Posts in category: nVidia AI

Inference, the work of using AI in applications, is moving into mainstream uses, and it’s running faster than ever.

NVIDIA GPUs won all tests of AI inference in data center and edge computing systems in the latest round of the industry’s only consortium-based and peer-reviewed benchmarks.

NVIDIA A100 Tensor Core GPUs extended the performance leadership we demonstrated in the first AI inference tests held last year by MLPerf, an industry benchmarking consortium formed in May 2018.

The A100, introduced in May, outperformed CPUs by up to 237x in data center inference, according to the MLPerf Inference 0.7 benchmarks. NVIDIA T4 small form factor, energy-efficient GPUs beat CPUs by up to 28x in the same tests.

To put this into perspective, a single NVIDIA DGX A100 system with eight A100 GPUs now provides the same performance as nearly 1,000 dual-socket CPU servers on some AI applications.

This round of benchmarks also saw increased participation, with 23 organizations submitting — up from 12 in the last round — and with NVIDIA partners using the NVIDIA AI platform to power more than 85 percent of the total submissions.

A100 GPUs, Jetson AGX Xavier Take Performance to the Edge

While A100 is taking AI inference performance to new heights, the benchmarks show that T4 remains a solid inference platform for mainstream enterprise, edge servers and cost-effective cloud instances. In addition, the NVIDIA Jetson AGX Xavier builds on its leadership position in power constrained SoC-based edge devices by supporting all new use cases.

The results also point to our vibrant, growing AI ecosystem, which submitted 1,029 results using NVIDIA solutions representing 85 percent of the total submissions in the data center and edge categories. The submissions demonstrated solid performance across systems from partners including Altos, Atos, Cisco, Dell EMC, Dividiti, Fujitsu, Gigabyte, Inspur, Lenovo, Nettrix and QCT.

Expanding Use Cases Bring AI to Daily Life

Backed by broad support from industry and academia, MLPerf benchmarks continue to evolve to represent industry use cases. Organizations that support MLPerf include Arm, Baidu, Facebook, Google, Harvard, Intel, Lenovo, Microsoft, Stanford, the University of Toronto and NVIDIA.

The latest benchmarks introduced four new tests, underscoring the expanding landscape for AI. The suite now scores performance in natural language processing, medical imaging, recommendation systems and speech recognition as well as AI use cases in computer vision.

You need go no further than a search engine to see the impact of natural language processing on daily life.

“The recent AI breakthroughs in natural language understanding are making a growing number of AI services like Bing more natural to interact with, delivering accurate and useful results, answers and recommendations in less than a second,” said Rangan Majumder, vice president of search and artificial intelligence at Microsoft.

“Industry-standard MLPerf benchmarks provide relevant performance data on widely used AI networks and help make informed AI platform buying decisions,” he said.

AI Helps Saves Lives in the Pandemic 

The impact of AI in medical imaging is even more dramatic. For example, startup Caption Health uses AI to ease the job of taking echocardiograms, a capability that helped save lives in U.S. hospitals in the early days of the COVID-19 pandemic.

That’s why thought leaders in healthcare AI view models like 3D U-Net, used in the latest MLPerf benchmarks, as key enablers.

“We’ve worked closely with NVIDIA to bring innovations like 3D U-Net to the healthcare market,” said Klaus Maier-Hein, head of medical image computing at DKFZ, the German Cancer Research Center.

“Computer vision and imaging are at the core of AI research, driving scientific discovery and representing core components of medical care. And industry-standard MLPerf benchmarks provide relevant performance data that helps IT organizations and developers accelerate their specific projects and applications,” he added.

Commercially, AI use cases like recommendation systems, also part of the latest MLPerf tests, are already making a big impact. Alibaba used recommendation systems last November to transact $38 billion in online sales on Singles Day, its biggest shopping day of the year.

Adoption of NVIDIA AI Inference Passes Tipping Point

AI inference passed a major milestone this year.

NVIDIA GPUs delivered a total of more than 100 exaflops of AI inference performance in the public cloud over the last 12 months, overtaking inference on cloud CPUs for the first time. Total cloud AI Inference compute capacity on NVIDIA GPUs has been growing roughly tenfold every two years.

With the high performance, usability and availability of NVIDIA GPU computing, a growing set of companies across industries such as automotive, cloud, robotics, healthcare, retail, financial services and manufacturing now rely on NVIDIA GPUs for AI inference. They include American Express, BMW, Capital One, Dominos, Ford, GE Healthcare, Kroger, Microsoft, Samsung and Toyota.

Why AI Inference Is Hard

Use cases for AI are clearly expanding, but AI inference is hard for many reasons.

New kinds of neural networks like generative adversarial networks are constantly being spawned for new use cases and the models are growing exponentially. The best language models for AI now encompass billions of parameters, and research in the field is still young.

These models need to run in the cloud, in enterprise data centers and at the edge of the network. That means the systems that run them must be highly programmable, executing with excellence across many dimensions.

NVIDIA founder and CEO Jensen Huang compressed the complexities in one word: PLASTER. Modern AI inference requires excellence in Programmability, Latency, Accuracy, Size of model, Throughput, Energy efficiency and Rate of learning.

To power excellence across every dimension, we’re focussed on constantly evolving our end-to-end AI platform to handle demanding inference jobs.

AI Requires Performance, Usability

An accelerator like the A100, with its third-generation Tensor Cores and the flexibility of its multi-instance GPU architecture, is just the beginning. Delivering leadership results requires a full software stack.

NVIDIA’s AI software begins with a variety of pretrained models ready to run AI inference. Our Transfer Learning Toolkit lets users optimize these models for their particular use cases and datasets.

NVIDIA TensorRT optimizes trained models for inference. With 2,000 optimizations, it’s been downloaded 1.3 million times by 16,000 organizations.

The NVIDIA Triton Inference Server provides a tuned environment to run these AI models supporting multiple GPUs and frameworks. Applications just send the query and the constraints — like the response time they need or throughput to scale to thousands of users — and Triton takes care of the rest.

These elements run on top of CUDA-X AI, a mature set of software libraries based on our popular accelerated computing platform.

Getting a Jump-Start with Applications Frameworks

Finally, our application frameworks jump-start adoption of enterprise AI across different industries and use cases.

Our frameworks include NVIDIA Merlin for recommendation systems, NVIDIA Jarvis for conversational AI, NVIDIA Maxine for video conferencing, NVIDIA Clara for healthcare, and many others available today.

These frameworks, along with our optimizations for the latest MLPerf benchmarks, are available in NGC, our hub for GPU-accelerated software that runs on all NVIDIA-certified OEM systems and cloud services.

In this way, the hard work we’ve done benefits the entire community.

3D artists and video editors have long used real-time AI features to improve their work and speed up how they turn inspiration into finished art. Now, those benefits are extending to Adobe Photoshop users with the introduction of GPU-accelerated neural filters.

These AI-powered tools, leveraging NVIDIA RTX GPUs with the Adobe creative applications, are being showcased at Adobe MAX, which is bringing together creators from around the world virtually through Oct. 22.

Neural filters are a new feature set for artists to try AI-powered tools that enable them to explore creative ideas and make amazing, complex adjustments to images in just seconds. Done manually, these adjustments would take artists hours of tedious work. AI allows artists to make these changes almost instantaneously.

NVIDIA GPUs accelerate nearly all these new filters. We’ll explain how to get the most out of them at a session at Adobe MAX.

Adobe and NVIDIA are closely collaborating on AI technology to improve creative tools in Creative Cloud and Photoshop. This collaboration includes the new Smart Portrait Filter, which is powered by NVIDIA StyleGAN2 technology and runs best on NVIDIA RTX GPUs.

With Smart Portrait in Photoshop, artists can easily experiment, making edits to facial characteristics, such as gaze direction and lighting angles, simply by dragging a slider. These types of complex corrections and adjustments would typically entail multiple manual steps. But Smart Portrait uses AI — based on a deep neural network developed by NVIDIA Research and trained on numerous portrait images — to achieve breathtaking results in seconds.

This gives artists greater flexibility with their images long after the photo shoot has ended. And they retain full control over their work with a non-destructive workflow, while the effects blend naturally into the original image.

Video editors in Adobe Premiere Pro also benefit from NVIDIA RTX GPUs with virtually all GPU-accelerated decoding offloaded to dedicated VRAM, resulting in smoother video playback and sharper responsiveness when scrubbing through footage, especially with ultra-high resolution and multistream footage. Advanced, AI-powered features such as Scene Edit Detection and Auto Reframe automate manual tasks, speeding up final exports and saving editors valuable time.

For the first time, Adobe Premiere Elements adds GPU acceleration to enable instant playback of popular video effects such as adding a lens flare or an animated overlay, cropping of videos, and overall playback in real-time, all without prerendering, rapidly speeding up the editing process.

AI and GPU-accelerated workflows are the result of the ongoing collaboration between teams at NVIDIA and Adobe. Over the years, we’ve developed tools and helped accelerate workflows in Adobe Photoshop, Lightroom, Premiere Pro, After Effects, Illustrator, Dimension, Substance Alchemist, Substance Painter and Substance Designer. As Adobe continues to build amazing software experiences, NVIDIA will be there to power and accelerate them, giving creators more time for creativity.

Working Smarter: Tapping into AI to Boost Creativity

Adobe is hosting more than 350 sessions across 10 tracks at this year’s MAX conference. Creators looking for new ways to improve their work while cutting down on the tasks that take away precious time can learn how to get the most out of new AI tools across Adobe creative apps.

NVIDIA is hosting an Adobe MAX session where attendees will discover new ways to tap into the power of AI. Whether a graphic artist, video editor, motion graphics professional, Photoshop professional, concept artist or other creator who needs computing speed, you’ll leave with valuable, time-saving tips.

Session attendees will discover:

• How to improve creations with more precision, clarity and quality
• How to let AI do the work under the hood, giving you more time to create
• The NVIDIA Studio ecosystem of tools and products designed to supercharge creativity

Visit the session catalog to learn more and tune in on Wednesday, Oct. 21, from 11-11:30 a.m. Pacific time.

October Studio Driver Ready For Download

Alongside these updates to Adobe Photoshop, Adobe Premiere Pro and Adobe Premiere Elements, there are new releases of Adobe After Effects, Adobe Substance Alchemist, Notch and Daz 3D — all supported in the new October NVIDIA Studio Driver. Studio Drivers are built specifically for creators and tested extensively against top creative apps and workflows.

Download the new Studio Driver (release 456.71) today through GeForce Experience or from the driver download page.

Learn more about NVIDIA Studio hardware and software for creators on the NVIDIA Studio website.

You can also stay up to date on the latest apps through NVIDIA’s Studio YouTube channel, featuring tutorials, tips and tricks by industry-leading artists.

Amid a pandemic that’s put much of the world’s work, learning, even family reunions online, two of the leaders who have made today’s virtual world possible met Thursday on, where else — Zoom — to talk about what’s next.

NVIDIA CEO Jensen Huang and Zoom CEO Eric Yuan spoke Thursday at the online video conference company’s Zoomtopia user event in a casual, wide-ranging conversation.

“If not for what Zoom has done, the recent pandemic would be unbearable,” Huang said. The present situation, Huang explained, “has accelerated the future, it has brought forward the urgency of a digital future.”

In front of a virtual audience from all over the globe, the two spoke about their entrepreneurial journeys, NVIDIA’s unique company culture, and how NVIDIA is knitting together the virtual and real worlds to help NVIDIA employees collaborate.

Huang’s appearance at Zoomtopia follows NVIDIA’s GPU Technology Conference last week, where Huang outlined NVIDIA’s view of data center computing and introduced new technologies in data centers, edge AI and healthcare.

Yuan playfully wore a leather jacket, matching Huang’s trademark attire—and briefly displayed a sleek virtual kitchen as his backdrop, paying tribute to the presentations Huang has given from his kitchen this year—began their conversation with Huang by asking about his early life.

“I was fortunate that my parents worked hard and all of the people I was surrounded by worked hard,” Huang said, adding that he was focused on on school and sports, especially table tennis. “To me working is living, working is breathing and, to me, it’s not work at all — I enjoy it too much.”

It’s NVIDIA’s mission, Huang said, that continues to motivate him, as the company has gone from inventing the GPU to pioneering new possibilities in robotics and AI.

The common thread: since the beginning, NVIDIA has had a singular focus on accelerated computing.

“We built a time machine,” Huang said, touching on NVIDIA’s work in drug discovery as an example. “So, instead of a particular drug taking 10 years to discover, we would like drugs and therapies and vaccines to be discovered in months.”

Zoom and NVIDIA, Huang said, share a “singular purpose and a sense of destiny,” Huang said, one that has made the world a better place.

“The fact that Zoom existed and your vision came to reality means we can be together even if we’re not together,” Huang said.

“You can look at your work and imagine the impact on society and the benefits it will bring and somehow it’s your job to do it,” Huang said. “If you don’t do it, no one else will — and that’s thrilling to me, I love that feeling.”

Yuan also asked about NVIDIA’s culture and the future of work, one which Huang believes will increasingly meld the physical and the virtual worlds.

Today, for example, we might report to your colleagues that we’ll be WFH, or working from home.

Office lingo, however, may change to reflect the new reality, where being at the office isn’t necessarily the norm.

“In the future we will say we’re ‘going to the office,’” Huang said. “Today we say ‘WFH,’ in the future we will say ‘GTO.’”

Tools such as Zoom enable colleagues to meet, face to face, from home, from an office, from anywhere in the world.

More and more, work will take place in a hybrid of office and home, physical and virtual reality.

NVIDIA, for example, has created a platform called NVIDIA Omniverse that lets colleagues working in different places and with different tools collaborate in real time.

“The Adobe world can connect to the Catia world and so on,” Huang said. “We can have different designers working with each other at their homes.”

The present moment has “brought forward the urgency of a digital future, it has made us aware that completely physical is not sufficient, that completely digital is not sufficient,” Huang said. “The future is a mixed reality world.”

Four new supercomputers backed by a pan-European initiative will use NVIDIA’s data center accelerators, networks and software to advance AI and high performance computing.

They include one system dubbed Leonardo, unveiled today at Italy’s CINECA research center, using NVIDIA technologies to deliver the world’s most powerful AI system. The four mark the first of eight systems to be announced this year targeting spots among the world’s 50 most powerful computers.

Together, they’ll form a regional network, “an engine to power Europe’s data economy,” said EuroHPC, the group driving the effort, in a white paper outlining its goals.

The systems will apply AI and data analytics across scientific and commercial applications that range from fighting COVID-19 and climate change to the design of advanced airplanes, cars, drugs and materials.

Joining Leonardo are a wave of new AI supercomputers planned for the Czech Republic, Luxembourg and Slovenia that will act as national centers of competence, expanding skills and creating jobs.

NVIDIA GPUs, InfiniBand Power Latest Systems

All four supercomputers announced use NVIDIA Ampere architecture GPUs and NVIDIA Mellanox HDR InfiniBand networks to tap an ecosystem of hundreds of HPC and AI applications. Atos, an NVIDIA systems partner headquartered in France, will build three of the four systems; Hewlett Packard Enterprise will construct the fourth.

The new systems join 333 of the world’s TOP500 supercomputers powered by NVIDIA GPUs, networking or both.

NVIDIA GPUs accelerate 1,800 HPC applications, nearly 800 of them available today in the GPU application catalog and NGC, NVIDIA’s hub for GPU-optimized software.

The new systems all use HDR 200Gb/s InfiniBand for low latency, high throughput and in-network computing. It’s the latest version of InfiniBand, already powering supercomputers across Europe.

A Brief Tour of Europe’s Latest Supercomputers

Leonardo will be the world’s fastest AI supercomputer. Atos is harnessing nearly 14,000 NVIDIA Ampere architecture GPUs and HDR 200Gb/s InfiniBand networking to deliver a system with 10 exaflops of AI performance. It will use the InfiniBand Dragonfly+ network topology to deliver both flexibility and scalable performance.

Researchers at CINECA will apply that power to advance science, simulating planetary forces behind climate change and molecular movements inside a coronavirus. The center is perhaps best known for its work on Quantum Espresso, a suite of open source codes for modeling nanoscale materials for jobs such as engineering better batteries.

A new supercomputer in Luxembourg called MeluXina, also part of the EuroHPC network, will connect 800 NVIDIA A100 GPUs on HDR 200Gb/s InfiniBand links. The system, to be built by Atos and powered by green energy from wood waste, will pack nearly 500 petaflops of AI performance.

MeluXina will address commercial applications and scientific research. It plans to offer access to users leveraging HPC and AI to advance work in financial services as well as manufacturing and healthcare.

Eastern Europe Powers Up

The new Vega supercomputer at the Institute of Information Science in Maribor, Slovenia, (IZUM) will be based on the Atos BullSequana XH2000 system. The supercomputer, named after Slovenian mathematician Jurij Vega, includes 240 A100 GPUs and 1,800 HDR 200Gb/s InfiniBand end points.

Vega will help “ensure a new generation of experts and developers, as well as the wider Slovenian community, can meet new challenges within our national consortium and contribute to regional and European HPC initiatives,” said Aleš Bošnjak, IZUM’s director in a statement issued by EuroHPC.

The IT4Innovations National Supercomputing Center will host what’s expected to become the most powerful supercomputer in the Czech Republic. It will use 560 NVIDIA A100 GPUs to deliver nearly 350 petaflops of AI performance — 7x the capabilities of the center’s existing system.

The supercomputer will be based on the HPE Apollo 6500 systems from Hewlett Packard Enterprise (HPE). It will serve researchers at the VSB – Technical University of Ostrava, where it’s based, as well as an expanding set of external academic and industrial users employing a mix of simulations, data analytics and AI.

The story of Europe’s ambitions in HPC and AI is still being written.

EuroHPC has yet to announce its plans for systems in Bulgaria, Finland, Portugal and Spain. And beyond that work, the group has already sketched out plans that stretch to 2027.

Using AI and a supercomputer simulation, Ken Dill’s team drew the equivalent of wanted posters for a gang of proteins that make up COVID-19. With a little luck, one of their portraits could identify a way to arrest the coronavirus with a drug.

When the pandemic hit, “it was terrible for the world, and a big research challenge for us,” said Dill, who leads the Laufer Center for Physical & Quantitative Biology at Stony Brook University, in Long Island, New York.

For a decade, he helped the center assemble the researchers and tools needed to study the inner workings of proteins — complex molecules that are fundamental to cellular life. The center has a history of applying its knowledge to viral proteins, helping others identify drugs to disable them.

“So, when the pandemic came, our folks wanted to spring into action,” he said.

AI, Simulations Meet at the Summit

The team aimed to use a combination of physics and AI tools to predict the 3D structure of more than a dozen coronavirus proteins based on lists of the amino acid strings that define them. It won a grant for time on the IBM-built Summit supercomputer at Oak Ridge National Laboratory to crunch its complex calculations.

“We ran 30 very extensive simulations in parallel, one on each of 30 GPUs, and we ran them continuously for at least four days,” explained Emiliano Brini, a junior fellow at the Laufer Center. “Summit is a great machine because it has so many GPUs, so we can run many simulations in parallel,” he said.

“Our physics-based modeling eats a lot of compute cycles. We use GPUs almost exclusively for their speed,” said Dill.

Sharing Results to Help Accelerate Research

Thanks to the acceleration, the predictions are already in. The Laufer team quickly shared them with about a hundred researchers working on a dozen separate projects that conduct painstakingly slow experiments to determine the actual structure of the proteins.

“They indicated some experiments could be done faster if they had hunches from our work of what those 3D structures might be,” said Dill.

Now it’s a waiting game. If one of the predictions gives researchers a leg up in finding a weakness that drug makers can exploit, it would be a huge win. It could take science one step closer to putting a general antiviral drug on the shelf of your local pharmacy.

Melding Machine Learning and Physics

Dill’s team uses a molecular dynamics program called MELD. It blends physical simulations with insights from machine learning based on statistical models.

AI provides MELD key information to predict a protein’s 3D structure from its sequence of amino acids. It quickly finds patterns across a database of atomic-level information on 200,000 proteins gathered over the last 50 years.

MELD uses this information in compute-intensive physics simulations to determine the protein’s detailed structure. Further simulations then can predict, for example, what drug molecules will bind tightly to a specific viral protein.

“So, both these worlds — AI inference and physics simulations — are playing big roles in helping drug discovery,” said Dill. “We get the benefits of both methods, and that combination is where I think the future is.”

MELD runs on CUDA, NVIDIA’s accelerated computing platform for GPUs. “It would take prohibitively long to run its simulations on CPUs, so the majority of biological simulations are done on GPUs,” said Brini.

Playing a Waiting Game

The COVID-19 challenge gave Laufer researchers with a passion for chemistry a driving focus. Now they await feedback on their work on Summit.

“Once we get the results, we’ll publish what we learn from the mistakes. Many times, researchers have to go back to the drawing board,” he said.

And every once in a while, they celebrate, too.

Dill hosted a small, socially distanced gathering for a half-dozen colleagues in his backyard after the Summit work was complete. If those results turn up a win, there will be a much bigger celebration extending far beyond the Stony Brook campus.

In the battle against COVID-19, Greater Paris University Hospitals – Public Assistance Hospital of Paris (AP-HP is the French acronym) isn’t just on the medical front lines — it’s on the data front lines as well.

With a network of 39 hospitals treating 8.3 million patients each year, AP-HP is a major actor in the fight against COVID-19.

Along with its COVID-19 cases comes an awful lot of data, including now geodata that can potentially help lessen the impact of the pandemic. AP-HP, which partners with seven universities, already had the ability to analyze large amounts of medical data. It had previously created dashboards that combined cancer cases and geodata. So, it was logical to pursue and extend its role during the pandemic.

The expected volume of COVID-19 data and geodata would probably have tested AP-HP’s data crunching capacity. To mitigate this critical challenge, the hospital’s information systems administrators turned to Kinetica, a provider of streaming data warehouses and real-time analytics and a member of the NVIDIA Inception program for AI startups.

Kinetica’s offering harnesses the power of NVIDIA GPUs to quickly convert case location data into usable intelligence. And in the fight against COVID-19, speed is everything.

The project team also used NVIDIA RAPIDS to speed up the machine learning algorithms integrated into the platform. RAPIDS accelerates analytics and data science pipelines on NVIDIA GPUs by taking advantage of GPU parallelism and high memory bandwidth.

“Having the ability to perform this type of analysis in real time is really important during a pandemic,” said Hector Countouris, the project lead at AP-HP. “And more data is coming.”

Analyzing COVID Contact Data

What Countouris and his colleagues are most focused on is using COVID-related geodata to understand where virus “hot spots” are and the dynamic of the outbreak. Looking for cluster locations can help decision-making at the district or region level.

In addition, they’re looking at new signals to improve early detection of COVID patients. This includes working with data from other regional agencies.

If patients are diagnosed with COVID, they’ll be asked by the relevant agencies via a phone call about their recent whereabouts and contacts to help with contact tracing. This is the first time that a wide range of data from different partners in the Paris area will be integrated to allow for contact tracing and timely alerts about a potential exposure. The result will be a newfound ability to see how clusters of COVID-19 cases evolve.

“We hope that in the near future we will be able to follow how a cluster evolves in real time,” said Countouris.

The goal is to enable public health decision-makers to implement prevention and control measures and assess their effectiveness. The data can also be integrated with other demographic data to study the viral spread and its possible dependency on socio-economics and other factors.

Attacking Bottlenecks with GPUs

Prior to engaging with Kinetica, such data-intensive projects involved so much time for loading the data that they couldn’t be analyzed quickly enough to deliver real-time benefits.

“Now, I don’t feel like I have a bottleneck,” said Countouris. “We are continuously integrating data and delivering dashboards to decision makers within hours. And with robust real-time pipelines allowing for continuous data ingestion, we can now focus on building better dashboards.”

In the past, to get data in a specific and usable format, they would need to do a lot of pre-processing. With Kinetica’s Streaming Data Warehouse powered by NVIDIA V100 Tensor Core GPUs, that’s no longer the case. Users can access the much richer datasets they demand.

Kinetica’s platform is available on NVIDIA NGC, a catalog of GPU-optimized AI containers that let enterprises quickly operationalize extreme analytics, machine learning and data visualization. This eliminates complexity and lets organizations deploy cloud, on-premises or hybrid models for optimal business operations.

“I don’t think we could meet user expectations for geodata without GPU power,” he said. “There is just too much data and geodata to provide for too many users at the same time.”

AP-HP’s COVID-related work has already built a foundation upon which to do follow-up work related to emergency responses in general. The hospital information system’s interest for that kind of data is far from over.

“The fact that we helped the decision-making process and that officials are using our data is the measure of success,” said Countouris. “We have a lot to do. This is only the beginning.”

Countouris presented the team’s work last week at the GPU Technology Conference. Registered GTC attendees can view the talk on demand. It will be available for replay to the general public early next month.

Kinetica will also be part of the NVIDIA Startup Village Booth at the HLTH conference, presenting on Oct. 16 at 2 p.m. Pacific time.

Not everyone needs to be a developer, but everyone will need to be an AI decision maker.

That was the message behind a panel discussion on Advancing Equitable AI, which took place at our GPU Technology Conference last week. It was one of several GTC events advancing the conversation on diversity, equity and ethics in AI.

This year, we strengthened our support for women and underrepresented developers and scientists at GTC by providing conference passes to members of professional organizations supporting women, Black and Latino developers. Professors at historically Black colleges and universities — including Prairie View A&M University, Hampton University and Jackson State University — as well as groups like Black in AI and LatinX in AI received complimentary access to training from the NVIDIA Deep Learning Institute.

A Forbes report last year named GTC as one of the U.S.’s top conferences for women to attend to further their careers in AI. At this month’s event, women made up better than one in five registered attendees — doubling last year’s count and an almost 4x increase since 2017 — and more than 100 of the speakers.

And in a collaboration with the National Society of Black Engineers that will extend beyond GTC, we created opportunities for the society’s collegiate and professional developers to engage with NVIDIA’s recruiting team, which provided guidance on navigating the new world of virtual interviewing and networking.

“We’re excited to be embarking on a partnership with NVIDIA,” said Johnnie Tangle, national finance chairman of NSBE Professionals. “Together, we are both on the mission of increasing the visibility of Blacks in development and showing why diversity in the space enhances the community as a whole.”

Panel Discussions: Paving Pathways for Equitable AI

Two power-packed, all-female panels at GTC focused on a roadmap for responsible and equitable AI.

In a live session that drew over 250 attendees, speakers from the University of Florida, the Boys and Girls Club of Western Pennsylvania and AI4All — a nonprofit working to increase diversity and inclusion in AI — discussed the importance of AI exposure and education for children and young adults from underrepresented groups.

When a broader group of young people has access to AI education, “we naturally see a way more diverse and interesting set of problems being addressed,” said Tess Posner, CEO of AI4All, “because young people and emerging leaders in the field are going to connect the technology to a problem they’ve seen in their own lives, in their own experience or in their communities.”

The conversation also covered the role parents and schools play in fostering awareness and exposure to STEM subjects in their children’s schools, as well as the need for everyone — developers or not — to have a foundational understanding of how AI works.

“We want students to be conscious consumers, and hopefully producers,” said Christina Gardner-McCune, associate professor and director of the Engaging Learning Lab at the University of Florida, and co-chair of the AI4K12 initiative. “Everybody is going to be making decisions about what AI technologies are used in their homes, what AI technologies their children interact with.”

Later in the week, a panel titled “Aligning Around Common Values to Advance AI Policy” explored ideas to pave the way for responsible AI on a global scale.

The webinar featured representatives from the U.S. National Institute of Standards and Technology, Scotland-based innovation center The Data Lab, and C Minds, a think tank focused on AI initiatives in Latin America. Speakers shared their priorities for developing trustworthy AI, and defined what success would like to them five years in the future.

Dinner with Strangers: Developer Diversity in AI

In a virtual edition of the popular Dinner with Strangers networking events at GTC, experts from NVIDIA and NSBE partnered to moderate two conversations with GTC attendees. NVIDIA employees shared their experiences and tips with early-career attendees, offering advice on how to build a personal brand in a virtual world, craft a resume and prepare for interviews.

For more about GTC, watch NVIDIA founder and CEO Jensen Huang’s keynote below.

When large organizations require translation services, there’s no room for the amusing errors often produced by automated apps. That’s where Lilt, an AI-powered enterprise language translation company, comes in.

Lilt CEO Spence Green spoke with AI Podcast host Noah Kravitz about how the company is using a human-in-the-loop process to achieve fast, accurate and affordable translation.

Lilt does so with a predictive typing software, in which professional translators receive AI-based suggestions of how to translate content. By relying on machine assistance, Lilt’s translations are efficient while retaining accuracy.

However, including people in the company’s workflow also makes localization possible. Professional translators use cultural context to take direct translations and adjust phrases or words to reflect the local language and customs.

Lilt currently supports translations of 45 languages, and aims to continue improving its AI and make translation services more affordable.

Key Points From This Episode:

• Green’s experience living in Abu Dhabi was part of the inspiration behind Lilt. While there, he met a man, an accountant, who had immigrated from Egypt. When asked why he no longer worked in accounting, the man explained that he didn’t speak English, and accountants who only spoke Arabic were paid less. Green didn’t want the difficulty of adult language learning to be a source of inequality in a business environment.
• Lilt was founded in 2015, and evolved from a solely software company into a software and services business. Green explains the steps it took for the company to manage translators and act as a complete solution for enterprises.

Tweetables:

“We’re trying to provide technology that’s going to drive down the cost and increase the quality of this service, so that every organization can make all of its information available to anyone.” — Spence Green [2:53]

“One could argue that [machine translation systems] are getting better at a faster rate than at any point in the 70-year history of working on these systems.” — Spence Green [14:01]

You Might Also Like:

Hugging Face’s Sam Shleifer Talks Natural Language Processing

Hugging Face is more than just an adorable emoji — it’s a company that’s demystifying AI by transforming the latest developments in deep learning into usable code for businesses and researchers, explains research engineer Sam Shleifer.

Credit Check: Capital One’s Kyle Nicholson on Modern Machine Learning in Finance

Capital One Senior Software Engineer Kyle Nicholson explains how modern machine learning techniques have become a key tool for financial and credit analysis.

A Conversation with the Entrepreneur Behind the World’s Most Realistic Artificial Voices

Voice recognition is one thing, creating natural sounding artificial voices is quite another. Lyrebird co-founder Jose Solero speaks about how the startup is using deep learning to create a system that’s able to listen to human voices and generate speech mimicking the original human speaker.

Tune in to the AI Podcast

Get the AI Podcast through iTunes, Google Podcasts, Google Play, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn. If your favorite isn’t listed here, drop us a note.

Make the AI Podcast Better

Have a few minutes to spare? Fill out this listener survey. Your answers will help us make a better podcast.

The popularity of public cloud offerings is evident — just look at how top cloud service providers report double-digit growth year over year.

However, application performance requirements and regulatory compliance issues, to name two examples, often require data to be stored locally to reduce distance and latency and to place data entirely within a company’s control. In these cases, standard private clouds also may offer less flexibility, agility or on-demand capacity.

To help resolve these issues, Lenovo, Microsoft and NVIDIA have engineered a hyperconverged hybrid cloud that enables Azure cloud services within an organization’s data center.

By integrating Lenovo ThinkAgile SX, Microsoft Azure Stack Hub and NVIDIA Mellanox networking, organizations can deploy a turnkey, rack-scale cloud that’s optimized with a resilient, highly performant and secure software-defined infrastructure.

Fully Integrated Azure Stack Hub Solution

Lenovo ThinkAgile SX for Microsoft Azure Stack Hub satisfies regulatory compliance and removes performance concerns. Because all data is kept on secure servers in a customer’s data center, it’s much simpler to comply with the governance laws of a country and implement their own policies and practices.

Similarly, by reducing the distance that data must travel, latency is reduced and application performance goals can be more easily achieved. At the same time, customers can cloud-burst some workloads to the Microsoft Azure public cloud, if desired.

Lenovo, Microsoft and NVIDIA worked together to make sure everything performs right out of the box. There’s no need to worry about configuring and adjusting settings for virtual or physical infrastructure.

The power and automation of Azure Stack Hub software, the convenience and reliability of Lenovo’s advanced servers, and the high performance of NVIDIA networking combine to enable an optimized hybrid cloud. Offering the automation and flexibility of Microsoft Azure Cloud with the security and performance of on-premises infrastructure, it’s an ideal platform to:

• deliver Azure cloud services from the security of your own data center,
• enable rapid development and iteration of applications with on-premises deployment tools,
• unify application development across entire hybrid cloud environments, and
• easily move applications and data across private and public clouds.

Agility of a Hybrid Cloud 

Azure Stack Hub also seamlessly operates with Azure, delivering an orchestration layer that enables the movement of data and applications to the public cloud. This hybrid cloud protects the data and applications that need protection and offers lower latencies for accessing data. And it still provides the public cloud benefits organizations may need, such as reduced costs, increased infrastructure scalability and flexibility, and protection from data loss.

A hybrid approach to cloud computing keeps all sensitive information onsite and often includes centrally used applications that may have some of this data tied to them. With a hybrid cloud infrastructure in place, IT personnel can focus on building proficiencies in deploying and operating cloud services — such as IaaS, PaaS and SaaS — and less on managing infrastructure.

Network Performance 

A hybrid cloud requires a network that can handle all data communication between clients, servers and storage. The Ethernet fabric used for networking in the Lenovo ThinkAgile SX for Microsoft Azure Stack Hub leverages NVIDIA Mellanox Spectrum Ethernet switches — powered by the industry’s highest-performing ASICs — along with NVIDIA Cumulus Linux, the most advanced open network operating system.

At 25Gb/s data rates, these switches provide cloud-optimized delivery of data at line-rate. Using a fully shared buffer, they support fair bandwidth allocation and provide predictably low latency, as well as traffic flow prioritization and optimization technology to deliver data without delays, while the hot-swappable redundant power supplies and fans help provide resiliency for business-sensitive traffic.

Modern networks require advanced offload capabilities, including remote direct memory access (RDMA), TCP, overlay networks (for example, VXLAN and Geneve) and software-defined storage acceleration. Implementing these at the network layer frees expensive CPU cycles for user applications while improving the user experience.

To handle the high-speed communications demands of Azure Stack Hub, Lenovo configured compute nodes with a dual-port 10/25/100GbE NVIDIA Mellanox ConnectX-4 Lx, ConnectX-5 or ConnectX-6 Dx NICs. The ConnectX NICs are designed to address cloud, virtualized infrastructure, security and network storage challenges. They use native hardware support for RoCE, offer stateless TCP offloads, accelerate overlay networks and support NVIDIA GPUDirect technology to maximize performance of AI and machine learning workloads. All of this results in much needed higher infrastructure efficiency.

RoCE for Improved Efficiency 

Microsoft Azure Stack Hub leverages Storage Spaces Direct (S2D) and Microsoft’s Server Message Block Direct 3.0. SMB Direct uses high-speed RoCE to transfer large amounts of data with little CPU intervention. SMB Multichannel allows servers to simultaneously use multiple network connections and provide fault tolerance through the automatic discovery of network paths.

The addition of these two features allows NVIDIA RoCE-enabled ConnectX Ethernet NICs to deliver line-rate performance and optimize data transfer between server and storage over standard Ethernet. Customers with Lenovo ThinkAgile SX servers or the Lenovo ThinkAgile SX Azure Hub can deploy storage on secure file servers while delivering the highest performance. As a result, S2D is extremely fast with disaggregated file server performance, almost equaling that of locally attached storage.

Run More Workloads

By using intelligent hardware accelerators and offloads, the NVIDIA RoCE-enabled NICs offload I/O tasks from the CPU, freeing up resources to accelerate application performance instead of making data wait for the attention of a busy CPU.

The result is lower latencies and an improvement in CPU efficiencies. This maximizes the performance in Microsoft Azure Stack deployments by leaving the CPU available to run other application processes. Efficiency gets a boost since users can host more VMs per physical server, support more VDI instances and complete SQL Server queries more quickly.

A Transformative Experience with a ThinkAgile Advantage

Lenovo ThinkAgile solutions include a comprehensive portfolio of software and services that supports the full lifecycle of infrastructure. At every stage — planning, deploying, supporting, optimizing and end-of-life — Lenovo provides the expertise and services needed to get the most from technology investments.

This includes single-point-of-contact support for all the hardware and software used in the solution, including Microsoft’s Azure Stack Hub and the ConnectX NICs. Customers never have to worry about who to call — Lenovo takes calls and drives them to resolution.

Learn more about Lenovo ThinkAgile SX for Microsoft Azure Stack Hub with NVIDIA Mellanox networking.

Music and engineering were completely separate parts of Bryce Denney’s life until the pandemic hit.

By day, the Massachusetts-based NVIDIA engineer helped test processors. On nights and weekends, he played piano chamber music and accompanied theater troupes that his wife, Kathryn, sang in or led.

It was a good balance for someone who graduated with a dual major in physics and piano performance.

Once COVID-19 arrived, “we had to take the calendar off the wall — it was too depressing to look at everything that was canceled,” Bryce said.

“I had this aimless sense of not feeling sure who I was anymore,” said Kathryn, a former public school teacher who plays French horn and conducts high school and community theater groups. “This time last year I was working in five shows and a choir that was preparing for a tour of Spain,” she said.

That’s when Bryce got an idea for some musical engineering.

Getting Wired for Sound

He wanted to help musicians in separate spaces hear each other without nagging delays. As a proof of concept, he ran cables for mics and headphones from a downstairs piano to an upstairs bedroom where Kathryn played her horn.

The duet’s success led to convening a quartet from Kathryn’s choir in the driveway, singing safely distanced in separate cars with wired headsets linked to a small mixer. The Driveway Choir was born.

“We could hear each other breathe and we joked back and forth,” said Kathryn.

“It was like an actual rehearsal again and so much more rewarding” than Zoom events or virtual choirs where members recorded one part at a time and mixed them together, said Bryce.

But it would take a rat’s nest of wires to link a full choir of 20 singers, so Bryce boned up on wireless audio engineering.

Physics to the Rescue

He reached out to people like David Newman, a voice teacher at James Madison University, who was also experimenting with choirs in cars. And he got tips about wireless mics that are inexpensive and easy to clean.

Newman and others coached him on broadcasting over FM frequencies, and how to choose bands to avoid interference from local TV stations.

“It was an excuse to get into physics again,” said Bryce.

Within a few weeks he assembled a system and created a site for his driveway choir, where he posted videos of events, a spreadsheet of options for configuring a system, and packing lists for how to take it on the road. A basic setup for 16 singers costs $1,500 and can scale up to accommodate 24 voices.

“Our goal is to make this accessible to other groups, so we choose less-expensive equipment and write out a step-by-step process,” said Kathryn, who has helped organize 15 events using the gear.

Jan Helbers, a neighbor with wireless expertise, chipped in by designing an antenna distribution system that can sit on top of a car on rainy days.

Bryce posted instructions on how to build it for about $300 complete with a bill of materials and pictures of the best RF connectors to use. A commercial antenna distribution system of this size would cost thousands.

“I was excited about that because here in Marlborough it will be snowy soon and we want to keep singing,” said Bryce.

From Alaska to the Times

The Denneys helped the choir at St. Anne’s Episcopal church in nearby Lincoln, Mass., have its first live rehearsal in four months and record a track used in a Sunday service. Now the choir is putting together its own system.

The church is one of at least 10 groups that have contacted the Denneys about creating driveway choirs of their own, including one in Alaska. They expect more calls after a New York Times reporter joined one of their recent events and wrote a story about his experience.

There’s no shortage of ideas for what’s next. Driveway choirs for nursing homes, singalongs in big mall parking lots or drive-in theaters, Christmas caroling for neighbors.

“I wouldn’t be surprised if we did a Messiah sing,” said Kathryn, who has been using some of her shelter-in-place time to start writing a musical called Connected.

“I think about that song, ‘How Can I Keep from Singing,’ that’s the story of our lives,” she said.

At top: Kathryn conducts and Bryce plays piano at a Driveway Choir event with 24 singers in Concord, MA.