Posts in category: nVidia AI

This back-to-school season, many universities are powering on brand new AI supercomputers. Researchers and students working in fields from basic science to liberal arts can’t wait to log on.

“They would like to use it right now,” said James Wilgenbusch, director of research computing at the University of Minnesota, speaking of Agate, an accelerated supercomputer Hewlett Packard Enterprise is building.

It’s one of at least eight new academic systems lighting up around the world — four in America’s heartland and two in the U.K.

Before the semester’s end, Agate will deliver seven petaflops of umph. It will crunch through “research from socio-economic trends to celestial objects — it really will serve the full gamut,” he said of the system to be housed at the Minnesota Supercomputing Institute (MSI) that will link 265 NVIDIA A100 Tensor Core GPUs on an NVIDIA HDR 200Gb/s InfiniBand network.

Agate will serve about 4,500 users, working under a thousand principal investigators who since January have already run a whopping 138,612 GPU-accelerated jobs on MSI’s existing systems.

“We’re seeing annual user growth, the greatest amount of it in life sciences and liberal arts — fields like geology, history, poli-sci, marketing — anywhere people have vast quantities of unstructured data and they’re attempting to make sense of it,” he said.

AI Supercomputer Helps Fight COVID

Demonstrating the power of accelerated computing, the Minnesota Department of Health reserved a portion of MSI’s system in its fight against COVID-19. It’s sequencing genomes for contract tracing and to track variants of the coronavirus.

“Collaborations like this make the role of universities in innovation and life saving more obvious to the public,” said Wilgenbusch, pointing to articles in a Minneapolis newspaper.

Virtual GPUs Power Indiana Classrooms

Some 600 miles southeast, Indiana University (IU) is standing up two AI supercomputers packing a total of 616 A100 GPUs.

Big Red 200, built by Hewlett Packard Enterprise, will serve the nine IU campuses. Jetstream-2, built by Dell Technologies, will power work at several partner institutions from Cornell to the University of Hawaii.

Tapping the A100’s ability to offer fractions of a processor, Jetstream-2 will host classes with hundreds of students, each using a slice of a GPU’s performance to learn popular AI skills like image classification. One IU researcher presented a paper last November benchmarking the virtual GPU capability.

“Now whole classrooms can be trained in one go, so more people get access,” said Winona Snapp-Childs, chief operating officer of IU’s Pervasive Technology Institute and leader of an AI-for-everyone initiative.

A Vision of Ubiquitous AI

More than 2,500 students use IU’s current GPU-accelerated systems. They ran more than 40 percent of the work for the university’s record $1 billion of research contracts and grants spread across 178 departments last year.

“Funding agencies realize the importance of machine learning in academic fields across the spectrum,” said Snapp-Childs.

“AI and accelerated computing help push the boundaries of science, and I can imagine they will come to handle half of our research over the next 5 to 10 years as these techniques become ubiquitous and imperative for research,” she added.

The work spans a spectrum that can set your head spinning. Researchers are tapping AI for everything from tracking down COVID misinformation on social networks to studying the genome of rice to improve harvests.

Delta Pioneers Accessible Supercomputing

Next door, the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign is expanding use of accelerated computing with Delta, an AI supercomputer packing more than 800 A100 GPUs.

“We will help emerging research areas such as computational archaeology and digital agriculture take advantage of new computing methods and hardware while making advanced systems more usable and accessible to a broad community of researchers,” said William Gropp, a principal investigator and NCSA director who oversees Delta.

The system is one way the National Science Foundation is spreading GPU-based computing as a common tool for accelerating research. The work includes an initiative to make Delta and future systems more accessible to people with disabilities.

Florida Spreads the AI Sunshine

A thousand miles south, the University of Florida’s HiPerGator AI system provides another shining example of accelerated computing.

In a recent article in the Gainesville Sun, provost Joe Glover said the system will spread AI skills much like Henry Ford’s first assembly line made cars affordable for Americans. The university aims to add 100 AI-focused faculty to make machine learning ubiquitous across its curriculum with a stated goal of creating 30,000 AI-enabled graduates by 2030.

HiPerGator AI linked a whopping 1,120 A100 GPUs on a HDR 200Gb/s InfiniBand network to take the No. 22 spot in the latest TOP500 list of the world’s fastest supercomputers. It was built in just a few weeks thanks to its use of the NVIDIA DGX SuperPOD reference architecture, a recipe for stacking NVIDIA DGX systems in Lego-like style.

Studying Abroad: AI Supercomputing’s Far Reach

These five AI supercomputers represent just a few peaks in a rising range that crisscrosses the U.S. and Europe.

• On the UC Berkeley campus, researchers just turned on Perlmutter, the world’s fifth fastest system, packing 6,144 A100 GPUs.
• The University of Cambridge debuted CSD3, a cloud-native supercomputer built on Dell EMC PowerEdge, which is now the fastest academic system in the U.K. and hit No. 3 on the Green500 list of the world’s most energy-efficient systems.
• The University of Edinburgh is building a system with 448 A100 GPUs, the latest in the four-system network run by the DiRAC research group in the U.K.
• And Linköping University is now home to Sweden’s largest supercomputer, BerzeLiUs, which will serve a national AI initiative and be shared with researchers at Singapore’s Nanyang Technical University.

They are among high-performance systems sprinkled around the world, advancing science with machine learning and accelerated computing.

Photo at top: From left, Winona Snapp-Childs and Sheri Sanders, Director of the National Center for Genome Analysis Support, give students Christine Campbell and Lyric Cooper a tour of the Jetstream data center at Indiana University.

When you shop for a car, the first question is what model — a Honda Civic for low-cost commuting, a Chevy Corvette for looking good and moving fast, or maybe a Ford F-150 to tote heavy loads.

For the journey to AI, the most transformational technology of our time, the engine you need is a machine learning model.

What Is a Machine Learning Model?

A machine learning model is an expression of an algorithm that combs through mountains of data to find patterns or make predictions. Fueled by data, machine learning (ML) models are the mathematical engines of artificial intelligence.

For example, an ML model for computer vision might be able to identify cars and pedestrians in a real-time video. One for natural language processing might translate words and sentences.

Under the hood, a model is a mathematical representation of objects and their relationships to each other. The objects can be anything from “likes” on a social networking post to molecules in a lab experiment.

ML Models for Every Purpose

With no constraints on the objects that can become features in an ML model, there’s no limit to the uses for AI. The combinations are infinite.

Data scientists have created whole families of machine learning models for different uses, and more are in the works.

A Brief Taxonomy of ML Models

For instance, linear models use algebra to predict relationships between variables in financial projections. Graphical models express as diagrams a probability, such as whether a consumer will choose to buy a product. Borrowing the metaphor of branches, some ML models take the form of decision trees or groups of them called random forests.

In the Big Bang of AI in 2012, researchers found deep learning to be one of the most successful techniques for finding patterns and making predictions. It uses a kind of machine learning model called a neural network because it was inspired by the patterns and functions of brain cells.

An ML Model for the Masses

Deep learning took its name from the structure of its machine learning models. They stack layer upon layer of features and their relationships, forming a mathematical hero sandwich.

Thanks to their uncanny accuracy in finding patterns, two kinds of deep learning models, described in a separate explainer, are appearing everywhere.

Convolutional neural networks (CNNs), often used in computer vision, act like eyes in autonomous vehicles and can help spot diseases in medical imaging. Recurrent neural networks and transformers (RNNs), tuned to analyze spoken and written language, are the engines of Amazon’s Alexa, Google’s Assistant and Apple’s Siri.

Pssssst, Pick a Pretrained Model

Choosing the right family of models — like a CNN, RNN or transformer — is a great beginning. But that’s just the start.

If you want to ride the Baja 500, you can modify a stock dune buggy with heavy duty shocks and rugged tires, or you can shop for a vehicle built for that race.

In machine learning, that’s what’s called a pretrained model. It’s tuned on large sets of training data that are similar to data in your use case. Data relationships — called weights and biases — are optimized for the intended application.

It takes an enormous dataset, a lot of AI expertise and significant compute muscle to train a model. Savvy buyers shop for pretrained models to save time and money.

Who Ya Gonna Call?

When you’re shopping for a pretrained model, find a dealer you can trust.

NVIDIA puts its name behind an online library called the NGC catalog that’s filled with vetted, pretrained models. They span the spectrum of AI jobs from computer vision and conversational AI and more.

Users know what they’re getting because models in the catalog come with résumés. They’re like the credentials of a prospective hire.

Model resumes show you the domain the model was trained for, the dataset that trained it, and how it’s expected to perform. They provide transparency and confidence you’re picking the right model for your use case.

More Resources for ML Models

What’s more, NGC models are ready for transfer learning. That’s the one final tune-up that torques models for the exact road conditions over which they’ll ride — your application’s data.

NVIDIA even provides the wrench to tune your NGC model. It’s called TAO and you can sign up for early access to it today.

To learn more, check out:

• Our web page on pretrained models
• A guide to the NGC catalog
• Our web page on Tao and related tools
• A developer blog on using pretrained models for computer vision to build a gesture recognition app
• A talk from GTC 21 on transfer learning (free to view with registration)

Award-winning research, stunning demos, a sweeping vision for how NVIDIA Omniverse will accelerate the work of millions more professionals, and a new pro RTX GPU were the highlights at this week’s SIGGRAPH pro graphics conference.

Kicking off the week, NVIDA’s SIGGRAPH special address featuring Richard Kerris, vice president, Omniverse, and Sanja Fidler, senior director, AI research, with an intro by Pixar co-founder Alvy Ray Smith gathered more than 1.6 million views in just 48 hours.

A documentary launched Wednesday, “Connecting in the Metaverse: The Making of the GTC Keynote”  – a behind-the-scenes view into how a small team of artists were able to blur the lines between real and rendered in NVIDIA’s GTC21 keynote achieved more than 360,000 views within the first 24 hours.

In all, NVIDIA brought together professionals from every corner of the industry, hosting over 12 sessions and launching 22 demos this week.

Among the highlights:

• NVIDIA announced a major expansion of NVIDIA Omniverse — the world’s first simulation and collaboration platform — through new integrations with Blender and Adobe that will open it to millions more users.
• NVIDIA’s research team took the coveted SIGGRAPH Best of Show award for a stunning new demo.
• And NVIDIA’s new NVIDIA RTX A2000 GPU, announced this week, will expand the availability of RTX technology to millions more pros.

It was a week packed with innovations, many captured in a new sizzle reel crammed with new technologies.

Sessions from  the NVIDIA Deep Learning Institute brought the latest ideas to veteran developers and students alike.

And the inaugural gathering of the NVIDIA Omniverse User Group brought more than 400 graphics professionals from all over the world together to learn about what’s coming next for Omniverse, to celebrate the work of the community, and announce the winners of the second #CreatewithMarbles: Marvelous Machine contest.

“Your work fuels what we do,” Rev Lebaredian, vice president of Omniverse engineering and simulation at NVIDIA told the scores of Omniverse users gathered for the event.

NVIDIA has been part of the SIGGRAPH community since 1993, with close to 150 papers accepted and NVIDIA employees leading more than 200 technical talks.

And SIGGRAPH has been the venue for some of NVIDIA’s biggest announcements — from OptiX in 2010 to the launch of NVIDIA RTX real-time ray tracing in 2018.

NVIDIA RTX A2000 Makes RTX More Accessible to More Pros

Since then, thanks to its powerful real-time ray tracing and AI acceleration capabilities, NVIDIA RTX technology has transformed design and visualization workflows for the most complex tasks.

Introduced Tuesday, the new NVIDIA RTX A2000 — our most compact, power-efficient GPU — makes it easier to access RTX from anywhere. With the unique packaging of the A2000, there are many new form factors, from backs of displays to edge devices, that are now able to incorporate RTX technology.

The RTX A2000 is designed for everyday workflows, so more professionals can develop photorealistic renderings, build physically accurate simulations and use AI-accelerated tools.

The GPU has 6GB of memory capacity with error correction code, or ECC, to maintain data integrity for uncompromised computing accuracy and reliability.

With remote work part of the new normal, simultaneous collaboration with colleagues on projects across the globe is critical.

NVIDIA RTX technology powers Omniverse, our collaboration and simulation platform that enables teams to iterate together on a single 3D design in real time while working across different software applications.

The A2000 will serve as a portal into this world for millions of designers.

Building the Metaverse

NVIDIA also announced a major expansion of NVIDIA Omniverse — the world’s first simulation and collaboration platform — through new integrations with Blender and Adobe that will open it to millions more users.

Omniverse makes it possible for designers, artists and reviewers to work together in real-time across leading software applications in a shared virtual world from anywhere.

Blender, the world’s leading open-source 3D animation tool, will now have Universal Scene Description, or USD, support, enabling artists to access Omniverse production pipelines.

Adobe is collaborating with NVIDIA on a Substance 3D plugin that will bring Substance Material support to Omniverse, unlocking new material editing capabilities for Omniverse and Substance 3D users.

So far, professionals at over 500 companies, including BMW, Volvo, SHoP Architects, South Park and Lockheed Martin, are evaluating the platform. Since the launch of its open beta in December, Omniverse has been downloaded by over 50,000 individual creators.

NVIDIA Research Showcases Digital Avatars at SIGGRAPH

More innovations are coming.

Highlighting their ongoing contributions to cutting-edge computer graphics, NVIDIA researchers put four AI models to work to serve up a stunning digital avatar demo for SIGGRAPH 2021’s Real-Time Live showcase.

Broadcasting live from our Silicon Valley headquarters, the NVIDIA Research team presented a collection of AI models that can create lifelike virtual characters for projects such as  bandwidth-efficient video conferencing and storytelling.

The demo featured tools to generate digital avatars from a single photo, animate avatars with natural 3D facial motion and convert text to speech.

The demo was just one highlight among a host of contributions from the more than 200 scientists who make up the NVIDIA Research team at this year’s conference.

Papers presented include:

• Real-Time Neural Radiance Caching for Path Tracing
• Neural Scene Graph Rendering
• An Unbiased Ray-Marching Transmittance Estimator
• StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters

NVIDIA Deep Learning Institute

These innovations quickly become tools that NVIDIA is hustling to bring to graphics professionals.

Created to help professionals and students master skills that will help them quickly advance their work, NVIDIA’s Deep Learning Institute held sessions covering a range of key technologies at SIGGRAPH.

They included a self-paced training on Getting Started with USD, a live instructor-led course on fundamentals of ray tracing, Using NVIDIA Nsight Graphics and NVIDIA Nsight Systems, a Masterclass by the Masters series on NVIDIA Omniverse, and a Graphics and NVIDIA Omniverse Teaching Kit for educators looking to incorporate hands-on technical training into student coursework. 

NVIDIA also showcased how its technology is transforming workflows in several demos, including:

• Factory of the Future: Participants explored the next era of manufacturing with this demo, which showcases BMW Group’s factory of the future — designed, simulated, operated and maintained entirely in NVIDIA Omniverse.
• Multiple Artists, One Server: SIGGRAPH attendees could learn how teams can accelerate visual effects production with the NVIDIA EGX platform, which enables multiple artists to work together on a powerful, secure server from anywhere.
• 3D Photogrammetry on an RTX Mobile Workstation: Participants got to watch how NVIDIA RTX-powered mobile workstations help drive the process of 3D scanning using photogrammetry, whether in a studio or a remote location.
• Interactive Volumes with NanoVDB in Blender Cycles: Attendees learned how NanoVDB makes volume rendering more GPU memory efficient, meaning larger and more complex scenes can be interactively adjusted and rendered with NVIDIA RTX-accelerated ray tracing and AI denoising.

Want to catch up on all the news from SIGGRAPH? Visit our hub for all things NVIDIA and SIGGRAPH at https://www.nvidia.com/en-us/events/siggraph/. 

Calling all warriors. It’s a glorious week full of new games.

This GFN Thursday comes with the exciting release of the new battle royale NARAKA: BLADEPOINT, as well as the Hello Neighbor franchise as part of the 11 great games joining the GeForce NOW library this week.

Plus, the newest Assassin’s Creed Valhalla DLC has arrived on the cloud.

Real PC Games, Real PC Power

Gaming on GeForce NOW means having access to the real versions of PC games. And there are more than 1,000 PC titles streaming from the cloud, with more on the way every week. It also means being able to play these titles across devices like low-powered PCs, Macs, Chromebooks, SHIELD TVs or Android and iOS mobile devices with the power of the cloud.

Members can play new and exciting PC games like NARAKA:BLADEPOINT with the power of a gaming rig streaming to any GeForce NOW compatible device at GeForce-level performance.

Melee Meets Battle Royale

Only one can remain. The melee, combat battle royale NARAKA: BLADEPOINT is now available on Steam and can be streamed on GeForce NOW. It’ll also be available to stream from the Epic Games Store upon its release in September.

Sixty players, heroes from around the world, will gather on Morus Island — and one will emerge victorious. Explore the vast, interactive world with a vertical design and experience unique gameplay powered by parkour and grappling hook movement. Learn to best use the brand-new resurrection system and unique character skills of a roster of characters with powerful abilities. And enjoy a vast arsenal of melee and ranged weapons along with the thrill of clashing blades and arrows flying in the battlefield.

Make your move. Press the assault on enemies with a grappling hook that can be aimed at anyone, anywhere and used to zip through obstacles to pounce on targets. Ambush opponents by hiding in the darkness and waiting for the right moment with deadly long-range takedowns or sneaky melee attacks. And avoid fights with a quick escape from less-favorable battles with a well-aimed grappling hook maneuver. Play your way to achieve victory.

Thanks to the GeForce power of the cloud, gamers can battle with the best and all other online PC gamers playing awesome multiplayer games like NARAKA: BLADEPOINT.

“It’s great that GeForce NOW can introduce gamers playing on low-powered hardware to the stunning world of NARAKA,” said Ray Kuan, lead producer. “We love that more gamers will be able to enter the battlefield and enjoy the next generation of battle royale games in full PC glory across all of their devices.”

Become the last warrior standing and learn the truth of NARAKA’s world and its endless battles on GeForce NOW this week.

Hello, It’s the Games of the Week

This GFN Thursday is packed with 11 new titles available to stream on GeForce NOW, including the stealth horror franchise, Hello Neighbor.

What’s your neighbor hiding? Members can find out and play Hello Neighbor, a suspenseful story of sneaking into your neighbor’s house to figure out what horrible secrets he’s hiding in the basement. Don’t get too comfortable — The Neighbor will learn from your every move and leave nasty surprises for you.

And stream the dramatic prequel, Hello Neighbor: Hide and Seek, to follow the tragic story of the loss of a family member while playing a game of hide-and-seek that leads to the game that started it all.

The full list of awesome games joining the service this week includes:

• City of Gangsters (day-and date release on Steam, August 9)
• Car Mechanic Simulator 2021 (day-and-date release on Steam, August 11)
• NARAKA: BLADEPOINT (day-and-date release on Steam, August 11)
• Faraday Protocol (day-and-date release on Steam, August 12)
• Voidtrain (day-and-date release on Epic Games Store, August 12)
• 428 Shibuya Scramble (Steam)
• Blood of Steel (Steam)
• DARIUSBURST Chronicle Saviours (Steam)
• Hello Neighbor (Steam and Epic Games Store)
• Hello Neighbor: Hide and Seek (Steam and Epic Games Store)
• RimWorld (Epic Games Store)

Finally, members will be able to sack a famous city and play the glorious new Assassin’s Creed Valhalla: The Siege of Paris DLC upon release today on GeForce NOW.

While you plan your gaming escape this weekend, we’ve got an important question for you.

Some games are so gorgeous, they make us never want to leave.

If you had to spend your summer vacation in a game which one would it be?

— NVIDIA GeForce NOW (@NVIDIAGFN) August 11, 2021

Tell us on Twitter or in the comments below, and we’ll catch up next week!

Talk about a magic trick. One moment, NVIDIA CEO Jensen Huang was holding forth from behind his sturdy kitchen counter.

The next, the kitchen and everything in it slid away, leaving Huang alone with the audience and NVIDIA’s DGX Station A100, a glimpse at an alternate digital reality.

For most, the metaverse is something seen in sci-fi movies. For entrepreneurs, it’s an opportunity. For gamers, a dream.

For NVIDIA artists, researchers and engineers on an extraordinarily tight deadline last spring, it was where they went to work — a shared virtual world they used to tell their story and a milestone for the entire company.

Designed to inform and entertain, NVIDIA’s GTC keynote is filled with cutting-edge demos highlighting advancements in supercomputing, deep learning and graphics.

“GTC is, first and foremost, our opportunity to highlight the amazing work that our engineers and other teams here at NVIDIA have done all year long,” said Rev Lebaredian, vice president of Omniverse engineering and simulation at NVIDIA.

With this short documentary, “Connecting in the Metaverse: The Making of the GTC Keynote,” viewers get the story behind the story. It’s a tale of how NVIDIA Omniverse, a tool for connecting to and describing the metaverse, brought it all together this year.

To be sure, you can’t have a keynote without a flesh and blood person at the center. Through all but 14 seconds of the hour and 48 minute presentation — from 1:02:41 to 1:02:55 — Huang himself spoke in the keynote.

Creating a Story in Omniverse

It starts with building a great narrative. Bringing forward a keynote-worthy presentation always takes intense collaboration. But this was unlike any other — packed not just with words and pictures — but with beautifully rendered 3D models and rich textures.

With Omniverse, NVIDIA’s team was able to collaborate using different industry content-creation tools like Autodesk Maya or Substance Painter while in different places.

“There are already great tools out there that people use every day in every industry that we want people to continue using,” said Lebaredian. “We want people to take these exciting tools and augment them with our technologies.”

These were enhanced by a new generation of tools, including Universal Scene Description (USD), Material Design Language (MDL) and NVIDIA RTX real-time ray-tracing technologies. Together, they allowed NVIDIA’s team to collaborate to create photorealistic scenes with physically accurate materials and lighting.

An NVIDIA DGX Station A100 Animation

Omniverse can create more than beautiful stills. The documentary shows how, accompanied by industry tools such as Autodesk Maya, Foundry Nuke, Adobe Photoshop, Adobe Premiere, and Adobe After Effects, it could stage and render some of the world’s most complex machines to create realistic cinematics.

With Omniverse, NVIDIA was able to turn a CAD model of the NVIDIA DGX Station A100 into a physically accurate virtual replica Huang used to give the audience a look inside.

Typically this type of project would take a team months to complete and weeks to render. But with Omniverse, the animation was chiefly completed by a single animator and rendered in less than a day.

Omniverse Physics Montage

More than just machines, though, Omniverse can model the way the world works by building on existing NVIDIA technologies. PhysX, for example, has been a staple in the NVIDIA gaming world for well over a decade. But its implementation in Omniverse brings it to a new level.

For a demo highlighting the current capabilities of PhysX 5 in Omniverse, plus a preview of advanced real-time physics simulation research, the Omniverse engineering and research teams re-rendered a collection of older PhysX demos in Omniverse.

The demo highlights key PhysX technologies such as Rigid Body, Soft Body Dynamics, Vehicle Dynamics, Fluid Dynamics, Blast’s Destruction and Fracture, and Flow’s combustible fluid, smoke and fire. As a result, viewers got a look at core Omniverse technologies that can do more than just show realistic-looking effects — they are true to reality, obeying the laws of physics in real-time.

DRIVE Sim, Now Built on Omniverse

Simulating the world around us is key to unlocking new technologies, and Omniverse is crucial to NVIDIA’s self-driving car initiative. With its PhysX and Photorealistic worlds, Omniverse creates the perfect environment for training autonomous machines of all kinds.

For this year’s DRIVE Sim on Omniverse demo, the team imported a map of the area surrounding a Mercedes plant in Germany. Then, using the same software stack that runs NVIDIA’s fleet of self-driving cars, they showed how the next generation of Mercedes cars would perform autonomous functions in the real world.

With DRIVE Sim, the team was able to test numerous lighting, weather and traffic conditions quickly — and show the world the results.

Creating the Factory of the Future with BMW Group

The idea of a “digital twin” has far-reaching consequences for almost every industry.

This year’s GTC featured a spectacular visionary display that exemplifies what the idea can do when unleashed in the auto industry.

The BMW Factory of the Future demo shows off the digital twin of a BMW assembly plant in Germany. Every detail, including layout, lighting and machinery, is digitally replicated with physical accuracy.

This “digital simulation” provides ultra-high fidelity and accurate, real-time simulation of the entire factory. With it, BMW can reconfigure assembly lines to optimize worker safety and efficiency, train factory robots to perform tasks, and optimize every aspect of plant operations.

Virtual Kitchen, Virtual CEO

The surprise highlight of GTC21 was a perfect virtual replica of Huang’s kitchen — the setting of the past three pandemic-era “kitchen keynotes” — complete with a digital clone of the CEO himself.

The demo is the epitome of what GTC represents: It combined the work of NVIDIA’s deep learning and graphics research teams with several engineering teams and the company’s incredible in-house creative team.

To create a virtual Jensen, teams did a full face and body scan to create a 3D model, then trained an AI to mimic his gestures and expressions and applied some AI magic to make his clone realistic.

Digital Jensen was then brought into a replica of his kitchen that was deconstructed to reveal the holodeck within Omniverse, surprising the audience and making them question how much of the keynote was real, or rendered.

“We built Omniverse first and foremost for ourselves here at NVIDIA,” Lebaredian said. “We started Omniverse with the idea of connecting existing tools that do 3D together for what we are now calling the metaverse.”

More and more of us will be able to do the same, accelerating more of what we do together. “If we do this right, we’ll be working in Omniverse 20 years from now,” Lebaredian said.

Immersive 3D design and character creation are going sky high this week at SIGGRAPH, in a demo showcasing NVIDIA CloudXR running on Google Cloud.

The clip shows an artist with an untethered VR headset creating a fully rigged character with Masterpiece Studio Pro, which is running remotely in Google Cloud and interactively streamed to the artist using CloudXR.

Bringing Characters to Life in XR

The demo focuses on an interactive technique known as digital sculpting, which uses software to create and refine a 3D model as if it were made of a real-life substance such as clay. But moving digital sculpting into a VR space creates a variety of challenges.

First, setting up the VR environment can be complicated and expensive. It typically requires dedicated physical space for wall-mounted sensors. If an artist wants to interact with the 3D model or move the character around, they can get tangled up in the cord that connects their VR headset to their workstation.

CloudXR, hosted from Google Cloud on a tetherless HMD, addresses these challenges by providing artists with the freedom to create from virtually anywhere. With a good internet connection, there’s no need for users to be physically tethered to an expensive workstation to have a seamless design session in an immersive environment.

Masterpiece Studio Pro is a fully immersive 3D creation pipeline that simplifies the character design process. From blocking in basic shapes to designing a fully textured and rigged character, artists can easily work on a character face-to-face in VR, providing a more intuitive experience.

In Masterpiece Studio Pro, artists can work on characters at any scale and use familiar tools and hand gestures to sculpt and pose models — just like they would with clay figures in real life. And drawing bones in position couldn’t be easier, because artists can reach right into the limbs of the creature to place them.

Getting Your Head in the Cloud

Built on NVIDIA RTX technology, CloudXR solves immersive design challenges by cutting the cord. Artists can work with a wireless, all-in-one headset, like the HTC VIVE Focus 3, without having to deal with the hassles of setting up a VR space.

And with CloudXR on Google Cloud, artists can rent an NVIDIA GPU on a Google Cloud Virtual Workstation, powered by NVIDIA RTX Virtual Workstation technology, and stream their work remotely. The VIVE Focus 3 is HTC’s latest standalone headset, which has 5K visuals and active cooling for long design sessions.

“We’re excited to show how complex creative workflows and high-quality graphics come together in the ultimate immersive experience — all running in the cloud,” said Daniel O’Brien, general manager at HTC Americas. “NVIDIA CloudXR and the VIVE Focus 3 provide a high quality experience to immerse artists in a seamless streaming experience.”

With Masterpiece Studio Pro running on Google Cloud, and streaming with NVIDIA CloudXR, users can enhance the workflow of creating characters in an immersive environment — one that’s more intuitive and productive than before.

Check out our other demos at SIGGRAPH, and learn more about NVIDIA CloudXR on Google Cloud.

With deep learning, amputees can now control their prosthetics by simply thinking through the motion.

Jules Anh Tuan Nguyen spoke with NVIDIA AI Podcast host Noah Kravitz about his efforts to allow amputees to control their prosthetic limb — right down to the finger motions — with their minds.

Using neural decoders and deep learning, this system allows humans to control just about anything digital with their thoughts, including playing video games and a piano.

Nguyen is a postdoctoral researcher in the biomedical engineering department at the University of Minnesota. His work with his team is detailed in a paper titled “A Portable, Self-Contained Neuroprosthetic Hand with Deep Learning-Based Finger Control.”

Key Points From This Episode:

• Nguyen and his team created an AI-based system using receptors implanted in the arm to translate the electrical information from the nerves into commands to execute the appropriate arm, hand and finger movements — all built into the arm.
• The two main objectives of the system are to make the neural interface wireless and to optimize the AI engine and neural decoder to consume less power — enough for a person to use it for at least eight hours a day before having to recharge it.

Tweetables:

“To make the amputee move and feel just like a real hand, we have to establish a neural connection for the amputee to move their finger and feel it just like a missing hand.” — Jules Anh Tuan Nguyen [7:24]

“The idea behind it can extend to many things. You can control virtual reality. You can control a robot, a drone — the possibility is endless. With this nerve interface and AI neural decoder, suddenly you can manipulate things with your mind.” — Jules Anh Tuan Nguyen [22:07]

You Might Also Like:

AI for Hobbyists: DIYers Use Deep Learning to Shoo Cats, Harass Ants

Robots recklessly driving cheap electric kiddie cars. Autonomous machines shining lasers at ants — and spraying water at bewildered cats — for the amusement of cackling grandchildren. Listen in to hear NVIDIA engineer Bob Bond and Make: Magazine Executive Editor Mike Senese explain how they’re entertaining with deep learning.

A USB Port for Your Body? Startup Uses AI to Connect Medical Devices to Nervous System

Think of it as a USB port for your body. Emil Hewage is the co-founder and CEO at Cambridge Bio-Augmentation Systems, a neural engineering startup. The U.K. startup is building interfaces that use AI to help plug medical devices into our nervous systems.

Behind the Scenes at NeurIPS With NVIDIA and CalTech’s Anima Anandkumar

Anima Anandkumar, NVIDIA’s director of machine learning research and Bren professor at CalTech’s CMS Department, talks about NeurIPS and discusses the transition from supervised to unsupervised and self-supervised learning, which she views as the key to next-generation AI.

Tune in to the AI Podcast

Get the AI Podcast through iTunes, Google Podcasts, Google Play, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn. If your favorite isn’t listed here, drop us a note.

Make the AI Podcast Better

Have a few minutes to spare? Fill out this listener survey. Your answers will help us make a better podcast.

In a turducken of a demo, NVIDIA researchers stuffed four AI models into a serving of digital avatar technology for SIGGRAPH 2021’s Real-Time Live showcase — winning the Best in Show award.

The showcase, one of the most anticipated events at the world’s largest computer graphics conference, held virtually this year, celebrates cutting-edge real-time projects spanning game technology, augmented reality and scientific visualization. It featured a lineup of jury-reviewed interactive projects, with presenters hailing from Unity Technologies, Rensselaer Polytechnic Institute, the NYU Future Reality Lab and more.

Broadcasting live from our Silicon Valley headquarters, the NVIDIA Research team presented a collection of AI models that can create lifelike virtual characters for projects such as bandwidth-efficient video conferencing and storytelling.

The demo featured tools to generate digital avatars from a single photo, animate avatars with natural 3D facial motion and convert text to speech.

“Making digital avatars is a notoriously difficult, tedious and expensive process,” said Bryan Catanzaro, vice president of applied deep learning research at NVIDIA, in the presentation. But with AI tools, “there is an easy way to create digital avatars for real people as well as cartoon characters. It can be used for video conferencing, storytelling, virtual assistants and many other applications.”

AI Aces the Interview

In the demo, two NVIDIA research scientists played the part of an interviewer and a prospective hire speaking over video conference. Over the course of the call, the interviewee showed off the capabilities of AI-driven digital avatar technology to communicate with the interviewer.

The researcher playing the part of interviewee relied on an NVIDIA RTX laptop throughout, while the other used a desktop workstation powered by RTX A6000 GPUs. The entire pipeline can also be run on GPUs in the cloud.

While sitting in a campus coffee shop, wearing a baseball cap and a face mask, the interviewee used the Vid2Vid Cameo model to appear clean-shaven in a collared shirt on the video call (seen in the image above). The AI model creates realistic digital avatars from a single photo of the subject — no 3D scan or specialized training images required.

“The digital avatar creation is instantaneous, so I can quickly create a different avatar by using a different photo,” he said, demonstrating the capability with another two images of himself.

Instead of transmitting a video stream, the researcher’s system sent only his voice — which was then fed into the NVIDIA Omniverse Audio2Face app. Audio2Face generates natural motion of the head, eyes and lips to match audio input in real time on a 3D head model. This facial animation went into Vid2Vid Cameo to synthesize natural-looking motion with the presenter’s digital avatar.

Not just for photorealistic digital avatars, the researcher fed his speech through Audio2Face and Vid2Vid Cameo to voice an animated character, too. Using NVIDIA StyleGAN, he explained, developers can create infinite digital avatars modeled after cartoon characters or paintings.

The models, optimized to run on NVIDIA RTX GPUs, easily deliver video at 30 frames per second. It’s also highly bandwidth efficient, since the presenter is sending only audio data over the network instead of transmitting a high-resolution video feed.

Taking it a step further, the researcher showed that when his coffee shop surroundings got too loud, the RAD-TTS model could convert typed messages into his voice — replacing the audio fed into Audio2Face. The breakthrough text-to-speech, deep learning-based tool can synthesize lifelike speech from arbitrary text inputs in milliseconds.

RAD-TTS can synthesize a variety of voices, helping developers bring book characters to life or even rap songs like “The Real Slim Shady” by Eminem, as the research team showed in the demo’s finale.

SIGGRAPH continues through Aug. 13. Check out the full lineup of NVIDIA events at the conference and catch the premiere of our documentary, “Connecting in the Metaverse: The Making of the GTC Keynote,” on Aug. 11.

The future of 3D graphics is on display at the SIGGRAPH 2021 virtual conference, where NVIDIA Studio is leading the way, showcasing exclusive benefits that NVIDIA RTX technologies bring to creators working with 3D workflows.

It starts with NVIDIA Omniverse, an immersive and connected shared virtual world where artists create one-of-a-kind digital scenes, perfect 3D models, design beautiful buildings and more with endless creative possibilities. The Omniverse platform continues to expand, gaining Blender USD support, a new Adobe Substance 3D plugin, and a new extension, GANverse3D — designed to make 3D modeling easier with AI.

Omniverse is currently in open beta and free for NVIDIA RTX and GeForce RTX GPU users. With today’s launch of the NVIDIA RTX A2000 GPU, millions more 3D artists and content creators will have the opportunity to explore the platform’s capabilities.

The latest creative app updates, along with Omniverse and RTX A2000 GPUs, gain improved levels of support in the August NVIDIA Studio Driver, available for download today.

Omniverse Expands the 3D Metaverse at SIGGRAPH

NVIDIA announced that Blender, the world’s leading open-source 3D animation application, will include support for Pixar’s Universal Scene Description (USD) in the Blender 3.0 release, enabling artists to use the application with Omniverse production pipelines.

The open-source 3D file framework gives software partners and artists multiple ways to extend and connect to Omniverse through USD adoption, building a plugin, or an Omniverse Connector, extension or app.

NVIDIA also unveiled an experimental Blender alpha 3.0 USD branch that includes more advanced USD and material support, which will be available soon for Blender users everywhere.

In addition, NVIDIA and Adobe are collaborating on a new Substance 3D plugin that will enable Substance Material support in Omniverse.

With the plugin, materials created in Adobe Substance 3D or imported from the Substance 3D Asset Library can be adjusted directly in Omniverse. 3D artists will save valuable time when making changes as they don’t need to export and reupload assets from Substance 3D Designer and Substance 3D Sampler.

We’re also releasing a new Omniverse extension, GANverse3D – Image2Car, which makes 3D modeling easier with AI. It’s the first of a collection of extensions that will comprise the Omniverse AI Toy Box.

GANverse3D was built on a generative adversarial network trained on 2D photos, synthesizing multiple views of thousands of objects to predict 3D geometry, texture and part segmentation labels. This process could turn a single photo of a car into a 3D model that can drive around a virtual scene, complete with realistic headlights, blinkers and wheels.

The AI Toy Box extension allows inexperienced 3D artists to easily create scenes, and experienced artists to bring new enhancements to their multi-app workflows.

Here’s GANverse3D in action with an Omniverse-connected workflow featuring Omniverse Create, Reallusion Character Creator 3 and Adobe Photoshop.

For a further dive into the latest innovations in 3D, including Omniverse, watch the NVIDIA special address at SIGGRAPH on demand.

Omniverse plays a critical role in many creative projects, like the GTC keynote with NVIDIA CEO Jensen Huang.

Get a sneak peek of how a small team of artists was able to blur the line between real and rendered.

The full documentary releases alongside the NVIDIA SIGGRAPH panel on Wednesday, August 11, at 11 a.m. PT.

The world’s leading artists use NVIDIA RTX and Omniverse to create beautiful work and stunning worlds. Hear from them directly in the second edition of NVIDIA’s RTX All Stars, a free e-book that spotlights creative professionals.

More RTX, More 3D Creative Freedom

NVIDIA RTX A2000 joins the RTX lineup as the most powerful, low-profile, dual-slot GPU for 3D creators. The new desktop GPU encompasses the latest RTX technologies in the NVIDIA Ampere architecture, including:

• 2nd-gen RT Cores for real-time ray tracing with up to 5x performance from last gen with RTX ON.
• 3rd-gen Tensor Cores to power and accelerate creative AI features.
• 2x PCIe Gen 4 accelerating data paths in and out of the GPU and up to 6GB of GPU ECC memory for rendering and exporting large files.

RTX A2000-based systems will be available starting in October.

For on-the-go creators, the NVIDIA RTX A2000 laptop GPU — available in Studio laptops shipping today like the Lenovo ThinkPad P17 Gen 2 — is the most power-efficient, professional RTX laptop GPU bringing ray tracing and AI capabilities to thin and light mobile workstations.

The NVIDIA RTX A2000 GPUs support a wealth of creative workflows, including 3D modeling and Omniverse, whether behind a desktop or anywhere a laptop may travel.

August Brings Creative App Updates and Latest Studio Driver

Several exciting updates to cutting-edge creative apps shipped recently. The August Studio Driver, available today, sharpens support for all of them.

Topaz Sharpen AI v3.2 offers refinements to AI models accelerated by RTX GPUs and Tensor Cores, adding 1.5x motion blur and Too Soft/Very Blurry features further reducing artifacts.

In-app masking has also been improved with real-time processing of mask strokes and customization controls for the overlay display.

Reallusion Character Creator v3.43, the first third-party app with Audio2Face integration, now allows artists to export characters from Character Creator to Omniverse as USD files with Audio2Face-compliant meshes. This allows facial and lip animations to be completely AI-driven solely from voice input, regardless of language, simplifying the process of animating a 3D character.

Capture One 21 v14.3.0 adds a new Magic Brush tool to create complex masks for layer editing based on image content in a split second, working on an underlying processed image from the raw file. This process is hardware accelerated and is up to 3x faster when using the GPU compared to the CPU.

Support for these app updates, plus the new features in Omniverse, are only a click away. Download the August Studio Driver.

Best Studio Laptops for 3D Workflows

3D workflows range from modeling scenes in real time with complex lights and shadows, to visualizing architectural marvels, in or out of Omniverse, with massive exports. These necessitate major computational power, requiring advanced NVIDIA RTX and GeForce RTX GPUs to get jobs done quickly.

These Studio laptops are built to handle demanding 3D creative workflows:

• Lenovo P1 Gen 4 is stylish and lightweight, at less than 4 pounds. It comes in a ton of configurations, including GeForce RTX 3070 and 3080, plus RTX A4000 and A5000 laptop GPUs.
• Dell Precision 7760 is their thinnest, smallest and lightest 17-inch mobile workstation. With up to an RTX A5000 and 16GB of video memory, it’s great for working with massive 3D models or in multi-app workflows.
• Acer ConceptD 7 Ezel features their patented Ezel Hinge with a 15.6-inch, 4K PANTONE-validated touchscreen display. Available later this month, it also comes with up to a GeForce RTX 3080 laptop GPU and 16GB of video memory.

Set to make a splash later this year is the HP Zbook Studio G8. Engineered for heavy 3D work, it comes well-equipped with up to an RTX A5000 or GeForce RTX 3080 laptop GPU, perfect for on-the-go creativity.

Browse the NVIDIA Studio Shop for more great options.

Stay up to date on all things Studio by subscribing to the NVIDIA Studio newsletter and following us on Facebook, Twitter and Instagram.

What is the metaverse? The metaverse is a shared virtual 3D world, or worlds, that are interactive, immersive, and collaborative.

Just as the physical universe is a collection of worlds that are connected in space, the metaverse can be thought of as a bunch of worlds, too.

Massive online social games, like battle royale juggernaut Fortnite and user-created virtual worlds like Minecraft and Roblox, reflect some elements of the idea.

Video-conferencing tools, which link far-flung colleagues together amidst the global COVID pandemic, are another hint at what’s to come.

But the vision laid out by Neal Stephenson’s 1992 classic novel “Snow Crash” goes well beyond any single game or video-conferencing app.

The metaverse will become a platform that’s not tied to any one app or any single place — digital or real, explains Rev Lebaredian, vice president of simulation technology at NVIDIA.

And just as virtual places will be persistent, so will the objects and identities of those moving through them, allowing digital goods and identities to move from one virtual world to another, and even into our world, with augmented reality.

“Ultimately we’re talking about creating another reality, another world, that’s as rich as the real world,” Lebaredian says.

Those ideas are already being put to work with NVIDIA Omniverse, which, simply put, is a platform for connecting 3D worlds into a shared virtual universe.

Omniverse is in use across a growing number of industries for projects such as design collaboration and creating “digital twins,” simulations of real-world buildings and factories.

How NVIDIA Omniverse Creates, Connects Worlds Within the Metaverse

So how does Omniverse work? We can break it down into three big parts.

The first is Omniverse Nucleus. It’s a database engine that connects users and enables the interchange of 3D assets and scene descriptions.

Once connected, designers doing modeling, layout, shading, animation, lighting, special effects or rendering can collaborate to create a scene.

Omniverse Nucleus relies on USD, or Universal Scene Description, an interchange framework invented by Pixar in 2012.

Released as open-source software in 2016, USD provides a rich, common language for defining, packaging, assembling and editing 3D data for a growing array of industries and applications.

Lebardian and others say USD is to the emerging metaverse what hyper-text markup language, or HTML, was to the web — a common language that can be used, and advanced, to support the metaverse.

Multiple users can connect to Nucleus, transmitting and receiving changes to their world as USD snippets.

The second part of Omniverse is the composition, rendering and animation engine — the simulation of the virtual world.

Omniverse is a platform built from the ground up to be physically based. Thanks to NVIDIA RTX graphics technologies, it is fully path traced, simulating how each ray of light bounces around a virtual world in real-time.

Omniverse simulates physics with NVIDIA PhysX. It simulates materials with NVIDIA MDL, or material definition language.

And Omniverse is fully integrated with NVIDIA AI (which is key to advancing robotics, more on that later).

Omniverse is cloud-native, scales across multiple GPUs, runs on any RTX platform and streams remotely to any device.

The third part is NVIDIA CloudXR, which includes client and server software for streaming extended reality content from OpenVR applications to Android and Windows devices, allowing users to portal into and out of Omniverse.

You can teleport into Omniverse with virtual reality, and AIs can teleport out of Omniverse with augmented reality.

Metaverses Made Real

NVIDIA released Omniverse to open beta in December, and NVIDIA Omniverse Enterprise in April. Professionals in a wide variety of industries quickly put it to work.

At Foster + Partners, the legendary design and architecture firm that designed Apple’s headquarters and London’s famed 30 St Mary Axe office tower — better known as “the Gherkin” — designers in 14 countries worldwide create buildings together in their Omniverse shared virtual space.

Visual effects pioneer Industrial Light & Magic is testing Omniverse to bring together internal and external tool pipelines from multiple studios. Omniverse lets them collaborate, render final shots in real-time and create massive virtual sets like holodecks.

Multinational networking and telecommunications company Ericsson uses Omniverse to simulate 5G wave propagation in real-time, minimizing multi-path interference in dense city environments.

Infrastructure engineering software company Bentley Systems is using Omniverse to create a suite of applications on the platform. Bentley’s iTwin platform creates a 4D infrastructure digital twin to simulate an infrastructure asset’s construction, then monitor and optimize its performance throughout its lifecycle.

The Metaverse Can Help Humans and Robots Collaborate

These virtual worlds are ideal for training robots.

One of the essential features of NVIDIA Omniverse is that it obeys the laws of physics. Omniverse can simulate particles and fluids, materials and even machines, right down to their springs and cables.

Modeling the natural world in a virtual one is a fundamental capability for robotics.

It allows users to create a virtual world where robots — powered by AI brains that can learn from their real or digital environments — can train.

Once the minds of these robots are trained in the Omniverse, roboticists can load those brains onto a NVIDIA Jetson, and connect it to a real robot.

Those robots will come in all sizes and shapes — box movers, pick-and-place arms, forklifts, cars, trucks and even buildings.

In the future, a factory will be a robot, orchestrating many robots inside, building cars that are robots themselves.

How the Metaverse, and NVIDIA Omniverse, Enable Digital Twins

NVIDIA Omniverse provides a description for these shared worlds that people and robots can connect to — and collaborate in — to better work together.

It’s an idea that automaker BMW Group is already putting to work.

The automaker produces more than 2 million cars a year. In its most advanced factory, the company makes a car every minute. And each vehicle is customized differently.

BMW Group is using NVIDIA Omniverse to create a future factory, a perfect “digital twin.” It’s designed entirely in digital and simulated from beginning to end in Omniverse.

The Omniverse-enabled factory can connect to enterprise resource planning systems, simulating the factory’s throughput. It can simulate new plant layouts. It can even become the dashboard for factory employees, who can uplink into a robot to teleoperate it.

The AI and software that run the virtual factory are the same as what will run the physical one. In other words, the virtual and physical factories and their robots will operate in a loop. They’re twins.

No Longer Science Fiction

Omniverse is the “plumbing,” on which metaverses can be built.

It’s an open platform with USD universal 3D interchange, connecting them into a large network of users. NVIDIA has 12 Omniverse Connectors to major design tools already, with another 40 on the way. The Omniverse Connector SDK sample code, for developers to write their own Connectors, is available for download now.

The most important design tool platforms are signed up. NVIDIA has already enlisted partners from the world’s largest industries — media and entertainment; gaming; architecture, engineering and construction; manufacturing; telecommunications; infrastructure; and automotive.

And the hardware needed to run it is here now.

Computer makers worldwide are building NVIDIA-Certified workstations, notebooks and servers, which all have been validated for running GPU-accelerated workloads with optimum performance, reliability and scale. And starting later this year, Omniverse Enterprise will be available for enterprise license via subscription from the NVIDIA Partner Network.

Thanks to NVIDIA Omniverse, the metaverse is no longer science fiction.

Back to the Future

So what’s next?

Humans have been exploiting how we perceive the world for thousands of years, NVIDIA’s Lebaredian points out. We’ve been hacking our senses to construct virtual realities through music, art and literature for millennia.

Next, add interactivity and the ability to collaborate, he says. Better screens, head-mounted displays like the Oculus Quest, and mixed-reality devices like Microsoft’s Hololens are all steps toward fuller immersion.

All these pieces will evolve. But the most important one is here already: a high-fidelity simulation of our virtual world to feed the display. That’s NVIDIA Omniverse.

To steal a line from science-fiction master William Gibson: the future is already here; it’s just not very evenly distributed.

The metaverse is the means through which we can distribute those experiences more evenly. Brought to life by NVIDIA Omniverse, the metaverse promises to weave humans, AI and robots together in fantastic new worlds.