For telcos around the world, one of the biggest challenges to upgrading networks has always been the question, “If you build it, will they come?”
Asia’s leading telco, Singtel, believes the key to helping customers innovate with AI across industries — for everything from traffic and video analytics to conversational AI avatars powered by large language models (LLMs) — is to offer multi-access edge compute services on its high-speed, ultra-low-latency 5G network.
Multi-access edge computing, or MEC, moves the computing of traffic and services from a centralized cloud to the edge of the network, where it’s closer to the customer. Doing so reduces network latency and lowers costs through sharing of network resources.
Singtel is collaborating with Microsoft and NVIDIA to combine AI and 5G, so enterprises can boost their innovation and productivity. Using NVIDIA’s full-stack accelerated computing platform optimized for Microsoft Azure Public MEC, the telco is creating solutions that enable customers to leverage AI video analytics for multiple use cases and to deploy 5G conversational avatars powered by LLMs.
From Sea to Shore
Singtel has been rolling out enterprise 5G and MEC across ports, airports, manufacturing facilities and other locations. In addition to running low-latency applications at the edge using Singtel’s 5G network, the solution has the potential to transform operations in sectors such as public safety, urban planning, healthcare, banking, civil service, transportation and logistics. It also offers high security for public sector customers and better performance for end users, enabling new intelligent edge scenarios.
Customers can use these capabilities through Microsoft Azure, only paying for the amount of compute and storage they use for the duration in which they use it. This replicates the cloud consumption model at the network edge and lets users save on additional operational overhead.
Edge Technologies
Singtel is working with video analytics software-makers participating in NVIDIA Inception, a free program that offers startups go-to-market support, expertise and technology. These ISVs will be able to use the NVIDIA Jetson Orin module for edge AI and robotics in conjunction with Microsoft MEC to identify traffic flows at airports and other high-population areas, retail video analytics and other use cases.
Singtel and NVIDIA are also showcasing their technology and solutions, including a real-time LLM-powered avatar developed by system integrator Quantiphi and based on NVIDIA Omniverse digital twin technology, at a May 11 launch event in Singapore. The avatar, built with NVIDIA Riva speech AI and the NeMo Megatron transformer model, enables people to interact in natural language on any topic of interest. Businesses can deploy these avatars anywhere over 5G.
Using Singtel’s high-speed, low-latency 5G — combined with NVIDIA AI accelerated infrastructure and capabilities — enterprises can explore use cases on everything from computer vision and mixed reality to autonomous guided vehicles.
Singtel plans to expand these new capabilities beyond Singapore to other countries and affiliated telcos, as well. This collaboration will help redefine what’s possible through the powerful combination of compute and next-generation networks, unlocking new operational efficiencies, revenue streams and customer experiences.
Editor’s note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.
Creative studio Elara Systems doesn’t shy away from sensitive subjects in its work.
Part of its mission for a recent client was to use fun, captivating visuals to help normalize what could be considered a touchy health subject — and boost medical outcomes as a result.
In collaboration with Boston Scientific and the Sickle Cell Society, the Elara Systems team created a character-driven 3D medical animation using the NVIDIA Omniverse development platform for connecting 3D pipelines and building metaverse applications.
The video aims to help adolescents experiencing sickle cell disease understand the importance of quickly telling an adult or a medical professional if they’re experiencing symptoms like priapism — a prolonged, painful erection that could lead to permanent bodily damage.
“Needless to say, this is something that could be quite frightening for a young person to deal with,” said Benjamin Samar, technical director at Elara Systems. “We wanted to make it crystal clear that living with and managing this condition is achievable and, most importantly, that there’s nothing to be ashamed of.”
To bring their projects to life, the Elara Systems team turns to the USD Composer app, generative AI-powered Audio2Face and Audio2Gesture, as well as Omniverse Connectors to Adobe Substance 3D Painter, Autodesk 3ds Max, Autodesk Maya and other popular 3D content-creation tools like Blender, Epic Games Unreal Engine, Reallusion iClone and Unity.
For the sickle cell project, the team relied on Adobe Substance 3D Painter to organize various 3D environments and apply custom textures to all five characters. Adobe After Effects was used to composite the rendered content into a single, cohesive short film.
“USD is extremely powerful and solves a ton of problems that many people may not realize even exist when it comes to effectively collaborating on a project,” Samar said. “For example, I can build a scene in Substance 3D Painter, export it to USD format and bring it into USD Composer with a single click. Shaders are automatically generated and linked, and we can customize things further if desired.”
An Animation to Boost Awareness
Grounding the sickle cell awareness campaign in a relatable, personal narrative was a “uniquely human approach to an otherwise clinical discussion,” said Samar, who has nearly two decades of industry experience spanning video production, motion graphics, 3D animation and extended reality.
The team accomplished this strategy through a 3D character named Leon — a 13-year-old soccer lover who shares his experiences about a tough day when he first learned how to manage his sickle cell disease.
3D character Leon in the Omniverse viewport.
The project began with detailed discussions about Sickle Cell Society’s goals for the short, followed by scripting, storyboarding and creating various sketches. “Once an early concept begins to crystallize in the artists’ minds, the creative process is born and begins to build momentum,” Samar said.
Sketches for Leon’s story.
Then, the team created rough 2D mockups using the illustration app Procreate on a tablet. This stage of the artistic process centered on establishing character outfits, proportions and other details. The final concept art was used as a clear reference to drive the rest of the team’s design decisions.
Rough animations of Leon made with Procreate.
Moving to 3D, the Elara Systems team tapped Autodesk Maya to build, rig and fully animate the characters, as well as Adobe Substance 3D Painter and Autodesk 3ds Max to create the short’s various environments.
“I’ve found the animated point cache export option in the Omniverse Connector for Maya to be invaluable,” Samar said. “It helps ensure that what we’re seeing in Maya will persist when brought into USD Composer, which is where we take advantage of real-time rendering to create high-quality visuals.”
The real-time rendering enabled by Omniverse was “critically important, because without it, we would have had zero chance of completing and delivering this content anywhere near our targeted deadline,” the technical artist said.
Leon brought to life in 3D using Adobe Substance 3D Painter, Autodesk Maya and USD Composer.
“I’m also a big fan of the Reallusion to Omniverse workflow,” he added.
The Connector allows users to easily bring characters created using Reallusion iClone into Omniverse, which helps to deliver visually realistic skin shaders. And USD Composer can enable real-time performance sessions for iClone characters when live-linked with a motion-capture system.
“Omniverse offers so much potential to help streamline workflows for traditional 3D animation teams, and this is just scratching the surface — there’s an ever-expanding feature set for those interested in robotics, digital twins, extended reality and game design,” Samar said. “What I find most assuring is the sheer speed of the platform’s development — constant updates and new features are being added at a rapid pace.”
Check out artwork from other “Omnivores” and submit projects in the gallery. Connect your workflows to Omniverse with software from Adobe, Autodesk, Epic Games, Maxon, Reallusion and more.
Artificial intelligence is teaming up with crowdsourcing to improve the thermo-stability – the ability to avoid breaking down under heat stress – of mRNA vaccines, making distribution more accessible worldwide.
In this episode of NVIDIA’s AI Podcast, host Noah Kravitz interviewed Bojan Tunguz, a physicist and senior system software engineer at NVIDIA, and Johnny Israeli, senior manager of AI and cloud software at NVIDIA.
The guests delved into AI’s potential in drug discovery and the Stanford Open Vaccine competition, a machine-learning contest using crowdsourcing to tackle the thermo-stability challenges of mRNA vaccines.
Kaggle, the online machine learning competition platform, hosted the Stanford Open Vaccine competition. Tunguz, a quadruple Kaggle grandmaster, shared how Kaggle has grown to encompass not just competitions, but also datasets, code and discussions. Competitors can earn points, rankings and status achievements across these four areas.
The fusion of AI, crowdsourcing and machine learning competitions is opening new possibilities in drug discovery and vaccine distribution. By tapping into the collective wisdom and skills of participants worldwide, it becomes possible to solve pressing global problems, such as enhancing the thermo-stability of mRNA vaccines, allowing for a more efficient and widely accessible distribution process.
Teaching the AI brains of autonomous vehicles to understand the world as humans do requires billions of miles of driving experience. The road to achieving this astronomical level of driving leads to the virtual world. Learn how Waabi uses powerful high-fidelity simulations to train and develop production-level autonomous vehicles.
Driving enjoyment and autonomous driving capabilities can complement one another in intelligent, sustainable vehicles. Learn about the automaker’s plans to unveil its third vehicle, the Polestar 3, the tech inside it, and what the company’s racing heritage brings to the intersection of smarts and sustainability.
Humans playing games against machines is nothing new, but now computers can develop their own games for people to play. Programming enthusiast and social media influencer Harrison Kinsley created GANTheftAuto, an AI-based neural network that generates a playable chunk of the classic video game Grand Theft Auto V.
Subscribe to the AI Podcast: Now Available on Amazon Music
Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows.
3D artist Milan Dey finds inspiration in games, movies, comics and pop culture. He drew from all of the above when creating a stunning 3D scene of Mayan ruins, The Hidden Temple of Itzamná, this week In the NVIDIA Studio.
“One evening, I was playing an adventure game and wanted to replicate the scene,” Milan said. “But I wanted my version to have a heavy Mayan influence.”
Milan sought vast, detailed architecture and carved rocks that look like they’ve stood with pride for centuries, similar to what can be seen in the Indiana Jones movies. The artist’s goals for his scene were to portray mother nature giving humanity a reminder that she is the greatest, to kick off with a grand introduction shot with light falling directly on the camera lens to create negative spaces in the frame, and to evoke that wild, wet smell of greens.
Below, Milan outlines his creative workflow, which combines tenacity with technical ability.
And for more inspiration, check out the NVIDIA Studio #GameArtChallenge reel, which includes highlights from our video-game-themed #GameArtChallenge entries.
It Belongs in a Museum
First things first, Milan gathers reference material. For this scene, the artist spent an afternoon capturing hundreds of screenshots and walkthrough videos of the game. He spent the next day on Artstation and Adobe Behance gathering visuals and sorting out projects of ruins.
Next, Milan browsed the Epic Games marketplace, which offers an extensive collection of assets for Unreal Engine creators.
“It crossed my mind that Aztec and Inca cultures are a great choice for a ruins environment,” said Milan. “Tropical settings have a variety of vegetation, whereas caves are deep enough to create their own biology and ecosystem.” With the assets in place, Milan organized them by level to create a 3D palette.
He then began with the initial blockout to prototype, test and adjust the foundational scene elements in Unreal Engine. The artist tested scene basics, replacing blocks with polished assets and applying lighting. He didn’t add anything fancy yet — just a single source of light to mimic normal daylight.
Blocking out stone walls.
Milan then searched for the best possible cave rocks and rock walls, with Quixel Megascans delivering the goods. Milan revisited the blocking process with the temple courtyard, placing cameras in multiple positions after initial asset placements. Next came the heavy task of adding vegetation and greens to the stone walls.
Getting the stone details just right.
“I put big patches of moss decals all around the walls, which gives a realistic look and feel,” Milan said. “Placing large- and medium-sized trees filled in a substantial part of the environment without using many resources.”
Vegetation is applied in painstaking detail.
As they say, the devil is in the details, Milan said.
“It’s very easy to get carried away with foliage painting and get lost in the depths of the cave,” the artist added. It took him another three days to fill in the smaller vegetation: shrubs, vines, plants, grass and even more moss.
The scene was starting to become staggeringly large, Milan said, but his ASUS ROG Strix Scar 15 NVIDIA Studio laptop was up to the task. His GeForce RTX 3080 GPU enabled RTX-accelerated rendering for high-fidelity, interactive visualization of his large 3D environment.
Simply stunning.
NVIDIA DLSS technology increased interactivity of the viewport by using AI to upscale frames rendered at lower resolution while retaining photorealistic detail.
“It’s simple: NVIDIA nailed ray tracing.” Milan said. “And Unreal Engine works best with NVIDIA and GeForce RTX graphics cards.”
A famed professor of archaeology explores the Mayan ruins.
Milan lit his scene with the HDRI digital image format to enhance the visuals and save file space, adding select directional lighting with exponential height fog. This created more density in low places of the map and less density in high places, adding further realism and depth.
Height fog adds realism to the 3D scene.
“It’s wild what you can do with a GeForce RTX GPU — using ray tracing or Lumen, the global illumination calculation is instant, when it used to take hours. What a time to be alive!” — Milan Dey
The artist doesn’t take these leaps in technology for granted, he said. “I’m from an era where we were required to do manual bouncing,” Dey said. “It’s obsolete now and Lumen is incredible.”
Lumen is Unreal Engine 5’s fully dynamic global illumination and reflections system that brings realistic lighting to scenes.
Milan reviewed each camera angle and made custom lighting adjustments, sometimes removing or replacing vegetation to make them pop with the lighting. He also added free assets from Sketchfab and special water effects to give the fountain an “eternity” vibe, he said.
With the scene complete, Milan quickly exported final renders thanks to his RTX GPU. “Art is the expression of human beings,” he stressed. “It demands understanding and attention.
To his past self or someone at the beginning of their creative journey, Milan would advise, “Keep an open mind and be teachable.”
Kris Kersey is an embedded software developer with over 20 years of experience, an educational YouTuber with 30,000+ subscribers, and a lifelong lover of comics and cosplay.
These interests and expertise came together in his first-ever project using the NVIDIA Jetson platform for edge AI and robotics when he created a fully functional superhero helmet as portrayed in one of his favorite Marvel Comic films, Iron Man.
The 3D-printed helmet comes complete with computer-vision capabilities in a heads-up display (HUD) that presents information wherever the user’s looking, just like in the movie.
The NVIDIA Jetson platform processes data from two cameras — one by each eye slot — that see what the helmet’s wearer is seeing. The HUD then presents information including the current temperature, humidity, altitude and GPS location. It can also classify what’s in the user’s view based on deep neural networks for object detection.
To let others join in on the fun, Kersey shared his entire workflow on his popular YouTube channel, Kersey Fabrications.
Superhero films and science fiction remind Kersey that cutting-edge technology requires collaboration across disciplines, he said.
“Often, as with this project, artists and storytellers use their imaginations to come up with brilliant ideas — then, it’s up to scientists and engineers to make them real,” the developer said.
About the Maker
Kersey, who studied computer science at Southern Polytechnic State University — now part of Kennesaw State University — in Georgia, has broad experience working with embedded microprocessors and architectures. He specializes in the Linux operating system, which is compatible with the NVIDIA Jetson platform.
“Writing software on the Jetson platform didn’t require that I learn a new programming language or operating system, which made it very easy for me,” the maker said.
By day, he’s a software engineer at an Atlanta-based startup. By night, he’s working on projects in his personal makerspace.
“I’ve never used my garage for cars,” he said.
Instead, it’s full of tools, boards and other equipment that enable his marvelous projects.
Kersey emphasized that what’s important to him most of all, however, is his family, with whom he likes to play board games, watch movies and go on hikes.
His Inspiration
Kersey’s fascination with technology stemmed from his childhood. His mother was a teacher focused on computer-aided drafting and mechanical design.
“From a very early age, I could tinker with computers that she had access to, which always fascinated me,” he said. “My cousin also once gave me an old 8-bit computer, but there wasn’t much I could do with it, so I remember pulling out the manual and reading the whole thing — that taught me basic programming.”
More recently, Kersey got into 3D printing while helping his son with a project for Science Olympiad.
“From that moment on, I got really into 3D printing as a hobby — my son never really took to it a whole lot,” he mused.
In 2018, Kersey created his YouTube channel with a focus on 3D printing as a way to delve deeper into the maker community while teaching others what he’s learned along the way.
A Jetson-Powered Superhero Project
Kersey’s 3D-printed, fully functional, wireless Iron Man helmet — which he even sanded and painted himself — could be straight out of the iconic films.
“For this whole experience to feel as awesome as Iron Man’s tech, it has to be real time, low latency, high resolution and high frame rate,” Kersey said. “It also needs to display a lot of information on screen, which requires a powerful graphics processor — that’s why I chose the Jetson platform.”
Jetson developer kits are equipped with a powerful, onboard NVIDIA GPU and AI capabilities to supercharge embedded applications.
Kersey also tapped the NVIDIA TensorRT software development kit to enable high-performance deep-learning inference with low latency and high throughput for the project.
For the next generation of the helmet’s HUD — a project that’s “not finished till it’s finished,” according to the maker — Kersey used the NVIDIA Jetson Orin Nano developer kit. Launched in September, the kit has set a new standard for creating entry-level AI-powered robots, intelligent cameras and more.
It only took Kersey two hours to get from opening the Orin Nano box to having the software deployed and running, he said.
He’s now looking to upgrade the project with the Jetson Orin NX 16GB system-on-module, as well as build a full suit beyond the headgear, starting with prototype aluminum repulsors.
And the developer will soon make the project’s code open source, so others can easily turn themselves into superheroes, too.
Kersey plans to wear the upgraded superhero gear at Dragon Con — the world’s largest multimedia, popular culture convention — taking place in August. Plus, at this month’s MomoCon in Atlanta, he’ll present on a panel titled Making It Real: High Tech in Cosplay.
Asked if Iron Man is his favorite superhero, Kersey said with a smile: “He is right now.”
What has it got in its pocketses? More games coming in May, that’s what.
GFN Thursday gets the summer started early with two newly supported games this week and 16 more coming later this month — including The Lord of the Rings: Gollum.
Don’t forget to take advantage of the limited-time discount on six-month Priority memberships. Priority members get faster access to cloud gaming servers, as well as support for RTX ON in supported games — all for 40% off the normal price. But hurry, this offer ends Sunday, May 21.
And the fun in May won’t stop there.
Stay tuned for more news on Xbox games joining the GeForce NOW library soon.
How Precious
No need to be sneaky about it — The Lord of the Rings: Gollum from Daedalic Entertainment comes to GeForce NOW when it releases on Thursday, May 25.
The action-adventure game and epic interactive experience takes place in parallel to the events described in The Fellowship of the Ring. Play as the enigmatic Gollum on his perilous journey and find out how he outwitted the most powerful characters in Middle-earth.
Climb the mountains of Mordor, sneak around Mirkwood and make difficult choices. Who will gain the upper hand: the cunning Gollum or the innocent Smeagol? Priority and Ultimate members can experience the epic story with support for RTX ray tracing and DLSS technology for AI-powered high-quality graphics, streaming across nearly any device with up to eight-hour sessions. Go Ultimate today with the one cloud gaming membership that rules them all.
May-Day Game-Day
It’s gonna be May, and that means more of the best games joining the GeForce NOW library.
Welcome to a new Age of Wonders.
Age of Wonders 4 is the long-awaited sequel from Paradox Interactive. A blend of 4x strategy and turn-based combat, members can explore new magical realms and rule over a faction of their design that grows with expanding empires. Battle through each chapter and guide your empire to greatness.
It leads two new games joining the cloud this week:
Poker Club unfortunately couldn’t be added in April due to technical issues. Tin Hearts also didn’t make it in April, but is included in the May list due to a shift in its release date.
With so many titles streaming from the cloud, what device will you be streaming on? Let us know in the comments below, or on Twitter or Facebook.
one does not simply game in the cloud with only 1 device
AV1, the next-generation video codec, is expanding its reach with today’s release of OBS Studio 29.1. This latest software update adds support for AV1 streaming to YouTube over Enhanced RTMP.
All GeForce RTX 40 Series GPUs — including laptop GPUs and the recently launched GeForce RTX 4070 — support real-time AV1 hardware encoding, providing 40% more efficient encoding on average than H.264 and delivering higher quality than competing GPUs.
AV1 vs. H.264 encode efficiency based on BD-SNR.
This reduces the upload bandwidth needed to stream, a common limitation from streaming services and internet service providers. At higher resolutions, AV1 encoding is even more efficient. For example, AV1 enables streaming 4K at 60 frames per second with 10 Mbps upload bandwidth — down from 20 Mbps with H.264 — making 4K60 streaming available to a wider audience.
AV1 — The New Standard
As a founding member of the Alliance for Open Media, NVIDIA has worked closely with industry titans in developing the AV1 codec. This work was necessitated by gamers and online content creators who pushed the boundaries of old formats that were defined roughly 20 years ago. The previous standard for livestreaming, H.264, usually maxed out with 1080p at 60 fps at the commonly used bitrates of 6-8 Mbps, and often produced blocky, grainy images.
AV1’s increased efficiency enables streaming higher-quality images, allowing creators to stream at higher resolutions with smoother frame rates. Even in network-limited environments, streamers can now reap the benefits of high-quality video shared with their audience.
Support for AV1 on YouTube comes through the recent update to RTMP. The enhanced protocol also adds support for HEVC streaming, bringing new formats to users on the existing low-latency protocol they use for H.264 streaming. Enhanced RTMP ingestion has been released as a beta feature on YouTube.
Learn how to configure OBS Studio for streaming AV1 with GeForce RTX 40 Series GPUs in the OBS setup guide.
Better Streams With NVENC, NVIDIA Broadcast
GeForce RTX 40 Series GPUs usher in a new era of high-quality streaming with AV1 encoding support on the eighth-generation NVENC. A boon to streamers, NVENC offloads compute-intensive encoding tasks from the CPU to dedicated hardware on the GPU.
Comparison of 4K image quality in AV1 livestream1.
Designed to support the rigors of professional content creators, NVENC preserves video quality with a higher accuracy than competitive encoders. GeForce RTX users can stream higher-quality images at the same bitrate as competitive products or encode at a lower bitrate while maintaining a similar picture quality.
NVIDIA Broadcast, part of the exclusive NVIDIA Studio suite of software, transforms any room into a home studio. Livestreams, voice chats and video calls look and sound better with powerful AI effects like eye contact, noise and room echo removal, virtual background and more.
NVIDIA today introduced a wave of cutting-edge AI research that will enable developers and artists to bring their ideas to life — whether still or moving, in 2D or 3D, hyperrealistic or fantastical.
Around 20 NVIDIA Research papers advancing generative AI and neural graphics — including collaborations with over a dozen universities in the U.S., Europe and Israel — are headed to SIGGRAPH 2023, the premier computer graphics conference, taking place Aug. 6-10 in Los Angeles.
The papers include generative AI models that turn text into personalized images; inverse rendering tools that transform still images into 3D objects; neural physics models that use AI to simulate complex 3D elements with stunning realism; and neural rendering models that unlock new capabilities for generating real-time, AI-powered visual details.
Innovations by NVIDIA researchers are regularly shared with developers on GitHub and incorporated into products, including the NVIDIA Omniverse platform for building and operating metaverse applications and NVIDIA Picasso, a recently announced foundry for custom generative AI models for visual design. Years of NVIDIA graphics research helped bring film-style rendering to games, like the recently released Cyberpunk 2077 Ray Tracing: Overdrive Mode, the world’s first path-traced AAA title.
The research advancements presented this year at SIGGRAPH will help developers and enterprises rapidly generate synthetic data to populate virtual worlds for robotics and autonomous vehicle training. They’ll also enable creators in art, architecture, graphic design, game development and film to more quickly produce high-quality visuals for storyboarding, previsualization and even production.
AI With a Personal Touch: Customized Text-to-Image Models
Generative AI models that transform text into images are powerful tools to create concept art or storyboards for films, video games and 3D virtual worlds. Text-to-image AI tools can turn a prompt like “children’s toys” into nearly infinite visuals a creator can use for inspiration — generating images of stuffed animals, blocks or puzzles.
However, artists may have a particular subject in mind. A creative director for a toy brand, for example, could be planning an ad campaign around a new teddy bear and want to visualize the toy in different situations, such as a teddy bear tea party. To enable this level of specificity in the output of a generative AI model, researchers from Tel Aviv University and NVIDIA have two SIGGRAPH papers that enable users to provide image examples that the model quickly learns from.
One paper describes a technique that needs a single example image to customize its output, accelerating the personalization process from minutes to roughly 11 seconds on a single NVIDIA A100 Tensor Core GPU, more than 60x faster than previous personalization approaches.
A second paper introduces a highly compact model called Perfusion, which takes a handful of concept images to allow users to combine multiple personalized elements — such as a specific teddy bear and teapot — into a single AI-generated visual:
Serving in 3D: Advances in Inverse Rendering and Character Creation
Once a creator comes up with concept art for a virtual world, the next step is to render the environment and populate it with 3D objects and characters. NVIDIA Research is inventing AI techniques to accelerate this time-consuming process by automatically transforming 2D images and videos into 3D representations that creators can import into graphics applications for further editing.
A third paper created with researchers at the University of California, San Diego, discusses tech that can generate and render a photorealistic 3D head-and-shoulders model based on a single 2D portrait — a major breakthrough that makes 3D avatar creation and 3D video conferencing accessible with AI. The method runs in real time on a consumer desktop, and can generate a photorealistic or stylized 3D telepresence using only conventional webcams or smartphone cameras.
A fourth project, a collaboration with Stanford University, brings lifelike motion to 3D characters. The researchers created an AI system that can learn a range of tennis skills from 2D video recordings of real tennis matches and apply this motion to 3D characters. The simulated tennis players can accurately hit the ball to target positions on a virtual court, and even play extended rallies with other characters.
Beyond the test case of tennis, this SIGGRAPH paper addresses the difficult challenge of producing 3D characters that can perform diverse skills with realistic movement — without the use of expensive motion-capture data.
Not a Hair Out of Place: Neural Physics Enables Realistic Simulations
Once a 3D character is generated, artists can layer in realistic details such as hair — a complex, computationally expensive challenge for animators.
Humans have an average of 100,000 hairs on their heads, with each reacting dynamically to an individual’s motion and the surrounding environment. Traditionally, creators have used physics formulas to calculate hair movement, simplifying or approximating its motion based on the resources available. That’s why virtual characters in a big-budget film sport much more detailed heads of hair than real-time video game avatars.
The team’s novel approach for accurate simulation of full-scale hair is specifically optimized for modern GPUs. It offers significant performance leaps compared to state-of-the-art, CPU-based solvers, reducing simulation times from multiple days to merely hours — while also boosting the quality of hair simulations possible in real time. This technique finally enables both accurate and interactive physically based hair grooming.
Neural Rendering Brings Film-Quality Detail to Real-Time Graphics
After an environment is filled with animated 3D objects and characters, real-time rendering simulates the physics of light reflecting through the virtual scene. Recent NVIDIA research shows how AI models for textures, materials and volumes can deliver film-quality, photorealistic visuals in real time for video games and digital twins.
NVIDIA invented programmable shading over two decades ago, enabling developers to customize the graphics pipeline. In these latest neural rendering inventions, researchers extend programmable shading code with AI models that run deep inside NVIDIA’s real-time graphics pipelines.
In a sixth SIGGRAPH paper, NVIDIA will present neural texture compression that delivers up to 16x more texture detail without taking additional GPU memory. Neural texture compression can substantially increase the realism of 3D scenes, as seen in the image below, which demonstrates how neural-compressed textures (right) capture sharper detail than previous formats, where the text remains blurry (center).
Neural texture compression (right) provides up to 16x more texture detail than previous texture formats without using additional GPU memory.
A related paper announced last year is now available in early access as NeuralVDB, an AI-enabled data compression technique that decreases by 100x the memory needed to represent volumetric data — like smoke, fire, clouds and water.
NVIDIA also released today more details about neural materials research that was shown in the most recent NVIDIA GTC keynote. The paper describes an AI system that learns how light reflects from photoreal, many-layered materials, reducing the complexity of these assets down to small neural networks that run in real time, enabling up to 10x faster shading.
The level of realism can be seen in this neural-rendered teapot, which accurately represents the ceramic, the imperfect clear-coat glaze, fingerprints, smudges and even dust.
The neural material model learns how light reflects from the many-layered, photoreal reference materials.
More Generative AI and Graphics Research
These are just the highlights — read more about all the NVIDIA papers at SIGGRAPH. NVIDIA will also present six courses, four talks and two Emerging Technology demos at the conference, with topics including path tracing, telepresence and diffusion models for generative AI.
NVIDIA Research has hundreds of scientists and engineers worldwide, with teams focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.
Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows. We’re also deep diving on new GeForce RTX 40 Series GPU features, technologies and resources, and how they dramatically accelerate content creation.
Content creator Grant Abbitt embodies selflessness, one of the best qualities that a creative can possess. Passionate about giving back to the creative community, Abbitt offers inspiration, guidance and free education for others in his field through YouTube tutorials.
He designed Dragon, the 3D scene featured this week In the NVIDIA Studio, specifically to help new Blender users easily understand the steps in the creative process of using the software.
“Dragons can be extremely tough to make,” said Abbitt. While he could have spent more time refining the details, he said, “That wasn’t the point of the project. It’s all about the learning journey for the student.”
Abbitt understands the importance of early education. Providing actionable, straightforward instructions enables prospective 3D modelers to make gradual progress, he said. When encouraged, 3D artists keep morale high while gaining confidence and learning more advanced skills, Abbitt has noticed over his 30+ years of industry experience.
His own early days of learning 3D workflows presented unique obstacles, like software programs costing as much as the hardware, or super-slow internet, which required Abbitt to learn 3D through instructional VHS tapes.
Learning 3D modeling and animation on VHS tapes.
Undeterred by such challenges, Abbitt earned a media studies degree and populated films with his own 3D content.
Now a full-time 3D artist and content creator, Abbitt does what he loves while helping aspiring content creators realize their creative ambitions. In this tutorial, for example, Abbitt teaches viewers how to create a video game character in just 20 minutes.
Dragon Wheel
Abbitt described a different dynasty in this realm — how he created his Dragon piece.
“Reference images are a must,” stressed Abbitt. “Deviation from the intended vision is part of the creative process, but without a direction or foundation, things can quickly go off track.” This is especially important with freelance work and creative briefs provided by clients, he added.
Abbitt looked to Pinterest and ArtStation for creative inspiration and reference material, and sketched in the Krita app on his tablet. The remainder of the project was completed in Blender — the popular 3D creation suite — which is free and open source.
Reference imagery set a solid foundation for the project.
He began with the initial blockout, a 3D rough-draft level built using simple 3D shapes without details or polished art assets. The goal of the blockout was to prototype, test and adjust the foundational shapes of the dragon. Abbitt then combined block shapes into a single mesh model, the structural build of a 3D model, consisting of polygons.
More sculpting was followed by retopologizing the mesh, the process of simplifying the topology of a mesh to make it cleaner and easier to work with. This is a necessary step for images that will undergo more advanced editing and distortions.
Adding Blender’s multiresolution modifier enabled Abbitt to subdivide a mesh, especially useful for re-projecting details from another sculpt with a Shrinkwrap modifier, which allows an object to “shrink” to the surface of another object. It can be applied to meshes, lattices, curves, surfaces and texts.
At this stage, the power of Abbitt’s GeForce RTX 4090 GPU really started to shine. He sculpted fine details faster with Blender Cycles RTX-accelerated OptiX ray tracing in the viewport for fluid, interactive modeling with photorealistic detail. Baking and applying textures were done with buttery smooth ease.
Astonishing details for a single 3D model.
The RTX 4090 GPU also accelerated the animation phase, where the artist rigged and posed his model. “Modern content creators require GPU technology to see their creative visions fully realized at an efficient pace,” Abbitt said.
For the texturing, painting and rendering process, Abbitt said he found it “extremely useful to be able to see the finished results without a huge render time, thanks to NVIDIA OptiX.”
Rendering final files in popular 3D creative apps — like Blender, Autodesk Maya with Autodesk Arnold, OTOY’s OctaneRender and Maxon’s Redshift — is made 70-200% faster with an RTX 4090 GPU, compared to previous-generation cards. This results in invaluable time saved for a freelancer with a deadline or a student working on a group project.
Abbitt’s RTX GPU enabled OptiX ray tracing in Blender Cycles for the fastest final frame render.
That’s one scary dragon.
“NVIDIA GeForce RTX graphics cards are really the only choice at the moment for Blender users, because they offer so much more speed during render times,” said Abbitt. “You can quickly see results and make the necessary changes.”
Content creator Grant Abbitt.
Check out Abbitt’s YouTube channel with livestreams every Friday at 9 a.m. PT.
Customers from Japan to Ecuador and Sweden are using NVIDIA DGX H100 systems like AI factories to manufacture intelligence.
They’re creating services that offer AI-driven insights in finance, healthcare, law, IT and telecom — and working to transform their industries in the process.
Among the dozens of use cases, one aims to predict how factory equipment will age, so tomorrow’s plants can be more efficient.
Called Green Physics AI, it adds information like an object’s CO2 footprint, age and energy consumption to SORDI.ai, which claims to be the largest synthetic dataset in manufacturing.
Green Physics AI lets users model how objects age.
The dataset lets manufacturers develop powerful AI models and create digital twins that optimize the efficiency of factories and warehouses. With Green Physics AI, they also can optimize energy and CO2 savings for the factory’s products and the components that go into them.
Meet Your Smart Valet
Imagine a robot that could watch you wash dishes or change the oil in your car, then do it for you.
Boston Dynamics AI Institute (The AI Institute), a research organization which traces its roots to Boston Dynamics, the well-known pioneer in robotics, will use a DGX H100 to pursue that vision. Researchers imagine dexterous mobile robots helping people in factories, warehouses, disaster sites and eventually homes.
“One thing I’ve dreamed about since I was in grad school is a robot valet who can follow me and do useful tasks — everyone should have one,” said Al Rizzi, CTO of The AI Institute.
That will require breakthroughs in AI and robotics, something Rizzi has seen firsthand. As chief scientist at Boston Dynamics, he helped create robots like Spot, a quadruped that can navigate stairs and even open doors for itself.
Initially, the DGX H100 will tackle tasks in reinforcement learning, a key technique in robotics. Later, it will run AI inference jobs while connected directly to prototype bots in the lab.
“It’s an extremely high-performance computer in a relatively compact footprint, so it provides an easy way for us to develop and deploy AI models,” said Rizzi.
Born to Run Gen AI
You don’t have to be a world-class research outfit or Fortune 500 company to use a DGX H100. Startups are unboxing some of the first systems to ride the wave of generative AI.
For example, Scissero, with offices in London and New York, employs a GPT-powered chatbot to make legal processes more efficient. Its Scissero GPT can draft legal documents, generate reports and conduct legal research.
In Germany, DeepL will use several DGX H100 systems to expand services like translation between dozens of languages it provides for customers, including Nikkei, Japan’s largest publishing company. DeepL recently released an AI writing assistant called DeepL Write.
Here’s to Your Health
Many of the DGX H100 systems will advance healthcare and improve patient outcomes.
In Tokyo, DGX H100s will run simulations and AI to speed the drug discovery process as part of the Tokyo-1 supercomputer. Xeureka — a startup launched in November 2021 by Mitsui & Co. Ltd., one of Japan’s largest conglomerates — will manage the system.
Separately, hospitals and academic healthcare organizations in Germany, Israel and the U.S. will be among the first users of DGX H100 systems.
Lighting Up Around the Globe
Universities from Singapore to Sweden are plugging in DGX H100 systems for research across a range of fields.
A DGX H100 will train large language models for Johns Hopkins University Applied Physics Laboratory. The KTH Royal Institute of Sweden will use one to expand its supercomputing capabilities.
Among other use cases, Japan’s CyberAgent, an internet services company, is creating smart digital ads and celebrity avatars. Telconet, a leading telecommunications provider in Ecuador, is building intelligent video analytics for safe cities and language services to support customers across Spanish dialects.
The eight H100 GPUs connect over NVIDIA NVLink to create one giant GPU. Scaling doesn’t stop there: organizations can connect hundreds of DGX H100 nodes into an AI supercomputer using the 400 Gbps ultra-low latency NVIDIA Quantum InfiniBand, twice the speed of prior networks.
Fueled by a Full Software Stack
DGX H100 systems run on NVIDIA Base Command, a suite for accelerating compute, storage, and network infrastructure and optimizing AI workloads.
They also include NVIDIA AI Enterprise, software to accelerate data science pipelines and streamline development and deployment of generative AI, computer vision and more.
The DGX platform offers both high performance and efficiency. DGX H100 delivers a 2x improvement in kilowatts per petaflop over the DGX A100 generation.
NVIDIA DGX H100 systems, DGX PODs and DGX SuperPODs are available from NVIDIA’s global partners.
Manuvir Das, NVIDIA’s vice president of enterprise computing, announced DGX H100 systems are shipping in a talk at MIT Technology Review’s Future Compute event today. A link to his talk will be available here soon.
NVIDIA GeForce NOW (@NVIDIAGFN) 
