Posts in category: nVidia AI

Professionals, teams, creators and others can tap into the power of AI to create high-quality audio and video effects — even using standard microphones and webcams — with the help of NVIDIA Maxine.

The suite of GPU-accelerated software development kits and cloud-native microservices lets users deploy AI features that enhance audio, video and augmented-reality effects for real-time communications services and platforms. Maxine will also expand features for video editing, enabling teams to reach new heights in video communication.

Plus, an NVIDIA Research demo at this week’s SIGGRAPH conference displays how AI can take video conferencing to the next level with 3D features.

NVIDIA Maxine Features Expand to Video Editing

Wireless connectivity has enabled people to join virtual meetings from more locations than ever. Typically, audio and video quality are heavily impacted when a caller is on the move or in a location with poor connectivity.

Advanced, real-time Maxine features — such as Background Noise Removal, Super Resolution and Eye Contact — allow remote users to enhance interpersonal communication experiences.

In addition, Maxine can now be used for video editing. NVIDIA partners are transforming this professional workflow with the same Maxine features that elevate video conferencing. The goal when editing a video, whether a sales pitch or a webinar, is to engage the broadest audience possible. Using Maxine, professionals can tap into AI features that enhance audio and video signals.

With Maxine, a spokesperson can look away from the screen to reference notes or a script while their gaze remains as if looking directly into the camera. Users can also film videos in low resolution and enhance the quality later. Plus, Maxine lets people record videos in several different languages and export the video in English.

Maxine features to be released in early access this year include:

• Interpreter: Translates from simplified Chinese, Russian, French, German and Spanish to English while animating the user’s image to show them speaking English.
• Voice Font: Enables users to apply characteristics of a speaker’s voice and map it to the audio output.
• Audio Super Resolution: Improves audio quality by increasing the temporal resolution of the audio signal and extending bandwidth. It currently supports upsampling from 8,000Hz to 16,000Hz as well as from 16,000Hz to 48,000Hz. This feature is also updated with more than 50% reduction in latency and up to 2x better throughput.
• Maxine Client: Brings the AI capabilities of Maxine’s microservices to video-conferencing sessions on PCs. The application is optimized for low-latency streaming and will use the cloud for all of its GPU compute requirements. Thin Client will be available on Windows this fall, with additional OS support to follow.

Maxine can be deployed in the cloud, on premises or at the edge, meaning quality communication can be accessible from nearly anywhere.

Taking Video Conferencing to New Heights

Many partners and customers are experiencing high-quality video conferencing and editing with Maxine. Two features of Maxine — Eye Contact and Live Portrait — are now available in production releases on the NVIDIA AI Enterprise software platform. Eye Contact simulates direct eye contact with the camera by estimating and aligning the user’s gaze with the camera. And Live Portrait animates a person’s portrait photo through their live video feed.

Software company Descript aims to make video a staple of every communicator’s toolkit, alongside docs and slides. With NVIDIA Maxine, professionals and beginners who use Descript can access AI features that improve their video-content workflows.

“With the NVIDIA Maxine Eye Contact feature, users no longer have to worry about memorizing scripts or doing tedious video retakes,” said Jay LeBoeuf, head of business and corporate development at Descript. “They can maintain a perfect on-screen presence while nailing their script every time.”

Reincubate’s Camo app aims to broaden access to great video by taking advantage of the hardware and devices people already own. It does this by giving users greater control over their image and by implementing a powerful, efficient processing pipeline for video effects and transformation. Using technologies enabled by NVIDIA Maxine, Camo can offer users an easier way to achieve incredible video creation.

“Integrating NVIDIA Maxine into Camo couldn’t have been easier, and it’s enabled us to get high performance from users’ RTX GPUs right out of the box,” said Aidan Fitzpatrick, founder and CEO of Reincubate. “With Maxine, the team’s been able to move faster and with more confidence.”

Quicklink’s Cre8 is a powerful video production platform for creating professional, on-brand productions, virtual and hybrid live events. The user-friendly interface combines an intuitive design with all the tools needed to build, edit and customize a professional-looking production. Cre8 incorporates NVIDIA Maxine technology to maximize productivity and the quality of video productions, offering complete control to the operator.

“Quicklink Cre8 now offers the most advanced video production platform on the planet,” said Richard Rees, CEO of Quicklink. “With NVIDIA Maxine, we were able to add advanced features, including Auto Framing, Video Noise Removal, Noise and Echo Cancellation, and Eye Contact Simulation.”

Los Angeles-based company gemelo.ai provides a platform for creating AI twins that can scale a user’s voice, content and interactions. Using Maxine’s Live Portrait feature, the gemelo.ai team can unlock new opportunities for scaled, personalized content and one-on-one interactions.

“The realism of Live Portrait has been a game-changer, unlocking new realms of potential for our AI twins,” said Paul Jaski, CEO of gemelo.ai. “Our customers can now design and deploy incredibly realistic digital twins with the superpowers of unlimited scalability in content production and interaction across apps, websites and mixed-reality experiences.”

NVIDIA Research Shows How 3D Video Enhances Immersive Communication

In addition to powering the advanced features of Maxine, NVIDIA AI enhances video communication with 3D. NVIDIA Research recently published a paper demonstrating how AI could power a 3D video-conferencing system with minimal capture equipment.

3D telepresence systems are typically expensive, require a large space or production studio, and use high-bandwidth, volumetric video streaming — all of which limits the technology’s accessibility. NVIDIA Research shared a new method, which runs on a novel VisionTransformer-based encoder, that takes 2D video input from a standard webcam and turns it into a 3D video representation. Instead of requiring 3D data to be passed back and forth between the participants in a conference, AI enables bandwidth requirements for the call to stay the same as for a 2D conference.

The technology takes a user’s 2D video and automatically creates a 3D representation called a neural radiance field, or NeRF, using volumetric rendering. As a result, participants can stream 2D videos, like they would for traditional video conferencing, while decoding high-quality 3D representations that can be rendered in real time. And with Maxine’s Live Portrait, users can bring their portraits to life in 3D.

AI-mediated 3D video conferencing could significantly reduce the cost for 3D capture, provide a high-fidelity 3D representation, accommodate photorealistic or stylized avatars, and enable mutual eye contact in video conferencing. Related research projects show how AI can help elevate communications and virtual interactions, as well as inform future NVIDIA technologies for video conferencing.

See the system in action below. SIGGRAPH attendees can visit the Emerging Technologies booth, where groups will be able to simultaneously view the live demo on a 3D display designed by New York-based company Looking Glass.

Availability

Learn more about NVIDIA Maxine, which is now available on NVIDIA AI Enterprise.

And see more of the research behind the 3D video conference project.

Featured image courtesy of NVIDIA Research.

AI and accelerated computing were in the spotlight at SIGGRAPH — the world’s largest gathering of computer graphics experts — as NVIDIA founder and CEO Jensen Huang announced during his keynote address updates to NVIDIA Omniverse, a platform for building and connecting 3D tools and applications, as well as acceleration for Universal Scene Description (known as OpenUSD), the open and extensible ecosystem for 3D worlds.

This follows the recent announcement of NVIDIA joining Pixar, Adobe, Apple and Autodesk to form the Alliance for OpenUSD. It marks a major leap toward unlocking the next era of 3D graphics, design and simulation by ensuring compatibility in 3D tools and content for digitalization across industries.

NVIDIA launched three new desktop workstation Ada Generation GPUs — the NVIDIA RTX 5000, RTX 4500 and RTX 4000 — which deliver the latest AI, graphics and real-time rendering technology to professionals worldwide.

Shutterstock is bringing generative AI to 3D scene backgrounds with a foundation model trained using NVIDIA Picasso, a cloud-based foundry for building visual generative AI models. Picasso-trained models can now generate photorealistic, 8K, 360-degree high-dynamic-range imaging (HDRi) environment maps for quicker scene development. Autodesk will also integrate generative AI content-creation services — developed using foundation models in Picasso — with its popular Autodesk Maya software.

Each month, NVIDIA Studio Driver releases provide artists, creators and 3D developers with the best performance and reliability when working with creative applications. Available today, the August NVIDIA Studio Driver gives creators peak reliability for using their favorite creative apps. It includes support for updates to Omniverse, XSplit Broadcaster and Reallusion iClone.

Plus, this week’s featured In the NVIDIA Studio artist Andrew Averkin shows how AI influenced his process in building a delightful cup of joe for his Natural Coffee piece.

Omniverse Expands

Omniverse received a major upgrade, bringing new connectors and advancements to the platform.

These updates are showcased in Omniverse foundation applications, which are fully customizable reference applications that creators, enterprises and developers can copy, extend or enhance.

Upgraded Omniverse applications include Omniverse USD Composer, which lets 3D users assemble large-scale, OpenUSD-based scenes. Omniverse Audio2Face — which provides generative AI application programming interfaces that create realistic facial animations and gestures from only an audio file — now includes multilingual support and a new female base model.

The update brings boosted efficiency and an improved user experience. New rendering optimizations take full advantage of the NVIDIA Ada Lovelace architecture enhancements in NVIDIA RTX GPUs with DLSS 3 technology fully integrated into the Omniverse RTX Renderer. In addition, a new AI denoiser enables real-time 4K path tracing of massive industrial scenes.

New application and experience templates provide developers getting started with OpenUSD and Omniverse a major headstart with minimal coding.

A new Omniverse Kit Extension Registry, a central repository for accessing, sharing and managing Omniverse extensions, lets developers easily turn functionality on and off in their application, making it easier than ever to build custom apps from over 500 core Omniverse extensions provided by NVIDIA.

New extended-reality developer tools let users build spatial-computing options natively into their Omniverse-based applications, giving users the flexibility to experience their 3D projects and virtual worlds however they like.

Expanding their collaboration across Adobe Substance 3D, generative AI and OpenUSD initiatives, Adobe and NVIDIA announced plans to make Adobe Firefly, Adobe’s family of creative generative AI models, available as APIs in Omniverse, enabling developers and creators to enhance their design processes.

Developers and industrial enterprises have new foundation apps and services to optimize and enhance 3D pipelines with the OpenUSD framework and generative AI.

Studio professionals can connect the world of generative AI to their workflows to accelerate entire projects — from environment creation and character animation to scene-setting and more. With Kit AI Agent, OpenUSD Connectors and extensions to prompt top generative AI tools and APIs, Omniverse can aggregate the final result in a unified viewport — collectively reducing the time from conception to creation.

RTX: The Next Generation

The new NVIDIA RTX 5000, RTX 4500 and RTX 4000 Ada Generation professional desktop GPUs feature the latest NVIDIA Ada Lovelace architecture technologies, including DLSS 3, for smoother rendering and real-time interactivity in 3D applications such as Unreal Engine.

These workstation-class GPUs feature third-generation RT Cores with up to 2x the throughput of the previous generation. This enables users to work with larger, higher-fidelity images in real time, helping artists and designers maintain their creative flow.

Fourth-generation Tensor Cores deliver up to 2x the AI performance of the previous generation for AI training and development as well as inferencing and generative AI workloads. ‌Large GPU memory enables AI-augmented multi-application workflows with the latest generative AI-enabled tools and applications.

The Ada architecture provides these new GPUs with up to twice the video encode and decode capability of the previous generation, encoding up to 8K60 video in real time, with support for AV1 encode and decode. Combined with next-generation AI performance, these capabilities make the new professional GPUs ideal for multi-stream video editing workflows with high-resolution content using  AI-augmented video editing applications such as Adobe Premiere and DaVinci Resolve.

Designed for high-end creative, multi-application professional workflows that require large models and datasets, these new GPUs provide large GDDR6 memory: 20GB for the RTX 4000, 24GB for the RTX 4500 and 32GB for the RTX 5000 — all supporting error-correcting code for error-free computing.

A Modern-Day Picasso

3D artists regularly face the monumental task of bringing scenes to life by artistically mixing hero assets with props, materials, backgrounds and lighting. Generative AI technologies can help streamline this workflow by generating secondary assets, like environment maps that light the scene.

At SIGGRAPH, Shutterstock announced that it’s tapping into NVIDIA Picasso to train a generative AI model that can create 360 HDRi photorealistic environment maps. The model is built using Shutterstock’s responsibly licensed libraries.

Previously, artists needed to use expensive 360-degree cameras to create backgrounds and environment maps from scratch, or choose from fixed options that may not precisely match their 3D scene. Now, from simple prompts or using their desired background as a reference, the Shutterstock generative AI feature will quickly generate custom 360-degree, 8K-resolution, HDRi environment maps, which artists can use to set a background and light a scene. This allows more time to work on hero 3D assets, which are the primary assets of a 3D scene that viewers will focus on.

Autodesk also announced that it will integrate generative AI content-creation services — developed using foundation models in Picasso — with its popular 3D software Autodesk Maya.

August Studio Driver Delivers

The August Studio Driver supports these updates and more, including the latest release of XSplit Broadcaster, the popular streaming software that lets users simultaneously stream to multiple platforms.

XSplit Broadcaster 4.5 introduces NVIDIA Encoder (NVENC) AV1 support. GeForce and NVIDIA RTX 40 Series GPU users can now stream in high-quality 4K 60 frames per second directly to YouTube Live, dramatically improving video quality.

Streaming in AV1 with RTX GPUs provides 40% better efficiency than H.264, reducing bandwidth requirements for livestreaming or reducing file size for high-quality local captures.

An update to the Reallusion iClone Omniverse Connector includes new features such as real-time synchronization of projects, as well as enhanced import functionality for OpenUSD. This makes work between iClone and Omniverse quicker, smoother and more efficient.

Brew-tiful Artwork

Words can’t espresso the stunning 3D scene Natural Coffee.

NVIDIA artist Andrew Averkin has over 15 years of experience in the creative field. He finds joy in a continuous journey — blending art and technology — to bring his vivid imagination to life.

His work, Natural Coffee, has a compelling origin story. Once upon a time, in a bustling office, there was a cup of “natural coffee” known for its legendary powers. It gave artists nerves of steel at work, improved performance across the board and, as a small bonus, offered magical music therapy.

Averkin used an image generator to quickly cycle through visual ideas created from simple text-based prompts. Using AI to brainstorm imagery at the beginning of creative workflows is becoming more popular by artists looking to save time on iteration.

With a visual foundation, Averkin speeds up the process by acquiring 3D assets from online stores to quickly build a 3D blockout of the scene, a rough-draft level built using simple 3D shapes without details or polished details.

Next, Averkin polished individual assets in Autodesk 3ds Max, sculpting models with fine detail, testing and applying different textures and materials. His GeForce RTX 4090 GPU unlocked RTX-accelerated AI denoising — with the default Autodesk Arnold renderer — delivering interactive 3D modeling, which helped tremendously while composing the scene.

“I chose a GeForce RTX graphics card for quality, speed and safety, plain and simple,” said Averkin.

Averkin then exported Natural Coffee to the NVIDIA Omniverse USD Composer app via the Autodesk 3ds Max Connector. “Inside USD Composer I added more details, played a lot with a built-in collection of materials, plus did a lot of lighting work to make composition look more realistic,” he explained.

One of the biggest benefits in USD Composer is the ability to review scenes rendering in real time with photorealistic light, shadows, textures and more. This dramatically improves the process of editing massive 3D scenes, making it quicker and easier. Averkin was even able to add a camera fly animation, further elevating the scene.

The final step was to add a few touch-ups in Adobe Photoshop. Over 30 GPU-accelerated features gave Averkin plenty of options for playing with colors and contrast, and making final image adjustments smoothly and quickly.

Averkin encourages advanced 3D artists to experiment with the OpenUSD framework. “I use it a lot in my work at NVIDIA and in personal projects,” he said. “OpenUSD is very powerful. It helps with work in multiple creative apps in a non-destructive way, and other great features make the entire process easier and more flexible.”

Check out Averkin’s portfolio on ArtStation.

Follow NVIDIA Studio on Instagram, Twitter and Facebook. Access tutorials on the Studio YouTube channel and get updates directly in your inbox by subscribing to the Studio newsletter.

Picture this: Creators can quickly create and customize 3D scene backgrounds with the help of generative AI, thanks to cutting-edge tools from Shutterstock.

The visual-content provider is building services using NVIDIA Picasso — a cloud-based foundry for developing generative AI models for visual design.

The work incorporates Picasso’s latest feature — announced today during NVIDIA founder and CEO Jensen Huang’s SIGGRAPH keynote — which will help artists enhance and light 3D scenes based on simple text or image prompts, all with AI models built using fully licensed, rights-reserved data.

From these prompts, the new gen AI feature quickly generates custom 360-degree, 8K-resolution, high-dynamic-range imaging (HDRi) environment maps, which artists can use to set a background and light a scene.

This expands on NVIDIA’s collaboration with Shutterstock to empower the next generation of digital content-creation tools and accelerate 3D model generation.

To meet a surging demand for immersive visuals in films, games, virtual worlds, advertising and more, the 3D artist community is rapidly expanding, with over 20% growth in the past year.

Many of these artists are tapping generative AI to bolster their complex workflows — and will be able to use the technology to quickly create and customize environment maps. This allows more time to work on hero 3D assets, which are the primary assets of a 3D scene that viewers will focus on. It makes a panoramic difference when creating compelling 3D visuals.

“We’re committed to hyper-enabling 3D artists and collaborators — helping them build the immersive environments they envision faster than ever before and streamlining their content-creation workflows using NVIDIA Picasso,” said Dade Orgeron, vice president of 3D innovation at Shutterstock.

Generating Photorealistic Environment Maps

Previously, artists needed to buy expensive 360-degree cameras to create backgrounds and environment maps from scratch, or choose from fixed options that may not precisely match their 3D scene.

Now, users can simply provide a prompt — whether that’s text or a reference image — and the 360 HDRi services built on Picasso will quickly generate panoramic images. Plus, thanks to generative AI, the custom environment map can automatically match the background image that’s inputted as a prompt.

Users can then customize the maps and quickly iterate on ideas until they achieve the vision they want.

Collaboration to Boost 3D World-Building

Autodesk, a provider of 3D software and tools for creators in media and entertainment, is focused on giving artists the creative freedom to inspire and delight audiences worldwide.

Enabling artists to trade mundane tasks for unbridled creativity, Autodesk will integrate generative AI content-creation services — developed using foundation models in Picasso — with its popular 3D software Maya.

Supercharging Autodesk customer workflows with AI allows artists to focus on creating — and to ultimately produce content faster.

Generative AI Model Foundry

Picasso is part of NVIDIA AI Foundations, which advances enterprise-level generative AI for text, visual content and even biology.

The foundry will also adopt new NVIDIA research to generate physics-based rendering materials from text and image prompts, demonstrated at SIGGRAPH’s Real-Time Live competition. This will enable content providers to create 3D services, software and tools that enhance and expedite the simulation of diverse physical materials, such as tiles, metals and wood — complete with texture-mapping techniques, including normal, roughness and ambient occlusion.

Picasso runs on the NVIDIA Omniverse Cloud platform-as-a-service and is accessible via a serverless application programming interface that content and service providers like Shutterstock can easily connect to their websites and applications.

Learn about the latest advances in generative AI, graphics and more by joining NVIDIA at SIGGRAPH, running through Thursday, Aug. 10.

DENZA, the luxury EV brand joint venture between BYD and Mercedes-Benz, has collaborated with marketing and communications giant WPP and NVIDIA Omniverse Cloud to build and deploy its next generation of car configurators, NVIDIA founder and CEO Jensen Huang announced at SIGGRAPH.

WPP is using Omniverse Cloud — a platform for developing, deploying and managing industrial digitalization applications — to help unify the automaker’s highly complex design and marketing pipeline.

Omniverse Cloud enables WPP to build a single, physically accurate, real-time digital twin of the DENZA N7 model by integrating full-fidelity design data from the EV maker’s preferred computer-aided design tools via Universal Scene Description, or OpenUSD.

OpenUSD is a 3D framework that enables interoperability between software tools and data types for the building of virtual worlds.

The implementation of a new unified asset pipeline breaks down proprietary data silos, fostering enhanced data accessibility and facilitating collaborative, iterative reviews for the organization’s large design teams and stakeholders. It enables WPP to work on launch campaigns earlier in the design process, making iterations faster and less costly.

Unifying Asset Pipelines With Omniverse Cloud

Using Omniverse Cloud, WPP’s teams can connect their own pipeline of OpenUSD-enabled design and content creation tools such as Autodesk Maya and Adobe Substance 3D Painter to develop a new configurator for the DENZA N7. With a unified asset pipeline in Omniverse, WPP’s teams of artists can iterate and edit in real time a path-traced view of the full engineering dataset of the DENZA N7 — ensuring the virtual car accurately represents the physical car.

Traditional car configurators require hundreds of thousands of images to be prerendered to represent all possible options and variants. OpenUSD makes it possible for WPP to create a digital twin of the car that includes all possible variants in one single asset. No prerendered images are required.

In parallel, WPP’s environmental artists create fully interactive, live 3D virtual sets. These can start with a scan of a real-world environment, such as those WPP captures with their robot dog, or tap into generative AI tools from providers such as Shutterstock to instantly generate 360-degree HDRi backgrounds to maximize opportunity for personalization.

Shutterstock is using NVIDIA Picasso — a foundry for building generative AI visual models — to  develop a variety of generative AI services to accelerate 3D workflows. At SIGGRAPH, Shutterstock announced the first offering of these new services – 360 HDRi – to create photorealistic HDR environment maps to relight a scene. With this feature, artists can rapidly create custom environments that fit their needs.

One-Click Publish to GDN

Once the 3D experience is complete, with just one click, WPP can publish it to Graphics Delivery Network (GDN), part of NVIDIA Omniverse Cloud. GDN is a network of data centers built to serve real-time, high-fidelity 3D content to nearly any web device, enabling interactive experiences in the dealer showroom as well as on consumers’ mobile devices.

This eliminates the tedious process of manually packaging, deploying, hosting and managing the experience themselves. If updates are needed, just like with the initial deployment, WPP can publish them with a single click.

CTA: Learn more about Omniverse Cloud and GDN.

Microsoft Azure users can now turn to the latest NVIDIA accelerated computing technology to train and deploy their generative AI applications.

Available today, the Microsoft Azure ND H100 v5 VMs using NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking — enables scaling generative AI, high performance computing (HPC) and other applications with a click from a browser.

Available to customers across the U.S., the new instance arrives as developers and researchers are using large language models (LLMs) and accelerated computing to uncover new consumer and business use cases.

The NVIDIA H100 GPU delivers supercomputing-class performance through architectural innovations, including fourth-generation Tensor Cores, a new Transformer Engine for accelerating LLMs and the latest NVLink technology that lets GPUs talk to each other at 900GB/sec.

The inclusion of NVIDIA Quantum-2 CX7 InfiniBand with 3,200 Gbps cross-node bandwidth ensures seamless performance across the GPUs at massive scale, matching the capabilities of top-performing supercomputers globally.

Scaling With v5 VMs

ND H100 v5 VMs are ideal for training and running inference for increasingly complex LLMs and computer vision models. These neural networks drive the most demanding and compute-intensive generative AI applications, including question answering, code generation, audio, video and image generation, speech recognition and more.

The ND H100 v5 VMs achieve up to 2x speedup in LLMs like the BLOOM 175B model for inference versus previous generation instances, demonstrating their potential to further optimize AI applications.

NVIDIA and Azure

NVIDIA H100 Tensor Core GPUs on Azure provide enterprises the performance, versatility and scale to supercharge their AI training and inference workloads. The combination streamlines the development and deployment of production AI with the NVIDIA AI Enterprise software suite integrated with Azure Machine Learning for MLOps, and delivers record-setting AI performance in industry-standard MLPerf benchmarks.

In addition, by connecting the NVIDIA Omniverse platform to Azure, NVIDIA and Microsoft are providing hundreds of millions of Microsoft enterprise users with access to powerful industrial digitalization and AI supercomputing resources.

Learn more about new Azure v5 instances powered by NVIDIA H100 GPUs.

One pandemic and one generative AI revolution later, NVIDIA founder and CEO Jensen Huang returns to the SIGGRAPH stage next week to deliver a live keynote at the world’s largest professional graphics conference.

The address, slated for Tuesday, Aug. 8, at 8 a.m. PT in Los Angeles, will feature an exclusive look at some of NVIDIA’s newest breakthroughs, including award-winning research, OpenUSD developments and the latest AI-powered solutions for content creation.

Huang’s address comes after NVIDIA joined forces last week with Pixar, Adobe, Apple and Autodesk to found the Alliance for OpenUSD, a major leap toward unlocking the next era of interoperability in 3D graphics, design and simulation.

The group will standardize and extend OpenUSD, the open-source Universal Scene Description framework that’s the foundation of interoperable 3D applications and projects ranging from visual effects to industrial digital twins.

Huang will also offer a perspective on what’s been a raucous year for AI, with wildly popular new generative AI applications — including ChatGPT and Midjourney — providing a taste of what’s to come as developers worldwide get to work.

Throughout the conference, NVIDIA will participate in sessions on immersive visualization, 3D interoperability and AI-mediated video conferencing and presenting 20 research papers. Attendees will also get the opportunity to join hands-on labs.

Join SIGGRAPH to witness the evolution of AI and visual computing. Watch the keynote on this page.

Image source: Ron Diering, via Flickr, some rights reserved.

Goran Vuksic is the brain behind a project to build a real-world pit droid, a type of Star Wars bot that repairs and maintains podracers which zoom across the much-loved film series.

The edge AI Jedi used an NVIDIA Jetson Orin Nano Developer Kit as the brain of the droid itself. The devkit enables the bot, which is a little less than four feet tall and has a simple webcam for eyes, to identify and move its head toward objects.

Vuksic — originally from Croatia and now based in Malmö, Sweden — recently traveled with the pit droid across Belgium and the Netherlands to several tech conferences. He presented to hundreds of people on computer vision and AI, using the droid as an engaging real-world demo.

A self-described Star Wars fanatic, he’s upgrading the droid’s capabilities in his free time, when not engrossed in his work as an engineering manager at a Copenhagen-based company. He’s also co-founder and chief technology officer of syntheticAIdata, a member of the NVIDIA Inception program for cutting-edge startups.

The company, which creates vision AI models with cost-effective synthetic data, uses a connector to the NVIDIA Omniverse platform for building and operating 3D tools and applications.

About the Maker

Named a Jetson AI Specialist by NVIDIA and an AI “Most Valuable Professional” by Microsoft, Vuksic got started with artificial intelligence and IT about a decade ago when working for a startup that classified tattoos with vision AI.

Since then, he’s worked as an engineering and technical manager, among other roles, developing IT strategies and solutions for various companies.

Robotics has always interested him, as he was a huge sci-fi fan growing up.

“Watching Star Wars and other films, I imagined how robots might be able to see and do stuff in the real world,” said Vuksic, also a member of the NVIDIA Developer Program.

Now, he’s enabling just that with the pit droid project powered by the NVIDIA Jetson platform, which the developer has used since the launch of its first product nearly a decade ago.

Apart from tinkering with computers and bots, Vuksic enjoys playing the bass guitar in a band with his friends.

His Inspiration

Vuksic built the pit droid for both fun and educational purposes.

As a frequent speaker at tech conferences, he takes the pit droid on stage to engage with his audience, demonstrate how it works and inspire others to build something similar, he said.

“We live in a connected world — all the things around us are exchanging data and becoming more and more automated,” he added. “I think this is super exciting, and we’ll likely have even more robots to help humans with tasks.”

Using the NVIDIA Jetson platform, Vuksic is at the forefront of robotics innovation, along with an ecosystem of developers using edge AI.

His Jetson Project

Vuksic’s pit droid project, which took him four months, began with 3D printing its body parts and putting them all together.

He then equipped the bot with the Jetson Orin Nano Developer Kit as the brain in its head, which can move in all directions thanks to two motors.

The Jetson Orin Nano enables real-time processing of the camera feed. “It’s truly, truly amazing to have this processing power in such a small box that fits in the droid’s head,” said Vuksic.

He also uses Microsoft Azure to process the data in the cloud for object-detection training.

“My favorite part of the project was definitely connecting it to the Jetson Orin Nano, which made it easy to run the AI and make the droid move according to what it sees,” said Vuksic, who wrote a step-by-step technical guide to building the bot, so others can try it themselves.

“The most challenging part was traveling with the droid — there was a bit of explanation necessary when I was passing security and opened my bag which contained the robot in parts,” the developer mused. “I said, ‘This is just my big toy!’”

Learn more about the NVIDIA Jetson platform.

To grow and succeed, organizations must continuously focus on technical skills development, especially in rapidly advancing areas of technology, such as generative AI and the creation of 3D virtual worlds.  

NVIDIA Training, which equips teams with skills for the age of AI, high performance computing and industrial digitalization, has released new courses that cover these technologies. The program has already equipped hundreds of thousands of students, developers, researchers and data scientists with critical technical skills.  

With its latest courses, NVIDIA Training is enabling organizations to fully harness the power of generative AI and virtual worlds, which are transforming the business landscape. 

Get Started Building Generative AI Applications     

Generative AI is revolutionizing the ways organizations work. It enables users to quickly generate new content based on a variety of inputs, including text, images, sounds, animation, 3D models and other data types.  

New NVIDIA Training courses on gen AI include:         

• Generative AI Explained — Generative models are accelerating application development for many use cases, including question-answering, summarization, textual entailment, 2D and 3D image and audio creation. In this two-hour course, Bryan Catanzaro, vice president of applied deep learning research at NVIDIA, provides an overview of gen AI’s major developments, where it stands now and what it could be capable of in the future. He’ll discuss technical details and popular generative AI applications, as well as how businesses can responsibly use the technology. 

• Generative AI With Diffusion Models — Thanks to improvements in computing power and scientific theory, generative AI is more accessible than ever. Get started with gen AI application development with this hands-on course where students will learn how to build a text-to-image generative AI application using the latest techniques. Generate images with diffusion models and refine the output with various optimizations. Build a denoising diffusion model from the U-Net architecture to add context embeddings for greater user control. 

To see a complete list of courses on generative AI and large language models, check out these NVIDIA Training Learning Paths. 

Building Digital 3D Worlds

Advancements in digital world-building are transforming media and entertainment, architecture, engineering, construction and operations, factory planning and avatar creation, among other industries.

Immersive 3D environments elevate user engagement and enable innovative solutions to real-world problems. NVIDIA Omniverse, a platform for connecting and developing 3D tools and applications, lets technical artists, designers and engineers quickly assemble complex and physically accurate simulations and 3D scenes in real time, while seamlessly collaborating with team members.

New NVIDIA Training courses on this topic include:

• Essentials of USD in NVIDIA Omniverse — Universal Scene Description, or OpenUSD, is transforming 3D workflows across industries. It’s an open standard enabling 3D artists and developers to connect, compose and simulate in the metaverse. Students will learn what makes OpenUSD unique for designing 3D worlds. The training covers data modeling using primitive nodes, attributes and relationships, as well as custom schemas and composition for scene assembly and collaboration. 

• Developing Omniverse Kit Applications — Learn how to use the NVIDIA Omniverse Kit development framework to build applications, custom extensions and microservices. Applications may comprise many extensions working in concert to address specific 3D workflows, like industrial digitalization and factory planning. Students will use Omniverse reference applications, like Omniverse USD Composer and USD Presenter, to kickstart their own application development.
   
• Bootstrapping Computer Vision Models With Synthetic Data — Learn how to use NVIDIA Omniverse Replicator, a core Omniverse extension, to accelerate the development of computer vision models. Generate accurate, photorealistic, physics-conforming synthetic data to ease the expensive, time-consuming task of labeling real-world data. Omniverse Replicator accelerates AI development at scale and reduces time to production. 

To see a complete list of courses on graphics and simulation, check out these NVIDIA Training Learning Paths.  

Wide Portfolio of Courses 

NVIDIA Training offers courses and resources to help individuals and organizations develop expertise in using NVIDIA technologies to fuel innovation. In addition to those above, a wide range of courses and workshops covering AI, deep learning, accelerated computing, data science, networking and infrastructure are available to explore in the training catalog. 

At the SIGGRAPH conference session “Reimagine Your Curriculum With OpenUSD and NVIDIA Omniverse,” Laura Scholl, senior content developer on the Omniverse team at NVIDIA, will discuss how to incorporate OpenUSD and Omniverse into an educational setting using teaching kits, programs for educators and other resources available from NVIDIA.  

Learn about the latest advances in generative AI, graphics and more by joining NVIDIA at SIGGRAPH. NVIDIA founder and CEO Jensen Huang will deliver a keynote address on Tuesday, Aug. 8, at 8 a.m. PT.

The Ultimate upgrade is complete — GeForce NOW Ultimate performance is now streaming all throughout North America and Europe, delivering RTX 4080-class power for gamers across these regions. Celebrate this month with 41 new games, on top of the full release of Baldur’s Gate 3 and the first Bethesda titles coming to the cloud as the NVIDIA and Microsoft partnership benefits gamers everywhere.

And catch GeForce NOW at QuakeCon — the popular bring-your-own-PC mega-event running Aug. 10-13 — where the in-person and digital GeForce NOW Ultimate challenge will kick off.

Plus, game on with gaming peripherals and accessories company SteelSeries, which will be giving away codes for three-day GeForce NOW Ultimate and Priority memberships, along with popular GeForce NOW games and in-game goodies.

The Ultimate Rollout

The rollout of GeForce RTX 4080 SuperPODs across the world this year lit up cities with cutting-edge performance from the cloud. RTX 3080 members were introduced to the Ultimate membership, featuring gaming at 4K resolution 120 frames per second, or even up to 240 fps with ultra-low latency thanks to NVIDIA Reflex technology.

Ultimate memberships also bring the benefits of the NVIDIA Ada Lovelace architecture — including DLSS 3 with frame generation for the highest frame rates and visual fidelity, and full ray tracing for the most immersive, cinematic, in-game lighting experiences. Plus, ultrawide resolutions were supported for the first time ever from the cloud.

And members can experience it all without having to upgrade a single piece of hardware. With RTX 4080-class servers fully deployed, gamers can now experience ultra-high fps streaming from GeForce RTX 4080-class power in the cloud and see how an Ultimate membership raises the bar on cloud gaming.

To celebrate, the GeForce NOW team will be showing off Ultimate at QuakeCon with a special GeForce NOW Ultimate challenge. Members can register now to be first in line to get a free one-day upgrade to an Ultimate membership and see how their skills improve with 240 fps gaming when the challenge launches next week. Top scorers at QuakeCon can win various prizes, along with those participating in the challenge from home. Keep an eye out on GeForce NOW’s Twitter and Facebook accounts for more details.

It’s Party Time

The best thing to pair with an Ultimate membership are the best games in the cloud. Members have been enjoying early access to Baldur’s Gate 3 from Larian Studios, the role-playing game set in the world of Dungeons and Dragons that raised the bar for the RPG genre.

Now, the full PC game launches and is streamable from GeForce NOW today. Choose from a wide selection of D&D races and classes, or play as an origin character with a handcrafted background. Adventure, loot, battle and romance while journeying through the Forgotten Realms and beyond. The game features a turn-based combat system, a dialogue system with choices and consequences, and a rich story that adapts to player actions and decisions.

Stream it across devices, whether solo or with others in online co-op mode. Those playing from the cloud will be able to enjoy it without worrying about download times or system requirements.

The Ultimate Shooters

Several titles from Bethesda’s well-known franchises — DOOM, Quake and Wolfenstein — will join the cloud this month for a mix of modern and classic first-person shooter games to enjoy across nearly all devices.

Feel the heat with the DOOM franchise, recognizable through its fast-paced epic gameplay and iconic heavy-metal soundtrack. Players take on the role of the DOOM Slayer to fight hordes of invading demons.

In addition, the Quake series features single- and multiplayer campaigns with gritty gameplay and epic music scores in which members can enjoy two sides of the legendary series.

The modern Wolfenstein games feature intense first-person combat against oversized Nazi robots, hulking super soldiers and elite shock troops. Discover an unfamiliar world ruled by a familiar enemy — one that’s changed and twisted history as you know it.

Experience all of these iconic franchises with an Ultimate or Priority membership. Priority members get faster access to GeForce RTX servers in the cloud over free members, along with up to six-hour gaming sessions. Ultimate members can raze their enemies in epic 4K and ultrawide resolution, with up to eight-hour gaming sessions.

Ready, Set, Play!

GeForce NOW and SteelSeries are rewarding gamers ‌throughout‌ August as part of the SteelSeries’ Game On sweepstakes.

Each week, gamers will have a chance to win three-day GeForce NOW Ultimate and Priority codes bundled with popular titles supported in the cloud — RuneScape, Genshin Impact, Brawlhalla and Dying Light 2 — as well in-game goodies.

Check GFN Thursday each week to see what the reward drop will be and head over to the SteelSeries Games site for more details on how to enter. Plus, save 20% with code “NVIDIAGAMEON” this month for premium SteelSeries products, which are perfect to pair with GeForce NOW cloud gaming.

Members can look forward to the 10 new games joining this week:

• F1 Manager 2023 (New release on Steam, July 31)
• Bloons TD 6 (Free on Epic Games Store, Aug. 3)
• Bloons TD Battles 2 (Steam)
• Brick Rigs (Steam)
• Demonologist (Steam)
• Empires of the Undergrowth (Steam)
• Stardeus (Steam)
• The Talos Principle (Steam)
• Teenage Mutant Ninja Turtles: Shredder’s Revenge (Steam)
• Yet Another Zombie Survivors (Steam)

And here’s what the rest of August looks like:

• WrestleQuest (New release on Steam, Aug. 7)
• I Am Future (New release on Steam, Aug. 8)
• Atlas Fallen (New release on Steam, Aug. 10)
• Sengoku Dynasty (New release on Steam, Aug. 10)
• Tales & Tactics (New release on Steam, Aug. 10)
• Moving Out 2 (New release on Steam, Aug. 15)
• Hammerwatch II (New release on Steam, Aug. 15)
• Desynced (New release on Steam, Aug. 15)
• Wayfinder (New release on Steam, Aug. 15)
• The Cosmic Wheel Sisterhood (New release on Steam, Aug. 16)
• Gord (New release on Steam, Aug. 17)
• Book of Hours (New release on Steam, Aug. 17)
• Shadow Gambit: The Cursed Crew (New release on Steam, Aug. 17)
• The Texas Chain Saw Massacre (New release on Steam, Aug. 18)
• Bomb Rush Cyberfunk (New release on Steam, Aug. 18)
• Jumplight Odyssey (New release on Steam, Aug. 21)
• Blasphemous 2 (New release on Steam, Aug. 24)
• RIDE 5 (New release on Steam, Aug. 24)
• Sea of Stars (New release on Steam, Aug. 29)
• Trine 5: A Clockwork Conspiracy (New release on Steam, Aug. 31)
• Deceit 2 (New release on Steam, Aug. 31)
• Inkbound (Steam)
• LEGO Brawls (Epic Games Store)
• Regiments (Steam)
• Session (Epic Games Store)
• Smalland: Survive the Wilds (Epic Games Store)
• Superhot (Epic Games Store)
• Terra Invicta (Epic Games Store)
• Wall World (Steam)
• Wild West Dynasty (Epic Games Store)
• WRECKFEST (Epic Games Store)
• Xenonauts 2 (Epic Games Store)

A Jammin’ July

On top of the 14 games announced in July, four extra joined the cloud last month:

• Let’s School (New release on Steam, July 26)
• Grand Emprise: Time Travel Survival (New release on Steam, July 27)
• Dragon’s Dogma: Dark Arisen (Steam)
• OCTOPATH TRAVELER (Epic Games Store)

What are you looking forward to streaming this month? Let us know your answer on Twitter or in the comments below.

August is here and we can’t wait for you to see what surprises we have in store for this month.

— NVIDIA GeForce NOW (@NVIDIAGFN) August 2, 2023