Powerful AI technologies are making a massive impact in 3D content creation and game development. Whether creating realistic characters that show emotion or turning simple texts into imagery, AI tools are becoming fundamental to developer workflows — and this is just the start.
At NVIDIA GTC and the Game Developers Conference (GDC), learn how the NVIDIA Omniverse platform for creating and operating metaverse applications is expanding with new Connectors and generative AI services for game developers.
Part of the excitement around generative AI is because of its ability to capture the creator’s intent. The technology learns the underlying patterns and structures of data, and uses that to generate new content, such as images, audio, code, text, 3D models and more.
Announced today, the NVIDIA AI Foundations cloud services enable users to build, refine and operate custom large language models (LLMs) and generative AI trained with their proprietary data for their domain-specific tasks.
And through NVIDIA Omniverse, developers can get their first taste of using generative AI technology to enhance game creation and accelerate development pipelines with the Omniverse Audio2Face app.
Accelerating 3D Content With Generative AI
Specialized generative AI tools can boost creator productivity, even for users who don’t have extensive technical skills. Anyone can use generative AI to bring their creative ideas to life, producing high-quality, highly iterative experiences — all in a fraction of the time and cost of traditional game development.
For example, NVIDIA Omniverse Avatar Cloud Engine (ACE) offers the fastest, most versatile solution for bringing interactive avatars to life at scale. Game developers could leverage ACE to seamlessly integrate NVIDIA AI into their applications, including NVIDIA Riva for creating expressive character voices using speech and translation AI, or Omniverse Audio2Face and Live Portrait for AI-powered 2D and 3D character animation.
Today, game developers are already taking advantage of Audio2Face, where artists are more efficiently animating secondary characters without a tedious manual process. The app’s latest release brings major quality, usability and performance updates, including headless mode and a REST API — enabling developers to run the app and process numerous audio files from multiple users in the data center.
Mandarin Chinese language support can now be previewed in Audio2Face, along with improved lip-sync quality, more robust multi-language support and a new pretrained female model. The world’s first fully real-time, ray-traced subsurface scattering shader is also demonstrated in the demo with Diana, a new digital human model.
GSC Game World, one of Europe’s leading game developers, is adopting Omniverse Audio2Face in its upcoming game, S.T.A.L.K.E.R. 2 Head of Chernobyl. Join the NVIDIA and GCS session at GDC to learn how developers are implementing generative AI technology in Omniverse.
Fallen Leaf, an indie game developer, is also using Omniverse Audio2Face for character facial animation in Fort Solis, a third-person sci-fi thriller game that takes place on Mars.
New generative AI services such as NVIDIA Picasso, announced at GTC, preview the future of building and deploying assets for game production pipelines. Omniverse is opening portals to enrich workflows with generative AI tools powered by NVIDIA and its partners, and the momentum around unifying the game asset pipeline is growing.
Unifying Game Asset Pipelines With Universal Scene Description
Based on the Universal Scene Description (USD) framework, NVIDIA Omniverse is the connecting fabric that helps creators and developers build interoperability between their favorite tools — like Autodesk Maya, Autodesk 3ds Max and Adobe Substance 3D Painter — or make their own custom applications.
And with USD — an open, extensible framework and ecosystem for composing, simulating and collaborating within 3D worlds — developers can achieve non-destructive, collaborative workflows when creating scenes, as well as simplify asset aggregation so content creation teams can iterate faster.
Tencent Games is adopting USD workflows based on Omniverse to better streamline content creation pipelines. To create vast worlds in every level of a game, the artists at Tencent use design tools such as Autodesk Maya, SideFX Houdini and Unreal Engine to produce up to millions of trees, buildings and other properties to enrich their scenes. The technical artists often look to optimize their content creation pipelines to speed up this process, so they developed a proprietary Unreal Engine workflow powered by OmniObjects.
With USD, Tencent Games’ teams saw the opportunity to easily streamline and seamlessly connect their workflows. Building on Omniverse as the platform for developing USD workflows, the artists at Tencent no longer need to install plug-ins for each software they use. Using just one USD plug-in enables interoperability across all their favorite software tools. Learn more about Tencent Games by joining this session at GDC.
New and updated Omniverse Connectors for game engines are also now available.
The open-beta Omniverse Connector for Unity workflows helps users of Omniverse and Unity collaborate on projects. Developed by NVIDIA, the Connector delivers USD support alongside Unity workflows, enabling Unity users to take advantage of interoperable workflows. It offers Omniverse Nucleus connection and browsing, USD geometry export, lights, cameras, Material Definition Language and preview for USD materials. Early features also include physics export, USD import and unidirectional live sync.
And with the Unreal Engine Connector’s latest release, Omniverse users can now use Unreal Engine’s USD import utilities to add skeletal mesh blend shape importing, and Python USD bindings to access stages on Omniverse Nucleus. The latest release also delivers improvements in import, export and live workflows, as well as updated software development kits.
Learn more about these latest technologies by joining NVIDIA at GDC.
And catch up on all the groundbreaking announcements in generative AI and the metaverse by watching the NVIDIA GTC keynote.
Follow NVIDIA Omniverse on Instagram, Medium, Twitter and YouTube for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.