Posts in category: nVidia AI

The latest release of NVIDIA Omniverse Enterprise, available now, brings increased performance, generational leaps in real-time RTX ray and path tracing, and streamlined workflows to help teams build connected 3D pipelines, and develop and operate large-scale, physically accurate, virtual 3D worlds like never before.

Artists, designers, engineers and developers can benefit from various enhancements across common Omniverse use cases, including breaking down 3D data silos through aggregation, building custom 3D pipeline tools and generating synthetic 3D data.

The release includes support for the breakthrough innovations within the new NVIDIA Ada Lovelace architecture, including third-generation RTX technology and DLSS 3, delivering up to 3x performance gains when powered by the latest GPU technology, such as NVIDIA RTX 6000 Ada Generation, NVIDIA L40 and OVX systems.

The update also delivers features and capabilities such as new Omniverse Connectors, layer-based live workflows, improved user experience and customization options, including multi-view ports, editable hotkeys, lighting presets and more.

Enhancing 3D Creation

The Omniverse ecosystem is expanding, with more capabilities and tools that allow teams to elevate 3D workflows and reach new levels of real-time, physically accurate simulations. The latest additions include:

• New Connectors: Omniverse Connectors enable more seamless connected workflows between disparate 3D applications. New Adobe Substance 3D Painter, Autodesk Alias, PTC Creo, Kitware’s Paraview Omniverse Connectors are now supported on Omniverse Enterprise. Users can also easily export data created in NX Software from Siemens Digital Industries Software.
• Omniverse DeepSearch: Now generally available, this AI-powered service lets teams intuitively search through extremely large, untagged 3D databases with natural language or using 2D reference images. This unlocks major value in previously unwieldy digital backlots and asset collections that lacked the ability to search.
• Omniverse Farm: A completely renewed user interface provides improved usability and performance, plus Kubernetes support.
• Omniverse Cloud: New cloud containers for Enterprise Nucleus Server, Replicator, Farm and Isaac Sim for AWS provide enterprises more flexibility in connecting and managing distributed teams all over the world. AWS’ security, identity and access-management controls allow teams to maintain complete control over their data. Containers are now available on NVIDIA NGC.

Strengthening Core Components for Building 3D Worlds

Omniverse Enterprise is designed for maximum flexibility and scalability. This means creators, designers, researchers and engineers can quickly connect tools, assets and projects to collaborate in a shared virtual space.

Omniverse Enterprise brings updates to the core components of the platform, including:

• Omniverse Kit SDK, the powerful toolkit for building extensions, apps, microservices or plug-ins, now makes it easier than ever to build advanced tools and Omniverse applications with new templates and developer workflows.
• Omniverse Create, a reference app for composing large-scale, USD-based worlds, now includes NVIDIA DLSS 3 and multi-viewport support, making it easier for Omniverse Enterprise users to fluidly interact with extremely large and complex scenes.
• Omniverse View, a reference app for reviewing 3D scenes, has been streamlined to focus purely on the review and approval experience. New collaborative, real-time, interactive capabilities — including markup, annotation, measure and simple navigation — make stakeholder presentations easier and more interactive than ever.
• Omniverse Nucleus, the database and collaboration engine, now includes improved IT management tools, such as expanded version control to handle atomic checkpoints on the server. Updated Large File Transfer service enables users to move files between servers, on premises or in the cloud to benefit hybrid workflows. And new self-service deployment instructions for Enterprise Nucleus Server on AWS are now available, letting customers deploy and manage Nucleus in the cloud.

Customers Dive Into Omniverse Enterprise 

Many customers around the world have experienced enhanced 3D workflows with Omniverse Enterprise.

Dentsu International, one of the largest global marketing and advertising agency networks, always looks for solutions that enable collaborative and seamless work, with a central repository for completed projects.

“There’s great value when multiple artists with different tools, located in different places, are able to work together, and NVIDIA Omniverse Enterprise makes that possible,” said Melody Jeter, vice president of Creative Technology at Dentsu. “The collaboration tackles the robust challenge of laborious file transferring — now, the workflows are all synced live and in real time.”

In addition to enhancing current pipelines with Omniverse Enterprise, Dentsu is looking to incorporate NVIDIA generative AI into its 3D design pipeline with software development kits like Omniverse ACE and Audio2Face.

Mercedes Benz, the German premium vehicle manufacturer, is using Omniverse Enterprise at its sites world wide to design, plan and optimize its manufacturing and assembly facilities. By developing full-fidelity digital twins of their production environments, globally dispersed teams will open up new abilities to collaborate in real time, accelerate decision-making and identify opportunities to reduce waste, decrease energy consumption and continuously enhance quality.

Zaha Hadid Architects (ZHA) is a renowned architectural design firm that has created some of the world’s most singular building designs. ZHA focuses on creating transformative cultural, corporate and residential spaces through cutting-edge technologies. With Omniverse Enterprise, the team can accelerate and automate its workflows, as well as develop custom tools within the platform.

“We are working with NVIDIA to incorporate Omniverse as the connective infrastructure of our tech stack. Our goal is to retain design intent across the various project stages and improve productivity,” said Shajay Bhooshan, associate director at Zaha Hadid Architects. “We expect NVIDIA Omniverse to play a critical, supportive role to our efforts to create a platform that’s agnostic, version-controlled and a single source of truth for design data, as it evolves from idea to delivery.”

NVIDIA Omniverse Enterprise is available by subscription from BOXX Technologies, Dell Technologies, Z by HP and Lenovo, and channel partners including Arrow, ASK, PNY and Leadtek. The platform is optimized to run on NVIDIA-Certified, RTX-enabled desktop and mobile workstations, as well as servers, with new support for NVIDIA RTX Ada generation systems.

Watch the NVIDIA special address at CES on demand. Learn more about NVIDIA Omniverse Enterprise and try Omniverse for free.

AI is extending further into the vehicle as autonomous-driving technology becomes more prevalent.

With the NVIDIA DRIVE platform, automakers can design and implement intelligent interior features to continuously surprise and delight customers.

It all begins with the compute architecture. The recently introduced NVIDIA DRIVE Thor platform unifies traditionally distributed functions in vehicles  — including digital cluster, infotainment, parking and assisted driving — for greater efficiency in development and faster software iteration.

NVIDIA DRIVE Concierge, built on the DRIVE IX software stack, runs an array of safety and convenience features, including driver and occupant monitoring, digital assistants and autonomous-vehicle visualization.

Automakers can benefit from NVIDIA data center solutions even if they aren’t using the NVIDIA DRIVE platform. With cloud technology, vehicles can stream the NVIDIA GeForce NOW cloud-gaming service without any special equipment. Plus, developers can train, test and validate in-vehicle AI models on NVIDIA DGX servers.

The same data center technology that’s accelerating AI development — in combination with the NVIDIA Omniverse platform for creating and operating metaverse applications — is also revolutionizing the automotive product cycle. Using NVIDIA DRIVE Sim built on Omniverse, automakers can design vehicle interiors and retail experiences entirely in the virtual world.

Easing Pain Points From Concept to Customer

Designing and selling vehicles requires the highest levels of organization and orchestration. The cockpit alone has dozens of components — such as steering wheel, cluster and infotainment — that developers must create and integrate with the rest of the car.

These processes are incredibly time- and resource-intensive — there are countless configurations, and chosen designs must be built out and tested prior to production. Vehicle designers must collaborate on various layouts, which must then be validated and approved. Customers must travel to dealerships to experience various options, and the ability to test features depends on a store’s inventory at any given time.

In the virtual world, developers can easily design vehicles, and car buyers can seamlessly test them, leading to an optimal experience on both ends of the production pipeline.

Design and Collaboration

Automakers operate design centers around the world, tapping into expertise from North America, Europe, Asia and other automotive hubs. Working on user experience concepts across these locations requires frequent international travel and close coordination.

With DRIVE Sim, designers and engineers anywhere in the world can work together to develop the cockpit experience, without having to leave their desks.

Design teams can also save time and valuable resources by testing concepts in the virtual world, without having to wait for physical prototypes. Decision-makers can review designs and ensure they meet relevant safety standards in DRIVE Sim before sending them to production.

Transforming the Customer Experience

The benefits of in-vehicle simulation extend far beyond the design phase.

Consumers are increasingly expecting full-service digital retail experiences. More than 60% of shoppers want to conduct more of the car-buying process online compared to the last time they bought a vehicle, while more than 75% are open to buying a car entirely online, according to an Autotrader survey.

The same tools used to design the vehicle can help meet these rising consumer expectations.

With DRIVE Sim, car buyers can configure and test the car from the comfort of their homes. Customers can see all potential options and combinations of vehicle features at the push of a button and take their dream car for a virtual spin — no lengthy trips to the dealership required.

From concept design to customer experience, DRIVE Sim is easing the process and opening up new ways to design and enjoy intelligent vehicles.

Building state-of-the-art factories requires a state-of-the art planning system.

Mercedes-Benz announced at CES that it is taking the next step in digitizing its production process, using the NVIDIA Omniverse platform to design and plan manufacturing and assembly facilities.

By tapping into NVIDIA AI and metaverse technologies, the automaker can create feedback loops to reduce waste, decrease energy consumption and continuously enhance quality.

Mercedes-Benz has been working with NVIDIA to develop software-defined vehicles. Its upcoming fleets will be built on NVIDIA DRIVE Orin centralized compute, with intelligent driving capabilities tested and validated in the NVIDIA DRIVE Sim platform, built on Omniverse.

The automaker’s latest announcement infuses AI and metaverse technologies even further into the Mercedes-Benz development process with smarter, more efficient manufacturing.

Electric-First Platform Built on Digital-First Process

Vehicle production is a colossal undertaking that requires thousands of parts and workers moving in harmony. Any supply chain or production issues can lead to costly delays.

On top of that, when automakers roll out a new model, they must reconfigure the layout of production plants to account for the new vehicle design. This process can take significant portions of the factory offline, pausing manufacturing for existing vehicles.

Mercedes-Benz plans to start production of its new dedicated platform for electric vehicles at its plant in Rastatt, Germany. The site currently manufactures the automaker’s A- and B-Class as well as the compact SUV GLA and the all-electric Mercedes-Benz EQA.

Experts from NVIDIA and Mercedes-Benz operations are setting up a “digital first” – planning process for the plant that won’t disrupt the current production of compact car models at the site. This blueprint will be rolled out to other parts of the global Mercedes-Benz production network for more agile vehicle manufacturing.

Developing With Digital Twins

NVIDIA Omniverse is an open 3D development platform enabling enterprises and institutions across all industries to build and operate digital twins for industrial and scientific use cases. It’s based on Universal Scene Description (USD), allowing enterprises and developers to construct custom 3D pipelines to break down data silos and interact with a single-source-of-truth view of their aggregated simulations.

With Omniverse, Mercedes-Benz planners can access the digital twin of the factory, reviewing and optimizing the plant as needed. Every change can be quickly evaluated and validated in the virtual world, then implemented in the real world to ensure maximum efficiency and ergonomics for factory workers.

Additionally, Mercedes-Benz can synchronize plant locations anywhere in the world by connecting Omniverse with its in-house MO360 Data Platform.

This capability streamlines operations across the global production network and enables over-the-air software updates to manufacturing equipment.

By planning production on NVIDIA Omniverse, Mercedes-Benz is implementing a manufacturing system as intelligent as its vehicles.

Autonomous and electric vehicles are making personal transportation safer and more sustainable — as well as more entertaining.

At CES today, NVIDIA announced that the NVIDIA GeForce NOW cloud gaming service will be coming to cars, with no special equipment needed. Hyundai Motor Group, BYD and Polestar — already members of the NVIDIA DRIVE ecosystem — are the first automakers working with NVIDIA to deliver GeForce NOW in their vehicles.

“We are excited to offer our customers the best in technology, comfort, design from BYD and now the latest in gaming through NVIDIA GeForce NOW high-performance cloud gaming service,” said Stella Li Vice President of BYD and CEO of BYD America. “The experience of driving our cars is becoming increasingly sophisticated, entertaining and sustainable.”

Thanks to cloud technology and the mobile internet, vehicle occupants can keep boredom at bay with the ability to enjoy music, movies and now video games in the car. Drivers and front seat passengers can access games while parked or charging, while those in the backseat of vehicles where rear entertainment is enabled can play anytime on the road.

Hyundai Motor Group, which houses the Hyundai, Kia and Genesis brands, features NVIDIA DRIVE in-vehicle infotainment across its entire lineup.

In March, BYD, the world’s leading manufacturer of new energy vehicles, announced it would build its NEVs on the NVIDIA DRIVE Hyperion platform, starting in the first half of 2023.

Polestar is also using NVIDIA DRIVE for its software-defined architecture, with the upcoming Polestar 3 powered by the NVIDIA DRIVE Orin system-on-a-chip.

Now, these trailblazing automakers will offer vehicles that are as entertaining as they are intelligent with the addition of GeForce NOW.

1,000+ Games, Anywhere, Anytime

The next generation of vehicles are transforming from fixed function to software-defined. At the same time, cars are becoming more connected, opening up new services that weren’t possible before.

GeForce NOW delivers a full PC-gaming experience to nearly any device, including laptops, mobile devices, smart TVs — and now, personal vehicles. The GeForce NOW client for auto can run on Android- or browser-based in-vehicle infotainment systems, providing an app to easily select and launch favorite games.

It uses breakthroughs in low-latency cloud-streaming technology for real-time game play powered by GeForce servers in the cloud. GeForce NOW features more than 1,500 titles — over 1,000 playable with a controller — including top-rated games such as A Plague Tale: Requiem, The Witcher 3: Wild Hunt and Cyberpunk 2077, from leading stores such as Steam, Epic Games Store, the EA app, Ubisoft and GOG.com.

The service also includes many of the most-played free-to-play games like Fortnite, Lost Ark and Destiny 2.

Experiencing the Future

The ability to stream games to the car is a key component of the future of in-vehicle experiences.

As autonomous-driving technology becomes more prevalent, the interior will play a much larger role in differentiating brand identities. Consumers will focus more on features such as digital assistants, beautiful graphic displays and streaming content as they make vehicle buying decisions.

This shift is turning vehicles into living spaces, where passengers can enjoy what’s important to them — whether it be work or play — rather than the stress of driving.

By providing GeForce NOW in their vehicles, Hyundai Motor Group, BYD and Polestar are taking a significant step into this new era of personal transportation.

The future of content creation was on full display today during NVIDIA’s virtual special address at CES.

Fueled by powerful NVIDIA RTX technology and backed by the NVIDIA Studio platform for creators, a creative revolution is underway as a wave of 2D artists moves to 3D, video workflows move to real time and AI tools help artists create content faster.

For them, and the 110 million professional and hobbying PC creators, there’s NVIDIA Studio. It’s a platform for content creators that supercharges 110 creative apps with RTX GPUs, provides lasting stability with NVIDIA Studio Drivers, and includes a powerful suite of Studio apps — Omniverse, Canvas, Broadcast and RTX Remix.

Creators can access all of this with any RTX GPU, including the new 40 Series laptops and GeForce RTX 4070 Ti graphics cards, powered by the ultra-efficient Ada Lovelace GPU architecture, which were introduced today at CES.

The heartbeat of the Studio platform lies in NVIDIA Omniverse, a platform where creators can connect their 3D apps and collaborate in real time. Omniverse is expanding with Blender enhancements, a new suite of experimental generative AI tools for 3D artists and thousands of new, free Universal Scene Description (USD) assets available in the content browser in Omniverse Create.

Built on Omniverse, NVIDIA RTX Remix is a free modding platform to quickly create mods for classic games with full ray tracing, enhanced materials, DLSS 3 and NVIDIA Reflex. We’ve enabled two amazing community modders to bring RTX to Portal’s most famous mod, Portal Prelude, preserving the timeless gameplay, while relighting it with full ray tracing.

The NVIDIA Broadcast and Canvas apps, also in the exclusive Studio software suite, have upcoming upgrades. Broadcast is adding a new Eye Contact feature, and Canvas will introduce 360-degree images that can be used as environment maps in 3D apps.

Coming to GeForce RTX 40 and 30 Series GPUs next month, the RTX Video Super Resolution feature uses AI to improve the quality of any video watched in a browser by removing blocky compression artifacts and upscaling video resolution. This improves video sharpness and clarity, and lets people watch online content in its native resolution on high-resolution displays. RTX Video Super Resolution will be available in February for Chrome and Edge browsers.

And to celebrate CES, influential Blender artist CG Geek will take on a three-day design challenge, powered by NVIDIA Studio technologies. Plus, explore how seven freelance artists from across the world created an experimental short film together, in real time, using Studio hardware, Omniverse and the Nucleus Cloud — this week In the NVIDIA Studio.

From Concept to Completion, Faster With New NVIDIA Studio Laptops

GeForce RTX GPUs provide massive speedups in 3D, video and broadcast workflows, as well as a myriad of AI tools, thanks to dedicated ray tracing, AI and video-encoding hardware. New Studio laptops with GeForce RTX 40 Series Laptop GPUs use the NVIDIA Ada Lovelace architecture — with support for DLSS 3, AV1 and more — and fifth-generation Max-Q technologies for maximum performance and efficiency.

The new laptops provide a quantum leap in performance over the last generation, and a massive boost in efficiency that allows OEMs to include more powerful GPUs in thinner, more portable systems. NVIDIA Studio laptops are purpose-built for creators, with the right combination of hardware for creative workflows, color-accurate displays, and preinstalled Studio Drivers and Studio software.

The instantly recognizable Razer Blade laptop gets an RTX 40 Series upgrade with up to a GeForce RTX 4090 Laptop GPU. The 16-inch, HDR-capable, dual-mode, mini-LED display boasts 1,000 nits peak brightness, which pairs nicely with a Creator mode that enables sharp, ultra-high-definition+ native resolution at 120Hz.

MSI’s Stealth series will launch with a wide range of Studio options. With 14- to 17-inch models and up to GeForce RTX 4090 Laptop GPUs, creators can pick the laptop that best suits their needs.

The increased efficiency of RTX 40 Series Laptop GPUs and fifth-generation Max-Q Technologies deliver high performance to laptops as slim as 14 inches, with up to GeForce RTX 4070 Laptop GPUs, a new milestone. The ASUS Zenbook Pro 14 OLED is one example, with a GeForce RTX 4070 Laptop GPU in a super-thin 14-inch design, and accompanied by an all-new, cutting-edge 2.8K 120Hz OLED NanoEdge Dolby Vision display.

GeForce RTX 4070 Ti: 40 Series Benefits and Faster Than a 3090 Ti

For artists who prefer to create from their desktops, the new GeForce RTX 4070 Ti graphics cards, available starting Thursday, Jan. 5, offers incredible performance that’s up to 15% faster than a GeForce RTX 3090 Ti in 3D rendering. Like the RTX 4090 and 4080, the newest RTX 40 Series GPU features dual AV1 video encoders. When enabled in top video-editing and livestreaming apps — such as Adobe Premiere Pro (via the Voukoder plug-in), DaVinci Resolve, OBS and Jianying — export times are cut in half with improved video quality.

3D creators see up to a 70% increase in performance compared to the GeForce RTX 3070 Ti in popular apps like Autodesk Maya, Blender, Chaos V-Ray, Epic Games Unreal Engine and Unity. Users can also unlock creative freedom by unifying 3D assets, libraries and tools in Omniverse.

And all creators can benefit from the new fourth-generation Tensor Cores for AI tools, providing up to 2x increased performance compared to the previous generation.

The January Studio Driver, supporting the GeForce RTX 4070 Ti launch, will be available for download later this week.

Create Easily With Studio Software

NVIDIA RTX GPUs unlock exclusive software for creators: Omniverse, RTX Remix, Canvas and Broadcast. They’re all getting upgrades early this year.

Based on the USD framework, Omniverse enables artists to connect their favorite tools from Adobe, Autodesk, Epic Games, SideFX and more. Creators can see their scenes instantly come together without lengthy import or export cycles. Changes happen in real time across the connected apps — whether a single artist is working in multiple apps simultaneously or collaborating with another artist from across the globe.

The Blender alpha branch release, part of a series of updates to Omniverse, is now available in the Omniverse Launcher. It enables users to repair geometry, generate automatic UVs and decimate high-resolution CAD data to more usable polycounts.

Performance updates to Audio2Face, Audio2Gesture and Audio2Emotion — powerful AI tools within Omniverse — better enable instant, realistic animation of characters. The AI ToyBox of experimental tools is now available in the Omniverse Extension Manager. Thousands of new, free 3D assets have also been made available worldwide for users to build and create within Omniverse.

RTX Remix, which is built on Omniverse and was used to create the jaw-dropping Portal with RTX, is approaching its early-access release. Nicolas “NykO18” Grevet — the original creator of Portal: Prelude — is using RTX Remix to remaster his unofficial Portal prequel from 2008. Grevet is working with modder David “Kralich” Driver-Gomm to modernize the assets and relight the game with stunning, full ray tracing. Portal: Prelude RTX will soon become free to download from mod sites like ModDB and Nexus Mods.

Canvas allows creators to paint by material, rather than color, using simple brushstrokes and AI to quickly conceptualize a beautiful image. Canvas is getting a new 360 image feature, which will become available in an upcoming free update for RTX users, helping artists create panoramic scenes and export them into any 3D app to use as an environment map. Such maps are used to change the ambient lighting of a 3D scene and appear in reflections for added realism.

Later this month, Broadcast will add a new Eye Contact feature that can change the position of the subject’s eyes to appear focused on the camera, as well as a new vignette effect. The update also improves the popular virtual background feature, adding temporal information for increased stability. The virtual background improvements can also be found in the recent OBS Studio 29.0 software release, and will be updated SDKs for developers later this week.

3D Need for Speed

To showcase the speed at which Studio creators flow, talented 3D artist CG Geek has been challenged to create an animated Blender scene during CES. He’ll have just three days, starting Thursday, Jan. 5, to blitz through tough, typically time-consuming tasks.

He’s equipped with a GeForce RTX 4090 GPU, RTX acceleration and AI features throughout his entire workflow.

Follow his progress on his YouTube channel or NVIDIA Studio profiles on Instagram and Twitter. Look for the final render and a deep dive on his process in next week’s In the NVIDIA Studio blog.

In the NVIDIA Studio With Omniverse

Several 3D creators, including In the NVIDIA Studio artist Jae Solina, are showcasing the collaborative elements of Omniverse. Ashley Goldstein, Edward McEvenue, Jeremy Lightcap, Pekka Varis, Rafi Nizam and Shangyu Wang joined Solina — each using their favorite 3D tools, NVIDIA Studio laptops, new GeForce RTX GPUs and Omniverse Nucleus Cloud — to build an experimental short film together in real time.

The group started by pulling in a pair of Nizam’s characters: “3D” built in ShapesXR and “Figgy” in Maya. Both were dropped into Omniverse seamlessly as USD files. Prompted by Lightcap, Nizam added knitted textures to “3D” using Adobe Substance 3D Painter, and the group immediately saw this through the cloud.

Solina then offered to help with motion capture using Xsens. As he added the rigging, Lightcap brought in a background created in Blender. Lightcap noted that “RTX is insane,” as his animated western landscape rendered nearly instantly.

With the scene coming together, Varis changed lighting elements, looking to make things “more epic,” he said. To incorporate a storm into the scene, McEvenue added a rain simulation created in Houdini. The film’s characters make a quick escape, hijacking the hot air balloon of Toy Jensen, the AI avatar of NVIDIA founder and CEO Jensen Huang.

For the ascent, Goldstein used her GeForce RTX 4090 GPU to add steam thrusters using OmniGraph.

With the characters now in outer space, Wang imported a satellite created in Autodesk Maya. The scene concludes as the space adventurers take off for Jupiter.

While showcasing their talents, these 3D artists displayed the capabilities of the NVIDIA Studio platform. Powerful RTX GPUs, acceleration in popular 3D apps and exclusive Omniverse software that enabled collaboration from all over the world in real time.

The future of creativity is shining bright. And the NVIDIA Studio ecosystem is lighting the scene.

Download monthly NVIDIA Studio Drivers for the latest reliability and performance improvements. Access tutorials and more on the Studio YouTube channel; follow Studio on Instagram, Twitter and Facebook; and get updates directly in your inbox by subscribing to the Studio newsletter.

Demand for intelligent robots is growing as more industries embrace automation to address supply chain challenges and labor force shortages.

The installed base of industrial and commercial robots will grow more than 6.4x — from 3.1 million in 2020 to 20 million in 2030, according to ABI Research. Developing, validating and deploying these new AI-based robots requires simulation technology that places them in realistic scenarios.

At CES, NVIDIA announced major updates to Isaac Sim, its robotics simulation tool to build and test virtual robots in realistic environments across varied operating conditions. Now accessible from the cloud, Isaac Sim is built on NVIDIA Omniverse, a platform for creating and operating metaverse applications.

Powerful AI-Driven Capabilities for Roboticists 

With humans increasingly working side by side with collaborative robots (cobots) or autonomous mobile robots (AMRs), it’s critical that people and their common behaviors are added to simulations.

Isaac Sim’s new people simulation capability allows human characters to be added to a warehouse or manufacturing facility and tasked with executing familiar behaviors— like stacking packages or pushing carts. Many of the most common behaviors are already supported, so simulating them is as simple as issuing a command.

To minimize the difference between results observed in a simulated world versus those seen in the real world, it’s imperative to have physically accurate sensor models.

Using NVIDIA RTX technology, Isaac Sim can now render physically accurate data from sensors in real time. In the case of an RTX-simulated lidar, ray tracing provides more accurate sensor data under various lighting conditions or in response to reflective materials.

Isaac Sim also provides numerous new simulation-ready 3D assets, which are critical to building physically accurate simulated environments. Everything from warehouse parts to popular robots come ready to go, so developers and users can quickly start building.

Significant new capabilities for robotics researchers include advances in Isaac Gym for reinforcement learning and Isaac Cortex for collaborative robot programming. Additionally, a new tool, Isaac ORBIT, provides simulation operating environments and benchmarks for robot learning and motion planning.

For the large community of Robot Operating System (ROS) developers, Isaac Sim upgrades support for ROS 2 Humble and Windows. All of the Isaac ROS software can now be used in simulation.

Expanding Isaac Platform Capabilities and Ecosystem Drives Adoption 

The large and complex robotics ecosystem spans multiple industries, from logistics and manufacturing to retail, energy, sustainable farming and more.

The end-to-end Isaac robotics platform provides advanced AI and simulation software as well as accelerated compute capabilities to the robotics ecosystem. Over a million developers and more than a thousand companies rely on one or many parts of it. This includes many companies that have deployed physical robots developed and tested in the virtual world using Isaac Sim.

Telexistence has deployed beverage restocking robots across 300 convenience stores in Japan. To improve safety, Deutsche Bahn is training AI models to handle very important but unexpected corner cases that happen rarely in the real world — like luggage falling on a train track. Sarcos Robotics is developing robots to pick and place solar panels in renewable energy installations.

Festo uses Isaac Cortex to simplify programming for cobots and transfer simulated skills to the physical robots. Fraunhofer is developing advanced AMRs using the physically accurate and full-fidelity visualization features of Isaac Sim. Flexiv is using Isaac Replicator for synthetic data generation to train AI models.

While training robots is important, simulation is playing a critical role in training the human operators to work with and program robots. Ready Robotics is teaching programming of industrial robots with Isaac Sim. Universal Robotics is using Isaac Sim for workforce development to train end operators from the cloud.

Cloud Access Puts Isaac Platform Within Reach Everywhere

With Isaac Sim available in the cloud, global, multidisciplinary teams working on robotics projects can collaborate with increased accessibility, agility and scalability for testing and training virtual robots.

A lack of adequate training data often hinders deployment when building new facilities with robotics systems or scaling existing autonomous systems. Isaac Sim taps into Isaac Replicator to enable developers to create massive ground-truth datasets that mimic the physics of real-world environments.

Once deployed, dynamic route planning is required to operate an efficient fleet of hundreds of robots as automation requirements scale. NVIDIA cuOpt, a real-time fleet task-assignment and route-planning engine improves operational efficiencies with automation.

Get Started on Isaac Sim 

Download Isaac Sim today.

Watch NVIDIA’s special address at CES, where its executives unveiled products, partnerships and offerings in autonomous machines, robotics, design, simulation and more.

Whether creating realistic digital humans that can express raw emotion or building immersive virtual worlds, those in the design, engineering, creative and other industries across the globe are reaching new heights through 3D workflows.

Animators, creators and developers can use new AI-powered tools to reimagine 3D environments, simulations and the metaverse — the 3D evolution of the internet.

Based on the Universal Scene Description (USD) framework, the NVIDIA Omniverse platform — which enables the development of metaverse applications — is expanding with Blender enhancements and a new suite of experimental generative AI tools for 3D artists.

In a special address at CES, NVIDIA announced these features, as well as Omniverse preinstallation on NVIDIA Studio laptops and thousands of new, free USD assets to help accelerate adoption of 3D workflows.

_{NVIDIA Studio 3D creators Jeremy Lightcap, Edward McEvenue, Rafi Nizam, Jae Solina, Pekka Varis, Shangyu Wang, Ashley Goldstein collaborate across multiple 3D design tools, time zones and RTX systems with Omniverse.}

Plus, a new release for Blender, now available in the Omniverse Launcher, is bringing 3D generative AI capabilities to Blender users everywhere. A new panel lets Blender users easily transfer shape keys and rigged characters.  The challenge of reattaching a rigged character’s head can now be solved with a one-button operation from Omniverse Audio2Face — an AI-enabled tool that automatically generates realistic facial expressions from an audio file.

Another new panel for scene optimization lets users create USD scenes within their multi-app 3D workflows more easily and in real time.

In addition, Audio2Face, Audio2Gesture and Audio2Emotion — generative AI tools that enable instant 3D character animation — are getting performance updates that make it easier for developers and creators to integrate into their current 3D pipelines.

Creators can generate facial expressions from an audio file using Audio2Face; realistic emotions ranging from happy and excited to sad and regretful with Audio2Emotion; and realistic upper-body movement using Audio2Gesture. These audio-to-animation tools are game-changers for 3D artists, eliminating the need to perform tedious, manual tasks.

AI-assisted creator tools are expanding to even more communities of creative and technical professionals. When NVIDIA Canvas was introduced, it empowered artists to seamlessly generate landscapes and iterate on them with simple brushstrokes and AI. Coming soon, all RTX users will be able to download an update to Canvas that introduces 360 surround images to create and conceptualize panoramic environments and beautiful images. The AI ToyBox, which features extensions derived from NVIDIA Research, enables creators to generate 3D meshes from 2D inputs.

Omniverse’s powerful AI tools simplify complex tasks. Creators of all levels can tap into these resources to produce high-quality outputs that meet the growing demands for content and virtual worlds in the metaverse.

“The demand for 3D skills is skyrocketing, but learning 3D can be pretty scary to some, and definitely time consuming,” said Jae Solina, aka JSFilmz. “But these new platform developments not only let creatives and technical professionals continue to work in their favorite 3D tools, but also supercharge their craft and even use AI to assist them in their workflows.”

Omniverse Launcher, the portal to download Omniverse content and reference applications, has also been made available to system builders so they can preinstall it, enabling optimized, out-of-the-box experiences for 3D creators on NVIDIA Studio-validated laptops. GIGABYTE and AORUS will be the first laptops launching in 2023 with Omniverse Launcher preinstalled, expanding platform access to a growing number of 3D content creators.

NVIDIA RTX Remix is a free modding platform, built on Omniverse, that enables modders to

quickly create and share #RTXON mods for classic games, each with full ray tracing, enhanced materials, NVIDIA DLSS 3 and NVIDIA Reflex. Its release in early access is coming soon. The jaw-dropping Portal with RTX was built with RTX Remix, and to demonstrate how easy it is for modders to turn RTX ON in their mods, we shared RTX Remix with the original creator of Portal: Prelude, an unofficial Portal prequel released in 2008.

Omniverse users can also access thousands of new, free USD assets, including a USD-based NVIDIA RTX Winter World Minecraft experience, and learn to create their own NVIDIA SimReady assets for complex simulation building. Using Omniverse, creators can supercharge their existing workflows using familiar tools such as Autodesk Maya, Autodesk 3ds Max, Blender, Adobe Substance 3D Painter, and more with AI, simulation tools and real-time RTX-accelerated rendering.

All types of 3D creators can take advantage of these new tools to push the boundaries of 3D simulation and virtual world-building. Users can reimagine digital worlds and animate lifelike characters with new depths of creativity through the bridging of audio-to-animation tools, generative AI and the metaverse.

Latest Omniverse Platform Updates

The latest updates within Omniverse include:

• Early access for the Unity Omniverse Connector is now available.
• Blender alpha release, now available in the Omniverse Launcher, enables users to repair geometry, generate automatic UVs and decimate high-resolution CAD data to more usable polycounts.
• Audio2Face, Audio2Emotion and Audio2Gesture updates better enable instant, realistic animation of characters, now available in Omniverse Audio2Face and Omniverse Machinima.
• NVIDIA Canvas is coming soon to the Omniverse Launcher with new capabilities that enable the creation of 360-degree landscapes with simple brushstrokes. Users can import the environments into 3D apps to test different settings and lighting.
• AI ToyBox of experimental tools, built by NVIDIA Research, that include GET3D, an Omniverse extension that generates trainable 3D models from 2D images, letting developers use their own datasets to rapidly create models for 3D virtual worlds, is now available in the Omniverse Extension Manager.
• Thousands of new, free 3D assets are now available worldwide for users to build and create within Omniverse.

Watch the NVIDIA special address at CES on demand.

Creators can download NVIDIA Omniverse for free, submit their work to the NVIDIA Omniverse gallery, and find resources through forums, Medium, Twitter, YouTube, Twitch, Instagram and Discord.

Follow NVIDIA Studio on Instagram, Twitter and Facebook and access tutorials — including on Omniverse — on the Studio YouTube channel. Get the latest Studio updates directly in your inbox by subscribing to the NVIDIA Studio newsletter.

Developers and teams building avatars and virtual assistants can now register to join the early-access program for NVIDIA Omniverse Avatar Cloud Engine (ACE), a suite of cloud-native AI microservices that make it easier to build and deploy intelligent virtual assistants and digital humans at scale.

Omniverse ACE eases avatar development, delivering the AI building blocks necessary to add intelligence and animation to any avatar, built on virtually any engine and deployed on any cloud. These AI assistants can be designed for organizations across industries, enabling organizations to enhance existing workflows and unlock new business opportunities.

ACE is one of several generative AI applications that will help creators accelerate the development of 3D worlds and the metaverse. Members who join the program will receive access to the prerelease versions of NVIDIA’s AI microservices, as well as the tooling and documentation needed to develop cloud-native AI workflows for interactive avatar applications.

Bring Interactive AI Avatars to Life With Omniverse ACE

Methods for developing avatars often require expertise, specialized equipment and manually intensive workflows. To ease avatar creation, Omniverse ACE enables seamless integration of NVIDIA’s AI technologies — including pre-built models, toolsets and domain-specific reference applications — into avatar applications built on most engines and deployed on public or private clouds.

Since it was unveiled in September, Omniverse ACE has been shared with select partners to capture early feedback. Now, NVIDIA is looking for partners who will provide feedback on the microservices, collaborate to improve the product, and push the limits of what’s possible with lifelike, interactive digital humans.

The early-access program includes access to the prerelease versions of ACE animation AI and conversational AI microservices, including:

• 3D animation AI microservice for third-party avatars, which uses Omniverse Audio2Face generative AI to bring to life characters in Unreal Engine and other rendering tools by creating realistic facial animation from just an audio file.
• 2D animation AI microservice, called Live Portrait, enables easy animation of 2D portraits or stylized human faces using live video feeds.
• Text-to-speech microservice uses NVIDIA Riva TTS to synthesize natural-sounding speech from raw transcripts without any additional information, such as patterns or rhythms of speech.

Program members will also get access to tooling, sample reference applications and supporting resources to help get started.

Avatars Make Their Mark Across Industries

Omniverse ACE can help teams build interactive, digital humans that elevate experiences across industries, providing:

• Easy animation of characters, so users can bring them to life with minimal expertise.
• The ability to deploy on cloud, which means avatars will be usable virtually anywhere, such as a quick-service restaurant kiosk, a tablet or a virtual-reality headset.
• A plug-and-play suite, built on NVIDIA Unified Compute Framework (UCF), which enables interoperability between NVIDIA AI and other solutions, ensuring state-of-the-art AI that fits each use case.

Partners such as Ready Player Me and Epic Games have experienced how Omniverse ACE can enhance workflows for AI avatars.

The Omniverse ACE animation AI microservice supports 3D characters from Ready Player Me, a platform for building cross-game avatars.

“Digital avatars are becoming a significant part of our daily lives. People are using avatars in games, virtual events and social apps, and even as a way to enter the metaverse,” said Timmu Tõke, CEO and co-founder of Ready Player Me. “We spent seven years building the perfect avatar system, making it easy for developers to integrate in their apps and games and for users to create one avatar to explore various worlds — with NVIDIA Omniverse ACE, teams can now more easily bring these characters to life.”

Epic Games’ advanced MetaHuman technology transformed the creation of realistic, high-fidelity digital humans. Omniverse ACE, combined with the MetaHuman framework, will make it even easier for users to design and deploy engaging 3D avatars.

Digital humans don’t just have to be conversational. They can be singers, as well — just like the AI avatar Toy Jensen. NVIDIA’s creative team quickly created a holiday performance by TJ, using Omniverse ACE to extract the voice of a singer and turn it into TJ’s voice. This enabled the avatar to sing at the same pitch and with the same rhythm as the original artist.

Many creators are venturing into VTubing, a new way of livestreaming. Users embody a 2D avatar and interact with viewers. With Omniverse ACE, creators can move their avatars into 3D from 2D animation, including photos and stylistic faces. Users can render the avatars from the cloud and animate the characters from anywhere.

Additionally, the NVIDIA Tokkio reference application is expanding, with early partners building cloud-native customer service avatars for industries such as telco, banking and more.

Join the Early-Access Program

Early access to Omniverse ACE is available to developers and teams building avatars and virtual assistants.

Watch the NVIDIA special address at CES on demand. Learn more about NVIDIA Omniverse ACE and register to join the early-access program.

Those looking to join the ranks of AI trailblazers or chart a new course in their careers need look no further.

At NVIDIA’s latest GTC conference, industry leaders in a panel called “5 Paths to a Career in AI” shared tips and insights on how to make a mark in this rapidly evolving field.

Representing diverse sectors such as healthcare, automotive, augmented and virtual reality, climate and energy, and manufacturing, these experts offered valuable advice for all seeking to build a career in AI.

Here are five key takeaways from the discussion:

1. Be curious and constantly learn: “I think in order to break into this field, you’ve got to be curious. It’s so important to always be learning [and] always be asking questions,” emphasized Chelsea Sumner, healthcare AI startups lead for North and Latin America at NVIDIA. “If we’re not asking questions, and we’re not learning, we’re not growing.”
2. Tell your story effectively to different audiences: “Your ability to tell your story to a variety of different audiences is essential,” noted Justin Taylor, vice president of AI at Lockheed Martin. “So for them to understand what you’re doing [with AI], how you’re doing it, why you’re doing it is essential.”
3. Embrace challenges and be resilient: “When you have all of these different experiences, you understand that it’s not always going to be perfect,” advised Laura Leal-Taixé, professor at the Technical University of Munich and principal scientist at Argo AI. “And when things aren’t always perfect, you’re able to have competence because [you know that you] did that really hard thing and was able to get through it.”
4. Understand the purpose behind your work: “Understand the baseline, how do you collect the data baseline — understand the physical, the bottom line. What’s the purpose, what do you want to do?” advised Jay Lee, Ohio eminent scholar of the University of Cincinnati and board member of Foxconn.
5. Collaborate and seek support from others: “It’s so important for resiliency to find people across different domains and really tap into that,” said Carrie Gotch, creator and content strategy for 3D/AR at Adobe. “No one does it alone, right? You’re always part of a system, part of a team of people.”

The panelists stressed the importance of staying up to date and curious, gaining practical experience, collaborating with others and taking risks when building a career in AI.

Start your journey to an AI career by signing up for NVIDIA GTC, running in March, where you can network, get trained on the latest tools and hear from thought leaders about the impact of AI in various industries.

It could be the first step toward a rewarding AI career that takes you into 2023 and beyond.

Editor’s note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.

Stephen Tong, aka Funky Boy, has always loved music and photography. He’s now transferring the skills developed over the years as a music producer — shooting time lapses, creating audio tracks and more — to a new passion of his: 3D content creation.

Tong began creating 3D renders and animations earlier this year, using the NVIDIA Omniverse platform for building and connecting custom 3D pipelines.

Within just a couple months of learning to use Omniverse, Tong created a music video with the platform. The video received honorable mention in the inaugural #MadeInMachinima contest last March, which invited participants to remix popular characters from games like Squad, Mount & Blade II: Bannerlord and MechWarrior Mercenaries 5 using the Omniverse Machinima app.

In September, Tong participated in the first-ever Omniverse developer contest, which he considered the perfect way to learn about extending the platform and coding with the popular Python programming language. He submitted three Omniverse extensions — core building blocks that let anyone create and extend functions of Omniverse apps — aimed at easing creative workflows like his own.

Ringing in the Season the Omniverse Way

The artist also took part in the #WinterArtChallenge this month from NVIDIA Studio, a creative community and platform of NVIDIA RTX and AI-accelerated creator apps. Creatives from around the world shared winter-themed art on social media using the hashtag.

Tong said his scene was inspired by cozy settings he often associates with the holidays.

First, the artist used AI to generate a mood board. Once satisfied with the warm, cozy mood, he modeled a winter chalet — complete with a snowman, wreaths and sleigh — using the Marbles RTX assets, free to use in the Omniverse Launcher, as well as some models from Sketchfab.

Tong collected the assets in Unreal Engine before rendering the 3D scene using the Omniverse Create and Blender apps. The Universal Scene Description (USD) framework allowed him to bring the work from these various applications together.

“USD enables large scenes to be loaded fast and with ease,” he said. “The system of layers makes Omniverse a powerful tool for collaboration and iterations.”

With his festive creativity on a roll, Tong also orchestrated an animated quartet lip-syncing to “Carol of the Bells” using Omniverse Audio2Face, an AI app that quickly and easily generates expressive facial animations from just an audio source, as well as the DaVinci Resolve application for video editing.

Watch to keep up the holiday spirit:

Join In on the Creation

Creators and developers across the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.

To hear the latest made possible by accelerated computing, AI and Omniverse, watch NVIDIA’s special address at CES on Tuesday, Jan. 3, at 8 a.m. PT.

Check out more artwork from Tong and other “Omnivores” in the gallery. Connect your workflows to Omniverse with software from Adobe, Autodesk, Epic Games, Maxon, Reallusion and more.

Follow NVIDIA Omniverse on Instagram, Medium, Twitter and YouTube for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.