Editor’s note: This is part of a series profiling people advancing science with high performance computing.
Ryan Coffee makes movies of molecules. Their impacts are huge.
The senior scientist at the SLAC National Accelerator Laboratory (above) says these visualizations could unlock the secrets of photosynthesis. They’ve already shown how sunlight can cause skin cancer.
Long term, they may help chemists engineer life-saving drugs and batteries that let electric cars go farther on a charge.
To make films that inspire that kind of work, Coffee’s team needs high-performance computers, AI and an excellent projector.
A Brighter Light
The projector is called the Linac Coherent Light Source (LCLS). It uses a linear accelerator a kilometer long to pulse X-rays up to 120 times per second.
That’s good enough for a Hollywood flick, but not fast enough for Coffee’s movies.
“We need to see how electron clouds move like soap bubbles around molecules, how you can squeeze them in certain ways and energy comes out,” said Coffee, a specialist in the physics at the intersection of atoms, molecules and optics.
So, an upgrade next year will let the giant instrument take 100,000 frames per second. In two years, another enhancement, called LCLS II, will push that to a million frames a second.
Sorting the frames that flash by that fast — in random order — is a job for the combination of high performance computing (HPC) and AI.
AIs in the Audience
Coffee’s goal is to sit an AI model in front of the LCLS II. It will watch the ultrafast movies to learn an atomic dance no human eyes could follow.
The work will require inference on the fastest GPUs available running next to the instrument in Menlo Park, Calif. Meanwhile, data streaming off LCLS II will be used to constantly retrain the model on a bank of NVIDIA A100 Tensor Core GPUs at the Argonne National Laboratory outside Chicago.
It’s a textbook case for HPC at the edge, and one that’s increasingly common in an era of giant scientific instruments that peer up at stars and down into atoms.
A look at part of the LCLS instrument. (For more details, see this blog.)
So far, Coffee’s team has been able to retrain an autoencoder model every 10-20 minutes while it makes inferences 100,000 times a second.
“We’re already in the realm of attosecond pulses where I can watch the electron bubbles slosh back and forth,” said Coffee, a core member of SLAC’s overall AI initiative.
A Broader AI Collaboration
The next step is even bigger.
Data from Coffee’s work on molecular movies will be securely shared with data from Argonne’s Advanced Proton Source, a kind of ultra-high-resolution still camera.
“We can use secure, federated machine learning to pull these two datasets together, creating a powerful, shared transformer model,” said Coffee, who’s collaborating with multiple organizations to make it happen.
Coffee in the ‘projection room’ where the light in his next molecular movies will first appear.
The transformer will let scientists generate synthetic data for many data-starved applications such as research on fusion reactors.
It’s an effort specific to science that parallels work in federated learning in healthcare. Both want to build powerful AI models for their fields while preserving data privacy and security.
“We know people get the best results from large language models trained on many languages,” he said. “So, we want to do that in science by taking diverse views of the same things to create better models,” he said.
The Quantum Future
The atomic forces that Coffee studies may power tomorrow’s computers, the scientist explains.
“Imagine a stack of electron bubbles all in the same quantum state, so it’s a superconductor,” he said. “When I add one electron at the bottom, one pops to the top instantaneously because there’s no resistance.”
The concept, called entanglement in quantum computing, means two particles can switch states in lock step even if they’re on opposite sides of the planet.
That would give researchers like Coffee instant connections between powerful instruments like LCLS II and remote HPC centers training powerful AI models in real time.
Sounds like science fiction? Maybe not.
Coffee foresees a time when his experiments will outrun today’s computers, a time that will require alternative architectures and AIs. It’s the kind of big-picture thinking that excites him.
“I love the counterintuitiveness of quantum mechanics, especially when it has real, measurable results humans can apply — that’s the fun stuff.”
When NVIDIA co-founder Chris Malachowsky approached University of Florida Provost Joe Glover with the offer of an AI supercomputer, he couldn’t have predicted the transformative impact it would have on the university. In just a short time, UF has become one of the top public colleges in the U.S. and developed a groundbreaking neural network for healthcare research.
In a recent episode of NVIDIA’s AI Podcast, host Noah Kravitz sat down with Glover, who is also senior vice president of academic affairs at UF. The two discussed the university’s efforts to put AI to work across all aspects of higher education, including a public-private partnership with NVIDIA that has helped transform UF into one of the leading AI universities in the country.
Just a year after the partnership was unveiled in July 2020, UF rose to No. 5 on the U.S. News and World Report’s list of the best public colleges in the U.S. The ranking was, in part, a recognition of UF’s vision for infusing AI into its teaching and research.
Last March, UF Health, the university’s academic health center, teamed with NVIDIA to develop GatorTron, a neural network that generates synthetic clinical data researchers can use to train other AI models in healthcare.
According to Glover, the success of UF’s AI initiatives can be attributed to “a combination of generous philanthropy, some good decisions, a little inspiration and a few miracles here and there along the way.”
He believes that the university’s AI-powered vision has significantly impacted its teaching and research and will continue to do so in the future.
Pindar Van Arman, an American artist and roboticist, designs painting robots that explore the differences between human and computational creativity. Since his first system in 2005, he has built multiple artificially creative robots. The most famous, Cloud Painter, was awarded first place at Robotart 2018.
Steven Frank is a partner at the law firm Morgan Lewis, specializing in intellectual property and commercial technology law. He’s also half of the husband-wife team that used convolutional neural networks to authenticate artistic masterpieces, including da Vinci’s Salvador Mundi, with AI’s help.
Humans playing games against machines is nothing new, but now computers can develop games for people to play. Programming enthusiast and social media influencer Harrison Kinsley created GANTheftAuto, an AI-based neural network that generates a playable chunk of the classic video game Grand Theft Auto V.
Subscribe to the AI Podcast: Now Available on Amazon Music
Powerful new GeForce RTX GPUs, a new generation of hyper-efficient laptops and new Omniverse capabilities and partnerships across the automotive industry were highlights of a news-packed address ahead of this week’s CES trade show in Las Vegas.
“AI will define the future of computing and this has influenced much of what we’re covering today,” said Jeff Fisher, senior vice president for gaming products at NVIDIA, as he kicked off the presentation.
Fisher was joined by several leaders from NVIDIA to introduce products and partnerships across gaming and content creation, robotics and next-generation automobiles.
Introducing GeForce RTX 40 Series Laptops, RTX 4070 Ti Graphics Cards and DLSS 3 Games
Fisher said the performance and power efficiency of the NVIDIA GeForce RTX 40 Series Laptop GPUs enable the greatest ever generational leap, including 14-inch gaming and creating powerhouse laptops, starting at $999 in February.
New GeForce RTX 4070 Ti graphics cards for desktops are faster than last generation’s RTX 3090 Ti at nearly half the power, bringing the NVIDIA Ada Lovelace architecture down to $799, with availability starting Jan. 5.
And DLSS 3 is being adopted by developers faster than any prior NVIDIA tech, with 50 released and upcoming titles, including Witchfire, The Day Before, Warhaven, THRONE AND LIBERTY and Atomic Heart.
In addition, RTX 4080 performance is coming to the NVIDIA GeForce NOW cloud-gaming service. As a result, Fisher said millions more gamers will have access to the NVIDIA Ada architecture with GeForce NOW’s Ultimate membership.
The new tier will bring NVIDIA Reflex and 240 frames per second streaming to the cloud for the first time, along with full ray tracing and DLSS 3 in games like Portal With RTX.
Momentum for NVIDIA RTX continues to build, Fisher said. “Creating has grown beyond photos and videos to virtual worlds rendered with 3D cinematic graphics and true-to-life physics,” Fisher said. “The RTX platform is powering this growth.”
Ray tracing and AI are defining the next generation of content, and NVIDIA Studio is the platform for this new breed of content creators. The heartbeat of Studio is found in NVIDIA Omniverse, where creators can connect accelerated apps and collaborate in real time.
NVIDIA’s Stephanie Johnson, vice president of consumer marketing, introduced a new suite of generative AI tools and experimental plug-ins that harness the power of AI.
Built with NVIDIA RTX, Omniverse is a platform enabling 3D artists to connect their favorite tools from Adobe, Autodesk, SideFX, Unreal Engine and more. And Omniverse now has a new Connector for Unity, said Stephanie Johnson, vice president of consumer marketing at NVIDIA.
Johnson introduced a suite of new generative AI tools and experimental plug-ins using the power of AI as the ultimate creative assistant. Audio2Face and Audio2Gesture generate animations from an audio file. The AI ToyBox by NVIDIA Research lets users generate 3D meshes from 2D inputs.
Companies have used generative AI technology to build Omniverse Connectors and extensions. Move.AI’s Omniverse extension, for example, enables video-to-animation. Lumirithmic generates 3D mesh for heads from facial scans. And Elevate3D generates photorealistic 3D visualizations of products from 360-degree video recordings.
Johnson also announced that NVIDIA RTX Remix, which is built on Omniverse and is “the easiest way to mod classic games,” will be entering early access soon. “The modding community can’t wait to get their hands on Remix,” she said.
NVIDIA Isaac Sim Brings Significantly Improved Features, Tools for Developing Intelligent Robots
Simulation plays a vital role in the lifecycle of a robotics project, explained Deepu Talla, vice president of embedded and edge computing at NVIDIA. Partners are using NVIDIA Isaac Sim to create digital twins that help speed the training and deployment of intelligent robots.
NVIDIA’s Deepu Talla, vice president of embedded and edge computing, announced the next release of Isaac Sim, NVIDIA’s robotics simulation application and synthetic data generation tool.
To revolutionize the way the robotics ecosystem develops the next generation of autonomous robots, Talla announced major updates to the next release of Isaac Sim. This includes improved sensor and lidar support to more accurately model real-world performance, a new conveyor-building tool, a new utility to add people to the simulation environment, a collection of new sim-ready warehouse assets and a host of new popular robots that come pre-integrated.
For the open-source ROS developer community, this release upgrades support for ROS 2 Humble and Windows, Talla added. And for robotics researchers, NVIDIA is introducing a new tool called Isaac ORBIT, which provides operating environments for manipulator robots. NVIDIA has also improved Isaac Gym for reinforcement learning and updated Isaac Cortex for collaborative robot programming.
“We are committed to advancing robotics and arguably investing more than anyone else in the world,” Talla said. “We are well on the way to having a thousand to million times more virtual robots for every physical robot deployed.”
Mercedes-Benz to Create Digital Twins; Foxconn Building EVs on NVIDIA DRIVE; Geforce NOW Streams to Cars
The NVIDIA DRIVE platform is open and easy to program, said Ali Kani, vice president of automotive at NVIDIA.
Hundreds of partners across the automotive ecosystem are now developing software on NVIDIA DRIVE, including 20 of the top 30 manufacturers building new energy vehicles, many of the industry’s top tier one manufacturers and software makers, plus eight of the largest 10 trucking and robotaxi companies.
It’s a number that continues to grow, with Kani announcing a partnership with Foxconn, the world’s largest technology manufacturer and service provider, to build electric vehicles based on NVIDIA DRIVE Hyperion.
NVIDIA’s Ali Kani, vice president of automotive, announced a partnership with Foxconn, that GeForce NOW will be “coming to screens in your car” and that Mercedes-Benz is using NVIDIA digital twin technology to plan and build more efficient production facilities.
“With Hyperion adoption, Foxconn will manufacture vehicles with leading electric range as well as state-of-the-art AV technology while reducing time to market,” Kani said.
Kani touched on how, as next-generation cars become autonomous and electric, interiors are transformed into mobile living spaces, complete with the same entertainment available at home. GeForce NOW will be “coming to screens in your car,” Kani said.
While gamers will enjoy virtual worlds from inside their cars, tools such as the metaverse are critical to the development and testing of new autonomous vehicles.
Kani announced that Mercedes-Benz is using digital twin technology to plan and build more efficient production facilities. “The applications for Omniverse in the automotive market are staggering,” Kani said.
The latest release of NVIDIA Omniverse Enterprise, available now, brings increased performance, generational leaps in real-time RTX ray and path tracing, and streamlined workflows to help teams build connected 3D pipelines, and develop and operate large-scale, physically accurate, virtual 3D worlds like never before.
Artists, designers, engineers and developers can benefit from various enhancements across common Omniverse use cases, including breaking down 3D data silos through aggregation, building custom 3D pipeline tools and generating synthetic 3D data.
The release includes support for the breakthrough innovations within the new NVIDIA Ada Lovelace architecture, including third-generation RTX technology and DLSS 3, delivering up to 3x performance gains when powered by the latest GPU technology, such as NVIDIA RTX 6000 Ada Generation, NVIDIA L40 and OVX systems.
The update also delivers features and capabilities such as new Omniverse Connectors, layer-based live workflows, improved user experience and customization options, including multi-view ports, editable hotkeys, lighting presets and more.
Enhancing 3D Creation
The Omniverse ecosystem is expanding, with more capabilities and tools that allow teams to elevate 3D workflows and reach new levels of real-time, physically accurate simulations. The latest additions include:
New Connectors: Omniverse Connectors enable more seamless connected workflows between disparate 3D applications. New Adobe Substance 3D Painter, Autodesk Alias, PTC Creo, Kitware’s Paraview Omniverse Connectors are now supported on Omniverse Enterprise. Users can also easily export data created in NX Software from Siemens Digital Industries Software.
Omniverse DeepSearch: Now generally available, this AI-powered service lets teams intuitively search through extremely large, untagged 3D databases with natural language or using 2D reference images. This unlocks major value in previously unwieldy digital backlots and asset collections that lacked the ability to search.
Omniverse Farm: A completely renewed user interface provides improved usability and performance, plus Kubernetes support.
Omniverse Cloud: New cloud containers for Enterprise Nucleus Server, Replicator, Farm and Isaac Sim for AWS provide enterprises more flexibility in connecting and managing distributed teams all over the world. AWS’ security, identity and access-management controls allow teams to maintain complete control over their data. Containers are now available on NVIDIA NGC.
Strengthening Core Components for Building 3D Worlds
Omniverse Enterprise is designed for maximum flexibility and scalability. This means creators, designers, researchers and engineers can quickly connect tools, assets and projects to collaborate in a shared virtual space.
Omniverse Enterprise brings updates to the core components of the platform, including:
Omniverse Kit SDK, the powerful toolkit for building extensions, apps, microservices or plug-ins, now makes it easier than ever to build advanced tools and Omniverse applications with new templates and developer workflows.
Omniverse Create, a reference app for composing large-scale, USD-based worlds, now includes NVIDIA DLSS 3 and multi-viewport support, making it easier for Omniverse Enterprise users to fluidly interact with extremely large and complex scenes.
Omniverse View, a reference app for reviewing 3D scenes, has been streamlined to focus purely on the review and approval experience. New collaborative, real-time, interactive capabilities — including markup, annotation, measure and simple navigation — make stakeholder presentations easier and more interactive than ever.
Omniverse Nucleus, the database and collaboration engine, now includes improved IT management tools, such as expanded version control to handle atomic checkpoints on the server. Updated Large File Transfer service enables users to move files between servers, on premises or in the cloud to benefit hybrid workflows. And new self-service deployment instructions for Enterprise Nucleus Server on AWS are now available, letting customers deploy and manage Nucleus in the cloud.
Customers Dive Into Omniverse Enterprise
Many customers around the world have experienced enhanced 3D workflows with Omniverse Enterprise.
Dentsu International, one of the largest global marketing and advertising agency networks, always looks for solutions that enable collaborative and seamless work, with a central repository for completed projects.
“There’s great value when multiple artists with different tools, located in different places, are able to work together, and NVIDIA Omniverse Enterprise makes that possible,” said Melody Jeter, vice president of Creative Technology at Dentsu. “The collaboration tackles the robust challenge of laborious file transferring — now, the workflows are all synced live and in real time.”
In addition to enhancing current pipelines with Omniverse Enterprise, Dentsu is looking to incorporate NVIDIA generative AI into its 3D design pipeline with software development kits like Omniverse ACE and Audio2Face.
Mercedes Benz, the German premium vehicle manufacturer, is using Omniverse Enterprise at its sites world wide to design, plan and optimize its manufacturing and assembly facilities. By developing full-fidelity digital twins of their production environments, globally dispersed teams will open up new abilities to collaborate in real time, accelerate decision-making and identify opportunities to reduce waste, decrease energy consumption and continuously enhance quality.
Zaha Hadid Architects (ZHA) is a renowned architectural design firm that has created some of the world’s most singular building designs. ZHA focuses on creating transformative cultural, corporate and residential spaces through cutting-edge technologies. With Omniverse Enterprise, the team can accelerate and automate its workflows, as well as develop custom tools within the platform.
“We are working with NVIDIA to incorporate Omniverse as the connective infrastructure of our tech stack. Our goal is to retain design intent across the various project stages and improve productivity,” said Shajay Bhooshan, associate director at Zaha Hadid Architects. “We expect NVIDIA Omniverse to play a critical, supportive role to our efforts to create a platform that’s agnostic, version-controlled and a single source of truth for design data, as it evolves from idea to delivery.”
NVIDIA Omniverse Enterprise is available by subscription from BOXX Technologies, Dell Technologies, Z by HP and Lenovo, and channel partners including Arrow, ASK, PNY and Leadtek. The platform is optimized to run on NVIDIA-Certified, RTX-enabled desktop and mobile workstations, as well as servers, with new support for NVIDIA RTX Ada generation systems.
AI is extending further into the vehicle as autonomous-driving technology becomes more prevalent.
With the NVIDIA DRIVE platform, automakers can design and implement intelligent interior features to continuously surprise and delight customers.
It all begins with the compute architecture. The recently introduced NVIDIA DRIVE Thor platform unifies traditionally distributed functions in vehicles — including digital cluster, infotainment, parking and assisted driving — for greater efficiency in development and faster software iteration.
NVIDIA DRIVE Concierge, built on the DRIVE IX software stack, runs an array of safety and convenience features, including driver and occupant monitoring, digital assistants and autonomous-vehicle visualization.
Automakers can benefit from NVIDIA data center solutions even if they aren’t using the NVIDIA DRIVE platform. With cloud technology, vehicles can stream the NVIDIA GeForce NOW cloud-gaming service without any special equipment. Plus, developers can train, test and validate in-vehicle AI models on NVIDIA DGX servers.
The same data center technology that’s accelerating AI development — in combination with the NVIDIA Omniverse platform for creating and operating metaverse applications — is also revolutionizing the automotive product cycle. Using NVIDIA DRIVE Sim built on Omniverse, automakers can design vehicle interiors and retail experiences entirely in the virtual world.
Easing Pain Points From Concept to Customer
Designing and selling vehicles requires the highest levels of organization and orchestration. The cockpit alone has dozens of components — such as steering wheel, cluster and infotainment — that developers must create and integrate with the rest of the car.
These processes are incredibly time- and resource-intensive — there are countless configurations, and chosen designs must be built out and tested prior to production. Vehicle designers must collaborate on various layouts, which must then be validated and approved. Customers must travel to dealerships to experience various options, and the ability to test features depends on a store’s inventory at any given time.
In the virtual world, developers can easily design vehicles, and car buyers can seamlessly test them, leading to an optimal experience on both ends of the production pipeline.
Design and Collaboration
Automakers operate design centers around the world, tapping into expertise from North America, Europe, Asia and other automotive hubs. Working on user experience concepts across these locations requires frequent international travel and close coordination.
With DRIVE Sim, designers and engineers anywhere in the world can work together to develop the cockpit experience, without having to leave their desks.
Design teams can also save time and valuable resources by testing concepts in the virtual world, without having to wait for physical prototypes. Decision-makers can review designs and ensure they meet relevant safety standards in DRIVE Sim before sending them to production.
Transforming the Customer Experience
The benefits of in-vehicle simulation extend far beyond the design phase.
Consumers are increasingly expecting full-service digital retail experiences. More than 60% of shoppers want to conduct more of the car-buying process online compared to the last time they bought a vehicle, while more than 75% are open to buying a car entirely online, according to an Autotrader survey.
The same tools used to design the vehicle can help meet these rising consumer expectations.
With DRIVE Sim, car buyers can configure and test the car from the comfort of their homes. Customers can see all potential options and combinations of vehicle features at the push of a button and take their dream car for a virtual spin — no lengthy trips to the dealership required.
From concept design to customer experience, DRIVE Sim is easing the process and opening up new ways to design and enjoy intelligent vehicles.
Building state-of-the-art factories requires a state-of-the art planning system.
Mercedes-Benz announced at CES that it is taking the next step in digitizing its production process, using the NVIDIA Omniverse platform to design and plan manufacturing and assembly facilities.
By tapping into NVIDIA AI and metaverse technologies, the automaker can create feedback loops to reduce waste, decrease energy consumption and continuously enhance quality.
Mercedes-Benz has been working with NVIDIA to develop software-defined vehicles. Its upcoming fleets will be built on NVIDIA DRIVE Orin centralized compute, with intelligent driving capabilities tested and validated in the NVIDIA DRIVE Sim platform, built on Omniverse.
The automaker’s latest announcement infuses AI and metaverse technologies even further into the Mercedes-Benz development process with smarter, more efficient manufacturing.
Electric-First Platform Built on Digital-First Process
Vehicle production is a colossal undertaking that requires thousands of parts and workers moving in harmony. Any supply chain or production issues can lead to costly delays.
On top of that, when automakers roll out a new model, they must reconfigure the layout of production plants to account for the new vehicle design. This process can take significant portions of the factory offline, pausing manufacturing for existing vehicles.
Mercedes-Benz plans to start production of its new dedicated platform for electric vehicles at its plant in Rastatt, Germany. The site currently manufactures the automaker’s A- and B-Class as well as the compact SUV GLA and the all-electric Mercedes-Benz EQA.
Mercedes-Benz plant in Rastatt, Germany, where the all-electric EQA is manufactured.
Experts from NVIDIA and Mercedes-Benz operations are setting up a “digital first” – planning process for the plant that won’t disrupt the current production of compact car models at the site. This blueprint will be rolled out to other parts of the global Mercedes-Benz production network for more agile vehicle manufacturing.
Developing With Digital Twins
NVIDIA Omniverse is an open 3D development platform enabling enterprises and institutions across all industries to build and operate digital twins for industrial and scientific use cases. It’s based on Universal Scene Description (USD), allowing enterprises and developers to construct custom 3D pipelines to break down data silos and interact with a single-source-of-truth view of their aggregated simulations.
With Omniverse, Mercedes-Benz planners can access the digital twin of the factory, reviewing and optimizing the plant as needed. Every change can be quickly evaluated and validated in the virtual world, then implemented in the real world to ensure maximum efficiency and ergonomics for factory workers.
Additionally, Mercedes-Benz can synchronize plant locations anywhere in the world by connecting Omniverse with its in-house MO360 Data Platform.
This capability streamlines operations across the global production network and enables over-the-air software updates to manufacturing equipment.
By planning production on NVIDIA Omniverse, Mercedes-Benz is implementing a manufacturing system as intelligent as its vehicles.
Autonomous and electric vehicles are making personal transportation safer and more sustainable — as well as more entertaining.
At CES today, NVIDIA announced that the NVIDIA GeForce NOW cloud gaming service will be coming to cars, with no special equipment needed. Hyundai Motor Group, BYD and Polestar — already members of the NVIDIA DRIVE ecosystem — are the first automakers working with NVIDIA to deliver GeForce NOW in their vehicles.
“We are excited to offer our customers the best in technology, comfort, design from BYD and now the latest in gaming through NVIDIA GeForce NOW high-performance cloud gaming service,” said Stella Li Vice President of BYD and CEO of BYD America. “The experience of driving our cars is becoming increasingly sophisticated, entertaining and sustainable.”
Thanks to cloud technology and the mobile internet, vehicle occupants can keep boredom at bay with the ability to enjoy music, movies and now video games in the car. Drivers and front seat passengers can access games while parked or charging, while those in the backseat of vehicles where rear entertainment is enabled can play anytime on the road.
In March, BYD, the world’s leading manufacturer of new energy vehicles, announced it would build its NEVs on the NVIDIA DRIVE Hyperion platform, starting in the first half of 2023.
Polestar is also using NVIDIA DRIVE for its software-defined architecture, with the upcoming Polestar 3 powered by the NVIDIA DRIVE Orin system-on-a-chip.
Now, these trailblazing automakers will offer vehicles that are as entertaining as they are intelligent with the addition of GeForce NOW.
1,000+ Games, Anywhere, Anytime
The next generation of vehicles are transforming from fixed function to software-defined. At the same time, cars are becoming more connected, opening up new services that weren’t possible before.
GeForce NOW delivers a full PC-gaming experience to nearly any device, including laptops, mobile devices, smart TVs — and now, personal vehicles. The GeForce NOW client for auto can run on Android- or browser-based in-vehicle infotainment systems, providing an app to easily select and launch favorite games.
It uses breakthroughs in low-latency cloud-streaming technology for real-time game play powered by GeForce servers in the cloud. GeForce NOW features more than 1,500 titles — over 1,000 playable with a controller — including top-rated games such as A Plague Tale: Requiem, The Witcher 3: Wild Hunt and Cyberpunk 2077, from leading stores such as Steam, Epic Games Store, the EA app, Ubisoft and GOG.com.
The service also includes many of the most-played free-to-play games like Fortnite, Lost Ark and Destiny 2.
Experiencing the Future
The ability to stream games to the car is a key component of the future of in-vehicle experiences.
As autonomous-driving technology becomes more prevalent, the interior will play a much larger role in differentiating brand identities. Consumers will focus more on features such as digital assistants, beautiful graphic displays and streaming content as they make vehicle buying decisions.
This shift is turning vehicles into living spaces, where passengers can enjoy what’s important to them — whether it be work or play — rather than the stress of driving.
By providing GeForce NOW in their vehicles, Hyundai Motor Group, BYD and Polestar are taking a significant step into this new era of personal transportation.
Fueled by powerful NVIDIA RTX technology and backed by the NVIDIA Studio platform for creators, a creative revolution is underway as a wave of 2D artists moves to 3D, video workflows move to real time and AI tools help artists create content faster.
For them, and the 110 million professional and hobbying PC creators, there’s NVIDIA Studio. It’s a platform for content creators that supercharges 110 creative apps with RTX GPUs, provides lasting stability with NVIDIA Studio Drivers, and includes a powerful suite of Studio apps — Omniverse, Canvas, Broadcast and RTX Remix.
The heartbeat of the Studio platform lies in NVIDIA Omniverse, a platform where creators can connect their 3D apps and collaborate in real time. Omniverse is expanding with Blender enhancements, a new suite of experimental generative AI tools for 3D artists and thousands of new, free Universal Scene Description (USD) assets available in the content browser in Omniverse Create.
Built on Omniverse, NVIDIA RTX Remix is a free modding platform to quickly create mods for classic games with full ray tracing, enhanced materials, DLSS 3 and NVIDIA Reflex. We’ve enabled two amazing community modders to bring RTX to Portal’s most famous mod, Portal Prelude, preserving the timeless gameplay, while relighting it with full ray tracing.
The NVIDIA Broadcast and Canvas apps, also in the exclusive Studio software suite, have upcoming upgrades. Broadcast is adding a new Eye Contact feature, and Canvas will introduce 360-degree images that can be used as environment maps in 3D apps.
Coming to GeForce RTX 40 and 30 Series GPUs next month, the RTX Video Super Resolution feature uses AI to improve the quality of any video watched in a browser by removing blocky compression artifacts and upscaling video resolution. This improves video sharpness and clarity, and lets people watch online content in its native resolution on high-resolution displays. RTX Video Super Resolution will be available in February for Chrome and Edge browsers.
And to celebrate CES, influential Blender artist CG Geek will take on a three-day design challenge, powered by NVIDIA Studio technologies. Plus, explore how seven freelance artists from across the world created an experimental short film together, in real time, using Studio hardware, Omniverse and the Nucleus Cloud — this week In the NVIDIA Studio.
From Concept to Completion, Faster With New NVIDIA Studio Laptops
GeForce RTX GPUs provide massive speedups in 3D, video and broadcast workflows, as well as a myriad of AI tools, thanks to dedicated ray tracing, AI and video-encoding hardware. New Studio laptops with GeForce RTX 40 Series Laptop GPUs use the NVIDIA Ada Lovelace architecture — with support for DLSS 3, AV1 and more — and fifth-generation Max-Q technologies for maximum performance and efficiency.
The new laptops provide a quantum leap in performance over the last generation, and a massive boost in efficiency that allows OEMs to include more powerful GPUs in thinner, more portable systems. NVIDIA Studio laptops are purpose-built for creators, with the right combination of hardware for creative workflows, color-accurate displays, and preinstalled Studio Drivers and Studio software.
The Razer Blade 16 is available with up to a GeForce RTX 4090 Laptop GPU.
The instantly recognizable Razer Blade laptop gets an RTX 40 Series upgrade with up to a GeForce RTX 4090 Laptop GPU. The 16-inch, HDR-capable, dual-mode, mini-LED display boasts 1,000 nits peak brightness, which pairs nicely with a Creator mode that enables sharp, ultra-high-definition+ native resolution at 120Hz.
The MSI Stealth 17 Studio A13V is available with up to a GeForce RTX 4090 Laptop GPU.
MSI’s Stealth series will launch with a wide range of Studio options. With 14- to 17-inch models and up to GeForce RTX 4090 Laptop GPUs, creators can pick the laptop that best suits their needs.
Powered by a GeForce RTX 4070 Laptop GPU, the ASUS Zenbook Pro 14 showcases a new level of performance for a 14-inch laptop.
The increased efficiency of RTX 40 Series Laptop GPUs and fifth-generation Max-Q Technologies deliver high performance to laptops as slim as 14 inches, with up to GeForce RTX 4070 Laptop GPUs, a new milestone. The ASUS Zenbook Pro 14 OLED is one example, with a GeForce RTX 4070 Laptop GPU in a super-thin 14-inch design, and accompanied by an all-new, cutting-edge 2.8K 120Hz OLED NanoEdge Dolby Vision display.
GeForce RTX 4070 Ti: 40 Series Benefits and Faster Than a 3090 Ti
For artists who prefer to create from their desktops, the new GeForce RTX 4070 Ti graphics cards, available starting Thursday, Jan. 5, offers incredible performance that’s up to 15% faster than a GeForce RTX 3090 Ti in 3D rendering. Like the RTX 4090 and 4080, the newest RTX 40 Series GPU features dual AV1 video encoders. When enabled in top video-editing and livestreaming apps — such as Adobe Premiere Pro (via the Voukoder plug-in), DaVinci Resolve, OBS and Jianying — export times are cut in half with improved video quality.
GeForce RTX 4070 Ti graphics cards.
3D creators see up to a 70% increase in performance compared to the GeForce RTX 3070 Ti in popular apps like Autodesk Maya, Blender, Chaos V-Ray, Epic Games Unreal Engine and Unity. Users can also unlock creative freedom by unifying 3D assets, libraries and tools in Omniverse.
And all creators can benefit from the new fourth-generation Tensor Cores for AI tools, providing up to 2x increased performance compared to the previous generation.
The January Studio Driver, supporting the GeForce RTX 4070 Ti launch, will be available for download later this week.
Create Easily With Studio Software
NVIDIA RTX GPUs unlock exclusive software for creators: Omniverse, RTX Remix, Canvas and Broadcast. They’re all getting upgrades early this year.
Based on the USD framework, Omniverse enables artists to connect their favorite tools from Adobe, Autodesk, Epic Games, SideFX and more. Creators can see their scenes instantly come together without lengthy import or export cycles. Changes happen in real time across the connected apps — whether a single artist is working in multiple apps simultaneously or collaborating with another artist from across the globe.
The Blender alpha branch release, part of a series of updates to Omniverse, is now available in the Omniverse Launcher. It enables users to repair geometry, generate automatic UVs and decimate high-resolution CAD data to more usable polycounts.
Performance updates to Audio2Face, Audio2Gesture and Audio2Emotion — powerful AI tools within Omniverse — better enable instant, realistic animation of characters. The AI ToyBox of experimental tools is now available in the Omniverse Extension Manager. Thousands of new, free 3D assets have also been made available worldwide for users to build and create within Omniverse.
‘Portal: Prelude RTX’ is coming soon to mod sites ModDB and Nexus Mods.
RTX Remix, which is built on Omniverse and was used to create the jaw-dropping Portal with RTX, is approaching its early-access release. Nicolas “NykO18” Grevet — the original creator of Portal: Prelude — is using RTX Remix to remaster his unofficial Portal prequel from 2008. Grevet is working with modder David “Kralich” Driver-Gomm to modernize the assets and relight the game with stunning, full ray tracing. Portal: Prelude RTX will soon become free to download from mod sites like ModDB and Nexus Mods.
Canvas allows creators to paint by material, rather than color, using simple brushstrokes and AI to quickly conceptualize a beautiful image. Canvas is getting a new 360 image feature, which will become available in an upcoming free update for RTX users, helping artists create panoramic scenes and export them into any 3D app to use as an environment map. Such maps are used to change the ambient lighting of a 3D scene and appear in reflections for added realism.
Later this month, Broadcast will add a new Eye Contact feature that can change the position of the subject’s eyes to appear focused on the camera, as well as a new vignette effect. The update also improves the popular virtual background feature, adding temporal information for increased stability. The virtual background improvements can also be found in the recent OBS Studio 29.0 software release, and will be updated SDKs for developers later this week.
3D Need for Speed
To showcase the speed at which Studio creators flow, talented 3D artist CG Geek has been challenged to create an animated Blender scene during CES. He’ll have just three days, starting Thursday, Jan. 5, to blitz through tough, typically time-consuming tasks.
He’s equipped with a GeForce RTX 4090 GPU, RTX acceleration and AI features throughout his entire workflow.
The group started by pulling in a pair of Nizam’s characters: “3D” built in ShapesXR and “Figgy” in Maya. Both were dropped into Omniverse seamlessly as USD files. Prompted by Lightcap, Nizam added knitted textures to “3D” using Adobe Substance 3D Painter, and the group immediately saw this through the cloud.
Solina then offered to help with motion capture using Xsens. As he added the rigging, Lightcap brought in a background created in Blender. Lightcap noted that “RTX is insane,” as his animated western landscape rendered nearly instantly.
Anticipation for precipitation, McEvenue’s Houdini-based rainy day simulation.
With the scene coming together, Varis changed lighting elements, looking to make things “more epic,” he said. To incorporate a storm into the scene, McEvenue added a rain simulation created in Houdini. The film’s characters make a quick escape, hijacking the hot air balloon of Toy Jensen, the AI avatar of NVIDIA founder and CEO Jensen Huang.
For the ascent, Goldstein used her GeForce RTX 4090 GPU to add steam thrusters using OmniGraph.
With the characters now in outer space, Wang imported a satellite created in Autodesk Maya. The scene concludes as the space adventurers take off for Jupiter.
While showcasing their talents, these 3D artists displayed the capabilities of the NVIDIA Studio platform. Powerful RTX GPUs, acceleration in popular 3D apps and exclusive Omniverse software that enabled collaboration from all over the world in real time.
The future of creativity is shining bright. And the NVIDIA Studio ecosystem is lighting the scene.
Demand for intelligent robots is growing as more industries embrace automation to address supply chain challenges and labor force shortages.
The installed base of industrial and commercial robots will grow more than 6.4x — from 3.1 million in 2020 to 20 million in 2030, according to ABI Research. Developing, validating and deploying these new AI-based robots requires simulation technology that places them in realistic scenarios.
At CES, NVIDIA announced major updates to Isaac Sim, its robotics simulation tool to build and test virtual robots in realistic environments across varied operating conditions. Now accessible from the cloud, Isaac Sim is built on NVIDIA Omniverse, a platform for creating and operating metaverse applications.
Powerful AI-Driven Capabilities for Roboticists
With humans increasingly working side by side with collaborative robots (cobots) or autonomous mobile robots (AMRs), it’s critical that people and their common behaviors are added to simulations.
Isaac Sim’s new people simulation capability allows human characters to be added to a warehouse or manufacturing facility and tasked with executing familiar behaviors— like stacking packages or pushing carts. Many of the most common behaviors are already supported, so simulating them is as simple as issuing a command.
To minimize the difference between results observed in a simulated world versus those seen in the real world, it’s imperative to have physically accurate sensor models.
Using NVIDIA RTX technology, Isaac Sim can now render physically accurate data from sensors in real time. In the case of an RTX-simulated lidar, ray tracing provides more accurate sensor data under various lighting conditions or in response to reflective materials.
Isaac Sim also provides numerous new simulation-ready 3D assets, which are critical to building physically accurate simulated environments. Everything from warehouse parts to popular robots come ready to go, so developers and users can quickly start building.
Significant new capabilities for robotics researchers include advances in Isaac Gym for reinforcement learning and Isaac Cortex for collaborative robot programming. Additionally, a new tool, Isaac ORBIT, provides simulation operating environments and benchmarks for robot learning and motion planning.
For the large community of Robot Operating System (ROS) developers, Isaac Sim upgrades support for ROS 2 Humble and Windows. All of the Isaac ROS software can now be used in simulation.
Expanding Isaac Platform Capabilities and Ecosystem Drives Adoption
The large and complex robotics ecosystem spans multiple industries, from logistics and manufacturing to retail, energy, sustainable farming and more.
The end-to-end Isaac robotics platform provides advanced AI and simulation software as well as accelerated compute capabilities to the robotics ecosystem. Over a million developers and more than a thousand companies rely on one or many parts of it. This includes many companies that have deployed physical robots developed and tested in the virtual world using Isaac Sim.
Telexistence has deployed beverage restocking robots across 300 convenience stores in Japan. To improve safety, Deutsche Bahn is training AI models to handle very important but unexpected corner cases that happen rarely in the real world — like luggage falling on a train track. Sarcos Robotics is developing robots to pick and place solar panels in renewable energy installations.
Festo uses Isaac Cortex to simplify programming for cobots and transfer simulated skills to the physical robots. Fraunhofer is developing advanced AMRs using the physically accurate and full-fidelity visualization features of Isaac Sim. Flexiv is using Isaac Replicator for synthetic data generation to train AI models.
While training robots is important, simulation is playing a critical role in training the human operators to work with and program robots. Ready Robotics is teaching programming of industrial robots with Isaac Sim. Universal Robotics is using Isaac Sim for workforce development to train end operators from the cloud.
Cloud Access Puts Isaac Platform Within Reach Everywhere
With Isaac Sim available in the cloud, global, multidisciplinary teams working on robotics projects can collaborate with increased accessibility, agility and scalability for testing and training virtual robots.
A lack of adequate training data often hinders deployment when building new facilities with robotics systems or scaling existing autonomous systems. Isaac Sim taps into Isaac Replicator to enable developers to create massive ground-truth datasets that mimic the physics of real-world environments.
Once deployed, dynamic route planning is required to operate an efficient fleet of hundreds of robots as automation requirements scale. NVIDIA cuOpt, a real-time fleet task-assignment and route-planning engine improves operational efficiencies with automation.
Watch NVIDIA’s special address at CES, where its executives unveiled products, partnerships and offerings in autonomous machines, robotics, design, simulation and more.
Whether creating realistic digital humans that can express raw emotion or building immersive virtual worlds, those in the design, engineering, creative and other industries across the globe are reaching new heights through 3D workflows.
Animators, creators and developers can use new AI-powered tools to reimagine 3D environments, simulations and the metaverse — the 3D evolution of the internet.
In a special address at CES, NVIDIA announced these features, as well as Omniverse preinstallation on NVIDIA Studio laptops and thousands of new, free USD assets to help accelerate adoption of 3D workflows.
Plus, a new release for Blender, now available in the Omniverse Launcher, is bringing 3D generative AI capabilities to Blender users everywhere. A new panel lets Blender users easily transfer shape keys and rigged characters. The challenge of reattaching a rigged character’s head can now be solved with a one-button operation from Omniverse Audio2Face — an AI-enabled tool that automatically generates realistic facial expressions from an audio file.
Another new panel for scene optimization lets users create USD scenes within their multi-app 3D workflows more easily and in real time.
In addition, Audio2Face, Audio2Gesture and Audio2Emotion — generative AI tools that enable instant 3D character animation — are getting performance updates that make it easier for developers and creators to integrate into their current 3D pipelines.
Creators can generate facial expressions from an audio file using Audio2Face; realistic emotions ranging from happy and excited to sad and regretful with Audio2Emotion; and realistic upper-body movement using Audio2Gesture. These audio-to-animation tools are game-changers for 3D artists, eliminating the need to perform tedious, manual tasks.
AI-assisted creator tools are expanding to even more communities of creative and technical professionals. When NVIDIA Canvas was introduced, it empowered artists to seamlessly generate landscapes and iterate on them with simple brushstrokes and AI. Coming soon, all RTX users will be able to download an update to Canvas that introduces 360 surround images to create and conceptualize panoramic environments and beautiful images. The AI ToyBox, which features extensions derived from NVIDIA Research, enables creators to generate 3D meshes from 2D inputs.
Omniverse’s powerful AI tools simplify complex tasks. Creators of all levels can tap into these resources to produce high-quality outputs that meet the growing demands for content and virtual worlds in the metaverse.
“The demand for 3D skills is skyrocketing, but learning 3D can be pretty scary to some, and definitely time consuming,” said Jae Solina, aka JSFilmz. “But these new platform developments not only let creatives and technical professionals continue to work in their favorite 3D tools, but also supercharge their craft and even use AI to assist them in their workflows.”
Omniverse Launcher, the portal to download Omniverse content and reference applications, has also been made available to system builders so they can preinstall it, enabling optimized, out-of-the-box experiences for 3D creators on NVIDIA Studio-validated laptops. GIGABYTE and AORUS will be the first laptops launching in 2023 with Omniverse Launcher preinstalled, expanding platform access to a growing number of 3D content creators.
NVIDIA RTX Remix is a free modding platform, built on Omniverse, that enables modders to
quickly create and share #RTXON mods for classic games, each with full ray tracing, enhanced materials, NVIDIA DLSS 3 and NVIDIA Reflex. Its release in early access is coming soon. The jaw-dropping Portal with RTXwas built with RTX Remix, and to demonstrate how easy it is for modders to turn RTX ON in their mods, we shared RTX Remix with the original creator of Portal: Prelude, an unofficial Portal prequel released in 2008.
Omniverse users can also access thousands of new, free USD assets, including a USD-based NVIDIA RTX Winter WorldMinecraft experience, and learn to create their own NVIDIA SimReady assets for complex simulation building. Using Omniverse, creators can supercharge their existing workflows using familiar tools such as Autodesk Maya, Autodesk 3ds Max, Blender, Adobe Substance 3D Painter, and more with AI, simulation tools and real-time RTX-accelerated rendering.
All types of 3D creators can take advantage of these new tools to push the boundaries of 3D simulation and virtual world-building. Users can reimagine digital worlds and animate lifelike characters with new depths of creativity through the bridging of audio-to-animation tools, generative AI and the metaverse.
Latest Omniverse Platform Updates
The latest updates within Omniverse include:
Early access for the Unity Omniverse Connector is now available.
Blender alpha release, now available in the Omniverse Launcher, enables users to repair geometry, generate automatic UVs and decimate high-resolution CAD data to more usable polycounts.
Audio2Face, Audio2Emotion and Audio2Gesture updates better enable instant, realistic animation of characters, now available in Omniverse Audio2Face and Omniverse Machinima.
NVIDIA Canvas is coming soon to the Omniverse Launcher with new capabilities that enable the creation of 360-degree landscapes with simple brushstrokes. Users can import the environments into 3D apps to test different settings and lighting.
AI ToyBox of experimental tools, built by NVIDIA Research, that include GET3D, an Omniverse extension that generates trainable 3D models from 2D images, letting developers use their own datasets to rapidly create models for 3D virtual worlds, is now available in the Omniverse Extension Manager.
Thousands of new, free 3D assets are now available worldwide for users to build and create within Omniverse.
