Posts in category: nVidia AI

For more than two decades, NVIDIA has supported graduate students doing GPU-based work through the NVIDIA Graduate Fellowship Program. Today we’re announcing the latest awards of up to $50,000 each to 10 Ph.D. students involved in GPU computing research.

Selected from a highly competitive applicant pool, the awardees will participate in a summer internship preceding the fellowship year. The work they’re doing puts them at the forefront of GPU computing, with fellows tackling projects in deep learning, robotics, computer vision, computer graphics, architecture, circuits, high performance computing, life sciences and programming systems.

“Our fellowship recipients are among the most talented graduate students in the world,” said NVIDIA Chief Scientist Bill Dally. “They’re working on some of the most important problems in computer science, and we’re delighted to support their research.”

The NVIDIA Graduate Fellowship Program is open to applicants worldwide.

Our 2022-2023 fellowship recipients are:

• Davis Rempe, Stanford University — Modeling 3D motion to solve pose estimation, shape reconstruction and motion forecasting, which enables intelligent systems that understand dynamic 3D objects, humans and scenes.
• Hao Chen, University of Texas at Austin — Developing next-generation VLSI physical synthesis tools capable of generating sign-off quality layouts in advanced manufacturing nodes, particularly in analog/mixed-signal circuits.
• Mohit Shridhar, University of Washington — Connecting language to perception and action for vision-based robotics, where representations of vision and language are learned through embodied interactions rather than from static datasets.
• Sai Praveen Bangaru, Massachusetts Institute of Technology — Developing algorithms and compilers for the systematic differentiation of numerical integrators, allowing them to mix seamlessly with machine learning components.
• Shlomi Steinberg, University of California, Santa Barbara — Developing models and computational tools for physical light transport — the computational discipline that studies the simulation of partially coherent light in complex environments.
• Sneha Goenka, Stanford University — Exploring genomic analysis pipelines through hardware-software co-design to enable the ultra-rapid diagnosis of genetic diseases and accelerate large-scale comparative genomic analysis.
• Yufei Ye, Carnegie Mellon University — Building agents that can perceive physical interactions among objects, understand the consequences of interactions with the physical world, and even predict the potential effects of specific interactions.
• Yuke Wang, University of California, Santa Barbara — Exploring novel algorithm- and system-level designs and optimizations to accelerate diverse deep-learning workloads, including deep neural networks and graph neural networks.
• Yuntian Deng, Harvard University — Developing scalable, controllable and interpretable natural language generation approaches using deep generative models with potential applications in long-form text generation.
• Zekun Hao, Cornell University — Developing algorithms that learn from real-world visual data and apply that knowledge to help human creators build photorealistic 3D worlds.

We also acknowledge the 2022-2023 fellowship finalists:

• Enze Xie, University of Hong Kong
• Gokul Swamy, Carnegie Mellon University
• Hong-Xing (Koven) Yu, Stanford University
• Suyeon Choi, Stanford University
• Yash Sharma, University of Tübingen

Step inside an auto assembly plant. See workers ratcheting down nuts to bolts. Hear the whirring of air tools. Watch pristine car bodies gliding along the line and robots rolling up with parts.

Now, fire up its digital twin in 3D online. See animated digital humans at work in the exact same, but digital version of the plant. Drag and drop in robots to move heavy materials, and run simulations for optimizations, taking in real-time factory floor data for improvements. That’s a digital twin.

A digital twin is a virtual representation — a true-to-reality simulation of physics and materials — of a real-world physical asset or system, which is continuously updated.

Digital twins aren’t just for inanimate objects and people. They can be a virtual representation of computer networking architecture used as a sandbox for cyberattack simulations. They can replicate a fulfillment center process to test out human-robot interactions before activating certain robot functions in live environments. The applications are as wide as the imagination.

Digital twins are shaking up operations of businesses. The worldwide market for digital twin platforms is forecast to reach $86 billion by 2028, according to Grand View Research. Its report cites COVID-19 as a catalyst for the adoption of digital twins in specific industries.

What’s Driving Digital Twins? 

The Internet of Things is revving up digital twins.

IoT is helping to enable connected machines and devices to share data with their digital twins and vice versa. That’s because digital twins are always on and up-to-date computer-simulated versions of real-world IoT-connected physical things or processes they represent.

Digital twins are virtual representations that can capture the physics of structures and changing conditions internally and externally, as measured by myriad connected sensors driven by edge computing. They can also run simulations within the virtualizations to test for problems and seek improvements through service updates.

Robotics development and autonomous vehicles are just a couple of the growing number of examples used in digital twins to mimic physical equipment and environments.

“Autonomous vehicles at a very simple level are robots that operate in the open world, striving to avoid contact with anything,” said Rev Lebaredian, vice president of Omniverse and Simulation Technology at NVIDIA. “Eventually we’ll have sophisticated autonomous robots working alongside humans in settings like kitchens — manipulating knives and other dangerous tools. We need digital twins of the worlds they are going to be operating in, so we can teach them safely in the virtual world before transferring their intelligence into the real world.”

Digital Twins in 3D Virtual Environments  

Shared virtual 3D worlds are bringing people together to collaborate on digital twins.

The interactive 3D virtual universe is evident in gaming. Online social games such as Fortnite and the user-generated virtual world of Roblox offer a glimpse of the potential of interactions.

Video conferencing calls in VR, with participants existing as avatars of themselves in a shared virtual conference room, are a step toward realizing the possibilities for the enterprise.

Today the tools exist to develop each of these shared virtual worlds in a shared virtual collaboration platform within this environment.

Omniverse Replicator for Digital Twin Simulations

At GTC, NVIDIA unveiled Omniverse Replicator to help develop digital twins. It’s a  synthetic-data-generation engine that produces physically simulated data for training deep neural networks.

Along with this, the company introduced two implementations of the engine for applications that generate synthetic data: NVIDIA DRIVE Sim, a virtual world for hosting the digital twin of autonomous vehicles, and NVIDIA Isaac Sim, a virtual world for the digital twin of manipulation robots.

Autonomous vehicles and robots developed using this data can master skills across an array of virtual environments before applying them in the real world.

Based on Pixar’s Universal Scene Description and NVIDIA RTX technology, NVIDIA Omniverse is the world’s first scalable, multi-GPU physically accurate world simulation platform.

Omniverse offers users the ability to connect to multiple software ecosystems — including Epic Games Unreal Engine, Reallusion, OnShape, Blender and Adobe — that can assist millions of users.

The reference development platform is modular and can be extended easily. Teams across NVIDIA have enlisted the platform to build core simulation apps such as the previously mentioned NVIDIA Isaac Sim for robotics and synthetic data generation, and NVIDIA DRIVE Sim.

DRIVE Sim enables recreating real-world driving scenarios in a virtual environment to enable testing and development of rare and dangerous use cases.  In addition, because the simulator has a perfect understanding of the ground-truth in any scene, the data from the simulator can be used for training the deep neural networks used in autonomous vehicle perception.

As shown in BMW Group’s factory of the future, Omniverse’s modularity and openness allows it to utilize several other NVIDIA platforms such as the NVIDIA Isaac platform for robotics, NVIDIA Metropolis for intelligent video analytics, and the NVIDIA Aerial software development kit, which brings GPU-accelerated, software-defined 5G wireless radio access networks to environments as well as third-party software for users and companies to continue to use their own tools.

How Are Digital Twins Coming Online?

When building a digital twin and deploying its features, corralling AI resources is necessary.

NVIDIA Base Command Platform enables enterprises to deploy large-scale AI infrastructure. It optimizes resources for users and teams, and it can monitor the workflow from early development to production.

Base Command was developed to help support NVIDIA’s in-house research team with AI resources. It helps manage the available GPU resources, select databases, workspaces and container images available.

It manages the lifecycle of AI development, including workload management and resource sharing, providing both a graphical user interface and a command line interface, and integrated monitoring and reporting dashboards. It delivers the latest NVIDIA updates directly into your AI workflows.

Think of it as the compute engine of AI.

How Are Digital Twins Managed?

NVIDIA Fleet Command provides remote AI management.

Implementing AI from digital twins into the real world requires a deployment platform to handle the updates to the thousands, or even millions, of machines and devices of the edge.

NVIDIA Fleet Command is a cloud-based service accessible from the NVIDIA NGC hub of GPU-accelerated software to securely deploy, manage and scale AI applications across edge-connected systems and devices.

Fleet Command enables fulfillment centers, manufacturing facilities, retailers and many others to remotely implement AI updates.

How Are Digital Twins Advancing?

Digital twins enable the autonomy of things. They can be used to control a physical counterpart autonomously.

An electric vehicle maker, for example, might use a digital twin of a sedan to run simulations on software updates. And when the simulations show improvements to the car’s performance or solve a problem, those software updates can be dispatched over the air to the physical vehicle.

Siemens Energy is creating digital twins to support predictive maintenance of power plants. A digital twin of this scale promises to reduce downtime and help save utility providers an estimated $1.7 billion a year, according to the company.

Passive Logic, a startup based in Salt Lake City, offers an AI platform to engineer and autonomously operate the IoT components of buildings. Its AI engine understands how building components work together, down to the physics, and can run simulations of building systems.

The platform can take in multiple data points and make control decisions to optimize operations autonomously. It compares this optimal control path to actual sensor data, applies machine learning and learns improvements about operating the building over time.

Trains are on a fast track to autonomy as well, and digital twins are being developed to help get there. They’re being used in simulations for features such as automated braking and collision detection systems, enabled by AI run on NVIDIA GPUs.

What Is the History of Digital Twins?

By many accounts, NASA was the first to introduce the notion of the digital twin. While clearly not connected in the Internet of Things way, NASA’s early twin concept and its usage share many similarities with today’s digital twins.

NASA began with the digital twin idea as early as the 1960s. The space agency illustrated its enormous potential in the Apollo 13 moon mission. NASA had set up simulators of systems on the Apollo 13 spacecraft, which could get updates from the real ship in outer space via telecommunications. This allowed NASA engineers to run situation simulations between astronauts and engineers ahead of departure, and it came in handy when things went awry on that mission in 1970.

Engineers on the ground were able to troubleshoot with the astronauts in space, referring to the models on Earth and saving the mission from disaster.

What Types of Digital Twins Are There?

Smart Cities Sims

Smart cities are popping up everywhere. Using video cameras, edge computing and AI, cities are able to understand everything from parking to traffic flow to crime patterns. Urban planners can study the data to help draw up and improve city designs.

Digital twins of smart cities can enable better planning of construction as well as constant improvements in municipalities. Smart cities are building 3D replicas of themselves to run simulations. These digital twins help optimize traffic flow, parking, street lighting and many other aspects to make life better in cities, and these improvements can be implemented in the real world.

Dassault Systèmes has helped build digital twins around the world. In Hong Kong, the company presented examples for a walkability study, using a 3D simulation of the city for visualization.

NVIDIA Metropolis is an application framework, a set of developer tools and a large ecosystem of specialist partners that help developers and service providers better instrument physical space and build smarter infrastructure and spaces through AI-enabled vision. The platform spans AI training to inference, facilitating edge-to-cloud deployment, and it includes enterprise management tools like Fleet Command to better manage fleets of edge nodes.

Earth Simulation Twins 

Digital twins are even being applied to climate modeling.

NVIDIA CEO Jensen Huang disclosed plans to build the world’s most powerful AI supercomputer dedicated to predicting climate change.

Named Earth-2, or E-2, the system would create a digital twin of Earth in Omniverse.

Separately, the European Union has launched Destination Earth, an effort to build a digital simulation of the planet. The plan is to help scientists accurately map climate development as well as extreme weather.

Supporting an EU mandate for achieving climate neutrality by 2050, the digital twin effort would be rendered at one-kilometer scale and based on continuously updated observational data from climate, atmospheric and meteorological sensors. It also plans to take into account measurements of the environmental impacts of human activities.

It is predicted that the Destination Earth digital twin project would require a system with 20,000 GPUs to operate at full scale, according to a paper published in Nature Computational Science. Simulation insights can enable scientists to develop and test scenarios. This can help inform policy decisions and sustainable development planning.

Such work can help assess drought risk, monitor rising sea levels and track changes in the polar regions. It can also be used for planning on food and water issues, and renewable energy such as wind farms and solar plants. The goal is for the main digital modeling platform to be operating by 2023, with the digital twin live by 2027.

Data Center Networking Simulation

Networking is an area where digital twins are reducing downtime for data centers.

Over time, networks have become more complicated. The scale of networks, the number of nodes and the interoperability between components add to their complexity, affecting preproduction and staging operations.

Network digital twins speed up initial deployments by pretesting routing, security, automation and monitoring in simulation. They also enhance ongoing operations, including validating network change requests in simulation, which reduces maintenance times.

Networking operations have also evolved to more advanced capabilities with the use of APIs and automation. And streaming telemetry — think IoT-connected sensors for devices and machines — allows for constant collection of data and analytics on the network for visibility into problems and issues.

The NVIDIA Air infrastructure simulation platform enables network engineers to host digital twins of data center networks.

Ericsson, a maker of telecommunications equipment, is combining decades of radio network simulation expertise with NVIDIA Omniverse Enterprise.

The Stockholm-based company is building city-scale digital twins in NVIDIA Omniverse to help accurately simulate the interplay between 5G cells and the environment to maximize performance and coverage.

Automotive Manufacturing Twins

BMW Group, which has 31 factories around the world, is collaborating with NVIDIA on digital twins. The German automaker is relying on NVIDIA Omniverse Enterprise to run factory simulations to optimize its operations.

Its factories provide more than 100 options for each car, and more than 40 BMW models, offering 2,100 possible configurations of a new vehicle. Some 99 percent of the vehicles produced in BMW factories are custom configurations, which creates challenges for keeping materials stocked on the assembly line.

To help maintain the flow of materials for its factories, BMW Group is also harnessing the NVIDIA Isaac robotics platform to deploy a fleet of robots for logistics to improve the distribution of materials in its production environment. These human-assisting robots, which are put into simulation scenarios with digital humans in pre-production, enable the company to safely test out robot applications on the factory floor of the digital twin before launching into production.

Virtual simulations also enable the company to optimize the assembly line as well as worker  ergonomics and safety. Planning experts from different regions can connect virtually with NVIDIA Omniverse, which lets global 3D design teams work together simultaneously across multiple software suites in a shared virtual space.

NVIDIA Omniverse Enterprise is enabling digital twins for many different industrial applications.

Architecture, Engineering and Construction

Building design teams face a growing demand for efficient collaboration, faster iteration on renderings, and expectations for accurate simulation and photorealism.

These demands can become even more challenging when teams are dispersed worldwide.

Creating digital twins in Omniverse for architects, engineers and construction teams to assess designs together can quicken the pace of development, helping contracts run on time.

Teams on Omniverse can be brought together virtually in a single, interactive platform — even when simultaneously working in different software applications — to rapidly develop architectural models as if they are in the same room and simulate with full physical accuracy and fidelity.

Retail and Fulfillment

Logistics for order fulfillment is a massive industry of moving parts. Fulfillment centers now are aided by robots to help workers avoid injury and boost their efficiency. It’s an environment filled with cameras driven by AI and edge computing to help rapidly pick, pull and pack products. It’s how one-day deliveries arrive at our doors.

The use of digital twins means that much of this can be created in a virtual environment, and simulations can be run to eliminate bottlenecks and other problems.

Kinetic Vision is reinventing intelligent fulfillment and distribution centers with digital twins through digitization and AI. Successfully implementing a network of intelligent stores and fulfillment centers needs robust information, data, and operational technologies to enable innovative edge computing and AI solutions like real-time product recognition. This drives faster, more agile product inspections and order fulfillments.

Energy Industry Twins 

Siemens Energy is relying on the NVIDIA Omniverse platform to create digital twins to support predictive maintenance of power plants.

Using NVIDIA Modulus software frameworks, running on NVIDIA A100 Tensor Core GPUs, Siemens Energy can simulate the corrosive effects of heat, water and other conditions on metal over time to fine-tune maintenance needs.

Hydrocarbon Exploration 

Oil companies face huge risks in seeking to tap new reservoirs or reassess production stage fields with the least financial and environmental downside. Drilling can cost hundreds of millions of dollars. After locating hydrocarbons, these energy companies need to quickly figure out the most profitable strategies for new or ongoing production

Digital twins for reservoir simulations can save many millions of dollars and avoid environmental problems. Using technical software applications, these companies can model how water and hydrocarbons flow under the ground amid wells. This allows them to evaluate potentially problematic situations and virtual production strategies on supercomputers.

Having assessed the risks beforehand, in the digital twins, these exploration companies can minimize losses when committing to new projects. Real-world versions in production can also be optimized for better output based on analytics from their digital doppelganger.

Airport Efficiencies

Digital twins can enable airports to improve customer experiences. For instance, video cameras could monitor the Transportation Security Administration, or TSA, and apply AI to look for ways to analyze bottlenecks at peak hours. Those could be addressed in digital models, and then moved into production to reduce missed flights. Baggage handling video can be assessed to improve ways in the digital environment to ensure luggage arrives on time.

Airplane turnarounds can benefit, too. Many vendors service arriving planes to get them turned around and back on the runway for departures. Video can help airlines track these vendors to ensure timely turnarounds. Digital twins can also analyze the services coordination to optimize workflows before changing things up.

Airlines can then hold their vendors accountable to quickly carrying out services. Caterers, cleaners, refueling, trash and waste removal and other service providers all have what’s known as service-level agreements with airlines to help keep the planes running on time. All of these activities can be run in simulations in the digital world and then applied to scheduling in production for real-world results to help reduce departure delays.

NVIDIA Metropolis helps to process massive amounts of video from the edge so that airports and other industries can analyze operations in real time and derive insights from analytics.

What’s the Future for Digital Twins?

Digital twin simulations have been simmering for half a century. But the past decade’s advances in GPUs, AI and software platforms are heating up their adoption amid this higher-fidelity era of more immersive experiences.    

Increasing penetration of virtual reality and augmented reality will accelerate this work.

Worldwide sales of VR headsets are expected to increase from roughly 7 million in 2021 to more than 28 million in 2025, according to analyst firm IDC.

That’s a lot more headset-connected, content-consuming eyeballs for virtual environments.

And all those in it will be able to access the NVIDIA Omniverse platform for AI, human and robot interactions, and infinite simulations, driving a wild ride of advances from digital twins.

“There has been talk of virtual worlds and digital twins for years. We’re right at the beginning of this transition into reality, much as AI became viable and created an explosion of possibilities,” said NVIDIA’s Lebaredian.

Buckle up for the adventure.

It was the kind of message Connor McCluskey loves to find in his inbox.

As a member of the product innovation team at FirstEnergy Corp. — an electric utility serving 6 million customers from central Ohio to the New Jersey coast — his job is to find technologies that open new revenue streams or cut costs.

In the email, Chris Ricciuti, the founder of Noteworthy AI, explained his ideas for using edge computing to radically improve how utilities track their assets. For FirstEnergy, those assets include tens of millions of devices mounted on millions of poles across more than 269,000 miles of distribution lines.

Bucket Trucks Become Smart Cameras

Ricciuti said his startup aimed to turn every truck in a utility’s fleet into a smart camera that takes pictures of every pole it passes. What’s more, Noteworthy AI’s software would provide the location of the pole, identify the gear on it and help analyze its condition.

“I saw right away that this could be a game changer, so I called him,” said McCluskey.

In the U.S. alone, utilities own 185 million poles. They spend tens, if not hundreds, of millions of dollars a year trying to track the transformers, fuses and other devices on them, as well as the vegetation growing around them.

Utilities typically send out workers each year to manually inspect a fraction of their distribution lines. It’s an inventory that can take a decade, yet the condition of each device is critical to delivering power safely.

5x More Images in 30 Days

In a pilot test last summer, Noteworthy AI showed how edge computing gets better results.

In 30 days, two FirstEnergy trucks, outfitted with the startup’s smart cameras, collected more than 5,000 high-res images of its poles. That expanded the utility’s database more than fivefold.

“People were astounded at what we could do in such a short time frame,” said McClusky.

What’s more, the pictures were of higher quality than those in the utility’s database. That would help eliminate wasted trips when actual line conditions vary from what engineers expect to find.

Use Cases Multiply

News of the pilot program spread to other business units.

A team that inspects FirstEnergy’s 880,000 streetlights and another responsible for tracking vegetation growth around its lines wanted to try the technology. Both saw the value of having more and better data.

So, an expanded pilot is in the works with more trucks over a larger area.

It’s too early to estimate the numbers, but McCluskey “felt right away we could find some significant cost savings with this technology — in a couple years I can imagine its use expanded to all our states,” he said.

An Inside Look at Edge Computing

In a unit the size of a small cake box that attaches to a truck with magnets or suction cups, Noteworthy AI packs two cameras and communications gear. It links to a smaller unit inside the cab that processes the images and AI on an NVIDIA Jetson Xavier NX.

“We developed a pretty sophisticated workflow that runs at the edge on Jetson,” Ricciuti said.

It uses seven AI models. One model looks for poles in images taken at 30 frames/second. When it finds one, it triggers a higher res camera to take bursts of 60-megabyte pictures.

Other models identify gear on the poles and determine which images to send to a database in the cloud.

“We’re doing all this AI compute at the edge on Jetson, so we don’t have to send all the images to the cloud — it’s a huge cost savings,” Ricciuti said.

“With customer use cases growing, we’ll graduate to products like Jetson AGX Orin in the future — NVIDIA has been awesome in computing at the edge,” he added.

Software, Support Speeds Startup

The startup uses NVIDIA TensorRT, code that keeps its AI models trim, so they run fast. It also employs the NVIDIA JetPack SDK with drivers and libraries for computer vision and deep learning as well as ROS, an operating system, now accelerated on Jetson.

In addition, Ricciuti ticks off three benefits from being part of NVIDIA Inception, a program designed to nurture cutting-edge startups.

“When we have engineering questions, we get introduced to technical people who unblock us; we meet potential customers when we’re ready to go to market; and we get computer credits for GPUs in the cloud to train our models,” he said.

AI Spells Digital Transformation

The GPUs, software and support help Ricciuti do the work he loves: finding ways AI can transform legacy practices at large, regulated companies.

“We’re just seeing the tip of the iceberg of what we can do as people are being forced to innovate in the face of problems like climate change, and we’re getting a lot of interest from utilities with large distribution networks,” he said.

Learn more about how NVIDIA is accelerating innovation in the energy industry.

2021 trends are charging into 2022, heralding a new era of autonomous transportation and opening up business models and services never before dreamed of.

In the next year, software-defined compute architectures, electric powertrains, high-fidelity simulation, AI assistants and autonomous trucking solutions are set to transform the transportation industry.

This past year, key technologies saw significant progress, including centralized high-performance compute, data center solutions, simulation and more. These breakthroughs will give rise to even greater innovation next year, ushering in new technology, improving current offerings and accelerating the deployment of safer, more efficient vehicles.

Following Mercedes-Benz’s announcement in 2020 of its upcoming software-defined fleets built on NVIDIA DRIVE Orin, 2021 saw nearly a dozen companies transition their vehicles to the high-performance, centralized compute platform, including NIO, SAIC, Xpeng and more.

Simulation, a crucial component of the autonomous vehicle development pipeline, further narrowed the gap between the virtual and real worlds, using technologies such as NVIDIA Omniverse and synthetic data generation.

And, as the pandemic continued, increased demand for delivery, as well as the worsening driver shortage, invigorated efforts to deploy autonomous trucking solutions.

An Always-New Set of Wheels

AI is transforming the personal vehicle experience. Vehicle development cycles have traditionally lasted around two years, and the end product is fixed with the technology it rolls off the manufacturing line with.

A centralized, software-defined vehicle architecture built on high-performance compute, such as NVIDIA DRIVE Orin, is richly programmable, streamlining development, and can continually improve over time.

In the next year, more automakers will move away from their traditional manufacturing practices and architect vehicles with high-performance compute headroom and full-stack software from the start. As a result, next-gen models can benefit from new apps and features, via over-the-air software updates, so the vehicle gets better and safer over time.

These vehicles will also continue to transition to electric powertrains for intelligent transportation that’s also more sustainable. Automakers have already pledged to increase the share of electric vehicles in their fleets, while newcomers begin to roll out cutting-edge production EVs.

Reality Goes Virtual

Autonomous vehicles are born in the data center, and simulation is a key component of this training and validation process.

In the past, simulation platforms have used gaming engines to generate virtual worlds. However, these engines have serious limitations in accurately recreating the physics and vehicle dynamics of a car driving in the real world.

NVIDIA DRIVE Sim is built on our core technologies, including NVIDIA RTX, Omniverse and AI, to create a true digital twin environment of the world. It uses NVIDIA Omniverse Replicator to generate physically based sensor data for camera, radar, lidar and ultrasonics, along with labeled ground truth data to reduce valuable development time and cost.

The combination of these technologies has significantly narrowed the gap between the virtual and physical worlds, delivering a comprehensive AV training, testing and validation platform. Equipped with DRIVE Sim, AV manufacturers can hit the accelerator on deployment plans in 2022.

Truly Personal Transportation

In addition to high-fidelity AV simulation, NVIDIA Omniverse is paving the way for a seamless intelligent assistant experience.

With NVIDIA DRIVE Concierge, vehicle occupants have access to AI services that are always on, using NVIDIA DRIVE IX and NVIDIA Omniverse Avatar for real-time interactions.

Omniverse Avatar connects speech AI, computer vision, natural language understanding, recommendation engines and simulation. Avatars created on the platform are interactive characters with ray-traced 3D graphics that can see, speak and converse on a wide range of subjects, and understand naturally spoken intent.

The technology of Omniverse Avatar enables DRIVE Concierge to serve as everyone’s digital assistant, providing recommendations and alerts, booking reservations and making phone calls. It’s personalized to each driver and passenger, giving every vehicle occupant their own personal concierge. And with Omniverse Avatar technology, these assistants will have incredible intelligence.

Keep on Trucking

As demand for ecommerce goods and freight continues to grow, the industry is increasingly investing in autonomous trucking solutions.

E-commerce orders increased nearly 60 percent year-over-year in 2020, according to last-mile technology vendor Convey Inc., with more than a third of shoppers opting for same-day delivery. At the same time, the trucking industry is experiencing a 90-percent-plus turnover rate; with the American Trucking Association estimating it will be short 160,000 drivers by 2028.

AI-enabled, highly automated and fully autonomous trucks as well as last-mile delivery vehicles, such as those under development by Volvo Autonomous Solutions, Kodiak Robotics, Embark, TuSimple, Plus, Einride and more, are an essential element of our transportation future.

These next-generation vehicles are built on the high-performance, energy-efficient compute of NVIDIA DRIVE to enhance the safety and quality of life for truck drivers and increase productivity and efficiency.

As the industry continues to adopt these transformative technologies, the next year will see rapid growth toward a truly autonomous future.

In a new evaluation of enterprise AI infrastructure providers, Forrester Research Monday recognized NVIDIA and three of its key partners — Amazon Web Services, Google and Microsoft — as the leaders in AI infrastructure.

The “Forrester Wave: AI Infrastructure, Q4 2021” report states that “​​NVIDIA’s DNA is in every other AI infrastructure solution we evaluated. It’s an understatement to say that NVIDIA GPUs are synonymous with AI infrastructure.”

The report recognizes NVIDIA DGX systems and NVIDIA-accelerated offerings not only from AWS, Google and Microsoft, but also from Dell, Exxact, Hewlett Packard Enterprise, IBM, Inspur, Lambda, Run: AI, Spell and Supermicro.

NVIDIA software, including NVIDIA AI Enterprise, brings full-stack AI development to the broad ecosystem of providers recognized in the Forrester report.

The news comes as multitrillion-dollar industries — from healthcare to finance to retail — are racing to put AI to work and relying on NVIDIA AI infrastructure, including NVIDIA DGX Foundry to accelerate their business transformations.

‘Rocking the World’

“AI is rocking the world,” Forrester’s team wrote.

The firm noted that 74 percent of surveyed global data and analytics decision-makers whose firm is implementing or expanding its use of AI said that adoption has had a positive impact.

“It’s rapidly gone from ‘if’ to ‘when’ to ‘now,’” Forrester reported.

The report from a leading independent research firm signals the arrival of enterprise AI in the mainstream and provides support for even more widespread adoption.

NVIDIA Positioned as a Leader for Its DGX Systems

And that requires flexible, enterprise-grade infrastructure.

“AI platform software that many of these enterprises rely on is all well and good, but it needs hearty infrastructure — compute, storage, networking — to keep AI teams working, not waiting,” Forrester’s team wrote.

NVIDIA’s DGX systems, engineered for enterprise AI workloads, scored the highest in the market presence category and received among the highest scores in the strategy category.

“Breakthroughs in deep learning around 2012 brought AI into focus, but only NVIDIA had the strategy, vision, and roadmap to invest in supporting these now mainstream AI workloads,” the Forrester report noted.

The Forrester report cited NVIDIA’s strengths in “architectural components, throughput, latency, and overall product strategy.”

“The vendor’s sweet spot for its DGX systems are for customers that want a complete system that is engineered by NVIDIA to include its latest component technology for AI workloads,” the Forrester report noted.

NVIDIA AI Available Across the Industry

With NVIDIA AI, customers can choose from any cloud provider and server maker and select from systems located in their data centers or colocation facilities, in the cloud and even on the desktop.
NVIDIA AI infrastructure supports every major cloud service provider. It’s available on the desktop from every major server and system vendor. And NVIDIA technologies can be deployed at the edge by enterprises in autonomous vehicles, robots and embedded systems.

As a result, every vendor mentioned in Forrester’s report is an NVIDIA customer, partner or NVIDIA Inception member, underscoring NVIDIA’s unique role in the industry as a full-stack AI company providing semiconductors, software and systems.

“Reference customers appreciate the vendor’s thought leadership in AI, its frameworks designed to run on NVIDIA GPUs, and having first access to chips coming out of NVIDIA R&D,” the Forrester report stated.

With widespread reports of positive results, universal support for NVIDIA enterprise AI and recognition from one of the most widely respected industry analyst firms, enterprise AI adoption is poised to accelerate.

Enterprises can experience NVIDIA-accelerated AI with NVIDIA AI Enterprise and NVIDIA DGX Foundry through the NVIDIA LaunchPad program, available free of charge in nine regions worldwide.

‘Tis the season for all content creators, especially 3D artists, this month on NVIDIA Studio.

Blender, the world’s most popular open-source 3D creative application, launched a highly anticipated 3.0 release, delivering extraordinary performance gains powered by NVIDIA RTX GPUs, with added Universal Scene Description (USD) support for NVIDIA Omniverse.

Faster 3D creative workflows are made possible by a significant upgrade to Isotropix Clarisse’s redesigned renderer Angie, and Cyberlink’s PowerDirector integration with NVIDIA Broadcast technology — all backed by the December Studio Driver, ready for download.

Plus, NVIDIA Studio and Adobe continue to share the love this holiday season, offering three free months of Adobe Creative Cloud with the purchase of a select NVIDIA Studio laptop or desktop.

And, in the spirit of giving, check out the NVIDIA Studio Facebook, Twitter and Instagram channels all month long for a chance to win a new NVIDIA Studio laptop.

Blender and RTX Render Better Together

Blender 3.0 marks the beginning of a new era for 3D content creation, with new features including a more responsive viewport, reduced shadow artifacts, an asset library to access owned and borrowed assets quickly, and additional customization options.

Blender Cycles renderer has been completely overhauled, maximizing NVIDIA RTX GPU RT Cores for OptiX ray tracing and Tensor Cores for OptiX AI denoising.

This enables rendering nearly 12x and 15x faster than with a MacBook Pro M1 Max or CPU alone, respectively, with the GeForce RTX 3080 laptop GPU.

3D artists can get instant feedback when working in Cycles as the viewport renderer. OptiX support has also been added to model rendering with materials, delivering stunning performance.

Blender 3.0 also adds USD file importing. USD files are the foundation of NVIDIA Omniverse, allowing scenes to be edited by multiple creative applications simultaneously.

The new importer converts USD geometry, lights, cameras, time-sampled animation and preview surface materials to their Blender representations — a massive leap in the Omniverse ecosystem.

Omniverse users can access an alpha build of Blender 3.1, unlocking additional features such as  instancing and added support for USD material exporting, which includes basic NVIDIA Material Definition Language conversions.

To download Omniverse 3.1 alpha, open the launcher and look for the Blender 3.1 build.

Getting started? Check out the Omniverse Showroom app, which gives beginners a look into the foundational technologies of Omniverse, with new content releases over time.

A December (Studio Driver) to Remember

Isotropix Clarisse is a fully interactive computer graphics toolset specialized for set-dressing, look development, lighting and rendering.

The new release, Clarisse 5.5 with Angie, a redesigned renderer, significantly speeds up renders and enables interactive rendering of huge scenes with ray-traced acceleration, exclusively for NVIDIA RTX GPUs.

Clarisse 5.5 with Angie in beta is available for download today.

Cyberlink’s video editing creative app, PowerDirector, has added NVIDIA Broadcast integration and its advanced AI tools, including video denoising to remove grainy noise from video shot in low-light conditions; audio denoising to remove unwanted background noise; and audio dereverb to get rid of room echos.

Now, video and audio enhancements can be done in post-production, adding more flexibility and power to a video editor’s toolbox.

These incredible creator app upgrades — as well as optimizations in Blender 3.0, NVIDIA Omniverse, Isotropix Clarisse, Cyberlink PowerDirector, Blackmagic’s DaVinci Resolve, BorisFX Sapphire, JangaFX Embergen and more — are all supported by the December Studio Driver (472.84), available for download now.

Join the Studio Holiday Fun

NVIDIA Studio is sharing the love in December by way of featurette videos, created by some of the most popular content creators, who discuss their journeys into content creation and share tips on how to get started.

These one-of-a-kind insights into various creative fields will feature giveaways that include the same model of NVIDIA Studio laptops the creators use.

Visit the NVIDIA Studio Facebook, Twitter and Instagram channels to learn more and enter.

Get Studio Support During Winter Break

For new ways to create, including exclusive step-by-step tutorials from industry-leading artists, inspiring community showcases and more, visit the NVIDIA Studio YouTube channel.

This year, we’ve worked with 52 leading creative professionals to launch 137 videos, which feature 75+ RTX-accelerated apps.

The channel’s videos have over 3 million minutes watched and 14,000+ shares, so thank you for your continued support and be sure to subscribe for new weekly videos.

Finally, check out Adobe’s The Great Shoecase contest and customize a model shoe by applying and painting materials in Substance 3D Painter. Win epic prizes including an NVIDIA Studio laptop or NVIDIA SHIELD TV. Submissions end on Thursday, Dec. 16.

Stay up to date on all things Studio by subscribing to the NVIDIA Studio newsletter.

Building AI projects to aid Alzheimer’s patients and monitor pedestrian safety might not be the typical way teens spend their summer.

But that’s what a dozen teens with the Boys & Girls Clubs of Hudson County, in densely packed northeastern New Jersey, did as part of the AI Pathways Institute program.

They spent three weeks brainstorming, coding and traversing spinning robots while using NVIDIA Jetson Nano 2GB developer kits and Jetbot robotics toolkits to create projects that tapped into a multitude of real-world issues. The aim was “to leave our environment and society better than we found it,” said Gail Grant, teen tech coordinator at the nonprofit.

The AI Pathways Institute program in Hudson County is an outgrowth of a successful 2019 pilot program that NVIDIA and the Boys & Girls Clubs of Western Pennsylvania partnered on. Its goals are to introduce high school students to AI and machine learning through a three-week summer camp and provide them with hands-on experience with relevant projects.

NVIDIA and the Western Pennsylvania Clubs announced this spring a new three-year partnership to scale the program to more students through the development of the AI Pathways Toolkit. The toolkit strives to make it easy for other clubs and organizations to deliver hands-on AI and robotics education to youth.

The Hudson County club — which has worked for nearly 130 years to enrich the lives of young people from disadvantaged circumstances — is among the first to adopt it.

Participating students received a cash stipend, new laptop and certificate of achievement from Carnegie Mellon University, which had previously worked with the Western Pennsylvania club. Students who scored well on a post-program exam qualified themselves for future internships.

Emboldening Students in AI

Grant said the program’s goal was to provide students with a broader sense of what AI truly entails. Moreover, she hoped to embolden students to develop and pursue newfound interests in AI and technology by providing speaker events where STEM and AI professionals highlighted their career pathways.

Sekou Sy, a 16-year-old in the AI Pathways program, was surprised to learn about the breadth of robotics — and just how much could be done with AI.

“AI is used in so many objects and ideas that I didn’t think of before,” Sy said. “It’s not just in robots, but other areas like healthcare and the environment, all of which will expand even more in the future.”

Another student, Moureau Tillman (pictured above), was already familiar with AI and the Python programming language, but AI Pathways’ focus on hands-on learning allowed her to dive deeper into AI’s real-world impact.

“Seeing as I already worked with AI, I thought the course was going to be a re-learning experience,” Tillman said. “But I learned a lot of new things through working with robots and coming up with a project that would help people other than myself.”

AI Projects in Action

The program culminated in a presentation in which students explained how they used Python to program NVIDIA Jetson-based projects that would respond to prominent, practical issues.

One project, titled “Forget-Me-Not,” employed AI to help elders and Alzheimer’s patients by alerting them about tasks and appointments, as well as providing medicine or prescription reminders.

“When brainstorming for this project, I thought about what I struggled with myself,” said Sy, who worked on the project with two teammates. “My memory is not so good, and I lose things a lot. We came up with this app where you can put in your data, and it gets to know your daily routines. Then, it reminds you about just about everything you might need.”

Other groups created proposals for projects like “Ecobot,” a robot that roams beaches and discards trash; “Safety First,” which uses AI to monitor streets and call the police when a pedestrian is in danger; and “Fresh,” a device to be placed in users’ cars and monitor outdoor air pollution.

Although not all were able to see their projects to completion, students witnessed how AI could be applied to a variety of problems. Some were even inspired to pursue tech as a career.

“These projects made me think about real-life situations that people are dealing with,” Sy said. “I loved taking it one step further by incorporating the AI aspect, as well as seeing the impact that AI can have in solving these problems. In fact, AI is a field I’m really excited to pursue in the future.”

The Hudson County club plans to reproduce the program early next year. Moving forward, Grant hopes that it’ll be a recurring one that’s implemented throughout the year.

To learn more about entry-level education on AI and robotics, watch the on-demand NVIDIA GTC sessions “Begin Your AI Journey With NVIDIA Jetson Nano” (A31723) and “Getting Started With the Edge AI and Robotics Teaching Kit” (A31535).

Get into the game quicker with the latest GeForce NOW update starting to roll out this GFN Thursday.

Learn more about our latest app update — featuring Ubisoft Connect account linking for faster game launches — now rolling out to members, and the six new games joining the GeForce NOW library.

The update also improves the streaming experience on Mac. So, this GFN Thursday also takes a look at how GeForce NOW transforms Macs into powerful PC gaming rigs.

Streamline Your Sign-In

The newest GeForce NOW app update is rolling out to members on PC and Mac, beginning this week. Version 2.0.36 includes a new feature that links NVIDIA and Ubisoft accounts to enable faster Ubisoft Connect game launches by automatically logging into a user’s account. Skip the sign-in process and stream your favorite Ubisoft games this week.

The update also includes a fix for streaming at the correct aspect ratio on the Apple MacBook Pro M1 Max, as well as improvements to the countdown timer when viewed on the in-game overlay.

PC Gaming on a Mac? Here’s How It Works

It used to be a difficult choice: do you want to be a Mac user or a PC gamer? With GeForce NOW, you can have your Mac and PC game, too.

GeForce NOW transforms nearly any Mac into a high-end gaming rig, thanks to the power of the cloud. NVIDIA data centers do the heavy lifting, rendering games at full quality and streaming them down to Macbook Pro, Macbook Air, iMac and iOS. Get all the benefits of PC gaming, without leaving the Apple ecosystem.

On GeForce NOW, you play the real PC versions of your games, without having to worry if something has been ported to Mac. Software compatibility for the new M1 Mac isn’t a problem either, since the native PC version of games streams straight from the cloud.

GeForce NOW also handles game saves for supported games, so members can play on their Macs, as well as any other supported device, without losing progress.

That means the next time your squad readies up in Apex Legends, you can join the fray from your iMac. Jealous of your friends building their ultimate Viking community in Valheim? With GeForce NOW, you can join them without leaving your Mac. Build your gaming library with weekly free games from the Epic Games Store, with offers like Dead By Daylight.

Plus, GeForce NOW RTX 3080 members can now play at native resolution on their M1 Macbook Air or Macbook Pro, at glorious 1600p. Stream with even longer sessions lengths — up to eight hours. And with RTX ON for both RTX 3080 and Priority members, experience games like Cyberpunk 2077 and Control with real-time ray tracing, without upgrading to a PC.

Playing PC games with GeForce NOW on a Mac is like having your cake and eating it, too.

Yes, Your Mac Can Run Crysis Now

Ready for the ultimate battle on your Mac? For a limited time, get a copy of Crysis Remastered free with select GeForce NOW memberships. Purchase a six-month Priority membership, or the new GeForce NOW RTX 3080 membership, and get a free redeemable code for Crysis Remastered on the Epic Games Store. Terms and conditions apply.

Do Pass Go. Do Collect New Games.

GFN Thursday always means new games coming to the cloud. Six titles are being added to the GeForce NOW library this week, including two day-and-date releases:

• A-Train: All Aboard! Tourism (new game launch on Steam, December 7)
• White Shadows (new game launch on Steam, December 7)
• Monopoly Madness (new game launch on Ubisoft Connect, December 9)
• Anno 1404 History Edition (free on Ubisoft Connect, December 6-14)
• Prison Architect (free on Epic Games Store, December 9-16)
• Super Magbot (Steam)
• Untitled Goose Game (Epic Games Store)

We make every effort to launch games on GeForce NOW as close to their release as possible, but, in some instances, games may not be available immediately.

Also, we had hoped to add both Syberia: The World Before in December. However, these games have shifted their release dates to next year and will be coming to the cloud in the future.

Finally, in case you missed it – Fortnite flipped. Explore new locations, take on enemies with new weapons, and discover what’s new on the Island in Fortnite Chapter 3 – streaming now!

For the members who have experienced the magic of PC gaming on a Mac, we’ve got a question for you. Tell us on Twitter or down below in the comments.

describe how it felt to play a PC game on a Mac for the first time using 𝙤𝙣𝙡𝙮 𝙂𝙄𝙁𝙨

— NVIDIA GeForce NOW (@NVIDIAGFN) December 8, 2021

AI is enabling brighter financial futures for consumers and businesses. From traditional banks to new fintechs, the financial services industry is powering use cases with AI such as preventing payments fraud, automating insurance claims, and accelerating trading strategies.

The latest episode in the I AM AI video series brings these technology stories to life by featuring global financial enterprises and startups transforming banking, insurance and payments.

Automating Insurance Claims and Document Processing

Ping An, China’s largest property and casualty insurer, uses NVIDIA GPU-powered image analysis and AI to rapidly calculate damages caused by vehicle collisions, automate claims handling for simple and clean cases, estimate costs and identify fraudulent claims. This automated experience leads to better customer service, fewer cases of insurance fraud and more efficient delivery of services.

CAPE Analytics, a computer vision startup, is transforming the property insurance industry by analyzing geospatial data to inform more accurate underwriting decisions and mitigate wildfire disasters. The NVIDIA Inception member uses AI to produce detailed data on the vegetation density, roof material and proximity to surrounding structures — more accurately calculating risk and helping homeowners take actions to reduce potential property damage.

Applica, a fintech, deploys progressive AI to streamline text-based workflows that deliver better-than-human performance. Its robotic text automation platform uses NVIDIA GPUs for training machine learning models and inference in production. This eliminates up to 90 percent of manual errors, boosts document turnover rate to less than one second, and reduces physical workforce effort by up to 75 percent.

Banks Adopt AI to Accelerate Model Training and Cut Costs

Bank of Montreal runs complex derivative models to find fair prices for financial contracts used in valuation and risk management. These AI-informed models — trained by Riskfuel, a Toronto-based startup and member of NVIDIA Inception, on 650 million data points and deployed for inference on NVIDIA A100 or T4 Tensor Core GPUs — can drive higher trade flows, generate new risk insights and lead to better product design and selection for Riskfuel’s clients.

Capital One uses Dask and RAPIDS, a suite of GPU-optimized libraries for accelerating data science and analytics pipelines, to achieve 100x improvement in model training times and reduce costs by nearly 98 percent. Its team of data scientists and machine learning engineers use accelerated and distributed data processing for financial and credit analysis.

AI Virtual Assistants Improve the Customer Experience

Square, a global leader in payments, powers its virtual assistant, Square Assistant, using conversational AI to schedule appointments with new and returning customers. These AI models are trained using large hyperparameter jobs running on NVIDIA GPUs in AWS. Once they’re trained and ready for deployment, Square found that inference jobs on large models such as RoBERTa run 10x faster on the AWS GPU service than on CPUs.

Intuit uses conversational AI and intelligent AI assistants to empower financial futures for individuals, self-employed workers and small business owners. The company uses AI technologies, such as knowledge engineering, machine learning and natural language processing and understanding, to provide targeted and personalized assistance with virtual experts, automate financial documents processing, and even forecast cash flow for small businesses.

Funding the Future of Financial Services with AI

NVIDIA’s full-stack accelerated computing platform enables banks, traders, payments providers, insurers and fintechs to deliver enhanced offerings that boost lifetime value for customers and reduce operational costs across their and their customers’ businesses.

Explore NVIDIA solutions for financial services and learn from more industry leaders, such as American Express and PayPal.

With their marbled counters, neoclassical oven alcove and iconic bouquets of spatulas, the “kitchen keynotes” delivered by NVIDIA founder and CEO Jensen Huang during pandemic-era GTCs have been a memorable setting for the highly anticipated events.

The keynotes were initially delivered from his real kitchen, in response to workplace closures. But last spring, the kitchen faded away to reveal a realistic digital replica — one that not only surprised viewers, but also showcased the powerful capabilities of the NVIDIA Omniverse virtual world simulation and collaboration platform.

Now, audiences can get a closer look at all the scenes from the latest virtual kitchen in the Virtual Kitchen Tone Poem.

The project, which launched at GTC in November, is a cinematic homage to the elaborate, painstaking work that went into recreating every detail of the kitchen, from its glistening chrome water fixtures to its earthenware salt cellar.

To accomplish the feat, a team of highly skilled artists collaborated across multiple continents and time zones using Omniverse.

It’s All in the Details

The virtual kitchen got its start during the video shoot for GTC in the fall of 2020, when an onsite crew captured high-resolution images of Huang’s kitchen. The lead environment artist used this footage as the main reference to build a virtual set.

The creative team’s project lead researched detailed references of everything in the kitchen, including the appliance models, oil tins, salt box brands, and even the screws within the cabinets.

A team of eight NVIDIA artists and 10 freelance creators built the cinematic with an Omniverse workflow. In Omniverse, each artist worked within their preferred software, then used Omniverse Connectors to bring all the models and data together, leading to a much smoother animation pipeline and publishing workflow.

The 3D modeling of 57 unique assets and 6,240 total scene objects was done in Autodesk 3ds Max, Autodesk Maya and Pixologic Zbrush. The artists used Adobe Substance Painter and Photoshop for texturing, and the rigging and animation was done in Maya. The team used Nuke for scene composition, while the editing was done with DaVinci Resolve.

Omniverse was where everything converged for lighting and rendering. Omniverse Nucleus acted as the universal exchange and collaboration hub for all the USD-based assets, which helped it all come together. Nucleus facilitated remote access, smart local caching and built-in versioning.

Producing the Virtual Kitchen Tone Poem was also an opportunity to further develop Omniverse Farm — a newly released systems layer that connects multiple computer systems to jointly execute batch operations — and Shot Manager extensions across multiple teams.

With Omniverse Farm, a team of artists can iterate on rendering in an organized, repeatable fashion, bringing flexibility and structure to the rendering process — similar to what animation and visual effects studios would expect.

Omniverse Farm enabled the team to easily batch render 40,000 frames for GTC totaling four terabytes of content, rendered across disparate workstations, on-premises data centers, and cloud servers with a peak of 1,200 GPUs running simultaneously. With the ability to easily contribute a workstation or new server to Farm, the teams could scale to meet their needs.

Visualizing the Future of VFX and Animation

The Virtual Kitchen Tone Poem showcases how it’s possible to have a workflow that’s iterative, scalable and streamlined under short deadlines. These are some of the biggest requirements for artists working in animation and VFX production studios.

Omniverse provided all the tools that enabled the creative team to efficiently render high-quality content for the latest GTC keynote, which shifted across virtual environments, including Huang’s kitchen, a data center, and NVIDIA’s campus in Silicon Valley. NVIDIA technology provided a new level of collaboration for people across the globe that wasn’t available before.

NVIDIA technology also allowed for a non-destructive workflow, which was crucial to a project of this nature and scale, as it helped the team streamline remote and cross-platform collaboration.

The Tone Poem showcases the potential of Omniverse, and how animation and VFX studios can use the platform to enhance workflows, including for production-style projects.

Learn more about NVIDIA Omniverse for professional media & entertainment teams and individual creators.