Editor’s note: This is part of a series profiling researchers advancing science with high performance computing.
Maria Girone is expanding the world’s largest network of scientific computers with accelerated computing and AI.
Since 2002, the Ph.D. in particle physics has worked on a grid of systems across 170 sites in more than 40 countries that support CERN’s Large Hadron Collider (LHC), itself poised for a major upgrade.
A high-luminosity version of the giant accelerator (HL-LHC) will produce 10x more proton collisions, spawning exabytes of data a year. That’s an order of magnitude more than it generated in 2012 when two of its experiments uncovered the Higgs boson, a subatomic particle that validated scientists’ understanding of the universe.
The Call of Geneva
Girone loved science from her earliest days growing up in Southern Italy.
“In college, I wanted to learn about the fundamental forces that govern the universe, so I focused on physics,” she said. “I was drawn to CERN because it’s where people from different parts of the world work together with a common passion for science.”
Tucked between Lake Geneva and the Jura mountains, the European Organization for Nuclear Research is a nexus for more than 12,000 physicists.
A map of CERN and the LHC below it on the French-Swiss border. (Image courtesy of CERN)
Its 27-kilometer ring is sometimes called the world’s fastest racetrack because protons careen around it at 99.9999991% the speed of light. Its superconducting magnets operate near absolute zero, creating collisions that are briefly millions of times hotter than the sun.
Opening the Lab Doors
In 2016, Girone was named CTO of CERN openlab, a group that gathers academic and industry researchers to accelerate innovation and tackle future computing challenges. It works closely with NVIDIA through its collaboration with E4 Computer Engineering, a specialist in HPC and AI based in Italy.
In one of her initial acts, Girone organized the CERN openlab’s first workshop on AI.
Industry participation was strong and enthusiastic about the technology. In their presentations, physicists explained the challenges ahead.
“By the end of the day we realized we were from two different worlds, but people were listening to each other, and enthusiastically coming up with proposals for what to do next,” she said.
A Rising Tide of Physics AI
Today, the number of publications on applying AI across the whole data processing chain in high-energy physics is rising, Girone reports. The work attracts young researchers who see opportunities to solve complex problems with AI, she said.
Meanwhile, researchers are also porting physics software to GPU accelerators and using existing AI programs that run on GPUs.
“This wouldn’t have happened so quickly without the support of NVIDIA working with our researchers to solve problems, answer questions and write articles,” she said. “It’s been extremely important to have people at NVIDIA who appreciate how science needs to evolve in tandem with technology, and how we can make use of acceleration with GPUs.”
“We’re working on experiments on a number of projects like porting to lower power architectures, and we look forward to evaluating the next generation of lower power processors,” she said.
Digital Twins and Quantum Computers
To prepare for the HL-LHC, Girone, named head of CERN openlab in March, seeks new ways to accelerate science with machine learning and accelerated computing. Other tools are on the near and far horizons, too.
The group recently won funding to prototype an engine for building digital twins. It will provide services for physicists, as well as researchers in fields from astronomy to environmental science.
A look inside the accelerator. (Image courtesy of CERN)
CERN also launched a collaboration among academic and industry researchers in quantum computing. The technology could advance science and lead to better quantum systems, too.
A Passion for Diversity
In another act of community-making, Girone was among four co-founders of a Swiss chapter of the Women in HPC group. It will help define specific actions to support women in every phase of their careers.
“I’m passionate about creating diverse teams where everyone feels they contribute and belong — it’s not just a checkbox about numbers, you want to realize a feeling of belonging,” she said.
Girone was among thousands of physicists who captured some of that spirit the day CERN announced the Higgs boson discovery.
She recalls getting up at 4 a.m. to queue for a seat in the main auditorium. It couldn’t hold all the researchers and guests who arrived that day, but the joy of accomplishment followed her and others watching the event from a nearby hall.
“I knew the contribution I made,” she said. “I was proud being among the many authors of the paper, and my parents and my kids felt proud, too.”
The season of hot sun and longer days is here, so stay inside this summer with 20 games joining GeForce NOW in June. Or stream across devices by the pool, from grandma’s house or in the car — whichever way, GeForce NOW has you covered.
Titles from the Age of Empires series are the next Xbox games to roll out to GeForce NOW, giving members plenty to do this summer, especially with over 1,600 games part of the GeForce NOW library.
Expand Your Empire
From the Stone Age to the cloud.
NVIDIA released the first Xbox games to the cloud last month as part of its ongoing partnership with Microsoft. Now it’s the first to bring a smash hit Xbox series to the cloud with Ensemble Studios’ Age of Empires titles.
Since the first release in 1997, Age of Empires has established itself as one of the longest-running real-time strategy (RTS) series in existence. The critically acclaimed RTS series puts players in control of an entire empire with the goal of expanding and evolving to become a flourishing civilization.
All four of the franchise’s latest Steam versions will join GeForce NOW later this month: Age of Empires: Definitive Edition, Age of Empires II: Definitive Edition, Age of Empires III: Definitive Edition and Age of Empires IV: Anniversary Edition. Each title will also support new content and updates, like upcoming seasons or the recently released “Return of Rome” expansion for Age of Empires II: Definitive Edition.
Members will be able to rule from PC, Mac, Chromebooks and more when the Definitive Editions of these games join the GeForce NOW library later this month. Upgrade to Priority membership to skip the waiting lines and experience extended gaming sessions for those long campaigns. Or go for an Ultimate membership to conquer enemies at up to 4K resolution and up to eight-hour sessions.
Game on the Go
Now available in Europe, the Logitech G Cloud gaming handheld supports GeForce NOW, giving gamers a way to stream their PC library from the cloud. It features a seven-inch, full-HD touchscreen with a 60Hz refresh rate and precision controls to stream over 1,600 games in the GeForce NOW library.
Pick up the device and get one month of Priority membership for free to celebrate the launch, from now until Thursday, June 22.
“Look at You, Hacker…”
Welcome to Citadel Station.
Stay cool as a cucumber in the remake of System Shock, the hit game from Nightdive Studios. This game has everything — first-person shooter, role-playing and action-adventure. Fight to survive in the depths of space after a hacker who awakens from a six-month coma finds the space station overrun with hostile mutants and rogue AI. Explore, use hacker skills and unravel the mysteries of the space station streaming System Shock in the cloud.
In addition, members can look for the following two games this week:
Conquerer’s Blade didn’t make it in May, so stay tuned to GFN Thursday for any updates.
Finally, before moving forward into the weekend, lets take things back with our question of the week. Let us know your answer on Twitter or in the comments below.
Neuralangelo, a new AI model by NVIDIA Research for 3D reconstruction using neural networks, turns 2D video clips into detailed 3D structures — generating lifelike virtual replicas of buildings, sculptures and other real-world objects.
Like Michelangelo sculpting stunning, life-like visions from blocks of marble, Neuralangelo generates 3D structures with intricate details and textures. Creative professionals can then import these 3D objects into design applications, editing them further for use in art, video game development, robotics and industrial digital twins.
Neuralangelo’s ability to translate the textures of complex materials — including roof shingles, panes of glass and smooth marble — from 2D videos to 3D assets significantly surpasses prior methods. The high fidelity makes its 3D reconstructions easier for developers and creative professionals to rapidly create usable virtual objects for their projects using footage captured by smartphones.
“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,” said Ming-Yu Liu, senior director of research and co-author on the paper. “This tool will eventually enable developers to import detailed objects — whether small statues or massive buildings — into virtual environments for video games or industrial digital twins.”
In a demo, NVIDIA researchers showcased how the model could recreate objects as iconic as Michelangelo’s David and as commonplace as a flatbed truck. Neuralangelo can also reconstruct building interiors and exteriors — demonstrated with a detailed 3D model of the park at NVIDIA’s Bay Area campus.
Neural Rendering Model Sees in 3D
Prior AI models to reconstruct 3D scenes have struggled to accurately capture repetitive texture patterns, homogenous colors and strong color variations. Neuralangelo adopts instant neural graphics primitives, the technology behind NVIDIA Instant NeRF, to help capture these finer details.
Using a 2D video of an object or scene filmed from various angles, the model selects several frames that capture different viewpoints — like an artist considering a subject from multiple sides to get a sense of depth, size and shape.
Once it’s determined the camera position of each frame, Neuralangelo’s AI creates a rough 3D representation of the scene, like a sculptor starting to chisel the subject’s shape.
The model then optimizes the render to sharpen the details, just as a sculptor painstakingly hews stone to mimic the texture of fabric or a human figure.
The final result is a 3D object or large-scale scene that can be used in virtual reality applications, digital twins or robotics development.
Find NVIDIA Research at CVPR, June 18-22
Neuralangelo is one of nearly 30 projects by NVIDIA Research to be presented at the Conference on Computer Vision and Pattern Recognition (CVPR), taking place June 18-22 in Vancouver. The papers span topics including pose estimation, 3D reconstruction and video generation.
One of these projects, DiffCollage, is a diffusion method that creates large-scale content — including long landscape orientation, 360-degree panorama and looped-motion images. When fed a training dataset of images with a standard aspect ratio, DiffCollage treats these smaller images as sections of a larger visual — like pieces of a collage. This enables diffusion models to generate cohesive-looking large content without being trained on images of the same scale.
The technique can also transform text prompts into video sequences, demonstrated using a pretrained diffusion model that captures human motion:
Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows. We’re also deep diving on new GeForce RTX 40 Series GPU features, technologies and resources, and how they dramatically accelerate content creation.
New 14-inch NVIDIA Studio laptops, equipped with GeForce RTX 40 Series Laptop GPUs, give creators peak portability with a significant increase in performance over the last generation. AI-dedicated hardware called Tensor Cores power time-saving tasks in popular apps like Davinci Resolve. Ray Tracing Cores together with our neural rendering technology, DLSS 3, boost performance in real-time 3D rendering applications like D5 Render and NVIDIA Omniverse.
NVIDIA also introduced a new method for accelerating video encoding. Simultaneous Scene Encoding sends independent groups of frames, or scenes, to each NVIDIA Encoder (NVENC). With multiple NVENCs fully utilized, video export times can be reduced significantly, without affecting image quality. The first software to integrate the technology is the popular video editing app CapCut.
The May Studio Driver is ready for download now. This month’s release includes support for updates to MAGIX VEGAS Pro, D5 Render and VLC Media Player — in addition to CapCut — plus AI model optimizations for popular apps.
COMPUTEX, Asia’s biggest annual tech trade show, kicks off a flurry of updates, bringing creators new tools and performance from the NVIDIA Studio platform — and plenty of AI power.
During his keynote address at COMPUTEX, NVIDIA founder and CEO Jensen Huang introduced a new generative AI to support game development, NVIDIA Avatar Cloud Engine (ACE) for Games. The platform adds intelligence to non-playable characters (NPCs) in gaming, with AI-powered natural language interactions.
The Kairos demo — a joint venture with Convai led by NVIDIA Creative Director Gabriele Leone — demonstrates how a single model can transform into a living, breathing, lifelike character this week In the NVIDIA Studio.
Ultraportable, Ultimate Performance
NVIDIA Studio laptops, powered by the NVIDIA Ada Lovelace architecture, are the world’s fastest laptops for creating and gaming.
For the first time, GeForce RTX performance comes to 14-inch devices. In the process, it’s transforming the ultraportable market, delivering the ultimate combination of performance and portability.
ASUS Zenbook Pro 14 comes with up to a GeForce RTX 4070 Laptop GPU.
These purpose-built creative powerhouses do it all. Backed by NVIDIA Studio, the platform supercharges over 110 creative apps, provides lasting stability with NVIDIA Studio Drivers and includes a powerful suite of AI-powered Studio software, such as NVIDIA Omniverse, Canvas and Broadcast.
Fifth-generation Max-Q technologies bring an advanced suite of AI-powered technologies that optimize laptop performance, power and acoustics for peak efficiency. Battery life improves by up to 70%. And DLSS is now optimized for laptops, giving creators incredible 3D rendering performance with DLSS 3 optical multi-frame generation and super resolution in Omniverse and D5 Render, and in hit games like Cyberpunk 2077.
As the ultraportable market heats up, PC laptop makers are giving creators more options than ever. Recently announced models, with more on the way, include the Acer Swift X 14, ASUS Zenbook Pro 14, GIGABYTE Aero 14, Lenovo’s Slim Pro 9i 14 and MSI Stealth 14.
Visit the Studio Shop for the latest GeForce RTX-powered NVIDIA Studio systems and explore the range of high-performance Studio products.
Simultaneous Scene Encoding
The recent release of Video Codec SDK 12.1 added support for multi-encoder support, which can cut export times in half. Our previously announced split encoding method — which splits a frame and sends each section to an encoder — now has an API that app developers can expose to their end users. Previously, split encoding would be engaged automatically for 4K or higher video and the faster export presets. With this update, developers can simply allow users to toggle on this option.
Video Codec SDK 12.1 also introduces a new encoding method: simultaneous scene encoding. Video apps can split groups of pictures or scenes as they’re sent into the rendering pipeline. Each group can then be rendered independently and ordered properly on the final output.
The result is a significant increase in encoding speed — approximately 80% for dual encoders, and further increases when more than two NVENCs are present, like in the NVIDIA RTX 6000 Ada Generation professional GPU. Image quality is also improved compared to current split encoding methods, where individual frames are sent to each encoder and then stitched back together in the final output.
CapCut users will be the first to experience this benefit on RTX GPUs with two or more encoders, starting with the software’s current release, available today.
Massive May Studio Driver Drops
The May Studio Driver features significant upgrades and optimizations.
MAGIX partnered with NVIDIA to move its line of VEGAS Pro AI models on WinML, enabling video editors to apply AI effects much faster.
The driver also optimizes AI features for applications running on WinML, including Adobe Photoshop, Lightroom, MAGIX Vegas Pro, ON1 and DxO, among many others.
The real-time ray tracing renderer D5 Render also added NVIDIA DLSS 3, delivering a smoother viewport experience to navigate scenes with super fluid motion, massively benefiting architects, designers, interior designers and all professional 3D artists.
D5 Render and DLSS 3 work brilliantly to create photorealistic imagery.
NVIDIA RTX Video Super Resolution — video upscaling technology that uses AI and RTX Tensor Cores to upscale video quality — is now fully integrated into VLC Media Player, no longer requiring a separate download. Learn more.
During NVIDIA founder and CEO Jensen Huang’s keynote address at COMPUTEX, he introduced NVIDIA ACE for Games, a new foundry that adds intelligence to NPCs in gaming with AI-powered natural language interactions.
Game developers and studios can use ACE for Games to build and deploy customized speech, conversation and animation AI models in their software and games. The AI technology can transform entire worlds, breathing new life into individuals, groups or an entire town’s worth of characters — the sky’s the limit.
ACE for Games builds on technology inside NVIDIA Omniverse, an open development platform for building and operating metaverse applications, including optimized AI foundation models for speech, conversation and character animation.
This includes the NVIDIA NeMo for conversational AI fine-tuned for game characters, NVIDIA Riva for automatic speech recognition and text-to-speech, and Omniverse Audio2Face for instantly creating expressive facial animation of game characters to match any speech tracks. Audio2Face features Omniverse connectors for Unreal Engine 5, so developers can add facial animation directly to MetaHuman characters.
Seeing Is Believing: Kairos Demo
Huang debuted for COMPUTEX attendees ACE for Games — and provided a sneak-peek of the future of gaming — in a demo dubbed Kairos.
Convai, an NVIDIA Inception startup, specializes in cutting-edge conversational AI for virtual game worlds. NVIDIA Lightspeed Studios, led by Creative Director and 3D artist Gabriele Leone, built the remarkably realistic scene and demo. Together, they’ve showcased the opportunity developers have to use NVIDIA ACE for Games to build NPCs.
In the demo, players interact with Jin, owner and proprietor of a ramen shop. The photorealistic shop was modeled after the virtual ramen shop built in NVIDIA Omniverse.
For this, an NVIDIA artist traveled to a real ramen restaurant in Tokyo and collected over 2,000 high-resolution reference images and videos. Each captured aspects from the kitchen’s distinct areas for cooking, cleaning, food preparation and storage. “We probably used 70% of the existing models, 30% new and 80% retextures,” said Leone.
Kairos: Beautifully rendered in Autodesk Maya, Blender, Unreal Engine 5 and NVIDIA Omniverse.
In the digital ramen shop, objects were modeled in Autodesk 3ds Max with RTX-accelerated AI denoising, and Blender benefiting from RTX-accelerated OptiX ray tracing for smooth, interactive movement in the viewport — all powered by the team’s arsenal of GeForce RTX 40 Series GPUs.
“It’s fair to say that without GeForce RTX GPUs and Omniverse, this project would’ve been impossible to complete without adding considerable time” — Gabriele Leone
The texture phase in Adobe Substance 3D Painter used NVIDIA Iray rendering technology with RTX-accelerated light and ambient occlusion, baking large assets in mere moments.
Next, Omniverse and the Audio2Face app, via the Unreal Engine 5 Connector, allowed the team to add facial animation and audio directly to the ramen shop NPC.
Although he is an NPC, Jin replies to natural language realistically and consistent with the narrative backstory — all with the help of generative AI.
Lighting and animation work was done in Unreal Engine 5 aided by NVIDIA DLSS using AI to upscale frames rendered at lower resolution while still retaining high-fidelity detail, again increasing interactivity in the viewport for the team.
Direct your ramen order to the NPC, ahem, interactive, conversational character.
Suddenly, NPCs just got a whole lot more engaging. And they’ve never looked this good.
MediaTek, a leading innovator in connectivity and multimedia, is teaming with NVIDIA to bring drivers and passengers new experiences inside the car.
The partnership was announced today at a COMPUTEX press conference with MediaTek CEO Rick Tsai and NVIDIA founder and CEO Jensen Huang.
“NVIDIA is a world-renowned pioneer and industry leader in AI and computing. With this partnership, our collaborative vision is to provide a global one-stop shop for the automotive industry, designing the next generation of intelligent, always-connected vehicles,” said Tsai. “Through this special collaboration with NVIDIA, we will together be able to offer a truly unique platform for the compute-intensive, software-defined vehicle of the future.”
“AI and accelerated computing are fueling the transformation of the entire auto industry,” said Huang. “The combination of MediaTek’s industry-leading system-on-chip plus NVIDIA’s GPU and AI software technologies will enable new user experiences, enhanced safety and new connected services for all vehicle segments, from luxury to entry-level.”
A Collaboration to Transform Automotive
The partnership combines the best competencies of each company to deliver the most compelling solutions for the next generation of connected vehicles.
Today, NVIDIA offers GPUs for laptops, desktops, workstations and servers, along with systems-on-chips (SoCs) for automotive and robotics applications. With this new GPU chiplet, NVIDIA can extend its GPU and accelerated compute leadership across broader markets.
MediaTek will develop automotive SoCs and integrate the NVIDIA GPU chiplet, featuring NVIDIA AI and graphics intellectual property, into the design architecture. The chiplets are connected by an ultra-fast and coherent chiplet interconnect technology.
In addition, MediaTek will run the NVIDIA DRIVE OS, DRIVE IX, CUDA and TensorRT software technologies on these new automotive SoCs to enable connected infotainment and in-cabin convenience and safety functions. This partnership makes more in-vehicle infotainment options available to automakers on the NVIDIA DRIVE platform.
MediaTek will develop automotive SoCs integrating NVIDIA GPU chiplet. Image courtesy of MediaTek.
By tapping NVIDIA’s core expertise in AI, cloud, graphics technology and software ecosystem, and pairing it with NVIDIA advanced driver assistance systems, MediaTek can bolster the capabilities of its Dimensity Auto platform.
A Rich Heritage of Innovation
This collaboration empowers MediaTek’s automotive customers to offer cutting-edge NVIDIA RTX graphics and advanced AI capabilities, plus safety and security features enabled by NVIDIA DRIVE software, for all types of vehicles. According to Gartner, the infotainment and instrument cluster SoCs used within vehicles is projected to reach $12 billion in 2023.*
MediaTek’s Dimensity Auto platform draws on its decades of experience in mobile computing, high-speed connectivity, entertainment and extensive Android ecosystem. The platform includes the Dimensity Auto Cockpit, which supports smart multi-displays, high-dynamic range cameras and audio processing, so drivers and passengers can seamlessly interact with cockpit and infotainment systems.
For well over a decade, automakers have been turning to NVIDIA to help modernize their vehicle cockpits, using its technology for infotainment systems, graphical user interfaces and touchscreens.
By integrating the NVIDIA GPU chiplet into its automotive offering, MediaTek aims to enhance the performance capabilities of its Dimensity Auto platform to deliver the most advanced in-cabin experience available in the market. The platform also includes Auto Connect, a feature that will ensure drivers remain wirelessly connected with high-speed telematics and Wi-Fi networking.
With today’s announcement, MediaTek aims to raise the bar even higher for its automotive offerings — delivering intelligent, connected in-cabin solutions that cater to the evolving needs and demands of customers, while providing a safe, secure and enjoyable experience in the car.
*Gartner, Forecast Analysis: Automotive Semiconductors, Worldwide, 2021-2031; Table 1 – Automotive Semiconductor Forecast by Application (Billions of U.S. Dollars), January 18, 2023. Calculation performed by NVIDIA based on Gartner research.
In his first live keynote since the pandemic, NVIDIA founder and CEO Jensen Huang today kicked off the COMPUTEX conference in Taipei, announcing platforms that companies can use to ride a historic wave of generative AI that’s transforming industries from advertising to manufacturing to telecom.
“We’re back,” Huang roared as he took the stage after years of virtual keynotes, some from his home kitchen. “I haven’t given a public speech in almost four years — wish me luck!”
Speaking for nearly two hours to a packed house of some 3,500, he described accelerated computing services, software and systems that are enabling new business models and making current ones more efficient.
“Accelerated computing and AI mark a reinvention of computing,” said Huang, whose travels in his hometown over the past week have been tracked daily by local media.
In a demonstration of its power, he used the massive 8K wall he spoke in front of to show a text prompt generating a theme song for his keynote, singable as any karaoke tune. Huang, who occasionally bantered with the crowd in his native Taiwanese, briefly led the audience in singing the new anthem.
“We’re now at the tipping point of a new computing era with accelerated computing and AI that’s been embraced by almost every computing and cloud company in the world,” he said, noting 40,000 large companies and 15,000 startups now use NVIDIA technologies with 25 million downloads of CUDA software last year alone.
The DGX GH200 packs an exaflop of performance and 144 terabytes of shared memory, nearly 500x more than in a single NVIDIA DGX A100 320GB system. That lets developers build large language models for generative AI chatbots, complex algorithms for recommender systems, and graph neural networks used for fraud detection and data analytics.
Google Cloud, Meta and Microsoft are among the first expected to gain access to the DGX GH200, which can be used as a blueprint for future hyperscale generative AI infrastructure.
NVIDIA’s DGX GH200 AI supercomputer delivers 1 exaflop of performance for generative AI.
“DGX GH200 AI supercomputers integrate NVIDIA’s most advanced accelerated computing and networking technologies to expand the frontier of AI,” Huang told the audience in Taipei, many of whom had lined up outside the hall for hours before the doors opened.
NVIDIA is building its own massive AI supercomputer, NVIDIA Helios, coming online this year. It will use four DGX GH200 systems linked with NVIDIA Quantum-2 InfiniBand networking to supercharge data throughput for training large AI models.
The DGX GH200 forms the pinnacle of hundreds of systems announced at the event. Together, they’re bringing generative AI and accelerated computing to millions of users.
Zooming out to the big picture, Huang announced more than 400 system configurations are coming to market powered by NVIDIA’s latest Hopper, Grace, Ada Lovelace and BlueField architectures. They aim to tackle the most complex challenges in AI, data science and high performance computing.
Acceleration in Every Size
To fit the needs of data centers of every size, Huang announced NVIDIA MGX, a modular reference architecture for creating accelerated servers. System makers will use it to quickly and cost-effectively build more than a hundred different server configurations to suit a wide range of AI, HPC and NVIDIA Omniverse applications.
MGX lets manufacturers build CPU and accelerated servers using a common architecture and modular components. It supports NVIDIA’s full line of GPUs, CPUs, data processing units (DPUs) and network adapters as well as x86 and Arm processors across a variety of air- and liquid-cooled chassis.
QCT and Supermicro will be the first to market with MGX designs appearing in August. Supermicro’s ARS-221GL-NR system announced at COMPUTEX will use the Grace CPU, while QCT’s S74G-2U system, also announced at the event, uses Grace Hopper.
ASRock Rack, ASUS, GIGABYTE and Pegatron will also use MGX to create next-generation accelerated computers.
5G/6G Calls for Grace Hopper
Separately, Huang said NVIDIA is helping shape future 5G and 6G wireless and video communications. A demo showed how AI running on Grace Hopper will transform today’s 2D video calls into more lifelike 3D experiences, providing an amazing sense of presence.
Laying the groundwork for new kinds of services, Huang announced NVIDIA is working with telecom giant SoftBank to build a distributed network of data centers in Japan. It will deliver 5G services and generative AI applications on a common cloud platform.
The data centers will use NVIDIA GH200 Superchips and NVIDIA BlueField-3 DPUs in modular MGX systems as well as NVIDIA Spectrum Ethernet switches to deliver the highly precise timing the 5G protocol requires. The platform will reduce cost by increasing spectral efficiency while reducing energy consumption.
The systems will help SoftBank explore 5G applications in autonomous driving, AI factories, augmented and virtual reality, computer vision and digital twins. Future uses could even include 3D video conferencing and holographic communications.
Turbocharging Cloud Networks
Separately, Huang unveiled NVIDIA Spectrum-X, a networking platform purpose-built to improve the performance and efficiency of Ethernet-based AI clouds. It combines Spectrum-4 Ethernet switches with BlueField-3 DPUs and software to deliver 1.7x gains in AI performance and power efficiency over traditional Ethernet fabrics.
NVIDIA Spectrum-X, Spectrum-4 switches and BlueField-3 DPUs are available now from system makers including Dell Technologies, Lenovo and Supermicro.
NVIDIA Spectrum-X accelerates AI workflows that can experience performance losses on traditional Ethernet networks.
Bringing Game Characters to Life
Generative AI impacts how people play, too.
Huang announced NVIDIA Avatar Cloud Engine (ACE) for Games, a foundry service developers can use to build and deploy custom AI models for speech, conversation and animation. It will give non-playable characters conversational skills so they can respond to questions with lifelike personalities that evolve.
New and enhanced tools, frameworks and drivers are making it easier for PC developers to develop and deploy AI. For example, the Microsoft Olive toolchain for optimizing and deploying GPU-accelerated AI models and new graphics drivers will boost DirectML performance on Windows PCs with NVIDIA GPUs.
The collaboration will enhance and extend an installed base of 100 million PCs sporting RTX GPUs with Tensor Cores that boost performance of more than 400 AI-accelerated Windows apps and games.
Digitizing the World’s Largest Industries
Generative AI is also spawning new opportunities in the $700 billion digital advertising industry.
For example, WPP, the world’s largest marketing services organization, is working with NVIDIA to build a first-of-its kind generative AI-enabled content engine on Omniverse Cloud.
In a demo, Huang showed how creative teams will connect their 3D design tools such as Adobe Substance 3D, to build digital twins of client products in NVIDIA Omniverse. Then, content from generative AI tools trained on responsibly sourced data and built with NVIDIA Picasso will let them quickly produce virtual sets. WPP clients can then use the complete scene to generate a host of ads, videos and 3D experiences for global markets and users to experience on any web device.
“Today ads are retrieved, but in the future when you engage information much of it will be generated — the computing model has changed,” Huang said.
Factories Forge an AI Future
With an estimated 10 million factories, the $46 trillion manufacturing sector is a rich field for industrial digitalization.
“The world’s largest industries make physical things. Building them digitally first can save billions,” said Huang.
The keynote showed how electronics makers including Foxconn Industrial Internet, Innodisk, Pegatron, Quanta and Wistron are forging digital workflows with NVIDIA technologies to realize the vision of an entirely digital smart factory.
They’re using Omniverse and generative AI APIs to connect their design and manufacturing tools so they can build digital twins of factories. In addition, they use NVIDIA Isaac Sim for simulating and testing robots and NVIDIA Metropolis, a vision AI framework, for automated optical inspection.
The latest component, NVIDIA Metropolis for Factories, can create custom quality-control systems, giving manufacturers a competitive advantage. It’s helping companies develop state-of-the-art AI applications.
AI Speeds Assembly Lines
For example, Pegatron — which makes 300 products worldwide, including laptops and smartphones — is creating virtual factories with Omniverse, Isaac Sim and Metropolis. That lets it try out processes in a simulated environment, saving time and cost.
Foxconn Industrial Internet, a service arm of the world’s largest technology manufacturer, is working with NVIDIA Metropolis partners to automate significant portions of its circuit-board quality-assurance inspection points.
Crowds lined up for the keynote hours before doors opened.
In a video, Huang showed how Techman Robot, a subsidiary of Quanta, tapped NVIDIA Isaac Sim to optimize inspection on the Taiwan-based giant’s manufacturing lines. It’s essentially using simulated robots to train robots how to make better robots.
In addition, Huang announced a new platform to enable the next generation of autonomous mobile robot (AMR) fleets. Isaac AMR helps simulate, deploy and manage fleets of autonomous mobile robots.
A large partner ecosystem — including ADLINK, Aetina, Deloitte, Quantiphi and Siemens — is helping bring all these manufacturing solutions to market, Huang said.
It’s one more example of how NVIDIA is helping companies feel the benefits of generative AI with accelerated computing.
“It’s been a long time since I’ve seen you, so I had a lot to tell you,” he said after the two-hour talk to enthusiastic applause.
As mobile robot shipments surge to meet the growing demands of industries seeking operational efficiencies, NVIDIA is launching a new platform to enable the next generation of autonomous mobile robot (AMR) fleets.
Isaac AMR brings advanced mapping, autonomy and simulation to mobile robots and will soon be available for early customers, NVIDIA founder and CEO Jensen Huang announced during his keynote address at the COMPUTEX technology conference in Taipei.
Isaac AMR is a platform to simulate, validate, deploy, optimize and manage fleets of autonomous mobile robots. It includes edge-to-cloud software services, computing and a set of reference sensors and robot hardware to accelerate development and deployment of AMRs, reducing costs and time to market.
Mobile robot shipments are expected to climb from 251,000 units in 2023 to 1.6 million by 2028, with revenue forecast to jump from $12.6 billion to $64.5 billion in the period, according to ABI Research.
Simplifying the Path to Autonomy
Despite the explosive adoption of robots, the intralogistics industry faces challenges.
Traditionally, software applications for autonomous navigation are often coded from scratch for each robot, making rolling out autonomy across different robots complex. Also, warehouses, factories and fulfillment centers are enormous, frequently running a million square feet or more, making them hard to map for robots and keep updated. And integrating AMRs into existing workflows, fleet management and warehouse management systems can be complicated.
For those working in advanced robotics and seeking to migrate traditional forklifts or automated guided vehicles to fully autonomous mobile robots, Isaac AMR provides the blueprint to accelerate the migration to full autonomy, reducing costs and speeding deployment of state-of-the-art AMRs.
Orin-Based Reference Architecture
Isaac AMR is built on the foundations of the NVIDIA Nova Orin reference architecture.
Nova Orin is the brains and eyes of Isaac AMR. It integrates multiple sensors including stereo cameras, fisheye cameras, 2D and 3D lidars with the powerful NVIDIA Jetson AGX Orin system-on-module. The reference robot hardware comes with Nova Orin pre-integrated, making it easy for developers to evaluate Isaac AMR in their own environments.
The compute engine of Nova is Orin, which delivers access to some of the most advanced AI and hardware-accelerated algorithms that can be run using 275 tera operations per second (TOPS) of edge computing in real time.
The synchronized and calibrated sensor suite offers sensor diversity and redundancy for real-time 3D perception and mapping. Cloud-native tools for record, upload and replay enable easy debugging, map creation, training and analytics.
Isaac AMR: Mapping, Autonomy, Simulation
Isaac AMR offers a foundation for mapping, autonomy and simulation.
Isaac AMR accelerates mapping and semantic understanding of large environments by tying into DeepMap’s cloud-based service to help accelerate robot mapping of large facilities from weeks to days, offering centimeter-level accuracy without the need for a highly skilled team of technicians. It can generate rich 3D voxel maps, which can be used to create occupancy maps and semantic maps for multiple types of AMRs.
Additionally, Isaac AMR shortens the time to develop and deploy robots in large, highly dynamic and unstructured environments with autonomy that’s enabled by multimodal navigation with cloud-based fleet optimization using NVIDIA cuOpt software.
An accelerated and modular framework enables real-time camera and lidar perception. Planning and control using advanced path planners, behavior planners and use of semantic information make the robot operate autonomously in complex environments. A low-code, no-code interface makes it easy to rapidly develop and customize applications for different scenarios and use cases.
Finally, Isaac AMR simplifies robot operations by tapping into physics-based simulation from Isaac Sim, powered by NVIDIA Omniverse, an open development platform for industrial digitalization. This can bring digital twins to life, so the robot application can be developed, tested and customized for each customer before deploying in the physical world. This significantly reduces the operational cost and complexity of deploying AMRs.
How do you help robots build better robots? By simulating even more robots.
NVIDIA founder and CEO Jensen Huang today showcased how leading electronics manufacturer Quanta is using AI-enabled robots to inspect the quality of its products.
For example, robots from Quanta subsidiary Techman Robot tapped NVIDIA Isaac Sim — a robotics simulation application built on NVIDIA Omniverse — to develop a custom digital twin application to improve inspection on the Taiwan-based electronics provider’s manufacturing line.
The below demo shows how Techman uses Isaac Sim to optimize the inspection of robots by robots on the manufacturing line. In effect, it’s robots building robots.
Automated optical inspection, or AOI, helps manufacturers more quickly identify defects and deliver high-quality products to their customers around the globe. The NVIDIA Metropolis vision AI framework, now enabled for AOI, is also used to optimize inspection workflows for products ranging from automobiles to circuit boards.
Techman developed AOI with its factory-floor robots by using Isaac Sim to simulate, test and optimize its state-of-the-art collaborative robots, or cobots, while using NVIDIA AI and GPUs for training in the cloud and inference on the robots themselves.
Isaac Sim is built on NVIDIA Omniverse — an open development platform for building and operating industrial metaverse applications.
Unique features of Techman’s robotic AOI solutions include their placement of the inspection camera directly on articulated robotic arms and GPUs integrated in the robot controller.
This allows the bots to inspect areas of products that fixed cameras simply can’t access, as well as use AI at the edge to instantly detect defects.
“The distinctive features of Techman’s robots — compared to other robot brands — lie in their built-in vision system and AI inference engine,” said Scott Huang, chief operations officer at Techman. “NVIDIA RTX GPUs power up their AI performance.”
But programming the movement of these robots can be time consuming.
A developer has to determine the precise arm positions, as well as the most efficient sequence, to capture potentially hundreds of images as quickly as possible.
This can involve several days of effort, exploring tens of thousands of possibilities to determine an optimal solution.
The solution: robot simulation.
Using Omniverse, Techman built a digital twin of the inspection robot — as well as the product to be inspected — in Isaac Sim.
Programming the robot in simulation reduced time spent on the task by over 70%, compared to programming manually on the real robot. Using an accurate 3D model of the product, the application can be developed in the digital twin even before the real product is manufactured, saving valuable time on the production line.
Then, with powerful optimization tools in Isaac Sim, Techman explored a massive number of program options in parallel on NVIDIA GPUs.
The end result was an efficient solution that reduced the cycle time of each inspection by 20%, according to Huang.
Every second saved in inspection time will drop down to the bottom line of Techman’s manufacturing customers.
Gathering and labeling real-world images of defects is costly and time consuming, so Techman turned to synthetic data to improve the quality of inspections. It used the Omniverse Replicator framework to quickly generate high-quality synthetic datasets.
These perfectly labeled images are used to train the AI models in the cloud and dramatically enhance their performance.
And dozens of AI models can be run at the edge — efficiently and with low latency thanks to NVIDIA technology — while inspecting particularly complicated products, some of which take more than 40 models to scrutinize their different aspects.
Learn more about how Isaac Sim on Omniverse, Metropolis and AI are streamlining the optical inspection process across products and industries by joining NVIDIA at COMPUTEX, where the Techman cobots will be on display.
The $46 trillion global electronics manufacturing industry spans more than 10 million factories worldwide, where much is at stake in producing defect-free products. To drive product excellence, leading electronics manufacturers are adopting NVIDIA Metropolis for Factories.
More than 50 manufacturing giants and industrial automation providers — including Foxconn Industrial Internet, Pegatron, Quanta, Siemens and Wistron — are implementing Metropolis for Factories, NVIDIA founder and CEO Jensen Huang announced during his keynote address at the COMPUTEX technology conference in Taipei.
NVIDIA Metropolis for Factories is a collection of factory automation workflows that enables industrial technology companies and manufacturers to develop, deploy and manage customized quality-control systems that offer a competitive advantage.
Manufacturers globally spend more than $6 trillion a year in pursuit of quality control, and they apply defect detection on nearly every product line. But manual inspections can’t keep up with the demands.
Many manufacturers have automated optical inspection (AOI) systems that can help, but often these have high false detection rates, requiring labor-intensive and costly secondary manual inspections in an already challenging labor market, reducing their value.
NVIDIA Metropolis for Factories now offers a state-of-the-art AI platform and workflows for the development of incredibly accurate inspection applications such as AOI.
Pegatron Drives AOI With Metropolis for Factories
Leading manufacturer Pegatron, based in Taipei’s Beitou district, is using NVIDIA Metropolis for Factories on its production lines.
Pegatron manufactures everything from motherboards to smartphones, laptops and game consoles. With a dozen manufacturing facilities handling more than 300 products and more than 5,000 parts per day, Pegatron has a lot of quality control to manage across its product portfolio. Further, frequent product updates require ongoing revisions to its AOI systems.
Pegatron is using the entire Metropolis for Factories workflow to support its printed circuit board (PCB) factories with simulation, robotics and automated production inspection. Metropolis for Factories enables the electronics manufacturing giant to quickly update its defect detection models and achieve 99.8% accuracy on its AOI systems, starting with small datasets.
Pegatron uses NVIDIA Isaac Sim, a robotic simulator, to program robotic arms in simulation and to model the performance of its fleets of mobile robots.
In Metropolis, NVIDIA TAO Toolkit allows Pegatron to access pretrained models and transfer learning to build its highly accurate defect detection models from its enhanced datasets.
The NVIDIA DeepStream software development kit can be used to develop optimized intelligent video applications that handle multiple video, image and audio streams. Using DeepStream, Pegatron was able to achieve a 10x improvement in throughput.
Moreover, Omniverse enables Pegatron to run digital twins of its inspection equipment, so it can simulate future inspection processes, promising increased efficiencies to its production workflow.
It’s also used by Quanta subsidiary Techman Robot, which taps Isaac Sim to optimize the inspection of robots by robots on their manufacturing line.
Metropolis for Factories is helping manufacturers like Pegatron to increase production line throughput, reduce costs and improve production quality.
Growing Partner Ecosystem Supports Metropolis
Metropolis for Factories can be deployed from the enterprise industrial edge to the cloud, and a large and growing ecosystem of partners is helping bring it to market.
A host of specialists are joining forces on this effort including sensor makers, application partners, inspection equipment makers and integration partners.
Basler, a leading maker of imaging sensors and systems, has partnered with NVIDIA to help developers build AI-enabled inspection systems faster through tighter integration with the NVIDIA DeepStream SDK.
Quantiphi, a Metropolis partner, is working with one of the world’s largest beverage producers to automate inspections of fully packed pallets with GPU-powered vision AI.
Overview and Advantech — both NVIDIA Metropolis partners — are collaborating to build a real-time AI-based inspection system to support industrial inspection, product counting and assembly verification.
Metropolis partners Siemens and Data Monsters are working together to build industrial inspection systems, bringing together Omniverse Replicator synthetic data generation, NVIDIA TAO training, DeepStream runtime and Siemens’ NVIDIA Jetson-powered industrial personal computers.
Generative AI is rapidly ushering in a new era of computing for productivity, content creation, gaming and more. Generative AI models and applications — like NVIDIA NeMo and DLSS 3 Frame Generation, Meta LLaMa, ChatGPT, Adobe Firefly and Stable Diffusion — use neural networks to identify patterns and structures within existing data to generate new and original content.
When optimized for GeForce RTX and NVIDIA RTX GPUs, which offer up to 1,400 Tensor TFLOPS for AI inferencing, generative AI models can run up to 5x faster than on competing devices. This is thanks to Tensor Cores — dedicated hardware in RTX GPUs built to accelerate AI calculations — and regular software improvements. Enhancements introduced last week at the Microsoft Build conference doubled performance for generative AI models, such as Stable Diffusion, that take advantage of new DirectML optimizations.
As more AI inferencing happens on local devices, PCs will need powerful yet efficient hardware to support these complex tasks. To meet this need, RTX GPUs will add Max-Q low-power inferencing for AI workloads. The GPU will operate at a fraction of the power for lighter inferencing tasks, while scaling up to unmatched levels of performance for heavy generative AI workloads.
To create new AI applications, developers can now access a complete RTX-accelerated AI development stack running on Windows 11, making it easier to develop, train and deploy advanced AI models. This starts with development and fine-tuning of models with optimized deep learning frameworks available via Windows Subsystem for Linux.
Developers can then move seamlessly to the cloud to train on the same NVIDIA AI stack, which is available from every major cloud service provider. Next, developers can optimize the trained models for fast inferencing with tools like the new Microsoft Olive. And finally, they can deploy their AI-enabled applications and features to an install base of over 100 million RTX PCs and workstations that have been optimized for AI.
“AI will be the single largest driver of innovation for Windows customers in the coming years,” said Pavan Davuluri, corporate vice president of Windows silicon and system integration at Microsoft. “By working in concert with NVIDIA on hardware and software optimizations, we’re equipping developers with a transformative, high-performance, easy-to-deploy experience.”
To date, over 400 RTX AI-accelerated apps and games have been released, with more on the way.
This custom AI model foundry service transforms games by bringing intelligence to non-playable characters through AI-powered natural language interactions. Developers of middleware, tools and games can use ACE for Games to build and deploy customized speech, conversation and animation AI models in their software and games.
Generative AI on RTX, Anywhere
From servers to the cloud to devices, generative AI running on RTX GPUs is everywhere. NVIDIA’s accelerated AI computing is a low-latency, full-stack endeavor. We’ve been optimizing every part of our hardware and software architecture for many years for AI, including fourth-generation Tensor Cores — dedicated AI hardware on RTX GPUs.
Regular driver optimizations ensure peak performance. The most recent NVIDIA driver, combined with Olive-optimized models and updates to DirectML, delivers significant speedups for developers on Windows 11. For example, Stable Diffusion performance is improved by 2x compared to the previous interference times for developers taking advantage of DirectML optimized paths.
And with the latest generation of RTX laptops and mobile workstations built on the NVIDIA Ada Lovelace architecture, users can take generative AI anywhere. Our next-gen mobile platform brings new levels of performance and portability — in form factors as small as 14 inches and as lightweight as about three pounds. Makers like Dell, HP, Lenovo and ASUS are pushing the generative AI era forward, backed by RTX GPUs and Tensor Cores.
“As AI continues to get deployed across industries at an expected annual growth rate of over 37% now through 2030, businesses and consumers will increasingly need the right technology to develop and implement AI, including generative AI. Lenovo is uniquely positioned to empower generative AI spanning from devices to servers to the cloud, having developed products and solutions for AI workloads for years. Our NVIDIA RTX GPU-powered PCs, such as select Lenovo ThinkPad, ThinkStation, ThinkBook, Yoga, Legion and LOQ devices, are enabling the transformative wave of generative AI for better everyday user experiences in saving time, creating content, getting work done, gaming and more.” — Daryl Cromer, vice president and chief technology officer of PCs and Smart Devices at Lenovo
“Generative AI is transformative and a catalyst for future innovation across industries. Together, HP and NVIDIA equip developers with incredible performance, mobility and the reliability needed to run accelerated AI models today, while powering a new era of generative AI.” — Jim Nottingham, senior vice president and general manager of Z by HP
“Our recent work with NVIDIA on Project Helix centers on making it easier for enterprises to build and deploy trustworthy generative AI on premises. Another step in this historic moment is bringing generative AI to PCs. Think of app developers looking to perfect neural network algorithms while keeping training data and IP under local control. This is what our powerful and scalable Precision workstations with NVIDIA RTX GPUs are designed to do. And as the global leader in workstations, Dell is uniquely positioned to help users securely accelerate AI applications from the edge to the datacenter.” — Ed Ward, president of the client product group at Dell Technologies
“The generative AI era is upon us, requiring immense processing and fully optimized hardware and software. With the NVIDIA AI platform, including NVIDIA Omniverse, which is now preinstalled on many of our products, we are excited to see the AI revolution continue to take shape on ASUS and ROG laptops.” — Galip Fu, director of global consumer marketing at ASUS
Soon, laptops and mobile workstations with RTX GPUs will get the best of both worlds. AI inference-only workloads will be optimized for Tensor Core performance while keeping power consumption of the GPU as low as possible, extending battery life and maintaining a cool, quiet system. The GPU can then dynamically scale up for maximum AI performance when the workload demands it.
NVIDIA GeForce NOW (@NVIDIAGFN) 
