In a talk, now available online, NVIDIA Chief Scientist Bill Dally describes a tectonic shift in how computer performance gets delivered in a post-Moore’s law era.
Each new processor requires ingenuity and effort inventing and validating fresh ingredients, he said in a recent keynote address at Hot Chips, an annual gathering of chip and systems engineers. That’s radically different from a generation ago, when engineers essentially relied on the physics of ever smaller, faster chips.
The team of more than 300 that Dally leads at NVIDIA Research helped deliver a whopping 1,000x improvement in single GPU performance on AI inference over the past decade (see chart below).
It’s an astounding increase that IEEE Spectrum was the first to dub “Huang’s Law” after NVIDIA founder and CEO Jensen Huang. The label was later popularized by a column in the Wall Street Journal.
The advance was a response to the equally phenomenal rise of large language models used for generative AI that are growing by an order of magnitude every year.
“That’s been setting the pace for us in the hardware industry because we feel we have to provide for this demand,” Dally said.
In his talk, Dally detailed the elements that drove the 1,000x gain.
The largest of all, a sixteen-fold gain, came from finding simpler ways to represent the numbers computers use to make their calculations.
The New Math
The latest NVIDIA Hopper architecture with its Transformer Engine uses a dynamic mix of eight- and 16-bit floating point and integer math. It’s tailored to the needs of today’s generative AI models. Dally detailed both the performance gains and the energy savings the new math delivers.
Separately, his team helped achieve a 12.5x leap by crafting advanced instructions that tell the GPU how to organize its work. These complex commands help execute more work with less energy.
As a result, computers can be “as efficient as dedicated accelerators, but retain all the programmability of GPUs,” he said.
In addition, the NVIDIA Ampere architecture added structural sparsity, an innovative way to simplify the weights in AI models without compromising the model’s accuracy. The technique brought another 2x performance increase and promises future advances, too, he said.
Dally described how NVLink interconnects between GPUs in a system and NVIDIA networking among systems compound the 1,000x gains in single GPU performance.
No Free Lunch
Though NVIDIA migrated GPUs from 28nm to 5nm semiconductor nodes over the decade, that technology only accounted for 2.5x of the total gains, Dally noted.
That’s a huge change from computer design a generation ago under Moore’s law, an observation that performance should double every two years as chips become ever smaller and faster.
Those gains were described in part by Denard scaling, essentially a physics formula defined in a 1974 paper co-authored by IBM scientist Robert Denard. Unfortunately, the physics of shrinking hit natural limits such as the amount of heat the ever smaller and faster devices could tolerate.
An Upbeat Outlook
Dally expressed confidence that Huang’s law will continue despite diminishing gains from Moore’s law.
For example, he outlined several opportunities for future advances in further simplifying how numbers are represented, creating more sparsity in AI models and designing better memory and communications circuits.
Because each new chip and system generation demands new innovations, “it’s a fun time to be a computer engineer,” he said.
Dally believes the new dynamic in computer design is giving NVIDIA’s engineers the three opportunities they desire most: to be part of a winning team, to work with smart people and to work on designs that have impact.
Pixellot is scoring with vision AI — making it easier for organizations to deliver real-time sports broadcasting and analytics to viewers across the globe.
A member of the NVIDIA Metropolis vision AI partner ecosystem, the company based near Tel Aviv offers an AI-powered platform that automates the capturing, streaming and analysis of sporting events.
It’s changing the game for fans, coaches and players of nearly 20 different sports — not just basketball and soccer but also rugby and handball — as it broadcasts events and provides analytics from more than 30,000 venues across 70+ countries. In the U.S., Pixellot powers the broadcasting of over a million games every year through its partnership with the NFHS Network, a leader in streaming live and on-demand high school sports.
Through its broadcasting partners like the NFHS Network, MLB and others, Pixellot provides professional analytics, post-match breakdowns and highlights based on jersey numbers with shot charts and heat maps — which can be especially useful for coaches and players of school and pro sports alike as they study their moves to up their game. It also enables interactive experiences for users, who can manipulate viewframes and cut their own highlights for a game.
Recently, SuperSport Schools, a company based in Cape Town, South Africa, deployed the Pixellot platform to power an app that broadcasts student athletics across the nation, where more than 1,500 high schools are active in sports.
“Our goal is to democratize the coverage of sports with the help of AI and automation,” said Yossi Tarablus, who leads marketing at Pixellot, a member of the NVIDIA Inception program for cutting-edge startups. “Using the NVIDIA Jetson platform for edge AI, Pixellot brings powerful technology for sports broadcasting and analytics to some of the world’s most remote areas.”
How Pixellot Works
During peak sports seasons, about 200,000 games a month are broadcasted across the globe using the Pixellot platform, according to Tarablus.
Lightweight Pixellot cameras powered by NVIDIA Jetson capture high-quality video of games, matches and even practices — and livestream them in high definition to users through an app in real time with an overlaid scoreboard, live stats, commentary and more.
The platform creates an automatic viewframe that simulates a camera operator, optimizes videos and corrects scene lighting using NVIDIA RTX ray-tracing technology.
In addition, the platform helps organizations and companies monetize sports while making them more accessible to viewers, as it enables over-the-top, or OTT, streaming — direct streaming over the internet without the need for a traditional cable or satellite TV provider.
In all of its camera setups, the Metropolis member runs the NVIDIA DeepStream software development kit for AI-powered video streaming analytics. And the company relies on the NVIDIA TensorRT SDK for high-performance deep learning inference.
“NVIDIA Jetson made it possible for Pixellot to create the most accurate and affordable AI-powered camera solution for broadcasting live sporting events,” said Gal Oz, chief technology officer and cofounder of Pixellot. “The versatility of Jetson modules in terms of camera pipeline, encoders and AI capabilities enabled Pixellot to develop multiple products based on the same hardware and software platform.”
Broadcasting South African School Sports
High-quality, real-time broadcasts of athletics are difficult to produce without access to a slew of graphics and data.
As the NVIDIA Jetson Orin NX module enables AI-powered video processing and GPU-accelerated computing right at the edge — on the field or at courtside — Pixellot lets organizations broadcast sports from anywhere.
“It’s amazing how many people have told us stories about a moment they were empowered to share with their children thanks to SuperSport Schools and Pixellot, because they couldn’t be there physically but were present through live or on-demand video,” said Kelvin Watt, managing director of Capitalize Media and SuperSport Schools, on the Pixellot deployment in South Africa.
The SuperSport Schools app, which is free and recently reached 600,000 subscribers, was the first to broadcast a junior nationals track race in the country.
At the event last year, a student named Viwe Jingqi broke 50-plus-year national records for both the 100- and 200-meter races for South African girls under 18 years old. People all over the world could easily witness these historic victories through the SuperSport Schools app, powered by Pixellot.
Building a Smart Sports City in China
In China, tech giant Baidu and the Chengdu Sports Authority are using Pixellot technology in an initiative to develop a smart sports city, with an initial focus on broadcasting community soccer.
Chengdu, the capital of southwestern China’s Sichuan province, is a sports-oriented city and was the host of this year’s World University Games, an event sanctioned by the International University Sports Federation.
“Pixellot’s AI-driven sports production solutions are a perfect fit for our strategic vision of delivering innovative technology solutions to communities,” said Liu Chuan, solution director of the intelligent cloud sports industry at Baidu.
“Broadcasting community soccer with vision AI is part of the Chengdu initiative’s efforts to emphasize the health benefits of engaging in sports recreationally,” said Tarablus. “It moves the spotlight from pro or Olympic sports to the importance of athletics for all.”
Learn more about the NVIDIA Metropolis application framework, developer tools and partner ecosystem.
The wait is over. GeForce NOW Ultimate members can experience Cyberpunk 2077:Phantom Liberty on GOG.com at full GeForce RTX 4080 quality, with support for NVIDIA DLSS 3.5 technology.
It’s part of an action-packed GFN Thursday, with 26 more games joining the cloud gaming platform’s library, including Quake II from id Software.
A New Look for Night City
Experience NVIDIA DLSS 3.5 in Cyberpunk 2077’s spy-thriller expansion.
Take on a thrilling challenge with Phantom Liberty, an all-new adventure for Cyberpunk 2077. When the orbital shuttle of the President of the New United States of America is shot down over the deadliest district of Night City, there’s only one person who can save her. Become V, a cyberpunk for hire, and dive into a tangled web of espionage and political intrigue, unraveling a story that connects the highest echelons of power with the brutal world of black-market mercenaries.
Ultimate members can return to the neon lights of Night City and experience the benefits of NVIDIA DLSS 3.5 and its new Ray Reconstruction technology. These updates enhance the quality of full ray tracing in Cyberpunk 2077’s Ray Tracing: Overdrive Mode, as part of the game’s 2.0 update available for the base game for free and included with the Phantom Liberty expansion. Upgrade to a GeForce NOW Ultimate membership today to see Night City at its best.
Prepare for War
id Software’s classic first-person shooter is better than ever, streaming from the cloud.
Experience the authentic, enhanced and complete version of id Software’s critically acclaimed first-person shooter, Quake II, now streaming from the cloud.
Humankind is at war with the Strogg, a hostile alien race that’s attacked Earth. In response, humanity launched a strike on the Strogg homeworld — which failed. Outnumbered and outgunned, battle through fortified military installations to shut down the enemy’s war machine. Only then will the fate of humanity be decided.
Quake II includes a new, enhanced version of id Software’s classic, along with both original mission packs: “The Reckoning” and “Ground Zero.” Plus, battle through 28 campaign levels in MachineGames’ all-new “Call of the Machine” expansion and play through the exclusive levels from Quake II 64 for the first time on PC. Blast the Stroggs or friends in classic multiplayer modes at up to 4K and 120 frames per second or with ultrawide resolutions for GeForce NOW Ultimate members.
Nice Shootin,’ Tex
Ultimate leads the way.
The GeForce NOW Ultimate KovaaK’s challenge is complete, and the results are in: Ultimate power means more ultimate wins. Members worked to sharpen their skills and win amazing prizes during the challenge while experiencing the power of Ultimate for themselves. With up to 240 fps streaming at ultra-low latency, gamers are playing up to their ultimate potential, just by upgrading to an Ultimate membership.
The proof is in the data. Check out some powerful stats showing what Ultimate members accomplished:
Nearly 15,000 people took on the challenge, playing over 120,000 sessions.
Members saw a 2x boost in scores when playing on Ultimate over a Free membership.
All of the leaderboard’s top 25 slots were filled with those playing on Ultimate.
But don’t just take our word for it. Here’s what it felt like for TinooQ, who placed third overall in the challenge:
“As a long-time KovaaK’s user, transitioning to this platform was seamless, as the precision and responsiveness was nothing short of extraordinary.
“The minimal latency and the consistent 240 fps made me think that many people could rely solely on the GeForce NOW Ultimate plan and a monitor, that’s all. I found it perfect for top gaming, saving a lot of money and the PC hardware headaches I suffered when building mine.” — TinooQ
And check out what the press had to say about the Ultimate membership tier:
“GeForce NOW KovaaK’s Challenge Proves Gaming At Glorious 240 FPS Matters” – Hot Hardware
“Hot dang does GeForce Now Ultimate ever deliver.” – Tom’s Guide
“NVIDIA GeForce NOW has held the crown for cloud gaming performance for a while now, and it’s just getting better.” – 9to5Google
“Unsurprisingly, NVIDIA says 98% of the users who tried Kovaak’s Challenge on the GeForce NOW Ultimate tier have seen improvements in their test results over the free tier.” – Wccftech
“Nvidia has taken a different approach to cloud gaming: Instead of boosting their library and settling for 1080p 60fps, Nvidia’s GeForce Now service prioritizes performance, implementing faster graphics cards for players to use.” – SlashGear
“One of the distinguishing factors of GeForce Now is its superior image quality and lack of noticeable input delay. ” – Game Is Hard
“NVIDIA’s GeForce NOW is widely considered one of the best cloud gaming platforms in terms of latency, visual fidelity, and overall experience.” — TweakTown
Everyone’s a winner when they play on Ultimate. Upgrade today for the best performance in the cloud, even when streaming popular shooters like Counter Strike, Destiny 2, Tom Clancy’s Rainbow Six Siege and more, where every frame counts.
Even better, Ultimate members get a free copy of KovaaK’s – the world’s best aim trainer. Don’t miss this chance to claim the reward, starting today, only for Ultimate members. Be on the lookout for an email starting today, available only for a limited time.
Challenge Accepted
Be a hero. Be Dai.
Square Enix’s Infinity Strash: DRAGON QUEST The Adventure of Dai leads 26 new titles in the GeForce NOW library this week. In this action role-playing game based on the popular anime and manga series of the same name, Dai and his friends must fight the Dark Lord Hadlar and his evil army of monsters. Fulfill Dai’s dream of becoming a hero in this game, which features fast-paced, dynamic combat, stunning anime-style graphics and a rich storyline.
Here’s the full list of what’s joining this week:
These Doomed Isles (New release on Steam, Sept. 25)
DENZA, the luxury electric-vehicle brand and joint venture between BYD and Mercedes-Benz, is debuting new intelligent driving features for its entire N7 model lineup, powered by the NVIDIA DRIVE Orin system-on-a-chip (SoC).
The N7 series was introduced earlier this year as a family of spacious five-seater SUVs for commuters looking to sport a deluxe EV with advanced driving functionality.
All N7 models can be equipped with the NVIDIA DRIVE Orin SoC for high-performance compute to simultaneously run in-vehicle applications and deep neural networks for automated driving.
NVIDIA DRIVE Orin serves as the brain behind DENZA’s proprietary Commuter Smart Driving system, which offers an array of smart features, including:
Navigate on autopilot for high-speed, all-scenario assisted driving.
Intelligent speed-limit control and emergency lane-keeping aid, for safer commutes on urban roads and highways.
Enhanced automatic emergency braking and front cross-traffic alert for increased safety at intersections and on narrow streets.
Automated parking assist, which scouts for parking spots, identifying horizontal, vertical and diagonal spaces to ease the challenge of parking in crowded areas.
Next-Gen Car Configuration
In addition to adopting accelerated computing in the car, DENZA is one of the flagship automotive trailblazers using the NVIDIA Omniverse Cloud platform to build and deploy next-generation car configurators to deliver greater personalization options for the consumer’s vehicle-purchasing experience.
Medical-device company Invenio Imaging is developing technology that enables surgeons to evaluate tissue biopsies in the operating room, immediately after samples are collected — providing in just three minutes AI-accelerated insights that would otherwise take weeks to obtain from a pathology lab.
In a surgical biopsy, a medical professional removes samples of cells or tissue that pathologists analyze for diseases such as cancer. By delivering these capabilities through a compact, AI-powered imaging system within the treatment room, Invenio aims to support rapid clinical decision-making.
“This technology will help surgeons make intraoperative decisions when performing a biopsy or surgery,” said Chris Freudiger, chief technology officer of Silicon Valley-based Invenio. “They’ll be able to rapidly evaluate whether the tissue sample contains cancerous cells, decide whether they need to take another tissue sample and, with the AI models Invenio is developing, potentially make a molecular diagnosis for personalized medical treatment within minutes.”
Quicker diagnosis enables quicker treatment. It’s especially critical for aggressive types of cancer that could grow or spread significantly in the weeks it takes for biopsy results to return from a dedicated pathology lab.
Invenio is a member of NVIDIA Inception, a program that provides cutting-edge startups with technological support and AI platform guidance. The company accelerates AI training and inference using NVIDIA GPUs and software libraries.
Laser Focus on Cancer Care
The NIO Laser Imaging System accelerates the imaging of fresh tissue biopsies.
Invenio’s NIO Laser Imaging System is a digital pathology tool that accelerates the imaging of fresh tissue biopsies. It’s been used in thousands of procedures in the U.S. and Europe. In 2021, it received the CE Mark of regulatory approval in Europe.
“We’re building a layer of AI models on top of our imaging capabilities to provide physicians with not just the diagnostic image but also an analysis of what they’re seeing,” Freudiger said. “With the AI performance provided by NVIDIA Jetson at the edge, they’ll be able to quickly determine what kinds of cancer cells are present in a biopsy image.”
Invenio uses a cluster of NVIDIA RTX A6000 GPUs to train neural networks with tens of millions of parameters on pathologist-annotated images. The models were developed using the TensorFlow deep learning framework and trained on images acquired with NIO imaging systems.
“The most powerful capability for us is the expanded VRAM on the RTX A6000 GPUs, which allows us to load large batches of images and capture the variability of features,” Freudiger said. “It makes a big difference for AI training.”
On the Path to Clinical Deployment
One of Invenio’s AI products, NIO Glioma Reveal, is approved for clinical use in Europe and available for research use in the U.S. to help identify areas of cancerous cells in brain tissue.
A team of Invenio’s collaborators from the University of Michigan, New York University, University of California San Francisco, the Medical University of Vienna and University Hospital of Cologne recently developed a deep learning model that can find biomarkers of cancerous tumors with 93% accuracy in 90 seconds.
With this ability to analyze different molecular subtypes of cancer within a tissue sample, doctors can predict how well a patient will respond to chemotherapy — or determine whether a tumor has been successfully removed during surgery.
Beyond its work on brain tissue analysis, Invenio this year announced a clinical research collaboration with Johnson & Johnson’s Lung Cancer Initiative to develop and validate an AI solution that can help evaluate lung biopsies. The AI model will help doctors rapidly determine whether collected tissue samples contain cancer.
Lung cancer is the world’s deadliest form of cancer, and in the U.S. alone, lung nodules are found in over 1.5 million patients each year. Once approved for clinical use, Invenio’s NIO Lung Cancer Reveal tool aims to shorten the time needed to analyze tissue biopsies for these patients.
As part of this initiative, Invenio will run a clinical study before submitting the NVIDIA Jetson-powered AI solution for FDA approval.
As generative AI sweeps across corporate boardrooms around the world, global telecommunications companies are exploring how to cost-effectively deliver many of these new AI applications to the edge over 5G and upcoming 6G networks.
Telcos plan to deploy over 17 million 5G microcells and towers worldwide by 2025. Building, managing and optimizing this new infrastructure while maintaining quality-of-service delivery and maximizing the customer experience is the industry’s next big challenge.
Today, NTT DOCOMO announced it is deploying a GPU-accelerated wireless solution in its network in Japan. This makes it the first-ever telco in the world to deploy a GPU-accelerated commercial 5G network.
DOCOMO’s move aims to address the multibillion dollar problem of driving improvements in performance, total cost of ownership and energy efficiency while unlocking the flexibility, scalability and supply chain diversity promise of Open RAN.
The 5G Open RAN solution uses a high-performance 5G virtual radio access network (vRAN) from Fujitsu built on the NVIDIA Aerial vRAN stack and NVIDIA Converged Accelerators. This combination enables telcos to create a fully software- and cloud-defined network that can dynamically allocate resources using industry-standard equipment.
“Open RAN offers the opportunity to build next-generation 5G networks with unprecedented flexibility and scalability thanks to multivendor connections,” said Sadayuki Abeta, global head of Open RAN solutions at NTT DOCOMO. “We look forward to continuing working with NVIDIA on infrastructure solutions that meet those needs.”
The 5G Open RAN solution is the first 5G vRAN for telco commercial deployment using the NVIDIA Aerial platform, with a hardened, carrier-grade vRAN stack. The platform brings together the NVIDIA Aerial vRAN stack for 5G, AI frameworks, accelerated compute infrastructure and long-term software support and maintenance.
Cost and Energy-Efficiency Benefits of GPU Acceleration
Working with offerings from Fujitsu and Wind River, the new 5G solution uses the NVIDIA Aerial platform to lower costs and reduce power consumption. Compared to its existing 5G network deployments, DOCOMO says the solution reduces total costs by up to 30%, network design utilization by up to 50%, and power consumption at base stations by up to 50%.
“Delivering a 5G Open RAN network that meets stringent performance requirements of operators is a significant accomplishment,” said Masaki Taniguchi, senior vice president and head of the Mobile System Business Unit at Fujitsu Limited. “Using our Fujitsu vCU/vDU, in combination with NVIDIA Aerial platform, will help network operators to efficiently build high-performance 5G networks for consumers and businesses alike.”
NVIDIA’s contribution to the DOCOMO rollout is part of a growing portfolio of 5G solutions that are driving transformation in the telecommunications industry. Anchored on NVIDIA Aerial vRAN stack and NVIDIA Converged Accelerators — combined with NVIDIA BlueField data processing units (DPUs) and a suite of AI frameworks — NVIDIA provides a high-performance, software-defined, cloud-native, AI-enabled 5G for on-premises and telco operators’ RAN.
Fujitsu, NVIDIA and Wind River have been working under the OREX (5G Open RAN service brand), which was launched by DOCOMO in February 2021, to develop the Open RAN 5G vRAN. OREX has been deployed in Japan based on Fujitsu’s virtualized DU (vDU) and virtualized CU (vCU) and leverages commercial off-the-shelf servers, the Wind River cloud platform, Fujitsu’s 5G vRAN software and the NVIDIA Aerial vRAN stack and NVIDIA Converged Accelerators.
“Wind River is delighted to work with NTT DOCOMO, Fujitsu and NVIDIA towards a vision of improved efficiency of RAN deployments and operations,” said Paul Miller, chief technology officer at Wind River. “Wind River Studio provides a cloud-native, distributed cloud, automation, and analytics solution based on open source, so that operators can deploy and manage their 5G edge networks globally at high scale with faster innovation. This solution is proven in highly scaled production deployments today.”
OREX: Building Out From Japan and Beyond
DOCOMO and its partners in OREX are promoting a multivendor, Open RAN-compliant 5G vRAN to the global operator community. The commercial deployment in Japan is a first step in the vision of OREX, where members can commercially validate their solutions and then promote them to other operators globally.
NVIDIA is working with DOCOMO and other partners to support operators around the world to deploy high-performance, energy-efficient, software-defined, commercial 5G vRAN.
Denny’s CEO Kelli Valade was joined Tuesday by NVIDIA CEO Jensen Huang Tuesday to unveil a plaque at the Silicon Valley Denny’s where NVIDIA’s founders hatched their idea for a chip that would enable realistic 3D graphics on personal computers.
“This is a place where we fuel ideas, your story is so inspiring it will continue to inspire people at Dennny’s,” Valade said, as she presented the plaque to Huang.
“Denny’s has taught me so many lessons,” Huang said.
Both CEOs got their start working in diners. Valade got her first job as a waitress at a diner when she was 16. Huang got his first job at Denny’s in Portland when he was 15.
“I was a dishwasher, I was a busboy, I waited tables,” Huang said. “No one can carry more coffee cups than I can.”
To fuel even more great ideas, Valade announced the Denny’s Trillion-Dollar Incubator Contest — offering $25,000 in seed money for the next $1 trillion idea.
The contest is open to anyone with a creative and innovative idea that could impact the world. The only catch? The idea must originate — like NVIDIA — in a Denny’s booth.
NVIDIA, the leading accelerated computing and AI company, got its start at the 24-hour diner chain known for favorites such as its signature Grand Slam combo.
In 1993, three friends — Huang, Chris Malachowsky and Curtis Priem — met at Denny’s to discuss creating a chip that would enable realistic 3D graphics on personal computers.
The Denny’s just off a busy thoroughfare in the heart of Silicon Valley was the perfect place to start a business, said Huang, who lived nearby at the time with his wife and kids.
“It had all the coffee you could drink and no one could chase you out,” Huang told Valade.
Tuesday’s event took place in a corner of the bustling restaurant — one of the most popular Denny’s locations in Northern California — as families, retirees, and workers coming off the night shift piled in for plates piled high with eggs and pancakes, sausage and bacon.
Huang was among them, starting the day with a meeting where he and his table polished off a Lumberjack Slam, Moons Over My Hammy, and a Super Bird sandwich — washed down with plenty of coffee.
Huang, whose family immigrated to the United States from Taiwan when he was a child, told Valade he had his first hamburger at Dennys and his first milkshake.
“We make the best pancakes here,” Huang said.
“I love how you still say ‘we,’” Valade said.
“Made fresh every day,” Huang added with a grin.
Valade and Huang said Denny’s can be a great launching pad, not just for great ideas, but for great careers.
“Start your first job in the restaurant business,” Huang said. “It teaches you humility, it teaches you hard work, it teaches you hospitality.”
Valade agreed wholeheartedly, who, after talking shop with Huang, checked in with diners like Alfred, who was tucking into a stack of pancakes amid the morning rush.
For people across Silicon Valley, it’s the place to be. “I come here every day,” the retired roofer said.
Full contest details for the Denny’s Trillion Dollar Incubator Contest can be found at www.dennys.com/trilliondollarincubator. Contestants can submit their ideas online or at Denny’s restaurant by Nov. 21st at 8:59 am. The winner will be announced early next year.
Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks and demonstrates how NVIDIA Studio technology improves creative workflows. We’re also deep-diving on new GeForce RTX 40 Series GPUfeatures, technologies and resources and how they dramatically accelerate content creation.
Improving gaming, creating and everyday productivity, NVIDIA RTX graphics cards feature specialized Tensor Cores that deliver cutting-edge performance and transformative capabilities for AI.
An arsenal of 100 million AI-ready, RTX-powered Windows 11 PCs and workstations are primed to excel in AI workflows and help launch a new wave of training models and apps.
This week In the NVIDIA Studio, learn more about these AI power-players and read about Victor de Martrin, a self-taught 3D artist who shares the creative process behind his viral video Ascension, which features an assist from AI and uses a new visual effect process involving point clouds.
AI on the Prize
AI can elevate both work and play.
Content creators with NVIDIA Studio hardware can access over 100 AI-enabled and RTX-accelerated creative apps, including the Adobe Creative Cloud suite, Blender, Blackmagic Design’s DaVinci Resolve, OBS Studio and more.
This includes the NVIDIA Studio suite of AI tools and exclusive next-generation technology such as NVIDIA DLSS 3.5, featuring Ray Reconstruction and OptiX. AI image generation on a GeForce RTX 4090-powered PC can be completed at lightning speed — the same task would take 8x longer with an Apple M2 Ultra.
3D modelers can benefit immensely from AI, with dramatic improvements in real-time viewport rendering and upscaling with DLSS in NVIDIA Omniverse, Adobe Substance, Adobe Painter, Blender, D5 Render, Unreal Engine and more.
D5 Render full ray-tracing preview with Ray Reconstruction.
Gamers are primed for AI boosts with DLSS in over 300 games, including Alan Wake 2, coming soon, and Cyberpunk 2077: Phantom Liberty, available for download today.
Modders can breathe new life into classic games with NVIDIA RTX Remix’s revolutionary AI upscaling and texture enhancements. Game studios deploying the NVIDIA Avatar Cloud Engine (ACE) can build intelligent game characters, interactive avatars and digital humans in apps at scale — saving time and money.
Everyday tasks also get a boost. AI can draft emails, summarize content and upscale video to stunning 4K on YouTube and Netflix with RTX Video Super Resolution. Frequent video chat users can harness the power of NVIDIA Broadcast for AI-enhanced voice and video features. And livestreamers can take advantage of AI-powered features like virtual backgrounds, noise removal and eye contact, which moves the eyes of the speaker to simulate eye contact with the camera.
NVIDIA hardware powers AI natively and in the cloud — including for the world’s leading cloud AIs like ChatGPT, Midjourney, Microsoft 365 Copilot, Adobe Firefly and more.
Get further with AI — faster on RTX — and learn more to get started.
State-of-the-art technologies behind the Windows platform and NVIDIA’s AI hardware and software enable GPU-accelerated workflows for training and deploying AI models, exclusive tools, containers, software development kits and new open-source models optimized for RTX.
100 million RTX-powered Windows 11 PCs and workstations are already AI-ready.
With these integrations, Windows and NVIDIA are making it easier for developers to create the next generation of AI-powered Windows apps.
AI on Viral Content
3D content creator Martrin saw content creation as a means to gain financial, creative and geographical freedom — as well as a sense of gratification that he wasn’t getting in his day job in advertising.
“To gain recognition as an artist nowadays, unless you have connections, you’ve got to grind it out on social media,” admitted Martrin. As he began to showcase artwork to wider audiences, Martrin unexpectedly enjoyed the process of creating art aimed at garnering social media attention.
Best known for his viral video series Satisfying Marble Music — in which he replicates melodies by simulating a marble bouncing on xylophone notes — Martrin also takes a keen interest in the latest trends and advancements in the world of 3D to consistently meet his clients’ evolving demands.
In his recent project Ascension, he experimented with a point-cloud modeling technique for the first time, aiming to create a transition between reality and 3D.
Martrin used several 3D apps to refine video footage and animate himself in 3D. The initial stages involved modeling and animating a portrait of himself, which he used Daz Studio and Blender to accomplish. His PC, equipped with two NVIDIA GeForce RTX 4090 GPUs, easily handled this task.
Animations come to life in Blender.
He then used the GPU-accelerated Marvelous Designer cloth simulation engine to craft and animate his clothing.
Realistic clothes generated in Marvelous Designer.
Martrin stressed the importance of GPU acceleration in his creative workflow. “When did I use it? The countless times I checked my live viewer and during the creation of every render,” said Martrin.
From there, Martrin tested point-cloud modeling techniques. He first 3D-scanned his room. Then, instead of using the room as conventional 3D mesh, he deployed it as a set of data points in space to create a unique visual style.
Background elements added in Cinema 4D.
He then used Cinema 4D and OctaneRender with RTX-accelerated ray tracing to render the animation.
Martrin completed the project by applying a GPU-accelerated composition and special effects in Adobe After Effects, achieving faster rendering with NVIDIA CUDA technology.
“No matter how tough it is at the start, keep grinding and challenging — odds are, success will eventually follow,” said Martrin.
In the wake of ChatGPT, every company is trying to figure out its AI strategy, work that quickly raises the question: What about security?
Some may feel overwhelmed at the prospect of securing new technology. The good news is policies and practices in place today provide excellent starting points.
Indeed, the way forward lies in extending the existing foundations of enterprise and cloud security. It’s a journey that can be summarized in six steps:
Expand analysis of the threats
Broaden response mechanisms
Secure the data supply chain
Use AI to scale efforts
Be transparent
Create continuous improvements
AI security builds on protections enterprises already rely on.
Take in the Expanded Horizon
The first step is to get familiar with the new landscape.
Security now needs to cover the AI development lifecycle. This includes new attack surfaces like training data, models and the people and processes using them.
Extrapolate from the known types of threats to identify and anticipate emerging ones. For instance, an attacker might try to alter the behavior of an AI model by accessing data while it’s training the model on a cloud service.
The security researchers and red teams who probed for vulnerabilities in the past will be great resources again. They’ll need access to AI systems and data to identify and act on new threats as well as help build solid working relationships with data science staff.
Broaden Defenses
Once a picture of the threats is clear, define ways to defend against them.
Monitor AI model performance closely. Assume it will drift, opening new attack surfaces, just as it can be assumed that traditional security defenses will be breached.
Also build on the PSIRT (product security incident response team) practices that should already be in place.
For example, NVIDIA released product security policies that encompass its AI portfolio. Several organizations — including the Open Worldwide Application Security Project — have released AI-tailored implementations of key security elements such as the common vulnerability enumeration method used to identify traditional IT threats.
Adapt and apply to AI models and workflows traditional defenses like:
Protect the datasets used to train AI models. They’re valuable and vulnerable.
Once again, enterprises can leverage existing practices. Create secure data supply chains, similar to those created to secure channels for software. It’s important to establish access control for training data, just like other internal data is secured.
Some gaps may need to be filled. Today, security specialists know how to use hash files of applications to ensure no one has altered their code. That process may be challenging to scale for petabyte-sized datasets used for AI training.
The good news is researchers see the need, and they’re working on tools to address it.
Scale Security With AI
AI is not only a new attack area to defend, it’s also a new and powerful security tool.
Machine learning models can detect subtle changes no human can see in mountains of network traffic. That makes AI an ideal technology to prevent many of the most widely used attacks, like identity theft, phishing, malware and ransomware.
NVIDIA Morpheus, a cybersecurity framework, can build AI applications that create, read and update digital fingerprints that scan for many kinds of threats. In addition, generative AI and Morpheus can enable new ways to detect spear phishing attempts.
Machine learning is a powerful tool that spans many use cases in security.
Security Loves Clarity
Transparency is a key component of any security strategy. Let customers know about any new AI security policies and practices that have been put in place.
For example, NVIDIA publishes details about the AI models in NGC, its hub for accelerated software. Called model cards, they act like truth-in-lending statements, describing AIs, the data they were trained on and any constraints for their use.
NVIDIA uses an expanded set of fields in its model cards, so users are clear about the history and limits of a neural network before putting it into production. That helps advance security, establish trust and ensure models are robust.
Define Journeys, Not Destinations
These six steps are just the start of a journey. Processes and policies like these need to evolve.
The emerging practice of confidential computing, for instance, is extending security across cloud services where AI models are often trained and run in production.
The industry is already beginning to see basic versions of code scanners for AI models. They’re a sign of what’s to come. Teams need to keep an eye on the horizon for best practices and tools as they arrive.
Along the way, the community needs to share what it learns. An excellent example of that occurred at the recent Generative Red Team Challenge.
In the end, it’s about creating a collective defense. We’re all making this journey to AI security together, one step at a time.
Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks and demonstrates how NVIDIA Studio technology improves creative workflows.
Backed by the NVIDIA Studio platform, the Surface Laptop Studio 2, announced today, offers maximum stability with preinstalled Studio Drivers, plus exclusive tools to accelerate professional and creative workflows.
NVIDIA today also launched DLSS 3.5, adding Ray Reconstruction to the suite of AI-powered DLSS technologies. The latest feature puts a powerful new AI neural network at the fingertips of creators on RTX PCs, producing higher-quality, lifelike ray-traced images in real time before generating a full render.
Chaos Vantage is the first creative application to integrate Ray Reconstruction. For gamers, Ray Reconstruction is now available in Cyberpunk 2077 and is slated for the Phantom Liberty expansion on Sept. 26.
Blackmagic Design has adopted NVIDIA TensorRT acceleration in update 18.6 for its popular DaVinci Resolve software for video editing, color correction, visual effects, motion graphics and audio post-production. By integrating TensorRT, the software now runs AI tools like Magic Mask, Speed Warp and Super Scale over 50% faster than before. With this acceleration, AI runs up to 2.3x faster on GeForce RTX and NVIDIA RTX GPUs compared to Macs.
The September Studio Driver is now available for download — providing support for the Surface Laptop Studio 2, new app releases and more.
NVIDIA and GeForce RTX GPUs get NVIDIA Studio Drivers free of charge.
This week’s In the NVIDIA Studio installment features Studio Spotlight artist Gavin O’Donnell, who created a Wild West-inspired piece using a workflow streamlined with AI and RTX acceleration in Blender and Unreal Engine running on a GeForce RTX 3090 GPU.
Create Without Compromise
The Surface Laptop Studio 2, when configured with NVIDIA laptop GPUs, delivers up to 2x the graphics performance compared to the previous generation, and is NVIDIA Studio validated.
The versatile Microsoft Surface Laptop Studio 2.
In addition to NVIDIA graphics, the Surface Laptop Studio 2 comes with 13th Gen Intel Core processors, up to 64GB of RAM and a 2TB SSD. It features a bright, vibrant 14.4-inch PixelSense Flow touchscreen, with true-to-life color and up to 120Hz refresh rate, and now comes with Dolby Vision IQ and HDR to deliver sharper colors.
The system’s unique design adapts to fit any workflow. It instantly transitions from a pro-grade laptop to a perfectly angled display for entertainment to a portable creative canvas for drawing and sketching with the Surface Slim Pen 2.
NVIDIA Studio systems deliver upgraded performance thanks to dedicated ray tracing, AI and video encoding hardware. They also provide AI app acceleration, advanced rendering and Ray Reconstruction with NVIDIA DLSS, plus exclusive software like NVIDIA Omniverse, NVIDIA Broadcast, NVIDIA Canvas and RTX Video Super Resolution — helping creators go from concept to completion faster.
The Surface Laptop Studio 2 will be available beginning October 3.
An AI for RTX
NVIDIA GeForce and RTX GPUs feature powerful local accelerators that are critical for AI performance, supercharging creativity by unlocking AI capabilities on Windows 11 PCs — including the Surface Laptop Studio 2.
AI-powered Ray Resstruction.
With the launch of DLSS 3.5 with Ray Reconstruction, an NVIDIA supercomputer-trained AI network replaces hand-tuned denoisers to generate higher-quality pixels between sampled rays.
Ray Reconstruction improves the real-time editing experience by sharpening images and reducing noise — even while panning around a scene. The benefits of Ray Reconstruction shine in the viewport, as rendering during camera movement is notoriously difficult.
Chaos Vantage DLSS 3.5 with Ray Reconstruction technology.
By adding DLSS 3.5 support, Chaos Vantage will enable creators to explore large scenes in a fully ray-traced environment at high frame rates with improved image quality. Ray Reconstruction joins other AI-accelerated features in Vantage, including the NVIDIA OptiX denoiser and Super Resolution.
DLSS 3.5 and AI-powered Ray Reconstruction today join performance-multiplying AI Frame Generation in Cyberpunk 2077 with the new 2.0 update. On Sept. 26, Cyberpunk 2077: Phantom Liberty will launch with full ray tracing and DLSS 3.5.
NVIDIA TensorRT — the high-performance inference optimizer that delivers low latency and high throughput for deep learning inference applications and features — has been added to DaVinci Resolve in version 18.6.
Performance testing conducted by NVIDIA in August 2023. NVIDIA Driver 536.75. Windows 11. Measures time to apply various AI effects in DaVinci Resolve 18.6: Magic Mask, Speed Warp, Super Res, Depth Map.
The added acceleration dramatically increases performance. AI effects now run on a GeForce RTX 4090 GPU up to 2.3x faster than on M2 Ultra, and 5.4x faster than on 7900 XTX.
Stay tuned for updates in the weeks ahead for more AI on RTX, including DLSS 3.5 support coming to Omniverse this October.
Welcome to the Wild, Wild West
Gavin O’Donnell — an Ireland-based senior environment concept artist at Disruptive Games — is no stranger to interactive entertainment.
He also does freelance work on the side including promo art, environment design and matte painting for an impressive client list, including Disney, Giant Animation, ImagineFX, Netflix and more.
O’Donnell’s series of Western-themed artwork — the Wild West Project — was directly inspired by the critically acclaimed classic open adventure game Red Dead Redemption 2.
Prospector, lawman or vigilante?
“I really enjoyed how immersive the storyline and the world in general was, so I wanted to create a scene that might exist in that fictional world,” said O’Donnell. Furthermore, it presented him an opportunity to practice new workflows within 3D apps Blender and Unreal Engine.
‘Wild West Project’ brings the American Frontier to life.
Workflows combining NVIDIA technologies including RTX GPU acceleration at the intersection of AI were of especially great interest — accelerated by his GeForce RTX 3090 laptop GPU.
In Blender — O’Donnell sampled AI-powered Blender Cycles RTX-accelerated OptiX ray tracing in the viewport for interactive, photoreal rendering for modeling and animation.
On a beautiful journey.
Meanwhile — in Unreal Engine — O’Donnell sampled NVIDIA DLSS to increase the interactivity of the viewport by using AI to upscale frames rendered at lower resolution while still retaining high-fidelity detail. On top of RTX-accelerated rendering for high-fidelity visualization of 3D designs, virtual production and game development, the artist could simply create better, more detailed artwork, faster and easier.
O’Donnell credits his success to a constant state of creative evaluation — ensuring everything from his content creation techniques, methods of gaining inspiration, and technological knowledge — enabling the highest quality artwork possible — all while maintaining resource and efficiency gains.
As such, O’Donnell recently upgraded to an NVIDIA Studio laptop equipped with a GeForce RTX 4090 GPU with spectacular results. His rendering speeds in Blender, already very fast, sped up 73%, a massive time savings for the artist.
Senior environment concept artist Gavin O’Donnell.
And finally, don’t forget to enter the #StartToFinish community challenge! Show us a photo or video of how one of your art projects started — and then one of the final result — using the hashtag #StartToFinish and tagging @NVIDIAStudio for a chance to be featured! Submissions considered through Sept. 30.
The contest is open to anyone with a creative and innovative idea that could impact the world. The only catch? The idea must originate — like NVIDIA — in a Denny’s booth.
