Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows. We’re also deep diving on new GeForce RTX 40 Series GPU features, technologies and resources, and how they dramatically accelerate content creation.
An update is now available for NVIDIA Canvas, the free beta app that harnesses the power of AI to help artists quickly turn simple brushstrokes into realistic landscapes.
This version 1.4 update includes a new Panorama mode, which 3D artist Dan “Greenskull” Hammill explores this week In the NVIDIA Studio.
The #GameArtChallenge charges ahead with this sensationally scary The Last of Us-themed 3D animation by @Noggi29318543.
Share game-inspired art using the #GameArtChallenge hashtag through Sunday, April 30, for a chance to be featured on the @NVIDIAStudio or @NVIDIAOmniverse channels.
Panorama Comes to Canvas
NVIDIA Canvas 1.4 adds Panorama mode, allowing for the creation of 4K equirectangular landscapes for 3D workflows. Graphic designers will be able to apply Canvas AI-generated scenes to their workflow, allowing for quick and easy iterations.
Users can select between Standard and Panorama workspace each time they open or create a new file.
For 3D artist and AI aficionado Dan “Greenskull” Hammill, Canvas technology invokes an intentional change of tone.
“The therapeutic nature of painting a landscape asks me to slow things down and let my inner artist free,” said Greenskull. “The legendary Bob Ross is a clear inspiration for how I speak during my videos. I want the viewer to both be fascinated by the technology and relaxed by the content.”
For a recent piece, called “The Cove,” Greenskull took a few minutes to create his preferred landscape — an ocean view complete with hills, foliage, sand and darker skies for a cloudy day — with the few strokes of a digital pen in Canvas, all accelerated by his GeForce RTX 4090 GPU.
‘The Cove’ was created in NVIDIA Canvas and completed in mere minutes.
The artist refined his landscape in even more detail with an expanded selection of brush-size options included in the Canvas 1.4 release. Once satisfied with the background, Greenskull reviewed his creation. “I can look at the 3D view, review, see how it looks, and it’s looking pretty good, pretty cool,” he said.
Greenskull then uploaded his landscape into a video game within Unreal Engine 5 as the skybox or enclosed world. Now his Canvas texture makes up the background.
“This really does open up a lot of possibilities for game designers, especially indie developers who quickly want to create something and have it look genuinely unique and great,” Greenskull said.
“NVIDIA has powered both my casual and professional life for countless years. To me, NVIDIA is reliable, powerful and state of the art. If I’m going to be on top of my game, I should have the hardware that will keep up and push forward.” — Dan “Greenskull” Hammill
With his new virtual world complete, Greenskull prepared to create videos for his social media platforms.
“I hit record, run my DSLR through a capture card, record dialog through Adobe Audition, and grab a second screen capture with a separate PC,” explained Greenskull.
Greenskull then pieced everything together, syncing the primary video, secondary PC video captures and audio files. He reviewed the clips, made minor edits and exported final videos.
Using his favorite video editing app, Adobe Premiere Pro, Greenskull tapped his NVIDIA RTX 4090 GPU’s dual AV1 video encoders via the Voukoder plug-in, cutting export times in half with improved video quality.
Download the Canvas beta, free for NVIDIA and GeForce RTX GPU owners.
Learn more about these latest technologies by joining us at the Game Developers Conference. And catch up on all the groundbreaking announcements in generative AI and the metaverse by watching the NVIDIA GTC keynote.
Disney Dreamlight Valley is streaming from Steam and Epic Games Store on GeForce NOW starting today.
It’s one of two new games this week that members can stream with beyond-fast performance using a GeForce NOW Ultimate membership. Game as if using a PC on any device — at up to 4K resolution and 120 frames per second — even on a Mac.
Game Different
I’m a Mac, and I’m now a gaming PC.
GeForce NOW gives members the unique ability to play over 1,500 games with the power of a gaming PC, on nearly any device.
The new Ultimate membership taps into next-generation NVIDIA SuperPODs that stream GeForce RTX 4080-class performance. With support for 4K resolution at up to 120 fps or high-definition gaming at 240 fps on both PCs and Macs, even Mac users can say they’re PC gamers.
“For Mac users, GeForce NOW is an opportunity to finally play the most advanced games available on the computer they love, which is exciting.” — MacStories.net
Macs with the latest Apple silicon — M2 and M1 chips — run the GeForce NOW app natively, without the need to install or run Rosetta. GeForce NOW members on a Mac get the best of PC gaming, on the system they love, without ever leaving the Apple ecosystem. This results in incredible performance from popular PC-only games without downloads, updates or patches.
“Any laptop can be a gaming laptop, even a MacBook.” — Laptop Mag
MacBook Pro 16-inch laptops with 3,456×2,234 ProMotion 120Hz refresh-rate displays enable gaming in 4K high dynamic range at up to 120 fps. With NVIDIA DLSS 3 technology, these Macs can even run graphically intense games like The Witcher 3 and Warhammer 40,000: Darktide at 4K 120 fps. MacBook Pro laptops with smaller displays and MacBook Airs with 2,560×1,664 displays transform into gaming PCs, running titles like Cyberpunk 2077 in 1440p HDR at liquid-smooth frame rates.
“NVIDIA’s GeForce NOW Ultimate changes everything. Suddenly, the Mac became a brilliant gaming platform.” — Forbes
GeForce NOW opens a world of gaming possibilities on Mac desktops — like the Mac mini, Mac Studio and iMac. Connect an ultrawide monitor and take in all the HDR cinematic game play at up to 3,840×1,600 and 120 fps in PC games such as Destiny 2 and Far Cry 6. With Macs connected to a 240Hz monitor, GeForce NOW Ultimate members can stream with the lowest latency in the cloud, enabling gaming at 240 fps in Apex Legends, Tom Clancy’s Rainbow Six Siege and nearly a dozen other competitive titles.
And it’s not just new Macs that can join in PC gaming. Any Mac system introduced in 2009 or later is fully supported.
We’ve Got Games, Say Cheers!
Oh boy! “Disney Dreamlight Valley” is streaming on GeForce NOW.
Help restore Disney magic to the Valley and go on an enchanting journey in Gameloft’s Disney Dreamlight Valley — a life-sim adventure game full of quests, exploration and beloved Disney and Pixar friends.
Peter Ma was bored in his high school computer science class. So he decided to teach himself something new: how to use artificial intelligence to find alien life.
In this episode of the NVIDIA AI Podcast, host Noah Kravitz interviews Ma, who is now an undergraduate student at the University of Toronto.
Ma tells Kravitz how he stumbled upon this problem and how he developed an AI algorithm that outperformed traditional methods in the search for extraterrestrial intelligence.
Sequoia Capital’s Pat Grady and Sonya Huang on Generative AI Pat Grady and Sonya Huang, partners at Sequoia Capital, to discuss their recent essay, “Generative AI: A Creative New World.” The authors delve into the potential of generative AI to enable new forms of creativity and expression, as well as the challenges and ethical considerations of this technology. They also offer insights into the future of generative AI.
Real or Not Real? Attorney Steven Frank Uses Deep Learning to Authenticate Art Steven Frank is a partner at the law firm Morgan Lewis, specializing in intellectual property and commercial technology law. He’s also half of the husband-wife team that used convolutional neural networks to authenticate artistic masterpieces, including da Vinci’s Salvador Mundi, with AI’s help.
GANTheftAuto: Harrison Kinsley on AI-Generated Gaming Environments Humans playing games against machines is nothing new, but now computers can develop games for people to play. Programming enthusiast and social media influencer Harrison Kinsley created GANTheftAuto, an AI-based neural network that generates a playable chunk of the classic video game Grand Theft Auto V.
Subscribe, Review and Follow NVIDIA AI on Twitter
If you enjoyed this episode, subscribe to the NVIDIA AI Podcast on your favorite podcast platform and leave a rating and review. Follow @NVIDIAAI on Twitter or email the AI Podcast team to get in touch.
Global leader in convenient foods and beverages PepsiCo is deploying advanced machine vision technology from startup KoiReader Technologies, powered by the NVIDIA AI platform and GPUs, to improve efficiency and accuracy in its distribution process.
PepsiCo has identified KoiReader’s technology as a solution to enable greater efficiency in reading warehouse labels. This AI-powered innovation helps read warehouse labels and barcodes in fast-moving environments where the labels can be in any size, at any angle or even partially occluded or damaged.
This is up and running in a PepsiCo distribution center in the Dallas-Fort Worth area, with plans for broader deployment this year.
“If you find the right lever, you could dramatically improve our throughput,” said Greg Bellon, senior director of digital supply chain at PepsiCo.
KoiReader’s AI-powered innovation helps read warehouse labels and barcodes in fast-moving environments.
KoiReader’s technology is being used to train and run the deep learning algorithms that power PepsiCo’s AI label and barcode scanning system.
Once near-perfect accuracy was achieved, its application is being expanded to validate customer deliveries to ensure 100% accuracy of human-assisted picking operations.
At the Dallas facility where PepsiCo is testing the technology, Koi’s AutonomousOCR technology scans some of the most complex warehouse labels quickly and accurately on fast-moving conveyor belts.
It also is being investigated to assist warehouse workers as they scan pallets of soda and snacks. The same AutonomousOCR technology has also been deployed to automate yard operations as tractors and trailers enter and exit PepsiCo’s distribution center in Texas.
“KoiReader’s capability offers up the potential for many use cases — starting small and demonstrating capability is key to success,” Bellon says.
The system is already generating valuable real-time insights, Bellon reports.
Koi’s technology can accurately track regular or irregularly shaped products, with and without labels, as well as count how long it takes workers to pack boxes, how many items they are packing, and how long it takes them to retrieve items for boxes.
It acts as a real-time industrial engineering study answering many questions about the influence of people, process and technology on throughput.
A broad array of the NVIDIA stack is being used by KoiReader across its diverse solutions portfolio and customer workflows.
And every aspect of Koi’s applications are built cloud-native, using containerization, Kubernetes and microservices.
Additionally, the NVIDIA AI Enterprise software suite promises to help PepsiCo confidently scale up and manage its applications and AI deployments.
“The KoiVision Platform was built to deliver logistics, supply chain, and industrial automation for enterprise customers. Our solution suite is helping PepsiCo improve operational efficiency and accuracy in its distribution process,” said Ashutosh Prasad, founder and CEO of KoiReader.
“We’re testing out object- and activity-detection capabilities and computer vision today to figure out what kind of data we want to collect with that sort of application,” Bellon said.
Bellon said he’s excited for what’s next. “We’re going to be on a journey together,” he said.
It all started with two software engineers and a tomato farmer on a West Coast road trip.
Visiting farms to survey their needs, the three hatched a plan at an apple orchard: build a highly adaptable 3D vision AI system for automating field tasks.
Verdant, based in the San Francisco Bay Area, is developing AI that promises versatile farm assistance in the form of a tractor implement for weeding, fertilizing and spraying.
Founders Lawrence Ibarria, Gabe Sibley and Curtis Garner — two engineers from Cruise Automation and a tomato farming manager — are harnessing the NVIDIA Jetson edge AI platform and NVIDIA Metropolis SDKs such as TAO Toolkit and DeepStream for this ambitious slice of farm automation.
The startup, founded in 2018, is commercially deployed in carrot farms and in trials at apple, garlic, broccoli and lettuce farms in California’s Central Valley and Imperial Valley, as well as in Oregon.
Verdant plans to help with organic farming by lowering production costs for farmers while increasing yields and providing labor support. It employs the tractor operator, who is trained to manage the AI-driven implements. The company’s robot-as-service model, or RaaS, enables farmers to see metrics on yield improvements and reductions in chemical costs, and pay by the acre for results.
“We wanted to do something meaningful to help the environment,” said Ibarria, Verdant’s chief operating officer. “And it’s not only reducing costs for farmers, it’s also increasing their yield.”
The company recently landed more than $46 million in series A funding.
Another recent event at Verdant was hiring as its chief technology officer Frank Dellaert, who is recognized for using graphical models to solve large-scale mapping and 4D reconstruction challenges. A faculty member at Georgia Institute of Technology, Dellaert has led work at Skydio, Facebook Reality Labs and Google AI while on leave from the research university.
“One of the things that was impressed upon me when joining Verdant was how they measure performance in real-time,” remarked Dellaert. “It’s a promise to the grower, but it’s also a promise to the environment. It shows whether we do indeed save from all the chemicals being put into the field.”
Verdant is a member of NVIDIA Inception, a free program that provides startups with technical training, go-to-market support, and AI platform guidance.
Verdant is working with Bolthouse Farms, based in Bakersfield, Calif., to help its carrot-growing business transition to regenerative agriculture practices. The aim is to utilize more sustainable farming practices, including reduction of herbicides.
Verdant is starting with weeding and expanding next into precision fertilizer applications for Bolthouse.
The computation and automation from Verdant have enabled Bolthouse Farms to understand how to achieve its sustainable farming goals, according to the farm’s management team.
Riding With Jetson AGX Orin
Verdant is putting the Jetson AGX Orin system-on-module inside tractor cabs. The company says that Orin’s powerful computing and availability with ruggedized cases from vendors makes it the only choice for farming applications. Verdant is also collaborating with Jetson ecosystem partners, including RidgeRun, Leopard Imaging and others.
The module enables Verdant to create 3D visualizations showing plant treatments for the tractor operator. The company uses two stereo cameras for its field visualizations, for inference and to gather data in the field for training models on NVIDIA DGX systems running NVIDIA A100 Tensor Core GPUs back at its headquarters. DGX performance allows Verdant to use larger training datasets to get better model accuracy in inference.
“We display a model of the tractor and a 3D view of every single carrot and every single weed and the actions we are doing, so it helps customers see what the robot’s seeing and doing,” said Ibarria, noting this can all run on a single AGX Orin module, delivering inference at 29 frames per second in real time.
DeepStream-Powered Apple Vision
Verdant relies on NVIDIA DeepStream as the framework for running its core machine learning to help power its detection and segmentation. It also uses custom CUDA kernels to do a number of tracking and positioning elements of its work.
Verdant’s founder and CEO, Sibley, whose post-doctorate research was in simultaneous localization and mapping has brought this expertise to agriculture. This comes in handy to help present a logical representation of the farm, said Ibarria. “We can see things, and know when and where we’ve seen them,” he said.
This is important for apples, he said. They can be challenging to treat, as apples and branches often overlap, making it difficult to find the best path to spray them. The 3D visualizations made possible by AGX Orin allow a better understanding of the occlusion and the right path for spraying.
“With apples, when you see a blossom, you can’t just spray it when you see it, you need to wait 48 hours,” said Ibarria. “We do that by building a map, relocalizing ourselves saying, ‘That’s the blossom, I saw it two days ago, and so it’s time to spray.’”
NVIDIA TAO for 5x Model Production
Verdant relies on NVIDIA TAO Toolkit for its model building pipeline. The transfer learning capability in TAO Toolkit enables it to take off-the-shelf models and quickly refine them with images taken in the field. For example, this has made it possible to change from detecting carrots to detecting onions, in just a day. Previously, it took roughly five days to build models from scratch that achieved an acceptable accuracy level.
“One of our goals here is to leverage technologies like TAO and transfer learning to very quickly begin to operate in new circumstances,” said Dellaert.
While cutting model building production time by 5x, the company has also been able to hit 95% precision with its vision systems using these methods.
“Transfer learning is a big weapon in our armory,” he said.
The mics were live and tape was rolling in the studio where the Miles Davis Quintet was recording dozens of tunes in 1956 for Prestige Records.
When an engineer asked for the next song’s title, Davis shot back, “I’ll play it, and tell you what it is later.”
Like the prolific jazz trumpeter and composer, researchers have been generating AI models at a feverish pace, exploring new architectures and use cases. Focused on plowing new ground, they sometimes leave to others the job of categorizing their work.
A team of more than a hundred Stanford researchers collaborated to do just that in a 214-page paper released in the summer of 2021.
In a 2021 paper, researchers reported that foundation models are finding a wide array of uses.
They said transformer models, large language models (LLMs) and other neural networks still being built are part of an important new category they dubbed foundation models.
Foundation Models Defined
A foundation model is an AI neural network — trained on mountains of raw data, generally with unsupervised learning — that can be adapted to accomplish a broad range of tasks, the paper said.
“The sheer scale and scope of foundation models from the last few years have stretched our imagination of what’s possible,” they wrote.
Two important concepts help define this umbrella category: Data gathering is easier, and opportunities are as wide as the horizon.
No Labels, Lots of Opportunity
Foundation models generally learn from unlabeled datasets, saving the time and expense of manually describing each item in massive collections.
Earlier neural networks were narrowly tuned for specific tasks. With a little fine-tuning, foundation models can handle jobs from translating text to analyzing medical images.
Foundation models are demonstrating “impressive behavior,” and they’re being deployed at scale, the group said on the website of its research center formed to study them. So far, they’ve posted more than 50 papers on foundation models from in-house researchers alone.
“I think we’ve uncovered a very small fraction of the capabilities of existing foundation models, let alone future ones,” said Percy Liang, the center’s director, in the opening talk of the first workshop on foundation models.
AI’s Emergence and Homogenization
In that talk, Liang coined two terms to describe foundation models:
Emergence refers to AI features still being discovered, such as the many nascent skills in foundation models. He calls the blending of AI algorithms and model architectures homogenization, a trend that helped form foundation models. (See chart below.)
The field continues to move fast.
A year after the group defined foundation models, other tech watchers coined a related term — generative AI. It’s an umbrella term for transformers, large language models, diffusion models and other neural networks capturing people’s imaginations because they can create text, images, music, software and more.
Generative AI has the potential to yield trillions of dollars of economic value, said executives from the venture firm Sequoia Capital who shared their views in a recent AI Podcast.
A Brief History of Foundation Models
“We are in a time where simple methods like neural networks are giving us an explosion of new capabilities,” said Ashish Vaswani, an entrepreneur and former senior staff research scientist at Google Brain who led work on the seminal 2017 paper on transformers.
That work inspired researchers who created BERT and other large language models, making 2018 “a watershed moment” for natural language processing, a report on AI said at the end of that year.
Google released BERT as open-source software, spawning a family of follow-ons and setting off a race to build ever larger, more powerful LLMs. Then it applied the technology to its search engine so users could ask questions in simple sentences.
In 2020, researchers at OpenAI announced another landmark transformer, GPT-3. Within weeks, people were using it to create poems, programs, songs, websites and more.
“Language models have a wide range of beneficial applications for society,” the researchers wrote.
Their work also showed how large and compute-intensive these models can be. GPT-3 was trained on a dataset with nearly a trillion words, and it sports a whopping 175 billion parameters, a key measure of the power and complexity of neural networks.
The growth in compute demands for foundation models. (Source: GPT-3 paper)
“I just remember being kind of blown away by the things that it could do,” said Liang, speaking of GPT-3 in a podcast.
The latest iteration, ChatGPT — trained on 10,000 NVIDIA GPUs — is even more engaging, attracting over 100 million users in just two months. Its release has been called the iPhone moment for AI because it helped so many people see how they could use the technology.
One timeline describes the path from early AI research to ChatGPT. (Source: blog.bytebytego.com)
From Text to Images
About the same time ChatGPT debuted, another class of neural networks, called diffusion models, made a splash. Their ability to turn text descriptions into artistic images attracted casual users to create amazing images that went viral on social media.
The first paper to describe a diffusion model arrived with little fanfare in 2015. But like transformers, the new technique soon caught fire.
Researchers posted more than 200 papers on diffusion models last year, according to a list maintained by James Thornton, an AI researcher at the University of Oxford.
In a tweet, Midjourney CEO David Holz revealed that his diffusion-based, text-to-image service has more than 4.4 million users. Serving them requires more than 10,000 NVIDIA GPUs mainly for AI inference, he said in an interview (subscription required).
Dozens of Models in Use
Hundreds of foundation models are now available. One paper catalogs and classifies more than 50 major transformer models alone (see chart below).
The Stanford group benchmarked 30 foundation models, noting the field is moving so fast they did not review some new and prominent ones.
Startup NLP Cloud, a member of the NVIDIA Inception program that nurtures cutting-edge startups, says it uses about 25 large language models in a commercial offering that serves airlines, pharmacies and other users. Experts expect that a growing share of the models will be made open source on sites like Hugging Face’s model hub.
Experts note a rising trend toward releasing foundation models as open source.
Foundation models keep getting larger and more complex, too.
That’s why — rather than building new models from scratch — many businesses are already customizing pretrained foundation models to turbocharge their journeys into AI.
Foundations in the Cloud
One venture capital firm lists 33 use cases for generative AI, from ad generation to semantic search.
Major cloud services have been using foundation models for some time. For example, Microsoft Azure worked with NVIDIA to implement a transformer for its Translator service. It helped disaster workers understand Haitian Creole while they were responding to a 7.0 earthquake.
In February, Microsoft announced plans to enhance its browser and search engine with ChatGPT and related innovations. “We think of these tools as an AI copilot for the web,” the announcement said.
Google announced Bard, an experimental conversational AI service. It plans to plug many of its products into the power of its foundation models like LaMDA, PaLM, Imagen and MusicLM.
“AI is the most profound technology we are working on today,” the company’s blog wrote.
Startups Get Traction, Too
Startup Jasper expects to log $75 million in annual revenue from products that write copy for companies like VMware. It’s leading a field of more than a dozen companies that generate text, including Writer, an NVIDIA Inception member.
Other Inception members in the field include Tokyo-based rinna that’s created chatbots used by millions in Japan. In Tel Aviv, Tabnine runs a generative AI service that’s automated up to 30% of the code written by a million developers globally.
A Platform for Healthcare
Researchers at startup Evozyne used foundation models in NVIDIA BioNeMo to generate two new proteins. One could treat a rare disease and another could help capture carbon in the atmosphere.
Evozyne and NVIDIA described a hybrid foundation model for creating proteins in a joint paper.
BioNeMo, a software platform and cloud service for generative AI in drug discovery, offers tools to train, run inference and deploy custom biomolecular AI models. It includes MegaMolBART, a generative AI model for chemistry developed by NVIDIA and AstraZeneca.
“Just as AI language models can learn the relationships between words in a sentence, our aim is that neural networks trained on molecular structure data will be able to learn the relationships between atoms in real-world molecules,” said Ola Engkvist, head of molecular AI, discovery sciences and R&D at AstraZeneca, when the work was announced.
Separately, the University of Florida’s academic health center collaborated with NVIDIA researchers to create GatorTron. The large language model aims to extract insights from massive volumes of clinical data to accelerate medical research.
A Stanford center is applying the latest diffusion models to advance medical imaging. NVIDIA also helps healthcare companies and hospitals use AI in medical imaging, speeding diagnosis of deadly diseases.
AI Foundations for Business
Another new framework, NVIDIA NeMo Megatron, aims to let any business create its own billion- or trillion-parameter transformers to power custom chatbots, personal assistants and other AI applications.
It created the 530-parameter Megatron-Turing Natural Language Generation model (MT-NLG) that powers TJ, the Toy Jensen avatar that gave part of the keynote at NVIDIA GTC last year.
Foundation models — connected to 3D platforms like NVIDIA Omniverse — will be key to simplifying development of the metaverse, the 3D evolution of the internet. These models will power applications and assets for entertainment and industrial users.
Factories and warehouses are already applying foundation models inside digital twins, realistic simulations that help find more efficient ways to work.
Foundation models can ease the job of training autonomous vehicles and robots that assist humans on factory floors and logistics centers like the one described below.
New uses for foundation models are emerging daily, as are challenges in applying them.
Several papers on foundation and generative AI models describing risks such as:
amplifying bias implicit in the massive datasets used to train models,
introducing inaccurate or misleading information in images or videos, and
violating intellectual property rights of existing works.
“Given that future AI systems will likely rely heavily on foundation models, it is imperative that we, as a community, come together to develop more rigorous principles for foundation models and guidance for their responsible development and deployment,” said the Stanford paper on foundation models.
Current ideas for safeguards include filtering prompts and their outputs, recalibrating models on the fly and scrubbing massive datasets.
“These are issues we’re working on as a research community,” said Bryan Catanzaro, vice president of applied deep learning research at NVIDIA. “For these models to be truly widely deployed, we have to invest a lot in safety.”
It’s one more field AI researchers and developers are plowing as they create the future.
It’s a thrilling GFN Thursday with GRID Legends racing to the cloud this week. It leads a total of eight new games expanding the GeForce NOW library. New content for Rainbow Six Siege is also now streaming.
Plus, two new cities are now online with GeForce RTX 4080 performance for cloud gaming. Chicago and Montreal have completed upgrades to RTX 4080 SuperPODs, delivering next-generation cloud streaming to GeForce NOW Ultimate members.
Shifting Up
Beyond-fast gaming meets beyond-fast racing.
Jump into the spectacular action of GRID Legends, the racing game from EA with drama at every turn. Battle for glory with a variety of incredible cars on stellar tracks featuring iconic landmarks from the streets of London and Moscow.
Navigate the world of high-stakes racing as a rookie behind the wheel, with a documentary team capturing every sensational moment. Conquer hundreds of events and challenges, and create a dream racing team in the unique, cinematic story mode. Join up online with other legends and race against friends, or challenge them to a race designed in the Race Creator mode.
GeForce NOW members can experience it all with high dynamic range on PC, Mac and SHIELD TV for a smooth, ultra-crisp driving experience, even under intense racing conditions.
Upgrade Roll Call
Order up! Servers are now live in Chicago and Montreal.
The RTX 4080 SuperPODs have been rolling out around the world for a couple months now, so it’s time to check in with a RTX 4080 roll call.
Chicago and Montreal bring the number of cities on the server update map to 10, joining Ashburn, Dallas, Los Angeles and San Jose in the U.S., and Amsterdam, Frankfurt, London and Paris in Europe. Now past it’s halfway point, the rollout is expected to be completed by mid-year.
Here’s a few reasons to upgrade:
NVIDIA DLSS 3 technology is enabled for AI-powered performance boosts on supported games like HITMAN World of Assassination and Marvel’s Midnight Suns. This means 4K streaming from the cloud results in the smoothest game play at up to 120 frames per second, even when settings are cranked to the max.
NVIDIA Reflex delivers ultra-low latency. Paired with DLSS 3, the technology enables Ultimate members to stream games like Rainbow Six Siege and Apex Legends at up to 240 fps on PCs and Macs, with as low as 35 milliseconds of total latency for a streaming experience that feels nearly indistinguishable from being on a local desktop.
Ultrawide resolutions are supported for the first time ever from the cloud, giving Ultimate members the most immersive game play in No Man’s Sky, Cyberpunk 2077 and Assassin’s Creed Valhalla.
Ultimate members in and around the 10 cities on the map are streaming with new performance today, and can take full advantage of these RTX technologies in the cloud. Level up to next-generation cloud streaming today for beyond-fast gaming.
Spring Forward With New Games
“Brava will do what’s needed. Even when it requires sacrifice.”
Operation Commanding Force is the newest season of Year 8 for Tom Clancy’s Rainbow Six Siege, now available for members to stream. The update brings a new attacker named Brava, equipped with the Kludge Drone, a gadget that can disrupt enemy electronics devices and even take over some of them to turn the tides of battle.
That’s on top of the eight games joining the cloud this week:
There’s a whole lot of games streaming from the cloud, and we want to know your top three. Let us know in the comments below or on Twitter and Facebook.
The Academy Award nominations are in — and for the 15th year in a row, NVIDIA technologies worked behind the scenes of every film nominated for Best Visual Effects.
The five VFX contenders for the 95th annual Academy Awards, taking place on Sunday, March 12, include:
All Quiet on the Western Front
Avatar: The Way of Water
The Batman
Black Panther: Wakanda Forever
Top Gun: Maverick
For over a decade, filmmakers and VFX studios around the world have used NVIDIA technologies to power the most advanced, visually rich movies ever made. Today, creators and artists are transforming VFX using advanced capabilities in graphics, like real-time ray tracing, simulation, AI and virtual production — all powered by NVIDIA RTX technologies.
Diving Into Natural Wonders With Cutting-Edge Graphics
Award-winning studio Wētā FX created the stunning visuals for director James Cameron’s much-anticipated sequel, Avatar: The Way of Water. The film is one of Wētā’s largest VFX projects to date. The team created 3,240 shots — which is 98% of the total shots in the film, more than two-thirds of which featured water.
In computer graphics (CG), making water look natural and realistic — from how it moves off a character’s skin to how it drips from clothing — is one of the biggest challenges for visual effects artists. But for this film, Wētā developed and implemented a new water toolset that advanced their capabilities across simulation, rendering and more.
The team started with pre-production and performance capture using a real-time, GPU-based ocean spectrum deformer, which served as a consistent, physically based starting point for water on set. From there, Wētā created a new suite of water solvers — many of them within Loki, the studio’s in-house multiphysics simulation framework. Loki allows coupling of multiple solvers in any configuration. For example, hair, cloth, air and water can all be simulated together.
Other key innovations from Wētā centered on both dry and wet performance capture, new deep learning models to process stereo camera images and generate depth maps for compositing, and neural networks to assist with facial animation and muscle systems.
Creating Captivating Car Chases Through Gritty Gotham
Wētā FX was also behind the cinematic visuals for The Batman. The team, led by VFX supervisor Anders Langlands, worked on the gripping highway chase between Batman and the infamous villain, the Penguin. As they race through the city of Gotham under heavy rainfall, the Penguin sets off a sequence of car crashes and explosions.
To create a feeling of danger and exhilaration, the team put the car chase scene together through heavily enhanced live action and completely CG shots. Rendering the proper lighting; simulating realistic raindrops colliding with multiple surfaces, hydroplaning and wheel spray; and illuminating rain through headlights and streetlights all added to the complexity of these shots. Wētā also worked on background environments for scenes in the Batcave and Gotham’s City Hall.
Taking CGI to the Sky
The practical effects and cinematography behind Top Gun: Maverick was an instant highlight of this heart-pounding Hollywood blockbuster film. But to add more layers of realism to those outstanding aerial shots, VFX Supervisor Ryan Tudhope and the team at Method Studios partnered with the camera department, aerial coordinators and the United States Navy to film extensive air-to-air and ground-to-air footage of real jets. They captured over 800 hours of aerial stunts, mounts and plates to provide their team with a practical foundation for the visual effects work.
The Top Gun: Maverick team implemented various VFX techniques, creating a surprising 2,400 VFX shots for the movie. The visual effects included creating and adding CG planes in scenes, as well as adding missiles, smoke and explosions in various action sequences. The invisible nature of the visual effects in Top Gun: Maverick make it a top contender for the Academy Award for Best Visual Effects.
A New Swimlane for Underwater Worlds
In Black Panther: Wakanda Forever, Wētā FX further demonstrated its leadership in creating photorealistic underwater sequences. Chris White, visual effects supervisor for the film, was tasked with creating the Mesoamerican-inspired Talokan underwater kingdom.
To get a realistic look for the characters in this undersea world, Wētā used a combination of live-action sequences shot in water tanks and dry-for-wet shots that helped capture realistic underwater motion for the characters, clothes and hair.
Wētā also reflected how various skin tones would react to light with the added complexity of a murky underwater environment. The bar for realistic water simulation has once again been raised by Wētā FX in Blank Panther: Wakanda Forever.
All Action on the VFX Front
Movie magic is made when visual effects are so seamless that the audience remains completely immersed in the story, not realizing that what they’re seeing is an effect. This is how VFX supervisor Markus Frank and production company Cine Chromatix earned their Best Visual Effects nomination for All Quiet on the Western Front.
To authentically tell the story of two young soldiers during World War I, Cine Chromatix and the film’s visual effects teams focused on the fine details needed to craft VFX that are hidden in plain sight.
The result is stunning. Even after watching Cine Chromatix’s VFX breakdown reel for the film, viewers may find themselves scrubbing back and forth to decipher fact from fiction.
See How Oscar-Nominated VFX Are Created at GTC
NVIDIA congratulates all of this year’s nominees for the Academy Award for Best Visual Effects.
Learn more about visual effects, AI, virtual production and animation at NVIDIA GTC, a global technology conference taking place online March 20-23. Register for free and hear from industry luminaries creating stunning visuals in film and TV. Check out all the media and entertainment sessions at GTC.
Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows.
An adrenaline-fueled virtual ride in the sky is sure to satisfy all thrill seekers — courtesy of 3D artist Kosei Wano’s sensational animation, Moon Hawk. Wano outlines his creative workflow this week In the NVIDIA Studio.
Plus, join the #GameArtChallenge — running through Sunday, April 30 — by using the hashtag to share video game fan art, character creations and more for a chance to be featured across NVIDIA social media channels.
Original game content can be made with NVIDIA Omniverse — a platform for creating and operating metaverse applications — using the Omniverse Machinima app. This enables users to collaborate in real time when animating characters and environments in virtual worlds.
Who Dares, Wins
Wano often finds inspiration exploring the diversity of flora and fauna. He has a penchant for examining birds — and even knows the difference in wing shapes between hawks and martins, he said. This interest in flying entities extends to his fascination with aircrafts. For Moon Hawk, Wano took on the challenge of visually evolving a traditional, fuel-based fighter jet into an electric one.
With reference material in hand, Wano opened the 3D app Blender to scale the fighter jet to accurate, real-life sizing, then roughly sketched within the 3D design space, his preferred method to formulate models.
“Moon Hawk” in its traditional form.
The artist then deployed several tips and tricks to model more efficiently: adding Blender’s automatic detailing modifier, applying neuro-reflex modeling to change the aircraft’s proportions, then dividing the model’s major 3D shapes into sections to edit individually — a step Wano calls “dividing each difficulty.”
Neuro-reflex modeling enables Wano to change proportions while maintaining model integrity.
Blender Cycles RTX-accelerated OptiX ray tracing, unlocked by the artist’s GeForce RTX 3080 Ti GPU, enabled interactive, photorealistic modeling in the viewport. “Optix’s AI-powered denoiser renders lightly, allowing for comfortable trial and error,” said Wano, who then applied sculpting and other details. Next, Wano used geo nodes to add organic style and customization to his Blender scenes and animate his fighter jet.
Applying geo nodes.
Blender geo nodes make modeling an almost completely procedural process — allowing for non-linear, non-destructive workflows and the instancing of objects — to create incredibly detailed scenes using small amounts of data.
The “Moon Hawk” model is nearly complete.
For Moon Hawk, Wano applied geo nodes to mix materials not found in nature, creating unique textures for the fighter jet. Being able to make real-time base mesh edits without the concern of destructive workflows gave Wano the freedom to alter his model on the fly with an assist from his GPU. “With the GeForce RTX 3080 Ti, there’s no problem, even with a model as complicated as this,” he said.
Animations accelerated at the speed of light with Wano’s GeForce RTX GPU.
Wano kicked off the animation phase by selecting the speed of the fighter jet and roughly designing its flight pattern.
Mapping the flight path in advance.
The artist referenced popular fighter jet scenes in cinema and video games, as well as studied basic rules of physics, such as inertia, to ensure the flight patterns in his animation were realistic. Then, Wano returned to using geo nodes to add 3D lighting effects without the need to simulate or bake. Such lighting modifications helped to make rendering the project simpler in its final stage.
Parameters were edited with ease, in addition to applying particle simulations and manually shaking the camera to add more layers of immersion to the scenes.
Final color edits in Blender.
With the animation complete, Wano added short motion blur. Accelerated motion blur rendering enabled by his RTX GPU and the NanoVBD toolset for easy rendering of volumes let him apply this effect quickly. And RTX-accelerated OptiX ray tracing in Blender Cycles delivered the fastest final frame renders.
Wano imported final files into Blackmagic Design’s DaVinci Resolve application, where GPU-accelerated color grading, video editing and color scopes helped the artist complete the animation in record time.
3D artist Kosei Wano.
Choosing GeForce RTX was a simple choice for Wano, who said, “NVIDIA products have been trusted by many people for a long time.”
View more of Wano’s impressive portfolio on ArtStation.
Who Dares With Photogrammetry, Wins Again
Wano, like most artists, is always growing his craft, refining essential skills and learning new techniques, including photogrammetry — the art and science of extracting 3D information from photographs.
In the NVIDIA Studio artist Anna Natter recently highlighted her passion for photogrammetry, noting that virtually anything can be preserved in 3D and showcasing features that have the potential to save 3D artists countless hours. Wano saw this same potential when experimenting with the technology in Adobe Substance 3D Sampler.
“Photogrammetry can accurately reproduce the complex real world,” said Wano, who would encourage other artists to think big in terms of both individual objects and environments. “You can design an entire realistic space by placing it in a 3D virtual world.”
Try out photogrammetry and post your creations with the #StudioShare hashtag for a chance to be featured across NVIDIA Studio’s social media channels.
Preparing a retailer’s online catalog once required expensive physical photoshoots to capture products from every angle. A Tel Aviv startup is saving brands time and money by transforming these camera clicks into mouse clicks.
Hexa uses GPU-accelerated computing to help companies turn their online inventory into 3D renders that shoppers can view in 360 degrees, animate or even try on virtually to help their buying decisions. The company, which recently announced a $20.5 million funding round, is working with brands in fashion, furniture, consumer electronics and more.
“The world is going 3D,” said Yehiel Atias, CEO of Hexa. “Just a few years ago, the digital infrastructure to do this was still so expensive that it was more affordable to arrange a photographer, models and lighting. But with the advancements of AI and NVIDIA GPUs, it’s now feasible for retailers to use synthetic data to replace physical photoshoots.”
Hexa’s 3D renders are used on major retail websites such as Amazon, Crate & Barrel and Macy’s. The company creates thousands of renders each month, reducing the need for physical photoshoots of every product in a retailer’s catalog. Hexa estimates that it can save customers up to 300 pounds of carbon emissions for each product imaged digitally instead of physically.
From Physical Photoshoots to AI-Accelerated Renders
Hexa can reconstruct a single 2D image, or a set of low-quality 2D images, into a high-fidelity 3D asset. The company uses differing levels of automation for its renders depending on the complexity of the shape, the amount of visual data that needs to be reconstructed, and the similarity of the object to Hexa’s existing dataset.
To automate elements of its workflow, the team uses dozens of AI algorithms that were developed using the PyTorch deep learning framework and run on NVIDIA Tensor Core GPUs in the cloud. If one of Hexa’s artists is reconstructing a 3D toaster, for example, one algorithm can identify similar geometries the team has created in the past to give the creator a head start.
Another neural network can scan a retailer’s website to identify how many of its products Hexa can support with 3D renders. The company’s entire rendering pipeline, too, runs on NVIDIA GPUs available through Amazon Web Services.
“Accessing compute resources through AWS gives us the option to use thousands of NVIDIA GPUs at a moment’s notice,” said Segev Nahari, lead technical artist at Hexa. “If I need 10,000 frames to be ready by a certain time, I can request the hardware I need to meet the deadline.”
Nahari estimates that rendering on NVIDIA GPUs is up to 3x faster than relying on CPUs.
Broadening Beyond Retail, Venturing Into Omniverse
Hexa developers are continually experimenting with new methods for 3D rendering — looking for workflow improvements in preprocessing, object reconstruction and post-processing. The team recently began working with NVIDIA GET3D, a generative AI model by NVIDIA Research that generates high-fidelity, three-dimensional shapes based on a training dataset of 2D images.
By training GET3D on Hexa’s dataset of shoes, the team was able to generate 3D models of novel shoes not part of the training data.
In addition to its work in ecommerce, Hexa’s research and development team is investigating new applications for the company’s AI software.
“It doesn’t stop at retail,” Atias said. “Industries from gaming to fashion and healthcare are finding out that synthetic data and 3D technology is a more efficient way to do things like digitize inventory, create digital twins and train robots.”
The team credits its membership in NVIDIA Inception, a global program that supports cutting-edge startups, as a “huge advantage” in leveling up the technology Hexa uses.
“Being part of Inception opens doors that outsiders don’t have,” Atias said. “For a small company trying to navigate the massive range of NVIDIA hardware and software offerings, it’s a door-opener to all the cool tools we wanted to experiment with and understand the potential they could bring to Hexa.”
Hexa is testing the NVIDIA Omniverse Enterprise platform — an end-to-end platform for building and operating metaverse applications — as a tool to unify its annotating and rendering workflows, which are used by dozens of 3D artists around the globe. Omniverse Enterprise enables geographically dispersed teams of creators to customize their rendering pipelines and collaborate to build 3D assets.
“Each of our 3D artists has a different software workflow that they’re used to — so it can be tough to get a unified output while still being flexible about the tools each artist uses,” said Jonathan Clark, Hexa’s CTO. “Omniverse is an ideal candidate in that respect, with huge potential for Hexa. The platform will allow our artists to use the rendering software they’re comfortable with, while also allowing our team to visualize the final product in one place.”
pic.twitter.com/ticn42QMTt
NVIDIA GeForce NOW (@NVIDIAGFN) 
