Posts in category: nVidia AI

Building next-generation intelligent vehicles requires an AI infrastructure that pushes the cutting edge.

Electric vehicle maker NIO is using NVIDIA HGX to build a comprehensive data center infrastructure for developing AI-powered, software-defined vehicles. With high-performance compute, the automaker can continuously iterate on sophisticated deep learning models, creating robust autonomous driving algorithms in a closed-loop environment.

“The complex scenarios faced by mass-produced cars and the massive amount of data these fleets generate are the cornerstones of NIO’s autonomous driving capabilities,” said Bai Yuli, head of AI Platforms at NIO. “By using NVIDIA high-performance compute solutions, NIO can accelerate the path to autonomous driving.”

NIO has already launched intelligent vehicles developed on this infrastructure, such as its fully electric, intelligent flagship sedan, the ET7. Its mid-size performance sedan, the ET5 is scheduled to debut in September.

In addition to high-performance data center development, both models are built on the Adam supercomputer, powered by four NVIDIA DRIVE Orin systems-on-a-chip. These vehicles feature autonomous driving and intelligent cockpit capabilities that are continuously iterated upon and improved in the data center for a revolutionary customer experience.

Building a High-Performance AI Infrastructure With NVIDIA GPUs and Networking

The role of the data center is to ingest, curate and label massive amounts of data for AI model training at scale.

Data collection fleets generate hundreds of petabytes of data and billions of images each year. This data is then used to optimize the deep neural networks (DNNs) that will run in the vehicles.

NIO’s scalable AI infrastructure is powered by NVIDIA HGX with eight A100 Tensor Core GPUs and NVIDIA ConnectX-6 InfiniBand adapters. This scalable supercomputer cluster consists of NVME SSD servers and interconnects through the high-speed NVIDIA Quantum InfiniBand network platform.

This powerful infrastructure allows large amounts of deep learning training data to be transferred to supercomputer memory or NVIDIA A100 video memory at ultra-high speeds of up to 200Gbps.

NVIDIA HGX A100 is a high-performance server platform designed for AI scenarios, including big datasets and complicated models like those that power autonomous vehicles. It incorporates a fully optimized NVIDIA AI software stack in NGC.

The platform sets a new compute density benchmark, condensing 5 petaflops of AI performance and replaces siloed infrastructures with a single platform for a wide range of complex AI applications.

With the HGX A100, NIO is able to flexibly develop and deploy scalable AI systems. It also enables the company to increase model development efficiency by up to 20x, allowing it to launch autonomous vehicles sooner and evolve to newer, faster architectures.

Forging Ahead

NIO is already off to the races with its software-defined lineup, announcing plans to double the capacity of its plant in Hefei, China, to 240,000 vehicles per year, with a facility capable of producing up to 300,000.

As NIO scales production capabilities and continues its expansion into global markets, NVIDIA HGX is scaling with them, enabling the deployment of one of the most advanced AI platforms in the automotive industry.

Fortnite on GeForce NOW with touch controls on mobile is now available to all members, streaming through the Safari web browser on iOS and the GeForce NOW Android app.

The full launch — including the removal of the waitlist — follows a successful beta period in which more than 500,000 participants streamed over 4 million sessions to hundreds of mobile device types.

The closed beta  gave the GeForce NOW team the opportunity to test and learn, which resulted in optimized on-screen touch controls and game menus, with gameplay that feels intuitive. The end result is a touch experience for gamers to enjoy.

We’re thanking the beta participants, whose millions of streamed sessions made these improvements possible, with a three-day Priority membership — or three-day extension for existing RTX 3080 and Priority members. See below for details.

Streaming ‘Fortnite’ on GeForce NOW

GeForce NOW provides millions of gamers the opportunity to stream their favorite PC games — like Fortnite — to nearly any device. Games render on PCs in the cloud with high-performance NVIDIA GPUs and deliver gameplay back to those devices in a fraction of a second.

Getting started is easy. Visit the GeForce NOW membership page and choose from one of three levels of performance, including a Free membership option. RTX 3080 and Priority members get an upgraded experience with higher quality graphics, faster access to servers and more features. After signing up, download native GeForce NOW game streaming apps for PC, Mac, Android or TV, or start streaming right away from play.geforcenow.com.

Members can link their Epic Games account from the Settings > Connections menu to automatically sign in to Fortnite upon game launch.

With the addition of touch controls, Fortnite mobile players get GeForce performance they can touch. RTX 3080 members stream improved graphics that render at up to 120 frames per second on select 120Hz Android devices and can play with millions of other Fortnite players around the world.

GeForce NOW manages patches and game updates for members. As new seasons arrive or a new mode drops, members are always game ready. When no-build Battle Royale arrived in Fortnite with the launch of Fortnite Zero Build, GeForce NOW members were able to play right away.

For tips on gameplay mechanics or a refresher on playing Fortnite with touch controls, check out Fortnite’s Getting Started page.

A Three-Day Thank You to Beta Participants

As a thank you to members who helped improve Fortnite mobile touch controls during the closed beta period, participants can redeem a three-day Priority membership to GeForce NOW. Priority members get higher performance, access to premium servers and extended six-hour session lengths, for an upgraded gaming experience.

Members redeem the free promotional membership — or extension of their existing Founders, Priority or RTX 3080 membership — by logging into the redemption portal and following the instructions to activate. Beta participants will also receive an email in the coming days with additional details.

Get to Gaming

GFN Thursday always means more games. Members can find these and more streaming on the cloud this week:

• DEADCRAFT (New release on Steam)
• Old World (New release on Steam)
• Vampire: The Masquerade Swansong (New release on Epic Games Store)
• Dolmen (New Release on Steam, May 20)
• Trigon: Space Story (Epic Games Store)

Sign up for an RTX 3080 or Priority membership to play these and any of the 1,300 games available on the service.

With the power to take great gaming on the go with you, we’ve got a question this weekend. Let us know your answer on Twitter or in the comments below.

what’s the best place to game in the cloud that’s 𝙣𝙤𝙩 at home?

bonus points for pictures

— NVIDIA GeForce NOW (@NVIDIAGFN) May 18, 2022

Real-time rendering is helping one studio take virtual production to impossible heights.

In their latest project, the creators at Los Angeles-based company Impossible Objects were tasked with depicting an epic battle between characters from the upcoming video game, Diablo Immortal. But the showdown had to take place on the surface of a Google Pixel phone, set in the living room of a live actor.

The team at Impossible Objects brought this vision to life using accelerated virtual production workflows to blend visual effects with live action. Using Epic Games’ Unreal Engine and NVIDIA A6000-powered Dell Precision 7920 workstations, the team created all the stunning cinematics and graphics, from high-fidelity textures and reflections to realistic camera movement and lighting.

These advanced technologies helped the artists make instant creative decisions as they could view high-quality virtual imagery rendered in real time.

“We can build larger, photorealistic worlds and not worry about relying on outdated creative workflows,” said Joe Sill, founder of Impossible Objects. “With Unreal Engine and NVIDIA RTX-powered Dell Precision workstations, we brought these Diablo Immortal characters to life.”

Real-Time Technologies Deliver Impossible Results

Previously, to tackle a project like this, the Impossible Objects team would look at concept art and storyboards to get an idea of what the visuals were supposed to look like. But with virtual production, the creators can work in a nonlinear way, bridging the gap between imagination and the final high-resolution images faster than before.

For the Diablo Immortals project, Impossible Objects used Unreal Engine for previsualization — where the artists were able to make creative, intentional decisions because they were experiencing high-fidelity images in real time. Moreover, the previsualization happened simultaneously with the virtual art department and layout phases.

The team used NVIDIA A6000-powered Dell Precision 7920 workstations — an advanced combination that allowed the artists to enhance the virtual production and creative workflows. The A6000 GPU delivers 48 gigabytes of VRAM, a crucial spec when offline rendering in Unreal Engine. With more GPU memory, the team had room for more geometry and higher resolution textures.

“Rendering would not have been possible without the A6000 — we maxed out on its 48 gigs of memory, using all that room for textures, environments and geometry,” said Luc Delamare, head of Technology at Impossible Objects. “We could throw anything at the GPU, and we’d still have plenty of performance for real-time workflows.”

Typically, this project would have taken up to six months to complete. But the nonlinear approach enabled by the real-time pipeline allowed Impossible Objects to cut the production time in half.

The video game characters in the commercial were prebuilt and provided by Blizzard. Impossible Objects used Autodesk Maya to up-res the characters and scale them to perform better in a cinematic setting.

The team often toggled between compositing software, Autodesk Maya and Unreal Engine as they ported animation back and forth between the applications. And as the project started to get bigger, Impossible Objects turned to another solution: NVIDIA Deep Learning Super Sampling, an AI rendering technology that uses a neural network to boost frame rates and produce sharp images.

“NVIDIA DLSS was incredibly important, as we were able to use it in the real-time workflow, even with characters that had high polygon counts,” said Delamare. “This solution became really helpful, especially as the project started to get denser and denser.”

At the animation stage, Unreal Engine and NVIDIA RTX allowed the team to simultaneously update cinematography and lighting in real time. The end result was fewer department handoffs, which resulted in time saved and efficient creative communication gained.

With all of these advanced technologies combined, Impossible Objects had the power to create a more efficient, iterative process — one that allowed the team to ditch linear pipelines and instead take on a much more creative, collaborative workflow.

To learn more about the project, watch the video below:

Learn more about NVIDIA RTX technology in media and entertainment.

Eyal Ben-Ari just took his first shot on a goal of bringing professional-class analytics to amateur soccer players.

The CEO of startup Track160, in Tel Aviv, has seen his company’s AI-powered sports analytics software tested and used in the big leagues. Now he’s turning his attention to underserved amateurs in the clubs and community teams he says make up “the bigger opportunity” among the world’s 250 million soccer players.

“Almost everyone in professional sports uses data analytics today. Now we’re trying to enable any team at any level to capture their own data and analytics, and the only way to do it is leveraging AI,” he said.

A Kickoff Down Under

In April, the company launched its Coach160 software in Australia, where it’s getting kudos from amateur soccer clubs in Victoria and Queensland. It uses computer vision to let teams automatically generate rich reports and annotated videos with an off-the-shelf camera and a connection to the cloud.

“The analysis and data provided by Track160 will prove a wonderful resource for our coaches and players,” said Vaughn Coveny, a retired pro soccer player now working with multiple youth teams in the region.

Startup With an AI Heritage

Miky Tamir, a serial entrepreneur in sports tech, co-founded Track160 in 2017. The company’s investors include the Deutsche Fussball Liga, Germany’s national soccer league, which contributed annotated datasets from several of its seasons.

“That helped set a baseline, then we applied transfer learning and developed an ever-growing internal database,” said Tamir Anavi, Track160’s CTO.

Using video from a single camera, the company’s software identifies and tracks players as 3D skeletons, then tags events and actions as they move.

“We use deep learning in every step to understand where the camera is, where the pitch is and where the players are on it,” Anavi said.

With that information, the software delivers detailed analytics and more. It constructs a 3D model so players and coaches can view any part of the game from any perspective, providing what Ben-Ari calls “a metaverse experience.”

Software Certified by the Pros

The Coach160 software got high scores for speed and accuracy in a benchmark for electronic tracking systems created by FIFA, the global federation of more than 200 pro soccer leagues. “We delivered the same performance as others who used six times more cameras,” said Anavi.

One pro league uses the code to get real-time data on game days. It processes 4K video streams with four NVIDIA GPUs and libraries that accelerate the work.

When it comes to AI, Track160 relies on NVIDIA TensorRT to make its models lean so they run fast.

“We couldn’t do inference without it. The work went from being impossible to running smoothly and that got our system from a prototype to production,” said Anavi.

Track160 frequently signed on as a member of NVIDIA Metropolis, a program for companies in intelligent video analytics. Ben Ari says he’ll tap the program’s early access to technology and expertise to accelerate his company’s growth.

Looking Beyond Oz

Australia was a natural first target given its penchant for new technology and large number of amateur soccer players and clubs, said Ben-Ari, who is already planning a launch in the U.S.

Long term, the company plans to train models for other sports, too.

“We see a kind of viral effect where everyone will want to have this,” he said.

“As a dad, I want to know what’s happening when my daughter plays, and even if they’re not pros, people want to know their performance,” said Ben-Ari, who likes to pour over his stats from  triathlons.

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology accelerates creative workflows. 

Concept artist Pablo Muñoz Gómez dives In the NVIDIA Studio this week, showcasing artwork that depicts a fantastical myth.

Gómez, a creator based in Australia, is equally passionate about helping digital artists, teaching 3D classes and running the Zbrush guides website with his creative specialties: concept and character artistry.

“For me, everything starts with a story,” Muñoz Gómez said.

His 3D Forest Creature contains a fascinating myth. “The story of the forest creature is rather simple … a very small fantasy character that lives in the forest and spends his life balancing rocks, the larger the stones he manages to balance and stack on top of each other, the larger he’ll grow and the more invisible he’ll become. Eventually, he’ll reach a colossal size and disappear.”

Gómez begins his journey in a 2D app, Krita, with a preliminary sketch. The idea is to figure out how many 3D assets will be needed while adding a little bit of color as reference for the palette later on.

Next, Gómez moves to Zbrush, where he uses custom brushes to sculpt basic models for the creature, rocks and plants. It’s the first of multiple leaps in his 2D to 3D workflow, detailed in this two-part 3D Forest Creature tutorial.

Gómez then turns to Adobe Substance 3D Painter to apply various colors and materials directly to his 3D models. Here, the benefits of NVIDIA RTX acceleration shine. NVIDIA Iray technology in the viewport enables Gómez to edit in real time and use ray-traced baking for faster rendering speeds — all accelerated by his GeForce RTX 3090 GPU.

Seeking further customization for his background, Gómez downloads and imports a grass asset from the Substance 3D asset library into Substance 3D Sampler, adjusting a few sliders to create a photorealistic material. RTX-exclusive interactive ray tracing lets Gómez apply realistic wear-and-tear effects in real time, powered by his GPU.

3D workflows can be incredibly demanding. As Gómez notes, the right GPU allows him to focus on content creation. “Since I switched to the GeForce RTX 3090, I’m simply able to spend more time in the ‘creative stages’ and testing things to refine my concept when I don’t have to wait for a render or worry about optimizing a scene so I can see it in real time,” he said.

Gómez sets up his scene in Marmoset 4, critically changing the denoiser from CPU to GPU. Doing so unlocks real-time ray tracing and smooth visuals in the viewport while he works. This can be done by accessing the Lighting then Ray Tracing selections in the main menu and changing the denoiser from CPU to GPU.

With the scene in a good place after some edits, Gómez generates his renders.

He makes final composition, lighting and color correction in Adobe Photoshop. With the addition of a new background, the scene is complete.

More 3D to Explore

Gómez has created several tutorials demonstrating 3D content creation techniques to aspiring artists. Check out this one on how to build a 3D scene from scratch.

Part one of the Studio Session, Creating Stunning 3D Crystals, offers an inside look at sketching and concepting in Krita and modeling in Zbrush, while part two focuses on baking in Adobe Substance 3D Painter and texturing in Marmoset Toolbag 4.

Generally, low-polygon models for 3D workflows are great to work with on hardware that can’t handle high-poly counts. Gómez’s Studio Session, Creating a 3D Low-Poly Floating Island, demonstrates how to build low-poly models like his Floating Island within Zbrush and touch up in Adobe Photoshop.

However, with the graphics horsepower and AI benefits of NVIDIA RTX and GeForce RTX GPUs, 3D artists can work with high-polygon models quickly and easily.

Learning how to create in 3D takes ingenuity, notes Gómez: “You become more resourceful making your tools work for you in the way you want, even if that means finding a better tool to solve a particular process.” But with enough practice, as seen from the variety of Gómez’s portfolio, the results can be stunning.

Gómez is the founder of ZBrushGuides and the 3DConceptArtist academy. View his courses, tutorials, projects and more on his website.

Follow NVIDIA Studio on Facebook, Twitter and Instagram. Access tutorials on the Studio YouTube channel and get updates directly in your inbox by subscribing to the NVIDIA Studio newsletter.

When it comes to safety, efficiency and sustainability, autonomous vehicles are delivering a clean sweep.

Autonomous vehicle company and NVIDIA Inception member WeRide this month began a public road pilot of its Robo Street Sweepers. The vehicles, designed to perform round-the-clock cleaning services, are built on the high-performance, energy-efficient compute of NVIDIA.

The fleet of 50 vehicles is sweeping, sprinkling and spraying disinfectant in Guangzhou, China, all without a human driver at the wheel. The robo-sweepers run on a cloud-based fleet management platform that automatically schedules and dispatches vehicles using real-time information on daily traffic and routes.

Street sweeping is a critical municipal service. In addition to keeping dirt and debris off the road, it helps ensure trash and hazardous materials don’t flow into storm drains and pollute local waterways.

As cities grow, applying autonomous driving technology to street cleaning vehicles enables these fleets to run more efficiently and maintain cleaner and healthier public spaces.

Sweep Smarts

While street sweepers typically operate at lower speeds and in more constrained environments than robotaxis, trucks or other autonomous vehicles, they still require robust AI compute to safely operate.

Street cleaning vehicles must be able to drive in dense urban traffic, as well as in low-visibility conditions, such as nighttime and early morning. In addition, they have to detect and classify objects in the road as they clean.

To do so without a human at the wheel, these vehicles must process massive amounts of data from onboard sensors in real time. Redundant and diverse deep neural networks (DNNs) must work together to accurately perceive relevant information from this sensor data.

As a high-performance, software-defined AI compute platform, NVIDIA’s solution is designed to handle the large number of applications and DNNs that run simultaneously in autonomous vehicles, while achieving systemic safety standards.

A Model Lineup

The WeRide Robo Street Sweepers are the latest in the company’s stable of autonomous vehicles and its second purpose-built and mass-produced self-driving vehicle model.

WeRide has been developing autonomous technology on NVIDIA since 2017, building robotaxis, mini robobuses and robovans with the goal of accelerating intelligent urban transportation.

Its robotaxis have already provided more than 350,000 rides for 180,000 passengers since 2019, while its mini robobuses began pilot operations to the public in January.

The company is currently building its next-generation self-driving solutions on NVIDIA DRIVE Orin, using the high-performance AI compute platform to commercialize its autonomous lineup.

And with the addition of these latest vehicles, WeRide’s fleets are set to make a clean sweep.

Bengal tigers, red pandas and mountain gorillas are among the world’s most familiar endangered species, but tens of thousands of others — like the Karpathos frog, the Perote deer mouse or the Mekong giant catfish — are largely unknown.

Typically perceived as lacking star quality, these species are now roaming massive billboards in one of the world’s busiest destinations. An AI-powered initiative is spotlighting lesser-known endangered creatures on Times Square billboards this month, nightly in the few minutes before midnight across nearly 100 screens.

The project, dubbed Critically Extant, uses AI to illustrate the limited public data available on critically endangered flora and fauna. It’s the first deep learning art display in the Times Square Arts program’s decade-long history.

“A neural network can only create images based on what it’s seen in training data, and there’s very little information online about some of these critically endangered species,” said artist Sofia Crespo, who created the work with support from Meta Open Arts, using NVIDIA GPUs for AI training and inference. “This project is ultimately about representation — for us to recognize that we are biased towards some species versus others.”

These biases in representation have implications on the effort and funding given to save different species. Research has shown that a small subset of endangered species that are considered charismatic, cute or marketable receive more funding than they need, while most others receive little to no support.

When endangered species of any size — such as insects, fungi or plants — are left without conservation resources, they’re more vulnerable to extinction, contributing to a severe loss of biodiversity that makes ecosystems and food webs less resilient.

Intentionally Imperfect Portraits

The AI model, created by Crespo and collaborator Feileacan McCormick, was trained on a paired dataset of nearly 3 million nature images and text describing around 10,000 species. But this still wasn’t enough data to create true-to-life portraits of the less popular endangered species.

So the deep learning model, a generative adversarial network, does the best it can, guessing the features of a given endangered species based on related species. Due to the limited source data, many of the AI-generated creatures have a different color or body shape than their real-life counterparts — and that’s the point.

“Part of the project was relying on the open-source data that’s available right now,” said Crespo. “If that’s all the data we have, and species go extinct, what kind of knowledge and imagination do we have about the world that was lost?”

Critically Extant features more than 30 species, including amphibians, birds, fish, flowering plants, fungi and insects. After feeding species names to the generative AI model, Crespo animated and processed the synthetic images further to create the final moving portraits.

The AI model behind this project was trained using a cluster of NVIDIA Tensor Core GPUs. Crespo used a desktop NVIDIA RTX A6000 GPU for what she called “lightning-quick” inference.

AI in the Public Square

Critically Extant’s Times Square display premiered on May 1 and will be shown nightly through the end of the month.

The three-minute display features all 30+ specimens in a randomized arrangement that shifts every 30 seconds or so. Crespo said that using the NVIDIA RTX A6000 GPU was essential to generate the high-resolution images needed to span dozens of digital billboards.

Crespo and McCormick, who run an ecology and AI-focused studio, also enhanced the art display with an AI-generated soundtrack trained on a diverse range of animal sounds.

“The idea is to show diversity with many creatures, and overwhelm the audience with creatures that look very different from one another,” Crespo said.

The project began as an exhibition on Instagram, with the goal of adding representation of critically endangered species to social media conversations. At Times Square, the work will reach an audience of hundreds of thousands more.

“Crespo’s work brings the natural world directly into the center of the very urban environment at odds with these at-risk species, and nods to the human changes that will be required to save them,” reads the Times Square Arts post.

Crespo and McCormick have showcased their work at NVIDIA GTC, most recently an AI-generated fragment of coral reef titled Beneath the Neural Waves.

Learn more about AI artwork by Crespo and McCormick on the NVIDIA AI Art Gallery, and catch Critically Extant in Times Square through May 31.

Times Square images courtesy of Times Square Arts, photographed by Michael Hull. Artwork by Sofia Crespo. 

Good. Bad. You’re the Guy With the Gun this GFN Thursday.

Get ready for some horrifyingly good fun with Evil Dead: The Game streaming on GeForce NOW tomorrow at release. It’s the 1,300th game to join GeForce NOW, joining on Friday the 13th.

And it’s part of eight total games joining the GeForce NOW library this week.

Hail to the King, Baby

Step into the shoes of Ash Williams and friends from the iconic Evil Dead franchise in Evil Dead: The Game (Epic Games Store), streaming on GeForce NOW at release tomorrow.

Work together in a game loaded with over-the-top co-op and PvP action across nearly all your devices. Grab your boomsticks, chainsaws and cleavers to fight against the armies of darkness, even on a Mac. Or take control of the Kandarian Demon to hunt the heroes by possessing Deadites, the environment and even the survivors themselves with a mobile phone.

For RTX 3080 members, the horror comes to life with realistic visuals and a physics-based gore system, enhanced by NVIDIA DLSS – the groundbreaking AI rendering technology that increases graphics performance by boosting frame rates and generating beautiful, sharp images.

Plus, everything is better in 4K. Whether you’re tearing a Deadite in two with Ash’s chainsaw hand or flying through the map as the Kandarian Demon, RTX 3080 members playing from the PC and Mac apps can bring the bloodshed in all its glory, streaming at up to 4K resolution and 60 frames per second.

There’s No Time Like Playtime

Not a spooky fan? That’s okay. There’s fun for everyone with eight new games streaming this week:

• Achilles: Legends Untold (New release on Steam)
• Brigandine The Legend of Runersia (New release on Steam)
• Neptunia x SENRAN KAGURA: Ninja Wars (New release on Steam)
• Songs of Conquest (New release on Steam and Epic Games Store)
• Cepheus Protocol Anthology (New release on Steam, May 13)
• Evil Dead: The Game (New release on Epic Games Store, May 13)
• Pogostuck: Rage With Your Friends (Steam)
• Yet Another Zombie Defense HD (Steam)

With the armies of darkness upon us this weekend, we’ve got a question for you. Let us know how your chances are looking on Twitter or in the comments below.

Quick! You have to fight an army of darkness.

The first thing to your left is your weapon.

How are your chances looking?

— NVIDIA GeForce NOW (@NVIDIAGFN) May 11, 2022

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology accelerates creative workflows.

The future of content creation is in AI. This week In the NVIDIA Studio, discover how AI-assisted painting is bringing a new level of inspiration to the next generation of artists.

San Francisco-based creator Karen X. Cheng is on the forefront of using AI to design amazing visuals. Her innovative work produces eye-catching effects to social media videos for brands like Adobe, Beats by Dre and Instagram.

Cheng’s work bridges the gap between emerging technologies and creative imagery, and her inspiration can come from anywhere. “I usually get ideas when I’m observing things — whether that’s taking a walk or scrolling in my feed and seeing something cool,” she said. “Then, I’ll start jotting down ideas and sketching them out. I’ve got a messy notebook full of ideas.”

When inspiration hits, it’s important to have the right tools. Cheng’s ASUS Zenbook Pro Duo — an NVIDIA Studio laptop that comes equipped with up to a GeForce RTX 3080 GPU — gives her the power she needs to create anywhere.

Paired with the NVIDIA Canvas app, a free download available to anyone with an NVIDIA RTX or GeForce RTX GPU, Cheng can easily create and share photorealistic imagery. Canvas is powered by the GauGAN2 AI model and accelerated by Tensor Cores found exclusively on RTX GPUs.

The app uses AI to interpret basic lines and shapes, translating them into realistic landscape images and textures. Artists of all skill levels can use this advanced AI to quickly turn simple brushstrokes into realistic images, speeding up concept exploration and allowing for increased iteration, while freeing up valuable time to visualize ideas.

“I’m excited to use NVIDIA Canvas to be able to sketch out the exact landscapes I’m looking for,” said Cheng. “This is the perfect sketch to communicate your vision to an art director or location scout. I never had much drawing skill before, so I feel like I have art superpowers with this thing.”

Cheng plans to put these superpowers to the test in an Instagram live stream on Thursday, May 12, where she and her AI Sketchpad collaborator Don Allen Stevenson III will race to paint viewer challenges using Canvas.

Tune in to contribute, and download NVIDIA Canvas to see how easy it is to paint by AI.

With AI, Anything Is Possible

Empowering scribble-turn-van Gogh painting abilities is just one of the ways that NVIDIA Studio is transforming creative technology through AI.

NVIDIA Broadcast uses AI running on RTX GPUs to improve audio and video for broadcasters and live streamers. The newest version can run multiple neural networks to apply background removal, blur and auto-frame for webcams, and remove noise from incoming and outgoing sound.

3D artists can take advantage of AI denoising in Autodesk Maya and Blender software, refine color detail across high-resolution RAW images with Lightroom’s Enhance Details tool, enable smooth slow motion with retained b-frames using DaVinci Resolve’s SpeedWarp and more.

NVIDIA AI researchers are working on new models and methods to fuel the next generation of creativity. At GTC this year, NVIDIA debuted Instant NeRF technology, which uses AI models to transform 2D images into high-resolution 3D scenes, nearly instantly.

Instant NeRF is an emerging AI technology that Cheng already plans to implement. She and her collaborators have started experimenting with bringing 2D scenes to 3D life.

More AI Tools In the NVIDIA Studio

AI is being used to tackle complex and incredibly challenging problems. Creators can benefit from the same AI technology that’s applied to healthcare, automotive, robotics and countless other fields.

The NVIDIA Studio YouTube channel offers a wide range of tips and tricks, tutorials and sessions for beginning to advanced users.

CGMatter hosts Studio speedhack tutorials for beginners, showing how to use AI viewport denoising and AI render denoising in Blender.

Many of the most popular creative applications from Adobe have AI-powered features to speed up and improve the creative process.

Neural Filters in Photoshop, Auto Reframe and Scene Edit Detection in Premiere Pro, and Image to Material in Substance 3D all make creating quicker and easier through the power of AI.

Follow NVIDIA Studio on Instagram, Twitter and Facebook; access tutorials on the Studio YouTube channel; and get updates directly in your inbox by signing up for the NVIDIA Studio newsletter.

What’s your favorite NVIDIA Canvas creation?

Show us below.

— NVIDIA Studio (@NVIDIACreators) May 9, 2022

Working as an aerospace engineer in Malaysia, Chee How Lim dreamed of building a startup that could really take off. Today his company, Tapway, is riding a wave of computer vision and AI adoption in Southeast Asia.

A call for help in 2019 with video analytics led to the Kuala Lumpur-based company’s biggest project to date.

Malaysia’s largest operator of toll highways, PLUS, wanted to reduce congestion for its more than 1.5 million daily travelers. A national plan called for enabling car, taxi, bus and truck traffic to flow freely across multiple lanes — but that posed several big challenges.

Unsnarling Traffic Jams

The highways charge five classes of tolls depending on vehicle type. Drivers pay using four different systems, and often enter the highway using one payment system, then exit using another, making it hard to track vehicles.

Dedicated lanes for different vehicle classes forced users to stop, slowing traffic, so too booth operators could identify the specific vehicle. Even then some drivers scammed the system, exchanging cards on the highway to get lower tolls.

“We showed them how with computer vision — just a camera and AI — you could solve all that,” said Lim.

AI Smooths the Flow

Using NVIDIA GPUs and software, Tapway trained and ran AI models that could read a vehicle’s license plate and detect its class, make and color in just 50 milliseconds, about one tenth of one eye blink — even if it’s traveling at up to 40 kilometers/hour while approaching a toll plaza.

Tapway’s VehicleTrack software works in all light and weather conditions with a consistent 97 percent accuracy. And thanks in part to NVIDIA Triton Inference Server, a single GPU can manage up to 50 simultaneous video streams.

PLUS has installed 577 cameras so far, and plans to expand to nearly 900 in 92 toll plazas to meet its goal of freely flowing traffic.

Inside a Computer Vision System

Under the hood, the system depends on smart AI models trained in the cloud on a network of NVIDIA A100 and V100 Tensor Core GPUs.

They use a dataset of up to 100,000 images to prepare a new model for a Tapway customer in a few hours, a huge improvement on a CPU-based system that used to take several days, Lim said.

But the real magic comes with inference, running those models in production to process up to 28,800 images a minute on edge servers using NVIDIA A10, A30 and T4 GPUs.

Software Makes it Sing

Tapway uses the NVIDIA DeepStream software development kit to build its computer vision apps, NVIDIA TensorRT to keep its AI models lean and fast, and Triton to play traffic cop, directing AI inference jobs.

“Triton is a real lifesaver for us,” said Lim. “We had some scaling problems doing inference and multithreading on our own and couldn’t scale beyond 12 video streams in a server, but with Triton we easily handle 20 and we’ve tested it on up to 50 simultaneous streams,” he said.

In February, Tapway officially became an NVIDIA Metropolis partner. The program gives companies in intelligent video analytics early access to technology and expertise.

“We had to pass stress tests in areas like multistreaming and security — that helped us strengthen our product offering — and from a business perspective it’s a way to be recognized and further establish ourselves as an AI expert in the region,” Lim said.

AI Covers the Waterfront

Since its start in 2014, Tapway has deployed 3,000 sensors in 500 locations throughout Malaysia and Singapore. Off the road, they help malls and retailers understand customer shopping habits, and now the company is gearing up to help manufacturers like the region’s car makers and palm oil producers inspect products for quality control.

“The demand has never been better, there are a lot of vision challenges in the world, and quite a few exciting projects we hope to land soon,” he said.

To learn more, watch Lim’s talk at GTC (free with registration). And download this free e-book to learn how NVIDIA Metropolis is helping build smarter and safer spaces around the world.