Posts in category: nVidia AI

Cooking on video calls with friends. Getting to the end of Netflix’s endless content well. Going 10 months without a haircut.

Over the past year, we all found different ways to keep ourselves occupied.

Gaming, however, is a longer-term trend that promises to continue remaking global culture for years to come.

Over 2.5 billion gamers are now engaged in playing, creating, sharing and connecting with one another.

Together we watched over 100 billion hours of gaming content on YouTube — twice as much as 2018. That’s like 11 million people watching non-stop for a year.

And esports viewership is nearly half a billion people globally — up 100 million from 2018 — with another 150 million new viewers expected over the next three years.

Building on a long-term surge that has made gaming an integral part of all our lives over the past decade, market researcher Newzoo expects gaming revenue to rise 8 percent in 2021.

The Gaming Market Set New Records (Again)

And gaming revenue — projected by analysts at $175 billion last year — already towers over other global consumer entertainment markets such as music and radio, internet-based or “over the top” video of all kinds, and cinema.

There are now 2.6 billion people playing — over half the global online population.

The amount of time people played grew dramatically, with gameplay time up 26 percent in the U.S. over the past six months.

Participation on Discord — the leading real-time voice chat app for gamers — is up 2x in the past two years, with 140 million monthly active users who connect in over 4 billion minutes of conversation each day.

The Thrill of Victory

Today’s games are getting more realistic and immersive, from lifelike graphics to AI-based gameplay and realistic physics simulation.

And to play the latest titles, gamers want the latest hardware — whether a recently launched console or a PC with the latest graphics technology, NVIDIA GeForce RTX.

Two years ago, NVIDIA introduced a breakthrough in graphics — real-time ray tracing for the ultimate realism and AI-based DLSS (deep learning super sampling) to “magically” improve performance.

We named it RTX.

Together with Microsoft and top developers and game engines, we’re working to bring the visual realism of movies to fully interactive gaming.

The momentum is unstoppable. The latest consoles and the rest of the gaming ecosystem are now onboard: Ray tracing is the new standard. There are already over 30 titles with ray tracing and many more on the way. You can find a list of select titles here.

And some of the most popular games over the past few years — Minecraft, with over 200 million copies sold and 126 million people playing per month, and Fortnite, with over 350 million registered users — have introduced real-time ray tracing enhancements to their games, enabling much more realistic and immersive experiences.

New Hardware Enables Better Games

The hits keep on coming.

Cyberpunk 2077, released a few weeks ago, generated 8 million pre-orders (almost 5 million for PC), and sold 13 million copies in its first 10 days. It shattered the single-player record for number of concurrent Steam users within two hours of launch with over 1 million.

Cyberpunk 2077 is just one example of how production value of games continues to increase. Bigger worlds and cinematic graphics demand more of the GPU. A survey of GeForce gamers showed that 59 percent of GPU or PC upgrades are due to the low performance of a game they’re playing or requirements for one that they’re anticipating.

This increase in production value is evident in the latest sequels to these AAA games, where the GeForce GTX 1060, one of our most popular GPUs of all time, struggles to keep up.

For instance, gamers were playing Watch Dogs 2 (released in 2016) with high settings at 60 frames per second on a GTX 1060. But when they moved on to Watch Dogs: Legion, released in October, they only saw 24 frames per second on that system.

And turning on ray tracing to get the best visuals makes Watch Dogs: Legion virtually unplayable on a GTX 1060.

Fear not, however. The just-announced GeForce RTX 3060 based on the NVIDIA Ampere architecture brings the game back to life, at a price that makes RTX technology available to all PC gamers.

With tens of millions of GeForce GTX GPUs in use today, the upgrade opportunity is enormous. The performance gains and new features in GeForce RTX 30 Series GPUs make it a great time for all PC gamers to upgrade.

Competitive Gaming Takes Center Stage

Weekend basketball player? We already know you love the NBA. It’s no surprise, then, that hundreds of millions of video game players like to watch pros play their games, too.

There are professional leagues for top games like Counter-Strike, League of Legends and Dota 2. The stakes in these contests are serious. In 2019, the prize pool for the Dota 2 International tournament reached $34 million. That’s more than triple the $11 million prize purse for the U.S. Masters golf tournament.

When the stakes are high, quality gear matters. Response time is critical to be most competitive at esports games — typically shooters — like Counter-Strike or Fortnite. Every millisecond matters.

Esports pros and enthusiasts strive for the lowest latency — down to zero if they could get there. So NVIDIA invented Reflex to help get the latency of the system as low as possible.

NVIDIA Reflex optimizes the rendering pipeline across the CPU and GPU to reduce latency by up to 50 percent, giving gamers a precious advantage. A 20-millisecond advantage can mean the difference between winning and losing in the physical sports world and esports.

With Reflex and our Game Ready Drivers, over 100 million GeForce gamers are instantly more competitive. Fortnite, Apex Legends, Valorant and Call of Duty: Warzone were among the first to integrate Reflex technology. With this week’s announcements, seven of the top 10 competitive shooter games now support Reflex. And more are on the way.

It’s no wonder that 99 percent of esports pros play on GeForce GPUs.

You Can Take It With You

Laptops are the fastest growing gaming platform, increasing 7x in seven years. And the power of these laptops has inspired gamers to find new ways to play, whether in console mode connected to a big screen TV using a controller, or driving an ultra-wide monitor with a keyboard and mouse.

Laptop PCs were flying off shelves so fast last year, manufacturers couldn’t keep up with demand. Laptop PC sales were up 26 percent over 2019, the largest gain in many years. It would have been even higher if more supply were available.

Many laptop buyers are looking for their PCs to perform a wide variety of activities — from keeping up on social media and streaming videos to creating and editing videos and playing the latest games. They want as much performance as they can get without having to sacrifice portability.

To address that demand, we’ve just announced that the GeForce RTX 30 Series, powered by the NVIDIA Ampere architecture, will soon be available in laptops. These laptops can handle all the latest tasks and are available in slim designs using our third-generation Max-Q technologies.

RTX 30 Series laptops bring exceptional power to gamers and creators, with the best laptops for creators meeting the specifications of our NVIDIA Studio program.

And there are laptops available specifically for esports players that include 240Hz displays for fast response and low latency. Gamers can compete at the highest level on these devices.

This week we announced that our partners are introducing over 70 laptops — it’s our biggest GeForce laptop launch ever. These are the world’s fastest laptops that give gamers and creators a huge variety to choose the right device for their needs.

More to Come

The GeForce RTX 30 Series GPUs shipping in desktops and, very shortly, in laptops, coupled with ray tracing in games, will fuel the next round of gaming PCs and upgrades.

Over the past two decades, GPUs have revolutionized modern graphics again and again. Once the holy grail of computer graphics, ray tracing is now the standard.

We look forward to more revolutions to come: if the last 20 years of graphics and gaming were amazing, the next 20 will seem nothing short of science fiction.

Bringing more gaming capabilities to millions more gamers, NVIDIA on Tuesday  announced more than 70 new laptops will feature GeForce RTX 30 Series Laptop GPUs and unveiled the NVIDIA GeForce RTX 3060 graphics card for desktops.

All are powered by the award-winning NVIDIA Ampere GPU architecture, the second generation of RTX with enhanced Ray Tracing Cores, Tensor Cores, and new streaming multiprocessors.

The announcements were among the highlights of a streamed presentation from Jeff Fisher, senior vice president of NVIDIA’s GeForce business.

Amid the unprecedented challenges of 2020, “millions of people tuned into gaming — to play, create and connect with one another,” Fisher said. “More than ever, gaming has become an integral part of our lives.” Among the stats he cited:

• Steam saw its number of concurrent users more than double from 2018
• Discord, a messaging and social networking service most popular with gamers, has seen monthly active users triple to 140 million from two years ago
• In 2020 alone, more than 100 billion hours of gaming content have been watched on YouTube
• Also in 2020, viewership of esports reached half a billion people

Meanwhile, NVIDIA has been delivering a series of major gaming advancements, Fisher explained.

RTX ‘the New Standard’

Two years ago, NVIDIA introduced a breakthrough in graphics real-time ray tracing and AI-based DLSS (deep learning super sampling), together called RTX, he said.

NVIDIA quickly partnered with Microsoft and top developers and game engines to bring the visual realism of movies to fully interactive gaming, Fisher said.

In fact, 36 games are now powered by RTX. They include the #1 Battle Royale game, the #1 RPG, the #1 MMO and the #1 best-selling game of all time – Minecraft.

Now, we’re announcing more games that support RTX technology, including DLSS, which is coming to both Call of Duty: Warzone and Square Enix’s new IP, Outriders. And Five Nights at Freddy’s: Security Breach and F.I.S.T.: Forged in Shadow Torch will be adding ray tracing and DLSS.

For more details, read our January 2021 RTX Games article.

“The momentum is unstoppable,” Fisher said. “As consoles and the rest of the ecosystem are now onboard — ray tracing is the new standard.

last year, NVIDIA launched its second generation of RTX, the GeForce RTX 30 Series GPUs. Based on the NVIDIA Ampere architecture, it represents “our biggest generational leap ever,” Fisher said.

NVIDIA built NVIDIA Reflex to deliver the lowest system latency for competitive gamers – from mouse click to display. Since Reflex’s launch in September, a dozen games have added support.

Fisher announced that Overwatch and Rainbow Six Siege are also adopting NVIDIA Reflex. Now, 7 of the top 10 competitive shooters support Reflex.

And over the past four months, NVIDIA has launched four NVIDIA Ampere architecture-powered graphics cards, from the ultimate BFGPU — the GeForce RTX 3090 priced at $1,499 — to the GeForce RTX 3060 Ti at $399.

“Ampere has been our fastest selling architecture ever, selling almost twice as much as our prior generation,” Fisher said.

GeForce RTX 3060: An NVIDIA Ampere GPU for Every Gamer

With gaming now a key part of global culture, the new GeForce RTX 3060 brings the power of the NVIDIA Ampere architecture to every gamer, Fisher said.

“The RTX 3060 offers twice the raster performance of the GTX 1060 and 10x the ray-tracing performance,” Fisher said, noting that the GTX 1060 is the world’s most popular GPU. “The RTX 3060 powers the latest games with RTX On at 60 frames per second.”

The RTX 3060 has 13 shader teraflops, 25 RT teraflops for ray tracing, and 101 tensor teraflops to power DLSS, an NVIDIA technology introduced in 2019 that uses AI to accelerate games. And it boasts 12 gigabytes of GDDR6 memory.

“With most of the installed base underpowered for the latest games, we’re bringing RTX to every gamer with the GeForce RTX 3060,” Fisher said.

The GeForce RTX 3060 starts at just $329 and will be available worldwide in late February.

“Amazing Gaming Doesn’t End at the Desktop”

NVIDIA also announced a new generation of NVIDIA Ampere architecture-powered laptop GPUs.

Laptops, Fisher explained, are the fastest-growing gaming platform. There are now 50 million gaming laptops, which powered over 14 billion gaming hours last year.

“Amazing gaming doesn’t end at the desktop,” Fisher said.

These high-performance machines also meet the needs of 45 million creators and everyone working and studying from home, Fisher said.

The new generation of NVIDIA Ampere architecture-powered laptops, with second-generation RTX and third-generation Max-Q technologies, deliver twice the power efficiency of previous generations.

Efficiency Is Paramount in Laptops

That’s why NVIDIA introduced Max-Q four years ago, Fisher explained.

Max-Q is a system design approach that delivers high performance in thin and light gaming laptops.

“It has fundamentally changed how laptops are built, every aspect — the CPU GPU, software, PCB design, power delivery, thermals — are optimized for power and performance,” Fisher said.

NVIDIA’s third-gen Max-Q technologies use AI and new system optimizations to make high-performance gaming laptops faster and better than ever, he said.

Fisher introduced Dynamic Boost 2.0, which for the first time uses AI to shift power between the CPU, GPU and now, GPU memory.

“So your laptop is constantly optimizing for maximum performance,” Fisher said.

Fisher also introduced WhisperMode 2.0, which delivers a new level of acoustic control for gaming laptops.

Pick your desired acoustics and WhisperMode 2.0’s AI-powered algorithms manage the CPU, GPU system temperature and fan speeds to “deliver great acoustics at the best possible performance,” Fisher explained.

Another new feature, Resizable BAR, uses the advanced capabilities of PCI Express to boost gaming performance.

Games use GPU memory for textures, shaders and geometry — constantly updating as the player moves through the world.

Today, only part of the GPU’s memory can be accessed at any one time by the CPU, requiring many memory updates, Fisher explained.

With Resizable BAR, the game can access the entire GPU memory, allowing for multiple updates at the same time, improving performance, Fisher said.

Resizable BAR will also be supported on GeForce RTX 30 Series graphics cards for desktops, starting with the GeForce RTX 3060. NVIDIA and GPU partners are readying VBIOS updates for existing GeForce RTX 30 series graphics cards starting in March.

Finally, NVIDIA DLSS offers a breakthrough for gaming laptops. It uses AI and RTX Tensor Cores to deliver up to 2x to performance in the same power envelope.

World’s Fastest Laptops for Gamers and Creators

Starting at $999, RTX 3060 laptops are “faster than anything on the market today,” Fisher said.

They’re 30 percent faster than the PlayStation 5 and deliver 90 frames per second on the latest games at ultra settings 1080p, Fisher said.

Starting at $1,299, GeForce RTX 3070 laptops are “a 1440p gaming beast.”

Boasting twice the pixels of 1080p, this new generation of laptops “provides the perfect mix of high-fidelity graphics and great performance.”

And starting at $1,999, GeForce RTX 3080 laptops will come with up to 16 gigabytes of GDDR6 memory.

They’re “the world’s fastest laptop for gamers and creators,” Fisher said, delivering hundreds of frames per second with RTX on.

As a result, laptop gamers will be able to play at 240 frames per second, across top titles like Overwatch, Rainbow Six Siege, Valorant and Fortnite, Fisher said.

Availability

Manufacturers worldwide, starting Jan. 26, will begin shipping over 70 different GeForce RTX gaming and creator laptops featuring GeForce RTX 3080 and GeForce RTX 3070 laptop GPUs, followed by GeForce RTX 3060 laptop GPUs on Feb. 2.

The GeForce RTX 3060 graphics card will be available in late February, starting at $329, as custom boards — including stock-clocked and factory-overclocked models — from top add-in card providers such as ASUS, Colorful, EVGA, Gainward, Galaxy, Gigabyte, Innovision 3D, MSI, Palit, PNY and Zotac.

Look for GeForce RTX 3060 GPUs at major retailers and etailers, as well as in gaming systems by major manufacturers and leading system builders worldwide.

“RTX is the new standard, and the momentum continues to grow,” Fisher said.

The latest NVIDIA Studio laptops, powered by new NVIDIA GeForce RTX 30 Series Laptop GPUs, are empowering the next generation of creativity. And they bring a host of updates to change how fast creators work.

New models come equipped with up to 16GB of video memory, pixel-accurate displays with 1440p and 4K options, and GPU acceleration for ray tracing, AI and video encoding that allow artists to create in record time.

NVIDIA Studio laptops with new RTX 30 Series Laptop GPUs offer improved performance.

Built to Create, Anywhere

GeForce RTX 30 Series Laptop GPUs accelerate rendering performance by up to 2x and introduce third-generation Max-Q technologies. Max-Q helps manufacturers develop thin and light laptops that maintain great performance, enabling creators to conceptualize incredible pieces of content using hundreds of GPU-accelerated apps.

Creator workflows require multiple PC components — the GPU, GPU memory and CPU — to execute rapidly. New, AI-powered Dynamic Boost 2.0 intelligently shifts power among these components, improving performance so laptops run faster and longer.

NVIDIA Studio laptops are designed to provide the best performance and acoustics, but sometimes you need your laptop to be even quieter while giving a presentation or working in a public place. WhisperMode 2.0 uses AI-powered algorithms to dynamically manage the CPU, GPU and fan speeds to deliver great acoustics and the best possible performance.

Creators need to have pixel-accurate displays to preview with the maximum fidelity. NVIDIA Studio laptops use factory-calibrated displays with wide color gamuts and up to 4K resolution. And now, many NVIDIA Studio laptops offer 1440p configurations for the perfect balance between pixel density and performance.

The New 30 Series Studio Laptops

New Studio laptops with GeForce RTX 3060, 3070 and 3080 Laptop GPUs will start rolling out later this month. The first models announced include:

• Asus ZenBook Pro Duo UX582 — featuring GeForce RTX 3070 graphics. Artists can create their best thanks to next-gen performance and a stunning dual display, giving them new ways of using a laptop.

• MSI Creator 15 — offering GeForce RTX 3060, 3070 and 3080 variants. MSI’s built-for-creators laptop delivers blazing-fast performance with hushed tones as it leverages both Dynamic Boost 2.0 and WhisperMode 2.0.

• Gigabyte Aero 15 and Aero 17 — with multiple screen sizes to choose from and configurable with GeForce RTX 3060, 3070 or 3080 graphics. The Aero 15 features Dynamic Boost 2.0, WhisperMode 2.0 and Advanced Optimus (on the 1080p display) to deliver peak performance while on the go. The larger format Aero 17 also features Dynamic Boost 2.0.

• Razer Blade 15 and Blade Pro 17 — offer stunning 1440p and 4K display options in an elegant, high-quality chassis. The Blade 15 Base model is configurable with a GeForce RTX 3070 and a 1440p display; the Razer Blade Advanced model with GeForce RTX 3070 or 3080 and 1440p or 4K displays; the Blade Pro 17 with a GeForce RTX 3080 and 4K 120Hz display.

Accelerating Creativity

GeForce RTX 30 Series Studio laptops give video editors mobile access to edit up to 8K HDR RAW footage, use AI to simplify workflows, and reduce encode times by up to 75 percent with the NVIDIA Encoder. When connected to an external G-SYNC display — an NVIDIA technology that synchronizes the display to content — editors can preview their content at the exact frame rate it will be exported and played back.

Artists can get up to 16GB of graphics memory to work with huge assets and across multiple apps at the same time.

AI-Powered Workflows

NVIDIA Studio laptops take advantage of industry-leading creative apps to empower creators with AI.

Adobe Photoshop recently introduced Neural Filters, which use AI to quickly make complex edits to photographs.

Blackmagic Design’s Davinci Resolve 17 implements a new Magic Mask feature that uses AI to speed up mask creation and tracking.

The free NVIDIA Broadcast app turns any room into a home studio by enhancing your microphone, speakers and webcam with AI-powered features such as virtual background effects, audio noise removal and webcam auto frame.

NVIDIA Optix AI denoising in Blender cleans up noise on the fly, quickly resolving 3D images.

And NVIDIA Omniverse Audio2Face uses AI to animate unrigged 3D characters, based solely on an audio source.

Studio Drivers

NVIDIA Studio laptops are supported with Studio Drivers that are built specifically to meet creators’ needs for both performance and reliability. They’re tested extensively against top creative apps and workflows.

The latest Studio Driver can be downloaded through GeForce Experience or from our driver download page.

New GeForce RTX 30 Series NVIDIA Studio laptops from ASUS, Gigabyte, MSI and Razer will begin rolling out later this month.

Read more about the GeForce RTX 30 Series laptop announcement, including additional options for gamers and creators starting at $999. Visit the GeForce Laptop website for even more information.

Learn more about NVIDIA Studio hardware and software for creators on the NVIDIA Studio website.

And stay up to date on the latest apps through the NVIDIA Studio YouTube channel, which features tutorials, tips and tricks by industry-leading artists.

We’re at a pivotal moment to unlock a new, AI-accelerated era of discovery and medicine, says Kimberly Powell, NVIDIA’s vice president of healthcare.

Speaking today at the J.P. Morgan Healthcare conference, held virtually, Powell outlined how AI and accelerated computing are enabling scientists to take advantage of the boom in biomedical data to power faster research breakthroughs and better patient care.

Understanding disease and discovering therapies is our greatest human endeavor, she said — and the trillion-dollar drug discovery industry illustrates just how complex a challenge it is.

How AI Can Drive Down Drug Discovery Costs

The typical drug discovery process takes about a decade, costs $2 billion and suffers a 90 percent failure rate during clinical development. But the rise of digital data in healthcare in recent years presents an opportunity to improve those statistics with AI.

“We can produce today more biomedical data in about three months than the entire 300-year history of healthcare,” she said. “And so this is now becoming a problem that no human really can synthesize that level of data, and we need to call upon artificial intelligence.”  

Powell called AI “the most powerful technology force of our time. It’s software that writes software that no humans can.”

But AI works best when it’s domain specific, combining data and algorithms tailored to a specific field like radiology, pathology or patient monitoring. The NVIDIA Clara application framework bridges this gap by providing researchers and clinicians the tools for GPU-accelerated AI in medical imaging, genomics, drug discovery and smart hospitals.

Downloads of NVIDIA Clara grew 5x last year, Powell shared, with developers taking up our new platforms for conversational AI and federated learning.

Healthcare Ecosystem Rallies Around AI

She noted that amid the COVID-19 pandemic, momentum around AI for healthcare has accelerated, with startups estimated to have raised well over $5 billion in 2020. More than 1,000 healthcare startups are in the NVIDIA Inception accelerator program, up 4x since 2017. And over 20,000 AI healthcare papers were submitted last year to PubMed, showing exponential growth over the past decade.

Leading research institutions like the University of California, San Francisco, are using NVIDIA GPUs to power their work in cryo-electron microscopy, a technique used to study the structure of molecules — such as the spike proteins on the COVID-19 virus — and accelerate drug and vaccine discovery.

And pharmaceutical companies, including GlaxoSmithKline, and major healthcare systems, like the U.K.’s National Health Service, will harness the Cambridge-1 supercomputer — an NVIDIA DGX SuperPOD system and the U.K.’s fastest AI supercomputer — to solve large-scale problems and improve patient care, diagnosis and delivery of critical medicines and vaccines.

Software-Defined Instruments Link AI Innovation and Medical Practice

Powell sees software-defined instruments — devices that can be regularly updated to reflect the latest scientific understanding and AI algorithms — as key to connecting the latest research breakthroughs with the practice of medicine.

“Artificial intelligence, like the practice of medicine, is constantly learning. We want to learn from the data, we want to learn from the changing environment,” Powell said.

By making medical instruments software-defined, tools like smart cameras for patient monitoring or AI-guided ultrasound systems can not only be developed in the first place, she said, but also retain their value and improve over time.

U.K.-based sequencing company Oxford Nanopore Technologies is a leader in software-defined instruments, deploying a new generation of DNA sequencing technology across an electronics-based platform. Its nanopore sequencing devices have been used in more than 50 countries to sequence and track new variants of the virus that causes COVID-19, as well as for large-scale genomic analyses to study the biology of cancer.

The company uses NVIDIA GPUs to power several of its instruments, from the handheld MinION Mk1C device to its ultra-high throughput PromethION, which can produce more than three human genomes’ worth of sequence data in a single run. To power the next generation of PromethION, Oxford Nanopore is adopting NVIDIA DGX Station, enabling its real-time sequencing technology to pair with rapid and highly accurate genomic analyses.

For years, the company has been using AI to improve the accuracy of basecalling, the process of determining the order of a molecule’s DNA bases from tiny electrical signals that pass through a nanoscale hole, or nanopore.

This technology “truly touches on the entire practice of medicine,” Powell said, whether COVID epidemiology or in human genetics and long read sequencing. “Through deep learning, their base calling model is able to reach an overall accuracy of 98.3 percent, and AI-driven single nucleotide variant calling gets them to 99.9 percent accuracy.”

Path Forward for AI-Powered Healthcare

AI-powered breakthroughs like these have grown in significance amid the pandemic, said Powell.

“The tremendous focus of AI on a single problem in 2020, like COVID-19, really showed us that with that tremendous focus, we can see every piece and part that can benefit from artificial intelligence,” she said. “What we’ve discovered over the last 12 months is only going to propel us further in the future. Everything we’ve learned is applicable for every future drug discovery program there is.”

Across fields as diverse as genome analysis, computational drug discovery and clinical diagnostics, healthcare heavyweights are making strides with GPU-accelerated AI. Hear more about it on Jan. 13 at 11 a.m. Pacific, when Powell joins a Washington Post Live conversation on AI in healthcare.

Subscribe to NVIDIA healthcare news here.

When photographers take long-exposure photos, they maximize the amount of light their camera sensors receive. The technique helps capture scenes like the night sky, but it introduces blurring in the final image, as in the example at right.

It’s not too different from cryo-electron microscopy, or cryo-EM, which scientists use to study the structure of tiny molecules frozen in vitreous ice. But while motion-induced blur in photography can create beautiful images, in structural biology it’s an unwanted side effect.

Protein samples for cryo-EM are frozen at -196 degrees Celcius to protect the biological structures, which would otherwise be destroyed by the microscope’s high-energy electron beam. But even when frozen, samples are disturbed by the powerful electron dose, causing motion that would blur a long-exposure photo.

To get around it, UCSF researchers use specialized cameras to instead capture videos of the biological molecules, so they appear nearly stationary in each frame of the video. Correcting the motion across frames is a computationally demanding task — but can be done in seconds on NVIDIA GPUs.

“If the motion was left uncorrected, we’d lose the high-resolution picture of a molecule’s 3D structures,” said Shawn Zheng, scientific software developer at the University of California, San Francisco and Howard Hughes Medical Institute. “And knowing the structure of a molecule is critical to understanding its function.”

Zheng and his colleagues run MotionCor2, the world’s most widely used motion-correction application, on NVIDIA GPUs to align each molecule in the video from frame to frame — creating a clean image researchers can turn into a 3D model.

These 3D models are essential for scientists to understand the complex chains of interactions taking place in an individual protein, such as spike proteins on the COVID-19 virus, speeding drug and vaccine discovery.

Solving the Bottleneck

UCSF, a leader in cryo-EM research, has been the source of groundbreaking work to improve the resolution of microscopy images. The technology enables scientists to visualize proteins at an atomic scale — something considered impossible just a decade ago.

But the pipeline is lengthy, involving freezing samples, capturing them on multimillion dollar cryo-EM microscopes, correcting their motion and then reconstructing detailed 3D models of the molecules. To keep things running smoothly, it’s critical that the motion-correction process runs fast enough to keep pace with the new data being collected.

“Cryo-EM microscopes are very expensive instruments. You don’t want it just sitting there idle. But if we have a backlog of movies piled up in the machine’s data storage, nobody else can collect more,” said Zheng. “It’d be a waste of this expensive instrument, and slow down the research of others.”

To achieve rapid motion correction, UCSF’s Center of Advanced Electron Microscopy uses workstations with eight NVIDIA GPUs for each microscope. These workstations are needed to keep up with the cryo-EM data collection, which acquires four movies per microscope per minute.

The GPU setup can run eight jobs concurrently, taking on the iterative process of motion correction for videos with as many as 400 frames, each with nearly 100 million pixels.

To speed the development of new applications, Zheng, who’s used NVIDIA GPUs for his research for a decade, uses a workstation powered by two NVIDIA Tensor Core GPUs. The system can analyze a 70GB microscope movie in under a minute.

Accelerating COVID Research

Zheng and his colleagues also use GPUs to run alignment software for cryo-electron tomography, or cryo-ET. This technique is better suited to study slightly heterogeneous specimens like macromolecules and cells. Samples are tilted at different angles, collecting a series of images that can be aligned and reconstructed into a detailed 3D model.

NVIDIA GPUs can fully automate the reconstruction process, taking a half hour on a single GPU, he says.

In a recent paper in Science, Zheng collaborated with lead researchers from the Netherlands’ Leiden University Medical Center to use cryo-ET to study molecular pores involved in COVID-19 virus replication in cells. A better understanding of this pore structure could help scientists develop a drug that targets it, blocking the virus from replicating in an infected patient.

To learn more about Zheng’s work, watch this on-demand talk from the GPU Technology Conference.

Main image shows a cryo-EM density map for the enzyme beta-galactosidase, showing the gradual increase in quality of the cryo-EM structures from low to high resolution. Image by Veronica Falconieri and Sriram Subramaniam, licensed from the National Cancer Institute under public domain.

The journey to making the upcoming film Gods of Mars changed course dramatically once real-time rendering entered the picture.

The movie, currently in production, features a mix of cinematic visual effects with live-action elements. The film crew had planned to make the movie primarily using real-life miniature figures. But they switched gears once they experienced the power of real-time NVIDIA RTX graphics and Unreal Engine.

Director Peter Hyoguchi and producer Joan Webb used an Epic MegaGrant from Epic Games to bring together VFX professionals and game developers to create the film. The virtual production started with scanning the miniature models and animating them in Unreal Engine.

“I’ve been working as a CGI and VFX supervisor for 20 years, and I never wanna go back to older workflows,” said Hyoguchi. “This is a total pivot point for the next 100 years of cinema — everyone is going to use this technology for their effects.”

Hyoguchi and team produced rich, photorealistic worlds in 4K to create rich, intergalactic scenes using a combination of NVIDIA Quadro RTX 6000 GPU-powered Lenovo ThinkStation P920 workstations, ASUS ProArt Display PA32UCX-P monitors, Blackmagic Design cameras and DaVinci Resolve, and the Wacom Cintiq Pro 24.

Stepping Outside the Ozone: Technology Makes Way for More Creativity

Gods of Mars tells the tale of a fighter pilot who leads a team against rebels in a battle on Mars. The live-action elements of the film are supported by LED walls with real-time rendered graphics created from Unreal Engine. Actors are filmed on-set, with a virtual background projected behind them.

To keep the set minimal, the team only builds what actors will physically interact with, and then uses the projected environment from Unreal Engine for the rest of the scenes.

One big advantage of working with digital environments and assets is real-time lighting. When previously working with CGI, Hyoguchi and his team would pre-visualize everything inside a grayscale environment. Then they’d wait hours for one frame to render before seeing a preview of what an image or scene would look like.

With Unreal Engine, Hyoguchi can have scenes ray-trace rendered immediately with lights, shadows and colors. He can move around the environment and see how everything would look in the scene, saving weeks of pre-planning.

Real-time rendering also saves money and resources. Hyoguchi doesn’t need to spend thousands of dollars for render farms, or wait weeks for one shot to complete rendering. The RTX-powered ThinkStation P920 renders everything in real time, which leads to more iterations, making way for a much more efficient, flexible and faster creative workflow.

“Ray tracing is what makes this movie possible,” said Hyoguchi. “With NVIDIA RTX and the ability to do real-time ray tracing, we can make a movie with low cost and less people, and yet I still have the flexibility to make more creative choices than I’ve ever had in my life.”

Hyoguchi and his team are shooting the film with Blackmagic Design’s new URSA Mini Pro 12K camera. Capturing such high-resolution footage provides more options in post-production. They can crop images or zoom in for a close-up shot of an actor without worrying about losing resolution.

They can also color and edit scenes in real time using Blackmagic DaVinci Resolve Studio, which uses NVIDIA GPUs to accelerate editing workflows. With the 32-inch ASUS ProArt Display PA32UCX-P monitors, the team calibrated their screens so all the artists can see the same rendered color and details, even while working in different locations across the country.

The Wacom Cintiq Pro 24 pen displays speed up the 3D artist’s workflow, and provides a natural connection between the artist and the Unreal editor, both when moving scene elements around to create the 3D environment and when keyframing actors for animation.

Learn more about Gods of Mars and NVIDIA RTX.

Electric vehicle upstarts have gained a foothold in the industry and are using NVIDIA DRIVE to keep that momentum going.

Nowhere is the trend of electric vehicles more apparent than in China, the world’s largest automotive market, where electric vehicle startups have exploded in popularity. NIO, Li Auto and Xpeng are bolstering the initial growth in new energy vehicles with models that push the limits of everyday driving with extended battery range and AI-powered features.

All three companies doubled their sales in 2020, with a combined volume of more than 103,000 vehicles.

Along with more efficient powertrains, these fleets are also introducing new and intelligent features to daily commutes with NVIDIA DRIVE.

NIO Unveils a Supercharged Compute Platform

Last week, NIO announced a supercomputer to power its automated and autonomous driving features, with NVIDIA DRIVE Orin at its core.

The computer, known as Adam, achieves over 1,000 trillion operations per second (TOPS) of performance with the redundancy and diversity necessary for safe autonomous driving. It also enables personalization in the vehicle, learning from individual driving habits and preferences while continuously improving from fleet data.

The Orin-powered supercomputer will debut in the flagship ET7 sedan, scheduled for production in 2022, and will be in every NIO model to follow.

The ET7 leapfrogs current model capabilities, with more than 600 miles of battery range and advanced autonomous driving. As the first vehicle equipped with Adam, the EV can perform point-to-point autonomy, leveraging 33 sensors and high-performance compute to continuously expand the domains in which it operates  — from urban to highway driving to battery swap stations.

With this centralized, software-defined computing architecture, NIO’s future fleet of EVs will feature the latest AI-enabled capabilities designed to make its vehicles perpetually upgradable.

Li Auto Powers Ahead

In September, standout EV maker Li Auto said it would develop its next generation of electric vehicles using NVIDIA DRIVE AGX Orin.

These new vehicles are being developed in collaboration with tier 1 supplier Desay SV and feature advanced autonomous driving features, as well as extended battery range for truly intelligent mobility.

This high-performance platform will enable Li Auto to deploy an independent, advanced autonomous driving system with its next-generation fleet.

The automaker began rolling out its first vehicle, the Li Auto One SUV, in November 2019. Since then, sales have skyrocketed, with a 530 percent increase in volume in December, year-over-year, and a total of 32,624 vehicles in 2020.

Li Auto plans to continue this momentum with its upcoming models, packed with even more intelligent features enabled by NVIDIA DRIVE.

Cruising on Xpeng XPilot

Xpeng has been developing on DRIVE since 2018, developing a level 3 autopilot system in collaboration with Desay.

The technology debuted last April with the Xpeng P7, an all-electric sports sedan developed from the ground up for an intelligent driving future.

The XPilot 3.0 level 3 autonomous driving system leverages NVIDIA DRIVE AGX Xavier as well as a redundant and diverse halo of sensors for automated highway driving and valet parking. XPilot was born in the data center, with NVIDIA’s AI infrastructure for training and testing self-driving deep neural networks.

With high-performance data center GPUs and advanced AI learning tools, this scalable infrastructure allows developers to manage massive amounts of data and train autonomous driving DNNs.

The burgeoning EV market is driving the next decade of personal transportation. And with NVIDIA DRIVE at the core, these vehicles have the intelligence and performance to go the distance.

Chinese electric automaker NIO will use NVIDIA DRIVE for advanced automated driving technology in its future fleets, marking the genesis of truly intelligent and personalized NIO vehicles.

During a global reveal event, the EV maker took the wraps off its latest ET7 sedan, which starts shipping in 2022 and features a new NVIDIA-powered supercomputer, called Adam, that uses NVIDIA DRIVE Orin to deploy advanced automated driving technology.

“The cooperation between NIO and NVIDIA will accelerate the development of autonomous driving on smart vehicles,” said NIO CEO William Li. “NIO’s in-house developed autonomous driving algorithms will be running on four industry-leading NVIDIA Orin processors, delivering an unprecedented 1,000+ trillion operations per second in production cars.”

The announcement marks a major step toward the widespread adoption of intelligent, high-performance electric vehicles, improving standards for both the environment and road users.

NIO has been a pioneer in China’s premium smart electric vehicle market. Since 2014, the automaker has been leveraging NVIDIA for its seamless infotainment experience. And now, with NVIDIA DRIVE powering automated driving features in its future vehicles, NIO is set to redefine mobility with continuous improvement and personalization.

“Autonomy and electrification are the key forces transforming the automotive industry,” said Jensen Huang, NVIDIA founder and CEO. “We are delighted to partner with NIO, a leader in the new energy vehicle revolution—leveraging the power of AI to create the software-defined EV fleets of the future.”

An Intelligent Creation

Software-defined and intelligent vehicles require a centralized, high-performance compute architecture to power AI features and continuously receive upgrades over the air.

The new NIO Adam supercomputer is one of the most powerful platforms to run in a vehicle. With four NVIDIA DRIVE Orin processors, Adam achieves more than 1,000 TOPS of performance.

Orin is the world’s highest-performance, most-advanced AV and robotics processor. This supercomputer on a chip is capable of delivering up to 254 TOPS to handle the large number of applications and deep neural networks that run simultaneously in autonomous vehicles and robots, while achieving systematic safety standards such as ISO 26262 ASIL-D.

By using multiple SoCs, Adam integrates the redundancy and diversity necessary for safe autonomous operation. The first two SoCs process the 8 gigabytes of data produced by the vehicle’s sensor set every second. The third Orin serves as a backup to ensure the system can still operate safely in any situation, while the fourth enables local training, improving the vehicle with fleet learning as well as personalizing the driving experience based on individual user preferences.

With high-performance compute at its core, Adam is a major achievement in the creation of automotive intelligence and autonomous driving.

Meet the ET7

NIO took the wraps off its much-anticipated ET7 sedan — the production version of its original EVE concept first shown in 2017.

The flagship vehicle leapfrogs current model capabilities, with more than 600 miles of battery range and advanced autonomous driving. As the first vehicle equipped with Adam, the ET7 can perform point-to-point autonomy, leveraging 33 sensors and high-performance compute to continuously expand the domains in which it operates  — from urban to highway driving to battery swap stations.

The intelligent sedan ensures a seamless experience from the moment the driver approaches the car. With a highly accurate digital key and soft-closing doors, users can open the car with a gentle touch. Enhanced driver monitoring and voice recognition enable easy interaction with the vehicle. And sensors on the bottom of the ET7 detect the road surface so the vehicle can automatically adjust the suspension for a smoother ride.

With AI now at the center of the NIO driving experience, the ET7 and upcoming NVIDIA-powered models are heralding the new generation of intelligent transportation.

The AI cockpit has reached galactic proportions with the new Mercedes-Benz MBUX Hyperscreen.

During a digital event, the luxury automaker unveiled the newest iteration of its intelligent infotainment system — a single surface extending from the cockpit to the passenger seat displaying all necessary functions at once. Dubbed the MBUX Hyperscreen, the system is powered by NVIDIA technology and shows how AI can create a truly intuitive and personalized experience for both the driver and passengers.

“The MBUX Hyperscreen reinvents how we interact with the car,” said Sajjad Khan, executive vice president at Mercedes-Benz. “It’s the nerve center that connects everyone in the car with the world.”

Like the MBUX system recently unveiled with the new Mercedes-Benz S-Class, this extended-screen system runs on high-performance, energy-efficient NVIDIA GPUs for instantaneous AI processing and sharp graphics.

A vehicle’s cockpit typically requires a collection of electronic control units and switches to perform basic functions, such as powering entertainment or adjusting the temperature. Using NVIDIA technology, Mercedes-Benz consolidated these components into one AI platform — with three separate screens under one glass surface — to simplify the architecture while creating more space to add new features.

“Zero Layer” User Interface

The driving principle behind the MBUX Hyperscreen is that of the “zero layer” — every necessary driving feature is delivered with a single touch.

However, developing the largest screen ever mounted in a series-built Mercedes-Benz was not enough to achieve this groundbreaking capability. The automaker also leveraged AI to promote commonly used features at relevant times while pushing those not needed to the background.

The deep neural networks powering the system process datasets such as vehicle position, cabin temperature and time of day to prioritize certain features — like entertainment or points of interest recommendations — while always keeping navigation at the center of the display.

“The system always knows what you want and need based on emotional intelligence,” Khan explained.

And these features aren’t just for the driver. Front-seat passengers get a dedicated screen for entertainment and ride information that doesn’t interfere with the driver’s display. It also enables the front seat passenger to share content with others in the car.

Experience Intelligence

This revolutionary AI cockpit experience isn’t a mere concept — it’s real technology that will be available in production vehicles this year.

The MBUX Hyperscreen will debut with the all-electric Mercedes-Benz EQS, combining electric and artificial intelligence. With the first generation MBUX now in 1.8 million cars, the next iteration coming in the redesigned S-Class, and now, the MBUX Hyperscreen in the EQS, customers will have a range of AI cockpit options.

And with the entire MBUX family powered by NVIDIA, these systems will constantly deliver new, surprising and intelligent features with high performance and a seamless experience.

Canadian high schooler Ana DuCristea has a clever solution for the quarantine slump.

Using AI and natural language processing, she programmed an app capable of setting customizable reminders so you won’t miss any important activities, like baking banana bread or whipping up Dalgona coffee.

The project’s emblematic of how a new generation – with access to powerful technology and training — approaches the once exotic domain of AI.

A decade ago, deep learning was the stuff of elite research labs with big budgets.

Now it’s the kind of thing a smart, motivated high school student can knock out to solve a tangible problem.

DuCristea’s been interested in coding from childhood, and spends her spare time teaching herself new skills and taking online AI courses. After winning a Jetson Nano Developer Kit this summer at AI4ALL, an AI camp, she set to work remedying one of her pet peeves — the limited functionality of reminder applications.

She’s long envisioned a more useful app that could snooze for more specific lengths of time, and set reminders for specific tasks, dates and times. Using the Nano and her background on Python, DuCristea spent her after-school hours creating an app that does just that.

With the app, users can message a bot on Discord requesting a reminder for a specific task, date and time. DuCristea has shared the app’s code on Github, and is planning to continue training it to increase its accuracy and capabilities.

Key Points From This Episode:

Her first hands-on experience with the Jetson Nano has only strengthened her intent to pursue software or computer engineering at college, where she’ll continue to learn more about what area of STEM she’d like to focus on.

• DuCristea’s interest in programming and electronics started at age nine, when her father gifted her a book on Python and she found it so interesting that she worked through it in a week. Since then, she’s taken courses on coding and shares her most recent projects on GitHub.
• Programming the app took some creativity, as DuCristea didn’t have a large dataset to train on. After trying neural networks and vectorization, she eventually found that template searches worked best for her limited list of examples.

Tweetables:

“There’s so many programs, even exclusively for girls now in STEM — I would say go for them.” — Ana DuCristea [14:55]

“The Jetson Nano is a lot more accessible than most things in AI right now.” — Ana DuCristea [18:51]

You Might Also Like:

AI4Good: Canadian Lab Empowers Women in Computer Science

Doina Precup, associate professor at McGill University and research team lead at AI startup DeepMind, speaks about her personal experiences, along with the AI4Good Lab she co-founded to give women more access to machine learning training.

Jetson Interns Assemble! Interns Discuss Amazing AI Robots They’re Building

NVIDIA’s Jetson interns, recruited at top robotics competitions, discuss what they’re building with NVIDIA Jetson, including a delivery robot, a trash-disposing robot and a remote control car to aid in rescue missions.

A Man, a GAN and a 1080 Ti: How Jason Antic Created ‘De-Oldify’

Jason Antic explains how he created his popular app, De-Oldify, with just an NVIDIA GeForce 1080 Ti and a generative adversarial network. The tool colors old black-and-white shots for a more modern look.

Tune in to the AI Podcast

Get the AI Podcast through iTunes, Google Podcasts, Google Play, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn. If your favorite isn’t listed here, drop us a note.

Make the AI Podcast Better

Have a few minutes to spare? Fill out this listener survey. Your answers will help us make a better podcast.