Posts in category: nVidia AI

Editor’s note: This is the latest post in our NVIDIA DRIVE Labs series, which takes an engineering-focused look at individual autonomous vehicle challenges and how NVIDIA DRIVE addresses them. Catch up on all of our automotive posts, here.

Even with advanced driver assistance systems automating more driving functions, human drivers must maintain their attention at the wheel and build trust in the AI system.

Traditional driver monitoring systems typically don’t understand subtle cues such as a driver’s cognitive state, behavior or other activity that indicates whether they’re ready to take over the driving controls.

NVIDIA DRIVE IX is an open, scalable cockpit software platform that provides AI functions to enable a full range of in-cabin experiences, including intelligent visualization with augmented reality and virtual reality, conversational AI and interior sensing.

Driver perception is a key aspect of the platform that enables the AV system to ensure a driver is alert and paying attention to the road. It also enables the AI system to perform cockpit functions that are more intuitive and intelligent.

In this DRIVE Labs episode, NVIDIA experts demonstrate how DRIVE IX perceives driver attention, activity, emotion, behaviour, posture, speech, gesture and mood with a variety of detection capabilities.

A Multi-DNN Approach

Facial expressions are complex signals to interpret. A simple wrinkle of the brow or shift of the gaze can have a variety of meanings.

DRIVE IX uses multiple DNNs to recognize faces and decipher the expressions of vehicle occupants. The first DNN detects the face itself, while a second identifies fiducial points, or reference markings — such as eye location, nose, etc.

On top of these base networks, a variety of DNNs operate to determine whether a driver is paying attention or requires other actions from the AI system.

The GazeNet DNN tracks gazes by detecting the vector of the driver’s eyes and mapping it to the road to check if they’re able to see obstacles ahead. SleepNet monitors drowsiness, classifying whether eyes are open or closed, running through a state machine to determine levels of exhaustion. Finally, ActivityNet tracks driver activity such as phone usage, hands on/off the wheel and driver attention to road events. DRIVE IX can also detect whether the driver is properly sitting in their seat to focus on road events.

In addition to driver focus, a separate DNN can determine a driver’s emotions — a key indicator of their ability to safely operate the vehicle. Taking in data from the base face-detect and fiducial-point networks, DRIVE IX can classify a driver’s state as happy, surprised, neutral, disgusted or angry.

It can also tell if the driver is squinting or screaming, indicating their level of visibility or alertness and state of mind.

A Customizable Solution

Vehicle manufacturers can leverage the driver monitoring capabilities in DRIVE IX to develop advanced AI-based driver understanding capabilities for personalizing the car cockpit.

The car can be programmed to alert a driver if their attention drifts from the road, or the cabin can adjust settings to soothe occupants if tensions are high.

And these capabilities extend well beyond driver monitoring. The aforementioned DNNs, together with gesture DNN and speech capabilities, enable multi-modal conversational AI offerings such as automatic speech recognition, natural language processing and speech synthesis.

These networks can be used for in-cabin personalization and virtual assistant applications. Additionally, the base facial recognition and facial key point models can be used for AI-based video conferencing platforms.

The driver monitoring capabilities of DRIVE IX help build trust between occupants and the AI system as automated driving technology develops, creating a safer, more enjoyable intelligent vehicle experience.

Arman Toorians isn’t your average classic car restoration hobbyist.

The NVIDIA engineer recently transformed a 1974 Triumph TR6 roadster at his home workshop into an EV featuring AI.

Toorians built the vehicle to show a classic car can be recycled into an electric ride that taps NVIDIA Jetson AI for safety, security and vehicle management features.

He hopes the car blazes a path for others to explore electric vehicle conversions that pack AI.

“My objective is to encourage others to take on this task — it’s a lot of fun,” he said.

Where It Began

The story begins when Toorians purchased the Triumph in “junkyard condition” for $4,000 from a retired cop who gave up on ambitions of restoring the car.

The conversion cost $20,000 in parts. He acquired a motor from NetGain Motors, electrical components from EV West, Triumph parts from Moss Motors, and five Tesla batteries capable of 70 miles on a charge for his setup. NVIDIA supplied the Jetson Xavier NX developer kit.

To find spare time, the quadrilingual Persian-Armenian musician took a hiatus from playing flamenco and jazz guitar. Co-workers and friends cheered on his effort, he said, while his wife and son gave a hand and advice as needed.

Three years in the making, the car’s sophisticated build with AI may just set a new bar for the burgeoning classic-to-electric conversion space.

Jetson for Security 

The car is smart. And it may have as much in common with a Tesla as it does with future generations of EVs from automakers everywhere adopting AI.

That’s because he installed the NVIDIA Jetson NX developer kit in the little sports car’s trunk. The power-efficient Jetson edge AI platform provides compact supercomputing performance capable of handling multi-modal inference, devouring data from the car’s sensors and camera.

“Jetson is a great platform that addresses many different markets and can also be very easily used to solve many problems in this DIY electric car market,” Toorians said.

Toorians, a director in the Jetson group, uses the Jetson Xavier NX for a growing list of cool features. For example, using a dash cam feed and the NVIDIA DeepStream SDK for intelligent video analytics, he’s using the Jetson to work on securing the car. Like the talking car KITT from the hit ‘80s TV series Knight Rider, the Triumph will be able to react with verbal warnings to anyone unauthorized to be in or around it.

If that doesn’t work to deter a threat to the vehicle, his plans are for Jetson to step it up a notch and activate alarms and email alerts.

Lighter, Faster, Cleaner

The sleek sports car has impressive specs. It’s now lighter, faster and cleaner on its energy usage than when it rolled out of the factory decades ago. It also doesn’t leave motor oil stains or the stench of gas and exhaust fumes in its wake.

In place of the greasy 350-pound six cylinder gas engine, Toorians installed a shiny 90-pound electrical motor. The new motor lives in the same spot under the hood and puts out 134 horsepower versus the stock engine’s 100 horses, providing peppy takeoffs.

He gets a lot of thumbs-up reactions to the car. But for him the big payoff is repurposing the older gasoline vehicle to electric to avoid environmental waste and exhaust pollutants.

“Electric cars help keep our air and environment clean and are the way toward a more sustainable future in transportation,” he said.

Jetson for Safety

Among its many tricks, the Triumph uses AI to recognize the driver so that only authorized users can start it — and only when seat belts are fastened by both the driver and passenger.

The dash camera running through Jetson — which can process more than 60 frames per second — is being used to generate lane departure warnings.

Front and rear lidar generate driver alerts if objects are in the Triumph’s path or too close. The front lidar and the dash cam can also be used to run the cruise control.

Jetson processes the front and rear battery temperatures and translates it for the analog temperature gauges. Jetson is also called on to read the battery capacity and display it on the analog fuel gauge, keeping it stock. Another nice one: the fuel cap now covers the charging port.

Touches like these — as well as keeping a functioning four-speed shifter — allowed the car to keep its original look.

“Rebuilding older generation cars with electric motors and computers helps us recycle good gasoline engine cars that otherwise would have to be destroyed,” he said.

DIY makers and businesses alike turn to NVIDIA Jetson for edge AI.

Flying, driving, gaming, racing… amid the first-ever virtual Consumer Electronics Show this week, NVIDIA-powered technologies spilled out in all directions.

In automotive, Chinese automakers SAIC and NIO announced they’ll use NVIDIA DRIVE in future vehicles.

In gaming, NVIDIA on Tuesday led off a slew of gaming announcements by revealing the affordable new RTX 3060 GPU and detailing the arrival of more than 70 30-series GPUs for gamers and creatives.

In robotics, the Skydio X2 drone has received the CES 2021 Best of Innovation Award for Drones and Unmanned Systems.

And in, well, a category all its own, the Indy Autonomous Challenge, unveiled Thursday, will pit college teams equipped with sleek, swift vehicles equipped with ADLINK DLP-8000 robot controller powered by NVIDIA GPUs against each other for a $1.5 million prize.

This week’s announcements were just the latest examples of how NVIDIA is driving AI and innovation into every aspect of our lives.

Game On

Bringing more gaming capabilities to millions more gamers, NVIDIA Tuesday announced more than 70 new laptops will feature GeForce RTX 30 Series Laptop GPUs and unveiled the NVIDIA GeForce RTX 3060 graphics card for desktops, priced at just $329.

All are powered by the award-winning NVIDIA Ampere GPU architecture, the second generation of RTX with enhanced Ray Tracing Cores, Tensor Cores, and new streaming multiprocessors.

NVIDIA also announced Call of Duty: Warzone and Square Enix’s new IP, Outriders. And Five Nights at Freddy’s: Security Breach and F.I.S.T.: Forged in Shadow Torch will be adding RTX ray tracing and DLSS.

The games are just the latest to support the real-time ray tracing and AI-based DLSS (deep learning super sampling) technologies, known together called RTX, which NVIDIA introduced two years ago.

The announcements were among the highlights of a streamed presentation from Jeff Fisher, senior vice president of NVIDIA’s GeForce business.

Amid the unprecedented challenges of 2020, “millions of people tuned into gaming — to play, create and connect with one another,” Fisher said. “More than ever, gaming has become an integral part of our lives.”

Hitting the Road

In automotive, two Chinese automakers announced they’ll be relying on NVIDIA DRIVE technologies.

Just as CES was starting electric car startup NIO announced a supercomputer to power its automated and autonomous driving features, with NVIDIA DRIVE Orin at its core.

The computer, known as Adam, achieves over 1,000 trillion operations per second of performance with the redundancy and diversity necessary for safe autonomous driving.

The Orin-powered supercomputer will debut in NIO’s flagship ET7 sedan, scheduled for production in 2022, and every NIO model to follow.

And on Thursday, SAIC, China’s largest automaker, announced it’s joining forces with online retail giant Alibaba to unveil a new premium EV brand, dubbed IM for “intelligence in motion.”

The long-range electric vehicles will feature AI capabilities powered by the high-performance, energy-efficient NVIDIA DRIVE Orin compute platform.

The news comes as EV startups in China have skyrocketed in popularity, with NVIDIA working with NIO along with Li Auto and Xpeng to bolster the growth of new-energy vehicles.

Taking to the Skies

Meanwhile, Skydio, the leading U.S. drone manufacturer and world leader in autonomous flight, today announced it received the CES 2021 Best of Innovation Award for Drones and Unmanned Systems for the Skydio X2.

Skydio’s new autonomous drone offers enterprise and public sector customers up to 35 minutes of autonomous flight time.

Packing six 4k cameras and powered by the NVIDIA Jetson TX2 mobile supercomputer, it’s built to offer situational awareness, asset inspection, and security patrol.

There’s a new brand of automotive intelligence equipped with the brains — and the battery — to go the distance.

SAIC, the largest automaker in China, joined forces with etail giant Alibaba to unveil a new premium EV brand, dubbed IM, or “intelligence in motion.” The long-range electric vehicles will feature AI capabilities powered by the high-performance, energy-efficient NVIDIA DRIVE Orin compute platform.

The first two vehicles in the lineup — a flagship sedan and SUV — will have autonomous parking and other automated driving features, as well as a 93kWh battery that comes standard. SAIC will begin taking orders for the sedan at the Shanghai Auto Show in April, with the SUV following in 2022.

These models will have multi NVIDIA Orin SoCs (system-on-a-chip) at the core of a centralized computer system, achieving 500 to 1,000+ TOPS of performance for automated and autonomous capabilities in addition to in-cabin personalization that is continuously upgradable over-the-air for a truly software-defined experience.

By centralizing and unifying the compute architecture, IM vehicles will be able to receive advanced software features as they’re developed. Just like a mobile phone, which periodically gets software updates, these software-defined vehicles will do the same.

Premium Vehicles Inside and Out

Developing a top-of-the-line premium electric brand requires best-in-class in-vehicle compute.

Orin is the world’s highest-performance, most-advanced AV and robotics processor. This supercomputer on a chip is capable of delivering up to 254 trillion operations per second (TOPS) to handle the large number of applications and deep neural networks that run simultaneously in autonomous vehicles and robots, while meeting systematic safety standards such as ISO 26262 ASIL-D.

With two Orin SoCs at the center of IM vehicles, these compute platforms will be able to deliver more than 500 TOPS of performance to achieve the redundancy and diversity necessary for autonomous operation.

Like all modern computing devices, these intelligent vehicles will be supported by a large team of AI and software engineers, dedicated to improving the performance and capability of the car as technology advances.

Intelligence in Motion

The new IM vehicle lineup is upping the ante for intelligent and electric mobility.

These electric vehicles will come standard with a 93kWh battery, and a 115kWh one on premium trims.

The software-defined IM vehicles don’t just improve like mobile devices, they also work seamlessly with such technology. The brand includes smartphone features for personalized driver and passenger experiences, creating a smart living space inside the car.

As a new, continuously upgradeable electric vehicle lineup, SAIC’s premium IM brand will drive further innovation in intelligent, personal transportation.

NVIDIA is the second-best place to work in the U.S. according to a ranking released today by Glassdoor.

The site’s Best Places to Work in 2021 list rates the 100 best U.S. companies with more than 1,000 employees, based on how their own employees rate career opportunities, company culture and senior management.

The survey’s top finisher was Bain & Company. Right behind in NVIDIA on the list are In-N-Out Burger, HubSpot and McKinsey & Company.

“This year’s winning employers have proven, according to employees, that even during extraordinary times, they’ll rise to the challenge to support their people,” said Christian Sutherland-Wong, Glassdoor chief executive officer.

Among other recent recognitions of NVIDIA’s efforts to take care of our people and communities amid the pandemic:

• No. 1 on Investor’s Business Daily’s 2020 Best ESG Companies List, covering environmental, social and governance factors.
• No. 2 in Newsweek’s list of America’s Most Responsible Companies.
• No. 2 on America’s Most JUST Companies.
• No. 4 in the People magazine’s 50 Companies That Care.
• ​No. 4 on Fortune’s Change the World list in recognition of the company’s contributions to healthcare.

Cooking on video calls with friends. Getting to the end of Netflix’s endless content well. Going 10 months without a haircut.

Over the past year, we all found different ways to keep ourselves occupied.

Gaming, however, is a longer-term trend that promises to continue remaking global culture for years to come.

Over 2.5 billion gamers are now engaged in playing, creating, sharing and connecting with one another.

Together we watched over 100 billion hours of gaming content on YouTube — twice as much as 2018. That’s like 11 million people watching non-stop for a year.

And esports viewership is nearly half a billion people globally — up 100 million from 2018 — with another 150 million new viewers expected over the next three years.

Building on a long-term surge that has made gaming an integral part of all our lives over the past decade, market researcher Newzoo expects gaming revenue to rise 8 percent in 2021.

The Gaming Market Set New Records (Again)

And gaming revenue — projected by analysts at $175 billion last year — already towers over other global consumer entertainment markets such as music and radio, internet-based or “over the top” video of all kinds, and cinema.

There are now 2.6 billion people playing — over half the global online population.

The amount of time people played grew dramatically, with gameplay time up 26 percent in the U.S. over the past six months.

Participation on Discord — the leading real-time voice chat app for gamers — is up 2x in the past two years, with 140 million monthly active users who connect in over 4 billion minutes of conversation each day.

The Thrill of Victory

Today’s games are getting more realistic and immersive, from lifelike graphics to AI-based gameplay and realistic physics simulation.

And to play the latest titles, gamers want the latest hardware — whether a recently launched console or a PC with the latest graphics technology, NVIDIA GeForce RTX.

Two years ago, NVIDIA introduced a breakthrough in graphics — real-time ray tracing for the ultimate realism and AI-based DLSS (deep learning super sampling) to “magically” improve performance.

We named it RTX.

Together with Microsoft and top developers and game engines, we’re working to bring the visual realism of movies to fully interactive gaming.

The momentum is unstoppable. The latest consoles and the rest of the gaming ecosystem are now onboard: Ray tracing is the new standard. There are already over 30 titles with ray tracing and many more on the way. You can find a list of select titles here.

And some of the most popular games over the past few years — Minecraft, with over 200 million copies sold and 126 million people playing per month, and Fortnite, with over 350 million registered users — have introduced real-time ray tracing enhancements to their games, enabling much more realistic and immersive experiences.

New Hardware Enables Better Games

The hits keep on coming.

Cyberpunk 2077, released a few weeks ago, generated 8 million pre-orders (almost 5 million for PC), and sold 13 million copies in its first 10 days. It shattered the single-player record for number of concurrent Steam users within two hours of launch with over 1 million.

Cyberpunk 2077 is just one example of how production value of games continues to increase. Bigger worlds and cinematic graphics demand more of the GPU. A survey of GeForce gamers showed that 59 percent of GPU or PC upgrades are due to the low performance of a game they’re playing or requirements for one that they’re anticipating.

This increase in production value is evident in the latest sequels to these AAA games, where the GeForce GTX 1060, one of our most popular GPUs of all time, struggles to keep up.

For instance, gamers were playing Watch Dogs 2 (released in 2016) with high settings at 60 frames per second on a GTX 1060. But when they moved on to Watch Dogs: Legion, released in October, they only saw 24 frames per second on that system.

And turning on ray tracing to get the best visuals makes Watch Dogs: Legion virtually unplayable on a GTX 1060.

Fear not, however. The just-announced GeForce RTX 3060 based on the NVIDIA Ampere architecture brings the game back to life, at a price that makes RTX technology available to all PC gamers.

With tens of millions of GeForce GTX GPUs in use today, the upgrade opportunity is enormous. The performance gains and new features in GeForce RTX 30 Series GPUs make it a great time for all PC gamers to upgrade.

Competitive Gaming Takes Center Stage

Weekend basketball player? We already know you love the NBA. It’s no surprise, then, that hundreds of millions of video game players like to watch pros play their games, too.

There are professional leagues for top games like Counter-Strike, League of Legends and Dota 2. The stakes in these contests are serious. In 2019, the prize pool for the Dota 2 International tournament reached $34 million. That’s more than triple the $11 million prize purse for the U.S. Masters golf tournament.

When the stakes are high, quality gear matters. Response time is critical to be most competitive at esports games — typically shooters — like Counter-Strike or Fortnite. Every millisecond matters.

Esports pros and enthusiasts strive for the lowest latency — down to zero if they could get there. So NVIDIA invented Reflex to help get the latency of the system as low as possible.

NVIDIA Reflex optimizes the rendering pipeline across the CPU and GPU to reduce latency by up to 50 percent, giving gamers a precious advantage. A 20-millisecond advantage can mean the difference between winning and losing in the physical sports world and esports.

With Reflex and our Game Ready Drivers, over 100 million GeForce gamers are instantly more competitive. Fortnite, Apex Legends, Valorant and Call of Duty: Warzone were among the first to integrate Reflex technology. With this week’s announcements, seven of the top 10 competitive shooter games now support Reflex. And more are on the way.

It’s no wonder that 99 percent of esports pros play on GeForce GPUs.

You Can Take It With You

Laptops are the fastest growing gaming platform, increasing 7x in seven years. And the power of these laptops has inspired gamers to find new ways to play, whether in console mode connected to a big screen TV using a controller, or driving an ultra-wide monitor with a keyboard and mouse.

Laptop PCs were flying off shelves so fast last year, manufacturers couldn’t keep up with demand. Laptop PC sales were up 26 percent over 2019, the largest gain in many years. It would have been even higher if more supply were available.

Many laptop buyers are looking for their PCs to perform a wide variety of activities — from keeping up on social media and streaming videos to creating and editing videos and playing the latest games. They want as much performance as they can get without having to sacrifice portability.

To address that demand, we’ve just announced that the GeForce RTX 30 Series, powered by the NVIDIA Ampere architecture, will soon be available in laptops. These laptops can handle all the latest tasks and are available in slim designs using our third-generation Max-Q technologies.

RTX 30 Series laptops bring exceptional power to gamers and creators, with the best laptops for creators meeting the specifications of our NVIDIA Studio program.

And there are laptops available specifically for esports players that include 240Hz displays for fast response and low latency. Gamers can compete at the highest level on these devices.

This week we announced that our partners are introducing over 70 laptops — it’s our biggest GeForce laptop launch ever. These are the world’s fastest laptops that give gamers and creators a huge variety to choose the right device for their needs.

More to Come

The GeForce RTX 30 Series GPUs shipping in desktops and, very shortly, in laptops, coupled with ray tracing in games, will fuel the next round of gaming PCs and upgrades.

Over the past two decades, GPUs have revolutionized modern graphics again and again. Once the holy grail of computer graphics, ray tracing is now the standard.

We look forward to more revolutions to come: if the last 20 years of graphics and gaming were amazing, the next 20 will seem nothing short of science fiction.

Bringing more gaming capabilities to millions more gamers, NVIDIA on Tuesday  announced more than 70 new laptops will feature GeForce RTX 30 Series Laptop GPUs and unveiled the NVIDIA GeForce RTX 3060 graphics card for desktops.

All are powered by the award-winning NVIDIA Ampere GPU architecture, the second generation of RTX with enhanced Ray Tracing Cores, Tensor Cores, and new streaming multiprocessors.

The announcements were among the highlights of a streamed presentation from Jeff Fisher, senior vice president of NVIDIA’s GeForce business.

Amid the unprecedented challenges of 2020, “millions of people tuned into gaming — to play, create and connect with one another,” Fisher said. “More than ever, gaming has become an integral part of our lives.” Among the stats he cited:

• Steam saw its number of concurrent users more than double from 2018
• Discord, a messaging and social networking service most popular with gamers, has seen monthly active users triple to 140 million from two years ago
• In 2020 alone, more than 100 billion hours of gaming content have been watched on YouTube
• Also in 2020, viewership of esports reached half a billion people

Meanwhile, NVIDIA has been delivering a series of major gaming advancements, Fisher explained.

RTX ‘the New Standard’

Two years ago, NVIDIA introduced a breakthrough in graphics real-time ray tracing and AI-based DLSS (deep learning super sampling), together called RTX, he said.

NVIDIA quickly partnered with Microsoft and top developers and game engines to bring the visual realism of movies to fully interactive gaming, Fisher said.

In fact, 36 games are now powered by RTX. They include the #1 Battle Royale game, the #1 RPG, the #1 MMO and the #1 best-selling game of all time – Minecraft.

Now, we’re announcing more games that support RTX technology, including DLSS, which is coming to both Call of Duty: Warzone and Square Enix’s new IP, Outriders. And Five Nights at Freddy’s: Security Breach and F.I.S.T.: Forged in Shadow Torch will be adding ray tracing and DLSS.

For more details, read our January 2021 RTX Games article.

“The momentum is unstoppable,” Fisher said. “As consoles and the rest of the ecosystem are now onboard — ray tracing is the new standard.

last year, NVIDIA launched its second generation of RTX, the GeForce RTX 30 Series GPUs. Based on the NVIDIA Ampere architecture, it represents “our biggest generational leap ever,” Fisher said.

NVIDIA built NVIDIA Reflex to deliver the lowest system latency for competitive gamers – from mouse click to display. Since Reflex’s launch in September, a dozen games have added support.

Fisher announced that Overwatch and Rainbow Six Siege are also adopting NVIDIA Reflex. Now, 7 of the top 10 competitive shooters support Reflex.

And over the past four months, NVIDIA has launched four NVIDIA Ampere architecture-powered graphics cards, from the ultimate BFGPU — the GeForce RTX 3090 priced at $1,499 — to the GeForce RTX 3060 Ti at $399.

“Ampere has been our fastest selling architecture ever, selling almost twice as much as our prior generation,” Fisher said.

GeForce RTX 3060: An NVIDIA Ampere GPU for Every Gamer

With gaming now a key part of global culture, the new GeForce RTX 3060 brings the power of the NVIDIA Ampere architecture to every gamer, Fisher said.

“The RTX 3060 offers twice the raster performance of the GTX 1060 and 10x the ray-tracing performance,” Fisher said, noting that the GTX 1060 is the world’s most popular GPU. “The RTX 3060 powers the latest games with RTX On at 60 frames per second.”

The RTX 3060 has 13 shader teraflops, 25 RT teraflops for ray tracing, and 101 tensor teraflops to power DLSS, an NVIDIA technology introduced in 2019 that uses AI to accelerate games. And it boasts 12 gigabytes of GDDR6 memory.

“With most of the installed base underpowered for the latest games, we’re bringing RTX to every gamer with the GeForce RTX 3060,” Fisher said.

The GeForce RTX 3060 starts at just $329 and will be available worldwide in late February.

“Amazing Gaming Doesn’t End at the Desktop”

NVIDIA also announced a new generation of NVIDIA Ampere architecture-powered laptop GPUs.

Laptops, Fisher explained, are the fastest-growing gaming platform. There are now 50 million gaming laptops, which powered over 14 billion gaming hours last year.

“Amazing gaming doesn’t end at the desktop,” Fisher said.

These high-performance machines also meet the needs of 45 million creators and everyone working and studying from home, Fisher said.

The new generation of NVIDIA Ampere architecture-powered laptops, with second-generation RTX and third-generation Max-Q technologies, deliver twice the power efficiency of previous generations.

Efficiency Is Paramount in Laptops

That’s why NVIDIA introduced Max-Q four years ago, Fisher explained.

Max-Q is a system design approach that delivers high performance in thin and light gaming laptops.

“It has fundamentally changed how laptops are built, every aspect — the CPU GPU, software, PCB design, power delivery, thermals — are optimized for power and performance,” Fisher said.

NVIDIA’s third-gen Max-Q technologies use AI and new system optimizations to make high-performance gaming laptops faster and better than ever, he said.

Fisher introduced Dynamic Boost 2.0, which for the first time uses AI to shift power between the CPU, GPU and now, GPU memory.

“So your laptop is constantly optimizing for maximum performance,” Fisher said.

Fisher also introduced WhisperMode 2.0, which delivers a new level of acoustic control for gaming laptops.

Pick your desired acoustics and WhisperMode 2.0’s AI-powered algorithms manage the CPU, GPU system temperature and fan speeds to “deliver great acoustics at the best possible performance,” Fisher explained.

Another new feature, Resizable BAR, uses the advanced capabilities of PCI Express to boost gaming performance.

Games use GPU memory for textures, shaders and geometry — constantly updating as the player moves through the world.

Today, only part of the GPU’s memory can be accessed at any one time by the CPU, requiring many memory updates, Fisher explained.

With Resizable BAR, the game can access the entire GPU memory, allowing for multiple updates at the same time, improving performance, Fisher said.

Resizable BAR will also be supported on GeForce RTX 30 Series graphics cards for desktops, starting with the GeForce RTX 3060. NVIDIA and GPU partners are readying VBIOS updates for existing GeForce RTX 30 series graphics cards starting in March.

Finally, NVIDIA DLSS offers a breakthrough for gaming laptops. It uses AI and RTX Tensor Cores to deliver up to 2x to performance in the same power envelope.

World’s Fastest Laptops for Gamers and Creators

Starting at $999, RTX 3060 laptops are “faster than anything on the market today,” Fisher said.

They’re 30 percent faster than the PlayStation 5 and deliver 90 frames per second on the latest games at ultra settings 1080p, Fisher said.

Starting at $1,299, GeForce RTX 3070 laptops are “a 1440p gaming beast.”

Boasting twice the pixels of 1080p, this new generation of laptops “provides the perfect mix of high-fidelity graphics and great performance.”

And starting at $1,999, GeForce RTX 3080 laptops will come with up to 16 gigabytes of GDDR6 memory.

They’re “the world’s fastest laptop for gamers and creators,” Fisher said, delivering hundreds of frames per second with RTX on.

As a result, laptop gamers will be able to play at 240 frames per second, across top titles like Overwatch, Rainbow Six Siege, Valorant and Fortnite, Fisher said.

Availability

Manufacturers worldwide, starting Jan. 26, will begin shipping over 70 different GeForce RTX gaming and creator laptops featuring GeForce RTX 3080 and GeForce RTX 3070 laptop GPUs, followed by GeForce RTX 3060 laptop GPUs on Feb. 2.

The GeForce RTX 3060 graphics card will be available in late February, starting at $329, as custom boards — including stock-clocked and factory-overclocked models — from top add-in card providers such as ASUS, Colorful, EVGA, Gainward, Galaxy, Gigabyte, Innovision 3D, MSI, Palit, PNY and Zotac.

Look for GeForce RTX 3060 GPUs at major retailers and etailers, as well as in gaming systems by major manufacturers and leading system builders worldwide.

“RTX is the new standard, and the momentum continues to grow,” Fisher said.

The latest NVIDIA Studio laptops, powered by new NVIDIA GeForce RTX 30 Series Laptop GPUs, are empowering the next generation of creativity. And they bring a host of updates to change how fast creators work.

New models come equipped with up to 16GB of video memory, pixel-accurate displays with 1440p and 4K options, and GPU acceleration for ray tracing, AI and video encoding that allow artists to create in record time.

NVIDIA Studio laptops with new RTX 30 Series Laptop GPUs offer improved performance.

Built to Create, Anywhere

GeForce RTX 30 Series Laptop GPUs accelerate rendering performance by up to 2x and introduce third-generation Max-Q technologies. Max-Q helps manufacturers develop thin and light laptops that maintain great performance, enabling creators to conceptualize incredible pieces of content using hundreds of GPU-accelerated apps.

Creator workflows require multiple PC components — the GPU, GPU memory and CPU — to execute rapidly. New, AI-powered Dynamic Boost 2.0 intelligently shifts power among these components, improving performance so laptops run faster and longer.

NVIDIA Studio laptops are designed to provide the best performance and acoustics, but sometimes you need your laptop to be even quieter while giving a presentation or working in a public place. WhisperMode 2.0 uses AI-powered algorithms to dynamically manage the CPU, GPU and fan speeds to deliver great acoustics and the best possible performance.

Creators need to have pixel-accurate displays to preview with the maximum fidelity. NVIDIA Studio laptops use factory-calibrated displays with wide color gamuts and up to 4K resolution. And now, many NVIDIA Studio laptops offer 1440p configurations for the perfect balance between pixel density and performance.

The New 30 Series Studio Laptops

New Studio laptops with GeForce RTX 3060, 3070 and 3080 Laptop GPUs will start rolling out later this month. The first models announced include:

• Asus ZenBook Pro Duo UX582 — featuring GeForce RTX 3070 graphics. Artists can create their best thanks to next-gen performance and a stunning dual display, giving them new ways of using a laptop.

• MSI Creator 15 — offering GeForce RTX 3060, 3070 and 3080 variants. MSI’s built-for-creators laptop delivers blazing-fast performance with hushed tones as it leverages both Dynamic Boost 2.0 and WhisperMode 2.0.

• Gigabyte Aero 15 and Aero 17 — with multiple screen sizes to choose from and configurable with GeForce RTX 3060, 3070 or 3080 graphics. The Aero 15 features Dynamic Boost 2.0, WhisperMode 2.0 and Advanced Optimus (on the 1080p display) to deliver peak performance while on the go. The larger format Aero 17 also features Dynamic Boost 2.0.

• Razer Blade 15 and Blade Pro 17 — offer stunning 1440p and 4K display options in an elegant, high-quality chassis. The Blade 15 Base model is configurable with a GeForce RTX 3070 and a 1440p display; the Razer Blade Advanced model with GeForce RTX 3070 or 3080 and 1440p or 4K displays; the Blade Pro 17 with a GeForce RTX 3080 and 4K 120Hz display.

Accelerating Creativity

GeForce RTX 30 Series Studio laptops give video editors mobile access to edit up to 8K HDR RAW footage, use AI to simplify workflows, and reduce encode times by up to 75 percent with the NVIDIA Encoder. When connected to an external G-SYNC display — an NVIDIA technology that synchronizes the display to content — editors can preview their content at the exact frame rate it will be exported and played back.

Artists can get up to 16GB of graphics memory to work with huge assets and across multiple apps at the same time.

AI-Powered Workflows

NVIDIA Studio laptops take advantage of industry-leading creative apps to empower creators with AI.

Adobe Photoshop recently introduced Neural Filters, which use AI to quickly make complex edits to photographs.

Blackmagic Design’s Davinci Resolve 17 implements a new Magic Mask feature that uses AI to speed up mask creation and tracking.

The free NVIDIA Broadcast app turns any room into a home studio by enhancing your microphone, speakers and webcam with AI-powered features such as virtual background effects, audio noise removal and webcam auto frame.

NVIDIA Optix AI denoising in Blender cleans up noise on the fly, quickly resolving 3D images.

And NVIDIA Omniverse Audio2Face uses AI to animate unrigged 3D characters, based solely on an audio source.

Studio Drivers

NVIDIA Studio laptops are supported with Studio Drivers that are built specifically to meet creators’ needs for both performance and reliability. They’re tested extensively against top creative apps and workflows.

The latest Studio Driver can be downloaded through GeForce Experience or from our driver download page.

New GeForce RTX 30 Series NVIDIA Studio laptops from ASUS, Gigabyte, MSI and Razer will begin rolling out later this month.

Read more about the GeForce RTX 30 Series laptop announcement, including additional options for gamers and creators starting at $999. Visit the GeForce Laptop website for even more information.

Learn more about NVIDIA Studio hardware and software for creators on the NVIDIA Studio website.

And stay up to date on the latest apps through the NVIDIA Studio YouTube channel, which features tutorials, tips and tricks by industry-leading artists.

We’re at a pivotal moment to unlock a new, AI-accelerated era of discovery and medicine, says Kimberly Powell, NVIDIA’s vice president of healthcare.

Speaking today at the J.P. Morgan Healthcare conference, held virtually, Powell outlined how AI and accelerated computing are enabling scientists to take advantage of the boom in biomedical data to power faster research breakthroughs and better patient care.

Understanding disease and discovering therapies is our greatest human endeavor, she said — and the trillion-dollar drug discovery industry illustrates just how complex a challenge it is.

How AI Can Drive Down Drug Discovery Costs

The typical drug discovery process takes about a decade, costs $2 billion and suffers a 90 percent failure rate during clinical development. But the rise of digital data in healthcare in recent years presents an opportunity to improve those statistics with AI.

“We can produce today more biomedical data in about three months than the entire 300-year history of healthcare,” she said. “And so this is now becoming a problem that no human really can synthesize that level of data, and we need to call upon artificial intelligence.”  

Powell called AI “the most powerful technology force of our time. It’s software that writes software that no humans can.”

But AI works best when it’s domain specific, combining data and algorithms tailored to a specific field like radiology, pathology or patient monitoring. The NVIDIA Clara application framework bridges this gap by providing researchers and clinicians the tools for GPU-accelerated AI in medical imaging, genomics, drug discovery and smart hospitals.

Downloads of NVIDIA Clara grew 5x last year, Powell shared, with developers taking up our new platforms for conversational AI and federated learning.

Healthcare Ecosystem Rallies Around AI

She noted that amid the COVID-19 pandemic, momentum around AI for healthcare has accelerated, with startups estimated to have raised well over $5 billion in 2020. More than 1,000 healthcare startups are in the NVIDIA Inception accelerator program, up 4x since 2017. And over 20,000 AI healthcare papers were submitted last year to PubMed, showing exponential growth over the past decade.

Leading research institutions like the University of California, San Francisco, are using NVIDIA GPUs to power their work in cryo-electron microscopy, a technique used to study the structure of molecules — such as the spike proteins on the COVID-19 virus — and accelerate drug and vaccine discovery.

And pharmaceutical companies, including GlaxoSmithKline, and major healthcare systems, like the U.K.’s National Health Service, will harness the Cambridge-1 supercomputer — an NVIDIA DGX SuperPOD system and the U.K.’s fastest AI supercomputer — to solve large-scale problems and improve patient care, diagnosis and delivery of critical medicines and vaccines.

Software-Defined Instruments Link AI Innovation and Medical Practice

Powell sees software-defined instruments — devices that can be regularly updated to reflect the latest scientific understanding and AI algorithms — as key to connecting the latest research breakthroughs with the practice of medicine.

“Artificial intelligence, like the practice of medicine, is constantly learning. We want to learn from the data, we want to learn from the changing environment,” Powell said.

By making medical instruments software-defined, tools like smart cameras for patient monitoring or AI-guided ultrasound systems can not only be developed in the first place, she said, but also retain their value and improve over time.

U.K.-based sequencing company Oxford Nanopore Technologies is a leader in software-defined instruments, deploying a new generation of DNA sequencing technology across an electronics-based platform. Its nanopore sequencing devices have been used in more than 50 countries to sequence and track new variants of the virus that causes COVID-19, as well as for large-scale genomic analyses to study the biology of cancer.

The company uses NVIDIA GPUs to power several of its instruments, from the handheld MinION Mk1C device to its ultra-high throughput PromethION, which can produce more than three human genomes’ worth of sequence data in a single run. To power the next generation of PromethION, Oxford Nanopore is adopting NVIDIA DGX Station, enabling its real-time sequencing technology to pair with rapid and highly accurate genomic analyses.

For years, the company has been using AI to improve the accuracy of basecalling, the process of determining the order of a molecule’s DNA bases from tiny electrical signals that pass through a nanoscale hole, or nanopore.

This technology “truly touches on the entire practice of medicine,” Powell said, whether COVID epidemiology or in human genetics and long read sequencing. “Through deep learning, their base calling model is able to reach an overall accuracy of 98.3 percent, and AI-driven single nucleotide variant calling gets them to 99.9 percent accuracy.”

Path Forward for AI-Powered Healthcare

AI-powered breakthroughs like these have grown in significance amid the pandemic, said Powell.

“The tremendous focus of AI on a single problem in 2020, like COVID-19, really showed us that with that tremendous focus, we can see every piece and part that can benefit from artificial intelligence,” she said. “What we’ve discovered over the last 12 months is only going to propel us further in the future. Everything we’ve learned is applicable for every future drug discovery program there is.”

Across fields as diverse as genome analysis, computational drug discovery and clinical diagnostics, healthcare heavyweights are making strides with GPU-accelerated AI. Hear more about it on Jan. 13 at 11 a.m. Pacific, when Powell joins a Washington Post Live conversation on AI in healthcare.

Subscribe to NVIDIA healthcare news here.

When photographers take long-exposure photos, they maximize the amount of light their camera sensors receive. The technique helps capture scenes like the night sky, but it introduces blurring in the final image, as in the example at right.

It’s not too different from cryo-electron microscopy, or cryo-EM, which scientists use to study the structure of tiny molecules frozen in vitreous ice. But while motion-induced blur in photography can create beautiful images, in structural biology it’s an unwanted side effect.

Protein samples for cryo-EM are frozen at -196 degrees Celcius to protect the biological structures, which would otherwise be destroyed by the microscope’s high-energy electron beam. But even when frozen, samples are disturbed by the powerful electron dose, causing motion that would blur a long-exposure photo.

To get around it, UCSF researchers use specialized cameras to instead capture videos of the biological molecules, so they appear nearly stationary in each frame of the video. Correcting the motion across frames is a computationally demanding task — but can be done in seconds on NVIDIA GPUs.

“If the motion was left uncorrected, we’d lose the high-resolution picture of a molecule’s 3D structures,” said Shawn Zheng, scientific software developer at the University of California, San Francisco and Howard Hughes Medical Institute. “And knowing the structure of a molecule is critical to understanding its function.”

Zheng and his colleagues run MotionCor2, the world’s most widely used motion-correction application, on NVIDIA GPUs to align each molecule in the video from frame to frame — creating a clean image researchers can turn into a 3D model.

These 3D models are essential for scientists to understand the complex chains of interactions taking place in an individual protein, such as spike proteins on the COVID-19 virus, speeding drug and vaccine discovery.

Solving the Bottleneck

UCSF, a leader in cryo-EM research, has been the source of groundbreaking work to improve the resolution of microscopy images. The technology enables scientists to visualize proteins at an atomic scale — something considered impossible just a decade ago.

But the pipeline is lengthy, involving freezing samples, capturing them on multimillion dollar cryo-EM microscopes, correcting their motion and then reconstructing detailed 3D models of the molecules. To keep things running smoothly, it’s critical that the motion-correction process runs fast enough to keep pace with the new data being collected.

“Cryo-EM microscopes are very expensive instruments. You don’t want it just sitting there idle. But if we have a backlog of movies piled up in the machine’s data storage, nobody else can collect more,” said Zheng. “It’d be a waste of this expensive instrument, and slow down the research of others.”

To achieve rapid motion correction, UCSF’s Center of Advanced Electron Microscopy uses workstations with eight NVIDIA GPUs for each microscope. These workstations are needed to keep up with the cryo-EM data collection, which acquires four movies per microscope per minute.

The GPU setup can run eight jobs concurrently, taking on the iterative process of motion correction for videos with as many as 400 frames, each with nearly 100 million pixels.

To speed the development of new applications, Zheng, who’s used NVIDIA GPUs for his research for a decade, uses a workstation powered by two NVIDIA Tensor Core GPUs. The system can analyze a 70GB microscope movie in under a minute.

Accelerating COVID Research

Zheng and his colleagues also use GPUs to run alignment software for cryo-electron tomography, or cryo-ET. This technique is better suited to study slightly heterogeneous specimens like macromolecules and cells. Samples are tilted at different angles, collecting a series of images that can be aligned and reconstructed into a detailed 3D model.

NVIDIA GPUs can fully automate the reconstruction process, taking a half hour on a single GPU, he says.

In a recent paper in Science, Zheng collaborated with lead researchers from the Netherlands’ Leiden University Medical Center to use cryo-ET to study molecular pores involved in COVID-19 virus replication in cells. A better understanding of this pore structure could help scientists develop a drug that targets it, blocking the virus from replicating in an infected patient.

To learn more about Zheng’s work, watch this on-demand talk from the GPU Technology Conference.

Main image shows a cryo-EM density map for the enzyme beta-galactosidase, showing the gradual increase in quality of the cryo-EM structures from low to high resolution. Image by Veronica Falconieri and Sriram Subramaniam, licensed from the National Cancer Institute under public domain.