Posts in category: nVidia AI

Penn State University pals Brad Bogolea and Mirza Shah were living in Silicon Valley when they pitched Jeff Gee on their robotics concepts. Fortunately for them, the star designer was working at the soon-to-shutter Willow Garage robotics lab.

So the three of them — Shah was also a software engineer at Willow — joined together and in 2014 founded Simbe Robotics.

The startup’s NVIDIA Jetson-powered bot, dubbed Tally, has since rolled into more than a dozen of the world’s largest retailers. The multitasking robot can navigate stores, scan barcodes and track as many as 30,000 items an hour.

Running on Jetson enables Tally to be more efficient — it can process data from several cameras and perform onboard deep computer vision algorithms. This powerful edge AI capability enhances Tally’s data capture and processing, providing Simbe’s customers with inventory and shelf information more quickly and seamlessly while minimizing costs.

Tally makes rounds to scan store inventory up to three times a day, increasing product availability and boosting sales for retailers through reduced out of stocks, according to the company.

“We’re providing critical information on what products are not on the shelf, which products might be misplaced or mispriced and up-to-date location and availability,” said Bogolea, Simbe’s CEO.

Forecasting Magic

Using Tally, retail stores are able to better understand what’s happening on store shelves, helping them recognize missed sale opportunities and the benefits of improved inventory management, said Bogolea.

Tally’s inventory data enables its retail partners to offer better visibility to store employees and customers about what’s on store shelves — even before they enter a store.

At Schnuck Markets, for example, where Tally is deployed in 62 stores across the midwest, the retailer integrates Tally’s product location and availability into the store’s loyalty app. This allows customers and Instacart shoppers to determine a store’s availability of products and find their precise locations while shopping.

This data has been helpful with addressing the surge in online shopping under COVID-19, enabling faster order picking through services like Instacart, helping to more quickly fulfill orders.

“Those that leverage technology and data in retail are really going to separate themselves from the rest of the pack,” said Bogolea.

There’s an added benefit for store employees, too: workers who were previously busy taking inventory can now focus on other tasks like improving customer service.

In addition to Schnucks, the startup has deployments with Carrefour, Decathlon Sporting Goods, Groupe Casino and Giant Eagle.

Cloud-to-Edge AI 

AI is the key technology enabling the Tally robots to navigate autonomously in a dynamic environment, analyze the vast amount of information collected by its sensors and report a wide range of metrics such as inventory levels, pricing errors and misplaced stock.

Simbe is using NVIDIA GPUs from the cloud to the edge, helping to train and inference a variety of AI models that can detect the different products on shelves, read barcodes and price labels and detect obstacles.

Analyzing the vast amount of 2D and 3D sensor data collected from the robot, NVIDIA Jetson has enabled extreme optimization of the Tally data capture system and has also helped with localization, according to the company.

Running Jetson on Tally, Simbe is able to process data locally in real time from lidar as well as 2D and 3D cameras to aid in both product identification and navigation. And Jetson has reduced its reliance on processing in the cloud.

“We’re capturing at a far greater frequency and fidelity than has really ever been seen before,” said Bogolea.

“One of the benefits of leveraging NVIDIA Jetson is it gives us a lot of flexibility to start moving more to the edge, reducing our cloud costs.”

Learn more about NVIDIA Jetson, which is used by enterprise customers, developers and DIY enthusiasts for creating AI applications, as well as students and educators for learning and teaching AI.

Brendon Cassidy, CTO and chief scientist at Super Hi-Fi, uses AI to give everyone the experience of a radio station tailored to their unique tastes.

Super Hi-Fi, an AI startup and member of the NVIDIA Inception program, develops technology that produces smooth transitions, intersperses content meaningfully and adjusts volume and crossfade. Started three years ago, Super Hi-Fi first partnered with iHeartRadio and is now also used by companies such as Peloton and Sonos.

Results are showing that users like this personalized approach. Cassidy notes that they tested MagicStitch, one of their tools that eliminates the gap between songs, and found that customers listening with MagicStitch turned on spent 10 percent more time streaming music.

Cassidy’s a veteran of the music industry — from Virgin Digital to the Wilshire Media Group — and recognizes this music experience is finally possible due to GPU acceleration, accessible cloud resources and AI powerful enough to process and learn from music and audio content from around the world.

Key Points From This Episode:

• Cassidy, a radio DJ during his undergraduate and graduate careers, notes how difficult it is to “hit the post” — or to stop speaking just as the singing of the next song begins. Super Hi-Fi’s AI technology is using deep learning to understand and achieve that timing.
• Super Hi-Fi’s technology is integrated into the iHeartRadio app, as well as Sonos Radio stations. Cassidy especially recommends the “Encyclopedia of Brittany” station, which is curated by Alabama Shakes’ musician Brittany Howard and integrates commentary and music.

Tweetables:

“This AI is trying to create a form of art in the listening experience.” — Brendon Cassidy [14:28]

“I hope we’re improving the enjoyment that listeners are getting from all of the musical experiences that we have.” — Brendon Cassidy [28:55]

You Might Also Like:

How Yahoo Uses AI to Create Instant eSports Highlight Reels

Like any sports fan, eSports followers want highlight reels of their kills and thrills as soon as possible, whether it’s StarCraft II, League of Legends or Heroes of the Storm. Yale Song, senior research scientist at Yahoo! Research, explains how AI can make instant eSports highlight reels.

Pierre Barreau Explains How Aiva Uses Deep Learning to Make Music

AI systems have been trained to take photos and transform them into the style of great artists, but now they’re learning about music. Pierre Barreau, head of Luxembourg-based startup Aiva Technologies, talks about the soaring music composed by an AI system — and used as the theme song of the AI Podcast.

How Tattoodo Uses AI to Help You Find Your Next Tattoo

What do you do when you’re at a tattoo parlor but none of the images on the wall strike your fancy? Use Tattoodo, an app that uses deep learning to help create a personalized tattoo.

Tune in to the AI Podcast

Get the AI Podcast through iTunes, Google Podcasts, Google Play, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn. If your favorite isn’t listed here, drop us a note.

Make the AI Podcast Better

Have a few minutes to spare? Fill out this listener survey. Your answers will help us make a better podcast.

Much of 2020 may look best in the rearview mirror, but the year also held many moments of outstanding work, gems worth hitting the rewind button to see again.

So, here’s a countdown — roughly in order of ascending popularity — of 10 favorite NVIDIA videos that hit YouTube in 2020. With two exceptions for videos that deserve a wide audience, all got at least 200,000 views and most, but not all, can be found on the NVIDIA YouTube channel.

#10 Coronavirus Gets a Close-Up

The pandemic was clearly the story of the year.

We celebrated the work of many healthcare providers and researchers pushing science forward to combat it, including the team that won a prestigious Gordon Bell award for using high performance computing and AI to see how the coronavirus works, something they explained in detail in their own video here.

In another one of the many responses to COVID-19, the Folding@Home project received donations of time on more than 200,000 NVIDIA GPUs to study the coronavirus. Using NVIDIA Omniverse, we created a visualization (described below) of data they amassed on their virtual exascale computer.

#9 Cruising into a Ray-Traced Future

Despite the challenging times, many companies continued to deliver top-notch work. For example, Autodesk VRED 2021 showed the shape of things to come in automotive design.

The demo below displays the power of ray tracing and AI to deliver realistic 3D visualizations in real time using RTX technology, snagging nearly a quarter million views. (Note: There’s no audio on this one, just amazing images.)

#8 A Test Drive in the Latest Mercedes

Just for fun — yes, even 2020 included fun — we look back at NVIDIA CEO Jensen Huang taking a spin in the latest Mercedes-Benz S-Class as part of the world premiere of the flagship sedan. He shared the honors with Grammy award-winning Alicia Keys and Formula One champ Lewis Hamilton.

The S-Class uses AI to deliver intelligent features like a voice assistant personalized for each driver. An engineer and a car enthusiast at heart, Huang gave kudos to the work of hundreds of engineers who delivered a vehicle that with over-the-air software updates will get better and better.

#7 Playing Marbles After Dark

The NVIDIA Omniverse team pointed the way to a future of photorealistic games and simulations rendered in real time. They showed how a distributed team of engineers and artists can integrate multiple tools to play more than a million polygons smoothly with ray-traced lighting at 1440p on a single GeForce RTX 3090.

The mesmerizing video captured the eyeballs of nearly half a million viewers.

#6 An AI Platform for the Rest of Us

Great things sometimes come in small packages. In October, we debuted the DGX Station A100, a supercomputer that plugs into a standard wall socket to let data scientists do world-class work in AI. More than 400,000 folks tuned in.

#5 Seeing Virtual Meetings Through a New AI

With online gatherings the new norm, NVIDIA Maxine attracted a lot of eyeballs. More than 800,000 viewers tuned into this demo of how we’re using generative adversarial networks to lower the bandwidth and turn up the quality of video conferencing.

#4 What’s Jensen Been Cooking?

Our most energy-efficient video of 2020 was a bit of a tease. It lasted less than 30 seconds, but Jensen Huang’s preview of the first NVIDIA Ampere architecture GPU drew nearly a million viewers.

#3 Voila, Jensen Whips Up the First Kitchen Keynote

In the days of the Great Depression, vacuum tubes flickered with fireside chats. The 2020 pandemic spawned a slew of digital events with GTC among the first of them.

In May, Jensen recorded in his California home the first kitchen keynote. In a playlist of nine virtual courses, he served a smorgasbord where the NVIDIA A100 GPU was an entrée surrounded by software side dishes that included frameworks for conversational AI (Jarvis) and recommendation systems (Merlin). The first chapter alone attracted more than 300,000 views.

And we did it all again in October when we featured the first DPU, its DOCA software and a framework to accelerate drug discovery.

#2 Delivering Enterprise AI in a Box

The DGX A100 emerged as one of the favorite dishes from our May kitchen keynote. The 5-petaflops system packs AI training, inference and analytics for any data center.

Some 1.3 million viewers clicked to get a virtual tour of the eight A100 GPUs and 200 Gbit/second InfiniBand links inside it.

#1 Enough of All This Hard Work, Let’s Have Fun!

By September it was high time to break away from a porcupine of a year. With the GeForce RTX 30 Series GPUs, we rolled out engines to create lush new worlds for those whose go-to escape is gaming.

The launch video, viewed more than 1.5 million times, begins with a brief tour of the history of computer games. Good days remembered, good days to come.

For Dessert: Two Bytes of Chocolate

We’ll end 2020, happily, with two special mentions.

Our most watched video of the year was a blistering five-minute clip of game play on DOOM Eternal running all out on a GeForce RTX 3080 in 4K.

And perhaps our sweetest feel good moment of 2020 was delivered by an NVIDIA engineer, Bryce Denney, who hacked a way to let choirs sing together safely in the pandemic. Play it again, Bryce!

2020 served up a global pandemic that roiled the economy. Yet the startup ecosystem has managed to thrive and even flourish amid the tumult. That may be no coincidence.

Crisis breeds opportunity. And nowhere has that been more prevalent than with startups using AI, machine learning and data science to address a worldwide medical emergency and the upending of typical workplace practices.

This is also reflected in NVIDIA Inception, our program to nurture startups transforming industries with AI and data science. Here are a few highlights from a tremendous year for the program and the members it’s designed to propel toward growth and success.

Increased membership:

• Inception hit a record 7,000 members — that’s up 25 percent on the year.
• IT services, healthcare, and media and entertainment were the top three segments, reflecting the global pandemic’s impact on remote work, medicine and home-based entertainment.
• Early-stage and seed-stage startups continue to lead the rate of joining NVIDIA Inception. This has been a consistent trend over recent years.

Startups ramp up: 

• 100+ Inception startups reached the program’s Premier level, which unlocks increased marketing support, engineering access and exposure to senior customer contacts.
• Developers from Inception startups enrolled in more than 2,000 sessions with the NVIDIA Deep Learning Institute, which offers hands-on training and workshops.
• GPU Ventures, the venture capital arm of NVIDIA Inception, made investments in three startup companies — Plotly, Artisight and Rescale.

Deepening partnerships: 

• NVIDIA Inception added Oracle’s Oracle for Startups program to its list of accelerator partners, which already includes AWS Activate and Microsoft for Startups, as well as a variety of regional programs. These tie-ups open the door for startups to access free cloud credits, new marketing channels, expanded customer networks, and other benefits across programs.
• The NVIDIA Inception Alliance for Healthcare launched earlier this month, starting with healthcare leaders GE Healthcare and Nuance, to provide a clear go-to-market path for medical imaging startups.

At its core, NVIDIA Inception is about forging connections for prime AI startups, finding new paths for them to pursue success, and providing them with the tools or resources to take their business to the next level.

Read more about NVIDIA Inception partners on our blog and learn more about the program at https://www.nvidia.com/en-us/deep-learning-ai/startups/.

A giant toaster with windows. That’s the image for many when they hear the term “robotaxi.” But there’s much more to these futuristic, driverless vehicles than meets the eye. They could be, in fact, the next generation of transportation.

Automakers, suppliers and startups have been dedicated to developing fully autonomous vehicles for the past decade, though none has yet to deploy a self-driving fleet at scale.

The process is taking longer than anticipated because creating and deploying robotaxis aren’t the same as pushing out next year’s new car model. Instead, they’re complex supercomputers on wheels with no human supervision, requiring a unique end-to-end process to develop, roll out and continually enhance.

The difference between these two types of vehicles is staggering. The amount of sensor data a robotaxi needs to process is 100 times greater than today’s most advanced vehicles. The complexity in software also increases exponentially, with an array of redundant and diverse deep neural networks (DNNs) running simultaneously as part of an integrated software stack.

These autonomous vehicles also must be constantly upgradeable to take advantage of the latest advances in AI algorithms. Traditional cars are at their highest level of capability at the point of sale. With yearslong product development processes and a closed architecture, these vehicles can’t take advantage of features that come about after they leave the factory.

Vehicles That Get Better and Better Over Time

With an open, software-defined architecture, robotaxis will be at their most basic capability when they first hit the road. Powered by DNNs that are continuously improved and updated in the vehicle, self-driving cars will constantly be at the cutting edge.

These new capabilities all require high-performance, centralized compute. Achieving this paradigm shift in personal transportation requires reworking the entire development pipeline from end to end, with a unified architecture from training, to validation, to real-time processing.

NVIDIA is the only company that enables this end-to-end development, which is why virtually every robotaxi maker and supplier — from Zoox and Voyage in the U.S., to DiDi Chuxing in China, to Yandex in Russia — is using its GPU-powered offerings.

Installing New Infrastructure

Current advanced driver assistance systems are built on features that have become more capable over time, but don’t necessarily rely on AI. Autonomous vehicles, however, are born out of the data center. To operate in thousands of conditions around the world requires intensive DNN training using mountains of data. And that data grows exponentially as the number of AVs on the road increases.

To put that in perspective, a fleet of just 50 vehicles driving six hours a day generates about 1.6 petabytes of sensor data daily. If all that data were stored on standard 1GB flash drives, they’d cover more than 100 football fields. This data must then be curated and labeled to train the DNNs that will run in the car, performing a variety of dedicated functions, such as object detection and localization.

This data center infrastructure isn’t also used to test and validate DNNs before vehicles operate on public roads. The NVIDIA DRIVE Sim software and NVIDIA DRIVE Constellation autonomous vehicle simulator deliver a scalable, comprehensive and diverse testing environment. DRIVE Sim is an open platform with plug-ins for third-party models from ecosystem partners, allowing users to customize it for their unique use cases.

This entire development infrastructure is critical to deploying robotaxis at scale and is only possible through the unified, open and high-performance compute delivered by GPU technology.

Re-Thinking the Wheel

The same processing capabilities required to train, test and validate robotaxis are just as necessary in the vehicle itself.

A centralized AI compute architecture makes it possible to run the redundant and diverse DNNs needed to replace the human driver all at once. This architecture must also be open to take advantage of new features and DNNs.

The DRIVE family is built on a single scalable architecture ranging from one NVIDIA Orin variant that sips just five watts of energy and delivers 10 TOPS of performance all the way up to the new DRIVE AGX Pegasus, featuring the next-generation Orin SoC and NVIDIA Ampere architecture for thousands of operations per second.

Such a high level of performance is necessary to replace and perform better than a human driver. Additionally, the open and modular nature of the platform enables robotaxi companies to create custom configurations to accommodate the new designs opened up by removing the human driver (along with steering wheel and pedals).

With the ability to use as many processors as needed to analyze data from the dozens of onboard sensors, developers can ensure safety through diversity and redundancy of systems and algorithms.

This level of performance has taken years of investment and expertise to achieve. And, by using a single scalable architecture, companies can easily transition to the latest platforms without sacrificing valuable software development time.

Continuous Improvement

By combining data center and in-vehicle solutions, robotaxi companies can create a continuous, end-to-end development cycle for constant improvement.

As DNNs undergo improvement and learn new capabilities in the data center, the validated algorithms can be delivered to the car’s compute platform over the air for a vehicle that is forever featuring the latest and greatest technology.

This continuous development cycle extends joy to riders and opens new, transformative business models to the companies building this technology.

And you think you’ve mastered social distancing.

Selene is at the center of some of NVIDIA’s most ambitious technology efforts.

Selene sends thousands of messages a day to colleagues on Slack.

Selene’s wired into GitLab, a key industry tool for tracking the deployment of code, providing instant updates to colleagues on how their projects are going.

One of NVIDIA’s best resources works just a block from NVIDIA’s Silicon Valley, Calif., campus, but Selene can only be visited during the pandemic only with the aid of a remote-controlled robot.

Selene is, of course, a supercomputer.

The world’s fastest commercial machine, Selene was named the world’s fifth-fastest supercomputer in the world on November’s closely watched list of TOP500 supercomputers.

Built with new NVIDIA A100 GPUs, Selene achieved 63.4 petaflops on HPL, a key benchmark for high-performance computing, on that same TOP500 list.

While the TOP500 benchmark, originally launched in 1993, continues to be closely watched, a more important metric today is peak AI performance.

By that metric, using the A100’s 3rd generation tensor core, Selene delivers over 2,795 petaflops*, or nearly 2.8 exaflops, of peak AI performance.

The new version of Selene doubles the performance over the prior version, which holds all eight performance records on MLPerf AI Training benchmarks for commercially available products.

But what’s remarkable about this machine isn’t its raw performance. Or how long it takes the two-wheeled, NVIDIA Jetson TX2 powered robot, dubbed “Trip,” tending Selene to traverse the co-location facility — a kind of hotel for computers — housing the machine.

Or even the quiet (by supercomputing standards) hum of the fans cooling its 555,520 computing cores and 1,120,000 gigabytes of memory, all connected by NVIDIA Mellanox HDR InfiniBand networking technology.

It’s how closely it’s wired into the day-to-day work of some of NVIDIA’s top researchers.

That’s why — with the rest of the company downshifting for the holidays — Mike Houston is busier than ever.

In Demand

Houston, who holds a Ph.D. in computer science from Stanford and is a recent winner of the ACM Gordon Bell Prize, is NVIDIA’s AI systems architect, coordinating time on Selene among more than 450 active users at the company.

Sorting through proposals to do work on the machine is a big part of his job. To do that, Houston says he aims to balance research, advanced development and production workloads.

NVIDIA researchers such as Bryan Catanzaro, vice president for applied deep learning research, say there’s nothing else like Selene.

“Selene is the only way for us to do our most challenging work,” Catanzaro said, whose team will be putting the machine to work the week of the 21st. “We would not be able to do our jobs without it.”

Catanzaro leads a team of more than 40 researchers who are using the machine to help advance their work in large-scale language modeling, one of the toughest AI challenges

His words are echoed by researchers across NVIDIA vying for time on the machine.

Built in just three weeks this spring, Selene’s capacity has more than doubled since it was first turned on. That makes it the crown jewel in an ever-growing, interconnected complex of supercomputing power at NVIDIA.

In addition to large-scale language modeling, and, of course, performance runs, NVIDIA’s computing power is used by teams working on everything from autonomous vehicles to next-generation graphics rendering to tools for quantum chemistry and genomics.

Having the ability to scale up to tackle big jobs, or tear off just enough power to tackle smaller tasks, is key, explains Marc Hamilton, vice president for solutions architecture and engineering at NVIDIA.

Hamilton matter of factly compares it to moving dirt. Sometimes a wheelbarrow is enough to get the job done. But for other jobs, where you need more dirt, you can’t get the job done without a dump truck.

“We didn’t do it to say it’s the fifth-fastest supercomputer on Earth, but because we need it, because we use it every day,” Hamilton says.

The Fast and the Flexible

It helps that the key component Selene is built with, NVIDIA DGX SuperPOD, is incredibly efficient.

A SuperPOD achieved 26.2 gigaflops/watt power-efficiency during its 2.4 HPL performance run, placing it atop the latest Green500 list of world’s most efficient supercomputers.

That efficiency is a key factor in its ability to scale up, or carry bigger computing loads, by merely adding more SuperPODs.

Each SuperPOD, in turn, is comprised of compact, pre-configured DGX A100 systems, which are built using the latest NVIDIA Ampere architecture A100 GPUs and  NVIDIA Mellanox InfiniBand for the compute and storage fabric.

Continental, Lockheed Martin and Microsoft are among the businesses that have adopted DGX SuperPODs.

The University of Florida’s new supercomputer, expected to be the fastest in academia when it goes online, is also based on SuperPOD.

Selene is now composed of four SuperPODs, each with a total of 140 nodes, each a NVIDIA DGX A100, giving Selene a total of 560 nodes, up from 280 earlier this year.

A Need for Speed

That’s all well and good, but Catanzaro wants all the computing power he can get.

Catanzaro, who holds a doctorate in computer science from UC Berkeley, helped pioneer the use of GPUs to accelerate machine learning a decade ago by swapping out a 1,000 CPU system for three off-the-shelf NVIDIA Geforce GTX 580 GPUs, letting him work faster.

It was one of a number of key developments that led to the deep learning revolution. Now, nearly a decade later, Catanzaro figures he has access to roughly a million times more power thanks to Selene.

“I would say our team is being really well supported by NVIDIA right now, we can do world-class, state-of-the-art things on Selene,” Catanzaro says. “And we still want more.”

That’s why — while NVIDIANs have set up Microsoft Outlook to respond with an away message as they take the week off — Selene will be busier than ever.

*2,795 petaflops FP16/BF16 with structural sparsity enabled.

AI is transforming businesses across every industry, but like any journey, the first steps can be the most important.

To help enterprises get a running start, we’re collaborating with Amazon Web Services to bring 21 NVIDIA NGC software resources directly to the AWS Marketplace. The AWS Marketplace is where customers find, buy and immediately start using software and services that run on AWS.

NGC is a catalog of software that is optimized to run on NVIDIA GPU cloud instances, such as the Amazon EC2 P4d instance featuring the record-breaking performance of NVIDIA A100 Tensor Core GPUs. AWS customers can deploy this software free of charge to accelerate their AI deployments.

We first began providing GPU-optimized software through the NVIDIA NGC catalog in 2017. Since then, industry demand for these resources has skyrocketed. More than 250,000 unique users have now downloaded more than 1 million of the AI containers, pretrained models, application frameworks, Helm charts and other machine learning resources available on the catalog.

Teaming Up for Another First in the Cloud

AWS is the first cloud service provider to offer the NGC catalog on their marketplace. Many organizations look to the cloud first for new deployment, so having NGC software available at the fingertips of data scientists and developers can help enterprises hit the ground running. With NGC, they can easily get started on new AI projects without having to leave the AWS ecosystem.

“AWS and NVIDIA have been working together to accelerate computing for more than a decade, and we are delighted to offer the NVIDIA NGC catalog in AWS Marketplace,” said Chris Grusz, director of AWS Marketplace at Amazon Web Services. “With NVIDIA NGC software now available directly in AWS Marketplace, customers will be able to simplify and speed up their AI deployment pipeline by accessing and deploying these specialized software resources directly on AWS.”

NGC AI Containers Debuting Today in AWS Marketplace

To help data scientists and developers build and deploy AI-powered solutions, the NGC catalog offers hundreds of NVIDIA GPU-accelerated machine learning frameworks and industry-specific software development kits. Today’s launch of NGC on AWS Marketplace features many of NVIDIA’s most popular GPU-accelerated AI software in healthcare, recommender systems, conversational AI, computer vision, HPC, robotics, data science and machine learning, including:

• NVIDIA AI: A suite of frameworks and tools, including MXNet, TensorFlow, NVIDIA Triton Inference Server and PyTorch.
• NVIDIA Clara Imaging: NVIDIA’s domain-optimized application framework that accelerates deep learning training and inference for medical imaging use cases.
• NVIDIA DeepStream SDK: A multiplatform scalable video analytics framework to deploy on the edge and connect to any cloud.
• NVIDIA HPC SDK: A suite of compilers, libraries and software tools for high performance computing.
• NVIDIA Isaac Sim ML Training: A toolkit to help robotics machine learning engineers use Isaac Sim to generate synthetic images to train an object detection deep neural network.
• NVIDIA Merlin: An open beta framework for building large-scale deep learning recommender systems.
• NVIDIA NeMo: An open-source Python toolkit for developing state-of-the-art conversation AI models.
• RAPIDS: A suite of open-source data science software libraries.

Instant Access to Performance-Optimized AI Software

NGC software in AWS Marketplace provides a number of benefits to help data scientists and developers build the foundations for success in AI.

• Faster software discovery: Through the AWS Marketplace, developers and data scientists can access the latest versions of NVIDIA’s AI software with a single click.
• The latest NVIDIA software: The NGC software in AWS Marketplace is federated, giving AWS users access to the latest versions as soon as they’re available in the NGC catalog. The software is constantly optimized, and the monthly releases give users access to the latest features and performance improvements.
• Simplified software deployment: Users of Amazon EC2, Amazon SageMaker, Amazon Elastic Kubernetes Service (EKS) and Amazon Elastic Container Service (ECS) can quickly subscribe, pull and run NGC software on NVIDIA GPU instances, all within the AWS console. Additionally, SageMaker users can simplify their workflows by eliminating the need to first store a container in Amazon Elastic Container Registry (ECR).
• Continuous integration and development: NGC Helm charts are also available in AWS Marketplace to help DevOps teams quickly and consistently deploy their services.

When it comes to future mobility, you may not have to pave as many paradises for personal car parking lots.

This week, autonomous mobility company Zoox unveiled its much-anticipated purpose-built robotaxi. Designed for everyday urban mobility, the vehicle is powered by NVIDIA and is one of the first level 5 robotaxis featuring bi-directional capabilities, providing a concrete view into the next generation of intelligent transportation.

Zoox and NVIDIA first announced their partnership in 2017, with the innovative startup leveraging the high-performance, energy-efficient compute of NVIDIA to build a level 5 vehicle from the ground up. It was a significant milestone toward an autonomous future. Zoox is also an alumnus of NVIDIA Inception, our accelerator program for startups transforming industries with AI and data science.

Robotaxis are set to transform the way we move. Experts at UBS estimate these vehicles could create a $2 trillion market globally by 2030, while reducing the cost of daily travel for riders by more than 80 percent. With greater affordability, robotaxis are expected to decrease car ownership in urban areas — a recent survey of 6,500 U.S. drivers showed nearly half would be willing to give up car ownership if robotaxis became widespread.

With Zoox and the openness and scalability of NVIDIA AI technology, this vision of safer and more efficient mobility is no longer a faraway future, but a close reality.

Autonomy Forwards and Backwards

Unlike current passenger vehicles that focus on the driver, Zoox is designed for riders. The vehicle was built from the start to optimize features necessary for autonomous, electric mobility, such as sensor placement and large batteries.

Each vehicle features four-wheel steering, allowing it to pull into tight curb spaces without parallel parking. This capability makes it easy for Zoox to pick up and drop off riders, quickly getting to the curb and out of the flow of traffic to provide a better and safer experience.

The vehicle is bidirectional, so there is no fixed front or back end. It can pull forward into a driveway and forward out onto the road without reversing. In the case of an unexpected road closure, the vehicle can simply flip directions or use four-wheel steering to turn around. No reversing required.

Inside the vehicle, carriage seating facilitates clear visibility of the vehicle’s surroundings as well as socializing. Each seat has the same amount of space and delivers the same experience — there’s no bad seat in the house. Carriage seating also makes room for a wider aisle, allowing passengers to easily pass by each other without getting up or contorting into awkward positions.

All together, these design details give riders the freedom of seamless mobility, backed by safety innovations not featured in conventional cars.

One Solution

NVIDIA provides the only end-to-end platform for developing software-defined vehicles with a centralized architecture, spanning from the data center to the vehicle.

For robotaxis, achieving level 5 autonomy requires compute with enough headroom to continuously add new features and capabilities. NVIDIA enables this level of performance, starting with the infrastructure for training and validation and extending to in-vehicle compute.

These vehicles can be continuously updated over the air with deep neural networks that are developed and improved in the data center.

The open and modular nature of the NVIDIA platform enables robotaxi companies to create custom configurations to accommodate new designs, such as Zoox’s symmetrical layout, with cameras, radar and lidar that achieve a 270-degree field of view on all four corners of the vehicle.

With the ability to use as many processors as needed to analyze data from the dozens of onboard sensors, developers can ensure safety through diversity and redundancy of systems and algorithms.

By leveraging NVIDIA, Zoox is using the only proven, high-performance solution for robotaxis, putting the vision of on-demand autonomous mobility within reach.

Once the founder of a wearable computing startup, Arye Barnehama understands the toils of manufacturing consumer devices. He moved to Shenzhen in 2014 to personally oversee production lines for his brain waves-monitoring headband, Melon.

It was an experience that left an impression: manufacturing needed automation.

His next act is Elementary Robotics, which develops robotics for manufacturing. Elementary Robotics, based in Los Angeles, was incubated at Pasadena’s Idealab.

Founded in 2017, Elementary Robotics recently landed a $12.7 million Series A round of funding, including investment from customer Toyota.

Elementary Robotics is in deployment with customers who track thousands of parts. Its system is constantly retraining algorithms for improvements to companies’ inspections.

“Using the NVIDIA Jetson edge AI platform, we put quite a bit of engineering effort into tracking for 100 percent of inferences, at high frame rates,” said Barnehama, the company’s CEO.

Jetson for Inspections

Elementary Robotics has developed its own hardware and software for inspections used in manufacturing. It offers a Jetson-powered robot that can examine parts for defects. It aims to improve quality with better tracking of parts and problems.

Detecting the smallest of defects on a fast moving production line requires processing of high-resolution camera data with AI in real time. This is made possible with the embedded CUDA-enabled GPU and the CUDA-X AI software on Jetson. As the Jetson platform makes decisions from video streams, these are all ingested into its cloud database so that customers are able to observe and query the data.

The results, along with the live video, are also then published to the Elementary Robotics web application, which can be accessed from anywhere.

Elementary Robotics’ system also enables companies to inspect parts from suppliers before putting them into the production line, avoiding costly failures. It is used for inspections of assemblies on production lines as well as for quality control at post-production.

Its applications include inspections of electronic printed circuit boards and assemblies, automotive components, and gears for light industrial use. Elementary Robotics customers also use its platform in packaging and consumer goods such as bottles, caps and labels.

“Everyone’s demand for quality is always going up,” said Barnehama. “We run real-time inference on the edge with NVIDIA systems for inspections to help improve quality.”

The Jetson platform recently demonstrated leadership in MLPerf AI inference benchmarks in SoC-based edge devices for computer vision and conversational AI use cases.

Elementary Robotics is a member of NVIDIA Inception, a virtual accelerator program that helps startups in AI and data science get to market faster.

Traceability of Operations

The startup’s Jetson-enabled machine learning system can handle split-second anomaly detection to catch mistakes on the production lines. And when there’s a defective part returned, companies that rely on Elementary Robotics can try to understand how it happened. Use cases include electronics, automotive, medical, consumer packaged goods, logistics and other applications.

For manufacturers, such traceability of operations is important so that companies can go back and find and fix the causes of problems for improved reliability, said Barnehama.

“You want to be able to say, ‘OK, this defective item got returned, let me look up when it was inspected and make sure I have all the inspection data,’”  added Barnehama.

NVIDIA Jetson is used by enterprise customers, developers and DIY enthusiasts for creating AI applications, as well as students and educators for learning and teaching AI.

Pinterest now has more than 440 million reasons to offer the best visual search experience. That’s because its monthly active users are tracking this high for its popular image sharing and social media service.

Visual search enables Pinterest users to search for images using text, screenshots or camera photos. It’s the core AI behind how people build their Boards of Pins — collections of images by themes —  around their interests and plans. It’s also how people on Pinterest can take action on the inspiration they discover, such as shopping and making purchases based on the products within scenes.

But tracking more than 240 billion images and 5 billion Boards is no small data trick.

This requires visual embeddings — mathematical representations of objects in a scene. Visual embeddings use models for automatically generating and evaluating visualizations to show how similar two images are — say, a sofa in a TV show’s living room compared to ones for sale at retailers.

Pinterest is improving its search results by pretraining its visual embeddings on a smaller dataset. The overall goal is to improve for one unified visual embedding that can perform well for its key business features.

Powered by NVIDIA V100 Tensor Core GPUs, this technique pre-trains Pinterest’s neural nets on a subset of about 1.3 billion images to yield improved relevancy across the wider set of hundreds of billions of images.

Improving results on the unified visual embedding in this fashion can benefit all applications on Pinterest, said Josh Beal, a machine learning researcher for Visual Search at the company.

“This model is fine-tuned on various multitask datasets. And the goal of this project was to scale the model to a large scale,” he said.

Benefitting Shop the Look 

With so many visuals, and new ones coming in all the time, Pinterest is continuously training its neural networks to identify them in relation to others.

A popular visual search feature, Pinterest’s Shop the Look enables people to shop for home and fashion items. By tapping into visual embeddings, Shop the Look can identify items in Pins and connect Pinners to those products online.

Product matches are key to its visual-driven commerce. And it isn’t an easy problem to solve at Pinterest scale.

Yet it matters. Another Pinterest visual feature is the ability to search specific products within an image, or Pin. Improving the accuracy or recommendations with visual embedding improves the magic factor in matches, boosting people’s experience of discovering relevant products and ideas.

An additional feature, Pinterest’s Lens camera search, aims to recommend visually relevant Pins based on the photos Pinners take with their cameras.

“Unified embedding for visual search benefits all these downstream applications,” said Beal.

Making Visual Search More Powerful

Several Pinterest teams have been working to improve visual search on the hundreds of billions of images within Pins. But given the massive scale of the effort and its cost and engineering resource restraints, Pinterest wanted to optimize its existing architecture.

With some suggested ResNeXt-101 architecture optimizations and by simply upgrading to the latest releases of NVIDIA libraries, including cuDNN v8, automated mixed precision and NCCL, Pinterest was able to improve training performance of their models by over 60 percent.

NVIDIA’s GPU-accelerated libraries are constantly being updated to enable companies such as Pinterest to get more performance out of their existing hardware investment.

“It has improved the quality of the visual embedding, so that leads to more relevant results in visual search,” said Beal.