In support of President Biden’s executive order on AI, the U.S. government will use an NVIDIA DGX SuperPOD to produce generative AI advances in climate science, healthcare and cybersecurity.
The executive order, issued in October, is aimed at ensuring U.S. leadership in AI and managing its risks. MITRE, a nonprofit organization that operates federally funded research and development centers, is implementing a new NVIDIA DGX SuperPOD system that will provide researchers and developers access to massive computing leaps.
The DGX SuperPOD will support MITRE’s Federal AI Sandbox, a platform to improve experimentation with next-generation, AI-enabled applications across federal government agencies.
“The recent executive order on AI encourages federal agencies to reduce barriers for AI adoptions, but agencies often lack the computing environment necessary for experimentation and prototyping,” said Charles Clancy, senior vice president and chief technology officer at MITRE. “Our new Federal AI Sandbox will help level the playing field, making the high-quality compute power needed to train and test custom AI solutions available to any agency.”
The Federal AI Sandbox will deliver federal agencies the computing gains needed to train large language models and other generative AI tools to develop cutting-edge applications.
The NVIDIA DGX SuperPOD system powering the sandbox is capable of an exaflop of 8-bit AI compute, meaning it performs a quintillion math operation each second to train and deploy custom LLMs and other AI solutions at scale.
The supercomputing initiative comes as the White House recently unveiled plans, which include NVIDIA, for a $110 million partnership to help universities teach AI skills.
Cybersecurity experts at the RSA Conference this week will be on the hunt for ways to secure their operations in the era of generative AI.
They’ll find many of the latest tools use AI and accelerated computing. This intersection of security and AI is coming into focus with collaborations that companies like NVIDIA and its partners will describe at the event.
Data Science for a Data Problem
Machine learning is a great tool for cybersecurity because data is exploding.
“With more devices and users expanding the landscape to defend, cybersecurity has become a data problem; and AI is a data solution,” said David Reber, NVIDIA’s chief security officer.
Today, security analysts can be overwhelmed by a data tsunami. Generative AI can provide security copilots that act like filters, extracting from the firehose flow of information the context and anomalies that need a human expert’s attention.
Generative AI also lets security managers interrogate their data directly instead of setting rules, chasing alerts and tracking dashboards. In the age of AI, security experts will move from a command line to a conversational interface.
AI Puts Security Context on Steroids
This shift takes context-based security to a new level, according to a talk Reber delivered at GTC.
The potential is huge, but it requires work to unearth it. At GTC, Reber encouraged cybersecurity experts to begin working with AI, starting with low-risk use cases to identify and secure gaps.
He also provided suggestions for how to go about securing machine learning processes, saying security experts need to:
secure data supply chains,
develop tests for securing AI models and datasets across the development lifecycle,
use model cards, data cards and software bills of materials to provide AI transparency and reliability,
participate in community testing events such as security hackathons, and
update policies on how to respond to AI security events.
Foundations for AI Cybersecurity
To give users a leg up, NVIDIA provides NVIDIA Morpheus, a cybersecurity AI framework that filters and classifies large volumes of real-time data. Morpheus, part of the NVIDIA AI Enterprise software suite, lets developers build applications that can detect spear phishing, insider threats and more.
A new release of NVIDIA DOCA — the software framework for programming NVIDIA BlueField DPUs and NVIDIA ConnectX NICs — provides another foundation for AI security. It now sports updated encryption features for network and storage data.
An Expanding AI Security Ecosystem
At RSA, many companies will show products built on NVIDIA technologies that extend security for the generative AI era, including:
AIC will demonstrate Qrypt’s key generation for quantum secure encryption running on a BlueField DPU in an AIC-designed server.
Bloombase will show an updated version of its StoreSafe Storage Firewall powered by Morpheus and BlueField DPUs and new use cases for threat detection and fast, quantum-safe encryption of AI models and data.
Check Point Software will show its AI Cloud Protect security solution on BlueField DPUs, Quantum Force security gateways on ConnectX NICs, and Quantum Maestro software on NVIDIA Spectrum switches.
Cisco will feature Cisco Hypershield, an AI-native security architecture, to protect against both known and unknown attacks. It will also discuss its expanding partnership with NVIDIA to help customers harness the power of AI.
CrowdStrike will show its CrowdStrike Falcon Foundry and CrowdStrike Falcon platform that employs NVIDIA’s GPU-optimized AI software, including NIM microservices.
Deloitte will showcase CyberSphere, a cyber operations platform that uses Morpheus to speed detection of cyber threats.
Palo Alto Networks will describe its collaboration with NVIDIA on two use cases, a next-generation reference architecture for securing generative AI deployments with NIM and its VM-Series Virtual Next-Generation Firewall, with expanded intelligent traffic offload (ITO), supporting BlueField-3 DPUs.
Qrypt will demonstrate its quantum-secure key generation for securing AI workloads running on BlueField-3 DPUs using DOCA.
Sygnia will announce the use of BlueField DPUs and Morpheus in Velocity Edge, its new hardware-accelerated MXDR service for the energy and industrial sectors.
They are part of the NVIDIA ecosystem building a new layer of security for generative AI. That community includes more than 20 companies at RSA this week from NVIDIA Inception, a program for cutting-edge startups.
At RSA, Daniel Rohrer, vice president of software product security at NVIDIA, will be part of the keynote panel on AI safety.
In addition, Kevin Deierling, senior vice president of networking at NVIDIA, will share insights on security at the Cloudflare Executive Supper Club. And NVIDIA will participate in an event about women in cybersecurity.
To get started with AI-powered cybersecurity, try a workflow in NVIDIA LaunchPad.
Intrinsic, a software and AI robotics company at Alphabet, has integrated NVIDIA AI and Isaac platform technologies to advance the complex field of autonomous robotic manipulation.
This week at the Automate trade show, in Chicago, Intrinsic is spotlighting leaps in robotic grasping and industrial scalability assisted by foundation models enabled by NVIDIA Isaac Manipulator, unlocking new value in industrial automation with AI.
NVIDIA unveiled Isaac Manipulator at GTC in March. Isaac Manipulator is a collection of foundation models and modular GPU-accelerated libraries that help industrial automation companies build scalable and repeatable workflows for dynamic manipulation tasks by accelerating AI model training and task reprogramming.
Foundation models are based on a transformer deep learning architecture that allows a neural network to learn by tracking relationships in data. They’re generally trained on huge datasets and can be used to process and understand sensor and robot information as magically as ChatGPT for text. This enables robot perception and decision-making like never before and provides zero-shot learning — the ability to perform tasks without prior examples.
NVIDIA’s collaboration with Intrinsic, a leading robotics software and AI company, demonstrates the potential for a universally applicable robotic-grasping skill to work across grippers, environments and objects.
“For the broader industry, our work with NVIDIA shows how foundation models can have a profound impact, including making today’s processing challenges easier to manage at scale, creating previously infeasible applications, reducing development costs, and increasing flexibility for end users,” said Wendy Tan White, CEO at Intrinsic, in a blog post announcing the collaboration with NVIDIA. (White will deliver a keynote address at Automate about what the rise of AI means for innovation and growth, on Thursday, May 9, at 7 a.m. PT.)
Developing Better Robot Grip With Isaac Manipulator
Grasping has been a long sought after robotics skill. So far it’s been time-consuming, expensive to program and difficult to scale. As a result, many repetitive pick-and-place conditions haven’t been seamlessly handled to date by robots.
Simulation is changing that. Enlisting NVIDIA Isaac Sim on the NVIDIA Omniverse platform, Intrinsic generated synthetic data for vacuum grasping using computer-aided design models of sheet metal and suction grippers. This allowed Intrinsic to create a prototype for its customer Trumpf Machine Tools, a leading maker of industrial machine tools.
The prototype uses Intrinsic Flowstate, a developer environment for AI-based robotics solutions, for visualizing processes, associated perception and motion planning. With a workflow that includes Isaac Manipulator, one can generate grasp poses and CUDA-accelerated robot motions, which can first be evaluated in simulation with Isaac Sim — a cost-saving step — before deployment in the real world with the Intrinsic platform.
Under the collaboration, NVIDIA and Intrinsic plan to bring state-of-the-art dexterity and modular AI capabilities for robotic arms, with a robust collection of foundation models and GPU-accelerated libraries to accelerate a greater number of new robotics and automation tasks.
On Tuesday, May 7, at 11 a.m. CT, NVIDIA Senior Research Scientist Adithya Murali and Intrinsic Chief Science Officer Torsten Kroeger will demonstrate the companies’ work in the session “Automating Smart Pick-and-Place With Intrinsic Flowstate and NVIDIA Isaac Manipulator ” in the Intrinsic booth 2808 at Automate. Join our speaking sessions at Automate.
Harnessing optimized AI models for healthcare is easier than ever as NVIDIA NIM, a collection of cloud-native microservices, integrates with Amazon Web Services.
NIM, part of the NVIDIA AI Enterprise software platform available on AWS Marketplace, enables developers to access a growing library of AI models through industry-standard application programming interfaces, or APIs. The library includes foundation models for drug discovery, medical imaging and genomics, backed by enterprise-grade security and support.
NIM is now available via Amazon SageMaker — a fully managed service to prepare data and build, train and deploy machine learning models — and AWS ParallelCluster, an open-source tool to deploy and manage high performance computing clusters on AWS. NIMs can also be orchestrated using AWS HealthOmics, a purpose-built service for biological data analysis.
Easy access to NIM will enable the thousands of healthcare and life sciences companies already using AWS to deploy generative AI more quickly, without the complexities of model development and packaging for production. It’ll also help developers build workflows that combine AI models across different modalities, such as amino acid sequences, MRI images and plain-text patient health records.
Pharma and Biotech Companies Adopt NVIDIA AI on AWS
BioNeMo is a generative AI platform of foundation models, training frameworks, domain-specific data loaders and optimized training recipes that support the training and fine-tuning of biology and chemistry models on proprietary data. It’s used by more than 100 organizations globally.
Amgen, one of the world’s leading biotechnology companies, has used the BioNeMo framework to train generative models for protein design, and is exploring the potential use of BioNeMo with AWS.
BioNeMo models for protein structure prediction, generative chemistry and molecular docking prediction are available as NIM microservices, pretrained and optimized to run on any NVIDIA GPU or cluster of GPUs. These models can be combined to support a holistic, AI-accelerated drug discovery workflow.
Biotechnology company A-Alpha Bio harnesses synthetic biology and AI to measure, predict and engineer protein-to-protein interactions. When its researchers moved from a generic version of the ESM-2 protein language model to a version optimized by NVIDIA running on NVIDIA H100 Tensor Core GPUs on AWS, they immediately saw a speedup of more than 10x. This lets the team sample a much more extensive field of protein candidates than they would have otherwise.
NVIDIA NIM includes genomics models from NVIDIA Parabricks, which are also available on AWS HealthOmics as Ready2Run workflows that enable customers to deploy pre-built pipelines.
Life sciences company Agilent used Parabricks genomics analysis tools running on NVIDIA GPU-powered Amazon Elastic Compute Cloud (EC2) instances to significantly improve processing speeds for variant calling workflows on the company’s cloud-native Alissa Reporter software. Integrating Parabricks with Alissa secondary analysis pipelines enables researchers to access rapid data analysis in a secure cloud environment.
Conversational AI Technology Supports Digital Health
In addition to models that can decode proteins and genomic sequences, NIM microservices offer optimized large language models for conversational AI and visual generative AI models for avatars and digital humans.
AI-powered digital assistants can enhance healthcare by answering patient questions and supporting clinicians with logistics. Trained on healthcare organization-specific data using RAG, they could connect to relevant internal data sources to synthesize research, surface insights and improve productivity.
Generative AI startup Hippocratic AI is in the final stages of testing AI-powered healthcare agents that focus on a wide range of tasks including wellness coaching, preoperative outreach and post-discharge follow-up.
The company, which uses NVIDIA GPUs through AWS, is adopting NVIDIA NIM and NVIDIA ACE microservices to power a generative AI agent for digital health.
The team used NVIDIA Audio2Face facial animation technology, NVIDIA Riva automatic speech recognition and text-to-speech capabilities, and more to power a healthcare assistant avatar’s conversation.
GeForce NOW brings 24 new games for members this month.
Ninja Theory’s highly anticipated Senua’s Saga: Hellblade II will be coming to the cloud soon — get ready by streaming the first in the series, Hellblade: Senua’s Sacrifice, part of the seven new games joining the GeForce NOW library this week.
Plus, game across more devices than ever as GeForce NOW adds improved support on Steam Deck this GFN Thursday.
Journey into Viking Hell
No sacrificing frame rates, streaming from the cloud.
Experience exceptional storytelling in Ninja Theory’s award-winning game Hellblade: Senua’s Sacrifice, available to stream from the cloud this week.
Set in a dark fantasy world inspired by Norse mythology and Celtic culture, the game follows the journey of Senua, a Pict warrior. Her quest is to reach Helheim, the realm of the dead,to rescue her deceased lover’s soul from the goddess Hela.
Solve intricate puzzles with observation, engage in melee combat and get pulled deep into Senua’s mind as she grapples with inner demons. Journey through the hauntingly beautiful landscapes of Helheim with ray tracing and high-dynamic range using an Ultimate or Priority membership for the most immersive and stunning visual fidelity.
Decked Out
Thanks to GeForce NOW, nearly any device can perform like a GeForce-powered PC gaming rig. Members who want to stream their favorite PC games to Valve’s Steam Deck now have an easier way to get started.
Members can use a new beta installation method to automatically configure GeForce NOW’s browser in Steam Deck’s Gaming Mode. The installation script automatically installs Google Chrome to the device, then adds all the settings needed to help members log into GeForce NOW and stream their favorite games.
The latest GeForce NOW update, released last week, also allows members to navigate GeForce NOW on a browser using a gamepad, including on the Steam Deck. That means it’s even easier to find and play supported titles with NVIDIA DLSS technology or real-time ray tracing, regardless of the handheld’s system specs.
Plus, members can easily play non-Steam games on the Steam Deck thanks to the cloud. This includes games from Battle.net, Epic Games Store, Ubisoft Connect, GOG.com and Xbox, as well as supported PC Game Pass titles. No more worrying about downloads or backend configurations. And with more than 1,900 games supported, there’s always something new to stream.
Get complete creative freedom in Paradox Interactive’s FOUNDRY, an exciting first-person factory-building sandbox game set in an infinite voxel world for expansive, ever-changing landscapes. Build a factory optimized to perfection or an artistic masterpiece, harvest resources, automate ever-growing production lines and manage complex systems to achieve mechanical mastery in FOUNDRY.
Check out the list of new games this week:
Hellblade: Senua’s Sacrifice (Steam and Xbox, available on PC Game Pass)
Stormgate Closed Beta (New release on Steam, April 30, sign up for access)
Gray Zone Warfare (New release on Steam, April 30)
Riot Games has rolled out the League of Legends 14.9 update globally, which adds the Vanguard security software to the game. Since Vanguard doesn’t support virtual machines like GeForce NOW, the game is under maintenance and no longer playable on the cloud gaming platform for the foreseeable future. Work is underway to find a solution for GeForce NOW members.
Lightyear Frontier (Xbox) didn’t make it in April due to technical issues. Stay tuned to GFN Thursday for updates.
What are you planning to play this weekend? Let us know on X or in the comments below.
Arthur.ai enhances the performance of AI systems across various metrics like accuracy, explainability and fairness. In this episode of the NVIDIA AI Podcast, recorded live at GTC 2024, host Noah Kravitz sits down with Adam Wenchel, cofounder and CEO of Arthur, to discuss the challenges and opportunities of deploying generative AI. Their conversation spans a range of topics, including AI bias, the observability of AI systems and the practical implications of AI in business. For more on Arthur, visit arthur.ai.
AI-driven change is in the air, as are concerns about the technology’s environmental impact. In this episode of NVIDIA’s AI Podcast, Daniel Castro, vice president of the Information Technology and Innovation Foundation and director of its Center for Data Innovation, speaks with host Noah Kravitz about the motivation behind his AI energy use report, which addresses misconceptions about the technology’s energy consumption.
DigitalPath is igniting change in the golden state — using computer vision, generative adversarial networks and a network of thousands of cameras to detect signs of fire in real-time. In the latest episode of NVIDIA’s AI Podcast, host Noah Kravtiz spoke with DigitalPath system architect Ethan Higgins about the company’s role in the ALERTCalifornia initiative, a collaboration between California’s wildfire fighting agency CAL FIRE and the University of California, San Diego.
Generative AI-based models can not only learn and understand natural languages — they can learn the very language of nature itself, presenting new possibilities for scientific research. On the latest episode of NVIDIA’s AI Podcast, host Noah Kravitz spoke with Anandkumar on generative AI’s potential to make splashes in the scientific community.
In this episode of the NVIDIA AI Podcast, host Noah Kravitz dives into an illuminating conversation with Alex Fielding, co-founder and CEO of Privateer Space. Privateer Space, Fielding’s latest venture, aims to address one of the most daunting challenges facing our world today: space debris.
YouTube robotics influencer Dave Niewinski has developed robots for everything from driveable La-Z-Boy chairs to an AI-guided cornhole tosser and horse-drawn chariot racing.
His recent Interactive Animatronic GLaDOS project was among nine winners in the Hackster AI Innovation Challenge. About 100 contestants vied for prizes from NVIDIA and Sparkfun by creating open-source projects to advance the use of AI in edge computing, robotics and IoT.
Niewinski won first place in the generative AI applications category for his innovative robot based on the GLaDOS guide from game series Portal, the first-person puzzle platform from video game developer Valve.
Other top winners included contestants Andrei Ciobanu and Allen Tao, who took first prize in the generative AI models for the edge and AI at the edge applications categories, respectively. Ciobanu used generative AI to help virtually try on clothes, while Tao developed a ROS-based robot to map the inside of a home to help find things.
Harnessing LLMs for Robots
Niewinski builds custom applications for robotics at his Armoury Labs business in Waterloo, Ontario, Canada, where he uses the NVIDIA Jetson platform for edge AI and robotics, creating open-source tutorials and YouTube videos following his experiences.
He built his interactive GLaDOS robot to create a personal assistant for himself in the lab. It handles queries using Transformer-based speech recognition, text-to-speech, and large language models (LLMs) running onboard an NVIDIA Jetson AGX Orin, which interfaces with a robot arm and camera for interactions.
GLaDOS can track his whereabouts in the lab, move in different directions to face him and respond quickly to queries.
“I like doing things with robots that people will look at and say it’s not what they had immediately expected,” he said.
He wanted the assistant to sound like the original GLaDOS from Portal and respond quickly. Fortunately, the gaming company Valve has put all of the voice lines from Portal and Portal 2 on its website, allowing Niewinski to download the audio to help train a model.
“Using Jetson, your average question-and-answer stuff runs pretty quick for speech,” he said.
Niewinski used NVIDIA’s open-source NeMo toolkit to fine-tune a voice for GLaDOS, training a spectrogram generator network called FastPitch and HiFiGAN vocoder network to refine the audio quality.
Both networks are deployed on Orin with NVIDIA Riva to enable speech recognition and synthesis that’s been optimized to run at many times the real-time rate of speech, so that it can run alongside the LLM while maintaining a smooth, interactive delivery.
For generating realistic responses from GLaDOS, Niewinski uses a locally hosted LLM called OpenChat that he runs in Docker from jetson-containers, saying that it was a drop-in replacement for OpenAI’s API. All of this AI is running on the Jetson module, using the latest open-source ML software stack built with CUDA and JetPack.
To enable GLaDOS to move, Niewinski developed the interactions for a Unitree Z1 robotic arm. It has a stereo camera and models for seeing and tracking a human speaking and a 3D-printed GLaDOS head and body shell around the arm.
Trying on Generative AI for Fashion Fit
Winner Ciobanu, based in Romania, aimed to improve the virtual clothing try-on experience with the help of generative AI, taking a top prize for his EdgeStyle: Fashion Preview at the Edge.
He used AI models such as YOLOv5, SAM and OpenPose to extract and refine data from images and videos. Then he used Stable Diffusion to generate the images, which he said was key to achieving accurate virtual try-ons.
This system taught the model how clothes fit different poses on people, which he said enhanced the realism of the try-ons.
“It’s quite handy as it allows users to see how clothes would look on them without actually trying them on,” said Ciobanu.
The NVIDIA JetPack SDK provided all the tools needed to run AI models smoothly on the Jetson Orin, he said.
“It’s super-helpful to have a stable set of tools, especially when you’re dealing with AI tech that keeps changing,” said Ciobanu. “It really cut down on the time and hassle for us developers, letting us focus more on the cool stuff we’re building instead of getting stuck on tech issues.”
Finding Lost Items With Robot Assistance
Winner Tao, based in Ontario, Canada, created a robot to lessen the burden of searching for things lost around the house. His An Eye for an Item project took top honors at the Hackster challenge.
“Finding lost objects is a chore, and recent developments in zero-shot object detection and LLMs make it feasible for a computer to detect arbitrary objects for us based on textual or pictorial descriptions, presenting an opportunity for automation,” said Tao.
Tao said he needed robot computing capabilities to catalog objects in any unstructured environment — whether a living room or large warehouse. And he needed it to also perform real-time calculations for localization to help with navigation, as well as running inference on larger object detection models.
“Jetson Orin was a perfect fit, supporting all functionality from text and image queries into NanoDB, to real-time odometry feedback, including leveraging Isaac ROS’ hardware-accelerated AprilTag detections for drift correction,” he said.
Other winners of the AI Innovation Challenge include:
George Profenza, Escalator people tracker, 2nd place, Generative AI Applications category
Dimiter Kendri, Cooking meals with a local AI assistant using Jetson AGX Orin, 3rd place, Generative AI Applications category
Vy Phan, ClearWaters Underwater Image Enhancement with Generative AI, 2nd place, Generative AI Models category
Huy Mai, Realtime Language Segment Anything on Jetson Orin, 2nd place, Generative AI Models category
Fakhrur Razi, Autonomous Intelligent Robotic Shopping Cart, 2nd place, AI at the Edge Open category
Team Kinetika, Counting for Inspection and Quality Control with TensorRT, 3rd place, AI at the Edge Open category
Learn more about NVIDIA Jetson Orin for robotics and edge AI applications. Get started creating your own projects at the Jetson AI Lab.
Editor’s note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and which showcases new hardware, software, tools and accelerations for RTX PC users.
Chatbots powered by large-language AI models have transformed computing, and NVIDIA ChatRTX lets users interact with their local data, accelerated by NVIDIA RTX-powered Windows PCs and workstations. A new update, first demoed at GTC in March, expands the power of this RTX-accelerated chatbot app with additional features and support for new models.
The NVIDIA RTX Remix beta update brings NVIDIA DLSS 3.5 with Ray Reconstruction to the modding platform for even more impressive real-time ray tracing.
Say It Out Loud
ChatRTX uses retrieval-augmented generation, NVIDIA TensorRT-LLM software and NVIDIA RTX acceleration to bring chatbot capabilities to RTX-powered Windows PCs and workstations. Backed by its powerful large language models (LLMs), users can query their notes and documents with ChatRTX, which can quickly generate relevant responses, while running locally on the user’s device.
The latest version adds support for additional LLMs, including Gemma, the latest open, local LLM trained by Google. Gemma was developed from the same research and technology used to create the company’s Gemini models and is built for responsible AI development. ChatRTX also now supports ChatGLM3, an open, bilingual (English and Chinese) LLM based on the general language model framework.
Users can also interact with image data thanks to support for Contrastive Language-Image Pre-training from OpenAI. CLIP is a neural network that, through training and refinement, learns visual concepts from natural language supervision — that is, the model recognizes what it’s “seeing” in image collections. With CLIP support in ChatRTX, users can interact with photos and images on their local devices through words, terms and phrases, without the need for complex metadata labeling.
The new ChatRTX release also lets people chat with their data using their voice. Thanks to support for Whisper, an automatic speech recognition system that uses AI to process spoken language, users can send voice queries to the application and ChatRTX will provide text responses.
With RTX Remix, modders can transform classic PC games into RTX remasters using AI-accelerated tools on the NVIDIA Omniverse platform.
Now, they can use DLSS 3.5 with Ray Reconstruction in their projects with just a few clicks, thanks to an update to RTX Remix available this week. Its advanced, AI-powered neural renderer improves the fidelity, responsiveness and quality of ray-traced effects, giving NVIDIA GeForce RTX gamers an even better experience.
AI powers other elements of the Remix workflow, too. Modders can use generative AI texture tools to analyze low-resolution textures from classic games, generate physically accurate materials — including normal and roughness maps — and upscale the resolution by up to 4x. Tools like this also save modders time, quickly handling a task that could otherwise become tedious.
Generative AI is transforming gaming, videoconferencing and interactive experiences of all kinds. Make sense of what’s new and what’s next by subscribing to the AI Decoded newsletter.
Talk about commitment. When startup SEA.AI, an NVIDIA Metropolis partner, set out to create a system that would use AI to scan the seas to enhance maritime safety, entrepreneur Raphael Biancale wasn’t afraid to take the plunge. He donned a lifejacket and jumped into the ocean.
It’s a move that demonstrates Biancale’s commitment and pioneering approach. The startup, founded in 2018 and based in Linz, Austria, with subsidiaries in France, Portugal and the US, had to build its first-of-its-kind training data from scratch in order to train an AI to help oceangoers of all kinds scan the seas.
And to do that, the company needed photos of what a person in the water looked like. That’s when Biancale, now the company’s head of research, walked the plank.
The company has come a long way since then, with a full product line powered by NVIDIA AI technology that lets commercial and recreational sailors detect objects on the seas, whether potential hazards or people needing rescue.
It’s an effort inspired by Biancale’s experience on a night sail. The lack of visibility and situational awareness illuminated the pressing need for advanced safety technologies in the maritime world that AI is bringing to the automotive industry.
AI, of course, is finding its way into all things aquatic. Startup Saildrone’s autonomous sailing vessels can help conduct data-gathering for science, fisheries, weather forecasting, ocean mapping and maritime security. Other researchers are using AI to interpret whale songs and even protect beachgoers from dangerous rip currents.
SEA.AI, however, promises to make the seas safer for everyone who steps aboard a boat. First introduced for ocean racing sailboats, SEA.AI’s system has quickly evolved into an AI-powered situational awareness system that can be deployed on everything from recreational sail and powerboats to commercial shipping vessels.
SEA.AI directly addresses one of the most significant risks for all these vessels: collisions. Thanks to SEA.AI, commercial and recreational oceangoers worldwide can travel with more confidence.
How SEA.AI Works
At the heart of SEA.AI’s approach is a proprietary database of over 9 million annotated marine objects which is growing constantly.
When combined with high-tech optical sensors and the latest AI technology from NVIDIA, SEA.AI’s systems can recognize and classify objects in real-time, significantly improving maritime safety.
SEA.AI technology can detect a person in water up to 700 meters — almost half a mile — away, a dinghy up to 3,000 meters, and motorboats up to 7,500 meters.
This capability ensures maritime operators can identify hazards before they pose a threat. It complements older marine safety systems that rely on radar and satellite signals.
SEA.AI solutions integrate with central marine display units from industry-leading manufacturers like Raymarine, B&G, Garmin and Furuno as well as Android and iOS-based mobile devices. This provides broad applicability across the maritime sector, from recreational vessels to commercial and government ships.
The NVIDIA Jetson edge AI platform is integral to SEA.AI’s success. The platform for robotics and embedded computing applications enables SEA.AI products to achieve unparalleled processing power and efficiency, setting a new standard in maritime safety by quickly detecting, analyzing and alerting operators to objects.
AI Integration and Real-Time Object Detection
SEA.AI uses NVIDIA’s AI and machine vision technology to offer real-time object detection and classification, providing maritime operators with immediate identification of potential hazards.
SEA.AI is bringing its approach to oceangoers of all kinds with three product lines.
One, SEA.AI Sentry, provides 360-degree situational awareness for commercial vessels and motor yachts with features like collision avoidance, object tracking and perimeter surveillance.
Another, SEA.AI Offshore, provides bluewater sailors with high-tech safety and convenience with simplified installation across several editions that can suit different detection and technical needs.
The third, SEA.AI Competition, offers reliable object detection for ocean racing and performance yacht sailors. Its ultra-lightweight design ensures maximum performance when navigating at high speeds.
With a growing team of more than 60 and a distribution network spanning over 40 countries, SEA.AI is charting a course to help ensure every journey on the waves is safer.
The latest trends and technologies in the automotive industry are in the spotlight at the Beijing International Automotive Exhibition, aka Auto China, which opens to the public on Saturday, April 27.
An array of NVIDIA auto partners is embracing this year’s theme, “New Era, New Cars,” by making announcements and showcasing their latest offerings powered by NVIDIA DRIVE, the platform for AI-defined vehicles.
NVIDIA Auto Partners Announce New Vehicles and Technologies
Image courtesy of JIYUE.
Electric vehicle (EV) makers Chery (booth E107) and JIYUE (booth W206), a joint venture between Baidu and Geely (booth W204), announced they have adopted the next-generation NVIDIA DRIVE Thor centralized car computer.
In addition, a number of automakers are building next-gen vehicles on NVIDIA DRIVE Orin, including:
smart, a joint venture between Mercedes-Benz and Geely, previewed its largest and most spacious model to date, an electric SUV called #5. It will be built on its Pilot Assist 3.0 intelligent driving-assistance platform, powered by NVIDIA DRIVE Orin, which supports point-to-point automatic urban navigation. smart #5 will be available for purchase in the second half of this year. smart will be at booth E408.
NIO, a pioneer in the premium smart EV market, unveiled its updated ET7 sedan, featuring upgraded cabin intelligence and smart-driving capabilities. NIO also showcased its 2024 ET5 and ES7. All 2024 models are equipped with four NVIDIA DRIVE Orin systems-on-a-chip (SoCs). Intelligent-driving capabilities in urban areas will fully launch soon. NIO will be at booth E207.
Image courtesy of GWM.
GWM revealed the WEY Blue Mountain (Lanshan) Intelligent Driving Edition, its luxury, high-end SUV. This upgraded vehicle is built on GWM’s Coffee Pilot Ultra intelligent-driving system, powered by NVIDIA DRIVE Orin, and can support features such as urban navigate-on-autopilot (NOA) and cross-floor memory parking. GWM will be at booth E303.
XPENG, a designer and manufacturer of intelligent EVs, announced that it is streamlining the design workflow of its flagship XPENG X9 using the NVIDIA Omniverse platform. In March, XPENG announced it will adopt NVIDIA DRIVE Thor for its next-generation EV fleets. XPENG will be at booth W402.
Innovation on Display
On the exhibition floor, NVIDIA partners are showcasing their NVIDIA DRIVE-powered vehicles:
Image courtesy of BYD.
BYD, DENZA and YANGWANG are featuring their latest vehicles built on NVIDIA DRIVE Orin. The largest EV maker in the world, BYD is building both its Ocean and Dynasty series on NVIDIA DRIVE Orin. In addition, BYDE, a subsidiary of BYD, will tap into the NVIDIA Isaac and NVIDIA Omniverse platforms to develop tools and applications for virtual factory planning and retail configurators. BYD will be at booth W106, DENZA at W408 and YANGWANG at W105.
DeepRoute.ai is showcasing its new intelligent driving-platform, DeepRoute IO, and highlighting its end-to-end model. Powered by NVIDIA DRIVE Orin, the first mass-produced car built on DeepRoute IO will focus on assisted driving and parking. DeepRoute.ai will be at booth W4-W07.
Hyper, a luxury brand owned by GAC AION, is displaying its latest Hyper GT and Hyper HT models, powered by NVIDIA DRIVE Orin. These vehicles feature advanced level 2+ driving capabilities in high-speed environments. Hyper recently announced it selected DRIVE Thor for its next-generation EVs with level 4 driving capabilities. Hyper will be at booth W310.
IM Motors is exhibiting the recently launched L6 Super Intelligent Vehicle. The entire lineup of the IM L6 is equipped with NVIDIA DRIVE Orin to power intelligent driving abilities, including urban NOA features. IM Motors will be at booth W205.
Li Auto is showcasing its recently released L6 model, as well as L7, L8, L9 and MEGA. Models equipped with Li Auto’s AD Max system are powered by dual NVIDIA DRIVE Orin SoCs, which help bring ever-upgrading intelligent functionality to Li Auto’s NOA feature. Li Auto will be at booth E405.
Image courtesy of Lotus.
Lotus is featuring a full range of vehicles, including the Emeya electric hyper-GT powered by NVIDIA DRIVE Orin. Lotus will be at booth E403.
Mercedes-Benz is exhibiting its Concept CLA Class, the first car to be developed on the all-new Mercedes-Benz Modular Architecture. The Concept CLA Class fully runs on MB.OS, which handles infotainment, automated driving, comfort and charging. Mercedes-Benz will be at booth E404.
Momenta is rolling out a new NVIDIA DRIVE Orin solution to accelerate commercialization of urban NOA capabilities at scale.
Image courtesy of Polestar.
Polestar is featuring the Polestar 3, the Swedish car manufacturer’s battery electric mid-size luxury crossover SUV powered by DRIVE Orin. Polestar will be at booth E205.
SAIC R Motors is showcasing the Rising Auto R7 and F7 powered by NVIDIA DRIVE Orin at booth W406.
WeRide is exhibiting Chery’s Exeed Sterra ET SUV and ES sedan, both powered by NVIDIA DRIVE Orin. The vehicles demonstrate progress made by Bosch and WeRide on level 2 to level 3 autonomous-driving technology. WeRide will be at booth E1-W04.
Xiaomi is displaying its NVIDIA DRIVE Orin-powered SU7 and “Human x Car x Home” smart ecosystem, designed to seamlessly connect people, cars and homes, at booth W203.
ZEEKR unveiled its SEA-M architecture and is showcasing the ZEEKR 007 powered by NVIDIA DRIVE Orin at booth E101.
Auto China runs through Saturday, May 4, at the China International Exhibition Center in Beijing.
NVIDIA GeForce NOW (@NVIDIAGFN) 
