Posts in category: nVidia AI

Building and deploying AI applications at scale requires a new class of computing infrastructure — one that can handle the massive amounts of data, compute power and networking bandwidth needed by generative AI models.

To better ensure these models perform optimally and efficiently, NVIDIA is teaming with Cisco to enable enterprise generative AI infrastructure.

Cisco’s new Nexus HyperFabric AI cluster solution, developed in collaboration with NVIDIA, provides a path for enterprises to operationalize generative AI. Cisco HyperFabric is an enterprise-ready, end-to-end infrastructure solution to scale generative AI workloads. It combines NVIDIA accelerated computing and AI software with Cisco AI-native networking and a robust VAST Data Platform.

“Enterprise applications are transforming into generative AI applications, significantly increasing data processing requirements and overall infrastructure complexity,” said Kevin Wollenweber, senior vice president and general manager of data center and provider connectivity at Cisco. “Together, Cisco and NVIDIA are advancing HyperFabric to advance generative AI for the world’s enterprises so they can use their data and domain expertise to transform productivity and insight.”

Powering a Full-Stack AI Fabric

Foundational to the solution are NVIDIA Tensor Core GPUs, which provide the accelerated computing needed to process massive datasets. The solution utilizes NVIDIA AI Enterprise, a cloud-native software platform that acts as the operating system for enterprise AI. NVIDIA AI Enterprise streamlines the development and deployment of production-grade AI copilots and other generative AI applications, ensuring optimized performance, security and application programming interface stability.

Included with NVIDIA AI Enterprise, NVIDIA NIM inference microservices accelerate the deployment of foundation models while ensuring data security. NIM microservices are designed to bridge the gap between complex AI development and enterprise operational needs. As organizations across various industries embark on their AI journeys, the combination of NVIDIA NIM and the Cisco Nexus HyperFabric AI cluster supports the entire process, from ideation to development and deployment of production-scale AI applications.

The Cisco Nexus HyperFabric AI cluster solution integrates NVIDIA Tensor Core GPUs and NVIDIA BlueField-3 SuperNICs and DPUs to enhance system performance and security. The SuperNICs offer advanced network capabilities, ensuring seamless, high-speed connectivity across the infrastructure. BlueField-3 DPUs offload, accelerate and isolate the infrastructure services, creating a more efficient AI solution.

BlueField-3 DPUs can also run security services like the Cisco Hypershield solution. It enables an AI-native, hyperdistributed security architecture, where security shifts closer to the workloads needing protection. Cisco Hypershield is another notable area of collaboration between the companies, focusing on creating AI-powered security solutions.

Join NVIDIA at Cisco Live

Learn more about how Cisco and NVIDIA power generative AI at Cisco Live — running through June 6 in Las Vegas — where the companies will showcase NVIDIA AI technologies at the Cisco AI Hub and share best practices for enterprises to get started with AI.

Attend these sessions to discover how to accelerate generative AI with NVIDIA, Cisco and other ecosystem partners:

• Keynote Deep Dive: “Harness a Bold New Era: Transform Data Center and Service Provider Connectivity” with NVIDIA’s Kevin Deierling and Cisco’s Jonathan Davidson, Kevin Wollenweber, Jeremy Foster and Bill Gartner — Wednesday, June 5, from 1-2 p.m. PT
• AI Hub Theater Presentation: “Accelerate, Deploy Generative AI Anywhere With NVIDIA Inference Microservices” with Marty Jain, vice president of sales and business development at NVIDIA — Tuesday, June 4, from 2:15-2:45 p.m. PT
• WWT AI Hub Booth: Thought leadership interview with NVIDIA’s Jain and WWT Vice President of Cloud, Infrastructure and AI Solutions Neil Anderson — Wednesday, June 5, from 10-11 a.m. PT
• NetApp Theater: “Accelerating Gen AI With NVIDIA Inference Microservices on FlexPod” with Sicong Ji, strategic platforms and solutions lead at NVIDIA — Wednesday, June 5, from 1:30-1:40 p.m. PT
• Pure Storage Theater: “Accelerating Gen AI With NVIDIA Inference Microservices on FlashStack” with Joslyn Shakur, sales alliance manager at NVIDIA — Wednesday, June 5, from 2-2:10 p.m. PT ​

Sign up for generative AI news to stay up to date on the latest breakthroughs, developments and technologies.

Read More

European neobank bunq is debunking financial fraudsters with the help of NVIDIA accelerated computing and AI.

Dubbed “the bank of the free,” bunq offers online banking anytime, anywhere. Through the bunq app, users can handle all their financial needs exclusively online, without needing to visit a physical bank.

With more than 12 million customers and 8 billion euros’ worth of deposits made to date, bunq has become one of the largest neobanks in the European Union. Founded in 2012, it was the first bank to obtain a European banking license in over three decades.

To meet growing customer needs, bunq turned to generative AI to help detect fraud and money laundering. Its automated transaction-monitoring system, powered by NVIDIA accelerated computing, greatly improved its training speed.

“AI has enormous potential to help humanity in so many ways, and this is a great example of how human intelligence can be coupled with AI,” said Ali el Hassouni, head of data and AI at bunq.

Faster Fraud Detection

Financial fraud is more prevalent than ever, el Hassouni said in a recent talk at NVIDIA GTC.

Traditional transaction-monitoring systems are rules based, meaning algorithms flag suspicious transactions according to a set of criteria that determine if an activity presents risk of fraud or money laundering. These criteria must be manually set, resulting in high false-positive rates and making such systems labor intensive and difficult to scale.

Instead, using supervised and unsupervised learning, bunq’s AI-powered transaction-monitoring system is completely automated and easily scalable.

Bunq achieved this using NVIDIA GPUs, which accelerated its data processing pipeline more than 5x.

In addition, compared with previous methods, bunq trained its fraud-detection model nearly 100x faster using the open-source NVIDIA RAPIDS suite of GPU-accelerated data science libraries.

RAPIDS is part of the NVIDIA AI Enterprise software platform, which accelerates data science pipelines and streamlines the development and deployment of production-grade generative AI applications.

“We chose NVIDIA’s advanced, GPU-optimized software, as it enables us to use larger datasets and speed the training of new models — sometimes by an order of magnitude — resulting in improved model accuracy and reduced false positives,” said el Hassouni.

AI Across the Bank

Bunq is seeking to tap AI’s potential across its operations.

“We’re constantly looking for new ways to apply AI for the benefit of our users,” el Hassouni said. “More than half of our user tickets are handled automatically. We also use AI to spot fake IDs when onboarding new users, automate our marketing efforts and much more.”

Finn, a personal AI assistant available to bunq customers, is powered by the company’s proprietary large language model and generative AI. It can answer user questions like, “How much did I spend on groceries last month?” and “What’s the name of the Indian restaurant I ate at last week?”

The company is exploring NVIDIA NeMo Retriever, a collection of generative AI microservices available in early access, to further improve Finn’s accuracy. NeMo Retriever is a part of NVIDIA NIM inference microservices, which provide models as optimized containers, available with NVIDIA AI Enterprise.

“Our initial testing of NeMo Retriever embedding NIM has been extremely positive, and our collaboration with NVIDIA on LLMs is poised to help us to take Finn to the next level and enhance customer experience,” el Hassouni said. 

Plus, for the digital bank’s marketing efforts, AI helps analyze consumer engagement metrics to inform future campaigns.

“We’re creating a borderless banking experience for our users, always keeping them at the heart of everything we do,” el Hassouni said.

Watch bunq’s NVIDIA GTC session on demand and subscribe to NVIDIA financial services news. 

Learn more about AI and financial services at Money20/20 Europe, a fintech conference running June 4-6 in Amsterdam, where NVIDIA will host an AI Summit in collaboration with AWS, and where bunq will present on a panel about AI for fraud detection.

Read More

“Generative AI is reshaping industries and opening new opportunities for innovation and growth,” NVIDIA founder and CEO Jensen Huang said in an address ahead of this week’s COMPUTEX technology conference in Taipei.

“Today, we’re at the cusp of a major shift in computing,” Huang told the audience, clad in his trademark black leather jacket. “The intersection of AI and accelerated computing is set to redefine the future.”

Huang spoke ahead of one of the world’s premier technology conferences to an audience of more than 6,500 industry leaders, press, entrepreneurs, gamers, creators and AI enthusiasts gathered at the glass-domed National Taiwan University Sports Center set in the verdant heart of Taipei.

The theme: NVIDIA accelerated platforms are in full production, whether through AI PCs and consumer devices featuring a host of NVIDIA RTX-powered capabilities or enterprises building and deploying AI factories with NVIDIA’s full-stack computing platform.

“The future of computing is accelerated,” Huang said. “With our innovations in AI and accelerated computing, we’re pushing the boundaries of what’s possible and driving the next wave of technological advancement.”
 

‘One-Year Rhythm’

More’s coming, with Huang revealing a roadmap for new semiconductors that will arrive on a one-year rhythm. Revealed for the first time, the Rubin platform will succeed the upcoming Blackwell platform, featuring new GPUs, a new Arm-based CPU — Vera — and advanced networking with NVLink 6, CX9 SuperNIC and the X1600 converged InfiniBand/Ethernet switch.

“Our company has a one-year rhythm. Our basic philosophy is very simple: build the entire data center scale, disaggregate and sell to you parts on a one-year rhythm, and push everything to technology limits,” Huang explained.

NVIDIA’s creative team used AI tools from members of the NVIDIA Inception startup program, built on NVIDIA NIM and NVIDIA’s accelerated computing, to create the COMPUTEX keynote. Packed with demos, this showcase highlighted these innovative tools and the transformative impact of NVIDIA’s technology.

‘Accelerated Computing Is Sustainable Computing’

NVIDIA is driving down the cost of turning data into intelligence, Huang explained as he began his talk.

“Accelerated computing is sustainable computing,” he emphasized, outlining how the combination of GPUs and CPUs can deliver up to a 100x speedup while only increasing power consumption by a factor of three, achieving 25x more performance per Watt over CPUs alone.

“The more you buy, the more you save,” Huang noted, highlighting this approach’s significant cost and energy savings.

Industry Joins NVIDIA to Build AI Factories to Power New Industrial Revolution

Leading computer manufacturers, particularly from Taiwan, the global IT hub, have embraced NVIDIA GPUs and networking solutions. Top companies include ASRock Rack, ASUS, GIGABYTE, Ingrasys, Inventec, Pegatron, QCT, Supermicro, Wistron and Wiwynn, which are creating cloud, on-premises and edge AI systems.

The NVIDIA MGX modular reference design platform now supports Blackwell, including the GB200 NVL2 platform, designed for optimal performance in large language model inference, retrieval-augmented generation and data processing.

AMD and Intel are supporting the MGX architecture with plans to deliver, for the first time, their own CPU host processor module designs. Any server system builder can use these reference designs to save development time while ensuring consistency in design and performance.

Next-Generation Networking with Spectrum-X

In networking, Huang unveiled plans for the annual release of Spectrum-X products to cater to the growing demand for high-performance Ethernet networking for AI.

NVIDIA Spectrum-X, the first Ethernet fabric built for AI, enhances network performance by 1.6x more than traditional Ethernet fabrics. It accelerates the processing, analysis and execution of AI workloads and, in turn, the development and deployment of AI solutions.

CoreWeave, GMO Internet Group, Lambda, Scaleway, STPX Global and Yotta are among the first AI cloud service providers embracing Spectrum-X to bring extreme networking performance to their AI infrastructures.

NVIDIA NIM to Transform Millions Into Gen AI Developers

With NVIDIA NIM, the world’s 28 million developers can now easily create generative AI applications. NIM — inference microservices that provide models as optimized containers — can be deployed on clouds, data centers or workstations.

NIM also enables enterprises to maximize their infrastructure investments. For example, running Meta Llama 3-8B in a NIM produces up to 3x more generative AI tokens on accelerated infrastructure than without NIM.

Nearly 200 technology partners — including Cadence, Cloudera, Cohesity, DataStax, NetApp, Scale AI, and Synopsys — are integrating NIM into their platforms to speed generative AI deployments for domain-specific applications, such as copilots, code assistants, digital human avatars and more. Hugging Face is now offering NIM — starting with Meta Llama 3.

“Today we just posted up in Hugging Face the Llama 3 fully optimized, it’s available there for you to try. You can even take it with you,” Huang said. “So you could run it in the cloud, run it in any cloud, download this container, put it into your own data center, and you can host it to make it available for your customers.”

NVIDIA Brings AI Assistants to Life With GeForce RTX AI PCs

NVIDIA’s RTX AI PCs, powered by RTX technologies, are set to revolutionize consumer experiences with over 200 RTX AI laptops and more than 500 AI-powered apps and games.

The RTX AI Toolkit and newly available PC-based NIM inference microservices for the NVIDIA ACE digital human platform underscore NVIDIA’s commitment to AI accessibility.

Project G-Assist, an RTX-powered AI assistant technology demo, was also announced, showcasing context-aware assistance for PC games and apps.

And Microsoft and NVIDIA are collaborating to help developers bring new generative AI capabilities to their Windows native and web apps with easy API access to RTX-accelerated SLMs that enable RAG capabilities that run on-device as part of Windows Copilot Runtime.

NVIDIA Robotics Adopted by Industry Leaders

NVIDIA is spearheading the $50 trillion industrial digitization shift, with sectors embracing autonomous operations and digital twins — virtual models that enhance efficiency and cut costs. Through its Developer Program, NVIDIA offers access to NIM, fostering AI innovation.

Taiwanese manufacturers are transforming their factories using NVIDIA’s technology, with Huang showcasing Foxconn’s use of NVIDIA Omniverse, Isaac and Metropolis to create digital twins, combining vision AI and robot development tools for enhanced robotic facilities.

“The next wave of AI is physical AI. AI that understands the laws of physics, AI that can work among us,” Huang said, emphasizing the importance of robotics and AI in future developments.

The NVIDIA Isaac platform provides a robust toolkit for developers to build AI robots, including AMRs, industrial arms and humanoids, powered by AI models and supercomputers like Jetson Orin and Thor.

“Robotics is here. Physical AI is here. This is not science fiction, and it’s being used all over Taiwan. It’s just really, really exciting,” Huang added.

Global electronics giants are integrating NVIDIA’s autonomous robotics into their factories, leveraging simulation in Omniverse to test and validate this new wave of AI for the physical world. This includes over 5 million preprogrammed robots worldwide.

“All the factories will be robotic. The factories will orchestrate robots, and those robots will be building products that are robotic,” Huang explained.

Huang emphasized NVIDIA Isaac’s role in boosting factory and warehouse efficiency, with global leaders like BYD Electronics, Siemens, Teradyne Robotics and Intrinsic adopting its advanced libraries and AI models.

NVIDIA AI Enterprise on the IGX platform, with partners like ADLINK, Advantech and ONYX, delivers edge AI solutions meeting strict regulatory standards, essential for medical technology and other industries.

Huang ended his keynote on the same note he began it on, paying tribute to Taiwan and NVIDIA’s many partners there. “Thank you,” Huang said. “I love you guys.”

Read More

Deploying generative AI in the enterprise is about to get easier than ever.

NVIDIA NIM, a set of generative AI inference microservices, will work with KServe, open-source software that automates putting AI models to work at the scale of a cloud computing application.

The combination ensures generative AI can be deployed like any other large enterprise application. It also makes NIM widely available through platforms from dozens of companies, such as Canonical, Nutanix and Red Hat.

The integration of NIM on KServe extends NVIDIA’s technologies to the open-source community, ecosystem partners and customers. Through NIM, they can all access the performance, support and security of the NVIDIA AI Enterprise software platform with an API call — the push-button of modern programming.

Serving AI on Kubernetes

KServe got its start as part of Kubeflow, a machine learning toolkit based on Kubernetes, the open-source system for deploying and managing software containers that hold all the components of large distributed applications.

As Kubeflow expanded its work on AI inference, what became KServe was born and ultimately evolved into its own open-source project.

Many companies have contributed to and adopted the KServe software that runs today at companies including AWS, Bloomberg, Canonical, Cisco, Hewlett Packard Enterprise, IBM, Red Hat, Zillow and NVIDIA.

Under the Hood With KServe

KServe is essentially an extension of Kubernetes that runs AI inference like a powerful cloud application. It uses a standard protocol, runs with optimized performance and supports PyTorch, Scikit-learn, TensorFlow and XGBoost without users needing to know the details of those AI frameworks.

The software is especially useful these days, when new large language models (LLMs) are emerging rapidly.

KServe lets users easily go back and forth from one model to another, testing which one best suits their needs. And when an updated version of a model gets released, a KServe feature called “canary rollouts” automates the job of carefully validating and gradually deploying it into production.

Another feature, GPU autoscaling, efficiently manages how models are deployed as demand for a service ebbs and flows, so customers and service providers have the best possible experience.

An API Call to Generative AI

The goodness of KServe will now be available with the ease of NVIDIA NIM.

With NIM, a simple API call takes care of all the complexities. Enterprise IT admins get the metrics they need to ensure their application is running with optimal performance and efficiency, whether it’s in their data center or on a remote cloud service — even if they change the AI models they’re using.

NIM lets IT professionals become generative AI pros, transforming their company’s operations. That’s why a host of enterprises such as Foxconn and ServiceNow are deploying NIM microservices.

NIM Rides Dozens of Kubernetes Platforms

Thanks to its integration with KServe, users will be able access NIM on dozens of enterprise platforms such as Canonical’s Charmed KubeFlow and Charmed Kubernetes, Nutanix GPT-in-a-Box 2.0, Red Hat’s OpenShift AI and many others.

“Red Hat has been working with NVIDIA to make it easier than ever for enterprises to deploy AI using open source technologies,” said KServe contributor Yuan Tang, a principal software engineer at Red Hat. “By enhancing KServe and adding support for NIM in Red Hat OpenShift AI, we’re able to provide streamlined access to NVIDIA’s generative AI platform for Red Hat customers.”

“Through the integration of NVIDIA NIM inference microservices with Nutanix GPT-in-a-Box 2.0, customers will be able to build scalable, secure, high-performance generative AI applications in a consistent way, from the cloud to the edge,” said the vice president of engineering at Nutanix, Debojyoti Dutta, whose team contributes to KServe and Kubeflow.

“As a company that also contributes significantly to KServe, we’re pleased to offer NIM through Charmed Kubernetes and Charmed Kubeflow,” said Andreea Munteanu, MLOps product manager at Canonical. “Users will be able to access the full power of generative AI, with the highest performance, efficiency and ease thanks to the combination of our efforts.”

Dozens of other software providers can feel the benefits of NIM simply because they include KServe in their offerings.

Serving the Open-Source Community

NVIDIA has a long track record on the KServe project. As noted in a recent technical blog, KServe’s Open Inference Protocol is used in NVIDIA Triton Inference Server, which helps users run many AI models simultaneously across many GPUs, frameworks and operating modes.

With KServe, NVIDIA focuses on use cases that involve running one AI model at a time across many GPUs.

As part of the NIM integration, NVIDIA plans to be an active contributor to KServe, building on its portfolio of contributions to open-source software that includes Triton and TensorRT-LLM. NVIDIA is also an active member of the Cloud Native Computing Foundation, which supports open-source code for generative AI and other projects.

Try the NIM API on the NVIDIA API Catalog using the Llama 3 8B or Llama 3 70B LLM models today. Hundreds of NVIDIA partners worldwide are using NIM to deploy generative AI.

Watch NVIDIA founder and CEO Jensen Huang’s COMPUTEX keynote to get the latest on AI and more.

Read More

Taiwan’s leading consumer electronics giants are making advances with AI automation for manufacturing, as fleets of robots and millions of cameras and sensors drive efficiencies across the smart factories of the future.

Dozens of electronics manufacturing and automation specialists — including Foxconn, Pegatron and Wistron — are showcasing their use of the NVIDIA software at COMPUTEX, in Taipei, and are called out in NVIDIA founder and CEO Jensen Huang’s keynote address.

Companies displayed the latest in computer vision and generative AI using NVIDIA Metropolis for everything from automating product manufacturing to improving worker safety and device performance.

Creating Factory Autonomy

With increasing production challenges, manufacturers are seeing a need to turn factories into autonomous machines, with generative AI and digital twins as a foundation. AI agents — driven by large language models (LLMs) — are being built that can talk and assist on warehouse floors to boost productivity and increase safety. And digital twins are helping manufacturers simulate and develop factories and AI-powered automation before being deployed in real factories.

Foxconn and its Ingrasys subsidiary use NVIDIA Omniverse and Metropolis to build digital twins for factories, planning efficiency optimizations and worker safety improvement at a number of manufacturing sites. At COMPUTEX, Foxconn is showing how it uses digital twins to plan placements of many video cameras in factories to optimize its data capture for collecting key insights.

Bringing Generative AI to the Factory Floor

Generative AI is creating productivity leaps across industries. Researcher McKinsey forecasts that generative AI will deliver as much as $290 billion in value for the advanced manufacturing industry, while bringing $4.4 trillion annually to the global economy.

At GTC in March, NVIDIA launched NVIDIA NIM, a set of microservices designed to speed up generative AI deployment in enterprises. Supporting a wide range of AI models, it ensures seamless, scalable AI inferencing, on premises or in the cloud, using industry-standard application programming interfaces.

Billions of IoT devices worldwide can tap into Metropolis and NVIDIA NIM for improvements in AI perception to enhance their capabilities.

Advancing Manufacturing With NVIDIA NIM

Linker Vision, an AI vision insights specialist, is adopting NVIDIA NIM to assist factories in deploying AI agents that can respond to natural language queries.

The Taipei company uses NVIDIA Visual Insight Agent (VIA) in manufacturing environments for always-on video feed monitoring of factory floors. With user prompts, these ChatGPT-like systems can enable operators to ask for video of factory floors to be monitored for insights and safety alerts, like when workers are not wearing hardhats.

Operators can ask questions and receive instant, context-aware responses from AI agents, which can tap into organizational knowledge via retrieval-augmented generation, an integration of AI that can enhance operational efficiency.

Leading manufacturer Pegatron has factories that span more than 20 million square feet and the facilities process and build more than 15 million assemblies per month, while deploying more than 3,500 robots across factory floors. It has announced efforts based on NVIDIA NIM and is using Metropolis multi-camera tracking reference workflows to help with worker safety and productivity on factory lines. Pegatron’s workflow fuses digital twins in Omniverse and Metropolis real-time AI to better monitor and optimize operations.

Boosting Automated Visual Inspections

Adoption of NVIDIA Metropolis is helping Taiwan’s largest electronics manufacturers streamline operations and reduce cost as they build and inspect some of the world’s most complex and high-volume products.

Quality control with manual inspections in manufacturing is a multitrillion-dollar challenge. While automated optical inspection systems have been relied upon for some time, legacy AOI systems have high false detection rates, requiring costly secondary manual inspections for verification.

NVIDIA Metropolis for Factories offers a state-of-the-art AI reference workflow for bringing sophisticated and accurate AOI inspection applications to production faster.

TRI, Taiwan’s leading AOI equipment maker, has announced integrating NVIDIA Metropolis for Factories workflow and capabilities into its latest AOI systems and is also planning to use NVIDIA NIM to further optimize system performance.

Wistron is expanding its OneAI platform for visual inspection and AOI with Metropolis. OneAI has been deployed in more than 10 Wistron factories globally, spanning hundreds of inspection points.

MediaTek, a leading innovator in connectivity and multimedia, and one of Taiwan’s largest IoT silicon vendors, announced at COMPUTEX that it’s teaming with NVIDIA to integrate NVIDIA TAO training and pretrained models into its AI development workflow for IoT device customers. The collaboration brings Metropolis and the latest advances in AI and visual perception to billions of IoT far-edge devices and streamlines software development for MediaTek’s next phase of growth in edge IoT.

Learn about NVIDIA Metropolis for Factories, NVIDIA NIM and the NVIDIA Metropolis multi-camera tracking workflow, which developers can use to build state-of-the-art real-time locating services and worker safety into their factory or warehouse operations. 

Read More

Foxconn operates more than 170 factories around the world — the latest one a virtual plant pushing the state of the art in industrial automation.

It’s the digital twin of a new factory in Guadalajara, hub of Mexico’s electronics industry. Foxconn’s engineers are defining processes and training robots in this virtual environment, so the physical plant can produce at high efficiency the next engine of accelerated computing, NVIDIA Blackwell HGX systems.

To design an optimal assembly line, factory engineers need to find the best placement for dozens of robotic arms, each weighing hundreds of pounds. To accurately monitor the overall process, they situate thousands of sensors, including many networked video cameras in a matrix to show plant operators all the right details.

Virtual Factories Create Real Savings

Such challenges are why companies like Foxconn are increasingly creating virtual factories for simulation and testing.

“Our digital twin will guide us to new levels of automation and industrial efficiency, saving time, cost and energy,” said Young Liu, chairman of the company that last year had revenues of nearly $200 billion.

Based on its efforts so far, the company anticipates that it can increase the manufacturing efficiency of complex servers using the simulated plant, leading to significant cost savings and reducing kilowatt-hour usage by over 30% annually.

Foxconn Teams With NVIDIA, Siemens

Foxconn is building its digital twin with software from the Siemens Xcelerator portfolio including Teamcenter and NVIDIA Omniverse, a platform for developing 3D workflows and applications based on OpenUSD.

NVIDIA and Siemens announced in March that they will connect Siemens Xcelerator applications to NVIDIA Omniverse Cloud API microservices. Foxconn will be among the first to employ the combined services, so its digital twin is physically accurate and visually realistic.

Engineers will employ Teamcenter with Omniverse APIs to design robot work cells and assembly lines. Then they’ll use Omniverse to pull all the 3D CAD elements into one virtual factory where their robots will be trained with NVIDIA Isaac Sim.

Robots Attend a Virtual School

A growing set of manufacturers is building digital twins to streamline factory processes. Foxconn is among the first to take the next step in automation — training their AI robots in the digital twin.

Inside the Foxconn virtual factory, robot arms from manufacturers such as Epson can learn how to see, grasp and move objects with NVIDIA Isaac Manipulator, a collection of NVIDIA-accelerated libraries and AI foundation models for robot arms.

For example, the robot arms may learn how to pick up a Blackwell server and place it on an autonomous mobile robot (AMR). The arms can use Isaac Manipulator’s cuMotion to find inspection paths for products, even when objects are placed in the way.

Foxconn’s AMRs, from Taiwan’s FARobot, will learn how to see and navigate the factory floor using NVIDIA Perceptor, software that helps them build a real-time 3D map that indicates any obstacles. The robot’s routes are generated and optimized by NVIDIA cuOpt, a world-record holding route optimization microservice.

Unlike many transport robots that need to stick to carefully drawn lines on the factory floor, these smart AMRs will navigate around obstacles to get wherever they need to go.

A Global Trend to Industrial Digitization

The Guadalajara factory is just the beginning. Foxconn is starting to design digital twins of factories around the world, including one in Taiwan where it will manufacture electric buses.

Foxconn is also deploying NVIDIA Metropolis, an application framework for smart cities and spaces, to give cameras on the shop floor AI-powered vision. That gives plant managers deeper insights into daily operations and opportunities to further streamline operations and improve worker safety.

With an estimated 10 million factories worldwide, the $46 trillion manufacturing sector is a rich field for industrial digitalization.

Delta Electronics, MediaTek, MSI and Pegatron are among other top electronics makers revealed at COMPUTEX this week how they’re using NVIDIA AI and Omniverse to build digital twins of their factories.

Like Foxconn, they’re racing to make their factories more agile, autonomous and sustainable to serve the demand for more than a billion smartphones, PCs and servers a year.

A reference architecture shows how to develop factory digital twins  with the NVIDIA AI and Omniverse platforms. And learn about the experiences of five companies doing this work.

Watch NVIDIA founder and CEO Jensen Huang’s COMPUTEX keynote to get the latest on AI and more.

Read More

Meta Llama 3, Meta’s openly available state-of-the-art large language model — trained and optimized using NVIDIA accelerated computing — is dramatically boosting healthcare and life sciences workflows, helping deliver applications that aim to improve patients’ lives.

Now available as a downloadable NVIDIA NIM inference microservice at ai.nvidia.com, Llama 3 is equipping healthcare developers, researchers and companies to innovate responsibly across a wide variety of applications. The NIM comes with a standard application programming interface that can be deployed anywhere.

For use cases spanning surgical planning and digital assistants to drug discovery and clinical trial optimization, developers can use Llama 3 to easily deploy optimized generative AI models for copilots, chatbots and more.

At COMPUTEX, one of the world’s premier technology events, NVIDIA today announced that hundreds of AI ecosystem partners are embedding NIM into their solutions.

More than 40 of these adopters are healthcare and life sciences startups and enterprises using the Llama 3 NIM to build and run applications that accelerate digital biology, digital surgery and digital health.

Advancing Digital Biology

Techbio and pharmaceutical companies, along with life sciences platform providers, use NVIDIA NIM for generative biology, chemistry and molecular prediction. With the Llama 3 NIM for intelligent assistants and NVIDIA BioNeMo NIM microservices for digital biology, researchers can build and scale end-to-end workflows for drug discovery and clinical trials.

Deloitte is driving efficiency for garnering data-based insights from gene to function for research copilots, scientific research mining, chemical property prediction and drug repurposing with its Atlas AI drug discovery accelerator, powered by the NVIDIA BioNeMo, NeMo and Llama 3 NIM microservices.

Transcripta Bio harnesses Llama 3 and BioNeMo for accelerated intelligent drug discovery. Its proprietary artificial intelligence modeling suite, Conductor AI, uses its Drug-Gene Atlas to help discover and predict the effects of new drugs at transcriptome scale.

Bolstering Clinical Trials

Quantiphi — an AI-first digital engineering company and an Elite Service Delivery Partner in the NVIDIA Partner Network — is using NVIDIA NIM to develop generative AI solutions for clinical research and development, diagnostics and patient care. These innovations are enabling organizations to save substantial cost, enhance workforce productivity and improve patient outcomes.

ConcertAI is advancing a broad set of translational and clinical development solutions within its CARA AI platform. The company has integrated the Llama 3 NIM to support population-scale patient matching to clinical trials, study automation and research site copilots with real-time insights and model management for large-scale AI applications.

Mendel AI is developing clinically focused AI solutions that can understand nuances in medical data at scale to provide actionable insights, with applications across clinical research, real-world evidence generation and cohort selection. It has deployed a fine-tuned Llama 3 NIM for its Hypercube copilot, offering a 36% performance improvement. Mendel is also exploring potential use cases with Llama 3 NIM to extract clinical information from patient records and to translate natural language into clinical queries.

Improving Digital Surgery

The operating room is bolstered by AI and the latest digital technologies, too.

Activ Surgical is using Llama 3 to accelerate development of its AI copilot and augmented-reality solution for real-time surgical guidance. The company’s ActivSight technology, which allows surgeons to view critical physiological structures and functions, aims to reduce surgical complication rates, improving patient care and safety.

Enhancing Digital Health

Generative AI-powered digital health applications enhance patient-doctor interactions, helping to improve patient outcomes and deliver more efficient healthcare.

Precision medicine company SimBioSys recently downloaded the Llama 3 NIM to help analyze a breast cancer patient’s diagnosis and tailor guidance for the physician regarding the patient’s unique characteristics.

Artisight, a startup focused on smart hospital transformation, uses Llama 3 to automate documentation and care coordination in all its clinical locations with ambient voice and vision systems.

AITEM, which offers medical and veterinary AI diagnostic solutions, is building healthcare-specific chatbots with the model.

And Abridge, which offers a generative AI platform for clinical conversations, is using the NIM to build a physician-patient encounter summarization solution.

Transcripta Bio, Activ Surgical, SimBioSys, Artisight, AITEM and Abridge are all members of NVIDIA Inception, a free program that helps startups evolve faster through cutting-edge technology, opportunities to connect with venture capitalists and access to the latest technical resources from NVIDIA.

The NVIDIA NIM collection of inference microservices is available with NVIDIA AI Enterprise, a software platform that streamlines development and deployment of production-grade copilots and other generative AI applications.

Download the Meta Llama 3 NIM now and learn more about how generative AI is reshaping healthcare and other industries by joining NVIDIA at COMPUTEX, running through June 7 in Taipei, Taiwan.

Read More

Fueled by generative AI, enterprises globally are creating “AI factories,” where data comes in and intelligence comes out.

Critical to this movement are validated systems and reference architectures that reduce the risk and time involved in deploying specialized infrastructure that can support complex, computationally intensive generative AI workloads.

At the COMPUTEX trade show, NVIDIA today announced the expansion of its NVIDIA-Certified Systems program, which designates leading partner systems as suited for AI and accelerated computing, so customers can confidently deploy these platforms from the data center to the edge.

Two new certification types are now included: NVIDIA-Certified Spectrum-X Ready systems for AI in the data center and NVIDIA-Certified IGX systems for AI at the edge.

Each NVIDIA-Certified System undergoes rigorous testing and is validated to provide enterprise-grade performance, manageability, security and scalability for NVIDIA AI Enterprise software workloads, including generative AI applications built with NVIDIA NIM inference microservices. The systems provide a trusted pathway to design and implement efficient, reliable infrastructure.

NVIDIA-Certified Spectrum-X Ready Systems

The world’s first Ethernet fabric built for AI, the NVIDIA Spectrum-X AI Ethernet platform combines the NVIDIA Spectrum-4 SN5000 Ethernet switch series, NVIDIA BlueField-3 SuperNICs and networking acceleration software to deliver 1.6x AI networking performance over traditional Ethernet fabrics.

NVIDIA-Certified Spectrum-X Ready servers will act as building blocks for high-performance AI computing clusters and support powerful NVIDIA Hopper architecture and NVIDIA L40S GPUs.

NVIDIA-Certified IGX Systems

NVIDIA IGX Orin is an enterprise-ready AI platform for the industrial edge and medical applications that features industrial-grade hardware, a production-grade software stack and long-term enterprise support. It includes the latest technologies in device security, remote provisioning and management, along with built-in extensions, to deliver high-performance AI and proactive safety for low-latency, real-time applications in such areas as medical diagnostics, manufacturing, industrial robotics, agriculture and more.

Expanding Partner Portfolio

Top NVIDIA ecosystem partners are set to achieve the new certifications.

ASUS, Dell Technologies, GIGABYTE, Hewlett Packard Enterprise, Ingrasys, Lenovo, QCT and Supermicro will soon offer NVIDIA-Certified Spectrum-X Ready systems.

And NVIDIA-Certified IGX systems will soon be available from ADLINK, Advantech, Aetina, Ahead, Cosmo Intelligent Medical Devices (a division of Cosmo Pharmaceuticals), Dedicated Computing, Leadtek, Onyx and YUAN.

Learn more about NVIDIA-Certified Systems and the latest generative AI technologies by joining NVIDIA at COMPUTEX.

Read More

Taiwan’s leading medical centers — the National Health Research Institute (NHRI) and Chang Gung Memorial Hospital (CGMH) — are set to advance biomedical research and healthcare for patients.

The centers are embracing accelerated computing and generative AI for everything from imaging to enhancing patient care, from streamlining clinical workflows to drug discovery research.

“The use of AI in healthcare will fundamentally change the way we approach disease prevention and treatment,” said Dr. Hung-Yi Chiou, director of the Institute of Population Health Sciences (IPHS) at NHRI. “With AI’s ability to analyze vast amounts of data quickly and accurately, we can develop personalized medicine strategies and early intervention methods that were previously unattainable.”

“The future of AI in healthcare is incredibly promising,” said Dr. Wen-Jin Cherng at CGMH.

With the assistance of AI in smart healthcare, future diagnoses will become more accurate, treatment plans will have better predictions and patients will experience faster recovery, Dr. Cherng explained. And in complex analytical processes, AI can enable more efficient and cost-effective decision-making in healthcare, he added.

“The transformative potential of the NVIDIA Blackwell platform allows us to integrate advanced AI capabilities into our medical practices, enhancing patient care and streamlining clinical workflows like never before,” he said.

NHRI, the leading medical research institution in Taiwan, plays a crucial role in advancing public health through biomedical research and innovation. The integration of NVIDIA accelerated computing into its IT infrastructure marks a significant leap forward in the realm of AI-driven healthcare.

NHRI’s collaboration with NVIDIA also extends to the development of large language models tailored specifically for Taiwan’s healthcare needs.

“Traditional Chinese medical records and genomic data present unique challenges that require localized solutions,” said Dr. Feng-Chi Chen, deputy director of IPHS at the NHRI.

These challenges include the complexity of language variations and the need for precise genomic interpretations specific to Taiwan’s population, Dr. Chen explained.

“NVIDIA accelerated computing enables us to create these solutions, ensuring that our healthcare system remains at the cutting edge of medical research,” he said.

CGMH, one of the largest healthcare systems in Taiwan, operates a network of 10 hospitals with a combined inpatient capacity of over 11,000 beds. It also serves millions of people in outpatient services. It’s a cornerstone of Taiwan’s healthcare system, which is one of the most advanced in the world.

“With the computational power of Blackwell, we can expand our language model services to all hospitals under our umbrella, enhancing professional support, patient care and streamlining clinical workflows,” said Dr. Chang-Fu Kuo, director of the AI center at CGMH. “It addresses the needs of various medical disciplines and diverse patient populations, enabling healthcare professionals to focus on critical clinical tasks and ultimately improve patient outcomes.”

NHRI, CGMH Pioneering Medical AI

NHRI currently uses six NVIDIA DGX A100 systems for cloud and data center services, focusing on biomedical model training and genomic analysis.

By harnessing the power of NVIDIA accelerated computing, NHRI is also tackling pressing public health issues. One of its key projects involves using AI to predict the risk of chronic diseases such as diabetes and cardiovascular conditions by analyzing a multitude of genetic and environmental parameters.

“This level of analysis was previously unattainable due to computational constraints,” said Dr. Chen. “Now, with the power of NVIDIA accelerated computing, we will be able to offer more accurate risk assessments and preventative strategies.”

CGMH also has a diverse array of NVIDIA hardware, including NVIDIA H100, A100, and other Tensor Core GPUs, which it uses for medical imaging development and deployment. The foundation serves 46 models daily and intends to use Blackwell for LLM training and the deployment of service robots in hospitals.

Running these systems on premises and keeping the data within the hospital’s infrastructure are key to ensuring patient data privacy as well as faster data processing and reduced latency, said Dr. Chihung Lin, deputy director of the CGMH AI center.

These technologies may be used in various medical applications, including:

• Clinical Decision Support System: Developed on premises to ensure patient data confidentiality and privacy, this system assists clinicians by providing access to up-to-date data and guidelines and using models to answer questions and prepare medical decisions.
• Patient Interaction System: Allows patients to interact with a robot to get answers about their medication and medical conditions, reducing the burden on medical staff. Medical staff review the robot’s responses to ensure accuracy.
• Medical Imaging: Enhances radiology and other imaging tasks using AI. This project is one of the most mature AI technologies in CGMH’s healthcare system.
• Precision Medicine: Handles large-scale genomic data and transforms sequences into readable medical reports for doctors. Focused on building computational facilities to support whole genome and exome sequencing.
• Expansion of AI Services: Aims to extend the language model services to all hospitals under CGMH’s umbrella, leveraging the computational capacity from the Blackwell platform to support this expansion.

Other applications include early detection of colorectal cancer via endoscopy, autoimmune disease screening through microscope images and kidney disease prediction using general imaging techniques.

NHRI and CGMH’s adoption of accelerated computing underscores the growing importance of AI and advanced computing in medical research and healthcare delivery.

With these tools, Taiwan is poised to make strides in improving patient outcomes and advancing biomedical science.

Read More

Weather forecasters in Taiwan had their hair blown back when they saw a typhoon up close, created on a computer that slashed the time and energy needed for the job.

It’s a reaction that users in many fields are feeling as generative AI shows them how new levels of performance contribute to reductions in total cost of ownership.

Inside the AI of the Storm

Tracking a typhoon provided a great test case of generative AI’s prowess. The work traditionally begins with clusters of CPUs cranking on complex algorithms to create atmospheric models with a 25-kilometer resolution.

Enter CorrDiff, a generative AI model that’s part of NVIDIA Earth-2, a set of services and software for weather and climate research.

Using a class of diffusion models that power today’s text-to-image services, CorrDiff resolved the 25-km models to two kilometers 1,000x faster, using 3,000x less energy for a single inference than traditional methods.

CorrDiff Cuts Costs 50x, Energy Use 25x

CorrDiff shines on the NVIDIA AI platform, even when retraining the model once a year and using statistical groups of a thousand forecasts to boost the accuracy of predictions. Compared to traditional methods under these conditions, it slashes cost by 50x cost and energy use by 25x a year.

That means work that used to require nearly $3 million for a cluster of CPUs and the energy to run them can be done for about $60,000 on a single system with an NVIDIA H100 Tensor Core GPU. It’s a massive reduction that shows how generative AI and accelerated computing increases energy efficiency and lowers total cost of ownership.

The technology also helps forecasters see more precisely where a typhoon will land, potentially saving lives.

“NVIDIA’s CorrDiff generative AI model opens the door to the use of AI-generated kilometer-scale weather forecasts, enabling Taiwan to prepare better for typhoons,” said Hongey Chen, a director of Taiwan’s National Science and Technology Center for Disaster Reduction.

The Taiwan forecasters could save nearly a gigawatt-hour a year, using CorrDiff. Energy savings could balloon if the nearly 200 regional weather data centers around the world adopt the technology for more sustainable computing.

Companies that sell commercial forecasts are also adopting CorrDiff, attracted by its speed and savings.

Broad Horizons for Energy Efficiency

NVIDIA Earth-2 takes these capabilities to a planetary scale. It fuses AI, physics simulations and observed data to help countries and companies respond to global issues like climate change. That will help address the impacts of climate change, which is expected to cost a million lives and $1.7 trillion per year by 2050.

Accelerated computing and generative AI are bringing new levels of performance and energy efficiency to many applications. Explainers on green computing and why GPUs are great for AI provide more context and some examples.

Compare the costs and energy consumption of popular workloads running on an x86 CPU-based server versus an NVIDIA GPU server with this simple calculator. And watch Huang’s keynote address at COMPUTEX to get the big picture.

Read More