Monthly Archives: January 2025

Data is the fuel of AI applications, but the magnitude and scale of enterprise data often make it too expensive and time-consuming to use effectively.

According to IDC’s Global DataSphere¹, enterprises will generate 317 zettabytes of data annually by 2028 — including the creation of 29 zettabytes of unique data — of which 78% will be unstructured data and 44% of that will be audio and video. Because of the extremely high volume and various data types, most generative AI applications use a fraction of the total amount of data being stored and generated.

For enterprises to thrive in the AI era, they must find a way to make use of all of their data. This isn’t possible using traditional computing and data processing techniques. Instead, enterprises need an AI query engine.

What Is an AI Query Engine?

Simply, an AI query engine is a system that connects AI applications, or AI agents, to data. It’s a critical component of agentic AI, as it serves as a bridge between an organization’s knowledge base and AI-powered applications, enabling more accurate, context-aware responses.

AI agents form the basis of an AI query engine, where they can gather information and do work to assist human employees. An AI agent will gather information from many data sources, plan, reason and take action. AI agents can communicate with users, or they can work in the background, where human feedback and interaction will always be available.

In practice, an AI query engine is a sophisticated system that efficiently processes large amounts of data, extracts and stores knowledge, and performs semantic search on that knowledge, which can be quickly retrieved and used by AI.

AI Query Engines Unlock Intelligence in Unstructured Data

An enterprise’s AI query engine will have access to knowledge stored in many different formats, but being able to extract intelligence from unstructured data is one of the most significant advancements it enables.

To generate insights, traditional query engines rely on structured queries and data sources, such as relational databases. Users must formulate precise queries using languages like SQL, and results are limited to predefined data formats.

In contrast, AI query engines can process structured, semi-structured and unstructured data. Common unstructured data formats are PDFs, log files, images and video, and are stored on object stores, file servers and parallel file systems. AI agents communicate with users and with each other using natural language. This enables them to interpret user intent, even when it’s ambiguous, by accessing diverse data sources. These agents can deliver results in a conversational format, so that users can interpret results.

This capability makes it possible to derive more insights and intelligence from any type of data — not just data that fits neatly into rows and columns.

For example, companies like DataStax and NetApp are building AI data platforms that enable their customers to have an AI query engine for their next-generation applications.

Key Features of AI Query Engines

AI query engines possess several crucial capabilities:

• Diverse data handling: AI query engines can access and process various data types, including structured, semi-structured and unstructured data from multiple sources, including text, PDF, image, video and specialty data types.
• Scalability: AI query engines can efficiently handle petabyte-scale data, making all enterprise knowledge available to AI applications quickly.
• Accurate retrieval: AI query engines provide high-accuracy, high-performance embedding, vector search and reranking of knowledge from multiple sources.
• Continuous learning: AI query engines can store and incorporate feedback from AI-powered applications, creating an AI data flywheel in which the feedback is used to refine models and increase the effectiveness of the applications over time.

Retrieval-augmented generation is a component of AI query engines. RAG uses the power of generative AI models to act as a natural language interface to data, allowing models to access and incorporate relevant information from large datasets during the response generation process.

Using RAG, any business or other organization can turn its technical information, policy manuals, videos and other data into useful knowledge bases. An AI query engine can then rely on these sources to support such areas as customer relations, employee training and developer productivity.

Additional information-retrieval techniques and ways to store knowledge are in research and development, so the capabilities of an AI query engine are expected to rapidly evolve.

The Impact of AI Query Engines

Using AI query engines, enterprises can fully harness the power of AI agents to connect their workforces to vast amounts of enterprise knowledge, improve the accuracy and relevance of AI-generated responses, process and utilize previously untapped data sources, and create data-driven AI flywheels that continuously improve their AI applications.

Some examples include an AI virtual assistant that provides personalized, 24/7 customer service experiences, an AI agent for searching and summarizing video, an AI agent for analyzing software vulnerabilities or an AI research assistant.

Bridging the gap between raw data and AI-powered applications, AI query engines will grow to play a crucial role in helping organizations extract value from their data.

NVIDIA Blueprints can help enterprises get started connecting AI to their data. Learn more about NVIDIA Blueprints and try them in the NVIDIA API catalog.

1.  IDC, Global DataSphere Forecast, 2024.

Read More

NVIDIA founder and CEO Jensen Huang kicked off CES 2025 with a 90-minute keynote that included new products to advance gaming, autonomous vehicles, robotics, and agentic AI.

AI has been “advancing at an incredible pace,” he said before an audience of more than 6,000 packed into the Michelob Ultra Arena in Las Vegas.

“It started with perception AI — understanding images, words, and sounds. Then generative AI — creating text, images and sound,” Huang said. Now, we’re entering the era of “physical AI, AI that can proceed, reason, plan and act.”

NVIDIA GPUs and platforms are at the heart of this transformation, Huang explained, enabling breakthroughs across industries, including gaming, robotics and autonomous vehicles (AVs).

Huang’s keynote showcased how NVIDIA’s latest innovations are enabling this new era of AI, with several groundbreaking announcements, including:

• The just-announced NVIDIA Cosmos platform advances physical AI with new models and video data processing pipelines for robots, autonomous vehicles and vision AI.
• New NVIDIA Blackwell-based GeForce RTX 50 Series GPUs offer stunning visual realism and unprecedented performance boosts.
• AI foundation models introduced at CES for RTX PCs feature NVIDIA NIM microservices and AI Blueprints for crafting digital humans, podcasts, images and videos.
• The new NVIDIA Project DIGITS brings the power of NVIDIA Grace Blackwell to developer desktops in a compact package that can practically fit in a pocket.
• NVIDIA is partnering with Toyota for safe next-gen vehicle development using the NVIDIA DRIVE AGX in-vehicle computer running NVIDIA DriveOS.

Huang started off his talk by reflecting on NVIDIA’s three-decade journey. In 1999, NVIDIA invented the programmable GPU. Since then, modern AI has fundamentally changed how computing works, he said. “Every single layer of the technology stack has been transformed, an incredible transformation, in just 12 years.”

Revolutionizing Graphics With GeForce RTX 50 Series

“GeForce enabled AI to reach the masses, and now AI is coming home to GeForce,” Huang said.

With that, he introduced the NVIDIA GeForce RTX 5090 GPU, the most powerful GeForce RTX GPU so far, with 92 billion transistors and delivering 3,352 trillion AI operations per second (TOPS).

“Here it is — our brand-new GeForce RTX 50 series, Blackwell architecture,” Huang said, holding the blacked-out GPU aloft and noting how it’s able to harness advanced AI to enable breakthrough graphics. “The GPU is just a beast.”

“Even the mechanical design is a miracle,” Huang said, noting that the graphics card has two cooling fans.

More variations in the GPU series are coming. The GeForce RTX 5090 and GeForce RTX 5080 desktop GPUs are scheduled to be available Jan. 30. The GeForce RTX 5070 Ti and the GeForce RTX 5070 desktops are slated to be available starting in February. Laptop GPUs are expected in March.

DLSS 4 introduces Multi Frame Generation, working in unison with the complete suite of DLSS technologies to boost performance by up to 8x. NVIDIA also unveiled NVIDIA Reflex 2, which can reduce PC latency by up to 75%.

The latest generation of DLSS can generate three additional frames for every frame we calculate, Huang explained. “As a result, we’re able to render at incredibly high performance, because AI does a lot less computation.”

RTX Neural Shaders use small neural networks to improve textures, materials and lighting in real-time gameplay. RTX Neural Faces and RTX Hair advance real-time face and hair rendering, using generative AI to animate the most realistic digital characters ever. RTX Mega Geometry increases the number of ray-traced triangles by up to 100x, providing more detail.

Advancing Physical AI With Cosmos|

In addition to advancements in graphics, Huang introduced the NVIDIA Cosmos world foundation model platform, describing it as a game-changer for robotics and industrial AI.

The next frontier of AI is physical AI, Huang explained. He likened this moment to the transformative impact of large language models on generative AI.

“The ChatGPT moment for general robotics is just around the corner,” he explained.

Like large language models, world foundation models are fundamental to advancing robot and AV development, yet not all developers have the expertise and resources to train their own, Huang said.

Cosmos integrates generative models, tokenizers, and a video processing pipeline to power physical AI systems like AVs and robots.

Cosmos aims to bring the power of foresight and multiverse simulation to AI models, enabling them to simulate every possible future and select optimal actions.

Cosmos models ingest text, image or video prompts and generate virtual world states as videos, Huang explained. “Cosmos generations prioritize the unique requirements of AV and robotics use cases like real-world environments, lighting and object permanence.”

Leading robotics and automotive companies, including 1X, Agile Robots, Agility, Figure AI, Foretellix, Fourier, Galbot, Hillbot, IntBot, Neura Robotics, Skild AI, Virtual Incision, Waabi and XPENG, along with ridesharing giant Uber, are among the first to adopt Cosmos.

In addition, Hyundai Motor Group is adopting NVIDIA AI and Omniverse to create safer, smarter vehicles, supercharge manufacturing and deploy cutting-edge robotics.

Cosmos is open license and available on GitHub.

Empowering Developers With AI Foundation Models

Beyond robotics and autonomous vehicles, NVIDIA is empowering developers and creators with AI foundation models.

Huang introduced AI foundation models for RTX PCs that supercharge digital humans, content creation, productivity and development.

“These AI models run in every single cloud because NVIDIA GPUs are now available in every single cloud,” Huang said. “It’s available in every single OEM, so you could literally take these models, integrate them into your software packages, create AI agents and deploy them wherever the customers want to run the software.”

These models — offered as NVIDIA NIM microservices — are accelerated by the new GeForce RTX 50 Series GPUs.

The GPUs have what it takes to run these swiftly, adding support for FP4 computing, boosting AI inference by up to 2x and enabling generative AI models to run locally in a smaller memory footprint compared with previous-generation hardware.

Huang explained the potential of new tools for creators: “We’re creating a whole bunch of blueprints that our ecosystem could take advantage of. All of this is completely open source, so you could take it and modify the blueprints.”

Top PC manufacturers and system builders are launching NIM-ready RTX AI PCs with GeForce RTX 50 Series GPUs. “AI PCs are coming to a home near you,” Huang said.

While these tools bring AI capabilities to personal computing, NVIDIA is also advancing AI-driven solutions in the automotive industry, where safety and intelligence are paramount.

Innovations in Autonomous Vehicles

Huang announced the NVIDIA DRIVE Hyperion AV platform, built on the new NVIDIA AGX Thor system-on-a-chip (SoC), designed for generative AI models and delivering advanced functional safety and autonomous driving capabilities.

“The autonomous vehicle revolution is here,” Huang said. “Building autonomous vehicles, like all robots, requires three computers: NVIDIA DGX to train AI models, Omniverse to test drive and generate synthetic data, and DRIVE AGX, a supercomputer in the car.”

DRIVE Hyperion, the first end-to-end AV platform, integrates advanced SoCs, sensors, and safety systems for next-gen vehicles, a sensor suite and an active safety and level 2 driving stack, with adoption by automotive safety pioneers such as Mercedes-Benz, JLR and Volvo Cars.

Huang highlighted the critical role of synthetic data in advancing autonomous vehicles. Real-world data is limited, so synthetic data is essential for training the autonomous vehicle data factory, he explained.

Powered by NVIDIA Omniverse AI models and Cosmos, this approach “generates synthetic driving scenarios that enhance training data by orders of magnitude.”

Using Omniverse and Cosmos, NVIDIA’s AI data factory can scale “hundreds of drives into billions of effective miles,” Huang said, dramatically increasing the datasets needed for safe and advanced autonomous driving.

“We are going to have mountains of training data for autonomous vehicles,” he added.

Toyota, the world’s largest automaker, will build its next-generation vehicles on the NVIDIA DRIVE AGX Orin, running the safety-certified NVIDIA DriveOS operating system, Huang said.

“Just as computer graphics was revolutionized at such an incredible pace, you’re going to see the pace of AV development increasing tremendously over the next several years,” Huang said. These vehicles will offer functionally safe, advanced driving assistance capabilities.

Agentic AI and Digital Manufacturing

NVIDIA and its partners have launched AI Blueprints for agentic AI, including PDF-to-podcast for efficient research and video search and summarization for analyzing large quantities of video and images — enabling developers to build, test and run AI agents anywhere.

AI Blueprints empower developers to deploy custom agents for automating enterprise workflows This new category of partner blueprints integrates NVIDIA AI Enterprise software, including NVIDIA NIM microservices and NVIDIA NeMo, with platforms from leading providers like CrewAI, Daily, LangChain, LlamaIndex and Weights & Biases.

Additionally, Huang announced new Llama Nemotron.

Developers can use NVIDIA NIM microservices to build AI agents for tasks like customer support, fraud detection, and supply chain optimization.

Available as NVIDIA NIM microservices, the models can supercharge AI agents on any accelerated system.

NVIDIA NIM microservices streamline video content management, boosting efficiency and audience engagement in the media industry.

Moving beyond digital applications, NVIDIA’s innovations are paving the way for AI to revolutionize the physical world with robotics.

“All of the enabling technologies that I’ve been talking about are going to make it possible for us in the next several years to see very rapid breakthroughs, surprising breakthroughs, in general robotics.”

In manufacturing, the NVIDIA Isaac GR00T Blueprint for synthetic motion generation will help developers generate exponentially large synthetic motion data to train their humanoids using imitation learning.

Huang emphasized the importance of training robots efficiently, using NVIDIA’s Omniverse to generate millions of synthetic motions for humanoid training.

The Mega blueprint enables large-scale simulation of robot fleets, adopted by leaders like Accenture and KION for warehouse automation.

These AI tools set the stage for NVIDIA’s latest innovation: a personal AI supercomputer called Project DIGITS.

NVIDIA Unveils Project Digits

Putting NVIDIA Grace Blackwell on every desk and at every AI developer’s fingertips, Huang unveiled NVIDIA Project DIGITS.

“I have one more thing that I want to show you,” Huang said. “None of this would be possible if not for this incredible project that we started about a decade ago. Inside the company, it was called Project DIGITS — deep learning GPU intelligence training system.”

Huang highlighted the legacy of NVIDIA’s AI supercomputing journey, telling the story of how in 2016 he delivered the first NVIDIA DGX system to OpenAI. “And obviously, it revolutionized artificial intelligence computing.”

The new Project DIGITS takes this mission further. “Every software engineer, every engineer, every creative artist — everybody who uses computers today as a tool — will need an AI supercomputer,” Huang said.

Huang revealed that Project DIGITS, powered by the GB10 Grace Blackwell Superchip, represents NVIDIA’s smallest yet most powerful AI supercomputer. “This is NVIDIA’s latest AI supercomputer,” Huang said, showcasing the device. “It runs the entire NVIDIA AI stack — all of NVIDIA software runs on this. DGX Cloud runs on this.”

The compact yet powerful Project DIGITS is expected to be available in May.

A Year of Breakthroughs

“It’s been an incredible year,” Huang said as he wrapped up the keynote. Huang highlighted NVIDIA’s major achievements: Blackwell systems, physical AI foundation models, and breakthroughs in agentic AI and robotics

“I want to thank all of you for your partnership,” Huang said.

See notice regarding software product information.

Read More

According to Gartner, the worldwide end-user spending on all IT products for 2024 was $5 trillion. This industry is built on a computing fabric of electrons, is fully software-defined, accelerated — and now generative AI-enabled. While huge, it’s a fraction of the larger physical industrial market that relies on the movement of atoms.

Today’s 10 million factories, nearly 200,000 warehouses and 40 million miles of highways form the “computing” fabric of our physical world. But that vast network of production facilities and distribution centers is still laboriously and manually designed, operated and optimized.

In warehousing and distribution, operators face highly complex decision optimization problems — matrices of variables and interdependencies across human workers, robotic and agentic systems and equipment. Unlike the IT industry, the physical industrial market is still waiting for its own software-defined moment.

That moment is coming.

NVIDIA today at CES announced “Mega,” an Omniverse Blueprint for developing, testing and optimizing physical AI and robot fleets at scale in a digital twin before deployment into real-world facilities.

Advanced warehouses and factories use fleets of hundreds of autonomous mobile robots, robotic arm manipulators and humanoids working alongside people. With implementations of increasingly complex systems of sensor and robot autonomy, it requires coordinated training in simulation to optimize operations, help ensure safety and avoid disruptions.

Mega offers enterprises a reference architecture of NVIDIA accelerated computing, AI, NVIDIA Isaac and NVIDIA Omniverse technologies to develop and test digital twins for testing AI-powered robot brains that drive robots, video analytics AI agents, equipment and more for handling enormous complexity and scale. The new framework brings software-defined capabilities to physical facilities, enabling continuous development, testing, optimization and deployment.

Developing AI Brains With World Simulator for Autonomous Orchestration

With Mega-driven digital twins, including a world simulator that coordinates all robot activities and sensor data, enterprises can continuously update facility robot brains for intelligent routes and tasks for operational efficiencies.

The blueprint uses Omniverse Cloud Sensor RTX APIs that enable robotics developers to render sensor data from any type of intelligent machine in the factory, simultaneously, for high-fidelity large-scale sensor simulation. This allows robots to be tested in an infinite number of scenarios within the digital twin, using synthetic data in a software-in–the-loop pipeline with NVIDIA Isaac ROS.

Supply chain solutions company KION Group is collaborating with Accenture and NVIDIA as the first to adopt Mega for optimizing operations in retail, consumer packaged goods, parcel services and more.

Jensen Huang, founder and CEO of NVIDIA, offered a glimpse into the future of this collaboration on stage at CES, demonstrating how enterprises can navigate a complex web of decisions using the Mega Omniverse Blueprint.

“At KION, we leverage AI-driven solutions as an integral part of our strategy to optimize our customers’ supply chains and increase their productivity,” said Rob Smith, CEO of KION GROUP AG. “With NVIDIA’s AI leadership and Accenture’s expertise in digital technologies, we are reinventing warehouse automation. Bringing these strong partners together, we are creating a vision for future warehouses that are part of a smart agile system, evolve with the world around them and can handle nearly any supply chain challenge.”

Creating Operational Efficiencies With Mega Omniverse Blueprint

Creating operational efficiencies, KION and Accenture are embracing the Mega Omniverse Blueprint to build next-generation supply chains for KION and its customers. KION can capture and digitalize a warehouse digital twin in Omniverse by using computer-aided design files, video, lidar, image and AI-generated data.

KION uses the Omniverse digital twin as a virtual training and testing environment for its industrial AI’s robot brains, powered by NVIDIA Isaac, tapping into smart cameras, forklifts, robotic equipment and digital humans. Integrating the Omniverse digital twin, KION’s warehouse management software can create and assign missions for robot brains, like moving a load from one place to another.

These simulated robots can carry out tasks by perceiving and reasoning in environments, and they’re capable of planning next motions and then taking actions that are simulated in the digital twin. The robot brains perceive the results deciding the next action, and this cycle continues with Mega precisely tracking the state and position of all the assets in the digital twin.

Delivering Services With Mega for Facilities Everywhere

Accenture, global leader in professional services, is adopting Mega as part of its AI Refinery for Simulation and Robotics, built on NVIDIA AI and Omniverse, to help organizations use AI simulation to reinvent factory and warehouse design and ongoing operations.

With the blueprint, Accenture is delivering new services — including Custom Robotics and Manufacturing Foundation Model Training and Finetuning; Intelligent Humanoid Robotics; and AI-Powered Industrial Manufacturing and Logistics Simulation and Optimization — to expand the power of physical AI and  simulation to the world’s factories and warehouse operators.  Now, for example, an organization can explore numerous options for their warehouse before choosing and implementing the best one.

“As organizations enter the age of industrial AI, we are helping them use AI-powered simulation and autonomous robots to reinvent the process of designing new facilities and optimizing existing operations,” said Julie Sweet, chair and CEO of Accenture. “Our collaboration with NVIDIA and KION will help our clients plan their operations in digital twins, where they can run hundreds of options and quickly select the best for current or changing market conditions, such as seasonal market demand or workforce availability.  This represents a new frontier of value for our clients to achieve using technology, data and AI.”

Join NVIDIA at CES. 

See notice regarding software product information.

Read More

Generative AI and foundation models let autonomous machines generalize beyond the operational design domains on which they’ve been trained. Using new AI techniques such as tokenization and large language and diffusion models, developers and researchers can now address longstanding hurdles to autonomy.

These larger models require massive amounts of diverse data for training, fine-tuning and validation. But collecting such data — including from rare edge cases and potentially hazardous scenarios, like a pedestrian crossing in front of an autonomous vehicle (AV) at night or a human entering a welding robot work cell — can be incredibly difficult and resource-intensive.

To help developers fill this gap, NVIDIA Omniverse Cloud Sensor RTX APIs enable physically accurate sensor simulation for generating datasets at scale. The application programming interfaces (APIs) are designed to support sensors commonly used for autonomy — including cameras, radar and lidar — and can integrate seamlessly into existing workflows to accelerate the development of autonomous vehicles and robots of every kind.

Omniverse Sensor RTX APIs are now available to select developers in early access. Organizations such as Accenture, Foretellix, MITRE and Mcity are integrating these APIs via domain-specific blueprints to provide end customers with the tools they need to deploy the next generation of industrial manufacturing robots and self-driving cars.

Powering Industrial AI With Omniverse Blueprints

In complex environments like factories and warehouses, robots must be orchestrated to safely and efficiently work alongside machinery and human workers. All those moving parts present a massive challenge when designing, testing or validating operations while avoiding disruptions.

Mega is an Omniverse Blueprint that offers enterprises a reference architecture of NVIDIA accelerated computing, AI, NVIDIA Isaac and NVIDIA Omniverse technologies. Enterprises can use it to develop digital twins and test AI-powered robot brains that drive robots, cameras, equipment and more to handle enormous complexity and scale.

Integrating Omniverse Sensor RTX, the blueprint lets robotics developers simultaneously render sensor data from any type of intelligent machine in a factory for high-fidelity, large-scale sensor simulation.

With the ability to test operations and workflows in simulation, manufacturers can save considerable time and investment, and improve efficiency in entirely new ways.

International supply chain solutions company KION Group and Accenture are using the Mega blueprint to build Omniverse digital twins that serve as virtual training and testing environments for industrial AI’s robot brains, tapping into data from smart cameras, forklifts, robotic equipment and digital humans.

The robot brains perceive the simulated environment with physically accurate sensor data rendered by the Omniverse Sensor RTX APIs. They use this data to plan and act, with each action precisely tracked with Mega, alongside the state and position of all the assets in the digital twin. With these capabilities, developers can continuously build and test new layouts before they’re implemented in the physical world.

Driving AV Development and Validation

Autonomous vehicles have been under development for over a decade, but barriers in acquiring the right training and validation data and slow iteration cycles have hindered large-scale deployment.

To address this need for sensor data, companies are harnessing the NVIDIA Omniverse Blueprint for AV simulation, a reference workflow that enables physically accurate sensor simulation. The workflow uses Omniverse Sensor RTX APIs to render the camera, radar and lidar data necessary for AV development and validation.

AV toolchain provider Foretellix has integrated the blueprint into its Foretify AV development toolchain to transform object-level simulation into physically accurate sensor simulation.

The Foretify toolchain can generate any number of testing scenarios simultaneously. By adding sensor simulation capabilities to these scenarios, Foretify can now enable  developers to evaluate the completeness of their AV development, as well as train and test at the levels of fidelity and scale needed to achieve large-scale and safe deployment. In addition, Foretellix will use the newly announced NVIDIA Cosmos platform to generate an even greater diversity of scenarios for verification and validation.

Nuro, an autonomous driving technology provider with one of the largest level 4 deployments in the U.S., is using the Foretify toolchain to train, test and validate its self-driving vehicles before deployment.

In addition, research organization MITRE is collaborating with the University of Michigan’s Mcity testing facility to build a digital AV validation framework for regulatory use, including a digital twin of Mcity’s 32-acre proving ground for autonomous vehicles. The project uses the AV simulation blueprint to render physically accurate sensor data at scale in the virtual environment, boosting training effectiveness.

The future of robotics and autonomy is coming into sharp focus, thanks to the power of high-fidelity sensor simulation. Learn more about these solutions at CES by visiting Accenture at Ballroom F at the Venetian and Foretellix booth 4016 in the West Hall of Las Vegas Convention Center.

Learn more about the latest in automotive and generative AI technologies by joining NVIDIA at CES.

See notice regarding software product information.

Read More

The next big moment in AI is in sight — literally.

Today, more than 1.5 billion enterprise level cameras deployed worldwide are generating roughly 7 trillion hours of video per year. Yet, only a fraction of it gets analyzed.

It’s estimated that less than 1% of video from industrial cameras is watched live by humans, meaning critical operational incidents can go largely unnoticed.

This comes at a high cost. For example, manufacturers are losing trillions of dollars annually to poor product quality or defects that they could’ve spotted earlier, or even predicted, by using AI agents that can perceive, analyze and help humans take action.

Interactive AI agents with built-in visual perception capabilities can serve as always-on video analysts, helping factories run more efficiently, bolster worker safety, keep traffic running smoothly and even up an athlete’s game.

To accelerate the creation of such agents, NVIDIA today announced early access to a new version of the NVIDIA AI Blueprint for video search and summarization. Built on top of the NVIDIA Metropolis platform — and now supercharged by NVIDIA Cosmos Nemotron vision language models (VLMs), NVIDIA Llama Nemotron large language models (LLMs) and NVIDIA NeMo Retriever — the blueprint provides developers with the tools to build and deploy AI agents that can analyze large quantities of video and image content.

The blueprint integrates the NVIDIA AI Enterprise software platform — which includes NVIDIA NIM microservices for VLMs, LLMs and advanced AI frameworks for retrieval-augmented generation — to enable batch video processing that’s 30x faster than watching it in real time.

The blueprint contains several agentic AI features — such as chain-of-thought reasoning, task planning and tool calling — that can help developers streamline the creation of powerful and diverse visual agents to solve a range of problems.

AI agents with video analysis abilities can be combined with other agents with different skill sets to enable even more sophisticated agentic AI services. Enterprises have the flexibility to build and deploy their AI agents from the edge to the cloud.

How Video Analyst AI Agents Can Help Industrial Businesses 

AI agents with visual perception and analysis skills can be fine-tuned to help businesses with industrial operations by:

• Increasing productivity and reducing waste: Agents can help ensure standard operating procedures are followed during complex industrial processes like product assembly. They can also be fine-tuned to carefully watch and understand nuanced actions, and the sequence in which they’re implemented.

• Boosting asset management efficiency through better space utilization: Agents can help optimize inventory storage in warehouses by performing 3D volume estimation and centralizing understanding across various camera streams.

• Improving safety through auto-generation of incident reports and summaries: Agents can process huge volumes of video and summarize it into contextually informative reports of accidents. They can also help ensure personal protective equipment compliance in factories, improving worker safety in industrial settings.

• Preventing accidents and production problems: AI agents can identify atypical activity to quickly mitigate operational and safety risks, whether in a warehouse, factory or airport, or at a traffic intersection or other municipal setting.

• Learning from the past: Agents can search through operations video archives, find relevant information from the past and use it to solve problems or create new processes.

Video Analysts for Sports, Entertainment and More

Another industry where video analysis AI agents stand to make a mark is sports — a $500 billion market worldwide, with hundreds of billions in projected growth over the next several years.

Coaches, teams and leagues — whether professional or amateur — rely on video analytics to evaluate and enhance player performance, prioritize safety and boost fan engagement through player analytics platforms and data visualization. With visually perceptive AI agents, athletes now have unprecedented access to deeper insights and opportunities for improvement.

During his CES opening keynote, NVIDIA founder and CEO Jensen Huang demonstrated an AI video analytics agent that assessed the fastball pitching skills of an amateur baseball player compared with a professional’s. Using video captured from the ceremonial first pitch that Huang threw for the San Francisco Giants baseball team, the video analytics AI agent was able to suggest areas for improvement.

The $3 trillion media and entertainment industry is also poised to benefit from video analyst AI agents. Through the NVIDIA Media2 initiative, these agents will help drive the creation of smarter, more tailored and more impactful content that can adapt to individual viewer preferences.

Worldwide Adoption and Availability 

Partners from around the world are integrating the blueprint for building AI agents for video analysis into their own developer workflows, including Accenture, Centific, Deloitte, EY, Infosys, Linker Vision, Pegatron, TATA Consultancy Services (TCS), Telit Cinterion and VAST.

Apply for early access to the NVIDIA Blueprint for video search and summarization.

See notice regarding software product information.

Editor’s note: Omdia is the source for 1.5 billion enterprise-level cameras deployed.   

Read More

GeForce RTX 50 Series Desktop and Laptop GPUs, unveiled today at the CES trade show, are poised to power the next era of generative and agentic AI content creation — offering new tools and capabilities for video, livestreaming, 3D and more.

Built on the NVIDIA Blackwell architecture, GeForce RTX 50 Series GPUs can run creative generative AI models up to 2x faster in a smaller memory footprint, compared with the previous generation. They feature ninth-generation NVIDIA encoders for advanced video editing and livestreaming, and come with NVIDIA DLSS 4 and up to 32GB of VRAM  to tackle massive 3D projects.

These GPUs come with various software updates, including two new AI-powered NVIDIA Broadcast effects, updates to RTX Video and RTX Remix, and NVIDIA NIM microservices — prepackaged and optimized models built to jumpstart AI content creation workflows on RTX AI PCs.

Built for the Generative AI Era

Generative AI can create sensational results for creators, but with models growing in both complexity and scale, generative AI can be difficult to run even on the latest hardware.

The GeForce RTX 50 Series adds FP4 support to help address this issue. FP4 is a lower quantization method, similar to file compression, that decreases model sizes. Compared with FP16 — the default method that most models feature — FP4 uses less than half of the memory and 50 Series GPUs provide over 2x performance compared to the previous generation. This can be done with virtually no loss in quality with advanced quantization methods offered by NVIDIA TensorRT Model Optimizer.

For example, Black Forest Labs’ FLUX.1 [dev] model at FP16 requires over 23GB of VRAM, meaning it can only be supported by the GeForce RTX 4090 and professional GPUs. With FP4, FLUX.1 [dev] requires less than 10GB, so it can run locally on more GeForce RTX GPUs.

With a GeForce RTX 4090 with FP16, the FLUX.1 [dev] model can generate images in 15 seconds with 30 steps. With a GeForce RTX 5090 with FP4, images can be generated in just over five seconds.

A new NVIDIA AI Blueprint for 3D-guided generative AI based on FLUX.1 [dev], which will be offered as an NVIDIA NIM microservice, offers artists greater control over text-based image generation. With this blueprint, creators can use simple 3D objects — created by hand or generated with AI — and lay them out in a 3D renderer like Blender to guide AI image generation.

A prepackaged workflow powered by the FLUX NIM microservice and ComfyUI can then generate high-quality images that match the 3D scene’s composition.

The NVIDIA Blueprint for 3D-guided generative AI is expected to be available through GitHub using a one-click installer in February.

Stability AI announced that its Stable Point Aware 3D, or SPAR3D, model will be available this month on RTX AI PCs. Thanks to RTX acceleration, the new model from Stability AI will help transform 3D design, delivering exceptional control over 3D content creation by enabling real-time editing and the ability to generate an object in less than a second from a single image.

Professional-Grade Video for All

GeForce RTX 50 Series GPUs deliver a generational leap in NVIDIA encoders and decoders with support for the 4:2:2 pro-grade color format, multiview-HEVC (MV-HEVC) for 3D and virtual reality (VR) video, and the new AV1 Ultra High Quality mode.

Most consumer cameras are confined to 4:2:0 color compression, which reduces the amount of color information. 4:2:0 is typically sufficient for video playback on browsers, but it can’t provide the color depth needed for advanced video editors to color grade videos. The 4:2:2 format provides double the color information with just a 1.3x increase in RAW file size — offering an ideal balance for video editing workflows.

Decoding 4:2:2 video can be challenging due to the increased file sizes. GeForce RTX 50 Series GPUs include 4:2:2 hardware support that can decode up to eight times the 4K 60 frames per second (fps) video sources per decoder, enabling smooth multi-camera video editing.

The GeForce RTX 5090 GPU is equipped with three encoders and two decoders, the GeForce RTX 5080 GPU includes two encoders and two decoders, the 5070 Ti GPUs has two encoders with a single decoder, and the GeForce RTX 5070 GPU includes a single encoder and decoder. These multi-encoder and decoder setups, paired with faster GPUs, enable the GeForce RTX 5090 to export video 60% faster than the GeForce RTX 4090 and at 4x speed compared with the GeForce RTX 3090.

GeForce RTX 50 Series GPUs also feature the ninth-generation NVIDIA video encoder, NVENC, that offers a 5% improvement in video quality on HEVC and AV1 encoding (BD-BR), as well as a new AV1 Ultra Quality mode that achieves 5% more compression at the same quality. They also include the sixth-generation NVIDIA decoder, with 2x the decode speed for H.264 video.

NVIDIA is collaborating with Adobe Premiere Pro, Blackmagic Design’s DaVinci Resolve, Capcut and Wondershare Filmora to integrate these technologies, starting in February.

3D video is starting to catch on thanks to the growth of VR, AR and mixed reality headsets. The new RTX 50 Series GPUs also come with support for MV-HEVC codecs to unlock such formats in the near future.

Livestreaming Enhanced

Livestreaming is a juggling act, where the streamer has to entertain the audience, produce a show and play a video game — all at the same time. Top streamers can afford to hire producers and moderators to share the workload, but most have to manage these responsibilities on their own and often in long shifts — until now.

Streamlabs, a Logitech brand and leading provider of broadcasting software and tools for content creators, is collaborating with NVIDIA and Inworld AI to create the Streamlabs Intelligent Streaming Assistant.

Streamlabs Intelligent Streaming Assistant is an AI agent that can act as a sidekick, producer and technical support. The sidekick that can join streams as a 3D avatar to answer questions, comment on gameplay or chats, or help initiate conversations during quiet periods. It can help produce streams, switching to the most relevant scenes and playing audio and video cues during interesting gameplay moments. It can even serve as an IT assistant that helps configure streams and troubleshoot issues.

Streamlabs Intelligent Streaming Assistant is powered by NVIDIA ACE technologies for creating digital humans and Inworld AI, an AI framework for agentic AI experiences. The assistant will be available later this year.

Millions have used the NVIDIA Broadcast app to turn offices and dorm rooms into home studios using AI-powered features that improve audio and video quality — without needing expensive, specialized equipment.

Two new AI-powered beta effects are being added to the NVIDIA Broadcast app.

The first, Studio Voice, enhances the sound of a user’s microphone to match that of a high-quality microphone. The other, Virtual Key Light, can relight a subject’s face to deliver even coverage as if it were well-lit by two lights.

Because they harness demanding AI models, these beta features are recommended for video conferencing or non-gaming livestreams using a GeForce RTX 5080 GPU or higher. NVIDIA is working to expand these features to more GeForce RTX GPUs in future updates.

The NVIDIA Broadcast upgrade also includes an updated user interface that allows users to apply more effects simultaneously, as well as improvements to the background noise removal, virtual background and eye contact effects.

The updated NVIDIA Broadcast app will be available in February.

Livestreamers can also benefit from NVENC — 5% BD-BR video quality improvement for HEVC and AV1 — in the latest beta of Twitch’s Enhanced Broadcast feature in OBS, and the improved AV1 encoder for streaming in Discord or YouTube.

RTX Video — an AI feature that enhances video playback on popular internet browsers like Google Chrome and Microsoft Edge, and locally with Video Super Resolution and HDR — is getting an update to decrease GPU usage by 30%, expanding the lineup of GeForce RTX GPUs that can run Video Super Resolution with higher quality.

The RTX Video update is slated for a future NVIDIA App release.

Unprecedented 3D Render Performance

The GeForce RTX 5090 GPU offers 32GB of GPU memory — the largest of any GeForce RTX GPU ever, marking a 33% increase over the GeForce RTX 4090 GPU. This lets 3D artists build larger, richer worlds while using multiple applications simultaneously. Plus, new RTX 50 Series fourth-generation RT Cores can run 3D applications 40% faster.

DLSS 4 debuts Multi Frame Generation to boost frame rates by using AI to generate up to three frames per rendered frame. This enables animators to smoothly navigate a scene with 4x as many frames, or render 3D content at 60 fps or more.

D5 Render and Chaos Vantage, two popular professional-grade 3D apps for architects and designers, will add support for DLSS 4 in February.

3D artists have adopted generative AI to boost productivity in generating draft 3D meshes, HDRi maps or even animations to prototype a scene. At CES, Stability AI announced SPAR3D, its new 3D model that can generate 3D meshes from images in seconds with RTX acceleration.

NVIDIA RTX Remix — a modding platform that lets modders capture game assets, automatically enhance materials with generative AI tools and create stunning RTX remasters with full ray tracing — supports DLSS 4, increasing graphical fidelity and frame rates to maximize realism and immersion during gameplay.

RTX Remix soon plans to support Neural Radiance Cache, a neural shader that uses AI to train on live game data and estimate per-pixel accurate indirect lighting. RTX Remix creators can also expect access to RTX Skin in their mods, the first ray-traced sub-surface scattering implementation in games. With RTX Skin, RTX Remix mods expect to feature characters with new levels of realism, as light will reflect and propagate through their skin, grounding them in the worlds they inhabit.

GeForce RTX 5090 and 5080 GPUs will be available for purchase starting Jan. 30 — followed by GeForce RTX 5070 Ti and 5070 GPUs in February and RTX 50 Series laptops in March.

All systems equipped with GeForce RTX GPUs include the NVIDIA Studio platform optimizations, with over 130 GPU-accelerated content creation apps, as well as NVIDIA Studio Drivers, tested extensively and released monthly to enhance performance and maximize stability in popular creative applications.

Stay tuned for more updates on the GeForce RTX 50 Series. Learn more about how the GeForce RTX 50 Series supercharges gaming, and check out all of NVIDIA’s announcements at CES. 

Every month brings new creative app updates and optimizations powered by the NVIDIA Studio 

Follow NVIDIA Studio on Instagram, X and Facebook. Access tutorials on the Studio YouTube channel and get updates directly in your inbox by subscribing to the Studio newsletter. 

See notice regarding software product information.

Read More

Artificial intelligence is entering a new era — agentic AI — where teams of specialized agents can help people solve complex problems and automate repetitive tasks.

With custom AI agents, enterprises across industries can manufacture intelligence and achieve unprecedented productivity. These advanced AI agents require a system of multiple generative AI models optimized for agentic AI functions and capabilities. This complexity means that the need for powerful, efficient, enterprise-grade models has never been greater.

To provide a foundation for enterprise agentic AI, NVIDIA today announced the Llama Nemotron family of open large language models (LLMs). Built with Llama, the models can help developers create and deploy AI agents across a range of applications — including customer support, fraud detection, and product supply chain and inventory management optimization.

To be effective, many AI agents need both language skills and the ability to perceive the world and respond with the appropriate action.

With new NVIDIA Cosmos Nemotron vision language models (VLMs) and NVIDIA NIM microservices for video search and summarization, developers can build agents that analyze and respond to images and video from autonomous machines, hospitals, stores and warehouses, as well as sports events, movies and news. For developers seeking to generate physics-aware videos for robotics and autonomous vehicles, NVIDIA today separately announced NVIDIA Cosmos world foundation models.

Open Llama Nemotron Models Optimize Compute Efficiency, Accuracy for AI Agents

Built with Llama foundation models — one of the most popular commercially viable open-source model collections, downloaded over 650 million times — NVIDIA Llama Nemotron models provide optimized building blocks for AI agent development. This builds on NVIDIA’s commitment to developing state-of-the-art models, such as Llama 3.1 Nemotron 70B, now available through the NVIDIA API catalog.

Llama Nemotron models are pruned and trained with NVIDIA’s latest techniques and high-quality datasets for enhanced agentic capabilities. They excel at instruction following, chat, function calling, coding and math, while being size-optimized to run on a broad range of NVIDIA accelerated computing resources.

“Agentic AI is the next frontier of AI development, and delivering on this opportunity requires full-stack optimization across a system of LLMs to deliver efficient, accurate AI agents,” said Ahmad Al-Dahle, vice president and head of GenAI at Meta. “Through our collaboration with NVIDIA and our shared commitment to open models, the NVIDIA Llama Nemotron family built on Llama can help enterprises quickly create their own custom AI agents.”

Leading AI agent platform providers including SAP and ServiceNow are expected to be among the first to use the new Llama Nemotron models.

“AI agents that collaborate to solve complex tasks across multiple lines of the business will unlock a whole new level of enterprise productivity beyond today’s generative AI scenarios,” said Philipp Herzig, chief AI officer at SAP. “Through SAP’s Joule, hundreds of millions of enterprise users will interact with these agents to accomplish their goals faster than ever before. NVIDIA’s new open Llama Nemotron model family will foster the development of multiple specialized AI agents to transform business processes.”

“AI agents make it possible for organizations to achieve more with less effort, setting new standards for business transformation,” said Jeremy Barnes, vice president of platform AI at ServiceNow. “The improved performance and accuracy of NVIDIA’s open Llama Nemotron models can help build advanced AI agent services that solve complex problems across functions, in any industry.”

The NVIDIA Llama Nemotron models use NVIDIA NeMo for distilling, pruning and alignment. Using these techniques, the models are small enough to run on a variety of computing platforms while providing high accuracy as well as increased model throughput.

The Llama Nemotron model family will be available as downloadable models and as NVIDIA NIM microservices that can be easily deployed on clouds, data centers, PCs and workstations. They offer enterprises industry-leading performance with reliable, secure and seamless integration into their agentic AI application workflows.

Customize and Connect to Business Knowledge With NVIDIA NeMo

The Llama Nemotron and Cosmos Nemotron model families are coming in Nano, Super and Ultra sizes to provide options for deploying AI agents at every scale.

• Nano: The most cost-effective model optimized for real-time applications with low latency, ideal for deployment on PCs and edge devices.
• Super: A high-accuracy model offering exceptional throughput on a single GPU.
• Ultra: The highest-accuracy model, designed for data-center-scale applications demanding the highest performance.

Enterprises can also customize the models for their specific use cases and domains with NVIDIA NeMo microservices to simplify data curation, accelerate model customization and evaluation, and apply guardrails to keep responses on track.

With NVIDIA NeMo Retriever, developers can also integrate retrieval-augmented generation capabilities to connect models to their enterprise data.

And using NVIDIA Blueprints for agentic AI, enterprises can quickly create their own applications using NVIDIA’s advanced AI tools and end-to-end development expertise. In fact, NVIDIA Cosmos Nemotron, NVIDIA Llama Nemotron and NeMo Retriever supercharge the new NVIDIA Blueprint for video search and summarization, announced separately today.

NeMo, NeMo Retriever and NVIDIA Blueprints are all available with the NVIDIA AI Enterprise software platform.

Availability

Llama Nemotron and Cosmos Nemotron models will be available soon as hosted application programming interfaces and for download on build.nvidia.com and Hugging Face. Access for development, testing and research is free for members of the NVIDIA Developer Program.

Enterprises can run Llama Nemotron and Cosmos Nemotron NIM microservices in production with the NVIDIA AI Enterprise software platform on accelerated data center and cloud infrastructure.

Sign up to get notified about Llama Nemotron and Cosmos Nemotron models, and join NVIDIA at CES.

See notice regarding software product information.

Read More

Autonomous vehicle (AV) development is made possible by three distinct computers: NVIDIA DGX systems for training the AI-based stack in the data center, NVIDIA Omniverse running on NVIDIA OVX systems for simulation and synthetic data generation, and the NVIDIA AGX in-vehicle computer to process real-time sensor data for safety.

Together, these purpose-built, full-stack systems enable continuous development cycles, speeding improvements in performance and safety.

At the CES trade show, NVIDIA today announced a new part of the equation: NVIDIA Cosmos, a platform comprising state-of-the-art generative world foundation models (WFMs), advanced tokenizers, guardrails and an accelerated video processing pipeline built to advance the development of physical AI systems such as AVs and robots.

With Cosmos added to the three-computer solution, developers gain a data flywheel that can turn thousands of human-driven miles into billions of virtually driven miles — amplifying training data quality.

“The AV data factory flywheel consists of fleet data collection, accurate 4D reconstruction and AI to generate scenes and traffic variations for training and closed-loop evaluation,” said Sanja Fidler, vice president of AI research at NVIDIA. “Using the NVIDIA Omniverse platform, as well as Cosmos and supporting AI models, developers can generate synthetic driving scenarios to amplify training data by orders of magnitude.”

“Developing physical AI models has traditionally been resource-intensive and costly for developers, requiring acquisition of real-world datasets and filtering, curating and preparing data for training,” said Norm Marks, vice president of automotive at NVIDIA. “Cosmos accelerates this process with generative AI, enabling smarter, faster and more precise AI model development for autonomous vehicles and robotics.”

Transportation leaders are using Cosmos to build physical AI for AVs, including:

• Waabi, a company pioneering generative AI for the physical world, will use Cosmos for the search and curation of video data for AV software development and simulation.
• Wayve, which is developing AI foundation models for autonomous driving, is evaluating Cosmos as a tool to search for edge and corner case driving scenarios used for safety and validation.
• AV toolchain provider Foretellix will use Cosmos, alongside NVIDIA Omniverse Sensor RTX APIs, to evaluate and generate high-fidelity testing scenarios and training data at scale.
• In addition, ridesharing giant Uber is partnering with NVIDIA to accelerate autonomous mobility. Rich driving datasets from Uber, combined with the features of the Cosmos platform and NVIDIA DGX Cloud, will help AV partners build stronger AI models even more efficiently.

Availability

Cosmos WFMs are now available under an open model license on Hugging Face and the NVIDIA NGC catalog. Cosmos models will soon be available as fully optimized NVIDIA NIM microservices.

Get started with Cosmos and join NVIDIA at CES.

See notice regarding software product information.

Read More

New NVIDIA AI Blueprints for building agentic AI applications are poised to help enterprises everywhere automate work.

With the blueprints, developers can now build and deploy custom AI agents. These AI agents act like “knowledge robots” that can reason, plan and take action to quickly analyze large quantities of data, summarize and distill real-time insights from video, PDF and other images.

CrewAI, Daily, LangChain, LlamaIndex and Weights & Biases are among leading providers of agentic AI orchestration and management tools that have worked with NVIDIA to build blueprints that integrate the NVIDIA AI Enterprise software platform, including NVIDIA NIM microservices and NVIDIA NeMo, with their platforms. These five blueprints — comprising a new category of partner blueprints for agentic AI — provide the building blocks for developers to create the next wave of AI applications that will transform every industry.

In addition to the partner blueprints, NVIDIA is introducing its own new AI Blueprint for PDF to podcast, as well as another to build AI agents for video search and summarization. These are joined by four additional NVIDIA Omniverse Blueprints that make it easier for developers to build simulation-ready digital twins for physical AI.

To help enterprises rapidly take AI agents into production, Accenture is announcing AI Refinery for Industry built with NVIDIA AI Enterprise, including NVIDIA NeMo, NVIDIA NIM microservices and AI Blueprints.

The AI Refinery for Industry solutions — powered by Accenture AI Refinery with NVIDIA — can help enterprises rapidly launch agentic AI across fields like automotive, technology, manufacturing, consumer goods and more.

Agentic AI Orchestration Tools Conduct a Symphony of Agents

Agentic AI represents the next wave in the evolution of generative AI. It enables applications to move beyond simple chatbot interactions to tackle complex, multi-step problems through sophisticated reasoning and planning. As explained in NVIDIA founder and CEO Jensen Huang’s CES keynote, enterprise AI agents will become a centerpiece of AI factories that generate tokens to create unprecedented intelligence and productivity across industries.

Agentic AI orchestration is a sophisticated system designed to manage, monitor and coordinate multiple AI agents working together — key to developing reliable enterprise agentic AI systems. The agentic AI orchestration layer from NVIDIA partners provides the glue needed for AI agents to effectively work together.

The new partner blueprints, now available from agentic AI orchestration leaders, offer integrations with NVIDIA AI Enterprise software, including NIM microservices and NVIDIA NeMo Retriever, to boost retrieval accuracy and reduce latency of agent workflows. For example:

• CrewAI is using new Llama 3.3 70B NVIDIA NIM microservices and the NVIDIA NeMo Retriever embedding NIM microservice for its blueprint for code documentation for software development. The blueprint helps ensure code repositories remain comprehensive and easy to navigate.
• Daily’s voice agent blueprint, powered by the company’s open-source Pipecat framework, uses the NVIDIA Riva automatic speech recognition and text-to-speech NIM microservice, along with the Llama 3.3 70B NIM microservice to achieve real-time conversational AI.
• LangChain is adding Llama 3.3 70B NVIDIA NIM microservices to its structured report generation blueprint. Built on LangGraph, the blueprint allows users to define a topic and specify an outline to guide an agent in searching the web for relevant information, so it can return a report in the requested format.
• LlamaIndex’s document research assistant for blog creation blueprint harnesses NVIDIA NIM microservices and NeMo Retriever to help content creators produce high-quality blogs. It can tap into agentic-driven retrieval-augmented generation with NeMo Retriever to automatically research, outline and generate compelling content with source attribution.
• Weights & Biases is adding its W&B Weave capability to the AI Blueprint for AI virtual assistants, which features the Llama 3.1 70B NIM microservice. The blueprint can streamline the process of debugging, evaluating, iterating and tracking production performance and collecting human feedback to support seamless integration and faster iterations for building and deploying agentic AI applications.

Summarize Many, Complex PDFs While Keeping Proprietary Data Secure 

With trillions of PDF files — from financial reports to technical research papers — generated every year, it’s a constant challenge to stay up to date with information.

NVIDIA’s PDF to podcast AI Blueprint provides a recipe developers can use to turn multiple long and complex PDFs into AI-generated readouts that can help professionals, students and researchers efficiently learn about virtually any topic and quickly understand key takeaways.

The blueprint — built on NIM microservices and text-to-speech models — allows developers to build applications that extract images, tables and text from PDFs, and convert the data into easily digestible audio content, all while keeping data secure.

For example, developers can build AI agents that can understand context, identify key points and generate a concise summary as a monologue or a conversation-style podcast, narrated in a natural voice. This offers users an engaging, time-efficient way to absorb information at their desired speed.

Test, Prototype and Run Agentic AI Blueprints in One Click

NVIDIA Blueprints empower the world’s more than 25 million software developers to easily integrate AI into their applications across various industries. These blueprints simplify the process of building and deploying agentic AI applications, making advanced AI integration more accessible than ever.

With just a single click, developers can now build and run the new agentic AI Blueprints as NVIDIA Launchables. These Launchables provide on-demand access to developer environments with predefined configurations, enabling quick workflow setup.

By containing all necessary components for development, Launchables support consistent and reproducible setups without the need for manual configuration or overhead — streamlining the entire development process, from prototyping to deployment.

Enterprises can also deploy blueprints into production with the NVIDIA AI Enterprise software platform on data center platforms including Dell Technologies, Hewlett Packard Enterprise, Lenovo and Supermicro, or run them on accelerated cloud platforms from Amazon Web Services, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure.

Accenture and NVIDIA Fast-Track Deployments With AI Refinery for Industry

Accenture is introducing its new AI Refinery for Industry with 12 new industry agent solutions built with NVIDIA AI Enterprise software and available from the Accenture NVIDIA Business Group. These industry-specific agent solutions include revenue growth management for consumer goods and services, clinical trial companion for life sciences, industrial asset troubleshooting and B2B marketing, among others.

AI Refinery for Industry offerings include preconfigured components, best practices and foundational elements designed to fast-track the development of AI agents. They provide organizations the tools to build specialized AI networks tailored to their industry needs.

Accenture plans to launch over 100 AI Refinery for Industry agent solutions by the end of the year.

Get started with AI Blueprints and join NVIDIA at CES.

See notice regarding software product information.

Read More

From creating the GPU, RTX real-time ray tracing and neural rendering to now reinventing computing for AI, NVIDIA has for decades been at the forefront of computer graphics — pushing the boundaries of what’s possible in media and entertainment.

NVIDIA Media2 is the latest AI-powered initiative transforming content creation, streaming and live media experiences.

Built on technologies like NVIDIA NIM microservices and AI Blueprints — and breakthrough AI applications from startups and software partners — Media2 uses AI to drive the creation of smarter, more tailored and more impactful content that can adapt to individual viewer preferences.

Amid this rapid creative transformation, companies embracing NVIDIA Media2 can stay on the $3 trillion media and entertainment industry’s cutting edge, reshaping how audiences consume and engage with content.

NVIDIA Technologies at the Heart of Media2

As the media and entertainment industry embraces generative AI and accelerated computing, NVIDIA technologies are transforming how content is created, delivered and experienced.

NVIDIA Holoscan for Media is a software-defined, AI-enabled platform that allows companies in broadcast, streaming and live sports to run live video pipelines on the same infrastructure as AI. The platform delivers applications from vendors across the industry on NVIDIA-accelerated infrastructure.

Delivering the power needed to drive the next wave of data-enhanced intelligent content creation and hyper-personalized media is the NVIDIA Blackwell architecture, built to handle data-center-scale generative AI workflows with up to 25x more energy efficiency over the NVIDIA Hopper generation. Blackwell integrates six types of chips: GPUs, CPUs, DPUs, NVIDIA NVLink Switch chips, NVIDIA InfiniBand switches and Ethernet switches.

Blackwell is supported by NVIDIA AI Enterprise, an end-to-end software platform for production-grade AI. NVIDIA AI Enterprise comprises NVIDIA NIM microservices, AI frameworks, libraries and tools that media companies can deploy on NVIDIA-accelerated clouds, data centers and workstations. Of the expanding list, these include:

• The Llama 3.1-405B-Instruct NIM microservice, which enables synthetic data generation, distillation and inference for chatbots, coding and domain-specific tasks.

• The Mistral-NeMo-12B-Instruct NIM microservice, which enables multilingual information retrieval — the ability to search, process and retrieve knowledge across languages. This is key in enhancing an AI model’s outputs with greater accuracy and global relevancy.

• The NVIDIA Omniverse Blueprint for 3D conditioning for precise visual generative AI, which can help advertisers easily build personalized, on-brand and product-accurate marketing content at scale using real-time rendering and generative AI without affecting a hero product asset.

• NVIDIA NeMo Retriever embedding and reranking NIM microservices, which can vectorize text documents, transcripts, news articles and other written content. Media companies can use these to expand their generative AI efforts and build accurate, multilingual systems.

• The NVIDIA Cosmos Nemotron vision language model NIM microservice, which is a multimodal VLM that can understand the meaning and context of text, images and video. With the microservice, media companies can query images and videos with natural language and receive informative responses.

• The NVIDIA AI Blueprint for video search and summarization (VSS), which integrates VLMs and LLMs and provides cloud-native building blocks to build video analytics, search and summarization applications.

• The NVIDIA Edify multimodal generative AI architecture, which can generate visual assets — like images, 3D models and HDRi environments — from text or image prompts. It offers advanced editing tools and efficient training for developers. With NVIDIA AI Foundry, service providers can customize Edify models for commercial visual services using NVIDIA NIM microservices.

Partners in the Media2 Ecosystem

Partners across the industry are adopting NVIDIA technology to reshape the next chapter of storytelling.

Getty Images and Shutterstock are intelligent content creation services built with NVIDIA Edify. The AI models have also been optimized and packaged for maximum performance with NVIDIA NIM microservices.

Bria is a commercial-first visual generative AI platform designed for developers. It’s trained on 100% licensed data and built on responsible AI principles. The platform offers tools for custom pipelines, seamless integration and flexible deployment, ensuring enterprise-grade compliance and scalable, predictable content generation. Optimized with NVIDIA NIM microservices, Bria delivers faster, safer and scalable production-ready solutions.

Runway is an AI platform that provides advanced creative tools for artists and filmmakers. The company’s Gen-3 Alpha Turbo model excels in video generation and includes a new Camera Control feature that allows for precise camera movements like pan, tilt and zoom. Runway’s integration of the NVIDIA CV-CUDA open-source library combined with NVIDIA GPUs accelerates preprocessing for high-resolution videos in its segmentation model.

Wonder Dynamics, an Autodesk company, recently launched the beta version of Wonder Animation, featuring powerful new video-to-3D scene technology that can turn any video sequence into a 3D-animated scene for animated film production. Accelerated by NVIDIA GPU technology, Wonder Animation provides visual effects artists and animators with an easy-to-use, flexible tool that significantly reduces the time, complexity and efforts traditionally associated with 3D animation and visual effects workflows — while allowing the artist to maintain full creative control.

Comcast’s Sky innovation team is collaborating with NVIDIA on lab testing NVIDIA NIM microservices and partner models for its global platforms. The integration could lead to greater interactivity and accessibility for customers around the world, such as enabling the use of voice commands to request summaries during live sports and access other contextual information.

Vū, a creative technology company and home to the largest network of virtual studios, is broadening access to the creation of virtual environments and immersive content with NVIDIA-accelerated generative AI technologies.

Twelve Labs, a member of the NVIDIA Inception program for startups, is developing advanced multimodal foundation models that can understand videos like humans, enabling precise semantic search, content analysis and video-to-text generation. Twelve Labs uses NVIDIA H100 GPUs to significantly improve the models’ inference performance, achieving up to a 7x improvement in requests served per second.

S4 Capital’s Monks is using cutting-edge AI technologies to enhance live broadcasts with real-time content segmentation and personalized fan experiences. Powered by NVIDIA Holoscan for Media, the company’s solution is integrated with tools like NVIDIA VILA to generate contextual metadata for injection within a time-addressible media store framework — enabling precise, action-based searching within video content.

Additionally, Monks uses NVIDIA NeMo Curator to help process data to build tailored AI models for sports leagues and IP holders, unlocking new monetization opportunities through licensing. By combining these technologies, broadcasters can seamlessly deliver hyper-relevant content to fans as events unfold, while adapting to the evolving demands of modern audiences.

Media companies manage vast amounts of video content, which can be challenging and time-consuming to locate, catalog and compile into finished assets. Leading media-focused consultant and system integrator Qvest has developed an AI video discovery engine, built on NIM microservices, that accelerates this process by automating the data capture of video files. This streamlines a user’s ability to both discover and contextualize how videos can fit in their intended story.

Verizon is transforming global enterprise operations, as well as live media and sports content, by integrating its reliable, secure private 5G network with NVIDIA’s full-stack AI platform, including NVIDIA AI Enterprise and NIM microservices, to deliver the latest AI solutions at the edge.

Using this solution, streamers, sports leagues and rights holders can enhance fan experiences with greater interactivity and immersion by deploying high-performance 5G connectivity along with generative AI, agentic AI, extended reality and streaming applications that enable personalized content delivery. These technologies also help elevate player performance and viewer engagement by offering real-time data analytics to coaches, players, referees and fans. It can also enable private 5G-powered enterprise AI use cases to drive automation and productivity.

Welcome to NVIDIA Media2

The NVIDIA Media2 initiative empowers companies to redefine the future of media and entertainment through intelligent, data-driven and immersive technologies — giving them a competitive edge while equipping them to drive innovation across the industry.

NIM microservices from NVIDIA and model developers are now available to try, with additional models added regularly.

Get started with NVIDIA NIM and AI Blueprints, and watch the CES opening keynote delivered by NVIDIA founder and CEO Jensen Huang to hear the latest advancements in AI.

See notice regarding software product information.

Read More