Editor’s note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.
While still in grad school, Antonio Serrano-Muñoz has helped author papers spanning planetary gravities, AI-powered diagnosis of rheumatoid arthritis and robots that precisely track millimetric-sized walkers, like ants.
Now, the Ph.D. student in applied robotics at Mondragon University in northern Spain, is using robotics and AI in remanufacturing to tackle issues of climate change and pollution.
In short, Serrano-Muñoz is busy discovering unique ways to apply technology to complex, real-world issues — and, in his free time, he makes extensions for NVIDIA Omniverse, a platform for real-time 3D design collaboration and world simulation.
Omniverse Extensions are core building blocks that enable anyone to create and extend the functionality of Omniverse Apps to meet the specific needs of their workflows with just a few lines of Python code.
Serrano-Muñoz has created six open-source Omniverse Extensions that can be accessed on GitHub, one of which enhances the capabilities of NVIDIA Isaac Sim, an Omniverse-powered application framework for creating photorealistic, physically accurate virtual environments in which to develop, train and simulate AI robots.
“Since the beginning of my Ph.D. studies, I decided to work with Isaac Sim to simulate robots,” Serrano-Muñoz said. “It offers a powerful, real-time simulation platform with an ultra-realistic physics engine and graphics — as well as a clean, simple application programming interface that makes it easy to interact with the whole system.”
Omniverse for Robotics Simulation
Serrano-Muñoz has pursued robotics for as long as he can remember. Growing up in Cuba, he always fiddled with electronics, and he fell deeper in love with technology when he began coding in college.
“Robots can assist humans with strenuous, high-precision, repetitive and sometimes hazardous tasks,” Serrano-Muñoz said. “They have the potential to improve our lives, and I hope my work advances robotics in a way that allows us to build a better present and achieve a better future.”
He believes Omniverse is crucial to his doctoral studies in applied robotics.
“Performing real-time, graphically realistic simulations of robotics environments wasn’t possible before Omniverse,” he said. “The platform opens the door to a new era of revolutionary changes in robotics, simulation and real-time collaboration.”
Omniverse links specialists of all kinds — engineers, designers, content creators — for the development of simulation systems, he added. Key for this is Universal Scene Description (USD), an open source 3D scene description and extensible file framework serving as the common language for virtual worlds.
“USD plays an important role in the process of authoring, composing and reading a hierarchically organized scene to create and manipulate its rendering elements and objects,” Serrano-Muñoz said.
Extending Omniverse for Isaac Sim
Using NVIDIA Omniverse, the developer has created robot simulations for industrial use cases and a digital twin of Mondragon University’s laboratory for robotics prototyping.
And while working on such projects, Serrano-Muñoz wanted to integrate with Isaac Sim a tool he was already familiar with: Robot Operating System, or ROS, a set of software libraries for building robot applications. So, he created an Omniverse Extension to enable just that.
The extension lets users manipulate simulated robotic systems in the Omniverse-powered Isaac Sim application via ROS control interfaces. ROS MoveIt, a motion planning framework for robots, can be used in conjunction with Isaac Sim’s dynamic control extension and PhysX capabilities, which bring physical accuracy to high-fidelity robotics simulations.
“It’s easy to develop code without leaving the Omniverse Kit,” Serrano-Muñoz said. “Omniverse Extensions come with a system-wide integration API, installation, activation and reload mechanisms to augment the functionality of Omniverse Apps.”
This particular extension for ROS, he added, boosts agile prototyping for robotics applications — which is further accelerated by his NVIDIA RTX 3080 Laptop GPU — making his workflow faster than ever.
Hear more from Serrano-Muñoz about using digital twins for industrial robotics by watching his NVIDIA GTC session on demand. And watch his Community Spotlight on the Omniverse Twitch channel happening Aug. 3 at 11 a.m. PT.
Join in on the Creation
Creators and developers across the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.
Developers like Serrano-Muñoz will join NVIDIA at SIGGRAPH, a global computer graphics conference running Aug. 8-11. Watch the Omniverse community livestream at SIGGRAPH on Tuesday, Aug. 9, at noon PT to learn how Omniverse and other design and visualization technologies are driving breakthroughs in graphics and GPU-accelerated software.
Plus, anyone can submit to the inaugural #ExtendOmniverse developer contest through Friday, Aug. 19. Create an Omniverse Extension using Omniverse Code for a chance to win an NVIDIA RTX GPU.
Check out artwork from other “Omnivores” and submit projects in the gallery. Connect your workflows to Omniverse with software from Adobe, Autodesk, Epic Games, Maxon, Reallusion and more.
Follow NVIDIA Omniverse on Instagram, Twitter, YouTube and Medium for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.
The post Meet the Omnivore: Developer Builds Bots With NVIDIA Omniverse and Isaac Sim appeared first on NVIDIA Blog.