Several of the world’s top server manufacturers today at COMPUTEX 2021 announced new systems powered by NVIDIA BlueField-2 data processing units. While systems with NVIDIA GPUs are already available now with the option of adding BlueField-2 DPUs, many applications and customer use cases may not require a GPU but can still benefit from a DPU.
The new servers from ASUS, Dell Technologies, GIGABYTE, QCT and Supermicro are ideal for enterprises seeking the additional performance, security and manageability that NVIDIA BlueField-2 DPUs offer.
The NVIDIA BlueField-2 DPU is the world’s leading data processing unit.
Servers that primarily run software-defined networking (for example, a stateful load balancer or distributed firewall), software-defined storage or traditional enterprise applications will all benefit from the DPU’s ability to accelerate, offload and isolate infrastructure workloads for networking, security and storage.
Systems running VMware vSphere, Windows or hyperconverged infrastructure solutions also benefit from including a DPU, whether running AI and machine learning applications, graphics-intensive workloads or traditional business applications.
NVIDIA BlueField DPUs shift infrastructure tasks from the CPU to the DPU, making more server CPU cores available to run applications, which increases server and data center efficiency. The DPU places a “computer in front of the computer” for each server, delivering separate, secure infrastructure provisioning that is isolated from the server’s application domain. This allows agentless workload isolation, security isolation, storage virtualization, remote management and telemetry on both virtualized and bare-metal servers.
Asus, Dell Technologies, GIGABYTE, QCT and Supermicro have announced plans to offer servers accelerated by NVIDIA BlueField-2 DPUs.
Customers and software makers can program BlueField DPUs easily using the NVIDIA DOCA SDK, the data-center-on-a-chip architecture that simplifies application development and ensures forward and backward compatibility with the NVIDIA BlueField-3 DPU, expected in 2022, and all future BlueField DPUs.
Enterprise customers are interested in deploying DPUs in many or even all of their data center servers. The availability of DPU-accelerated servers from leading system builders makes it simple and easy for customers to buy the market-leading BlueField-2 DPU pre-installed and pre-configured with their favorite servers.
BlueField-2 DPU-accelerated servers are expected this year, with several anticipated to submit to the NVIDIA-Certified Systems program when DPU server certification becomes available.
What’s a three-letter acronym for a “video-handling chip”? A GPU, of course. Who knew, though, that these parallel processing powerhouses could have a way with words, too.
Following a long string of victories for computers in other games — chess in 1997, go in 2016 and Texas hold’em poker in 2019 — a GPU-powered AI has beaten some of the world’s most competitive word nerds at the crossword puzzles that are a staple of every Sunday paper.
Dr.Fill, the crossword puzzle-playing AI created by Matt Ginsberg — a serial entrepreneur, pioneering AI researcher and former research professor — scored higher than any humans last month at the American Crossword Puzzle Tournament.
Dr.Fill’s performance against more than 1,300 crossword enthusiasts comes after a decade of playing alongside humans through the annual tournament.
Such games, played competitively, test the limits of how computers think and better understand how people do, Ginsberg explains. “Games are an amazing environment,” he says.
Dr.Fill’s edge? A sophisticated neural network developed by UC Berkeley’s Natural Language Processing team — trained in just days on an NVIDIA DGX-1 system and deployed on a PC equipped with a pair of NVIDIA GeForce RTX 2080 Ti GPUs — that snapped right into the system Ginsberg had been refining for years.
A way with words: A pair of NVIDIA RTX 2080 Tis powered a sophisticated neural network developed by UC Berkeley’s Natural Language Processing team.
“Crossword fills require you to make these creative multi-hop lateral connections with language,” says Professor Dan Klein, who leads the Natural Language Processing team. “I thought it would be a good test to see how the technology we’ve created in this field would handle that kind of creative language use.”
Given that unstructured nature, it’s amazing that a computer can compete at all. And to be sure, Dr.Fill still isn’t necessarily the best, and that’s not only because the American Crossword Puzzle Tournament’s official championship is reserved only for humans.
The contest’s organizer, New York Times Puzzle Editor Will Shortz, pointed out that Dr.Fill’s biggest advantage is speed: it can fill in answers in an instant that humans have to type out. Judged solely by accuracy, however, Dr.Fill still isn’t the best, making three errors during the contest, worse than several human contestants.
Nevertheless, Dr.Fill’s performance in a challenge that, unlike more structured games such as chess or go, rely so heavily on real-world knowledge and wordplay is remarkable, Shortz concedes.
“It’s just amazing they have programmed a computer to solve crosswords — especially some of the tricky hard ones,” Shortz said.
A Way with Words
Ginsberg, who holds a Ph.D. in mathematics from the University of Oxford and has 100 technical papers, 14 patents and multiple books to his name, has been a crossword fan since he attended college 45 years ago.
But his obsession took off when he entered a tournament more than a decade ago and didn’t win.
“‘The other competitors were so much better than I was, and it annoyed me, so I thought ‘Well, I should write a program,’ so I started Dr.Fill,” Ginsberg says.
Organized by Shortz, the American Crossword Tournament is packed with people who know their way around words.
Dr.Fill made its debut at the competition in 2012. Despite high expectations, Dr.Fill only managed to place 141st out of 600 contestants. Dr.Fill never managed a top 10 finish until this year.
In part, that’s because crosswords didn’t attract the kind of richly funded efforts that took on — and eventually beat — the best humans at chess and go.
It’s also partly because crossword puzzles are unique. “In go and chess and checkers, the rules are very clear,” Ginsberg says. “Crosswords are very interesting.”
Crossword puzzles often rely on cryptic clues that require deep cultural knowledge and an extensive vocabulary, as well as the ability to find answers that best slide into each puzzle’s overlapping rows and columns.
“It’s a messy thing,” Shortz said. “It’s not purely logical like chess or even like Scrabble, where you have a word list and every word is worth so many points.”
Crossword puzzles often rely on cryptic clues that require deep cultural knowledge and an extensive vocabulary.
A Winning Combination
The game-changer? Help from the Natural Language Processing team. Inspired by his efforts, the team reached out to Ginsberg a month before the competition began.
It proved to be a triumphant combination.
The Berkeley team focused on understanding each puzzle’s often gnomic clues and finding potential answers. Klein’s team of three graduate students and two undergrads took the more than 6 million examples of crossword clues and answers that Ginsberg had collected and poured them into a sophisticated neural network.
Ginsberg’s software, refined over many years, then handled the task of ranking all the answers that fit the confines of each puzzle’s grid and fitting them in with overlapping letters from other answers — a classic constraint satisfaction problem.
While their systems relied on very different techniques, they both spoke the common language of probabilities. As a result, they snapped together almost perfectly.
“We quickly realized that we had very complementary pieces of the puzzle,” Klein said.
Together, their models parallel some of the ways people think, Klein says. Humans make decisions by either remembering what worked in the past or using a model to simulate of what might work in the future.
“I get excited when I see systems that do some of both,” Klein said.
The result of combining both approaches: Dr.Fill played almost perfectly.
The AI made just three errors during the tournament. Its biggest edge, however, was speed. It dispatched most of the competition’s puzzles in under a minute.
AI Supremacy Anything But Assured
But since, unlike chess or go, crossword puzzles are ever-changing, another such showing isn’t guaranteed.
“It’s very likely that the constructors will throw some curveballs,” Shortz said.
Ginsberg says he’s already working to improve Dr.Fill. “We’ll see who makes more progress.”
The result may be out to be even more engaging crossword puzzles than ever.
“It turns out that the things that are going to stump a computer are really creative,” Klein said.
It will help piece together a 3D map of the universe, probe subatomic interactions for green energy sources and much more.
Perlmutter, officially dedicated today at the National Energy Research Scientific Computing Center (NERSC), is a supercomputer that will deliver nearly four exaflops of AI performance for more than 7,000 researchers.
That makes Perlmutter the fastest system on the planet on the 16- and 32-bit mixed-precision math AI uses. And that performance doesn’t even include a second phase coming later this year to the system based at Lawrence Berkeley National Lab.
More than two dozen applications are getting ready to be among the first to ride the 6,159 NVIDIA A100 Tensor Core GPUs in Perlmutter, the largest A100-powered system in the world. They aim to advance science in astrophysics, climate science and more.
A 3D Map of the Universe
In one project, the supercomputer will help assemble the largest 3D map of the visible universe to date. It will process data from the Dark Energy Spectroscopic Instrument (DESI), a kind of cosmic camera that can capture as many as 5,000 galaxies in a single exposure.
Researchers need the speed of Perlmutter’s GPUs to capture dozens of exposures from one night to know where to point DESI the next night. Preparing a year’s worth of the data for publication would take weeks or months on prior systems, but Perlmutter should help them accomplish the task in as little as a few days.
“I’m really happy with the 20x speedups we’ve gotten on GPUs in our preparatory work,” said Rollin Thomas, a data architect at NERSC who’s helping researchers get their code ready for Perlmutter.
Perlmutter’s Persistence Pays Off
DESI’s map aims to shed light on dark energy, the mysterious physics behind the accelerating expansion of the universe. Dark energy was largely discovered through the 2011 Nobel Prize-winning work of Saul Perlmutter, a still-active astrophysicist at Berkeley Lab who will help dedicate the new supercomputer named for him.
“To me, Saul is an example of what people can do with the right combination of insatiable curiosity and a commitment to optimism,” said Thomas, who worked with Perlmutter on projects following up the Nobel-winning discovery.
Supercomputer Blends AI, HPC
A similar spirit fuels many projects that will run on NERSC’s new supercomputer. For example, work in materials science aims to discover atomic interactions that could point the way to better batteries and biofuels.
Traditional supercomputers can barely handle the math required to generate simulations of a few atoms over a few nanoseconds with programs such as Quantum Espresso. But by combining their highly accurate simulations with machine learning, scientists can study more atoms over longer stretches of time.
“In the past it was impossible to do fully atomistic simulations of big systems like battery interfaces, but now scientists plan to use Perlmutter to do just that,” said Brandon Cook, an applications performance specialist at NERSC who’s helping researchers launch such projects.
That’s where Tensor Cores in the A100 play a unique role. They accelerate both the double-precision floating point math for simulations and the mixed-precision calculations required for deep learning.
Similar work won NERSC recognition in November as a Gordon Bell finalist for its BerkeleyGW program using NVIDIA V100 GPUs. The extra muscle of the A100 promises to take such efforts to a new level, said Jack Deslippe, who led the project and oversees application performance at NERSC.
Software Helps Perlmutter Sing
Software is a strategic component of Perlmutter, too, said Deslippe, noting support for OpenMP and other popular programming models in the NVIDIA HPC SDK the system uses.
Separately, RAPIDS, open-source code for data science on GPUs, will speed the work of NERSC’s growing team of Python programmers. It proved its value in a project that analyzed all the network traffic on NERSC’s Cori supercomputer nearly 600x faster than prior efforts on CPUs.
“That convinced us RAPIDS will play a major part in accelerating scientific discovery through data,” said Thomas.
Coping with COVID’s Challenges
Despite the pandemic, Perlmutter is on schedule. But the team had to rethink critical steps like how it ran hackathons for researchers working from home on code for the system’s exascale-class applications.
Meanwhile, engineers from Hewlett Packard Enterprise helped assemble phase 1 of the system, collaborating with NERSC staff who upgraded their facility to accommodate the new system. “We greatly appreciate the work of those people onsite bringing the system up, especially under all the special COVID protocols,” said Thomas.
At the virtual launch event, NVIDIA CEO Jensen Huang congratulated the Berkeley Lab crew on its plans to advance science with the supercomputer.
“Perlmutter’s ability to fuse AI and high performance computing will lead to breakthroughs in a broad range of fields from materials science and quantum physics to climate projections, biological research and more,” Huang said.
On Time for AI Supercomputing
The virtual ribbon cutting today represents a very real milestone.
“AI for science is a growth area at the U.S. Department of Energy, where proof of concepts are moving into production use cases in areas like particle physics, materials science and bioenergy,” said Wahid Bhimji, acting lead for NERSC’s data and analytics services group.
“People are exploring larger and larger neural-network models and there’s a demand for access to more powerful resources, so Perlmutter with its A100 GPUs, all-flash file system and streaming data capabilities is well timed to meet this need for AI,” he added.
Here’s a chance to become a marvel at marbles: the Marbles RTX playable sample is now available from the NVIDIA Omniverse launcher.
Marbles RTX is a physics-based mini-game level where a player controls a marble around a scene full of obstacles. The sample, which already has over 8,000 downloads, displays real-time physics with dynamic lighting and stunning, physically based materials.
The technology demo showcases NVIDIA Omniverse’s powerful suite of graphics, AI and simulation technologies. GeForce RTX gaming and NVIDIA RTX enthusiasts can download Marbles RTX and experience Omniverse’s advanced capabilities in real-time ray- and path-traced rendering, Deep Learning Super Sampling (DLSS) and complex physics simulation.
First previewed at GTC 2020, the Marbles RTX tech demo simulates the dynamic world in real time, without any precomputation or baking. It highlights NVIDIA’s advanced rendering and physics with exceptionally high-quality 3D content created from scratch.
The final Marbles RTX tech demo completed in Omniverse resulted in over 500GB of texture data, 165 unique assets that were modeled and textured by hand, more than 5,000 meshes and about 100 million polygons.
During the GeForce RTX 30 Series launch event in September, NVIDIA unveiled a more challenging take on the demo with the release of Marbles at Night RTX. This presented a night scene that contained hundreds of dynamic, animated lights. Based on NVIDIA Research, Marbles at Night RTX showcased how the power and beauty of RTX-enabled real-time ray tracing allows artists to render dynamic direct lighting and shadows from millions of area lights in real time.
The combination of physically based MDL materials and real-time, referenced path tracing in Omniverse brings high-quality details to the Marbles scene, enabling players to feel like they’re looking at real-world objects. The Omniverse RTX Renderer calculates reflections, refraction and global illumination accurately while the denoiser easily manages all the complex geometry across the entire scene.
NVIDIA PhysX 5 and Flow simulate the interaction of rigid-body objects and fluids in the scene in real time, and NVIDIA DLSS enhances the details in the image with powerful AI, allowing users to focus GPU resources on accuracy and fidelity. All these elements combined provide a unique look and feel in CGI that users typically can’t get from real-time games.
At GTC 2021, the artists behind Marbles RTX hosted an exclusive deep dive session detailing the creative and development process. Learn more about the Making of Marbles by watching the GTC session on demand, which is available now and free to access.
Download NVIDIA Omniverse and try the Marbles RTX playable sample today.
GFN Thursday comes roaring in with 22 games and support for three DLCs joining the GeForce NOW library this week.
Among the 22 new releases are five day-and-date game launches: Biomutant, Maneater, King of Seas, Imagine Earth and Warhammer Age of Sigmar: Storm Ground.
DLC, Without the Download
GeForce NOW ensures your favorite games are automatically up to date, avoiding game updates and patches. Simply log in, click PLAY and enjoy an optimal cloud gaming experience.
This includes supporting the latest expansions and other downloadable content — without any local downloads.
Three great games are getting new DLC, and they’re streaming on GeForce NOW.
Hunt: Showdown’s newest DLC adds new hunter Henry Monroe to the mix. He doesn’t look stable to us.
Hunt Showdown — The Committed DLC contains one Legendary Hunter (Monroe), a Legendary knife (Pane) and a Legendary Romero 77 (Lock and Key). It’s available on Steam, so members can start hunting now.
Isle of Siptah, the massive expansion to the open world survival game Conan Exiles, is exiting early access and releasing on Steam today. It features a vast new island to explore, huge and vile new creatures to slay, new building sets and a host of new features. Gamers have 40 new NPC camps and points of interest to explore, three new factions of NPCs, new ways of acquiring thralls and much more.
Announced last month, Iron Harvest – Operation Eagle, the new expansion to the critically acclaimed world of Iron Harvest set in the alternate reality of 1920, is available on Steam and streaming with GeForce NOW. Guide the new faction through seven new single-player missions, while learning how to use the game’s new Aircraft units across all of the game’s playable factions, including Polania, Saxony and Rusviet.
Newest Additions of the Week
GFN Thursday wouldn’t be complete without new games. The library evolved this week, but didn’t chew you up, with five day-and-date releases, including the launch of Biomutant, from Experiment 101 and THQ Nordic.
A gorgeous open world to explore as a weapon-wielding rodent? That’s our perfect weekend.
Explore a strange new world as an ever-evolving, weapon-wielding, martial arts master anthropomorphic rodent in this featured game of the week! For more information, read here.
Including Biomutant, members can expect a total of 22 games this week:
A few games that we planned to release in May didn’t quite make it this month. Some were due to technical issues, others are on the way. Look for updates to the below in the weeks ahead.
Beyond Good & Evil (Steam)
Child of Light (Russian version only, Ubisoft Connect)
Hearts of Iron III (Steam)
King’s Bounty: Dark Side (Steam)
Sabotaj (Steam)
Super Mecha Champions (Steam)
Thea: The Awakening (Steam)
Tomb Raider Legend (Steam)
What are you going to play? Let us know on Twitter or in the comments below.
Path-breaking work that translates an amputee’s thoughts into finger motions, and even commands in video games, holds open the possibility of humans controlling just about anything digital with their minds.
Using GPUs, a group of researchers trained an AI neural decoder able to run on a compact, power-efficient NVIDIA Jetson Nano system on module (SOM) to translate 46-year-old Shawn Findley’s thoughts into individual finger motions.
And if that breakthrough weren’t enough, the team then plugged Findley into a PC running Far Cry 5 and Raiden IV, where he had his game avatar move, jump — even fly a virtual helicopter — using his mind.
It’s a demonstration that not only promises to give amputees more natural and responsive control over their prosthetics. It could one day give users almost superhuman capabilities.
“The idea is intuitive to video gamers,” said Anh Tuan Nguyen, the paper’s lead author and now a postdoctoral researcher at the University of Minnesota advised by Associate Professor Zhi Yang.
“Instead of mapping our system to a virtual hand, we just mapped it to keystrokes — and five minutes later, we’re playing a video game,” said Nguyen, an avid gamer, who holds a bachelor’s degree in electrical engineering and Ph.D. in biomedical engineering.
Shawn Findley, who lost his hand following an accident 17 years ago, was able to use an AI decoder to translate his thoughts in real-time into actions.
In short, Findley — a pastor in East Texas who lost his hand following an accident in a machine shop 17 years ago — was able to use an AI decoder trained on an NVIDIA TITAN X GPU and deployed on the NVIDIA Jetson to translate his thoughts in real-time into actions inside a virtual environment running on, of course, yet another NVIDIA GPU, Nguyen explained.
Bionic Plan
Findley was one of a handful of patients who participated in the clinical trial supported by the U.S. Defense Advanced Research Projects Agency’s HAPTIX program.
The human physiology study is led by Edward Keefer, a neuroscientist and electrophysiologist who leads Texas-based Nerves Incorporated, and Dr. Jonathan Cheng at the University of Texas Southwestern Medical Center.
In collaboration with Yang’s and Associate Professor Qi Zhao’s labs at the University of Minnesota, the team collected large-scale human nerve data and is one of the first to implement deep learning neural decoders in a portable platform for clinical neuroprosthetic applications.
That effort aims to improve the lives of millions of amputees around the world. More than a million people lose a limb to amputation every year. That’s one every 30 seconds.
Prosthetic limbs have advanced fast over the past few decades — becoming stronger, lighter and more comfortable. But neural decoders, which decode movement intent from nerve data promise a dramatic leap forward.
With just a few hours of training, the system allowed Findley to swiftly, accurately and intuitively move the fingers on a portable prosthetic hand.
“It’s just like if I want to reach out and pick up something, I just reach out and pick up something,” reported Findley.
The key, it turns out, is the same kind of GPU-accelerated deep learning that’s now widely used for everything from online shopping to speech and voice recognition.
Teamwork
For amputees, even though their hand is long gone, parts of the system that controlled the missing hand remain.
Every time the amputee imagines grabbing, say, a cup of coffee with a lost hand, those thoughts are still accessible in the peripheral nerves once connected to the amputated body part.
To capture those thoughts, Dr. Cheng at UTSW surgically inserted arrays of microscopic electrodes into the residual median and ulnar nerves of the amputee forearm.
These electrodes, with carbon nanotube contacts, are designed by Keefer to detect the electrical signals from the peripheral nerve.
Dr. Yang’s lab designed a high-precision neural chip to acquire the tiny signals recorded by the electrodes from the residual nerves of the amputees.
Dr. Zhao’s lab then developed machine learning algorithms that decode neural signals into hand controls.
GPU-Accelerated Neural Network
Here’s where deep learning comes in.
Data collected by the patient’s nerve signals — and translated into digital signals — are then used to train a neural network that decodes the signals into specific commands for the prosthesis.
It’s a process that takes as little as two hours using a system equipped with a TITAN X or NVIDIA GeForce 1080 Ti GPU. One day users may even be able to train such systems at home, using cloud-based GPUs.
These GPUs accelerate an AI neural decoder designed based on a recurrent neural network running on the PyTorch deep learning framework.
Use of such neural networks has exploded over the past decade, giving computer scientists the ability to train systems for a vast array of tasks, from image and speech recognition to autonomous vehicles, too complex to be tackled with traditional hand-coding.
The challenge is finding hardware powerful enough to swiftly run this neural decoder, a process known as inference, and power-efficient enough to be fully portable.
Portable and powerful: Jetson Nano’s CUDA cores provide full support for popular deep learning libraries such as TensorFlow, PyTorch and Caffe.
So the team turned to the Jetson Nano, whose CUDA cores provide full support for popular deep learning libraries such as TensorFlow, PyTorch and Caffe.
“This offers the most appropriate tradeoff among power and performance for our neural decoder implementation,” Nguyen explained.
Deploying this trained neural network on the powerful, credit card sized Jetson Nano resulted in a portable, self-contained neuroprosthetic hand that gives users real-time control of individual finger movements.
Using it, Findley demonstrated both high-accuracy and low-latency control of individual finger movements in various laboratory and real-world environments.
The next step is a wireless and implantable system, so users can slip on a portable prosthetic device when needed, without any wires protruding from their body.
Nguyen sees robust, portable AI systems — able to understand and react to the human body — augmenting a host of medical devices coming in the near future.
The technology developed by the team to create AI-enabled neural interfaces is being licensed by Fasikl Incorporated, a startup sprung from Yang’s lab.
The goal is to pioneer neuromodulation systems for use by amputees and patients with neurological diseases, as well as able-bodied individuals who want to control robots or devices by thinking about it.
“When we get the system approved for nonmedical applications, I intend to be the first person to have it implanted,” Keefer said. “The devices you could control simply by thinking: drones, your keyboard, remote manipulators — it’s the next step in evolution.”
Banks use AI to determine whether to extend credit, and how much, to customers. Radiology departments deploy AI to help distinguish between healthy tissue and tumors. And HR teams employ it to work out which of hundreds of resumes should be sent on to recruiters.
These are just a few examples of how AI is being adopted across industries. And with so much at stake, businesses and governments adopting AI and machine learning are increasingly being pressed to lift the veil on how their AI models make decisions.
Charles Elkan, a managing director at Goldman Sachs, offers a sharp analogy for much of the current state of AI, in which organizations debate its trustworthiness and how to overcome objections to AI systems:
We don’t understand exactly how a bomb-sniffing dog does its job, but we place a lot of trust in the decisions they make.
To reach a better understanding of how AI models come to their decisions, organizations are turning to explainable AI.
What Is Explainable AI?
Explainable AI, or XAI, is a set of tools and techniques used by organizations to help people better understand why a model makes certain decisions and how it works. XAI is:
A set of best practices: It takes advantage of some of the best procedures and rules that data scientists have been using for years to help others understand how a model is trained. Knowing how, and on what data, a model was trained helps us understand when it does and doesn’t make sense to use that model. It also shines a light on what sources of bias the model might have been exposed to.
A set of design principles: Researchers are increasingly focused on simplifying the building of AI systems to make them inherently easier to understand.
A set of tools: As the systems get easier to understand, the training models can be further refined by incorporating those learnings into it — and by offering those learnings to others for incorporation into their models.
How Does Explainable AI Work?
While there’s still a great deal of debate over the standardization of XAI processes, a few key points resonate across industries implementing it:
Who do we have to explain the model to?
How accurate or precise an explanation do we need?
Do we need to explain the overall model or a particular decision?
Source: DARPA
Data scientists are focusing on all these questions, but explainability boils down to: What are we trying to explain?
Explaining the pedigree of the model:
How was the model trained?
What data was used?
How was the impact of any bias in the training data measured and mitigated?
These questions are the data science equivalent of explaining what school your surgeon went to — along with who their teachers were, what they studied and what grades they got. Getting this right is more about process and leaving a paper trail than it is about pure AI, but it’s critical to establishing trust in a model.
While explaining a model’s pedigree sounds fairly easy, it’s hard in practice, as many tools currently don’t support strong information-gathering. NVIDIA provides such information about its pretrained models. These are shared on the NGC catalog, a hub of GPU-optimized AI and high performance computing SDKs and models that quickly help businesses build their applications.
Explaining the overall model:
Sometimes called model interpretability, this is an active area of research. Most model explanations fall into one of two camps:
In a technique sometimes called “proxy modeling,” simpler, more easily comprehended models like decision trees can be used to approximately describe the more detailed AI model. These explanations give a “sense” of the model overall, but the tradeoff between approximation and simplicity of the proxy model is still more art than science.
Proxy modeling is always an approximation and, even if applied well, it can create opportunities for real-life decisions to be very different from what’s expected from the proxy models.
The second approach is “design for interpretability.” This limits the design and training options of the AI network in ways that attempt to assemble the overall network out of smaller parts that we force to have simpler behavior. This can lead to models that are still powerful, but with behavior that’s much easier to explain.
This isn’t as easy as it sounds, however, and it sacrifices some level of efficiency and accuracy by removing components and structures from the data scientist’s toolbox. This approach may also require significantly more computational power.
Why XAI Explains Individual Decisions Best
The best understood area of XAI is individual decision-making: why a person didn’t get approved for a loan, for instance.
Techniques with names like LIME and SHAP offer very literal mathematical answers to this question — and the results of that math can be presented to data scientists, managers, regulators and consumers. For some data — images, audio and text — similar results can be visualized through the use of “attention” in the models — forcing the model itself to show its work.
In the case of the Shapley values used in SHAP, there are some mathematical proofs of the underlying techniques that are particularly attractive based on game theory work done in the 1950s. There is active research in using these explanations of individual decisions to explain the model as a whole, mostly focusing on clustering and forcing various smoothness constraints on the underlying math.
The drawback to these techniques is that they’re somewhat computationally expensive. In addition, without significant effort during the training of the model, the results can be very sensitive to the input data values. Some also argue that because data scientists can only calculate approximate Shapley values, the attractive and provable features of these numbers are also only approximate — sharply reducing their value.
While healthy debate remains, it’s clear that by maintaining a proper model pedigree, adopting a model explainability method that provides clarity to senior leadership on the risks involved in the model, and monitoring actual outcomes with individual explanations, AI models can be built with clearly understood behaviors.
For a closer look at examples of XAI work, check out the talks presented by Wells Fargo and ScotiaBank at NVIDIA GTC21.
NVIDIA’s GPU Technology Conference is a hotbed for sharing groundbreaking innovations — making it the perfect forum for developers, students and professionals from underrepresented communities to discuss the challenges and opportunities surrounding AI.
It had 6x increase in female attendees from last fall’s event, a 6x jump in Black attendees and a 5x boost in Hispanic and Latino attendees. Dozens signed up for hands-on training from the NVIDIA Deep Learning Institute and joined networking sessions hosted by NVIDIA community resource groups in collaboration with organizations like Black in AI and LatinX in AI.
More than 1,500 sessions from GTC 2021 are now available for free replay on NVIDIA On-Demand — including panel discussions on AI literacy and efforts to grow the participation of underrepresented groups in science and engineering.
Advocating for AI Literacy Among Youth
In a session called “Are You Smarter Than a Fifth Grader Who Knows AI?,” STEM advocates Justin Shaifer and Maynard Okereke (known as Mr. Fascinate and the Hip Hop M.D., respectively) led a conversation about initiatives to help young people understand AI.
Given the ubiquity of AI technologies, being surrounded by it “is essentially just how they live,” said Jim Gibbs, CEO of the Pittsburgh-based startup Meter Feeder. “They just don’t know any different.”
But school curriculums often don’t teach young people how AI technologies work, how they’re developed or about AI ethics. So it’s important to help the next generation of developers prepare “to take advantage of all the new opportunities that there are going to be for people who are familiar with machine learning and artificial intelligence,” he said.
Panelist Lisa Abel-Palmieri, CEO of the Boys & Girls Clubs of Western Pennsylvania, described how her organization’s STEM instructors audited graduate-level AI classes at Carnegie Mellon University to inform a K-12 curriculum for children from historically marginalized communities. NVIDIA recently announced a three-year AI education partnership with the organization to create an AI Pathways Toolkit that Boys & Girls Clubs nationwide can deliver to students, particularly those from underserved and underrepresented communities.
And Babak Mostaghimi, assistant superintendent of Georgia’s Gwinnett County Public Schools shared how his team helps students realize how AI is relevant to their daily experiences.
“We started really getting kids to understand that AI is already part of your everyday life,” he said. “And when kids realize that, it’s like, wait a minute, let me start asking questions like: Why does the algorithm behind something cause a certain video to pop up and not others?”
Another panel, “Diversity Driving AI Innovation,” was led by Brennon Marcano, CEO of the National GEM Consortium, a nonprofit focused on diversifying representation in science and engineering.
Researchers and scientists from Apple, Amazon Web Services and the University of Utah shared their experiences working in AI, and the value that the perspectives of underrepresented groups can provide in the field.
“Your system on the outside is only as good as the data going in on the side,” said Marcano. “So if the data is homogeneous and not diverse, then the output suffers from that.”
But diversity of datasets isn’t the only problem, said Nashlie Sephus, a tech evangelist at Amazon Web Services AI who focuses on fairness and identifying biases. Another essential consideration is making sure developer teams are diverse.
“Just by having someone on the team with a diverse experience, a diverse perspective and background — it goes a long way. Teams and companies are now starting to realize that,” she said.
The panel described how developers can mitigate algorithmic bias, improve diversity on their teams and find strategies to fairly compensate focus groups who provide feedback on products.
“Whenever you are trying to create something in software that will face the world, the only way you can be precisely coupled to that world is to invite the world into that process,” said Rogelio Cardona-Rivera, assistant professor at the University of Utah. “There’s no way you will be able to be as precise if you leave diversity off the table.”
Every day, vast amounts of geospatial imagery are being collected, and yet, until recently, one of the biggest potential users of that trove — property insurers — had made surprisingly little use of it.
Now, CAPE Analytics, a computer vision startup and NVIDIA Inception member, seeks to turn that trove of geospatial imagery into better underwriting decisions, and is applying these insights to mitigate wildfire disasters.
Traditionally, the insurance industry could only rely on historic data for broad swaths of land, combined with an in-person visit. CAPE Analytics can use AI to produce detailed data on the vegetation density, roof material and proximity to surrounding structures. This provides a better way to calculate risk, as well as an avenue to help homeowners take actions to cut it.
“For the first time, insurers can quantify defensible space- the removal of flammable material, such as vegetation from around a home- with granular analytics,” said Kevin van Leer, director of Customer Success at CAPE Analytics. “CAPE allows insurance carriers to identify the vulnerability of a specific home and make recommendations to the homeowner. For example, our recent study shows that cutting back vegetation in the 10 feet surrounding a home is the most impactful action a homeowner can take to reduce their wildfire risk. It’s also much easier to achieve in comparison to the frequently recommended 30-to-100 foot buffer.”
As fire seasons grow longer and deadlier each year, and wildfires are driven by hotter, drier, and faster winds, the risk area widens into newer areas, not found on older maps. This makes up-to-date insights especially crucial.
“What’s unique about this dataset is that it’s very recent, and it’s high resolution,” said Kavan Farzaneh, head of marketing at the Mountain View, Calif., based company. “Using AI, we can analyze it at scale.”
Insights from such analysis extend beyond weather risk to “blue sky,” or day-to-day risk, as well. Whether that means determining the condition of a roof, factoring in new solar panels or detecting the presence of a trampoline, CAPE’s software seeks to optimize the underwriting process by helping insurers make more informed decisions about what policies to write.
And given that the six-year-old company already boasts more than 40 insurance industry customers and is backed by investments from several large insurance carriers, including the Hartford, State Farm and CSAA, CAPE Analytics appears to be on to something.
Creating More Accurate Records
For some time, insurance companies have used aerial imagery for claims verification, such as reviewing storm damage. But CAPE Analytics is converting that imagery into structured data that underwriters can use to streamline their decision-making process. The company is essentially creating more up-to-date property records, which traditionally come from tax assessor offices and other public records sources.
“We zeroed in on property underwriting because there was a void in accuracy, and data tends to be old,” said Busy Cummings, chief revenue officer at CAPE Analytics. “By using AI to tap into this objective ‘source of truth,’ we can improve the accuracy of existing data sources.”
And that means more efficiency for underwriters, who can avoid unnecessary inspections altogether thanks to having access to more current and complete data.
CAPE Analytics obtains its datasets from multiple imagery partners. Human labelers tag some of the data, and the company has trained algorithms that can then identify elements of an aerial image, such as whether a roof is flat or has gables, whether additional structures have been added, or if trees and brush are overgrowing the structure.
The company started training its models on several NVIDIA GPU-powered servers. It has since transitioned the bulk of its training activities to Amazon Web Services P3 instances running NVIDIA V100 Tensor Core GPUs.
Inferencing is running on the NVIDIA Triton inferencing server. CAPE Analytics relies on multiple Triton instances to run its models, with a load balancer distributing inferencing requests, allowing it to scale horizontally to meet shifting client demand. The company’s infrastructure makes it possible to do live inferencing of imagery, with geospatial data converted into actionable structured data in two seconds.
In Pursuit of Scale
Thanks to its membership in NVIDIA Inception, the company has recently been experimenting with the NVIDIA DGX A100 AI system to train larger networks on larger datasets. Jason Erickson, director of platform engineering at CAPE Analytics, said the experience with the DGX A100 has shown “what we could potentially achieve if we had unlimited resources.”
“We’ve been very fortunate to be a part of NVIDIA’s Inception program since 2017, which has afforded us opportunities to test new NVIDIA offerings, including data science GPU and DGX A100 systems, while engaging with the wider NVIDIA community,” said Farzaneh.
CAPE Analytics has every motivation to pursue more scale. Cummings said it has spent the past year focused on expanding from insurance underwriting into the real estate and mortgage markets, where there is demand to integrate property condition data into the tools that determine home values. The company also just announced it’s powering a new automated valuation model based on geospatial data.
With so many potential markets to explore, CAPE Analytics has to keep pushing the envelope.
“Machine learning is such a fast-moving world. Every day there are new papers and new methods and new models,” said Farzaneh. “We’re just trying to stay on the bleeding edge.”
GFN Thursday returns with a brand new adventure, exploring the unknown in Phantom Abyss, announced just moments ago by Devolver Digital and Team WIBY. The game launches on PC this summer, and when it does, it’ll be streaming instantly to GeForce NOW members.
No GFN Thursday would be complete without new games. And this week is chock full of new-new ones. Five PC games launching this week are joining the GeForce NOW library — including Saints Row: The Third Remastered — plus twelve additional titles.
It’s a good week to have your head in the clouds.
Adventure From the Clouds
Devolver and Team WIBY have an exciting spin on dungeon-delving, and they’re bringing it to GeForce NOW members at release.
“Phantom Abyss and GeForce NOW are a perfect match,” said Josh Sanderson, lead programmer at Team WIBY. “We can’t wait for gamers to explore each dungeon, and now those adventurers can stream from the cloud even if their gaming PC isn’t up to par.”
Members who purchase the game on Steam will be able to play Phantom Abyss across nearly all of their devices, at legendary GeForce quality. Away from your rig but looking for a challenge? Stream to your Android mobile device, Chromebook or any other supported device. You can bring the action with you, wherever you are.
Read on to learn more about Devolver’s freshly announced Phantom Abyss.
You’ll face perils in each of Phantom Abyss’ tombs.
Temple of Zoom
Phantom Abyss is a massive asynchronous multiplayer game that casts players into procedurally generated temples and tasks them with retrieving the sacred relics hidden within deadly chambers.
You’ll need to dodge hidden traps, leap across chasms and defeat foes as you navigate each labyrinth to claim the relic at the end. Oh, and you only get one chance per temple. Luckily, you’re joined on your quest by the phantoms of adventurers who tried before you, and you can learn from their run to aid your own.
To survive, you’ll need to learn from those who came before you.
Explore the perilous halls and colossal rooms of each temple alongside the phantoms of fallen players that came before you. Use their successes and failures to your advantage to progress deeper than they ever could’ve hoped. Watch and learn from the mistakes of up to 50 phantoms, including your Steam friends who have attempted the same temple, and steal their whips as they fall.
Those stolen whips matter, as they carry minor blessings to aid you on your run. But beware: they’re also cursed, and balancing the banes and boons is key to reaching your ultimate prize.
The competition if fierce. Succeed where others have failed, and the treasures of each tomb will be yours.
As you progress on each run, you’ll recover keys from chests to unlock deeper, deadlier sections of the temple that house more coveted relics. The more difficult a relic is to obtain, the greater the reward.
And if you succeed, the legendary relic at the bottom of each temple will be yours, sealing the temple and cementing your legacy.
Exclusive Tips & Tricks
The folks at Devolver shared a few tips and tricks to help you start your adventure in Phantom Abyss.
Before each run you can select between two standard whips and one legendary one that can be acquired in exchange for tokens that you’ve found in previous runs. Select carefully though, as each whip has its own blessings and curses, so it’s important to find the whip that boosts your play style.
The phantoms of ghosts aren’t just fun to watch meet their demise, they can be a helpful guide in your run! Phantoms can set off traps for you, which can be advantageous but also unexpected, so stay on your toes. If a phantom dies in front of you, you can pick up its whip if you find that it’s more beneficial to you on the run.
The guardians are relentless so always keep them in mind the deeper you get into a temple — they tend to cause complete chaos when in the same room as you!
Each temple has different levels and as players move down they can choose to take a more common relic and secure their lesser success or choose a different door to venture further into the Caverns and then the Inferno for even more treasure and glory.
Remastered and Returning
The remastered version of Volition’s classic returns to GeForce NOW.
Saints Row: The Third Remastered is both joining and returning to GeForce NOW. The game launches on Steam later this week and we’ll be working to bring it to GeForce NOW after it’s released. Aligned to the launch, the Epic Games Store version of the game will make its triumphant return to GeForce NOW.
The remastered edition includes enhanced graphics, plus all DLC included – the three expansion mission packs and 30 pieces of DLC from the original version.
Get Your Game On
That’s only the beginning. GFN Thursday means more games, and this week’s list includes four more day-and-date releases. Members can look forward to the following this week:
Snowrunner (day-and-date release on Steam, May 17)
Siege Survival Gloria Victis (day-and-date release on Steam, May 18)
Just Die Already (day-and-date release on Steam, May 20)
Researchers and scientists from Apple, Amazon Web Services and the University of Utah shared their experiences working in AI, and the value that the perspectives of underrepresented groups can provide in the field.
