Stanford AI Lab Papers and Talks at AAAI 2022

The 36th AAAI Conference on Artificial Intelligence (AAAI 2022) is being hosted virtually from February 22th – March 1st. We’re excited to share all the work from SAIL that’s being presented, and you’ll find links to papers, videos and blogs below. Feel free to reach out to the contact authors directly to learn more about the work that’s happening at Stanford.

List of Accepted Papers

Partner-Aware Algorithms in Decentralized Cooperative Bandit Teams


Authors: Erdem Bıyık, Anusha Lalitha, Rajarshi Saha, Andrea Goldsmith, Dorsa Sadigh

Contact: ebiyik@stanford.edu

Links: Paper | [Video](https://www.youtube.com/watch?v=MCHXAYvaB5Y | 2nd Video | Website

Keywords: bandits, multi-agent systems, collaboration, human-robot interaction, partner-awareness

Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning


Authors: Tong Mu, Georgios Theocharous, David Arbour, Emma Brunskill

Contact: tongm@stanford.edu

Links: Paper

Keywords: reinforcement learning, constraints

IS-Count: Large-scale Object Counting from Satellite Images with Covariate-based Importance Sampling


Authors: Chenlin Meng*, Enci Liu*, Willie Neiswanger, Jiaming Song, Marshall Burke, David Lobell, Stefano Ermon

Contact: jesslec@stanford.edu

Award nominations: Oral presentation

Links: Paper | Blog Post | Website

Keywords: remote sensing, sampling

PantheonRL


Authors: Bidipta Sarkar, Aditi Talati, Andy Shih, Dorsa Sadigh

Contact: bidiptas@stanford.edu

Links: Paper | Video | Website

Keywords: multiagent reinforcement learning; software package; web user interface; adaptive marl; dynamic training interactions

Synthetic Disinformation Attacks on Automated Fact Verification Systems


Authors: Yibing Du, Antoine Bosselut, Christopher D Manning

Contact: antoineb@cs.stanford.edu

Keywords: fact checking, fact verification, disinformation, synthetic text

Similarity Search for Efficient Active Learning and Search of Rare Concepts


Authors: Cody Coleman, Edward Chou, Julian Katz-Samuels, Sean Culatana, Peter Bailis, Alexander C. Berg, Robert Nowak, Roshan Sumbaly, Matei Zaharia, I. Zeki Yalniz

Contact: cody@cs.stanford.edu

Links: Paper

Keywords: active learning, computer vision, active search, large-scale, data-centric ai

We look forward to seeing you at AAAI 2022.

Read More

Can machine-learning models overcome biased datasets?

Artificial intelligence systems may be able to complete tasks quickly, but that doesn’t mean they always do so fairly. If the datasets used to train machine-learning models contain biased data, it is likely the system could exhibit that same bias when it makes decisions in practice.

For instance, if a dataset contains mostly images of white men, then a facial-recognition model trained with these data may be less accurate for women or people with different skin tones.

A group of researchers at MIT, in collaboration with researchers at Harvard University and Fujitsu Ltd., sought to understand when and how a machine-learning model is capable of overcoming this kind of dataset bias. They used an approach from neuroscience to study how training data affects whether an artificial neural network can learn to recognize objects it has not seen before. A neural network is a machine-learning model that mimics the human brain in the way it contains layers of interconnected nodes, or “neurons,” that process data.

The new results show that diversity in training data has a major influence on whether a neural network is able to overcome bias, but at the same time dataset diversity can degrade the network’s performance. They also show that how a neural network is trained, and the specific types of neurons that emerge during the training process, can play a major role in whether it is able to overcome a biased dataset.

“A neural network can overcome dataset bias, which is encouraging. But the main takeaway here is that we need to take into account data diversity. We need to stop thinking that if you just collect a ton of raw data, that is going to get you somewhere. We need to be very careful about how we design datasets in the first place,” says Xavier Boix, a research scientist in the Department of Brain and Cognitive Sciences (BCS) and the Center for Brains, Minds, and Machines (CBMM), and senior author of the paper.  

Co-authors include former MIT graduate students Timothy Henry, Jamell Dozier, Helen Ho, Nishchal Bhandari, and Spandan Madan, a corresponding author who is currently pursuing a PhD at Harvard; Tomotake Sasaki, a former visiting scientist now a senior researcher at Fujitsu Research; Frédo Durand, a professor of electrical engineering and computer science at MIT and a member of the Computer Science and Artificial Intelligence Laboratory; and Hanspeter Pfister, the An Wang Professor of Computer Science at the Harvard School of Enginering and Applied Sciences. The research appears today in Nature Machine Intelligence.

Thinking like a neuroscientist

Boix and his colleagues approached the problem of dataset bias by thinking like neuroscientists. In neuroscience, Boix explains, it is common to use controlled datasets in experiments, meaning a dataset in which the researchers know as much as possible about the information it contains.

The team built datasets that contained images of different objects in varied poses, and carefully controlled the combinations so some datasets had more diversity than others. In this case, a dataset had less diversity if it contains more images that show objects from only one viewpoint. A more diverse dataset had more images showing objects from multiple viewpoints. Each dataset contained the same number of images.

The researchers used these carefully constructed datasets to train a neural network for image classification, and then studied how well it was able to identify objects from viewpoints the network did not see during training (known as an out-of-distribution combination). 

For example, if researchers are training a model to classify cars in images, they want the model to learn what different cars look like. But if every Ford Thunderbird in the training dataset is shown from the front, when the trained model is given an image of a Ford Thunderbird shot from the side, it may misclassify it, even if it was trained on millions of car photos.

The researchers found that if the dataset is more diverse — if more images show objects from different viewpoints — the network is better able to generalize to new images or viewpoints. Data diversity is key to overcoming bias, Boix says.

“But it is not like more data diversity is always better; there is a tension here. When the neural network gets better at recognizing new things it hasn’t seen, then it will become harder for it to recognize things it has already seen,” he says.

Testing training methods

The researchers also studied methods for training the neural network.

In machine learning, it is common to train a network to perform multiple tasks at the same time. The idea is that if a relationship exists between the tasks, the network will learn to perform each one better if it learns them together.

But the researchers found the opposite to be true — a model trained separately for each task was able to overcome bias far better than a model trained for both tasks together.

“The results were really striking. In fact, the first time we did this experiment, we thought it was a bug. It took us several weeks to realize it was a real result because it was so unexpected,” he says.

They dove deeper inside the neural networks to understand why this occurs.

They found that neuron specialization seems to play a major role. When the neural network is trained to recognize objects in images, it appears that two types of neurons emerge — one that specializes in recognizing the object category and another that specializes in recognizing the viewpoint.

When the network is trained to perform tasks separately, those specialized neurons are more prominent, Boix explains. But if a network is trained to do both tasks simultaneously, some neurons become diluted and don’t specialize for one task. These unspecialized neurons are more likely to get confused, he says.

“But the next question now is, how did these neurons get there? You train the neural network and they emerge from the learning process. No one told the network to include these types of neurons in its architecture. That is the fascinating thing,” he says.

That is one area the researchers hope to explore with future work. They want to see if they can force a neural network to develop neurons with this specialization. They also want to apply their approach to more complex tasks, such as objects with complicated textures or varied illuminations.

Boix is encouraged that a neural network can learn to overcome bias, and he is hopeful their work can inspire others to be more thoughtful about the datasets they are using in AI applications.

This work was supported, in part, by the National Science Foundation, a Google Faculty Research Award, the Toyota Research Institute, the Center for Brains, Minds, and Machines, Fujitsu Research, and the MIT-Sensetime Alliance on Artificial Intelligence.

Read More

Toward a stronger defense of personal data

A heart attack patient, recently discharged from the hospital, is using a smartwatch to help monitor his electrocardiogram signals. The smartwatch may seem secure, but the neural network processing that health information is using private data that could still be stolen by a malicious agent through a side-channel attack.

A side-channel attack seeks to gather secret information by indirectly exploiting a system or its hardware. In one type of side-channel attack, a savvy hacker could monitor fluctuations in the device’s power consumption while the neural network is operating to extract protected information that “leaks” out of the device.

“In the movies, when people want to open locked safes, they listen to the clicks of the lock as they turn it. That reveals that probably turning the lock in this direction will help them proceed further. That is what a side-channel attack is. It is just exploiting unintended information and using it to predict what is going on inside the device,” says Saurav Maji, a graduate student in MIT’s Department of Electrical Engineering and Computer Science (EECS) and lead author of a paper that tackles this issue.

Current methods that can prevent some side-channel attacks are notoriously power-intensive, so they often aren’t feasible for internet-of-things (IoT) devices like smartwatches, which rely on lower-power computation.

Now, Maji and his collaborators have built an integrated circuit chip that can defend against power side-channel attacks while using much less energy than a common security technique. The chip, smaller than a thumbnail, could be incorporated into a smartwatch, smartphone, or tablet to perform secure machine learning computations on sensor values.

“The goal of this project is to build an integrated circuit that does machine learning on the edge, so that it is still low-power but can protect against these side channel attacks so we don’t lose the privacy of these models,” says Anantha Chandrakasan, the dean of the MIT School of Engineering, Vannevar Bush Professor of Electrical Engineering and Computer Science, and senior author of the paper. “People have not paid much attention to security of these machine-learning algorithms, and this proposed hardware is effectively addressing this space.”

Co-authors include Utsav Banerjee, a former EECS graduate student who is now an assistant professor in the Department of Electronic Systems Engineering at the Indian Institute of Science, and Samuel Fuller, an MIT visiting scientist and distinguished research scientist at Analog Devices. The research is being presented at the International Solid-States Circuit Conference.

Computing at random

The chip the team developed is based on a special type of computation known as threshold computing. Rather than having a neural network operate on actual data, the data are first split into unique, random components. The network operates on those random components individually, in a random order, before accumulating the final result.

Using this method, the information leakage from the device is random every time, so it does not reveal any actual side-channel information, Maji says. But this approach is more computationally expensive since the neural network now must run more operations, and it also requires more memory to store the jumbled information.

So, the researchers optimized the process by using a function that reduces the amount of multiplication the neural network needs to process data, which slashes the required computing power. They also protect the neutral network itself by encrypting the model’s parameters. By grouping the parameters in chunks before encrypting them, they provide more security while reducing the amount of memory needed on the chip.

“By using this special function, we can perform this operation while skipping some steps with lesser impacts, which allows us to reduce the overhead. We can reduce the cost, but it comes with other costs in terms of neural network accuracy. So, we have to make a judicious choice of the algorithm and architectures that we choose,” Maji says.

Existing secure computation methods like homomorphic encryption offer strong security guarantees, but they incur huge overheads in area and power, which limits their use in many applications. The researchers’ proposed method, which aims to provide the same type of security, was able to achieve three orders of magnitude lower energy use. By streamlining the chip architecture, the researchers were also able to use less space on a silicon chip than similar security hardware, an important factor when implementing a chip on personal-sized devices.

“Security matters”

While providing significant security against power side-channel attacks, the researchers’ chip requires 5.5 times more power and 1.6 times more silicon area than a baseline insecure implementation.

“We’re at the point where security matters. We have to be willing to trade off some amount of energy consumption to make a more secure computation. This is not a free lunch. Future research could focus on how to reduce the amount of overhead in order to make this computation more secure,” Chandrakasan says.

They compared their chip to a default implementation which had no security hardware. In the default implementation, they were able to recover hidden information after collecting about 1,000 power waveforms (representations of power usage over time) from the device. With the new hardware, even after collecting 2 million waveforms, they still could not recover the data.

They also tested their chip with biomedical signal data to ensure it would work in a real-world implementation. The chip is flexible and can be programmed to any signal a user wants to analyze, Maji explains.

“Security adds a new dimension to the design of IoT nodes, on top of designing for performance, power, and energy consumption. This ASIC [application-specific integrated circuit] nicely demonstrates that designing for security, in this case by adding a masking scheme, does not need to be seen as an expensive add-on,” says Ingrid Verbauwhede, a professor in the computer security and industrial cryptography research group of the electrical engineering department at the Catholic University of Leuven, who was not involved with this research. “The authors show that by selecting masking friendly computational units, integrating security during design, even including the randomness generator, a secure neural network accelerator is feasible in the context of an IoT,” she adds.

In the future, the researchers hope to apply their approach to electromagnetic side-channel attacks. These attacks are harder to defend, since a hacker does not need the physical device to collect hidden information.

This work was funded by Analog Devices, Inc. Chip fabrication support was provided by the Taiwan Semiconductor Manufacturing Company University Shuttle Program.

Read More

Robust Routing Using Electrical Flows

Posted by Ali Kemal Sinop and Kostas Kollias, Research Scientists, Google Research

In the world of networks, there are models that can explain observations across a diverse collection of applications. These include simple tasks such as computing the shortest path, which has obvious applications to routing networks but also applies in biology, e.g., where the slime mold Physarum is able to find shortest paths in mazes. Another example is Braess’s paradox — the observation that adding resources to a network can have an effect opposite to the one expected — which manifests not only in road networks but also in mechanical and electrical systems. For instance, constructing a new road can increase traffic congestion or adding a new link in an electrical circuit can increase voltage. Such connections between electrical circuits and other types of networks have been exploited for various tasks, such as partitioning networks, and routing flows.

In “Robust Routing Using Electrical Flows”, which won the Best Paper Award at SIGSPATIAL 2021, we present another interesting application of electrical flows in the context of road network routing. Specifically, we utilize ideas from electrical flows for the problem of constructing multiple alternate routes between a given source and destination. Alternate routes are important for many use cases, including finding routes that best match user preferences and for robust routing, e.g., routing that guarantees finding a good path in the presence of traffic jams. Along the way, we also describe how to quickly model electrical flows on road networks.

Existing Approaches to Alternate Routing
Computing alternate routes on road networks is a relatively new area of research and most techniques rely on one of two main templates: the penalty method and the plateau method. In the former, alternate routes are iteratively computed by running a shortest path algorithm and then, in subsequent runs, adding a penalty to those segments already included in the shortest paths that have been computed, to encourage further exploration. In the latter, two shortest path trees are built simultaneously, one starting from the origin and one from the destination, which are used to identify sequences of road segments that are common to both trees. Each such common sequence (which are expected to be important arterial streets for example) is then treated as a visit point on the way from the origin to the destination, thus potentially producing an alternate route. The penalty method is known to produce results of high quality (i.e., average travel time, diversity and robustness of the returned set of alternate routes) but is very slow in practice, whereas the plateau method is much faster but results in lower quality solutions.

An Alternate to Alternate Routing: Electrical Flows
Our approach is different and assumes that a routing problem on a road network is in many ways analogous to the flow of electrical current through a resistor network. Though the electrical current travels through many different paths, it is weaker along paths of higher resistance and stronger on low resistance ones, all else being equal.

We view the road network as a graph, where intersections are nodes and roads are edges. Our method then models the graph as an electrical circuit by replacing the edges with resistors, whose resistances equal the road traversal time, and then connecting a battery to the origin and destination, which results in electrical current between those two points. In this analogy, the resistance models how time-consuming it is to traverse a segment. In this sense, long and congested segments have high resistances. Intuitively speaking, the flow of electrical current will be spread around the entire network but concentrated on the routes that have lower resistance, which correspond to faster routes. By identifying the primary routes taken by the current, we can construct a viable set of alternates from origin to destination.

Example of how we construct the electrical circuit corresponding to the road network. The current can be decomposed into three flows, i1, i2 and i3; each of which corresponds to a viable alternate path from Fremont to San Rafael.

In order to compute the electrical flow, we use Kirchhoff’s and Ohm’s laws, which say respectively: 1) the algebraic sum of currents at each junction is equal to zero, meaning that the traffic that enters any intersection also exits it (for instance if three cars enter an intersection from one street and another car enters the same intersection from another street, a total of four cars need to exit the intersection); and 2) the current is directly proportional to the voltage difference between endpoints. If we write down the resulting equations, we end up with a linear system with n equations over n variables, which correspond to the potentials (i.e, the voltage) at each intersection. While voltage has no direct analogy to road networks, it can be used to help compute the flow of electrical current and thus find alternate routes as described above.

In order to find the electrical current i (or flow) on each wire, we can use Kirchhoff’s law and Ohm’s law to obtain a linear system of equations in terms of voltages (or potentials) v. This yields a linear system with three equations (representing Kirchhoff’s law) and three unknowns (voltages at each intersection).

So the computation boils down to computing values for the variables of this linear system involving a very special matrix called Laplacian matrix. Such matrices have many useful properties, e.g., they are symmetric and sparse — the number of off-diagonal non-zero entries is equal to twice the number of edges. Even though there are many existing near-linear time solvers for such systems of linear equations, they are still too slow for the purposes of quickly responding to routing requests with low latency. Thus we devised a new algorithm that solves these linear systems much faster for the special case of road networks1.

Fast Electrical Flow Computation
The first key part of this new algorithm involves Gaussian elimination, which is possibly the most well-known method for solving linear systems. When performed on a Laplacian matrix corresponding to some resistor network, it corresponds to the Y-Δ transformation, which reduces the number of nodes, while preserving the voltages. The only downside is that the number of edges may increase, which would make the linear system even slower to solve. For example, if a node with 10 connections is eliminated using the Y-Δ transformation, the system would end up with 35 new connections!

The Y-Δ transformation allows us to remove the middle junction and replace it with three connections (Ra, Rb and Rc) between N1, N2 and N3. (Image from Wikipedia)

However if one can identify parts of the network that are connected to the rest through very few nodes (lets call these connections bottlenecks), and perform elimination on everything else while leaving the bottleneck nodes, the new edges formed at the end will only be between bottleneck nodes. Provided that the number of bottleneck nodes is much smaller than the number of nodes eliminated with Y-Δ — which is true in the case of road networks since bottleneck nodes, such as bridges and tunnels, are much less common than regular intersections — this will result in a large net decrease (e.g., ~100x) in terms of graph size. Fortunately, identifying such bottlenecks in road networks can be done easily by partitioning such a network. By applying Y-Δ transformation to all nodes except the bottlenecks2, the result is a much smaller graph for which the voltages can be solved faster.

But what about computing the currents on the rest of the network, which is not made up of bottleneck nodes? A useful property about electrical flows is that once the voltages on bottleneck nodes are known, one can easily compute the electrical flow for the rest of the network. The electrical flow inside a part of the network only depends on the voltage of bottleneck nodes that separate that part from the rest of the network. In fact, it’s possible to precompute a small matrix so that one can recover the electrical flow by a single matrix-vector multiplication, which is a very fast operation that can be run in parallel.

Consider the imposed conceptual road network on Staten Island (left), for which directly computing the electrical flow would be slow. The bridges (red nodes) are the bottleneck points, and we can eliminate the whole road network inside the island by repeatedly applying Gaussian Elimination (or Y-Δ transformation). The resulting network (middle) is a much smaller graph, which allows for faster computation. The potentials inside the eliminated part are always a fixed linear combination of the bottleneck nodes (right).

Once we obtain a solution that gives the electrical flow in our model network, we can observe the routes that carry the highest amount of electrical flow and output those as alternate routes for the road network.

Results
Here are some results depicting the alternates computed by the above algorithm.

Different alternates found for the Bay Area. Different colors correspond to different routes from the origin (red icon toward the bottom) to the destination (blue icon toward the top).

Conclusion
In this post we describe a novel approach for computing alternate routes in road networks. Our approach is fundamentally different from the main techniques applied in decades of research in the area and provides high quality alternate routes in road networks by studying the problem through the lens of electrical circuits. This is an approach that can prove very useful in practical systems and we hope inspires more research in the area of alternate route computation and related problems. Interested readers can find a more detailed discussion of this work in our SIGSPATIAL 2021 talk recording.

Acknowledgements
We thank our collaborators Lisa Fawcett, Sreenivas Gollapudi, Ravi Kumar, Andrew Tomkins and Ameya Velingker from Google Research.

1Our techniques work for any network that can be broken down to smaller components with the removal of a few nodes. ↩
2 Performing Y-Δ transformation one-by-one for each node will be too slow. Instead we eliminate whole groups of nodes by taking advantage of the algebraic properties of Y-Δ transformation. ↩

Read More

Guinness World Record Awarded for Fastest DNA Sequencing — Just 5 Hours

Guinness World Records this week presented a Stanford University-led research team with the first record for fastest DNA sequencing technique — a benchmark set using a workflow sped up by AI and accelerated computing.

Achieved in five hours and two minutes, the DNA sequencing record can allow clinicians to take a blood draw from a critical-care patient and reach a genetic disorder diagnosis the same day. The recognition was awarded by a Guinness World Records adjudicator Wednesday at Stanford University’s Jen-Hsun Huang Engineering Center, named for NVIDIA’s founder and CEO, a Stanford alumnus.

The landmark study behind the world record was led by Dr. Euan Ashley, professor of medicine, of genetics and of biomedical data science at the Stanford School of Medicine. Collaborators include researchers from Stanford, NVIDIA, Oxford Nanopore Technologies, Google, Baylor College of Medicine and the University of California at Santa Cruz.

Caption: An adjudicator from Guinness World Records presented the record to the project’s collaborators this week. Image credit: Steve Fisch, courtesy of Stanford University.

“I think we are in unanimous agreement that this is nothing short of a miracle,” said Kimberly Powell, vice president of healthcare at NVIDIA, at the event. “This is an achievement that did go down in the history books, and will inspire another five and 10 years of fantastic work in the digital biology revolution, in which genomics is driving at the forefront.”

Diagnosing With a Genome in Record Time

The researchers achieved the record speed by optimizing every stage of the sequencing workflow. They used high-throughput nanopore sequencing on Oxford Nanopore’s PromethION Flow Cells to generate more than 100 gigabases of data per hour, and accelerated base calling and variant calling using NVIDIA GPUs on Google Cloud. A gigabase is one billion nucleotides.

“These innovations don’t come from one individual, or even one team,” said Greg Corrado, distinguished scientist at Google Research, at the event. “It really takes this group of people coming together to solve these problems.”

To accelerate every step — from Oxford Nanopore’s AI base calling to variant calling, where scientists identify the millions of variants in a genome — the researchers relied on the NVIDIA Clara Parabricks computational genomics application framework. They used a GPU-accelerated version of PEPPER-Margin-DeepVariant, a pipeline developed by Google and UC Santa Cruz’s Computational Genomics Laboratory.

“I believe that the innovations that we’ll see in biology and medicine in the coming century are going to depend on this kind of collaboration much more than the siloed R&D centers of the past,” Corrado said.

New Possibilities for Patient Care

Ultra-rapid genome sequencing isn’t about setting world records. Cutting down the turnaround for a genetic diagnosis from a couple weeks to just a few hours can provide doctors with rapid answers needed to treat critical care patients, where every second counts.

And, as the technology becomes more accessible, more hospitals and research centers will be able to use whole genome sequencing as a critical tool for patient care.

“Genomics is still at the beginning — it’s not the standard of care,” said Powell. “I believe we can help make it part of the standard by reducing the cost and the complexity and democratizing it.”

Not content with the five-hour record, the team is already exploring ways to decrease the DNA sequencing time even further.

“There’s one promise we will make. We will smash this record very quickly in collaboration with Euan and his team, and NVIDIA and Google,” said Gordon Sanghera, CEO of Oxford Nanopore Technologies.

Hear more about this research from Dr. Euan Ashley by registering free for NVIDIA GTC, where he’ll present a talk titled “When Every Second Counts: Accelerated Genome Sequencing for Critical Care” on Tues., March 22 at 2 p.m. Pacific.

Subscribe to NVIDIA healthcare news here.

The post Guinness World Record Awarded for Fastest DNA Sequencing — Just 5 Hours appeared first on The Official NVIDIA Blog.

Read More

Take the Future for a Spin at GTC 2022

Don’t miss the chance to experience the breakthroughs that are driving the future of autonomy.

NVIDIA GTC will bring together the leaders, researchers and developers who are ushering in the era of autonomous vehicles. The virtual conference, running March 21-24, also features experts from industries transformed by AI, such as healthcare, robotics and finance.

And it’s all free to attend.

The conference features a brilliant display of the latest in AI development with the opening keynote on March 22, delivered by NVIDIA CEO and founder Jensen Huang.

The whole week is packed with more than 900 sessions covering autonomous vehicles, AI, supercomputing and more. Conference-goers also have the opportunity to network and learn from in-house experts on the latest in AI and self-driving development.

Here’s a sneak peek of what to expect at GTC next month:

Learn From Luminaries

It seems that nearly every week there’s a new development in the field of AI, but how do these breakthroughs translate to autonomous vehicles?

Hear how industry leaders are harnessing the latest AI innovations to accelerate intelligent transportation — from global automakers and suppliers to startups and researchers.

Automotive session highlights include:

  • Stefan Sicklinger, head of Big Loop and advanced systems division at CARIAD/VW Group, covers the process of leveraging fleet data to develop and improve autonomous driving software at scale.
  • Magnus Östberg, chief software officer of Mercedes-Benz, discusses how the premium automaker is creating software-defined features for the next generation of luxury.
  • Xiaodi Hou, co-founder and CTO of TuSimple, walks through the autonomous trucking startup’s approach to achieving level 4 autonomy.
  • Raquel Urtasun, CEO of Waabi, details how the startup, which recently emerged from stealth, takes an AI-first approach to autonomous driving development.
  • Michael Keckeisen, director of ProAI at ZF Group, outlines the role of supercomputers in developing and deploying safer, more efficient transportation.
  • Developer-focused sessions from Cruise, Luminar, Microsoft, Ouster, Pony.ai, Zoox and more.

Inside NVIDIA DRIVE

Attendees also have the opportunity to get the inside scoop on the latest NVIDIA DRIVE solutions directly from the minds behind them.

NVIDIA DRIVE Developer Days is a series of deep-dive sessions on building safe and robust autonomous vehicles. These sessions are led by the NVIDIA engineering team, which will highlight the newest DRIVE features and discuss how to apply them to autonomous vehicle development.

Topics include:

This virtual content is available to all GTC attendees — register for free today and seize the opportunity to get a firsthand look at the autonomous future.

The post Take the Future for a Spin at GTC 2022 appeared first on The Official NVIDIA Blog.

Read More

Machine Learning for Mechanical Ventilation Control

Posted by Daniel Suo, Software Engineer and Elad Hazan, Research Scientist, Google Research, on behalf of the Google AI Princeton Team

Mechanical ventilators provide critical support for patients who have difficulty breathing or are unable to breathe on their own. They see frequent use in scenarios ranging from routine anesthesia, to neonatal intensive care and life support during the COVID-19 pandemic. A typical ventilator consists of a compressed air source, valves to control the flow of air into and out of the lungs, and a “respiratory circuit” that connects the ventilator to the patient. In some cases, a sedated patient may be connected to the ventilator via a tube inserted through the trachea to their lungs, a process called invasive ventilation.

A mechanical ventilator takes breaths for patients who are not fully capable of doing so on their own. In invasive ventilation, a controllable, compressed air source is connected to a sedated patient via tubing called a respiratory circuit.

In both invasive and non-invasive ventilation, the ventilator follows a clinician-prescribed breathing waveform based on a respiratory measurement from the patient (e.g., airway pressure, tidal volume). In order to prevent harm, this demanding task requires both robustness to differences or changes in patients’ lungs and adherence to the desired waveform. Consequently, ventilators require significant attention from highly-trained clinicians in order to ensure that their performance matches the patients’ needs and that they do not cause lung damage.

Example of a clinician-prescribed breathing waveform (orange) in units of airway pressure and the actual pressure (blue), given some controller algorithm.

In “Machine Learning for Mechanical Ventilation Control”, we present exploratory research into the design of a deep learning–based algorithm to improve medical ventilator control for invasive ventilation. Using signals from an artificial lung, we design a control algorithm that measures airway pressure and computes necessary adjustments to the airflow to better and more consistently match prescribed values. Compared to other approaches, we demonstrate improved robustness and better performance while requiring less manual intervention from clinicians, which suggests that this approach could reduce the likelihood of harm to a patient’s lungs.

Current Methods
Today, ventilators are controlled with methods belonging to the PID family (i.e., Proportional, Integral, Differential), which control a system based on the history of errors between the observed and desired states. A PID controller uses three characteristics for ventilator control: proportion (“P”) — a comparison of the measured and target pressure; integral (“I”) — the sum of previous measurements; and differential (“D”) — the difference between two previous measurements. Variants of PID have been used since the 17th century and today form the basis of many controllers in both industrial (e.g., controlling heat or fluids) and consumer (e.g., controlling espresso pressure) applications.

PID control forms a solid baseline, relying on the sharp reactivity of P control to rapidly increase lung pressure when breathing in and the stability of I control to hold the breath in before exhaling. However, operators must tune the ventilator for specific patients, often repeatedly, to balance the “ringing” of overzealous P control against the ineffectually slow rise in lung pressure of dominant I control.

Current PID methods are prone to over- and then under-shooting their target (ringing). Because patients differ in their physiology and may even change during treatment, highly-trained clinicians must constantly monitor and adjust existing methods to ensure such violent ringing as in the above example does not occur.

To more effectively balance these characteristics, we propose a neural network–based controller to create a set of control signals that are more broad and adaptable than PID-generated controls.

A Machine-Learned Ventilator Controller
While one could tune the coefficients of a PID controller (either manually or via an exhaustive grid search) through a limited number of repeated trials, it is impossible to apply such a direct approach towards a deep controller, as deep neural networks (DNNs) are often parameter-rich and require significant training data. Similarly, popular model-free approaches, such as Q-Learning or Policy Gradient, are data-intensive and therefore unsuitable for the physical system at hand. Further, these approaches don’t take into account the intrinsic differentiability of the ventilator dynamical system, which is deterministic, continuous and contact-free.

We therefore adopt a model-based approach, where we first learn a DNN-based simulator of the ventilator-patient dynamical system. An advantage of learning such a simulator is that it provides a more accurate data-driven alternative to physics-based models, and can be more widely distributed for controller research.

To train a faithful simulator, we built a dataset by exploring the space of controls and the resulting pressures, while balancing against physical safety, e.g., not over-inflating a test lung and causing damage. Though PID control can exhibit ringing behavior, it performs well enough to use as a baseline for generating training data. To safely explore and to faithfully capture the behavior of the system, we use PID controllers with varied control coefficients to generate the control-pressure trajectory data for simulator training. Further, we add random deviations to the PID controllers to capture the dynamics more robustly.

We collect data for training by running mechanical ventilation tasks on a physical test lung using an open-source ventilator designed by Princeton University’s People’s Ventilator Project. We built a ventilator farm housing ten ventilator-lung systems on a server rack, which captures multiple airway resistance and compliance settings that span a spectrum of patient lung conditions, as required for practical applications of ventilator systems.

We use a rack-based ventilator farm (10 ventilators / artificial lungs) to collect training data for a ventilator-lung simulator. Using this simulator, we train a DNN controller that we then validate on the physical ventilator farm.

The true underlying state of the dynamical system is not available to the model directly, but rather only through observations of the airway pressure in the system. In the simulator we model the state of the system at any time as a collection of previous pressure observations and the control actions applied to the system (up to a limited lookback window). These inputs are fed into a DNN that predicts the subsequent pressure in the system. We train this simulator on the control-pressure trajectory data collected through interactions with the test lung.

The performance of the simulator is measured via the sum of deviations of the simulator’s predictions (under self-simulation) from the ground truth.

While it is infeasible to compare real dynamics with their simulated counterparts over all possible trajectories and control inputs, we measure the distance between simulation and the known safe trajectories. We introduce some random exploration around these safe trajectories for robustness.

Having learned an accurate simulator, we then use it to train a DNN-based controller completely offline. This approach allows us to rapidly apply updates during controller training. Furthermore, the differentiable nature of the simulator allows for the stable use of the direct policy gradient, where we analytically compute the gradient of the loss with respect to the DNN parameters.  We find this method to be significantly more efficient than model-free approaches.

Results
To establish a baseline, we run an exhaustive grid of PID controllers for multiple lung settings and select the best performing PID controller as measured by average absolute deviation between the desired pressure waveform and the actual pressure waveform. We compare these to our controllers and provide evidence that our DNN controllers are better performing and more robust.

  1. Breathing waveform tracking performance:

    We compare the best PID controller for a given lung setting against our controller trained on the learned simulator for the same setting. Our learned controller shows a 22% lower mean absolute error (MAE) between target and actual pressure waveforms.

    Comparison of the MAE between target and actual pressure waveforms (lower is better) for the best PID controller (orange) for a given lung setting (shown for two settings, R=5 and R=20) against our controller (blue) trained on the learned simulator for the same setting. The learned controller performs up to 22% better.
  2. Robustness:

    Further, we compare the performance of the single best PID controller across the entire set of lung settings with our controller trained on a set of learned simulators over the same settings. Our controller performs up to 32% better in MAE between target and actual pressure waveforms, suggesting that it could require less manual intervention between patients or even as a patient’s condition changes.

    As above, but comparing the single best PID controller across the entire set of lung settings against our controller trained over the same settings. The learned controller performs up to 32% better, suggesting that it may require less manual intervention.

Finally, we investigated the feasibility of using model-free and other popular RL algorithms (PPO, DQN), in comparison to a direct policy gradient trained on the simulator. We find that the simulator-trained direct policy gradient achieves slightly better scores and does so with a more stable training process that uses orders of magnitude fewer training samples and a significantly smaller hyperparameter search space.

In the simulator, we find that model-free and other popular algorithms (PPO, DQN) perform approximately as well as our method.
However, these other methods take an order of magnitude more episodes to train to similar levels.

Conclusions and the Road Forward
We have described a deep-learning approach to mechanical ventilation based on simulated dynamics learned from a physical test lung. However, this is only the beginning. To make an impact on real-world ventilators there are numerous other considerations and issues to take into account. Most important amongst them are non-invasive ventilators, which are significantly more challenging due to the difficulty of discerning pressure from lungs and mask pressure. Other directions are how to handle spontaneous breathing and coughing. To learn more and become involved in this important intersection of machine learning and health, see an ICML tutorial on control theory and learning, and consider participating in one of our kaggle competitions for creating better ventilator simulators!

Acknowledgements
The primary work was based in the Google AI Princeton lab, in collaboration with Cohen lab at the Mechanical and Aerospace Engineering department at Princeton University. The research paper was authored by contributors from Google and Princeton University, including: Daniel Suo, Naman Agarwal, Wenhan Xia, Xinyi Chen, Udaya Ghai, Alexander Yu, Paula Gradu, Karan Singh, Cyril Zhang, Edgar Minasyan, Julienne LaChance, Tom Zajdel, Manuel Schottdorf, Daniel Cohen, and Elad Hazan.

Read More

The Balloon Learning Environment

Posted by Joshua Greaves, Software Engineer and Pablo Samuel Castro, Staff Software Engineer, Google Research, Brain Team

Benchmark challenges have been a driving force in the advancement of machine learning (ML). In particular, difficult benchmark environments for reinforcement learning (RL) have been crucial for the rapid progress of the field by challenging researchers to overcome increasingly difficult tasks. The Arcade Learning Environment, Mujoco, and others have been used to push the envelope in RL algorithms, representation learning, exploration, and more.

In “Autonomous Navigation of Stratospheric Balloons Using Reinforcement Learning”, published in Nature, we demonstrated how deep RL can be used to create a high-performing flight agent that can control stratospheric balloons in the real world. This research confirmed that deep RL can be successfully applied outside of simulated environments, and contributed practical knowledge for integrating RL algorithms with complex dynamical systems. Today we are excited to announce the open-source release of the Balloon Learning Environment (BLE), a new benchmark emulating the real-world problem of controlling stratospheric balloons. The BLE is a high-fidelity simulator, which we hope will provide researchers with a valuable resource for deep RL research.

Station-Keeping Stratospheric Balloons
Stratospheric balloons are filled with a buoyant gas that allows them to float for weeks or months at a time in the stratosphere, about twice as high as a passenger plane’s cruising altitude. Though there are many potential variations of stratospheric balloons, the kind emulated in the BLE are equipped with solar panels and batteries, which allow them to adjust their altitude by controlling the weight of air in their ballast using an electric pump. However, they have no means to propel themselves laterally, which means that they are subject to wind patterns in the air around them.

By changing its altitude, a stratospheric balloon can surf winds moving in different directions.

The goal of an agent in the BLE is to station-keep — i.e., to control a balloon to stay within 50km of a fixed ground station — by changing its altitude to catch winds that it finds favorable. We measure how successful an agent is at station-keeping by measuring the fraction of time the balloon is within the specified radius, denoted TWR50 (i.e., the time within a radius of 50km).

A station-seeking balloon must navigate a changing wind field to stay above a ground station. Left: Side elevation of a station-keeping balloon. Right: Birds-eye-view of the same balloon.

The Challenges of Station-Keeping
To create a realistic simulator (without including copious amounts of historical wind data), the BLE uses a variational autoencoder (VAE) trained on historical data to generate wind forecasts that match the characteristics of real winds. A wind noise model is then used to make the windfields more realistic to match what a balloon would encounter in real-world conditions.

Navigating a stratospheric balloon through a wind field can be quite challenging. The winds at any given altitude rarely remain ideal for long, and a good balloon controller will need to move up and down through its wind column to discover more suitable winds. In RL parlance, the problem of station-keeping is partially observable because the agent only has access to forecasted wind data to make those decisions. An agent has access to wind forecasts at every altitude and the true wind at its current altitude. The BLE returns an observation which includes a notion of wind uncertainty.

A stratospheric balloon must explore winds at different altitudes in order to find favorable winds. The observation returned by the BLE includes wind predictions and a measure of uncertainty, made by mixing a wind forecast and winds measured at the balloon’s altitude.

In some situations, there may not be suitable winds anywhere in the balloon’s wind column. In this case, an expert agent is still able to fly towards the station by taking a more circuitous route through the wind field (a common example is when the balloon moves in a zig-zag fashion, akin to tacking on a sailboat). Below we demonstrate that even just remaining in range of the station usually requires significant acrobatics.

An agent must handle long planning horizons to succeed in station-keeping. In this case, StationSeeker (an expert-designed controller) heads directly to the center of the station-keeping area and is pushed out, while Perciatelli44 (an RL agent) is able to plan ahead and stay in range longer by hugging the edge of the area.

Night-time adds a fresh element of difficulty to station-keeping in the BLE, which reflects the reality of night-time changes in physical conditions and power availability. While during the day the air pump is powered by solar panels, at night the balloon relies on its on-board batteries for energy. Using too much power early in the night typically results in limited maneuverability in the hours preceding dawn. This is where RL agents can discover quite creative solutions — such as reducing altitude in the afternoon in order to store potential energy.

An agent needs to balance the station-keeping objective with a finite energy allowance at night.

Despite all these challenges, our research demonstrates that agents trained with reinforcement learning can learn to perform better than expert-designed controllers at station-keeping. Along with the BLE, we are releasing the main agents from our research: Perciatelli44 (an RL agent) and StationSeeker (an expert-designed controller). The BLE can be used with any reinforcement learning library, and to showcase this we include Dopamine’s DQN and QR-DQN agents, as well as Acme’s QR-DQN agent (supporting both standalone and distributed training with Launchpad).

Evaluation performance by the included benchmark agents on the BLE. “Finetuned” is a fine-tuned Perciatelli44 agent, and Acme is a QR-DQN agent trained with the Acme library.

The BLE source code contains information on how to get started with the BLE, including training and evaluating agents, documentation on the various components of the simulator, and example code. It also includes the historical windfield data (as a TensorFlow DataSet) used to train the VAE to allow researchers to experiment with their own models for windfield generation. We are excited to see the progress that the community will make on this benchmark.

Acknowledgements
We would like to thank the Balloon Learning Environment team: Sal Candido, Marc G. Bellemare, Vincent Dumoulin, Ross Goroshin, and Sam Ponda. We’d also like to thank Tom Small for his excellent animation in this blog post and graphic design help, along with our colleagues, Bradley Rhodes, Daniel Eisenberg, Piotr Staczyk, Anton Raichuk, Nikola Momchev, Geoff Hinton, Hugo Larochelle, and the rest of the Google Brain team in Montreal.

Read More