RT-2: New model translates vision and language into action

RT-2: New model translates vision and language into action

Introducing Robotic Transformer 2 (RT-2), a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalised instructions for robotic control, while retaining web-scale capabilities. This work builds upon Robotic Transformer 1 (RT-1), a model trained on multi-task demonstrations which can learn combinations of tasks and objects seen in the robotic data. RT-2 shows improved generalisation capabilities and semantic and visual understanding, beyond the robotic data it was exposed to. This includes interpreting new commands and responding to user commands by performing rudimentary reasoning, such as reasoning about object categories or high-level descriptions. Read More

Using AI to fight climate change

Using AI to fight climate change

AI is a powerful technology that will transform our future, so how can we best apply it to help combat climate change and find sustainable solutions? The effects of climate change on Earth’s ecosystems are incredibly complex, and as part of our effort to use AI for solving some of the world’s most challenging problems, here are some of the ways we’re working to advance our understanding, optimise existing systems, and accelerate breakthrough science of climate and its effects.Read More

Exploring institutions for global AI governance

Exploring institutions for global AI governance

New white paper investigates models and functions of international institutions that could help manage opportunities and mitigate risks of advanced AI. Growing awareness of the global impact of advanced artificial intelligence (AI) has inspired public discussions about the need for international governance structures to help manage opportunities and mitigate risks involved. Many discussions have drawn on analogies with the ICAO (International Civil Aviation Organization) in civil aviation; CERN (European Organization for Nuclear Research) in particle physics; IAEA (International Atomic Energy Agency) in nuclear technology, and intergovernmental and multi-stakeholder organisations in many other domains. And yet, while analogies can be a useful start, the technologies emerging from AI will be unlike aviation, particle physics, or nuclear technology. To succeed with AI governance, we need to better understand: what specific benefits and risks we need to manage internationally, what governance functions those benefits and risks require, what organisations can best provide those functions.Read More