And researchers from the Toyota Research Institute, Columbia University and MIT were able to quickly teach robots to perform many new tasks using an AI learning technique called imitation learning, as well as generator. They believe that they have found a way to extend the technology that propels a generative AI in the field of text, images and videos in the field of robot movements.
Many others also took advantage of the generator. Covariant, a robotics startup that has been transferred from the now closed robotics research unit of Openai, built a multimodal model called RFM-1. He can accept prompts in the form of text, image, video, robot instructions or measurements. The generative AI allows the robot to understand the instructions and to generate images or videos relating to these tasks.
3. More data allow robots to learn more skills
The power of large AI models such as GPT-4 are found in the trains and trains resolved on the Internet. But it doesn’t really work for robots, which need data that has been specifically collected for robots. They need physical demonstrations on how washing machines and refrigerators are open, dishes picked up or folded linen. Currently, this data is very rare, and it takes a long time for humans to collect.
A new initiative launched by Google Deepmind, called the Open X-Embodiment Collaboration, aims to change this. Last year, the company has teamed up with 34 research laboratories and around 150 researchers to collect data from 22 different robots, including the hello robot section. The resulting data set, published in October 2023, consists of robots demonstrating 527 skills, such as picking, push and move.
The first signs show that more data lead to smarter robots. The researchers have built two versions of a model for robots, called RT-X, which could be executed locally on individual or accessible laboratory computers via the web. The larger and web accessible model has been pre-trained with Internet data to develop a “visual common sense” or a reference understanding of the world, from large models of language and image. When the researchers directed the RT-X model on many different robots, they discovered that robots were able to learn skills 50% more with success than in systems that each individual laboratory developed.
Deeper learning
Generative AI can transform your most precious memories into photos that have never existed
Maria grew up in Barcelona, Spain, in the 1940s. Her first memories of her father were lively. At the age of six, Maria would visit a neighbor’s apartment in her building when she wanted to see him. From there, she could look through the railing of a balcony in the prison below and try to see it by the small window of his cell, where he was locked up to oppose the dictatorship of Francisco Franco. There is no photo of Maria on this balcony. But it can now hold something like that: a false photo – or a reconstruction based on memory.