The researchers used the system, called Lucidesim, to form a robot dog in the parkour, make it rush on a box and mount the stairs even if it had never seen real data. The approach shows how useful the AI could be useful when it comes to teaching robots to perform difficult tasks. This also raises the possibility that we can finally train them in fully virtual worlds. THE research was presented at the Conference on Robot Learning (Corl) last week.
“We are in the midst of an industrial revolution for robotics,” said Ge Yang, a post-doctorator at the MIT computer and artificial intelligence laboratory, who worked on the project. “This is our attempt to understand the impact of these (AI generative) models outside their original ends, in the hope that this will lead us to the next generation of tools and models.”
Lucidsim uses a combination of generative AI models to create visual training data. The researchers first generated thousands of guests for Chatgpt, which in fact descriptions of a range of environments that represent the conditions that the robot would encounter in the real world, including different types of time, day and lighting time. These included “an old alley bordered by tea houses and small picturesque shops, each displaying traditional ornaments and calligraphy” and “the sun illuminates a somewhat assaulted lawn strewn with dry plates”.
These descriptions were introduced into a system that maps 3D geometry and physics data on images generated by AI, creating short videos mapping a trajectory for the robot to follow. The robot is based on this information to develop the height, the width and the depth of the things he has to navigate – a box or a staircase, for example.
The researchers tested Lucidim by instructing a four -legged robot equipped with a webcam to perform several tasks, including locating a traffic cone or a football ball, climbing on a box and going up and down the stairs. The robot has worked better than when it directed a system formed on traditional simulations. In 20 trials to locate the cone, Lucidsim had a 100% success rate, compared to 70% for systems formed on standard simulations. Likewise, Lucidsim has reached the football ball in 20 other 85% trials of time, and only 35% for the other system.
Finally, when the robot directed Lucidsim, he succeeded in the 10 staircase tests, against only 50% for the other system.
With the kind permission of the MIT CSAIL
These results are likely to improve even more in the future if Lucidsim is inspired directly with models of sophisticated generative video rather than a combination enlightened by models of language, image and physics, explains Phillip Isola, associate professor at MIT who worked on research.
The approach of researchers to use generating AI is a news that will open the way for new more interesting research, explains Mahi Shafiullah, doctoral student at New York University that uses AI models to form robots. He did not work on the project.