MIT and NVIDIA researchers have developed a framework that allows users to correct the actions of a robot in real time, using comments similar to those used in human interactions.
The frame was designed to help robots to perform tasks that align with the intention of the user without the need for additional data collection or recycling of automatic learning models. Instead, users can guide the robot using intuitive interactions, such as pointing to objects, tracing a trajectory on a screen or physically pushing the robot arm.
“We cannot expect the lay people to collect data and refine a neural network model. The consumer will expect the robot to work out of the box, and if he did not want, he would like an intuitive mechanism to personalize it. This is the challenge in electrical and computer engineering (EECS) Graduate Student and Lead Author.
Wang’s co-authors include Lirui Wang Phd ’24, Yilun du Phd ’24, and military personalities such as Julie Shah, Mit in aeronautics and astronautics and director of the Interactive Robotics Group at the Balakumar of Computer Science and Artificial (CSAIL), with Yuning Yang Chao, Claudia Perez-D’Arpino ’19 and Dieter Fox.
Research, which will be presented at the international conference on robotics and automation, highlights the capacity of the framework to provide users with a more accessible way to correct the actions of poorly aligned robots of their expectations.
By allowing human users to correct the behavior of the robot without inadvertently causing new errors inadvertently, the team aimed to create actions that are aligned with the intention of the user and achievable in execution. Wang explains: “We want to allow the user to interact with the robot without introducing this type of error, so we get much more aligned behavior with the intention of the user during deployment, but which is also valid and achievable.”
The frame includes three interaction methods: pointing out an object in the robot’s view interface, tracing a trajectory or physically move the robot arm. According to Wang, “when you map a 2D image of the environment in actions in a 3D space, some information is lost. Physically nudging The robot is the most direct way to specify the intention of the user without losing any information.”
To alleviate the risk of non -valid shares, a specific sampling procedure is used. This method guarantees that the robot selects the actions that best correspond to the user’s objectives from a valid set. “Rather than imposing the user’s desire, we give the robot an idea of what the user has the intention, but leave the sampling procedure oscillating around their own set of learned behaviors,” adds Wang.
The research presents an improved success rate of the completion of tasks of 21% compared to the alternatives which do not integrate human intervention, demonstrating improved performance in various simulations and scenarios of the real world involving a robot arm in a kitchen setting.
This progression could potentially lead to more user -friendly robots that adapt to new environments and tasks without in -depth recycling, with the additional advantage which repeated bend of elbows in similar situations could reinforce the learning of the robot for future tasks.
“But the key to this continuous improvement is to have a way for the user to interact with the robot, which we have shown here,” said Wang.
Future research aims to further optimize the speed and performance of the sampling procedure and to explore the generation of robot policies in unknown environments.