I2R Researchers from ASTAR Singapore and UM-CNRS LIRMM in France have recently developed a framework that improves the ability of robots to identify objects around them that could be potential tools and then use them to complete manual tasks, even if they have never encountered them before these objects. Their method, published in a paper in the journal Nature Machine Intelligence, can significantly improve the ability of the robotic system to complete challenging tasks that may require tools, without any prior tool training. "Humans are amazing at recognizing random objects in the environment as potential tools and using them as potential tools," Ganesh Gowrishankar, one of the researchers who conducted the study, told TechXplore. "Similar capabilities are useful for robots, allowing them to be innovative in "unstructured" (i.e., unmodelled and unpredictable) environments. For example, imagine a rescue robot capable of independent (without human help) in a disaster scenario ) solve the task and use the available fragments as tools to pass obstacles. ”
While many past studies in the field of robotics have highlighted the great potential of systems that can use tools to complete physical tasks, all methods proposed so far require prior use of tools. training. This is achieved in simulations, using videos of humans or other robots using tools, or using tools directly in the physical world.
On the other hand, the goal of the work by Keng Peng Tee, Gowrishankar and their colleagues was to create a framework that would allow robots to identify potentially useful tools on the fly based on their shape and size, even if they have never encountered these objects before. Never trained on how to use these objects or any other tools. Researchers have been working on this framework for several years.
Gowrishankar said: "In our new paper, we combine past work on human tool use, avatars, and human tool representation to develop a cognitive framework that enables robots with no tool experience to transform objects, even recognize objects they see for the first time as task-specific tools and use them immediately—just like humans."
While conducting previous research and reviewing existing literature, Tee and Gowrishankar found that humans may use the shape of their own hands or arms and the movements they make with their hands/arms as a reference to determine whether a tool can be used to complete a specific task. . The new framework they developed is based on this idea, as it encourages robots to use their limbs to determine whether objects around them can be used as tools.
The hand/arm-like shape of the tools and the movements we need to complete tasks with these tools are similar to the movements we perform with our hands while working on the same task
"Our framework leverages our findings to enable robots to intuitively use tools, "Tee explained. "Specifically, it enables a robot to isolate the 'functional' features of its own limbs that enable it to perform a task, use those features to identify objects as potential tools for the same task, and then use the skills the robot already possesses (the controller ) to develop successful actions using the tool."
Essentially, the researchers' framework allows robots to use tools to accomplish any task they are able to accomplish without tools (i.e., any task for which they have a so-called "controller" Task). It simply requires the robot to integrate cameras or sensors that allow it to "visually" perceive objects around it.
The new framework developed by Tee, Gowrishankar and colleagues can be used to enhance the capabilities of existing and newly developed robots, allowing them to better utilize the objects around them when completing specific tasks. However, the potential of this framework is currently limited as accurate visual perception of robots remains a technical challenge.
"In its current form, our framework is entirely based on visual perception," Tee said."It therefore applies to tools that enable 'motor augmentation', that is, an expansion of limb shape and size (that can be perceived visually). These tools include a large (perhaps most) set of tools we use in our daily lives: Spoons , rakes, tongs, plates, even chairs (for climbing), etc.”
While the researchers’ framework covers many simple tools we use in our daily lives, in its current form, it does not allow robots to recognize and use them. Tools such as hammers or levers that enable what is called "dynamic/force augmentation". The main reason is that identifying these tools requires not only looking at their shape and size, but also understanding their weight and dynamic characteristics.
Therefore, in order to recognize whether these tools are useful when provided, the robot needs a further layer of perception that enables it to relate the visual/physical characteristics of the object to its dynamic characteristics. To achieve this goal, Gowrishankar and colleagues are now planning to further develop their framework.
"In future research, we hope to extend our framework to support the use of dynamic tools," Gowrishankar added. "We also hope to combine our framework with learning techniques proposed by other tool usage studies. This is necessary to enable robots to use tools for tasks for which they do not have a controller (a problem not addressed by our framework), thereby Tool usage capabilities that bring robots closer to humans”
.
