According to a new study by USC computer scientists, to help robots succeed, you may need to show some harsh love.

In computer-simulated manipulation tasks, the researchers found that training a robot with a human opponent can greatly improve its grip on objects.

Research co-author, Assistant Professor of Computer Science Stefanos Nikolaidis said: "This is the first robot learning using adversarial human users.

"Think of it as a sport: if you play tennis with someone who always allows you to win, you won’t get better. Same as robots. If we want them to learn maneuvers such as grip Task, then they can help people, we need to challenge them."

The research titled "Robot Learning Through Human Confrontation Games" was published at the International Conference on Intelligent Robots and Systems on November 4. The students of University of Southern California Jiali Duan and Qian Wang are the lead authors, under the guidance of Professor CC Jay Kuo, and collaborator Lerrel Pinto of Carnegie Mellon University.

 Learn from practice

Nikolaidis joined the University of Southern California Viterbi School of Engineering in 2018. He and his team use reinforcement learning, an artificial intelligence program that "learns" through repeated experiments.

The robotic system is not limited to completing small-scale repetitive tasks such as industrial robots. It also "learns" based on previous examples, theoretically increasing the range of tasks it can perform.

However, the challenge of creating a universal robot is very daunting, partly because of the need for training. Robot systems need to look at a large number of examples to learn how to manipulate objects in a human-like manner.

For example, OpenAI's impressive robotic system has learned to solve the Rubik's Cube with humanoid hands, but it takes 10,000 years of simulation training to learn to operate the Rubik's Cube.

More importantly, the flexibility of the robot is very specific. Without extensive training, it cannot pick up an object, manipulate it with another grip or grasp and handle another object.

"As a person, even if I know the location of an object, I don't know how much it weighs, or how it will move or behave when I pick it up, but we can almost always succeed in doing this."

Click the link to watch the video:

https://mp.weixin.qq.com/s/HKH1Y8ZwQ8AoXkQbH5c1dQ

According to a new study by USC computer scientists, to help robots succeed, you may need to show some harsh love. In computer-simulated manipulation tasks, the researchers found that training the robot with a human opponent can greatly improve its grip on objects.

"This is because people are very intuitive about the behavior of the world, but robots are like newborn babies."

In other words, robotic systems are difficult to promote, which is a natural skill for humans. This may seem trivial, but it can have serious consequences. If auxiliary robotic devices such as gripping robots are to fulfill their promise of helping the disabled, the robotic system must be able to operate reliably in the real environment.

Human cycle

One study that has been very successful in overcoming this problem is the "human cycle". In other words, humans provide feedback to robotic systems by demonstrating their ability to complete tasks.

However, until now, these algorithms have made a strong assumption that a collaborative human supervisor is needed to assist the robot.

Nikoladis said: "I have always been committed to human-machine collaboration, but in fact, people don't always cooperate with robots in the wild."

For example, he pointed to a study conducted by Japanese researchers who let go of a robot in a public shopping mall and observed children "moving violently towards it" many times.

So, Nikoladis thought, what if we use human tendencies to make robots more difficult? What if you don't try to show how to better grasp the object? Through thinking and adding challenges, the system will learn to be more powerful against the complexity of the real world.

Challenge factor

The experiment is like this: In a computer simulation, the robot tries to grab an object. Humans observe the grip of the simulated robot on the computer. If the grasp is successful, the human will try to use the keyboard to indicate the direction, thereby snatching the object from the robot's grasp.

Adding a challenge element can help the robot understand the difference between a weaker grip (for example, placing a bottle on the top) and a firm grip (place it in the middle), which can make it more difficult for the opponent to steal.

Nikolaidis admits that this is a crazy idea, but it does work.

After training with opponents, the robot has stronger grip and it is difficult to snatch objects away

Researchers found that systems trained by human opponents rejected unstable grip and quickly learned about the robust grip of these objects. In an experiment, the model has a success rate of 52% under the control of a human opponent, and 26.5% under the control of a human collaborator.

Nikoladis said: "The robot not only learned how to grasp objects more firmly, but also learned to successfully use new objects in different directions because it learned a more stable grasp."

They also found that models trained by human opponents performed better than simulated opponents, which had a 28% success rate. Therefore, the robot system can learn the best things from flesh and blood opponents.

Nikoladis explained: "This is because humans understand stability and robustness better than their learned opponents."

"The robot tries to pick up something, and if a human tries to destroy something, it will result in a more stable grip. And because it learns a more stable grip, it will succeed more frequently even if the object is in a different position. In other words In other words, it has learned to generalize. This is very important."

Find balance

Nikolaidis hopes to make the system work on a real robot arm within a year. This will present a new challenge-in the real world, a little bit of friction or noise in the robot's joints will throw things away. But Nikolaidis is full of hope for the future of robot adversarial learning.

Nikoladis said: "I think we just explored the potential applications of learning through adversarial human games."

"We are also very happy to explore loop adversarial learning in other tasks, such as avoiding obstacles in robotic arms and mobile robots (such as self-driving cars)."

This begs the question: how far are we willing to engage in adversarial learning? Are we willing to kick and beat the robot to yield? Nikolaidis said that the answer lies in finding a difficult balance between love and encouragement with our robots.

Nikoladis said: "In the context of our proposed algorithm, I feel difficult love is like a sport again: it belongs to specific rules and constraints."

"If humans just break the robot's gripper, the robot will continue to fail and never learn. In other words, the robot needs to be challenged, but it can still learn successfully."