An Easier Way To Teach New Skills To Robots


With the influx of online purchases, a warehouse robot pulls mugs from the shelf and packs them into boxes for shipment. When the warehouse processes a modification and the robot must now grip taller, narrower faces that are upside down, everything is running well.

To reprogram that robot, you’ll need to manually label and train the system on thousands of photographs that demonstrate the robot how to hold these new cups.

On the other hand, MIT researchers have devised a novel approach that only calls only a small number of human demonstrations to reprogram the robot. Using a machine-learning technique, a robot can pick up and set things that it has never seen before in random positions. In around 10 to 15 minutes, the robot would be ready for a new pick-and-place operation.

In a neural network intended for 3D object reconstruction, the method works. By showing a few examples, the neural network learns about 3D geometry to identify new things that are comparable.

Robotic systems may be trained with only ten examples, demonstrating the effectiveness of the researchers’ system in manipulating previously unseen items such as cup and bowl sets and bottles that are placed in a variety of positions.

1. Geometry May Be Grasped

While a robot may have been taught to pick up a certain thing, if that object is lying on its side (perhaps because it tumbled over), the robot will perceive this as an entirely different situation altogether. As a result, generalization to new object orientations is difficult for machine learning systems.

The researchers developed a novel sort of neural network model, called a Neural Descriptor Field (PDF), to address this problem. A 3D point cloud, a collection of data points or coordinates in three dimensions, is used to generate the model’s geometric representation of a particular item.

Data points can be collected from a depth camera that offers information about the distance between the item and a perspective. Simulated 3D forms may be used to train the network, which can then identify real-world items.

Equivariance is a characteristic of the NDF that was designed by the team. As a result of this attribute, when the model sees an image of a cup of coffee that has been flipped on its side, it recognizes the cup as the same item.

To help the NDF learn how to reassemble the forms of similar things, it stores information on the parts with each shape. For example, even though some mugs are taller or broader or have smaller or longer handles, it learns that the handles of mugs are comparable.

To achieve this using a different technique, you would have to identify each part by hand. Our technique automatically detects these elements from the form reconstruction,” Du explains.

2. Determining The Victor

They put their design to the test using cups, bowls, and bottles as test objects on an accurate robotic arm and simulations. On pick-and-place tasks with new things in new orientations, their strategy achieved a success rate of 85 percent, compared to the best baseline of 45 percent. Putting mugs on a rack is an excellent example of a triumphant accomplishment.

These algorithms are more challenging to include equivariance if they employ 2D picture information rather than 3D geometry. One of the reasons NDF did so well was because of this.

Moreover, researchers were pleased with its results, and the approach is limited to the specific item category on which it is trained. If a robot has been trained to pick up mugs, it won’t be able to pick up boxes or headphones since the geometric properties of these things are too different from what the network was taught.

It would be wonderful if the concept of categories were abolished altogether in the future, Simeonov believes. Non-rigid items may also be included in the future, and a pick-and-place capability when the target region changes.

The Defense Advanced Research Projects Agency, the Singapore Defense Science and Technology Agency, and the National Science Foundation have provided some funding for this research.