Before a robot arm can reach into a tight space or pick up a delicate object, the robot needs to know precisely where its hand is. Researchers at Carnegie Mellon University’s Robotics Institute have shown that a camera attached to the robot’s hand can rapidly create a 3-D model of its environment and also locate the hand within that 3-D world.

Camera-mounted robot arm
Camera-mounted robot arm (Source: Carnegie Mellon University)

Doing so with imprecise cameras and wobbly arms in real-time is tough, but the CMU team found they could improve the accuracy of the map by incorporating the arm itself as a sensor, using the angle of its joints to better determine the pose of the camera. This would be important for a number of applications, including inspection tasks, said Matthew Klingensmith, a Ph.D. student in robotics.

Placing a camera or other sensor in the hand of a robot has become feasible as sensors have grown smaller and more power-efficient. But an eye in the hand isn’t much good if the robot can’t see its hand and doesn’t know where its hand is relative to objects in its environment. It’s a problem shared with mobile robots that must operate in an unknown environment. A popular solution for mobile robots is called simultaneous localization and mapping, or SLAM, in which the robot pieces together input from sensors such as cameras, laser radars and wheel odometry to create a 3-D map of the new environment and to figure out where the robot is within that 3-D world.

>Read more by Byron Spice, CMU News, May 16, 2016

Robot’s In-Hand Eye Maps Surroundings, Determines Hand’s Location