The new robot brings together advances in computer vision, speech recognition and force control to achieve a wide range of advanced capabilities.

1. Recognition of spoken commands, such as an order to pick up balls according to their color from a group of colored balls.
2. The ability to follow the trajectory of a moving ball and then hit and return it.
3. Recognition of individuals by using pattern recognition to compare people against stored image data.
4. Identify actions, such as indicating an individual waving his hands in the midst of a group of people.
5. Shake hands with person, with a movement that corresponds to that of the person.

Development background

The 21^st century will see robots not only programmed to act in certain ways, but able to respond to their environment, including working with and around people. Such robots will perform sophisticated tasks, including taking care of patients or executing complicated manufacturing processes, on the basis of simple instructions.

Currently, robots, and the objects they work on, must be precisely positioned, and robots require detailed programming to control their movements and actions. All this severely limits repertoire and usefulness.

Toshiba's new robot demonstrates solution to these problems that will bring greater flexibility and capabilities to robots. The prototype integrates a wide range of technologies, including high-speed image processing, speech recognition, face recognition, and force control. It will be demonstrated at Tomorrow 21, an exhibition of Toshiba's technologies, at Tokyo International Forum, next to Yurakucho Station, from March 4 to 9.

1. Two compact cameras are installed in the robot's head. Image data is processed and the 3-D position of a ball is computed at 60Hz by extracting only the specified color region in the image, in this case, the color of the ball. This capability allows the robot to identify and pick up the ball, to predict its trajectory when it is played to the robot, and to return it.

2. The robot detects motion in the image, analyzes the motion, and identifies only the registered motion, such as waving hands.

3. The robot is able to detect faces in the image, and identify registered individuals.

4. Speech recognition technology allows the robot to recognize and respond to spoken commands by analyzing the command and recognizing its frequency pattern.

5. The robot has a force sensor at the wrist and precisely controls force of each joint, the shoulder, elbow, and wrist. This enables the robot to shake hands with people naturally.