This Robot Can Turn a Duck. Can It Tell When It’s Done?

Show this robot a picture, and it turns the object in its hand to match. That is a real rubber duck moving between real robot fingers, and the small image in the footage is the target.

The clever part is how the robot gets there. The catch is how it decides it is done.

Why turning an object is hard

Try turning something in your own hand without putting it down. Every finger movement changes what you can see. Your fingers hide the object, then uncover a different side.

That creates a problem for a robot that first tries to calculate exactly how an object is positioned before deciding how to move it. Tracking an object’s position is hard when the hand keeps covering what it holds.

A picture instead of coordinates

ETH Zurich researchers, working with NVIDIA, tested a different route called GoalDex. The controller receives the current camera image, information about its own joints and a picture of the desired orientation, and turns those inputs into finger movements without a separate object pose estimator.

The request is not simply to hold the object. It is to turn a particular side toward the camera. The controller was trained in simulation and transferred to a real robot hand, and one policy handled five tested objects: a cube, a 3D printed toy, a tablet bottle, a rubber duck and a globe.

A prediction is not a verification

The green border in the demonstration footage marks a success prediction. That word matters. A prediction still needs checking.

In the researchers’ hardware evaluation, GoalDex succeeded in 80 of 100 trials, with 20 target images per object. The tablet bottle reached 13 of 20 and the cube 18. The strongest pose-based approach the researchers compared reached 27%. The most common failure was declaring success before the object was close enough to the goal. Those figures come from the paper’s evaluation, not from counting green flashes in an edited demonstration.

What it does not show

This is a research result on five tested objects, not proof that a robot can handle anything in your home. The paper notes that training covered 25 objects in simulation and does not yet generalize to entirely unseen object categories.

Bigger machines face the same question. As Atlas and the engine cover show, moving the arm is one problem, and knowing where the object actually ended up is another.

Whose version of done

The idea is still useful. Show a robot the result you want, then let it work toward that picture.

My takeaway: the next challenge is making sure the robot’s version of done matches yours.