Watch: Humanoid robot gets surprisingly good at tennis

This ain't teleoperation. Chinese researchers have tested a new, much quicker and easier method of teaching robots to play tennis, and the results look like a breakthrough in machine learning and real-world AI.

What does it take to be a decent sportsperson? Highly accurate perception, for a start – plus a lot of physical dexterity, excellent predictive abilities, fast reflex reactions, a sixth sense for angles, and no small amount of technique specific to the given sport.

The lattermost has been a challenge for robotics researchers; in tennis, as in most sports, wearable motion capture tech struggles to deal with how far tennis players run during a rally, and also can't yet read the tiny nuances of wrist angle and whatnot that separate a good shot from a bad one. It's far too dynamic a situation to make teleoperation an option.

And trying to divine this stuff from multi-camera TV footage using AI training software like nVidia's Vid2Player3D... Well, according to Zhang et al, authors of a new study, that's a "complex pipeline" that "may require substantial expertise and engineering efforts."

The team's new LATENT system goes back to motion capture, but only for the building blocks of technique, and it's designed to work with imperfect data. Effectively, in the current experiment, the researchers took some five hours' worth of motion capture data, in which human sportspeople demonstrated the "primitive skills" required for tennis: forehands, backhands, sideways shuffles and crossover steps, executed within a fraction of the area of a full-sized tennis court.

They crunched these motion captures to create a repertoire of human-like 'motion spaces,' then loaded these basic skills into the robots – in this case, Unitree's G1 humanoid, which you've seen all over the place doing everything from dance numbers to kickboxing, and which is now available from a pretty wild starting price of ~US$13,500.

Effectively, the LATENT system then more or less told the robots 'ok, there's how you should move. Now, using motions somewhat similar to those, your task is to see a tennis ball coming, and use your racket to hit it back over the net. Success is a ball landing on the opposite side of the court, within the white lines.'

With those basic skills and motions to choose from, the robots were then able to experiment with all the rest of the details; angles, timing, which movements to use for which purposes and when to move outside of the trained motions. The vast majority of this learning was done at greatly accelerated speed in simulation.

And the real-world results? Well, the G1 returned forehands at around 90% success and backhands at just under 80%, and looks remarkably agile and fluid and... An awful lot like a tennis player while doing it. Check it out:

Clearly, it's not ready for Wimbledon. Indeed, it's not ready for any sort of competitive match yet. But for an early-days effort, this represents remarkable progress.

It looks to me like it won't be long before a 10-grand Chinese robot will make a pretty dang decent tennis training partner, and the path is gradually being paved toward a world where the best professional tennis players have about as much chance of beating these things as a chess grandmaster has of beating an AI opponent.

Of course, pro tennis player isn't exactly the kind of routine, repetitive job people have been desperately hoping robots will take over. But robots will get some of the same benefits humans do out of sport – they'll learn to master their bodies under extreme circumstances, dealing with complex and highly dynamic situations, in ways that'll be useful in more practical tasks... Like, say, beating protestors about the head with Agassi-level style and flair!

The LATENT software is open-source and available at Github.