Science

New animation tech could make motion capture suits obsolete

New animation tech could make ...
The quick body movements of sparring martial artists are tracked by The Captury's software without the need for special markers
The quick body movements of sparring martial artists are tracked by The Captury's software without the need for special markers
View 9 Images
The Captury's motion capture software can generate realistic three-dimensional data such as the folds in this woman's dress
1/9
The Captury's motion capture software can generate realistic three-dimensional data such as the folds in this woman's dress
Fine-grain detail like the folds in clothing can be recreated using The Captury's motion capture software
2/9
Fine-grain detail like the folds in clothing can be recreated using The Captury's motion capture software
The dynamic folds and crinkles in cloth are captured with a high degree of accuracy by The Captury's software
3/9
The dynamic folds and crinkles in cloth are captured with a high degree of accuracy by The Captury's software
A close-up of the details achievable by The Captury's motion capture software
4/9
A close-up of the details achievable by The Captury's motion capture software
Another close-up of the details achievable by The Captury's motion capture software
5/9
Another close-up of the details achievable by The Captury's motion capture software
A martial artist's movements are tracked without any special markers by The Captury's motion capture software
6/9
A martial artist's movements are tracked without any special markers by The Captury's motion capture software
An actress is tracked without any special markers by The Captury's motion capture software
7/9
An actress is tracked without any special markers by The Captury's motion capture software
The Captury's software can apply the tracked motion to a digital model in real-time
8/9
The Captury's software can apply the tracked motion to a digital model in real-time
The quick body movements of sparring martial artists are tracked by The Captury's software without the need for special markers
9/9
The quick body movements of sparring martial artists are tracked by The Captury's software without the need for special markers
View gallery - 9 images

Actors may soon say good-bye to those humbling Lycra body suits commonly used in the visual effects industry, thanks to a group of researchers from the Max Planck Institute for Informatics (MPII). They've formed a start-up called The Captury that is set to deliver its proprietary markerless motion capture software later this year. Their software can even capture a costume's surface detail in three dimensions, like the draping folds in a ballroom dress.

The Captury's system requires between five and 12 ordinary cameras – far fewer than a typical Hollywood motion capture studio – making it easier to set up and more affordable for projects in multiple fields, like video games, live sports, and medical research. It generates a digital skeleton composed of 58 joints that moves in sync with the person shot on video, in just a few milliseconds.

“The system even detects a person’s movements when they are covered up by other objects or when there are disturbances in the background. This will allow us to shoot visual effects outside of the studio in the future, for example, out in open nature,” explains Nils Hasler, one of the computer scientists working with Prof. Christian Theobalt's Graphics, Vision, & Video team at MPII that developed the software.

“The field of medicine would also profit," he added. "It would be easier for doctors to depict and track the degree of recovery after operations on joints."

If you've seen the behind-the-scenes featurette detailing the creation of Gollum for Peter Jackson's film adaptation of The Lord of the Rings, then you can imagine how this software might be used. During filming, actor Andy Serkis had to wear a skin-tight body suit as he interacted with the other actors and the environment – scraping himself over rocks and braving freezing rivers. Instead of having computer animators painstakingly rotoscope Andy's movements, as they had to do for Gollum, the software could do most of the leg work automatically. Even more time and money could have been saved if they had the team's inpainting software, which deletes people from video footage.

Another exciting potential application would provide sports commentators with a three-dimensional replay of an athlete's performance on live television. Imagine how this could affect broadcasts of competitions like the Olympic games, and you begin to see the merits of The Captury's markerless system. The results may not be quite as accurate as those obtained with infrared markers, but they're improving all the time.

The Captury will commercialize the software later this year, having recently won top honors from the Gründerwettbewerb IKT Innovativ, a German start-up competition. The designers' presentation at CeBIT earlier this month has already generated interest from multiple companies hailing from TV and visual effects, the automobile industry, makers of sports equipment, medicine, and more.

The system can be seen in use, in the videos below.

Source: Max-Planck-Gesellschaft

The Captury - markerless motion capture

View gallery - 9 images
3 comments
Arahant
Those people are about to make alot of money, im jealous.
Dekarate
Another case of public funded research being taken private for the capture of profits?
DaddyHoggy
I'm impressed, but can it tell the difference between the human shape and the clothes they are wearing? If not, 3D party software and/or humans will still need to deconstruct the captured 3D animation and texture appropriately (given current techniques have heavy investment in tech that takes the movements of the "ping pong balls" and uses this as the underlying skeletal model which the 3D animated character uses for its movement - it will be interesting to see if this tech actually aids or degrades this process).