Actors may soon say good-bye to those humbling Lycra body suits commonly used in the visual effects industry, thanks to a group of researchers from the Max Planck Institute for Informatics (MPII). They've formed a start-up called The Captury that is set to deliver its proprietary markerless motion capture software later this year. Their software can even capture a costume's surface detail in three dimensions, like the draping folds in a ballroom dress.
The Captury's system requires between five and 12 ordinary cameras – far fewer than a typical Hollywood motion capture studio – making it easier to set up and more affordable for projects in multiple fields, like video games, live sports, and medical research. It generates a digital skeleton composed of 58 joints that moves in sync with the person shot on video, in just a few milliseconds.
“The system even detects a person’s movements when they are covered up by other objects or when there are disturbances in the background. This will allow us to shoot visual effects outside of the studio in the future, for example, out in open nature,” explains Nils Hasler, one of the computer scientists working with Prof. Christian Theobalt's Graphics, Vision, & Video team at MPII that developed the software.
“The field of medicine would also profit," he added. "It would be easier for doctors to depict and track the degree of recovery after operations on joints."
If you've seen the behind-the-scenes featurette detailing the creation of Gollum for Peter Jackson's film adaptation of The Lord of the Rings, then you can imagine how this software might be used. During filming, actor Andy Serkis had to wear a skin-tight body suit as he interacted with the other actors and the environment – scraping himself over rocks and braving freezing rivers. Instead of having computer animators painstakingly rotoscope Andy's movements, as they had to do for Gollum, the software could do most of the leg work automatically. Even more time and money could have been saved if they had the team's inpainting software, which deletes people from video footage.
Another exciting potential application would provide sports commentators with a three-dimensional replay of an athlete's performance on live television. Imagine how this could affect broadcasts of competitions like the Olympic games, and you begin to see the merits of The Captury's markerless system. The results may not be quite as accurate as those obtained with infrared markers, but they're improving all the time.
The Captury will commercialize the software later this year, having recently won top honors from the Gründerwettbewerb IKT Innovativ, a German start-up competition. The designers' presentation at CeBIT earlier this month has already generated interest from multiple companies hailing from TV and visual effects, the automobile industry, makers of sports equipment, medicine, and more.
The system can be seen in use, in the videos below.
Source: Max-Planck-Gesellschaft
