Motion capture system makes actors the camera instead of putting them in front of it
Computer-generated imagery (CGI) has become such a staple of modern movie-making that most people know what actors are doing when prancing around in front of green screens wearing skin-tight leotards with reflective balls affixed at various locations over their bodies - motion capture. In addition to the actor's performance, such techniques can also require the tracking of camera movements and props so that perspective is maintained when translating the movements into CGI. Now researchers have demonstrated a system that can perform motion capture almost anywhere and without the need to track a separate camera and it does this by mounting the cameras on the actors instead.
The wearable camera system developed by researchers at Disney Research, Pittsburgh (DRP) and Carnegie Mellon University (CMU) reconstructs the relative and global motions of an actor using a process called 'structure from motion' (SfM). CMU professor of computer science and robotics Takeo Kanade developed SfM 20 years ago as a means of determining the three-dimensional structure of an object by analyzing the images from a camera as it moves around the object, or as the object moves past the camera.
For use in motion capture, SfM isn't used primarily to analyze objects in the person's surroundings, but to estimate the pose of the cameras on the person. The researchers used Velcro to mount 20 lightweight, outward-facing cameras on the limbs and trunk of each subject and calibrated each camera using a reference structure. When each person performed a range-of-motion exercise the system was able to automatically build a digital skeleton and estimate where the cameras were positioned in respect to that skeleton.
As the actor moves through an environment, SfM estimates the rough position and orientation of limbs and collects some 3D information about the environment to provide context for the captured motion. The configuration of the body and its location in the environment are then refined to produce the final motion capture result.
"This could be the future of motion capture," said Takaaki Shiratori, a post-doctoral associate at DRP. "I think anyone will be able to do motion capture in the not-so-distant future," he said.
Although the quality of motion capture from body-mounted cameras isn't yet on a par with traditional techniques, Shiratori says it will improve as the resolution of small video cameras continues to improve. The technique also requires a hefty amount of computational power with just one minute of motion capture taking up to an entire day to process. DRP and CMU researchers are now working to find computational shortcuts, such as performing many of the steps simultaneously through parallel processing.
Shiratori gave a presentation about the new technique this week at SIGGRAPH 2011 in Vancouver and a paper detailing the technology and video of the system can also be found on the DRP website.