Typically, motion capture systems are confined to one studio containing multiple cameras, and they require subjects to wear sensor-equipped body suits. A new setup, however, is based around a single chest-mounted camera.
Known as MonoEye, the experimental system was designed by a Tokyo Institute of Technology team led by Prof. Hideki Koike, working with colleagues from Carnegie Mellon University.
It incorporates a camera fitted with an ultra-wide-angle lens, that is worn on the subject's chest. With its 280-degree field of view, that camera is able to image not only the environment in front of the subject, but also that person's arms, legs and face.
The video is analyzed via three specially designed deep neural networks, which respectively estimate the subject's three-dimensional body pose, their head pose, and the camera's orientation within its surroundings. That data is used to build an animated wire-frame computer model of the subject, which replicates their movements.
MonoEye's neural networks have so far been trained on a dataset made up of 680,000 renderings of people of different body shapes and sizes performing various activities, so some work still needs to be done before the system is optimized for use on physical bodies.
Ultimately, though, it is hoped that the technology may be refined into something as compact as a tie clip or brooch. It could then serve as a much cheaper and simpler alternative to conventional motion capture systems.
The research is being presented this week via the online 33rd ACM Symposium on User Interface Software and Technology.
Source: Tokyo Institute of Technology
