MIT researchers have developed a new time-of-flight (TOF) 3D "nano-camera" with the ability to work with translucent objects, motion, fog, rain, and other factors in the environment that totally confuse previous TOF cameras, such as Microsoft's second-gen Kinect. The MIT Media Lab team has added these new capabilities by introducing additional information into the illuminating light beam. The resulting camera costs less than US$500 in parts.

Time-of-flight cameras, a subclass of scannerless LIDAR (a portmanteau of light radar), have been around for nearly 25 years and find application in automated manufacturing, image recognition, human-computer interfaces, collision avoidance and automatic parking in cars. The new nano-camera promises to improve on the performance and capabilities of these cameras and many additional niches are expected to open as capabilities expand and prices plummet.

A TOF camera takes a 3D photo of a scene by measuring the time required for light to travel from a source (typically a laser or LED), and then to bounce back from objects in the field of view to be detected by the TOF camera. This time delay measurement is calculated pixel by pixel, and is converted into a distance from the object to the camera roughly by multiplying the time delay by the speed of light and a geometrical factor.

This simple method, however, is not capable of determining distances when the field of view contains objects which are not opaque. Known as multipath interference, the problem arises when a single pixel in the image is illuminated by light returning from a variety of distances.

A simple example would be a clear glass vase in front of a wall, as appears in the lead picture above. The light reflecting from the vase does not travel as far as the light that reflects from the wall behind it and shines through the vase. This one pixel thus contains time delay information for two different surfaces, preventing a simple TOF camera from extracting accurate distance information.

To sort out the distance information when the light illuminating a pixel comes from paths having different lengths requires considerably more information than is provided by the simple TOF camera. The MIT Media Lab team, led by Associate Professor Ramesh Raskar, has figured out how to insert "considerably more information" into the modulation of the light that is illuminating the 3D scene.

It works like this. A complex, but still periodic, modulation is applied to the light illuminating a 3D scene. Among other things, the modulation of the light intensity changes between the early part and the late part of the periodic cycle. This means that if light arrives at a pixel from reflections at two different distances, the light from the two paths will have different modulations in time, allowing the two signals to be separated from one another.

The output of the TOF will then be that reflections from two different points in space were received at the same pixel, and the phase shift from the illumination for each is unambiguously determined. The spatial resolution for depth is about 2 cm in MIT's demonstration camera, which corresponds to a time resolution of about 700 picoseconds.

This means that the shape, outer dimensions, and inner dimensions of a translucent vase can be determined by such sophisticated TOF cameras. If you picture a wall through a fishtank, you will be able to determine the distance of the fish tank from the wall, as well as the size and shape of the fish tank.

The MIT TOF camera can even see an object on the far side of an optical diffuser by ignoring the time frame in which light returns from the diffuser, allowing the camera to focus on the object. These and many other feats impossible for a simple TOF camera are now within our grasp. In fact, many existing TOF cameras could adopt this mode of operation with a field software upgrade.

The prototype TOF camera was built in the MIT Media Lab using components on hand. There are now available less expensive versions of the components used, and the team estimates that a DIY'er could probably assemble a version of their advanced TOF camera for about US$500 worth of parts.

The nano-camera is explained in the following video.

View gallery - 8 images