What if the light in the room could sense you waving your hand as you enter? And what if it responded by introducing minute light changes that instructed your smart coffee machine to switch on? Researchers at Dartmouth College have developed a sensing system called LiSense that aims to make the light around us "smart." Not only does it use light to sense people’s movements, but it also allows them to control devices in their environment with simple gestures, using light to transmit information.

The goal is to use light to gesture and interact with objects in a room, just like how you’d use a Wii or Kinect to interact with a TV. Xia Zhou, lead author of a paper on the research, wants to use smart light to turn every indoor space into a cognitive space.

"Using purely visible light, we can not only stay connected to the internet, but also have the environment know and respond to what we do, how we behave, and how we feel," Zhou tells Gizmag. "Smart light can bring intelligence to all the devices immersed in the light and allow them to act based on our behaviours."

To get LiSense to track a person’s movements only through light, the researchers built a light-sensing testbed, with LED lights in the ceiling and light sensors on the floor. The system uses the shadows cast by a person standing on the testbed to reconstruct their 3D human skeletal posture in real time. LiSense, the team states, essentially works on the same principle as a shadow puppet, where a hand held before a light blocks certain light rays and not others.

"Consider a person standing under several lights," Zhou explains. "If we can recover the shadow cast by each light in a different direction, we can aggregate the shadow information and collect the blockage information of a large number of light rays. We then use the information to search for a 3D skeleton posture that best matches the blockage information revealed by these shadows."

To get their shadow-based human sensing to work, the researchers had to crack two critical challenges. Since multiple ceiling lights lead to diminished and complex shadow patterns on the floor, they came up with light beacons. These separate the light rays from different light sources to recover the shadow pattern cast by each light. The team also designed an algorithm capable of taking the collected low resolution, 2D shadow maps from sensors in the floor and reconstructing a person’s posture in 3D.

LiSense makes use of advances in visible light communication (VLC), where information is encoded as light intensity changes at high frequency. Since most of the smart devices commonly available contain light sensors, they’re capable of receiving data by monitoring changes in light.

"The light changes are imperceptible to human eyes, yet the light sensors on our smart devices can sense the changes and decode data," Zhou tells us.

In tests, LiSense was able to reconstruct a 3D user skeleton within 16 milliseconds (ms) in real time. It was also able to produce shadow maps of all the LEDs every 11.8 ms, which is comparable to capturing video frames (without using any cameras). The system was also found to be robust in various light settings with users of varying body sizes and shapes.

Going forward, the researchers plan to minimize the number of light sensors used and have them seamlessly integrate into the environment. They also want to make the system sense more than low-level gestures, which could potentially open up all kinds of novel applications. For instance, smart light could monitor both our health and behaviour, to catch diseases in their early stages.

"If the light around us continuously monitors how we move and gesture over time, it might help detect early symptoms of diseases such as Parkinson’s, which has movement-related symptoms," Zhou explains. "Right now patients have to wear and carry bulky, cumbersome devices, or doctors have to videotape them. Light provides a new possibility—no on-body devices, no cameras."

A light-based sensing system also has numerous advantages over existing technologies, the team says. There’s no electromagnetic interference, it doesn’t penetrate walls and it reuses existing lighting infrastructure. It’s also secure, has a bandwidth 10,000 times greater than the radio frequency spectrum, and isn’t limited to classifying a pre-defined set of gestures and activities.

The researchers plan to present LiSense at MobiCom, the 21st annual International Conference on Mobile Computing and Networking, in early September. More information on the system is available in the following video.

View gallery - 3 images