Computers

SoundWave uses Doppler Effect to bring Kinect-like gesture recognition to PCs

SoundWave uses Doppler Effect ...
SoundWave detects motion inputs by detecting the frequency shift in tones emitted from the system's speaker
SoundWave detects motion inputs by detecting the frequency shift in tones emitted from the system's speaker
View 3 Images
SoundWave is responsive enough to play Tetris using gesture controls
1/3
SoundWave is responsive enough to play Tetris using gesture controls
SoundWave can be used on existing laptops with no additional hardware
2/3
SoundWave can be used on existing laptops with no additional hardware
SoundWave detects motion inputs by detecting the frequency shift in tones emitted from the system's speaker
3/3
SoundWave detects motion inputs by detecting the frequency shift in tones emitted from the system's speaker

Microsoft’s Kinect peripheral senses motion inputs using a combination of a depth sensor, RGB camera and a multi-array microphone. Now Microsoft’s research division, Microsoft Research, has developed a gesture recognition system called SoundWave that relies on sound instead of video to bring gesture recognition capabilities to a standard laptop or desktop computer without the need for any additional hardware.

The system uses a PC's existing speakers to emit an inaudible tone between 18 and 22 kHz. As the sound waves reflect off a moving object they change in frequency. The PC’s microphone detects this frequency change - known as the Doppler Effect - along with the sound wave’s speed, amplitude, direction and time variation.

SoundWave can be used on existing laptops with no additional hardware
SoundWave can be used on existing laptops with no additional hardware

Based on this information, an algorithm then translates the movements into gestures, which can be used to control things onscreen. SoundWave can detect multiple objects - or hands - and even works in noisy environments or when the laptop is being used to play music.

While the Kinect has found great success as a gaming peripheral with its ability to work at a bit of a distance, SoundWave appears to only detect gestures made relatively close to the computer. This might limit the technology to deskbound applications, but the number of unforeseen applications to which the Kinect has been adapted suggests SoundWave could find equally surprising uses for those chained to a desk.

SoundWave is responsive enough to play Tetris using gesture controls
SoundWave is responsive enough to play Tetris using gesture controls

The research team has already developed some examples that demonstrate the potential for the technology. Along with the standard scrolling with a wave of the hand, the responsiveness of the system has been demonstrated by using it to play Tetris. It could also be used in an office environment to automatically lock a user’s screen when they leave their desk.

The SoundWave technology is still under development in the Microsoft Research labs, and it’s not clear if or when we can expect to see this kind of gesture-recognition system released. While it looks promising, Microsoft might want to maximize the return on its Kinect for Windows before releasing gesture-recognition technology that does away with the need for purchasing any additional hardware.

Here’s a video from Microsoft Research showing SoundWave in action.

SoundWave: Using the Doppler Effect to Sense Gestures

Source: Microsoft Research via Extremetech

4 comments
Wyatt Bonnette
this is very interesting however at that particular kHz what are the long term side effects on the human subject. Studies have showed very high kHz above the human ears can cause headaches, dizziness, nausea feeling. It is interesting that it can determine different gestures, but wouldn't xbox's kinect addon work better and have more of a fine tuning ability? Or is this designed to work in conjunction with the kinect?
railwaymen
I agree that it looks promising and hopefully they'll be able to develop the idea as they want it to be. I'm counting on this, maybe The SoundWave Technology will create a chance to move more while working (I heard that sitting and not moving all day long is very bad for your health).
Charles Bosse
20 kHz is not inaudible to many people under 25 (a good share of gamers) and some over 25! People trying to use sounds above the hearing range need to realize that they either need to shell out for special drivers (40kHz speakers for instance) or realize that the reason computer speakers go up to 20 kHz is because some people can HEAR up to that point (I can hear clearly up to 19500 Hz and I am nearly 30 - and as teens both my mom and I could hear the low end of bats) and that those people will not only avoid your product at all costs, but also avoid anyone using the product.
Jürgen De Blonde
This is not something for people with dogs. And indeed, 18kHz is hardly inaudible...