Gesture recognition devices may indeed offer more functionality than is possible using just a keyboard and mouse, but in order to use them, users have to lift their hands up and away from that keyboard. A team at Microsoft Research decided to address that problem, and created a prototype mechanical keyboard that recognizes hand gestures performed on or immediately above the keys.
The Type-Hover-Swipe keyboard features a 16 x 4 array of infrared proximity sensors embedded in a printed circuit board. Each sensor node peeks out from between the regular key caps, tracking the movement of the user's hands as they touch the keyboard or hover above it.
Although the array only manages a resolution of 64 pixels, it operates at high frame rates – above 300 hertz. That's enough for it to detect rapid movement, thus allowing the keyboard's machine learning algorithm to recognize a number of both static and dynamic "motion signatures."
These include swiping up, down, left and right; changing applications by hovering the hand in place; pinching to zoom; pointing for index up and down; and even a steering wheel gesture that allows users to control an onscreen car in a racing game.
More details on how the keyboard learns and recognizes gestures are available in the video below. There's currently no word on possible commercialization of the technology.
Source: Microsoft Research
