Currently, if you want a VR setup to "know" what your individual fingers are doing, you either have to wear special gloves or place your hands directly in front of a camera. A new system, however, keeps tabs on the fingers via a simple wristband.
Known as TapID, the experimental technology is being developed by a team at Switzerland's ETH Zurich research institute. It's based on the fact that when each one of our fingers is tapped against a hard surface, it produces a vibration that's unique to that finger.
In the TapID system, a rubber wristband equipped with two accelerometers detects those vibrations as they travel up through the wrist bones. Machine learning-based software on a linked computer analyzes the wristband data in real time. It determines which finger is being tapped at what time, based on the detected vibrational profiles.
If the user is wearing VR glasses, a virtual interface such as a piano or computer keyboard can be projected onto a real-world surface such as desktop. A camera in the glasses will then keep track of which finger is adjacent to which key, so that the system will know what key is being pressed when the TapID wristband detects a finger-tap.
Although there are VR setups that use cameras alone to detect the actual finger-taps, they're generally not as sensitive as physical keyboards. By contrast, in lab testing that involved 18 volunteers, TapID proved to perform much better than such systems. Once the technology is developed further, it could conceivably be incorporated not only into a standalone wristband, but also into existing wearables such as smartwatches.
A paper on the research, which is being led by Prof. Christian Holz, was recently published in Proceedings of IEEE VR 2021. The system is demonstrated in the video below.
Source: ETH Zurich