Using your big ol' fingers to perform tasks on a smartphone's touchscreen can be difficult enough, with the smaller screen of a smartwatch presenting even more of a challenge. It was with this in mind that scientists at the University of Washington created FingerIO. The technology turns mobile devices into sonar systems that are capable of tracking the user's finger movements on nearby surfaces such as desk tops, or even in mid-air.
FingerIO works by repeatedly transmitting inaudible sound waves from the device's speakers. Those waves bounce off the user's finger, with their echoes then being detected by two of the device's mics – smartwatches typically have just one mic, so would need to be made with two in able to work with the system.
By analyzing the amount of time that elapses between the waves being emitted and their echoes subsequently received, it's possible to calculate the finger's position in space, in near-real time. It's accurate to within 8 mm.
Unlike a camera-based system, FingerIO works even when the device is covered by clothing – such as a front pocket in the case of a phone, or a sleeve in the case of a watch. Technologies such as radar, on the other hand, would require additional hardware along with increased processing power.
Suggested uses for the technology include things like flicking a finger to control music playback volume, making tapping motions to select items, or even writing text in the air. It's already been successfully tested using an Android app running on a Samsung Galaxy S4 smartphone, paired with a smartwatch customized with two microphones.
"You can't type very easily onto a smartwatch display, so we wanted to transform a desk or any area around a device into an input surface," says study lead author Rajalakshmi Nandakumar. "I don't need to instrument my fingers with any other sensors — I just use my finger to write something on a desk or any other surface and the device can track it with high resolution."
A paper on the technology is being presented in May at the CHI 2016 conference in San Jose, California. FingerIO is demonstrated in the video below.
Sources: University of Washington, FingerIO