For people who are deaf or hard of hearing, it can be difficult to pick out important sounds, such as a knock at the door or a fire alarm. Now researchers at the University of Washington have developed a smartwatch system called SoundWatch, which identifies sounds and alerts users to them with a buzz and a visual readout.
The project was led by Dhruv Jain, inspired by his own experience being hard of hearing. SoundWatch uses the smartwatch’s microphone to listen out for particular sounds, sends the data to a user’s connected phone for processing, then back to the watch to display in text what the sound is, and how loud it is. That’s accompanied by a vibration to notify the wearer that something has been detected. The SoundWatch system has been trained to identify common and important sounds, such as beeping microwaves, doorbells, car horns, dogs barking and babies crying.
“This technology provides people with a way to experience sounds that require an action — such as getting food from the microwave when it beeps,” says Jain. “But these devices can also enhance people’s experiences and help them feel more connected to the world. I use the watch prototype to notice birds chirping and waterfall sounds when I am hiking. It makes me feel present in nature. My hope is that other d/Deaf and hard-of-hearing people who are interested in sounds will also find SoundWatch helpful.”
It’s not perfect of course – the team says that the app sometimes gets it wrong, or takes too long to notify a user. But these issues are being worked on, as are improvements to how the system picks important sounds out of background noise, the ability to identify the direction it’s coming from, and personalization options.
“Disability is highly personal, and we want these devices to allow people to have deeper experiences,” says Jain. “We’re now looking into ways for people to personalize these systems for their own specific needs. We want people to be notified about the sounds they care about — a spouse’s voice versus general speech, the back door opening versus the front door opening, and more.”
The team is also developing an augmented reality version of the system called HoloSound, which can display captions and other information through HoloLens glasses.
The SoundWatch app should soon be available for download on Android devices, with beta testing underway.
Dhruv Jain demonstrates SoundWatch in the video below.
Source: University of Washington