Smartwatches

SoundWatch alerts deaf and hard-of-hearing people to important sounds

View 2 Images
The SoundWatch system alerts hard-of-hearing users with vibrations and visual information about important sounds
Jain et al./ASSETS 2020
The SoundWatch system can differentiate between common and important sounds, such as doorbells, car horns or babies crying
Jain et al./CHI 2020
The SoundWatch system alerts hard-of-hearing users with vibrations and visual information about important sounds
Jain et al./ASSETS 2020

For people who are deaf or hard of hearing, it can be difficult to pick out important sounds, such as a knock at the door or a fire alarm. Now researchers at the University of Washington have developed a smartwatch system called SoundWatch, which identifies sounds and alerts users to them with a buzz and a visual readout.

The project was led by Dhruv Jain, inspired by his own experience being hard of hearing. SoundWatch uses the smartwatch’s microphone to listen out for particular sounds, sends the data to a user’s connected phone for processing, then back to the watch to display in text what the sound is, and how loud it is. That’s accompanied by a vibration to notify the wearer that something has been detected. The SoundWatch system has been trained to identify common and important sounds, such as beeping microwaves, doorbells, car horns, dogs barking and babies crying.

“This technology provides people with a way to experience sounds that require an action — such as getting food from the microwave when it beeps,” says Jain. “But these devices can also enhance people’s experiences and help them feel more connected to the world. I use the watch prototype to notice birds chirping and waterfall sounds when I am hiking. It makes me feel present in nature. My hope is that other d/Deaf and hard-of-hearing people who are interested in sounds will also find SoundWatch helpful.”

The SoundWatch system can differentiate between common and important sounds, such as doorbells, car horns or babies crying
Jain et al./CHI 2020

It’s not perfect of course – the team says that the app sometimes gets it wrong, or takes too long to notify a user. But these issues are being worked on, as are improvements to how the system picks important sounds out of background noise, the ability to identify the direction it’s coming from, and personalization options.

“Disability is highly personal, and we want these devices to allow people to have deeper experiences,” says Jain. “We’re now looking into ways for people to personalize these systems for their own specific needs. We want people to be notified about the sounds they care about — a spouse’s voice versus general speech, the back door opening versus the front door opening, and more.”

The team is also developing an augmented reality version of the system called HoloSound, which can display captions and other information through HoloLens glasses.

The SoundWatch app should soon be available for download on Android devices, with beta testing underway.

Dhruv Jain demonstrates SoundWatch in the video below.

Source: University of Washington

  • Facebook
  • Twitter
  • Flipboard
  • LinkedIn
1 comment
SiteGuy
Absolutely brilliant and innovative use of technology to solve a vexing problem for those affected with severe hearing impairments. I believe that this will be widely adopted rapidly once it's ready for market. I predict that it will be spun off and then snapped up by a technology company that is savvy enough to see the enormous potential, and will want to integrate the product into its own product ecosystem. i.e. Apple.