Speechin necklace recognizes its wearer's silently spoken words

Speechin necklace recognizes its wearer's silently spoken words
An upward-facing camera on the necklace monitors the underside of the user's chin
An upward-facing camera on the necklace monitors the underside of the user's chin
View 1 Image
An upward-facing camera on the necklace monitors the underside of the user's chin
An upward-facing camera on the necklace monitors the underside of the user's chin

If you were in a business meeting or a quiet library, it wouldn't really be appropriate if you suddenly blurted out "Siri, check email." That's where the Speechin necklace is designed to come in, as it recognizes silent speech.

The experimental device is being developed by Cornell University's Asst. Prof. Cheng Zhang and doctoral student Ruidong Zhang. It builds upon the NeckFace necklace that Cheng Zhang unveiled last year, which monitored the wearer's facial expressions.

Along with a microprocessor, battery and Bluetooth module, Speechin also features an upward-facing infrared camera that images the underside of the wearer's chin. It's held in this orientation via a set of "wings" that extend out to either side, along with a coin that serves as a weight on its bottom. In order to address privacy concerns, it doesn't point directly at the user's face.

Utilizing machine-learning-based algorithms, the device is able to determine which commands its wearer is silently speaking, based on their chin movements. It can then relay those commands to a paired smartphone.

The system was initially trained by monitoring the chin movements of 20 volunteers as they silently spoke known words and phrases – 10 of those people spoke English, while the other 10 spoke Mandarin. In subsequent tests, the participants mouthed 54 commonly used commands in English, along with 44 Mandarin words and phrases.

The necklace proved to be 90.5 and 91.6 percent accurate at recognizing the English and Mandarin speech, respectively. Those figures did drop significantly when volunteers used the device while walking, as their individual walking styles caused their heads to move in an unpredictable fashion.

It is hoped that once developed further, the technology could be utilized not only in settings where people have to stay quiet, but also in noisy environments where smartphones are unable to hear their users. The Speechin necklace could additionally be used by people who lack the power of speech.

A paper on the research was recently published in the journal Proceedings of the Association of Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies. Speechin is demonstrated in the video below.

Silent speech device developed by Cornell Students

Source: Cornell University

I've been saying for a while that facial recognition ought to vastly improve all forms of voice to text dictation. Web cams would fit the bill nicely for this, though I would like to see additional privacy features, perhaps twin lights, one for operation, two for unwanted "guests", or possibly a warning chime. I'm sure the team has already realized the technologies potential reach and look forward to throwing my keyboard into the dust bin! haha
Because it's morally and ethically righteous to check your email when you're in a business meeting or classroom? Um...
Funny.. A friend years ago told me he wanted to invent a device so he could have a silent coversation on the bus. Reason was he was embarrassed be speaking in a relatively quiet place. I said... Man you are the only idiot who is worried about having a phone call on publuc transit. Now I can tell him, he has something to feed his insecurities! This is progress! In all seriousness, this is amazing that speech can be recognized by chin movement. I can imagine Mandarin is much harder than English, being a tonal language. Bravo to the researchers. I can see this being used by the CIA.