Good Thinking

Smart glove translates sign language gestures into text

The Language of Glove is a sensor-packed glove that translates the gestures of sign language into text
Timothy O'Connor/UC San Diego Jacobs School of Engineering
The Language of Glove is a sensor-packed glove that translates the gestures of sign language into text
Timothy O'Connor/UC San Diego Jacobs School of Engineering

Unless you're hard of hearing, or have hearing-impaired friends or relatives, you probably won't understand sign language, which is frustrating for those who rely on it to communicate. Now engineers at the University of California San Diego have developed a prototype of what they call "The Language of Glove," a Bluetooth-enabled, sensor-packed glove that reads the sign language hand gestures and translates them into text.

This isn't the first device designed to break down this particular language barrier. The 2012 Microsoft Imagine Cup was taken out by the EnableTalk gloves, which translate gestures into speech, and a London team developed a similar system a few years later called the SignLanguageGlove. Uni, meanwhile, is a tablet-like solution that uses infrared motion tracking to convert gestures to speech and text.

The Language of Glove uses a similar tracking method to the other glove-based systems. Nine stretchable sensors are attached to the knuckles of a leather athletic glove, two on each finger and one on the thumb. These are connected to a circuit board on the wrist, which generates a letter of the American Sign Language alphabet based on the position of the fingers.

The code works off a binary system. The sensors are designed to change their electrical resistance when they're bent or stretched, so when a finger is straight, the device registers a 0 input, while a bent finger sends a signal of 1. The system combines these signals from all nine sensors to generate a nine-digit code, which corresponds with a particular letter. For example, in sign language the letter A is made by keeping the thumb straight while bending all fingers, so the code is 011111111. To differentiate between similar gestures, the glove is also equipped with an accelerometer and pressure sensor.

The circuit board then translates these codes into a string of written letters, and sends them via Bluetooth to be displayed on a smartphone or computer screen. To keep the system accessible, the team created the Language of Glove using inexpensive materials and fabrication methods, and in future, the design could be used to control virtual or robotic hands, for entertainment or medical or military training.

"Gesture recognition is just one demonstration of this glove's capabilities," says Timothy O'Connor, first author of the Language of Glove study. "Our ultimate goal is to make this a smart glove that in the future will allow people to use their hands in virtual reality, which is much more intuitive than using a joystick and other existing controllers. This could be better for games and entertainment, but more importantly for virtual training procedures in medicine, for example, where it would be advantageous to actually simulate the use of one's hands."

The study was published in the journal PLOS ONE, and the Language of Glove can be seen in action in the video below.

Source: UCSD

  • Facebook
  • Twitter
  • Flipboard
  • LinkedIn
3 comments
Gizmowiz
They'll never succeed in translating ASL because ASL is not a written language and it changes daily because of that. It's also a 'local' language in that whatever city you live in will have different 'slang' and so different signs for each made up slang. Deaf kids continuously make up new signs for whatever topics they are communicating and then parents have to learn from their kids what the heck their saying. Some gets translated to other states, some does not. ASL users in Europe have totally different signs for the same thing as do almost all other countries, states, cities, and even inside a family unit! So therefore it's a language that can never be mastered. Being hearing impaired with 13 deaf kids, parents, grand children gives me the right to base this in fact.
MerlinGuy
Or they could just write it down. Or type it out. They are deaf folks, doesn't mean they don't know how to write or type.
CraigAllenCorson
From text, it's only a small step to voice. Has no one thought of that yet?