It's often during those early stages of learning to play a new instrument that many people give up in despair. Even though you swear that you're hitting the right notes, everything still sounds like an old gramophone recording played at the wrong speed. If only you could let someone take control of your hands to fast forward through the arduous repetition phase before muscle memory kicks in and the piece you're trying to play begins to sound more like it should. That's precisely the kind of potential offered by the PossessedHand project. Electrode-packing armbands placed on a user's forearm send electrical pulses through the muscles to take control of the movement of the hand - with fledgling Koto players testing the system having demonstrated greater accuracy and speedier progress.
The goal of the PossessedHand project was to develop a support system for playing musical instruments where users could quickly learn the gestures, positioning and timing needed for a piece of music without having to wear glove-type devices that digitally control the movement of their fingers. By positioning two belts on the forearm of a user - each containing a number of non-invasive electrode pads - researchers were able to effectively take control of five independent and 11 linked joints in the hand to electrically control the movement of individual fingers of the wearer's hand.
Many of the muscles used to move fingers - like the superficial flexor muscle, deep flexor muscle, long flexor muscle of the thumb, common digital extensor muscle, flexor carpi radialis muscle, long palmar muscle, and flexor carpi ulnaris muscle - are located in the forearm. Movement of the joints can be controlled by stimulating these muscles with electrical pulses. The researchers also discovered that muscles at different depths in the forearm respond to different strengths of voltage, resulting in the system being able to take over control of a total of 16 joints in the user's hand.
The PossessedHand prototype consists of a battery, a condenser, a timing volume, a switching board (Photo-MOS Relays Series AQV221), a microcontroller, and solid gel electrode pads on two belts. One armband is positioned on the upper part of the forearm and the other lower down and then connected to PC via USB and controlled by a specially-developed graphical user interface. The pulse width is 0.2 ms and the voltage range is 17 - 29 V.
To overcome difficulties in precise positioning of the electrodes over specific muscles in different users, the researchers crammed 14 electrodes into each of the two armbands and then developed an automated calibration system. The system estimates the relationship between the 28 electrodes, the level of stimulation and the amount of muscle movement required.
Volunteers were blindfolded and asked to identify which joints were being moved by the system and reactions to PossessedHand ranged from Scary, just scary to I felt like my forearm was pushed by someone to I felt like my body was hacked.
The team then transcribed Japanese Koto scores into PossessedHand scores, which included information on which finger should be moved and when. Two beginners then alternated between system supported and non-supported playing of the music, with arm positioning over the 13-stringed instrument being undertaken by the user in both cases. The team noted a significant reduction in timing errors and fewer fingering mistakes when PossessedHand was active.
It's also important when playing a musical instrument to apply the correct amount of force at the appropriate time. Sadly, the PossessedHand experiments could not electrically stimulate enough force to actually grasp an object or play an instrument. As such, this would seem to fit in with learning systems like Piano Hands and Piano Maestro.
Such technology might also find application in the treatment of folks who suffer from impaired muscle control or be used by professionals looking to quickly improve dexterity, such as surgeons.
The paper entitled PossessedHand: Techniques for controlling human handsusing electrical muscles stimuli by Emi Tamaki, Takashi Miyaki, and Jun Rekimoto was presented at the ACM CHI Conference on Human Factors in Computing Systems in Vancouver, Canada in May 2011.
The PossessedHand system was recently demonstrated at the World Haptics Conference in Istanbul, Turkey.