For the last three years, a small research team at McGill University has been working with a choreographer, a composer, dancers and musicians on a project named Instrumented Bodies. Three groups of sensor-packed, internally-lit digital music controllers that attach to a dancer's costume have been developed, each capable of wirelessly triggering synthesized music as the performer moves around the stage. Sounds are produced by tapping or stroking transparent Ribs or Visors, or by twisting, turning or moving Spines. Though work on the project continues, the instruments have already been used in a performance piece called Les Gestes which toured Canada and Europe during March and April.
"The project began as a collaboration between members of our group in the Input Devices and Music Interaction Lab (specifically Joseph Malloch, Ian Hattwick, and Marlon Schumacher, supervised by Marcelo Wanderley), a composer (Sean Ferguson, also at McGill), and a choreographer (Isabelle Van Grimde)," reveals Malloch. "In 2008 we worked with the same collaborators on a short piece for cello and dancer which made use of a digital musical instrument we had already developed called the T-Stick. We decided to apply for a grant to support a longer collaboration for which we would develop instruments specifically for dancers, but based loosely on the T-Stick."
The aim was to develop instruments that could be played on their own, or as part of the dancer's body. Rough sketches and outlines became early prototypes fashioned from foam, and then onto corrugated plastic. These were attached to dancers to explore the kind of movement possible while wearing what were now being referred to as prosthetic instruments. The designers settled on three basic types (a Visor, a Rib and a Spine), and work began on modifying the design, integrating sensor systems and selecting the right materials for working prototypes.
"The use of digital fabrication technologies allowed us to quickly iterate through variations of the prototypes," says Malloch. "To start out, we used laser-cutters at the McGill University School of Architecture and a 3D printer located at the Centre for Interdisciplinary Research in Music Media and Technology. As we moved to production we outsourced some of the laser-cutting to a commercial company."
The Ribs and Visors are made from laser-cut transparent acrylic and polycarbonate layers stuck firmly together with solvent. The laser cutter was also used to create eight touch-sensitive pads from a transparent conductive plastic film, which are connected to capacitive touch sensing circuitry at the base of the instruments by thin wire fed through grooves etched into the acrylic. The base is also home to a ZigBee radio transceiver, a 3-axis accelerometer, drivers for embedded LEDs and Li-Pol batteries.
Construction of the Spines is a little more involved. The laser cutter comes into play again to produce transparent acrylic vertebrae that slot onto PVC tubing. A thin PETg plastic rod is fixed to either end of the instrument, which allows the Spine to be bent and twisted. Also at each end of the Spine is an inertial measurement unit comprising a 3-axis accelerometer, a 3-axis gyro, a 3-axis magnetometer, and a microcontroller that runs custom firmware.
The latter uses a complementary filter to translate the sensor data into a pretty good approximation of the spine's orientation. "In this way we know the orientation of each end of the instrument (represented as quaternions), and we can interpolate between them to track or visualize the shape of the entire instrument," explains Malloch.
As with the Ribs and Visors, power for the Spines come from Li-Pol batteries, and there's a ZigBee radio transceiver, too. The sensor data from all active instruments is sent wirelessly to a computer for processing, before being made available for the control of synthesized media via an IDMIL-supported, open-source, cross-platform software library called libmapper.
All of the 30 or so instruments produced so far were designed to withstand extended use in live performances, and can be smoothly attached to, or detached from, a dancer's costume courtesy of 3D-printed mounts. The Ribs, Visors and Spines were also created to head out on the road without needing the developers to tag along. For the recent Les Gestes performances, however, technical support staff did accompany the two dancers and two musicians on the short tour.
"Their duties regarding the instruments were basically to change the batteries and make sure they were out of their cases before shows," says Hattwick. "We tried to make everything from the hard cases to batteries and computer interface as seamless as possible, so there was no need for any specialized knowledge on the part of the users. The software, on the other hand, required more attention and was the main focus of the tech staff running the show. I did actually go on part of the tour, but as an observer and not as tech staff."
Malloch told us that now the current funding period has come to an end, the development team is hoping to explore new ways of using the technology by setting up new collaborations. Work on new gesture mappings is also planned, along with refinements to some of the integrated electronics, and the design of new methods of attaching the instruments to a performer.
Malloch and Hattwick overview the Instrumented Bodies project in the short documentary below.
Source: IDMIL
