Electronics

Instrumented Bodies gives music and dance some backbone

Instrumented Bodies gives musi...
The Spine digital music instrument (Photo: Vanessa Yaremchuk)
The Spine digital music instrument (Photo: Vanessa Yaremchuk)
View 19 Images
The 30 or so Instrumented Bodies devices developed so far
1/19
The 30 or so Instrumented Bodies devices developed so far
Close up of the three sizes of Ribs
2/19
Close up of the three sizes of Ribs
The Spine is made up of transparent acrylic vertebrae that slot onto PVC tubing
3/19
The Spine is made up of transparent acrylic vertebrae that slot onto PVC tubing
The Spine is flexible enough to bend and twist, but manages to keep its shape
4/19
The Spine is flexible enough to bend and twist, but manages to keep its shape
Performers Marjolaine Lambert and Sophie Breton with a Rib instrument (Photo: Michael Slobodian)
5/19
Performers Marjolaine Lambert and Sophie Breton with a Rib instrument (Photo: Michael Slobodian)
Soula Trougakos uses a detached Rib instrument (Photo: Michael Slobodian)
6/19
Soula Trougakos uses a detached Rib instrument (Photo: Michael Slobodian)
The Rib instrument used as part of a performance (Photo: Michael Slobodian)
7/19
The Rib instrument used as part of a performance (Photo: Michael Slobodian)
Performer Soula Trougakos wearing Rib instruments (Photo: Michael Slobodian)
8/19
Performer Soula Trougakos wearing Rib instruments (Photo: Michael Slobodian)
A close up of the Spine instrument
9/19
A close up of the Spine instrument
Influencing sound and music through dance (Photo: Vanessa Yaremchuk)
10/19
Influencing sound and music through dance (Photo: Vanessa Yaremchuk)
Soula Trougakos wearing a Spine instrument (Photo: Vanessa Yaremchuk)
11/19
Soula Trougakos wearing a Spine instrument (Photo: Vanessa Yaremchuk)
The Spine is flexible enough to bend and twist, but manages to keep its shape (Photo: Vanessa Yaremchuk)
12/19
The Spine is flexible enough to bend and twist, but manages to keep its shape (Photo: Vanessa Yaremchuk)
Influencing sound and music through dance (Photo: Vanessa Yaremchuk)
13/19
Influencing sound and music through dance (Photo: Vanessa Yaremchuk)
The Spine digital music instrument (Photo: Vanessa Yaremchuk)
14/19
The Spine digital music instrument (Photo: Vanessa Yaremchuk)
Sophie Breton performing with the Spine (Photo: Vanessa Yaremchuk)
15/19
Sophie Breton performing with the Spine (Photo: Vanessa Yaremchuk)
Like the Ribs and Visors, the Spine can also be used as a stand-alone instrument (Photo: Vanessa Yaremchuk)
16/19
Like the Ribs and Visors, the Spine can also be used as a stand-alone instrument (Photo: Vanessa Yaremchuk)
Early Spine prototype
17/19
Early Spine prototype
A Visor digital music instrument
18/19
A Visor digital music instrument
Sophie Breton wearing the Visor instrument (Photo: Michael Slobodian)
19/19
Sophie Breton wearing the Visor instrument (Photo: Michael Slobodian)

For the last three years, a small research team at McGill University has been working with a choreographer, a composer, dancers and musicians on a project named Instrumented Bodies. Three groups of sensor-packed, internally-lit digital music controllers that attach to a dancer's costume have been developed, each capable of wirelessly triggering synthesized music as the performer moves around the stage. Sounds are produced by tapping or stroking transparent Ribs or Visors, or by twisting, turning or moving Spines. Though work on the project continues, the instruments have already been used in a performance piece called Les Gestes which toured Canada and Europe during March and April.

"The project began as a collaboration between members of our group in the Input Devices and Music Interaction Lab (specifically Joseph Malloch, Ian Hattwick, and Marlon Schumacher, supervised by Marcelo Wanderley), a composer (Sean Ferguson, also at McGill), and a choreographer (Isabelle Van Grimde)," reveals Malloch. "In 2008 we worked with the same collaborators on a short piece for cello and dancer which made use of a digital musical instrument we had already developed called the T-Stick. We decided to apply for a grant to support a longer collaboration for which we would develop instruments specifically for dancers, but based loosely on the T-Stick."

The aim was to develop instruments that could be played on their own, or as part of the dancer's body. Rough sketches and outlines became early prototypes fashioned from foam, and then onto corrugated plastic. These were attached to dancers to explore the kind of movement possible while wearing what were now being referred to as prosthetic instruments. The designers settled on three basic types (a Visor, a Rib and a Spine), and work began on modifying the design, integrating sensor systems and selecting the right materials for working prototypes.

Early Spine prototype
Early Spine prototype

"The use of digital fabrication technologies allowed us to quickly iterate through variations of the prototypes," says Malloch. "To start out, we used laser-cutters at the McGill University School of Architecture and a 3D printer located at the Centre for Interdisciplinary Research in Music Media and Technology. As we moved to production we outsourced some of the laser-cutting to a commercial company."

The Ribs and Visors are made from laser-cut transparent acrylic and polycarbonate layers stuck firmly together with solvent. The laser cutter was also used to create eight touch-sensitive pads from a transparent conductive plastic film, which are connected to capacitive touch sensing circuitry at the base of the instruments by thin wire fed through grooves etched into the acrylic. The base is also home to a ZigBee radio transceiver, a 3-axis accelerometer, drivers for embedded LEDs and Li-Pol batteries.

Like the Ribs and Visors, the Spine can also be used as a stand-alone instrument (Photo: Vanessa Yaremchuk)
Like the Ribs and Visors, the Spine can also be used as a stand-alone instrument (Photo: Vanessa Yaremchuk)

Construction of the Spines is a little more involved. The laser cutter comes into play again to produce transparent acrylic vertebrae that slot onto PVC tubing. A thin PETg plastic rod is fixed to either end of the instrument, which allows the Spine to be bent and twisted. Also at each end of the Spine is an inertial measurement unit comprising a 3-axis accelerometer, a 3-axis gyro, a 3-axis magnetometer, and a microcontroller that runs custom firmware.

The latter uses a complementary filter to translate the sensor data into a pretty good approximation of the spine's orientation. "In this way we know the orientation of each end of the instrument (represented as quaternions), and we can interpolate between them to track or visualize the shape of the entire instrument," explains Malloch.

As with the Ribs and Visors, power for the Spines come from Li-Pol batteries, and there's a ZigBee radio transceiver, too. The sensor data from all active instruments is sent wirelessly to a computer for processing, before being made available for the control of synthesized media via an IDMIL-supported, open-source, cross-platform software library called libmapper.

The Rib instrument used as part of a performance (Photo: Michael Slobodian)
The Rib instrument used as part of a performance (Photo: Michael Slobodian)

All of the 30 or so instruments produced so far were designed to withstand extended use in live performances, and can be smoothly attached to, or detached from, a dancer's costume courtesy of 3D-printed mounts. The Ribs, Visors and Spines were also created to head out on the road without needing the developers to tag along. For the recent Les Gestes performances, however, technical support staff did accompany the two dancers and two musicians on the short tour.

"Their duties regarding the instruments were basically to change the batteries and make sure they were out of their cases before shows," says Hattwick. "We tried to make everything from the hard cases to batteries and computer interface as seamless as possible, so there was no need for any specialized knowledge on the part of the users. The software, on the other hand, required more attention and was the main focus of the tech staff running the show. I did actually go on part of the tour, but as an observer and not as tech staff."

Malloch told us that now the current funding period has come to an end, the development team is hoping to explore new ways of using the technology by setting up new collaborations. Work on new gesture mappings is also planned, along with refinements to some of the integrated electronics, and the design of new methods of attaching the instruments to a performer.

Malloch and Hattwick overview the Instrumented Bodies project in the short documentary below.

Source: IDMIL

1 comment
Rafael Kireyev
Fun idea. Now you can try to create music of your body. And yet, it seems to me that if you add a little bit of brain impulses then the composition will be more fulfilling.