Erin Gee is making music from human emotions

Concordia graduate Erin Gee has developed technology to measure emotions like anger, feat and joy and translate them into computer-programmed music performed by an ensemble of robots.

Robots use biological feedback to produce music in Swarming Emotional Pianos

Erin Gee in her robotics lab at Eastern Bloc. (Frank Opolko/CBC)

The first time I met Montreal visual artist and composer Erin Gee in 2012, she was sticking a very small needle that reads electrical currents into the nerve behind her own knee. 

Gee’s goal was to see what the emotion of pain would sound like.

Two years later, Gee has elected to work with less intrusive sensors to measure the body’s response to emotions. She uses the measurements to compose music creating what she describes as the sound of human emotions. 

The composer and self-taught robotics specialist has spent the last two years building robots, measuring actors’ sweat production, heart rate, blood flow and breathing — all indicators of heightened emotions — to create the data for a computer program that transfers the data into musical notes and triggers a performance from her tubular bell-outfitted robots. 

The work is inspired by research into neurophysiology — the study of how the nervous system works — undertaken by Australian medical school professor Vaughan Macefield at the University of Western Sydney.

Macefield is studying the body’s response to pain. Gee is turning that kind of research into a visual art and music project. 

"Let’s make the inside of our bodies visible for other people to experience and that’s how I came up with the idea for these robots," Gee said.

Swarming Emotional Pianos

Gee began work on her "cybernetic musical performance" called Swarming Emotional Pianos for her Master's in Visual Arts at Concordia University in Montreal. 

She's been able to continue the project in collaboration with the Montreal chamber music series, Innovations en concert

Isak Goldschneider from Innovations en concert says Gee's work pushes the envelope of what a chamber music concert can be. 

"Combining science and art and kind of inhabiting that zone between science and art is something very new for us but we're very happy to have these robots as chamber musicians in our series," Goldschneider says.

Swarming Emotional Pianos features a video of two actors responding to Gee's vocal commands: fear, anger, joy.

A school of small robots spin on their bases flashing lights in response to the actors’ heartbeats. Mallets on the robots strike the tubular bells in a pattern developed using the biofeedback data from the actor's emotions.

Gee is looking forward to seeing the performance herself.  She has her fingers crossed that all the technology will work. 

"What I’m intrigued by is the combination of actors breathing, crying laughing pauses in robotic movement sound of their motors and the light they emit. I don`t think it will be like something most people have seen before but I'm very excited about seeing it all come together and seeing how people will react," she says.

On mobile? Listen here 


Jeanette Kelly works as the arts reporter at CBC Montreal. She's also the host of Cinq à Six, Quebec's Saturday afternoon culture show on CBC Radio One.