Jigs, reels, bits and bytes — these students are teaching a machine to play Irish music

A group of computer science students at Memorial University are hoping to use computers and artificial intelligence to generate some jigs and reels.

GAN Morrison learning jigs and reels — slowly

Zach Northcott and Caleb Graves have developed a neural network that is slowly learning how to play Irish music. (Francesca Swann/CBC)

A group of computer science students at Memorial University are hoping to use computers and artificial intelligence to generate some jigs and reels.

Zach Northcott and Caleb Graves, two of the six students working on the project, say they're using a type of neural network called a general adversarial network, or GAN — a project they're calling GANs and Reels.

"Basically, you can think of it almost like a cluster of nodes that are connected in similar ways to the brain, and when one activates it triggers activations elsewhere," Northcott said.

"So, I guess it's a bunch of little bits and bobs strung together to try and emulate knowledge or thought or learning in some sort of way."

It learns over time … to try and generate music that convincingly passes for traditional Irish music. - Zach Northcott

Graves said they've had to feed the network a huge amount of traditional Irish music for it to begin to emulate the style.

"It learns over time the characteristics of the music — it learns to recognize what Irish music is, what it thinks Irish music should sound like, and it uses that knowledge to try and generate music that convincingly passes for traditional Irish music."

And while they haven't settled on a name for the network, they're calling it GAN Morrison for now.

Vast amounts of Irish traditional music are fed into the GAN in hopes that it will learn how to produce its own music. (GANs & Reels/MUN Computer Science)

But does the music it generates really sound like traditional Irish music?

"Absolutely not," said Graves.

Have a listen for yourself in the audio clip below.

A group of computer science students at Memorial University have built a neural network and fed it Irish music, in hopes that it would one day generate a jig. Here's what it came up with. 1:02

Northcott said they believe part of the problem is in how the music is being encoded and decoded.

"You can't give a computer a pitch or sheet music and expect it to read it like a person would, so we've basically taken all the notes turned them into numbers, and then have the GAN work on that," he said.

A further challenge comes in getting the computer to play nothing at all.

"To try and represent a rest in music in our number system is difficult, because if you have middle C, you could say that's 60, and then C sharp is 61," said Northcott.

"But with a rest, it doesn't actually have a pitch, so we think that representing it as a zero is skewing what it thinks of in terms of pitches."

Improvements in leaps and bounds

Graves and Northcott said they have been working on the GAN since May, and improvements come in leaps and bounds in each successive iteration.

"Some of the changes we make will improve it significantly and some of them will sometimes even decrease it," said Graves.

To help track their progress, the students have also developed visual representations, representing pitches as grey scale pixels to compare with real Irish tunes put through the same visualization process.

Graves and Northcott use grey scale pixels to visually represent the music the GAN produces. (GANs & Reels/MUN Computer Science)

"We can kind of use just our intuition to say, is this working, is this not working? Is it getting better, is it not getting better?" Northcott said.

"We have some basic idea of music theory and what not, but we're not professional musicians by any regard, so having the visual cues helps us kind of refine things in a different way." 

A group of computer science students at Memorial University have built a neural network and fed it Irish music, in hopes that it would one day generate a jig. Here's what it came up with. 1:02
 

The hope is to push the use of neural networks to be used it ways they never have before and advance the use of computers to "learn." 

"We just want to see if there's, I guess, applications that haven't been considered, because traditionally, it is used for image manipulation," said Northcott.

"But if we can perhaps represent different data in a similar way to images, perhaps we have a way to manage data and draw conclusions from it." 

With files from On the Go

Read more articles from CBC Newfoundland and Labrador