Jigs, reels, bits and bytes — these students are teaching a machine to play Irish music
GAN Morrison learning jigs and reels — slowly
A group of computer science students at Memorial University are hoping to use computers and artificial intelligence to generate some jigs and reels.
Zach Northcott and Caleb Graves, two of the six students working on the project, say they're using a type of neural network called a general adversarial network, or GAN — a project they're calling GANs and Reels.
"Basically, you can think of it almost like a cluster of nodes that are connected in similar ways to the brain, and when one activates it triggers activations elsewhere," Northcott said.
"So, I guess it's a bunch of little bits and bobs strung together to try and emulate knowledge or thought or learning in some sort of way."
It learns over time … to try and generate music that convincingly passes for traditional Irish music. - Zach Northcott
Graves said they've had to feed the network a huge amount of traditional Irish music for it to begin to emulate the style.
"It learns over time the characteristics of the music — it learns to recognize what Irish music is, what it thinks Irish music should sound like, and it uses that knowledge to try and generate music that convincingly passes for traditional Irish music."
And while they haven't settled on a name for the network, they're calling it GAN Morrison for now.
But does the music it generates really sound like traditional Irish music?
"Absolutely not," said Graves.
Have a listen for yourself in the audio clip below.
Northcott said they believe part of the problem is in how the music is being encoded and decoded.
"You can't give a computer a pitch or sheet music and expect it to read it like a person would, so we've basically taken all the notes turned them into numbers, and then have the GAN work on that," he said.
A further challenge comes in getting the computer to play nothing at all.
"To try and represent a rest in music in our number system is difficult, because if you have middle C, you could say that's 60, and then C sharp is 61," said Northcott.
"But with a rest, it doesn't actually have a pitch, so we think that representing it as a zero is skewing what it thinks of in terms of pitches."
Improvements in leaps and bounds
Graves and Northcott said they have been working on the GAN since May, and improvements come in leaps and bounds in each successive iteration.
"Some of the changes we make will improve it significantly and some of them will sometimes even decrease it," said Graves.
To help track their progress, the students have also developed visual representations, representing pitches as grey scale pixels to compare with real Irish tunes put through the same visualization process.
"We can kind of use just our intuition to say, is this working, is this not working? Is it getting better, is it not getting better?" Northcott said.
"We have some basic idea of music theory and what not, but we're not professional musicians by any regard, so having the visual cues helps us kind of refine things in a different way."
The hope is to push the use of neural networks to be used it ways they never have before and advance the use of computers to "learn."
"We just want to see if there's, I guess, applications that haven't been considered, because traditionally, it is used for image manipulation," said Northcott.
"But if we can perhaps represent different data in a similar way to images, perhaps we have a way to manage data and draw conclusions from it."
With files from On the Go