Edmonton app Frettable transcribes live sound into sheet music
Machine learning can now help musicians put their melodies on paper
The Edmonton man who created an app that transcribes the sound of an instrument into sheet music hopes it reduces geographic barriers for composers.
Greg Burlet was an undergraduate student at the University of Alberta when his bandmate moved to Vancouver. They wanted to write rock songs together, so they tried emailing audio files and experimenting with Skype. But there were lags with both methods.
Burlet wondered if a machine could be taught to recognize musical notes and transcribe them into sheet music. After working on the idea since the fall of 2015 — along the way he earned a master's degree in music technology from McGill University, and a degree in computer science from the U of A — he launched the app on Jan. 25 this year. Thousands of people have used Frettable since then.
Burlet played a few bars of a riff on the intro to Wild Cherry's 1976 hit Play That Funky Music on CBC Radio's Edmonton AM on Tuesday, demonstrating how the app works.
Similar to speech recognition, Frettable uses artificial intelligence to analyze a recording and transform a melody into musical notes on the page. The more music it hears, the smarter the app gets at converting it.
"The way I like to think about it is, there's a robot up in the sky somewhere that's listening to what I've just played," Burlet said.
The app works for guitar, voice, brass and woodwinds.
Burlet said the app's closest competitor is ScoreCloud, a music notation service run by a Swedish company. Plenty of services transcribe audio into sheet music, but Frettable's machine learning component is novel.
The technology isn't perfect. It struggles to pick out a melody in a noisy environment and can miss a few notes here and there. It's also designed to detect only one instrument at a time.
Still, the app does a good job of capturing the basic structure of melodies, Burlet said. It could be handy for music teachers and studio engineers (who could record vocal melodies and synthesize them into accompanying string tracks).
Perhaps its simplest application is as a memory aid for songwriters.
"You can refer back to the sheet music if you forgot a riff you wrote a year ago," Burlet said.
The app is available for smartphones and tablets via the iTunes app store and Google Play. A premium version, which costs $4.99 US per month, offers users longer recording times and the ability to download transcription files and remove print watermarks from music.