PROFILE — These 2 teens built a sign language translator during the pandemic

Story by CBC Kids News • 2021-12-03 06:00

The ConchShell Bracelet turns sign language into spoken word


We're profiling cool kids doing cool things.
Know someone you think should be profiled on our site?
Email us at cbckidsnews@cbc.ca and tell us what makes them so awesome.


Name

Jin Schofield
Sarvnaz Ale Mohammad

Hometown

Richmond Hill, Ontario

Age

17

Claim to fame:

While the rest of us were napping during lockdown, two 17-year-olds from Richmond Hill, Ontario, were working hard to invent a new device to help people who are deaf.

The ConchShell Bracelet that Jin Schofield and Sarvnaz Ale Mohammad designed uses a camera and tiny computer to translate American Sign Language (ASL) into spoken English.

American Sign Language allows people to use hand movements and facial expressions to communicate without speaking.

While the device is still a work in progress, their pandemic project has already won at least $15,000 in prizes.

“It really gave us, like, a lot of purpose and motivation throughout all the time that we would just spend indoors and lockdown.” - Sarvnaz Ale Mohammad, 17

Jin Schofield, left, shows the codes behind the ConchShell Bracelet on a laptop. Sarvnaz Ale Mohammad, right, shows how the device is worn. (Image submitted by Jin Schofield)

The light bulb moment

Jin told CBC Kids News she was in Grade 9 when she volunteered at a laboratory and met a deaf student.

He had to work with a human translator to help him communicate with those who didn’t know sign language, and it stuck with her.

When Jin and Sarvnaz were brainstorming ways to stay busy during the pandemic, the idea to automate that translator came up.

“What if we put a camera on someone’s chest and it could translate ASL?” Jin remembers asking Sarvnaz, half-joking.

To Jin’s surprise, Sarvnaz took her seriously and said, “That’s a great idea!”

The inventors used 30,000 images of ASL hand gestures to build the software for the ConchShell Bracelet. (Image submitted by Jin Schofield)

Building on the idea

The two had basic programming knowledge from their robotics team but had no idea how to get started.

So they signed up for an online eight-week machine learning course designed by Stanford University in California.

They also set up one-time phone calls with local university professors who directed them to more resources.

Neither of the girls are members of the deaf community, so part of their skill-building was doing online research into what inventions were already out there.

The teens also reached out to people they knew who used ASL to ask them about their preferences.

The finished product

After doing their research, Jin and Sarvnaz decided to build a bracelet instead of a chest camera.

The bracelet has a front-facing camera that recognizes ASL.

Right-handed people wear it on their left hand and use their right to do sign language in front of the camera.

The device then uses specially designed computer software to translate out loud in English.

Left-handed people do the opposite.

The hardware in the ConchShell Bracelet consists of a microcomputer called a Raspberry Pi Zero W. It’s connected by a wire to another microcomputer that serves as a camera. (Image submitted by Jin Schofield)

The future of ConchShell

There’s still work to be done to improve their invention, but Jin and Sarvnaz are getting lots of encouragement.

In August 2020, the girls won the pitch competition at The DMZ, a tech startup program at Toronto’s Ryerson University, and took home $5,000.

They earned $10,000 a year later when they won the Youth Impact Challenge, a national competition hosted by two Canadian companies that funds businesses with positive environmental or social impacts.

The teens say they hope to use the money to continue experimenting with the device.

The biggest issue they’re facing is how to make it fully functional, said Sarvnaz.

“Increasing the accuracy,” she said, “is definitely something that takes time.”

What’s next for the girls

Sarvnaz said she hopes to attend university for either engineering or computer science, while Jin is thinking of doing life science and computer science.

The girls say they complement each other’s skills and hope to continue working together as a team.

That includes one day putting ConchShell Bracelets on the arms of those who need them the most.


TOP IMAGE CREDIT: Image submitted by Jin Schofield, graphic design by Philip Street/CBC

Get your class on the same page, add this to
Google Classroom