Brains 'listen' to speech through skin: researchers
Last Updated: Monday, November 30, 2009 | 8:15 AM ET
By John Bowman, CBC News
Our brains can be fooled into perceiving one speech sound as a completely different one if the sound is accompanied by an inaudible puff of air, Canadian researchers have found.
The research published this week in Nature suggests that our perception of language isn't made up of only sound and visual data, but tactile information such as airflow as well.
Bryan Gick and Donald Derrick at the University of British Columbia created an experiment where people were made to listen to recorded sounds while puffs of air were simultaneously applied to their hands or necks.
When the inaudible puff of air was applied, the subjects perceived speech sounds that don't normally come with a puff of air in English speech — "ba" and "da" — as different sounds that do — "pa" and "ta."
"'Ba' and 'pa' are only really distinguished by this puff of air, and we're used to hearing that, but we're not so used to feeling it on our skin, because you'd have to be pretty close to somebody to pick that information up," said Gick.
Gick said that because we don't normally feel puffs of air on our skin when we're listening to people, it's interesting that our brain would be fooled by this airflow information.
In a famous illusion known as the McGurk Effect, people can be fooled into thinking they're hearing "da" when they're actually hearing "ba" at the same time as seeing a face mouthing the syllable "ga."
Gick says one theory to explain this illusion is that the brain goes through a lifetime of learning to integrate visual and sound information to perceive speech, so it becomes fooled when that information is mismatched, as in the McGurk Effect.
Gick says his research suggests that this integration of different senses into the perception of speech may not be something that we learn through experience.
"It's much less frequent that you would hear somebody speak and feel these puffs of air on your skin at the same time. The fact that we can get the same kind of illusion with [air puffs] suggests that there might be something different going on," said Gick.
Gick said it's also interesting that our brains can be fooled by a puff of air that most people don't hear and don't realize they produce when they speak.
The research could lead to "possible future directions in audio and telecommunication applications and aids for the hearing impaired," Gick and Derrick wrote in Nature.
