Technology & Science

Scientists read minds to predict hand actions

Canadian and U.S. researchers have been able to predict what hand movement a person is going to make by reading a scan of their brain.

Brain-driven prosthetics possible, but 'Minority Report' scenario far off

Based on active areas in the brain, shown in an fMRI scan, researchers could predict 55 to 65 per cent of the time which of three actions a person was going to do, several seconds before it happened. ((University of Western Ontario))
Canadian and U.S. researchers have been able to predict what hand movement a person is going to make by reading a scan of their brain.

The scientists at the University of Western Ontario and the University of Oregon scanned the brains of nine volunteers at the Robarts Research Institute in London, Ont. They found they were able to distinguish somewhat accurately among plans to make three hand movements that were only slightly different from one another:

  • Grasping the top of an object.
  • Grasping the bottom of an object.
  • Touching the object.
Jody Culham and Jason Gallivan at the University of Western Ontario were the two lead authors of the study. ((University of Western Ontario))
"We're showing that you can decode little subtle differences in finger movements based on the goal of the movement," said Jason Gallivan, a Ph.D. student in neuroscience at the University of Western Ontario and the lead author of a study published in the Journal of Neuroscience this week.

Previously, scientists had only been able to make similar predictions for animals with electrodes inserted in their brains. Functional magnetic resonance imaging, or fMRI, is far less intrusive, said Jody Culham, a psychology professor at the University of Western Ontario who is Gallivan’s supervisor and co-author. That made it possible to do such an experiment in humans.

While the new discovery may bring to mind Minority Report, the 2002 movie starring Tom Cruise where criminals are caught before the crimes they commit, Gallivan said that type of scenario is a long way off.

What is fMRI?

Functional magnetic resonance imaging is a technique that shows what parts of the brain are active by detecting changes in blood flow and the amount of oxygen consumed in different areas. Areas that are more active use more oxygen and therefore require more blood flow.

The scanner contains a strong magnet that causes particles inside the brain's atoms to line up in a certain way, producing a signal that is different for different kinds of atoms. In the case of MRI, the signal from hydrogen atoms in water is measured. The signal varies in different parts of the head, so that some parts appear darker than others.

An additional signal is detected in fMRI. Hemoglobin, the protein that carries oxygen in the blood, produces a slightly different signal when it is bound to oxygen, versus when it is not.

"The concept of a Hollywood mind reading — I still think we're several years away from being able to do that," Gallivan said. "What we've done here is constrained to the three behaviours we're measuring."

The researchers could only predict the action four seconds before it happened, and their accuracy was typically no better than 55 to 65 per cent.

Gallivan acknowledged that the accuracy was fairly low, but said it was statistically significant. He said it was difficult to interpret the signals because they depended on the subjects' attention, motivation and focus and any movements by the subjects interfered with the signals.

"We view this study largely as proof of concept," he said.

He added that one possible near-term use for the findings is better brain-controlled prosthetics. Prosthetic arms driven by certain kinds of brain waves already exist, but amputees need to be trained to make those specific brain waves. Prosthetics that target brain signals linked to planning would make them operate more like real limbs.

The study showed that about 13 or 14 sites in the brain areas responsible for vision and motion were involved in planning the different kinds of movements in the study.

The researchers are now working on trying to predict the direction that people plan to move their arm or their eye, and are interested in whether planning is the same when a person is using a tool such as a fork or tennis racket.

The researchers could distinguish among plans to make three hand movements that were only slightly different from one another. ((University of Western Ontario))