Psychologists confront impossible finding, triggering a revolution in the field

*Originally published on November 1, 2019.

In 2011, an American psychologist named Daryl Bem proved the impossible. He showed that precognition — the ability to sense the future — is real. His study was explosive, and shook the very foundations of psychology. 

"This would probably be the most important research paper I would say ever published in any field, if it were true," said Jeff Galak, psychologist at Carnegie Mellon University. 

"If this paper were true, our understanding of the entire world, the universe, physics, [and] psychology, for sure, would be completely different," Galak said.

"We would no longer see time as this linear thing that we move through, but instead something that can go forwards and backwards. And we could reach into the future and pull information from that — if it were true. And 'if 'is a big part of that statement." 

Replication crisis

Daryl Bem is a professor emeritus of psychology at Cornell University. His paper, Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect, was published in the Journal of Personality and Social Psychology — a top-tier journal for the field.

"The [paper's] conclusion was ridiculous," said Chris Chambers, a professor of cognitive neuroscience at Cardiff University in Wales. He's the author of the book The Seven Deadly Sins of Psychology. 

"And this is really interesting because if a paper like this that's doing everything normally and properly can end up producing a ridiculous conclusion, then how many other papers that use those exact same methods that didn't reach ridiculous conclusions are similarly flawed?"

You have a right to scrutinize and verify and correct.​​​​- Simine Vazire, psychologist 

After Feeling the Future was published, a group called the Open Science Collaboration organized a massive replication study. And 270 scientists from 17 countries signed up. They picked 100 studies published in the year 2008 as their test sample — all from reputable, peer-reviewed psychology journals. 

The plan was to repeat all 100 experiments exactly as described, and then see what happens. The findings came out in 2015. The results were stunning: only 36 percent of replications were successful.

The ripple effect

University of Toronto psychologist Michael Inzlicht was shocked to find that research papers in his own area of research no longer held water. They could not be replicated under the filter of more rigorous methodology.

"I had grown up a scientist believing in the scientific method and the tools that we used, and all of a sudden, this one replication just made me question everything," said Inzlicht. 

"What was real, what could I trust? The things I was studying... were they real? Could I trust them?"

Inzlicht is one of the psychologists leading the way to set new research standards. He attributes the lapse in his field to the tremendous pressure researchers face to produce new, high-impact research.

"Basic science is not always about chasing the new," said Inzlicht. 

"It's not always about chasing something groundbreaking. It's about building a house. And if the foundations of the house are rotten, if from the beginning a discipline was built on shoddy foundations, the entire enterprise can fall."

Building a new foundation

Since 2011, standards for psychology research have indeed changed. 

What's known as 'pre-registration' is becoming more common: researchers writing up how they're going to conduct a study, what their hypotheses are, and how they intend to analyze the data before doing their experiment. This protocol prevents researchers from massaging the data and reporting a positive result until they actually find one.

More than 200 scientific journals, both inside and outside psychology, now publish "registered reports," reporting their decision whether to accept or reject studies that are submitted before the experiments have actually been performed. So the decision is based on the proposed methodology and not how exciting the results are.

Researchers also formed formed organizations like the Centre for Open Science and the Society for the Improvement of Psychological Science.

"If you're signing up to be a scientist, you're signing up to say 'check my work'," said Simine Vazire, psychologist at the University of California at Davis and one of the co-founders of the Society for the Improvement of Psychological Science. 

"Don't just take my word for it. Don't just trust me," said Vazire.  "You have a right to scrutinize and verify and correct."

Guests in this episode:

• Jeff Galak is a professor of marketing at Carnegie Mellon University. 
• Daryl Bem is professor emeritus of psychology at Cornell University. 
• Chris Chambers is a professor of psychology specializing in cognitive neuroscience at Cardiff University in Wales. He's also the author of the book, The Seven Deadly Sins of Psychology.
• Michael Inzlicht is a professor of psychology at the University of Toronto. He's also a principal investigator at the Toronto Laboratory for Social Neuroscience.
• Simine Vazire is an associate professor of psychology at U.C. Davis where she studies personality. She is one of the co-founders of the Society for the Improvement of Psychological Science.
• Harry Collins is a distinguished research professor of social science at Cardiff University, specializing in scientific knowledge. He's the author of several books including Forms of Life: The method and meaning of sociology.
• Alexa Tullet is an assistant professor of social psychology at the University of Alabama and co-founder of the Society for the Improvement of Psychological Science.
• Alison Ledgerwood is a professor of psychology at U.C. Davis.
  
  Special thanks to Dr. Ed Kroc, for help with statistics; Dr. Candis Callison, for help with philosophy of science; Tom Lowe, for recording Professor Collins in Wales; Emma Partridge, for booking Professor Bem; and Cited Media Productions, for supporting the making of this programme.
  

** This episode was written and produced by Alexander B. Kim of Cited Podast, with support from Ideas Senior Producer Nicola Luksic.