Exaggerated benefit found in 1 out of 3 early trials

For some mysterious reason, treatment effect can get smaller after a first clinical trial.

'Proteus Effect': No way to tell which new study will be the one where the benefit fades

Researchers were surprised to find that in about a third of studies, an early benefit faded over time. (Sinart Creative/Shutterstock)

This is an excerpt from Second Opinion, a weekly roundup of eclectic and under-the-radar health and medical science news emailed to subscribers every Saturday morning. If you haven't subscribed yet, you can do that by clicking here.

A new drug or medical device might show a promising benefit in the first trial, and even in the second trial. But for some mysterious reason that treatment effect can get smaller in followup studies.

"In reality, it shouldn't," said Dr. Hassan Murad, a Mayo Clinic epidemiologist and author of a new study on the vanishing treatment effect. "If everything is at random, you should expect that the earlier study or the later study should show an effect that is similar."

Murad's study compared results from more than 900 trials for drugs and devices to treat chronic medical conditions including cancer, diabetes, high blood pressure and heart disease. The researchers found that in about a third of the studies, an early benefit faded over time.

"We were surprised at the prevalence," said Murad, adding that it means one out of every three studies could show a benefit that won't show up in later studies.

The vanishing benefit is called the "Proteus Effect," named for a shape-shifting character from Greek mythology. And there's no way to tell which one out of three new studies will be the one where the benefit fades over time.

Unintentional bias?

"We tried to see if perhaps the size of the study matters, or how well the study was done, or whether it was a multi-centre versus single-centre. We tried to look at several explanations that would explain this, and we couldn't find any explanation."

Murad said one possible explanation might be that people who are chosen for the first trials are healthier, with fewer complications, a bias that might be unintentional.

"They will choose patients that are more likely to benefit." Murad said. "Whereas in subsequent trials, they choose everyone. That's like the real-world experience in subsequent trials."

The first trials also get the most publicity. And Murad said people are often eager to use the newest drugs.

"And unfortunately [with] the subsequent trials, sometimes people don't hear about them, because the results are not as impressive, and because the practice has already shifted."

To read the entire Second Opinion newsletter every Saturday morning, subscribe by clicking here.