How you are changing science journalism
One of the biggest changes that the internet has wrought in journalism has been the growing loudness and insistence of your voice, dear reader.
This has replaced what I have taken to calling the dictatorship of the author and the editor. In the past, after a reporter wrote about something, it was edited and then it appeared. Readers might respond, but at best a pittance of a pittance of those responses — think one or two — would make it on to a letters page.
Effectively, the author and editor ruled.
Now, if they are not libellous or completely scurrilous, all responses make it into a comment section following an online article. And because they do, they change the nature of what was originally presented.
With the readers' voices included, the piece becomes both what was originally written — or televised or broadcast — and the audience's view of what wasn't there or what needed to be explained further; what was confusing; what was simply wrong; what people were really interested in.
The dictatorship of author and editor hasn't been completely overthrown, but the process has been both democratized and challenged.
I say all this because it also seems to me that the democratization is changing the role of journalists — particularly science writers — in the media world. Now one of our duties is to revisit previously reported pieces and respond to readers' voices.
Case in point
With this in mind I want to have us return to a much commented-upon article about the differing health effects of an unhappy marriage on men and women that appeared on this site in early March.
The research out of the University of Utah was purported to show that while a strained marital relationship was equally depressing for both men and women, only in women did the depression translate into what is called "the metabolic syndrome" (MetS). This is a grouping of higher-than-normal levels of abdominal obesity, glucose, blood pressure and triglycerides, and below normal readings of the "good" cholesterol.
Together these conditions have been connected with significantly higher incidences of heart disease, stroke and diabetes.
The obvious implication, as a Globe and Mail headline put it, was "heartache harder on women." Or more tartly in the Tucson Medical Center's health blog, "Bad Marriages Break Women's Hearts."
The result was something that touched the chord of newsworthiness around the world. By Google News's count, more than 210 stories in various media and in more than a dozen languages appeared, not to mention 116 blog references and numerous other comments.
All of the news stories were, as far as I can see, similar to that appeared at CBC.ca. (Well, that's not precisely true, as the Deseret Morning News in Salt Lake City wondered whether the women in the study were more unhappy than others because many were likely to have been Mormons and the Mormon Church is strongly against the modern world's solution to a bad marriage — a good divorce.)
Not surprisingly, the CBC.ca story included comments from a number of people talking about how their toxic marriages had affected their health and what to do when a marriage goes bad. But lacking marriage counsellor credentials, I choose here to respond to Rita Guigon and the 26 readers who agreed with her when she wrote: "There's not enough information in this article. Who's to say which causes which?"
Interpreting the data
A very good question, it turns out; but first, some background information.
To begin with, the study did not appear in a peer-reviewed journal. It was rather an initial report at a conference on psychosomatic illnesses presented by lead author Nancy Henry, a PhD student in psychology.
This doesn't mean the study was untrue, but that its data was preliminary — so preliminary that some of the numbers in Henry's presentation changed in the period between a news release issued in advance of the talk and the talk itself.
What are the numbers?
There were 276 couples in the study, ranging in age from 32 and 76, who had been married 27.5 years on average. Mormons aside, a number had indeed experienced divorce — 20 per cent of the people were into a second marriage. They were also quite healthy, chosen because they didn't have any signs of coronary heart disease, weren't on blood pressure meds, nor were they what Henry described to me as "morbidly obese."
To determine their marital mind state, a number of tests designed to measure both happiness and unhappiness were administered.
When all the feedback from these people was totalled up, 70 women and 61 men qualified as seeing their marriages as unhappy. Henry says the gender difference in those figures was not statistically significant.
Physical measurements were made also made of the various conditions — think, for example, blood pressure — which when abnormal contributes to the metabolic syndrome. What the Utah group then did is take these physical results and correlate them with happiness and unhappiness in marriage.
Only the linkage was really quite abstract.
I will tell you, Rita, there were no graphs showing a direct correlation — such as how increasing unhappiness correlates with elevated blood pressure or a big belly. Nor was there any link between specific unhappiness and specific increase in any of the five MetS measures.
That is because the Utah group hasn't yet looked at these connections.
Rather, what Henry presented to the meeting was a non-specific equation that expressed a generalized correlation between unhappiness/depression and MetS in women. A similar equation said unhappily married men weren't statistically more likely to have MetS than happily married men.
Maybe. The Utah people tied their interpretation to increases in various levels of the conditions that go into MetS, but not to what would amount to a clinical definition of the syndrome. What constitutes that generally requires abnormal levels in three or more of the categories discussed above.
Timothy Smith, the professor under whom Nancy Henry is studying, told me: "I have a bit of quarrel with those who say, if you have three out of five conditions then you have the syndrome."
Effectively you ignore all the evidence that falls under the MetS cut-off line.
That makes sense. What doesn't make sense is that a group that includes five University of Utah scientists also calculated how many of the people in the study formally would qualify as having the metabolic syndrome.
Contradiction in figures
Given the fact that the largest American survey in the area reported that 22.8 per cent of men and 22.6 per cent of women in the U.S. met the MetS criteria, you would think that their heartache-filled, miserable marriages would make women's MetS levels higher than men's.
When the Utah group looked only at people whom doctors would diagnose as having the metabolic syndrome, there were 56 men and 33 women. That is 12 per cent of the women and 20 per cent of the men were in the MetS category.
"I don't have a good answer for that one," Smith said.
I agree, and then point out that is very worrisome when a sample produces a result that goes against most previous research. It makes you fear there was something very skewed in the selection process.
But back to causation.
Because they decided to focus their study on levels going up all over the place rather than look at the cut-off where MetS theoretically comes into being, neither Henry nor Smith know whether there are, statistically speaking, more unhappily married women than happily married women in the MetS category. It's possible an unhappy marriage raises a woman's risk factors but doesn't drive them into a truly medically dangerous area.
Smith doubts that will be the case, but as yet we don't know.
So as you see, Rita, there is no one-to-one correlation in anything that was presented. We don't yet know what causes what — or whether there is a causal relationship at all.
And that leads me to two responses to the concerned commenters on the previous article.
First, don't divorce your husband because you think it will make you heart-healthy. We have way too little evidence to justify that.
Second, there is a lesson here for the world's media in the age of reader comment: If you are looking for an object case about why research that has not gone through the peer-review process should be handled with a great deal of circumspection, this is probably it.
Preliminary findings are not necessarily wrong, but often are so skimpy and incomplete that highlighting them does a disservice to researchers, their work and the people reading about both.
And here again I tip my hat to science commenter Rita Guigon. She also wrote: "The problem with such stories is that either the study is very fuzzy or the results are not communicated in enough depth to justify the conclusions."
What can I say? I think Rita is absolutely right.