Analysis & Commentary
The Spin Cycle
By Ira Basen
July 7, 2004
"Tories Close in on Majority"
-Ottawa Citizen, June 12, 2004
"Harper Widens Seat Lead"
- National Post, June 22, 2004
-Globe and Mail, June 25, 2004
Let's face it, sometimes it's not easy being a pollster. Lots of Canadians made predictions about the outcome of this election, and most of us were completely wrong. But our humiliation was confined to the relative obscurity of friends, family or co-workers in the office pool.
But obscurity is not a luxury available to pollsters. Their spectacular flameout on June 28 was right there for all of us to see. And the certainty with which they normally foist their opinions upon the rest of us makes it hard not to get some sense of satisfaction when they are sometimes revealed to be only human.
But it would be easier to feel some pangs of compassion for their very public humiliation if they could only admit that something might have gone wrong this time around, and corrections need to be made.
But generally speaking, that's not what's happening. On the contrary, the industry appears to be circling the wagons. Frank Graves of Ekos Research has boldly asserted that "the possibility that all the polls were wrong is highly unlikely." Other pollsters have made similar claims.
And why not? Over the years, the polling industry has succeeded in manoeuvring itself into the rather extraordinary position of never being wrong. Imagine that! Everyone makes mistakes at work, except pollsters.
Voter preference polls, they are constantly reminding us, are simply "snapshots in time." They capture opinion at the moment the question is asked. They have no predictive value. What makes this all so bulletproof for the pollsters is that the only time we can empirically verify their results is on election day. But, of course, they don't publish polls on election day, so how would we know if the polls they took up to that point were accurate or not? We can't know. What a sweet deal for the pollsters.
Still, there is no question that what happened on the night of June 28 has not been good for the polling industry. They don't like to have people challenging the accuracy of their polls, which is why they have been spinning their "snapshot" story so vigorously since June 28.
By and large, these media-funded voter preference polls are loss leaders for the polling companies. They're useful for raising their public profile, but they're not highly profitable. The industry's bread and butter, if you will pardon the expression, is asking Canadians what they spread on their toast on behalf of the butter producers. If the butter boys start thinking that the data they're getting isn't accurate, they might not choose to commission another poll next year. And that's really bad for business.
But if you scratch beneath the surface of the post-election bravado, you begin to see an industry starting to ask some serious questions about its own viability. And among the people doing the asking is one of the godfathers of Canadian polling, Angus Reid. He founded the company now known as Ipsos-Reid, which polls for CTV and the Globe and Mail.
In an interview with the B.C. on-line journal the Tyee, conducted a few weeks before election day, Reid explained that these days, in order for pollsters to find the 1,000 randomly selected Canadians they need to get a statistically meaningful sample, they might have to make as many as 10,000 phone calls.
Thanks largely to call screening and answering machines, roughly half of all calls never actually reach a human being. And when live contact is made, about 80 per cent of people now refuse to answer pollsters' questions. Reid calls this "the big dirty secret of the industry."
How do we know that those few people who do agree to be interviewed are expressing opinions that are similar to those held by the vast majority who refuse to talk? This is the underlying assumption behind public opinion polling, but it might no longer be true. "Maybe Conservatives, being perhaps older or more cantankerous or whatever, are more likely to refuse than Liberals," Reid suggested. "In which case, the industry is going to be confused on election day."
The polling industry is sensitive to the issue of rising refusal rates. It argues that more sophisticated questioning, better-skilled interviewers, and other techniques can minimize the damage. But that requires money, and that's the other problem confronting the industry. Most clients, including the media companies, are cutting back on the money they're spending on polling. Angus Reid estimates that in the 1980s, the Southam chain of newspapers spent about $250,000 on election polling. In this election, he figures that the CanWest/Global chain, including the National Post, probably spent less than $50,000.
But polling is a "you get what you pay for" kind of thing. The more money you spend, the more reliable the results. Chris Baker, who runs an Ottawa polling company called Continuum Research, told The Hill Times newsweekly that in this election, media companies were only interested in getting horserace headlines from their polling, and that's all they would pay to get. He estimates it would have cost at least another $5,000 to add the extra few questions that would have helped determine the likelihood of a voter changing his or her mind, or not voting at all, but that's not what the media companies wanted or needed.
There is, of course, an alternative to our media-induced obsession with polls and horse races during election campaigns. The CBC decided not to commission voter preference polls in this campaign, and downplayed the coverage of polls done by others. But it's unlikely this trend will spread to its media rivals. Polls are like crack cocaine for political journalists, and it's hard to imagine a campaign without them playing a central role.
But in order to avoid a repeat of the embarrassment of June 28, several things will have to change. First, it appears that if the media companies who commission the polls want to get a more reliable picture of what's really happening, they'll have to spend more, not less money on polling. And, they will have to be much more careful about how they report their results.
Why not include refusal rates when reporting polls so we can make some independent judgments on their reliability? And both pollsters and the press will have to think twice about doing seat projections, which proved even more egregiously wrong on election night than the voter preference polls. Last week, Frank Graves of Ekos called seat projections a "mug's game," and suggested his company may not do them anymore.
Whether they admit it or not, whether it's justified or not, the polling industry suffered a blow to its credibility on June 28. And by extension, so, too, did the newspapers and TV networks that paid for and published the polls. All of this is good news from the point of view of the voters.
Turns out that we aren't as predictable or as easily packaged as the "spindustry" might have thought. They'll have to work harder and smarter to regain some lost credibility. Beneath the bravado, they're squirming a bit. It looks good on them.
|Past Spin Cycles
- Demanding Better Campaigns (June 25, 2004)
- MediaSpin II (June 23, 2004)
- An evening in Spin Alley.
(June 16, 2004)
- Can you handle the truth? (June 14, 2004)
Spin (June 11, 2004)
of their own Bamboozlement":
Inside the Campaign Strategy Story (June 7, 2004)
the pulse of polling (June 3, 2004)
"off" message (June 2, 2004)
spinner scorned (May 31, 2004)
political reporting (May 28, 2004)
leaks (May 26, 2004)
A primer for voters (May 23, 2004)
Ira Basen has been with CBC Radio since 1984 as a documentary
maker, senior producer and executive producer. He has written for
Saturday Night, The Globe and Mail and The Walrus, and has taught
a course titled "Spin: PR, Journalism and the Search for Truth"
at the University of Toronto and Ryerson University. Ira's book
on public relations and the media will be published by Penguin Books
You can subscribe to Ira's column and have each new edition
e-mailed to you.
||A view on the election from Dennis Trudeau. more »
||Keep track of the campaign with Olga Safroshkina. more »