Buyer beware: Sorting through the polls
Amid the flurry of poll numbers flying through the air during the federal election campaign, a voter could be forgiven for feeling confused.
One recent poll suggested the Conservatives had a significant lead in public support.
Another poll released the same day told a different tale, suggesting the Tories were gently falling back toward minority territory.
So how is a voter supposed to sort through all the numbers?
Pay attention to the basics of any poll, suggests the CBC's David Taylor, a senior producer on the Election Desk who keeps a close eye on polls.
Understand what kind of poll it is, how many people were surveyed, what period of time is represented by the data, what the margin of error is.
The pollsters know they don't have all the answers, Taylor says. We shouldn't expect to get all the answers either.
Nor should we use polls to predict exactly what will happen on Oct. 14. But we can gain some instant insight into the feeling of the electorate at a given moment, and depending on the type of poll, a sense of trends emerging among voters.
Types of polls
Some polls are the products of relatively rapid surveys — snapshot polls taken through a hefty number
of phone calls placed over a couple of nights, and the results turned around quickly.
Others take a different tack, doing what is called a rolling poll. A fixed starting date is chosen, and calls are made each night. After three or four nights, the results are added together and reported. The poll continues daily, with the results from the first night dropped off, and those from the most recent night added into the mix. And so on.
Snapshot polls have some advantages, says Taylor.
- Provide data from a moment in time.
- Tend to be a good-sized sample — 1,000 or 2,000 respondents — gathered quickly.
- Tend to ask more questions.
- Can be parsed in different ways; for example regional issues, as long as there's a valid sample size to consider.
The data is fresh, says Taylor. "That matters to folks in a dynamic environment."
There are drawbacks to snapshot polls, though, including the risk of trying to make a prediction based on the data. They don't predict a thing, Taylor says.
Rolling polls have their own advantages. They include:
- Daily reporting - "instant gratification," Taylor says.
- The ability to observe emerging trends as a rolling poll continues throughout a campaign.
"What gets interesting is what you see over time in the rear-view mirror," says Taylor.
But rolling polls also have their drawbacks:
- There's a risk in trying to make predictions, particularly if you use the numbers from daily samples, which could be smaller and not the rolling total that is reported.
- There's also a risk in trying to explain daily shifts, something Taylor cautions against, noting there are a lot of reasons public opinion moves.
While political parties have used rolling polls for years, they came to greater media attention during the 2006 federal election, when the last rolling poll conducted by SES Research came within one-10th of a percentage point of the outcome for the four major parties. Another rolling poll by the Strategic Counsel for the Globe and Mail and CTV was farther off the mark.
This time around, three polling companies have rolling polls: Canadian Press Harris/Decima, which the CBC is carrying as a regular service; Nanos (formerly SES Research); and EKOS.
If a poll finds the level of support for parties growing close, particularly if the difference between them is less than the poll's margin of error, then no one can be sure where things stand.
"The truth is nobody knows where public opinion is when it gets that close," says Taylor.