Election vote and seat projections
Full methodology for Éric Grenier's vote and seat projections
The following is a detailed explanation of the vote and seat projection methodology used for provincial and federal elections.
The vote projection model averages all publicly available opinion polls of decided and leaning voters that meet CBC standards. Polls are weighted by their age and sample size, as well as by the track record and past performance of the polling firm.
The weight of a poll is reduced by 35 per cent with each passing day once a campaign has officially begun. The date of the poll is determined by the last day the poll was in the field, unless the poll was conducted over a period of one week during an election campaign or two weeks outside of a campaign, in which case the date of the poll is considered to be the starting field date plus seven days (or 14 days outside of a campaign).
The sample size weighting is determined by the margin of error that applies to the poll, assuming a completely random sampling of the population. There is a cap of 2,000 on the sample size weighting — polls of more than 2,000 respondents are not given a corresponding boost in weight.
Pollster track record
Polling firms are weighted by their track record of accuracy over the last 10 years. Their accuracy rating is determined by three factors:
- The last poll the firm released during an election campaign.
- The average error for all parties that earned 3 per cent or more of the popular vote.
- The amount of time that has passed since the election.
In order to take into account changes of methodology or improvements made over time, the performance of a polling firm in a recent election is weighted more heavily than its performance in an older election.
The difficulty of each election is also taken into account: elections where the average error was smaller are weighted more heavily than elections in which the error was greater. This is meant to take into consideration elections in which there were particular factors contributing to pollster error that were outside of the pollster's control. Conversely, outlier pollsters are penalized more for elections where the consensus was close to the mark.
There is also a reduction in weight for polls that are outside of the confidence intervals of the vote projection, in order to reduce the extent to which potentially outlying polls make the projection swing dramatically.
Only polls commissioned by media outlets or independently self-commissioned by polling firms are included in the aggregate. Polls commissioned by political parties, interest groups or advocacy organizations are excluded.
The weightings for sample size, date and track record are combined to give each poll in the projection model an overall weight. No poll is ever awarded more than 66.7 per cent of the total weight, unless there have been no other recent polls, and a limit of 50 per cent is placed on the weight of any one poll when appropriate.
In short, this means that newer polls with larger sample sizes from polling firms with a good accuracy record are weighted more heavily than older and smaller polls from firms with a poorer track record.
After then estimating likely support for independents and smaller parties (based on performance in the last election and the number of candidates running this time), the projection model gives the best estimate of support that each party is likely to get in an election held on the last day of polling in the projection model.
Vote projection ranges
Despite performing better than most individual polls and an unweighted average of all polls, the vote projection is still heavily dependent on the accuracy of the polls. It can fail when the polls do, as occurred in the provincial elections in Alberta in 2012 and British Columbia in 2013.
The projection thus measures the likely error in the vote projection, using the degree of error polls have had in recent elections. The high and low ranges are based on 95 per cent of past election errors.
This means that the result of the election should fall within the high-to-low range 95 per cent of the time.
In five per cent of cases, the outcome will fall outside of the projected ranges.
Once the vote projection and likely ranges for each party are determined, the model uses this to make a seat projection.
The seat projection model uses a proportional swing method based on the difference between the results of the last three elections and current polls. The swing is weighted as follows: 50 per cent for the most recent election, 33.3 per cent for the next most recent, and 16.7 per cent for the election before that.
For example, if a party managed 20 per cent in Quebec in 2015 and is now polling at 40 per cent in Quebec, the party's results in each riding in Quebec in 2015 is doubled. If the party managed 10 per cent in Quebec in 2011, its results in a given Quebec riding in 2011 would be quadrupled. And if the party managed 40 per cent in Quebec in 2008, its results from that election would not be swung at all.
The image below shows how this method would have estimated the Liberals' support in the riding of Kingston and the Islands in the 2015 federal election, using the results of the election in Ontario in place of the polls:
This swing is applied to every party in each riding. As the result will not always add up to 100 per cent, the projection for each party is adjusted proportionately to equal 100 per cent.
Taking other factors into account
The swing model alone, however, cannot take into account the individual characteristics of each riding. Other factors need to be taken into account.
When a party lacks an incumbent candidate, its vote share is reduced by about 10 per cent of what it would have otherwise been.
The presence of a party leader in a riding is taken into account, as leaders have historically been able to outperform other candidates when a party is losing support in a region. Leaders running for the first time in a riding have also historically experienced a significant boost in support.
Similarly to when an incumbent candidate does not run again, there is a steep penalty when a leader vacates a riding, either because they lost it in the previous election or because they are not running for re-election.
Star candidates give their party a bonus of about 15 per cent. Star candidates are usually former legislators or cabinet ministers, party leaders or well-known figures from the private sector.
If an independent politician is running for re-election, his or her vote is dropped marginally from the previous election, according to the drop in support recorded by independents in other elections. The same penalty is applied to popular independent candidates who were never elected but are running again.
Politicians who left or were forced out of their party caucuses and are running for re-election as independents retain a proportion of their vote share from the previous election, based on the circumstances of their departure from caucus. Those who depart for positive reasons retain much more of their support than those who leave in disgrace. When the circumstances are hard to define, the average performance is used. A no-incumbent penalty is applied to the party the candidate left.
When an ex-legislator attempts a comeback as an independent, using past cases as a guide the independent ex-legislator retains a portion of the vote they received the last time they stood as a candidate.
By-elections are also taken into account when the incumbent party lost the seat or, when the by-election occurred after the most recent general election, if the results were significantly different from that previous vote (when a major party's support has doubled or been cut in half). The swing from by-elections is calculated by how current polling levels differ from where the parties stood in the polls at the time of the by-election.
When available, and when they differ from the projection's estimations, riding polls are also added to the projection for an individual riding. The weight of the poll is determined by the number of respondents but is capped at 50 per cent of the riding's projection. The riding poll's results are used as a new baseline, from which the numbers are adjusted as regional polling changes. When multiple riding polls are released during an election, only the latest one is taken into account.
Unique circumstances are also taken into account when looking at the three prior elections: for example, the presence of a very popular independent candidate may invalidate that election's results for the use of the swing model. A floor-crosser or leader that dramatically changed the outcome of an older election may also cause that election to be dropped from consideration.
When a party is running candidates where it did not have a name on the ballot in the previous election, the riding projection is equal to the regional vote projection for the party.
Likely seat ranges
In order to take into account error in polls and in the seat projection model itself, the vote projection ranges are used to determine likely seat ranges.
For example, if the high projected vote for a party in a given region is five per cent higher than the average projection, then the projected vote for the party in each riding in that region is increased by a factor of 1.05. If the projected high result for a party in a riding is greater than the projected low result for the party expected to win the seat, the seat is then considered winnable for the trailing party.
The low-to-high projected range thus reflects all of the seats a party could potentially win or lose at the 95 per cent confidence interval — 95 per cent of polling errors should be taken into account in this seat projection range.
Based on how the seat projection ranges overlap, 10,000 simulations are run to determine the number of cases in which each party can win a minority or majority government. These results reflect the probability that, taking into account past polling errors and if the vote were held today, a party would win the election.
To put probability into practical context, a party with a 25 per cent probability of winning has as much of a chance of winning the election as someone does of flipping a coin twice and getting "heads" both times. Unlikely — but still very possible.