The founder of Abacus Data, a Canadian polling firm, dropped kind of an interesting URL yesterday: abacus-weighting.com. It’s a advertisement in the form of a case study on why Abacus weights their political polls on past vote. It fits perfectly with the theme of yesterday’s post on how pollster’s get different results from the same data (the answer is they weight the raw data differently).
If you follow Nate Silver (or American political polling in general), you probably know that pollsters undercounted Trump support in all three elections where he was on the ballot. What I learned from this post is that support for the Conservative Party of Canada has been underestimated in their firm’s polling data in every polling wave for every election since 2011:
In every single wave, across every single election cycle, Conservative voters are underrepresented in our demographically weighted sample relative to their actual share of the vote. Not in most waves. Not in some elections. In every case we can observe.
Weighting for recalled past vote improves the estimate in every case, sometimes dramatically so:
In every election, past vote weighting moved our Conservative estimates upward and our Liberal estimates downward — consistently in the direction of the actual result. The 2021 election shows the most dramatic correction: a 7-point improvement in our Conservative estimate.
Another tidbit from the discussion section:
A critical secondary finding: the accuracy of recalled past vote diminishes over time. Short-term recall had an average absolute error of 2.3 pp relative to the actual result, increasing to 3.0 pp for long-term recall collected almost three years later. Long-term recall performed no better than the no-past-vote baseline. Short-term recall was the clear winner.
YouGov’s experimental work reinforced this. When they reweighted 2017 UK election data using past vote collected immediately after that election, 41% of respondents reported voting Labour. When collected from the same respondents two years later, only 33% recalled voting Labour. The difference in estimated Labour vote share between those two scenarios was three percentage points. The lesson is not that past vote weighting fails. It is that stale recall data undermines it.
I find it interesting that Abacus chose to give this write-up its own URL, basically vibe code up a nice little mini-site for what in the past might have been a much more mundanely formatted blog post (I suspect the text was partially AI-written too). I think we’ll be seeing a lot more of this kind of bespoke website in the future, now that every executive can conjure one up in a fraction of the time it would have taken them previously (and without any real coding skill).