Addendum: Large Polling Error in 2020 US Presidential

Early in 2022, I posted some preparatory observations about polling for the 2020 US Presidential election.  These were based off recommendations by AAPOR (American Association for Public Opinion Research), responding to widespread criticism of how the 2016 polls performed. 

The post-hoc evaluation of 2020 polling was released by AAPOR in July 2021.  Of the 5 ‘takeaways’ that I previously noted, the situation ostensibly improved in 2020.  State-level polling was a concern in 2016, but according to AAPOR, polls were “remarkably accurate” in 2020.  Mode effects (online, phone, etc.) didn’t seem to make an important contribution.  There was improved analysis of educational differences between Democratic and Republican votes.  But weighting for this didn’t adequately correct for polling error.  Nor did attention to partisanship, or demographics. 

In fact, the 2020 polls were worse than 2016.  National polling error was the highest in 40 years, and on the state level, it was the highest in 20 years.  Average topline results underestimated Trump certified vote tallies by 3.3 percentage points.  The Biden percentage of victory was erroneously predicted to be 1.0 percentage point higher than it was.

Briefly, their analysis of this historically poor polling result doesn’t shed much light on what went wrong, or how to fix it.  This is not a “shy Trump” question, like I raised 2 years ago.  The problem wasn’t that people gave socially-desirable answers. It appears there is essentially a sub-segment of Republicans who are systematically disinclined to respond to polls at all.  This might best be explained by distrust of polling. It results in a substantial block of voters that don’t get sampled.  Their invisibility produces error.  Unlike other overlooked respondents, many of them do go on and vote. 

The current reality is that polling is flummoxed by how to collect opinions from people who are so fundamentally withdrawn.  They won’t express their opinions in non-partisan channels.  They probably aren’t alienated only by pollsters, but by plenty of other institutions too.  Maybe there are psychographic markers to spot these lost respondents; certainly some experts are looking for them. The levels of polarization in US discourse might make it necessary to think about sampling plans within partisan categories.  It may not bode well for the future of polling, because persistently large error obviously threatens the value and usefulness of the results from polls.  Without objective polling results, the side that screams loudest may appear to be the largest block of voters.