October 19, 2020

Social Desirability Bias And The Hidden Trump Vote

While most pollsters got the 2016 national election trends right, several key state polls (most notably Pennsylvania, Michigan and Wisconsin) severely under-measured the vote for Donald Trump. Since American presidential elections are not really national, but a collection of 50 state elections funneled through the Electoral College, projections of the 2016 winner missed the mark. There were a myriad of reasons for this—methodological and otherwise—but one of the most intriguing was the impact of social desirability bias on survey results.

Basically, this is the tendency of respondents to tell a live pollster what they perceive to be the "acceptable" answer—rather than the response that's representative of their true thoughts and feelings. If anything, this phenomenon has intensified since 2016—with increased polarization putting a chilling effect on peoples' openness on politics. Under-reporting Trump voters in battleground states could once again result in polls missing a fundamental trend.

So, how can social desirability bias be better accounted for in surveys? At Polity Research, we have developed a four-question Respondent Comfort Index (RCI)—which tries to measure the level of anxiety a respondent might feel about being open with an interviewer. Now, pollsters are not social psychologists, so this process is inexact at best. But, our four RCI questions (on an interval scale of "1" to "5") try to measure: the overall comfort level with sharing information; how comfortable people feel about talking to family and friends about politics; their level of trust in polls and the media; and their likelihood to share views on social media sites. Taken together, these results are "crunched" into a numeric index that can give us a hint as to how open a respondent has been. 

When evaluating the impact of all this on voting results, we can look at the comfort scores for each candidate's backers. If one candidate's voters show a disproportionately high 'discomfort' score, it might be prudent to factor that into the evaluation of where the race really stands. While imperfect, it's a start.