Wednesday, June 12, 2013

What Gallup got wrong in 2012

The Gallup Poll predicted Mitt Romney would win the 2012 presidential election and consistently overestimated his support. This can be bad for business, so they have tried to figure out why that happened:

Here is a link to their report.

The review, led by Mr. Newport and Michael Traugott of the University of Michigan, was conducted by a team of statisticians, methodologists and analysts from Gallup, assisted by outside consultants. After identifying more than 20 possible elements that could have been related to the Republican skew in Gallup’s polls, the team isolated four factors as the likely causes.


Gallup’s model for identifying those most likely to vote — a series of seven questions — seemed to have failed in 2012, and the organization is re-evaluating its formula for ranking voters who will turn out.

Just as technology has changed the way campaigns work, it has altered the way survey researchers gather data. A change in the way that Gallup called respondents on land lines may have been a contributing factor that led to its sample to be older and more Republican. Half of their respondents, however, were reached on cellphones — a proportion that is at or above industry standards.

In addition, there were too many respondents from certain time zones. In the South and the Midwest, there were more respondents from the Central Time Zone, which tends to be more Republican, than the Eastern Time Zone, which skews Democratic.

Finally, the way Gallup asked callers about their race overrepresented some groups.