Why the polls get things wrong

Just posted this in reply to an article on LinkedIn. Thought it could also go here…

Folks trying to explain polls often seem to look for one or two sources of error. Even the UK polling enquiry seemed to be looking for that ‘one big thing’ and appeared to discount small issues that could have made a small difference for only some polls.

Having tinkered with polls and surveys over the last decade or so, I can confidently state that there is never one source of error. They are infinite, they often interact and amplify each other, and they can also cancel each other out

It’s not the job of a pollster to carry out the perfect poll. It’s their job to understand why they can’t, and to minimise sources of error as much as possible within time and budget constraints.

Companies can make a lot of money from polling and surveys under the illusion their polls are ‘profressionally conducted’, when they may not be doing anything at all to systematically understand and minimise error.

The real problem is that polling is easy to do, but it’s really hard to do well. 

Advertisements

2 thoughts on “Why the polls get things wrong

  1. The other issue is the volatility of the voting public – both in terms of their propensity to even vote, and their propensity to change their mind. I’m quite certain the Dewey Wins effect (they changed their minds for Truman in the last two weeks between polling and voting) occurs even within 24 hours between a ‘day before’ poll and voting today. In 1993 in NZ Heylen went back to those polled on Friday, and some 30% had changed their minds – with traffic going every which way, but in balance, slightly toward National. Bugger the pollsters! said the PM – but don’t blame us if voters change their minds. The recent Trump upset was always quite within the realm of possibilities given the see-sawing enthusiasm (or otherwise) for Clinton and Trump.

Make a comment...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s