What IS a pollster’s job anyway?

Wow, there has been a lot of online discussion about polls in the last few days. I’d like to make a point.

It seems a lot of people assume that pollsters are trying desperately to get a representative sample, and that they fail if they do not achieve that. To tell a pollster their data are flawed, it seems, is the ultimate insult!

That’s an incorrect assumption.

Any pollster, ever, who tells you their sample is perfectly representative either doesn’t know what they’re doing, or is lying. Yes! The data are flawed, and they always have been. This is nothing new, and has not just come about in recent years due to increasing non-coverage. No pollster can get a representative sample of eligible voters. It’s simply not possible.

The pollster’s job, in my view, is to try to understand why they can’t, and to attempt to disentangle the signal from the noise. It’s the job of a good pollster to spend hours thinking about sources of error, and considering ways to reduce it, cancel it out, or otherwise adjust for it. They can’t always get it right, but that’s the nature of measurement in a context where there are so many variables.

A poll is a measurement tool, and all measurements contain error.

That is all.


3 thoughts on “What IS a pollster’s job anyway?

  1. Over at one of the Public Address discussions someone asked why polling companies don’t make a “bias correction” to get a more accurate result.

    I would think that you wouldn’t want any ad-hoc corrections. If there’s a source of error you’d want to find it and eliminate it or account for it explicitly?

Make a comment...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s