Rob Salmond’s post on cell phone polling

I agree with Rob Salmond that within five years polling methodologies will likely change.

Some thoughts though..

1. The proportion of cell only households is much higher in the US than in New Zealand (where it’s around 12% to 14%), so of course there are more cell only polls in the US. Just because ‘they do it in the States’, doesn’t mean we should do the same here. The sorts of people who live in ‘cell only households’ in New Zealand are not necessarily the same as the sorts of people who live in cell only households in the States.

2. The company that I work for has no policy on “…refusing to call cell phones.” In fact, they do randomly dial cell phones for telephone surveys. They will also call them for the poll if a non-qualifying person in the household gives them a cell number to call.

3. Our decision about not randomly dialing cell phones (yet) has very little to do with the cost of calling cell phones (in fact the cost is not that substantial). It’s due more to a) the degree to which the additional sample frame will reduce bias versus increase variance (produce less stable results), and b) the structure of the New Zealand cell phone system.

4. When the decision is made to change polling approaches it may or may not be a decision to change to cell phone polling (Rob hints at this in his post).

5. At present my view is that, in New Zealand, non-response is a far far bigger source of error than non-coverage. If non-coverage of cell only households is such a big issue, how come most polls seem to over-state support for the Green party? And why don’t they under-state support for the Labour Party?

6. In New Zealand, does calling cell phones decrease non-response or increase it? Don’t underestimate the importance of this.

Calling cells is not, and will never be, the magic bullet for opinion polling.

UPDATE: I’ve read, here and there, some comments that polls use a) published landline listings, or b) an outdated list of number banks for RDD sampling. I can categorically state that ‘a’ is absolute rubbish. None of the main media-client public polls use published listings. At the company I work for ‘b’ is also rubbish. It’s quite possible to uncover new number ranges.

For those interested, RDD works by randomly generating numbers within number banks, then connection testing them, and then re-sampling the connected numbers.


5 thoughts on “Rob Salmond’s post on cell phone polling

  1. I think I’ve made the point elsewhere about non-response, and entirely agree. As an aside, however, we’ve just transitioned to being a cell only household…

  2. Thanks for commenting James. Sorry to hear you’re no longer in a landline RDD sample frame 😦

    I entirely agree that non-coverage is becoming an increasingly important issue for landline surveys. The thing is though, the potential sources of error in a poll are endless – anyone pointing to just one is only seeing the tip of the iceberg. The job of a good survey researcher is to try to understand them all, and how they might interact to influence the measurement of public opinion.

  3. Sorry I was actually meaning that I think the other problems are (currently) bigger than non-coverage, but then undermined myself by pointing out I am now out of frame. You should miss me! I was always very keen to participate…

    1. Oh I understood what you meant – I just took the opportunity to clarify my perspective. Thanks again for commenting – who knows – we may call your cell soon! 🙂

Make a comment...

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s