Think critically about cell phone polling

So the odd person or two comments that New Zealand polls should sample cell phone numbers as well as landline numbers. I’ve blogged on this before. When landline only polls begin to miss the mark, polling approaches will need to change.

I (and my colleagues) have done a wee bit of work investigating cell phone sampling and interviewing. I’m personally not too convinced by the ‘non-coverage of cell phone only households’ argument, for a number of reasons, but mainly because non-coverage is just one potential source of survey error. When it comes to polls and surveys, the potential sources of error are infinite. It’s the job of a good pollster to try and identify and reduce error wherever practically possible.

I’ve  read a couple of comments recently that the Roy Morgan poll is the most reliable poll because it samples cell phones.

This latest New Zealand Roy Morgan Poll on voting intention was conducted by telephone – both landline and mobile telephone, with a NZ wide cross-section of 894 electors from April 29 – May 12, 2013. Of all electors surveyed 5% (down 0.5%) didn’t name a party.

I don’t really have much reason to doubt the methodology that Roy Morgan uses, because I know almost nothing about how Roy Morgan carry out their poll.

Roy Morgan may have an innovative way to sample cell phones, so they don’t want to reveal too much about their approach. Okay. Fair enough. I work in a commercial research company myself, so I can accept that. What bugs me though is when people say that any poll is a reliable poll just because it calls cell phones. That is like saying a car is well tuned because the horn works!

If you’re the slightest bit interested in thinking critically about polls, and how to include cell numbers, here are a few questions to ponder:

  • How are the landline and cell numbers generated?
  • Are the cell numbers randomly generated or taken at random from a list?
  • What is the quality of the list? Is it a purchased list? Purchased lists can often be biased toward homeowners and higher income earners (marketing lists are often compiled from public database searches, such as homeowner database searches)
  • Do those interviewed via cell phone also have a landline? If they do, what is done to correct for the fact that they’re already covered by the landline sample frame?

Also, I just read this interesting article by Imogen Crispe on the TV3 wesbite.

A New Zealand internet provider has developed an app which lets people use their smartphone like a landline.

Communications technologies change rapidly. When it comes time to change polling approaches, who even knows if cell phone polling will be the most viable option? What about VOIP? Online polls? Who knows…?

Advertisements

Make a comment...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s