The whole landline vs cell phone thing

I originally posted this as a comment on Dimpost and it was reposted by the blog owner. I tried to fix up my typos and I’ve made some other minor edits.

The problem with calling cell phones doesn’t really lie in the cost of calls. For a polling company, calling a cell phone doesn’t cost that much more than calling a landline. The problem is the complexity and cost of employing dual sampling frames when the proportion of cell phone users without a landline is still very low. If the purpose of calling cell phones is to reduce non-coverage of likely voters, then you may actually need to ‘screen out’ those you call on cell phones who also have a landline (because they are already covered by the landline sample frame).

If we assume (hypothetically) that 6% of eligible voters have cell phones and no landline, that means that 94% of the people you call on a cell phone will not be eligible to take part (again, because they are already covered by the landline sample frame). This is where the cost would really begin to build up – all those interviewer hours required just to screen people out (eek!).

This is not the only way to deal with non-coverage – but it’s actually one of the more straight forward and ‘statistically pure’ ways (ie, you can develop some sort of weighting scheme, but the more you weight, the greater the design effect which increases the margin or error, and decreases the accuracy of a poll).

To make things more complex:

– Some people have more than one cellphone, meaning that the probability of them being called is higher, so additional weighting would need to be applied to adjust for the probability of selection (you may notice that some polls weight by household size and the number of landlines connected to a house – this is adjusting for the probability of selection)

– There are a lot of cell phone numbers that are out of use, but when they are called they still go through to a voice mail. Unlike landlines, it is very difficult (ie, near impossible) to determine if there is actually an eligible person at the end of a number, so you’ve got no clear measure of the success rate of your sampling approach (ie, refusal rates, response rates, qualifier rates etc).

– At the moment such a small proportion of New Zealanders have a cell phone with no landline that party support would need to be dramatically different among those people for this particular type of non-coverage to influence the poll results for party vote (eg, support for Labour among cell phone only voters may need to be twice what it is among landline voters for the party vote result to shift by more than, say, the margin of error).

When the proportion of people with cell phones and no landline is considerably larger than it is today (like it is in some other countries), then it will definitely make sense to employ a dual sampling frame approach. In NZ though (at least in 2011) most pollsters got things pretty close to the election day result so this would suggest non-coverage of cell phone only voters isn’t a big issue just yet. If cell phone plans get cheaper, then polling approaches will probably need to change to keep up.

Advertisements

11 thoughts on “The whole landline vs cell phone thing

  1. Martyn Bradbury has commented on David Farrar’s post about this. Weirdly he tries to use your comments against DPF when it looks like DPF was supportive of your views.

  2. “Unlike landlines, it is very difficult (ie, near impossible) to determine if there is actually an eligible person at the end of a number, so you’ve got no clear measure of the success rate of your sampling approach (ie, refusal rates, response rates, qualifier rates etc).”

    Hmmm, I’m not sure if it would be any more difficult. In my previous residence I was called on a regular (monthly) basis for polling. To determine my eligibility, the pollster simply asked for my details, especially “could I speask with the next/last person to have/had a birthday in your household”. (There were two of us in the household, plus a third boarder on occassion.)

    So if I was of a mind to be mischievous, I could given incorrect info (I didn’t).

    It occurs to be there’d be no more/less likelihood of a person talking to a pollster through a cellphone. Or skype, for that matter.

    But I see your point about individuals owning multiple cellphones. I know of several who do, to make use of various special deals offered by telcos.

    Perhaps the best, safest, most accurate option is going back to door-to-door sampling. More expensive, perhaps, and not practical when you want to get respondants from small, isolated towns.

    1. “Perhaps the best, safest, most accurate option is going back to door-to-door sampling. More expensive, perhaps, and not practical when you want to get respondants from small, isolated towns.”

      You’ll get no argument from me that door-to-door surveys are exceptional in many ways. (In fact, I’m running a fully national one one right now.)

      The population coverage for national door-to-door surveys is excellent. Under the approach we use, almost every person in NZ has a chance to be selected, excluding those living offshore or those who live in households that would require 4WD access. They also get response rates above 60%, and close to 80% in some cases (see this one from 2008).

      The problem is though, it takes 2-3 months to collect the data. Given the speed at which things happen in politics, by the time all the data waswere collected, a door-to-door political poll would be out of date. 😦

  3. I have some issues with non-sampling of cell phones:
    – telco unbundling of land lines from broadband plans means that survey numbers from 2-3 years ago or even the 2013 Census are not reliable as an estimate of real and current land line penetration
    – cell phone only vs cell phone mostly. This is a key concept not dealt with in your article. Amongst under 40s, a lot of people may technically have landlines (usually related to internet plans) but not use them. They don’t list them, they don’t give them out as a contact number, they don’t connect a hand set to them, even if they do connect a handset – they don’t answer them because non-one they know has the number so callers are all telemarketers. My guess is that cell phone mostly is a least 20% of the population now and growing rapidly.
    – can you tell where cell phone users are? Technically you could if you could access location information but that’s presumably difficult. The alternative is asking them where they live. Since you’re asking lots of other questions in a poll this shouldn’t be difficult. As an aside I guess location is important for a sampling frame, but since the party vote is a national concept it’s not as important as it was under FPP.
    – some personal comments. Prior to the 2011 (2007 – 2011) election I lived in a few flats that had land lines that were not well used. These same flats contained well paid 20-somethings who vote and were practicing cell phone only users by choice not out of necessity. So high household income bracket, but young, and never sampled by existing poll methodologies. I don’t believe it can be assumed that these people in their growing proportions vote identically to land line using households which will be skewed toward older cohorts.
    – so in summary landline proportions may overstate landline users; cell phone mostly users are growing very fast; it’s my guess that close to 0% of cell phone mostly people vote Conservative or NZ First calling those poll results and the polls overall into question; further in so far as the polls affect public opinion as well as monitoring it, it’s a concern if polling is skewed systematically by methodological problems; finally if we’ve learned anything from Nate Silver’s success with fivethirtyeight.com it’s beyond time that our polling started to meet the needs of a more diverse population.

Make a comment...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s