Campbell Live GCSB poll commits the ultimate sin of survey research

Campbell Live is to trying to run the biggest poll in New Zealand history.

C Live tweet

Here’s a really important point for anyone wanting to judge the accuracy of this poll – big numbers do not provide representative data!

Here’s why…

This poll commits the ultimate sin of survey research – it uses a self-selecting sample.

C Live poll

People choose to take part based on the topic. This means that the poll only represents the New Zealanders who feel strongly enough about the GCSB bill to take part in the poll.

The only way to attempt to gain a representative result is to take a random sample of New Zealanders, and to ask them the question. Preferably, the topic of the poll shouldn’t be given to people in advance, so their decision about whether to take part will not be based on the topic in question.

Does self-selection really make much of a difference? 

Yes, it can make an enormous difference.  Here’s an example, also from a Campbell Live poll.

C Live same-sex marriage poll

In April Campbell Live asked people to text in to show their support or opposition to the same-sex marriage bill. The poll showed 22% support for the bill. At the time John Campbell said that he believed that this poll generated the second largest number of responses ever received to one of their polls.  Wow – given that level of response it must be a reliable poll!

No. This result  was markedly different from results of all other random polls carried out on the topic.

Random polls on same-sex marriage

All the random polls asked slightly different questions, used different approaches to random selection, and were carried out at different times. While the result varied from random poll to random poll, you can see they all differed markedly from the Campbell Live self-selecting poll.

You cannot rely on results from self-selecting polls.

Advertisements

3 thoughts on “Campbell Live GCSB poll commits the ultimate sin of survey research

  1. Nobody has randomly selected me to participate in a poll on my thoughts about the Government spying on fellow citizens of my country. Only John Campbell has given me a place to register my opposition. Government MPs who say if you have done nothing wrong you have nothing to fear have to tell me who decides what is wrong!

Make a comment...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s