Campbell Live is to trying to run the biggest poll in New Zealand history.
Here’s a really important point for anyone wanting to judge the accuracy of this poll – big numbers do not provide representative data!
This poll commits the ultimate sin of survey research – it uses a self-selecting sample.
People choose to take part based on the topic. This means that the poll only represents the New Zealanders who feel strongly enough about the GCSB bill to take part in the poll.
The only way to attempt to gain a representative result is to take a random sample of New Zealanders, and to ask them the question. Preferably, the topic of the poll shouldn’t be given to people in advance, so their decision about whether to take part will not be based on the topic in question.
Does self-selection really make much of a difference?
Yes, it can make an enormous difference. Here’s an example, also from a Campbell Live poll.
In April Campbell Live asked people to text in to show their support or opposition to the same-sex marriage bill. The poll showed 22% support for the bill. At the time John Campbell said that he believed that this poll generated the second largest number of responses ever received to one of their polls. Wow – given that level of response it must be a reliable poll!
All the random polls asked slightly different questions, used different approaches to random selection, and were carried out at different times. While the result varied from random poll to random poll, you can see they all differed markedly from the Campbell Live self-selecting poll.
You cannot rely on results from self-selecting polls.