(I wrote the below article for the current edition of Interview – the magazine of the New Zealand Research Association.)
Polling is a small part of what we do as an industry, but as the most publicly facing part it’s important our polls are considered robust measures of voter sentiment. The recent failing of UK polls has brought New Zealand polling under even closer scrutiny. Although polling in New Zealand has stacked up fairly well over recent elections, changes to technology and the way we communicate mean that eventually, approaches will need to change.
Telephone polling still works, for now
For the time being, telephone polling still works. At the last Census 85.5% of households reported having a landline telephone. Although landline ownership will have declined further since then, pollsters have so far had some success at tweaking their approaches or making other adjustments to reduce the impact of non-coverage (and/or non-response). Landline non-coverage and non-response have not yet reached a critical level, but they probably will. So what’s next?
What about more cell phone polling?
Calling cell phones as well as landlines can help to reduce non-coverage error, but due to the number of New Zealanders with both a cell and landline it can be extremely inefficient at doing so. Also, while calling cell phones may reduce one source of error, it could introduce others. For example, generating a representative sample of random cell numbers is made difficult by a lack of accurate data on the structure of the New Zealand cell number system. If your random numbers aren’t representative to start with, your sample of voters won’t be either.
What about more online polling?
Online polling is carried out overseas with varying levels of success. In New Zealand though, I have my doubts that online panels can deliver better estimates than landline polls. We all know online panels suffer from various socio-economic and demographic skews. Pollsters could probably make up for these by setting careful quotas, but given the size of New Zealand panels and high polling frequencies, we’d be going back to some groups of people repeatedly, and in doing so could alter their interest in politics (making them different from similar New Zealanders who are not on the panel).
However the bigger problem with online panels for political polls is a combination of the high frequency of polling and being at the mercy of other projects being carried out using that panel at the same time. If you’re polling during a period of particularly high panel demand, your poll sample that month could well end up being skewed toward people who, say, don’t eat breakfast cereal or aren’t responsible for household shopping. This would ultimately lead to increased volatility of your party support estimates, and your changes between polls could appear odd and unreliable.
So what’s the way forward?
In my view the most promising avenue for forecasting elections lies in data analytics. This means stepping away from tried and true research approaches. It means instead of spending time and money on collecting data and generating representative samples, we spend it on adjusting and modelling unrepresentative data. As an industry we’ve been highly critical of self-selecting polls in the past, but there’s compelling evidence they can be useful, given the right combination of data and analytics skill. For example, a recent study published in the International Journal of Forecasting showed it was possible to forecast 2012 US election results using extremely large samples of highly unrepresentative data from opt-in polls taken over the Xbox gaming platform.
Evidence in New Zealand suggests conventional polling approaches still work, but the range and quantity of data we can access is increasing all the time. This change presents enormous opportunities to our industry, but it may mean reviewing old assumptions and learning new skills.
 Wang, W., Rothschild, D., Goel, D., & Gelman, A. (2015). Forecasting elections with non-representative polls. International Journal of Forecasting, 31, 980-991.