In brief:
- Poll results can be misleading without full context, such as methodology and sample size.
- Media often omits key details, making it harder to assess the accuracy of poll results.
- Leading or emotionally charged questions can influence responses and skew outcomes.
- Transparency is needed to ensure poll results are accurate and reliable.
Poll results often leave out important details
In political polling, much weight is given to the results, but these answers are only as valuable as the full context in which they were produced.
The way polling questions are crafted, the sample size, and the methodology may greatly influence the answers. Without knowing these details, the results may be misleading or incomplete.
As professional pollsters have pointed out, no poll is perfect, and how results are framed can create more confusion than clarity.
The need for transparency in polling
Despite the importance of transparency, media reports often omit details about the methodology, sample size, and the exact questions behind poll results. Without these, it’s hard to gauge the accuracy of the results.
For example, an RNZ article on a 1News-Verian poll leading up to the 2023 elections showed political standings but failed to reveal the exact questions asked. Similarly, a New Zealand Herald piece discussed demographic trends without explaining the poll’s methodology. These gaps make it difficult for readers to fully interpret the data and raise questions about the reliability of conclusions.
In another 1News article, Poll: Swarbrick says Kiwis already ‘incredibly frustrated’ with Govt, the results were presented without disclosing the specific wording of the questions.
In this case, TVNZ political editor Maiki Sherman’s coverage of the 1News-Varian poll generated over 300 complaints filed with the Broadcasting Standards Authority (BSA) regarding the “frothy” and dramatic presentation of the results.
Despite this, the BSA found the coverage balanced, stating that Sherman’s strong language fell within acceptable standards of political analysis and journalistic freedom and that their high standards for what constitutes a breach had not been met.
Given the high number of complaints, it raises the question of whether the BSA should consider transparency issues in its assessments.
Polling questions are not neutral
Polling questions are rarely neutral, and the phrasing or context in which they are asked can easily shape responses. For instance, emotionally charged or leading questions can prime respondents to answer in ways they might not have otherwise. As Satherley, Sporle, and Greaves noted, “responses to a poll question can vary markedly depending on how it is asked.”
This shows why pollsters and the media should provide full transparency regarding the exact wording and context of the questions to ensure accuracy and trust.
Is it worse to manipulate the perception of the results or the results themselves?
When a pollster compromises the methodology—through biased questions, self selected respondents, insufficient sample sizes, or incomplete data—it undermines the foundation of the poll and can produce misleading results.
On the other hand, a journalist who frames poll results in a skewed or sensational manner can distort the public’s understanding of the findings, even if the data itself is sound.
Both attempts to distort the results may help with advocating the proponent’s favoured result, in the short run. However, manipulating either the results or their presentation can erode public trust in the process.
