The dust is … well … not settling.
The only poll that ever mattered – the actual vote – has delivered a Brexit majority.
Last year, after the pollsters failed to detect a Conservative majority was on the cards, pollsters held an extensive industry-wide inquiry.
The much-respected academic institution, NatCen, followed the inquiry’s findings. It found that at the time of its survey the UK would have voted 53:47 in favour of Remain.
The outcome, of course, is not in line with that finding. Many other pollsters found a small Remain lead in the final days.
On the headline figures, not just NatCen but many pollsters were wrong.
For many PR professionals, polls and surveys are the go-to tool of their campaigns. So, can they have any confidence in them any more?
The clue is in that word: confidence.
Every poll comes with a measure of its uncertainty – often called a confidence interval, a credible interval or, most commonly, the margin of error.
In the skills guide I put together for the CIPR on using statistics in public relations, I said “be certain about your uncertainty”.
What does this mean?
A rule of thumb is that any figure reflects the true situation within three percentage points both up and down. Any decent pollster will tell you clearly what the particular interval is.
So for any poll that found Remain or Leave on 50 per cent, then the true figure was most likely to be in the range 50-3 per cent and 50+3 per cent, that is somewhere between 47 and 53 per cent.
But there’s more. There’s more to this likelihood. Because it too tells us something about the certainty.
That interval is not 100 per cent certain. The true figure could, actually, be higher or lower than the plus-or-minus-three suggests. It is often that the certainty is given at the 95 per cent level. That is, if the poll had been done 20 times, 19 of them would have the true figure in their respective interval, though not necessarily the same interval.
There is even more, though.
Any poll is only a snapshot of opinion at the time those opinions were sought. In the referendum, as with elections, opinions can switch rapidly.
And, any poll is only as good as its methodology. If there is a fundamental flaw, then it doesn’t matter how certain its results are. It’s like taking a temperature with a thermometer that has a flawed scale. The figure might go to tenths of a degree, and so be very precise, but it’s no use if it tells you that your temperature is 44 degrees!
PRs should not fret. But they should make sure they’re fully aware of the certainties and uncertainties. Read your pollsters’ small print. And design your campaigns, and advise your clients accordingly.
If you need to …
● make sense of the numbers, data and statistics you work with daily
● have greater impact with the data you collect and the surveys you commission
● develop confidence talking and writing about the numbers that matter