Adverts and percentages – is statistical beauty only skin deep?

I have developed a curious obsession.

Lately, I’ve been taking photographs of adverts for health and beauty products.

Most of them are from television. It’s not the products that interest me. It’s those captions at the bottom used to back up some voiceover assertion. Like this:

(Though it has Helen Mirren, one of my favourite actors, so what’s not to like?)

Some large – or even not so large percentage – is quoted of the number of women (no men’s products yet spotted) who agreed with some statement.

Some more of my photos are at the bottom of this post. Here’s a table of the figures from them:

 Total women %age Total times %age Rounded Rounded divided by Total In words 241 74% 178.34 178 73.9% 178 out of 241 women agreed 207 70% 144.90 145 70.0% 145 out of 207 women agreed 194 72% 139.68 140 72.2% 140 out of 194 women agreed 193 73% 140.89 141 73.1% 141 out of 193 women agreed 182 78% 141.96 142 78.0% 142 out of 182 women agreed 114 88% 100.32 100 87.7% 100 out of 114 women agreed 97 72% 69.84 70 72.2% 70 out of 97 women agreed 53 70% 37.10 37 69.8% 37 out of 53 women agreed 30 96% 28.80 29 96.7% 29 out of 30 women agreed

I’ve set out to two decimal places what the percentage means of the total number surveyed. I’ve also done the calculation using the rounded number of women. It looks like most advertisers round down on their percentages.

So, are these impressive figures? Who knows? The problem is that without knowing how the survey was set up, there are just too many ways in which figures like this might actually be mediocre.

These don’t seem to be like the surveys that you might be used to in political opinion polling – usually of around 1,000 voters with a lot of care taken to get a representative sample.

Nor do they seem to be the results you might associate with drug development with their extensive randomised controlled trials.

Cognitive bias

Psychological research tells us a lot about cognitive biases that can undermine any survey. Most people like to agree to questions put to them, for a start.

Even when we agree, we may disagree! It all depends on how the question is asked. I tweeted this screengrab from Twitter:

Different wording, different results. Different conclusions? Now I remember my quantum mechanics from my uni days and how something can be in two states at the same time – think Schrödinger’s Cat. But this result ain’t quantum. It’s just how phrasing affects people’s considerations.

So, back to the TV adverts. These companies make big profits. To paraphrase one, aren’t you worth it for them to spend a little more on their numbers?