Sample size numbers in adverts – what’s the point?

I was on the tube recently with low phone battery so, and this is obscene for a Londoner, decided to look up and check out the adverts – and one stood out.

It was a shampoo advert saying that 9 out of 10 consumers said their hair felt softer after using it. Unsurprisingly, the stat had an asterisk after the number 10. If readers cared enough they could gaze at the font size six at the footer telling them this stat was from an experiment ran showing that 335 out of 368 people agreed in a test. My only thought was – why do we bother telling people this?

Statistics in adverts

Using stats in adverts is pretty common. According to one source, roughly one in 20 advertising strategies use statistics. It’s not surprising either – it’s a compelling way to persuade consumers who may need convincing to buy a product, particularly if they’re unfamiliar with the product or the brand.

I’m all for using stats too. My problem lies with adverts showing sample sizes alongside.

The Advertising Standards Agency’s (ASA) code states that sample sizes don’t need to be included in marketing comms, unless it’d be misleading not to. And to quote its website “if results are based on a robust sample size in which statistically significant findings can be drawn, then there is no need to include the sample size in the ad” (link).

I think very few people understand that a sample size of 368 is robust to make a 9/10 claim statistically meaningful. Readers just assume this to be the case.

It could be argued that the sample sizes are added to show the hundreds of people who they tested, but I don’t believe this to be the case as often it’s put in tiny font in a footer. If it mattered that much, or was that influential, then it would be made more prominent.

Why are asterisks so common?

For me there’s two reasons why this could be happening; either to stop people complaining to the ASA about stats in adverts, or a misunderstanding of ASA’s guidelines. If it’s the former, then I’d be quite surprised as I don’t believe the number of complaints for this reason is great enough and would also take some understanding of sample size calculations. This must be a niche group (though happy to be proved wrong). If it’s due to misunderstanding, then there’s no real excuses.

You might be wondering how this article is related to Fairer Finance or financial services at all – yeh I’m getting there.

The reason I’m harping on about this is simple – to get rid of asterisks and small print in advertising. Or as much of it as possible. Asterisks are prominent in financial services’ advertising and they only add to consumers’ feelings of distrust. It’s hard to trust companies who, even when trying to entice you, appear to be trying to hide something. And they might not even be trying to hide anything – they may just have an over-zealous compliance department.

Obviously, it’s important that claims on adverts are justified. But why not force companies to publish this evidence on their websites? This will give competitors their chance to do the important job of checking up on the facts in their rivals’ adverts. But importantly, it will also remove some of the small print – which we all know, no one reads – from the bottom of the adverts.

I hear we’ve got a new PM starting soon. They can have that policy idea for free.