What does 5 stars really mean?

Star rating systems have long been a handy way of getting an idea whether a product or service is any good. Apparently, it was an American book editor who first came up with the idea (about 100 years ago) - assigning star ratings to short stories. Since then, they've become prominent in reviews of every form of the arts - from theatre to films to art exhibitions. But over the last 10 years, star ratings have started to stretch to just about anything you can think of.

Almost every product you can buy on Amazon will have star ratings from previous customers who have bought the item. Tripadvisor will give you star ratings on just about every restaurant or attraction in the world - not to mention star ratings on places to stay, of course.

And in the world of money, Defaqto will give you star ratings on every insurance or banking product that you might be thinking of buying.

Customers vs the experts

Each of these star ratings are useful in their own way - but they all have slightly different meanings. The AA's hotel ratings, for example, are about the quality of the accommodation and the facilities that are available. You should expect that an AA 5-star rated hotel is luxurious, expensive, and amongst the nicest places to stay in the world.

On Tripadvisor, however, 5 stars is not quite the same. Here - the ratings are provided by people, and not by experts. So the Premier Inn in Bournemouth gets 4.5 stars on Tripadvisor - which reflects the fact that people think it's great for the price that they've paid. That doesn't mean it's luxurious, it means it's perceived to be good value.

In contrast, the Park Lane Sheraton is rated 5 stars by the experts, but just 3 stars on Tripadvisor.

5 stars is not always the best

The same ratings confusion extends to the world of finance. Increasingly, companies like Trustpilot and reviews.co.uk are bringing star ratings to insurer and banking websites. These are based on customer reviews.

But alongside a Trustpilot score, you might also see a Defaqto star rating - which means something else entirely. A five star-rated Defaqto insurance policy is one that has all the bells and whistles - but it may not be any more suitable for you than a 3 star policy.

In fairness, Defaqto has always been perfectly clear on that point - but when its ratings are displayed on other companies' websites, the meaning is rarely obvious to the prospective customer.

Worse still is that the Defaqto ratings drive some very odd behaviour by insurers. In some of our recent work, we've tried to persuade insurers to remove elements of their policy which aren't much use. But we've been told that although no one uses that element of cover, they can't withdraw it or they'll lose Defaqto points. The loss of a few points could see an insurer dropping from five stars to four - which could easily convert to a loss of business.

But surely this is not what Defaqto means to achieve through its ratings. A five star policy might be significantly better than a four star one - but it might just be bulked up with a bunch of extra bits of cover that nobody uses.

Time for a rethink

At their core, I think Defaqto's ratings are useful - but it's time for them to reconsider how they go about their scoring so their ratings provide the right incentives for insurers.

In my view, they should also rethink their use of stars. Although I know it's not their intention, star ratings are open to misinterpretation in the context of all the other star ratings out there. Anyone who uses Gocompare and decides to pay a bit more for a policy with a higher Defaqto star rating may well be paying more for nothing.

As for customer-based star ratings - that's a whole different can of worms. Even if you can trust that the ratings are based on the views of real customers (and that's a big if), there's a bigger question about whether or not customers are the best judges of whether or not they're getting the best deal in the world of money. It's easy to be blissfully ignorant. But I'll save that debate for another blog. Or here's one I made earlier.