23 March 2023
How do you test your communications? Not the way you're doing it
A lot of firms we’ve been speaking to have been panicking about consumer testing over the last few months. The FCA has said that to be compliant with its new Consumer Duty rules – which come into force on 31 July – firms need to be testing the effectiveness of their communications. And while it acknowledges this could take various forms, some firms are struggling to work out exactly what it is they need to do – and possibly getting the wrong end of the stick.
Many that we have spoken to are taking the phrase “consumer testing” very literally. They want someone to help them fill up rooms with focus group participants – and then engineer some kind of comprehension test.
And I’ll confess that for a while, we did consider whether we could create something like that which would be meaningful for companies. But in my gut, I think I always knew this would be a futile exercise – which would probably leave firms with a false sense of security about how clear their communications were.
Fake tests, fake results
We’ve done this kind of exercise before and it is riddled with problems. If you simply put people in a room and ask them questions about clarity of language and messaging – they are prone to give answers which make them sound much more competent than they are. No one likes to feel stupid, so they will generally tell you that something is clear even when it isn’t. And of course consumers don’t know what they don’t know.
If you do create some kind of comprehension test – getting people to read documents and then answer questions – it’s so artificial that it’s a stretch to take the results and say that they will be the same amongst your actual customer base.
Our advice to firms has been to pull back from the very literal interpretation and think about what the FCA is trying to achieve. At the most basic level, the Consumer Duty places a requirement on firms to prove they are working to deliver good outcomes for their customers. For me, this means communications testing should be focused on the outcomes. That means being able to understand whether consumers are taking the actions that they should do in response to your communications – or acting in a way that demonstrates they do.
For example, if you send an insurance renewal letter, you want your customer to do three things – check their information is correct, check the cover is still suitable, and check they are happy with next year’s price. How do you prove that customers have done those things? Well, that’s not easy. But sending them multiple communications, in multiple channels at the right times in run up to renewal can at least put you in a position where you can show you are trying as hard as you can to get them to do those three things.
Do you have the right data?
You also need to have the right data to help you with this task. How many people are opening your emails? How many people are clicking on links within them? How many people are logging into your portal – or calling the call centre – on the days after a mass mailer lands?
For me, testing the effectiveness of communications is all about having the right data to understand the impact they are having.
There are of course other meaningful tests that can be done – such as assessing your documents’ reading ages, and design. Accreditations such as Fairer Finance’s Clear & Simple Mark, or the Plain English Campaign crystal mark provide a “test” of your communications against a set of criteria which are based on evidence around what makes documents easier to read. And if you don’t want to pay firms like us to help you, we’ve published all our criteria and detail on how we assess in booklets which are free to access – such as our guide on Clear Communications for the ABI.
Where the literal definition of consumer testing is relevant is in live online environments. Again, data is the key here – measuring how long customers are dwelling on key pages in your application journeys, and how many are clicking on key links. By using AB testing – you can make tweaks to how you present information and then prove which one is more effective by comparing the performance of the two different options. Testing the actions of real consumers, in live environments is definitely worthwhile – but when it comes to documents, that is very hard to do.
If you’re a large firm, there’s likely to be far more to do than you can possibly manage between now and July. But the key message from the FCA is to prioritise. Priority number one should be getting the right data in place to ensure you can see how effective your communications are, and where you need to do more to prevent those “foreseeable harms”.