AI and deceptive patterns: who are the winners and losers?

While I was on paternity leave, Sheldon Mills published his review into AI for the FCA. Lots to unpack in it (and if you haven’t yet, listen to our fascinating interview with Sheldon on the Fairer Finance podcast).

One thing jumped out to me: what is the end result of AI and deceptive patterns on different groups of consumers...?

Deceptive patterns are deployed by firms to influence online consumer behaviour, resulting in worse consumer outcomes.

Some deceptive patterns are obvious (10 steps to unsubscribe, anyone?). Some are a little sneakier (the big red button says, ‘continue my subscription’, and the small grey text underneath says ‘cancel’).

In theory, AI might make it easier for firms to use deceptive patterns, and use more effective, personalised deceptive patterns. As Sheldon says in his review:

Indeed, one of the more distinctive risks in AI-mediated journeys is the potential for highly personalised forms of deceptive design. In today’s digital markets, firms can already use presentation, defaults, friction and prompts to steer consumers towards particular choices. AI could amplify this risk by enabling firms to identify which forms of presentation, prompts or friction are most effective for each individual consumer. These steers can be adapted in real time and deployed at scale. This may allow firms to exploit consumers’ immediate circumstances, behavioural data, preferences, or vulnerabilities in ways that are more precise and difficult to detect than in conventional digital consumer journeys.

Of course, prudent firms realise that they have two good reasons not to deploy deceptive patterns in the first place: building long-term trust in their brand and avoiding the risk of regulatory intervention. But we might still end up with more deceptive patterns, especially in unregulated markets.

However, AI also gives consumers a tool to fightback. Again, Sheldon points out that:

Conversely, an independent AI agent - for example, where a consumer creates their own agent rather than using one from a firm - could protect consumers by filtering manipulative prompts, managing and controlling access to data, comparing options consistently across the market, and counteracting behavioural biases. This might offer even greater protection for consumers.

AI might make deceptive patterns more widespread, but some consumers will be able to insulate themselves from this. I’ve visualised this below.

In reality, each deceptive pattern affects different people differently. The y axis is therefore more of an average.

In fact, those consumers with their own AI agent could end up making better decisions than ever before. But what about those left behind?

Who are the winners and losers?

Perhaps the winners will be…

• More tech-savvy consumers, who are more confident in using their own AI agent?

• More engaged consumers, who know about the risks of deceptive patterns?

• Wealthier consumers, who can more easily afford to pay for their own AI agent?

This could be troubling. What if those least able to detect and avoid deceptive patterns were most exposed in the new world?

Perhaps a future policy aim should be fair access to the AI tools that will help consumers navigate important financial decisions with care.