← Journal

The Contemporary Marketing Management Journal

Persuasion at Machine Scale

Persuasion at Machine Scale
Persuasion at Machine Scale

Most of what marketing knows about influence was established in conditions that no longer describe how influence is practised.

Scarcity increases desirability. Social proof shifts choice. Defaults are rarely overridden. Losses loom larger than equivalent gains. These findings came from controlled studies on modest samples, and the ethical debate around them was conducted on the same terms: is it acceptable to arrange a choice so that the option you prefer is slightly easier to take? The consensus answer, for decades, was that it is — the effects were small, the customer retained their agency, and every act of communication arranges something.

That consensus was reasonable. It is also obsolete, for a reason that has nothing to do with the techniques and everything to do with the conditions of their application.

Three things that changed

First, optimisation replaced design. A nudge used to be chosen by a person who could be asked to justify it. It is now selected by a system that tries many variants and keeps whichever performs, with no representation of why it worked. Nobody chose the manipulation; it was discovered. And a process that searches a space of persuasive techniques while measuring only conversion will find the ones that exploit confusion just as readily as the ones that aid decision, because it has no way to tell them apart.

Second, the effects compound. A single nudge with a two per cent effect is trivial. A sequence of forty, each individually trivial, arranged across a customer journey and continuously re-optimised, is not — and no one is measuring the sequence, because each element is owned by a different team with its own test.

Third, the asymmetry became structural. The classical defence of persuasion assumed roughly comparable parties: a seller with an argument, a buyer with judgement. When one party has a model of the other's behaviour built from millions of prior interactions, and the other has an afternoon and a search box, describing the exchange as an argument between equals is no longer accurate.

Where the line falls

The useful distinction is not between influence and no influence — there is no such thing as neutral presentation — but between influence that improves a decision and influence that degrades it.

A technique that helps a customer act on what they already want is legitimate. A default that reflects what most people in their position would choose saves them work. A clear statement of scarcity, when the scarcity is real, is information.

A technique that works only because the customer has not understood something is not legitimate, and the test is straightforward: would it survive disclosure? If explaining the mechanism to the customer would cause them to reverse their decision, the mechanism was not helping them decide. It was substituting for their deciding.

This test has the merit of being operational. It can be applied to a specific interface element by a specific team on a specific afternoon, which is more than can be said for most ethical guidance in marketing.

Why self-regulation is failing here

Marketing has largely governed this territory through professional norms rather than rules, and the approach worked tolerably while practitioners chose the techniques themselves. Norms attach to people making decisions.

Automated optimisation removes the decision from the person. There is no moment at which anyone chooses the dark pattern; there is a test result, a metric that improved, and a variant promoted to production. The norm has nothing to attach to, which is why firms with genuinely sincere ethical commitments keep shipping practices those commitments would forbid.

The fix is not more principles. It is constraints in the system: outcomes the optimiser is not permitted to reach, regardless of what they do to conversion. That is a product and engineering decision, and it has to be made before the experiment runs, because afterwards there is only a number that went up and a great deal of reluctance to explain why it should be discarded.

Part of chapter: Chapter 41. Marketing Ethics