The argument in brief· 3–5 min read· full essay 24 pp

Abundant and Uncurated

What humans are still for, once the machine is the better judge.

The case for keeping people in the loop cannot rest on being better at the task. Machine judgement already beats human annotation on accuracy and cost. What survives is narrower and more durable: humans form the criteria and supply the signal from outside the system.

01The usual defence has already failed

The comfortable version of human-in-the-loop says people are the quality bar — machines draft, humans catch what they get wrong. That defence is finished, and pretending otherwise wastes the argument. On accuracy and on cost, applied judgement at scale is now the machine’s job. The essay concedes it up front and reallocates rather than retreating:

“Machines should absorb the application of judgement at scale, where they are demonstrably accurate and radically cheaper. Humans should own two things that cannot be delegated.” The reallocation

Note what that is not. It is not a claim that models are safe, or that review is optional. It is a claim about where the irreplaceable part sits — and it sits upstream of the labelling, in deciding what would count as good.

Model collapse is usually invoked here as the trump card, and the essay declines to lean on it: collapse describes a regime, not a fact about all synthetic data. A model that verifies its own output plateaus at what it already knows — which is a narrower and more defensible statement than the one the discourse usually makes, and it is the one that actually holds.

02Criteria cannot be written in advance

The second thing humans own is harder to automate because it is not a task at all. Evaluation criteria cannot be fixed before you have looked at the outputs. You discover what “good” means for a corpus by reading it, being surprised, and revising the rubric — a loop with no pre-specified target for a machine to optimise toward.

Which is why the economics look the way they do, and why the cost keeps being mistaken for a clerical one:

“Curation looks like a clerical cost because we have arranged for someone else, somewhere else, to absorb it.” On the economics of not looking

That is the sentence to argue with if you want to argue with the essay. The work of forming criteria did not disappear when annotation got cheap; it moved, and it moved somewhere the budget does not show it. The scarcity, in the end, is partly a scarcity of permission — of anyone being paid to look at the data long enough to know what the standard should be.