How did a 100,000-tests-a-day target make COVID numbers less trustworthy?
Once a government turned 'number of COVID tests' into a target to hit, the number it reported stopped telling anyone how much useful testing was happening.
▶ Start the storyBecause once the count of tests became a target, it stopped tracking what it was meant to measure. When the British government announced a target of 100,000 COVID-19 tests a day, it first defined it as tests actually carried out, and later as maximum testing capacity. When it announced that the target had been met, the number of useful diagnostic tests was far lower than the figure it reported.
100,000
This is a textbook case of what's known as Goodhart's law, named after British economist Charles Goodhart, usually summarized as 'when a measure becomes a target, it ceases to be a good measure.' Goodhart's own 1975 version, written about monetary policy, put it more technically: any observed statistical regularity tends to collapse once pressure is placed on it for control purposes. In economics this ties to rational expectations: people who know how a system rewards and punishes them will optimize their actions to do well inside it, so they start working on the number itself.
The pattern shows up everywhere people are measured and rewarded. An employee rewarded for the number of cars sold each month will sell more cars, even at a loss, because that's what the metric rewards. The correlation between a popular scientific productivity score, the h-index, and actual scientific awards has weakened since the h-index came into widespread use. Because a conservation group's official declaration that a species is extinct can be used to remove legal environmental protections, the group has become more cautious about declaring extinctions at all. Hospitals told to reduce how long patients stay can end up discharging people too early, leading to more emergency readmissions. And under the United States' No Child Left Behind Act, teachers often pushed students through grades before they were ready, to keep their school's funding.
In each case, what the number was meant to describe (useful testing, good science, a species' real status, a patient's recovery) is no longer what drives the number. As Goodhart's law puts it, the measure has become a target, and that is when it stops being a good measure.
Quiz me
0/3
Recap
The moment a number becomes a target, people start optimizing for the number, not for the reality it was supposed to describe.
Surprising fact · When Britain announced it had met its 100,000-tests-a-day target, useful diagnostic tests were far fewer than the reported figure.
Sources (1)
No source, no claim. Every fact in this lesson (15 claims) cites at least one of these.