Maths●●●●●Difficulty 3 of 5

Why is a brilliant performance usually followed by a worse one?

Flight instructors swore that yelling at cadets worked and praise backfired. They were fooled by a statistical illusion.

▶ Start the story

A brilliant performance is usually followed by a worse one because extreme results are partly luck, and luck rarely strikes twice in a row. This is regression toward the mean: when a measurement is unusually high or low, the next one is likely to be closer to average. Picture a test where everyone guesses randomly. The top 10% will, on a retest, score about 50 like everyone else. Real performances mix skill and luck, so the top scorers include some who were lucky. Next time, their luck won't hold.

Why the best tend to slip
  1. Step 1: A result is skill plus luck

    Everyone gets some good or bad luck

  2. Step 2: Pick the top performers

    They include some who were simply lucky

  3. Step 3: Measure again

    Luck resets; the lucky rarely repeat it

  4. Step 4: The group slips towards average

    No cause needed, just chance

The illusion can fool experts. The psychologist Daniel Kahneman once told flight instructors that praise works better than punishment. A veteran instructor objected: when he praised a cadet's perfect manoeuvre, the next one was usually worse; when he screamed at a bad one, the next was usually better. Kahneman realised the instructor was seeing regression to the mean. Great flights tend to be followed by more ordinary ones whatever you say. To prove it, he had the instructors toss coins at a target behind their backs: those who did best the first time mostly did worse the second.

The same trap explains the "sophomore slump" in sport and the supposed curse of appearing on a magazine cover. It's also why, in medicine, a worthless treatment given to the sickest patients can look like it works.

The cure is a comparison group. Only if treated people improve more than similar untreated people do you know something real happened.

Quiz me

0/3

  1. 1.Why do the top scorers on a test tend to score lower on a retest?
  2. 2.What was the flight instructor's mistake?
  3. 3.How do researchers avoid being fooled by regression in medical studies?

Recap

After an extreme result, expect something more ordinary, and use a comparison group before crediting any cause.

Surprising fact · Because we reward good performances and punish bad ones, regression makes punishment look effective and praise look harmful.

Sources (1)

No source, no claim. Every fact in this lesson (22 claims) cites at least one of these.

  1. [1]Regression toward the mean · Wikipedia
More lessons in ➗ Maths (3) See all maths lessons →

One more light on your map.

Get one lesson like this every day, about the things you love. Free, in two or five minutes.

Get the share card for this lesson ↗