Regression to the mean
Overview
Regression to the mean is the statistical tendency for an extreme measurement to be followed by one closer to the average. If a variable has any random component, the cases selected for being unusually high or low will, on re-measurement, tend to be less extreme, simply because some of what made them extreme was noise.
Why it misleads
- People seek treatment when symptoms are at their worst. Whatever they take next, they will usually feel better afterwards, because "worst" is by definition followed by "less bad". This is the single largest reason that Homeopathy, faith healing, and most folk remedies appear to work.
- The pilot praised after an excellent landing does worse next time; the pilot shouted at after a bad one does better. The instructor concludes that praise is harmful and criticism works. Neither had any effect. (This is Daniel Kahneman's example from Thinking, Fast and Slow, 2011.)
- The "Sports Illustrated curse", the sophomore slump, and the disappointing sequel are all regression to the mean given a name and a cause.
Notes
- The error is a special case of Post Hoc Ergo Propter Hoc: the improvement follows the intervention, so the intervention is credited.
- The defence is a control group. If the untreated improve as much as the treated, the treatment did nothing, however vivid the individual recoveries.