Goodhart’s law

Goodhart’s law states that when a measure becomes a target, it ceases to be a good measure. Once a metric is used to drive behavior rather than merely observe it, the relationship between the metric and the underlying quality it was meant to represent tends to break down.

The law is named after Charles Goodhart, a Bank of England adviser and London School of Economics economist, who set it out in a 1975 paper on UK monetary management. Goodhart’s original concern was narrow and technical. He observed that any statistical relationship between a monetary aggregate and the goals of policy tends to collapse once the central bank tries to exploit that relationship for control. The broader and now familiar phrasing is due to the anthropologist Marilyn Strathern, who generalized it in a 1997 essay on audit and academic research.

Why it happens

A measure is almost always a proxy for something harder to observe directly. Wait times stand in for quality of care. A test score stands in for competence. A velocity metric stands in for productivity. While the proxy is only being watched, the correlation between it and the underlying quality holds. As soon as people are evaluated, rewarded, or punished on the proxy alone, they have every incentive to move the proxy without moving the thing it tracks. The proxy is easier to game than the underlying quality is to improve.

The breakdown takes several common forms. The target may be met by redefining what counts. It may be met by concentrating effort on the measured cases at the expense of unmeasured ones. Or it may be met by outright manipulation. In each case the metric keeps rising while the underlying quality stays flat or falls.

Consider for example a hospital that was taking too long to admit patients. A response from management may be to set a target for maximum wait times. Since the clock starts ticking once the patient reaches the hospital, ambulance drivers may respond by slowing down their journeys to the hospital. The wait time target is met, but the underlying outcome, prompt treatment, is not.

Related formulations

Goodhart’s law is one of a family of similar observations. Campbell’s law, set out by the social scientist Donald T. Campbell in 1976, makes the same point for quantitative social indicators used in policy-making. The Lucas critique in macroeconomics reaches an analogous conclusion. Statistical regularities observed in past data cannot be relied on once a policy-maker tries to exploit them, because the behavior that produced them changes in response. The cobra effect names the more colorful cases where a perverse incentive actively makes the problem worse rather than merely failing to fix it.

See also

• 10x programmer
• A/B testing
• Fitness function
• Metrics

References

• Goodhart, C. (1975). Problems of Monetary Management: The UK Experience. Papers in Monetary Economics, Vol. I. Reserve Bank of Australia.
• Strathern, M. (1997). "Improving ratings": audit in the British university system. European Review, 5(3), 305–321.
• Campbell, D. T. (1976). Assessing the impact of planned social change. Dartmouth College, The Public Policy Center.