Human Decisions and Machine Predictions
Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, Sendhil Mullainathan
The Quarterly Journal of Economics / NBER · 2018
Abstract
Uses pretrial bail as a high-stakes test of whether machine-learning predictions can improve on human judges, while confronting the 'selective labels' problem that makes such comparisons hard.
Methodology
Builds a machine-learning model to predict a defendant's risk of failing to appear or reoffending using data available to judges, then evaluates policy simulations against actual judicial release decisions in New York City, carefully addressing the fact that outcomes are only observed for released defendants.
Findings
Algorithmic release rules could cut crime by up to about 24.8% with no increase in jail populations, or reduce jail populations by up to about 41.9% with no rise in crime. Judges misrank defendants and appear to overreact to salient but weakly predictive features, leaving large welfare gains on the table.
Applied nudge
Where consistent prediction beats human intuition, position the algorithm as the default recommendation and design the interface so experts override it only with cause—curbing noise-driven, salience-based deviations.
Citation
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human decisions and machine predictions. Quarterly Journal of Economics, 133(1), 237-293.
↗ Download PDF