Human Decisions and Machine Predictions: A Bail Algorithm vs. Judges
Edited by Paweł Raja, PhD · Published · Updated
Using 758,027 New York City cases, Kleinberg et al. (2018) estimate that an algorithmic release rule could have cut the pre-trial detention rate from 26.4% to 15.3% (a 41.9% decline) without raising crime, or cut crime by 24.7% while jailing the same share of defendants. These figures come from a retrospective simulation, not a field trial.
Defendants jailed before trial, holding the crime rate at the judges' level (policy simulation) · Retrospective ML evaluation
Judges' actual decisions: 26.4% → Algorithmic release rule (simulated): 15.3% (−11.1 pts, −42% relative)
A 41.9% decline. Alternatively, at the judges' release rate, the rule would cut crime by 24.7%. These are simulated, not observed, outcomes.
Study details
- Study design
- Retrospective ML evaluation
- Problem
- After an arrest, a judge decides whether the defendant waits for trial at home or in jail. In New York, that decision is supposed to rest on the risk that the defendant will not return to court. Jailing too many people is costly for defendants and taxpayers. Releasing high-risk defendants leads to missed court dates and new crimes.
- What was tested
- Not a nudge: no one's choices were changed. The authors trained a machine-learning model (gradient-boosted decision trees) on information available to judges, such as the current charge and prior record, to predict each defendant's risk. They then simulated release rules based on the model's predictions and compared them with the decisions judges actually made.
- Outcome
- Defendants jailed before trial, holding the crime rate at the judges' level (policy simulation): 26.4% (Judges' actual decisions) → 15.3% (Algorithmic release rule (simulated)); −11.1 pts, −42% relative
- Sample
- 758,027 New York City cases (2008-2013); 110,938-case hold-out test set
- Key takeaway
- Judges' decisions were noisy relative to predicted risk. The riskiest 1% of defendants, when released, failed to appear in court 56.3% of the time and were re-arrested 62.7% of the time, yet judges released 48.5% of them. The authors find the algorithm's gains can be had while also reducing racial disparities in who is jailed.
- Source
- Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human decisions and machine predictions. Quarterly Journal of Economics, 133(1), 237-293. doi:10.1093/qje/qjx032 ↗ sourceNumbers checked against: Abstract; policy-simulation section (release rate 73.6%, crime rate 11.3%, detention 26.4% to 15.3%)
What the study did
The data cover all arrests in New York City from November 2008 to November 2013, of which 758,027 cases faced a pre-trial release decision. The authors set aside 203,338 cases as a "lock box" that they did not touch until the final draft. They trained the model on part of the remaining data and evaluated it on a separate hold-out set of 110,938 cases.
Evaluation faces a selective-labels problem: outcomes are observed only for defendants whom judges released. To test whether the algorithm would do better when releasing people judges had jailed, the authors use the quasi-random assignment of cases to judges who differ in leniency, and they show the conclusions hold under fairly extreme assumptions about what judges see that the data do not.
New York law tells judges to consider flight risk, so the main outcome is failure to appear in court, which the authors call "crime" for short. The gains also show up for re-arrest, including for violent crimes. The authors report similar results in a national dataset.
By training a second model to predict the judges' own decisions, the authors find that a key problem appears to be that judges respond to "noise" as if it were signal. Their decisions vary in ways that do not track defendants' predicted risk.
Limitations and caveats
- This is a retrospective policy simulation, not a randomized trial or a deployed tool. The 24.7% and 41.9% figures are what an algorithmic rule could have achieved on historical data under the authors' assumptions.
- Outcomes are observed only for released defendants. Estimates for releasing people judges jailed rely on imputation and on the judge-leniency checks.
- Judges may care about things other than failure-to-appear risk. The authors check some of these, such as violent crime and racial equity, but cannot rule them all out.
- The NBER working-paper version reports 24.8% and 42.0%; we use the published figures (24.7% and 41.9%).
- This is not a nudge. It is included because it measures how far human judgment falls short of a statistical benchmark, which is the background to research on algorithm aversion.
Further sources
- Our research summary: Human Decisions and Machine Predictions
Cite this page
Raja, P. (Ed.). (2026, 8 Oct 2026). Human Decisions and Machine Predictions: A Bail Algorithm vs. Judges. Behavioral Economics Lab. https://www.behavioraleconomicslab.com/findings/judicial-bail-algorithms
Primary source: Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human decisions and machine predictions. Quarterly Journal of Economics, 133(1), 237-293.