Overcoming Algorithm Aversion: Letting People Slightly Modify an Algorithm's Forecasts
Edited by Paweł Raja, PhD · Published · Updated
In Study 1 of Dietvorst, Simmons and Massey (2018), 32% of participants chose to use an imperfect forecasting algorithm they could not change, compared with 76% when they could adjust its forecasts by up to 10 percentiles and 73% when they could override up to 10 of its 20 forecasts. Those who could modify it performed better as a result.
Participants choosing to use the algorithm's forecasts (Study 1) · Lab experiment
Algorithm could not be changed: 32% → Could adjust each forecast by up to 10 percentiles: 76% (+44.0 pts, +138% relative)
Third condition (could override up to 10 of 20 forecasts): 73%.
Study details
- Study design
- Lab experiment
- Problem
- Evidence-based algorithms usually forecast more accurately than people, yet people often refuse to use one after seeing it make mistakes. Dietvorst, Simmons and Massey (2015) called this algorithm aversion. This paper asks how to reduce it.
- Intervention
- In incentivized forecasting tasks, participants estimated real high-school seniors' percentile scores on a standardized math test. They were told a statistical model was off by 17.5 percentiles on average, then chose between their own forecasts and the model's. In the treatment conditions, choosing the model still allowed some control: adjusting each forecast by up to 10 percentiles, or overriding up to 10 of the 20 forecasts.
- Outcome
- Participants choosing to use the algorithm's forecasts (Study 1): 32% (Algorithm could not be changed) → 76% (Could adjust each forecast by up to 10 percentiles); +44.0 pts, +138% relative
- Sample
- Three incentivized experiments: Study 1 in the Wharton Behavioral Lab; Studies 2-3 online (Study 2: 816 participants)
- Key takeaway
- People want some control over the outcome, not necessarily much. In Study 2, use of the model was about the same whether participants could adjust it by 10, 5 or only 2 percentiles (71%, 71% and 68%, against 47% with no adjustment). A little control raised algorithm use, and with it forecast accuracy.
- Biases involved
- Algorithm Aversion
- Source
- Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64(3), 1155-1170. doi:10.1287/mnsc.2016.2643 ↗ full text (PDF)Numbers checked against: Study 1 results and Figure 2 (32% / 73% / 76%); Study 2 results (47% vs 71% / 71% / 68%); Study 3; General Discussion
What the study did
Study 1 ran in the Wharton Behavioral Lab. Participants earned a bonus of up to $5 that fell as their average forecasting error rose. They saw 20 students, one per page, with the same nine variables the model used. Participants who could modify the model chose it far more often, and because their final forecasts stayed close to the model's, they made smaller errors and earned more.
Study 2 ran online with 816 participants and asked whether the amount of control matters. It barely did: 71%, 71% and 68% chose the model when they could adjust it by 10, 5 or 2 percentiles, compared with 47% when they could not change it. Cutting the permitted adjustment by 80% did not detectably reduce willingness to use the model.
Study 3 looked at what happens after people use the algorithm and get feedback. Participants who had been able to modify the model in the first round were more satisfied with the process, more likely to believe the model was better than they were, and more likely to pick a model-based option for the second round. Most chose to combine their own judgment with the model (51% to 74%) rather than rely only on themselves (11% to 19%) or only on the model (10% to 32%).
In a further test, the option to adjust increased use of another person's forecasts (21% to 41%) about as much as use of the algorithm's (46% to 69%). This suggests the effect reflects a general wish for control over the outcome rather than something specific to algorithms.
Limitations and caveats
- Participants were lab participants and online workers doing a short task for small bonuses. Professional forecasters with high stakes may behave differently.
- All three studies used the same task, forecasting test-score percentiles, with a model whose average error participants were told. Other domains and less transparent models may give different results.
- Only adjustment limits of 2 to 10 percentiles were tested; the authors note they cannot say what happens with much larger or smaller limits.
Further sources
- Our research summary: Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err
Cite this page
Raja, P. (Ed.). (2026, 8 Oct 2026). Overcoming Algorithm Aversion: Letting People Slightly Modify an Algorithm's Forecasts. Behavioral Economics Lab. https://www.behavioraleconomicslab.com/findings/restoring-algorithm-trust
Primary source: Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64(3), 1155-1170.