RCTs to Scale: Comprehensive Evidence from Two Nudge Units
Stefano DellaVigna, Elizabeth Linos
Econometrica / NBER · 2022
Abstract
Audits the real-world effectiveness of nudges by analyzing every trial run by two large US 'nudge units,' and compares those effects to the nudges published in the academic literature—an unusually complete test for publication bias.
Methodology
Analysis of 126 randomized trials conducted by government nudge units, covering over 23 million participants, benchmarked against a sample of nudge RCTs published in academic journals.
Findings
Nudge-unit interventions raise take-up by about 1.4 percentage points on average—an order of magnitude below the ~8.7 point average in published academic studies. The gap is explained largely by publication bias and by academic studies' smaller samples and stronger interventions.
Applied nudge
Budget for a realistic effect size (low single-digit points), not the headline numbers from published trials—nudges are cost-effective at scale precisely because they are cheap, not because they are large.
Citation
DellaVigna, S., & Linos, E. (2022). RCTs to scale: Comprehensive evidence from two nudge units. Econometrica, 90(1), 81-116.
↗ Download PDF