Evaluating Replicability of Laboratory Experiments in Economics
Colin F. Camerer, Anna Dreber, Eskil Forsell, et al.
Science · 2016
Edited by Paweł Raja, PhD · Published · Updated
Abstract
Provides the first systematic estimate of how well experimental-economics findings replicate, addressing the field's exposure to the broader 'replication crisis' in the social sciences.
Methodology
Pre-registered high-powered replications of 18 laboratory experiments published in two leading economics journals, with replication success judged by significant effects in the same direction and by effect-size comparison.
Findings
A significant effect in the original direction replicated in about 61% of studies (11 of 18), with replication effect sizes averaging around three-quarters of the originals—higher than contemporaneous psychology replication rates but still leaving a substantial share unconfirmed.
Applied nudge
Treat a single published effect as provisional evidence; weight interventions by replication strength and run your own high-powered test before betting a product decision on a headline result.
Citation
Camerer, C. F., Dreber, A., Forsell, E., et al. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433-1436.
↗ Download PDF