EI Feature: "Selective Reporting of Placebo Tests in Top Economics Journals"
The EI Editors invite you to browse the July 2024 issue featured article:
"Selective Reporting of Placebo Tests in Top Economics Journals"
- Anna Dreber, Stockholm School of Economics
- Magnus Johannesson, Stockholm School of Economics
- Yifan Yang, Stockholm School of Economics
ABSTRACT: Placebo tests provide incentives to underreport statistically significant tests, a form of reversed p-hacking. We test for such underreporting in 11 top economics journals between 2009 and 2021 based on a pre-registered analysis plan. If the null hypothesis is true in all tests, 2.5% of them should be significant at the 5% level with an effect in the same direction as the main test (and 5% in total). The actual fraction of statistically significant placebo tests with an effect in the same direction is 1.29% (95% CI [0.83, 1.63]), and the overall fraction of statistically significant placebo tests is 3.10% (95% CI [2.2, 4.0]).
First published: 25 March 2024