2021 Best EI Article Award Announced!

"The Influence of Hidden Researcher Decisions in Applied Microeconomics"


The Editors of Economic Inquiry are pleased to congratulate the recipients of the 2021 Best Article Award!

  • Nick Huntington-Klein, Seattle University
  • Andreu Arenas, University of Barcelona & IEB
  • Emily Beam, University of Vermont 
  • Marco Bertoni, Padova University
  • Jeffrey R. Bloem, USDA Economic Research Service
  • Pralhad Burli, Idaho National Laboratory
  • Naibin Chen, Pennsylvania State University
  • Paul Grieco, Pennsylvania State University
  • Godwin Ekpe, Northern Illinois University
  • Todd Pugatch, Oregon State University
  • Martin Saavedra, Oberlin College
  • Yaniv Stopnitzky, University of San Francisco

This is a paper of substantial importance in improving our understanding of empirical research. There has been a credibility crisis sweeping through social science research for the past decade focusing on problems in replicating experiments and in how empirical researchers may be manipulating their data analysis to yield p-values in ranges more attractive to journal referees and editors. This past work has been important in pushing forward a conversation that has led to making all of our empirical work more rigorous and replicable. The part of this which has previously received little attention, despite being as important as those others, is the fact that as empirical researchers we will all make a number of what may seem small decisions about processing our data which we rarely document clearly but which could have substantial impact on our final results. This study highlights this problem by having several teams start with the same raw datasets as two published papers and attempt to answer the same research questions as those original papers. These research teams often end up finding substantively different conclusions than the original studies and even when they do end up with substantively similar conclusions the magnitudes of the estimated effects often vary substantially. This is in part because econometric specifications are often different across teams but also because of how different teams make difference choices on all of those supposedly small issues in processing the data sets into a usable form. These results make it clear how important it is for researchers to clearly document all of those choices and that we should maintain healthy skepticism about any empirical findings which have not been separately replicated in multiple studies. The implications should not be misunderstood to suggest that empirical research cannot be relied on, but rather that we should take more of a Bayesian approach to evaluating findings in that we increase our likelihood of determining a result to be valid only after seeing multiple pieces of evidence in its favor.

We invite you check it out online here...
 "The influence of hidden researcher decisions in applied microeconomics"