The Economics of Replication

Mueller-Langer, Frank; Fecher, Benedikt; Harhoff, Dietmar and Wagner, Gert G. (30. January 2017): The Economics of Replication. Discussion Papers in Economics 2017-3

Abstract

Replication studies are considered a hallmark of good scientific practice. Yet they are treated among researchers as an ideal to be professed but not practiced. To provide incentives and favorable boundary conditions for replication practice, the main stakeholders need to be aware of what drives replication. Here we investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that from 1974 to 2014 less than 0.1% of publications in the top-50 economics journals were replications. We do not find empirical support that mandatory data disclosure policies or the availability of data or code have a significant effect on the incidence of replication. The mere provision of data repositories may be ineffective, unless accompanied by appropriate incentives. However, we find that higher-impact articles and articles by authors from leading institutions are more likely to be subject of published replication studies whereas the replication probability is lower for articles published in higher-ranked journals.

• Item Type: Paper
• Keywords: Replication, economics of science, science policy, economic methodology
• Faculties: Economics; Economics > Munich Discussion Papers in Economics
• Subjects: 300 Social sciences > 330 Economics
• JEL Classification: A1,, B4,, C12,, C13
• Language: English

References

Anderson, R. & Kichkha, A. Replication versus meta-analysis in economics: Where do we stand 30 years after Dewald, Thursby and Anderson? Paper presented at the American Economic Association Annual Meeting (2017).

Andreoli-Versbach, P. & Mueller-Langer, F. Open access to data: An ideal professed but not practised. Research Policy 43, 1621–1633 (2014).

Card, D. & DellaVigna, S. Nine facts about top journals in economics. Journal of Economic Literature 51, 144–161 (2013).

Cyranoski, D. Rise and fall. news@nature (2006). doi:10.1038/news060109-8.

Dewald, W., Thursby, J. & Anderson, R. Replication in empirical economics: The Journal of Money, Credit and Banking Project. American Economic Review 76, 587–603 (1986).

Duvendack, M., Palmer-Jones, R. W. & Reed, W. R. Replications in economics: A progress report. Econ Journal Watch 12, 164–191 (2015).

Fecher, B., Fraessdorf, M. & Wagner, G. G. Perceptions and practices of replication by social and behavioral scientists: Making replications a mandatory element of curricula would be useful. IZA Discussion Paper Series No. 9896 (2016).

Furman, J., Jensen, K. & Murray, F. Governing knowledge in the scientific community: Exploring the role of retractions in biomedicine. Research Policy 41, 276–290 (2012).

Grant, P. Scientific credit and credibility. Nature Materials 1, 139–141 (2002).

Hamermesh, D. Viewpoint: Replication in economics. Canadian Journal of Economics 40, 715–733 (2007).

Hamermesh, D. What is replication? The possibly exemplary example of labor economics. Paper presented at the American Economic Association Annual Meeting (2017).

Hoeffler, J. H. Replication and economics journal policies. Paper presented at the American Economic Association Annual Meeting in Chicago (2017).

Lacetera, N. & Zirulia, L. The economics of scientific misconduct. Journal of Law, Economics, and Organization 27, 568–603 (2011).

Levelt, W. J. M., Drenth, P. & Noort, E. Flawed science: The fraudulent research practices of social psychologist Diederik Stapel. (Commissioned by the Tilburg University, University of Amsterdam and the University of Groningen, 2012).

Organisation for Economic Co-Operation and Development (OECD): Gross domestic expenditure on R-D by sector of performance and field of science: (http://stats.oecd.org/Index.aspx?DataSetCode=GERD_SCIENCE, 2016).

Sukhtankar, S. Replications in development. Paper presented at the American Economic Association Annual Meeting in Chicago (2017).