Three years ago, I drafted an editor’s note about the reproducibility crisis and how AJPOR might encourage research that would help to address the crisis. For some unremembered reason, I wrote a different note for the issue and never returned to the note until earlier this month. I’m happy that the note wasn’t published because my thoughts then were somewhat naïve. Three years later, when thinking about the articles that AJPOR has published subsequently, I can see more clearly the serious challenges most researchers encounter reproducing prior research.
The problem with the lack of reproducibility in the social and behavioral sciences (and other sciences) has been noted for about the past 15 years. Multiple factors have been raised as the cause of the problem but let me address only three here – the expectation that researchers, and especially academic researchers, publish original research. A second related reason is that researchers are concerned that replications are less likely to be published. Researchers don’t believe that duplicating other research is novel enough for publication. However, related to AJPOR, I think the most significant challenges are the cultural and linguistic differences across countries.
In the past three years, AJPOR has published many articles on two timely and important topics – fake news and COVID-19. Collectively, the articles on these topics provided sound information and understandings about the experiences in multiple Asian countries. However, the articles had different objectives, used both quantitative and qualitative methods, used different sampling techniques and populations, asked different survey questions, tested different hypotheses, and used different analysis procedures. AJPOR published a few papers where the research was on a similar topic and used somewhat similar methods, but the research projects had different objectives and were not replications in any sense.
It’s clear to me that replicating social and behavioral science studies across countries and cultures is complex and difficult. I’m not certain that if researchers had attempted replications of COVID-19 and fake news research that we would have learned more about the topics than having many researchers approach the topic from multiple perspectives and methods. Replication is important but for many topics, such as those published in AJPOR, diverse research methods may be more useful in the long run.
Replications are important but it’s not clear to me that replications advance science as much as multiple researchers examining the topics using multiple methods and objectives, especially across multiple countries, cultures, and languages. I welcome your thoughts and I would be happy to publish a summary in a later issue of AJPOR.