8 Easy Steps to Open Science: An Annotated Reading List (To Give to your busy Supervisor/Office Mate)

Background: “Open Science” is an umbrella term used to refer to the concepts of openness, transparency, rigour, reproducibility, replicability, and accumulation of knowledge, which are considered fundamental features of science. In recent years, psychological scientists have begun to adopt reforms to make our work better align with these principles and address the current “credibility revolution” (Vazire, 2018). For example, the Society for the Improvement of Psychological Science (SIPS; https://improvingpsych.org/mission/) is a membership society dedicated specifically to improving the methods and practices of the field. The proposed open science reforms are largely a response to realisations that standard research practices undermine fundamental principles of good and open science (e.g., Ioannidis, 2005; Open Science Collaboration, 2015; Simmons, Nelson, & Simonsohn, 2011). Most scientists agree that there is a reproducibility crisis, at least to some extent (Baker, 2016). However, not all psychological scientists have adopted best practices designed to make science more reproducible (Ioannidis, Munafò, Fusar-Poli, Nosek, & David, 2014; O’Boyle, Banks, & Gonzalez-Mulé, 2014). In part, this is because current incentive structures are misaligned with best practices (Bakker, van Dijk, & Wicherts, 2012; Higginson & Munafò, 2016). But there is also confusion and misinformation about what best practices are, whether and why they are necessary, and how to implement them (Houtkoop et al., 2018). In response, researchers have produced many excellent resources for each major facet of open science and methodological reforms that provide detailed instruction, context, and relevant empirical evidence. However, these articles are sometimes technical, are distributed across different journals and domains of psychology, and may be behind paywalls. Students and researchers with little background knowledge of open science may not easily find, access, or understand the resources that exist. Indeed, lack of information on the available resources and incentives for adopting best scientific practices have recently been identified as primary reasons for researchers in psychology not using best scientific practices (Washburn et al., 2018). Thus, an accessible and consolidated guide is needed to outline the best openly accessible resources related to best practices in (psychological) science. Objectives and Approach: In this paper, we provide a comprehensive and concise introduction to open science practices and resources that can help students and researchers with no background knowledge understand and implement best practices. Following the format of an annotated reading list introduced by Etz, Gronau, Dablander, Edelsbrunner, & Baribault (2018), we curate and summarise reviews, tutorials, and metascientific research related to eight topics: Open Science, Open Access, Open Data, Preregistration, Reproducible Analyses, Best Practices in Statistics, Replications, and Teaching Open Science. For each topic, we provide a summary of one publicly available, published, peer-reviewed article and suggest additional readings. Supporting a broader understanding of Open Science issues, this overview should enable researchers to engage with, improve, and implement current open, transparent, reproducible, replicable, and cumulative scientific practices. Conclusions and Implications: One of the greatest barriers preventing psychological scientists from adopting open science practices is a lack of training and easily-accessible guidance. In this paper, we aim to address this barrier by providing a high-level, low-effort summary and overview of issues and solutions surrounding open science. Readers of all backgrounds can consult this text to understand the purpose of specific practices, obtain information about how to implement specific practices, and find more detailed resources.We hope that this paper will benefit individual researchers and the field as a whole. For all of the steps presented in this annotated reading list, the time taken to understand the issue and develop better practices is rewarded in orders of magnitude. On an individual level, time is ultimately saved, errors are reduced, and one’s own research is improved through a greater adherence to openness and transparency. On a field-wide level, the more researchers invest effort in adopting these practices, the closer the field will come toward adhering to scientific norms and the values it claims to espouse.