Preliminary Sample and Dummy/Blinded Data for the Stage 1 Registered Report: Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template

Background: Preregistration, the practice of publishing a research plan before data collection or analysis, can help to reduce researcher degrees of freedom, but its value depends on whether researchers specify their study plan in sufficient detail and adhere to this plan. Previous findings indicate better, but still insufficient, specificity when preregistrations are based on structured compared to unstructured template formats. The planned study aims to build on these findings and investigate the specificity of preregistrations based on the PRP-QUANT Template, a comprehensive template that draws on insights from previous templates and aids the preregistration of quantitative studies in psychology. Additionally, we will inspect whether peer-review can help to restrict researcher degrees of freedom by comparing peer-reviewed and non-peer-reviewed preregistrations created with the PRP-QUANT Template. Lastly, adherence to the preregistered plans and risk of bias in reporting will be inspected in the publications associated with the examined preregistrations. Methods: Preregistrations based on the PRP-QUANT Template will be sampled from PsychArchives (non-peer-reviewed: n = 46, peer-reviewed: n = 27) and coded for their level of specificity using the coding scheme of Bakker et al. (2020) and Heirene et al. (2021). In addition, adherence and risk of bias in reporting will be coded for all identified accompanying publications. Wilcoxon-Mann-Whitney U tests will be used to compare the median specificity scores of preregistrations based on the PRP-QUANT Template versus preregistrations based on the Prereg Challenge Registration, and peer-reviewed PRP-QUANT preregistrations versus non-peer-reviewed PRP-QUANT preregistrations. Adherence and risk of bias in reporting will be analyzed descriptively.

Preregistration can help to restrict researcher degrees of freedom and thereby ensure the integrity of research findings. However, its ability to restrict such flexibility depends on whether researchers specify their study plan in sufficient detail and adhere to this plan. Previous research indicates higher restrictiveness when preregistrations are based on structured versus unstructured template formats, although there is room for further improvement. The planned study aims to build on these findings and investigate the restrictiveness of preregistrations based on the PRP-QUANT Template, an extensive template that aids the preregistration of quantitative studies in psychology. Preregistrations will be sampled from PsychArchives and coded for their level of restrictiveness using the coding scheme of Bakker et al. (2020) and Heirene et al. (2021). We predict that preregistrations based on the PRP-QUANT Template (N = [74]) are more restrictive than preregistrations based on the OSF Preregistration Template (N = 52, Bakker et al., 2020, hypothesis 1). We will also inspect whether peer review can contribute further to restricting flexibility and predict higher restrictiveness for peer-reviewed (n = [27]) than non-peer-reviewed preregistrations (n = [47], hypothesis 2), using nested Wilcoxon-Mann-Whitney tests. Additionally, we will examine adherence to the preregistered plans in the associated publications (N = [17]). [In line/in contrast] to hypothesis 1, PRP-QUANT preregistrations [had significantly/did not have] higher restrictiveness scores than OSF Preregistrations. Moreover, [consistent/inconsistent] with hypothesis 2, peer-reviewed preregistrations [had significantly/did not have] higher restrictiveness than non-peer-reviewed ones. […] percent of the associated articles included undeclared deviations. We discuss the implications of our findings for the PRP-QUANT Template and structured templates in general. Please note: This is a Stage 1 Registered Report that will be submitted to PCI RR. To maximize transparency in the further process, we have already formulated the results section and a description of the results in the abstract in past tense, but the analyses of this study have yet to be carried out. The results section is based on dummy/blinded data and, thus, values are nonsensical. To facilitate review, we have highlighted text parts that will be edited in brackets and color. In Stage 2, we will change the tense to past and append discussion and conclusion sections.

R Markdown file for the Stage 1 Registered Report: Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template, and all files necessary to knit the RMD

Coding Schemes for the Stage 1 Registered Report: Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template

Analysis Scripts for the Stage 1 Registered Report: Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template