Mechanisms, entities, and theory: A discussion of general rules to evaluate theories

The replication crisis is both the result of suboptimal theorizing and the consequence of problematic statistics. Psychological science suffers from "parallel" and "mini" theories, as well as analogies that stand-in for well-defined theoretical frameworks. This tendency leads to the frequent reporting of intriguing yet isolated findings, which exist outside of established theories. This practice not only overlooks existing theories but also hinders the integration of knowledge within a comprehensive theoretical framework. Kuhn (1996) suggested that science progresses in phases. He calls the initial stage of young disciplines protoscience. At this stage, disciplines gravitate towards exploratory rather than theory-driven research. His characterization of protoscience perfectly matches the current state of psychological science. According to Kuhn, the maturation of a field requires the development of a research paradigm. A paradigm serves as cohesive framework that unifies theory, measurement instruments, values, and underlying metaphysical assumptions. A paradigm is shared and agreed upon by the research community. While psychology offers such theories they are not widely used. I believe this is the result of the ubiquitous demand to make theoretical contributions. In contrast, paradigmatic research implies that scientists examine auxiliary assumptions while leaving the broader theoretical framework untouched. Rather than changing the theoretical framework, paradigmatic science strives for highly accurate measures and relatively small specifications of theoretical assumptions in the form of auxiliary assumptions. We will discuss whether psychology can adopt a paradigm and how changes in the research and review process can support this change.

In a globalised scientific community, it is common that different researchers work on similar topics, especially if these topics are novel or of broad societal interest. In the context of experimental research, this often goes along with each researcher or team of researchers using their own specific methods, including the definitions of the relevant constructs, their operationalisation, and the experimental paradigms. This variability in approaches is advantageous when the aim is to establish the generalisability of robust phenomena. However, in earlier stages of the knowledge generation process, it may be preferrable to use standardised approaches, meaning that different researchers use the same experimental paradigms and operationalisations of dependent and independent variables. Such a standardised approach does not only provide a basis for easy comparability of results, but also enables researchers to identify core phenomena, that is, robust patterns that arise across studies. Knowledge of such core phenomena is relevant because they define what theoretical approaches must be able to explain. I will demonstrate the usefulness of standardised approaches to experimental research in the context of laboratory research on advice taking.

Psychological science entered a crisis when a lack of reproducibility, outright fraud, and the discovery of the prevalence of questionable research practices converged. Since then, funders, scientists, and the public have proposed or even demanded open science as a remedy to increase the accessibility, transparency, and reproducibility/replicability of psychological research. While these practices are generally perceived as positive and laudable, they often require increased effort on the part of the researchers, at least if pursued in a meaningful way (cf. FAIR principles). At ZPID, our mission is to promote open science practices by providing free-of-charge tools that reduce the burden on researchers while ensuring quality standards. One part of my talk will cover some of these tools (e.g., P-TOS, PreReg, PsychArchives), their adoption, and also their weaknesses. Another part will look beyond ZPID to developments in the scientific publication system, such as Registered Reports and PCI Registered Reports. While I will occasionally present empirical data, this talk will mainly reflect my subjective perspective as a former researcher who has moved into a research infrastructure position at an Open Science Institute. This talk was part of the symposium "Towards a Sustainable Experimental Psychology: Theory, Replicability, Behavior" chaired by Lukas Thürmer.

In 2000, the APA’s Board of Scientific Affairs announced the Decade of Behavior (DoB) to promote the importance and support for the study of human action (Kendall, 2000). Nevertheless, the proportion of published studies that include measures of behavior has declined steadily (e.g., Baumeister Vohs, & Funder, 2007, Banks, Woznyj, & Mansfield, 2021). In this talk, we will provide a personal account on why and how we are enthusiastic about studying behavior. Specifically, we will (a) discuss why studying behavior yields uniquely informative and relevant insights, (b) provide a practical definition of what constitutes a behavioral measure, and (c) present a program of experimental research on intergroup criticism where behavioral measures have allowed us to make unique predictions and yield surprising findings. We will discuss how to increase actual behavioral research.