Why we study behavior—and how: On the utility of observing action

The replication crisis is both the result of suboptimal theorizing and the consequence of problematic statistics. Psychological science suffers from "parallel" and "mini" theories, as well as analogies that stand-in for well-defined theoretical frameworks. This tendency leads to the frequent reporting of intriguing yet isolated findings, which exist outside of established theories. This practice not only overlooks existing theories but also hinders the integration of knowledge within a comprehensive theoretical framework. Kuhn (1996) suggested that science progresses in phases. He calls the initial stage of young disciplines protoscience. At this stage, disciplines gravitate towards exploratory rather than theory-driven research. His characterization of protoscience perfectly matches the current state of psychological science. According to Kuhn, the maturation of a field requires the development of a research paradigm. A paradigm serves as cohesive framework that unifies theory, measurement instruments, values, and underlying metaphysical assumptions. A paradigm is shared and agreed upon by the research community. While psychology offers such theories they are not widely used. I believe this is the result of the ubiquitous demand to make theoretical contributions. In contrast, paradigmatic research implies that scientists examine auxiliary assumptions while leaving the broader theoretical framework untouched. Rather than changing the theoretical framework, paradigmatic science strives for highly accurate measures and relatively small specifications of theoretical assumptions in the form of auxiliary assumptions. We will discuss whether psychology can adopt a paradigm and how changes in the research and review process can support this change.

Challenges in the relationship between theory and observation, such as the theory-ladenness of observations, can be interpreted as a call for general methods to evaluate theories independently of observational data. In this presentation, I will discuss such methods, with a focus on theories about mechanisms of higher cognitive functions. Central to the discussion is the definition of a 'mechanism' as a set of entities with specific properties that lead to systematic activity. I will explore the following key points as problematic: Theories based on proposing novel, dedicated mechanisms (i.e., introducing new entities), theories based on concepts that are challenging to link to any tangible entity, and theories that are only specified at a singular level of abstraction (i.e., overlooking entities at lower levels). All three aspects relate to the principle of parsimony, which favors theories requiring fewer assumptions. However, this discussion aims to refine the definition of 'assumption,' to reduce ambiguity in applying the parsimony concept, thereby enhancing its utility in evaluating theories in cognitive sciences independently of observational data. (This talk was part of the symposium "Towards a Sustainable Experimental Psychology: Theory, Replicability, Behavior" organized by Lukas Thürmer.)

In a globalised scientific community, it is common that different researchers work on similar topics, especially if these topics are novel or of broad societal interest. In the context of experimental research, this often goes along with each researcher or team of researchers using their own specific methods, including the definitions of the relevant constructs, their operationalisation, and the experimental paradigms. This variability in approaches is advantageous when the aim is to establish the generalisability of robust phenomena. However, in earlier stages of the knowledge generation process, it may be preferrable to use standardised approaches, meaning that different researchers use the same experimental paradigms and operationalisations of dependent and independent variables. Such a standardised approach does not only provide a basis for easy comparability of results, but also enables researchers to identify core phenomena, that is, robust patterns that arise across studies. Knowledge of such core phenomena is relevant because they define what theoretical approaches must be able to explain. I will demonstrate the usefulness of standardised approaches to experimental research in the context of laboratory research on advice taking.

Psychological science entered a crisis when a lack of reproducibility, outright fraud, and the discovery of the prevalence of questionable research practices converged. Since then, funders, scientists, and the public have proposed or even demanded open science as a remedy to increase the accessibility, transparency, and reproducibility/replicability of psychological research. While these practices are generally perceived as positive and laudable, they often require increased effort on the part of the researchers, at least if pursued in a meaningful way (cf. FAIR principles). At ZPID, our mission is to promote open science practices by providing free-of-charge tools that reduce the burden on researchers while ensuring quality standards. One part of my talk will cover some of these tools (e.g., P-TOS, PreReg, PsychArchives), their adoption, and also their weaknesses. Another part will look beyond ZPID to developments in the scientific publication system, such as Registered Reports and PCI Registered Reports. While I will occasionally present empirical data, this talk will mainly reflect my subjective perspective as a former researcher who has moved into a research infrastructure position at an Open Science Institute. This talk was part of the symposium "Towards a Sustainable Experimental Psychology: Theory, Replicability, Behavior" chaired by Lukas Thürmer.