Open Science: Why, How, and Where Is It Going?

The replication crisis is both the result of suboptimal theorizing and the consequence of problematic statistics. Psychological science suffers from "parallel" and "mini" theories, as well as analogies that stand-in for well-defined theoretical frameworks. This tendency leads to the frequent reporting of intriguing yet isolated findings, which exist outside of established theories. This practice not only overlooks existing theories but also hinders the integration of knowledge within a comprehensive theoretical framework. Kuhn (1996) suggested that science progresses in phases. He calls the initial stage of young disciplines protoscience. At this stage, disciplines gravitate towards exploratory rather than theory-driven research. His characterization of protoscience perfectly matches the current state of psychological science. According to Kuhn, the maturation of a field requires the development of a research paradigm. A paradigm serves as cohesive framework that unifies theory, measurement instruments, values, and underlying metaphysical assumptions. A paradigm is shared and agreed upon by the research community. While psychology offers such theories they are not widely used. I believe this is the result of the ubiquitous demand to make theoretical contributions. In contrast, paradigmatic research implies that scientists examine auxiliary assumptions while leaving the broader theoretical framework untouched. Rather than changing the theoretical framework, paradigmatic science strives for highly accurate measures and relatively small specifications of theoretical assumptions in the form of auxiliary assumptions. We will discuss whether psychology can adopt a paradigm and how changes in the research and review process can support this change.

Challenges in the relationship between theory and observation, such as the theory-ladenness of observations, can be interpreted as a call for general methods to evaluate theories independently of observational data. In this presentation, I will discuss such methods, with a focus on theories about mechanisms of higher cognitive functions. Central to the discussion is the definition of a 'mechanism' as a set of entities with specific properties that lead to systematic activity. I will explore the following key points as problematic: Theories based on proposing novel, dedicated mechanisms (i.e., introducing new entities), theories based on concepts that are challenging to link to any tangible entity, and theories that are only specified at a singular level of abstraction (i.e., overlooking entities at lower levels). All three aspects relate to the principle of parsimony, which favors theories requiring fewer assumptions. However, this discussion aims to refine the definition of 'assumption,' to reduce ambiguity in applying the parsimony concept, thereby enhancing its utility in evaluating theories in cognitive sciences independently of observational data. (This talk was part of the symposium "Towards a Sustainable Experimental Psychology: Theory, Replicability, Behavior" organized by Lukas Thürmer.)

In a globalised scientific community, it is common that different researchers work on similar topics, especially if these topics are novel or of broad societal interest. In the context of experimental research, this often goes along with each researcher or team of researchers using their own specific methods, including the definitions of the relevant constructs, their operationalisation, and the experimental paradigms. This variability in approaches is advantageous when the aim is to establish the generalisability of robust phenomena. However, in earlier stages of the knowledge generation process, it may be preferrable to use standardised approaches, meaning that different researchers use the same experimental paradigms and operationalisations of dependent and independent variables. Such a standardised approach does not only provide a basis for easy comparability of results, but also enables researchers to identify core phenomena, that is, robust patterns that arise across studies. Knowledge of such core phenomena is relevant because they define what theoretical approaches must be able to explain. I will demonstrate the usefulness of standardised approaches to experimental research in the context of laboratory research on advice taking.

In 2000, the APA’s Board of Scientific Affairs announced the Decade of Behavior (DoB) to promote the importance and support for the study of human action (Kendall, 2000). Nevertheless, the proportion of published studies that include measures of behavior has declined steadily (e.g., Baumeister Vohs, & Funder, 2007, Banks, Woznyj, & Mansfield, 2021). In this talk, we will provide a personal account on why and how we are enthusiastic about studying behavior. Specifically, we will (a) discuss why studying behavior yields uniquely informative and relevant insights, (b) provide a practical definition of what constitutes a behavioral measure, and (c) present a program of experimental research on intergroup criticism where behavioral measures have allowed us to make unique predictions and yield surprising findings. We will discuss how to increase actual behavioral research.