Estimating Psychometric Reliability of Machine-Learning-Based Fluid Reasoning Scores Derived from Smartphone Data

Individual differences in cognitive abilities are known to predict various important life outcomes, making their study a critical area of interest for practitioners and researchers alike. While most research studied cognitive abilities within laboratory or achievement contexts, different lines of research investigated their role in everyday life, repeatedly linking them to our everyday behavior. However, as prior work mainly relied on reported behavior or simulated tasks, the relationship between cognitive abilities and objective behavior in everyday life remains unclear. The recent adaption of smartphone sensing and computational methods in psychology has demonstrated the potential of studying individual differences in real- world settings. In this fashion, the present study leverages digital footprints from everyday smartphone usage to investigate how fluid intelligence, one of the most central cognitive abilities within the Cattell-Horn-Carroll Theory (CHC; McGrew, 2009), is related to objective behavior in everyday life. More specifically, by means of a machine learning approach, we investigate (1) to what extent behavioral patterns in everyday smartphone usage predict fluid intelligence and (2) which behavioral patterns are most important for these predictions. For this purpose, we drew on existing literature to derive a comprehensive overview of behavioral correlates of fluid intelligence in everyday life capturable via logs of everyday smartphone usage. Translating these findings into features of multimodal smartphone usage data (e.g., phone usage duration, app installations, music consumption, typing patterns), we created a list of sensing features that correspond to the theory-based behavioral correlates and are described in this preregistration protocol. Using cross-validation, we will train linear and non-linear machine learning models (e.g., Elastic Net, Random Forest) based on these features and determine their predictiveness for participants’ composite scores of a fluid intelligence test. By means of interpretable machine learning techniques, we will examine which single features and feature groups contribute most to the predictive performance of these models.