Effective Features of Feedback in an Intelligent Tutoring System - A Randomized Controlled Field Trial (Pre-Registration)

Individualized, scaffolded feedback and motivation are essential for L2 learning. However, teachers often report that they are unable to cater to each student individually because of time constraints alone. Part of the solution is believed to be provided by intelligent tutoring systems (ITS) that aim at personalizing instructions to users by means of artificial intelligence technology. One such system for L2 learning is the ITS FeedBook for seventh-grade English as L2 learners in German secondary schools. In a first efficacy study with the FeedBook, Meurers et al. (2019) found larger learning gains for students who were provided with automatic, scaffolded, and individualized feedback on targeted grammatical structures as compared with students who only received unscaffolded, corrective feedback. We aim to replicate the findings of the Meurers et al. study by means of a follow-up trial with a larger sample of N = 616 students, testing for the effectiveness of the scaffolded feedback. As a second goal, we will examine whether there are additional positive effects on students' English proficiency regarding targeted grammatical structures if - besides the scaffolded, individualized feedback - criterion-referenced feedback and motivational elements were added to the FeedBook. Criterion-referenced feedback was realized by providing the students with a student dashboard including information about their learning progress and performance level in relation to a set learning goal; motivational elements were realized by adding a pedagogical agent presenting praise sentences after each exercise. In this pre-registration, we describe our theoretical rationales, our research questions and hypotheses, and our statistical analysis plan.