3 November 2006 (week 5)
Students use intelligent tutors and other types of interactive learning environments in a considerable variety of ways. In this talk, I will present research on automatically detecting and adapting to when students "game the system", attempting to succeed in a learning environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly.
I will present a set of studies that establish that gaming the system is replicably associated with low learning, and evidence that gaming has different effects, depending on when and why students game. I will then present a detector that reliably detects gaming, in order to drive adaptive support. I have validated that this detector transfers effectively between several intelligent tutoring lessons without re-training, despite the lessons varying considerably in their subject matter and user interfaces. In order to make accurate predictions both about which students game, and when a specific student is gaming, using a combination of labeled and unlabeled data at different grain-sizes, this model was trained using a combination of a psychometric modeling framework, Latent Response Models (Maris, 1995) and a machine-learning space-searching technique, Fast Correlation-Based Filtering (Yu and Liu, 2003).
I have used this detector to develop a tutoring lesson which responds to gaming. Within this lesson, a software agent ("Scooter the Tutor") indicates to the student and their teacher whether the student has been gaming recently. Scooter also gives students supplemental exercises, in order to offer the student a second chance to learn the material he/she had gamed through. Scooter reduces the frequency of gaming by over half, and Scooter's supplementary exercises are associated with substantially better learning; Scooter appears to have had virtually no effect on students who do not game.
Bio: Ryan S. J. d. Baker is a Research Fellow in the Learning Sciences Research Insitute at the University of Nottingham. He graduated from Carnegie Mellon University's School of Computer Science in December 2005, with a Ph.D. in Human-Computer Interaction. His interests are are at the intersection of data mining, educational psychology, and human-computer interaction. His long-term goal is to develop a general model of how students choose to use interactive learning environments, how those choices differ between environments, what motivations and antecedents explain those choices, and how environments can detect and respond appropriately to the range of student choices -- with the eventual goal of developing a model which, given an description of an interactive learning environment's user interface and domain ontology, can accurately predict how students will respond to the learning environment.