Design Recommendations for Intelligent Tutoring Systems Volume 1 Learner Modeling Edited by: Robert Sottilare Arthur Graesser Xiangen Hu Heather Holden A Book in the Adaptive Tutoring Series Copyright © 2013 by the U.S. Army Research Laboratory. Copyright not claimed on material written by an employee of the U.S. Government. All rights reserved. No part of this book may be reproduced in any manner, print or electronic, without written permission of the copyright holder. The views expressed herein are those of the authors and do not necessarily reflect the views of the U.S. Army Research Laboratory. Use of trade names or names of commercial sources is for information only and does not imply endorsement by the U.S. Army Research Laboratory. This publication is intended to provide accurate information regarding the subject matter addressed herein. The information in this publication is subject to change at any time without notice. The U.S. Army Research Laboratory, nor the authors of the publication, makes any guarantees or warranties concerning the information contained herein. Printed in the United States of America First Printing, July 2013 First Printing (errata addressed), August 2013 U.S. Army Research Laboratory Human Research & Engineering Directorate SFC Paul Ray Smith Simulation & Training Technology Center Orlando, Florida International Standard Book Number: 978-0-9893923-0-3 We wish to acknowledge the editing and formatting contributions of Carol Johnson, ARL Dedicated to current and future scientists and developers of adaptive learning technologies CONTENTS Preface i Section I: Fundamentals of Learner Modeling 1 Chapter 1 ‒ A Guide to Understanding Learner Models .......... 3 Arthur Graesser (University of Memphis) Chapter 2 ‒ Lowering the Barrier to Adoption of Intelligent Tutoring Systems through Standardization............................ 7 Robby Robson (Eduworks Corporation) Avron Barr (Aldo Ventures) Chapter 3 ‒ Important Considerations for Learner Models: Transfer Potential and Pedagogical Content Knowledge.... 15 Alan Lesgold (University of Pittsburgh) Arthur Graesser (University of Memphis) Chapter 4 ‒ Matching Learner Models to Instructional Strategies ................................................................................... 23 Andrew M. Olney (University of Memphis) Whitney L. Cade (University of Memphis) Chapter 5 ‒A Review of Student Models Used in Intelligent Tutoring Systems ................................................... 39 Philip I. Pavlik Jr. (University of Memphis) Keith Brawner (U.S. Army Research Laboratory) Andrew Olney (University of Memphis) Antonija Mitrovic (University of Canterbury) Section II: Current Learner Modeling Tools and Methods 69 Chapter 6 ‒Understanding Current Learner Modeling Approaches ............................................................................... 71 Heather K. Holden (U.S. Army Research Laboratory) Chapter 7 ‒Affective-Behavioral-Cognitive (ABC) Learner Modeling .................................................................... 75 Abhiraj Tomar (University of North Texas) Rodney D. Nielsen (University of North Texas) Chapter 8 ‒The Need for Empirical Evaluation of Learner Model Elements ......................................................... 87 Heather K. Holden (U.S. Army Research Laboratory) Anne M. Sinatra (U.S. Army Research Laboratory) Chapter 9 ‒On the Use of Learner Micromodels as Partial Solutions to Complex Problems in a Multiagent, Conversation-based Intelligent Tutoring System ................. 97 Xiangen Hu (University of Memphis) Donald M. Morrison (University of Memphis) Zhiqiang Cai (University of Memphis) Chapter 10 ‒Learner Models in the Large-Scale Cognitive Modeling (LSCM) Initiative ................................ 111 Scott A. Douglass (U.S. Air Force Research Laboratory) Section III: Emerging Learner Modeling Approaches 127 Chapter 11 ‒Emerging Learner Modeling Concepts ............. 129 Xiangen Hu (University of Memphis) Donald M. Morrison (University of Memphis) Chapter 12 ‒The Need for a Mathematical Model of Intelligent Tutoring ............................................................... 133 Robby Robson (Eduworks Corporation) Xiangen Hu (University of Memphis) Donald M. Morrison (University of Memphis) Zhiqiang Cai (University of Memphis) Chapter 13 ‒Modeling Student Competencies in Video Games Using Stealth Assessment ......................................... 141 Valerie Shute (Florida State University) Matthew Ventura (Florida State University) Matthew Small (Florida State University) Benjamin Goldberg (U.S. Army Research Laboratory) Chapter 14 ‒Assessing the Disengaged Behaviors of Learners .................................................................................. 153 Ryan S.J.d. Baker (Teachers College Columbia University) Lisa M. Rossi (Worcester Polytechnic Institute) Chapter 15 ‒Knowledge Component (KC) Approaches to Learner Modeling .................................................................. 165 Vincent Aleven (Carnegie Mellon University) Kenneth R. Koedinger (Carnegie Mellon University) Chapter 16 ‒Towards Learner Models based on Learning Progressions (LPs) in DeepTutor ......................................... 183 Vasile Rus (University of Memphis) William Baggett (University of Memphis) Elizabeth Gire (University of Memphis) Don Franceschetti (University of Memphis) Mark Conley (University of Memphis) Arthur Graesser (University of Memphis) Section IV: Future Learner Modeling Concepts 193 Chapter 17 ‒ Pushing and Pulling Toward Future ITS Learner Modeling Concepts ................................................. 195 Robert A. Sottilare (U.S. Army Research Laboratory) Chapter 18 ‒ Learner Modeling to Predict Real-Time Affect in Serious Games ....................................................... 199 James Lester (North Carolina State University) Bradford Mott (North Carolina State University) Jonathan Rowe (North Carolina State University) Jennifer Sabourin (North Carolina State University) Chapter 19 ‒Intelligent Creativity Support ............................ 209 Winslow Burleson (Arizona State University) Kasia Muldner (Arizona State University) Chapter 20 ‒Learner Modeling Considerations for a Personalized Assistant for Learning (PAL) ........................ 217 Damon Regan (The Tolliver Group, Inc.) Elaine M. Raybourn (Sandia National Laboratories) Paula J. Durlach (Advanced Distributed Learning Initiative) Chapter 21 ‒Eye-Tracking for Student Modelling in Intelligent Tutoring Systems ................................................. 227 Cristina Conati (University of British Columbia) Vincent Aleven (Carnegie Mellon University) Antonija Mitrovic (University of Canterbury) Chapter 22 ‒Shared Mental Models of Cognition for Intelligent Tutoring of Teams ............................................... 237 J.D. Fletcher (Institute for Defense Analyses) Robert A. Sottilare (U.S. Army Research Laboratory) Biographies 253 Acronyms List 267 Index 273 Design Recommendations for Intelligent Tutoring Systems - Volume 1: Learner Modeling Preface Robert A. Sottilare1, Arthur C. Graesser2, Xiangen Hu2, and Heather Holden1, Eds. U.S. Army Research Laboratory - Human Research and Engineering Directorate1 University of Memphis Institute for Intelligent Systems2 i Design Recommendations for Intelligent Tutoring Systems - Volume 1: Learner Modeling This book is the first in a planned series of books that examine key topics (e.g., learner modeling, instructional strategies, authoring, domain modeling, learning effect, and team tutoring) in intelligent tutoring system (ITS) design through the lens of the Generalized Intelligent Framework for Tutoring (GIFT; Sottilare, Brawner, Goldberg, and Holden, 2012), a modular, service-oriented architecture created to develop standards for authoring, managing instruction, and analyzing the effect of ITS technologies. This preface introduces tutoring functions, provides learner modeling examples, and examines the motivation for standards for the design, authoring, instruction, and analysis functions within ITSs. Next, we introduce GIFT design principles, and finally, we discuss how readers might use this book as a design tool. We begin by examining the concept of learner modeling. Learner modeling (also known as student modeling or user modeling) is one of the major components of ITS. Learner modeling is a key to the computer-based tutor’s understanding of the learner. Comprehensive, real-time modeling of the learner is a critical element in the design and development of truly adaptive tutoring systems that can tailor tutoring experiences to the needs of the individual learner and teams of learners. It is generally accepted that an ITS has four major components (Elson-Cook, 1993; Nkambou, Mizoguchi & Bourdeau, 2010; Graesser, Conley & Olney, 2012; Psotka & Mutter, 2008; Sleeman & Brown, 1982; VanLehn, 2006; Woolf, 2009): The domain model, the student model, the tutoring model, and the user- interface model. GIFT similarly adopts this four-part distinction, but with slightly different corresponding labels (domain module, learner module, pedagogical module, and tutor-user interface) and the addition of the sensor module, which can be viewed as an expansion of the user interface. (1) The domain model contains the set of skills, knowledge, and strategies of the topic being tutored. It normally contains the ideal expert knowledge and also the bugs, mal-rules, and misconceptions that students periodically exhibit. (2) The learner model consists of the cognitive,