Modes and Mechanisms of Game-Like Interventions in Intelligent Tutoring
Total Page：16
File Type：pdf, Size：1020Kb
Modes and Mechanisms of Game-like Interventions in Intelligent Tutoring Systems by Dovan Rai A Dissertation Submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in Computer Science April 2016 APPROVED: ______________________ ______________________ Professor Joseph E. Beck Professor Ivon Arroyo Advisor – WPI Co-Advisor - WPI ______________________ ______________________ Professor Charles Rich Dr. Kristen DiCerbo Committee Member - WPI External Committee Member - Pearson i DEDICATION To my parents ii ACKNOWLEDGEMENT I would like to thank the members of my committee for their guidance and support. I am deeply grateful to my advisors Joseph E. Beck and Ivon Arroyo for mentoring me throughout these years and giving me continuous support and encouragement during all phases of my research. I would also like to thank Charles Rich and Kristen DiCerbo for their insightful appraisal and invaluable feedback. During my dissertation period, I feel fortunate to have worked with and received guidance from different faculty members. I am thankful to Neil Heffernan, Ryan Baker, Janice Gobert, Matthew Kam, Beverly Woolf and Tom Murray. I also received tremendous help from my lab-mates and colleagues who not only helped me with my research work but also enriched my days as a graduate student. Heartfelt thanks to my friends Naomi Wixon, Michael Sao Pedro, Yue Gong, Yutao Wang, Zach Broderick, Adam Goldstein along with various Assistment and Mathspring team members. Thanks to my family, friends and relatives for their love and support. I would particularly like to express my gratitude to my friends and family members that have helped me during my stay in United States. Thanks to Deepa Rai, Sumugdha Rayamajhi, Dan Cooper, Shila Pradhan, Chama Rai, Sazu Rai and many others. My thanks also goes to Fulbright Commission for funding me for the three years of my stay in United States and other funding agencies and WPI graduate school for providing me with the funding and logistics in successfully completing my dissertation. iii ABSTRACT While games can be an innovative and a highly promising approach to education, creating effective educational games is a challenge. It requires effectively integrating educational content with game attributes and aligning cognitive and affective outcomes, which can be in conflict with each other. Intelligent Tutoring Systems (ITS), on the other hand, have proven to be effective learning environments that are conducive to strong learning outcomes. Direct comparisons between tutoring systems and educational games have found digital tutors to be more effective at producing learning gains. However, tutoring systems have had difficulties in maintaining students’ interest and engagement for long periods of time, which limits their ability to generate learning in the long-term. Given the complementary benefits of games and digital tutors, there has been considerable effort to combine these two fields. This dissertation undertakes and analyzes three different ways of integrating Intelligent Tutoring Systems and digital games. We created three game-like systems with cognition, metacognition and affect as their primary target and mode of intervention. Monkey's Revenge is a game-like math tutor that offers cognitive tutoring in a game-like environment. The Learning Dashboard is a game-like metacognitive support tool for students using Mathspring, an ITS. Mosaic comprises a series of mini-math games that pop-up within Mathspring to enhance students' affect. The methodology consisted of multiple randomized controlled studies ran to evaluate each of these three interventions, attempting to understand their effect on students’ iv performance, affect and perception of the intervention and the system that embeds it. Further, we used causal modeling to further explore mechanisms of action, the inter- relationships between student’s incoming characteristics and predispositions, their mechanisms of interaction with the tutor, and the ultimate learning outcomes and perceptions of the learning experience. v CONTENTS ACKNOWLEDGEMENT ................................................................................................. iii LIST OF TABLES ............................................................................................................. ix LIST OF FIGURES ............................................................................................................ x 1 Introduction ................................................................................................................. 1 2 Background Research ................................................................................................. 6 2.1 Games and learning: techno-cultural landscape ................................................................. 6 2.2 Games and Learning: threads of academic research ......................................................... 12 2.3 Game: Affordances ........................................................................................................... 18 2.4 Games: Constraints ........................................................................................................... 29 2.5 Empirical evaluation of effectiveness of games in learning ............................................. 33 2.6 Designing Educational Games .......................................................................................... 41 2.7 Effective integration of game design and instruction design ............................................ 45 2.8 Game elements, Game mechanics and Gamification ....................................................... 47 2.9 Intelligent Tutoring Systems and Educational games ...................................................... 52 3 OUR APPROACH .................................................................................................... 55 3.1 Games as affective, cognitive and metacognitive tool ..................................................... 57 3.2 Web of associations and Causal Mechanisms .................................................................. 59 3.3 Research Questions ........................................................................................................... 62 3.4 Description of three systems ............................................................................................. 66 3.4.1 Monkey’s Revenge: Coordinate geometry learning environment ............................ 66 vi 3.4.2 The Learning Dashboard (Student Progress Page) .................................................... 78 3.4.3 Mosaic: Math mini-games ......................................................................................... 97 4 Experiments and Analysis ...................................................................................... 102 4.1 Experiments with Monkey’s Revenge ............................................................................ 102 4.1.1 Mily's World ............................................................................................................ 103 4.1.2 Pilot study with Monkey’s Revenge ....................................................................... 107 4.1.3 Monkey’s Revenge : Experiment Design ................................................................ 107 4.1.4 Randomized Controlled Study- I ............................................................................. 114 4.1.5 Randomized Controlled Study- II ........................................................................... 119 4.1.6 Conclusions, Limitations and Future Work ............................................................ 127 4.2 Experiments with Learning Dashboard (Student Progress Page) ................................... 130 4.3 Experiment with Mosaic ................................................................................................. 143 5 Causal Modeling ...................................................................................................... 150 5.1 Causal Modeling of Monkey’s Revenge: a case study in Causal Modeling .................. 151 5.1.1 Causal modeling and correlation matrix ................................................................. 153 5.1.2 Causal structure, path orientation and domain knowledge ...................................... 157 5.1.3 Causal modeling and multiple regression ............................................................... 161 5.1.4 Causal modeling: confirmatory, exploratory and graphical tool ............................. 164 5.2 Causal modeling: guide for new experiments ................................................................ 168 5.2.1 Causal Modeling of Wayang OutPost ..................................................................... 168 5.2.2 Causal modeling with ASSISTments ...................................................................... 179 5.3 Causal Modeling with Mathspring ................................................................................. 183 5.3.2 Causal modeling of pre-survey variables ................................................................ 189 5.3.3 Causal modeling with within-tutor variables .......................................................... 193 5.3.4 Causal modeling of Pre-survey and within-tutor variables ..................................... 202 vii 5.3.5 Pre-Survey, within-tutor, Post-Survey variables ..................................................... 206 5.3.6 What do these causal models say about