Enriching the Student Model in an Intelligent Tutoring System
Total Page：16
File Type：pdf, Size：1020Kb
Enriching the Student Model in an Intelligent Tutoring System Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy of the Indian Institute of Technology, Bombay, India and Monash University, Australia by Ramkumar Rajendran Supervisors: Sridhar Iyer (IIT Bombay) Sahana Murthy (IIT Bombay) Campbell Wilson (Monash University) Judithe Sheard (Monash University) The course of study for this award was developed jointly by the Indian Institute of Technology, Bombay and Monash University, Australia and given academic recognition by each of them. The programme was administered by The IITB-Monash Research Academy. 2014 Dedicated to my Parents, Teachers and the Almighty iii iv Thesis Approval The thesis entitled Enriching the Student Model in an Intelligent Tutoring System by Ramkumar Rajendran (IIT Bombay Roll Number: 08407406, Monash Student ID Number: 22117954) is approved for the degree of Doctor of Philosophy Examiners 1. 2. 3. Supervisors 1. 2. 3. Chairman 1. Date: Place: v vi Declaration I declare that this written submission represents my ideas in my own words and where others' ideas or words have been included, I have adequately cited and refer- enced the original sources. I also declare that I have adhered to all principles of aca- demic honesty and integrity and have not misrepresented or fabricated or falsified any idea/data/fact/source in my submission. I understand that any violation of the above will be cause for disciplinary action by the Institute/the Academy and can also evoke penal action from the sources which have thus not been properly cited or from whom proper permission has not been taken when needed. Notice 1 Under the Copyright Act 1968, this thesis must be used only under normal conditions of scholarly fair dealing. In particular no results or conclusions should be extracted from it, nor should it be copied or closely paraphrased in whole or in part without the written consent of the author. Proper written acknowledgement should be made for any assistance obtained from this thesis. Notice 2 I certify that I have made all reasonable efforts to secure copyright permissions for third- party content included in this thesis and have not knowingly added copyright content to my work without the owners permission. Place Signature Date Name vii viii Abstract An intelligent tutoring system is a computer-based self-learning system which pro- vides personalized learning content to students based on their needs and preferences. The importance of a students' affective component in learning has motivated adaptive ITS to include learners' affective states in their student models. Learner-centered emotions such as frustration, boredom, and confusion are considered in computer learning envi- ronments like ITS instead of other basic emotions such as happiness, sadness and fear. In our research we detect and respond to students' frustration while they interact with an ITS. The existing approaches used to identify affective states include human obser- vation, self-reporting, modeling affective states, face-based emotion recognition systems, and analyzing data from physical and physiological sensors. Among these, data-mining approaches and affective state modeling are feasible for the large scale deployment of ITS. Systems using data-mining approaches to detect frustration have reported high accuracy, while systems that detect frustration by modeling affective states not only detect a stu- dent's affective state but also the reason for that state. In our approach we combine these approaches. We begin with the theoretical definition of frustration, and operationalize it as a linear regression model by selecting and appropriately combining features from log file data. We respond to students' frustration by displaying messages which motivate students to continue the session instead of getting more frustrated. These messages were created to praise the student's effort, attribute the results to external factors, to show sympathy for failure and to get feedback from the students. The messages were displayed based on the reasons for frustration. We have implemented our research in Mindspark, which is a mathematics ITS with a large scale deployment, developed by Educational Initiatives, India. The facial observa- tions of students were collected using human observers, in order to build a ground truth database for training and validating the frustration model. We used 932 facial observa- tions data from 27 students to create and validate our frustration model. Our approach shows comparable results to existing data-mining approaches and also with approaches that model the reasons for the students' frustration. Our approach to responding to frus- tration was implemented in three schools in India. Data from 188 students from the three schools, collected across two weeks was used for our analysis. The number of frustration instances per session after implementing our approach were analyzed. Our approach to responding to frustration reduced the frustration instances statistically significantly{(p < 0.05){in Mindspark sessions. We then generalized our theory-driven approach to detect ix other affective states. Our generalized theory-driven approach was used to create a bore- dom model which detects students' boredom while they interact with an ITS. The process shows that our theory-driven approach is generalizable to model not only frustration but also to model other affective states. x Publications from the thesis work • A Theory-Driven Approach to Predict Frustration in an ITS, Ramkumar Rajen- dran, Sridhar Iyer, Sahana Murthy, Campbell Wilson, and Judithe Sheard, IEEE Transactions on Learning Technologies, Vol 6 (4), pages 378{388, Oct-Dec 2013. • Responding to Students' Frustration while Learning with an ITS, To be submitted to the IEEE Transactions on Learning Technologies. • Literature Driven Method for Modeling Frustration in an ITS, Ramkumar Rajen- dran, Sridhar Iyer, and Sahana Murthy, International Conference on Advanced Learning Technologies (ICALT), 2012, Rome, Italy. • Automatic identification of affective states using student log data in ITS, Ramku- mar Rajendran, Doctoral Consortium in International Conference on Artificial In- telligence in Education (AIED), 2011, Auckland, New Zealand. xi xii Contents Page Abstract ix List of Tables xxv List of Figures xxv List of Abbreviations xxvi 1 Introduction 1 1.1 Objective . .4 1.2 Solution Approach . .6 1.3 Contribution . .7 1.4 Thesis structure . .8 2 Background 11 2.1 Intelligent Tutoring System . 11 2.1.1 History of ITS . 12 2.1.2 Generic Architecture of ITS . 13 2.1.3 Research areas in ITS . 16 2.2 Affective Computing . 17 2.2.1 Affective States . 17 2.3 Frustration . 19 2.3.1 Rosenweig's Frustration Theory (1934) . 20 xiii 2.3.2 Frustration Aggression Hypothesis (1939) . 20 2.3.3 Frustration and Goal-Blockage . 22 2.3.4 Experimental Research on Frustration . 22 2.3.5 Definition of Frustration Used in Our Research . 23 2.4 Motivation Theory . 24 2.4.1 Hull's Drive Theory . 24 2.4.2 Lewin's Field Theory . 25 2.4.3 Atkinson's theory of achievement motivation: . 25 2.4.4 Rotter's Social Learning Theory: . 26 2.4.5 Attribution Theory . 26 2.4.6 Discussion . 28 2.5 Responding to Frustration . 29 2.6 Summary . 31 3 Research Platform 33 3.1 Mindspark . 33 3.1.1 Student Interface . 35 3.1.2 Learning Content Database . 38 3.1.3 Browser Log and Student Model . 39 3.1.4 Adaptation Engine . 40 3.1.4.1 Cluster level adaptation logic . 40 3.1.4.2 Adaptation within a cluster . 41 3.1.5 Sparkies . 42 3.2 Discussion . 43 4 Related Work of Affective Computing 45 4.1 Affective States Detection . 45 4.2 Detecting Frustration from the Student's Interactions with the System . 52 4.2.1 Discussion . 56 xiv 4.3 Responding to Affective States . 58 4.3.1 Discussion . 60 4.4 Motivation for Our Work . 61 5 Detect Frustration 63 5.1 Theory-Driven Model to Detect Frustration . 64 5.2 Theory-Driven Model for Mindspark Log Data . 66 5.2.1 Frustration Model for Mindspark . 66 5.2.2 Feature Selection and Combination in the Theory-Driven Approach 70 5.3 Experiment Methodology . 72 5.3.1 Human Observation . 72 5.3.1.1 Sample . 73 5.3.1.2 Video Recording Procedure . 73 5.3.1.3 Instrument . 73 5.3.1.4 Observation Procedure . 74 5.3.1.5 Labeling Technique . 75 5.3.1.6 Universality in Facial Expressions . 75 5.3.2 Analysis Procedure . 77 5.3.3 Metrics to Validate Frustration Model . 78 5.4 Results . 80 5.5 Performance of Data-Mining Approaches Applied to the Data from Mindspark Log File . 83 5.6 Performance of Theory-Driven Constructed Features Applied to Other Clas- sifier Models . 87 5.7 Generalization of our Model Within Mindspark . 89 5.8 Discussion . 91 6 Responding to Frustration 93 6.1 Our Approach to Respond to Frustration . 93 xv 6.1.1 Reasons for Frustration . 95 6.1.2 Motivational Messages . 96 6.1.3 Algorithms to Display Motivational Messages . 98 6.2 Methodology to Show Motivational Messages in Mindspark . 102 6.2.1 Implementation of Affective Computing in Mindspark . 104 6.3 Impact of Motivational Messages on Frustration . 106 6.3.1 Methodology to Collect Data from Mindspark Sessions . 107 6.3.2 Sample . 107 6.3.3 Data Collection Procedure . 107 6.3.4 Data Analysis Technique . 109 6.4 Results . 110 6.4.1 Validation of Impact of Motivational Messages . 113 6.4.2 Detailed Analysis . 115 6.4.2.1 Impact of Motivational Messages on Learning Achievement 117 6.4.2.2 Impact of Motivational Messages on Average Time to An- swer the Questions in Mindspark Session . 118 6.4.2.3 Analysis on Ordering Effects - Removal of Motivational Messages . 119 6.4.3 Discussion . 120 6.5 Summary .