Chapter 10: Emotional Learning Analytics
Sidney K. D’Mello
Departments of Psychology and Computer Science & Engineering, University of Notre Dame, USA DOI: 10.18608/hla17.010
ABSTRACT
This chapter discusses the ubiquity and importance of emotion to learning. It argues that substantial progress can be made by coupling the discovery-oriented, data-driven, analytic PHWKRGVRIOHDUQLQJDQDO\WLFV /$ DQGHGXFDWLRQDOGDWDPLQLQJ ('0 ZLWKWKHRUHWLFDO advances and methodologies from the affective and learning sciences. Core, emerging, and future themes of research at the intersection of these areas are discussed. Keywords: Affect, affective science, affective computing, educational data mining
At the "recommendation" of a reviewer of one of my I was done. I felt contentment , relief , and a bit of pride . SDSHUV ' 0HOOR ,UHFHQWO\VRXJKWWROHDUQD $VWKLVH[DPSOHLOOXVWUDWHVWKHUHLVDQXQGHUFXUUHQW QHZ IRUPH VWDWLVWLFDOPHWKRGFDOOHGJHQHUDOL]HG of emotion throughout the learning process. This DGGLWLYHPL[HGPRGHOV *$00V 0F.HRZQ 6QHGGRQ is not unique to learning as all "cognition" is tinged *$00VDLPWRPRGHODUHVSRQVHYDULDEOHZLWK with "emotion". The emotions may not always be an additive combination of parametric and nonpara- FRQVFLRXVO\H[SHULHQFHG 2KPDQ 6RDUHV EXW metric smooth functions of predictor variables, while WKH\H[LVWDQGLQĂXHQFHFRJQLWLRQQRQHWKHOHVV$OVR addressing autocorrelations among residuals (for emotions do not occur in a vacuum; they are deeply WLPHVHULHVGDWD $WāUVW,ZDVPLOGO\ displeased at intertwined within the social fabric of learning. It the thought of having to do more work on this paper. does not take much to imagine the range of emotions Anxiety resulted from the thought that I might not H[SHULHQFHGE\WKHW\SLFDOVWXGHQWZKRVHSULQFLSOH have the time to learn and implement a new method RFFXSDWLRQLVOHDUQLQJ3HNUXQDQG6WHSKHQV to meet the revision deadline. So I did nothing. The call these "academic emotions" and group them into anxiety transitioned into mild panic as the deadline four categories. Achievement emotions (contentment, DSSURDFKHG,āQDOO\GHFLGHGWRORRNLQWR*$00VE\ DQ[LHW\DQGIUXVWUDWLRQ DUHOLQNHGWROHDUQLQJDFWLYL - downloading a recommended paper. The paper had ties (homework, taking a test) and outcomes (success, some eye-catching graphics, which piqued my curiosity failure). Topic emotions are aligned with the learning DQGPRWLYDWHGPHWRH[SORUHIXUWKHU7KH curiosity content (empathy for a protagonist while reading clas- quickly turned into interest as I read more about the sic literature). Social emotions such as pride, shame, method, and eventually into excitement when I real- and jealousy occur because education is situated in L]HGWKHSRZHURIWKHDSSURDFK7KLVPRWLYDWHGPHWR VRFLDOFRQWH[WV)LQDOO\ epistemic emotions arise from slog through the technical details, which led to some cognitive processing, such as surprise when novelty intense emotions - confusion and frustration when is encountered or confusion in the face of an impasse. things did not make sense, despair when I almost gave up, hope when I thought I was making progress, and Emotions are not merely incidental; they are func- eventually, delight and happiness when I actually did WLRQDORUWKH\ZRXOGQRWKDYHHYROYHG 'DUZLQ make progress. I then attempted to apply the method Tracy, 2014). Emotions perform signalling functions WRP\GDWDE\PRGLI\LQJVRPH5V\QWD[0RUH confu- 6FKZDU] E\KLJKOLJKWLQJSUREOHPVZLWKNQRZO- sion , frustration , and despair interspersed with hope , edge (confusion), problems with stimulation (boredom), delight , and happiness . I eventually got it all working FRQFHUQVZLWKLPSHQGLQJSHUIRUPDQFH DQ[LHW\ DQG and wrote up the results. Some more emotions oc- challenges that cannot be easily surpassed (frustra- curred during the writing and revision cycles. Finally, tion). They perform evaluative functions by serving as the currency by which people appraise an event in
CHAPTER 10 EMOTIONAL LEARNING ANALYTICS PG 115 terms of its value, goal relevance, and goal congruence adopts a data-driven analytic approach to study the ,]DUG (PRWLRQVSHUIRUP modulation functions LQFLGHQFHDQGLQĂXHQFHRIHPRWLRQVRQWKHSURFHVVHV E\FRQVWUDLQLQJRUH[SDQGLQJFRJQLWLYHIRFXVZLWK and products of learning. In this chapter, I highlight negative emotions engendering narrow, bottom-up, some of the core, emerging, and future themes in this and focused modes of processing (constrained focus) interdisciplinary research area. %DUWK )XQNH 6FKZDU] LQFRPSDULVRQWR Let us begin with a note on terminology. Emotion is positive emotions, which facilitate broader, top-down, related, but not equivalent to motivation, attitudes, JHQHUDWLYHSURFHVVLQJ H[SDQGHGIRFXV )UHGULFNVRQ preferences, physiology, arousal, and a host of other & Branigan, 2005; Isen, 2008). Indeed, emotions per- constructs often used to refer to it. Emotions are also vade thought as is evident by their effects on memory, distinct from moods and affective traits (Rosenberg, problem solving, decision making, and other facets of 1998). Emotion is not the same as a feeling. Hunger is a cognition (see Clore & Huntsinger, 2007, for a review). feeling but is not an emotion. Neither is pain. There is 6RZKDWH[DFWO\LVDQHPRWLRQ"7UXWKEHWROGZHGR also some contention as to what constitutes an emotion. QRWUHDOO\NQRZRUDWOHDVWZHGRQRWIXOO\DJUHH ,] - Anger is certainly an emotion, but what about con- ard, 2010). This can be readily inferred from the most IXVLRQ"&RQIXVLRQKDVDIIHFWLYHFRPSRQHQWV IHHOLQJV recent debates on the psychological underpinnings RIEHLQJFRQIXVHGFKDUDFWHULVWLFIDFLDOH[SUHVVLRQV of emotion - a debate sometimes referred to as the ' 0HOOR *UDHVVHUE EXWWKHUHLVVRPHGHEDWH "100 year old emotion war" (Lench, Bench, & Flores, DVWRZKHWKHULWLVDQHPRWLRQ +HVV 5R]LQ /LQGTXLVW6LHJHO4XLJOH\ %DUUHWW )RU - Cohen, 2003). Thus, in the remainder of this chapter, tunately, there is general agreement on the following I use the more inclusive term "affective state" rather key points. Emotions are conceptual entities that arise than the more restrictive term "emotion". IURPEUDLQ×ERG\×HQYLURQPHQWLQWHUDFWLRQV%XW\RX ZLOOQRWāQGWKHPE\ORRNLQJLQWKHEUDLQWKHERG\ CORE THEMES or the environment. Instead, emotions emerge (Lewis, ZKHQRUJDQLVP×HQYLURQPHQWLQWHUDFWLRQVWULJJHU I selected the following four themes to highlight the use changes across multiple time scales and at multiple RI/$('0PHWKRGVWRVWXG\DIIHFWGXULQJOHDUQLQJ, levels - neurobiological, physiological, behaviourally DOVRUHYLHZRQHRUWZRH[HPSODU\VWXGLHVZLWKLQHDFK H[SUHVVLYHDFWLRQRULHQWHGDQGFRJQLWLYHPHWDFRJ - theme in some depth rather than cursorily reviewing QLWLYHVXEMHFWLYH7KHHPRWLRQLVUHĂHFWHGLQWKHVH PDQ\VWXGLHV7KLVPHDQVWKDWPDQ\H[FHOOHQWVWXGLHV changes in a manner modulated by the ongoing sit- JRXQPHQWLRQHGEXW,OHDYHLWWRWKHUHDGHUWRH[SORUH XDWLRQDOFRQWH[W7KHVDPHHPRWLRQDOFDWHJRU\ HJ the body of work within each theme. I recommend DQ[LHW\ ZLOOPDQLIHVWGLIIHUHQWO\EDVHGRQDWULJJHULQJ review papers, when available, to facilitate this process. HYHQW 7UDF\ WKHVSHFLāFELRORJLFDOFRJQL - Affect Analysis from Click-Stream Data WLYHPHWDFRJQLWLYHSURFHVVHVLQYROYHG *URVV 2QHRIWKHPRVWEDVLFXVHVRI/$('0WHFKQLTXHV 0RRUV DQGVRFLRFXOWXUDOLQĂXHQFHV 0HVTXLWD is to use the rich stream of data generated during %RLJHU 3DUNLQVRQ)LVFKHU 0DQVWHDG interactions with learning technologies in order to )RUH[DPSOHDQDQ[LHW\LQGXFLQJHYHQWZLOOWULJJHU XQGHUVWDQGOHDUQHUV FRJQLWLYHSURFHVVHV &RUEHWW GLVWLQFWHSLVRGHVRIDQ[LHW\GHSHQGLQJRQWKHVSH - $QGHUVRQ 6LQKD-HUPDQQ/L 'LOOHQERXUJ FLāFFLUFXPVWDQFH SXEOLFVSHDNLQJWHVWWDNLQJ WKH 2014). A complementary set of insights can be gleaned WHPSRUDOFRQWH[W RQHGD\YVRQHPLQXWHEHIRUHWKH ZKHQDIIHFWLVLQFOXGHGLQWKHPL[DVLOOXVWUDWHGLQ speech), the neurobiological system (baseline arousal), the study below. DQGWKHVRFLDOFRQWH[W VSHDNLQJLQIURQWRIFROOHDJXHV %RVFKDQG' 0HOOR LQSUHVV FRQGXFWHGDODEVWXG\ vs. strangers). This level of variability and ambiguity RQWKHDIIHFWLYHH[SHULHQFHRIVWXGHQWVGXULQJWKHLU LVH[SHFWHGEHFDXVHKXPDQVDQGWKHLUHPRWLRQVDUH āUVWSURJUDPPLQJVHVVLRQ1RYLFHVWXGHQWV 1 dynamic and adaptive. Rigid emotions have little were asked to learn the fundamentals of computer evolutionary value. SURJUDPPLQJLQWKH3\WKRQODQJXDJHXVLQJDVHOI Where do learning analytics (LA) and educational data SDFHGFRPSXWHUL]HGOHDUQLQJHQYLURQPHQWLQYROYLQJD PLQLQJ ('0 āWLQ"2QRQHKDQGJLYHQWKHFHQWUDO 25-minute scaffolded learning phase and a 10-minute UROHRIHPRWLRQVLQOHDUQLQJDWWHPSWVWRDQDO\]H RU non-scaffolded fadeout phase. All instructional activities data mine) learning without considering emotion FRGLQJUHDGLQJWH[WWHVWLQJFRGHUHFHLYLQJHUURUV will be incomplete. On the other hand, giving the HWF ZHUHORJJHGDQGYLGHRVRIVWXGHQWV IDFHVDQG DPELJXLW\DQGFRPSOH[LW\RIHPRWLRQDOSKHQRPHQD computer screens were recorded. Students provided attempts to study emotions during learning without DIIHFWMXGJPHQWVDWDSSUR[LPDWHO\SRLQWV HYHU\ WKHPHWKRGVRI/$DQG('0ZLOORQO\\LHOGVKDOORZ seconds) over the course of viewing these videos im- insights. Fortunately, there is a body of work that mediately after the learning session via a retrospective
PG 116 HANDBOOK OF LEARNING ANALYTICS DIIHFWMXGJPHQWSURWRFRO 3RUD\VND3RPVWD0DYULNLV The more interesting transitions include affective ' 0HOOR&RQDWL %DNHU 7KHDIIHFWLYHVWDWHV states. In particular, confusion and frustration were RILQWHUHVWZHUHDQJHUDQ[LHW\ERUHGRPFRQIXVLRQ both preceded by an incorrect solution submission FXULRVLW\GLVJXVWIHDUIUXVWUDWLRQĂRZHQJDJHPHQW (SubmitError); these affective states were then fol- happiness, sadness, and surprise. Only engagement, lowed by a hint request (ShowHint), or by constructing confusion, frustration, boredom, and curiosity occurred code (Coding), which itself triggered confusion and ZLWKVXIāFLHQWIUHTXHQF\WRZDUUDQWIXUWKHUDQDO\VLV IUXVWUDWLRQ5HDGLQJLQVWUXFWLRQDOWH[WV LQFOXGLQJ 7KHDXWKRUVH[DPLQHGKRZLQWHUDFWLRQHYHQWVJLYHULVH problem descriptions) was a precursor of engagement, to affective states, and how affective states trigger curiosity, boredom, and confusion but not frustration. In other words, all the key affective states were related various behaviours. They constructed time series that to knowledge assimilation (reading) and construc- interspersed interaction events (from clickstream data) and affective states (self-reports) for each student tion (coding) activities. However, only confusion and during the scaffolded learning phase. Time series frustration accompanied failure (Submit Error) and subsequent help-seeking behaviours (ShowHint), PRGHOOLQJWHFKQLTXHV ' 0HOOR7D\ORU *UDHVVHU ZHUHXVHGWRLGHQWLI\VLJQLāFDQWWUDQVLWLRQV which are presumably learning opportunities. Taken between affective states and interaction events. The WRJHWKHUWKHWUDQVLWLRQPRGHOHPSKDVL]HVWKHNH\ resultant model is shown as a directed graph in Figure role of impasses and the resultant negative activating VWDWHVRIFRQIXVLRQDQGIUXVWUDWLRQWROHDUQLQJ ' 0HOOR 10.1. There were some transitions between interaction events that did not include an affective state (dashed *UDHVVHUE 9DQ/HKQ6LOHU0XUUD\ Figure 10.1. 6LJQLāFDQWWUDQVLWLRQVEHWZHHQDIIHFWLYHVWDWHVDQGLQWHUDFWLRQHYHQWVGXULQJVFDIIROGHGOHDUQLQJ RIFRPSXWHUSURJUDPPLQJ6ROLGOLQHVLQGLFDWHWUDQVLWLRQVLQFOXGLQJDIIHFW'DVKHGOLQHVLQGLFDWHWUDQVLWLRQV QRWLQYROYLQJDIIHFWLYHVWDWHV6KRZ3UREOHP VWDUWLQJDQHZH[HUFLVH 5HDGLQJ YLHZLQJWKHLQVWUXFWLRQVDQG or problem statement; Coding: editing or viewing the current code; ShowHint: viewing a hint; TestRunEr- URU FRGHZDVUXQDQGHQFRXQWHUHGDV\QWD[RUUXQWLPHHUURU 7HVW5XQ6XFFHVV FRGHUXQZLWKRXWV\QWD[RU runtime errors (but was not checked for correctness); SubmitError: code submitted and produced an error or incorrect answer; SubmitSuccess: code submitted and was correct. CHAPTER 10 EMOTIONAL LEARNING ANALYTICS PG 117 Affect Detection from Interaction 3HGUR%DNHU%RZHUVDQG+HIIHUQDQ DWWHPSWHG Patterns to predict college enrollment based on the automatic Affective states cannot be directly measured because GHWHFWRUV7KH\DSSOLHGWKHGHWHFWRUVWRH[LVWLQJORJ they are conceptual entities (constructs). However, āOHVIURPVWXGHQWVZKRLQWHUDFWHGZLWK$6 - WKH\HPHUJHIURPHQYLURQPHQW×SHUVRQLQWHUDFWLRQV SISTments from 2004 to 2009. College enrollment FRQWH[W DQGLQĂXHQFHDFWLRQE\PRGXODWLQJFRJQLWLRQ information for these students was obtained from Therefore, it should be possible to "infer" affect by the National Student Clearinghouse. Automatically DQDO\]LQJWKHXQIROGLQJFRQWH[WDQGOHDUQHUDFWLRQV PHDVXUHGDIIHFWLYHVWDWHVZHUHVLJQLāFDQWSUHGLFWRUV This line of work, referred to as "interaction-based", of college enrollment several years later, which is a ORJāOHEDVHGRUVHQVRUIUHHDIIHFWGHWHFWLRQZDV UDWKHULPSUHVVLYHāQGLQJ VWDUWHGPRUHWKDQDGHFDGHDJR $LHWDO ' 0HO - Affect Detection from Bodily Signals OR&UDLJ6XOOLQV *UDHVVHU DQGZDVUHFHQWO\ Affect is an embodied phenomenon in that it activates reviewed by Baker and Ocumpaugh (2015). bodily response systems for action. This should make $VDQH[DPSOHFRQVLGHU3DUGRV%DNHU6DQ3HGUR it possible to infer learner affect (a latent variable) DQG*RZGD ZKRGHYHORSHGDIIHFWGHWHFWRUV from machine-readable bodily signals (observables). for ASSISTments, an intelligent tutoring system (ITS) There is a rich body of work on the use of bodily for middle- and high-school mathematics, used by signals to detect affect as discussed in a number of DSSUR[LPDWHO\VWXGHQWVLQWKH86DVSDUWRI UHYLHZV &DOYR ' 0HOOR ' 0HOOR .RU\ WKHLUUHJXODUPDWKHPDWLFVLQVWUXFWLRQ 5D]]DTHWDO =HQJ3DQWLF5RLVPDQ +XDQJ 7KHUHVHDUFK 2005). The authors adopted a supervised learning has historically focused on interactions in controlled approach to build automated affect detectors. They environments, but researchers have begun to take this collected training data from 229 students while they work into the real world, notably computer-enabled used ASSISTments in school computer labs. Human FODVVURRPV7KHVWXG\UHYLHZHGEHORZUHĂHFWVRQH observers provided online observations (annotations) such effort by our research group and collaborators, of affect as students interacted with ASSISTments but the reader is directed to Arroyo et al. (2009) for XVLQJWKH%DNHU5RGULJR2EVHUYDWLRQ0HWKRG3URWRFRO their pioneering work on affect detection in comput- %5203 2FXPSDXJK%DNHU 5RGULJR $F - er-enabled classrooms. cording to this protocol, trained observers provide live %RVFK' 0HOOR%DNHU2FXPSDXJKDQG6KXWH annotations of affect based on observable behaviour, studied automated detection of affect from facial LQFOXGLQJH[SOLFLWDFWLRQVWRZDUGVWKHLQWHUIDFHLQ - features in a noisy real-world setting of a comput- teractions with peers and teachers, body movements, er-enabled classroom. In this study, 137 middle and JHVWXUHVDQGIDFLDOH[SUHVVLRQV7KHREVHUYHUVFRGHG high school students played a conceptual physics four affective states (boredom, frustration, engaged HGXFDWLRQDOJDPHFDOOHG3K\VLFV3OD\JURXQG 6KXWH concentration, and confusion) and two behaviours 9HQWXUD .LP LQVPDOOJURXSVIRUWRKRXUV (going off-task and gaming the system). Supervised across two days as part of their regular physics/ learning techniques were used to discriminate each physical science classes. Trained observers performed affective state from other states (e.g., bored vs. others) live annotations of boredom, confusion, frustration, XVLQJIHDWXUHVH[WUDFWHGIURP$66,67PHQWVORJāOHV HQJDJHGFRQFHQWUDWLRQDQGGHOLJKWXVLQJWKH%5203 (performance on problems, hint requests, response āHOGREVHUYDWLRQSURWRFRODVLQWKH$66,67PHQWVVWXG\ times, etc.). Affect detection accuracies ranged from GLVFXVVHGDERYH 3DUGRVHWDO 7KHREVHUYHUV WR PHDVXUHGZLWKWKH$SULPHPHWULF>VLPLODU also noted when students went off-task. to area under the receiver operating characteristic 9LGHRVRIVWXGHQWV IDFHVDQGXSSHUERGLHVZHUHUH - FXUYH×$8&RU$852&@ IRUDIIHFWDQGWR IRU FRUGHGGXULQJJDPHSOD\DQGV\QFKURQL]HGZLWKWKH EHKDYLRXUV7KHFODVVLāHUZDVYDOLGDWHGLQDPDQQHU affect annotations. The videos were processed using WKDWHQVXUHGJHQHUDOL]DELOLW\WRQHZVWXGHQWVIURP the FACET computer-vision program (Emotient, 2014), the same population by enforcing strict independence which provides estimates of the likelihood of 19 facial among training and testing data. action units (Ekman & Friesen, 1978) (e.g., raised brow, 3DUGRVHWDO DOVRSURYLGHGSUHOLPLQDU\HYLGHQFH tightened lips), head pose (orientation), and position on the predictive validity of their detectors. This was (see Figure 10.2 for screenshot). Body movement was GRQHE\DSSO\LQJWKHGHWHFWRUVRQORJāOHVIURPD DOVRHVWLPDWHGIURPWKHYLGHRVXVLQJPRWLRQāOWHULQJ different set of 1,393 students who interacted with DOJRULWKPV .RU\' 0HOOR 2OQH\ VHH)LJXUH $66,67PHQWVGXULQJWKH×VFKRRO\HDUV 10.3). Supervised learning methods were used to build several years before the measure was developed. Au- detectors of each affective state (e.g., bored vs. other tomatically measured affect and behaviour moderately VWDWHV XVLQJERWKIDFLDOH[SUHVVLRQVDQGERGLO\PRYH - FRUUHODWHGZLWKVWDQGDUGL]HGWHVWVFRUHV)XUWKHU6DQ PG 118 HANDBOOK OF LEARNING ANALYTICS Figure 10.2. $XWRPDWLFWUDFNLQJRIIDFLDOIHDWXUHVXVLQJWKH&RPSXWHU([SUHVVLRQ5HFRJQLWLRQ7RROER[7KH graphs on the right show likelihoods of activation of various facial features (e.g., brow lowered, eyelids tight- ening). Figure 10.3. Automatic tracking of body movement from video using motion silhouettes. The image on the right displays the areas of movement from the video playing on the left. The graph on the bottom shows the amount of movement over time. ments. The detectors were moderately successful with interaction-based detectors were less accurate than DFFXUDFLHV TXDQWLāHGZLWKWKH$8&PHWULFDVQRWHG the face-based detectors (Kai et al., 2015), but were DERYH UDQJLQJIURPWRIRUDIIHFWDQGIRU applicable to almost all of the cases. By combining the RIIWDVNEHKDYLRXUV)ROORZXSDQDO\VHVFRQāUPHGWKDW two, the applicability of detectors increased to 98% of WKHDIIHFWGHWHFWRUVJHQHUDOL]HGDFURVVVWXGHQWVPXO - the cases, with only a small reduction (<5% difference) tiple days, class periods, and across different genders in accuracy compared to face-based detection. and ethnicities (as perceived by humans). Integrating Affect Models in Affect-Aware One limitation of the face-based affect detectors is Learning Technologies that they are only applicable when the face can be The interaction- and bodily-based affect detectors automatically detected in the video stream. This is discussed above are tangible artifacts that can be QRWDOZD\VWKHFDVHGXHWRH[FHVVLYHPRYHPHQWRF - instrumented to provide real-time assessments of clusion, poor lighting, and other factors. In fact, the student affect during interactions with a learning face-based affect detectors were only applicable to WHFKQRORJ\7KLVDIIRUGVWKHH[FLWLQJSRVVLELOLW\RI RIWKHFDVHV7RDGGUHVVWKLV%RVFK&KHQ%DN - closing the loop by dynamically responding to the HU6KXWHDQG' 0HOOR XVHGPXOWLPRGDOIXVLRQ sensed affect. The aim of such affect-aware learning techniques to combine interaction-based (similar WHFKQRORJLHVLVWRH[SDQGWKHEDQGZLGWKRIDGDSWLY - to previous section) and face-based detection. The ity of current learning technologies by responding CHAPTER 10 EMOTIONAL LEARNING ANALYTICS PG 119 Figure 10.4. Affective AutoTutor: an intelligent tutoring system (ITS) with conversational dialogs that auto- matically detects and responds to learners’ boredom, confusion, and frustration. to what students feel in addition to what they think helped learning for low-domain knowledge learners DQGGR VHH' 0HOOR%ODQFKDUG%DNHU2FXPSDXJK during the second 30-minute learning session. The Brawner, 2014, for a review). Here, I highlight two such affective tutor was less effective at promoting learning V\VWHPVWKH$IIHFWLYH$XWR7XWRU ' 0HOOR *UDHVVHU IRUKLJKGRPDLQNQRZOHGJHOHDUQHUVGXULQJWKHāUVW D DQG81&,7632.( )RUEHV5LOH\ /LWPDQ 30-minute session. Importantly, learning gains increased from Session 1 to Session 2 with the affective tutor $IIHFWLYH$XWR7XWRU VHH)LJXUH LVDPRGLāHG whereas they plateaued with the non-affective tutor. version of AutoTutor - a conversational ITS that Learners who interacted with the affective tutor also KHOSVVWXGHQWVGHYHORSPDVWHU\RQGLIāFXOWWRSLFVLQ demonstrated improved performance on a subsequent 1HZWRQLDQSK\VLFVFRPSXWHUOLWHUDF\DQGVFLHQWLāF transfer test. A follow-up analysis indicated that learn- UHDVRQLQJE\KROGLQJDPL[HGLQLWLDWLYHGLDORJLQQDW - HUV SHUFHSWLRQVRIKRZFORVHO\WKHFRPSXWHUWXWRUV XUDOODQJXDJH *UDHVVHU&KLSPDQ+D\QHV 2OQH\ 7KHRULJLQDO$XWR7XWRUV\VWHPKDVDVHWRIIX]]\ resembled human tutors increased across learning production rules that are sensitive to the cognitive sessions, was related to the quality of tutor feedback, states of the learner. The Affective AutoTutor augments DQGZDVDSRZHUIXOSUHGLFWRURIOHDUQLQJ ' 0HOOR these rules to be sensitive to dynamic assessments of *UDHVVHUF 7KHSRVLWLYHFKDQJHLQSHUFHSWLRQV OHDUQHUV DIIHFWLYHVWDWHVVSHFLāFDOO\ERUHGRPFRQIX - was greater for the affective tutor. sion, and frustration. The affective states are sensed $VDVHFRQGH[DPSOHFRQVLGHU81&,7632.( )RUEHV5L - by automatically monitoring interaction patterns, ley & Litman, 2011), a speech-enabled ITS for physics JURVVERG\PRYHPHQWVDQGIDFLDOIHDWXUHV ' 0HOOR with the capability to automatically detect and respond *UDHVVHUD 7KH$IIHFWLYH$XWR7XWRUUHVSRQGV WROHDUQHUV FHUWDLQW\XQFHUWDLQW\LQDGGLWLRQWRWKH with empathetic, encouraging, and motivational dia- correctness/incorrectness of their spoken responses. ORJPRYHVDORQJZLWKHPRWLRQDOGLVSOD\V)RUH[DPSOH 8QFHUWDLQW\GHWHFWLRQZDVSHUIRUPHGE\H[WUDFWLQJDQG the tutor might respond to mild boredom with, "This DQDO\]LQJWKHDFRXVWLFSURVRGLFIHDWXUHVLQOHDUQHUV VWXIIFDQEHNLQGRIGXOOVRPHWLPHVVR, PJRQQDWU\ VSRNHQUHVSRQVHVDORQJZLWKOH[LFDODQGGLDORJEDVHG DQGKHOS\RXJHWWKURXJKLW/HW VJR7KHDIIHFWLYH IHDWXUHV81&,7632.(UHVSRQGHGWRXQFHUWDLQW\ responses are accompanied by appropriate emotional when the learner was correct but uncertain about the IDFLDOH[SUHVVLRQVDQGHPRWLRQDOO\PRGXODWHGVSHHFK response. This was taken to signal an impasse because HJV\QWKHVL]HGHPSDWK\RUHQFRXUDJHPHQW the learner is unsure about the state of their knowledge, despite being correct. The actual response strategy The effectiveness of Affective AutoTutor over the original non-affective AutoTutor was tested in a LQYROYHGODXQFKLQJH[SODQDWLRQEDVHGVXEGLDORJV EHWZHHQVXEMHFWVH[SHULPHQWZKHUHOHDUQHUV that provided added instruction to remediate the were randomly assigned to two 30-minute learning uncertainty. This could involve additional follow-up VHVVLRQVZLWKHLWKHUWXWRU ' 0HOOR/HKPDQ6XOOLQVHW TXHVWLRQV IRUPRUHGLIāFXOWFRQWHQW RUVLPSO\WKH al., 2010). The results indicated that the affective tutor assertion of the correct information with elaborated PG 120 HANDBOOK OF LEARNING ANALYTICS H[SODQDWLRQV IRUHDVLHUFRQWHQW measured in this study, behaviourally engaging with Forbes-Riley and Litman (2011) compared learning e-portfolios can be considered a sign of interest, which is a powerful motivating emotion. outcomes of 72 learners who were randomly assigned to receive adaptive responses to uncertainty (adaptive Sentiment Analysis of Discussion Forums condition), no responses to uncertainty (non-adaptive Language communicates feelings. Hence, sentiment control condition), or random responses to uncertainty DQDO\VLVDQGRSLQLRQPLQLQJWHFKQLTXHV 3DQJ /HH (random control condition). In this later condition, 2008) have considerable potential to study how stu- the added tutorial content from the sub-dialogs was GHQWV WKRXJKWV H[SUHVVHGLQZULWWHQODQJXDJH DERXW given for a random set of turns in order to control for DOHDUQLQJH[SHULHQFHSUHGLFWVUHOHYDQWEHKDYLRXUV the additional tutoring. The results indicated that the (most importantly attrition). In line with this, Wen, DGDSWLYHFRQGLWLRQDFKLHYHGVOLJKWO\ EXWQRWVLJQLā - Yang, and Rosé (2014) applied sentiment analysis cantly) higher learning outcomes than the random techniques on student posts on three Massive Open DQGQRQDGDSWLYHFRQWUROFRQGLWLRQV7KHāQGLQJV Online Courses (MOOCs). They observed a negative revealed that it was perhaps not the presence or ab- correlation between the ratio of positive to negative sence of adaptive responses to uncertainty, but the terms and dropout across time. More recently, Yang, number of adaptive responses that correlated with Wen, Howley, Kraut, and Rosé (2015) developed meth- learning outcomes. ods to automatically identify discussion posts that were indicative of student confusion. They showed EMERGING THEMES that confusion reduced the likelihood of retention, but this could be mitigated with confusion resolution Research at the intersection of emotions, learning, and other supportive interventions. /$DQG('0KDVW\SLFDOO\IRFXVHGRQRQHRQRQH Classroom Learning Analytics learning with intelligent tutoring systems (Forbes-Riley Recent advances in sensing and signal processing & Litman, 2011; Woolf et al., 2009), educational games technologies have made it possible to automatically (Conati & Maclaren, 2009; Sabourin, Mott, & Lester, PRGHODVSHFWVRIVWXGHQWV FODVVURRPH[SHULHQFHWKDW 2011), or interfaces that support basic competencies could previously only be obtained from self-reports OLNHUHDGLQJZULWLQJWH[WGLDJUDPLQWHJUDWLRQDQG DQGFXPEHUVRPHKXPDQREVHUYDWLRQV)RUH[DPSOH SUREOHPVROYLQJ ' 0HOOR *UDHVVHUD ' 0HOOR second-generation Kinects can detect whether the /HKPDQ 3HUVRQ ' 0HOOR 0LOOV $O - eyes or mouth are open, if a person is looking away, though these basic lines of research are quite active, DQGLIWKHPRXWKKDVPRYHGIRUXSWRVL[SHRSOHDWD UHFHQWZRUNKDVIRFXVHGRQDQDO\]LQJDIIHFWDFURVV time (Microsoft, 2015). In one pioneering study, Raca, PRUHH[SDQVLYHLQWHUDFWLRQFRQWH[WVWKDWPRUHFORVHO\ .LG]LQVNLDQG'LOOHQERXUJ WUDFNHGVWXGHQWVLQD FDSWXUHWKHEURDGHUVRFLRFXOWXUDOFRQWH[WVXUURXQGLQJ FODVVURRPXVLQJPXOWLSOHFDPHUDVDIā[HGDURXQGWKH OHDUQLQJ,EULHĂ\GHVFULEHIRXUWKHPHVRIUHVHDUFKWR blackboard area. Computer vision techniques were used LOOXVWUDWHDIHZRIWKHH[FLWLQJGHYHORSPHQWV for head detection and head-pose estimation, which Affect-Based Predictors of Attrition and were then used to train a detector of student atten- Dropout tion (validated via self-reports). This emerging area, Early indicators of risk and early intervention systems UHODWHGWRWKHāHOGRIPXOWLPRGDOOHDUQLQJDQDO\WLFV DUHVRPHRIWKHNLOOHUDSSVRI/$DQG('0 -D\ - (Blikstein, 2013), is poised for considerable progress DSUDNDVK0RRG\/DXU®D5HJDQ %DURQ 0RVW in years to come. āHOGHGV\VWHPVIRFXVRQDFDGHPLFSHUIRUPDQFHGDWD Teacher Analytics GHPRJUDSKLFVDQGDYDLODELOLW\RIāQDQFLDODVVLVWDQFH Teachers should not be left out of the loop since teach- These factors are undoubtedly important, but there HUSUDFWLFHVDUHNQRZQWRLQĂXHQFHVWXGHQWDIIHFW are likely alternate factors that come into play. With and engagement. Unfortunately, quantifying teacher WKLVLQPLQG$JXLDU$PEURVH&KDZOD*RRGULFKDQG instructional practices relies on live observations in Brockman (2014) compared the predictive power of classrooms (e.g., Nystrand, 1997), which makes the traditional academic and demographic features with UHVHDUFKGLIāFXOWWRVFDOH7RDGGUHVVWKLVUHVHDUFKHUV features indicative of behavioural engagement in pre- have begun to develop methods for automatic analysis dicting dropout from an Introduction to Engineering of teacher instructional practices. In a pioneering Course. Their key finding was that behaviourally study, Wang, Miller, and Cortina (2013) recorded engaging with e-portfolios, measured by number of classroom audio in 1st to 3rd grade math classes and logins, number of artifacts submitted, and number developed automatic methods to predict the level of of page hits, was a better predictor of dropout than discussions in these classes. This work was recently models constructed from academic performance and H[SDQGHGWRDQDO\]HVHYHUDODGGLWLRQDOLQVWUXFWLRQDO demographics alone. Although affect was not directly CHAPTER 10 EMOTIONAL LEARNING ANALYTICS PG 121 activities (lecturing, small group work, supervised ronment is my fellow-man. The consciousness of his seatwork, question/answer, and procedures and di- attitude towards me is the perception that normally rections) in larger samples of middle-school literature unlocks most of my shames and indignations and fears" and language-arts classes using teacher audio alone (p. 195). Research to date has mainly focused on the 'RQQHOO\HWDOD RUDFRPELQDWLRQRIWHDFKHUDQG achievement, epistemic, and topic emotions. However, FODVVURRPDXGLR 'RQQHOO\HWDOE %ODQFKDUGHW DQDQDO\VLVRIOHDUQLQJLQWKHVRFLRFXOWXUDOFRQWH[W DO XVHGWHDFKHUDXGLRWRDXWRPDWLFDOO\GHWHFW in which it is situated must adequately address the teacher questions, achieving a .85 correlation with social emotions of pride, shame, guilt, jealousy, envy, WKHSURSRUWLRQRIKXPDQFRGHGTXHVWLRQV7KHQH[W and so on. This is both a future theme and a grand step in this line of work is to use information on what research challenge. WHDFKHUVDUHGRLQJWRFRQWH[WXDOL]HKRZVWXGHQWVDUH IHHOLQJZKLFKLQWXUQLQĂXHQFHVZKDWWKH\WKLQNGR CONCLUSION and learn. Learning is not a cold intellectual activity; it is punc- FUTURE THEMES tuated with emotion. The emotions are not merely GHFRUDWLYHWKH\KDYHDJHQF\%XWHPRWLRQLVDFRPSOH[ /HWPHHQGE\EULHĂ\KLJKOLJKWLQJVRPHSRWHQWLDO phenomenon with multiple components that dynamically future themes of research. One promising area of unfold across multiple time scales. And despite great research involves a detailed analysis of the emotional VWULGHVLQWKHāHOGVRIDIIHFWLYHVFLHQFHVDQGDIIHFWLYH H[SHULHQFHRIOHDUQHUVDQGFRPPXQLWLHVRIOHDUQHUV neuroscience, we know little about emotions, and even DFURVVWKHH[WHQGHGWLPHVFDOHRIDWUDGLWLRQDOFRXUVH less on emotions during learning. This is certainly not DIOLSSHGFRXUVHRUD022& 'LOORQHWDO to imply that we should refrain from modelling emo- A second involves the study of emotion regulation WLRQXQWLOWKHUHLVPRUHWKHRUHWLFDOFODULW\4XLWHWKH GXULQJOHDUQLQJHVSHFLDOO\KRZ/$('0PHWKRGV opposite. It simply means that we need to be mindful of can be used to identify different regulatory strategies what we are modelling when we say we are modelling *URVV VRWKDWWKHPRUHEHQHāFLDORQHVFDQ emotion. We also need to embrace, rather than dilute, EHHQJHQGHUHG HJ6WUDLQ ' 0HOOR $WKLUG WKHFRPSOH[LW\DQGDPELJXLW\LQKHUHQWLQHPRWLRQ,I would jointly consider emotion alongside attentional anything, the discovery-oriented, data-driven, analytic states of mindfulness, mind wandering, and how PHWKRGVRI/$DQG('0DORQJZLWKDQHPSKDVLVRQ HPRWLRQDWWHQWLRQEOHQGVOLNHWKHĂRZH[SHULHQFH real-world data collection, has the unique potential to &VLNV]HQWPLKDO\L HPHUJHDQGPDQLIHVWLQWKH advance both the science of learning and the science body and in behaviour. A fourth addresses how the of emotion. It all begins by incorporating emotion into so-called "non-cognitive" (Farrington et al., 2012) traits the analysis of learning. like grit, self-control, and diligence modulate learner HPRWLRQVDQGHIIRUWVWRUHJXODWHWKHP HJ*DOODHW ACKNOWLEDGEMENTS DO $āIWKZRXOGPRQLWRUHPRWLRQVRIJURXSVRI learners during collaborative learning and collabora- This research was supported by the National Science tive problem solving (Ringeval, Sonderegger, Sauer, & )RXQGDWLRQ 16) '5/DQG,,6 WKH Lalanne, 2013) given the importance of collaboration %LOO 0HOLQGD*DWHV)RXQGDWLRQDQGWKH,QVWLWXWHIRU DVDFULWLFDOVWFHQWXU\VNLOO 2(&' Educational Sciences (R305A130030). Any opinions, āQGLQJVDQGFRQFOXVLRQVRUUHFRPPHQGDWLRQVH[ - )LQDOO\TXRWLQJ:LOOLDP-DPHV VFODVVLFWUHDWLVH pressed in this paper are those of the author and do not on emotion: "The most important part of my envi- QHFHVVDULO\UHĂHFWWKHYLHZVRIWKHIXQGLQJDJHQFLHV PG 122 HANDBOOK OF LEARNING ANALYTICS REFERENCES $JXLDU($PEURVH*$$&KDZOD19*RRGULFK9 %URFNPDQ- (QJDJHPHQWYVSHUIRUPDQFH 8VLQJHOHFWURQLFSRUWIROLRVWRSUHGLFWāUVWVHPHVWHUHQJLQHHULQJVWXGHQWSHUVLVWHQFH Journal of Learning Analytics, 1 × $L+/LWPDQ'-)RUEHV5LOH\.5RWDUX07HWUHDXOW-5 3XU$ 8VLQJV\VWHPDQGXVHUSHU - formance features to improve emotion detection in spoken tutoring dialogs. Proceedings of the 9 th Interna- tional Conference on Spoken Language Processing ,QWHUVSHHFK $UUR\R,:RROI%&RRSHU'%XUOHVRQ:0XOGQHU. &KULVWRSKHUVRQ5 (PRWLRQVHQVRUVJRWR VFKRRO,Q9'LPLWURYD50L]RJXFKL%'X%RXOD\ $*UDHVVHU (GV Proceedings of the 14 th International &RQIHUHQFHRQ$UWLāFLDO,QWHOOLJHQFHLQ(GXFDWLRQ SS× $PVWHUGDP ,263UHVV %DNHU5 2FXPSDXJK- ,QWHUDFWLRQEDVHGDIIHFWGHWHFWLRQLQHGXFDWLRQDOVRIWZDUH,Q5&DOYR6 ' 0HOOR-*UDWFK $.DSSDV (GV The Oxford handbook of affective computing SS× 1HZ %DUWK&0 )XQNH- 1HJDWLYHDIIHFWLYHHQYLURQPHQWVLPSURYHFRPSOH[VROYLQJSHUIRUPDQFH Cogni- tion and Emotion, 24 ×GRL %ODQFKDUG1'RQQHOO\32OQH\$06DPHL%:DUG%6XQ;' 0HOOR6. ,GHQWLI\LQJWHDFK - er questions using automatic speech recognition in live classrooms. Proceedings of the 17 th Annual SIGdial Meeting on Discourse and Dialogue 6,*',$/ SS × $VVRFLDWLRQIRU&RPSXWDWLRQDO/LQJXLVWLFV %OLNVWHLQ3 0XOWLPRGDOOHDUQLQJDQDO\WLFV Proceedings of the 3 rd International Conference on Learning Analytics and Knowledge /$. ×$SULO/HXYHQ%HOJLXP1HZ %RVFK1&KHQ+%DNHU56KXWH9 ' 0HOOR6. $FFXUDF\YVDYDLODELOLW\KHXULVWLFLQPXOWLPRGDO affect detection in the wild. Proceedings of the 17 th ACM International Conference on Multimodal Interaction (ICMI 2015). New York: ACM. %RVFK1 ' 0HOOR6. LQSUHVV 7KHDIIHFWLYHH[SHULHQFHRIQRYLFHFRPSXWHUSURJUDPPHUV International -RXUQDORI$UWLāFLDO,QWHOOLJHQFHLQ(GXFDWLRQ . %RVFK1' 0HOOR6%DNHU52FXPSDXJK- 6KXWH9 8VLQJYLGHRWRDXWRPDWLFDOO\GHWHFWOHDUQ - er affect in computer-enabled classrooms. ACM Transactions on Interactive Intelligent Systems, 6 (2), GRL &DOYR5$ ' 0HOOR6. $IIHFWGHWHFWLRQ $QLQWHUGLVFLSOLQDU\UHYLHZRIPRGHOVPHWKRGVDQGWKHLU applications. IEEE Transactions on Affective Computing, 1 ×GRL 7$))& &ORUH*/ +XQWVLQJHU-5 +RZHPRWLRQVLQIRUPMXGJPHQWDQGUHJXODWHWKRXJKW Trends in Cogni- tive Sciences, 11 × GRL MWLFV Conati, C., & Maclaren, H. (2009). Empirically building and evaluating a probabilistic model of user affect. User Modeling and User-Adapted Interaction, 19 × &RUEHWW$ $QGHUVRQ- .QRZOHGJHWUDFLQJ 0RGHOLQJWKHDFTXLVLWLRQRISURFHGXUDONQRZOHGJH User Modeling and User-Adapted Interaction, 4 × &VLNV]HQWPLKDO\L0 Flow: The psychology of optimal experience . New York: Harper & Row. ' 0HOOR6. 2QWKHLQĂXHQFHRIDQLWHUDWLYHDIIHFWDQQRWDWLRQDSSURDFKRQLQWHUREVHUYHUDQG self-observer reliability. IEEE Transactions on Affective Computing, 7 × ' 0HOOR6.%ODQFKDUG1%DNHU52FXPSDXJK- %UDZQHU. ,IHHO\RXUSDLQ $VHOHFWLYHUHYLHZ RIDIIHFWVHQVLWLYHLQVWUXFWLRQDOVWUDWHJLHV,Q56RWWLODUH$*UDHVVHU;+X %*ROGEHUJ (GV Design recommendations for adaptive intelligent tutoring systems: Adaptive instructional strategies 9ROSS × 2UODQGR)/ 86$UP\5HVHDUFK/DERUDWRU\ ' 0HOOR6&UDLJ66XOOLQV- *UDHVVHU$ 3UHGLFWLQJDIIHFWLYHVWDWHVH[SUHVVHGWKURXJKDQHPRWH DORXGSURFHGXUHIURP$XWR7XWRU VPL[HGLQLWLDWLYHGLDORJXH ,QWHUQDWLRQDO-RXUQDORI$UWLāFLDO,QWHOOLJHQFH in Education, 16 × CHAPTER 10 EMOTIONAL LEARNING ANALYTICS PG 123 ' 0HOOR6 *UDHVVHU$ D $XWR7XWRUDQG$IIHFWLYH$XWR7XWRU /HDUQLQJE\WDONLQJZLWKFRJQLWLYHO\DQG emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems, 2 (4), × ' 0HOOR6 *UDHVVHU$ E '\QDPLFVRIDIIHFWLYHVWDWHVGXULQJFRPSOH[OHDUQLQJ Learning and Instruc- tion, 22 ×GRL MOHDUQLQVWUXF ' 0HOOR6 *UDHVVHU$ F 0DOOHDELOLW\RIVWXGHQWV SHUFHSWLRQVRIDQDIIHFWVHQVLWLYHWXWRUDQGLWVLQ - ĂXHQFHRQOHDUQLQJ,Q* ' 0HOOR6 *UDHVVHU$ D ,QGXFLQJDQGWUDFNLQJFRQIXVLRQDQGFRJQLWLYHGLVHTXLOLEULXPZLWKEUHDN - down scenarios. Acta Psychologica, 151 × ' 0HOOR6. *UDHVVHU$& E &RQIXVLRQ,Q53HNUXQ //LQQHQEULQN*DUFLD (GV International handbook of emotions in education SS × 1HZ ' 0HOOR6. .RU\- $UHYLHZDQGPHWDDQDO\VLVRIPXOWLPRGDODIIHFWGHWHFWLRQV\VWHPV ACM Com- puting Surveys, 47 × ' 0HOOR6/HKPDQ% 3HUVRQ1 0RQLWRULQJDIIHFWVWDWHVGXULQJHIIRUWIXOSUREOHPVROYLQJDFWLYLWLHV ,QWHUQDWLRQDO-RXUQDORI$UWLāFLDO,QWHOOLJHQFHLQ(GXFDWLRQ × ' 0HOOR6/HKPDQ%6XOOLQV-'DLJOH5&RPEV59RJW.*UDHVVHU$ $WLPHIRUHPRWLQJ :KHQDIIHFWVHQVLWLYLW\LVDQGLVQ WHIIHFWLYHDWSURPRWLQJGHHSOHDUQLQJ,Q-.D\ 9$OHYHQ (GV Pro- ceedings of the 10 th International Conference on Intelligent Tutoring Systems SS× %HUOLQ+HLGHO - berg: Springer. ' 0HOOR6 0LOOV& (PRWLRQVZKLOHZULWLQJDERXWHPRWLRQDODQGQRQHPRWLRQDOWRSLFV Motivation and Emotion, 38 × ' 0HOOR67D\ORU56 *UDHVVHU$ 0RQLWRULQJDIIHFWLYHWUDMHFWRULHVGXULQJFRPSOH[OHDUQLQJ,Q '0F1DPDUD -7UDIWRQ (GV Proceedings of the 29th Annual Cognitive Science Society SS× Austin, TX: Cognitive Science Society. 'DUZLQ& The expression of the emotions in man and animals. /RQGRQ -RKQ0XUUD\ 'LOORQ-%RVFK1&KHWOXU0:DQLJDVHNDUD1$PEURVH*$6HQJXSWD% ' 0HOOR6. 6WXGHQW HPRWLRQFRRFFXUUHQFHDQGGURSRXWLQD022&FRQWH[W,Q7%DUQHV0&KL 0)HQJ (GV Proceed- ings of the 9th International Conference on Educational Data Mining ('0 SS× ,QWHUQDWLRQDO (GXFDWLRQDO'DWD0LQLQJ6RFLHW\ 'RQQHOO\3%ODQFKDUG16DPHL%2OQH\$06XQ;:DUG%' 0HOOR6. D $XWRPDWLFWHDFK - HUPRGHOLQJIURPOLYHFODVVURRPDXGLR,Q/$UR\R6' 0HOOR-9DVVLOHYD -%OXVWHLQ (GV Proceedings of the 2016 ACM International Conference on User Modeling, Adaptation, & Personalization $&080$3 SS× 1HZ 'RQQHOO\3%ODQFKDUG16DPHL%2OQH\$06XQ;:DUG%' 0HOOR6. E 0XOWLVHQVRU modeling of teacher instructional segments in live classrooms. Proceedings of the 18 th ACM International Conference on Multimodal Interaction ,&0, 1HZ (NPDQ3 )ULHVHQ: The facial action coding system: A technique for the measurement of facial move- ment. 3DOR$OWR&$ &RQVXOWLQJ3V\FKRORJLVWV3UHVV (PRWLHQW )$&(7 )DFLDOH[SUHVVLRQUHFRJQLWLRQVRIWZDUH )DUULQJWRQ&$5RGHULFN0$OOHQVZRUWK(1DJDRND-.H\HV76-RKQVRQ': %HHFKXP12 (2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school per- formance: A critical literature review . Chicago, IL: University of Chicago Consortium on Chicago School Research. )RUEHV5LOH\. /LWPDQ'- %HQHāWVDQGFKDOOHQJHVRIUHDOWLPHXQFHUWDLQW\GHWHFWLRQDQGDG - aptation in a spoken dialogue computer tutor. Speech Communication, 53 × ×GRL M VSHFRP PG 124 HANDBOOK OF LEARNING ANALYTICS )UHGULFNVRQ% %UDQLJDQ& 3RVLWLYHHPRWLRQVEURDGHQWKHVFRSHRIDWWHQWLRQDQGWKRXJKWDFWLRQ repertoires. Cognition & Emotion, 19 ×GRL *DOOD%03OXPPHU%':KLWH5(0HNHWRQ'' 0HOOR6. 'XFNZRUWK$/ 7KHDFDGHPLF GLOLJHQFHWDVN $'7 $VVHVVLQJLQGLYLGXDOGLIIHUHQFHVLQHIIRUWRQWHGLRXVEXWLPSRUWDQWVFKRROZRUN Con- temporary Educational Psychology, 39 × *UDHVVHU$&KLSPDQ3+D\QHV% 2OQH\$ $XWR7XWRU $QLQWHOOLJHQWWXWRULQJV\VWHPZLWK PL[HGLQLWLDWLYHGLDORJXH IEEE Transactions on Education, 48 ×GRL 7( *URVV- (PRWLRQUHJXODWLRQ,Q0/HZLV-+DYLODQG-RQHV /%DUUHWW (GV Handbook of emotions (3 rd HGSS × 1HZ +HVV8 1RZ\RXVHHLWQRZ\RXGRQ WWKHFRQIXVLQJFDVHRIFRQIXVLRQDVDQHPRWLRQ &RPPHQWDU\ RQ5R]LQDQG&RKHQ Emotion, 3 × ,VHQ$ 6RPHZD\VLQZKLFKSRVLWLYHDIIHFWLQĂXHQFHVGHFLVLRQPDNLQJDQGSUREOHPVROYLQJ,Q0 /HZLV-+DYLODQG-RQHV /%DUUHWW (GV Handbook of emotions (3 rd HGSS× 1HZ ,]DUG& 7KHPDQ\PHDQLQJVDVSHFWVRIHPRWLRQ 'HāQLWLRQVIXQFWLRQVDFWLYDWLRQDQGUHJXODWLRQ Emotion Review, 2 ×GRL -DPHV: :KDWLVDQHPRWLRQ" Mind, 9 × -D\DSUDNDVK600RRG\(:/DXU®D(-5HJDQ-5 %DURQ-' (DUO\DOHUWRIDFDGHPLFDOO\DW risk students: An open source analytics initiative. Journal of Learning Analytics, 1 × .DL63DTXHWWH/%DNHU5%RVFK1' 0HOOR62FXPSDXJK-9HQWXUD0 &RPSDULVRQRI IDFHEDVHGDQGLQWHUDFWLRQEDVHGDIIHFWGHWHFWRUVLQSK\VLFVSOD\JURXQG,Q&5RPHUR03HFKHQL]NL\ -%RWLFDULR 26DQWRV (GV Proceedings of the 8 th International Conference on Educational Data Mining ('0 SS× ,QWHUQDWLRQDO(GXFDWLRQDO'DWD0LQLQJ6RFLHW\ .RU\-' 0HOOR6. 2OQH\$ 0RWLRQWUDFNHU &DPHUDEDVHGPRQLWRULQJRIERGLO\PRYHPHQWVXV - ing motion silhouettes. PloS ONE, 10 GRL MRXUQDOSRQH Lench, H. C., Bench, S. W., & Flores, S. A. (2013). Searching for evidence, not a war: Reply to Lindquist, Siegel, 4XLJOH\DQG%DUUHWW Psychological Bulletin, 113 × /HZLV0' %ULGJLQJHPRWLRQWKHRU\DQGQHXURELRORJ\WKURXJKG\QDPLFV\VWHPVPRGHOLQJ Behavior- al and Brain Sciences, 28 × /LQGTXLVW.$6LHJHO(+4XLJOH\.6 %DUUHWW/) 7KHKXQGUHG\HDUHPRWLRQZDU $UHHPR - WLRQVQDWXUDONLQGVRUSV\FKRORJLFDOFRQVWUXFWLRQV"&RPPHQWRQ/HQFK)ORUHVDQG%HQFK Psycho- logical Bulletin, 139 × 0F.HRZQ*- 6QHGGRQ, 0RGHOLQJFRQWLQXRXVVHOIUHSRUWPHDVXUHVRISHUFHLYHGHPRWLRQXVLQJ JHQHUDOL]HGDGGLWLYHPL[HGPRGHOV Psychological Methods, 19 × 0HVTXLWD% %RLJHU0 (PRWLRQVLQFRQWH[W $VRFLRG\QDPLFPRGHORIHPRWLRQV Emotion Review, 6 × 0LFURVRIW .LQHFWIRU:LQGRZV6'.06'15HWULHYHG$XJXVWIURPKWWSV PVGQPLFURVRIWFRP HQXVOLEUDU\GQ DVS[ Moors, A. (2014). Flavors of appraisal theories of emotion. Emotion Review, 6 × Nystrand, M. (1997). Opening dialogue: Understanding the dynamics of language and learning in the English classroom. Language and Literacy Series 1HZ 2FXPSDXJK-%DNHU56 5RGULJR007 %DNHU5RGULJR2EVHUYDWLRQ0HWKRG3URWRFRO %5203 1.0. Training Manual version 1.0. New York. 2(&' 2UJDQLVDWLRQIRU(FRQRPLF&RRSHUDWLRQDQG'HYHORSPHQW 3,6$FROODERUDWLYHSUREOHP solving framework. CHAPTER 10 EMOTIONAL LEARNING ANALYTICS PG 125 2KPDQ$ 6RDUHV- 8QFRQVFLRXVDQ[LHW\ 3KRELFUHVSRQVHVWRPDVNHGVWLPXOL Journal of Abnormal Psychology, 103 × 3DQJ% /HH/ 2SLQLRQPLQLQJDQGVHQWLPHQWDQDO\VLV Foundations and Trends in Information Re- trieval, 2 × × 3DUGRV=%DNHU56-G6DQ3HGUR02&= *RZGD60 $IIHFWLYHVWDWHVDQGVWDWHWHVWV ,Q - YHVWLJDWLQJKRZDIIHFWWKURXJKRXWWKHVFKRRO\HDUSUHGLFWVHQGRI\HDUOHDUQLQJRXWFRPHV,Q'6XWKHUV .9HUEHUW('XYDO ;2FKRD (GV Proceedings of the 3 rd International Conference on Learning Analytics and Knowledge /$. ×$SULO/HXYHQ%HOJLXP SS× 1HZ 3DUNLQVRQ%)LVFKHU$+ 0DQVWHDG$6 Emotion in social relations: Cultural, group, and interper- sonal processes 3V\FKRORJ\3UHVV 3HNUXQ5 6WHSKHQV(- $FDGHPLFHPRWLRQV,Q.+DUULV6*UDKDP78UGDQ6*UDKDP-5R\HU 0=HLGQHU (GV APA educational psychology handbook, Vol 2: Individual differences and cultural and contextual factors SS× :DVKLQJWRQ'& $PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQ 3RUD\VND3RPVWD.0DYULNLV0' 0HOOR6.&RQDWL& %DNHU5 .QRZOHGJHHOLFLWDWLRQPHWKRGV for affect modelling in education. ,QWHUQDWLRQDO-RXUQDORI$UWLāFLDO,QWHOOLJHQFHLQ(GXFDWLRQ × 5DFD0.LG]LQVNL/ 'LOOHQERXUJ3 7UDQVODWLQJKHDGPRWLRQLQWRDWWHQWLRQ 7RZDUGVSURFHVVLQJRI VWXGHQW VERG\ODQJXDJH Proceedings of the 8 th International Conference on Educational Data Mining . Inter- QDWLRQDO(GXFDWLRQDO'DWD0LQLQJ6RFLHW\ 5D]]DT/)HQJ01X]]R-RQHV*+HIIHUQDQ17.RHGLQJHU.5-XQNHU%&KRNVH\6 7KH DVVLVWPHQWSURMHFW %OHQGLQJDVVHVVPHQWDQGDVVLVWLQJ,Q&/RL *0F&DOOD (GV Proceedings of the 12th ,QWHUQDWLRQDO&RQIHUHQFHRQ$UWLāFLDO,QWHOOLJHQFHLQ(GXFDWLRQ SS× $PVWHUGDP ,263UHVV 5LQJHYDO)6RQGHUHJJHU$6DXHU- /DODQQH' ,QWURGXFLQJWKH5(&2/$PXOWLPRGDOFRUSXVRI remote collaborative and affective interactions. Proceedings of the 2 nd International Workshop on Emotion Representation, Analysis and Synthesis in Continuous Time and Space (EmoSPACE) in conjunction with the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition . Washington, '& ,((( 5RVHQEHUJ( /HYHOVRIDQDO\VLVDQGWKHRUJDQL]DWLRQRIDIIHFW Review of General Psychology, 2 (3), ×GRL 5R]LQ3 &RKHQ$ +LJKIUHTXHQF\RIIDFLDOH[SUHVVLRQVFRUUHVSRQGLQJWRFRQIXVLRQFRQFHQWUDWLRQ DQGZRUU\LQDQDQDO\VLVRIPDWXUDOO\RFFXUULQJIDFLDOH[SUHVVLRQVRI$PHULFDQV Emotion, 3 × 6DERXULQ-0RWW% /HVWHU- 0RGHOLQJOHDUQHUDIIHFWZLWKWKHRUHWLFDOO\JURXQGHGG\QDPLF%D\HVLDQ QHWZRUNV,Q6' 0HOOR$*UDHVVHU%6FKXOOHU -0DUWLQ (GV Proceedings of the 4 th International Con- ference on Affective Computing and Intelligent Interaction SS× %HUOLQ+HLGHOEHUJ 6SULQJHU9HU - lag. 6DQ3HGUR0%DNHU56%RZHUV$- +HIIHUQDQ17 3UHGLFWLQJFROOHJHHQUROOPHQWIURPVWXGHQW LQWHUDFWLRQZLWKDQLQWHOOLJHQWWXWRULQJV\VWHPLQPLGGOHVFKRRO,Q6' 0HOOR5&DOYR $2OQH\ (GV Proceedings of the 6 th International Conference on Educational Data Mining ('0 SS× ,QWHU - QDWLRQDO(GXFDWLRQDO'DWD0LQLQJ6RFLHW\ 6FKZDU]1 )HHOLQJVDVLQIRUPDWLRQWKHRU\,Q39DQ/DQJH$.UXJODQVNL 7+LJJLQV (GV Hand- book of theories of social psychology SS × 7KRXVDQG2DNV&$ 6DJH 6KXWH9-9HQWXUD0 .LP<- $VVHVVPHQWDQGOHDUQLQJRITXDOLWDWLYHSK\VLFVLQ1HZWRQ VSOD\ - ground. The Journal of Educational Research, 106 × 6LQKD7-HUPDQQ3/L1 'LOOHQERXUJ3 6WUDLQ$ ' 0HOOR6 $IIHFWUHJXODWLRQGXULQJOHDUQLQJ 7KHHQKDQFLQJHIIHFWRIFRJQLWLYHUHDSSUDLVDO Applied Cognitive Psychology, 29 × GRL DFS PG 126 HANDBOOK OF LEARNING ANALYTICS 7UDF\-/ $QHYROXWLRQDU\DSSURDFKWRXQGHUVWDQGLQJGLVWLQFWHPRWLRQV Emotion Review, 6 × 312. 9DQ/HKQ.6LOHU60XUUD\& :DQJ=0LOOHU. &RUWLQD. 8VLQJWKH/(1$LQWHDFKHUWUDLQLQJ 3URPRWLQJVWXGHQWLQYROYHPHQW through automated feedback. Unterrichtswissenschaft, 4 × :HQ0 :RROI%%XUOHVRQ:$UUR\R,'UDJRQ7&RRSHU' 3LFDUG5 $IIHFWDZDUHWXWRUV 5HFRJQL]LQJ and responding to student affect. International Journal of Learning Technology, 4 × =HQJ=3DQWLF05RLVPDQ* +XDQJ7 $VXUYH\RIDIIHFWUHFRJQLWLRQPHWKRGV $XGLRYLVXDODQG VSRQWDQHRXVH[SUHVVLRQV IEEE Transactions on Pattern Analysis and Machine Intelligence, 31 × CHAPTER 10 EMOTIONAL LEARNING ANALYTICS PG 127