Automatically Analyzing Facial-Feature Movements to Identify Human Errors

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/224083535

Article in Intelligent Systems, IEEE · May 2011 DOI: 10.1109/MIS.2009.106 · Source: IEEE Xplore

CITATIONS READS 6 56

3 authors, including:

Maria E. Jabon Sun Joo-Grace Ahn Stanford University University of Georgia

9 PUBLICATIONS 317 CITATIONS 34 PUBLICATIONS 408 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Working as a software engineer at LinkedIn View project

All content following this page was uploaded by Maria E. Jabon on 28 May 2015.

The user has requested enhancement of the downloaded file. Facial Recognition

Automatically Analyzing Facial- Feature Movements to Identify Human Errors

Maria E. Jabon, Sun Joo Ahn, and Jeremy N. Bailenson, Stanford University

very day countless human errors occur around the globe. Although many Eof these errors are harmless, disastrous errors—such as Bhopal, Cher- Using facial feature nobyl, and Three Mile Island—demonstrate that developing ways to improve

points automatically human performance is not only desirable but crucial. Considerable research

extracted from short exists in human-error identification (HEI), a This enhancement takes our method of hu- field devoted to developing systems to pre- man error prediction beyond the capabilities video segments, dict human errors.1 However, these sys- of existing HEI techniques. tems typically predict only instantaneous researchers couple errors, not overall human performance. Current Approach Furthermore, they often rely on predefined To create our performance models, we first computer vision with hierarchies of errors and manual minute- collected videos and performance logs of by-minute analyses of users by trained participants performing a laboratory task. machine learning to analysts, making them costly and time- We synchronized the videos with the per- consuming to implement.1 (See the “Related formance logs and segmented the videos predict performance Work in Facial Recognition” sidebar for into meaningful chunks based on cues in more details on previous and ongoing the logs. An example of a meaningful chunk over an entire task research work.) might be the time interval preceding one Using facial feature points automati- error instance. and at any given cally extracted from short video segments We then extracted key facial feature points of participants’ faces during laboratory ex- from the videos, such as the mouth and instant within the periments, our work applies a bottom-up eye positions. We calculated time and fre- approach to predict human performance. Our quency domain statistics over the facial fea- task. method maximizes data usage and allows us ture points in each segment and ranked to predict both instantaneous errors (individ- these features according to their chi-square ual errors occurring at any time during the value. Finally, using the highest-ranked fea- task) and task-level performance (speed, ac- tures, we trained machine-learning classifi- curacy, and productivity over the entire task). ers to predict participant performance on the

IS-26-02-jabon.indd 54 16/03/11 3:10 PM Related Work in Facial Recognition

ith its ability to create more than 10,000 expres- Our work extends previous work in many ways. First, we sions, the face has greater variability than any predict a unique and multifaceted human behavior (human W other channel of nonverbal expression. Thus, performance). Furthermore, we use a bottom-up approach automated facial-feature tracking lets researchers tap into that lets us link specific facial features directly to human a rich resource of behavioral cues. In the past, researchers performance and evaluate those features over varying time have shown great interest in using micro-momentary facial intervals. We can identify the most valuable pre-error inter- expressions to predict human emotion and mental states.1–4 vals and the most informative interval lengths. We extend In a recent study, Rana El Kaliouby and Peter Robinson de- current HEI techniques in that we make predictions at two veloped a general computational model to recognize six different temporal layers: instantaneous and task level in classes of complex emotions and implemented this model an intuitive and relatively painless method with highly ac- as a real-time system.5 Their approach used dynamic Bayes- curate results. These advancements offer a cost-effective ian networks to recognize emotions. In a related study that solution in terms of labor, time, and finances in human- used computer vision to detect emotional states, Jeremy N. performance prediction. We feel that such merits will yield Bailenson and his colleagues demonstrated that automated significant benefits to both researchers and industry per- models could be developed to detect and categorize the sonnel who yearn to find answers within a face. felt emotion of individuals.2 By training machine-learning algorithms that link certain facial movements to subjective perception of emotions, they were able to create real-time References models that classified three emotions (sad, amused, and . 1 Z. Zeng et al., “A Survey of Affect Recognition Methods: neutral) based on facial features and physiological re- Audio, Visual, and Spontaneous Expressions,” IEEE Trans. sponses. Rosalind Picard and her colleagues in the Affective Pattern Analysis and Machine Intelligence, vol. 31, no. 1, 2009, pp. 39–58. Computing Research Group at Massachusetts Institute of 2. J.N. Bailenson et al., “Real-Time Classification of Evoked Technology also demonstrated across a number of systems Emotions Using Facial Feature Tracking and Physiological that tracking various facial aspects can give insight into the Responses,” Int’l J. Human Machine Studies, vol. 66, no. 5, mental state of the person whose face is being tracked.6,7 2008, pp. 303–317. Picard and colleagues then advanced the field of behav- 3. R. Picard and J. Klein, “Computers that Recognize and Re- ior prediction to include affective-state prediction. Partic- spond to User Emotion: Theoretical and Practical Implica- ularly interested in the development of affective-learning tions,” Interacting with Computers, vol. 14, no. 2, 2002, companions (also referred to as the Affective Intelligent pp. 144–169. Tutoring System), Picard emphasizes that technology that 4. Y.S. Shin, “Recognizing Facial Expressions with PCA and ICA onto Dimension of the Emotion,” Structural, Syntactic, and recognizes the user’s nonverbal and affective cues and Statistical Pattern Recognition, Springer, 2006, pp. 916–922. responds accurately to these cues will be most effective 5. R. El Kaliouby and P. Robinson, “Mind Reading Machines: 6 in human-computer interaction. Some of the current Automated Inference of Cognitive Mental States from Video,” work from the Affective Computing Research Group in- Proc. IEEE Int’l Conf. Systems, Man and Cybernetics, vol. 1, cludes using multimodal sensory inputs for the following IEEE Press, 2004, pp. 682–688. purposes: 6. R. Picard, Affective Computing, MIT Press, 1997. 7. R.W. Picard and K.K. Liu, “Relative Participative Count and 8 • Predicting frustration in a learning environment; Assessment of Interruptive Technologies Applied to Mobile • Detecting and responding to a learner’s affective and Monitoring of Stress,” Int’l J. Human-Computer Studies, cognitive state;9 and vol. 65, no. 4, 2007, pp. 361–375. • Assisting individuals diagnosed with autism spectrum 8. A. Kapoor, W. Burleson, and R. Picard, “Automatic Prediction disorders in social interaction.10 of Frustration,” Int’l J. Human-Computer Studies, vol. 65, no. 8, 2007, pp. 724–736. Following suit, researchers have now developed systems 9. S. D’Mello et al., “AutoTutor Detects and Responds to Learners to model deception. Thomas O. Meservy and his colleagues Affective and Cognitive States,” Proc. Workshop Emotional extracted macro features such as head and hand position and Cognitive Issues at Int’l Conf. Intelligent Tutoring Systems, and angle from video cameras taken during an experiment 2008, pp. 31–43; http://affect.media.mit.edu/pdfs/08. where a mock theft took place in the lab.11 Afterward, in dmello-etal-autotutor.pdf. an interview with a trained researcher, participants were 10. M. Madsen et al., “Technology for Just-In-Time In-Situ Learn- ing of Facial Affect for Persons Diagnosed with an Autism either truthful or deceptive regarding the theft. Using Spectrum Disorder,” Proc. 10th ACM Conf. Computers and machine-learning algorithms, the team was able to create Accessibility (ASSETS), ACM Press, 2008, pp. 19–26. models that obtained up to 71 percent correct classification 11. T.O. Meservy et al., “Deception Detection through Automatic, of truthful or deceptive participants based on just the fea- Unobtrusive Analysis of Nonverbal Behavior,” IEEE Intelligent tures extracted from the video recording of the subject. Systems, vol. 20, no. 5, 2005, pp. 36–43.

entire task (task-level performance) Task Setup task, participants had to pick up a and at any given instant within the Our experimental task consisted of screw (item one in Figure 2a) from one task (instantaneous errors). fitting screws into holes for half an of the virtual boxes using a Sensable Figure 1 illustrates these steps. hour (see Figure 2). To perform the Phantom Omni haptic pen (item four

March/April 2011 www.computer.org/intelligent 55

IS-26-02-jabon.indd 55 16/03/11 3:10 PM Facial Recognition

Video input Computer logs

Synchronize

Segment data

Extract facial points (raw data)

Compute statistics

Time domain Frequency features domain features

Chi-square feature extraction

Chance LogitBoost Decision Support vector classiﬁer classiﬁer table machine

Figure 1. Human error identification approach. To create our performance models, we followed a series of steps to extract training data from a set of videos and performance logs.

in Figure 2b) and insert it into a hole progress. One board refresh was termed Figure 2b) affixed to the top of the with the correct label. The pen, a de- one phase of the experiment. The first monitor captured the participants’ vice with six degrees of freedom (x, board refreshed after 45 seconds. If the faces at a rate of 15 frames per sec- y, z, pitch, yaw, and roll), let the user participant successfully filled two con- ond while video recording software “feel” the hardness and the depth secutive boards without any errors (in- (Video Capturix 2007) compressed of the box. A beep indicated the us- dicating that the level of difficulty was the data to AVI format. We also re- er’s success or failure to screw in the too low), the phase time was curtailed corded performance logs, includ- parts. The wooden boards refreshed by three seconds. ing measures such as time stamps for after a preprogrammed amount of A high-resolution Logitech Quick- each error (that is, placing a screw in time regardless of the participant’s Cam UltraVision webcam (item 5 in an incorrect hole, dropping a screw,

56 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

IS-26-02-jabon.indd 56 16/03/11 3:10 PM 1 5

(a) (b)

Figure 2. Experimental setup. The task was administered at a computer station with a flat-screen monitor adjusted to a 640 × 480 pixel resolution. (a) We presented the participants with three boxes, each containing a screw with a different part number. (b) The center of their screen contained a large wooden board with seven holes labeled with a randomly selected set of the different part numbers. The screen also contained a virtual screw box (1), virtual board (2), and virtual screw (3). The participants needed to put the screws in the holes using a haptic pen (4). We recorded each session using the mounted Web camera (5).

or failing to hold the screw in place phases of the task. We discarded any until the beep), each correctly placed intervals with average face-tracking screw, each board refresh, and overall confidence (that is, how confident time spent holding screws. In this OKAO was in its measurement) way, we measured each participant’s lower than 60 percent. instantaneous performance (errors) We then calculated means, medi- and performance (overall error rate ans, minimums, maximums, standard and speed of completion) over the en- deviations, ranges, and wavelet trans- tire experiment. formations on each of the raw facial- We collected data from 57 stu- feature points in each interval (see dents (25 female and 32 male) but Figure 4). We calculated these statis- discarded data from eight partici- tics because facial signals are dynamic, pants due to technical problems with and their micro-momentary move- collection. ments can leak information about the person’s internal state.2 Feature Computation Figure 3. OKAO computer vision We extracted facial-feature points Feature Selection algorithm tracking points on a from the videos using the OKAO vi- To speed up the training of our algo- participant’s face. We tracked 37 points sion library (see Figure 3). To map rithms, prevent over-fitting, and iden- on the face, along with head movements such as pitch, yaw, and roll and eye and the facial data with task perfor- tify which features were most useful mouth openness level. We tracked the mance, we then synchronized our fa- in predicting performance, we per- points relative to the captured frame. cial data with the performance logs. formed a chi-square feature selection However, for our calculations, we Finally, we programmatically seg- using the freely distributed machine- standardized all the points to be relative mented the data according to our learning software package Waikato to the center of the face. two prediction intervals: instanta- Environment for Knowledge Analy- neous and task. Instantaneous inter- sis (WEKA).3 We also performed a problems, such as Arabic text clas- vals corresponded to data from short best-first search to determine the op- sification4 and gene-based cancer time intervals directly preceding timal cutoff for features to keep in identification.5 error instances, and task level inter- our analyses. Similar methods have Figure 5 shows the performance vals corresponded to data for entire been successful in other classification curve of each analysis using different

March/April 2011 www.computer.org/intelligent 57

IS-26-02-jabon.indd 57 16/03/11 3:10 PM Facial Recognition

0.30

s 0.20

0.10

0.30

0.25

a3 0.20

0.15

0.10 0.10

d3 0

–0.10

0.10

0.05 d2 0

–0.05

0.05

d 1 0

–0.05

100 200 300 400 500 600 700

Figure 4. Wavelet decomposition of right eye openness level. The s is the original signal, a3 is the order one decomposed signal, and d1 through d3 represent the three levels of decomposition: s = a3 + d1 + d2 + d3.

numbers of features. Table 1 lists the and Bayesian nets, we found decision tunes its algorithms to predict unla- top 10 features for each analysis, and tables and LogitBoost classifiers to be beled vectors in a test set. Figure 6 shows the meaning of each the most powerful performance pre- Specifically, the decision table will, feature. On average, 60 percent of dictors. In many ways these algorithms given a training set and another set of the top features were wavelet coef- mirror the cognitive process used by unlabeled instances I, search for the ficients of face signals, which shows humans, whereby the sequence of set £ of instances that match the fea- the power of the wavelet analysis. behaviors (or features) that lead up tures of I. It will then predict I to be to an error are learned by observing of the majority class in £. If £ = ∅, Performance Prediction many examples. In machine learn- then the majority class of the training From our experimentations with de- ing, the examples are a set of vectors set is predicted.6 If the features are cision tables, support vector ma- containing all the features and a label. continuous they are divided into two chines (SVMs), LogitBoost classifiers, From this training set, the classifier discrete classes based on where they

58 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

IS-26-02-jabon.indd 58 16/03/11 3:10 PM 100 90 fall in relation to the median value of 80 the feature. 70 LogitBoost classifiers work by se- quentially applying a classification 60

algorithm to reweighted versions of Overall accuracy 50 a data set to make a series of classifi- 40 ers.7 A majority vote of these classifiers then determines the final class pre- 30 0 5 10 15 20 25 30 diction of unlabeled instances.7 For (a) Number of features our LogitBoost classifier, we chose a simple decision stump classifier as our 100 base algorithm and built a LogitBoost 90 classifier by performing 40 boosting iterations on the decision stump. 80 70 Results 60 To gauge the performance of our classifiers, we calculated three main Overall accuracy 50 measures for each classifier: overall 40 Decision table LogitBoost SVM accuracy, precision, and recall. Over- 30 all accuracy is the total number of 0 5 10 15 20 25 30 correctly classified instances. Preci- (b) Number of features sion is the number of instances correctly predicted to be in a class divided Figure 5. Accuracy of performance predictions. We used different numbers by the total number of instances for of features to determine the overall accuracy for (a) task performance and that class. Recall is the total number (b) instantaneous errors. of instances correctly predicted to be in a class divided by the total number phase completion time, total num- Although the overall accuracy of instances predicted to be in that ber of phases completed at maximum peaked for both classifiers when 20 class. speed, mean error rate, amount of time phases were used as input, the over- We then looked at the overall ac- spent holding screws, and mean num- all accuracy obtained with 10 phases curacy, precision, and recall of our ber of filled holes per box. Participants was only 2 to 10 percent lower than classifiers and compared them against with scores in the top quartile were the overall accuracy obtained with 20 the corresponding values from a chance labeled high performers, and those in phases. This demonstrates the power classifier. We define chancea classifier the bottom quartile were labeled low of our approach; using only small as a classifier that guesses the class performers. We used data from two amounts of facial data, we can gauge of an instance using the proportional independent data sets for testing and participant performance over the split of the data. training. This assured the generaliz- entire task. If early prediction were ability of our results across individuals. essential in an application, just 10 Task-Level Performance Figure 7 shows the face inputs we phases (five to seven minutes) of data Prediction used to predict task-level performance would be sufficient to classify partici- Our first goal was to predict partici- using varying amounts of data. The pants as high or low performers. pants’ overall task performance by results changed with differing num- Table 2 shows the face inputs we using only the first few minutes of bers of phases, but in general accu- used to predict task-level perfor- facial data in their videos—that is, racy increased with more phases. mance using 20 data phases. Boosting create models capable of prescreen- We noted significant drops in perfor- significantly improved results; the ing individuals for aptitude at the task. mance with certain numbers of phases LogitBoost performed almost 15 per- We defined overall performance as a (for example, 12 phases), which could cent better than the simple decision normalized sum of how many phases be caused if certain phases were less table, classifying participant perfor- each participant completed, fastest predictive than others. mance with 95.2 percent accuracy.

March/April 2011 www.computer.org/intelligent 59

IS-26-02-jabon.indd 59 16/03/11 3:10 PM Facial Recognition

Table 1. Top 10 chi-square features for each prediction. Level Statistic Feature Definition Chi value Task Average Average Y Vertical position of face 108.5 Maximum Average Y Vertical position of face 78.90

Wavelet Gaze tilt q (radians) 70.50

Minimum Average Y Vertical position of face 64.53

Wavelet Gaze tilt q (radians) 63.56

Average Lower lip center Y A (y coordinate) 61.80

Wavelet Right eye open level h (cm) 55.31

Wavelet Right eye ratio w/h 55.25

Wavelet Gaze tilt q (radians) 54.26

Wavelet Right eye ratio w/h 52.50

Instantaneous Velocity Roll y (radians) 790.8 Velocity Yaw b (radians) 449.7

Velocity Roll y (radians) 424.6

Wavelet Left outer eye corner Y D (y coordinate) 379.1

– Eye per close rate Percent of time both eyes h < .15 370.9

Wavelet Average X Horizontal position of face 370.6

Wavelet Left lower lip Y A (y coordinate) 367.1

Wavelet Left upper lip Y B (y coordinate) 356.0

Wavelet Left pupil Y E (y coordinate) 354.3

Wavelet Left upper lip X B (x coordinate) 353.2

This accuracy is more than 44 per- approximately equal number of ran- System Use cent higher than the chance level. Re- domly selected non-error intervals. Our results, most of which are in call was also notably strong for the We used two independent sets of par- the 90th percentile, indicate that our high performers in both algorithms, ticipants for test and training sets and method of using facial movements to indicating a low false-alarm rate. trained a LogitBoost classifier and a predict errors and model human per- The support vector machine per- decision table classifier on each set. formance has significant potential for formed worse than the decision table Figure 8a shows the facial fea- actual application. Human error plays and LogitBoost classifier, suggesting tures predicting errors one second a role in many industrial incidents; the support vector algorithm might before they occurred with varying I, the nuclear power industry attributes not be well suited for performance and Figure 8b shows the facial fea- up to 70 to 90 percent of failures to prediction. tures predicting errors at varying D human error, and other industries re- using two seconds of data. Perfor- port similar rates—90 percent for air- Instantaneous Error Prediction mance peaked using two seconds of lines and 98 percent in medicine.8,9 In our second analysis, we predicted data one second before the error oc- In these safety-critical applications, instantaneous errors. To do this, we curred. This suggests that the micro- warnings could be issued to eliminate compiled a data set of all the pre- momentary expressions indicative costly, or even deadly, incidents. error intervals, which we defined as of errors might be short in duration Moreover, given that our models the window of facial data of length I (less than three seconds) and that provide both micro- and macro-views beginning D seconds before the error, the expressions occur at one sec- of human-task errors, our methods where I ranged from one to five second or less before an error. Table 3 could also give managers a better under- onds and D ranged from one to three shows results for the most successful standing of worker performance at seconds. We then added to this set an classification. both the task and individual error levels.

60 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

IS-26-02-jabon.indd 60 16/03/11 3:10 PM α

w E h θ ψ C D

Figure 6. Significant facial features. Table 1 defines each feature.

This is an important contribution be- 100 cause simple aggregates of individual errors do not necessarily demonstrate a 90 person’s overall performance during a task. Companies could use these mod- 80 els to prescreen employees or match individuals to jobs better suited to their 70 skills , saving time and resources. Furthermore, our results indicate 60

that we are able to outperform Overall accuracy 50 human analysts with only a fraction of the effort required in the tradi- 40 tional HEI systems. Thus, our mod- LogitBoost Chance Decision table SVM els can either completely substitute 30 human data analysts or supplement 0 5 10 15 20 their work, saving valuable human Phases used resources. Finally, because of the ease of adop- Figure 7. Task-level performance results. The results indicate that we can gauge tion (our models require only a small participant performance over the entire task using only small amounts of facial data. webcam and processor), individuals Table 2. Prediction results for task performance using 20 data phases. and corporations alike could reap the benefits of performance prediction at Overall Precision Recall little cost, allowing more widespread Classifier accuracy (%) Class (%) (%) use and thus offering greater error Chance classifier 50.0 High 50.0 50.0 avoidance potential. Low 50.0 50.0 LogitBoost 95.2 High 91.3 100 classifier System Limitations Low 100 90.5 Despite these encouraging results, Decision table 79.8 High 71.2 100 the current study has several limita- Low 100 59.5 tions. Although the models can be Support vector 72.6 High 65.1 97.6 generalized across individuals, they machine cannot be generalized across tasks; Low 95.2 47.6 our models were based on arbitrary definitions of error and performance setting with the face clearly visible. In addition, the facial-recognition specifically tailored to our laboratory In work environments where the face software and machine-learning tech- experiment. Furthermore, we con- is not visible our models would be un- nology we used is not specifically ducted our experiment in a laboratory able to predict performance. tailored to the task of performance

March/April 2011 www.computer.org/intelligent 61

IS-26-02-jabon.indd 61 16/03/11 3:10 PM Facial Recognition

90 85 80 custom-made library that automati- 75 cally detects and tracks more points 70 on the face and works around ob- 65 structions on the face such as glasses 60 might yield better results. Similarly,

Overall accuracy alternative algorithms might also 55 yield better accuracies. 50 45 40 ven with the technical chal- 1 2 3 4 5 Elenges and limitations discussed (a) Interval length (sec) here, we believe that our video-based 90 performance prediction system demonstrates potential for many appli- 85 cations. Although it is difficult to 80 expect any system, even human, to 75 make perfectly accurate judgments 70 on processes as complex as human 65 behavior, our results are encourag- 60 ing and we anticipate improvements

Overall accuracy with future research. By incorpo- 55 rating these models into the work- 50 place and learning environment, our 45 LogitBoost Chance Decision table SVM models could serve as effective, cost- 40 efficient, and unobtrusive monitors 1 2 3 that assist both individuals and corpo- (b) Time before (sec) rations in achieving maximum safety and output. Figure 8. Instantaneous error prediction. We measured the facial features predicting errors (a) one second before they occurred with varying amounts of data (b) at Acknowledgments varying times before the error occurred using two seconds of data. We thank Ritsuko Nishide, Shuichiro Tsukiji, Hiroshi Nakajima, and Kimihiko Iwamura Table 3. Face input predicting errors one second before for early guidance on the project, and they occur using two seconds of data. OMRON Silicon Valley for partial funding of the project. We also thank Suejung Shin Overall and Steven Duplinsky for their help in mak- Classifier accuracy (%) Class Precision (%) Recall (%) ing the images, and Joris Janssen, Kathryn Chance 50.1 Error 47.2 47.2 Segovia, Helen Harris, Michelle Del Rosario, classifier and Solomon Messing for helpful comments Correct 52.8 52.8 on earlier drafts of this article. LogitBoost 82.0 Error 84.1 75.7 classifier Correct 80.4 87.5 References Decision table 84.7 Error 83.2 84.1 1. P. Salmon et al., “Predicting Design Correct 86.0 85.2 Induced Pilot Error: A Comparison Support vector 53.3 Error 0 0 of SHERPA, Human Error HAZOP, machine HEIST and HET, a Newly Devel- Correct 53.3 100 oped Aviation Specific HEI Method,” Human-Centered Computing: Cogni- prediction, and the performance of vision tracks only 37 points on tive, Social, and Ergonomic Aspects, our models is closely related to the the face and might not yield accurate Lawrence Erlbaum Associates, vol. 3, quality of our software. OKAO results for people wearing glasses. A 2003, pp. 567–571.

62 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

IS-26-02-jabon.indd 62 16/03/11 3:10 PM T h e A u t h o r s

Maria E. Jabon is a software engineer at LinkedIn. While attending Stanford, she worked 2. P. Ekman and E.L. Rosenberg, What as lead systems engineer in the Virtual Human Interaction Lab and research assistant in the Face Reveals: Basic and Applied the Nolan Lab. Her interests include machine learning and Web interfaces for the visu- alization of high-dimensional data sets. Jabon has an MS in electrical engineering from Studies of Spontaneous Expression Stanford University. Contact her at [email protected]. Using the Facial Action Coding System (FACS), Oxford Univ. Press, 1997. Sun Joo Ahn is a doctoral candidate at the Department of Communication at Stanford University. Her research interests include emotion and behavior prediction based on auto- 3. H.I. Witten and E. Fank, Data Mining: mated facial-feature tracking and consumer psychology within virtual environments. Ahn Practical Machine Learning Tools has an MA in communication from Stanford University. Contact her at [email protected]. and Techniques, 2nd ed., Morgan Jeremy N. Bailenson is the founding director of the Virtual Human Interaction Lab Kaufmann, 2005. and an associate professor in the Department of Communication at Stanford University. 4. P. Kosla, A Feature Selection Approach His main interest is the phenomenon of digital human representation, especially in the in Problems with a Great Number context of immersive virtual reality. Bailenson has a PhD in cognitive psychology from Northwestern University. Contact him at [email protected]. of Features, Springer, 2008, pp. 394–401. 5. X. Jin et al., “Machine Learning Machine Learning, Springer-Verlag, 9. A. Isaac, “Human Error in European Techniques and Chi-Square Feature 1995, pp. 174–189. Air Traffic Management: The HERA Selection for Cancer Classification 7. J.H. Friedman, T. Hastie, and R. project,” Reliability Eng. and Sys- Using SAGE Gene Expression Profile,” Tibshirani, “Additive Logistic Regression: tem Safety, vol. 75, no. 2, 2002, Proc. Data Mining for Biomedical Ap- A Statistical View of Boosting,” Annals of pp. 257–272. plications, LNBI 3916, Springer, 2006, Statistics, vol. 28, no. 2, 2000, pp. 337–407. pp. 106–115. 8. J.W. Senders and N. Moray, Human Er- Selected CS articles and columns 6. R. Kohavi, “The Power of Decision ror: Cause, Prediction, and Reduction, are also available for free at Tables,” Proc. 8th European Conf. Lawrence Erlbaum Associates, 1991. http://ComputingNow.computer.org.

March/April 2011 www.computer.org/intelligent 63

View publication stats

IS-26-02-jabon.indd 63 16/03/11 3:10 PM