Conditional Random Fields for Activity Recognition Douglas L. Vail CMU-CS-08-119 April, 2008 School of Computer Science Computer Science Department Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Manuela Veloso, Co-Chair John Lafferty, Co-Chair Carlos Guestrin Dieter Fox, University of Washington Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Copyright c 2008 Douglas L. Vail This research was sponsored by a Graduate Fellowship from the National Physical Science Consortium; Directorate of Contracting under grant number DABT63-99-1-0013; National Science Foundation under grant number IIS-9900298; Air Force Research Laboratory under cooperative agreement number F30602- 00-2-0549; L3 Communication Integrated Systems, L.D. under grant number 4500244745; SRI International under grant number 03-000211; Department of Justice under cooperative agreement number 2005IJCXK057; DARPA under grant number NBCH1040007. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S. government or any other entity. Keywords: Conditional Random Fields, Activity Recognition, Robotics, Undirected Graphical Models, Feature Selection, M-estimators For my father, who started it all 4 Abstract To act intelligently in the presence of others, robots must use information from their sensors to recognize the behaviors and activities of the other agents in their environment. Robots must map from low-level, difficult to interpret data, such as position information extracted from video, to abstract states, in particular, the activities of the other agents. In this thesis, we explore how to bridge the gap from noisy, continuous observations about the world to high-level, discrete activity labels for robots in the environment. We contribute the use of conditional random fields (CRFs) for activity recognition in multi- robot domains. We explore the appropriateness of CRFs with an empirical comparison to hidden Markov models. We elucidate the properties of CRFs that make them well suited to the activity recognition, namely discriminative training, the ability to robustly incorporate rich features of the observations, and their nature as conditional models, with a variety of synthetic and real robot data. Accurate activity recognition requires complex and rich features of the observations. We choose the most informative features from a large set of candidates using feature selection. We adapt two feature selection algorithms, grafting and `1 regularization, to conditional random fields. We also investigate a third feature selection algorithm, which was originally proposed for CRFs in a natural language processing domain, in an activity recognition context. In particular, we focus on scaling feature selection to very large sets of candidate features that we define succinctly using a rich relational feature specification language. The reduced feature sets that we discover via feature selection enable efficient, real-time inference. However, feature selection and training for conditional random fields is computa- tionally expensive. We adapt an M-estimator, introduced by Jeon and Lin for log-density estimation in ANOVA models, for fast, approximate parameter estimation in CRFs. We provided an in depth, empirical evaluation of the properties of the M-estimator and then we introduce a new, efficient feature selection algorithm for CRFs based around M-estimation to identify the most important features. 5 6 Acknowledgments I gratefully acknowledge the support of my advisors, Manuela Veloso and John Lafferty. It has been a privilege to work with two such extraordinary people and to grow under their guidance. These past seven years have been good for me, both as a person and a researcher, and I owe both Manuela and John a deep debt of thanks for their role as my mentors. I am grateful for the support and insight of my committee members, Carlos Guestrin and Dieter Fox. I thank them for their generosity with their time and suggestions for improving my thesis. I thank the members of the CMDragons07 robot soccer team, James Bruce, Michael Licitra, Stefan Zickler, and Manuela Veloso for sharing their log data from the RoboCup 2007 robot soccer championship. In particular, James Bruce and Stefan Zickler helped me work with their simulator and the data. I am grateful for the support of my friends in the robot soccer lab, James Bruce, Sonia Chernova, Scott Lenser, Colin McMillen, Maayan Roth, and the other members of the Coral research group. 7 8 Table of Contents 1 Introduction 21 1.1 Characteristics of Sensor Data . 21 1.2 Approach . 22 1.3 Contributions . 23 1.4 Guide to the Thesis . 24 2 Conditional Random Fields for Activity Recognition 25 2.1 Introduction . 25 2.1.1 Activity Recognition . 26 2.1.2 The Sequential Supervised Learning Problem . 27 2.1.3 The Robot Tag Domain . 28 2.2 Representation . 29 2.2.1 Conditional Independence Assumptions . 30 2.2.2 Clique Potentials . 32 9 2.2.3 Features . 33 2.3 Inference . 35 2.3.1 Computing the Normalization Constant . 36 2.3.2 Marginal Probabilities of Labels . 38 2.4 Parameter Estimation . 40 2.4.1 Computing the log likelihood and its gradient . 40 2.4.2 Training with multiple sequences . 42 2.4.3 Regularization . 43 2.5 Implementation . 45 2.5.1 Log-domain addition . 46 2.5.2 The Forward-Backward Algorithm . 47 2.5.3 Computing `(Y jX)............................ 48 2.5.4 Optimization . 49 2.6 Properties of Conditional Random Fields . 50 2.6.1 Generative versus Discriminative Models . 51 2.6.2 Non-independent Observations . 52 2.6.3 Transitions that Depend on Observations . 53 2.7 Tag Activity Recognition Experiments . 56 2.7.1 Notation . 57 2.7.2 Experiments . 57 10 2.7.3 Feature Combinations and Results . 58 2.7.4 Discriminative Training and Smoothing in the HMM . 63 2.8 Chapter Summary . 67 3 Feature Selection 69 3.1 Introduction . 69 3.1.1 Types of Features . 70 3.1.2 Types of Feature Selection Algorithms . 71 3.1.3 Evaluating Feature Selection Methods . 73 3.2 Feature Selection in Conditional Random Fields . 75 3.2.1 Defining \Feature" . 75 3.2.2 Grafting . 76 3.2.3 `1 Regularization . 77 3.2.4 Greedy Feature Induction . 80 3.3 Experiments with Synthetic Data . 83 3.3.1 Synthetic Data with Weakly Relevant Features . 83 3.3.2 Weakly Relevant Features with Grafting . 85 3.3.3 Weakly Relevant Features with the Mean Field Heuristic . 85 3.3.4 Model Selection . 92 3.4 Tag Experiments . 97 11 3.4.1 Features in the CRF . 97 3.4.2 Tag Simulation Results . 99 3.4.3 Tag Real Data Results . 100 3.5 Chapter Summary . 101 4 Multiple Robots and Many Features 105 4.1 The RoboCup Small Size League . 105 4.1.1 Roles . 107 4.1.2 The Classification Task . 108 4.2 Activity Recognition with Simple Features . 109 4.3 Scaling to Many Features . 110 4.3.1 Lack of Sparsity in Features in Activity Recognition . 110 4.3.2 Conditional Random Fields: The Price of Training . 111 4.3.3 Scaling Grafting . 112 4.3.4 Scaling the Mean Field Heuristic . 113 4.3.5 Scaling `1 Regularization . 115 4.4 Relational Spatial Features . 116 4.4.1 A Motivating Example . 117 4.4.2 Components of Relational Spatial Features . 119 4.4.3 Relational Markov Networks . 119 12 4.5 Relational Features for RoboCup . 120 4.5.1 Object Sets, Selection Operators, and Evaluation Operators . 120 4.5.2 Features of the Small Size Domain . 122 4.5.3 Experiments . 129 4.6 Multi-Robot Activity Recognition . 130 4.6.1 Reasoning about the Roles of Other Agents . 130 4.6.2 Automatically Constructing Features . 133 4.6.3 Multi-Robot Relational Feature Experiments . 135 4.7 Chapter Summary . 141 5 An M-estimator for Fast Training and Feature Selection 143 5.1 Introduction . 143 5.1.1 Approximate Parameter Estimation . 144 5.1.2 M-estimators . 145 5.1.3 CRF Training as M-Estimation . 146 5.2 A Computationally Efficient M-Estimator . 147 5.2.1 The Base Model . 149 5.2.2 Asymptotic Consistency . 152 5.2.3 Parameter Estimation . 153 5.2.4 Inference . 154 13 5.3 Experiments . 154 5.3.1 Synthetic Data . 155 5.3.2 Robot Tag . 158 5.3.3 Shallow Parsing . 172 5.4 Feature Selection . 174 5.5 Chapter Summary . 177 6 Related Work 179 6.1 Activity Recognition . 179 6.1.1 Traditional Classifiers . 179 6.1.2 Hidden Markov Models . 180 6.1.3 Augmented Markov Models . 180 6.1.4 Dynamic Baysian Networks . ..