Group Activity Recognition Using Wearable Sensing Devices
Total Page：16
File Type：pdf, Size：1020Kb
Group Activity Recognition Using Wearable Sensing Devices In Fulfillment of the Requirements for the Degree of Doctor of Engineering Faculty of Informatics Karlsruhe Institute of Technology (KIT) Approved Dissertation by Dawud Gordon Born in Istanbul, Turkey Examination Date: 17 Apr 2014 Primary Examiner: Prof. Dr.-Ing. Michael Beigl Secondary Examiner: Prof. Dr. Gerhard Tröster For David Blair, (1952 – 2013) who believed in me when no one else did, including myself. –AND– For Dorothea Erhart, (1953 – 2013) who accepted me as family unconditionally. I would like to thank my father for being my hero, whose strength and warrior mentality in the face of adversity never ceases to amaze and humble me. I thank my mother and my sister Sara for understanding and supporting my decisions, even if it meant not seeing me for 10 years. I would also like to thank my other family, the Erharts, who selflessly welcomed me into their home and lives in that time for absolutely no reason I can think of. My friends who made daily life bearable with this workload, especially Dejan, Tirdad and Markus, I salute you. I thank my colleagues at TecO who put up with my quirks and provided a fantastic platform for creativity and constructive debate. I would also like to thank the myriad of Bachelor, Master and Diploma students who helped me along the way with experiments and implementations. My high school science teachers, John Hoffman and Stephen Edelglass who inspired me to always question and to enjoy finding the answers. Many thanks are due to my secondary adviser Prof. Gerhard Tröster for taking on this role, and for giving me the opportunity to work with him and his group in Zürich. Finally I cannot thank my adviser Prof. Michael Beigl enough for giving me the opportunity to be a part of his research team. Over the years he has always been there with ideas and inspiration, giving me the freedom to follow my own creativity, and the endless pep talks, support and advice I needed to see it through. Abstract One goal of ubiquitous and pervasive computing is to enable devices to adapt to the contexts and activities of the individuals interacting with them through a process called context or activity recognition. However, humans are social and spend most of their time in groups, and the behavior of the group is fundamentally different than the behavior of each individual in it, referred to as “emergent behavior” in social psychology. As the number of devices and users increases, using infrastructure and server-side processing for recognition becomes infeasible due to state space explosion and the “curse of dimensionality.” As a result, peer- to-peer (P2P) methods for detecting this behavior within the network of sensing devices are needed. Machine learning and classification are demonstrably effective for recognizing the behavior of individuals based on wearable sensing devices, especially accele- rometers. However, systems which recognize contexts and activities of groups of individuals are required in order to support the needs of the group. The mobile devices we carry present an optimal platform, but aggregating data from the mul- titude of embedded sensors which observe behavior can congest networks. P2P approaches are attractive but challenging, since the behavior of the group cannot be observed at any single location. The challenges are (1) to understand the data which is required for recognition, (2) to detect different groups who may be in the same environment, and (3) to recognize the physical behavior or activities of the group, all in a P2P fashion. Furthermore, all of this must be done while (4) respecting the limited resources and primary functions of the sensing devices, e.g. mobile phones. The contribution of this dissertation consists of the following: A formal definition of group activity recognition and differentiation from • multi-user activity recognition. Methods for using vibration sensors to reduce the power consumption of • physical activity sensing. Methods for reducing the consumption of the recognition toolchain using • prediction for dynamic sensor selection. An analysis of different abstraction levels for sensor observations with respect • to group activity recognition. Algorithms for detection group affiliation in P2P networks of mobile, wearable • devices. Algorithms for inferring emergent group behavior in P2P networks of mobile, • wearable devices. This dissertation presents novel methods for reducing the power consumption of the machine learning tool-chain for recognition of human behavior in distributed systems. First a vibrational sensor is investigated with respect to its potential for activity recognition. The sensor proves useful for recognizing activities with high- frequency vibrational components such as riding a bicycle, as well as activities with impacts or concussions, such as walking or jogging. The approach consumes 50 times less than an accelerometer and samples high motion frequency information (3 8 kHz) which the accelerometer can not. Methods for online sensor selection − using the predictability of human behavior are explored, which can greatly reduce the energy footprint of activity sensing without sacrificing the accuracy of recogni- tion. By predicting likely and unlikely activities for the immediate future, sensors required to differentiate unlikely activities can be turned off to conserve energy. Thereby energy savings of around 85%- 90% where achieved in turn for a loss of 1.5 - 3 percentage points of recognition accuracy To approach the problem of distributed recognition of emergent group activities, the level of abstraction at which to fuse individual behavior information into group information is evaluated. Here power consumption and accuracy of inference is explored for using local features, local behavior classes, as well as unsupervised soft and hard clustering as the basis for inference of group activities. Using single-user activities saves 40% of power consumed by communication, but causes a 47% loss in recognition effectiveness due to technical issues Unsupervised clustering has a high potential, reducing energy consumption by 36%, while only causing 2.8% reduction in recognition rates. The unsupervised clustering abstraction level is therefore used as the abstraction level of choice for this dissertation. Using unsupervised clustering of behavioral observations, statements can be made about the behavior of a group, based on distributed observations of individual constituents. A method using the Jeffrey’s divergence of behavioral clusters and P2P communication called divergence-based affiliation detection (DBAD) is introduced, whereby group affiliations can be detected in multi group environments. When compared to a centralized approach, the DBAD reduces power consumption by up to 43% while maintaining affiliation detection rates and at the same eliminating the need for centralized resources. A method for using distributed probabilistic inference with loopy belief propagation (DPI-LBP) is presented which allows emergent group behavior to be recognized by the distributed network, further reducing energy consumption without the need for infrastructure. Again it is shown that performance is comparable to monolithic approaches, while reducing overall impact on the power consumption of the devices by up to a factor of 40. Local processing and memory usage increase slightly, but are well within tolerable boundaries. vi The combined contribution is a methodology for practical recognition of groups and their emergent behavior in a P2P network of mobile phones. Applications are foreseen in adaptive intelligent environments, social networking, crowd monitoring and possibly even crowd management in emergency situations. vii Deutsche Zusammenfassung Eines der Ziele von Pervasive-Computing ist es, anhand mobiler und tragbarer Geräte, die Kontexte und Aktivitäten individueller Nutzer und Träger, zu verstehen. Dieses Ziel wird durch den Prozesses der so genannten Kontext- oder Aktivitäts- erkennung erreicht. Das Problem hierbei ist, dass das Verhalten eines Menschen in einer Gruppe ein grundlegend anderes ist, als das Verhalten der Gruppe selbst -auch „emergentes“ Verhalten“ genannt. Mit der stetig wachsenden Anzahl von Geräten und Nutzern steigen auch die zur Erkennung benötigten Ressourcen ex- ponential, verursacht durch die Zustandsexplosion und dem sogenannten „Curse of Dimensionality”. Dies führt dazu dass es fast unmöglich sein wird, diese Daten- menge an einem Ort zu verarbeiten. Aus diesem Grund werden Peer-to-Peer (P2P) Ansätze benötigt, da diese das Verhalten innerhalb des Netzwerks von mobilen und tragbaren Sensorgeräten erkennen können. Maschinelles Lernen und Klassifikation, auf Basis von Sensoren in tragbaren Geräten- insbesondere durch den Einsatz des Accelerometer, haben sich als effek- tive Werkzeuge zur Verhaltenserkennung einzelner Personen erwiesen. Um die Wünsche und Bedürfnisse einer Gruppe zu erfüllen, müssen die Kontexte und Aktivitäten der Gruppe erkannt werden, was durch die Erweiterung dieser Ansätze möglich wird. Für diesen Zweck stellen mobile und tragbare Geräte eine optimale Plattform dar. Die Aggregation dieser Daten jedoch, kann zu Überlastung der Infrastruktur führen. P2P-Ansätze sind attraktiv aber herausfordernd. Denn es ist nicht möglich, durch die Beobachtung der einzelnen Menschen unabhängig voneinander, auf das Verhalten der Gesamtheit zu schließen. Die Schwierigkeit ist es (1) mit einem P2P-System zu verstehen, welche Daten benötigt werden um Gruppenaktivitäten im