Smart assistants for smart homes KATHARINA RASCH Doctoral Thesis in Electronic and Computer Systems Stockholm, Sweden 2013 TRITA-ICT/ECS AVH 13:16 KTH School of Information and 1653-6363 Communication Technology KTH/ICT/ECS/AVH-13/16-SE SE 164-40 Kista 978-91-7501-837-9 SWEDEN Akademisk avhandling som med tillstånd av Kungliga Tekniska Högskolan framlägges till offentlig granskning för avläggande av teknologie doktorsexamen i elektronik och datorsystem den 11 Oktober 2013 klockan 13 i Sal E, Forum Isafjordsgatan 39, Kista, Kungliga Tekniska Högskolan. © Katharina Rasch, September 2013 Tryck: Universitetsservice US AB iii Abstract The smarter homes of tomorrow promise to increase comfort, aid elderly and disabled people, and help inhabitants save energy. Unfortunately, smart homes today are far from this vision – people who already live in such a home struggle with complicated user interfaces, inflexible home configurations, and difficult installation procedures. Under these circumstances, smart homes are not ready for mass adoption. This dissertation addresses these issues by proposing two smart assistants for smart homes. The first assistant is a recommender system that suggests useful services (i.e actions that the home can perform for the user). The recommended services are fitted to the user’s current situation, habits, and preferences. With these recommendations it is possible to build much simpler user interfaces that highlight the most interesting choices currently available. Configuration becomes much more flexible: since the recommender system automatically learns user habits, user routines no longer have to be manually described. Evaluations with two smart home datasets show that the correct service is included in the top five recommendations in 90% of all cases. The second assistant addresses the difficult installation procedures. The unique feature of this assistant is that it removes the need for manually describing device functionalities (such descriptions are needed by the recommender system). Instead, users can simply plug in a new device, begin using it, and let the installation assistant identify what the device is doing. The installation assistant has minimal requirements for manufacturers of smart home devices and was successfully integrated with an existing smart home. Evaluations performed with this smart home show that the assistant can quickly and reliably learn device functionalities. v Acknowledgements First and foremost I am grateful to my supervisor Professor Rassul Ayani for all the support and guidance that he has given me over the past five years. Without him, this dissertation would not have been possible. I am thankful to my supervisor Professor Christian Schulte for all the helpful comments and ideas I got from him for my work. I also want to thank Johan Schubert for providing valuable feedback in the final year of my studies. Many thanks go to Professor Alois Ferscha for taking the time to be my opponent and for reading and commenting on this dissertation. A big thank you also goes to Marianne Hellmin and Sandra Gustavsson Nylén for helping me with all the bureaucratic hurdles. I am thankful to KTH and Sweden for giving me this opportunity. This research was partially funded by the “Smart homes for all” project (SM4All, EU FP7 STREP Grant No. 224332) and I would like to thank all my colleagues from this project. In particular, I am grateful to Fei Li and Sanjin Sehic from Vienna University of Technology for our collaboration, which resulted in a number of joint publications that form the basis for this dissertation. I am grateful to the all-knowing Thesaurus for never failing to show me alternative paths. I want to thank Eric-Oluf for proofreading and for all those lunches in Gallerian. Thank you to my fellow PhD students Imran, Shahab, Nima, Kathrin, Voravit, Irfan, Natalja, Salman, Byron, and many others for our semi-regular fikas and for making my workplace a pleasant one. Ett stort tack till Petter, Marianela, Edward, Annika och alla mina andra vänner i Sverige för mycket nöje och distraktion från jobbet. Vielen Dank an Sandra, Tobias, Tanja, Sebastian, Luise und die Familie für die vielen Besuche und für (manchmal dringend nötige) Ablenkung und Freude. Contents List of Figures xi List of Tables xiii List of Algorithms xv 1 Introduction 1 1.1 Visions for the home of the future . 1 1.2 Actual user experiences . 3 1.3 Smart assistants for smart homes . 3 1.3.1 A recommender system for smart homes . 3 1.3.2 An installation assistant for smart homes . 4 1.4 Contributions . 5 1.5 Source material . 8 1.6 Dissertation structure . 9 2 Background 11 2.1 Ubiquitous computing . 11 2.2 Smart homes . 12 2.3 Contributing technologies . 13 2.3.1 Sensing . 13 2.3.2 Reasoning . 14 2.3.3 Acting . 16 2.3.4 Human-Computer Interaction . 17 2.4 User expectations and experiences . 18 2.4.1 Studies of potential smart home users . 18 2.4.2 Studies of actual smart home users and smart home test labs . 19 2.4.3 Study results . 21 2.4.4 Discussion . 24 2.5 Ethical considerations . 26 2.5.1 Privacy . 26 2.5.2 Addressing important human needs . 27 2.6 Summary . 27 vii viii CONTENTS 3 Hyperspace Analogue to Context (HAC) 29 3.1 Requirements . 29 3.1.1 Context sources . 29 3.1.2 User context and context changes . 32 3.1.3 Service precondition and effect . 32 3.1.4 Summary of the requirements . 33 3.2 Hyperspace Analogue to Context (HAC) . 33 3.3 Context points . 35 3.3.1 Definition . 35 3.3.2 Current user context . 36 3.3.3 Changing the user context . 36 3.3.4 Modeling time information . 37 3.4 Context scopes . 39 3.5 Context-altering services . 40 3.5.1 Service preconditions . 40 3.5.2 Service effects . 41 3.5.3 Formal definition of context-altering services . 43 3.6 Operations in HAC . 44 3.6.1 Basis . 44 3.6.2 Projection . 45 3.6.3 Containment . 46 3.6.4 Scope combination . 47 3.7 Smart home datasets used for evaluation . 48 3.7.1 van Kasteren houseA and houseB . 48 3.7.2 Domus . 48 3.7.3 SM4All smart home . 49 3.7.4 Converting the datasets into HAC . 49 3.8 Summary . 52 4 A recommender system for smart homes 53 4.1 Example scenario . 53 4.2 Utilizing the recommendation results . 55 4.2.1 Active context-awareness . 55 4.2.2 Passive context-awareness . 55 4.2.3 Mixed strategy . 56 4.3 Performance requirements . 57 4.4 Overview of the recommender system . 58 4.5 Filtering unavailable services . 59 4.5.1 An algorithm for service filtering . 60 4.5.2 Reuse of previous filtering results . 60 4.5.3 Updated algorithm with reuse of previous results . 61 4.5.4 Runtime analysis . 62 4.6 Evaluation of service filtering . 62 4.6.1 Experimental setup . 62 CONTENTS ix 4.6.2 Inspecting the rate of affected services in smart home datasets . 63 4.6.3 Runtime of the filtering algorithms on smart home datasets . 63 4.6.4 Scalability of the filtering algorithms . 64 4.7 Summary . 66 5 Ranking services based on user habits 67 5.1 Example scenario . 67 5.2 Overview of the proposed method . 68 5.3 Training phase . 69 5.3.1 Extracting observation tuples . 69 5.3.2 Dealing with temporal data . 71 5.3.3 Counting the observations . 72 5.3.4 Incremental learning . 73 5.4 Calculate the service ranking . 73 5.4.1 Context change prediction . 73 5.4.2 Service matching . 76 5.4.3 Interpretation of the results . 77 5.4.4 Runtime analysis . 78 5.5 Evaluation . 79 5.5.1 Choice of datasets . 80 5.5.2 Procedure . 80 5.5.3 Metrics . 80 5.5.4 Comparison with Naïve Bayes . 81 5.5.5 Size of the training dataset . 82 5.5.6 Choice of time intervals . 83 5.5.7 Exploring conflict and uncertainty . 84 5.5.8 Runtimes on test datasets . 86 5.5.9 Algorithm scalability . 87 5.6 Summary . 87 6 Ranking services based on user preferences 89 6.1 User preference model . 90 6.2 Calculate the service ranking . 90 6.2.1 Distance between current context and user preference . 91 6.2.2 Service score . 92 6.2.3 Service ranking . 94 6.2.4 Runtime analysis . 96 6.3 Evaluation . 96 6.3.1 Choice of datasets . ..