End-User Service Composition in Ubiquitous Computing Environments Mark Webster Newman Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2007-138 http://www.eecs.berkeley.edu/Pubs/TechRpts/2007/EECS-2007-138.html November 27, 2007 Copyright © 2007, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. End-User Service Composition in Ubiquitous Computing Environments by Mark Webster Newman B.A. (Macalester College) 1992 M.S. (University of California, Berkeley) 2000 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Professor James Anthony Landay, Co-chair Professor John Canny, Co-chair Professor Athony D. Joseph Professor Peter Lyman Fall 2007 The dissertation of Mark Webster Newman is approved: ___________________________________________________________________ Co-chair Date ___________________________________________________________________ Co-chair Date ___________________________________________________________________ Date ___________________________________________________________________ Date University of California, Berkeley Fall 2007 End-User Service Composition in Ubiquitous Computing Environments Copyright Fall 2007 by Mark Webster Newman 1 ABSTRACT End-User Service Composition in Ubiquitous Computing Environments by Mark Webster Newman Doctor of Philosophy in Computer Science University of California, Berkeley Professor James A. Landay, Co-chair Professor John Canny, Co-chair The era of ubiquitous computing is upon us. We are seeing a sustained explosion in the numbers and types of networked devices and services with which users can interact. This means that great new capabilities are available to end-users, but such capabilities may come at a cost in terms of the complexity of understanding and managing multiple heterogeneous devices and services. In this dissertation, I present work on the design, development, and evaluation of three systems that offer solutions to existing approaches’ shortcomings with regard to developing networked devices and services for end-users. The Obje Framework is a distributed middleware platform that overcomes the problem of piecemeal interoperability by providing a robust interoperability solution for distributed services by dictating minimal up-front agreements and allowing the details of interoperation to be supplied at runtime through the delivery of mobile code. 2 The Obje Display Mirror (ODM) is a service that allows users to connect their laptops to any shared display device within a collaborative work environment, enabling seamless access to and interaction with remote devices. A study of ODM usage across six months indicated that its adoption had impacted workplace information sharing practices in a positive way. OSCAR is an application that allows users to discover, control, and connect devices and services in a home media network. It leverages Obje to provide solutions to both piecemeal interaction and sluggish adaptation by allowing integrated control of all devices on the home network and allowing end-users to compose their own functionality from disparate devices and services. A two-phase user study involving 18 participants with varying degrees of technical skill demonstrates that users could employ OSCAR to create and access a range of functionality and that users were able to identify a wide variety of needs for which OSCAR would provide assistance. The experiences with these systems reported in this dissertation point towards principles for designing frameworks and end-user tools to support an integrated, yet flexible and customizable user experience of ubiquitous computing environments. ________________________________________________ Professor James A. Landay Dissertation Committee Co-chair ________________________________________________ Professor John Canny Dissertation Committee Co-chair i To my amazing long-term domestic partner, Valerie Jo Taylor ii Table of Contents CHAPTER 1 INTRODUCTION 1 1.1 Problems and Opportunities with Networked Services 2 1.1.1 Near-Future Home Networking Scenario 3 1.2 Towards an Integrated User Experience 7 1.3 Thesis and Contributions 11 1.4 Dissertation Outline 14 CHAPTER 2 RELATED WORK 15 2.1 Overview of Related Work 15 2.2 Interoperability Frameworks 18 2.2.1 Universal Plug and Play and the Digital Living Network Alliance 18 2.2.2 Jini Network Technology 20 2.2.3 Web Services 20 2.2.4 Shortcomings of Existing Interoperability Frameworks 21 2.3 Ubicomp Application Frameworks 22 2.3.1 Toolkits for Building Specific Types of Ubicomp Application 23 2.3.2 General-Purpose Ubicomp Application Frameworks 25 2.3.3 Shortcomings of General Ubicomp Frameworks 27 2.4 Automatic Service Composition 28 2.4.1 Automatic Data Flow Path Creation 28 2.4.2 User-Initiated Automatic Service Composition 29 2.4.3 Shortcomings of Automatic Service Composition 30 2.5 Universal Remote Control 31 2.5.1 Infrared-based Universal Remotes 31 iii 2.5.2 Software-based Universal Remote Control of Networked Devices 32 2.5.3 Providing Devices with Web Presence 33 2.5.4 Shortcomings of Universal Remote Control 34 2.6 End-User Composition of Networked Services 35 2.6.1 End-User Programming in the Digital Home 36 2.6.2 Design and Prototyping Tools for Ubicomp Applications 39 2.6.3 Shortcomings of Existing End-User Composition Systems 40 2.7 Summary 42 CHAPTER 3 THE OBJE FRAMEWORK 44 3.1 Introduction 44 3.2 The User Experience of Interoperability 45 3.2.1 Patterns of Communication 47 3.2.2 The Recombinant Computing approach 49 3.2.3 The Role of User-Supplied Semantics 52 3.3 The Obje Framework 54 3.3.1 Bootstrap Protocol and Code Formats 56 3.3.2 The Obje Programming Model 60 3.3.3 Obje Service Roles and Communication Patterns 67 3.3.4 Integrated User Control 85 3.4 Experiences with the Obje Framework 90 3.4.1 Specialized Applications 92 3.4.2 Generic Tools for Service Composition 96 3.5 Discussion 102 3.5.1 Contributions of the Obje Framework 102 3.5.2 Discussion of Specialized and Generic Application Types 104 3.6 Summary 106 CHAPTER 4 THE OBJE DISPLAY MIRROR 109 4.1 Initial Goals and Context 110 4.2 Supporting Unremarkable Computing 113 4.3 Observing Device-Oriented Practices in the Workplace 116 iv 4.3.1 Studying Meeting Rooms 116 4.3.2 Observation Methods 118 4.3.3 Initial Study Results and Observations 121 4.3.4 Preparing to Intervene 128 4.4 The Display Mirror 129 4.4.1 Looking Into the Display Mirror 133 4.5 Discussion 144 4.5.1 Studying Boring Things 144 4.5.2 Challenges for Sustained Deployment, Usage, and Evaluation of Ubicomp Technology 149 4.5.3 Supporting Micro-tasks: Micro-applications or General Tools? 153 4.5.4 The Obje Advantage 157 4.5.5 Contributions and Limitations 159 4.6 Summary 161 CHAPTER 5 DESIGNING AND BUILDING OSCAR 163 5.1 Motivation for OSCAR 164 5.1.1 Stuck in the Middle 164 5.1.2 Focusing on Home Networking 169 5.2 Designing OSCAR 175 5.2.1 Issues with “Custom Configurations” 176 5.2.2 Personas and Scenarios 178 5.2.3 First OSCAR Prototype: Paper Prototype 183 5.2.4 First Expert Evaluation 204 5.2.5 Second OSCAR Prototype: Medium Fidelity Mockups 208 5.3 OSCAR 1: The First Interactive Prototype 226 5.3.1 The OSCAR Testbed 226 5.3.2 The OSCAR1 User Interface 228 5.4 Summary: Towards OSCAR2 240 CHAPTER 6 THE OSCAR USER STUDY 241 6.1 User Study Design 242 6.2 User Study Sessions 247 v 6.2.1 Introduction and scene setting 247 6.2.2 Tasks 248 6.2.3 Questionnaire 249 6.2.4 Screen-by-screen usability assessment 251 6.2.5 Interview 251 6.3 Results of the OSCAR1 Study 257 6.3.1 Screen-by-screen assessment results 263 6.3.2 Reactions to the Recipe Metaphor 265 6.3.3 Subjective Impressions and Interview Results 266 6.4 The OSCAR2 Prototype 267 6.5 Results of the OSCAR2 Study 272 6.5.1 Analysis of breakdowns 276 6.5.2 Additional usability issues 285 6.5.3 Screen-by-screen assessment of OSCAR2 287 6.5.4 Subjective Perceptions of OSCAR1 and OSCAR2 290 6.5.5 Correlations with Computer and Networking Experience 292 6.6 Interview Results 296 6.6.1 Projected Activities 296 6.6.2 Sharing Attitudes 301 6.6.3 Form Factor 304 6.7 Discussion 305 6.7.1 Contributions of OSCAR 307 6.7.2 Significance of the OSCAR User Study 312 6.8 Summary 315 CHAPTER 7 FUTURE WORK 317 7.1 Improvements to OSCAR 318 7.1.1 Usability Improvements 318 7.1.2 Increasing OSCAR’s range of functionality 329 7.2 Studying OSCAR in daily use 333 7.3 Sharing Devices and Media 334 7.4 Sharing Setups Across Households 338 vi 7.4.1 Support for Wide-Area Sharing: Communities of Composition 345 7.5 Summary 347 CHAPTER 8 CONCLUSION 348 8.1 Thesis Statement 348 8.2 Contributions 350 8.3 An Integrated User Experience of Ubiquitous Computing 353 CHAPTER BIBLIOGRAPHY 354 vii List of Figures Figure 1-1: Functionality can be delivered to users in ways that span a spectrum between ease-of-use for any given task and the flexibility to accomplish various different tasks. This dissertation explores end-user composition, which is an attempt to locate an optimal tradeoff between ease-of-use and flexibility in the domain of networked services.