2006:60 DOCTORAL T H E SI S Interaction Aspects of Wearable Computing for Human Communication Mikael Drugge Luleå University of Technology Department of Computer Science and Electrical Engineering Media Technology Research Group 2006:60|: 02-544|: - -- 06 ⁄60 -- Interaction Aspects of Wearable Computing for Human Communication Mikael Drugge Media Technology Research Group Department of Computer Science and Electrical Engineering Luleå University of Technology SE–971 87 Luleå Sweden December 2006 Supervisor Ph.D. Peter Parnes, Luleå University of Technology ii Abstract This thesis presents the use of wearable computers for aiding human communication over a distance, focusing on interaction aspects that need to be resolved in order to realize this goal. As wearable computers by definition are highly mobile, always on, and always accessible, the ability to communicate becomes independent of place, time and situation. This also imposes new requirements on the user interface of the wearable computer, calling for natural and unobtrusive interaction with the user. One of the key challenges in wearable computing today is to streamline the user’s inter- action, so that it is tailored for the situation at hand. A user interface that takes too much effort to use, interrupts or requires more than a minimum of attention, will inevitably ham- per the user’s ability to perform tasks in real life. At the same time, human communication involves both effort, interruptions and paying attention, so the key is to find a balance where wearable computers can aid human communication without being intrusive. To design user interfaces supporting this, we need to know what roles different aspects of interaction have in the field of wearable computing. In this thesis, the use of wearable computing for aiding human communication is explored around three aspects of interaction. The first aspect deals with how information can be conveyed by the wearable computer user, allowing a user to retrieve advice and guidance from experts, and remote persons to share experiences over a distance. The thesis presents findings of using wearable computing for sharing knowledge and experience, both for informal exchange among work colleagues, as well as enabling more efficient communication among health-care personnel. The second aspect is based on findings from these trials and concerns how the wearable computer inter- acts with the user. As the user performs tasks in the real world, it is important to determine how different methods of notifying the user affects her attention and performance, in order to design interfaces that are efficient yet pleasant to use. The thesis presents user studies examin- ing the impact of different methods of interruption, and provides guidelines for how to make notifications less intrusive. The third and final aspect considers how the user’s physical inter- action with the wearable computer can be improved. The thesis presents rapid prototyping of systems employing user centric design. Furthermore, a framework for ubiquitous multimedia communication is presented, enabling wearable computers to be dynamically configurable and utilize resources in the environment to supplement the user’s equipment. All in all, the thesis presents how wearable communications systems can be developed and deployed, how their human-computer interaction should be designed for unobtrusive operation, and how they can come to practical use in real world situations. iii iv Contents Abstract iii Preface xi Publications xiii Acknowledgments xv 1 Thesis Introduction 1 1.1 Introduction . .................................. 3 1.2ThesisOrganization............................... 3 1.3 Background and Motivation .......................... 4 1.3.1 WearableComputing.......................... 4 1.3.2 UbiquitousandPervasiveComputing................. 6 1.3.3 VideoConferencingandE-meetings.................. 7 1.3.4 MobileE-meetings........................... 8 1.3.5 MotivationofThesis.......................... 11 1.4ResearchQuestions............................... 11 1.5ScopeandDelimitationoftheThesis...................... 14 1.6 Research Methodology . .......................... 14 1.7SummaryofIncludedPublications....................... 16 1.8WearableComputingforHumanCommunication............... 18 1.8.1 Mobile E-Meetings through Wearable Computing ........... 19 1.8.2 ManagingInterruptionsandNotifications............... 22 1.8.3 PrototypingandDeployingMobileE-MeetingSystems........ 24 1.9Discussion.................................... 28 1.9.1 FutureResearchDirections....................... 31 v vi Contents 1.9.2 Conclusions............................... 31 1.10PersonalContribution.............................. 32 2 Sharing Experience and Knowledge with Wearable Computers 35 2.1 Introduction . .................................. 37 2.1.1 EnvironmentforTesting........................ 38 2.2RelatedWork.................................. 38 2.3TheMobileUser................................ 38 2.3.1 HardwareEquipment.......................... 39 2.3.2 SoftwareSolution............................ 40 2.4 Beyond Communication . .......................... 41 2.4.1 BecomingaKnowledgeableUser................... 41 2.4.2 InvolvingExternalPeopleinMeetings................. 42 2.4.3 WhenWearableComputerUsersMeet................. 43 2.5Evaluation.................................... 44 2.5.1 TheImportanceofText......................... 44 2.5.2 CameraandVideo........................... 46 2.5.3 Microphone and Audio . ................... 46 2.5.4 TransmissionofKnowledge...................... 46 2.6Conclusions................................... 47 2.6.1 FutureWork.............................. 47 2.7Acknowledgements............................... 47 3 Experiences of Using Wearable Computers for Ambient Telepres- ence and Remote Interaction 49 3.1 Introduction . .................................. 51 3.1.1 RelatedWork.............................. 52 3.2EverydayTelepresence............................. 54 3.3WearableComputers.............................. 56 3.4ExperiencesofTelepresence.......................... 58 3.4.1 UserInterfaceProblems........................ 59 3.4.2 ChoiceofMediaforCommunicating................. 61 3.5Evaluation.................................... 62 3.5.1 TimeforSetupandUse........................ 62 3.5.2 DifferentLevelsofImmersion..................... 63 Contents vii 3.5.3 AppearanceandAesthetics....................... 66 3.5.4 RemoteInteractionsmadePossible.................. 68 3.5.5 Summary................................ 68 3.6Conclusions................................... 69 3.6.1 FutureWork.............................. 69 3.7Acknowledgments............................... 70 4 Methods for Interrupting a Wearable Computer User 71 4.1 Introduction . .................................. 73 4.1.1 RelatedWork.............................. 74 4.2Experiment................................... 75 4.2.1 RealWorldTask............................ 75 4.2.2 InterruptionTask............................ 76 4.2.3 CombiningtheTasks.......................... 76 4.2.4 Treatments............................... 77 4.3UserStudy................................... 79 4.3.1 TestSession............................... 79 4.3.2 Apparatus................................ 80 4.4Results...................................... 82 4.4.1 ComparisonwithBaseCases...................... 83 4.4.2 PairwiseComparisonofTreatments.................. 84 4.4.3 ComparisonwithOriginalStudy.................... 85 4.4.4 SubjectiveComments......................... 85 4.5Conclusions................................... 86 4.5.1 FutureWork.............................. 86 4.6Acknowledgments............................... 86 5 Using the "HotWire" to Study Interruptions in Wearable Com- puting Primary Tasks 87 5.1 Introduction . .................................. 89 5.1.1 Motivation............................... 89 5.1.2 Outline . .............................. 90 5.2RelatedWork.................................. 90 5.3Experiment................................... 91 5.3.1 PrimaryTask.............................. 91 viii Contents 5.3.2 InterruptionTask............................ 92 5.3.3 Methods for Handling Interruptions . ............... 92 5.4UserStudy................................... 93 5.4.1 Apparatus................................ 94 5.5Results...................................... 96 5.5.1 Time.................................. 98 5.5.2 Contacts................................ 99 5.5.3 Errorrate................................101 5.5.4 Averageage...............................101 5.6Evaluatingtheapparatus............................101 5.7Conclusions...................................102 5.7.1 FutureWork..............................103 5.8Acknowledgments...............................103 6 Wearable Systems in Nursing Home Care: Prototyping Experi- ence 105 6.1 Introduction . ..................................107 6.2ScopingtheProject...............................108 6.3PaperPrototyping................................109 6.3.1 Paper,Pen,andPlastic.........................109 6.3.2 PaperPrototypingBenefits.......................110 6.4 Moving to Multimodal Devices . ...................111 6.4.1 WearablePrototype...........................111 6.4.2 CommunicationApplication......................111 6.4.3 WizardofOzTesting..........................112 6.4.4 FeedbackFromtheNurses.......................113 6.5FinalRemarks..................................114 6.6Acknowledgments...............................115 7 Enabling Multimedia