Wearable Computer Interaction Issues in Mediated Human to Human Communication Mikael Drugge Division of Media Technology Department of Computer Science and Electrical Engineering Luleå University of Technology SE–971 87 Luleå Sweden November 2004 Supervisor Peter Parnes, Ph.D., Luleå University of Technology ii Abstract This thesis presents the use of wearable computers as mediators for human to human commu- nication. As the market for on-body wearable technology grows, the importance of efficient interactions through such technology becomes more significant. Novel forms of communi- cation is made possible due to the highly mobile nature of a wearable computer coupled to a person. A person can, for example, deliver live video, audio and commentary from a re- mote location, allowing local participants to experience it and interact with people on the other side. In this way, knowledge and information can be shared over a distance, passing through the owner of the wearable computer who acts as a mediator. To enable the mediator to perform this task in the most efficient manner, the interaction between the user, the wear- able computer and the other people involved, needs to be made as natural and unobtrusive as possible. One of the main problemsof today is that the virtualworld offered by wearable computers can become too immersive, thereby distancing its user from interactions in the real world. At the same time, the very same immersion serves to let the user sense the remote participants as being there, accompanying and communicating through the virtual world. The key here is to get the proper balance between the real and the virtual worlds; remote participants should be able to experience a distant location through the user, while the user should similarly experience their company in the virtual world. To make both experiences as efficient and nat- ural as possible, the user’s interaction with the wearable computer needs to be exceptionally streamlined. In this thesis, the use of wearable computing technology as a means for enhanced human to human communicationis explored. This is done by extending traditional e-meetings with a mobile participant using a wearable computer. Through experiences at different events, fairs and exhibitions in both controlled and uncontrolled environments, the communication system has been studied in real life situations. Focus has been laid on the interaction aspect, what advantages and drawbacks there are with the system and how the issues can be resolved. Furthermore, a user study has been performed on how the wearable computer should no- tify the user when involved in real life tasks — highly applicable for the purpose of mobile e-meetings, where the user may interact with persons in the real and virtual world simultane- ously. On a similar note, a novel yet intuitive interaction method is introduced, demonstrating all the pros and cons of the natural way of physically “throwing” information among users. iii iv Contents Publications vii Acknowledgments ix 1 Thesis Introduction and Summary 1 1.1 Introduction................................... 3 1.2 Background................................... 4 1.3 Motivation.................................... 6 1.4 TheNomadicCommunicator. 7 1.5 SharingExperiences .............................. 9 1.6 SummaryandConclusions . 12 1.7 ThesisOrganization. .... .... ... .... .... .... ... ... 14 2 Sharing Experience and Knowledge with Wearable Computers 17 2.1 Introduction................................... 19 2.2 RelatedWork .................................. 20 2.3 TheMobileUser ................................ 20 2.4 BeyondCommunication ............................ 22 2.5 Evaluation.................................... 25 2.6 Conclusions................................... 28 2.7 Acknowledgements............................... 28 3 Experiences of Using Wearable Computers for Ambient Telepres- ence and Remote Interaction 29 3.1 Introduction................................... 31 v vi Contents 3.2 EverydayTelepresence . 34 3.3 WearableComputers .............................. 36 3.4 ExperiencesofTelepresence . ... 37 3.5 Evaluation.................................... 41 3.6 Conclusions................................... 47 3.7 Acknowledgments ............................... 48 4 Methods for Interrupting a Wearable Computer User 49 4.1 Introduction................................... 51 4.2 Experiment ................................... 53 4.3 UserStudy ................................... 56 4.4 Results...................................... 59 4.5 Conclusions................................... 63 4.6 Acknowledgments ............................... 64 5 Eventcasting with a Wearable Computer 65 5.1 Introduction................................... 67 5.2 RelatedWork .................................. 68 5.3 TheoryandDesign ............................... 69 5.4 Prototype .................................... 75 5.5 Discussion.................................... 76 5.6 Conclusions................................... 76 5.7 Acknowledgements............................... 77 Publications This licentiate thesis consists of an introduction and four published papers. The introductory chapter provides a discussion of all papers and their relationship with each other, together with ideas for future work in the area of research. All papers have been published at inter- national peer reviewed conferences or workshops. I am the main author of three papers and co-author of one paper. Paper 1 Marcus Nilsson, Mikael Drugge, and Peter Parnes, “Sharing Experience and Knowledge with Wearable Computers”, In Proceedings of Pervasive 2004 Workshop on Memory and Sharing of Experiences, April 2004. Paper 2 Mikael Drugge, Marcus Nilsson, Roland Parviainen, and Peter Parnes, “Experiences of Using Wearable Computers for Ambient Telepresence and Re- mote Interaction”, In Proceedings of the 2004 ACM SIGMM Workshop on Effective Telepresence, October 2004. Paper 3 Mikael Drugge, Marcus Nilsson, Urban Liljedahl, Kåre Synnes, and Peter Parnes, “Methods for Interrupting a Wearable Computer User”, In Proceedings of the 8th IEEE International Symposium on Wearable Computers (ISWC’04), November 2004. Paper 4 Mikael Drugge, Marcus Nilsson, Kåre Synnes, and Peter Parnes, “Eventcasting with a Wearable Computer”, In Proceedings of the 4th International Workshop on Smart Appliances and Wearable Computing (IWSAWC’04), March 2004. vii viii Acknowledgments First, I would like to thank my supervisor Dr. Peter Parnes for all his guidance, support and encouragement to always strive for excellence. I would also like to thank my secondary ad- visor Dr. Kåre Synnes for valuable comments, discussions and advice given. A posthumous thanks goes to the late Dr. Dick Schefström for his grand visions that served as inspiration when I first started working here. Thanks also to my colleagues at the Division of Media Technology and the Centre for Distance-spanning Technology for making this such a wonderful place to work and conduct research in. Without your wits, wisdom and friendship, it would never have been this reward- ing and fun to work here. During my time here, I have worked in close collaboration with Marcus Nilsson who deserves a special thanks since much of our research is the result of joint efforts. Special thanks also to my friends both inside and outside of academia, I won’t mention any names but I am quite certain you know who you are. Finally, I would like to thank my family for always supporting me in whatever endeavour I have undertaken. Luleå, November 2004 Mikael Drugge ix x Part 1 Thesis Introduction and Summary 1 Thesis Introduction and Summary 3 1.1 Introduction This thesis presents the use of wearable computers as mediators for human to human com- munication. Throughout history, communication among humans has constituted a part of the evolution of mankind; early cave paintings, the tradition of storytelling and the art of writ- ing are all examples of ways of sharing knowledge and experiences. Advances in technology have eased how this information is produced, ranging from the use of cuneiform writing tools for clay and stone, to pens and pencils for writing on paper and brushes for painting pictures. The invention of the printing press and photography enabled an easier way to produce this information, while telegraphs, telephones, television and, in the recent decades, computer networks and mobile phones, made it easier to convey over a distance. The Internet of today allows audio, video, commentary and illustrations to be shared in real-time, with little or no regard to the physical distance between people. One thing worth noticing, is that how the user interacts with this technology is essential for how well the information can be produced, conveyed and shared with others. For example, the typewriter is superseded by computers that allows changing textual contents more easily, the recording and playback of audio and video can be replaced with digital equivalents for similar reasons, while this digital data can be transmitted to specific people regardless of their whereabouts. In a sense, less focus needs to be given to the underlying technology, allowing a person to pay more attention to the con- tents of the communication — that is, after all, what remains important regardless of any changes in technology. In communicating for the purpose of sharing experiences, the mobility of the user is im- portant. For an artist to accurately depict a scene taken from nature, she must be at that location. For a reporter