Understanding Remote Collaboration in Video Collaborative Virtual Environments A thesis submitted in partial fulfilment of the requirements for the Degree of Doctor of Philosophy in the University of Canterbury by J¨org Hauber Supervising Committee Dr. Andy Cockburn Supervisor Dr. Mark Billinghurst Co-Supervisor Dr. Holger Regenbrecht Associate Supervisor Dr. Dean Owen Associate Supervisor University of Canterbury 2008 Publications from this dissertation Material from this dissertation has been previously published in the peer- reviewed papers, posters, and extended abstracts listed below. The chapters of this thesis that relate to each publication are noted. I also comment on my individual contribution to each of these papers in the alongside paragraphs. 1. Hauber, J., Regenbrecht, H., Hills, A., Cockburn, A. and Billinghurst, M. (2005) Social Presence in Two- and Three-dimensional Videoconferencing. Presented at PRESENCE 2005, 8th Annual International Workshop on Pres- ence, (London, Sep. 21-23, 2005). (Chapter 4) 2. Hills, A., Hauber, J. and Regenbrecht, H. (2005) Videos in Space: A study on Presence in Video Mediating Communication Systems. Presented at the 15th International Conference on Artificial Reality and Telexistence (ICAT 2005), (Christchurch, Dec. 5-8, 2005). (Chapter 5) 3. Regenbrecht, H., Haller, M., Hauber, J., Billinghurst, M. (2006) Carpeno: Interfacing Remote Collaborative Virtual Environments with Table-Top In- teraction. Virtual Reality - Systems, Development and Applications, 10(2) (September 2006). Springer/London, 95–107. (Chapter 6) 4. Hauber, J., Regenbrecht, H., Billinghurst, M., and Cockburn, A. (2006) Spatiality in videoconferencing: trade-offs between efficiency and social pres- ence. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (Banff, Alberta, Canada, November 04-08, 2006). CSCW ’06. ACM Press, New York, NY, 413–422. (Chapter 6) 5. Hauber, J. (2006) Improving Video-Mediated Communication with a Col- laborative Virtual Environment Approach. Doctoral Colloquium Abstract, ACM Conference on Computer Supported Co-operative work (CSCW ’06), Banff, Alberta, Canada, November 2006. 6. Hauber, J., Regenbrecht, H., Billinghurst, M., Teoh, H. (2006) Presence in Video-mediated Communication: Does being there help to come to- gether? Poster, ACM Conference on Computer Supported Co-operative work (CSCW ’06), Banff, Alberta, Canada, November 2006. (Chapter 7) Most of the papers resulted from collaborative work. My contributions to these papers were (numbers correspond to list numbers above): 1. This study was run by me with the assistance of Aimee Hills and under the supervision of Dr. Regenbrecht. I was responsible for experiment design and data analysis. I also wrote the paper with help from Dr. Regenbrecht, Dr. Cockburn, and Dr. Billinghurst. 2. This study was designed and conducted collaboratively by Aimee Hills and me, again under supervision and assistance of Dr. Regenbrecht. However, I analysed the gathered data separately and wrote most of the paper. 3. The Carpeno project was primarily a research effort by Dr. Regen- brecht and Dr. Haller. I helped with the implementation of the proto- type and conducted two informal user studies during its demonstration at two local conferences. I also wrote the part of the paper describing findings of these studies. The experiences gathered with both hardware and user feedback influenced the design of my follow-up experiments. 4. This was my own research. My supervisors Dr. Regenbrecht, Dr. Billinghurst, and Dr. Cockburn consulted me during the design and data analysis phase and repeatedly reviewed the resulting paper I wrote. 5. This extended abstract was written by me with assistance of my super- visors Dr. Cockburn and Dr. Billinghurst. 6. This poster was designed by me and reflected my ideas for the design of a study that I carried out later in a slightly modified form (see Chapter 7). Signature Date Dedicated to Sandra Abstract Video-mediated communication (VMC) is currently the prevalent mode of telecommunication for applications such as remote collaboration, telecon- ferencing, and distance learning. It is generally assumed that transmitting real-time talking-head videos of participants in addition to their audio is ben- eficial and desirable, enabling remote conferencing to feel almost the same as face-to-face collaboration. However, compared to being face-to-face, VMC still feels distant, artificial, cumbersome, and detached. One limitation of standard video-collaboration that contributes to this feeling is that the 3D context between people and their shared workspace given in face-to-face col- laboration is lost. It is therefore not possible for participants to tell from the video what others are looking at, what they are working on, or who they are talking to. Video Collaborative Virtual Environments (video-CVEs) are novel VMC interfaces which address these problems by re-introducing a virtual 3D con- text into which distant users are mentally “transported” to be together and interact with the environment and with each other, represented by their spa- tially controllable video-avatars. To date, research efforts following this ap- proach have primarily focused on the demonstration of working prototypes. However, maturation of these systems requires a deeper understanding of human factors that emerge during mediated collaborative processes. This thesis contributes to a deeper understanding of human factors. It in- vestigates the hypothesis that video-CVEs can effectively support face-to-face aspects of collaboration which are absent in standard video-collaboration. This hypothesis is tested in four related comparative user studies involv- ing teams of participants collaborating in video-CVEs, through standard video-conferencing systems, and being face-to-face. The experiments apply and extend methods from the research fields of human-computer interaction, computer-supported cooperative work, and presence. Empirical findings indicate benefits of video-CVEs for user experience dimensions such as social presence and copresence, but also highlight chal- lenges for awareness and usability that need to be overcome to unlock the full potential of this type of interface. Table of Contents List of Figures viii List of Tables xi Chapter 1: Introduction 1 1.1 Research context . 2 1.2 Problem statement and research hypothesis . 2 1.3 Contributions . 4 1.3.1 Methodological contributions . 4 1.3.2 Substantive contributions . 4 1.4 Structure of this thesis . 5 Chapter 2: Background 8 2.1 Getting a grip on communication . 9 2.1.1 Shannon and Weaver’s model of communication . 9 2.1.2 Interpersonal communication . 12 2.1.3 Collaboration . 17 2.1.4 Criticisms of transmission models . 20 2.1.5 Summary . 21 2.2 Mediated communication . 22 2.2.1 Media characteristics . 22 2.2.2 Media selection . 30 2.2.3 Remote collaboration in shared workspaces . 32 2.3 Video-mediated communication . 35 2.3.1 Review of empirical work in video-mediated communi- cation . 36 2.3.2 Improving video-mediated communication . 47 2.4 Using virtual reality (VR) as a communication medium . 57 2.4.1 The sense of presence in virtual environments . 58 2.4.2 Collaborative virtual environments (CVE) . 63 2.4.3 The affordances of CVEs for remote collaboration . 64 2.4.4 Examples of CVEs . 67 2.4.5 Empirical research in CVEs . 70 2.5 Chapter summary . 73 Chapter 3: Research Approach 75 3.1 A CVE approach to video-conferencing . 75 3.1.1 The video-CVE “cAR/PE!” . 77 3.2 Method . 82 3.2.1 A comparative approach . 83 3.2.2 Independent measures . 84 3.2.3 Dependent measures . 87 3.3 Overview of the experiments . 88 Chapter 4: Experiment: “Desert Survival Game” 93 4.1 Experiment design . 95 4.1.1 Experimental task . 95 4.1.2 Experiment conditions . 96 4.1.3 Apparatus . 97 4.1.4 Participants . 97 4.1.5 Procedure . 98 4.1.6 Measures . 100 4.2 Data analysis . 100 4.2.1 Social presence as quality of the medium . 100 4.2.2 Social presence as the experience of the other . 101 4.2.3 Exploratory questionnaire items . 101 4.2.4 Scale validity . 102 4.2.5 Age and simulator experience effects . 103 4.3 Results . 103 4.3.1 Social presence as a quality of the medium . 103 4.3.2 Social presence as the experience of the other . 104 4.3.3 Exploratory questionnaire items . 105 4.3.4 Scale validity . 106 ii 4.3.5 Age and simulator experience effects . 108 4.3.6 User observation in condition vCVE . 108 4.4 Discussion . 110 4.5 Experiment summary . 111 Chapter 5: Experiment: “Dream House” 114 5.1 Experiment design . 115 5.1.1 Experimental task . 115 5.1.2 Experiment conditions . 116 5.1.3 Apparatus . 117 5.1.4 Participants . 118 5.1.5 Procedure . 119 5.1.6 Measures . 119 5.2 Data analysis . 120 5.2.1 Social presence . 120 5.2.2 Physical presence . 121 5.2.3 The relation between physical presence and social pres- ence . 121 5.2.4 Reliability analysis . 121 5.2.5 Preference ratings . 121 5.2.6 Exploration of other factors . 121 5.3 Results . 122 5.3.1 Differences in social presence . 122 5.3.2 Differences in physical presence . 122 5.3.3 Exploring the relationship between social presence and physical presence . 122 5.3.4 Reliability analysis . 124 5.3.5 User comments and preference . 126 5.3.6 Gender and relationship effects . 127 5.4 Discussion . 127 5.5 Experiment summary . 128 Chapter 6: Experiment: “Dogs and Owners” 130 6.1 Experiment design . 132 iii 6.1.1 Experimental task . 132 6.1.2 Experiment conditions . 133 6.1.3 Apparatus . 135 6.1.4 Participants . 137 6.1.5 Procedure . 138 6.2 Measures . 139 6.2.1 Social presence . 139 6.2.2 Copresence . 139 6.2.3 Usability parameters . 140 6.2.4 Video recordings . 140 6.3 Data analysis .