Increasing Passersby Engagement with Public Large Interactive Surfaces by Victor Cheung A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Doctor of Philosophy in Systems Design Engineering Waterloo, Ontario, Canada, 2016 ©Victor Cheung 2016 AUTHOR'S DECLARATION This thesis consists of material all of which I authored or co-authored: see Statement of Contributions included in the thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. ii Statement of Contributions The DISCOVER interaction model introduced in Chapter 3, and the studies presented in the subsequent chapters, are the result of collaborative work between me and my research colleagues. It is often difficult to state the exact amount of each person’s contributions or the exact evolution of the research. However, the following notes provide some indication for the amount of their involvement. 1. The DISCOVER interaction model was developed by me, in collaboration with Dr. Stacey Scott. An earlier, simplified version of this model was used by Mindy Seto in her thesis work (Seto, 2012) and a study on menu discoverability (Seto et al., 2012). I was the author and presenter of the paper describing the early stages of this model in a peer-reviewed international conference under the following citation: Cheung, V. (2014). Improving Interaction Discoverability in Large Interactive Displays. In Doctoral Symposium of ITS 2014: ACM Interactive Tabletops and Surfaces. Dresden, Germany, November 16-19, 2014. 2. The laboratory-based study methodology presented in Chapter 4 was developed by me, in collaboration with Dr. Stacey Scott. I was the lead author and presenter of the paper describing this methodology in a peer-reviewed international conference under the following citation: Cheung, V., Scott, S.D. (2015). A Laboratory-based Study Methodology to Investigate Attraction Power of Large Public Interactive Displays. In Proceedings of UbiComp 2015: ACM International Joint Conference on Pervasive and Ubiquitous Computing, p. 1239- 1250. Osaka, Japan, September 7-11, 2015. 3. The pilot study presented in Chapter 5 was conducted by me, in collaboration with Dr. Stacey Scott and co-op student Frank Cento. The application used in the study was implemented by me, with the assistance from undergraduate research assistant Joanne Leong. iii 4. The improved study presented in Chapter 6 was conducted by me, in collaboration with Dr. Stacey Scott. The application used in the study was implemented by me, with the assistance from co-op student Shrey Khosla. I was the lead author and presenter of the paper reporting the findings from this study in a peer-reviewed international conference under the following citation: Cheung, V., Scott, S.D. (2015). Studying Attraction Power in Proxemics-Based Visual Concepts for Large Public Interactive Displays. In Proceedings of ITS 2015: ACM Interactive Tabletops and Surfaces, p. 93-102. Madeira, Portugal, November 15-18, 2015. 5. The field experiment presented in Chapter 7 was conducted by me, in collaboration with Dr. Stacey Scott and master’s student Mojgan Ghare. The application used in the study was implemented by me. The video annotation analysis was performed in collaboration with undergraduate research assistant Caroline Wong. iv Abstract Despite the proliferation of Public Large Interactive Surfaces (PLISs), and their potential to provide a more engaging and interactive user experience, these surfaces often go unnoticed by passersby, or not immediately comprehensible in terms of usage. Current research in addressing this problem involves modeling the user-surface interaction through observational studies, and deriving recommendations for interface design to facilitate the interaction. This approach is often context-specific, requires elaborate setup, and lacks experimental control. To mitigate this problem, an interaction model, named DISCOVER, was developed by drawing ideas from classic usability research and focusing on the discoverability aspect of the interaction. This approach allows the model to serve as a lens for understanding and synthesizing existing work on PLISs, and to be used as an evaluation framework to assess effectiveness of potential designs. To accompany this evaluation capability, a laboratory-based evaluation methodology was developed to allow researchers to quickly implement and evaluate potential designs, particularly for the early stages of interaction that precede the more commonly studied explicit and direct interaction (e.g., touches, mid-air gestures). Using the model and the evaluation methodology, a proximity-based interaction mechanism using animated content and shadow visualizations was designed and evaluated as an effective technique in drawing attention from unknowing study participants. A follow-up, more conventional in-the-wild study also verified this finding, and further demonstrated the usefulness of shadow visualizations in drawing attention from passersby, retaining them, and enticing playful interaction. The goal of this thesis is to better equip researchers and practitioners of PLISs with tools that allow them to evaluate and improve existing interfaces, and to provide them with insights into designing future ones employing better and more engaging technologies. v Acknowledgements Rome was not built in a day, nor was it built by one person. This thesis will not be possible without the generous help and support from many important people, whom I am deeply grateful. I will try my best to thank those who have made the most profound impact on this research and to me personally as I have spent countless days and nights to its fruition. First, I thank my supervisors, Dr. Stacey Scott and Dr. Edward Lank for taking me in as their student. They have been my advisors, mentors, and friends. I cannot thank them enough for their insights, support and encouragement. I really appreciate the different perspectives they have and I hope to enjoy many years of continued work with them. I would also like to extend my thanks to my comprehensive examination committee, Dr. Karen Collins, Dr. Carolyn MacGregor, and Dr. Daniel Stashuk, for their feedback in shaping my thesis research early on, and their understanding in the time required for my research to complete. I would also like to thank Dr. Andrés Lucero for being my external examiner, and bringing new perspectives to my research. I am also grateful to my funding sources, which have allowed me to focus my time and efforts on this research: Natural Sciences and Engineering Research Council of Canada (NSERC), NSERC Digital Surface Software Application Network (SurfNet), the Engineering School at University of Waterloo, and University of Waterloo. I would also like to thank all the research colleagues I have had the pleasure of working with. To the people at Collaborative Systems Lab directed by Dr. Stacey Scott, the people at TouchLab directed by Dr. Mark Hancock, the people at Social Usable Interactive Technologies directed by Dr. Parmit Chilana, the people at Games Institute directed by Dr. Neil Randall, and the people at User Interface & Software Engineering at Otto-von-Guericke-Universität Magdeburg, Germany, directed by Dr. Raimund Dachselt. I would also like to acknowledge several people who have gone the extra metres for me during this time. To Yu-Ling Betty Chang, my lab-mate and good friend, who always gives me good insights into my research and writings, and feeds me with endless delicious snacks. To Diane Watson, my colleague and good friend, who brain-storms with me the craziest research ideas and shows me a person can both be an excellent researcher and cook. To John Harris, my colleague and good friend, who introduced me to the amazing world of game design and development that inspires my research projects. vi This thesis would have never been completed without the continuous support of my family and friends. They have provided their support in each of their own special ways that no others can replace. Thanks to my parents Anita and Felix, and my brother Kelvin, for believing in me and their unconditional support in every aspect. Thanks to my family and friends for understanding it is not appropriate to keep asking a PhD student when he is going to graduate. Lastly, to my Lord, who guides me through all stages of my life. May this thesis glorify His name. vii Table of Contents AUTHOR'S DECLARATION ............................................................................................................... ii Statement of Contributions ................................................................................................................... iii Abstract .................................................................................................................................................. v Acknowledgements ............................................................................................................................... vi Table of Contents ................................................................................................................................ viii List of Figures ..................................................................................................................................... xiv List of Tables .....................................................................................................................................