Using&and&Collecting&Annotated&Behavioral& Trace&Data&for&Designing&and&Developing& Context9Aware&Application& by& Yung9Ju&Chang& A&dissertation&submitted&in&partial&fulfillment&& of&the&requirements&for&the&degree&of&& Doctor&of&Philosophy&& (Information)& in&the&University&of&Michigan& 2016& Doctoral&Committee:&& Associate&Professor&Mark&W.&Newman,&Chair& Professor&Mark&S.&Ackerman& Associate&Professor&Natalie&Colabianchi& Assistant&Professor&Predrag&Klasnja ACKNOWLEDGEMENTS First and foremost, I would like to thank my research advisor Mark W. Newman at the Interaction Ecologies Group for his great support and advice for my research work and career plan throughout my time as a Ph.D. student. Thanks for all the help, encouragement, critiques, feedback, advice, and time. Especially thanks to his guiding me back to my research when I was attempting to launch a startup in my third year of Ph.D. I also want to thank him for allowing me to explore different research topics over these years, including information seeking and environmental cognition. Although I am not able to include the pieces of work in this dissertation, my literature background in these areas are important knowledge assets allowing me to explore new directions at the intersection between these areas and my dissertation. I want to thank all of the dissertation committee members: Mark Ackerman, Pedja Klasnja, and Natalie Colabianchi. It has been a wonderful intellectual experience working with them. I want to thank their thoughtful comments that improve the focus and the scope of the dissertation and suggest areas to explore in my future research. I want to thank John Tang, my mentor at Microsoft Research. I first want to thank him choosing me as an intern at Microsoft Research in 2012, for which I was able to start my journey in mobile communication and interruptibility, a very important research direction in this dissertation and of my future research agenda. It was an excellent experience working with John, and this internship was my first time starting to use a mixed-method approach to study mobile users’ behaviors. Without his mentorship, I would not have been able to use these methods in my other research studies. I also want to thank Soo Young Rieh and Malcolm McCullough, my committee members of the preliminary ii exam, for providing me with thoughtful comments and feedback on the topic of information seeking in the urban environment from different perspectives. I want to thank all of the (former) colleagues at the Interaction Ecologies Group, who collaborated with me and contributed to the projects I managed. In no particular order (probably omitting some): Dong Tao, Rayoung Young, Pei-Yao (Perry) Hung, I-Chun Hsiao, Manchul Han, Chuan-Che (Jeff) Huang, Shriti Raj. Special thanks to Rayoung, who has been with me, been thoughtful and considerate, and encourages me when I was stressed throughout this journey. I have been enjoying the time discussing our own research with Rayoung. It is my best pleasure to graduate with you at the same time! I want to thank all of friends and colleagues who helped the Ubicomp labeling study, including Hsin-Ying Wu, Hsin-Yu Lin, and Noureen Dharani. Thanks to all faculty members who have been teaching me skills and knowledge in the past few years. Thanks to colleagues in the ACM SIGCHI community for their inputs and discussions on my research projects at conferences. Thanks to the research and administration staff at UMSI for providing reliable assistance for my research. Thanks to Judy Dyer, Christine Feak, and Pamela Bogart who helped me improve my English skills and this dissertation through their thoughtful and thorough feedback. Thanks to Shih-Hsuan Chou, Min-Chih Liu, Surong Ruan, and Lezong Li, who started the language learning project with me, and Yi-Wei Chia, Kevin Chang, Kerry Kao, and Morgan Chen who were willing to continue this project, although we could not make it eventually. It was a pity that we were not able to continue the project together, for now, because my choice of focusing on my dissertation. But it has been a great experience in working with you guys. And I believe at some point the plan is going to work out. Last but not least, thanks to my dear family members and friends. Without your love, support, and encouragement, this wouldn’t have been possible. iii TABLE OF CONTENTS ACKNOWLEDGEMENTS .............................................................................................. ii LIST OF FIGURES ......................................................................................................... x LIST OF TABLES ........................................................................................................ xv ABSTRACT ............................................................................................................... xvi CHAPTER 1. INTRODUCTION .......................................................................................1 1.1 THESIS STATEMENT AND CONTRIBUTIONS .................................................................. 5 1.2 DISSERTATION OUTLINE .......................................................................................... 7 CHAPTER 2 LITERATURE BACKGROUND ......................................................................9 2.1 USING CAPTURED BEHAVIORAL AND CONTEXTUAL DATA IN INTERACTION DESIGN ............ 9 2.2 SUPPORTING DATA COLLECTION IN MOBILE CROWDSENSING/SOURCING ...................... 12 2.2.1 Support Participants .................................................................................... 15 2.2.2 Assessing Participants’ Data Contributions................................................. 21 2.2.3 Participants Selection, Recruitment, and Task Distribution ........................ 22 2.2.4 Mobile Crowdsensing/sourcing Platforms and Campaigns ........................ 25 2.3 MOBILE DATA COLLECTION SYSTEMS ...................................................................... 26 2.3.1 Mobile Data Collection Systems for Mobile Crowedsensing ....................... 27 2.3.2 Mobile Data Collection Systems Supporting Context-Awareness ............... 28 2.4 COLLECTING ANNOTATIONS ON BEHAVIORAL AND CONTEXTUAL DATA ......................... 32 2.4.1 Collecting Annotations From Users Who Provide the data ......................... 32 2.5 MOBILE INTERRUPTIBILITY, RECEPTIVITY, AND OPPORTUNE MOMENTS ......................... 35 CHAPTER 3 USING CAPTURE-AND-PLAYBACK TO SUPPORT PROTOTYPING, TESTING, AND EVALUATION OF CONTEXT-AWARE APPLICATIONS: FINDINGS AND LESSONS LEARNED ................................................................................................................. 44 3.1 INTRODUCTION .................................................................................................... 44 3.2 RESEARCH GOALS AND APPROACH .......................................................................... 51 3.2.1 Case Studies ................................................................................................. 52 iv 3.2.2 User Studies and Continue System Improvement ....................................... 53 3.3 CASE STUDIES ...................................................................................................... 54 3.3.1 Case Study 1: LoungeBoard ......................................................................... 54 3.3.2 Case Study 2: BusBuddy .............................................................................. 64 3.3.3 Lesson Learned from the Case Studies ........................................................ 72 3.3.4 Summary ..................................................................................................... 76 3.4 THE REPLAY USER STUDY ...................................................................................... 76 3.4.1 The Testbed: Here & Now (H&N)................................................................ 77 3.4.2 Participants ................................................................................................. 81 3.4.3 Study Tasks and Procedure ......................................................................... 82 3.4.4 Study Results and Findings .......................................................................... 84 3.5 INITIAL IMPROVEMENTS: TRACEVIZ ......................................................................... 90 3.5.1 A Scenario of Using TraceViz ....................................................................... 91 3.5.2 The TraceViz Interface ................................................................................. 92 3.5.3 Brushing to Explore and Filter Traces .......................................................... 93 3.5.4 The TraceViz User Study .............................................................................. 97 3.6 TOWARDS A COMPREHENSIVE TOOLSET: CAPLA ...................................................... 100 3.6.1 The Clip Browser ........................................................................................ 101 3.6.2 The Clip Editor ........................................................................................... 103 3.6.3 The Clip Player ........................................................................................... 104 3.6.4 Extensions: Labels, Markup, and Transforms ............................................ 105 3.6.5 The Annotation and Markup Pipeline ....................................................... 107 3.6.6 Implementation Details ............................................................................. 110 3.6.7 The