Mobile Web Surveys: a First Look at Measurement, Nonresponse, and Coverage Errors by Christopher Antoun A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Survey Methodology) in The University of Michigan 2015 Doctoral Committee: Professor Frederick G. Conrad, Chair Professor Mick P. Couper Assistant Professor Predrag Klasnja Assistant Professor Brady T. West © Christopher Antoun All rights reserved 2015 DEDICATION To Rachel and baby Rose ii ACKNOWLEDGMENTS I would never have been able to finish my dissertation without support and encouragement from many people. I would like to thank the members of my dissertation committee – Fred Conrad, Mick Couper, Brady West, and Predrag Klasnja – for their time and effort. I am grateful to the staff of CentERdata at Tilburg University in the Netherlands because this dissertation makes use of data obtained through the LISS panel administered by CentERdata and they carried out the experiment that is the foundation of this dissertation. In particular, I would like to thank Maarten Streefkerk who was my point of contact for the project. I appreciate the generous financial support provided by Dr. James Morgan by way of a SEARCH Grant and Ms. Marilyn Lieberman by way of the Daniel Katz Dissertation Fellowship. These grants helped me focus on my dissertation research during this past year. I also appreciate the financial support provided by Ms. Jean Campbell through the Angus Campbell Graduate Fellowship. I would like to thank my fellow students, including those how have moved on to other things, for their support, feedback, and friendship. I want to thank Kirsten Alcer for supervising me in my first research position at Survey Research Operations and for supporting my decision to enroll in the PhD program. I would like to express my sincere thanks to Mick Couper who introduced me to the topic of mobile Web surveys and provided a two-semester guided reading course that helped shape this dissertation. Many (if not most) of the ideas in this dissertation came iii from our thought-provoking conversations, which always made me feel excited about this line of research. I would like to express my deepest gratitude to my advisor and dissertation chair, Fred Conrad, for his guidance and encouragement. I was lucky to have his trust and support along the way. Despite his other commitments, he always made time for our weekly meetings (which I will greatly miss), interesting conversations about research ideas, and editing drafts of my work. Most importantly, he instilled in me a sense of confidence that I was up for task of producing original research, for which I cannot thank him enough. Finally, I want to acknowledge the sacrifices made by my dear wife, Rachel, while I pursued this degree that are too numerous to describe. She was always there for me through the easy times and challenging times to ground me and motivate me to give my best effort. iv TABLE OF CONTENTS DEDICATION .................................................................................................................... ii ACKNOWLEDGMENTS ................................................................................................. iii LIST OF TABLES ........................................................................................................... viii LIST OF FIGURES ............................................................................................................ x LIST OF APPENDICES ................................................................................................... xii ABSTRACT ..................................................................................................................... xiii Chapter 1: Introduction ....................................................................................................... 1 Introduction ..................................................................................................................... 1 Measurement on small devices ....................................................................................... 5 Nonresponse in mobile Web surveys .............................................................................. 9 Coverage error and total error in mobile Web surveys ................................................. 11 Conclusions ................................................................................................................... 12 Chapter 2: Effects of Mobile versus PC Web on Survey Response Quality: a Crossover Experiment in a Probability Web Panel ............................................................................ 14 Summary ....................................................................................................................... 14 Introduction ................................................................................................................... 15 Methods......................................................................................................................... 25 v Results ........................................................................................................................... 39 Discussion ..................................................................................................................... 55 Chapter 3: Influence of Sample Person Characteristics on Nonresponse in a Mobile Web Survey ............................................................................................................................... 62 Summary ....................................................................................................................... 62 Introduction ................................................................................................................... 63 Methods......................................................................................................................... 72 Results ........................................................................................................................... 80 Discussion ..................................................................................................................... 90 Chapter 4: Decomposing Mobile versus PC Web Mode Effects into Different Error Components in a Probability Web Panel .......................................................................... 95 Summary ....................................................................................................................... 95 Introduction ................................................................................................................... 96 Methods....................................................................................................................... 105 Results ......................................................................................................................... 114 Discussion ................................................................................................................... 121 Chapter 5: Conclusion..................................................................................................... 126 Summary ..................................................................................................................... 126 The future of mobile Web ........................................................................................... 128 Further research/Research agenda .............................................................................. 134 vi Appendices ...................................................................................................................... 139 References ....................................................................................................................... 154 vii LIST OF TABLES Table 1.1: Summary of observed mode differences between mobile and PC Web ............ 7 Table 2.1: Factors that could differ between PC Web and mobile Web survey modes and the implications for data quality in mobile Web ............................................................... 19 Table 2.2: Completion rates in mobile Web and PC Web by period ................................ 30 Table 2.3: Demographics of participants in mobile Web and PC Web ............................ 32 Table 2.4: Indicators of satisficing.................................................................................... 34 Table 2.5: Adjusted differences in indicators of satisficing between mobile and PC Web ........................................................................................................................................... 42 Table 2.6: Estimated parameters in multivariate models regressing satisficing indicators on experimental factors, demographic control variables, attention to the survey, and device familiarity .............................................................................................................. 44 Table 2.7: Adjusted differences in socially undesirable responding between mobile and PC Web ............................................................................................................................. 46 Table 2.8: Estimated parameters in multivariate models regressing socially undesirable responding on experimental factors, demographic control variables, context variables, and user characteristics. .................................................................................................... 48 Table 3.1: Attributes hypothesized to affect participation