The challenge of web design guidelines: Investigating issues of awareness, interpretation, and efficacy by Stephen James Szigeti A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Faculty of Information University of Toronto © Copyright by Stephen James Szigeti 2012 The challenge of web design guidelines: Investigating issues of awareness, interpretation, and efficacy Stephen James Szigeti Doctor of Philosophy Faculty of Information University of Toronto 2012 Abstract Guidelines focusing on web interface design allow for the dissemination of complex and multidisciplinary research to communities of practice. Motivated by the desire to better understand how research evidence can be shared with the web design community, this dissertation investigates the role guidelines play in the design process, the attitudes designers hold regarding guidelines, and whether evidence based guidelines can be consistently interpreted by designers. Guidelines are a potential means to address the knowledge gap between research and practice, yet we do not have a clear understanding of the relationship between research evidence, guideline sets and web design practitioners. In order to better understand how design guidelines are used by designers in the practice of web interface design, four sequential studies were designed; the application of a guideline subset to a design project by 16 students, the assessment of ten health information websites by eight designers using a guideline subset, a web based survey of 116 designers, and interviews with 20 designers. The studies reveal that guideline use is dependent on the perceived trustworthiness of the guideline, its source and the alignment between guideline advice and designer experience. The first two studies found that guidelines are inconsistently interpreted. One third of the guidelines used in the second study were interpreted differently by participants, ii an inconsistency which represents a critical problem in guideline use. Findings showed no difference in the characteristics of guidelines which were consistently interpreted and those for which interpretation was the most inconsistent. Further, research evidence was not a factor in guideline use, less than half the designers are aware of evidence-based guideline sets, and guidelines are predominantly used as memory aids. Ultimately alternatives to guidelines, such as checklists or pattern libraries, may yield the best results in our efforts to share research knowledge with communities of practice. iii Acknowledgements While thanks are due to many people who helped me in my journey to complete this dissertation, I reserve the greatest thanks and gratitude to my supervisor, Dr. Joan Cherry. Her guidance, wisdom and support throughout my time in the doctoral program have shaped my understanding of what it means to be a good researcher. I am grateful for the excellent advice and guidance offered by my committee members Dr. Mark Chignell, Dr. Kelly Lyons and Dr. Chun Wei Choo. Mark not only contributed to the design of the research, but offered support and a different perspective that enriched my work. At various key moments, Kelly and Chun Wei offered a welcome mix of reassurance and suggestions. Thanks to my external appraiser, Dr. Gary Perlman, for his insights and invaluable comments. I am also grateful for the involvement of Dr. Elaine Toms, under whose guidance I started this journey, and Dr. Paul Muter, who was a great source of advice in the early stages. I wish to thank fellow researchers who provided help, including M. Max Evans, Luanne Freud, Glen Farrelly, Colin Furness, Rhonda McEwen, and Scott Pacquette. Thanks to Adam Fiser and Tamás Dobozy for good advice. Thanks are due to the Inforum staff for their help, and to my colleagues in the field of design for their support, including Peter Mansour and Petros Petrakis. Involving the design community in my research was made easier through the help of Karel Vredenburg and Ian Chalmers. Thanks to my brother Dave for not only finding me a place to write in Berlin, but for his support and good humour. Thanks to Alex, Zoë and Will who are not only glad to have their dad back from his studies, but who can finally see that school does indeed have an end. Finally, thanks to my partner Natalie for her patience, encouragement, support and for being a keen and tireless reader during the final years of my doctoral research. iv Table of Contents Abstract ............................................................................................................................... ii Acknowledgements ............................................................................................................... iv List of Figures ........................................................................................................................ viii List of Tables ........................................................................................................................ ix List of Appendices ................................................................................................................ xi Chapter 1: Introduction ......................................................................................................... 1 1.1 Introduction ......................................................................................................... 1 1.2 Research context .................................................................................................. 1 1.3 Definitions............................................................................................................ 3 1.4 Purpose and significance of research .................................................................. 7 1.5 Scope and delimitations ....................................................................................... 8 1.6 Central research question and structure of dissertation ...................................... 8 Chapter 2: Literature Review ................................................................................................ 10 2.1 Introduction ......................................................................................................... 10 2.2 Guidelines outside the domain of Human Computer Interaction ....................... 10 2.3 Knowledge translation theory .............................................................................. 13 2.4 Interface Design Guidelines ................................................................................ 18 2.4.1 Guidelines supported by expert opinion .............................................. 21 2.4.2 Guidelines supported by research evidence ......................................... 32 2.5 Research regarding interface design guidelines .................................................. 42 2.5.1 Interpretation of guidelines .................................................................. 43 2.5.2 Tools for the use of interface design guidelines .................................. 46 2.5.3 The relationship between user performance and compliance with interface design guidelines ........................................................ 51 2.6 Research regarding the user of guidelines: the designer ..................................... 55 2.7 Discussion and related research questions .......................................................... 57 Chapter 3: The application of interface design guidelines (Study #1) .................................. 61 3.1 Introduction ......................................................................................................... 61 3.2 Methodology ....................................................................................................... 64 3.2.1 Method ................................................................................................. 62 3.2.2 Apparatus ............................................................................................. 63 3.2.3 Participants ........................................................................................... 64 3.2.4 Procedure ............................................................................................. 65 3.2.5 Measures .............................................................................................. 65 3.3 Results ............................................................................................................ 66 3.3.1 Guideline use in the design of projects ................................................ 66 3.3.2 Focus group sessions ............................................................................ 71 3.4 Summary ............................................................................................................ 82 v Chapter 4: The interpretation and use of guidelines for web-site assessment (Study #2) .... 87 4.1 Introduction ......................................................................................................... 87 4.2 Methodology ....................................................................................................... 87 4.2.1 Method ................................................................................................. 87 4.2.2 Apparatus ............................................................................................. 89 4.2.3 Participants ........................................................................................... 91 4.2.4 Procedure ............................................................................................