The Design and Evaluation of Multiple Interfaces: A Solution for Complex Software by Joanna McGrenere A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Department of Computer Science The University of Toronto © Copyright by Joanna McGrenere 2002 ii Abstract Computer software has become increasingly complex as advances in technology permit substantially more functionality to be provided to the user, a phenomenon which has led some people to describe today’s heavily featured software as “bloated”. Despite the prevalence of this trend, the impact of complexity on the user has received little attention in the research community. This dissertation describes research that addresses that problem. Study One, a comprehensive study that looked at the experience of 53 users of Microsoft Word, showed that while many users would like to have unused functions “tucked away”, most users were uncomfortable with the complete removal of unused functions. These findings suggested personalization as a promising direction for design and led to our Pilot Study which evaluated a multiple-interfaces prototype for Microsoft Word, where one of the interfaces was personalized to the user’s individual needs. Results from that informal Wizard-of-Oz evaluation with 4 participants encouraged refinement of our prototype. Study Two, a field study that included 20 participants, tested the effects of different interface designs on users’ satisfaction and their perceived ability to navigate, control, and learn the software. There were two conditions: Microsoft Word with adaptive menus, and our user- adaptable multiple-interfaces prototype. Results showed that participants felt that they were better able to both navigate through the menus and toolbars and to learn with our multiple- interfaces prototype. There were also important differences in satisfaction and control with the new design. iii This dissertation shows multiple interfaces to be a promising design solution to complex software. The novelty of our design is the combination of three design elements: 1) Multiple interfaces, one is personalized, one is the full set of functions, and switching between interfaces requires a single button click. 2) The personal interface is adaptable by the user with an easy-to-understand adaptation mechanism. 3) The personal interface begins small and, therefore, unless the user adds many functions, it will be a minimal interface relative to the full interface. Important advances in the understanding of individual differences with respect to the perception of complex software and in the understanding of adaptive versus adaptable interfaces are made. iv Acknowledgements I am very grateful to my multidisciplinary supervisory committee. Co-supervisors: Dr. Ronald Baecker (Computer Science, University of Toronto) who agreed to supervise this research even though it was outside the scope of his main projects at the time. Among other things, Ron encouraged me to evaluate the prototype early in the design cycle, which became our Pilot Study, and provided insightful input into the Study Two protocol. Dr. Kellogg Booth (Computer Science, University of British Columbia) who has proven that supervision at a distance can be done. Kelly was present throughout this research and at difficult times he always seemed to be there with sound judgment and direction. The spirit with which he supervised, his keen attention to detail, and his mega emails will not be forgotten. Surrogate supervisor for Study One and committee member: Dr. Gale Moore (Sociologist, Knowledge Media Design Institute, University of Toronto) who played an instrumental role in the early stages of this work and generously gave her time and expertise to mentoring a young academic. Together Gale and I shared both the joys and the challenges of doing cross-disciplinary research. Committee members: Dr. Mark Chignell (Mechanical Industrial Engineering, University of Toronto) who brought substantial expertise in the area of experimental design and data analysis. Mark made me welcome in his lab and allowed me to take part in his research group. Dr. Ken Sevcik (Computer Science, University of Toronto) who always participated with a smile. Ken was thoroughly reliable and was never short of thoughtful comments. v Others who have directly contributed to this dissertation: Dr. Mary Czerwinski (Cognitive Psychology, Microsoft Research) who facilitated access to a number of internal-use-only Microsoft tools and provided significant assistance with both the experimental design of Study Two and the resulting data analysis. Dr. Saul Greenberg (Computer Science, University of Calgary) who acted as the external examiner at my Senate Oral and made valuable recommendations that improved the quality of this dissertation. Dr. Ian Spence (Psychology, University of Toronto) who participated at my Senate Oral and gave his stamp of approval on the exploratory experimental approach taken in this work. And lastly, those who contributed indirectly to this work: Dr. Kori Inkpen who provided first rate accommodation at all the CHI conferences along with lots of late night conversation. My fellow DGP lab mates who made a fun and collegial research environment. Chris, Gwen, Tom, Scott, Tim, Melanie, and Joan who were always there providing much love and encouragement. I gratefully acknowledge the sources of funding and in-kind support that made this research possible: Communications and Information Technology Ontario (CITO) funded the Learning Complex Software project from 1998 to 2000. Our Study One was the first research to emerge from this cross-disciplinary project. Dr. Gale Moore of the Knowledge Media Design Institute is the principal investigator of this project. IBM Toronto Lab provided 3 years of graduate funding in the form of an IBM Center for Advanced Studies (CAS) Fellowship. vi Microsoft provided two software loggers (MSTracker and MS Instrumented Version) and an MSOffice expertise screening questionnaire. These tools were all developed for internal Microsoft use only but were made available for use in this research. NSERC provided 2 years of graduate funding in the form of a Post Graduate Scholarship B (PGSB). Substantial portions of Chapter Three and Chapter Five of this dissertation have already been published in the proceedings of the CIPS Graphics Interface 2000 conference and the ACM CHI 2002 conference. The questionnaire Experiencing Word Processing, copyright by Moore and McGrenere 1998, is included as Appendix F. vii viii Table of Contents CHAPTER 1 Introduction................................................................................................. 1 1.1 Research Motivation................................................................................................. 1 1.2 Research Objectives and Overview .......................................................................... 2 CHAPTER 2 Related Work .............................................................................................. 9 2.1 How is Software Changing? ..................................................................................... 9 2.2 Why Featurism? Why the Complexity?.................................................................. 11 2.3 What is the User’s Experience of Complex Software?........................................... 13 2.3.1 What is the Impact on Learning?.................................................................... 14 2.3.2 What Functions are Actually Being Used?..................................................... 18 2.3.3 Summary......................................................................................................... 20 2.4 Design Approaches for Complex Software ............................................................ 21 2.4.1 Functionality Blocking....................................................................................21 2.4.2 Menu Design................................................................................................... 22 2.4.3 Customization/Personalization ....................................................................... 27 2.4.4 Intelligent User Interfaces (IUIs).................................................................... 30 2.4.5 Summary......................................................................................................... 48 CHAPTER 3 Study One – Capturing Users’ Experience of Complex Software........ 51 3.1 Study Design........................................................................................................... 52 3.1.1 Software Application Studied......................................................................... 52 3.1.2 Participants...................................................................................................... 53 3.1.3 Instruments and Data Collection..................................................................... 54 3.1.4 How Our Work May Be Differentiated from that of Others .......................... 56 3.2 Results..................................................................................................................... 57 ix 3.2.1 Quantitative method: Counting functions....................................................... 57 3.2.2 Qualitative Method: The Users’ Experience................................................... 60 3.2.3 Summary......................................................................................................... 68 3.3 Evaluating and Extending Microsoft’s Study on User Profiling ...........................