Open Research Online The Open University’s repository of research publications and other research outputs The role of metaphor in user interface design Thesis How to cite: Treglown, Mark (2002). The role of metaphor in user interface design. PhD thesis. The Open University. For guidance on citations see FAQs. c 2002 The Author Version: Version of Record Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyright owners. For more information on Open Research Online’s data policy on reuse of materials please consult the policies page. oro.open.ac.uk The Role of Metaphor in User Interface Design Mark Treglown MSc Applied Artificial Intelligence, University of Aberdeen, 1991 BSc (Hons.) Computer Science, University of Sussex, 1990 Submitted for the degree of Doctor of Philosophy in Human-Computer Interaction Institute of Educational Technology, The Open University, Milton Keynes. December 1999 THE OPEN UNlVE RESEARCH SCHO Library Authorisation Please return this form to the Research School with the two bound copies of your thesis to be deposited with the University Library. AIl candidates should complete parts one and two of the form. Part three only applies to PhD candidates. Part One: Candidates Details Name: .. M A. RK ... T.R~.t;[r.~ ~.~ ............................ PI: ...~.7..~.~.~ 7. 2..' ..... Degree: .....r. .~.~........................................................................................... ,...... .. Thesis title: ...'THE ... gQ~.e. ....QF.. .. M.€T.~.PHQR.. ..IN ... u SE. ~... .'.~R.FA'E. .. ......... ......... .l>.~~.~~N ....................................................................................... .. Part Two: Open University Library Authorisation I confirm that I am willing for my thesis to be made available to readers by the Open University Library, ::n:t Itd:.d°~~:.;;tl~C~~O~Of.~e~i~~:~: ... tt: ... ~ .. ~~ .. Part Three: British Library Authorisation [PhD candidates only] If you want a copy of your PhD thesis to be available on loan to the British Library Thesis Service as and when it is requested, you must sign a British Library Doctoral Thesis Agreement Form. Please return it to the Research School with this form. The British Library will publicise the details of your thesis and may request a copy on loan from the University Library. Information on the presentation of the thesis is given in the Agreement Foml. Please note the British Library have requested that theses should be printed on one side only to enable them to produce a clear microfilm. The Open University Library sends the fully bound copy of theses to the British Library. The University has agreed that your participation in the British Library Thesis Service should be voluntary. Please tick either (a) or (b) to indicate your intentions. ~ I am willing for the Open University to loan the British Library a copy of my thesis: A signed Agreement Form is attached D I do not wish the Open University to loan the British Library a copy of my thesis. Signed: .... I.1.~ .. --:-..I.~.r.J.n ....................... Date: ..L+.: ... ~ .. ~~ .. h:vaxm\word\fonlls\EX12 Abstract The thesis discusses the question of how unfamiliar computing systems, particularly those with graphical user interfaces, are learned and used. In particular, the approach of basing the design and behaviour of on-screen objects in the system's model world on a coherent theme and employing a metaphor is explored. The drawbacks, as well as the advantages, of this approach are reviewed and presented. The use of metaphors is also contrasted with other forms of users' mental models of interactive systems, and the need to provide a system image from which useful mental models can be developed is presented. Metaphors are placed in the context of users' understanding of interactive systems and novel application is made of the Qualitative Process Theory (QPT) qualitative reasoning model to reason about the behaviour of on-screen objects, the underlying system functionality, and the relationship between the two. This analysis supports re­ evaluation of the domains between which user interface metaphors are said to form mappings. A novel user interface design, entitled Medusa, that adopts guidelines for the design of metaphor-based systems, and for helping the user develop successful mental models, based on the QPT analysis and an empirical study of a popular metaphor-based system, is described. The first Medusa design is critiqued using well-founded usability inspection method. Employing the Lakoff/lohnson theory, a revised verSIOn of the Medusa user interface is described that derives its application semantics and dialogue structures from the entailments of the knowledge structures that ground understanding of the interface metaphor and that capture notions of embodiment in interaction with computing devices that QPT descriptions cannot. Design guidelines from influential existing work, and new methods of reasoning about metaphor-based designs, are presented with a number of novel graphical user interface designs intended to overcome the failings of existing systems and design approaches. Acknowledgements My grateful thanks go to my supervisor Tim O'Shea, for his advice, encouragement, editorial advice, and for his patience. I would like to thank Debbie Stone and Vanessa Evers for proofreading previous drafts, and for their invaluable comments. The work reported in this thesis benefited from conversations with Gregory Abowd, Mark Elsom-Cook, Alan Dix, Thomas Green, Kim Issroff, Tim O'Shea, Steve Schneider, Randall Smith, and Debbie Stone among many others. Thank you to Matt Smith, Don Clark and Mark Elsom-Cook for lending me hardware and office space to conduct the MacLeaming study reported in this thesis. Many thanks and sincere apologies are due to Richard Bomat, Gill Ritchie, Pete Woodward, Ben du Boulay, Des Watson and Roger Sinnhuber for help with empirical work that eventually proved impossible to conduct. I would like to thank Simon Holland who was responsible in large part for my studying at the Open University. Thank you to Eileen Scanlon and all in the CALRG. Thanks are due to Marian Petre for providing Sun workstations, and to Yibing Li for making sure they ran more often than not. I would like to thank Dave Perry, Jon Oliver, Dave Signorini, Rob Griffiths and Colin Pink for technical support. Thank you to everyone in cyberspace who provided advice, references, code fragments and who sent useful papers; keep on rockin' in the free world. Louis Feinberg and Jerome and Maurice Horowitz also provided invaluable assistance. "Technical texts are generally understood to report work that their authors have done; they are focused on machinery in a broad sense, be it hardware, software, or mathematics. They open by making claims - 'Our machinery can do such and such and others' cannot' - and they confine themselves to demonstrating these claims in a way that others can replicate. They close by sketching further work - more problems, more solutions. Critical texts, by contrast, are the work that their authors have done. Their textuality is in the foreground, and they are focused on theoretical categories. - They open by situating a problematic in an intellectual tradition, and they proceed by narrating their materials in a way that exhibits the adequacy of certain categories and the inadequacy of others. They close with a statement of moral purpose." - P. E. Agre (1997) Computation and Human Experience, Cambridge University Press: xiii. "This is the time, and this is the record of the time. 11 - Laurie Anderson (1979) United States live I-W, Warner Brothers Records. Table of Contents Chapter 1: Introduction 1 1.1 The Problem 1 1.2 A Solution - Metaphor Recommended 3 1.3 A Solution? Metaphor Also Considered Harmful 4 1.4 A Solution - Ne\v Metaphors and Approaches to Metaphor 5 1.5 Overview of the Thesis 6 Chapter 2: Existing Approaches to the Use of Metaphor and 13 Analogy in User Interface Design 2.1 Introduction 13 2.1.1 WIMP Systems 14 2.2 The Desktop 17 2.3 Rooms 18 2.4 The Al ternate Reali ty Ki t 24 2.5 Metaphor and non-visual representation 29 2.5.1 Auditory Icons 30 2.5.2 SonicFinder 32 2.5.3 SharedARK 33 2.6 The "Reality" Metaphor and New Interaction Styles 34 2.6.1 Optical Metaphors 36 2.7 Conclusions 39 Chapter 3: An Empirical Study of First-time Macintosh Users 41 3.1 Overview of the study 43 3.1.1 The Subjects 43 3.1.2 Methodology 43 3.1.3 Tasks Performed by Subjects 44 3.1.4 Caveats 45 3.2 Observations 46 3.2.1 Using the Manual 46 3.2.2 Using the On-line Help Facility 47 3.2.3 Interpreting the Desktop Metaphor 48 3.2.4 Basic User Interaction 52 3.3 Conclusions 56 Chapter 4: Drawbacks to Employing Metaphors and 58 Analogies in Interactive User Interfaces 4.1 Introduction 58 4.2 Operational Metaphors 59 4.3 Structural A pproaches to Metaphor 63 4.4 Structural Approaches to Metaphor and Learning of Computer- 68 Based Systems 4.5 The Pragmatics of Metaphor 72 4.5.1 WIMP Systems 73 4.5.2 Instruction 73 ii 4.5.3 Basic user interaction 74 4.5.4 The Desktop 75 4.6 Discussion - Metaphor and System Learning and Use 76 4.7 Types and Theories of Metaphor 82 4.7.1 Interaction Theories 83 4.7.2 Metaphor and Analogy 85 4.8 Is Metaphorical Understanding of User Interfaces 88 Possible? 4.9 Cognitive Semantics of User Interface Metaphors 93 4.9.1 Image Schemata and Metaphorical Projection for 94 Understanding 4.9.2 The Lakoffllohnson Theory in HCI 98 4.10 Conclusions 101 Chapter 5: Users' Models of Interactive Systems 102 5.1 Introduction 102 5.2 Types of Users' Models of Systems 105 5.2.1 Networks 108 5.2.2 Glass Box Models 109 5.2.3 Surrogates 110 5.2.4 Task-action Mappings 115 5.2.5 Qualitative Models as Mental Models 116 5.3 The Role of the Display as Source of Information in System 123 Learning and Use 5.4 Using the Lakoffllohnson Model for Analysis and 125 Design of User Interfaces 5.4.1 Case Study I:An Immersive Environment 126 5.4.2 Case Study 2:Snap-Dragging 129 5.4.3 Case Study 3:The Apple Macintosh Trashcan 131 5.