Understanding Interactivity Steps to a Phenomenology of Human-Computer Interaction Dag Svanæs Preface and Acknowledgements iii Contents Preface and Acknowledgements..............................................................................................vii PART I: THEORY Chapter 1 Introduction .........................................................................................................1 1.1 Beyond Metaphor...............................................................................................................................3 1.2 Methodology: How to Study Interactivity? ....................................................................................7 1.3 The Larger Cultural Backdrop......................................................................................................10 1.4 Research Questions and Structure of the Study ..........................................................................17 Chapter 2 Interactivity in the HCI Literature.....................................................................20 2.1 Narrowing the Scope: Three Scenarios.........................................................................................21 2.2 Seven Theoretical Frameworks......................................................................................................27 2.3 The Model Human Processor.........................................................................................................30 2.4 Cognitive Science .............................................................................................................................33 2.5 Phenomenology and Speech Act Theory.......................................................................................44 2.6 Ethnomethodology...........................................................................................................................49 2.7 Activity Theory.................................................................................................................................54 2.8 Semiotics............................................................................................................................................60 2.9 Computers as Theatre .....................................................................................................................68 2.10 What do the Theories Shed Light on?.........................................................................................75 2.11 Blind Spots ......................................................................................................................................77 2.12 A Common Theme in the Blind Spots.........................................................................................78 2.13 Related Discourses and Comparisons..........................................................................................80 Chapter 3 Non-Cartesian Alternatives................................................................................86 3.1 The Human Body in Heidegger......................................................................................................87 3.2 Maurice Merleau-Ponty's "Phenomenology of Perception"......................................................88 3.3 An Example: The Mr. Peters Button.............................................................................................93 3.4 Merleau-Ponty on the Three Scenarios.........................................................................................97 3.5 Mark Johnson's "The Body in the Mind"....................................................................................99 3.6 Discussion........................................................................................................................................101 Chapter 4 Computer as Material ......................................................................................102 4.1 The Nature of the Computer ........................................................................................................102 4.2 The Abstract Art of Kandinsky ...................................................................................................103 4.3 Abstraction in the Computer Medium........................................................................................108 4.4 How is Abstract Art Experienced?..............................................................................................110 iv Understanding Interactivity PART II: EXPERIMENTS Chapter 5 Doing Empirical Research on Interactivity......................................................115 5.1 Relevant Research Traditions ......................................................................................................116 5.2 Evaluation Criteria........................................................................................................................121 5.3 Overall Experiment Design...........................................................................................................124 5.4 Measures Taken to Meet the Criteria .........................................................................................126 Chapter 6 Square World ...................................................................................................128 Chapter 7 Exp. A: Metaphors We Interact By ..................................................................133 7.1 Method.............................................................................................................................................134 7.2 Three Cases in Great Detail..........................................................................................................136 7.3 Results .............................................................................................................................................143 Coding scheme and method of analysis...................................................................................143 The use of verbs ...............................................................................................................144 The metaphors .................................................................................................................146 Between-example effects ....................................................................................................154 The diversity of the descriptions ...........................................................................................155 7.4 Discussion........................................................................................................................................157 Chapter 8 Exp. B: Designing Interactive Behavior ..........................................................162 8.1 Method.............................................................................................................................................163 8.2 The State-of-the-Art Editors ........................................................................................................165 An event-action language....................................................................................................165 A rule-sheet editor ............................................................................................................167 Programming-by-demonstration ...........................................................................................168 State-Transition Diagrams...................................................................................................169 The improved version ........................................................................................................170 8.3 The Empirically-Based Editors....................................................................................................171 A natural-language editor....................................................................................................171 The physical metaphors ......................................................................................................172 The house metaphor ..........................................................................................................173 Designing with Interaction Gestalts .......................................................................................174 The improved version ........................................................................................................177 8.4 Discussion........................................................................................................................................178 Chapter 9 Exp. C: Participatory Tool Design ...................................................................182 9.1 Method.............................................................................................................................................183 9.2 The First 10 minutes......................................................................................................................185 9.3 Results .............................................................................................................................................191 9.4 Discussion........................................................................................................................................196 Preface and Acknowledgements v PART III: REFLECTIONS Chapter 10 Making Sense of Square World......................................................................203 10.1 The Interactive Experience.........................................................................................................203