Practical Strategies for Integrating a Conversation Analyst in an Iterative Design Process We present a case study of an iterative design process significant difficulties frequently arise in applying rich, highly-contextual qualitative that includes a conversation analyst. We discuss potential findings to the process of designing and benefits of conversation analysis for design, and we building a concrete artifact. There are many mismatches between social sciences and describe our strategies for integrating the conversation other disciplines such as engineering or analyst in the design process. Since the analyst on our design – professional vocabulary, accus- tomed time scale, goals of inquiry, even team had no previous exposure to design or engineering, philosophical outlook [23]; when combined and none of the other members of our team had any with the time limitations endemic in engi- neering environments, these mismatches experience with conversation analysis, we needed to build typically (at least in the reported literature – a foundation for our interaction. One of our key strategies see, e.g., [19]) result in fairly restricted forms of engagement between social was to pair the conversation analyst with a designer in scientists and designers. a highly interactive collaboration. Our tactics have been In the course of a recent project, we were effective on our project, leading to valuable results that we strongly motivated to incorporate a partic- ular qualitative method, conversation believe we could not have obtained using another method. analysis (CA) [21], into our design process. We hope that this paper can serve as a practical guide The project goal was to build an electronic audio guidebook that would enhance social to those interested in establishing a productive and efficient interaction between tourists. CA focuses on working relationship between a conversation analyst and the processes by which humans organize the other members of a design team. their activities and interactions; as such, CA has obvious potential to inform the design of computer systems that affect human- KEYWORDS INTRODUCTION human interaction. While the CA literature Conversation analysis, electronic Social science research methods are often has been used as a source of design princi- guidebooks, iterative design, qualitative applied in the design of interactive ples in HCI (e.g., [10]), actually applying CA research methods systems. In particular, qualitative methods as a method in iterative design seems far such as ethnography can provide useful more problematic. CA involves extended, insights about the environments into which detailed examination of recorded interac- a new system will be introduced. However, tions; the analytic framework emphasizes Permission to make digital or hard Allison Woodruff copies of all or part of this work for Margaret H. Szymanski personal or classroom use is granted without fee provided that copies are Rebecca E. Grinter not made or distributed for profit or Paul M. Aoki commercial advantage, and that copies bear this notice and the full Palo Alto Research Center citation on the first page. To copy 3333 Coyote Hill Road otherwise, to republish, to post on Palo Alto, CA 94304-1314 USA servers or to redistribute to lists, requires prior specific permission and/or a fee." ACM-1-58113-515-7/02/0006...$5.00 DIS2002 | 19 objectively-observed behavioral instances design team, and in particular, between began. The design work is primarily done rather than interpretations of behavior by a conversation analyst and a designer. by the designer (who has an interdiscipli- the analyst; and the resulting findings are nary background, including computer The remainder of the paper is organized as often quite subtle and require considerable science) in conjunction with one of the follows. Since this paper is about experi- background knowledge to understand. computer systems researchers and the ences as well as methods, we provide conversation analyst, although many In this paper, we share our experiences in some necessary background by describing design meetings involve all members of (successfully) integrating a conversation relevant aspects of our project. We then the team. analyst into an iterative design process. We describe conversation analysis and how believe this report will be useful to others design teams might profit from its use, Prototypes for at least two reasons. First, we do not turning to an extended example of how CA We have developed two main prototypes know of a previous detailed discussion of has helped improve our own system. In the (Figure 1). Each prototype has two main this topic. Second, we describe a process following section, we discuss the strategies components, a visual interface (which is the that proceeded “from the ground up” – the we have used to integrate the conversation same in each prototype) and an information conversation analyst on our team had no analyst in the team to achieve these bene- delivery mechanism (which is different in previous exposure to design or engi- fits. Finally, we discuss related work and the two prototypes). neering, and none of the other members of conclude. our team had any experience with conver- Prototype 1 sation analysis. THE ELECTRONIC GUIDEBOOK At the beginning of the project, we PROJECT We report the methods by which we have performed a task analysis and developed a To provide context for the rest of the paper, integrated the conversation analyst in the design for a visual interface [3]. Individual we briefly describe our project, including design process. For example, the conver- visitors use this interface to obtain informa- the team members involved, the prototypes sation analyst is paired with the designer in tion about objects in their environment. The developed, and the studies conducted. a highly interactive collaboration: the interface resembles a set of Web browser Most of this information has been reported conversation analyst and the designer often imagemaps; at a given time, the visitor sees previously and in more detail [2, 27]. work together on both the conversation a single photographic imagemap that analysis and the design. depicts one wall of a room in a historic Team house. When visitors tap on an imagemap Using the methods to be described, we Over a two-year period, six members of target, the guidebook delivers a description have found conversation analytic methods Xerox PARC have worked intermittently on of that object. to be enormously valuable to our work. the electronic guidebook project: an inter- While CA is time- and resource-intensive, action designer, a conversation analyst, In Prototype 1, visitors had a choice of we have found ways to manage these a fieldworker, two computer systems information delivery modes: text descrip- costs, and we believe that the analysis has researchers, and a computer science tions, audio descriptions played through yielded results that we would not have intern. None of these members are full-time headphones, or audio descriptions played achieved using another method. We hope on this project, and some of these through speakers. Prototype 1 users that this paper can serve as a practical members joined the project after it was predominantly chose audio played through guide to those interested in establishing a already underway; of specific relevance is speakers. productive and efficient working relation- that the designer was one of the founders ship between a conversation analyst and of the project, and the conversation analyst joined the project about half a year after it Prototype 2 uses eavesdropping (headphone audio + wireless networking) Both employ a visual interface based on imagemaps, using tap tips to indicate targets as needed Prototype 1 uses open air model (speaker audio) Figure 1: Comparison of Prototype 1 (left) and Prototype 2 (right). 20 | DIS2002 Prototype 2 structured interviews with visitors after they research, is concerned with describing the The main difference between Prototype 1 used the electronic guidebooks. The three methods by which the members of a and Prototype 2 lies in the information studies can be summarized as follows: culture engage in social interaction. A key delivery mechanism. Realizing that audio goal of conversation analysis is to examine played through speakers is not a feasible Study 1 (October 2000) social interaction to reveal organized prac- solution in a public space due to noise Study 1 participants used Prototype 1, tices or patterns of actions, under the disturbance issues, we moved in Prototype which offered a choice of delivery mode fundamental assumption that interaction is 2 to a single information delivery mode: an (text, audio through headphones, or audio structurally organized. Social actions audio eavesdropping model that uses through speakers). Study 1 included 14 include talk, gesture, and use of objects. headsets but preserves the shared listening individuals (seven pairs) from the Xerox While ethnomethodology and conversation capability of audio played through PARC community. Participants visited two analysis share this concern for how actions speakers. Specifically, devices are paired. rooms on a day Filoli was closed to the are organized, the goal of CA is to describe Each visitor in a pair always hears the general public. both how sequences of action are orga- content they select themselves, and addi- nized and situated in a particular instance tionally, each visitor has a volume control Study 2 (July 2001) of activity, as well as to abstract features for determining how loudly they hear Study 2 participants used Prototype 2, that generalize across a collection of similar content from their companion. The volume which had eavesdropping. Study 2 instances. can be set to “Off,” “Quiet,” or “Loud” included 12 individuals (six pairs) from the (“Loud” being the same volume as clips Xerox PARC community. Like Study 1, A conversation analytic research program selected on one’s own device). A priority Study 2 was conducted on a day Filoli was involves analyzing a collection of interactive model addresses overlapping clips; if visi- closed to members of the general public.