How to Apply the Anthropological Technique of Participant Observation to Knowledge Acquisition for Expert Systems

Home , Participant observation

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 5, SEPTEMBERIOCTOBER 1792 983 How to Apply the Anthropological Technique of Participant Observation to Knowledge Acquisition for Expert Systems

Mary A. Meyer

Abstract-Participant observation requires that the researcher (interviewing), having the individual think aloud while per- not only observes but participates in the activities of those forming a particular task (verbal reports’), and/or observing persons being studied. The paper describes and illustrates how the individual’s behavior. the anthropological method of participant observation can be used in manual knowledge acquisition to: 1) familiarize the Participant observation relates to the aspects of knowledge knowledge engineer with the domain prior to interviews of the acquisition concerned with gathering information from human experts or users; 2) obtain information about the experts’ or sources-experts and users. The performance of participant users’ unconscious behaviors; 3) learn the experts’ or users’ observation is likely to overlap with the other techniques for views of themselves and their actions; 4) investigate the social gathering information from experts and users. For instance, processes by which expertise is developed and supported; and 5) aid the knowledge engineer’s construction of personal models of participant observation places researchers in a face-to-face the events and objects in the domain. Participant observation is situation with the experts/users, thereby giving the researcher compared to similar, more commonly used knowledge acquisi- ideas for future interview questions or for testing by other tion techniques, such as apprenticeship learning. Suggestions are methods. given on how to use participant observation within knowledge I propose that participant observation can serve the follow- engineering projects. ing five purposes: 1) to familiarize knowledge engineers with the domain as I. INTRODUCTION a first step prior to any elicitations; N PARTICIPANT OBSERVATION, the researcher is not a 2) to obtain information on the experts’ and users’ behav- Idetached observer but a participant in the activities of those iors that cannot be reliably obtained through interviews; being studied. In cultural anthropology, participant observation 3) to place knowledge engineers in a position to hear the is defined as a experts’ or users’ views of themselves, their own, and others’ actions; field method whereby the ethnographer is immersed 4) to investigate the social processes by which expertise is in the day-to-day activities of the community being developed and supported; and studied . . . The objective of this method is to minimize 5) to aid knowledge engineers in constructing personal the presence of the field worker as a factor affecting models of the events and objects in the domain. the responses of the people and to provide a record of observed behavior under varying conditions [ 11. Recently anthropological approaches have been proposed for use in knowledge acquisition [2], [3]. Forsythe and I believe that participant observation is a valuable strat- Buchanan [3] reason that anthropological techniques are egy for knowledge acquisition, defined here as the acquiring applicable because of the overlap between the goals of artificial and modeling of appropriate problem solving knowledge for intelligence and anthropology. Both fields seek to investigate the purpose of building expert or knowledge-based systems. “how human beings understand, organize, process and make Knowledge acquisition usually involves the following pro- use of symbolic information” and rely on the same basic data cesses: gathering knowledge from human or textual sources gathering methods-interviews with individuals, text analysis, (books, reports, data bases, case studies, and empirical data); and observations. They note that anthropology, because of interpreting, modeling, and analyzing this information; and its longer history and extensive literature on its methods, representing it in a machine-readable form that can be entered has much to offer the subfield of knowledge acquisition. into the system. Human sources may include experts whose In particular, they consider participant observation a valid problem-solving knowledge is being elicited or potential users technique for knowledge engineering. of the system whose needs and patterns of interaction with This paper is organized as follows. In Section 11, I present the computer are being determined. With humans, knowl- background information on participant observation: the history edge acquisition usually involves some type of questioning of its practice in cultural anthropology, the type of thinking that it involves, and a comparison of it to other related knowledge Manuscript received June 17, 1991; revised February 21, 1992. M. A. Meyer is with the Statistics Group, MS F600, at the Los Alamos acquisition techniques. In Section 111, I give examples of the National Laboratory, Los Alamos, NM 87545. IEEE Log Number 9201397. As described in Ericsson and Simon [30].

U.S. Government work not protected by US. Copyright 984 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 5, SEPTEMBERIOCTOBER 1992 application of participant observation to the five proposed the observed. Connected knowing is a way of thinking based goals. In Section IV, I describe some of the assumptions about on empathy. In connected knowing, the researcher “ . . . uses knowledge that are implicit in participant observation and offer him or herself as a tool for the investigation. The researchers guidance on the use of participant observation in knowledge [6] use their reactions to understand others’ view and to acquisition. Section IV is a summary. formulate hypotheses about the other participant’s reactions.” In this sense, participant observation is a kind of “disciplined 11. BACKGROUND subjectivity” [ 61.

A. History of Participant Observation in Anthropology C. Relation of Participant Observation to Other KA Practices and Techniques The development of participant observation is formally credited to Brownislaw Malinowski, a Polish anthropologist, Participant observation resembles two techniques commonly who began his field studies in the early 1900’s [l]. He used in knowledge acquisihion-on-site observation and ap- and other pioneering anthropologists initiated the practice of prenticeship learningteachback interviews. On-site observa- residing in the community of study for extended periods of tion is described as watching the expert solve real prob- time. These anthropologists went beyond relying on second- lems on the job [7]. Thus, on-site observation differs from hand, and usually erroneous reports of the people from traders participant observation in that passive observations do not and missionaries by actually living with the natives and involve participation or systematic empathizing. For example, observing them. in conducting on-site observations of experts, the knowledge Participant observation has traditionally been seen as means engineer would merely watch detachedly in the background; of improving rapport with those being studied [4].In addition, whereas in participant observation, the knowledge engineer anthropologists have found that participant observation has would, at least, mentally and, usually physically, put himself or frequently stimulated them to generate hypotheses which could herself in the place of the expert (e.g., performing the activities then be examined through the use of more structured methods, that the expert did). Thus, in doing participant observation, as such as interviewing and tests. For example, after conducting opposed to passive observation, the knowledge engineers have participant observation, the researcher might hypothesize that the input of their own responses as a participant in forming members of one group exhibited one behavior and those of hypotheses. another, a slightly different behavior in the same situation. Apprenticeship learning or teachback interviews [8] involve Crane and Agrosino [5] argue that participant observation learning from one expert at a time and having the knowledge is not so much a technique as a state of mind.= By this they engineers demonstrate their understanding by solving a prob- mean that one does not do participant observation in the lem. In contrast, participant observation commonly involves sense that one does a kinship chart or opinion survey. Instead, learning from multiple experts’ interactions, some of which participant observation is more a framework for living in the may not be directed toward problem solving so much as field. Direct experience of others’ way of life is viewed by building expertise (e.g., through arguments among experts). many anthropologists as the most effective way of reducing I consider apprenticeship learning or teachback interviews to one’s natural ethnocentric biases. A researcher’s ethnocentrism be a subvariety of participant observation. Indeed, participant arises from being raised in a society that possesses particular observation has been defined as “when the knowledge engineer ethical, social, work, religious, and sanitary values. Ethnocen- becomes an apprentice or otherwise participates in the expert’s trism results from the researcher seeing things in terms of the problem solving” [SI. culture of which he or she is a product and therefore tends to In anthropology, participant observation usually is combined view as superior, or at least acceptable. with more analytical and manipulative methods. For this reason, in knowledge acquisition, I would recommend its use B. Participant Observation as a Way of Thinking in combination with other methods of knowledge acquisition and statistics. However care is needed in combining participant I consider participant observation to be a way of thinking observation with other methods; Shaw and Woodward [9] distinct from that traditionally used in the research associated caution against ad hoc combinations of knowledge acquisition with the physical sciences. Belenky, Clinchy, Goldberger, techniques which may be associated with conflicting assump- and Tarule [6] who have done an epistemological study of tions about the nature of knowledge. (See Section IV for women’s ways of thinking about the world, describe partici- further discussion of the assumptions inherent in participant pant observation as a sophisticated form of connected knowing. observation.) Connected and separate knowing are two general types of thinking. Separate knowing is the way of thinking that has been associated traditionally with scientific thinking, namely empiri- 111. USES OF PARTICIPANT OBSERVATION IN cism. In separate knowing, researchers strive for objectivity KNOWLEDGEACQUISITION by attempting to detach themselves from what is observed so How participant observation is enacted may differ depend- that they avoid projecting their own thoughts and biases into ing on the purposes for which it is being used. For example, if participant observation is simply being used to familiarize ’While I agree with this characterization of participant observation, I refer to it as a technique, mainly to distinguish it from the automated roo1.y of the knowledge engineers with the domain, it will be less knowledge acquisition. intensive than if its purpose is stimulate their formulation MEYER: ANTHROPOLOGICAL TECHNIQUE OF PARTICIPANT OBSERVATION 985

of hypotheses for model development. The different uses of Los Alamos National Laboratory [ll]. This study relates to participant observation are likely to overlap; that is, participant learning how experts solve problems and how expertise is observation done for the purpose of becoming acquainted with developed within an organization. The first use of participant the domain is also likely to provide information on individuals’ observation was to familiarize myself with the domain. I behavior and may allow the knowledge engineer to overhear sought to learn in a concrete way what individuals of these job the participants’ interpretation of some event. categories, scientists and technicians, typically did at work. For instance, how did they receive their work assignments, what reference materials, tools, and resources did they use, and A. To Familiarize the Knowledge Engineer with the Domain with whom they did they work? I spent several days apiece Participant observation can serve to familiarize the knowl- following and observing four employees, two scientists and edge engineer with the domain and with those who are two technicians. being studied. For example, Boose and Shaw [lo] mention The preliminary participant observation revealed that scien- participant observation as something that knowledge engineers tists received project assignments from their managers and/or can do before their first interview with a source expert, and, I created their own, such as by obtaining funding to work on would add, before their first interview with a potential system some research that interested them. Technicians, on the other user. hand, received their assignments from scientists who were How long the familiarization phase will take depends on working on the project to which the technician had been the complexity of the domain and the availability of textual assigned or from their direct supervisors, often high-level knowledge sources. If the domain is difficult to understand and technicians. Both scientists and technicians used textbooks, if there are no textbooks, reports, case studies, or data that the manuals, computers, and calculators in their work. Scientists, knowledge engineer can peruse, this initial participant obser- however, used technicians as resources for building equipment vation will take longer. Typically, participant observation for for running large experiments; technicians used machining and familiarization involves more observation than participation. electrical tools for building or repairing equipment. Scientists Unless the experts’ tasks or the users’ needs are very simple, and technicians worked with a variety of people. For example, knowledge engineers are more likely to observe and ask a few some scientists collaborated with other scientists at different questions to clarify their observations than to try to do the national laboratories or universities. Or, scientists might work task themselves with the assistance of the user or expert. At with technicians within their own organizational group or this stage in knowledge acquisition, the knowledge engineer outside it. is unlikely lo have acquired the concepts or skills to do even a semblance of the experts’ tasks. B. To Obtain Information about the Experts’ or Users’ In my experience, the relatively small amount of time spent Behavior in this preliminary participant observation provides dispropor- tionately large gains in information and understanding. What Participant observation does not involve giving instructions the knowledge engineer is most likely to learn is how the or setting particular tasks; thus, it allows the knowledge experts or users go about their work. For instance, in the engineer access to knowledge without controlling or altering case of the expert, this may include how the expert uses the expert’s/user’s environment or behavior. Participant obser- references and interacts with other experts and the client in vation also allows the knowledge engineer to fill information reaching a solution. In the case of the system user, participant gaps and to correct misinformation that results from relying on observation may reveal how the task comes to the user verbal descriptions of behavior. People are frequently unable through the channels of the organization, how and where to describe their problem-solving behaviors either because the task is executed, and the means by which the finished these behaviors are unconscious or because their knowledge product or solution is passed from the user to its destination. of their problem-solving processes is informal and uncodified Information on how experts/users do their work cannot usually [3].Another explanation is that people cannot describe aspects be obtained from textual references or from interviewing of their behavior if they have been asked for this description Yet, a basic understanding of this process is critical to the out of the context in which the behavior OCCU~S.~ knowledge engineer’s later asking of questions to probe on Proponents of this view (from Conversation Analysis and the expert’s problem solving or the user’s needs. In addition, ethnomethodology) portray knowledge as actions taking place preliminary participant observation will give the knowledge in the “stream of experience envelops the person”; a stream engineer an impression of the subcultures of the experts or of experience which is defined to include “the person’s self, users. Knowing the subculture is an advantage in selecting the things in the environment, and the factors which provide methods for later knowledge acquisition, in formulating the the background against which that person creates meaning role of the knowledge engineers so that they will be understood for him or herself.” [12] In other words, this view portrays by and acceptable to the insider, and in designing the expert system or the user interface. 3Suchman [31] has explained that peoples’ actions take place in the context An Illustration of Participant Observation to Familiarize of particular, concrete circumstance and that these actions are essentially the Researcher with the Domain: Participant observation was ad hoc because the circumstances surrounding them constantly change. She argues that people can create a retrospective rationale for their actions, if used in a study that investigated the work-related attitudes asked to do so, but that this description filters out the contextual details that and behaviors of a group of scientists and technicians at would truly explain their actions. 986 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 5, SEPTEMBER/OCTOBER 1992

“peoples’ knowledge as not being just in their heads but in rapp~rt.~For example, participation would enable the knowl- the environment in which they are emt~edded.”~ edge engineers to put themselves in the user’s shoes to learn When people are able to provide descriptions of their their needs and preferences for the system. As mentioned behavior, the descriptions are often unreliable. Some experts earlier, many anthropologists believe that the most effective provide descriptions that do not match their actual perfor- means for minimizing one’s own ethnocentric bias is partic- mance, particularly in the sequence of actions [13]. Others tend ipation in the insider’s activities. Knowledge engineers, as to give idealized versions of their behaviors either because this human beings, tend to interpret new information in light of is what they think the researchers wish to hear or because they what they already have learned and have come to value. Thus, are more aware of rules and norms than they are of their actual knowledge engineers are prone to misinterpreting and misrep- behavior [ 111. Participant observation enables the knowledge resenting the expert’s verbalizations or behaviors. Participant engineer to obtain information about the expert’s or user’s observation provides a check on the knowledge engineer’s behavior while avoiding these pitfalls. misunderstanding of the experts’ reports of their actions. An Illustration of Participant Observation to Obtain In- Illustrations of Participant Observation to Learn Insiders’ formation about the Zndividuals’ Behavior: In the Laboratory Views: In the lab study, participant observation revealed how study, after the preliminary participant observation, I focused the scientists and technicians viewed themselves and each on the scientists’ and technicians’ interactions, because these other. Scientists, especially those with Ph.D.’s, saw themselves demonstrate the presence of occupational stereotyping. Many as problem solvers, idea people, and thinkers; whereas the interactional patterns, such as who sits with whom, are un- technicians viewed themselves as doers, those who really got conscious; thus, the employees would not have been able the work done. Each group took pride in their contribution to describe their interactional behaviors. For this reason, I and considered it the most important role in the laboratory. focused on observing scientists’ and technicians’ relations as Employees’ comments revealed that scientists viewed heavy characterized by jokes, employee seating arrangements during job involvement as pleasurable and good, and technicians interactions, deferential behavior, and socializing. Scientists’ viewed it as something to be avoided (e.g., technicians said and technicians’ relations were observed in four different they tried to leave their workproblems at work.) In the extreme, organizational groups. To understand the interactions, since scientists viewed technicians as lazy, as more concerned with employees interacted with as many as 50 people in a day, I work perks than the work itself; whereas technicians saw had the individual that I was observing provide a list of the scientists as workaholics and inept in everyday life. people he saw and their job series (scientist, technician, or As another example, participant observation can provide the secretary). knowledge engineer with acceptable terms of address for those Participant observation revealed that behaviors indicative studied. How the knowledge engineer refers to a group of of occupational stereotyping occurred where the tasks of the people is very important because it describes how the people scientists and technicians were the most different [ 111. In the see themselves and/or wish to be seen. Using the wrong terms, organizational groups in which the scientists and technicians especially in the early stages of a study can have severe performed essentially the same job, the stereotyping behaviors repercussions: 1) it can irritate or offend people and negatively were largely absent. In groups where the Ph.D.’s planned and affect their decision to cooperate with the knowledge engineer, oversaw the experiments and the technicians did the manual and 2) it can repeatedly draw attention to the knowledge work of building and installing the experimental components, engineer’s outsider status. For instance, in another project the stereotyping behaviors were more prevalent. that involved obtaining experts’ estimates of the safety of selected components under different environmental conditions, participant observation was not conducted. As a result, I C. To Learn How the Experts or Users View Themselves, made the mistake of referring, orally and in writing, to Their Own, and Others’ Behavior the participants as experts; they complained and, with some Many psychologists, psychiatrists, and practitioners of irritation, asked me to stop doing so. To them, expert connoted knowledge acquisition believe that the drawing of distinctions a breadth of knowledge in a well-researched field. My labeling is at the base of what humans experience and learn. Participant them experts made them fear that their judgments would be observation places the researcher in a convenient position to interpreted as being more firm than these judgments could be, hear the insiders’ descriptions of distinctions that they are given the lack of data and high uncertainties in the phenomena. making (e.g., who they consider to be insiders in various situations or what distinguishes a good problem solution). D. To Investigate the Social Processes by which Expertise Thus, participant observation allows the researcher to learn is Developed and Supported what meaning the insiders’ actions hold for them in the Observing experts in situ offers the knowledge engineer insiders’ own words. In addition, participant observation may deepen the knowl- ’Similarly participant observation seems to foster the insiders’ feelings of rapport with the knowledge engineer. Insiders have told me that the use of edge engineer’s understanding of the insider’s activities and participant observation, which they have likened to walking a mile in their moccasins, made them believe that the knowledge engineer sincerely wished to understand them and their point of view. Hence with this technique, they have been willing to carefully consider the knowledge engineers’ hypotheses 4This is quoted from Gaine’s summary talk at the European Knowledge about their actions (i.e., check it against their intuitive understanding of their Acquisition Workshop 1990, Amsterdam. own actions). MEYER: ANTHROPOLOGICAL TECHNIQUE OF PARTICIPANT OBSERVATION 987 unique insights into the social process of expertise, how were being reached through consensual behavioral means, in expertise evolves through the individual expert’s receiving particular, group think. Group think, also called the follow- of feedback from other experts and clients [14]. Expertise the-leader effect, is a form of social pressure occurring in a is continuously developed through the experts’ exposure to group that induces individuals to slant their responses or to practical problems (through the client) and the data of others silently acquiesce to what is acceptable to other members. (experts) in their professional community. Random differences The observer can watch for several signs of group think in early performance may be amplified as the most interesting (further described in Meyer and Booker [20]). However, problems are given to those with the strongest records of in general, if no difference of opinion is voiced, and the previous successes. Invitations to scientific congresses and experts appear to defer, verbally or behaviorally, to another awards of scholarships may further magnify these effects. The member of the group or to each other, group think is a strong role of experts in society can thus be seen as trading their possibility. The reason that a knowledge engineer would want advice for access to problem solving and, hence, learning to know of this possibility is because group think is considered opportunities. undesirable; decisions that are acquiesced to, rather than To examine a social process, such as the development and arrived at by thorough consideration of diverse viewpoints, support of expertise, requires an observational technique. Ob- are suspect. servations are required because much of the experts’ actions, In the laboratory study, participant observation and other particularly in interaction with other experts or clients, do methods yielded findings on how expertise is developed within not lend themselves to verbal description. The interactions an organization. I had observed that the scientists, particularly are complex and much information is being communicated those with Ph.D.’s seemed more involved in their work and simultaneously by verbal and nonverbal means. Participant to identify with their work to a greater degree than the observation is an excellent technique for this purpose. technicians. For example, during lunch or any social activity Understanding the social process of expertise may be valu- at work, the scientists tended to talk about their latest project able in designing expert or knowledge-based systems. For or paper; whereas the technicians seemed to talk about their instance, Gaines [14] and Chandrasekaran [15] focus on char- sports activities, hobbies, or families. This difference had been acteristics of human expertise and propose that systems be the subject of informal conversation and jokes by the scientists designed to have these traits. To illustrate, Gaines proposes and technicians. that, like experts, knowledge-based systems have the capability These observations stimulated other questions, such as: of updating their knowledge. Chandrakekaran proposes that Why would the scientists identify with their work to a greater the framework of expert systems, the substructures of the degree than the technicians; and Do the scientists and techni- knowledge, be organized to reflect how a community of cians exhibit differing work behaviors, corresponding to their experts uses their knowledge. attitudes? These research questions were investigated using I would add that knowing how experts reach decisions participant observation, intensive interviewing, questionnaires, in the face of dissenting information from other experts is and records research. Their findings allow me to propose also of interest in designing systems. Previously, knowl- the following process by which expertise was developed and edge engineers tended to start working intensively with only maintained in the laboratory. one or two experts. However in the late 1980’s, knowledge The laboratory differentially gave the Ph.D. scientists the engineers were encouraged to begin their elicitations with greatest job rewards: interesting work, work which made diverse experts [lo], [ 161. Additionally, Gaines’ social model good use of their skills, the opportunity to see the results of expertise [14] would predict that systems be based on of their labors, and autonomy. The nature of these rewards “groups of experts rather than individuals.” It follows that caused a positive feedback loop. The autonomy, accorded a major question in knowledge acquisition will be how to the Ph.D.’s allowed them to study the research questions integrate or combine the differences in experts’ problem- which personally interested them, which in turn increased solving processes and judgments in the system. There are the magnitude of the job rewards that they were already two basic approaches for handling multiple and differing receiving (e.g., the work became even more interesting expert knowledge. The first approach is to have the experts because it was what most interested them, and it increasingly reach consensus during the elicitation and enter only this used their skills). Their job satisfaction, involvement, and one piece of expert data in the expert system [17]. The time at work increased, for which they were rewarded second approach is to collect information from multiple ex- with salary increases and greater job rewards, and so perts, to compare that information for areas of agreement on. and disagreement, and then, in some cases, to select the In sum, the organization was differentially developing the most appropriate information for presentation to a user [17], expertise of its Ph.D. scientists, the group which could most [W. enhance its probability of continued existence (cf. [14]). These Illustrations of Participant Observation to Investigate Social scientists, because of the status afforded to Ph.D.’s in this Processes of Expertise: In several studies, such as Meyer society, were more likely to do research, publish, and receive and Johnson [19], participant observation, or simply obser- outside honors. Through their publications and honors, they vation, has been helpful in determining how experts in groups could enhance the reputation of the organization, bring in arrive at their final judgments. For instance, I have used additional prestigious research and funds, and increase the participant observation to learn if the experts’ final judgments organizations’ chances of continued success. 988 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 5, SEPTEMBERiOCTOBER 1992

E. To Aid in the Construction of Personal Models of the mation will allow the reader to assess the validity of participant Events and Objects in the Domain observation for specific applications and for combination with Participation aids knowledge engineers in constructing per- other techniques. sonal models of the objects and events within the domain which, in turn, can be used in building the knowledge base. A. Assumptions of Knowledge Implicit in Participant Clancey (211 has likened knowledge bases to qualitative Observation models. He further defines qualitative models as describing “systems in the world in terms of causal, compositional, or A number of assumptions are implicit to the technique of subtypical relationships among objects and events.” Following participant observation. First however, I consider an assump- this view, then, knowledge engineers are likely to need a vari- tion that is not made. In my opinion, participant observation ety of techniques and tools to assist them in building models. makes few assumptions about how people process information. I propose that participant observation provides the knowledge Shaw and Woodward [9] argue that the implicit assumptions engineer with ideas about relationships between the elements about the nature of knowledge that underlie many knowledge of the domain through an unusual avenue-participation. Par- acquisition techniques influence how the knowledge is repre- ticipation allows the knowledge engineers to receive their own sented. For example, the use of interviewing, which usually responses to situations as data on relationships. From this involves questioning the expert in retrospect, assumes the data, the knowledge engineer can formulate hypotheses, and expert’s answers are based on what the expert actually did or then test them through the use of additional methods before thought. These assumptions, in turn, rest on certain models of including these relationships in the knowledge base. what the expert can recall or can bring into short-term memory. An Illustration of Participant Observation As an Aid in Participant observation is not based on these information Constructing Models: In the lab study, I used my own processing assumptions because when participant observation reactions to the organization to form hypotheses about the is used, it is the environment and not the knowledge engineer employees’ receipt of job rewards, job satisfaction, job that provides the cues or prompts for the expert’s responses. identification, and work behaviors (e.g., amount of time spent A second assumption is that the participants’ actions and at work). My reaction was that I would spend more time interactions are behavioral outputs of their knowledge and, as working and identify with my job, if I received the rewards such, are more reliable indicators of the participants’ mental of the Ph.D. scientists and that I would limit the amount of models than retrospective or concurrent descriptions of their time and energy I devoted to my work, if I were a technician decision making. Using participant observation, the researcher because this role was of lower status and offered fewer rewards can examine differences in participant’s behavior in different than that of the scientists. situations, uncover patterns in their decision making processes, These hypotheses, such as the Ph.D.’s identify to a greater and perhaps discover rules or distinctions, Similarly, a third degree with their work and spend more time at work because assumption is that first-hand information is less prone to of the job rewards that they receive, were further examined by distortion than that relayed through other parties. other methods. For example, I analyzed records of the number A fourth assumption is that by observing and participating of sick days taken by the different groups and found that the in the participants’ everyday life, by sharing the same Ph.D.’s took the least, Master and Bachelor-degreed scientists environment and experiences, the researcher gains the insiders’ took more, and the technicians, the most. Furthermore, the view and a greater understanding of the participants’ thoughts amount of sick leave taken correlated statistically with the (cf. [22]). This fourth assumption is important and merits different groups’ receipt of job rewards and job satisfaction, further discussion. as these were determined from questionnaires. First, it may be argued that the researcher may not understand the participants’ views even though he or she participates in their environment. After all, researchers and insiders can Iv. GUIDANCEON THE USE OF PARTICIPANT OBSERVATION interpret and respond to the experiences very differently as a As knowledge engineers, our conception of knowledge result of their previous experiences and socialization. How- acquisition, the nature of knowledge, and the goal of our ever, as Keeney and Ross [23] note “what one perceives is a applications influence which methods we select as appropriate. consequence of how one participates in perceiving which in Elicitation techniques, such as participant observation, carry turn is a consequence of one’s social context.” Social context their own assumptions and provide information in a particular includes the person’s self, the things in the environment, and form. Shaw and Woodward [9] have drawn attention to the background factors from which the person creates meaning patchwork problem of combining differing knowledge acqui- [12]. Because the environment is at least part of what we sition techniques, typically borrowed from other disciplines. perceive, it would make sense to try to participate in it as Knowledge acquisition techniques make implicit, often con- a aid to understanding the perceptions of those we study. In flicting assumptions about the nature of knowledge and its other words, as researchers or knowledge engineers, we have characteristics. little control over the aspects of ourselves and other factors In light of Shaw and Woodward’s efforts to promote a which would cause us to interpret a situation differently than methodology for the logical selection of knowledge acquisition those we study. Yet, we do have control over our decision to techniques, I describe some of the assumptions that participant participate in their environment and should not discard this observation implicitly makes. I hope that the following infor- approach simply because it does not guarantee that we will MEYER: ANTHROPOLOGICAL TECHNIQUE OF PARTICIPANT OBSERVATION 989 think and feel as the other participants. often with some flourish. I later learned that they saw this token In addition, I believe that the emphasis in participant ob- sum as meaning the difference between accepting charity (a servation on obtaining the insiders’ views tends to constrain negative situation associated with loosing their independence) the researchers from absolute projection of their views onto and paying for their own meal [25].This instance illustrates those of the insiders. Also, the practice of testing hypotheses how small actions on the part of those providing the service against observations or data obtained from more structured (e.g., requesting a voluntary donation of a quarter) can make methods should prevent the knowledge engineer’s views from a critical difference in the users’ acceptance and enjoyment diverging too much from those of the insiders. In the end, of the service. however, I recognize that participant observation leads to an In general, participant observation would be helpful in researcher’s model of the insiders’ environment and thinking applications where portions of the knowledge are informal or and not the insiders’ model. (I am indebted to Brian Woodward viewed by the insiders as being common sense. Insiders are of University of Calgary for voicing this actuality.) However, unable to articulate uncodified knowledge, particularly their I would add that this is all any tool or technique can offer. As heuristics for doing what seems to them to be just common two psychiatrists, Thomas and Patrick Malone [24], note the sense. best we can do is accept another’s reality, and that is the only Participant observation not a good choice if the knowledge real understanding that we can ever have. engineer is likely to have any difficulty obtaining entry into the experts’ society. Entry is more difficult if the domain is classified or proprietary, if the groups are opposed to outsiders, B. Applications for Participant Observation or if the groups do not have a concept of what it is to study or Most knowledge acquisition applications would be suitable conduct research (i.e., anthropologists have frequently entered for using participant observation. However, participant obser- groups by likening their position to that of students).‘ vation works best in applications where the expert interacts with others (e.g., experts, coworkers, or clients) or in some C. Considerations in Using Participant Observation tangible way with the physical environment (e.g., nature or In addition, there are a few other considerations in us- human tools). By contrast, if there is little to observe of the ing participant observation: The level of participation can expert’s work, participant observation would not be suitable. be adjusted to the application. For example, the knowledge For example, if the expert was a theoretical mathematician engineers are likely to observe more than participate, if their sitting alone in her or his office and staring at the ceiling most purpose is to become acquainted with the domain. In some of the day, participant observation would not be a good choice. cases, the knowledge engineer may be precluded from active Participant observation is well suited to applications where participation by the nature of task. For instance, if the experts there are multiple experts and/or forms in which the knowledge are involved in responding to life-threatening emergencies, is encoded (written, verbal, and in artifacts). Participant obser- the knowledge engineer might follow the experts but not vation allows the knowledge engineer to learn how individual physically perform their tasks because this could imperil the experts evaluate and integrate others’ knowledge with their knowledge engineer, the experts, or those to whom the experts own. The technique also reveals how the experts produce the are responding. different forms of knowledge and how these different forms Participant observation can be used to acquire different are combined in the practicalities of their use. granularities of information, from the larger picture to the Participant observation is a good technique for any applica- detailed data on the individual’s behaviors and attitude^.^ For tion that studies how individuals acquire expertise. Similarly, instance, participant observation used to become familiar with participant observation can be used to study the practice of the domain provides information of coarse granularity, while knowledge acquisition in its larger context, particularly some that used to learn individual’s behaviors or views provides of its approaches and problems in relation to those of other data of a finer granularity. Many of the manual techniques for fields. For instance, Forsythe and Buchanan [3] mention their knowledge acquisition are only suited to obtaining one level of on-going study of how “knowledge engineers acquire and granularity. For example, verbal reports, in which the experts order the domain knowledge to be encoded in their systems.” or users are asked to think aloud as they work on some task, Participant observation is an excellent technique for studying user needs. In recent years, it has been used by market 6However, I note that once entry is gained into a group, the participant research companies to learn what consumers want in particular observer is rarely viewed as being intrusive. Indeed, in my experience, products, how they use the products, and how they shop for individuals quickly become accustomed to having participant observers and even miss their presence when this method has been concluded. 1 attribute similar products. The market researcher actually goes into the this common reaction to people enjoying having someone intensely interested consumers’ homes, cupboards, watches the consumer using in them and their activities, someone whose presence gives special meaning the product, and observes the consumer as he or she shops. to everything they do. Participant observation is very helpful in uncovering the needs ’How we are able to change granularities is explained in a study of peoples’ browsing behavior by Kwasnik [12]. She describes the notion of view as and constraints that affect peoples’ use of product, but of which the means by which we are able to change conceptual and physical focus. they are unaware. For instance, in doing participant observa- For instance, a knowledge engineer standing by a group of people may say tion of senior citizens’ use of a federally-funded program to “problem-solving session” and that identifies the knowledge engineer’s view as including the experts and clients in interaction. The knowledge engineers provide them with meals, I observed that the vast majority of then focus on one expert (the view is now the individual) and monitors their attendees made the optional donation of a quarter per meal, particular problem-solving activities. 990 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22. NO. 5, SEPTEMBERiOCTOBER 1992

provide only fine granularity data, such as how the individual R. C. Paton, H. S. Nwana, M. J. R. Shave, and T. 3. M. Bench-Capon, thinks through the problem, their assumptions and heuristics “Toward a classification of approaches to knowledge acquisition: A proposal,” MEKAS report no. 15, Dept. Comput. Sci., Univ. Liverpool, [201, ~261,~71. Liverpool, L69 3BX U. K., Dec. 8, 1990. Participant observation is time consuming and its returns D. E. Forsythe and B. G. Buchanan, “Knowledge acquisition for (e.g., in providing insider insight’s or in stimulating the knowl- expert systems: Some pitfalls and suggestions,” IEEE Trans. Syst., Man, Cybern., vol. 19, pp. 435442, 1989. edge engineer’s formulation of hypotheses) may be difficult to C. P. Kottak, Anthropology: The Exploration ofHuman Diversity. New measure. Forsythe and Buchanan have noted that heavy time York: Random House, 1974. J. G. Crane and M. V. Agrosino, Field Projects in Anthropology. and money constraints have led many knowledge engineers Morristown, NJ: General Learning Press, 1974, p. 63. to spend less rather than more time with their experts [3]. M. F. Belenky, B. M. Clinchy, N. R. Goldberger, and J. M. Tarule, While I generally agree with their assessment, I have noticed Woman’s Ways of Knowing: The Development of Self; Voice, and Mind. New York: Basic Books, 1986, p. 226. a counter trend among system specialists in a few areas, such D. A. Waterman, A Guide to Expert Systems. Reading, MA: Addison- as that of human-computer interface (HCI) and user interface Wesley, 1986, p. 158. design. In these two areas, at least, knowledge engineers, J. H. Boose, “A survey of knowledge acquisition techniques and tools,” Knowledge Acquisition, vol. 1, no. l., pp. 3-37, 1989, p. 12. software engineers, and interface designers are employing time M. L. G. Shaw and J. B. Woodward, “Mental models in the knowledge consuming sociological and anthropological approaches, such acquisition process,” in Proc. 4th Knowledge Acquisition for Knowledge- Based Systems Workshop, Banff, Canada, Oct. 1989, pp. 29-1 to 29-24. as conversation analysis and ethnomethodology [28], [29]. J. H. Boose and M. L. G. Shaw, “Knowledge acquisition for knowledge- I would advise the knowledge engineer to expect to be based systems,” in Proc. AAAI-89 Tutorial Notes of the Eleventh Int. uncomfortable with having a marginal status of not being Conf Artificial Intell., Detroit, MI, Aug. 1989, pp. 6-62. M. A. Meyer, “Jobs, work-related values, and attitudes of staff and wholly a researcher, in the traditional empirical sense, nor an technicians at a national laboratory,” Los Alamos Nat. Lab. rep. LA- actual insider while doing participant observation. Then too, 10497-T, Los Alamos, NM, 1985. trying to emulate tasks at which the expert is skilled, often B. H. Kwasnik, “An analysis by means of naturalistic approaches of two complex behaviors,” in Proc. AAAI Workshop on Complex Systems, makes the knowledge engineer feel temporarily inept. Par- Ethnomethodology and Interaction Analysis, Boston, MA, July, 1990, ticipant observation should generally be used in combination pp. 123-135. with more structured and analytic techniques, as it has been J. Breuker and B. Wielinga, “Use of models in the interpretation of verbal data,” Knowledge Acquisition for Expert Systems: A Practical in anthropology. For instance, in the study of scientists and Handbook, A. L. Kidd, Ed. New York: Plenum, 1987, pp. 1742. technicians, participant observation was supplemented with B. R. Gaines, “Social and cognitive processes in knowledge acquisition,” questionnaires and analyses of the organization’s records. Knowledge Acquisition, vol 1, no. 1, pp. 39-58, 1989. B. Chandrasekaran, “Toward a taxonomy of problem solving types,” AI Mag., vol. 4, no. 1, pp. 9-17, 1983. J. H. Boose and B. R. Gaines, “Knowledge acquisition for knowledge- V. CONCLUSION based systems,” in Proc. AAAI-88 Tutorial Notes Tenth Int. Conf Arti- ficial Intell., St. Paul, MN, Aug. 1988. I have described participation observation to allow others K. L. Mc Graw and K. Harbison-Briggs, Knowledge Acquisition: to use it for manual knowledge acquisition. In particular, Principles and Guidelines. Englewood Cliffs, NJ: Prentice Hall, 1989. B. R. Gaines and M. L. G. Shaw, “Comparing the conceptual systems I believe that participant observation offers the knowledge of experts, in Proc. Eleventh Joint Conf Artificial Intell., Detroit, MI, engineer unique benefits in understanding the users’ needs and Aug. 1989, pp. 633438. subculture, the experts’ domain and in formulating hypotheses M. A. Meyer and E. R. Johnson, “Proceedings from the technology control panel workshop,” Los Alamos Nat. Lab. tech. rep. LA-UR on the relationship of the expert’s actions to situational factors. 854078, Los Alamos, NM, 1985. I have recommended that participant observation be used in Meyer, M. A. and J. M. Booker, Eliciting and Analyzing Expert combination with other more structured techniques, especially Judgment: A Practical Guide, Knowledge Acquisition for Knowledge- Based Systems Series, vol. 5. London: Academic, 1991. if the performance of participant observation has led the W. J. Clancey, “Viewing knowledge bases as qualitative models,” IEEE knowledge engineer to formulate hypotheses about aspects Expert, vol. 4, no. 2, pp. 9-24, 1989. of the participants’ behavior. My aim in this paper was to A. Salmond, “Theoretical landscapes: On cross-cultural conceptions of knowledge,” Semantic Anthropology, ASA Monograph 22, D. Parkin Ed. provide a more explicit, disciplined approach to participant London: Academic, 1982, pp. 6687. observation. To this end, I have described the uses, strengths, B. Keeney and J. Ross, Mind in Therapy: Constructing Systemic Family Therapies. New York: Basic Books, 1985, p. 11. and difficulties of participant observation, the way of thinking T. P. Malone and P. T. Malone, The Art of Intimacy. New York: that it involves, and its implicit assumptions as to the nature Prentice Hall, 1987, p. 116. of knowledge and human information processing. M. A. Meyer, “Factors affecting Albuquerque elderly’s use of the social services,” unpublished Master’s thesis, Dept. Anthropology, Univ. New Mexico, Albuquerque, NM, 1979. J. M. Booker and M. A. Meyer, “Sources and effects of interexpert ACKNOWLEDGMENT correlation: An empirical study,” IEEE Trans. Syst., Man, Cybern., vol. 8, pp. 135-142, 1988. I am indebted to the participants in the studies for their gra- M. A. Meyer, S. M. Mniszewski, and A.T. Peaslee, Jr, “Using three cious cooperation. Thanks also go to the anonymous reviewers minimally-biasing elicitation techniques for knowledge acquisition,” and to John Mott, IS-11, Los Alamos National Laboratory, and Knowledge Acquisition, vol. I,no. 1, pp. 59-71, 1989. P. Luff, N. Gilbert, and D. Frohlich, Computers and Conversation, Debbie Leishman, University of Calgary, for their insightful Computers and People Series. London: Academic, 1990. comments on this paper. N. Gilbert, Ed., in Proc. AAA1 Workshop on Complex Systems, Eth- nomethodology and Interaction Analysis, Boston, MA, July, 1990. K. A. Ericsson and H. A. Simon, Protocol Analysis: Verbal Reports as REFERENCES Data. Boston, MA: MIT Press, 1984. L. A. Suchman, Plans and Situated Actions: The Problem of Human- [1] D. E. Hunter and P. Whitten, Eds., Encyclopedia ofAnthropology. New Machine Communication. Cambridge, MA: Cambridge Univ. Press, York: Harper and Row, 1976, p. 294. 1987, pp. viii-ix. MEYER ANTHROPOLOGICAL TECHNIQUE OF PARTICIPANT OBSERVATION 991

Mary 2. Meyer received the B.A. degree in phi- studying the native peoples of Chile and Argentina to understand the social losophy in 1973 from Harvard University, Cam- aspects of cognition and memory and then spent three years in the United bridge, MA, after being wounded at Kent State States with Rand Corporation developing biochemical models for cognition and spending three years studying Abbie Hoffman and memory. It was during the development of these biochemical models in seclusion. She dropped out of sight after the that her work gravitated toward the problems of artificial intelligence with upheavals of the 1960’s to re-emerge a Buddhist applications toward cognition, voice pattern recognition, and ethnographic having studied in India and Tibet. She received knowledge elicitation. She is currently applying her theoretical work as a the M.A. and the Ph.D. degrees in anthropological Research Staff Scientist in a statistical elicitation research group at the Los genetics and social altruism, in 1977 and 1978 Alamos National Laboratory, Los Alamos, New Mexico. She is completing a respectively, from Stanford University, Stanford, project of computerized knowledge elicitation with voice pattem recognition CA. She received the M.F. degree in social systems so that probability distributions of answers can be developed by analyzing the science and the Ph.D. degree in computation algorithmic mathematics in 1980 experts’ voices while the computer continues to develop new questions. This and 1982, respectively, from Duke University, Durham, NC. will be added to her knowledge updating algorithms that allow computers She has held teaching and research positions in the United States, Asia, to leam from feedback. She is starting a project on human motivational New Guinea, Chile, and Argentina. In 1984, while climbing in Patagonia, objectives which will enable computers to coach humans to excel using she began to concentrate on human cognition receptors and the biochemical voice recognition, knowledge acquisition, knowledge updating algorithms, problems of memory retention and cell communication. She spent two years and motivational feedback theories.