Visual Interaction: A LinkLink Between Perception and Problem Solving

Erika Rogers and Ronald C. ArkinArkin

Abdmcl-TheAbstract-The approach taken in this research is to de­de- is to consider how to use automation to enhance the ef­ef- velop a cognitive model of how a human observer extracts fectiveness of the human performing a complex task, to information from a visual display and then u~eeuses this per-per­ build "tools“tools instead of prostheses”prostheses" [3,5]. HoltzmanHoltsman [6] ceptual information in a decision-making task. Knowledge states that "effective“effective decision systems must concentrate about this relationship provides information about the oc­oc- on assisting the decision-maker to gain insight into the currence of perceptual events in the course of problem-problem­ decision problem at hand rather than on merely supply­supply- solving activities, and suggests that perceptualperceptud assistance ing a somehow 'right'‘right’ answer".answer”. It is also stressed that the in the form of image enhancements is a useful supplement way in which this is to be achieved must involve an un-un­ to the user'suser’s own abilities. This knowledge is then to be derstanding of the cognitive abilities of the human user in embedded in an intelligent computerized assistant which order to implement a "cognitive“cognitivecooperation” cooperation" [3] between isie designed to facilitate and stimulateI/timulate the human problem-problem­ the human and the computer system. With respect to solving process. these stated philosophies, the research presented in this paper is in complete agreement. I. INTRODUCTION However, in practice, the implementations of these ideas have concentrated primarily on the development of With the advent of powerful new technologies for dis­dis- expert-like systems, both in medicine and industry. Such playing multi-dimensional scientific data, the develop­develop- systems usually achieve their cognitive plausibility by ex­ex- ment of new strategies for efficient use of these capabili­capabili- tracting domain-specific knowledge from one or more ex­ex- ties is offoremost concern. It is becoming more and more perts in an area, and then reformulating this knowledge evident that in order to perform complex tasks, man and in a rule-based format. There are three main drawbacks machine can no longer be treated as separate entities, but of such systems: 1) the knowledge extracted tends to be must be considered together as a unified decision-making shallow; 2) experts can also be fallible, and therefore a system. The powerful computational resources of com­com- range of experience would be more desirable for our mod-mod­ puter technology must be coupled with powerful human elling effort; and 3) expert systems tend to rely on verbal perceptual and problem-solving capabilities in order to representations of knowledge, and few are designed to ac­ac- achieve this goal. commodate "image"“image” or "spatial“spatial reasoning”reasoning" that is needed Efforts to apply artificial intelligence techniques in pur-pur­ for task domains where the decision-making process must suit of this goal have resulted in a number of different rely on interpretation of a visual image. Therefore, our approaches. One direction has been the development of approach is to first develop a detailed cognitive model automated reasoning systems, where conceptually, the of the human capabilities in our area of interest. This "faulty"“faulty” aspects of human decision-making are replaced model is then used to predict the type of assistance that with more mathematically precise components. At one will be most useful to the user at various stages of the time, it was thought that such systems might one day re­re- problem solving or decision-making process. place humans altogether. However, more recently, there Systems which do address image reasoning issues may has been a shift in emphasis, especially in light of the be found primarily in domains that contain sensor­sensor- fact that these autonomous intelligent systems have not derived data. Examples of such systems include AXON - a really been as successful as hoped in complex task do­ AXON really been as successful as hoped in complex task do- computer-based intelligent assistant for retrieval ofradio­of radio- mains. Rubin et ala1 [17] note that "knowledge-based“knowledge-based com­com- graphic studies [2], Intelligent Atlas - an expert system puter control systems are unable to effectively handle - puter control systems are unable to effectively handle for neuroanatomy diagnosis, which presents the inference degraded information or novel situations with ambigu­ambigu- process with both verbal expressions and image presen-presen­ ous conditions in which previous guidance of an explicit tations from a pictorial database [12], and research by nature is not available".available”. The more contemporary trend Kraiss on an intelligent dual-screen workstation for fea­fea- ture extraction and interpretation of sonar data [7]. It is interesting to note that although these systems present perceptual material in the form of images, and perhaps Task even allow perceptual enhancements in the form of im­im- Analysis age processing menu choices, the primary "intelligence"“intelligence” still focuses on reasoning capabilities. The perceptual capabilities of the user have not been addressed, particu­particu- larly with respect to effects on performance. The choices of image enhancements are left up to the user, and yet there is no guarantee that improvement in appearance will lead to improvement in perfomance.performance. Our own apap­ //======'0;:;: proach to this problem is best expressed in the following (JENCODING SCHEME)) working hypotheses: 1. The application of cognitive science methods leads to anan understanding of the close interaction be-be­ Full Protocol tween the user’suser's perceptual processes and his prob-prob­ Analysis lem solving capabilities in domains where computer­computer- displayed images form an integral part of the prob-prob­ lem solving process. Cognitive Descriptive \\ 2. The resultant cognitive models can be embedded Elements Elements Elements ) in interactive, cooperative computer systems, which are designed to provide intelligent assistance to the human agent. Such intelligence consists of knowing Contextual what type of assistance is needed, and when it may Analysis be cognitivelycognitivelyeffective effective to afford it. 3. By providing appropriate image enhancements at

key nodes in the problem-solving process, the hu-hu­ Patterns Patterns Patterns man user’suser's own perceptual abilities are enhanced, and overall performance in the task is improved. 4 The emphasis of this paper is on the exploration of the Cognitive Model first of these hypotheses and presents the components of of the cognitive model which has been developed through LVisual Interaction experiment and analysis. 11.II. PROJECT BACKGROUND Figure 1: Data Analysis Methodology Our preliminary work [15] has introduced the concept of Visual Interaction as the process which links percep-percep­ The collection of both observational data and concur-concur­ tion and problem solving such that problem solving is af-af­ rent protocol data was organized in order to develop a fected by what is seen, and conversely, what is seen and thorough understanding of the task environment, and the perceived is affected by the current state of the prob-prob­ task requirements. Details of the experimental data col-col­ lem solving process. Furthermore, the early phases of lection and preliminary results are reported in [15,16]. development of a cognitively-based model of the visual An overview of the Data Analysis Methodology devel-devel­ interaction process have been described for the domain oped to extract perceptual and problem solving concepts of diagnostic radiology. This development domain was from the verbal protocols is presented in Figure 1. The chosen because the specialists are trained to extract in-in­ details of the three main stages, namely, Task Analysis, formation from data that already has a known visual Full Protocol Analysis, and Contextual Analysis are pre-pre­ representation. That is, the numerical values produced sented in [15]. In this paper, we present more concrete by the various imaging technologies are presented in the results from the Contextual Analysis stage, which have form of images of human anatomy which are particularly had a direct impact on our modelling process. meaningful to medically-trained personnel. Investigation of visual interaction in the context of such “visual"visual famil-famil­ 111.III. RESULTS OF ANALYSIS iarity”iarity" facilitates our task considerably. Furthermore, by studying an environment in which the natural layout en-en­ By partitioning the protocol statements according to tails a static stimulus (i.e.,(Le., the image) and a relatively task-related categories, and by maintaining the tempo-tempo­ stationary observer, we constrain our problem, while still ral order of these statements, we see clusters of activity obtaining realistic data. That is, we circumvent the ac-ac­ which are common to the majority of subjects for each cusation, endemic to much AI research, of creating com-com­ case. Many of these clusters can be classified as percep-percep­ pletely artificial laboratory conditions that have no con-con­ tual or problem solving events, and they provide clues to nection with the real world. thethe understanding of the visual interaction process. The firstfirst typetype of visual interactioninteraction behavior thatthat we I. Modifies 1. Testsfor noticed isis what we have termedtermed “immediate"immediate visual cap-cap­ Solution ture”.ture". That is,is, certain typestypes of findings inin thethe imageimage (e.g.,(e.g., lunglung mass, hilar adenopathy) seem toto attract immediateimmediate attention, and, for most of thethe subjects, are mentioned as soon as thethe verbal reporting begins. These findings as as 5. Directs are thenthen described with respect toto features such as size, (De/ailttl! shape, location, texture,texture, and edges. The findings them-them­ selves vary according toto different levels of specificity, and thethe amount of description associated with each level also varies. These kinds of perceptual events occur very early Plans 3. Directs in certain cases, and thenthen trigger a set of one or more 6. Samples (High Level) candidate diagnosis hypotheses. A different type of behavior expressed by the major­ A different type of behavior expressed by the major- Figure 2: Perception-Problem Solving Interaction ity of subjects in virtually all of the cases is called “de-"de­ liberate landmark search”.search". Here the subjects examine landmarks in the chest, and classify them according toto 1.1. Important descriptive features are linked to findings whether they appear “normal”"normal" or not. The classification in the image. This implies that guiding the user to is fairly rapid, and appears to be almost a “check-list”"check-list" look at these features will be a useful enhancement. type of activity. Although the order itself varies from individual to individual, the main anatomical landmarks 2.2. Findings in the image are linked to a set of candidate mentioned are common to most of the subjects. If there diagnostic hypotheses. This suggests that maintain-maintain­ is immediate visual capture of an abnormality, then the ing and displaying hypothesis information will be deliberate landmark search takes place later in the pro-pro­ helpful to the user by relieving some of the cognitive tocol. It is sometimes interrupted by changes in focus loading on short term memory. of attention, and then resumed at a later stage. If the 3. Hypotheses areare linked to particular kinds of evi­evi- case presents a normal chest, or one in which there is dence. Therefore it will be useful to let some expec­expec- no immediately evident abnormality, then the subjects tation guide the search for further perceptual evi-evi­ usually begin with the deliberate landmark search, and dence. continue until either an abnormality is detected, or they 4. Feature combinations and relevant secondary find-find­ are satisfied that there is none. ings can override incorrect hypotheses. This sug-sug­ Further results show that often secondary evidence gests that providing enhancements of the features and/or case history information is needed to disam-disam­ of image findings and anatomical landmarks should biguate between diagnostic hypotheses. Sometimes in-in­ improve the user’suser's assessment of these features and correct anatomical localization (e.g., mediastinal tumor therefore lead to improved diagnostic performance. vs. lung tumor) can lead to an incorrect diagnostic hy-hy­ 5. Goal-oriented behavior occurs throughout the diag- pothesis. However, there is also evidence that sometimes 5. Goal-oriented behavior occurs throughout the diag­ nostic process. Therefore plan-like structures will be perceptual evidence in the form of combinations of criti­criti- useful to guide the diagnostic strategies. cal features (e.g., size and shape) can override the incor­incor- useful to guide the diagnostic strategies. rect diagnostic hypothesis, and lead to a correct one. IV. DESCRIPTION OF MODEL The particular patterns of activity found in our data, (such as deliberate landmark search, anatomicallocaliza­anatomical localiza- In the course of normal human activities, it can be tion, gathering of secondary evidence, etc.), imply a cer­cer- shown that there must be a relationship between per­per- tain amount of goal-oriented behavior. The use of such ception and problem solving such that perception "deliv­“deliv- plans may be driven by a strategy of collecting general ers"ers” information about the environment to the problem feature information about a finding, without any com­com- solving process, and, conversely, the problem solving pro­pro- mitment to a particular hypothesis, or it may reflect a cess communicates "directions"“directions” to the perceptual process strategy of collecting particular evidence for a particular (e.g., I need this type of information rather than that hypothesis. Furthermore these strategies may vary de­de- type). Moreover, we are interested in the class of prob­prob- pending on the experience of the subject, and the nature lems that requires extensive interaction between percep­percep- of the case under consideration. ItIt is also possible for a tion and problem solving, where the problem input is in particular subject to use a number of strategies during visual format and the task is to interpret this input in a the course of a single case. This implies that, although meaningful way. there may be a plan-like structure, it must be flexible Based on current models of perception (e.g., [1,9]) and enough to allow changes in strategy to occur. problem solving (e.g., [8,11]), the mechanisms for such These results have a number of important implications two-way communication are already potentially in place, for the cognitive model and its subsequent effects on the and are conceptually illustrated in Figure 2. Constraints computerized assistant. To summarize: placed on the components labelled Model and Plans are critical to our understanding of this interaction. For ex-ex­ ample, the Model should be able to accommodate knowl-knowl­ Mental edge from both sides: visual information delivered by thethe Model perceptual process (e.g., percepts that describe findings in the image), and decision-related knowledge based on Perceptual ProblemProblem the current state of the problem solving process (e.g., Buffer SolvingSolving what hypotheses are active, what kinds of information do ’iBufferBuffer they need for evidence, etc.). Therefore we need a way Visual Percepts -Interaction ..-) HypotheSERepR \ Problem to reconcile and combine these different types of infor-infor­ Perceptual / Process Soh ing Process mation in the Model. In addition, there should be a way A \ to account for different levels of Plans. For example, a Dctadcd fi@ Plan Level plan to pursue hypothesis-directed search vs. data-driven P13n search is at a different level of abstraction than the de-de­ tailed plan for gathering the specific perceptual evidence required by a particular hypothesis. Therefore we need Long Term Memory a mechanism that coordinates these different levels, and Y ensures that the plans are executed, modified or aban-aban­ doned according to both current perceptual information, and the current state of the decision-making process. We therefore hypothesize that there is a mediating process, which we call the Visual Interaction Pro-Pro­ Figure 3: Detailed Model of Visual InteractionInteraction cess, which oversees the transfer of information and in-in­ structions between the perceptual cycle and the problem solver (which will henceforth be called the Perceptual Long Term Memory also contains some “Mental"Mental Mod-Mod­ Process and the Problem Solving Process, respec-respec­ els”els" used in problem solving, or at leastleast thethe components tively). This mediating process is responsible for main-main­ required to construct these mental models. Pate1Patel et a1al taining a MMentalental Model depending on current information use the following expression: “...when"...when a doctor isis pre-pre­ fromfrom both of the other two processes, and also for man-man­ sented with a problem that he or she isis familiarfamiliar with, aging the transition from high-level plans to detailed low-low­ an appropriate prototype would be invoked toto suggest levellevel plans. In addition to the primary processes, we also possible findings [13]”.[13]". The threethree primary processes of take into account the need for both Long Term Memory our model access this knowledge at various timestimes inin bothboth and Working Memory as parts of the cognitive architec-architec­ bottom-up and topdowntop-down fashion. This isis inin keeping with ture, and describe the components of our model in the both our own observations, and thosethose of researchersresearchers suchsuch following sections. as Reiman and Chi [14]. A. Long Term Memory B. Working Memory In addition to general knowledge about the world, the In addition to general knowledge about the world, the On the other hand, Working Memory isis characterized Long Term Memory contains domain-specific knowledge as a mental workspace that is easily accessible, holds cur-cur­ thatthat is relevant to the problem-solving task, and persists rent information about the task (in particular, thethe “pro-"pro­ over longlong periods of time (e.g., the career of an active totype”totype" or “Mental"Mental Model”Model" acquired from Long Term radiologist). The data which we have collected indicates Memory), and is quickly updated as more or new infor-infor­ thatthat this knowledge falls into a number of categories, mation becomes available. As with Long Term Memory, which includeinclude the following: the Working Memory of our model isis also accessed by all three basic processes, and is divided intointo a number • Different levels of findings (e.g., object findings of components which include Percepts, Mental Model, such as "masses",“masses”, landmark-related findings such as Hypothesis Report, High-Level Plan and Detailed Plan. "adenopathy"“adenopathy” , etc.);etc.) ; The entire model is illustrated in Figure 3. In describ-describ­ • Features associatedassociated with the findings (e.g., size, ing the requirements for a cognitive architecture, Newel1Newell shape,shape, texture);texture); et ala1 [10][lo] discuss the need for “interfaces"interfaces that connect the sensory and motor devices to the symbol system”.system". • Solution hypotheses (complete or partial, e.g., dif­dif- They state that “the"the external world and internalinternal sym-sym­ ferential disease diagnoses); bolic world proceed asynchronously and thereforetherefore therethere must be a buflering of information between the two in • Previous theoretical knowledge (e.g., medical train-train­ must be a buffering of information between the two in ing in anatomy and signs/symptoms of disease); both directions”.directions". The structures in our Working Mem-Mem­ ory can be grouped together toto reflect theirtheir relationshiprelationship • Previous experiences (e.g., previouslypreviously seen x-ray to the three primary processes. The Perceptual Process cases). and the Visual Interaction Process communicate via thethe Perceptual Buffer, and likewise, the Visual Interac­Interac- current candidate hypotheses. It is thethe task of thethe Visual tiontion Process and the Problem Solving Process commu­commu- Interaction Process to initialize and update thisthis Mental nicate via thethe Problem Solving Buffer. Each Buffer Model based on the Percepts delivered by thethe Perceptual contains both "data"“data” components (i.e., Percepts and Hy­Hy- Process, and also on information dictated by thethe Problem pothesis Report, respectively) and "control"“control” components Solving Process. (i.e.,(i.e., Detailed Plan and High Level Plan, respectively). The followingfollowing is a description ofeach ofthese components E. Hypothesis Report togethertogether with that of the Mental Model. The process of making a decision and completing thethe C. Percepts task frequently involves the evaluation of thethe current sta-sta­ tus of the candidate hypotheses. Frequently thethe subjects We recall from our data the evidence for "immediate“immediate generate a list of candidate hypotheses, and rankrank themthem visual capture".capture”. This early information may be used as as at various times in the problem solving task.task. Sometimes an initial cue to obtain relevant domain-specific infor­ an initial cue to obtain relevant domain-specific infor- an initial ranking is expressed, which might later change mation from Long Term Memory. This is supported by an initial ranking is expressed, which might later change mation from Long Term Memory. This is supported by according to the evidence gathered. At some point thethe findingsfindings in the literature as well. For example, Reiman as subjects decide whether they have acquired enough in-in­ and Chi mention the use ofsuch a cue to "trigger“trigger a partic-partic­ formation to make a final decision. This decision may ular [problem][problem] schema"schema” [14], and, more generally, Newel1Newell be of the form of a definite, single diagnosis, a rankedranked et ala1 [10][lo] point out the need for access to distal mem-mem­ list of differential diagnoses, an unranked listlist of differ-differ­ ory structures. Later, as more information is added, or ential diagnoses, or simply a list of findings. This typetype different objects (such as findings or landmarks) capture as of reasoning does not seem to involve all thethe informa-informa­ attention, there is also a need for a structure which allows tion currently gathered in the Mental Model, but ratherrather transfertransfer of this information from the Perceptual Process appears to focus on the hypothesis-related components. toto thethe Visual Interaction Process. We say that this struc­struc- This suggests that from the above Mental Model isis ex-ex­ tureture consists ofone or more Percepts which constitute the tracted a substructure that contains a summary or re-re­ output of perceptual processing, and can be described port of the current status of the candidate hypotheses. as labelledlabelled objects with some number of features (e.g., as This report reflects the strength of thethe evidence avail-avail­ size, shape) attached. The purpose of this structure is able at that stage, thus allowing thethe Problem Solving toto supply the perceptual cue(s) used by the Visual In­In- Process to produce a ranking, and toto make a decision teractionteraction Process to initialize and update the current in-in­ about whether to continue seeking further evidence, or formationformation thatthat is maintained in the Mental Model during to stop. We therefore call this the Hypothesis Report, thethe problem-solving task. which is delivered by the Visual Interaction Process toto D. Mental Model the Problem Solving Process. The next observation that we make from the data F. High Level Plan is that during the problem solving task, the types of knowledge available to the subjects include: a candidate The Problem Solving Process, having received thethe Hy-Hy­ listlist of descriptive features associated with the finding, pothesis Report from the Visual Interaction Process, candidate anatomical locations, candidate diagnostic hy­hy- must evaluate this information and determine whether potheses, candidate secondary findings, and remaining there is enough to make a commitment toto a decision. anatomical landmarks.landmarks. Descriptive features (or combi­combi- If not, then it must communicate directions toto acquire nations of features) and anatomical locations are utilized whatever further information may be necessary toto get as evidence for particular diagnostic hypotheses. On the closer to the goal. The Problem Solver thereforetherefore formu-formu­ other hand, sometimes the hypotheses are directly used lates a High-Level Plan that reflects thethe best strategy toto control thethe acquisition of particular feature/finding in­in- needed. For example, if the hypotheses can be ranked,ranked, formation.formation. This implies a tight coupling between the per-per­ then the plan may indicate that evidence for thethe first-first­ ceptual type of information and the candidate hypothe-hypothe­ ranked hypothesis should be acquired first, followed by ses. Our data shows that both bottom-up and top-downtopdown that for the second, etc. On the other hand, ifif thethe hy-hy­ processing occur at various times in the subjects'subjects’ pro-pro­ potheses cannot be ranked due to insufficient informa-informa­ tocols,tocols, and furthermore, that precedence can alternate tion, the plan may indicate that more features of thethe (e.g.,(e.g., sometimes combinations of features can override primary finding should be collected, and may order some incorrectincorrect hypotheses). This is consistent with the results of these features according to their ability toto distinguish of Reiman and Chi [14] and PatelPate1 et ala1 [13]. between hypotheses. It is also possible thatthat thethe Problem Therefore, the Mental Model maintained in working Solving Process may “discover”"discover" a new hypothesis (or(or set memory contains a combination of relevant finding and of hypotheses) that it wants to explore. This new require-require­ hypothesis information,information, and reflects the relationship be-be­ ment is reflected in the plan, togethertog.ether with instructions tweentween current finding/feature information, and the ev-ev­ on whether to completely abandon the old hypotheses, idenceidence obtained and/or needed to distinguish between or perhaps to maintain them in a lower priority. G.G. Detailed Plan [3][3] Hoc, J.M., "Cognitive“Cognitive approaches to process control".control”. In G. Tiberghien (Ed.), Advances in Cognitivecognitive Science, When the Visual Interaction Process receives the High­High- Vol. 2.-2: Theory and Applications. Chichester: Horwood, Level Plan from the Problem Solving Process, it first uses 1989. the plan to reorganize the Mental Model to reflect the [4][4] Hochberg, J., "On”On cognition in perception: Perceptual current priorities - either rankings of hypotheses, or fea­fea- coupling and unconscious inference",inference”, Cognition 10,1981, tures, or perhaps the addition or deletion of hypotheses. pp. 127-134. Then, in order to meet the goals of the High-Level Plan, [5][5] Hollnagel, E., Mancini, G. and Woods, D.D. (Eds.) Intel-Intel­ detailed instructions must be passed on to the Percep­Percep- ligent Decision Support in Process Environments. Berlin: tual Process. This is done in the form of a Detailed Plan, Springer-Verlag, 1985. which is formulated by the Visual Interaction Process using the information in the Mental Model. This plan [6][6] Holtzman, S. Intelligent Decision Systems. Reading, contains goals related to particular features of a finding MA: Addison-Wesley, 1989. (e.g., size, shape, texture, etc.), different types of find­find- [7][7] Kraiss, K.-F., "Knowledge-based“Knowledge-based classification with in­in- ings (e.g., fluid, collapse, etc.) or different landmarks teractive graphics".graphics”. In E. Hollnagel, G. Mancini and (e.g., heart, bones, etc.), and contains directions such as D.D. Woods (Eds.), Intelligent Decision Support in Pro-Pro­ "LooK-AT(lungs)",“LOOK-AT(lungs)”, "LooK-FoR(fluid)", “LOOK-FOR(fluid)”, etc. This is in cess Environments. Berlin: Springer-Verlag, 1985. keeping with the earlier description of the perceptual cy­cy- [8][8] Mayer, R.E., "Human“Human non nonadversaryadversary problem solving".solving”. cle, which uses the model to formulate plans for action, In K.J. Gilhooly (Ed.), Human and Machine Problem which then interact with the outside world. Solving New York: Plenum Press, 1989. [9][9] Neisser, U. Cognition and Reality. San Fransisco: W.H. VI. CONCLUSION Freeman, 1976. The approach described in this paper has a number of [10][IO] Newell, A., Rosenbloom, P.S. and Laird, J.E., "Symbolic“Symbolic advantages. It provides a methodology for developing a architectures for cognition".cognition”. In M.J. Posner (Ed.), Foun-Foun­ cognitive model of visual interaction, and then utilizes dations of Cognitive Science. Cambridge, MA: The MIT this knowledge to provide perceptual assistance in the Press, 1989. course of the problem-solving process. Unlike automatic [11][ll] Newell, A. and Simon, H.A. Human Problem Solving. detection or automatic decision-making systems, the as­as- Englewood Cliffs, NJ: Prentice-Hall, 1972. sistance is supplied at the interface between the two poles [12][12] Ohe, K. and Kaihara, S., "Intelligent“Intelligent Atlas: A method of perception and problem solving. The user must be of perception and problem solving. The user must to support physicians'physicians’ spatial reasoning".reasoning’!. In MEDINFO guided through the process of assessing stimulus input, 89, Proceedings of the 6th Conference on Medical Infor­Infor- extracting relevant information, making a decision about matics. Amsterdam: North-Holland, 1989, pp. 175-179. the quality of that information, and perhaps returning again to the stimulus to obtain more evidence for the [13] Patel, V.L., Evans, D.A. and Kaufman, D.R., "Biomed­“Biomed- ical knowledge and clinical reasoning", In D.A. Evans problem-solving process. Furthermore, this enables us ical knowledge and clinical reasoning”, In D.A. Evans and V.L. Patel (Eds.), Cognitive Science in Medicine. to design a computer system which utilizes knowledge Cambridge, MA: The MIT Press, 1989. of how image processing affects perception of images to strategically implement physical image enhancements at [14] Reiman, P. and Chi, M., "Human“Human expertise".expertise”. In K.J. key nodes in the user'suser’s problem-solving process. This Gilhooly (Ed.), Human and Machine};lachine Problem Solving latter work is currently in progress. New York: Plenum Press, 1989. [15] Rogers, E., Arkin, R.C., and Baron, M., "Visual“Visual in­in- ACKNOWLEDGMENTS teraction in diagnostic radiology".radiology”. In Computer-Based The authors would like to thank the following for their assis­assis- Medical Systems, Proceedings of the 4th Annual IEEE tance to this project: Dr. Murray Baron and Dr. Ernest Gar­Gar- Symposium. Los Alamitos, CA: IEEElEEE Computer Society cia, Dept of Radiology, Emory University Hospital; Dr. Nor-Nor­ Press, 1991, pp. 170-177. berto Ezquerra, Office of Interdisciplinary Programs, Georgia [16J[16; Rogers, E., Arkin, R.C., Baron, M., Ezquerra, N., and Tech; Dr. John Pani, Dept. of Psychology, Emory Univer-Univer­ Garcia, E., "Visual“Visual protocol collection for the enhance­enhance- sity. ment of the radiological diagnostic process’!.process". In Proceed-Proceed­ ings of the First Conference on Visualization in Biomed-Biomed­ REFERENCES ical Computing. Los Alamitos, CA: IEEE Computer So­So- ciety Press, 1990, pp. 208-215. [17] Rubin, K.S., Jones, P.M. and Mitchell, C.M., [1] Arkin, R., "The‘The impact of cybernetics on the design of "OFMspert:“OFMspert: Inference of operator intentions in super­super- a mobile robot system",system”, IEEE Transactions on Systems, visory control using a blackboard architecture",architecture”, IEEE Man, and Cybernetics 20, No.No.6, 6, 1990, pp. 1245-1257. Transactions on Systems, Man, and Cybernetics 18, No. [2] Cohn, A.I., Miller, P.L., Fisher, P.R., Mutalik, P.G., 4, 1988, pp. 618-636. and Swett, H.A., "Knowledge-based“Knowledge-based radiologic image re-re­ trieval using axes of clinical relevance",relevance”, Computers and Biomedical ReeearchResearch 29,23, 1990, pp. 199-221.