A Case-Based Approach to Diagrammatic Reasoning
Total Page:16
File Type:pdf, Size:1020Kb
From: AAAI Technical Report FS-97-03. Compilation copyright © 1997, AAAI (www.aaai.org). All rights reserved. Using Diagrams to Understand Diagrams: A Case-Based Approach to Diagrammatic Reasoning Dale E. Fish and Robert McCartney Department of ComputerScience and Engineering University of Connecticut Storrs, CT 06269-3155 [email protected] robert @eng2.uconn.edu Abstract diagrammatic reasoning (DR) but little work using case- Weare exploring the possibility of using case-based based reasoning (CBR) as an approach to DR. We think reasoning as an approachto diagrammaticreasoning. This this maybe a natural fit, reasoning with diagrams about paper describes a work in progress on a system that diagrams. This work explores the possible advantages of ’understands’a diagramof a problemsituation by finding using a store of case diagramsto interpret newdiagrams. correspondencesbetween it and similar diagramsin its case This paper describes a workin progress on a system that memory.The input to the system is a diagram of the ’understands’ a diagram of a problemsituation by finding problemsituation consisting of somenumber of simple correspondencesbetween it and similar diagrams in its case elementssuch as circles and lines. Thegoal of the systemis memory. The input to the system is a diagram of the to determine the significance of the diagram and its constituent elementsby recognizingthe structure of the problem situation consisting of some number of simple diagramand structures withinit. Theapproach is to use high elementssuch as circles and lines. The goal of the systemis level perception(recognition of instancesof categoriesand to determine the significance of the diagram and its relations) to build a representationof the problemsituation constituent elements by recognizing the structure of the (a diagram), and infer additional informationby finding diagram and structures within it. The approach is to use correspondencesbetween this representation and stored high level perception (recognition of instances of cases. Theoutput is the problemsituation diagramannotated categories and relations) to build a representation of the with descriptions of the elementsand relationships between problem situation (a diagram), and infer additional them and any inferences the system maybe able to make information by finding correspondences between this alongwith their justifications. representation and stored cases. The output is the problem situation diagram (or just problemdiagram) annotated with descriptions of the elements and relationships between Introduction them and any inferences the system may be able to make There are a numberof problem domains where information alongwith their justifications. is more easily and naturally represented using diagrams. In the next section, we state the problem we are While humans are adept at using diagrams, automated addressing with this workand follow that with a discussion reasoning systems typically use pre-distilled codified of the approach we are taking. The following section representations. This means that in domains where elaborates on the approach by providing an example. We diagrammatic representations are the norm, the diagrams then discuss someother work related to our approach, some must first be interpreted by humans, which (in extreme problems to be addressed and possible extensions to our cases) results in either simplified representations where work, and we conclude with a brief discussion of the determinations of relevancy and saliency have already been contribution we believe this workprovides. made, or representations that (attempt to) include all the information present in the original diagrams. The former reduces the system’s reasoning to a form of unification and Problem Statement the latter can be very difficult if not impossible given the Weare interested in looking at the utility of using diagrams density and complexity of information inherent even in we knowand understand to reason about new diagrams. In simple diagrams. An automated system that reasons general terms, the question we are dealing with is can directly from diagrams both precludes the necessity of diagrams be useful in understanding other diagrams. For spoon feeding the system interpreted representations and humansworking in domains where information is routinely retains the original information-rich diagrams. Using the represented diagrammatically, the answer seems to be an actual diagrams in reasoning is especially important when obvious yes. Architects represent plans with diagrams and there is limited domainknowledge (i.e. the first principles understand these representations because of their of a domainare unknown)or flexibility is required, both of experience in working with them. The military uses whichare characteristics of learning systems. diagrams to represent tactical scenarios, sports teams use There has been considerable interest in the area of diagrams to represent plays, and directors and actors use 6 diagrams to represent sets and stage movement. The terms of offensive plays with whichit is familiar. Figure 2 usefulness of diagrammatic representations in these and shows a diagram of a problem situation as it would be other domains is the efficiency with which they convey presented to the system. The diagramconsists of 11 circles information, and the efficiency with which the people representing wherethe players on offense line up relative involved understand the diagrams is due in large part to to one another for the given play. their experience in workingwith them. We would like to develop a CBRsystem that uses diagrams of situations as cases and reasons from these cases about new diagrammatically represented problem situations. Weare assumingthat a problemsituation can be represented using some number of simple pictorial elements such as basic geometric figures, and that the identity of these elements along with their location and orientation is given, sidestepping the issue of low level recognition. (Whatthis really meansis that the system is ! given instructions on howto draw the problemsituation as opposed to giving it a bit map.) What these elements represent at some basic level may also be knowngiven somedomain, but the roles the individual elements play in a particular diagram must be inferred by recognizing the structures and relationships inherent in the diagram. This seems to be a difficult problem. Take a simple relationship like grouping: if an element can be in at most Figure 1: Sample play diagram showing player alignment one group, the numberof possible groupings is given by and routes of receivers. the size of the powerset of the elements. Evenwith a small number of elements, a small number of relationships between elements and/or groups, and a small number of possible categories to assign to the elements, the problem space gets large in a hurry. Onerole of the cases then is to focus attention, sort of like saying look for somethinglike this around here. The structures discovered in the problem O O diagram may be mappedto the structures in the case or O cases that prompted their discovery along with the corresponding elements, allowing inferences to be formed 000 00 0 about the elementsin the problemdiagram (i.e. if this maps ! to that, thenthis is like that). Figure 2: A problemsituation. The domain we have chosen is the game of football which is a domain where diagrams are routinely used to Whatwe would like to do is take a diagram such as this represent plans called plays. These play diagrams showthe and makeuseful inferences to achieve somegoal. The goal initial locations of the players and their assignmentsduring maybe something as nebulous as ’understand the picture’ the play. Figure 1 shows an example of a play diagram. or somethingas specific as ’identify the primary receiver’. This domain can be quite complicated although the For now, our concern is to be able to makegood judgments diagrams themselves are very simple, using a small set of about similarity between the problem situation and cases symbols, mostly easily recognizable geometric shapes. The and find reasonable correspondences, or mappings, simplicity of these diagrams makes it a good domain for between the elements of two similar diagrams. Webelieve initial investigation (since we are more interested in that this is useful since missing information about an conceptual analysis of a diagram rather than low level element in the problem diagram maybe inferred from the recognition of complexelements within a diagram) while information knownabout its corresponding element in a the complexity inherent in the domain should provide similar case. The desired output of the system is the fertile ground for interesting extensions. Weare of course problem diagram annotated with labels identifying the very interested in provingclaims of generality by using this types of the player elements and the discovered structures approach on other domains. and relationships; the mappings are included as For the present we are ignoring the action part of the justifications for the inferences madeand to indicate which diagrams (the lines showing player movementduring the groups in the problemdiagram are inferred to share roles play) and concentrating on the initial positioning, or with which groups in the case diagram. In other words, the alignment, of the players. This is the problemsituation the system’sgoal is to flesh out the diagram. system is to understand. The perspective is that of the defense trying to recognize a novel