sign in sign up
<<
Home , Abstraction, Abstract structure

A Grounded of in Artificial Author(s): Jean-Daniel Zucker Source: Philosophical Transactions: Biological , Vol. 358, No. 1435, The Abstraction Paths: From Experience to (Jul. 29, 2003), pp. 1293-1309 Published by: The Royal Society Stable URL: http://www.jstor.org/stable/3558222 Accessed: 09/04/2009 01:23

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=rsl.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission.

JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that promotes the discovery and use of these resources. For more information about JSTOR, please contact [email protected].

The Royal Society is collaborating with JSTOR to digitize, preserve and extend access to Philosophical Transactions: Biological Sciences.

http://www.jstor.org THE ROYAL Publishedonline 19 May 2003 SOCIETY

A grounded theory of abstraction in artificial intelligence

Jean-Daniel Zucker LIM&BIO, EPML-CNRS IAPuces, Universite Paris XIII, 74 rue Marcel Cachin 93017 Bobigny Cedex, France, and LIP6-Departement IA, Universite Paris VI, 4 place Jussieu, 75252 Paris Cedex, France ([email protected]) In artificial intelligence, abstraction is commonly used to account for the use of various levels of details in a given representation or the ability to change from one level to another while preserving useful properties. Abstraction has been mainly studied in , theorem proving, representation (in particular for spatial and temporal reasoning) and machine . In such contexts, abstraction is defined as a mapping between formalisms that reduces the computational complexity of the task at stake. By analysing the notion of abstraction from an information quantity point of view, we pinpoint the differences and the complementary role of reformulation and abstraction in any represen- tation change. We contribute to extending the existing semantic of abstraction to be grounded on , where the notion of information quantity is easier to characterize formally. In the author's view, abstraction is best represented using abstraction operators, as they provide for classifying different and support the automation of representation changes. The usefulness of a grounded theory of abstraction in the cartography domain is illustrated. Finally, the importance of explicitly rep- resenting abstraction for designing more autonomous and adaptive systems is discussed. Keywords: artificial intelligence; abstraction; reformulation; representation change; machine learning

1. INTRODUCTION:THE NOTIONOF defined in a constructive way but rather a posteriori as a ABSTRACTIONIN ARTIFICIALINTELLIGENCE particular representation change that reduces the compu- tational complexity of the task at stake. Abstracting is a pervasive activity in perception, In and theorem abstraction is conceptualization and reasoning. In AI there is a consen- problem solving proving, often associated with a transformation of the sus (Ram & Jones 1995) that this ability to 'distil the problem rep- resentation that allows a theorem to be a essence from its superficial trappings' (Goldstone & Bar- proved (or prob- lem to be more i.e. with a reduced salou 1998) is a key issue, and that finding an adequate solved) easily, to the described in representationis often the hard part of the problem to be computational effort, according process 1. of solved when building 'intelligent' systems. In fact, in the figure This pragmatic view abstraction proved very early 1990s, Brooks did challenge the tenet of the 'good useful to its intended goal and provides a means to charac- old fashioned AI' stating that finding a good abstraction terize a transformation that is or is not an abstraction. The of a problem was the essence of intelligence and was, in intuitive is that a representation change is an abstrac- many AI systems, done by the researcherhimself (Brooks tion if the computational cost to solve a class of problems 1990, 1991). Thus, it is not surprising that a large pro- or demonstrate a class of theorems is significantly portion of AI successes rely as much on intelligent - reduced. Representation changes may be recursively ing as on adequate problem representation. applied defining a of representations of increas- Notwithstanding this fundamentalrole in AI, little study ing level of abstraction (Sacerdoti 1974; Christensen has been carried out directly on the subject, which has 1990; Ellman 1993; Knoblock 1994; Shawe-Taylor et al. emerged as a side-effect of the investigation of knowledge 1998). However, this view may not be sufficient for build- representation and reasoning. In AI this ability to forget ing new abstractions that rely on the definition of new con- irrelevantdetails and to find simpler descriptions has been cepts. Indeed, computational issues, even though investigated, with few exceptions, either in problem solv- important, are subsequent to the establishing of meaning- ing (Sacerdoti 1974; Plaisted 1981; Giunchiglia & Walsh ful relations between the '' and their in 1992; Ellman 1993; Knoblock 1994; Holte et al. 1996), the world. In concept representation, in fact, the role of or in problem reformulation (Amarel 1983; Lowry 1987; abstraction seems more related to 'making sense' of the Subramanian 1990). An account of these researches is perception of the world, by transforming it into a set of presented in Holte & Choueiry's (2003) contribution to meaningful 'concepts', prior to an efficient use of them. this issue. In these studies, abstraction is usually not Abstraction is thus a fundamental mechanism for saving cognitive efforts, by offering a 'higher' level view of our physical and intellectual environment. Goldstone & Barsalou (1998) have recently advocated a stricter link One contribution of 16 to a Theme Issue 'The abstraction paths: from experience to concept'. between perception and conceptualization in cognitive

Phil. Trans. R. Soc. Lond. B (2003) 358, 1293-1309 1293 ( 2003 The Royal Society DOI 10.1098/rstb.2003.1308 1294 J.-D. Zucker Abstractionin artificial intelligence

(a) (h)

\f -- abstractionand problemreformulation

I 7kyabstract Figure 2. (a) A set of objects on a bar table. (b) This image I aKproblem is more abstract, because the glass is hidden. This N transformation is a domain hiding abstraction. .\ abstractsolving direct solving ~6 (more efficient)

I enquire into the abstraction process in general, and to pro- pose an operational framework to devise systems that !3ii _h- solution automatically and iteratively build their own abstractions. refinement To achieve this goal it is necessary not only to define abstraction(s), but also to propose the means for exploring ground different abstractions and them. solution operationalizing To achieve a better of the abstraction process, several problems remain to be solved. First, a definition of abstraction should be provided, at least in a 1. Abstraction process for problem solving. Step 1 Figure circumscribed sense. Abstraction is intuitively related to concerns a representationchange justified by the need to the notion of 'simplicity', but this link does not make its reduce the computational complexity to solve a so-called definition as seems to be an groundproblem. Step 2 concerns solving the abstracted any easier, simplicity equally elusive notion. different be problem. Step 3 concerns the refinement of the abstract Moreover, properties may solution back to the ground representation . required for abstraction processes involved in different tasks. The problem is thus twofold: (i) to identify what properties are likely to be useful for a task; and (ii) once . Their approach offers a cognitive foundation to they have been found, how to formally represent them. A our grounded model of abstraction. more concrete problem is to define mechanisms to per- In this paper I am interested in the role played by form abstraction in practice, i.e. to identify operators that abstraction in a phase preceding problem solving, namely carry out transformations between different abstraction the phase of conceptualizing a domain, when a set of levels. Finally, one of the most challenging problems is to appropriate, possibly interrelated, concepts is defined. In explain how useful abstractions are acquired and/or a domain, concepts are used for a variety of different tasks; formed. The rest of the paper is organized as follows: ? 2 they must then be internally organized in a flexible and illustrates examples of abstraction, to give the reader an dynamic way, allowing the properties that are relevant for intuitive semantics for the transformations referred to as the task at hand to be easily and quickly retrieved. As abstract. Section 3 briefly reviews existing theories of works on selective attention confirm (Goldstone & Barsa- abstraction. Section 4 presents a grounded framework for lou 1998), show an amazing ability to change the abstraction in concept representation, whereas ? 5 intro- representation level of details and choose relevant infor- duces the notion of abstract operators. Section 6 gives an mation. However, when currently irrelevant details are illustration of the interest of a grounded theory of abstrac- removed, they must not be deleted, but simply hidden, tion in the cartography domain, and finally, ? 7 suggests because they may become useful for another task. Hence, directions for future work. a model of the abstraction process should also accommo- date different in both directions. linking representations, 2. INFORMALPRESENTATION OF its diffuse in human Notwithstanding presence percep- PERCEPTION-BASEDABSTRACTIONS tion and reasoning, abstraction is an elusive notion, diffi- cult to capture in formal terms. Given the multiplicity of In this section I provide some examples of transform- goals and tasks in which abstraction is involved, the identi- ations1 that I intend the notion of abstraction to capture. fication of a unique set of properties which it has to satisfy To provide the reader with an intuitive idea of the seman- may not be possible. In fact, there is currently no general tics of these transformations, I have chosen transform- framework that provides the means for understanding the ations of images. The different transformations of images different facets of abstraction. A comprehensive review presented differ by their properties, be it their content and a unified approach to problem-solving-oriented (domain related) or their description (co-domain related). abstraction have been proposed by Giunchiglia & Walsh Whenever possible, these transformations are related to (1992) and Choueiry et al. (2003). their cognitive equivalents as described in the seminal If abstraction is difficult to formalize, devising different work by Goldstone & Barsalou (1998). abstractions from a given initial representation is likely to be even more complex. However, the ability of an agent (a) Domain hiding to change or adapt its representation of the world is In figure 2, a number of objects are situated on a table undoubtedly essential for surviving and solving complex in a bar. A filter placed on the top hides the glass. It is problems. The practical motivation in this article is to still there, but it can no longer be seen unless the filter is

Phil. Trans. R. Soc. Lond. B (2003) Abstractionin artificial intelligence J.-D. Zucker 1295

(a) (b) (hb

Figure 3. By hiding the colour of an image of a rose (a), a more abstract image is obtained (b) This transformation, that hides elements of the description, is called co-domain Figure 5. By thresholding with only two levels a 256 grey- hiding. level picture (a), a much less detailed picture is obtained (b). This transformation is called coarsequantization in image (a) (b) compression.

one, associating to the compound the mean value of col- ours and brightness of the four component pixels. The result is a change in resolution, and we consider the image in figure 4b more abstract than that in figure 4a. Abstrac- tion, in this case, reduces the domain of a function, making sets of points (sixteen adjacent pixels) indis- tinguishable. This transformation corresponds to the blur- ring cognitive operation defined by Goldstone & Barsalou Figure 4. By lowering the resolution of an image (a), a more (1998). In image compression, this transformation is an abstract one (b) is obtained. This transformation corresponds implementation of the sub-sampling technique2 (Toelg & to a lossy sub-samplingin image compression. Poggio 1994). Sub-sampling reduces the number of bits required to describe an image and the quality of the sub- removed. From a cognitive point of view, this transform- sampled image is said to be lower than the quality of the ation corresponds to a focalization (Goldstone & Barsalou original. It is called a lossy compression algorithm.3 1998). Abstraction reduces (hides) part of the domain, to Domain reduction thus corresponds to any abstraction focus, for example, on the word 'Ciao' that has been where a group of objects is replaced by a new prototypic drawn in the foam of the coffee by the deft Italian barman. . Simons & Levin (1Q)7) have shown that when a part of an image is slowly hidden (such as the glass in figure 2b), (d) Co-domain reduction one may be blind to this change. This change blindness Starting from a grey-level picture of a person (see figure phenomenon in visual perception characterizes the fact 5a), a thresholding can be performed using only two grey that very large changes occurring in full view in a visual levels (black/white) (see figure 5b), thus hiding many of scene may go unnoticed (O'Regan 2001). Domain hiding the details of the original image. In this case, abstraction is arguably the most basic and common form of abstrac- reduces the co-domain of a function, making sets of values tion explored in AI. indistinguishable. In image processing this transformation is called quantization (Toelg & Poggio 1994). Coarse (b) Co-domain hiding quantization is similar to sub-sampling in that information In figure 3, two pictures of a rose are shown: figure 3a is discarded, but the compression is accomplished by is a coloured image, and figure 3b is a grey-level image. reducing the numbers of bits used to describe each pixel, Changing from the first to the second picture, the infor- rather than reducing the number of pixels. Each pixel is mation about the colour has been hidden. This transform- reassigned an alternative value and the number of alterna- ation is related to what Goldstone & Barsalou (1998) call tive values is less than that in the original image. This type selectivity, i.e. the ability not to pay attention to useless of abstraction is widely used at the level of symbolic perceptual features in a given task. We may notice the description to discretize numerical attributes. The intuitive asymmetry of the relation between figure 3a and b: idea is to associate a to a set of values that are whereas figure 3b can be obtained from figure 3a, the considered indiscernible (Dougherty et al. 1995; Kohavi & reverse is not possible (unless the removed information Sahami 1996). Co-domain reduction corresponds to all can be memorized). In fact, re-colouring the picture by abstractions where the domain of values taken by object means of a graphic program is indeed possible, but there descriptors, function or relation is reduced. is no guarantee that the resulting picture actually corre- sponds to the original coloured picture of the world that (e) Domain aggregation is captured by the same optical instrument. Co-domain Finally, in figure 6 a very important type of abstraction hiding corresponds to another widely considered abstrac- is described. If we were asked, in front of the desk rep- tion, where part of the object description is hidden. resented in figure 6a, what we see on the table, most of us would answer a 'computer', and not 'a monitor, a key- (c) Domain reduction board and a mouse'. We have spontaneously grouped In figure 4 a different case is illustrated. Figure 4a is together a set of objects that are functionally related into transformed by grouping four adjacent pixels into a single a composite object, i.e. a 'computer'. In figure 6, we may

Phil. Trans. R. Soc. Lond. B (2003) 1296 J.-D. Zucker Abstractionin artificial intelligence

(a) (b)

I I u-----U

Figure 6. A monitor, a PC tower, a keyboard and a mouse are each represented as objects on top of a table (a). By aggregating them, a more abstract image-where a computer is viewed as a whole-is obtained (b).

P I? . (/010 0 . co-domain domain mage reduction o00oo0 0 0 hidingo *@00G)0 @000 * 0

# x y intensity # x y intensity # x y intensity 1 1 0.18 co-domain 1 1 1 low 1 1 2 high relational 2 1 2 0.98 2 1 2 high domain 2 2 3 high 1 3 table 3 3 0.34 reduction' 3 1 3 low hiding 3 2 4 high ...... - ::: ' ' :. ..4 3 2 high 16 4 4 0.65 16 4 4 high4 high " . 6g 4 4 high ...... _hi h

Spot(sl) Spot(sl) Spot(s'1) Pos(sl,l,l) Pos(sl,1,1) Pos(s'1,1,2) Intensity(s1,18%) Intensity(s1,low) Intensity(s'1,high) predicate I Spot(s 6) Spot(sl6) Spot(s'6) Pos(s16,4,4) Pos(s16,4,4) Pos(s'6,4,4) Intensity(s16,65%). Intensity(sl6, high). Intensity(s'6,high)...... ------... -= -,; R .------

Figure 7. Illustrative examples of a similar abstraction viewed as a transformation of an image, a relational table and facts in a predicate language. This abstraction is at each level a combination of a co-domain reduction (discretize the spot intensity) and domain hiding (hides spot of low intensity). notice that, even though the whole computer is perceived hidden). At the relational level, this transformation at first sight, the components do not disappear; in fact, as respectively corresponds to the inclusion and select soon as we speak of 'computer configuration', they can be relational database operations (Goldstein & Storey 1999). retrieved and used again. This transformation corresponds At the predicate level, this transformation corresponds to to the productivity human cognitive operation defined by a value discretization followed by an object selection. Goldstone & Barsalou (1998). Domain aggregation is a transformation that to abstraction where corresponds 3. ABSTRACTION: DEFINITIONS AND are to build new ones. objects composed APPROACHES IN ARTIFICIAL INTELLIGENCE

(f) Abstract transformations in other knowledge The previous section aimed at illustrating a set of representation formalisms examples of transformations that correspond to abstrac- Domain hiding, co-domain hiding, domain reduction, tions. They illustrated that the notion of abstraction does co-domain reduction and domain aggregation are typical not have to be solely related to the reduced computational transformations that illustrate the notion of perception-based complexity. The notion of information decrease does in fact abstraction. In fact, similar transformations are used within better characterize these different examples. This section non-graphical knowledge representation formalisms, be it defines the notion of abstraction, and contrasts it with that logical or not. Relational databases, propositional, or first- of reformulation and representationchange. order logic are other types of representation formalism where abstractions are typically considered. Figure 7 illus- (a) Representation changes, abstraction trates an abstraction within three different formalisms. At and reformulation the perception level, this abstraction corresponds to a com- There are many tasks or problems for which there exists bination of both co-domain reduction (as the resolution of one good representation,4 which allows the definition of the image has been lowered) and domain hiding (as spots efficient and sound algorithms. This said, there are also of pixels whose intensity is not above a threshold are numerous problems for which a good representation is

Phil. Trans. R. Soc. Lond. B (2003) Abstraction in artificial intelligence J.-D. Zucker 1297 either as yet unknown or exists independently of precise expressive or simpler formalism to become possible. In this goals. Since the beginning of AI, mechanisms allowing context, the term reformulation is restricted to precisely changes of representation (Amarel 1968; Korf 1980; characterize a change of the sole representation formal- Benjamin et al. 1990) and problem reformulation (Riddle ism.6 1990; Wneck & Michalski 1994; Lavrac & Flach 2001; Choueiry et al. 2003) have been envisioned. Definition 3.2. A reformulation is a change of representation, one to A seminal contribution to the understanding of rep- from anotherformalism, preserving the quantity of infor- mation involved. resentation changes in AI comes from Korf (1980). Within he between trans- problem solving, distinguishes A review of the relation between the notion of abstraction formations that are isomorphisms and those that are homo- and 'approximation' is beyond the scope of this article. morphisms. Isomorphisms preserve the quantity of However, there is a large body of research on approximate information of the initial problem but modify the structure reasoning7 (Imielinski 1987; Lesser et al. 1988; Subra- that represents this information. Changing the represen- manian 1990), 'qualitative reasoning' (Kuipers 1986), tation of a circle of radius R from 2 + y2 = R2 to p= R temporal reasoning (Bettini et al. 1998) and spatial and 0 E [0,27-] is a typical case. By contrast, homomor- reasoning (Forbus et al. 1991). Allen's famous set of phisms change the formalism but modify the infor- relations between interval diagrams is a good illustration mation quantity.5 of links between approximation and abstractions (Allen 1984). It describes all possible relations among inter- A abstraction (b) definition of vals according to a logical framework: before, meets, over- Before a definition of abstraction, it giving general laps, during, starts, finishes and equals. should be noted that the term 'abstraction' has been widely used in AI, and in a wide range of contexts: abstract problem solving and planning (Sacerdoti 1974; Giunchig- (c) The difficulty to define the notions of lia & Walsh 1992), abstract states (Provan 1995; Holte 'simplicity', 'details' and 'desirable properties' et al. 1996), temporal abstraction (Euzenat 1995; Shahar Definition 3.1 remains elusive, and, in effect, when one 1997; Bettini et al. 1998), space abstraction (Forbus et al. tries to formalize the definition of an abstraction one 1991), abstracted causal model (Iwasaki 1990), , encounters the formalization of the notions of details, desir- classes and abstract types (Briot & Cointe 1987), abstract able and simplicity. The following is a quick survey, interpretation (Cousot 2000), knowledge and abstract more particularly in the context of a machine learning reasoning (Hobbs 1985; Imielinski 1987; Smoliar 1991), task, which clearly shows the variety of characterization abstract proof (Giunchiglia & Walsh 1992), abstract given for these three notions upon which the definition of lambda-calculus (Orlarey et al. 1994), abstraction relies. This diversity underlines the difficulty (Zucker & Ganascia 1996), abstract conceptual graph in formalizing definition 3.1. (Bournaud et al. 2000), abstract data (Smith & Smith The notion of 'detail' is often associated with that of rel- 1977; Goldstein & Storey 1999) and abstract genotype evance to a class of tasks (Subramanian et al. 1997; Lavrac (Bentley & Kumar 1999), etc. et al. 1998). Details that are hidden are indeed defined as The common idea that underlies these various uses of 'less relevant' to these tasks. In machine learning, the word abstraction echoes the human capacity to focus for example, Blum & Langley (1997) have given several on simpler descriptions in perception and conceptualiz- definitions of attribute relevance. Their definitions are ation as well as in reasoning (Goldstone & Barsalou 1998, related to measures that quantify the information brought p. 62). Giunchiglia & Walsh (1992), whose work is among by an attribute with respect to other attributes, the class the most cited in the field of abstraction, define abstrac- or the sample distribution. Based on these definitions a tion as a process of mapping an initial representation of a filter approach to feature selection (a kind of co-domain hid- problem. The latter is built by hiding details from the ing at the language level) may be applied to an initial rep- ground one, to simplify the exploration space preserving resentation of examples to ease the learning task by some 'properties' desirable to map back the abstract sol- reducing the number of features. Domain hiding is also ution. Definition 1 is an attempt to summarize the com- used in machine learning to reduce the number of mon view of abstraction in AI. examples to speed up instance-based algorithms such as k-Nearest Neighbors (kNN) (Wilson & Martinez 1998). Definition 3.1. An abstraction is a change of representation,in In problem solving where a basic mechanism to speed up a same formalism, that hides details and preserves desirable the search is to hide parts of the operators' pre-conditions, properties. the 'criticity' to satisfy an operator pre-condition may be used to rank the pre-condition and decide the ones that This definition is more restrictive than that proposed by will be dropped (Bundy et al. 1996; Yang 1997). It is also Giunchiglia & Walsh (1992), as it requires that the formal- possible to define a domain-independent notion of detail. ism remain unchanged in the transformation. The useful- In that case, it is related to the notion of scale or granularity ness of this restriction is better in distinguishing the of descriptions (Hobbs 1985; Imielinski 1990; Smoliar essence of an abstraction from the general notion of rep- 1991). resentation change. Nevertheless, frequently an abstrac- The notion of 'desirable properties' depends on the field tion and a change of formalism are combined into a where abstractions are used. In problem solving, for representation change as described in figure 8a. Indeed, example, a classic desired property is the 'monotonicity' abstraction makes it possible to change the quantity of (Sacerdoti 1974; Knoblock 1994). This property states information represented, allowing a reformulation in a less that operators' pre-condition does not interfere once

Phil. Trans.R. Soc. Lond. B (2003) 1298 J.-D. Zucker Abstractionin artificial intelligence

(a) (b) less information natural text _ representation r------informationquantity axis I the quickbro abstracion l / abstraction - fox jumps over ;abstraction-fox-'ump over-the I < thelazy dog. The Tal -o. The fox I \ X I-- --'<" s/ fox is happy. i a - app. I I reformulation I reformulation representation 4 chan e I brown i change ' dog fox l' happy is jumps I . lazy I over quick the 'bag of words' represent,7u, tin( 7. l I

Figure 8. (a) Change of representation viewed as a combination of abstraction and reformulation. (b) A change of representation from a natural text representation to a 'bag of words' representation. The abstraction assumption is that the presence of a word is relevant and not its position. This representation change is often used in text indexing. abstracted. Another useful property is the 'downward understanding the that justify the choice of a parti- refinement' (Bacchus & Yang 1994) that states that no cular abstraction and the search for better abstractions. A backtrack in a hierarchy of abstract space is necessary to comprehensive theory of the principles underlying build the refined plan. In theorem proving, a desirable abstractions is useful for a number of reasons. From a property states that for each theorem in the ground rep- practical point of view it may provide: (i) the means for resentation there exists a corresponding abstract one in clearly understanding the different types of abstraction the abstract representation (this property is called TI used in past work; (ii) semantic and computational justifi- abstraction). In machine learning, a desirable property cations for using abstractions; and (iii) justifications to states that generalization order between hypotheses is pre- automatically useful abstractions. served (Giordana & Saitta 1990). In constraint satisfaction Moreover, an understanding of different abstractions within programming, a desirable property is that the set of vari- a common framework can allow the transfer of techniques ables that are abstracted into one have 'interchangeable between disparate domains (Nayak & Levy 1995). There supports' (Freuder 1991). More generally, there is a have been several attempts to propose such a type of domain-independent desirable property: it states that a theory. Many of them have been carried out in theorem given order relation is preserved by an abstraction (Hobbs proving, where the role of abstraction is to find a sketch 1985). of a proof, whose details can be filled in later. This section The notion of 'simplicity' plays a key role in the charac- gives a brief account of existing theories of abstraction and terization of abstracted representation. There is abundant their associated definition of abstraction. literature discussing the simplicity of a representation. In machine learning, this notion is fundamental because it (a) Abstraction as syntactical mapping guides the choice between equally accurate concept (i) Abstraction as predicate mapping descriptions. Indeed, the 'simplest' concept is invariably Predicate mapping (Plaisted 1981; Tenenberg 1987) is favoured following the parsimony principle or Occam's a class of abstractions based on the that the razor8 (Blumer et al. 1987; Domingos 1998). There are distinctions between a set of predicates P1, ..., Pn in a different measures of the simplicity of a hypothesis space theory may become irrelevant for certain inferences. An that have been proposed. The capacity,9 for example, is a abstract theory can be constructed by replacing all occur- measure of the power of a collection of functions to dis- rences of the predicates Pi in the base theory by a single criminate between classes. Some other principles or meas- abstract predicate Pa. Let us consider, for example, the ures of simplicity are used in machine learning: the following base theory that states that Hondas are Japanese Minimum Description Length (Rissanen 1985) and the cars, BMW are European cars, and both European and Minimum Message Length (Wallace & Boulton 1968). Japanese cars (among others) are types of car: Finally, mention should be made of the rich literature on BASE THEORY HONDA(x) = JAPANESECAR(x), (4.1) information-theoretic approaches to the notion of sim- plicity (Kolmogorov 1965; Li & Vitanyi 1993), compu- BMW(x) => EUROPEANCAR(x), (4.2) tational learning (Blumer et al. 1987; Kearns 1990) and JAPANESECAR(x) = CAR(x), (4.3) statistical learning (Vapnik 1995). EUROPEANCAR(x) = CAR(x). (4.4) The between the predicates 4. THEORIESOF ABSTRACTION Pi = JAPANESECAR and P2 = EUROPEANCAR may Section 3 has provided a workable definition of a trans- become irrelevant (e.g. when trying to answer a query formation that is or is not an abstraction. An abstraction CAR(A)), and therefore these predicates can both be theory goes beyond a definition, and is a step towards mapped to Pa = JAPEUROCAR.

Phil. Trans. R. Soc. Lond. B (2003) Abstractionin artificial intelligence J.-D. Zucker 1299

PREDICATEMAPPING JAPANESECAR- JAPEUROCAR, the description of a situation, i.e. a set of ground facts, a EUROPEANCAR - JAPEUROCAR. widely used abstraction consists in hiding part of precon- dition operators as it happens in STRIPS.10 The This abstraction to axioms the applied (4.1)-(4.4) yields is to build an abstract plan that ignores the non-critical abstract following simpler theory: preconditions of operators and then refines it. Such abstraction may also be categorized as a TI abstraction as SYNTACTICALLY ABSTRACTED THEORY HONDA(x) JAPEUROCAR(x, (4.5) all the plans valid in the ground space are also valid in the BMW(x) > JAPEUROCAR(x), (4.6) abstract space. Their theory of abstraction11 is not directly JAPEUROCAR(x)= CAR(x). (4.7) related to the notion of simplicity, and although it is a powerful framework to describe contributions, it is not However, suppose the base theory also includes the fol- meant to support the construction of new abstractions. lowing axioms stating that European cars are fast and Japanese cars are reliable: (b) Abstraction as semantic mapping BASE THEORY = EUROPEANCAR(x) FAST(x), (4.8) (i) Semantic mapping of interpretation models JAPANESECAR(x) > RELIABLE(x). (4.9) The fundamental shortcomings of the syntactic theory of abstraction are that: (i) while it captures the final result the same to axioms Applying syntactical mapping (4.8) of an abstraction, it does not capture the underlying justi- and would result in the (4.9) following: fication that leads to the abstraction; (ii) TI abstractions ABSTRACT THEORY JAPEUROCAR(x)= FAST(x), (4.10) potentially lead to false proofs; and (iii) the abstracted theory may not be the strongest one given the abstraction. JAPEUROCAR(x) > RELIABLE(x). (4.11) For example, it appears that adding axiom (4.12) yields However, combining these two axioms with axioms (4.5) the strongest theory that removes predicates JAPANESE- and (4.6), leads to undesirable false proofs (Plaisted CAR and EUROPEANCAR and still does not admit 1981). For example, one can infer that Honda cars are false proofs: fast, and BMWs are reliable, inferences not sanctioned by SEMANTICALLY ABSTRACT THEORY JAPEUROCAR(x) the base theory (Nayak & Levy 1995). To avoid this prob- > (FAST(x) V RELIABLE(x)). (4.12) lem, Tenenberg (1987) has proposed a restricted type of After of the in predicate mapping. replacement predicates Nayak & Levy (1995)-extending Tenenberg's work- the the clauses that do not inconsist- clauses, only bring have proposed a semantic theory of abstraction. Their are ency kept. theory defines abstraction as a model-level mapping rather than predicate mapping. Instead of viewing abstraction as (ii) Abstraction as mapping between formal systems a syntactic mapping (like Giunchiglia & Walsh 1992), they Giunchiglia & Walsh (1992) have extended the syntac- view abstraction as a two-step process: a kind of semantic tic view of abstraction of Plaisted (1981) to a mapping mapping. The first step consists in abstracting the between formal systems. A I is defined by a 'domain', and the second one in constructing a set of triple (A,Ai,2), where A is the language, A is the deductive abstract formulae to capture the abstracted domain. This mechanism and D is the set of axioms. They define an semantic theory yields abstractions that are weaker than abstraction as follows: the base theory, i.e. they are strictly a subset of TD. Recall that Giunchiglia and Walsh do not advocate TD abstrac- Definition 4.1 (formal abstraction; Giunchiglia & Walsh tions (for problem solving), because, sound, - although they 1992). An abstraction,denoted f: X, 2, is the union of three in - 'lose completeness' comparison with TC abstractions, functions fA : A -A A2, f,A: A, A2, and ff : [2, - Q2, which which preserve completeness, and TI, which lose sound- respectivelytransform the formulas of the initial language, inference ness. and introduce two notions: rulesand axioms respectivelyinto the abstractformulas, the abstract Nayak Levy important rules and the abstractaxioms. MI abstractions (MI that are a strict subset of TD, see defi- nition 4) and simplifying assumptions (which allow one to According to Giunchiglia & Walsh (1992), the majority evaluate the utility of an abstraction depending on the of abstractions in problem solving and theorem proving reliability of this assumption). They defined abstraction as may be represented within this framework. They also follows: an abstraction mapping H7: interpretations == notice that most abstractions modify neither the axioms (Lbase) interpretations (Labs) is a model-level specifi- nor the inference rules. Abstractions are therefore, in most cation of how the interpretations of Lbase are to be cases, mapping between languages. They have introduced abstracted to interpretations of Labs. a useful distinction between abstraction: TD and TI. This distinction is used to classify the various abstractions Definition 4.2 (model-increasing abstraction; Nayak & found in problem-solving and theorem-proving literature. Levy 1995). The theory Tabsis a model-increasingabstraction for TI abstractions are those whose image by fA of each the- Tbase with respectto an abstractionmapping H if for every model a orem from the ground space is a theorem in the abstract Mbase of Tbase (the set of sentences in Lbase), H(Mbase is modelof Tabs (the set sentences in Lbase). space. The TD abstractions are, on the contrary, abstrac- of tions such that images of certain theorems in the ground space are no longer a theorem in the abstract space. The work of Nayak and Levy, although only considering Giunchiglia and Walsh argue that useful abstractions for a limited subset of the possible abstractions (i.e. 'domain the resolution of problems are the TI abstractions because abstraction'), is, in my view, a fundamental contribution they preserve all theorems. In problem solving, given S to a theory of abstraction.

Phil. Trans. R. Soc. Lond. B (2003) ,...... _n_...... th...e.o...y

1300 J.-D. Zucker Abstractionin artificial intelligence

- u ? -r___ _.I __.______.___._lI__.____ 1___

model ground interpretation mapping . abstract structure (Giordana& Saitta 1990; structure_ <^'^> Nayak & Levy 1995) I A

ground - - _syntactical mapping abstract language (Giunchiglia& Walsh 1992) language A

clauses ground __ mapping__ abstract theory (Plaisted 1981) theory

Figure 9. Theoretical approaches to abstraction according to the level of representation they operate at: mapping of theories (Plaisted 1981), mapping of clauses (Giunchiglia & Walsh 1992) and mapping of interpretation models (Giordana & Saitta 1990; Nayak & Levy 1995).

(ii) Semantic mapping of formulae latter approaches give the means to capture what makes Historically, Giordana & Saitta (1990) were precursors the essence of the abstraction: a 'simplifying assumption'. in the semantic approach to predicate mapping. Indeed, Figure 9 summarizes these different contributions by con- they did not consider abstraction as a purely syntactic trasting the type of representation level at which they transformation but as a semantic one that preserves the operate. generality order between formulae.12 The abstraction of a clause C, expressed in a language f is another clause C', (d) Practical utility of abstraction within where relations in C have been renamed or grouped into representation change new ones defined over an abstract language f'. In their To conclude this brief of the art of theories of theory, an abstraction is defined as follows: abstraction, it is worth assessing the utility of abstractions in practice. The vast majority of works that mention Definition 4.3 (abstraction; Giordana & Saitta 1990). An 'abstraction' underline its positive effects, in particular abstract theory, mapping a language of clauses f to an abstract performance increase. Reported speed-ups are often f', is a consistent set the language of definitions of type exponential (Giunchiglia 1996). However, in certain cases Vx3 y = such that f ... (Yi=1,n) [T(x) (x,y) V V fn(,y), abstraction can lead to reductions in that where ris an abstractpredicate off' andfi= ,n areformulas based performance13 are worse than the best benefits & on predicatesin fapplied to the variablesx and Yi=I,n possible (Backstrom Jonsson 1995). Although Giunchiglia has shown that the An abstraction is, in their view, the definition of new negative results of Backstrom and Jonsson correspond to it is neverthe- concepts at the extensional level: a specific correspon- pathological problems (Giunchiglia 1996), dence between the variables of the abstract formula and less highly probable that there exists, for abstraction, an those of the initial one. These abstractions are a subset of analogue of the no-free-lunch-theorem (Wolpert 1995). It is thus essential to choose abstractions To the TD abstractions and are particularly useful to define carefully.14 the of an abstraction is related to the the notion of term abstraction as described in ? 2e and fig- summarize, utility of the that is associated with ure 6. In their theory, an abstraction of five pixels centred utility representation change it. Several factors must be taken into on a pixel x into a 'cross'-shaped larger pixel would be account, including: represented as follows: (i) the computational cost of the reformulation from the Vx3y,z,t,uCross(x) (Pixel(x) A Pixel(y) A Pixel(z) A Pixel(t) A Pixel(u) ground problem to the new representation; A Above(y,x) A Below(z,x) A Right(t,x) A Left(u,x)). (ii) the ratio between the complexity of the abstract problem and that of the initial problem; (c) Abstract view on abstraction theories (iii) the computational cost of the reformulation of the Although the theories presented in this section bear abstract solutions back to the initial representation; similar titles, they are quite different according to the rep- (iv) the density of solutions of the initial problem within resentation level to which the abstraction is considered. the new representation. The Plaisted (1981) theory considers the abstraction at the level of logical theories. Giunchiglia & Walsh (1992) have a level and are interested adopted syntactical mainly 5. A GROUNDED DEFINITIONOF ABSTRACTION in TI abstractions. Nayak & Levy (1995) as well as Giordana & Saitta (1990) have proposed to define seman- Most of the representation changes called abstraction tical abstractions at the interpretation model level. These in AI reduce the quantity of information. If the theories

Phil. Trans. R. Soc. Lond. B (2003) Abstraction in artificial intelligence J.-D. Zucker 1301 presented in ? 4 are good at characterizing and classifying perceived object attributes, FUNC is the set of perceived existing abstractions, they fail to offer a constructive functional links and REL is the set of perceived relations. approach to the problem. In this section, a somewhat We underline that in Wthere is no explicit notion of attri- novel perspective on abstraction is proposed that orig- bute or relation, but that these are meta-notions developed inates from the observation that the conceptualization of by the observer through a previous, separate process, in a domain involves, simultaneously, at least four different order to avoid dealing with the original stimuli for each levels. Given these four levels, a theory is proposed that new task. In the case of figure 10, an object x is the signal is grounded in the lowest level (the perception level) but on a spot, an attribute can be its intensity, its colour or offers the means to build compatible abstractions at any its surface, a function might be a link between the coordi- upper level of representation. nate of the spot and the identifier of the corresponding gene. Finally, a relation could specify the relative positions (a) Representation levels and description of two probes (this information is in fact useful to spatially framework normalize the signal-to-noise ratio on the chip). Underlying any source of experience there is the world At the perception level, the percepts 'exist' only for the W, which for the sake of simplicity we assume is not observer, and only during the act of perceiving. Their changing over time. The world is not really known, consists in the stimuli sent to the observer. In the because we, or artificial systems, only have a mediated considered example of figure 10, the stimuli come from access to it through our/their perception. Thus, what is the microarray, and each occurrence of a signal in a spot important for an observer is not the world per se, but the will decay without leaving any long-term effect perception P that she/he has of it. P specifies the nature (fluorescence is naturally decaying). To allow the stimuli of the elements that constitute the result of perception; for to become available over time, for retrieval and further example, P may say that the perception consists of the reasoning, they must be memorized and organized into a brightness and colour of a matrix of pixels, or of a given structure S (Van Dalen 1983). This structure is an exten- number of objects, described by shape and size. An actual sional representation of the perceived world, in which world perception P is obtained through a process J' of stimuli perceptually related to one another are stored signal/information acquisition from/about the world: together. In the example of figure 10, signals must be memorized with their attributes and organized into some P= y(W). structure, in order to lend themselves to their intended The above notation synthetically indicates that the percep- elaboration. A way of organizing them is to record the tion process /?applied to the world Wprovides particular measures of the associate attributes values in a table inside content to the elements specified by P. For the sake of a database. Obviously, in a natural system, stimuli from exemplification, let us consider a sensor used in molecular the world are elaborated by their natural perceptual sys- biology to measure the expression of genes, such as the tems, which have the definition of both f and P embed- one reported in figure 10. ded in their anatomo-physiology, as well as suitable P defines the elements of the world that we consider memorization mechanisms and appropriate relations with elementary (or atomic) percepts. To specify the reasons action. In an artificial system, storage occurs in a relational why these elements are selected is beyond the scope of database, manipulated via relational algebra operators this paper. Even though these elements may be rather (Ullman 1983). We will denote by . fthe memorization complex, they are considered the basic building blocks of process, which generates a structure S: any further conceptualization. In the case of figure 10, for S =./f(P). instance, the process /, which underlies the Microarray detector, is the hybridization of probes with sample and Finally, to describe, in a symbolic way, the perceived control cDNA, and the perception P consists of the signal world, and to communicate with other agents, a language intensity generated by the sum of the fluorescence green L is needed. L allows the perceived world to be described corresponding to the concentration of sample and cor- intentionally. Assigning names to the tables themselves, to responding to the concentration of control. A 'natural' the objects, attributes, functions and relations is then a object to consider is a spot. Nevertheless, the spot is itself process of description: built of pixels. They could equally well have been selected as basic objects.15 L= (/(S). The basic stimuli in P can be categorized according to In the example of figure 10, for instance, the intensity , including objects, attributes, functions and of the signal on a spot will be called 'intensity', the pos- relations. More precisely, objects can be atomic or com- itions of the detectors will be related to their 'x- and y- pound; atomic objects do not have parts, whereas com- coordinates', the physical/biological contiguity of detec- pound objects do have parts that are themselves objects: tors will be called 'adjacency', and so on. Finally, a theory a part-of hierarchy relates compound objects to their con- enables reasoning about the world. The theory may also stituents. Both atomic and compound objects have contain general background knowledge, which does not properties, which we call attributes. Other types of proper- belong to any specific domain. At the theory level, infer- ties involve groups of objects, resulting in functions and ence rules are used. We call theorization the process of relations. The percepts in P, generated by J'(W), can be expressing the theory in the language L (possibly enriched grouped into four classes: to accommodate domain-independent background P= < OBJ, ATT, FUNC, REL > . knowledge): OBJ is the set of perceived objects, ATT is the set of T= (LL).

Phil. Trans. R. Soc. Lond. B (2003) 1302 J.-D. Zucker Abstractionin artificial intelligence

J

l 10. i

Figure 10. Picture of a microarray, a modem tool for analysing gene expression. It consists of a small membrane, or glass slide, containing samples of many genes arranged in a regular two-dimensional array (up to 40 000 genes). Each spot on an array is associated with a particular gene. The location and intensity of a colour provides an estimate of the expression level of the gene(s) in the sample (diseased) and in the control biological extract (healthy). Green and red spots represent underexpressed and overexpressed genes, respectively, in the sample versus the control.

^af^jiyH^J^H\ ~~cDNA microarray descriptionframework

signals / 5 scanned hybridized microarray peceptual . Sf(P anchoring bl *s=S=X(P) 7: L(P) * 1 11221133 2 ~ tables I > tables I genes expression ratio o structure ?

r I t language*- T 0? L = y( W) interpretation 2.. L (W) cloneid facts . ratio attribute/valuerepresentation ?o o ,iacs .- function . language process *1 l_ theory T= -(wpinterpretatiQ' if ratio formulat ":. classification rules ?? _ . ?theory ..thenmous..(Gene223) > 3 ...... mouse is obese

Figure 11. The four levels involved in representing and reasoning about a world W in the KRA model. P denotes a perception of objects and their physical links in W. S is a set of tables, each one grouping objects sharing some property. L is a , whose semantics is evaluated on the tables of S. Finally, T is a theory formulated using L, in which the properties of the world and general knowledge are embedded. General background knowledge may provide inputs at any level.

In the example of figure 10, a theory is needed to inter- Barsalou (1998). This dependency, however, does not pret the gene expression variation, hence transforming mean that the content of the levels is generated strictly pure measurements into experimental evidence. More- bottom-up: a complex bi-directional interplay may exist, over, formal knowledge, such as the symmetry of the and conceptualization or task information may even affect adjacency relation, can be added to T. perception (Goldstone & Barsalou 1998). In figure 11, The four levels are ordered as in figure 11. They rep- background knowledge may be the database that contains resent the basis of our KRA model. An ascending (or the clone identifier of each gene of each spot on the descending) arrow from a box X to Y on figure 11 indi- microarray. cates that the syntactic and semantic definition of level Y A deeper analysis of the relations among the introduced must be interpreted (or is a description) based on the con- representation levels is out of the scope of computer tent of level X. As no world is totally isolated, a body of science, because it requires contributions from philosophy background knowledge provides, in principle, input to and cognitive science, at the very least. Thus, in this paper each level, especially to the theory, where general laws and the levels are considered as given, and the nature of the domain-independent facts may be needed. The primary -, /f, X) and 7processes are not discussed. Instead, we role of perception (level P in figure 11) in biasing the concentrate on representational issues and define a upper levels is strongly advocated by Goldstone & description framework @i(W), over a world W, as the 4-

Phil. Trans. R. Soc. Lond. B (2003) Abstractionin artificial intelligence J.-D. Zucker 1303

Lg,= g(W . La = a(W) . language .. language

?_ ?_ IC mm ? ? ? ? ? ? Tg =g(W) * Ta (W) theory . theory

Figure 12. A grounded abstraction is a mapping between a ground representation framework Dg(W) = (Pg,Sg,Lg,Tg) to another Da( W) = (Pa,Sa,La,Ta)= t(Dg(W)) such that the perception of the latter (Pa) is simpler than that of the first (Pg). The perception on the right is simpler than the one on the left according to definition 3.2.

tuple 5((W) = (P,S,L, T). Before proceeding to any formal effect on the simplicity of a perceived world, when a higher definitions, it is worth spending some time on the notion syntactic complexity implies more work to handle the con- of world W. In fact, it is by no means intended that the veyed information. This definition is general. It does not world W be restricted to the physical one, and that the impose any semantic link between Fl and F2; it only states perception P and the perception process v are only that F2 is simpler to describe. related to our five senses. A world W could be a concep- This definition respects the transitive aspect of an tual one, for instance, a text, in which a reading process abstraction (Iwasaki 1990; Yoshida & Motoda 1990), and identifies words and phrases, which form a 'perception' chains of abstraction mappings can be considered. A dis- _ of the text (see figure 8b). cussion of what concerns the relation between simplicity and information content can be found elsewhere (Li & (b) Formal definition of simplicity in a description Vitanyi 1993). Here, it is sufficient to say that abstraction framework is not concerned with the probability distribution of the Given a world W, let P be a perception of W resulting configurations belonging to a set of F: just one configur- from a process e that uses a set of sensors, each one tail- ation, the observed one, y, is relevant. Moreover, the rel- ored to capture a particular signal. Each sensor has a resol- evant problem is not recognizing the configuration, but ution threshold that establishes the minimum difference describing it. between two signals in order to consider them as distinct. Given a world W, let Dg(W) = (Pg,Sg,Lg,Tg) and A set of values provided by the sensors in v is called a Da(W) = (Pa,Sa,La,Ta) be two description frameworks over signal pattern or a configuration. Let F be the set of poss- the same world W, which we conventionally label as ible configurations detectable by Y. ground and abstract, respectively. An abstraction is a map- ping X; defined as follows: Definition 5.1 (perception simplicity; Saitta & Zucker 2001). Given a world W, let -, and 2 be two perceptionprocesses generatingP, and P2, respectively.Let Fr and F2 be the correspond- Definition 5.2 (grounded abstraction; Saitta & Zucker ing configurationsets. The perceivedworld P2 will be said simpler 2001). An abstractionis a mapping _/ from a ground description than P1 if and only if K(F2) < K(FD, whereK is the Kolmogorov framework Dg(W) onto an abstraction Da(W) such that complexityof a configurationset (Kolmogorov1965; Li & Vit- Pa = -a(W) is simpler-according to definition 5.1--than dnyi 1993). Pg(W) .

The above definition has the advantage of linking sim- This definition is illustrated in figure 12. According to plicity to its semantic meaning of the cognitive effort of this definition, all the transformations introduced in ? 2a-d information processing, rather than to its syntactic are formally defined as abstractions, as can be demon- expression. Obviously, syntactic complexity may have an strated using definition 5.1.

Phil. Trans. R. Soc. Lond. B (2003) 1304 J.-D. Zucker Abstractionin artificial intelligence

O0 Pap Pa(W) signals

O 4fa" C A # x y intensity 1 1 1 0.18 # x y intensity 0 1 I1 green 2 1 2 0.98 2 1 2 black 3 1 3 0.34 3 1 3 black . . ... 25 5 5 0.65 is255 j5 j5 reen gee.

Figure 13. An abstract structure Sa may be obtained by applying an S-operator a to the ground structure Sg, given that it is compatible with the P-operator co defined at the perception level.

6. ABSTRACTIONOPERATORS As for T-operators that simplify theories, the Prolog language or other dedicated rewriting languages are good Section 5 provides a formal definition of abstraction as examples of programming languages that support the a functional mapping X between a given perception of a writing of transformation algorithms. world and a simpler one. To build abstractions that are grounded on perception in a constructive manner, this section introduces the notion of abstraction operators at (b) Compatible operators Given an abstraction there are two the levels of structure, language and theory. To guarantee P-operator wo, ways to build an abstract structure The first consists in that the operators at the different levels of the description Sa. path co to the to obtain the abstract framework are compatible with the abstraction defined at applying ground perception then it into the the perceptual level, a notion of compatibility is introduced. perception Pa, 'memorizing' (Ja(Pa)) abstract structure Sa (see path 1-2 in figure 13). A second to the use of an that (a) P-operators, S-operators, L-operators path corresponds S-operator a( oper- ates on the in and T-operators directly ground structure Sg (see path 1'-2' Given an abstraction -, as defined in definition 5.2, an figure 13). The first one to build the abstract P-operator w is defined as follows: path requires explicitly abstract world perception Pa and then to memorize it. Definition 6.1 (abstract perception operator; Saitta & Because the memorization step of a new perception is dif- Zucker 2001). An abstractionP-operator o denotesa procedure ficult to automate fully, it is more useful to have a trans- that takes as input a perceptionPg(W) of a world W and outputs formation that works directly on the ground structure Sg. a the same world. simplerperception Pa(W) of The notion of operator compatibility is introduced to express the fact that both the mentioned paths must lead In other a a trans- words, P-operator represents generic to the same structure. This compatibility provides the of world to a formation , which, applied parti- semantic foundation to the abstract S-operators at the cular perception, produces an abstraction of it. There exist structure level. an infinite number of transformations of a given world that the above definition. Among all P- perception satisfy Definition 6.2 (S-operator compatibility). An S-operatora, we are interested in the ones that have a certain operators, applicableat the structurelevel, is compatiblewith a P-operatorw degree of generality (universality), i.e. the ones that have at the world perception level, if and only if or(ffg(Pg)) larger domains of application. P-operators classically oper- = fI( (O(Pa)). ate at the level of signals or a stream (be it an image, a sound, a text, ...). Filters in image analysis are typical The compatibility of L-operators and T-operators is example of P-operators (e.g. the ones that transform the defined in an analogous manner. It is important to insist images of figures 2a, 3a, 4a and 5a). on the fact that abstraction operators are usually The notion of abstract S-operators, L-operators and T- combined with reformulation operators. Whereas in operators are defined in an analogous fashion (Saitta & approaches to abstraction based on mappings at the Zucker 2001). The key idea is that each of these operators language level the consistency problem may emerge represents a type of algorithm that takes input knowledge (Tenenberg 1987; Nayak & Levy 1995), in the grounded represented at a given formalism level and produces a approach presented, consistency is guaranteed by the fact more abstract representation of this knowledge in the that the considered abstractions correspond to actually same formalism. SELECTION and DELETE are typical possible perceptions. However, a caveat is that it may be abstract S-operators on relational databases (Goldstein & impossible to define compatible transformation rules at Storey 1999). At the language level, XSLT16 is an example the language level. Goldstone & Barsalou (1998) have also of programming language that supports transformations. mentioned this point in relation to human perception.

Phil. Trans. R. Soc. Lond. B (2003) Abstractionin artificial intelligence J.-D. Zucker 1305

7. ILLUSTRATIVEEXAMPLES OF GROUNDED for abstraction and knowledge for reformulation, as ABSTRACTIONIN CARTOGRAPHY explained below. In this an is of the use of the section, example provided (b) Abstraction in the cartographic generalization of abstraction in the presented grounded theory cartogra- process domain. It concerns: the of the knowl- phy (i) modelling The third step is usually repeated for each desired scale of and the edge acquisition process map design; (ii) partial (see step 3' in figure 14), and consists in using the GDB automation of a of this called part process cartographic to define the objects to be symbolized on the map (e.g. generalization. which sets of lines are considered as houses), their position (e.g. a house may have to be moved to be more readable), (a) Abstraction in the map design process their level of detail (e.g. the sinuosity of a road). This step A domain where the description frameworkintroduced is called cartographic generalization (Muller et al. 1995). in the previous sections finds a natural application is the This operation is far more complex than a simple cartographydomain. Consider the process of map design reduction, and it involves abstracting details so that the (Brassel & Weibel 1988; Armstrong 1991; Mustiere & map is readable at the chosen scale. The basic operation Zucker 2002), described in figure 14 (steps 1-4). The map that an expert uses to perform cartographic generalization production process closely follows the model of abstrac- is to apply various transformation algorithms to the GDB. tion presented in this paper. The creation of maps from We have developed a machine learning approach to learn geographical data is a multiple-step process that involves how to apply these operators to produce acceptable maps. several intermediate representations.The first step of car- Because of the complexity of the task, it appears (both tography is to collect data from the geographical world theoretically and experimentally) that a direct approach, (W), or part of it. This is usually done through aerial consisting in learning from the GDB, is inappropriate. To photographs (Yg(W)) or satellite images. The photo- address this problem, the modification of the represen- graphsproduced are the perceived world P. From this pic- tation language is a promising approach that increases ture, an expert (a photogrammetrist or a stereoplotter both accuracy and rule comprehensibility. operator) manually extracts a GDB (also called Digital The abstraction process is to progress from a detailed Landscape Model or DLM by Brassel & Weibel (1988)). description of each part of a geographical object to a more A GDB contains the coordinates of all the points and lines global description, containing only those properties of the that were identified by the photogrammetrist on the object relevant to the map users' needs. For example, an images. The photogrammetristperforms, simultaneously, abstraction is to progress from a complete description of an abstraction and a reformulation of the image. In the geometry of a set of streets in a town to their descrip- addition, during this second step, she/he associates a tion as a grid (Mustiere et al. 1999). A concrete descrip- category (road, building, river, field, etc.) to the lines tion of the proposed model has been done in a identified. In the third step, the GDB is displayed by cartographic generalization of buildings and roads means of cartographic applied to objects stored (Mustiere et al. 2000), leading to a partial automation of in it. This step is performed by a cartographerand corre- the process and very significant results. As a tangible proof sponds to the choice of a language, L, which, in this case, of the cartographic quality of the results obtained, the is an iconic one, consisting of symbols such as roads, cit- algorithms developed are now incorporated into the mass ies, buildings, rivers, etc. Finally, maps are not an end production of maps at the French National Geographic in themselves; maps are created for space and landscape Institute (IGN). analysis, for itinerary search, measuring distances, esti- or other mating population density, geographical theory 8. CONCLUSION construction (T). The map may therefore be completed in a fourth step by various types of information Abstraction is an elusive notion that plays a fundamen- corresponding to the theme of the map (geology, rainfall, tal role in both and AI. In the latter, population, tourist, , etc.). The distinction into lev- the common idea that lies behind most abstractions is that els, represented in figure 14 by the vertical axis, is only of a change of the representation that reduces the compu- one aspect of the cartographicprocess of designing maps. tational cost for solving a class of task. There are countless All steps of map creation involve both knowledge rep- illustrative examples of changes of representation. They resentation and knowledge abstraction, because each step often lead to an exponential increase in problem-solving retains only part of the information available;a GDB does performances. In this paper, it has been argued that this not contain all the information seen on the image by the pragmatic definition of abstraction does not support build- photogrammetrist.17Thus, map creation is best rep- ing representation changes that lead to simpler represen- resented as a process combining, at each step, an abstrac- tation. By analysing the notion of simplicity from an tion (a simplification) and a reformulation (a change of information quantity point of view, the complementary formalism). The KRA model introduced in ? 5 supports roles of reformulation and abstraction in any represen- the modelling of both the process of abstraction (change tation change have been considered. The model we have of level of detail) and the process of reformulation(change introduced captures some important aspects of the pre- of language). Indeed, usually these two processes are viously proposed theory of abstractions and limits itself to closely intertwined in cartography and difficult to grounded abstraction, i.e. abstraction that may be charac- distinguish from one another. This distinction provides terized by an information loss at the perception level. the basis for automating cartographic knowledge acqui- Although inherently limited, this definition of abstraction sition as a combined acquisition of specific knowledge supports the definition of abstraction operators at different

Phil. Trans. R. Soc. Lond. B (2003) 1306 J.-D. Zucker Abstractionin artificial intelligence

geographicworld abstraction lb..- m

Ce)i0 Kdata collection

image (

geographical database 1? 1/25000 cartographicgeneralization and symbolization

map map 1/25 000 [::??^<^ 1/100000 thematic symbolization c * ._o

42 (D thematicmap i

Figure 14. Two types of process are intermixed in the cartographic process of conceiving and drawing a map. They are representation changes through the different levels of representation involved in the process of map drawing (Mustiere & Zucker 2002). Abstractions are represented by horizontal arrows, reformulations by vertical arrows, and other arrows represent representation changes. Such diagrams are called Reformulation/Abstraction Description (RAD) diagrams. levels of knowledge representation (raw data, relational et al. 2003), in applying abstraction operators to explore tables, languages, theories, etc.). These operators capture a space of representations to improve the learning of the essence of abstraction and provide a means of both anchors in autonomous robots, are a promising step for characterizing existing abstraction and operationalizing designing more autonomous and adaptive systems. the building of new abstraction at the database, language and theory level, provided compatible operators exist. This work is a synthesis of different contributions made in col- Operators have proved particularly useful in formalizing laboration with Lorenza Saitta. The author benefits from a the abstraction performed in cartographic generalization grant to his EPML-32 CNRS IAPuces team. and significantly improved its partial automation (Mustiere et al. 2000). ENDNOTES This grounded approach to abstraction still leaves open many fundamental questions. One question concerns the 1The words 'mapping', 'function', 'map', 'operator','transformation' and study of the most relevant property that operators ought to 'morphism' are all synonyms; they are often used in the representation literature to denote the relation between an initial and a for deductive, abductive and . change preserve changed representation. Another question concerns an algebraic formalization of 2Sub-samplingcan be implemented by keeping only a fraction of the pix- abstraction operators. Such a formalization would be use- els from the original. This latter transformationis the most basic of all ful to combine abstraction operators and define the image compression techniques and may be described as domain hiding. 3The of a does not result in an to be a decompression lossy compressed image properties preserved. Furthermore, comprehensive image similar to the original one. description of useful operators in the different fields of AI 4Let us underline the ambiguityof the term representation,which refers to would offer a way to classify existing contributions to both the notion of representationformalism as well as a specific description in a formalism. abstraction. An exciting direction for research includes the given 5'Changes of [logical] representationare characterizedas isomorphisms automatic change of representation by composing abstract and homomorphisms, correspondinglyto changes of information struc- and reformulation operators. Early experiments (Bredeche ture and information quantity, respectively' (Korf 1980).

Phil. Trans. R. Soc. Lond. B (2003) Abstractionin artificial intelligence J.-D. Zucker 1307

6Definitions 3.1 and 3.2 are related to the notions of isomorphisms and Bettini, C., Wang, X. S., Jajodia, S. & Jia-Ling, L. 1998 Dis- introduced Korf homomorphisms by (1980). covering temporal relationships with multiple granularities 7Imielinski (1987) has proposed an approximate reasoning framework for in time IEEE Trans. Know. Data abstraction. He called such a kind of reasoning 'limited', because it is sequences. Engng 10, weaker than the general predicate logic-proof method, but it leads to sig- 222-237. nificant computational advantages. Blum, A. & Langley, P. 1997 Selection of relevant features and 8Occam's razor is a logical principle attributed to the medieval philos- examples in machine learning. Artif. Intell. 1-2, 245-271. opher William of Occam (or Ockham). The principle states that one Blumer, A., Ehrenfeucht, A., Haussler, D. & Warmuth, M. K. should not make more assumptions than the minimum needed. 1987 Occam's razor. Process.Lett. 377-380. 9The be esimated the dimen- Inf. 24, capacity may using Vapnik-Chervonenkis M. & 2000 Abstractions sion (VC-dim). Bournaud, I., Courtine, Zucker, J.-D. 'OA STRIPS operator is a triple (Precondition, Deletion, Addition), for knowledge organization of relational descriptions. In Lec- respectively corresponding to a set of ground facts, denoting the oper- tures Notes in ComputerScience 1864 (ed. B. Y. Choueiry & ator's preconditions, deleted facts and added facts. This type of represen- T. Walsh), pp. 86-106. New York: Springer. tation is widely used in planning, and ABSTRIPS is a well-known abstract Brassel, K. & Weibel, R. 1988 A review and conceptual frame- that extends STRIP abilities (Yang 1997). planner work of automated map generalization. Int. J. Geogr. "'Notice that we do not formally study the requirementfor simplicity or Inf: 229-244. any more global requirementsfor increased efficiency. This would require Syst. 2, some complexityarguments which will be discussed in subsequent papers' Bred&che, N., Chevaleyre, Y., Zucker, J.-D., Drogoul, A. & (Giunchiglia & Walsh 1992, p. 5). Sabah, G. 2003 A meta-learning approach to anchor visual 12Thisproperty is indeed much desired in machine learning as it used to percepts. Robot. Auton. Syst. J. (Special issue on Anchoring the of that and not counter- explore space hypothesis generalize examples Symbols to Sensor Data in Single and Multiple Robot examples of a concept. 3Backstrom& Jonsson (1995) have showed that there are domains for Systems.) which the ALPINE (Knoblock 1994) and HIGHPOINT (Bacchus & Briot, J.-P. & Cointe, P. 1987 A uniform model for object- Yang 1994) algorithmsmay generate exponentiallylonger plans than opti- oriented languages using the class abstraction. In Int. Joint mal, despite the downwardrefinement property. Conf. on Artificial Intelligence(IJCAI '87), Milan, Italy (ed. "The no-free-lunch-theoremstates that there is no such thing as a univer- J. McDermott), pp. 40-43. San Mateo, CA: Morgan Kauf- sal 'best' algorithm for all possible search problems. mann. '5The of what is and what is not an is too difficult problem defining object R. A. 1990 don't chess. Robot. Auton. a problem. If, historically, the primary goal of computer vision has been Brooks, Elephants play to identify an object (Stone 1993), a unique operationaldefinition of what Syst. 6, 3-15. an object is has not emerged. In the field of human vision, several Brooks, R. A. 1991 Intelligence without representation. Artif. researcherssuggest that anything perceived that we can either act or talk Intell. 47, 139-159. upon may be considered as an object (Buser & Imbert 1992). Bundy, A., Giunchiglia, F., Sebastiani, R. & Walsh, T. 1996 '6XSLT Transformations)is a (eXtensible Stylesheet Language language criticalities. Intell. 88, 39-67. for transformingXML documents into other XML documents. Calculating Artificial '7The ideal geographical dataset that would be obtained without any Buser, P. & Imbert, M. 1992 Vision. Cambridge, MA: MIT abstractionis called the nominalground. It is considered to represent the Press. true value for the values contained in the geographicaldataset, and is used Choueiry, B., Iwasaki, Y., McIlraith, S. 2003 Towards a prac- to evaluate the quality of the geographicaldataset. tical theory for reformulation for reasoning about physical systems. Artif. Intell. (In the press.) REFERENCES Christensen, J. 1990 A hierarchical planner that generates its own . In Proc. 8th National Conferenceon Artificial Allen, J. 1984 Towards a general theory of action and time. Intelligence,29 July-3 August 1990. Boston, MA: AAAI Press. Artif Intell. 23, 123-154. Cousot, P. 2000 Abstract interpretation based formal methods Amarel, S. 1968 On representations of problems of reasoning and future challenges. In Lecture notes in computerscience about actions. Machine Intell. 3, 131-171. 2000 (ed. R. Wilhelm), pp. 138-156. Heidelberg: Amarel, S. 1983 Problems of representation in heuristic prob- Springer-Verlag. lem solving: related issues in the development of expert sys- Domingos, P. 1998 Occam's two razors: the and the tems. In Methods of heuristics(ed. M. Groner, R. Groner & sharp blunt. In Proc. 4th International on Dis- W. Bischov), 245-350. Hillsdale, NJ: Lawrence Conference Knowledge pp. Data Erlbaum. coveryand Mining (KDD-98), New York, 27-31 August 1998 (ed. R. P. E. Stolorz & G. Armstrong, M. P. 1991 Knowledge classification and organis- Agrawal, Piatetsky-Shapiro). MA: AAAI Press. ation. In Map generalization:making rulesfor knowledgerep- Boston, R. & M. 1995 and resentation(ed. B. P. Buttenfield & R. B. McMaster), pp. Dougherty, J., Kohavi, Sahami, Supervised discretization of continuous features. In Proc. 86-102. Essex, UK: Longman. unsupervised 12th Int. on Machine A. Prieditis & S. Bacchus, F. & Yang, Q. 1994 Downward refinement and the Conf. Learning (ed. 194-202. San CA: Kaufmann. efficiency of hierarchical problem solving. Artif. Intell. 71, Russell), pp. Mateo, Morgan 43-100. Ellman, T. 1993 Synthesis of abstraction hierarchies for con- Backstrom, C. & Jonsson, P. 1995 Planning with abstraction straint satisfaction by clustering approximately equivalent hierarchies can be exponentially less efficient. In Proc. 15th objects. In Proc. 10th Int. Conf. on Machine Learning, Int. Joint Conf on A.I (ed. M. E. Pollack), pp. 1599-1604. Amherst, MA (ed. C. Sammut & A. Hoffmann), pp. 104- San Mateo, CA: Morgan Kaufmann. 111. San Mateo, CA: Morgan Kaufmann. Benjamin, D. P., Dorst, L., Mandhyan, I. & Rosar M. 1990 Euzenat, J. 1995 An algebraic approach for granularity in An algebraic approach to abstraction and representation qualitative time representation. In Actes 14th Int. Joint Con- change. In Proc. AAAI Workshopon Automatic Generationof ference Artif. Intelligence,Montreal, Canada, 21-25 August Approximationsand Abstractions,Boston, MA, pp. 47-52. 1995, pp. 894-900. San Mateo, CA: Morgan Kaufmann. Bentley, P. & Kumar, S. 1999 Three ways to grow designs: a Forbus, K., Nielsen, P. & Faltings, B. 1991 Qualitative spatial comparison of embryogenies for an evolutionary design reasoning: the CLOCK project. Artif. Intell. 51, 95-143. problem. In Proceedingsof the Geneticand EvolutionaryCom- Freuder, E. 1991 Eliminating interchangeable values in con- putation Conf., Orlando, USA, vol. 1 (ed. W. Banzhaf, J. straint satisfaction problems. In Proc. Third Int. Conference Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jaki- on Innovative Applicationsof Artificial Intelligence(ed. R. G. ela & R. E. Smith), pp. 35-43. San Mateo, CA: Morgan Smith & A. C. Scott), pp. 227-233. Anaheim, CA: MIT Kaufmann. Press.

Phil. Trans.R. Soc. Lond. B (2003) 1308 J.-D. Zucker Abstraction in artificial intelligence

Giordana, A. & Saitta, L. 1990 Abstraction: a general frame- Generalisationet representationsmultiples (ed. A. Ruas), pp. work for learning. In WorkingNotes of Workshopon Auto- 353-368. Paris: Hermes. mated Generationof Approximationsand Abstractions,Boston, Mustiere, S., Zucker, J.-D. & Saitta, L. 1999 Cartographic MA, pp. 245-256. generalization as a combination of representing and Giunchiglia, F. 1996 Using abstrips abstractions. Where do we abstracting knowledge. In Proc. ACM/GIS'99. New York: stand? Artif. Intell. Rev. 13, 201-213. ACM Press. Giunchiglia, F. & Walsh, T. 1992 A theory of abstraction. Mustiere, S., Zucker, J.-D. & Saitta, L. 2000 An abstraction- Artif. Intell. 56, 323-390. based machine learning approach to cartographic generaliz- Goldstein, R. C. & Storey, V. C. 1999 Data abstractions: why ation. In Proceedingsof the 9th InternationalSymposium on and how? Data Knowl. Engng 29, 293-311. Spatial Data Handling, Beijing, China, August 2000 (ed. A. Goldstone, R. & Barsalou, L. 1998 Reuniting perception and Jeh), pp. 50-63. conception. Cognition65, 231-262. Nayak, P. P. & Levy, A. Y. 1995 A semantic theory of abstrac- Hobbs, J. 1985 Granularity. In InternationalJoint Conference tion. In Proc. FourteenthInternational Joint Conferenceon Arti- on Artificial Intelligence,Los Angeles, CA, USA, 1985 (ed. A. ficial Intelligence(IJCAI-95), Montreal, Canada, 20-25 August Toshi), pp. 432-435. San Mateo, CA: Morgan Kaufmann. 1995 (ed. A. Toshi), pp. 196-202. Holte, R. C. & Choueiry, B. Y. 2003 Abstraction and O'Regan, J. K. 2001 on change blindness. In Vision reformulation in artificial intelligence. Phil. Trans. R. Soc. and attention (ed. L. R. Harris & M. Jenkin), pp. 281-302. Lond. B 358, 1197-1204. (DOI 10.1098/rstb.2003.1317.) Berlin: Springer. Holte, R. C., Mkadmi, T., Zimmer, R. M. & MacDonald, A. J. Orlarey, Y., Fober, D., Letz, S. & Bilton, M. 1994 Lambda 1996 Speeding up problem-solving by abstraction: a graph- calculus and music calculi. In Proc. Int. ComputerMusic oriented approach. Artif. Intell. 85, 321-361. Conf., pp. 243-250. San Francisco, CA: International Com- Imielinski, T. 1987 Domain abstraction and limited reasoning. puter Music Association. In Proc. 10th InternationalJoint Conferenceon ArtificialIntelli- Plaisted, D. 1981 Theorem proving with abstraction. Artif. gence, Milan, Italy, 1987, pp. 997-1003. San Mateo, CA: Intell. 16, 47-108. Morgan Kaufmann. Provan, G. M. 1995 Abstraction in belief networks: the role Imielinski, T. 1990 Abstraction in query processing. J. Assoc. of intermediate states in diagnostic reasoning. In Conf. on Comput. Machinery (JACM) 38, 534-558. Uncertaintyin Artificial Intelligence,Montreal, Canada (ed. P. Iwasaki, Y. 1990 Reasoning with multiple abstraction models. Mesnard & S. Hanks), pp. 464-471. San Mateo, CA: Mor- In Proc. AAAI Workshopon Automatic Generationof Approxi- gan Kaufmann. mations and Abstractions,Boston, MA, pp. 122-133. Ram, A. & Jones, E. 1995 Foundations of foundations of arti- ficial Phil. 193-199. Kearns, M. J. 1990 The computationalcomplexity of machine intelligence. Psychol. 8, P. 1990 learning. Cambridge, MA: MIT Press. Riddle, Automating problem reformulation. In Change and inductivebias D. P. Knoblock, C. 1994 Automatic generation of abstraction for of representation (ed. Benjamin), pp. 105-124. MA: Kluwer. planning. Artif. Intell. 68, 243-302. Boston, 1985 Minimum In Kohavi, R. & Sahami, M. 1996 Error-based and entropy-based Rissanen, J. description length principle. statistical vol. 5 (ed. S. Kotz & N. L. discretization of continuous features. In Proceedingsof 2nd Encyclopediaof sciences, 523-527. New York: International Conferenceon Knowledge Discovery and Data Johnson), pp. Wiley. Sacerdoti, E. 1974 in a of abstraction Mining (KDD-96), 114-119. Menlo Park, CA: AAAI Planning hierarchy pp. Intell. Press. . Artif. 5, 115-135. Saitta, L. & Zucker, J.-D. 2001 A model of abstraction in visual Kolmogorov, A. N. 1965 Three approaches to the quantitative perception. Appl. Artif. Intell. 15, 761-776. definition of information. Prob. Inf. Transmission1, 4-7. Shahar, Y. 1997 A framework for knowledge-based temporal Korf, R. E. 1980 Towards a model for representation change. abstraction. Artif. Intell. 90, 79-133. Artif: Intell. 14, 41-78. P. Williamson, R. & B. 1986 simulation. Intell. Shawe-Taylor, J., Bartlett, L., Anthony, Kuipers, Qualitative Artif. 29, M. 1998 Structural risk minimization over 289-338. data-dependent hierarchies. NeuroCOLT technical report, NC-TR-96-053, Lavrac, N. & Flach, P. A. 2001 An extended transformation ftp://ftp.dcs.rhbnc.ac.uk /pub/neurocolt/tech reports. approach to inductive logic ACM Trans. Com- programming. Simons, D. J. & Levin, D. T. 1997 Change blindness. Trends 458-494. put. Logic 2, Cogn. Sci. 1, 261-267. Lavrac, N., D. & P. D. 1998 A Gamberger, Turney, relevancy Smith, J. M. & Smith, D. 1977 Database abstractions: aggre- filter for constructive induction. In Feature con- extraction, gation and generalization. ACM Trans. Database Syst. 2, struction,and selection:a data miningperspective (ed. H. Liu & 105-133. H. 137-154. The Netherlands: Motoda), pp. Dordrecht, Smoliar, S. 1991 Algorithms for musical composition: a ques- Kluwer. tion of granularity. Computer(July), 54-56. V. & E. H. 1988 Lesser, R., Pavlin, J. Durfee, Approximate Stone, J. V. 1993 Computer vision: what is the object? In Arti- in processing real-time problem solving. Artif. Intell. Maga- ficial intelligence and simulation of behaviour, Birmingham, zine 9, 49-61. England (ed. A. Sloman, D. Hogg, G. Humphrey, A. Ram- Li, M. & Vitanyi, P. 1993 An introductionto Kolmogorovcom- say & D. Partridge), pp. 199-208. Amsterdam: IOS Press. plexity and its applications.New York: Springer. Subramanian, D. 1990 A theory of justified reformulations. In Lowry, M. 1987 The abstraction/implementation model of Change of representationand inductive bias (ed. D. P. problem reformulation. In InternationalJoint Conferenceon Benjamin), pp. 147-167. Boston, MA: Kluwer. ArtificialIntelligence, Milan, Italy, 1987, pp. 1004-1010. San Subramanian, D., Greiner, R. & Pearl, J. (eds) 1997 Artif. Mateo, CA: Morgan Kaufmann. Intell. 97. Special Issue on Relevance. Muller, J.-C., Lagrange, J.-P., Weibel, R. & Salge, F. 1995 Tenenberg, J. 1987 Preserving consistency across abstraction Generalization: state of the art and issues in. In GIS and mappings. In Proc. IJCAI-87, Milan, Italy, 1987 (ed. J. generalization(ed. J.-C. Muller, J.-P. Lagrange & R. Weibel), McDermott), pp. 1011-1014. pp. 3-17. London: Taylor & Francis. Toelg, S. & Poggio, T. 1994 Towards an example-based image Mustiere, S. & Zucker, J.-D. 2002 Generalisation cartogra- compression for video conferencing. A.I. memo phique et apprentissage automatique partir d'exemples. In no. 1494, MIT, Boston, MA, June.

Phil. Trans. R. Soc. Lond. B (2003) Abstractionin artificial intelligence J.-D. Zucker 1309

Ullman, J. D. 1983 Principles of database systems, vol. 1, 2. Yoshida, K. & Motoda, H. 1990 Towards automatic gener- Rockville, MD: Press. ation of hierarchical knowledge bases. In Proc. AAAI Work- Van Dalen, D. 1983 Logic and structure.Berlin: Springer. shop on Automatic Generation of Approximations and Vapnik, V. N. 1995 The natureof statisticallearning theory. New Abstractions,Boston, AMA,pp. 98-109. York: Springer. Zucker, J.-D. & Ganascia, J.-G. 1996 Changes of represen- Wallace, C. S. & Boulton, D. M. 1968 An information meas- tation for efficient learning in structural domains. In Proc. ure for classification. Comput.J. 11, 185-194. 13th Int. Conf. on Machine Learning, Bari, Italy, 3-6 July Wilson, D. R. & Martinez, T. R. 1998 Reduction techniques 1996, pp. 543-551. San Mateo, CA: Morgan Kaufmann. for exemplar-based learning algorithms. Machine learning. Mach. Learn. 38, 257-268. Wneck, J. & Michalski, R. 1994 Hypothesis-driven construc- GLOSSARY tive induction in AQ17-HCI: a method and experiments. AI: artificial intelligence Mach. Learn. 14, 139-168. GDB: database Wolpert, D. H. (ed.) 1995 The mathematicsof generalization. geographical KRA: reformulation and abstraction Santa Fe Institute Studies in the Sciences of Complexity and knowledge Addison Wesley. MI: model increasing Yang, Q. 1997 Intelligentplanning: a decompositionand abstrac- TD: theorem decreasing tion based approach.Berlin: Springer. TI: theorem increasing

Phil. Trans. R. Soc. Lond. B (2003)