Antonymy and Conceptual Vectors Didier Schwab, Mathieu Lafourcade and Violaine Prince LIRMM Laboratoire d'informatique, de Robotique et de Micro´electronique de Montpellier MONTPELLIER - FRANCE. schwab,lafourca,prince @lirmm.fr http://www.lirmm.fr/f ~ schwab,g lafourca, prince f g Abstract a kernel of manually indexed terms is necessary For meaning representations in NLP, we focus for bootstrapping the analysis. The transver- 1 our attention on thematic aspects and concep- sal relationships , such as synonymy (LP01), tual vectors. The learning strategy of concep- antonymy and hyperonymy, that are more or tual vectors relies on a morphosyntaxic analy- less explicitly mentioned in definitions can be sis of human usage dictionary definitions linked used as a way to globally increase the coher- to vector propagation. This analysis currently ence of vectors. In this paper, we describe a doesn't take into account negation phenomena. vectorial function of antonymy. This can help This work aims at studying the antonymy as- to improve the learning system by dealing with pects of negation, in the larger goal of its inte- negation and antonym tags, as they are often gration into the thematic analysis. We present a present in definition texts. The antonymy func- model based on the idea of symmetry compat- tion can also help to find an opposite thema to ible with conceptual vectors. Then, we define be used in all generative text applications: op- antonymy functions which allows the construc- posite ideas research, paraphrase (by negation tion of an antonymous vector and the enumer- of the antonym), summary, etc. ation of its potentially antinomic lexical items. Finally, we introduce a measure which evaluates 2 Conceptual Vectors how a given word is an acceptable antonym for We represent thematic aspects of textual seg- a term. ments (documents, paragraph, syntagms, etc) by conceptual vectors. Vectors have been used 1 Introduction in information retrieval for long (SM83) and Research in meaning representation in NLP is for meaning representation by the LSI model an important problem still addressed through (DDL+90) from latent semantic analysis (LSA) several approaches. The NLP team at LIRMM studies in psycholinguistics. In computational currently works on thematic and lexical disam- linguistics, (Cha90) proposes a formalism for biguation text analysis (Laf01). Therefore we the projection of the linguistic notion of se- built a system, with automated learning capa- mantic field in a vectorial space, from which bilities, based on conceptual vectors for mean- our model is inspired. From a set of elemen- ing representation. Vectors are supposed to en- tary concepts, it is possible to build vectors code `ideas' associated to words or to expres- (conceptual vectors) and to associate them to sions. The conceptual vectors learning system lexical items2. The hypothesis3 that considers automatically defines or revises its vectors ac- a set of concepts as a generator to language cording to the following procedure. It takes, as has been long described in (Rog52). Polysemic an input, definitions in natural language con- words combine different vectors corresponding tained in electronic dictionaries for human us- age. These definitions are then fed to a morpho- 1well known as lexical functions (MCP95) 2 syntactic parser that provides tagging and anal- Lexical items are words or expressions which consti- tute lexical entries. For instance, ,car- or ,white ant- are ysis trees. Trees are then used as an input lexical items. In the following we will (some what) use to a procedure that computes vectors using sometimes word or term to speak about a lexical item. tree geometry and syntactic functions. Thus, 3that we call thesaurus hypothesis. to different meanings. This vector approach (45 degrees) A and B are thematically close and π is based on known mathematical properties, it share many concepts. For DA(A; B) 4 , the is thus possible to undertake well founded for- thematic proximity between A and B ≥would be π mal manipulations attached to reasonable lin- considered as loose. Around 2 , they have no guistic interpretations. Concepts are defined relation. DA is a real distance function. It ver- from a thesaurus (in our prototype applied to ifies the properties of reflexivity, symmetry and French, we have chosen (Lar92) where 873 con- triangular inequality. We have, for example, cepts are identified). To be consistent with the the following angles(values are in radian and de- thesaurus hypothesis, we consider that this set grees). constitutes a generator family for the words and DA(V(,tit-), V(,tit-))=0 (0) their meanings. This familly is probably not DA(V(,tit-), V(,bird-))=0.55 (31) DA(V(,tit-), V(,sparrow-))=0.35 (20) free (no proper vectorial base) and as such, any DA(V(,tit-), V(,train-))=1.28 (73) word would project its meaning on it according DA(V(,tit-), V(,insect-))=0.57 (32) to the following principle. Let be a finite set The first one has a straightforward interpreta- C of n concepts, a conceptual vector V is a linear tion, as a ,tit- cannot be closer to anything else combinaison of elements c of . For a meaning than itself. The second and the third are not i C A, a vector V (A) is the description (in exten- very surprising since a ,tit- is a kind of ,sparrow- sion) of activations of all concepts of . For ex- which is a kind of ,bird-. A ,tit- has not much C ample, the different meanings of ,door- could be in common with a ,train-, which explains a large projected on the following concepts (the CON- angle between them. One can wonder why there CEPT[intensity] are ordered by decreasing val- is 32 degrees angle between ,tit- and ,insect-, ues): V(,door-) = (OPENING[0.8], BARRIER[0.7], which makes them rather close. If we scruti- LIMIT[0.65], PROXIMITY [0.6], EXTERIOR[0.4], IN- nise the definition of ,tit- from which its vector TERIOR[0.39], . is computed (Insectivourous passerine bird with In practice, the larger is, the finer the mean- colorful feather.) perhaps the interpretation of ing descriptions are. InCreturn, the computing these values seems clearer. In effect, the the- is less easy: for dense vectors4, the enumera- matic is by no way an ontological distance. tion of activated concepts is long and difficult 2.2 Conceptual Vectors Construction. to evaluate. We prefer to select the themati- cally closest terms, i.e., the neighbourhood. For The conceptual vector construction is based on instance, the closest terms ordered by increas- definitions from different sources (dictionaries, ing distance to ,door- are: (,door-)=,portal-, synonym lists, manual indexations, etc). Defini- ,portiere-, ,opening-, ,gate-, ,barrierV -,. tions are parsed and the corresponding concep- tual vector is computed. This analysis method 2.1 Angular Distance shapes, from existing conceptual vectors and Let us define Sim(A; B) as one of the similar- definitions, new vectors. It requires a bootstrap ity measures between two vectors A et B, of- with a kernel composed of pre-computed vec- ten used in information retrieval (Mor99). We tors. This reduced set of initial vectors is man- can express this function as: Sim(A; B) = ually indexed for the most frequent or difficult A B terms. It constitutes a relevant lexical items cos(A; B) = · with \ " as the scalar A B · basis on which the learning can start and rely. product. We ksuppk×kosek here that vector com- One way to build an coherent learning system ponendts are positive or null. Then, we define is to take care of the semantic relations between an angular distance D between two vectors A A items. Then, after some fine and cyclic compu- and B as D (A; B) = arccos(Sim(A; B)). In- A tation, we obtain a relevant conceptual vector tuitively, this function constitutes an evaluation basis. At the moment of writing this article, of the thematic proximity and measures the an- our system counts more than 71000 items for gle between the two vectors. We would gener- French and more than 288000 vectors, in which ally consider that, for a distance D (A; B) π A ≤ 4 2000 items are concerned by antonymy. These 4Dense vectors are those which have very few null items are either defined through negative sen- coordinates. In practice, by construction, all vectors are tences, or because antonyms are directly in the dense. dictionnary. Example of a negative definition: ,non-existence-: property of what does not exist. of the terms implies the negation of the other. Example of a definition stating antonym: ,love-: Complementary antonymy presents two kinds antonyms: ,disgust-, ,aversion-. of symmetry, (i) a value symmetry in a boolean system, as in the examples above and (ii) a sym- 3 Definition and Characterisation of metry about the application of a property (black Antonymy is the absence of color, so it is \opposed" to all We propose a definition of antonymy compat- other colors or color combinaisons). ible with the vectorial model used. Two lexi- cal items are in antonymy relation if there is 4 Antonymy Functions a symmetry between their semantic components 4.1 Principles and Definitions. relatively to an axis. For us, antonym construc- tion depends on the type of the medium that The aim of our work is to create a function supports symmetry. For a term, either we can that would improve the learning system by sim- have several kinds of antonyms if several possi- ulating antonymy. In the following, we will be bilities for symmetry exist, or we cannot have mainly interested in antonym generation, which an obvious one if a medium for symmetry is not gives a good satisfaction clue for these functions. to be found. We can distinguish different sorts We present a function which, for a given lex- of media: (i) a property that shows scalar val- ical item, gives the n closest antonyms as the neighbourhood function provides the n clos- ues (hot and cold which are symmetrical values V of temperature), (ii) the true-false relevance or est items of a vector.