From: AAAI Technical Report SS-95-01. Compilation copyright © 1995, AAAI (www.aaai.org). All rights reserved. Lexical Acquisition via Constraint Solving * Ted Pedersen Weidong Chen Departmentof ComputerScience & Engineering SouthernMethodist University Dallas,TX 75275 {pedersen, wchen}eseas, smu. edu Abstract havea solution.Otherwise, constraints are inferred forunknown words that will make the sentencevalid. This paperdescribes a methodto auto- We choose to use Link Grammar[Sleatorand maticallyacquire the syntacticand se- Temperley,1991] as it providesa convenientmeans manticclassifications of unknown words. for expressingbidirectional constraints. Among the Our methodreduces the searchspace of otherframeworks we have investigatedwere Depen- thelexical acquisition problem by utiliz- dency Grammar[Mel~uk, 1988], Categorial Gram- ingboth the left and the right context of mar [Oehrleet aI., 1988],and WordGrammar [Hud- the unknownword. Link Grammarpro- son,1984] all of whichare lexically-based.We se- ridesa convenientframework in whichto lectedLink Grammar due to itsexplicit use of right implementour method. andleft context and the availability of an implemen- tationthat includes a 24,000word lexicon. However, our approachis applicableto any systemthat inte- 1 Introduction gratesbidirectional constraints explicitly in the lex- A robustNatural Language Processing (NLP) sys- icon. tem mustbe ableto processsentences that contain Thispaper begins with an introductionto Link wordsunknown to its lexicon.The syntacticand Grammar.We describethe processof acquiringthe semanticproperties of unknownwords are derived syntaxof unknownwords and outlinethe processof fromthose of knownwords in a sentence,assuming semanticacquisition. We close with a discussionof thatthe given sentence is valid. related work and our plans for the future. Theunderlying linguistic framework plays a criti- calrole in lexicalacquisition. Linguistic frameworks 2 Link Grammar canbe broadlyclassified into two groups: those with phrasestructure rules and thosewithout. The lexi- Link Grammar[Sleator and Temperley, 1991] is a con of knownwords and any phrasestructure rules context-free linguistic framework that is lexically thatexist determine the size of thesearch space for based. It differs from other context-free grammars the classificationof unknown words. In general,the in thatthere are no decomposableconstituent struc- morecomplex the phrasestructure rules, the larger turesand its grammar rules are implicit in thelexi- thesearch space. con. Thispaper explores lexical acquisition in a frame- Eachword in the grammaris definedby a syntac- workwithout phrase structure rules. All constraints ticconstraint that is expressedin a disjunctivenor- on the usageof wordsare integratedinto the lexi- realform. Each disjunct consists of a pairof ordered con.We use a novellexical representation that ex- listsof theform ((11, ..., /m-l, Im)(rn, m-l, ..., plicitlyspecifies what syntactic and semantic classes wherethe lefthand list is made up of connectors of wordsmay appear to theleft and to theright of a thatmust link to wordsto theleft of theword in the wordin a validsentence. If allwords are known in a sentenceand likewisefor theright hand list. Each sentence,it is validonly if theassociated constraints wordcan have multiple disjuncts, which implies that it canbe usedin varioussyntactic contexts. *Supportedin partby theNational Science Founda- The followingis a simpleexample of a LinkGram- tionunder Grant No. IRI-9212074. mar: 118 big, yellow: (() (A)) is to select one disjunct for each word in a sentence car, corn, condor, that will lead to &atisfaction of the the meta-rules. gasoline, meat: ((A,D,,O,) ((X,Ds)(Ss)) 3 Syntactic Acquisition ((Ds)(Ss)) Syntactic acquisition is the proce~ of mapping an ((D,,O,) unknownword to a finite set of syntactic categories. (COs)()) In Link Grammar syntactic categories are repre- eats: ((ss)(o)) sented by the constraints that are expressed as dis- ss)()) juncts. Our lexical acquisition system is not called the: l(() (D)) upon to create or identify new syntactic categories aa we assume that these are already known. Parsing a sentence in Link Grammarconsists of Given a sentence with unknown words the dis- choosing one disjunct for each word such that it can juncts of unknown words are determined based upon be connected to the surrounding words aa specified the syntactic constraints of the knownwords in the in that disjunct. For a simple example consider the sentence. sequence of words: The condor eats the meat" and For instance suppose that snipe is an unknown the following choices of disjuncts for each word from word in the sentence: "The snipe eats meat". The the lexicon above: following lists all the choices for the disjuncts of the the: (() (D)) known words which come from the lexicon. condor: ((Ds) (Ss)) the: (() (D)) eats: ((ss)(o)) snipe: ((?) (?)) the: (() (V)) eats: ((s,)(o)) meat: ((Ds,Os) ((as) meat: ((A,Ds,Os)()) ((A,Ds)(Ss)) The following diagram (called a linkage) shows the ((Ds)(Ss)) links amongthe words that justify the validity of the ((D,,O,)()) sentence according to Link Grammar. ((o,)()) + .... 08--+ It must be determined what disjnnct associated +-Ds--+-Ss--+ +-Ds-+ with ’snipe’ will allow for the selection of a single I disjunct for every known word such that each word 1:he condor eL~s 1:he mea~ can have its disjtmct satisfied in accordance with In general, a sequence of words is a sentence if it the meta-rules previously discussed. There are 10 is possible to draw links amongthe words in such distinct disjnncts in the above gr~,mmar and any one a way that the syntactic constraint of every word of those could be the proper syntactic category for is satisfied and all the following meta-rules are ob- ’snipe’. served: We could attempt to parse by blindly assigning to ’snipe’ each of these disjuncts and see which led ¯ Planarity: Links drawn above the sentence do to a valid linkage. Howeverthis is impractical since not intersect. more complicated grammars will have hundreds or ¯ Connectivity: There is a path from any word even thousands of known disjnncts. In fact, in the in the sentence to any other word via the links. current 24,000 word lexicon there are approximately ¯ Ordering: For each disjunct of a word w, of the 6,500 different syntactic constraints. A blind ap- form((iz, ..., In-z, l,n)(r,, m-x, ..., rx)), proach would assign all of these disjnncts to ’snipe’ m _> 0 and n >_ 0, the left hand list of con- and then attempt to parse. It is possible to greatly nectors indicates links to words to the left of reduce the number of candidate disjnncts by analyz- w, and likewise for the right hand list. In ad- ing the disjuncts for the known words. Those dis- dition, the larger the subscript of a connector, juncts that violate the constraints of the meta-rules the further away the word with the matching are eliminated. connector is from w. The disjuncts ((A,Ds)(Ss)) and ((Ds)(Ss)) ’meat’ are immediately eliminated as they can never ¯ Exclusion: No two links may connect the same be satisfied since there are no words to the right of pair of words. ’meat’. Parsing in Link Grammarcorresponds to constraint The disjunct ((A,Ds,Os)()) for ’meat’ can solving according to these meta-rules. The objective be eliminated. If the A connector is to be satisfied 119 it would have to be sat/~ied by ’snipe’. The or- +-Ds--+-Ss--+~Os-+ dering meta-rule implies that the Ds connector in I I I I ’meat’ wouldhave to be satisfied by ’the’ but then the snipe eal:s mea~ the remaining Os connector in ’meat’ would not be satiMiable since there are no wordspreceding ’the’. 4 Semantic Acquisition That leaves the disjnncts ((Ds,Os)()) and )) as the remainingpossibilities for ’meat’. Thedis- Acquisition of lexical semantics is defined in junct ((Ds,Os)()) can be eliminated since the [Berwick, 1983; Granger, 1977; Hastings, 1994; Rns- words that can satisfy the Ds connector are ’the’or sell, 1993] as mappingunknown words to knowncon- cepts. [Hastings, 1994; Russell, 1993] assumethat ’snipe’. Again the ordering meta-ru/e makesit im- possible to satisfy the Os connector. Thus the only the knowledgebase is a concept hierarchy structured remainingcandidate disjunct for ’meat’ is ((Os)()). as a tree where children are morespecific concepts The next wordconsidered is ’eats’. There are two than their parents. There are separate hierarchies pouihle disjuncts and neither can he immediately for nouns and verbs. Rather than usin~ concept hi- eliminated. The left handside of each disjunct con- erarchies [Berwick,1983; Granger,1977J used scripts s/sts of an Ss connector.This could only be satisfied and causal networks to represent a sequence of re- by ’snipe’ whichtherefore must have an Ss connector lated events. In their workLexical Acquisition con- sists of mappingan unknown word into a knownse- in its right hand side. Recall that the left handside quence of ’meat’ consists of an Os connector. This could be of events. Weadopt the convention of [Hast- satisfied either by the ((Ss)(O)) disjunct for ’eats’ ings, 1994; Russell, 1993Jand attempt to map un- if the right handside of ’snipe’ consists of ((Os,Ss)). knownwords into a concept hierarchy. The left handside of ’snipe’ need only consist of a In order to semantically classify an unknownword D connector in order to satisfy the right hand s/de the lexical entries of known words must be aug- of ’the’. Thus the disjunct for ’snipe’ must be ei- mentedwith semantic information derived from the actual usage of themin a variety of contexts. ther ((D)(Ss)) or ((D)(Os,Ss)) and we have eliminated any of the candidate disjuncts for ’eats’. As sentences with no unknownwords are parsed, Unfortunately the meta-rules do not allow for the each connector in the syntactic constraints of nouns further elimination of candidate disjuncts. and verbs is tagged with the noun or verb with which it connects to. For instance given the sentence: "The In cases suchas this the lexicon is usedaa a knowl- condor eats meat", the nouns and verbs are tagged edge source and will be used to resolve the issue.