Determining Word Sense Dominance Using a Thesaurus

Saif Mohammad and Graeme Hirst Department of Computer Science University of Toronto

Toronto, ON M5S 3G4, Canada g fsmm,gh @cs.toronto.edu

Abstract of insufficient evidence. The dominance values may be used as prior probabilities for the differ- The degree of dominance of a sense of a ent senses, obviating the need for labeled train- word is the proportion of occurrences of ing data in a sense disambiguation task. Natural that sense in text. We propose four new language systems can choose to ignore infrequent methods to accurately determine word senses of words or consider only the most domi- sense dominance using raw text and a pub- nant senses (McCarthy et al., 2004). An unsuper- lished thesaurus. Unlike the McCarthy vised algorithm that discriminates instances into et al. (2004) system, these methods can different usages can use word sense dominance to be used on relatively small target texts, assign senses to the different clusters generated. without the need for a similarly-sense- Sense dominance may be determined by sim- distributed auxiliary text. We perform an ple counting in sense-tagged data. However, dom- extensive evaluation using artificially gen- inance varies with domain, and existing sense- erated thesaurus-sense-tagged data. In the tagged data is largely insufficient. McCarthy process, we create a word–category co- et al. (2004) automatically determine domain- occurrence matrix, which can be used for specific predominant senses of words, where the unsupervised word sense disambiguation domain may be specified in the form of an un- and estimating distributional similarity of tagged target text or simply by name (for exam- word senses, as well. ple, financial domain). The system (Figure 1) automatically generates a thesaurus (Lin, 1998) us- 1 Introduction ing a measure of distributional similarity and an The occurrences of the senses of a word usually untagged corpus. The target text is used for this have skewed distribution in text. Further, the dis- purpose, provided it is large enough to learn a the- tribution varies in accordance with the domain or saurus from. Otherwise a large corpus with sense topic of discussion. For example, the ‘assertion distribution similar to the target text (text pertain- of illegality’ sense of charge is more frequent in ing to the specified domain) must be used. the judicial domain, while in the domain of eco- The thesaurus has an entry for each word type, nomics, the ‘expense/cost’ sense occurs more of- which lists a limited number of words (neigh- ten. Formally, the degree of dominance of a par- bors) that are distributionally most similar to it. ticular sense of a word (target word) in a given Since Lin’s distributional measure overestimates text (target text) may be defined as the ratio of the the distributional similarity of more-frequent word occurrences of the sense to the total occurrences of pairs (Mohammad and Hirst, Submitted), the the target word. The sense with the highest domi- neighbors of a word corresponding to the predom- nance in the target text is called the predominant inant sense are distributionally closer to it than sense of the target word. those corresponding to any other sense. For each Determination of word sense dominance has sense of a word, the distributional similarity scores many uses. An unsupervised system will benefit of all its neighbors are summed using the semantic by backing off to the predominant sense in case similarity of the word with the closest sense of the

121 SIMILARLY SENSE DISTRIBUTED We therefore rely on first-order co-occurrences, which we believe are better indicators of a word’s WORDNET characteristics than second-order co-occurrences A (distributionally similar words). U C X O TARGET LIN’S I R L P 2 Thesauri TEXT D THESAURUS I U A S R Published thesauri, such as Roget’s and Mac- DOMINANCE VALUES Y quarie, divide the English vocabulary into around a thousand categories. Each category has a list Figure 1: The McCarthy et al. system. of semantically related words, which we will call category terms or c-terms for short. Words with multiple meanings may be listed in more than one category. For every word type in the vocabulary of the thesaurus, the index lists the categories that PUBLISHED THESAURUS A include it as a c-term. Categories roughly cor- U C respond to coarse senses of a word (Yarowsky, X O TARGET I R 1992), and the two terms will be used interchange- WCCM L P TEXT D I U ably. For example, in the Macquarie Thesaurus, A S bark is a c-term in the categories ‘animal noises’ R DOMINANCE VALUES Y and ‘membrane’. These categories represent the coarse senses of bark. Note that published the- Figure 2: Our system. sauri are structurally quite different from the “thesaurus” automatically generated by Lin (1998), neighbor as weight. The sense that gets the highest wherein a word has exactly one entry, and its score is chosen as the predominant sense. neighbors may be semantically related to it in any The McCarthy et al. system needs to re-train of its senses. All future mentions of thesaurus will (create a new thesaurus) every time it is to de- refer to a published thesaurus. termine predominant senses in data from a differ- While other sense inventories such as WordNet ent domain. This requires large amounts of part- exist, use of a published thesaurus has three dis- of-speech-tagged and chunked data from that do- tinct advantages: (i) coarse senses—it is widely main. Further, the target text must be large enough believed that the sense distinctions of WordNet are to learn a thesaurus from (Lin (1998) used a 64- far too fine-grained (Agirre and Lopez de Lacalle million-word corpus), or a large auxiliary text with Lekuona (2003) and citations therein); (ii) compu- a sense distribution similar to the target text must tational ease—with just around a thousand cate- be provided (McCarthy et al. (2004) separately gories, the word–category matrix has a manage- used 90-, 32.5-, and 9.1-million-word corpora). able size; (iii) widespread availability—thesauri By contrast, in this paper we present a method are available (or can be created with relatively that accurately determines sense dominance even less effort) in numerous languages, while Word- in relatively small amounts of target text (a few Net is available only for English and a few ro- hundred sentences); although it does use a corpus, mance languages. We use the Macquarie The- it does not require a similarly-sense-distributed saurus (Bernard, 1986) for our experiments. It corpus. Nor does our system (Figure 2) need consists of 812 categories with around 176,000 any part-of-speech-tagged data (although that may c-terms and 98,000 word types. Note, however, improve results further), and it does not need to that using a sense inventory other than WordNet generate a thesaurus or execute any such time- will mean that we cannot directly compare perfor- intensive operation at run time. Our method stands mance with McCarthy et al. (2004), as that would on the hypothesis that words surrounding the tar- require knowing exactly how thesaurus senses get word are indicative of its intended sense, and map to WordNet. Further, it has been argued that that the dominance of a particular sense is pro- such a mapping across sense inventories is at best portional to the relative strength of association be- difficult and maybe impossible (Kilgarriff and Yal- tween it and co-occurring words in the target text. lop (2001) and citations therein).

122 3 Co-occurrence Information are incremented. The steps of word sense disambiguation and creating new bootstrapped WCCMs 3.1 Word–Category Co-occurrence Matrix can be repeated until the bootstrapping fails to im- The strength of association between a particular prove accuracy signiﬁcantly. category of the target word and its co-occurring The cells of the WCCM are populated using a words can be very useful—calculating word sense large untagged corpus (usually different from the dominance being just one application. To this target text) which we will call the auxiliary cor- end we create the word–category co-occurrence pus. In our experiments we use a subset (all except

matrix (WCCM) in which one dimension is the every twelfth sentence) of the British National ;::: list of all words (w1 ; w2 ) in the vocabulary, Corpus World Edition (BNC) (Burnard, 2000) as and the other dimension is a list of all categories

the auxiliary corpus and a window size of ¦5 ;::: (c1 ; c2 ). words. The remaining one twelfth of the BNC is

used for evaluation purposes. Note that if the tar- ::: c c ::: c

1 2 j get text belongs to a particular domain, then the :::

w1 m11 m12 ::: m1 j creation of the WCCM from an auxiliary text of ::: w2 m21 m22 ::: m2 j the same domain is expected to give better results

. . . .. :::

. . . . ::: than the use of a domain-free text. ::: wi mi1 mi2 ::: mij ...... 3.2 Analysis of the Base WCCM ...... The use of untagged data for the creation of the A particular cell, mij, pertaining to word wi and base WCCM means that words that do not re- category c j, is the number of times wi occurs in ally co-occur with a certain category but rather a predetermined window around any c-term of c j do so with a homographic word used in a differ- in a text corpus. We will refer to this particular ent sense will (erroneously) increment the counts WCCM created after the ﬁrst pass over the text corresponding to the category. Nevertheless, the as the base WCCM. A contingency table for any strength of association, calculated from the base particular word w and category c (see below) can WCCM, of words that truly and strongly co-occur be easily generated from the WCCM by collaps- with a certain category will be reasonably accurate ing cells for all other words and categories into despite this noise. one and summing up their frequencies. The ap- We demonstrate this through an example. As- plication of a suitable statistic will then yield the sume that category c has 100 c-terms and each c- strength of association between the word and the term has 4 senses, only one of which corresponds category. to c while the rest are randomly distributed among

c :c other categories. Further, let there be 5 sentences

w nwc nw: each in the auxiliary text corresponding to every

: :: w n:c n c-term–sense pair. If the window size is the com- Even though the base WCCM is created from plete sentence, then words in 2,000 sentences will unannotated text, and so is expected to be noisy, increment co-occurrence counts for c. Observe we argue that it captures strong associations rea- that 500 of these sentences truly correspond to cat- sonably accurately. This is because the errors egory c, while the other 1500 pertain to about 300 in determining the true category that a word co- other categories. Thus on average 5 sentences cor- occurs with will be distributed thinly across a respond to each category other than c. Therefore number of other categories (details in Section 3.2). in the 2000 sentences, words that truly co-occur Therefore, we can take a second pass over the cor- with c will likely occur a large number of times, pus and determine the intended sense of each word while the rest will be spread out thinly over 300 or using the word–category co-occurrence frequency so other categories. (from the base WCCM) as evidence. We can We therefore claim that the application of a thus create a newer, more accurate, bootstrapped suitable statistic, such as odds ratio, will result WCCM by populating it just as mentioned ear- in signiﬁcantly large association values for word– lier, except that this time counts of only the co- category pairs where the word truly and strongly occurring word and the disambiguated category co-occurs with the category, and the effect of noise

123 will be insigniﬁcant. The word–category pairs Weighted Unweighted having low strength of association will likely be voting voting adversely affected by the noise, since the amount Implicit sense DI,W DI,U of noise may be comparable to the actual strength disambiguation of association. In most natural language applica- Explicit sense D D tions, the strength of association is evidence for a disambiguation E,W E,U particular proposition. In that case, even if association values from all pairs are used, evidence from Figure 3: The four dominance methods. less-reliable, low-strength pairs will contribute lit- tle to the ﬁnal cumulative evidence, as compared

to more-reliable, high-strength pairs. Thus even if four methods (Figure 3) to determine dominance

; ; ;

; ; ; the base WCCM is less accurate when generated (DI ;W DI U DE W and DE U ) and the underlying from untagged text, it can still be used to provide assumptions of each. association values suitable for most natural lan- DI ;W is based on the assumption that the more guage applications. Experiments to be described dominant a particular sense is, the greater the in section 6 below substantiate this. strength of its association with words that co-occur with it. For example, if most occurrences of bank 3.3 Measures of Association in the target text correspond to ‘river bank’, then The strength of association between a sense or the strength of association of ‘river bank’ with all category of the target word and its co-occurring of bank’s co-occurring words will be larger than words may be determined by applying a suitable the sum for any other sense. Dominance DI ;W of a statistic on the corresponding contingency table. sense or category (c) of the target word (t) is:

Association values are calculated from observed

; µ A ´w c

∑w¾T

´ ; µ =

D ; t c (1)

; ;

; I W

: :: ¼

frequencies (nwc n:c nw and n ), marginal fre-

´ ; µ

¼ A w c

´ µ ¾

∑c ¾senses t ∑w T

= · = · =

: :£ : :: £

quencies (nw£ nwc nw ; n n c n ; n c

= ·

· where A is any one of the measures of association £: : ::

nwc n:c; and n nw n ), and the sample

· · ·

= from section 3.3. Metaphorically, words that co- : :: size (N nwc n:c nw n ). We provide ex- perimental results using Dice coefficient (Dice), occur with the target word give a weighted vote to cosine (cos), pointwise mutual information (pmi), each of its senses. The weight is proportional to odds ratio (odds), Yule’s coefficient of colligation the strength of association between the sense and (Yule), and phi coefficient (φ)1. the co-occurring word. The dominance of a sense is the ratio of the total votes it gets to the sum of 4 Word Sense Dominance votes received by all the senses. We examine each occurrence of the target word A slightly different assumption is that the more in a given untagged target text to determine dom- dominant a particular sense is, the greater the num-

inance of any of its senses. For each occurrence ber of co-occurring words having highest strength ¼ t ¼ of a target word t, letT be the set of words of association with that sense (as opposed to any

(tokens) co-occurring within a predetermined win- other). This leads to the following methodol- ¼ dow around t ¼ ; letT be the union of all such T ogy. Each co-occurring word casts an equal, un-

¼ weighted vote. It votes for that sense (and no

jX j and let Xt be the set of all such T . (Thus t is

other) of the target word with which it has the j equal to the number of occurrences of t, and jT is equal to the total number of words (tokens) in the highest strength of association. The dominance windows around occurrences of t.) We describe DI ;U of the sense is the ratio of the votes it gets to the total votes cast for the word (number of co- 1 Measures of association (Sheskin, 2003): occurring words).

nwc nwc ¢ N

´ ; µ = ; ´ ; µ = ;

Ô Ô

¾ ´ ; µ = gj

cos w c pmi w c log jfw T : Sns1 w t c

¢ ¢

£ £ £

n £ n n n

; µ = w c w c ´

DI ;U t c (2)

Ô j

´ ; µ ¢

nwc n:: odds w c 1

; µ = ; ´ ; µ = ;

odds ´w c Yule w c

µ ; µ = ´ ; Sns ´w t argmax A w c (3)

¢ 1

´ ; µ· :

nw : n c odds w c 1

´ µ

c ¾senses t

µ ´ ¢ ¢ ¢ µ ´

: :

2 nwc nwc n:: nw n c

; µ = ; ´ ; µ =

Dice´w c φ w c

· ¢ ¢ ¢

£ £ :£ £ £: n £ n n n n n w c w c Observe that in order to determine DI ;W or

DI ;U , we do not need to explicitly disambiguate

124 the senses of the target word’s occurrences. We ideas of Leacock et al. (1998). Around 63,700 now describe alternative approaches that may be of the 98,000 word types in the Macquarie The- used for explicit sense disambiguation of the target saurus are monosemous—listed under just one word’s occurrences and thereby determine sense of the 812 categories. This means that on aver- dominance (the proportion of occurrences of that age around 77 c-terms per category are monose-

sense). DE ;W relies on the hypothesis that the in- mous. Pseudo-thesaurus-sense-tagged (PTST) tended sense of any occurrence of the target word data for a non-monosemous target word t (for has highest strength of association with its co- example, brilliant) used in a particular sense or occurring words. category c of the thesaurus (for example, ‘intel-

ligence’) may be generated as follows. Identify

¼ ¼

¾ X ´ ; µ = gj

jfT t : Sns2 T t c monosemous c-terms (for example, clever) be-

´ ; µ =

DE ;W t c

X j j t longing to the same category as c. Pick sentences

(4) containing the monosemous c-terms from an un-

¼ ¼

´ µ = ´ ; µ

Sns2 T ; t argmax ∑ A w c (5) tagged auxiliary text corpus.

¾ ´ µ c senses t w¾T Hermione had a clever plan. Metaphorically, words that co-occur with the target word give a weighted vote to each of its senses In each such sentence, replace the monosemous word with the target word t. In theory the c- just as in DI ;W . However, votes from co-occurring words in an occurrence are summed to determine terms in a thesaurus are near-synonyms or at least the intended sense (sense with the most votes) of strongly related words, making the replacement of the target word. The process is repeated for all one by another acceptable. For the sentence above, occurrences that have the target word. If each we replace clever with brilliant. This results in word that co-occurs with the target word votes as (artiﬁcial) sentences with the target word used in a sense corresponding to the desired category. described for DI ;U , then the following hypothesis Clearly, many of these sentences will not be lin- forms the basis of DE ;U : in a particular occurrence, the sense that gets the maximum votes from its guistically well formed, but the non-monosemous neighbors is the intended sense. c-term used in a particular sense is likely to have

similar co-occurring words as the monosemous c- ¼

¼ 2

¾ X ´ ; µ = gj

jfT t : Sns3 T t c term of the same category. This justiﬁes the use

´ ; µ =

DE ;U t c (6)

X j j t of these pseudo-thesaurus-sense-tagged data for

the purpose of evaluation.

¼ ¼ ¼

´ ; µ = ¾ ´ ; µ = gj

Sns3 T t argmax jfw T : Sns1 w t c We generated PTST test data for the head words

´ µ c ¾senses t 3 (7) in SENSEVAL-1 English lexical sample space us-

ing the Macquarie Thesaurus and the held out sub- ; In methods DE ;W and DE U , the dominance of BNC a sense is the proportion of occurrences of that set of the (every twelfth sentence). sense. 6 Experiments The degree of dominance provided by all four methods has the following properties: (i) The We evaluate the four dominance methods, like dominance values are in the range 0 to 1—a score McCarthy et al. (2004), through the accuracy of of 0 implies lowest possible dominance, while a a naive sense disambiguation system that always score of 1 means that the dominance is highest. gives out the predominant sense of the target word. (ii) The dominance values for all the senses of a In our experiments, the predominant sense is de- word sum to 1. termined by each of the four dominance methods, individually. We used the following setup to study 5 Pseudo-Thesaurus-Sense-Tagged Data the effect of sense distribution on performance. To evaluate the four dominance methods we would 2Strong collocations are an exception to this, and their effect must be countered by considering larger window sizes. ideally like sentences with target words annotated Therefore, we do not use a window size of just one or two with senses from the thesaurus. Since human an- words on either side of the target word, but rather windows notation is both expensive and time intensive, we of ¦5 words in our experiments. 3SENSEVAL-1 head words have a wide range of possible present an alternative approach of artiﬁcially gen- senses, and availability of alternative sense-tagged data may erating thesaurus-sense-tagged data following the be exploited in the future.

125 1 upper bound upper bound 1 upper bound upper bound DE,W (pmi, odds, Yule) DE,W (odds), bootstrapped 0.9 DI,W (pmi) 0.9 DI,W (odds), base DI,U (phi, pmi, Accuracy odds, Yule) Accuracy 0.8 0.8 DE,U (phi, pmi, odds, Yule) 0.7 0.7

0.6 0.6 baseline baseline 0.5 0.5 baseline baseline

0.4 0.4

0.3 0.3

0.2 0.2

0.1 lower bound lower bound 0.1 lower bound lower bound

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Distribution (alpha) Distribution (alpha) Mean distance below upper bound Mean distance below upper bound DE,W (pmi, odds, Yule): .02 DI,W (pmi): .03 DI,U (phi, pmi, odds, Yule): .11 DE,U (phi, pmi, odds, Yule): .16 DE,W (odds), bootstrapped: .02 DI,W (odds), base: .08 Figure 4: Best results: four dominance methods Figure 5: Best results: base vs. bootstrapped

6.1 Setup 6.2 Results For each target word for which we have PTST Highest accuracies achieved using the four dom- data, the two most dominant senses are identiﬁed, inance methods and the measures of association say s1 and s2. If the number of sentences annotated that worked best with each are shown in Figure 4.

with s1 and s2 is x and y, respectively, where x > y, The table below the ﬁgure shows mean distance then all y sentences of s2 and the ﬁrst y sentences below upper bound (MDUB) for all α values of s1 are placed in a data bin. Eventually the bin considered. Measures that perform almost iden- contains an equal number of PTST sentences for tically are grouped together and the MDUB val- the two most dominant senses of each target word. ues listed are averages. The window size used was

Our data bin contained 17,446 sentences for 27 ¦5 words around the target word. Each dataset nouns, verbs, and adjectives. We then generate dif- dα, which corresponds to a different target text in

ferent test data sets dα from the bin, where α takes Figure 2, was processed in less than 1 second on

;: ;:::;

values 0 ;:1 2 1, such that the fraction of sen- a 1.3GHz machine with 16GB memory. Weighted ; tences annotated with s1 is α and those with s2 is voting methods, DE ;W and DI W , perform best with

1 α. Thus the data sets have different dominance MDUBs of just .02 and .03, respectively. Yule’s values even though they have the same number of coefﬁcient, odds ratio, and pmi give near-identical, sentences—half as many in the bin. maximal accuracies for all four methods with a

Each data set dα is given as input to the naive slightly greater divergence in DI ;W , where pmi sense disambiguation system. If the predominant does best. The φ coefﬁcient performs best for sense is correctly identiﬁed for all target words, unweighted methods. Dice and cosine do only then the system will achieve highest accuracy, slightly better than the baseline. In general, re-

whereas if it is falsely determined for all target sults from the method–measure combinations are : words, then the system achieves the lowest ac- symmetric across α = 0 5, as they should be. curacy. The value of α determines this upper Marked improvements in accuracy were

bound and lower bound. Ifα is close to 0:5, then achieved as a result of bootstrapping the WCCM even if the system correctly identiﬁes the predom- (Figure 5). Most of the gain was provided by inant sense, the naive disambiguation system can- the ﬁrst iteration itself, whereas further iterations not achieve accuracies much higher than 50%. On resulted in just marginal improvements. All the other hand, if α is close to 0 or 1, then the bootstrapped results reported in this paper pertain system may achieve accuracies close to 100%. A to just one iteration. Also, the bootstrapped disambiguation system that randomly chooses one WCCM is 72% smaller, and 5 times faster at of the two possible senses for each occurrence of processing the data sets, than the base WCCM, the target word will act as the baseline. Note that which has many non-zero cells even though the no matter what the distribution of the two senses corresponding word and category never actually (α), this system will get an accuracy of 50%. co-occurred (as mentioned in Section 3.2 earlier).

126 6.3 Discussion 7 Related Work Considering that this is a completely unsupervised The WCCM has similarities with latent semantic approach, not only are the accuracies achieved us- analysis, or LSA, and speciﬁcally with work by ing the weighted methods well above the baseline, Sch¨utze and Pedersen (1997), wherein the dimen- but also remarkably close to the upper bound. This sionality of a word–word co-occurrence matrix is is especially true for α values close to 0 and 1. The reduced to create a word–concept matrix. How- lower accuracies for α near 0.5 are understandable ever, there is no non-heuristic way to determine as the amount of evidence towards both senses of when the dimension reduction should stop. Fur- the target word are nearly equal. ther, the generic concepts represented by the re- Odds, pmi, and Yule perform almost equally duced dimensions are not interpretable, i.e., one well for all methods. Since the number of times cannot determine which concepts they represent two words co-occur is usually much less than in a given sense inventory. This means that LSA the number of times they occur individually, pmi cannot be used directly for tasks such as unsuper- tends to approximate the logarithm of odds ra- vised sense disambiguation or determining seman- tio. Also, Yule is a derivative of odds. Thus all tic similarity of known concepts. Our approach three measures will perform similarly in case the does not have these limitations. co-occurring words give an unweighted vote for Yarowsky (1992) uses the product of a mutual the most appropriate sense of the target as in DI ;U

information–like measure and frequency to iden- ; and DE ;U . For the weighted voting schemes, DI W tify words that best represent each category in the and DE ;W , the effect of scale change is slightly Roget’s Thesaurus and uses these words for sense higher in DI ;W as the weighted votes are summed over the complete text to determine dominance. In disambiguation with a Bayesian model. We im- proved the accuracy of the WCCM using sim-

DE ;W the small number of weighted votes summed to determine the sense of the target word may be ple bootstrapping techniques, used all the words the reason why performances using pmi, Yule, and that co-occur with a category, and proposed four odds do not differ markedly. Dice coefficient and new methods to determine sense dominance— cosine gave below-baseline accuracies for a num- two of which do explicit sense disambiguation. ber of sense distributions. This suggests that the Véronis (2005) presents a graph theory–based ap- normalization4 to take into account the frequency proach to identify the various senses of a word in a of individual events inherent in the Dice and co- text corpus without the use of a dictionary. Highly sine measures may not be suitable for this task. interconnected components of the graph represent The accuracies of the dominance methods re- the different senses of the target word. The node main the same if the target text is partitioned as per (word) with the most connections in a component the target word, and each of the pieces is given in- is representative of that sense and its associations dividually to the disambiguation system. The av- with words that occur in a test instance are used as erage number of sentences per target word in each evidence for that sense. However, these associations are at best only rough estimates of the associ- dataset dα is 323. Thus the results shown above correspond to an average target text size of only ations between the sense and co-occurring words, 323 sentences. since a sense in his system is represented by a single (possibly ambiguous) word. Pantel (2005) We repeated the experiments on the base proposes a framework for ontologizing lexical re- WCCM after filtering out (setting to 0) cells with sources. For example, co-occurrence vectors for frequency less than 5 to investigate the effect on the nodes in WordNet can be created using the co- accuracies and gain in computation time (propor- occurrence vectors for words (or lexicals). How- tional to size of WCCM). There were no marked ever, if a leaf node has a single lexical, then once changes in accuracy but a 75% reduction in size the appropriate co-occurring words for this node of the WCCM. Using a window equal to the com- are identified (coup phase), they are assigned the plete sentence as opposed to ¦5 words on either same co-occurrence counts as that of the lexical.5 side of the target resulted in a drop of accuracies. 4If two events occur individually a large number of times, 5A word may have different, stronger-than-chance then they must occur together much more often to get sub- strengths of association with multiple senses of a lexical. stantial association scores through pmi or odds, as compared These are different from the association of the word with the to cosine or the Dice coefficient. lexical.

127 8 Conclusions and Future Directions References Eneko Agirre and O. Lopez de Lacalle Lekuona. 2003. We proposed a new method for creating a word– Clustering WordNet word senses. In Proceedings category co-occurrence matrix (WCCM) using a of the Conference on Recent Advances on Natural published thesaurus and raw text, and applying Language Processing (RANLP’03), Bulgaria. simple sense disambiguation and bootstrapping J.R.L. Bernard, editor. 1986. The Macquarie The- techniques. We presented four methods to deter- saurus. Macquarie Library, Sydney, Australia. mine degree of dominance of a sense of a word us- Lou Burnard. 2000. Reference Guide for the British ing the WCCM. We automatically generated sen- National Corpus (World Edition). Oxford Univer- tences with a target word annotated with senses sity Computing Services. from the published thesaurus, which we used to perform an extensive evaluation of the dominance Adam Kilgarriff and Colin Yallop. 2001. What’s in a thesaurus. In Proceedings of the Second Interna- methods. We achieved near-upper-bound results tional Conference on Language Resources and Eval-

using all combinations of the the weighted meth- uation (LREC), pages 1371–1379, Athens, Greece. ; ods (D ; and D ) and three measures of asso- I W E W Claudia Leacock, Martin Chodrow, and George A. ciation (odds, pmi, and Yule). Miller. 1998. Using corpus statistics and WordNet We cannot compare accuracies with McCarthy relations for sense identification. Computational et al. (2004) because use of a thesaurus instead Linguistics, 24(1):147–165. of WordNet means that knowledge of exactly how Dekang Lin. 1998. Automatic retrieval and clustering the thesaurus senses map to WordNet is required. of similar words. In Proceedings of the 17th Inter- We used a thesaurus as such a resource, unlike national Conference on Computational Linguistics WordNet, is available in more languages, pro- (COLING-98), pages 768–773, Montreal, Canada. vides us with coarse senses, and leads to a smaller Diana McCarthy, Rob Koeling, Julie Weeds, and John WCCM (making computationally intensive oper- Carroll. 2004. Finding predominant senses in ations viable). Further, unlike the McCarthy et untagged text. In Proceedings of the 42nd An- nual Meeting of the Association for Computational al. system, we showed that our system gives accu- Linguistics (ACL-04), pages 280–267, Barcelona, rate results without the need for a large similarly- Spain. sense-distributed text or retraining. The target texts used were much smaller (few hundred sen- Saif Mohammad and Graeme Hirst. Submitted. Dis- tributional measures as proxies for semantic related- tences) than those needed for automatic creation ness. of a thesaurus (few million words). Patrick Pantel. 2005. Inducing ontological co- The WCCM has a number of other applications, occurrence vectors. In Proceedings of the 43rd An- as well. The strength of association between a nual Meeting of the Association for Computational word and a word sense can be used to determine Linguistics (ACL-05), pages 125–132, Ann Arbor, the (more intuitive) distributional similarity of Michigan. word senses (as opposed to words). Conditional Hinrich Schütze and Jan O. Pedersen. 1997. A probabilities of lexical features can be calculated cooccurrence-based thesaurus and two applications from the WCCM, which in turn can be used in un- to information retreival. Information Processing supervised sense disambiguation. In conclusion, and Management, 33(3):307–318. we provided a framework for capturing distribu- David Sheskin. 2003. The handbook of paramet- tional properties of word senses from raw text and ric and nonparametric statistical procedures. CRC demonstrated one of its uses—determining word Press, Boca Raton, Florida. sense dominance. Jean Véronis. 2005. Hyperlex: Lexical cartography for information retrieval. To appear in Computer Acknowledgments Speech and Language. Special Issue on Word Sense Disambiguation. We thank Diana McCarthy, Afsaneh Fazly, and Suzanne Stevenson for their valuable feedback. David Yarowsky. 1992. Word-sense disambiguation using statistical models of Roget’s categories trained This research is financially supported by the Natu- on large corpora. In Proceedings of the 14th Inter- ral Sciences and Engineering Research Council of national Conference on Computational Linguistics Canada and the University of Toronto. (COLING-92), pages 454–460, Nantes, France.

128