IndoWordnet Pushpak Bhattacharyya Department of Computer Science and Engineering Indian Institute of Technology Bombay E-mail: [email protected] Abstract India is a multilingual country where machine translation and cross lingual search are highly relevant problems. These problems require large resources- like wordnets and lexicons- of high quality and coverage. Wordnets are lexical structures composed of synsets and semantic relations. Synsets are sets of synonyms. They are linked by semantic relations like hypernymy (is-a) , meronymy (part-of), troponymy (manner-of) etc. IndoWordnet is a linked structure of wordnets of major Indian languages from Indo-Aryan, Dravidian and Sino-Tibetan families. These wordnets have been created by following the expansion approach from Hindi wordnet which was made available free for research in 2006. Since then a number of Indian languages have been creating their wordnets. In this paper we discuss the methodology, coverage, important considerations and multifarious benefits of IndoWordnet. Case studies are provided for Marathi, Sanskrit, Bodo and Telugu, to bring out the basic methodology of and challenges involved in the expansion approach. The guidelines the lexicographers follow for wordnet construction are enumerated. The difference between IndoWordnet and EuroWordnet also is discussed. matrix rows represent word meanings and columns the 1. Introduction forms . For example, in Table 1, the column F2 shows Wordnets have emerged as crucial resources for Natural different meanings of bank , i.e. , the polysemy of bank , Language Processing (NLP). Wordnets are lexical while the rows M1 and M2 show different synonyms of bank . structures composed of synsets and semantic relations (Fellbaum, 1998). Synsets are sets of synonyms. They are linked by semantic relations like hypernymy (is-a) , Word Word Forms meronymy (part-of), troponymy (manner-of) etc. The first Meanings F1 F2 F3 … F k wordnet in the world was built for English at Princeton M1 depend bank Rely University 1 . Then followed wordnets for European M2 bank embankment Languages: Eurowordnet 2 (Vossen, 1998). Since 2000, M3 wordnets for a number of Indian languages are getting … bank 3 built, led by the Hindi wordnet (Narayan et. al. , 2001) Mn effort at Indian Institute of Technology Bombay 4 (IITB). Table 1: Lexical matrix showing the word bank In wordnet creation, the focus shifts from words to It is clear from the presence of other words in the same concepts. For example, सूयϕ (Sun ), पृवी (Earth ), जल , पानी row ( e.g. , depend in M1 and embankment in M2) what (Water ) etc. are very common concepts . After selecting a these meanings or senses are. This is the principle of concept, all the words standing for that concept are stored relational semantics. Words when put together in a as the set of synonymous words. common set disambiguate each other. Such sets are In what follows we first describe the general known as synsets. methodology used in wordnet construction in section 2. There are three principles the synset construction The points made therein are substantiated through a case process must adhere to. Minimality principle insists on study of Hindi and Marathi wordnets construction in capturing that minimal set of the words in the synset section 3. Section 4 is on the process details of which uniquely identifies the concept. For example IndoWordnet construction. Section 5 describes the {family, house} uniquely identifies a concept ( e.g. “he is experiences of a few Indian languages in expanding from from the house of the King of Jaipur”} . Coverage Hindi wordnet. Section 6 enumerates some guiding principle then stresses on the completion of the synset, i.e., capturing ALL the words that stand for the concept principles of IndoWordnet construction. Section 7 is on expressed by the synset ( e.g., {family, house, household, difference between IndoWordnet (IWN) and ménage} completes the synset ). Within the synset the EuroWordnet (EWN). Section 8 concludes the paper and words should be ordered according their frequency in the points to future directions. corpus. Replaceability demands that the most common words in the synset, i.e., words towards the beginning of 2. General methodology for wordnet the synset should be able to replace one another in the creation example sentence associated with the synset. The foundation of wordnet construction is relational Wordnets are constructed by following either the merge semantics (Cruse, 1986) . Words and concepts can be approach or the expansion approach (Vossen, 1998). In looked upon as forming entries in a structure called the the former- which can be said to be wordnet construction Lexical Matrix. Table 1 illustrates this. In the lexical from first principles- exhaustive sense repository of each word is first recorded. Then the lexicographers constructs 1 a synset for each sense, obeying the above three principles. http://www.wordnet.princeton.edu In the expansion approach, the synsets of the wordnet of a 2 http:// http://www.illc.uva.nl/EuroWordNet/ 3 given source language LS are provided. Each synset is http://www.cfilt.iitb.ac.in/wordnet/webhwn carefully studied for its meaning. Then the words of the 4 http://www.iitb.ac.in target language LT, representing that meaning are ‘tree’ . Hindi and Marathi being close members of the collected and put together in a set in frequency order. same language family, many Hindi words have the same meaning in Marathi. This is especially so for tatsam 2.1 Comparing merge and expansion words, which are directly borrowed from Sanskrit. The approaches to wordnet building semantic relations can be transferred directly, thus saving time and effort. Both the merge and expansion approaches have their advantages and disadvantages. In the former, there is no HWN entry : distracting influence of another language, which happens {पेड़ , वृϓ , पादप , Lुम , तď , िवटप , Đϓ , Đख , अिMप , तďवर } when the lexicographer encounters culture and region ‘tree ’ specific concepts of the source language. The quality of peR, vriksh, paadap, drum, taru, viTap, ruuksh, ruukh, the wordnet is good, provided the synset maker is well 5 adhrip, taruvar versed with the nuances of the language. But the process synset is typically slow. In the latter approach, the whole wordnet making process is well guided in the sense of जड़ , ताना , शाखा , तथा पिēयĪ से युŎ बćवषĖय वनपित following the synsets of the source language. Also it has jaR,tanaa, shaakhaa, tathaa pattiyo se yukt bahuvarshiya the advantage of being able to borrow the semantic vanaspati ‘perennial woody plant having root, stem, relations of the given wordnet. This saves an enormous branches and leaves ’ amount of time. However, the lexicographer oftentimes is Gloss distracted by synsets standing for highly culture and region specific concepts. Also common is the problem of peR manushya ke lie bahut hi upayogii hai not finding the target language’s “own concepts” . पेड़ मनुय के िलए बćत ही उपयोगी है ‘trees are very One finds the predominance of the expansion useful to men ’ approach in the wordnet building community. Many Example sentence concepts are common across languages. Creating synsets for these universal concepts should be the first step in the MWN entry : construction of any wordnet. If a language has already { झाड , वृϓ , तďवर , Lुम , तĐ , पादप } ‘tree ’ done this job, it makes sense to leverage from this work. jhaaR, vriksh, taruvar, drum, taruu, paadap This fact and the fact of being able to borrow the semantic relations from the source language tilt the balance in मुले, खोड़, फाघा , पाने इयादीनी योŎ असा वनपितिवशेष favour of the expansion approach. If the source and target mule, khoR, phaanghaa, pane ityaadiinii yokt asaa languages happen to have strong kinship relationship, the vanaspativishesh ‘perennial woody plant having root, expansion approach becomes all the more attractive, since stem, branches and leaves ’ distracting influences of culture and region specific concepts is minimal in this case. ती दमून झाडाϢया सावलीत बसली tii damuun jhaadacyaa In the next section, we present a case study to saavlit baslii ‘Being exhausted she sat under the shadow elucidate the above ideas. of the tree’ 3. A case study: creation of Hindi Figure 1: MWN synset creation from HWN wordnet (HWN) and Marathi wordnet (MWN) 3.1 Synset making We follow Chakrabarty et. al. (2007) in this section. We The principles of minimality , coverage and replaceability have, for long, been engaged in building lexical resources govern the creation of the synsets: for Indian languages with focus on Hindi and Marathi (http://www.cfilt.iitb.ac.in). The Hindi and Marathi (i) Minimality : Only the minimal set that uniquely wordnets (HWN and MWN) and the Hindi Verb identifies the meaning is first used to create the sysnet, Knowledge Base (HVKB) (Chakrabarty et. al ., 2007) e.g. , have been given special attention. The wordnets more or {ghar, kamaraa} (room) less follow the design principles of the Princeton Wordnet ghar- which is ambiguous- is not by itself sufficient to for English while paying particular attention to language denote the concept of a room. The addition of kamaraa to specific phenomena (such as complex predicates ) the synset brings out this meaning uniquely. whenever they arise. While HWN has been created by manually looking (ii) Coverage : Next, the synset should contain all the up the various listed meanings of words in different words denoting a particular meaning. The words are listed dictionaries, MWN has been created by expansion from in order of decreasing frequency of their occurrence in the HWN. That is, the synsets of HWN are adapted to MWN corpus. via addition or deletion of synonyms in the synset. {ghar, kamaraa, kaksh} (room) Figure 1 shows the creation of the synset for the word peR ‘tree’ in MWN via addition and deletion of synonyms (iii) Replaceability : The words forming the synset should from HWN.