A morphological processor based on foma for Biscayan (a Basque dialect) Iñaki Alegria1, Garbiñe Aranbarri2, Klara Ceberio3, Gorka Labaka1, Bittor Laskurain 2, Ruben Urizar1 1 IXA research group. University of the Basque Country. 649 PK 20080 Donostia 2 Eleka S.L. Zelai Haundi kalea, 3 - Osinalde industrialdea - 20170 Usurbil 3 Elhuyar Foundation. Zelai Haundi kalea, 3 - Osinalde industrialdea - 20170 Usurbil E-mail: [email protected] Abstract We present a new morphological processor for Biscayan, a dialect of Basque, developed on the description of the morphology of standard Basque. The database for the standard morphology has been extended for dialects and an open- source tool for morphological description named foma is used for building the processor. XuxenB, a spelling checker/corrector for this dialect, is the first application of this work. 1. Basque and Biscayan morphology foma is licensed/certified under the GNU license: in Basque is an agglutinative language with rich keeping with traditions of free software, the distribution morphology. Standard Basque morphology was that includes the source code comes with a user manual described by Alegria et al. (1995; 2002) using finite-state and a library of examples. The compiler and library are morphology. implemented in C and an API is available (foma. The Biscayan dialect of Basque (Arejita et al., 2002, sourceforge.net). 2005), also called Western Basque (Zuazo, 2008), is a Standard Basque morphology was migrated to open- dialect of the Basque language spoken in the western source finite-state technology using hunspell (Nemeth et part of the Basque speaking area, mainly in the province al., 2004) first and then to foma (Alegria et al., 2009). In of Biscay, but also in southwest Guipuzcoa and the the last case, it is worth pointing out the process of Basque speaking areas of Álava. Although it is the most rewriting parallel two-level phonological rules into widespread of Basque dialects, it differs considerably sequential replacement rules. Due to the fact that in the from standard Basque, heavily based on the Guipuzcoan context of the sequential rules it is not necessary to take dialect. in account the rest of the rules, the use of sequential While the standard written Basque is used in all the replacement rules increases flexibility in the description, levels of education and the media intended for the whole which allows to include linguistic variants. In contrast, Basque speaking population, there is an increasing special attention must be taken to place the rules in the interest in using Biscayan in local media and informal correct order.. communication forums (blogs, chats, phones). It is quite difficult to calculate the number of Biscayan speakers, as # parallel rules the majority of surveys done regarding the use of Basque R->r ; +->0 ; language focus mainly on Basque speakers as a whole. h->0 || # b e R:r 0:r +:0 _ Vowel; Nevertheless, according to the estimate made by the 0->r || R:r _ +:0 (h:0) Vowel; Labayru institute (www.labayru.org), a cultural e->i || # b _ R +:0 Cons; institution working for the promotion of Biscayan, speakers of this dialect amount to about 250,000. # sequential rules h->0 || # b e R + _ Vowel; 2. foma 0->r || R _ + Vowel; e->i || # b _ R + Cons; foma (Hulden, 2009) is a finite-state tool designed for R->r ; +->0 ; multi-purpose use with explicit support for automata theory research, constructing lexical analyzers for Table 1. Simplified comparison between parallel and programming languages, and building morpho- sequential rules in Basque morphology. phonological analyzers, as well as spell-checking applications. In Table 1 we provide an example of the differences The compiler allows users to specify finite-state between both descriptions. The example rules illustrate a automata and transducers incrementally in a similar number of phenomena. In Basque, the prefix ber- fashion to AT&T’s fsm, Sproat’s Lextools, the (equivalent to English 're-') mostly precedes verbal Xerox/PARC finite-state toolkit and the SFST toolkit. stems. Before a vowel, the final character of this prefix One of foma’s design goals has been compatibility with (r) is doubled; this is expressed by the capital R (beR). the Xerox/PARC toolkit (Beesley and Karttunen, 2003). 828 When preceding a consonant the prefix ber- changes into house'), verb participles ending in –atu become -au in bir-. In addition to this, if the first character of the verb their dialectal forms (errezau / errezatu 'to pray') and the stem is h, this h is erased. Thus, the chain beR+egin nominal form of these verbs takes –eta along with the (lexical expression) generates berregin 'redo', while standard -t(z)e (errezetan / errezatzen 'praying'). beR+gai+tu generates birgaitu 'rehabilitate', and In order to develop the lexicon for the Biscayan dialect beR+has+i changes into berrasi 'restart'. we used the standard version of the analyzer. This was applied on a list of 2775 words made up by the Labayru 3. The database for Biscayan Institute from a list of dictionary entries and a small The description of the lexicon and the morphotactics (or corpus written in the dialect. On the one hand, the word grammar) for standard Basque was carried out standard analyzer could not identify 1984 correct using a relational database (Aduriz et al., 1998). Biscayan words in the list, and, on the other hand, 771 Currently, around 100,000 entries are recorded for the words in the list, which were not acceptable in the description of standard Basque, divided into (a) lexical dialect, were analyzed as correct since there are entries (b) inflectional and derivational morphemes and acceptable in standard Basque. (c) inflected verb forms. The lexical description needed for the morphological analysis of text words is obtained 4. The morphophonological rules for using an export process from the database. Biscayan The database for the Biscayan dialect was built based on It has been possible to adapt the morphophonological the one for standard Basque. When new dialectal variants rules for standard Basque to the ones for Biscayan using were added, they were linked to the standard entries, sequential replacement rules, without changing the which now became non-standard variants in the dialectal original ones. database. For example, the Biscayan verb form gagoz Ten new sequential rules have been added, most of them ('we are/stay') is stored in the Biscayan database linked close to the lexicon (in the beginning of the rules system to gaude, its corresponding entry in standard Basque, before the standard rules) and only one rule in the bottom which, at the same time, is non-standard in the dialect. part of the rules system. Most of these rules deal with These links will be exploited when we want to obtain the several changes in vowels (vowel harmonization) and in standard form from the dialectal one or vice versa. This sibilants (fricativization of affricates) in the morpheme is very useful for indexation, spelling correction and border. other applications. Two new features have been used: one (5 is used for this Most of the new entries (1,661) correspond to inflected feature) intended to give an account of the verb forms, since the main difference between both morphophonological modification taking place when the variants of the language are auxiliary verbs. For instance, singular article is added to a word ending in a (a + a = the triadic auxiliary verb forms for indicative in Biscayan ea) as in alabea, the 'the daughter' (alabA + 5a) and take the -eu(t)s- stem while in standard Basque they take another one for a special ending in verb roots. -i- e.g. ekarri deutso / dio (brought have-it-to_him/her- In Table 2 an example of the new rules is shown. he/she; 'she has brought it to her'). As for lexical entries (nouns, adjectives, adverbs, lexical # final A is realized as e verbs, determiners and pronouns), many of the standard # in Biscayan before words belong to the common core of the language. # morpheme border (MM), However, over 900 new lexical entries have been added # 5 feature and open vowel to the database for Biscayan. Most of the variations are A -> e || \i _ MM 5 (E) OpenVowel ; due to phonological phenomena such as i/u alternations # alabA+5a:alabea (huri / hiri 'city', gitxi / gutxi 'little', ule / ile 'hair'), loss of intervocalic nasal in –sio/-sino or -zio/-zino suffix Table 2. Example of additional rules for Byscayan. (telebisio / telebisino 'television', akusazio / akusazino 'accusation') or final -a / -e alternations (lora / lore An analyzer was built Using foma, and the 'flower', laba / labe 'oven', hoba / hobe 'better'). morphological description was tested and debugged For those standard entries which do not have an based on a corpus of Biscayan. equivalent in Biscayan, we decided to keep them in order After refining the lexicon and the rules several times in to improve coverage. It is very common to find this kind an iterative way, 93.95% of the dialectal words that the of words in dialectal texts due to the increasing influence standard processor could not analyze were recognised of standard Basque over the rest of the dialects especially now, and 97.92% of the non-Biscayan words that the in written texts. standard processor analyzed as correct were now Regarding inflectional morphemes, just a few changes discarded. Thus, average precision of the analyzer on had to be made. For the sociative case ('with'), the -gaz these problematic words improved up to 95.06%. morpheme is used in the Biscayan dialect along with the standard -rekin (amarekin / amagaz 'with mum'), -runtz 5. Tools and application ('towards') is also acceptable together with the the In addition to the basic analyzer used for spelling, a new standard -rantz (etxerantz / etxeruntz 'towards the transducer has been included (Alegria et al., 2002).