DOCSLIB.ORG

Morphosyntactic Disambiguation in an Endangered Language Setting

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Morphosyntactic Disambiguation in an Endangered Language Setting Morphosyntactic Disambiguation in an Endangered Language Setting Jeff Ens♠ Mika Ham¨ al¨ ainen¨ ♦ Jack Rueter♦ Philippe Pasquier♠ ♠ School of Interactive Arts & Technology, Simon Fraser University ♦ Department of Digital Humanities, University of Helsinki [email protected], [email protected], [email protected], [email protected] Abstract available in the infrastructure, as they are not free Endangered Uralic languages present a of errors (Ham¨ al¨ ainen¨ et al., 2018; Ham¨ al¨ ainen,¨ high variety of inflectional forms in their 2018). morphology. This results in a high num- This paper presents a method to learn the mor- ber of homonyms in inflections, which in- phosyntax of a language on an abstract level by troduces a lot of morphological ambigu- learning patterns of possible morphologies within ity in sentences. Previous research has sentences. The resulting models can be used employed constraint grammars to address to evaluate the existing rule-based disambigua- this problem, however CGs are often un- tors, as well as to directly disambiguate sentences. able to fully disambiguate a sentence, and Our work focuses on the languages belonging to their development is labour intensive. We the Finno-Permic language family: Finnish (fin), present an LSTM based model for auto- Northern Sami (sme), Erzya (myv) and Komi- matically ranking morphological readings Zyrian (kpv). The vitality classification of the of sentences based on their quality. This three latter languages is definitely endangered ranking can be used to evaluate the exist- (Moseley, 2010). ing CG disambiguators or to directly mor- phologically disambiguate sentences. Our 2 Motivation approach works on a morphological ab- There are two main factors motivating this re- straction and it can be trained with a very search. First of all, data is often very scarce when small dataset. dealing with endangered Uralic langauges. Apart 1 Introduction from Northern Sami, other endangered Uralic lan- guages may have a very small set of annotated Most of the languages in the Uralic language fam- samples at best, and no gold standard data at worst. ily are endangered. The low number of speak- As a result, evaluating disambiguated sentences ers, limited linguistic resources and the vast com- can often only be conducted by consulting native plexity in morphology typical to these languages speakers of the language or by relying on the re- makes their computational processing quite a chal- searcher’s own linguistic intuition. lenge. Over the past years, a great deal of work Secondly, canonical approaches involving Part- related to language technology for endangered of-Speech (POS) tagging will not suffice in this Uralic languages has been released openly on context due to the rich morphology of Uralic lan- the Giellatekno infrastructure (Moshagen et al., guages. For example the Finnish word form voita 2014). This includes lexicographic resources, FST can be lemmatized as voi (the singular partitive of (finite-state transducer) based morphological ana- butter), vuo (the plural partitive of fjord), voittaa lyzers and CG (constraint grammar) disambigua- (the imperative of win) or voitaa1 (the connegative tors. form of spread butter). Despite being a great resource, the Giellatekno The approach described in this paper, addresses infrastructure has tools and data originating from these two issues, as we use a generalized sen- different sources by different authors. Recent re- tence representation based on morphological tags search conducted with the resources for Komi- to capture morphological patterns. Moreover, our Zyrian, Skolt Sami, Erzya and Moksha has identi- fied a need for proper evaluation of the resources 1A non-standard form produced by the Finnish analyzer 345 Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 345–349 Turku, Finland, 30 September – 2 October, c 2019 Linköping University Electronic Press models can be trained on low resource langauges, biguation is needed in order to improve the per- and models that have been trained on high re- formance of higher-level NLP tools. In addition, source languages can be applied to low or no re- we do not want to assume bi-lingual parallel data source languages with reasonable success. or access to word embeddings as we want our ap- proach to be applicable for truly endangered lan- 3 Related Work guages with extremely limited resources. The problem of morphological tagging in the 4 The Rule-based Tools and Data context of low-resource languages has been ap- proached using parallel text (Buys and Botha, We use the morphological FST analyzers in the 2016). From the aligned parallel sentences, their Gieallatekno infrastructure to produce morpholog- Wsabie-based model can learn to tag the low- ical readings with UralicNLP (Ham¨ al¨ ainen,¨ 2019). resource language based on the morphological They operate on a word level. This means that tags of the high-resource language sentences in for an input word form, they produce all the pos- the training data. A limitation of this approach is sible lemmas together with their parts-of-speech the morphological relatedness of the high-resource and morphological readings, without any weights and low-resource languages. to indicate which reading is the most probable one. A method for POS tagging of low-resource The existing CG disambiguators get the mor- languages has been proposed by Andrews et al. phological readings produced by the FST for each (2017). They use a bi-lingual dictionary between word in a sentence and apply their rules to re- a low and high-resource language together with move the non-possible readings. In some cases, monolingual data to build cross-lingual word em- a CG disambiguator might produce a fully disam- beddings. The POS tagger is trained on an LSTM biguated sentence, however these models are often neural network, and their approach performs con- unable to resolve all morphological ambiguity. sistently better than the other benchmarks they re- In this paper, we use the UD Treebanks for our port. languages of interest. For Finnish, we use Turku Lim et al. (2018) present work conducted on Dependency Treebank (Haverinen et al., 2014) syntactically parsing Komi-Zyrian and Northern with 202K tokens (14K sentences). The Northern Sami using multilingual word-embeddings. They Sami Treebank (Sheyanova and Tyers, 2017) is the use pretrained word-embeddings for Finnish and largest one for the endangered languages with 26K Russian, and train word-embeddings for the low- tokens (3K sentences). For Komi-Zyrian, we use resource languages from small corpora. These in- the Komi-Zyrian Lattice Treebank (Partanen et al., dividual word-embeddings are then projected into 2018) of 2K tokens (189 sentences) representing a single space by using bilingual dictionaries. The the standard written Komi. The Erzya Treebank parser was implemented as an LSTM model and it (Rueter and Tyers, 2018) is the second largest en- performed better in a POS tagging task than in pre- dangered language one we use in our research with dicting syntactic relations. The key finding for our 15k tokens (1,500 sentences). purposes is that including a related high-resource 5 Sentence Representation language (Finnish in this case) improved the accu- racy. We represent each word as a non-empty set of DsDs (Plank and Agic,´ 2018) is a neural net- morphological tags. This representation does not work based POS tagger for low-resource lan- contain the word form itself nor its lemma, as guages. The idea is to use a bi-LSTM model to we aim for a more abstract level morphologi- project POS tags from one language to another cal representation. This representation is meant with the help of word-embeddings and lexical in- to capture the possible morphologies following formation. In a low-resource setting, they find that each other in a sentence to learn morphosyntactic adding word-embeddings boosts the model, but inter-dependencies such as agreement rules. This lexical information can also help to a smaller de- level of abstraction makes it possible to apply the gree. learned structures for other morphosyntactically Much of the related work deals with POS tag- similar languages. ging. However, as the Uralic languages are mor- As we are looking into morphosyntax, we train phologically rich, a full morphological disam- our model only with the morphosyntactically rele- 346 vant morphological tags. These are case, number, 1.0 voice, mood, person, tense, connegative and verb form. This means that morphological tags such as 0.9 clitics and derivational morphology are not taken into account. We are also ignoring the dependency 0.8 information in the UD Treebanks as dependencies are not available for text and languages outside of 0.7 the Treebanks due to the fact that there are no ro- 0.6 bust dependency parsers available for many of the endangered Uralic language. 0.5 Gold Standard Sentence in Top-K Frequency 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Each sentence is simply a sequence of mor- K phological tag sets, represented as a sequence kpv sme fin myv of integers with a special token SP demar- cating spaces between words. For exam- Figure 1: The frequency with which the gold stan- ple the sentence ”Nyt on lungisti ottamisen dard sentence is ranked in the top-k with 1000 tri- aika.” (now it is time to relax), is encoded als per model averaged over 10 data splits. as [150, SP, 121, 138, 168, 178, 205, 214, 221, SP 150, SP, 25, 138, 158, SP, 31, 138, 158, SP, 165]. 7 Evaluation Equation 1 is used to measure the distance be- tween two sentences containing n words, where xi We produce all the morphological readings for denotes set of morphological tags associated with each word in a gold standard sentence (GSS) using the ith word in x, denotes the number of FST analyzers, and construct incorrect sentences || · || elements in a set, and denotes the symmetric (INS) of varying quality by randomly selecting 4 difference of two sets.
Load more

Recommended publications
  • Annotation Schemes in North Sami Dependency Parsing
    Annotation schemes in North Sámi dependency parsing Mariya Sheyanova Fundamental and Applied Linguistics Higher School of Economics Moscow, Russia [email protected] Francis M. Tyers Giela ja kultuvrra instituhtta UiT Norgga árktalaš universitehta N-9018 Romsa, Norway [email protected] Abstract In this paper we describe a comparison of two annotation schemes for de- pendency parsing of North Sámi, a Finno-Ugric language spoken in the north of Scandinavia and Finland. The two annotation schemes are the Giellatekno (GT) scheme which has been used in research and applications for the Sámi languages and Universal Dependencies (UD) which is a cross-lingual scheme aiming to unify annotation stations across languages. We show that we are able to deterministi- cally convert from the Giellatekno scheme to the Universal Dependencies scheme without a loss of parsing performance. While we do not claim that either scheme is a priori a more adequate model of North Sámi syntax, we do argue that the choice of annotation scheme is dependent on the intended application. This work is licensed under a Creative Commons Attribution–NoDerivatives 4.0 International Licence. Licence details: http://creativecommons.org/licenses/by-nd/4.0/ 1 66 The 3rd International Workshop for Computational Linguistics of Uralic Languages, pages 66–75, St. Petersburg, Russia, 23–24 January 2017. c 2017 Association for Computational Linguistics 1 Introduction Dependency parsing is an important step in many applications of natural language processing, such as information extraction, machine translation, interactive language learning and corpus search interfaces. There are a number of approaches to depen- dency parsing.
    [Show full text]
  • Word Classes and Part-Of-Speech 5 Tagging
    Speech and Language Processing: An introduction to speech recognition, computational linguistics and natural language processing. Daniel Jurafsky & James H. Martin. Copyright c 2006, All rights reserved. Draft of July 30, 2007. Do not cite without permission. WORD CLASSES AND PART-OF-SPEECH 5 TAGGING Conjunction Junction, what’s your function? Bob Dorough, Schoolhouse Rock, 1973 A gnostic was seated before a grammarian. The grammarian said, ‘A word must be one of three things: either it is a noun, a verb, or a particle.’ The gnostic tore his robe and cried, “Alas! Twenty years of my life and striving and seeking have gone to the winds, for I laboured greatly in the hope that there was another word outside of this. Now you have destroyed my hope.’ Though the gnostic had already attained the word which was his purpose, he spoke thus in order to arouse the grammarian. Rumi (1207–1273), The Discourses of Rumi, Translated by A. J. Arberry Dionysius Thrax of Alexandria (c. 100 B.C.), or perhaps someone else (exact au- thorship being understandably difficult to be sure of with texts of this vintage), wrote a grammatical sketch of Greek (a “techne¯”) which summarized the linguistic knowl- edge of his day. This work is the direct source of an astonishing proportion of our modern linguistic vocabulary, including among many other words, syntax, diphthong, PARTS­OF­SPEECH clitic, and analogy. Also included are a description of eight parts-of-speech: noun, verb, pronoun, preposition, adverb, conjunction, participle, and article. Although ear- lier scholars (including Aristotle as well as the Stoics) had their own lists of parts-of- speech, it was Thrax’s set of eight which became the basis for practically all subsequent part-of-speech descriptions of Greek, Latin, and most European languages for the next 2000 years.
    [Show full text]
  • Analysing Constraint Grammar with SAT I L
    T D L P Analysing Constraint Grammar with SAT I L Department of Computer Science and Engineering C U T U G Gothenburg, Sweden 2016 Analysing Constraint Grammar with SAT I L © Inari Listenmaa, 2016. Technical Report 154L ISSN 1652-876X Research groups: Functional Programming / Language Technology Department of Computer Science and Engineering C U T U G SE-412 96 Gothenburg Sweden Telephone +46 (0)31-772 1000 Typeset in Palatino and Monaco by the author using X TE EX Printed at Chalmers Gothenburg, Sweden 2016 Abstract Constraint Grammar (CG) is a robust and language-independent formalism for part-of-speech tagging and shallow parsing. A grammar consists of disambiguation rules for initially am- biguous, morphologically analysed text: the correct analysis for the sentence is attained by removing improper readings from ambiguous words. Wide-coverage Constraint Grammars, containing some thousands of rules, generally achieve very high accuracy for free text: thanks to the flexible formalism, new rules can be written to address even the rarest phenomena, without compromising the general tendencies. Even with a smaller set of rules, CG-based taggers have been competetive with statistical taggers. In addition, CG has been used in more experimental settings: dependency syntax, dialogue system for language learning and anaphora resolution. This thesis presents two contributions to the study of CG. Firstly, we model a parallel variant of CG as a Boolean satisfiability (SAT) problem, and implement a CG-engine by using a SAT-solver. This is attractive for several reasons: formal logic is well-studied, and serves as an abstract language to reason about the properties of CG.
    [Show full text]
  • When Grammar Can't Be Trusted
    Department of Language and Culture (ISK) When grammar can’t be trusted - Valency and semantic categories in North Sámi syntactic analysis and error detection — Linda Wiechetek A dissertation for the degree of Philosophiae Doctor – December 2017 When grammar can’t be trusted - Valency and semantic categories in North Sámi syntactic analysis and error detection Linda Wiechetek A dissertation for the degree of Philosophiae Doctor Department of Language and Culture (ISK) December 2017 Für meine Eltern & buot sámegiela oahpahalliide Contents I Beginning 3 1 Introduction 7 2 Background and methodology 13 2.1 Theoretical background: Valency theory . 14 2.1.1 Syntactic valency . 16 2.1.1.1 Obligatoriness . 17 2.1.1.2 Syntactic tests . 19 2.1.2 Selection restrictions and semantic prototypes . 21 2.1.3 Semantic valency . 23 2.1.3.1 Semantic roles vs. syntactic functions . 24 2.1.3.2 Semantic roles vs. referential semantics . 25 2.1.4 Semantic verb classes . 25 2.1.5 Criteria for potential governors . 27 2.2 Valency theory in Sámi research . 28 2.2.1 Case and valency . 28 2.2.2 Rection and valency . 30 2.2.3 Transitivity and valency . 32 2.2.4 Syntactic valency . 34 2.2.5 Governors . 35 2.2.6 Selection restrictions . 36 2.2.7 Semantic valency . 38 2.3 Human-readable and machine-readable valency resources . 42 2.3.1 Human-readable valency resources . 42 2.3.2 Machine-readable valency resources . 43 2.4 Methodology and framework . 46 2.4.1 Methodology . 46 2.4.2 Framework . 50 2.4.2.1 The Constraint Grammar formalism .
    [Show full text]
  • A Constraint Grammar Parser for Spanish
    Proceedings of the International Joint Conference IBERAMIA/SBIA/SBRN 2006 - 4th Workshop in Information and Human Language Technology (TIL’2006), Ribeir˜aoPreto, Brazil, October 23–28, 2006. CD-ROM. ISBN 85-87837-11-7 A Constraint Grammar Parser for Spanish Eckhard Bick Institute of Language and Communication, University of Southern Denmark [email protected] Abstract. In this paper we describe and evaluate a Constraint Grammar parser for Spanish, HISPAL. The parser adopts the modular architecture of the Portuguese PALAVRAS parser, and in a novel porting approach, the linguist- written Portuguese CG rules for morphological and syntactic disambiguation were ªcorrectedº and appended for Spanish in a corpus-based fashion, rather than rewritten from scratch. As part of the 5 year project, a 74.000 lexeme lexicon was developed, as well as a morphological analyzer and semantic ontology for Spanish. An evaluation of the the system©s tagger/parser modules indicated F-scores of 99% for part-of-speech tagging and 96% for syntactic function assignment. HISPAL has been used for the grammatical annotation of 52 million words of text, including the Europarl and Wikipedia text collections. Keywords: Spanish parser, Constraint Grammar, NLP, Corpus annotation 1 Introduction As results for several languages have shown (e.g. English [7], Portuguese [3], French [5], Danish [4] and Norwegian [6]), Constraint Grammar (CG) based NLP systems can achieve a high level of robustness and accuracy in the annotation of running text. However, as rule-based systems, they normally demand not only a full lexicon-based morphological analysis as input, but also a large linguist-written disambiguation grammar.
    [Show full text]
  • Finnpos: an Open-Source Morphological Tagging and Lemmatization Toolkit for Finnish
    Noname manuscript No. (will be inserted by the editor) FinnPos: An Open-Source Morphological Tagging and Lemmatization Toolkit for Finnish Miikka Silfverberg · Teemu Ruokolainen · Krister Lindén · Mikko Kurimo the date of receipt and acceptance should be inserted later Abstract This paper describes FinnPos, an open-source morphological tagging and lemmatization toolkit for Finnish. The morphological tagging model is based on the averaged structured perceptron classifier. Given training data, new taggers are esti- mated in a computationally efficient manner using a combination of beam search and model cascade. The lemmatization is performed employing a combination of a rule-based morphological analyzer, OMorFi, and a data-driven lemmatization model. The toolkit is readily applicable for tagging and lemmatization of running text with models learned from the recently published Finnish Turku Dependency Treebank and FinnTreeBank. Empirical evaluation on these corpora shows that FinnPos performs favorably compared to reference systems in terms of tagging and lemmatization ac- curacy. In addition, we demonstrate that our system is highly competitive with regard to computational efficiency of learning new models and assigning analyses to novel sentences. Keywords Morphological tagging · Data-driven lemmatization · Averaged perceptron · Finnish · Open-source M. Silfverberg ( ) · K. Lindén University of Helsinki, Helsinki, Finland M. Silfverberg E-mail: mpsilfve@iki.fi T. Ruokolainen · M. Kurimo Aalto University, Helsinki, Finland 2 Miikka Silfverberg et al. 1 Introduction This paper presents FinnPos, an open-source morphological tagging and lemmati- zation toolkit for Finnish. Our work stems from the recently published Turku De- pendency Treebank (Haverinen et al, 2009, 2014) and FinnTreeBank (Voutilainen, 2011). The Turku Dependency Treebank has been prepared by manually correcting the output of an automatic annotation process, whereas the FinnTreeBank has been prepared completely manually.
    [Show full text]
  • Arxiv:2105.12428V1 [Cs.CL] 26 May 2021
    Neural Morphology Dataset and Models for Multiple Languages, from the Large to the Endangered Mika Hämäläinen, Niko Partanen, Jack Rueter and Khalid Alnajjar Faculty of Arts University of Helsinki [email protected] Abstract machine translation (Pirinen et al., 2017). The transducers are also in constant use in several We train neural models for morpholog- real world applications such as online dictionar- ical analysis, generation and lemmatiza- ies (Rueter and Hämäläinen, 2019), spell check- tion for morphologically rich languages. ers (Trosterud and Moshagen, 2021), online cre- We present a method for automatically ative writing tools (Hämäläinen, 2018), automated extracting substantially large amount of news generation (Alnajjar et al., 2019), language training data from FSTs for 22 languages, learning tools (Antonsen and Argese, 2018) and out of which 17 are endangered. The neu- documentation of endangered languages (Gersten- ral models follow the same tagset as the berger et al., 2017; Wilbur, 2018). As an ad- FSTs in order to make it possible to use ditional important application we can mention them as fallback systems together with the the wide use of FSTs in the creation of Uni- FSTs. The source code1, models2 and versal Dependencies treebanks for low-resource datasets3 have been released on Zenodo. languages, at least with Erzya (Rueter and Ty- 1 Introduction ers, 2018), Northern Saami (Tyers and Sheyanova, 2017) Karelian (Pirinen, 2019a) and Komi-Zyrian Morphology is a powerful tool for languages to (Partanen et al., 2018). form new words out of existing ones through in- Especially in the context of endangered lan- flection, derivation and compounding.
    [Show full text]
  • Natural Language Parsing with Graded Constraints
    Natural Language Parsing with Graded Constraints Dissertation zur Erlangung des Doktorgrades am Fachbereich Informatik der Universität Hamburg vorgelegt von Ingo Schröder aus Uelzen Hamburg im Jahr 2002 Genehmigt vom Fachbereich Informatik der Universität Hamburg auf Antrag von Betreuer: Prof. Dr.-Ing. Wolfgang Menzel Fachbereich Informatik Universität Hamburg 2. Gutachter: Prof. Dr. Christopher Habel Fachbereich Informatik Universität Hamburg Externe Gutachterin: Prof. Mary P. Harper, Ph.D. School of Electrical and Computer Engineering Purdue University, IN, USA Hamburg, den 15. April 2002 Dekan Prof. Dr. Siegfried Stiehl Dedicated to my mother Ursula Schröder Abstract Based on the working hypothesis that gradation is a central phenomenon in natural language, this thesis proposes the grammar formalism of weighted constraint dependency grammars (WCDG) as a unified framework for the treatment of gradation. Gradation can be found in different forms in natural language: as a preference for one outcome over another or as an extended notion of grammaticality which accepts not only `well-formed' sentences but also deviant structures if no other analysis is feasible. It also occurs in inherently uncertain information such as that from automatic recognition of speech or blurred hand-writing. WCDGs not only offer a well-defined formalism to concisely express what the underlying con- straints of natural language understanding are but additionally allow one to state which status a constraint has, be it a strict rule which describes what structures are possibly imaginable or be it a slight preference which only selects the preferred one among a set of ambiguous analyses for a sentence. The formalism of WCDGs is formally defined both syntactically by specifying the constraint language and semantically by providing a concise mapping from the WCDG parsing problem to the well-known class of constraint satisfaction optimization problems.
    [Show full text]
  • CG3 Beyond Classical Constraint Grammar
    CG-3 - Beyond Classical Constraint Grammar Eckhard Bick Tino Didriksen University of Southern Denmark GrammarSoft ApS [email protected] [email protected] Abstract genre, but in programming terms, it is implemented procedurally as a set of This paper discusses methodological consecutively iterated rules that add, remove or strengths and shortcomings of the select tag-encoded information. In its classical Constraint Grammar paradigm (CG), form (Karlsson, 1990; Karlsson et al., 1995), showing how the classical CG formalism Constraint Grammar relies on a morphological can be extended to achieve greater analyzer providing so-called cohorts of possible expressive power and how it can be readings for a given word, and uses constraints enhanced and hybridized with techniques that are largely topological1 in nature, for both from other parsing paradigms. We present part-of-speech disambiguation and the a new, largely theory-independent CG assignment of syntactic function tags. (a-c) framework and rule compiler (CG-3), that provide examples for close context (a) and wide allows the linguist to write CG rules context (b) POS rules, and syntactic mapping (c). incorporating different types of linguistic information and methodology from a wide (a) REMOVE VFIN IF (0 N) (-1 ART OR range of parsing approaches, covering not <poss> OR GEN); remove a finite verb reading only CG©s native topological technique, if self (0) can also be a noun (N), and if there is but also dependency grammar, phrase an article (ART), possessive (<poss>) or structure grammar and unification genitive (GEN) 1 position left (-1). grammar. In addition, we allow the (b) SELECT VFIN IF (NOT *1 VFIN) (*-1C integration of statistical-numerical CLB-WORD BARRIER VFIN); select a finite constraints and non-discrete tag and string verb reading, if there is no other finite verb sets.
    [Show full text]
  • Applying Constraint Grammar to Tibetan NLP Edward Garrett & Nathan Hill
    Applying Constraint Grammar to Tibetan NLP Edward Garrett & Nathan Hill Project website: http://larkpie.net/tibetancorpus/ Workflow – Some rules Workflow – Flagging the rule Workflow – Flagging the rule Workflow – Fixing the rule Workflow – Fixing the rule Workflow – Fixing the rule Workflow – Fixing the rule Workflow – Fixing the rule Regex rules The regex tagger consists of a sequence of rules, applied in order to a horizontal text. Each rule consists of two parts, the pattern (before the < symbol) and the replacement (after the < symbol). In horizontal format, a single space marks the boundary between words, and line breaks separate sentences. Each word consists of a word form followed by a tag, with the pipe character in between. Whitespace is not permitted within words. ར་|[case.term][cv.term][dunno][n.count][skt] བs་|[n.count][v.fut][v.fut.v.pres][v.imp][v.pres] ནས་|[case.ela] [cv.ela][dunno][n.mass] ཡབ་|[n.count][v.fut] kི་|[case.gen][cv.cont][cv.gen] ཞལ་ཆེམས་|[n.count] བཞིན་|[n.count] [n.rel] ཕ་uལ་|[n.count] b|[n.count] ས་|[case.agn][cv.agn][dunno][n.count][n.mass][n.rel][skt] འཛན་|[v.pres] d་| [case.term][cv.term] འjག་པ|[n.v.fut.n.v.pres][n.v.pres] ར་|[case.term][cv.term][dunno][n.count][skt] u་| [n.count][v.fut][v.fut.v.pres][v.imp][v.past][v.past.v.pres][v.pres] bས་པ|[n.v.past] ས|[case.agn][n.count][n.rel] །| [punc] Regex rules Regex rules Regex rules Regex rules – summary Within short order, it became evidence that updating and maintaining regex rules would require a regular expressions wizard with a keen eye for slashes.
    [Show full text]
  • Solving Translational Problems Through Constraint Grammar - a Practical Case Study of Possession Homographs
    Liv Molich Bachelor thesis (20 ECTS), January 10th 2019 Department of Language, Literature & Media Institute of Culture, Language & History Ilisimatusarfik, University of Greenland Solving translational problems through constraint grammar - a practical case study of possession homographs Supervisor: Karen Langgård Character count: 71,218 Liv Molich Department of Language, Literature & Media Institute of Culture, Language & History Ilisimatusarfik, University of Greenland Introduction 4 Hypothesis 6 Outline 6 Materials and methods 7 Corpora 7 The parser (GCG) 8 Methods 11 Part I: Disambiguation in theory 13 Possession in Greenlandic 13 Head and dependent marking, and implicit or explicit possessor 14 The inflectional system of Greenlandic 16 Syncretism 16 Semantics 18 Using context to unravel the syntax and disambiguate the possessives 19 How 1st, 2nd and 3rd persons differ 24 Coordination can clarify the meaning 26 NIQ is a special case 28 Summary of part I 28 Part II: Disambiguation in practice 30 Local level rules for the GCG based on the morphological tagging 30 Global level rules for the GCG based on the morphological tagging 36 Rules for the GCG based on the syntactic roles 38 Summary of part II 39 2 Liv Molich Department of Language, Literature & Media Institute of Culture, Language & History Ilisimatusarfik, University of Greenland Part III: Proof checking and discussion 41 Proof checking the rules on regression corpus 41 Proof checking on extended corpus 41 Discussion 43 Conclusion 45 Literature 47 Appendix I: The tag set 52 Word
    [Show full text]
  • Instant Annotations in ELAN Corpora of Spoken and Written Komi, an Endangered Language of the Barents Sea Region
    Instant annotations in ELAN corpora of spoken and written Komi, an endangered language of the Barents Sea region Ciprian Gerstenberger∗ Niko Partanen UiT The Arctic University of Norway University of Hamburg Giellatekno – Saami Language Technology Department of Uralic Studies [email protected] [email protected] Michael Rießler University of Freiburg Freiburg Institute for Advanced Studies [email protected] Abstract NLP methods to more efficiently annotate qualita- tively and quantitatively language data for endan- The paper describes work-in-progress by gered languages, and this, despite the fact that the the Izhva Komi language documentation relevant computational methods and tools are well- project, which records new spoken lan- known from corpus-driven linguistic research on guage data, digitizes available recordings larger written languages. With respect to the data and annotate these multimedia data in types involved, endangered language documenta- order to provide a comprehensive lan- tion generally seems similar to corpus linguistics guage corpus as a databases for future re- (i.e. “corpus building”) of non-endangered lan- search on and for this endangered – and guages. Both provide primary data for secondary under-described – Uralic speech commu- (synchronic or diachronic) data derivations and nity. While working with a spoken vari- analyses (for data types in language documenta- ety and in the framework of documentary tion, cf. Himmelmann 2012; for a comparison be- linguistics, we apply language technol- tween corpus linguistics and language documenta- ogy methods and tools, which have been tion, cf. Cox 2011). applied so far only to normalized writ- The main difference is that traditional cor- ten languages.
    [Show full text]