191 on the DEVELOPMENT of an Interdisciplinary
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Compilation of a Swiss German Dialect Corpus and Its Application to Pos Tagging
Compilation of a Swiss German Dialect Corpus and its Application to PoS Tagging Nora Hollenstein Noemi¨ Aepli University of Zurich University of Zurich [email protected] [email protected] Abstract Swiss German is a dialect continuum whose dialects are very different from Standard German, the official language of the German part of Switzerland. However, dealing with Swiss German in natural language processing, usually the detour through Standard German is taken. As writing in Swiss German has become more and more popular in recent years, we would like to provide data to serve as a stepping stone to automatically process the dialects. We compiled NOAH’s Corpus of Swiss German Dialects consisting of various text genres, manually annotated with Part-of- Speech tags. Furthermore, we applied this corpus as training set to a statistical Part-of-Speech tagger and achieved an accuracy of 90.62%. 1 Introduction Swiss German is not an official language of Switzerland, rather it includes dialects of Standard German, which is one of the four official languages. However, it is different from Standard German in terms of phonetics, lexicon, morphology and syntax. Swiss German is not dividable into a few dialects, in fact it is a dialect continuum with a huge variety. Swiss German is not only a spoken dialect but increasingly used in written form, especially in less formal text types. Often, Swiss German speakers write text messages, emails and blogs in Swiss German. However, in recent years it has become more and more popular and authors are publishing in their own dialect. Nonetheless, there is neither a writing standard nor an official orthography, which increases the variations dramatically due to the fact that people write as they please with their own style. -
Semantic Approaches in Natural Language Processing
Cover:Layout 1 02.08.2010 13:38 Seite 1 10th Conference on Natural Language Processing (KONVENS) g n i s Semantic Approaches s e c o in Natural Language Processing r P e g a u Proceedings of the Conference g n a on Natural Language Processing 2010 L l a r u t a This book contains state-of-the-art contributions to the 10th N n Edited by conference on Natural Language Processing, KONVENS 2010 i (Konferenz zur Verarbeitung natürlicher Sprache), with a focus s e Manfred Pinkal on semantic processing. h c a o Ines Rehbein The KONVENS in general aims at offering a broad perspective r on current research and developments within the interdiscipli - p p nary field of natural language processing. The central theme A Sabine Schulte im Walde c draws specific attention towards addressing linguistic aspects i t of meaning, covering deep as well as shallow approaches to se - n Angelika Storrer mantic processing. The contributions address both knowledge- a m based and data-driven methods for modelling and acquiring e semantic information, and discuss the role of semantic infor - S mation in applications of language technology. The articles demonstrate the importance of semantic proces - sing, and present novel and creative approaches to natural language processing in general. Some contributions put their focus on developing and improving NLP systems for tasks like Named Entity Recognition or Word Sense Disambiguation, or focus on semantic knowledge acquisition and exploitation with respect to collaboratively built ressources, or harvesting se - mantic information in virtual games. Others are set within the context of real-world applications, such as Authoring Aids, Text Summarisation and Information Retrieval. -
Text Corpora and the Challenge of Newly Written Languages
Proceedings of the 1st Joint SLTU and CCURL Workshop (SLTU-CCURL 2020), pages 111–120 Language Resources and Evaluation Conference (LREC 2020), Marseille, 11–16 May 2020 c European Language Resources Association (ELRA), licensed under CC-BY-NC Text Corpora and the Challenge of Newly Written Languages Alice Millour∗, Karen¨ Fort∗y ∗Sorbonne Universite´ / STIH, yUniversite´ de Lorraine, CNRS, Inria, LORIA 28, rue Serpente 75006 Paris, France, 54000 Nancy, France [email protected], [email protected] Abstract Text corpora represent the foundation on which most natural language processing systems rely. However, for many languages, collecting or building a text corpus of a sufficient size still remains a complex issue, especially for corpora that are accessible and distributed under a clear license allowing modification (such as annotation) and further resharing. In this paper, we review the sources of text corpora usually called upon to fill the gap in low-resource contexts, and how crowdsourcing has been used to build linguistic resources. Then, we present our own experiments with crowdsourcing text corpora and an analysis of the obstacles we encountered. Although the results obtained in terms of participation are still unsatisfactory, we advocate that the effort towards a greater involvement of the speakers should be pursued, especially when the language of interest is newly written. Keywords: text corpora, dialectal variants, spelling, crowdsourcing 1. Introduction increasing number of speakers (see for instance, the stud- Speakers from various linguistic communities are increas- ies on specific languages carried by Rivron (2012) or Soria ingly writing their languages (Outinoff, 2012), and the In- et al. -
World Languages Using Latin Script
World languages using Latin script Source: http://www.omniglot.com/writing/langalph.htm https://www.ethnologue.com/browse/names Sort order : Language status, ISO 639-3 Lang, ISO Language name Classification Population status Language map Comment 639-3 (EGIDS) Botswana, Lesotho, South Indo-European, Germanic, West, Low 1. Afrikaans, afr 7,096,810 1 Africa and Saxon-Low Franconian, Low Franconian SwazilandNamibia Azerbaijan,Georgia,Iraq 2. Azeri,Azerbaijani azj Turkic, Southern, Azerbaijani 24,226,940 1 Jordan and Syria Indo-European Balto-Slavic Slavic West 3. Czech Bohemian Cestina ces 10,619,340 1 Czech Republic Czech-Slovak Chamorro,Chamorru Austronesian Malayo-Polynesian Guam and Northern 4. cha 94,700 1 Tjamoro Chamorro Mariana Islands Seychelles Creole,Seselwa Creole, Creole, Ilois, Kreol, 5. Kreol Seselwa, Seselwa, crs Creole, French based 72,700 1 Seychelles Seychelles Creole French, Seychellois Creole Indo-European Germanic North East Denmark Finland Norway 6. DanishDansk Rigsdansk dan Scandinavian Danish-Swedish Danish- 5,520,860 1 and Sweden Riksmal Danish AustriaBelgium Indo-European Germanic West High Luxembourg and 7. German Deutsch Tedesco deu German German Middle German East 69,800,000 1 NetherlandsDenmark Middle German Finland Norway and Sweden 8. Estonianestieesti keel ekk Uralic Finnic 1,132,500 1 Estonia Latvia and Lithuania 9. English eng Indo-European Germanic West English 341,000,000 1 over 140 countries Austronesian Malayo-Polynesian 10. Filipino fil Philippine Greater Central Philippine 45,000,000 1 Filippines L2 users population Central Philippine Tagalog Page 1 of 48 World languages using Latin script Lang, ISO Language name Classification Population status Language map Comment 639-3 (EGIDS) Denmark Finland Norway 11. -
Parsing Approaches for Swiss German
Master’s thesis presented to the Faculty of Arts and Social Sciences of the University of Zurich for the degree of Master of Arts UZH nmod case compound amod Parsing Approaches for Swiss German NOUN NOUN ADP ADJ NOUN Author: Noëmi Aepli Student ID Nr.: 10-719-250 Examiner: Prof. Dr. Martin Volk Supervisor: Dr. Simon Clematide Institute of Computational Linguistics Submission date: 10.01.2018 nmod case compound amod Parsing Approaches for Swiss German Abstract NOUN NOUN ADP ADJ NOUN Abstract This thesis presents work on universal dependency parsing for Swiss German. Natural language pars- ing describes the process of syntactic analysis in Natural Language Processing (NLP) and is a key area as many applications rely on its output. Building a statistical parser requires expensive resources which are only available for a few dozen languages. Hence, for the majority of the world’s languages, other ways have to be found to circumvent the low-resource problem. Triggered by such scenarios, research on different approaches to cross-lingual learning is going on. These methods seem promising for closely related languages and hence especially for dialects and varieties. Swiss German is a dialect continuum of the Alemannic dialect group. It comprises numerous vari- eties used in the German-speaking part of Switzerland. Although mainly oral varieties (Mundarten), they are frequently used in written communication. On the basis of their high acceptance in the Swiss culture and with the introduction of digital communication, Swiss German has undergone a spread over all kinds of communication forms and social media. Considering the lack of standard spelling rules, this leads to a huge linguistic variability because people write the way they speak. -
Ročník 68, 2017
2 ROČNÍK 68, 2017 JAZYKOVEDNÝ ČASOPIS ________________________________________________________________VEDEcKÝ ČASOPIS PrE OtáZKY tEórIE JAZYKA JOUrNAL Of LINGUIStIcS ScIENtIfIc JOUrNAL fOr thE thEOrY Of LANGUAGE ________________________________________________________________ hlavná redaktorka/Editor-in-chief: doc. Mgr. Gabriela Múcsková, PhD. Výkonní redaktori/Managing Editors: PhDr. Ingrid Hrubaničová, PhD., Mgr. Miroslav Zumrík, PhD. redakčná rada/Editorial Board: doc. PhDr. Ján Bosák, CSc. (Bratislava), PhDr. Klára Buzássyová, CSc. (Bratislava), prof. PhDr. Juraj Dolník, DrSc. (Bratislava), PhDr. Ingrid Hrubaničová, PhD. (Bra tislava), Doc. Mgr. Martina Ivanová, PhD. (Prešov), Mgr. Nicol Janočková, PhD. (Bratislava), Mgr. Alexandra Jarošová, CSc. (Bratislava), prof. PaedDr. Jana Kesselová, CSc. (Prešov), PhDr. Ľubor Králik, CSc. (Bratislava), PhDr. Viktor Krupa, DrSc. (Bratislava), doc. Mgr. Gabriela Múcsková, PhD. (Bratislava), Univ. Prof. Mag. Dr. Ste- fan Michael Newerkla (Viedeň – Rakúsko), Associate Prof. Mark Richard Lauersdorf, Ph.D. (Kentucky – USA), doc. Mgr. Martin Ološtiak, PhD. (Prešov), prof. PhDr. Slavomír Ondrejovič, DrSc. (Bratislava), prof. PaedDr. Vladimír Patráš, CSc. (Banská Bystrica), prof. PhDr. Ján Sabol, DrSc. (Košice), prof. PhDr. Juraj Vaňko, CSc. (Nitra), Mgr. Miroslav Zumrík, PhD. prof. PhDr. Pavol Žigo, CSc. (Bratislava). technický_______________________________________________________________ redaktor/technical editor: Mgr. Vladimír Radik Vydáva/Published by: Jazykovedný ústav Ľudovíta Štúra Slovenskej -
Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects, Pages 1–15, Valencia, Spain, April 3, 2017
VarDial 2017 Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial’2017) Proceedings of the Workshop April 3, 2017 Valencia, Spain c 2017 The Association for Computational Linguistics Order copies of this and other ACL proceedings from: Association for Computational Linguistics (ACL) 209 N. Eighth Street Stroudsburg, PA 18360 USA Tel: +1-570-476-8006 Fax: +1-570-476-0860 [email protected] ISBN 978-1-945626-43-2 ii Preface VarDial is a well-established series of workshops held annually and co-located with top-tier international NLP conferences. Previous editions of VarDial were VarDial’2014, which was co- located with COLING’2014, LT4VarDial’2015, which was held together with RANLP’2015, and finally VarDial’2016 co-located with COLING’2016. The great interest of the community has made possible the fourth edition of the Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial’2017), co-located with EACL’2017 in Valencia, Spain. The VarDial series has attracted researchers working on a wide range of topics related to linguistic variation such as building and adapting language resources for language varieties and dialects, creating language technology and applications that make use of language closeness, and exploiting existing resources in a related language or a language variety. We believe that this is a very timely series of workshops, as research in language variation is much needed in today’s multi-lingual world, where several closely-related languages, language varieties, and dialects are in daily use, not only as spoken colloquial language but also in written media, e.g., in SMS, chats, and social networks. -
Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss German
Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss German Pierre-Edouard Honnet∗;1, Andrei Popescu-Belis∗;2, Claudiu Musat3, Michael Baeriswyl3 1 Telepathy Labs 2 HEIG-VD / HES-SO 3 Swisscom (Schweiz) AG Schutzengasse¨ 25 Route de Cheseaux 1, CP 521 Genfergasse 14 CH-8001 Zurich¨ CH-1401 Yverdon-les-Bains CH-3011 Bern Switzerland Switzerland Switzerland pierre-edouard.honnet andrei.popescu-belis [email protected] @telepathy.ai @heig-vd.ch [email protected] Abstract The goal of this work is to design a machine translation (MT) system for a low-resource family of dialects, collectively known as Swiss German, which are widely spoken in Switzerland but seldom written. We collected a significant number of parallel written resources to start with, up to a total of about 60k words. Moreover, we identified several other promising data sources for Swiss German. Then, we designed and compared three strategies for normalizing Swiss German input in order to address the regional diversity. We found that character-based neural MT was the best solution for text normalization. In combination with phrase-based statistical MT, our solution reached 36% BLEU score when translating from the Bernese dialect. This value, however, decreases as the testing data becomes more remote from the training one, geographically and topically. These resources and normalization techniques are a first step towards full MT of Swiss German dialects. Keywords: machine translation, low-resource languages, spoken dialects, Swiss German, character-based neural MT 1. Introduction present the Swiss German dialects and review the scarce In the era of social media, more and more people make monolingual and even scarcer parallel language resources online contributions in their own language. -
Text Corpora and the Challenge of Newly Written Languages Alice Millour, Karën Fort
Text Corpora and the Challenge of Newly Written Languages Alice Millour, Karën Fort To cite this version: Alice Millour, Karën Fort. Text Corpora and the Challenge of Newly Written Languages. 1st Joint SLTU and CCURL Workshop (SLTU-CCURL 2020), May 2020, Marseille, France. hal-02611209 HAL Id: hal-02611209 https://hal.archives-ouvertes.fr/hal-02611209 Submitted on 18 May 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Text Corpora and the Challenge of Newly Written Languages Alice Millour∗, Karen¨ Fort∗y ∗Sorbonne Universite´ / STIH, yUniversite´ de Lorraine, CNRS, Inria, LORIA 28, rue Serpente 75006 Paris, France, 54000 Nancy, France [email protected], [email protected] Abstract Text corpora represent the foundation on which most natural language processing systems rely. However, for many languages, collecting or building a text corpus of a sufficient size still remains a complex issue, especially for corpora that are accessible and distributed under a clear license allowing modification (such as annotation) and further resharing. In this paper, we review the sources of text corpora usually called upon to fill the gap in low-resource contexts, and how crowdsourcing has been used to build linguistic resources. -
Compilation of a Swiss German Dialect Corpus and Its Application to Pos Tagging
Compilation of a Swiss German Dialect Corpus and its Application to PoS Tagging Nora Hollenstein Noemi¨ Aepli University of Zurich University of Zurich [email protected] [email protected] Abstract Swiss German is a dialect continuum whose dialects are very different from Standard German, the official language of the German part of Switzerland. However, dealing with Swiss German in natural language processing, usually the detour through Standard German is taken. As writing in Swiss German has become more and more popular in recent years, we would like to provide data to serve as a stepping stone to automatically process the dialects. We compiled NOAH’s Corpus of Swiss German Dialects consisting of various text genres, manually annotated with Part-of- Speech tags. Furthermore, we applied this corpus as training set to a statistical Part-of-Speech tagger and achieved an accuracy of 90.62%. 1 Introduction Swiss German is not an official language of Switzerland, rather it includes dialects of Standard German, which is one of the four official languages. However, it is different from Standard German in terms of phonetics, lexicon, morphology and syntax. Swiss German is not dividable into a few dialects, in fact it is a dialect continuum with a huge variety. Swiss German is not only a spoken dialect but increasingly used in written form, especially in less formal text types. Often, Swiss German speakers write text messages, emails and blogs in Swiss German. However, in recent years it has become more and more popular and authors are publishing in their own dialect. Nonetheless, there is neither a writing standard nor an official orthography, which increases the variations dramatically due to the fact that people write as they please with their own style. -
Collecting and Annotating Corpora for Three Under-Resourced Languages of France: Methodological Issues
Vol. 15 (2021), pp. 316–357 http://nflrc.hawaii.edu/ldc http://hdl.handle.net/10125/74645 Revised Version Received: 13 Apr 2021 Collecting and annotating corpora for three under-resourced languages of France: Methodological issues Delphine Bernhard Pascale Erhart Université de Strasbourg, Université de Strasbourg, LiLPa, UR 1339 LiLPa, UR 1339 Anne-Laure Ligozat Jean Sibille LISN, ENSIIE CLLE, UMR 5263, Université Toulouse Jean Jaurès, CNRS Myriam Bras Amalia Todirascu CLLE, UMR 5263, Université de Strasbourg, Université Toulouse Jean Jaurès, CNRS LiLPa, UR 1339 Fanny Martin Philippe Boula de Mareüil Habiter Le Monde, EA 4287, LISN, CNRS Université de Picardie Jules Verne Marianne Vergez-Couret Dominique Huck FoReLLIS, EA 3816, Université de Strasbourg, Université de Poitiers LiLPa, UR 1339 In contrast to French, the vast majority of regional languages of France can be considered as under-resourced. In this article, we present the results of a research project aiming to produce annotated resources for three regional languages of France: Alsatian, Occitan, and Picard. These languages cover three different lan- guage families (Germanic and two subfamilies of Romance, Oïl and Oc languages) and different sociolinguistic situations. Yet, they all face issues common to many under-resourced languages: lack of human and financial resources and presence of geolinguistic variation. The originality of this project is that it brought together researchers from different fields (sociolinguistics, descriptive linguistics, dialectol- ogy, natural language processing, digital humanities) to work together towards the common goal of developing annotated corpora for Alsatian, Occitan, and Pi- card. This created a favorable and stimulating working environment which could not have been achieved had different research groups worked independently, each on a single language. -
Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, Pages 1–10, Dublin, Ireland, August 23 2014
COLING 2014 The 1st Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects Proceedings of the Workshop August 23, 2014 Dublin, Ireland c 2014: Papers marked with a Creative Commons or other specific license statement are copyright of their respective authors (or their employers). ISBN 978-1-873769-39-3 Proceedings of VarDial: Applying NLP Tools to Similar Languages, Varieties and Dialects Marcos Zampieri, Liling Tan, Nikola Ljubešic´ and Jörg Tiedemann (eds.) ii Introduction The interest in language resources and computational models for the study of similar languages, varieties and dialects has been growing substantially in the last few years. The first edition of the Workshop on Applying NLP tools to similar languages, varieties and dialects (VarDial) confirms the interest in the topic. Within the NLP community, the impact of language variation in the development of language resources and NLP applications has been explored in recent years with experiments in different directions. For example, automatic classification or identification of closely related languages such as in Huang and Lee (2008) and Tiedemann and Ljubešic´ (2012); corpus-driven studies focusing on lexical variation between varieties such as the one by Piersman et al. (2010) or Ljubešic´ and Fišer (2013); and finally, the adaptation of language models in the context of machine translation such as in Nakov and Tiedemann (2012). Together with the VarDial workshop we organized the Discriminating between Similar Languages (DSL) shared task. Discriminating between similar languages and language varieties is one of the bottlenecks of state-of-the-art language identification and it has been topic of a number of papers published in the last years.