Modèles Et Outils Pour Des Bases Lexicales "Métier" Multilingues Et

Total Page:16

File Type:pdf, Size:1020Kb

Modèles Et Outils Pour Des Bases Lexicales THÈSE Pour obtenir le grade de DOCTEUR DE LA COMMUNAUTE UNIVERSITE GRENOBLE ALPES Spécialité : INFORMATIQUE Arrêté ministériel : 7 août 2006 Présentée par Ying ZHANG Thèse dirigée par Christian BOITET et codirigée par Valérie BELLYNCK et Mathieu MANGEOT-NAGATA préparée au sein du Laboratoire d’Informatique de Grenoble dans l'École Doctorale « Mathématiques, Sciences et Technologies de l’Information, informatique » Modèles et outils pour des bases lexicales "métier" multilingues et contributives de grande taille, utilisables tant en traduction automatique et automatisée que pour des services dictionnairiques variés Thèse soutenue publiquement le 28 juin 2016 , devant le jury composé de : Prof. Ahmed LBATH Professeur, UGA, Président Prof. Denis MAUREL Professeur, Tours, Rapporteur Prof. Alain POLGUERE Professeur, Nancy, Rapporteur MdC. Mathieu LAFOURCADE Maître de conférence, Montpellier II, Examinateur Prof. Antoine Chalvin Professeur, INaLCO, Examinateur Prof. Christian BOITET Professeur, UGA, Directeur MdC. Valérie BELLYNCK Maître de conférence, Grenoble-INP, Co-Directrice MdC. Mathieu MANGEOT-NAGATA Maître de conférence, UdSavoie, Co-Directeur UNIVERSITÉ DE GRENOBLE ° N attribué par la bibliothèque /__/__/__/__/__/__/__/__/__/ THÈSE pour obtenir le grade de DOCTEUR ÈS SCIENCES délivré par l'UNIVERSITÉ GRENOBLE ALPES Spécialité : “INFORMATIQUE” Thèse préparée au laboratoire GETALP-LIG (CNRS-INPG-UJF) dans le cadre de l'École Doctorale “Mathématiques, Sciences et Technologies de l 'Information, Informatique” présentée et soutenue publiquement par Ying ZHANG Le 28/6/2016 Modèles et outils pour des bases lexicales "métier" multilingues et contributives de grande taille, utilisables tant en traduction automatique et automatisée que pour des services dictionnairiques variés JURY M. Ahmed LBATH , prof. UGA Président M. Denis MAUREL , prof. Tours Rapporteur M. Alain POLGUERE , prof. Nancy Rapporteur M. Mathieu LAFOURCADE , MdC Montpellier II Examinateur M. Antoine Chalvin, prof. INaLCO Examinateur M. François BROWN DE COLSTOUN , PDG de L&M Invité M. Christian BOITET , prof. UGA Directeur de thèse Mme Valérie BELLYNCK , MdC G-INP Codirecteur de thèse M. Mathieu MANGEOT -NAGATA , MdC UdSavoie Codirecteur de thèse 1/202 Remerciements Les travaux présentés dans cette thèse ont fait l'objet d'une convention CIFRE entre la société Lingua et Machina et le GETALP (Groupe d'Étude pour la Traduction Automatique et le Traitement Automatisé des Langues et de la Parole) du LIG (Laboratoire d'Informatique de Grenoble). Je voudrais tout d'abord remercier le professeur Christian Boitet, mon directeur de thèse, qui est à l'origine de ce travail. C'est un honneur pour moi de travailler avec lui et je ne peux qu'admirer son talent. Je lui suis infiniment reconnaissante, non seulement parce qu'il a accepté de me prendre en thèse, mais aussi parce qu'il a partagé ses idées avec moi. Il a dirigé ma thèse avec beaucoup de patience et il a dédié beaucoup de temps à mon travail en étant toujours très disponible et en venant me chercher très souvent pour que l'on discute, ce qui m'a énormément encouragée. Je le remercie aussi d'avoir lu très sérieusement beaucoup de versions préliminaires de ces travaux. J'adresse de chaleureux remerciements à mon co-directeur M. Mathieu Mangeot. Il m'a accueilli gentiment chaque fois que je venais l'embêter pour lui poser des questions et ses réponses m'ont toujours éclairci les idées. J'admirerai toujours son savoir ainsi que sa capacité à l'exposer et à le partager. Il a aussi dédié beaucoup de son temps à discuter avec moi à propos de mon avenir et je lui en suis très reconnaissante. Un grand merci à mon autre co-directeur Mme Valérie Bellynck. Tout d'abord pour son accueil, mais surtout pour les nombreuses discussions que nous avons eues sur son concept de "langage narratif" et pour son aide constante en ce qui concerne les supports techniques pour mes développements. Elle m'a beaucoup appris, non seulement sur la vie scientifique, mais aussi sur la vie culturelle, et sur la vie tout court. J'ai énormément apprécié son enthousiasme et sa sympathie. Je tiens à remercier particulièrement le président de Lingua et Machina, M. François Brown de Colstoun, sans qui cette thèse CIFRE n'aurait sûrement jamais vu le jour. Mes remerciements s'adressent également aux professeurs Denis Maurel et Alain Polguère pour avoir accepté de rapporter sur ma thèse et pour l'intérêt qu'ils ont porté à ces travaux. J'espère qu'à l'avenir nous serons amenés à échanger de nouveau sur ce sujet passionnant. Un grand merci aussi au professeur Ahmed Lbath pour m'avoir fait l'honneur de présider le jury de ma thèse. Antoine Chalvin, professeur d'estonien à l'INaLCO, et contributeur majeur du projet GDEF (Grand Dictionnaire Estonien-Français) développé en JIBIKI avec le support de M. Mangeot depuis 2003, a accepté de participer à ce jury malgré ses lourdes tâches à Paris. Je l'en remercie d'autant plus. Mathieu Lafourcade, Maître de Conférences HDR à Montpellier, et spécialiste de la création contributive de grandes bases lexicales vers des "jeux sérieux", et de désambiguïsation lexicale, a accepté de faire partie de mon jury. C'est un grand honneur pour moi de les remercier. Mes derniers remerciements iront évidemment à ma famille. Je pense tout d'abord à mes parents qui m'ont toujours épaulée dans mes études, mes décisions et mes choix, même si je n'ai pas pu être à leurs côtés pendant une longue période. Ensuite, je voudrais remercier mon mari Wang Ling Xiao, qui partage ma vie, et a fait aussi sa thèse en CIFRE avec L&M, sur un sujet complémentaire (les bases de données "corporales") et mon fils Wang Xin Di qui m'a apporté beaucoup de joie durant ma vie de doctorante. 2/202 Résumé en français Notre recherche se situe en lexicographie computationnelle, et concerne non seulement le support informatique aux ressources lexicales utiles pour la TA (traduction automatique) et la THAM (traduction humaine aidée par la machine), mais aussi l'architecture linguistique des bases lexicales supportant ces ressources, dans un contexte opérationnel (thèse CIFRE avec L&M). Nous commençons par une étude de l'évolution des idées, depuis l'informatisation des dictionnaires classiques jusqu'aux plates-formes de construction de vraies "bases lexicales" comme JIBIKI -1 [Mangeot, M. et al., 2003 ; Sérasset, G., 2004] et JIBIKI -2 [Zhang, Y. et al., 2014]. Le point de départ a été le système PIVAX -1 [Nguyen, H.-T. et al., 2007 ; Nguyen, H. T. & Boitet, C., 2009] de bases lexicales pour systèmes de TA hétérogènes à pivot lexical supportant plusieurs volumes par "espace lexical" naturel ou artificiel (UNL). En prenant en compte le contexte industriel, nous avons centré notre recherche sur certains problèmes, informatiques et lexicographiques. Pour passer à l'échelle, et pour profiter des nouvelles fonctionnalités permises par JIBIKI -2, dont les "liens riches", nous avons transformé PIVAX -1 en PIVAX -2, et réactivé le projet GBD LEX -UW++ commencé lors du projet ANR TRAOUIERO , en réimportant toutes les données (multilingues) supportées par PIVAX -1, et en les rendant disponibles sur un serveur ouvert. Partant d'un besoin de L&M concernant les acronymes, nous avons étendu la "macrostructure" de PIVAX en y intégrant des volumes de "prolexèmes", comme dans PROLEXBASE [Tran, M. & Maurel, D., 2006]. Nous montrons aussi comment l'étendre pour répondre à de nouveaux besoins, comme ceux du projet INNOVALANGUES . Enfin, nous avons créé un "intergiciel de lemmatisation", LEXTOH , qui permet d'appeler plusieurs analyseurs morphologiques ou lemmatiseurs, puis de fusionner et filtrer leurs résultats. Combiné à un nouvel outil de création de dictionnaires, CREAT DICO , LEXTOH permet de construire à la volée un "mini-dictionnaire" correspondant à une phrase ou à un paragraphe d'un texte en cours de "post-édition" en ligne sous IMAG/SECT RA , ce qui réalise la fonctionnalité d'aide lexicale proactive prévue dans [Huynh, C.-P., 2010]. On pourra aussi l'utiliser pour créer des corpus parallèles "factorisés" pour construire des systèmes de TA en MOSES . Abstract in English Our research is in computational lexicography, and concerns not only the computer support to lexical resources useful for MT (machine translation) and MAHT (Machine Aided Human Translation), but also the linguistic architecture of lexical databases supporting these resources in an operational context (CIFRE thesis with L&M). We begin with a study of the evolution of ideas in this area, since the computerization of classical dictionaries to platforms for building up true "lexical databases" such as JIBIKI -1 [Mangeot, M. et al., 2003 ; Sérasset, G., 2004] and JIBIKI -2 [Zhang, Y. et al., 2014]. The starting point was the PIVAX -1 system [Nguyen, H.-T. et al., 2007 ; Nguyen, H. T. & Boitet, C., 2009] designed for lexical bases for heterogeneous MT systems with a lexical pivot, able to support multiple volumes in each "lexical space", be it natural or artificial (as UNL). Considering the industrial context, we focused our research on some issues, in informatics and lexicography. To scale up, and to add some new features enabled by JIBIKI -2, such as the "rich links", we have transformed PIVAX -1 into PIVAX -2, and reactivated the GBD LEX -UW++ project that started during the ANR TRAOUIERO project, by re-importing all (multilingual) data supported by PIVAX -1, and making them available on an open server. Hence a need for L&M for acronyms, we expanded the "macrostructure" of PIVAX incorporating volumes of "prolexemes" as in PROLEXBASE [Tran, M. & Maurel, D., 2006]. We also show how to extend it to meet new needs such as those of the INNOVALANGUES project. Finally, we have created a "lemmatisation middleware", LEXTOH , which allows calling several morphological analyzers or lemmatizers and then to merge and filter their results. Combined with a new dictionary creation tool, CREAT DICO , LEXTOH allows to build on the fly a "mini-dictionary" corresponding to a sentence or a paragraph of a text being "post-edited" online under IMAG/SECT RA , which performs the lexical proactive support functionality foreseen in [Huynh, C.-P., 2010].
Recommended publications
  • Building a Collation Element Table for a Large Chinese Character Set in YES
    Building a Collation Element Table for a Large Chinese Character Set in YES Xiaoheng Zhang1 and Xiaotong Li2 1 Department of Chinese and Bilingual Studies, Hong Kong Polytechnic University [email protected] 2 College of International Exchange, Shenzhen University [email protected] Abstract. YES is a simplified stroke-based method for sorting Chinese characters. It is free from stroke counting and grouping, and thus much faster and more accurate than the traditional method. This paper presents a collation element table built in YES for a large joint Chinese character set covering (a) all 20,902 characters of Unicode CJK Unified Ideographs, (b) all 11,408 characters in the Complete List of Chinese Characters Used by the Media in 2013, (c) all 13,000 plus characters in the latest versions of Xinhua Dictionary(v11) and Contemporary Chinese Dictionary(v6). Of the 20,902 Chinese characters in Unicode, 97.23% have one-to-one relationship with their stroke order codes in YES, comparing with 90.69% of the traditional method. Enhanced with the secondary and tertiary sorting levels of stroke layout and Unicode value, there is a guarantee of one-to-one relationship between the characters and collation elements. The collation element table has been successfully applied to sorting CC-CEDICT, a Chinese-English dictionary of over 112,000 word entries. Keywords: Chinese characters, collation, Unicode, YES 1 Introduction Collation, or determining the sorting order of strings of characters, is a key function in computer systems and a basic need of the human society. Whenever a list of texts—such as the records of a database and the entries of a dictionary—is presented to users, they are likely to want it in a sorted order so that they can easily and reliably find their target words.
    [Show full text]
  • In This Issue
    译风 The ATA Chinese Language Division Newsletter|Winter 2019|美国翻译协会中文翻译分会通讯 IN THIS ISSUE From the Editors 01 Farewell from Our Outgoing Layout Editor 03 This issue of Yifeng witnesses yet more Since our last issue on the new A Little Story About Our changes as we bid farewell to our layout Chinese-to-English ATA certification New Layout Editor editor, Tianlu Redmon (贾天璐) and exam, CLD member Rony Gao (高嵘) welcome new layout editor Pearl Zheng not only obtained the certification but 04 (郑绍娴) on page 4. Many thanks to figured out how to do so "in ten both of them for their hard work hours"! See how he achieved this feat Letter from the CLD ensuring that Yifeng is not just interesting on page 16 and learn some tips for Administrator to read but also pleasant to look at. preparing for the certification exam. 05 ATA annual conferences share much in Finally, in the Resources Roundup on common, but each offers a unique page 19, Yifeng editor Trevor Cook My First ATA experience. Read about first-time reviews the indispensable Pleco app, Conference - More than attendee Jessie Liu’s experience in New and, in Bird's Corner, CLD Just an Escape from the Orleans (pg. 7) and how Sean Song 蔡晓萍 Administrator Pency Tsai ( ) Kids (宋俊杰) has improved his conference shares a fresh take on adopting experience through repeat attendance translation and interpretation 07 (pg. 9). If you missed her at this year’s technology on page 22. conference, you can also meet CLD Finding My Niche for a member Gigi Yau (尤建之) in our new Better Conference feature 15 Minutes with June |译者访谈 Experience 录, by Junqiao Chen (陈俊巧), on page 12.
    [Show full text]
  • For Pocket PC Pleco Software Incorporated
    Version 1.0.3 Revision 2 (updated install) For Pocket PC Pleco Software Incorporated www.pleco.com - 1 - Introduction..................................................................................................................................... 4 System Requirements ..................................................................................................................... 5 Installation ...................................................................................................................................... 6 Before You Install....................................................................................................................... 6 Installing PlecoDict..................................................................................................................... 7 Installation on Windows Mobile 2003........................................................................................ 8 Installing the Demo Version ................................................................................................... 8 Upgrading from the Demo to the Paid Version .................................................................... 12 Reinstalling the Paid Version of PlecoDict .......................................................................... 14 Installation on Windows Mobile 5............................................................................................ 15 Installing the Demo Version ................................................................................................
    [Show full text]
  • Design of a Cloud Service for Learning Chinese Pronunciation
    54 JOURNAL OF ELECTRONIC SCIENCE AND TECHNOLOGY, VOL. 12, NO. 1, MARCH 2014 Design of a Cloud Service for Learning Chinese Pronunciation Hsueh-Ting Chu, Wei-Shan Tsai, and Shao-Yu Lee AbstractWith the fast growing of cloud computing Chinese articles in textbooks usually requires the help of infrastructure, learning from cloud services has become additional phonetic symbols. There are two Chinese more and more convenient for people worldwide. In phonetic systems: Pinyin and Zhuyin. Pinyin is currently order to integrate the cloud computing technology and the official Romanization which was published by the different e-learning platforms including variant mobile People's Republic of China (PRC) in 1958[6]. Before 1958, apps, Windows and web-based applications, we develop Zhuyin was the standardized phonetic system in entire our Chinese learning system “analytic Chinese helper” China for Chinese (Mandarin) pronunciation. The Zhuyin with a service-oriented architecture (SOA). Based on the system incorporates 37 Bopomofo symbols (Fig. 1) which new architecture we designed and developed a cloud transcribe precise sounds of Chinese characters among service for the e-learning of Chinese language on the different Chinese dialects. At present, Zhuyin is still the Internet as a convenient resource for foreign students, major phonetic system in Taiwan’s education system for especially in the reading of Chinese texts. teaching and learning Chinese. At the same time, Pinyin is There are two Chinese phonetic systems: Pinyin and also used in traffic signs and maps in Taiwan. Zhuyin. Pinyin is the official Romanization of Chinese Moreover, the learning of Chinese characters is also characters, and Zhuyin incorporates additional difficult for foreigners because there are two different Bopomofo symbols which transcribe precise sounds of writing systems of Chinese characters: Traditional Chinese Chinese characters.
    [Show full text]
  • Free Download Chinese Dictionaries
    Free download chinese dictionaries NOTE: if you experience errors trying to download our app, see for other download options. Pleco is the ultimate Chinese learning. Easily learn Chinese & English with Chinese English Dictionary & Translator app! Free download & no Internet connection required! The Chinese English. WHAT MAKES OXFORD DICTIONARY OF CHINESE BETTER THAN OTHER DICTIONARIES? • The very latest vocabulary, with over , Over words and expressions! FREE!* Works offline! No Internet connection needed, the perfect translator for your trips, your studies, or when no data. Download Chinese Dictionary. Free and safe download. Download the latest version of the top software, games, programs and apps in English Chinese Dictionary - Lite - Convert your laptop into an English Chinese dictionary - electronic translator! English Chinese Dictionary. chinese dictionary free download. KeePass KeePass Password Safe is a free, open source, lightweight, and easy-to-use password manager for. Download this app from Microsoft Store for Windows 10, Windows Chinese English Dictionary. Hein Htat. Rated: stars out of 5. (82) reviews. Free. Search Hanzi/Chinese, Pinyin or English. Also displays Zhuyin (Bopomofo) • Both Simplified and. CC-CEDICT download. What is CC-CEDICT? The word dictionary of this website is based on CC-CEDICT. CC-CEDICT is a continuation of the CEDICT project. We review the best Chinese-English Dictionary Apps so you don't have to It offers a chic, colorful design and most importantly, a free Chinese. download translate: (尤指从因特网或较大型计算机上)下载, 下载程序;下载信息. Learn more in the Cambridge Translation of "download" - English-Mandarin Chinese dictionary. See all translations 下载程序;下载信息. a free download. LG smartphone, open up Google Play, search for “Pleco” and tap “Pleco Chinese Dictionary” to instantly download the free basic version of Pleco for Android.
    [Show full text]
  • Keeping the Meanings of the Source Text: an Introduction to Yes Translate
    Keeping the Meanings of the Source Text: An introduction to Yes Translate Xiaoheng Zhang Department of Chinese and Bilingual Studies, Hong Kong Polytechnic University [email protected] Abstract. The primary task of language translation is to faithfully pass the meaning(s) of the source text to the target language. Unfortunately, meanings often get lost or distorted in machine translation, including state-of-the-art Google Translate and Baidu Translate. Yes Translate is a Chinese-English translation tool to be maximally loyal to the source text while maintaining adequate fluency. This is implementable by avoiding risky actions of word deleting, adding and re-ordering. The tool is supported by an 116,000-words Dictionary. In an experiment on natural news articles freely selected by themselves, 10 postgraduate students with good command of Chinese and English all agreed or strongly agreed that the general meaning of the translation by Yes Translate was correct and understandable. And 9 out of the 10 students agreed or strongly agreed that the general meaning of each sentence was correct. Keywords: Machine translation, meanings, Yes Translate, YES-CEDICT 1 Introduction The basic goal of translation is to faithfully convey the intended meaning(s) of the author in one language to the reader in a different language. However, there is large room for improvement in this aspect in machine transition (MT) (Bhattacharyya, 2015, p30), which is true even to the most popular and state-of-the-art systems such as Google Translate and Baidu Translate, as demonstrated by the following example. The source text is a Chinese article from Yahoo News (Yahoo, 2016).
    [Show full text]
  • Tocharian Loanwords in Chinese Tocharské Výpůjčky V Čínštině
    Univerzita Karlova Diplomová práce Ústav obecné lingvistiky Diplomová práce Bc. Jan Židek Tocharian Loanwords in Chinese Tocharské výpůjčky v čínštině Praha 2017 vedoucí Ronald Kim, Ph.D., konzultant doc. Mgr. Lukáš Zádrapa, Ph.D. I would like to express my utmost gratitude towards my thesis supervisor Ronald Kim, Ph.D. and consultant doc. Mgr. Lukáš Zádrapa, Ph.D., I would also like to thank everyone who took care of my academic needs throughout all those years of my university studies, especially Mgr. Jan Bičovský, Ph.D. who made me complete my bachelor studies and Mgr. Jakub Maršálek, Ph.D. who taught me basics of Classical Chinese, essentially pointing me in the direction of this work. I would also like to thank my mother for the material support and Buddha-like patience. Also, I hereby thank all my colleagues and friends who supported me in uncountable ways. Prohlašuji, že jsem diplomovou práci vypracoval samostatně, že jsem řádně citoval všechny použité prameny a literaturu a že práce nebyla využita v rámci jiného vysokoškolského studia či k získání jiného nebo stejného titulu. V Praze dne 28.5.2017, ………………………. Abstract This work was created to review the evidence for lexical borrowing from the Tocharian languages to the Chinese languages. The used methodology relies on lexical lists, previous etymological findings, linguistic typology and anthropological input. For preparatory data manipulation, a set of semi- automatic scripts has been created. Presented is a qualitative research based on previous findings assisted by raw data. The outcome of this work should be testable findings which could be extracted to a computer processable form.
    [Show full text]
  • A User's Guide to the Unihan Database
    UTR #38: A User’s Guide to the Unihan Database Page 1 of 48 Technical Reports Proposed Draft Unicode Technical Report #38 A USER’S GUIDE TO THE UNIHAN DATABASE Authors John H. Jenkins 井作恆 Richard Cook Date 2006-05-12 This Version http://www.unicode.org/reports/tr38/tr38-3.html Previous Version None Latest Version http://www.unicode.org/reports/tr38/ Tracking Number 3 Summary This document describes the organization and content of the Unihan database. Status This document is a Proposed Draft Unicode Technical Report. Publication does not imply endorsement by the Unicode Consortium. This is a draft document which may be updated, replaced, or superseded by other documents at any time. This is not a stable document; it is inappropriate to cite this document as other than a work in progress. A Unicode Technical Report (UTR) contains informative material. Conformance to the Unicode Standard does not imply conformance to any UTR. Other specifications, however, are free to make normative references to a UTR. Please submit corrigenda and other comments with the online reporting form [Feedback]. Related information that is useful in understanding this document is found in References. For the latest version of the Unicode Standard see [Unicode]. For a list of current Unicode Technical Reports see [Reports]. For more information about versions of the Unicode Standard, see [Versions]. [Note to reviewers: This document currently duplicates much of the content of the Unihan.html file in the Unicode database. The version here is not yet the official,
    [Show full text]
  • Chinese Listeni/Tʃaɪˈniːz/ (汉语 / 漢語; Hànyǔ Or 中文; Zhōngwén) Is
    Chinese Listeni/tʃaɪˈniːz/ (汉语 / 漢語; Hànyǔ or 中文; Zhōngwén) is a group of related but in many cases mutually unintelligible language varieties, forming a branch of the Sino-Tibetan language family. Chinese is spoken by the Han majority and many other ethnic groups in China. Nearly 1.2 billion people (around 16% of the world's population) speak some form of Chinese as their first language. The varieties of Chinese are usually described by native speakers as dialects of a single Chinese language, but linguists note that they are as diverse as a language family.[a] The internal diversity of Chinese has been likened to that of the Romance languages, but may be even more varied. There are between 7 and 13 main regional groups of Chinese (depending on classification scheme), of which the most spoken, by far, is Mandarin (about 960 million), followed by Wu (80 million), Yue (70 million) and Min (70 million). Most of these groups are mutually unintelligible, although some, like Xiang and the Southwest Mandarin dialects, may share common terms and some degree of intelligibility. All varieties of Chinese are tonal and analytic. Standard Chinese (Putonghua/Guoyu/Huayu) is a standardized form of spoken Chinese based on the Beijing dialect of Mandarin. It is the official language of China and Taiwan, as well as one of four official languages of Singapore. It is one of the six official languages of the United Nations. The written form of the standard language (中文; Zhōngwén), based on the logograms known as Chinese characters (汉字/漢字; hànzi), is shared by literate speakers of otherwise unintelligible dialects.
    [Show full text]
  • Corpus of Chinese Dynastic Histories: Gender Analysis Over Two Millennia
    Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 785–793 Marseille, 11–16 May 2020 c European Language Resources Association (ELRA), licensed under CC-BY-NC Corpus of Chinese Dynastic Histories: Gender Analysis over Two Millennia Sergey Zinin, Yang Xu University of Massachusetts Amherst, University of Toronto Warring States Workshop, Amherst, Massachusetts, USA, Department of Computer Science, Toronto, Canada [email protected], [email protected] Abstract Chinese dynastic histories form a large continuous linguistic space of approximately 2000 years, from the 3rd century BCE to the 18th century CE. The histories are documented in Classical (Literary) Chinese in a corpus of over 20 million characters, suitable for the computational analysis of historical lexicon and semantic change. However, there is no freely available open-source corpus of these histories, making Classical Chinese low-resource. This project introduces a new open-source corpus of twenty-four dynastic histories covered by Creative Commons license. An original list of Classical Chinese gender-specific terms was developed as a case study for analyzing the historical linguistic use of male and female terms. The study demonstrates considerable stability in the usage of these terms, with dominance of male terms. Exploration of word meanings uses keyword analysis of focus corpora created for gender- specific terms. This method yields meaningful semantic representations that can be used for future studies of diachronic semantics. Keywords: Classical Chinese, dynastic histories, corpus linguistics, historical linguistics, semantics, gender, keyword analysis digitization of the dynastic histories3. However, despite 1. Introduction having been digitized and placed online, this resource is not available to the community at large.
    [Show full text]
  • UAX #38: Unicode Han Database (Unihan)
    Technical Reports Proposed Update Unicode® Standard Annex #38 Version Unicode 9.0.0 Editors John H. Jenkins 井作恆 Richard Cook 曲理查 Ken Lunde 小林劍 Date 2015-09-22 This Version http://www.unicode.org/reports/tr38/tr38-20.html Previous http://www.unicode.org/reports/tr38/tr38-19.html Version Latest http://www.unicode.org/reports/tr38/ Version Latest http://www.unicode.org/reports/tr38/proposed.html Proposed Update Revision 20 Summary This document describes the organization and content of the Unihan database. Status This is a draft document which may be updated, replaced, or superseded by other documents at any time. Publication does not imply endorsement by the Unicode Consortium. This is not a stable document; it is inappropriate to cite this document as other than a work in progress. A Unicode Standard Annex (UAX) forms an integral part of the Unicode Standard, but is published online as a separate document. The Unicode Standard may require conformance to normative content in a Unicode Standard Annex, if so specified in the Conformance chapter of that version of the Unicode Standard. The version number of a UAX document corresponds to the version of the Unicode Standard of which it forms a part. Please submit corrigenda and other comments with the online reporting form [Feedback]. Related information that is useful in understanding this annex is found in Unicode Standard Annex #41, “Common References for Unicode Standard Annexes.” For the latest version of the Unicode Standard, see [Unicode]. For a list of current Unicode Technical Reports, see [Reports]. For more information about versions of the Unicode Standard, see [Versions].
    [Show full text]