DOCSLIB.ORG

Using Alternate Representations of Text for Natural Language Understanding

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Using Alternate Representations of Text for Natural Language Understanding Using Alternate Representations of Text for Natural Language Understanding Venkata sai Varada, Charith Peris, Yangsook Park, Christopher DiPersio Amazon Alexa AI, USA {vnk, perisc, yangsoop, dipersio}@amazon.com Abstract ping list म dove soap bar जोड़’ (‘add dove soap bar to my shopping list’). NLU models ex- One of the core components of voice assis- pect ASR to return the text in the above form, tants is the Natural Language Understand- ing (NLU) model. Its ability to accurately which matches with the transcription of the classify the user’s request (or “intent”) and training data for NLU models. Then, NLU recognize named entities in an utterance is model would return the action as Add Items pivotal to the success of these assistants. To Shopping List, while it also recognizes ‘dove NLU models can be challenged in some soap bar’ as the actual item name to be added languages by code-switching or morpho- to the shopping list. However, ASR could in- logical and orthographic variations. This consistently return ‘ shopping list dove work explores the possibility of improving मेरी म soap ’, where the English word ‘bar’ is the accuracy of NLU models for Indic lan- बार जोड़ guages via the use of alternate represen- recognized as a Hindi word ‘बार’. Note that tations of input text for NLU, specifically while the Hindi word ‘बार’ means something ISO-15919 and IndicSOUNDEX, a custom different than the English ‘bar’, their pronun- SOUNDEX designed to work for Indic lan- ciation is similar enough to mislead the ASR guages. We used a deep neural network model in the context of mixed-language utter- based model to incorporate the informa- ance. In this case, the NLU model should be- tion from alternate representations into the come robust against such ASR mistakes, and NLU model. We show that using alternate representations significantly improves the learn that ‘dove soap बार’ is equivalent to ‘dove overall performance of NLU models when soap bar’ in order to correctly recognize it the amount of training data is limited. as an item name. The phenomenon of code- switching is common amongst other Indic lan- 1 Introduction guages as well. Building NLU models can be more challeng- ASR inconsistencies can cause significant ing for languages that involve code-switching. challenges for NLU models by introducing new Recent times have seen a significant surge of data that the models were not trained on. interest in voice-enabled smart assistants, such Such inconsistencies can occur due to 1) code- as Amazon’s Alexa, Google Assistant, and Ap- switching in the data, especially when the in- ple’s Siri. These assistants are powered by put text contains more than one script (char- several components, which include Automatic acter set), 2) orthographic or morphological Speech Recognition (ASR) and Natural Lan- variations that exist in the input text, or 3) guage Understanding (NLU) models. The in- due to requirements for multilingual support. put to an NLU model is the text returned by In a language such as English in the United the ASR model. With this input text, there States, where code-switching is less common, are two major tasks for an NLU model: 1) In- both ASR and NLU models can perform quite tent Classification (IC), and 2) Named Entity well, as input data tend to be more consistent. Recognition (NER). However, when it comes to Indic languages Code-switching is a phenomenon in which such as Hindi, where code-switching is much two or more languages are interchanged within more common, the question arises as to which a sentence or between sentences. An exam- representation of the input text would work ple of code-switching from Hindi is ‘मेरी shop- best for NLU models. 1 Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 1–10 July 9, 2020. c 2020 Association for Computational Linguistics There is ongoing research on solving mod- on one high-resource Indic language (Hindi) eling tasks for data with code-switching. In and three low-resource Indic languages (Tamil, Geetha et al. (2018), the authors explore using Marathi, Telugu). In Section 2, we describe character and word level embeddings for NER the two alternate representations of text that tasks. Some of this research focuses on us- we explore. In Section 3, we describe our data, ing alternate representations of text for NLU model architecture, and detail our experimen- modeling. For example, Johnson et al. (2019) tal setup. In Section 4, we present our results explore the use of cross-lingual transfer learn- followed by our conclusions in Section 5. ing from English to Japanese for NER by ro- manizing Japanese input into a Latin-based 2 Alternate representations of text character set to overcome the script differences between the language pair. Zhang and Le- Using data transcribed in the original script of Cun (2017) has shown using romanization for a language can cause problems for both mono- Japanese text for sentiment classification hurt lingual and multilingual NLU models. For a the performance of the monolingual model. monolingual model in a language where code- In this paper, we explore the possibility of switching, orthographic variations, or rich mitigating the problems related to ASR in- morphological inflections are common, NLU consistency and code-switching in our input models may not be able to perform well on data by using two alternate representations of all the variations, depending on the frequency text in our NLU model: ISO-15919 and In- of these variations in the training data. For dicSOUNDEX. ISO-159191 was developed as multilingual models, words with similar sound a standardized Latin-based representation for and meaning across different languages (e.g., Indic languages and scripts. SOUNDEX is an loanwords, common entities) cannot be cap- algorithm that provides phonetic-like represen- tured if the words are written in their original tations of words. Most work on SOUNDEX script. For example, the same proper noun ‘tel- algorithms has focused primarily on monolin- ugu’ is written as ‘तेलुगु’ in Hindi, as ‘ெத’ gual solutions. One of the best known imple- in Tamil, and as ‘లుగు’ in Telugu. Although mentations with a multilingual component is they sound similar and mean the same thing, Beider-Morse Phonetic Matching Beider and NLU model will see them as unrelated tokens Morse (2010); however, it only identifies the if they are represented in their original scripts language in which a given word is written in the input data. to choose which pronunciation rules to apply. From the NLU point of view, a text repre- Other attempts at multilingual SOUNDEX sentation that can minimize variations of the algorithms, particularly for Indic languages, same or similar words within a language and were smaller studies focused on two Indic lan- across different languages would be beneficial guages with or without English as a third lan- for both IC and NER tasks. In this section, guage. Chaware and Rao (2012) developed a we explore two alternative ways of text rep- custom SOUNDEX algorithm for monolingual resentations for Indic languages: ISO-15919 Hindi and Marathi word pair matching. Shah and a SOUNDEX-based algorithm, which we and Singh (2014) describe an actual multilin- call IndicSOUNDEX. Compared to using the gual SOUNDEX implementation designed to original scripts, these two alternatives can rep- cover Hindi, Gujarati, and English, which, in resent the variants of the same word or root addition to the actual algorithm implementa- in the same way. For example, in the origi- tion, was aided by a matching threshold declar- nal Hindi script (i.e., Devanagari), the word ing two conversions a match even if they dif- for ‘volume/voice’ can be represented in two fered in up to two characters. forms: ‘आवाज़’ and ‘आवाज’. These two forms, In this study, we use ISO-15919 and Indic- however, are uniformly represented as ‘āvāj’ SOUNDEX representations of text in a deep in ISO-15919 and as the string ‘av6’ in Indic- neural network (DNN) to perform multi-task SOUNDEX. Similarly, the English word ‘list’ modeling of IC and NER. We experiment may be transcribed as ‘list’ or as ‘’ in Tel- ugu; however, they map to the same Indic- 1https://en.wikipedia.org/wiki/ISO_15919 SOUNDEX representation, ‘ls8’. 2 2.1 ISO-15919 3 Experimental Setup ISO-15919 is a standardized representation of 3.1 Data Indic scripts based on Latin characters, which For our experiments, we chose datasets from maps the corresponding Indic characters onto four Indic languages: Hindi, Tamil, Marathi, the same Latin character. For example, ‘क’ in and Telugu. For Hindi, we use an inter- Devanagari, ‘క’ in Telugu, and ‘க’ in Tamil are nal large-scale real-world dataset; for the all represented by ‘k’ in the ISO-15919 stan- other three languages, we use relatively small dard. Consequently, ISO-15919 can be used datasets collected and annotated by third as a neutral common representation between party vendors. We perform a series of ex- Indic scripts. However, ISO-15919 has some periments to evaluate the use of the alternate downsides. Indic scripts often rely on implicit representations of text, ISO-15919 and Indic- vowels which are not represented in the orthog- SOUNDEX, described in Section 2. raphy, which means they cannot be reliably Our Hindi training dataset consists of 6M added to a transliterated word. Additionally, data. We separate a portion (1̃%) of the data Indic scripts have a character called a halant, into an independent test set. We execute a or virama, which is used to suppress an in- stratified split on the remainder, based onin- herent vowel. This character, although usu- tent, and choose 10% of this data for validation ally included orthographically in a word, does and the rest as training data.
Load more

Recommended publications
  • Technical Reference Manual for the Standardization of Geographical Names United Nations Group of Experts on Geographical Names
    ST/ESA/STAT/SER.M/87 Department of Economic and Social Affairs Statistics Division Technical reference manual for the standardization of geographical names United Nations Group of Experts on Geographical Names United Nations New York, 2007 The Department of Economic and Social Affairs of the United Nations Secretariat is a vital interface between global policies in the economic, social and environmental spheres and national action. The Department works in three main interlinked areas: (i) it compiles, generates and analyses a wide range of economic, social and environmental data and information on which Member States of the United Nations draw to review common problems and to take stock of policy options; (ii) it facilitates the negotiations of Member States in many intergovernmental bodies on joint courses of action to address ongoing or emerging global challenges; and (iii) it advises interested Governments on the ways and means of translating policy frameworks developed in United Nations conferences and summits into programmes at the country level and, through technical assistance, helps build national capacities. NOTE The designations employed and the presentation of material in the present publication do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The term “country” as used in the text of this publication also refers, as appropriate, to territories or areas. Symbols of United Nations documents are composed of capital letters combined with figures. ST/ESA/STAT/SER.M/87 UNITED NATIONS PUBLICATION Sales No.
    [Show full text]
  • Names of Countries, Their Capitals and Inhabitants
    United Nations Group of Experts on Geographical Names (UNGEGN) East Central and South-East Europe Division (ECSEED) ___________________________________________________________________________ The Nineteenth Session of the East Central and South-East Europe Division of the UNGEGN Zagreb, Croatia, 19 – 21 November 2008 Item 9 and 10 of the agenda Document Symbol: ECSEED/Session.19/2008/10 Names of countries, their capitals and inhabitants Submitted by Poland* ___________________________________________________________________________ * Prepared by Maciej Zych, Commission on Standardization of Geographical Names Outside the Republic of Poland, Poland. 19th Session of the East, Central and South-East Europe Division of the United Nations Group of Experts on Geographical Names Zagreb, 19 – 21 November 2008 Names of countries, their capitals and inhabitants Maciej Zych Commission on Standardization of Geographical Names Outside the Republic of Poland 1 Names of countries, their capitals and inhabitants In 1997 the Commission on Standardization of Geographical Names Outside the Republic of Poland published the first list of Names of countries, their capitals and inhabitants, comprising both independent countries as well as non-self-governing and autonomous territories. The second edition appeared in 2003. Numerous changes occurred in geographical names in the six years since the previous list was published. New countries and new non-self-governing and autonomous territories appeared, some countries changed their names, other their capital or its name, the Polish names for several countries and their capitals also changed as did the recommended principles for the Romanization of several languages using non-Roman systems of writing. The third edition of Names of countries, their capitals and inhabitants appeared in the end of 2007, the data it contained being updated for mid-July 2007.
    [Show full text]
  • Evitalia NORMAS ISO En El Marco De La Complejidad
    No. 7 Revitalia NORMAS ISO en el marco de la complejidad ESTEQUIOMETRIA de las relaciones humanas FRACTALIDAD en los sistemas biológicos Dirección postal Calle 82 # 102 - 79 Bogotá - Colombia Revista Revitalia Publicación trimestral Contacto [email protected] Web http://revitalia.biogestion.com.co Volumen 2 / Número 7 / Noviembre-Enero de 2021 ISSN: 2711-4635 Editor líder: Juan Pablo Ramírez Galvis. Consultor en Biogestión, NBIC y Gerencia Ambiental/de la Calidad. Globuss Biogestión [email protected] ORCID: 0000-0002-1947-5589 Par evaluador: Jhon Eyber Pazos Alonso Experto en nanotecnología, biosensores y caracterización por AFM. Universidad Central / Clúster NBIC [email protected] ORCID: 0000-0002-5608-1597 Contenido en este número Editorial p. 3 Estequiometría de las relaciones humanas pp. 5-13 Catálogo de las normas ISO en el marco de la complejidad pp. 15-28 Fractalidad en los sistemas biológicos pp. 30-37 Licencia Creative Commons CC BY-NC-ND 4.0 2 Editorial: “En armonía con lo ancestral” Juan Pablo Ramírez Galvis. Consultor en Biogestión, NBIC y Gerencia Ambiental/de la Calidad. [email protected] ORCID: 0000-0002-1947-5589 La dicotomía entre ciencia y religión proviene de la edad media, en la cual, los aspectos espirituales no podían explicarse desde el método científico, y a su vez, la matematización mecánica del universo era el único argumento que convencía a los investigadores. Sin embargo, más atrás en la línea del tiempo, los egipcios, sumerios, chinos, etc., unificaban las teorías metafísicas con las ciencias básicas para dar cuenta de los fenómenos en todas las escalas desde lo micro hasta lo macro.
    [Show full text]
  • Transliteration of Tamil and Other Indic Scripts Ram Viswanadha
    Transliteration of Tamil and Other Indic Scripts Ram Viswanadha Unicode Software Engineer IBM Globalization Center of Competency, California, USA ___________________________________________________________________________ Main points of Powerpoint presentation This talk gives an overview and discusses the issues with transliteration of Indic scripts to Latin and between different Indic scripts, e.g: Tamil-Telugu, Gujarati-Tamil, etc. It is often perceived that transliteration between different Indic scripts is straightforward because all Indic scripts have a common origin in Brahmi script. The ISCII standard is based on this similarity between scripts, and the placement of Unicode code points for each Indic script is based on an early version of ISCII. Correct transliteration between Indic scripts takes advantage of this allocation but handles special cases. The ICU implementation uses a script-neutral pivot range in Unicode. --- Agenda • What is ICU? • Terminology & Concepts • Standards for Romanization • Problems in Romanization • Problems in Inter-Indic Transliteration • Implementation approaches • Implementation in ICU •Summary •A brief overview of what International Components for Unicode (ICU) is. •Some terms and concepts which are important for the scope of this presentation are discussed •Different standards for Romanization, ISCII and ISO 15919 in particular are presented •Some problems in Romanization and Inter-Indic transliteration are discussed •Different implementation approaches, their deficiencies and how ICU’s implementation
    [Show full text]
  • A Könyvtárüggyel Kapcsolatos Nemzetközi Szabványok
    A könyvtárüggyel kapcsolatos nemzetközi szabványok 1. Állomány-nyilvántartás ISO 20775:2009 Information and documentation. Schema for holdings information 2. Bibliográfiai feldolgozás és adatcsere, transzliteráció ISO 10754:1996 Information and documentation. Extension of the Cyrillic alphabet coded character set for non-Slavic languages for bibliographic information interchange ISO 11940:1998 Information and documentation. Transliteration of Thai ISO 11940-2:2007 Information and documentation. Transliteration of Thai characters into Latin characters. Part 2: Simplified transcription of Thai language ISO 15919:2001 Information and documentation. Transliteration of Devanagari and related Indic scripts into Latin characters ISO 15924:2004 Information and documentation. Codes for the representation of names of scripts ISO 21127:2014 Information and documentation. A reference ontology for the interchange of cultural heritage information ISO 233:1984 Documentation. Transliteration of Arabic characters into Latin characters ISO 233-2:1993 Information and documentation. Transliteration of Arabic characters into Latin characters. Part 2: Arabic language. Simplified transliteration ISO 233-3:1999 Information and documentation. Transliteration of Arabic characters into Latin characters. Part 3: Persian language. Simplified transliteration ISO 25577:2013 Information and documentation. MarcXchange ISO 259:1984 Documentation. Transliteration of Hebrew characters into Latin characters ISO 259-2:1994 Information and documentation. Transliteration of Hebrew characters into Latin characters. Part 2. Simplified transliteration ISO 3602:1989 Documentation. Romanization of Japanese (kana script) ISO 5963:1985 Documentation. Methods for examining documents, determining their subjects, and selecting indexing terms ISO 639-2:1998 Codes for the representation of names of languages. Part 2. Alpha-3 code ISO 6630:1986 Documentation. Bibliographic control characters ISO 7098:1991 Information and documentation.
    [Show full text]
  • Finite-State Script Normalization and Processing Utilities: the Nisaba Brahmic Library
    Finite-state script normalization and processing utilities: The Nisaba Brahmic library Cibu Johny† Lawrence Wolf-Sonkin‡ Alexander Gutkin† Brian Roark‡ Google Research †United Kingdom and ‡United States {cibu,wolfsonkin,agutkin,roark}@google.com Abstract In addition to such normalization issues, some scripts also have well-formedness constraints, i.e., This paper presents an open-source library for efficient low-level processing of ten ma- not all strings of Unicode characters from a single jor South Asian Brahmic scripts. The library script correspond to a valid (i.e., legible) grapheme provides a flexible and extensible framework sequence in the script. Such constraints do not ap- for supporting crucial operations on Brahmic ply in the basic Latin alphabet, where any permuta- scripts, such as NFC, visual normalization, tion of letters can be rendered as a valid string (e.g., reversible transliteration, and validity checks, for use as an acronym). The Brahmic family of implemented in Python within a finite-state scripts, however, including the Devanagari script transducer formalism. We survey some com- mon Brahmic script issues that may adversely used to write Hindi, Marathi and many other South affect the performance of downstream NLP Asian languages, do have such constraints. These tasks, and provide the rationale for finite-state scripts are alphasyllabaries, meaning that they are design and system implementation details. structured around orthographic syllables (aksara)̣ as the basic unit.1 One or more Unicode characters 1 Introduction combine when rendering one of thousands of leg- The Unicode Standard separates the representation ible aksara,̣ but many combinations do not corre- of text from its specific graphical rendering: text spond to any aksara.̣ Given a token in these scripts, is encoded as a sequence of characters, which, at one may want to (a) normalize it to a canonical presentation time are then collectively rendered form; and (b) check whether it is a well-formed into the appropriate sequence of glyphs for display.
    [Show full text]
  • Inventory of Romanization Tools
    Inventory of Romanization Tools Standards Intellectual Management Office Library and Archives Canad Ottawa 2006 Inventory of Romanization Tools page 1 Language Script Romanization system for an English Romanization system for a French Alternate Romanization system catalogue catalogue Amharic Ethiopic ALA-LC 1997 BGN/PCGN 1967 UNGEGN 1967 (I/17). http://www.eki.ee/wgrs/rom1_am.pdf Arabic Arabic ALA-LC 1997 ISO 233:1984.Transliteration of Arabic BGN/PCGN 1956 characters into Latin characters NLC COPIES: BS 4280:1968. Transliteration of Arabic characters NL Stacks - TA368 I58 fol. no. 00233 1984 E DMG 1936 NL Stacks - TA368 I58 fol. no. DIN-31635, 1982 00233 1984 E - Copy 2 I.G.N. System 1973 (also called Variant B of the Amended Beirut System) ISO 233-2:1993. Transliteration of Arabic characters into Latin characters -- Part 2: Lebanon national system 1963 Arabic language -- Simplified transliteration Morocco national system 1932 Royal Jordanian Geographic Centre (RJGC) System Survey of Egypt System (SES) UNGEGN 1972 (II/8). http://www.eki.ee/wgrs/rom1_ar.pdf Update, April 2004: http://www.eki.ee/wgrs/ung22str.pdf Armenian Armenian ALA-LC 1997 ISO 9985:1996. Transliteration of BGN/PCGN 1981 Armenian characters into Latin characters Hübschmann-Meillet. Assamese Bengali ALA-LC 1997 ISO 15919:2001. Transliteration of Hunterian System Devanagari and related Indic scripts into Latin characters UNGEGN 1977 (III/12). http://www.eki.ee/wgrs/rom1_as.pdf 14/08/2006 Inventory of Romanization Tools page 2 Language Script Romanization system for an English Romanization system for a French Alternate Romanization system catalogue catalogue Azerbaijani Arabic, Cyrillic ALA-LC 1997 ISO 233:1984.Transliteration of Arabic characters into Latin characters.
    [Show full text]
  • Transliteration of <Script Name>
    Transliteration of Bengali, Assamese & Manipuri 1/5 BENGALI, ASSAMESE & MANIPURI Script: Bengali* ISO 15919 UN ALA-LC 2001(1.0) 1977(2.0) 1997(3.0) Vowels অ ◌ a a a আ ◌া ā ā ā ই ি◌ i i i ঈ ◌ী ī ī ī উ ◌ু u u u ঊ ◌ূ ū ū ū ঋ ◌ৃ r̥ ṛ r̥ ৠ ◌ৄ r̥̄ — r̥̄ ঌ ◌ৢ l̥ — l̥ ৡ ◌ৣ A l̥ ̄ — — এ ে◌ e(1.1) e e ঐ ৈ◌ ai ai ai ও ে◌া o(1.1) o o ঔ ে◌ৗ au au au অ�া a:yā — — Nasalizations ◌ং anunāsika ṁ(1.2) ṁ ṃ ◌ঁ candrabindu m̐ (1.2) m̐ n̐, m̐ (3.1) Miscellaneous ◌ঃ bisarga ḥ ḥ ḥ ◌্ hasanta vowelless vowelless vowelless (1.3) ঽ abagraha Ⓑ :’ — ’ ৺ isshara — — — Consonants ক ka ka ka খ kha kha kha গ ga ga ga ঘ gha gha gha ঙ ṅa ṅa ṅa চ ca cha ca ছ cha chha cha জ ja ja ja ঝ jha jha jha ঞ ña ña ña ট ṭa ṭa ṭa ঠ ṭha ṭha ṭha ড ḍa ḍa ḍa ঢ ḍha ḍha ḍha ণ ṇa ṇa ṇa ত ta ta ta Thomas T. Pedersen – transliteration.eki.ee Rev. 2, 2005-07-21 Transliteration of Bengali, Assamese & Manipuri 2/5 ISO 15919 UN ALA-LC 2001(1.0) 1977(2.0) 1997(3.0) থ tha tha tha দ da da da ধ dha dha dha ন na na na প pa pa pa ফ pha pha pha ব ba ba ba(3.2) ভ bha bha bha ম ma ma ma য ya ja̱ ya র ⒷⓂ ra ra ra ৰ Ⓐ ra ra ra ল la la la ৱ ⒶⓂ va va wa শ śa sha śa ষ ṣa ṣha sha স sa sa sa হ ha ha ha ড় ṛa ṙa ṛa ঢ় ṛha ṙha ṛha য় ẏa ya ẏa জ় za — — ব় wa — — ক় Ⓑ qa — — খ় Ⓑ k̲h̲a — — গ় Ⓑ ġa — — ফ় Ⓑ fa — — Adscript consonants ◌� ya-phala -ya(1.4) -ya ẏa ৎ khanda-ta -t -t ṯa ◌� repha r- r- r- ◌� baphala -b -b -b ◌� raphala -r -r -r Vowel ligatures (conjuncts)C � gu � ru � rū � Ⓐ ru � rū � śu � hr̥ � hu � tru � trū � ntu � lgu Thomas T.
    [Show full text]
  • Massively Multilingual Creation of Language Tools Using Interlinear Glossed Text
    From Aari to Zulu: Massively Multilingual Creation of Language Tools using Interlinear Glossed Text Ryan Georgi A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Washington 2016 Reading Committee: Fei Xia, Chair William D. Lewis Emily M. Bender Program Authorized to Offer Degree: UW Department of Linguistics c Copyright 2016 Ryan Georgi University of Washington Abstract From Aari to Zulu: Massively Multilingual Creation of Language Tools using Interlinear Glossed Text Ryan Georgi Chair of the Supervisory Committee: Professor Fei Xia Linguistics This dissertation examines the suitability of Interlinear Glossed Text (IGT) as a computa- tional, semi-structured resource for creating NLP tools for resource-poor languages, with a focus on the tasks of word alignment, part-of-speech (POS) tagging, and dependency pars- ing. The creation of a massively multilingual database of IGT instances called the Online Database of INterlinear text (Odin) made possible the potential for creating tools to har- ness this particular data format on a large scale. Xia and Lewis (2007); Lewis and Xia (2008) demonstrated the potential of using IGT instances from Odin to answer some typological questions such as basic word order for a large number of languages by means of utilizing the language{gloss{translation line structure of IGT instances to bootstrap word alignment, and consequentially syntactic projection. This dissertation seeks to perform a thorough investigation as to the potential for creating these NLP tools for endangered or otherwise resource-poor languages with nothing more than the IGT instances found in Odin. After introducing the IGT data type and the particulars of the resources that will be used (Sections 3.1 to 4.4), this thesis presents each task in detail.
    [Show full text]
  • Proceedings of the 2Nd Workshop on Natural Language Processing for Conversational AI, Pages 1–10 July 9, 2020
    ACL 2020 NLP for Conversational AI Proceedings of the 2nd Workshop July 9, 2020 c 2020 The Association for Computational Linguistics Order copies of this and other ACL proceedings from: Association for Computational Linguistics (ACL) 209 N. Eighth Street Stroudsburg, PA 18360 USA Tel: +1-570-476-8006 Fax: +1-570-476-0860 [email protected] 978-1-952148-08-8 ii Introduction Welcome to the ACL 2020 Workshop on NLP for Conversational AI. Ever since the invention of the intelligent machine, hundreds and thousands of mathematicians, linguists, and computer scientists have dedicated their career to empowering human-machine communication in natural language. Although the idea is finally around the corner with a proliferation of virtual personal assistants such as Siri, Alexa, Google Assistant, and Cortana, the development of these conversational agents remains difficult and there still remain plenty of unanswered questions and challenges. Conversational AI is hard because it is an interdisciplinary subject. Initiatives were started in different research communities, from Dialogue State Tracking Challenges to NIPS Conversational Intelligence Challenge live competition and the Amazon Alexa prize. However, various fields within the NLP community, such as semantic parsing, coreference resolution, sentiment analysis, question answering, and machine reading comprehension etc. have been seldom evaluated or applied in the context of conversational AI. The goal of this workshop is to bring together NLP researchers and practitioners in different fields, alongside experts in speech and machine learning, to discuss the current state-of-the-art and new approaches, to share insights and challenges, to bridge the gap between academic research and real- world product deployment, and to shed the light on future directions.
    [Show full text]
  • The Issue of the Assamese Script Misrepresentation / Nonrepresentation in the International Standards
    Dated Guwahati the 2nd of August 2014 To Mr Tarun Gogoi Hon©ble Chief Minister of Assam Guwahati, Assam Subject : Petition for urgent attention and steps regarding two important issues concerning the well being and the interests of the Assamese people viz. 1. Issue of misrepresentation/nonrepresentation of the Assamese script in the various International Standards. 2. Issue of the welfare of beleaguered historically dispersed Assamese people in Myanmar and Bangladesh. Sir, With due respect we wish to draw your attention to the two important issues concerning the well being and the interests of the Assamese community and pray for urgent steps to expedite the matters. THE ISSUE OF THE ASSAMESE SCRIPT MISREPRESENTATION / NONREPRESENTATION IN THE INTERNATIONAL STANDARDS The status of Assamese script in the International Standards are as follows : 1 A. ISO 15924 International Standard for Names of the Scripts Status of Assamese script in ISO 15924 : Not included Hence there is nothing called Assamese script in International Standards B. ISO 10646 = Unicode Standard Assamese script misrepresented as Bengali there, meaning Assamese language as per Unicode standard is written by Bengali script. C. ISO 15919 International Standard for Indic Scripts Transliteration. There is no Transliteration Standard for Assamese in this standard, because of the wrong view that Bengali transliteration scheme will work for Assamese as well. D. ALA-LC Romanization Table Transliteration charts maintained by US Library of Congress. The Bengali Romanization scheme is being presented there as the Assamese transliteration scheme. CONSEQUENCES OF THE MISREPRESENTATIONS/NONREPRESENTATIONS OF THE ASSAMESE SCRIPT IN INTERNATIONAL STANDARDS 2 1. Loss of identity of the Assamese Script 2.
    [Show full text]
  • A Könyvtárüggyel Kapcsolatos Nemzetközi Szabványok
    A könyvtárüggyel kapcsolatos nemzetközi szabványok 1. Bibliográfiai feldolgozás és adatcsere, transzliteráció ISO 10754:1996 Information and documentation. Extension of the Cyrillic alphabet coded character set for non-Slavic languages for bibliographic information interchange ISO 11940:1998 Information and documentation. Transliteration of Thai ISO 11940-2:2007 Information and documentation. Transliteration of Thai characters into Latin characters. Part 2: Simplified transcription of Thai language ISO 15919:2001 Information and documentation. Transliteration of Devanagari and related Indic scripts into Latin characters ISO 15924:2004 Information and documentation. Codes for the representation of names of scripts ISO 21127:2014 Information and documentation. A reference ontology for the interchange of cultural heritage information ISO 233:1984 Documentation. Transliteration of Arabic characters into Latin characters ISO 233-2:1993 Information and documentation. Transliteration of Arabic characters into Latin characters. Part 2: Arabic language. Simplified transliteration ISO 233-3:1999 Information and documentation. Transliteration of Arabic characters into Latin characters. Part 3: Persian language. Simplified transliteration ISO 25577:2013 Information and documentation. MarcXchange ISO 259:1984 Documentation. Transliteration of Hebrew characters into Latin characters ISO 259-2:1994 Information and documentation. Transliteration of Hebrew characters into Latin characters. Part 2. Simplified transliteration ISO 3602:1989 Documentation. Romanization of Japanese (kana script) ISO 5963:1985 Documentation. Methods for examining documents, determining their subjects, and selecting indexing terms ISO 639-2:1998 Codes for the representation of names of languages. Part 2. Alpha-3 code ISO 6630:1986 Documentation. Bibliographic control characters ISO 7098:2015 Information and documentation. Romanization of Chinese ISO 843:1997 Information and documentation. Conversion of Greek characters into Latin characters ISO 8459:2009 Information and documentation.
    [Show full text]