DOCSLIB.ORG

Finite-State Script Normalization and Processing Utilities: the Nisaba Brahmic Library

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Finite-State Script Normalization and Processing Utilities: the Nisaba Brahmic Library Finite-state script normalization and processing utilities: The Nisaba Brahmic library Cibu Johny† Lawrence Wolf-Sonkin‡ Alexander Gutkin† Brian Roark‡ Google Research †United Kingdom and ‡United States {cibu,wolfsonkin,agutkin,roark}@google.com Abstract In addition to such normalization issues, some scripts also have well-formedness constraints, i.e., This paper presents an open-source library for efficient low-level processing of ten ma- not all strings of Unicode characters from a single jor South Asian Brahmic scripts. The library script correspond to a valid (i.e., legible) grapheme provides a flexible and extensible framework sequence in the script. Such constraints do not ap- for supporting crucial operations on Brahmic ply in the basic Latin alphabet, where any permuta- scripts, such as NFC, visual normalization, tion of letters can be rendered as a valid string (e.g., reversible transliteration, and validity checks, for use as an acronym). The Brahmic family of implemented in Python within a finite-state scripts, however, including the Devanagari script transducer formalism. We survey some com- mon Brahmic script issues that may adversely used to write Hindi, Marathi and many other South affect the performance of downstream NLP Asian languages, do have such constraints. These tasks, and provide the rationale for finite-state scripts are alphasyllabaries, meaning that they are design and system implementation details. structured around orthographic syllables (aksara)̣ as the basic unit.1 One or more Unicode characters 1 Introduction combine when rendering one of thousands of leg- The Unicode Standard separates the representation ible aksara,̣ but many combinations do not corre- of text from its specific graphical rendering: text spond to any aksara.̣ Given a token in these scripts, is encoded as a sequence of characters, which, at one may want to (a) normalize it to a canonical presentation time are then collectively rendered form; and (b) check whether it is a well-formed into the appropriate sequence of glyphs for display. sequence of aksara.̣ This can occasionally result in many-to-one map- Brahmic scripts are heavily used across South pings, where several distinctly-encoded strings can Asia and have official status in India, Bangladesh, result in identical display. For example, Latin Nepal, Sri Lanka and beyond (Cardona and Jain, script letters with diacritics such as “é” can gener- 2007; Steever, 2019). Despite evident progress ally be encoded as either: (a) a pair of the base let- in localization standards (Unicode Consortium, ter (e.g., “e” which is U+0065 from Unicode’s Ba- 2019) and improvements in associated technolo- sic Latin block, corresponding to ASCII) and a dia- gies such as input methods (Hinkle et al., 2013) and critic (in this case U+0301 from the Combining Dia- character recognition (Pal et al., 2012), Brahmic critical Marks block); or (b) a single character that script processing still poses important challenges represents the grapheme directly (U+00E9 from the due to the inherent differences between these writ- Latin-1 Supplement Unicode block). Both encod- ing systems and those which historically have been ings yield visually identical text, hence text is of- more dominant in information technology (Sinha, ten normalized to a conventionalized normal form, 2009; Bhattacharyya et al., 2019). such as the well-known Normalization Form C In this paper, we present Nisaba, an open-source (NFC), so that visually identical words are mapped software library,2 which provides processing utili- to a conventionalized representative of their equiv- ties for ten major Brahmic scripts of South Asia: alence class for downstream processing. Critically, Bengali, Devanagari, Gujarati, Gurmukhi, Kan- NFC normalization falls far short of a complete nada, Malayalam, Oriya (Odia), Sinhala, Tamil, handling of such many-to-one phenomena in Uni- 1See §3 for details on the scripts. code. 2https://github.com/google-research/nisaba/ 14 Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 14–23 April 19 - 23, 2021. ©2021 Association for Computational Linguistics and Telugu. In addition to string normaliza- Murikinati et al., 2020), named entity recogni- tion and well-formedness processing, the library tion (Huang et al., 2019) and part-of-speech tag- also includes utilities for the deterministic and re- ging (Cardenas et al., 2019). versible romanization of these scripts, i.e., translit- Similar to the software mentioned above, our li- eration from each script to and from the Latin brary does provide romanization, but unlike some script (Wellisch, 1978). While the resulting roman- of the packages, such as URoman, we guarantee izations are standardized in a way that may or may reversibility from Latin back to the native script. not correspond to how native speakers tend to ro- Similar to others we do not focus on faithful in- manize the text in informal communication (see, vertible transliteration of named entities which e.g., Roark et al., 2020), such a default romaniza- typically requires model-based approaches (Se- tion can permit easy inspection of an approximate quiera et al., 2014). Unlike the IndicNLP pack- version of the linguistic strings for those who read age, our software does not provide morphologi- the Latin script but not the specific Brahmic script cal analysis, but instead offers significantly richer being examined. script normalization capabilities than other pack- As a whole, the library provides important utili- ages. These capabilities are functionally sepa- ties for language processing applications of South rated into normalization to Normalization Form Asian languages using Brahmic scripts. The de- C (NFC) and visual normalization. Additionally, sign is based on the observation that, while there our library provides extensive script-specific well- are considerable superficial differences between formedness grammars. Finally, in contrast to these these scripts, they follow the same encoding model other approaches, grammars in our library are in Unicode, and maintain a very similar char- maintained separately from the code for compila- acter repertoire having evolved from the same tion and application, allowing for maintenance of source — the Brāhmī script (Salomon, 1996; Fe- existing scripts and languages plus extension to dorova, 2012). This observation lends itself to the new ones without having to modify any code. This script-agnostic design (outlined in §4) that, unlike is particularly important given that Unicode stan- other approaches reviewed in §2, is based on the dards do change over time and there remain many weighted finite state transducer (WFST) formal- languages left to cover. ism (Mohri, 2004). The details of our system are To the best of our knowledge this is the first provided in §5. publicly available general finite-state grammar ap- proach for low-level processing of multiple Brah- 2 Related Work mic scripts since the early formal syntactic work by Datta (1984) and is the first such library de- The computational processing of Brahmic scripts signed based on an observation by Sproat (2003) is not a new topic, with the first applications that the fundamental organizing principles of the dating back to the early formal syntactic work Brahmic scripts can be algebraically formalized. by Datta (1984). With an increased focus on the In particular, all the core components of our li- South Asian languages within the NLP commu- brary (inverse romanization, normalization and nity, facilitated by advances in machine learning well-formedness) are compactly and efficiently and the increased availability of relevant corpora, represented as finite state transducers. Such for- multiple script processing solutions have emerged. malization lends itself particularly well to run-time Some of these toolkits, such as statistical ma- or offline integration with any finite state process- chine translation-based Brahmi-Net (Kunchukut- ing pipeline, such as decoder components of in- tan et al., 2015), are model-based, while oth- put methods (Ouyang et al., 2017; Hellsten et al., ers, such as URoman (Hermjakob et al., 2018), 2017), text normalization for automatic speech IndicNLP (Kunchukuttan, 2020), and Akshar- recognition and text-to-speech synthesis (Zhang mukha (Rajan, 2020), employ rules. The main fo- et al., 2019), among other natural language and cus of these libraries is script conversion and ro- speech applications. manization. In this capacity they were success- fully employed in diverse downstream multilin- 3 Brahmic Scripts: An Overview gual NLP tasks such as neural machine transla- tion (Zhang et al., 2020; Amrhein and Sennrich, The scripts of interest have evolved from the an- 2020), morphological analysis (Hauer et al., 2019; cient Brāhmī writing system that was recorded 15 Name Id iv dv c co Visual Legacy sequence NFC normalized Bengali BENG 16 13 43 5 ऩ NA NUKTA (U+0928 U+093C) NNNA (U+0929) Devanagari DEVA 19 17 45 4 क़ QA (U+0958) KA NUKTA (U+0915 U+093C) Gujarati GUJR 16 15 39 5 Gurmukhi GURU 12 9 39 8 Table 2: NFC examples for Devanagari. Kannada KNDA 17 15 39 3 Malayalam MLYM 16 16 38 10 bols, like anusvara, chandrabindu, and visarga, Oriya ORYA 14 13 38 5 Sinhala SINH 18 17 41 2 which modify the aksarạ as a whole (and follow Tamil TAML 12 11 27 1 and vowel signs in the memory representation). Fi- Telugu TELU 16 15 38 5 nally, there is a class of special characters, such as the religious symbol Om ॐ, that behave like inde- Table 1: Sizes of core graphemic classes: Independent pendent aksara.̣ 4 vowels (iv), dependent vowel diacritics (dv), conso- nants ( ), coda symbols ( ). c co Unicode Normalization Unicode defines sev- from the 3rd century BCE and fell out of use eral normalization forms which are used for check- by the 5th century CE (Salomon, 1996; Strauch, ing whether the two Unicode strings are equiv- 2012; Fedorova, 2012). The main unit of lin- alent to each other (Unicode Consortium, 2019).
Load more

Recommended publications
  • Cross-Language Framework for Word Recognition and Spotting of Indic Scripts
    Cross-language Framework for Word Recognition and Spotting of Indic Scripts aAyan Kumar Bhunia, bPartha Pratim Roy*, aAkash Mohta, cUmapada Pal aDept. of ECE, Institute of Engineering & Management, Kolkata, India bDept. of CSE, Indian Institute of Technology Roorkee India cCVPR Unit, Indian Statistical Institute, Kolkata, India bemail: [email protected], TEL: +91-1332-284816 Abstract Handwritten word recognition and spotting of low-resource scripts are difficult as sufficient training data is not available and it is often expensive for collecting data of such scripts. This paper presents a novel cross language platform for handwritten word recognition and spotting for such low-resource scripts where training is performed with a sufficiently large dataset of an available script (considered as source script) and testing is done on other scripts (considered as target script). Training with one source script and testing with another script to have a reasonable result is not easy in handwriting domain due to the complex nature of handwriting variability among scripts. Also it is difficult in mapping between source and target characters when they appear in cursive word images. The proposed Indic cross language framework exploits a large resource of dataset for training and uses it for recognizing and spotting text of other target scripts where sufficient amount of training data is not available. Since, Indic scripts are mostly written in 3 zones, namely, upper, middle and lower, we employ zone-wise character (or component) mapping for efficient learning purpose. The performance of our cross- language framework depends on the extent of similarity between the source and target scripts.
    [Show full text]
  • Grammatica Et Verba Glamor and Verve
    “GippertOVprint” — 2014/4/24 — 22:14 — page 81 —#1 Grammatica et verba Glamor and verve Studies in South Asian, historical, and Indo-European linguistics in honor of Hans Henrich Hock on the occasion of his seventy-fifth birthday edited by Shu-Fen Chen and Benjamin Slade Beech Stave Press Ann Arbor New York • “GippertOVprint” — 2014/4/24 — 22:14 — page 82 —#2 © Beech Stave Press, Inc. All rights reserved. No part of this publication may be reproduced, translated, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission from the publisher. Typeset with LATEX using the Galliard typeface designed by Matthew Carter and Greek Old Face by Ralph Hancock. The typeface on the cover is Post Hock by Steve Peter. Library of Congress Cataloging-in-Publication Data Grammatica et verba : glamor and verve : studies in South Asian, historical, and Indo- European linguistics in honor of Hans Henrich Hock on the occasion of his seventy- fifth birthday / edited by Shu-Fen Chen and Benjamin Slade. pages cm Includes bibliographical references. ISBN ---- (alk. paper) . Indo-European languages. Lexicography. Historical linguistics. I. Hock, Hans Henrich, - honoree. II. Chen, Shu-Fen, editor of compilation. III. Slade, Ben- jamin, editor of compilation. –dc Printed in the United States of America “GippertOVprint” — 2014/4/24 — 22:14 — page 83 —#3 TableHHHHHH ofHHHHHHHHHHHHHHH ContentsHHHHHHHHHHHH Preface . vii Bibliography of Hans Henrich Hock . ix List of Contributors . xxi , Traces of Archaic Human Language Structure Anvita Abbi in the Great Andamanese Language. , A Study of Punctuation Errors in the Chinese Diamond Sutra Shu-Fen Chen Based on Sanskrit Texts .
    [Show full text]
  • SC22/WG20 N896 L2/01-476 Universal Multiple-Octet Coded Character Set International Organization for Standardization Organisation Internationale De Normalisation
    SC22/WG20 N896 L2/01-476 Universal multiple-octet coded character set International organization for standardization Organisation internationale de normalisation Title: Ordering rules for Khmer Source: Kent Karlsson Date: 2001-12-19 Status: Expert contribution Document type: Working group document Action: For consideration by the UTC and JTC 1/SC 22/WG 20 1 Introduction The Khmer script in Unicode/10646 uses conjoining characters, just like the Indic scripts and Hangul. An alternative that is suitable (and much more elegant) for Khmer, but not for Indic scripts, would have been to have combining- below (and sometimes a bit to the side) consonants and combining-below independent vowels, similar to the combining-above Latin letters recently encoded, as well as how Tibetan is handled. However that is not the chosen solution, which instead uses a combining character (COENG) that makes characters conjoin (glue) like it is done for Indic (Brahmic) scripts and like Hangul Jamo, though the latter does not have a separate gluer character. In the Khmer script, using the COENG based approach, the words are formed from orthographic syllables, where an orthographic syllable has the following structure [add ligation control?]: Khmer-syllable ::= (K H)* K M* where K is a Khmer consonant (most with an inherent vowel that is pronounced only if there is no consonant, independent vowel, or dependent vowel following it in the orthographic syllable) or a Khmer independent vowel, H is the invisible Khmer conjoint former COENG, M is a combining character (including COENG, though that would be a misspelling), in particular a combining Khmer vowel (noted A below) or modifier sign.
    [Show full text]
  • Technical Reference Manual for the Standardization of Geographical Names United Nations Group of Experts on Geographical Names
    ST/ESA/STAT/SER.M/87 Department of Economic and Social Affairs Statistics Division Technical reference manual for the standardization of geographical names United Nations Group of Experts on Geographical Names United Nations New York, 2007 The Department of Economic and Social Affairs of the United Nations Secretariat is a vital interface between global policies in the economic, social and environmental spheres and national action. The Department works in three main interlinked areas: (i) it compiles, generates and analyses a wide range of economic, social and environmental data and information on which Member States of the United Nations draw to review common problems and to take stock of policy options; (ii) it facilitates the negotiations of Member States in many intergovernmental bodies on joint courses of action to address ongoing or emerging global challenges; and (iii) it advises interested Governments on the ways and means of translating policy frameworks developed in United Nations conferences and summits into programmes at the country level and, through technical assistance, helps build national capacities. NOTE The designations employed and the presentation of material in the present publication do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The term “country” as used in the text of this publication also refers, as appropriate, to territories or areas. Symbols of United Nations documents are composed of capital letters combined with figures. ST/ESA/STAT/SER.M/87 UNITED NATIONS PUBLICATION Sales No.
    [Show full text]
  • The Festvox Indic Frontend for Grapheme-To-Phoneme Conversion
    The Festvox Indic Frontend for Grapheme-to-Phoneme Conversion Alok Parlikar, Sunayana Sitaram, Andrew Wilkinson and Alan W Black Carnegie Mellon University Pittsburgh, USA aup, ssitaram, aewilkin, [email protected] Abstract Text-to-Speech (TTS) systems convert text into phonetic pronunciations which are then processed by Acoustic Models. TTS frontends typically include text processing, lexical lookup and Grapheme-to-Phoneme (g2p) conversion stages. This paper describes the design and implementation of the Indic frontend, which provides explicit support for many major Indian languages, along with a unified framework with easy extensibility for other Indian languages. The Indic frontend handles many phenomena common to Indian languages such as schwa deletion, contextual nasalization, and voicing. It also handles multi-script synthesis between various Indian-language scripts and English. We describe experiments comparing the quality of TTS systems built using the Indic frontend to grapheme-based systems. While this frontend was designed keeping TTS in mind, it can also be used as a general g2p system for Automatic Speech Recognition. Keywords: speech synthesis, Indian language resources, pronunciation 1. Introduction in models of the spectrum and the prosody. Another prob- lem with this approach is that since each grapheme maps Intelligible and natural-sounding Text-to-Speech to a single “phoneme” in all contexts, this technique does (TTS) systems exist for a number of languages of the world not work well in the case of languages that have pronun- today. However, for low-resource, high-population lan- ciation ambiguities. We refer to this technique as “Raw guages, such as languages of the Indian subcontinent, there Graphemes.” are very few high-quality TTS systems available.
    [Show full text]
  • Names of Countries, Their Capitals and Inhabitants
    United Nations Group of Experts on Geographical Names (UNGEGN) East Central and South-East Europe Division (ECSEED) ___________________________________________________________________________ The Nineteenth Session of the East Central and South-East Europe Division of the UNGEGN Zagreb, Croatia, 19 – 21 November 2008 Item 9 and 10 of the agenda Document Symbol: ECSEED/Session.19/2008/10 Names of countries, their capitals and inhabitants Submitted by Poland* ___________________________________________________________________________ * Prepared by Maciej Zych, Commission on Standardization of Geographical Names Outside the Republic of Poland, Poland. 19th Session of the East, Central and South-East Europe Division of the United Nations Group of Experts on Geographical Names Zagreb, 19 – 21 November 2008 Names of countries, their capitals and inhabitants Maciej Zych Commission on Standardization of Geographical Names Outside the Republic of Poland 1 Names of countries, their capitals and inhabitants In 1997 the Commission on Standardization of Geographical Names Outside the Republic of Poland published the first list of Names of countries, their capitals and inhabitants, comprising both independent countries as well as non-self-governing and autonomous territories. The second edition appeared in 2003. Numerous changes occurred in geographical names in the six years since the previous list was published. New countries and new non-self-governing and autonomous territories appeared, some countries changed their names, other their capital or its name, the Polish names for several countries and their capitals also changed as did the recommended principles for the Romanization of several languages using non-Roman systems of writing. The third edition of Names of countries, their capitals and inhabitants appeared in the end of 2007, the data it contained being updated for mid-July 2007.
    [Show full text]
  • Schwa Deletion: Investigating Improved Approach for Text-To-IPA System for Shiri Guru Granth Sahib
    ISSN (Online) 2278-1021 ISSN (Print) 2319-5940 International Journal of Advanced Research in Computer and Communication Engineering Vol. 4, Issue 4, April 2015 Schwa Deletion: Investigating Improved Approach for Text-to-IPA System for Shiri Guru Granth Sahib Sandeep Kaur1, Dr. Amitoj Singh2 Pursuing M.E, CSE, Chitkara University, India 1 Associate Director, CSE, Chitkara University , India 2 Abstract: Punjabi (Omniglot) is an interesting language for more than one reasons. This is the only living Indo- Europen language which is a fully tonal language. Punjabi language is an abugida writing system, with each consonant having an inherent vowel, SCHWA sound. This sound is modifiable using vowel symbols attached to consonant bearing the vowel. Shri Guru Granth Sahib is a voluminous text of 1430 pages with 511,874 words, 1,720,345 characters, and 28,534 lines and contains hymns of 36 composers written in twenty-two languages in Gurmukhi script (Lal). In addition to text being in form of hymns and coming from so many composers belonging to different languages, what makes the language of Shri Guru Granth Sahib even more different from contemporary Punjabi. The task of developing an accurate Letter-to-Sound system is made difficult due to two further reasons: 1. Punjabi being the only tonal language 2. Historical and Cultural circumstance/period of writings in terms of historical and religious nature of text and use of words from multiple languages and non-native phonemes. The handling of schwa deletion is of great concern for development of accurate/ near perfect system, the presented work intend to report the state-of-the-art in terms of schwa deletion for Indian languages, in general and for Gurmukhi Punjabi, in particular.
    [Show full text]
  • Assessment of Options for Handling Full Unicode Character Encodings in MARC21 a Study for the Library of Congress
    1 Assessment of Options for Handling Full Unicode Character Encodings in MARC21 A Study for the Library of Congress Part 1: New Scripts Jack Cain Senior Consultant Trylus Computing, Toronto 1 Purpose This assessment intends to study the issues and make recommendations on the possible expansion of the character set repertoire for bibliographic records in MARC21 format. 1.1 “Encoding Scheme” vs. “Repertoire” An encoding scheme contains codes by which characters are represented in computer memory. These codes are organized according to a certain methodology called an encoding scheme. The list of all characters so encoded is referred to as the “repertoire” of characters in the given encoding schemes. For example, ASCII is one encoding scheme, perhaps the one best known to the average non-technical person in North America. “A”, “B”, & “C” are three characters in the repertoire of this encoding scheme. These three characters are assigned encodings 41, 42 & 43 in ASCII (expressed here in hexadecimal). 1.2 MARC8 "MARC8" is the term commonly used to refer both to the encoding scheme and its repertoire as used in MARC records up to 1998. The ‘8’ refers to the fact that, unlike Unicode which is a multi-byte per character code set, the MARC8 encoding scheme is principally made up of multiple one byte tables in which each character is encoded using a single 8 bit byte. (It also includes the EACC set which actually uses fixed length 3 bytes per character.) (For details on MARC8 and its specifications see: http://www.loc.gov/marc/.) MARC8 was introduced around 1968 and was initially limited to essentially Latin script only.
    [Show full text]
  • Evitalia NORMAS ISO En El Marco De La Complejidad
    No. 7 Revitalia NORMAS ISO en el marco de la complejidad ESTEQUIOMETRIA de las relaciones humanas FRACTALIDAD en los sistemas biológicos Dirección postal Calle 82 # 102 - 79 Bogotá - Colombia Revista Revitalia Publicación trimestral Contacto [email protected] Web http://revitalia.biogestion.com.co Volumen 2 / Número 7 / Noviembre-Enero de 2021 ISSN: 2711-4635 Editor líder: Juan Pablo Ramírez Galvis. Consultor en Biogestión, NBIC y Gerencia Ambiental/de la Calidad. Globuss Biogestión [email protected] ORCID: 0000-0002-1947-5589 Par evaluador: Jhon Eyber Pazos Alonso Experto en nanotecnología, biosensores y caracterización por AFM. Universidad Central / Clúster NBIC [email protected] ORCID: 0000-0002-5608-1597 Contenido en este número Editorial p. 3 Estequiometría de las relaciones humanas pp. 5-13 Catálogo de las normas ISO en el marco de la complejidad pp. 15-28 Fractalidad en los sistemas biológicos pp. 30-37 Licencia Creative Commons CC BY-NC-ND 4.0 2 Editorial: “En armonía con lo ancestral” Juan Pablo Ramírez Galvis. Consultor en Biogestión, NBIC y Gerencia Ambiental/de la Calidad. [email protected] ORCID: 0000-0002-1947-5589 La dicotomía entre ciencia y religión proviene de la edad media, en la cual, los aspectos espirituales no podían explicarse desde el método científico, y a su vez, la matematización mecánica del universo era el único argumento que convencía a los investigadores. Sin embargo, más atrás en la línea del tiempo, los egipcios, sumerios, chinos, etc., unificaban las teorías metafísicas con las ciencias básicas para dar cuenta de los fenómenos en todas las escalas desde lo micro hasta lo macro.
    [Show full text]
  • Myanmar Script Learning Guide Burmese 1,2,3 Tone System Is Developed by Naing Tin-Nyunt-Pu Copyright © 2013-2017, Naing Tinnyuntpu & Asia Pearl Travels
    Myanmar Script Learning guide (Revision F) by Naing Tinnyuntpu WITH FREE ONLINE AUDIO SUPPORT https://www.asiapearltravels.com/language/myanmar-script-learning-guide.php 1 of 104 Menu ≡ Introduction ........................................................................................4 Vowel: A .............................................................................................6 Vowel: E or I ....................................................................................13 Vowel: U ..........................................................................................20 Vowel: O ..........................................................................................24 Vowel: Ay .........................................................................................28 Vowel: Au ........................................................................................33 Vowel: Un or An ..............................................................................37 Vowel: In ..........................................................................................42 Vowel: Eare ......................................................................................46 Vowel: Ain .......................................................................................51 Vowel: Ome .....................................................................................53 Vowel: Ine ........................................................................................56 Vowel: Oon ......................................................................................59
    [Show full text]
  • Proposal to Encode 0D5F MALAYALAM LETTER ARCHAIC II
    Proposal to encode 0D5F MALAYALAM LETTER ARCHAIC II Shriramana Sharma, jamadagni-at-gmail-dot-com, India 2012-May-22 §1. Introduction In the Malayalam Unicode encoding, the independent letter form for the long vowel Ī is: ഈ where the length mark ◌ൗ is appended to the short vowel ഇ to parallel the symbols for U/UU i.e. ഉ/ഊ. However, Ī was originally written as: As these are entirely different representations of Ī, although their sound value may be the same, it is proposed to encode the archaic form as a separate character. §2. Background As the core Unicode encoding for the major Indic scripts is based on ISCII, and the ISCII code chart for Malayalam only contained the modern form for the independent Ī [1, p 24]: … thus it is this written form that came to be encoded as 0D08 MALAYALAM LETTER II. While this “new” written form is seen in print as early as 1936 CE [2]: … there is no doubt that the much earlier form was parallel to the modern Grantha , both of which are derived from the old Grantha Ī as seen below: 1 ma - īdṛgvidhā | tatarāya īṇ Old Grantha from the Iḷaiyānputtūr copper plates [3, p 13]. Also seen is the Vatteḻuttu Ī of the same time (line 2, 2nd char from right) which also exhibits the two dots. Of course, both are derived from old South Indian Brahmi Ī which again has the two dots. It is said [entire paragraph: Radhakrishna Warrier, personal communication] that it was the poet Vaḷḷattōḷ Nārāyaṇa Mēnōn (1878–1958) who introduced the new form of Ī ഈ.
    [Show full text]
  • Proposal for a Kannada Script Root Zone Label Generation Ruleset (LGR)
    Proposal for a Kannada Script Root Zone Label Generation Ruleset (LGR) Proposal for a Kannada Script Root Zone Label Generation Ruleset (LGR) LGR Version: 3.0 Date: 2019-03-06 Document version: 2.6 Authors: Neo-Brahmi Generation Panel [NBGP] 1. General Information/ Overview/ Abstract The purpose of this document is to give an overview of the proposed Kannada LGR in the XML format and the rationale behind the design decisions taken. It includes a discussion of relevant features of the script, the communities or languages using it, the process and methodology used and information on the contributors. The formal specification of the LGR can be found in the accompanying XML document: proposal-kannada-lgr-06mar19-en.xml Labels for testing can be found in the accompanying text document: kannada-test-labels-06mar19-en.txt 2. Script for which the LGR is Proposed ISO 15924 Code: Knda ISO 15924 N°: 345 ISO 15924 English Name: Kannada Latin transliteration of the native script name: Native name of the script: ಕನ#ಡ Maximal Starting Repertoire (MSR) version: MSR-4 Some languages using the script and their ISO 639-3 codes: Kannada (kan), Tulu (tcy), Beary, Konkani (kok), Havyaka, Kodava (kfa) 1 Proposal for a Kannada Script Root Zone Label Generation Ruleset (LGR) 3. Background on Script and Principal Languages Using It 3.1 Kannada language Kannada is one of the scheduled languages of India. It is spoken predominantly by the people of Karnataka State of India. It is one of the major languages among the Dravidian languages. Kannada is also spoken by significant linguistic minorities in the states of Andhra Pradesh, Telangana, Tamil Nadu, Maharashtra, Kerala, Goa and abroad.
    [Show full text]