DOCSLIB.ORG

Categorization of Various Essential Datasets and Methods for Textual Spelling Detection and Normalization

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Categorization of Various Essential Datasets and Methods for Textual Spelling Detection and Normalization 10 Categorization of Various Essential Datasets and Methods for Textual Spelling Detection and Normalization Molouk Sadat Hosseini Beheshti PhD in General Linguistics; Assistant Professor; Iranian Research Institute for Information Science and Technology (IranDoc); Tehran, Iran; Corresponding Auther [email protected] Hadi Abdi Ghavidel Msc. Graduate in Computational Linguistics; Sharif University of Technology; Tehran, Iran [email protected] Received: 26, Jan. 2016 Accepted: 2, Aug. 2016 Abstract: One of the most primary phases of automatic text processing is spelling error detection and grapheme normalization. Storing textual Iranian Research Institute for Science and Technology documents faces several problems without passing this phase, which ISSN 2251-8223 causes a disturbance in retrieving the documents automatically. Therefore, eISSN 2251-8231 specialists in the fields of natural language processing and computational Indexed by SCOPUS, ISC, & LISTA linguistics usually make an attempt to sample various data through Vol. 32 | No. 4 | pp. 1143-1170 presenting ideal methods and algorithms in order to reach the normalized Summer 2017 data. Several researches have been conducted on English and some other languages, which have been followed by a certain amount of researches on Farsi too. Sometimes, these several researches have remained to be a pure study and sometimes they have been released as a product. This paper carries out the categorization of the different methods and essential datasets in these researches and depicts each category individually and the evaluation measurements methods generally. Moreover, it describes the performance of the monolingual Farsi systems and the way they meet the Farsi challenges. Keywords: Spelling Error Detection, Grapheme Normalization, Categorization of the Methods, Monolingual Farsi Systems, Farsi Language Challenges xiii طبقه بندی انواع دادگان مورد نياز و روش های خطايابی و استانداردسازی متنی ملوک السادات حسينی بهشتی دكتری زبان شناسی همگانی؛ استاديار؛ پژوهشگاه علوم و فناوري اطﻻعات ايران )اي رانداک(؛ پديدآور رابط [email protected] هادی عبدی قويدل كارشناسی ارشد زبان شناسی رايانشی؛ دانشگاه صنعتی شريف [email protected] دريافت: 06/11/1394 پذيرش: 12/05/1395 مقاله برای اصﻻح به مدت 31 روز نزد پديدآوران بوده است. | چكيــده: يکــی از پايه ایتريــن مراحــل پــردازش خودكار متــن، تشــخیص خطاهای فصلنامه علمی پژوهشی پژوهشگاه علوم و ّفناوری اطﻻعات ايران امﻻيــی و استانداردســازی نويسه هاســت. بــدون گــذر از ايــن مرحلــه، ذخیرهســازی شاپا )چاپی( 2251-8223 مســتندات متنــی بــا مشــکﻻت متعــددی مواجــه شــده و موجــب اختــﻻل در بازيابــی شاپا )الكترونيكی( 2251-8231 نمايه در ISC ،LISTA، و SCOPUS ماشــینی آن هــا می گــردد. بدين ترتیــب، متخصصــان حوزه هــای پــردازش زبــان jipm.irandoc.ac.ir طبیعــی و زبانشناســی رايانشــی همــواره در تــﻻش هســتند تــا بــا ارائــة روش هــا و دورۀ 32 | شمارۀ 4 | ص ص 1170-1143 الگوريتم هــای مطلــوب انــواع داده هــا را در بوتــة پــردازش قــرار داده و بــه داده ای تابستان 1396 اســتاندارد دســت يابنــد. در زبــان انگلیســی و برخــی زبان هــای ديگــر، تحقیقــات متعــددی در ايــن زمینــه انجــام شــده و بهدنبــال آن زبــان فارســی نیــز در ايــن زمینــه مــورد تحقیــق قــرار گرفتــه اســت. ايــن تحقیقــات متعــدد گاهــی در حــد پژوهــش بــه قــوت خــود باقــی مانــده و گاهــی نیــز در قالــب محصــول عرضــه شــده اســت. مقالــة حاضــر بــه طبقه بنــدی انــواع روش هــا و دادگان مــورد نیــاز در ايــن تحقیقــات پرداختــه و فراينــد هــر كــدام از آن هــا را بهطــور خــاص و نحــوة ســنجش میــزان دقــت پــردازش آنهــا را بهطــور عــام شــرح می دهــد. در ايــن مقالــه همچنیــن، نحــوة عملکــرد ســامانه های تکزبانــة فارســی توصیــف شــده و بــه نحــوة برخــورد آنهــا بــا چالش هــای زبــان فارســی اشــاره می گــردد. کليدواژه هــا: تشــخیص خطاهــای امﻻيــی، استانداردســازی نويســه ها، طبقهبنــدی روشهــا، ســامانه های تکزبانــة فارســی، چالش هــای زبــان فارســی 1143 تابستان 1396 | دورة 32 | شمارة 4 1. مقدمه نــگارش متــن همــواره دســتخوش ســلیقههای مختلفــی در طــول تاريــخ بــوده اســت. هیــچ گاه بــر ســر اي نكــه كــدام حالــت نوشــتاری درســت اســت و كــدام غلــط، اتفــاق نظــر كاملــی وجــود نداشــته و تنــوع نــگارش در انــواع متــون بهوفــور ديــده میشــود. گرچــه تنــوع نگارشــی را شــايد حاصــل خﻻقیــت ذهنــی بشــر بدانیــم، امــا ايــن خﻻقیــت پــردازش ماشــینی متــن را بــا چالش هــای متعــددی روبــه رو می كنــد و دقــت پــردازش دادههــا را بــه میــزان چشــمگیری كاهــش میدهــد. در كنــار تنــوع نگارشــی، غلط هــای ســهوی امﻻيــی نیــز وجــود دارد كــه فحــوای متــن را منحــرف كــرده و ماشــین را از انجــام پردازش هــای دقیــق در حوزههايــی نظیــر ترجمــة ماشــینی، پــردازش و تشــخیص گفتــار، بازيابــی اطﻻعــات و غیــره بازمــی دارد. بدي نترتیــب، متخصصــان حوزههــای پــردازش زبــان طبیعــی و زبانشناســی رايانشــی همــواره ســعی دارنــد كــه بــا ارائــة روش هــا و آزمايــش الگوري تمهــای دادهكاوی و پاﻻيــش داده بــه تشــخیص غلط هــای امﻻيــی بپردازنــد و نوي ســههای متنــی را اســتاندارد كننــد. در ايــن صــورت، اســتانداردترين ِمتــن ممکــن بــر طبــق معیارهــای زبانشــناختی توســط ماشــین دريافــت شــده و بــه تبــع آن، محصول هــای دقیقتــری نیــز از نظــر پايايــی و روايــی توســط ماشــین تولیــد میشــود. از طريــق مــاژول استانداردســازی مــواردی ماننــد نــوع نوشــتن تاريــخ، حــروف فارســی و عربــی، فاصلــه و نیمفاصلــه، خــط تیــره، عﻻئــم نگارشــی و كاربردهــای متنــوع آنهــا، واحدهــای اندازه گیــری مثــل كیلوگــرم و kg و يــا ک.گ.، اعــداد )به صــورت حروفــی و عــددی(، فاصلــة حاشــیة پاراگراف هــا و در نهايــت، ســاختار بخشهــای مختلــف مســتندات نظیــر مقــاﻻت و صفحــات وب در سرتاســر دادگان متنــی همگــن و يکنواخــت میشــوند. از طريــق مــاژول خطايابــی امﻻيــی، انــواع خطاهــای امﻻيــی غیرعمــدی شناســايی میشــوند. ايــن خطاهــا را »ژورافســکی و جیمــز« چنیــن طبقهبنــدی كرده انــد: الــف. خطاهــای غیركلمــه ای1: در ايــن نــوع خطاهــا، كلمــة صحیــح بــه كلمــة ناموجــود در فرهنــگ لغــت يــک زبــان تبديــل میشــود؛ ماننــد: مرســه )امــﻻی غلــط كلمــة مدرســه(، المــا )امــﻻی غلــط كلمــة امــﻻ( ب. خطاهــای كلمــه ای2: در ايــن نــوع خطاهــا، دو كلمــه بــه لحــاظ شــباهت امﻻيــی و شــناختی 1. non-word 2. real-word 1144 طبقه بندی انواع دادگان مورد نیاز و روش های خطایابی و استانداردسازی متنی | حسینی بهشتی و عبدی قویدل )آوايــی( بهطــور غلــط جابه جــا می شــوند؛ ماننــد شــعر و شــر )امﻻيــی( و ماننــد ثــواب و صــواب )شــناختی( )Jurafsky and James 2009(. وجــود انبــوه داده در زبان هــای مختلــف، پژوهشــگران متخصــص در علــوم زبانشناســی رايانشــی و پــردازش زبــان طبیعــی را بــر آن داشــته تــا در مســیر خودكارســازی استانداردســازی و خطايابــی دادههــای متنــی گام بردارنــد. تنهــا دلیــل ايــن امــر را میتــوان در صــرف وقــت و هزينــة زيــاد در عملیــات پیش پــردازش داده يافــت. بدي نترتیــب، در زبــان انگلیســی و برخــی زبان هــای ديگــر نظیــر عربــی، چینــی و عبــری، تحقیقــات متعــددی در زمینــة ســاخت استانداردســاز و خطايــاب انجــام شــده اســت و بهدنبــال آن زبــان فارســی نیــز در ايــن زمینــه مــورد تحقیــق قــرار گرفتــه اســت. بســیاری از ايــن تحقیقــات گاه در قالــب پژوهــش آزمــون و خطــا انجــام شــده و گاه در قالــب محصــول تجــاری در بــازار بین الملــل عرضــه شــده اســت. هیچ كــدام از تحقیقــات انجامشــده بهدقــت 100 درصــد نرســیده اســت و بســیاری از پژوهشــگران، اكنــون نیــز بــه ســاختن استانداردســاز و خطايــاب مشــغول هســتند. از ايــن رو، بررســی نحــوة ســاخت داده و همچنیــن، روش هــای انجامشــده می توانــد بهعنــوان مرجعــی هدايتگــر بــرای پژوهشــگران حــال حاضــر عمــل كــرده و مســیر استانداردســازی و خطايابــی متــن را تبییــن نمايــد. در ايــن مقالــه ســعی داريــم ايــن تحقیقــات متعــدد را در قالــب دادگان مــورد نیــاز و روش هــا طبقهبنــدی كــرده و بــه شــرح فراينــد هــر كــدام بهطــور خــاص و نحــوة ســنجش میــزان دقــت پــردازش آن هــا بهطــور عــام بپردازيــم. همچنیــن، قصــد داريــم چالشهــای زبــان فارســی را نیــز طبقهبنــدی كنیــم و نحــوة عملکــرد ســامانه های تک زبانــة فارســی را در راســتای چالشهــای زبــان فارســی شــرح دهیــم. 2. دادگان مورد نياز و روش های خطايابی و استانداردسازی متن 2-1. دادگان مورد نياز در ايــن بخــش بــه طبقه بنــدی انــواع داده و ســاختار آن بــرای استانداردســازی و خطايابــی متنــی می پردازيــم. جــدول 1، بــه ايــن داده هــا و انــواع ســاختار آن اشــاره می كنــد. 1145 تابستان 1396 | دورة 32 | شمارة 4 جدول 1. انواع داده و ساختار آن داده ساختار داده پیکره پیکرة موازی پیکرة غلط فرهنگ لغت فرهنگ لغت نمايه گذاری نشده فرهنگ لغت نمايه گذاری شده در ادامه، هر كدام از انواع داده های موجود در جدول 1، تشريح می شود. 2-1-1. پيكره 2-1-1-1. پيكــرۀ مــوازی1: پیکرههــا ی مــوازی مناســب بــرای انجــام استانداردســازی و خطايابــی، از جمــﻻت درســت و غلــط همترازشــده متشــکل هســتند كــه بــا بهكارگیــری الگوريتم هــای حــوزة ترجمــة ماشــینی آمــوزش داده شــده و قابــل اســتفاده میباشــند )Schlippe et al. 2010(. 2-1-1-2. پيكــرۀ غلــط2: ايــن نــوع پیکــره از مجموعــة متونــی جمــعآوری میشــود كــه به طــور طبیعــی دارای غلــط امﻻيــی هســتند. معروفتريــن ايــن نــوع پیکــره، پیکــرة انگلیســی »بیركبــک«3 دانشــگاه آكســفورد و همچنیــن، مجموعــة غلطهــای امﻻيــی »پیتــر نورويــگ«4 اســت كــه از »ويکی پديــا« و »بیركبــک« اســتخراج شــده اســت. پیکــرة »بیركبــک« دارای 36133 شــکل ِغلــط 6136 كلمــة انگلیســی از گروه هــای ســنی مختلــف اســت. 2-1-2. فرهنگ لغت5 2-1-2-1. فرهنــگ لغــت نمايه گذارینشــده: فرهنــگ لغت هايــی كــه بهطــور بهینــه نمايه گــذاری نشــده باشــند، در زمــرة ايــن گروه هــا قــرار می گیرنــد. اين گونــه فرهنــگ لغت هــا، در كارهــای پردازشــی حجیــم بــه هیچوجــه قابل اســتفاده نیســتند. 2-1-2-2. فرهنــگ لغــت نمايه گذاری شــده: ايــن نــوع فرهنــگ لغتهــا بــا توجــه بــه نــوع تحقیــق و يــا نرمافــزاری كــه در آن بــهكار مــی رود، نمايه
Load more

Recommended publications
  • GNU Emacs GNU Emacs
    GNU Emacs GNU Emacs Reference Card Reference Card Computing and Information Technology Computing and Information Technology Replacing Text Replacing Text Replace foo with bar M-x replace-string Replace foo with bar M-x replace-string foo bar foo bar Query replacement M-x query replace Query replacement M-x query replace then replace <Space> then replace <Space> then skip to next <Backspace> then skip to next <Backspace> then replace and exit <Period> then replace and exit <Period> then replace all ! then replace all ! then go back ^ then go back ^ then exit query <Esc> then exit query <Esc> Setting a Mark Setting a Mark Set mark here C-<Space> or C-@ Set mark here C-<Space> or C-@ Exchange point and mark C-x C-x Exchange point and mark C-x C-x Mark paragraph M-h Mark paragraph M-h Mark entire file C-x h Mark entire file C-x h Sorting Text Sorting Text Normal sort M-x sort-lines Normal sort M-x sort-lines Reverse sort <Esc>-1 M-x sort lines Reverse sort <Esc>-1 M-x sort lines Inserting Text Inserting Text Scan file fn M-x view-file fn Scan file fn M-x view-file fn Write buffer to file fn M-x write-file fn Write buffer to file fn M-x write-file fn Insert file fn into buffer M-x insert-file fn Insert file fn into buffer M-x insert-file fn Write region to file fn M-x write-region fn Write region to file fn M-x write-region fn Write region to end of file fn M-x append-to-file fn Write region to end of file fn M-x append-to-file fn Write region to beginning of Write region to beginning to specified buffer M-x prepend-to-buffer specified buffer
    [Show full text]
  • Emacspeak — the Complete Audio Desktop User Manual
    Emacspeak | The Complete Audio Desktop User Manual T. V. Raman Last Updated: 19 November 2016 Copyright c 1994{2016 T. V. Raman. All Rights Reserved. Permission is granted to make and distribute verbatim copies of this manual without charge provided the copyright notice and this permission notice are preserved on all copies. Short Contents Emacspeak :::::::::::::::::::::::::::::::::::::::::::::: 1 1 Copyright ::::::::::::::::::::::::::::::::::::::::::: 2 2 Announcing Emacspeak Manual 2nd Edition As An Open Source Project ::::::::::::::::::::::::::::::::::::::::::::: 3 3 Background :::::::::::::::::::::::::::::::::::::::::: 4 4 Introduction ::::::::::::::::::::::::::::::::::::::::: 6 5 Installation Instructions :::::::::::::::::::::::::::::::: 7 6 Basic Usage. ::::::::::::::::::::::::::::::::::::::::: 9 7 The Emacspeak Audio Desktop. :::::::::::::::::::::::: 19 8 Voice Lock :::::::::::::::::::::::::::::::::::::::::: 22 9 Using Online Help With Emacspeak. :::::::::::::::::::: 24 10 Emacs Packages. ::::::::::::::::::::::::::::::::::::: 26 11 Running Terminal Based Applications. ::::::::::::::::::: 45 12 Emacspeak Commands And Options::::::::::::::::::::: 49 13 Emacspeak Keyboard Commands. :::::::::::::::::::::: 361 14 TTS Servers ::::::::::::::::::::::::::::::::::::::: 362 15 Acknowledgments.::::::::::::::::::::::::::::::::::: 366 16 Concept Index :::::::::::::::::::::::::::::::::::::: 367 17 Key Index ::::::::::::::::::::::::::::::::::::::::: 368 Table of Contents Emacspeak :::::::::::::::::::::::::::::::::::::::::: 1 1 Copyright :::::::::::::::::::::::::::::::::::::::
    [Show full text]
  • Emacspeak User's Guide
    Emacspeak User's Guide Jennifer Jobst Revision History Revision 1.3 July 24,2002 Revised by: SDS Updated the maintainer of this document to Sharon Snider, corrected links, and converted to HTML Revision 1.2 December 3, 2001 Revised by: JEJ Changed license to GFDL Revision 1.1 November 12, 2001 Revised by: JEJ Revision 1.0 DRAFT October 19, 2001 Revised by: JEJ This document helps Emacspeak users become familiar with Emacs as an audio desktop and provides tutorials on many common tasks and the Emacs applications available to perform those tasks. Emacspeak User's Guide Table of Contents 1. Legal Notice.....................................................................................................................................................1 2. Introduction.....................................................................................................................................................2 2.1. What is Emacspeak?.........................................................................................................................2 2.2. About this tutorial.............................................................................................................................2 3. Before you begin..............................................................................................................................................3 3.1. Getting started with Emacs and Emacspeak.....................................................................................3 3.2. Emacs Command Conventions.........................................................................................................3
    [Show full text]
  • Spelling Correction: from Two-Level Morphology to Open Source
    Spelling Correction: from two-level morphology to open source Iñaki Alegria, Klara Ceberio, Nerea Ezeiza, Aitor Soroa, Gregorio Hernandez Ixa group. University of the Basque Country / Eleka S.L. 649 P.K. 20080 Donostia. Basque Country. [email protected] Abstract Basque is a highly inflected and agglutinative language (Alegria et al., 1996). Two-level morphology has been applied successfully to this kind of languages and there are two-level based descriptions for very different languages. After doing the morphological description for a language, it is easy to develop a spelling checker/corrector for this language. However, what happens if we want to use the speller in the "free world" (OpenOffice, Mozilla, emacs, LaTeX, ...)? Ispell and similar tools (aspell, hunspell, myspell) are the usual mechanisms for these purposes, but they do not fit the two-level model. In the absence of two-level morphology based mechanisms, an automatic conversion from two-level description to hunspell is described in this paper. previous work and reuse the morphological information. 1. Introduction Two-level morphology (Koskenniemi, 1983; Beesley & Karttunen, 2003) has been applied successfully to the 2. Free software for spelling correction morphological description of highly inflected languages. Unfortunately there are not open source tools for spelling There are two-level based descriptions for very different correction with these features: languages (English, German, Swedish, French, Spanish, • It is standardized in the most of the applications Danish, Norwegian, Finnish, Basque, Russian, Turkish, (OpenOffice, Mozilla, emacs, LaTeX, ...). Arab, Aymara, Swahili, etc.). • It is based on the two-level morphology. After doing the morphological description, it is easy to The spell family of spell checkers (ispell, aspell, myspell) develop a spelling checker/corrector for the language fits the first condition but not the second.
    [Show full text]
  • Hunspell – the Free Spelling Checker
    Hunspell – The free spelling checker About Hunspell Hunspell is a spell checker and morphological analyzer library and program designed for languages with rich morphology and complex word compounding or character encoding. Hunspell interfaces: Ispell-like terminal interface using Curses library, Ispell pipe interface, OpenOffice.org UNO module. Main features of Hunspell spell checker and morphological analyzer: - Unicode support (affix rules work only with the first 65535 Unicode characters) - Morphological analysis (in custom item and arrangement style) and stemming - Max. 65535 affix classes and twofold affix stripping (for agglutinative languages, like Azeri, Basque, Estonian, Finnish, Hungarian, Turkish, etc.) - Support complex compoundings (for example, Hungarian and German) - Support language specific features (for example, special casing of Azeri and Turkish dotted i, or German sharp s) - Handle conditional affixes, circumfixes, fogemorphemes, forbidden words, pseudoroots and homonyms. - Free software (LGPL, GPL, MPL tri-license) Usage The src/tools dictionary contains ten executables after compiling (or some of them are in the src/win_api): affixcompress: dictionary generation from large (millions of words) vocabularies analyze: example of spell checking, stemming and morphological analysis chmorph: example of automatic morphological generation and conversion example: example of spell checking and suggestion hunspell: main program for spell checking and others (see manual) hunzip: decompressor of hzip format hzip: compressor of
    [Show full text]
  • Finite State Recognizer and String Similarity Based Spelling
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by BRAC University Institutional Repository FINITE STATE RECOGNIZER AND STRING SIMILARITY BASED SPELLING CHECKER FOR BANGLA Submitted to A Thesis The Department of Computer Science and Engineering of BRAC University by Munshi Asadullah In Partial Fulfillment of the Requirements for the Degree of Bachelor of Science in Computer Science and Engineering Fall 2007 BRAC University, Dhaka, Bangladesh 1 DECLARATION I hereby declare that this thesis is based on the results found by me. Materials of work found by other researcher are mentioned by reference. This thesis, neither in whole nor in part, has been previously submitted for any degree. Signature of Supervisor Signature of Author 2 ACKNOWLEDGEMENTS Special thanks to my supervisor Mumit Khan without whom this work would have been very difficult. Thanks to Zahurul Islam for providing all the support that was required for this work. Also special thanks to the members of CRBLP at BRAC University, who has managed to take out some time from their busy schedule to support, help and give feedback on the implementation of this work. 3 Abstract A crucial figure of merit for a spelling checker is not just whether it can detect misspelled words, but also in how it ranks the suggestions for the word. Spelling checker algorithms using edit distance methods tend to produce a large number of possibilities for misspelled words. We propose an alternative approach to checking the spelling of Bangla text that uses a finite state automaton (FSA) to probabilistically create the suggestion list for a misspelled word.
    [Show full text]
  • SDL Tridion Docs Release Notes
    SDL Tridion Docs Release Notes SDL Tridion Docs 14 July 2019 ii SDL Tridion Docs Release Notes 1 Welcome to Tridion Docs Release Notes 1 Welcome to Tridion Docs Release Notes This document contains the complete Release Notes for SDL Tridion Docs 14. Customer support To contact Technical Support, connect to the Customer Support Web Portal at https://gateway.sdl.com and log a case for your SDL product. You need an account to log a case. If you do not have an account, contact your company's SDL Support Account Administrator. Acknowledgments SDL products include open source or similar third-party software. 7zip Is a file archiver with a high compression ratio. 7-zip is delivered under the GNU LGPL License. 7zip SFX Modified Module The SFX Modified Module is a plugin for creating self-extracting archives. It is compatible with three compression methods (LZMA, Deflate, PPMd) and provides an extended list of options. Reference website http://7zsfx.info/. Akka Akka is a toolkit and runtime for building highly concurrent, distributed, and fault tolerant event- driven applications on the JVM. Amazon Ion Java Amazon Ion Java is a Java streaming parser/serializer for Ion. It is the reference implementation of the Ion data notation for the Java Platform Standard Edition 8 and above. Amazon SQS Java Messaging Library This Amazon SQS Java Messaging Library holds the Java Message Service compatible classes, that are used for communicating with Amazon Simple Queue Service. ANTLR ANTLR is a powerful parser generator that you can use to read, process, execute, or translate structured text or binary files.
    [Show full text]
  • Spelling Correction: from Two-Level Morphology to Open Source
    Spelling Correction: from two-level morphology to open source Iñaki Alegria, Klara Ceberio, Nerea Ezeiza, Aitor Soroa, Gregorio Hernandez Ixa group. University of the Basque Country / Eleka S.L. 649 P.K. 20080 Donostia. Basque Country. [email protected] Abstract Basque is a highly inflected and agglutinative language (Alegria et al., 1996). Two-level morphology has been applied successfully to this kind of languages and there are two-level based descriptions for very different languages. After doing the morphological description for a language, it is easy to develop a spelling checker/corrector for this language. However, what happens if we want to use the speller in the "free world" (OpenOffice, Mozilla, emacs, LaTeX, ...)? Ispell and similar tools (aspell, hunspell, myspell) are the usual mechanisms for these purposes, but they do not fit the two-level model. In the absence of two-level morphology based mechanisms, an automatic conversion from two-level description to hunspell is described in this paper. previous work and reuse the morphological information. 1. Introduction Two-level morphology (Koskenniemi, 1983; Beesley & Karttunen, 2003) has been applied successfully to the 2. Free software for spelling correction morphological description of highly inflected languages. Unfortunately there are not open source tools for spelling There are two-level based descriptions for very different correction with these features: languages (English, German, Swedish, French, Spanish, • It is standardized in the most of the applications Danish, Norwegian, Finnish, Basque, Russian, Turkish, (OpenOffice, Mozilla, emacs, LaTeX, ...). Arab, Aymara, Swahili, etc.). • It is based on the two-level morphology. After doing the morphological description, it is easy to The spell family of spell checkers (ispell, aspell, myspell) develop a spelling checker/corrector for the language fits the first condition but not the second.
    [Show full text]
  • Ocrspell: an Interactive Spelling Correction System for OCR Errors in Text
    UNLV Retrospective Theses & Dissertations 1-1-1996 OCRspell: An interactive spelling correction system for OCR errors in text Eric Stofsky University of Nevada, Las Vegas Follow this and additional works at: https://digitalscholarship.unlv.edu/rtds Repository Citation Stofsky, Eric, "OCRspell: An interactive spelling correction system for OCR errors in text" (1996). UNLV Retrospective Theses & Dissertations. 3222. http://dx.doi.org/10.25669/t1t6-9psi This Thesis is protected by copyright and/or related rights. It has been brought to you by Digital Scholarship@UNLV with permission from the rights-holder(s). You are free to use this Thesis in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/ or on the work itself. This Thesis has been accepted for inclusion in UNLV Retrospective Theses & Dissertations by an authorized administrator of Digital Scholarship@UNLV. For more information, please contact [email protected]. INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter &ce, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely afreet reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted.
    [Show full text]
  • IRIX® 6.5.19 Update Guide
    IRIX® 6.5.19 Update Guide 1600 Amphitheatre Pkwy. Mountain View, CA 94043-1351 Telephone (650) 960-1980 FAX (650) 961-0595 February 2003 Dear Valued Customer, SGI® is pleased to present the new IRIX® 6.5.19 maintenance and feature release. Starting with IRIX® 6.5, SGI created a new software upgrade strategy, which delivers both the maintenance (6.5.19m) and feature (6.5.19f) streams to our support contract customers. This upgrade is part of a family of releases that periodically enhances IRIX 6.5. There are several benefits to this release strategy: it provides periodic fixes to IRIX, it assists in managing upgrades, and it supports all platforms. Additional information on this strategy and how it affects you is included in the updated Installation Instructions manual contained in this package. If you need assistance, please visit the Supportfolio™ online website at http://support.sgi.com/ or contact your local support provider. In conjunction with the release of IRIX® 6.5.15, SGI added to the existing life cycle management categories the Limited Support Mode that customizes services we deliver to our users. This new support mode is targeted for open source products. We now offer eight modes of service 2 for software supported by SGI: Active, Maintenance, Limited, Legacy, Courtesy, Divested, Retired, and Expired. Active Mode is our highest level of service. It applies to products that are being actively developed and maintained and are orderable through general distribution. Software fixes for all levels of problems can be expected. Maintenance Mode software is maintained and is still an important part of our product mix.
    [Show full text]
  • A Survey on Creation of Hindi-Spell Checker to Improve the Processing of OCR
    International Journal of Research in Engineering, Science and Management 517 Volume-2, Issue-3, March-2019 www.ijresm.com | ISSN (Online): 2581-5792 A Survey on Creation of Hindi-Spell Checker to Improve the Processing of OCR Shabana Pathan1, Nikhil Khuje2, Punam Kolhe3 1Assistant Professor, Department of Information Technology, SVPCET, Nagpur, India 2,3Student, Department of Information Technology, SVPCET, Nagpur, India Abstract: This project investigates the principles of Optical categorized under non-word errors which occur when the Character Recognition used in the Tesseract OCR engine and correct spelling of the word is known but the word is mistyped techniques to improve its efficiency and processing. This mainly by mistake. These errors are mostly related to the wrong key focuses on improving the Tesseract OCR efficiency for Hindi/Devanagari languages. Improving Hindi/Devanagari text press. For example, typing आपमान for अपमान. Real-word extraction will increase Tesseract's performance and in turn will errors are those error words that are acceptable words in the draw developers to contribute towards Hindi /Devanagari OCR. dictionary but not correct according to sentence. For example, The project introduces the efficiency of new Tesseract OCR मेरा घर उस और है(incorrect) for मेरा घर उस ओर है(correct) और version 4.0 beta and heads us towards its capabilities which reflects in its output. Spell checker analyzes the written text in is an acceptable word in the Hindi dictionary but it occurs as an order to identify any misspellings and gives best correct error for ओर word. Possibility of spelling mistakes in Hindi suggestions for those misspellings.
    [Show full text]
  • Learning GNU Emacs Other Resources from O’Reilly
    Learning GNU Emacs Other Resources from O’Reilly Related titles Unix in a Nutshell sed and awk Learning the vi Editor Essential CVS GNU Emacs Pocket Reference Version Control with Subversion oreilly.com oreilly.com is more than a complete catalog of O’Reilly books. You’ll also find links to news, events, articles, weblogs, sample chapters, and code examples. oreillynet.com is the essential portal for developers interested in open and emerging technologies, including new platforms, pro- gramming languages, and operating systems. Conferences O’Reilly brings diverse innovators together to nurture the ideas that spark revolutionary industries. We specialize in document- ing the latest tools and systems, translating the innovator’s knowledge into useful skills for those in the trenches. Visit con- ferences.oreilly.com for our upcoming events. Safari Bookshelf (safari.oreilly.com) is the premier online refer- ence library for programmers and IT professionals. Conduct searches across more than 1,000 books. Subscribers can zero in on answers to time-critical questions in a matter of seconds. Read the books on your Bookshelf from cover to cover or sim- ply flip to the page you need. Try it today with a free trial. THIRD EDITION Learning GNU Emacs Debra Cameron, James Elliott, Marc Loy, Eric Raymond, and Bill Rosenblatt Beijing • Cambridge • Farnham • Köln • Paris • Sebastopol • Taipei • Tokyo Learning GNU Emacs, Third Edition by Debra Cameron, James Elliott, Marc Loy, Eric Raymond, and Bill Rosenblatt Copyright © 2005 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
    [Show full text]