OCR Post-Processing of Historical Swedish Text Using Machine Learning Techniques

Total Page:16

File Type:pdf, Size:1020Kb

OCR Post-Processing of Historical Swedish Text Using Machine Learning Techniques OCR post-processing of historical Swedish text using machine learning techniques Master’s thesis in Computer science and engineering Simon Persson Department of Computer Science and Engineering CHALMERS UNIVERSITY OF TECHNOLOGY UNIVERSITY OF GOTHENBURG Gothenburg, Sweden 2019 Master’s thesis 2019 OCR post-processing of historical Swedish text using machine learning techniques Simon Persson Department of Computer Science and Engineering Chalmers University of Technology University of Gothenburg Gothenburg, Sweden 2019 OCR post-processing of historical Swedish text using machine learning techniques Simon Persson © Simon Persson, 2019. Supervisor: Dana Dannélls, Språkbanken Examiner: Krasimir Angelov, University of Gothenburg Master’s Thesis 2019 Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg SE-412 96 Gothenburg Telephone +46 31 772 1000 Gothenburg, Sweden 2019 iv OCR post-processing of historical Swedish text using machine learning techniques Simon Persson Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg Abstract We present an OCR post-processing method that utilizes machine learning tech- niques and is targeted at historical Swedish texts. The method is developed com- pletely independent of any OCR-tool with the aim is to avoid bias towards any single OCR-tool. The purpose of the method is to solve the two main problems of OCR post-processing, i.e. detecting and correcting errors caused by the OCR-tool. Our method is divided into two main parts, each solves one of these problems. Error detection is solved by a Support Vector Machine (SVM) that classifies each word to be either valid or erroneous. In order for the SVM to classify the words, each word is converted into a feature vector that contains several word features for indicating the validity of the word. The error correction part of the method takes the words that have been classified as erroneous and tries to replace them with the correct word. The error correction algorithm is based upon Levenshtein edit distance combined with the frequency wordlists. We OCR processed a collection of 400 documents from the 19th century using three OCR-tools: Ocropus, Tesseract and ABBYY, and used the output result from each tool to develop our method. Experiments and evalua- tions were carried out against the ground truth of the documents. The method is built in a modular fashion and evaluation was performed on each module. We report quantitative and qualitative results showing varying degrees of OCR post-processing complexity. Keywords: machine-learning, NLP, OCR, post-processing v Acknowledgements I would like to express my deepest gratitude towards my supervisor at Språkbanken, Dana Dannélls for her excellent help and patience. I would also like to thank my examiner Krasimir Angelov at University of Gothenburg for the support and feed- back. Lastly, I would like to thank my family and close friend for the support and encouragement throughout this project. Simon Persson, Gothenburg, June 2019 vii Contents List of Figures xiii List of Tables xv 1 Introduction1 1.1 Aim....................................2 1.2 Delimitations...............................2 2 Background3 2.1 OCR of historical texts..........................3 2.2 Optical Character Recognition......................4 2.2.1 OCR-process...........................5 2.2.1.1 Pre-processing.....................5 2.2.1.2 Character recognition.................6 2.2.1.3 Post-processing.....................7 2.3 OCR-tools.................................7 2.3.1 Abbyy Finereader.........................8 2.3.2 Ocropus..............................8 2.3.3 Tesseract.............................8 2.4 Support Vector Machine.........................9 2.5 Word feature............................... 10 2.6 Levenshtein distance........................... 11 2.7 Evaluation................................. 11 2.7.1 Precision, Recall and F1-score.................. 12 2.7.2 OCR evaluation.......................... 12 2.7.3 Word metrics evaluation..................... 14 3 Data 17 3.1 Targeted text............................... 17 3.1.1 Then Swänska Argus....................... 17 3.1.2 Grepect.............................. 18 3.2 Project Runeberg............................. 18 4 Methods 21 4.1 Data pre-processing............................ 21 4.2 Post-processing.............................. 22 4.2.1 Error detection.......................... 22 ix Contents 4.2.1.1 Training data generation................ 22 4.2.1.2 Feature vector..................... 23 4.2.1.3 Support Vector Machine................ 23 4.2.2 Error correction.......................... 24 4.2.2.1 Candidate search.................... 25 4.2.2.2 Candidate ranking................... 26 4.3 OCR evaluation.............................. 26 4.4 Project baseline.............................. 26 5 Results 29 5.1 Error detection.............................. 29 5.1.1 Word features........................... 30 5.1.1.1 Training data filtering................. 30 5.1.1.2 Tri-gram database size................. 30 5.1.1.3 Word frequencies database size............ 31 5.1.1.4 Overall performance.................. 31 5.1.2 Support Vector Machine..................... 32 5.1.2.1 Kernel selection..................... 32 5.1.2.2 Parameter selection................... 33 5.2 Error correction.............................. 34 5.2.1 Input filtering........................... 34 5.2.2 Error correction algorithm.................... 35 5.2.3 Word database size........................ 36 5.3 Finalized performance.......................... 37 6 Discussion 41 6.1 Error detection.............................. 41 6.1.1 Word features........................... 41 6.1.1.1 Tri-gram database size................. 42 6.1.1.2 Word frequencies database size............ 42 6.1.1.3 Training data filtering................. 43 6.1.1.4 SVM........................... 43 6.2 Error correction.............................. 44 6.2.1 Input filtering........................... 44 6.2.2 Correction algorithm....................... 44 6.2.3 Word database size........................ 45 6.3 Overall performance........................... 45 6.4 Bias for any OCR-tool.......................... 45 6.5 Suitability of a machine learning approach............... 45 6.6 Limitations caused by the targeted text................. 46 6.7 Future work................................ 46 6.7.1 Word features........................... 46 6.7.2 Error detection.......................... 47 7 Conclusion 49 Bibliography 51 x Contents A Appendix 1I xi Contents xii List of Figures 2.1 An example of deteriorated page. The prints from nearby pages has left marks on the current page......................4 2.2 A diagram of the different blocks in the OCR-process.........5 2.3 Illustration of a text getting segmented by the layout analysis during the pre-processing step in the OCR-process...............6 2.4 Depiction of an image before (left) and after (right) binarisation....6 2.5 A diagram of the different parts of Ocropus...............8 2.6 A diagram of the Tesseract structure...................9 2.7 Classification problem with two classes (triangles and circles) labeled with values one and zero. The diagonal full line is a possible solution to the problem. By maximizing the distance to the margins (dotted lines) the solution is optimized...................... 11 2.8 The Levenshtein distance function (LD) applied to three pairs of strings. 11 2.9 Classification problem with two classes (triangles and circles) labeled with values one and zero. The diagonal full line is a possible solution to the problem............................... 13 2.10 Example of averages of two word features divided by the validity class. 14 2.11 Example of the percent difference metric for two word features.... 15 3.1 Histogram of word length distribution for the targeted text...... 18 3.2 Example of pages from both Then Swänska Argus (left) and Grepect (right). Notice the ink “bleed-through” and mixture of typefaces........ 19 4.1 Simplified diagram of the complete OCR process including a more detailed version of the post-processing step................ 21 4.2 Visualization of the data pre-processing. The second line is the pro- cessed version of the first line....................... 21 4.3 Example of how a word is split and some candidates for each split. The winning candidate is chosen by ranking the candidates by Lev- enshtein distance (LD) and the frequency of the word.......... 25 5.1 The order in which each subsystem will be evaluated. Each sub- system will be evaluated with optimal configurations for all previous subsystems. The dashed boxes indicate which problem each subsytem contribute to solving............................ 29 xiii List of Figures 5.2 Performance of the word features without any of the two filters, last char filter and only non-alpha filter. The performance is displayed in the percent difference mean metric................... 31 5.3 Performance of the word feature with only last char filter. The per- formance is displayed in the percent difference mean metric...... 32 5.4 Performance of the word feature with only only non-alpha filter. The performance is displayed in the percent difference mean metric..... 33 5.5 Performance of the word features with both only non-alpha filter and last character filter. The performance is displayed in the percent difference mean metric........................... 34 5.6 Performance of the Swedishness metric with different
Recommended publications
  • OCR Pwds and Assistive Qatari Using OCR Issue No
    Arabic Optical State of the Smart Character Art in Arabic Apps for Recognition OCR PWDs and Assistive Qatari using OCR Issue no. 15 Technology Research Nafath Efforts Page 04 Page 07 Page 27 Machine Learning, Deep Learning and OCR Revitalizing Technology Arabic Optical Character Recognition (OCR) Technology at Qatar National Library Overview of Arabic OCR and Related Applications www.mada.org.qa Nafath About AboutIssue 15 Content Mada Nafath3 Page Nafath aims to be a key information 04 Arabic Optical Character resource for disseminating the facts about Recognition and Assistive Mada Center is a private institution for public benefit, which latest trends and innovation in the field of Technology was founded in 2010 as an initiative that aims at promoting ICT Accessibility. It is published in English digital inclusion and building a technology-based community and Arabic languages on a quarterly basis 07 State of the Art in Arabic OCR that meets the needs of persons with functional limitations and intends to be a window of information Qatari Research Efforts (PFLs) – persons with disabilities (PWDs) and the elderly in to the world, highlighting the pioneering Qatar. Mada today is the world’s Center of Excellence in digital work done in our field to meet the growing access in Arabic. Overview of Arabic demands of ICT Accessibility and Assistive 11 OCR and Related Through strategic partnerships, the center works to Technology products and services in Qatar Applications enable the education, culture and community sectors and the Arab region. through ICT to achieve an inclusive community and educational system. The Center achieves its goals 14 Examples of Optical by building partners’ capabilities and supporting the Character Recognition Tools development and accreditation of digital platforms in accordance with international standards of digital access.
    [Show full text]
  • Gradu04243.Pdf
    Paperilomakkeesta tietomalliin Kalle Malin Tampereen yliopisto Tietojenkäsittelytieteiden laitos Tietojenkäsittelyoppi Pro gradu -tutkielma Ohjaaja: Erkki Mäkinen Toukokuu 2010 i Tampereen yliopisto Tietojenkäsittelytieteiden laitos Tietojenkäsittelyoppi Kalle Malin: Paperilomakkeesta tietomalliin Pro gradu -tutkielma, 61 sivua, 3 liitesivua Toukokuu 2010 Tässä tutkimuksessa käsitellään paperilomakkeiden digitalisointiin liittyvää kokonaisprosessia yleisellä tasolla. Prosessiin tutustutaan tarkastelemalla eri osa-alueiden toimintoja ja laitteita kokonaisjärjestelmän vaatimusten näkökul- masta. Tarkastelu aloitetaan paperilomakkeiden skannaamisesta ja lopetetaan kerättyjen tietojen tallentamiseen tietomalliin. Lisäksi luodaan silmäys markki- noilla oleviin valmisratkaisuihin, jotka sisältävät prosessin kannalta oleelliset toiminnot. Avainsanat ja -sanonnat: lomake, skannaus, lomakerakenne, lomakemalli, OCR, OFR, tietomalli. ii Lyhenteet ADRT = Adaptive Document Recoginition Technology API = Application Programming Interface BAG = Block Adjacency Graph DIR = Document Image Recognition dpi= Dots Per Inch ICR = Intelligent Character Recognition IFPS = Intelligent Forms Processing System IR = Information Retrieval IRM = Image and Records Management IWR = Intelligent Word Recognition NAS = Network Attached Storage OCR = Optical Character Recognition OFR = Optical Form Recognition OHR = Optical Handwriting Recognition OMR = Optical Mark Recognition PDF = Portable Document Format SAN = Storage Area Networks SDK = Software Development Kit SLM
    [Show full text]
  • JETIR Research Journal
    © 2019 JETIR June 2019, Volume 6, Issue 6 www.jetir.org (ISSN-2349-5162) IMAGE TEXT CONVERSION FROM REGIONAL LANGUAGE TO SPEECH/TEXT IN LOCAL LANGUAGE 1Sayee Tale,2Manali Umbarkar,3Vishwajit Singh Javriya,4Mamta Wanjre 1Student,2Student,3Student,4Assistant Professor 1Electronics and Telecommunication, 1AISSMS, Institute of Information Technology, Pune, India. Abstract: The drivers who drive in other states are unable to recognize the languages on the sign board. So this project helps them to understand the signs in different language and also they will be able to listen it through speaker. This paper describes the working of two module image processing module and voice processing module. This is done majorly using Raspberry Pi using the technology OCR (optical character recognition) technique. This system is constituted by raspberry Pi, camera, speaker, audio playback module. So the system will help in decreasing the accidents causes due to wrong sign recognition. Keywords: Raspberry pi model B+, Tesseract OCR, camera module, recording and playback module,switch. 1. Introduction: In today’s world life is too important and one cannot loose it simply in accidents. The accident rates in today’s world are increasing day by day. The last data says that 78% accidents were because of driver’s fault. There are many faults of drivers and one of them is that they are unable to read the signs and instructions written on boards when they drove into other states. Though the instruction are for them only but they are not able to make it. So keeping this thing in mind we have proposed a system which will help them to understand the sign boards written in regional language.
    [Show full text]
  • An Open-Source Tool Providing a (Semi-)Automatic OCR Workflow For
    Article OCR4all - An Open-Source Tool Providing a (Semi-)Automatic OCR Workflow for Historical Printings Christian Reul 1, Dennis Christ 1, Alexander Hartelt 1, Nico Balbach 1, Maximilian Wehner 1, Uwe Springmann 2, Christoph Wick 1, Christine Grundig 3, Andreas Büttner 4, and Frank Puppe 1 1 Chair for Artificial Intelligence and Applied Computer Science, University of Würzburg, 97074 Würzburg, Germany 2 Center for Information and Language Processing, LMU Munich, 80538 Munich, Germany 3 Institute for Modern Art History, University of Zurich, 8006 Zurich, Switzerland 4 Institute for Philosophy, University of Würzburg, 97074 Würzburg, Germany * Correspondence: [email protected] Version September 11, 2019 submitted to Appl. Sci. Abstract: Optical Character Recognition (OCR) on historical printings is a challenging task mainly due to the complexity of the layout and the highly variant typography. Nevertheless, in the last few years great progress has been made in the area of historical OCR, resulting in several powerful open-source tools for preprocessing, layout recognition and segmentation, character recognition and post-processing. The drawback of these tools often is their limited applicability by non-technical users like humanist scholars and in particular the combined use of several tools in a workflow. In this paper we present an open-source OCR software called OCR4all, which combines state-of-the-art OCR components and continuous model training into a comprehensive workflow. A comfortable GUI allows error corrections not only in the final output, but already in early stages to minimize error propagations. Further on, extensive configuration capabilities are provided to set the degree of automation of the workflow and to make adaptations to the carefully selected default parameters for specific printings, if necessary.
    [Show full text]
  • Ocrocis Tutorial
    Ocrocis A high accuracy OCR method to convert early printings into digital text A Tutorial Uwe Springmann Center for Information and Language Processing (CIS) Ludwig-Maximilians-University, Munich Email: springmann (at) cis.lmu.de version history: version 0.95: 05. March 2015 version 0.96: 11. August 2015 This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0. Ocrocis v. 0.96 2 Contents 1 Abstract 2 2 Introduction 3 3 The OCR workflow 8 3.1 Image acquisition ............................... 8 3.2 Preprocessing ................................. 8 3.3 Training ..................................... 10 3.3.1 Annotation ............................... 10 3.3.2 Training a model ............................ 13 3.4 Character recognition ............................. 16 3.5 Postprocessing ................................. 16 4 A practical example 16 4.1 Download the images ............................. 17 4.2 Binarize the images .............................. 17 4.3 Segment into lines ............................... 18 4.4 Choose images to annotate .......................... 18 4.5 Train a model .................................. 19 4.6 Test the models ................................. 20 4.7 Recognize the book .............................. 23 4.8 Extract the text ................................. 23 5 Demo data 24 6 Overview of the command sequence 24 7 References 24 1 Abstract This tutorial describes the first successful application of OCR to convert scanned images of books over the complete history of modern printing since Gutenberg into highly accu- rate digital text (above 95% correctly recognized characters for even the earliest books with good scans). This opens up the possibility of transforming our textual culturage heritage by OCR methods into electronic text in a much faster and cheaper way than by manual transcription.
    [Show full text]
  • Geometric Layout Analysis of Scanned Documents
    Geometric Layout Analysis of Scanned Documents Dissertation submitted to the Department of Computer Science Technical University of Kaiserslautern for the fulfillment of the requirements for the doctoral degree Doctor of Engineering (Dr.-Ing.) by Faisal Shafait Thesis supervisors: Prof. Dr. Thomas Breuel, TU Kaiserslautern Prof. Dr. Horst Bunke, University of Bern Chair of supervisory committee: Prof. Dr. Karsten Berns, TU Kaiserslautern Kaiserslautern, April 22, 2008 D 386 Abstract Layout analysis–the division of page images into text blocks, lines, and determination of their reading order–is a major performance limiting step in large scale document dig- itization projects. This thesis addresses this problem in several ways: it presents new performance measures to identify important classes of layout errors, evaluates the per- formance of state-of-the-art layout analysis algorithms, presents a number of methods to reduce the error rate and catastrophic failures occurring during layout analysis, and de- velops a statistically motivated, trainable layout analysis system that addresses the needs of large-scale document analysis applications. An overview of the key contributions of this thesis is as follows. First, this thesis presents an efficient local adaptive thresholding algorithm that yields the same quality of binarization as that of state-of-the-art local binarization methods, but runs in time close to that of global thresholding methods, independent of the local window size. Tests on the UW-1 dataset demonstrate a 20-fold speedup compared to traditional local thresholding techniques. Then, this thesis presents a new perspective for document image cleanup. Instead of trying to explicitly detect and remove marginal noise, the approach focuses on locating the page frame, i.e.
    [Show full text]
  • Improving OCR Accuracy for Classical Critical Editions
    Improving OCR Accuracy for Classical Critical Editions Federico Boschetti, Matteo Romanello, Alison Babeu, David Bamman, and Gregory Crane Tufts University, Perseus Digital Library, Eaton 124, Medford MA, 02155, USA Abstract. This paper describes a work-flow designed to populate a digital library of ancient Greek critical editions with highly accurate OCR scanned text. While the most recently available OCR engines are now able after suitable training to deal with the polytonic Greek fonts used in 19th and 20th century editions, further improvements can also be achieved with postprocessing. In particular, the progressive multiple alignment method applied to different OCR outputs based on the same images is discussed in this paper. 1 Introduction The new generation of Greek and Latin corpora that has increasingly become available has shifted the focus from creating accurate digital texts to sophisti- cated digital editions. Previously prefaces, introductions, indexes, bibliographies, notes, critical apparatus (usually at the end of the page, in footnote size), and textual variations of different editions have either been discarded or systemat- ically ignored in the creation of early digital collections. The ancient text that we read in modern editions, however, is the product of editors' choices, where they have evaluated the most probable variants attested in the manuscripts or the best conjectures provided by previous scholars. Humanists thus need both textual and paratextual information when they deal with ancient works. Critical editions of classics are challenging for OCR systems in many ways. First, the layout is divided into several text flows with different font sizes: the author's text established by the editor, the critical apparatus where manuscript variants and scholars' conjectures are registered and, optionally, boxes for notes or side by side pages for the parallel translation.
    [Show full text]
  • Design and Implementation of a System for Recording and Remote Monitoring of a Parking Using Computer Vision and Ip Surveillance Systems
    VOL. 11, NO. 24, DECEMBER 2016 ISSN 1819-6608 ARPN Journal of Engineering and Applied Sciences ©2006-2016 Asian Research Publishing Network (ARPN). All rights reserved. www.arpnjournals.com DESIGN AND IMPLEMENTATION OF A SYSTEM FOR RECORDING AND REMOTE MONITORING OF A PARKING USING COMPUTER VISION AND IP SURVEILLANCE SYSTEMS Albeiro Cortés Cabezas, Rafael Charry Andrade and Harrinson Cárdenas Almario Department of Electronic Engineering, Surcolombiana University Grupo de Tratamiento de Señales y Telecomunicaciones, Pastrana Av. Neiva Colombia, Huila E-Mail: [email protected] ABSTRACT This article presents the design and implementation of a system for detection of license plates for a public parking located at the municipality of Altamira at the state of Huila in Colombia. The system includes also, a module of surveillance cameras for remote monitoring. The detection system consists of three steps: the first step consists in locating and cutting the vehicle license plate from an image taken by an IP camera. The second step is an optical character recognition (OCR), responsible for identifying alphanumeric characters included in the license plate obtained in the first step. And the third step consists in the date and time register operation for the vehicles entry and exit. This data base stores also the license plate number, information about the contract between parking and vehicle owner and billing. Keywords: image processing, vision, monitoring, surveillance, OCR. 1. INTRODUCTION vehicles in a parking and store them in digital way to gain In recent years the automatic recognition plates control of the activity in the parking, minimizing errors numbers technology -ANPR- (Automatic Number Plate that entail manual registration and storage of information Recognition) has had a great development in congestion in books accounting and papers.
    [Show full text]
  • Optical Character Recognition
    MQP CDR#GXS1201 Optical Character Recognition A Major Qualifying Project Report submitted to the faculty of the WORCESTER POLYTECHNIC INSTITUTE in partial fulfillment of the requirements for the Degree of Bachelor of Science by ________________________ Sean O’Brien ________________________ Dhia Ben Haddej April 26, 2012 ___________________________________ Professor Gábor N. Sárközy, Major Advisor ___________________________________ Professor Stanley M. Selkow, Co-Advisor Abstract Our project aimed to understand, utilize and improve the open source Optical Character Recognizer (OCR) software, OCRopus, to better handle some of the more complex recognition issues such as unique language alphabets and special characters such as mathematical symbols. We extended the functionality of OCRopus to work with any language by creating support for UTF-8 character encoding. We also created a character and language model for the Hungarian language. This will allow other users of the software to preform character recognition on Hungarian input without having to train a completely new character model. i Acknowledgments András Kornai, Project Advisor and SZTAKI Contact Attila Zséder, SZTAKI Colleague Gábor Sárközy, MQP Advisor Stanley Selkow, MQP Co-Advisor Tom Breuel, OCRopus Developer Worcester Polytechnic Institute MTA-SZTAKI o Information Lab, MTA-SZTAKI ii Contents Chapter 1: Background .................................................................................................................. 3 1.1 Introduction ....................................................................................................................
    [Show full text]
  • Important New Developments in Arabographic Optical Character Recognition (OCR)
    Important New Developments in Arabographic Optical Character Recognition (OCR) BENJAMIN KIESSLING, MATTHEW THOMAS MILLER, MAXIM G. ROMANOV, & SARAH BOWEN SAVANT 1.1 Summary of Results The Open Islamicate Texts Initiative (OpenITI) team1—building on the foundational open- source OCR work of the Leipzig University (LU) Alexander von Humboldt Chair for Digital Humanities—has achieved Optical Character Recognition (OCR) accuracy rates for printed classical Arabic-script texts in the high nineties. These numbers are based on our tests of seven different Arabic-script texts of varying quality and typefaces, totaling over 7,000 lines (~400 pages, 87,000 words; see Table 1 for full details). These accuracy rates not only represent a distinct improvement over the actual2 accuracy rates of the various proprietary OCR options for printed classical Arabic-script texts, but, equally important, they are produced using an open-source OCR software called Kraken (developed by Benjamin Kiessling, LU), 1. The co-PIs of the Open Islamicate Texts Initiative (OpenITI) are Sarah Bowen Savant (Aga Khan University, Institute for the Study of Muslim Civilisations, London; [email protected]), Maxim G. Romanov (Leipzig University through June 2017; now University of Vienna; [email protected]), and Matthew Thomas Miller (Roshan Institute for Persian Studies, University of Maryland, College Park; [email protected]). Benjamin Kiessling can be contacted at [email protected]. 2. Proprietary OCR programs for Persian and Arabic (e.g., Sakhr’s Automatic Reader, ABBYY Finereader, Readiris) overpromise the level of accuracy they deliver in practice when used on classical texts (in particular, highly vocalized texts). These companies claim that they provide accuracy rates in the high 90 percentages (e.g., Sakhr claims 99.8% accuracy for high-quality documents).
    [Show full text]
  • Extending the Page Segmentation Algorithms of the Ocropus Document Layout Analysis System
    EXTENDING THE PAGE SEGMENTATION ALGORITHMS OF THE OCROPUS DOCUMENTATION LAYOUT ANALYSIS SYSTEM by Amy Alison Winder A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Science Boise State University August 2010 c 2010 Amy Alison Winder ALL RIGHTS RESERVED BOISE STATE UNIVERSITY GRADUATE COLLEGE DEFENSE COMMITTEE AND FINAL READING APPROVALS of the thesis submitted by Amy Alison Winder Thesis Title: Extending the Page Segmentation Algorithms of the OCRopus Document Layout Analysis System Date of Final Oral Examination: 28 June 2010 The following individuals read and discussed the thesis submitted by student Amy Alison Winder, and they evaluated her presentation and response to questions during the final oral examination. They found that the student passed the final oral examination. Elisa Barney Smith, Ph.D. Co-Chair, Supervisory Committee Timothy Andersen, Ph.D. Co-Chair, Supervisory Committee Amit Jain, Ph.D. Member, Supervisory Committee The final reading approval of the thesis was granted by Elisa Barney Smith, Ph.D., Co- Chair of the Supervisory Committee. The thesis was approved for the Graduate College by John R. Pelton, Ph.D., Dean of the Graduate College. Dedicated to my parents, Mary and Robert, and to my children, Samantha and Thomas. iv ACKNOWLEDGMENTS The author wishes to express gratitude to Dr. Barney Smith for providing a structured and supportive environment in which to formulate, develop, and complete this thesis. Not only did she provide the initial concept for the work, but she was instrumental in selecting the specific topic and followed through by meeting with me weekly to guide and support this effort.
    [Show full text]
  • Cooperative Human-Machine Data Extraction from Biological Collections
    2016 IEEE 12th International Conference on e-Science Cooperative Human-Machine Data Extraction from Biological Collections Icaro Alzuru, Andréa Matsunaga, Maurício Tsugawa, José A.B. Fortes Advanced Computing and Information Systems (ACIS) Laboratory University of Florida, Gainesville, USA Abstract—Historical data sources, like medical records or Biocollections (iDigBio: https://www.idigbio.org), the Global biological collections, consist of unstructured heterogeneous Biodiversity Information Facility (GBIF: http://www.gbif.org), content: handwritten text, different sizes and types of fonts, and and the Atlas of Living Australia (ALA: http://www.ala.org.au), text overlapped with lines, images, stamps, and sketches. The are stable providers of universal access to millions of specimens. information these documents can provide is important, from a historical perspective and mainly because we can learn from it. The challenge is that the actual number of specimens to The automatic digitization of these historical documents is a digitize, as stored in collections worldwide, has been calculated complex machine learning process that usually produces poor in more than a billion [4]. Considering the current digitization results, requiring costly interventions by experts, who have to rate, which is on the order of minutes per specimen, these data transcribe and interpret the content. This paper describes hybrid could take several decades to be processed [3]. This time does (Human- and Machine-Intelligent) workflows for scientific data not account for the time to train the personnel who perform the extraction, combining machine-learning and crowdsourcing digitization or the domain expert’s time to manage the process software elements. Our results demonstrate that the mix of human and validate results.
    [Show full text]