DOCSLIB.ORG

A Neural Network Approach to Arbitrary Symbol Recognition on Modern Smartphones

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Neural Network Approach to Arbitrary Symbol Recognition on Modern Smartphones A Neural Network Approach to Arbitrary Symbol Recognition on Modern Smartphones FINAL SAMUEL WEJÉUS Master’s Thesis at CSC, KTH Supervisor: Jens Lagergren Examiner: Anders Lansner Project Sponsor: Bontouch AB TRITA xxx yyyy-nn Abstract Making a computer understand handwritten text and sym- bols have numerous applications ranging from reading of bank checks, mail addresses or digitalizing arbitrary note taking. Benefits include atomization of processes, efficient electronic storage, and possible augmented usage of parsed content. This report will provide an overview of how off-line hand- writing recognition systems can be constructed. We will show how such systems can be split into isolated modules which can be constructed individually. Focus will be on handwritten single symbol recognition and we will present how this could be accomplished using convolutional neural networks on a modern smartphone. A symbol recognition prototype application for the Apple iOS operation system will be constructed and evaluated as a proof-of-concept. Results obtained during this project shows that it is pos- sible to train a classifier to understand arbitrary symbols without the need to manually crafting class separating fea- tures and instead rely on deep learning for automatic struc- ture discovery. Referat Identifiering av symboler genom användning av neurala nätverk på moderna mobiltelefoner Att få en dator att förstå handskriven text och symboler har många användningsområden rörande allt från att läsa bankcheckar, postadresser eller att digitalisera godtyckliga notater. Fördelar av en sådan process inkluderar automa- tisering av procedurer, elektronisk lagring samt möjlig ut- ökad användning av inläst material. Denna rapport avser att ge en översikt hur system för igen- känning av handskriven text i off-line läge kan skapas. Vi ämnar visa hur sådana system kan delas upp i mindre iso- lerade delar som kan realiseras individuellt. Fokus kommer att läggas på igenkänning av enskilda handskrivna symbo- ler och vi kommer presentera hur detta kan göras med hjälp av konvolutionella neurala nätverk på en modern mobiltele- fon. En applikation som utför igenkänning av nämnda sym- boler kommer att skapas för Apples iOS operativsystem i from av en proof-of-concept applikation. Resultat erhållna genom detta projekt visar på att det är möjligt att träna en klassifierare att känna igen godtyckliga symboler utan att behöva manuellt skapa klassseparerande karaktärsdrag utan istället använda sig av djup inlärning för automatisk igenkänning av strukturer. Contents List of Figures List of Tables Acronyms 1 Introduction 1 1.1 History . 2 1.2 The Client . 2 1.3 Applications . 2 1.4 Problem Statement . 3 1.5 Challenges . 3 1.6 Limitations of Scope . 4 2 Related Work 5 2.1 The MNIST Database . 5 2.2 Current State of Field . 6 2.3 Handwriting Recognition . 8 2.3.1 Writing Styles and Related Issues . 8 2.3.2 Writer-dependence vs. Writer-independence . 9 2.3.3 On-line vs. Off-line handwriting . 9 2.3.4 Segmentation . 9 2.3.5 Features and Feature Selection . 11 2.4 Problem Reduction Techniques . 13 2.5 Description of a Complete HWR System . 13 3 Theory 15 3.1 Choice of Symbol Classifier . 15 3.2 Artificial Neural Networks . 16 3.2.1 Network Structure and Network Training . 20 3.2.2 Deep Learning of Representations . 22 3.2.3 Convolutional Neural Networks . 23 4 Results 27 4.1 Comparison of Neural Network Libraries . 27 4.1.1 Investigated Libraries . 27 4.1.2 Evaluation . 29 4.2 Lua . 29 4.3 Torch . 30 4.4 Prototype . 30 4.4.1 Network Training . 31 4.4.2 iPhone Application . 33 4.4.3 Testing . 36 5 Discussion 39 5.1 Conclusions . 41 6 Future Work 43 References 47 List of Figures 2.1 Samples from the MNIST dataset . 6 2.2 Hard samples from the MNIST set . 6 2.3 Example of recognition using the Evernote OCR system. 7 2.4 Variation in writing style . 8 2.5 Example of a captured word image before explicit segmentation . 10 2.6 Example of different results after preforming explicit segmentation . 10 2.7 Example of sliding-window technique . 11 2.8 Hypothetical HWR System Pipeline . 14 3.1 Visual model of the McCulloch and Pitt’s neuron . 17 3.2 Overview of a simple neural network model . 18 3.3 Plot of network performance over time . 22 3.4 A typical two stage ConvNet . 24 3.5 Edge detection using convolution . 25 4.1 Training using different learning rates . 32 4.2 Input views of application prototype . 33 4.3 Example of user drawing custom shapes . 34 4.4 Drawing classification pipeline . 34 4.5 Classification using camera capture . 35 4.6 Pre-processing stages of sample captured with camera . 36 4.7 Number of correct classifications using prototype (individual numbers) . 36 4.8 Number of correct classifications using prototype (total) . 37 5.1 30 filters trained on the MNIST set . 40 List of Tables 2.1 Best results reported on the MNIST set for various ML techniques. 7 3.1 Mathematical notation used when describing neural network algorithms 16 4.1 Freehand drawing speed performance of prototype on various devices . 37 4.2 Camera capture speed performance of prototype on various devices . 37 Chapter 1 Introduction The process of parsing samples of a handwritten text into symbols is usually referred to as recognition or classification. The purpose of this representation is that it can be interpreted by a machine. One usually makes a distinction between two types of recognition. If the characters are printed typewriter fonts the recognition of such are referred to as Optical Character Recognition (OCR), and if the characters are written by hand we call the process Handwriting Recognition (HWR). Furthermore, handwriting can be distinguished as being either on-line or off-line, depending on when the text is captured. If the text is captured while the author is writing it is referred to as on-line mode otherwise it is referred to as off-line. A complete recognition system consists of three parts commonly corresponding to three separate problems: localization, segmentation, and recognition. The goal for each of these parts are: isolating and finding contours of individual words (lo- calization), separating a word into individual characters (segmentation), and finally mapping segmented chunks with the correct interpretation (recognition). Today the most popular approaches for character recognition involves using some form of Machine Learning (ML) technique. A classifier is the set of techniques used and can be seen as a black box producing output in the form of a classification given some input. Classification is regarded as an instance of supervised learning. That is, learning is performed using a set of correctly identified observations. Machine learning can be used for a complete recognition system, or for specific parts. A recognition system involves complex tasks that need large computational resources. Building recognition systems for smartphones has not yet been widely investigated [24], consequently it is of interest to determine a set of suitable technologies for these type of devices. This report will focus on the off-line case of handwriting recognition on a smart- phone using neural networks. We will give an introduction to the various problems faced when building HWR systems. No assumptions or clear goal was given at the start of this project other than the question of what is the state of field today, what could be accomplished, and how would you do it. The outline of this report is of investigative nature. To validate results found a prototype for character recognition 1 on a smartphone will be presented as a proof-of-concept for how neural networks can be used efficiently on smartphones. 1.1 History Character Recognition (CR), was first studied in the beginning of the 1900s taking a mechanical approach using photocells [1]. Common techniques investigated included simple template matching and structural analysis. Initial development came to a halt when researchers realized the huge diversity and variability in text when it comes to handwritten input [48]. The history of research efforts made from its infancy have not been a linear process. The problem of CR was at first very popular subject in the field of pattern recognition since it was regarded an easy problem to solve. Like many other fields in science and technology process is usually tangled and progress is often made when research diverge and then cross-breeding different re- sults [48]. Modern state of the art recognition systems uses techniques from various fields of pattern recognition, machine learning and artificial intelligence. Today on-line HWR is considered being a close to solved problem [54]. The problem of off-line handwriting is however much harder and is considered an open question in the research community [55]. 1.2 The Client Bontouch AB is a IT consulting company whose aim is to partner in long-term collaborations with its customers. The company focus on mobile solutions for plat- forms like Android (Google)1 and iOS (Apple)2. Bontouch is located in Stockholm but have a global market. Projects developed for clients includes, among others, Sweden’s first banking app for Skandinaviska Enskilda Banken AB (SEB)3 which make use of off-line OCR scanning of a predefined printed OCR font on invoices. For future projects Bontouch is very interested in how current recognition sys- tems can be extended, or replaced to recognize arbitrary input. The main interest for Bontouch AB is to get an overview of the state of the field. What can be ac- complished today and how could a recognition system be implemented on a mobile platform? 1.3 Applications HWR software simplifies the process of extracting data from the handwritten doc- uments and storing it in electronic formats.
Load more

Recommended publications
  • Transfer Learning Using CNN for Handwritten Devanagari Character
    Accepted for publication in IEEE International Conference on Advances in Information Technology (ICAIT), ICAIT - 2019, Please refer IEEE Explore for final version Transfer Learning using CNN for Handwritten Devanagari Character Recognition Nagender Aneja∗ and Sandhya Aneja ∗ ∗Universiti Brunei Darussalam Brunei Darussalam fnagender.aneja, sandhya.aneja [email protected] Abstract—This paper presents an analysis of pre-trained models to recognize handwritten Devanagari alphabets using transfer learning for Deep Convolution Neural Network (DCNN). This research implements AlexNet, DenseNet, Vgg, and Inception ConvNet as a fixed feature extractor. We implemented 15 epochs for each of AlexNet, DenseNet 121, DenseNet 201, Vgg 11, Vgg 16, Vgg 19, and Inception V3. Results show that Inception V3 performs better in terms of accuracy achieving 99% accuracy with average epoch time 16.3 minutes while AlexNet performs fastest with 2.2 minutes per epoch and achieving 98% accuracy. Index Terms—Deep Learning, CNN, Transfer Learning, Pre-trained, handwritten, recognition, Devanagari I. INTRODUCTION Handwriting identification is one of the challenging research domains in computer vision. Handwriting character identification systems are useful for bank signatures, postal code recognition, and bank cheques, etc. Many researchers working in this area tend to use the same database and compare different techniques that either perform better concerning the accuracy, time, complexity, etc. However, the handwritten recognition in non-English is difficult due to complex shapes. Fig. 1: Devanagari Alphabets Many Indian Languages like Hindi, Sanskrit, Nepali, Marathi, Sindhi, and Konkani uses Devanagari script. The alphabets of Devanagari Handwritten Character Dataset (DHCD) [1] is shown in Figure 1. Recognition of Devanagari is particularly challenging due to many groups of similar characters.
    [Show full text]
  • A Novel Approach to On-Line Handwriting Recognition Based on Bidirectional Long Short-Term Memory Networks
    A Novel Approach to On-Line Handwriting Recognition Based on Bidirectional Long Short-Term Memory Networks Marcus Liwicki1 Alex Graves2 Horst Bunke1 Jurgen¨ Schmidhuber2,3 1 Inst. of Computer Science and Applied Mathematics, University of Bern, Neubruckstr.¨ 10, 3012 Bern, Switzerland 2 IDSIA, Galleria 2, 6928 Manno-Lugano, Switzerland 3 TU Munich, Boltzmannstr. 3, 85748 Garching, Munich, Germany Abstract underlying this paper we aim at developing a handwriting recognition system to be used in a smart meeting room sce- In this paper we introduce a new connectionist approach nario [17], in our case the smart meeting room developed in to on-line handwriting recognition and address in partic- the IM2 project [11]. Smart meeting rooms usually have ular the problem of recognizing handwritten whiteboard multiple acquisition devices, such as microphones, cam- notes. The approach uses a bidirectional recurrent neu- eras, electronic tablets, and a whiteboard. In order to allow ral network with long short-term memory blocks. We use for indexing and browsing [18], automatic transcription of a recently introduced objective function, known as Connec- the recorded data is needed. tionist Temporal Classification (CTC), that directly trains In this paper, we introduce a novel approach to on-line the network to label unsegmented sequence data. Our new handwriting recognition, using a single recurrent neural net- system achieves a word recognition rate of 74.0 %, com- work (RNN) to transcribe the data. The key innovation is a pared with 65.4 % using a previously developed HMM- recently introduced RNN objective function known as Con- based recognition system. nectionist Temporal Classification (CTC) [5].
    [Show full text]
  • March 2012 Version 2
    WWRF VIP WG CONNECTED VEHICLES White Paper Connected Vehicles: The Role of Emerging Standards, Security and Privacy, and Machine Learning Editor: SESHADRI MOHAN, CHAIR CONNECTED VEHICLES WORKING GROUP PROFESSOR, SYSTEMS ENGINEERING DEPARTMENT UA LITTLE ROCK, AR 72204, USA Project website address: www.wwrf.ch This publication is partly based on work performed in the framework of the WWRF. It represents the views of the authors(s) and not necessarily those of the WWRF. EXECUTIVE SUMMARY The Internet of Vehicles (IoV) is an emerging technology that provides secure vehicle-to-vehicle (V2V) communication and safety for drivers and their passengers. It stands at the confluence of many evolving disciplines, including: evolving wireless technologies, V2X standards, the Internet of Things (IoT) consisting of a multitude of sensors that are housed in a vehicle, on the roadside, and in the devices worn by pedestrians, the radio technology along with the protocols that can establish an ad-hoc vehicular network, the cloud technology, the field of Big Data, and machine intelligence tools. WWRF is presenting this white paper inspired by the developments that have taken place in recent years in standards organizations such as IEEE and 3GPP and industry consortia efforts as well as current research in academia. This white paper provides insights into the state-of-the-art regarding 3GPP C-V2X as well as security and privacy of ETSI ITS, IEEE DSRC WAVE, 3GPP C-V2X. The White Paper further discusses spectrum allocation worldwide for ITS applications and connected vehicles. A section is devoted to a discussion on providing connected vehicles communication over a heterogonous set of wireless access technologies as it is imperative that the connectivity of vehicles be maintained even when the vehicles are out of coverage and/or the need to maintain vehicular connectivity as a vehicle traverses multiple wireless access technology for access to V2X applications.
    [Show full text]
  • Unconstrained Online Handwriting Recognition with Recurrent Neural Networks
    Unconstrained Online Handwriting Recognition with Recurrent Neural Networks Alex Graves Santiago Fernandez´ Marcus Liwicki TUM, Germany IDSIA, Switzerland University of Bern, Switzerland [email protected] [email protected] [email protected] Horst Bunke Jurgen¨ Schmidhuber University of Bern, Switzerland IDSIA, Switzerland and TUM, Germany [email protected] [email protected] Abstract In online handwriting recognition the trajectory of the pen is recorded during writ- ing. Although the trajectory provides a compact and complete representation of the written output, it is hard to transcribe directly, because each letter is spread over many pen locations. Most recognition systems therefore employ sophisti- cated preprocessing techniques to put the inputs into a more localised form. How- ever these techniques require considerable human effort, and are specific to par- ticular languages and alphabets. This paper describes a system capable of directly transcribing raw online handwriting data. The system consists of an advanced re- current neural network with an output layer designed for sequence labelling, com- bined with a probabilistic language model. In experiments on an unconstrained online database, we record excellent results using either raw or preprocessed data, well outperforming a state-of-the-art HMM based system in both cases. 1 Introduction Handwriting recognition is traditionally divided into offline and online recognition. Offline recogni- tion is performed on images of handwritten text. In online handwriting the location of the pen-tip on a surface is recorded at regular intervals, and the task is to map from the sequence of pen positions to the sequence of words. At first sight, it would seem straightforward to label raw online inputs directly.
    [Show full text]
  • Start, Follow, Read: End-To-End Full Page Handwriting Recognition
    Start, Follow, Read: End-to-End Full-Page Handwriting Recognition Curtis Wigington1,2, Chris Tensmeyer1,2, Brian Davis1, William Barrett1, Brian Price2, and Scott Cohen2 1 Brigham Young University 2 Adobe Research [email protected] code at: https://github.com/cwig/start_follow_read Abstract. Despite decades of research, offline handwriting recognition (HWR) of degraded historical documents remains a challenging problem, which if solved could greatly improve the searchability of online cultural heritage archives. HWR models are often limited by the accuracy of the preceding steps of text detection and segmentation. Motivated by this, we present a deep learning model that jointly learns text detection, segmentation, and recognition using mostly images without detection or segmentation annotations. Our Start, Follow, Read (SFR) model is composed of a Region Proposal Network to find the start position of text lines, a novel line follower network that incrementally follows and preprocesses lines of (perhaps curved) text into dewarped images suitable for recognition by a CNN-LSTM network. SFR exceeds the performance of the winner of the ICDAR2017 handwriting recognition competition, even when not using the provided competition region annotations. Keywords: Handwriting Recognition, Document Analysis, Historical Document Processing, Text Detection, Text Line Segmentation. 1 Introduction In offline handwriting recognition (HWR), images of handwritten documents are converted into digital text. Though recognition accuracy on modern printed documents has reached acceptable performance for some languages [28], HWR for degraded historical documents remains a challenging problem due to large variations in handwriting appearance and various noise factors. Achieving ac- curate HWR in this domain would help promote and preserve cultural heritage by improving efforts to create publicly available transcriptions of historical doc- uments.
    [Show full text]
  • Contributions to Handwriting Recognition Using Deep Neural Networks and Quantum Computing Bogdan-Ionut Cirstea
    Contributions to handwriting recognition using deep neural networks and quantum computing Bogdan-Ionut Cirstea To cite this version: Bogdan-Ionut Cirstea. Contributions to handwriting recognition using deep neural networks and quantum computing. Artificial Intelligence [cs.AI]. Télécom ParisTech, 2018. English. tel-02412658 HAL Id: tel-02412658 https://hal.archives-ouvertes.fr/tel-02412658 Submitted on 15 Dec 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. 2018-ENST-0059 EDITE - ED 130 Doctorat ParisTech THÈSE pour obtenir le grade de docteur délivré par TELECOM ParisTech Spécialité Signal et Images présentée et soutenue publiquement par Bogdan-Ionu¸tCÎRSTEA le 17 décembre 2018 Contributions à la reconnaissance de l’écriture manuscrite en utilisant des réseaux de neurones profonds et le calcul quantique Directrice de thèse : Laurence LIKFORMAN-SULEM T Jury Thierry PAQUET, Professeur, Université de Rouen Rapporteur H Christian VIARD-GAUDIN, Professeur, Université de Nantes Rapporteur È Nicole VINCENT, Professeur, Université Paris Descartes Examinateur Sandrine COURCINOUS, Expert, Direction Générale de l’Armement (DGA) Examinateur S Laurence LIKFORMAN-SULEM, Maître de conférence, HDR Télécom ParisTech Directrice de thèse E TELECOM ParisTech école de l’Institut Mines-Télécom - membre de ParisTech 46 rue Barrault 75013 Paris - (+33) 1 45 81 77 77 - www.telecom-paristech.fr Acknowledgments I would like to start by thanking my PhD advisor, Laurence Likforman-Sulem.
    [Show full text]
  • Unsupervised Feature Learning for Writer Identification
    Unsupervised Feature Learning for Writer Identification A Master’s Thesis Submitted to the Faculty of the Escola Tecnica` d’Enginyeria de Telecomunicacio´ de Barcelona Universitat Politecnica` de Catalunya by Advisors: Stefan Fiel Student: Manuel Keglevic Eduard Pallas` Arranz Robert Sablatnig and Josep R. Casas In partial fulfillment of the requirements for the degree of MASTER IN TELECOMMUNICATIONS ENGINEERING Barcelona, July 2018 Unsupervised Feature Learning for Writer Identification Eduard Pallas` Arranz Title of the thesis: Unsupervised feature learning for writer identification Author: Eduard Pallas` Arranz 1 Abstract Our work presents a research on unsupervised feature learning methods for writer identification and retrieval. We want to study the impact of deep learning alter- natives in this field by proposing methodologies which explore different uses of autoencoder networks. Taking a patch extraction algorithm as a starting point, we aim to obtain charac- teristics from patches of handwritten documents in an unsupervised way, mean- ing no label information is used for the task. To prove if the extraction of fea- tures is valid for writer identification, the approaches we propose are evaluated and compared with state-of-the-art methods on the ICDAR2013 and ICDAR2017 datasets for writer identification. Unsupervised Feature Learning for Writer Identification Eduard Pallas` Arranz Dedication: This thesis is dedicated to the loved ones waiting at home, to Giulia, and to all those who have made Vienna an unforgettable experience. 1 Unsupervised Feature Learning for Writer Identification Eduard Pallas` Arranz Acknowledgements First of all, I would like to sincerely thank Josep Ramon Casas for all the support and dedication of all these years, and also during the development of the thesis.
    [Show full text]
  • Fast Multi-Language LSTM-Based Online Handwriting Recognition
    Noname manuscript No. (will be inserted by the editor) Fast Multi-language LSTM-based Online Handwriting Recognition Victor Carbune · Pedro Gonnet · Thomas Deselaers · Henry A. Rowley · Alexander Daryin · Marcos Calvo · Li-Lun Wang · Daniel Keysers · Sandro Feuz · Philippe Gervais Received: Nov. 2018 / Accepted: - Abstract We describe an online handwriting system that is able to support 102 languages using a deep neu- ral network architecture. This new system has com- pletely replaced our previous Segment-and-Decode-based system and reduced the error rate by 20%-40% relative for most languages. Further, we report new state-of- the-art results on IAM-OnDB for both the open and closed dataset setting. The system combines methods from sequence recognition with a new input encoding using Bézier curves. This leads to up to 10x faster recog- nition times compared to our previous system. Through a series of experiments we determine the optimal con- figuration of our models and report the results of our Fig. 1 Example inputs for online handwriting recognition in dif- setup on a number of additional public datasets. ferent languages. See text for details. 1 Introduction alphabetic language, groups letters in syllables leading In this paper we discuss online handwriting recognition: to a large “alphabet” of syllables. Hindi writing often Given a user input in the form of an ink, i.e. a list of contains a connecting ‘Shirorekha’ line and characters touch or pen strokes, output the textual interpretation can form larger structures (grapheme clusters) which of this input. A stroke is a sequence of points (x; y; t) influence the written shape of the components.
    [Show full text]
  • When Are Tree Structures Necessary for Deep Learning of Representations?
    When Are Tree Structures Necessary for Deep Learning of Representations? Jiwei Li1, Minh-Thang Luong1, Dan Jurafsky1 and Eduard Hovy2 1Computer Science Department, Stanford University, Stanford, CA 94305 2Language Technology Institute, Carnegie Mellon University, Pittsburgh, PA 15213 jiweil,lmthang,[email protected] [email protected] Abstract For tasks where the inputs are larger text units (e.g., phrases, sentences or documents), a compo- Recursive neural models, which use syn- sitional model is first needed to aggregate tokens tactic parse trees to recursively generate into a vector with fixed dimensionality that can be representations bottom-up, are a popular used as a feature for other NLP tasks. Models for architecture. However there have not been achieving this usually fall into two categories: re- rigorous evaluations showing for exactly current models and recursive models: which tasks this syntax-based method is Recurrent models (also referred to as sequence appropriate. In this paper, we benchmark models) deal successfully with time-series data recursive neural models against sequential (Pearlmutter, 1989; Dorffner, 1996) like speech recurrent neural models, enforcing apples- (Robinson et al., 1996; Lippmann, 1989; Graves et to-apples comparison as much as possible. al., 2013) or handwriting recognition (Graves and We investigate 4 tasks: (1) sentiment clas- Schmidhuber, 2009; Graves, 2012). They were ap- sification at the sentence level and phrase plied early on to NLP (Elman, 1990), by modeling level; (2) matching questions to answer- a sentence as tokens processed sequentially and at phrases; (3) discourse parsing; (4) seman- each step combining the current token with pre- tic relation extraction. viously built embeddings.
    [Show full text]
  • A Connectionist Recognizer for On-Line Cursive Handwriting Recognition
    A CONNECTIONIST RECOGNIZER FOR ON-LINE CURSIVE HANDWRITING RECOGNITION Stefan Manke and Ulrich Bodenhausen University of Karlsruhe, Computer Science Department, D-76 128 Karlsruhe, Germany ABSTRACT Neural Network (MS-TU”), an extension of the TU”, coin- bines the high accuracy character recognition capahilities of ;I In this paper we show how the Multi-State Tim Delay Neural TDNN with a non-linear time alignment procedure (Dynamic Network (MS-TDNN), which is already used successfully in Time Warping) [7] for finding an optimal alignment between continuous speech recognition tasks, can be applied both to on- strokes and characters in handwritten continuous words. line single character and cursive (continuous) handwriting recog- The following section describes the basic network architecture nition. The MS-TDNN integrates the high accuracy single char- and training method of the MS-TDNN, followed by a description acter recognition capabilities of a TDNN with a non-linear time of the input features used in this paper (section 3). Section 4 pre- alignment procedure (dynamic time warping algorithm) for hd- sents results both on different writer dependent/writer indepn- ing stroke and character boundaries in isolated, handwritten char- dent, single character recognition task and writer dependent, acters ‘and words. In this approach each character is modelled by cursive handwriting recognition tasks. up to 3 different states and words are represented as a sequence of these characters. We describe the basic MS-TDNN architec- 2. THE MULTI-STATE TIME DELAY NEURAL ture and the input features used in this paper, and present results NETWORK (MS-TDNN) (up to 97.7% word recognition rate) both on writer dependent/ independent, single character recognition tasks and writer depen- The Time Delay Neural Network provides a time-shrft invariant dent, cursive handwriting tasks with varying vocahulary sizes up architecture for high performance on-line single character recog- to 20000 words.
    [Show full text]
  • Candidate Fusion: Integrating Language Modelling Into a Sequence-To-Sequence Handwritten Word Recognition Architecture
    Candidate Fusion: Integrating Language Modelling into a Sequence-to-Sequence Handwritten Word Recognition Architecture Lei Kang∗†, Pau Riba∗, Mauricio Villegasy, Alicia Forn´es∗, Mar¸calRusi~nol∗ ∗Computer Vision Center, Barcelona, Spain flkang, priba, afornes, [email protected] yomni:us, Berlin, Germany flei, [email protected] Abstract Sequence-to-sequence models have recently become very popular for tackling handwritten word recognition problems. However, how to effectively inte- grate an external language model into such recognizer is still a challenging problem. The main challenge faced when training a language model is to deal with the language model corpus which is usually different to the one used for training the handwritten word recognition system. Thus, the bias between both word corpora leads to incorrectness on the transcriptions, pro- viding similar or even worse performances on the recognition task. In this work, we introduce Candidate Fusion, a novel way to integrate an external language model to a sequence-to-sequence architecture. Moreover, it pro- vides suggestions from an external language knowledge, as a new input to the sequence-to-sequence recognizer. Hence, Candidate Fusion provides two improvements. On the one hand, the sequence-to-sequence recognizer has the flexibility not only to combine the information from itself and the lan- arXiv:1912.10308v1 [cs.CV] 21 Dec 2019 guage model, but also to choose the importance of the information provided by the language model. On the other hand, the external language model has the ability to adapt itself to the training corpus and even learn the most commonly errors produced from the recognizer.
    [Show full text]
  • Giant List of AI/Machine Learning Tools & Datasets
    Giant List of AI/Machine Learning Tools & Datasets AI/machine learning technology is growing at a rapid pace. There is a great deal of active research & big tech is leading the way. Luckily there are also a lot of resources out there for the technologist to utilize. So many we had to cherry pick what look like the most legit & useful tools. 1. Accord Framework http://accord-framework.net 2. Aligned Face Dataset from Pinterest (CCO) https://www.kaggle.com/frules11/pins-face-recognition 3. Amazon Reviews Dataset https://snap.stanford.edu/data/web-Amazon.html 4. Apache SystemML https://systemml.apache.org 5. AWS Open Data https://registry.opendata.aws 6. Baidu Apolloscapes http://apolloscape.auto 7. Beijing Laboratory of Intelligent Information Technology Vehicle Dataset http://iitlab.bit.edu.cn/mcislab/vehicledb 8. Berkley Caffe http://caffe.berkeleyvision.org 9. Berkley DeepDrive https://bdd-data.berkeley.edu 10. Caltech Dataset http://www.vision.caltech.edu/html-files/archive.html 11. Cats in Movies Dataset https://public.opendatasoft.com/explore/dataset/cats-in-movies/information 12. Chinese Character Dataset http://www.iapr- tc11.org/mediawiki/index.php?title=Harbin_Institute_of_Technology_Opening_Recognition_Corpus_for_Chinese_Characters_(HIT- OR3C) 13. Chinese Text in the Wild Dataset (CC4.0) https://ctwdataset.github.io 14. CelebA Dataset (research only) http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html 15. Cityscapes Dataset https://www.cityscapes-dataset.com | License 16. Clash of Clans User Comments Dataset (GPL 2) https://www.kaggle.com/moradnejad/clash-of-clans-50000-user-comments 17. Core ML https://developer.apple.com/machine-learning 18. Cornell Movie Dialogs Corpus http://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html 19.
    [Show full text]