Support Vector Machine for Handwritten Devanagari Numeral Recognition Shailedra Kumar Shrivastava Sanjay S
Total Page:16
File Type:pdf, Size:1020Kb
International Journal of Computer Applications (0975 – 8887) Volume 7– No.11, October 2010 Support Vector Machine for Handwritten Devanagari Numeral Recognition Shailedra Kumar Shrivastava Sanjay S. Gharde HOD, Information Technology Department Research Scholar, Information Technology Department Samrat Ashok Technological Institute Samrat Ashok Technological Institute Vidisha, (M. P.) INDIA Vidisha, (M. P.) INDIA ABSTRACT characters by applying different techniques, but very less work Support Vector Machines (SVM) is used for classification in has been performed on Handwritten Devanagari numerals. So, pattern recognition widely. This paper applies this technique for this research work has been conducted on Handwritten recognizing handwritten numerals of Devanagari Script. Since Devanagari numeral. benchmark database does not exist globally, this system is constructed database by implementing Automated Numeral Recognition of Handwritten Devanagari Numerals or Extraction and Segmentation Program (ANESP). Preprocessing is Characters [1, 2] is a complicated task due to the unconstrained manifested in the same program which reduces most of the shape variations, different writing style and different kinds of efforts. 2000 samples are collected from 20 different people noise. Also, handwriting depends much on the writer and because having variation in writing style. Moment Invariant and Affine we do not always write the same digit in exactly the same way, Moment Invariant techniques are used as feature extractor. These building a general recognition system that would recognize any techniques extract 18 features from each image which is used in digit with good reliability in every application is not possible. In Support Vector Machine for recognition purpose. Binary recent time, specialists have made use of different techniques classification techniques of Support Vector Machine is such as Modified Discrimination Function (MQDF) [10], implemented and linear kernel function is used in SVM. This Multilayer Perceptron (MLP) [11], Principal Component Analysis linear SVM produces 99.48% overall recognition rate which is (PCA) [12], K-Nearest Neighbor [13], Wavelet-based multi the highest among all techniques applied on handwritten resolution [13], Quadratic classifier [14] have been applied to Devanagari numeral recognition system. solve this problem. These recognition systems are produces the General Terms recognition rate between 89% and 99.04%. To achieve these Machine Learning, Pattern Recognition. accuracies the researchers used various feature extraction techniques also. They extracted large number of features for Keywords recognition. Support Vector Machine, Devanagari Numeral Recognition, Moment Invariant, Affine Moment Invariant. Machine Learning [15, 16, 17, 18, 19] is a subfield of artificial intelligence that is concerned with the design and development of 1. INTRODUCTION algorithms and techniques that allow computers to "learn". Handwritten Character Recognition [1] was begun to come Machine Learning can be classified broadly into Supervised into existence in research during 1980s. Many researches had Learning, Unsupervised Learning, Semi-supervised Learning, been done till date. Due to which many documents are computer Reinforcement Learning. Support Vector machine is one of the aided in the present era. Handwritten recognition system is having supervised learning method. First practical implementation of its own importance and it is adoptable in various fields such as SVM had been executed in early nineties. It is most efficient online handwriting recognition on computer tablets, recognize zip family of algorithms in Machine Learning and computationally codes on mail for postal mail sorting, processing bank check efficient. Support Vector Machines (SVM) [20] are learning amounts, numeric entries in forms filled up by hand (for example systems that use a hypothesis space of linear functions in a high - tax forms) and so on [2]. There are different challenges faced dimensional feature space, trained with a learning algorithm from while attempting to solve this problem. The Handwritten digits optimization theory that implements a learning bias derived from are not always of the same size, thickness, or orientation and statistical learning theory [21]. This learning strategy is position relative to the margins. Numerous amount of work has introduced by Vapnik and co-workers. Support vector machine is been done on English and other sub continental languages [3] and one of the best techniques used for linear and nonlinear even on Indian script [1, 2, 3], but the results didn’t come out of classification and regression. Therefore, it is used in recognition the constraint of laboratories. Diverse algorithms/schemes for of Handwritten Devanagari numerals. The SVM classifier was handwritten character recognition have been evolved in [4, 5]. originally developed for two-class or binary classification and the demanding applications of pattern recognition led to the design of Handwritten recognition has always been a challenging task in multi-class SVM classifiers using the binary SVM classifiers pattern recognition. Many systems and classification algorithms [22]. have been proposed in the past years on handwritten character/numeral recognition in various languages like English [6], Arabian [7], Persian [8], Chinese [9] and Devanagari scripts also. Researchers had been worked on Handwritten Devanagari 9 International Journal of Computer Applications (0975 – 8887) Volume 7– No.11, October 2010 2. SUPPORT VECTOR MACHINE Hindi, Konkani and Nepali. As no standardized database for Devanagari Handwritten Characters is available, first the relevant Support Vector Machine is supervised Machine Learning database has been created [1, 2]. technique. The existence of SVM is shown in fig 1. Computer Vision [18] is the broad area whereas Machine Learning is one of 4. DATA PREPARATION the application domains of Artificial Intelligence along with pattern recognition, Robotics, Natural Language Processing. Automated Numeral Extraction and Segmentation Program Supervised learning, Un-supervised learning, Semi-supervised (ANESP) is a new research program implemented by us in learning and reinforcement learning are various types of Machine MATLAB 7.5b version. This program extracts each numeral Learning. image from the sheet and stores the numeral image by giving file Support Vector Machine (SVM) was first heard in 1992, name in a separate folder assigned in the program path. Input sheet to ANESP should be approximately 1050 X 797 pixels in introduced by Boser, Guyon, and Vapnik in COLT-92. Support vector machines (SVMs) are a set of related supervised learning size. Writer should write the numerals on standardized sample methods used for classification and regression [23]. They belong sheet having specific dimension of block size. Columns should be to a family of generalized linear classifiers. In another terms, separated by approximately 115 pixels and Rows should be Support Vector Machine (SVM) is a classification and regression separated by approximately 120 pixels. Input sheet is processed prediction tool that uses Machine Learning theory to maximize by ANESP. Fig. 2 shows the processing on single Handwritten predictive accuracy while automatically avoiding over-fit to the Numeral using ANESP. Similar kind of processing is performed data. Support Vector machines can be defined as systems which on each handwritten numeral written on Input Sheet. Each isolated numeral is stored in bitmap file of specified folder for use hypothesis space of a linear functions in a high dimensional feature space, trained with a learning algorithm from optimization that numeral only. ANESP performs following operations on theory that implements a learning bias derived from statistical input sheet: learning theory. The foundations of Support Vector Machines Crop single handwritten numeral from input sheet and (SVM) have been developed by Vapnik [4] and gained popularity stored it in a bitmap file. due to many promising features such as better empirical performance. The formulation uses the Structural Risk Normalize and Resize the file in 40 X 40 pixel size by Minimization (SRM) principle, which has been shown to be utilizing only 1 KB memory size. superior, [4], to traditional Empirical Risk Minimization (ERM) Remove the Noise from the image file. principle, used by conventional neural networks. SRM minimizes an upper bound on the expected risk, where as ERM minimizes Convert color image into black and white image the error on the training data. It is this difference which equips (Binarizing image). SVM with a greater ability to generalize, which is the goal in statistical learning. SVMs were developed to solve the Bitmap file is stored in specified folder for that numeral. classification problem, but recently they have been extended to solve regression problems [22]. Figure 2: Processing of single Handwritten Numeral using ANESP. Figure 1: Existence of Support Vector Machine. This program prepares and designs the complete dataset by overcoming the problems in the input sheet such as [1] 3. DEVANAGARI SCRIPT Overwriting, India is a multilingual country of more than 1.2 billion populations with 18 constitutional languages and 10 different Striking out of words, scripts. Devanagari, an alphabetic script, is used by a number of Indian Languages. It was developed to write Sanskrit but was Words crossing the edges of boxes, later adapted to write many other languages such