Handwritten Punjabi Character Recognition Using Convolutional Neural Networks A Thesis submitted in partial fulfillment of the requirements for the award of the degree of Master Of Engineering in Department of Computer Science and Engineering by Sonia Mittal (801532052) Under the supervision of Dr. Karun Verma Dr. Ravinder Kumar AssistantProfessor,CSED AssistantProfessor,CSED COMPUTER SCIENCE AND ENGINEERING DEPARTMENT THAPAR UNIVERSITY PATIALA - 147004 July 2017 Certificate I hereby certify that the work, which is being presented in the thesis, entitled Handwrit- ten Punjabi Character Recognition Using Convolutional Neural Networks, in partial fulfillment of the requirements for the award of the degree of Master Of Engi- neering and submitted to the institution is an authentic record of my own work carried out during the period July 2015 to July 2017 under the supervision of Assistant Pro- fessor Karun Verma and Assistant Professor Ravinder Kumar. I have also cited the reference about the text(s)/figure(s)/table(s) from where they have been taken. The matter presented in this thesis has not been submitted elsewhere for the award of any other degree or diploma from any institution. Date: Sonia Mittal Candidate This is to certify that the above statement made by the candidate is correct to the best of our knowledge. Date: Date: Dr. Karun Verma Dr. Ravinder Kumar Supervisor Supervisor Assistant Professor Assistant Professor The M.E. Viva-Voice examination of Sonia Mittal, has been held on 09th August, 2017 . i Acknowledgement First of all, I would like to express my gratitude towards Thapar University, for pro- viding me a platform to do my thesis work at such an esteemed institute. I wish to express my respect, deep sense of gratitude and idebtedness to my guide Mr. Karun Verma and Mr. Ravinder Kumar, Assistant Professor, Computer Science And Engineering Department, Thapar University, Patiala for their invaluable and en- thusiastic guidance, useful suggestions, unfailing patience and sustained encouragement throughout this work. I would like to thank Dr. Maninder Singh, Head of Computer Science And Engineering Department, Thapar University, Patiala for kind help, guidance, encouragement and providing the necessary facilities to carry out my research. I am indebted to the faculty members of the department for valuable suggestions, friendly support and full cooperation rendered by all of them. Last, but not the least, I am thankful to supreme power “The GOD” one who has always guided me to work on the right path of the life. Without his grace, this would never come to be today³s reality. With special thanks, I dedicate this thesis to GOD. Sonia Mittal ii Abstract Today, computers have influenced the life of human beings to a great extent. To provide the communication between computers and users, natural language processing techniques have proven to be very efficient way to exchange the information with less personnel requirement. In this thesis work, natural handwriting technique is used to recognize the online handwritten Punjabi characters as natural handwritten characters are less error prone as compared to the input taken via mouse or keyboard. This thesis describes the implementation of handwritten Punjabi character recognition using deep learning technique named as Convolutional Neural Networks (CNNs). The main problem occurs in the recognition of handwritten characters is due to the occurrence of variation in the handwriting style of different users because each person has their own style of writing and also the variability in the writing style of his/her own style due to change in mood, speed of writing at different instant of time. Punjabi script is chosen for this research work as it comes on 14th position in the spo- ken languages and less work is done on Punjabi script as compared to work done on other scripts such as English, Devanagari, Gujarati, Chinese. CNN is chosen for the implementation as it is proven to be very efficient technique to recognize and classify the recognized handwritten characters into their respective classes as it concentrates on the dynamic features of the input handwritten character which is obtained from the random generated character matrices. Here, we used 5-layer CNN having stride value of one for the classification of handwrit- ten images into one of the large number of classes (430 classes) available. Punjabi script has total of 430 classes consisting of 35 consonants, 10 vowel identifiers and their corre- sponding combination characters. In our dataset, each class contains 100 images thereby providing a total of 43,000 number of character images dataset. We divide our dataset in the ratio of 65:25:10, 55:35:10, 45:45:10 training:testing:validation samples data respec- tively. Training, testing and validation accuracy at different number of epochs (consist of forward pass and backward pass) for these different sample ratios are calculated and thus compared. Keywords: Convolutional Neural Networks (CNNs), Rectied Linear Unit (reLu), Dropout, Backpropagation, Tensorflow iii Table of Contents Title Page No. Certificate ....................................... i Acknowledgement .................................. ii Abstract ........................................ iii Table of Contents .................................. iv List of Figures .................................... vii List of Tables ..................................... ix Chapter 1 A Brief Introduction Of Handwritten Recognition And Con- volutional Neural Networks .................... 1 1.1 Introduction.................................. 1 1.2 WhyNeuralNetworks?............................ 2 1.3 Why We Choose Convolutional Neural Network Over Simple Neural Net- work? ..................................... 4 1.4 Layers And Other Hyper-parameters Used To Build Convolutional Neural Network .................................... 4 1.4.1 Convolutional Layer . 5 1.4.2 Pooling Layer . 6 1.4.3 Fully Connected Layer . 7 1.4.4 Padding ................................ 7 1.4.5 Stride ................................. 8 1.4.6 Rectified Linear Units (reLu) Is Used As Activation Function . 9 1.4.7 BackPropagation ........................... 9 1.5 Steps Used In Handwritten Character Recognition . ... 10 1.5.1 Preprocessing ............................. 10 1.5.2 Segmentation ............................. 11 1.5.3 FeatureExtraction .......................... 11 1.5.4 Recognition and Classification . 12 1.6 Applications Of Handwritten Punjabi Character Recognition . .... 12 iv Chapter 2 Literature Survey ........................... 13 Chapter 3 Problem Statement ......................... 23 3.1 Problem Formulation . 23 3.2 ResearchGaps................................. 23 3.3 ResearchObjectives.............................. 24 Chapter 4 Data Collection, Pre-Processing And Segmentation ..... 25 4.1 OverviewOfPunjabiScript . 25 4.2 Punjabi consonants and vowel identifiers . .. 27 4.3 Overview Of Data Collection And Pre-Processing . 28 4.4 Data Collection . 29 4.5 PreprocessingPhase ............................. 30 4.5.1 Size normalization and Centering Of Stroke . 32 4.5.2 Interpolation of Missing Points . 34 4.5.3 Resampling of Points . 35 4.6 Segmentation ................................. 36 Chapter 5 Implementation And Experimental Results .......... 37 5.1 HowTensorflowworks?............................ 38 5.2 Convert handwritten Punjabi character images to Tensorflow format . 39 5.3 HardwareAndSoftwareRequirements . 40 5.4 TensorflowInstallation ............................ 40 5.5 Different Layers Used To Build Convolutional Neural Network In Our Pro- posedSystem ................................. 42 5.5.1 Convolutional Layer 1 And Subsampling Layer 1 . 42 5.5.2 Convolutional Layer 2 And Subsampling Layer 2 . 43 5.5.3 Convolutional Layer 3 And Subsampling Layer 3 . 44 5.5.4 Convolutional Layer 4 And Subsampling Layer 4 . 45 5.5.5 Convolutional Layer 5 And Subsampling Layer 5 . 46 5.5.6 Fully Connected Layer 1 . 46 5.5.7 Fully Connected Layer 2 . 47 5.6 ActivationFunctions ............................. 48 5.6.1 Why We Used Activation Functions? . 48 5.6.2 Types Of Activation Functions . 50 5.7 BackPropagation ............................... 51 5.8 Dropout .................................... 51 5.9 ExperimentalResults............................. 52 v Chapter 6 Conclusion And Future Work ................... 57 6.1 Conclusion................................... 57 6.2 SummaryOfContributions . .. .. 58 6.3 FutureWork.................................. 58 References ....................................... 59 List of Publications ................................. 63 vi List of Figures Figure No. Title Page No. 1.1 Simpleneuralnetworkstructure . 2 1.2 MultiLayerperceptronneuralnetwork . 3 1.3 General Convolutional Neural Network (CNN) Architecture . ..... 5 1.4 Example of convolutional layer . 5 1.5 Example of max pooling layer having 2 × 2 kernel size window . 6 1.6 Example of zero padding on 6 × 6 volume size image . 8 1.7 Example of stride value of 2 applied on 7 × 7 that will output 3 × 3 volume Image ..................................... 8 1.8 Steps Used In Handwritten Punjabi Character Recognition . ..... 11 4.1 Different zones and headline of Punjabi script . 25 4.2 Sample of Haa'haa Punjabi character with vowel identifiers (Adhdhak, Au ˙nkar, Bih¯ar¯i, Dul¯am. v¯amandDulai˙nkar . 28 4.3 Sample of Haa'haa Punjabi character with vowel identifiers (H¯or.¯a, Kanaur.¯a, Kann¯a, L¯am. v¯am. and T. ipp¯i ......................... 29 4.4 Character “Kak'kaa”writtenusingtwostrokes