Deep Convolutional Neural Network for Recognition of Unified Multi-Language Handwritten Numerals

2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR)

Jaafar Alghazo Ghazanfar Latif Loay Alzubaidi Computer Engineering Department, Computer Science Department, Computer Science Department, Prince Mohammad Bin Fahd University, Prince Mohammad Bin Fahd University, Prince Mohammad Bin Fahd University, Al Khobar, Saudi Arabia. Al Khobar, Saudi Arabia. Al Khobar, Saudi Arabia. [email protected] [email protected] [email protected] M. Muzzamal Naseer Yazan Alghazo College of Engineering, Humanities Department, Prince Australian National University, Mohammad Bin Fahd University, Al Canberra, Australia. Khobar, Saudi Arabia. [email protected] [email protected]

Abstract— Deep learning systems have recently gained handwritten. The classification phase involves using well- importance as the architecture of choice in artificial intelligence developed classification techniques such as Multilayer (AI). Handwritten numeral recognition is essential for the Perceptron (MLP) and Support Vector Machines (SVM) among development of systems that can accurately recognize digits in many others. This phase involves training the Artificial different languages which is a challenging task due to variant Intelligence (AI) system and then recognizing the remaining writing styles. This is still an open area of research for developing portion of the dataset or new data. These algorithms can also an optimized Multilanguage writer independent technique for utilize either supervised or unsupervised learning while training numerals. In this paper, we propose a deep learning architecture the AI system on the dataset. It is observed that mostly research for the recognition of handwritten Multilanguage (mixed papers have only tackled numerals written in one language numerals belongs to multiple languages) numerals (Eastern Arabic, Persian, Devanagari, Urdu, Western Arabic). The overall The importance of recognizing handwritten numerals is accuracy of the combined Multilanguage database was 99.26% apparent due to the increase usage of handheld devices in both with a precision of 99.29% on average. The average accuracy of personal and professional settings. Banking industry relies each individual language was found to be 99.322%. Results heavily on handwritten numeral recognition especially for indicate that the proposed deep learning architecture produces numerals written on checks. Other industries also would benefit better results compared to methods suggested in the previous from handwritten numeral recognition algorithms in addition to literature. individuals with different handheld device applications developed that allow users to write numbers rather than input Keywords—Arabic Numberals; Mul-Language Numerals Recognition; Hand Written Numerals; Deep Convolutional Neural them using a keyboard. Networks In this research, a deep Convolutional Neural Network (CNN) architecture is developed for handwritten Eastern Arabic I. INTRODUCTION and Persian numerals while experiments are also performed on some other languages including Urdu, Devanagari, Western Due to the infinite number of variations of handwritten Arabic which in total, are spoken by approximately 1.86 billion numerals that are subject to many variable, the recognition of people. The rest of this paper is structured as follows: section 2 handwritten numeral is a complex problem that has been subject confers the literature review, section 3 presents the proposed of research. A large number of algorithms have been developed system, section 4 details the experimental datasets, section 5 to tackle this problem for both offline and online recognition. discusses the results of the proposed recognition system and Offline recognition refers to recognizing numerals written on section 6 concludes the paper. paper using a pen and online recognition refers to numerals written on handheld devices e.g. stylus. Handwritten numerals can differ even for the same writer when writing on paper or a II. LITERATURE REVIEW handheld device and can vary even in the same medium. The Though the concept of Deep Learning has been around for algorithm consisting of pre-processing, feature extraction and quite some time, yet the utilization of Deep learning to its full the final phase of classification is used for both cases. In the potential was not achieved until recently with advances in preprocessing phase, the image is extracted whether in the technology, computing power and computing resources. Deep spatio-temporal or spatio-luminance representation, noise learning algorithms have achieved new records in different reduction, centering the image, etc. In the feature extraction fields such as image recognition and speech recognition among phase, researchers proposed different methods for feature others [1-2]. The concept of Deep Learning relies on what is extraction that can be used for in final step of classification of known as Convolutional Neural Networks (ConvNets or CNN).

Authorized licensed use limited to: Prince Mohammad Bin Fahd University. Downloaded on August 13,2020 at 11:49:58 UTC from IEEE Xplore. Restrictions apply. 2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR)

LeNet 5 developed by LeCun, Y. et al. was one of the first are used in [20] for improving handwritten Chinese text convolutional networks used in the 1990s [3]. In 2012, recognition. Deep Neural Network is used as part of an Krizhevsky, A. et al. [4] introduced a newer CNN that was wider algorithm implemented in [21] for SAR target configuration and deeper than LeNet and subsequently won the ImageNet- recognition. Fisher Vector (FV) and CNN are used in [22] for 2012 Large Scale Visual Recognition Challenge (ILSVRC). visual recognition of the ancient inscriptions. This is considered by many as the initial point of the subsequent Since Deep learning algorithm handwritten numeral interest in Deep Learning. The accuracy rate achieved was recognition rate will be compared to widely studied Artificial 84.7%. The Architecture of [4] was modified into a new Intelligence Algorithms, a few references will be mentioned architecture proposed by M. D. Zeiler and R. Fergus in [5]. This here in the literature review. Different novel feature extraction new architecture ended up winning the ILSVRC 2013. It methods were developed and used in [23-25] for numeral and achieved an accuracy rate of 88.3%. An Inception module was character recognition. Different classifier have also been used to developed that shrinks the number of parameters in the network test the proposed algorithms. and was the contribution of the CNN developed by Szegedy, C. et al. and ended up winning the ILSVRC 2014 [6]. The accuracy rate achieved was 93.33%. In the same year and in the same III. PROPOSED SYSTEM competition Simonyan, K., & Zisserman, A. introduced a CNN proving that the number of layers in a deep network affects the A. Preprocessing performance [7]. They showed that results are improved by Numerals in Arabic, Urdu and Persian are written from left increasing the depth to 16-19 weighted layers. Their network to right. Their digits are mostly similar except for the numbers ended up winning second place in the 2014 ILSVRC. He, K. et (4) four, (5) five and (6) six. In this paper, we target the al. in [8] introduced residual network CNN and won the recognition of handwritten digits of these languages in addition ILSVRC 2015. The error rate achieved was 3.6%. Huang et al. to Western Arabic (English) and Devanagari which are all [9] introduced the densely connected CNN which exhibited shown in table 1. improvement over some previous architectures. Alghazo et al. also proposed an efficient geometric feature based numeral TABLE I. REPRESENTATION NUMERALS IN EASTERN ARABIC, recognition system for Arabic and Persian numerals [10]. Yet WESTERN ARABIC, PERSIAN, URDU AND DEVANAGARI the CNN introduced in [8] is still the most popular and preferred Western Eastern CNN architecture for practical applications. Persian Urdu Devanagari Arabic Arabic ० ٠ ٠ A number of papers have already been completed on digit or 0 १ ١ ١ character recognition using some types of Deep Learning 1 Architectures. For example, Singh et al. [11] compared the use २ ٢ ٢ of the deep Convolutional Neural Network and the fully 2 ३ ٣ ٣ connected Feed-forward Neural Network for the recognition of 3 the Devangari handwritten characters. They reported a 98.11% ४ ۴ ٤ 4 recognition rate. In [12], Deep-Learning Feed forward ۵ ५ ٥ Backpropagation Neural Network (DFBNN) along with 5 ۶ ६ ٦ Extreme Learning Machine (ELM) were used in handwritten 6 Numeral Recognition. The methods were applied to datasets in ७ ٧ ٧ Thai, Bangla and Devangari. DRBNN outperformed ELM 7 ८ ٨ ٨ slightly. In DFBNN, the accuracy rates were 98.4% for Thai, 8 95.84% for Bangla, and 78.4% for Devangari. Deep Belief ९ ٩ ٩ 9 Networks (DBN), A Deep learning Structure, was used in [13] for Bangla Handwritten Numeral Recognition. The recognition accuracy was 90.27%. DBN was also used in [14] for high- Handwriting styles are different from one person to another resolution Synthetic Aperture Radar (SAR) image classification. and even the handwriting of the same person can be different Alghazo proposed an online numerals recognition system using according to several variables. For example, more than 52 structural features [15]. In [16], Deep Learning algorithm writing classes exist for Arabic and Persian digits only [26]. The consisting of Back-propagation with many hidden layers and similarity between the digits in different languages shown in deformed images for training was applied on the MNIST table 1 makes the recognition of handwritten numerals a difficult Handwritten dataset and achieved a low 0.35% error rate. In task. Fig. 1 shows a sample of handwritten numbers for the [17], deep convolutional Neural Networks (CNN) is used for different languages targeted in this work. numeral and character recognition respectively. The error rate The preprocessing phase is important in automatic was as low as 0.19% and 0.27% respectively. handwritten numeral recognition due to the variations of size, Two hidden layers were used in [18] for the recognition of location, shape, noise and angle of handwritten numerals. The postal code in Urdu, Telugu, English, Tamil and Devanagari and following steps are performed in the preprocessing phase of the achieved a 96.53%. Deep Autoencoder (DA) based on MLP was proposed system. used in [19] for Bangali handwritten numeral recognition with • Binarization of handwritten numerals from grayscale is 97.74% recognition rate for deep learning DA algorithm. Neural done through Otsu Thresholding [27]. Network Language Models and Convolutional neural networks

• Removing noise from the binarized image which is done which in turn helps to train deep networks due to by a 3 3 window of disk shaped structure through computational advantages. morphological operation. • Another disadvantage of fully connected neural network • Separating each digit referred to as Segmentation. is that they are very prone to overfitting when compared to convolution neural networks. The problem of • Each digit is then centered within the window. overfitting occurs when a neural network cannot Each digit of all the languages is normalized and converted generalize well on the unseen data. to a size of 28 28 to make identical input for the Deep CNN. The proposed network architecture contains input layer of 28 28 which is equal to image size, next is the hidden convolution layer with 20 24 24 units followed by a pooling layer with 20 12 12 units and next is another convolution layer with 10088 units again followed by pooling layer with 10044 units and finally, the fully connected output layer generating actual results of classification. Fig. 2 shows the architecture of our convolution neural network. In the proposed convolution neural network architecture, a regularization technique called "dropout" [29] to deal with the problem of overfitting is used. Dropout technique ignores randomly selected neurons during training. These ignored neurons cannot contribute to activations during forwardpass and their corresponding weights are not updated during backward Fig. 1. Samples of Handwritten numerals for most popular five languages. pass. Dropout technique causes a neural network to learn Columns from left to right representing Eastern Arabic, Persian, Urdu, Western multiple internal representations which in turn makes the neural Arabic (English) and Devanagari digits. network to better generalize and less likely to overfit the training data. In addition to using dropout as regularization, a small B. Deep CNN Architecture weight penalty is used while training our network to keep weights under control and avoid overfitting. The proposed algorithm in this article is based on deep learning. A convolution neural network CNN, a three We used stochastic gradient descent equation 1 with a batch dimensional (3D) volume of neurons, to classify numerals. The size of one as an optimization technique to update the weights deep learning architecture proposed in this study is designed as during training of our architecture. follows: the input layer is equivalent to image size which is 28x28 units then two hidden convolution layers with 5X5 window size are used also called local receptive field and both convolution layers are then followed by pooling layers with Where is the cost function, is the step size and is max-pooling of 2×2. The convolution layers in the proposed the sample size and equal to one. network extract and learn image features that are distributed across the entire image. Every convolution layer is composed of IV. EXPERIMENTAL DATA multiple feature map with different weights to extract multiple As previously shown, the complexity of recognition of features from each location of the input image. All units in a handwritten numerals depends on many variables and is single feature map share the same set of weights (5×5) and biases increased due to variables such as the variation of handwritings. to detect the same feature at all possible locations on the input. The proposed method is validated with 5 well known large In this way, every feature map is trying to detect different local databases of 5 different languages. Modified Arabic feature. The purpose of the pooling layers is to condense and Handwritten Digits Databases (MADBase) is the first database simplify the information at the output of these convolution used. MADBase was compiled from 700 writers [30], and layers. There are several reasons to choose convolution layers comprises of 70,000 handwritten Arabic digits with 300 dpi over fully connected neural nets, some of these includes. resolution at 28 ×28 pixels. The Modified National Institute of • Standards and Technology (MNIST) is the second database used From experience convolution networks tends to perform is Eastern Arabic. MNIST was developed using 250 writers and better than the fully connected neural networks. consists of 60,000 training and 10,000 testing numerals [31]. • Convolutional networks are robust to shifts and The HODA database is used for Persian numerals [32]. HODA distortions in the image as output of the feature map will contains 80,000 Persian numerals compiled from 12,000 be shifted by the same amount as the input image [28]. registration forms from university entrance exams. In this paper, a database was developed for the Urdu numerals database • Using individual image pixel as input in fully connected consisting 8,500 samples from which 6,500 numerals are used neural nets would not take advantage of the fact that for training and 2,000 are used for testing. Experiments are also images are highly spatially coordinated. performed on Devanagari numerals datasets named DHCD • Since hidden convolution layers use shared weights and respectively [33]. A summary of the datasets used for the testing biases, so it greatly reduces the number of parameters of the proposed system are shown in Table 2.

Fig. 2. Proposed Deep CNN Architecture for the Numeral Recognition

V. RESULTS AND DISCUSSIONS TABLE III. COMPARISON OF RECOGNITION RATES FOR DIFFERENT The proposed deep learning architecture was tested on the LANGUAGES NUMERALS WITH 1 CNN LAYER AND 2 CNN LAYERS above mentioned datasets. After preprocessing the digits in the CNN 1 Layer CNN 2 Layers databases, the numerals are input to the deep learning architecture developed in this study. As shown in table 3, the Eastern Arabic 99.21% 99.30% database were input to the proposed CNN with 1 hidden layer Persian 98.57% 98.82% and 2 hidden layers respectively. As seen, the average Devanagari 99.47% 99.73% recognition rate using 1 hidden layer is 99.10% and the average Urdu 99.14% 99.33% recognition rate using 2 hidden layers is 99.32%. It has been Western Arabic (English) 99.13% 99.43% proven in previous literature that more hidden layers would produce better recognition rate. The results indicate that with two hidden layers better recognition rates were achieved. The combined Multilanguage unified system can However, with more hidden layers, the more complex the recognize numerals of any language without predefining the network and more computational time is needed to produce specific language which make the recognition task more results. For the purpose of this study, the recognition rate of challenging. Table 4 show the accuracy rates for all 5 languages 99.32% is adequate and adding more layers increasing the time separately and for each digit within each language. For example, to produce results will not be justified. The rest of the results in the accuracy rate for digit 0 has an average of 98.96% in all this section are tested on a 2 hidden layer architecture. languages while digit 0 in Western Arabic (English) has 99.29% accuracy rate. In general, the average accuracy for all digits in all 5 languages is greater than 99.33%. The comparison of the TABLE II. DETAILS OF THE DATASETS USED FOR TESTING THE PROPOSED SYSTEM proposed method shows better recognition rates as compared to those methods discussed in the previous studies in the literature Numeral Training Testing Database Data Source review section along with their experimental results. Language Dataset Dataset Eastern Arabic MADBase 700 volunteers 60,000 10,000 Western Arabic MNIST 250 participants 60,000 10,000 TABLE IV. COMPARISON OF ACCURACY RATES FOR EACH INDIVIDUAL (English) NUMERAL OF ALL LANGUAGES Persian HODA 12000 Forms 60,000 20,000 Digit Eastern Western Urdu PMUdb 170 volunteers 6,500 2,000 Persian Devanagari Urdu Average Class Arabic Arabic Devanagari DHCD NA 17,000 3,000 0 98.01% 98.73% 99.29% 100.00% 98.76% 98.96% 1 99.10% 97.87% 99.91% 99.67% 98.74% 99.06% Fig. 3 shows the error rate for each numeral in each language and indicates the confusion between certain digits in each 2 98.52% 98.22% 99.32% 99.34% 98.57% 98.79% language. For example, in Arabic the error rate for the digit zero 3 99.90% 98.74% 99.51% 100.00% 99.73% 99.58% is about 0.996%, however, what Fig. 2 shows is that the digit 0 in Arabic is confused as the digits 5 for about 0.896% of the error 4 99.80% 98.71% 99.19% 99.67% 99.20% 99.31% rate. This is a logical results since the numeral 5 has similar 5 98.80% 98.81% 99.22% 99.67% 99.13% 99.12% geometrical features with the numeral 0. In Persian, the digit 0 is mostly confused with the digit 1 and digit 5. To explain the 6 99.70% 99.00% 99.79% 99.67% 99.60% 99.55% definition of target and predicted, an example will be used. For 7 99.90% 99.55% 99.22% 100.00% 99.90% 99.71% example if we input an image for the digit 0 (target image) and 8 99.90% 99.65% 99.59% 99.34% 99.90% 99.68% after all three phases are complete, the system recognizes the image wrongly as 5 (predicted). 9 99.40% 98.95% 99.60% 100.00% 99.77% 99.54%

3.00% 0 1 2 3 4 5 6 7 8 9 2.70% 2.40% 2.10% 1.80% 1.50%

Error Rate 1.20% 0.90% 0.60% 0.30% 0.00% 01234567890123456789012345678901234567890123456789 Arabic Persian English Dawanageri Urdu Numerals of different Languages

Fig. 3. Comparison of Targets vs. Predicted digits error rates

Finally and as an ultimate test for the proposed CNN, the TABLE V. RECOGNITION RATES FOR COMBINED ALL 5 LANGUAGES database for all 5 languages is combined into one gigantic NUMERALS Digit FP Language TP Rate Precision database and table 6 shows the results. The average accuracy rate Class Rate achieved was 99.26% and average FP rate 0.02% and average 0 98.10% 0.10% 99.10% precision of 99.29%. These results show the superiority of the proposed architecture in the recognition of the multi-language 2 99.20% 0.10% 98.20% numerals. Eastern Arabic, Persian, 3 98.40% 0.00% 99.40% Urdu 5 98.80% 0.10% 98.70% VI. CONCLUSION 8 99.90% 0.00% 99.90% In this study, a novel deep learning CNN is proposed for the 9 99.10% 0.10% 99.30% recognition of multi-language numerals. It was shown that the Eastern Arabic, Persian 7 99.60% 0.00% 99.50% more the hidden layers the better the recognition rate, however, Eastern Arabic 4 99.20% 0.00% 99.90% the more the hidden layers the more time and complexity in Eastern Arabic, Urdu 6 99.70% 0.00% 99.60% producing results. It was determined that 2 hidden layers is a 6 99.20% 0.00% 99.30% compromise that produces adequate recognition rates. The Persian experimental results indicate that the proposed architecture 4 99.20% 0.00% 99.30% produces better accuracies on both the individual languages or 4 99.20% 0.10% 99.40% Urdu combined languages database that have similar geometrical 7 99.90% 0.00% 99.70% features or have combined different geometrical features 2 99.50% 0.00% 99.40% respectively. The overall average accuracy for the combined Multilanguage database was 99.26% with an average precision 4 99.40% 0.00% 99.20% 5 99.20% 0.00% 99.00% of 99.29. The average accuracy for each individual language was Western Arabic 99.322%. The results achieved in this study far exceeds those 6 98.90% 0.00% 99.50% mentioned in previous literature for systems proposed to 7 99.40% 0.00% 98.80% recognize these languages. For future work, the authors will 8 99.30% 0.00% 99.40% explore a deep learning architecture for universal numeral 0 98.80% 0.10% 98.40% recognition. This will be also expanded for character recognition Western Arabic, Devanagari as well. 3 99.20% 0.00% 99.40% 1 99.70% 0.00% 99.30% 2 99.00% 0.00% 98.00% 4 99.70% 0.00% 99.70% Devanagari 5 99.00% 0.00% 99.70% 6 98.70% 0.00% 99.70% 7 99.70% 0.00% 100.00% 8 100.00% 0.00% 99.70% Eastern Arabic, Persian, 1 99.60% 0.10% 98.80% Urdu. Western Arabic Average 99.26% 0.02% 99.29%

REFERENCES [17] Ciresan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2011, September). Convolutional neural network committees for handwritten character classification. In Document Analysis and Recognition (ICDAR), [1] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep 2011 International Conference on (pp. 1135-1139). IEEE. learning. Nature, 521(7553), 436-444. [18] Asthana, S., Haneef, F., & Bhujade, R. K. (2011). Handwritten multiscript [2] Schmidhuber, J. (2015). Deep learning in neural networks: An numeral recognition using artificial neural networks. International overview. Neural networks, 61, 85-117. Journal of Soft Computing and Engineering, 1(1), 1-5. [3] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning [19] Pal, A., Khonglah, B. K., Mandal, S., Choudhury, H., Prasanna, S. R. M., Applied to Document Recognition, Proceedings of the IEEE, Rufiner, H. L., & Balasubramanian, V. N. (2016, March). Online Bengali 86(11):2278-2324, November 1998 handwritten numerals recognition using Deep Autoencoders. [4] Krizhevsky, A., Sutskever, I. and Hinton, G. E. ImageNet Classification In Communication (NCC), 2016 Twenty Second National Conference with Deep Convolutional Neural Networks NIPS 2012: Neural on (pp. 1-6). IEEE. Information Processing Systems, Lake Tahoe, Nevada [20] Wu, Y. C., Yin, F., & Liu, C. L. (2017). Improving handwritten Chinese [5] M. D. Zeiler and R. Fergus. Stochastic pooling for regularization of deep text recognition using neural network language models and convolutional convolutional neural networks. CoRR, abs/1301.3557, 2013. neural network shape models. Pattern Recognition, 65, 251-264. [6] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, [21] Huang, X., Nie, X., Wu, W., Qiao, H., & Zhang, B. (2017). SAR target D., Vanhoucke, V. & Rabinovich, A. (2015). Going deeper with configuration recognition based on the biologically inspired convolutions. In Proceedings of the IEEE Conference on Computer model. Neurocomputing, 234, 185-191. Vision and Pattern Recognition (pp. 1-9). [22] Amato, G., Falchi, F., & Vadicamo, L. (2016). Visual recognition of [7] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional ancient inscriptions using convolutional neural network and fisher networks for large-scale image recognition. arXiv preprint vector. Journal on Computing and Cultural Heritage (JOCCH), 9(4), 21. arXiv:1409.1556 [23] Boukharouba, A., & Bennia, A. (2015). Novel feature extraction [8] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for technique for the recognition of handwritten digits. Applied Computing image recognition. In Proceedings of the IEEE Conference on Computer and Informatics. Vision and Pattern Recognition (pp. 770-778). [24] Karimi, H., Esfahanimehr, A., Mosleh, M., Salehpour, S., & Medhati, O. [9] Huang, G., Liu, Z., Weinberger, K. Q., & van der Maaten, L. (2016). (2015). Persian Handwritten Digit Recognition Using Ensemble Densely connected convolutional networks. arXiv preprint Classifiers. Procedia Computer Science, 73, 416-425. arXiv:1608.06993. [25] Jie, M., Aziz, I. A., Hasbullah, H., & Azizan, S. A. B. (2016, August). [10] Alghazo, J. M., Latif, G., Elhassan, A., Alzubaidi, L., Al-Hmouz, A., & Handwritten digits recognition based on improved label propagation Al-Hmouz, R. (2017). An Online Numeral Recognition System Using algorithm. In Computer and Information Sciences (ICCOINS), 2016 3rd Improved Structural Features–A Unified Method for Handwritten Arabic International Conference on (pp. 345-350). IEEE. and Persian Numerals. Journal of Telecommunication, Electronic and [26] Al_barraq, M.O. and Mehrotra, S.C., 2015. Recognition of Arabic Computer Engineering (JTEC), 9(2-10), 33-40. Handwritten Amount in Cheque through Windowing Approach. [11] Singh P., Verma A., Chaudhari N.S. (2016) Deep Convolutional Neural International Journal of Computer Applications, 115(10), pp. 33-38. Network Classifier for Handwritten Devanagari Character Recognition. [27] Fan, J. L., & Zhao, F. (2007). Two-dimensional Otsu's curve thresholding In: Satapathy S., Mandal J., Udgata S., Bhateja V. (eds) Information segmentation method for gray-Level images. Dianzi Xuebao(Acta Systems Design and Intelligent Applications. Advances in Intelligent Electronica Sinica), 35(4), 751-755. Systems and Computing, vol 434. Springer, New Delhi. [28] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based [12] Iamsa-at, S., & Horata, P. (2013, December). Handwritten character learning applied to document recognition. Proceedings of the recognition using histograms of oriented gradient features in deep IEEE, 86(11), 2278-2324. learning of artificial neural network. In IT Convergence and Security [29] Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., & (ICITCS), 2013 International Conference on (pp. 1-5). IEEE. Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural [13] Sazal, M. M. R., Biswas, S. K., Amin, M. F., & Murase, K. (2014, networks from overfitting. Journal of Machine Learning Research, 15(1), February). Bangla handwritten character recognition using deep belief 1929-1958. network. In Electrical Information and Communication Technology [30] Abdelazeem, S., 2009. Comparing arabic and latin handwritten digits (EICT), 2013 International Conference on (pp. 1-5). IEEE. recognition problems. World Academy of Science, Engineering and [14] Zhao, Z., Jiao, L., Zhao, J., Gu, J., & Zhao, J. (2017). Discriminant deep Technology, 54. belief network for high-resolution sar image classification. Pattern [31] Deng, L., 2012. The MNIST database of handwritten digit images for Recognition, 61, 686-701. machine learning research [best of the web]. IEEE Signal Processing [15] Alghazo, J. M., Latif, G., Elhassan, A., Alzubaidi, L., Al-Hmouz, A., & Magazine, 29(6), pp.141-142. Al-Hmouz, R. (2017). An Online Numeral Recognition System Using [32] Khosravi, H. and Kabir, E., 2007. Introducing a very large dataset of Improved Structural Features–A Unified Method for Handwritten Arabic handwritten Farsi digits and a study on their varieties. Pattern recognition and Persian Numerals. Journal of Telecommunication, Electronic and letters, 28(10), pp.1133-1141. Computer Engineering (JTEC), 9(2-10), 33-40. [33] Acharya, Shailesh, Ashok Kumar Pant, and Prashnna Kumar Gyawali. [16] Ciresan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J., 2010, "Deep learning based large scale handwritten Devanagari character Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition, recognition." In Software, Knowledge, Information Management and Neural Computation, 22(12) 2010, pp. 1-14. Applications (SKIMA), 2015 9th International Conference on, pp. 1-6. IEEE, 2015.

Authorized licensed use limited to: Prince Mohammad Bin Fahd University. Downloaded on August 13,2020 at 11:49:58 UTC from IEEE Xplore. Restrictions apply.