Identification of Dialects: Survey

International Journal of Advanced Science and Technology Vol. 29, No. 3, (2020), pp. 10733 - 10739 `

S. Shivaprasad 1 Dr. M Sadanandam2 1 Research Scholar , Department of CSE, Kakatiya University ,Warangal & Assistant Professor Department of CSE ,VFSTR University, GUNTUR. 2Assistant Professor,Department of CSE , Kakatiya University ,Warangal 1 [email protected]

ABSTRACT Automatic Dialect Identification plays a crucial role for constructing an Automatic speech recognition system in an appreciable manner in signal processing. We can mention dialect as property of a language that varies from standard version of that language depending upon the region. Dialect can be identified from speaker’s vocabulary, articulation, grammar and some other aspects like loudness, tonality and nasality. Identifying dialect exactly and properly will help in making some applications and services to work in an efficient manner such as e-learning and many such fields that is more helpful for homebound, aged ones. Dealing with dialect identification is very difficult due to factors like insufficient databases, subtle to regional boundaries, variations in languages. It is a tedious analysis procedure. Due to this factor dialect identification became crucial among research topics. Through this paper, we explain the process that is performed in identification various dialects and also the work done up to now, thereby providing information on what might be expected in coming years. Keywords: Dialects, Automatic Dialect Identification (ADI), Speech recognition Systems (SRS), automatic speech recognition ( ASR)

1. INTRODUCTION Language acts a medium of communication for humans. In speech recognition systems we use same language for various purposes. Dialect of language is considered as one of the problem for automatic speech recognition systems. Some speech recognition systems directly deal with the input audio and others deals the transformed audio signal. There is a need for finding the features of the audio signal to classify the audio. Recognition of audio signal based on dialects is next level for categorizing the speech recognition system. As of now many researchers has been working towards dialect recognition system in various languages. Dialect is defined as the language that has been habituated by the people of certain area.

International Journal of Advanced Science and Technology Vol. 29, No. 3, (2020), pp. 10733 - 10739 `

Automated dialect recognition systems are used to identify the dialect of a particular language. The training of the dialect recognition system is done by means of audio samples collected from a particular region. Automatic speech recognition systems has gained their importance in both academics and in industries [1]. It has a high impact on the society. Now-a-days speech recognition systems are given more importance by people rather than text formats because of having faster query processing as its advantage [3] [18]. Speech recognition systems are present now-a-days in electronic gadgets and there is a need for dialect recognition system for its improvement [19]. Dialect recognition systems are useful for people who are old and homebound by providing good e-health, telecommunication services. Day-by-day there has been an improvement in automated speech recognition systems. Dialect recognition systems will have an impact on speech recognition systems because it is adding an important feature for identifying the speaker [20]. In speech dialect is present in different segmental levels they are segmental, supra-segmental and sub-segmental. Developing a a good dialect recognition system may cause  Improvements in applications that work with human interaction.  Securing the communications that involve remote accessing.  Refining the searches in electronic gadgets which work with the help of speech recognition systems. Dialect recognition system is considered as a more complex problem because there exists more likeness between vernaculars of same language [21]. Language tongues are utilized by the individuals to post status, speak with companions and other important aspects in social media. There is a requirement for new models to identify what is present in internet as well as news articles. Therefore, inorder to understand all these things dialect recognition system is considered as very important.

2. LITERATURE SURVEY George [1] conducted surveys in 1877 to identify the dialect. Barly [2] worked on Midland dialect whether it is presnt or not. He identified that the dialect will not deepends upon people prounciation of similar community region. From this, work on dialect is started. Imène GUELLIL, Faiçal AZOUAOU [3] worked on Arabic dialects identification and they identified new way about dialect identification. Thy applied techniques and algorithm that used in Algerian dialect. For this they had built and developed some dictionary of words that provides transfiguration between Algerian dialect and French with 25086 words. After that they proposed a calculation to cut down the messages in online networking into expressions and attempted to perceive every single expression by utilizing the words that were at that point developed in the dictionary.

International Journal of Advanced Science and Technology Vol. 29, No. 3, (2020), pp. 10733 - 10739 `

They considered mainly three techniques for identification. They are total, partial and by using Improved Levenshtein distance in which the terms in the constructed dictionary are identified but with differences of few letters. It included the length of the word thus differs from classical distance of Levenshtein. In order to apply their approach they relied on a corpus (collection) that was extracted using Facebook API from January 2015.They had considered a small corpus (with 100 messages in order to analyze their algorithm by focusing on the time taken to process such a huge collection [22] [23]. The outcomes of that algorithm were good enough and gave rating exceeding 60% for all three above mentioned types of identification with a distance less than or equal to 0.3, consequently with small differences in letters). It is to extend the work to EGY, TUN, Iraqi and other. To extend the dialect lexicon that is used by using an API on the net in order to collect the words from Algerian dialect. So, they startedwith a larger lexicon and enriched it [24]. Sreeraj V V Rajeev Rajan [4] had proposed new model consists feature-level fusion of MFCC (Mel Frequency Cepstral Coefficient) and TEO (teager energy operator). In classification phase a classifier based upon Support Vector Machine (SVM) is used. The systematic evaluation of the system which was proposed is executed upon a database containing the Malayalam dialect that was developed in studio environment. The database will have 4 dialects and each dialect will have 300 speech samples. The system based upon MFCC reported 65% of accuracy, system based upon TEO gave 73.33% of accuracy. The combined system exhibited some development with 78% of total accuracy.

In [14], Dialect Identification of Assamese Language was proposed. Thy applied different methods like GMM and GMM-UBM. For the identidication purpose they created 13 hours and 30 minutes data of spontaneous speech. Even some sort of similarities is present between Goalparia,Kamrupi dialects and Assamese language, GMM-UBM provides good accuracy

International Journal of Advanced Science and Technology Vol. 29, No. 3, (2020), pp. 10733 - 10739 `

with 98.3% compare with GMM accuracy of 85.7%. So that thy conclude GMM-UBM is best modelling technique but they are not applied prosodic features and they does not check accuracy is incresed or decreased if database size is increased. Tanvira Ismail, Gaurab Krishnan Deka,[15] they are identifid Kamrupi Dialect with data set used is 10 hrs and 32 mins of spontaneous speech. These are applied GMM and GMM-UBM to identify the dialects. They identified GMM-UBM provides 99.5% compare to GMM with accuracy of 98.6% . Hence, they identified that GMM-UBM is better compared with GMM. However, they are not worked on the prosodic features . In [6],Mahnoosh Mehrabani proposed an analysis for identifying dialect/language sets automatically, which is based ontheprimitive differences between them. The differences were explored on the basis of the data that is available.. Initially, a method to measure the spectral acoustic differences between dialects was proposed which is on the basis of the volume space analysis within a 3D model. It uses MFCC derived log likelihood score distributions and Gaussian Mixture Models(GMM) [25]. Later, for studying the excitation structure differences among the dialects, energy and pitch contour primitives based text-independent prosody features were put forward. Dialect proximity measures were also proposed which were evaluated on Arabic dialects and on South languages of India. The measures presented by them are shown to be consistent and repeatable. However, they did not consider the lexical contrasts like word selection, grammatical structure or wider supra segmental differences for the evolution of specific dialects [26]. In [17], a user-friendly prototype was developed which can be installed on NAO robots to command them using speech. HMM-GMM was primarily adopted in this. The outcome from the experiments showed that a high accuracy was achieved by the prototype. But in this, prototype one was used and it can be further trained to identify distinct accents in different languages [27]. In[18] it was suggested that we can group dialects based on the speech sign's acoustic attributes. They used word level features to identify the dialects. The Intonational Variations in English (IViE) discourse corpus was thought of. For the development of the element vectors, acoustic properties are gotten from word level and for the extraction of colossal models, SVM and tree-based XGB(Xtreme Gradient Boosting) group calculations were utilized to separate speech. Be that as it may, phoneme or syllables for recognizing vernaculars were not considered here. In[7], a strategy for recognizing Spoken Dialects of Indonesian by Classification technique utilizing SVM and K-implies procedure for grouping was proposed. Eight tongues of the Indonesian language were distinguished. They isolated their investigation into three basing on

International Journal of Advanced Science and Technology Vol. 29, No. 3, (2020), pp. 10733 - 10739 `

Mel Frequency Cepstral Coefficient (MFCC), otherworldly transition, and ghostly centroid and afterward contrasted the outcome with a model that just has MFCC highlights. One- against-every last one without a moment's delay were utilized for drawing contrast and the best outcome, which was 55% of precision, is acquired from SVM one-against-one with three highlights.

Mengistu, Melesew[16], they worked on identify dialects in Amharic Language. They used hybrid approach VQ and GMM.For this purpose they considered 100 speakers for each region dialect. They used different feature extraction techniques like Mel frequency cepstral coefficients (MFCC) , ΔMFCC and ΔΔMFCC. They got 85.9% accuracy with 25 speakers and 92.7% for 100 speakers. In [5], Saud Khan used MFCC and SVM. Vocal samples were collected from different categories of people. The optimum class boundaries Cepstral coefficients and statistical parameters are used by support vector machines, defines are present in the feature sets that are derived from the database. In was concluded that SVM based dialect identification system outputs an optimal result in differentiation of dialects. László Czap[4] the Shaanxi Xi’an dialect’s phonetical transcription was addressed which was developed as a phonetical alphabet that a computer can read to cover the fundamental phonemes of this dialect. This transcription was set depending upon the Shaanxi Xi’an dialect’s phonetical alphabet which consists of similar phonemes to Mandarin and other unique phonemes of the dialect. This can be used in the development of the basics of a talking head which is the Shaanxi Xi’an dialect’s animated articulation model. Suwon Shon1, Ahmed Ali [3] proposed a system using both voice and dialectal characteristics for the MGB-3 challenge. Various ways to handle the variations among dialects, and domain dissimilarities between the training as well as test sets were addressed in this. Without having an information about the domain of test where the system will be used, a Siamese network’s i-

International Journal of Advanced Science and Technology Vol. 29, No. 3, (2020), pp. 10733 - 10739 `

vector dimensionality reduction was found to be useful, where an interpolated i-vector dialect model exhibited relatively less performance while dealing with small portions of information from test domain collected from development data. In both cases, combination of audio and dialectal features guaranteed significant improvements in dialect recognition.

3. CONCLUSION In this paper, we have explained about what are different methods used to automatically identify the dialects of a language by using Spectral and Prosodic features. Now a day’s research in dialects is having an equal importance to Language identification. But in future, we should implement standardization mechanisms to get more accurate results to identify dialects of language because as it is more sensitive and also drastically changes by the nature.

REFERENCES [1]. Imene Guellil, Faical Azouaou. "Arabic Dialect Identification with an Unsupervised Learning (Based on a Lexicon). Application Case: ALGERIAN Dialect",IEEE Intl Conferenceon Computational Science and Engineering .DCABES 2016. [2]. V V Sreeraj, Rajeev Rajan. "Automatic dialect recognition using feature fusion", International Conference on Trends in Electronics and Informatics (ICEI), 2017. [3]. Suwon Shon, Ahmed Ali, James Glass. "MITQCRI Arabic dialect identification system for the 2017 multi-genre broadcast challenge", IEEE Automatic Speech Recognition 2017. [4]. Laszlo Czap, Lu Zhao. "Phonetic aspects of Chinese Shaanxi Xi'an dialect", IEEE International Conference on Cognitive Info communications (CogInfoCom),2017. [5]. Saud Khan, Haider Ali, Khalil Ullah. "Pashto language dialect recognition using mel frequency cepstral coefficient and support vector machines", International Conference on Innovations in Electrical Engineering and Computational Technologies(ICIEECT), 2017. [6]. Mehrabani, Mahnoosh, and John H. L. Hansen. "Automatic analysis of dialect/languagesets", International Journal of Speech Technology, 2015. [7]. Jacqueline Ibrahim, Dessi Puji Lestari. "Classification and clustering to identify spokendialects in Indonesian", 2017 International Conference on Data and Software Engineering(ICoDSE), 2017 [8]. Mona Abdullah Al-Walaie, Muhammad Badruddin Khan. "Arabic dialects classification using text mining techniques", International Conference on Computer and Applications (ICCA), 2017 [9]. Tanvira Ismail, L. Joyprakash Singh. "Identification of Goalparia dialect and similar languages", International Conference on Multimedia, Signal Processing and Communication Technologies (IMPACT), 2017 [10]. Intelligent Systems Conference (IntelliSys), 2015 [11]. Automatic Text Independent Language Identification Using Reduct Set of Feature Vectors by M. Sadanandam, V. Kamakshi Prasad, Springer,2013. [12]. Predicting the Intelligibility of Noisy and Nonlinearly Processed Binaural Speech Asger Heidemann Andersen, Jan Mark de Haan, Zheng-Hua Tan, Senior Member, IEEE, and Jesper Jensen,2016. [13]. Automatic Dialect and Accent Recognition and its Application to Speech Recognition by Fadi Biadsy,IEEE 2011. [14]. Dialect Identification of Assamese Language using Spectral Features by Tanvira Ismail and L. Joyprakash Singh Indian Journal of Science and Technology,May 2017 [15]. Identification of Kamrupi Dialect and Similar Languages by Tanvira Ismail, Gaurab Krishnan Deka International Conference on Signal Processing and Integrated Networks (SPIN),2017 [16]. Text Independent Amharic Language Dialect Recognition: A Hybrid Approach of VQ and GMM by Abrham Debasu Mengistu and Dagnachew Melesew International Journal of Signal Processing 2017. [17]. A Novel Approach of System Design for Dialect Speech Interaction with NAO Robot by Ming CHEN1 , Lujia WANG2, Cheng-zhong XU3, ICAR 2017.

International Journal of Advanced Science and Technology Vol. 29, No. 3, (2020), pp. 10733 - 10739 `

[18]. K. S (2020),” An Iterative Group Based Anomaly Detection Method For Secure Data Communication in Networks”, Journal of Critical Reviews, Vol 7, Issue 6, pp:208-212. doi: 10.31838/jcr.07.06.39. [19]. Banavathu Mounika, (2020), “ Use of BlockChain Technology In Providing Security During Data Sharing”, Journal of Critical Reviews, Vol 7, Issue 6, pp:338-343. doi: 10.31838/jcr.07.06.59. [20]. V.L.N, BNS,(2020),” Fuzzy Base Artificial Neural Network Model For Text Extraction From Images”, Journal of Critical Reviews, Vol 7, Issue 6,pp:350-354, doi: 10.31838/jcr.07.06.61. [21]. VLN,APG (2020),” Accurate Identification And Detection Of Outliers In Networks Using Group Random Forest Methodoly”, Journal of Critical Reviews, Vol 7, Issue 6,pp:381-384, doi: 10.31838/jcr.07.06.67. [22]. Sandhya Pasala, (2020),” Identification Of Attackers Using Blockchain Transactions Using Cryptography Methods”, Journal of Critical Reviews, Vol 7, Issue 6,pp:368-375, doi: 10.31838/jcr.07.06.65 [23]. C.R.Bharathi, L.V. Ramesh, (2020),” Secure Data Communication Using Internet of Things”, International Journal of Scientific & Technology Research, Volume 9, Issue 04,pp:3516-3520. [24]. Bharathi C R ,(2018),“Multi-mode Routing Algorithm with Cryptographic Techniques and Reduction of Packet Drop using 2ACK scheme in MANETs”, Smart Intelligent Computing and Applications, Vo1.1, pp.649-658. DOI: 10.1007/978-981-13-1921-1_63 [25]. Bharathi C R, (2018), “Effective multi-mode routing mechanism with master-slave technique and reduction of packet droppings using 2-ACK scheme in MANETS”, Modelling, Measurement and Control A, Vol.91, Issue.2, pp.73-76. DOI: 10.18280/mmc_a.910207 [26]. A Peda Gopi and N.Ashok Kumar,(2018),“ Different techniques for hiding the text information using text steganography techniques: A survey”, Ingénierie des Systèmes d'Information, Vol.23, Issue.6,pp.115-125. DOI: 10.3166/ISI.23.6.115-125 [27]. Chaitanya, K., and S. Venkateswarlu,(2016),"DETECTION OF BLACKHOLE & GREYHOLE ATTACKS IN MANETs BASED ON ACKNOWLEDGEMENT BASED APPROACH." Journal of Theoretical and Applied Information Technology 89.1: 228. [28]. Gopi, A., et al. "Designing an Adversarial Model Against Reactive and Proactive Routing Protocols in MANETS: A Comparative Performance Study." International Journal of Electrical & Computer Engineering (2088-8708) 5.5 (2015). [29]. Kumar, S. Ashok, et al. "An Empirical Critique of On-Demand Routing Protocols against Rushing Attack in MANET." International Journal of Electrical and Computer Engineering5.5 (2015). [30]. Acoustic Features based Word Level Dialect Classification using SVM and Ensemble Methods by Nagaratna B. Chittaragi*, Shashidhar G. Koolagudi, 2017 Tenth International Conference on Contemporary Computing ( IC3), 10-12 August 2017, Noida, [31]. Vasanthi, P., Jonnala, P., Janardhan Reddy, U., (2020), “Effective ensemble strategies for predicting the cardiac diseases”, International Journal of Advanced Science and Technology Vol. 29 No. 8s, pp: 1665- 1674. [32]. Janardhan Reddy, U., Krishna Rao, K.S., Nirupama Bhat, M., Sastry, K.B.S., (2016), “Falsification: An advanced tool for detection of Duplex code “, Indian Journal of Science and Technology. Vol:9, Issue:39, pp: 1-5.https://doi.org/10.1748/ijst/2016/v9i39/96195 [33]. Reddy, U.J., Dhanalakshmi, P., Reddy, P.D.K., (2019), “Image segmentation technique using SVM classifier for detection of medical disorders”, Ingenierie des Systemes d'Information, Vol. 24, No. 2, pp: 173-176. https://doi.org/10.18280/isi.240207 [34]. P. Sandhya Krishna, Ummadi Janardhan Reddy, R.S.M. Lakshmi Patibandla, Sk. Reshmi Khadherbhi (2020), “Identification of Lung Cancer Stages Using Efficient Machine Learning Framework” . Journal of Critical Reviews, 7 (6), 385-390. https://doi.org/10.31838/jcr.07.06.68