applied sciences Article A Convolutional Neural Network Based Auto Features Extraction Method for Tea Classification with Electronic Tongue Yuan hong Zhong 1,* , Shun Zhang 1, Rongbu He 2, Jingyi Zhang 1, Zhaokun Zhou 1, Xinyu Cheng 1, Guan Huang 1 and Jing Zhang 1 1 Department of School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China; [email protected] (S.Z.); [email protected] (J.Z.); [email protected] (Z.Z.); [email protected] (X.C.); [email protected] (G.H.); [email protected] (J.Z.) 2 Electric Power Research Institute of Guizhou Power Grid Co., Ltd., Guizhou 550007, China; [email protected] * Correspondence: [email protected]; Tel.: +86-1375-2908-255 Received: 11 May 2019; Accepted: 18 June 2019; Published: 20 June 2019 Abstract: Feature extraction is a key part of the electronic tongue system. Almost all of the existing features extraction methods are “hand-crafted”, which are difficult in features selection and poor in stability. The lack of automatic, efficient and accurate features extraction methods has limited the application and development of electronic tongue systems. In this work, a convolutional neural network-based auto features extraction strategy (CNN-AFE) in an electronic tongue (e-tongue) system for tea classification was proposed. First, the sensor response of the e-tongue was converted to time-frequency maps by short-time Fourier transform (STFT). Second, features were extracted by convolutional neural network (CNN) with time-frequency maps as input. Finally, the features extraction and classification results were carried out under a general shallow CNN architecture. To evaluate the performance of the proposed strategy, experiments were held on a tea database containing 5100 samples for five kinds of tea. Compared with other features extraction methods including features of raw response, peak-inflection point, discrete cosine transform (DCT), discrete wavelet transform (DWT) and singular value decomposition (SVD), the proposed model showed superior performance. Nearly 99.9% classification accuracy was obtained and the proposed method is an approximate end-to-end features extraction and pattern recognition model, which reduces manual operation and improves efficiency. Keywords: electronic tongue; tea classification; auto features extraction; convolutional neural network 1. Introduction Tea is one of the most prevailing beverages across the world. The practice of drinking tea has been a long history in China. Tea contains theine, cholestenone, inose, folic acid and other components, which can improve humanity health. In the actual tasting process, this kind of mellow and fragrant taste of tea stimulates people’s taste buds. There are many external conditions that affect the taste of tea, for example, number of tea leaves added, tea making utensils, tea making time, water quality and the way tea is stored. The synergistic effects of these factors make the unique taste of tea. Organic compounds with different chemical structures and concentrations play a significant role in the quality of tea. The ingredients in tea are very complicated, the most important of which are tea polyphenols, amino acids, alkaloids and other aromatic substances. Tea classification has a wide range of application scenarios. It plays an important role in tea quality estimation, for example, new or old, true or fake judgment of tea. Moreover, the classification Appl. Sci. 2019, 9, 2518; doi:10.3390/app9122518 www.mdpi.com/journal/applsci Appl. Sci. 2019, 9, 2518 2 of 15 Appl. Sci. 2019, 9, x FOR PEER REVIEW 2 of 15 and identification of tea provide a reference for healthy tea drinking. What’s more, the classification and recognitionrecognition ofof tea tea are are more more likely likel toy provideto provide the th basise basis for thefor designthe design of smart of s teapotmart teapot in the future.in the future.Traditional tea quality assessment methods are based on different analytical instruments, such as highTraditional performance tea liquid quality chromatography assessment methods [1], gas are chromatography based on different [2] andanalytical plasma instruments, atomic emission such asspectrometry high performance [3]. However, liquid these chromatography methods require [1] a, lotgas of chromatography technical personnel, [2] materialand plasma and financial atomic emissionsupport, whichspectrometry lead to [3]. low However, efficiency these and largermethods overhead require [a4 ].lot With of technical the development personnel, ofmaterial sensor andtechnology, financial the support, advantages which of lead methods to low based efficiency on sensor and technology larger overhead are more [4]. distinguished.With the development The typical of sensoradvantages technology, are high the accuracy, advantages simple of methods operation based and on fast sensor detection, technology which are improve more distinguished the efficiency. Theof tea typical quality advantages inspection are obviously. high accuracy, At the simple same time, operation the electronic and fast nosedetection, for gas which analysis improve has alsothe efficiencymade technological of tea quality breakthroughs inspection [obviously5,6]. The arrays. At the of same electrochemical time, the electronic sensors and nose devices for gas have analysis been hasdesigned also made for the technological analysis of complex breakthroughs liquid samples,[5,6]. The such arrays as tasteof electrochemical sensor and electronic sensors tongues and devices [7,8]. Ashave a modernbeen designed intelligent for sensorythe analysis instrument, of complex electronic liquid tongue samples, is skillful such as in taste monitoring sensor theand production electronic toncyclegues of beverage,[7,8]. As a and modern has the intelligent advantages sensory of simple, instrument, fast and electronic low cost, tongue which showsis skillful huge in monitoring potential in thebeverage production quality cycle evaluation of beverage, [9]. and has the advantages of simple, fast and low cost, which shows hugeFigure potential1 shows in beverage the basic quality flow of evaluation the electronic [9]. tongue system for beverage detection and quality evaluation. First, the response signals from the e-tongue hardware system are collected. Then, features of sampling raw data are extracted for pattern recognition. Electronic tongue Sensor response Feature extraction Pattern recognition hardware system Figure 1. TheThe basic basic flow flow of the electronic tongue system for liquid detection. Pattern recognition and features extraction are the two most important parts of the electronic Pattern recognition and features extraction are the two most important parts of the electronic tongue system for liquid classification. Many scholars have contributed to tea pattern recognition. tongue system for liquid classification. Many scholars have contributed to tea pattern recognition. For instance, principal component analysis (PCA) [10], artificial neural network (ANN) [11], support For instance, principal component analysis (PCA) [10], artificial neural network (ANN) [11], support k k vector machine (SVM) [9,12 [9,12]],, k--nearestnearest neighbor algorithm ( k--NN)NN) [13] [13],, r randomandom forest (RF) [14] [14] and autoregressive (AR)(AR) model model [4 ][4 have] have been been put put forward forward for classificationfor classification analysis analysis in the e-tonguein the e-tongue system. system.Features Features extraction extraction is another is an criticalother critical step of step the of e-tongue the e-tongue as the as quality the quality of features of features selection selection will willdirectly directly affect theaffect quality the ofquality pattern of recognition. pattern rec Generallyognition. speaking,Generally feature speaking, extraction feature is advantageousextraction is advantageousfor three main purposes,for three namely:main purposes, (1) Reduction namely: of random (1) Reduction noise, (2) of reduction random ofnoise, unwanted (2) reduction systematic of variationsunwanted whichsystematic are often variations due to experimental which are conditions, often due (3) to data experimental compression inconditions order to capture, (3) data the compressionmost relevant in information order to capture from signals.the most There relevant are diinformationfferences in from the implementation signals. There are details differences of feature in theextraction implementation methods in details different of typesfeature of electronicextraction tonguemethods systems. in different However, type ass farof aselectronic we know, tongue these systems.methods areHowever, similar inas principle,far as we whichknow, canthese be dividedmethods into are the similar following in principle three categories:, which can Features be divided with physicalinto the meaning,following features three acquiredcategories: by mathematicalFeatures with transformation, physical meaning, features features in frequency acquired domain. by mathematicalIn terms of voltammetry transformation e-tongue,, features in the in early frequency stage, thedomain. features In extractionterms of voltammetry methods of samples e-tongue, were in therelatively early stage, simple, the for features example, extraction raw data methods which wereof sampl collectedes were at relatively a fix frequency simple, from for samplesexample, were raw datatreated which as features were collected [15]. Then, at peaka fix valuefrequency and inflectionfrom sampl pointes were (the maximum treated as value, features