The Development of a Skin Image Analysis Tool by Using Machine Learning Algorithms
cosmetics
Article The Development of a Skin Image Analysis Tool by Using Machine Learning Algorithms
Perry Xiao 1,* , Xu Zhang 2, Wei Pan 3, Xiang Ou 4, Christos Bontozoglou 1 , Elena Chirikhina 1 and Daqing Chen 1
1 School of Engineering, London South Bank University, 103 Borough Road, London SE1 0AA, UK; [email protected] (C.B.); [email protected] (E.C.); [email protected] (D.C.) 2 Auckland Tongji Medical & Rehabilitation Equipment Research Centre, Tongjing Zhejiang College, JiaXing 314000, China; [email protected] 3 School of Computer Science and Engineering, Changshu Institute of Technology, Changshu 215500, China; [email protected] 4 College of Communication Engineering, Chongqing University, Chongqing 400044, China; [email protected] * Correspondence: [email protected]; Tel.: +44-20-7815-7569; Fax: +44-20-7815-7561
Received: 23 July 2020; Accepted: 20 August 2020; Published: 22 August 2020
Abstract: We present our latest research work on the development of a skin image analysis tool by using machine-learning algorithms. Skin imaging is very import in skin research. Over the years, we have used and developed different types of skin imaging techniques. As the number of skin images and the type of skin images increase, there is a need of a dedicated skin image analysis tool. In this paper, we report the development of such software tool by using the latest MATLAB App Designer. It is simple, user friendly and yet powerful. We intend to make it available on GitHub, so that others can benefit from the software. This is an ongoing project; we are reporting here what we have achieved so far, and more functions will be added to the software in the future.
Keywords: graphic user interface; deep learning; machine learning; capacitive imaging; skin image analysis
1. Introduction Skin imaging is very import in skin research. To date, there are many popular skin imaging technologies. Dermatoscopy is probably the most widely used skin imaging technology and is commonly used to examine skin lesions with a dermatoscope [1,2]. A dermatoscope typically consists of a magnifier, a light source (polarized or non-polarized) and a transparent plate. Skin photos and videos can be recorded by using a digital camera. DermLite is a small dermatoscope that can be attached to a smart phone. It claims to be the world’s best-selling dermatoscope [3]. DermLite II MultiSpectral is another pocket size dermatoscope, it is button activated, with cross-polarized 4-color imaging [4]. Nerasolutions provides several imaging devices, not only for skin imaging, but also for hair imaging [5]. Visia Skin Analysis System takes multiple photos of face and analyzes the melanin distributions, the texture, the wrinkle, the pores, etc. [6]. EPISCAN I-200 is a high resolution ultrasound imaging system that can take skin images using ultrasound up to 50 MHz [7]. Last not the least, Epsilon permittivity imaging system is a capacitive image technology developed in our research group [8–13]. It is based on dielectric constant measurement principle, and has been used for skin hydration imaging, skin solvent penetration imaging, as well as hair and nail imaging. With all these skin imaging technologies, there is a genuine need for a general purpose, user friendly and yet powerful skin software analysis tool. In this paper, we present our latest work on the development of a
Cosmetics 2020, 7, 67; doi:10.3390/cosmetics7030067 www.mdpi.com/journal/cosmetics Cosmetics 2020, 7, 67 2 of 12 Cosmetics 2020, 7, x FOR PEER REVIEW 2 of 13 developmentskin image analysis of a skin tool image based onanalysis machine-learning tool based on algorithms. machine-learning We will first algorithms. present the We design will first and presentthe structure the design of the skinand the software structure analysis of the tool, skin then software present analysis the results tool, and then discussions present the and results finally, and the discussionsfuture works. and finally, the future works.
2. Materials Materials and and Methods
2.1. Front-End Front-End GUI The front-end GUI (graphical user interface) is designed and developed by using MATLAB App Designer, which is relatively new and provides much better user interface components and a better integratedintegrated developmentdevelopment environment environment than than the the previous previous MATLAB MATLAB GUIDE. GUIDE. With With MATLAB MATLAB App AppDesigner, Designer, you can you easily can easily design design and develop and develop a user a friendly user friendly software software interface. interface. It offers It aoffers grid layouta grid layoutmanager manager to organize to organize the user the interface user interface and automatic and automatic reflow reflow options options to make to make the app the detectapp detect and andrespond respond to changes to changes in screen in screen size. size. The The coding coding is is also also a a lot lot simpler, simpler, with with machinemachine generatedgenerated code automatically grayed out. Figure Figure 11 showsshows thethe GUIGUI ofof thethe skinskin imageimage analysisanalysis softwaresoftware tool.tool. TheThe leftleft panel shows the original image and analyzed results. The right panel contains several tabs with each tab is dedicated to didifferentfferent functions. Apart Apart from from reading reading from from image files, files, it can also get image frames from webcams and other USB based video de devices.vices. This This is is done done by by using using “MATLAB “MATLAB Support Package forfor USB USB Webcams” Webcams” Add-On Add-On module module which haswhich shown has significant shown performancesignificant improvementperformance improvementthan the existing than video the existing input function video input from fu thenction image from acquisition the image toolbox. acquisition toolbox.
Figure 1. TheThe GUI (graphical user interface) of the skin image analysis tool.
2.2. Back-End Back-End MATLAB MATLAB Files The back-end of the program is implemented in a number of MATLAB functions saved as standard M-files. M-files. MATLAB MATLAB functions, functions, also also called called modules modules or or subroutines in other programming languages, accept input arguments and return output arguments. They operate on variables within their ownown workspace,workspace, di differentfferent from from the the main main program. program. Table Table1 shows 1 shows the structure the structure of the skin of analysisthe skin analysissoftware software tool. tool. Table 1. Structure of the skin analysis tool. Table 1. Structure of the skin analysis tool. Files Comments Files Comments PX_DL_GUI.mlappPX_DL_GUI.mlapp Main Main GUIGUI filefile AlexnetTraining.mAlexnetTraining.m AlexNetAlexNet trainingtraining functionfunction AlexnetClassify.m AlexNet classification function AlexnetClassify.m AlexNet classification function GoogLeNetTraining.m GoogLeNet training function GoogLeNetTraining.m GoogLeNet training function GoogLeNetClassify.m GoogLeNet classification function GoogLeNetClassify.mGabor_Calculate.m GaborGoogLeNet wavelet classification transform calculationfunction function Gabor_Calculate.mGabor_Search.m Gabor Gabor waveletwavelet transformtransform searchcalculation function function Gabor_Search.m...... Gabor wavelet transform search function … … … …
Cosmetics 2020, 7, 67 3 of 12
2.3. FunctionalitiesCosmetics 2020, 7, x FOR PEER REVIEW 3 of 13
The2.3. Functionalities main purpose of this work is to develop a general purpose skin image analysis software with many useful skin image analysis functions. It is also designed to be flexible, so more new functions can The main purpose of this work is to develop a general purpose skin image analysis software be added in the future. with many useful skin image analysis functions. It is also designed to be flexible, so more new Thefunctions current can implementedbe added in the functions future. are image classifications [8,9,13], skin texture analysis and skin imageThe retrieval current implemented by using Gabor functions wavelet are image transform, classifications PCA (principle [8,9,13], skin components texture analysis analysis) and and GLCMskin (gray-level image retrieval co-occurrence by using Gabor matrix) wavelet [10,11 ],tran assform, well asPCA skin (principle live image components analysis. analysis) and GLCM (gray-level co-occurrence matrix) [10,11], as well as skin live image analysis. 3. Results and Discussions 3. Results and Discussions 3.1. Skin Image Classification 3.1. Skin Image Classification Skin image classification is the most important function of the skin analysis tool. With skin image classification,Skin weimage would classification be able tois the diff erentiatemost important normal function skin from of the damaged skin analysis skin, tool. healthy With skin skin from diseasedimage skin. classification, It can also we would potentially be able be to useddifferentiate for identifying normal skin di fromfferent damaged types ofskin, skin healthy diseases skin and skin cancers.from diseased Skin skin. image It can classification also potentially is achievedbe used for through identifying transfer different learning types of byskin using diseases pre-trained and skin cancers. Skin image classification is achieved through transfer learning by using pre-trained deep-learning neural networks such as AlexNet [14], GoogLeNet [15], VGG19 [16] and ResNet101 [17]. deep-learning neural networks such as AlexNet [14], GoogLeNet [15], VGG19 [16] and ResNet101 The performance[17]. The performance of image of image classification classification is tested is tested on on four four di differentfferent types types of of images, images, as shown as shown in in FiguresFigures2–5. 2–5. All theAll the images images are are randomly randomly divided divided intointo training training set set (70%) (70%) and and test test set set(30%) (30%) and and automaticallyautomatically resized resized to 224to 224 224× 224 ×3 3 (except (except for AlexNet, AlexNet, 227 227 × 227227 × 3) before3) before training. training. × × × × The grayscaleThe grayscale images images also also automatically automatically convertedconverted to to color color images images prior prior to training. to training. During During the the training,training, it uses it uses stochastic stochastic gradient gradient descent descent with with momentum, momentum, the the initial initial learning learning rate rate is is 10ˆ-4, 10^-4, mini mini batch size isbatch 5, max size epochs is 5, max is 6,epochs and itis also6, and shu it fflalsoes shuf trainingfles training between between every epoch.every epoch. The trainingsThe trainings are doneare on ® a standarddone on desktop a standard computer desktop with computer Intel® withCore Intel™ i7-3770 Core™ CPU i7-3770 @3.4 CPU GHz, @3.4 8 GHz, cores, 8 16cores, GB 16 RAM GB and WindowsRAM 8.1and operating Windows system.8.1 operating system.
palm neck leg
face forearm palm
FigureFigure 2. Sample 2. Sample skin skin capacitive capacitive images images from from di ffdifferenterent skin skin sites sites of of a a healthy healthy volunteer. volunteer. ThereThere areare total 90 imagestotal 90 divided images intodivided six categories,into six categories, palm, palm, neck, neck, leg, face, leg, face, forearm forearm and and palm. palm.
FigureFigure2 shows 2 shows the the skin skin capacitive capacitive images images acquired acquired by using our our Epsilon Epsilon permittivity permittivity imaging imaging systemsystem [8,9]. [8,9]. There There are are total total 90 90 images images divided divided intointo six categories, categories, palm, palm, neck, neck, leg, leg, face, face, forearm forearm and and palm. The image classification results are shown in the first row of Table 2. AlexNet has the shortest palm. The image classification results are shown in the first row of Table2. AlexNet has the shortest training time and VGG19 has the longest training time. The classification overall accuracy is good, except VGG19 and GoogLeNet gives an impressive 100% classification accuracy. Cosmetics 2020, 7, x FOR PEER REVIEW 4 of 13
Cosmeticstraining2020 time, 7, 67and VGG19 has the longest training time. The classification overall accuracy is good,4 of 12 except VGG19 and GoogLeNet gives an impressive 100% classification accuracy.
Benign
Malignant
CosmeticsFigure 2020 3., 7,Sample x FOR PEER skin REVIEW cancer images from ISIC database [18]. There are total 600 images divided into5 of 13 twoFigure categories, 3. Sample Benign skin cancer and Malignant. images from ISIC database [18]. There are total 600 images divided into two categories, Benign and Malignant.
Figure 3 shows the skin cancer images from ISIC database [18]. These are standard digital photo images. There are totally 600 images divided into two categories: Benign and Malignant. The image classification results are shown in the second row of Table 2. AlexNet again has the shortest training time, and VGG19 has the longest. GoogLeNet and ResNet101 have the best classification accuracy. Figure 4 shows the skin ultrasound images acquired by using EPISCAN I-200 ultrasound instrument. There are totally 610 images measured from six different skin sites, chin, cheeks, nose, under eye, lips and forearm. The image classification results are shown in the third row of Table 2. AlexNet again has the shortest training time and VGG19 the longest. ResNet101 has the best classification accuracy. The accuracy is slightly lower than lower than previous two types of skin images, this is understandable, as skin ultrasound images have higher similarities.
Chin cheeks nose
under eye lips forearm
Figure 4.4. Sample skin ultrasound images from didifferentfferent skin sites of aa healthyhealthy volunteer.volunteer. There are total 610 imagesimages divideddivided intointo sixsix categories,categories, chin,chin, cheeks,cheeks, nose,nose, underunder eye,eye, lipslips andand forearm.forearm.
Cosmetics 2020, 7, 67 5 of 12
Figure3 shows the skin cancer images from ISIC database [ 18]. These are standard digital photo images. There are totally 600 images divided into two categories: Benign and Malignant. The image classification results are shown in the second row of Table2. AlexNet again has the shortest training time, and VGG19 has the longest. GoogLeNet and ResNet101 have the best classification accuracy. Figure4 shows the skin ultrasound images acquired by using EPISCAN I-200 ultrasound instrument. There are totally 610 images measured from six different skin sites, chin, cheeks, nose, under eye, lips and forearm. The image classification results are shown in the third row of Table2. AlexNet again has the shortest training time and VGG19 the longest. ResNet101 has the best classification accuracy. The accuracy is slightly lower than lower than previous two types of skin images,Cosmetics this is 2020 understandable,, 7, x FOR PEER REVIEW as skin ultrasound images have higher similarities. 6 of 13
COVID-19
Normal
Figure 5.FigureChest 5. Chest X-ray X-ray images images from from University University ofof MontrealMontreal [19] [19 and] and Kaggle Kaggle [20]. [There20]. Thereare total are 50 total 50 images dividedimages divided into two into categories,two categories, COVID-19 COVID-19 and and normal.
Table 2. Image classification results. Currently, the coronavirus COVID-19 has become a global pandemic and is affecting more than Table Column Head 21 million people in moreImage than Types 200 countries and territories around the world. We wanted to see if our software tool can effectively workModels on COVID-19 Training chest Time X-ray Accuracy images, as shown in Figure5. AlexNet 1 min 6 s 83.33% The COVID-19 chest X-ray images are from a research group in University of Montreal [19] and Capacitive GoogLeNet 1 min 39 s 100.00% the normal chest X-rayskin images images are fromVGG19 Kaggle [20]. 12 There min 5 ares totally 58.33% 50 images divided into two categories: COVID-19 and normal. TheResNet101 image classification 9 min 12 s results87.50% are shown in the third row of Table2. AlexNet, not surprisingly, has theAlexNet shortest training 8 min time37 s and VGG19 73.89% the longest. The overall classification accuracy isSkin high cancer (>85%) andGoogLeNet ResNet101 has13 min an 23 impressive s 77.78% 100% classification accuracy. In summary, AlexNetimages is consistently VGG19 the fastest 91 in min image 15 s classification 75.00% training, while VGG19 is the slowest. The ResNet101 has theResNet101 highest overall 66 min classification 40 s 77.78% accuracy and VGG19 has the AlexNet 4 min 56 s 69.44% lowest overall classificationSkin accuracy. For skin capacitive image classification, GoogLeNet is the best. GoogLeNet 10 min 51 s 56.67% ultrasound For other image classification, e.g., skin cancerVGG19 images, 71skin min ultrasound 59 s images60.56% and Chest X-ray images, images ResNet101 is the best. ResNet101 63 min 3 s 70.56% AlexNet 43 s 92.86% Chest X-ray GoogLeNet 1 min 30 s 92.85% images VGG19 6 min 44 s 85.71% ResNet101 4 min 42 s 100.00%
Currently, the coronavirus COVID-19 has become a global pandemic and is affecting more than 21 million people in more than 200 countries and territories around the world. We wanted to see if our software tool can effectively work on COVID-19 chest X-ray images, as shown in Figure 5. The
Cosmetics 2020, 7, 67 6 of 12
Table 2. Image classification results.
Table Column Head Image Types Models Training Time Accuracy AlexNet 1 min 6 s 83.33% GoogLeNet 1 min 39 s 100.00% Capacitive skin images VGG19 12 min 5 s 58.33% ResNet101 9 min 12 s 87.50% AlexNet 8 min 37 s 73.89% GoogLeNet 13 min 23 s 77.78% Skin cancer images VGG19 91 min 15 s 75.00% ResNet101 66 min 40 s 77.78% AlexNet 4 min 56 s 69.44% GoogLeNet 10 min 51 s 56.67% Skin ultrasound images VGG19 71 min 59 s 60.56% ResNet101 63 min 3 s 70.56% AlexNet 43 s 92.86% GoogLeNet 1 min 30 s 92.85% Chest X-ray images VGG19 6 min 44 s 85.71% ResNet101 4 min 42 s 100.00%
The most significant benefit of skin image classification is that users would be able to use the skin image analysis tool with a webcam camera to take a picture of a skin lesion, the software will be able to tell whether it is benign or malignant. Users can also train the neural networks on their own skin image data, to recognize different types of skin diseases/cancers. This could be a potential low cost skin diseases/cancer diagnosis device. The classification accuracy depends on the number and the quality of skin images, as well as the deep-learning neural networks. To further improve the accuracy, we will need train the neural networks on larger number of skin images and better quality of skin images. Currently we only have tested four different deep-learning neural networks, in the future, we will implement more neural networks. In addition, all the neural networks are trained on a set of default optimized values, next step is to allow users to specify their own training parameter values.
3.2. Skin Texture Analysis and Image Retrieval Skin surface is not even, it is rich with patterns and lines. Our previous studies show that we can analyze the skin texture by using Gabor wavelet transform, PCA (principle components analysis) and GLCM (gray-level co-occurrence matrix). With Gabor wavelet transform, we can get skin features vectors on color, shape and texture [10]. With GLCM (gray-level co-occurrence matrix), we can get four GLCM feature vectors, e.g., angular second moment (ASM), entropy (ENT), contrast (CON) and correlation (COR). The results showed that the angular second moment increases as age increases and entropy decreases as age increases [11]. We can also analyze the skin micro-relief, the results showed that the change in the intensity of primary and secondary lines during arm extension is increasing against chronological age, while the average number of closed polygons per square mm is decreasing against chronological age [12]. Another useful usage of skin texture analysis is image retrieval or image searching. This is particularly suitable for skin images, where similarities are much higher than normal images. Figure6a shows the skin cancer image searching results using Gabor wavelet transform. As we use the same image database for training and searching, the best matching result is the query image itself. The second to sixth best matching results are also from the same category of the query image, i.e., benign images. Figure6b–d show the image searching results using Gabor wavelet transform on skin capacitive images, skin ultrasound images and chest X-ray images. Similar to Figure6a, all the best matching results are from the same category of the query image. Cosmetics 2020, 7, 67 7 of 12
Table3 shows the image searching results of di fferent algorithms on different types of images. Calculation time is how long each algorithm takes to calculate the feature vectors of all the images and built a database of the results. Searching time is how long each algorithm takes to search the database and find the six best matching results. The error rate is calculated as how many images in the best six matching results are in wrong category. Figure6 shows the best scenario of Gabor wavelet transform searching results, the error rates are 0 out of 6, while Table3 shows the average error rates. The overall results show that Gabor wavelet transform general takes longer time to calculate and to search, while PCA is the shortest for calculation and GLCM is the shortest for searching. Although Gabor wavelet transform is slower compared with other image retrieval techniques, such as PCA and GLCM, Gabor wavelet transform is the best for retrieving images. This agrees well our previous studies on skin capacitive contact images and digital facial images [10]. All the algorithms are running on a set Cosmetics 2020, 7, x FOR PEER REVIEW 8 of 13 of optimized default values, in the future, we will allow users to specify their own parameter values.
(a)
(b)
Figure 6. Cont.
Cosmetics 2020, 7, 67x FOR PEER REVIEW 98 of 13 12
(c)
(d)
Figure 6. Image searching results using Gabor wavelet transform on different type of images. (a) skin Figurecapacitive 6. Image images; searching (b) skin results cancer using images; Gabor (c) skin wavelet ultrasound transform images; on different (d) chest type X-ray of images. (a) skin capacitive images; (b) skin cancer images; (c) skin ultrasound images; (d) chest X-ray images. Table 3. Image searching results.
Table 3 shows the image searching resultsTable of diff Columnerent algorithms Head on different types of images. Calculation Imagetime is Types how long each algorithm takes to calculate the feature vectors of all the images Algorithms Calculation Time Searching Time Error Rate and built a database of the results. Searching time is how long each algorithm takes to search the database and find the six best matchingGabor results. The 153 error s rate is calculated 1059 ms as how many 1/6 images in Capacitive skin images PCA 1.6 s 221 ms 3/6 the best six matching results are in wrong category. Figure 6 shows the best scenario of Gabor GLCM 1.1 s 25 ms 2/6 wavelet transform searching results, the error rates are 0 out of 6, while Table 3 shows the average Gabor 370 s 2834 ms 0/6 error rates. The overall results show that Gabor wavelet transform general takes longer time to Skin cancer images PCA 54 s 1560 ms 2/6 calculate and to search, while PCAGLCM is the shortest 482 for s calculation and 698 msGLCM is the 2/shortest6 for searching. Although Gabor wavelet transform is slower compared with other image retrieval techniques, such as PCA and GLCM, Gabor wavelet transform is the best for retrieving images. This agrees well our previous studies on skin capacitive contact images and digital facial images [10]. All
Cosmetics 2020, 7, x FOR PEER REVIEW 10 of 13 the algorithms are running on a set of optimized default values, in the future, we will allow users to specify their own parameter values. Cosmetics 2020, 7, 67 9 of 12 Table 3. Image searching results.
Table TableColumn 3. Cont. Head Image Types Calculation Algorithms Searching Time Error Rate TimeTable Column Head Image Types Gabor Algorithms153 Calculation s Time 1059 Searching ms Time Error 1/6 Rate Capacitive PCA 1.6 s 221 ms 3/6 skin images Gabor 324 s 2743 ms 2/6 Skin ultrasound imagesGLCM PCA 1.1 s 29 s 25 ms 2409 ms 2/6 3/6 Gabor GLCM370 s 118 s 2834 ms 220 ms 0/6 1/6 Skin cancer PCA Gabor 54 s 35 s 1560 ms 813 ms 2/6 0/6 images Chest X-ray imagesGLCM PCA482 s 2.6 s 698 ms 162 ms 2/6 2/6 GLCM 30 s 563 ms 0/6 Skin Gabor 324 s 2743 ms 2/6 ultrasound PCA 29 s 2409 ms 3/6 3.3. Skinimages Live Image AnalysisGLCM 118 s 220 ms 1/6 Gabor 35 s 813 ms 0/6 ChestApart X-ray from analyze image files, it can also analyze skin live images captured from webcams PCA 2.6 s 162 ms 2/6 and otherimages USB based video devices. The live image capture is achieved by using the latest MATLAB Add-On module called “MATLABGLCM Support Package 30 s for USB Webcams”. 563 ms Figure7 shows the 0/6 live image analysis of facial skin using the ProScope HR with 50 times magnification [21]. The facial features such 3.3.as skin Skin texture, Live Image speckles Analysis and facial hair are clear to see. The software can divide the live image into threeApart color channels,from analyze red, greenimage and files, blue it (RGB)can also and an thereforealyze skin perform live images the RGB captured calculation from in webcams real time. andFigure other7a showsUSB based the original video devices. image andThe processedlive image image capture (green is achieved channel by only), using Figure the latest7b shows MATLAB the Add-Onoriginal imagemodule and called processed “MATLAB image (redSupport channel Package only). for As weUSB can Webcams”. see, green channelFigure 7 has shows more the surface live imageinformation, analysis while of facial red channel skin using has more the ProScope information HR on with melanin 50 times (facial magnification hair) and skin [21]. color. The facial featuresFigure such7c as shows skin thetexture, original speckles image and and facial the processed hair are clear image, to see. calculated The software by adding can divide green channel the live 0.7 with blue channel 0.7, without red channel. As we can see, this enhances the surface texture image× into three color ×channels, red, green and blue (RGB) and therefore perform the RGB calculationinformation in of thereal skin. time. Users Figure can 7a specify shows they the own original RGB channelimage and calculations. processed Facial image skin (green has enormous channel only),cosmetic Figure values, 7b thisshows functionality the original could image be veryand processed useful for analyzingimage (red skin channel facial features,only). As such we can as facial see, greenhairs, smoothness,channel has speckles,more surface scars, information, wrinkles, all inwhile real red time! channel More functions has more will information be added. on The melanin current (facialversion hair) of the and program skin color. is available as a MATLAB App Installer format on GitHub [22].
(a)
Figure 7. Cont. Cosmetics 20202020,, 77,, 67x FOR PEER REVIEW 1011 of 1213
(b)
(c)
Figure 7. Live image analysis usingusing the ProScope HR with 50 timestimes magnification.magnification. (a) originaloriginal imageimage and processedprocessed imageimage (green (green channel channel only); only); (b )( originalb) original image image and and processed processed image image (red channel(red channel only); (only);c) original (c) original image image and the and processed the proc imageessed (noimage red (no channel, red channel, green green0.7 + ×blue 0.7 + blue0.7). × 0.7). × × 4. ConclusionsFigure 7c shows the original image and the processed image, calculated by adding green channelWe have× 0.7 developedwith blue channel a skin image × 0.7, analysiswithout toolred basedchannel. on As machine-learning we can see, this algorithms. enhances the It uses surface the latesttexture MATLAB information App of Designer the skin. to Users create can the specify graphic they user own interface RGB channel on the frontcalculations. end and Facial use MATLAB skin has M-filesenormous as thecosmetic backend. values, The this current functionality version could provides be very functions useful such for analyzing as skin image skin facial classifications, features, skinsuch imageas facial retrieval hairs, smoothness, and skin live speckles, image analysis. scars, wr Theinkles, skin all image in real classification time! More isfunctions done by will using be deep-learningadded. The current neural networksversion of such the asprogram AlexNet, is GoogLeNet, available as VGG19 a MATLAB and ResNet101. App Installer The classification format on GitHub [22].
Cosmetics 2020, 7, 67 11 of 12 results show that GoogLeNet is the best for skin capacitive image classifications, while ResNet101 is the best for skin cancer image, skin ultrasound image and chest X-ray image classifications. The biggest advantage of this skin image analysis tool is that users would be able to retrain the deep-learning neural networks on their own skin image data and therefore perform their own skin image classifications, such as recognizing different types of skin diseases. This could lead to a potential low cost skin disease/cancer diagnose device. The skin image retrieval is done by using Gabor wavelet transform, PCA and GLCM. The results show that Gabor wavelet transform has the overall best accuracy despite of being the slowest. In skin live image analysis, by manipulating the RGB values of the images, it has potentials for analyzing skin facial features, such as facial hairs, smoothness, speckles, scars, wrinkles in real time. More functionality will be added on in the future. The software tool can work on a variety of different types of skin images, such as skin digital images, skin capacitive images, skin ultrasound images and more. Our study shows the apart from skin images, it can also work on other types of images, such as chest X-ray images. We have made the program available on GitHub so that others can benefit from it. For the future work, we would like to test the skin image analysis tool on larger skin image dataset, on more volunteers, of different ages, genders, races and colors, as well as different skin conditions (dry, normal, silky, rough, etc.) and different skin diseases/cancers. For image classifications, more deep-learning neural networks will be implemented, and users would be able to select different training parameters. For skin image retrieval, users would be able to specify different parameter values for Gabor wavelet transform, PCA and GLCM. For live image analysis, more functions, such as image segmentation, object detections and so on, will be implemented.
Author Contributions: Conceptualization, P.X. and D.C.; methodology, P.X., X.O., X.Z., W.P., C.B. and E.C.; formal analysis, P.X., X.O., X.Z., W.P. and C.B.; investigation, X.O., X.Z., W.P. and E.C.; data curation, P.X. and C.B.; original draft preparation, P.X.; review and editing of manuscript, P.X. and D.C.; supervision, P.X. and D.C. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Acknowledgments: We thank the London South Bank University and Biox Systems, Ltd., UK for the financial support. We also thank Longport, Inc., USA for loaning the EPISCAN I-200 instrument. Conflicts of Interest: The authors declare no conflict of interest.
References
1. Argenziano, G.; Soyer, H.P. Dermoscopy of pigmented skin lesions–a valuable tool for early diagnosis of melanoma. Lancet Oncol. 2011, 2, 443–449. [CrossRef] 2. Vestergaard, M.E.; Macaskill, P.; Holt, P.E.; Menzies, S.W. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: A meta-analysis of studies performed in a clinical setting. Br. J. Dermatol. 2008, 159, 669–676. [CrossRef][PubMed] 3. Dermlite. Available online: https://dermlite.com/ (accessed on 18 March 2020). 4. DermLite II Multi-Spectral. Available online: https://www.eufoton.com/en/products/accessories/dermlite- multispectral-ii (accessed on 18 March 2020). 5. Nerasolutions. Available online: https://www.nerasolutions.co.uk/skin-analysis-devices (accessed on 18 March 2020). 6. Visia Skin Analysis System. Available online: https://www.canfieldsci.com/imaging-systems/visia- complexion-analysis/ (accessed on 18 March 2020). 7. EPISCAN I-200. Available online: https://www.longportinc.com/about-longport (accessed on 18 March 2020). 8. Zhang, X.; Pan, W.; Bontozoglou, C.; Chirikhina, E.; Chen, D.; Xiao, P. Skin Capacitive Imaging Analysis Using Deep Learning GoogLeNet. In Intelligent Computing. SAI 2020. Advances in Intelligent Systems and Computing; Arai, K., Kapoor, S., Bhatia, R., Eds.; Springer: Cham, Switzerland, 2020; Volume 1229. [CrossRef] 9. Zhang, X.; Pan, W.; Xiao, P. In-vivo Skin Capacitive Image Classification Using AlexNet Convolution Neural Network. In Proceedings of the 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), Chongqing, China, 27–29 June 2018; pp. 439–443. [CrossRef] Cosmetics 2020, 7, 67 12 of 12
10. Ou, X.; Pan, W.; Zhang, X.; Xiao, P. Skin image retrieval using Gabor wavelet texture feature. Int. J. Cosmet. Sci. 2016, 38, 607–614. [CrossRef][PubMed] 11. Ou, X.; Pan, W.; Xiao, P. In-Vivo Skin Capacitive imaging Analysis by using Grey Level Co-occurrence Matrix (GLCM). Int. J. Pharm. 2014, 460, 28–32. [CrossRef][PubMed] 12. Bontozoglou, C.; Zhang, X.; Xiao, P. Micro-relief analysis with skin capacitive imaging. Skin Res. Technol. 2019, 25, 165–170. [CrossRef][PubMed] 13. Bianconi, F.; Chirikhina, E.; Smeraldi, F.; Bontozoglou, C.; Xiao, P. Personal identification based on skin texture features from the forearm and multi-modal imaging. Skin Res. Technol. 2017, 23, 392–398. [CrossRef] [PubMed] 14. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. In Proceedings of the International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–8 December 2012; pp. 1097–1105. 15. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going Deeper With Convolutions. In Proceedings of the The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. 16. Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. ICLR 2015, arXiv:1409.1556v6. 17. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. arXiv 2015, arXiv:1512.03385. 18. ISIC—The International Skin Imaging Collaboration (Skin Disease Database). Available online: https: //isic-archive.com/#images (accessed on 20 March 2020). 19. University of Montreal’s X-ray Images of COVID-19 Cases. Available online: https://github.com/ieee8023/ covid-chestxray-dataset (accessed on 20 March 2020). 20. Kaggle Chest X-Ray Images (Pneumonia). Available online: https://www.kaggle.com/paultimothymooney/ chest-xray-pneumonia (accessed on 20 March 2020). 21. ProScope, H.R. The ProScope Mobile & HR USB Handheld Digital Microscope. Available online: https: //www.bodelin.com/ (accessed on 20 July 2020). 22. Skin Image Analysis Tool GitHub Site. Available online: https://github.com/PerryXiao2015/LSBU-Skin- Image-Analysis-Tool (accessed on 21 August 2020).
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).