Traditional Chinese Medicine Pulse Diagnosis on a Smartphone Using Skin Impedance at Acupoints: a Feasibility Study

sensors

Article Traditional Chinese Medicine Pulse Diagnosis on a Smartphone Using Skin Impedance at Acupoints: A Feasibility Study

Kun-Chan Lan 1,2,*, Gerhard Litscher 2,3,* and Te-Hsuan Hung 1 1 Department of CSIE, National Cheng Kung University, Tainan 701, Taiwan; [email protected] 2 School of Chinese Medicine, China Medical University, Taichung 404, Taiwan 3 Research Unit of Biomedical Engineering in Anesthesia and Intensive Care Medicine, Research Unit for Complementary and Integrative Laser Medicine, and Traditional Chinese Medicine (TCM) Research Center Graz, Medical University of Graz, 8036 Graz, Austria * Correspondence: [email protected] (K.-C.L.); [email protected] (G.L.)

Received: 30 June 2020; Accepted: 6 August 2020; Published: 17 August 2020

Abstract: In traditional Chinese medicine (TCM), pulse diagnosis is one of the most important methods for diagnosis. A pulse can be felt by applying firm fingertip pressure to the skin where the arteries travel. The pulse diagnosis has become an important tool not only for TCM practitioners but also for several areas of Western medicine. Many pulse measuring devices have been proposed to obtain objective pulse conditions. In the past, pulse diagnosis instruments were single-point sensing methods, which missed a lot of information. Later, multi-point sensing instruments were developed that resolved this issue but were much higher in cost and lacked mobility. In this article, based on the concept of sensor fusion, we describe a portable low-cost system for TCM pulse-type estimation using a smartphone connected to two sensors, including one photoplethysmography (PPG) sensor and one galvanic skin response (GSR) sensor. As a proof of concept, we collected five-minute PPG pulse information and skin impedance on 24 acupoints from 80 subjects. Based on these collected data, we implemented a fully connected neural network (FCN), which was able to provide high prediction accuracy (>90%) for patients with a TCM wiry pulse.

Keywords: pulse diagnosis instrument; photoplethysmography; galvanic skin response; resonance theory; smartphone

1. Introduction For several thousands of years, the pulse diagnosis has been one of the important diagnostic methods used in traditional Chinese medicine (TCM). A pulse can be felt by applying firm fingertip pressure to the skin at sites where the arteries travel near the surface of the skin. The pulse diagnosis has become a necessary procedure in clinical diagnosis not only for Chinese medicine practitioners but also in several areas of Western medicine. In the past, Chinese medicine practitioners relied on their accumulated experience and intuition, where the sensitivity of different physicians varied. It was difficult to transfer personal experiences to other practitioners. Currently, through a combination of medical expertise and digital signal processing technologies, TCM pulse-measuring instruments not only greatly improve reproducibility, but also instantly visualize and analyze the measured pulse profile. There are about 29 pulse types in a TCM pulse diagnosis, including the floating pulse, scattered pulse, hollow pulse, deep pulse, hidden pulse, firm pulse, slow pulse, moderate pulse, swift pulse, surging pulse, thready pulse, long pulse, short pulse, feeble pulse, weak pulse, faint pulse, replete pulse, slippery pulse, stirred pulse, unsmooth pulse, wiry pulse, tight pulse, tympanic pulse, soggy pulse, irregularly intermittent pulse, irregular pulse, irregular-rapid pulse, and intermittent pulse [1,2].

Sensors 2020, 20, 4618; doi:10.3390/s20164618 www.mdpi.com/journal/sensors Sensors 2020, 20, 4618 2 of 14

The key point of pulse-measuring instruments is to obtain important information from the pulse in order to distinguish between different pulse types. A comprehensive analysis of the pulse types can lead to an understanding of a disease and provide a basis for the dispensing of prescriptions [3]. While many pulse-measuring devices have been developed based on different sensing techniques, such as piezoresistive, photoelectric, and piezoelectric methods [4–7], they are mostly single-probe sensors that convert the spatial variations of pulses into a simple output of electrical signals. These single-point sensing devices can only obtain limited information because they cannot reflect the detailed changes in the pulse. Physiologically speaking, the TCM pulse type is “felt” by many different receptors on the fingertip of a TCM practitioner. Each of these receptors can be considered to be a single sensor. Intuitively, data obtained from only a single pulse sensor will not have enough information to correctly model the “pulse feeling” based on data from multiple neural receptors on the fingertip. Chung et al. [8] recently developed a multi-point sensing instrument based on a 3 4 sensor × array. While their device is able to identify the important characteristics of the wrist radial artery, including strength, rate, length, width, and trends in pulse conditions, using three-dimensional pulse mapping (3DPM) [8] of the pulse and simulating the sensation of actual touch, it is too costly and requires running its sophisticated signal processing algorithms on a PC. To address the shortcomings of these prior studies [5–8], in this work, we propose a smartphone-based sensor system for TCM pulse diagnosis built on the concept of “sensor fusion.” The idea of sensor fusion is to combine sensory data from multiple sources such that the resulting information is more complete than would be possible using these sources individually. To be more specific, we consider additional information from recording the conductivity of acupuncture points to complement the use of a single-probe pulse sensor when inferring the TCM pulse type. Acupuncture points have been proven to have distinct electrical properties. These properties include increased conductance, reduced impedance and resistance, increased capacitance, and elevated electrical potential compared to adjacent non-acupuncture points [9]. Based on these properties, skin impedance at acupuncture points (APs) has been used as a diagnostic aid for various diseases for more than 50 years [9]. In this work, three reasons motivated the use of information about AP skin impedance to assist the inference of TCM pulse type. First of all, both TCM pulse and AP impedance are basically biological features that reflect the conditions of the body. While these two biomarkers are different, hypothetically, their data distributions could be correlated when they are measured at the same time for a given physical state. Theoretically, if two data distributions, say p and A, are correlated, augmented data from A (e.g., acupoint conductance) may be helpful as a way to implicitly learn about p (e.g., pulse characteristics) [10]. Second, skin impedance information related to APs can be non-intrusively and easily collected through a low-cost galvanic skin response (GSR) sensor. Finally, from the perspective of sources of sensing, there are many acupoints on the body, which in aggregation can provide rich information about the condition of the human body (considering the amount of data collection time, in this work, we used only 24 acupoints). Our contribution in this work is threefold. First, we designed and implemented a low-cost smartphone-based sensor system consisting of one photoplethysmography (PPG) sensor and one galvanic skin response (GSR) sensor to infer TCM pulse type based on the concept of sensor fusion. Second, as a proof of concept, we collected single-point PPG pulse information and GSR data for 24 acupoints from 80 subjects with a “wiry pulse,” a commonly seen TCM pulse type usually found in patients with liver diseases. Eighteen features were computed from PPG signals and twenty-four features were extracted from GSR data, respectively. We collected five-minute PPG measurements from each subject. A sliding-window (with a window size of 10 s) approach was implemented to segment PPG data for computing the eighteen features. The GSR data for each acupoint were sampled for five seconds and the median of the sampled GSR was recorded. These data were used to verify our hypothesis suggesting a correlation between TCM pulse and acupoint impedance. Finally, based on these collected data, we implemented a binary classifier based on a fully connected neural network (FCN), which was able to accurately predict the wiry pulse. To the best of our knowledge, this is the first Sensors 2020, 20, x FOR PEER REVIEW 3 of 14 portableSensors 2020 sensor, 20, 4618 system that can correctly infer the TCM pulse type. As compared to prior work3 of by 14 Chung et al. [6], our system is more affordable for use by TCM practitioners. The remainder of this paper is organized as follows: We present our methods in Section 2 and low-cost (without considering the smartphone, the cost of our system is under $200) portable sensor evaluate the proposed system in Section 3. Finally, we conclude this paper and provide suggestions system that can correctly infer the TCM pulse type. As compared to prior work by Chung et al. [6], for future work in Section 4. our system is more affordable for use by TCM practitioners. 2. MethodThe remainder of this paper is organized as follows: We present our methods in Section2 and evaluate the proposed system in Section3. Finally, we conclude this paper and provide suggestions for futureThe work hardware in Section architecture4. of the proposed system is shown in Figure 1. It is divided into three parts, including one PPG sensor and one GSR sensor, a smartphone, and a cloud server. PPG is a signal2. Method that is recorded by using a skin area to absorb light energy according to the light sensing element. Since the hemoglobin in the blood absorbs light, the amplitude of the PPG signal changes The hardware architecture of the proposed system is shown in Figure1. It is divided into three proportionally with the blood flow in and out of tissue. In this study, we used the GSR sensor to parts, including one PPG sensor and one GSR sensor, a smartphone, and a cloud server. PPG is a signal record the skin impedance of acupoints. In TCM, it is generally believed that acupoints are the that is recorded by using a skin area to absorb light energy according to the light sensing element. Since reaction points that reflect health conditions and pathological changes in the human body. Yoshio the hemoglobin in the blood absorbs light, the amplitude of the PPG signal changes proportionally Nakatani [9] found that patients with various visceral diseases had specific skin resistance response with the blood flow in and out of tissue. In this study, we used the GSR sensor to record the skin patterns at some particular reaction points called Ryodoraku points. Incidentally, these Ryodoraku impedance of acupoints. In TCM, it is generally believed that acupoints are the reaction points that points appear to be at the same location as 24 known TCM acupoints (as shown later in Table 1). reflect health conditions and pathological changes in the human body. Yoshio Nakatani [9] found that The sensor platform first collects GSR and PPG signals from the subject. These sensor data are patients with various visceral diseases had specific skin resistance response patterns at some particular sent through the OTG (On-The-Go) cable to the smartphone and then transmitted wirelessly to the reaction points called Ryodoraku points. Incidentally, these Ryodoraku points appear to be at the same cloud server to predict the TCM pulse type using an artificial neural network. location as 24 known TCM acupoints (as shown later in Table1).

Figure 1. System architecture. Figure 1. System architecture. The sensor platform first collects GSR and PPG signals from the subject. These sensor data are 2.1.sent Sensors through the OTG (On-The-Go) cable to the smartphone and then transmitted wirelessly to the cloudFigure server 2 toshows predict the the wiring TCM diagram pulse type of usingthe GSR an artificial(galvanic neural skin response) network. sensing unit, which is comprised of an Arduino beetle board, a GSR module, and a probe. A mobile phone is used to 2.1. Sensors transmit the GSR data to the cloud as well as to provide 5 V power to the Arduino beetle through a USB FigureOTG. 2The shows GSR themodule wiring (Seeed diagram Studio, of the Shenzhen, GSR (galvanic China) skinconsists response) of a bridge sensing circuit unit, and which an amplifier.is comprised The of bridge an Arduino circuit is beetle used board,to measure a GSR resistance. module, andIts working a probe. principle A mobile is phonesimilar isto used that of to atransmit potentiometer. the GSR After data the to themeasured cloud as resistance well as to is provideconverted 5 V to power a voltage to the through Arduino the beetlebridge through circuit, thea USB amplifier OTG. Theamplifies GSR modulethe signals (Seeed to the Studio, desired Shenzhen, magnification. China) The consists amplified of a signals bridge are circuit read and by the an ADCamplifier. (analogue The bridge digital circuit converter) is used on to the measure Arduino resistance. beetle and Its then working relayed principle to the isphone similar over to thatUART of (universala potentiometer. asynchronous After the receiver measured transmitter). resistance Given is converted that most to phones a voltage usually through do not the have bridge externally circuit, availablethe amplifier UARTs, amplifies a USB-serial the signals converter to the desiredis employ magnification.ed for the communication The amplified between signals arethe readphone by and the theADC Arduino (analogue beetle. digital converter) on the Arduino beetle and then relayed to the phone over UART (universal asynchronous receiver transmitter). Given that most phones usually do not have externally available UARTs, a USB-serial converter is employed for the communication between the phone and the Arduino beetle.

Sensors 2020, 20, x FOR PEER REVIEW 4 of 14 Sensors 2020, 20, 4618 4 of 14 Sensors 2020, 20, x FOR PEER REVIEW 4 of 14

Figure 2. Wiring diagram of the GSR (galvanic skin response) sensor. Figure 2. WiringWiring diagram of the GSR ( (galvanicgalvanic skin response) sensor. The wiring diagram of the PPG (photoplethysmography) sensor module is similar to the design of theThe GSR wiring sensor. diagram The PPG of thepulse PPG sensor (photoplethysmography)(photoplethysmogra is connected to anphy) Arduino sensor beetle module board, is similar and we to theutilize design the ofsmartphone the GSR sensor. sensor. to read The ThePPG PPG PPG data pulse pulse from sensor sensorthe beetle is isconnected connected board as to welltoan anArduino as Arduino to power beetle beetle the board, board. board, and In and we this weutilize work, utilize the 5- theminutesmartphone smartphone PPG tomeasurements read to read PPG PPG data were data from collected from the thebeetle beetlefrom board the board wristas well as of well asthe to as user. power to power The the GSR the board. board.data In were this In this collectedwork, work, 5- 5-minutefromminute 24 PPGacupoints PPG measurements measurements based on werethe were Ryodoraku collected collected from frompoints the the proposed wrist wrist of of the theby user.Dr. user. Yoshio The The GSR GSR Nakatani data were [9], and collected each fromacupoint 2424 acupointsacupoints was sampled based for on 5 s. thethe RyodorakuRyodoraku pointspoints proposed by Dr.Dr. Yoshio Nakatani [[9],9], and each acupoint was sampled for 5 s. 2.2. Smartphone 2.2. Smartphone 2.2. SmartphoneThe proposed system was implemented on the Android platform. Both the control packets and The proposed system was implemented on the Android platform. Both the control packets and data Thepackets proposed were system sent over was implementedUART between on thethe Anphonedroid and platform. the PPG/GSR Both the controlsensor. packetsNote that, and data packets were sent over UART between the phone and the PPG/GSR sensor. Note that, considering consideringdata packets the were resolution sent over of the UART ADC onbetween the Arduin the ophone beetle and board the is 10PPG/GSR bit while sensor. the packet Note size that, of the resolution of the ADC on the Arduino beetle board is 10 bit while the packet size of UART is only UARTconsidering is only the one resolution byte, we of used the ADC a simple on the LZ-based Arduin omethod beetle boardto compress is 10 bit the while sensor the packetdata into size one of one byte, we used a simple LZ-based method to compress the sensor data into one byte. During the byte.UART During is only the one measurements, byte, we used as a simpleshown LZ-basedin Figure 3a,method one endto compress of the GSR the probe sensor was data put into on onethe measurements, as shown in Figure3a, one end of the GSR probe was put on the palm while the other palmbyte. whileDuring the the other measurements, end was placed as shown on the intargeted Figure acupuncture 3a, one end point.of the ForGSR the probe PPG wasmeasurements, put on the end was placed on the targeted acupuncture point. For the PPG measurements, the sensor was placed thepalm sensor while was the placedother end on wasthe “Guan”placed on position, the targeted which acupuncture is adjacent to point. the styloid For the process PPG measurements, of the radius on the “Guan” position, which is adjacent to the styloid process of the radius bone (at the medial bonethe sensor (at the was medial placed portion on the of “Guan” the bulge position, of the prominent which is adjacent bone proximal to the styloid to the palm)process [11], of thefollowing radius portion of the bulge of the prominent bone proximal to the palm) [11], following the practice used thebone practice (at the medialused in portion traditional of the Chinese bulge ofmedicine. the prominent We used bone adhesive proximal tape to theto wrap palm) the [11], PPG following sensor in traditional Chinese medicine. We used adhesive tape to wrap the PPG sensor around the wrist, aroundthe practice the wrist, used asin showntraditional in Figure Chinese 3b, tomedicine. reduce movement We used adhesiveartifacts during tape to the wrap measurements. the PPG sensor The as shown in Figure3b, to reduce movement artifacts during the measurements. The sample rates for samplearound therates wrist, for theas shown PPG and in Figure GSR sensor3b, to redu werece 25 movement Hz and 200artifacts Hz, respectively.during the measurements. For the PPG, Thewe the PPG and GSR sensor were 25 Hz and 200 Hz, respectively. For the PPG, we collected all of the raw collectedsample rates all offor the the raw PPG data. and WeGSR only sensor recorded were 25 the Hz median and 200 of Hz, the respectively. GSR measurements For the PPG,for each we data. We only recorded the median of the GSR measurements for each acupuncture point. acupuncturecollected all point.of the raw data. We only recorded the median of the GSR measurements for each acupuncture point.

(a) (b) Figure ( 3.a )Placement of the ( a) GSR and ( b) PPG(b ) sensors. Figure 3. Placement of the (a) GSR and (b) PPG sensors. A simple user interface was implemented on the phone that allowed the user to startstart/stop/stop the sensorA datasimple collection user interface and view was the implemented obtained signals on the in realphone time. that The allowed measured the data user were to start/stop first ﬁrst stored the insensor the phone data collection and thenthen and transmittedtransmitted view the toto obtained thethe cloudcloud signals serverserver in atat real thethe time. endend ofTheof eacheach measured measurementmeasurement data were process.process. first stored in the phone and then transmitted to the cloud server at the end of each measurement process.

Sensors 2020, 20, 4618 5 of 14 Sensors 2020, 20, x FOR PEER REVIEW 5 of 14

2.3. Cloud2.3. Cloud Server Server We usedWe used MySQL MySQL to store to store the GSRthe GSR and PPGand PPG signals signal (includings (including 18 features 18 features computed computed from from the the PPGPPG signals signals (which (which will bewill discussed be discussed shortly). shortly). The ER The (entity-relationship) ER (entity-relationship) model model of the of SQL the tables SQL istables shownis shown in Figure in 4Figure. 4.

FigureFigure 4. ER 4. model ER model of the of SQL the SQL tables. tables.

To predictTo predict the pulsethe pulse type, type, we utilizedwe utilized forty-two forty-two features features computed computed from from PPG PPG and and GSR GSR data data (collected(collected at various at various acupoints), acupoint ass), listed as listed in Table in Table1. 1. EighteenEighteen features features were were computed computed from from PPG PPG signals. signal Theses. These features features consisted consisted of 8 time-domainof 8 time-domain featuresfeatures and 10and frequency-domain 10 frequency-domain features, features, as shown as show in Figuresn in Figures5 and 56 and, respectively. 6, respectively. These These features features werewere selected selected based based on an on extensive an extensive survey survey of common of common features features used used in previous in previous studies studies related related to to pulsepulse diagnosis diagnosis [4–9 ,[4–9,12–18].12–18]. We implementedWe implemented a simple a simple PMAF PMAF [19] method[19] method to remove to remove motion motion artifacts artifacts and noisesand noises in the in PPGthe PPG signals signals before before computing computing the time-the time- and and frequency-domain frequency-domain features features of the of PPG the PPG data.data. The The PMAF PMAF was was based based on the on quasi-periodicitythe quasi-periodicity of the of PPGthe PPG signals, signals, which whic ﬁrsth first segments segments the PPGthe PPG signalsignal into into periods periods and thenand then resamples resamples each each period period [20]. [20].

TableTable 1. PPG 1. PPG and GSRand GSR features features used used to estimate to estimate the traditional the traditional Chinese Chinese medicine medicine (TCM) (TCM) pulse pulse type. type.

Time-DomainTime-Domain Features Features of PPG of PPG Data (AsData Shown (As inShown Figure 5)in Figure 5) Pulse amplitude diﬀerence between pulse wave systolic peak (PWSP) 1 Pulse amplitude difference between pulse wave systolic peak 1 and pulse wave diastolic peak (PWDP) (PWSP) and pulse wave diastolic peak (PWDP) 2 Amplitude of pulse wave diastolic peak (PWDP) 2 Amplitude of pulse wave diastolic peak (PWDP) 3 Angle of pulse wave systolic peak (PWSP) 3 Angle of pulse wave systolic peak (PWSP) 4 Pulse wave amplitude (PWA) 4 Pulse wave amplitude (PWA) 5 Duration of systolic phase 5 Duration of systolic phase 6 Duration of diastolic phase 6 Duration of diastolic phase Pulse propagation time (PPT) between pulse wave systolic peak (PWSP) 7 Pulse propagation time (PPT) between pulse wave systolic 7 and pulse wave diastolic peak (PWDP) peak (PWSP) and pulse wave diastolic peak (PWDP) 8 Inter-beat interval (IBI) 8 Inter-beat interval (IBI) Frequency domain features of PPG data

Sensors 2020, 20, 4618 6 of 14

Table 1. Cont.

Time-Domain Features of PPG Data (As Shown in Figure 5) Frequency domain features of PPG data the nth harmonic of base frequency (e.g., if the heartrate is SensorsSensors 2020 2020,, ,20 20,, ,xx x FORFOR FOR PEERPEER PEER REVIEWREVIEW REVIEW 66 ofof 1414 SensorsSensors 2020 2020,,96 ,20 20, bpm,, , xx x FORFOR FOR then PEERPEER PEER C1 REVIEWREVIEW REVIEW Corresponding TCM Meridian 66 ofof 1414 corresponds to 1.6 Hz, C2 thethethecorresponds nth nthnth harmonic harmonicharmonic to 3.2 Hz,of ofof base basebase... ., frequency frequency etc.)frequency (e.g., (e.g.,(e.g., if ifif the thethe thethethe nth nthnth harmonic harmonicharmonic of ofof base basebase frequency frequencyfrequency (e.g., (e.g.,(e.g., if ifif the thethe heartrateheartrate is is 96 96 bpm, bpm, then then C1 C1 corresponds corresponds to to 1.6 1.6 CorrespondingCorresponding TCM TCM Meridian Meridian heartrateheartrate is is 96 96 C1bpm, bpm, then then C1 C1 corresponds corresponds to to 1.6 1.6 Corresponding LiverCorresponding Meridian TCM TCM Meridian Meridian Hz,Hz, C2 C2 corresponds corresponds to to 3.2 3.2 Hz, Hz, …., …., etc.) etc.) Hz,Hz, C2 C2 corresponds correspondsC2 to to 3.2 3.2 Hz, Hz, …., …., etc.) etc.) Kidney Meridian C1C1 LiverLiver Meridian Meridian C3C1C1 Spleen MeridianLiverLiver Meridian Meridian C2C2 KidneyKidney Meridian Meridian C2C2 KidneyKidney Meridian Meridian C4C2C2 Lung MeridianKidneyKidney Meridian Meridian C3C3 SpleenSpleen Meridian Meridian C3C3 SpleenSpleen Meridian Meridian C5C4C4 Stomach MeridianLungLung Meridian Meridian C4C4 LungLung Meridian Meridian C6C5C5 GallbladderStomachStomach Meridian Meridian Meridian C5C5 StomachStomach Meridian Meridian C7C6C6 BladderGallbladderGallbladder Meridian Meridian Meridian C6C6 GallbladderGallbladder Meridian Meridian C7C7 BladderBladder Meridian Meridian C8C7C7 Large IntestineBladderBladder Meridian Meridian Meridian C8C8 Large Large Intestine Intestine Meridian Meridian C9C8C8 San-yin-jiao Large Large MeridianIntestine Intestine Meridian Meridian C9C9 San-yin-jiaoSan-yin-jiao Meridian Meridian C10C9C9 Small IntestineSan-yin-jiaoSan-yin-jiao Meridian Meridian Meridian C10C10 Small Small Intestine Intestine Meridian Meridian C10C10 Small Small Intestine Intestine Meridian Meridian Skin impedanceC10C10 of acupoints from 12 meridians (including Small Small both Intestine Intestine left and right Meridian Meridian hands) SkinSkin impedance impedance of of acupoints acupoints from from 12 12 meridi meridiansans (including (including both both left left and and right right hands) hands) SkinSkin impedance impedance of of acupoints acupoints from from 12 12 meridi meridiansans (including (including both both left left and and right right hands) hands)

ShenmenShenmen (HT7) (HT7) TaiyuanTaiyuan (LU9)(LU9) (LU9) DalingDaling (PC7)(PC7) ShenmenShenmenShenmen (HT7) (HT7)(HT7) TaiyuanTaiyuan (LU9) (LU9) DalingDaling (PC7) (PC7) ShenmenShenmen (HT7) (HT7)

YangguYanggu (SI5) (SI5) YangchiYangchi (TE4) (TE4) YangxiYangxi (LI5) (LI5) YangguYanggu (SI5)(SI5) (SI5) YangchiYangchi (TE4)(TE4) (TE4) YangxiYangxiYangxi (LI5) (LI5)(LI5)

TaichongTaichong (LV3) (LV3) ShuguShugu (BL65) (BL65) TaichongTaichong (LV3) (LV3) ChongyangChongyang (ST42) (ST42) ShuguShugu (BL65)(BL65) TaichongTaichongTaichong (LV3)(LV3) (LV3) ChongyangChongyangChongyang (ST42) (ST42)(ST42)

TaibaiTaibai (SP3) (SP3) ChiuChiu Hsi Hsi (GB40) (GB40) DazhongDazhong (KD4) (KD4) TaibaiTaibai (SP3) (SP3)(SP3) ChiuChiuChiu Hsi HsiHsi (GB40)(GB40) DazhongDazhongDazhong (KD4) (KD4)

AfterAfter removingremoving thethe noisenoise fromfrom thethe PPGPPG waves,waves, wewe firstfirstfirst consideredconsideredconsidered thethethe threethreethree featurefeaturefeature pointspointspoints forforfor AfterAfter removingremoving thethe noisenoise fromfrom thethe PPGPPG waves,waves, wewe firstfirstfirst consideredconsideredconsidered thethethe threethreethree featurefeaturefeature pointspointspoints forforfor thetheAfterthe time-domain time-domaintime-domain removing features: features:features: the noise the thethe from pulse pulsepulse thewave wave PPG systolic systolic waves, peak peak we (PWSP), (PWSP), ﬁrst considered the the pulse pulse wave wave the begin begin three (PWB), (PWB), feature and and points the the for thethethe time-domain time-domaintime-domain features: features:features: the thethe pulse pulsepulse wave wave systolic systolic peak peak (PWSP), (PWSP), the the pulse pulse wave wave begin begin (PWB), (PWB), and and the the pulsepulse wave wave diastolic diastolic peak peak (PWDP), (PWDP), as as shown shown in in Figure Figure 5. 5. Once Once these these feature feature points points were were located, located, the the the time-domainpulsepulse wave wave diastolic diastolic features: peak peak the (PWDP), (PWDP), pulse as as wave shown shown systolic in in Figure Figure peak 5. 5. Once Once (PWSP), these these thefeature feature pulse points points wave were were begin located, located, (PWB), the the and eighteight time-domaintime-domain featuresfeatures shownshown inin FigureFigure 55 coulcouldd bebe computedcomputed accordingly.accordingly. ToTo detectdetect thethe pulsepulse the pulseeighteight wavetime-domaintime-domain diastolic featuresfeatures peak shown (PWDP),shown inin Figure asFigure shown 55 coulcoul indd Figure bebe computedcomputed5. Once accordingly.accordingly. these feature ToTo points detectdetect the werethe pulsepulse located, wavewave systolic systolic peak peak (PWSP), (PWSP), we we implemented implemented an an algorithm algorithm based based on on an an event-related event-related moving moving average average the eightwavewave systolic time-domainsystolic peak peak (PWSP), (PWSP), features we we implemented shownimplemented in Figure an an algorithm algorithm5 could based based be computed on on an an event-related event-related accordingly. moving moving To average average detect the [20].[20].[20]. ThisThisThis methodmethodmethod consistsconsistsconsists ofofof threethreethree stages:stages:stages: signalsignalsignal pre-processingpre-processingpre-processing (including(including(including bandpassbandpassbandpass filteringfilteringfiltering andandand pulse[20].[20].[20]. wave ThisThisThis systolic methodmethodmethod peak consistsconsistsconsists (PWSP), ofofof threethreethree we stages:stages:stages: implemented signalsignalsignal pre-processingpre-processingpre-processing an algorithm (including(including(including based on bandpassbandpassbandpass an event-related filteringfilteringfiltering andandand moving squaring),squaring), generationgeneration ofof blocksblocksblocks ofofof interestinterestinterest usingusingusing twotwotwo movingmovingmoving averages,averages,averages, andandand classification-basedclassification-basedclassification-based averagesquaring),squaring), [20]. This generationgeneration method ofof consists blocksblocksblocks of ofof threeinterestinterestinterest stages: usingusingusing signal twotwotwo movingmovingmoving pre-processing averages,averages,averages, (including andandand classification-basedclassification-basedclassification-based bandpass ﬁltering adaptiveadaptive thresholding.thresholding. TheThe locationlocation ofof thethe pulsepulse wavewave beginbegin (PWB)(PWB) cancan bebe detecteddetected usingusing thethe samesame adaptiveadaptive thresholding.thresholding. TheThe locationlocation ofof thethe pulsepulse wavewave beginbegin (PWB)(PWB) cancan bebe detecteddetected usingusing thethe samesame algorithmalgorithm byby changingchanging thethe inputinput tototo thethethe reversedreversedreversed PPGPPGPPG signalssignalssignals (so(so(so thatthatthat PWBPWBPWB becomesbecomesbecomes thethethe maximamaximamaxima ininin algorithmalgorithm byby changingchanging thethe inputinput tototo thethethe reversedreversedreversed PPGPPGPPG signalssignalssignals (so(so(so thatthatthat PWBPWBPWB becomesbecomesbecomes thethethe maximamaximamaxima ininin thethethe signals, signals,signals, as asas shown shownshown in inin Figure FigureFigure 6). 6).6). thethethe signals, signals,signals, as asas shown shownshown in inin Figure FigureFigure 6). 6).6).

Sensors 2020, 20, 4618 7 of 14 and squaring), generation of blocks of interest using two moving averages, and classiﬁcation-based adaptive thresholding. The location of the pulse wave begin (PWB) can be detected using the same algorithm by changing the input to the reversed PPG signals (so that PWB becomes the maxima in the signals, asSensorsSensors shown 2020 2020, in20, 20, x, Figure xFOR FOR PEER PEER6). REVIEW REVIEW 7 7of of 14 14

FigureFigureFigure 5. Time-domain 5. 5. Time-domain Time-domain featuresfeatures features of of ofthe the thePPG PPG PPG signal. signal. signal.

Figure 6.FigureFigureDetection 6. 6. Detection Detection of the of of pulsethe the pulse pulse wave wave wave systolic systolic systolic peak peak (PWSP) (PWSP) (PWSP) and and andpulse pulse pulsewave wave begin wavebegin (PWB). (PWB). begin (PWB).

To detectToTo diastolic detect detect diastolic diastolic peaks, peaks, peaks, we we first we first first computed computed computed th thethe esecond-order second-order derivates derivates derivates of of the the PPG PPG of thewaveforms, waveforms, PPG waveforms, which were then smoothed using a moving average filter. Generally speaking, the largest minima which werewhich then were smoothed then smoothed using using a moving a moving average average filter. filter. Generally Generally speaking, speaking, the largest the minima largest minima withinwithin the the second-order second-order derivative derivative waveform waveform corr correspondespond to to the the systolic systolic peaks, peaks, and and the the minima minima within thefollowingfollowing second-order these these typically typically derivative correspond correspond waveform to to the the diastolic diastolic correspond peaks. peaks. Once Once to the the locations locations systolic of of the peaks,the systolic systolic and peaks peaks the minima followingwere thesewere identified, identified, typically as as above, correspondabove, we we could could toperform perform the diastolic peak peak detection detection peaks. on on the Oncethe inverted inverted the locationssecond-order second-order of derivative thederivative systolic peaks were identified,waveformwaveform as to above,to find find the the we local local could maxima maxima perform corresponding corresponding peak detectionto to the the diastolic diastolic on thepeak. peak. inverted The The timing timing second-order of of the the diastolic diastolic derivative peakpeak was was then then recorded recorded as as the the timing timing of of the the minima minima following following the the systolic systolic peak peak in in each each PPG PPG wave. wave. waveform to find the local maxima corresponding to the diastolic peak. The timing of the diastolic OnceOnce the the pulse pulse wave wave systolic systolic peak peak (PWSP), (PWSP), the the pulse pulse wave wave begin begin (PWB), (PWB), and and the the pulse pulse wave wave peak wasdiastolic thendiastolic recorded peak peak (PWDP) (PWDP) as the were were timing identified, identified, of the we we minima could could then then following compute compute the thethe eight eight systolic time-domain time-domain peak in features, features, eachPPG as as wave. Onceshownshown the pulse from from Equation waveEquation systolic (1) (1) to to Equation Equation peak (PWSP),(8). (8). Specifically, Specifically, the pulsethe the location location wave of of beginthe the PWSP, PWSP, (PWB), PWB, PWB, andand and PWDP thePWDP pulse wave diastolic peakwerewere given given (PWDP) a a 2D 2D coordinate coordinate were identified, (x,y), (x,y), where where wex x is is the couldthe amp amplitude thenlitude of of compute the the feature feature thepoint, point, eight and and y y time-domainis is the the timing timing in in features, thethe signals. signals. For For example, example, PWSP(x,y) PWSP(x,y) refers refers to to the the 2D 2D coordinate coordinate of of PWSP, PWSP, and and PWSP(x) PWSP(x) indicates indicates the the as shown from Equation (1) to Equation (8). Specifically, the location of the PWSP, PWB, and PWDP xx coordinate coordinate of of PWSP. PWSP. Feature Feature 1, 1, Feature Feature 2, 2, and and Feature Feature 4 4 could could be be computed computed using using the the basic basic were givenPythagoreanPythagorean a 2D coordinate theorem. theorem. The (x,y), The PWSP PWSP where angle angle x (Feature is(Feature the amplitude 3) 3) was was estimated estimated of the by by first feature first connecting connecting point, the the and PWSP PWSP y is with with the timing in the signals.thethe For sampled sampled example, point point before PWSP(x,y)before and and the the referssampled sampled to point thepoint 2Dafte after coordinate rinto into a a triangle. triangle. of The PWSP,The PWSP PWSP and angle angle PWSP(x) could could then then indicates be be the x coordinatecomputedcomputed of PWSP. using using Feature the the inverse inverse 1, Feature trigonometric trigonometric 2, and function. Featurefunction. 4 could be computed using the basic Pythagorean theorem. The PWSP angle𝐹𝑒𝑎𝑡𝑢𝑟𝑒𝐹𝑒𝑎𝑡𝑢𝑟𝑒 (Feature 1= 3) 1= (P(P wasWSPWSP(x,y)–PWDP(x,y)) estimated(x,y)–PWDP(x,y)) by first2-(P2-(PWSPWSP connecting(x)–PWDP(x))(x)–PWDP(x)) the PWSP with(1)(1) the sampled point before and the sampled𝐹𝑒𝑎𝑡𝑢𝑟𝑒𝐹𝑒𝑎𝑡𝑢𝑟𝑒 point 2= after 2= (PWDP(x,y)–(PWDP(x,y)– into atriangle. PWE(x,y)) PWE(x,y))2-(PWDP(x)–PWE(x)) The2-(PWDP(x)–PWE(x)) PWSP angle could then(2) be(2) computed using the inverse trigonometric function. 𝑎𝑎+𝑏+𝑏−𝑐−𝑐 𝐹𝑒𝑎𝑡𝑢𝑟𝑒𝐹𝑒𝑎𝑡𝑢𝑟𝑒 3=∠ 3=∠𝐶𝐶 = = 𝑎𝑟𝑐𝑐𝑜𝑠 𝑎𝑟𝑐𝑐𝑜𝑠 (3)(3) q 2𝑎𝑏2𝑎𝑏 2 2 Feature 1 =𝐹𝑒𝑎𝑡𝑢𝑟𝑒𝐹𝑒𝑎𝑡𝑢𝑟𝑒(PWSP 4= ( 4= x,(PWSP(x,y)–PWB(x,y))(PWSP(x,y)–PWB(x,y)) y) PWDP(x, y))-(PWSP(x)–PWB(x))2-(PWSP(x)–PWB(x))(PWSP(x) PWDP (x)) (4)(4) (1) − − − q Feature 2 = (PWDP(x, y) PWE(x, y))2 (PWDP(x) PWE(x)) (2) − − − a2 + b2 c2 Feature 3 = ∠C = arccos − (3) 2ab Sensors 2020, 20, x FOR PEER REVIEW 8 of 14

𝐹𝑒𝑎𝑡𝑢𝑟𝑒 5 =PWSP x – PWBx (5) 𝐹𝑒𝑎𝑡𝑢𝑟𝑒 6 =PWSP x – PWEx (6) 𝐹𝑒𝑎𝑡𝑢𝑟𝑒 7 =PWSP x – PWDPx (7) Sensors 2020, 20, 4618 8 of 14 𝐹𝑒𝑎𝑡𝑟𝑒𝑢 8 = next PWSPx −PWSPx (8)

The frequency-domain featuresq were based on the resonance theory proposed by Wang [13]. He found that the intensityFeature 4 of= the (resonancePWSP(x, waves y) PWB of different(x, y))2 frequencies(PWSP( xand) PWBthe physiological(x)) and (4) pathological changes of different organs have a− relationship [21].− Based on several− animal and clinical observations, Wang claimed thatFeature C0 is the5 = totalPWSP load(x )of thePWB heart(x )in a cardiac cycle. There are (5) − corresponding relationships between different harmonics and TCM meridians (as shown in Table 1). Feature 6 = PWSP(x) PWE(x) (6) In this work, we collected 5-minute PPG measurements from− each subject. A sliding-window (with a window size of 10 s) approachFeature was implemente7 = PWSPd (tox )segmentPWDP the( xPPG) data for the purpose of (7) computing the frequency-domain features. An FFT (fast Fourier− transformation) was used to convert the PPG signals to frequency-domainFeatreu information8 = next PWSPfor each(x 10-second) PWSP segment.(x) Based on the subject’s (8) − heart rate calculated in each window, we computed the base frequency and found the harmonic of theThe base frequency-domain frequency from C1 to features C10, as shown were based in Table on 1. the resonance theory proposed by Wang [13]. He foundThe that features the intensitytaken from of the the GSR resonance data included waves skin of di impedancefferent frequencies for 12 acupoints, and the including physiological andTaiyuan pathological (LU9), changesDaling (PC7), of di ffShenmenerent organs (HT7), have Yanggu a relationship (SI5), Yangchi [21 (TE4),]. Based Yangxi on several (LI5), Shugu animal and clinical(BL65), observations, Taichong (LV3), Wang Chongyang claimed that(ST42), C0 Taibai is the (SP3), total loadChiu ofHsi the (GB40), heart and in a Dazhong cardiac cycle.(KD4), There as are correspondingshown in Table relationships 1. The selection between of these di acupointfferent harmonicss was based and on the TCM Ryodoraku meridians points (as proposed shown in by Table 1). In thisDr. work,Yoshio weNakatani collected [9]. 5According min PPG to measurements traditional Chinese from medicine each subject. (TCM), A these sliding-window acupoints are (with a windowpresent size on ofboth 10 s)the approach left side and was the implemented right side ofto the segment body, and the they PPG are data symmetrical. for the purpose (Incidentally, of computing as a side observation, based on measurements from 20 volunteers, we found the skin impedance the frequency-domain features. An FFT (fast Fourier transformation) was used to convert the PPG taken from the acupoint on the left side of the body had a strong correlation with that from the same signals to frequency-domain information for each 10 s segment. Based on the subject’s heart rate acupoint on the right side of the body, as in the examples shown in Figure 7.) Nevertheless, in this calculatedstudy, we in used each the window, 24 acupoints we computedfrom both sides the baseof the frequencybody in ourand experiments. found the harmonic of the base frequencyFinally, from with C1 tothe C10, features as shown extracted in Table from1 .the PPG and GSR data, we trained a simple neural networkThe features (NN) model taken based from on thea 5-fold GSR validation data included scheme (i.e., skin 80% impedance of the data for were 12 used acupoints, for training, including Taiyuanand 20% (LU9), were Daling used for (PC7), testing) Shenmen to predict (HT7), the TC YangguM pulse (SI5), type. Yangchi Specifically, (TE4), we Yangxi implemented (LI5), Shugu a fully (BL65), Taichongconnected (LV3), neural Chongyang network (FCN), (ST42), for Taibai which (SP3),the inputs Chiu were Hsi the (GB40), PPG and and GSR Dazhong data. The (KD4), input layers as shown in Tablehad1. 42 The neurons selection (18 of them these were acupoints extracted was PPG based features on the and Ryodoraku the other 24 points were from proposed the GSR by data). Dr. Yoshio NakataniThe output [9]. Accordinglayers had a to single traditional output that Chinese indicated medicine the probability (TCM), these of predicting acupoints a specific are present pulse type. on both the Three hidden layers with 30 neurons in each layer were employed in the experiments. In this study, left side and the right side of the body, and they are symmetrical. (Incidentally, as a side observation, we focused on the prediction of a “wiry pulse,” which is a commonly seen pulse type usually based on measurements from 20 volunteers, we found the skin impedance taken from the acupoint on observed in people who are stressed or have liver-related diseases. To obtain the ground truth for the thepulse left side type, of three the body independent, had a strong experienced correlation TCM withdoctors that were from responsible the same for acupoint the subject on theselection. right side of theA body, subject as inwas the only examples included shown into this in study Figure if7 he/s.) Nevertheless,he was diagnosed in this with study, the same we usedpulse thetype 24 by acupoints all fromthree both TCM sides doctors. of the body in our experiments.

(a) (b)

FigureFigure 7. 7.(a )(a The) The correlation correlation betweenbetween the the Taiyuan Taiyuan ac acupointsupoints (left (left hand hand vs. vs.right righthand). hand). (b) The ( b) The correlationcorrelation between between Taichong Taichong acupointsacupoints (left (left leg leg vs. vs. right right leg). leg).

Finally, with the features extracted from the PPG and GSR data, we trained a simple neural network (NN) model based on a 5-fold validation scheme (i.e., 80% of the data were used for training, and 20% were used for testing) to predict the TCM pulse type. Speciﬁcally, we implemented a fully connected neural network (FCN), for which the inputs were the PPG and GSR data. The input layers had 42 neurons (18 of them were extracted PPG features and the other 24 were from the GSR data). The output layers had a single output that indicated the probability of predicting a speciﬁc pulse type. Three hidden layers with 30 neurons in each layer were employed in the experiments. In this study, we focused on the prediction of a “wiry pulse,” which is a commonly seen pulse type usually Sensors 2020, 20, 4618 9 of 14 observed in people who are stressed or have liver-related diseases. To obtain the ground truth for the pulse type, three independent, experienced TCM doctors were responsible for the subject selection. A subject was only included into this study if he/she was diagnosed with the same pulse type by all three TCM doctors.

3. ResultsSensors 2020, 20, x FOR PEER REVIEW 9 of 14

Before3. Results we reported our results on TCM pulse type classiﬁcation, we ﬁrst validated the PPG and GSR sensorBefore devices we reported employed our results in this on work TCM against pulse type some classification, commercial we products.first validated Next, the PPG as a and proof of concept,GSR wesensor recruited devices 80 employed subjects in (among this work them, agains 40tsubjects some commercial had a “wiry products. pulse” Next, based as a on proof agreement of amongconcept, three we TCM recruited doctors) 80 withsubjects the (among goal of them, determining 40 subjects if thehad proposeda “wiry pulse” sensor-fusion based on agreement system could predictamong the samethree TCM results doctors) agreed with upon theby goal the of TCMdetermi doctors.ning if the proposed sensor-fusion system could predict the same results agreed upon by the TCM doctors. 3.1. Validation of PPG/GSR Sensor Devices 3.1. Validation of PPG/GSR Sensor Devices For the PPG comparison, we used a commercial product called ANSwatch (Model TS-0411) [22] as our benchmark.For the PPG We comparison, compared we three used metrics,a commercial including product heart called rate, ANSwatch heart rate(Model variability TS-0411) (SDNN),[22] and LFas/ HF,our benchmark. obtained from We ANSwatchcompared three and ourmetrics, PPG including sensor device. heart rate, We foundheart rate a strong variability correlation (SDNN), (above and LF/HF, obtained from ANSwatch and our PPG sensor device. We found a strong correlation 90%) between ANSwatch and our device, based on one of the subjects, as shown in Figure8. (above 90%) between ANSwatch and our device, based on one of the subjects, as shown in Figure 8.

(a)

(b)

(c)

FigureFigure 8. ( 8.a)The (a)The HR HR (heart (heart rate)rate) correlation. correlation. (b) (Theb) The HRV HRV (SDNN) (SDNN) correlation. correlation. (c) The LF/HF (c) The ratio LF /HF correlation. ratio correlation. For validation of the proposed GSR sensor device, we used a commercial device called ARDK® (Automatic Reflexodiagnostic Komplex; AleXorA BioMedical Technologies Inc. Quezon City, Philippine) [23] as our benchmark. In addition, we also validated the output of our GSR device with

Sensors 2020, 20, 4618 10 of 14

For validation of the proposed GSR sensor device, we used a commercial device called ARDK® (AutomaticSensors 2020, 20 Reﬂexodiagnostic, x FOR PEER REVIEW Komplex; AleXorA BioMedical Technologies Inc. Quezon10 City,of 14 Philippine) [23] as our benchmark. In addition, we also validated the output of our GSR device with a a multi-meter (CHY-36C). Four resisters were used (their resistances were 384 kΩ, 806 kΩ, 3.52 MΩ, multi-meter (CHY-36C). Four resisters were used (their resistances were 384 kΩ, 806 kΩ, 3.52 MΩ, 990 KΩ, and 3.81 MΩ, respectively), and our GSR device output the same readings as those from the 990 KΩ, and 3.81 MΩ, respectively), and our GSR device output the same readings as those from multi-meter when measuring these resisters. When measuring the acupoints, the output of the the multi-meter when measuring these resisters. When measuring the acupoints, the output of the proposed GSR device was consistent with the ARDK output (the correlation on average, based on 20 proposed GSR device was consistent with the ARDK output (the correlation on average, based on participants, was higher than 95%), as shown in the example in Figure 9. 20 participants, was higher than 95%), as shown in the example in Figure9.

Figure 9. Comparison of the proposed GSR sensor with the Automatic Reflexodiagnostic Komplex Figure 9. Comparison of the proposed GSR sensor with the Automatic Reflexodiagnostic Komplex (ARDK). (ARDK). 3.2. Prediction of the Wiry Pulse 3.2. Prediction of the Wiry Pulse In this study, we recruited 80 subjects. Among them, 40 patients had a “wiry pulse,” and the others did notIn (amongthis study, them, we 38 recruited had a “normal 80 subjects. pulse” Among and 2 had them, a “slippery 40 patients pulse”). had The a “wiry identification pulse,” ofand pulse the typeothers for did each not subject (among was them, based 38 had on a “normal joint agreement pulse” and of three 2 had di a ff“slipperyerent TCM pulse”). doctors. The The identification sampling rateof pulse of the type GSR for sensor each subject was 200 was Hz, based and on that a ofjoint the agreement PPG sensor of three was 25 different Hz. The TCM duration doctors. of The the PPGsampling measurements rate of the wasGSR five sensor minutes. was 200 A sliding-windowHz, and that of the (with PPG a window sensor wa sizes 25 of Hz. 10s) The approach duration was of implementedthe PPG measurements to segment was the five PPG minutes. data before A sliding-window feeding the data (with into a window the neural size network. of 10 s) approach The total numberwas implemented of PPG data to segments segment forthe eachPPG subjectdata before was 290,feeding and the 18data time-domain into the neural and network. frequency-domain The total PPGnumber features of PPG were data computed segments for for each each segment subject (notewas 290, that and we computedthe 18 time-domain the average and value frequency- of each time-domaindomain PPG features feature forwere each computed segment). for each The segment GSR data (note for that each we acupoint computed were the sampled average value for five of seconds.each time-domain For each acupoint,feature for only each the segment). median The of the GSR sampled data for GSR each data acupoint was recorded were sampled as the featurefor five point.seconds. The For computed each acupoint, PPG and only GSR the features median were of the used sampled as the GSR input data for was theneural recorded network. as the Afeature fully connectedpoint. The neuralcomputed network PPG (FCN) and GSR was features implemented were withused Keras,as the whereinput thefor the input neural layers network. had 42 neurons. A fully Inconnected other words, neural each network subject (FCN) provided was 290 implemented samples. Each with sample Keras, contained where the 42 input features layers (18 ofhad them 42 wereneurons. extracted In other PPG words, features, each and subject the otherprovided 24 were 290 samples. from the Each GSR sample data). Sincecontained for each 42 features subject we(18 onlyof them measured were extracted GSR data PPG once features, for each and acupoint, the other the 24 GSRwere features from the were GSR thedata). same Since in allfor 290each samples subject forwe theonly same measured subject. GSR We data employed once for aeach five-fold acupoint, cross-validation the GSR features to select were thethe trainingsame in all data. 290 Insamples other words,for the insame each subject. experiment, We employed data from a 64five-fold subjects cross- werevalidation used for the to training,select the and training data fromdata. the In other 16words, subjects in each were experiment, used for testing. data fr Inom the 64 testing subjects group, were eight used subjects for the hadtraining, a “wiry and pulse,” data from and the other eight16 subjects subjects were did used not. for The testing. age range In the for testing the subjects group, who eight did subjects not have had a wirya “wiry pulse pulse,” ranged and from the other 15 to 96eight (mean subjectsSD: did 55.06 not. The19.2) age years range including for the 22subjec malests who and 18did females not have (BMI: a wiry 23.29 pulse2.42). ranged The from subject 15 ± ± ± groupto 96 (mean that did ± SD: not 55.06 have ± a 19.2) wiry years pulse including ranged in 22 age ma fromles and 13 to 18 85 females (mean (BMI:SD: 23.29 59.52 ± 2.42).15.28), The including subject ± ± 16group males that and did 24 not females have a (BMI: wiry 23.87pulse ranged4.3). in age from 13 to 85 (mean ± SD: 59.52 ± 15.28), including ± 16 malesTo determine and 24 females the level (BMI: of importance 23.87 ± 4.3). of different features, we also adjusted the input layer of the neuronTo networkdetermine for the diff erentlevel setsof importance of features, of as differen shown int features, Table2, where we also the adjusted GSR or PPG the featuresinput layer were of the neuron network for different sets of features, as shown in Table 2, where the GSR or PPG features were used alone. The prediction accuracy was generally less than 80%. However, when we combined both features together, the accuracy increased to 91%, which shows the benefit of using data fusion for TCM pulse type identification.

Sensors 2020, 20, 4618 11 of 14 used alone. The prediction accuracy was generally less than 80%. However, when we combined both features together, the accuracy increased to 91%, which shows the beneﬁt of using data fusion for TCM pulse type identiﬁcation.

Table 2. Comparison of feature accuracy using ﬁve-fold cross-validation.

Feature Used Accuracy Only 24 GSR features 62.5% 18 PPG (time-/frequency-domain) features 80% 8 PPG time-domain features 65% 10 PPG frequency-domain features 62.5% 8 PPG time-domain and 24 GSR features 72.5% 10 PPG frequency-domain and 24 GSR features 75% 18 PPG (time-/frequency-domain) features and 24 GSR features 91%

In addition, we evaluated the importance of speciﬁc features through random decision forests [24]. As shown in Table3, the top three features with the highest weight in terms of classifying the wiry pulse were feature 2 and feature 3 in the PPG time-domain and the amplitude of the second harmonic in the PPG frequency-domain data. Speciﬁcally, subjects who had a wiry pulse tended to have a larger angle for the systolic peak, and their diastolic peaks were usually closer to the systolic peak over time. In addition, their second harmonics, corresponding to the kidney meridian, were often weaker as compared to the control group (i.e., the group that did not have a wiry pulse).

Table 3. Order of importance of features.

Wiry Pulse Group Non-Wiry Pulse Group Feature Weight p Value (mean std) (mean std) ± ± PPG (2) 0.24636 2.92 0.0303 1.534 0.0838 <0.001 ** ± ± PPG (3) 0.225339 155.0205 20.5951 47.8 13.2765 <0.001 ** ± ± PPG-C2 0.214329 0.0046 0.0049 0.0179 0.0091 <0.001 ** ± ± ** p < 0.001.

Finally, to further validate our results, we recruited another 10 subjects with a wiry pulse. Again, the determination of the pulse type was based on the joint agreement of three experienced TCM doctors. We use the previously trained model (based on data from the original 80 subjects) to classify these subjects. An accuracy of 90% was obtained in this experiment (i.e., the pulse type of one of these 10 subjects was not correctly predicted). In our review of previous studies on pulse type prediction, most of them validated their results based on a small number of subjects. Some of the prior work also employed artiﬁcial neural networks (ANNs) for predicting pulse type. For example, Wang et al. [25] implemented a simple three-layer ANN to identify the wiry pulse and the slippery pulse. They reported an 87% predictive accuracy. Xu et al. [26] utilized seventeen features from the time-domain of pulse waves to train a three-layer ANN for predicting eight pulse conditions (not speciﬁed in their paper) and obtained 86% accuracy. As compared to these existing works, our study was based on a much larger and more diverse subject pool and thus had higher predictive accuracy.

4. Conclusions and Suggestions for Future Work For several thousands of years, the pulse diagnosis has been one of the important diagnostic methods in traditional Chinese medicine (TCM). While a significant amount of effort has been put into the development of pulse diagnostic instruments, they are generally single-probe sensors that can only obtain limited information about pulse characteristics. In this work, we propose a low-cost smartphone-based system to infer the TCM pulse type based on the concept of sensor fusion. Specifically, Sensors 2020, 20, 4618 12 of 14

by combining PPG and GSR data collected from a smartphone and running the prediction on top of a neural network, we were able to detect the wiry pulse, a commonly seen TCM pulse type, with a high degree of accuracy. As a feasibility study, our current results are quite limited since they are only based on a small set of people with one specific TCM pulse type. Originally, we tried to formulate the problem as a multi-class classification problem. However, given that we did not have sufficient data for the “slippery pulse,” the results were poor. Therefore, we grouped the subjects with a “normal pulse” and “slippery pulse” together into a “non-wiry pulse” class, and only performed a binary classification in this study. However, we believe that our results can be generalized to infer other TCM pulse types, and we are currently working toward this goal by recruiting more subjects with different TCM pulse types (such as the slippery pulse, which is commonly seen in pregnant women). In addition, establishing an agreement of a TCM pulse type among different TCM practitioners is challenging. In this work, we evaluated our prediction results based on the opinion of a small panel of three TCM doctors. A quantifiable traditional Chinese medicine (TCM) pulse diagnostic model and its validation is another related topic we aim to investigate in the future. Finally, in this work, we computed the PPG and GSR data features, which were then used as inputs for a simple three-layer ANN to predict the wiry pulse. Recently, various deep neural network structures, such as RNN [27,28] and CNN [29], have been proposed for signal processing [30]. However, these deep architecture models generally suffer from over-fitting problems when the training data are small [31]. Due to the limited size of our dataset, we did not adopt these deep neural network models in this study. Nevertheless, we plan to investigate their applications to TCM pulse type prediction once we collect more data in the future.

Author Contributions: Conceptualization, K.-C.L.; Data curation, K.-C.L. and T.-H.H.; Funding acquisition, K.-C.L.; Investigation, K.-C.L.; Methodology, K.-C.L.; Project administration, K.-C.L.; Resources, K.-C.L.; Software, T.-H.H.; Supervision, K.-C.L.; Validation, K.-C.L. and T.-H.H.; Writing—original draft, K.-C.L. and T.-H.H; Writing—review & editing, K.-C.L. and G.L. All authors have read and agreed to the published version of the manuscript. Funding: This joint publication was supported by funds from the MOST (Ministry of Science and Technology) by a MOST Add-on Grant for International Cooperation (leader in Taiwan: Kun-Chan Lan (K.-C.L.), leader in Austria: Gerhard Litscher (G.L.)). The research was supported by the European Academy of TCM (EATCM); the Initiative for Medicine without Side Effects (nonprofit organization); the German Academy of Acupuncture; and the TCM Research Center Graz, Medical University of Graz. Acknowledgments: Our deepest gratitude goes to the anonymous reviewers and the editor for their careful work and thoughtful suggestions that have helped improve the quality of this paper substantially. Conflicts of Interest: The authors declare no conflicts of interest regarding the publication of this work. Data Availability: The original data used to support the findings of this study are available from the first author upon request. Disclosure: Gerhard Litscher is also a visiting professor at the China Medical University, Taichung, Taiwan.

References

1. Wang, L.; Shang, Q.Q.; Wang, Y.Q.; Qian, P.; Guo, R.; Yan, H.X. Discussion on international standard of English translation for TCM pulse condition name terms. Chin. J. Inf. TCM 2017, 24, 5–8. 2. Litscher, G.; Yannacopoulos, T.; Kreisl, P. Nogier reflex: Physiological and experimental results in auricular medicine—A new hypothesis. Medicines 2018, 5, 132. [CrossRef][PubMed] 3. Chen, Y.Y.; Chang, R.S.; Jwo, K.W.; Hsu, C.C.; Tsao, C.P. A non-contact pulse automatic positioning measurement system for traditional Chinese medicine. Sensors 2015, 15, 9899–9914. [CrossRef][PubMed] 4. Lygouras, J.N.; Tsalides, P.G. Optical-fiber finger photo-plethysmograph using digital techniques. IEEE Sens. J. 2002, 2, 20–25. [CrossRef] 5. Yoon, Y.Z.; Lee, M.H.; Soh, K.S. Pulse type classification by varying contact pressure. IEEE Eng. Med. Biol. 2000, 19, 106–110. [CrossRef] 6. Lin, Y.; Leng, H.; Yang, G.; Cai, H. An intelligent noninvasive sensor for driver pulse wave measurement. IEEE Sens. J. 2007, 7, 790–799. [CrossRef] Sensors 2020, 20, 4618 13 of 14

7. Joshi, A.B.; Kalange, A.E.; Bodas, D.; Gangal, S.A. Simulations of piezoelectric pressure sensor for radial artery pulse measurement. Mater. Sci. Eng. B 2010, 168, 250–253. [CrossRef] 8. Chung, Y.F.; Chu, Y.W.; Chung, C.Y.; Hu, C.S.; Luo, C.H.; Yeh, C.C. New Vision of the Pulse Conditions Using Bi-Sensing Pulse Diagnosis Instrument. In Proceedings of the 1st International Conference on Orange Technologies (ICOT), Tainan, Taiwan, 12–16 March 2013; IEEE: Piscataway Township, NJ, USA, 2013. [CrossRef] 9. Yuan, H. Chinese Pulse Diagnosis; Elsevier Urban Fischer: München, Germany, 2009. 10. Hernandez-Garc, A.; Konig, P. Data augmentation instead of explicit regularization. arXiv 2018, arXiv:1806.03852. 11. Tago, K.; Wang, H.; Jin, Q. Classification of TCM Pulse Diagnoses Based on Pulse and Periodic Features from Personal Health Data. In Proceedings of the Global Communications Conference (GLOBECOM), Wuhan, China, 6–8 July 2007. 12. Moser, M.; Frühwirth, M.; Messerschmidt, D.; Goswami, N.; Dorfer, L.; Bahr, F.; Opitz, G. Investigation of a micro-test for circulatory autonomic nervous system responses. Front. Physiol. 2017, 8, 448. [CrossRef] [PubMed] 13. Wang, W. Movement of Qi; Chinese People’s University Press: Beijing, China, 2006; p. 212. 14. Wei, L.Y.; Winchester, T. Electronic diagnose of arterial pulse. J. Med. Eng. Technol. 1985, 9, 183–186. [CrossRef][PubMed] 15. Zhang, X. Study on the Correlation between Viscera Theory of TCM and Organ Resonance Theory by Analyzing Pulse Harmonic Spectrum. Ph.D. Thesis, China Institute of Medicine, China Medical College of Taiwan, Taichung, Taiwan, 1993. 16. Zhang, Y.; Liu, J. Design and Realization of TCM Pulse Analysis and Management System. In Proceedings of the 2018 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), Xiamen, China, 25–26 January 2018; pp. 488–491. [CrossRef] 17. Xue, L.; Fu, Z.; Qian, P.; Wang, L.; Zhang, H.; Zhou, X.; Zhang, W.; Li, F. Computerized Wrist Pulse Signal Diagnosis Using Gradient Boosting Decision Tree. In Proceedings of the 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain, 3–6 December 2018; pp. 1941–1947. [CrossRef] 18. Yan, H.X.; Wang, Y.Q.; Guo, R.; Liu, Z.R.; Li, F.F.; Run, F.Y.; Hong, Y.J.; Yan, J.J. Feature Extraction and Recognition for Pulse Waveform in Traditional Chinese Medicine Based on Hemodynamics Principle. In Proceedings of the IEEE ICCA, Xiamen, China, 9–11 June 2010; pp. 972–976. [CrossRef] 19. Lee, H.W.; Lee, J.W.; Jung, W.G.; Lee, G.K. The Periodic Moving Average Filter for Removing Motion Artifacts from PPG Signals. Int. J. Control Autom. Syst. 2007, 5, 701–706. 20. Elgendi, M.; Norton, I.; Brearley, M.; Abbott, D.; Schuurmans, D. Systolic peak detection in acceleration photoplethysmograms measured from emergency responders in tropical conditions. PLoS ONE 2013, 8, e76585. [CrossRef][PubMed] 21. Aoyagi, T.; Miyasaka, K. Pulse oximetry: Its invention, contribution to medicine, and future tasks. Anesth. Analg. 2002, 94, 51–53. 22. Cowie, M.R.; Sarkar, S.; Koehler, J. Development and validation of an integrated diagnostic algorithm derived from parameters monitored in implantable devices for identifying patients at risk for heart failure hospitalization in an ambulatory setting. Eur. Heart J. 2013, 34, 2472–2480. [CrossRef][PubMed] 23. O’Rourke, M.F.; Taylor, M.G. Vascular impedance of the femoral bed. Circ. Res. 1966, 18, 126. [CrossRef] 24. Ho, T.K. Random decision forests. In Proceedings of the 3rd International Conference on Document Analysis and Recognition, Montreal, QC, Canada, 14–16 August 1995; pp. 278–282. [CrossRef] 25. Tang, A.; Chung, J.; Wong, T. Digitalizing Traditional Chinese Medicine Pulse Diagnosis with Artificial Neural Network. Telemed. J. E-Health 2011, 18, 446–453. [CrossRef][PubMed] 26. Tang, A.C. Review of Traditional Chinese Medicine Pulse Diagnosis Quantification. Complement. Ther. Contemp. Healthc. 2012, 10, 5044–5772. 27. Balouji, E.; Gu, I.Y.H.; Bollen, M.H.J.; Bagheri, A.; Nazari, M. A LSTM-based deep learning method with application to voltage dip classification. In Proceedings of the 2018 18th International Conference on Harmonics and Quality of Power (ICHQP), Ljubljana, Slovenija, 13–16 May 2018; pp. 1–5. [CrossRef] 28. Suh, H.; Seo, S.; Kim, Y.H. Deep Learning-Based Hazardous Sound Classification for the Hard of Hearing and Deaf. In Proceedings of the 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Louisville, KY, USA, 6–8 December 2018; pp. 073–077. [CrossRef] Sensors 2020, 20, 4618 14 of 14

29. Kiranyaz, S.; Ince, T.; Abdeljaber, O.; Avci, O.; Gabbouj, M. 1-D Convolutional Neural Networks for Signal Processing Applications. In Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12–17 May 2019; pp. 8360–8364. [CrossRef] 30. Kwon, J.; Lee, S.; Kwak, N. Human Detection by Deep Neural Networks Recognizing Micro-Doppler Signals of Radar. In Proceedings of the 2018 15th European Radar Conference (EuRAD), Madrid, Spain, 26–28 September 2018; pp. 198–201. [CrossRef] 31. Pasupa, K.; Sunhem, W. A comparison between shallow and deep architecture classiﬁers on small dataset. In Proceedings of the 2016 8th International Conference on Information Technology and Electrical Engineering (ICITEE), Yogyakarta, Japan, 5–6 October 2016; pp. 1–6. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).