Photoplethysmography Signal Analysis for Optimal Region-Of-Interest Determination in Video Imaging on a Built-In Smartphone Under Different Conditions

sensors

Article Photoplethysmography Signal Analysis for Optimal Region-of-Interest Determination in Video Imaging on a Built-In Smartphone under Different Conditions

Yunyoung Nam 1 ID and Yun-Cheol Nam 2,*

1 Department of Computer Science and Engineering, Soonchunhyang University, Asan 31538, Korea; [email protected] 2 Department of Architecture, Joongbu University, Goyang 10279, Korea * Correspondence: [email protected]; Tel.: +82-31-8075-1622

Received: 23 August 2017; Accepted: 18 October 2017; Published: 19 October 2017

Abstract: Smartphones and tablets are widely used in medical fields, which can improve healthcare and reduce healthcare costs. Many medical applications for smartphones and tablets have already been developed and widely used by both health professionals and patients. Specifically, video recordings of fingertips made using a smartphone camera contain a pulsatile component caused by the cardiac pulse equivalent to that present in a photoplethysmographic signal. By performing peak detection on the pulsatile signal, it is possible to estimate a continuous heart rate and a respiratory rate. To estimate the heart rate and respiratory rate accurately, which pixel regions of the color bands give the most optimal signal quality should be investigated. In this paper, we investigate signal quality to determine the best signal quality by the largest amplitude values for three different smartphones under different conditions. We conducted several experiments to obtain reliable PPG signals and compared the PPG signal strength in the three color bands when the flashlight was both on and off. We also evaluated the intensity changes of PPG signals obtained from the smartphones with motion artifacts and fingertip pressure force. Furthermore, we have compared the PSNR of PPG signals of the full-size images with that of the region of interests (ROIs).

Keywords: healthcare; smartphone; photoplethysmography (PPG); motion artifact; ROI

1. Introduction Mobile healthcare applications using smartphones have been developed exponentially in recent years and are now an essential part of the daily lives of people in various environments. Mobile healthcare applications can collect health-related information using a smartphone via body-worn or implanted sensors, and can deliver health-promoting messages in situations and throughout people’s everyday life [1]. The use of smartphones is attracting more attention in healthcare. Mobile healthcare applications make smartphones useful tools in the practice of evidence-based medicine at the point of care, in addition to their use in mobile clinical communication [2]. In addition, smartphones can play a very important role in patient education, disease self-management, and remote monitoring of patients. Smartphones have fast microprocessors, multiple sensors, large data storage, software ﬂexibility, and media capabilities. In other words, they are suitable for developing monitoring systems that can potentially be used by the general public [3]. They have accordingly been found to be accurate in a variety of vital sign monitoring applications [4–7]. Recently, our research group proposed an approach using dual cameras consisting of contact and noncontact video monitoring directly implemented on a smartphone for estimation of heart rate (HR) and the mean respiratory rate (RR), respectively [8].

Sensors 2017, 17, 2385; doi:10.3390/s17102385 www.mdpi.com/journal/sensors Sensors 2017, 17, 2385 2 of 18

Smartphone devices can not only record PPG signals with a camera and a flashlight, but also serve as an accurate monitor for several physiological variables, based on their ability to record and analyze the varying color signals of a fingertip placed in contact with its optical sensor. However, there are some issues in acquiring PPG signals from a camera on a smartphone [9]. First, different smartphone models lead to different color saturation in the captured frames; second, shifting the finger with respect to the camera lens creates motion artifacts and, as a result, causes wrong segmentation; third, different fingertip pressure force on the camera lens as well as different features of the tissue change the level of light absorption when it passes through the finger, thereby causing different color ratios. Selection of the optimal region of interest (ROI) is one of important issues in the collection of clean PPG signals. The quality of the PPG signals depends on the properties of the skin area, camera characteristics, light emitting diode (LED) usage, and light conditions [9]. Of particular importance is the area of a person’s skin in one or more image frames that should be analyzed, and an ROI should be defined to obtain strong PPG signals. To achieve this goal, Optimal ROI Selector (OptROIS) not only searches for the most stable skin region, since stable regions generally provide the strongest PPG signals, but also selects the optimal ROIs to record clean PPG signals. Thus, OptROIS should analyze spatial pixel uniformity inside the ROI and the strength of a PPG signal. In order to select the location of optimal ROIs that contain strong and clean signals. The obtained images are converted from YUV420sp to RBGA. Then, these sampled signals are filtered with a band-pass filter. After selecting a color channel [6], the signal is interpolated at 30 Hz via a cubic spline algorithm to obtain a fixed sampling rate. Finally, peak-to-peak intervals (PPI) are derived from interpolated signals. In our previous work [4–6], for the video recordings of Samsung Galaxy S3, iPhone, iPad, and iPod, four different pixel region sizes were examined for determining the optimal signal quality. In these works, a larger pixel resolution did not necessarily result in a better signal quality. In fact, in most scenarios, a 50 × 50-pixel resolution was just as good as or better than an HVGA resolution. In addition, the closest region to the flash was in most cases associated with higher signal quality, which is both logical and expected. Finally, using the optimum pixel size, location, and color band of a pulsatile signal (PS), accurate heart rate and respiratory rate can be estimated. Recently, two other methods were developed to improve the reliability. In [10], after binarizing each frame, the radius of the fitting circle was calculated. In [11], a frame adaptive ROI was proposed to handle the color saturation and cut-off distortion caused by the limited dynamic range of smartphone cameras. However, it focused on improvement of the signal quality rather than reducing the extraction time. Both of these methods had a time-varying ROI, which was confusing due to the time-varying intensity of pixels. In this paper, we show experimental results using three different smartphones to obtain reliable PPG signals, and compare the PPG signal strength in three color bands when the flashlight was both on and off. We also evaluated the intensity changes of PPG signals obtained from smartphones with motion artifacts and fingertip pressure force. The rest of this paper is organized as follows. The proposed method is explained in Section2. In Section3, the implementation of the proposed method and its performance are presented. Finally, conclusions are given with discussions in Section4.

2. Method

2.1. Data Collection Data were collected on 30 healthy subjects (18 males, 12 females) with yellow skin color using three different devices: a Samsung Galaxy S6, a HTC One M8, and a Samsung Galaxy Note 3. The room temperature was kept stable at approximately 22 ◦C, so that the subjects were comfortable. Soonchunhyang Hospital’s Institutional Review Board approved the data collection technique. For the pulsatile signal (PS) acquisition, Java was used to develop the Android smartphone on the mobile platform Android 4.1 (Jelly Bean). Speciﬁcally, we used Eclipse Integrated Development Environment (IDE) Indigo R2 as an integrated development environment for development and debugging purposes. Sensors 2017, 17, 2385 3 of 18

For theSensors video 2017,recordings, 17, 2385 we examined 8 × 8 ROIs for determining the optimal signal quality. The3 of 18 PS wassignal obtained quality. by averaging The PS was the obtained entire pixel by averaging size for eachthe entire of the pixel three size color for each bands of (red,the three green, color and bands blue) for every(red, green, frame. and All blue) three for devices every providedframe. All athree sampling devices rate provided close to a 30sampling frames rate per second.close to 30 However, frames whenper the second. video However, sampling when rate was the lowervideo thansampling 30 Hz, rate a cubicwas lower spline than algorithm 30 Hz, a was cubic used spline to interpolatealgorithm the signalwas used to 30 to Hz.interpolate the signal to 30 Hz. NoNo subject subject had had cardiorespiratory cardiorespiratory pathologies. pathologies. All All three three devices devices were were testedtested using the the same same subjects,subjects, at the at the same same location, location, and and under under the the same same test test conditions. conditions. DataData werewere collectedcollected in in the the sitting sitting uprightupright position, position, and and the the sensor sensor was was placed placed in in proximity proximity to to the the subject’ssubject’s left index or or middle middle finger ﬁnger.. FigureFigure1 shows 1 shows data data collection collectio onn theon the three three devices devices by by placing placing a ﬁngertipa fingertip on on the the video video camera. camera.

Camera Flash

(a)

(b) (c) (d)

FigureFigure 1. Smartphones 1. Smartphones used used for for evaluating evaluating performance: performance (: a()a) General General scheme scheme toto acquireacquire videos from from threethree devices; devices (b); Samsung(b) Samsung Galaxy Galaxy S6; S6 (c;) ( HTCc) HTC One One M8; M8 (;d ()d Samsung) Samsung Galaxy Galaxy Note Note 3.3.

Having established that the green color signal provides the best signal amplitude values for smartphonesHaving established [5], we systematically that the green investigated color signal which provides pixel the regions best signal of the amplitudecolor bandsvalues gave the for smartphonesoptimal signal [5], wequality, systematically as determined investigated by the largest which amplitude pixel regionsvalues for of all the three color devices. bands As gave shown the optimalin Table signal 1, 15 quality, different as determinedconditions were by the investigated largest amplitude for comparing values PPG for allsignal three quality. devices. As shown in Table1, 15 different conditions were investigated for comparing PPG signal quality. Table 1. Experimental conditions. Table 1. Experimental conditions. No. Conditions No.1 Red channel, Conditions LED on 2 Green channel, LED on 1 Red channel, LED on 3 Blue channel, LED on 2 Green channel, LED on 34 Red channel Bluewith channel, horizontal LED waving, on LED on 45 Green Red channel withwith horizontalhorizontal waving, waving, LED LED on on 56 Blue Green channel channel with with horizontal horizontal waving, waving, LED onon 67 BlueRed channel withwith horizontal vertical waving, waving, LED LED on on 78 Green Red channel withwith vertical vertical waving, waving, LED LED on on 8 Green channel with vertical waving, LED on 9 Blue channel with vertical waving, LED on 9 Blue channel with vertical waving, LED on 1010 Red Red channel channel with with pressure, LEDLED on on 1111 Green Green channel channel with with pressure, LEDLED on on 1212 Blue Blue channel channel with with pressure, LEDLED on on 1313 Red Red channel, channel, LED offoff 1414 Green Green channel, channel, LED offoff 15 Blue channel, LED off 15 Blue channel, LED off

Sensors 2017, 17, 2385 4 of 18

Thirty seconds of data were collected for each subject. A set of these 15 conditions was repeated five times in the same order. Each 30-s condition was separated by 10-s rest periods. When the LED flash is turned off, we observed that signals are weak and contain more white noise under weak (60 watts, fluorescent lamp, 120–210 lux) light.

2.2. Data Analysis Most smartphone image sensors capture at 30 Hz or 60 Hz. More recently, a new image sensor is capable of 240 fps video capture at full 1080 p resolution. Owing to the frame rate variability, we interpolated the PS to 30 Hz using a cubic spline algorithm followed by peak detection. The peak detection algorithm incorporated a ﬁlter bank with variable cut-off frequencies. To extract the experimental data, we set the window size to 60 samples with 50% overlapping segments between consecutive windows at a 30 Hz sampling frequency. To measure the intensity change of signals obtained from a smartphone camera over time, the change of intensity i∗ is calculated using the following equation:

n ∑j=1 arg max xppg kj − min xppg kj i∗ = k k , (1) n where xppg kj is signal in window j.

3. Experimental Results

3.1. PPG Signal Quality Analysis We investigated the signal quality among the red, green, and blue color bands for each pixel region for all three devices. Figure2a shows the results obtained from the Galaxy S6 using a grid size of 8 × 8. In the figure, the x-axis and y-axis are depicted in grid scale to improve visualization of the intensity (z-axis) of the images when the flashlight was on. Even though the LED flash is placed on the right of the camera, Android device’s orientation is set to landscape mode while acquiring signals. In this case, the right side of the portrait mode ROI is turned to be on top, and the left side is on the bottom. As the ROIs get closer to the flashlight, i∗ of the green and blue color bands gets higher. However, in Figure2a, i∗ of the red color band in the ROIs is 0 when close to the flashlight because the intensity values are always 255, which is the maximum value of intensity. Figure2b shows the intensity of the three color bands over time when the flashlight was on. In the figure, the average values of i∗ of the red, green, and blue color bands are 0, 38.7, and 26.59, respectively. We found that the green color consistently provided significantly higher mean amplitude values than either the blue or the red color. We also found that the region closest to the flash in most cases resulted in a higher signal quality, which is logical and expected. Sensors 2017, 17, 2385 5 of 18 Sensors 2017, 17, 2385 5 of 18

Sensors 2017, 17, 2385 5 of 18

(a) (b) (a) (b) Figure 2. ExampleExample of of quality quality analysis analysis of of PPG PPG signals signals obtained obtained from from a subject a subject using using the theGalaxy Galaxy S6: ( S6:a) Figure 2. Example of quality analysis of PPG signals obtained from a subject using the Galaxy S6: (a) intensity(a) intensity change change distribution distribution;; (b) (intensityb) intensity of PPG of PPG signa signal.l. intensity change distribution; (b) intensity of PPG signal. Figure 3 shows the results obtained from the HTC One M8, when the grid size was 8 × 8. In this FigureFigure3 shows 3 shows the resultsthe results obtained obtained from from the the HTC HTC OneOne M8,M8, when when the the grid grid size size was was 8 × 8 8. ×In 8.this In this case, the LED flash is placed on the left of the camera, and Android device’s orientation is set to case,case, the LEDthe LED flash flash is placedis placed on on the the left left of of thethe camera,camera, and and Android Android device device’s’s orientation orientation is set is to set to landscape mode while acquiring the signals. The right side of the portrait mode ROI is turned to be landscapelandscape mode mode while while acquiring acquiring the the signals. signals. The The right right side side of of thethe portraitportrait mode mode ROI ROI is is turned turned to to be be on on top,on andtop, theand leftthe leftside side is on is onthe the bottom. bottom. As As the the ROIsROIs also got got closer closer to to the the flashlight, flashlight, 푖∗ of푖∗ the of green the green top, and the left side is on the bottom. As the ROIs also got closer to the flashlight, i∗ of the green and and blueand bluecolor color bands bands also also got got higher. higher. However, However, inin Figure 3a 3a, , 푖∗푖 ∗of of the the red red color color band band in ROIs in ROIs when when blue color bands also got higher. However, in Figure3a, i∗ of the red color band in ROIs when close close closeto the to flashlightthe flashlight is 0is because0 because most most intensity intensity values areare 255, 255, which which is theis the maximum maximum value value of of to the flashlight is 0 because most intensity values are 255, which is the maximum value of intensity. intensity.intensity. Figure Figure 3b shows 3b shows the the intensity intensity of of the the three three color bandsbands when when the the flashlight flashlight was was on. Ason. shown As shown Figure3b shows the intensity of the* three color bands when the flashlight was on. As shown in the in thein figure, the figure, average average values values of ofi* ofi of the the red, red, green,green, and blueblue color color bands bands are are 1.77, 1.77, 52.75, 52.75, and and24.69, 24.69, figure,respectively. average values of i* of the red, green, and blue color bands are 1.77, 52.75, and 24.69, respectively. respectively.

(a) (b)

Figure 3. Example of quality analysis of PPG signals obtained from a subject using the HTC One M8: Figure 3. Example of quality analysis of PPG signals obtained from a subject using the HTC One M8: (a) intensity change distribution; (b) intensity of PPG signal. (a) intensity change distribution;(a) (b) intensity of PPG signal. (b) Figure 3. Example of quality analysis of PPG signals obtained from a subject using the HTC One M8: ( a) intensity change distribution; (b) intensity of PPG signal.

SensorsSensors2017, 201717, 2385, 17, 2385 66 of of 18 18

Figure 4 shows the results obtained from the Galaxy Note 3 when the grid size is 8 × 8. In this

case,FigureSensors the4 2017 LEDshows, 17 ,flash 2385 the is results placed obtained on the bottom from of the the Galaxy camera, Note and 3 Android when the device grid’s size orientation is 8 6× of8. 18 is In set this to case,landscape the LED flashmode is while placed acquiring on the bottomthe signals. of the The camera, right side and of Android the portrait device’s mode orientationROI is turned is setto be to landscapeon top, mode Figureand the while4 shows left acquiringside the isresults on thethe obtained signals.bottom. from As The the ROIs right Galaxy got side Notecloser of 3 the whento portraitthe the flashlight, grid mode size iis ROI* of8 × green is8. In turned this and toblue be case, the LED flash is placed on the bottom of the camera,∗ and Android device’s orientation is set to on top,color and bands the leftalso sidegot higher. is on the However, bottom. in As Figure ROIs 4a, got 푖 closer of the to red the color flashlight, band ini *ROIsof green when and close blue to thelandscape flashlight mode is 0 while because acquiring the intensity the signals. values The right are∗ side always of the 255, portrait which mode is ROI the maximumis turned to be value of color bandson top, also and got the higher.left side However,is on the bottom. in Figure As ROIs4a, i gotof closer the red to the color flashlight, band in i* ROIsof green when and closeblue to the intensity. Figure 4b shows the intensity of the three color bands when the flashlight was on. In the flashlightcolor is bands 0 because also got the higher. intensity However, values in areFigure always 4a, 푖∗255, of the which red color is the band maximum in ROIs when value close of intensity.to figure, average values of 푖∗ of the red, green, and blue color bands are 0, 66.38, and 0.34, respectively. Figure4theb shows flashlight the intensity is 0 because of the the three intensity color values bands are when always the 255, flashlight which is was theon. maximum In the figure, value of average Figure 5 shows boxplots with the median calculated by the average values of 푖∗ of the three values ofintensityi∗ of the. Figure red, 4b green, shows and the blueintensity color of bandsthe three are color 0, 66.38, bands andwhen 0.34, the respectively.flashlight was on. In the color bands obtained from∗ the 64 ROIs of the Galaxy S6 when the flashlight was on. The lower Figurefigure,5 shows average boxplots values of with푖 of the the red, median green, calculated and blue color by bands the average are 0, 66.38, values and of0.34,i∗ respectively.of the three color boundaryFigure of the5 shows box indicatesboxplots withthe 25ththe median percentile, calculated a line by within the average the box values marks of the푖∗ of median, the three and the bands obtained from the 64 ROIs of the Galaxy S6 when the flashlight was on. The lower boundary of uppercolor boundary bands obtained of the frombox indicates the 64 ROIs the of75th the percen Galaxytile. S6 whenWhiskers the flashlight(error bars) was above on. The and lower below the the box indicates the 25th percentile, a line within the box marks the median, and the upper boundary boxboundary indicate ofthe the 90th box and indicates 10th percentiles.the 25th percentile, Therefore, a line the within area the of boxthe marksblue box the ismedian, an indication and the of the of thespread. boxupper indicates Red boundary crosses the of represent 75ththe box percentile. indicates the outliers. the Whiskers 75th In percen the (error casetile. of Whiskers bars) the Galaxy above (error S6 and bars), harmonics below abovethe and of box belowgreen indicate the and blue the 90thco andlorbox10th bands indicate percentiles. provide the 90th a Therefore,strongerand 10th percentiles.PPG the signal area Therefore, of than the the blue othersthe box areais while of an the indication most blue boxROIs is of anof the aindication red spread. color of Red band the crosses have spread. Red crosses represent the outliers. In the case of the Galaxy S6, harmonics of green and blue representlow 푖 the∗ values outliers.. In the In thecase case of the of Galaxy the Galaxy Note S6,3, 1 harmonics–3 or 6–8 ROIs of green and their and harmonics blue color of bands all color provide bands a color bands provide a stronger PPG signal than the others while most ROIs of a red color band∗ have strongerprovide PPG strong signal PPG than signals. the others The average while most values ROIs of of푖∗ awere red colorfound bandto be havehighest low ini bluevalues. and Ingreen the low 푖∗ values. In the case of the Galaxy Note 3, 1–3 or 6–8 ROIs and their harmonics of all color bands casecolor of the bands Galaxy as Noteshown 3, in 1–3 Table or 6–8 2. ROIs and their harmonics∗ of all color bands provide strong PPG provide strong PPG signals.∗ The average values of 푖 were found to be highest in blue and green signals.color The bands average as shown values in of Tablei were 2. found to be highest in blue and green color bands as shown in Table2.

(a) (b) (a) (b) Figure 4. Example of quality analysis of PPG signals obtained from a subject using the Galaxy Note Figure 4. Example of quality analysis of PPG signals obtained from a subject using the Galaxy Note 3: Figure3: (a) 4intensity. Example change of quality distribution analysis; (b) intensityof PPG signalsof PPG signalobtained. from a subject using the Galaxy Note (a) intensity3: (a) intensity change change distribution; distribution (b) intensity; (b) intensity of PPG of PPG signal. signal.

(a) (d)

Figure 5. Cont. Sensors 2017, 17, 2385 7 of 18 Sensors 2017, 17, 2385 7 of 18

(b) (e)

푖∗ FigureFigure 5. 5Box. Box plot plot of of the the average average valuesvalues of i∗ ofof thethe threethree color bands bands obtained obtained from from the the 64 64 ROIs ROIs of of the Galaxy S6 and the Galaxy Note 3 from 30 subjects when the flashlight was on: (a) Red (Galaxy S6); the Galaxy S6 and the Galaxy Note 3 from 30 subjects when the ﬂashlight was on: (a) Red (Galaxy S6); (b) Green (Galaxy Note 3); (c) Blue (Galaxy Note 3); (d) Red (Galaxy Note 3); (e) Green (Galaxy Note (b) Green (Galaxy Note 3); (c) Blue (Galaxy Note 3); (d) Red (Galaxy Note 3); (e) Green (Galaxy Note 3); 3); (f) Blue (Galaxy Note 3). (f) Blue (Galaxy Note 3).

Table 2. The average values of 푖∗ of the three color bands obtained from 64 ROIs when the flashlight Table 2. The average values of i∗ of the three color bands obtained from 64 ROIs when the flashlight was on. was on. Color Band Galaxy S6 HTC One M8 Galaxy Note 3 Color BandRed Galaxy2.77 ± 9.96 S6 18.7 HTC ± 27.12 One M84.11 Galaxy ± 11.46 Note 3 RedGreen 26.39 2.77 ± ± 9.9620.62 35.05 18.7 ± ±19.4727.12 17.74 4.11 ± 14.83± 11.46 GreenBlue 26.3941.07± ± 20.6221.34 7.51 35.05 ± ±7.419.47 1.97 17.74 ± 2.87± 14.83 Blue 41.07 ± 21.34 7.51 ± 7.4 1.97 ± 2.87 3.2. PPG Signal Quality Analysis with Motion Artifacts 3.2. PPGMotion Signal artifacts Quality are Analysis known with to distort Motion PPG Artifacts recordings, causing erroneous estimation of HR and respiratory rate. There are three distinct sources of artifacts that can impair PPG recordings: Motion artifacts are known to distort PPG recordings, causing erroneous estimation of HR environmental, physiological, and experimental artifacts, which are due to power interference and respiratory rate. There are three distinct sources of artifacts that can impair PPG recordings: surrounding the body, other physiological signals, and experimental conditions. We investigated the environmental, physiological, and experimental artifacts, which are due to power interference signal quality among the red, green, and blue color bands for each of the pixel regions for all three surrounding the body, other physiological signals, and experimental conditions. We investigated the devices with hand shaking. The participants sat in a chair with their left hand on a desk, with the signalsensors quality and amonginstruments thered, required green, for and conventional blue color laboratory bands for eachmeasurements of the pixel attached regions to forthe allbody three devicesand left with index hand finger. shaking. Then, The the participantssubjects were sat asked in a to chair hold with the theirsmartphones left hand in ontheir a desk, right withhand, the sensorskeeping and their instruments right index required finger on for the conventional camera with the laboratory front of the measurements screen vertical attached on a firm to cushion the body andon left their index knee. finger. The subjects Then, were the subjectsasked to weregrip the asked smartphone to hold thesufficiently smartphones firmly into ensure their right that hand,the keepingfinger- theirsmartphone right index contact finger was onconstant the camera despite with the theexistence front of motion the screen [11] vertical. on a firm cushion on theirAfter knee. baseline The subjects recording were for asked1 minute to gripwithout the smartphoneany movement, sufficiently motion artifacts firmly were to ensure induced that by the finger-smartphonemovement of the contact subject’ wass hand constant in both despite the horizontal the existence and of the motion vertical [11 directions.]. Subjects were instructedAfter baseline to introduce recording motion for artifacts 1 minute horizontally without any and movement, vertically for motion specific artifacts time intervals. were induced For byexample, movement if a ofsubject the subject’swas instructed hand to in perform both the left horizontal-right movements and the at vertical2 Hz or 3 directions. Hz to distinguish Subjects were instructed to introduce motion artifacts horizontally and vertically for specific time intervals.

Sensors 2017, 17, 2385 8 of 18

Sensors 2017, 17, 2385 8 of 18 For example, if a subject was instructed to perform left-right movements at 2 Hz or 3 Hz to distinguish betweenbetween hearhear raterate andand motionmotion frequency.frequency. WeWe observedobserved thatthat thethe spectrumspectrum byby motionmotion artifactsartifacts waswas betweenbetween 22 HzHz andand 33 Hz.Hz. TheThe shakingshaking intensityintensity waswas suchsuch thatthat visiblevisible noisenoise waswas seenseen onon thethe recordedrecorded PPGPPG waveform.waveform. InIn thisthis period,period, therethere werewere twotwo motionmotion artifactartifact conditions:conditions: horizontalhorizontal motionmotion artifactartifact (HMA)(HMA) andand verticalvertical motion artifact artifact (VMA), (VMA), and and one one baseline baseline (BL) (BL) condition. condition. In Inthe the HMA HMA and and the theVMA, VMA, the theexperimenter experimenter shook shook the thecushion cushion horizontally horizontally and and vertically, vertically, respectively. respectively. In the In theBL, BL,the theexperimenter experimenter simply simply held held the the cushion cushion motionless. motionless. The The experiments experiments of of the the two motion motion artifact artifact conditionsconditions werewere conductedconducted onon thethe three three smartphones. smartphones. FigureFigure6 6shows shows the the results results obtained obtained from from the the Galaxy Galaxy S6 S6 while while shaking shaking the the cushion cushion horizontally horizontally whenwhen thethe flashlightflashlight waswas on.on. InIn thethe figure,figure, thethe signalsignal isis unstableunstable underunder thethe influenceinfluence ofof thethe motionmotion artifact.artifact. WeWe measuredmeasured thethe red,red, green,green, andand blueblue lightlight PPGsPPGs usingusing aa smartphonesmartphone whilewhile addingadding motionmotion * artifacts.artifacts. EvenEven thoughthough aa motionmotion artifactartifact cancan affectaffect thethe signal,signal, asas ROIsROIs gotgot closercloser toto thethe flashlight,flashlight,i i* ofof ∗ thethe greengreen andand blueblue colorcolor bandsbands gotgot higher. As shown in Figure 55bb,, i푖∗ ofof the the red, red, green, green, andand blueblue colorcolor bandsbands werewere 0,0, 77.19,77.19, andand 67.39,67.39, respectively.respectively.Even Eventhough though thethe PPGPPG signalssignals werewere slightslight affectedaffected byby thethe motionmotion artifact,artifact, thethe heartheart raterate cancan bebe measuredmeasured fromfrom thethe signalssignals ofof thethe greengreen andand blueblue colorcolor bands.bands. WeWe observed observed it it in in other other smartphone smartphone models. models.

(a) (b)

FigureFigure6. 6. ExampleExample ofof qualityquality analysisanalysis ofof PPGPPG signalssignals obtainedobtained fromfrom aa subjectsubject usingusing thethe GalaxyGalaxy S6S6 withwith horizontalhorizontal movements:movements: ((aa)) intensityintensity changechange distribution;distribution; ( b(b)) intensity intensity of of PPG PPG signal. signal.

Figure 7 shows the results obtained from the Galaxy S6 while shaking the cushion vertically Figure7 shows the results obtained from the Galaxy S6 while shaking the cushion vertically when when the flashlight was on. In the figure, the signals of the green and blue color bands were stable the flashlight was on. In the figure, the signals of the green and blue color bands were stable under under the influence of the motion artifact. Because the signal of the blue color band is sometimes the influence of the motion artifact. Because the signal of the blue color band is sometimes unstable, unstable, the accuracy of the heart rate estimated from the signal of the green color band is higher the accuracy of the heart rate estimated from the signal of the green color band is higher than that than that from the signal of the blue color band. In addition, after reconstructing the MNA from the signal of the blue color band. In addition, after reconstructing the MNA contaminated data contaminated data segments, accurate heart rates and respiratory rates can be measured [6]. We segments, accurate heart rates and respiratory rates can be measured [6]. We observed it in other observed it in other smartphone models. smartphone models. Figures 8 and 9 show boxplots with the median calculated by the average values of 푖∗ of the Figures8 and9 show boxplots with the median calculated by the average values of i∗ of the three three color bands obtained from the 64 ROIs of the Galaxy S6 and the Galaxy Note 3 while shaking color bands obtained from the 64 ROIs of the Galaxy S6 and the Galaxy Note 3 while shaking the the cushion horizontally and vertically, respectively, when the flashlight was on. The boxplots of cushion horizontally and vertically, respectively, when the flashlight was on. The boxplots of Figures8 Figures 8 and 9 are similar to those of Figure 5. The average values of 푖∗ was found to be highest in and9 are similar to those of Figure5. The average values of i∗ was found to be highest in blue and blue and green color bands as shown in Tables 3. green color bands as shown in Table3.

Sensors 2017, 17, 2385 9 of 18 Sensors 2017, 17, 2385 9 of 18

Sensors 2017, 17, 2385 9 of 18

(a) (b) (a) (b) Figure 7. Example of quality analysis of PPG signals obtained from a subject using the Galaxy S6 with FigureFigure 7. 7Example. Example of of quality quality analysis analysis ofof PPG signals obtained obtained from from a asubject subject using using the the Galaxy Galaxy S6 S6with with vertical movements: (a) intensity change distribution; (b) intensity of PPG signal. verticalvertical movements: movements: (a ()a) intensity intensity change change distribution;distribution; ( (b)) intensity intensity of of PPG PPG signa signal.l.

(a()a ) (d(d) )

(b) (e) (b) (e) Figure 8. Cont.

Sensors 2017, 17, 2385 10 of 18 Sensors 2017, 17, 2385 10 of 18 Sensors 2017, 17, 2385 10 of 18

(c) (f) (c) (f) Figure 8. Box plot of the average values of 푖∗∗ of the three color bands obtained from the 64 ROIs of FigureFigure 8. 8Box. Box plot plot of of thethe averageaverage values of i푖∗ ofof thethe threethree color bands bands obtained obtained from from the the 64 64 ROIs ROIs of of the Galaxy S6 and the Galaxy Note 3 from 30 subjects while shaking the cushion horizontally: (a) Red thethe Galaxy Galaxy S6 S6 and and the the Galaxy Galaxy NoteNote 33 fromfrom 3030 subjects while shaking shaking the the cushion cushion horizontally horizontally:: (a ()a Red) Red (Galaxy S6); (b) Green (Galaxy S6); (c) Blue (Galaxy S6); (d) Red (Galaxy Note 3); (e) Green (Galaxy (Galaxy(Galaxy S6); S6) (;b ()b) Green Green (Galaxy (Galaxy S6);S6); ((cc)) BlueBlue (Galaxy(Galaxy S6 S6);); ( (dd)) Red Red ( (GalaxyGalaxy Note Note 3) 3);; (e ()e Green) Green (Galaxy (Galaxy Note 3); (f) Blue (Galaxy Note 3). NoteNote 3); 3) (;f )(f Blue) Blue (Galaxy (Galaxy Note Note 3). 3).

(a) (d) (a) (d)

(b) (e) (b) (e)

Figure 9. Cont.

Sensors 2017, 17, 2385 11 of 18 Sensors 2017, 17, 2385 11 of 18

Figure 9. Box plot of the average values of 푖∗ of the three color bands obtained from the 64 ROIs of Figure 9. Box plot of the average values of i∗ of the three color bands obtained from the 64 ROIs of the the Galaxy S6 and the Galaxy Note 3 from 30 subjects while shaking the cushion vertically: (a) Red Galaxy S6 and the Galaxy Note 3 from 30 subjects while shaking the cushion vertically: (a) Red (Galaxy (Galaxy S6); (b) Green (Galaxy S6); (c) Blue (Galaxy S6); (d) Red (Galaxy Note 3); (e) Green (Galaxy S6); (b) Green (Galaxy S6); (c) Blue (Galaxy S6); (d) Red (Galaxy Note 3); (e) Green (Galaxy Note 3); Note 3); (f) Blue (Galaxy Note 3). (f) Blue (Galaxy Note 3).

Table 3. The average values of 푖∗ of the three color bands obtained from 64 ROIs while shaking the Table 3. The average values of i∗ of the three color bands obtained from 64 ROIs while shaking the cushion horizontally and vertically when the flashlight was on. cushion horizontally and vertically when the flashlight was on. Motion Artifacts Color band Galaxy S6 HTC One M8 Galaxy Note 3 Motion Artifacts ColorRed band Galaxy2.97 ± 9.4 S6 13.42 HTC ± One 21.14 M8 0.86 Galaxy ± 2.03 Note 3 HMA RedGreen 36.57 2.97 ±± 69.089.4 61.07 13.42 ± ±32.3821.14 24.89 0.86 ± ±15.592.03 HMA GreenBlue 67.21 36.57 ±± 35.2169.08 7.92 61.07 ± ±9.0732.38 5.2 24.89 ± 35.32± 15.59 BlueRed 67.212.66 ±± 8.5235.21 8.06 7.92 ± 15.08± 9.07 2.1 5.2 ±± 6.7535.32 VMA RedGreen 65.91 2.66 ±± 21.898.52 49.78 8.06 ±± 28.6215.08 29.91 2.1 ±± 19.76.75 VMA GreenBlue 71.74 65.91 ±± 27.8321.89 8.58 49.78 ± ±8.6728.62 7.61 29.91 ± 27.95± 19.7 Blue 71.74 ± 27.83 8.58 ± 8.67 7.61 ± 27.95 3.3. PPG Signal Quality Analysis with Fingertip Pressure Force 3.3. PPGAll Signal subjects Quality were Analysis instructed with to Fingertip put the Pressureindex fingertip Force on the smartphone camera lens with differentAll subjects contacting were forces instructed over a torange put between the index 0.2 fingertip N and 1.8 on N the. This smartphone modification camera was made lens within differentconsideration contacting of the forces nature over of the a present range between experiment, 0.2 Nthat and is, 1.8adding N. Thismotion modification artifact [12] was. However, made in considerationit is possible that of the the nature finger of-phone the present contact experiment,pressure was that increased, is, adding which motion could artifact in turn [cause12]. However, a local itdecrease is possible in thatarteriolar the finger-phone pressure. In this contact case, pressure it was observed was increased, that the pulse which wave could disappears in turn cause when a the local decreasefinger isin pressed arteriolar very pressure.firmly against In this the case,smartphone it was observedimage sensor. that Fi thegures pulse 10 and wave 11 show disappears the results when theof fingerall color is pressedbands with very a firmlyfingertip against pressure the smartphoneforce on the camera image sensor.lens. From Figures these 10 figures, and 11 HRshow and the respiratory rate cannot estimated from these signals. results of all color bands with a fingertip pressure force on the camera lens. From these figures, HR and respiratory rate cannot estimated from these signals.

Sensors 2017, 17, 2385 12 of 18 Sensors 2017, 17, 2385 12 of 18

Sensors 2017, 17, 2385 12 of 18

FigureFigure 10.10. Intensity 10. Intensity change change distribution distribution of of of PPG PPGPPG signals signals withwith with a a afingertip fingertip ﬁngertip pressure pressure pressure force force force:: (a) :Galaxy ( (aa)) Galaxy Galaxy S6; S6; S6; (b) HTC(b) HTC One One M8; M8; (c) GalaxyGalaxy(c) Galaxy NoteNote Note 3.3 .3 .

FigureFigure 11. PPG 11. PPG signals signals with with a ﬁngertipa fingertip pressure pressure force force:: (a (a) )Signal Signal of ofthe the red red color color band band of Galaxy of Galaxy S6; S6; (b) Signal of the green color band of Galaxy S6; (c) Signal of the blue color band of Galaxy S6; (d) (b) Signal of the green color band of Galaxy S6; (c) Signal of the blue color band of Galaxy S6; (d) Signal Signal of the red color band of HTC One M8; (e) Signal of the green color band of HTC One M8; (f) Figureof the red 11. colorPPG bandsignals of with HTC a One fingertip M8; (e pressure) Signal offorce the: green (a) Signal color of band the ofred HTC color One band M8; of ( fGalaxy) Signal S6; of Signal of the blue color band of HTC One M8; (g) Signal of the red color band of Galaxy Note 3; (h) (b) Signal of the green color band ofg Galaxy S6; (c) Signal of the blue color band of hGalaxy S6; (d) the blueSignal color of the band green of HTCcolor Oneband M8;of Galaxy ( ) Signal Note of3; ( thei) Signal red color of the band blue ofcolor Galaxy band Noteof Galaxy 3; ( Note) Signal 3. of the Signalgreen colorof the band red color of Galaxy band Note of HTC 3; (i )One Signal M8; of (e the) Signal blue colorof the band green of color Galaxy band Note of 3.HTC One M8; (f) Signal3.4. PPG of Signalthe blue Qua colorlity Analysisband of WhenHTC Onethe Flashlight M8; (g) Signalis Off of the red color band of Galaxy Note 3; (h) Signal of the green color band of Galaxy Note 3; (i) Signal of the blue color band of Galaxy Note 3. 3.4. PPG SignalWhen Qualitythe LED Analysis flash is off, When a change the Flashlight of intensity is Off of the signals of the three color bands was observed in most smartphone models except the Galaxy Note 3. This change of intensity depended 3.4. PPGWhen Signal the Qua LEDlity ﬂash Analysis is off, When a change the Flashlight of intensity is Off of the signals of the three color bands was observed in most smartphone models except the Galaxy Note 3. This change of intensity depended When the LED flash is off, a change of intensity of the signals of the three color bands was observed in most smartphone models except the Galaxy Note 3. This change of intensity depended

Sensors 2017, 17, 2385 13 of 18 SensorsSensors 2017 2017, 17, 17, 2385, 2385 1313 of of 18 18 ononon thethe the typetype type ofof of smartphone.smartphone. smartphone. Fig FiguresFigureures s12 1212–––114144 show showshow the the results results obtained obtained obtained from from from all all all smartphones smartphones smartphones when when when the the the flashlightflashlightflashlight waswas was off.off. off. From From the thethe figure figures,figures,s ,the thethe change change of of intensity intensity was was was observed observed observed mainly mainly mainly in in inthe the the center center center of of of images.images.images. InIn In thethe the casecase case of of the the the Galaxy Galaxy Galaxy S6, S6,S6, the thethe intensities intensities in in in the the the signals signals signals of of ofall all all of of ofthe the the color color color bands bands bands were were were strongly strongly strongly detecteddetecteddetected withoutwithout without the the flashlightflashlight flashlight of ofof the thethe smartphone. smartphone.smartphone. In In the the the case case case of of of the the the HTC HTC HTC One One One M8, M8, M8, the the the heart heart heart rate rate rate can can can bebebe measured measured measured onlyonly only from from the the signals signals of ofof the thethe green greengreen or or red red color color band band band while while while the the the heart heart heart rate rate rate can can can be be measured be measured measured fromfromfrom only only only thethe the signalssignals signals of of theth the ered red color color band bandband in inin the thethe case casecase of of Galaxy Galaxy Note Note Note 3. 3. 3.

(a()a ) (b(b) )

FigureFigureFigure 12.12 12. Example.Example Example ofof of qualityquality quality analysisanalysis analysis ofof of PPG PPG PPG signals signals signals obtained obtained obtained from from from a a subject asubject subject using using using the the the Galaxy Galaxy Galaxy S6 S6 withS6 ﬂashwithwith off:flash flash (a off) off intensity: (: a()a )intensity intensity change change change distribution; distribution distribution (b);intensity (; b(b) )intensity intensity of PPG of of P PPG signal.PG signal signal. .

(a()a ) (b(b) ) Figure 13. Example of quality analysis of PPG signals obtained from a subject using HTC One M8 FigureFigure 13.13. ExampleExample ofof qualityquality analysisanalysis ofof PPG PPG signals signals obtained obtained from from a a subject subject using using HTC HTC One One M8 M8 with withwith the the flash flash off: off: ( a()a )intensity intensity change change distribution distribution; (; b(b) )intensity intensity of of PPG PPG signal signal. . the ﬂash off: (a) intensity change distribution; (b) intensity of PPG signal.

Sensors 2017, 17, 2385 14 of 18

Sensors 2017, 17, 2385 14 of 18 Sensors 2017, 17, 2385 14 of 18

(a) (b) (a) (b) Figure 14. Example of quality analysis of PPG signals obtained from a subject using the Galaxy Note Figure 14. Example of quality analysis of PPG signals obtained from a subject using the Galaxy Note 3Figure with flash 14. Example off: (a) intensity of quality change analysis distribution of PPG signals; (b) intensity obtained of from PPG a subjectsignal. using the Galaxy Note 3 3 with flash off: (a) intensity change distribution; (b) intensity of PPG signal. with ﬂash off: (a) intensity change distribution; (b) intensity of PPG signal. ∗ FigureFigure 15 shows 15 shows boxplots boxplots with with the the median median calculated by by the the average average values values of 푖of∗ of푖 the of three the three colorcolorFigure bands bands 15 obtained shows obtained boxplotsfrom from the the with 64 64 ROIs theROIs median of of thethe Galaxy calculated S6S6 and and by the the Galaxy average Galaxy Note values Note 3 when 3 of wheni∗ theof theflashlightthe threeflashlight color ∗ wasbands offwas obtained. The off. Theaverage fromaverage values the values 64 ROIsof of푖 푖 of∗was was the found Galaxyfound toto S6be andhighesthighest the in Galaxyin red red and and Note green green 3 whencolor color bands the bands ﬂashlight as shown as shown wasin off. in TheTable averageTable 4. 4. values of i∗ was found to be highest in red and green color bands as shown in Table4.

(a) (d)

(b) (e)

Figure 15. Cont.

Sensors 2017, 17, 2385 15 of 18 Sensors 2017, 17, 2385 15 of 18

∗ Figure 1 15.5. Box plot of the average valuesvalues ofofi ∗푖 of of the the three three color color bands bands obtained obtained from from the the 64 ROIs64 ROIs of the of theGalaxy Galaxy S6 and S6 and the Galaxythe Galaxy Note Note 3 from 3 from 30 subjects 30 subjects with ﬂash with off: flash (a) off: Red (a (Galaxy) Red (Galaxy S6); (b) S6 Green); (b) (Galaxy Green (S6);Galaxy (c) Blue S6); (Galaxy(c) Blue ( S6);Galaxy (d) RedS6); (Galaxy(d) Red ( NoteGalaxy 3); Note (e) Green 3); (e (Galaxy) Green ( NoteGalaxy 3); Note (f) Blue 3); (Galaxy(f) Blue ( NoteGalaxy 3). Note 3). Table 4. The average values of i∗ of the three color bands obtained from 64 ROIs when the ﬂashlight ∗ Tablewas on. 4. The average values of 푖 of the three color bands obtained from 64 ROIs when the flashlight was on.

Color BandBand Galaxy Galaxy S6 S6 HTC HTC One One M8 M8 Galaxy Galaxy Note Note 3 3 RedRed 34 34 ±± 0.820.82 5.19 5.19 ± 0.25± 0.25 22.85 22.85 ± ±0.110.11 ± ± ± GreenGreen 20.3 20.3 ± 32.3732.37 16.1 16.1 ± 0.180.18 6.79 6.79 ± 0.150.15 Blue 2.87 ± 9.93 11.2 ± 0 9.3 ± 0 Blue 2.87 ± 9.93 11.2 ± 0 9.3 ± 0

3.5. PPG Signal Quality Analysis of the ROI 3.5. PPG Signal Quality Analysis of the ROI When acquiring PPG signals from a smartphone camera, instead of a series of full-size images, this When acquiring PPG signals from a smartphone camera, instead of a series of full-size images, downsized image, namely, the ROI, requires less computational resources and produces more reliable this downsized image, namely, the ROI, requires less computational resources and produces more PPG signals [13,14]. Proper selection of the ROI is one of important steps to collect reliable signals. reliable PPG signals [13,14]. Proper selection of the ROI is one of important steps to collect reliable There are two signiﬁcant advantages to employing an optimized ROI. One is that the computing load signals. There are two significant advantages to employing an optimized ROI. One is that the is decreased by reducing the image size used to extract the PPG signal. The other is that stronger and computing load is decreased by reducing the image size used to extract the PPG signal. The other is more robust PPG signals are obtained by removing the noisy regions. In the experiments, the PPG that stronger and more robust PPG signals are obtained by removing the noisy regions. In the signal of full-size images was compared with that of ROIs. The ROIs were selected by the optimal experiments, the PPG signal of full-size images was compared with that of ROIs. The ROIs were ROI determination method [15], which recursively splits regions to locate the region that produces the selected by the optimal ROI determination method [15], which recursively splits regions to locate the regionstrongest that PPG produces signal. the Figure strongest 16 shows PPG the signal. intensity Figure changes 16 shows of the the full-size intensity images changes and of selected the full ROIs-size imagesobtained and from selected the Galaxy ROIs S6.obtained When from the PPG the signalsGalaxy ofS6. the When full-size the PPG images signals and theof the selected full-size ROIs images were i∗ i∗ andobtained, the selected average ROIs values were of obtained,were 22.36average and values 32.49, of respectively. 푖∗ were 22.36 In and other 32.49, words, respectively.of the selectedIn other i∗ words,ROIs is 푖 45.3%∗ of the higher selected than ROIs that is of 45.3% full-size higher images. than Figurethat of 17full shows-size imagesand. theFigure peak 17 signal-to-noiseshows 푖∗ and i∗ theratio peak (PSNR). signal In-to the-noise ﬁgure, ratio the (PSNR). wider In the the ROI, figure, the lowerthe widergets. the TheROI, wider the lower the ROI,푖∗ gets. the higherThe wider the thePSNR ROI, gets. the In higher other words,the PSNR while gets. intensity In other of thewords, selected while ROIs intensi is strongerty of the than selected that of ROIs full-size is stronger images, thanPPG signalsthat of full obtained-size images, from the PPG selected signals ROIs obtained can be from more the affected selected by motionROIs can artifacts. be more affected by motion artifacts.

Sensors 2017, 17, 2385 16 of 18 Sensors 2017, 17, 2385 16 of 18 Sensors 2017, 17, 2385 16 of 18

FigureFigure 1 16.6 1.6 GalaxyGalaxy. Galaxy S6 S6 S6 with with LED LED On:On: (( aa)) all all regions regions;;; (( (bb)) )selected selected selected dominant dominant dominant region region region.. .

Figure 17. PSNR and i* with ROIs. Figure 1 17.7. PSNR and i* with ROIs ROIs.. 4. Conclusions and Discussion 4. Conclusion ConclusionsWe haves and investigated Discussion signal quality among the red, green, and blue color bands to determine theWe best have signal investigated quality by the signal largest quality amplitude among values the red,for all green, three devices and blue under color different bands conditions to determine. theWe best best have signal signal evaluated quality quality theby by thequality the largest largest of PPG amplitude amplitude signals values valuesobtained for for from all all three three smartphones devices devices under underwhen different the different flashlight conditions conditions. was . on and off. We found that the region closest to the flashlight in most cases resulted in a higher signal We havehave evaluatedevaluated the quality of PPG signals obtained from smartphones when the flashlight flashlight was quality, which is logical and expected. When the LED flash was off, only the signal of the red color on and off. We We found that the region closest to the flashlight flashlight in most cases resulted in a higher signal quality, which is logical and expected. When the the LED flash flash was off, only the signal of the red color

Sensors 2017, 17, 2385 17 of 18 band in all three devices was observed as having a change of intensity, while signals of the green and blue color bands in the Galaxy Note 3 were not observed as having a change of intensity. The intensity change of a green color band is strongest in the HTC One M8 and the Galaxy Note 3. However, intensity change of a blue color band is strongest in the Galaxy S6. Each smartphone has different image sensors. The Samsung Galaxy S6 has the 16-megapixel Sony Exmor RS IMX240 sensor, which is the same sensor used in the Galaxy Note 4. The HTC One M8 has 4-megapixel OmniVision OV4688 which is a 1/3-inch CMOS sensor. The camera of the Galaxy Note 3 is a Sony IMX135 Exmor RS of 12.8 megapixels. In darker environments, images obtained from the IMX240 sensor of the Galaxy S6 can contain a significant blue tint, which results in strong intensity changes of the blue color band. Even though strong intensity changes of the blue color band were observed in the Galaxy S6, the signals of the green color band obtained from all three devices were stable under conditions except fingertip pressure force conditions. We also evaluated the quality of the PPG signals obtained from the smartphones with motion artifacts and fingertip pressure force. Even though the signals of all color bands were unstable under the influence of a motion artifact, heart rate can still be estimated using several bandpass filters to cut out a specified frequency band. In addition, after reconstructing the MNA contaminated data segments, accurate heart rates and respiratory rates can be estimated. Furthermore, we have compared not only the intensity changes of the PPG signals of the full-size images with those of the ROIs, but also the PSNR of PPG signals of the full-size images with that of the ROIs. While the intensity changes of the full-size images were lower than those of selected ROIs, the PSNR of the full-size images was higher than that of the selected ROIs. We hope that our results will help to achieve robust measurements especially in ambulatory settings, where motion-induced artifacts are likely to be a significant issue.

Acknowledgments: This work was supported by the Soonchunhyang University Research Fund and also supported by the Bio & Medical Technology Development Program of the NRF funded by the Korean government, MSIP(NRF-2015M3A9D7067219). Author Contributions: Yunyoung Nam designed and performed the experiments, analyzed the data, and wrote the manuscript. Yun-Cheol Nam oversaw and guided the study and the writing of the manuscript. Conﬂicts of Interest: The authors declare no conﬂict of interest.

References

1. Rahmati, A.; Zhong, L. Context-for-Wireless: Context-Sensitive Energy-Efﬁcient Wireless Data Transfer. In Proceedings of the 5th International Conference on Mobile Systems, Applications and Services, ACM, New York, NY, USA, 11–14 June 2007. 2. Mosa, A.S.M.; Yoo, I.; Sheets, L. A Systematic Review of Healthcare Applications for Smartphones. BMC Med. Inform. Decis. Mak. 2012, 12, 67. [CrossRef][PubMed] 3. Reyes, B.A.; Reljin, N.; Kong, Y.; Nam, Y.; Ha, S.; Chon, K.H. Employing an Incentive Spirometer to Calibrate Tidal Volumes Estimated from a Smartphone Camera. Sensors 2012, 16, 397. [CrossRef][PubMed] 4. Scully, C.G.; Lee, J.; Meyer, J.; Gorbach, A.M.; Granquist-Fraser, D.; Mendelson, Y.; Chon, K.H. Physiological Parameter Monitoring from Optical Recordings with a Mobile Phone. IEEE Trans. Biomed. Eng. 2012, 59, 303–306. [CrossRef][PubMed] 5. Lee, J.; Reyes, B.A.; McManus, D.D.; Maitas, O.; Chon, K.H. Atrial Fibrillation Detection Using an iPhone 4S. IEEE Trans. Biomed. Eng. 2012, 60, 203–206. [CrossRef][PubMed] 6. Nam, Y.; Lee, J.; Chon, K.H. Respiratory Rate Estimation from the Built-in Cameras of Smartphones and Tablets. Ann. Biomed. Eng. 2014, 42, 885–898. [CrossRef][PubMed] 7. Reljin, N.; Reyes, B.A.; Chon, K.H. Tidal Volume Estimation Using the Blanket Fractal Dimension of the Tracheal Sounds Acquired by Smartphone. Sensors 2015, 15, 9773–9790. [CrossRef][PubMed] 8. Nam, Y.; Kong, Y.; Reyes, B.; Reljin, N.; Chon, K.H. Monitoring of Heart and Respiratory Rates Using Dual Cameras on a Smartphone. PLoS ONE 2016, 11, e0151013. [CrossRef][PubMed] Sensors 2017, 17, 2385 18 of 18

9. Lamonaca, F.; Kurylyak, Y.; Grimaldi, D.; Spagnuolo, V. Reliable Pulse Rate Evaluation by Smartphone. In Proceedings of the 2012 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Budapest, Hungary, 18–19 May 2012. 10. Kurylyak, Y.; Lamonaca, F.; Grimaldi, D. Smartphone Based Photoplethysmogram Measurement. In Digital Image and Signal Processing for Measurement Systems; River Publishers: Aalborg, Denmark, 2012; pp. 135–164. 11. Po, L.M.; Xu, X.; Feng, L.; Li, Y.; Cheung, K.W.; Cheung, C.H. Frame Adaptive ROI for Photoplethysmography Signal Extraction from Fingertip Video Captured by Smartphone. In Proceedings of the 2015 IEEE International Symposium on Circuits and Systems (ISCAS) IEEE, Lisbon, Portugal, 24–27 May 2015; pp. 1634–1637. 12. Matsumura, K.; Yamakoshi, T. iPhysioMeter: A New Approach for Measuring Heart Rate and Normalized Pulse Volume Using Only a Smartphone. Behav. Res. Method. 2013, 45, 1272–1278. [CrossRef][PubMed] 13. Peng, R.C.; Yan, W.R.; Zhang, N.L.; Lin, W.H.; Zhou, X.L.; Zhang, Y.T. Investigation of Five Algorithms for Selection of the Optimal Region of Interest in Smartphone Photoplethysmography. J. Sens. 2016, 2016. [CrossRef] 14. Calvo-Gallego, E.; de Haan, G. Automatic ROI for Remote Photoplethysmography Using PPG and Color Features. In Proceedings of the 10th International Conference on Computer Vision Theory and Applications—Volume 1: VISAPP, Berlin, Germany, 11–14 March 2015; pp. 357–364. 15. Lee, K.; Nam, Y.; Nam, Y. Optimal ROI Determination for Obtaining PPG Signals from a Camera on a Smartphone. J. Elect. Eng. Technol. 2017, in press.

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).