arXiv:1603.03031v3 [q-bio.NC] 2 Apr 2016 obik a eue odtc rwiesbfr tcauses it before drowsiness workload detect of to increase to used the be known with can is blinks correlated rat rate that so inversely Blink noted [6]. blink is conditions is drowsy and (BR) It and [5]. monotonous rate in drivers blink decrease of increases drowsiness accidents car the workload of to percent due 30 Causatio that happen Crash found Vehicle globally, has Motor death, (NMVCCS) National of Survey The Organization cause accidents. ninth Health car the World is th that in The announced people is has accidents. role disabled a (WHO) important car for an use of play possibilities blinks we prevention new where to area how has Another door blinks to [4]. of wide use similarly a the of computer opened implementation This a mouse. de control computer Afte a with Brain blinks. could used between was some that information duration the In t analysis, the this fields. blinks and collecting eye various pattern analyzed the in e researchers determine the impact (BCI) of huge Interface collecting detection Computer a of The had brain methods information. blinks easier interior activity for to brain look o linked internal is humans detection be as blink to of importance accuracy known Syndrome the Eye, Vision been Increasing Dry Computer activities. have the as of Blinks known symptom also [3]. major blin dryness a Reduced and is [2]. redness which eye conversation Researchers causes face-to-face rate continuous. in listen as between speaker blinking of and world in behavior areas the synchronous not shown several have experiences changes does in environmental one detecting activity blinks that for suppresses or responsible sudden blinking brain saccades The [1] the of continuity to dust. act of experience due like very subjective our bodies image with foreign an interfere and in lights changes stimuli, bright damaging potentially as against protected eyes keep E artifacts. EEG of quality the algorithm. st its assessing detection this for discuss In blink selected processes. proposed were and mental recordings the method novel on EEG light a 14 detection some variability, shed blink rate might presen is blink that a the extracting it propose of for aim thus application to The and is accurately. carr blinks processes study blinks detect However, mental to signals. about important EEG information analyzing important while noise and xrcigBikRt aiblt rmEGSignals EEG from Variability Rate Blink Extracting Keywords Abstract lnigi eiuooi lsn fteeeld.Blinks lids. eye the of closing semiautonomic a is Blinking 60 hnjo-o yogho-yo,Dnnmg,Cheo Dongnam-gu, Byeongcheon-myeon, Chungjeol-ro, 1600, Gnrly lnsaetetdo qa ihartifacts with equal on treated are blinks —Generally, — PREPRINT dynamics, interval blink inter variability, rate blink .I I. an Gradinscak Marija Gebrehiwot, Temesgen Paprocki, Rafal NTRODUCTION mi:[email protected],[email protected] [email protected], email: oe nvriyo ehooyadEducation and Technology of University Korea eulco Korea of Republic concept high f The . such vice udy, so , the ted ye er o n k y e e r osse fafiemnt edn eso n v minute five a and session procedur reading The minute interventions. five any a require of not consisted does it that way Procedures Testing B. to se EEG experimental analyze fig.1. The blink to activity. in eye brain shown planning of is of are types purpose we various the future, detect for Curren the recorded resistance. In were electrode-skin detection. signals the hol EEG decrease electrodes’ 10-2 the to into international injected order the was in The to Electro-gel software. a [20]. according WinEEG employed system placed to we used were signals, drive and electrodes EEG amplifier disk of wer 201 a recording signals Mitsar-EEG on EEG the Simultaneously, For stored future. recorded. the was in stream processed be Video camera. USB Acquisition Data A. counting. manual by for and BRV algorithm the construct by number We blinks the (BRV variability. compare detected variability further of We rate rate tests. memory heart blink taking subjects the to in coin analogy the we extracting in for that algorithm intervals detection inter-bli pr blink blink We brain of accurately. the blinks clean variation main apply detect obtain to and to the important to focusing blinks is of signals it are analyze intervals the one To Researchers of values. is [19]. parts signal blink signals these Eye removing EEG on data. the remove brain in (EEG) artifacts acquire by electroencephalographs to driven many them sig turn brain as non-invasive in to comes measuring it are alway when not signals which are desired Blinks most cycles, [18]. of [17], [16], attention method processing audienc emotional by same among driven synchronization the eye blink is the experienced eye of who The [14 synchronicity storytelling. audiences, [13], the [12], in studied ac- [11], blinks have their [10], Researchers makes periphera important not which [15]. non-invasive more does activity a detection detection dopamine as curate blink central regarded of of are use markers Touret Blinks The many there. and [9]. in stop disease [8], studied Parkinson’s [7], been syndrome like has a diseases BR play neurological Spontaneous disorder can brain activities. blinks different many brain that detecting shown in role have significant Researchers [6]. damage h etn otaewsdvlpduigJv nsc a such in Java using developed was software testing The Flea3 Pointgrey a with captured was stream video The a-i hncennmd 31253, Chungcheongnam-do nan-si, I E II. re Lenskiy Artem d PRMNA SETUP XPERIMENTAL om opose and s the s low ter- tup tly, nal nk es te ], 0 e e s ) l input: X is 2 × N fp1-fp3 and fp2-fp4 signals output: y is the blink component H is the Heaviside function filt(X,fl,fh) is a sub-band filter, where fl and fh are low and high cut off frequencies CDF (x) estimates the cumulative distribution function for x for k =1 to 2 do t =6 · σ(Xk) if skewness(Xk) ≥ t then Xk,{i|Xk ≤t} = t end Xk = filt(Xk,fl,fh) X = Xk−E[Xk ] Figure 1. Experimental setup k σ(Xk ) ˆ FXk = CDF (Xk) r = max(Xk) − min(Xk) memory test session. Before the memory test, a passage −1 Xk,{i|Xk ≥xmax} = min(Xk)+ r · FXk (0.99) about Ethiopia is given. After reading the passage, users are −1 Xk,{i|Xk ≤xmin} = min(Xk)+ r · FXk (0.02) presented questions one by one. In this paper, we focus on end detecting blinks while subjects are reading the passage and answering questions about the passage. Overall, 28 people S = ICA(X) participated in the experiment. Among them 14 subjects were if H(kS1k− 3 · σ(S1)) > H(|S2|− 3 · σ(S2) then dropped due to falling asleep, adjusting the cap or constant P P head movements that resulted in significant noise. y = S1 else S III. EYEBLINKDETECTIONPROCEDURE y = 2 end Electrodes are applied to the head according to the 10-20 Algorithm 1: The preprocessing stage system. We used bipolar montage, which means we determined the potential between Fp1 and Fp3, along with Fp2 and Fp4. Fig. 2 presents EEG signals for both pairs. EEG signals were The next step consists of mixing signals from two pairs of recorded while participants were taking the tests and imported electrodes fp1-fp3 and fp2-fp4 in such a way that led to a in form of CSV files to Matlab for further analysis. The cleaner signal. process of blink detection can be divided into two stages: preprocessing and blink detection. The preprocessing stage consists of the following steps: (a) normalization and bandpass filtering (b) cutting of extreme amplitudes using the estimated cumulative distribution function that characterizes the signal amplitude’s distribution, (c) independent component analysis, and (d) selection of the component with eye blinks. The blink detection stage consists of (e) signal thresholding, (f) candidate extraction, and (g) polynomial fitting with finding maximum electorde pairs electorde in the polynomial function. Both stages, preprocessing and blink detection, have been presented in listings algorithm 1 and algorithm 2 respectively. The first step in the process of blink detection consists of 100 120 140 160 180 200 220 240 260 280 300 cutting off very extreme amplitudes, that usually caused by time(s) touching the electrodes or cap adjustment. Right after, the signal is band-pass filtered and normalized. We applied 50th- Figure 2. Original Fp1-Fp3 and Fp2-Fp4 electrode pairs order bandpass filter with finite impulse response. The lowest and highest normalized cut off frequencies are fL = 0.02 Usually, we want to get rid of ocular artifacts from EEG and fH = 0.08 correspondingly. In fig. 3 the EEG signals signals, as the eye blink is an artifact and leads to interpretation after bandpass filtering and normalization are presented. The problems [21]. However, our goal is on contrary aims at signals became smoother and the lower frequency components extracting blinks from EEG. We employ the fastICA [22] responsible for trends in the signals disappear after filtering. algorithm for solving Blind Source Separation (BSS) [23], For the filtered signal an amplitude cumulative distribution which allows us to differentiate neural activity from muscle function (CDF)PREPRINT is estimated. Using the CDF, we cut 2 percent and blinks [24]. Independent component analysis (ICA) con- of all amplitudes from the top and 1 percent from the bottom. sists of two steps. The first is responsible for decorrelation or whitening, when a correlation in the data is removed. The second stage is responsible for the separation, which is an input: y - blink component, orthogonal transformation of whitened signals (rotation of the fs - sampling frequency output: t - position of blinks joint density). The task is to find an orthogonal matrix U measured in samples such that the projections on the orthogonal axis are non- Gaussian [22]. One by one, we look for the rows of the lmin - minimum arc length of a blink wave T matrix U = (u1,u2) so a measure of non-Gaussianity wmin, wmax - min/max blink duration T |E(G(uk xst))| is maximized by such uk that the length of {Y, s} = segment(y) - extracts continuous segments of uk is one and orthogonal to the remain row. The function G non-zero values from y and stores them as a list in Y, s can be any nonquadratic function, which is twice continuously contains positions of segments in y. differentiable with G(0) = 0 . We tested a number of nonlinear fs fs lmin = , wmin = , wmax = fs functions and the one that results in a better component separa- 50 25 2 2 y{i|y<σ(y)} =0 tion is the g(z)= z skewness measure. The g(z)= z (skew) {Y,s} = segment(y) nonlinearity finds skew sources, but in the case of symmetric l =0 sources is not efficient. In our data, the skew measure shows for j =1 to dim(Y) − 1 do best results due to the fact that the waveforms corresponding N = dim(Yj ) to blinks are asymmetrical (fig. 4). To distinguish which of the x = {i|1 ≤ i ≤ N} components corresponds to the blink component y, we select 3 n n n the component based on the following rule a ←− argminkYj − n=0 an · (x0 , x2 , ..., xN−1)k2 a P 3 n n n yˆ = (ˆy0, yˆ1, ..., yˆN−1)= n=0 an ·(x0 , x1 , ..., xN−1) H(|S1|− 3 · σ(S1)) > H(|S2|− 3 · σ(S2)) blinks N−1 2 y = alen = i=0 ([ˆyi] − yˆPi+1) +1/N Potherwise P if wminP
100 120 140 160 180 200 220 240 260 280 300 time(s)
100 120 140 160 180 200 220 240 260 280 300 Figure 4. Independent components time(s) To construct blink rate variability, the time of occurrences of Figure 3. Normalized and band-pass filtered signals consecutive blinks are subtracted. The interval between blinks PREPRINTis stack-up into a series that constitutes blink rate variability (fig. 7). Table I. COMPARISONOF AUTOMATICAND MANUALBLINK DETECTIONDURINGREADINGSTAGE
False False True Subject Precision Recall positive negative positive 1 3 0 100 0.97 1.00 2 0 3 180 1.00 0.98 3 0 0 94 1.00 1.00 4 0 0 61 1.00 1.00 5 0 0 60 1.00 1.00 6 0 1 94 1.00 0.99 7 1 0 133 0.99 1.00 8 2 1 49 0.96 0.98 9 4 1 148 0.97 0.99 10 0 0 139 1.00 1.00 11 3 0 62 0.95 1.00 140 141 142 143 144 145 146 12 1 0 76 0.99 1.00 13 2 3 113 0.98 0.97 Figure 5. An independent component with blinks 14 0 1 122 1.00 0.99
the true positives are the correctly detected blinks. Based on these three categories we calculated the precision and recall characteristics that are shown in tables 1 and 2. The average precision during reading stage is 0.99 ± 0.02 and the average of recall is 0.99 ± 0.01. The average precision during memory testing stage is 0.98 ± 0.03 and the average of recall is 0.99 ± 0.02.
2
26 391 391.5 392 392.5 393 393.5 394 394.5 time(s) 23
22 Figure 6. Detected blinks 20
5 16
4.5 15
4 14
3.5 13
3 12 2.5
2 5
1.5 2
1 1
0.5 0 20 40 608 0 100 120 140 160 1 0 0 50 100 150 Blinks Figure 8. Extracted blink rate variability for each subject during reading Figure 7. Example of blink rate variability
V. CONCLUSION IV. RESULTS AND DISCUSSION Blinking is a natural, biological, semiautomatic process. It Figure 8 demonstrates BRVs for all of the subjects during is linked to interior brain activity and relationships between the memory test. The abscissa is the blink interval and the blinks and performing tasks. The blink rate variability might ordinate represents the length of inter-blink intervals. The BRV find applications in variety of fields, including car safety, shows the dynamics of blink intervals extracted while memory psychology or BCI. testing for each subject. In order to automatically extract blinks from large datasets of To access the quality of blink detection we manually calculated EEG recordings, we proposed the blink detection algorithm. the number of false positives, false negatives and true positives Basic steps such as band-pass filtering and thresholding are during the reading and the testing stages. The false positives applied in the algorithm. Then fastICA is applied to find an are mistakenlyPREPRINT detected blinks at places where blinks did not independent component within the blinks. The blink candidates occur. The false negatives are the blinks that were missed, and were filtered based on the heuristics that arc length of the blink Table II. COMPARISONOF AUTOMATICAND MANUALBLINK DETECTIONDURINGTESTINGSTAGE [16] Ryota Nomura, Kojun Hino, Makoto Shimazu, Yingzong Liang and Takeshi Okada, Emotionally excited eyeblink-rate variability predicts an False False True experience of transportation into the narrative world Subject Precision Recall positive negative positive [17] Nomura, R., and Okada, T. (2014). Spontaneous synchronization of 1 0 1 187 1.00 0.99 eye-blinks during story-telling performance. Cogn. Stud. 21, 225244. 2 0 3 212 1.00 0.99 3 1 1 76 0.99 0.99 [18] Nakano, T., Yamamoto, Y., Kitajo, K., Takahashi, T., and Kitazawa, 4 3 3 63 0.95 0.95 S.(2009). Synchronization of spontaneous eyeblinks while viewing video 5 0 2 146 1.00 0.99 stories. Proc. R. Soc. B Biol. Sci. 275, 3535 3544. 6 0 0 124 1.00 1.00 7 0 0 144 1.00 1.00 [19] Chambayil B., Singla R., Jha R. , EEG Eye Blink Classification Using 8 5 0 67 0.93 1.00 Neural Network, Proceedings of the World Congress on Engineering, 9 1 1 197 0.99 0.99 WCE 2010, London, U.K 10 3 4 184 0.98 0.98 [20] Malmivuo J., Plonsey R., Bioelectromagnetism, Principles and Appli- 11 4 0 51 0.93 1.00 cations of Bioelectrical and Biomagnetic Fields, page 258, New York, 12 1 3 88 0.99 0.97 13 0 8 181 1.00 0.96 Oxford, Oxford University Press, 1995 14 0 0 201 1.00 1.00 [21] Croft R.J., Barry R.J. Removal of ocular artifact from the EEG: A review. Clin. Neurophysiol. 2000;30:519 [22] Miettinen J., Nordhausen K., Oja H.and Taskinen S.. Deflation-based waveform should be above a certain threshold and the slopes FastICA with adaptive choices of nonlinearities. IEEE TRANSACTIONS of the blink waveform are within a [80o, 100o] range. ON SIGNAL PROCESSING, VOL. , NO. , 2014 [23] Comon P. Independent component analysis, A new concept?. Signal Calculated the recall and precision characteristics show that Processing 35 (1994) 287-314 the proposed algorithm is suitable for blink rate variability [24] Bell A. J., Sejnowski T. J.1995. An information-maximization approach extraction. to blind separation and blind deconvolution. Neural Computation,7, 11291159 REFERENCES [1] Gawne T.J., Martin J.M., Responses of primate visual cortical neurons to stimuli presented by flash, saccade, blink, and external darkening. J Temesgen Gebrehowt Received first BS from Neurophysiol. 2002 Nov;88(5):2178-86. Hilcoe school of computer science science in 2011 and completing his second bachelors in communica- [2] Nakano, T., and Kitazawa, S. (2010). Eyeblink entrainment at breakpoints tion engineering at Korea University of Technology of speech. Exp. Brain Res. 205, 577581. and Education (South Korea). [3] Polatsek P., Eye Blink Detection. Slovak University of Technology in Bratislava. Faculty of Informatics and Information Technologies. IIT.SRC 2013, Bratislava, April 23, 2013, pp. 18. [4] Chau M. and Betke, M. Real Time Eye Tracking and Blink Detection with USB Cameras. Computer Science Department Boston University Rafal Paprocki Completed the Bachelors and the Boston, MA 02215, USA. Masters degerees in Military University of Technol- [5] National Motor Vehicle Crash Causation Survey(NMVCCS) ogy(Warsaw, Poland) in 2010 and 2011 respectively. [6] Barbato G., Ficca G., Muscettola G., Fichele M., Beatrice M., Rinaldi In 2010, Mr Paprocki did an internship at the Uni- F. Diurnal variation in spontaneous eye-blink rate. Psychiatry Res. versit degli Studi di Camerino (Italy). Currently he is 2000;93:145151. a doctor course student at Korea university of Tech- [7] Agostino R., Berardelli A., Cruccu G., Stocchi F., Manfredi M. Corneal nology and Education (South Korea). His research and blink reflexes in Parkinsons disease with onoff fluctuations. Mov. interest is brain signal processing with applications Disord. 1987; 2: 227235. to inter-blink intervals analysis. [8] Shlik J., Zhou Y., Koszycki D., Vaccarino F.J., Bradwejn J. Effects of Marija Gradinscak is an assistant professor at CCK-4 infusion on the acoustic eye-blink startle and psychophysiological the Korea University of Technology and Education, measures in healthy volunteers. J. Psychopharmacol. 1999; 13: 385390. South Korea. From 1996 she was actively involved [9] Karson C.N., Kaufmann C.A., Shapiro A.K., Shapiro E. Eye-blink rate in teaching and research. She received her MEng and in Tourettes syndrome. J. Nerv. Ment. Dis. 1985; 173: 555 559. her Ph.D from Victoria University, Melbourne. [10] Chen E.Y., Lam L.C., Chen R.Y., Nguyen D.G. Blink rate, neurocogni- tive impairments, and symptoms in schizophrenia. Biol. Psychiatry 1995; 40: 597503. [11] Helms P.M., Godwin C.D. Abnormalities of blink rate in psychoses: A preliminary report. Biol. Psychiatry 1985; 20: 103 105. Artem Lenskiy Received BSc and MSc degrees [12] Mackert A, Woyth C, Flechtner KM, Volz HP. Increased blink rate in computer science form Novosibirsk State Tech- in drug-naive acute schizophrenic patients. Biol. Psychiatry 1990; 27: nical University, Russian in 2002 and 2004, respec- 11971202. tively.He joined doctor course at the University of Ulsan, Korea. He was awarded the Ph.D in 2010 [13] Sandyk R. The significance of eye blink rate in parkinsonism: A from the same university. After conducting research hypothesis. Int. J. Neurosci. 1990; 51: 99103. as a postdoc fellow at the Ulsan University, he joined [14] Jacobsen L.K., Hommer D.W., Hong W.L. Blink rate in childhood-onset Korea University of Technology and Education as an schizophrenia: Comparison with normal and attention-deficit hyperactiv- assistant professor in 2011. ity disorder controls. Biol. Psychiatry 1995; 40: 12221229. [15] Karson C.N.,PREPRINT Dykman R., Paige S.R.. Blink rates in schizophrenia. Schizophr. Bull. 1990; 15: 345354.