Discrete Cosine Transform Based Causal Convolutional Neural Network for Drift Compensation in Chemical Sensors

DISCRETE COSINE TRANSFORM BASED CAUSAL CONVOLUTIONAL NEURAL NETWORK FOR DRIFT COMPENSATION IN CHEMICAL SENSORS Diaa Badawi∗ Agamyrat Agambayevy Sule Ozevy A. Enis Çetin∗ ∗ Department of Electrical & Computer Engineering, University of Illinois at Chicago, Chicago, IL y School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ ABSTRACT identify the gas analyte identity. Others have employed dif- ferent methods to counteract the sensor drift for the task of Sensor drift is a major problem in chemical sensors that re- analyte identification, such as domain adaptation [5,6], semi- quires addressing for reliable and accurate detection of chem- supervised learning [7], and deep stacked autoencoders [8]. ical analytes. In this paper, we develop a causal convolu- In the aforementioned papers, the features extracted from the tional neural network (CNN) with a Discrete Cosine Trans- sensors become unreliable as the sensor ages. On the other form (DCT) layer to estimate the drift signal. In the DCT hand, other approaches try to estimate the drift directly from module, we apply soft-thresholding nonlinearity in the trans- the time-series measurements of the sensors using iterative form domain to denoise the data and obtain a sparse represen- interpolation techniques [9], or by using Independent Com- tation of the drift signal. The soft-threshold values are learned ponent Analysis [10]. After the drift signal is estimated, it during training. Our results show that DCT layer-based CNNs can be subtracted from the sensor measurements to determine are able to produce a slowly varying baseline drift signal. We the actual sensor response. train the CNN on synthetic data and test it on real chemical The wide success of deep convolutional neural networks sensor data. Our results show that we can have an accurate (CNNs) has been demonstrated by their ability to achieve and smooth drift estimate even when the observed sensor sig- state-of-the-art performance in many time-series recogni- nal is very noisy. tion tasks. CNNs have been used in analyte classification Index Terms— chemical sensor drift, chemical sensor, problems [4, 11]. One particular variant of convolutional time series analysis, discrete cosine transform, convolutional neural networks, the temporal convolutional neural network neural networks (TCNN), has been shown to outperform recurrent neural networks on many benchmark data sets [12, 13]. This motivates 1 Introduction us to employ a TCNN-based framework to address the prob- Chemical sensors have been used for detecting and identify- lem of drift correction in chemical sensor data. Given the ing chemical analytes in a wide array of industrial and safety fact that sensor drift is a slowly varying signal, we propose applications [1]. Despite the fact that chemical sensor tech- utilizing the Discrete Cosine Transform (DCT) to extract nology provides practical solutions, sensor responses may de- a baseline drift signal from the observed sensor signal. In grade with time, resulting in inconsistent results.This phe- particular, we propose a new type of layer to be integrated nomenon is known as sensor drift. Sensor drift arises be- into the conventional TCNN, which we call DCT-based sub- cause of internal factors such as sensor poisoning and ag- network. In the DCT-based sub-network, we compute the ing, and/or external factors such as humidity and temperature DCT over sliding short-time windows of the temporal feature changes [2]. The chemical sensory system becomes unreli- maps. We then apply the soft-thresholding nonlinearity in the able over time, if the sensor drift signal is not properly esti- transform domain and compute the inverse DCT to transform mated. the features back to time domain. In the transform domain, There has been extensive work to address the drift prob- we use soft-thresholding that is parametrized by threshold lem using signal processing and machine learning. One com- variables, which are learned during training by the stan- mon approach is to extract features carefully from the time dard backpropagation algorithm. Our results show that the series sensor measurements and optimize a machine learning the nonlinear denoising and smoothing in transform domain algorithm to recognize patterns from these features [3, 4]. In using the DCT-based structure removes the high-frequency [3], discriminative features are extracted from the responses features, i.e., induces smoothness in the features and, sub- of the sensors and then fed into support vector machines to sequently, the constructed output baseline drift signal. Any significant deviation from the sensor baseline signal indicates This work is being supported in part by NSF grants 1739396 (UIC) and 1739451 (ASU). The authors, Badawi and Cetin, additionally thank NVIDIA the existence of chemical vapors. ICASSP 2021 - IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | 978-1-7281-7605-5/20/$31.00 ©2021 DOI: 10.1109/ICASSP39728.2021.9414512 for an equipment grant. The organization of this paper is as follows: First we ex- 978-1-7281-7605-5/21/$31.00 ©2021 IEEE 8012 ICASSP 2021 Authorized licensed use limited to: University of Illinois at Chicago Library. Downloaded on June 02,2021 at 22:23:21 UTC from IEEE Xplore. Restrictions apply. plain the drift problem, TCNN-framework and the DCT-based 1.000 0.995 structure for drift estimation in Section 2. In Section 3, we 0.990 present experimental results. Finally, in Section 4 we con- Response 0.985 clude our work. 0.980 0 200 400 600 800 1000 2 Causal DCT-based Real-Time Sen- Time (mins) sor Drift Estimation Framework Fig. 1: Overall sensor measurement signal (blue) and the un- derlying drift signal (red) The drift response of ChemFET sensors is modeled as a linear is used. The PG method is an iterative method involving se- combination of exponential decays [14–18]. In our ChemFET quential forward and inverse Fourier Transform computations sensors we can approximate the drift waveform using the fol- over blocks of sensor data. It requires some time-domain in- lowing model [19, 20] formation about the drift signal and it is assumed that the sen- t t d(t) = (R0 + R) + Rf exp − + Rs exp − sor is not exposed to gas vapor initially. To interpolate the τf + f τs + s “missing” drift portions where p(t) = 0, the authors also (1) assumed a low-pass bandwidth (BW)6 for the drift signal in where d(0) = R0 + R + Rf + Rs is the sensor conversion the Fourier domain [9]. In contrast, we do not assume any factor at the initial time after deployment and reset, Rf , Rs prior bandwidth information. The neural network automati- are fast and slow drift coefficients, τf , τs are fast and slow cally imposes the sparsity constraint on the drift signal in the drift time constants, and R, f , s are the corresponding er- transform domain during training by learning DCT domain ror terms for each of the model variables. It is not possible soft thresholds. Furthermore, we do not use future samples to know the values of the time constants and other parameters during the DCT computation to estimate the drift signal d(t). in Eq. 1 but this model enables us to generate a random pop- We use only the past and current samples of y(t) to estimate ulation of sensors with individual sensitivities and individual the drift signal d(t). As a result, we compute a real-time esti- time constants that can differ significantly. mate of d(t)1. Let a sensor time measurement be y(t) at time t. Let d(t) be the drift signal at time t. Let p(t) be the desired sensor re- 2.1 TCNN with DCT layer sponse without the drift component (drift-corrected response We developed a Temporal Convolutional Neural Network in absence of noise). The observed sensor measurement is (TCNN) with a DCT layer to extract the drift signal d[n] y(t) = d(t) + p(t) + v(t), where v(t) is the noise signal. from the observed sensor signal y[n] 2. In TCNN, the convo- Given y(t), we are interested in estimating the drift signal lutional layers implement causal convolution. Therefore, in d(t) so that we can estimate p(t). this approach, given a time series y[t] for t = 1; 2; ::: n , When a sensor is exposed to a gas vapour, it starts absorb- f g one can determine a drift estimate d^[n] and, subsequently, ing the vapour and its response changes accordingly. Once the p^[n] = y[n] d[n]. TCNN implements the so-called dilated sensor is no longer exposed to the gas vapour, it starts exuding convolution− in their convolutional layers, which can be ex- the vapour it had absorbed earlier. Absorbing and exuding the K−1 pressed as y [n] := P h[k]x[n rk]; where h is the one vapour depends on the sensor material, the analyte type and r k=0 dimensional temporal filter of length−K, and r is the dilation the environment. The ideal sensor response in the absence of rate. drift can be approximated analytically as follows [21] TCNN is made up of successive convolutional blocks. 8 0 t Ts Furthermore, residual connections are also used between suc- <> ≤ −1 t−Ts p(t) = βτ tan Ts t Ts + ∆T cessive blocks of convolutional layers. Our TCN design goes τ ≤ ≤ :> −1 t−Ts −1 t−Ts−∆T as follows: We first apply an ordinary convolutional layer βτ tan τ tan τ t Ts + ∆T − ≥ (2) with filters of length 5. Afterwards, we apply convolutional where Ts is the starting time of the exposure, and ∆T is blocks with dilated convolution and residual connections. the exposure duration.

Load more