Adaptive Predictors for Extracting Physiological Signals in Two
ADAPTIVE PREDICTORS FOR EXTRACTING PHYSIOLOGICAL SIGNALS IN TWO
MODERN BIOINSTRUMENTS
by
Brently W. Robinson
APPROVED BY SUPERVISORY COMMITTEE:
Dr. Mohammad Saquib
Dr. Issa Panahi
Dr. Naofal Al-Dhahir
Dr. P.K. Rajasekaran Copyright 2019
Brently W. Robinson
All rights reserved To All My Family. ADAPTIVE PREDICTORS FOR EXTRACTING PHYSIOLOGICAL SIGNALS IN TWO
MODERN BIOINSTRUMENTS
by
BRENTLY W. ROBINSON, BS, MS
DISSERTATION
Presented to the Faculty of
The University of Texas at Dallas
in Partial Fulfillment
of the Requirements
for the Degree of
DOCTOR OF PHILOSOPHY IN
ELECTRICAL ENGINEERING
THE UNIVERSITY OF TEXAS AT DALLAS
August 2019 ACKNOWLEDGMENTS
I would like to express deep gratitude and appreciation to Dr. Mohammad Saquib, my PhD advisor, who consistently and patiently encouraged me through periods of personal challenges and academic development. Apart from his support and guidance this research would not have come to fruition.
I would also like to acknowledge and thank, my supervisory committee members, Dr. Issa Panahi, Dr. Naofal Al-Dhahir, and Dr. P.K. Rajasekaran. They provided feedback and direction through their comments and suggestions that helped to develop my research into a more comprehensive work.
Finally, I would like to acknowledge and remember Dr. Phillip Loizou, an original supervisory committee member, whose guidance brought me to UTD and who was always wiling to share both personal and academic advice.
My research was, in part, funded by the Raytheon Educational Assistance and Advanced Studies Program.
July 2019
v ADAPTIVE PREDICTORS FOR EXTRACTING PHYSIOLOGICAL SIGNALS IN TWO
MODERN BIOINSTRUMENTS
Brently W. Robinson, PhD The University of Texas at Dallas, 2019
Supervising Professor: Dr. Mohammad Saquib
Physiological signals are at the core of understanding, diagnosing, and treating the human body. Those provide valuable insight into the internal function and state of systems within the anatomy. Depending on the system being observed, a physiological signal can either be a source of information or a source of interference.
This dissertation first examines physiological hand tremor, the body’s response to stress, tiredness, or hunger, as a source of interference during microsurgery. It then examines the electrocardiogram (ECG), a source of vital information about the condition of the heart, when corrupted by broadband interference. Examination of these two physiological signals, obtained by bioinstruments, leads us to develop novel real-time adaptive predictors. Based on Kalman adaptation principle, an adaptive predictor is developed for removing physio- logical hand tremor and a scalable, cascaded predictor is designed for removing broadband interference from the ECG.
Due to the real-time requirement of bioinstruments, this dissertation addresses the issues with implementing adaptive algorithms in fixed-point representation. A proposed modified binary floating-point format is presented and is shown to overcome the prior known issues associated with fixed-point implementations and demonstrated for removal of physiological hand tremor.
vi TABLE OF CONTENTS
ACKNOWLEDGMENTS ...... v ABSTRACT ...... vi LIST OF FIGURES ...... ix LIST OF TABLES ...... xi CHAPTER 1 INTRODUCTION ...... 1 1.1 Overview ...... 1 1.2 Background ...... 2 1.2.1 Cancelling Broadband Interference ...... 3 1.2.2 Cancelling Narrowband Interference ...... 6 1.3 Motivation ...... 7 1.4 Outline ...... 9 1.5 Contributions ...... 9 CHAPTER 2 ADAPTIVE PREDICTORS ...... 11 2.1 Introduction ...... 11 2.2 Wiener Filter ...... 11 2.3 Adaptive Filter ...... 14 2.3.1 Cost Function ...... 16 2.3.2 Minimization Algorithm ...... 17 2.4 Adaptive Predictor ...... 19 2.5 Conclusion ...... 25 CHAPTER 3 PREDICTING PHYSIOLOGICAL HAND TREMOR DURING MICRO- SURGERY ...... 26 3.1 Introduction ...... 26 3.2 System Model ...... 28 3.3 Proposed Algorithm ...... 29 3.4 Performance ...... 33 3.5 Hardware Implementation ...... 33 3.5.1 Fixed-Point Representation ...... 34
vii 3.5.2 Floating-Point Representation ...... 40 3.5.3 Comparison ...... 48 3.6 Conclusion ...... 50 CHAPTER 4 ENHANCEMENT OF THE ELECTROCARDIOGRAM USING ADAP- TIVE COHERENT AVERAGING ...... 51 4.1 Introduction ...... 51 4.2 Proposed Structure ...... 55 4.3 Cascaded Architecture ...... 58 4.4 Analysis ...... 61 4.4.1 Derivation of Optimum Filter at Stage 1 ...... 61 4.4.2 Derivation of Optimum Filter at Stage n ...... 62 4.4.3 Derivation of MSE at Stage n ...... 64 4.5 Performance ...... 66 4.5.1 Least Mean Squares ...... 67 4.5.2 Recursive Least Squares ...... 67 4.5.3 Numerical Results ...... 68 4.5.4 Synthetic ECG Results ...... 69 4.6 Conclusion ...... 70 CHAPTER 5 CONCLUSIONS ...... 80 5.1 Summary ...... 80 5.2 Future Works ...... 81 REFERENCES ...... 82 BIOGRAPHICAL SKETCH ...... 87 CURRICULUM VITAE
viii LIST OF FIGURES
1.1 The adaptive noise canceller [55]...... 2 1.2 The implementation of adaptive cancellation for electrosurgical interference [57]. 3 1.3 The filter model for the cancellation of broadband interference [55]...... 4 1.4 The time-sequenced adaptive filter [13]...... 5 1.5 The adaptive recurrent filter [47]...... 5 1.6 The filter model for the cancellation of narrowband interference [55]...... 6 1.7 The Fourier linear combiner [51]...... 7 1.8 The weighted-frequency Fourier linear combiner [35]...... 8 2.1 The FIR transversal filter structure...... 13 2.2 The Wiener filter structure...... 13 2.3 The adaptive filter structure...... 14 2.4 The adaptive predictor structure...... 20 2.5 The adaptive predictor for cancelling broadband interference...... 22 2.6 The adaptive predictor for cancelling narrowband interference...... 23 2.7 The adaptive predictor with signal leakage...... 24 3.1 The hand tremor measurement and cancellation system model...... 28 3.2 The adaptive predictor used for hand tremor cancellation...... 29 3.3 Example of explosive divergence and the stalling effect...... 35 3.4 The mean square error of 64 bit fixed-point RLS...... 37 3.5 The mean square error of 64 bit fixed-point Kalman...... 38 3.6 The mean square error of 32 bit fixed-point RLS...... 38 3.7 The mean square error of 32 bit fixed-point Kalman...... 39 3.8 The mean square error of 16 bit fixed-point RLS...... 39 3.9 The mean square error of 16 bit fixed-point Kalman...... 40 3.10 The 16 bit half precision binary floating-point format...... 41 3.11 The binary floating-point unpack process...... 41 3.12 The mean square error of 64 bit floating-point RLS...... 44 3.13 The mean square error of 64 bit floating-point Kalman...... 44
ix 3.14 The mean square error of 32 bit floating-point RLS...... 45 3.15 The mean square error of 32 bit floating-point Kalman...... 45 3.16 The mean square error of 16 bit floating-point RLS...... 46 3.17 The mean square error of 16 bit floating-point Kalman...... 46 3.18 The 24 bit extended mantissa half precision binary floating-point format. . . . . 47 3.19 The mean square error of 24 bit floating-point RLS...... 47 3.20 The mean square error of 24 bit fixed-point Kalman...... 48 4.1 The synthetic ECG signal model...... 52 4.2 The frequency spectrum of the synthetic ECG signal...... 53 4.3 The reoccurence interval of the synthetic ECG signal model...... 54 4.4 The adaptive predictor structure...... 55 4.5 The adaptive coherent averaging structure...... 56 4.6 The cascade form of the ACA structure...... 58 4.7 The mean square error for the ACA Stage 1 (SNR = −5dB)...... 72 4.8 The mean square error for the ACA Stage 2 (SNR = −5dB)...... 72 4.9 The mean square error for the ACA Stage 3 (SNR = −5dB)...... 73 4.10 The mean square error for the ACA Stage 4 (SNR = −5dB)...... 73 4.11 The mean square error for the ACA Stage 5 (SNR = −5dB)...... 74 4.12 The mean square error for the ACA Stage 6 (SNR = −5dB)...... 74 4.13 The mean square error for the ACA Stage 7 (SNR = −5dB)...... 75 4.14 The mean square error for the ACA Stage 8 (SNR = −5dB)...... 75 4.15 The mean square error for the ACA Stage 1 (SNR = 5dB)...... 76 4.16 The mean square error for the ACA Stage 2 (SNR = 5dB)...... 76 4.17 The mean square error for the ACA Stage 3 (SNR = 5dB)...... 77 4.18 The mean square error for the ACA Stage 4 (SNR = 5dB)...... 77 4.19 The mean square error for the ACA Stage 5 (SNR = 5dB)...... 78 4.20 The mean square error for the ACA Stage 6 (SNR = 5dB)...... 78 4.21 The mean square error for the ACA Stage 7 (SNR = 5dB)...... 79 4.22 The mean square error for the ACA Stage 8 (SNR = 5dB)...... 79
x LIST OF TABLES
3.1 The comparison between RLS and derived Kalman algorithms...... 32 3.2 The RLS and Kalman simulation results for hand tremor cancellation...... 33 3.3 The 3-stage binary floating-point multiplication...... 42 3.4 The 3-stage binary floating-point divide...... 42 3.5 The 3-stage binary floating-point addition/subtraction...... 43 3.6 The fixed-point RLS results for hand tremor cancellation...... 48 3.7 The fixed-point Kalman results for hand tremor cancellation...... 49 3.8 The floating-point RLS results for hand tremor cancellation...... 49 3.9 The floating-point Kalman results for hand tremor cancellation...... 50 4.1 The output SNDR for different input SNR of synthetic ECG and AWGN. . . . . 70 4.2 The output SNDR for different input SNR of synthetic ECG and AWGN. . . . . 70
xi CHAPTER 1
INTRODUCTION
1.1 Overview
Physiological signals are generated within the human body and represent the internal func- tion and state of the eleven major organ systems. These signals are transmitted and measured in various forms, including: electrical, acoustic, magnetic, optical, mechanical, and chemi- cal. A physiological signal can represent a single independent response or a complex set of time dependent responses. Since these signals originate within the body but are typically measured outside of the body, they are subject to both internal interference and external measurement noise. Because each of the major organ systems generates physiological signals within the body, each signal can be considered either a source of information or a source of interference with respect to determining the state of a particular system.
Biointruments are used to measure a physiological signal from the body. The measure- ment is either processed in real-time by the bioinstrument or offline by a general purpose processor in order to extract the desired signal. Adaptive signal processing has made sig- nificant contributions toward offline processing of physiological signals, but contributions toward real-time processing has been slow. This is primarily due to the barriers present when implementing adaptive algorithms in real-time specialized processors within a bioin- strument.
In this chapter we first look at the origin of the adaptive filter and limitations of its implementation. We then discuss how the adaptive filter is used to address two different classes of noise cancellation. The first class is the cancellation of broadband interference in the presence of a narrowband signal. The second class is the cancellation of narrowband interference in the presence of a broadband signal. In each of these classes we show exam- ples of how variants of the adaptive filter have been proposed to extend and specialize the
1 general adaptive filter. We then consider cancellation of broadband noise in the presence of a quasi-periodic signal. This quasi-periodicity motivates a novel variant of the adaptive
filter proposed in this dissertation.
1.2 Background
Research on the enhancement of physiological signals gained significant momentum with the introduction of the adaptive filter, shown in Fig. 1.1, by Widrow et al. in 1975. In this article they demonstrated multiple applications of the adaptive filter, but emphasized its use toward the cancellation of noise for two physiological signals, the electrocardiogram
(ECG) and speech. The adaptive filter is shown to obtain the equivalent performance as the Wiener filter for stationary signals and close to optimal performance for nonstationary signals. The adaptive filter has two significant advantages over the Wiener filter. The first advantage is the adaptive filter does not require the same amount of a priori knowledge about the statistical characteristics of the noise. The second advantage is the adaptive filter significantly reduced the computational complexity by the introduction of recursion. The trade-off incurred for these two advantages is that the adaptive filter did not perform as well as the Wiener filter on its first iteration, but required a number of iterations (time) in order to converge to the optimum weights obtained by the Wiener filter [55].
Figure 1.1. The adaptive noise canceller [55].
2 Over the next eight years, significant research was invested by Treichler [50] and Zeidler et al. [2, 3, 33, 58] into the analysis of the adaptive filter parameters required to minimize the time to convergence to the optimum weights obtained by the Wiener filter.
Following the introduction and analysis of the adaptive filter was the implementation into a bioinstrument in 1983 by Yelderman et al. [57]. In this article they presented a software implementation of the adaptive filter using a minicomputer programmed in assembly language. This implementation, shown in Fig. 1.2, was used for the adaptive cancellation of electrosurgical interference. Due to the computational complexity and the limited dynamic range of fixed-point, adaptive filters remain implemented in software using floating-point and do not advance into real-time hardware for many decades.
Figure 1.2. The implementation of adaptive cancellation for electrosurgical interference [57].
1.2.1 Cancelling Broadband Interference
The first class of adaptive filters we consider are focused on cancelling broadband interference in the presence of a narrowband signal. The filter model for this class of adaptive filters is shown in Fig. 1.3. In this model, the output is obtained from the filter output. The filter will
3 iteratively adaptive to form a bandpass filter around the narrowband signal. Unfortunately, broadband interference within the narrowband spectrum will not be eliminated. Within this class of adaptive filters multiple variants have been proposed. We first consider a variant that uses a delayed reference input and then consider a variant using a synthetic reference input.
Figure 1.3. The filter model for the cancellation of broadband interference [55].
In 1981, the time-sequenced adaptive filter, shown in Fig. 1.4, was proposed by Ferrara and Widrow [13], which focused on optimum estimation of nonstationary signals with recur- ring statistical character. This adaptive filter variant specifically considered applications of the ECG because of the quasi-periodic nature of the signal. The time-sequenced adaptive fil- ter increased performance by introducing an array of transversal filters, which also increased computational complexity. The introduction of the array required an input sequence num- ber, which was expected to be obtained by a reference signal. The trade-off for performance was both computational complexity and additional knowledge about the signal of interest in order to provide an accurate sequence number. In 1991, the adaptive recurrent filter, shown in Fig. 1.5, was proposed by Thakor and Zhu [47], which reduced the computational complexity of the time-sequence adaptive filter by significantly reducing the number of transversal filters. The adaptive recurrent filter still required a precise reference input to denote the beginning of a sequence. Both the
4 Figure 1.4. The time-sequenced adaptive filter [13]. time-sequenced adaptive filter and the recurrent filter suffered from the same limitation and require an reference input similar to that obtained by a matched filter in order to improve upon the adaptive filter.
Figure 1.5. The adaptive recurrent filter [47].
5 1.2.2 Cancelling Narrowband Interference
The second class of adaptive filters we consider focused on cancelling narrowband interfer- ence in the presence of a broadband signal. The filter model for this class of adaptive filters is shown in Fig. 1.6. In this model, the output is obtained by subtracting the filter output from the primary input. The filter will iteratively adapt to form a bandpass filter around the narrowband interference, which is then used to subtract the narrowband interference from the broadband signal. Unlike the first class where residual broadband interference per- sisted, the second class is subject to distortion of the broadband signal when the narrowband interference has overlapping signal spectrum. Similar to the prior class, multiple adaptive filters variants have been proposed, each slightly improving on the ability to estimate the narrowband interference.
Figure 1.6. The filter model for the cancellation of narrowband interference [55].
In 1994, the adaptive Fourier linear combiner (FLC), shown in Fig. 1.7, was proposed by Vaz and Thakor [51], which adaptively adjusted the amplitudes of sinusoidal compo- nents to maximally represent the signal to be removed. The FLC increased performance by synthetically estimating the narrowband interference through the combining of multiple si- nusoids, however, it was limited by predetermined frequencies of the sinusoidal components. The trade-off for performance was both computational complexity and additional knowledge about the frequency components of the narrowband interference.
6 Figure 1.7. The Fourier linear combiner [51].
In 1996, the weighted-frequency Fourier linear combiner (WFLC), shown in Fig. 1.8, was proposed by Riviere and Thakor [35], which improved on the prior FLC by adaptively ad- justing both the amplitudes and phases of the sinusoidal components to maximally represent the signal to be removed. The WFLC is not limited like the FLC requiring knowledge of the frequency components of the narrowband interference, however, this is a trade-off obtained at the cost of increased computational complexity and latency.
1.3 Motivation
From the background review of adaptive filters, we observe that the adaptive filter output is either subject to residual interference within the signal spectrum or signal distortion. We also note that adaptive filters are constrained to software implementations in floating- point due to the dynamic range required for the adaptive parameters and the computational complexity of each variant.
7 Figure 1.8. The weighted-frequency Fourier linear combiner [35].
This dissertation first examines physiological hand tremor, as a source of interference during microsurgery. Using the adaptive filter for cancellation of narrowband interference in the presence of a broadband signal, we develop a novel real-time adaptive predictor based on Kalman adaptation principle suitable for implementation into hardware.
We then demonstrate the issues with implementing the adaptive filter in fixed-point representation, which is a common hardware implementation. A proposed modified binary
floating-point format is presented and is shown to overcome the prior known issues associated with fixed-point implementations. This allows the adaptive filter to achieve the real-time requirement of modern bioinstruments.
Finally we examine the ECG, a source of vital information about the condition of the heart, when corrupted by broadband noise. Using the adaptive filter for cancellation of broadband interference in the presence of a periodic signal, we develop a scalable, cascaded structure based on the adaptive predictor capable of removing broadband interference from the ECG while preserving the morphological features of the signal.
8 1.4 Outline
The remainder of this dissertation is organized in the following manner.
In Chapter 2, we give an overview of the adaptive predictor, providing a foundation for this dissertation.
In Chapter 3, we investigate physiological hand tremor as a source of interference limiting precision during microsurgery. Using an adaptive linear prediction system model, we develop a modified Kalman adaptation algorithm for the prediction and removal of physiological hand tremor. The derived Kalman algorithm is compared with the existing recursive least squares
(RLS) adaptation algorithm. Additionally, we examine the barriers and tradeoffs in real-time implementations of adaptive predictors. A modified binary floating-point format is proposed that efficiently uses native hardware multipliers and overcome the issues. Analysis of both
fixed and floating-point precision is performed.
In Chapter 4, we examine the ECG when corrupted by broadband interference. Ex- panding on the adaptive predictor structure presented in Chapter 3, we develop an adaptive coherent averaging (ACA) structure. The ACA structure is capable of enhancing a quasi- periodic signal in the presence of broadband interference while preserving the morphological features of the signal. Analysis is performed to derive the optimum filter and the mean square error (MSE) for each stage of the cascaded ACA structure. This analysis demon- strates the cascading benefit over the existing adaptive predictor structure. We then present the performance of the ACA structure using two common adaptation algorithms: the least mean squares (LMS) and the RLS.
Finally, in Chapter 5 we present the conclusion and future work.
1.5 Contributions
Some of the results of this dissertation have been presented previously.
9 • Chapter 3 was presented at the 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) and also published in IEEE Journal of Selected Topics in Signal Processing (JSTSP) in June 2010.
• Chapter 4 was presented at the 2014 International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) and will be submitted to the IEEE Journal of Engineering in Medicine and Biology Society (EMBS) in Fall of 2019.
10 CHAPTER 2
ADAPTIVE PREDICTORS
2.1 Introduction
In this chapter we review the foundations of signal processing leading up to the adaptive
predictor. We begin by introducing the Wiener filter, which preceded the adaptive filter. The
Wiener filter obtains the optimum solution in the mean square sense, but is computationally
complex and requires noise characteristics that are not always available. This motivated the
development of the adaptive filter, which overcame many of the prior issues by introducing
a recursive adaptation algorithm. Finally we review the adaptive predictor, a specialized
version of the adaptive filter that only requires a single input.
2.2 Wiener Filter
One focus of signal processing is the use of a priori information about the statistics of a input signal in order to provide an estimated or enhanced output signal. In 1931, Wiener and Hopf developed the set of equations for optimum estimation, now commonly referred to as the Wiener-Hopf equations. These equations describes the relationship between a primary input vector d(k), of length p, given by
> d(k) = d(k), d(k − 1), . . . , d(k − p), d(k − p + 1) , (2.1)
and a reference input vector x(k), of length p, given by
> x(k) = x(k), x(k − 1), . . . , x(k − p), x(k − p + 1) . (2.2)
Pxd, is the cross-correlation vector between the reference input vector x(k), and the primary input d(k) denoted by
11 Pxd = E[x(k)d(k)]. (2.3)
Rxx, is the autocorrelation matrix of the reference input vector x(k) denoted by
rxx(0) rxx(1) rxx(2) . . . rxx(p − 1) rxx(1) rxx(2) rxx(3) . . . rxx(p − 2) . . . . Rxx = r (2) ...... , (2.4) xx ...... . . . . . rxx(p − 1) rxx(p − 2) ...... rxx(0) where
rxx(j) = E[x(k)x(k − j)]. (2.5)
In 1947, Levinson described the Wiener-Hopf equation for discrete time in its more com- mon matrix form [17] is written as
Rxxwopt = Pxd. (2.6)
The Wiener-Hopf equation can provide the optimum coefficients wopt, in the mean square
−1 sense by multiplying both sides of (2.6) by Rxx . This form of the equation, commonly referred to as the Wiener filter solution is expressed as
−1 wopt = Rxx Pxd, (2.7) where the optimum coefficients wopt, are given by
> wopt = w1, w2, . . . , wp−1, wp . (2.8)
12 x(k) x(k-1) x(k-p) x(k-p+1) z-1 z-1 … z-1
w1 x w2 x wp-1 x wp x y(k) + … + +
Figure 2.1. The FIR transversal filter structure.
Using the optimum coefficients wopt, in a Finite Impulse Response (FIR) transversal filter, shown in Fig. 2.1, to process the reference input vector x(k), we can describe the Wiener filter output y(k) as
p X y(k) = wix(k − i), (2.9) i=1 which can also be written in vector form as
> y(k) = woptx(k). (2.10)
The Wiener filter structure, shown in Fig. 2.2, shows the relationship between the primary input d(k), the reference input x(k), and the Wiener filter output y(k).
( ) + ( ) - ( )
( ) FIR Filter =
Figure 2.2. The Wiener filter structure.
If we consider the difference between the primary input d(k), and the Wiener filter output y(k), we can describe the error e(k) as
13 e(k) = d(k) − y(k), (2.11) which can be expanded by replacing, y(k), with its vector form from (2.10) to yield
> e(k) = d(k) − woptx(k). (2.12)
Given the primary input and the reference input are statistically stationary, the Wiener
filter error e(k), represents the Minimum Means Squared Error (MMSE).
While the Wiener filter is the optimal filter in the mean square sense, it requires two pieces of a priori information that are not always available: the autocorrelation matrix of the reference input and the cross-correlation between the reference input and primary input. In addition it is computationally expensive to invert the autocorrelation matrix. The limitations of the Wiener filter inspired the derivation of the adaptive filter.
2.3 Adaptive Filter
The adaptive filter was proposed by Widrow et al. in 1975 as a means of enhancing a signal corrupted by additive noise or interference [55]. The adaptive filter, shown in Fig. 2.3, resembles the Wiener filter shown in Fig. 2.2.
( ) ( ) + - ( )
( ) Adaptive Filter ( ) Adaptive Algorithm
Figure 2.3. The adaptive filter structure.
14 Similar to the Wiener filter, the adaptive filter processes on a primary input vector
d(k) and a reference input vector x(k), both of length p, as shown in (2.1) and (2.2), respectively.Where the adaptive filter deviates from the Wiener filter is in its calculation of the filter coefficients used by the transversal filter. To reduce the computational complexity and the dependency upon a priori noise characteristics required by the Wiener filter, the adaptive filter uses recursion to iteratively adjust time-varying adaptive filter coefficients w(k), given by
> w(k) = w1(k), w2(k), . . . , wp−1(k), wp(k) . (2.13)
The reference input vector x(k), is processed by an FIR transversal filter using the time-
varying adaptive filter coefficients w(k), to form the adaptive filter output y(k), described
as
p X y(k) = w(k)x(k − i), (2.14) i=1 which can be written in vector form as
y(k) = w>(k)x(k). (2.15)
If we consider the difference between the primary input d(k), and the adaptive filter
output y(k), as shown in Fig. 2.3, we can describe the error e(k) as (2.11). By replacing y(k)
with its vector form from (2.15) we get
e(k) = d(k) − w>(k)x(k). (2.16)
The adaptive filter expands on the Wiener filter by the introduction of the adaptation
algorithm, shown in Fig. 2.3. The adaptive algorithm is composed of two parts, 1) the cost
function and 2) the minimization algorithm. The cost function generates an error surface,
15 and the minimization algorithm uses the output error e(k), to iteratively adjust the time- varying adaptive filter coefficients w(k), in order to find the error surface minima, resulting in the optimum coefficients.
2.3.1 Cost Function
The cost function is the performance measure by which the adaptive algorithm iteratively adjust the weights. There are two commonly used cost functions, both require the error, e(k), as previously defined in (2.11).
The most commonly used cost function is the Mean Square Error (MSE) approach, which optimizes the statistical measure of the squared error. The MSE cost function is defined as
J(k) = E[e2(k)]. (2.17)
The second most commonly used cost function is the Weighted Least Squares (WLS) approach, which optimizes a weighted sum of squared errors. The WLS cost function is defined as
n X J(k) = λn−ke2(k), (2.18) k=0 where 0 ≤ λ ≤ 1 denotes the forgetting factor.
Both the MSE and WLS functions are convex and have a unique minimum point, thus yielding a single optimum solution for the weight vector. Those are also both positive and monotonically increasing, thus presenting no additional minima for the algorithm to stall out.
16 2.3.2 Minimization Algorithm
The minimization algorithm iteratively adjusts the time-varying adaptive filter coefficients, w(k), to minimize the cost function, J(k). There are two distinct classes of algorithms, one for each of the previously defined cost functions.
The first class of adaptive algorithms minimize the MSE cost function, defined in (2.17).
This class is typically referred to as gradient descent algorithms. The most common gradient descent algorithm is the least mean squares (LMS) algorithm, first introduced by Widrow-
Hoff in 1960 [56].
The second class of adaptive algorithms minimize the WLS cost function, defined in
(2.18). This class is typically referred to as least squares algorithms. The two most common least squares algorithms are the recursive least squares (RLS) algorithm derived long before adaptive filtering by Gauss in 1821 and the algorithm introduced by Kalman in 1960 [21, 22].
We will briefly review the two most commonly used minimization algorithms, LMS and
RLS, which will be used throughout the remainder of this dissertation.
Least Mean Squares
The LMS algorithm uses a simple and robust gradient descent method to minimize the MSE cost function[17] defined in (2.17) as
J(k) = E[e2(k)]. (2.19)
Substituting the error e(k), from (2.16), into the MSE cost function in (2.19) yields
E[e2(k)] = E[(d(k) − w>(k)x(k))2]. (2.20)
Expanding the MSE cost function in (2.20) gives us
17 E[e2(k)] = E[(x2(k)] − 2E[(x(k)x>(k)]w(k) + w>(k)E[x(k)x>(k)]w(k). (2.21)
Using the cross-correlation vector Pxd, defined in (2.3) as
Pxd = E[x(k)d(k)], (2.22)
and the autocorrelation Rxx, defined in (2.4) as
> Rxx = E[x(k)x (k)], (2.23)
to substitute into the expanded MSE cost function in (2.21) yields
2 2 > > E[e (k)] = E[(x (k)] − 2Pxdw(k) + w (k)Rxxw(k), (2.24)
Widrow et al. have shown that by utilizing gradient descent to minimize the MSE of the
quadratic function shown in (2.24), the optimum coefficients wopt, also known as the Wiener filter solution is achieved [55], and is expressed as
−1 wopt = Rxx Pxd. (2.25)
The LMS algorithm implements gradient descent through an iterative weight update
given by
w(k) = w(k − 1) − 2µe(k)x(k), (2.26)
where, 0 ≤ µ ≤ 1/λmax, is the step size and λmax is the largest eigenvalue of the input autocorrelation matrix.
18 Recursive Least Squares
The RLS algorithm minimizes the WLS cost function [17] defined in (2.18) as
n X J(k) = λn−ke2(k). (2.27) k=0 Substituting the error e(k), from (2.11), into the WLS cost function in (2.27) yields
n n X X λn−ke2(k) = λn−k(d(k) − y(k))2]. (2.28) k=0 k=0 The expansion of this cost function is shown in [17]. Similar to LMS, the RLS algorithm implements an iterative weight update given by
w(k) = w(k − 1) − K(k)e(k), (2.29)
where K(k), is defined as
P(k − 1)x(k) K(k) = , (2.30) λ + x>(k)P(k − 1)x(k) where 0 ≤ λ ≤ 1 denotes the forgetting factor and P(k), is defined as
1 P(k) = [P(k − 1) − K(k)x>(k)P(k − 1)]. (2.31) λ The LMS and RLS adaptation algorithms both attempt to iteratively converge to the optimum Wiener filter solution, defined in (2.7). However, each algorthim is distinctly differ- ent in the computational complexity, speed of convergence, tracking ability, misadjustment and stability.
2.4 Adaptive Predictor
The adaptive predictor, also referred to as the adaptive line enhancer (ALE), was proposed by Widrow et al. in 1975 as a simplified version of the adaptive filter [55]. The adaptive
19 predictor, shown in Fig. 2.4, is nearly identical to the adaptive filter, shown in Fig. 2.3, with one advantage, it only requires a single input.
( ) = ( ) ( ) + - Δ ( )
( − Δ) Adaptive Filter ( ) Adaptive Algorithm
Figure 2.4. The adaptive predictor structure.
Similar to the adaptive filter, the adaptive predictor processes a primary input vector d(k) = x(k), of length p, given by given by (2.1). However, since the adaptive predictor only has one input, the reference input vector x(k − ∆), of length p, is a time-delayed version of the primary input, given by
x(k − ∆) x(k − ∆ − 1) . x(k − ∆) = . . (2.32) x(k − ∆ − p) x(k − ∆ − p + 1) The adaptive predictor recursively updates time-varying adaptive filter coefficients w(k), as found in (2.13). The reference input vector x(k − ∆), is processed by a FIR transversal filter using the time-varying adaptive filter coefficients w(k), to form the adaptive filter output y(k), described as
p X y(k) = w(k)x(k − ∆ − i), (2.33) i=1
20 which can be written in vector form as
y(k) = w>(k)x(k − ∆). (2.34)
If we consider the difference between the primary input d(k), and the adaptive filter output y(k), as shown in Fig. 2.4, we can describe the error e(k) as (2.11). By replacing y(k) with its vector form from (2.34) we get
e(k) = d(k) − w>(k)x(k − ∆). (2.35)
Similar to the adaptive filter, the adaptive algorithm uses the output error e(k), to iteratively adjust the time-varying adaptive filter coefficients w(k), in order to find the error surface minima, resulting in the optimum coefficients.
Because the adaptive predictor only has a single input, the time-delay on the reference input must be adjusted to be sufficient length to decorrelate the broadband components of the reference input from the broadband components of the of the primary input. If this can be accomplished, then the broadband and narrowband components can be separated. We will now review how the adaptive predictor can be used for the removal of either broadband or narrowband interference.
Cancelling Broadband Interference
The adaptive predictor is capable of cancelling broadband interference in the presence of narrowband signal. When the primary input contains both broadband interference and narrowband signal and the time-delay on the reference input is adjusted to decorrelate the broadband components of the reference input from the broadband components of the primary input, the broadband and narrowband components can be separated. Because of the time- delay decorrelation, the input signals can be represented as the model shown in Fig. 2.5.
21 ( ) ( ) ( ) + - ( )
( ) Output ( ) Adaptive Filter ( ) Adaptive Algorithm
Figure 2.5. The adaptive predictor for cancelling broadband interference.
In this model the primary input d(k), containing both narrowband signal and broadband interference can be expressed as
d(k) = s(k) + n(k), (2.36) and the reference input x(k), containing narrowband signal correlated through a linear time- invariant (LTI) process g(k), to the narrowband signal in the primary input as
x(k) = s(k) ∗ g(k). (2.37)
The output of the adaptive predictor is obtained from the filter output. The filter will iteratively adapt to form a bandpass filter around the narrowband signal. Unfortunately, broadband interference overlapping the narrowband spectrum will not be eliminated.
Cancelling Narrowband Interference
The adaptive predictor is capable of cancelling narrowband interference in the presence of broadband signal. When the primary input contains both narrowband interference and broadband signal and the time-delay on the reference input is adjusted to decorrelate the broadband components of the reference input from the broadband components of the primary
22 input, the broadband and narrowband components can be separated. Because of the time- delay decorrelation, the input signals can be represented as the model shown in Fig. 2.6.
( ) ( ) Output ( ) + - ( )
( ) ( ) ℎ( ) Adaptive Filter ( ) Adaptive Algorithm
Figure 2.6. The adaptive predictor for cancelling narrowband interference.
In this model the primary input d(k), containing both broadband signal and narrowband interference can be expressed as
d(k) = s(k) + n(k), (2.38) and the reference input x(k), containing narrowband interference correlated through a LTI process h(k), to the narrowband interference in the primary input as
x(k) = n(k) ∗ h(k). (2.39)
The output is obtained by subtracting the filter output from the primary input. The filter will iteratively adapt to form a bandpass filter around the narrowband interference, which is then used to subtract the narrowband interference from the broadband signal. Unlike the prior model where residual broadband interference persisted, this model is subject to distortion of the broadband signal when the narrowband interference has overlapping signal specturm.
23 Signal Leakage
In either of the two previous models, if the time-delay in the reference input is not able to decorrelate the broadband components in the reference input from the broadband compo- nents in the primary input, the adaptive predictor will be suboptimal. In this case the input signals can be represented as the model shown in Fig. 2.7.
Broadband ( ) ( ) Output ( ) + - ( ) Narrowband ( ) Output ( ) ℎ( ) Adaptive Filter ( ) Adaptive Algorithm
Figure 2.7. The adaptive predictor with signal leakage.
In this model the primary input d(k), containing both signal and interference can be expressed as
d(k) = s(k) + n(k), (2.40) and the reference input x(k), containing signal correlated through a LTI process g(k), to the signal in the primary input and interference correlated through a LTI process h(k), to the interference in the primary input as
x(k) = s(k) ∗ g(k) + n(k) ∗ h(k). (2.41)
The narrowband output of the adaptive predictor is obtained from the filter output and the broadband output is obtained by subtracting the filter output from the primary input. When the time-delay of the reference input is not able to completely decorrelate
24 either the narrowband or broadband components, the filter will be suboptimal and both the narrowband and broadband output will be contaminated by signal leakage from the correlated components.
2.5 Conclusion
We have reviewed the foundations of signal processing leading up to the adaptive predictor. We began with the Wiener filter, presenting a direct form calculation to the optimal filter coefficients in the mean square sense. We compared the Wiener filter to the adaptive filter, which arrives at the same solution through iterative updates using an adaptation algorithm. We discussed similarities and differences between the Wiener filter and the adaptive filter, along with the trade-offs for implementation. Finally we presented the adaptive predictor as a simple variant of the adaptive filter, which reduced the need for an additional input signal. We discussed how the adaptive filter can be used for cancelling either broadband or narrowband interference by adjusting the time-delay value. In the following chapters we will apply the adaptive predictor to two modern bioinstruments where our goal is to extract the desired physiological signals and overcome the difficulties with implementing the adaptive predictor in real-time hardware.
25 CHAPTER 3
PREDICTING PHYSIOLOGICAL HAND TREMOR DURING
MICROSURGERY12
3.1 Introduction
Physiological hand tremor fundamentally limits the accuracy with which humans can directly manipulate objects. Imprecision in microsurgery due to hand tremor has long been a concern and subject of previous work in the areas of robotic-assisted surgery [19] and active handheld microsurgical instruments [34, 35]. Physiological tremor is an involuntary oscillatory (roughly sinusoidal) movement with frequency content between 8-12 Hz [35]. In fact, its frequency is independent of the mechanical properties of the hand and arm [11]. The magnitude of physiological tremor during the most delicate part of a retinal microsurgery has been reported to be as large as 38µm rms [42]. This inherent human limitation complicates and, in some cases, prohibits the use of many delicate surgical procedures such as the injection of anticoagulants using intraocular cannulation for the treatment of retinal vein occlusions
[44, 54].
The model of physiological tremor derived in [32] is based on the process that arises from random mechanical impulses applied to a resonant system. As a result, a low order autoregressive (AR) or autoregressive-moving-average (ARMA) model was suggested for the modeling of physiological hand tremor. Furthermore, tremor is a non-stationary process [35].
A comparison between nonparametric (fast Fourier transform) and parametric (autoregres-
1 c 2010 IEEE. Portions adapted, with permission, from B. Robinson, D. Hernandez-Garduno, and M. Saquib, Fixed and Floating-Point Implementations of Linear Adaptive Techniques for Predicting Physiologi- cal Hand Tremor in Microsurgery, IEEE Journal of Selected Topics in Signal Processing, no. 4, pp. 659-667, July 2010.
2 c 2010 IEEE. Portions adapted, with permission, from B. Robinson, D. Hernandez-Garduno, and M. Saquib, Fixed and floating-point analysis of linear predictors for physiological hand tremor in microsurgery, IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 578-581, March 2010.
26 sive models) power spectrum estimation of the physiological hand tremor was reported in [43]. Handheld microsurgical instruments require compensation of the physiological hand tremor effects. In [34, 35], a weighted-frequency Fourier linear combiner (WFLC) was developed. In [59, 60], the hand tremor was modeled as a third-order autoregressive (AR-3) process. The coefficients of the linear predictor were obtained using the recursive least squares (RLS) adaptive algorithm. In contrast, the effect of hand tremor during microsurgery using robotic approaches [10, 45] can be filtered out, but these systems are costly, obtrusive, and have potential safety and liability issues. Kalman filtering has been previously applied to AR processes in other applications such as speech enhancement [30], and thus it is reasonable to explore its application in hand tremor prediction. This chapter presents the real-time implementation of hand tremor predictors using RLS and a proposed Kalman adaptation algorithm. The newly derived algorithm improves the convergence speed and minimum mean square error (MMSE) with respect to RLS. Our analysis identifies the condition for which both RLS and Kalman algorithms are identical, however, that condition is not favorable from the system performance point of view. The dynamic range and numerical precision required for these algorithms is addressed when they are implemented in fixed-point and floating-point precision. A modified binary floating- point format is proposed and is shown to overcome the prior known issues associated with fixed-point implementations of adaptive algorithms. The remainder of this chapter is organized as follows. Section 3.2 describes the hand tremor prediction system and reviews the AR model to be used. Section 3.3 presents the derivation of the hand tremor predictor equations based on Kalman filtering and compares them with their RLS counterpart. Section 3.4 analyzes and compares the results between the RLS and proposed Kalman adaptation algorithms. Section 3.5 discusses real-time adaptive signal processing trade-offs between fixed and floating-point implementations. Conclusions are finally presented in section 3.6.
27 3.2 System Model
The block diagram of the hand tremor measurement and cancellation system is shown in
Fig. 3.1. The tremor is measured using lightweight piezoresistive accelerometers attached to the belly of the hand as described in [49], measuring the acceleration in the direction of the
3 degrees of freedom. To prevent aliasing effects and undesired slow drifts, the measured acceleration goes through a 0.5Hz - 50Hz analog bandpass filter. An analog-to-digital con- verter digitizes the signal at a sample frequency of F s = 1kHz, which can then be used by the hand tremor canceller. The sampled data used for this work was taken from [48].
Figure 3.1. The hand tremor measurement and cancellation system model.
The hand tremor canceller is shown in Fig. 3.2. If the physiological tremor is modeled as an AR random process, as suggested in [43, 59, 60], the predictor can consist of an adaptive
finite impulse response (FIR) filter. In other words,
> w(k) = w1(k), w2(k), . . . wp(k), (3.1) is the filter coefficient vector at time k, and
d(k − 1) d(k − 2) u(k) = . (3.2) . d(k − p)
28 is the input vector to the filter, with d(k) being the original input signal with tremor. The predicted tremor dˆ(k) can then be subtracted from the original signal to yield an output signal y(k) = e(k) = d(k) − dˆ(k) with the tremor cancelled. To obtain the filter coefficients w(k) as k → ∞, the LMS, the RLS, or potentially Kalman based algorithms can be used. Articles [59, 60] showed that RLS outperforms the LMS algorithm in terms of convergence speed and MSE. The present work proposes an algorithm directly derived from Kalman filters to achieve further improvement in convergence speed and MSE.
Output Signal with ( ) ( ) Tremor Cancelled + - ( ) ( ) Adaptive FIR Filter = ( − 1)
Adaptive Algorithm
Figure 3.2. The adaptive predictor used for hand tremor cancellation.
3.3 Proposed Algorithm
The general discrete Kalman filter state equations can be written as [21, 22]
x(k) = Ax(k − 1) + Bs(k) + q(k − 1), (3.3)
z(k) = Hx(k) + v(k). (3.4)
In (3.3), known as the process equation, x(k) represents the state vector to be estimated at time k, A is the transition matrix, s(k) is an optional control input with a scaling vector B, and q(k) is the process noise with zero mean and covariance Q. In (3.4), known as the
29 measurement equation, z(k) represents the measured (observed) vector at time k, H is the measurement matrix, and v(k) is the measurement noise with zero mean and covariance R.
The corresponding discrete Kalman filter time update equations are:
ˆx−(k) = Aˆx(k − 1) + Bs(k), (3.5)
P−(k) = AP(k − 1)A> + Q, (3.6)
− where ˆx(k) and ˆx (k) are the aposteriori and apriori state estimates, and Pk = E[(x(k) −
> − − − > ˆx(k))(x(k) − ˆx(k)) ] and Pn (k) = E[(x(k) − ˆx (k))(x(k) − ˆx (k)) ] are the corresponding aposteriori and apriori error covariance matrices. The discrete Kalman filter measurement update equations are:
K(k) = Pˆ −(k)H>(HPˆ −(k)H> + R)−1, (3.7)
ˆx(k) = ˆx−(k) + K(k)(z(k) − Hˆx−(k), (3.8)
Pˆ −(k) = (I − K(k)H)Pˆ −(k). (3.9)
These equations can be mapped to the adaptive predictor by letting vector ˆx(k) be the coefficient vector w(k) in Fig. 3.2 and the measured vector z(k) be the desired output d(k).
We can then rewrite the unforced Kalman filter state equations as
ˆw−(k) = w(k − 1), (3.10)
d(k) = u>(k)w(k − 1) + e(k). (3.11)
30 Notice that in this model, A = I (identity matrix), B = [0, 0,... 0]> , and w(k) = [0, 0,... 0]> for all k. Furthermore, H = u>(k) , and e(k) replaces the measurement noise v(k). The measurement noise variance is given by = E[|e(k)|2]. However, since its value can not be known apriori, a proposed value needs to be used to compute the Kalman gain K(k) [31]. Using these results along with the Kalman filter time and measurement update equations, the following recursive algorithm for the update of the predictor coefficients is obtained:
1 1. Initialize P(k) = I and w(0) = 0, with δ being a small positive number. δ
2. Find the prediction error (which is actually the output)
e(k) = d(k) − u>(k)w(k − 1). (3.12)
3. Compute the Kalman gain
P(k − 1)u(k) K(k) = . (3.13) + u>(k)P(k − 1)u(k)
4. Update the predictor coefficients
w(k) = w(k − 1) + K(k)e(k). (3.14)
5. Update the error covariance matrix
P(k) = P(k − 1) − K(k)u(k)P(k − 1). (3.15)
6. Repeat steps 2 through 5 recursively.
Parameter in (3.13) affects the speed of convergence and the mean square error. For faster convergence, << 1 . Larger values of result in smaller initial estimates for the Kalman gain and thus slower convergence. Furthermore, if << u>(k)P(k − 1)u(k), then
31 the specific value of has no significant influence on the successive estimates of coefficients
w(k) [15, 31].
The derived Kalman filter update equations (3.12 - 3.15) can be compared to the RLS
algorithm in [59, 60] as summarized in Table 3.1. The relationship between Kalman filtering
and RLS has already been extensively studied [17, 40]. In particular, if A = λ−1/2I (where λ is the forgetting factor in RLS) and the measurement noise is assumed to have unit variance, both algorithms are identical. The difference with the presented model arises from choosing
A = I and << 1.
The comparison in Table 3.1 shows that if = λ, the only difference between the derived algorithm and RLS is in the equation for computing the covariance P(k). In RLS, the forgetting factor λ is included in the equations for K(k) and P(k). For quasi-stationary processes, λ ≈ 1, while choosing λ << 1 gives faster convergence and better tracking at the expense of larger mean square error. In the derived Kalman equations, affects the Kalman gain K(k), but it is not present in the computation of P(k). As a result, Kalman can provide a better mean square error (MSE) for small values of , while as mentioned before, << 1 provides faster convergence, and thus better performance for non-stationary processes such as tremor. Both algorithms are identical only when = λ = 1, however it is not an optimum operating condition.
Table 3.1. The comparison between RLS and derived Kalman algorithms. RLS Kalman
e(k) = d(k) − u>(k)w(k − 1) e(k) = d(k) − u>(k)w(k − 1) P(k − 1)u(k) P(k − 1)u(k) K(k) = K(k) = λ + u>(k)P(k − 1)u(k) + u>(k)P(k − 1)u(k) w(k) = w(k − 1) + K(k)e(k) w(k) = w(k − 1) + K(k)e(k)
1 P(k) = λ [P(k − 1) − K(k)u(k)P(k − 1)] P(k) = P(k − 1) − K(k)u(k)P(k − 1)
32 3.4 Performance
A summary of the simulation parameters and results for RLS and the derived Kalman algorithms are shown in Table 3.2. It is clear that the derived Kalman predictor significantly improves the minimum achievable MSE and convergence speed.
Table 3.2. The RLS and Kalman simulation results for hand tremor cancellation. Parameter RLS Kalman −1 1e5 1e5 λ 0.9997 0.00001 Iterations 2000 100 MSE 1.593e-6 9.845e-8
A derived Kalman adaptation algorithm for prediction of physiological hand tremor has been presented. Its performance was analyzed and compared to the previously proposed
RLS hand tremor predictors. The derived Kalman outperforms RLS with faster convergence and lower mean square error. Since the number of computations was reduced in the update equations, it is also more suitable for real-time implementation.
3.5 Hardware Implementation
Real-time signal processing imposes three basic requirements on arithmetic hardware: 1) high speed, to accommodate wide signal bandwidths; 2) high precision, to minimize rounding errors; and 3) wide dynamic range, to handle various signal to noise levels without overflow or underflow [12]. These requirements typically conflict with each other creating a basis of tradeoffs, i.e. higher precision usually entails a larger number of bits that in turn requires a speed reduction to be able to complete the arithmetic computations.
One of the most common implementation tradeoffs is choosing between fixed-point and
floating-point arithmetic. Fixed-point arithmetic is easier to implement and has a lower level of computational complexity, therefore requires less logic to implement. It, however, suffers
33 from limited dynamic range. Conversely, floating-point arithmetic has a higher dynamic
range, yet can be cumbersome to implement and has a much greater level of computational
complexity that in turn requires more logic to implement.
Adaptive algorithms have typically been implemented using floating-point arithmetic
because of instability caused by the limited dynamic range and finite precision of fixed-point
arithmetic as discussed in [6]. This requirement limited realizations of adaptive algorithms
to DSPs. However, FPGAs have begun to compete with DSPs because of the numerous
features added with each generation. Along with an ever increasing gate count, FPGAs have
added dedicated carry chains, embedded multipliers, extended precision multiply-accumulate
blocks and symmetric rounding [23]. The culmination of these features along with the prior
research from [16, 23, 25, 26, 29, 39, 41, 46] makes it feasible to implement complex floating-
point algorithms such as RLS and Kalman in FPGAs.
3.5.1 Fixed-Point Representation
As mentioned earlier, fixed-point arithmetic has limited dynamic range and finite precision.
These two drawbacks make it difficult to implement adaptive algorithms with a small word
length. Articles [1, 6, 8] show that the RLS algorithm can suddenly become unstable when
implemented in fixed-point. They note specifically that the large dynamic range of the
adaptive algorithm variables (δ−1, , and λ) could cause unwanted effects: 1) overflow or underflow, resulting in explosive divergence and 2) the stalling phenomena, resulting in no further update to the algorithm. These two effects have been recreated and plotted in
Fig. 3.3.
Explosive divergence can be seen around iteration 160 and the stalling effect after iteration
190. When explosive divergence occurs, there will be abrupt change in the MSE, but it can recover. However, when the coefficients stall, the MSE will converge to a non-optimum value.
These two behaviors are shown in the reduced word length cases below.
34 16 bit Fixed Point RLS - Estimated Coefficient Convergence for lamda = 0.997 30
20
10
0
Coefficients -10
-20
-30 0 50 100 150 200 250 300 Iteration Number
16 bit Fixed Point RLS - Mean Square Error for lamda = 0.997 1.5
1 MSE 0.5
0 0 50 100 150 200 250 300 Iteration Number
Figure 3.3. Example of explosive divergence and the stalling effect.
These issues become more evident as the word length is reduced. For proper comparison
to floating-point, we implemented fixed-point RLS and Kalman algorithms for a third-order
AR model predictor. It has already been shown in [59, 60] that higher-order predictors are
not required, since higher order coefficients have values close to zero. Standard word lengths
of 64 bits, 32 bits and 16 bits were initially used and then compared with a word length of
24 bits.
To evaluate the trade-off between performance and the word length in the finite precision
arithmetic, the mean square error as defined below can be used:
2 > 2 MSEp = E[|ep(k)| ] = E[ d(k) − up (k)wp(k − 1) ], (3.16) where E[·] denotes expected value and p is the order of precision in terms of number of bits.
35 The use of finite precision p introduces a distortion or error (ξp(n)) into the output of the ideal predictor which has p = ∞. Therefore, we can write
> > up (k)wp(k − 1) = u∞(k)w∞(k − 1) + ξP (k). (3.17)
Applying (3.17) to (3.16), we obtain
2 2 MSEp = E[|ep(k)| ] = E[|e∞(k) − ξp(k)| ]. (3.18)
Assuming the prediction error due to the system (e∞(k)) and the numerical error (ξP (k)) as uncorrelated zero mean random variables, we get
2 MSEp = MSE∞ + E[|ξP (k)| ], (3.19) which yields
2 E[|ξP (k)| ] = MSEp − MSE∞, (3.20)
2 where E[|ξp(k)| ] is the penalty imposed to the system due to the use of finite word lengths. Notice that the numerical error directly contributes additively to the mean square error.
Fig. 3.4 and Fig. 3.5 show the MSE as a function of the iteration number for RLS and
Kalman, respectively. As it can be observed, the 64 bit word length implementations of both
RLS and Kalman appear stable and quickly converge. When the word length is reduced to 32 bits, the RLS algorithm shows signs of underflow resulting in explosive divergence, causing the MSE to rapidly increase. In contrast, Kalman appears stable. This can be seen in Fig. 3.6 and Fig. 3.7. The reason for differing results between RLS and Kalman is attributed to the additional division in the computation of P (n) in the RLS algorithm, as noted in Section 3.3, which leads to the underflow. Reducing the word length even further to 16 bits reveals both prior noted phenomena. RLS and Kalman see underflows early in the calculations, becoming
36 extremely unstable. This continues until P (n) is reduced to a zeros matrix, which causes the weight update to freeze. Once the update stalls, the MSE slowly diverges, as prior discussed.
These two issues can be seen in Fig. 3.8 and Fig. 3.9. It is clear from these results that RLS is more sensitive to a reduction in word length than the proposed Kalman predictor.
64 BIT FIXED POINT RLS - MEAN SQUARE ERROR FOR LAMDA = 0.9997 & DELTA = 100000 -5 x 10 9
8
7
6
5 MSE 4
3
2
1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.4. The mean square error of 64 bit fixed-point RLS.
37 64 BIT FIXED POINT KALMAN - MEAN SQUARE ERROR FOR EPSILON = 0.0001 & DELTA = 10000 -7 x 10 7
6
5
4 MSE 3
2
1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.5. The mean square error of 64 bit fixed-point Kalman.
32 BIT FIXED POINT RLS - MEAN SQUARE ERROR FOR LAMDA = 0.9997 & DELTA = 10000 -4 x 10
3
2 MSE
1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.6. The mean square error of 32 bit fixed-point RLS.
38 32 BIT FIXED POINT KALMAN - MEAN SQUARE ERROR FOR EPSILON = 0.001 & DELTA = 1000 -6 x 10 4
3.5
3
2.5
2 MSE
1.5
1
0.5
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.7. The mean square error of 32 bit fixed-point Kalman.
16 BIT FIXED POINT RLS - MEAN SQUARE ERROR FOR LAMDA = 0.9997 & DELTA = 10
15
10 MSE
5
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.8. The mean square error of 16 bit fixed-point RLS.
39 3.5.2 Floating-Point Representation
As mentioned in Section 3.5, floating-point is more difficult to implement and requires in- creased computational complexity, however, it provides a wider dynamic range. The in- creased dynamic range make it possible to represent the wide range in the adaptive algorithm variables and avoid underflow on operations close to zero. In this dissertation, we choose to comply with the Binary Floating-Point Standard, IEEE 754-2008 [5], to implement the double precision 64 bit format, the single precision 32 bit format and the newly presented half precision 16 bit format. We then present an extended mantissa half precision 24 bit format in order to attain better precision. These results are then compared with the results from the fixed-point implementation. The drive for a smaller word format originates from the desire to use a single 18-bit hard macro multiplier for each multiplication. This reduces the total amount of logic while preserving a higher clock speed.
16 BIT FIXED POINT KALMAN - MEAN SQUARE ERROR FOR EPSILON = 0.1 & DELTA = 10
0.025
0.02
0.015 MSE
0.01
0.005
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.9. The mean square error of 16 bit fixed-point Kalman.
40 The half precision format is briefly described below. The other formats have been thor- oughly described in [5]. The half precision format consists of a 1 bit sign (S), a 5 bit exponent
(E), and a 10 bit mantissa (M) as shown in Fig. 3.10. The sign of the mantissa is represented by the sign bit and the sign of the exponent is represented in excess representation. This means the exponent is treated as a positive number from which a constant is subtracted to form the final exponent. Since the mantissa is stored in a normalized binary floating-point, the MSB will always be one; therefore, this bit is not stored and is implied.
S N = (−1) × .1 M × 2(E−15)
Figure 3.10. The 16 bit half precision binary floating-point format.
The computational complexity of a floating-point implementation comes primarily from the modified arithmetic blocks. Addition, subtraction, multiplication and division are the fundamental building blocks of adaptive algorithms. Binary floating-point addition, subtrac- tion and multiplication have been previously covered by the following articles [16, 23, 25, 26,
29, 39, 41], while division has been covered by [25, 29, 39, 41, 46]. We provide below a brief summary of these Three-Stage blocks in Fig. 3.11 and Table 3.3, 3.4, and 3.5.
Figure 3.11. The binary floating-point unpack process.
41 Table 3.3. The 3-stage binary floating-point multiplication. N3 = N1 x N2 Stage 1: Unpack into Sign, Exponent and Fraction Check for denormalized, infinity or zero Stage 2: S3 = S1 XOR S2 E3 = E1 + E2 M3 = M1 x M2, store most significant bits Stage 3: Normalization - Shift M3 at most by two bits Pack into Binary Floating-Point Format
Table 3.4. The 3-stage binary floating-point divide. N3 = N1 / N2 Stage 1: Unpack into Sign, Exponent and Fraction Check for denormalized, infinity or zero Stage 2: S3 = S1 XOR S2 E3 = E1 - E2 M3 = M1 / M2 (Using iterative subtraction) Stage 3: Normalization - Shift M3 at most by two bits Pack into Binary Floating-Point Format
Combining these basic building blocks, we are able to form both RLS and Kalman al- gorithms. Similar to fixed-point, we implemented floating-point RLS and Kalman using a word length of 64 bits, 32 bits and 16 bits.
Similar to the fixed-point implementation, the 64 bit word length provides a basis to compare the reduced word length implementations. The 64 bit word length implementation of both RLS and Kalman yield similar results to the fixed-point, appearing stable and con- verging very quickly as shown in Fig. 3.12 and Fig. 3.13. When the word length was reduced to 32 bits, the results are similar to the 64 bit word length, as can be seen in Fig. 3.14 and Fig. 3.15. However, reducing the word length even further to 16 bits yields underflow.
42 Table 3.5. The 3-stage binary floating-point addition/subtraction. N3 = N1 +/- N2 Stage 1: Unpack into Sign, Exponent and Fraction Check for denormalized, infinity or zero Compare E1 and E2, if E2 >E1, swap N1 and N2 Calculate and store |E2 − E1| Stage 2: Shift M2 to the right by |E2 − E1| calculated in the previous stage Fill left most bits with zeros If S1 = S2, M3 = M1 + M2, otherwise M3 = M1 - M2 Set S3 and E3 to the values from N1 Stage 3: Normalization - Shift M3 to the left until there is a ‘1’ in the MSB Adjust E3 by the number of left shifts in the prior step Pack into Binary Floating-Point Format
This underflow appears when doing calculations on numbers very close to zero, resulting in explosive divergence as shown in Fig. 3.16 and Fig. 3.17. One way to alleviate this problem is to reduce the dynamic range of the adaptive algo- rithm variables. Unfortunately, these variables affect the rate of convergence, the final value of the MSE, and the ability for the algorithm to track, as discussed in Section 3.3. There- fore, reducing these variables directly impacts the performance of predicting physiological hand tremor. Since the goal is to maintain maximum performance in the implementation, an alternative way to gain the needed precision without substantially increasing the logic is required. Binary floating-point with a 16 bit word length appears to satisfy the dynamic range needed for RLS and Kalman, however, it lacks precision in calculations around zero. In order to overcome this issue, we extend the mantissa of the half precision by 8 bits, forming a 24 bit binary floating-point format. The extended mantissa half precision format consists of a 1 bit sign (S), a 5 bit exponent (e), and a 18 bit mantissa (M) as shown in Fig. 3.18. This format avoids substantially increasing logic by utilizing the full 18 bit hard macro multipliers
43 64 BIT FLOATING POINT RLS - MEAN SQUARE ERROR FOR LAMDA = 0.9997 & DELTA = 100000 -4 x 10
3
2 MSE
1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.12. The mean square error of 64 bit floating-point RLS.
64 BIT FLOATING POINT KALMAN - MEAN SQUARE ERROR FOR EPSILON = 1e-006 & DELTA = 100000 -7 x 10 7
6
5
4 MSE 3
2
1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.13. The mean square error of 64 bit floating-point Kalman.
44 32 BIT FLOATING POINT RLS - MEAN SQUARE ERROR FOR LAMDA = 0.9997 & DELTA = 10000 -4 x 10
2 MSE
1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.14. The mean square error of 32 bit floating-point RLS.
32 BIT FLOATING POINT KALMAN - MEAN SQUARE ERROR FOR EPSILON = 0.001 & DELTA = 100 -6 x 10 8
7
6
5
4 MSE
3
2
1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.15. The mean square error of 32 bit floating-point Kalman.
45 16 BIT FLOATING POINT RLS - MEAN SQUARE ERROR FOR LAMDA = 0.9997 & DELTA = 10
0.8
0.7
0.6
0.5
0.4 MSE
0.3
0.2
0.1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.16. The mean square error of 16 bit floating-point RLS.
16 BIT FLOATING POINT KALMAN - MEAN SQUARE ERROR FOR EPSILON = 0.1 & DELTA = 10
450
400
350
300
250 MSE 200
150
100
50
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.17. The mean square error of 16 bit floating-point Kalman.
46 within the FPGA, unlike the single precision 32 bit implementation which would require two multipliers for each 23 bit multiplication of the mantissas as shown in [53].
N = (−1)S × .1 M × 2(E−15)
Figure 3.18. The 24 bit extended mantissa half precision binary floating-point format.
This format is still compliant with the 16 bit half precision standard by zero padding the 8 least significant bits. The implementation of the 24 bit format shows both RLS and
Kalman algorithms converge properly. Explosive divergence still exists; however, compared to the 16 bit format, it has been reduced by a factor of 106 for RLS and by a factor of 105 for Kalman. These results can be seen in Fig. 3.19 and Fig. 3.20.
24 BIT FLOATING POINT RLS - MEAN SQUARE ERROR FOR LAMDA = 0.9997 & DELTA = 100 -3 x 10 1
0.9
0.8
0.7
0.6
0.5 MSE
0.4
0.3
0.2
0.1
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.19. The mean square error of 24 bit floating-point RLS.
47 24 BIT FLOATING POINT KALMAN - MEAN SQUARE ERROR FOR EPSILON = 0.01 & DELTA = 100 -3 x 10 3
2.5
2
1.5 MSE
1
0.5
0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ITERATION NUMBER
Figure 3.20. The mean square error of 24 bit fixed-point Kalman.
3.5.3 Comparison
A summary of the fixed-point implementation results is shown in Table 3.6 and 3.7. For the purpose of our numerical analysis, we use p = 64 as the ideal case, and using (20) we define the percentage of penalty paid by the system for using p order precision as
MSEP − MSE64 PenaltyP = × 100 (3.21) MSE64
Table 3.6. The fixed-point RLS results for hand tremor cancellation. RLS p 64 bit 32 bit 24 bit 16 bit δ−1 1e5 1e4 100 10 0.9997 0.9997 0.9997 0.9997 Iterations 7000 9000 N/A* N/A* MSEP 1.392e-6 5.311e-6 6.634e-5 4.619e-4 PenaltyP − 3.82e+2 4.77e+3 3.32e+4 * Coefficients do not converge to optimum values
48 Table 3.7. The fixed-point Kalman results for hand tremor cancellation. Kalman p 64 bit 32 bit 24 bit 16 bit δ−1 1e4 1e3 100 10 0.0001 0.001 0.01 0.1 Iterations 100 800 4000 N/A* MSEP 2.857e-7 5.054e-7 1.182e-6 2.547e-5 PenaltyP − 3.63e+1 8.49e+1 1.83e+3 * Coefficients do not converge to optimum values
It is clear that the derived Kalman predictor improves the minimum achievable MSE and convergence speed for 64 bit and 32 bit word lengths. For example, the speed of convergence for the 32 bit word length improved by an order of magnitude. The derived Kalman also allows for a 24 bit implementation which was not feasible with RLS. Furthermore, notice that Kalman is less sensitive to finite precision for the 32 bit case, as the penalty is an order of magnitude smaller compared to RLS, and that allows the Kalman algorithm to accommodate more penalty and still converge at 24 bit. The floating-point implementation results are shown in Table 3.8 and 3.9. Similar to the fixed-point implementation, it can be observed that the derived Kalman predictor improves MSE and convergence speed for 64 bit, 32 bit and 24 bit word lengths. In this case, the 32 bit word length resulted in an improvement by a factor of 5. Similar to fixed-point results, Kalman is less sensitive to the reduction of p compared to RLS, as shown by the penalty values.
Table 3.8. The floating-point RLS results for hand tremor cancellation. RLS p 64 bit 32 bit 24 bit 16 bit δ−1 1e5 1e4 100 10 0.9997 0.9997 0.9997 0.9997 Iterations 2000 5000 9000 N/A* MSEP 1.593e-6 1.231e-6 3.286e-5 1.200e-3 PenaltyP − 8.84e+1 2.36e+3 8.62e+4 * Coefficients do not converge to optimum values
49 Table 3.9. The floating-point Kalman results for hand tremor cancellation. Kalman p 64 bit 32 bit 24 bit 16 bit δ−1 1e5 1e3 100 10 0.00001 0.001 0.01 0.1 Iterations 100 1000 4000 N/A* MSEP 9.845e-8 4.086e-7 2.591e-6 2.726e-4 PenaltyP − 2.94e+1 1.86e+2 1.96e+4 * Coefficients do not converge to optimum values
From the above results, we can also observe that floating-point is required for RLS if a 24 bit word length is to be implemented. In contrast, using fixed-point arithmetic does not significantly degrade the MMSE of the derived Kalman when compared to floating-point. This shows that the derived Kalman is more robust to finite precision for reduced bit word lengths.
3.6 Conclusion
A derived Kalman adaptation algorithm for prediction of physiological hand tremor has been presented. Its performance was analyzed and compared to the previously proposed RLS hand tremor predictors. Both predictors have been implemented using fixed-point and floating- point arithmetic. The fixed-point RLS results show a need for increased dynamic range and improved precision, which were overcome by a proposed 24 bit Binary Floating-Point format. This alleviates the explosive divergence and stalling effect. The derived Kalman outperforms RLS with faster convergence and lower mean square error. It also offers more robustness to finite precision, as the word length is reduced, making it feasible to realize a 24 bit fixed-point implementation.
50 CHAPTER 4
ENHANCEMENT OF THE ELECTROCARDIOGRAM USING ADAPTIVE
COHERENT AVERAGING1
4.1 Introduction
The electrocardiogram (ECG) is a recording of the hearts electrical potentials over time. The most common single diode representation of the ECG complex is shown in Fig. 4.1. The ECG provides physicians with a graphical representation of the hearts functionality and is used to expose physiological and pathological irregularities of the cardiac rhythm and circulatory system. Unfortunately, multiple sources of interference contaminate the ECG signal during acquisition, corrupting the morphology and making it difficult for physicians to provide an accurate diagnosis. These sources of interference can be grouped into two categories: narrowband and broadband. The major contributing narrowband interference are as follows: power line interference (PLI), baseline wander (BW), motion artifacts (MA) and electrode motion artifacts (EM). The remaining broadband interference are as follows: electromyographic (EMG), and elec- trode contact noise [14]. Typically, the frequency of the narrowband interference is either known in advance or can be determined through analysis. This allows for the interference to be attenuated by either a simple FIR notch filter or a slightly more complex adaptive noise canceller (ANC) as shown in [55]. The advantage of using an ANC over the more simplistic solution is that the filter can track the frequency of the interference if it drifts over time. It is discussed in [47] that even when the narrowband interference exhibits spectral overlap with the ECG, a notch filter can be used with minimal effect to the ECG morphology.
1 c 2014 IEEE. Portions adapted, with permission, from B. Robinson and M. Saquib, Adaptive coherent averaging for real-time electrocardiogram enhancement, 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5048-5051, Aug 2014.
51 Synthetic ECG Complex 1.5
1
0.5 ECG mV
0
−0.5 0.5 1 1.5 Time in seconds
Figure 4.1. The synthetic ECG signal model.
It is more difficult, however, to remove the broadband interference when it shares the frequency spectra of the ECG. The reason for difficulty is because the ECG is composed of multiple harmonic components, as shown in Fig. 4.2. Many linear adaptive approaches have been applied to remove the broadband interference from the ECG including the use of an adaptive noise canceller by Yelderman et al. [57], the time-sequenced adaptive filter by Ferrara [13], the adaptive recurrent filter by Thakor and Zhu [47], the Fourier linear combiner (FLC) by Vaz et al. [51], the wavelet transform by Li et al. [24], and the adaptive comb filter (ACF) by Cyrill et al. [9]. There are advantages and disadvantages to each of these techniques, but all are sensitive to the quasi-periodic nature of the ECG. This makes their performance highly dependent on accurately determining the reoccurrence interval of the ECG complex, shown in Fig. 4.3. Determining this reoccurrence interval requires an additional pre-processing step that is usually accomplished with a matched filter.
52 FFT of Synthetic ECG Signal 1
0.9
0.8
0.7
0.6
0.5
0.4
Normalized Magnitude 0.3
0.2
0.1
0 0 5 10 15 20 25 30 Frequency (Hz)
Figure 4.2. The frequency spectrum of the synthetic ECG signal.
More recently, eigenanalysis techniques have been applied to remove the broadband in- terference from the ECG. These include both principal component analysis and independent component analysis [4, 7, 18]. The two primary drawbacks of the eigenanalysis techniques are they require considerably more resources and computational time due to the increased computational complexity. This makes them less suitable for real-time implementation into hardware. In this chapter, an adaptive coherent averaging (ACA) structure is proposed to remove contaminating broadband interference while preserving the morphological features of the ECG signal. Consecutive ECG complexes resemble each other and are generally uncorre- lated with the interference, however, the time varying delay between complexes creates a quasi-periodic nature. This nature is exploited by adaptively averaging multiple consecu- tive ECG complexes, thus improving the signal-to-noise plus distortion ratio (SNDR). The
53 Multiple Synthetic ECG Complexes 1.5
1
0.5 ECG mV
0
−0.5 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Time in seconds
Figure 4.3. The reoccurence interval of the synthetic ECG signal model.
proposed structure does not require a priori knowledge of either the primary signal or the interference characteristics. The cascade structure is scalable and can be implemented in real-time hardware.
The remainder of the chapter is organized as follows. Section 4.2 proposes the ACA structure as a slight variant of the adaptive predictor. Section 4.3 describes the scalable casacaded ACA architecture. Section 4.4 analyzes the ACA by deriving the filter output and the theoretical MSE for each stage. Sections 4.5 contains the analytical performance along with the performance of the LMS and RLS adaptive algorithms for both wide sense stationary (WSS) and synthetic ECG signals. Conclusions are presented in Section 4.6.
54 4.2 Proposed Structure
The adaptive predictor, shown in Fig. 4.4, was introduced in 1975 by Widrow et al. [55] as a simplified form of an adaptive filter. The adaptive predictor is nearly identical to the adaptive filter with one advantage, it only requires a single input. Similar to the adaptive filter, the adaptive predictor processes a primary input vector d(k) = x(k), and a reference input vector x(k − ∆), which is a time-delayed version of the primary input.
( ) = ( ) ( )
Δ
( − Δ) ( ) ( ) Adaptive Filter - + Adaptive Algorithm
Figure 4.4. The adaptive predictor structure.
Since the adaptive predictor operates on a single input, it does not require a priori knowledge of the characteristics of the signal of interest or the additive noise. The adaptive predictor is able to accomplish this by appropriately choosing the size of the time-delay in the reference input, in order to exploit the difference in correlation lengths between the narrowband signal and the broadband interference. When attenuating additive white Gaus- sian noise (AWGN), the adaptive predictor only requires one delay element, due to the fact that white Gaussian noise samples are independent and identically distributed (i.i.d) random variables. This case is known as a one-step adaptive linear predictor. The proposed ACA structure, show in Fig. 4.5, is a modified version of the adaptive predictor, designed to be cascaded in order to take advantage of coherent averaging of the adaptive filter output y(k), with the primary input d(k) in order to produce a cascade output, c(k).
55 1 − ( ) = ( ) ( ) x +
x Δ
( − Δ) ( ) - ( ) Adaptive Filter + Adaptive Algorithm
Figure 4.5. The adaptive coherent averaging structure.
Similar to the adaptive predictor, the ACA contains a primary input vector d(k), of length p, recall that
> d(k) = x(k), x(k − 1), . . . , x(k − p), x(k − p + 1) , (4.1) and a reference input vector x(k − ∆), of length p, which is a time-delayed version of the primary input expressed as
x(k − ∆) x(k − ∆ − 1) . x(k − ∆) = . . (4.2) x(k − ∆ − p) x(k − ∆ − p + 1)
Recall that the time-varying adaptive predictor coefficients w(k), given by
> w(k) = w1(k), w2(k), . . . , wp−1(k), wp(k) . (4.3)
The reference input vector x(k−∆), is processed by a FIR transversal filter using the time- varying adaptive filter coefficients w(k), to form the adaptive filter output y(k), described
56 as p X y(k) = w(k)x(k − ∆ − i), (4.4) i=1 which can be written in vector form as
y(k) = w>(k)x(k − ∆). (4.5)
If we consider the difference between the primary input d(k) = x(k), and the adaptive
filter output y(k), as shown in Fig. 4.5. We can describe the error e(k) as
e(k) = d(k) − y(k), (4.6) which can be expanded by replacing y(k) with its vector form from (4.5) to yield
e(k) = d(k) − w>(k)x(k − ∆). (4.7)
Similar to the adaptive predictor, the adaptive algorithm shown in Fig. 4.5, will use the output error e(k), to recursively adjust time-varying adaptive predictor coefficients w(k), in order to minimize the expected error power based on the cost function.
The ACA structure adds to the adaptive predictor by the introduction of an average output c(k), as shown in Fig. 4.5. This scaled output takes advantage of the inherent coherent averaging of the primary input d(k) and adaptive filter output y(k). We can describe the cascade output c(k) as
c(k) =ad ¯ (k) + ay(k), (4.8) which can be expanded by replacing y(k) with its vector form from (4.5) to yield
c(k) =ad ¯ (k) + aw>(k)x(k − ∆), (4.9)
57 where 0 ≤ a ≤ 1 is a scaling factor anda ¯ = 1 − a. It is important to observe that the proposed ACA structure, shown in Fig. 4.5, is identical to the adaptive predictor structure, shown in Fig. 4.4, when a = 1.
4.3 Cascaded Architecture
The ACA was intended to be cascaded to take advantage of the coherent averaging among stages. The cascade form of the ACA structure is shown in Fig. 4.6. The cascade form is show with block elements, each with a primary input (P), a reference input (R) and a cascaded output (C). The input vector x(k), is used as both the primary and reference input on the first stage, but only the reference input for each subsequent stage with the greatest delay on the lowest stage. The cascade output of a given stage is the primary input to the following stage. ( −( −1)Δ) ( − Δ) ( ) … Δ Δ
P CCC… P P
Δ ACA ACA ACA
R R R
Stage 1 Stage N-1 Stage N
Figure 4.6. The cascade form of the ACA structure.
In order to represent multiple stages, we introduce n to represent each stage, where n = 1,...,N. Using n, we can rewrite (4.9) as
> cn(k) =a ¯nd(k) + anw (k)x(k − ∆), (4.10)
where 0 ≤ an ≤ 1 is a scaling factor for stage n anda ¯n = 1 − an. By expressing an as
58 1 a = , (4.11) n n the cascade output cn(k), of each stage to have equal contribution to the final output, cN (k). Also, the first stage of the cascaded architecture is equivalent to the adaptive predictor structure and each subsequent stage is the ACA structure.
Using (4.10), we obtain the cascade outputs of the first three stages as follows.
Stage 1:
> c1(k) = w1 (k)x(k − N∆). (4.12)
Stage 2:
> c2(k) =a ¯2c1(k) + a2w2 (k)x(k − (N − 2)∆). (4.13)
Stage 3:
> c3(k) =a ¯3c2(k) + a3w3 (k)x(k − (N − 3)∆). (4.14)
By taking advantage of the recursive nature of (4.12 - 4.14), the recursive equation for the output of stage n of the ACA structure can be expressed as
> cn(k) =a ¯ncn−1(k) + anwn (k)x(k − (N − n)∆). (4.15)
For notational simplicity, we define the input signal at stage n as
xn(k) = x(k − (N − n)∆), (4.16) and then substitute (4.16) into (4.15) to yield
> cn(k) =a ¯ncn−1(k) + anwn (k)xn(k). (4.17)
59 Expanding (4.17) to define the output at stage 4 gives us
> > c4(k) =¯a4a¯3a¯2a1w1 (k)x1(k) +a ¯4a¯3a2w2 (k)x2(k) (4.18) > > +a ¯4a3w3 (k)x3(k) + a4w4 (k)x4(k). Reversing the order of (4.18) to represent descending stages is expressed as
> > c4(k) =a4w4 (k)x4(k) +a ¯4a3w3 (k)x3(k) (4.19) > > +a ¯4a¯3a2w2 (k)x2(k) +a ¯4a¯3a¯2a1w1 (k)x1(k). Taking advantage of the repeating coefficient terms in (4.19), we can generalize (4.17) for stage n as follows
> > cn(k) =anwn (k)xn(k) +a ¯nan−1wn−1(k)xn−1(k) (4.20) > > +a ¯na¯n−1an−2wn−2(k)xn−2(k) + ... +a ¯na¯n−1 ... a¯2a1w1 (k)x1(k). If we define the product of the averaging coefficients from stage n down to i as
An,i =a ¯na¯n−1 ... a¯i, (4.21) where i = n, . . . , 2 and An,n+1 = 1, then we can substitute (4.21) into (4.20) to simplify as
> > cn(k) =An,n+1anwn (k)xn(k) + An,nan−1wn−1(k)xn−1(k) (4.22) > > + An,n−1an−2wn−2(k)xn−2(k) + ... + An,2a1w1 (k)x1(k). We can further simplify (4.22) to define the filter output at stage n, given by
n X > cn(k) = An,i+1aiwi (k)xi(k). (4.23) i=1 Equation (4.23) reveals the relationship between the cascade output at stage n with all of the previous stages including the current one.
60 4.4 Analysis
In this section we perform analysis to understand the advantages and limitations of the proposed ACA structure. We begin by deriving the optimum filter at stage 1. Because the single stage ACA structure is identical to the adaptive predictor when a = 1, this analysis is consistent with the results shown for the adaptive predictor in Chapter 2. We then extend the analysis to show the optimum filter at stage n of the cascaded ACA structure. Finally, we derive the MSE at stage n for the cascaded ACA structure.
Throughout this section we assume xi is a WSS processes with a cross-correlation matrix given by
> Ri,j = E[xi(k)xj (k)], (4.24) where i = j = 0,...,N.
4.4.1 Derivation of Optimum Filter at Stage 1
Input x0(k) is the desired reference and the primary input is x(k − (N − 1)∆). The cross- correlation vector between them is given by
r = E[x0(k)x(k − (N − 1)∆)]. (4.25)
Using the principle of orthogonality [28] at the first stage, we get
> E[x0(k)(x(k − (N − 1)∆) − x0(k) w1))] = 0, (4.26) which yields the optimum filter at stage n = 1 as
−1 w1 = R r. (4.27)
The WSS process assumption allows us to use R instead of R0,0 in the above equation.
61 4.4.2 Derivation of Optimum Filter at Stage n
At stage n the primary input cn−1(k) can be expressed as
n−1 X > cn−1(k) = An−1,i+1aiwi (k)xi(k). (4.28) i=1
The cross-correlation vector of xn(k) and cn−1(k) is denoted by
rn = E[xn(k)cn−1(k)]. (4.29)
Using the principle of orthogonality [28] at stage n implies
> E[xn(k)(cn−1(k) − xn(k) wn)] = 0. (4.30)
Substituting (4.28) into (4.30) yields
" n−1 !# X > > E xn(k) An−1,i+1aiwi xi(k) − xn(k) wn = 0. (4.31) i=1
By distributing xn(k), we rewrite the above equation as
n−1 > X > > E[xn(k)xn(k) ]wn = An−1,i+1aiE[xn(k)xi(k) ]wi , (4.32) i=1 which can be further simplified to
n−1 X Rn,nwn = An−1,i+1aiRn,iwi. (4.33) i=1
Since xn(k) is derived from a WSS process, we can use R in place of Rn,n yielding
n−1 X Rwn = An−1,i+1aiRn,iwi. (4.34) i=1 Multiplying both sides by R−1, yields the optimum filter at stage n > 1 as
62 n−1 X −1 wn = An−1,i+1aiR Rn,iwi. (4.35) i=1 We can show the relationship between the optimum filter at stage n and the optimum
filter at stage 1 by further simplifying (4.35). In order to accomplish this, we define Cn,i as
−1 Cn,i = An−1,i+1aiR Rn,i. (4.36)
Substituting (4.36) into (4.35) yields
n−1 X wn = Cn,iwi = Bnw1, (4.37) i=1 where n > 1 and Bi is given by
n−1 X Bn = Cn,iBi i=1 . . . .
B3 = C3,1B1 + C3,2B2
B2 = C2,1
B1 = I .
In (4.37) we can see a direct relationship between optimum filter at stage n and the optimum filter at stage 1. Substituting (4.27) into (4.37) we get
−1 wn = BnR r = B˜ nr, (4.38)
−1 where B˜ n = BnR . Equation (4.38) shows how the optimum filter at stage n can be derived directly from the autocorrelation function of the input signal, xn(k).
63 4.4.3 Derivation of MSE at Stage n
Now our goal is to analyze the MSE expression for each stage. Using the input in (4.5), the converged filter output at stage n is expressed as
> yn(k) = wn xn(k), (4.39)
which can also be written as
yn(k) = αnd(k) + zn(k). (4.40)
where d(k) is the desired sample which we would like to estimate, αn is the distortion caused
by the filter at stage n on the desired sample and zn(k) is the AWGN contribution which is independent to the signal and modeled as uncorrelated from one filter to another according
to
2 2 σ ||wi|| ; i = j E[zi(k)zj(k)] = 0; otherwise,
where σ2 is the average noise power and the error signal at the output for filter n is
en(k) = d(k) − yn(k). (4.41)
Substituting (4.40) into (4.41) yields
en(k) = d(k) − αnd(k) + zn(k), (4.42)
which can also be expressed as
en(k) = (1 − αn) d(k) − zn(k). (4.43)
64 Since zi(k) is an uncorrelated noise sequence for i 6= j, it is easy to show that
2 2 2 2 E[(1 − αi) d (k) + σ ||wi|| ]; i = j E[ei(k)ej(k)] = (4.44) 2 E[(1 − αi)(1 − αj)d (k)]; i 6= j.
The desired signal sample can be expressed as
d(k) = and(k) +a ¯nd(k). (4.45)
Expanding (4.45), we can write the following equalities
d(k) = and(k) +a ¯nan−1d(k) +a ¯na¯n−1d(k)
= and(k) +a ¯nan−1d(k) +a ¯na¯n−1an−2d(k)
+ ... +a ¯na¯n−1a¯n−2 ··· a¯2a1d(k) n X = An,i+1aid(k). (4.46) i=1 Now we can rewrite the error at stage n by substituting the expansion shown in (4.46) into (4.41) to yield
n n X X n(k) = An,i+1aid(k) − An,i+1aiyi(k). (4.47) i=1 i=1 Combining terms we obtain
n X n(k) = An,i+1ai(d(k) − yi(k)), (4.48) i=1 which further simplifies to
n X n(k) = An,i+1aiei(k). (4.49) i=1
65 The mean square error at stage n can be expressed as
2 E[n(k)] = E[n(k)n(k)]. (4.50)
Substituting (4.49) into (4.50) yields
n n 2 X X E[n(k)] = An,i+1An,j+1aiajE[ei(k)ej(k)]. (4.51) i=1 j=1
Using (4.44) in (4.51) exploits the fact that noise at different filter outputs are uncorrelated and zero mean resulting in
n n n 2 X X 2 2 X 2 E[n(k)] = An,i+1An,j+1aiajE[(1 − αi)(1 − αj)d (k)] + σ An,i+1ai||wi|| . (4.52) i=1 j=1 i=1
Examining the MSE at stage n in (4.52), we can see it is composed of a signal distortion component and a noise component. The last term, which is the contribution from the noise component, is the lower bound of the MSE at stage n. Next we will consider the LMS and
RLS adaptation algorithms and then use the above analysis along with those two adaptation algorithms to gain further insight into the cascaded ACA structure.
4.5 Performance
Our performance results are divided into two sections. The first section examines the cas- caded ACA structure based on the analysis presented in Section 4.4. The second section applies the cascaded ACA structure to the enhancement of the synthetic ECG. In both sec- tions we implement the LMS and RLS adaptation algorithms. We will first briefly review these algorithms described in more detail in Chapter 2
66 4.5.1 Least Mean Squares
The LMS algorithm uses a simple and robust gradient descent method that minimizes the MSE cost function[17] defined in (2.17). Widrow et al. have shown that by utilizing gradient
descent to minimize the MSE, the optimum coefficients wopt, also known as the Wiener filter solution is achieved [55]. The LMS algorithm implements gradient descent through an iterative weight update given by
wn(k) = wn(k − 1) − 2µen(k)xn(k), (4.53)
where, 0 ≤ µ ≤ 1/λmax, is the step size and λmax is the largest eigenvalue of the input autocorrelation matrix. The LMS adaptive algorithm is simple, robust and well known. We use it to baseline the performance analysis.
4.5.2 Recursive Least Squares
The RLS algorithm minimizes the WLS cost function [17] defined in (2.18). Similar to LMS, the RLS algorithm implements an iterative weight update given by
wn(k) = wn(k − 1) − K(k)e(k), (4.54)
where K(k), is defined as
P(k − 1)xn(k) K(k) = > , (4.55) λ + xn (k)P(k − 1)xn(k) and where 0 ≤ λ ≤ 1 denotes the forgetting factor and P(k), is defined as
1 P(k) = [P(k − 1) − K(k)x>(k)P(k − 1)]. (4.56) λ n The RLS adaptive algorithm converges faster and tracks better than LMS, but it is not as robust. RLS will be used to evaluate tracking of the quasi-periodic nature of the ECG signal and compared to the LMS baseline algorithm.
67 4.5.3 Numerical Results
We examine the cascaded ACA structure based on the analysis presented in Section 4.4. In order to obtain the optimum filter at stage n expressed by (4.35) and the MSE at stage n in (4.52), we setup an experimental environment as follows. We use an autoregressive moving average (ARMA) model to generate the WSS signal and this signal is received in the presence of AWGN. We filter a white noise process with a 10th order Butterworth low pass filter, with a passband of 0.10 to generate the WSS signal. Next we create a single input vector and the corresponding autocorrelation matrix for each stage. We then calculate the optimum filter for n = 2,..., 8 using (4.35). Using the optimum filters for each stage, we numerically determine the MSE and derive the corresponding lowerbound, which is the last term in (4.52). To implement ACA we use an = 1/n, for the LMS a step size µ = 0.05 and for the RLS, a forgetting factor λ = 0.997 with the initial covariance matrix P(k) = (1/δ)I, where δ = 1. First we calculate the MSE (normalized by the average signal power) for each of the 8 stages of ACA along with the corresponding lowerbound, where the input SNR = −5dB. The performance of the LMS and RLS adaptation algorithms are also calculated. Results are shown in Fig. 4.7- 4.14. Here we can observe that the benefit of adding an ACA stage has a diminishing return in terms of MSE as the number of stages increases. This can be seen by comparing the decrease in MSE from stage 1 to stage 2 with the decrease in MSE from stage 2 to stage 3. In addition, the difference between the MSE and the corresponding lowerbound diminishes as the number of stages increases. It can also be observed that RLS and LMS perform closer to the lowerbound with the increase in the number of stages. The above experiment is also performed for input SNR = 5dB. Results are shown in Fig. 4.15- 4.22. Comparing these results with the previous results for input SNR = −5dB, we can infer that the difference between the simulated MSE and the lowerbound is composed of both the signal distortion component and the noise component. When SNR = −5dB, the
68 signal power is low, allowing the simulated MSEs to converge closer to the lowerbound as shown in Fig. 4.14. However, when the SNR = 5dB, the signal power is high, the simulated MSEs are noticeably above the lowerbound due to the signal distortion component, as shown in Fig. 4.22. An important observation is that MSEs are inversely related to the input SNR. This can be seen by comparing MSEs of stage n for different SNR. In the next section we will apply the ACA structure to the enhancement of the synthetic ECG.
4.5.4 Synthetic ECG Results
Although consecutive ECG complexes resemble each other, the quasi-periodic nature of the ECG introduces a varying delay between complexes. This requires the adaptive algorithm to continuously adjust the filter coefficients, otherwise known as tracking. For our analysis, we implement the LMS and RLS adaptation algorithms. The output SNDR was the criteria chosen for analyzing the performance of the ACA structure. The output SNDR at stage n is defined as
! E[s2(k)] SNDR = 10 log , (4.57) n 10 E[(s(k) − sˆ(k))2] where, s(k) is the uncorrupted ECG ands ˆ(k) is the enhanced ECG given by the cascade
output cn(k) in (4.23). We use the synthetic ECG model introduced by McSharry et al. [27] in order to attain an uncorrupted ECG signal. This model produces realistic quasi-periodic ECG complexes based on three coupled ordinary differential equations. A comparison of the cascaded ACA results for both LMS and RLS adaptation algorithms are shown in Table 4.1. It is clear that RLS provides a greater initial gain for the adaptive predictor, which becomes more noticeable as the SNR increases. However, LMS provides the better multi-stage ACA gain, whereas RLS appears to remain relatively flat.
69 Table 4.1. The output SNDR for different input SNR of synthetic ECG and AWGN. Output SNDR (dB) Adaptive Predictor 2-Stage ACA 4-Stage ACA 8-Stage ACA Input SNR (dB) LMS RLS LMS RLS LMS RLS LMS RLS -10 1.79 2.33 2.02 2.25 2.04 2.21 2.36 2.52 -5 5.22 5.28 5.23 5.21 5.23 5.20 5.70 5.66 0 8.76 8.75 8.95 8.72 9.35 8.72 10.23 9.25 5 11.94 12.17 12.44 12.20 13.39 12.22 14.96 12.57 10 14.17 15.68 14.84 15.93 16.39 16.17 19.16 16.22
The results of Table 4.1 lead us to modify the cascade structure from a homogenous LMS or RLS structure to one that takes advantage of the algorithm strengths observed in the prior paragraph. The modified hybrid structure uses RLS in the first block of Fig. 4.6 and LMS in all remaining ACA blocks. Results from this hybrid configuration are shown in Table 4.2.
Table 4.2. The output SNDR for different input SNR of synthetic ECG and AWGN. Output SNDR (dB) Input SNR (dB) Adaptive Predictor 2-Stage ACA 4-Stage ACA 8-Stage ACA -10 2.33 2.28 2.25 2.56 -5 5.28 5.23 5.28 5.83 0 8.75 8.91 9.29 10.23 5 12.17 12.74 13.55 15.01 10 15.68 16.71 17.75 19.80
4.6 Conclusion
In this chapter, we reviewed the adaptive predictor and proposed a novel variant that in- troduced an averaging output. This new variant called ACA is both scalable and suitable for implementation into hardware. To understand the cascaded structure, we performed an analysis to derive the optimum filter and the MSE of each stage for a WSS signal in the pres- ence of AWGN. For each stage a lowerbound is also derived. These analytical results were used as the baseline for the cascaded ACA structure implemented with both the LMS and
70 RLS adaptation algorithms. Our numerical results demonstrated that the ACA structure was capable of outperforming the existing adaptive predictor. As expected, the cascading gain of ACA diminishes with the number of stages. We then consider the performance of the cascaded ACA structure for the enhancement of the synthetic ECG, a quasi-stationary signal, in the presence of AWGN. The RLS algorithm provided greater initial gain, while the LMS algorithm provided better multi-stage gain. This observation leads us from a homogenous LMS or RLS structure to a hybrid RLS-LMS structure that takes advantage of each algorithm strengths. Our results for the synthetic ECG signal showed that ACA consistently outperformed the existing adaptive predictor.
71 5
0 LMS RLS −5 MSE Lowerbound
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.7. The mean square error for the ACA Stage 1 (SNR = −5dB).
5 LMS RLS MSE 0 Lowerbound
−5
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.8. The mean square error for the ACA Stage 2 (SNR = −5dB).
72 5 LMS RLS MSE 0 Lowerbound
−5
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.9. The mean square error for the ACA Stage 3 (SNR = −5dB).
5 LMS RLS MSE 0 Lowerbound
−5
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.10. The mean square error for the ACA Stage 4 (SNR = −5dB).
73 5 LMS RLS MSE 0 Lowerbound
−5
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.11. The mean square error for the ACA Stage 5 (SNR = −5dB).
5 LMS RLS MSE 0 Lowerbound
−5
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.12. The mean square error for the ACA Stage 6 (SNR = −5dB).
74 5 LMS RLS MSE 0 Lowerbound
−5
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.13. The mean square error for the ACA Stage 7 (SNR = −5dB).
5 LMS RLS MSE 0 Lowerbound
−5
−10 MSE (dB)
−15
−20
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.14. The mean square error for the ACA Stage 8 (SNR = −5dB).
75 5
0 LMS −5 RLS −10 MSE Lowerbound −15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.15. The mean square error for the ACA Stage 1 (SNR = 5dB).
5 LMS RLS 0 MSE −5 Lowerbound
−10
−15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.16. The mean square error for the ACA Stage 2 (SNR = 5dB).
76 5 LMS 0 RLS MSE −5 Lowerbound −10
−15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.17. The mean square error for the ACA Stage 3 (SNR = 5dB).
5 LMS 0 RLS MSE −5 Lowerbound −10
−15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.18. The mean square error for the ACA Stage 4 (SNR = 5dB).
77 5 LMS 0 RLS MSE −5 Lowerbound −10
−15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.19. The mean square error for the ACA Stage 5 (SNR = 5dB).
5 LMS 0 RLS MSE −5 Lowerbound −10
−15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.20. The mean square error for the ACA Stage 6 (SNR = 5dB).
78 5 LMS 0 RLS MSE −5 Lowerbound −10
−15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.21. The mean square error for the ACA Stage 7 (SNR = 5dB).
5 LMS 0 RLS MSE −5 Lowerbound −10
−15
−20
−25 MSE (dB) −30
−35
−40
−45
−50
0 2000 4000 6000 8000 10000 Iteration Number (k)
Figure 4.22. The mean square error for the ACA Stage 8 (SNR = 5dB).
79 CHAPTER 5
CONCLUSIONS
5.1 Summary
In this dissertation, we have shown significant contributions toward the use of adaptive predictors for extracting and enhancing physiological signals in two modern bioinstruments.
An adaptation algorithm, an arithmetic format and an adaptive structure were proposed and analyzed toward the advancement of adaptive predictor implementations in real-time hardware.
In Chapter 3, a proposed Kalman adaptation algorithm was developed for the predic- tion of physiological hand tremor as a source of interference limiting precision during mi- crosurgery. It was demonstrated that the proposed Kalman outperforms RLS with faster convergence and lower mean square error. It also offered more robustness to finite precision, as the word length is reduced. Additionally, we implemented both predictors using fixed- point arithmetic. The fixed-point results showed a need for increased dynamic range and improved precision due to the observed explosive divergence and stalling effect. To address this issue we proposed and implemented a 24 bit Binary Floating-Point format. Analysis of both fixed and floating-point precision were presented.
In Chapter 4, an ACA structure was proposed for enhancing the ECG when corrupted by broadband noise while preserving the morphological features of the signal. The cascade nature of the proposed ACA structure is both scalable and suitable for implementation in real-time hardware. Analysis was performed to derive the optimum filter and the MSE at stage n for the cascaded ACA structure. This analysis was used to demonstrate the cascading benefit over the adaptive predictor structure. The proposed ACA was implemented using both the LMS and RLS adaptation algorithms and shown to outperform the adaptive predictor in terms of SNDR.
80 These contributions have introduce a new scalable variant of the adaptive filter and opened the door for future adaptive predictor implementations in real-time hardware.
5.2 Future Works
There are multiple bioinstruments where this dissertation can be extended and have potential to provide significant improvement. The first is for the real-time removal of the maternal ECG in a fetal electrocardiogram (fECG). Because of the quasi-periodic nature of both signals, the ACA structure is well suited for the application. The second is for the detection of low level somatosensory evoked potentials (SEP) in a real-time array. Because the ACA structure is able to be efficiently implemented in hardware using the 24 bit Binary Floating-Point format and has low latency, it is highly capable of supporting a real-time array implementation. In addition to applications toward bioinstruments, this dissertation could be used toward defense applications such as unknown pulse detection in sonar and coherent averaging of pulses for radar. We leave this ideas along with the above ideas for future research.
81 REFERENCES
[1] Adali, T. and S. H. Ardalan (1991). Analysis of a stabilization technique for the fixed- point prewindowed rls algorithm. IEEE Trans. Signal Processing 39 (9), 2079–2082.
[2] Alexander, S. T. and J. R. Zeidler (1977, Dec). Detection of narrowband signals in noise using adaptive linear prediction filters. pp. 519–528.
[3] Anderson, C., E. Satorius, and J. Zeidler (1983, February). Adaptive enhancement of finite bandwidth signals in white gaussian noise. IEEE Transactions on Acoustics, Speech, and Signal Processing 31 (1), 17–28.
[4] Barros, A. K., A. Mansour, and N. Ohnishi (1998). Removing artifacts from electro- cardiographic signals using independent components analysis. Neurocomputing 22 (1), 173–186.
[5] Board, I. S. (2008). IEEE standard for binary floating-point arithmetic. Technical report.
[6] Bottomley, G. and S. Alexander (1991). A novel approach for stabilizing recursive least squares filters. IEEE Trans. Signal Process. SP-39 8, 1770–1779.
[7] Castells, F., P. Laguna, L. S¨ornmo,A. Bollmann, and J. Roig (2007, Dec). Principal component analysis in ecg signal processing. EURASIP Journal on Advances in Signal Processing 2007 (1), 074580.
[8] Ciochina, S., C. Paleologu, and A. Enescu (2003). On the behavior of rls adaptive algorithm in fixed-point implementation. IEEE International Symposium on Circuits and Systems 1, 57–60.
[9] Cyrill, D., J. McNames, and M. Aboy (2003, Sept). Adaptive comb filters for quasiperi- odic physiologic signals. 3, 2439–2442 Vol.3.
[10] Das, H., H. Zak, J. Johnson, J. Crouch, and D. Frambach (1999). Evaluation of a telerobotic system to assist surgeons in microsurgery. Computer Aided Surg. 4 (7), 15– 25.
[11] Elble, R. J. and W. C. Koller (1990). Tremor. Baltimore: Johns Hopkins University Press.
[12] Eldon, J. and C. Robertson (1982). A floating point format for signal processing. IEEE.
[13] Ferrara, E. and B. Widrow (1981, June). The time-sequenced adaptive filter. IEEE Transactions on Circuits and Systems 28 (6), 519–523.
82 [14] Friesen, G., T. Jannett, M. Jadallah, S. Yates, S. Quint, and H. Nagle (1990, Jan). A comparison of the noise sensitivity of nine qrs detection algorithms. IEEE Transactions on Biomedical Engineering 37 (1), 85–98.
[15] Godard, D. (1974). Channel equalization using a Kalman filter for fast data transmis- sion. IBM Journal of Research Development 18 (3), 267–273.
[16] Govindu, G., L. Zhuo, S. Choi, and V. Prasanna (2004). Analysis of high-performance floating-point arithmetic on FPGAs. IEEE International Parallel and Distributed Pro- cessing Symposium, 26–30.
[17] Haykin, S. (2002). Adaptive Filter Theory (4 ed.). Prentice-Hall.
[18] He, T., G. Clifford, and L. Tarassenko (2006). Application of independent component analysis in removing artefacts from the electrocardiogram. Neural Computing and Ap- plications 15 (2), 105–116.
[19] Hunter, I. W., T. D. Tilemachos, S. R. Lafontaine, P. G. Charette, L. A. Jones, and et al. (1993). A teleoperated microsurgical robot and associated virtual environment for eye surgery. Presence 2, 265–280.
[20] Hyv¨arinen,A. and E. Oja (2000). Independent component analysis: algorithms and applications. Neural networks 13 (4), 411–430.
[21] Kalman, R. E. (1960). A new approach to linear prediction and filtering theory. Trans- actions of the ASME–Journal of Basic Engineering 81, 35–45.
[22] Kalman, R. E. and R. S. Bucy (1961). New results in linear filtering and prediction theory. Transactions of the ASME–Journal of Basic Engineering 95, 107–115.
[23] Karlstrom, P., A. Ehliar, and D. Liu (2008). High-performance low-latency field- programmable gate array-based floating-point adder and multiplier units in a Virtex 4. IET Computers and Digital Techniques 2 (4), 305–313.
[24] Li, C., C. Zheng, and C. Tai (1995, Jan). Detection of ecg characteristic points using wavelet transforms. IEEE Transactions on Biomedical Engineering 42 (1), 21–28.
[25] Ligon, W., S. McMillan, G. Monn, K. Schoonover, F. Stivers, and K. Underwood (1998). A re-evaluation of the practicality of floating-point operations on FPGAs. IEEE Sym- posium on FPGAs for Custom Computing Machines, 206.
[26] Louca, L., T. Cook, and W. Johnson (1996). Implementation of IEEE single precision floating point addition and multiplication on FGPAs. Proc. IEEE Symp. FPGAs for Custom Computing Machines.
83 [27] McSharry, P. E., G. D. Clifford, L. Tarassenko, and L. A. Smith (2003). A dynam- ical model for generating synthetic electrocardiogram signals. IEEE Transactions on Biomedical Engineering 50 (3), 289–294.
[28] Papoulis, A. and S. U. Pillai (2002). Probability, Random Variables, and Stochastic Processes (4 ed.). McGraw Hill.
[29] Paschalakis, S. and P. Lee (2003). Double precision floating-point arithmetic on FPGAs. IEEE International Conference on Field-Programmable Technology, 352–358.
[30] Popescu, D. C. and I. Zeljkovic (1998). Kalman filtering of colored noise for speech enhancement. IEEE International Conference on Acoustics Speech and Signal Process- ing 2, 997–1000.
[31] Popescu, O., M. Saquib, D. C. Popescu, and M. D. Rice (2007). Interference mitigation in aeronautical telemetry systems using Kalman filter. IEEE Trans. Aerospace and Electronic Systems 43 (4), 16241630.
[32] Randall, J. E. (1973). A stochastic time series model for hand tremor. J Appl Phys- iol 34 (3), 390–395.
[33] Rickard, J. and J. Zeidler (1979, February). Second-order output statistics of the adaptive line enhancer. IEEE Transactions on Acoustics, Speech, and Signal Process- ing 27 (1), 31–39.
[34] Riviere, C. N., W. T. Ang, and P. K. Khosla (2003). Toward active tremor canceling in handheld microsurgical instruments. IEEE Trans. Robotics and Automation 19 (5), 793–800.
[35] Riviere, C. N., R. S. Radder, and N. V. Thakor (1998). Adaptive canceling of physio- logical tremor for improved precision in microsurgery. IEEE Trans. Biomedical Engi- neering 45 (7), 839–846.
[36] Robinson, B. W., D. Hernandez-Garduno, and M. Saquib (2010a, March). Fixed and floating point analysis of linear predictors for physiological hand tremor in microsurgery. 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 578– 581.
[37] Robinson, B. W., D. Hernandez-Garduno, and M. Saquib (2010b, July). Fixed and floating-point implementations of linear adaptive techniques for predicting physiological hand tremor in microsurgery. IEEE Journal of Selected Topics in Signal Processing 4, 659 – 667.
[38] Robinson, B. W. and M. Saquib (2014, Aug). Adaptive coherent averaging for real- time electrocardiogram enhancement. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 5048–5051.
84 [39] Salcic, Z., J. Cao, and S. K. Nguang (2006). A floating-point FPGA-based self-tuning regulator. IEEE Trans. Industrial Electronics 53 (2), 693–704.
[40] Sayed, A. H. (2008). Adaptive Filters. Wiley.
[41] Shirazi, N., A. Walters, and P. Athanas (1995). Quantitative analysis of floating point arithmetic on FPGA based custom computing machines. Proc. IEEE Symposium on FPGA’s for Custom Computing Machines, 155.
[42] Singh, S. P. N. and C. N. Riviere (2002). Physiological tremor amplitude during retinal microsurgery. In Proc. IEEE 28th Annu. Northeast Bioeng. Conf., pp. 171–172.
[43] Spyers-Ashby, J. M., P. G. Bain, and S. J. Roberts (1998). A comparison of fast fourier transform and autoregressive spectral estimation techniques for the analysis of tremor data. Journal of Neuroscience Methods 83, 35–43.
[44] Tang, W. M. and D. P. Han (2000). A study of surgical approaches to retinal vascular occlusions. Archives of Ophthalmology 118, 138–143.
[45] Taylor, R., P. Jensen, L. Whitcomb, A. Barnes, R. Kumar, and et al. (1999). A steady- hand robotic system for microsurgical augmentation. Medical Image Computing and Computer-Assisted Intervention Springer, 1031–1041.
[46] Thakkar, A. and A. Ejnioui (2006). Design and implementation of double precision floating point division and square root on FPGAs. IEEE Aerospace Conference.
[47] Thakor, N. and Y.-S. Zhu (1991, Aug). Applications of adaptive filtering to ecg anal- ysis: noise cancellation and arrhythmia detection. IEEE Transactions on Biomedical Engineering 38 (8), 785–794.
[48] Timmer, J. (2009). Bivariate tremor time series. http://webber.physik.uni-freiburg.de/~jeti/FSP/SSP_tremor.html.
[49] Timmer, J., M. Lauk, S. Hauler, V. Radt, B. Kster, and et al. (2000). Cross-spectral analysis of tremor time series. Int J Bif Chaos 10, 2595–2610.
[50] Treichler, J. (1979, February). Transient and convergent behavior of the adaptive line enhancer. IEEE Transactions on Acoustics, Speech, and Signal Processing 27 (1), 53–62.
[51] Vaz, C., X. Kong, and N. Thakor (1994, Jan). An adaptive estimation of periodic signals using a fourier linear combiner. IEEE Transactions on Signal Processing 42 (1), 1–10.
[52] Vaz, C. A. and N. V. Thakor (1989, April). Adaptive fourier estimation of time-varying evoked potentials. IEEE Transactions on Biomedical Engineering 36 (4), 448–455.
85 [53] Venishetti, S. and A. Akoglu (2007). A highly parallel FPGA based IEEE-754 compli- ant double-precision binary floating-point multiplication algorithm. IEEE International Conference on Field-Programmable Technology, 145–152.
[54] Weiss, J. N. (1998). Treatment of central retinal vein occlusion by injection of tissue plasminogen activator into a retinal vein. Am. J. Ophtalmol 126, 142–144.
[55] Widrow, B., J. R. Glover, J. M. McCool, J. Kaunitz, C. S. Williams, R. H. Hearn, J. R. Zeidler, J. E. Dong, and R. C. Goodlin (1975). Adaptive noise cancelling: Principles and applications. Proceedings of the IEEE 63 (12), 1692–1716.
[56] Widrow, B. and M. E. Hoff. Adaptive switching circuits. 1960 IRE WESCON Conven- tion Record, 96–104.
[57] Yelderman, M., B. Widrow, J. Cioffi, E. Hesler, and J. A. Leddy (1983, July). Ecg enhancement by adaptive cancellation of electrosurgical interference. IEEE Transactions on Biomedical Engineering BME-30 (7), 392–398.
[58] Zeidler, J., E. Satorius, D. Chabries, and H. Wexler (1978, June). Adaptive enhancement of multiple sinusoids in uncorrelated noise. IEEE Transactions on Acoustics, Speech, and Signal Processing 26 (3), 240–254.
[59] Zhang, J. and F. Chu (2005). Real-time modeling and prediction of physiological hand tremor. IEEE International Conference on Acoustics Speech and Signal Processing 5 (1), 645–648.
[60] Zhang, J., F. Chu, and N. Mohammed (2005). Adaptive signal processing of physiolog- ical hand tremor. Proc. IEEE American Control Conference 2005 3, 1569–1574.
86 BIOGRAPHICAL SKETCH
Brent W. Robinson received his BS degrees in Computer Engineering and Electrical Engi- neering in 2003 and his MS degree in Electrical Engineering in 2005 from North Carolina State University, Raleigh, NC. In 2005 he joined Raytheon where he is currently a Systems Engineer. He is pursuing his PhD degree in Electrical Engineering at The University of Texas at Dallas.
His research interests include design, analysis and implementation of algorithms in both defense and biomedical applications.
87 CURRICULUM VITAE
Brent W. Robinson July 16, 2019
Educational History: B.S., Electrical Engineering, North Carolina State University, 2003 B.S., Computer Engineering, North Carolina State University, 2003 M.S., Electrical Engineering, North Carolina State University, 2005 Ph.D., Electrical Engineering, The University of Texas at Dallas, 2019
Employment History: Systems Engineer, Raytheon, May 2005 – Present Electrical Engineer, Lockheed Martin, May 2001 – May 2005
Professional Memberships: Institute of Electrical and Electronics Engineers (IEEE), 2005 – present