ANALYSIS OF ADAPTIVE VOLTERRA FILTERS Analysis of Adaptive Volterra Filters with LMS and RLS Algorithms Amrita Rai and Dr. Amit Kumar Kohli Department of Electronics and Communication Engineering, Thapar University, Patiala 147004 Punjab [email protected], [email protected] ______
Abstract -- Linear filters have played a very crucial role in real world applications. While Volterra filters have been the development of various signal processing techniques applied in many applications, they still present some and are relatively simple from conceptual and limitations because of their computational complexity which implementation view points. However, there are several increases exponentially with the filter order. When the situations in which the performance of linear filters is nonlinear system order is unknown, adaptive methods and unacceptable. At that situation adaptive polynomial filters algorithms are widely used for the Volterra kernel estimation. are used which perform satisfactorily. Adaptive Polynomial filters are a nonlinear generalization of Accuracy of the Volterra kernels will determine the accuracy adaptive linear filters that are based on nonrecursive and of the system model and the accuracy of the inverse system recursive linear difference equations. Polynomial filters used for compensation. The speed of kernel estimation often refer to as Volterra filter when input and output process is also important. A fast kernel estimation method signals are related through the Volterra series expansion. may allow the user to construct a higher order model that In a non-stationary or time-varying environment, the gives an even better system representation. adaptive polynomial filter helps track the statistics of the input data or the dynamics of a system. This article Recently, more or less general representations of the Volterra explains details about adaptive Volterra filter with filter with its truncated version have received increasing different algorithms such as LMS (Least Mean Square ) attention in nonlinear signal processing fields. There are and RLS ( Recursive Least Square). Also discussed are two important properties of the Volterra filter that can the current research areas and problems associated with further account for the attention paid to such nonlinear the nonlinear adaptive filters. structures [4-5].
Keywords: - Volterra Series, Least Mean Square, Recursive 1. One important property relies on the fact that the output Least Square, System Identification. of a Volterra filter depends linearly on the coefficients of the filter itself. In other words, the Volterra filter may I. INTRODUCTION be interpreted as extension of linear filters to the ADAPTIVE filters are used in various areas where the nonlinear case. Therefore, many linear filters with the statistical knowledge of the signals to be filtered /analyzed is corresponding adaptive algorithms can be extended to the not known a priori or the signal may be slowly time-invariant. polynomial filters. Moreover, this characteristic can be In Adaptive filtering, the adjustable filter parameters are to be largely used to analyze quadratic filters, to find new optimized. The notion of making filters adaptive, i.e., to alter implementations. parameters (coefficients) of a filter according to some 2. Another interesting property results from the algorithm, tackles the problems that we might not in advance representation of the nonlinearity by means of know, e.g., the characteristics of the signal, or of the unwanted multidimensional operators working on products of input signal, or of a system influence on the signal that we like to samples. Such characteristic allows for the description of compensate. Adaptive filters can adjust to unknown the filter behavior in the frequency domain by means of a environment, and even track signal or system characteristics type of multidimensional convolution. varying over time [1-4]. The current trend in the telecommunication systems design is General characteristics of adaptive filters: the identification and compensation of unwanted 1. Automatically adjustable: adapt in a changing system. nonlinearities. It was demonstrated that unwanted 2. Can perform specific filtering or decision-making. nonlinearities in the system will have a detrimental effect on 3. Have adaptation algorithms for adjusting parameters. its performance. There are various ways of reducing the effects of undesired nonlinearities. The use of nonlinear Over the last decade, Volterra filters or polynomial filters and models to characterize and compensate harmful nonlinearities nonlinear adaptive infinite impulse-response filters have been offers a possible solution. The Volterra series have been appealing areas of research and have been considered in many widely applied as nonlinear system modeling technique with
9 AKGEC JOURNAL OF TECHNOLOGY, Vol. 2, No. 1 Therefore, estimating coefficients becomes nonlinear considerable success. However, at present, no one general estimation problem about the global optimal. Moreover, stable method exists to calculate the Volterra kernels for nonlinear and convergence performance of adaptive algorithm cannot be systems, although they can be calculated for systems whose settled by these architectures. Recently, two nonlinear blind order is known and finite. adaptive interference cancellation algorithms (exact Newton and approximate Newton algorithms) based on the second- II. VOLTERRA FILTER THEORY order Volterra expansion were proposed and developed to The Volterra filter theory was first studied and developed by overcome the multiple access interference [10]. Wiener, who mainly worked on the analysis of nonlinear systems using white Gaussian input and so-called G functions. III. VOLTERRA SERIES EXPANSION FOR Following his works, many researchers used the truncated NONLINEAR SYSTEMS Volterra series for estimation and identification of nonlinear Let x[n] and y[n] represent the input and output signals, systems [4]. However, for higher order or memory Volterra respectively, of a discrete-time and causal nonlinear system. filters, large amount of computational burdens are prohibitive The Volterra series expansion for y[n] using x[n] is given by: for most practical applications. yn h0 h1 m1 xn m1 To overcome the computational complexity, Koh and Powers m1 0 propose an iterative factorization technique to design a subclass of the SOV filters which can alleviate the complexity h2 m1, m2 xn m1 xn m2 ... of the filtering operations considerably and apply to nonlinear m1 0 m2 0 system identification for Gaussian input signals. Furthermore, ... h m , m , ..., m x n m x n m ... x n m ...) reduction implementation of computational loads is proposed p 1 2 p 1 2 p m 0 m 0 m 0 that are composed of a square with subsequent linear filtering 1 2 p (1) or of two linear filters whose outputs are multiplied. In (l), hp [ml, m2,..., mp] is known as the p-the order Volterra kernel of the system. Without any loss of generality, one can A more general approximation to the quadratic filter is assume that the Volterra kernels are symmetric, i.e., h [m , investigated, which is called multi memory decomposition p l m . ..., m ] is left unchanged for any of the possible p! (MMD) and is composed of three linear filters connected by a 2 p Permutations of the indices m , m ..., m . One can think of the multiplier [11]. With these structures, though the l 2, p Volterra series expansion as a Taylor series expansion with computational complexity can be reduced significantly memory. The imitations of the Volterra series expansion are compared with the direct-form SOV filter, the stable similar to those of the Taylor series expansion both performance is not guaranteed, and the drawbacks of these expansions do not do well when there are discontinuities in approaches are that the decomposition of Volterra series is not the system description. Volterra series expansion exists for unique in the identification procedures. systems involving such type of nonlinearity. Even though clearly not applicable in all situations, Volterra system models Moreover, based on the alternative adaptation of the have been successfully employed in a wide variety of coefficients of the linear filters, a problem of local minima applications, and such models continue to be popular with may exist. Consequently, the convergence is not easily researchers in this area. established, especially for higher-order kernels. An alternative Among the early works on nonlinear system analysis is a very very effective method based on a parallel-cascade structure for important contribution by Wiener. His analysis technique adaptive Volterra filter is first proposed by applying singular involved white Gaussian input signals and used value (SV) decomposition to the coefficient matrix in order to “G-functionals” to characterize nonlinear system behavior. obtain an approximation based on its most important Following his work, several researchers employed Volterra eigenvectors [5]. series expansion and related representations for estimation and time-invariant or time variant nonlinear system identification. Volterra filter using the normalized least mean square Since an infinite series expansion like (1) is not useful in (NLMS) to reduce its implementation complexity by using filtering applications, one must work with truncated Volterra fewer than the maximum number of branches required [7]. series expansions. M.M. Banat proposed a pipelined Volterra filter utilizing the recursive equation, and a pipelined implementation of IV. VOLTERRA KERNELS ESTIMATION BY THE LMS quadratic adaptive Volterra filters based on NLMS algorithm ADAPTIVE ALGORITHM was presented. Though these can effectively reduce The Volterra filter of fixed order and fixed memory adapts to complexity of the implementation structure, the output of the the unknown nonlinear system using one of the various system becomes a nonlinear function of the filter coefficients [8].
10 adaptive algorithms. The use of adaptive techniques for T Volterra kernel estimation has been well published. Most of H nh10, h11,....,h1N 1,h2 0,0,h2 0,1,....hQ N 1,...., N 1 (6) the previous work considers 2nd order Volterra filters. A Thus, (6) Volterra Filter input and output can be compactly simple and commonly used algorithm uses an LMS adaptation rewritten as
T yn H nX e n (7)
The error signal e(n) is formed by subtracting y(n) from the noisy desired response d(n), i.e.,
T endn yn dn H nX e n (8)
For the LMS algorithm we minimize the Eq. (7)
2 T Ee n Edn H nX e n (9) Figure 1. A Block Diagram of Adaptive Volterra Filter. The well Known LMS update equation for a first order filter is criterion. Though the LMS algorithm has its weaknesses, such as its dependence on signal statistics, which can lead to Hn 1 Hn en X e n (10) low speed or large residual errors, it is very simple to implement and well behaved compared to the faster recursive where µ is small positive constant (referred to as the step size) algorithms [4]. A typical adaptive technique used for Volterra that determines the speed of convergence and also affects the kernels identification is shown in Fig.2 final error of the filter output [5]. The extension of the LMS algorithm to higher order (nonlinear) Volterra filters involves a few simple changes. Firstly the vector of the impulse response coefficients becomes the vector of Volterra kernels coefficients. Also the input vector, which for the linear case contained only a linear combination, for nonlinear time varying Volterra filters, complicates treatment.
V. VOLTERRA KERNEL ESTIMATION USING THE RLS ADAPTIVE ALGORITHM The RLS (recursive least squares) algorithm is another algorithm for determining the coefficients of an adaptive filter. In contrast to the LMS algorithm, the RLS algorithm Figure 2. Volterra Kernel Identification by Adaptive LMS uses information from all past input samples (and not only from the current tap-input samples) to estimate the (inverse Method. of the) autocorrelation matrix of the input vector. This section is to discuss the extension of the algorithm to the nonlinear case using the previously defined input vectors. The To decrease the influence of input samples from the far past, a weighting factor for the influence of each sample is discrete time impulse response of a first order (linear) system with memory span is aggregate of all the N most recent inputs used. A typical adaptive technique is shown in Figure 3. and their nonlinear combinations into one expanded input vector as The Volterra filter of a fixed order and a fixed memory adapts to the unknown nonlinear system using one of the various
T adaptive algorithms. The use of adaptive techniques for X n xn, xn 1,...., xn N 1, x 2 n xnxn 1,...., xQ n N 1 e Volterra kernel estimation has been well studied. Most of the (5) previous research considers 2nd order Volterra filters and Similarly, the expanded filter coefficients vector H(n) is given some consider the 3rd order case [15]. by ANALYSIS OF ADAPTIVE VOLTERRA FILTERS AKGEC JOURNAL OF TECHNOLOGY, Vol. 2, No. 1
11 H[n] can be recursively updated by realizing that
Cn C n 1 X n X T n (15)
and Pn Pn 1 dn X n (16)
One can simplify the computational complexity a little bit by making use of the matrix inversion lemma for inverting C[n]. This will result in the algorithm given below in eq.17. The derivation is similar to that for the RLS linear adaptive filter [15,2].
C 1n 1C 1 n 1 1knX T nC 1n 1 (17)
Figure 3. Volterra Kernal Identification by Adaptive RLS Method. VI. SIMULATION RESULTS A simple and commonly used algorithm is based on the LMS In this section, we examine both adaptive second order adaptation criterion. Adaptive Volterra filters based on the Volterra LMS filter and adaptive second order Volterra RLS LMS adaptation algorithm are computational simple but suffer filter. The left side graph of the figure 4 and 5 shows the from slow and input signal dependant convergence behavior adaptive filter coefficients after convergence which is almost and hence are not useful in many applications [15]. identical to the unknown filter h. As in the linear case, the adaptive nonlinear system minimizes The right side graph shows the square error in dB versus time the following cost function at each time: during the adaptation process. The lower limit of the error signal power in the learning curve is defined here by the n 2 Jn nk dk H T nX k (11) additive white noise added at the filter output (-60 dB). k 0 where, H(n) and X (n) are the coefficients and the input signal In figure 4, Sample per sample filtering and coefficient update vectors, respectively, as defined in (4) and (5), is a factor using the Second Order Volterra Least Mean Squares or one that controls the memory span of the adaptive filter and d(k) of its variants. The LMS second order Volterra filter learning represents the desired output. The solution of equation (11) curve shows satisfactory results. can be obtained each time can be easily found by differentiating J[n] with respect to H[n], setting the For further improvement in learning curve, we examine derivative to zero, and solving for H[n]. The optimal solution Sample per sample filtering and coefficients update using the at time n is given by [15,2] Second Order Volterra Recursive Least Squares (SOVRLS) adaptive algorithm implements the second order Volterra RLS Hn C 1nPn (12) filter (SOVRLS). where, The SOVRLS algorithm calculates the filter output and n Cn nk X kX T k (13) updates the filter coefficients vector . The filter output is k0 the sum of the outputs of the linear filter part and the and nonlinear part as given in Appendix.
n Pnnk dk X k k0 Simulation results show improvement in second figure i.e (14) SOVRLS is more fissible for the system identification as SOVLMS.
12 ANALYSIS OF ADAPTIVE VOLTERRA FILTERS
Figure 4: (a) The adaptive filter coefficients after convergence and (b) The learning curve for the FIR system identification problem using the SOVLMS algorithm.
.
Figure 5: (a)The adaptive filter coefficients after convergence and (b) The learning curve for the FIR system identification problem using the SOVRLS algorithm. . VII. CONCLUSION In recent years, most significant work is on a comparative It is observed that Adaptive Polynomial filters are useful in evaluation of the tracking behaviors of the LMS and RLS large number of applications. Most of the Adaptive algorithm. Due to degradation in the tracking performance of Polynomial system relations with nonlinearity can relate the LMS and RLS algorithm, the Kalman filter is the optimum through a Volterra Series Expansion or a recursive nonlinear linear tracking device. In reality using the Kalman filter difference equation. The Volterra filter recently gained theory, a constant which is clearly not the way to solve the significant interest in many advanced applications, including tracking problem for a nonstationary environment. acoustic echo cancellation, Channel equalization, biological system modeling and image processing. VIII. REFERENCES [1]. Simon Haykin, “Adaptive Filter Theory”, Fourth Edition, The Volterra filter used here is either truncated Volterra series Pearson Education, 2008. or fixed order Volterra series. The least mean square LMS [2]. V. John Mathews, “Adaptive Polynomial Filters”, IEEE SP Magazine, July 1991. algorithm and the recursive least-squares RLS have [3]. John Leis, “Adaptive Filter Lecture Notes & Examples”, established themselves as principal tools for linear adaptive November 1, 2008 filtering. www.usq.edu.au/users/leis/notes/sigproc/adfilt.pdf .
13 AKGEC JOURNAL OF TECHNOLOGY, Vol. 2, No. 1
[4]. Tuncer C. Aysal and Kenneth E. Barner, “Myriad-Type Filters: Making Quadratic Filters Converge Like Linear Filters”, Polynomial Filtering”, IEEE Transactions on Signal IEEE Transactions on Signal Processing, vol. 47, no. 4, Processing, vol. 55, no. 2, February 2007. April 1999. [5]. Ezio Biglieri, Sergio Barberis, and Maurizio Catena, “Analysis and Compensation of Nonlinearities in Digital Amrita Rai is currently pursuing Transmission Systems”, IEEE Journal on selected areas in PhD in DSP & VLSI design from Communications, vol. 6, no. 1, January 1988. Thapar University, Patiala. After [6] Roberto López-Valcarce and Soura Dasgupta, “Second- obtaining BTech in ECE from Order Statistical Properties of Nonlinearly Distorted Phase- College of Engineering Chandrapur (Nagpur University) in 1998, she Shift Keyed (PSK) Signals”, IEEE Communications received MTech as a university Letters, vol. 7, no. 7, July 2003. topper from Maharshi Dayanand [7] Dong-Chul Park and Tae-Kyun Jung Jeong, “Complex- University, Rohtak in the field of Bilinear Recurrent Neural Network for Equalization of a Power Electronics & Electrical Digital Satellite Channel”, IEEE Transactions on Neural Drives. Networks, vol. 13, no. 3, May 2002. [8] John Tsimbinos and Langford B. White, “Error Earlier worked in Superior Product Propagation and Recovery in Decision-Feedback Industry Ltd. as a Design & Development Engineer for four Equalizers for Nonlinear Channels”, IEEE Transactions on years. Communications, vol. 49, no. 2, February 2001. Since 2005 she is teaching at Lingaya’s Institute of Management & [9] Christoph Krall, Klaus Witrisal, Geert Leus and Heinz Technology, Faridabad. Published papers in international journals like IFSA Koeppl, “Minimum Mean-Square Error Equalization for (International Frequency and Sensor Association Publication) and attended Second-Order Volterra Systems”, IEEE Transactions on national and international IEEE Conference. Signal Processing, vol. 56, no. 10, October 2008. [10] Alexandre Guérin, Gérard Faucon, and Régine Le Dr Amit Kumar Kohli is currently Bouquin-Jeannès, “Nonlinear Acoustic Echo Cancellation Assistant Professor in the t Based on Volterra Filters”, IEEE Transactions on Speech Department of Electronics and and Audio Processing, vol. 11, no. 6, November 2003. Communication [11] Yang-Wang Fang, Li-Cheng Jiao, Xian-Da Zhang and Jin Engineering, Thapar University, Pan, “On the Convergence of Volterra Filter Equalizers Patiala. He specializes in the area of Signal Processing and Using a Pth-Order Inverse Approach”, IEEE Transactions Wireless Communication on Signal Processing, vol. 49, no. 8, August 2001. Engineering. He obtained PhD from [12] Kenneth E. Barner and Tuncer Can Aysal, “Polynomial IIT Roorkee in 2006 and ME from Weighted Median Filtering”, IEEE Transactions on Signal Thapar University in 2002. Processing, vol. 54, no. 2, February 2006. [13] Georgeta Budura and Corina Botoca, “Efficient Received invitation from American Implementation of the Third Order RLS Adaptive Volterra Journal Experts for significant Filter”, FACTA Universitatis (NIS) Ser.: Elec. Energ. vol. contribution in the field of Signal Processing. Published a large 19, no. 1, April 2006. number of papers in national and [14] A. Zaknich, “Principal of Adaptive Filter and Self Learning international journals. System”, Springer Link –2005. He is a reviewer and member of the editorial board of journals like IEEE, [15] Charles W. Therrien, W. Kenneth Jenkins, and Xiaohui Li, Elsevier and Springer. Won several awards during his student days and “Optimizing the Performance of Polynomial Adaptive professional career.
IX. APPENDIX The filter output is the sum of the outputs of the linear filter part and the nonlinear part as follows:
14