Optimised Wiener Filtering in Overdetermined Systems

Joshua L. Sendall Warren P. du Plessis Council for Scientiﬁc and Industrial Research University of Pretoria Pretoria, South Africa Pretoria, South Africa

Abstract—The Wiener filter is prevalent in many adaptive matrix approximation is analysed. The validity of the Toeplitz filtering applications, but can also be a computationally bur- approximation on the filter’s effectiveness is analysed for filter densome part of a signal processing chain. This paper presents sizes in overdetermined systems. methods to reduce the computational complexity and memory footprint of Wiener filters where the filter is highly overdetermined. The optimisations are demonstrated in the context of passive radar and result in a reduction of over 40 times in II.WIENER FILTER processing time. Index Terms—Adaptive filter, passive radar, passive coherent location (PCL), direct path interference (DPI), clutter cancella- The Wiener filter minimizes the distance between a measured tion, radar clutter, radar signal processing. stationary process and a desired process. The exposition below is based on the outline provided by [2]. I.INTRODUCTION The filter is described by defining a measured signal as Covariance matrices are an important concept in many adap- xn = sn + zn n = 0, 1,...,N − 1 (1) tive signal processing applications. These include minimum variance distortionless response (MVDR) filtering [1], signal where N is the number of samples, and sn and zn are the cancellation [2], and space-time adaptive processing (STAP) nth samples of the desired and undesired signal components radar [3]. While these techniques can theoretically result in respectively. The output of the filter is then the estimate of the optimal performance, in practical situations, efficiently estimat- desired signal ing the covariance is still an open topic in research. Typically, sbn = xn − zbn (2) there are two aspects of covariance matrix estimation that are where s is the estimate of the desired signal, and z is the challenging. Either there is limited information and therefore b b estimated undesired signal obtained from insufficient samples to accurately estimate the covariance matrix M−1 [4], [5], or the number of samples is so large that the estimation X of the covariance matrix becomes computationally intensive. zbn = hmyn−m (3) The first case is handled using rank reduction methods m=0 [3]. A solution for reducing the computational burden for the with M the number of filter coefficients, y a template of the second case is presented in this paper. Many signal cancellation desired signal, and hm the filter weights. environments require delayed versions of a template signal to The undesired signal can be estimated with the minimum be removed, although in some instances with a frequency offset squared error by determining filter weights by solving the [6]. This structure induces redundancy, which can be exploited Wiener equations for all values of l = 0, 1,...,M − 1 at to reduce computation time and memory requirements. sample n [2], This paper presents two strategies for efficiently estimating M−1 the covariance matrix for highly overdetermined systems. The X ∗ ∗ E yn−myn−l hm = E xnyn−l (4) first exploits the redundancy in the inner product produced m=0 by the covariance matrix’s tapped delay line structure by Ryyh = Rxy (5) breaking up the matrix-matrix multiplication into a matrix- vector multiplication and a set of rank-1 updates. The second where E[·], ·∗, and ·H denote the expectation operator, complex method further exploits the redundancy by computing the conjugation, and the Hermitian transpose respectively. The matrix-vector product using fast Fourier transforms (FFTs). M × 1 vector h is formed from the filter coefficients, Ryy Both methods significantly reduce the computational time is the M × M auto-covariance matrix of y, and Rxy is the of the filter. The memory requirements of the filters are M × 1 covariance matrix of x and y. also significantly reduced as the sample matrix is implicitly In practical systems the covariance matrix is not usually represented, rather than having to be stored in memory. Then known and it is necessary to estimate the covariance matrix the computational benefit of using a Toeplitz covariance from sample data. Given N samples from the reference channel, TABLE I: The approximate computational cost of the cancellation algorithms considered.

Algorithm Computational cost 2 3 2 LU factorisation 3 M + 2NM 1 3 2 Cholesky factorisation 3 M + 2NM 2 3 2 QR factorisation − 3 M + 2NM CGLS (P + 1)(4NM) it is possible to form the N × M matrix  ∗ ∗  y0 y1 . . . yM−1 Fig. 1: Procedure for filling Rb yy.  y y . . . y∗   1 0 M−2  Y =   (6)  . . .. .   . . . .    factorisations comprises two terms. The M 3 term is related y y . . . y 2 N−1 N−2 N−M to the factorisation of Ryy in (5), while the NM term is by delaying each column by an additional sample, thereby related to the calculation of Rb yy in (7). The majority of the computational load is associated with the calculation of Ryy allowing Ryy to be estimated as b under the assumption N M. 1 H A further point to note is that R is a Hemitian Toeplitz Rb yy = Y Y. (7) yy N matrix because x is stationary. However, Rb yy is only approx- Further, placing n samples from the measured signal in the imately Toeplitz as practical environments are not stationary N × 1 vector x allows Rxy to be approximated by processes [9]. The effects of this observation are further explored in Section III-E. The use of Toeplitz solvers, such as 1 H Rb xy = Y x. (8) N the Bareiss algorithm [10], results in sub-optimal cancellation. Substituting (7) and (8) into (5) results in III.OPTIMISATION 1 1 This section shows how the mathematical properties of YH Yh = YH x (9) N N the computations described in Section II can be exploited which can be simplified to the form of the least squares (LS) to significant reduce the number of computations required to problem implement a Wiener filter. Yh = x (10) A. Hermitian Structure of Covariance Matrix under the assumption that the inverse of YH exists. An LS The first opportunity for optimisation emerges from the method, such as QR decomposition, in used to determine the exploitation of the Hermitian structure of Rb yy. Only the lower filter weights. Alternatively, a square system solver, such as triangle of Rb yy needs be explicitly calculated, while the upper LU decomposition, can be employed to solve (5) directly by triangle can be filled using using the approximate values of Rb yy and Rb xy from (7) and R (k, l) = R∗ (l, k) (8). b yy b yy (11) In order to achieve an accurate estimate of the covariance where A(k, l) is the element of A in kth row and lth column. 2 matrices, N should be much greater than M [4]. With this in The complexity in calculating Rb yy is reduced from O(2NM ) mind, the approximate number of complex operations required to O(NM 2). to solve the system via a number of methods is shown in Table I. The LU and Cholesky factorisations [7] solve (5), B. Tapped Delay Line Covariance Matrix while the QR factorisation [7] and conjugate-gradient least The tapped delay line structure of Y can be exploited to squares (CGLS) [8] solve (10). reduce the computational load. This observation means that is The QR filter uses Householder reflections to directly invert only necessary to store N + M − 1 unique samples, and not a rectangular matrix, where N > M. Hence, the terms seen all N × M elements, as the elements in Y are repeated. in Table I are both from the factorisation. Due to the negative This reduces the storage requirements, and the resulting term, the QR filter requires fewer operations than the Wiener lower number of unique memory accesses also improves cache filter (using LU and Cholesky Factorisation). The conjugate- efficiency, which is especially valuable on processors such as gradient least squares (CGLS) algorithm does not directly solve graphics processing units (GPUs) which have a high arithmetic (10), but instead iterates towards the solution. The number of intensity [11]. iterations is P . The tapped delay line structure of Y gives Ryy a Toeplitz The computational complexity of the LU and Cholesky structure. The approximation of Rb yy as a Toeplitz matrix reduces the computation of Rb yy further by only requiring the E. Toeplitz Approximation full inner dot product of the first row of Rb yy (2NM complex For a stationary process, the auto-covariance matrix is operations). a Toeplitz positive semi-definite matrix, with each element The remainder of Rb yy can be populated by following a defined by sequential update approach for each element of the first row, shown in Fig. 1. Each (gray) element is calculated along the Rij = E [(Yi − µi)(Yj − µj)] (16) diagonal as where µi is the mean of the ith random vector. For the tapped ∗ delay-line case where Rb yy(k, l) = Rb yy(k − 1, l − 1) − Y(k, k − 1)Y (l, k − 1) ∗ + Y(k, k + N)Y (l, k + N). (12) Yi(t0) = Yj(t0 + Ts(j − i)) (17) where T is the sample period, (16) becomes Therefore, the calculation of Rb yy has a complexity of s O(2NM + (3/2) M 2). Rij = E [(Y(t0 + Tsi) − µ(t0 + Tsi)) C. Exploiting Fast Correlation (Y(t0 + Tsj) − µ(t0 + Tsj))] (18) Fast correlation can be used to calculate Rb xy and the first where Y is the signal in question and µ is its mean. Considering row of R . This involves defining, b yy that practial scenarios are non-stationary processes i.e. h iT yT 0 a = 1,M−1 (13) µ(t0) 6= µ(t0 + τ), (19) where y is the N × 1 vector of reference samples, ·T denotes it stands that a transpose operation, and 0k,l represents an k × l matrix of Rij 6= R(i+n)(j+n). (20) zeros, giving It should also be noted that the maximum deviation in time is

b = [y0, y1, . . . , yN−1, yM−1, yM−2, . . . , y−1] . (14) between R11 and RMM , which is thus TsM. In a sample covariance matrix there will be 2M samples These 1 × (M + N − 1) vectors are then correlated using difference between Rb 11 and Rb MM . The covariance matrix, fast correlation which requires a and b to be transformed to with N M, will therefore be pseudo-Toeplitz. Using this the frequency domain. The element-wise product is then taken approximation allows Rb yy to be represented by an M element such that vector. c = F(a) ◦ F(b) (15) The Toeplitz sample covariance vector is defined as where F represents the Fourier transform, and ◦ represents H Tbyy = y Y (21) the Hadamard product. The first row of Rb yy is calculated by taking the inverse Fourier transform of c. The first row of Rb yy where y is the N × 1 vector containing the reference- can be extracted from the first M elements of the convolved channel samples (the first column of Y). This matrix-vector vector. multiplication can be implemented with the same methods The calculation of Rb yy’s first row has a complexity of described in Section III, except that the fill algorithm is not implemented. O(6(N +M)[log2(N +M)+1]), compared to the conventional O(2NM). The correlative method is less complex than the The major advantage of this method is the ability to use multiplicative method for most practical values of N with a Toeplitz matrix solver which has significantly reduced M > 80. Rxy can be calculated in the same manner. computational requirements. The Toeplitz solver used in this study is detailed in the appendix, and has a complexity of D. Factorisation Algorithm O((11/2) N 2). Since the computations required to form Rb yy can be significantly reduced using (12), the dominant term for computational F. Performance Comparison load becomes the factorisation of Rb yy. The performance of each solver is shown in Fig. 2. It can For a generic matrix, the filter coefficients can be found using be seen that the QR solver was the slowest for most system LU decomposition. However, the Hermitian nature of Rb yy an sizes, and the Toeplitz solver was the fastest. LDL factorisation [7] can be used. It should be noted that The LDL solver was 1.3 times slower than the Cholesky row/column pivoting is required in order for these factorisations solver for large problem sizes despite having the same dominant to be numerically stable [7]. Due to the low arithmetic intensity complexity term. This demonstrates the performance reduction of the pivoting, it may be a bottleneck for smaller problem is caused by row/column pivoting. The LU solver was faster sizes. The Cholesky decomposition will not always successfully than the LDL solver when M ≤ 1194. factorise Rb yy, as real data results in Rb yy being positive-semi definite. As such, the success of the factorisation must be IV. NUMERICAL EXAMPLES monitored, and when failure occurs the filter must resort to In passive radar, the undesired components are the direct one of the other factorisation algorithms. path interference (DPI) and clutter [2]. The Wiener filter can 105 TABLE IV: Cancellation filter throughputs for the FM config- Toeplitz uration. 104 Cholesky LDL Cancellation Filter Mean throughput (MS/s) 103 LU QR QR least-squares 0.033 2 10 CGLS 2.46 101 LU Wiener 0.564 Rectangular LU Weiner 7.07 100 FFT LU Weiner 6.37 -1

Average execution time (ms) 10 Toeplitz Weiner 23.7 10-2 102 103 104 TABLE V: Cancellation ﬁlter throughputs for the DAB conﬁg- Filter length uration.

Fig. 2: Mean execution time for square system solvers. Cancellation Filter Mean throughput (MS/s) QR least-squares 5.40 × 10−3 TABLE II: Signal properties of transmission signals. CGLS 1.11 FM Radio DAB DVB-T LU Wiener 0.118 Typical transmission power 250 10 8 Rectangular LU Weiner 3.80 (kW) [13] FFT LU Weiner 5.01 Maximum bandwidth 150 1 712 7 600 Toeplitz Weiner 9.91 (kHz) Sampling frequency (kHz) 200 2 048 8 124 and an LU factorisation to determine the filter weights. The TABLE III: Cancellation filter configuration. optimisation in Section III-B is termed “rectangular LU Wiener,” the optimisation scheme presented in Section III-C is termed Estimation samples Filter taps “FFT LU Wiener,” and the approximation from Section III-E 3 FM radio 40 × 10 512 is termed “Toepitz Wiener.” DAB 400 × 103 1 750 In order to place the results presented into a more general DVB-T 800 × 103 4 096 context, two other methods for performing clutter cancellation are also included in Tables IV to VI. These are the CGLS (implemented with 15 iterations) [2] solver and QR least- determine these undesired components from the reference squares factorisation. signal. The above algorithms were evaluated on three real-world A. FM Configuration scenarios based on each of the main broadcast signals which have been investigated for passive radar, frequency modulation The achieved throughput of the filters for the FM configura- (FM) terrestrial radio, digital audio broadcast (DAB), and tion is shown in Table IV. digital video broadcasting – terrestrial (DVB-T) [12]. The Aside from the QR least-squares filter, all the filters in most relevant properties of each of these broadcast signals are Table IV achieved a mean throughput greater than 200 kS/s shown in Table II. allowing them to run in real time (i.e. samples were processed These signal properties translate to three configurations for at least as fast as they were generated). The rectangular LU passive radar systems. Clutter is removed at Doppler frequency Wiener filter achieved the higher throughput than the FFT LU offsets lower than 5 Hz, and is implemented using a batched Wiener filter, demonstrating that the overhead incurred using approach [6]. The cancellation filter configuration for each the FFT method outweighs the computational benefits for this system is shown in Table III. configuration. Each of the implementations was then run over a set of As expected, the Toeplitz Wiener filter achieved the highest recorded data from an FM based passive radar. While the throughput, which was 3.35 times higher than the rectangular data set is not representative of the digital transmissions, the LU filter, 42 times higher than the conventional LU Wiener execution times of the cancellation algorithms are largely filter, and 9.63 times higher than the CGLS filter. invariant of the underlying data as long as the covariance matrices are non-singular. B. DAB Configuration For the results, the baselines for comparisons are “QR least- The results for the DAB filter are shown in Table V. squares,” which is a least-squares solver, and “LU Wiener,” The increased solution size and data rate meant that only which uses a matrix multiply to form the covariance matrices the rectangular Wiener, FFT LU Wiener and Toeplitz Wiener TABLE VI: Cancellation filter throughputs for the DVB-T configuration. -100

Cancellation Filter Mean throughput (MS/s) -110 CGLS 0.372 Rectangular LU Weiner 1.13 -120 Original FFT LU Weiner 1.23 -130 Wiener Toeplitz Weiner 3.22 Toeplitz -140

filters presented in presented in Section III were able to -150 operate in real-time (mean throughput higher than 2 MS/s). Estimated noise floor (dBm) The FFT LU Wiener filter outperformed the rectangular Wiener -160 implementation, demonstrating that the overhead associated with the FFTs is justified by the additional computational -170 benefit for the larger filter size. 0 500 1000 1500 Once again, the Toeplitz Wiener filter achieved the highest Time (s) throughput, 2.6 times higher than the rectangular Wiener filter, Fig. 3: Noise floor for M = 512 and N = 4 000. 2.0 times higher than the FFT LU Wiener filter, and 84 times higher than the LU Wiener filter. -100 C. DVB-T Configuration

The results for the DVB-T filter are shown in Table VI. -110 For this configuration, none of the methods which required the explicit storage of the cancellation matrix could be -120 implemented as the memory footprint was larger that the test Original platform could accommodate; these include QR least-squares -130 and the LU Wiener filters. None of the methods implemented Wiener Toeplitz could achieve real-time operation (a mean throughput of at -140 least 8 MS/s), with the fastest throughput achieved being 2.5 times too slow for real-time operation. -150 Again, the Toeplitz Wiener filter achieved the best throughput, Estimated noise floor (dBm) which was 8.7 times higher than the throughput of the CGLS -160 equivalent. -170 D. Cancellation effectiveness 0 500 1000 1500 The cancellation effectiveness of sub-optimal cancellation Time (s) algorithms such as CGLS has already been established in Fig. 4: Noise floor for M = 512 and N = 20 000. Note that literature [2], [14]. In order to gauge the validity of the the Wiener and Toeplitz graphs are indistinguishable. Toeplitz approximation, the noise floor for the test data set was compared. The noise floor was estimated by taking the median magnitude of the range-Doppler maps. after the LU Wiener filter was −153.0 dBm, and the Toeplitz Fig. 3 shows the noise floor for the FM configuration, at Wiener filter displayed a difference of 0.008 dB which is the batch size where the Toeplitz approximation begins to negligible. The Toeplitz approximation would be completely fail. For the relatively stationary environment for which the valid in this configuration. data was recorded, this occurred when N = 2 000. With no cancellation, the mean noise floor was −116.0 dBm. The LU V. CONCLUSION Wiener cancellation filter (Wiener in Fig. 3) reduced the noise Optimised implementations of the required Wiener filtering floor to −155.8 dBm, and the Toeplitz Wiener filter achieved were presented and evaluated. These implementations exploit a mean noise floor of −154.8 dBm. In this configuration, the the cancellation filters’ tapped delay line and overdetermined Toeplitz approximation raised the noise floor (i.e. reduced the structure. Two optimisation strategies were presented. The first, signal-to-noise ratio (SNR)) by 1.08 dB. rectangular optimisation, was found to be optimal for small Comparatively, for the original filter length (N = 20 000) for problem sizes, while the second, FFT optimisation, was found the FM configuration, the noise floor is shown in Fig. 4. Here, to be optimal for larger problem sizes. A Hermitian Toeplitz the Toeplitz Wiener filter resulted in near identical performance solver was also presented as an approximation to the Wiener to the LU Wiener filter. The mean noise floor for the data set filter in order to further accelerate the filters. The rectangular and FFT approximations were evaluated Algorithm 1: Hermitian Toeplitz solver in three configuration. These filters were found to increase Input: N element Toeplitz matrix vector a, N element system throughput between 12.5 and 42.5 times. Furthermore, solution vector b these optimisations allowed a significant decrease in memory Output: The N element solution, x requirements. It was shown that for typical passive radar Solve for the first principle minor: problem sizes, the Toeplitz approximation results in a SNR 1: r1 = a(1) deterioration of only 1.08 dB and 0.008 dB for FM and DAB 2: x(1) = b(1)/r1 passive radars respectively. Solve for the second principle minor: 3: r6 = a(2) APPENDIX 4: r3 = −r6/r1 HERMITIAN TOEPLITZ SOLVER ∗ 5: r1 = r1 + r3r6 This section presents the Hermitian Toeplitz solver which is 6: c2(1) = r3 ∗ used above. The algorithm is modified from a general Toeplitz 7: r6 = (b(2) − a(2) x(1))/r1 solver found in the Toeplitz library [15]. 8: x(1) = x(1) + c2(1)r6 An M × M Hermitian Toeplitz matrix, A, can be fully 9: x(2) = r6 described by its first row or column. For this implementation Solve for the rest of the system: the first row is selected, such that each element of this row is 10: for i = 3 to N do a1,i where i = 1, 2 ...N. The solution to 11: r6 = a(i) ∗ 12: c1(i − 1) = r Ax = b 3 (22) 13: for j = 1 to i − 2 do can be computed by Algorithm 1. 14: r6 = r6 + a(j + 1)c2(j) In Algorithm 1, vector elements are denoted x(i). The 15: end for 16: r = −r /r dimension dependant variables required for storage are c1, 3 6 1 17: r = r + r r∗ c2, and c3 which are each N − 1 element vectors. 1 1 3 6 By exploiting the Hermitian structure of A, the number of 18: c2(i − 1) = 0 interior multiplications and additions were halved. Furthermore, 19: for j = 1 to i − 1 do the loops between 19 and 25 were originally combined (which 20: c3(j) = c2(j) didn’t require an explicit copy), but it was expanded to allow 21: end for the loop to be parallelised. 22: for j = 2 to i − 1 do 23: c2(j) = c1(j)r3 + c3(j − 1) REFERENCES ∗ 24: c1(j) = c1(j) + c3(j − 1)r3 [1] J. Capon, “Maximum-likelihood spectral estimation,” in Nonlinear 25: end for Methods of Spectral Analysis, ser. Topics in Applied Physics, S. Haykin, 26: c2(1) = r3 Ed., vol. 34. Springer Berlin Heidelberg, 1983, pp. 155–179. [2] J. Palmer and S. Searle, “Evaluation of adaptive filter algorithms for 27: r5 = 0 clutter cancellation in passive bistatic radar,” in IEEE Radar Conference 28: for j = 1 to i − 1 do (RADAR), May 2012, pp. 493–498. ∗ 29: r5 = r5 + x(i − j)a(j + 1) [3] J. R. Guerci, J. S. Goldstein, and I. S. Reed, “Optimal and adaptive reduced-rank stap,” IEEE Transactions on Aerospace and Electronic 30: end for Systems, vol. 36, no. 2, pp. 647–663, Apr 2000. 31: r6 = (b(i) − r5)/r1 [4] I. S. Reed, J. D. Mallett, and L. E. Brennan, “Rapid convergence rate 32: for j = 1 to i − 1 do in adaptive arrays,” IEEE Transactions on Aerospace and Electronic Systems, vol. AES-10, no. 6, pp. 853–863, Nov 1974. 33: x(j) = x(j) + c2(j) ∗ r6 [5] B. Kang, V. Monga, and M. Rangaswamy, “Computationally efficient 34: end for toeplitz approximation of structured covariance under a rank constraint,” 35: x(i) = r6 IEEE Transactions on Aerospace and Electronic Systems, vol. 51, no. 1, pp. 775–785, January 2015. 36: end for [6] F. Colone, D. O’Hagan, P. Lombardo, and C. Baker, “A multistage 37: return x processing algorithm for disturbance removal and target detection in passive bistatic radar,” IEEE Transactions on Aerospace and Electronic Systems, vol. 45, no. 2, pp. 698–722, April 2009. [7] L. Trefethen and D. Bau, Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997. [12] P. Howland, “Editorial: Passive radar systems,” IEE Radar, Sonar and [8] Z. Zlatev and H. Nielsen, “Solving large and sparse linear least-squares Navigation, vol. 152, no. 3, pp. 105–106, June 2005. problems by conjugate gradient algorithms,” Computers & Mathematics with Applications, vol. 15, no. 3, pp. 185–202, 1988. [Online]. Available: [13] H. D. Griffiths and C. J. Baker, “Passive coherent location radar systems. http://www.sciencedirect.com/science/article/pii/0898122188901708 Part 1: Performance prediction,” IEE Radar, Sonar and Navigation, vol. [9] M. A. Richards, J. A. Scheer, and W. A. Holm, “Constant false alarm rate 152, no. 3, pp. 153–159, June 2005. detectors,” in Principles of Modern Radar, Volume I – Basic Principles. [14] C. Tong, “A scalable real-time processing chain for radar exploiting SciTech Publishing, 2010, pp. 589–623. illuminators of opportunity,” 2014. [10] W. R. Freund, “A look-ahead Bareiss algorithm for general Toeplitz matrices,” Numerische Mathematik, vol. 68, no. 1, pp. 35–69, 1994. [15] O. Arushanian, M. Samarin, V. Voevodin, E. Tyrtyshnikov, B. Garbow, [11] NVIDIA Corporation, CUDA C Programming Guide. NVIDIA J. Boyle, W. Cowell, and K. Dritz, The TOEPLITZ Package User’s Guide. Corporation, 2015. Argonne National Laboratory, 1983.