arXiv:math/0010243v1 [math.NA] 25 Oct 2000 ulwno) swl scanletmto rbesi iia commu digital t in computing problems estimation when channel behavior as well same as (yieldin the window), [25] dual with functions systems window block-Toeplitz decaying nite Weyl-He polynomially include or examples T Other exponentially to [13]). with leading e.g. sup decay, (see fast compact property this show - heriting frequently stronger also even the equations or often integral (since - of decay example one exponential polynomia In are has or deblurring function exponential diagonal. image of the in is off arising decay rate equations this decay problems certain practical a and enjoy cal matrix the properties of decay entries certain with systems. systems Toeplitz-type Toeplitz-type definite to solut positive attention (approximate) hermitian the finite with vast dealing and a are infinite, contain paper this and in applications told of stories number The a describe statis 17] and [4, engineering, surveys communications processing, signal matics, atmti,fiiescinmethod. section finite matrix, lant -al tomrmt.cai.d.Ti okwspartial was work This [email protected]. E-mail: n[ on bevtosmd ysm eerhr ncneto ihde with connection in provide researchers to some allow by dou which made m resul derived, observations of Toeplitz quantitative are solution embedding the some circulant Finally, the of via entries for systems. the estimates circulant of error finite decay give the we d on Furthermore, then only We secti depends finite the rate behavior. by systems decay Toeplitz (bi)infinite this of solution of o e characterization Gelfand the of general plays results a Classical role inverses. crucial their and A matrices systems. Toeplitz-type biinfinite ∗ M ujc classifications. subject AMS Abstract. e words. Key − Let .Introduction. 0. eateto ahmtc,Uiest fClfri,Davi California, of University Mathematics, of Department 2 1 , C 2 1 ORSOTSOISAOTTELT MATRIX TOEPLITZ ABOUT STORIES SHORT FOUR .Frall For ). [ − 1 2 , h tre odi hsppraedaigwt h ouino solution the with dealing are paper this in told stories The 1 2 ) opizmti,Luetoeao,dcyo nes matrix inverse of decay operator, Laurent matrix, Toeplitz etesto l -eidc otnos elvle ucin defi functions real-valued continuous, 1-periodic, all of set the be a f k ∈ = C opiztp qain rs nmn plctosi mathe- in applications many in arise equations Toeplitz-type − Z [ 2 1 1 2 − f 1 2 , ( 2 1 ω ) ) let , 51,4A0 51,65F10 65D10, 42A10, 65T10, CALCULATIONS HMSSTROHMER THOMAS e 2 πiωk ω k dω, 1 nmto,soigta h approximation the that showing method, on l ncmuaieBnc lersyield algebras Banach commutative on al. t 0 = so h osrcino preconditioners of construction the on ts yspotdb S rn 9973373. grant NSF by supported ly hoeia xlnto o numerical for explanation theoretical a -ignldcybhvo fToeplitz of behavior decay ff-diagonal ovlto problems. convolution , ∗ ± rv siae o h approximate the for estimates erive l nnt ovlto ytm by systems convolution infinite bly 1 , ti n t odto number. condition its and atrix ± 2 , . . . , ,C 51-63 USA; 95616-8633, CA s, ntesneta the that sense the in rcniinr circu- preconditioner, , ot 1] Kernels [19]. port) is h excellent The tics. elt ytm in- systems oeplitz nt,ifiie and infinite, finite, f ito references. of list ye Toeplitz type. l o fbiinfinite, of ion snegframes isenberg epyspecial pay We aytheoreti- many iain [21]. nications on spread point eso-called he biinfi- g ned 2 Thomas Strohmer be the Fourier coefficients of f. Since f is real-valued, we have ak = a k. − A Laurent operator or multiplication operator associated with its defining function f can be represented by the doubly infinite hermitian matrix L = [Lkl]k,l∞ = with Z n 1 −∞ entries Lkl = ak l for k,l . For all n 1 let Ln = [(Ln)kl]k,l−= n+1 be the Toeplitz − ∈ ≥ − matrix of size (2n 1) (2n 1) with entries (Ln)kl = ak l for k,l = n+1,...,n 1. − × − − − − Ln is a finite section of the biinfinite Toeplitz matrix L. A Toeplitz operator with symbol f can be represented by the singly infinite matrix
T = [Tkl]k,l∞ =0 with Tkl = ak l for k,l = 0, 1,... . In this case we define Tn = − n 1 [(Tn)kl] − as the n n matrix with entries (Tn)kl = ak l for k,l =0,...,n 1. Of k,l=0 × − − course Ln = T2n 1, but in what follows it will sometimes be convenient to use the − notations Ln and Tn. As mentioned earlier, a crucial role throughout the paper plays the decay behav- ior of Toeplitz matrices and their inverses. Classical results of Gelfand et al. lead to a general characterization of this decay behavior for biinfinite Toeplitz matrices, see section 1. Section 2 is concerned with the approximate solution of (bi)infinite Toeplitz systems using the finite section method. Explicit error estimates are derived, showing that the approximation rate depends only on the condition number of the matrix and its decay properties. In section 3 we analyze the approximate solution of convolution equations via circulant matrices. Finally, in section 4, we derive some quantitative results for preconditioning of Toeplitz matrices by circulant embedding. Among others, we provide a theoretical explanation of numerical observations made by Nagy et al. in connection with (non)banded Toeplitz systems.
1. On the decay of inverses of Toeplitz-type matrices. It is helpful to review a few results on the decay of inverses of certain matrices. In what follows, if not otherwise mentioned, the 2-norm of a matrix or a vector will be denoted by . k k without subscript. The following theorem about the decay of the inverse of a band matrix is due to Demko, Moss, and Smith [8]. Theorem 1.1. Let A be a matrix acting on ℓ2( ), where = 0, 1,...,N 1 , Z, I I { − } or N, and assume A to be hermitian positive definite and s-banded (i.e., Akl = 0 if 1 √κ 1 1 k l >s). Set κ = A A− , q = − and λ = q s . Then | − | k kk k √κ+1
1 k l A− cλ| − | , | k,l |≤ where
2 1 (1 + √κ) c = A− max 1, . k k { 2κ }
One notes that the inverse of a banded matrix is in general not banded, the 1 type of decay changes when we switch from A to A− (although exponential decay is “almost as good” as bandedness). This observation suggests to look at other classes Four short stories about Toeplitz matrix calculations 3 of matrices, for which the type of decay is preserved under inversion. This leads naturally to the following
Definition 1.2. Let A = [Ak,l]k,l be a matrix, where the index set is = Z, N ∈I I or 0,...,N 1 . { − } (i) A belongs to the space if the coefficients A satisfy Eγ,λ kl λ γ k l A < ce− | − | for γ,λ> 0, | kl| and some constant c> 0. If λ =1 we simply write . Eγ (ii) A belongs to the space if the coefficients A satisfy Qs kl s A
Theorem 1.4. Let A = akl be a hermitian positive definite biinfinite Toeplitz 1 { } matrix with inverse A− = α . Let v(k) be a positive (weight) function with { kl} v(k + l) v(k)v(l) , ≤ such that
∞ a v(k) < . (1.1) | k| ∞ k= X−∞ 4 Thomas Strohmer
If 1 lim =1 and lim n v(n)=1, (1.2) n n n →∞ v( n) →∞ − p then p
∞ α v(k) < . (1.3) | k| ∞ k= X−∞ In particular,
1 if A for s> 1, then A− ; (1.4) ∈Qs ∈Qs 1 if A for 0 <λ< 1, then A− . (1.5) ∈Eγ,λ ∈Eγ,λ
Proof. Since A is positive definite we have
∞ 2πikω f(ω)= ake > 0 (1.6) k= X−∞ and by the properties of Laurent operators [12]
∞ 2πikω 1 1/f(ω)= αke , where (A− )k,l = αk l. − k= X−∞ k We denote by W [v] the set of all formal series f = k∞= akX for which −∞ P ∞ f = a v(k) < . k k | k| ∞ k= X−∞ It follows from Chapter 19.4 of [10] that W [v] is a Banach algebra with respect to the multiplication (discrete convolution)
∞ l ∞ ∞ l fg = clX = al kbk X , − l= l= k= ! X−∞ X−∞ X−∞ k k where f = k akX and g = k bkX . By Theorem 2 on page 24 in [10] an element of W [v] has an inverse in W [v] if it is not contained in a maximal ideal of W [v]. Any P P maximal ideal of W [v] consists of elements of the form (cf. Chapter 19.4 in [10])
∞ k akξ =0, k= X−∞ where ξ = ρe2πiω with
ρ ρ ρ , 1 ≤ ≤ 2 Four short stories about Toeplitz matrix calculations 5 and
1 n ρ1 = lim n and ρ2 = lim √vn. n √v n n →∞ − →∞
Due to assumption (1.2) we get ρ1 = ρ2 = 1, hence ρ = 1. Thus a necessary and sufficient condition for an element in W [v] to be not contained in a maximal ideal of W [v] is a e2πikω = 0 for all ω. By assumption A is positive definite, hence k k 6 f(ω)= a e2πikω > 0 for all ω and (1.3) follows. kPk StatementsP (1.4) and (1.5) are now clear, since in both cases we can easily find a weight function such that (1.1) and (1.2) are satisfied. Remark 1.5. (i) Theorem 2.11 in [9] by Domar and Theorem V B in [1] by Beurl- ing are closely related to Theorem 1.4. Their results are concerned with (non)quasi- analytic functions, for which they have to impose the more restrictive condition
∞ log[v(kx)] < , for all x, k2 ∞ kX=1 on the weight function (called Beurling-Domar condition in [22]). For instance the k function v(k) = exp( 1+log(| | k ) ), k = 0 satisfies condition (1.2), but not the Beurling- | | 6 Domar condition. (ii) Using Theorem 8.1 on page 830 in [12] we can extend Theorem 1.4 to biinfinite block-Toeplitz matrices with finite-dimensional non-Toeplitz blocks (i.e., Laurent op- erators with matrix-valued symbol). These matrices play an important role in filter bank theory [25]. (iii) Note that v(n) = exp(γn) does not satisfy condition (1.2), that is why we have 1 to introduce an exponent γ1 <γ in order to estimate the decay of A− , cf. also The- orem 1.3. However if A γ,λ with λ < 1, then condition 1.2 is satisfied and – as ∈ E 1 we have seen – the decay of the entries of A− can be bounded by using the same parameters γ, λ.
2. Approximation of infinite-dimensional Toeplitz-type systems. Infi- nite Toeplitz systems arise for instance in the discretization of Wiener-Hopf integral equations or, more generally, in one-sided infinite convolution equations, see [11]. Bi- infinite Toeplitz-type systems are encountered in doubly infinite (discrete) convolution equations, as well as e.g. in filter bank theory [25] or in the inverse heat problem [4]. In order to solve these problems we have to introduce a finite-dimensional model. For let A : ℓ2(Z) ℓ2(Z) be a hermitian positive definite (hpd for short) biinfinite 7→ ℓ2 Z Toeplitz matrix given by ak,l k∞= . Let y = yk k∞= ( ). We want to solve { } −∞ { } −∞ ∈ the system Ax = y. For n N and y ℓ2(Z) define the orthogonal projections P by ∈ ∈ n
Pny = (..., 0, 0,y n+1,...,yn 1, 0, 0,... ). (2.1) − − 6 Thomas Strohmer
2n 1 By identifying the image of P with the 2n 1-dimensional space C − we can express n − the (2n 1) (2n 1) matrix A as − × − n
An = PnAPn,
(n) where we have used that P ∗ = P . The n-th approximation x to x is then given by the solution of the finite-dimensional system of equations
(n) (n) Anx = y
(n) where y := Pny. If A is a singly infinite Toeplitz matrix and y ℓ2(N), we proceed analogously by ∈ defining Pn as
Pny = (y0,y1,...,yn 1, 0, 0,... ). − This approach to approximate the solution of Ax = y is usually called the finite section method, cf. [11]. The first question that arises when considering this method is “does x(n) converge to x?”. For the case when A is not hpd this question has lead to deep mathematical results. See the book [11] and chapter 7 in [2] for more details. For the case when A is hpd the answer is easy and always positive. To see this, recall that since A is hpd it follows that An is also hermitian positive definite, see [15]. Furthermore, An A 1 1 k k≤k k and (An)− A− for n = 1, 2,... . Applying the Lemma of Kantorovich [23] k k≤k1 k 1 (n) yields that (A )− converges strongly to A− for n , i.e., x converges to x in n → ∞ the ℓ2-norm for any y ℓ2(Z) (or for any y ℓ2(N) if A is singly infinite). ∈ ∈ An important aspect for applications is if we can give an estimate on how fast x(n) converges to x. It will be shown that the rate of approximation depends on the decay behavior and the condition number of the matrix.
Theorem 2.1. Let Lx = y be given, where L = ak,l is a hermitian positive (n) {1 (n)} definite biinfinite Toeplitz matrix and denote x = Ln− y . (a) If there exist constants c,c′ such that
γ k γ k a ce− | | and y c′e− | |, γ> 0 (2.2) | k|≤ | k|≤ then there exists a γ1 with 0 <γ1 <γ and a constant c1 depending only on γ1 and on the condition number of L such that
(n) γ1n x x ce− . (2.3) k − k≤
(b) If there exist constants c,c′ such that
s s a c(1 + k )− and y c′(1 + k )− , s> 1, (2.4) | k|≤ | | | k|≤ | | then there exists a constant c1 depending only on the condition number of L such that
(n) (1 2s)/2 x x c n − . (2.5) k − k≤ 1 Four short stories about Toeplitz matrix calculations 7
Proof. We have
(n) 1 1 (n) 1 1 (n) x x = L− y L− y L− y LL− y k − k k − n k≤k kk − n k 1 (n) 1 (n) L− y y + (L L)L− y . (2.6) ≤k k k − k k n − n k To prove statement (a) we note that by Theorem 1.3(a) there exists a γ2 < γ 1 γ2 k l such that (L− ) c e− | − | with a constant c depending only on γ and on the n kl ≤ 2 2 2 condition number of Ln. Since σ(Ln) [fmin,fmax] we get cond(Ln) cond(L) for all ⊆ (n) ≤ 1 (n) n. That means we can choose c independently of n. Write z = (L L)L− y 2 n − n and note that z(n) = 0 for k < n. Since the non-zero entries of (L L) decay k | | n − exponentially, it is easy to show that there exists a γ1 with 0 < γ1 < γ2 such that (n) γ1n (n) z c3e− for some constant c3. It is trivial that y y also decays k k ≤ 1 k − k exponentially for n . We absorb L− and the other constants in the constant → ∞ k k c1 and get the desired result. For the proof of part (b) we proceed analogously to above by applying Theo- rem 1.3(b) to conclude that
1 (n) s (L− y ) c (1 + k)− | n k|≤ 2 for some constant c depending only on cond(L) and on s. The norm y y(n) can 2 k − k be estimated via
∞ ∞ 1 2s (n) 2 2 2s 2s n − y y = y 2c (1 + k)− 2c (1 + x)− dx 2c , k − k | k| ≤ ≤ ≤ 2s 1 k n k=n nZ1 − |X|≥ X − (2.7)
(n) (n) 1 (n) similarly for z where z := (L L)L− y . Since all arising constants - k k n − n absorbed in one constant c1 - depend only on cond(L) and on the exponent s, the proof is complete. Remark 2.2. Theorem 2.1 holds if we replace the system Lx = y by a singly infinite Toeplitz system T x = y with corresponding decay conditions on T and y and (n) (n) approximate its solution by considering the finite system Tnx = y . The proof of Theorem 2.1 is essentially based on the fact that under appropriate (n) 1 1 (n) decay conditions on L and y (resp. Ln and y ) L− and Ln− y have similar decay 1 1 properties. Thus, if one can show that Ln− has the same decay properties as L− one can use Theorem 1.4 in order to generalize Theorem 2.1 to various other decay conditions. This may however not always lead to simple and closed-form expres- 1 (n) sions for (L L)L− y , therefore I have restricted myself to the most frequently k n − n k encountered decay properties. Example 1: We illustrate Theorem 2.1 by a numerical example. We consider s Lx = y, where L is the biinfinite Toeplitz matrix with entries a = (1+ k )− , k Z for k | | ∈ s = 2 and y consists of random entries having the same polynomial decay rate as the 8 Thomas Strohmer
1 10 actual error ||x−x(n)|| estimated error
0 10
−1 10 error
−2 10
−3 10
−4 10 0 50 100 150 200 250 300 350 n (size of circulant matrix is 2n+1)
Fig. 2.1. Actual approximation error and error estimate (2.5) from Theorem 2.1(b) for the system Lx = y, where L is a biinfinite hermitian Toeplitz matrix with polynomial decay.
entries a . To compare the error x x(n) with the error estimate (2.5) we would need k k − k the true solution x. Since the solution of this biinfinite system cannot be computed (n0) (n0) analytically we compute the “true” solution of Lx = y by solving Ln0 x = y for very large n0 (we choose n0 = 32768). Using (2.6) and (2.12) we can estimate that (32768) 6 (32768) in the worst case x x 10− , so that x can mimick the true solution k − k≈ with sufficiently high accuracy for this experiment.
Then we approximate this solution by the finite section method as in Theorem 2.1 for n = 0,..., 350 and compute for each n the error x x(n) as well as the error k − k estimate in (2.5). Note that an explicit expression for the constant c1 in (2.5) is not known, we only know that it depends on the condition number of L. In this example we use c1 = cond(L) (a different example may require a different choice). The result, illustrated in Figure 2.1, shows that the asymptotic behavior of the error rate is well estimated by the given error bound.
It is well-known that the product of two Laurent operators and the inverse of a Laurent operator (if it exists) is again a Laurent operator. This is of course not true for singly infinite or finite Toeplitz matrices (and this is one of the reasons which makes the “Toeplitz business” so interesting). Hence one may argue that the “canonical” finite-dimensional analogue of Laurent operators are not Toeplitz matrices but circulant matrices, since they also form an algebra. Thus for a given biinfinite hermitian Toeplitz matrix L with entries Lkl = ak l we define the hermitian circulant − Four short stories about Toeplitz matrix calculations 9 matrix C of size (2n 1) (2n 1) by n − × −
a0 a1 ... an 2 an 1 an 1 an 2 ... a1 − − − − a1 a0 a1 ... a2 C = . (2.8) n . .. . . . . a ... a a 1 1 0 n 1 We also say that Cn is generated by ak k=− n+1. { } − Remark 2.3. Cn does not have to be positive definite if L is positive definite, e.g. see [5]. However - as pointed out in [5] - if L is at least in Wiener’s algebra then one can always find an N such that Cn is invertible for all n>N. The faster the decay of the entries of L the smaller this N has to be. We can do even a little better and estimate how well the extrema of the defining function of L are approximated by the extreme eigenvalues of Cn.
Lemma 2.4. Let L be a biinfinite hermitian Toeplitz matrix with entries Lk,l = 2πikω ak l where a = ak k∞= and set f(ω)= k∞= ake . Let Cm be the associated − { } −∞ −∞ (2m 1) (2m 1) circulant matrix with first row (a0, a1,..., am 1,am 1,...,a1). − × − P (m−) −(m) Denote the maximum and minimum eigenvalue resp. of Cm by λmax and λmin. (a) If L is n-banded with m>n, then πn λ(m) f λ(m) + 2 sin a , (2.9) max ≤ max ≤ max 2(2m 1)2 k k1 πn− λ(m) f λ(m) 2 sin a . (2.10) min ≥ min ≥ min − 2(2m 1)2 k k1 − γ k (b) If a ce− | | for k Z,c> 0, then | k|≤ ∈ (m) 2c πm γm fmax λmax γ 2 sin 2 + e− , (2.11) | − |≤ 1 e− 2(2m 1) − h − i (m) a similar estimate holds for fmin λmin . s | − | (c) If a c(1 + k )− for k Z,s> 1,c> 0, then | k|≤ | | ∈ (m) 2c πm 1 s f λ 2 sin + m − , (2.12) | max − max|≤ s 1 2(2m 1)2 − h − i a similar estimate holds for f λ(m) . | min − min| Proof. (a): It is well-known [7] that the eigenvalues of Cm are given by
m 1 − 2πikl/(2m 1) a e − , for l = m +1,...,m 1. (2.13) k − − k= m+1 X− l For case (a) that means they are regularly spaced samples f( 2m 1 ) of the function − f, which immediately yields the left hand side of the inequalities (2.9) and (2.10). In order to prove the right hand side of (2.9) and (2.10) it is sufficient to estimate l + ω l max f( ) f( ) , ω,l 2m 1 − 2m 1 − −
10 Thomas Strohmer
1 1 where ω [ 2(2m 1) , 2(2m 1) ] and l = m +1,...,m 1. Define the sequence m ∈ − − − − − a˜k k= m bya ˜k = ak if k n anda ˜k = 0 if k >n. There holds { } − | |≤ | | m l l + ω 2πikl/(2m 1) 2πikω/(2m 1) max f( ) f( ) = max a˜ke − (e − 1) ω,l 2m 1 − 2m 1 ω,l − k= m − − X− n 2πikω/ (2m 1) max e − 1 ak ≤ ω, k n − | | | |≤ k= n X− πm max 2 sin(πkω/(2m 1)) a 1 2 sin a 1. (2.14) ≤ ω, k n | − |k k ≤ 2(2m 1)2 k k | |≤ − Relations (2.9) and (2.10) follow now from this estimate. Statements (b) and (c) can be proved similarly by using
m 1 − l + ω (m) 2πikω/(2m 1) f λ a e − 1 + a , (2.15) | 2m 1 − l |≤ | k|| − | | k| − k= m+1 k m X− | X|≥ and applying the corresponding decay properties to (2.15). Remark 2.5. The left part of inequality (2.10) reads λm f . This im- min ≥ min plies that Strang’s preconditioner is always positive definite for n-banded hermitian Toeplitz matrices of size > (2n 2n) with positive generating function. Hence the × “sufficiently large n”-condition at the end of section 2 in [14] can be omitted. It is obvious that decay properties for circulant matrices cannot be defined in the same way as for non-circulant matrices. Hence, by stating that Cm has, say, m 1 exponentially decaying entries, we mean that the generating sequence ak k=− m+1 γ k { } − satisfies a ce− | |, in which case the entries of C will decay exponentially off | k| ≤ m the corners of the matrix (instead of off the diagonal). In analogy to the theorems in section 1 it is natural to ask if the inverse of Cm also inherits these decay properties. The following theorem shows that at least for m sufficiently large this is the case. The 1 1 entries of Cm− uniformly approximate the entries of L− with an error rate depends on the decay properties and the condition number of L. Theorem 2.6. Let L be a hermitian positive definite Laurent operator with entries Lkl = ak l and let Cm be the associated circulant (2m 1) (2m 1) matrix − 1 1 − × − 1 as defined in (2.8). Denote the entries of L− by (L− )kl = αk l and let (Cm)− − m 1 { } (if it exists) be generated by βk k=− m+1. { } − (i) If L , then for sufficiently large m ∈Eγ
γ1m α β ce− , (2.16) | k − k|≤ with 0 <γ γ and some constant c depending on cond(L) and γ . 1 ≤ 1 (ii) If A for s> 1, then for sufficiently large m ∈Qs m1 s α β c − (2.17) | k − k|≤ s 1 − with some constant c depending on cond(L). Four short stories about Toeplitz matrix calculations 11
2πikx m 1 2πikx Proof. (i): Set f(x)= k∞= ake and fm(x)= k=− m+1 ake . By the −∞ − properties of Laurent operators [12] αk k Z is given by P { } ∈ P 1/2 1 α = e2πikxdx, k Z, (2.18) k f(x) ∈ 1Z/2 −
By Remark 2.3 and Lemma 2.4 we can easily find an N such that Cm is invertible for all m>N, which implies that fm > 0. In this case by the properties of circulant m 1 1 matrices [7] the entries βk k=− m+1 of Cm− can be computed as { } − m 1 1 − 1 2πikl/(2m 1) βk = e − , k = m +1,...,m 1. (2.19) 2m 1 f ( l ) − − l= m+1 m 2m 1 − −X − Now consider 1/2 m 1 1 2πikx − 1 2πikl/(2m 1) αk βk = e dx e − | − | f(x) − f ( l ) l= m+1 m 2m 1 1Z/2 − − −X 1/2 m 1 − 1 2πikx 1 1 2πikl/(2m 1) e dx e − + (2.20) ≤ f(x) − 2m 1 f( l ) l= m+1 2m 1 1Z/2 − − − −X m 1 m 1 1 − 1 2πikl/(2m 1) − 1 2πikl/(2m 1) + e − e − (2.21) 2m 1 f( l ) − f ( l ) l= m+1 2m 1 l= m+1 m 2m 1 − − − −X −X We estimate the expression above in two steps: 1 2πikx ∞ 1. We first consider (2.20). Note that f(x) = k= αke and −∞ m 1 P ∞ − ∞ 1 2πikl/(2m 1) 2πilp/(2m 1) = αke − = αp+(2m 1)q e − . f( l ) − 2m 1 k= p= m+1 q= ! − X−∞ X− X−∞
By setting δp = q∞= αp+(2m 1)q we get −∞ − P m 1 1 − 2πilp/(2m 1) = δ e − , l = m +1,...,m 1, l p − − f( 2m 1 ) p= m+1 − X− and m 1 − 1 1 2πikp/(2m 1) δ = e− − . k 2m 1 f( p ) p= m+1 2m 1 − X− − Hence 1/2 m 1 1 2πikx 1 − 1 2πikl/(2m 1) e dx e − f(x) − 2m 1 f( l ) l= m+1 2m 1 1Z/2 − − − −X
= αk δk αk+(2m 1)q . (2.22) | − |≤ | − | q=0 X6 12 Thomas Strohmer
1 By Theorem 1.3(a) A implies A− 1 . Hence we get for k = m+1,...,m 1 ∈Eγ ∈Eγ − −
∞ γ1(k+(2m 1)q) αk+(2m 1)q 2c1 e− − | − |≤ q=0 q=1 X6 X ∞ γ1m γ1((2m 1)q m+1) 2c1e− 2c1 e− − − = . (2.23) ≤ 1 e γ1(2m 1) q=1 − − X −
2. Now we estimate (2.21):
m 1 m 1 1 − 1 2πikl/(2m 1) − 1 2πikl/(2m 1) e − e − 2m 1 l − l l= m+1 f( 2m 1 ) l= m+1 fm( 2m 1 ) − −X − −X − m 1 1 − 1 1 ≤ 2m 1 f( l ) − f ( l ) l= m+1 2m 1 m 2m 1 − −X − − m 1 1 − 1 1 l l f( ) fm( ) ≤ 2m 1 f( l ) f ( l ) 2m 1 − 2m 1 l= m+1 2m 1 m 2m 1 − −X − − − − m 1 1 1 1 − max ak ≤ 2m 1 l m f( l ) f ( l ) | | − | |≤ 2m 1 m 2m 1 l= m+1 k >m − − −X | X| 1 1 A− C− a . (2.24) ≤k kk m k | k| k >m | X|
By Lemma 2.4 we can easily find for any ε > 0 an N such that for all m>N there 1 1 holds C− (1 + ε) A− . Thus k m k≤ k k
1 1 1 2 ∞ A− C− a (1 + ε) A− 2 a k kk m k | k|≤ k k | k| k >m k=m | X| X γm 1 2 e− 1 2 γ1m (1 + ε) A− 2 (1 + ε) A− c e− (2.25) ≤ k k 1 e γ ≤ k k 2 − −
for some γ1 < γ and some constant c2. By combining (2.22), (2.23), and (2.25) and 1 hiding expressions as (1 + ε) A− in the constant c, we obtain estimate (2.16). k k (ii): The proof of (2.17) is similar to the proof of (2.16). The only steps that require a modification are (2.23) and (2.25). By Theorem 1.3(a) A implies ∈ Qs Four short stories about Toeplitz matrix calculations 13
1 A− s. Hence we can estimate q=0 αk+(2m 1)q as follows. ∈Q 6 | − | P s αk+(2m 1)q c′ (1 + (2m 1)q + k )− | − |≤ | − | q=0 q=0 X6 X6 s ∞ s =2c′(1 + 2m 1+ k )− +2c′ (1 + (2m 1)q + k )− | − | | − | q=2 X s ∞ s 2c′(1 + 2m 1 m +1 )− +2c′ (1 + (2m 1)q m +1 )− ≤ | − − | | − − | q=2 X ∞ s s 2c′(1 + m)− +2c′ (1 + (2m 1)x m +1 )− dx ≤ | − − | Z1 s s s (1 + m)− 2c(1 + m)− =2c′(1 + m)− +2c′ . (2.26) (2m 1)(s 1) ≤ s 1 − − − By adapting (2.25) to the case of polynomial decay we can estimate (2.21) by
1 s 1 1 m − 2c A− C− . (2.27) k kk m k s 1 − Combining (2.26) with (2.27) yields the desired result. The main portion of the proof of statement (ii) above is due to Gabriele Steidl [24]. 3. Error estimates for approximate solution of deconvolution problems. ℓ2 Z Consider the convolution of two sequences a = ak k∞= ,c = ck k∞= ( ), { } −∞ { } −∞ ∈ given by b = a c. Here a may represent an impulse response or a blurring function. ∗ Given a and b our goal is to compute c. This is known as deconvolution. In matrix notation the problem can be expressed as Lc = b where L is a biinfinite Toeplitz matrix with entries Lkl = ak l. − Sometimes a and c have compact support, in which case b also has compact (al- though larger) support and c can be computed by solving a finite banded Toeplitz system, see [19]. It is well-known that this can be done efficiently via FFT by em- bedding the Toeplitz matrix into a circulant matrix. Of course, this approach is very attractive from a numerical viewpoint, at least if the system is well-conditioned (we discuss the ill-conditioned case in section 4). However, if either a or c does not have compact support, the reduction of Lc = b to a finite Toeplitz system obviously will introduce a truncation error and embedding the Toeplitz matrix into a circulant yields an additional (perturbation) error. We nevertheless can try to make use of the FFT-based approach with the hope to get a good approximation to the solution. According to [13] it has been shown in [18] that the solution of doubly infinite convolution systems can be approximated by solutions of finite circulant systems. (Note that the approximation by finite circulant systems does not apply to one-sided infinite convolution equations.) Hence we are concerned with the problem of how good the approximation is obtained in that way and how fast the approximation converges to the true solution. 14 Thomas Strohmer
To answer these questions we proceed as follows. In the first step we study the approximate solution of a finite Toeplitz system by circulant embedding. In the second step we combine the obtained results with Theorem 2.1 to derive estimates for the approximate solution of doubly infinite convolution equations by finite circulant systems. The results that we will collect in the first step will also be very useful in section 4 for the analysis of circulant preconditioners. (n) (n) Let Anx = y be given, where An is an hpd n n Toeplitz matrix. Of course, (n) × we have in mind that An and y are finite sections of the biinfinite Toeplitz matrix L and the right-hand side y, respectively. As usual, we embed A into a circulant matrix C of size 2n 2n as follows: n 2n ×
An Bn∗ C2n = , (3.1) "Bn An# where Bn is the n n Toeplitz matrix with first row given by bk = an k for k = × − 1,...,n 1. If a is known we set b = a , otherwise we define b = 0. For the − n 0 n 0 following considerations it does not matter if we embed An into a circulant matrix of size (2n 1) (2n 1) or of size 2n 2n (or larger). Choosing B to be of the same − × − × n size as An is just more convenient for the proofs below. Since C2n is circulant we can
find An again in the center of C2n (and at any other position along the main diagonal of C2n), and express the embedding of An into C2n as follows
× × × C2n = A . × n × × × × In spite of the biinifite system Lx = y in the background it is useful to embed y(n) symmetrically into a vectory ˜(n) of length 2n as follows
y˜(n) = [0,..., 0,y(n), 0,..., 0].
We assume for the moment that C2n is invertible (and will justify this assumption (n) (n) (n) later). An approximate solution z to Anx = y is now obtained by solving (n) 2n n/2 C z˜(n) =y ˜(n) and setting z(n) = z˜ − (i.e., we take as approximation the 2n { k }k=n/2+1 central part ofz ˜(n) corresponding to the embedding of y(n) intoy ˜(n)). 1 We partition C2−n in the same way as C2n as follows
1 Mn Tn∗ C2−n = , " Tn Mn# where Mn is a Toeplitz matrix and by Cauchy’s interlace theorem (cf. [15]) invertible. 1 (n) Define Sn := Mn− , then it is easy to see that z can be obtained as the solution of
(n) (n) Snz = y .
The question is now how well does z(n) approximate x(n). Four short stories about Toeplitz matrix calculations 15
(n) (n) Theorem 3.1. Let Anx = y be given where An is an n n hermitian positive 1 × 2 2πiωk definite Toeplitz matrix with (An)kl = ak l and ak = f(ω)e dω. Assume that − 1 − 2 γ k (n) γ n/2 k R ak ce− | | and (y )k c′e− | − |. Suppose that C2n as defined in (3.1) is | | ≤ (|n) | ≤ (n) (n) invertible and let z be the solution of Snz = y , where Sn is the n n leading 1 × principal submatrix of C2−n . Then there exists a γ1 with 0 < γ1 < γ and a constant c1 depending only on fmin and fmax and on γ1 such that
(n) (n) γ1n x z c e− . (3.2) k − k≤ 1
Proof. Similar to Theorem 2.1 we write
(n) (n) 1 1 (n) x z A− (A S )S− y . (3.3) k − k≤k n kk n − n n k
Using the Schur complement [15] we can write Sn as
1 S = A B A− B∗ . (3.4) n n − n n n We will first show that the entries of the matrix A S are exponentially decaying n − n off the corners of the matrix. Note that equation (3.4) implies
A S = E n − n n 1 where En := BnAn− Bn∗. We analyze the decay behavior of En in two steps by 1 1 considering first BnAn− and then (BnAn− )Bn∗ . γ k l γ(n k l ) Recall that (A ) ce− | − | and note that (B∗) ce− −| − | . By | n kl| ≤ | n kl| ≤ 1 γ2 k l Theorem 1.3 we know that (A− ) c e− | − | for some 0 < γ < γ, where c | n kl| ≤ 2 2 2 depends on γ and on f and f , but is independent of n. We set δ = γ γ . 2 min max − 2 There holds
n 1 n 1 − − 1 1 γ(n k j ) γ2 j l (B A− ) = (B ) (A− ) cc e− −| − | e− | − |. (3.5) | n n kl| | n kj n jl|≤ 2 j=0 j=0 X X For simplicity we will absorb any constants arising throughout this proof that depend solely on γ (or γ2) in the constant c. We analyze the sum (3.5) further by splitting it 1 up into three parts and in addition consider first the entries (B A− ) with k l. n n kl ≥ (i) 0 j (iii) k j 1 γ2(n k l ) (B A− ) ce− −| − | . | n n kl|≤ 1 1 We now estimate the decay of the entries of BnAn− Bn. Since BnAn− Bn∗ is her- 1 mitian, it is sufficient to consider only the entries (B A− B∗) with k l. There n n n kl ≥ holds n 1 − 1 γ2(n k j ) γ(n j l ) (B A− B∗) c e− −| − | e− −| − | . (3.7) | n n n kl|≤ j=0 X As before we proceed by splitting up this sum into three parts. (i) 0 j 1 2γ2n γ2(k+l) 2γ2n γ2 k l ) γ2(k+l) (B A− B∗) c(e− e + e− e | − | + e− ). (3.8) | n n n kl|≤ The entries of C2n satisfy γ k l ce− | − | for k l =0,...,n 1, (C2n)lk | − | − (3.9) γ(2n k l ) | |≤ ce− −| − | for k l = n, . . . , 2n 1. | − | − Four short stories about Toeplitz matrix calculations 17 By Theorem 2.6 there exists a γ3 <γ and a constant c3 depending on γ3 and on fmin and fmax such that γ3 k l 1 c3e− | − | for k l =0,...,n 1, (C2−n )lk | − | − (3.10) γ3(2n k l ) | |≤ c e− −| − | for k l = n, . . . , 2n 1. 3 | − | − (n) 1 (n) (n) γ3 n/2 k (n) (n) Hence z := S− y satisfies z c e− | − |. Set u = (A S )z . After n | k |≤ 3 n − n some lengthy but straightforward computations we get γ1(n/2+k) (n) c1e− for k =0,...,n/2, (u )k (3.11) | |≤ c e γ1(3n/2 k) for k = n/2,...,n 1 − − (n) γ3n/2 for some γ <γ . Hence u ce− , which together with (3.3) completes the 1 3 k k≤ proof. Corollary 3.2. Let Lx = y be given where L is a biinfinite hermitian positive definite Toeplitz matrix with entries Lkl = ak l and let Sn be as defined in (3.4). − γ k (n) (n) (n) Assume L and y e− | | and let z be the solution of S z = y . Then ∈Eγ | k|≤ n there exists an N such that for all n>N (n) γ1n x z ce− , k − k≤ for some 0 <γ1 <γ and a constant c independent of n. Proof. First note that by Remark 2.3 we can always find an N such that Sn exists for all n>N. There holds (n) 1 1 (n) 1 1 (n) 1 (n) 1 (n) x z = L− y S− y L− y L− y + L− y S− y (3.12) k − k k − n k≤k − n k k n − n k where A is an n n finite section of L. The result follows now by applying Theo- n × rem 2.1 and Theorem 3.1. Theorem 2.1 and Corollary 3.2 provide two different ways to approximate the solution of biinfinite Toeplitz systems. Which of the two is preferable? This depends on two criteria: (i) The accuracy of the approximation for given dimension n; (ii) the computational costs for solving each of the finite-dimensional systems. The solution of the circulant system in Corollary 3.2 can be computed via 3 FFTs of size 2n. The Toeplitz system in Theorem 2.1 can be solved by the conjugate gradient method in approximately 3k FFTs of size 2n, where k is the number of iterations. Of course, additional preconditioning can significantly reduce this number at the cost of two additional FFTs per iteration (see also Section 4). For both, the circulant and the Toeplitz system, zeropadding can be used to extend the vectors to “power-of-two”-length. Example 2: We consider the same biinfinite Toeplitz system as in Example 1. We compare the error when approximating the solution by using the circulant system of Corollary 3.2 and by the Toeplitz system of Theorem 2.1. We compute for each 18 Thomas Strohmer 1 10 error via Toeplitz system error via circulant system 0 10 −1 10 error −2 10 −3 10 −4 10 0 50 100 150 200 250 300 350 dimension n Fig. 3.1. Comparison of error for the solution of a biinfinite Toeplitz system with polynomial decay. We compare the approximation error of the Toeplitz system described in Theorem 2.1 to that of the circulant system of Corollary 3.2 for increasing matrix dimension. The approximation error of both methods is almost identical, so that the difference between the two graphs is hardly visible. n =1,..., 350 the approximation error x x(n) and x z(n) respectively. As can k − k k − k be seen from Figure 3.1 both methods give almost the same error, in fact the two lines showing the error are hardly distinguishable. A similar behavior can be observed for other examples involving biinfinite Toeplitz matrices with fast decay. Since solving a circulant system is cheaper than solving a Toeplitz system, the approximation scheme of Corollary 3.2 seems to be preferable in such situations. Many variations of the theme are possible. For instance if An is an s-banded (biinfinite) Toeplitz matrix with s < n/2, we could use Strang’s preconditioner as approximate inverse. Due to the explicit constants in Theorem 1.1 this approach allows us to give an error estimate with explicit constants (cf. also Theorem 5 in [25]). (n) (n) Theorem 3.3. Let Anx = y be given where An is an n n hermitian s- × (n) banded Toeplitz matrix with s (n) (n) γn γs γ(s+1) 3 x z 3√2cλ− (λ− λ− )− k − k≤ − where c and λ are as in Theorem 1.1. Proof. The proof is similar to that of Theorem 3.1. To avoid unnecessary repeti- tions we only indicate the modifications, that are required. Four short stories about Toeplitz matrix calculations 19 By Remark 2.3 Cn is invertible. Note that Cn is a matrix with three bands, one band is centered at the main diagonal, and the two other bands of width 2s are located at the lower left and upper right corner of the matrix. It follows from Proposition 5.1 1 in [8] that the entries of Cn− decay exponentially off the diagonal and off the lower right and upper left corner. More precisely, k l 1 cλ| − | if 0 k l n (C− )k,l ≤ | − |≤ n 2n+1 k l | |≤ cλ −| − | if n +1 k l 2n , ≤ | − |≤ 1 where c is as in Theorem 1.1 with λ = q 2s . With this result at hand it is easy to show 1 (n) that the entries of Cn− y decay exponentially. A C has a simple form, it is a Toeplitz matrix with first row n − n (0,..., 0,asas 1,...,a1). − (n) 1 (n) When we compute u := (A C )(C− y ) the non-zero entries of (A n − n n n − Cn) are multiplied by exponentially decaying entries due to the exponential decay of 1 (n) (n) s s+1 3 n ((C− y ). This leads to the estimate u 2√2c(λ λ )− λ . n k k≤ − An interesting alternative to periodic boundary conditions is the use of Neumann boundary conditions considered in [20]. This modification will be discussed elsewhere. 4. Preconditioning by embedding and exponentially decaying Toeplitz (n) (n) matrices. The accuracy of the solution of the Toeplitz system Anx = y by 1 using Sn− as approximate inverse depends crucially on the decay properties of the right hand side y(n). If y(n) does not have appropriate decay conditions the approach in section 3 may not yield an approximation with sufficient accuracy. But we can still 1 (n) (n) use Sn− as preconditioner and solve Anx = y by the preconditioned conjugate gradient method [4]. The construction of preconditioners via circulant embedding is well known, it has been thoroughly investigated in [6] and for the special case of band Toeplitz matrices in [19, 14]. Compared to the certainly more elegant and more general approach in [6], the approach undertaken in this section has the advantage that it yields some quantitative 1 results. It shows that the clustering behavior of the preconditioned matrix Sn− An is the stronger the faster the decay of An is. Moreover, our approach allows us to prove a conjecture by Nagy et al., cf. [19], and will provide a theoretical explanation for some numerical results presented in [19] and [14]. The theoretical analysis of the clustering behavior of the eigenvalues of the pre- 1 conditioned matrix Sn− An is inspired by the work of Raymond Chan [3]. We will 1 1 show that Sn− An can be written as Sn− An = In + Rn + Kn, where In is the identity matrix, Rn is a matrix of small rank, and Kn is a matrix of small 2-norm. Theorem 4.1. Let An be a hermitian Toeplitz matrix whose entries ak decay γ k 1 exponentially, i.e., a Ce | |, k = 0,...,n 1. Set S = A B A− B∗, where | k| ≤ − n n − n n n Bn is as defined in (3.1). Then for all ε > 0, there exist N and M such that for all n>N at most M eigenvalues of A S have absolute value exceeding ε. n − n 20 Thomas Strohmer Proof. By definition of Sn we have A S = E , n − n n 1 where En = BnAn− Bn∗. We know from equation (3.8) in the proof of Theorem 3.1 that the entries of En can be bounded by 2γ1n γ1(k+l) 2γ1n γ1 k l γ1(k+l) (E ) c(e− e + e− e | − | + e− ). (4.1) | n kl|≤ n For x = [x0, x1,...,xn 2, xn 1] C and N < n/2 we define the orthogonal − − ∈ projection PN by PN x = [0,..., 0, xN , xN+1,...,xn N 1, 0,..., 0], − − n 2N (N) (N) and identify the image of PN with C − . We set En = PN EnPN . In words, En is obtained from En by taking only the central (n 2N) (n 2N) submatrix of En − × − (N) and setting the other entries surrounding this block equal to zero. Then E En n − has 2N “full” rows and n 2N “sparse” rows, where each of the latter rows has − non-zero entries only at the first N and last N coordinates. Thus the dimension of (N) the space spanned by the sparse rows is at most 2N. Hence rank(E En ) 4N. n − ≤ Due to the decay properties of En it is easy to see that n N 1 − − E(N) = (E(N)) . k n k1 | n Nl| lX=N Using (4.1) we get after some straightforward calculations (N) γ1n γ1(n 2N) 2γ1N E c(e− + e− − + e− ). (4.2) k n k1 ≤ It is obvious that for each given ε > 0 we can find an N such that for all n > 2N (N) (N) (N) (N) En 1 ε. Since En is hermitian, we have En 1 = En . Thus k k ≤ k k k k∞ (N) (N) (N) 1/2 En 2 ( En 1 En ) ε. k k ≤ k k k k∞ ≤ (N) Hence for large n the spectrum of En lies in ( ε,ε). By the Cauchy interlace − theorem we conclude that at most 4N eigenvalues of A S have absolute value n − n exceeding ε. Lemma 2.4 implies that for any ε> 0 we can find an M such that for all n>M 1 1 Sn and Sn− exist and Sn− is bounded by fmin ε> 0. Proceeding as in [3], Chapter 1 k k − 2, we express Sn− An as 1 1 S− A = I + S− (A S ) n n n n n − n and arrive at the following Corollary 4.2. Let A = ak,l be a hermitian positive definite Toeplitz matrix γ k { } with a ce− | | for γ > 0. Then for all ε> 0 there exist N and M > 0 such that | k|≤ Four short stories about Toeplitz matrix calculations 21 1 for all n>M at most N eigenvalues of S− A I have absolute values larger than n n − n ε. With the results presented in this paper it should not be difficult for the reader to derive Theorem 4.1 and Corollary 4.2 for Toeplitz matrices with polynomial decay. Theorem 4.1 (in particular (4.2)) and Corollary 4.2 show that the clustering behavior 1 of Sn− An is the stronger the faster the decay of the entries of the Toeplitz matrix is. In [19] and in [14] Nagy et al. consider the solution of convolution equations and banded Toeplitz systems by the preconditioned conjugate gradient method using a preconditioner similar to the one in this section. In [19] the Toeplitz matrix is rectan- gular, but the embedding is in principle the same. The theoretical results presented in [19, 14] only hold for banded Toeplitz systems, but in the numerical experiments the authors consider also non-banded Toeplitz systems, where the Toeplitz matrix has exponential decay (see [19]) or polynomial decay (see Example 4 in [14]). It is noted in [19] that “it is surprising that the number of iterations is still quite small”. The authors also point out that the numerical experiments indicate additional clustering around one of the spectrum of the preconditioner matrix, which is not covered by their theoretical results. With Theorem 4.1 and Corollary 4.2 at hand we can provide a theoretical explanation for the numerical observations in [19] and [14], at least for the 1-D case. The key lies in the fast decay of the inverse of the Toeplitz matrix. Acknowledgement. I want to thank Thomas Kailath and Ali Sayed for fruitful discussions on this topic, Torsten Ehrhardt for pointing out reference [10] to me, Gabriele Steidl for communicating to me part of the proof of Theorem 2.6 and the referees for a careful reading of the manuscript. REFERENCES [1] A. Beurling, Sur les int´egrales de Fourier absolument convergentes et leur application `aune transformation fonctionelle, in Ninth Scandinavian Math. Congress, Helsingfors, 1938, pp. 345–366. [2] A. Bottcher¨ and B. Silbermann, Analysis of Toeplitz operators, Springer-Verlag, Berlin, 1990. [3] R. Chan, Circulant preconditioners for hermitian Toeplitz systems, SIAM J. Matrix Anal. Appl., 10 (1989), pp. 542–550. [4] R. Chan and M. Ng, Conjugate gradient methods for Toeplitz systems, SIAM Review, 38 (1996), pp. 427–482. [5] R. Chan and G. Strang, Toeplitz equations by conjugate gradients with circulant precondi- tioner, SIAM J. Sci. Stat. Comput., 10 (1989), pp. 104–119. [6] R. H. Chan and M. Ng, Toeplitz preconditioners for Hermitian Toeplitz systems, Linear Algebra Appl., 190 (1993), pp. 181–208. [7] P. Davis, Circulant Matrices, John Wiley, 1979. [8] S. Demko, W. Moss, and P. Smith, Decay rates for inverses of band matrices, Math. Comp., 43 (1984), pp. 491–499. [9] Y. Domar, Harmonic analysis based on certain commutative Banach algebras, Acta Math., 96 (1956), pp. 1–66. 22 Thomas Strohmer [10] I. Gelfand, D. Raikov, and G. Shilov, Commutative normed rings, Chelsea Publishing Co., New York, 1964. Translated from the Russian. [11] I. Gohberg and I. Fel’dman, Convolution equations and projection methods for their solution, American Mathematical Society, Providence, R.I., 1974. Translated from the Russian by F. M. Goldware, Translations of Mathematical Monographs, Vol. 41. [12] I. Gohberg, S. Goldberg, and M. A. Kaashoek, Classes of linear operators. Vol. II, vol. 63 of Operator Theory: Advances and Applications, Birkh¨auser Verlag, Basel, 1993. [13] I. Gohberg, M. Hanke, and I. Koltracht, Fast preconditioned conjugate gradient algorithms for Wiener-Hopf integral equations, SIAM J. Numer. Anal., 31 (1994), pp. 429–443. [14] M. Hanke and J. G. Nagy, Toeplitz approximate inverse preconditioner for banded Toeplitz matrices, Numer. Algorithms, 7 (1994), pp. 183–199. [15] R. Horn and C. Johnson, Topics in matrix analysis, Cambridge University Press, Cambridge, 1994. [16] S. Jaffard, Propri´et´es des matrices “bien localis´ees” pr`es de leur diagonale et quelques appli- cations, Ann. Inst. H. Poincar´eAnal. Non Lin´eaire, 7 (1990), pp. 461–476. [17] T. Kailath and A. H. Sayed, Displacement structure: theory and applications, SIAM Rev., 37 (1995), pp. 297–386. [18] S. Levin, Asymptotic properties of Toeplitz matrices, PhD thesis, The Weizmann Institute, Rehovot, Israel, 1980. [19] J. Nagy, R. Plemmons, and T. Torgersen, Iterative image restoration using approximate inverse preconditioning, IEEE Trans. Image Processing, 15 (1996), pp. 1151–1162. [20] M. K. Ng, R. H. Chan, and W.-C. Tang, A fast algorithm for deblurring models with Neu- mann boundary conditions, SIAM J. Sci. Comput., 21 (1999), pp. 851–866 (electronic). [21] J. G. Proakis, Digital Communications, McGraw-Hill, New York, 3rd ed., 1995. [22] H. Reiter, Classical Harmonic Analysis and Locally Compact Abelian Groups, Oxford Uni- versity Press, 1968. [23] R. Richtmyer and K. Morton, Difference Methods for Initial-Value Problems, Krieger Pub- lishing Company, Malabar, Florida, 1994. [24] G. Steidl, Personal communication, 2000. [25] T. Strohmer, Rates of convergence for the approximation of dual shift-invariant systems in 2 ℓ (Z), J. Four. Anal. Appl., 5 (2000), pp. 599–615.