Optimal Conditioning of Vandermonde-Like Matrices And

Home , Vandermonde matrix

OPTIMAL CONDITIONING OF VANDERMONDE-LIKE MATRICES AND

A MEASUREMENT PROBLEM

A dissertation submitted to

Kent State University in partial

fulﬁllment of the requirements for the

degree of Doctor of Philosophy

Mykhailo Kuian

May, 2019 Dissertation written by

Mykhailo Kuian

B.S.,Taras Shevchenko National University of Kyiv, 2010

M.S.,Taras Shevchenko National University of Kyiv, 2012

M.S., Kent State University, 2014

Ph.D., Kent State University, May, 2019

Approved by

Lothar Reichel, Sergij Shiyanovskii , Chairs, Doctoral Dissertation Committee

Jing Li , Members, Doctoral Dissertation Committee

Xiaoyu Zheng ,

Arden Ruttan , Robin Selinger

Accepted by

Dr. Andrew Tonge , Chair, Department of Mathematical Sciences

Dr. James Blank , Dean, College of Arts and Sciences TABLE OF CONTENTS

TABLE OF CONTENTS ...... iii

ACKNOWLEDGEMENTS ...... v

1 Introduction ...... 1

1.1 Conditioning and condition numbers ...... 1

1.2 Interpolation, least squares problems and Vandermonde matrices ...... 4

1.3 Description of the measurement problem ...... 7

2 Optimally conditioned Vadermonde-like matrices ...... 9

2.1 Introduction ...... 9

2.2 Square Vandermonde-like matrices ...... 11

2.3 Rectangular Vandermonde-like matrices ...... 15

2.4 Szeg˝o-Vandermonde matrices ...... 24

2.5 General Vandermonde-type matrices ...... 35

2.6 Chapter summary ...... 42

3 Fast factorization of rectangular Vandermonde matrices with Cheby-

shev nodes ...... 43

3.1 Introduction ...... 43

3.2 Fast factorization methods for Vandermonde matrices deﬁned by zeros of

Chebyshev polynomials...... 48

iii 3.3 Fast factorization methods for Vandermonde matrices deﬁned by extrema

of Chebyshev polynomials and extensions ...... 52

3.4 Numerical experiments...... 54

3.5 Chapter summary ...... 59

4 Conditioning optimization of a measurement problem ...... 62

4.1 Introduction ...... 62

4.2 Determination of control parameters for CMPM systems ...... 64

4.3 Application of the proposed method to the PolScope ...... 70

4.4 Noise contamination of the measurement process ...... 80

4.5 Chapter summary ...... 84

5 Conclusion and future work ...... 86

5.1 Conclusion ...... 86

5.2 Future work ...... 87

Bibliography ...... 89

iv ACKNOWLEDGEMENTS

First of all, I would like to thank my advisors, Dr. Lothar Reichel and Dr. Sergij

Shiyanovskii for wisdom, guidance and numerous advises which made it possible for this work to come true. They both have a great impact on my knowledge, professional devel- opment, and shared a lot of exciting ideas. I am very happy that I met those people in my life.

I am grateful to all professors I have had a pleasure to take classes from at Kent State and Kyiv State Universities, and to my high school teachers.

I would especially like to thank my family who always helped and motivated me.

v CHAPTER 1

Introduction

1.1 Conditioning and condition numbers

The main purpose of this dissertation is to describe several problems of the optimization of conditioning. Conditioning is one of the fundamental issues of numerical analysis, an essential tool for error analysis. Computations are performed in ﬂoating point arithmetic with inevitable and unavoidable errors of rounding. Furthermore, the data itself may be the result of an experiment or measurement that is subject to errors. Conditioning indicates how rounding errors in computations and errors in the data aﬀect the computed solution.

Let us start with the deﬁnition of conditioning. Conditioning of a problem concerns the sensitivity to perturbation of a mapping f from a normed vector space of data X to

a normed vector space of solutions Y . The conditioning measures how much the output

Y can change due to a small perturbation in the input X, which may stem from errors

in the data or round-oﬀ errors introduced during the solution process. In other words,

conditioning is an upper bound of the ratio of the size of the solution error to the size of

the data error. The conditioning is measured by a condition number.

Deﬁnition 1.1.1 Let δx denote small perturbation of x and let δf = f(x + δx) − f(x).

The absolute condition number κ˜ =κ ˜(x) is defined as kδfk κ˜ = lim sup . δ&0 kδxk≤δ kδxk Definition 1.1.2 The relative condition number κ˜ =κ ˜(x) is defined as kδf(x)k kδxk κ˜ = lim sup / . δ&0 kδxk≤δ kf(x)k kxk

1 A small relative condition number indicates that a small relative error in the data only

causes a small relative error in the solution, in this case problem is called well-conditioned.

Conversely, a large condition number signals that the computed solution may be very

sensitive to an error in the data; such a problem is called ill-conditioned.

The conditioning of the system of linear equations

(1.1) Ax = b, A ∈ Cn×n, x ∈ Cn, b ∈ Cn, where the matrix A is nonsingular is of fundamental importance in numerical linear algebra.

Theorem 1.1.1 Let the vector b be ﬁxed and consider the problem of computing the solution x of the system (1.1). The relative condition number of this problem with respect to perturbations in the matrix A is

κ(A) = kAkkA−1k.

Theorem 1.1.2 Let matrix A be ﬁxed and consider the problem of computing solution x of the system (1.1). The relative condition number of this problem with respect to perturbations in the vector b is also

κ(A) = kAkkA−1k.

The least-squares problem can be formulated as residual minimization

(1.2) minkAx − bk2, x∈Cn

N×n N where A ∈ C is of full rank N ≥ n, b ∈ C and k·k2 denotes the Euclidian vector norm. The solution x of (1.2) is given by

x = A†b,

2 where A† denotes the Moore-Penrose pseudoinverse of A,

A† = (A∗A)−1A∗, when N ≥ n, and A† = A(AA∗)−1, when n > N.

Obviously A† reduces to the inverse A−1 when n = N.

The product

(1.3) κ(A) = kAkkA†k is deﬁned as condition number of matrix A.

The next theorem describes conditioning properties of the least squares problem.

Theorem 1.1.3 Let b ∈ CN , the matrix A ∈ CN×n be of full rank, and deﬁne y = AA†b, kAkkxk kyk η = , θ = arccos . The least squares problem (1.2) has the following spec- kyk kbk tral norm relative condition numbers:

a) The relative condition number with respect to perturbations in the vector b equals

κ(A) (1.4) κ = . b→x η cos θ

b) The relative condition number with respect to perturbations in the matrix A equals

κ(A)2 tan θ (1.5) κ = ; A→x η

see Trefethen and Bau [50] for more details. As we can see from Theorems 1.1.1 and 1.1.2, the condition number of the matrix (1.3) furnishes a bound for the relative error in the solution of a linear system of equations (1.1). At the same time, from relations (1.4) and

(1.5) of Theorem 1.1.3, the condition number also furnishes a bound for the relative error in the solution of least squares problems. Minimization of the matrix condition number

(1.1.1) minimizes an upper bound for relative error in the computed solutions.

The following section brieﬂy describes Vandermonde matrices.

3 1.2 Interpolation, least squares problems and Vandermonde matrices

Vandermonde matrices, see (1.6), arise frequently in computational mathematics in

problems that require polynomial approximation, diﬀerentiation, or integration. These

matrices are deﬁned by a set of N distinct nodes x1, x2,...,xN and a monomial basis,

  n−1 1 x1 ··· x1    n−1   1 x2 ··· x2  (1.6) V :=   ∈ N×n. N,n  . . .  C  . . .   . . ··· .    n−1 1 xN ··· xN

When n = N, the Vandermonde matrix (1.6) has an explicit formula for the determinant, Y that equals (xi − xj). Quadratic, and consequently also rectangular Vandermonde 1≤j

The polynomial interpolation problem with distinct interpolation points and the polynomial represented in the power basis gives rise to a linear system of equations with a

Vandermonde matrix. Suppose we are given n distinct points x1, ..., xn ∈ C and data y1, ..., yn ∈ C at these points. Then there exists a unique polynomial

n−1 (1.7) p(x) = c0 + c1x + ... + cn−1x ,

such that p(xi) = yi, i = 1, ..., n. The relationship between the sets {xi} and {yi} is expressed via a Vandermode matrix

4       n−1 1 x1 ··· x1 c0 y1        n−1   .   .   1 x2 ··· x2   .   .  (1.8)     =    . . .       . . .       . . ··· .            n−1 1 xn ··· xn cn−1 yn

Least squares ﬁtting problem is an indispensable tool, where rectangular Vandermonde matrices appear

N n−1 X 2 X j T (1.9) min (p(c, xi) − yi) , p(c, x) = cjx , c = [c0, c1, . . . , cn−1] , c∈Rn i=1 j=0

PN 2 2 The sum of squares i=1 (p(c, xi) − yi) is equal to the square of the residual kV x − bk2 for the rectangular Vandermonde system       n−1 1 x1 ··· x1 c0 y1        n−1   .   .   1 x2 ··· x2   .   .      ≈    . . .       . . .       . . ··· .            n−1 1 xN ··· xN cn−1 yN

A diﬃculty with Vandermonde matrices is that they typically are highly ill-conditioned when the nodes are real. The ill-conditioning can be reduced by using a basis of orthogonal polynomials instead of monomials.

Definition 1.2.1 A family of orthogonal polynomials is a sequence of polynomials such that any two polynomials of different degrees are orthogonal to each other with respect to some inner product in either a finite or infinite interval [a, b].

5 Deﬁnition 1.2.2 An inner product (relative to the measure dλ(x)) of two polynomials p(x) and q(x) is deﬁned as

Z b (1.10) (p, q)λ := p(x)q(x)dλ(x). a

The discrete inner product has form

X (1.11) (p, q)λ := p(xi)q(xi)λ(xi). i

Let x1, x2, . . . , xN be distinct nodes in the complex plane C and let p0, p1, p2,... be a orthogonal polynomial family with deg(pj) = j. Matrices of the form

  p0(x1) p1(x1) ··· pn−1(x1)      p0(x2) p1(x2) ··· pn−1(x2)  (1.12) V :=   ∈ N×n N,n  . . .  C  . . .   . . ··· .    p0(xN ) p1(xN ) ··· pn−1(xN ) are known as Vandermonde-like matrices. Optimally conditioned and optimally scaled square Vandermonde and Vandermonde-like matrices with real nodes were ﬁrstly analyzed by Walter Gautschi.

Chapter 2 extends Gautschi’s analysis to rectangular Vandermonde-like matrices with real nodes, as well as to Vandermonde-like matrices with nodes on the unit circle in the complex plane. Existence and uniqueness of optimally conditioned Vandermonde-like matrices are investigated. Using classical Posse’s theorem Gautschi showed uniqueness of optimally conditioned quadratic Vandermonde-like matrices. In this Chapter we show generalization of Posse theorem for Gauss quadrature rule. Using this result we extend Gautschi’s analysis to rectangular Vandermonde-like matrices. Next we also derive an extension of Posse’s

6 theorem for Gauss–Szeg˝oquadrature rules and analyze Szeg˝o-Vandermonde-like matrices.

The ﬁnal part of Chapter 2 discuss properties of rectangular general Vandermonde-type

matrices VN,n of order N × n, N 6= n, with Chebyshev nodes or equidistant nodes on the

unit circle in the complex plane. It is shown that the condition number of these matrices

is independent of the number of nodes.

Chapter 3 describes explicit QR and QR-like factorizations of Vandermonde matrices

with Chebyshev nodes and Chebyshev extreme nodes. These matrices arise in polynomial

least squares approximation problems and yield the polynomial in a particularly simple

form for further processing, including diﬀerentiation and integration. Gautschi showed

that the condition number of a square Vandermonde matrix V ∈ Rn×n with Chebyshev nodes is near-optimal, which makes it even more reasonable to use them. Based on QR and QR-like factorizations we derive fast solution methods of least squares problem (1.2), where A is Vandermonde matrices with Chebyshev nodes. Both methods are compared to a factorization approach shown by Eisinberg et al. in [13], and demonstrate more than double acceleration in computational speed.

1.3 Description of the measurement problem

Chapter 4 presents a conditioning analysis of a measurement problem. Speciﬁcally, we investigate combined multi-measuring systems that determine several unknown quantities

Λ from measurements of a single variable bi at diﬀerent preprogrammed conditions determined by control parameters Ai. The considered measurement problem can be described by a nonlinear system of equations

(1.13) Φ(Ai, Λ) = bi, i = 1, ..., n.

7 Perturbations in the control parameters Ai and in measured data bi are considered. The mathematical model of conditioning of the system (1.13) are analytically derived, with applicability to real-world measurements problem, and are substantiated through numerical experiments. The choice of optimal control parameters Ai is based on the minimization of the conditioning optimization over all possible range of values of determined quantities Λ.

Using the submultiplicativity of the spectral and Frobenius matrix norms, we construct an

n upper bound of the error function and determine the set of control parameters A = {Ai}i=1 by minimizing this bound. To demonstrate the capability of the proposed method, we apply it to the polarized light microscopy technique called LC-PolScope, equipped with controlled liquid crystal plates. LC-PolScope is used for determining inhomogeneous two-dimensional

fields of optical retardation, and orientation of optical axis in thin organic and inorganic samples. We compare the computed set of control parameters A with other sets, including the one used in the PolScope, and demonstrate the benefits of conditioning optimization even though it does not take into account any specific features of the PolScope.

8 CHAPTER 2

Optimally conditioned Vadermonde-like matrices

2.1 Introduction

As we noted before the main diﬃculty using Vandermonde matrices (1.6) is the fact that they typically are quite ill-conditioned when the nodes are real, having exponential growth of the conditional number with n. The ill-conditioning can be reduced by using a basis of orthogonal polynomials p0, p1, ..., pn, with deg(pj) = j. This was ﬁrst observed by Gautschi. The matrices so obtained are commonly referred to as Vandermonde-like and are of the form (1.12). Gautschi analyzed optimally conditioned and optimally scaled square Vandermonde and Vandermonde-like matrices with real nodes. In this chapter we extend Gautschi’s analysis to rectangular Vandermonde-like matrices with real nodes, as well as to Vandermonde-like matrices with nodes on the unit circle in the complex plane. Additionally, we investigate existence and uniqueness of optimally conditioned

Vandermonde-like matrices. Finally, we discuss properties of rectangular Vandermonde and Vandermonde-like matrices VN,n of order N × n, N 6= n, with Chebyshev nodes or

with equidistant nodes on the unit circle in the complex plane, and show that the condition

number of these matrices can be bounded independently of the number of nodes.

Baz´an[2] observed that rectangular Vandermonde matrices with n N can be fairly

well-conditioned when the nodes xi are close to the unit circle in the complex plane and

pairwise not too close. However, the situation when the nodes are real is quite diﬀerent.

Gautschi [17] has shown that the inverse of a square Vandermonde matrix is of large norm

when the nodes are real. This results in a large condition number. Further investigations

9 by Gautschi [16] and Gautschi and Inglese [19] provide bounds for condition numbers; the

latter work shows that the condition number of square Vandermonde matrices with real

nodes grows exponentially with the number of nodes. The conditioning of square Vander-

monde matrices also is investigated by Beckermann [3], Eisinberg et al. [13], Gautschi [23],

and Tyrtyshnikov [52]. To circumvent the ill-conditioning of square Vandermonde matrices

with real nodes, Gautschi [18] introduced square Vandermonde-like matrices in which the

power basis is replaced by a basis of polynomials that are orthonormal with respect to an

inner product deﬁned by a non-negative measure with support on the real axis.

This chapter is organized as follows. Section 2.2 reviews results by Gautschi on the

conditioning of square Vandermonde and Vandermonde-like matrices. Section 2.3 extends

Gautschi’s analysis to rectangular Vandermonde and Vandermonde-like matrices. Using a

result by Posse [41], Gautschi [18, 23] showed that square Chebyshev-Vandermonde ma-

n trices Vn,n = [Ti−1(xj)]i,j=1, where the Ti−1 are Chebyshev polynomials of the ﬁrst kind orthogonal on the interval [a, b], and the nodes x1, x2, . . . , xn are the zeros of polynomial

Tn, are the only Vandermonde-like matrices that are optimally conditioned for all n ≥ 1.

In Section 2.3 we provide generalization of Posse theorem for Gauss quadrature rule. Using

this result we extend Gautschi’s analysis to rectangular Vandermonde-like matrices VN,n, where N 6= n. Section 2.4 considers square and rectangular Vandermonde-like matrices deﬁned by polynomials that are orthogonal with respect to an inner product deﬁned by a non-negative measure on the unit circle in the complex plane C and by nodes that are the abscissas of a Gauss–Szeg˝oquadrature rule associated with this inner product. There we derive an extension of Posse’s theorem for Gauss–Szeg˝oquadrature rules. Section 2.5 discusses the conditioning of Vandermonde-like matrices determined by a general polynomial basis and Chebyshev nodes, and extends results shown by Eisinberg et al. [13].

10 Speciﬁcally, we show that the Frobenius and spectral condition numbers of Vandermonde

and Vandermonde-like matrices with Chebyshev do not depend on number of nodes. An

analogous result for Vandermonde matrices with equidistant nodes on the unit circle in C also is shown. Finally, Section 2.6 contains concluding remarks.

2.2 Square Vandermonde-like matrices

This section reviews results shown by Gautschi [18] for square Vandermonde-like ma-

trices. Let p0, p1, p2,... be a family of polynomials, with deg(pj) = j, that are orthogonal with respect to an inner product determined by a real non-negative measure dλ with support on the real axis, Z (2.1) (f, g)λ := f(x)g(x)dλ(x).

Further, assume that the polynomials are normalized to be of unit length with respect to

the norm associated with this inner product. Thus,   0, j 6= k, (2.2) (pj, pk)λ =  1, j = k.

Introduce the N-node Gauss quadrature rule associated with the measure dλ,

N X (N) (N) (2.3) GN f = λk f(xk ). k=1

(N) (N) (N) The nodes x1 , x2 , . . . , xN are known to be distinct and in the convex hull of the support

(N) (N) (N) of dλ, and the weights λ1 , λ2 , . . . , λN are positive. This quadrature rule can be applied to approximate the integral Z If = f(x)dλ(x).

It is characterized by the property

If = GN f ∀f ∈ P2N−1,

11 where P2N−1 denotes the set of polynomials of degree at most 2N − 1; see, e.g., Gautschi [21] and Szeg˝o[49] for discussions on Gauss quadrature. The Christoﬀel function associated

with the Gauss rule (2.3) can be expressed as

N−1 !−1 X 2 (2.4) ΛN (x) = pj (x) ; j=0

see, e.g., Szeg˝o[49, Chapter 2]. Evaluation of this function at the Gaussian nodes yields

the Gaussian weights,

(N) (N) (2.5) λk = ΛN (xk ), 1 ≤ k ≤ N,

which also are known as Christoﬀel numbers. Gautschi [18, Theorem 2.1] showed the

following result for square Vandermonde-like matrices.

Proposition 2.2.1 Let the Vandermonde-like matrix deﬁned by (1.12) with n = N be determined by the orthonormal polynomials that satisfy (2.2) and by the Gaussian nodes

(N) xk := xk , 1 ≤ k ≤ N. Then

N N !1/2 −1 X (N) X (N) (2.6) κF (VN,N ) = λk · λk , k=1 k=1

(N) where the λk are the Gaussian weights (2.5).

The Cauchy inequality applied to the right-hand side of (2.6) shows that

κF (VN,N ) ≥ N

(N) with equality if and only if all the Christoﬀel numbers λk are equal. Gautschi [18] showed that the Chebyshev measure

(2.7) dλ(x) = (1 − x2)−1/2dx, −1 < x < 1,

12 determines quadrature nodes that give optimally conditioned Vandermonde-like matrices

VN,N for all N ≥ 1. To prove this, Gautschi [18] applied the following result shown by

Posse [41].

Theorem 2.2.1 (Posse, 1875) Let dλ be a non-negative measure deﬁned on the interval

[−1, 1] and let p0, p1, p2,... be a sequence of normalized orthogonal polynomials, i.e., they satisfy deg(pj) = j for all j = 0, 1, 2,... , and (2.2). Denote the zeros of pN by

(N) (N) x1 , . . . , xN . If for all N = 1, 2, 3,... , it holds that

Z 1 N X (N) p(x)dλ(x) = νN p(xk ) −1 k=1 for some scalar νN and all polynomials p ∈ P2N−1, then dλ is the Chebyshev measure (2.7).

We recall that the orthonormal polynomials with respect to this measure are given by  r  1  T0(x), j = 0,  π (2.8) pj(x) = r  2  T (x), j = 1, 2,...,  π j where the Tj(x) are known as Chebyshev polynomials of the ﬁrst kind. For interval −1 ≤ x ≤ 1 they can be deﬁned as

(2.9) Tj(x) = cos(j arccos(x)), j = 0, 1, 2,...,

The Gaussian nodes and weights associated with the measure (2.7) are given by

2k − 1 π (2.10) x(N) = cos π , λ(N) = , 1 ≤ k ≤ N. k 2N k N

Obviously the result of Theorem 2.2.1 can be extended for any interval [a, b], with corresponding transformation of the measure dλ and nodes (2.10). Square Vandermonde-like matrices with distinct nodes (1.12) are known to be full rank; see, e.g., [48, Section 3.6].

13 Let the Vandermonde-like matrix VN,N have the singular values σ1 ≥ σ2 ≥ ... ≥ σN > 0.

Gautschi’s proof [18, Theorem 2.1] of Proposition 2.2.1 shows that the singular values are

square roots of the reciprocal Christoﬀel numbers, up to a renumbering.

Corollary 2.2.1 All singular values of the Vandermonde-like matrix VN,N are equal if the

polynomials pj are deﬁned by (3.10) and the nodes are the Chebyshev points (2.10). Then

VN,N is optimally conditioned, i.e., κF (VN,N ) = N, κ2(VN,N ) = 1.

Let p0, p1, p2,... be a family of polynomials that are orthogonal with respect to an inner product (2.1) with a nonnegative measure with support on the real axis.

Lemma 2.2.1 Any square Vandermonde-like matrix that is deﬁned by such a family of polynomials p0, p1, p2,... can be made optimally conditioned by an appropriate choice of nodes and row scaling.

Proof: This can be shown as follows. Let c ∈ R be a constant and let the nodes

x1, x2, . . . , xN be the zeros of the polynomial pN (x) − c pN−1(x). It follows from [49, Theo- rem 3.3.4] that the zeros of pN (x)−c pN−1(x) are distinct and real. The Christoﬀel-Darboux formula for i 6= j yields

N−1 X µN−1 pN (xi)pN−1(xj) − pN (xj)pN−1(xi) (2.11) pk(xi)pk(xj) = , µN xi − xj k=0 where µk is the leading coeﬃcient of pk(x). Since pN (xi) = c pN−1(xi), we have

pN (xi)pN−1(xj) = c pN−1(xi)pN−1(xj), pN (xj)pN−1(xi) = c pN−1(xj)pN−1(xi).

Substitution into (2.11) yields

N−1 X pk(xi)pk(xj) = 0, i 6= j. k=0

Hence, the rows of the matrix VN,N are orthogonal. Normalizing the rows of VN,N makes

the matrix orthogonal and, therefore, optimally conditioned. 14 2.3 Rectangular Vandermonde-like matrices

Here we extend the results of the previous section to rectangular Vandermonde-like matrices.

Lemma 2.3.1 Let p0, p1, . . . , pn−1 be sequence of orthogonal polynomials polynomials such that deg(pj) = j, and assume that the points x1, x2, . . . , xN are distinct in the complex plane. Then the rectangular Vandermonde-like matrix (1.12) is of full rank.

Proof: Let m = min{n, N}. The leading m × m principal submatrix of the matrix (1.12) is nonsingular by [48, Theorem 3.6.11].

Lemma 2.3.2 An optimally conditioned matrix A ∈ CN×n can be written as A = σQ, where σ is positive constant and the matrix Q has orthonormal columns if N ≥ n, and orthonormal rows if N ≤ n.

Proof: The lemma follows from the SVD of A = UΣW ∗. Here U ∈ CN×N and W ∈ Cn×n

N×n are unitary matrices. Assume that N ≥ n. Then Σ = diag[σ1, σ2, . . . , σn] ∈ R with ˆ N×n σ1 = ... = σn. Let the matrix U ∈ C be made up of the n ﬁrst columns of U. Then ˆ ∗ ˆ ∗ A = σ1UW , where the matrix UW has orthonormal columns. The proof for N < n proceeds similarly.

Lemma 2.3.3 Let the polynomials pj in (1.12) be Chebyshev polynomials (3.10) and let the nodes xj be the zeros (2.10) of pN . Then r Nn kV k = . N,n F π

Proof: The polynomials (3.10) satisfy for 0 ≤ i, j < N,  N  X  0, i 6= j, (2.12) pj(xk)pi(xk) = N k=1  , i = j.  π 15 Therefore, Nn kV k2 = trace(V ∗ V ) = . n,N F n,N N,n π

Theorem 2.3.1 Rectangular Vandermonde matrices VN,n of normalized Chebyshev poly-

nomials (3.10) with the nodes (2.10) are optimally conditioned with respect to the Frobenius

norm for all 1 ≤ n ≤ N. Clearly, they also are optimally conditioned with regard to the

spectral norm.

Proof: The polynomials (3.10) are orthogonal with respect to a discrete inner product; N cf. (2.12). This equation implies that V ∗ V = I, from which it follows that all N,N N,N π p singular values are N/π. Consider the Chebyshev-Vandermonde matrix VN,N−1 obtained by removing the last column of VN,N . The singular values of VN,N−1 interlace those of VN,N ; p see, e.g., [32, Theorem 7.3.3]. Therefore, all singular values of VN,N−1 are equal to N/π.

We now remove the last column of VN,N−1 to obtain the matrix VN,N−2. By the same argument, all singular values of the latter matrix are equal to pN/π. We proceed by repeatedly removing the last column until the matrix VN,n is obtained. All singular values p of this matrix are N/π. It follows that κF (VN,n) = n and κ2(VN,n) = 1. We now consider the situation when N ≤ n.

Corollary 2.3.1 Rectangular Vandermonde matrices VN,n of normalized Chebyshev poly-

nomials (3.10) with the nodes (2.10) are optimally conditioned with respect to the Frobenius

norm for all 1 ≤ N ≤ n. Clearly, they also are optimally conditioned with regard to the

spectral norm.

Proof: The result can be shown similarly as Theorem 2.3.1. Thus, we observe that all p singular values of VN,N equal N/π. Now remove the last row of VN,N to obtain the

16 matrix VN−1,N . By the same argument as in the proof of Theorem 2.3.1 all singular values p of VN−1,N have the value N/π. We continue to remove last rows until the matrix VN,n is obtained. All its singular values are equal. Therefore it is optimally conditioned with respect to both the Frobenius and spectral norms.

Remark 2.3.1 The Christoﬀel numbers for the Gauss rule (2.3) associated with the Cheby- shev measure (2.7) easily can be determined by using the fact that the matrix VN.N is op- N timally conditioned. We have from the proof of Theorem 2.3.1 that V ∗ V = I. N,N N,N π N It follows that V V ∗ = I. The diagonal entries are the reciprocal values of the N,N N,N π Christoﬀel numbers; cf. (2.10).

We note that the matrix VN,N is closely related to the discrete cosine transform DCT-III

matrix, which is important in numerous applications in science and engineering; see [44]. r π The latter matrix is obtained from V by scaling by the factor , using the relation N,N N (N) (2.9), and the fact that the nodes xk are given by (2.10). This yields the orthonormal cosine transform matrix

 r 1 r 2 2π r 2 (N − 1)π   cos ··· cos   N N 2N N 2N   r r r   1 2 2 · 3π 2 (N − 1)3π   cos ··· cos    (2.13)  N N 2N N 2N  .  . . . .   . . . .     r r r   1 2 (2N − 1)π 2 (N − 1)(2N − 1)π  cos ··· cos N N 2N N 2N

Since the above matrix is orthogonal, we obtain optimally conditioned submatrices by re-

moving either selected rows or selected columns; similarly as in the proofs of Theorem 2.3.1

or Corollary 2.3.1. We would like to explore whether the measure that generates optimally

17 conditioned Vandermonde matrices of Theorem 2.3.1 and Corollary 2.3.1 is unique. To-

wards this end, we ﬁrst show the following result, which gives a diﬀerent characterization

of the measure (2.7) than Theorem 2.2.1.

Theorem 2.3.2 Let dλ be a positive measure deﬁned on the interval [−1, 1] and let p0(x), p1(x),...

be a sequence of associated monic orthogonal polynomials. Denote the zeros of pN (x) by

(N) (N) x1 , . . . , xN . If for all N = 1, 2, 3,... , it holds that

Z 1 N X (N) (2.14) p(x)dλ(x) = νN p(xj ) −1 j=1

for some scalar νN and all polynomials p ∈ P2, then dλ is the Chebyshev measure (2.7) (possibly scaled).

Proof: It suﬃces to show that if for any N, there is a constant νN such that

N Z 1 m m X (N) (2.15) x dλ(x) = νN xk for m = 0, 1, 2, −1 k=1 then dλ is the measure (2.7) or a scaling thereof. Consider (2.15) for increasing values of m. For m = 0, we have Z 1 dλ(x) = NνN . −1 R 1 We may assume that −1 dλ(x) = 1. Then νN = 1/N. Turning to m = 1, we get

Z 1 N 1 X (N) (2.16) µ := x dλ(x) = x . 1 N k −1 k=1

Express the monic orthogonal polynomial pN ∈ PN in the form

N N−1 N−2 pN (x) = x + aN,N−1x + aN,N−2x + ... + aN,0.

18 The relation between the zeros and coeﬃcients of pN (also known as Vieta’s formulas)

yields

N X (N) (2.17) xk = −aN,N−1, k=1 X (N) (N) (2.18) xk xl = aN,N−2. 1≤k

with support on a real interval satisﬁes a recurrence relation of the form

(2.19) pk(x) = (x − αk)pk−1(x) − βk−1pk−2(x), k = 2, 3,... ;

see, for example, [21, Theorem 1.27]. Here we assume that the measure has inﬁnitely many

points of support. Comparing coeﬃcients of xk−1 in the right-hand side and left-hand side of (2.19) for k = N,N − 1,..., 2, we obtain the relations

aN,N−1 = −αN + aN−1,N−2,

aN−1,N−2 = −αN−1 + aN−2,N−3,

...

a2,1 = −α2 + a1,0.

Summing these relations yields

aN,N−1 = −(αN + αN−1 + ... + α2) + a1,0.

It follows from p1(x) = x + a1,0 = x − α1 that

N X aN,N−1 = − αk. k=1 Using (2.17) and (2.16) gives N X αk = Nµ1. k=1 19 Letting N = 1, 2,... in the above sum, we obtain

(2.20) µ1 = α1 = α2 = ... = αN .

Hence,

(2.21) aN,N−1 = −Nµ1.

This relation holds for N = 1, 2, 3,... . Consider the case m = 2. We have

N Z 1 2 1 X (N) µ := x2dλ(x) = x . 2 N k −1 k=1 Using N N !2 2 X (N) X (N) X (N) (N) xk = xk − 2 xk xl , k=1 k=1 1≤k

N (2.22) a = (Nµ2 − µ ). N,N−2 2 1 2

Comparing coeﬃcients of the power xk−2 in the left-hand side and right-hand side of

equation (2.19) for k = N + 2,N + 1,..., 2, gives the relations, in order,

aN+2,N = aN+1,N−1 − αN+2aN+1,N − βN+1,

aN+1,N−1 = aN,N−2 − αN+1aN,N−1 − βN ,

...

a3,1 = a2,0 − α3a2,1 − β2,

a2,0 = −α2a1,0 − β1.

Summing these relations and using (2.20) yields

N−1 N−1 X X aN,N−2 = −µ1 ak,k−1 − βk. k=1 k=1 20 Taking into account that ak,k−1 = −kµ1 for k = 1, 2, 3,... , cf. (2.21), gives another relation for coeﬃcients aN,N−2,

N−1 N−1 N−1 X X N(N − 1) X a = µ2 k − β = µ2 − β . N,N−2 1 k 1 2 k k=1 k=1 k=1 It now follows from (2.22) that

N−1 X N β = (µ − µ2). k 2 2 1 k=1 Letting N = 2, 3, 4,... in the above sum, we obtain

β 1 1 = β = β = ... = β = (µ − µ2). 2 2 3 N 2 2 1

To simplify the notation, let σ2 1 := (µ − µ2). 4 2 2 1 Then σ2 σ2 β = , β = β = ... = β = . 1 2 2 3 N 4

Substituting these values of βk and the values (2.20) of the αk into (2.19), we obtain the

recurrent relations

p0(x) = 1, p1(x) = x − µ1, (2.23) σ2 pk(x) = (x − µ1)pk−1(x) − 2 pk−2(x), k = 2, 3,...,N + 1.

Let Tbk denote the monic Chebyshev polynomial of the ﬁrst kind of degree k associated

with the measure (2.7). Comparing the recursion relation (2.23) with that for the Tbk shows that

x − µ (2.24) p (x) = σkT 1 , k = 0, 1, 2,.... k bk σ

It follows from (2.24) that the zeros of the polynomial pN (x) are

2k − 1 x(N) = µ + σ cos π , k = 1, 2,...,N. k 1 2N 21 They lie in the interval [−σ + µ1, σ + µ1]. By [49, Theorem 6.1.1] the zeros of any family of

orthogonal polynomials are dense on the support of the measure dλ, which by assumption

is [−1, 1]. That means [−σ + µ1, σ + µ1] = [−1, 1]. This implies that µ1 = 0 and σ = 1.

Hence, dλ(θ) is 1/π times the measure (2.7).

Theorem 2.3.3 Let p0, p1, . . . , pn−1 be a family of orthonormal polynomials with respect

to an inner product determined by a positive (real) measure dλ with support on the real

(N) (N) (N) axis, i.e., the polynomials satisfy (2.2). Let x1 , x2 , . . . , xN denote the nodes of an N-point Gauss quadrature rule associated with the measure dλ. Consider the real N × n

Vandermonde-like matrix VN,n determined by the polynomials pj, 0 ≤ j < n, and nodes

(N) xk = xk , 1 ≤ k ≤ N. Let n ≥ 3, then the matrix VN,n is optimally conditioned for all N ≥ n if and only if dλ(θ) is a Chebyshev measure of the ﬁrst kind.

Proof: Suppose that the matrix VN,n of the form (1.12) with n ≥ 3 is optimally conditioned for any N. The columns of VN,n then are orthogonal and of the same length. In other words,

N X (2.25) pi(xk)pj(xk) = 0, 0 ≤ i, j < n, k=1 and there exists a constant cN such that N X 2 cN = pi (xk), 0 ≤ i < n. k=1

Any polynomial l ∈ P2 can be presented in the form

l(x) = d2p2(x) + d1p1(x) + d0p0(x)

R 1/2 for certain scalar coeﬃcients dk. Let µ0 := dλ(x) > 0. Then p0(x) ≡ 1/µ0 . Due to the

orthogonality of the polynomials p0, p1, p2 and relation (2.25), we have Z 1/2 l(x)dλ(x) = d0(p0, 1)λ = d0µ0

22 and N N X X N l(xk) = d0 p0(xk) = d0 1/2 . k=1 k=1 µ0

This shows that (2.14) holds for the constant νN = µ0/N. It follow from Theorem 2.3.2 that dλ is a scaled and possibly translated Chebyshev measure (2.7).

The following result shows that if the Vandermonde-like matrix VN,n has suﬃciently

many more rows than columns, then suitable row scaling of VN,n will render a matrix with orthogonal columns.

Lemma 2.3.4 Let the rectangular Vandermonde-like matrix VN,n be deﬁned by a family

of orthonormal polynomials p0, p1, . . . , pn−1 with respect to a nonnegative measure dλ, i.e.,

the polynomials satisfy (2.2). Let x1, x2, . . . , xN be real distinct nodes and assume that

N−1 n < 2 . Then, generally, the columns of VN,n can be made orthonormal by diagonal scaling.

Proof: Given arbitrary real distinct nodes x1, x2, . . . , xN , one can determine real weights

λ1, λ2, . . . , λN such that Z (2.26) p(x)dλ(x) = p(x1)λ1 + p(x2)λ2 + ... + p(xN )λN ∀p ∈ PN−1; see, e.g., [49, Theorem 3.4.1]. The right-hand side of (2.26) is known as an interpolatory quadrature rule. We will assume that all weights λk are nonvanishing. This is the generic

N−1 situation. Since n < 2 , we have pipj ∈ PN−1 for all 0 ≤ i, j < n. Therefore, N Z X δij = pi(x)pj(x)dλ(x) = pi(xk)pj(xk)λk, k=1 1/2 1/2 1/2 where δij is the Kronecker delta. Let Λ = diag[λ1 , λ2 , . . . , λN ]. Then the matrix ΛVN,n

has orthogonal columns of Euclidean norm one. Note that if a weight λj is negative, then

the corresponding diagonal entry of Λ is purely imaginary. 23 2.4 Szeg˝o-Vandermonde matrices

In this section we consider polynomials that are orthogonal with respect to an inner

product on the unit circle in C of the form

Z π √ (2.27) (f, g)λ := f(z)g(z)dλ(θ), z = exp(iθ), i = −1. −π

The bar denotes complex conjugation, dλ is a nonnegative measure with inﬁnitely many points of increase, and the functions f and g are polynomials. Let p0, p1, p2,... be a family

of polynomials that are orthonormal with respect to the inner product (2.27), i.e.,   1, j = k, (pj, pk)λ =  0, j 6= k,

with deg(pj) = j for all j ≥ 0. Polynomials that are orthogonal with respect to the

inner product (2.27) are commonly referred to as Szeg˝opolynomials. They have many

applications in signal processing and frequency analysis; see, e.g., [29, 45]. Integrals of the

form Z π If := f(exp(iθ))dλ(θ) −π can be approximated by the quadrature formula

N X (N) (N) (2.28) SN f := λk f(zk ), k=1

(N) (N) where the λk > 0 are weights and the zk are distinct nodes on the unit circle in C. The quadrature rule (2.28) is said to be an N-point Gauss–Szeg˝orule if

(2.29) If = SN f ∀f ∈ Λ−N+1,N−1,

where

−1 2 −2 N−1 −N+1 Λ−N+1,N−1 := span{1, z, z , z , z , . . . , z , z }.

24 Existence of Gauss–Szeg˝oquadrature rules is shown in, e.g., [33, Theorem 7.1]. The weights

of Gauss–Szeg˝orules are unique and the nodes are unique up to a rotation on the unit

(N) (N) (N) circle, i.e., z1 can be chosen arbitrarily on the unit circle. The other nodes, z2 , . . . , zN , are then uniquely determined. Let (2.28) be an N-point Gauss–Szeg˝orule. The Christoﬀel

function associated with the measure dλ is given by

N−1 !−1 X 2 (2.30) ΛN (z) := |pk(z)| ; k=0 see [38]. This expression is analogous to (2.4). Similarly to (2.5), the weights of the N-point

Gauss–Szeg˝orule (2.28) are given by

(N) (N) λk = ΛN (zk ), k = 1, 2,...,N.

(N) The nodes zk , k = 1, 2,...,N, of the Gauss–Szeg˝orule (2.28) are zeros of a so-called “para-orthogonal” polynomial,

∗ (2.31) BN (z, wN ) := pN (z) + wN pN (z),

that satisﬁes the orthogonality relations

Z π k (2.32) z BN (z)dλ(θ) = 0, z = exp(iθ), 1 ≤ k ≤ N − 1, −π

∗ where the parameter wN ∈ C can be chosen arbitrarily such that |wN |= 1, and pN (z) :=

N z pN (1/z) denotes the reversed polynomial associated with pN ; see Gonz´aleset al. [25] or

Jones [33] for details. Note that k > 0 in (2.32). A diﬀerent approach to deﬁne the nodes

(N) zk is described by Gragg [26]. Eﬃcient algorithms for computing the nodes and weights of Gauss-Szeg˝orules are described in [1, 27, 28, 30, 47].

We show a few properties of Vandermonde-like matrices deﬁned by Szeg˝opolynomials.

These properties are analogous to those of the Vandermonde-like considered in Sections

4.2 and 2.3.

25 Lemma 2.4.1 Let p0, p1, . . . , pN−1 be Szeg˝opolynomials associated with a nonnegative

(N) (N) (N) measure dλ on the unit circle. In particular, deg(pj) = j for all j. Let z1 , z2 , . . . , zN be nodes of the N-point Gauss–Szeg˝orule (2.28). Then the Szeg˝o–Vandermonde matrix

 (N) (N) (N)  p0(z1 ) p1(z1 ) ··· pN−1(z1 )    (N) (N) (N)   p0(z2 ) p1(z2 ) ··· pN−1(z2 )  V =   N,N  . . .   . . .   . . ··· .    (N) (N) (N) p0(zN ) p1(zN ) ··· pN−1(zN ) can be row-scaled to become optimally conditioned.

Proof: The existence of a Gauss-Szeg˝orule (2.28) implies discrete orthogonality of the

N−1 Szeg˝opolynomials. We will use this to show the lemma. Clearly, pj(z) ∈ span{1, z, . . . , z }

for 0 ≤ j < N. For node z on the unit circle, we havez ¯ = z−1, and, therefore,

−1 −N+1 pj(z) ∈ span{1, z , . . . , z } for 0 ≤ j < N. It now follows from (2.28) and (2.29) that  N Z π  X (N) (N) (N)  1, i = j, λk pi(zk )pj(zk ) = pi(z)pj(z)dλ(θ) = k=1 −π  0, i 6= j. Let q q q (N) (N) (N) D = diag λ1 , λ2 ,..., λN .

Then the matrix Q = DVN,N is unitary and, consequently, optimally conditioned.

Theorem 2.4.1 Let z1, z2, . . . , zN be distinct complex numbers, and let p0, p1, . . . , pN−1 be a family of orthonormal polynomials with respect to the inner product (2.27). Then

N N !1/2 X X −1 κF (VN,N ) ≥ ΛN (zi) · (ΛN (zi)) , i=1 i=1 where the matrix VN,N is determined by the values of the polynomials pj at the nodes zk, and ΛN is the Christoﬀel function (2.30) associated with the measure dλ.

26 Proof: Consider the Lagrange polynomials

N Y z − zi (2.33) lk(z) = , k = 1, 2,...,N, zk − zi i=1 i6=k which form a basis for PN−1. The polynomials p0, p1, . . . , pN−1 are linearly independent.

Therefore, the Lagrange polynomials can be expressed as linear combinations of the pj,

      a11 a12 ··· a1N p0(z) l1(z)          .   .   a21 a22 ··· a2N   .   .  (2.34)     =   .  . . .       . . .       . . ··· .            aN1 aN2 ··· aNN pN−1(z) lN (z)

Similarly, the polynomials pk can be expressed as linear combinations of Lagrange polyno-

mials,

      p0(z1) p0(z2) ··· p0(zN ) l1(z) p0(z)          .   .   p1(z1) p1(z2) ··· p1(zN )   .   .  (2.35)     =   .  . . .       . . .       . . ··· .            pN−1(z1) pN−1(z2) ··· pN−1(z1) lN (z) pN−1(z)

−1 N It follows from (2.34) and (2.35) that VN,N = [aij]i,j=1. Consequently,

Z π N Z π N N N ! X 2 X X X |li(z)| dλ(θ) = aijpj−1(z) aikpk−1(z) dλ(θ) −π i=1 −π i=1 j=1 k=1 N N N X X X Z π = aijaik pj−1(z)pk−1(z)dλ(θ) i=1 j=1 k=1 −π N N X X 2 = |a |2= V −1 . ij N,N F i=1 j=1

27 Nevai [38] shows that the Christoﬀel function associated with a nonnegative measure dλ on the unit circle satisﬁes

Z π 2 (2.36) ΛN (z) = min |p(ζ)| dλ(θ), ζ = exp(iθ). p∈PN−1 −π p(z)=1

The Lagrange polynomials (2.33) satisfy li ∈ PN−1 and li(zi) = 1. It follows from (2.36) that Z π Z π 2 2 |li(ζ)| dλ(θ) ≥ min |p(ζ)| dλ(θ) = ΛN (zi), ζ = exp(iθ). −π p∈PN−1 −π p(zi)=1 Therefore,

Z π N N 2 X 2 X V −1 = |l (z)| dλ(θ) ≥ Λ (z ), z = exp(iθ). N,N F i N i −π i=1 i=1 In view of (2.30), we obtain

N 2 X −1 kVN,N kF = (ΛN (zi)) . i=1 Hence, N N 2 −1 2 X X −1 kVN,N kF kVN,N kF ≥ ΛN (zi) · (ΛN (zi)) , i=1 i=1 and the theorem follows. A special type of para-orthogonal polynomials (2.31), sometimes referred to as Delsarte-

Genin para-orthogonal polynomials, satisfy the recurrence relations

(1) ∗ (2) ∗ (2.37) RN (z) = zpN−1(z) + pN (z), (z − 1)RN (z) = zpN (z) − pN (z),

for N = 0, 1, 2,... , with p−1(z) ≡ 0. These polynomials were ﬁrst considered in Delsarte and Genin [11]. They are associated with symmetric or skew-symmetric measures on the unit circle in C, and are important because they relate Szeg˝opolynomials to orthogonal polynomials on the interval [−1, 1]; see [11, 12, 54]. Applications include frequency analysis;

28 see [7]. The following result for the polynomials (2.37) is analogous to Posse’s theorem for

polynomials that are orthogonal on a real interval.

(1) ∞ (2) ∞ Theorem 2.4.2 Let {RN }N=0 and {RN }N=0 be families of para-orthogonal polynomials (2.37) associated with a symmetric nonnegative measure dλ on the unit circle with inﬁnitely

many points of support. If for all N = 2, 3,... , there is an N-node Gauss-Szeg˝oquadrature

(1) (2) rule, whose nodes are the zeros of RN or RN with all weights equal, then dλ(θ) is a multiple of dθ. When dλ is a skew-symmetric nonnegative measure on the unit circle, there is no such Gauss-Szeg˝oquadrature rule.

Proof: Assume that for every N ≥ 2 there is a real scalar νN such that

N Z π m m X (N) (2.38) µm := z dλ(θ) = νN zk , m = 0, 1, 2,..., −π k=1

(N) (N) (N) (1) where z = exp(iθ) and the nodes z1 , z2 , . . . , zN are the zeros of the polynomials RN (2) R π or RN . We may scale the measure dλ so that µ0 = −π dλ(θ) = 1. Setting m = 0 gives

N Z π X 1 = dλ(θ) = νN 1 = νN N, −π k=1

1 i.e., νN = N . Turning to m = 1, we obtain

Z π N 1 X (N) µ = zdλ(θ) = z . 1 N k −π k=1

(1) Delsarte and Genin [11] showed that both families of para-orthogonal polynomials RN and

(2) RN , deﬁned by (2.37) and associated with a symmetric or skew-symmetric measure dλ, satisfy a three-term recurrence relation of the form

RN (z) = (z + 1)RN−1(z) − 4dN zRN−2(z),N = 2, 3,..., (2.39)

R0(z) = 1,R1(z) = z + 1;

29 see also [8] for more details. Consider the coeﬃcient aN,N−1 of the polynomial

N N−1 N−2 (2.40) RN (z) = z + aN,N−1z + aN,N−2z + ... + aN,0.

We obtain from Vieta’s formula that

N X (N) (2.41) aN,N−1 = − zk = −Nµ1,N ≥ 2. k=1

Comparing the coeﬃcients aN,N−1 in the left-hand sides of (2.39) and (2.40) for decreasing degrees N, we obtain

aN,N−1 = aN−1,N−2 + (1 − 4dN ),

aN−1,N−2 = aN−2,N−3 + (1 − 4dN−1),

(2.42) ...

a2,1 = a1,0 + (1 − 4d2),

a1,0 = 1.

The relations (2.41) and (2.42) give

a2,1 = 2 − 4d2 = −2µ1.

1 Hence, d2 = 2 (1 + µ1). For N ≥ 3, we obtain similarly that

(2.43) aN,N−1 = −(N − 1)µ1 + (1 + 4dN ) = −Nµ1

1 and, therefore, dN = 4 (1 + µ1). It follows that d 1 (2.44) 2 = d = ... = d = (1 + µ ). 2 3 N 4 1

Bracciali et al. [8] show that the coeﬃcients d2, d3, . . . , dN are positive. Hence, µ1 > −1.

Turning to the coeﬃcient aN,N−2 in (2.40), we derive from Vieta’s formula that

X (N) (N) aN,N−2 = zi zj . 1≤i

30 It now follows from Z π 2 1 (N) 2 (N) 2 µ2 = z dλ(θ) = ((z1 ) + ... + (zN ) ) −π N that, for N ≥ 3,

1 (N) (N) (N) (N) a = (z + ... + z )2 − ((z )2 + ... + (z )2) N,N−2 2 1 N 1 N (2.45) N = (Nµ2 − µ ). 2 1 2

Comparing the coeﬃcients aN,N−2 on the left-hand sides of (2.39) and (2.40), we get

(2.46) aN,N−2 = aN−1,N−3 + aN−1,N−2 − (1 + µ1)aN−2,N−3.

Using the relation (2.43) twice (with N replaced by N − 1 and N − 2) and (2.45) (with N

replaced by N − 1) in (2.46) gives

2 (2.47) µ2 = 3µ1 + 2µ1.

Now consider the coeﬃcient aN,N−3. We will derive an expression for the third moment Z π 3 1 (N) 3 (N) 3 µ3 = z dλ(θ) = (z1 ) + ... + (zN ) , z = exp(iθ), −π N

in terms of µ1. Vieta’s formula applied to (2.40) yields

X (N) (N) (N) (2.48) aN,N−3 = − zi zj zk . 1≤i

For any set of N complex numbers z1, z2, . . . , zN , the following identity is fulﬁlled

 N !3 N ! N ! N ! X 1 X X X X z z z = z − 3 z2 z + 2 z3 . i j k 6  i i i i  1≤i

(N) (N) (N) Application of this relation to the nodes z1 , z2 , . . . , zN in (2.48), and (2.38), gives relation

1 (2.49) a = − (N 3µ3 − 3N 2µ µ + 2Nµ ). N,N−3 6 1 2 1 3 31 Comparing the coeﬃcients aN,N−3 in the left-hand sides of (2.39) and (2.40) yields

aN,N−3 = aN−1,N−4 + aN−1,N−3 − (1 + µ1)aN−2,N−4,N ≥ 4, and using the relations (2.45), (2.47), and (2.49) ﬁnally gives formula

3 2 (2.50) µ3 = 10µ1 + 12µ1 + 3µ1.

From (2.32) the para-orthogonal polynomial R2 satisﬁes

Z π zR2(z)dλ(θ) = 0; −π 1 see, e.g., [33]. From combining recurrence relation (2.39) and d = (1 + µ ) it follows 2 2 1 2 R2(z) = z − 2µ1z + 1. Hence,

Z π 2 z(z − 2µ1z + 1)dλ(θ) = µ3 − 2µ1µ2 + µ1 = 0. −π

Using (2.47) and (2.50) yields

3 2 2 4µ1 + 8µ1 + 4µ1 = 4µ1(µ1 + 1) = 0.

Since µ1 > −1, µ1 can only be 0. Therefore, (2.44) simpliﬁes to

d 1 2 = d = ... = d = . 2 3 N 4

N It follows from the recursion formula (2.39) that RN (z) = z + 1 for N ≥ 0. The zeros of

RN are given by 2k − 1 z(N) = exp i π , k = 1,...,N. k N Thus, they are equidistant on the unit circle. These nodes deﬁne an N-node Gauss-Szeg˝o quadrature rule. This rule is exact for all functions in Λ−N+1,N+1; cf. (2.29). Therefore,

32 by (2.38),

N Z π j j X (N) µj = z dλ(θ) = νN zk −π k=1 N j = ±1, ±2,..., ±(N − 1). X 2k − 1 = ν exp ij π = 0, N N k=1 so, all moments except for µ0 vanish. When dλ(θ) is a symmetric measure, this only leaves 1 one possible solution for dλ(θ), namely dλ(θ) = dθ, because µ is assumed to be one. 2π 0

If, instead, dλ(θ) is a skew-symmetric measure, then also µ0 vanish. This contradicts the assumption that µ0 = 1. 1 The monic Szeg˝opolynomials associated with the measure dλ(θ) = dθ are 2π

N pN (z) = z ,N = 0, 1, 2,....

The nodes of an N-point Gauss-Szeg˝orule associated with this measure are equidistant on

the unit circle. They are given by

2π (2.51) z = exp(ikθ ), θ = θ + k, k = 1, 2,...,N, k k k 0 N

where θ0 ∈ R is arbitrary. Consider the Vandermonde matrix   N−1 1 z1 ··· z1    N−1   1 z2 ··· z2  (2.52) V =   N,N  . . .   . . .   . . ··· .    N−1 1 zN ··· zN deﬁned by the nodes (2.51). It is easy to show that

∗ VN,N VN,N = NI.

1/2 Hence, all singular values of the matrix VN,N equal N . It follows that κ2(VN,N ) = 1 and

κF (VN,N ) = N. Thus, the matrix (2.52) is optimally conditioned. We will show that this

also holds for the corresponding rectangular Vandermonde matrices.

33 Lemma 2.4.2 Rectangular n × N Vandermonde matrices   N−1 1 z1 ··· z1    N−1   1 z2 ··· z2  (2.53) V =   n,N  . . .   . . .   . . ··· .    N−1 1 zn ··· zn with nodes 2π z = exp(ikθ ), θ = θ + (k − 1), k = 1, 2, . . . , n, k k k 0 M

where M = max{n, N} and θ0 ∈ R is arbitrary, are optimally conditioned. Moreover, √ √ r † 1 † m (2.54) kVn,N k2= M, kV k2= √ , kVn,N kF = Mm, kV kF = , n,N M n,N M where m = min{n, N}.

Proof: The result follows by applying [32, Theorem 7.3.3] to matrix VM,M in the same

way as in the proof of Theorem 2.3.1, and by observing that for all n, N, singular values √ m {σi}i=1 of Vn,N are all equal to M, thus √ † 1 1 kVn,N k2= σ1 = M, kVn,N k2= = √ , σ1 M √ r p † q m kV k = σ2 + ... + σ2 = mM, kV k = σ−2 + ... + σ−2 = n,N F 1 m n,N F 1 m M The above lemma is in agreement with the observation by Baz´an [2] that matrices of the form (2.53), with N ≥ n, are fairly well conditioned when the nodes zj are close to the unit circle and no pair of distinct nodes are very close to each other.

Theorem 2.4.3 For any N and n, N ≥ n, matrices of the form (2.53) can be row-scaled to be optimally conditioned if and only if the arguments θk of the nodes zk = ρk exp(iθk) 2π are of the form θ = θ + j for 1 ≤ k ≤ n, where the j are integers such that the k 0 n k k

nodes z1, z2, . . . , zn are distinct.

34 Proof: Consider the row-scaling DVn,N of the matrix (2.53), where

D = diag[d1, d2, . . . , dn], dk > 0 ∀k.

It follows from Lemma 2.3.2 that if the matrix DVn,N is optimally conditioned, then its rows are orthogonal and have all the same Euclidean norm. Let the nodes of the matrix

(2.53) be of the form zk = ρk exp(iθk), k = 1, 2, . . . , n, where θk ∈ R. Assume that the rows of DVn,N are orthogonal. This implies that the nodes z1, z2, . . . , zn are distinct. The orthogonality of consecutive rows of the matrix DVn,N yields

2 2 N−1 N−1 dkdk+1 + dkdk+1zkzk+1 + dkdk+1zkzk+1 + ... + dkdk+1zk zk+1 = 0, k = 1, 2, . . . , n − 1.

Let φk := θk+1 − θk. The above equations can be written as

2 2 N−1 N−1 (2.55) 1 + ρkρk+1 exp(iφk) + ρkρk+1 exp(i2φk) + ... + ρk ρk+1 exp(i(N − 1)φk) = 0 for k = 1, 2, . . . , n − 1. The equations (2.55) imply that

N N ρk ρk+1 exp(iNφk) = 1, k = 1, 2, . . . , n − 1.

2π It follows that φk = N jk for k = 1, 2, . . . , n − 1, where the jk are integers such that the nodes z1, z2, . . . , zn are distinct. We choose the scaling factors dj so that all rows of DVn,N have the same norm. The matrix DVn,N then satisﬁes the conditions of Lemma 2.3.2 and therefore is optimally conditioned. r 1 The discrete Fourier transform (DFT) matrix can be expressed in the form V . N N,N

Denote the nodes by z1, z2, . . . , zN . Theorem 2.4.3 shows that the DFT matrix is orthogonal if and only if the nodes are equidistant on the unit circle.

2.5 General Vandermonde-type matrices

This section considers Vandermonde matrices, whose nodes are zeros of Chebyshev orthogonal polynomials on an interval or on the unit circle. Eisinberg et al. [13] showed that

35 N×n the spectral condition number κ2(Vn,N ) of a rectangular Vandermonde matrix Vn,N ∈ R , with N ≥ n and Chebyshev nodes, does not depend on the number of nodes N. We extend

this result to Vandermonde-type matrices of the form (1.12), where the polynomials pi are

not required to be orthogonal and deg(pj) may be diﬀerent from j.

n−1 Theorem 2.5.1 Let {pi}i=0 be a set of linearly independent polynomials in Pm and let

n−1 {li}i=0 be an arbitrary set of polynomials in Pm. Hence, m ≥ n − 1. The polynomials {li} may be linearly dependent. Let the nodes x1, . . . , xN be zeros of the Chebyshev polynomial

TN (x) where N ≥ n and deﬁne the Vandermonde-type matrices

    p0(x1) ··· pn−1(x1) l0(x1) ··· ln−1(x1)          p0(x2) ··· pn−1(x2)   l0(x2) ··· ln−1(x2)  (2.56) P =   ,L =   . N,n  . . .  N,n  . . .   . . .   . . .   . . .   . . .      p0(xN ) ··· pn−1(xN ) l0(xN ) ··· ln−1(xN )

6 There are constants {dj}j=1 that can be chosen independently of N, such that r N r π kP k = d , kP † k = d , N,n F π 1 N,n F N 2 r N r π kP k = d , kP † k = d , N,n 2 π 3 N,n 2 N 4 r r N N kL k = d , kL k = d N,n F π 5 N,n 2 π 6 for all N ≥ m.

Proof: The polynomials pi and li, 0 ≤ i < n, can be expressed as

m m X ˜ X ˜ pi(x) = ci,kTk(x), li(x) = cˆi,kTk(x), k=0 k=0

36 ˜ where the Tk denote normalized Chebyshev polynomials. We express the matrices (2.56) in factored form, ˆ PN,n(x) = TN,mC,LN,n = TN,mC,

where   ˜ ˜ T0(x1) ··· Tm(x1)      ·········    (2.57) TN,m =      ·········    ˜ ˜ T0(xN ) ··· Tm(xN ) and     c1,0 ··· c1,n−1 cˆ1,0 ··· cˆ1,n−1          ·········   ·········    ˆ   C =   , C =   .      ·········   ·········      cm,0 ··· cm,n−1 cˆm,0 ··· cˆm,n−1

Since PN,n is of full rank, so are the matrices TN,m and C. It follows from Theorem 2.3.1

q N and Lemma 2.3.2 that TN,m(x) = π U, where the matrix U has orthonormal columns. We obtain r N kP k = kT Ck = kCk , N,n F N,m F π F r π kP † k = kC†T † k = kC†k , N,n F N,m F N F r π kL k = kT Cˆk = kCˆk . N,n F N,m F N F

Similarly, r N r π r π kP k = kCk , kP † k = kC†k , kL k = kCˆk . N,n 2 π 2 N,n 2 N 2 N,n 2 N 2

Letting

† † ˆ ˆ d1 = kCkF , d2 = kCk2, d3 = kC kF , d4 = kC k2, d5 = kCkF , d6 = kCk2

37 n N Chebyshev nodes optimal nodes 5 5 6.1 · 100 5.2 · 100 10 10 1.3 · 101 1.1 · 101 20 20 2.9 · 101 2.3 · 101 35 35 5.4 · 101 4.3 · 101 40 40 6.2 · 101 5.0 · 101 5 10 6.1 · 100 5.0 · 100 10 20 1.3 · 101 1.0 · 101 20 40 2.9 · 101 2.2 · 101 35 70 5.4 · 101 4.1 · 101 40 80 6.2 · 101 4.7 · 101

Table 2.1: Example 5.1. Frobenius condition numbers of Vandermonde-like matrices VN,n deﬁned by Legendre polynomials and Chebyshev or optimal nodes. concludes the proof.

If deg(pi) = i for 0 ≤ i < n, then PN,n = TN,nC for some nonsingular upper triangular

n×n matrix C ∈ R . This means that the QR factorization of PN,n is explicitly known. The following result follows directly from Theorem 2.5.1,

Corollary 2.5.1 Let the matrix PN,n be given by (2.56). Then the condition numbers

κF (PN,n) and κ2(PN,n) can be bounded independently of N.

Let for the moment the nodes x1, x2, . . . , xN in the matrix (2.57) be distinct, but otherwise arbitrary, and let C ∈ Rm×n. Then

† † † † (2.58) kTN,mCkF kC TN,mkF ≤ kCkF kC kF kTN,mk2kTN,mk2,

† † † † (2.59) kTN,mCk2kC TN,mk2 ≤ kCk2kC k2kTN,mk2kTN,mk2.

N Thus, letting {xj}j=1 be Chebyshev nodes minimizes upper bounds in (2.58) and (2.59). Example 5.1. We compare numerically condition numbers of Vandermonde-like and

Vandermonde matrices determined by Chebyshev nodes and optimal nodes. The optimal nodes, i.e., the nodes that minimize the condition number κF (VN,n), are determined with

Wolfram Mathematica Software using the minimizer NMinimize. We cannot be certain that

38 n N Chebyshev nodes optimal nodes 5 5 8.2 · 100 5.5 · 100 10 10 2.3 · 101 1.3 · 101 20 20 6.4 · 101 3.2 · 101 35 35 1.5 · 102 7.0 · 101 40 40 1.8 · 102 8.5 · 101 5 10 8.2 · 100 5.0 · 100 10 20 2.3 · 101 1.1 · 101 20 40 6.4 · 101 2.8 · 101 35 70 1.5 · 102 6.0 · 101 40 80 1.8 · 102 7.3 · 101

Table 2.2: Example 5.1. Frobenius condition numbers of Vandermonde-like matrices VN,n deﬁned by Chebyshev polynomials of the second kind and Chebyshev or optimal nodes.

n N Chebyshev nodes optimal nodes 5 5 2.8 · 101 2.3 · 101 10 10 2.3 · 103 1.6 · 103 20 20 1.5 · 107 9.9 · 106 35 35 8.3 · 1012 5.1 · 1012 40 40 3.3 · 1013 2.6 · 1013 5 10 2.8 · 101 1.7 · 101 10 20 2.3 · 103 1.2 · 103 20 40 1.5 · 107 7.5 · 106 35 70 8.3 · 1012 4.0 · 1012 40 80 3.3 · 1013 1.6 · 1013

Table 2.3: Example 5.1. Frobenius condition numbers of Vandermonde matrices VN,n with Chebyshev and optimal nodes.

39 the nodes so computed indeed are optimal, but the minimization function NMinimize gave the same nodes for many diﬀerent initial node choices. Computed condition numbers for

Vandermonde-like matrices deﬁned by Legendre polynomials and Chebyshev polynomials of the second kind for Chebyshev and optimal nodes are shown in Table 2.1 and Table 2.2, respectively. Corresponding results for (standard) Vandermonde matrices are displayed in Table 2.3. As it can be expected, the matrices of the latter table have much larger condition numbers than the matrices of the former tables. Chebyshev nodes can be seen to be “near-optimal” in the sense that the condition numbers for the Chebyshev nodes for all examples are larger by only a fairly small factor than the condition numbers for optimal nodes for all combinations of n and N reported.

Numerical experiments suggest that the nodes that minimize the condition number

κF (VN,n) are unique. A similar observation for square Vandermonde-like matrices is reported by Gautschi [23]. In fact, our experiments suggest that κF (VN,n) is a locally convex

function of the nodes in a neighborhood of the optimal nodes. We remark that it is not

hard to show, that for the special case when " # c0 c1 + c2x1 V2,2(x1, x2) = , c0 c1 + c2x2

the condition number κF (V2,2(x1, x2)) is strongly convex in the whole region −∞ < x1 <

2 x2 < ∞. This follows from the fact that the Hessian of κF (V2,2(x1, x2)) is positive deﬁnite. 2

The following result is an analogue of Theorem 2.5.1 for polynomials with nodes on the

unit circle.

n−1 Theorem 2.5.2 Let {pi}i=0 be a set of linearly independent polynomials of the form

k1 k1+1 k1+m pi(z) = ci,1z + ci,2z + ... + ci,mz

40 n−1 with m ≥ n − 1 and let k1 ≥ 0 be an arbitrary integer. Further, let {li}i=0 be a set of arbitrary polynomials of the form

k2 k2+1 k2+m li(z) =c î,1z +c î,2z + ... +c î,mz

with k2 ≥ 0 an arbitrary integer. Let the nodes z1, z2, . . . , zN be equidistant on the unit

2πk circle, i.e.,. zk = exp(ikθk), where θk = θ0 + N , where θ0 ∈ R is arbitrary. Consider the Vandermonde-like matrices

    p0(z1) ··· pn−1(z1) l0(z1) ··· ln−1(z1)          p0(z2) ··· pn−1(z2)   l0(z2) ··· ln−1(z2)      PN,n =   ,LN,n =   .      ·········   ·········      p0(zN ) ··· pn−1(zN ) l0(zN ) ··· ln−1(zN )

6 There are constants {dj}j=1, that can be chosen independently of N, such that

√ † d kP k = Nd , kP k = √2 , N,n F 1 N,n F N √ † d kP k = Nd , kP k = √4 , N,n 2 3 N,n 2 N √ √ kLN,nkF = Nd5, kLN,nk2 = Nd6

for all N ≥ m.

Proof: We have the factorizations

ˆ PN,n = D1ZC,LN,n = D2ZC,

where

k1 k1 k1 k2 k2 k2 D1 = diag[z1 , z2 , . . . , zN ],D2 = diag[z1 , z2 , . . . , zN ],

41 and   m 1 z1 ··· z1    m   1 z2 ··· z   2  Z =   .    ············    m 1 zN ··· zN

Moreover,     c1,0 ··· c1,n−1 cˆ1,0 ··· cˆ1,n−1          ·········   ·········    ˆ   C =   , C =   .      ·········   ·········      cm,0 ··· cm,n−1 cˆm,0 ··· cˆm,n−1

The matrices D1 and D2 are unitary. Using the bounds (2.54), we obtain √ ˆ ˆ ˆ kLN,nkF = kD2ZCkF = kZk2kCkF = NkCkF , 1 † † † † † † † kP kF = kC Z D kF = kZ k2kC kF = √ kC kF . N,n 1 N

The remaining bounds follow similarly.

2.6 Chapter summary

Gautschi investigated the conditioning of square Vandermonde-like matrices determined by orthogonal polynomials with respect to an inner product deﬁned by a measure with support on the real axis. This chapter extends his results to rectangular Vandermonde-like matrices with real nodes as well as to Vandermonde-like matrices with nodes on the unit circle in the complex plane, called Szeg˝o-Vandermonde matrices. In this chapter we also generalize classical Posse theorem for Gauss and Gauss–Szeg˝oquadrature rules.

42 CHAPTER 3

Fast factorization of rectangular Vandermonde matrices with Chebyshev

nodes

3.1 Introduction

In this chapter we consider QR factorization of rectangular Vandermonde matrices   n−1 1 x1 ··· x1    n−1   1 x2 ··· x2  (3.1) V =   ∈ RN×n,N ≥ n,  . . .   . . .   . . ··· .    n−1 1 xN ··· xN whose nodes xi are the zeros of a Chebyshev polynomial of the ﬁrst kind of degree N for the interval [−1, 1],

2i − 1 (3.2) x = cos π , i = 1, 2,...,N. i 2N

We also will discuss the situation when the nodes are extreme points (also called Chebyshev points of the second kind) of a Chebyshev polynomial of the ﬁrst kind of degree N − 1.

These nodes are given by

i − 1 (3.3) x = cos π , i = 1, 2,...,N. i N − 1

Rectangular Vandermonde matrices arise in the polynomial least squares approximation

problem

N n−1 X 2 X j T (3.4) min (p(c, xi) − yi) , p(c, x) = cjx , c = [c0, c1, . . . , cn−1] . c∈Rn i=1 j=0

43 T where the vector y = [y1, y2, . . . , yN ] represents data that is to be ﬁtted in the least squares

sense. The minimization problem (3.4) can be written in the form

(3.5) min kV c − yk2 . c∈Rn

The use of the power form representation of the least square polynomial p is convenient

when the polynomial is to be diﬀerentiated or integrated.

Although Vandermonde matrices with real nodes are known to be generally quite ill-

conditioned, the use of Chebyshev nodes (3.2) reduces this somewhat.

Gautschi [23] showed that for the Frobenius matrix norm, the condition number of a

square Vandermonde matrix V ∈ Rn×n, with real nodes allocated to minimize the condition √ number, grows exponentially at a rate slightly less than (1 + 2)n. When the matrix norm

is induced by the uniform vector norm, that for an arbitrary vector u is deﬁned as

(3.6) kuk∞= max{|ui|} i and the nodes are given by (3.2) with N = n, Gautschi [17] proved that the condition 3 √ 3 4 n number for square Vandermonde matrices to be asymptotically 4 (1 + 2) as n → ∞. This result indicates that it is beneﬁcial to choose Chebyshev nodes for Vandermonde

matrices.

Moreover, it is known that the approximation of functions on the interval [−1, 1] by

polynomial interpolation at Chebyshev nodes gives near-optimal approximants in the sense

that the polynomial interpolant is close to the best polynomial approximant of the same

degree in the uniform norm. An excellent interpolating properties of nodes (3.2) and (3.3)

are supported by classical theorems below.

Theorem 3.1.1 (Optimal node placement in Lagrange interpolation) If x0, x1, ..., xn are

44 distinct points in the interval [−1, 1] and f ∈ Cn+1[−1, 1], and P (x) the nth degree inter-

polating Lagrange polynomial, then ∀x ∈ [−1, 1], ∃ξ(x) ∈ (−1, 1) so that

n n f (n+1)(ξ(x)) Y f (n+1)(ξ(x)) Y f(x) − P (x) = (x − xk) ≤ max max (x − xk) , (n + 1)! ξ (n + 1)! x k=0 k=0

n Y and Chebyshev zeros (3.2) minimize max (x − xk) . x k=0 See [48, Ch.2] for more details.

Theorem 3.1.2 (Convergence of interpolants at Chebyshev points)

(v) For any n > 0, let pn(x) denote the interpolant to function f(x) ∈ C [−1, 1] at the nodes

(3.3) having variation V (v) = max f (v)(x) − min f (v)(x) Then for any n > v x∈[−1,1] x∈[−1,1]

V (v) (3.7) kf − p k ≤ 4 ; n ∞ π(n − v)v

see, for instance, Trefethen [51] for details.

Inequality (3.7) guarantees convergence of the interpolation polynomials pn as n →

∞ to the function f when f is differentiable. However, if function f is continious but not differentiable then convergence is not guaranteed for any interpolations points. A famous result of Faber and Bernstein [15], [4], asserts that there is no sequence of sets of interpolation points that yields convergence as n → ∞ for all continuous functions. If a polynomial approximant of a function is desired on a bounded interval [a, b] different from

[−1, 1], then it is convenient to map this interval to [−1, 1] by a linear transformation and compute a polynomial interpolant or least squares approximant on the latter interval.

Eisinberg et al. [13] were the ﬁrst to describe a fast factorization method of a rectangular

Vandermonde matrix (3.1) determined by Chebyshev nodes (2.10). They presented the

45 factorization

(3.8) V = HUD,

N×n where H = [hij] ∈ R has the entries

(2i − 1)(j − 1) h = cos , i = 1, 2, . . . , N, j = 1, 2, . . . , n, ij 2N

n×n and the upper triangular matrix U = [uij] ∈ R has the elements

jnk jnk u = C(j−i), i = 1, 2,..., , j = i, i + 1,..., , 2i,2j 2j−1 2 2 lnm lnm u = C(j−i), i = 1, 2,..., , j = i, i + 1,..., , 2i−1,2j−1 2j−2 2 2 lnm u = C(j−1), j = 1, 2,..., . 1,2j−1 2j−3 2

Here b·c and d·e denote the “ﬂoor” and “ceiling” functions, respectively, and the

k! (3.9) C(`) = k (k − `)! `!

are binomial coeﬃcients. They arise in the expansion of the powers xk in terms of Cheby-

shev polynomials

(3.10) Ti(x) = cos(i arccos(x)), −1 ≤ x ≤ 1, i = 0, 1, . . . , k; see, for instance, Cody [9], Mason and Handscomb [36, Chapter 2], or (3.19) below. The

n×n matrix D = diag[d11, d22, . . . , dnn] ∈ R has the entries

1 d = , i = 1, 2, . . . , n. ii 2i−1

The representation (3.8) allows Eisinberg et al. [13] to express the Moore–Penrose pseudoinverse of V in factored form,

1 (3.11) V † = D−1QBHT , N 46 n×n where the matrix Q = [qij] ∈ R is the inverse of U. It has the entries

2j − 1 jnk jnk q = (−1)i+j C(j−i) , i = 1, 2,..., , j = i, i + 1,..., , 2i,2j 2i − 1 i+j−2 2 2 j − 1 lnm lnm q = (−1)i+j C(j−i), i = 1, 2,..., , j = i, i + 1,..., , 2i−1,2j−1 i − 1 2j−2 2 2 lnm q = (−1)j+1, j = 1, 2,..., . 1,2j−1 2

Moreover,

B = diag[1, 2, 2,..., 2] ∈ Rn×n.

Eisinberg et al. [13] compute the solution of (3.5) by evaluating

(3.12) c = V †y, using the factorization (3.11). The computation of the vector (3.12) in this manner requires

5Nn+2n2 arithmetic floating point operations (flops), assuming that the required binomial coefficients (3.9) are explicitly known. The flop count does not include the evaluation of the entries hij of the matrix H. Here and throughout this chapter, our flop counts only give the leading terms. In computations reported in Section 3.5, we precompute the coefficients

(3.9) that will be used and store them in a ﬁle. We will refer to this fast solver due to

Eisinberg et al. [13] as the EFS factorization algorithm.

It is the purpose of this chapter to discuss alternative fast algorithms for the factorization of rectangular Vandermonde matrices and the solution of (3.5). These algorithms exploit the structure of the matrix (3.1) in a diﬀerent manner than Eisinberg et al. [13].

We derive, in Section 4.2, a QR factorization

(3.13) V = QR, where the Vandermonde matrix (3.1) is determined by the nodes (2.10), the matrix Q ∈

RN×n has orthonormal columns, and the matrix R ∈ Rn×n is upper triangular. The entries

47 of the matrices Q and R are explicitly known, which makes fast solution of the least squares

problem (3.5) possible.

Another fast solution method for (3.5) is obtained by applying the formula

(3.14) V R˜ = Q.

The matrices V and Q in (3.13) and (3.14) are the same, and the matrix R˜ ∈ Rn×n in (3.14) is the inverse of R in (3.13). Thus, R˜ is upper triangular. Its entries are explicitly known.

It follows that (3.14) can be applied to derive a fast solution method for (3.5). The formula

(3.14) was derived by Li [35], who applied it to investigate the conditioning of rectangular

Vandermonde matrices with Chebyshev nodes; Li did not explore the application of the decomposition (3.14) in a fast solver for (3.5). Such a solver is described in Section 3.2.

Formulas analogous to (3.13) and (3.14) for the situation when the nodes are extreme points of a Chebyshev polynomial (3.3) also can be derived. The columns of the matrix

Q in the formulas then are orthonormal with respect to a weighted discrete inner product and associated norm. These formulas are discussed in Section 3.3, where we also consider the situation when the nodes x1, x2, . . . , xN are zeros of a classical orthogonal polynomial.

Computed examples that compare the accuracy of the computed solutions and the ex- ecution time required of the fast solvers outlined above and of a structure-ignoring QR factorization method are presented in Section 3.4. Concluding remarks for this chapter can be found in Section 3.5.

3.2 Fast factorization methods for Vandermonde matrices deﬁned by zeros of

Chebyshev polynomials.

This section discusses fast solvers for (3.5) based on the formulas (3.13) and (3.14) for

Vandermonde matrices with the nodes (2.10).

48 Introduce the normalized Chebyshev polynomials

 r 1   Tk(x), if k = 0, ˜ π (3.15) Tk(x) = r  2  T (x), if k > 0.  π k Lemma 3.2.1 The matrix

  ˜ ˜ T0(x1) ··· Tn−1(x1)   r  ˜ ˜  π  T0(x2) ··· Tn−1(x2)  (3.16) Q =   ∈ RN×n  . .  N  . .   . ··· .    ˜ ˜ T0(xN ) ··· Tn−1(xN ) has orthonormal columns for any 1 ≤ n ≤ N.

Proof: The orthonormality of the columns immediately follows from Theorem 2.3.1.

Proposition 3.2.1 The matrix (3.1) has a QR factorization (3.13) with Q ∈ RN×n given

n×n by (3.16) and the entries of the upper triangular matrix R = [rij] ∈ R deﬁned by

 √ j−i  1−j ( 2 )  2 NCj−1 , if j is odd, (3.17) r1j =  0, if j is even, and  q j−i  2−j N ( 2 )  2 2 Cj−1 , if j − i is even, (3.18) rij =  0, if j − i is odd or j < i.

49 Proof: Results by Cody [9] or Mason and Handscomb [36, Chapter 2] can be used to show j r j−i π X ( ) (3.19) xj = 21−j 0 C 2 T˜ (x), 2 j i i=0 j−i even √ where the ’ indicates that the ﬁrst term (for i = 0 and j even) is to be multiplied by 1/ 2.

It follows from (3.13) that every element of the matrix V can be expressed as j+1 j X xk = qkiri,j+1, i=1 where the qki are the entries of the matrix (3.16). Using (3.16) and (3.19) gives (3.17) and

(3.18). The QR factorization (3.13) can be used to solve the minimization problem (3.5) by

ﬁrst evaluating the vector QT y and then solving

Rc = QT y

by back substitution. These computations are backward stable; see [50]. The ﬂop count

for this solution method is 2Nn + n2 when n N: the evaluation of QT y requires 2Nn

ﬂops, and back substitution costs n2/2 ﬂops. The latter count utilizes the zero structure of

the matrix R. When n ≈ N, the vector QT b can be evaluated more rapidly (in O(N log N)

ﬂops) by application of the fast Fourier transform; see, e.g., [36, Chapter 4]. We are

primarily interested in the situation when n N.

Our MATLAB implementation of this solution method for (3.5) uses the standard

MATLAB function c = linsolve(R,Q’*y,opts)1 with opts.UT = true2 for back substi-

tution. This function does not exploit the zero structure of R. Back substitution therefore

requires n2 ﬂops in our implementation. We refer to this solution method as the explicit

QR factorization algorithm. 1x = linsolve(A,b,opts) solves the linear system Ax = b, using the solver that is most appropriate given the properties of the matrix A, speciﬁed in opts. 2opts.UT = true is set to make linsolve use a solver designed for upper triangular matrices.

50 We turn to a solution method based on the formula (3.14). The entries of the upper

triangular matrix

  r˜1,1 r˜1,2 r˜1,3 ··· r˜1,n      r˜ r˜ ··· r˜   2,2 2,3 2,n    (3.20) R˜ =   ∈ Rn×n  r˜3,3 ··· r˜3,n     . .   .. .      r˜n,n can be determined from the formula i b 2 c X i (k) T (x) = (−1)k C 2i−2k−1xi; i i − k i−k k=0 √ see [36, Chapter 2] for a proof. We obtainr ˜1,1 = 1/ N and r 2 j − 1 r˜ = (−1)i C(i) 2j−2i−2, r˜ = 0 j,j−2i N j − i − 1 j−i−1 j,j−2i−1 for j = 2, 3, . . . , n and i = 0, 1 ..., bj/2c; see also Li [35].

It follows from (3.14) that V † = RQ˜ T . The solution of (3.5) can be computed by matrix-vector product evaluations with the matrices QT and R˜:

(3.21) c = R˜(QT y).

Straightforward evaluation of (3.21) using the MATLAB command R*(Q’*y)˜ requires

2Nn + n2 ﬂops, assuming that the entries of the matrices R˜ and Q are known. Note that the matrix-vector product evaluation with R˜ does not use the zero-structure of R˜.

We will refer to the expression (3.14) as a QR-like factorization of V , and to the solution method based on the computations (3.21) as the explicit QR-like factorization algorithm.

The computations (3.21) are backward stable.

51 We will compare the fast algorithms discussed to the structure-ignoring QR factoriza-

tion method that is implemented by the MATLAB function [Q,R]=qr(V). This function

computes a QR factorization of the matrix V by Householder triangularization. The com-

2 2 3 putation of this factorization requires 2Nn − 3 n flops; see, e.g., [50] for details. Evaluation of the solution vector c demands 2Nn + n2 additional flops. The total flop count therefore

2 2 3 2 is 2N(n +n)− 3 n +n . We refer to this solution method as the standard QR factorization algorithm.

3.3 Fast factorization methods for Vandermonde matrices deﬁned by extrema

of Chebyshev polynomials and extensions

In this section the nodes xk are the extreme points (3.3) of the Chebyshev polynomial

TN−1.

Lemma 3.3.1 Deﬁne the matrices   ˜ ˜ T0(x1) ··· Tn−1(x1)    ˜ ˜  r π  T0(x2) ··· Tn−1(x2)  (3.22) Q =   ∈ RN×n  . .  N − 1  . .   . ··· .    ˜ ˜ T0(xN ) ··· Tn−1(xN ) and h √ √ i (3.23) E = diag 1/ 2, 1, 1,..., 1, 1/ 2 ∈ RN×N .

T 2 1/2 Introduce the inner product (u, v)E = u E v and associated norm kvkE= (v, v)E for u, v ∈ RN . Then the matrix Q has orthonormal columns with respect to this inner product

and norm.

Proof: Introduce the discrete inner product N−1 1 X 1 [f, g] = f(x )g(x ) + f(x )g(x ) + f(x )g(x ) 2 1 1 i i 2 N N i=2 52 for polynomials f and g of degree at most N − 1. It is well known that the Chebyshev

polynomials (3.10) satisfy the orthogonality relation   0, j 6= k,   N−1 (3.24) [Tj,Tk] = , j = k 6= 0,  2   N − 1, j = k = 0; see, e.g., [36, p. 87] for a proof. Rescaling according to (3.15) shows the lemma. The following result can be shown similarly as Proposition 3.2.1. We refer to a factorization

V = QR ∈ RN×n as a QR-type factorization if the columns of Q ∈ RN×n are orthonormal with respect to a weighted inner product and associated norm, and R ∈ Rn×n is upper triangular.

Proposition 3.3.1 Let the nodes of the Vandermonde matrix (3.1) be Chebyshev extreme points (3.3) and let E be deﬁned by (3.23). Then the matrix V has a QR-type factorization

V = QR, where Q, deﬁned by (3.22), has orthonormal columns with respect to the inner product (·, ·)E and associated norm, and the entries of the upper triangular matrix R =

n×n [rij] ∈ R are given by  √ j−i  1−j ( 2 )  2 N − 1Cj−1 , if j is odd, r1j =  0, if j is even, and  q j−i  2−j N−1 ( 2 )  2 2 Cj−1 , if j − i is even, rij =  0, if j − i is odd or j < i. The matrices of Proposition 3.3.1 can be applied to solve the weighted least squares problem

(3.25) minkV c − ykE. c∈Rn

Its solution can be computed as

Rc = QT E2y.

53 The vector c is evaluated by back substitution. A QR-like factorization, analogous to

(3.14), also can be derived. We omit the details.

Let dµ be a nonnegative measure with support on the real axis and let p0, p1, p2,... denote a family of orthonormal polynomials associated with this measure. One can derive a QR-type factorization of a Vandermonde matrix V = QR ∈ RN×n, whose nodes x1, x2, . . . , xN are the zeros of pN , by using the orthogonality of the polynomials p0, p1, . . . , pn−1 with respect to a discrete inner product deﬁned by the N-point Gauss quadrature rule associated with dµ. Thus, the columns of Q ∈ RN×n are orthonormal with respect to a weighted inner product and associated norm determined by the Gauss quadrature rule; the matrix R ∈ Rn×n is upper triangular. The computation of this factorization requires the evaluation of the nodes and weights of this Gauss rule. This can be carried out rapidly in several ways when the recursion coeﬃcients of the polynomials pj are known or easily computable; see [6, 20, 24, 34]. One obtains a factorization V = QR, where R ∈ Rn×n is upper triangular, and the columns of Q ∈ RN×n are orthonormal with respect to a discrete inner product and associated norm determined by the N-point Gauss rule. The matrix R is analogous to the upper triangular matrix of Proposition 3.3.1. Its entries are explicitly known for many families of classical orthogonal polynomials; see, e.g., Szeg˝o[49].

3.4 Numerical experiments.

We report numerical experiments that shed light on the performance of the EFS, explicit

QR, and explicit QR-like factorization algorithms for Vandermonde matrices V ∈ RN×n given by (3.1) with the nodes (2.10). For each pair (n, N) with N ≥ n, we generate 10000 vectors y with uniformly distributed entries in the interval [−1, 1], and compute for each one of these vectors the solution c of the least squares problem (3.5) so deﬁned by using the fast algorithms as well as the “slow” standard structure-ignoring QR factorization

54 algorithm based on the use of Householder matrices. The exact solution c∗ is calculated with Mathematica using high-precision arithmetic; these computations are carried out with 50 signiﬁcant decimal digits. All other computations are carried out in MATLAB with about 15 signiﬁcant decimal digits on a Dell computer with an i7-4770 processor running at 3.44 GHz.

n=20 1 expl. QR fact. 0.9 expl. QR-like fact. EFS fact. 0.8 stand. QR fact.

0.7

0.6

0.5

time, seconds 0.4

0.3

0.2

0.1

0 100 150 200 250 300 350 400 450 500 N

Figure 3.1: Computing time as a function of N for n = 20 for the explicit QR, explicit QR-like, EFS, and standard QR factorization algorithms for Vandermonde matrices with Chebyshev nodes.

Figure 3.5 displays the CPU time required by the algorithms as a function of N for

55 n = 20. The graphs show the total time required by each method for 10000 experiments.

The CPU time is seen to grow linearly with N for all algorithms in agreement with the ﬂop counts reported in Sections 3.1 and 3.2. The CPU time for the explicit QR and QR-like algorithms grows the slowest with N.

N=500 102 expl. QR fact. expl. QR-like fact. EFS fact. stand. QR fact. 101

100 time, seconds

10-1

10-2 5 10 15 20 25 30 35 40 n

Figure 3.2: Computing time as a function of n for N = 500 for the explicit QR, explicit QR-like, EFS, and standard QR factorization algorithms for Vandermonde matrices with Chebyshev nodes.

Figure 3.2 shows how the CPU time depends on the parameter n for N = 500. The times

are for 10000 experiments. The CPU time required by the explicit QR, explicit QR-like,

56 and EFS factorization algorithms is seen to grow roughly linearly with n, while the CPU

time for the standard QR factorization algorithm grows quadratically with n, in agreement

with the ﬂop counts for these methods. Figures 3.1 and 3.2 show the structure exploiting

explicit QR and explicit QR-like factorization algorithms of Section 3.2 to determine the

fastest solutions.

N=100 100 expl. QR-like fact. stand. QR fact.

10-5

10-10 relative mean error

10-15

10-20 0 5 10 15 20 25 30 35 40 n

Figure 3.3: Mean error as a function of n for N = 100 for the explicit QR, explicit QR-like, EFS, and standard QR factorization algorithms for Vandermonde matrices with Chebyshev nodes.

∗ kc − c k2 Figure 3.1 displays the mean relative error over 10000 samples for each ∗ kc k2

57 pair (n, N) for 2 ≤ n ≤ 40 and N = 100 for the four algorithms in our comparison.

The ﬁgure shows all the algorithms to give roughly the same accuracy for all n-values; for

small n-values the slow structure-ignoring QR factorization algorithm gives slightly higher

accuracy than the fast algorithms, but for n close to 40 the fast algorithms yield more accurate solutions. The accuracy achieved with the algorithms of Section 4.2 is almost indistinguishable. Figure 3.2 diﬀers from Figure 3.1 only in that N = 1000. The relative performance of the methods compared is quite similar in Figures 3.1 and 3.2. The average errors in the computed solutions can be seen to grow exponentially with n for ﬁxed N.

Comparing Figures 3.1 and 3.2 shows that the error does not grow signiﬁcantly with N for ﬁxed n. This behavior is in agreement with the result by Eisinberg et al. [13] that the spectral condition number of a rectangular Vandermonde matrix V with Chebyshev nodes is independent of the number of nodes N ≥ n.

Fitting a straight line to logarithmically scaled data in the least square sense, we ﬁnd the error growth to be proportional to (2.4)n. This holds for the data of both Figures 3.1 and 3.2. This growth rate is close to the growth of the condition number of Vandermonde matrices with Chebyshev nodes; see Section 3.1.

Figure 4.4 shows the maximum component-wise relative error. Each point of each graph is the maximum relative error over 10000 least squares problems (3.5) with data-vectors y ∈ RN with uniformly distributed entries in [−1, 1]. This error is seen to be larger than the mean relative error shown in Figure 3.2, but just like in the latter ﬁgure the errors in the computed solutions of the algorithms compared are of about the same size.

58 3.5 Chapter summary

In this chapter we present an explicit QR and QR-like factorizations of rectangular

Vandermonde matrices with Chebyshev nodes and Chebyshev extreme points. Such matrices arise in polynomial least squares approximation problems and yield the polynomial in a particularly simple form for further processing, including diﬀerentiation and integration. Based QR and QR-like factorizations new fast methods for solving least squares problem are proposed. The new methods are found to be approximately 2.5 time faster as factorization method provided by Eisinberg et al. [13]. The computed examples show the accuracy of our methods to be about the same as that of the much slower standard QR factorization algorithm.

59 N=1000 100 expl. QR fact. expl. QR-like fact. EFS fact. stand. QR fact. 10-5

10-10 relative mean error

10-15

10-20 0 5 10 15 20 25 30 35 40 n

Figure 3.4: Mean error as a function of n for N = 1000 for the explicit QR, explicit QR-like, EFS, and standard QR factorization algorithms for Vandermonde matrices with Chebyshev nodes.

60 N=1000 100 expl. QR fact. expl. QR-like fact. 10-2 EFS fact. stand. QR fact. 10-4

10-6

10-8 relative mean error 10-10

10-12

10-14 0 5 10 15 20 25 30 n

Figure 3.5: Maximum component-wise relative error as a function of n for N = 1000 for the explicit QR, explicit QR-like, EFS, and standard QR factorization algorithms for Vandermonde matrices with Chebyshev nodes.

61 CHAPTER 4

Conditioning optimization of a measurement problem

4.1 Introduction

In this chapter the mathematical model of conditioning of a real-world measurements problem is derived, and numerical experiments are provided.

Today’s state-of-the-art experimental techniques have been greatly inﬂuenced by integration of powerful computers in the experimental setups used in microscopy, tomogra- phy, material science, biology, etc. A computer in a measuring system can have several functions. First, it can provide fast real-time calculations that mitigate disadvantages of indirect measurements, when an unknown quantity λ is determined from a measured value

b through the measuring function Φ(λ) = b. Second, it allows one to perform simultaneous

measurements when several unknown quantities Λ = [λ1, ..., λn] are determined from the

measurements of the same or larger, number of diﬀerent physical quantities b = [b1, ..., bn]

by solving a system of equations Φi(λ) = bi possibly in the least squares sense; see e.g. [43].

Third, the computer can control and change measuring conditions. In this case, several

stationary or slowly changing unknown quantities Λ = [λ1, ..., λn] can be determined in

an experimental setup with a single measuring unit by measuring a related single physical

variable at preprogrammed diﬀerent conditions, deﬁned by control parameter(s) Ai,

(4.1) Φ(Ai, Λ) = bi, i = 1, ..., n,

where Ai may be a scalar Ai = ai or a k-dimensional vector Ai = [ai1, ..., aik], controlled by a computer. Measurements of the form (4.1) are referred to as combined measurements [43].

62 For simplicity we call combined multi-parameter measuring systems as CMPM systems.

The problem of best estimation of unknown quantities Λ from a system (4.1) for the case

of a linear measuring functions Φ were considered in [5], [43] with the solution determined by the least squares method. We consider the problem how to properly select the control parameters. The accuracy of the measured quantities Λ is determined not only by the accuracy of the measurements themselves, but also by the accuracy of the settings of the control parameters A = [A1, ..., An], where Ai = [ai1, ..., aik], i = 1, ..., n. An a priory selection should be “optimal” for all possible values of Λ. We consider the class of CMPM systems that are described by a separable measuring function,

n X (4.2) Φ(Ai, Λ) = mj(Ai)vj(Λ), i = 1, ..., n, j=1 where mj, vj, j = 1, ..., n, are smooth nonlinear functions. We develop a method for selection of control parameters in such systems that are near-optimal for all possible values of determined quantities. Using the submultiplicativity of the spectral and Frobenius matrix norms we construct the upper bound of the error function as a product of two functions, one of which depends only on control parameters and another one depends only on unknown quantities. We determine the set of control parameters by minimizing the former function. To demonstrate the full capability of the proposed method, we use

CMPM system with the following features:

(a) The measuring function (4.2) has a simple dependence on several control parameters and unknown quantities;

(b) many sets of unknown quantities are determined concurrently at the same values of control parameters;

(c) the measurement errors are dependent on both the control parameters and unknown quantities, and thus cannot be minimized simultaneously at each point of the sample. As

63 an example of such measuring system we considered the PolScope microscope.

The PolScope (also called LC-PolScope) described in [37],[39],[46] and its advanced ver-

sions, the Exicor MicroImager (Hinds Instruments) [14], and the Phi-Viz Imaging System

(Polaviz) [42] are among the latest polarized light microscopy techniques. The PolScope

determines two-dimensional ﬁelds of phase retardation and of the optical axis direction of

thin anisotropic samples by measuring transmitted light intensity under diﬀerent condi-

tions. The PolScope is equipped with a standard set of light source, monochromatic ﬁlter,

polarizers, lenses, and CCD camera. The additional variable optical retarders are inserted

in the optical path; for details see Section 4.3 and Figure 4.1. For each point the sample

quantities ∆(x, y) and φ(x, y) are determined by measuring transmitted light intensities for diﬀerent settings of two variable retarders. The intensity is a product of Jones matrices of individual optical elements and has the separable form (4.2). We apply our method to

compute the control parameters for PolScope. We compare the computed set of control

parameters with other sets, including the one used in the PolScope, and demonstrate that

our choice of control parameters works very well even though it does not take into account

any speciﬁc features of the PolScope.

4.2 Determination of control parameters for CMPM systems

For separable measuring functions of the form (4.2), the nonlinear system (4.1) can be

presented in matrix form

(4.3) M(A)V (Λ) = b,

where M(A) is a matrix with elements Mij = mj(Ai), i, j = 1, ..., n, and V (Λ) is a vector

of functions vj(Λ), j = 1, ..., n. The elements of the vectors b, Λ, and Ai may correspond

to diﬀerent physical quantities and have diﬀerent dimensions. In this case, we scale the

64 matrix M(A) and the vectors b, Λ, Ai and V (Λ) to make them dimensionless. In this case, we scale the matrix M(A) and the vectors b, Λ, Ai and V (Λ) to make them dimensionless.

The error-vector δΛ˜ = Λ∗ − Λ has two sources: the errors caused by inaccuracies of the

∗ ∗ measuring unit, δb = b − b and the errors of control parameters δA = A − A = {δaij},

∗ ∗ ∗ i = 1, ..., n, j = 1, ..., k. Here Λ , Ai and b are the true error-free values that obey the

∗ ∗ equations Φ(Ai , Λ ), i = 1, ..., n and thus can be presented in the form

(4.4) M(A∗)V (Λ∗) = M(A + δA)V (Λ + δΛ)˜ = b + δb

Unlike the classical least squares problem considered in previous chapters, in (4.4) we have a more complicated case; the perturbations are presented not only in measured data vector b, but also in the matrix A. We establish the dependence of the unknown errors δΛ˜ of the control parameters and the measurement errors using methods of perturbation analysis

[31], applying ﬁrst-order Taylor expansion, V (Λ + δΛ) = V (Λ) + δV , where δV = D(Λ)δΛ˜ ∂vj(Λ) and D(Λ) is the Jacobian matrix with elements Dji = , i, j = 1, ..., n. Similarly, ∂λi M(A + δA) = M(A) + δM, where deviations of elements of the control matrix,

(4.5) δMij = ∇mj(Ai)δAi,

are caused by the errors in the control parameters δAi = [δαi1, ..., δαik]. In (4.5) ∇ = h ∂ ∂ i , ..., is the gradient of the control parameters. Neglecting the second order ∂αi1 ∂αik term δMδV in equation (4.4) and subtracting equation (4.3) leads to the equation

(4.6) M(A)δV + δMV (Λ) = δb,

which transforms into an expression for the measurement error vector:

(4.7) δΛ˜ = D−1(Λ)M −1(A)δb − D−1(Λ)M −1(A)δM(A, δA)V (Λ).

65 ˜ We can scale the error vector δΛ=ΘδΛ by selecting the numerical values of Θii , i = 1, ..., n,

based on the “importance” of determined values λi; however, as one can see below, the

proposed selection of the control parameters A does not depend on Θ. We measure the v u n uX 2 error vector with the spectral norm kδΛk2= t δλi . From equation (4.7), the vector i=1 δΛ consists of two terms, δΛ = δΛM + δΛB, where the error

−1 −1 −1 (4.8) δΛM = ΘD D (Λ)M (A)δM(A, δA)V (Λ),

is caused by errors in the setting of the control parameters, and

−1 −1 (4.9) δΛB = −ΘD (Λ)M (A)δb

stems from measurement inaccuracies. Elements of the vectors δΛM and δΛB are lin-

ear functions of respectively, δαij and δbi , which we assume to be accidental errors, i.e.

mutually independent, random zero-mean quantities:

2 2 (4.10) E[δαijδαi0j0] = σ (δij)δii0,E[δbjδbj0] = σ (δbj)δjj0,E[δaijδbk] = 0,

here E is the expectation operator and σ is the standard deviation. With this assumption

we obtain the equation:

2 2 2 (4.11) E[kδΛk2] = E[kδΛM k2] + E[kδΛBk2],

where

n k 2 2 X X 2 ∂λi 2 (4.12) E[kδΛM k2] = Θii σ (δαlj), ∂αlj i,l=1 j=1

and

n 2 2 X 2∂λi 2 (4.13) E[kδΛBk2] = Θii σ (δbi), ∂bi i 66 deﬁne the contributions caused by the errors in the control parameters and measured values,

respectively. Both functions depend on the control parameters A and on the quantities Λ; the former are usually a single set selected before the experiment, the latter are unknown and may change during the experiment. A single set A cannot concurrently minimize

2 2 E[kδΛM k2] and E[kδΛBk2] or their sum (4.11) for all measured values of Λ. Thus, we consider them separately and minimize their upper bounds using the sub-multiplicative property of the spectral matrix norm and the linearity of the expectation operator:

(4.14)

2 −1 −1 2 −1 2 2 −1 2 E[kδΛM k2] = E[kΘD (Λ)M (A)δMV (Λ)k2] ≤ kΘD (Λ)k2kV (Λ)k2E[kM (A)δMk2],

2 −1 −1 2 −1 2 −1 2 (4.15) E[kδΛBk2] = E[kΘD (Λ)M (A)δbk2] ≤ kΘD (Λ)k2kE[kM (A)δbk2],

kW xk2 The spectral matrix norm of an arbitrary matrix W is kW k2= sup . To mini- x6=0 kxk2 −1 2 2 mize (4.14), we keep the factors kΘD (Λ)k2 and kV (Λ)k2, which are independent of

−1 2 A, and bound the spectral norm in E[kM (A)δMk2] by the Frobenius matrix norm

−1 2 −1 2 kM (A)δMk2≤ kM (A)δMkF , [50], because the square of the Frobenius norm, deﬁned

2 T X X 2 as kW kF = trace(W W ) = Wij, is easily explicitly calculated as i j

n n n k 2 −1 2 X X X X ∂ms(Ai) (4.16) kM (A)δMkF = ψpi δαij , ∂αij p=1 s=1 i=1 j=1

−1 −1 where ψpi = [M (A)]pi are the elements of the inverse matrix M (A) . Using equations

2 (4.16) and (4.10), we obtain an upper bound for E[kδΛM k2],

n n n k 2 2 −1 2 2X X X X 2 2 ∂ms(Ai) (4.17) E[kδΛM k2] ≤ kΘD (Λ)k2kV (Λ)k2 σij(δαij)ψpi . ∂αij p=1 s=1 i=1 j=1

2 2 If we can assume that the absolute errors δαij are identically distributed, σij = σα, then

2 2 −1 2 2 a (4.18) E[kδΛM k2] ≤ σαkΘD (Λ)k2kV (Λ)k2PM (A),

67 where

n n n k 2 a X X X X 2 2 ∂ms(Ai) (4.19) PM (A) = σijψpi ∂αij p=1 s=1 i=1 j=1

2 is the function that governs the minimization of E[kδΛM k2]. If the errors in the control parameters δαij are proportional to their magnitudes αij, then δij can be expressed as αij, where is a random variable, σ(τ) = σ, σ(δαij) = σαij, and

2 2 −1 2 2 r (4.20) E[kδΛM k2] ≤ σ kΘD (Λ)k2kV (Λ)k2PM (A), is determined by the function

n n n k 2 r X X X X 2 2 ∂ms(Ai) (4.21) PM (A) = αijψpi . ∂αij p=1 s=1 i=1 j=1

2 Now, we consider the function E[kδΛBk2], see equation 4.15, which depends on A through

−1 2 E[kM (A)δbk2]. If the measurement errors δbi do not depend on bi and are identically

−1 2 −1 2 2 distributed, σ(δbj) = σb, then E[kM (A)δbk2] = kM (A)kF σb and

2 2 −1 2 a (4.22) E[kδΛBk2] ≤ σb kΘD (Λ)k2Pb (A) is determined by the function

n n a −1 2 X X 2 (4.23) Pb (A) = kM (A)kF = ψij(A). i=1 j=1

If the measurement errors δbi are proportional to bi , then the error vector δb can be expressed as δb = Ξb , where Ξ is a diagonal matrix with identically distributed random elements Ξii, σ(Ξii) = σΞ. From the equality M(A)V (Λ) = b, we get

2 2 2 −1 2 r (4.24) E[kδΛBk2] ≤ σΞkV (Λ)k2kΘD (Λ)k2Pb (A),

r where Pb (A) is condition number of M(A), n n r −1 2 2 X X 2 2 (4.25) Pb (A) = kM (A)kF kM(A)kF = ψij(A)mps(A). i,j=1 p,s=1

68 Using equations (4.18), (4.20), (4.22) and (4.24), one can see that the “optimal” choice of the control parameters that minimizes the upper bound for the total error (4.11) can be found by minimization of the weighted function

ξ η (4.26) Ptot(A) = wPM (A) + (1 − w)Pb (A), where ξ and η are either a, see equations (4.19) and (4.23), or r, see (4.21) and (4.25), depending on whether the errors in the control parameters and the measured values are absolute or relative. The weighting coeﬃcient w is determined by the products in equations

(4.18), (4.20), (4.22) and (4.24).

If the errors in the measured values bi have relative form, see equation (4.24), then w is determined only by the standard deviations σν and σΞ,

2 σν (4.27) w = 2 2 , σν + σΞ where the parameter ν is either a, or , depending on whether the errors in the control parameters and the measured values are absolute or relative. In this case, equation (4.26) does not depend on the unknown quantities Λ and, therefore, allows one to determine the control parameters A by minimization of the function (4.26). If the errors in the measured values are absolute, see equation (4.22), then the weighting coeﬃcient

2 2 kV (Λ)k2σν (4.28) w = 2 2 2 , kV (Λ)k2σν + σb depends on Λ. Additional assumptions are required to justify the minimization of (4.26).

2 If either term in the denominator of (4.28) is or kV (Λ)k2 remains almost constant in the domain of possible values of Λ, then one may assume w to be constant and minimize

(4.26). In the case when w in (4.28) strongly depends on Λ, we propose the following procedure, which is similar to single parameter optimization. For several selected values of

69 w, we calculate the corresponding “optimal” sets A by minimizing the function (4.26), and

between them choose the best set by comparing the total error functions (4.11). Further

reﬁnement of w can be performed if necessary. To summarize, the goal of our method is to

construct an eﬀective error function (4.11), which does not depend on unknown quantities

and is an upper bound for the true error function (4.11). We then minimize function (4.26) and determine the optimal set of control parameters A. Below we demonstrate how the proposed method is implemented for PolScope.

4.3 Application of the proposed method to the PolScope

In this section we apply the proposed method to PolScope [46], [39], [40]. PolScopes are widely used to study patterns with submicron resolution in various inhomogeneous anisotropic ﬁlms, in particular, biological cells, polymer samples, liquid crystal ﬁlms, etc.

The PolScope is a polarized light microscope, where a quarter-wave plate and two liquid crystal (LC) variable optical retarders LCA and LCB, see Figure 4.1, are added to

a standard set of optical elements: light source, monochromatic ﬁlter, polarizer, lenses,

analyzer and CCD camera. The PolScope determines in-plane, two-dimensional ﬁelds of

phase retardance ∆(x, y) and of the azimuth φ(x, y) of slow optic axis from a sequence of

measurements of transmitted light intensity obtained for the diﬀerent retardances of LCA

and LCB set by the computer controlled applied voltages. The phase retardance ∆(x, y)

and azimuth φ(x, y) are determined by the four-frame algorithm [46], [39], [40]. The corresponding function of the output intensity at the CCD camera Iout can be derived within the Jones calculus, which is an eﬃcient technique to analyze optical devices consisting of linear non-reﬂecting polarization, sensitive optical elements.

70 Figure 4.1: Scheme of the PolScope microscope. Liquid-crystal plates LCA and LCB have variable values of retardances, α1 and α2; λ/4 is a quarter-wave plate with phase shift between polarization components equal π/2, P and A are the linear polarizer and analyzer transmit only horizontal component of the beam.

The basic elements of the Jones calculus is the 2D Jones vector j = [Ex Ey], which is the complex amplitude of the electric field E = jexp(2π(z − ct)/λ) of the plane monochromatic wave with the wavelength λ, and 2 × 2 Jones matrix of an optical element J that defines the transformation of the Jones vector jout = Jjin (4.11). The optical scheme of the PolScope system, shown on Figure 4.1, contains two types of optical elements: linear polarizers and phase retarders. The Jones matrix of a linear polarizer, which transmits " 1 0 # horizontal polarization along the Ox axis, is JH = . The Jones matrix of a phase 0 0 retarder is defined by its phase retardance ψ and the angle θ between its slow optic axis and the Ox axis,

(4.29) Jr(ξ, θ) = R(−θ)Jr(ξ, 0)R(θ),

" e−iξ/2 0 # where Jr = . is the Jones matrix if the slow axis is parallel to Ox axis, 0 eiξ/2

" cos(θ) − sin(θ) # and R(`) = is the rotation matrix. Then, the Jones matrices of the sin(θ) cos(θ)

71 optical elements in the PolScope are: J1 = JH for the linear polarizer, J2 = Jr(α1, π/4) for the liquid-crystal variable retarder LCA rotated by π/4, J3 = Jr(α2, 0) for the liquid- crystal variable retarder LCB, J4 = Jr(∆(x, y), φ(x, y)) for the (x, y) pixel of the specimen,

J5 = Jr(π/4, π/4) for the quarter-wave plate rotated by π/4, and J6 = JH for the linear analyzer. Thus, the transmission coeﬃcient the PolScope TPS is determined by the Jones matrix JPS = J6J5J4J3J2 that transforms the normalized Jones vector after the polarizer jin = [1 0] into the Jones vector after the analyzer jout = JPSjin :

(4.30) 1 T = j∗ j = (1 + sin α cos α cos ∆ − sin α sin α cos 2φ sin ∆ + cos α sin 2φ sin ∆), PS out out 2 1 2 1 2 1

∗ where jout denotes complex conjugate of vector jout.

Considering the inhomogeneous distributions of the input intensity Iin(x, y) and of the depolarized leakage Ileak(x, y), one obtains the output intensity distribution at the CCD camera Iout(x, y) as Iout = Ileak + IinTPS,

(4.31) 1 I = I + I (1 + sin α cos α cos ∆ − sin α sin α cos 2φ sin ∆ + cos α sin 2φ sin ∆), out leak 2 in 1 2 1 2 1

The intensity function (4.31) has separable form (4.2) and contains the four-dimensional

T vector of unknown quantities Λ = [Ileak,Iin, ∆, φ] that requires four measurements, i = 1, ..., 4, of the intensity Iout = Ii with diﬀerent sets of the control parameters Ai =

[αi1, αi2]. Thus, Λ is determined from equation (4.3) where

  1 sin α11 cos α12 sin α11 sin α12 cos α11      1 sin α21 cos α22 sin α21 sin α22 cos α21    (4.32) M(A) =  ,    1 sin α31 cos α32 sin α31 sin α32 cos α31    1 sin α41 cos α42 sin α41 sin α42 cos α41

72   Iin   Ileak + I1  2     I cos ∆     in   I2   2    (4.33) V (Λ) =   , b =   ,  Iin sin ∆cos 2φ   −   I3   2        Iin sin ∆sin 2φ I 2 4

In equations (4.8) and (4.9), we choose the diagonal matrix Θ = diag[0, 0, 1, 1] because

we are not interested in values of Iout and Iin, and the “importance” of the dimensionless

quantities ∆ and φ is assumed to be equal.

  0 0 0 0      0 0 0 0  1   −1   (4.34) ΘD =  sin 2φ  , Iin  0 0 −2 cos ∆ cos 2φ   sin ∆   sin 2φ   0 0 2 cos ∆ sin 2φ  sin ∆

multiplies both error vectors (4.8) and (4.9). Taking into account that the depolarized

leakage Ileak depends linearly on Iin , and Ileak Iin, we assume that the measuring

errors do not signiﬁcantly depend on Iin and Ileak. We set Iin = 1, and Ileak = 0. For

the PolScope, the errors in the control parameters δαij are absolute, i.e. do not depend

on the parameter values aij, and the measurement errors δbi are relative, δbi = Ξiibi, where Ξii ∼ 1% [46]. Thus, we start the search of the set of “optimal” values of A by separately minimizing the functions (4.19) and (4.25). To determine these minima, we apply the Nelder-Mead algorithm implemented in Wolfram Mathematica. The function

73 (4.19) achieves its minimum 4.907 at

(4.35) A = AM = [[180◦, 90◦], [51.36◦, 180◦], [51.36◦, 0◦], [91.83◦, 90◦]],

where for better visibility we present here and below the variable phase redardances α1 and

a B α2 in degrees, rather than in radians. The function (4.25) reaches minPM (A ) = 4.909 at

(4.36) A = AB = [[180◦, 90◦], [51.58◦, 180◦], [51.58◦, 0◦], [91.56◦, 90◦]],

M B a B Note that the sets A and A are almost the same, and the value PM (A ) = 4.909 is close

a M r M to the minimum of the function PM (A ), while Pb (A ) = 3.906 is close to minimum of the

r B function Pb (A ). Thus, the set of “optimized” control parameters is almost independent of the weighting coeﬃcient in the weighted function (4.26) and we can choose either set

of the control parameters AM or AB. We select A = AB as the “optimal” set of control

parameters. To compare diﬀerent sets of control parameters, we represent their values

Ai = [αi1, αi2] by the Jones vector of the light entering the sample. A linear polarizer

◦ ◦ P (0 ) and two liquid crystal plates LCA(α1, 45 ) and LCB(α2, 0), see ﬁgure (4.1), form the

universal compensator (4.10), because the variable retardance values α1 := αi1, α2 := αi2

provide the transformation of the unpolarized illumination light beam into any polarization

state, with the Jones vector:

T h αi1 αi2 αi1 αi2 π i (4.37) j = cos e−i 2 sin ei( 2 − 2 ) , 3 2 2

Figure 4.2 shows the control parameters Ai = [αi1, αi2] and the corresponding polarization

state (4.37) using standard representations with the Stokes parameters, S0 = 1, S1, S2, S3,

and the Poincar´esphere, [10]:

S1 = cos αi1 = cos χ cos ψ,

(4.38) S2 = sin αi1 sin αi2 = cos χ sin ψ,

S3 = − sin αi1 cos αi2 = sin χ,

74 Figure 4.2: Representation of the control parameters of measurement Ai = [αi1, αi2] through the Jones vector (4.36) on the Poincar´esphere in the space of the Stokes parameters, S1, S2, S3. χ is the ellipticity angle and ψ deﬁnes the orientation of the major axis of the polarization ellipse.

On the Poincar´esphere, right and left circular polarizations correspond to the north and south poles, respectively, and the linear polarizations lie on the equator. In the four- frame algorithm, for the samples with full range of possible retardance values the following set of control parameters was used:

(4.39) A90 = [[90◦, 180◦], [90◦ − X, 180◦], [90◦ + X, 180◦], [90◦, 180◦ − X]],X = 90◦

To study biological samples, e.g., living cells, when retardance values of the sample have

75 mostly small values, Oldenbourg and Shribak [46] proposed to use the set of control parameters:

(4.40) A11 = [[90◦, 180◦], [90◦ − X, 180◦], [90◦ + X, 180◦], [90◦, 180◦ − X]],X = 10.8◦

We also examine a conﬁguration on the Poincar´esphere that is interesting from a symmetry point of view - the right tetrahedron set:

(4.41) AT = [[90◦, 180◦], [30◦, 0◦], [115.66◦, 56.3◦], [115.66◦, −56.3◦]],

The polarization states of the four illumination beam settings (4.36), (4.39), (4.40), (4.41) are shown on the Poincar´esphere in Figure 4.3.

76 Figure 4.3: Sets of control parameters AB, A90, A11, AT shown on the Poincar´esphere.

To compare the eﬀect of the selected control parameter sets on the measurement accuracy, we calculate the errors of phase retardance δ∆ and azimuth of the slow optical axis δφ using equation (4.7). The errors δ∆ and δφ split into the control parameters errors, δ∆M , δφM , (4.8), and the measuring errors, δ∆B, δφB, (4.9). Considering the control parameter errors δA = αij, i = 1, ..., 4, j = 1, 2, to be uniformly distributed, √ √ δαij ∼ unif[− 3σα, 3σα] with standard deviation σα, we obtain from (4.12) the standard

deviations of the propagated errors δ∆M and δφM ,

77 v 4 2 v 4 2 u 2 u 2 uX X ∂∆M uX X ∂φM (4.42) σ(δ∆M ) = σαt , σ(δφM ) = σαt . ∂αij ∂αij i=1 j=1 i=1 j=1

σ(δ∆M ) σ(δφM ) The normalized standard deviations , are shown in ﬁgure (4.4) as func- σα σα tions of ∆ for several ﬁxed values of φ for the set A = AB.

Figure 4.4: Normalized standard deviations σ(δ∆M )/σα and vs. σ(δφM )/σα for diﬀerent

values of φ and the set A = AB.

We assume that the diagonal elements Ξii that determine the relative measurement er- √ √ rors δbi = Ξiibi are identically uniformly distributed random variables Ξii ∼ unif[− 3σΞ, 3σΞ] with standard deviation σΞ = σ(δΞii). Then from equation (4.13), we obtain the standard deviations for the propagated errors δ∆B and δφB, v v 4 4 u 2 u 2 uX ∂∆B uX ∂φB (4.43) σ(δ∆B) = σΞt bi , σ(δφB) = σΞt bi , ∂bi ∂bi i=1 i=1

σ(δ∆M ) σ(δφM ) The normalized standard deviations and are shown in ﬁgure (4.5) as σΞ σΞ 78 functions of ∆ for several ﬁxed values of φ for the set A = AB.

Figure 4.5: Normalized standard deviations σ(δ∆B)/σΞ and σ(δφB)/σΞ vs. ∆ for diﬀerent

values of φ, set A = AB.

For the other sets A90, A11, and AT , the functions (4.42) and (4.43) exhibit similar

weak dependence on φ and strong dependence on ∆. Thus to compare the sets AB, A90,

A11, and AT , we use the functions over the whole interval of possible values of φ, e.g.

1 Z π (4.44) δ∆M = σ(δ∆M )dφ πσα 0

and the averaged functions δφM , δ∆B, δφB are deﬁned similarly as in (4.44). Figure 4.6 demonstrates that the parameter set AB determined by our method provides the smallest values of δ∆M , δφM , δ∆B, and δφB in almost the entire range of ∆. Only in the case of small values of retardation ∆, the functions δ∆B and δφB take the smallest values for the

set A11, which has been specially designed by Oldenbourg and Schribak to study living

cells and other objects with small retardance [39]. Note that the proposed method works

well despite the presence of singularities at points ∆ = 0 and ∆ = π.

79 E 90 Figure 4.6: φ−averaged standard deviations δ∆M , δφM , δ∆B, δφB vs. ∆ for sets A , A ,

11 T π A , A . Inset shows the same plots for small values of ∆ ∈ [0, 180 ].

4.4 Noise contamination of the measurement process

To illustrate how the selected control parameters affect the measuring accuracy, we numerically simulate the effect of the artificially added noise to the typical PolScope file of a liquid crystal sample, similar to ones presented in [53]. The file contains 2D fields of retardation ∆∗(x, y), 4.7a, and optical axis orientation φ∗(x, y), 4.8a, which we consider as

∗ the error-free 2D ﬁeld of quantities Λ (x, y), assuming that Iin = 1 and Ileak = 0. Then for the selected A we perform the following steps for each pixel (x, y):

(1) We calculate the error-free vector of measured values b∗ = M(A)V (Λ∗).

80 (2) To simulate eﬀect from errors in control parameters, we contaminate data with ar-

tiﬁcial noise by adding random, uniformly distributed values δA = {δαkj}, k = 1, ..., 4, j = h 3π 3π i 1, 2, α ∼ unif − , to the set A. kj 180 180

(3) We determine the vector V (Λ) = [ν1, ν2, ν3, ν4] from the equation M(A + δA)V (Λ) =

∗ b via LU decomposition [50] of the matrix M and calculate the retardation ∆M = ν2 ν3 arccos and the orientation of the slow optical axis φ = 1 arccot p 2 2 2 M 2 ν ν2 + ν3 + ν4 4 contaminated with noise in the control parameters.

(4) We introduce the relative noise of measuring values as δb = Ξb∗ is a diagonal ma- h 3 3 i trix with identically distributed random elements Ξ ∼ unif − , . ii 100 100

ν2 (5) We calculate the retardation ∆ = arccos and . orientation of B p 2 2 2 ν2 + ν3 + ν4 1 ν 3 ∗ the slow optical axis φB = arccot from the equation M(A)V (Λ) = b + δb similar 2 ν4 to step (3).

81 Figure 4.7: Eﬀect of noise contamination on retardation: a) original (noise-free) image

∗ B ∆ (x, y); b) image with noise in control parameters ∆M (x, y) when A = A ; c) ∆M (x, y)

when A = A90; d) image with relative noise in measured values A = AB when A = AB; e)

90 ∆B(x, y) when A = A . The grayscale bar represents values of the optical retardation.

Figure 4.7 exhibits the eﬀect of control parameters errors on the retardation for the sets AB and A90 after applying steps (1)-(3) for each pixel. One can see that in comparison with the original error-free image ∆∗(x, y), Figure 4.7a, the measurement process at

B A = A has a substantially smaller image degradation ∆M (x, y), Figure 4.7b, than when

A = A90, Figure 4.7c. On the other hand, after steps (4)-(5) images with relative mea-

B 90 suring errors ∆Bx, y show similar weak degradation for both sets A and A , Figure 4.7d

82 and Figure 4.7e, respectively. To explain the observed image degradation, we computed

∗ ∗ sample standard deviations of the errors δ∆M,B = ∆M,B − ∆M,B, δ∆M,B = ∆M,B − ∆M,B, s 1 X sM,B = δ∆2 (x, y), that equal, respectively, sM = 0.09, sB = 0.024 for ∆ N − 1 M,B ∆ ∆ x,y 90 M B B M A = A and s∆ = 0.042, s∆ = 0.023 for A = A . As we can see, s∆ is signiﬁcantly

B 90 B smaller for A than for A = A , while s∆ is almost the same for both sets.

Figure 4.8: Eﬀect of noise contamination on retardation: a) original (noise-free) image

∗ B φ (x, y); b) image with noise in control parameters φM (x, y) when A = A ; c) φM (x, y)

when A = A90; d) image with relative noise in measured values A = AB when A = AB; e)

90 φB(x, y) when A = A . The grayscale bar represents values of the optical retardation.

83 The eﬀect of noise contamination on the optical slow axis orientation data is shown

B in Figure 4.8. The choice of the set A leads to smaller image degradation φM (x, y),

see Figure 4.8b, of the error-free image φ∗(x, y), Figure 4.8a, than the set A90, 4.8c.

At the same time images with relative noise in the measured values φB(x, y) exhibit a

similar slight degree of degradation for the sets AB and A90, Figures 4.8d and 4.8e, re-

spectively. The values of sample standard deviations of the slow axis orientation errors, s 1 X sM,B = δφ2 (x, y), where δφ (x, y) = φ (x, y) − φ∗(x, y), characterize φ N − 1 M,B M,B M,B x,y M B the visual degree of image degradation shown on Figure 4.8, sφ = 0.11, sφ = 0.029 for

90 M B B A = A and s∆ = 0.051, s∆ = 0.027 for A = A . One can see that the relations between

M M B B B 90 the sample standard deviations s∆ , sφ , s∆, sφ for both sets A , A match well the re-

lations between the averaged standard deviations δ∆M , δφM , δ∆B, δφB, shown in Figures

4.6a-d.

4.5 Chapter summary

In this chapter we investigated the conditioning of CMPM systems that determine several unknown quantities by successively measuring a single physical variable under diﬀerent experimental conditions, deﬁned by the control parameters. The errors of the determined quantities are caused by the measurement errors and the errors in the setting of the control parameters. One of the main problems in CMPM systems is the determination of a suitable set of control parameters that provides the best possible accuracy for the entire range of the unknown quantities. We analytically derived the method of the measurement noise reduction based on conditioning optimization, which is applicable to real-world measurement problem and substantiated through numerical experiments. As an error function, we consider the mathematical expectation of the length of the dimensionless, scaled vector of errors of the unknown quantities (4.7). When the measuring function has separable

84 form (4.2), the error function splits into two independent functions, describing respectively

the effects of the measurement errors and the control parameters errors, (4.11). Using submultiplicativity of the spectral and Frobenius matrix norms, we represent these two functions as products of factors that dependent only on control parameters and factors that dependent only on unknown quantities. Substituting the factors that depend only on the unknown quantities with a weighting coefficient, we construct the effective error function (4.26), which is the upper bound of the true error function (4.11). We determine an optimal set of control parameters by minimizing (4.26). To demonstrate the capability of our method, we apply it to the PolScope polarized light microscope. In the PolScope,

2D distributions of optical retardation and optical slow axis orientation are determined from four measurements of the light intensity coming through the optical scheme and controlled by variable retarders. We have found that for the PolScope, our method provides almost the same set of control parameters both when minimizing the control parameter error and when minimizing the measuring error, so the optimization of (4.26) is essentially independent on the weighting coeﬃcient. We compare the computed optimal set of control parameters with other sets including those used in the PolScope and demonstrate that our computed set works very well for the entire range of determined quantities. The proposed method is applicable to any error distributions of the control parameters and of the measured values, and can be used for optimization of various CMPM systems, in particular, for latest “polarized light microscopy” techniques, Exicor and Polaviz.

85 CHAPTER 5

Conclusion and future work

5.1 Conclusion

In this work several aspects of conditioning optimization are presented. Firstly, we analyzed the conditioning of rectangular Vandermonde and Vandermonde-like matrices.

Vandermonde matrices (1.6) are known to be highly ill-conditioned when the nodes are real. One approach to reduce ill-conditioning is based on using a basis of orthogonal polynomials. The matrices so obtained are commonly referred to as Vandermonde-like.

Gautschi analyzed optimally conditioned and optimally scaled square Vandermonde and

Vandermonde-like matrices with real nodes. In Chapter 2 Gautschi’s analysis is extended to rectangular Vandermonde-like matrices with real nodes, as well as to Vandermonde-like matrices with nodes on the unit circle in the complex plane. There we generalize classical

Posse theorem for Gauss and Gauss–Szeg˝oquadrature rules. Using these results existence and uniquenceness of optimally conditioned Vandermonde-like matrices is analyzed. In the ﬁnal part of Chapter 2 properties of rectangular general Vandermonde-type matrices with Chebyshev nodes or with equidistant nodes on the unit circle in the complex plane are discussed. We show that condition number of these matrices is independent of the numbers of nodes.

In Chapter 3 an explicit QR and QR-like factorization for rectangular Vandermonde matrices with Chebyshev nodes are presented. The use of Chebyshev nodes is known to reduce the conditioning of Vandermonde matrices. Based on QR and QR-like factorizations two new fast methods for solving least squares problem for Vandermonde matrices with

86 Chebyshev nodes are derived. Both methods are compared to state-to-art factorization method derived by Eisinberg et al. in [13], showing more than double acceleration in computational speed.

Chapter 4 discuss conditioning analysis of the measurement problem. There we investigate combined multi-measuring systems that determine several unknown quantities from measurements of a single variable at diﬀerent preprogrammed conditions determined by control parameters. Such measurements are described by nonlinear systems of equations where perturbations are present simultaneously in both the control parameters and measured data. The errors in the measured quantities are caused by measurement errors and errors in the setting of the control parameters. To provide better accuracy for the entire range of the unknown quantities, a model of conditioning of combined multi-measuring systems is derived analytically. The set of control parameters is determined by optimizing the conditioning. To demonstrate the capability of the proposed method, we apply it to the polarized light microscopy technique called LC-PolScope. We compare the computed optimal set of control parameters with other sets including those used in the PolScope and demonstrate that our computed set works very well for the entire range of determined quantities. We believe that the described method can be applied to a wide range of measurement systems.

5.2 Future work

Here we describe several problems that arise from the current work and are left unex- plored.

For general Vandermonde-like matrices, described in Chapter 2, the problem of computing optimal set of nodes that minimizes the condition number remains unsolved. Our numerical experiments suggest that the set of optimal nodes minimizing the condition

87 number is always unique. However, the proof of this conjecture is challenging and left open.

Explicit QR factorization of Vandermonde matrices with Chebyshev zeros, described in Chapter 3, may be extended for zeros of any orthogonal polynomials.

For measurement problem, the dependence of conditioning from number of measurements remains unsolved. Investigating this problem would be useful for designing measurement algorithms.

88 Bibliography

[1] G. S. Ammar, L. Reichel, and D. C. Sorensen, Algorithm 730: An implementation

of a divide and conquer algorithm for the unitary eigenproblem, ACM Trans. Math.

Software, 18 (1992), pp. 292–307, and 20 (1994), p. 161.

[2] F. S. V. Baz´an, Conditioning of rectangular Vandermonde matrices with nodes in the

unit disk, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 679–693.

[3] B. Beckermann, The condition number of real Vandermonde, Krylov, and positive

deﬁnite Hankel matrices, Numer. Math., 85 (2000), pp. 553–577.

[4] S. N. Bernstein, Quelques remarques sur l’interpolation, Math. Annal. 79 (1919), pp.

1–12.

[5] A. Bjerhammar, Theory of errors and generalized matrix inverses, Elsevier, Amster-

dam, New York, 1973.

[6] I. Bogaert, Iteration-free computation of Gauss-Legendre quadrature nodes and

weights, SIAM J. Sci. Comput., 36, 2014, pp. A1008–A1026.

[7] C. Bracciali, X. Li, and A. Sri Ranga, Real orthogonal polynomials in frequency anal-

ysis, Math. Comp., 74 (2004), pp. 341–362.

[8] C. Bracciali, A. Sri Ranga, and A. Swaminathan, Para-orthogonal polynomials on

the unit circle satisfying three term recurrence formulas, Applied Numer. Math., 109

(2016), pp. 19–40.

89 [9] W. J. Cody, A survey of practical rational and polynomial approximation of functions,

SIAM Rev., 12, 1970, pp. 400–423.

[10] E. Collett, Field guide to polarization, SPIE Press, Bellingham, Wash., 2005.

[11] P. Delsarte and Y. Genin, The split Levinson algorithm, IEEE Trans. Acoust. Speech

Signal Process., 34 (1986) pp. 470–478.

[12] P. Delsarte and Y. Genin, The tridiagonal approach to Szeg˝o’sorthogonal polynomials,

Toeplitz linear system, and related interpolation problems, SIAM J. Math. Anal., 19

(1988) pp. 718–735.

[13] A. Eisinberg, G. Franz´e, and N. Salerno, Rectangular Vandermonde matrices on

Chebyshev nodes, Linear Algebra Appl., 338 (2001), pp. 27–36.

[14] https://www.hindsinstruments.com/products/exicor.

[15] G. Faber, U˝ ber die interpolatorische Darstellung stetiger Funktionen (On the Inter-

polation of Continuous Functions), Jahresber. Deutsch. Math. Verein. 23 (1914).

[16] W. Gautschi, Optimally conditioned Vandermonde matrices, Numer. Math., 24 (1975),

pp. 1–12.

[17] W. Gautschi, On inverses of Vandermonde and conﬂuent Vandermonde matrices III,

Numer. Math., 29 (1978), pp. 445–450.

[18] W. Gautschi, The condition of Vandermonde-like matrices involving orthogonal poly-

nomials, Linear Algebra Appl., 52/53 (1983), pp. 293–300.

[19] W. Gautschi and G. Inglese, Lower bounds for the condition number of Vandermonde

matrices, Numer. Math., 52 (1988), pp. 241–250.

90 [20] W. Gautschi, The interplay between classical analysis and (numerical) linear algebra

- a tribute to Gene H. Golub, Electron. Trans. Numer. Anal., 13, 2002, pp. 119-147.

[21] W. Gautschi, Orthogonal Polynomials: Approximation and Computation, Oxford Uni-

versity Press, Oxford, 2004.

[22] W. Gautschi, Numerical Analysis, 2nd ed., Birkh¨auser,Basel, 2012.

[23] W. Gautschi, Optimally scaled and optimally conditioned Vandermonde and

Vandermonde-like matrices, BIT, 51 (2011), pp. 103–125.

[24] G. H. Golub and J. H. Wilkinson, Note on the iterative reﬁnement of least squares

solution, Numer. Math., 9 (1966), pp. 139–148.

[25] P. Gonz´alez-Vera, J. C. Santos-Le´on,and O. Nj˚astad, Some results about numerical

quadrature on the unit circle, Adv. Comput. Math., 5 (1996), pp. 297–328.

[26] W. Gragg, Positive deﬁnite Toeplitz matrices, the Arnoldi process for isometric opera-

tors, and Gaussian quadrature on the unit circle, J. Comput. Appl. Math., 46 (1993),

pp. 183–198. This is a slightly edited version of a paper ﬁrst published in Russian in

Numerical Methods in Linear Algebra, ed. E. S. Nikolaev, Moscow University Press,

Moscow, 1982, pp. 16–32.

[27] W. B. Gragg, The QR algorithm for unitary Hessenberg matrices, J. Comput. Appl.

Math., 16 1986), pp. 1–8.

[28] W. B. Gragg and L. Reichel, A divide and conquer method for unitary and orthogonal

eigenproblems, Numer. Math., 57 (1990), pp. 695–718.

91 [29] U. Grenander and G. Szeg˝o, Toeplitz Forms and Their Applications, Chelsea, New

York, 1984.

[30] M. Gu, R. Guzzo, X.-B. Chi, and X.-Q. Cao, A stable divide and conquer algorithm

for the unitary eigenproblem, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 385–404.

[31] N. J. Higham, Accuracy and stability of numerical algorithms, Society for Industrial

and Applied Mathematics, Philadelphia, (2002), 2nd ed edn.

[32] R. A. Horn and C. R. Johnson, Matrix Analysis, 2nd ed., Cambridge University Press,

Cambridge, 2013.

[33] W. Jones, O. Nj˚astad, and W. J. Thron, Moment theory, orthogonal polynomials,

quadrature and continued fractions associated with the unit circle, Bull. London Math.

Soc., 21 (1989), pp. 113–152.

[34] D. P. Laurie, Computation of Gauss-type quadrature formulas, J. Comput. Appl.

Math., 127, 2001, pp. 201–217.

[35] R. C. Li, Vandermonde matrices with Chebyshev nodes, Linear Algebra Appl., 428,

2008, pp. 1803–1832.

[36] J. C. Mason and D. C. Handscomb, Chebyshev Polynomials, CRC Press, Boca Raton,

2003.

[37] S. B. Mehta, M. Shribak, and R. Oldenbourg, Polarized light imaging of birefrin-

gence and diattenuation at high resolution and high sensitivity, J Optics-Uk 15, 094007

(2013).

92 [38] P. Nevai, G´ezaFreud, orthogonal polynomials and Christoﬀel functions. A case study,

J. Approx. Theory, 48 (1986), pp. 3–167.

[39] R. Oldenbourg, Polarized light ﬁeld microscopy: an analytical method using a mi-

crolens array to simultaneously capture both conoscopic and orthoscopic views of bire-

fringent objects, J Microsc-Oxford 231, 419 (2008).

[40] R. Oldenbourg and G. Mei, J Microsc-Oxford 180, 140-147, (1995).

[41] C. Posse, Sur les quadratures, Nouv. Ann. Math., 14 (1875), pp. 49–62.

[42] http://polaviz.com.

[43] S. G. Rabinovich, Evaluating measurement accuracy: a practical approach, Springer,

New York, 2013.

[44] K. R. Rao and P. Yip, Discrete Cosine Transform, Academic Press, Boston, 1991.

[45] L. Reichel and G. S. Ammar, Fast approximation of dominant harmonics by solving

an orthogonal eigenvalue problem, in Mathematics in Signal Processing II, ed. J. G.

McWhirter, Oxford University Press, Oxford, 1990, pp. 575–591.

[46] M. Shribak and R. Oldenbourg, Techniques for fast and sensitive measurements of

two-dimensional birefringence distributions, Appl. Opt. 42, 3009, (2003).

[47] M. Stewart, An error analysis of a unitary Hessenberg QR algorithm, SIAM J. Matrix.

Anal. Appl., 28 (2006), pp. 40–67.

[48] J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, 3rd ed., Springer, New

York, 2002.

93 [49] G. Szeg˝o, Orthogonal Polynomials, 4th ed., Amer. Math. Soc., Providence, 1975.

[50] L. N. Trefethen and D. Bau III, Numerical Linear Algebra, SIAM, Philadelphia, 1997.

[51] L. N. Trefethen, Approximation Theory and Approximation Practice, SIAM, 2013.

[52] E. E. Tyrtyshnikov, How bad are Hankel matrices?, Numer. Math., 67 (1994), pp.

261–269.

[53] J. Xiang, S. V. Shiyanovskii, C. T. Imrie, and O. D. Lavrentovich, Electrooptic Re-

sponse of Chiral Nematic Liquid Crystals with Oblique Helicoidal Director, Phys. Rev.

Lett. 112, 217801 (2014).

[54] A. Zhedanov, On some classes of polynomials orthogonal on arcs of the unit circle

connected with symmetric orthogonal polynomials on an interval, J. Approx. Theory,

94 (1998), pp. 73–106.