Numerical Condition in Polynomial Equation Solving Numerical Condition in Polynomial Equation Solving Peter B¨urgisser Technische Universit¨atBerlin Meeting on Algebraic Vision 2015 Berlin, October 8, 2015 Numerical Condition in Polynomial Equation Solving General phenomena Part I: Definition of condition and general phenomena Explained for linear equation solving I Relative error k∆xk=kxk of input x causes relative error k∆yk=kyk of output y I Condition number κ(f ; x) of f at x: k∆yk=kyk . κ(f ; x) k∆xk=kxk: I Formal definition if f is differentiable: kxk κ(f ; x) := kDf (x)k kf (x)k where kDf (x)k denotes the operator norm of the derivative of f at x. Numerical Condition in Polynomial Equation Solving General phenomena Condition number as norm of derivative I Numerical computation problem (\solution map") p q f : R ! R ; x 7! y = f (x) Fix norms k k on Rp; Rq. I Condition number κ(f ; x) of f at x: k∆yk=kyk . κ(f ; x) k∆xk=kxk: I Formal definition if f is differentiable: kxk κ(f ; x) := kDf (x)k kf (x)k where kDf (x)k denotes the operator norm of the derivative of f at x. Numerical Condition in Polynomial Equation Solving General phenomena Condition number as norm of derivative I Numerical computation problem (\solution map") p q f : R ! R ; x 7! y = f (x) Fix norms k k on Rp; Rq. I Relative error k∆xk=kxk of input x causes relative error k∆yk=kyk of output y I Formal definition if f is differentiable: kxk κ(f ; x) := kDf (x)k kf (x)k where kDf (x)k denotes the operator norm of the derivative of f at x. Numerical Condition in Polynomial Equation Solving General phenomena Condition number as norm of derivative I Numerical computation problem (\solution map") p q f : R ! R ; x 7! y = f (x) Fix norms k k on Rp; Rq. I Relative error k∆xk=kxk of input x causes relative error k∆yk=kyk of output y I Condition number κ(f ; x) of f at x: k∆yk=kyk . κ(f ; x) k∆xk=kxk: Numerical Condition in Polynomial Equation Solving General phenomena Condition number as norm of derivative I Numerical computation problem (\solution map") p q f : R ! R ; x 7! y = f (x) Fix norms k k on Rp; Rq. I Relative error k∆xk=kxk of input x causes relative error k∆yk=kyk of output y I Condition number κ(f ; x) of f at x: k∆yk=kyk . κ(f ; x) k∆xk=kxk: I Formal definition if f is differentiable: kxk κ(f ; x) := kDf (x)k kf (x)k where kDf (x)k denotes the operator norm of the derivative of f at x. I Theorem. The condition number of f at A equals κ(A) := κ(f ; A) = kAk kA−1k: This is well known as \the condition number of the matrix A". I κ(A) was first introduced by A. Turing in 1948. I Warnung: a different computational problem related to A has a different condition number. I For computing the eigenvalues λ1; : : : ; λn of A have n condition numbers κ(A; λ1); : : : ; κ(A; λn) (Wilkinson). Numerical Condition in Polynomial Equation Solving General phenomena Condition number of matrix inversion I Consider matrix inversion m×m −1 f : GL(m; R) ! R ; A 7! A : We measure errors with the spectral norm. I κ(A) was first introduced by A. Turing in 1948. I Warnung: a different computational problem related to A has a different condition number. I For computing the eigenvalues λ1; : : : ; λn of A have n condition numbers κ(A; λ1); : : : ; κ(A; λn) (Wilkinson). Numerical Condition in Polynomial Equation Solving General phenomena Condition number of matrix inversion I Consider matrix inversion m×m −1 f : GL(m; R) ! R ; A 7! A : We measure errors with the spectral norm. I Theorem. The condition number of f at A equals κ(A) := κ(f ; A) = kAk kA−1k: This is well known as \the condition number of the matrix A". I Warnung: a different computational problem related to A has a different condition number. I For computing the eigenvalues λ1; : : : ; λn of A have n condition numbers κ(A; λ1); : : : ; κ(A; λn) (Wilkinson). Numerical Condition in Polynomial Equation Solving General phenomena Condition number of matrix inversion I Consider matrix inversion m×m −1 f : GL(m; R) ! R ; A 7! A : We measure errors with the spectral norm. I Theorem. The condition number of f at A equals κ(A) := κ(f ; A) = kAk kA−1k: This is well known as \the condition number of the matrix A". I κ(A) was first introduced by A. Turing in 1948. I For computing the eigenvalues λ1; : : : ; λn of A have n condition numbers κ(A; λ1); : : : ; κ(A; λn) (Wilkinson). Numerical Condition in Polynomial Equation Solving General phenomena Condition number of matrix inversion I Consider matrix inversion m×m −1 f : GL(m; R) ! R ; A 7! A : We measure errors with the spectral norm. I Theorem. The condition number of f at A equals κ(A) := κ(f ; A) = kAk kA−1k: This is well known as \the condition number of the matrix A". I κ(A) was first introduced by A. Turing in 1948. I Warnung: a different computational problem related to A has a different condition number. Numerical Condition in Polynomial Equation Solving General phenomena Condition number of matrix inversion I Consider matrix inversion m×m −1 f : GL(m; R) ! R ; A 7! A : We measure errors with the spectral norm. I Theorem. The condition number of f at A equals κ(A) := κ(f ; A) = kAk kA−1k: This is well known as \the condition number of the matrix A". I κ(A) was first introduced by A. Turing in 1948. I Warnung: a different computational problem related to A has a different condition number. I For computing the eigenvalues λ1; : : : ; λn of A have n condition numbers κ(A; λ1); : : : ; κ(A; λn) (Wilkinson). I The Eckart-Young Theorem from 1936 states that kAk κ(A) = kAk kA−1k = : dist(A; Σ) where dist either refers to operator norm or to Frobenius norm (Euclidean norm on Rn×n). I This is a prototype of a Condition Number Theorem; see Demmel. Numerical Condition in Polynomial Equation Solving General phenomena Distance to ill-posedness m×m I We call the set of singular matricesΣ ⊆ R the set of ill-posed instances for matrix inversion. Clearly, A 2 Σ , det A = 0. I This is a prototype of a Condition Number Theorem; see Demmel. Numerical Condition in Polynomial Equation Solving General phenomena Distance to ill-posedness m×m I We call the set of singular matricesΣ ⊆ R the set of ill-posed instances for matrix inversion. Clearly, A 2 Σ , det A = 0. I The Eckart-Young Theorem from 1936 states that kAk κ(A) = kAk kA−1k = : dist(A; Σ) where dist either refers to operator norm or to Frobenius norm (Euclidean norm on Rn×n). Numerical Condition in Polynomial Equation Solving General phenomena Distance to ill-posedness m×m I We call the set of singular matricesΣ ⊆ R the set of ill-posed instances for matrix inversion. Clearly, A 2 Σ , det A = 0. I The Eckart-Young Theorem from 1936 states that kAk κ(A) = kAk kA−1k = : dist(A; Σ) where dist either refers to operator norm or to Frobenius norm (Euclidean norm on Rn×n). I This is a prototype of a Condition Number Theorem; see Demmel. I conjugate gradient method for solving linear equations (Hestenes and Stiefel) I interior point methods for linear optimization (Renegar) I Newton homotopy methods to solve systems of polynomial equations (Shub and Smale) Less obvious, but true: Even when assuming infinite precision arithmetic, the condition of an input often dominates the running time of iterative algorithms. Three important examples for this phenomenon: Numerical Condition in Polynomial Equation Solving General phenomena Role of condition numbers Obvious: Condition numbers are a crucial issue for designing \numerically stable" algorithms. I conjugate gradient method for solving linear equations (Hestenes and Stiefel) I interior point methods for linear optimization (Renegar) I Newton homotopy methods to solve systems of polynomial equations (Shub and Smale) Three important examples for this phenomenon: Numerical Condition in Polynomial Equation Solving General phenomena Role of condition numbers Obvious: Condition numbers are a crucial issue for designing \numerically stable" algorithms. Less obvious, but true: Even when assuming infinite precision arithmetic, the condition of an input often dominates the running time of iterative algorithms. I interior point methods for linear optimization (Renegar) I Newton homotopy methods to solve systems of polynomial equations (Shub and Smale) Numerical Condition in Polynomial Equation Solving General phenomena Role of condition numbers Obvious: Condition numbers are a crucial issue for designing \numerically stable" algorithms. Less obvious, but true: Even when assuming infinite precision arithmetic, the condition of an input often dominates the running time of iterative algorithms. Three important examples for this phenomenon: I conjugate gradient method for solving linear equations (Hestenes and Stiefel) I Newton homotopy methods to solve systems of polynomial equations (Shub and Smale) Numerical Condition in Polynomial Equation Solving General phenomena Role of condition numbers Obvious: Condition numbers are a crucial issue for designing \numerically stable" algorithms. Less obvious, but true: Even when assuming infinite precision arithmetic, the condition of an input often dominates the running time of iterative algorithms. Three important examples for this phenomenon: I conjugate gradient method for solving linear equations (Hestenes and Stiefel) I interior point methods for linear optimization (Renegar) Numerical Condition in Polynomial Equation Solving General phenomena Role of condition numbers Obvious: Condition numbers are a crucial issue for designing \numerically stable" algorithms.