Julia Eaton Workshop on Numerical Linear Algebra and Optimization [email protected] University of British Columbia University of Washington Tacoma Vancouver, BC Canada

Variational properties of polynomial root functions and spectral max functions

Abstract 1 Spectral Functions 4 Damped oscillator abscissa (see panel 3) 9 Applications 14 n×n Eigenvalue optimization problems arise in the control of continuous We say ψ :C →R=R ∪ {+∞} is a spectral function if it • Smooth everywhere but -2 and 2. Recall damped oscillator problem in panel 3. The characteristic and discrete time dynamical systems. The spectral abscissa (largest • depends only on the eigenvalues of its argument • Non-Lipschitz at -2 and 2 polynomial of A(x) is: p(λ, x) = λ(λ+x)+1 = λ2+xλ+1. real part of an eigenvalue) and spectral radius (largest eigenvalue • is invariant under permutations of those eigenvalues • Non-convex and abscissa graph is shown in panel 3. in modulus) are examples of functions of eigenvalues, or spectral • ∂hˆ (2) = [−1/2, ∞) = ∂h(2) A spectral max function ϕ: n×n → is a spectral fcn. of the form Consider the parameterization H : → P2, where functions, connected to these problems. A related class of functions C R • subdifferentially regular at 2 R ϕ(A) = max{f (λ) | det(λI −A) = 0} H(x) = (λ−λ )2+h (x)(λ−λ )+h (x), λ ∈ , h , h : → . are polynomial root functions. In 2001, Burke and Overton showed 0 1 0 2 0 C 1 2 R R where f : C → R. The spectral abscissa and radius are spectral max 2 2 that the abscissa mapping on polynomials is subdifferentially Variational properties of spectral functions 10 Require p(λ, x) = λ +xλ+1 = (λ−λ0) +h1(x)(λ−λ0)+h2(x) so functions. (Note: f C → R is treated as a function from R2 to R.) 2 T regular on the monic polynomials of degree n. In 2012 we extended Familiar spectral functions: abscissa α and radius ρ. h1(x) = 2λ0+x, h2(x) = λ0+xλ0+1, ∇H(x) = [0, 1, λ0] n×n these results to a class of max polynomial root functions which • α and ρ continuous on C , not Lipschitz continuous Set h = a ◦ H, so h(x) = a(p(λ, x)). By chain rule (panel 10), Optimizing Spectral Functions 5 includes both the polynomial abscissa and the polynomial radius. • α and ρ are not differentiable in general ∂hˆ (x) = ∇H(x)T ∂ˆa(p(λ, x)). We are currently working to extend these results to the matrix setting. We saw an example of a simple abscissa minimization problem Compute regular subgradients? If subdifferentially regular... e e e where the matrix depended on a single parameter. More generally Global min of h is 2, so p(λ, 2) = λ2+2λ+1 = (λ+1)2 and (Chain Rule, Rockafellar & Wets ‘98) ˆ T 2 we can consider min ψ( A(x)) ∂h(2) = ∇H(2) ∂a((λ+1) ). Let g : m → be regular at H(x), H smooth, h=g ◦ H. CQ: the only 2 x∈U R R e 2 Motivation k From 2001, ∂a((λ+1) ) = {z |z = −(λ+1)/2 + µ , Re (µ ) ≤ 0}. where U ⊂C is some parameter set. How do we verify a candidate vector z ∈∂∞g(H(x)) with ∇H(x)∗z =0 is z =0. If (CQ) is satisfied, then 2 2 Consider the linear continuous-time dynamical system e e D E 0 soln is optimal? We need to know about the structure of ψ. ∂h(x)=∇H(x)∗∂g(H(x)), ∂∞h(x)=∇H(x)∗∂∞g(H(x)). ˆ T 2 (DE) y = Ay. e e e e e e Set λ0 =−1: ∂h(2) = [0, 1, −1] , ∂a((λ+1) ) n×n where A∈C . What is the long-term behavior of a solution? • Gradients can give necessary conditions, provided they exist. = {Re(−1/2−µ )| Re(µ ) ≤ 0} = [−1/2, ∞) α regular at X iff X’s active evals are nonderogatory (Burke, Overton 01) 2 2 • Parameterization means a composite fcn—use a chain rule? Therefore 0∈int (∂hˆ (2)). Minimizer 2 is sharp.

Decomposition 11 n Some variational analysis 6 •P : linear space of polynomials of degree 0, 1, . . . , n Applying the polynomial radius to the family 15 Let ye be an equilibrium solution to dy/dt = f(t, y). Let h : E → . •M n ⊂Pn: polynomials of degree n 2 R {p(λ, x)}x∈R = {λ +xλ+1}x∈R, • ye is Lyapunov stable if for any ε > 0, ∃ δε > 0 such that The regular subdifferential of h at x is the set • Define Φ: n×n → Pn by Φ(A) = det(λI −A). e C we have ( √ |y(t) − ye| ≤ ε whenever |y(0) − ye| ≤ δε ˆ Define the max root function f : Pn → by (|x| + x2 − 4)/2 |x| ≥ 2 ∂h(xe) = {z | h(x) − h(xe) ≥ hz, x−xei + o(kx−xek), ∀x} R • y asymptotically stable ∃γ > 0 r(p(λ, x)) = e is if it is LS and such that We say x is sharp if ∃ δ, ε>0 such that h(x)−h(x)≥δ kx−xk f(p) = max{f (λ) | p(λ) = 0}. 1 |x| < 2 |y(t) − y | → 0 |y(t) − y | < γ e e e e whenever e for all x for which kx−xk<ε. Then ϕ = f ◦ Φ. Let h = r ◦ H, so h(x) = r(p(λ, x)). A global minimum is at 2. e D E A stable solution ye to (DE) is asymptotically stable if the ∂hˆ (2) = [0, 1, −1]T , ∂r((λ+1)2 ) Fact: (Burke, Overton, 2000): xe is a sharp local minimizer of h iff Properties of polynomial root functions 12 eigenvalues of A lie in the left-half plane. ˆ 0∈int(∂h(xe)). If f is continuous on its domain, = {Re(1/2−µ2) |Re(µ2) ≤ 1/2} • Define the spectral abscissa α : n×n → by • f Mn ∩ dom (f) = [0, ∞) C R Example reg. subgradients of the continuous on √ , not generally Lipschitz α(A) = max{Re(λ) | det(λI −A)=0} n n ˆ damped oscillator abscissa:  a(λ − ε) = ε 0 ∈/ int(∂h(2)), so minimizer is not sharp. • y α(A)<0 Then e is asympotically stable if . Set h(·)=α(A(·)): • f not differentiable in general n n • Also, the rate of decay of a soln. depends on α(A). z ∈{−1/2}=∂hˆ (−3/5) • f is unbounded in a neighborhood of q for all q(λ)∈P \M n−1 ˆ ˆ  let q(λ)∈P , then q(λ)(ελ−1) → q(λ), Matrix Setting 16 k+1 k n×n z ∈∂h(−2), z ∈∂h(2) Next consider the discrete-time analog: y =Ay ,A∈C . but root 1/ε is unbounded In 2001 Burke & Overton derived many properties of a broad class The subderivative (generalizes the directional derivative) • Asymptotic stability tied to max modulus of eigenvalues of A. Restrict our attention to Mn. of spectral functions as well as the subdifferential regularity of the n×n dh(x)(v)= lim inf ( h(x+tv) − h(x))/t. • The spectral radius ρ : C → R is given by e e e e spectral abscissa at a matrix with nonderogatory active eigenvalues. t&0,v→ve Polynomial max root history ρ(A)=max{|λ| | det(λI −A)=0}. In 2013 Grundel & Overton derived the formula for the A key subderivative & regular subdifferential relationship: • A stable soln is asymptotically stable if ρ(A)<1. Let f : C → R be proper, convex, lsc. subdifferential of the simplest example of a derogatory, defective ∂ˆh(x)={z | hz, vi ≤ dh(x)(v) ∀ v} e e f(q)=max{f (q) | q(λ) = 0} matrix: a 3 × 3 matrix w/ one eigenvalue and two Jordan blocks. • If f =Re(·), the polynomial abscissa a. Well studied for general We have been working to extend the abscissa results to a broader Example 3 The general subdifferential (limits of regular subgradients) 7 p=Q(λ−λ )nj ∈Mn (Burke & Overton ‘01) 00 0   j 1 class of spectral functions. We have shown the subdifferential (Damped oscillator) Consider the system v + xv + v = 0. ∃{xν}⊆dom (h) and {zν}⊆E such that • For general f , f is studied for polynomials (λ−λ )n, regularity  0     ∂h(xe) = z ˆ 0 regularity in the case of nonderogatory active eigenvalues for a very v 0 1 v zν ∈∂h(xν) ∀ ν, xν →x,e h(xν)→h(xe), and zν →z not shown. (Burke, Lewis, Overton ‘05) Linearizing gives 0 = 0 , narrow class of functions which includes the abscissa but not the v −1 −x v The horizon subdifferential (captures non-Lipschitz-ness) • Done: fill the gap, and clarify process, go beyond max root fcn. √   radius. Extending the methods in the 2001 paper has proven 2 ∞ ∃{xν}⊆dom (h), {tν}⊆[0, ∞), {zν}⊆E, s.t. ∀ ν • To do: push through to the matrix setting. The matrix (call it A(x)) has eigenvalues (−x ± x − 4)/2 ∂ h(x) = z difficult. As demonstrated in G & O’s 2013 paper, even the 3 × 3 √ e zν ∈∂ˆh(xν) xν →x, h(xν)→h(x), tν ↓0 and tνzν →z ( 2 e e case is extremely challenging and does not easily lend itself to (−x + x − 4)/2 |x|≥2 Polynomial radius subdifferential 13 α (A(x)) = ˆ ˆ ∞ ∞ generalization. We are currently exploring other avenues to derive Computing, h is regular at x if ∂h(x)=6 ∅, ∂h(x)=∂h(x) and ∂h(x) =∂ h(x) r(p) = max{|λ| | p(λ) = 0} −x/2 |x|<2 e e e e e e variational properties of spectral functions. (All convex functions are regular on their domains.) 2001: for λ0 =6 0,  Pn n−s  What value of x minimizes α(A(x))? n z = s=1 µs(λ − λ0) , µ1 =−λ0/(n |λ0|) ∂r((λ − λ0) )= z 2 Negative absolute value: y = −|x|. 8 Re(λ0 µ2) ≤ |λ0| /n It’s easy: arg min α ( A(x) ) = 2 References Qm n Grundel, Overton. Var. Analy. of Spect. Absc. at a Matrix w/ Nongeneric Mult. Eval. Set-Val. & Var. Analy. 2013. • piecewise linear, non-diff at 0 2012: For p = (λ − λj) j , λ1, . . . , λm distinct, x∈R j=1 Burke, Eaton. On the subdifferential regularity of polynomial max root functions. J. Nonlinear Analysis 2012.  m n  ˆ P Q nk P j nj−s Burke, Lewis, Overton. Variational Analysis of functions of the roots of polynomials. Mathematical Prog., 2005. • ∂(−| · |)(0) = ∅ z = j=1 k6=j(λ − λk) s=1 µjs(λ − λj)   J. G Opt. P Burke, Lewis, Overton. Variational Analysis of Abscissa Mapping...via the Gauss-Lucas Theorem. , 2004. • ∂(−| · |)(0) = {−1, 1} ∂r(p)= z ∃{γj}j∈I(p) ⊂ [0, 1] with γj =1 such that Burke, Overton. Variational Analysis of the Abscissa Mapping for Polynomials. SIAM J. on Control & Opt., 2001.  2  Burke, Overton. Variational Analysis of non-Lipschitz spectral functions. Mathematical Programming, 2001. • not subdifferentially regular  µj1 =−γjλj/(nj |λj|) and Re(λj µj2)≤γj |λj| /nj ∀ j ∈ I(p)  Burke, Lewis, Overton. Optimizing Matrix Stability. AMS Proceedings, 2000.