Julia Eaton Workshop on Numerical Linear Algebra and Optimization [email protected] University of British Columbia University of Washington Tacoma Vancouver, BC Canada Variational properties of polynomial root functions and spectral max functions Abstract 1 Spectral Functions 4 Damped oscillator abscissa (see panel 3) 9 Applications 14 n×n Eigenvalue optimization problems arise in the control of continuous We say :C !R=R [ f+1g is a spectral function if it • Smooth everywhere but -2 and 2. Recall damped oscillator problem in panel 3. The characteristic and discrete time dynamical systems. The spectral abscissa (largest • depends only on the eigenvalues of its argument • Non-Lipschitz at -2 and 2 polynomial of A(x) is: p(λ, x) = λ(λ+x)+1 = λ2+xλ+1: real part of an eigenvalue) and spectral radius (largest eigenvalue • is invariant under permutations of those eigenvalues • Non-convex and abscissa graph is shown in panel 3. in modulus) are examples of functions of eigenvalues, or spectral • @h^ (2) = [−1=2; 1) = @h(2) A spectral max function ': n×n ! is a spectral fcn. of the form Consider the parameterization H : !P2, where functions, connected to these problems. A related class of functions C R • subdifferentially regular at 2 R '(A) = maxff (λ) j det(λI −A) = 0g H(x) = (λ−λ )2+h (x)(λ−λ )+h (x); λ 2 , h ; h : ! . are polynomial root functions. In 2001, Burke and Overton showed 0 1 0 2 0 C 1 2 R R where f : C ! R. The spectral abscissa and radius are spectral max 2 2 that the abscissa mapping on polynomials is subdifferentially Variational properties of spectral functions 10 Require p(λ, x) = λ +xλ+1 = (λ−λ0) +h1(x)(λ−λ0)+h2(x) so functions. (Note: f C ! R is treated as a function from R2 to R.) 2 T regular on the monic polynomials of degree n. In 2012 we extended Familiar spectral functions: abscissa α and radius ρ. h1(x) = 2λ0+x, h2(x) = λ0+xλ0+1, rH(x) = [0; 1; λ0] n×n these results to a class of max polynomial root functions which • α and ρ continuous on C , not Lipschitz continuous Set h = a ◦ H, so h(x) = a(p(λ, x)): By chain rule (panel 10), Optimizing Spectral Functions 5 includes both the polynomial abscissa and the polynomial radius. • α and ρ are not differentiable in general @h^ (x) = rH(x)T @^a(p(λ, x)): We are currently working to extend these results to the matrix setting. We saw an example of a simple abscissa minimization problem Compute regular subgradients? If subdifferentially regular... e e e where the matrix depended on a single parameter. More generally Global min of h is 2, so p(λ, 2) = λ2+2λ+1 = (λ+1)2 and (Chain Rule, Rockafellar & Wets ‘98) ^ T 2 we can consider min ( A(x)) @h(2) = rH(2) @a((λ+1) ): Let g : m ! be regular at H(x), H smooth, h=g ◦ H. CQ: the only 2 x2U R R e 2 Motivation k From 2001, @a((λ+1) ) = fz jz = −(λ+1)=2 + µ ; Re (µ ) ≤ 0g: where U ⊂C is some parameter set. How do we verify a candidate vector z 2@1g(H(x)) with rH(x)∗z =0 is z =0. If (CQ) is satisfied, then 2 2 Consider the linear continuous-time dynamical system e e D E 0 soln is optimal? We need to know about the structure of . @h(x)=rH(x)∗@g(H(x));@1h(x)=rH(x)∗@1g(H(x)): ^ T 2 (DE) y = Ay: e e e e e e Set λ0 =−1: @h(2) = [0; 1; −1] ;@a((λ+1) ) n×n where A2C . What is the long-term behavior of a solution? • Gradients can give necessary conditions, provided they exist. = fRe(−1=2−µ )j Re(µ ) ≤ 0g = [−1=2; 1) α regular at X iff X’s active evals are nonderogatory (Burke, Overton 01) 2 2 • Parameterization means a composite fcn—use a chain rule? Therefore 02int (@h^ (2)). Minimizer 2 is sharp. Decomposition 11 n Some variational analysis 6 •P : linear space of polynomials of degree 0; 1; : : : ; n Applying the polynomial radius to the family 15 Let ye be an equilibrium solution to dy=dt = f(t; y). Let h : E ! . •M n ⊂Pn: polynomials of degree n 2 R fp(λ, x)gx2R = fλ +xλ+1gx2R, • ye is Lyapunov stable if for any " > 0, 9 δ" > 0 such that The regular subdifferential of h at x is the set • Define Φ: n×n !Pn by Φ(A) = det(λI −A): e C we have ( p jy(t) − yej ≤ " whenever jy(0) − yej ≤ δ" ^ Define the max root function f : Pn ! by (jxj + x2 − 4)=2 jxj ≥ 2 @h(xe) = fz j h(x) − h(xe) ≥ hz; x−xei + o(kx−xek); 8xg R • y asymptotically stable 9γ > 0 r(p(λ, x)) = e is if it is LS and such that We say x is sharp if 9 δ; ">0 such that h(x)−h(x)≥δ kx−xk f(p) = maxff (λ) j p(λ) = 0g. 1 jxj < 2 jy(t) − y j ! 0 jy(t) − y j < γ e e e e whenever e for all x for which kx−xk<". Then ' = f ◦ Φ. Let h = r ◦ H, so h(x) = r(p(λ, x)). A global minimum is at 2. e D E A stable solution ye to (DE) is asymptotically stable if the @h^ (2) = [0; 1; −1]T ;@r((λ+1)2 ) Fact: (Burke, Overton, 2000): xe is a sharp local minimizer of h iff Properties of polynomial root functions 12 eigenvalues of A lie in the left-half plane. ^ 02int(@h(xe)). If f is continuous on its domain, = fRe(1=2−µ2) jRe(µ2) ≤ 1=2g • Define the spectral abscissa α : n×n ! by • f Mn \ dom (f) = [0; 1) C R Example reg. subgradients of the continuous on p , not generally Lipschitz α(A) = maxfRe(λ) j det(λI −A)=0g n n ^ damped oscillator abscissa: a(λ − ") = " 0 2= int(@h(2)), so minimizer is not sharp. • y α(A)<0 Then e is asympotically stable if . Set h(·)=α(A(·)): • f not differentiable in general n n • Also, the rate of decay of a soln. depends on α(A). z 2f−1=2g=@h^ (−3=5) • f is unbounded in a neighborhood of q for all q(λ)2P n M n−1 ^ ^ let q(λ)2P , then q(λ)(ελ−1) ! q(λ), Matrix Setting 16 k+1 k n×n z 2@h(−2), z 2@h(2) Next consider the discrete-time analog: y =Ay ;A2C . but root 1=" is unbounded In 2001 Burke & Overton derived many properties of a broad class The subderivative (generalizes the directional derivative) • Asymptotic stability tied to max modulus of eigenvalues of A. Restrict our attention to Mn. of spectral functions as well as the subdifferential regularity of the n×n dh(x)(v)= lim inf ( h(x+tv) − h(x))=t: • The spectral radius ρ : C ! R is given by e e e e spectral abscissa at a matrix with nonderogatory active eigenvalues. t&0;v!ve Polynomial max root history ρ(A)=maxfjλj j det(λI −A)=0g. In 2013 Grundel & Overton derived the formula for the A key subderivative & regular subdifferential relationship: • A stable soln is asymptotically stable if ρ(A)<1. Let f : C ! R be proper, convex, lsc. subdifferential of the simplest example of a derogatory, defective @^h(x)=fz j hz; vi ≤ dh(x)(v) 8 vg e e f(q)=maxff (q) j q(λ) = 0g matrix: a 3 × 3 matrix w/ one eigenvalue and two Jordan blocks. • If f =Re(·), the polynomial abscissa a. Well studied for general We have been working to extend the abscissa results to a broader Example 3 The general subdifferential (limits of regular subgradients) 7 p=Q(λ−λ )nj 2Mn (Burke & Overton ‘01) 00 0 j 1 class of spectral functions. We have shown the subdifferential (Damped oscillator) Consider the system v + xv + v = 0. 9fxνg⊆dom (h) and fzνg⊆E such that • For general f , f is studied for polynomials (λ−λ )n, regularity 0 @h(xe) = z ^ 0 regularity in the case of nonderogatory active eigenvalues for a very v 0 1 v zν 2@h(xν) 8 ν; xν !x;e h(xν)!h(xe); and zν !z not shown. (Burke, Lewis, Overton ‘05) Linearizing gives 0 = 0 , narrow class of functions which includes the abscissa but not the v −1 −x v The horizon subdifferential (captures non-Lipschitz-ness) • Done: fill the gap, and clarify process, go beyond max root fcn. p radius. Extending the methods in the 2001 paper has proven 2 1 9fxνg⊆dom (h); ftνg⊆[0; 1), fzνg⊆E; s.t. 8 ν • To do: push through to the matrix setting. The matrix (call it A(x)) has eigenvalues (−x ± x − 4)=2 @ h(x) = z difficult. As demonstrated in G & O’s 2013 paper, even the 3 × 3 p e zν 2@^h(xν) xν !x; h(xν)!h(x); tν #0 and tνzν !z ( 2 e e case is extremely challenging and does not easily lend itself to (−x + x − 4)=2 jxj≥2 Polynomial radius subdifferential 13 α (A(x)) = ^ ^ 1 1 generalization. We are currently exploring other avenues to derive Computing, h is regular at x if @h(x)=6 ;, @h(x)=@h(x) and @h(x) =@ h(x) r(p) = maxfjλj j p(λ) = 0g −x=2 jxj<2 e e e e e e variational properties of spectral functions.