Numerische Mathematik Numerische https://doi.org/10.1007/s00211-021-01215-6 Mathematik

Fiber product homotopy method for multiparameter eigenvalue problems

Jose Israel Rodriguez1 · Jin-Hong Du2 · Yiling You3 · Lek-Heng Lim4 Author Proof

Received: 30 June 2018 / Revised: 14 May 2021 / Accepted: 24 May 2021 © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2021

1 Abstract 2 We develop a new homotopy method for solving multiparameter eigenvalue problems 3 (MEPs) called the fiber product homotopy method. For a k-parameter eigenvalue 4 problem with matrices of sizes n1,...,nk O(n), fiber product homotopy method = 5 requires deformation of O(1) linear equations, while existing homotopy methods for 6 MEPs require O(n) nonlinear equations. We show that the fiber product homotopy 7 method theoretically finds all eigenpairs of an MEP with probability one. It is especially 8 well-suited for a class of problems we call dimension-deficient singular problems that 9 are generic with respect to intrinsic dimension, as the fiber product homotopy method 10 is provably convergent with probability one for such problems as well, a fact borne 11 out by numerical experiments. More generally, our numerical experiments indicate 12 that the fiber product homotopy method significantly outperforms the standard Delta 16 13 method in terms of accuracy, with consistent backward errors on the order of 10− 14 without any use of extended precision. In terms of speed, it significantly outperforms 15 previous homotopy-based methods on all problems and outperforms the Delta method

The work in this article is generously supported by DARPA D15AP00109, HR00112190040, and NSF IIS 1546413, DMS 1854831. LHL has also received support from the Eckhardt Faculty Fund.

B Jose Israel Rodriguez [email protected] Jin-Hong Du [email protected] Yiling You [email protected] Lek-Heng Lim [email protected]

1 Department of Mathematics, University of Wisconsin, Madison, WI, USA 2 Department of Statistics, University of Chicago, Chicago, IL, USA 3 Department of Mathematics,uncorrected University of California, Berkeley, CA, USA proof 4 Computational and Applied Mathematics Initiative, University of Chicago, Chicago, IL, USA 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

1 16 on larger problems, and is also highly parallelizable. We show that the fiber product 17 MEP that we solve in the fiber product homotopy method, although mathematically 2 18 equivalent to a standard MEP, is typically a much better conditioned problem.

19 Mathematics Subject Classification 65H20 65H17 65H10 35P30 · · ·

20 1 Introduction Author Proof 21 A multiparameter eigenvalue problem (MEP) is, in an appropriate sense, a system of 22 linear equations

23 a11x1 a12x2 a1k xk b1, + +···+ = 24 a21x1 a22x2 a2k xk b2, + +···+ = . . . . 25 . . . .

26 ak1x1 ak2x2 akkxk bk, (1) + +···+ =

27 where the coefficients aij’s and bi ’s are matrices, and where equality is interpreted 28 to mean on a point in a product of projective spaces (this will be made precise later). 29 These coefficients are square matrices but are of different dimensions in general, so 30 one may not usually regard (1) as a linear system over a matrix ring. There is a rich 31 mathematical theory behind MEP [2,3] that places it at the crossroad of linear and 32 multilinear algebra, ordinary and partial differential equations, spectral theory and 33 Sturm–Liouville theory, among other areas. The problem appeared as early as 1836 in 34 the works of Sturm and Liouville on periodic heat flow in a bar, and was studied over 35 the years by many: Klein, Lamé, Heine, Stieltjes, Pell, Carmichael, Bocher, Hilbert 36 among them (see [2, Preface] and [3, Chapter 1]). 37 An MEP encompasses many known types of eigenvalue problems: Standard eigen- 38 value problems Ax λx; generalized eigenvalue problems Ax λBx; quadratic = 2 = 39 eigenvalue problems (λ A λB C)x 0; polynomial eigenvalue problems m m 1 + + = 40 (λ Am λ − Am 1 A0)x 0; quadratic two-parameter eigenvalue problems + − +···+ =

2 2 41 (A00 λA10 µA01 λ A20 λµA11 µ A02)x1 0, + + + + + = 2 2 42 (B00 λB10 µB01 λ B20 λµB11 µ B02)x2 0 + + + + + = ;

43 may all be reduced to mathematically equivalent MEPs. 44 Nevertheless MEP remains in the blind spot of most modern mathematicians, 45 whether pure or applied. This is not for its lack of applications; as we pointed out, 46 the problem in fact originated from a study of heat flow, and we will see yet other 47 applications of MEP in Sect. 7 and that it contains eigenvalue problem and linear 48 system, both ubiquitous in science and engineering, as special cases. We think that a 49 main reason for the obscurityuncorrected of MEPs is that there are not many effective proof methods 50 for its computation and there is thus little to be gained from formulating a problem 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

51 as an MEP. It is with this in mind that we propose a new homotopy method based on 52 what we call fiber product homotopy for computing MEP solutions. 53 We will now formally define an MEP in more conventional notations. Instead of 54 having a single eigenvalue parameter λ, an MEP has multiple eigenvalue parameters 55 λ (λ1,...,λk). We will call =

56 H(λ) A0 λ1 A1 λk Ak, := − −···−

57 a linear polynomial matrix in k parameters λ1,...,λk with matrix coefficients Author Proof n n n 58 A0,...,Ak C . We will write P for the complex projective n-space. ∈ × n n 59 Definition 1 For a fixed k 2 and given matrices Aij C i i with j 0, 1,...,k, ≥ ∈ × = 60 i 1,...,k, consider the linear polynomial matrices =

61 Hi (λ) Ai0 λ1 Ai1 λ2 Ai2 λk Aik. := − − −···−

62 The multiparameter eigenvalue problem (MEP), or, more precisely, a k-parameter 63 eigenvalue problem, is to find λ1,...,λk C and corresponding (x1,...,xk) n 1 n 1 ∈ ∈ 64 P 1 P k such that − ×···× −

65 Hi (λ)xi 0, i 1,...,k. (2) = =

k n1 1 nk 1 66 A solution (λ1,...,λk, x1,...,xk) C P − P − to the MEP is called ∈ ×n1 1 ×···×nk 1 67 an eigenpair,thek-tuple (x1,...,xk) P − P − an eigenvector, and the k ∈ ×···× 68 k-tuple λ (λ1,...,λk) C an eigenvalue. = ∈ 69 Written out in full, (2) takes the form

70 (λ1 A11 λ2 A12 λk A1k)x1 A10 x1, + +···+ = 71 (λ1 A21 λ2 A22 λk A2k)x2 A20 x2, + +···+ = . . . . 72 . . . .

73 (λ1 Ak1 λ2 Ak2 λk Akk)xk Ak0 xk. (3) + +···+ =

74 With λi ’s playing the role of xi ’s, Aij’s and Ai0’s playing the roles of aij’s and bi ’s 75 respectively in (1), and interpreting equality of the ith equation in (1) to mean equality n 1 76 on some xi P i , we may view (3) as an analogue of a linear system that we ∈ − 77 referred to at the beginning. The analogy is precise when n1 nk 1—(3)is =···= = 78 a linear system in the usual sense. 79 When k 1, (3) is a generalized eigenvalue problem. More generally, if Aij 0 = = 80 for all i j and j 0, then (3) is decoupled into k generalized eigenvalue problems. = = 81 Hence (3) contains both eigenvalue problems and linear systems as special cases. The 82 multiparameter eigenvalue problem is well studied and readers may refer to the books 83 [2,3,23] for a comprehensive treatment. 84 Since any scalar multiple of xi is also an eigenvector it is fitting to consider xi as an 85 Pni 1 element of the projective spaceuncorrected− although for practical reason oneproof might prefer 86 to simply normalize xi to have unit norm. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

87 We develop a new homotopy method to solve a multiparameter eigenvalue problem 88 effectively, where effectiveness is measured by the following factors:

89 – Speed as measured by wall time. We record time per path, maximum time over 90 all paths, and total track time of all paths. Our algorithm is highly parallelizable 91 and the per-path times give good speed estimates when there are enough cores to 92 track all paths in parallel. 93 – Accuracy as measured by the backward error. We use the normwise backward error 94 in [14, Theorem 2] for an approximate eigenpair. Our homotopy method tracks

Author Proof 95 several copies of the eigenvalue λ; they should all converge to the same value if 96 our method performs correctly and we include the difference between copies of 97 λ’s as another measure of accuracy. 98 – Certificates of quadratic convergence in terms of Shub–Smale α-theory. 99 – Number of divergent paths that fail to converge to the solutions.

100 The last two measures only apply to methods based on homotopy continuation. We 101 will compare our method to two existing methods:

102 (i) The Delta method [2], which is the de facto standard method for solving MEPs 103 by transforming them into a coupled system of generalized eigenvalue problems; 104 we use the MultiParEig package [17] in our experiments with this method. For 105 singular MEPs, we perform Delta method after extracting the common regular 106 parts of the Delta matrices with a staircase algorithm [17]. 107 (ii) The diagonal coefficient homotopy method recently proposed in [8] for solving 108 MEPs, where the start system is a random choice of diagonal matrices and the 109 homotopy is a straight-line homotopy that deforms n1 nk of n1 nk k +···+ +···+ + 110 equations.

111 The numerical experiments in [8] show that the diagonal coefficient homotopy method 112 outperforms the Delta method in terms of memory usage and is also faster for large 113 n1,...,nk. Both methods find all eigenpairs of an MEP. 114 Our fiber product homotopy method adopts an alternative approach—we solve an 115 MEP (2) by solving a mathematically equivalent system that we will call the fiber 1 116 product multiparameter eigenvalue problem:

117 Hi (λi )xi 0, λ1 λ2 λk, i 1,...,k, (4) = = =···= =

118 where λ1,...,λk are to be regarded as different copies of λ. Our corresponding homo- 119 topy has a start system that captures more structure of the MEP and allows us to deform 120 far fewer equations. By introducing k(k 1) auxiliary unknowns, we deform at most 2 − 121 k(k 1) of the n1 nk k equations. For a fixed k and n1,...,nk O(n), fiber − +···+ + = 122 product homotopy deforms O(1) equations whereas diagonal coefficient homotopy 123 deforms O(n) equations. Furthermore, fiber product homotopy deforms only linear 124 equations whereas diagonal coefficient homotopy deforms nonlinear equations, which 125 means that the paths of the fiber product homotopy are easier to track. As we will see 126 later, for dimension-deficient and singular MEPs, the number of paths to track in a

1 k2 More precisely, we solve Hi (λuncorrectedi )xi 0, G1 Gk 0, i 1,...,k, where proof each Gi C k 1 λ λ = = ··· = = = : → C − is a linear function of 1,..., k , chosen so that the resulting system is equivalent to (4). See Sect. 5.2. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

127 fiber product homotopy can be substantially smaller than using a diagonal coefficient 128 homotopy. 129 An eigenpair (λ, x1,...,xk) of an MEP is said to be regular if the eigenvalue λ is 130 isolated and has multiplicity one (see [14] for definitions of algebraic and geometric 131 multiplicity). Since the expected number of regular eigenpairs to an MEP is n1 nk, ··· 132 one often considers only relatively small values of k to n when finding all eigenpairs. 133 Our fiber product homotopy method is guaranteed to compute all regular eigenpairs 134 in theory—we show in Theorem 1 that our start system in Sect. 5 is chosen correctly 135 with probability one. Author Proof 136 The fiber product homotopy method is motivated by several geometric insights. In 137 Sect. 3, we define two algebraic varieties associated with a fiber product MEP: mul- 138 tiparameter eigenvalue variety and multiparameter eigenpair variety. Fiber products, 139 a notion well-known in areas from algebraic geometry to relational database, will be 140 reviewed in Sect. 4. The name for our method comes from the fact that a k-parameter 141 eigenpair variety is a fiber product of k one-parameter eigenpair varieties. In Sect. 11, 142 we rely on geometry to define a condition number for the fiber product MEP (4), 143 which differs from the condition number for the standard MEP (3), these being two 144 different problems, albeit having the same solutions. Our condition number arises 145 from intersecting an algebraic variety (our multiparameter eigenpair variety) with a 146 varying linear space (defined by our start system). 147 We implemented a purely numerical version (in particular, it does not use multi- 148 precision) of fiber product homotopy in Matlab and a mixed symbolic-numerical 149 version in Bertini with Macaulay2 for comparison. We did extensive numerical 150 experiments with both implementations: randomly generated MEPs in Sect. 8;the 151 Mathieu two-parameter eigenvalue problem arising from a real-world application—an 152 elliptic membrane vibration problem—in Sect. 9; and MEPs that are both dimension- 153 deficient and singular, a challenging class of problems with a deficiency in the number 154 of eigenpairs that breaks most other methods, in Sect. 10. 155 We applied a broad range of measures for speed and accuracy to our numerical 156 experiments to stress test the robustness of our method. In Sect. 8, speed is measured 157 via both wall time and iteration count (number of Newton steps); accuracy is measured 158 via both relative backward error of the computed eigenpairs and the deviation in the 159 multiple paths used to track the eigenvalues. In Sect. 9, we test the effect of reducing the 160 number of Newton steps (by early stopping) on the accuracy of our method and certify 161 its final quadratic convergence speed using Shub–Smale α-theory. For the dimension- 162 deficient singular MEPs in Sect. 10, what breaks other homotopy methods is the 163 issue of divergent paths and so we use the number of divergent paths as a measure 164 of effectiveness. The take-away is that our method saw zero divergent paths in the k 165 cases we tested, where the MEP has exactly i 1 di eigenvalues with (d1,...,dk) = 166 the intrinsic dimension. In Sects. 8 and 10, we also provide the time it takes to track a  167 single path, which gives a good estimate of the speed under parallel execution of our 168 method (since each path can be tracked independently of others). uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

169 2 Homotopy methods

170 We recall the straight-line homotopy and describe the diagonal coefficient homotopy 171 method used in [8] but will defer the description of our fiber product homotopy method 172 to Sect. 5. 173 A homotopy deforms solutions of a start system Q(z) 0 to solutions of a target = 174 system P(z) 0. More precisely, a straight-line homotopy with path parameter t is = 175 defined as 176 H(z, t) (1 t) Q(z) t P(z), t 0, 1 . (5) Author Proof := − + ∈[ ] 177 When t 0, H(z, 0) Q(z) 0 is the start system and when t 1, H(z, 1) = = = = = 178 P(z) 0 is the target system. = 179 Definition 2 A start system for the homotopy (5)issaidtobechosen correctly [15]if 180 the following properties hold: 181 (i) the solution set of the start system Q(z) 0 are known or easy to obtain; = 182 (ii) the solution set of H(z, t) 0for0 t < 1 consists of a finite number of = ≤ 183 smooth paths, each parametrized by t in 0, 1); [ 184 (iii) for each isolated solution of the target system P(z) 0, there is some path = 185 originating from a solution of the start system that leads to it. ni ni 186 Let Dij denote diagonal matrices in C × .Thediagonal coefficient homotopy 187 method for solving MEP is the straight-line homotopy given by:

188 Hdc(λ, x1,...,xk, t) (1 t) Q t Pdc, := − dc + (D10 λ1 D11)x1 + . 189 Q (λ, x1,...,xk) . , dc := ⎡ . ⎤ (Dk0 λk Dkk)xk ⎢ + ⎥ ⎣ ⎦ H1(λ)x1 . 190 P (λ, x1,...,xk) . (6) dc := ⎡ . ⎤ H (λ)x ⎢ k k⎥ ⎣ ⎦ 191 where one regards

k n1 1 nk 1 n1 nk 192 H C (P − P − ) C C +···+ , dc : × ×···× × → k n1 1 nk 1 n1 nk 193 Q , Pdc C (P − P − ) C +···+ dc : × ×···× →

194 as polynomial maps. The homotopy method proposed in [8] is an example of a diagonal 195 coefficient homotopy method. 196 To implement the homotopy above, one has to account for the scaling of the eigen- 197 vectors by introducing k constraints. One way to do this is by scaling each xi so that 198 it has norm one. This is the approach undertaken in [8]. Another way to do this is to 2 199 place a generic affine constraint on each xi , which is what we will do in our fiber 200 product homotopy in Sect. 5.

2 uncorrected proof Here “generic” is used in its usual sense in algebraic geometry. Those unfamiliar with this notion may assume that it is synonymous with “random.” 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

201 3 Multiparameter eigenvarieties

202 We will define two algebraic varieties associated with an MEP.

203 Definition 3 In algebraic geometric terms [10], the coordinates of H(λ)x are poly- 204 nomials that form a subset of C λ, x C λ1,...,λk, x1,...,xn and define an [ ]= [ ] 205 algebraic variety

k n 1 206 EP(H) (λ, x) C P − H(λ)x 0 .

Author Proof := { ∈ × : = }

207 We will call this the eigenpair variety of H. In the context of an MEP, we will call the 208 Cartesian product EP(H1,...,Hk) EP(H1) EP(Hk) the multiparameter := ×···× 209 eigenpair variety of H1,...,Hk. Explicitly,

2 k n1 1 nk 1 210 EP(H1,...,Hk) (λ1,...λk, x1,...,xk) C P − P − ={ ∈ × ×···× : 211 H1(λ1)x1 0,...,Hk(λk)xk 0 . (7) = = }

212 The multiparameter eigenvalue variety of H1,...,Hk is the coordinatewise projection k2 213 of the multiparameter eigenpair variety to C and will be denoted by EV H1,...,Hk . 214 Alternatively, it can be defined explicitly as

k2 215 EV(H1,...,Hk) (λ1,...λk) C ={ ∈ : 216 det H1(λ1) 0,...,det Hk(λk) 0 . (8) = = }

217 Throughout our article, N will denote the set of positive integers. k 218 Definition 4 AnMEPissaidtohaveintrinsic dimension (d1,...,dk) N if the total ∈ 219 degree of the polynomial det Hi (λi ) is di , i 1,...,k. Such an MEP is said to be = 220 generic with respect to intrinsic dimension if the hypersurface defined by det Hi (λi ) 3 221 is generically reduced for i 1,...,k. = 222 As di ’s are required to be positive, none of the det Hi (λi )’s are the zero poly- 223 nomial. By Bezout’s theorem, the degree of the multiparameter eigenvalue variety 2 k 224 EV k (H1,...,Hk) C is at most i 1 di . So the number of isolated regular points ⊆ = 225 in the intersection of EV(H1,...,Hk) with a codimension-k(k 1) affine linear space k2 k  − 226 in C is at most i 1 di . We state this formally below. = k 227 Proposition 1 An MEP with intrinsic dimension (d1,...,dk) has at most i 1 di = 228 regular eigenvalues and eigenpairs. This bound is tight if the MEP is generic with  229 respect to intrinsic dimension.

230 The next observation will be a key to our fiber product homotopy method.

231 Lemma 1 Let D denote the linear space defined by

2 k n1 1 nk 1 232 D (λ1,...λk, x1,...,xk) C P − P − λ1 λk . := { uncorrected∈ × ×···× : proof=···= } 3 This is an algebraic geometry term that implies the polynomial det Hi (λi ) is square-free. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

233 Then the intersection D EP(H1,...,Hk) gives the set of eigenpairs of (2). ∩ k 234 In fact, D EP(H1,...,Hk) is the fiber product of EP(H1),...,EP(Hk) over C , ∩ 235 a standard notion in algebraic geometry [10]. This is the impetus for the name of our 236 homotopy method—fiber product homotopy.

237 4 Fiber products

Author Proof 238 Knowledge of the fiber product’s formal properties at the level of, say, [10] is unnec- 239 essary for us. All we need is the notion of fiber product of sets—an important and 240 well-known concept in relational database theory [16, Section 6.3]. 241 Let X, Y , A be sets and ϕ X A and ψ Y A be maps. : → : →

Y ψ XA 242 ϕ

243 The fiber product X A Y of X and Y over A is the subset of X Y given by × ×

244 X A Y (x, y) X Y ϕ(x) ψ(y) . × := { ∈ × : = }

245 Note that a fiber product depends on the maps ϕ and ψ although this is not reflected 246 in the notation X A Y , which is nevertheless standard. The fiber product satisfies the × 247 following commutative diagram where π1(x, y) x and π2(x, y) y are projection = = 248 maps:

X A YY × π2 π1 ψ XA 249 ϕ

250 We will illustrate fiber products with a few examples.

251 Example 1 (Relational database) Let X 1, 2, 3, 4 , Y a, b, c, d, e , and A ={ } ={ } = 252 1, 1 . Let the maps ϕ X A and ψ Y A be given by {− + } : → : → 1 x is odd, 1 y is a vowel, 253 ϕ(x) + ψ(y) + = 1 x is even, = 1 y is a consonant. − − 254 Then the fiber product of Xuncorrectedand Y over A is proof 255 X A Y (1, a), (1, e), (2, b), (2, c), (2, d), (3, a), (3, e), (4, b), (4, c), (4, d) . × ={ } 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method... Author Proof

4 Fig. 1 X A Y and X Y are subsets of R , but we only plot the coordinates (z , z , t ). Left: The fiber × × 1 2 1 product X A Y is the union of the blue curve and the magenta curve. The dotted blue line and dotted × magenta ellipse are the (z1, z2)-coordinate projections of the corresponding curve onto the t1 0 plane. Right: The (z , z , t )-coordinate projection of the Cartesian product X Y = 1 2 1 ×

256 Incidentally, this toy example underlies the JOIN operation in the structured query 257 language (sql) of a relational database management system (rdbms). See [16, Sec- 258 tion 6.3] for more information.

2 259 Example 2 (Algebraic geometry) Consider the following cubic curves in R :

2 260 X (t1, z1) R t1 z1(z1 1)(z1 2) 1 , ={ ∈ : = − − + } 2 261 Y (t2, z2) R t2 z2(z2 1)(z2 2) 1 . ={ ∈ : = − − + }

262 Let A R and consider the maps =

263 ϕ X A,ϕ(t1, z1) t1 and ψ Y A,ψ(t2, z2) t2. : → = : → =

264 Their fiber product,

4 265 X A Y (t1, t2, z1, z2) R (t1, z1) X,(t2, z2) Y , t1 t2 , × ={ ∈ : ∈ ∈ = }

266 isshowninFig.1. While the Cartesian product X Y is an irreducible surface, the × 267 fiber product X A Y is a union of two curves—a point (t1, t2, z1, z2) X A Y × ∈ × 268 satisfies

2 2 269 z1 z2 or z z1z2 z 3z1 3z2 2. = 1 + + 2 = + −

270 One of the curves projects onto a line and the other projects onto an ellipse. Whereas 271 the Cartesian product of two irreducible curves is always an irreducible surface, this 272 example shows that the fiber product of two irreducible curves does not need to be 273 irreducible. uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

274 Example 3 (Multiparameter eigenvalue problem) Consider a two-parameter eigen- 275 value problem (2) and the eigenpair varieties of H1 and H2,

2 n1 1 276 X (λ1, x1) C P − H1(λ1)x1 0 , ={ ∈ × : = } 2 n2 1 277 Y (λ2, x2) C P − H2(λ2)x2 0 . ={ ∈ × : = } 2 278 Let A C and consider the maps =

Author Proof 279 ϕ X A,ϕ(λ1, x1) λ1 and ψ Y A,ψ(λ2, x2) λ2. : → = : → =

280 Their fiber product is

4 n1 1 n2 1 281 X A Y (λ1, λ2, x1, x2) C P − P − × ={ ∈ × × : 282 (λ1, x1) X,(λ2, x2) Y , λ1 λ2 EP(H1, H2), ∈ ∈ = }=

283 the two-parameter eigenpair variety of H1, H2.

284 5 Fiber product homotopy method

285 Fiber products have previously appeared in numerical algebraic geometry in various 286 contexts: study of exceptional sets [20], algorithms to intersect varieties [12], and 287 numerical computations of Galois groups [11, Section 4]. However, the use of fiber 288 products in a homotopy method for solving MEPs is, as far as we know, new. We will 289 now describe this method.

290 5.1 Start system

291 We choose our start system by the following observation. Let H(λ) A0 λ1 A1 = − − 292 λk Ak be a linear polynomial matrix and L(λ) Aλ b be an affine linear ···− (k 1) k k 1 = − 293 function for some A C and b C . We claim that ∈ − × ∈ −

294 H(λ)x 0, L(λ) 0 = =

295 is equivalent to a generalized eigenvalue problem, that we will call the associated 296 generalized eigenvalue problem or associated GEP for short. k 297 To see this, let λ βq p where p, q C are such that L( p) 0 and = + ∈ = 298 q ker( L), where denotes the total derivative (also known as total differential). ∈ ∇ ∇ 299 Note that the previous statement says nothing more than A p b and Aq 0. = = 300 Eliminating q and p from

301 H(βq p)x 0, q ker( L), L( p) 0, (9) + = ∈ ∇ = 302 then gives us a GEP with uncorrectedβ the generalized eigenvalue and x the generalized proof eigen- 303 vector. We will see in Example 4 how one may obtain a GEP from (9) but expressing 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

304 the GEP in terms of general A, b, and A0,...,Ak is complicated and unilluminating. 305 The associated eigenpairs are

k n 1 306 (λ, x) C P − λ βq p, H(βq p)x 0 . (10) { ∈ × : = + + = }

307 As each coordinate of H(βq p)x is homogeneous in x, H(βq p)x 0is n 1 + + = 308 well-defined for x P − . ∈ k k 1 309 For an MEP (2), let Li C C − , i 1,...,k, be affine linear maps. Thus Author Proof : → = 310 each Li (λi ) 0 is an affine linear equation in λi , i 1,...,k. We obtain k associated = = 311 GEPs: 312 Hi (λi )xi 0, Li (λi ) 0, i 1,...,k. (11) = = =

313 The set of solutions to (11) will be called start solutions or start points and denoted 314 S. If the MEP has intrinsic dimension (d1,...,dk), then the ith associated GEP has 315 at most di generalized eigenvalues. Thus if Si denotes the set of associated eigenpairs 316 that have distinct generalized eigenvalues, then S is given by the Cartesian product

317 S S1 Sk = ×···×

k k 318 and it has cardinality i 1 Si i 1 di . k2 = |n |≤1 = n 1 n n k(k 1) 319 We define Q C P 1 P k C 1 k C by fp : × − ×···× − → +···+ × −

H1(λ1)x1 . ⎡ . ⎤ Hk(λk)xk 320 Qfp(λ1,...,λk, x1,...,xk) ⎢ ⎥ (12) := ⎢ L1(λ1) ⎥ ⎢ ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ ⎥ ⎢ L (λ ) ⎥ ⎢ k k ⎥ ⎣ ⎦ k k 321 and choose our start system to be the i 1(k 1 ni ) equations in i 1(k ni ) = − + = + 322 variables

323 Q (λ1,...,λk, x1,...,xk) 0. (13) fp = n 1 324 Again note that Hi (λi )xi 0 is well-defined for xi P i , i 1,...,k.Wehave = ∈ − = 325 the following easy observation.

326 Lemma 2 The points in S are regular solutions to the start system (13).IftheMEPis k 327 generic with respect to intrinsic dimension, then S i 1 di for any generic choice | |= = 328 of affine linear maps L1,...,Lk.  329 We now give an illustration of how an MEP can be transformed into a system of 330 GEPs by imposing randomuncorrected affine constraints. proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

331 Example 4 Consider the two-parameter eigenvalue problem given by

23 11 13 23 29 332 H1(λ1) λ11 λ12 , = 57 + 17 19 + 31 37    12 31 11 22 333 H2(λ2) λ21 λ22 , = 15 71 + 11 + 22   

334 where we write λi (λi1,λi2), i 1, 2. This two-parameter eigenvalue problem is = = Author Proof 335 singular and has only two regular eigenvalues. 336 We pick two random affine linear polynomials L1 and L2, e.g.,

337 L1(λ1) (0.6909 0.2745i)λ11 (0.4277 0.1333i)λ12 1, = + + − − 338 L2(λ2) ( 0.1443 0.5711i)λ21 ( 0.0735 1.8085i)λ22 1. = − + + − + −

339 Then we have

0.41 0.31i 0.52 0.79i 1.2501 0.4967i 0 340 q , q , p , p − + − + − . [ 1 2 1 2]= 0.85 0.02i 0.22 0.21i 0 0.0224 0.5520i + − − − 

341 The polynomial matrices for the two associated GEP are

11.75 5.46i 13.25 6.45i 15.17 4.02i 19.48 4.79i 342 H1(βq p ) − + − + β + + , 1 + 1 = 16.25 8.44i 16.75 9.43i − 19.55 6.10i 23.87 6.87i − + − +  + +  12.04 1.10i 31.04 1.10i 0.08 0.35i 0.08 0.35i 343 H2(βq p ) + + β − + − + . 2 + 2 = 15.04 1.10i 71.04 1.10i − 0.08 0.35i 0.08 0.35i + +  − + − + 

344 The first GEP has two finite eigenvalues: 0.9978 1.1933i and 0.5637 0.3035i. − + − + 345 The second GEP has only one finite eigenvalue at 3.6333 28.4804i. Note that the − − 346 values d1 2, d2 1 are in agreement with the number of regular eigenvalues. Thus = = 347 the start solutions, i.e., the solutions to our start system, of our homotopy is a set of 348 two points with (λ1, λ2)-coordinates below.

λ1 λ2

349 ( 0.9978 1.1933i)q1 p1 ( 3.6333 28.4804i)q2 p2 (−0.5637 + 0.3035i)q + p (−3.6333 − 28.4804i)q + p − + 1 + 1 − − 2 + 2

350 5.2 Target system

(k 1) k(k 1) k2 k 1 351 For i 1,...,k,letRi C be generic and let Gi C C be the = ∈ − × − : → − 352 linear function defined by

λ1 Ik Ik . −. . k(k 1) k2 353 Gi (λ1,...,λk) Ri Dk . , Dk .. .. C − × , (14) := uncorrected⎡ . ⎤ := ⎡ ⎤ ∈ proof λk Ik Ik ⎢ ⎥ ⎢ − ⎥ ⎣ ⎦ ⎣ ⎦ 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

(k 1) k(k 1) 354 where Ik is the k k identity matrix. If the matrices R1,...,Rk C are × ∈ − × − 355 generic, then

k2 356 (λ1,...λk) C G1(λ1,...,λk) Gk(λ1,...,λk) 0 { ∈ : =···= = } k2 357 (λ1,...λk) C λ1 λk , (15) ={ ∈ : =···= }

358 and therefore, Author Proof 2 k n1 1 nk 1 359 (λ1,...λk, x1,...,xk) C P − P − { ∈ × ×···× : 360 G1(λ1,...,λk) Gk(λ1,...,λk) 0 =···= = } 2 k n1 1 nk 1 361 (λ1,...λk, x1,...,xk) C P − P − λ1 λk D, ={ ∈ × ×···× : =···= }=

362 i.e., the linear space in Lemma 1. Hence the system

Hi (λi )xi 0, G1(λ1,...,λk) Gk(λ1,...,λk) 0, i 1,...,k, = =···= = = 363 (16) 364 is equivalent to the fiber product MEP (4), which is equivalent to the original MEP in 365 (2). We define

H1(λ1)x1 . ⎡ . ⎤ Hk(λk)xk 366 Pfp(λ1,...,λk, x1,...,xk) ⎢ ⎥ , (17) := ⎢G1(λ1,...,λk)⎥ ⎢ ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ ⎥ ⎢G (λ ,...,λ )⎥ ⎢ k 1 k ⎥ ⎣ ⎦ 367 and choose our target system to be

368 P (λ1,...,λk, x1,...,xk) 0, fp =

369 which is of course just (16). By Proposition 1, our target system (17) yields the 370 eigenpairs of the MEP.

371 5.3 Fiber product homotopy

372 The main objective of this section is to show that our start system is chosen correctly 373 with probability one.

374 Definition 5 A fiber product homotopy for the MEP with linear polynomial matrices 375 H1,...,Hk is a straight-line homotopy from t 0tot 1 given by the polynomial uncorrected= = proof 376 map 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

2 k n1 1 nk 1 n1 nk (k 1)k 377 H C (P − P − ) C C +···+ C − , fp : × ×···× × → × H1(λ1)x1 . ⎡ . ⎤ Hk(λk)xk 378 Hfp(λ1,...,λk, x1,...,xk, t) ⎢ ⎥ . (18) := ⎢(1 t)L1(λ1) tG1(λ1,...,λk)⎥ ⎢ − + ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ ⎥ ⎢(1 t)Lk(λk) tGk(λ1,...,λk)⎥

Author Proof ⎢ − + ⎥ ⎣ ⎦ 379 Note that Hfp (1 t) Q t Pfp. = − fp + 380 Throughout the article we assume that k 2 so that we indeed have a multiparameter ≥ 381 eigenvalue problem. If k 1, then k(k 1) 0, and (18) will not involve the path = − = 382 parameter t. If the reader is wondering whether our homotopy method applies to a 383 standard or generalized eigenvalue problem like in [24], this shows that the answer is 384 no.

385 Theorem 1 The fiber product homotopy for an MEP (18) with intrinsic dimension 386 (d1,...,dk) has a start system chosen correctly with probability one if the start solu- k 387 tions S has S i 1 di . | |= = 388 Proof By Lemma 2, the start solutions S are known after solving k generalized eigen- 389 value problems of dimensions n1,...,nk respectively. Consider the variety

2 k n1 1 nk 1 390 C (λ1,...,λk, x1,...,xk) C P − P − := { ∈ × ×···× : 391 L(λ1) 0,...,L(λk) 0 . = = } k 392 C EP The intersection (H1,...,Hk) consists of S i 1 di points. As each Gi ∩ (k 1) k(k 1) | |= = 393 is a linear function and each Ri C − × − is generic, i 1,...,k, it follows ∈  = 394 from the gamma trick [21, Lemma 7.1.3] that for regular eigenpairs, the homotopy 395 has a start system chosen correctly with probability one. ⊓⊔ 396 We will provide more extensive numerical experiments in Sect. 7 but here we 397 illustrate our method with a small example: k 3 and n1 n2 n3 2. = = = = 398 Example 5 We generate an MEP by randomly choosing the 2 2 coefficient matri- × 399 ces Aij for i 1, 2, 3 and j 0, 1, 2, 3. There are eight solutions to the start = = 400 and target systems. Note that fiber homotopy method requires that we work with 9 1 1 1 401 (λ1, λ2, λ3, x1, x2, x3) C P P P . The end point will however be of the form ∈9 ×1 ×1 ×1 3 1 1 1 402 (λ, λ, λ, x1, x2, x3) C P P P where (λ, x1, x2, x3) C P P P ∈ × × × ∈ × × × 403 is a multiparameter eigenpair. 404 With our homotopy (18), we deform from t 0tot 1. Note that λi 3 = = = 405 (λi1,λi2,λi3) C , i 1, 2, 3. In the left plot of Fig. 2, we track the λi1-coordinate of ∈ = 406 all eights paths for t 0.9, 1 , i 1, 2, 3. The horizontal and vertical axes represent ∈[ ] = 407 the real and imaginary axes. We see eight sets of three paths (colored red, blue, magenta 408 to represent i 1, 2, 3), each converging to a point that represents the first coordinate = uncorrected proof 409 of an eigenvalue. Wepicked the λi1-coordinate arbitrarily and could have done the same 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

1 3

2 0.5

1

0 0

-1 -0.5

-2

-1 -3 Author Proof

-1.5 -4 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -3 -2 -1 0 1 2 3 4

Fig. 2 Left: The λi1 coordinates of all eight paths for t 0.95, 1 , i 1, 2, 3. Right: The λi1 coordinates of the path highlighted in the left panel for t 0, 1 , i ∈[1, 2, 3. The] horizontal= and vertical axes represent the real and imaginary axes ∈[ ] =

9 1 1 1 410 plot for any of the 15 coordinates in (λ1, λ2, λ3, x1, x2, x3) C P P P . ∈ × × × 411 What we are witnessing is a one-dimensional projection of the homotopy path in 9 1 1 1 412 C P P P converging to the eight eigenpairs of the MEP. Note that “one × × × 413 dimension” here means “one complex dimension” which translates to the two real 414 dimensions we see in Fig. 2. 415 The left plot shows only the behavior of the homotopy path towards the end, i.e., 416 only for t 0.95, 1 . The right plot in Fig. 2 shows the full homotopy path, i.e., for ∈[ ] 417 all t 0, 1 ,oftheλi1-coordinates for one of the eight solutions of the left plot. ∈[ ] 418 We would like to emphasize that in Example 5, the homotopy path is confined to a 9 1 1 1 419 six-dimensional subspace of C P P P as only 6 k(k 1) equations involve × × × = − 420 the path parameter t. The following difference between the fiber product homotopy 421 and diagonal coefficient homotopy for MEP cannot be overstated. For the former, 422 at most k(k 1) equations involve the path parameter, all of which are linear in − 423 λ1,...,λk. For the latter, n1 n2 nk equations involve the path parameter, all + +···+ 424 of which are multilinear. Even when k(k 1)>n1 nk, the paths of the fiber − +···+ 425 product homotopy would still be easier to track than those of the diagonal coefficient 426 homotopy, as we will see later.

427 6 Continuation procedure

428 Observe that there are k more variables than equations in (18) because the eigenvectors 429 are unique only up to scaling, i.e., they are indeed points in projective spaces. To 430 account for this arbitrary scaling, we fix a generic affine chart in each projective space ni 1 T ni 431 P − , i.e., by introducing the affine constraints di xi 1 with di C , i 1,...,k. ni ni = ∈ = 432 Let Aij C × , j 0, 1,...,k, i 1,...,k be the matrices of our MEP as in ∈ = k2 = k 1 k k 1 433 Definition 1. Recall the maps Gi C C − in (14) and Li C C − in (11). k2 : k→1 : → 434 For i 1,...,k,weletLi C C be defined by = uncorrected: → − proof 435 Li (λ1,...,λk) Li (λi ), := 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

k k2 436 i.e., Li extends the domain of Li from C to C . m n n 437 In the following, we write Om n C × and On C for the zero matrix and n × ∈ ∈ 438 zero vector, and 1n C for the all ones vector. To track the homotopy, we use ∈ 439 an Euler–Newton predictor-corrector method. The Euler step gives an approximate 440 eigenpair whereas the Newton step refines the approximation: 2 2 441 Euler step: Solve (k n1 nk) (k n1 nk) linear system + +···+ × + +···+

λ1 Author Proof diag B (x ),...,B (x ) diag H (λ ),...,H (λ ) 1 1 k k 1 1 k k . O O T T . n1 nk k k k2 diag(d1,...,dk ) ⎡ ⎤ +···+ + × L1 G1 ⎡ O ⎤ λk 442 (1 t) L1 t G1 (k 1) (n1 nk ) ⎡ − ⎤ − ∇ + ∇ − × +···+ ⎢ ⎥ . , ⎢ . . ⎥ ⎢x1⎥ = . ⎢ . . ⎥ ⎢ ⎥ ⎢ ⎥ ⎢. ⎥ ⎢ O ⎥ ⎢ . ⎥ ⎢ Lk Gk ⎥ ⎢ (1 t) Lk t Gk (k 1) (n1 nk ) ⎥ ⎢ ⎥ ⎢ − ⎥ ⎢ − ∇ + ∇ − × +···+ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎢xk ⎥ ⎣ ⎦ n k 443 where Bi (xi ) Ai1xi ,...,Aikxi  C i , i 1,...,k. := −[ ]∈ × = 444 Here we regard the total derivatives as Jacobian matrices, i.e., (k 1) k2 445 Li , Gi C , i 1,...,k.AsLi ’s and Gi ’s are affine ∇ ∇ ∈ − × = 446 linear maps, their Jacobians are constant matrices that do not depend 447 on λi ’s. 448 Predictor: This is given by

λ1 λ1 λ1 . . . ⎡ . ⎤ ⎡ . ⎤ ⎡ . ⎤   λ λ λ 2 k k k n1 nk k 449 p ⎢ ⎥ ⎢ ⎥ h ⎢ ⎥ C +···+ + := ⎢x1⎥ = ⎢x1⎥ + ⎢x1⎥ ∈ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢. ⎥ ⎢ . ⎥ ⎢. ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢x ⎥ ⎢x ⎥ ⎢x ⎥ ⎢ k⎥ ⎢ k⎥ ⎢ k⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ 450 where his the step size. The predictor is the input to the Newton 451 step.

uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

2 2 452 Newton step: Solve (k n1 nk) (k n1 nk) linear system + +···+ × + +···+

∆λ1 B (x(ℓ)),...,B (x(ℓ)) (H λ(ℓ)),...,H (λ(ℓ)) diag 1 1 k k diag 1 1 k k . O T T ⎡ . ⎤ k k2 diag(d1,...,dk) ⎡ × O ⎤ ∆λk 453 (1 t) L1 t G1 (k 1) (n1 nk ) ⎢ ⎥ − ∇ + ∇ − × +···+ ⎢∆x ⎥ ⎢ . . ⎥ ⎢ 1⎥ ⎢ . . ⎥ ⎢ . ⎥ ⎢ O ⎥ ⎢ . ⎥ ⎢ (1 t) Lk t Gk (k 1) (n1 nk ) ⎥ ⎢ ⎥ ⎢ − ∇ + ∇ − × +···+ ⎥ Author Proof ⎢∆x ⎥ ⎣ ⎦ ⎢ k⎥ λ(ℓ) (ℓ) ⎣ ⎦ H1( 1 )x1 . ⎡ . ⎤ λ(ℓ) (ℓ) ⎢Hk( k )xk ⎥ T 454 ⎢ (ℓ) ⎥ ⎢ d1x 1 ⎥ =−⎢ 1 − ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ T ⎥ ⎢ d x(ℓ) 1 ⎥ ⎢ k k ⎥ ⎢ v(ℓ)− ⎥ ⎢ ⎥ ⎣ ⎦ 455 where Bi is as defined in the Euler step and

λ(ℓ) 1 (1 t) L1 t G1 1 k 1 − (ℓ) − ∇ . + ∇ . . k(k 1) 456 v . . (1 t) . C − . := ⎡ . ⎤ ⎡ . ⎤ + − ⎡ . ⎤ ∈ (ℓ) (1 t) Lk t Gk λ 1k 1 ⎢ − ∇ + ∇ ⎥ ⎢ k ⎥ ⎢ − ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ (0) (0) (0) (0) T 457 λ λ The initial approximation 1 ,..., k , x1 ,...,xk is given 458 by p.   459 Corrector: This is given by the solution to the Newton step

∆λ1 . ⎡ . ⎤ λ 2 ∆ k n1 nk k 460 c ⎢ ⎥ C +···+ + , (19) = ⎢∆x1⎥ ∈ ⎢ ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ ⎥ ⎢∆x ⎥ ⎢ k⎥ ⎣ ⎦

uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

461 which is then added to refine the approximation,

λ(ℓ 1) λ(ℓ) λ 1 + 1 ∆ 1 . +. ⎡ . ⎤ ⎡ . ⎤ λ(ℓ 1) λ(ℓ) λ ⎢ k + ⎥ ⎢ k ∆ k ⎥ 462 ⎢ (ℓ 1)⎥ ⎢ (ℓ) + ⎥ . ⎢x + ⎥ = ⎢x ∆x1⎥ ⎢ 1 ⎥ ⎢ 1 + ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ . ⎥

Author Proof ⎢ ⎥ ⎢ ⎥ ⎢x(ℓ 1)⎥ ⎢x(ℓ) ∆x ⎥ ⎢ k + ⎥ ⎢ k k⎥ ⎣ ⎦ ⎣ + ⎦ 463 Our Euler–Newton predictor-corrector method uses a predictor p with size h, followed 464 by Newton steps until the norm of c is sufficiently small or when the maximum number 465 of iterations is reached. If t h < 1, we return to the Euler step. If t h 1, we + + ≥ 466 update h so that t h 1, and do one final Euler step followed by Newton steps. + = 467 For a rough idea of the relative costs, tracking one path, i.e., one eigenpair, typically 468 takes a total of 50 Euler steps and a total of 400 Newton steps. We provide actual 469 implementation details in Sect. 7. 470 The matrices in the Euler and Newton steps will in general look like the one depicted T T 471 in Fig. 3 for a four-parameter eigenvalue problem. The k row vectors d1,...,dk near 2 k 472 the bottom right corner and the (k k) k block representing (1 t) Li t Gi − × − ∇ + ∇ i 1 473 on the bottom left corner of the matrix are dense—recall from Lemma 2, Theorem=1,   474 and the first paragraph of this section that we require the entries in these blocks be 475 generic, i.e., there is zero probability that any of these entries is zero. On the other n n 476 hand, if the matrices Aij C i i , j 0, 1,...,k, i 1,...,k, defining the MEP ∈ × = = 477 are sufficiently sparse, then the lighter shaded blocks in the top right part of the matrix (ℓ) 478 representing Hi (λi ) (Euler step) or Hi (λ ) (Newton step), being the sum of k 1 i + 479 sparse matrices, will also be sparse. The darker shaded blocks on the top left part of (ℓ) 480 the matrix representing Bi (xi ) (Euler step) or Bi (xi ) (Newton step) are expected to 481 be dense, as the column vectors in these blocks are each a product of a sparse matrix 482 and a dense vector. 483 The Euler–Newton predictor-corrector method will converge to the solution of the 484 target system if the step sizes are sufficiently small and the approximate start solution 485 is sufficiently close to the actual start solution [1, Theorem 5.2.1]. Quantifying the 486 “sufficiently small” and “sufficiently close” is still an active area of research, but 487 we will see in Sects. 8–10 ample numerical evidence of stable convergence to true 488 solutions (even for dimension-deficient singular MEPs). Indeed, the results in Sects. 8– 489 10 are from thousands of MEPs — a single value in the tables/figures represents an 490 aggregate over tens or hundreds of runs— and we did not encounter any instance 3 491 where Euler–Newton failed to converge. uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

k2 n1 + ···+ nk Author Proof

n1 + ···+ nk

k k2 − k

Fig. 3 Pictorial representation of a typical matrix in the Euler and Newton steps. Here k 4andn1 n n n . The darker blocks are almost always dense; the lighter blocks are sparse if the= input matrices= 2 = 3 = 4 Aij’s are sufficiently sparse; the white areas are always zero

492 7 Implementation

4 493 We implemented our method in both Matlab and Bertini/Macaulay2, catering 494 respectively to the numerical computing and symbolic computing communities. All 495 experiments are carried out on single node, an Intel E5-2680v4 2.4 GHz processor 496 with 28 threads and 56 GB of RAM, in the University of Chicago Research Computing 497 Center. The parameters below can be readily changed by the user. n n 498 Inputs: The inputs are the coefficients Aij C i i , j ∈ × = 499 0, 1,...,k, i 1,...,k, of the polynomial matrices = 500 H1,...,Hk, as in Definition 1. 501 Start solutions: Solutions to the start system (12) are obtained as fol- 502 lows. For each i, we set the constant terms of Li (λi ) 503 to be 1, and generate other coefficients from the stan- − 504 dard complex Gaussian distribution using randn (Matlab) 505 or random CC (Macaulay2). In our Matlab imple- 506 mentation, we determine a null vector q ker( Li ) i ∈ ∇ 507 using null, a particular solution p to Li (λi ) 0using i = 508 mldivideuncorrected(i.e., backslash), and the associated proof eigenpairs 4 https://github.com/JoseMath/MEP_Homotopy. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

509 using eig. In our Bertini/Macaulay2 implementation, 510 we use bertiniZeroDimSolve (Bertini) with default 511 settings to determine the solutions to Hi (λi )xi 0 and = 512 Li (λi ) 0. = 513 Continuation: The bulk of our computations is in running the Euler–Newton 514 predictor-corrector method in Sect. 6. In our Matlab imple- 515 mentation, the linear systems in the Euler and Newton steps 516 are solved using mldivide. We use two sets of crite- 517 ria for controlling the step size and termination. In the Author Proof 518 conservative criterion, the step size h in the predictor 519 p is updated as follows: if the number of Newton iterations 520 i 2, we double h;ifi > 8, we halve h; else we keep the nt ≤ nt 521 same h. The Newton iteration continues until the corrector c 9 522 satisfies c < 10− or when number of iterations exceeds ∞ 523 max 20, k max(n1,...,nk) 5 .Inthefast criterion, the { + } 524 step size h in the predictor p is updated as follows: if the num- 525 ber of Newton iterations int 2, we double h;ifint > 5, we ≤ 526 halve h;elsewekeepthesameh. The Newton iteration con- 8 527 tinues until the corrector c satisfies c 2 < 10 or when the − 528 number of iterations exceeds five. The maximum and mini- 2 6 529 mum step sizes are 10− and 10− respectively for both two 530 criteria. The fast criterion is the same one used in [8], and 531 we will use it when comparing with the results given by the 532 method in [8]. Otherwise, we will use the conservative 533 criterion for better accuracy. These choices are heuristical 534 but may be easily fine-tuned. 535 Bertini is specifically designed with the homotopy method 536 in mind and these tasks are built-in and automated; users 537 need only specify the configurations they wish to change in 538 the input file. In our Bertini/Macaulay2 implementation, 539 we may simply use runBertini with default configura- 540 tions. 541 Stopping conditions: In our Matlab implementation, we estimate the endpoint 542 of the homotopy as follows: when t h > 1, we perform + 543 the Euler step with step size h 1 t and refine our end- = − 544 point with Newton’s method until the change in the update 9 545 is sufficiently small (the default is c < 10− ) or when ∞ 546 the maximum number of iterations is reached (default is set 547 to max 20, k max(n1,...,nk) 5 ). Again, Bertini is { · + } 548 designed for homotopy method and has a variety of built- 549 in options for estimating endpoint. Among other things, 550 Bertini has implemented various sophisticated “endgames” 551 to identify solutions with multiplicities larger than one and 552 return the values of these multiplicities, a feature that is too 553 involveduncorrected to replicate in our Matlab implementation. proof We use 554 the default “fractional power series endgame.” 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

555 In the next three sections, we present our numerical results on MEPs that are (i) 556 randomly generated (Sect. 8), (ii) from a real-world application (Sect. 9), and (iii) 557 dimension-deficient and singular (Sect. 10). We compare speed and accuracy of our 558 method with those of Delta and diagonal coefficient homotopy methods.

559 8 Numerical results I: randomly generated MEPs

ni ni 560 Here we randomly generate our inputs Aij C × , j 0, 1,...,k, i 1,...,k, Author Proof ∈ = = 561 from the standard complex Gaussian distribution. For convenience, we assume that 562 n1 nk n so that there are just two parameters k and n to consider. We write =···= = 563 N for the number of eigenpairs, given by Proposition 1. 564 We report the maximum time it takes to track one path, denoted by tpath, and the 565 average number of Newton iterations during the path tracking, denoted by φ(k, n). 566 The value of tpath is important as path-tracking is a task with high parallelism and tpath 567 provides a good estimate of the time it takes to run fiber product homotopy method in 568 parallel. Nevertheless, we are only able to report tpath for our Matlab implementation 569 as Bertini deals with path-tracking in a more sophisticated and automated manner 570 that offers users no easy way of determining the total track time of one path. 571 We investigate the stability of our method and the accuracy of our solutions by 572 examining the backward error as defined in [14, Section 3]. The normwise backward 573 error of an approximate eigenpair (λ, x1,...,xk) of an MEP with coefficients Aij n n ∈ 574 C i i , j 0, 1,...,k, i 1,...,k, and polynomial matrices H1,...,Hk,asin × = = 575 Definition 1 is given by

576 η(λ, x1,...,xk) min ε R (Hi (λ) ∆Hi (λ))xi 0, := { ∈ : + = 577 ∆Aij ε Aij , i 1,...,k, j 0,...,k , ≤ = = } k 578 ∆Hi (λ) ∆Ai0 λ j ∆Aij, i 1,...,k. j 1 := − = =  579 To compute η(λ, x1,...,xk) we take advantage of [14, Theorem 2], which says

Hi (λ)xi 580 η(λ, x1,...,xk) max . = i 1,...,k k = Ai0 j 1 λ j Aij  + = | | 

581 8.1 Fixed k, varying n

582 Here we fix k 3 and n1 n2 n3 n. For each value of n we generate ten three- = = = = 583 parameter eigenvalue problems. Note that the expected number of eigenpairs in these 3 584 problems is N n . As our Matlab implementation of fiber product homotopy = 585 method computes in parallel with 28 threads, the wall time reported in Table 1 is 586 estimated by multiplying the real wall time by 28. In Table 1, we see that our method 587 is faster than the timings reported for the diagonal coefficient homotopy method in 588 [8]. The Delta method failsuncorrected for larger values of n—it crashes with an proof out-of-memory 589 error in every instance when n 30. = 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

Table 1 Elapsed timings (in seconds) for Delta method, fiber product homotopy method (averaged over ten runs), and diagonal coefficient homotopy method [8]

nN Wall time tpath φ(3, n) n3 Delta Mtd. Fiber Prod. Diag. Coeff. Fiber Prod.

10 1000 1.44 264.38 779.47 0.22 386 15 3375 19.14 906.00 2888.73 0.25 398 20 8000 150.59 2547.29 7857.44 0.31 402 25 15625 815.9125 5905.11 17169.53 0.37 408 Author Proof 30 27000 failed 12820.89 32786.64 0.46 418

-7 -7

-8 -8

-9 -9

-10 -10

-11 -11

-12 -12

-13 -13

-14 -14

-15 -15

-16 -16

-17 -17 0 100 200 300 400 500 600 700 800 900 1000 0 500 1000 1500 2000 2500 3000

-7 -7

-8 -8

-9 -9

-10 -10

-11 -11

-12 -12

-13 -13

-14 -14

-15 -15

-16 -16

-17 -17 0 1000 2000 3000 4000 5000 6000 7000 8000 0 0005 00001 00051

Fig. 4 The log10 backward errors (vertical axis) of fiber product method (blue) and the Delta method (red) are plotted against the eigenvalues (horizontal axis) ordered by increasing norm from left to right

590 Our results for backward errors are presented in Fig. 4, where it is clear that fiber 591 product homotopy method (blue plot) has significantly smaller backward error than 592 the Delta method (red plot) in our numerical experiments. The experiments in Table 1 593 and Fig. 4 find all eigenpairs by tracking all start solutions of (12). 594 In our next experiment, our goal is to examine the effect of a substantial increase 595 in n on speed and accuracyuncorrected of our fiber product homotopy method. Finding proof all eigen- 596 pairs would take too long and serves little purpose for this next experiment. Instead, 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

Table 2 Timings (in seconds) and accuracy for fiber product homotopy method (100 runs)

n Wall time Backward error Best Avg Worst Best Avg Worst

30 0.71 1.12 4.45 7.47 10 17 1.81 10 16 1.42 10 15 × − × − × − 70 2.88 4.27 8.85 8.18 10 17 1.66 10 16 1.26 10 15 × − × − × − 150 16.08 29.07 91.45 1.05 10 16 1.96 10 16 1.36 10 15 × − × − × − Author Proof

597 we randomly generate 100 MEPs for each n and track one random start solution. The 598 timings in Table 2 are rough estimates of the time it would have taken to find all eigen- 599 pairs, obtained by multiplying average wall time for one randomly chosen eigenpair 3 600 by n . The backward errors in Table 2 are for one randomly chosen eigenpair (and not 3 601 multiplied by n ). Evidently, increasing n has negligible effect on backward errors, 15 602 which are all splendidly small—on the order of 10− or less.

603 8.2 Fixed n, varying k

604 This time we fix n1 nk 3 and vary k from 3,...,9. The expected number of =···=k = 605 eigenpairs is then N 3 . If the reader is wondering why we do not increase k inamore = 606 drastic manner, note that increasing k produces a corresponding exponential increase k 607 in the number of solutions—for each randomly generated MEP,there are 3 eigenpairs. 608 Also, unlike changing n, which just changes the dimension of the problem, changing k 609 gives a different class of problems—for example, a two-parameter eigenvalue problem 610 is qualitatively different from a one-parameter eigenvalue problem (i.e., a GEP)— 611 and each k deserves a careful examination. Compared to the numbers for diagonal 612 coefficient homotopy method in [8, Table 3], the numbers in Table 3 show that the 613 fiber product homotopy method is significantly faster and also more stable in the sense 614 that every path converged and did not need to be rerun. 615 The reader is reminded that fiber product homotopy method tracks k paths each 616 carrying a copy of the eigenvalue; these k copies λ1,...,λk converge to the same 617 eigenvalue λ1 λk if the method runs correctly (see Example 5). This is indeed =···= 618 the case as the reader can see from the near zero values of δ(k) max2 i k( λ1 := ≤ ≤ − 619 λi 1) reported in Table 3.

620 8.3 Comparisons on Bertini

621 The results in Tables 1 and 3 compare the Matlab implementation of fiber product 622 homotopy method (18) to the results for diagonal coefficient homotopy method (6) 5 623 reported in [8]. Here we will compare them on Bertini. As Bertini is designed 624 for homotopy continuation methods, we do not include a non-homotopy method like 625 Delta method in our comparisonuncorrected on this platform. proof 5 WehavealsousedaMacaulay2 [9] package [4] to produce the relevant input files. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

Table 3 Elapsed timings (in seconds) for Delta method and the two homotopy methods, all in Matlab. Accuracy for our method. Here δ(k) max2 i k ( λ1 λi 1) = ≤ ≤ − kN Wall time δ(k)φ(k, 3) tpath 3k Delta Mtd. Fiber Prod. Diag. Coeff. Fiber Prod.

4810.13 50.81 77.63 2.90 10 15 382 0.27 × − 5 243 0.19 89.44 288.24 5.43 10 15 403 0.26 × − 6 729 1.32 269.92 1089.10 1.37 10 14 433 0.32 × − 14 Author Proof 7 2187 12.30 920.54 3979.94 3.18 10 443 0.40 × − 8 6561 172.44 3355.81 13445.96 7.35 10 13 457 0.49 × − 9 19683 failed 12893.37 48624.78 1.06 10 13 466 0.64 × −

Table 4 Elapsed timings (in nN Bertini Matlab seconds) for comparing homotopy methods and n2 Diag. Coeff. Fiber Prod. Fiber Prod. implementations 525 0.25 0.45 0.61 15 225 31.93 29.36 3.02 30 900 1158.61 882.68 18.99 50 2500 18661.62 14840.28 110.79

626 In this numerical experiment, we look at a two-parameter eigenvalue problem, i.e., 2 627 k 2, with n1 n2 n. The dimension of the problem is thus N n . From Table 4, = = = = 628 the fiber product homotopy method is consistently faster that the diagonal coefficient 629 homotopy method in Bertini. We include the timings of our Matlab implementation 630 of the fiber product homotopy method in the last column for comparison.

631 9 Numerical results II: Mathieu’s systems

632 An example where multiparameter eigenvalue problems surface is Mathieu’s systems, 633 which in turn arises from studies of vibration of a fixed elliptic membrane [18,23]. We 634 will test our Matlab implementation of fiber product homotopy method against the 635 Delta method on this problem. 636 We refer the reader to [18, Section 2] for a discussion of how a coupled system 637 of two-point boundary value problems, representing Mathieu’s angular and radial 638 equations, yields a two-parameter eigenvalue problem (thus k 2) upon Chebyshev = 639 collocation discretization. The dimensions of the matrices n1 and n2 correspond to the 640 number of points used in the discretization. 641 In Fig. 5, we present accuracy result for a two-parameter eigenvalue problem with 642 n1 18 and n2 38 coming from a Mathieu system. The horizontal axis represents = = 643 the n1n2 684 eigenvalues, ordered from the smallest to largest by the norm of λ1. = 644 The vertical axis measuresuncorrected backward errors on a log scale. The blue proof plot shows the 645 backward error of our fiber product homotopy method whereas the red plot is that for 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

-10 -10

-11 -11

-12 -12

-13 -13

-14 -14

-15 -15

-16 -16 Author Proof

-17 -17

-18 -18 0 100 200 300 400 500 600 0 100 200 300 400 500 600

Fig. 5 The log10 backward errors (vertical axis) of fiber product method (blue) and Delta method (red) plot- ted against eigenvalues (horizontal axis) ordered from smallest to largest. Left figure: 81 Newton iterations, taking 19 seconds. Right figure: 25 Newton iterations at the end of the path, taking 16 seconds

646 the Delta method. With few exceptions, fiber product homotopy produces significantly 647 more accurate results than the Delta method by orders of magnitude. Indeed, every 15 648 eigenpair computed with our method here has a backward error that is less than 10− . 649 The left and right figures in Fig. 5 differ by Nen, the number of Newton iterations 650 used in the Newton step at the end of the path—the left plot uses the conservative 651 stopping condition of 81 max(20, k max n1,...,nk 5) iterations whereas the = { }+ 652 right plot uses an early stopping condition of 25 iterations. We see no discernible 653 difference in the backward errors but the left plot took 19 seconds with 28 threads 654 whereas the right plot took only 16 seconds to compute. This suggests that there is 655 room for further fine-tuning to improve speed without sacrificing stability. 656 We next employ Shub–Smale α-theory [5] to certify the quadratic convergence of 657 the Newton steps in a neighborhood of the end point. Briefly, in this theory there is a n n 658 function α that takes a polynomial system f C C and an approximate solution n : → 659 z C as its input and returns a positive real number. If α( f , z) is less than the ∈ 660 constant (13 3√17)/4, then one has certified quadratic convergence of Newton’s − 661 method on the approximate solution z to the polynomial system f [5, Theorem 2, 662 p. 160]. With our default tolerances we are able to certify the smallest 550 of the 684 663 eigenvalues—see Fig. 6.

664 10 Numerical results III: dimension-deficient singular MEPs

665 To each MEP with k parameters there correspond k 1 Delta matrices ∆0,...,∆k + 666 [2]. The nonhomogeneous problem (3)issingular if ∆0 is singular [13, Section 2]. 667 Such problems have fewer than the expected n1 nk eigenpairs. That a singular k- ··· 668 parameter eigenvalue problem presents computational difficulties is already evident 669 when k 1: a GEP with a singular matrix pencil A λB is well-known to be = − 670 challenging computationally. Among the singular MEPs, our fiber product homotopy 671 method is well-suited for dimension-deficientuncorrected MEPs that are generic proof with respect to 672 intrinsic dimension, i.e., where the intrinsic dimensions (d1,...,dk)<(n1,...,nk). 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

10 10

8 8

6 6

4 4

2 2

0 0

-2 -2

-4 -4

-6 -6 Author Proof

-8 -8

-10 -10 0 100 200 300 400 500 600 700 0 100 200 300 400 500 600 700

Fig. 6 Computed bound (vertical axis) on log10 α for approximate eigenpairs of Mathieu MEP plotted against eigenvalues (horizontal axis) ordered from smallest to largest. If the computed bound is below log10 (13 3√17)/4 (red dashed line), the eigenpair is certified. Left figure: 81 Newton iterations. Right figure:[ 25− Newton iterations]

673 We emphasize that these form a subclass of the singular problems — not all singular 674 MEPs satisfy the aforementioned criteria. 675 Such dimension-deficient singular MEPs are inevitable when one linearizes 676 quadratic multiparameter eigenvalue problems (QMEPs) [13]: Given quadratic poly- 677 nomial matrices

2 2 678 Q1(λ, µ) B00 λB10 µB01 λ B20 λµB11 µ B02, := + + + + + 2 2 679 Q2(λ, µ) C00 λC10 µC01 λ C20 λµC11 µ C02, (20) := + + + + + n n n n 680 where Bij C 1 1 and Cij C 2 2 , i, j 0, 1, 2, solve ∈ × ∈ × =

681 Q1(λ, µ)x1 0, Q2(λ, µ)x2 0 = = n n 682 for all possible λ, µ C and nonzero x1 C 1 , x2 C 2 . It is straightforward to ∈ ∈ ∈ 683 generalize this to a quadratic k-parameter eigenvalue problem but we will only study 684 the case k 2 here. = 685 The quadratic two-parameter eigenvalue problem is mathematically equivalent to 686 a two-parameter eigenvalue problem [13]:

B00 B10 B01 0 B20 B11 00B02 687 H1(λ, µ) 0 I 0 λ I 00 µ 00 0 , = ⎡ 00− I ⎤ + ⎡00 0⎤ + ⎡I 00⎤ − ⎣C00 C10 C01⎦ ⎣0 C20 C11⎦ ⎣00C02⎦ 688 H2(λ, µ) 0 I 0 λ I 00 µ 00 0 . (21) = ⎡ 00− I ⎤ + ⎡00 0⎤ + ⎡I 00⎤ − ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ 689 Cni ni Whereas the original coefficientuncorrected matrices in Qi are in × , the coefficient proof matrices 3n 3n 690 of Hi are in C i i , i 1, 2. However, the real catch is that the two-parameter × = 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

Table 5 Number of divergent paths and elapsed timings (in seconds) for the diagonal coefficient (DC), fiber product (FP) homotopy methods, and Delta method (∆M). tpath is the time taken to track a single path, i.e., roughly how fast FP would have run in parallel

nN # of divergent paths Wall time tpath 4n2 Bertini Matlab Bertini Matlab DC FP FP DC FP ∆MFP

2 16 20 0 0 0.49 0.38 0.07 0.55 0.19 5 100 125 0 0 23.72 8.60 0.11 1.36 0.22 Author Proof 10 400 500 0 0 801.24 244.01 0.95 5.41 0.30 20 1600 2000 0 0 38761.58 14079.69 24.98 56.56 0.79 40 6400 8000 0 0 33 days 8 days 1156.38 1133.90 4.40

691 eigenvalue problem (21) is singular and its intrinsic dimension (2n1, 2n2) is strictly 692 smaller than (3n1, 3n2). n n 693 In our experiment, we set n1 n2 n, randomly generate Bij, Cij C , = = ∈ × 694 and compare fiber product and diagonal coefficient homotopy methods in Bertini. 695 The difficulty of dimension-deficient MEP is conspicuously reflected in our observed 2 2 696 results: The diagonal coefficient homotopy method has 9n start solutions and 5n of 697 the paths tracked does not converge for all values of n we tested, which we recorded 698 in Table 5. On the other hand, in these experiments where the MEPs are generic with 699 respect to intrinsic dimension, our fiber product homotopy method did not produce a 700 single divergent path. For n 40, the wall time of the diagonal coefficient homotopy = 701 method in Bertini is estimated based on the number of paths it tracked when the fiber 702 product homotopy method finishes. 703 We next compare fiber product homotopy and Delta methods [17]inMatlab.From 704 the backward errors in Fig. 7 for dimension-deficient singular MEPs, we see a similar 705 pattern as in the nonsingular MEPs in Fig. 4—fiber product homotopy method is 706 more accurate than the Delta method by orders of magnitude. More importantly, Delta 707 method loses accuracy as n increases but fiber product homotopy method maintains it 16 708 at 10− even after n is doubled four times. 709 From the last three columns of Table 5, we see that while the Delta method beats 710 fiber product homotopy method in terms of speed for smaller values of n, the situation 711 reverses when n 40. Furthermore, the value of t in the last column indicates that = path 712 had we run fiber product homotopy method in parallel on a machine with sufficiently 713 many cores, all eigenpairs can be obtained in seconds. On the other hand, even though 714 the program eig in Matlab has the ability to use multiple cores and processors 715 for the Delta method, parallelization is not straightforward as the problem involves k 716 large Delta matrices of sizes n1 nk n1 nk that need to be formed and stored ··· × ··· 717 in advance. As such, the Delta method is infeasible for large matrices. uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

-7 -7

-8 -8

-9 -9

-10 -10

-11 -11

-12 -12

-13 -13

-14 -14

-15 -15 Author Proof

-16 -16

-17 -17 0 10 20 30 40 50 60 70 80 90 100 0 50 100 150 200 250 300 350 400

-7 -7

-8 -8

-9 -9

-10 -10

-11 -11

-12 -12

-13 -13

-14 -14

-15 -15

-16 -16

-17 -17 0 200 400 600 800 1000 1200 1400 1600 0 1000 2000 3000 4000 5000 6000

Fig. 7 The log10 backward errors (vertical axis) of fiber product homotopy (blue) and Delta methods (red) for the dimension-deficient singular MEP (21), plotted against eigenvalues (horizontal axis) ordered in increasing norm from left to right

718 In this section we have limited ourselves to singular MEPs that are (i) dimension- 719 deficient and (ii) generic with respect to intrinsic dimension. One situation when (ii) 6 720 is not satisfied is an example

721 11 Conditioning of the fiber product MEP and standard MEP

722 Recall that condition number [7, Chapter 14] depends on the problem —two different 723 problems that always give the same solution will in general have different condition 724 numbers. The condition number for MEP as defined in [14] applies to the standard 725 formulation of MEP in (2). Since we are solving a different problem (4), or more 726 generally (16), albeit one that gives the same solutions as (2), it will have a different 727 condition number.

6 The example is implemented in our GitHub codes at https://github.com/JoseMath/MEP_Homotopy/blob/ master/MATLAB/examples/demo_flutter1.m for aerospace engineering [19], which remains a challenge for our method. In general, when the solutions to (3) are not the same as the regular eigenpairs of the MEP, the fiber product homotopy method may lose some regular eigenpairs. To address this issue, one has to know the connection between theuncorrectedHi (λ) and the corresponding system of generalized proof eigenvalue problems on ∆-matrices but this is unfortunately not well-understood. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

728 In this section, we discuss the condition number for the fiber product MEP (4) and 729 compare it to the condition number for the standard MEP as defined in [14]. They are 730 expected to be quite different by the common interpretation of condition number as 731 measuring the change in output produced by a change in input. The standard condition 732 number for MEP in [14] measures the change in eigenvalues (output) produced by a 733 change in the coefficient matrices (input). In fiber product homotopy method, we do 734 not deform these coefficient matrices; instead, the relevant notion of condition number 735 is one that measures the change in eigenvalues (output) produced by a change in the 736 linear space defined by G G 0asin(16) (input). So in our setting, we Author Proof 1 k = ··· = = 737 require the condition number of a variety intersected with a varying linear subspace. 738 In the following, by a projective linear subspace of a projective space, we mean n m (n 1) n 1 739 π(x) P M x 0, x 0 for some M C × + and where π C + { ∈ n : = = } ∈ : \ 740 0 P is the canonical projection. Henceforth, we will write M π ker(M) { }→ m (n 1) := n \ 741 0 for the projective linear subspace that M C × + defines in P . We write { } ∈ n 742 x0 x1 xn for homogeneous coordinates in P . [ : : ···: ]

743 11.1 Intersecting a variety with a varying linear subspace

744 We briefly review some relevant ideas in [6], on which our condition number in n 745 Sect. 11.2 is based. Let Z be a degree-p variety in P of codimension n d.The − 746 Hurwitz variety of Z is a subvariety of the Grassmannian G(n d, n) of (n d)- n − − 747 dimensional projective linear subspaces in P defined by

748 HZ M G(n d, n) Z M does not consist of p reduced points . := { ∈ − : ∩ }

749 The Hurwitz variety is an irreducible hypersurface defined by a polynomial called 750 the Hurwitz form in the coordinate ring of the Grassmannian [22]. This variety can 751 be regarded as the set of ill-posed instances of the problem of intersecting a variety 752 by a varying linear space [6]. By [6, Definition 1.1 and Theorem 1.4], we have the 753 following definition.

n 754 Definition 6 Let Z be a d-dimensional irreducible projective variety in P and M ∈ 755 G(n d, n).Letz Z M and α be the minimum angle between the tangent spaces − ∈ ∩ 756 Tz(Z) and Tz(M).Theintersection condition number of M at z with respect to Z is

1 757 κZ (M, z) := sin α

758 if z is a smooth point of Z and M intersects Z transversally at z, and κZ (M, z) := ∞ 759 otherwise.

760 The elements of HZ are preciselyuncorrected the projective linear subspaces where proof the intersection 761 condition number is infinite [6, Theorem 1.6]. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

762 11.2 Condition number of the fiber product MEP

k2 763 Consider the multiparameter eigenvalue variety EV(H1,...,Hk) C in (8). k2 ⊆ 764 Throughout this section, Z P will always denote the projective variety ⊆ k2 765 Z 1 λ1 λk P (λ1,...,λk) EV(H1,...,Hk) , (22) := {[ : : ···: ]∈ : ∈ } k2 766 where we have denoted the homogeneous coordinates of P by Author Proof

767 λ0 λ11 λ1k λk1 λkk , [ : : ···: : ···: : ···: ] λ1 λk       768 with λ0 as the homogenizing coordinate. The defining equations of Z are homogeneous 769 polynomials in λ0, λ1,...,λk such that setting λ0 1 gives the defining equations = 770 for EV(H1,...,Hk) in (8) 771 We will apply the notion of an intersection condition number in Sect. 11.1 to define 772 a condition number for fiber product MEP.

773 Definition 7 Given a fiber product multiparameter eigenvalue problem (4), let Z be 774 the projective variety in (22), Mfp be the projective linear space

k2 775 Mfp 1 λ1 λk P λ1 λk := {[ : : ···: ]∈ : =···= } k2 776 λ0 λ1 λk P λ1 λk , (23) ={[ : : ···: ]∈ : =···= }

777 and z be the point

778 z 1 λ1 λk Z Mfp. := [ : : ···: ]∈ ∩

779 The condition number of the fiber product MEP (4) is given by

780 κ (λ1,...,λk, H1,...,Hk) κZ (M , z). fp := fp k2 781 Note that Z Mfp consists precisely of points 1 λ1 λk P where ∩ k2 [ : : ···: ]∈ 782 (λ1,...,λk) C are the eigenvalues of the MEP. We next show how to com- ∈ 783 pute κZ (M, z) for any projective linear subspace M—as we will see later, for fiber 784 product homotopy method, we would also be interested in κZ (M, z) for projective 785 linear subspaces M other than Mfp. n n 786 Theorem 2 Let Aij C i i ,i 1,...,k, j 0, 1,...,k, be the matrices of an ∈ × = = 787 MEP as in Definition 1. Let Z be the projective variety in (22), M π(ker(M) 0 ) (k2 k) (k2 1) = \{ } 788 for some M C − × + , and z 1 λ1 λk Z M be a smooth point ∈ n =[7: : ···: ]∈ ∩ 789 of Z.Ifxi , y C i are nonzero vectors such that i ∈ T 790 Hi (λi )xi 0, Hi (λi ) y 0, i 1,...,k, =uncorrectedi = = proof 7 These are respectively right and left eigenvectors of the polynomial matrix. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

791 then the intersection condition number 1 792 κZ (M, z) , = sin α

793 where α is the minimum angle between M and J π ker(J) 0 , = \{ } T T y1 A10 x1 y1 B1(x1) yT A x yT B (x ) 2 20 1 2 2 2 k (1 k2) Author Proof 794 J ⎡ . . ⎤ C × + (24) := . .. ∈ ⎢ T T ⎥ ⎢ y A x y B (x )⎥ ⎢ k k0 k k k k ⎥ ⎣ ⎦ n k 795 with Bi (xi ) Ai1xi ,...,Aikxi C i ,i 1,...,k. := −[ ]∈ × = 796 Proof By Definition 6, it suffices to show that Tz(Z) Tz(J). First, recall from = 797 Definition 3 that the multiparameter eigenvalue variety EV(H1,...,Hk) is a pro- 798 jection of the multiparameter eigenpair variety EP(H1,...,Hk). So the projective k2 n 1 n 1 799 closure of EP(H1,...,Hk) in P P 1 P k , which we will denote by × − ×···× − 800 EP(H1,...,Hk), projects onto the variety Z. 801 For a linear polynomial matrix

802 H(λ) A0 λ1 A1 λk Ak, = − −···−

803 we introduce a homogenizing variable λ0 and define a homogenized H as

804 H(λ0, λ) λ0 A0 λ1 A1 λk Ak. := − −···−

805 We homogenize H1,..., Hk in this manner, write w (λ0, λ1,...,λk, x1,...,xk) := 806 for brevity, and then let

H1(λ0, λ1)x1 . 807 w . H( ) ⎡ . ⎤ . :=  H (λ , λ )x ⎢ k 0 k k⎥ ⎣ ⎦ T 808 C w Write H h1,...,hn1 nk wherehi , i 1,...,n1 nk,are =[ +···+ ] ∈ [ ] = +···+ 809 multihomogeneous polynomials—homogeneous in the variables λ0, λ1,...,λk and 810 homogeneous in each of the variables x1,...,xk. Note that the hi ’s vanish on the vari- 811 ety EP(H1,...,Hk); in fact, the set of polynomials that vanish on EP(H1,...,Hk) 812 is given by

m 813 C w C w N h λ0 h g1h1 gn1 nk hn1 nk , gi , m . { ∈ [ ]: = +···+ +···+ +···+ ∈ [ ] ∈ } 2 814 The (n1 nk) (1 k n1 nk) Jacobian matrix, +···+ × + + +···+ ∂uncorrected proof 815 H(w) H, λ H,..., λ H, x H,..., x H , ∇ = ∂λ ∇ 1 ∇ k ∇ 1 ∇ k 0  123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

816 when evaluated at a point

817 w0 ( 1 λ1 λk , x1,...,xk) (z, x1, , xk) EP(H1,...,Hk), := [ : :···: ] = ··· ∈

818 has the form

A10 x1 B1(x1) H1(1, λ1) A20 x1 B2(x2) H2(1, λ2) 819 H(w ) ⎡ ⎤ . 0 . ..  .. Author Proof ∇ = . . . ⎢  ⎥ ⎢A x B (x ) H (1, λ )⎥ ⎢ k0 k k k k k ⎥ ⎣ ⎦ ( 2 ) 820 Cm k 1 O  For any L × + , if each row of the matrix L, m (n1 nk ) m (k2 1 n ∈ n ) [ × +···+ ]∈ 821 C 1 k is in the row space of H(w0), then L π ker(L) 0 is × + + +···+ ∇ = \{ } 822 such that Tz(Z) Tz(L). This is because Z is the projection of EP(H1,...,Hk) to k2 1 ⊆2 823 P in the first k 1 coordinates. With this in mind, we multiply H(w0) on the + + ∇ 824 left by

T y1 yT 2 k (n1 nk ) 825 ⎡ . ⎤ C × +···+ .. ∈ ⎢ T ⎥ ⎢ y ⎥ ⎢ k⎥ ⎣ ⎦ 2 826 to obtain the k (1 k n1 nk) matrix × + + +···+ yT A x yT B (x ) OT 1 10 1 1 1 1 n1 nk yT A x yT B (x ) OT +···+ 2 20 1 2 2 2 n1 nk 827 ⎡ +···+ ⎤ O . . . J, k (n1 nk ) , . .. . =[ × +···+ ] ⎢ T T T ⎥ ⎢ y A x y B (x ) O ⎥ ⎢ k k0 k k k k n1 nk ⎥ ⎣ +···+ ⎦ 828 showing that Tz(Z) Tz(J). When J is full rank, we have that ⊂

829 codim Tz(J) k codim Tz(Z) = =

830 when z is a smooth point. It follows that Tz(Z) Tz(J). = ⊓⊔ 831 We show how we obtain the condition number κfp(λ1,...,λk, H1,...,Hk). Con- 832 sider the following one-parameter family of projective linear spaces induced by a 833 one-parameter family of matrices: For a fixed t 0, 1 ,let ∈[ ]

1k 1 L1 Ok 1 G1 − − ∇ − ∇ 1k L Ok G 1 2 1 2 k(k 1) (k2 1) Mt (1 t) ⎡− . − ∇ . ⎤ t ⎡ .− ∇ . ⎤ C − × + , := − . .. + . . ∈ ⎢ 1 ⎥ ⎢O ⎥ ⎢ k 1 uncorrectedLk⎥ ⎢ k 1 Gk⎥ proof ⎢− − ∇ ⎥ ⎢ − ∇ ⎥ 834 ⎣ ⎦ ⎣ ⎦ (25) 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

835 where L1,...,Lk are as in (11) and G1,...,Gk are as in (14). Let

k2 836 Mt π ker(Mt ) 0 P := \{ } ⊆

837 be the corresponding projective linear subspace. Note that Mt dehomogenizes (by k2 838 setting λ0 1) to the affine linear space in C defined by =

839 0 (1 t)L1(λ1) tG1(λ1,...,λk), = − + Author Proof . . 840 . .

841 0 (1 t)Lk(λk) tGk(λ1,...,λk), = − +

842 where the right-hand sides are the last k(k 1) linear polynomials in (18). − 843 Therefore, for any t 0, 1), the projective closure of the set of solutions to ∈[ 844 Hfp(λ1,...,λk, x1,...,xk, t) 0 defined in (18) projects onto Z Mt . = ∩ 845 When t 1, by (15) and (23), we have = k2 846 M1 λ0 λ1 λk P λ1 λk Mfp, ={[ : : ···: ]∈ : =···= }=

847 and for z 1 λ1 λk Z M1, we obtain, by Definition 7, =[ : : ···: ]∈ ∩

848 κZ (M1, z) κfp(λ1,...,λk, H1,...,Hk). =

849 The intersection condition number κZ (Mt , z) for t 0, 1) is also useful as it measures ∈[ 850 conditioning of the subproblems encountered during path tracking in the fiber product 851 homotopy method. The next theorem shows that κZ (Mt , z) is almost always finite 852 and Example 6 indicates that it is typically small.

853 Theorem 3 For any t 0, 1), κZ (Mt , z) is finite with probability one. ∈[ 854 Proof Since the Hurwitz variety HZ comprises projective linear subspaces M with 855 κZ (M, z) , it suffices to show that for any t 0, 1), Mt / HZ with probability 2 =∞ ∈[ k ∈n1 1 nk 1 856 one. By Theorem 1, the fiber product homotopy Hfp C (P − P − ) n n k(k 1) : × ×···× × 857 C C 1 k C has a start system chosen correctly with probability one. → +···+ × − 858 Thus H (λ1,...,λk, x1,...,xk, t) 0, t 0, 1), has smooth solution paths with fp = ∈[ 859 probability one. 860 If Mt HZ , then these solution paths would not be smooth as the projective closure ∈ 861 of the set of solutions to H (λ1,...,λk, x1,...,xk, t) 0 projects onto Z Mt . fp = ∩ 862 Thus, with probability one, κZ (Mt , z) is finite for t 0, 1). ∈[ ⊓⊔ 863 For comparison, κ(λ, H1,...,Hk), the condition number of a standard MEP (2)as 864 defined in [14] captures how small perturbations in the inputs Aij, i 1,...,k, j = = 865 0,...,k, affect the eigenvalue λ (λ1,...,λk), i.e., = k 866 lim sup ∆λ /ε (Ai0 ∆Ai0 (λ j ∆λ j )(Aij ∆Aij))(xi ∆xi ) 0, j 1 ε 0 : + − = + + + = →  uncorrected proof 867 ∆Aij ε Aij , i 1,...,k, j 0,...,k ≤ = =  123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

10

9

8

7

6

Author Proof 5

4

3

2

1

0 0 100 200 300 400 500 600 700

Fig. 8 The condition numbers (vertical axis) κfp(λ1, λ2, H1, H2) for fiber product homotopy (blue) and κ(λ, H1, H2) for Delta method (red) plotted respectively against eigenvalues λ1 and λ (horizontal axis), ordered in increasing norm. The vertical axis is in log10-scale

868 where (x1,...,xk) is the eigenvector, i.e., Hi (λ)xi 0, i 1,...,k. We will also T = = 869 λ let ( y1,..., yk) be the left eigenvector, i.e., Hi ( ) yi 0, i 1,...,k. k = = 870 Let θi Ai0 j 1 λ j Aij , i 1,...,k.Theθ-weighted norm, := + = | | =

k 871 M θ max M z 2 z C , zi θi , i 1,...,k , := { : ∈ | |= = } k k 1 872 for any M C .By[14, Theorem 6], κ(λ, H1,...,Hk) M θ where ∈ × = −

y A11x1 y A12 x1 y A1k x1 1∗ 1∗ ··· 1∗ y2∗ A21x2 y2∗ A22 x2 y2∗ A2k x2 873 M ⎡ . . ··· . ⎤ . := . . . ⎢ ⎥ ⎢ y Ak1xk y Ak2 xk y Akkxk⎥ ⎢ k∗ k∗ ··· k∗ ⎥ ⎣ ⎦ 874 We next show that the same Mathieu problem formulated as a fiber product MEP 875 (4) and a standard MEP (2) can have vastly different condition numbers.

876 Example 6 [Conditioning of Mathieu problem] We generate an instance of the Mathieu 877 problem in Sect. 9 with n1 18 and n2 38. In Fig. 8, we plot the fiber product mul- = = 878 tiparameter eigenvalue problem condition number κ (λ1, λ2, H1, H2) κZ (M1, z) fp = 879 λ and the standard multiparameteruncorrected eigenvalue problem condition number proofκ( , H1, H2), 880 against λ1 and λ respectively in increasing norms. 123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex Fiber product homotopy method...

881 The difference is striking—we emphasize that the vertical axis of the figure is in 882 λ λ log10-scale. The values of κfp( 1, 2, H1, H2) are all close to 1 and certainly less than 883 10. So small perturbations of the projective linear space M1 yield small changes in the 884 points of intersection of Z with M1, which correspond to the eigenvalues we seek. On 885 the other hand, the values of κ(λ, H1, H2) are vastly larger, with a majority exceeding 5 886 10 and those near the right end of the plot corresponding to the largest eigenvalues 9 887 are as large as 10 . 888 Apart from κZ (M1, z), we have also computed the intersection condition number 889 κZ (M , z) as t varies from 0 to 1 in every path we tracked. We ran our implementation Author Proof t 890 five times and found that κZ (Mt , z) does not exceed 21.85 for every t we encountered 891 in every path and in every run.

892 12 Conclusions

893 The fiber product homotopy method solves an MEP by solving a mathematically 894 equivalent problem, the fiber product MEP, via a homotopy algorithm designed to 895 exploit its structure. Our numerical experiments show that the fiber product homotopy 896 method: (i) is much faster than diagonal coefficient homotopy method on all instances 897 we tested and is faster than the Delta method on large instances; (ii) is extremely 16 898 accurate, producing relative backward errors on the order of 10− , especially in 11 899 comparison with the 10− errors in the Delta method; (iii) is unique in that it maintains 900 the same high degree of accuracy across a robust range of parameters — irrespective 901 of the dimensions of the matrices or the magnitudes of the eigenvalues. 902 We proffer two insights to explain its strength: (a) it deforms only linear equations 903 whereas the diagonal coefficient homotopy method deforms nonlinear equations; (b) 904 the problem that it solves, the fiber product MEP, is much better conditioned than the 905 equivalent standard MEP with the same solutions.

906 Acknowledgements We are extremely grateful for the reviewer’s exceptionally helpful comments and 907 suggestions that led to a much improved article.

908 References

909 1. Allgower, E.L., Georg, K.: Introduction to Numerical Continuation Methods. Classics in Applied 910 Mathematics, vol. 45. SIAM, Philadelphia (2003) Classics in Applied Mathematics Classics in Applied 911 Mathematics, vol. 45. SIAM, Philadelphia (2003) 912 2. Atkinson, F.V.: Multiparameter Eigenvalue Problems. Academic Press, New York (1972) 913 3. Atkinson, F.V., Mingarelli, A.B.: Multiparameter Eigenvalue Problems. CRC Press, Boca Raton (2011) 914 4. Bates, D., Gross, E., Leykin, A., Rodriguez, J.: Bertini for Macaulay 2. preprint arXiv:1310.3297 915 (2013) 916 5. Blum, L., Cucker, F., Shub, M., Smale, S.: Complexity and Real Computation. Springer, New York 917 (1998) 918 6. Bürgisser, P.: Condition of intersecting a projective variety with a varying linear subspace. SIAM J. 919 Appl. Algebra Geom. 1(1), 111–125 (2017) 920 7. Bürgisser, P., Cucker, F.: Condition, Grundlehren der Mathematischen Wissenschaften, vol. 349. 921 Springer, Heidelberg (2013)uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex J. I. Rodriguez et al.

922 8. Dong, B., Yu, B., Yu, Y.: A homotopy method for finding all solutions of a multiparameter eigenvalue 923 problem. SIAM J. Matrix Anal. Appl. 37(2), 550–571 (2016) 924 9. Grayson, D.R., Stillman, M.E.: Macaulay2, a software system for research in algebraic geometry. 925 Available at http://www.math.uiuc.edu/Macaulay2/ 926 10. Hartshorne, R.: Algebraic Geometry. Graduate Texts in Mathematics, vol. 52. Springer, New York 927 (1977) 928 11. Hauenstein, J.D., Rodriguez, J.I., Sottile, F.: Numerical computation of Galois groups. Found. Comput. 929 Math. 2, 95 (2017) 930 12. Hauenstein, J.D., Wampler, C.W.: Unification and extension of intersection algorithms in numerical 931 algebraic geometry. Appl. Math. Comput. 293, 226–243 (2017) 932 13. Hochstenbach, M.E., Muhiˇc,A., Plestenjak, B.: On linearizations of the quadratic two-parameter Author Proof 933 eigenvalue problem. Linear Algebra Appl. 436(8), 2725–2743 (2012) 934 14. Hochstenbach, M.E., Plestenjak, B.: Backward error, condition numbers, and pseudospectra for the 935 multiparameter eigenvalue problem. Linear Algebra Appl. 375, 63–81 (2003) 936 15. Li, T.Y.: Numerical solution of multivariate polynomial systems by homotopy continuation methods. 937 In: Acta Numerica, 1997, Acta Numeric, vol. 6, pp. 399–436. Cambridge University Press, Cambridge 938 (1997) 939 16. Mazzola, G., Milmeister, G., Weissmann, J.: Comprehensive Mathematics for Computer Scientists, 940 vol. 1, 2nd edn. Universitext. Springer, Berlin (2006) 941 17. Hochstenbach, M.E., Muhiˇc,A., Plestenjak, B.: On the quadratic two-parameter eigenvalue problem 942 and its linearization. Linear Algebra Appl. 432(10), 2529–2542 (2010) 943 18. Plestenjak, B., Gheorghiu, C.I., Hochstenbach, M.E.: Spectral collocation for multiparameter eigen- 944 value problems arising from separable boundary value problems. J. Comput. Phys. 298, 585–601 945 (2015) 946 19. Pons, A.D.: Aeroelastic flutter as a multiparameter eigenvalue problem, Master’s Thesis, University 947 of Canterbury (2015). Retrieved March 9, 2021, from https://ir.canterbury.ac.nz/handle/10092/11265 948 20. Sommese, A.J., Wampler, C.W.: Exceptional sets and fiber products. Found. Comput. Math. 8(2), 949 171–196 (2008) 950 21. Sommese, A.J., Wampler, C.W.: The Numerical Solution of Systems of Polynomials. World Scientific 951 Publishing Co. Pte. Ltd., Hackensack (2005) 952 22. Sturmfels, B.: The Hurwitz form of a projective variety. J. Symbolic Comput. 79(1), 186–196 (2017) 953 23. Volkmer, H.: Multiparameter Eigenvalue Problems and Expansion Theorems. Lecture Notes in Math- 954 ematics, vol. 1356. Springer, Berlin (1988) 955 24. Zhang, T., Law, K.H., Golub, G.H.: On the homotopy method for perturbed symmetric generalized 956 eigenvalue problems. SIAM J. Sci. Comput. 19(5), 1625–1645 (1998)

957 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps 958 and institutional affiliations.

uncorrected proof

123

SPI Journal: 211 Article No.: 1215 TYPESET DISK LE CP Disp.:2021/7/29 Pages: 36 Layout: Small-Ex