Chapter 7 Proximity problems In the “extremely large-scale case” (N of order of tens and hundreds of thousands), [iteration cost O(N 3)] rules out all advanced convex optimization techniques, including all known polynomial time algorithms. Arkadi Nemirovski, 2004 ¸ − ¥ ¦ A problem common to various sciences is to find the Euclidean distance matrix (EDM) D EDMN closest in some sense to a given complete matrix of measurements H under a ∈ § constraint on affine dimension 0 r N 1 (§2.3.1, 5.7.1.1); rather, r is bounded above by desired affine dimension ρ . ≤ ≤ − 7.0.1 Measurement matrix H N N Ideally, we want a given matrix of measurements H R × to conform with the first ∈ three Euclidean metric properties (§5.2); to belong to the intersection of the orthant N N N of nonnegative matrices R+ × with the symmetric hollow subspace Sh (§2.2.3.0.1). Geometrically, we want H to belong to the polyhedral cone (§2.12.1.0.1) N N N , S R × (1463) K h ∩ + Yet in practice, H can possess significant measurement uncertainty (noise). Sometimes realization of an optimization problem demands that its input, the given matrix H , possess some particular characteristics; perhaps symmetry and hollowness or nonnegativity. When that H given does not have the desired properties, then we must impose them upon H prior to optimization: When measurement matrix H is neither symmetric or hollow, taking its symmetric hollow part is equivalent to orthogonal projection on the symmetric hollow N subspace Sh . When measurements of distance in H are negative, zeroing negative entries effects N N unique minimum-distance projection on the orthant of nonnegative matrices R+ × 2 N in isomorphic R (§E.9.2.2.3). Dattorro, Convex Optimization Euclidean Distance Geometry 2ε, εβoo, v2018.09.21. 457 M 458 CHAPTER 7. PROXIMITY PROBLEMS 7.0.1.1 Order of imposition Since convex cone (1463) is the intersection of an orthant with a subspace, we want to project on thatK subset of the orthant belonging to the subspace; on the nonnegative orthant in the symmetric hollow subspace that is, in fact, the intersection. For that reason alone, unique minimum-distance projection of H on (that member of closest to H 2 K K in isomorphic RN in the Euclidean sense) can be attained by first taking its symmetric hollow part, and only then clipping negative entries of the result to 0 ; id est, there is only one correct order of projection, in general, on an orthant intersecting a subspace: project on the subspace, then project the result on the orthant in that subspace. (confer §E.9.5) In contrast, order of projection on an intersection of subspaces is arbitrary. That order of projection rule applies more generally, of course, to intersection of any convex set with any subspace. Consider the proximity problem7.1 over convex feasible N C N N set S given nonsymmetric nonhollow H R × : h ∩ C ∈ 2 minimize B H F B SN k − k ∈ h (1464) subject to B ∈ C N a convex optimization problem. Because the symmetric hollow subspace Sh is orthogonal N N N to the antisymmetric antihollow subspace R × ⊥ (§2.2.3), then for B S h ∈ h 1 tr BT (H HT) + δ2(H) = 0 (1465) 2 − µ µ ¶¶ so the objective function is equivalent to 1 2 1 2 B H 2 B (H + HT) δ2(H) + (H HT) + δ2(H) (1466) k − kF ≡ − 2 − 2 − ° µ ¶°F ° °F ° ° ° ° ° ° ° ° This means the antisymmetric° antihollow part° of given° matrix H would be° ignored by minimization with respect to symmetric hollow variable B under Frobenius’ norm; id est, minimization proceeds as though given the symmetric hollow part of H . This action of Frobenius’ norm (1466) is effectively a Euclidean projection N (minimum-distance projection) of H on the symmetric hollow subspace Sh prior to minimization. Thus minimization proceeds inherently following the correct order for N N projection on Sh . Therefore we may either assume H Sh , or take its symmetric hollow part prior to∩ C optimization. ∈ 7.0.1.2 Flagrant input error under nonnegativity demand More pertinent to the optimization problems presented herein where N N N N , EDM = S R × (1467) C ⊆ K h ∩ + then should some particular realization of a proximity problem demand input H be nonnegative, and were we only to zero negative entries of a nonsymmetric nonhollow input H prior to optimization, then the ensuing projection on EDMN would be guaranteed incorrect (out of order). 7.1 There are two equivalent interpretations of projection ( §E.9): one finds a set normal, the other, minimum distance between a point and a set. Here we realize the latter view. 459 .......................................................... .................................. .................................. ..................... ..................... ................. ................. ............... ✧❜ ............... ............ ............ ............ ............ .......... ✧ ❜ .......... .......... .......... ......... ✧ ❜ ......... ......... ......... ........ ✧ ❜ ........ ....... ....... ....... ✟ ....... ....... ✧ ❜ ....... ....... ✟ ...... ...... ✧ ✟ ❜ ...... ...... ...... ...... ✧ ✟ ❜ ...... ..... ..... ..... ✟ ..... ..... ✧ ❜ .... .... ✟ .... .... ✧ ❜ .... .... ✟ ... ... ✧ ❜ .... ... ✟ ... ... ✟ ... ... ✧ ❜ ... ... ✟ .. .. ✧ ❜ .. ... ✟ .. .. ✧ ❤❤ ❜ .. .. 0 ❤❤❤ .. ✧ ❝ ❤❤ N ❜ .. ❤❤ . ❤❤ EDM . ✧ N ❝ ❤❤❤ ❜ . ❤❤ . ❜ S ❤❤ ✧ . h ❝ ❤ . .. ❜ ✧ .. .. .. .. ❝ .. .. ❜ ✧ ... .. ❝ ... ... ❜ ✧ ... ... ... ... ❜ ❝ ✧ ... .... .... .... ❜ ✧ .... .... ❝ .... ..... ..... .... ❜ N N N ✧ .... ..... ❝ ..... ..... ❜ = Sh R+ × ✧ ..... ...... ..... ...... ❜ ❝K ∩ ✧ ...... ...... ...... ...... N ....... ....... ❜ ❝ ✧ ....... ....... ....... .......S ❜ ✧ ....... ........ ❝ ........ ......... ❜ ✧ ........ ........ ......... .......... ❝ .......... .......... ❜ ✧ .......... ........... ........... ............ ❜ ✧ ............ .............. .............. ............... ................ ................... ❜✧ ................... N N .......................... .......................... R × ..................................................................... ....................... Figure 179: Pseudo-Venn diagram: EDM cone EDMN belongs to intersection of symmetric hollow subspace with nonnegative orthant; EDMN (1048). EDMN cannot exist outside N N N ⊆ K of Sh , but R+ × does. 0 H N EDMN Sh N N N = S R × K h ∩ + Figure 180: Pseudo-Venn diagram from Figure 179 showing elbow placed in path of N N projection of H on EDM Sh by an optimization problem demanding nonnegative input matrix H . The first two⊂ line segments, leading away from H , result from correct order of projection required to provide nonnegative H prior to optimization. Were H N nonnegative, its projection on Sh would instead belong to ; making the elbow disappear. (confer Figure 197) K 460 CHAPTER 7. PROXIMITY PROBLEMS Now comes a surprising fact: Even were we to correctly follow the order of projection rule so as to provide H prior to optimization, then the ensuing projection on EDMN will be incorrect whenever∈ K input H has negative entries and some proximity problem demands nonnegative input H . This is best understood referring to Figure 179: Suppose nonnegative input H is N demanded, and then the problem realization correctly projects its input first on Sh and then directly on = EDMN . That demand for nonnegativity effectively requires imposition of on inputC H prior to optimization so as to obtain correct order of projection N K N (on Sh first). Yet such an imposition prior to projection on EDM generally introduces an elbow into the path of projection (illustrated in Figure 180) caused by the technique itself; that being, a particular proximity problem realization requiring nonnegative input. Any procedure, for imposition of nonnegativity on input H , can only be incorrect in this circumstance. There is no resolution unless input H is guaranteed nonnegative with no tinkering. Otherwise, we have no choice but to employ a different problem realization; one not demanding nonnegative input. 7.0.2 Least lower bound Most of the problems we encounter in this chapter have the general form: minimize B A F B k − k (1468) subject to B ∈ C m n where A R × is given data. This particular objective denotes Euclidean projection ∈ (§E) of vectorized matrix A on the set which may or may not be convex. When is convex, then projection is unique minimum-distanceC because Frobenius’ norm squareC is a strictly convex function of variable B and because the optimal solution is the same regardless of the square (524). When is a subspace, then the direction of projection is orthogonal to . C C T T Denoting by A, UA ΣAQA and B , UBΣBQB their full singular value decompositions (whose singular values are always nonincreasingly ordered (§A.6)), there exists a tight lower bound on the objective over the manifold of orthogonal matrices; ΣB ΣA F inf B A F (1469) k − k ≤ UA ,UB , QA , QB k − k m n This least lower bound holds more generally for any orthogonally invariant norm on R × § § (§2.2.1) including the Frobenius and spectral norm [370, II.3]. [233, 7.4.51] 7.0.3 Problem approach. stress/sstress problems traditionally posed in terms of point position x Rn, i=1 ...N { i ∈ } 2 minimize ( xi xj hij ) (1470) xi k − k − { } i , j X∈ I 2 2 minimize ( xi xj hij ) (1471) xi k − k − { } i , j X∈ I (where is an abstract set of indices and hij is given data) are everywhere converted herein toI the distance-square variable D or to Gram matrix G ; the Gram matrix acting as bridge between position and distance. (That conversion is performed regardless of whether known data is complete.) Then the techniques of chapter 5 or chapter 6 are applied to find relative or