Arxiv:2007.08909V2 [Math.DG] 1 Dec 2020 D but Will Not Be the Focus of This Article

MINIMALITY OF TENSORS OF FIXED MULTILINEAR RANK

ALEXANDER HEATON, KHAZHGALI KOZHASOV, AND LORENZO VENTURELLO

Abstract. We discover a geometric property of the space of tensors of fixed multilinear (Tucker) rank. Namely, it is shown that real tensors of fixed multilinear rank form a minimal submanifold of the Euclidean space of tensors endowed with the Frobenius inner product. We also establish the absence of local extrema for linear functionals restricted to the submanifold of rank-one tensors, finding application in statistics.

1. Introduction

n1 nd In the following by R ⊗ ⋅ ⋅ ⋅ ⊗ R we denote the space of real (n1, . . . , nd)-tensors whose n1 nd elements are identiﬁed with arrays T = (ti1...id ) of real numbers. The space R ⊗ ⋅ ⋅ ⋅ ⊗ R is endowed with the standard Frobenius inner product that is deﬁned by

d nj

(1.1) ⟨T,S⟩ = Q Q ti1...id si1...id , j=1 ij =1

n1 nd where T = (ti1...id ), S = (si1...id ). The multilinear rank of T ∈ R ⊗ ⋯ ⊗ R is the tuple

d nj µrank(T ) = min (r1, . . . , rd) ∈ N0 ∶ T ∈ W1 ⊗ ⋯ ⊗ Wd,Wj ⊆ R , dim Wj = rj ,

nj where N0 = {0, 1, 2,... } and each Wj is a linear subspace of R . The number rj is equal to the rank of the nj × ∏k≠j nk matrix obtained by flattening or unfolding or matricizing the tensor along mode j [14, Theorem 6.13]. In the case d = 2 the multilinear rank (r1, r2) of a matrix T satisfies r1 = r2 = rank(T ), since the rank of the row and column space of any matrix coincide. For tensors with more than 2 modes, the multilinear rank is different than the classical rank r nj rank(T ) = min r ∈ N0 ∶ T = Q v1,i ⊗ ⋯ ⊗ vd,i, vj,i ∈ R ¡ . i=1 The latter notion of rank is also important for many applications (see [15], [18, Section 3]), arXiv:2007.08909v2 [math.DG] 1 Dec 2020 d but will not be the focus of this article. For d ≥ 1 and r ∈ N0, let Td,r be the set of order d tensors with multilinear rank r. It is well-known (see, for example, [26]) that this set is a smooth submanifold of Rn1 ⊗ ⋯ ⊗ Rnd of dimension

d d dim(Td,r) = Q rj(nj − rj) + M rj. j=1 j=1

2010 Mathematics Subject Classification. 15A69, 53A10, 53A45, 49Q05. Key words and phrases. higher-order singular value decomposition (HOSVD), multilinear rank, tensor rank, Tucker decomposition, minimal surface, minimal submanifold, mean curvature. 1 The topology of Td,r was recently studied in [5]. They show, for example, that Td,r is path- connected unless nj = rj = ∏k≠j rj for some j. The notion of multilinear rank was introduced in [21] and popularized in [7] where it was used to show that the Tucker decomposition (see [25] and [18, Section 4]) provides a convincing generalization of the matrix singular value decomposition (SVD). The set Td,r and the Tucker decomposition in general have been used in numerous interesting applications, including the technique TensorFaces in computer vision [27]. Note that throughout this article we refer to the set of tensors of fixed multilinear rank, rather than the related subspace variety consisting of all tensors with µrank(T ) ≤ (r1, . . . , rd), which is also a well-studied object of interest [22, Chapter 7]. A common task is to find the best low-rank approximation of a tensor [8]. For matrices, the celebrated Eckart-Young theorem [10] states that truncating the SVD yields a closed- form solution, but for tensor rank the problem is ill-posed [9] due to the phenomenon of border rank. The subspace variety is Zariski closed, providing a nice setting for existence, but is not smooth. As a consequence, computing the best multilinear approximation over the subspace variety becomes more difficult, since not all points admit tangent spaces. In contrast, the problem of finding the best multilinear low-rank approximation is well- posed on Td,r with a unique solution with respect to the Frobenius norm [9, Corollary 4.5]. The manifold structure of low-rank tensors has been used in numerical analysis and computa- tional physics (see [13] for an overview) and, in general, methods of Riemannian optimization can be utilized [1, 11]. In recent work [20], the authors studied the problem of tensor completion, filling in the missing entries of a tensor to achieve a low-rank tensor. Tensors of fixed multilinear rank Td,r were used due to their manifold structure, implementing a ver- sion of nonlinear conjugate gradient method, see also [6]. Td,r was also used in [12, 23] to formulate the tensor approximation problem over a product of Grassmann manifolds. In [17], the manifold structure of Td,r was used to derive nonlinear differential equations whose solution is a time-varying family of tensors S(t) ∈ Td,r which provide the best multilinear rank approximation to a given family T (t) ∈ Rn1 ⊗⋯⊗Rnd , a problem called dynamical tensor approximation. Therefore the geometry of the set Td,r is important for applications. Our main result discovers a new geometric property of Td,r, namely its minimality.

d Theorem 1.1. For any r ∈ N0 the manifold Td,r is a minimal submanifold of the Euclidean space (Rn1 ⊗ ⋅ ⋅ ⋅ ⊗ Rnd , ⟨⋅, ⋅⟩), that is, its mean curvature vector ﬁeld is identically zero.

Minimal submanifolds are mathematical models of soap films: they minimize the volume locally around every point. When d = 2 the manifold T2,r, r = (r, r), consists of n1 × n2 matrices of rank r. Its minimality was recently proved in [2, 19] and for n1 = n2 = r +1 earlier in [24]. Thus Theorem 1.1 generalizes this property from matrices to higher order tensors. The case r = (1,..., 1) is of particular interest, since the elements of Td,(1,...,1) are precisely tensors of rank 1. The manifold Td,(1,...,1) is the (affine) cone over the real part of the (projective) Segre variety. In what follows we use the term “Segre variety” when referring to the manifold Td,(1,...,1) and denote it for simplicity by Sd. If we slice Td,(1,...,1) with the affine hyperplane A containing the tensors with sum of coordinates equal to 1 and consider its nonnegative part, we obtain a statistical model M which parametrizes d-tuples of in-

dependent discrete random variables (X1,...,Xd). The tensor T = (ti1...id ) expresses the 2 joint probability ti1...id = P(X1 = i1,...,Xd = id), and the condition of having rank 1 corresponds to statistical independence of X1,...,Xd. The popular Wasserstein distance between two probability distributions arises from optimal transport, measuring how much work is required to “move” one distribution to the other. In this discrete setting, the Wasserstein distance between a distribution T ∈ A and M is the smallest scaling factor for which a certain polyhedron centered at T intersects M. The closest points on M to T are those in the intersection. In [3, Section 6] it was experimentally observed that the optimal solution to this problem is never attained in the relative interior of a face of maximal dimension. One possible explanation for this experimental observation would be that the restriction of a linear functional to M attains no local minima (respectively maxima) in its relative interior. Corollary 1.4 below shows that it is indeed the case. This in turn follows from the following related result for the Segre variety Sd = Td,(1,...,1).

n1 nd Theorem 1.2. Let T ∈ Sd and let Pn⃗ = {T + V ∶ ⟨n,⃗ V ⟩ = 0} ⊂ R ⊗ ⋯ ⊗ R be an affine hyperplane containing the tangent space to Sd at T . Then no neighborhood of T in Sd is ± completely contained in one of the half spaces Pn⃗ = {T + V ∶ ±⟨n,⃗ V ⟩ ≥ 0}. In particular, the linear functional Ln⃗(S) = ⟨n,⃗ S⟩ does not attain a local maximum or minimum on Sd. A minimal surface in R3 with non-degenerate second fundamental form has saddle-like shape locally around every point. In particular, no linear functional can be minimized or maximized on such a surface. An analogous fact is true for minimal submanifolds in Rn satisfying some non-degeneracy conditions. In view of this and the minimality of Td,r (see Theorem 1.1), the assertion of Theorem 1.2 is anticipated. However to prove it, in Section 4 we deal with finer geometric information of Sd than just its mean curvature. Corollary 1.3. Let A be an affine hyperplane in Rn1 ⊗⋅ ⋅ ⋅⊗Rnd , and let L ∶ Rn1 ⊗⋅ ⋅ ⋅⊗Rnd → R be a linear functional not constant on A. Then the restriction of L to Sd ∩ A has no local minima (respectively maxima). If we take A to be the affine hyperplane of Rn1 ⊗ ⋅ ⋅ ⋅ ⊗ Rnd consisting of tensors with entries summing up to 1, we obtain the following application in statistics. Corollary 1.4. Let M be the statistical model of d independent discrete random variables. Then any linear functional which is not constant on M has no local minimum (or maximum) on the relative interior of M.

2. Preliminaries This section reviews basic concepts we will need. For more on tensors and tensor decompositions, see [18, 22], while for differential geometry see [4, 16]. We first recall a definition of the mean curvature vector field of a submanifold of an Euclidean space, see [16, Ch. VII] for more details. Let (Rn, ⟨⋅, ⋅⟩) be an Euclidean space and let M ⊂ Rn be a smooth m-dimensional submanifold. Consider a local parametrization r ∶ U → M of M, where U is an open subset of m R . The first order partial derivatives ∂u1 r(u), . . . , ∂um r(u) form a basis of the tangent space

Tr(u)M of M at r(u) and we denote by Gr(u) = ⟨∂ui r(u), ∂uj r(u)⟩ the Gram matrix of the 3 metric of M with respect to this basis. Thus, Gr(u), u ∈ U, is a smooth field of positive definite matrices along r(U) ⊂ M. The second fundamental form of M is a symmetric bilinear form b on the tangent bundle TM to M with values in the normal bundle (TM)⊥ ⊂ Rn to M defined at a point r(u) ∈ M via

m ⊥ ( ) = i j 2 ( ) (2.1) b u, v Q u v ∂uiuj r u , i,j=1

m i m j where u = ∑i=1 u ∂ui r(u), v = ∑j=1 v ∂uj r(u) ∈ Tr(u)M are arbitrary tangent vectors and ⊥ 2 ( ) ∈ ( )⊥ 2 ( ) ∈ n ∂uiuj r u Tr(u)M is the normal component of the vector ∂uiuj r u R . The mean curvature vector of M at r(u) ∈ M is deﬁned to be

m m ⊥ = −1 ( ) ( ) = −1 2 ( ) (2.2) Hr(u) Q Gr(u) b ∂ui r u , ∂uj r u Q Gr(u) ∂uiuj r u , i,j=1 ij i,j=1 ij

−1 where Gr(u) denotes the inverse to the positive definite matrix Gr(u). The definition depends only on the embedding M ⊂ V and not on the choice of the local parametrization. The smooth field Hr(u), u ∈ U, of normal vectors is the mean curvature vector field of M along the open set r(U) ⊂ M. By gluing “local” definitions of the mean curvature vector field along open sets r(U) from an open cover of M one obtains the smooth field of normal vectors called the mean curvature vector field of M. It will be convenient to adapt the following “formal” writing of (2.2)

−1 2 ⊥ (2.3) Hr(u) = Tr Gr(u) ⋅ d r(u) ,

2 ( ) ( ) 2 ( ) (⋅)⊥ where d r u is the symmetric matrix whose i, j -th entry is the vector ∂uiuj r u and is applied entry-wise. A submanifold M ⊂ Rn of the Euclidean space (Rn, ⟨⋅, ⋅, ⟩) is called minimal if the mean curvature vector field vanishes. Thus, in order to prove minimality of M ⊂ Rn one can show that for each point x ∈ M there is a local parametrization r ∶ U → M with x = r(u) and such that Hr(u) = 0. Next, we state the definition of multilinear rank of a tensor in terms of ranks of its flattening matrices. n1 nd For a tensor T = (ti1...id ) ∈ R ⊗ ⋯ ⊗ R and an index j ∈ [d] = {1, . . . , d} a j-th flattening of T is the nj × ∏k≠j nk matrix whose rows are indexed by ij ∈ [nj], columns are indexed by (d − 1)-tuples (i1, . . . , ij−1, ij+1, . . . , id) ∈ [n1] × ⋯ × [nj−1] × [nj+1] × ⋯ × [nd] (ordered in an arbitrary but a priori fixed way) and its (ij, (i1, . . . , ij−1, ij+1, . . . , id))-th entry equals ti1...ij−1ij ij+1...id . It is well known that T has multilinear rank r = (r1, . . . , rd) if and only if for j ∈ [d] the j-th flattening matrix of T has rank rj, see [14, Theorem 6.13]. Finally, note that ′ the dot products between the rows of the j-th flattening of T with indices λ, λ ∈ [nj] equals

nk ′ (2.4) Q Q ti1...ij−1λij+1...id ti1...ij−1λ ij+1...id . k∈[d]∖j ik=1 4 3. Proof of main theorem

n1 nd The product of orthogonal groups O(n) = O(n1) × ⋯ × O(nd) acts on R ⊗ ⋯ ⊗ R . More 1 d n1 nd explicitly, g = (g , . . . , g ) ∈ O(n) acts on the tensor T = (ti1...id ) ∈ R ⊗ ⋅ ⋅ ⋅ ⊗ R as

⎛ d nj ⎞ ∗ 1 d (3.1) g T = Q Q g . . . g tk ...k , ⎝ i1k1 idkd 1 d ⎠ j=1 kj =1 preserving the Frobenius inner product (1.1). This action also preserves the multilinear rank and hence restricts to an action on manifolds Td,r. In the following, we ﬁx an orthonormal j j nj ∗ basis {e1, . . . , enj } of R . In view of the above we have HT = 0 if and only if Hg T = 0 for any g ∈ O(n). In particular, since the orthogonal group O(nj) acts transitively on the nj Grassmannian Gr(rj, nj) of rj-planes in R , to prove that the mean curvature vector HT at j j T ∈ Td,r is zero we can ﬁrst assume that T ∈ W1 ⊗ ⋅ ⋅ ⋅ ⊗ Wd with Wj = Span{e1, . . . , erj },

d rj T = t e1 ⊗ ⋯ ⊗ ed . Q Q i1...id i1 id j=1 ij =1

j For 1 ≤ α, β ≤ nj let Eαβ be the (α, β)-th matrix unit of size nj × nj, 1, if α = α′ and β = β′ Ej = , αβ α′β′ 0, otherwise

j j j j and let Lαβ = Eβα − Eαβ. Skew-symmetric matrices Lαβ, 1 ≤ α < β ≤ nj, form a basis of the tangent space T1O(nj) to the orthogonal group at the identity matrix 1 ∈ O(nj). For uL L ∈ T1O(nj) let u ↦ e be the one-parameter subgroup of orthogonal matrices such that 0L d uL uL uL e = 1 and du e = Le = e L and deﬁne a family of matrices in O(nj) via

rj nj j j j uαβ Lαβ j j (3.2) g(u ) = M M e , u = (uαβ), α=1 β=rj +1

uj Lj where orthogonal matrices e αβ αβ in the product are ordered according to the order

(3.3) (1, rj + 1), (2, rj + 1),..., (rj, rj + 1), (1, rj + 2),..., (rj, rj + 2),..., (1, nj),..., (rj, nj)

j on the set of pairs (α, β). Note that the associated family g(u )Wj ⊂ Gr(rj, nj) of rj-planes contains a neighborhood of Wj ∈ Gr(rj, nj). We will need formulas for partial derivatives of (3.2). By Leibniz’s rule we obtain for 1 ≤ j ≤ d and 1 ≤ λ ≤ rj, rj + 1 ≤ µ ≤ nj

rj µ−1 λ−1 rj rj nj ∂g ⎛ uj Lj ⎞ j j uj Lj j j j ⎛ uj Lj ⎞ (uj) = M M e αβ αβ M euαµLαµ e λµ λµ L M euαµLαµ M M e αβ αβ . j ⎝ ⎠ λµ ⎝ ⎠ ∂uλµ α=1 β=rj +1 α=1 α=λ+1 α=1 β=µ+1 Evaluation of the last expression at uj = 0 gives

∂g j (3.4) j (0) = Lλµ. ∂uλµ 5 Now, the second order partial derivatives of (3.2) evaluated at uj = 0 equal 2 ∂ g j j j j ′ ′ (3.5) j j (0) = LλµLλ′µ′ = −δµµ Eλλ′ − δλλ Eµµ′ , ∂uλµ∂uλ′µ′ where (λ, µ) ≤ (λ′, µ′) with respect to the above deﬁned order. Next we deﬁne a local

parametrization of Td,r around T = (ti1...id ) by

d rj (3.6) T (u1,..., ud,S) = (t + s ) g(u1)e1 ⊗ ⋯ ⊗ g(ud)ed , Q Q i1...id i1...id i1 id j=1 ij =1

j rj (nj −rj ) r1 rd where u ∈ R , j ∈ [d], and S = (si1...id ) ∈ R ⊗ ⋯ ⊗ R together constitute a family of d d parameters of dimension ∑j=1 rj(nj − rj) + ∏j=1 rj = dim(Td,r). Note that at 0 = (0,..., 0, 0) = (u1,..., ud,S) we have T (0) = T . The ﬁrst order partial derivatives of (3.6) evaluated at 0 equal ∂T (3.7) (0) = e1 ⊗ ⋅ ⋅ ⋅ ⊗ ed i1 id ∂si1...id and ∂T d rk (0) = Q Q t e1 ⊗ ⋅ ⋅ ⋅ ⊗ ej−1 ⊗ Lj ej ⊗ ej+1 ⊗ ⋅ ⋅ ⋅ ⊗ ed j i1...id i1 ij−1 λµ ij ij+1 id ∂u k=1 i =1 (3.8) λµ k rk = Q Q t e1 ⊗ ⋅ ⋅ ⋅ ⊗ ej−1 ⊗ ej ⊗ ej+1 ⊗ ⋅ ⋅ ⋅ ⊗ ed , i1...ij−1λ ij+1...id i1 ij−1 µ ij+1 id k∈[d]∖j ik=1 where in (3.8) we use (3.4) and the formula ⎪⎧ ej , if i = λ ⎪ µ j Lj ej = ⎨ −ej , if i = µ . λµ ij λ j ⎪ ⎩⎪ 0, otherwise

Note that λ ∈ {1, 2, . . . , rj}, µ ∈ {rj + 1, . . . , nj}, and the sum in (3.8) in the ij-th index runs over ij ∈ {1, 2, . . . , rj} so that ij = λ is the only relevant case. It immediately follows from (3.7) and (3.8) that ∂T ∂T (3.9) d (0), (0)i = δi1j1 . . . δidjd , ∂si1...id ∂sj1...jd ∂T ∂T (3.10) d (0), (0)i = 0, ∂s j i1...id ∂uλµ ∂T ∂T rk d ( ) ( )i = ′ ′ ′ (3.11) j 0 , j′ 0 δjj δµµ Q Q ti1...ij−1λij+1...id ti1...ij−1λ ij+1...id , ∂uλµ ∂uλ′µ′ k∈[d]∖j ik=1 ′ for every (i1, . . . , id), (j1, . . . , jd) ∈ [r1] × ⋅ ⋅ ⋅ × [rd], j, j ∈ [d], (λ, µ) ∈ [rj] × {rj + 1, . . . , nj} and ′ ′ (λ , µ ) ∈ [rj′ ] × {rj′ + 1, . . . , nj′ }. With these formulas on hand we compute the Gram matrix GT =T (0). The rows and columns of this symmetric dim(Td,r) × dim(Td,r) matrix are indexed by the local coordinates in the parametrization (3.6), and each entry of GT is the inner product of the corresponding partial derivatives of (3.6) evaluated at (u1,..., ud,S) = 0. 6 First, formulas (3.9), (3.10) and (3.11) imply that GT has a block diagonal structure, with the d + 1 many blocks given as ⎛ ⎞ ⎜ ⎟ ⎜G ∂T 0 ⋯ 0 0 ⎟ ⎜ (0) ⎟ ⎜ ∂u1 ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ 0 G ∂T ⋱ ⋱ ⋮ ⎟ ⎜ 2 (0) ⎟ ⎜ ∂u ⎟ ⎜ ⎟ ⎜ ⎟ = ⎜ ⎟ GT ⎜ ⋮ ⋱ ⋱ ⋱ ⋮ ⎟ . ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ 0 ⋱ ⋱ G ∂T 0 ⎟ ⎜ d (0) ⎟ ⎜ ∂u ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ 0 ⋯ ⋯ 0 G∂T ⎝ ∂S (0)⎠ ′ Entries of the block G ∂T (0), j ∈ [d], are inner products (3.11) with j = j , whereas the last ∂uj d d block G ∂T ( ) of inner products (3.9) is the (∏j=1 rj × ∏j=1 rj) identity matrix. If we order ∂S 0 ′ the rows and columns of the block G ∂T (0) according to (3.3), then (3.11) with j = j implies ∂uj that G ∂T (0) has a further block structure, ∂uj

⎛ ⋯ ⎞ ⎜ Aj 0 0 ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ 0 A ⋱ ⋮ ⎟ ⎜ j ⎟ G ∂T (0) = ⎜ ⎟ . ∂uj ⎜ ⎟ ⎜ ⎟ ⎜ ⋮ ⋱ ⋱ 0 ⎟ ⎜ ⎟ ⎜ ⎟ ⎝ 0 ⋯ 0 Aj ⎠

More precisely, G ∂T (0) is a block diagonal (rj(nj − rj) × rj(nj − rj)) matrix, with nj − rj ∂uj ′ ′ identical blocks Aj. It follows from (3.11) with j = j , µ = µ and (2.4) that the rj × rj matrix Aj is the Gram matrix of the rows of the j-th ﬂattening of the tensor T . Since the j-th entry rj in the multilinear rank r = (r1, . . . , rj, . . . , rd) of T ∈ Td,r equals the rank of the j-th ﬂattening (see Section2), the matrix Aj and hence G ∂T (0) are nonsingular. Repeating the ∂uj same argument for every block shows that also GT is nonsingular and hence vectors (3.7) and (3.8) form a basis of the tangent space of Td,r at T .

The block diagonal structure of GT and of G ∂T (0), j ∈ [d], shows that the inverse matrix ∂uj −1 GT is also block diagonal, with blocks of the same size. In particular we have −1 (GT )αβ = 0, in the following cases: 7 j - α corresponds to a parameter si1...id and β corresponds to a parameter uλµ. ′ j j ′ - α corresponds to a parameter uλµ and β corresponds to a parameter uλ′µ′ , with j ≠ j or with j = j′ but µ ≠ µ′. Next we concentrate on the computation of the mean curvature vector at the point T . By (2.3) we need to show that

−1 2 ⊥ (3.12) HT = Tr GT ⋅ d T (0) = 0,

2 ⊥ where (d T (0)) is the dim(Td,r)×dim(Td,r) matrix whose entries are normal components of second order partial derivatives of the parametrization (3.6) evaluated at 0 = (u1,..., ud,S). −1 Taking into account the above discovered structure of GT , we note that the only second order derivatives of (3.6) that are relevant for the computation of HT are ∂2T (0) = 0, (i1, . . . , id), (j1, . . . , jd) ∈ [r1] × ⋅ ⋅ ⋅ × [rd], ∂si1...id ∂sj1...jd and ∂2T d rk ∂2g (3.13) (0) = Q Q t e1 ⊗ ⋯ ⊗ ej−1 ⊗ (0)ej ⊗ ej+1 ⊗ ⋯ ⊗ ed , j j i1...id i1 ij−1 j j ij ij+1 id ∂uλµ∂uλ′µ k=1 ik=1 ∂uλµ∂uλ′µ ′ where j ∈ [d], λ, λ ∈ [rj] and µ ∈ {rj + 1, . . . , nj}. Using (3.5) we obtain 2 ∂ g j j j j j (0)e = −δ ′ E − E ′ e = −δ ′ e j j ij λλ µµ λλ ij ij λ λ ∂uλµ∂uλ′µ and applying it to (3.13) gives

2 rk ∂ T 1 j−1 j j+1 d (0) = − Q Q t ′ e ⊗ ⋯ ⊗ e ⊗ e ⊗ e ⊗ ⋯ ⊗ e . j j i1...ij−1λ ij+1...id i1 ij−1 λ ij+1 id ∂uλµ∂uλ′µ k∈[d]∖j ik=1 Although this last computation reveals a nonzero vector we observe that all the indices (i1, . . . , ij−1, λ, ij+1, . . . , id) appearing in its expression range in [r1] × ⋯ × [rj] × ⋯ × [rd]. This implies that (3.13) lies in the tangent space TT Td,r, and hence its normal component ⊥ ⎛ ∂2T ⎞ (0) is zero. Thus, (3.12) holds and this completes the proof. ⎝ j j ⎠ ∂uλµ∂uλ′µ 4. Linear optimization over the Segre variety

In this section we consider the Segre variety, i.e., the manifold Sd = Td,(1,...,1) of tensors of order d and multilinear rank (1,..., 1). Our goal is to prove Theorem 1.2. Let T ∈ Sd and let Pn⃗ be any aﬃne hyperplane containing the tangent space to Sd at T . We will show that in any neighborhood of T ∈ Sd there are rank-one tensors lying on both sides of Pn⃗. + − + − To do so, we construct two curves γ , γ ∶ (−ε, ε) → Sd passing through T = γ (0) = γ (0) and such that ±⟨γ±(u) − T, n⃗⟩ > 0 for 0 < u < ε. The last condition simply means that the rank-one tensors γ+(u) and γ−(u) lie in the opposite open half-spaces formed by the aﬃne hyperplane Pn⃗. Note that the restriction of the action (3.1) of O(n) = O(n1) × ⋅ ⋅ ⋅ × O(nd) to Sd is transitive. This together with the fact that (3.1) preserves the inner product (1.1) 1 d imply that it is enough to prove the statement when T = e1 ⊗ ⋅ ⋅ ⋅ ⊗ e1 ∈ Sd. 8 Before we proceed with the construction of curves γ+ and γ−, let us investigate the structure 1 d of the normal space NT Sd to the Segre variety Sd at T = e1 ⊗ ⋅ ⋅ ⋅ ⊗ e1 ∈ Sd. We have an orthogonal direct sum decomposition: ⊥ ⊥ ⊥ NT Sd = N2 ⊕ N3 ⊕ ⋯ ⊕ Nd, where

1 d ij ⊥ Nk = Span {e1 ⊗ ⋯ ⊗ wi1 ⊗ ⋯ ⊗ wik ⊗ ⋯ ⊗ e1 ∶ 1 ≤ i1 < ⋯ < ik ≤ d, wij ∈ (e1 ) }. With these notations we may also write 1 d N0 = Span {e1 ⊗ ⋯ ⊗ e1} = Span {T } and 1 i−1 i+1 d i ⊥ N1 = Span {e1 ⊗ ⋯ ⊗ e1 ⊗ wi ⊗ e1 ⊗ ⋯ ⊗ e1 ∶ 1 ≤ i ≤ d, wi ∈ (e1) }, where N0 +N1 = TT Sd is the tangent space to Sd at T . It is straightforward to check that the subspaces Nk, k = 0, . . . , d, are pairwise orthogonal with respect to the inner product (1.1). By assumption the vector n⃗ belongs to the normal space NT Sd. Let k ≥ 2 be the smallest natural number such that n⃗ has nonzero component in Nk. For example, if k = 2, the vector 1 i1 i2 n⃗ has nonzero coeﬃcient in front of one of the basic tensors e1 ⊗ ⋯ ⊗ eα1 ⊗ ⋯ ⊗ eα2 ⊗ ⋯ ⊗ e1 for α1, α2 > 1 and some indices 1 ≤ i1 < i2 ≤ d. We ﬁrst prove an auxiliary lemma.

Lemma 4.1. Let n⃗ have nonzero coeﬃcient c(n,⃗ α1, . . . , αk) ≠ 0 in front of the basic tensor e1 ⊗ ⋯ ⊗ ek ⊗ ek+1 ⊗ ⋯ ⊗ ed ∈ N , where α , . . . , α > 1. Then the curve α1 αk 1 1 k 1 k 1 k uL1,α 1 uL1,α k k+1 d (4.1) γ = {(e 1 e1) ⊗ ⋯ ⊗ (e k e1) ⊗ e1 ⊗ ⋯ ⊗ e1 ∶ u ∈ (−ε, ε)} ⊂ Sd (j) (k) satisﬁes ⟨γ (0), n⃗⟩ = 0 for all j ∈ {0, 1, . . . , k − 1} and ⟨γ (0), n⃗⟩ = k! c(n,⃗ α1, . . . , αk). Proof: This follows from Leibniz’ rule. Indeed we have that 1 d γ(0) = e1 ⊗ ⋯ ⊗ e1 = T ∈ N0 k ′( ) = 1 ⊗ ⋯ ⊗ j ⊗ ⋯ ⊗ d ∈ γ 0 Q e1 eαj e1 N1 j=1 ⋮ γ(k)(0) = W + k! e1 ⊗ ⋯ ⊗ ek ⊗ ek+1 ⊗ ⋯ ⊗ ed, α1 αk 1 1 ⊥ ⊥ ⊥ with W ∈ N0 ⊕ N1 ⊕ ⋯⊕ Nk−1. The condition n⃗ ∈ Nk, orthogonality of N0,N1,...,Nk−1,Nk and orthogonality of basic tensors imply the claim. ◻ We now proof the main result of this section.

⊥ ⊥ Proof of Theorem 1.2: Let n⃗ ∈ Nk ⊕ ⋅ ⋅ ⋅ ⊕ Nd, k ≥ 2, and γ ∶ (−ε, ε) → R be as in Lemma 4.1. The Taylor expansion u2 uk−1 uk γ(u) = γ(0) + γ′(0)u + γ′′(0) + ⋯ + γ(k−1)(0) + γ(k)(0) + ⋯ 2! (k − 1)! k! and Lemma 4.1 imply that the (local) position of γ(u) (for small u > 0) with respect to (k) Pn⃗ = {T + V ∶ ⟨n,⃗ V ⟩ = 0} is determined by the sign of ⟨γ (0), n⃗⟩ = k! c(n,⃗ α1, . . . , αk). If 9 + − − c(n,⃗ α1, . . . , αk) > 0 deﬁne γ to be γ, while if c(n,⃗ α1, . . . , αk) < 0 set γ = γ. The curve γ (respectively γ+) is then deﬁned to be

1 2 k −uL1,α 1 uL1,α 2 uL1,α k k+1 d γ˜ = {(e 1 e1) ⊗ (e 2 e1) ⊗ ⋯ ⊗ (e k e1) ⊗ e1 ⊗ ⋯ ⊗ e1 ∶ u ∈ (−ε, ε)} ⊂ Sd. Note that the single difference is in the sign in front of the parameter u in the first tensor factor. This change in the sign is needed to flip the sign of the inner product ⟨γ(k)(0), n⃗⟩ as (j) (k) ⟨γ˜ (0), n⃗⟩ = 0 for j = 0, 1, . . . , k − 1 and ⟨γ˜ (0), n⃗⟩ = −k! c(n,⃗ α1, . . . , αk). This computation follows exactly the same path as the proof of Lemma 4.1. In both cases it follows from the definition of γ± and Taylor expansion that ±⟨γ±(u) − T, n⃗⟩ > 0 for small enough u > 0. ⊥ ⊥ In general, if n⃗ ∈ Nk ⊕ ⋅ ⋅ ⋅ ⊕ Nd has a non-zero coefficient in front of the basic tensor 1 i1 ik d e1 ⊗ ⋅ ⋅ ⋅ ⊗ eα1 ⊗ ⋅ ⋅ ⋅ ⊗ eαk ⊗ ⋅ ⋅ ⋅ ⊗ e1 ∈ Nk with 1 ≤ i1 < ⋅ ⋅ ⋅ < ik ≤ d and α1, . . . , αk > 1, one has to consider the curve i1 ik 1 uL1,α i1 uL1,α ik d γ = { e1 ⊗ ⋅ ⋅ ⋅ ⊗ (e 1 e1 ) ⊗ ⋯ ⊗ (e k e1 ) ⊗ ⋅ ⋅ ⋅ ⊗ e1 ∶ u ∈ (−ε, ε)} ⊂ Sd instead of (4.1). All other steps are identical. The above proves that there are tensors in an arbitrarily small neighborhood of T ∈ Sd that lie on either side of the affine hyperplane Pn⃗. In particular, the restriction of a linear functional to Sd attains no local minima or maxima.

We now derive Corollary 1.3, which asserts that (when nonconstant) the restriction of a linear functional to the aﬃne-linear slice Sd ∩ A of the Segre variety has no local minima or maxima as well.

Proof of Corollary 1.3: Let A be the affine hyperplane {T ∈ Rn1 ⊗ ⋅ ⋅ ⋅ ⊗ Rnd ∶ ⟨T, a⃗⟩ = c}. Observe first that the Segre variety Sd is conical, i.e., λT ∈ Sd for any T ∈ Sd and any λ ∈ R ∖ {0}. Assume that T ∈ Sd ∩ A is a local minimum (maximum) for the restriction of a linear functional Ln⃗(V ) = ⟨n,⃗ V ⟩ on Sd ∩ A. Since the restriction of Ln⃗ to A is assumed to be nonconstant, we have in particular that n⃗ is not proportional to a⃗. Then n,⃗ a⃗ span a two-dimensional subspace, which must intersect any hyperplane. Therefore there exists µ ∈ R with v⃗ = n⃗ + µa⃗ orthogonal to T . If A is a linear hyperplane (c = 0), then ⟨n,⃗ T ⟩ = 0 holds automatically because in this case Sd ∩ A remains conical and T ∈ TT (Sd ∩ A). Note that the coefficient of n⃗ can be assumed equal to 1 since any rescaling of n⃗ would produce another linear functional which also has T as a local minimum (maximum) on Sd ∩ A. The condition Lv⃗(T ) = 0 implies that T ∈ Sd ∩ A is a local minimum (maximum) of Lv⃗ restricted to Sd. This contradicts to Theorem 1.2.

Observe that, while Corollary 1.3 shows that the absence of local minima and maxima is preserved under aﬃne hyperplane sections, the same is not true for minimality. Figure1 depicts the mean curvature ﬁeld (blue vectors) for (the black pointed surface) 2 2 S2 ∩ A, A = {S = (sij)i,j=1,2 ∈ R ⊗ R ∶ s11 + s12 + s21 + s22 = 1}.

Since this vector ﬁeld is evidently nonzero, this surface S2 ∩ A is not minimal. If we restrict ourselves to tensors with nonnegative entries we obtain a statistical independence model, as explained in the introduction. This is precisely the portion of S2 ∩ A captured in Figure1. We make the code to compute the mean curvature for this example available at 10 Figure 1. Mean curvature vector ﬁeld on M, the independence model for two binary discrete random variables.

https://mathrepo.mis.mpg.de/tensorminimality.

References [1] P.-A. Absil, R. Mahony, and R. Sepulchre. Optimization algorithms on matrix manifolds. Princeton University Press, Princeton, NJ, 2008. [2] M. Bordemann, J. Choe, and J. Hoppe. The minimality of determinantal varieties. arXiv:2003.00233. [3]T. O.¨ Çelik, A. Jamneshan, G. Montúfar,B. Sturmfels, and L. Venturello. Wasserstein distance to independence models. Journal of Symbolic Computation, 104:855–873, 2021. [4] I. Chavel. Riemannian geometry: a modern introduction, volume 98 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, second edition, 2006. [5] P. Comon, L.-H. Lim, Y. Qi, and K. Ye. Topology of tensor ranks. Adv. Math., 367:107–128, 2020. [6] C. Da Silva and F. J. Herrmann. Optimization on the hierarchical Tucker manifold – applications to tensor completion. Linear Algebra Appl., 481:131–173, 2015. [7] L. De Lathauwer, B. De Moor, and J. Vandewalle. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl., 21(4):1253–1278, 2000. [8] L. De Lathauwer, B. De Moor, and J. Vandewalle. On the best rank-1 and rank-(R1,R2, ⋯,RN ) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl., 21(4):1324–1342, 2000. [9] V. de Silva and L.-H. Lim. Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Anal. Appl., 30(3):1084–1127, 2008. [10] C. Eckart and G. M. Young. The approximation of one matrix by another of lower rank. Psychometrika, 1:211–218, 1936. [11] A. Edelman, T. A. Arias, and S. T. Smith. The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl., 20(2):303–353, 1999. [12] L. Eldénand B. Savas. A Newton-Grassmann method for computing the best multilinear rank-(r1, r2, r3) approximation of a tensor. SIAM J. Matrix Anal. Appl., 31(2):248–271, 2009. [13] L. Grasedyck, D. Kressner, and C. Tobler. A literature survey of low-rank tensor approximation tech- niques. GAMM-Mitt., 36(1):53–78, 2013. [14] W. Hackbusch. Tensor Spaces and Numerical Tensor Calculus, volume 42. Springer, 01 2012. 11 [15] F. L. Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics, 6(1-4):164–189, 1927. [16] Sh. Kobayashi and K. Nomizu. Foundations of Differential Geometry, volume 2. Interscience publishers, 1969. [17] O. Koch and C. Lubich. Dynamical tensor approximation. SIAM J. Matrix Anal. Appl., 31(5):2360– 2375, 2010. [18] T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAM Rev., 51(3):455–500, 2009. [19] Kh. Kozhasov. On minimality of determinantal varieties. arXiv:2003.01049. [20] D. Kressner, M. Steinlechner, and B. Vandereycken. Low-rank tensor completion by Riemannian optimization. BIT Numer Math., 54(2):447–468, 2014. [21] J. B. Kruskal. Rank, decomposition, and uniqueness for 3-way and N-way arrays. In Multiway data analysis (Rome, 1988), pages 7–18. North-Holland, Amsterdam, 1989. [22] J. M. Landsberg. Tensors: geometry and applications, volume 128 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2012. [23] B. Savas and L.-H. Lim. Quasi-Newton methods on Grassmannians and multilinear approximations of tensors. SIAM J. Sci. Comput., 32(6):3352–3393, 2010. [24] V. Tkachev. Minimal cubic cones via Clifford algebras. Complex Anal. Oper. Theory, 4:685–700, 2010. [25] L. R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31:279–311, 1966. [26] A. Uschmajew and B. Vandereycken. The geometry of algorithms using hierarchical tensors. Linear Algebra Appl., 439(1):133–166, 2013. [27] M. Vasilescu, Alex O., and D. Terzopoulos. Multilinear analysis of image ensembles: Tensorfaces. In Proceedings of the 7th European Conference on Computer Vision-Part I, ECCV ’02, pages 447–460, Berlin, Heidelberg, 2002. Springer-Verlag.

Email address: [email protected], [email protected] Max Planck Institute for Mathematics in the Sciences, Leipzig, Technische Universitat¨ Berlin

Email address: [email protected] Technische Universitat¨ Braunschweig, Institut fur¨ Analysis und Algebra

Email address: [email protected], [email protected] Max Planck Institute for Mathematics in the Sciences, Leipzig