The cut locus and distance function from a closed subset of a Finsler manifold ∗† Minoru TANAKA · Sorin V. SABAU Abstract We characterize the differentiable points of the distance function from a closed subset N of an arbitrary dimensional Finsler manifold in terms of the number of N-segments. In the case of a 2-dimensional Finsler manifold, we prove the structure theorem of the cut locus of a closed subset N, namely that it is a local tree, it is made of countably many rectifiable Jordan arcs except for the endpoints of the cut locus and that an intrinsic metric can be introduced in the cut locus and its intrinsic and induced topologies coincide. We should point out that these are new results even for Riemannian manifolds. 1 Introduction Among the variational problems, there is an interesting problem which is called the Zer- melo navigation problem: Find the paths of shortest travel time from an originating point to a destination under the influence of a wind or a current when we travel by boat capable of a certain maximum speed. The shortest paths of this problem are geodesics of a Finsler metric. Notice that the shortest paths are geodesics of a Riemannian metric only when there is no wind and no current. Hence, if we restrict ourselves to the variational problems of a Riemannian manifold, we must exclude such an interesting variational problem. This is the reason why we are interested in the variational problems of a Finsler manifold. arXiv:1207.0918v3 [math.DG] 15 Dec 2012 Any geodesic γ emanating from a point p in a compact Riemannian manifold looses the minimizing property at a point q on γ. Such a point q is called a cut point of p along γ. The cut locus of a point p is the set of all cut points along geodesics emanating from p. The cut locus often appears as an obstacle when we try to prove some global structure theorems of a Riemannian manifold. For example, Ambrose’s problem is easily solved if the cut locus of the base point is empty. Hebda ([H]) and Itoh ([I]) solved this problem affirmatively and independently in the 2-dimensional case, by proving that the 1- dimensional Hausdorff measure of the cut locus of a point in an arbitrary compact subset ∗Mathematics Subject Classification (2010) : 53C60, 53C22. †Keywords: non-reversible Finsler surfaces, N-segment, cut locus, local tree, distance function. 1 is finite. It is still open for an arbitrary dimensional Riemannian manifold. The cut locus is also a vital notion in analysis, where the cut locus appears as a singular set. In fact, the cut locus of a point p in a complete Riemannian manifold equals the closure of the set of all non-differentiable points of the distance function from the point p. By being motivated by optimal control problems in space and quantum dynamics, the joint work ([BCST]) was accomplished by Bonnard, Caillau, Sinclair and Tanaka. In this paper, the structure of the cut locus was determined for a class of 2-spheres of revolution which contains oblate ellipsoids, and this structure theorem gives global optimal results in orbital transfer and for Lindblad equations in quantum control. The following property of the cut locus has played a crucial role in optimal transport problems (see [V1]). The distance function to the cut locus of a point p of a complete Riemannian manifold is locally Lipschitz on the unit sphere in the tangent space at p. This property is often applied in many papers of optimal transport problems: For example see Loeper-Villani ([LV]), Figalli-Rifford ([FR]), Figalli-Rifford-Villani ([FRV1], [FRV2], [FRV3], [FRV4]), Figalli-Villani ([FV]) and Villani ([V2]). Since H. Poincar´eintroduced the notion of the cut locus in 1905, the cut locus of a point or a submanifold in a Riemannian manifold has been investigated by many researchers ( [B]). In spite of this fact, any structure theorem of the cut locus has not been established yet except for special Riemannian manifolds. The main difficulty of formulating and proving such a theorem lies in the fact that the cut locus can be as complicated as a fractal set. In fact, Gluck and Singer ([GS]) constructed a smooth 2-sphere of revolution with positive Gaussian curvature admitting a point whose cut point is non-triangulable. Moreover, it is conjectured that the Hausdorff dimension of the cut locus would be a non-integer, if the manifold is not smooth enough. However, it has been shown that the Hausdorff dimension of the cut locus of a point is an integer for a smooth Riemannian manifold (see [ITd]) and that the distance function to the cut locus of a closed submanifold is locally Lipschitz for a smooth Riemannian manifold as well as for the Finslerian case (see [IT], [LN]). Hence, the cut locus has enough differentiability for nonsmooth analysis ([Cl]). Nevertheless, if we restrict ourselves to a surface, the structure theorem for the Rie- mannian cut locus has been established. Indeed, the detailed structure of the cut locus of a point or a smooth Jordan arc in a Riemannian 2-manifold have been thoroughly investigated (see [SST], [H]). For example, Hebda proved in [H] that the cut locus Cp of a point p in a complete 2-dimensional Riemannian manifold has a local-tree structure and that any two cut points of p can be joined by a rectifiable arc in the cut locus Cp if these two cut points are in the same connected component. Here, a topological space X is called a local tree if for any x ∈ X and any neighborhood U of x, there exists an open neighborhood V ⊂ U of x such that any two points in V can be joined by a unique continuous arc. In the present paper, the structure theorem of the cut locus of a closed subset of a Finsler surface will be proved. It should be noted that the investigations of the cut locus of a closed subset are scarce even in the case of a Riemannian manifold. We will also investigate the differentiability of the distance function from a closed subset of an 2 arbitrary dimensional Finsler manifold. It is well-known that the differentiability of the distance function is closely related to the cut locus. For example, it is known that the squared distance function from a point p in a complete Riemannian manifold is differentiable at a point q if and only if there exists a unique minimal geodesic segment joining p to q and that the squared distance function is smooth outside of the cut locus of p. One of our main theorems (Theorem A) generalizes the facts above. Let N be a closed subset of a forward complete Finsler manifold (M, F ). Roughly speaking, the Finsler manifold is a differentiable manifold with a norm on each tangent space. The precise definition of the Finsler manifold and some necessary fundamental notation and formulas will be reviewed later. A (unit speed) geodesic segment γ : [0, a] → M is called an N-segment if d(N,γ(t)) = t holds on [0, a], where d(N,p) := min{ d(q,p) | q ∈ N} for each point p ∈ M. If a non- constant unit speed N-segment γ : [0, a] → M is maximal as an N-segment, then the endpoint γ(a) is called a cut point of N along γ. The cut locus CN of N is the set of all cut points along all non-constant N-segments. One of our main theorems is on the distance function from a closed subset of a Finsler manifold. The research of the distance function dN (·) := d(N, ·) from the closed subset N is fundamental in the study of variational problems. For example, the viscosity solution of the Hamilton-Jacobi equation is given by the flow of the gradient vector of the distance function dN , when N is the smooth boundary of a relatively compact domain in Euclidean space (see [LN]). Although we do not assume any differentiability condition for the closed subset N ⊂ M, we may prove the following remarkable result. Theorem A Let N be a closed subset of a forward complete arbitrary dimensional Finsler manifold (M, F ). Then, the distance function dN from the subset N is differ- entiable at a point q ∈ M \ N if and only if q admits a unique N-segment. Theorem B and Theorem C are our main theorems on the cut locus. The theorems corresponding to Theorems B and C have been proved in [ShT] for the cut locus of a compact subset of an Alexandrov surface. We should point out that the Toponogov comparison theorem was a key tool for proving main theorems in [ShT], but the Toponogov comparison theorem does not hold for Finsler manifolds. Hence, completely different proofs will be given to Theorems B and C. Theorem B Let N be a closed subset of a forward complete 2-dimensional Finsler man- ifold (M, F ). Then, the cut locus CN of N satisfies the following properties: 1. CN is a local tree and any two cut points on the same connected component of CN can be joined by a rectifiable curve in CN . 2. The topology of CN induced from the intrinsic metric δ coincides with the topology induced from (M, F ). 3. The space CN with the intrinsic metric δ is forward complete. 3 4. The cut locus CN is a union of countably many Jordan arcs except for the endpoints of CN . Theorem C There exists a set E ⊂ [0, sup dN ) of measure zero with the following prop- erties: −1 1.