Sub-Riemannian Geometry on Some Step-Two Carnot Groups

Sub-Riemannian geometry on some step-two Carnot groups

Hong-Quan Li, Ye Zhang

Abstract. This paper is a continuation of the previous work of the first author. We characterize a class of step-two groups introduced in [88], saying GM-groups, via some basic sub-Riemannian geometric properties, including the squared Carnot-Carathéodory distance, the cut locus, the classical cut locus, the optimal synthesis, etc. Also, the shortest abnormal set can be exhibited easily in such situation. Some examples of such groups are step-two groups of corank 2, of Kolmogorov type, or those associated to quadratic CR manifolds. As a byproduct, the main goal in [19] is achieved from the setting of step-two groups of corank 2 to all possible step-two groups, via a completely different method. A partial answer to the open questions [20, (29)-(30)] is provided in this paper as well. Moreover, we provide a entirely different proof, based yet on [88], for the Gaveau-Brockett optimal control problem on the free step-two Carnot group with three generators. As a byproduct, we provide a new and independent proof for the main results obtained in [103], namely, the exact expression of d(g)2 for g belonging to the classical cut locus of the identity element o, as well as the determination of all shortest geodesics joining o to such g. Mathematics Subject Classification (2010): 22E25, 53C17, 53C22 Key words and phrases: Carnot-Carathéodory distance, Gaveau-Brockett optimal control problem, step-two Carnot group, cut locus, shortest geodesic, optimal synthesis

1 Introduction

arXiv:2102.09860v1 [math.DG] 19 Feb 2021 In the past several decades, step-two groups and their sub-Laplacians, as special Lie groups of polynomial volume growth or perfect sub-Riemannian manifolds, have attracted wide attention from experts in various ﬁelds, such as complex analysis, control theory, geometric measure theory, harmonic analysis, heat kernel, Lie group theory, probability analysis, PDE, sub-Riemannian geometry, etc. We only mention some relevant works here, [4–18,20–28,30,32,34–48,50–53,55–63,65–72,74–76,78,80–87,90,91,96,97,103,105, 114, 118, 123, 124, 131, 132]. The list is far from exhaustive and in fact is rather limited. More related papers can be found in the references therein as well as their subsequent researches. In this present paper, we will restrict our attention to the sub-Riemannian geometry on step-two Carnot groups. Many relevant works can be found in the literature as cited before. However, some most fundamental problems are far from being solved, or even

1 poorly known, in this very fine framework. Recently, in [88] (cf. also [89]), the first author used Loewner’s theorem to study two basic problems of sub-Riemannian geometry on 2-step groups: one is to obtain the exact formula for the sub-Riemannian distance, that is the Gaveau-Brockett optimal control problem; another is to characterize all (shortest) normal geodesics from the identity element o to any given g 6= o. In particular, there exists an enormous class of 2-step groups, saying GM-groups (see Subsection 2.3 below for the definition), which have some consummate sub-Riemannian geometric properties. More precisely, the squared Carnot-Carathéodory distance d(g)2 := d(o, g)2 and the cut locus 2 of o, Cuto (namely the set of points where d is not smooth) can be characterized easily in such situation. For example, all Heisenberg groups even generalized Heisenberg-type groups (so step-two groups of corank 1), and star graphs are GM-groups. We emphasize that in general, the expression of d(g)2 is extremely complicated. It is impossible to provide an explicit expression, via a relatively simple inverse function, as in the most special situation of generalized Heisenberg-type groups (cf. [63], [27] and [88]). We refer the reader to [88] for more details. The work is a continuation of [88], one of our main goals is to provide various equivalent characterizations of GM-groups via basic sub-Riemannian geometric properties. Moreover, the sub-Riemannian geometry in the setting of 2-step groups is not well understood. Roughly speaking, the main reason for this is that the well-understood examples are merely the Heisenberg group (cf. [63]) and generalized Heisenberg groups (cf. [27]). Other known cases, such as generalized Heisenberg-type groups as well as the direct product of a generalized Heisenberg group with a Euclidean space (in particular, step-two groups of corank 1), are essentially the same. Hence, it is very meaningful and exigent to supply some examples possessing richer sub-Riemannian geometric properties. Here, we will provide more examples of GM-groups, such as groups of corank 2, of Kolmogorov type, or those associated to quadratic CR manifolds. In particular, the aforementioned groups ∗ may have complicated shortest abnormal set of o, Abno, that is, the set of the endpoints of abnormal shortest geodesics starting from o. The existence of non-trivial abnormal shortest geodesics is closely related to the regularity of the Carnot-Carathéodory distance. And its appearance makes an obstacle for us to deal with some topics, such as the heat kernel asymptotics and geometric inequalities, etc. See for example [2, 20, 29, 30, 88, 105] and the references therein for more details. Recall that (cf. [118,119]) a sub-Riemannian manifold is called ideal if it is complete and has no non-trivial abnormal shortest geodesics. In our setting, a step-two group G is ideal if and only if it is of Métiviertype (see Subsection 2.2.1 for the definition). Optimal syntheses (namely the collection of all arclength parametrized geodesics with their cut times) are generally very difficult to obtain. In the setting of step-two groups, as far as we know, a correct result about them can be found only on nonisotropic Heisenberg groups. See [2, § 13] and Remark 1 below for more details. However, we can now give the optimal synthesis from the identity element o on GM-groups. As a result, the classical CL cut locus of o, Cuto , that is the set of points where geodesics starting at o cease to be shortest, can be characterized on such groups as well. 2 We say that d is semiconcave (resp. semiconvex) in a neighborhood of g0 if there exist

2 C > 0 and δ > 0 such that

0 2 0 2 2 0 2 0 2 d(g1 + g ) + d(g1 − g ) − 2 d(g1) ≤ C |g | (resp. ≥ − C |g | ), (1.1) DSSCC

0 q m for all g1 ± g ∈ B(g0, δ) = {g ∈ R × R ; |g − g0| < δ}. Here we stress that | · | denotes 0 the usual Euclidean norm and g1 ± g the usual operation in the Euclidean space. We also remark that this deﬁnition is independent of the choice of local coordinates around 2 g0 (here we use the canonical one) since d is locally Lipschitz w.r.t. the usual Euclidean distance (see for example [118,119]). Set in the sequel1

− 2 SCo := g; d fails to be semiconcave in any neighborhood of g , (1.2) DoFSC1 + 2 SCo := g; d fails to be semiconvex in any neighborhood of g . (1.3) DoFSC2

− ∗ Recall that SCo = Abno in the setting of Métiviergroups, all free Carnot groups of step 2, as well as some other sub-Riemannian structures. See [35], [54], [102], [20, § 4.1 and § 4.2] and references therein for more details. And an open problem is raised in [20, (29)], − ∗ which asks whether it holds SCo = Abno in the more general sub-Riemannian setting ∗ ∗ (where our Abno is noted by Abn(o)). In the framework of GM-groups, Abno can be described easily; as a byproduct, we give a positive answer to this open problem. Also, other related results and step-two groups can be found in Subsection 2.5. In addition, the most challenging problem should be to study the sub-Riemannian ∼ k k(k−1) geometry in the setting of free step-two groups with k generators Nk,2 = R × R 2 (k ≥ 3). Indeed, for any step-two group G with k generators, that is the first layer in the stratification of Lie algebra has dimension k, there exists some relation between G and Nk,2 by Rothschild-Stein lifting theorem (see [124] or [32]). Observe that N2,2 is exactly the Heisenberg group, which is well-known (cf. [63] or [27]). Recall that (cf. [63] and [34]) the original Gaveau-Brockett optimal control problem is to determine the sub- Riemannian distance on Nk,2 with k ≥ 3. This is a long-standing open problem. Recently, it is completely solved on N3,2 in [88, § 11]. Also remark that the main idea and method in [88] can be adapted to general step-two groups and other situations. CL In the setting of N3,2, the classical cut locus of o, Cuto , has been determined in [113] and [103] by completely different techniques; furthermore, the expression of d(g)2 with CL g ∈ Cuto has been obtained in [103]. Strictly speaking, we have used the above known ∗ CL results in the proof of [88]. Also notice that Abno and Cuto on N3,2 are relatively very simple. However, it is still an open problem to characterize the classical cut locus of o on Nk,2 with k ≥ 4, see [123] for more details. Motivated by this problem, we ask naturally 2 CL if we can determine first d(g) for any g then Cuto on N3,2. This is exactly another main purpose of this work. In the framework of step-two groups, first we recall that all shortest geodesics are normal (cf. [3] or [119, § 2.4]). Next, up to a subset of measure zero, all normal geodesics

1We would like to thank L. Rizzi for informing us of the addendum of [20] that the deﬁnition of the failure of semiconcavity/semiconvexity for the open questions should be the one stated here in our situation (which is consist with the classical deﬁnition of local semiconcavity/semiconvexity), rather than the one given in [20]. For more details, we refer to the addendum on L. Rizzi’s homepage.

3 from o to any given g 6= o have been characterized by [88, Theorem 2.4]. Moreover, it follows from [88, Theorem 2.5] that the squared distance has been determined in a symmetric, scaling invariant subset with non-empty interior. In particular, for the special case of N3,2, we can simplify the Gaveau-Brockett problem via an orthogonal-invariant property, and some useful results can be found in [88, § 11]. Based on these known results, we can describe the squared distance on N3,2 first on some dense open subset, then on whole space via a limiting argument. Then, from the regularity of the squared sub- Riemannian distance, we can further determine the cut locus Cuto. Finally, all shortest CL geodesics joining o to any given point in Cuto are obtained by approximating them c with that joining o to some points in (Cuto) , which are relatively easy to describe. As a consequence, we supply an independent and new proof for the main results obtained in [103]. Some applications will be given in a future work. This paper is organized as follows. In Section 2, we collect some preliminary materials and give our main results, which will be proven in Section 3. In Section 4, we provide a sufficient condition for a step-two group to be GM-group by using semi-algebraic theory. As a consequence, we find that all step-two groups of corank 2 are GM-groups. Furthermore, we also prove in this section that there exist Métiviergroups of corank 3 and of sufficiently large dimension which are not of GM-type. In Section 5, we consider the sub-Riemannian geometry in the setting of step-two K-type groups. Step-two groups associated to quadratic CR manifolds will be studied in Section 6. Finally, we give in Section 7 a completely different proof, based on [87], for the Gaveau-Brockett optimal control problem on N3,2. As an application, we provide a new and independent proof for the main results obtained in [103].

2 Preliminaries and main results s2 2.1 Step-two Carnot groups s21 Recall that a connected and simply connected Lie group G is a step-two Carnot group if its left-invariant Lie algebra g admits a stratiﬁcation

g = g1 ⊕ g2, [g1, g1] = g2, [g1, g2] = {0},

where [·, ·] denotes the Lie bracket on g. We identify G and g via the exponential map. As a result, G can be considered as Rq × Rm, q, m ∈ N∗ = {1, 2, 3,...} (in this paper we use N to denote the set of natural numbers {0, 1, 2,...}), with the group law 1 (x, t) · (x0, t0) = x + x0, t + t0 + h x, x0i , g := (x, t) ∈ q × m, 2 U R R

where

0 (1) 0 (m) 0 m hUx, x i := (hU x, x i,..., hU x, x i) ∈ R .

4 Here U = {U (1),...,U (m)} is an m-tuple of linearly independent q × q skew-symmetric matrices with real entries and h·, ·i (or · in the sequel when there is no ambiguity) denotes the usual inner product on Rq. Furthermore, in this article, we will not distinguish row vectors from column vectors and we may write a column vector t with scalar coordinates t1, . . . , tm, simply as (t1, . . . , tm) unless otherwise stated in the context. Note that m ≤ q(q−1) 2 . We call such a group a step-two group of type (q, m, U), which is denoted by G(q, m, U) or G for simplicity. One can refer to [49] or [32] for more details. (j) (j) Let U = (Ul,k )1≤l,k≤q (1 ≤ j ≤ m). The canonical basis of g1 is deﬁned by the left-invariant vector ﬁelds on G: m q ∂ 1 X X (j) ∂ Xl(g) := + Ul,k xk , 1 ≤ l ≤ q. ∂xl 2 ∂tj j=1 k=1

q P 2 And the canonical sub-Laplacian is ∆ = Xl . l=1

2.2 Left-invariant sub-Riemannian geometry on G: some elementary properties Let us ﬁrst recall some basic facts about the sub-Riemannian geometry in the framework of 2-step groups. In our setting, we will sometimes use equivalent deﬁnitions for some concepts in order to avoid recalling too many notations. We refer the reader to [2,28–30, 105, 119, 129] and references therein for further details. Also notice that partial but not all results below remain valid in some more general setting. The group G = G(q, m, U) is endowed with the sub-Riemannian structure, namely a scalar product on g1, with respect to which {Xl}1≤l≤q are orthonormal (and the norm induced by this scalar product is denoted by k · k). In the sequel, m is called the corank of G(q, m, U). A horizontal curve γ : [0, 1] → G is an absolutely continuous path such that

q X γ˙ (s) = uj(s)Xj(γ(s)) for a.e. s ∈ [0, 1], j=1 and we deﬁne its length as follows v Z 1 Z 1 u q uX 2 `(γ) := kγ˙ (s)k ds = t |uj(s)| ds. 0 0 j=1

The Carnot-Carath´eodory (or sub-Riemannian) distance between g, g0 ∈ G is then d(g, g0) := inf {`(γ); γ(0) = g, γ(1) = g0, γ horizontal} .

A geodesic is a horizontal curve γ satisfying: kγ˙ (s)k is constant and for any s0 ∈ [0, 1] there exists a neighborhood I of s0 in [0, 1] such that `(γ|I ) is equal to the distance

5 between its endpoints. And a shortest geodesic is a geodesic γ which realizes the distance between its extremities, that is, `(γ) = d(γ(0), γ(1)). By slightly abusing of notation in the sequel, 0 denotes the number 0 or the origin in the Euclidean space. Let o = (0, 0) denote the identity element of G. It is well-known that d is a left-invariant distance on G. Hence we set in the following d(g) := d(g, o). Recall that d2 is locally Lipschitz on G with respect to the usual Euclidean distance. The dilation on G is deﬁned by 2 δr(x, t) := (r x, r t), ∀ r > 0, (x, t) ∈ G. (2.1) nDS And the following scaling property is well-known:

2 d(r x, r t) = r d(x, t), ∀ r > 0, (x, t) ∈ G. (2.2) scap

2.2.1 Sub-Riemannian Hamiltonian and normal geodesics starting from o ns221 In the setting of step-two Carnot groups, it is well-known that all shortest geodesics are projections of normal Pontryagin extremals, that is integral curves of the sub-Riemannian Hamiltonian in T ∗G. See for example [3, § 20.5] or [119, Theorem 2.22]. ∗ ∼ q m q m More precisely, the sub-Riemannian Hamiltonian in T G = (R × R ) × (R × R ) is deﬁned by

q m q ! 1 X 1 X X (k) H = H(x, t, ξ, τ) := ζ2, ζ := ξ + U x τ , 1 ≤ j ≤ q. 2 j j j 2 j,l l k j=1 k=1 l=1

And a normal Pontryagin extremal,

∗ γ(s) := (x(s), t(s)), ξ(s), τ(s) : [0, 1] −→ T G, with γ(0) = o,

is a solution of

∂H ∂H ˙ ∂H ∂H x˙ k = , t˙j = , ξk = − , τ˙j = − , 1 ≤ k ≤ q, 1 ≤ j ≤ m. (2.3) HJE ∂ξk ∂τj ∂xk ∂tj

The covector (ξ(0), τ(0)) (resp. (ξ(1), τ(1))) is called the initial (resp. ﬁnal) covector of (γ(s), ξ(s), τ(s)). Its projection

γ(s) := γ(ξ(0), τ(0); s) = γ(ξ(0),τ(0))(s) = (x(s), t(s)) : [0, 1] −→ G is said to be the normal geodesic starting from o with initial covector (ξ(0), τ(0)). Note that H is independent of t. Hence we have

m τ(s) ≡ τ(0) := 2 θ ∈ R . (2.4) Set in the following

m X (j) m Ue(θ) := θj U and U(θ) := i Ue(θ), for θ = (θ1, . . . , θm) ∈ R . (2.5) Du1 j=1

6 Recall that a step-two group G is a M´etiviergroup (or of M´etiviertype) if U(θ) is invertible for any θ 6= 0 (cf. [98]). Let ζ(s) := ξ(s) + Ue(θ) x(s). Remark that ξ(0) = ζ(0). A simple calculation implies that Z s Z s 2 s U(θ) 1 ζ(s) = e e ζ(0), x(s) = ζ(r) dr, t(s) = hUx(r), ζ(r)i dr. (2.6) GEn 0 2 0 In particular, we have

Z 1 sin U(θ) x(1) = ζ(r) dr = eUe(θ) ζ(0), (2.7) endpointx 0 U(θ)

and γ(ξ(0), τ(0); s) = (x(s), t(s)) is extendable and real analytic on [0, +∞). It is easy to check the following homogeneity property:

q m γ(α ζ0, 2 α θ0; s) = γ(ζ0, 2 θ0; α s), ∀ α > 0, s ≥ 0, (ζ0, 2 θ0) ∈ R × R .

From now on, the domain of the normal geodesic γ(s) = γ(ζ0, 2 θ0; s) is [0, +∞) and

that of γ(s) = γ(ζ0,2 θ0)(s) is [0, 1] by default. Also remark that γ = o if ζ0 = 0, which is trivial. And all normal geodesics are by convention starting from o in this work. Let (x, t) denote the endpoint of γ(ζ,τ), then by (2.6), that of γ(−ζ,τ) is (−x, t). More-

over, both γ(ζ,τ) and γ(−ζ,τ) have length |ζ|, where | · | denotes the usual Euclidean norm. Combining this with the fact that the Carnot-Carath´eodory distance is a left-invariant distance on G, we have the following simple but useful observation: Lemma 1. In the setting of step-two groups, it holds that

d(x, t) = d(−x, t) = d(x, −t) = d(−x, −t), ∀ (x, t) ∈ G. (2.8) te1

The following basic property is well-known:

0 0 BPG Lemma 2. Let 0 ≤ s1 < s2. Assume that γ(ζ0, 2 θ0; s) = γ(ζ , 2 θ ; s) for all s1 ≤ s ≤ s2. 0 0 0 Then we have γ(ζ0, 2 θ0; ·) ≡ γ(ζ , 2 θ ; ·). Moreover, it holds that ζ0 = ζ . More information about such geodesics can be found in Proposition 1 below.

2.2.2 Sub-Riemannian exponential map, cut point and optimal synthesis The sub-Riemannian exponential map based at o is the smooth map deﬁned by

q m exp : R × R −→ G

(ζ0, 2 θ0) 7−→ γ(ζ0, 2 θ0; 1).

In our setting, it is surjective and has the following property:

q m γ(ζ0, 2 θ0; s) = exp{s (ζ0, 2 θ0)}, ∀ s ≥ 0, (ζ0, 2 θ0) ∈ R × R .

See for example [2, § 8.6]. Furthermore, we have the following simple observations:

7 Lemma 3. Suppose that exp(w, τ) = (x, t). Then we have

exp(r w, τ) = (r x, r2 t), ∀ r 6= 0, (2.9) symN2

exp(−eUe(τ) w, −τ) = (−x, −t). (2.10) symN3

Indeed, using (2.6), (2.9) is trivial, and an elementary computation implies (2.10). Now assume that γ(s) = exp{s (ζ0, 2 θ0)} is parametrized by arclength (or arclength parametrized), namely |ζ0| = 1. Let g0 = γ(s0). We say that g0 is conjugate to o along γ if s0 (ζ0, 2 θ0) is a critical point of exp. The cut time along γ is deﬁned as

hcut := hcut(γ) = sup{s > 0; γ|[0, s] is a shortest geodesic}. (2.11)

When hcut < +∞, γ(hcut) is said to be the cut point of o along γ. And we say γ has no cut point if hcut = +∞. The optimal synthesis from o is the collection of all arclength parametrized geodesics with their cut times.

2.2.3 Shortest abnormal set and cut locus ns223 A normal geodesic is said to be abnormal (i.e. singular) if it has two (so infinitely many) different normal lifts (see [120, Remark 8] and [119, Remark 2.4]). However, we stress that our definition of abnormal geodesic is not complete in general. In particular, on some sub-Riemannian manifolds, excluding our step-two groups, there are shortest geodesics which are not projections of normal Pontryagin extremals. For the original definition of abnormal geodesic as well as counter-examples, we refer the reader to [2,92,104,105,119] and the references therein for more details. In this work, the (normal-) abnormal set of o, Abno is defined by

Abno := {g; there exists an abnormal (which is also normal) geodesic joining o to g}. And we deﬁne the shortest abnormal set of o as follows: ∗ Abno := {g; there exists an abnormal shortest geodesic joining o to g},

which is a subset of Abno. The main difference between the two sets is that: in the definition of Abno, we do not care about minimality of geodesics, while this is needed in ∗ ∗ nor that of Abno. Notice that o ∈ Abno. Also remark that our Abno is exactly Abn (e) in [82, § 2.7]. The following characterization of abnormal geodesics, which can be also considered as an improvement of Lemma 2, can be easily verified by (2.6) (see also [102, § 3.1.1] for an explanation from the original definition of abnormal (-normal) geodesics).

cAg Proposition 1. Let θ 6= θ0. Then γ(w, 2 θ; ·) ≡ γ(w, 2 θ0; ·) if and only if for σ = θ − θ0 ∈ Rm \{0}, we have k U(σ) U(θ) w = 0, ∀ k ∈ N, (2.12) CharaAbn or equivalently,

s U(θ) U(σ) e e w = 0, ∀ s ∈ R. (2.13) AGc

8 That is, γ(w,2 θ) (or γ(w, 2 θ; ·)) is abnormal if and only if there exists some σ 6= 0 such that (2.12) (or equivalently (2.13)) satisﬁes. As a consequence, we get the following known fact:

Corollary 1. If γ(w,2 θ) is abnormal, then so does γ(a w,2 b θ) for any a, b ∈ R. In particular, for any 0 < a < 1, the restriction of γ(w,2 θ) in [0, a], γ(w,2 θ)|[0, a] = γ(a w,2 a θ) is also abnormal.

A normal geodesic is called strictly normal if it is not abnormal. Let γ = γ(w,2 θ) (resp.

γ(w, 2 θ; ·)) and 0 ≤ s1 < s2 ≤ 1 (resp. 0 ≤ s1 < s2 < +∞). We consider the restriction

of γ in [s1, s2], γ|[s1, s2] as well as

s1,s2 −1 γ (s) := γ(s1) · γ(s1 + s(s2 − s1)), s ∈ [0, 1]. By (2.6), a simple calculation shows that

s1,s2 γ = γ 2 s U(θ) , ((s2−s1) e 1 e w, 2 (s2−s1) θ) which is a normal geodesic starting from o. If γs1,s2 is abnormal, then it follows from Proposition 1 that there exists a σ ∈ Rm \{0} such that

k+1 k 2 s1 Ue(θ) (s2 − s1) U(σ) U(θ) e w = 0, ∀ k ∈ N,

which implies that γ = γ(w,2 θ) itself (so γ(w, 2 θ; ·)) is also abnormal by (2.13). We say a s1,s2 normal geodesic γ(w,2 θ) (resp. γ(w, 2 θ; ·)) does not contain abnormal segments if γ is

not abnormal for any 0 ≤ s1 < s2 ≤ 1 (resp. 0 ≤ s1 < s2 < +∞). In conclusion, we get the following: nLN Lemma 4. In the framework of step-two groups, any strictly normal geodesic does not contain abnormal segments. It is worthwhile to point out that the above property is no longer valid in general. See [99] for more details.

In this paper, the cut locus of o, Cuto, is deﬁned as c 2 ∞ Cuto := S , with S := {g; d is C in a neighborhood of g}. (2.14) DCUT Recall that (see for example [2, § 11.1]) S = {g; there exists a unique shortest geodesic γ from o to g, which is not abnormal, and g is not conjugate to o along γ}, (2.15) CCL1

and it is open and dense in G. Hence Cuto is closed. Furthermore, it has measure zero ∗ (cf. [118, Proposition 15]). Remark also that o ∈ Abno ⊆ Cuto. CL The classical cut locus of o, Cuto is defined as the set of points where geodesics starting at o cease to be shortest, that is CL Cuto := {g; g is the cut point of o along some arclength parametrized normal geodesic}. Now, we can give an affirmative answer to the open question [20, first part of (30)] in our framework, which follows from (2.15), Lemma 4 and [2, Theorem 8.72].

9 CL ∗ t2 Theorem 1. In the setting of step-two Carnot groups, it holds that Cuto = Cuto ∪Abno.

2.3 Notations and results from [88] s22 Let us begin by recalling the initial reference set and the reference function, introduced in [88], which are deﬁned respectively by m 2 2 m Ω∗ := τ ∈ R ; maxhU(τ) x, xi < π = {τ ∈ R ; kU(τ)k < π}, (2.16) oa |x|=1

φ(g; τ) = hU(τ) cot U(τ) x, xi + 4 t · τ, τ ∈ Ω∗, g = (x, t) ∈ G. (2.17) RFn Notice that the function φ(g; ·) is well-deﬁned provided the spectrum of U(τ) does

not contain any k π (k ∈ Z \{0}). Also, we will use its usual extension on Ω∗ (which is denoted by φ(g; ·) as well). And we have eP1 Proposition 2 ( [88], Proposition 2.1 and Remark 2.1). For any g, φ(g; ·) is smooth and

concave in Ω∗. Moreover, for every g, there exists an θg ∈ Ω∗ such that

φ(g; θg) = sup φ(g; τ). τ∈Ω∗

∂ ∂ m Let ∇θ = ,..., denote the usual gradient on . Recall that (cf. [88, 2]) ∂θ1 ∂θm R § 1 q e := x, − ∇θhU(θ) cot U(θ) x, xi ; x ∈ , θ ∈ Ω∗ , (2.18) nOM M 4 R

which is the union of disjoint and nonempty subsets

M := {g; ∃ θ ∈ Ω∗ s.t. θ is a nondegenerate critical point of φ(g; ·) in Ω∗}

= {g; ∃ θ ∈ Ω∗ s.t. the set of global maximizers of φ(g; ·) in Ω∗ is {θ}} , (2.19) m

and

Me 2 := {g; the set of global maximizers of φ(g; ·) in Ω∗ has at least two points}. (2.20) dtM2

∗ Also recall that M is an open set, M ⊆ S, o ∈ Me 2 ⊆ Abno ⊆ Cuto and (cf. [88, § 2])

2 2 d(g) = max φ(g; τ) for g ∈ Me , and d(g) = sup φ(g; τ) for g ∈ Me . (2.21) Csrd τ∈Ω∗ τ∈Ω∗ And we have the following

q RLT Theorem 2 ( [88], Theorems 2.4 and 2.5). Assume that ζ0 ∈ R \{0} and θ0 ∈ Ω∗. Then exp{(ζ0, 2 θ0)} = g0 := (x0, t0) if and only if U(θ ) −1 1 0 −Ue(θ0) x0 = e ζ0, t0 = − ∇θhU(θ0) cot U(θ0) x0, x0i. sin U(θ0) 4

10 Furthermore, in such case, we have

2 2 2 U(θ0) d(g0) = |ζ0| = x0 = φ(g0; θ0), sin U(θ0)

and the unique shortest geodesic from o to g0 is exp{s (ζ0, 2 θ0)} (0 ≤ s ≤ 1), which is strictly normal if and only if g0 ∈ M. As a consequence, we yield immediately

Nc1 Corollary 2. Let γ(s) := exp{s (ζ0, τ0)} be an arclength parametrized geodesic, that is |ζ0| = 1. Then its cut time hcut = +∞ if τ0 = 0, and in such case γ is a ray in the ﬁrst layer. In addition, we have hcut ≥ 2π/kU(τ0)k when τ0 6= 0.

Rkn1 Remark 1. It follows from [88, Proposition 5.1 and/or Corollary 2.2] that if G is not of m M´etiviertype, then there exist 0 6= θ0 ∈ R and 0 6= x0 ∈ ker Ue(θ0) such that |x0| = 1 and exp{s (x0, θ0)} = exp{s (x0, 0)} for all s > 0. Hence the cut time of exp{s (x0, θ0)} is equal to +∞ and the statement of [19, Theorems 6 and 7] is misleadingly phrased. However, a correct statement and their generalization can be found in Theorem 4, Corollaries 8 or 9 below. Also notice that our method to determine the cut time is completely diﬀerent from theirs.

Another easy but very useful consequence is the following:

CL c nCcc Corollary 3. It holds that Cuto ⊆ Me . Combining [88, Proposition 5.1 (b)] with Theorem 2 as well as Proposition 1, we have ∗ the following characterization of Me 2, Abno and Abno: NPA1 Proposition 3. It holds that:

n 2π o Me 2 = γ(s) = γ(ζ, τ; s); γ is abnormal, |ζ| = 1 and 0 ≤ s < kU(τ)k , ∗ Abno = {γ(s) = γ(ζ, τ; s); γ is abnormal, |ζ| = 1 and 0 ≤ s ≤ hcut(γ)} ,

Abno = {γ(s) = γ(ζ, τ; s); γ is abnormal, |ζ| = 1, s ≥ 0} .

Remark 2. Obviously, the arclength parametrized geodesic γ(ζ0, τ0; s) is abnormal if and

only if there exists a s0 > 0 such that kU(s0 τ0)k < 2π and exp{s0(ζ0, τ0)} ∈ Me 2.

∗ In order to describe Abno and Abno, it suﬃces to determine Me 2, which is much less diﬃcult. Moreover, we have

q M2AbsAb Corollary 4. In the setting of step-two groups, if Me 2 ⊆ {(x, 0); x ∈ R }, then Abno = ∗ Abno = Me 2.

11 q q nRna Remark 3. (a) Recall that R × {0} ⊆ Me . If (x0, 0) ∈ Abno for some x0 ∈ R , then we ∗ have also (x0, 0) ∈ Me 2 ⊆ Abno. ∗ q 3 2 (b) In general, Abno = Abno does not imply Me 2 ⊆ R × {0}. In fact, let H = R × R 3 3 ∼ 4 2 denote the Heisenberg group of real dimension 3 and consider G = H × H = R × R . We have ∗ 3 3 Abno = Abno = {oH3 } × H ∪ H × {oH3 } , but 4 {o} ∪ ({oH3 } × MH3 ) ∪ (MH3 × {oH3 }) = Me 2 6⊆ R × {0}, 2 where MH3 = {(x, t); x ∈ R \{0}} and oH3 denote the corresponding set M and identity element in the setting of H3 respectively. ∗ (c) The example in (b) also provides a group on which Me 2 $ Abno since ((0, 0, 1), oH3 ) ∈ ∗ 3 3 ∗ Abno \ Me 2 ⊆ H × H = G. Furthermore, another example of Me 2 $ Abno can be found ∗ in the proof of Proposition 7 (see Subsection 6.2 below), and that of Abno $ Abno in Subsection 6.3 below. ∗ (d) Obviously, a step-two group is of M´etiviertype if and only if Abno = {o}.

Recall that the nonempty open subset M ⊂ G is the set of points, g, where the reference function φ(g; ·) has a nondegenerate critical point in the initial reference set Ω∗. Observe that M is symmetric and scaling invariant; namely, if g ∈ M, then we have −1 g = −g ∈ M and δr(g) ∈ M for all r > 0. A step-two group G is said to be a GM-group (or of type GM ) if it satisﬁes M = G. (GM)

Notice that if both G1 and G2 satisfy (GM), then so does the direct product G1 ×G2. See Appendix B for more details. Also remark that GM groups form a wild set. Indeed, for any given G(q, m, U), we can construct an uncountable number of GM-groups G(q+2n, m, Ue). See [88, § 8.1] for more details. Recall that the global reference set is a compact set in Rm deﬁned by (cf. [88, § 2.6]) 1 R := R, with R := θ = ∇ d(g)2; g = (x, t) ∈/ Cut open. (2.22) GRs 4 t o

It follows from [102, § 3] or [88, Proposition 5.2] that R ∩ Ω∗ is dense in Ω∗. Then

R ⊇ Ω∗ = {θ; kU(θ)k ≤ π} . (2.23) grinc

Set

m ∗ V := {ϑ ∈ R ; det(kπ − U(ϑ)) 6= 0, ∀ k ∈ N }

and

q c W := exp(R × (2 V )), (2.24) nW

12 which is the set of the endpoints of “bad” normal geodesics, where “bad” normal geodesic c (resp. “good” normal geodesic) means γ = γ(w,2 θ) with θ ∈ V (resp. θ ∈ V). It is clearly that W is of measure zero. Finally some notations of special functions related to −s cot s are also recalled:

2s − sin (2s) f(s) f(s) := 1 − s cot s, µ(s) := f 0(s) = , ψ(s) := . (2.25) EFs 2 sin2 s s2

2.4 Main results

Our ﬁrst result is the following:

2.4.1 Properties of the global reference set R The following theorem should be useful to determine all shortest geodesics in the setting of step-two groups.

t1 Theorem 3. Let o 6= g ∈ G, and γg(s) (0 ≤ s ≤ 1) be a shortest geodesic joining o to g. q Then there exist ζ ∈ R with |ζ| = d(g) and θ ∈ R such that γg(s) = exp{s (ζ, 2 θ)} for all 0 ≤ s ≤ 1.

The following property is a direct consequence of (2.15) together with Lemma 4 and [2, Theorem 8.72]:

bLn Lemma 5. Suppose that g0 = exp{(ζ0, τ0)} ∈ S and exp{s (ζ0, τ0)} (0 ≤ s ≤ 1) is the −1 unique shortest geodesic between o and g0. Then exp{s (ζ0, τ0)} ∈ S and 2 s τ0 ∈ R for all 0 < s ≤ 1.

c nRnM Remark 4. Recall that Me = M∪Me 2 and Me 2 ⊆ Cuto = S . By the fact that exp{(ζ, 2 τ)} ∈ Me whenever τ ∈ Ω∗, a direct consequence of Lemma 5 is that M 6= ∅. See also [97, Propo- sition 6 and § 3] for another explanation. It follows from Lemma 5 that R is star-shaped w.r.t. the origin 0, that is, if τ ∈ R, then we have s τ ∈ R for all s ∈ [0, 1]; in particular, it is path connected. Furthermore, it is the smallest compact set which satisﬁes the property in Theorem 3. Also notice that Lemma 5 provides a theoretical basis for the method proposed in [88, § 11] to determine the squared sub-Riemannian distance for general non-GM groups.

RKn2 Remark 5. Theorem 3 could be considered as a somewhat converse statement of [120, Proposition 4] in our setting. In fact, from the proof of Theorem 3, we know that there +∞ exist {gj}j=1 ⊆ S with γ(ζ(j),2 θ(j)) the shortest geodesic joining o to gj such that gj → g and (ζ(j), 2 θ(j)) → (ζ, 2 θ) as j → +∞. As a result, every shortest geodesic can be induced by some limiting sub-diﬀerential in our situation.

Combining this with another basic property of R, namely [88, Corollary 2.4], this is why it is called the global reference set.

13 2.4.2 Other sub-Riemannian geometric properties on step-two groups Let us begin with an upper bound about the cut time of an arclength parametrized geodesic:

c1 Corollary 5. Let

m CR := max kU(τ)k, Cτ := sup {s > 0; r τ ∈ R, ∀ 0 ≤ r ≤ s} (τ ∈ R ). τ∈R

For any arclength parametrized geodesic γ(s) = exp{s (ζ, τ)} = γ(ζ, τ; s), its cut time satisﬁes

m hcut ≤ sup {2 Cσ; σ ∈ R , γ(ζ, σ; ·) = γ(ζ, τ; ·)} 2 CR ≤ sup ; σ ∈ m, γ(ζ, σ; ·) = γ(ζ, τ; ·) (2.26) nCUTUn kU(σ)k R

2 CR with the understanding 0 = +∞. In particular, assume moreover that γ(s) = exp{s (ζ, τ)} 2 CR is not abnormal, then hcut ≤ 2 Cτ ≤ kU(τ)k .

m nRKnSM Remark 6. Fix (ζ, τ) and let Π(ζ,τ) := {σ − τ ∈ R ; γ(ζ, σ; ·) = γ(ζ, τ; ·)}. Proposition m 1 implies that Π(ζ,τ) is a linear subspace of R . It is clear that the continuous function σ 7→ kU(σ)k deﬁned on τ + Π(ζ,τ) attains its minimum. Hence, the last “ sup” in (2.26) can be replaced by “ max”.

Moreover, we have the following:

2 ∗ NLA1 Lemma 6. In the framework of step-two Carnot groups, Abno is a closed set.

CL CL ∗ On a step-two group G, notice that o 6∈ Cuto . Now assume that Cuto ∩Abno = ∅ and ∗ Abno 6= {o}. Let γ(s) = exp{s (ζ, τ)} = γ(ζ, τ; s) be an arclength parametrized abnormal geodesic. Then it follows from Lemma 6 that its cut time is +∞. Using Corollary 5 and Remark 6, we obtain γ(ζ, τ; ·) = γ(ζ, 0; ·), which implies {γ(s); s ≥ 0} ⊆ Rq × {0} by (2.6). In conclusion, combining this with Corollaries 4 and 3, we get the following:

CL ∗ NCorA1 Corollary 6. In the setting of step-two Carnot groups, Cuto ∩ Abno = ∅ if and only if q Me 2 ⊆ R × {0}. To ﬁnish this subsection, we provide the following:

CLMMW Proposition 4. In the context of step-two Carnot groups, it holds that Me = M. 2 ∗ For a general sub-Riemannian manifold M, we can deﬁne the shortest abnormal set of y ∈ M, Abny, as the set of the endpoints of abnormal (not necessarily normal) shortest geodesics starting from y and ∗ Abny is a closed set as well. This is a result of the characterization of abnormal Pontryagin extremals via Lagrange multipliers rule and the compactness of minimal controls. We would like to thank L. Rizzi for informing us of this general result and providing a sketched proof. For the sake of completeness, we give a proof in the setting of step-two groups without using the notion of the endpoint map in Section 3.

14 2.4.3 Characterizations of GM-groups GM-groups have very ﬁne properties of sub-Riemannian geometry. More precisely,

NTh1 Theorem 4. The following properties are equivalent: (i) G is of type GM; (ii) Me is dense in G; (iii) d(g)2 = sup φ(g; θ) for all g ∈ G; θ∈Ω∗

(iv) The global reference set R is equal to Ω∗ = {θ; kU(θ)k ≤ π}; (v) For any arclength parametrized, strictly normal geodesic γ(s) = exp{s (ζ, τ)}, its cut time is equal to hcut(τ) := 2π/kU(τ)k, with the understanding hcut(0) = +∞; c (vi) Cuto ∩ ∂M = ∅. Remark 7. (1) The condition (ii) should be the easiest to check among all those. (2) In Section 4, we can ﬁnd that every step-two group of corank 1 or 2 is of type GM. As a result, from the above Property (v), we obtain the cut time of any arclength parametrized, strictly normal geodesic on G(q, m, U) with m = 1 or 2, which coincides with that in [19, Theorems 6 and 7]. However, considering Remark 1, their results for the cut time of abnormal geodesics need more explanations. See (ii) of Corollary 9 and Remark 9 below for more details. Furthermore, we emphasize that Property (v) is an equivalent characterization of GM-groups and thus we have found all possible step-two groups satisfying this ﬁne property.

Recall that M is an open set. As a consequence, we obtain the following improvement of [88, Theorem 2.7]:

Nc2 Corollary 7. A step-two group G is of type GM if and only if Cuto = ∂M. NcMc Corollary 8. Assume that G is of M´etiviertype. Then G is a GM-group iﬀ. for any arclength parametrized geodesic γ(s) = exp{s (ζ, τ)}, its cut time is 2π/kU(τ)k.

If G is of GM-type, then our Theorem 3 can be improved. Indeed, the parameter θ can be further chosen as a maximum point of the reference function φ(g; ·) on Ω∗. In other words, we have nThm1 Theorem 5. Assume that G is of GM-type and o 6= g ∈ G. For any shortest geodesic q γg(s) (0 ≤ s ≤ 1) joining o to g, there exist ζ ∈ R and θ ∈ Ω∗ such that

2 2 φ(g; θ) = d(g) = sup φ(g; τ) = |ζ| , γg(s) = exp{s (ζ, 2 θ)}, ∀ 0 ≤ s ≤ 1. τ∈Ω∗

c Moreover, we have that θ ∈ ∂Ω∗ if g ∈ Me .

0 Remark 8. Suppose further that Ω∗ is strictly convex, namely, for any τ 6= τ ∈ Ω∗ and 0 0 < s < 1, we have s τ + (1 − s) τ ∈ Ω∗. For example, all M´etiviergroups satisfy this c property, see [88, Lemma 9.1]. Then, for any g0 ∈ Me , the concave function φ(g0; ·) has a unique maximum point on ∂Ω∗. This simple observation is very useful to determine all c shortest geodesic(s) from o to g0 ∈ Me .

15 Note that in (v) of Theorem 4, we only consider the cut time of any strictly normal geodesic. However, we can characterize GM-groups via the optimal synthesis from o, as well as the classical cut locus of o. More precisely, we have the following:

NThA1 Corollary 9. The following properties are equivalent: (i) G is of type GM; (ii) For any arclength parametrized geodesic γ(s) = exp{s (ζ, τ)} = γ(ζ, τ; s), its cut time is given by

2π h = max ; σ ∈ m, γ(ζ, σ; ·) = γ(ζ, τ; ·) , cut kU(σ)k R

2π with the understanding 0 = +∞; CL c (iii) Cuto = Me . nRKn9 Remark 9. It follows from (ii) of Corollary 9 that the statement in [19, Theorems 6 and 7] is correct if we choose the covector suitably. To be more precise, let G be a step-two group of corank 1 or 2, so it is a GM-group from Corollary 13 in Section 4 below. Let γ(s) = exp{s (ζ, τ)} = γ(ζ, τ; s) be an arclength parametrized geodesic on G. Recall that m Π(ζ,τ) := {σ − τ ∈ R ; γ(ζ, σ; ·) = γ(ζ, τ; ·)} is a linear subspace. Assume further that the continuous function σ 7→ kU(σ)k deﬁned on τ + Π(ζ,τ) attains its minimum at τ. Then the cut time of γ is given by 2π/kU(τ)k.

2 2.5 On the lack of semi-concavity of d on Me 2 s23 Recall that o ∈ Me 2 ⊂ G is the set of points, g, where the reference function φ(g; ·) has a degenerate (so infinite) critical point in the initial reference set Ω∗. The following theorem is a kind of generalization of the first result in [102, Theorem 1.1], which will be proven by a completely different method.

m p1 Theorem 6. Let g0 = (x0, t0) ∈ Me 2. Then there exist a unit vector ν0 ∈ R and a constant c0 > 0 such that

2 2 2 d(x0, t0 + h ν0) + d(x0, t0 − h ν0) − 2 d(x0, t0) ≥ c0 h, ∀ h > 0. (2.27) failconc

In particular, together with (a) in Remark 3 and (2.8), it gives immediately

∗ m Corollary 10. If g0 = (x0, 0) ∈ Abno, there exist a unit vector ν∗ ∈ R and a constant c∗ > 0 such that

2 2 d(x0, h ν∗) − d(x0, 0) ≥ c∗|h|, ∀ h ∈ R. (2.28) failconc0

A direct consequence of (2.27) is the following signiﬁcantly weaker estimate:

0 2 0 2 2 d(g0 + g ) + d(g0 − g ) − 2 d(g0) lim sup 0 2 = +∞, (2.29) nMBn g0−→o |g |

16 2 which implies, from (1.1), the lack of semi-concavity of d for any g0 ∈ Me 2. ∗ It follows from Lemma 6 that Me 2 ⊆ Abno. A very interesting phenomenon is that

(2.29) can be no longer valid for g0 ∈ Me 2 \ Me 2 even in the setting of GM-groups. A concrete example will be provided in Subsection 6.2 below. − However, recall that our SCo is defined by (1.2) instead of as the set of points where − (2.29) satisfies. Obviously, SCo is closed. When the underlying group is of type GM, ∗ combining (ii) of Corollary 9 with Theorem 6 obtained above, we can characterize Abno via Me 2; as a byproduct, we answer the open problem [20, (29)] affirmatively:

− ∗ NThA2 Theorem 7. In the setting of GM-groups, it holds that SCo = Abno = Me 2. Furthermore, combining Theorem 7, Lemmas 4 and 6 with [20, Corollary 30] (with little modiﬁcation in its proof), we answer the open question [20, second part of (30)], when the underlying group is GM-group: + − Corollary 11. In the framework of GM-groups, we have Cuto = SCo ∪ SCo . Similarly, by Corollaries 4 and 6, we have the following result that provides an aﬃr- mative, also partial, answer to the open questions [20, (29)-(30)]:

q Corollary 12. Let G be a step-two group such that Me 2 ⊆ {(x, 0); x ∈ R }. Then it holds − ∗ + − that SCo = Abno (= Me 2) and Cuto = SCo ∪ SCo .

Recall that Me 2 = {o} if and only if G is of M´etiviertype. In the sequel, a step-two group G is said to be a SA-group (or of type SA) if it satisﬁes

q {o}= 6 Me 2 ⊆ {(x, 0); x ∈ R } . (SA)

Notice that star graphs and N3,2, namely the free step-two Carnot group with three generators, are SA-groups. See [88] or §5 and §7 below for more details. Also remark that star graphs are both GM-groups and SA-groups, N3,2 is the simplest example of SA-group that is not of type GM. Of course, we can provide an uncountable number of SA but not GM groups. Furthermore, a trivial method to construct SA-groups can be found in Proposition 11 of Appendix B. See Appendix C for another method which is much more meaningful. In particular, SA-groups of corank 1 are the direct product of a Euclidean space Rk with a generalized Heisenberg group. However, for any m ≥ 2, SA-groups with corank m form a very complicated set. To ﬁnish this section, we point out the following facts: (1) The second result of [102, Theorem 1.1] says that on the free step-two Carnot group ∗ with k (k ≥ 4) generators, Nk,2, for any g0 = (x0, t0) ∈ Abno, there exist a unit vector k (k−1)/2 ν∗ ∈ R and a constant c∗ > 0 such that we have the following counterpart of (2.28): d(x0, t0 + h ν∗) − d(x0, t0) ≥ c∗|h|, −c∗ ≤ h ≤ c∗. ∗ Hence, (2.29) is valid on Nk,2 for any g0 ∈ Abno.

17 k (2) Nk,2 (k ≥ 4) neither is a GM-group nor satisﬁes Me 2 ⊆ (x, 0); x ∈ R .

3 Proof of main results nsP 3.1 Proof of Theorem 3 Proof. We ﬁrst assume that the shortest geodesic joining o to g is unique. By [88, Corol- q lary 2.4], there exist w ∈ R with |w| = d(g) and θ ∈ R such that γ(w,2 θ) is a shortest geodesic joining o to g. By uniqueness we must have γg(s) = γ(w,2 θ)(s) = exp(s (w, 2 θ)) for all 0 ≤ s ≤ 1, which ends the proof in this case. In general, from the characterization of the shortest geodesic in Subsection 2.2.1, there q m exist w∗ ∈ R with |w∗| = d(g) and θ∗ ∈ R (not necessarily belonging to R) such that

γg = γ(w∗,2 θ∗). We now prove that for each s∗ ∈ (0, 1), the restriction of γg on the interval 0,s∗ [0, s∗], γg|[0, s∗] = γ(s∗ w∗,2 s∗ θ∗) := (γg) is the unique shortest geodesic joining o to 0,s∗ γg(s∗) = exp(s∗ w∗, 2 s∗ θ∗). Notice that (γg) is a shortest geodesic joining o to γg(s∗) since γg itself is shortest. To prove uniqueness, we argue by contradiction. Assume that 0,s∗ there is another shortest geodesic γs∗ 6= (γg) joining o to γg(s∗) with constant speed s∗|w∗|. Then we construct a horizontal curve with constant speed |w∗| = d(g) deﬁned by

( s γs , 0 ≤ s ≤ s∗, γ (s) := ∗ s∗ es∗ γg(s), s∗ ≤ s ≤ 1.

Obviously, γ is a shortest geodesic joining o to g as well. Again from the characterization es∗ q of the shortest geodesic in Subsection 2.2.1, there exist w∗∗ ∈ R with |w∗∗| = d(g) and θ ∈ m such that γ = γ . ∗∗ R es∗ (w∗∗,2 θ∗∗) By the fact that γ (s) = γ (s) when s ∈ [s , 1], it follows from Lemma 2 that γ and g es∗ ∗ g γ coincide on the whole interval [0, 1]. In particular, γ (s) = γ (s) for all 0 ≤ s ≤ s , es∗ g es∗ ∗ 0,s∗ that is (γg) = γs∗ , which contradicts with our assumption. For each s∗ ∈ (0, 1), what we have proven at the beginning shows that there exist q w(s∗) ∈ R with |w(s∗)| = s∗d(g) and θ(s∗) ∈ R such that

0,s∗ exp(s (w(s∗), 2 θ(s∗))) = (γg) (s) = exp(s (s∗ w∗, 2 s∗ θ∗)), ∀ 0 ≤ s ≤ 1. (3.1) equstar

+∞ From compactness of R, we extract a sequence {sj}j=1 ⊆ (0, 1) such that sj → 1, w(sj) → w with |w| = d(g) and θ(sj) → θ ∈ R as j → +∞. With s∗ replaced by sj in (3.1) and letting j → +∞, we obtain that exp(s (w∗, 2 θ∗)) = γg(s) = exp(s (w, 2 θ)) for all 0 ≤ s ≤ 1, which ends the proof of the theorem.

3.2 Proof of Corollary 5 Proof. For convenience, we set

m eh = eh(ζ, τ) := sup {2 Cσ; σ ∈ R , γ(ζ, σ; ·) = γ(ζ, τ; ·)} .

18 For any s∗ ∈ (0, hcut), we know that γ(s∗ ζ, s∗ τ) is a shortest geodesic. It follows from Theorem 3 that there exist ζ ∈ q and θ ∈ R such that γ = γ . (s∗) R (s∗) (s∗ ζ, s∗ τ) (ζ(s∗), 2 θ(s∗))

Then Lemma 2 implies that s∗ ζ = ζ(s∗) and

2 θ γ(ζ, σ ; ·) = γ(ζ, τ; ·), with σ := (s∗) . (s∗) (s∗) s∗

s∗ σ(s∗) Recalling that R is star-shaped w.r.t. 0, by the fact that θ(s∗) = 2 ∈ R, we have

s∗ eh ≤ Cσ ≤ , 2 (s∗) 2 which implies s∗ ≤ eh. Since s∗ ∈ (0, hcut) is arbitrary, we obtain hcut ≤ eh. To prove the m CR second inequality, it suﬃces to observe that for each σ ∈ R , we have Cσ ≤ kU(σ)k and this ﬁnishes the proof of Corollary 5.

3.3 Proof of Lemma 6

+∞ ∗ Proof. For any {gj}j=1 ⊆ Abno such that gj → g ∈ G as j → +∞, our aim is to prove ∗ ∗ g ∈ Abno as well. For each j ∈ N , let γj(s) (0 ≤ s ≤ 1) be an abnormal shortest geodesic (j) q (j) (j) joining o to gj. By Theorem 3, there exist ζ ∈ R with |ζ | = d(gj) and θ ∈ R such that γj = γ(ζ(j), 2 θ(j)). Since γ(ζ(j), 2 θ(j)) is abnormal, from Proposition 1 there exists a σ(j) ∈ Sm−1 such that (j) (j) k (j) U(σ ) U(θ ) ζ = 0, ∀ k ∈ N. (3.2) Abnj

(j) Notice that |ζ | = d(gj) → d(g) as j → +∞. From compactness, up to subsequences, (j) (j) (j) m−1 we may assume that ζ → ζ0 with |ζ0| = d(g), θ → θ0 ∈ R and σ → σ0 ∈ S as j → +∞. Observe that

(j) (j) γ(ζ ,2 θ )(1) = exp(ζ0, 2 θ0) = lim exp(ζ , 2 θ ) = lim gj = g. 0 0 j→+∞ j→+∞

By the fact that |ζ0| = d(g), we obtain that γ(ζ0,2 θ0) is a shortest geodesic joining o to g.

It remains to prove that γ(ζ0,2 θ0) is also abnormal. In fact, letting j → +∞ in (3.2), we get

k U(σ0) U(θ0) ζ0 = 0, ∀ k ∈ N,

which implies γ(ζ0,2 θ0) is abnormal by Proposition 1. This ends the proof of Lemma 6.

3.4 Proof of Proposition 4 Proof. Set

q Ξ1 := {(x, θ) ∈ R × Ω∗; det(−HessθhU(θ) cot U(θ) x, xi) > 0}, q Ξ2 := {(x, θ) ∈ R × Ω∗; det(−HessθhU(θ) cot U(θ) x, xi) = 0},

19 and the map

q κ : R × Ω∗ −→ Me 1 (x, θ) 7−→ x, − ∇ hU(θ) cot U(θ) x, xi . 4 θ

q It follows from Proposition 2 that Ξ1 ∪ Ξ2 = R × Ω∗. Recall that (cf. (2.18)-(2.20)) κ(Ξ1) = M and κ(Ξ2) = Me 2. Observe that the function (x, θ) 7→ det(−HessθhU(θ) cot U(θ) x, xi) is real analytic q q in R × Ω∗. We claim that Ξ2 has an empty interior. Otherwise Ξ2 should be R × Ω∗ (cf. [73, § 3.3 (b)]), which means Ξ1 = ∅ and thus M = ∅. This leads to a contradiction q since M 6= ∅ from Remark 4. In conclusion, Ξ1 is dense in R × Ω∗. Now, we shall show that Me 2 ⊆ M. Fix (x, t) ∈ Me 2. There exists a θ ∈ Ω∗ such that q (j) (j) +∞ (x, θ) ∈ Ξ2 and κ(x, θ) = (x, t). Since Ξ1 is dense in R ×Ω∗, there are {(x , θ )}j=1 ⊆ (j) (j) (j) (j) Ξ1 such that (x , θ ) → (x, θ) as j → +∞. Hence, M 3 κ(x , θ ) → κ(x, θ) = (x, t) as j → +∞. As a result, we obtain that Me 2 ⊆ M and thus Me = M, which ends the proof of the proposition.

3.5 Proof of Theorem 4 Proof. (i) ⇒ (ii): This is evident. (ii) ⇒ (iii): Just use (2.21). (iii) ⇒ (iv): For any given g = (x, t) ∈ S, under our assumption, it follows from

Proposition 2 that there exists a θ0 ∈ Ω∗ such that

2 d(x, t) = φ((x, t); θ0) = hU(θ0) cot U(θ0) x, xi + 4 t · θ0. (3.3) tmin

Since S is open, there exists a r0 > 0 such that

{x} × B(t, r0) = {x} × {τ; |τ − t| < r0} ⊆ S.

Then it follows from (iii) that we have for s ∈ B(t, r0),

2 d(x, s) ≥ φ((x, s); θ0) = hU(θ0) cot U(θ0) x, xi + 4 s · θ0. (3.4) sgmin

2 So, the function s 7−→ d(x, s) − 4 s · θ0 has a local minimum at the point s = t. As a 1 2 result, we have 4 ∇td(g) = θ0 ∈ Ω∗ and consequently R ⊆ Ω∗. The inverse inclusion is given by (2.23) and we obtain (iv). (iv) ⇒ (v): Just combine Corollary 2 with Corollary 5. c (v) ⇒ (vi): We argue by contradiction. Assume that there exists a g ∈ Cuto ∩ ∂M. ∗ Since Me 2 ⊆ Abno ⊆ Cuto, we have g ∈ Me \ Me . Then from [88, (1) of Remark 2.6] there q exist ζ ∈ R with |ζ| = d(g) and θ ∈ ∂Ω∗ such that γ(ζ,2 θ) is a shortest geodesic joining c o to g. Since g ∈ Cuto = S, it follows from (2.15) that γ(ζ,2 θ) is strictly normal. As a CL result, (v) implies g ∈ Cuto ⊆ Cuto and we obtain a contradiction.

20 c (vi) ⇒ (i): We argue by contradiction. Assume that M $ G. Since Cuto is dense in c G by [2, Theorem 11.8], we can pick a g ∈ (G \ M) ∩ Cuto. From the characterization of the smooth points (2.15), there exists a unique shortest geodesic γ = γ(w,2 θ) joining o to g, which is not abnormal. We ﬁrst claim that θ∈ / Ω∗, otherwise θ should be a critical point of φ(g; ·) in Ω∗ by Theorem 2 and thus g ∈ Me = M ∪ Me 2. Since g∈ / M, then g ∈ Me 2 ⊆ Cuto, which gives a contradiction and proves this assertion. We further claim that θ∈ / ∂Ω∗. Otherwise, Lemma 5 and Theorem 2 should imply that for any s∗ ∈ (0, 1), we have that exp(s∗ (w, 2 θ)) ∈ M. Thus g = exp(w, 2 θ) ∈ M, c contradicting with our assumption that g ∈ (G \ M) ∩ Cuto. As a result, there exists a s0 ∈ (0, 1) such that s0 θ ∈ ∂Ω∗. Set

c g0 = exp(s0 (w, 2 θ)) ∈ Cuto = S, where the “∈” is given by Lemma 5. Similarly, we get that exp(s∗ (w, 2 θ)) ∈ M for all s∗ ∈ (0, s0) and g0 ∈ M. Now we are in a position to show that g0 ∈/ M. We argue by contradiction. Assume that g0 ∈ M, then there exists (w∗, 2 θ∗) such that θ∗ ∈ Ω∗ and γ(w∗,2 θ∗) is the unique shortest geodesic joining o and g0. So γ(w∗,2 θ∗) coincides with the restriction of γ in [0, s0], 0,s0 0,s0 namely γ = γ(s0 w,2 s0 θ). Hence, γ admits two different normal lifts. Consequently γ0,s0 is also abnormal by definition, which contradicts with, by Lemma 4, the fact that γ is strictly normal. c After all, we have that g0 ∈ ∂M ∩ Cuto, which leads to a contradiction. Therefore we finishes the proof.

3.6 Proof of Theorem 5

Proof. By the deﬁnition of Me (see (2.18)), the second claim in Theorem 5 is a direct consequence of the ﬁrst one, that needs to be proven. Indeed, from Corollary 7 we (j) (j) +∞ have S = M. By Remark 5, there exist {gj = (x , t )}j=1 ⊆ S = M with γ(ζ(j),2 θ(j)) (j) +∞ ({θ }j=1 ⊆ Ω∗) the unique shortest geodesic joining o to gj such that:

(j) (j) gj −→ g, (ζ , 2 θ ) −→ (ζ, 2 θ) as j → +∞, γ(ζ,2 θ) = γg.

2 It remains to prove that φ(g; θ) = d(g) when θ ∈ ∂Ω∗. 2 m 2 Notice that U(τ) is semi-positive deﬁnite for every 0 6= τ ∈ R . Let 0 ≤ λ1(τ) ≤ 2 q ... ≤ λq(τ) (λl(τ) ≥ 0, 1 ≤ l ≤ q) denote its eigenvalues and {Pl(τ)}l=1 the corresponding q q set of pairwise orthogonal projections (that is, (Pk(τ))(R ) ⊥ (Pl(τ))(R ) for k 6= l). Then we have q 2 X 2 U(τ) = λl(τ) Pl(τ). (3.5) spec l=1

It follows from [77, Chapter two] that for every 1 ≤ l ≤ q, λl(τ) is a continuous function of τ 6= 0 and homogeneous of degree 1, namely λl(s τ) = s λl(τ) for s > 0. However, Pl(τ)

21 is not necessarily continuous, but it can be chosen to be symmetric and homogeneous of degree 0, namely

Pl(r τ) = Pl(τ) ∀ r 6= 0, 1 ≤ l ≤ q. (3.6) homoP

For the θ ∈ ∂Ω∗ obtained before, there exists an L ∈ {1, . . . , q} such that

2 2 2 2 2 λL−1(θ) < π when L > 1, and λL(θ) = ... = λq(θ) = π .

q π2 |θ| From the continuity of {λl(τ)}l=1, there exist δ ∈ 0, 4 and r0 ∈ 0, 2 such that for

τ ∈ B(θ, r0) = {τ; |τ − θ| < r0}, we have

2 2 2 2 2 2 λL−1(τ) ≤ π − 4δ when L > 1, and π − δ ≤ λL(τ) ≤ ... ≤ λq(τ) ≤ π + δ.

For τ ∈ B(θ, r0), let us set

 L−1 1 R 2 −1 P 2  z (z − U(τ) ) dz = λl(τ) Pl(τ) when L > 1 2πi Γ1 V (τ) := l=1 , (3.7)  0 when L = 1 and the projection on (near π2)-eigenspaces of U(τ)2

q 1 Z X Q(τ) := (z − U(τ)2)−1dz = P (τ), (3.8) 2πi l Γ2 l=L where the contours Γ1, Γ2 ⊆ C are deﬁned by

2 2 2 Γ1 := {z; dist(z, [0, π − 4δ]) = δ} and Γ2 := {z; dist(z, [π − δ, π + δ]) = δ} respectively, with the counterclockwise orientation. From the integral representation, it is easy to see that the operator functions V (τ) and Q(τ) are continuous in B(θ, r0). It deduces from Theorem 2 that

q (j) 2 (j) 2 X λl(θ ) (j) (j) 2 U(θ ) (j) (j) 2 2 (j) |Pl(θ ) x | = (j) x = |ζ | → |ζ| , as j → +∞. sin λl(θ ) sin U(θ ) l=1

As a result, since

q 2 (j) (j) 2 X (j) (j) 2 |Q(θ) x| = lim |Q(θ ) x | = lim |Pl(θ ) x | j→+∞ j→+∞ l=L q (j) −2 (j) 2 X λl(θ ) λl(θ ) (j) (j) 2 = lim (j) (j) |Pl(θ ) x | , j→+∞ sin λl(θ ) sin λl(θ ) l=L and sin s ∼ (π − s) for s near π, we get immediately

|Q(θ) x|2 = 0. (3.9) Px

22 Similarly, by the fact that q q V (θ(j)) cot V (θ(j)) x(j), x(j)

q (j) (j) (j) (j) (j) (j) 2 X (j) (j) (j) (j) 2 = hU(θ ) cot U(θ ) x , x i + |Q(θ ) x | − λl(θ ) cot λl(θ ) |Pl(θ ) x | , l=L we yield that D E q q pV (θ) cot pV (θ) x, x = lim V (θ(j)) cot V (θ(j)) x(j), x(j) j→+∞

= lim hU(θ(j)) cot U(θ(j)) x(j), x(j)i. (3.10) vcotv j→+∞

Hence, it follows from Theorem 2 that

2 2 (j) d(g) = lim d(gj) = lim φ(gj; θ ) j→+∞ j→+∞ = lim (hU(θ(j)) cot U(θ(j)) x(j), x(j)i + 4 t(j) · θ(j)) j→+∞ D E = pV (θ) cot pV (θ) x, x + 4 t · θ.

Combining this with θ ∈ ∂Ω∗ and the fact that the orthogonal projection of x on π2-eigenspace of U(θ)2 is zero (cf. (3.9)), it follows from [88, Remark 2.1] that

d(g)2 = hU(θ) cot U(θ) x, xi + 4 t · θ = φ((x, t); θ). (3.11) expphibd

This ends the proof of Theorem 5.

3.7 Proof of Corollary 9 Proof. (i) ⇒ (ii): Just combine Corollary 2 with Corollary 5 (cf. also Remark 6). (ii) ⇒ (i): It is trivial because Theorem 4 (v) satisﬁes under our assumption. c CL (iii) ⇒ (i): It follows from Theorem 1 and [118, Proposition 15] that Me = Cuto ⊆ Cuto is a set of measure zero, which means Me is dense in G. This is exactly (ii) of Theorem 4 and we obtain that G is of type GM. c CL c (i) + (ii) ⇒ (iii): From Corollary 3 it suﬃces to prove Me ⊆ Cuto . Fix g ∈ Me . It is q clear that g 6= o. Theorem 5 guarantees that there exist ζ ∈ R \{0} and θ ∈ ∂Ω∗ such that γ∗ = γ(ζ,2 θ) is a shortest geodesic joining o to g. Consider the arclength parametrized geodesic γ := γ(ζ, τ; ·), where τ := 2 θ . Here and in the sequel, we adopt the convention e b |ζ|

( u , if u ∈ R` \{0}, u := |u| (3.12) defhat b 0, if u = 0.

CL To prove that g ∈ Cuto , it remains to show that the cut time of γe, hcut(γe), equals |ζ|. First, notice that γe|[0, |ζ|] = γ(ζ,2 θ) = γ∗ is a shortest geodesic. Hence we get hcut(γe) ≥ |ζ|.

23 m On the other hand, for any σ ∈ R such that γ(ζ,b σ; ·) = γ(ζ,b τ; ·), we have γ∗ = c γ(ζ,|ζ| σ). Since g ∈ Me , it follows from Theorem 2 that |ζ| σ U ≥ π, 2 or equivalently 2π ≤ |ζ|. Then from (ii), we obtain that h (γ) ≤ |ζ|. kU(σ)k cut e Therefore, we ﬁnish the proof of Corollary 9.

3.8 Proof of Theorem 6

Proof. Let g0 = (x0, t0) ∈ Me 2. Then (2.21) and (2.20) imply that there exists a θ0 ∈ Ω∗ 2 such that d(g0) = φ(g0; θ0). Since Ω∗ is open, there exists a r0 > 0 such that B(θ0, 2 r0) = m {τ ∈ R ; |τ − θ0| < 2 r0} ⊆ Ω∗. From [88, Proposition 5.1 (c)] there exists a unit vector ν0 such that

t0 · ν0 = 0, φ(g0; θ0 + s ν0) = φ(g0; θ0), ∀ s ∈ R with θ0 + s ν0 ∈ Ω∗. Consequently, for any h > 0, using [88, Theorem 2.1], we have that

2 d(x0, t0 + h ν0) ≥ φ((x0, t0 + h ν0); θ0 + r0 ν0) = φ(g0; θ0) + 4 h r0 + 4 h ν0 · θ0, 2 d(x0, t0 − h ν0) ≥ φ((x0, t0 − h ν0); θ0 − r0 ν0) = φ(g0; θ0) + 4 h r0 − 4 h ν0 · θ0. As a result, we obtain

2 2 2 d(x0, t0 + h ν0) + d(x0, t0 − h ν0) − 2 d(g0) ≥ 8 r0 h, ∀ h > 0, which ﬁnishes the proof of this theorem.

3.9 Proof of Theorem 7 Proof. First, for any arclength parametrized abnormal geodesic γ(s) = exp{s (ζ, τ)} =

γ(ζ, τ; s) with cut time hcut and s∗ < hcut, we claim that γ(s∗) = γ(ζ, τ; s∗) ∈ Me 2. In m 2π fact, from (ii) of Corollary 9, there exists a σ ∈ R such that s∗ < kU(σ)k and γ(ζ, σ; ·) = γ(ζ, τ; ·). Then it follows from the ﬁrst equation of Proposition 3 that

γ(s∗) = γ(ζ, τ; s∗) = γ(ζ, σ; s∗) ∈ Me 2.

∗ Thus, the second equation in Proposition 3 implies that Abno ⊆ Me 2. − Next, from deﬁnition the set SCo is closed. Moreover, Theorem 6 implies that Me 2 ⊆ − SCo . In conclusion, we have that

∗ − Abno ⊆ Me 2 ⊆ SCo .

∗ − ∗ Finally, we recall that Abno is closed (cf. Lemma 6), so the inclusion SCo ⊆ Abno can be deduced from [35, Theorem 1], which ends the proof of Theorem 7.

24 4 Step-two groups of Corank 2 are GM-groups s3 The purpose of this section is twofold. On one hand, we provide a suﬃcient condition for M = G by means of semi-algebraic theory. As a byproduct, we show that all G(q, 2, U) are of type GM. On the other hand, we prove that there exists a M´etiviergroup G(4N, 3, UN ), which is not of type GM, for any N ∈ N∗. Let us begin by recalling (cf. [31, Chapter 2]):

4.1 Semi-algebraic sets, mappings and dimension

A set A ⊆ Rq is semi-algebraic if it is the result of a finite number of unions and inter- sections of sets of the form {f = 0}, {g > 0}, where f, g are polynomials on Rq. If A is a semi-algebraic set, then its complement, boundary and any Cartesian projection of A are semi-algebraic sets. If A and B are semi-algebraic sets, then so does A × B. Furthermore, any semi-algebraic set A ⊆ Rq is the disjoint union of a finite number of semi-algebraic sets q q dim Mi Mi in R where each Mi is a smooth submanifold in R and diffeomorphic to (0, 1) . If A ⊆ Rq and B ⊆ Rr are two semi-algebraic sets. A mapping h : A → B is semi- algebraic if its graph is a semi-algebraic set in Rq+r. If S ⊆ A is a semi-algebraic set and h : A → B is a semi-algebraic mapping, then h(S) is a semi-algebraic set in Rr. Let A ⊆ Rq be a semi-algebraic set. Its dimension, dim A, can be defined in some algebraic way. The basic properties that we will use later are: (1) If A is the finite union of semi-algebraic sets A1,...,Ap, then dim A = max dim Ai. (2) If A and B are 1≤i≤p semi-algebraic sets, then dim(A × B) = dim A + dim B. (3) If h : A → B is a semi- algebraic mapping, then dim h(A) ≤ dim A. (4) Moreover, if A is a semi-algebraic set as well as a smooth submanifold in Rq, its dimension as a semi-algebraic set coincides with its dimension as a smooth manifold. As a result, if A ⊆ Rq is a semi-algebraic set with dim A < q, then it has measure 0 in Rq by the usual Morse-Sard-Federer Theorem (cf. [79, p. 72]).

4.2 A suﬃcient condition, from an algebraic point of view, for M = G m Recall that Ω∗ is deﬁned by (2.16) and U(θ)(θ ∈ R ) by (2.5). For θ 6= 0, let M(θ) denote the multiplicity of the maximal eigenvalue of U(θ)2, and

M := min M(θ). (4.1) θ6=0

We have the following: t3 Theorem 8. If M ≥ m, then M = G.

m Proof. First observe that the open set Ω∗ is a semi-algebraic set in R by the fact that

c 2 2 2 Ω∗ = π2 {(x, τ); hU(τ) x, xi ≥ π } ∩ {(x, τ); |x| = 1} ,

25 q m m where π2 denotes the projection from R × R to the second entry R . Then, ∂Ω∗ is a semi-algebraic set and dim (∂Ω∗)(= dim (Ω∗ \ Ω∗)) ≤ m − 1 by [31, Propositon 2.8.13]. Set 2 2 q q Σ := {(U(θ) − π ) y; θ ∈ ∂Ω∗, y ∈ R } ⊆ R ,

that is, the set of points x such that there exists a θ ∈ ∂Ω∗ satisfying that the orthogonal projection of x on π2-eigenspace of U(θ)2 is zero. From [88, Proposition 2.2], it remains to prove that Σ has measure 0. Now, consider the map deﬁned by m q q Ψ: R × R −→ R (θ, y) 7−→ Ψ(θ, y) := (U(θ)2 − π2) y.

q Notice that it is a semi-algebraic mapping. Then Σ = Ψ(∂Ω∗ × R ) is a semi-algebraic set. It suﬃces to prove that dim Σ ≤ q − 1. For r ∈ N satisfying r ≤ q, set

Πq,r := {L; there exist 1 ≤ j1 < . . . < jr ≤ q such that L = span{ej1 , . . . , ejr }}, q where {e1, . . . , eq} denotes the standard orthonormal basis of R and we adopt the con- r vention that span{∅} = {0}. Then Πq,r is a ﬁnite set of Cq elements. Remark that for a q q × q real matrix S with rank(S) ≤ r, there exists an L ∈ Πq,r such that S(R ) = S(L). 2 2 Under our assumption, we have rank(U(θ) − π ) ≤ q − m for any θ ∈ ∂Ω∗. Hence, we get q Σ = Ψ(∂Ω∗ × R ) = ∪ Ψ(∂Ω∗ × L). L∈Πq,q−m

It is clear that for each L ∈ Πq,q−m, we have

dim(∂Ω∗ × L) = dim(∂Ω∗) + dim L ≤ (m − 1) + (q − m) = q − 1, and as a result,

dim Σ = max dim Ψ(∂Ω∗ × L) ≤ max dim(∂Ω∗ × L) ≤ q − 1, L∈Πq,q−m L∈Πq,q−m which ends the proof of this theorem.

By the fact that Ue(θ) is skew-symmetric for any θ, we have M ≥ 2. Then c2 Corollary 13. If G = G(q, 1, U) or G(q, 2, U), namely G is a step-two group of Corank 1 or 2, then it is a GM-group. r1 Remark 10. (1) Theorem 8 is sharp in the sense that G may not be of type GM if M < m. The simplest example is the free Carnot group of step two and 3 generators studied in [88]. Notice that in such case, we have M = 2 < 3 = m. Other interesting examples can be found in Subsection 4.3 below. (2) Remark also that M ≥ m is in general not necessary for M = G. See for example the star graphs studied in [88]. (3) We do not know whether there exists a simple algebraic characterization for GM- groups similar to that of M´etiviergroups.

26 4.3 Not all Métiviergroups are of type GM SS43 In the sequel, we will illustrate that when m = 3, the condition that q (instead of M) is sufficiently larger than m cannot guarantee M = G, even in the case that G is a Métivier group.

p4 Proposition 5. For any N ∈ N, there exists a M´etiviergroup G = G(4N + 4, 3, UN ) which is not a GM-group.

∗ Proof. Let H(4n, 3) = G(4n, 3, UH(4n,3))(n ∈ N ) denote the (4n + 3)-dimensional H-type

group, that is UH(4n,3) satisﬁes the following condition (cf. (2.5) for the related deﬁnition):

0 0 0 0 3 UH(4n,3)(λ) UH(4n,3)(λ ) + UH(4n,3)(λ ) UH(4n,3)(λ) = 2 λ · λ I4n, ∀ λ, λ ∈ R . We remark that the (4n + 3)-dimensional quaternionic Heisenberg group provides a good example for it. 3 For a row vector τ = (τ1, τ2, τ3) ∈ R , set  −1 −1 −1  0 2 τ1 2 τ2 2 τ3 −1 −2 τ1 0 −τ3 τ2  X(τ) := i  −1  , −2 τ2 τ3 0 −τ1  −1 −2 τ3 −τ2 τ1 0

∗ U0(τ) := X(τ) and for N ∈ N , τ UH(4N,3) 2 O(4N)×4 UN (τ) := . O4×(4N) X(τ) Observe that 1 2 2 4 |τ| I4N+1 UN (τ) = 2 3 T , (4.2) nEe1 |τ| I3 − 4 τ τ

2 |τ|2 whose eigenvalues are |τ| with the multiplicity 2 and 4 with the multiplicity 4N + 2. Hence, we get a M´etivier group, saying that G = G(4N + 4, 3, UN ). We only consider the case N ∈ N∗ here and the proof is similar when N = 0. From now on, we ﬁx N ∈ N∗, and write

∗ 4N 3 4N+4 x = (xe , xe1, xe∗) ∈ R × R × R = R . In our situation, by (4.2), a direct calculation shows that the initial reference set, deﬁned by (2.16), is given by Ω∗ = {τ; |τ| < π}, and the reference function (cf. (2.17)) is |τ| |τ| φ((x, t); τ) = 4 t · τ + cot |x∗|2 + x2 + |x · τ|2 2 2 e e1 e∗ b 2 + (|τ| cot |τ|) |xe∗ − (xe∗ · τb) τb| .

27 Here we have used the convention (3.12). Observe that ∗ ∗ φ(((xe , xe1, xe∗), t); τ) = φ(((xe , xe1, xe∗), −t); −τ), (4.3) symM1 ∗ ∗ φ(((xe , xe1,O xe∗), O t); O τ) = φ(((xe , xe1, xe∗), t); τ), ∀ O ∈ O3, (4.4) symM2

where O3 denotes the 3 × 3 orthogonal group. Without loss of generality, we may assume in the sequel that

xe∗ = |xe∗| e1 = |xe∗|(1, 0, 0), t = (t1, t2, 0) with t1, t2 ≥ 0. (4.5) nIc Now, by recalling that (see (2.25)) 1 − s cot s 2s − sin (2s) ψ(s) := , µ(s) := −(s cot s)0 = , s2 2 sin2 s we can write |τ| |τ| φ((x, t); τ) = 4 t · τ + |x |2 + cot (|x∗|2 + x2) e∗ 2 2 e e1 |τ| τ 2 − ψ 1 |x |2 − ψ(|τ|)(τ 2 + τ 2)|x |2. 2 4 e∗ 2 3 e∗

Suppose that θ ∈ Ω∗ is a critical point of φ((x, t); ·) for some (x, t) satisfying (4.5). Then we have |θ| |x∗|2 + x2 |θ| θ2 |x |2 |θ| |x |2 4 t = µ e e1 θ + ψ0 1 e∗ θ + ψ e∗ θ e 2 2|θ| 2 4 2|θ| 2 2 1 1 |x |2 + ψ0(|θ|)(θ2 + θ2) e∗ θ + 2 ψ(|θ|) |x |2 (θ e + θ e ). 2 3 |θ| e∗ 2 2 3 3

∗ 2 2 We further assume that |xe | + xe1 6= 0 and xe∗ 6= 0. Using [88, Lemmas 3.1 and 3.2], the fact that t3 = 0 implies θ3 = 0. Thus t |θ| |x∗|2 + x2 θ |θ| θ2 |x |2 θ |θ| |x |2 θ 4 1 = µ e e1 1 + ψ0 1 e∗ 1 + ψ e∗ 1 t2 2 2|θ| θ2 2 4 2|θ| θ2 2 2 0 2 0 2 |xe∗| θ1 2 0 + ψ (|θ|) θ2 + 2 ψ(|θ|) |x∗| |θ| θ2 e θ2 ∗ := Υ((xe , xe1, xe∗); (θ1, θ2)). Next, following the proof of [88, Proposition 10.3], we can establish the following lemma. For completeness, we include its proof in “Appendix A”. ∗ 2 2 LMng Lemma 7. Suppose that |xe | + xe1 6= 0 and xe∗ 6= 0. Let q 2 2 2 BR2 (0, π) := (v1, v2) ∈ R ; v1 + v2 < π ,

π u2 2(x∗, x , x ) := (u , u ) ∈ 2; |u | < 2 + |x∗|2 + x2 + |x |2 . Rr e e1 e∗ 1 2 R 1 2 e e1 e∗ 4 |xe∗| ∗ ∞ 2 ∗ Then Υ((xe , xe1, xe∗); ·) is a C -diﬀeomorphism from BR2 (0, π) onto Rr(xe , xe1, xe∗).

28 Combining this with (4.3) and (4.4), we get that G \ Me contains the subset ( ) |t · x | 1 t · x x 2 |x∗|2 + x2 + |x |2 (x, t); x 6= 0, |x∗|2 + x2 6= 0, e∗ > t − e∗ e∗ + e e1 e∗ , e∗ e e1 2 π|xe∗| |xe∗| |xe∗| |xe∗| 16

which implies immediately M ⊆ Me $ G.

5 Sub-Riemannian geometry on step-two K-type groups s6

Let p0, p1 ∈ {2, 3, 4,...} with p0 ≥ p1. Consider a p1 × p0 full-rank real matrix

 T  eb1  .  p0 B =  .  with column vectors ebj ∈ R \{0}, 1 ≤ j ≤ p1. T ebp1

(K) A step-two Kolmogorov type group (or K-type group, in short) of type B, GB , is deﬁned (K) by G(1 + p0, p1, UB ) with (see [32, § 4])

T ! (K),(j) 0 ebj UB = , 1 ≤ j ≤ p1. −ebj Op0×p0

Notice that we have for τ ∈ Rp1 T T 2 (K) 0 τ B (K) 2 |B τ| O1×p0 UB (τ) = i T ,UB (τ) = T T . (5.1) Ku2 −B τ Op0×p0 Op0×1 B τ τ B

Hence the initial reference set, deﬁned by (2.16), is

(K) p1 T Ω∗ := ΩB = {τ ∈ R ; |B τ| < π}.

In the rest of this section, we write

p0 1+p0 x = (x1, x∗) ∈ R × R = R . A simple calculation shows that the reference function in this setting is given by

(K) 2 2 T 2 T φ((x, t); τ) := φB ((x, t); τ) = |x| + 4 t · τ − x1 f(|B τ|) + |τ · Bx∗| ψ(|B τ|) .

Remark that the case B = In corresponds to the star graph K1,n, on which the squared sub-Riemannian distance and the cut locus have been characterized by [88, Theorem 10.1]. (K) We will use this known result to deduce the counterpart for general GB .

29 Notice that BBT is positive deﬁnite, then we can deﬁne an isomorphism:

1 p1 p1 T 2 TB : R −→ R , TB(τ) := BB τ. (5.2) TBn

−1 Let TB denote its inverse. Set

−1 p1 −1 p0 T := T (t) = TB (t), t ∈ R ; X∗ := X∗(x∗) = TB (Bx∗) , x∗ ∈ R , (5.3) TX* and

T −1 T T −1 p0 X∗∗ := X∗∗(x∗) = x∗ − B TB (X∗) = x∗ − B (BB ) Bx∗, x∗ ∈ R . (5.4)

T −1 p1 Observe that B TB is an isometry on R , then we have T −1 T −1 T −1 −1 τ · Bx∗ = hB TB τ, B TB Bx∗i = (BB TB τ) · (TB Bx∗) = (TBτ) · X∗, (5.5) nIso

T T −1 and similarly t · τ = T · (TBτ). Moreover, by the fact that B (BB ) B is a projection p0 T p1 on R and |TB(τ)| = |B τ| for any τ ∈ R , we ﬁnd that

(K) (K) 2 φ (((x1, x∗), t); τ) = φ (((x1,X∗),T ); TB(τ)) + |X∗∗| . B Ip1 Thus, the following result is a direct consequence of [88, Theorem 10.1]: t5 Theorem 9. (1) We have

2 (K) (K) d(g) = sup φB (g; τ), ∀ g ∈ GB . (K) τ∈ΩB

(2) The cut locus of o, Cuto, is exactly

( 2 r ) c |X∗| = ((0, x∗), t); |X∗ · T | ≤ √ T − (Xc∗ · T ) Xc∗ , M π

c where we have used the convention (3.12). And for ((0, x∗), t) ∈ M , it holds that

2 2 d((0, x∗), t) = |x∗| + 4π T − (Xc∗ · T ) Xc∗ .

(K) (3) If (x, t) = ((x1, x∗), t) ∈ M, then there exists a unique θ = θ(x, t) ∈ ΩB such that 1 t = ∇ x2 f(|BTθ|) + |θ · Bx |2 ψ(|BTθ|) . 4 θ 1 ∗ Furthermore, we have 2 U (K)(θ) d(x, t)2 = φ(K)((x, t); θ) = B x B (K) sin UB (θ) " # |BTθ| 2 |BTθ| 2 |θ · Bx |2 = x2 + |x |2 + − 1 ∗ . sin |BTθ| 1 ∗ sin |BTθ| |BTθ|2

30 r5 Remark 11. 1. Using [88, Corollary 10.1] and (iii) of Corollary 9, we ﬁnd that

p0 ∗ CL c Me 2 = {((0, x∗), 0); x∗ ∈ R } = Abno = Abno, Cuto = M \ Me 2.

(K) 2. GB is of type GM, then other properties in Theorem 4, Corollaries 7 and 9 are also valid.

To end this section, we describe all

5.1 Shortest geodesic(s) joining o to any given g 6= o, as well as “bad” normal geodesics

Let us begin with the

5.1.1 Concrete expression of γ(w, 2 θ; s) on K-type groups

p0 1+p0 p1 Let x = (x1, x∗), xe = (xe1, xe∗) ∈ R × R = R and τ ∈ R . A simple calculation gives that

(K) (K) p1 hUB x, xei · τ = hUeB (τ) x, xei = (xe1 B x∗ − x1 B xe∗) · τ, ∀ τ ∈ R , (5.6) skew which implies (K) hUB x, xei = xe1 B x∗ − x1 B xe∗ := x ? x.e (5.7) skew2

(K) 1+p0 p1 Next we consider exp{UeB (τ)} ζ(0) with ζ(0) ∈ R and τ ∈ R . It can be treated via the Spectral Theorem or directly by the fact that (K) (K) (K) (K) (K) sin UB (τ) exp{UeB (τ)} = exp{−i UB (τ)} = cos UB (τ) + UeB (τ) (K) , UB (τ) which is, by (5.1), equal to ! cos (|BTτ|) sin (|BTτ|) 0 τ TB T cos (|B τ|)−1 T T + T T . (5.8) Ip0 + |BTτ|2 B ττ B |B τ| −B τ Op0×p0

1+p0 Now, we describe γ(w, 2 θ; s) := (x(s), t(s)) for given w := (w1, w∗) ∈ R and θ ∈ Rp1 . Let us introduce the column vectors ! ! ! θ · Bw∗ w1 0 v1 := , v2 := , v3 := , (5.9) n1v T θ·Bw∗ T θ·Bw∗ T −w1 B θ |BTθ|2 B θ w∗ − |BTθ|2 B θ 2 (θ · Bw∗) w := v ? v = − w2 + BBTθ, (5.10) w1 1 1 2 1 |BTθ|2 (θ · Bw )2 w := v ? v = −(θ · Bw ) Bw + ∗ BBTθ, (5.11) 2 1 3 ∗ ∗ |BTθ|2 θ · Bw∗ w := v ? v = −w B w − BTθ . (5.12) w3 3 2 3 1 ∗ |BTθ|2

31 It follows from (2.6) that sin(2 s |BTθ|) ζ(s) = v + cos(2 s |BTθ|) v + v , (5.13) expzeta |BTθ| 1 2 3 1 − cos(2 s |BTθ|) sin(2 s |BTθ|) x(s) = v + v + s v . (5.14) expx 2|BTθ|2 1 2|BTθ| 2 3

The calculation of t(s) is cumbersome. However, (5.7)-(5.12), (2.6) together with (5.13) and (5.14) imply that 1 t˙(s) = x(s) ? ζ(s) 2 cos(2 s |BTθ|) − 1 1 1 − cos(2 s |BTθ|) sin(2 s |BTθ|) = w + − s w 4 |BTθ|2 1 2 2 |BTθ|2 |BTθ| 2 1 sin(2 s |BTθ|) + − s cos(2 s |BTθ|) w . (5.15) expt 2 2 |BTθ| 3 Noticing that Z s T T T T −2 s |B θ| cos(2 s |B θ|) + sin(2 s |B θ|) r sin(2 r |B θ|) dr = T 2 , (5.16) n*n1 0 4 |B θ| Z s T T T T 2 s |B θ| sin(2 s |B θ|) − 1 + cos(2 s |B θ|) r cos(2 r |B θ|) dr = T 2 , (5.17) n*n2 0 4 |B θ| from (5.15), we get that sin(2 s |BTθ|) − 2 s |BTθ| t(s) = w 8 |BTθ|3 1 s |BTθ| + s |BTθ| cos(2 s |BTθ|) − sin(2 s |BTθ|) + w 4 |BTθ|3 2 −s |BTθ| sin(2 s |BTθ|) + 1 − cos(2 s |BTθ|) + w . (5.18) exptp 4 |BTθ|2 3 In particular, we can get some information about the set of the endpoints of nontrivial “bad” normal geodesics, as well as

5.1.2 The nontrivial “bad” normal geodesics ss512 We suppose in this subsection that

T ∗ w = (w1, w∗) 6= 0, and |B θ| = kπ with k ∈ N .

T p1 Recall that the isomorphism TB is deﬁned by (5.2) and |TB(τ)| = |B τ| for any τ ∈ R . Set

TB(θ) TB(θ) TB(θ) W := T−1 (Bw ) , η := = = . (5.19) nWn ∗ B ∗ b T |TB(θ)| |B θ| k π

32 Observe that from (5.5), we have that

θ · Bw∗ TB(θ) · W∗ = = η · W := w . (5.20) nX |BTθ| |BTθ| b ∗ e1 Let

(x, t) = ((x1, x∗), t) := exp(w, 2 θ) = γ(w,2θ)(1), (x(s), t(s)) := γ(w,2θ)(s).

Taking s = 1 and |BTθ| = k π in (5.14), we deduce that θ · Bw∗ x = 0, w − BTθ , (5.21) nxz ∗ |BTθ|2 namely x1 = 0 and BTθ x = w − (η · W ) , (5.22) xstar ∗ ∗ b ∗ |BTθ| where we have used (5.20). Similarly, (5.18) implies that 1 1 t = − w + w (5.23) ntz 4 |BTθ|2 1 2 |BTθ|2 2 θ · Bw |BTθ|2 w2 + 3 (θ · Bw )2 = − ∗ Bw + 1 ∗ BBTθ. 2 |BTθ|2 ∗ 4 |BTθ|4 In other words, by (5.20), we have

2 2 η · W∗ w + 3 (η · W∗) t = − b Bw + 1 b BBTθ. (5.24) n1t 2 |BTθ| ∗ 4 |BTθ|2

T 1 −1 −1 −1 Recall that TB := (BB ) 2 , X∗ := TB (Bx∗) and T := TB (t). Applying TB B to both sides of (5.22), it follows from (5.19) and (5.20) that

X∗ = W∗ − (ηb · W∗) ηb = W∗ − we1 η.b (5.25) Xstar −1 And similarly, applying TB to both sides of (5.24), we obtain 2 2 w1 w + 3 w T = − e W + 1 e1 η. (5.26) TWstar 2 k π ∗ 4 k π b We split it into cases.

Case 1. t = 0 so T = 0. In such case, taking inner product with ηb on both sides of (5.26) and using (5.20), we yield w1 = we1 = 0. So θ · Bw∗ = 0, and w∗ = x∗ because of (5.21). Hence, the vectors deﬁned by (5.9)-(5.12) in this situation are w1 = w2 = w3 = 0, v1 = v2 = 0 and v3 = x respectively. In conclusion, by (5.14) and (5.18), a simple calculation shows that the “bad” normal geodesic from o to ((0, x∗), 0) is the the straight segment γ((0,x∗),0)(s). From now on, we further assume that:

33 2 2 Case 2. t 6= 0 so T 6= 0. In such case, we have w1 + we1 > 0. Taking inner product with ηb on both sides of (5.25), by (5.20), we get X∗ · ηb = 0. Inserting (5.25) into (5.26), we obtain that

2 2 w1 w + w T = − e X + 1 e1 η. (5.27) TXstar 2 k π ∗ 4 k π b

And we will consider the cases X∗ 6= 0 and X∗ = 0. (I) Assume that X 6= 0. In such case, X and the unit vector η = TBθ are orthogonal. ∗ ∗ b k π It is easy to solve out (w1, we1) as well as θ from (5.27). Using (5.19), a direct calculation shows that ! T · Xc∗ −1 T − (T · Xc∗) Xc∗ we1 = −2 k π , θ = k π TB , |X∗| |T − (T · Xc∗) Xc∗| v !2 (5.28) nP1n u u 2 2 T · Xc∗ w1 = ±t4 k π T − (T · Xc∗) Xc∗ − 4 k π . |X∗|

(II) Assume that X∗ = 0, that is Bx∗ = 0. In such case, by (5.19), (5.27) implies that

−1 p θ = k π TB Tb , (w1, we1) = 4 k π |T | (cos σ, sin σ)(σ ∈ R). (5.29) nP2n

In conclusion, we have always

X∗ · ηb = 0. (5.30) o1n1

And (x, t) = ((0, x∗), t)(t 6= 0) is the endpoint of some “bad” normal geodesic γ((w1,w∗),2 θ) satisfying |BTθ| = k π (k ∈ N∗) if and only if

2 r |X∗| |X∗ · T | ≤ √ T − (Xc∗ · T ) Xc∗ . (5.31) kπ

In such case, taking (w1, we1, θ) as in (5.28) for X∗ 6= 0, or in (5.29) for X∗ = 0, it follows from (5.22) and (5.20) that

T · Xc∗ T we1 T w∗ = x∗ − 2 B θ = x∗ + B θ. (5.32) w*n |X∗| k π

θ·Bw∗ T Now recall that (see (5.21)) x∗ = w∗ − |BTθ|2 B θ. Substituting this as well as (5.20) and |BTθ| = kπ in (5.9)-(5.12), we get that

0 w w v = , v = 1 , v = k π e1 , 3 2 we1 T 1 w1 T x∗ k π B θ − k π B θ w2 = −(θ · Bw∗) Bx∗ = −k π we1 Bx∗, w3 = −w1 Bx∗,

34 and by (5.23), T 2 w1 = 2 w2 − 4 |B θ| t = −2 k π (we1 Bx∗ + 2 k π t) . Substituting them into (5.14) and (5.18), and replacing |BTθ| by k π, we obtain ﬁnally the expression of γ((w1,w∗),2 θ)(s) as follows:  ! ! !  0 sin(2 s k π) w1 1−cos(2 s k π) we1 x(s) = s + 2 k π + 2 k π  x we1 BTθ − w1 BTθ  ∗ k π k π  sin(2 s k π) 1−cos(2 s k π) (5.33) bngE t(s) = s − t + s w1 Bx∗  2 k π 4 k π e    s k π sin(2 s kπ)−1+cos(2 s k π) + 4 k2 π2 w1 Bx∗. Moreover, by (5.22) and (5.20), we get that

T 2 2 B θ 2 2 |w∗| = x∗ + w1 = |x∗| + w , e |BTθ| e1 since (5.5) says that BTθ T θ x · = X · B = X · η = 0, ∗ |BTθ| ∗ |BTθ| ∗ b where we have used (5.30) in the last equality. In conclusion, `2(γ ) = w2 + |w |2 = |x |2 + w2 + w2 ((w1,w∗),2 θ) 1 ∗ ∗ 1 e1

2 = |x∗| + 4 k π T − (T · Xc∗) Xc∗ . (5.34) Kgl

5.1.3 Shortest geodesic(s), as well as normal geodesics from o to any given g 6= o Recall that the “good” normal geodesics from o to any given g 6= o are characterized by [88, Theorem 2.4] in the general setting of step-two groups. Also in our special setting of K-type groups, the “bad” normal geodesics from o to any given g ∈ W \ {o} are characterized in the last subsection. Hence, we only study in the sequel the shortest geodesic(s) from o to any given g0 6= o. ∗ ∗ CL Consider the cases g0 ∈ M, g0 ∈ Abno \{o} and g0 ∈ Cuto \ Abno = Cuto . 1. If g0 ∈ M, there exists a unique shortest geodesic steering o to g0, and its equation is well-known by Theorem 2 together with (5.14) and (5.18). ∗ 2. For g0 ∈ Abno \{o}, the unique shortest geodesic is a straight segment and it is abnormal. ∗ 3. From now on, assume that g0 ∈ Cuto \ Abno. Using Theorem 5, any shortest geodesic joining o to g0 is given by γ(w,2 θ), where

1+p0 p1 (w, 2 θ) := ((w1, w∗), 2 θ) ∈ R × R with |w| = d(g0), T |B θ| = π, and exp(w, 2 θ) = g0. More precisely, by the results known in Subsection 5.1.2, we yield:

35 ∗ CL Proposition 6. Let g0 ∈ Cuto \ Abno = Cuto , namely g0 = ((0, x∗), t) with t 6= 0 and

2 r |X∗| |X∗ · T | ≤ √ T − (Xc∗ · T ) Xc∗ , π

where T and X∗ are deﬁned by (5.3). Then any shortest geodesic from o to g0,

γ((w1,w∗),2 θ)(s) = (x(s), t(s)),

can be written as in (5.33) with k = 1, w∗ deﬁned by (5.32), and (w1, we1, θ) by (5.28) for X∗ 6= 0, or by (5.29) for X∗ = 0. r |X |2 Remark 12. If we suppose further that X 6= 0 and |X · T | < √∗ T − (X · T ) X , ∗ ∗ π c∗ c∗

(5.28) implies that there are exactly two distinct shortest geodesics joining o to g0. To end this part, we determine on K-type groups

5.1.4 Optimal synthesis In the special case of K-type groups, combining (ii) of Corollary 9 with Proposition 1 and the result in Case 1 of Subsection 5.1.2, we obtain the following:

Corollary 14. Let exp{s (w, 2 θ)} be an arclength parametrized geodesic. Then its cut time hcut equals +∞ when w = (0, w∗) with θ · Bw∗ = 0, and π/kU(θ)k otherwise.

6 Sub-Riemannian geometry on step-two groups associated to quadratic CR manifolds s7 Let m, n ∈ N∗ with n ≥ m. Consider a full-rank m × n real matrix

T m A = (ea1,..., ean) with column vectors eaj := (a1,j, . . . , am,j) ∈ R .

(CR) A step-two group associated to quadratic CR manifolds of type A, GA , is deﬁned by (CR) G(2n, m, UA ) with (cf. for example [114] for more details)   0 aj,1  −a 0   j,1  (CR),(j)  ..  UA =  .  , 1 ≤ j ≤ m.    0 aj,n  −aj,n 0

36 For example, if m = 1 and A = (1,..., 1), (CR) is the Heisenberg group of real 1 GA1 dimension 2n + 1, 2n+1. Moreover, for m = n and A = , (CR) is the direct product H In GIn of n copies of Heisenberg group H3, namely

(CR) = 3 × · · · × 3. GIn H H

Observe that we have for τ ∈ Rm,   0 ea1 · τ  −a · τ 0   e1  (CR)  ..  UA (τ) = i  .  . (6.1) ucr    0 ean · τ  −ean · τ 0 Then the initial reference set in this situation is given by

n (CR) \ m Ω∗ := ΩA = {τ ∈ R ; |eaj · τ| < π} . (6.2) rsCR j=1

We identify R2 with C in the usual way, so R2n with Cn. Write in the sequel

n z = (z1, . . . , zn) ∈ C . A simple calculation shows that the reference function is

n (CR) X 2 φ((z, t); τ) := φA ((z, t); τ) = |zj| (eaj · τ) cot(eaj · τ) + 4 t · τ. (6.3) nRFe j=1

Recall that f(s) = 1 − s cot s and µ(s) = f 0(s) (see (2.25)). The gradient and the Hessian matrix of φ((z, t); ·) at τ ∈ Ω∗ are clearly

n X 2 ∇τ φ((z, t); τ) = − µ(eaj · τ) |zj| eaj + 4 t, (6.4) HMnn0 j=1 T T p p Hessτ φ((z, t); τ) = −A Λ(z; τ) A = −A Λ(z; τ) A Λ(z; τ) , (6.5) HMnn respectively, where

 0 2  µ (ea1 · τ) |z1|  ..  Λ(z; τ) =  .  ≥ 0, ∀ τ ∈ Ω∗, 0 2 µ (ean · τ) |zn| since µ0(s) > 0 for all −π < s < π (see for example Lemma 8 in Subsection 7.2 below). Recall that M is deﬁned in Subsection 2.3. We can characterize the squared sub- Riemannian distance as well as the cut locus in the following theorem.

37 t6 Theorem 10. (1) We have

2 (CR) (CR) d(g) = sup φA (g; τ), ∀ g ∈ GA . (CR) τ∈ΩA

c (2) The cut locus of o, Cuto, is M , where

( n ) 1 X = (z, t); span{a ; |z |= 6 0} = m and ∃ θ ∈ Ω s.t. t = µ(a · θ) |z |2 a . M ej j R ∗ 4 ej j ej j=1

(CR) (3) If (z, t) ∈ M, then there exists a unique θ = θ(z, t) ∈ ΩA such that

n 1 X t = µ(a · θ) |z |2 a . (6.6) 4 ej j ej j=1

Moreover, we have

n 2 X aj · θ d(z, t)2 = φ((z, t); θ) = e |z |2 sin(a · θ) j j=1 ej n X 2 = (eaj · θ) cot(eaj · θ) |zj| + 4 t · θ. (6.7) j=1

Proof. Let r(A) denote the rank of a real matrix A. Using the basic property r(AAT) = r(A), it deduces from (6.5) that

p r(Hessτ φ((z, t); τ)) = r A Λ(z; τ) = dim span{eaj; |zj|= 6 0},

since µ0(s) > 0 for s ∈ (−π, π) (see Lemma 8 in Subsection 7.2 below). Combining this with (6.4), we get immediately the characterization of M. And the third assertion of this theorem follows directly from Theorem 2. Moreover, it follows from [88, Proposition 2.2] that

M ⊇ {(z, t); |zj|= 6 0, ∀ 1 ≤ j ≤ n}, (6.8) charM

(CR) (CR) which is dense in GA . Then GA is of type GM. By Theorem 4 and Corollary 7, we c deduce the ﬁrst assertion, Cuto = M = ∂M, as well as other sub-Riemannian geometric properties.

Now we describe

6.1 Shortest geodesic(s) joining o to any g 6= o nSs61 Let us begin with the

38 6.1.1 Sub-Riemannian exponential map In the setting of step-two groups associated to quadratic CR manifolds, the equation of the normal geodesic (cf. (2.6)) as well as the sub-Riemannian exponential map becomes very concise via their special group structure, namely (6.1). More precisely, if p = (p1, . . . , pn) ∈ n ∼ 2n m C = R and θ ∈ R , let exp {s (p, 2 θ)} := (z(p, 2 θ; s), t(p, 2 θ; s)) := (z(s), t(s)), (6.9)

and exp(p, 2 θ) := (z(p, 2 θ), t(p, 2 θ)) := (z, t). (6.10)

It follows from (2.6) that

1 − e−2 i s (eaj ·θ) −2 i s (aj ·θ) ζj(s) = e e pj, zj(s) = pj, 1 ≤ j ≤ n, 2 i (eaj · θ) and for 1 ≤ k ≤ m,

n X Z s 0 1 t (s) = 2−1 a z (r), ζ (r) dr k k,j −1 0 j j j=1 0 n Z s −1 X = −2 ak,j < i zj(r) · ζj(r) dr j=1 0 n X 2 s (aj · θ) − sin (2 s (aj · θ)) = e e |p |2 a . 8 (a · θ)2 j k,j j=1 ej

In conclusion, the normal geodesic with the initial covector (p, 2 θ), γ(p, 2 θ; s), is given by

 1−e−2 i s (aej ·θ) zj(s) = pj, 1 ≤ j ≤ n,  2 i (eaj ·θ) n (6.11) ngE P 2 s (eaj ·θ)−sin (2 s (eaj ·θ)) 2  t(s) = 2 |pj| aj.  8 (aj ·θ) e j=1 e

In particular, we yield the expression of exp{(p, 2θ)},

 1−e−2 i (aej ·θ) zj(p, 2 θ) = pj, 1 ≤ j ≤ n,  2 i (eaj ·θ) n (6.12) expmap3 P 2 (eaj ·θ)−sin(2 (eaj ·θ)) 2  t(p, 2 θ) = 2 |pj| aj.  8 (aj ·θ) e j=1 e

6.1.2 Shortest geodesic(s) joining o to any given g 6= o ss612 We are in a position to determine all shortest geodesics from o to any given o 6= g0 := (0) (0) (z0, t0) with z0 := (z1 , . . . , zn ). First, notice that in our framework, the set deﬁned by

39 (2.18) is

( n ) 1 X 2 e = (z, t); ∃ θ ∈ Ω∗ s.t. t = µ(aj · θ)|zj| aj . M 4 e e j=1

c And we consider the following two cases g0 ∈ Me \{o} and g0 ∈ Me . Case 1. g0 ∈ Me \{o}, namely φ(g0; ·) attains its maximum at some point in Ω∗, saying θ0. It follows from Theorem 2 that there exists a unique shortest geodesic joining o to g0, that is, exp{s (p(g0), 2 θ0)} (0 ≤ s ≤ 1) with

U(θ0) aj · θ0 (0) −Ue(θ0) e i (eaj ·θ0) p(g0) = e z0, i.e. pj(g0) = e zj , ∀ 1 ≤ j ≤ n. (6.13) sin U(θ0) sin (eaj · θ0) Substituting this in (6.11), we yield its concise expression. Moreover, it is strictly normal if g0 ∈ M and also abnormal for g0 ∈ Me \ (M ∪ {o}). c Case 2. g0 ∈ Me , that is φ(g0; ·) only attains its supremum in Ω∗ at some θ ∈ ∂Ω∗. In order to characterize all shortest geodesics from o to g0, we set 2 E(g0) := θ ∈ ∂Ω∗; φ(g0; θ) = sup φ(g0; τ) = d(g0) . τ∈Ω∗

From Theorem 5, it remains to determine all (p(g0), 2 θ(g0)) such that

θ(g0) ∈ E(g0), |p(g0)| = d(g0) and exp{(p(g0), 2 θ(g0))} = g0. (6.14) crIC

Up to rearrangements, we may assume that there exists an L ∈ N, L < n such that

|eaj · θ(g0)| < π for 1 ≤ j ≤ L, and |eaj · θ(g0)| = π if L + 1 ≤ j ≤ n. (6.15) CRc1 By (6.12), we obtain that

2 i (eaj · θ(g0)) (0) (0) pj(g0) = zj , ∀ 1 ≤ j ≤ L, zj = 0 for L + 1 ≤ j ≤ n, (6.16) equgeocr 1 − e−2 i (eaj ·θ(g0)) and (6.14) holds if and only if (pL+1(g0), . . . , pn(g0)) is a solution of the following equation:

n L X 1 2 X (0) 2 |pj(g0)| eaj = 4 t0 − µ(eaj · θ(g0)) |zj | eaj. (6.17) equgeoCR aj · θ(g0) j=L+1 e j=1

2 2 To prove this result, it suﬃces to show that |p(g0)| = d(g0) under our assumptions (6.15)-(6.17), and other claims are clear. Indeed, from (3.11) and (6.16), we have

L 2 X (0) 2 d(g0) = φ(g0; θ(g0)) = (eaj · θ(g0)) cot(eaj · θ(g0))|zj | + 4 t0 · θ(g0). (6.18) expphibdCR j=1

40 When (6.17) holds, taking inner product with θ(g0) on both sides of (6.17), we obtain that n L X 2 X (0) 2 |pj(g0)| = 4 t0 · θ(g0) − (eaj · θ(g0)) µ(eaj · θ(g0)) |zj | . j=L+1 j=1

L P 2 Summing with |pj(g0)| on both sides of last equality, it follows from (6.16) that j=1

L 2 ! X aj · θ(g0) (0) |p(g )|2 = 4 t · θ(g ) + e − (a · θ(g )) µ(a · θ(g )) |z |2. 0 0 0 sin(a · θ(g )) ej 0 ej 0 j j=1 ej 0

By (6.18) and the elementary identity

s 2 − s µ(s) = s cot s, sin s

2 2 we get |p(g0)| = d(g0) . In particular, if ((p1(g0), . . . , pL+1(g0), . . . pn(g0)), 2 θ(g0)) satisﬁes the condition (6.14), then so does ((p1(g0), . . . , wL+1 pL+1(g0), . . . , wn pn(g0)), 2 θ(g0)) for any complex numbers (wL+1, . . . , wn) satifying |wL+1| = ... = |wn| = 1.

c Thus, there exist inﬁnitely many shortest geodesics from o to g0 ∈ Me if

L X (0) 2 4 t0 − µ(eaj · θ(g0)) |zj | eaj 6= 0. j=1

In such case, it follows from [123, Lemma 9] that g0 belongs to the classical cut locus of CL c CL o, Cuto . This provides another explanation for such g0 ∈ Me belonging to Cuto besides (iii) of Corollary 9. Finally we remark that the meaning for the case L = 0 is clear in the above discussion. Step-two groups associated to quadratic CR manifolds have very rich sub-Riemannian geometric properties. First, we provide an example of such groups on which (2.27) is no ∗ longer valid for some g0 ∈ Abno \ Me 2.

6.2 (2.29) can be false at a point belonging to the shortest abnormal set on GM-groups s71 (CR) ∗ p5 Proposition 7. There exist GA , g0 ∈ Abno \ Me 2 and c0 > 0 such that

2 2 2 d(g0 + hg) + d(g0 − hg) − 2 d(g0) ≤ 0, (6.19) semiconc

n m 2 2 2 for all g = (z, t) ∈ C × R with |g| = |z| + |t| = 1 and 0 < h < c0.

41 Proof. Setting

1 1 1 a := , a := , a := e1 0 e2 1 e3 −1 and A := (ea1, ea2, ea3), we obtain a step-two group associated to quadratic CR manifolds (CR) GA . In our case, we have

(CR) Ω∗ = ΩA = {τ; −π < τ1 ± τ2 < π}.

3 2 Consider g0 := (0, e1) ∈ C × R with e1 = (1, 0). Using the ﬁrst result in Theorem 10 and (6.3), we have that g0 6∈ Me (see (2.18) for its deﬁnition) since the unique θ0 ∈ Ω∗ satisfying

2 d(0, e1) = sup φ((0, e1); τ) = sup 4 (e1 · τ) = 4π = 4 (e1 · θ0), (6.20) element2 τ∈Ω∗ τ∈Ω∗ is θ0 = (π, 0) ∈ ∂Ω∗. ∗ Next, we will show that g0 ∈ Abno. Let

√ 3 p = (p1, p2, p3) := (2 π, 0, 0) ∈ C . Observe that, by (6.12), we have

exp(p, 2 θ0) = (0, e1). (6.21) element1

Combining this with (6.20), we ﬁnd that γ(p,2 θ0) is a shortest geodesic from o to g0. It remains to prove that it is also abnormal. (CR) k ∗ Indeed, set e2 = (0, 1). From (6.1), p as well as UA (θ0) p (k ∈ N ) belongs to the (CR) kernel of UA (e2), namely

(CR) (CR) k UA (e2) UA (θ0) p = 0, ∀ k ∈ N. (6.22) element3

∗ Hence, Proposition 1 implies that γ(p,2 θ0) is also abnormal. So (0, e1) ∈ Abno. Now, we are in a position to prove (6.19) under our assumptions. Set in the sequel

3π Ω := τ; < τ ± τ < π ⊆ Ω . ♦ 4 1 2 ∗

First, we suppose that g = (z, t) with zj 6= 0 for all j = 1, 2, 3. Then (6.8) implies that g(±h) := g0 ± h g ∈ M for any h 6= 0. Let us begin with g(h). By Theorem 2, there 3 exists a unique (ζ(h), θ(h)) ∈ (C \{0}) × Ω∗ such that exp{(ζ(h), 2 θ(h))} = g(h). Next, we claim that θ(h) ∈ Ω♦ for any |g| = 1 and any 0 < h ≤ c0 with c0 < 1/4 small ∗ ∗ 3 enough. Otherwise, by compactness there exists a (ζ , θ ) ∈ (C \{0}) × Ω∗ \ Ω♦ such ∗ ∗ ∗ that |ζ | = d(g0) and exp(ζ , 2 θ ) = g0, which implies γ(ζ∗,2 θ∗) is also a shortest geodesic

42 ∗ ∗ ∗ from o to g0. From Case 2 in Subsection 6.1.2, we have that γ(ζ ,2 θ ) = γ(p ,2 θ0) since ∗ ∗ ∗ ∗ 3 ∗ 2 E(g0) = {θ0}. Furthermore, p := (p1, p2, p3) ∈ C , satisfying |p | = 4π, is a solution of

3 X ∗ 2 |pj | eaj = 4π e1. (6.23) element4 j=1

From Lemma 2, we get p∗ = ζ∗, and Proposition 1 implies that

(CR) ∗ ∗ UA (θ0 − θ ) p = 0. (6.24) element5

∗ ∗ ∗ ∗ If p1 6= 0, using (6.1), we obtain that ea1 · θ = ea1 · θ0 = π, so θ = (π, 0) since θ ∈ Ω∗, which contradicts with the fact θ∗ ∈ Ω \ Ω . In the opposite case p∗ = 0, it follows from √ ∗ ♦ 1 ∗ ∗ ∗ (6.23) that |p2| = |p3| = 2π. Using (6.1), we get that eaj · (θ0 − θ ) = 0 for j = 2, 3 since ∗ ∗ ∗ p2, p3 6= 0. Observe that ea2 and ea3 are linearly independent, and thus we have θ0 = θ , which leads to a contradiction as well. Consequently, for such g(h), by (3) of Theorem 10, we get that

3 2 X 2 2 d(g(h)) = φ(g(h); θ(h)) = (eaj · θ(h)) cot(eaj · θ(h)) |zj| h + 4 (e1 + h t) · θ(h) j=1

≤ 4 (e1 + h t) · θ(h),

where we have used in the inequality that θ(h) ∈ Ω♦. Let (a, b) := 4 (e1 + h t). Observe 1 that a ≥ 2 ≥ |b| since 0 < h ≤ c0 < 4 and |t| ≤ 1 from our assumption. So the function deﬁned on Ω∗, κ(τ1, τ2) := a τ1 + b τ2 attains its maximum at θ0. In conclusion,

2 d(g(h)) ≤ 4 (e1 + h t) · θ0, ∀ |g| = 1, 0 < h < c0 1. (6.25) estplus

And similarly, we have

2 d(g(−h)) ≤ 4 (e1 − h t) · θ0, ∀ |g| = 1, 0 < h < c0 1. (6.26) estminus

We use (6.20) together with (6.25) and (6.26), and obtain (6.19) under the additional condition that (z, t) satisfying zj 6= 0 for all j = 1, 2, 3. Finally a limiting argument ﬁnishes the proof of this proposition.

(CR) ∗ To ﬁnish this section, we provide an example of GA on which Abno 6= Abno.

6.3 The shortest abnormal set is not always equal to the abnormal set s72 (CR) p6 Proposition 8. There exists a step-two group associated to quadratic CR manifolds GA ∗ such that Abno $ Abno.

43 Proof. Setting 2−1 1 1 a = , a = , a = and A = (a , a , a ), e1 0 e2 1 e3 −1 e1 e2 e3

(CR) we obtain a step-two group associated to quadratic CR manifolds GA . And we will 3 2 ∗ show that the point g0 := (0, e1) ∈ C × R with e1 = (1, 0) satisﬁes g0 ∈ Abno \ Abno. In our situation, (6.2) implies that the initial reference set is given by

(CR) Ω∗ = ΩA = {τ = (τ1, τ2); −π < τ1 ± τ2 < π}. We argue as in the proof of Proposition 7, and get that 2 d(g0) = sup φ(g0; τ) = sup 4 (e1 · τ) = 4π = 4 (e1 · θ0), τ∈Ω∗ τ∈Ω∗

c where θ0 = (π, 0) ∈ ∂Ω∗ is the unique maximum point of φ(g0; ·) in Ω∗. So g0 ∈ Me . Via a simple calculation, Case 2 in Subsection 6.1.2 implies that any shortest geodesic joining o to g can be written as γ (s) := exp{s (p, 2 θ )} (0 ≤ s ≤ 1) where p := (0, p , p ) ∈ 3 0 √ p 0 2 3 C with |p2| = |p3| = 2π. We claim that all γ are strictly normal, so g ∈/ Abn∗. We argue by contradiction: p 0 √o ∗ ∗ ∗ ∗ ∗ assume that there exists p = (0, p2, p3) with |p2| = |p3| = 2π such that γp∗ is abnormal. It follows from Proposition 1 that there exists a σ ∈ R2 \{0} such that (CR) (CR) k ∗ UA (σ) UA (θ0) p = 0, ∀ k ∈ N. (6.27)

(CR) ∗ In particular, we yield UA (σ) p = 0. Using (6.1), we obtain that eaj · σ = 0 for j = 2, 3 ∗ ∗ since p2, p3 6= 0. Notice that ea2 and ea3 are linearly independent. Hence we have σ = 0, which leads to a contradiction. On the other hand, we set √ 3 p∗ = (2 2π, 0, 0) ∈ C , θ∗ = (2π, 0),

and consider the normal geodesic γ∗(s) := exp(s (p∗, 2 θ∗)) (0 ≤ s ≤ 1). It follows from (6.12) that exp{(p∗, 2 θ∗)} = g0, that is, γ∗ is a normal geodesic joining o to g0. Furthermore, let e2 = (0, 1). From (6.1), a simple computation shows that

(CR) (CR) k UA (e2) UA (θ∗) p∗ = 0, ∀ k ∈ N. (6.28)

Combining this with Proposition 1, γ∗ is also abnormal. As a result, we have g0 ∈ Abno, which ends the proof of this proposition.

7 Gaveau-Brockett optimal control problem on N3,2 s5 The purpose of this section is to provide a new and independent proof, based on [88], for the Gaveau-Brockett optimal control problem on the free Carnot group of step two and 3 generators N3,2. More precisely, we will give a diﬀerent proof for Theorem 12 below. For this purpose, we start by

44 7.1 Preliminaries and known results obtained in [88]

3 3 Recall that N3,2 = R × R with   0 −τ3 τ2 3 U(τ) := i  τ3 0 −τ1 , τ = (τ1, τ2, τ3) ∈ R , −τ2 τ1 0 that is, hUx, x0i = x × x0, where “×” denotes the cross product on R3 and hence Ue(τ) x = τ × x. In our situation, by considering τ as a column vector, U(τ) |τ| |τ| ττ T = − − 1 sin U(τ) sin |τ|I3 sin |τ| |τ|2 and the initial reference set and the reference function are given respectively by Ω∗ = {τ; |τ| < π} and 1 − |τ| cot |τ| φ((x, t); τ) = (|τ| cot |τ|)|x|2 + (τ · x)2 + 4 t · τ. (7.1) |τ|2

See [88, § 11] for more details. We will solve the Gaveau-Brockett optimal control problem on N3,2, namely to ﬁnd the exact expression of d(x, t)2. Using a limiting argument, the scaling property (see (2.2)), and an orthogonal invariance, namely (cf. [88, Lemma 11.1]) 2 2 d(x, t) = d(O x, O t) , ∀ (x, t) ∈ N3,2, ∀ O ∈ O3, (7.2) nOIP

2 where O3 denotes the 3 × 3 orthogonal group, it suﬃces to determine d(e1, t1 e1 + t2 e2) with t2 > 0. To begin with, we recall some notations and known results: 2 2 2 2 Ω+ := {(v1, v2) ∈ R ; v2 > 0, v1 + v2 < π }, (7.3) O+ 2 2 := (u , u ) ∈ 2; u > √ p|u | ≥ 0 , (7.4) R> 1 2 R 2 π 1 2 √ 2 := (u , u ); u > 0, 0 < u < √ u . (7.5) R<,+ 1 2 1 2 π 1 Set in the sequel √ 3 2π < ϑ < π such that tan ϑ = ϑ , (7.6) 1 2 1 1 ψ0(r) q K (v , v ) := 2ψ(r) + v2 with r := v2 + v2, (7.7) nK3N 3 1 2 r 2 1 2 and q 2 2 Ω−,4 := (v1, v2); v2 < 0, π < v1 < r = v1 + v2 < ϑ1, K3(v1, v2) < 0

= {(v1, v2); v2 < 0 < v1, K3(v1, v2) < 0, π 6= r < ϑ1} . (7.8) Omega-4

45 Indeed, to show the last equality in (7.8), by [88, Lemma 3.4], we have that π < r < ϑ1 and

" +∞ +∞ # X −2 X −1 1 0 > K (v , v ) = 4 v2 (j π)2 − r2 + (j π)2 − r2 − 3 1 2 2 r2 − π2 j=1 j=2 π2 − v2 > 4 1 , (r2 − π2)2

which implies v1 > π since v1 > 0. Moreover, for suitable E ⊆ R2, we deﬁne the smooth function Λ, ψ0(r) Λ(v , v ) := v v v + 2 ψ(r) e , v = (v , v ) ∈ E, r = |v|. (7.9) 32N96 1 2 2 r 2 2 1 2

The following results can be found or deduced directly from [88, § 11]:

RLT1 Theorem 11 ( [88]). It holds that: (i) d(e1, t1 e1 + t2 e2) = d(e1, |t1| e1 + t2 e2). ∞ 2 (ii) Λ is a C -diﬀeomorphism from Ω+ onto R>. 2 (iii) For suitable (θ1, θ2) ∈ R , set 1 θ := (θ , θ , 0), t := (Λ(θ , θ ), 0), g := (e , t ). (7.10) n78n 1 2 θ 4 1 2 θ 1 θ Then

2 2 2 2 θ1 θ2 U(θ) d(gθ) = φ(gθ; θ) = + = e1 , ∀ (θ1, θ2) ∈ Ω+. (7.11) dEn |θ|2 sin |θ| sin U(θ)

−1 α2 2 2 2 2 (iv) For any α ≥ 0, et(α) := 4 ( π , π α) ∈ ∂R> and d(e1, (et(α), 0)) = 1 + α . ∗ 3 (v) Abno = Abno = {(x, 0); x ∈ R } = Me 2 and x x 1 p (x, t); x 6= 0, t 6= 0, t − ht, i > √ |x| |t · x| ⊆ S. |x| |x| π

∞ 2 (vi) Λ is a C -diﬀeomorphism from Ω−,4 onto R<,+. (vii) We have d(0, t)2 = 4π|t| for all t ∈ R3.

2 In conclusion, via a limiting argument, it remains to determine d(gθ) with (θ1, θ2) ∈ Ω−,4. Indeed, we have the following:

t4 Theorem 12 ( [88], Theorem 11.3). (7.11) remains valid for any (θ1, θ2) ∈ Ω−,4.

46 7.2 Properties of some functions related to −s cot s s4 Recall that the functions f, µ and ψ are deﬁned by (2.25). The following lemma can be found in [63, Lemme 3, p. 112] or [27, Lemma 1.33]:

NL31 Lemma 8. The function µ is an odd function, and a monotonely increasing diﬀeomor- phism between (−π, π) and R. In the sequel, let us deﬁne

s 2 ϕ (s) := − 1, s ∈ , (7.12) nvp0 0 sin s R and for s > 0,

h(s) := ψ0(s)s3 sin2 s = s2 + s sin s cos s − 2 sin2 s, (7.13) ndH 2 2 s − sin s ϕ0(s) ϕ0(s) ϕ (s) := = = , (7.14) nvp1 1 s − sin s cos s µ(s) s2 ψ0(s) + 2s ψ(s) 2 2 s (s − sin s) ϕ0(s) ϕ0(s) ϕ (s) := = = , (7.15) nvp2 2 s2 + s sin s cos s − 2 sin2 s µ(s) − 2s ψ(s) s2 ψ0(s)

and p ϕ3(s) := ϕ1(s) ϕ2(s). (7.16) nvp3

∗ 1 For k ∈ N , let ϑk denote the unique solution of s = tan s on kπ, (k + 2 )π . We will need the following lemma in order to prove Theorem 12:

l1 Lemma 9. We have (1) h(r) > 0 for all r > 0. So, ψ0(r) > 0 for 0 < r 6∈ {kπ; k ∈ N∗}. (2) ϕ1 is strictly increasing on (0, +∞). +∞ (3) ϕ2 is strictly increasing on ∪k=1(kπ, ϑk). (4) ϕ3 is strictly increasing on (π, +∞). Proof. Notice that (1) can be found in [103, Lemma 3.1], and (2) as well as the strict +∞ monotonicity of ϕ3 on ∪k=1(kπ, ϑk) can be found in the proof of [103, Lemma 3.4]. For the sake of clarity, we will provide a complete proof which is not complicated. We begin with the proof of (1). Obviously, it suﬃces to prove the ﬁrst claim. Indeed, π when r ≥ 2 , we have 3 π − 3 h0(r) = r [2 + cos(2r)] − sin(2r) ≥ > 0. (7.17) fprime 2 2

π In the opposite case r ∈ 0, 2 , we have the elementary inequality sin r > r cos r and it is clear that

h00(r) = 2 − 2 cos(2r) − 2r sin(2r) = 4 sin r (sin r − r cos r) > 0,

47 which implies that h0(r) > lim h0(r) = 0 for r ∈ 0, π . Combining this with (7.17), we r→0+ 2 get that h(r) > lim h(r) = 0 for r > 0, which ends the proof of the ﬁrst assertion. r→0+ To prove (2), let us set

r − sin r cos r − sin2 r + r sin r cos r F (r) := and G(r) := . (7.18) dFG r2 − sin2 r r (r2 − sin2 r)

Remark that 1 1 1 F (r) = ,F (r) + 2G(r) = ,F (r) + G(r) = . (7.19) Fp2G ϕ1(r) ϕ2(r) r

A simple computation gets that

(r cos r − sin r)2 F 0(r) = −2 < 0, ∀ r ∈ (0, +∞) \{ϑ ; k ∈ ∗}, (7.20) doF (r2 − sin2 r)2 k N which implies the strict monotonicity of ϕ1. We return to the proof of (3). Using (7.19), we have that

1 0 2 (r cos r − sin r)2 = 2(F (r) + G(r))0 − F 0(r) = − + 2 2 2 2 2 ϕ2(r) r (r − sin r) (r2 cos r − r sin r − r2 + sin2 r)(r2 cos r − r sin r + r2 − sin2 r) = 2 . r2 (r2 − sin2 r)2

Observe that for r ∈ (kπ, ϑk) with k odd, we have that r cos r < sin r < 0, so

( r2 cos r − r sin r − r2 + sin2 r = r (r cos r − sin r) − (r2 − sin2 r) < 0 . r2 cos r − r sin r + r2 − sin2 r = r2 (cos r + 1) − sin r (r + sin r) > 0

Similarly, if r ∈ (kπ, ϑk) with k even, we have that 0 < sin r < r cos r and

( r2 cos r − r sin r − r2 + sin2 r = r2 (cos r − 1) + sin r (−r + sin r) < 0 . r2 cos r − r sin r + r2 − sin2 r = r (r cos r − sin r) + (r2 − sin2 r) > 0

0 1 +∞ Hence we have that < 0 on ∪ (kπ, ϑk). Finally, a direct computation gives ϕ2 k=1

ϕ2(kπ) = kπ and ϕ2(ϑk) = ϑk ∀ k ≥ 1,

+∞ which ﬁnishes the proof of the strict monotonicity of ϕ2 on ∪k=1(kπ, ϑk). We are in a position to prove the strict monotonicity of ϕ3. By the fact that ϕ3 = √ +∞ ϕ1 ϕ2, it following from (2) and (3) that ϕ3 is strictly increasing on ∪k=1(kπ, ϑk). Then

48 +∞ it remains to prove that it is also strictly increasing on ∪k=1(ϑk, (k + 1)π). Indeed, by using (7.19) again, we have that

0 0 1 0 2 2 = [(F (r) + 2G(r))F (r)] = − F (r) F (r) ϕ3(r) r 2 0 = − F (r) F (r) + (F (r) + 2G(r))F 0(r) r 2 = − F (r) + 2G(r)F 0(r). r2

From (7.18) and (7.20), the last term equals

(r − sin r cos r)(r2 − sin2 r)2 + 2r (r cos r − sin r)2 (− sin2 r + r sin r cos r) −2 . r2 (r2 − sin2 r)3

Note that we have for r > ϑ1 > 4,

(r − sin r cos r)(r2 − sin2 r)2 + 2 r (r cos r − sin r)2(− sin2 r + r sin r cos r) 1 r ≥ r − (r2 − 1)2 − 2 r (r + 1)2 + 1 2 2 1 = (r + 1)2 (r − )(r − 1)2 − r2 − 2r 2 ≥ (r + 1)2 3 (r − 1)2 − r2 − 2r > 2 r (r − 4) + 3 > 0, which proves our lemma.

7.3 Determination of W in our situation In order to prove Theorem 12, we will use [88, Corollary 2.1] in which we have assumed that g does not belong to W (cf. (2.24)). Let us begin with the

7.3.1 Expression of γ(w, 2 θ; s) = (x(s), t(s)) on N3,2 We ﬁrst recall the convention (3.12). As in the setting of K-type groups, an elementary computation gives that

cos(2 s U(θ)) w = cos (2 s |θ|)(w − (w · θb) θb) + (w · θb) θ,b sin(2 s U(θ)) sin(2 s |θ|) w = (w − (w · θb) θb) + 2s (w · θb) θ,b U(θ) |θ| sin(2 s U(θ)) Ue(θ) w = sin(2 s |θ|)(θb× w), U(θ)

49 where we have used the fact that Ue(τ) x = τ × x. Consequently, using (2.6), we obtain that sin(2 s U(θ)) ζ(s) =x ˙(s) = cos(2 s U(θ)) w + Ue(θ) w U(θ) = cos(2 s |θ|)(w − (w · θb) θb) + (w · θb) θb+ sin(2 s |θ|)(θb× w), sin(2 s |θ|) 1 − cos(2 s |θ|) x(s) = (w − (w · θb) θb) + s (w · θb) θb+ (θb× w). (7.21) expxN 2|θ| 2|θ|

Then using (2.6) again we have 1 1 sin(2 s |θ|) 1 − cos(2 s |θ|) t˙(s) = x(s) × ζ(s) = − s cos(2 s |θ|) u + u 2 2 2|θ| 1 4|θ| 2 1 1 − cos(2 s |θ|) + s sin(2 s |θ|) − u , (7.22) exptN 2 2|θ| 3 with

u1 = (w − (w · θb) θb) × [(w · θb) θb] = −(w · θb)(θb× w), (7.23) 2 2 u2 = (w − (w · θb) θb) × (θb× w) = (|w| − (w · θb) ) θ,b (7.24)

u3 = (w · θb) θb× (θb× w) = −(w · θb)[w − (w · θb) θb], (7.25) where we have used the well-known vector triple product expansion:

3 a × (b × c) = (a · c) b − (a · b) c, ∀ a, b, c ∈ R . (7.26) triex

As a result, by using (5.16) and (5.17), we can write 1 − cos(2 s |θ|) − s |θ| sin(2 s |θ|) 2 s |θ| − sin(2 s |θ|) t(s) = u + u 4|θ|2 1 8|θ|2 2 sin(2 s |θ|) − s |θ| − s |θ| cos(2 s |θ|) + u . (7.27) exptpN 4|θ|2 3

Now, we provide the

1 7.3.2 Description of W ∩ (e1, 4 (u1, u2, 0)); u1, u2 ∈ R In this subsection, we suppose that |θ| = kπ with k ∈ N∗. Substituting this with s = 1 in (7.21) and (7.27), (x(w, 2 θ), t(w, 2 θ)) := exp(w, 2 θ) is given by ( x(w, 2 θ) = (w · θb) θb 2 2 . (7.28) expmap 1 1 w·θb |w| −3 (w·θb) t(w, 2 θ) = 4|θ| u2 − 2|θ| u3 = 2|θ| w + 4|θ| θb

Then we have the following lemma, which implies that the set W is negligible in the “subspace” as well.

50 1 1 2 l2 Lemma 10. Let t = 4 (u, 0) = 4 (u1, u2, 0) such that (e1, t) ∈ W. Then we have 16u1 = 2 2 4 ∗ k π u2 for some k ∈ N .

3 3 Proof. Let (w, 2 θ) ∈ R × R with w := (w1, w2, w3) such that exp(w, 2 θ) = (e1, t) and |θ| = kπ for some k ∈ N∗. It follows from (7.28) that

w · θb |w|2 − 3 (w · θb)2 e1 = (w · θb) θ,b t = w + θ.b (7.29) n723n 2|θ| 4|θ|

The ﬁrst equality implies that θ = ±(kπ, 0, 0) := (θ1, 0, 0), w · θb = ±1. Furthermore, w · θb and θ1 have the same sign. Taking inner product on both sides of the second identity in (7.29) with θb, we obtain

|w|2 − (w · θb)2 t · θb = , (7.30) n724n 4|θ|

and θ1u1 ≥ 0. Multiplying both sides of (7.30) by −θb, and summing with both sides of the second equation in (7.29) respectively, we have

w · θb t − (t · θb) θb = (w − (w · θb) θb). (7.31) tminus 2|θ|

w·θb In particular, we get that w3 = 0 and u2 = 2 |θ| w2. By Pythagoras Theorem, we can write

(w · θb)2 |t|2 = |(t · θb) θb|2 + |t − (t · θb) θb|2 = (t · θb)2 + (|w|2 − (w · θb)2) 4|θ|2 1 = (t · θb)2 + (4 t · θ) 4|θ|2 1 = (t · θ)2 + (t · θ) , |θ|2

where we have used (7.31) and Pythagoras Theorem in the second “=”, (7.30) and |w·θb| = 1 1 in the third “=”. Inserting t = 4 (u1, u2, 0) and θ = ±(kπ, 0, 0) in the last equation gives the desired result. √ r2 Remark 13. Assume t = 1 (u , √2 u , 0) with u > 0 and k ∈ ∗. It follows from the 4 1 k π 1 1 N proof above that the “bad” normal geodesic joining o to (e1, t) ∈ W is p γ(w,2 θ)(s) (0 ≤ s ≤ 1) with w = (1, kπ u1, 0) and θ = (kπ, 0, 0).

51 More precisely, from (7.21) and (7.27), γ(w,2 θ)(s) := (x(s), t(s)) (0 ≤ s ≤ 1) is given by:

s  0   0  sin(2 s kπ) √ 1 − cos(2 s kπ) x(s) = 0 + kπ u + 0   2kπ  1 2kπ √  0 0 kπ u1   0  1 sin(2 s kπ) − s kπ − s kπ cos(2 s kπ) √ t(s) = − kπ u 4  k2π2  1 0  0  kπ u  s kπ sin(2 s kπ) − 1 + cos(2 s kπ) 2 s kπ − sin(2 s kπ) 1 + 0 + 0 . k2π2 √  2k2π2   kπ u1 0

We are in a position to provide the

7.4 Proof of Theorem 12

1−s cot s Recall that ψ(s) = s2 and ϕ0(s) is deﬁned by (7.12). Set in the following:

w2 Φ(w) := ϕ (|w|) 2 , w = (w , w , w ) ∈ 3. 0 |w|2 1 2 3 R

Let (θ1, θ2) ∈ Ω−,4. Recall that (cf. (7.10))

1 θ := (θ , θ , 0), (u , u ) := Λ(θ , θ ) ∈ 2 , t := (u , u , 0), g := (e , t ). 1 2 1 2 1 2 R<,+ θ 4 1 2 θ 1 θ

By Lemma 10, via a limiting argument, we may suppose in the sequel that gθ ∈/ W. Then all the normal geodesics joining o to gθ are “good” ones, so [88, Corollary 2.1] gives that

" 2 2# 2 τ1 τ2 d(e1, tθ) = inf 2 + = inf {Φ(τ) + 1} , (7.32) τ∈Υθ |τ| sin |τ| τ∈Υθ

where Υθ denotes the set of τ = (τ1, τ2, 0) such that Λ(τ1, τ2) = (u1, u2) (see (7.9)), namely,

 0 (0 <) u = ψ (|τ|) τ τ 2  1 |τ| 1 2 . (7.33) nELn 0  ψ (|τ|) 2  (0 <) u2 = τ2 K3(τ1, τ2) = τ2 |τ| τ2 + 2 ψ(|τ|)

Here are some direct observations.

52 Observations: Under the above assumptions, for any τ ∈ Υθ, we have ∗ 1. |τ| ∈/ {k π; k ∈ N } since (e1, tθ) ∈/ W. 0 2. Moreover τ2 6= 0 and τ1 > 0 since u1 > 0 and ψ > 0 from (1) of Lemma 9. 3. Furthermore |τ| > π. Otherwise |τ| < π, u2 > 0 and the second equation in (7.33) 2 imply that τ2 > 0. So τ ∈ Ω+. Hence it follows from (ii) of Theorem 11 that (u1, u2) ∈ R>. This leads to a contradiction. 4. The following equalities hold, in particular for θ,

s |τ| τ1 τ2 τ2 Φ(τ) = ϕ2(|τ|) u1 = ϕ1(|τ|) u1 + u2 = ϕ3(|τ|) u1 u1 + u2 . (7.34) ndEn τ1 |τ| |τ| τ1

Indeed, the first “=” follows from the definition of ϕ2 (see (7.15)) and the first equation in (7.33), the second (resp. third) one from (7.33) and (7.14) (resp. (7.16) and the first two equalities). It remains to show that

Φ(θ) < Φ(τ), ∀ τ ∈ Υθ \{θ}. (7.35) nGn

And we split the proof into three cases.

Case 1: τ ∈ Υθ with |τ| < |θ|. In such case, we get |τ| ∈ (π, |θ|). Moreover, we have τ2 > 0. Otherwise τ2 < 0, and combining with the fact that u2 > 0 and the second ψ0(|τ|) 2 equation in (7.33), we have K3(τ1, τ2) = |τ| τ2 + 2ψ(|τ|) < 0. So (τ1, τ2) ∈ Ω−,4. Thus it follows from (vi) of Theorem 11 that (τ1, τ2) = (θ1, θ2), which is a contradiction. 1 Next, remark that s ψ(s) = s − cot s (< 0) is strictly increasing on (π, ϑ1), then we have 2 |τ| ψ(|τ|) < 2 |θ| ψ(|θ|). By (7.33), we can write

|τ| |τ| |θ| |θ| u2 − u1 = 2 |τ| ψ(|τ|) < 2 |θ| ψ(|θ|) = u2 − u1 . τ2 τ1 θ2 θ1

By the fact that u1, u2, τ1, τ2, θ1 > 0 and θ2 < 0, the last inequality implies that s s τ θ |τ | τ 2 θ 2 |θ | 0 < 1 < 1 and so 2 = 1 − 1 > 1 − 1 = 2 > 0. |τ| |θ| |τ| |τ| |θ| |θ|

Then we have " # " # |θ| 2 θ 2 |τ| 2 τ 2 Φ(θ) = − 1 2 < − 1 2 = Φ(τ), sin |θ| |θ| sin |τ| |τ|

s 2 since the function sin s (> 1) is strictly decreasing on (π, ϑ1) (cf. [27, (1.45)]), which ends the proof in this case.

53 Case 2: τ ∈ Υθ with |τ| ≥ |θ|, τ 6= θ and τ2 < 0. We argue as in the beginning of ψ0(|τ|) 2 Case 1, we have that K3(τ1, τ2) = |τ| τ2 + 2ψ(|τ|) < 0 and |τ| ∈/ (π, ϑ1). Moreover, 0 +∞ since ψ is always positive (see (1) of Lemma 9) and ψ is negative only on ∪k=1(kπ, ϑk), +∞ then we get that |τ| ∈ ∪k=2(kπ, ϑk). |τ2| |θ2| We begin with the case where |τ| ≥ |θ| . Then we yield that τ θ |τ| |θ| 0 < 1 ≤ 1 , so ≥ > 0. |τ| |θ| τ1 θ1 Combining this with the ﬁrst equality in (7.34), we get |θ| |τ| Φ(θ) = ϕ2(|θ|) u1 < ϕ2(|τ|) u1 = Φ(τ), θ1 τ1 where we have used, in the inequality, the fact that u1 > 0 and ϕ2 (> π) is strictly +∞ increasing on ∪k=1(k π, ϑk) from Lemma 9. |τ2| |θ2| θ2 τ2 We continue with the opposite case |τ| < |θ| , which is equivalent to |θ| < |τ| < 0. τ1 θ1 Similarly, we have |τ| > |θ| > 0. Hence, via the second equality in (7.34), θ θ τ τ 0 < Φ(θ) = ϕ (|θ|) u 1 + u 2 < ϕ (|τ|) u 1 + u 2 = Φ(τ), 1 1 |θ| 2 |θ| 1 1 |τ| 2 |τ| where we have used the fact that u1, u2 > 0 and ϕ1 (> 0) is strictly increasing on (0, +∞) from Lemma 9.

Case 3: τ ∈ Υθ with |τ| ≥ |θ| and τ2 > 0. By the third equality in (7.34), we get s s θ2 τ2 Φ(θ) = ϕ3(|θ|) u1 u1 + u2 < ϕ3(|τ|) u1 u1 + u2 = Φ(τ), θ1 τ1

θ2 τ2 where we have used, in the inequality, the fact that u1, u2 > 0, < 0 < , and ϕ3 (> 0) θ1 τ1 is strictly increasing on (π, +∞) from Lemma 9. This ﬁnishes the proof of Theorem 12.

7.5 Some consequences

In this sub-section, we provide some applications of Theorems 11 and 12. More precisely, we determine the exact formulas of d(g)2 on the whole space via a limiting argument, the cut locus Cuto as well as all shortest geodesics from o to any given g 6= o. For the sake of clarity, we will ﬁrst reformulate Theorem 12. 3 Recall that ϑ1 is the unique solution of tan s = s on (π, 2 π). Also for π < s < ϑ1, f(s) s 2 f(s) = 1 − s cot s, µ(s) = f 0(s), ψ(s) = , ϕ (s) = − 1, s2 0 sin s ϕ (s) ϕ (s) ϕ (s) = 0 , ϕ (s) = 0 , ϕ (s) = pϕ (s) ϕ (s). 1 µ(s) 2 s2 ψ0(s) 3 1 2

54 √ √2 nT4n Theorem 13. Let u1, u2 > 0 such that u2 < π u1. Suppose that

θe := θe(u1, u2) = (θ1, θ2) with θ2 < 0 < θ1 < |θe| (6= π) < ϑ1

is the unique solution of:

0 0 ! ψ (|θe|) 2 ψ (|θe|) 2 u1 = θ1 θ2 u2 = θ2 θ2 + 2 ψ(|θe|) . |θe| |θe|

Then we have θ1 > π and

2 !2 !2 ! 1 θ1 θ2 θ1 θ2 d e1, (u1, u2, 0) = + = ϕ1(|θe|) u1 + u2 + 1 4 |θe| sin |θe| |θe| |θe| s |θe| θ2 = ϕ2(|θe|) u1 + 1 = ϕ3(|θe|) u1 u1 + u2 + 1. (7.36) EDnD θ1 θ1

Combining this with Theorem 11, it only remains to ﬁnd the

β 2 7.5.1 Exact expression of d(e1, 4 e1) with β > 0 We have the following result, which is exactly [103, Theorem 1.4] up to a scaling property (cf. (2.2)) and an orthogonal invariance (see (7.2)) combining with (i) of Theorem 11.

1 c3 Corollary 15. Let t(β) = 4 (β, 0, 0) with β > 0. Then it holds that

2 d(e1, t(β)) = ϕ3(r) β + 1,

where r is the unique solution of the following equation in (π, ϑ1): s ψ(r) −2 ψ(r) r2 + 2 r = β. (7.37) DCUTP ψ0(r) √ √2 1 Proof. Let 0 < < π β and t(β, ) = 4 (β, , 0). Suppose that θe := (θ1(), θ2()) is the unique solution of

θ2() < 0 < π < θ1() < |θe| < ϑ1,

0 0 ! ψ (|θe|) 2 ψ (|θe|) 2 β = θ1() θ2(), = θ2() θ2() + 2 ψ(|θe|) . (7.38) ncin |θe| |θe|

+ By the compactness of BR2 (0, ϑ1), up to subsequences, we may take j −→ 0 as (0) (0) (0) j −→ +∞ such that the corresponding θej −→ θe0 := (θ1 , θ2 ). Obviously π ≤ θ1 ≤ ϑ1 (0) and θ2 ≤ 0.

55 (0) We claim that θ2 6= 0 so θe0 ∈/ {(π, 0), (ϑ1, 0)} and π < r := |θe0| < ϑ1 by (7.38). Indeed, this is ensured by the choice of Ω−,4 in [88, § 11]. More precisely, we argue by − contradiction: suppose that θ2(j) −→ 0 . Then the ﬁrst equation in (7.38) implies that

0 ψ (|θej |) + + lim = +∞, so |θe | −→ π and θ1(j) −→ π , j−→+∞ j |θej |

0 + since π < θ1(j) < |θej | < ϑ1 and ψ (s) −→ +∞ (π < s < ϑ1) only if s −→ π . Moreover, a direct calculation shows that (see also [88, Lemma 3.4]) 1 1 lim (s − π) ψ(s) = − , lim (s − π)2 ψ0(s) = . s−→π+ π s−→π+ π

Combining this with (7.38), we get that

!2 1 θ ( ) 2 θ ( ) 2 lim 2 j = β, 0 = − lim 2 j = √ pβ > 0. j−→+∞ π π j−→+∞ π |θej | − π |θej | − π

This leads to a contradiction. In conclusion, by the continuity of d2 and the last equality in (7.36), we obtain that 2 d(e1, t(β)) = ϕ3(r) β + 1, where π < r < ϑ1 satisﬁes

0 0 ψ (r) (0) (0) ψ (r) (0) β = θ (θ )2, (θ )2 + 2 ψ(r) = 0. (7.39) NICN r 1 2 r 2 That is s q ψ(r) β = −2 ψ(r) r2 − (θ(0))2 = −2 ψ(r) r2 + 2 r . 2 ψ0(r)

4 Note that the RHS of the last equality is exactly P (s) with the function P deﬁned in [103, (3.3)]. Then from [103, Lemma 3.5], we know that P is a strictly increasing dif- feomorphism between (π, ϑ1) and (0, +∞), which justiﬁes the uniqueness of the solution r in (π, ϑ1).

7.5.2 The cut locus on N3,2

2 We can characterize the cut locus of o in N3,2 from the exact formula for d as well. Recall that S denotes the set of points g such that d2 is C∞ in a neighborhood of g. We have the following result: nP6n Proposition 9. It holds that S ⊇ {(x, t); x and t are linearly independent} on N3,2. Proof. Using the scaling property (cf. (2.2)), the orthogonal invariance (see (7.2)) as well 2 1 as (i) of Theorem 11, it suﬃces to show that d is smooth at (e1, 4 (u1, u2, 0)) with u2 > 0 and u1 ≥ 0. By recalling the notations deﬁned by (7.3)-(7.9), we divide it into cases.

56 2 Case (1): (u1, u2) ∈ R<,+. Consider the smooth map

3 3 Π:O3 × (0, +∞) × Ω−,4 −→ R × R = N3,2 r2 (O, r, (θ , θ )) 7−→ (r O e , O (Λ(θ , θ ), 0)), 1 2 1 4 1 2 where O3 denotes the 3 × 3 orthogonal group. A direct computation shows that the dif- −1 ∞ ferential of Π at the point (I3, 1, Λ (u1, u2)) is invertible since Λ is a C -diﬀeomorphism 2 from Ω−,4 onto R<,+ by (vi) of Theorem 11. As a result, from the inverse function theorem, Theorem 13 and the fact that the function

!2 θ2 θ r2 1 + r2 2 θ2 + θ2 p 2 2 1 2 sin θ1 + θ2

1 is smooth on (0, +∞) × Ω−,4, we have (e1, 4 (u1, u2, 0)) ∈ S. 2 1 Case (2): (u1, u2) ∈ R>. Similarly, we have (e1, (u1, u2, 0)) ∈ S. √ 4 √2 Case (3): (u1, π u1) with u1 > 0. In such case, it can be proven by using [20, Theorem 26] with the function deﬁned in [88, § 11.2, Step 2], via Lemma 4 and the concrete ∗ characterization of Abno obtained in (v) of Theorem 11. However, we will provide here a direct proof, which is of independent interest. Set 2 2 Ω+,1 := Ω+ ∩ {(v1, v2); v1 > 0}, R>,+ := R> ∩ {(u1, u2); u1 > 0}. 2 Using the polar coordinate in R \{(v1, 0); v1 ≤ 0},(r, η) where v1 = r cos η and v2 = sin η r sin η with −π < η < π, we introduce another map Θ : (r, η) 7→ (r, ρ := sin r ) with suitable domain. π π Notice that we have r > 0 and η ∈ 0, 2 (resp. − 2 , 0 ) on Ω+,1 (resp. Ω−,4). Moreover, it is not hard to show that Θ is injective on Ω+,1 (resp. Ω−,4) and its Jacobian cos η ∞ determinant is sin r . It follows from the global inverse function Theorem that Θ is a C - diﬀeomorphism from Ω+,1 (resp. Ω−,4) onto Ωe+ := Θ(Ω+,1) (resp. Ωe− := Θ(Ω−,4)). From (7.3), (7.8), (7.7) and (7.13), a direct calculation yields that

Ωe+ = {(r, ρ); 0 < ρ sin r < 1, 0 < r < π}, 2 2 Ωe− = {(r, ρ); π < r < ϑ1, ρ > 0, ρ h(r) + 2 r ψ(r) < 0}.

−1 ∞ By (ii) and (vi) of Theorem 11, Ξ := Λ ◦ Θ :(r, ρ) 7→ (u1, u2) is a C -diﬀeomorphism 2 2 from Ωe+ (resp. Ωe−) onto R>,+ (resp. R<,+). Using (7.9), (7.13) and the deﬁnition of ψ (cf. (2.25)), it can be written explicitly:

 2 sin r p 2 2 2  u1 = r + sin r cos r − 2 r 1 − ρ sin r ρ . (7.40) Xi sin2 r 3 sin r  u2 = sin r r + sin r cos r − 2 r ρ + 2 r − cos r ρ

57 ρ

4 {π} × (0, +∞) −→

q 2 ρ = 1 ρ = − 2r ψ(r) sin r &. h(r) 2

Ωe+ Ωe− (ϑ , 0) . 1 π π 3π 5π 3π r π 4 2 4 4 2

Figure 1: Plot of Ωe = Ωe+ ∪ Ωe− ∪ ({π} × (0, +∞)) fig1

A key observation is that Ξ is also meaningful on (π, ρ) for ρ > 0 and Ξ(π, ρ) = n √ o 2 √2 (π ρ , 2 ρ) is a bijection from {π} × (0, +∞) to (u1, u2): u2 = π u1 > 0 . Set

Ωe := Ωe+ ∪ Ωe− ∪ ({π} × (0, +∞)). See the plot in Figure 1. Moreover, notice that Ξ is C∞ on Ω.e A direct computation shows that the Jacobian determinant of Ξ at (π, ρ)(ρ > 0) equals J(Ξ)(π, ρ) = 2 π2ρ4 + 4ρ2 > 0.

As a consequence, Ξ is a C∞-diﬀeomorphism from Ωe onto (0, +∞) × (0, +∞). In conclusion, by (iii), (iv) of Theorem 11 as well as Theorem 12, we have 1 d(e , (u , u , 0))2 = (r2 − sin2 r) ρ2 + 1, ∀ u , u > 0, (7.41) 1 4 1 2 1 2 −1 where (r, ρ) = Ξ (u1, u2). Finally, by using the smooth map 3 3 Π:Oe 3 × (0, +∞) × Ωe −→ R × R = N3,2 R2 (O,R, (r, ρ)) 7−→ (R O e , O (Ξ(r, ρ), 0)), 1 4 √ 1 √2 we can get that (e1, 4 (u1, π u1, 0) ∈ S. This completes the proof of this proposition.

58 Indeed, the relation “⊇” in Proposition 9 can be improved to “=”. In other words, we have the following:

c4 Corollary 16. On N3,2, we have Cuto = {(x, t); x and t are linearly dependent}. Proof. It follows from Proposition 9 that

Cuto ⊆ {(x, t); x and t are linearly dependent}. (7.42) cutA

Moreover, we have that (see (v) of Theorem 11):

∗ 3 Abno = {(x, 0); x ∈ R } = Me 2. (7.43) abnN By Theorem 1, it remains to show that the classical cut locus of o is

CL Cuto = {(x, t); t 6= 0 and x = λt for some λ ∈ R} := E, (7.44) n32CC which has been already proven in [113] and [103] by completely diﬀerent technique. In- deed, once we get (7.42) and (7.43), via [123, Lemma 9], (7.44) is a direct consequence of the simple fact that there exist two distinct shortest geodesics from o to any g ∈ E. See Subsection 7.5.3 below for more details.

7.5.3 Description of shortest geodesic(s) from o to any given g 6= o SS76 Let us begin by the following observation: n32nLn Lemma 11. Let SO3 denote the 3 × 3 special orthogonal group, and (x, t) = exp(w, τ). Then we have

exp(O w, O τ) = (O x, O t), ∀ O ∈ SO3, (7.45) symN1

Ue(τ) exp(O e w, O τ) = (O x, O t), ∀ O ∈ O3 \ SO3. (7.46) symN4

Proof. Using the well-known basic property of the cross product:

O (τ × η) = (O τ) × (O η), ∀ O ∈ SO3, (7.47) orthocro

(7.45) can be checked directly by (7.21) and (7.27), or explained by [101, § 2.1]. To obtain (7.46), we use the fact that Ue(τ) η = τ × η as well as (7.47), and get that

T T Ue(O τ) η = (O τ) × η = O (τ × (O η)) = O Ue(τ) O η, ∀ O ∈ SO3, (7.48) orthoU which implies that

T Ue(O τ) = O Ue(τ) O , ∀ O ∈ SO3. (7.49) orthoU2

Then for any O ∈ O3 \ SO3, we have −O ∈ SO3, and (7.45) implies that exp(−O w, −O τ) = (−O x, −O t).

Consequently, applying (2.10) to the last equation and using (7.49), we obtain (7.46).

59 Combining (7.45) and (7.46) with (2.9), it suﬃces to determine all shortest geodesic(s) 1 from o to g, where: (1) g = (e1, 4 (u1, u2, 0)) with u1 ≥ 0 and u2 > 0; (2) g = (e1, 0); (3) 1 g = (0, e1) or g = (e1, 4 β e1) with β > 0. 1 Case 1. g = (e1, 4 (u1, u2, 0)) with u1 ≥ 0 and u2 > 0. In such case, we have g ∈ S. So there exists a unique shortest geodesic from o to g, which is strictly normal. If (u1, u2) ∈ 2 2 R> ∪R<,+, it follows from [88, Theorem 2.4 and Theorem 2.5] that the shortest geodesic is U(θ) −Ue(θ) −1 2 √ given by γ(w,2 θ) with w = e e1 and θ = (Λ (u1, u2), 0). For u2 = √ u1 > 0, sin U(θ) √ π then it is given by γ(w,2 θ) with w = (1, π u1, 0) and θ = (π, 0, 0), see Remark 13 for more details. ∗ Case 2. g = (e1, 0) ∈ Abno \{o}. The unique shortest geodesic joining o to g is a straight segment and it is abnormal. 1 Case 3. g = (0, e1) or g = (e1, 4 β e1) with β > 0. In such case, g ∈ E. By (7.45), a trivial observation is that there exist at least two distinct shortest geodesics from o to g. Indeed, a complete description can be found in [103, § 3]. However, we will provide a completely diﬀerent method to get it, which can be considered as a direct consequence of our main results, namely Theorems 11 and 13. More precisely, we will use Remark 5 to determine the parameter (w, θ) of any shortest geodesic from o to g, γ(w,2 θ). 1 Case 3 (a). Let us begin with the case g(β) = (e1, 4 (β, 0, 0)) where β > 0. By Remark (n) (n) +∞ c 5, there exist {gn := (x , t )}n=1 ⊆ Cuto, with the corresponding unique shortest (n) (n) geodesic γ(w(n),2 θ(n)), such that (gn, w , θ ) → (g(β), w, θ) as n → +∞. Without loss of generality, we may assume that |x(n)|= 6 0 and x(n) · t(n) > 0 for all n ≥ 1. For each n ≥ 1, (n) we pick an orthogonal matrix O ∈ O3 such that

(n) 2 |x | (n) (n) (n) (n) O(n) x(n) = |x(n)| e ,O(n) t(n) = (u , u , 0) with u , u > 0. (7.50) icNN 1 4 1 2 1 2

(n) (n) 1 Combining this with the fact that (x , t ) → (e1, 4 β e1) as n → +∞, we get that (n) (n) (u1 , u2 ) → (β, 0) as n → +∞. By arguing as in the proof of (7.39) and using (7.45), (7.46) as well as (2.9), we get that s s ψ(r) ψ(r) O(n) θ(n) → (Θ , Θ , 0) with Θ = r2 + 2 r , Θ = − −2 r , 1 2 1 ψ0(r) 2 ψ0(r) where r is the unique solution of (7.37) in (π, ϑ1). Since O3 is compact, up to subsequences, we may further assume that O(n) → O0 as n → +∞. Moreover, it follows 0 0 from the ﬁrst equation in (7.50) that the orthogonal matrix O satisﬁes O e1 = e1. Since 0 O θ = (Θ1, Θ2, 0), we yield that

θ = (Θ1, |Θ2| cos σ, |Θ2| sin σ) for some σ ∈ R, (n) (n) U(θ ) −Ue(θ(n)) (n) U(θ) −Ue(θ) w = lim w = lim e x = e e1, n→+∞ n→+∞ sin U(θ(n)) sin U(θ) where we have used (2.7) in the second “=” of the last formula.

60 In other words, we have proven that every shortest geodesic from o to g(β) can be expressed as γ(w,2 θ), where the parameter (w, θ) have the form U(θ(σ)) θ(σ) := (Θ , |Θ | cos σ, |Θ | sin σ), w(σ) := e−Ue(θ(σ)) e . (7.51) thetaw 1 2 2 sin U(θ(σ)) 1

Furthermore, we will prove that the converse is also valid, namely every such parameter provides a shortest geodesic steering o to g(β).

Let us ﬁx a such shortest geodesic γ(w(σ0),2 θ(σ0)), with σ0 ∈ R. By (7.45), for any

α ∈ R, γ(O(α) w(σ0),2 O(α) θ(σ0)) is also a shortest geodesic from o to g(β), where

 1 0 0  O(α) =  0 cos α − sin α  ∈ SO3. 0 sin α cos α

It remains to show that

(O(α) w(σ0),O(α) θ(σ0)) = (w(σ0 + α), θ(σ0 + α)), ∀ α ∈ R, (7.52) nCvn since (w(σ0 + α), θ(σ0 + α)) runs over all possible (w(σ), θ(σ)) as the parameter α runs over R. In fact, it follows from (7.49) that U(θ(σ )) 0 −Ue(θ(σ0)) T O(α) w(σ0) = O(α) e O(α) O(α) e1 sin U(θ(σ0)) U(O(α) θ(σ )) 0 −Ue(O(α) θ(σ0)) = e e1. (7.53) invarw sin U(O(α) θ(σ0))

To ﬁnish the proof of (7.52), it suﬃces to notice that we have obviously

O(α) θ(σ0) = θ(σ0 + α).

Via some elementary but tedious calculations, our result can be identiﬁed with that √ √ ⊥ β β 2 3 of [103, Theorem 3.2] with (y, y , θ) = 2 e2, 2 e3, r therein, where we identify ∧ R with R3 via the map T deﬁned by 2 3 3 T: ∧ R −→ R a ∧ b 7−→ a × b.

Case 3 (b). In the opposite case where g = (0, e1), similarly, it follows from Remark 5 and (7.28) that all shortest geodesics joining o to g are given by γ(w,2 θ), where √ θ = (π, 0, 0), w = 2 π (0, − cos σ, − sin σ) with σ ∈ R. Via some elementary but tedious calculations, our result can be identiﬁed with that ⊥ of [103, Theorem 3.2] with (y, y , θ) = (e2, e3, π) therein (via the map T as well).

61 8 Appendix A: Proof of Lemma 7 Axa The proof essentially follows that of [88, Proposition 10.3] and we include it for the sake of completeness. 2 ∗ ∗ 2 2 Set v := (v1, v2) ∈ R . Then from the deﬁnition of Υ((xe , xe1, xe∗); ·), for |xe | + xe1 6= 0 and xe∗ 6= 0, we have ∗ (u1, u2) := Υ((xe , xe1, xe∗); (v1, v2)) v2 |v| |v| = ∇ 1 ψ + v2 ψ(|v|) |x |2 + f (|x∗|2 + x2) . (8.1) graexpUps v 4 2 2 e∗ 2 e e1

More precisely,

 h ∗ 2 2 2 2 2 2 i |v| |xe | +xe1 0 |v| v1 |xe∗| |v| |xe∗| 0 2 |xe∗|  u1 = v1 µ + ψ + ψ + ψ (|v|) v  2 2|v| 2 4 2|v| 2 2 2 |v| . (8.2) expUps h |x∗|2+x2 v2 2 2 i  u = v µ |v| e e1 + ψ0 |v| 1 |xe∗| + ψ0(|v|) v2 |xe∗| + 2 ψ(|v|) |x |2  2 2 2 2|v| 2 4 2|v| 2 |v| e∗

It follows from [88, Lemma 3.3] that

ψ0(|v|) π ψ0(|v|) 2 0 ≤ |v | v2 ≤ v2 2 ψ(|v|) + v2 , ∀ |v| < π, |v| 1 2 4 2 |v| 2

which implies that |v| |x∗|2 + x2 |v| |v|2 |v| |v| |u | ≤ µ e e1 + ψ0 + ψ |x |2 1 2 2 2 8 2 2 e∗ π ψ0(|v|) 2 + v2 2 ψ(|v|) + v2 |x |2. 4 2 |v| 2 e∗

Using the identity s2 ψ0(s) + 2s ψ(s) = µ(s), the second equality in (8.2) implies that

|v| |x∗|2 + x2 + |x |2 π u2 π u2 |u | ≤ µ e e1 e∗ + 2 < 2 + |x∗|2 + x2 + |x |2 , 1 2 2 e e1 e∗ 2 2 4 |xe∗| 4 |xe∗| where we have used in “<” the fact that µ(π/2) = π/2 and µ is strictly increasing on ∗ 2 ∗ (−π, π) (see Lemma 8), so Υ((xe , xe1, xe∗); ·) is from BR2 (0, π) to Rr(xe , xe1, xe∗). ∗ Next, it follows from (8.1) that the Jacobian of Υ((xe , xe1, xe∗); ·) is given by v2 |v| |v| Hess 1 ψ + v2 ψ(|v|) |x |2 + f (|x∗|2 + x2) > 0, v 4 2 2 e∗ 2 e e1

∗ 2 2 2 since |xe | + xe1 6= 0, xe∗ 6= 0, Hessv(f(|v|)) > 0 and Hessv(v1 ψ(|v|)) ≥ 0 (so symmetrically 2 Hessv(v2 ψ(|v|)) ≥ 0) for |v| < π, which can be found in the proof of [88, Proposition 10.1]. ∗ Finally, using Hadamard’s theorem, it remains to show that Υ((xe , xe1, xe∗); ·) is proper. (j) +∞ (j) (j) (j) To this end, we consider the behaviour of {v }j=1 ⊆ BR2 (0, π) satisfying v = (v1 , v2 ) −→

62 ∂BR2 (0, π) and we split it into cases. (j) − (j) ∗ (j) (1) If |v | −→ π and |v2 | ≥ ε > 0, then Υ((xe , xe1, xe∗); v ) −→ ∞. (j) − (j) + ∗ (j) (j) (j) (2) If |v | −→ π and |v2 | −→ 0 , set Υ((xe , xe1, xe∗); v ) = (u1 , u2 ). (j) ∗ (j) In the case |u1 | −→ +∞, it is easy to see Υ((xe , xe1, xe∗); v ) −→ ∞. (j) In the opposite case, assume |u1 | −→ a (≥ 0). Then from the ﬁrst equation of (8.2), using the fact that s2 ψ0(s) + 2s ψ(s) = µ(s) as well as µ(π/2) = π/2, we have that

(j) !2 π ∗ 2 2 2 1 v2 2 a = |x | + x1 + |x∗| + lim |x∗| , 4 e e e j−→+∞ π π − |v(j)| e

where we have used the fact that (see also [88, Lemma 3.4]) 1 lim (π − s)2 ψ0(s) = . s−→π− π Since 1 lim (π − s) ψ(s) = , s−→π− π the second equation of (8.2) gives that 2 |v(j)| 2 |x |r π (j) 2 2 e∗ ∗ 2 2 2 lim |u2 | = lim |x∗| = √ a − (|x | + x1 + |x∗| ). j−→+∞ j−→+∞ π π − |v(j)| e π 4 e e e

∗ (j) 2 ∗ (j) In conclusion, we have Υ((xe , xe1, xe∗); v ) −→ ∂Rr(xe , xe1, xe∗) when v −→ ∂BR2 (0, π) in BR2 (0, π), which means it is proper. This ﬁnishes the proof of this lemma.

9 Appendix B: Properties of the direct product of two 2-step groups Axb Assume that Gj = G(qj, mj, Uj)(j = 1, 2), where

(1) (mj ) Uj = {Uj ,..., Uj }.

Consider the direct product G1 × G2 := G(q1 + q2, m1 + m2, U) with U deﬁned by

(1) (m1) U1 U1 Oq1×q1 Oq1×q1 ,..., , (1) ,..., (m2) , Oq2×q2 Oq2×q2 U2 U2

where Ok1×k2 denotes the k1 ×k2 null matrix. Let gj := (xj, tj) ∈ Gj, j = 1, 2. We identify (g1, g2) with ((x1, x2), (t1, t2)). Then, it is clear that

(G1×G2) (G1) (G2) Ω∗ = Ω∗ × Ω∗ ,

(G1×G2) (G1) (G2) φ ((g1, g2); (θ(1), θ(2))) = φ (g1; θ(1)) + φ (g2; θ(2)),

(G1×G2) (G1) (G2) 2 2 2 M = M × M , dG1×G2 ((g1, g2)) = dG1 (g1) + dG2 (g2) ,

63 and the meaning of the notations herein is obvious. It is clear that (γG1 (s), γG2 (s)) is

a normal geodesic if and only if both γG1 and γG2 are normal geodesics. Moreover, the

normal geodesic (γG1 (s), γG2 (s)) is abnormal if and only if γG1 or γG2 is abnormal. Also,

it is shortest if and only if both γG1 and γG2 are shortest. Hence, we have the following well-known properties: on G1 × G2, it holds that: (G1×G2) (G1) (G2) Cuto = Cuto × G2 ∪ G1 × Cuto , (9.1) CL (G1×G2) CL (G1) CL (G2) Cuto = Cuto × G2 ∪ G1 × Cuto , (9.2) ∗ (G1×G2) ∗ (G1) ∗ (G2) Abno = Abno × G2 ∪ G1 × Abno , (9.3)

and the meaning of the notations herein is obvious. By the fact that M(G1×G2) = M(G1) × M(G2), we have the following:

Proposition 10. G1 × G2 is of type GM if and only if both G1 and G2 are GM-groups.

Similarly, on the direct product of the Euclidean space Rk with a step-two group G, we have

k k (R ×G) k (G) CL (R ×G) k CL (G) Cuto = R × Cuto , Cuto = R × Cuto , (9.4) k ∗ (R ×G) k ∗ (G) Abno = R × Abno , (9.5) and the meaning of the notations herein is obvious. In particular,

NSAa Proposition 11. Let G be a step-two group. Consider the direct product of the Euclidean space Rk with G. Then:

(1) Rk × G is of type GM if and only if G is a GM-group;

(2) Rk × G is of type SA if and only if G is a M´etivieror SA-group.

10 Appendix C: Construction of SA-groups Axc In [88, § 8.1], there is a simple method to construct an uncountable number of GM-groups (resp. GM-groups of M´etiviertype) from any given step-two group (resp. M´etiviergroup). We will use the same method to produce SA-groups. More precisely, assume that m ≥ 2 and Gj = G(qj, m, Uj)(j = 1, 2), where

(1) (m) Uj = {Uj ,..., Uj }.

We consider G := G(q1 + q2, m, U) with U deﬁned by

( (1) ! (m) !) U1 U1 (1) ,..., (m) . U2 U2

64 q1 q2 m Let g = (x1, x2, t) ∈ R × R × R . Obviously, we have:

(G) (G1) (G2) Ω∗ = Ω∗ ∩ Ω∗ ,

(G) (G1) (G1) (G2) (G2) φ (g; τ) = hU (τ) cot U (τ) x1, x1i + hU (τ) cot U (τ) x2, x2i + 4 t · τ,

(G) and the meaning of the notations herein is clear. Moreover, g ∈ Me 2 if and only if there (G1) (G2) 0 00 m are θ ∈ Ω∗ ∩ Ω∗ and t , t ∈ R such that: 1 t = t0 + t00, t0 = − ∇ hU (G1)(θ) cot U (G1)(θ) x , x i, 4 θ 1 1 1 t00 = − ∇ hU (G2)(θ) cot U (G2)(θ) x , x i, 4 θ 2 2 and the sum of two positive semideﬁnite matrices

(G1) (G1) (G2) (G2) −HessθhU (θ) cot U (θ) x1, x1i + −HessθhU (θ) cot U (θ) x2, x2i

0 (G1) 00 (G2) is singular. In such case, we have (x1, t ) ∈ Me 2 and (x2, t ) ∈ Me 2 . Thus, we get immediately the following:

Proposition 12. With the above notations, G is a SA-group when G1 is of type SA and G2 is of type M´etivieror SA.

Acknowledgement

This work is partially supported by NSF of China (Grants No. 11625102 and No. 11831004) and “The Program of Shanghai Academic Research Leader” (18XD1400700). The authors would like to thank L. Rizzi for many useful suggestions.

References

ABB12 [1] A. Agrachev, D. Barilari, and U. Boscain. On the Hausdorﬀ volume in sub- Riemannian geometry. Calc. Var. Partial Diﬀerential Equations, 43(3-4):355–388, 2012.

ABB20 [2] A. Agrachev, D. Barilari, and U. Boscain. A comprehensive introduction to sub- Riemannian geometry, volume 181 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 2020. From the Hamiltonian viewpoint, With an appendix by Igor Zelenko.

AS04 [3] A. A. Agrachev and Y. L. Sachkov. Control theory from the geometric viewpoint, volume 87 of Encyclopaedia of Mathematical Sciences. Springer-Verlag, Berlin, 2004. Control Theory and Optimization, II.

65 A92 [4] G. Alexopoulos. An application of homogenization theory to harmonic analysis: Harnack inequalities and Riesz transforms on Lie groups of polynomial growth. Canad. J. Math., 44(4):691–727, 1992.

A94 [5] G. Alexopoulos. Spectral multipliers on Lie groups of polynomial growth. Proc. Amer. Math. Soc., 120(3):973–979, 1994.

A02 [6] G. K. Alexopoulos. Sub-Laplacians with drift on Lie groups of polynomial volume growth. Mem. Amer. Math. Soc., 155(739):x+101, 2002.

AR04 [7] L. Ambrosio and S. Rigot. Optimal mass transportation in the Heisenberg group. J. Funct. Anal., 208(2):261–301, 2004.

BR18 [8] Z. Badreddine and L. Riﬀord. Measure contraction properties for two-step analytic sub-Riemannian structures and Lipschitz Carnot groups. To appear in A.I.F. arXiv e-prints, page arXiv:1712.09900, Dec. 2017.

BFKG16 [9] H. Bahouri, C. Fermanian-Kammerer, and I. Gallagher. Dispersive estimates for the Schr¨odingeroperator on step-2 stratiﬁed Lie groups. Anal. PDE, 9(3):545–574, 2016.

BGX00 [10] H. Bahouri, P. Gérard,and C.-J. Xu. Espaces de Besov et estimations de Strichartz généraliséessur le groupe de Heisenberg. J. Anal. Math., 82:93–118, 2000.

BBBC08 [11] D. Bakry, F. Baudoin, M. Bonnefont, and D. Chafa¨ı. On gradient bounds for the heat kernel on the Heisenberg group. J. Funct. Anal., 255(8):1905–1938, 2008.

BGL14 [12] D. Bakry, I. Gentil, and M. Ledoux. Analysis and geometry of Markov diﬀusion operators, volume 348 of Grundlehren der Mathematischen Wissenschaften [Funda- mental Principles of Mathematical Sciences]. Springer, Cham, 2014.

BCC19 [13] A. Baldi, G. Citti, and G. Cupini. Schauder estimates at the boundary for sub- laplacians in Carnot groups. Calc. Var. Partial Diﬀerential Equations, 58(6):Paper No. 204, 43, 2019.

BFMT12 [14] Z. M. Balogh, K. F¨assler,P. Mattila, and J. T. Tyson. Projection and slicing theorems in Heisenberg groups. Adv. Math., 231(2):569–604, 2012.

BKS18 [15] Z. M. Balogh, A. Krist´aly, and K. Sipos. Geometric inequalities on Heisenberg groups. Calc. Var. Partial Diﬀerential Equations, 57(2):Paper No. 61, 41, 2018.

BKS19 [16] Z. M. Balogh, A. Krist´aly, and K. Sipos. Jacobian determinant inequality on corank 1 Carnot groups with applications. J. Funct. Anal., 277(12):108293, 36, 2019.

BTW09 [17] Z. M. Balogh, J. T. Tyson, and B. Warhurst. Sub-Riemannian vs. Euclidean dimension comparison and fractal geometry on Carnot groups. Adv. Math., 220(2):560– 619, 2009.

BGM18 [18] S. Banerjee, M. Gordina, and P. Mariano. Coupling in the Heisenberg group and its applications to gradient estimates. Ann. Probab., 46(6):3275–3312, 2018.

66 BBG12 [19] D. Barilari, U. Boscain, and J.-P. Gauthier. On 2-step, corank 2, nilpotent sub- Riemannian metrics. SIAM J. Control Optim., 50(1):559–582, 2012.

BR19 [20] D. Barilari and L. Rizzi. Sub-Riemannian interpolation inequalities. Invent. Math., 215(3):977–1038, 2019.

BB16 [21] F. Baudoin and M. Bonnefont. Reverse Poincar´einequalities, isoperimetry, and Riesz transforms in Carnot groups. Nonlinear Anal., 131:48–59, 2016.

BG17 [22] F. Baudoin and N. Garofalo. Curvature-dimension inequalities and Ricci lower bounds for sub-Riemannian manifolds with transverse symmetries. J. Eur. Math. Soc. (JEMS), 19(1):151–219, 2017.

BGM19 [23] F. Baudoin, M. Gordina, and P. Mariano. On the Cheng-Yau gradient estimate for Carnot groups and sub-Riemannian manifolds. Proc. Amer. Math. Soc., 147(7):3181–3189, 2019.

BHT08 [24] F. Baudoin, M. Hairer, and J. Teichmann. Ornstein-Uhlenbeck processes on Lie groups. J. Funct. Anal., 255(4):877–890, 2008.

BK14 [25] F. Baudoin and B. Kim. Sobolev, Poincar´e,and isoperimetric inequalities for subelliptic diﬀusion operators satisfying a generalized curvature dimension inequality. Rev. Mat. Iberoam., 30(1):109–131, 2014.

BGG96 [26] R. Beals, B. Gaveau, and P. Greiner. The Green function of model step two hypoelliptic operators and the analysis of certain tangential Cauchy Riemann complexes. Adv. Math., 121(2):288–345, 1996.

BGG00 [27] R. Beals, B. Gaveau, and P. C. Greiner. Hamilton-Jacobi theory and the heat kernel on Heisenberg groups. J. Math. Pures Appl. (9), 79(7):633–689, 2000.

BR96 [28] A. Bella¨ıche and J.-J. Risler, editors. Sub-Riemannian geometry, volume 144 of Progress in Mathematics. Birkh¨auserVerlag, Basel, 1996.

B88 [29] G. Ben Arous. D´eveloppement asymptotique du noyau de la chaleur hypoelliptique hors du cut-locus. Ann. Sci. Ecole´ Norm. Sup. (4), 21(3):307–331, 1988.

B84 [30] J.-M. Bismut. Large deviations and the Malliavin calculus, volume 45 of Progress in Mathematics. Birkh¨auserBoston, Inc., Boston, MA, 1984.

BCR98 [31] J. Bochnak, M. Coste, and M.-F. Roy. Real algebraic geometry, volume 36 of Ergeb- nisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Re- lated Areas (3)]. Springer-Verlag, Berlin, 1998. Translated from the 1987 French original, Revised by the authors.

BLU07 [32] A. Bonﬁglioli, E. Lanconelli, and F. Uguzzoni. Stratiﬁed Lie groups and potential theory for their sub-Laplacians. Springer Monographs in Mathematics. Springer, Berlin, 2007.

67 BCH20 [33] M. Bonnefont, D. Chafa¨ı,and R. Herry. On logarithmic Sobolev inequalities for the heat kernel on the Heisenberg group. Annales de la Facultédes sciences de Toulouse : Mathématiques, 29(2):335–355, 2020. identifier: 10.5802/afst.1633.

B82 [34] R. W. Brockett. Control theory and singular Riemannian geometry. In New direc- tions in applied mathematics (Cleveland, Ohio, 1980), pages 11–27. Springer, New York-Berlin, 1982.

CL08 [35] P. Cannarsa and L. Rifford. Semiconcavity results for optimal control problems ad- mitting no singular minimizing controls. Ann. Inst. H. PoincaréAnal. Non Linéaire, 25(4):773–802, 2008.

CWW95 [36] A. Carbery, S. Wainger, and J. Wright. Hilbert transforms and maximal functions associated to ﬂat curves on the Heisenberg group. J. Amer. Math. Soc., 8(1):141– 179, 1995.

CC15 [37] V. Casarino and P. Ciatti. A restriction theorem for M´etiviergroups. Adv. Math., 245:52–77, 2013.

CFZ20 [38] M. Chatzakou, S. Federico, and B. Zegarli´nski. q-Poincar´einequalities on Carnot Groups. arXiv e-prints, page arXiv:2007.04689, July 2020.

C85 [39] M. Christ. Hilbert transforms along curves. I. Nilpotent groups. Ann. of Math. (2), 122(3):575–596, 1985.

C91 [40] M. Christ. Lp bounds for spectral multipliers on nilpotent groups. Trans. Amer. Math. Soc., 328(1):73–81, 1991.

CG84 [41] M. Christ and D. Geller. Singular integral characterizations of Hardy spaces on homogeneous groups. Duke Math. J., 51(3):547–598, 1984.

C93 [42] T. Coulhon. It´erationde Moser et estimation gaussienne du noyau de la chaleur. J. Operator Theory, 29(1):157–165, 1993.

CMZ96 [43] T. Coulhon, D. M¨uller,and J. Zienkiewicz. About Riesz transforms on the Heisen- berg groups. Math. Ann., 305(2):369–379, 1996.

CS08 [44] T. Coulhon and A. Sikora. Gaussian heat kernel upper bounds via the Phragm´en- Lindel¨oftheorem. Proc. Lond. Math. Soc. (3), 96(2):507–544, 2008.

C79 [45] J. Cygan. Heat kernels for class 2 nilpotent groups. Studia Math., 64(3):227–238, 1979.

DP89 [46] E. B. Davies and M. M. H. Pang. Sharp heat kernel bounds for some Laplace operators. Quart. J. Math. Oxford Ser. (2), 40(159):281–290, 1989.

DH05 [47] M. Del Hierro. Dispersive and Strichartz estimates on H-type groups. Studia Math., 169(1):1–20, 2005.

68 DLLW19 [48] X. T. Duong, H.-Q. Li, J. Li, and B. D. Wick. Lower bound of Riesz transform kernels and commutator theorems on stratiﬁed nilpotent Lie groups. J. Math. Pures Appl. (9), 124:273–299, 2019.

E03 [49] P. Eberlein. The moduli space of 2-step nilpotent Lie algebras of type (p, q). In Explorations in complex and Riemannian geometry, volume 332 of Contemp. Math., pages 37–72. Amer. Math. Soc., Providence, RI, 2003.

E09 [50] N. Eldredge. Precise estimates for the subelliptic heat kernel on H-type groups. J. Math. Pures Appl. (9), 92(1):52–85, 2009.

E10 [51] N. Eldredge. Gradient estimates for the subelliptic heat kernel on H-type groups. J. Funct. Anal., 258(2):504–533, 2010.

FF15 [52] F. Ferrari and B. Franchi. Harnack inequality for fractional sub-Laplacians in Carnot groups. Math. Z., 279(1-2):435–458, 2015.

FJ08 [53] A. Figalli and N. Juillet. Absolute continuity of Wasserstein geodesics in the Heisen- berg group. J. Funct. Anal., 255(1):133–141, 2008.

FR10 [54] A. Figalli and L. Riﬀord. Mass transportation on sub-Riemannian manifolds. Geom. Funct. Anal., 20(1):124–159, 2010.

F73 [55] G. B. Folland. A fundamental solution for a subelliptic operator. Bull. Amer. Math. Soc., 79:373–376, 1973.

F75 [56] G. B. Folland. Subelliptic estimates and function spaces on nilpotent Lie groups. Ark. Mat., 13(2):161–207, 1975. ¯ FS74 [57] G. B. Folland and E. M. Stein. Estimates for the ∂b complex and analysis on the Heisenberg group. Comm. Pure Appl. Math., 27:429–522, 1974.

FS82 [58] G. B. Folland and E. M. Stein. Hardy spaces on homogeneous groups, volume 28 of Mathematical Notes. Princeton University Press, Princeton, N.J.; University of Tokyo Press, Tokyo, 1982.

FLW95 [59] B. Franchi, G. Lu, and R. L. Wheeden. Representation formulas and weighted Poincaréinequalities for Hörmandervector fields. Ann. Inst. Fourier (Grenoble), 45(2):577–604, 1995.

FSS01 [60] B. Franchi, R. Serapioni, and F. Serra Cassano. Rectiﬁability and perimeter in the Heisenberg group. Math. Ann., 321(3):479–531, 2001.

FSS03 [61] B. Franchi, R. Serapioni, and F. Serra Cassano. On the structure of ﬁnite perimeter sets in step 2 Carnot groups. J. Geom. Anal., 13(3):421–466, 2003.

FL12 [62] R. L. Frank and E. H. Lieb. Sharp constants in several inequalities on the Heisenberg group. Ann. of Math. (2), 176(1):349–381, 2012.

69 G77 [63] B. Gaveau. Principe de moindre action, propagation de la chaleur et estim´eessous elliptiques sur certains groupes nilpotents. Acta Math., 139(1-2):95–153, 1977.

GR07 [64] I. S. Gradshteyn and I. M. Ryzhik. Table of integrals, series, and products. El- sevier/Academic Press, Amsterdam, seventh edition, 2007. Translated from the Russian, Translation edited and with a preface by Alan Jeﬀrey and Daniel Zwill- inger, With one CD-ROM (Windows, Macintosh and UNIX).

HK00 [65] P. Haj lasz and P. Koskela. Sobolev met Poincar´e. Mem. Amer. Math. Soc., 145(688):x+101, 2000.

HZ10 [66] W. Hebisch and B. Zegarli´nski.Coercive inequalities on metric measure spaces. J. Funct. Anal., 258(3):814–851, 2010.

H67 [67] L. H¨ormander. Hypoelliptic second order diﬀerential equations. Acta Math., 119:147–171, 1967.

HL10 [68] J.-Q. Hu and H.-Q. Li. Gradient estimates for the heat semigroup on H-type groups. Potential Anal., 33(4):355–386, 2010.

HM89 [69] H. Hueber and D. M¨uller.Asymptotics for some Green kernels on the Heisenberg group and the Martin boundary. Math. Ann., 283(1):97–119, 1989.

H76 [70] A. Hulanicki. The distribution of energy in the Brownian motion in the Gaussian ﬁeld and analytic-hypoellipticity of certain subelliptic operators on the Heisenberg group. Studia Math., 56(2):165–173, 1976.

J86 [71] D. Jerison. The Poincaréinequality for vector fields satisfying Hörmander’scondi- tion. Duke Math. J., 53(2):503–523, 1986.

JL88 [72] D. Jerison and J. M. Lee. Extremals for the Sobolev inequality on the Heisenberg group and the CR Yamabe problem. J. Amer. Math. Soc., 1(1):1–13, 1988.

J82 [73] F. John. Partial diﬀerential equations, volume 1 of Applied Mathematical Sciences. Springer-Verlag, New York, fourth edition, 1982.

J09 [74] N. Juillet. Geometric inequalities and generalized Ricci bounds in the Heisenberg group. Int. Math. Res. Not. IMRN, (13):2347–2373, 2009.

J14 [75] N. Juillet. Diﬀusion by optimal transport in Heisenberg groups. Calc. Var. Partial Diﬀerential Equations, 50(3-4):693–721, 2014.

K80 [76] A. Kaplan. Fundamental solutions for a class of hypoelliptic PDE generated by composition of quadratic forms. Trans. Amer. Math. Soc., 258(1):147–153, 1980.

K95 [77] T. Kato. Perturbation theory for linear operators. Classics in Mathematics. Springer- Verlag, Berlin, 1995. Reprint of the 1980 edition.

KR95 [78] A. Kor´anyi and H. M. Reimann. Foundations for the theory of quasiconformal mappings on the Heisenberg group. Adv. Math., 111(1):1–87, 1995.

70 KP13 [79] S. G. Krantz and H. R. Parks. The implicit function theorem. Modern Birkh¨auser Classics. Birkh¨auser/Springer,New York, 2013. History, theory, and applications, Reprint of the 2003 edition.

LO14 [80] J. Lauret and D. Oscari. On non-singular 2-step nilpotent Lie algebras. Math. Res. Lett., 21(3):553–583, 2014.

LLMV13 [81] E. Le Donne, G. P. Leonardi, R. Monti, and D. Vittone. Extremal curves in nilpotent Lie groups. Geom. Funct. Anal., 23(4):1371–1401, 2013.

LDMOPV16 [82] E. Le Donne, R. Montgomery, A. Ottazzi, P. Pansu, and D. Vittone. Sard property for the endpoint map on some Carnot groups. Ann. Inst. H. Poincar´eAnal. Non Lin´eaire, 33(6):1639–1666, 2016.

LDR17 [83] E. Le Donne and S. Rigot. Besicovitch covering property for homogeneous distances on the Heisenberg groups. J. Eur. Math. Soc. (JEMS), 19(5):1589–1617, 2017.

Li06 [84] H.-Q. Li. Estimation optimale du gradient du semi-groupe de la chaleur sur le groupe de Heisenberg. J. Funct. Anal., 236(2):369–394, 2006.

Li07 [85] H.-Q. Li. Estimations asymptotiques du noyau de la chaleur sur les groupes de Heisenberg. C. R. Math. Acad. Sci. Paris, 344(8):497–502, 2007.

Li09 [86] H.-Q. Li. Fonctions maximales centr´ees de Hardy-Littlewood sur les groupes de Heisenberg. Studia Math., 191(1):89–100, 2009.

Li10 [87] H.-Q. Li. Estimations optimales du noyau de la chaleur sur les groupes de type Heisenberg. J. Reine Angew. Math., 646:195–233, 2010.

Li19 [88] H.-Q. Li. The Carnot-Carath´eodory distance on 2-step groups. 2019.

Li19O [89] H.-Q. Li. The Carnot-Carath´eodory distance on 2-step groups. To appear in the proceedings of the 8th ICCM, 2019.

LQ14 [90] H.-Q. Li and B. Qian. Centered Hardy-Littlewood maximal functions on Heisenberg type groups. Trans. Amer. Math. Soc., 366(3):1497–1524, 2014.

LZ19 [91] H.-Q. Li and Y. Zhang. Revisiting the heat kernel on isotropic and nonisotropic Heisenberg groups. Comm. Partial Diﬀerential Equations, 44(6):467–503, 2019.

LS95 [92] W. Liu and H. J. Sussman. Shortest paths for sub-Riemannian metrics on rank-two distributions. Mem. Amer. Math. Soc., 118(564):x+104, 1995.

LP03 [93] F. Lust-Piquard. A simple-minded computation of heat kernels on Heisenberg groups. Colloq. Math., 97(2):233–249, 2003.

LP04 [94] F. Lust-Piquard. Riesz transforms on generalized Heisenberg groups and Riesz transforms associated to the CCR heat ﬂow. Publ. Mat., 48(2):309–333, 2004.

71 LP10 [95] F. Lust-Piquard. Ornstein-Uhlenbeck semi-groups on stratiﬁed groups. J. Funct. Anal., 258(6):1883–1908, 2010.

Ma02 [96] V. Magnani. Elements of geometric measure theory on sub-Riemannian groups. Scuola Normale Superiore, Pisa, 2002.

MaM16 [97] A. Martini and D. M¨uller.Spectral multipliers on 2-step groups: topological versus homogeneous dimension. Geom. Funct. Anal., 26(2):680–702, 2016.

M80 [98] G. M´etivier.Hypoellipticit´eanalytique sur des groupes nilpotents de rang 2. Duke Math. J., 47(1):195–221, 1980.

MR20 [99] T. Mietton and L. Rizzi. Branching geodesics in sub-riemannian geometry. Geom. Funct. Anal., 30(4):1139–1151, 2020.

M85 [100] J. Mitchell. On Carnot-Carath´eodory metrics. J. Diﬀerential Geom., 21(1):35–45, 1985.

MPAM06 [101] F. Monroy-P´erezand A. Anzaldo-Meneses. The step-2 nilpotent (n, n(n + 1)/2) sub-Riemannian geometry. J. Dyn. Control Syst., 12(2):185–216, 2006.

MM16 [102] A. Montanari and D. Morbidelli. On the lack of semiconcavity of the subRiemannian distance in a class of Carnot groups. J. Math. Anal. Appl., 444(2):1652–1674, 2016.

MM17 [103] A. Montanari and D. Morbidelli. On the subRiemannian cut locus in a model of free two-step Carnot group. Calc. Var. Partial Diﬀerential Equations, 56(2):Paper No. 36, 26, 2017.

Mont94 [104] R. Montgomery. Abnormal minimizers. SIAM J. Control Optim., 32(6):1605–1620, 1994.

M02 [105] R. Montgomery. A tour of subriemannian geometries, their geodesics and applications, volume 91 of Mathematical Surveys and Monographs. American Mathematical Society, Providence, RI, 2002.

M90 [106] D. M¨uller. A restriction theorem for the Heisenberg group. Ann. of Math. (2), 131(3):567–587, 1990.

MR96 [107] D. M¨uller and F. Ricci. Solvability for a class of doubly characteristic diﬀerential operators on 2-step nilpotent groups. Ann. of Math. (2), 143(1):1–49, 1996.

MRS95 [108] D. M¨uller,F. Ricci, and E. M. Stein. Marcinkiewicz multipliers and multi-parameter structure on Heisenberg (-type) groups. I. Invent. Math., 119(2):199–233, 1995.

MS04 [109] D. M¨ullerand A. Seeger. Singular spherical maximal operators on a class of two step nilpotent Lie groups. Israel J. Math., 141:315–340, 2004.

MS15 [110] D. M¨ullerand A. Seeger. Sharp Lp bounds for the wave equation on groups of Heisenberg type. Anal. PDE, 8(5):1051–1100, 2015.

72 MS94 [111] D. M¨ullerand E. M. Stein. On spectral multipliers for Heisenberg and related groups. J. Math. Pures Appl. (9), 73(4):413–440, 1994.

MS99 [112] D. M¨ullerand E. M. Stein. Lp-estimates for the wave equation on the Heisenberg group. Rev. Mat. Iberoamericana, 15(2):297–334, 1999.

My02 [113] O. Myasnichenko. Nilpotent (3, 6) sub-Riemannian problem. J. Dynam. Control Systems, 8(4):573–597, 2002.

NRS01 [114] A. Nagel, F. Ricci, and E. M. Stein. Singular integrals with ﬂag kernels and analysis on quadratic CR manifolds. J. Funct. Anal., 181(1):29–118, 2001.

NSW85 [115] A. Nagel, E. M. Stein, and S. Wainger. Balls and metrics deﬁned by vector ﬁelds. I. Basic properties. Acta Math., 155(1-2):103–147, 1985.

Q11 [116] B. Qian. Positive curvature property for some hypoelliptic heat kernels. Bull. Sci. Math., 135(3):262–278, 2011.

RS87 [117] F. Ricci and E. M. Stein. Harmonic analysis on nilpotent groups and singular integrals. I. Oscillatory integrals. J. Funct. Anal., 73(1):179–194, 1987.

R13 [118] L. Riﬀord. Ricci curvatures in Carnot groups. Math. Control Relat. Fields, 3(4):467– 487, 2013.

R14 [119] L. Riﬀord. Sub-Riemannian geometry and optimal transport. SpringerBriefs in Mathematics. Springer, Cham, 2014.

RT05 [120] L. Riﬀord and E. Tr´elat. Morse-Sard type results in sub-Riemannian geometry. Math. Ann., 332(1):145–159, 2005.

R04 [121] S. Rigot. Counter example to the Besicovitch covering property for some Carnot groups equipped with their Carnot-Carath´eodory metric. Math. Z., 248(4):827–848, 2004.

R05 [122] S. Rigot. Mass transportation in groups of type H. Commun. Contemp. Math., 7(4):509–537, 2005.

RS17 [123] L. Rizzi and U. Serres. On the cut locus of free, step two Carnot groups. Proc. Amer. Math. Soc., 145(12):5341–5357, 2017.

RS76 [124] L. P. Rothschild and E. M. Stein. Hypoelliptic diﬀerential operators and nilpotent groups. Acta Math., 137(3-4):247–320, 1976.

S90 [125] L. Saloff-Coste. Analyse sur les groupes de Lie àcroissance polynômiale. Ark. Mat., 28(2):315–331, 1990.

S96 [126] A. Sikora. Sharp pointwise estimates on heat kernels. Quart. J. Math. Oxford Ser. (2), 47(187):371–382, 1996.

S04 [127] A. Sikora. Riesz transform, Gaussian bounds and the method of wave equation. Math. Z., 247(3):643–662, 2004.

73 S20 [128] G. Stefani. Generalized Bakry-Emery´ curvature condition and equivalent entropic inequalities in groups. arXiv e-prints, page arXiv:2008.13731, Aug. 2020.

S86 [129] R. S. Strichartz. Sub-Riemannian geometry. J. Diﬀerential Geom., 24(2):221–263, 1986.

S89 [130] R. S. Strichartz. Corrections to: “Sub-Riemannian geometry” [J. Diﬀerential Geom. 24 (1986), no. 2, 221–263; MR0862049 (88b:53055)]. J. Diﬀerential Geom., 30(2):595–596, 1989.

ERS99 [131] A. F. M. ter Elst, D. W. Robinson, and A. Sikora. Riesz transforms and Lie groups of polynomial growth. J. Funct. Anal., 162(1):14–51, 1999.

VSC92 [132] N. T. Varopoulos, L. Saloﬀ-Coste, and T. Coulhon. Analysis and geometry on groups, volume 100 of Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, 1992.

Hong-Quan Li, Ye Zhang School of Mathematical Sciences/Shanghai Center for Mathematical Sciences Fudan University 220 Handan Road Shanghai 200433 People’s Republic of China E-Mail: hongquan [email protected] [email protected] or [email protected]