The Nonlinear Fractional Relativistic Schr¨Odinger

DISCRETE AND CONTINUOUS doi:10.3934/dcds.2021092 DYNAMICAL SYSTEMS

THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION: EXISTENCE, MULTIPLICITY, DECAY AND CONCENTRATION RESULTS

Vincenzo Ambrosio

Dipartimento di Ingegneria Industriale e Scienze Matematiche Universit`aPolitecnica delle Marche Via Brecce Bianche, 12, 60131 Ancona, Italy

(Communicated by Enrico Valdinoci)

Abstract. In this paper we study the following class of fractional relativistic Schrödingerequations: 2 s N (−∆ + m ) u + V (ε x)u = f(u) in R , s N N u ∈ H (R ), u > 0 in R , where ε > 0 is a small parameter, s ∈ (0, 1), m > 0, N > 2s,(−∆ + m2)s is N the fractional relativistic Schrödingeroperator, V : R → R is a continuous potential satisfying a local condition, and f : R → R is a continuous subcritical nonlinearity. By using a variant of the extension method and a penalization technique, we first prove that, for ε > 0 small enough, the above problem admits a weak solution uε which concentrates around a local minimum point of V as ε → 0. We also show that uε has an exponential decay at infinity by constructing a suitable comparison function and by performing some refined estimates. Secondly, by combining the generalized Nehari manifold method and Ljusternik-Schnirelman theory, we relate the number of positive solutions with the topology of the set where the potential V attains its minimum value.

1. Introduction. In this paper we consider the following class of nonlinear fractional elliptic problems:

(−∆ + m2)su + V (ε x)u = f(u) in N , R (1) u ∈ Hs(RN ), u > 0 in RN , where ε > 0 is a small parameter, s ∈ (0, 1), N > 2s, m > 0, V : RN → R is a continuous potential and f : R → R is a continuous nonlinearity. The nonlocal operator (−∆ + m2)s appearing in (1) is deﬁned via Fourier transform by

2 s −1 2 2 s N (−∆ + m ) u(x) := F ((|k| + m ) Fu(k))(x), x ∈ R ,

2020 Mathematics Subject Classiﬁcation. Primary: 35R11, 35J10, 35J20; Secondary: 35J60, 35B09, 58E05. Key words and phrases. fractional relativistic Schr¨odingeroperator, extension method, variational methods, Ljusternik-Schnirelman theory.

1 2 VINCENZO AMBROSIO for any u : RN → R belonging to the Schwartz space S(RN ) of rapidly decaying functions, or equivalently (see [25, 35]) as (−∆ + m2)su(x) := m2su(x)

N+2s Z u(x) − u(y) 2 N + C(N, s)m P.V. N+2s K N+2s (m|x − y|) dy, x ∈ R , (2) N 2 R |x − y| 2 where P.V. stands for the Cauchy principal value, Kν is the modiﬁed Bessel function of the third kind (or Macdonald function) of index ν (see [8, 23]) which satisﬁes the following well-known asymptotic formulas for ν ∈ R and r > 0: Γ(ν) r −ν K (r) ∼ as r → 0, for ν > 0, (3) ν 2 2 1 K (r) ∼ log as r → 0, 0 r r π − 1 −r K (r) ∼ r 2 e as r → ∞, for ν ∈ , (4) ν 2 R and C(N, s) is a positive constant whose exact value is given by

− N+2s +1 − N 2s s(1 − s) C(N, s) := 2 2 π 2 2 . Γ(2 − s) Equations involving (−∆ + m2)s arise in the study of standing waves ψ(x, t) for Schrödinger-Klein-Gordonequations of the form ∂ψ ı = (−∆ + m2)sψ − f(x, ψ), in N × , ∂t R R which describe the behavior of bosons,√ spin-0 particles in relativistic fields. In particular, when s = 1/2, the operator −∆ + m2 − m plays an important role in relativistic quantum mechanics because it corresponds to the kinetic energy of a relativistic particle with mass m > 0. If p is the momentum of the particle then its relativistic kinetic energy is given by E = pp2 + m2. In the process of quantization the momentum p is replaced by the differential operator −ı∇ and the √quantum analog of the relativistic kinetic energy is the free relativistic Hamiltonian −∆ + m2 − m. Physical models related to this operator have been widely studied over the past 30 years and there exists a huge literature on the spectral properties of relativistic Hamiltonians, most of it has been strongly influenced by the works of Lieb on the stability of relativistic matter; see [32, 36, 35, 48, 49] for more physical background. On the other hand, there is also a deep connection between (−∆+m2)s and the theory of Lévyprocesses. Indeed, m2s − (−∆ + m2)s is the infinitesimal 2s,m generator of a Lévyprocess Xt called 2s-stable relativistic process having the following characteristic function

0 ık·X2s,m −t[(|k|2+m2)s−m2s] N E e t = e , k ∈ R ; we refer to [11, 14, 41] for a more detailed discussion on relativistic stable processes. When m = 0, the previous operator boils down to the fractional Laplacian operator (−∆)s which has been extensively studied in these last years due to its great applications in several fields of the research; see [9, 21, 38] for an introduction on this topic. In particular, a great interest has been devoted to the existence and multiplicity of solutions for fractional Schrödingerequations like 2s s N ε (−∆) u + V (x)u = f(u) in R , (5) THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 3 and the asymptotic behavior of its solutions as ε → 0; see for instance [2,5,6, 7, 19, 29] and the references therein. We also mention [18] in which the authors investigated a nonlocal Schrödingerequation with Dirichlet datum. When m > 0 and ε = 1 in (1), some interesting existence, multiplicity, and quali- tative results can be found in [4, 10, 16, 17, 27, 40, 42], while in [15]√ the semiclassical limit ε → 0 for a pseudo-relativistic Hartree-equation involving − ε2 ∆ + m2 is considered. Motivated by the above papers, in this work we focus our attention on the existence, multiplicity, decay and concentration phenomenon as ε → 0 of solutions to (1). Firstly, we suppose that V : RN → R is a continuous function which satisfies the following del Pino-Felmer type conditions [20]: 2s (V ) there exists V ∈ (0, m ) such that −V := inf N V (x), 1 1 1 x∈R N (V2) there exists a bounded open set Λ ⊂ R such that

−V0 := inf V (x) < min V (x), x∈Λ x∈∂Λ

with V0 > 0. We also set M := {x ∈ Λ: V (x) = −V0}. Without loss of generality, we may assume that 0 ∈ M. Concerning the nonlinearity f, we assume that f : R → R is continuous, f(t) = 0 for t ≤ 0, and f fulﬁlls the following hypotheses: f(t) (f1) limt→0 t = 0, f(t) ∗ ∗ 2N (f2) lim supt→∞ tp < ∞ for some p ∈ (1, 2s − 1), where 2s := N−2s is the fractional critical exponent, ∗ (f3) there exists θ ∈ (2, 2s) such that 0 < θF (t) ≤ tf(t) for all t > 0, where R t F (t) := 0 f(τ) dτ, f(t) (f4) the function t 7→ t is increasing in (0, ∞). The ﬁrst main result of this paper can be stated as follows:

Theorem 1.1. Assume that (V1)-(V2) and (f1)-(f4) are satisﬁed. Then, for every small ε > 0, there exists a solution uε to (1) such that uε has a maximum point xε satisfying lim dist(ε xε,M) = 0, ε→0 and for which −c|x−xε| N 0 < uε(x) ≤ Ce for all x ∈ R , for suitable constants C, c > 0. Moreover, for any sequence (εn) with εn → 0, there exists a subsequence, still denoted by itself, such that there exist a point x0 ∈ M s N with εn xεn → x0, and a positive least energy solution u ∈ H (R ) of the limiting problem 2 s N (−∆ + m ) u − V0u = f(u) in R , for which we have

uεn (x) = u(x − xεn ) + Rn(x), where lim kR k s N = 0. n→∞ n H (R ) The proof of Theorem 1.1 is obtained through suitable variational techniques. Firstly, we start by observing that (−∆+m2)s is a nonlocal operator and that does not scale like the fractional Laplacian operator (−∆)s. More precisely, the first operator is not compatible with the semigroup R+ acting on functions as t ∗ u 7→ u(t−1x) for t > 0. This means that several arguments performed to deal with (5) do not work in our context. To overcome the nonlocality of (−∆ + m2)s, we use 4 VINCENZO AMBROSIO a variant of the extension method [12] (see [16, 25, 45]) which permits to study via local variational methods a degenerate elliptic equation in a half-space with a nonlinear Neumann boundary condition. Clearly, some additional difficulties arise in the investigation of this new problem because we have to handle the trace terms of the involved functions and thus a more careful analysis will be needed. Due to the lack of information on the behavior of V at infinity, we carry out a penalization argument [20] which consists in modifying appropriately the nonlinearity f outside Λ, and thus consider a modified problem whose corresponding energy functional fulfills all the assumptions of the mountain pass theorem [3]. Then we need to check that, for ε > 0 small enough, the solutions of the auxiliary problem are indeed solutions of the original one. This goal will be achieved by combing an appropriate Moser iteration argument [39] with some regularity estimates for degenerate elliptic equations; see Lemmas 4.1 and 4.2. To our knowledge, this is the first time that the penalization trick is used to study concentration phenomena for the fractional relativistic Schrödingeroperator (−∆+m2)s for all s ∈ (0, 1) and m > 0. Moreover, we show that the solutions of (1) have an exponential decay, contrary to the case m = 0 for which is well-known that the solutions of (5) satisfy the power-type decay |x|−(N+2s) as |x| → ∞; see [6, 26]. To investigate the decay of solutions to (1), we construct a suitable comparison function and we carry out some refined estimates which take care of an adequate estimate concerning 2s-stable relativistic density with parameter m found in [31], and that the modified Bessel function Kν has an exponential decay at infinity. We stress that exponential type estimates 1 for equations like (1), appear in [16] where s = 2 , V is bounded, f is a Hartree type nonlinearity, and in [27] where s ∈ (0, 1), m = 1, V ≡ 0 and |f(u)| ≤ C|u|p ∗ for some p ∈ (1, 2s − 1). Anyway, our approach to obtain the decay estimate is completely different from the above mentioned papers, it is more general and we believe that can be applied in other situations to deal with fractional problems driven by (−∆ + m2)s. In the second part of this paper, we provide a multiplicity result for (1). In this case, we assume the following conditions on V : 0 2s (V ) there exists V ∈ (0, m ) such that −V := inf N V (x), 1 0 0 x∈R 0 N (V2 ) there exists a bounded open set Λ ⊂ R such that

−V0 < min V (x), x∈∂Λ

and 0 ∈ M := {x ∈ Λ: V (x) = −V0}. In order to state precisely the next theorem, we recall that if Y is a given closed subset of a topological space X, we denote by catX (Y ) the Ljusternik-Schnirelman category of Y in X, that is the least number of closed and contractible sets in X which cover Y (see [50]). Our second main result reads as follows: 0 0 Theorem 1.2. Assume that (V1 )-(V2 ) and (f1)-(f4) hold. Then, for any δ > 0 such that N Mδ := {x ∈ R : dist(x, M) ≤ δ} ⊂ Λ, there exists εδ > 0 such that, for any ε ∈ (0, εδ), problem (1) has at least catMδ (M) positive solutions. Moreover, if uε denotes one of these solutions and xε is a global maximum point of uε, then we have

lim V (ε xε) = −V0. ε→0 The proof of Theorem 1.2 will be obtained by combining variational methods, a penalization technique and Ljusternik-Schnirelman category theory. Since the THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 5 nonlinearity f is only continuous, we cannot use standard arguments for C1-Nehari manifold (see [50]) and we overcome the non differentiability of the Nehari manifold by taking advantage of some variants of critical point theorems from Szulkin and Weth [46]. We stress that a similar approach for nonlocal problems appeared in [5, 28], but with respect to these papers our analysis is rather tough due to the different structure of the energy functional associated to the extended modified problem (this difficulty is again caused by the lack of scaling invariance for (−∆ + m2)s). Therefore, some clever and intriguing estimates will be needed to achieve our purpose. As far as we known, this is the first time that the penalization scheme and topological arguments are mixed to get multiple solutions for fractional relativistic Schrödingerequations. The paper is organized as follows. In section 2 we collect some notations and preliminary results which will be used along the paper. In section 3 we introduce a penalty functional in order to apply suitable variational arguments. The section 4 is devoted to the proof of Theorem 1.1. The proof of Theorem 1.2 is given in Section 5. Finally, we give some interesting results for (−∆ + m2)s in the appendices A and B.

2. Preliminaries.

2.1. Notations and functional setting. We denote the upper half-space in RN+1 by N+1 N+1 R+ := (x, y) ∈ R : y > 0 , N+1 p 2 2 and for (x, y) ∈ R+ we consider the Euclidean norm |(x, y)| := |x| + y . By N N Br(x) we mean the (open) ball in R centered at x ∈ R with radius r > 0. When x = 0, we simply write Br. Let p ∈ [1, ∞] and A ⊂ RN be a measurable set. The notation Ac := RN \ A stands for the complement of A in RN . We indicate by Lp(A) the set of measurable functions u : RN → R such that ( 1/p R |u|p dx < ∞ if p < ∞, |u|Lp(A) := A esssupx∈A|u(x)| if p = ∞. N If A = , we simply write |u|p instead of |u|Lp( N ). With kwk p N+1 we denote R R L (R+ ) p N+1 + the norm of w ∈ L (R+ ). For a generic real-valued function v, we set v := − max{v, 0}, v := min{v, 0}, and v− := max{−v, 0}. Let D ⊂ RN+1 be a bounded domain, that is a bounded connected open set, 0 N+1 N with boundary ∂D. We denote by ∂ D the interior of D ∩ ∂R+ in R , and we set ∂00D := ∂D \ ∂0D. For R > 0, we put + N+1 BR := {(x, y) ∈ R+ : |(x, y)| < R}, 0 0 + N+1 ΓR := ∂ BR = {(x, 0) ∈ ∂R+ : |x| < R}, + 00 + N+1 ΓR := ∂ BR = {(x, y) ∈ R : y ≥ 0, |(x, y)| = R}. Now, we introduce the Lebesgue spaces with weight (see [24, 34] for more details). N+1 r 1−2s Let D ⊂ R+ be an open set and r ∈ (1, ∞). Denote by L (D, y ) the weighted Lebesgue space of all measurable functions v : D → R such that 1 ZZ r 1−2s r kvkLr (D,y1−2s) := y |v| dxdy < ∞. D 6 VINCENZO AMBROSIO

We say that v ∈ H1(D, y1−2s) if v ∈ L2(D, y1−2s) and its weak derivatives, col- lectively denoted by ∇v, exist and belong to L2(D, y1−2s). The norm of v in H1(D, y1−2s) is given by

1 ZZ 2 1−2s 2 2 kvkH1(D,y1−2s) := y (|∇v| + v ) dxdy . D It is clear that H1(D, y1−2s) is a Hilbert space with the inner product ZZ y1−2s(∇v · ∇w + vw) dxdy. D s N ∞ N Let H (R ) be the fractional Sobolev space deﬁned as the completion of Cc (R ) with respect to the norm

1 Z 2 2 2 s 2 |u| s N := (|k| + m ) |Fu(k)| dk . H (R ) N R s N p N ∗ Then, H (R ) is continuously embedded in L (R ) for all p ∈ [2, 2s) and compactly p N ∗ s N+1 in Lloc(R ) for all p ∈ [1, 2s); see [1,8, 21, 35]. Next we deﬁne X (R+ ) := 1 N+1 1−2s ∞ N+1 H (R+ , y ) as the completion of Cc (R+ ) with respect to the norm 1 ZZ ! 2 1−2s 2 2 2 kuk s N+1 := y (|∇u| + m u ) dxdy . X (R+ ) N+1 R+ s N+1 By Lemma 3.1 in [25], we deduce that X (R+ ) is continuously embedded in 2γ N+1 1−2s L (R+ , y ), that is s N+1 kuk 2γ N+1 1−2s ≤ C∗kuk s N+1 for all u ∈ X ( + ), (6) L (R+ ,y ) X (R+ ) R 2 r N+1 1−2s where γ := 1 + N−2s , and L (R+ , y ) is the weighted Lebesgue space, with r ∈ (1, ∞), endowed with the norm

1 ZZ ! r 1−2s r kuk r N+1 1−2s := y |u| dxdy . L (R+ ,y ) N+1 R+ s N+1 Moreover, by Lemma 3.1.2 in [22], we also have that X (R+ ) is compactly em- 2 + 1−2s bedded in L (BR , y ) for all R > 0. From Proposition 5 in [25], we know that s N+1 s N there exists a (unique) linear trace operator Tr : X (R+ ) → H (R ) such that √ s N+1 σs|Tr(u)|Hs( N ) ≤ kuk s N+1 for all u ∈ X ( + ), (7) R X (R+ ) R 1−2s s where σs := 2 Γ(1−s)/Γ(s). We also note the (7) and the deﬁnition of H -norm imply that Z ZZ 2s 2 1−2s 2 2 2 σsm |Tr(u)| dx ≤ y (|∇u| + m u ) dxdy. (8) N N+1 R R+ In what follows, in order to simplify the notation, we denote Tr(u) by u(·, 0). s N+1 s N s N q N Since Tr(X (R+ )) ⊂ H (R ) and the embedding H (R ) ⊂ L (R ) is con- ∗ ∗ tinuous for all q ∈ [2, 2s) and locally compact for all q ∈ [1, 2s), we obtain the following result:

s N+1 q N Theorem 2.1. Tr(X (R+ )) is continuously embedded in L (R ) for all q ∈ ∗ q N ∗ [2, 2s) and compactly embedded in Lloc(R ) for all q ∈ [1, 2s). THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 7

In order to circumvent the nonlocal character of the pseudo-diﬀerential operator (−∆ + m2)s, we make use of a variant of the extension method [12] given in [16, 25, 45]. More precisely, for any u ∈ Hs(RN ) there exists a unique function U ∈ s N+1 X (R+ ) solving the following problem 1−2s 2 1−2s N+1 − div(y ∇U) + m y U = 0 in R+ , N+1 ∼ N U(·, 0) = u on ∂R+ = R . The function U is called the extension of u and possesses the following properties: (i)

∂U 1−2s ∂U 2 s := − lim y (x, y) = σs(−∆ + m ) u(x) in distribution sense, ∂ν1−2s y→0 ∂y √ s N+1 (ii) σs|u|Hs( N ) = kUk s N+1 ≤ kV k s N+1 for all V ∈ X ( + ) such that R X (R+ ) X (R+ ) R V (·, 0) = u. N ∞ N+1 N+1 (iii) if u ∈ S(R ) then U ∈ C (R+ ) ∩ C(R+ ) and it can be expressed as Z U(x, y) = Ps,m(x − z, y)u(z) dz, N R with N+2s N+2s 0 2s 2 − 2 Ps,m(x, y) := cN,sy m |(x, y)| K N+2s (m|(x, y)|), 2 and N+2s −1 0 2 2 cN,s := pN,s N+2s , Γ( 2 ) where pN,s is the constant for the (normalized) Poisson kernel with m = 0; see [45]. p 2 2 We note that Ps,m is the Fourier transform of k 7→ ϑ( |k| + m ) and that Z Ps,m(x, y) dx = ϑ(my), (9) N R 1 1−2s where ϑ ∈ H (R+, y ) solves the following ordinary differential equation 00 1−2s 0 ϑ + y ϑ − ϑ = 0 in R+, ϑ(0) = 1, limy→∞ ϑ(y) = 0. We also recall that ϑ can be expressed via modified Bessel functions, more 2 r s precisely, ϑ(r) = Γ(s) ( 2 ) Ks(r); see [25] for more details. Taking into account the previous facts, problem (1) can be realized in a local manner through the following nonlinear boundary value problem: 1−2s 2 1−2s N+1 − div(y ∇w) + m y w = 0 in R+ , ∂w N (10) ∂ν1−2s = σs[−Vε(x)w(·, 0) + f(w(·, 0))] on R , where Vε(x) := V (ε x). For simplicity of notation, we will omit the constant σs from the second equation in (10). For all ε > 0, we define Z s N+1 2 Xε := u ∈ X (R+ ): Vε(x)u (x, 0) dx < ∞ N R endowed with the norm 1 Z 2 2 2 kuk := kuk N+1 + V (x)u (x, 0) dx . ε Xs( ) ε R+ N R 8 VINCENZO AMBROSIO

We note that k · kε is actually a norm. Indeed, Z Z 2 2 2 2 kuk = kuk N+1 − V u (x, 0) dx + (V (x) + V )u (x, 0) dx, ε Xs( ) 1 ε 1 R+ N N R R and using (8) and (V1) we can see that Z V1 2 2 2 2 1 − kuk N+1 ≤ kuk N+1 − V u (x, 0) dx ≤ kuk N+1 2s Xs( ) Xs( ) 1 Xs( ) m R+ R+ N R+ R that is Z 2 2 kuk N+1 − V u (x, 0) dx Xs( ) 1 R+ N R 2 is a norm equivalent to k · k s N+1 . This observation yields the required claim. X (R+ ) s N+1 Clearly, Xε ⊂ X (R+ ), and by (8) and (V1), we have 2s 2 m 2 kuk N+1 ≤ kuk for all u ∈ X . (11) Xs( ) 2s ε ε R+ m − V1

Moreover, Xε is a Hilbert space endowed with the inner product ZZ Z 1−2s 2 hu, viε := y (∇u · ∇v + m uv) dxdy + Vε(x)u(x, 0)v(x, 0) dx N+1 N R+ R ∗ for all u, v ∈ Xε. With Xε we will denote the dual space of Xε. 2.2. Elliptic estimates. For reader’s convenience, we list some results about local Schauder estimates for degenerate elliptic equations involving the operator −div(y1−2s∇v) + m2y1−2sv, with m > 0. Firstly we give the following deﬁnition:

N+1 0 Deﬁnition 2.2. Let D ⊂ R+ be a bounded domain with ∂ D 6= ∅. Let f ∈ 2N N+2s 0 1 0 Lloc (∂ D) and g ∈ Lloc(∂ D). Consider −div(y1−2s∇v) + m2y1−2sv = 0 in D, ∂v 0 (12) ∂νa = f(x)v + g(x) on ∂ D. We say that v ∈ H1(D, y1−2s) is a weak supersolution (resp. subsolution) to (12) 1 0 in D if for any nonnegative ϕ ∈ Cc (D ∪ ∂ D), ZZ Z y1−2s(∇v · ∇φ + m2vϕ) dxdy ≥ (≤) [f(x)v(x, 0) + g(x)]ϕ(x, 0) dx. D ∂0D We say that v ∈ H1(D, y1−2s) is a weak solution to (12) in D if it is both a weak supersolution and a weak subsolution.

For R > 0, we set QR := BR × (0,R). Then we recall the following version of De Giorgi-Nash-Moser type theorems established in [25] (see also [24, 34] for the case m = 0).

q N Proposition 1. [25] Let f, g ∈ L (B1) for some q > 2s . 1 1−2s (i) Let v ∈ H (Q1, y ) be a weak subsolution to (12) in Q1. Then + + + 2 1−2s q sup v ≤ C(kv kL (Q1,y ) + |g |L (B1)), Q1/2 + q where C > 0 depends only on m, N, s, q, |f |L (B1). THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 9

1 1−2s (ii) (weak Harnack inequality) Let v ∈ H (Q1, y ) be a nonnegative weak supersolution to (12) in Q1. Then for some p0 > 0 and any 0 < µ < τ < 1 we have

q p 1−2s inf v + |g−|L (B1) ≥ CkvkL 0 (Qτ ,y ), Q¯µ

q where C > 0 depends only on m, N, s, q, µ, τ, |f−|L (B1). 1 1−2s (iii) Let v ∈ H (Q1, y ) be a nonnegative weak solution to (12) in Q1. Then 0,α v ∈ C (Q1/2) and in addition

kvk 0,α ≤ C(kvk 2 + |g| q ), C (Q1/2) L (Q1) L (B1)

q with C > 0, α ∈ (0, 1) depending only on m, N, s, p, |f|L (B1).

3. The penalization argument. In order to find solutions to (10), we adapt the penalization approach in [20] which permits to study our problem via variational V1 θ f(a) V1 arguments. Fix κ > max{ 2s , } > 1 and a > 0 such that = . Define m −V1 θ−2 a κ ˜ f(t) for t < a, f(t) := V1 κ t for t ≥ a. Let us consider the following Carathéodory function ˜ N g(x, t) := χΛ(x)f(t) + (1 − χΛ(x))f(t) for (x, t) ∈ R × R, R t where χΛ denotes the characteristic function of Λ. Set G(x, t) := 0 g(x, τ) dτ. By (f1)-(f4) it follows that g(x,t) N (g1) limt→0 t = 0 uniformly in x ∈ R , N (g2) g(x, t) ≤ f(t) for all x ∈ R , t > 0, (g3)( i) 0 < θG(x, t) ≤ tg(x, t) for all x ∈ Λ and t > 0, V1 2 c (ii) 0 ≤ 2G(x, t) ≤ tg(x, t) ≤ κ t for all x ∈ Λ and t > 0, g(x,t) (g4) for each x ∈ Λ the function t 7→ t is increasing in (0, ∞), and for each c g(x,t) x ∈ Λ the function t 7→ t is increasing in (0, a). Consider the following modified problem: 1−2s 2 1−2s N+1 − div(y ∇u) + m y u = 0 in R+ , ∂u N (13) ∂ν1−2s = −Vε(x)u(·, 0) + gε(·, u(·, 0)) on R , where we set gε(x, t) := g(ε x, t). Obviously, if uε is a positive solution of (13) c N satisfying uε(x, 0) < a for all x ∈ Λε, where Λε := {x ∈ R : ε x ∈ Λ}, then uε is indeed a positive solution of (10). We associate with problem (13) the energy functional Jε : Xε → R defined as Z 1 2 Jε(u) := kukε − Gε(x, u(x, 0)) dx. 2 N R 1 Clearly, Jε ∈ C (Xε, R) and its differential is given by ZZ Z 0 1−2s 2 hJε(u), vi = y (∇u · ∇v + m uv) dxdy + Vε(x)u(x, 0)v(x, 0) dx N+1 N R+ R Z − gε(x, u(x, 0))v(x, 0) dx N R for all u, v ∈ Xε. Thus, the critical points of Jε correspond to the weak solutions of (13). To find these critical points, we will apply the mountain pass theorem [3]. We start by proving that Jε satisfies the geometric features required by this theorem. 10 VINCENZO AMBROSIO

Lemma 3.1. Jε has a mountain pass geometry, that is: (i) Jε(0) = 0, (ii) there exist α, ρ > 0 such that Jε(u) ≥ α for all u ∈ Xε such that kukε = ρ, (iii) there exists v ∈ Xε such that kvkε > ρ and Jε(v) < 0.

Proof. Clearly, (i) is true. By (f1), (f2), (g1), (g2), we deduce that for all η > 0 there exists Cη > 0 such that

2∗−1 N |gε(x, t)| ≤ η|t| + Cη|t| s for all (x, t) ∈ R × R, (14) and η Cη ∗ 2 2s N |Gε(x, t)| ≤ |t| + ∗ |t| for all (x, t) ∈ R × R. (15) 2 2s 2s Fix η ∈ (0, m − V1). Using (15), (8), (11) and Theorem 2.1, we obtain

∗ 1 2 η 2 Cη 2s Jε(u) ≥ kuk − |u(·, 0)| − |u(·, 0)| ∗ ε 2 ∗ 2s 2 2 2s ∗ 1 2 η 2s 2 Cη 2s = kuk − m |u(·, 0)| − |u(·, 0)| ∗ ε 2s 2 ∗ 2s 2 2m 2s ∗ 1 2 η 2 2s ≥ kuk − kuk N+1 − C Ckuk ε 2s Xs( ) η s N+1 2 2m R+ X (R+ )

∗ 1 η 2 2s ≥ − 2s kukε − CηCkukε 2 2(m − V1) from which we deduce that (ii) holds. ∞ N+1 To prove (iii), let v ∈ Cc (R+ ) be such that v 6≡ 0 and supp(v(·, 0)) ⊂ Λε. Then, by (f3), we have 2 Z t 2 Jε(tv) = kvkε − F (tv(x, 0)) dx 2 N R 2 Z t 2 θ θ 0 ≤ kvkε − Ct |v(x, 0)| dx + C |Λε| → −∞ as t → ∞. 2 Λε This completes the proof of the lemma.

By Lemma 3.1 and using a variant of the mountain pass theorem without the Palais-Smale condition (see Theorem 2.9 in [50]), we can ﬁnd a sequence (un) ⊂ Xε such that 0 ∗ Jε(un) → cε and Jε(un) → 0 in Xε , as n → ∞, where

cε := inf max Jε(γ(t)) γ∈Γε t∈[0,1] and

Γε := {γ ∈ C([0, 1],Xε): γ(0) = 0,Jε(γ(1)) < 0}.

In view of (f4), it is standard to check (see [50]) that

cε = inf Jε(u) = inf max Jε(tu), u∈Nε u∈Xε\{0} t≥0 where 0 Nε := {u ∈ Xε : hJε(u), ui = 0} THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 11 is the Nehari manifold associated with Jε. From the growth conditions of g, we see that for any ﬁxed u ∈ Nε and δ > 0 small enough Z 0 2 0 = hJε(u), ui = kukε − gε(x, u)u dx N R ∗ 2 2 2s ≥ kukε − δC1kukε − Cδkukε ∗ 2 2s ≥ C2kukε − Cδkukε , so there exists r > 0, independent of u, such that

kukε ≥ r for all u ∈ Nε. (16)

Now we prove the following fundamental compactness result:

Lemma 3.2. Jε satisﬁes the Palais-Smale condition at any level c ∈ R.

Proof. Let c ∈ R and (un) ⊂ Xε be a Palais-Smale sequence at the level c, namely

0 ∗ Jε(un) → c and Jε(un) → 0 in Xε , as n → ∞. By this fact, (g3), (8) and (11), for n big enough, we have 1 c + 1 + ku k ≥ J (u ) − hJ 0 (u ), u i n ε ε n θ ε n n Z 1 1 2 1 = − kunkε + gε(x, un(x, 0))un(x, 0) − θGε(x, un(x, 0)) dx 2 θ θ N R Z 1 1 2 1 1 V1 2 ≥ − kunkε − − un(x, 0) dx 2 θ 2 θ κ N R Z 1 1 2 1 1 V1 2s 2 = − kunkε − − 2s m un(x, 0) dx 2 θ 2 θ κm N R 1 1 2 1 1 V1 2 ≥ − kunkε − − 2s kunkXs( N+1) 2 θ 2 θ κm R+ 1 1 V1 2 ≥ − 1 − 2s kunkε. 2 θ κ(m − V1)

V1 Since θ > 2 and κ > 2s , we deduce that (un) is bounded in Xε. Hence, up m −V1 to a subsequence, we may assume that un * u in Xε. Now we prove that this ∞ N+1 convergence is indeed strong. Using (g1), (g2), (f2), the density of Cc (R+ ) in 0 Xε, and applying Theorem 2.1, it is easy to check that hJε(u), ϕi = 0 for all ϕ ∈ Xε. In particular, Z Z 2 2 2 kuk N+1 − V |u(·, 0)| + (V (x) + V )u (x, 0) dx + C (x, u(x, 0)) dx Xs( ) 1 2 ε 1 ε R+ c Λε Λε Z = f(u(x, 0))u(x, 0) dx. (17) Λε

2 where Cε(x, t) := (Vε(x) + V1)t − gε(x, t)t. Note that, by (g3)-(ii), it holds

1 (V (x) + V )t2 ≥ C (x, t) ≥ 1 − (V (x) + V )t2 ≥ 0 for all (x, t) ∈ Λc × [0, ∞). ε 1 ε κ ε 1 ε (18) 12 VINCENZO AMBROSIO

0 On the other hand, by hJε(un), uni = on(1), we get Z Z 2 2 2 ku k N+1 − V |u (·, 0)| + (V (x) + V )u (x, 0) dx + C (x, u (x, 0)) dx n Xs( ) 1 n 2 ε 1 n ε n R+ c Λε Λε Z = f(un(x, 0))un(x, 0) dx + on(1). (19) Λε

Since Λε is bounded, by the compactness of Sobolev embeddings in Theorem 2.1 we have Z Z lim f(un(x, 0))un(x, 0) dx = f(u(x, 0))u(x, 0) dx, (20) n→∞ Λε Λε Z Z 2 2 lim (Vε(x) + V1)un(x, 0) dx = (Vε(x) + V1)u (x, 0) dx. (21) n→∞ Λε Λε In view of (17), (19), (20), (21), we obtain Z ! 2 2 lim sup ku k N+1 − V |u (·, 0)| + C (x, u (x, 0)) dx n Xs( ) 1 n 2 ε n n→∞ R+ c Λε Z 2 2 = kuk N+1 − V |u(·, 0)| + C (x, u(x, 0)) dx. Xs( ) 1 2 ε R+ c Λε On the other hand, by (18) and Fatou’s lemma, we get Z ! 2 2 lim inf ku k N+1 − V |u (·, 0)| + C (x, u (x, 0)) dx n Xs( ) 1 n 2 ε n n→∞ R+ c Λε Z 2 2 ≥ kuk N+1 − V |u(·, 0)| + C (x, u(x, 0)) dx. Xs( ) 1 2 ε R+ c Λε Hence,

h 2 2i 2 2 lim kunk s N+1 − V1|un(·, 0)|2 = kuk s N+1 − V1|u(·, 0)|2 (22) n→∞ X (R+ ) X (R+ ) and Z Z lim Cε(x, un(x, 0)) dx = Cε(x, u(x, 0)) dx. n→∞ c c Λε Λε The last limit combined with (18) yields Z Z 2 2 (Vε(x) + V1)u (x, 0) dx ≤ lim inf (Vε(x) + V1)un(x, 0) dx c n→∞ c Λε Λε Z 2 ≤ lim sup (Vε(x) + V1)un(x, 0) dx n→∞ c Λε κ Z ≤ lim sup Cε(x, un(x, 0)) dx κ − 1 n→∞ c Λε κ Z = Cε(x, u(x, 0)) dx κ − 1 c Λε Z κ 2 ≤ (Vε(x) + V1)u (x, 0) dx. κ − 1 c Λε THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 13

By sending κ → ∞, we ﬁnd Z Z 2 2 lim (Vε(x) + V1)un(x, 0) dx = (Vε(x) + V1)u (x, 0) dx, n→∞ c Λε Λε which together with (21) implies that Z Z 2 2 lim (Vε(x) + V1)un(x, 0) dx = (Vε(x) + V1)u (x, 0) dx. (23) n→∞ N N R R From (22) and (23), we have 2 2 lim kunkε = kukε, n→∞ and since Xε turns out to be a Hilbert space, we deduce that un → u in Xε as n → ∞.

In light of Lemmas 3.1 and 3.2, we can apply the mountain pass theorem [3] to see that, for all ε > 0, there exists uε ∈ Xε \{0} such that 0 Jε(uε) = cε and Jε(uε) = 0. (24)

0 − N+1 Since g(·, t) = 0 for t ≤ 0, it follows from hJε(uε), uε i = 0 that uε ≥ 0 in R+ . Next, for µ > −m2s, we consider the following autonomous problem related to (10):

1−2s 2 1−2s N+1 − div(y ∇w) + m y w = 0 in R+ , ∂w N (25) ∂ν1−2s = −µw(·, 0) + f(w(·, 0)) on R , and the corresponding energy functional 1 Z L (u) := kuk2 − F (u(x, 0)) dx, µ Yµ 2 N R s N+1 which is well-deﬁned on Yµ := X (R+ ) endowed with the norm

1 2 2 2 kukYµ := kuk s N+1 + µ|u(·, 0)|2 . X (R+ )

N+1 Note that k · kYµ is a norm equivalent to k · k s . In fact, using (8), for X (R+ ) s N+1 u ∈ X (R+ ), 2 2 2 A kuk N+1 ≤ kuk ≤ B kuk N+1 , m,s s Yµ m,s s X (R+ ) X (R+ ) where 1 1 A := min{µ + m2s, m2s} > 0 and B := max{µ + m2s, m2s} > 0. m,s m2s m,s m2s

Clearly, Yµ is a Hilbert space with the inner product ZZ Z 1−2s 2 hu, viYµ := y (∇u · ∇v + m uv) dxdy + µ u(x, 0)v(x, 0) dx, N+1 N R+ R for all u, v ∈ Yµ. Denote by Mµ the Nehari manifold associated with Lµ , that is 0 Mµ := {u ∈ Yµ : hLµ(u), ui = 0}.

It is easy to check that Lµ has a mountain pass geometry [3]. Then, using a variant of the mountain pass theorem without the Palais-Smale condition (see 14 VINCENZO AMBROSIO

Theorem 2.9 in [50]), we can ﬁnd a Palais-Smale sequence (un) ⊂ Yµ at the mountain pass level dµ of Lµ. Note that (un) is bounded in Yµ. Indeed, by (f3), we get 1 C(1 + ku k ) ≥ L (u ) − hL0 (u ), u i n Yµ µ n θ µ n n 1 1 1 Z = − ku k2 + f(u (x, 0))u (x, 0) − θF (u (x, 0)) dx n Yµ n n n 2 θ θ N R 1 1 2 ≥ − kunk 2 θ Yµ which implies the boundedness of (un) in Yµ. By (f4), we also have that

dµ = inf Lµ(u) = inf max Lµ(tu). u∈Mµ u∈Yµ\{0} t≥0 Next we show the existence of a positive ground state solution to (25). We ﬁrst prove some useful technical lemmas. The ﬁrst one is a vanishing Lions type result (see [37]).

∗ s N+1 Lemma 3.3. Let t ∈ [2, 2s) and R > 0. If (un) ⊂ X (R+ ) is a bounded sequence such that Z t lim sup |un(x, 0)| dx = 0, n→∞ N z∈R BR(z) r N ∗ then un(·, 0) → 0 in L (R ) for all r ∈ (2, 2s). ∗ N Proof. Take q ∈ (t, 2s). Given R > 0 and z ∈ R , by using the H¨olderinequality, we get 1−λ λ |un(·, 0)|Lq (B (z)) ≤ |un(·, 0)| t |un(·, 0)| 2∗ for all n ∈ , R L (BR(z)) L s (BR(z)) N where 1 − λ λ 1 + ∗ = . t 2s q Now, covering RN by balls of radius R, in such a way that each point of RN is contained in at most N + 1 balls, we ﬁnd

q (1−λ)q λq |un(·, 0)| ≤ (N + 1)|un(·, 0)| t |un(·, 0)| ∗ , q L (BR(z)) 2s which combined with Theorem 2.1 and the assumptions yields

q (1−λ)q λq |un(·, 0)| ≤ C(N + 1)|un(·, 0)| t kunk N+1 q L (BR(z)) s X (R+ ) (1−λ)q ≤ C(N + 1) sup |un(·, 0)| t → 0 as n → ∞. L (BR(z)) N z∈R r N A standard interpolation argument leads to un(·, 0) → 0 in L (R ) for all r ∈ ∗ (2, 2s).

Lemma 3.4. Let (un) ⊂ Yµ be a Palais-Smale sequence for Lµ at the level c ∈ R and such that un * 0 in Yµ. Then we have either (a) un → 0 in Yµ, or N (b) there exist a sequence (zn) ⊂ R and constants R, β > 0 such that Z 2 lim inf un(x, 0) dx ≥ β. n→∞ BR(zn) THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 15

Proof. Assume that (b) does not occur. Then, for all R > 0, we have Z 2 lim sup un(x, 0) dx = 0. n→∞ N z∈R BR(z) q N ∗ Using Lemma 3.3, we can see that un(·, 0) → 0 in L (R ) for all q ∈ (2, 2s). This fact and (f1)-(f2) imply that Z f(un(x, 0))un(x, 0) dx → 0 as n → ∞. N R Hence, using hL0 (u ), u i = o (1), we get ku k2 = o (1), that is u → 0 in Y as µ n n n n Yµ n n µ n → ∞.

Now we prove the following existence result for (25). Theorem 3.5. Let µ > −m2s. Then (25) has a positive ground state solution.

Proof. Since Lµ has a mountain pass geometry [3], we can ﬁnd a Palais-Smale sequence (un) ⊂ Yµ at the level dµ. Thus (un) is bounded in Yµ and there exists u ∈ Yµ such that un * u in Yµ. Using the growth assumptions on f and the density ∞ N+1 s N+1 0 of Cc (R+ ) in X (R+ ), it is easy to check that hLµ(u), ϕi = 0 for all ϕ ∈ Yµ. If u = 0, then un 6→ 0 in Yµ because dµ > 0. Hence we can use Lemma 3.4 to N deduce that for some sequence (zn) ⊂ R , vn(x, y) := un(x + zn, y) is a bounded Palais-Smale sequence at the level dµ and having a nontrivial weak limit v. Hence, v ∈ Mµ. Moreover, using the weak lower semicontinuity of k · kYµ ,(f4) and Fatou’s lemma, it is easy to see that Lµ(v) = dµ. When u 6= 0, as before, we can deduce that u is a ground state solution to (25). In conclusion, for µ > −m2s, there exists a ground state solution w = wµ ∈ Yµ \{0} such that 0 Lµ(w) = dµ and Lµ(w) = 0.

0 − N+1 Since f(t) = 0 for t ≤ 0, it follows from hLµ(w), w i = 0 that w ≥ 0 in R+ and w 6≡ 0. A Moser iteration argument (see Lemma 4.1 below) shows that w(·, 0) ∈ q N ∞ N+1 L (R ) for all q ∈ [2, ∞], and that w ∈ L (R+ ). Using Proposition1-( iii), we 0,α N+1 obtain that w ∈ C (R+ ) for some α ∈ (0, 1). By Proposition1-( ii), we conclude that w is positive.

In the next lemma we establish an important connection between cε and dV (0) = 2s d−V0 (we remark that V (0) = −V0 > −m ):

Lemma 3.6. The numbers cε and dV (0) verify the following inequality:

lim sup cε ≤ dV (0). ε→0 Proof. By Theorem 3.5, we know that there exists a positive ground state solution w ∞ to (25) with µ = V (0). Let η ∈ Cc (R) be such that 0 ≤ η ≤ 1, η = 1 in [−1, 1] and η = 0 in R \ (−2, 2). Suppose that B2 ⊂ Λ. Deﬁne wε(x, y) := η(ε |(x, y)|)w(x, y) s N+1 and note that supp(wε(·, 0)) ⊂ Λε. It is easy to prove that wε → w in X (R+ ) and that LV (0)(wε) → LV (0)(w) as ε → 0. On the other hand, by deﬁnition of cε, we have 2 Z tε 2 cε ≤ max Jε(twε) = Jε(tεwε) = kwεkε − F (tεwε(x, 0)) dx (26) t≥0 2 N R 16 VINCENZO AMBROSIO for some tε > 0. Recalling that w ∈ MV (0) and using (f4), it is readily seen that tε → 1 as ε → 0. Note that 2 Z tε 2 Jε(tεwε) = LV (0)(tεwε) + (Vε(x) − V (0))wε (x, 0) dx. (27) 2 N R Since Vε(x) is bounded on the support of wε(·, 0), and Vε(x) → V (0) as ε → 0, we can apply the dominated convergence theorem and use (26) and (27) to conclude the proof.

0 0 Remark 1. Under assumptions (V1 )-(V2 ), we have that limε→0 cε = dV (0).

Now we come back to study (13) and consider the mountain pass solutions uε satisfying (24).

∗ N Lemma 3.7. There exist r, β, ε > 0 and (yε) ⊂ R such that Z 2 ∗ uε(x, 0) dx ≥ β, for all ε ∈ (0, ε ). Br (yε)

Proof. Since uε veriﬁes (24), it follows from the growth assumptions on g that there exist α > 0, independent of ε > 0, such that 2 kuεkε ≥ α for all ε > 0. (28)

Let (εn) ⊂ (0, ∞) be such that εn → 0. If by contradiction there exists r > 0 such that Z lim sup u2 (x, 0) dx = 0, εn n→∞ N y∈R Br (y) q N ∗ thus we can use Lemma 3.3 to deduce that uεn (·, 0) → 0 in L (R ) for all q ∈ (2, 2s).

Then, (24) and the growth assumptions on g imply that kuεn kεn → 0, as n → ∞, and this contradicts (28).

N Lemma 3.8. For any εn → 0, consider the sequence (yεn ) ⊂ R given in Lemma

3.7 and wn(x, y) := uεn (x + yεn , y). Then there exists a subsequence of (wn), still s N+1 denoted by itself, and w ∈ X (R+ ) \{0} such that s N+1 wn → w in X (R+ ).

Moreover, there exists x0 ∈ Λ such that

εn yεn → x0 and V (x0) = −V0.

Proof. In what follows, we denote by (yn) and (un), the sequences (yεn ) and (uεn ), respectively. Since each un satisﬁes (24), we can argue as in the proof of Lemma 3.2 s N+1 and use Lemma 3.6 and (11) to deduce that (un) is bounded in X (R+ ). Thus s N+1 s N+1 (wn) is bounded in X (R+ ) and there exists w ∈ X (R+ ) \{0} such that s N+1 wn * w in X (R+ ) as n → ∞, (29) and, by Lemma 3.7, Z w2(x, 0) dx ≥ β > 0. (30) Br N Next we show that (εn yn) is bounded in R . First of all, we prove that

dist(εn yn, Λ) → 0 as n → ∞. (31) THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 17

If (31) does not hold, there exists δ > 0 and a subsequence of (εn yn), still denoted by itself, such that dist(εn yn, Λ) ≥ δ for all n ∈ N. c Then there is R > 0 such that BR(εn yn) ⊂ Λ for all n ∈ N. By the deﬁnition s N+1 of X (R+ ) and using the fact that w ≥ 0, we know that there exists (ψj) ⊂ s N+1 N+1 X (R+ ) such that ψj ≥ 0, ψj has compact support in R+ and ψj → w in s N+1 0 X (R+ ) as j → ∞. Fix j ∈ N. Taking ψj as test function in hJε(un), φi = 0 for φ ∈ Xε, we get ZZ Z 1−2s 2 y (∇wn · ∇ψj + m wnψj ) dxdy + V (εn x + εn yn)wn(x, 0)ψj (x, 0) dx N+1 N R+ R Z = g(εn x + εn yn, wn(x, 0))ψj (x, 0) dx. (32) N R On the other hand, by the deﬁnition of gε and (g3), there holds Z g(εn x + εn yn, wn(x, 0))ψj (x, 0) dx N R Z Z = g(εn x + εn yn, wn(x, 0))ψj (x, 0) dx + g(εn x + εn yn, wn(x, 0))ψj (x, 0) dx c B R B R εn εn Z Z V1 ≤ wn(x, 0)ψj (x, 0) dx + f(wn(x, 0))ψj (x, 0) dx. κ c B R B R εn εn

Then, using (V1) and (32), we can see that ZZ Z 1−2s 2 1 y (∇wn · ∇ψj + m wnψj) dxdy − V1 1 + wn(x, 0)ψj(x, 0) dx N+1 κ N R+ R Z ≤ f(wn(x, 0))ψj(x, 0) dx. c B R εn

Taking into account that ψj has compact support, εn → 0, the growth assumptions on f, and (29), we deduce that, as n → ∞, Z f(wn(x, 0))ψj(x, 0) dx → 0, c B R εn and ZZ Z 1−2s 2 1 y (∇wn · ∇ψj + m wnψj) dxdy − V1 1 + wn(x, 0)ψj(x, 0) dx N+1 κ N R+ R ZZ Z 1−2s 2 1 → y (∇w · ∇ψj + m wψj) dxdy − V1 1 + w(x, 0)ψj(x, 0) dx. N+1 κ N R+ R The previous relations of limits give ZZ Z 1−2s 2 1 y (∇w · ∇ψj + m wψj) dxdy − V1 1 + w(x, 0)ψj(x, 0) dx ≤ 0, N+1 κ N R+ R and passing to the limit as j → ∞ we ﬁnd 2 1 2 kwk N+1 − V 1 + |w(·, 0)| ≤ 0. Xs( ) 1 2 R+ κ 18 VINCENZO AMBROSIO

V1 Thus (8) and κ > 2s yield m −V1

V1 1 2 0 ≤ 1 − 1 + kwk N+1 ≤ 0, 2s Xs( ) m κ R+ which implies w ≡ 0 in RN and this is in contrast with (30). Consequently, there exist a subsequence of (εn yn), still denoted by itself, and x0 ∈ Λ such that εn yn → x0 as n → ∞. Next we prove that x0 ∈ Λ. Using (g2) and (32), we know that

ZZ Z 1−2s 2 y (∇wn · ∇ψj + m wnψj ) dxdy + V (εn x + εn yn)wn(x, 0)ψj (x, 0) dx N+1 N R+ R Z ≤ f(wn(x, 0))ψj (x, 0) dx. N R

Letting n → ∞ and using (29) and the continuity of V , we ﬁnd

ZZ Z 1−2s 2 y (∇w · ∇ψj + m wψj) dxdy + V (x0)w(x, 0)ψj(x, 0) dx N+1 N R+ R Z ≤ f(w(x, 0))ψj(x, 0) dx. N R

By passing to the limit as j → ∞, we obtain

ZZ Z Z 1−2s 2 2 2 2 y (|∇w| + m w ) dxdy + V (x0)w (x, 0) dx ≤ f(w(x, 0))w(x, 0) dx. N+1 N N R+ R R

Hence there exists t1 ∈ (0, 1) such that t1w ∈ MV (x0). In view of Lemma 3.6, we have

dV (x ) ≤ LV (x )(t1w) ≤ lim inf Jε (un) = lim inf cε ≤ dV (0) 0 0 n→∞ n n→∞ n

from which dV (x0) ≤ dV (0) and thus V (x0) ≤ V (0) = −V0. Since −V0 = infx∈Λ V (x), we achieve V (x0) = −V0. Using (V2), we conclude that x0 ∈ Λ. s N+1 Finally, we show that wn → w in X (R+ ) as n → ∞. For all n ∈ N and x ∈ RN , deﬁne

Λ − εn y˜n Λ˜ n := , εn and

˜ 1 1 if x ∈ Λn, χñ(x) := ˜ c 0 if x ∈ Λn, 2 1 χñ(x) := 1 − χñ(x). THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 19

Let us also consider the following functions for x ∈ RN and n ∈ N: 1 1 h1 (x) := − (V (ε x + ε y ) + V )w2 (x, 0)˜χ1 (x), n 2 θ n n n 1 n n 1 1 h1(x) := − (V (x ) + V )w2(x, 0), 2 θ 0 1 1 1 h2 (x) := − (V (ε x + ε y ) + V )w2 (x, 0) n 2 θ n n n 1 n 1 + g(ε x + ε y , w (x, 0))w (x, 0) − G(ε x + ε y , w (x, 0)) χ˜2 (x) θ n n n n n n n n n n 1 1 1 ≥ − − (V (ε x + ε y˜ ) + V )w2 (x, 0)˜χ2 (x), 2 θ 2κ n n n 1 n n 1 h3 (x) := g(ε x + ε y , w (x, 0))w (x, 0) − G(ε x + ε y , w (x, 0)) χ˜1 (x) n θ n n n n n n n n n n 1 = (f(w (x, 0))w (x, 0) − F (w (x, 0))) χ˜1 (x), θ n n n n 1 h3(x) := (f(w(x, 0))w(x, 0) − F (w(x, 0))) . θ

From (f3), (g3), (V1) and our choice of κ, we see that the above functions are nonnegative in RN . Since N wn(x, 0) → w(x, 0) a.e. x ∈ R , εn yn → x0 ∈ Λ, as n → ∞, we get 1 1 1 2 3 3 N χ˜n(x) → 1, hn(x) → h (x), hn(x) → 0 and hn(x) → h (x) a.e. x ∈ R . 2 2 Hence, observing that k · k s N+1 − V1| · |2 is weakly lower semicontinuous, and X (R+ ) using Fatou’s lemma and the invariance of RN by translation, we have 1 0 dV (0) ≥ lim sup cεn = lim sup Jεn (un) − hJεn (un), uni n→∞ n→∞ θ Z 1 1 2 2 1 2 3 ≥ lim sup − kwnkXs( N+1) − V1|wn(·, 0)|2 + (hn + hn + hn) dx 2 θ R+ N n→∞ R Z 1 1 2 2 1 2 3 ≥ lim inf − kwnkXs( N+1) − V1|wn(·, 0)|2 + (hn + hn + hn) dx n→∞ 2 θ R+ N R Z 1 1 2 2 1 3 ≥ − kwkXs( N+1) − V1|w(·, 0)|2 + (h + h ) dx = dV (0). 2 θ R+ N R Accordingly, 2 2 2 2 lim kwnk s N+1 − V1|wn(·, 0)|2 = kwk s N+1 − V1|w(·, 0)|2, (33) n→∞ X (R+ ) X (R+ ) and 1 1 2 3 3 1 N hn → h , hn → 0 and hn → h in L (R ). Then, Z Z 2 2 lim (V (εn x + εn yn) + V1)wn(x, 0) dx = (V (x0) + V1)w (x, 0) dx, n→∞ N N R R 20 VINCENZO AMBROSIO which implies that

2 2 lim |wn(·, 0)|2 = |w(·, 0)|2. (34) n→∞

s N+1 Putting together (33) and (34), and using the fact that X (R+ ) is a Hilbert space, we attain

kwn − wk s N+1 → 0 as n → ∞. X (R+ ) This ends the proof of the lemma.

4. The proof of Theorem 1.1. In this last section we give the proof of Theorem 1.1. We start by proving the following lemma which will be crucial to study the behavior of maximum points of the solutions. The proof is based on a variant of the Moser iteration argument [39].

Lemma 4.1. Let (wn) be the sequence deﬁned as in Lemma 3.8. Then, wn(·, 0) ∈ L∞(RN ) and there exists C > 0 such that

|wn(·, 0)|∞ ≤ C for all n ∈ N.

∞ N+1 Moreover, wn ∈ L (R+ ) and there exists K > 0 such that

kwnk ∞ N+1 ≤ K for all n ∈ . L (R+ ) N

Proof. We note that wn is a weak solution to

1−2s 2 1−2s N+1 − div(y ∇wn) + m y wn = 0 in R+ , ∂wn N ∂ν1−2s = −V (εn x + εn yn)wn(·, 0) + g(εn x + εn yn, wn(·, 0)) on R . (35)

2β For each n ∈ N and L > 0, let wn,L := min{wn,L} and zn,L := wnwn,L , where β > 0 will be chosen later. Taking zL,n as test function in the weak formulation of (35), we deduce that ZZ ZZ 1−2s 2β 2 2 2 1−2s 2β 2 y wn,L(|∇wn| + m wn) dxdy + 2βy wn,L|∇wn| dxdy N+1 R+ Dn,L Z 2 2β = − V (εn x + εn yn)wn(x, 0)wn,L(x, 0) dx N ZR 2β + g(εn x + εn yn, wn(x, 0))wn(x, 0)wn,L(x, 0) dx, (36) N R N+1 where Dn,L := {(x, y) ∈ R+ : wn(x, y) ≤ L}. It is easy to check that ZZ ZZ 1−2s β 2 1−2s 2β 2 y |∇(wnwn,L)| dxdy = y wn,L|∇wn| dxdy N+1 N+1 R+ R+ ZZ 2 1−2s 2β 2 + (2β + β )y wn,L|∇wn| dxdy. (37) Dn,L THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 21

Then, putting together (36), (37), (V1), (f1)-(f2), (g1)-(g2), we get ZZ β 2 1−2s β 2 2 2 2β kwnwn,Lk s N+1 = y (|∇(wnwn,L)| + m wnwn,L) dxdy X (R+ ) N+1 R+ ZZ 1−2s 2β 2 2 2 = y wn,L(|∇wn| + m wn) dxdy N+1 R+ ZZ β + 2β 1 + y1−2sw2β |∇w |2 dxdy 2 n,L n Dn,L "ZZ ZZ # 1−2s 2β 2 2 2 1−2s 2β 2 ≤ cβ y wn,L(|∇wn| + m wn) dxdy + 2βy wn,L|∇wn| dxdy N+1 R+ Dn,L Z 2 2β = cβ − V (εn x + εn yn)wn(x, 0)wn,L(x, 0) dx N R Z 2β + g(εn x + εn yn, wn(x, 0))wn(x, 0)wn,L(x, 0) dx N R Z 2 2β p+1 2β ≤ cβ (V1 + 1)wn(x, 0)wn,L(x, 0) + C1wn (x, 0)wn,L(x, 0) dx (38) N R where β c := 1 + > 0. β 2 Now, we prove that there exist a constant c > 0 independent of n, L, β, and N/2s N hn ∈ L (R ), hn ≥ 0 and independent of L and β, such that 2 2β p+1 2β 2 2β N (V1 + 1)wn(·, 0)wn,L(·, 0) + C1wn (·, 0)wn,L(·, 0) ≤ (c + hn)wn(·, 0)wn,L(·, 0) on R . (39) Firstly, we notice that 2 2β p+1 2β (V1 + 1)wn(·, 0)wn,L(·, 0) + C1wn (·, 0)wn,L(·, 0) 2 2β p−1 2 2β N ≤ (V1 + 1)wn(·, 0)wn,L(·, 0) + C1wn (·, 0)wn(·, 0)wn,L(·, 0) on R . Moreover, p−1 N wn (·, 0) ≤ 1 + hn on R , (40) p−1 N/2s N where hn := χ{wn(·,0)>1}wn (·, 0) ∈ L (R ). In fact, we can observe that p−1 p−1 p−1 wn (·, 0) = χ{wn(·,0)≤1}wn (·, 0) + χ{wn(·,0)>1}wn (·, 0) p−1 N ≤ 1 + χ{wn(·,0)>1}wn (·, 0) on R . N s N If (p − 1) 2s < 2 then, recalling that (wn(·, 0)) is bounded in H (R ), we have

Z N Z 2s (p−1) 2 χ{wn(·,0)>1}(wn(x, 0)) dx ≤ χ{wn(·,0)>1}wn(x, 0) dx ≤ C, for all n ∈ N. N N R R N N ∗ If 2 ≤ (p − 1) 2s , we deduce that (p − 1) 2s ∈ [2, 2s], and by Theorem 2.1 and the s N+1 boundedness of (wn) in X (R+ ), we ﬁnd

Z N N (p−1) 2s (p−1) 2s χ{wn(·,0)>1}(wn(x, 0)) dx ≤ Ckwnk s N+1 ≤ C, N X (R+ ) R for some C > 0 depending only on N, s and p. Hence, (40) holds. Taking into account (38), (39), and (40), we obtain that Z β 2 2 2β kw w k N+1 ≤ c (c + h (x))w (x, 0)w (x, 0) dx, n n,L Xs( ) β n n n,L R+ N R 22 VINCENZO AMBROSIO and, by applying Fatou’s lemma and the monotone convergence theorem, we can pass to the limit as L → ∞ to get Z Z β+1 2 2(β+1) 2(β+1) kw k N+1 ≤ cc w (x, 0) dx + c h (x)w (x, 0) dx. (41) n Xs( ) β n β n n R+ N N R R

Fix M > 1 and let Ω1,n := {hn ≤ M} and Ω2,n := {hn > M}. Then, by using the H¨olderinequality, Z 2(β+1) hn(x)wn (x, 0) dx N R Z Z 2(β+1) 2(β+1) = hn(x)wn (x, 0) dx + hn(x)wn (x, 0) dx Ω1,n Ω2,n β+1 2 β+1 2 ≤ M|w (·, 0)| + (M)|w (·, 0)| ∗ , (42) n 2 n 2s where

2s Z ! N N/2s (M) := sup hn dx → 0 as M → ∞, n∈N Ω2,n

s N due to the fact that wn(·, 0) → w(·, 0) in H (R ). In view of (41) and (42), we have

β+1 2 β+1 2 β+1 2 kwn k s N+1 ≤ cβ(c + M)|wn (·, 0)|2 + cβ(M)|wn (·, 0)|2∗ . (43) X (R+ ) s We note that Theorem 2.1 yields

β+1 2 2 β+1 2 |wn (·, 0)|2∗ ≤ C2∗ kwn k s N+1 . (44) s s X (R+ ) Then, choosing M large so that

2 1 (M)cβC2∗ < , s 2 and using (43) and (44), we obtain that

β+1 2 2 β+1 2 |w (·, 0)| ∗ ≤ 2C ∗ cβ(c + M)|w (·, 0)| . (45) n 2s 2s n 2

2∗ N Then we can start a bootstrap argument: since w (·, 0) ∈ L s ( ) and |w (·, 0)| ∗ ≤ n R n 2s N C for all n ∈ N, we can apply (45) with β1 + 1 = N−2s to deduce that wn(·, 0) ∈ 2 ∗ 2N (β +1)2 N 2 N L 1 s (R ) = L (N−2s) (R ). Applying again (45), after k iterations, we ﬁnd 2Nk k N q N wn(·, 0) ∈ L (N−2s) (R ), and so wn(·, 0) ∈ L (R ) for all q ∈ [2, ∞) and |wn(·, 0)|q ≤ C for all n ∈ N. ∞ N q N Now we prove that actually wn(·, 0) ∈ L (R ). Since wn(·, 0) ∈ L (R ) for all N N q ∈ [2, ∞), we have that hn ∈ L s ( ) and |hn| N ≤ D for all n ∈ . Then, by the R s N generalized H¨olderinequality and Young’s inequality with λ > 0, we can see that for all λ > 0 Z 2(β+1) β+1 β+1 h (x)w (x, 0) dx ≤ |h | N |w (·, 0)| |w (·, 0)| ∗ n n n n 2 n 2s N s R β+1 2 1 β+1 2 ≤ D λ|wn (·, 0)|2 + |wn (·, 0)|2∗ . λ s THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 23

Consequently, using (41) and (44), we deduce that β+1 2 2 β+1 2 |wn (·, 0)|2∗ ≤ C2∗ kwn k s N+1 s s X (R+ )

2 β+1 2 2 cβD β+1 2 ≤ cβC2∗ (c + Dλ)|wn (·, 0)|2 + C2∗ |wn (·, 0)|2∗ . (46) s s λ s Taking λ > 0 such that 2 cβDC2∗ 1 s = , λ 2 we obtain that β+1 2 2 β+1 2 β+1 2 |w (·, 0)| ∗ ≤ 2cβ(c + Dλ)C ∗ |w (·, 0)| = Mβ|w (·, 0)| , n 2s 2s n 2 n 2 where 2 Mβ := 2cβ(c + Dλ)C ∗ . 2s

Now we can control the dependence on β of Mβ as follows: √ 2 2 2 2 β+1 Mβ ≤ Ccβ ≤ C(1 + β) ≤ M0 e , for some M0 > 0 independent of β. Consequently,

1 1 β+1 √ |w (·, 0)| ∗ ≤ M e β+1 |w (·, 0)| . n 2s (β+1) 0 n 2(β+1)

As before, iterating this last relation and choosing β0 = 0 and 2(βj+1 + 1) = ∗ 2s(βj + 1) we have that Pj 1 Pj √ 1 i=0 βi+1 i=0 β +1 |w (·, 0)| ∗ ≤ M e i |w (·, 0)| . n 2s (βj +1) 0 n 2(β0+1) We note that N j β = − 1, (47) j N − 2s so the series ∞ ∞ X 1 X 1 and √ β + 1 β + 1 i=0 i i=0 i are convergent. Recalling that |wn(·, 0)|q ≤ C for all n ∈ N and q ∈ [2, ∞), we get

|wn(·, 0)|∞ = lim |wn(·, 0)|2∗(β +1) ≤ M for all n ∈ . j→∞ s j N This proves the L∞-desired estimate for the trace. At this point, we prove that there exists R > 0 such that

kwnk ∞ N+1 ≤ R for all n ∈ . (48) L (R+ ) N

Using (46) with λ = 1 and that |wn(·, 0)|q ≤ C for all q ∈ [2, ∞], we deduce that β+1 2 2(β+1) kwn k s N+1 ≤ cc˜ βC for all n ∈ N, X (R+ ) for somec, ˜ C > 0 independent on β and n. On the other hand, from (6), we obtain that β+1 ZZ ! 2γ(β+1) 1−2s 2γ(β+1) β+1 y wn dxdy = kwn k 2γ N+1 1−2s N+1 L (R+ ,y ) R+ β+1 ≤ C∗kwn k s N+1 X (R+ ) 24 VINCENZO AMBROSIO which combined with the previous inequality yields

1 0 ˜ 2(β+1) kwnk 2γ(β+1) N+1 1−2s ≤ C (C∗cβ) for all n ∈ . L (R+ ,y ) N Since 1 β β 1 log 1 + ≤ ≤ for all β > 0, β + 1 2 2(β + 1) 2

0 1 we can see that there exists C¯ > 0 such that C (C˜∗cβ) 2(β+1) ≤ C¯ for all β > 0, and so ¯ kwnk 2γ(β+1) N+1 1−2s ≤ C for all n ∈ , β > 0. L (R+ ,y ) N ¯ N+1 Now, ﬁx R > C and deﬁne Σn := {(x, y) ∈ R+ : wn(x, y) > R}. Hence, for all n, j ∈ N, we get 1 ZZ ! 2γ(βj +1) ¯ 1−2s 2γ(βj +1) C ≥ y wn dxdy N+1 R+ 1 γ(β +1)−1 j ZZ 2γ(βj +1) γ(βj +1) 1−2s 2 ≥ R y wn dxdy , Σn which yields β +1− 1 1 ¯ j γ ZZ 2γ C 1 1−2s 2 ≥ 1 y wn dxdy , γ R C¯ Σn where βj is given in (47). Letting j → ∞, we have that βj → ∞ and then ZZ 1−2s 2 y wn dxdy = 0 for all n ∈ N, Σn which implies that |Σn| = 0 for all n ∈ N. Consequently, (48) holds true.

Lemma 4.2. The sequence (wn) satisﬁes wn(·, 0) → 0 as |x| → ∞ uniformly in n ∈ N.

Proof. By Lemma 4.1 and Proposition1-( iii), we obtain that each wn is continuous N+1 q N in R+ . Note that, by Lemma 4.1, wn(·, 0) → w(·, 0) in L (R ) for all q ∈ [2, ∞). s N+1 On the other hand, from (6) and wn → w in X (R+ ), we have that wn → w 2γ N+1 1−2s N in L (R+ , y ). Fixx ¯ ∈ R . Using (V1) and (14), we see that wn is a weak subsolution to 1−2s 2 1−2s −div(y ∇wn) + m y wn = 0 in Q1(¯x, 0) := B1(¯x) × (0, 1), ∗ ∂wn 2s −1 ∂ν1−2s = (V1 + η)wn(·, 0) + Cηwn (·, 0) on B1(¯x), 2s where η ∈ (0, m − V1) is ﬁxed. Applying Proposition1-( i) and observing that L2γ (A, y1−2s) ⊂ L2(A, y1−2s) for any bounded set A ⊂ RN , we get ∗ 2s −1 2γ 1−2s q 0 ≤ sup wn ≤ C(kwnkL (Q1(¯x,0),y ) + |wn (·, 0)|L (B1(¯x))) for all n ∈ N, Q1(¯x,0)

N where q > 2s is ﬁxed and C > 0 is a constant depending only on N, m, s, q, γ and ∗ independent of n ∈ N andx ¯. Note that q(2s − 1) ∈ (2, ∞) because N > 2s and N 2γ N+1 1−2s q > 2s . Using the strong convergence of (wn) in L (R+ , y ) and (wn(·, 0)) in q N L (R ), respectively, we infer that wn(¯x, 0) → 0 as |x¯| → ∞ uniformly in n ∈ N. THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 25

Remark 2. The proofs of Lemmas 4.1 and 4.2 can be also adapted to the case m = 0. In this way, we can give an alternative proof of Lemma 2.6 in [2] which, on −1 2s −1 the contrary, is based on some properties of the kernel Ks(x) = F ((|k| +1) )(x) proved in [26]. The next comparison principle for (−∆ + m2)s in general domains will be very useful later.

Theorem 4.3. (Comparison principle) Let Ω ⊂ RN be an open set, γ < m2s, s N N 2 s u1, u2 ∈ H (R ) be such that u1 ≤ u2 in R \ Ω and (−∆ + m ) u1 − γu1 ≤ 2 s (−∆ + m ) u2 − γu2 in Ω, that is Z 2s (m − γ) u1(x)v(x) dx N R ZZ N+2s (u1(x) − u1(y))(v(x) − v(y)) 2 + C(N, s)m N+2s K N+2s (m|x − y|) dxdy N 2 R |x − y| 2 Z 2s ≤ (m − γ) u2(x)v(x) dx N R ZZ N+2s (u2(x) − u2(y))(v(x) − v(y)) 2 + C(N, s)m N+2s K N+2s (m|x − y|) dxdy N 2 R |x − y| 2 (49)

s N N N for all v ∈ H (R ) such that v ≥ 0 in R and v = 0 in R \ Ω. Then, u1 ≤ u2 in RN . + Proof. Set w := u1 − u2 and take v = w as test function in (49). Then, observing that + + 2 + + |x − y | ≤ (x − y)(x − y ) for all x, y ∈ R, we have 2 Z N+2s ZZ |v(x) − v(y)| 2s 2 2 (m − γ) v (x) dx + C(N, s)m N+2s K N+2s (m|x − y|) dxdy N N 2 R R |x − y| 2 Z ≤ (m2s − γ) w(x)v(x) dx N R N+2s ZZ (w(x) − w(y))(v(x) − v(y)) 2 + C(N, s)m N+2s K N+2s (m|x − y|) dxdy N 2 R |x − y| 2 ≤ 0 from which v = 0 in RN and thus we get the thesis. Now we have all tools to give the proof of our ﬁrst main result of this work.

Proof of Theorem 1.1. We begin by proving that there exists ε0 > 0 such that, for any ε ∈ (0, ε0) and any mountain pass solution uε ∈ Xε of (13), it holds

|u (·, 0)| ∞ c < a. (50) ε L (Λε)

Assume by contradiction that for some subsequence (εn) such that εn → 0, we can ﬁnd u := u ∈ X such that J (u ) = c , J 0 (u ) = 0 and n εn εn εn n εn εn n

|un(·, 0)|L∞(Λc ) ≥ a. (51) εn N In view of Lemma 3.8, we can ﬁnd (yn) ⊂ R such that wn(x, y) := un(x+yn, y) → s N+1 w in X (R+ ) and εn yn → x0 for some x0 ∈ Λ such that V (x0) = −V0. 26 VINCENZO AMBROSIO

Now, if we choose r > 0 such that Br(x0) ⊂ B2r(x0) ⊂ Λ, we can see that x0 B r ( ) ⊂ Λε . Then, for any x ∈ B r (yn) it holds εn εn n εn

x0 x0 1 2r x − ≤ |x − yn| + yn − < (r + on(1)) < for n suﬃciently large. εn εn εn εn Therefore, c c Λε ⊂ B r (yn) (52) n εn for any n big enough. Using Lemma 4.2, we see that

wn(x, 0) → 0 as |x| → ∞ uniformly in n ∈ N. (53) Therefore, there exists R > 0 such that

wn(x, 0) < a for any |x| ≥ R, n ∈ N. c Hence, un(x, 0) = wn(x − yn, 0) < a for any x ∈ BR(yn) and n ∈ N. On the other hand, by (52), there exists n0 ∈ N such that for any n ≥ n0 we have c c c Λε ⊂ B r (yn) ⊂ BR(yn), n εn which implies that u (x, 0) < a for any x ∈ Λc and n ≥ n . This is impossible n εn 0 according to (51). Since uε ∈ Xε satisﬁes (50), by the deﬁnition of g it follows that uε is a solution of (10) for ε ∈ (0, ε0). From Proposition1-( ii), we conclude that N uε(·, 0) > 0 in R . In what follows, we study the behavior of the maximum points of solutions to problem (1). Take εn → 0 and let (un) ⊂ Xεn be a sequence of solutions to (13) as above. Consider the translated sequence wn(x, y) := un(x + yn, y), where (yn) is given by Lemma 3.8. Let us prove that there exists δ > 0 such that

|wn(·, 0)|∞ ≥ δ for all n ∈ N. (54)

Assume by contradiction that |wn(·, 0)|∞ → 0 as n → ∞. Using (f1), we can ﬁnd ν ∈ N such that f(|w (·, 0)| ) V n ∞ < 1 for all n ≥ ν. |wn(·, 0)|∞ κ From hJ 0 (u ), u i = 0, (g ) and (f ), we can see that for all n ≥ ν εn n n 2 4 2 2 2 2 kwnk s N+1 − V1|wn(·, 0)|2 = kunk s N+1 − V1|un(·, 0)|2 X (R+ ) X (R+ ) Z ≤ f(un(x, 0))un(x, 0) dx N R Z f(|wn(·, 0)|∞) 2 ≤ wn(x, 0) dx N |w (·, 0)| R n ∞ Z V1 2 ≤ wn(x, 0) dx κ N R which combined with (8) yields V1 1 2 1 − 1 + kw k N+1 ≤ 0. 2s n Xs( ) m κ R+

V1 Since κ > 2s , we get kwnk s N+1 = 0 for all n ≥ ν, which is a contradiction. m −V1 X (R+ ) Therefore, if qn is a global maximum point of wn(·, 0), we deduce from Lemma 4.1 and (54) that there exists R0 > 0 such that |qn| < R0 for all n ∈ N. Thus THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 27 xn := qn + yn is a global maximum point of un(·, 0), and εn xn → x0 ∈ M. This fact combined with the continuity of V yields

lim V (εn xn) = V (x0) = −V0. n→∞

Finally, we prove a decay estimate for un(·, 0). Using (f1), the definition of g and (53), we can find R1 > 2 sufficiently large such that 2 g(εn x + εn yn, wn(x, 0))wn(x, 0) ≤ δwn(x, 0) for |x| > R1, (55)

2s N where δ ∈ (0, m − V1) is fixed. Pick a smooth cut-off function φ defined in R such that 0 ≤ φ ≤ 1, φ(x) = 0 for |x| ≥ 1, and φ 6≡ 0. By Riesz representation theorem, there exists a unique functionw ¯ ∈ Hs(RN ) such that 2 s N (−∆ + m ) w¯ − (V1 + δ)w ¯ = φ in R . (56)

Sincew ¯ = B2s,m ∗φ, for some positive kernel B2s,m whose expression is given below, we can see thatw ¯ ≥ 0 in RN andw ¯ 6≡ 0. Denote by W the extension ofw ¯, namely N W (x, y) := (Ps,m(·, y) ∗ w¯)(x). Fix x ∈ R . Then, from Young’s inequality and (9), we can see that kW k 2 1−2s ≤ kW k 2 N 1−2s = kP (·, y) ∗ w¯k 2 N 1−2s L (B1(x)×[0,1],y ) L (R ×[0,1],y ) s,m L (R ×[0,1],y ) 1 Z 1 2 2 1−2s ≤ |Ps,m(·, y) ∗ w¯|2 y dy 0 1 Z 1 2 2 2 1−2s ≤ |Ps,m(·, y)|1|w¯|2 y dy 0 1 Z 1 2 2 1−2s ≤ |w¯|2 ϑ (my)y dy 0 ≤ cs,m|w¯|2, for some constant cs,m > 0. Therefore, by Proposition1-( i), we deduce thatw ¯ ∈ L∞(RN ), and thus, by interpolation,w ¯ ∈ Lq(RN ) for any q ∈ [2, ∞]. Hence, 2 s ∞ N (−∆ + m ) w¯ = (V1 + δ)w ¯ + φ ∈ L (R ), and applying Corollary3 (see also Theorem 6.2-(v) and Proposition1-( iii)) we obtain thatw ¯ is H¨oldercontinuous in RN . Using Proposition1-( ii), we have thatw ¯ > 0 in RN . Moreover, we can prove that there exist c, C > 0 such that

−c|x| N 0 < w¯(x) ≤ Ce for all x ∈ R . (57) Assume for the moment that (57) holds and we postpone the proof of it after showing how (57) yields the desired decay estimate for un(·, 0). We know that 2 s N (−∆ + m ) w¯ − (V1 + δ)w ¯ = 0 in R \ BR1 . (58)

On the other hand, by (V1) and (55), we can see that 2 s N (−∆ + m ) wn(·, 0) − (V1 + δ)wn(·, 0) ≤ 0 in R \ BR1 . (59)

Set b := min w¯(x) > 0 and zn := (`+1)w ¯−bwn(·, 0), where ` := supn∈ |wn(·, 0)|∞. x∈BR1 N

We note that zn ≥ 0 in BR1 and that 2 s N (−∆ + m ) zn − (V1 + δ)zn ≥ 0 in R \ BR1 . 28 VINCENZO AMBROSIO

2s N Since V1 + δ < m , we can apply Theorem 4.3 with Ω = R \ BR1 to deduce that N zn ≥ 0 in R . In the light of (57), we obtain that there exist c, C > 0 such that −c|x| N 0 ≤ wn(x) ≤ Ce for all x ∈ R , n ∈ N, which combined with un(x, 0) = wn(x − yn, 0), xn = qn + yn and |qn| < R yields

0 −c|x−xn| N un(x, 0) = wn(x − yn, 0) ≤ C e for all x ∈ R , n ∈ N. In what follows, we focus our attention on the estimate (57). We recall thatw ¯ = B2s,m ∗ φ, where − N −1 2 2 s −1 B2s,m(x) := (2π) 2 F ([(|k| + m ) − (V1 + δ)] )(x). Since φ has compact support, the exponential decay ofw ¯ at inﬁnity follows if we show the exponential decay of B2s,m(x) for big values of |x|. After that, due to the fact thatw ¯ is continuous in RN , we can deduce the exponential decay ofw ¯ in the N whole of R . Next we prove the exponential decay of B2s,m(x) for |x| large. Then, Z 1 ık·x 1 B2s,m(x) = N e 2 2 s dk (2π) N [(|k| + m ) − (V + δ)] R 1 ∞ 1 Z Z 2 2 s ık·x −t[(|k| +m ) −(V1+δ)] = N e e dt dk (2π) N R 0 Z ∞ Z −γt 1 ık·x −t[(|k|2+m2)s−m2s] = e N e e dk dt (2π) N 0 R Z ∞ −γt = e ps,m(x, t) dt, (60) 0 where 2s γ := m − (V1 + δ) > 0, and ∞ 2 2s Z 1 |x| 2 m t − 4z −m z ps,m(x, t) := e N e e ϑs(t, z) dz 0 (4πz) 2 is the 2s-stable relativistic density with parameter m (see pag. 4 formula (7) in [41], and pag. 4875 formula (2.12) and Lemma 2.2 in [11]), and ϑs(t, z) is the density s function of the s-stable process whose Laplace transform is e−tλ . Using the scaling N 2s property ps,m(x, t) = m ps,1(mx, m t) (see pag. 4876 formula (2.15) in [11]) and Lemma 2.2 in [31], we can see that for some C > 0 (depending only on N, s, m) mx 1 mx N ps,m(x, t) ≤ C gm2st √ + tν √ for all x ∈ R , t > 0, (61) 2 2 where 1 |x|2 − 4t gt(x) := N e , (4πt) 2 and νm is the L´evymeasure of relativistic process with parameter m > 0 given by

2s−N N+2s 2 m 2s2 2 m ν (x) := N K N+2s (m|x|) π 2 Γ(1 − s) |x| 2 (see pag. 4877 formula (2.17) in [11]). Therefore, (60) and (61) yield Z ∞ Z ∞ −γt mx −γt 1 mx B2s,m(x) ≤ C e gm2st √ dt + C e tν √ dt 0 2 0 2 =: I1(x) + I2(x). (62) THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 29

We start with the estimate of I1(x) for |x| ≥ 2. Observing that m2−2s m2−2s γt + |x|2 ≥ γt + for all |x| ≥ 2, t > 0, 8t 2t 2 1 2 and that ab ≤ a + 4 b for all a, b ≥ 0 and > 0 gives 2−2s 1−s m 2 m √ N γt + |x| ≥ √ |x| γ for all x ∈ R , t > 0, 8t 2 we deduce that for all |x| ≥ 2 and t > 0 m2−2s t m2−2s m1−s √ γt + |x|2 ≥ γ + + √ |x| γ. 8t 2 4t 2 2

Thus, using the deﬁnition of gt, we can see that for all |x| ≥ 2

t Z ∞ −γ 2−2s 1−s √ e 2 m − m√ |x| γ − 4t 2 2 I1(x) ≤ C N e e dt 0 t 2 ∞ −γ t Z e 2 m2−2s −c|x| − 4t −C2|x| ≤ Ce N e dt ≤ C1e , (63) 0 t 2 where we used the fact that ∞ −αt Z e β − t υ e dt < ∞ ∀α, β, υ > 0. 0 t

Now we estimate I2(x) for large values of |x|. Recalling formula (4) concerning the asymptotic behavior of Kν at inﬁnity, we deduce that there exists r0 > 0 such that K (r) e−r ν ≤ C0 for all r ≥ r , ν ν+ 1 0 r r 2 and thus m √ K N+2s ( √ |x|) e−c¯|x| 2 2 2 ¯ 0 N+2s ≤ C N+2s+1 for all |x| ≥ r0 := r0. 2 |x| 2 m √m |x| 2 1 0 Consequently, using the deﬁnition of ν , for all |x| ≥ r0 we get

−c¯|x| Z ∞ −C4|x| ¯ e −γt e I2(x) ≤ C N+2s+1 t e dt ≤ C3 N+2s+1 . (64) |x| 2 0 |x| 2 0 Gathering (60), (62), (63), and (64), we ﬁnd that for any |x| ≥ max{r0, 2} e−C4|x| −C2|x| −C6|x| B2s,m(x) ≤ C1e + C3 N+2s+1 ≤ C5e . |x| 2 Then (57) holds true and this ends the proof of Theorem 1.1.

1 Remark 3. When s = , p 1 (x, t) can be calculated explicitly (see [11, 35]) and 2 2 ,m is given by

N+1 m 2 N+1 mt 2 2 − 4 p 2 2 p 1 ,m(x, t) = 2 t e (|x| + t ) K N+1 (m |x| + t ). 2 2π 2

Remark 4. By the deﬁnitions of B2s,m and ps,m, it follows that B2s,m is radial, positive, decreasing in |x|, and smooth on RN \{0}. 30 VINCENZO AMBROSIO

5. A multiplicity result. This section is devoted to the multiplicity of positive ˜ solutions to (1). In this case, we define f as in Section 3 by replacing V1 by V0, and 2V0 we choose κ > 2s . m −V0 5.1. The Nehari manifold approach. Let us consider + + Xε := {u ∈ Xε : |supp(u (·, 0)) ∩ Λε| > 0}. + Let Sε be the unit sphere of Xε and we define Sε := Sε ∩ Xε. We observe that + + + Xε is open in Xε. By the definition of Sε and the fact that Xε is open in Xε, it + 1,1 follows that Sε is a non-complete C -manifold of codimension 1, modeled on Xε + + + and contained in the open Xε ; see [46]. Then, Xε = TuSε ⊕ Ru for each u ∈ Sε , where + TuSε := {v ∈ Xε : hu, viε = 0}. Since f is only continuous, the next results will be fundamental to overcome the + non-differentiability of Nε and the incompleteness of Sε . 0 0 Lemma 5.1. Assume that (V1 )-(V2 ) and (f1)-(f4) hold. Then: + (i) For each u ∈ Xε , let h : R+ → R be defined by hu(t) := Jε(tu). Then, there is a unique tu > 0 such that 0 hu(t) > 0 in (0, tu), 0 hu(t) < 0 in (tu, ∞). + (ii) There exists τ > 0 independent of u such that tu ≥ τ for any u ∈ Sε . + Moreover, for each compact set K ⊂ Sε there is a positive constant CK such that tu ≤ CK for any u ∈ K. + (iii) The map mˆ ε : Xε → Nε given by mˆ ε(u) := tuu is continuous and mε := + −1 u mˆ ε| + is a homeomorphism between and Nε. Moreover, m (u) = . Sε Sε ε kukε + + (iv) If there is a sequence (un) ⊂ Sε such that dist(un, ∂Sε ) → 0, then kmε(un)kε → ∞ and Jε(mε(un)) → ∞. 1 Proof. (i) Clearly, hu ∈ C (R+, R), and arguing as in the proof of Lemma 3.1, it is easy to verify that hu(0) = 0, hu(t) > 0 for t > 0 small enough and hu(t) < 0 for t > 0 sufficiently large. Therefore, maxt≥0 hu(t) is achieved at some tu > 0 verifying 0 hu(tu) = 0 and tuu ∈ Nε. Now, we note that Z 0 2 gε(x, tu) hu(t) = 0 ⇐⇒ kukε = u dx =: ζ(t). N t R + By the definition of g,(g4) and u ∈ Xε , we see that the function t 7→ ζ(t) is increasing in (0, ∞). Hence, there exists a unique tu > 0 satisfying the desired properties. + 0 (ii) Let u ∈ Sε . By (i) there exists tu > 0 such that hu(tu) = 0, that is Z tu = gε(x, tuu)u dx. N R By (14) and Theorem 2.1, for all η > 0 there exists Cη > 0 such that ∗ 2s −1 tu ≤ ηtuC1 + CηC2tu , with C1,C2 > 0 independent of η. Taking η > 0 sufficiently small, we obtain that + there exists τ > 0, independent of u, such that tu ≥ τ. Now, let K ⊂ Sε be a compact set and we show that tu can be estimated from above by a constant depending on K. Assume by contradiction that there exists a sequence (un) ⊂ K THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 31

such that tn := tun → ∞. Therefore, there exists u ∈ K such that un → u in Xε. From (iii) in Lemma 3.1, we get

Jε(tnun) → −∞. (65)

0 0 Fix v ∈ Nε. Then, using hJε(v), vi = 0, (V1 ), (g3) and (8), we can infer 1 J (v) = J (v) − hJ 0 (v), vi ε ε θ ε Z 1 1 2 1 = − kvkε + [gε(x, v(x, 0))v(x, 0) − θGε(x, v(x, 0))] dx 2 θ θ N R Z 1 1 2 1 1 1 2 ≥ − kvkε − − V0v (x, 0) dx 2 θ 2 θ κ c Λε Z 1 1 2 1 1 2 = − kvk N+1 + − (V (x) + V )v (x, 0) dx Xs( ) ε 0 2 θ R+ 2 θ N R Z 1 2 − 1 + V0 v (x, 0) dx κ N R Z 1 1 2 1 1 1 2 ≥ − kvk N+1 − − 1 + V v (x, 0) dx Xs( ) 0 2 θ R+ 2 θ κ N R Z 1 1 2 1 1 1 V0 2s 2 = − kvk N+1 − − 1 + m v (x, 0) dx Xs( ) 2s 2 θ R+ 2 θ κ m N R 1 1 1 V0 2 ≥ − 1 − 1 + kvk N+1 > 0 (66) 2s Xs( ) 2 θ κ m R+

V0 because κ > 2s . Taking into account that (tu un) ⊂ Nε and ktu unkε = tu → m −V0 n n n ∞, from (66) we deduce that (65) does not hold. −1 (iii) Firstly, we note thatm ˆ ε, mε and mε are well-deﬁned. Indeed, by (i), for + each u ∈ Xε there exists a unique mε(u) ∈ Nε. On the other hand, if u ∈ Nε then + + u ∈ Xε . Otherwise, if u∈ / Xε , we have + |supp(u (·, 0)) ∩ Λε| = 0, which together with (g3)-(ii) and (8) gives Z 2 kukε = gε(x, u(x, 0))u(x, 0) dx N ZR Z = gε(x, u(x, 0))u(x, 0) dx + gε(x, u(x, 0))u(x, 0) dx c Λε Λε Z + + = gε(x, u (x, 0))u (x, 0) dx c Λε V Z ≤ 0 u2(x, 0) dx. κ c Λε Therefore, Z Z 2 2 1 2 kuk N+1 ≤ − (V (x) + V )u (x, 0) dx + 1 + V u (x, 0) dx Xs( ) ε 0 0 R+ N κ N R R 1 V0 2 ≤ 1 + kuk N+1 , 2s Xs( ) κ m R+ 32 VINCENZO AMBROSIO

V0 and using u 6≡ 0 and κ > 2s , we ﬁnd m −V0 1 V0 2 0 < 1 − 1 + kuk N+1 ≤ 0, 2s Xs( ) κ m R+ that is a contradiction. Consequently, m−1(u) = u ∈ +, m−1 is well-deﬁned ε kukε Sε ε and continuous. + Let u ∈ Sε . Then,

−1 −1 tuu u mε (mε(u)) = mε (tuu) = = = u ktuukε kukε from which mε is a bijection. Now, our aim is to prove thatm ˆ ε is a continuous + + + function. Let (un) ⊂ Xε and u ∈ Xε such that un → u in Xε . Hence, un u → in Xε. kunkε kukε

un Set vn := and tn := tv . By (ii) there exists t0 > 0 such that tn → t0. Since kunkε n tnvn ∈ Nε and kvnkε = 1, we have Z 2 tn = gε(x, tnvn(x, 0))tnvn(x, 0) dx. N R Passing to the limit as n → ∞, we obtain Z 2 t0 = gε(x, t0v(x, 0))t0v(x, 0) dx, N R u where v := , and thus t0v ∈ Nε. By (i) we deduce that tv = t0, and this shows kukε that un u mˆ ε(un) =m ˆ ε → mˆ ε =m ˆ ε(u) in Xε. kunkε kukε

Therefore,m ˆ ε and mε are continuous functions. + + (iv) Let (un) ⊂ Sε be such that dist(un, ∂Sε ) → 0. Then, by Theorem 2.1 and 0 0 ∗ (V1 )-(V2 ), for each q ∈ [2, 2s] there exists Cq > 0 such that + q q |u (·, 0)| q ≤ inf |un(·, 0) − v(·, 0)| q n L (Λε) + L (Λε) v∈∂Sε q ≤ Cq inf kun − vk + ε v∈∂Sε + q ≤ Cq dist(un, ∂Sε ) , for all n ∈ N.

Thus, by (g1), (g2), and (g3)-(ii), we can infer that, for all t > 0, Z Gε(x, tun(x, 0)) dx N R Z Z = Gε(x, tun(x, 0)) dx + Gε(x, tun(x, 0)) dx c Λε Λε 2 Z Z t 2 ≤ V0 un(x, 0) dx + F (tun(x, 0)) dx κ c Λε Λε 2 t Z Z ∗ Z ∗ 2 2 + 2 2s + 2s ≤ V0 un(x, 0) dx + C1t (un (x, 0)) dx + C2t (un (x, 0)) dx κ N R Λε Λε 2 t Z ∗ ∗ 2 0 2 + 2 0 2s + 2s ≤ V0 un(x, 0) dx + C1t dist(un, ∂Sε ) + C2t dist(un, ∂Sε ) . (67) κ N R THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 33

+ Bearing in mind the deﬁnition of mε(un), and using (8), (11), (67), dist(un, ∂Sε ) → 0 and kunkε = 1, we have

lim inf Jε(mε(un)) n→∞

≥ lim inf Jε(tun) n→∞ Z 2 1 2 V0 2 0 2 + 2 ≥ lim inf t kunkε − un(x, 0) dx − C1t dist(un, ∂Sε ) n→∞ 2 κ N R ∗ ∗ o 0 2s + 2s −C2t dist(un, ∂Sε ) Z 2 1 2 V0 2 = lim inf t kunkε − un(x, 0) dx n→∞ 2 κ N R 2 1 2 V0 2 ≥ lim inf t ku k − ku k N+1 n ε 2s n Xs( ) n→∞ 2 κm R+ 2 1 V0 2 ≥ lim inf t − 2s kunkε n→∞ 2 κ(m − V0) 2 1 V0 = t − 2s for all t > 0. 2 κ(m − V0)

2V0 Recalling that κ > 2s and sending t → ∞, we get m −V0

lim Jε(mε(un)) = ∞. n→∞

In particular, by the deﬁnition of Jε and (g3), we obtain

1 2 lim inf kmε(un)kε ≥ lim inf Jε(mε(un)) = ∞ n→∞ 2 n→∞ which gives kmε(un)kε → ∞ as n → ∞. The proof of the lemma is now completed.

Let us deﬁne the maps ˆ + + ψε : Xε → R and ψε : Sε → R, ˆ ˆ by ψε(u) := Jε(m ˆ ε(u)) and ψε := ψε| + . Sε From Lemma 5.1 and arguing as in the proofs of Proposition 9 and Corollary 10 in [46], we may obtain the following result.

0 0 Proposition 2. Assume that (V1 )-(V2 ) and (f1)-(f4) hold. Then, ˆ 1 + (a) ψε ∈ C (Xε , R) and

ˆ0 kmˆ ε(u)kε 0 hψε(u), vi = hJε(m ˆ ε(u)), vi, kukε + for every u ∈ Xε and v ∈ Xε. 1 + (b) ψε ∈ C (Sε , R) and 0 0 hψε(u), vi = kmε(u)kεhJε(mε(u)), vi, + for every v ∈ TuSε . (c) If (un) is a Palais-Smale sequence for ψε, then (mε(un)) is a Palais-Smale sequence for Jε. If (un) ⊂ Nε is a bounded Palais-Smale sequence for Jε, then −1 (mε (un)) is a Palais-Smale sequence for ψε. 34 VINCENZO AMBROSIO

(d) u is a critical point of ψε if and only if mε(u) is a nontrivial critical point for Jε. Moreover, the corresponding critical values coincide and

inf ψε(u) = inf Jε(u). + u∈N u∈Sε ε Remark 5. As in [46], we have the following variational characterization of the inﬁmum of Jε over Nε:

cε = inf Jε(u) = inf max Jε(tu) = inf max Jε(tu). u∈N + t>0 + t>0 ε u∈Xε u∈Sε + Corollary 1. The functional ψε satisﬁes the (PS)c condition on Sε at any level c ∈ R.

Proof. Let (un) be a Palais-Smale sequence for ψε at the level c. Then 0 ψε(un) → c and kψε(un)k∗ → 0, + ∗ as n → ∞, where k · k∗ is the norm in the dual space (Tun Sε ) . It follows from Proposition2-( c) that (mε(un)) is a Palais-Smale sequence for Jε in Xε at the level + c. Then, using Lemma 3.2, there exists u ∈ Sε such that, up to a subsequence,

mε(un) → mε(u) in Xε. + Applying Lemma 5.1-(iii) we can infer that un → u in Sε . It is clear that the previous results can be easily extended for the autonomous case. More precisely, for µ > −m2s, we deﬁne + + Yµ := {u ∈ Yµ : |supp(u (·, 0))| > 0}. + + + Let Sµ be the unit sphere of Yµ and we put Sµ := Sε ∩ Yµ . We note that Sµ is a 1,1 non-complete C -manifold of codimension 1, modeled on Yµ and contained in the + + + open Yµ . Then, Yµ = TuSµ ⊕ Ru for each u ∈ Sµ , where + TuSµ := {v ∈ Yµ : hu, viYµ = 0}. Arguing as in the proof of Lemma 5.1 we have the following result.

Lemma 5.2. Assume that (f1)-(f4) hold. Then, + (i) For each u ∈ Yµ , let h : R+ → R be deﬁned by hu(t) := Lµ(tu). Then, there is a unique tu > 0 such that 0 hu(t) > 0 in (0, tu), 0 hu(t) < 0 in (tu, ∞). + (ii) There exists τ > 0 independent of u such that tu ≥ τ for any u ∈ Sµ . + Moreover, for each compact set K ⊂ Sµ there is a positive constant CK such that tu ≤ CK for any u ∈ K. + (iii) The map mˆ µ : Yµ → Mµ given by mˆ µ(u) := tuu is continuous and mµ := + −1 u mˆ µ| + is a homeomorphism between µ and Mµ. Moreover, mµ (u) = . Sµ S kukYµ + + (iv) If there is a sequence (un) ⊂ Sµ such that dist(un, ∂Sµ ) → 0 then kmµ(un)kYµ → ∞ and Lµ(mµ(un)) → ∞. Let us deﬁne the maps ˆ + + ψµ : Yµ → R and ψµ : Sµ → R, THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 35

ˆ ˆ by ψµ(u) := Lµ(m ˆ µ(u)) and ψµ := ψµ| + . Sµ The next result is a direct consequence of Lemma 5.2 and some arguments found in Proposition 9 and Corollary 10 in [46].

Proposition 3. Assume that (f1)-(f4) hold. Then, ˆ 1 + (a) ψµ ∈ C (Yµ , R) and kmˆ (u)k ˆ0 µ Yµ 0 hψµ(u), vi = hLµ(m ˆ µ(u)), vi, kukYµ + for every u ∈ Yµ and v ∈ Yµ. 1 + (b) ψµ ∈ C (Sµ , R) and 0 0 hψµ(u), vi = kmµ(u)kYµ hLµ(mµ(u)), vi, + for every v ∈ TuSµ . (c) If (un) is a Palais-Smale sequence for ψµ, then (mµ(un)) is a Palais-Smale sequence for Lµ. If (un) ⊂ Mµ is a bounded Palais-Smale sequence for Lµ, −1 then (mµ (un)) is a Palais-Smale sequence for ψµ. (d) u is a critical point of ψµ if and only if mµ(u) is a nontrivial critical point for Lµ. Moreover, the corresponding critical values coincide and

inf ψµ(u) = inf Lµ(u). + u∈M u∈Sµ µ Remark 6. As in [46], we have the following variational characterization of the inﬁmum of Lµ over Mµ:

dµ = inf Lµ(u) = inf max Lµ(tu) = inf max Lµ(tu). u∈M + t>0 + t>0 µ u∈Yµ u∈Sµ The next lemma is a compactness result for the autonomous problem which will be used later.

Lemma 5.3. Let (un) ⊂ Mµ be a sequence such that Lµ(un) → dµ. Then (un) has a convergent subsequence in Yµ.

Proof. Since (un) ⊂ Mµ and Lµ(un) → dµ, we can apply Lemma 5.2-(iii), Propo- sition3-( d) and the deﬁnition of dµ to infer that

−1 un + vn := mµ (un) = ∈ Sµ , for all n ∈ N, kunkYµ and

ψµ(vn) = Lµ(un) → dµ = inf ψµ(v). + v∈Sµ + Let us introduce the following map F : Sµ → R ∪ {∞} deﬁned by setting

( + ψµ(u) if u ∈ Sµ , F(u) := + ∞ if u ∈ ∂Sµ . We note that + • (Sµ , δµ), where δµ(u, v) := ku − vkYµ , is a complete metric space; + •F∈ C(Sµ , R ∪ {∞}), by Lemma 5.2-(iv); •F is bounded below, by Proposition3-( d). 36 VINCENZO AMBROSIO

Hence, by applying Ekeland’s variational principle, we can ﬁnd a Palais-Smale se- + quence (ˆvn) ⊂ Sµ for ψµ at the level dµ with kvˆn − vnkYµ = on(1). Now the remainder of the proof follows from Proposition3, Theorem 3.5 and arguing as in the proof of Corollary1.

5.2. Technical results. In this subsection we make use of the Ljusternik-Schnirelman category theory to obtain multiple solutions to (1). In particular, we relate the number of positive solutions of (13) to the topology of the set M. For this reason, we take δ > 0 such that

N Mδ = {x ∈ R : dist(x, M) ≤ δ} ⊂ Λ, and consider a smooth nonicreasing function η : [0, ∞) → R such that η(t) = 1 if δ 0 0 ≤ t ≤ 2 , η(t) = 0 if t ≥ δ, 0 ≤ η ≤ 1 and |η (t)| ≤ c for some c > 0. For any z ∈ M, we introduce

ε x − z Ψ (x, y) := η(|(ε x − z, y)|)w , y , ε,z ε where w ∈ YV (0) is a positive ground state solution to the autonomous problem (25) with µ = V (0) = −V0, whose existence is guaranteed by Theorem 3.5. Let tε > 0 be the unique number such that

Jε(tεΨε,z) := max Jε(tΨε,z). t≥0

Finally, we consider Φε : M → Nε deﬁned by setting

Φε(z) := tεΨε,z.

Lemma 5.4. The functional Φε satisﬁes the following limit

lim Jε(Φε(z)) = dV (0), uniformly in z ∈ M. ε→0

Proof. Assume by contradiction that there exist δ0 > 0, (zn) ⊂ M and εn → 0 such that

|Jεn (Φεn (zn)) − dV (0)| ≥ δ0. (68)

Applying the dominated convergence theorem, we know that

kΨ k2 → kwk2 ∈ (0, ∞), εn,zn εn YV (0) (69) and Z Z

F (Ψεn,zn (x, 0)) dx → F (w(x, 0)) dx. (70) N N R R

0 0 Note that, for all n ∈ N and for all x ∈ B δ , we have εn x ∈ Bδ and thus εn 0 0 εn x−zn εn x + zn ∈ Bδ(zn) ⊂ Mδ ⊂ Λ. By using the change of variable x := , the εn THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 37 deﬁnition of η and the fact that G = F in Λ × R, we can write

Jεn (Φεn (zn)) 2 ZZ Z ! tεn 1−2s 2 2 2 2 = y (|∇Ψεn,zn | + m Ψεn,zn ) dxdy + Vεn (x)Ψεn,zn (x, 0) dx 2 N+1 N R+ R Z − G(Ψεn,zn (x, 0)) dx N R 2 ZZ ! tεn 1−2s 0 0 2 2 0 0 2 0 = y (|∇(η(|(εn x , y)|)w(x , y))| + m (η(|(εn x , y)|)w(x , y)) dx dy 2 N+1 R+ 2 Z tεn 0 0 0 2 0 + V (εn x + zn)(η(|(εn x , 0)|)w(x , 0)) dx 2 N R Z 0 0 0 − F (tεn η(|(εn x , 0)|)w(x , 0)) dx . (71) N R On the other hand, since hJ 0 (Φ (z )), Φ (z )i = 0 and g = f in Λ × , we have εn εn n εn n R that ZZ ! 2 1−2s 0 0 2 2 0 0 2 0 tε y (|∇(η(|(εn x , y)|)w(x , y))| + m (η(|(εn x , y)|)w(x , y)) dx dy n N+1 R+ Z + t2 V (ε x0 + z )(η(|(ε x0, 0)|)w(x0, 0))2 dx0 εn n n n N Z R 0 0 0 0 0 = f(tεn η(|(εn x , 0)|)w(x , 0))tεn η(|(εn x , 0)|)w(x , 0) dx . (72) N R

Let us show that tεn → 1 as n → ∞. If by contradiction tεn → ∞, observing that η(|x|) = 1 for x ∈ B δ and that B δ ⊂ B δ for all n large enough, it follows from 2 2 2 εn (f4) that Z f(t w(x0, 0)) kΨ k2 ≥ εn w2(x0, 0) dx0 εn,zn εn 0 B δ tεn w(x , 0) 2 f(t w(ˆx, 0)) Z ≥ εn w2(x0, 0) dx0, (73) tεn w(ˆx, 0) B δ 2 where w(ˆx, 0) := min w(x, 0) > 0. Combining (69) with (73) and using (f ), x∈B δ 3 2 we obtain a contradiction. Hence, (tεn ) is bounded in R and, up to subsequence, we may assume that tεn → t0 for some t0 ≥ 0. In particular, t0 > 0. In fact, if t0 = 0, we see that (16) and (72) imply that Z 0 0 0 0 0 r ≤ f(tεn η(|(εn x , 0)|)w(x , 0))tεn η(|(εn x , 0)|)w(x , 0) dx . N R Using (f1), (f2), (69) and the above inequality, we obtain an absurd. Therefore, t0 > 0. By (69) and (70), we can pass to the limit in (72) as n → ∞ to see that Z 2 f(t0w(x, 0)) 2 kwkV (0) = w (x, 0) dx. N t w(x, 0) R 0 Bearing in mind that w ∈ MV (0) and using (f4), we infer that t0 = 1. Letting n → ∞ in (71) and using tεn → 1, we conclude that

lim Jε (Φε (zn)) = LV (0)(w) = dV (0), n→∞ n n which is in contrast with (68). 38 VINCENZO AMBROSIO

N N Let us ﬁx ρ = ρ(δ) > 0 satisfying Mδ ⊂ Bρ, and we consider Υ : R → R given by x if |x| < ρ, Υ (x) := ρx |x| if |x| ≥ ρ. N Then we deﬁne the barycenter map βε : Nε → R as follows Z Υ (ε x)u2(x, 0) dx N βε(u) := R Z . u2(x, 0) dx N R

Lemma 5.5. The functional Φε satisﬁes the following limit

lim βε(Φε(z)) = z, uniformly in z ∈ M. (74) ε→0

Proof. Suppose by contradiction that there exist δ0 > 0, (zn) ⊂ M and εn → 0 such that

|βεn (Φεn (zn)) − zn| ≥ δ0. (75) 0 εn x−zn Using the deﬁnitions of Φε (zn), βε , η and the change of variable x = , we n n εn see that R 0 0 0 2 0 N [Υ (εn x + zn) − zn](η(|(εn x , 0)|)ω(x , 0)) dx β (Φ (z )) = z + R . εn εn n n R 0 0 2 0 N (η(|(εn x , 0)|)ω(x , 0)) dx R Taking into account (zn) ⊂ M ⊂ Bρ and the dominated convergence theorem, we infer that

|βεn (Φεn (zn)) − zn| = on(1) which contradicts (75). The next compactness result will play a fundamental role to prove that the solutions of (13) are also solution to (10).

Proposition 4. Let εn → 0 and (un) ⊂ Nεn be such that Jεn (un) → dV (0). Then N there exists (˜zn) ⊂ R such that vn(x, y) := un(x +z ˜n, y) has a convergent subsequence in YV (0). Moreover, up to a subsequence, zn := εn z˜n → z0 for some z0 ∈ M. Proof. Since hJ 0 (u ), u i = 0 and J (u ) → d , it is easy to see that ku k ≤ εn n n εn n V (0) n εn

C for all n ∈ N. Let us observe that kunkεn 6→ 0 since dV (0) > 0. Therefore, N arguing as in the proof of Lemma 3.4, we can ﬁnd a sequence (˜zn) ⊂ R and constants R, β > 0 such that Z 2 lim inf un(x, 0) dx ≥ β. n→∞ BR(˜zn)

Set vn(x, y) := un(x +z ñ, y). Then, (vn) is bounded in YV (0), and we may assume that vn * v in YV (0), for some v 6≡ 0. Let tn > 0 be such thatv ñ := tnvn ∈ MV (0), and set zn := εn zñ.

Then, using (g2) and un ∈ Nεn , we see that

dV (0) ≤ LV (0)(˜vn) ≤ Jεn (tnun) ≤ Jεn (un) = dV (0) + on(1), which gives

LV (0)(˜vn) → dV (0) and (˜vn) ⊂ MV (0). (76) THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 39

In particular, (76) yields that (˜vn) is bounded in YV (0), so we may assume that v˜n * v˜ in YV (0). Since vn 6→ 0 in YV (0) and (˜vn) is bounded in YV (0), we deduce that (tn) is bounded in R and, up to a subsequence, we may assume that tn → t0 ≥ 0.

If t0 = 0, from the boundedness of (vn), we get kv˜nkYV (0) = tnkvnkYV (0) → 0, which yields LV (0)(˜vn) → 0 in contrast with the fact dV (0) > 0. Then, t0 > 0. From the uniqueness of the weak limit we havev ˜ = t0v andv ˜ 6≡ 0. Using Lemma 5.3 we deduce that

v˜n → v˜ in YV (0), (77) which implies that vn → v in YV (0) and

0 LV (0)(˜v) = dV (0) and hL0(˜v), v˜i = 0.

Now, we show that (zn) admits a subsequence, still denoted by (zn), such that zn → z0 for some z0 ∈ M. Assume by contradiction that (zn) is not bounded in N R . Then there exists a subsequence, still denoted by (zn), such that |zn| → ∞ as n → ∞. Take R > 0 such that Λ ⊂ BR, and assume that |zn| > 2R for n large.

Thus, for any x ∈ BR/ εn , we get | εn x + zn| ≥ |zn| − | εn x| > R for all n large enough. Hence, by using the deﬁnition of g, we have Z 2 kvnkV (0) ≤ g(εn x + zn, vn(x, 0))vn(x, 0) dx N ZR Z ˜ ≤ f(vn(x, 0))vn(x, 0) dx + f(vn(x, 0))vn(x, 0) dx. N BR/ εn R \BR/ εn

Since vn → v in YV (0) as n → ∞, it follows from the dominated convergence theorem that Z f(vn(x, 0))vn(x, 0) dx = on(1). N R \BR/ εn

Therefore,

V Z kv k2 ≤ 0 v2 (x, 0) dx + o (1), n V (0) κ n n BR/ εn which combined with (8) and V (0) = −V0 yields V0 1 2 1 − 1 + kv k N+1 ≤ o (1) 2s n Xs( ) n m κ R+

s N+1 V0 and this gives a contradiction due to vn → v in X ( ), v 6≡ 0 and κ > 2s . R+ m −V0 N Thus, (zn) is bounded in R and, up to a subsequence, we may assume that zn → z0. c If z0 ∈/ Λ, then there exists r > 0 such that zn ∈ Br/2(z0) ⊂ Λ for any n large s N+1 enough. Reasoning as before, we reach vn → 0 in X (R+ ), that is a contradiction. Hence, z0 ∈ Λ. Now, we show that V (z0) = V (0). Assume by contradiction that V (z0) > V (0). Taking into account (77), V (z0) > V (0) = −V0, Fatou’s lemma and 40 VINCENZO AMBROSIO the invariance of RN by translations, we have

dV (0) = LV (0)(˜v) < LV (z0)(˜v) ZZ Z 1 1−2s 2 2 2 1 2 = y (|∇v˜| + m v˜ ) dxdy + (V (z0) + V0)˜vn(x, 0) dx 2 N+1 2 N R+ R Z Z V0 2 − vñ(x, 0) dx − F (˜vn(x, 0)) dx 2 N N R R " ZZ 1 1−2s 2 2 2 ≤ lim inf y (|∇vñ| + m vñ) dxdy n→∞ 2 N+1 R+ Z Z 1 2 + V (εn x + zn)˜vn(x, 0) dx − F (˜vn(x, 0)) dx 2 N N R R " 2 ZZ tn 1−2s 2 2 2 = lim inf y (|∇un| + m un) dxdy n→∞ 2 N+1 R+ 2 Z Z tn 2 + V (εn x)un(x, 0) dx − F (tnun(x, 0)) dx 2 N N R R ≤ lim inf Jε (tnun) ≤ lim inf Jε (un) = dV (0) n→∞ n n→∞ n 0 which gives a contradiction. Therefore, in view of (V2 ), we can conclude that z0 ∈ M.

Now, we consider the following subset of Nε: Neε := u ∈ Nε : Jε(u) ≤ dV (0) + h(ε) , where h(ε) := supz∈M |Jε(Φε(z)) − dV (0)| → 0 as ε → 0 as a consequence of Lemma 5.4. By the deﬁnition of h(ε), we know that, for all z ∈ M and ε > 0, Φε(z) ∈ Neε and Neε 6= ∅. We present below an interesting relation between Neε and the barycenter map βε. Lemma 5.6. For any δ > 0, there holds that

lim sup dist(βε(u),Mδ) = 0. ε→0 u∈Neε

Proof. Let εn → 0 as n → ∞. By deﬁnition, there exists (un) ⊂ Neεn such that

sup inf |βεn (u) − z| = inf |βεn (un) − z| + on(1). z∈Mδ z∈Mδ u∈Neεn

Therefore, it suﬃces to prove that there exists (zn) ⊂ Mδ such that

lim |βε (un) − zn| = 0. (78) n→∞ n

Observing that LV (0)(tun) ≤ Jεn (tun) for all t ≥ 0 and (un) ⊂ Neεn ⊂ Nεn , we deduce that

dV (0) ≤ cεn ≤ Jεn (un) ≤ dV (0) + h(εn), N which implies that Jεn (un) → dV (0). Using Proposition4, there exists (˜zn) ⊂ R such that zn = εn zñ ∈ Mδ for n sufficiently large. Thus R 2 N [Υ (εn x + zn) − zn]v (x, 0) dx β (u ) = z + R n = z + o (1) εn n n R 2 n n N v (x, 0) dx R n THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 41 since vn(x, y) := un(· +z ñ, y) strongly converges in YV (0) and εn x + zn → z0 ∈ Mδ. Therefore, (78) holds true.

+ We end this section by proving a multiplicity result for (13). Since Sε is not a completed metric space, we make use of the abstract category result in [46]; see also [5, 28].

Theorem 5.7. For any δ > 0 such that Mδ ⊂ Λ, there exists ˜εδ > 0 such that, for any ε ∈ (0, ˜εδ), problem (13) has at least catMδ (M) positive solutions. + Proof. For any ε > 0, we consider the map αε : M → Sε deﬁned as αε(z) := −1 mε (Φε(z)). Using Lemma 5.4, we see that

lim ψε(αε(z)) = lim Jε(Φε(z)) = dV (0), uniformly in z ∈ M. (79) ε→0 ε→0 Set + + Seε := {w ∈ Sε : ψε(w) ≤ dV (0) + h(ε)}, where h(ε) := supz∈M |ψε(αε(z)) − dV (0)|. It follows from (79) that h(ε) → 0 as + ε → 0. Then, Seε 6= ∅ for all ε > 0. From the above considerations and in the light of Lemma 5.4, Lemma 5.1-(iii), Lemma 5.5 and Lemma 5.6, we can ﬁnd ¯ε = ¯εδ > 0 such that the diagram of continuous applications bellow is well-deﬁned for ε ∈ (0, ¯ε)

−1 Φε mε mε βε M → Φε(M) → αε(M) → Φε(M) → Mδ.

Thanks to Lemma 5.5, and decreasing ¯ε if necessary, we see that βε(Φε(z)) = δ z + θ(ε, z) for all z ∈ M, for some function θ(ε, z) satisfying |θ(ε, z)| < 2 uniformly in z ∈ M and for all ε ∈ (0, ¯ε). Deﬁne H(t, z) := z + (1 − t)θ(ε, z) for (t, z) ∈ [0, 1] × M. Then H : [0, 1] × M → Mδ is continuous. Clearly, H(0, z) = βε(Φε(z)) and H(1, z) = z for all z ∈ M. Consequently, H(t, z) is a homotopy between βε ◦ Φε = (βε ◦ mε) ◦ αε and the inclusion map id : M → Mδ. This fact yields

catαε(M)αε(M) ≥ catMδ (M). (80) Applying Corollary1, Lemma 3.6 (see also Remark1), and Theorem 27 in [46] with c = cε ≤ dV (0) + h(ε) = d and K = αε(M), we obtain that ψε has at least + catαε(M)αε(M) critical points on Seε . Taking into account Proposition2-( d) and

(80), we infer that Jε admits at least catMδ (M) critical points in Neε. We also have the following fundamental result.

Lemma 5.8. Let εn → 0 and (un) ⊂ Neεn be a sequence of solutions to (13). Then, ∞ N vn(x, y) := un(x +z ˜n, y) satisﬁes vn(·, 0) ∈ L (R ) and there exists C > 0 such that |vn(·, 0)|∞ ≤ C for all n ∈ N, (81) where (˜zn) is given by Proposition4. Moreover,

lim vn(x, 0) = 0 uniformly in n ∈ N. (82) |x|→∞

Proof. Since Jεn (un) ≤ dV (0) + h(εn) with h(εn) → 0 as n → ∞, we can argue as at the beginning of the proof of Proposition4 to deduce that Jεn (un) → dV (0). N Thus we may invoke Proposition4 to obtain a sequence (˜zn) ⊂ R such that εn z˜n → z0 ∈ M and vn(x, y) := un(x +z ˜n, y) strongly converges in YV (0). At this point we can proceed as in the proofs of Lemmas 4.1 and 4.2 to obtain the assertions. 42 VINCENZO AMBROSIO

We conclude this section by giving the proof of Theorem 1.2.

Proof of Theorem 1.2. Let δ > 0 be such that Mδ ⊂ Λ. Firstly, we claim that there exists ˆεδ > 0 such that for any ε ∈ (0, ˆεδ) and any solution u ∈ Neε of (13), it holds

|u(·, 0)| ∞ c < a. (83) L (Λε)

We argue by contradiction and assume that for some subsequence εn → 0 we can obtain u = u ∈ N such that J 0 (u ) = 0 and n εn eεn εn n

|un(·, 0)|L∞(Λc ) ≥ a. (84) εn

Since Jεn (un) ≤ dV (0) + h(εn), we can argue as in the first part of Proposition4 to N deduce that Jεn (un) → dV (0). In view of Proposition4, there exists (˜zn) ⊂ R such that εn zñ → z0 for some z0 ∈ M and vn(x, y) := un(x+z ñ, y) strongly converges in YV (0). Then we can proceed as in the proof of Theorem 1.1 to get a contradiction. Let ˜εδ > 0 be given by Theorem 5.7 and we set εδ := min{˜εδ, ˆεδ}. Fix ε ∈ (0, εδ).

Applying Theorem 5.7, we obtain at least catMδ (M) positive solutions to (13). If u ∈ Xε is one of these solutions, then u ∈ Neε, and in view of (83) and the deﬁnition of g, we infer that u is also a solution to (10). Then, u(·, 0) is a solution to (1), and we deduce that (1) has at least catMδ (M) positive solutions. To study the behavior of the maximum points of the solutions, we argue as in the proof of Theorem 1.1.

6. Appendix A: Bessel potentials. In this appendix we collect some useful results concerning Bessel potentials. For more details we refer to [8, 13, 30, 43].

Deﬁnition 6.1. Let α > 0. The Bessel potential of order α of u ∈ S(RN ) is deﬁned as Z − α Jαu(x) := (1 − ∆) 2 u(x) = (Gα ∗ u)(x) = Gα(x − y)u(y) dy, N R where Gα given through the Fourier transform

− N 2 − α FGα(k) := (2π) 2 (1 + |k| ) 2 is the so-called Bessel kernel.

Remark 7. If α ∈ R (or α ∈ C), then we may deﬁne the Bessel potential of a temperate distribution u ∈ S0(RN ) (see [13]) by setting

2 − α F(Jαu)(k) := (1 + |k| ) 2 Fu(k).

It is possible to prove (see [8]) that

1 α−N N−α 2 Gα(x) = N+α−2 N K (|x|)|x| . 2 2 α 2 2 π Γ( 2 )

Thus, Gα(x) is a positive, decreasing function of |x|, analytic except at x = 0, and N for x ∈ R \{0}, Gα(x) is an entire function of α. Moreover, from (3) and (4), we THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 43 have  Γ( N−α )  2 |x|α−N , if 0 < α < N,  N α  2απ 2 Γ( )  2    1 1 log , if α = N, Gα(x) ∼ N N 2N−1π 2 Γ( ) |x|  2    Γ( α−N )  2  N , if α > N,  N 2 α 2 π Γ( 2 ) as |x| → 0, and

1 α−N−1 2 −|x| Gα(x) ∼ N+α−1 N−1 |x| e 2 2 α 2 π Γ( 2 ) 1 N as |x| → ∞. Hence, Gα ∈ L (R ) for all α > 0, and

Z N Gα(x) dx = (2π) 2 FGα(0) = 1. N R

By the deﬁnition of Gα, it is evident that the following composition formula holds:

Gα+β = Gα ∗ Gβ, α, β > 0. We also mention the integral formula (see [43])

Z ∞ 2 1 − |x| −δ α−N dδ G (x) = e 4δ e δ 2 . α N α 2 δ (4π) Γ 2 0 One the most interesting facts concerning Bessel potentials is they can be employed to deﬁne the Bessel potential spaces; see [1,8, 13, 30, 43]. For p ∈ [1, ∞] and α ∈ R, we deﬁne the Banach space

p p N p N Lα := Gα(L (R )) = {u : u = Gα ∗ f, f ∈ L (R )} endowed with the norm

p kukLα := |f|p with u := Gα ∗ f. p p N Thus Lα is a subspace of L (R ) for all α ≥ 0. We summarize some of its properties. Theorem 6.2. [1, 13, 43] N p (i) If α ≥ 0 and 1 ≤ p < ∞, then D(R ) is dense in Lα. 0 p (ii) If 1 < p < ∞ and p its conjugate exponent, then the dual of Lα is isometri- p0 cally isomorphic to L−α. p p (iii) If β < α, then Lα is continuously embedded in Lβ. Np (iv) If β ≤ α and if either 1 < p ≤ q ≤ N−(α−β)p < ∞ or p = 1 and 1 ≤ q < N p q N−α+β , then Lα is continuously embedded in Lβ. N p 0,µ N (v) If 0 < µ ≤ α − p < 1, then Lα is continuously embedded in C (R ). p k,p N 2 α,2 N (vi) Lk = W (R ) for all k ∈ N and 1 < p < ∞, Lα = W (R ) for any α. (vii) If 1 < p < ∞ and ε > 0, then for every α we have the following continuous embeddings: p α,p N p Lα+ε ⊂ W (R ) ⊂ Lα−ε. 44 VINCENZO AMBROSIO

In order to accomplish some useful regularity results for equations driven by (−∆ + m2)s, with m > 0 and s ∈ (0, 1), we introduce the H¨older-Zygmund (or Lipschitz) spaces Λα; see [13, 30, 43]. If α > 0 and α∈ / N, then we set Λα := [α],α−[α] N ∗ C (R ). If α = k ∈ N, then we set Λα := Λk where ( ) ∗ ∞ N N |u(x + h) + u(x − h) − 2u(x)| Λ1 := u ∈ L (R ) ∩ C(R ) : sup < ∞ N |h| x,h∈R ,|h|>0 if k = 1, and ∗ k−1 N γ ∗ Λk := u ∈ C (R ): D u ∈ Λ1 for all |γ| ≤ k − 1 if k ∈ N, k ≥ 2.

The deﬁnition of Λα spaces can be also extended for α ≤ 0; see [33, 47]. We recall the following useful result (for a proof we refer to Proposition 8.6.6. in [33] and Theorem 6 in [47]; see also [13, 43]).

Theorem 6.3. [13, 33, 43, 47] Let α, β ∈ R. Then J2α maps Λβ isomorphically onto Λβ+2α. As a consequence of Theorem 6.3 and the definition of Hölder-Zygmund spaces, we easily deduce the following regularity result. Corollary 2. Let m > 0, s ∈ (0, 1) and α ∈ (0, 1). Assume that f ∈ C0,α(RN ) and that u ∈ L∞(RN ) is a solution to (−∆ + m2)su = f in RN . • If α + 2s < 1, then u ∈ C0,α+2s(RN ). • If 1 < α + 2s < 2, then u ∈ C1,α+2s−1(RN ). • If 2 < α + 2s < 3, then u ∈ C2,α+2s−2(RN ). ∗ • If α + 2s = k ∈ {1, 2}, then u ∈ Λk. We also have a regularity result for solutions u ∈ L∞(RN ) to (−∆ + m2)su = f in RN with f ∈ L∞(RN ). Indeed, arguing as in the proof of Theorem 4 at page 149 in [43] (see also Theorem 5 in [47]) and using formula (59) with l = 2 and β = 2s in this theorem, we can use the characterization of Λα in terms of the Poisson integral given in [43, 47] to infer that the solutions u of the previous equation belong to Λ2s. Then, in view of the definition of Λα given above, we deduce the following result. Corollary 3. Let m > 0 and s ∈ (0, 1). Assume that f ∈ L∞(RN ) and that u ∈ L∞(RN ) is a solution to (−∆ + m2)su = f in RN . • If 2s < 1, then u ∈ C0,2s(RN ). ∗ • If 2s = 1, then u ∈ Λ1. • If 2s > 1, then u ∈ C1,2s−1(RN ). Remark 8. When m = 0, the conclusions in Corollary2 and Corollary3 can be found in [44].

7. Appendix B: An integral representation formula for (−∆+m2)s. Bearing in mind the asymptotic estimates (3) and (4) for Kν , we are able to gain an integral representation formula for (−∆ + m2)s, with m > 0 and s ∈ (0, 1), in the spirit of Lemma 3.2 in [21]. Theorem 7.1. Let m > 0 and s ∈ (0, 1). Then, for all u ∈ S(RN ), (−∆ + m2)su(x) = m2su(x)

C(N, s) N+2s Z 2u(x) − u(x + y) − u(x − y) 2 + m N+2s K N+2s (m|y|) dy. 2 N 2 R |y| 2 THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 45

Proof. Choosing the substitution z = y − x in (2), we obtain (−∆ + m2)su(x)

N+2s Z u(x) − u(y) 2s 2 = m u(x) + C(N, s)m P.V. N+2s K N+2s (m|x − y|) dy N 2 R |x − y| 2 N+2s Z u(x) − u(x + z) 2s 2 = m u(x) + C(N, s)m P.V. N+2s K N+2s (m|z|) dz. (85) N 2 R |z| 2 By substitutingz ˜ = −z in the last term in (85), we get Z u(x + z) − u(x) Z u(x − z˜) − u(x) P.V. N+2s K N+2s (m|z|) dz = P.V. N+2s K N+2s (m|z˜|) dz,˜ N 2 N 2 R |z| 2 R |z˜| 2 (86) and so, after relabelingz ˜ as z, Z u(x + z) − u(x) 2P.V. N+2s K N+2s (m|z|) dz N 2 R |z| 2 Z u(x + z) − u(x) = P.V. N+2s K N+2s (m|z|) dz N 2 R |z| 2 Z u(x − z) − u(x) + P.V. N+2s K N+2s (m|z|) dz N 2 R |z| 2 Z u(x + z) + u(x − z) − 2u(x) = P.V. N+2s K N+2s (m|z|) dz. (87) N 2 R |z| 2 Hence, if we rename z as y in (85) and (87), we can write (−∆ + m2)s as (−∆ + m2)su(x)

C(N, s) N+2s Z 2u(x) − u(x + y) − u(x − y) 2s 2 = m u(x) + m P.V. N+2s K N+2s (m|y|) dy. 2 N 2 2 R |y| (88) Now, by using a second order Taylor expansion, we see that

2u(x) − u(x + y) − u(x − y) |D2u| ∞ N+2s K N+2s (m|y|) ≤ N+2s−4 K N+2s (m|y|). 2 2 |y| 2 |y| 2 From (3) we deduce that 2 |D u|∞ C N+2s−4 K N+2s (m|y|) ∼ N+2s−2 as |y| → 0 |y| 2 2 |y| which implies the integrability near 0. On the other hand, using (4), we get

2u(x) − u(x + y) − u(x − y) C|u| ∞ N+2s K N+2s (m|y|) ≤ N+2s K N+2s (m|y|) 2 2 |y| 2 |y| 2

C −m|y| ∼ N+2s+1 e as |y| → ∞ |y| 2 which gives the integrability near ∞. Therefore, we can remove the P.V. in (88).

Acknowledgments. The author would like to thank the anonymous referee for her/his careful reading of the manuscript and valuable suggestions. 46 VINCENZO AMBROSIO

REFERENCES

[1] R. A. Adams, Sobolev Spaces, Pure and Applied Mathematics, Vol. 65 Academic Press, New York-London, 1975. [2] C. O. Alves and O. H. Miyagaki, Existence and concentration of solution for a class of frac- N tional elliptic equation in R via penalization method, Calc. Var. Partial Differential Equa- tions, 55 (2016), art. 47, 19 pp. [3] A. Ambrosetti and P. H. Rabinowitz, Dual variational methods in critical point theory and applications, J. Funct. Anal., 14 (1973), 349–381. [4] V. Ambrosio, Ground states solutions for a non-linear equation involving a pseudo-relativistic Schrödingeroperator, J. Math. Phys., 57 (2016), 051502, 18 pp. [5] V. Ambrosio, Multiplicity of positive solutions for a class of fractional Schrödingerequations via penalization method, Ann. Mat. Pura Appl. (4), 196 (2017), 2043–2062. [6] V. Ambrosio, Concentrating solutions for a class of nonlinear fractional Schrödingerequations N in R , Rev. Mat. Iberoam., 35 (2019), 1367–1414. N [7] V. Ambrosio, Concentration phenomena for a class of fractional Kirchhoff equations in R with general nonlinearities, Nonlinear Anal., 195 (2020), 111761, 39 pp. [8] N. Aronszajn and K. T. Smith, Theory of Bessel potentials. I, Ann. Inst. Fourier (Grenoble), 11 (1961), 385–475. [9] C. Bucur and E. Valdinoci, Nonlocal Diffusion and Applications, Lecture Notes of the Unione Matematica Italiana, 20. Springer, Unione Matematica Italiana, Bologna, 2016. xii+155 pp. [10] H. Bueno, O. H. Miyagaki and G. A. Pereira, Remarks about a generalized pseudo-relativistic Hartree equation, J. Differential Equations, 266 (2019), 876–909. [11] T. Byczkowski, J. Malecki and M. Ryznar, Bessel potentials, hitting distributions and Green functions, Trans. Amer. Math. Soc., 361 (2009), 4871–4900. [12] L. Caffarelli and L. Silvestre, An extension problem related to the fractional Laplacian, Comm. Partial Differential Equations, 32 (2007), 1245–1260 [13] A.-P. Calderón,Lebesgue spaces of differentiable functions and distributions, Proc. Sympos. Pure Math., American Mathematical Society, Providence, R.I., 4 (1961), 33–49. [14] R. Carmona, W. C. Masters and B. Simon, Relativistic Schrödingeroperators: Asymptotic behavior of the eigenfunctions, J. Func. Anal., 91 (1990), 117–142. [15] S. Cingolani and S. Secchi, Semiclassical analysis for pseudo-relativistic Hartree equations, J. Differential Equations, 258 (2015), 4156–4179. [16] V. Coti Zelati and M. Nolasco, Existence of ground states for nonlinear, pseudo-relativistic Schrödingerequations, Atti Accad. Naz. Lincei Rend. Lincei Mat. Appl., 22 (2011), 51–72. [17] V. Coti Zelati and M. Nolasco, Ground states for pseudo-relativistic Hartree equations of critical type, Rev. Mat. Iberoam., 29 (2013), 1421–1436. [18] J. Dávila,M. del Pino, S. Dipierro and E. Valdinoci, Concentration phenomena for the nonlocal Schrödingerequation with Dirichlet datum, Anal. PDE, 8 (2015), 1165–1235. [19] J. Dávila,M. del Pino and J. Wei, Concentrating standing waves for the fractional nonlinear Schrödingerequation, J. Differential Equations, 256 (2014), 858–892. [20] M. del Pino and P. L. Felmer, Local mountain passes for semilinear elliptic problems in unbounded domains, Calc. Var. Partial Differential Equations, 4 (1996), 121–137. [21] E. Di Nezza, G. Palatucci and E. Valdinoci, Hitchhiker’s guide to the fractional Sobolev spaces, Bull. Sci. math., 136 (2012), 521–573. [22] S. Dipierro, M. Medina and E. Valdinoci, Fractional Elliptic Problems with Critical Growth n in the Whole of R , Appunti. Scuola Normale Superiore di Pisa (Nuova Serie) [Lecture Notes. Scuola Normale Superiore di Pisa (New Series)], 15. Edizioni della Normale, Pisa, 2017. [23] A. Erdélyi,W. Magnus, F. Oberhettinger and F. G. Tricomi, Higher Transcendental Func- tions. Vol. II, Based on notes left by Harry Bateman. Reprint of the 1953 original. Robert E. Krieger Publishing Co., Inc., Melbourne, Fla., 1981. [24] E. B. Fabes, C. E. Kenig and R. P. Serapioni, The local regularity of solutions of degenerate elliptic equations, Comm. Partial Differential Equations, 7 (1982), 77–116. [25] M. M. Fall and V. Felli, Unique continuation properties for relativistic Schrödingeroperators with a singular potential, Discrete Contin. Dyn. Syst., 35 (2015), 5827–5867. [26] P. Felmer, A. Quaas and J. Tan, Positive solutions of the nonlinear Schrödingerequation with the fractional Laplacian, Proc. Roy. Soc. Edinburgh Sect. A, 142 (2012), 1237–1262. [27] P. Felmer and I. Vergara, Scalar field equation with non-local diffusion, NoDEA Nonlinear Differential Equations Appl., 22 (2015), 1411–1428. THE NONLINEAR FRACTIONAL RELATIVISTIC SCHRODINGER¨ EQUATION 47

[28] G. M. Figueiredo and J. R. Santos, Multiplicity and concentration behavior of positive solutions for a Schrödinger-Kirchhoff type problem via penalization method, ESAIM Control Optim. Calc. Var., 20 (2014), 389–415. [29] G. M. Figueiredo and G. Siciliano, A multiplicity result via Ljusternick-Schnirelmann category N and Morse theory for a fractional Schrödingerequation in R , NoDEA Nonlinear Differential Equations Appl., 23 (2016), art. 12, 22 pp. [30] L. Grafakos, Modern Fourier analysis, Third edition. Graduate Texts in Mathematics, 250. Springer, New York, 2014. [31] T. Grzywny and M. Ryznar, Two-sided optimal bounds for Green functions of half-spaces for relativistic α-stable process, Potential Anal., 28 (2008), 201–239. [32] I. W. Herbst, Spectral theory of the operator (p2 + m2)1/2 − Ze2/r, Comm. Math. Phys., 53 (1977), 285–294. [33] L. Hörmander, Lectures on Nonlinear Hyperbolic Differential Equations, Mathématiques& Applications (Berlin) [Mathematics & Applications], 26. Springer-Verlag, Berlin, 1997. [34] T. Jin, Y. Li and J. Xiong, On a fractional Nirenberg problem, part I: Blow up analysis and compactness of solutions, J. Eur. Math. Soc. (JEMS), 16 (2014), 1111–1171. [35] E. H. Lieb and M. Loss, Analysis, Graduate Studies in Mathematics, 14. American Mathe- matical Society, Providence, RI, 1997. [36] E. H. Lieb and H. T. Yau, The Chandrasekhar theory of stellar collapse as the limit of quantum mechanics, Comm. Math. Phys., 112 (1987), 147–174. [37] P.-L. Lions, The concentration-compactness principle in the calculus of variations. The locally compact case. II, Ann. Inst. H. PoincaréAnal. Non Linéaire, 1 (1984), 223–283. [38] G. Molica Bisci, V. R˘adulescuand R. Servadei, Variational Methods for Nonlocal Fractional Problems, Cambridge University Press, 162 Cambridge, 2016. [39] J. Moser, A new proof of De Giorgi’s theorem concerning the regularity problem for elliptic differential equations, Comm. Pure Appl. Math., 13 (1960), 457–468. [40] D. Mugnai, Pseudorelativistic Hartree equation with general nonlinearity: existence, non- existence and variational identities, Adv. Nonlinear Stud., 13 (2013), 799–823. [41] M. Ryznar, Estimate of Green function for relativistic α-stable processes, Potential Analysis, 17 (2002), 1–23. [42] S. Secchi, On some nonlinear fractional equations involving the Bessel potential, J. Dynam. Differential Equations, 29 (2017), 1173–1193. [43] E. Stein, Singular Integrals and Differentiability Properties of Functions, Princeton Mathe- matical Series, No. 30 Princeton University Press, Princeton, N.J. 1970. [44] P. R. Stinga, User’s guide to the fractional Laplacian and the method of semigroups, Handbook of Fractional Calculus with Applications, De Gruyter, Berlin, 2 (2019), 235–265. [45] P. R. Stinga and J. L. Torrea, Extension problem and Harnack’s inequality for some fractional operators, Comm. Partial Differential Equations, 35 (2010), 2092–2122. [46] A. Szulkin and T. Weth, The method of Nehari manifold, Handbook of Nonconvex Analysis and Applications, Int. Press, Somerville, MA, 2010, 597–632. [47] M. H. Taibleson, On the theory of Lipschitz spaces of distributions on Euclidean n-space. I. Principal properties, J. Math. Mech., 13 (1964), 407–479. [48] R. A. Weder, Spectral properties of one-body relativistic spin-zero Hamiltonians, Ann. Inst. H. PoincaréSect. A (N.S.), 20 (1974), 211–220. [49] R. A. Weder, Spectral analysis of pseudodifferential operators, J. Functional Analysis, 20 (1975), 319–337. [50] M. Willem, Minimax Theorems, Progress in Nonlinear Differential Equations and their Ap- plications 24, BirkhäuserBoston, Inc., Boston, MA, 1996. Received December 2020; revised April 2021. E-mail address: [email protected]