Ann. I. H. Poincaré – PR 43 (2007) 481–507 www.elsevier.com/locate/anihpb

Self-intersection local time of (α,d,β)-

L. Mytnik a,2, J. Villa b,∗,1

a Technion-Israel Institute of Technology, Faculty of Industrial Engineering and Management, Haifa 32000, Israel b Universidad Autónoma de Aguascalientes, Departamento de Matemáticas y Física, Av. Universidad 940, C.P. 20100, Aguascalientes, Ags., Mexico Received 6 March 2006; accepted 25 July 2006 Available online 27 February 2007

Abstract The existence of self-intersection local time (SILT), when the time diagonal is intersected, of the (α,d,β)-superprocess is proved − for d/2 <αand for a renormalized SILT when d/(2 + (1 + β) 1)<α d/2. We also establish Tanaka-like formula for SILT. © 2007 Elsevier Masson SAS. All rights reserved. Résumé L’existence de temps locaux d’auto-intersection est établie pour certaines classes de superprocessus, ainsi qu’une formule ana- logue à celle de Tanaka. © 2007 Elsevier Masson SAS. All rights reserved.

MSC: primary 60G57; secondary 60H15

Keywords: Self-intersection local time; Infinite variance superprocess; Tanaka-like formula

1. Introduction and statement of results

This paper is devoted to the proof of existence of self-intersection local time (SILT) of (α,d,β)- for d d 0 <β<1. Let us introduce some notation. Let Bb(R ) (respectively Cb(R )) be the family of all bounded (respec- d d tively, bounded continuous) Borel measurable functions on R , and MF (R ) be the set of all finite Borel measures on Rd . The integral of a function f with respect to a measure μ is denoted by μ(f ).IfE is a metric space we denote by D([0, +∞), E) the space of all càdlàg E-valued paths with the Skorohod topology. We will use c to denote a positive and finite constant whose value may vary from place to place. A constant of the form c(a,b,...) means that this constant depends on parameters a,b,....     Let (Ω , F , F· ,P ) be a filtered probability space where the (α,d,β)-superprocess X ={Xt : t  0} is defined. d That is, by X we mean a MF (R )-valued, time homogeneous, strong Markov process with càdlàg sample paths, such

✩ Research supported in part by the Israel Science Foundation. * Corresponding author. E-mail addresses: [email protected] (L. Mytnik), [email protected] (J. Villa). 1 Thanks go to UAA for allow this postdoctoral stay. 2 Research supported in part by Henri Gutwirth Fund for the Promotion of Research.

0246-0203/$ – see front matter © 2007 Elsevier Masson SAS. All rights reserved. doi:10.1016/j.anihpb.2006.07.005 482 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507

d that for any non-negative function ϕ ∈ Bb(R ), E exp −Xt (ϕ) |X0 = μ = exp −μ Vt (ϕ) ,

d where μ ∈ MF (R ) and Vt (ϕ) denotes the unique non-negative solution of the following evolution equation t 1+β vt = St ϕ − St−s (vs) ds, t  0. 0

Here {St : t  0} denotes the semigroup corresponding to the fractional Laplacian operator α. Another way to characterize the (α,d,β)-superprocess X is by means of the following martingale problem: ⎧ ∈ ∈ M Rd ⎨ For all ϕ D(α) (domain of α) and μ F ( ), = = − − t (1.1) ⎩ X0 μ, and Mt (ϕ) Xt (ϕ) X0(ϕ) 0 Xs(αϕ)ds, F  is a t -martingale.

If β = 1 then M·(ϕ) is a continuous martingale. In this paper we are interested in the case of 0 <β<1, and here Mt (ϕ) is a purely discontinuous martingale. This martingale can be expressed as t

Mt (ϕ) = ϕ(x)M(ds, dx), (1.2) 0 Rd where M(ds, dx) is a martingale measure, it and the stochastic integral with respect to such martingale measure is defined in [11] (or in Section II.3 of [8]). The SILT is heuristically defined by

γX(B) = δ(x − y)Xs(dx)Xt (dy)ds dt, B R2d where B ⊂[0, ∞) ×[0, ∞) is a bounded Borel set and δ is the . Let D ={(t, t): t>0} be the time diagonal on R+ × R+.Forβ = 1, B ∩ D = ∅ and d  7, Dynkin [6] proved the existence of SILT, γX,fora very general class of continuous superprocesses. Also, from the Dynkin’s works follows the existence of SILT when β = 1, B ∩ D = ∅ and d  3 (see [1]). For β = 1, d = 4, 5 and B ∩ D = ∅, Rosen [15] proved the existence of a renormalized SILT for the (α, d, 1)-superprocess. A Tanaka-like formula for the local time of (α, d, 2)-superprocess was established by Adler and Lewin in [3]. The same authors derived a Tanaka-like formula for self-intersection local time for (α, d, 2)-superprocess (see [2]). In this paper we are going to extend the above results for the case of 0 <β<1. The usual way to give a rigorous definition of SILT is to take a sequence (ϕε)ε>0 of smooth functions that converges in distribution to δ, define the approximating SILTs

γX,ε(B) = ϕε(x − y)Xs(dx)Xt (dy)ds dt, B ⊂ R+ × R+, B R2d

2  1+β  1  and prove that (γX,ε(B))ε>0 converges, in some sense (it is usually taken L (P ), L (P ), L (P ), distribution or in probability), to a random variable γX(B). In what follows we choose ϕε = pε, where pε is the α-stable density, given by 1 − · − − | |α p (x, y) = e i(z (x y)) ε z dz, x, y ∈ Rd , ε (2π)d Rd when 0 <α<2 and

1 −|x−y|2/2ε d pε(x, y) = e ,x,y∈ R , (2πε)d/2 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 483 for α = 2. In this paper we will consider the particular case when B ={(t, s):0 s  t  T }. Here we denote γX,ε(B) by γX,ε(T ), that is T t

γX,ε(T ) = pε(x − y)Xs(dx)Xt (dy)ds dt, ∀T  0. 0 0 R2d Moreover, we are going to consider the renormalized SILT T λε λ,ε γ˜X,ε(T ) = γX,ε(T ) − e Xs G (x −·) Xs(dx)ds, 0 Rd where ∞ λ,ε −λt G (x) = e pt (x) dt, λ,ε  0. ε Notice that Gλ,ε(x) ↑ Gλ,0(x) = Gλ(x) as ε ↓ 0 for all x ∈ Rd , and if d>αthen − G(x) = G0,0(x) = c(α,d)|x|α d , (1.3) where ((d − α)/2) c(α,d) = , 2α/2π d/2 (α/2) and is the usual Gamma function. G is called Green function of α. Also notice that, for λ>0, we have λ,ε λ,ε −λε d αG (x) = λG (x) − e pε(x), x ∈ R . (1.4) Now we are ready to present our main result.

Theorem 1. Let X be the (α,d,β)-superprocess with initial measure X0(dx) = μ(dx) = h(x) dx, where h is bounded and integrable with respect to Lebesgue measure on Rd . Let M be the martingale measure which appears in the martingale problem (1.1) for X.

(a) Let d/2 <α. Then there exists a process γX ={γX(T ): T  0} such that for every T>0, δ>0 P sup γX,ε(t) − γX(t) >δ → 0, as ε ↓ 0. tT Moreover, for any λ>0,

T t T λ λ γX(T ) = λ G (x − y)Xs(dx)Xt (dy)dt ds − XT G (x −·) Xs(dx)ds 0 0 R2d 0 Rd T T t λ λ + Xs G (x −·) Xs(dx)ds + G (x − y)M(ds, dy)Xt (dx)dt, a.s. (1.5) 0 Rd 0 Rd 0 Rd −1 (b) Let d/(2 + (1 + β) )<α d/2. Then there exists a process γ˜X ={˜γX(T ): T  0} such that for every T>0, δ>0 P sup γ˜X,ε(t) −˜γX(t) >δ → 0, as ε ↓ 0. tT Moreover, for any λ>0, 484 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507

T t T λ λ γ˜X(T ) = λ G (x − y)Xs(dx)Xt (dy)dt ds − XT G (x −·) Xs(dx)ds 0 0 R2d 0 Rd T t λ + G (x − y)M(ds,dy)Xt (dx)dt, a.s. (1.6) 0 Rd 0 Rd

The processes γX and γ˜X are called SILT and renormalized SILT of X, respectively, and (1.5) and (1.6) are called Tanaka-like formula for SILT.

Remark 1. It is interesting to note that our bound on dimensions − d< 2 + (1 + β) 1 α, for renormalized SILT does not converge, as β ↑ 1, to the bound d<3α established by Rosen [15] for finite variance superprocess (β = 1). For example, simple algebra shows that for 5/3 <α<2, there is a SILT for finite variance superprocess in dimensions d  5. However for any β ∈ ((3α − 5)/(5 − 2α),1) we get the existence of SILT only in dimensions d  4 and this bound not improve to d  5ifβ ↑ 1. So, our bound is more restrictive, and we believe that it is related to the fact that (α,d,β)-superprocess (with β<1) has jumps. Our conjecture is that for β<1, the renormalized SILT defined by (1.6) does not exist in dimensions greater than (2 + (1 + β)−1)α.

The common ways to prove the existence of SILT for the finite variance superprocesses (see e.g. [2,15]) do not work here. The reason for this is that such proofs strongly rely on the existence of high moments of X (atleastof order four), and (α,d,β)-superprocess X has moments of order less than 1 + β. To overcome this difficulty let us consider the path properties of X more carefully. It is well known (see Theorem 6.1.3 of [4]) that, for 0 <β<1, the d (α,d,β)-superprocess X is a.s. discontinuous and has jumps of the form Xt = rδx ,forsomer>0,x∈ R .Here δx denotes the Dirac measure concentrated at x.Let NX(dx, dr, ds) = δ(x,r,s), (1.7) {(x,r,s): Xs =rδx } d be a random point measure on R × R+ × R+ with compensator measure NX given by −2−β NX(dx, dr, ds) = ηr drXs(dx)ds, (1.8) where β(β + 1) η = . (1 − β) Let K>0 fix. From [7] and [4] we have that the (α,d,β)-superprocess X has the following decomposition: Let ϕ ∈ D(α), t  0, t t t K Xt (ϕ) = μ(ϕ) + Xs(αϕ)ds − Cβ (K) Xs(ϕ) ds + rϕ(x)NX(dx, dr, ds) 0 0 0 0 Rd t ∞

+ rϕ(x)NX(dx, dr, ds), (1.9) 0 K Rd where NX = NX − NX is a martingale measure and η C (K) = . β βKβ As we have mentioned already, one of the problems of working with the (α,d,β)-superprocess X is dealing with “big” jumps. In fact, the “big” jumps produce the infinite variance of the process and they appear in the term corresponding L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 485 to the integral with respect to NX on (1.9). So, the first step in the establishing the existence of SILT for (α,d,β)- superprocess X is to “eliminate” those jumps. This is achieved via introducing the following auxiliary process. ◦ = [ ∞ Rd F ◦ = B ◦ F ◦ = { K   } Let us considerer the canonical space, Ω D( 0, ), MF ( )), (Ω ) and t σ Yr :0 r t , K ◦ = ◦ ∈ Rd ◦ F ◦ where Yr (ω ) ω (r). For any μ MF ( ) there exists (see [4]) a measure Qμ on (Ω , ), such that for any d non-negative function ϕ ∈ Bb(R ) − K F ◦ = − K K ∀  Eμ exp Yt (ϕ) s exp Ys Vt−s(ϕ) , 0 0 (1.11) K (notice that the expectation is taken here with respect to the measure Qμ). Vt is the unique non-negative solution for the non-linear equation K K K K ∂v /∂t = (α − Cβ (K))v − Φ (v ), t t t (1.12) K = v0 ϕ, where K − − − ΦK (x) = η e ux − 1 + ux u β 2 du. (1.13) 0 Note that when K =∞the resulting process Y ∞ and the regular (α,d,β)-superprocess X have the same distribution. Now, for any K>0, define the τK = inf t>0: |Xt | >K . (1.14)

In Section 2 we will show that if we define the process which evolves as X up to time τK and then continues to K K ↑∞ evolve as Y starting at XτK −, then this process has the same law as Y . This together with the fact that τK as K →∞(see Lemma 2) implies that it is enough to show existence of the SILT for the process Y K . This task will be accomplished in Section 3, modulo some technical moment estimates that will be derived in Section 4. The main steps leading to the proof of Theorem 1 will be described in the next section.

2. Proof of Theorem 1

The process Y K whose Laplace transform is given by (1.10), (1.11) has the following decomposition: t t t K K = + K − K + ∀  Yt (ϕ) μ(ϕ) Ys (αϕ)ds Cβ (K) Ys (ϕ) ds rϕ(x)NY K (dx, dr, ds), t 0, (2.1) 0 0 0 0 Rd = − where NY K NY K NY K , and = NY K (dx, dr, ds) δ(x,r,s), { K = } (x,r,s): Ys rδx = −2−β K NY K (dx, dr, ds) η1(0,K](r)r drYs (dx)ds.

Note that NY K is defined in a way analogous to (1.7), however it does not have jumps “greater” than K. In the following lemma we are going to construct the probability space where Y K coincides with X up to the stopping time τK .

Lemma 1. There exists a probability space on which a pair of processes (Y¨ K ,X)is defined and possesses the following properties: (a) Y¨ K coincides in law with Y K , ¨ K = ∀ (b) Yt Xt , t<τK . 486 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507

Proof. Define ≡  × ◦ F ≡ F  × F ◦ F ≡ F  × F ◦ Ω Ω Ω , , t t t , and let   ¨ K  ◦ ≡ Xt (w ), t < τK (w ), Yt (w ,w ) ◦   w (t − τK (w )), t  τK (w ). Define the measure P on (Ω, F):   τK (w )   P(B× C) = 1B (w )P (C)P (dw ), Ω where  ◦ ◦  ◦ τK (w ) = ∈ ¨ K ∈ P (C) QX  − w Ω : Y (w ,w ) C . (2.2) τK (ω ) ¨ K Let ϕ ∈ Dom(α) and t>0. From the definition of Y we have t t t K ¨ K = + ¨ K − ¨ K + Yt (ϕ) μ(ϕ) Ys (αϕ)ds Cβ (K) Ys (ϕ) ds rϕ(x)NY¨ K (dx, dr, ds), (2.3) 0 0 0 0 Rd where N ¨ K is defined by (1.7) for t<τK and Y = NY¨ K (dx, dr, ds) δ(x,r,s), { ¨ K = } (x,r,s): Ys rδx  t K F ∈ F  ∈ F ◦ for t τK . Let us check that Rd rϕ(x)NY¨ K (dx, dr, ds) is an t -martingale: For any t>u, B u, C u , 0 0  we obtain by using the definition (2.2) of P τK (w ) t K uK − P 1B×C rϕ(x)NY¨ K (dx, dr, ds) rϕ(x)NY¨ K (dx, dr, ds) 0 0 Rd 0 0 Rd t K uK    = τK (w ) − P 1C rϕ(x)NY¨ K (dx, dr, ds) rϕ(x)NY¨ K (dx, dr, ds) P (dw ) B 0 0 Rd 0 0 Rd  t K (t∧τK(w ))∨uK    = τK (w ) − P 1C rϕ(x)NY¨ K (dx, dr, ds) rϕ(x)NY¨ K (dx, dr, ds) P (dw ) B 0 0 Rd 0 0 Rd  (t∧τK(w ))∨uK uK    + τK (w ) − P 1C rϕ(x)NY¨ K (dx, dr, ds) rϕ(x)NY¨ K (dx, dr, ds) P (dw ) B 0 0 Rd 0 0 Rd t K  ◦   = τK (w ) F P QX  − 1C rϕ(x)N ¨ K (dx, dr, ds) ∧  ∨ P (dw ) τK (w ) Y t τK (w )) u  B (t∧τK (w ))∨u 0 Rd  (t∧τK(w ))∨uK−  τK (w )   + P 1C rϕ(x)NX(dx, dr, ds) P (dw ) B u 0 Rd t K  ◦   = τK (w ) F P 1CQX  − rϕ(x)N ¨ K (dx, dr, ds) ∧  ∨ P (dw ) τK (w ) Y (t τK (w )) u  B (t∧τK (w ))∨u 0 Rd L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 487

 (t∧τK(w ))∨uK−  τK (w )   + P (C) rϕ(x)NX(dx, dr, ds)P (dw ) B u 0 Rd  (t∧τK(w ))∨uK− =   F    1C X·∧u(ω ) 1{u<τK }P rϕ(x)NX(dx, dr, ds) u P (dw ), B u 0 Rd   F where in the last equality for the first term we have used the fact that for P -a.s. ω , N ¨ K is a (QX  − , t )- Y τK (w ) d martingale measure on R × R+ ×[τK ∧ t,t]. As for the second term we have used the simple identity   τK (w ) = P (C)1{u<τK } 1C X·∧u(ω ) 1{u<τK } ∈ F ◦  F  for any C u . Now use the fact that NX is a (P , t )-martingale measure to get that  (t∧τK(w ))∨uK−  F ◦ = P rϕ(x)NX(dx, dr, ds) u 0, u 0 Rd t K and the proof that 0 0 Rd rϕ(x)NY¨ K (dx, dr, ds) is a martingale is complete. Then, due to the uniqueness of the decomposition ([7], Theorem 7) we conclude that Y¨ K has the same distribution as Y K . 2

Convention. Based on the above lemma, from now on we will assume that Y K ,Xare defined on the same probability K = ∀ space and Yt Xt , t<τK .

Now we are going to show that time τK can be made greater than any constant T with probability arbitrary close to 1 by taking K sufficiently large.

Lemma 2. For every T>0 and ε>0, there exists K>0 such that P(τK  T) ε.

K [ ]×Rd K = [ +∞ ×[ ]×Rd Proof. Let ZT the number of jumps of height greater than K in 0,T , that is ZT N( K, ) 0,T ). K Then there exists (see [13], page 1430) a standard Poisson process At such that K = K ZT A −1−β T Rd , cβ K 0 Xs ( ) ds for some positive constant cβ . Then from the Markov inequality we have, P(τ  T)= P ZK  1 K T

= K  P A −1−β T Rd 1 cβ K 0 Xs ( ) ds

T cβ − = P AK  1, X Rd ds  K 1 −1−β T Rd 1+β s cβ K 0 Xs ( ) ds K 0 T cβ − + P AK  1, X Rd ds

T  −β Rd + − K = cβ K Eμ Xs 1 P AK−1 0 0 d −β −1 = cβ Tμ R K + 1 − exp −K . The result follows, since the right-hand side goes to 0 as K →∞. 2

Now the proof of Theorem 1 relies on the following proposition.

K K = = Proposition 1. Let K>0 and Y be the truncated (α,d,β)-superprocess with initial measure Y0 (dx) μ(dx) h(x) dx, where h is bounded and integrable with respect to Lebesgue measure dx on Rd .

(a) For d/2 <αthere exists a process γ K ={γ K (T ): T  0} such that Y K Y K K − K = ∀ lim E sup γY K ,ε(t) γY K (t) 0, T>0, ε↓0 t0,

T t T K = − λ − K K − K λ −· K γY K (T ) λ Cβ (K) G (x y)Ys (dx)Yt (dy)dt ds YT G (x ) Ys (dx) 0 0 R2d 0 Rd T T t + K λ −· K + λ − K K Ys G (x ) Ys (dx)ds G (x y)M (ds,dy)Yt (dx)dt, a.s. 0 Rd 0 Rd 0 Rd

(b) For d/(2 + (1 + β)−1)<α d/2 there exists a process γ˜ K ={˜γ K (T ): T  0} such that Y K Y K ˜ K −˜K = ∀ lim E sup γY K ,ε(t) γY K (t) 0, T>0, ε↓0 t0,

T t T ˜ K = − λ − K K − K λ −· K γY K (T ) λ Cβ (K) G (x y)Ys (dx)Yt (dy)dt ds YT G (x ) Ys (dx)ds 0 0 R2d 0 Rd T t + λ − K K G (x y)M (ds,dy)Yt (dx)dt, a.s. 0 Rd 0 Rd

Proof is postponed. The above proposition immediately yields:

= K Proof of Theorem 1. Fix arbitrary ε, δ > 0 and let d/2 <α. Since Xt Yt for any t<τK , we immediately get that = ∀ γX(t) γY K (t), t<τK , and γX(t) satisfies Tanaka formula (1.5) for t<τK . Moreover, since by Lemma 2, τK ↑∞,asK →∞, there is no problem to define γX(t) satisfying (1.5) for any t>0. Now let us check the convergence part of the theorem. For any T>0, by Lemma 2, we can fix K>0 such that P(τK  T) δ. Then L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 489 lim P sup γX,ε (t) − γX(t) >ε  lim P sup γX,ε (t) − γX(t) >ε,τK >T + P(τK  T) ↓ 1 ↓ 1 ε1 0 tT ε1 0 tT  lim P sup γ K (t) − γ K (t) >ε,τK >T + δ ↓ Y ,ε1 Y ε1 0 tT = δ, and since δ,ε > 0 were arbitrary the proof of convergence is complete. The proof of part (b) of the theorem goes along the same lines. 2

3. Existence of SILT for Y K — proof of Proposition 1

K { K  } Fix arbitrary K>0. First, we derive very useful moment estimates for Y .Let St : t 0 denote the solution of the partial differential equation ∂vK t =  − C (K) vK . ∂t α β t { K  } That is, St : t 0 is the semigroup defined as − K = Cβ (K)t St e St . (3.1) K  Notice that St ϕ St ϕ, for all non-negative bounded measurable functions ϕ. d Following Theorem 3.1 of [9] we have that for ϕ,ψ ∈ Bb(R ), t − K − K K = = − K E exp Yt (ϕ) Ys (ψ) ds Y0 μ exp μ Vt (ϕ, ψ) , (3.2) 0 ∈ M Rd K where μ F ( ) and Vt (ϕ, ψ) denotes the unique non-negative solution to the following evolution equation t t K = K + K − K K K  vt St ϕ Ss (ψ) ds St−s Φ vs ds, t 0, (3.3) 0 0 where ∞ (−1)m ΦK (x) = χ(m)xm (3.4) m! m=2 and ηKm−1−β χ(m)= . (3.5) m − 1 − β Now we are going to calculate the first two moments of Y K .

d Lemma 3. Let ϕ be a non-negative function on Bb(R ) and t>0. Then K = K Eμ Yt (ϕ) μ St ϕ , (3.6) t K 2 = K 2 + K K 2 Eμ Yt (ϕ) μ St ϕ χ(2)μ St−s Ss ϕ ds . 0

Proof. From (3.2) we have −λY K (ϕ) −μ(vK (λ)) Eμ e t = e t (3.7) 490 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 where t K = K − K K K vt (λ) λSt ϕ St−s Φ vs (λ) ds. (3.8) 0 Using the elementary inequality e−x − 1 + x  x2/2, x  0, and (1.13) we have χ(2) ΦK (x)  x2,x 0. 2 ·  K  K    Let ∞ be the supremum norm, then 0 vt (λ) λSt ϕ λ ϕ ∞ and the previous inequality implies t  2 K K K χ(2) ϕ ∞t 2 S − Φ v (λ) ds  × λ . t s s 2 0 Further from (3.8) we get

t K − K λSt ϕ vt (λ) = 1 K K K lim lim St−s Φ vs (λ) ds λ↓0 λ λ↓0 λ 0 2 χ(2)ϕ∞t  lim × λ = 0, λ↓0 2 and we write this like K = K − vt (λ) λSt ϕ o(λ). (3.9) This implies

− K − [ λYt (ϕ)] K = 1 Eμ e Eμ Yt (ϕ) lim λ↓0 λ − K + 1 − e λμ(St ϕ) o(λ) = lim λ↓0 λ − K + 1 − e λμ(St ϕ) o(λ) λμ(SK ϕ) − o(λ) = lim lim t = μ SK ϕ . ↓ K − ↓ t λ 0 λμ(St ϕ) o(λ) λ 0 λ Now, to calculate the second moment we follow the ideas used in the proof of Proposition 11 of Chapter II from [10]: 2 2 − K K = λYt (ϕ) − + K Eμ Yt (ϕ) lim Eμ e 1 λYt (ϕ) λ↓0 λ2 2 − K = μ(vt (λ)) − + K lim e 1 λμ St ϕ λ↓0 λ2 2 − K + t K K K = λμ(St ϕ) μ( 0 St−s (Φ (vs (λ))) ds) − + K lim e 1 λμ St ϕ λ↓0 λ2 t ∞ n = 2 1 K K K − K − + K lim μ St−s Φ vs (λ) ds λμ St ϕ 1 λμ St ϕ . λ↓0 2 ! λ = n n 0 0 Using the series expansion (3.4) for ΦK and (3.9) we obtain

t t K K K = K K K − St−s Φ vs (λ) ds St−s Φ λSs ϕ o(λ) ds 0 0 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 491

t K χ(2) 2 K 2 2 = S − λ S ϕ + o λ ds t s 2! s 0 t χ(2) 2 K K 2 2 = λ S − S ϕ ds + o λ . 2 t s s 0 Then t K 2 = 2 χ(2) 2 K K 2 + 2 Eμ Yt (ϕ) lim λ μ St−s Ss ϕ ds o λ λ↓0 λ2 2 0 t 2 1 χ(2) 2 K K 2 K 2 + λ μ S − S ϕ ds − λμ S ϕ + o λ 2! 2 t s s t 0 t = 2 χ(2) 2 K K 2 + 1 2 K 2 + 2 lim λ μ St−s Ss ϕ ds λ μ St ϕ o λ λ↓0 λ2 2 2 0 t = K K 2 + K 2 χ(2)μ St−s Ss ϕ ds μ St ϕ , 0 and we are done. 2

Remark 2. Using binary directed graphs, Dynkin in [6] gives a formula for the moments of superprocesses, where the Laplace functional (3.2) has an evolution equation (3.3) with only one term m = 2 in (3.4). For the Y K superprocess it is also possible, but here the main difference is that the directed graphs are not necessarily binary.

d Corollary 1. Let ϕ,ψ be non-negative functions on Bb(R ) and t  s>0. Then s K K = K K + K K K Eμ Yt (ϕ)Ys (ψ) μ St ϕ μ Ss ψ χ(2)μ Sr St−r ϕSs−r ψ dr . 0

Proof. First, use the for Y K to get K |F = K K Eμ Yt (ϕ) s Ys St−sϕ . Therefore K K = K K K Eμ Yt (ϕ)Ys (ψ) Eμ Ys St−sϕ Ys (ψ) 1 K K K 2 K K K 2 = E Y S − ϕ + Y (ψ) − E Y S − ϕ − Y (ψ) , 4 μ s t s s μ s t s s and we are done by Lemma 3. 2

d d Corollary 2. Let ϕ be non-negative functions on Bb(R × R ) and t  s>0. Then K K Eμ ϕ(x,y)Yt (dx)Ys (dy) d d R×R

= μ(dx1)μ(dx2)pt (z1 − x1)ps(z2 − x2)ϕ(z1,z2) dz1 dz2 R4d 492 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507

s

+ χ(2) μ(dx)pr (y − x)dypt−r (z1 − y)ps−r (z2 − y)ϕ(z1,z2) dz1 dz2 dr. 0 R4d Proof. Use Corollary 1 and approximation of the ψ(x,y) by functions in the form i ϕi(x)φi(y) to derive the result. We leave the details to the reader. 2

Next proposition gives bounds on some fractional moments of Y K and requires much more work than we have done in Lemma 3. Hence its proof will be postponed till Section 4.

Proposition 2. Let 1 + β

Remark 3. When K goes to infinity, then χ(2) goes to infinity, hence c(K,p,d,α,β) goes to infinity and this is because χ(2) is part of c(K,p,d,α,β). The moment in (3.10) is infinity when K =+∞, because the (α,d,β)- superprocess has moments of order less than 1 + β and p>1 + β.

Proof is postponed. Now we can write the Tanaka-like formula for the approximating SILT of the truncated superprocess Y K .From Fubini theorem, (1.4), (1.2) and (1.1) (the martingale problem for the truncated superprocess Y K ,[7])wehave T t K = − K K γY K ,ε(T ) pε(x y)Ys (dx)Yt (dy)ds dt 0 0 R2d T t = λε λ,ε − K K λe G (x y)Ys (dx)Yt (dy)dt ds 0 0 R2d T T − λε λ,ε − K K e αG (x y)Yt (dy)dtYs (dx)ds 0 Rd s Rd T t = − λε λ,ε − K K λ Cβ (K) e G (x y)Ys (dx)Yt (dy)ds dt 0 0 R2d T T − λε K λ,ε −· K + λε K λ,ε −· K e YT G (x ) Ys (dx)ds e Ys G (x ) Ys (dx)ds 0 Rd 0 Rd T T + λε λ,ε − K K e G (x y)M (dt,dy)Ys (dx)ds, (3.11) 0 Rd s Rd L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 493 and T ˜ K = K − λε K λ,ε −· K γY K ,ε(T ) γY K ,ε(T ) e Ys G (x ) Ys (dx)ds 0 Rd T t T = − λε λ,ε − K K − λε K λ,ε −· K λ Cβ (K) e G (x y)Ys (dx)Yt (dy)ds dt e YT G (x ) Ys (dx)ds 0 0 R2d 0 Rd T T + λε λ,ε − K K e G (x y)M (dt,dy)Ys (dx)ds. (3.12) 0 Rd s Rd Note that stochastic integrals in (3.11) and (3.12) are well defined due to the moment bound given by Proposition 2.

Proof of Proposition 1. We are going to prove Proposition 1 via letting ε → 0 in (3.11), and checking convergence of all the terms. By Corollary 2 and simple estimates we get

T K λ,ε −· K E Ys G (x ) Ys (dx)ds 0 T λ  μ(dx1)μ(dx2)ps(z1 − x1)ps(z2 − x2)G (z1 − z2) dz1 dz2 ds 0 R4d T s λ + χ(2) μ(dx)pr (y − x)dyps−r (z1 − y)ps−r (z2 − y)G (z1 − z2) dz1 dz2 dr ds 0 0 R4d T ∞ −λu  μ(dx1)μ(dx2) e pu(z1 − z2)ps(z1 − x1)ps(z2 − x2) dz1 dz2 du ds 0 R4d 0 T s ∞ −λu + χ(2) μ(dx)pr (y − x)dy e pu(z1 − z2)ps−r (z1 − y)dz1ps−r (z2 − y)dz2 du dr ds 0 0 R4d 0 T ∞ −λu  e pu+2s(x1 − x2)μ(dx1)μ(dx2) du ds 0 0 T s ∞ −λu + χ(2) μ(dx)pr (y − x)dy e pu+2s−2r (0) du dr ds 0 0 0 T s ∞ −1 −λu −d/α  h∞μ(1)T λ + μ(1)χ(2) e (u + 2s − 2r) du dr ds, ∀T>0, (3.13) 0 0 0 where the last integral is convergent if d<2α. Using (3.13), the bound Gλ  Gλ,ε and the monotone convergence theorem to get t K λ −· − λ,ε −· K lim E sup Ys G (x ) G (x ) Ys (dx)ds ε↓0 t

T  K λ −· − λ,ε −· K lim E Ys G (x ) G (x ) Ys (dx)ds ε↓0 0 = 0, ∀T>0. (3.14) In a similar way we can prove that T t λ − − λ,ε − K K = lim E sup G (x y) G (x y) Ys (dx)Yt (dy)ds dt 0, (3.15) ε↓0 T0 and d<3α. Now let us deal with the stochastic integral T F ε(t, x)MK (dt, dx), 0 where t ε = λ,ε − K F (t, x) G (x y)Ys (dy)ds. 0 Rd This integral is well defined if (see [11]) 1/2 ε K 2 +∞ E F t,Yt < , (3.17) t∈J ∩[0,T ] where J denotes the set of all jump times of X.Let 1 d<α 2 + , (3.18) 1 + β hence we can choose p ∈ (1 + β,2) such that 1 d<α 2 + . (3.19) p Since p ∈ (1 + β,2) we can use the Jensen inequality to get p/2  p/2 ai ai , i∈I i∈I if ai  0 for all i ∈ I . This yields 1/2 1/p ε K 2  ε K p E F t,Yt E F t,Yt t∈J ∩[0,T ] t∈J ∩[0,T ] T K 1/p = −β−2 ε p K E η u F (t, uδx) duYt (dx)dt 0 0 T t p 1/p = λ,ε − K K c E G (x y)Ys (dy)ds Yt (dx) dt . 0 0 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 495

Since p satisfies (3.19), the condition (3.17) follows from Proposition 2. Let F = F 0. By Burkholder–Davis–Gundy inequality (see [11]) and the previous argument we get t − ε K E sup F(s,x) F (s, x) M (ds, dx) tT 0 t  E sup F(s,x)− F ε(s, x) MK (ds, dx) tT 0 1/2  − ε K 2 cE F F t,Yt t∈J ∩[0,T ] T t p 1/p  λ − λ,ε − K K c E G G (x y) Ys (dy)ds Yt (dx) dt 0 0 → 0, as ε ↓ 0, ∀T>0, (3.20) where the last convergence follows by Proposition 2 and the monotone convergence theorem. Now combine (3.14)– (3.16), (3.18) and (3.20) to get that all the terms in (3.11) converge and the proof of part (a) is complete. By (3.15), (3.16), (3.18) and (3.20) we get that all the terms in (3.12) converge and hence the part (b) of the proposition fol- lows. 2

4. Proof of Proposition 2: estimation of fractional moments

In what follows we will use the following well known equalities. For p ∈ (1, 2) ∞ p−1 −λz −p z = ηp 1 − e λ dλ (4.1) 0 and ∞ p −λz −p−1 z = pηp e − 1 + λz λ dλ (4.2) 0 where p − 1 η = . p (2 − p)

Proposition 3. Let 1 < 1 + β0. (4.3) 0 0 0

Proof. Fix an arbitrary t>0. By (4.2) we obtain

t ∞ p − − − t K K K = p 1 K λ 0 Ys (ψ) ds Eμ Yt (ϕ) Ys (ψ) ds pηp λ E Yt (ϕ) e 0 0 t − K + K K E Yt (ϕ) λE Yt (ϕ) Ys (ψ) ds dλ. (4.4) 0 Now we will bound the moments on the right-hand side of the above expression. First of all, by Corollary 1, we have

t t t s K K = K K + K K K Eμ Yt (ϕ) Ys (ψ) ds μ St ϕ μ Ss ψ ds χ(2)μ Sr St−r ϕSs−r ψ dr ds . (4.5) 0 0 0 0 Moreover, from Fubini theorem we get the following useful equality t s t r K K K = K K K Sr St−r ϕSs−r ψ dr ds St−r Sr ϕ Ss ψ ds dr. (4.6) 0 0 0 0 Now let us estimate the remaining moment. Use the Laplace transform (3.2) and the dominated convergence theorem to obtain − t K 1 − t K − K − t K K λ 0 Ys (ψ) ds =− λ 0 Ys (ψ) ds εYt (ϕ) − λ 0 Ys (ψ) ds Eμ Yt (ϕ) e lim Eμ e e ε↓0 ε 1 − K − K =−lim e μ(Vt (εϕ,λψ)) − e μ(Vt (0,λψ)) . ε↓0 ε From (3.2), we can easily derive that N  N  ∀  Vs (ϕ, ψ) Vs (0,ψ) 0, s 0, (4.7) and hence by the dominated convergence theorem, we get K → K ↓ Vt (εϕ, λψ) Vt (0,λψ), as ε 0. Using the same argument we get K K − t K − K V (εϕ, λψ) − V (0,λψ) K λ 0 Ys (ψ) ds = μ(Vt (0,λψ)) t t E Yt (ϕ) e e μ lim . ε↓0 ε K Following the argument in Section 6.3 of [5] we can show that Ut (ϕ, λψ) defined by K − K K = Vt (εϕ, λψ) Vt (0,λψ) Ut (ϕ, λψ) lim ε↓0 ε L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 497 satisfies the following equation, t K = K − K K K  K Ut (ϕ, λψ) St ϕ St−s Us (ϕ, λψ) Φ Vs (0,λψ) ds, (4.8) 0 with Φ(x) = dΦ(x)/dx. Use (3.6), (4.4)–(4.6) and (4.8) to get t p K K E Yt (ϕ) Ys (ψ) ds 0 ∞ t − K = μ(Vt (0,λψ)) K + K K pηp e μ Ut (ϕ, λψ) λμ St ϕ μ Ss ψ ds 0 0 t s − K + K K K −p−1 μ St ϕ λχ(2)μ St−s Ss ϕ Sr ψ dr ds λ dλ 0 0 ∞ t − t K = K λμ( 0 Ss ψ ds) − K + K K pηpμ St ϕe St ϕ λSt ϕμ Ss ψ ds 0 0 t s − K − t K + K μ(Vt (0,λψ)) − K λμ( 0 Ss ψ ds) + K K K St ϕe St ϕ e λχ(2) St−s Ss ϕ Sr ψ dr ds 0 0 t − K  − − − μ(Vt (0,λψ)) K K K K p 1 e St−s Us (ϕ, λψ) Φ Vs (0,λψ) ds λ dλ 0 t p = K K + + + μ St ϕ μ Ss ψ ds I1 I2 I3 0 where ∞ − K − t K − − = K μ(Vt (0,λψ)) − λμ( 0 Ss ψ ds) p 1 I1 pηpμ St ϕ e e λ dλ , 0 ∞ t s = K K K I2 pηpμ λχ(2) St−s Ss ϕ Sr ψ dr ds 0 0 0 t − K K K  K −p−1 St−s Us (ϕ, λψ) Φ Vs (0,λψ) ds λ dλ , 0 ∞ t − K  − − = − μ(Vt (0,λψ)) K K K K p 1 I3 pηpμ 1 e St−s Us (ϕ, λψ) Φ Vs (0,λψ) dsλ dλ . 0 0 By the elementary inequality 1 − e−x  x,forx  0, and (3.5) we have −  K1 β ΦK (x)  η x = χ(2)x. (4.9) 1 − β 498 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507

K  Using (3.3) and (4.7) it is easy to derive that Ut (ϕ, λψ) 0 and K  K Ut (ϕ, λψ) St ϕ, (4.10) t K  K Vt (0,λψ) λ Ss ψ ds. (4.11) 0 The above inequalities and (4.8) yield the following bound on I3: ∞t − K − −  K K K − μ(Vt (0,λψ)) p 1 I3 cμ St−s Ss ϕVs (0,λψ) ds 1 e λ dλ 0 0 t s ∞ − t K −  K K K − λμ( 0 Sl ψ dl) p cμ St−s Ss ϕ Sr ψ dr ds 1 e λ dλ 0 0 0 t p−1 t s  K K K K c μ Ss ψ ds μ St−s Ss ϕ Sr ψ dr ds , (4.12) 0 0 0 where the last inequality follows by (4.1). Let us take care of I2. By (4.8) we get ∞ t s t = K K K − K K K  K I2 pηpμ λχ(2) St−s Ss ϕ Sr ψ dr ds St−s Ss ϕ Φ Vs (0,λψ) ds 0 0 0 0 t s + K K  K K K K  K −p−1 St−s Φ Vs (0,λψ) Ss−r Ur (ϕ, λψ) Φ Vr (0,λψ) dr ds λ dλ 0 0 = J1 + J2, (4.13) where ∞ t s t = K K K − K K K  K −p−1 J1 pηpμ λχ(2) St−s Ss ϕ Sr ψ dr ds St−s Ss ϕ Φ Vs (0,λψ) ds λ dλ , 0 0 0 0 ∞t s = K K  K K K K  K −p−1 J2 pηpμ St−s Φ Vs (0,λψ) Ss−r Ur (ϕ, λψ) Φ Vr (0,λψ) dr dsλ dλ . 0 0 0 Let us estimate J2. First, by (1.13), (4.1) and (4.2) we obtain ∞ ∞K  − − K − − − K K p = − wVs (0,λψ) β 1 p Φ Vs (0,λψ) λ dλ η 1 e w dwλ dλ 0 0 0 K∞ −λw s S ψ dr −p −β−1  η 1 − e 0 r λ dλw dw 0 0 K s p−1 = −1 K −β−1 ηηp w Sr ψ dr w dw 0 0 s p−1 = K c Sr ψ dr . 0 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 499

Use this and (4.9)–(4.11) to get ∞t s  K K  K K K K −p−1 J2 cμ St−s Φ Vs (0,λψ) Ss−r Sr ϕVr (0,λψ) dr dsλ dλ 0 0 0 t s r ∞  K K K K K  K −p cμ St−s Ss−r Sr ϕ Su ψ du dr Φ Vs (0,λψ) λ dλ ds 0 0 0 0 t s r s p−1  K K K K K cμ St−s Ss−r Sr ϕ Su ψ du dr Sl ψ dl ds . (4.14) 0 0 0 0

Now let us estimate J1: ∞ t s = K K K J1 pηpμ λχ(2) St−s Ss ϕ Sr ψ dr 0 0 0 t K − K − − − − − K K − wVs (0,λψ) β 1 p 1 St−s Ss ϕη 1 e w dw ds λ dλ 0 0 t ∞ s K − K − − − − = K K K + wVs (0,λψ) − β 1 p 1 pηpμ St−s Ss ϕ λχ(2) Sr ψ dr η e 1 w dw λ dλ ds . 0 0 0 0 Using the identity K − χ(2) = η w β dw, 0 we obtain t ∞K s − K − − − − = K K wVs (0,λψ) − + K β 1 p 1 J1 pηpμ St−s Ss ϕη e 1 λw Sr ψ dr w dwλ dλ ds 0 0 0 0 t K ∞ s − s − − − − = K K λw 0 Sr ψ dr − + K p 1 β 1 pηηpμ St−s Ss ϕ e 1 λw Sr ψ dr λ dλw dw 0 0 0 0 K ∞ −wV K (0,λψ) −λw s SK ψ dr −p−1 −β−1 + e s − e 0 r λ dλw dw ds 0 0 t K s p = K K K −β−1 μ St−s Ss ϕη w Sr ψ dr w dw 0 0 0 t K∞ − K − s K − − − − + K K wVs (0,λψ) − λw 0 Sr ψ dr p 1 β 1 pηηpμ St−s Ss ϕ e e λ dλw dw ds 0 0 0 t s t p = K K K + K K cμ St−s Ss ϕ Sr ψ dr ds pηηpμ St−s Ss ϕQ(s) ds , (4.15) 0 0 0 500 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 where K∞ −wV K (0,λψ) λw s SK ψ dr −p−1 −β−1 Q(s) = e s − e 0 r λ dλw dw 0 0 K ∞ s  K − K −p−1 −β−1 wVs (0,λψ) λw Sr ψ drλ dλw dw 0 0 0 K ∞ s = + K − K −p−1 c Vs (0,λψ) Sr (λψ) drλ dλ 0 K 0 = (Q1 + Q2)(s). By (3.3) Ks = K K K −p−1 Q1(s) c Ss−r Φ Vr (0,λψ) drλ dλ. 0 0  Due to 1 < 1 + β

Ks   K K p −p−1 Q1(s) c Ss−r Vr (0,λψ) drλ dλ 0 0  Ks r p  K K −p−1 c Ss−r Su (λψ) du drλ dλ 0 0 0  s r p = K K c Ss−r Su ψ du dr. (4.16) 0 0

Apply triangle inequality and (4.11) to bound Q2: ∞ s = K − K −p−1 Q2(s) c Vs (0,λψ) Sr (λψ) drλ dλ K 0 ∞s  K −p−1 2c Sr (λψ) drλ dλ K 0 s = K c Sr ψ dr. (4.17) 0

Finally, let us estimate I1. Proceeding as before we have K ∞ − K − t K − − = K + μ(Vt (0,λψ)) − λμ( 0 Ss ψ ds) p 1 I1 pηpμ St ϕ e e λ dλ 0 K L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 501 K t ∞  K K − K −p−1 + K −p−1 pηpμ St ϕ μ Vt (0,λψ) μ Ss ψ ds λ dλ pηpμ St ϕ 2λ dλ 0 0 K K t   K K K p −p−1 + K cμ St ϕ μ St−s Vs (0,λψ) ds λ dλ cμ St ϕ 0 0 t s  p = K K K + K cμ St ϕ μ St−s Sr ψ dr ds cμ St ϕ . (4.18) 0 0 Combining (4.12)–(4.18) and (3.6) we obtain (4.3). 2

Now, the proof of Proposition 2 is based on the bounds that we will get on all the terms on the right-hand side of (4.3).

Lemma 4. Let μ(dx) = h(x) dx, where h is bounded and integrable. Then λ sup μ SsG (··−x)  c4.19(h, λ) < ∞. (4.19) x∈Rd

Proof. Using the explicit expression for μ we have, λ λ μ SsG (··−x) = ps(y − z)G (z − x)dzμ(dy) λ  ps(y − z)G (z − x)dzh∞ dy λ −1 = ∞ = ∞ h G 1 h λ , recall that ·∞ is the supremum norm. 2

In the next two lemmas we are going to use the following basic inequalities: For d>αand δ ∈ (0, 1),wehave ([12], Lemma 4) δ−1 α−d−αδ d pt (x)  ct |x| ,t>0,x∈ R \{0}, (4.20) and the Riesz convolution formula − − + − |x − z|a d |z − y|b d dz = c|x − y|a b d , (4.21)

Rd whenever a,b > 0, a + b

Lemma 5. Let α 0 are constants.

Proof. First let us prove (4.22) for the case α − δα < a. Use (4.20) and (4.21) to get 502 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 −a −a ps(y − z)|z − x| dz = ps(y − x − z)|z| dz Rd Rd −a = + ps(y − x − z)|z| dz | | | | z >1 z 1 − − − −  1 + sδ 1|y − x − z|α d δα|z| a dz |z|1 − − −  1 + csδ 1|y − x|α a δα − − − − − = 1 + csδ 1 |y − x|α a δα1 |y − x|  1 +|y − x|α a δα1 |y − x| > 1 − − −  1 + csδ 1 |y − x|α a δακ(y − x) + 1 . Now, suppose α − δα  a. Using a simple coupling argument, as in Lemma 5.1 of [14], we have −a −a ps(y − x − z)|z| dz  ps(z)|z| dz. Rd Rd By the scaling relationship −d/α −1/α d pt (x) = t p1 t x ,t>0,x∈ R , (4.23) we get −a −a/α −1/α −1/α −a −d/α ps(z)|z| dz = s p1 s z s z s dz Rd Rd −a/α −a = s p1(z)|z| dz. Rd Therefore, −a −a/α ps(y − z)|z − x| dz  cs 1 |s|  1 + 1 |s| > 1 Rd −  csδ 1 + c, and we are done. 2

∈ ∞ Lemma 6. For any δ (0, 1) there exists c(t) such that for any T>0, supt

Proof. Let α

+ ∞ Lemma 7. Let 1 0, supt

Proof. Since d<α(2 + 1/q) it is easy to check that we can fix δ ∈ (0, 1) sufficiently small such that, q(2α − d − δα) + α − δα > 0. (4.27) By Lemma 6, s q λ q(2α−d−δα) Sr G (·−x)dr (y)  sup c(s) |y − x| κ(y − x) + 1 . (4.28) st 0 Now take a =−q(2α − d − δα).Ifa<0 then the result follows trivially, due to the fact that then the right-hand side of (4.28) is uniformly bounded for any y,x ∈ Rd . If a  0, we apply again Lemma 5 to conclude that the result follows if q(2α − d − δα) + α − δα > 0. But this is exactly the condition (4.27) which is satisfied due to the choice of δ. 2

K  Proof of Proposition 2. From (3.1) we see that St St , hence Proposition 3 implies t p K ·− K λ ··− Eμ Yt pε( x) Ys G ( x) ds dx Rd 0 t p λ  c μ St pε(·−x) dx + μ St pε(·−x) μ SsG (··−x)ds dx Rd Rd 0 t s  p λ + μ St pε(·−x) μ St−s Sr G (··−x)dr ds dx Rd 0 0 t s λ + μ St−s Sspε(·−x) Sr G (··−x)dr ds dx Rd 0 0 t s t p−1 λ λ + μ St−s Sspε(·−x) Sr G (··−x)dr ds μ SsG (··−x)ds dx Rd 0 0 0 t s r s p−1 λ λ + μ St−s Ss−r Sr pε(·−x) SuG (··−x)du dr Sr G (··−x)dr ds dx Rd 0 0 0 0 t s p λ + μ St−s Sspε(·−x) Sr G (··−x)dr ds dx Rd 0 0  t s r p 8 λ + μ St−s Sspε(·−x) Ss−r SuG (··−x)du dr ds dx = c Ii(ε). = Rd 0 0 0 i 1 504 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507

We will check the boundedness of all the terms Ii(ε), i = 1,...,8. First note, that for d  α all the terms Ii(ε), i = 1,...,8, can be bounded very easily, and we leave it to check to the reader. We will consider the case α

I3(ε) + I8(ε)  μ(1)c(t).

For I7(ε) we get the following t s p λ I7(ε) = pt−s(y − z)ps+ε(z − x) Sr G (z − x)dr ds dx dzμ(dy) 0 Rd ×Rd ×Rd 0 t s p λ = pt−s(y − z) dzSs+ε Sr G (·) dr (0) dsμ(dy) 0 Rd ×Rd 0 t s p λ = μ(1) Ss+ε Sr G (·) dr (0) ds 0 0  μ(1)c(t), (4.29) where the last inequality follows by Lemma 7. It is also easy to check that t s p λ I4(ε)  μ St−s Sspε(·−x) Sr G (··−x)dr + 1 ds dx Rd 0 0  I7(ε) + μ(1)t, (4.30) and  p−1 I5(ε) c4.19 I4(ε).

The last term we have to handle is I6(ε): t s r λ I6(ε)  + pt−s(y − y1) ps−r (y1 − z)pr+ε(z − x) SuG (z − x)du dr

R3d x: |y1−x|1 x: |y1−x|>1 0 0 0 s p−1 λ × Sr G (y1 − x)dr ds dy1 dz dxμ(dy) 0 = I6,1(ε) + I6,2(ε). Our condition on d implies that we can choose δ ∈ (0, 1/3) sufficiently small such that (2α − d)(p + 1) − δα(p + 2)>−d. (4.31) By (4.20), Lemmas 5 and 6 we get t s δ−1 I6,1(ε)  c(t) pt−s(y − y1) ps−r (y1 − z)(r + ε) R3d | − | x: y1 x 1 0 0 3α−2d−2δα α−d−δα (2α−d−δα)(p−1) × |z − x| κ(z− x) +|z − x| dr |y1 − x| + 1 ds dy1 dz dxμ(dy) L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 505

t s δ−1 = c(t) pt−s(y − y1) (r + ε) R2d x: |y −x|1 0 0 1 3α−2d−2δα α−d−δα × ps−r (y1 − z) |z − x| κ(z− x) +|z − x| dz dr Rd (2α−d−δα)(p−1) × |y1 − x| + 1 ds dy1 dxμ(dy)

t s δ−1  c(t) pt−s(y − y1) (r + ε) R2d x: |y −x|1 0 0 1 δ−1 4α−2d−3δα 2α−d−2δα × 1 + (s − r) |y1 − x| +|y1 − x| + 1 dr (2α−d−δα)(p−1) × |y1 − x| + 1 ds dy1 dxμ(dy) t s δ−1 = c(t) pt−s(y − y1) (r + ε) dr R2d 0 0 s (2α−d−δα)(p−1) δ−1 δ−1 × 1 +|y1 − x| dx + (r + ε) (s − r) dr x: |y −x|1 0 1 (2α−d−δα)(p−1)+4α−2d−3αδ × 1 +|y1 − x|

x: |y1−x|1 − − − + − − − − − +|y − x|(2α d δα)(p 1) 2α d 2αδ +|y − x|(2α d δα)(p 1) 1 1 4α−2d−3αδ 2α−d−2αδ +|y1 − x| +|y1 − x| dx dy1μ(dy)

 c(t)μ(1), where the last inequality follows by (4.31). As for the I6,2(ε),byLemma6weget t p−1 t λ I6,2(ε)  sup Sr G (y1 − x)dr pt−s(y − y1) x,y1: |y1−x|>1 0 R3d 0 s r ∞ −λv × ps−r (y1 − z)pr+ε+u+v(0) e dv du dr ds dy1 dzμ(dy) 0 0 0 r s r ∞ − −  c(t)μ(1) (r + ε + u + v) d/α e λv dv du dr ds 0 0 0 0 and the last integral is bounded if d<3α. By combining all the above estimates we are done with the first part of the proposition. Now we are going to prove the second part of the proposition. Take t p = λ,ε − K ∈ Rd ϕ(y) G (y x)Ys (dx)ds ,y . 0 506 L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507

For each n ∈ N define the truncations functions, ϕn = ϕ ∧ n. Then 0  ϕn ↑ ϕ,asn → 0. Since ϕn is bounded and K p1(z) dzYt (dy) is a finite measure, we have by the dominated convergence theorem and the scaling relationship (4.23) the following estimation K 1/α K ϕn(x)Y (dx) = lim ϕn δ z + x p (z) dzY (dx) t ↓ 1 t δ 0  lim inf ϕ δ1/αz + x p (z) dzYK (dx) ↓ 1 t δ 0 = − K lim inf pδ(x y)Yt (dx)ϕ(y)dy. δ↓0 Letting n →∞, by the monotone convergence theorem, we have t t p p K λ,ε ··− K  K λ,ε ··− K ·− Ys G ( x) ds Yt (dx) lim inf Ys G ( x) ds Yt pδ( x) dx. δ↓0 0 0 From the Fatou lemma we get t t p p K λ,ε ··− K  K λ,ε ··− K ·− E Ys G ( x) ds Yt (dx) lim inf E Ys G ( x) ds Yt pδ( x) dx . δ↓0 0 0 Since Gλ,ε  Gλ we have by the already proven part of Proposition 2, t t p p K λ,ε ··− K  K λ ··− K ·− E Ys G ( x) ds Yt (dx) lim inf E Ys G ( x) ds Yt pδ( x) dx δ↓0 0 0  c(K,p,d,α,β). Using once again the monotone convergence theorem, as ε → 0, we arrive at t p K λ ··− K E Ys G ( x) ds Yt (dx)

Acknowledgements

The second author J.V. would like to express his gratitude for the opportunity to spend some time as a postdoctoral fellow at the Technion (Haifa, Israel) where this research was done.

References

[1] R.J. Adler, Superprocess local and intersection local times and their corresponding particle pictures, in: Seminar on Stochastic Processes, 1992, Birkhäuser, 1993, pp. 1–42. [2] R.J. Adler, M. Lewin, An evolution equation for the intersection local times of superprocesses, in: M.T. Barlow, N.H. Bingham (Eds.), Stochastic Analysis, 1991, pp. 1–22. [3] R.J. Adler, M. Lewin, Local time and Tanaka formulae for super and super stable processes, . Appl. 41 (1992) 45–67. [4] D. Dawson, Measure-valued Markov processes, in: École d’Été de Probabilitiés de Saint Flour XXI, in: Lecture Notes in Math., vol. 1541, Springer, Berlin, 1993, pp. 1–260. [5] D. Dawson, Infinitely divisible random measure and superprocesses, in: Stochastic Analysis and Related Topics, Birkhäuser, Boston, 1992, pp. 1–130. [6] E. Dynkin, Representation for functionals of superprocesses by multiples stochastic integrals, with applications to self-intersection local times, Astérisque 157–158 (1988) 147–171. L. Mytnik, J. Villa / Ann. I. H. Poincaré – PR 43 (2007) 481–507 507

[7] N. El Karoui, S. Roelly, Propriétés de martingales, explosion et représentation de Lévy–Khintchine d’une classe de processus de branchement à valeurs measures, Stochastic Process. Appl. 38 (1991) 239–266. [8] K. Ikeda, S. Watanabe, Stochastic Differential Equations and Diffusion Processes, North-Holland Publishing Company and Kodansha Ltd., 1981. [9] I. Iscoe, A weighted occupation time for a class of measure-valued branching processes, Probab. Theory Related Fields 71 (1986) 85–116. [10] J.F. Le Gall, Spatial Branching Processes Random Snakes and Partial Differential Equations, Birkhäuser, 1999. [11] J.F. Le Gall, L. Mytnik, Stochastic integral representation and regularity of the density for the exit measure of super-Brownian motion, Ann. Probab. 33 (1) (2005) 194–222. [12] L. Mytnik, Collision measure and collision local time for (α,d,β)-superprocesses, J. Theoret. Probab. 11 (3) (1998) 733–763. [13] L. Mytnik, E. Perkins, Regularity and irregularity of (1 + β)-stable super-Brownian motion, Ann. Probab. 31 (3) (2003) 1413–1440. [14] L. Mytnik, E. Perkins, A. Sturm, On pathwise uniqueness for stochastic heat equations with non-Lipschitz coefficients, Ann. Probab. 34 (5) (2006) 1910–1959. [15] J. Rosen, Renormalizations and limit theorems for self-intersections of superprocesses, Ann. Probab. 20 (3) (1992) 1341–1368.