Interacting Superprocesses with Discontinuous Spatial Motion

Forum. Math. Galley Proof, 41 pages Forum Mathematicum DOI 10.1515/FORM.2011.100 © de Gruyter 2011

Interacting superprocesses with discontinuous spatial motion

Zhen-Qing Chen, Hao Wang and Jie Xiong Communicated by Ichiro Shigekawa

Abstract. A class of interacting superprocesses arising from branching particle systems with continuous spatial motions, called superprocesses with dependent spatial motion (SDSMs), has been introduced and studied by Wang and by Dawson, Li and Wang. In this paper, we extend the model to allow discontinuous spatial motions. Under Lipschitz condition for coefﬁcients, we show that under a proper rescaling, branching particle systems with jump-diffusion underlying motions in a random medium converge to a measure- valued process, called stable SDSM. We further characterize this stable SDSM as a unique solution of a well-posed martingale problem. To prove the uniqueness of the martingale 2 problem, we establish the C C -regularity for the transition semigroup of a class of jump- diffusion processes, which may be of independent interest. Keywords. Superprocess, Brownian sheet, interaction, symmetric stable process, frac- 2 tional Laplacian, C C -regularity of transition semigroup, branching mechanism, scaling limit, duality method, martingale problem. 2010 Mathematics Subject Classiﬁcation. Primary 60J68, 60K37; secondary 60G52, 60J80.

1 Introduction

Dawson–Watanabe superprocesses arise naturally as the scaling limit of branching processes where each particle moves independently of each other. In this paper, we are concerned with the scaling limit of an interacting branching particle system where the motion of each particle is governed by the following equation: for each i N, 2 Z t Z tZ Z t i i i i i i i zt z0 c.zs /dBs h.y zs /W.dy; ds/ b.zs / dSs ; D 0 C 0 R C 0 (1.1)

The research is supported in part by NSF Grants DMS-0600206 and DMS-0906743 (first author), by a research Grant of University of Oregon (second author) and by NSF Grant DMS-0906907 (third author). 2 Z.-Q.Chen, H. Wang and J. Xiong where Bi i 1 are independent standard Brownian motions on R, W is a ¹ I º Brownian sheet on R2 (see definition below) and S i i 1 are independent ¹ I º 1-dimensional symmetric ˛-stable processes with ˛ .0; 2/. The processes W , 2 Bi i 1 and S i i 1 are assumed to be independent of each other. Equa- ¹ I º ¹ I º tion (1.1) says that each particle is subject to a common random medium force, which is described by the Brownian sheet W , and each particle has its own jump- diffusion dynamics, which are described by an individual Brownian motion and an individual symmetric stable process. We will show in this paper that under suit- able conditions and scaling, the branching particle system converges weakly to a measure-valued process, which we call an interacting superprocess. We will further show that this measure-valued process is the unique solution to a related martingale problem. When b 0 (that is, there is no stable-noise motion S i in the par- D ticle system), such a model has been studied in Wang [32,33], which has been further investigated in subsequent papers by Dawson et al. [10] and Li et al. [24, 25]. However, many physical, economic and biological systems are better modeled by using discontinuous stochastic processes because the jumps exhibited by discontinuous processes reveal sudden big changes when viewed in a long time scale. Thus it is natural and meaningful to study the scaling limit of interacting branching processes with discontinuous spatial motion such as those described by (1.1). The presence of jumps introduces many new challenges. For example, to establish the uniqueness of the martingale problem of the interacting superprocess, one needs to establish the C 2-regularity for the semigroup of the n-particle system of (1.1). However unlike the diffusion case, due to the presence of (discontinuous) stable Lévy motion, the infinitesimal of such a semigroup is a non-local operator. Establishing regularity results for pseudo-differential operators is typically a challenging problem and is, in fact, of current research interest (see, e.g., [2], es- pecially item 6 of its Section 7.7). One of the main results of this paper is to show that under certain conditions there is some .0; 1/ so that for every n 1, 2 2 n the semigroup of the n-particle system of (1.1) maps Cb C .R / into itself. The following is a more detailed description on the content of this paper. Let L2.R/ be the Hilbert space of square-integrable functions on R and let m m Bb.R / be the space of bounded Lebesgue measurable functions on R . Denote by C.Rm/ and C j .Rm/ the space of continuous functions on Rm and the space of continuous functions on Rm with continuous derivatives up to and including m k m order j , respectively. Denote by Cb.R / and Cb .R / the space of bounded continuous functions on Rm and the space of bounded continuous functions that have bounded derivatives up to and including order k, respectively. We use Lip.R/ to denote the space of Lipschitz functions on R; that is, f Lip.R/ if there is a 2 constant M > 0 such that f .x/ f .y/ M x y for every x; y R. The class j j Ä j j 2 of bounded Lipschitz functions on R will be denoted by Lipb.R/. Interacting superprocesses with discontinuous spatial motion 3

Under the condition that b; c Lip .R/ and h L2.R/ Lip.R/, we will 2 b 2 \ show that the interacting stochastic differential equation (SDE) (1.1) has a unique strong solution. From the strong solution of (1.1), we can construct a family of interacting branching particle systems. Let > 0 and > 1 be ﬁxed constants. .n/ i For n 1, suppose that initially there are m0 number of particles located at z0, .n/ 1 i m , and each has mass n. These particles evolve according to Ä Ä 0 (1.1) and branch independently at rate n. The branching mechanism for each particle is assumed to be state independent, independent to each other and iden- tically distributed. After branching, the offsprings of each particle evolves independently according to (1.1) and then branches again. The common nth-stage branching mechanism q.n/ q.n/ k 0; 1; : : : is assumed to be critical (that WD ¹ k I D º is, the average number of offspring is 1), and it cannot produce 1 or more than n n Pm.n/ number of children. Under the assumption that the initial state ı k of k 1 z0 D.n/ the particles converges to a measure 0 and that the branching function q converges uniformly to a limiting branching function having ﬁnite second moment n P and some additional conditions, we show that the empirical process ı k k 1 zt converges to a measure-valued process. By Itô’s formula and the independence of motions and branching, we will show that the limiting measure-valued process has the following formal generators (usually called pregenerators):

LF ./ AF ./ BF ./; (1.2) WD C Z 2 1 2 ı F ./ BF ./ 2 .dx/ (1.3) WD 2 R ı.x/ and

Z Â 2 Ã 1 d ˛ ˛=2 ıF ./ AF ./ 2 a.x/ 2 b.x/ .dx/ WD R dx C j j ı.x/ (1.4) 1 Z Z @2 Â ı2F ./ Ã .x y/ .dx/.dy/ C 2 R R @x@y ı.x/ı.y/ for F ./ D.L/ C.MF .R//. Here > 0 is related to the branching rate of 2 the particle system and 2 > 0 is the variance of the limiting offspring distribution; MF .R/ denotes the Polish space of all ﬁnite measures on R with weak topology, C.MF .R// is the space of all continuous functions on MF .R/. The variational derivative is deﬁned by

ıF ./ F . "ıx/ F ./ lim C (1.5) ı.x/ WD " 0 " I # 4 Z.-Q.Chen, H. Wang and J. Xiong the fractional Laplacian ˛=2 on C 2.R/ is deﬁned for C 2.R/, b 2 b Z ˛=2 c˛ .x/ .x / .x/ 0.x/1 1 1 ˛ d (1.6) WD R 0 C ¹j jÄ º C n¹ º j j and Z .x/ h.y x/h.y/dy; a.x/ c2.x/ .0/; (1.7) WD R WD C and D.L/, the domain of the pregenerator L, consists of functions of the form

F ./ f . 1; ;:::; ; /; D h i h k i 2 2 k with i C .R/, f C .R /, k N . In the usual models (for example, 2 c 2 b 2 .˛; d; ˇ/-superprocesses, see Dawson [8], Dynkin [13] or Dynkin et al. [14]) in which the motions of particles are independent and the motions are independent of branching, the particle systems have the following multiplicative property. If two branching Markov processes evolve independently, with initial state m1 and m2 respectively, then their sum has the same distribution as the branching process with initial state m1 m2. It is well known that the log-Laplace functional (or C evolution equation) technique can be applied to these models in order to construct the limiting measure-valued process. However, for our model and pregenerator, it is obvious that the motions of particles are not independent and this destroys the multiplicative property. Thus, just as in Wang [33] and Dawson et al. [11], the usual log-Laplace functional method is not applicable to our new model. In order to construct the branching particle system, we show that under Lipschitz condition on functions c, h and b, SDE (1.1) has a strong solution and the solution is pathwise unique. Since the symmetric ˛-stable process S i is not square-integrable and is a martingale only when ˛ .1; 2/, the Picard’s successive approximation 2 method is not directly applicable to (1.1) and a truncation procedure on the jumps of S i is needed. To prove the uniqueness of the martingale problem for L for the measure-valued interacting process, we use a duality method due to Dawson and Kurtz [9]. To apply Dawson and Kurtz’s duality method in our context, we need to find, for every 2 m m integer m 1, a linear subspace D of Cb .R / so that Pt D D for every m t > 0. Here Pt t 0 is the transition semigroup of the underlying motion of ¹ I º 2 m m-particles given by (1.1). We show in this paper that we can take C C .R / for such D for some .0; 1/. That is, we show that there is some .0; 1/ so m 2 2 m 2 m 2 that P f C C .R / for each fixed t > 0 and f C C .R /. This result t 2 b 2 b has its own independent interest and is one of the main results of this paper. For earlier regularity results for non-local operators, we refer the reader to [2, 6, 28] Interacting superprocesses with discontinuous spatial motion 5 and the references therein. Note that the infinitesimal generator of m-particles .z1; : : : ; zm/ of (1.1) is given by (see the proof of (3.4) below)

.m/ L f .x1; : : : ; xm/ m 1 X @2 ij .x1; : : : ; xm/ f .x1; : : : ; xm/ WD 2 @xi @xj i;j 1 D m X ˛ ˛=2 b.xj / f .x1; : : : ; xm/ C j j xj j 1 D m (1.8) 1 X @2 ij .x1; : : : ; xm/ f .x1; : : : ; xm/ D 2 @xi @xj i;j 1 D m X Z Â f .x1; : : : ; xj w; : : : ; xm/ f .x1; : : : ; xm/ C C j 1 R 0 D X¹ º Ã ˛ @ c˛ b.xj / f .x1; : : : ; xm/w1 w 1 j 1 ˛j dw; @xj ¹j jÄ º w j j C where ´ a.xi / if i j , ij .x1; : : : ; xm/ D (1.9) WD .xi xj / if i j . ¤ ˛=2 Here xj f is the fractional Laplacian deﬁned in (1.6) applied to the function

xj f .x1; : : : ; xj ; : : : ; xm/: 7! 2 m The C C -regularity of Pt will be proved by a perturbation method, treating Pm ˛ ˛=2 .m/ the non-local operator part j 1 b.xj / xj of L as a perturbation of the 1 PDmj j @2 elliptic differential operator 2 i;j 1 ij .x/ @x @x . D i j In this paper, to better illustrate our approach to the study of interacting superprocesses whose underlying spatial motions are discontinuous, we restrict our- selves to a speciﬁc type of jump-diffusions for the spatial motions given by (1.1). However the method of this paper works for a more general class of jump-diffu- R t i sions. For example, there can be a drift term 0 r.zs/ds in (1.1) and the symmetric stable process S i can be replaced by some other Lévy process of pure jump type such as mixed stable processes. Moreover, S i i 1 can even be replaced by ¹ I º a certain family of independent random measures (see [20]). Furthermore, the spatial motions zi in (1.1) can also be multidimensional. 6 Z.-Q.Chen, H. Wang and J. Xiong

While the component of ˛-stable noise in the spatial motion is captured by the fractional Laplacian ˛=2 in the expression (1.4) of A for the pregenerator L of the interacting superprocess, the superprocess itself has continuous trajectories. This is because in this paper, we have critical branching at every stage and the branching function q.n/ converges to a limiting branching function in which the offspring distribution has a finite second moment. These assumptions imply that the limiting interacting measure-valued process has null Lévy (jumping) measure for its branching mechanism and therefore the superprocess is continuous. This is in analog to the case of Dawson–Watanabe process that it has continuous paths if and only if the Lévy (jumping) measure for the branching mechanism is zero (see Fitzsimmons [18] and El Karoui–Roelly [15]). If the offspring distribution determined by the limiting branching function of q.n/ has infinite second moment, for example, a stable branching mechanism or a general branching mechanism, then the limiting measure-valued process is discontinuous. For the latter case, we believe that the conditional log-Laplace functional would be a good tool to construct the limiting measure-valued process. See [8] and [25] for related work in this direction on interacting superprocesses with continuous spatial motion. The remainder of this paper is organized as follows. Section 2 is devoted to establish the strong existence and pathwise uniqueness for solutions of (1.1). This is the basis for the construction of the branching particle systems. In Section 3, we construct branching particle system. The tightness of the corresponding empir- n ical measure-valued processes t t 0 is established in Section 4. We prove 2 ¹ I º n the L -convergence of each term in the the decomposition of ; t and then 0 h i derive the corresponding decomposition of ; t for any weak sequential limit 0 n h i t t 0 of t t 0 n 1. The latter implies that the measure-valued pro- ¹ I º ¹ I º cess 0 t 0 solves the martingale problem for the generator .L; D.L// ¹ t I º of (1.2). Moreover, we show that 0 t 0 is a continuous process taking ¹ t I º values in MF .R/. We then use Dawson–Kurtz’s duality method to show that the martingale problem for .L; D.L// is well-posed. This in particular implies n that t 0 converges weakly in the Skorokhod space D.Œ0; /; MF .R// ¹ t I º 1 to a continuous measure-value process that solves the martingale problem for L. 2 .m/ The proof of the C C -regularity property of the semigroup Pt , which is used in a crucial way in the proof of well-posedness of the aforementioned martingale problem, is given in Section 5. For the reader’s convenience, let us recall the definition of the Brownian sheet W on R. For a D Rm or D R R , let B.D/ be the Borel -field D D C on D. By abusing the notation, the Lebesgue measure on R and on R2 will all be denoted by m. Let .; F ; Ft t 0; P/ be a filtered probability space with a right ¹ º Interacting superprocesses with discontinuous spatial motion 7

continuous filtration Ft t 0. A random set function W on B.R R / defined ¹ º C on .; F ; Ft t 0; P/ is called a Brownian sheet or a space-time white noise on ¹ º R if (i) For A B.R R / having finite Lebesgue measure, W.A/ is a Gaussian 2 C random variable with mean zero and variance m.A/.

(ii) If Ai B.R R /, i 1; 2, have ﬁnite Lebesgue measure and satisfy 2 C D A1 A2 , then W.A1/ and W.A2/ are independent and \ D;

W.A1 A2/ W.A1/ W.A2/ P-a.s. [ D C (iii) For every A B.R/ having ﬁnite Lebesgue measures, 2

M.A/t W.A Œ0; t/; WD

as a process in t 0, is a square-integrable Ft -martingales. ¹ º In fact, it follows from (i) and (ii) that for every A B.R/ having finite 2 Lebesgue measures, t M.A/t is a centered Gaussian process with indepen- 7! dent stationary increments with EŒM.A/2 m.A/t. Thus it has a continuous t D modification and is a Brownian motion in R. Condition (iii) above puts in an additional requirement that M.A/ be an Ft -martingale. It is a consequence of ¹ º above (ii) and (iii) that for every A; B B.R/ with finite Lebesgue measures, the 2 covariance process for martingales M.A/ and M.B/ satisfies

M.A/; M.B/ t m.A B/ t; t 0: h i D \ For more detailed information on Brownian sheet, the reader is referred to Walsh [31, Chapter 2] and Dawson [8, Section 7.1].

2 Strong existence and pathwise uniqueness of underlying motions

Recall that a one-dimensional symmetric stable process of index ˛ .0; 2 is a 2 Lévy process S St t 0 such that WD ¹ I º h i ˛ E ei.St S0/ e t for every R: D j j 2 Note that when ˛ 2, S is just a Brownian motion with speed running twice D fast as the standard Brownian motion. More generally, given a ﬁltered probability space .; F ; Ft t 0; P/, a process S St t 0 is said to be a Ft -sym- ¹ º WD ¹ I º ¹ º 8 Z.-Q.Chen, H. Wang and J. Xiong

metric ˛-stable process if St is Ft -measurable for every t 0 and for every t > r > 0, St Sr is independent of Fr and h i ˛ E ei.St Sr / e .t r/ for every R: D j j 2

c˛ The Lévy measure for such a process is given by w 1 ˛ dw, where c˛ is a constant depending only on ˛. Its role can be understoodj j C in the following Lévy system formula for a one-dimensional Ft -symmetric ˛-stable process S. For any ¹ º Ft -predictable process H Ht t 0 and any Borel measurable function F ¹ º 2 R t R WD ¹ I º c˛ on R satisfying 0 R HuF .Su;Su w/ w 1 ˛ dw du < , we have for every j C j C 1 t > 0, j j Z tZ X c˛ t HuF .Su ;Su/ HuF .Su;Su w/ dw du (2.1) w 1 ˛ 7! u t 0 R C C Ä j j is a local martingale with respect to the filtration Ft t 0. It is a martingale with ¹ º respect to the filtration Ft t 0 if in addition H is a bounded process and ¹ º Ä t Z Z c˛ E HuF .Su;Su w/ 1 ˛ dwdu < R j C j w 1 0 j j C for every t > 0. Here for any process with jumps we use Xs limt s Xt to WD " denote the left hand limit and Xs Xs Xs to denote the jump at time s. See, 4 WD for example, [4], [26], [20] and [30] for more information on the stochastic analysis of processes with jumps. The L2-infinitesimal generator K of a symmetric ˛-stable process is the fractional Laplacian . /˛=2. It can be defined in terms of Fourier transform as follows: ° ± Dom.K/ f L2.R/ ˛fb ./ L2.R/ ; WD 2 W j j 2 Kf ./ ˛fb ./ for f Dom.K/: WD j j 2 2 R ix Here for f L .R/, fb ./ e f .x/dx. It can be shown (see, e.g., [3]) 2 WD R that C 2.R/ Dom.K/ and Kf ˛=2f for f C 2.R/, where the latter is c D 2 c defined by (1.6). b Theorem 2.1. Let h L2.R/ Lip.R/, b; c Lip .R/. Let W be a Brown- 2 \ 2 b ian sheet, B Bt t 0 be a standard one-dimensional Brownian motion WD ¹ I º and S St t 0 be a one-dimensional symmetric ˛-stable process with WD ¹ I º ˛ .0; 2/. Assume that W , B and S are defined on a common filtered probabil- 2 ity space .; F ; Ft t 0; P/ and are independent of each other. Then for every ¹ º Interacting superprocesses with discontinuous spatial motion 9

F0-measurable random starting point Z0 R the stochastic integral equation 2 Z t Z t Z tZ Zt Z0 c.Zs /dBs b.Zs / dSs h.y Zs /W.dy; ds/ D C 0 C 0 C 0 R (2.2) has a unique strong solution.

Proof. Note that the symmetric ˛-stable process S is not square-integrable and is a martingale only when ˛ .1; 2/. So the standard Picard’s successive approxi- 2 mation method does not work here. However, it can be made to work with a truncation argument. Such a truncation method is known to experts. However, for the reader’s convenience, we will spell out the details of the proof. We make the convention S0 S0. With the above notations, it is well known D that X bS t St Ss1 Ss >1 ; t 0; (2.3) WD s t 4 ¹j j º Ä c˛ is a Lévy process with Lévy measure w 1 ˛ 1 w 1 dw and that bS and S bS are C ¹j jÄ º independent (cf. [4, pp. 13–15]). Notej thatj the process bS is a pure jump, square- integrable martingale and

bS t St for t < T1 inf r > 0 Sr > 1 : (2.4) D WD ¹ W j4 j º We now use Picard’s successive approximation method to construct a strong solution Y for (2.2) but with bS in place of S. Later on, we will use Y to construct the strong solution for (2.2). .1/ For any given real-valued F0-measurable random variable Z0, let Y Z0 t WD for t 0, and deﬁne for k 2 and t 0, Z t Z t .k/ .k 1/ .k 1/ Y Z0 c.Y /dBs b.Y / dbS s t WD C s C s 0 0 (2.5) Z tZ .k 1/ h.y Ys /W.dy; ds/: C 0 R Thus for k 3 and t 0, we have Z t .k/ .k 1/ .k 1/ .k 2/ Á Yt Yt c.Ys / c.Ys / dBs D 0 Z t .k 1/ .k 2/ Á b.Ys / b.Ys / dbS s C 0 Z .k 1/ .k 2/ Á h.y Ys / h.y Ys / W.dy; ds/: C R 10 Z.-Q.Chen, H. Wang and J. Xiong

As the three terms on the right hand side are square-integrable martingales that are independent to each other, by Doob’s maximal inequality we have

Ä ˇ ˇ2 ˇ .k/ .k 1/ˇ fk.t/ E sup ˇYs Ys ˇ Á 0 s t Ä Ä Ä 2 ˇ .k/ .k 1/ˇ 12E ˇY Y ˇ Ä ˇ t t ˇ ÄZ t 2 .k 1/ .k 2/ Á 12E c.Ys / c.Ys / ds D 0 ÄZ tZ 2 2 .k 1/ .k 2/ Á c˛ w 12E b.Ys / b.Ys / j1 j˛ 1 w 1 dwds C R w ¹j jÄ º 0 j j C ÄZ tZ 2 .k 1/ .k 2/Á 4E h.y Ys / h.y Ys dy ds C 0 R ÄZ t 2 2 2 2 .k 1/ .k 2/Á c c0 b0 h0 E Ys / Ys ds Ä k k1 C k k1 C k k1 0 Z t c1 fk 1.s/ds; (2.6) Ä 0 where is the supremum norm. A similar calculation shows that k k1 Ä ˇ ˇ2 Äˇ ˇ2 ˇ .2/ .1/ˇ ˇ .2/ .1/ˇ E sup ˇYs Ys ˇ 4E ˇYt Yt ˇ 0 s t Ä Ä Ä ÄÂZ t Z t 4E c.Z0/dBs b.Z0/dbS s D 0 C 0 Z tZ Ã2 h.y Z0/W.dy; ds/ C 0 R Ä Z 2 2 2 c˛ w 4E c.Z0/ t b.Z0/ j1 j˛ 1 w 1 dw D C R w ¹j jÄ º j j C Z tZ 2 h.y Z0/ dyds C 0 R 2 2 c2 . c b .0// t c3 t: Ä k k1 C k k1 C DW It then follows from above that, for n 3, Ä ˇ ˇ2 t n 1 ˇ .n/ .n 1/ˇ n 1 E sup ˇYs Ys ˇ c4 ; 0 s t Ä .n 1/Š Ä Ä Interacting superprocesses with discontinuous spatial motion 11

where c4 c1 c3. This implies that for every t > 0, D _ v u ÄÂ Ã2 u X1 ˇ .n/ .n 1/ˇ tE sup ˇYs Ys ˇ ˇ ˇ n 1 0 s t D Ä Ä s Ä 2 X1 ˇ .n/ .n 1/ˇ E sup ˇYs Ys ˇ < : Ä ˇ ˇ 1 n 1 0 s t D Ä Ä P ˇ .n/ .n 1/ˇ n Therefore 1 sup ˇYs Ys ˇ converges a.s. and consequently Y n 1 0 s t converges uniformlyD onÄ eachÄ Œ0; t to a càdlàg process Y a.s. Clearly we also have from above that for every t > 0,

Ä ˇ ˇ2 ˇ .n/ ˇ lim E sup ˇYs Ysˇ 0: n 0 s t D !1 Ä Ä Thus we deduce by the same argument as that for (2.6) that

Z t Z t Z tZ .k 1/ .k 1/ .k 1/ c.Ys /dBs b.Ys / dbS s h.y Ys /W.dy; ds/ 0 C 0 C 0 R converges in L2 as k to ! 1 Z t Z t Z tZ c.Ys /dBs b.Ys / dbS s h.y Ys /W.dy; ds/: 0 C 0 C 0 R This implies by (2.5) that

Z t Z t Z tZ Yt z c.Ys /dBs b.Ys / dbS s h.y Ys /W.dy; ds/: (2.7) D C 0 C 0 C 0 R

Moreover, Y is the unique strong solution for (2.7). Indeed, suppose Yb is another strong solution of (2.7). Then for every t 0, Z t Á Z t Á Yt Ybt c.Ys / c.Ybs / dBs b.Ys / b.Ybs / dbS t D 0 C 0 Z Á h.y Ys / h.y Ybs / W.dy; ds/: C R By a similar argument as that for (2.6), we have

Ä ˇ ˇ2 Ä ˇ ˇ2 ˇ ˇ ˇ ˇ E sup ˇYs Ybsˇ c5t E sup ˇYs Ybsˇ : 0 s t Ä 0 s t Ä Ä Ä Ä 12 Z.-Q.Chen, H. Wang and J. Xiong

ˇ ˇ It follows that sup0 s t ˇYs Ybsˇ 0 a.s. for every t Œ0; 1=c5/, and therefore Ä Ä D 2 for every t Œk=c5; .k 1/=c5/. This proves that Y Yb a.s. 2 C D Recall the stopping time T1 in (2.4). Define ´ Yt if t < T1; Zt WD YT1 b.YT1 /.ST1 ST1 / if t T1: C D Then, in view of (2.3), Z is the unique strong solution for SDE (2.2) over the random time interval Œ0; T1. For k 1, define Tk Tk 1 T1 Tk 1 ; WD C ı where t t > 0 are shift operators on the canonical path space . In other ¹ I ºth words, Tk is the k time that the symmetric ˛ stable process S has a jump of size larger than 1. Since t St is right continuous and has left hand limit, S can only 7! have finite many jumps of size larger than 1 over any compact time intervals. So limk Tk almost surely. !1 D 1 Note that BT1 t BT1 t 0 is a Brownian motion, ST1 t ST1 t 0 ¹ C I º ¹ C I º is symmetric ˛-stable process and W ..y1; y2; ŒT1;T1 t/ t 0 is a Brow- ¹ C I º nian sheet that are independent to each other and are independent of FT1 . Sup- 2 pose Y t 0 is the strong solution of (2.7) but with ZT in place of Z0, and ¹ t I º 1 BT1 t BT1 t 0 , ST1 t ST1 t 0 and W ..y1; y2; ŒT1;T1 t/ in ¹ C I º ¹ C I º C place of B, S and W ..y1; y2; Œ0; t/ there, respectively. Define

´ 2 Yt if t < T1 T1 ; ZT1 t 2 2 ı C WD Y b.Y /.S S / if t T1 : T1 T T1 T T2 T2 T1 ı 1 C ı 1 D ı

It is easy to see that such deﬁned Z solves SDE (2.2) for t T2. Iterating this Ä procedure, we can deﬁne Z on Œ0; T for every k 1 and hence on Œ0; /. Thus k 1 we have obtained a strong solution Z for (2.2) for t Œ0; /. Such a strong 2 1 solution is unique as it is unique on each time interval ŒTk 1;Tk. Here we take T0 0. D

3 Branching particle systems

In order to construct branching particle systems, ﬁrst we need to introduce an index set to identify each particle in the branching tree structure. Let be the < set of all multi-indices, i.e., strings of the form n1n2 : : : n , where the ni ’s D k are non-negative integers. Let denote the length of . We provide with j j < the arboreal ordering: m1m2 : : : mp n1n2 : : : nq if and only if p q and Ä Interacting superprocesses with discontinuous spatial motion 13

m1 n1; : : : ; mp np. If p, then has exactly p 1 predecessors, which D D j j D we shall denote respectively by 1, 2; : : : ; 1. For example, with j j C 6879, we get 1 687, 2 68 and 3 6. D D D D Let B be an independent family of standard Brownian motions in R, ¹ I 2 <º S be an independent family of one-dimensional symmetric ˛-stable ¹ I 2 <º processes with ˛ .0; 2/ and W be a Brownian sheet. Assume further that W , 2 B and S are deﬁned on a common ﬁltered probability space ¹ I 2 <º ¹ I 2 <º .; F ; Ft t 0; P/, and independent of each other. For every index and ¹ º 2 < initial data z0, by Theorem 2.1 there is a unique strong solution zt for Z t Z t z z c.z /dB b.z / dS t D 0 C s s C s s 0 0 (3.1) Z tZ h.y zs /W.dy; ds/: C 0 R

Since the strong solution of (3.1) only depends on the initial state z0, the Brownian motion B Bt t 0 , the symmetric ˛-stable process S St t 0 WD ¹ I º WD ¹ I º and the common white noise W , we can write z ˆ.z ;B ;S ; t/ for some t D 0 measurable real-valued map ˆ (omitting W in the notation as it is selected and ﬁxed once and for all).

Lemma 3.1. There is a constant c > 0 such that for every C 2.R/, 2 b ˛=2 c 00 : k k1 Ä k k1 C k k1 Proof. By the mean-value theorem, we have ˇZ ˇ ˛=2 ˇ c˛ ˇ .x/ supˇ .x w/ .x/ 0.x/w1 w 1 1 ˛ dwˇ k k1 D x Rˇ R 0 C ¹j jÄ º w C ˇ X¹ º j j Â2 Z Z Ã 1 ˛ 1 ˛ c 00 w dw w dw Ä k k1 w 1 j j C k k1 w >1 j j j jÄ j j < : 1 This proves the lemma.

Deﬁne X S S S 1 ; bt t s Ss >1 WD s t 4 ¹j4 j º Ä which is a Lévy process independent of S bS . Note that zt b.zt / St . 4 D 4 14 Z.-Q.Chen, H. Wang and J. Xiong

So by the property of Lévy measure (see (2.1)), we have for every C 2.R/, 2 b ; X h i M .z / .z / 0b .z / S 1 t s s s s Ss 1 WD s t 4 ¹j4 jÄ º Ä Z t ÂZ Á zs b.zs /w .zs / 0 R 0 C X¹ º Ã Á c˛ 0b .zs / w1 w 1 1 ˛ dw ds ¹j jÄ º w j j C is a square-integrable martingale with

Ä 2 ÄZ tZ ; Á E Mt E .zs b.zs /w/ .zs / D 0 R C 2 Á c˛ .0b/.zs /w1 w 1 1 ˛ dwds ¹j jÄ º w j j C ÄZ t ÂZ 1 2 2 (3.2) E 1 00b w 1 w 1 zs @ Ä 0 ¹ 6D º R 2k k1 j j ¹j jÄ º Ã 2 Á c˛ 2 1 w >1 1 ˛ dw ds C k k1 ¹j j º w j j C 2 c b 00 t: Ä k k1 C k k1 1 ˛ Observe that due to the symmetry of the kernel c˛= w , j j C Z ˛=2 c˛ .z/ .z w/ .z/ 0.z/w1 w h 1 ˛ dw D R 0 C ¹j jÄ º w C X¹ º j j for every h > 0. So by a change of variable, we have Z c˛ .x b.x/w/ .x/ .0b/.x/w1 w 1 1 ˛ dw R 0 C ¹j jÄ º w C X¹ º j j b.x/ ˛˛=2.x/: D j j It follows that

; X Á M .z / .z / 0b .z / S 1 t s s s s Ss 1 D s t 4 ¹j4 jÄ º Ä (3.3) Z t ˛ ˛=2 Á b .zs /ds: 0 j j Interacting superprocesses with discontinuous spatial motion 15

With C 2.R/, we have by Itô’s formula that for every t > 0, 2 b Z t Z tZ .zt / .z0/ 0c .zs /dBs 0.zs /h.y zs /W.dy; ds/ D 0 C 0 R Z t Z t 1 2 0b .zs / dSs ..0/ c .zs //00.zs /ds C 0 C 2 0 C X h i .zs / .zs / 0b .zs / Ss C s t 4 Ä Z t Z tZ 0c .zs /dBs 0.zs /h.y zs /W.dy; ds/ D 0 C 0 R Z t Z t 1 0b .zs / dbS s a00 .zs /ds C 0 C 2 0 X h i .z / .z / 0b .z / S 1 s s s s Ss 1 C s t 4 ¹j4 jÄ º Ä Z t Z tZ 0c .zs /dBs 0.zs /h.y zs /W.dy; ds/ D 0 C 0 R Z t ; 0b .zs / dbS s Mt C 0 C Z t 1 ˛ ˛=2 Á 2 a00 b .zs /ds: (3.4) C 0 C j j We now consider the branching particle systems in which each particle’s spatial motion is modeled by the SDE (3.1). For every positive integer n 1, there is an n n initial system of m0 particles. Each particle has mass and branches independently at rate n. The branching mechanism is assumed to be state independent. n Let qk denote the probability of having k offsprings. We assume that qn 0 if k 1 or k n 1; k D D C and n X n n kq 1 and lim sup q pk 0; k D n j k j D k 0 !1 k 0 D where p ; k 0; 1; 2; : : : is the limiting offspring distribution which is assumed ¹ k D º to satisfy following conditions:

X1 X1 2 p1 0; kp 1 and m2 k p < : D k D WD k 1 k 0 k 0 D D 16 Z.-Q.Chen, H. Wang and J. Xiong

Let mn Pn .k 1/4qn. Here mn n 1 may be unbounded, but we c WD k 0 k ¹ c I º assume that D mn lim c 0 for any 2. n 2n D !1 We will see that the limiting offspring distribution is the offspring distribution of the Stable SDSM, the limiting measure-valued process we will construct. We n n assume that the initial number of particles m0 , where > 0 and 2 are n Pn 2 n Ä„2 n „ 2 ﬁxed constants. Deﬁne m2 k 0 k qk , n m2 1 and m2 1. 2 2 WD D WD WD Note that n and are the variance of the n-stage and the limiting offspring 2 2 2 distribution, respectively. We have n < and limn n . 1 !1 D Let C be a family of i.i.d. real-valued exponential random variables ¹ I 2 <º with parameter n, which serve as lifetimes of the particles, and D be ¹ I 2 <º a family of i.i.d. random variables with P.D k/ qn for k 0; 1; 2; : : : , D D k D where qn k 0; 1; : : : is the offspring distribution satisfying above conditions. ¹ k I D º We assume W , B , S , C and D are all ¹ I 2 <º ¹ I 2 <º ¹ I 2 <º ¹ I 2 <º independent. The birth time ˇ./ of the particle x is given by

´P 1 j j jj j1 C if D 2 for every j 1; : : : ; 1, ˇ./ D D j j WD ; otherwise: 1

The death time of x is given by ./ ˇ./ C and the indicator function of D C the lifespan of x is denoted by h 1 .t/. t WD Œˇ./;.// Recall that @ denotes the cemetery point. Define x @ if either t < ˇ./ or t D t ./. We make a convention that any function f defined on R is automatically extended to R @ by setting f .@/ 0. This convention allows us to keep track [ ¹ º D of only those particles that is alive at any given time. n n n Pm0 n Given 0 MF .R/, let 0 1 ıx be such that 0 0 as 2 WD 0 ) n (see [33]). We are thus provided withD collection of initial starting points ! 1 x for each n 1. ¹ 0º Let N n 1; 2; : : : ; mn be the set of indices for the first generation of parti- 1 WD ¹ 0º cles. For any N n , define 2 1 \ < ´ ˆ.x ;B ;S ; t/ if 0 t < C ; x 0 Ä (3.5) t WD @ if t C ; and x @ for any .N N n/ and t 0: t Á 2 n 1 \ < Interacting superprocesses with discontinuous spatial motion 17

n If 0 N and D 0 .!/ k 2, deﬁne for every 0i i 1; 2; : : : ; k , 2 1 D 2 ¹ I D º ´ ˆ.x0 ;B ;S ; t ˇ.// if t Œˇ./; .//; .0/ xt 2 (3.6) WD @ otherwise.

0 If D .!/ 0, deﬁne xt @ for 0 t < and for every 0i i 1 . D Á Ä 1 n 2 ¹ I º More generally for an integer m 1, let Nm be the set of all indices for th n < the living particles in m -generation. If 0 N and D 0 .!/ k 2, deﬁne 2 m D for 0i i 1; 2; : : : ; k 2 ¹ I D º ´ ˆ.x0 ;B ;S ; t ˇ.// if t Œˇ./; .//; .0/ xt 2 (3.7) WD @ otherwise:

If D0 .!/ 0, deﬁne D x @ for 0 t < and for every 0i i 1 : t Á Ä 1 2 ¹ I º Continuing in this way, we get a branching tree of particles for any given ! with n ® 1 2 m0 ¯ random initial state taking values in x0 ; x0 ; : : : ; x0 . This gives us our branching particle systems on the real line where particles undergo a ﬁnite-variance branching at independent exponential times and have interacting spatial motion powered by jump-diffusions and a common white noise.

4 Tightness and the limiting martingale problem

Recall the branching particle system constructed in the last section. Deﬁne its associated empirical process by

n 1 X t .A/ ı .A/ for A B.R/: (4.1) WD n xt 2 2< n In the following, we will show that t 0 converges in D.Œ0; /; MF .R// ¹ t I º 1 weakly as n and its weak limit is the stable SDSM. ! 1 For any t > 0, deﬁne

n X .D 1/ M .A .0; t/ n 1 x A; ./ t ; (4.2) WD ./ ¹ 2 Ä º 2< which gives the total mass along each survived trajectory in set A up to time t. 18 Z.-Q.Chen, H. Wang and J. Xiong

Note that (3.4) implies that for every C 2.R/, 2 b n n 1 n 1 n n n ; t ; 0 Ut ./ Ht ./ Xt ./ Yt ./ h i h i D p n C p n C C (4.3) 1 n n Jt ./ Mt ./; C p n C where, recalling hs 1 .s/, D Œˇ./; .// Z t n 1 X Ut ./ hs 0.xs /c.xs /dBs ; WD p n 0 2< Z tZ n n Xt ./ h.y /0. /; s W.dy; ds/ ; WD 0 Rh i Z t n 1 ˛ ˛=2 n Yt ./ 2 a00 b ; s ds; WD 0 h C j j i Z t n 1 X Ht ./ hs 0.xs /b.xs /dbS s ; WD p n 0 2< 1 X ; J n./ M t WD p n t 2< Z tZ n n Mt ./ .x/M .dx; ds/ WD 0 R X .D 1/ n .x./ / 1 ./ t : D ¹ Ä º 2< The six terms in (4.3) represent the respective components of the overall mo- n tion of the ﬁnite particle systems ; t contributed by the individual Brownian n h i n motions (Ut ./), the random medium (Xt ./), the mean effect of jump-diffusive n n motions (Yt ./), the branching mechanism (Mt ./), the individual stable mo- n tions (Ht ./) with jump size bounded by one and the individual compensated n jump sums (Jt ./). Using a result of Dynkin [12, Theorem 10.25 on p. 325], we get at once the following theorem.

Theorem 4.1. For any n N, n defined by (4.1) is a right continuous strong 2 t Markov process which is the unique strong solution of (4.3) in the sense that it is a unique solution of (4.3) for a given probability space .; F ; P/ and given W , B , S , C , D defined on .; F ; P/. Furthermore, n t 0 are all ¹ º ¹ º ¹ º ¹ º ¹ t I º defined on the common probability space .; F ; P/. Interacting superprocesses with discontinuous spatial motion 19

For each t 0, let F n denote the -algebra generated by the collection of t processes ° ± n./; U n./; X n./; Y n./; M n./; H n./; J n./; C 2.R/; t 0 : t t t t t t t 2 b

Note that according to our assumption, the fourth moment of D ,

mn E .D 1/4 ; c WD ¹ º n 2n is ﬁnite and limn mc 0 for any 2. !1 D Lemma 4.2. Based on the above notations, we have the following:

2 n n (i) For every Cb .R/, M ./ Mt ./ t 0 is a purely discontinuous square-integrable2 martingale withWD ¹ I º

Z t n 2 2 n M ./ t n ; u du for every t 0: h i D 0 h i

(ii) For any t 0 and n 1, we have h n 2i n 2 2 n E sup 1; s 2 1; 0 8n t 1; 0 : 0 s th i Ä h i C h i Ä Ä There is a constant > 0 such that for every t 0, h n 4i 3 6 3 n 2 4 2 n 2 E sup 1; s n t 1; 0 n t 1; 0 0 s th i Ä h i C h i Ä Ä mn Á c t 1; n 1; n 4 : C 2n h 0i C h 0i

(iii) Let n t 0 be deﬁned by (4.1). Then n t 0 is tight as a family of ¹ t I º ¹ t I º processes with sample paths in D.Œ0; /; MF .R//. 1 Proof. (i) Recall that C are i.i.d. exponential random variables with ¹ I 2 <º parameter n, D 1 are i.i.d. random variables with zero mean and ¹ I 2 <º these two families are independent. Thus E M n./ 0 for every t > 0 and ¹ t º D S.R/. Since this holds for any initial state n, by the Markov property of 2 0 n t 0 , we have for every t; s > 0, ¹ t I º n n ˇ n n n E M ./ M ./ E n M ./ M ./ 0: t s t ˇ F t t s 0 C D D 20 Z.-Q.Chen, H. Wang and J. Xiong

n This shows that M ./ is a martingale. Clearly, it is purely discontinuous. Note 1 Ä add some E 2.x / ./ t wording ./ I Ä or replace period by Ä colon? E 1 .ˇ./ C /2.x / Œ0;t .ˇ./ C / D C C ÄZ 1 2 n nu E 1Œ0;t.ˇ./ u/ .x.ˇ./ u/ / e du D 0 C C ÄZ 1 2 n E 1Œ0;t.ˇ./ u/ .x.ˇ./ u/ / 1 C >u du by independence D 0 C C ¹ º ÄZ 1 2 n E 1Œ0;t.ˇ./ u/ .xˇ./ u/ du by the deﬁnition of x D 0 C C ÄZ 1 2 n E 1Œ0;t.v/ .xv/ dv D ˇ./ ÄZ t n 2 E 1Œˇ./;.//.v/ .xv/dv : D 0 As C and D are all independent and ED 1, we conclude ¹ I 2 <º ¹ I 2 <º D that Ä n 2 X 2n h 2i 2 E Mt ./ E .D 1/ E .x./ /1../ t/ D Ä 2< ÄZ t 2n 2 X n 2 n E 1Œˇ./;.//.v/ .xv/dv (4.4) D 0 2< ÄZ t 2 2 n n E ; v dv : D 0 h i n Note that identity (4.4) holds for any initial distribution 0. Again by the Markov property of n t 0 , we have for every t; s > 0, ¹ t I º Ä Z t s ˇ n 2 n 2 2 C 2 n ˇ n E Mt s./ Mt ./ n ; v dv ˇ F t C t h i ˇ Ä Z s n 2 2 2 n E n M ./ ; dv 0: t s n v D 0 h i D This shows that M n./2 2 R t 2; n dv is a martingale. Hence we conclude t n 0 h vi that M n./ is a purely discontinuous square-integrable martingale with Z t n 2 2 n M ./ t n ; u du for every t 0: h i D 0 h i Interacting superprocesses with discontinuous spatial motion 21

(ii) Since 1; n n M n.1/ is a zero-mean martingale, by Doob’s maximal h t 0i D t inequality, we have Ä Ä n 2 n 2 n 2 E sup 1; s 2E sup Ms .1/ 2 1; 0 0 s th i Ä 0 s t C h i Ä Ä Ä Ä 8E M n.1/2 2 1; n 2 Ä t C h 0i 8 2t 1; n 2 1; n 2: Ä n h 0i C h 0i n P D 1 Note that Mt .1/ n 1 ./ t is a purely discontinuous martingale D ¹ Ä º and D 1 are i.i.d2< random variables with zero mean and are independent ¹ I 2 <º of C . Thus ¹ I 2 <º Ä Â 2 2 Ã h n 4i X .D 1/ .D 1/ E Mt .1/ E 2n 1 ./ t 2n 1 ./ t D ¹ Ä º ¹ Ä º ; ; 2< ¤ ÄX .D 1/4 E 4n 1 ./ t C ¹ Ä º 2< 4 Ä n Ä n X mc X 4n E 1 ./ t 1 ./ t 4n E 1 ./ t D ¹ Ä º ¹ Ä º C ¹ Ä º ; ; 2< ¤ 2< 4 n ÄZ t n X mc n 4n E 1 ./ t 1 ./ t 2n E 1; v dv : D ¹ Ä º ¹ Ä º C 0 h i ; ; 2< ¤ For ; with , C and C are independent and so 2 < ¤ E 1 ./ t 1 ./ t ¹ Ä Äº ¹ Ä º E 1 .ˇ./ C / 1 .ˇ./ C / D Œ0;t C Œ0;t C ÄZ Z 2 2n 1 1 nu nv E 1Œ0;t.ˇ./ u/1Œ0;t.ˇ./ v/e e dudv D 0 0 C C ÄZ Z 2 2n 1 1 E 1Œ0;t.ˇ./ u/1Œ0;t.ˇ./ v/ Ä 0 0 C C Á 1 x @ 1 x @ dudv ˇ./ u ˇ.Á/ u ¹ C 6D º ¹ C 6D º ÄÂZ ÃÂZ Ã 2 2n 1 1 E 1Œ0;t.r/1 dr 1Œ0;t.s/1 Á ds xr @ xs @ D ˇ./ ¹ 6D º ˇ./ ¹ 6D º ÄÂZ t ÃÂZ t Ã 2 2n E 1Œˇ./; .//.r/dr 1Œˇ./; .//.s/ds : D 0 0 22 Z.-Q.Chen, H. Wang and J. Xiong

Therefore X E 1 ./ t 1 ./ t ¹ Ä º ¹ Ä º ; ; 2< ¤ ÄÂZ t ÃÂZ t Ã X 2 2n E 1Œˇ./; .//.r/dr 1Œˇ./; .//.s/ds Ä 0 0 ; 2< 2 3 Â Z t Ã2 2 2n X E 4 1Œˇ./; .//.r/dr 5 D 0 2< ÄÂZ t Ã2 2 4n n E 1; r dr : D 0 h i It then follows 4 ÄÂZ t Ã2 n ÄZ t h n 4i n 2 4n n mc n E Mt .1/ 4n E 1; r dr 2n E 1; v dv Ä 0 h i C 0 h i n 2 4 2 h n 2i mc n n t E sup 1; r 2n 1; 0 t D r Œ0;th i C h i 2 mn 2 4t 2 8 2t 1; n 2 1; n 2 c 1; n t Ä n n h 0i C h 0i C 2n h 0i mn 83 6t 3 1; n 22 4t 2 1; n 2 c t 1; n : (4.5) D n h 0i C n h 0i C 2n h 0i By Doob’s maximal inequality,

h n 4i h n n n 4i E sup 1; s E sup 1; s 0 1; 0 0 s th i Ä 0 s t h i C h i Ä Ä Ä Ä h n 4i n 4 8E sup Ms .1/ 8 1; 0 Ä 0 s t j j C h i Ä Ä h 4i 8.4=3/4 E M n.1/ 8 1; n 4 Ä t C h 0i 3 6t 3 1; n 2 4t 2 1; n 2 Ä n h 0i C n h 0i mn Á c t 1; n 1; n 4 : C 2n h 0i C h 0i n (iii) We ﬁrst prove the tightness of in D.Œ0; /; MF .R//, where R is the ¹ º 1 O O one-point compactiﬁcation of R. Note that (ii) above implies the compact containment property for n . By Theorems 4.5.4 and 4.6.1 in Dawson [7], we then ¹ º only need to prove that, for any given " > 0, > 0, T > 0, C 2.R/ and 2 b Interacting superprocesses with discontinuous spatial motion 23

any stopping time n bounded by T , there exist ı > 0 and n0 1 such that sup sup P n ./ n ./ > " . n n0 t Œ0;ı ¹j n t n j º Ä For this, ﬁrst2 observe thatC by (4.3), n n P. n t ./ n ./ > "/ j C j 1 h 2i E n ./ n ./ 2 n t n Ä " C Ä 6 1 2 1 2 E U n ./ U n ./ H n ./ H n ./ 2 n n t n n n t n t Ä " C C C C n n 2 n n 2 Xn t ./ Xn ./ Yn t ./ Yn ./ C C C C 1 2 2 J n ./ J n ./ M n ./ M n ./ : n n t n t n t n C C C C C Note that by the independence of B , ¹ I 2 <º ÄZ n t h 2i 1 X E U n ./ U n ./ E C h . c/2.x /ds n t n n s 0 s C D n 2< ÄZ n t C ˝ 2 n˛ E .0c/ ; s ds D n h ˝ 2 n˛i t E sup .0c/ ; s Ä s T t Ä C 2 h n i t 0c E sup 1; s : Ä k k1 s T t h i Ä C By the independence of bS and the Lévy system formula (2.1), ¹ I 2 <º h n n 2i E Hn t ./ Hn ./ C Ä 1 X X 2 2 n E hs .0b/ .xs /. bS s/ D 4 n

h n n 2i E Xn t ./ Xn ./ C ÄZ n tZ C n 2 E h.y /0. /; s dyds D n Rh i ÄZ n t ÂZ Z Ã C n n E .w z/0.w/0.z/s .dw/s .dz/ ds D n R R ÄZ n t 2 C n 2 2 2 h n 2i 0 E 1; s ds t h 2 0 E sup 1; s : Ä k k1k k1 n h i Ä k k k k s T th i Ä C In view of Lemma 3.1,

h n n 2i E Yn t ./ Yn ./ C Â Ã ÄZ n t 1 ˛ ˛=2 C n 2 a00 b E 1; s ds Ä 2k k1 C kj j k1 n h i Â Ã 1 ˛ ˛=2 h n 2i t a00 b E sup 1; s : Ä 2k k1 C kj j k1 s T th i Ä C By the independence of S , we have by a similar calculation as that for ¹ I 2 <º (3.2) that

h n n 2i E Jn t ./ Jn ./ C 1 X h ; 2i E M M ; n n t n D C 2< ÄZ n t 1 X 2 C n c b 00 E 1 x @ ds Ä k k1 C k k1 s n ¹ 6D º 2< ÄZ n t 2 C n c b 00 E 1; s ds D k k1 C k k1 n h i 2 h n i b 00 c t E sup 1; s : D k k1 C k k1 s T th i Ä C Finally, we have by part (i) of Lemma 4.2 that

~~ÄZ n t h n n 2i 2 C 2 n E Mn t ./ Mn ./ n E ; s ds C D n h i 2 2 h n i n t E sup 1; s : Ä k k1 s T th i Ä C Interacting superprocesses with discontinuous spatial motion 25~~

~~Therefore by part (ii) of Lemma 4.2 and Lemma 3.4 of Wang [33], we conclude that for every " > 0 there is a constant c > 0 such that~~

n n sup sup P n t ./ n ./ > " cı for every ı > 0; n 1 t Œ0;ı j C j Ä 2 which proves (iii). To complete the proof of the lemma, it remains to prove that the limit process does not hit the cemetery. This follows from the same arguments as those in the proof of Theorem 4.1 in Dawson et al. [10].

Let S.R/ be the space of Schwartz test functions, i.e., the space of inﬁnitely dif- ferentiable functions which, together with all their derivatives, are rapidly decreas- ing at inﬁnity. Let S 0.R/ denote the Schwartz space of tempered distributions, the dual space of S.R/.

~~Theorem 4.3. Based on the above notations, we have following conclusions: (i) .n;U n;Y n;M n;H n;J n/ is tight on~~

~~5 D.Œ0; /; MF .R// D.Œ0; /; .S .R// /: 1 1 0 (ii) (A Skorohod representation) Suppose that the joint distribution of~~

~~.nm ;U nm ;Y nm ;M nm ;H nm ;J nm ;W/~~

~~converges weakly to the joint distribution of~~

~~.0;U 0;Y 0;M 0;H 0;J 0; W /:~~

~~Then there exist a probability space .;e Fe; Pe/ and sequences~~

~~nm nm nm nm nm nm nm .e ; Ue ; Ye ; Mf ; He ; eJ ; We / 5 deﬁned on it with values in D.Œ0; /; MF .R// D.Œ0; /; .S 0.R// / such that 1 1~~

P .nm ;U nm ;Y nm ;M nm ;H nm ;J nm ;W/ 1 ı nm nm nm nm nm nm nm 1 Pe . ; Ue ; Ye ; Mf ; He ; eJ ; We / D ı e 6 holds and, Pe-almost surely on D.Œ0; /; MF .R// D.Œ0; /; .S .R// /, 1 1 0 nm nm nm nm nm nm nm e ; Ue ; Ye ; Mf ; He ; eJ ; We 0 0 0 0 0 0 0 ; Ue ; Ye ; Mf ; He ; eJ ; We ! e as m . ! 1 26 Z.-Q.Chen, H. Wang and J. Xiong

(iii) There exists a dense subset D Œ0; / such that Œ0; / D is at most 1 1 X countable and for each t D and each S.R/, as an R7-valued process 2 2 nm nm nm nm nm nm nm .et ./; Ue t ./; Yet ./; Mft ./; He t ./; eJ t ./; We t .// 0 0 0 0 0 0 0 . ./; Ue ./; Ye ./; Mf ./; He ./; eJ ./; We .// ! et t t t t t t 2 0 in L .;e Fe; Pe/ as m . Furthermore, let Fe t be the -algebra genera- 0 0 !0 1 0 0 0 0 ted by es ./, Ues ./, Yes ./, Mfs ./, He s ./, eJ s ./, We s ./ with run- 0 ning through S.R/ and s t. Then Mft ./ is a continuous, square-inte- 0 Ä grable Fe t -martingale with quadratic variation process Z t 0 2 2 0 Mft ./ ; eu du: h i D 0 h i

nm (iv) We 0.dy; ds/ and We .dy; ds/ are Brownian sheets. The continuous square- integrable martingale Z tZ nm ˝ nm ˛ nm Xe t ./ h.y /0. /; es We .dy; ds/ WD 0 R converges to Z tZ 0 ˝ 0˛ 0 Xe t ./ h.y /0. /; es We .dy; ds/ WD 0 R

in L2.;e Fe; Pe/ . 0 0 is a solution to the -martingale problem and 0 (v) e et t 0 .L; ı0 / e satisﬁesD ¹ I º Z tZ 0 0 0 0 ./ ./ Xe ./ .x/Mf .dx; ds/ et e0 D t C 0 R (4.6) Z t D 1 ˛ ˛=2 0E 2 a00 b ; es ds: C 0 C j j

n Proof. (i) The tightness of mu n 1 in D.Œ0; /; MF .R// has been estab- ¹ I º 1 lished in Lemma 4.2 (iii). So by a theorem of Mitoma [27], we only need to prove that for any S.R/ the sequence of laws of 2 .U n./; Y n./; M n./; H n./; J n.// is tight in D.Œ0; /; R5/. This is equivalent to proving that each component and 1 the sum of each pair of components are individually tight in D.Œ0; /; R/. Since 1 Interacting superprocesses with discontinuous spatial motion 27 the same idea works for each sequence, we only give the proof for M n./ . By ¹ º Lemma 4.2, we have 2 ÄZ t 2 2 n n 2 n n t n P Mt ./ > k 2 E ; u du k 2k1 1; 0 ; Ä k 0 h i Ä k h i which yields the compact containment condition. Now we use Kurtz’s tightness criterion (cf. Ethier–Kurtz [17, Theorem 8.6 on p. 137]) to prove the tightness of M n./ . ¹ Tº 2 2 n Let n .ı/ ın sup0 u T 1; u . Then for any 0 t ı T , WD k k1 Ä Ä h i Ä C Ä Ä Z t ı ˇ n n 2 ˇ n 2 C 2 n ˇ n E Mt ı ./ Mt ./ ˇ Ft E n ; u du ˇ Ft j C j D t h i ˇ T ˇ n E .ı/ ˇ F : Ä n t T n By Lemma 4.2, limı 0 supn EŒn .ı/ 0 holds, so M ./ n 1 is tight. ! D ¹ I º (ii) If we choose any countable dense subset gi i N of S.R/ and any enumer- ¹ º 2 ation tj j N of all rational numbers, then Theorem 1.7 of Jakubowski [21] shows ¹ º 2 that the countable family fij i; j N are continuous functions (with respect to ¹ I 2 º Skorohod topology on D.Œ0; /; S .R//) and separate points, where 1 0

fij x D.Œ0; /; S .R// fij .x/ arctan gi ; x.tj / Œ ; : W 2 1 0 ! WD h i 2 This proves that the space D.Œ0; /; S .R//, thus the space D.Œ0; /; .S .R//7/ 1 0 1 0 as well, satisﬁes the basic assumption for a version of the Skorohod Representation Theorem due to Jakubowski [22]. (iii) For each t D and each S.R/, from Lemma 4.2 we obtain the 2 2 uniform integrability of

~~® nm 2 nm 2 nm 2 nm 2 nm 2 nm 2 nm 2¯ et ./ ; Ue t ./ ; Yet ./ ; Mft ./ ; He t ./ ; eJ t ./ ; We t ./ :~~

So (ii) implies their convergence in L2.;e Fe; Pe/ as m . For each S.R/, 7 ! 1 2 Gi C .R / and any 0 < t1 tn s < t with ti ; t D, i 1; : : : ; n, let 2 b Ä Ä D 2 D n nm Y nm nm nm nm f .t1; : : : ; tn/ Gi ./; Ue ./; Ye ./; Mf ./; WD eti ti ti ti i 1 D nm nm nm Á He ti ./; eJ ti ./; We ti ./ :

~~Then we have~~

~~nm nm nm eE .Mf ./ Mf .//f .t1; : : : ; tn/ 0 (4.7) t s D 28 Z.-Q.Chen, H. Wang and J. Xiong and~~

~~ÄÂ Z t nm 2 2 2 n n 2 eE Mf ./ ; m du Mf m ./ t nm h eu i s 0 (4.8) Z s Ã 2 2; nm du f nm .t ; : : : ; t / 0: nm eu 1 n C 0 h i D~~

2 0 By the above convergence in L .;e Fe; Pe/, this implies that the processes Mft 0 2 2 R t 2 0 0 2 and Mf ./ ; du are Fe -martingales. Let K sup .x/. t 0 h eui t D x R Based on (4.5), we can get 2

n n 4 eE .Mf ./ Mf .// t s E.M n./ M n.//4 D t s Ä Â 2 2 X .D 1/ 2 .D 1/ E 2n .x./ /1 s<./ t 2n D ¹ Ä º ; ; 2< ¤ Ã 2 .x./ /1 s<./ t ¹ Ä º Ä Â 4 Ã X .D 1/ 4 E 4n .x./ /1 s<./ t C ¹ Ä º 2< Ä Â 2 2 Ã 2 X .D 1/ .D 1/ K E 2n 1 s<./ t 2n 1 s<./ t Ä ¹ Ä º ¹ Ä º ; ; 2< ¤ Ä Â 4 Ã 2 X .D 1/ K E 4n 1 s<./ t C ¹ Ä º 2< K2 83 6.t s/3 1; n 22 4.t s/2 1; n 2 Ä n h 0i C n h 0i mn Á c .t s/ 1; n : C 2n h 0i In particular, for any m 1 we have nm nm 4 2 3 6 3 nm eE .Mf ./ Mf .// K 8 .t s/ 1; t s Ä nm h 0 i 2 4 2 nm 2 2 n .t s/ 1; (4.9) C m h 0 i mnm Á c .t s/ 1; nm : C 2nm h 0 i Interacting superprocesses with discontinuous spatial motion 29

Letting m , we get ! 1 0 0 4 2 3 6 3 eE .Mft ./ Mfs .// K 8 .t s/ 1; 0 Ä h i (4.10) 2 4 2 2Á 2 .t s/ 1; 0 : C h i 0 Thus Mft has a continuous modiﬁcation according to Kolmogorov’s continuity criterion and Z t 0 2 2 0 Mft ./ ; eu du: h i D 0 h i 0 nm nm (iv) Since W , We , We have the same distribution, We 0 and We are Brown- ian sheets. The conclusion follows from (ii) and Theorem 2.1 of Cho [5]. nm 2 nm 2 nm 2 (v) Since Ue t ./ , He t ./ and eJ t ./ are uniformly integrable, we have Pe-a.s. and in L2.Pe/

1 nm 1 nm 1 nm lim Ue t ./ lim He t ./ lim eJ t ./ 0: m p nm D m p nm D m p nm D !1 !1 !1 By passing n along the subsequence nm m 1 in (4.3), we have ! 1 ¹ I º 0 0 0 0 0 ./ ./ Xe ./ Ye ./ Mf ./ for every S.R/ and t 0: et e0 D t C t C t 2 As Z t n D 1 ˛ ˛=2 nE Yet ./ 2 a00 b ; es ds D 0 C j j and Z tZ n n Mft ./ .x/Mf .dx; dy/; D 0 R we see from (ii) above that Z t 0 D 1 ˛ ˛=2 0E Yet ./ 2 a00 b ; es ds D 0 C j j and Z tZ 0 0 Mft ./ .x/Mf .dx; dy/: D 0 R 0 So, et satisﬁes (4.6). By Itô’s formula, we see that 0 t 0 is a solution to the martingale problem ¹et I º for .L; ı / with sample pathes in D.Œ0; /; MF .R//. 0 1 0 Theorem 4.3 (v) tells us that 0 t 0 is a solution to the mar- e D ¹et I º tingale problem for .L; ı0 /. For uniqueness, we will use a duality argument S m due to Dawson–Kurtz [9]. For f m1 1 Cb.R /, we write N.f / for m if m 2 D f Cb.R / and deﬁne 2 Z F.f / Ff ./ f .x1; : : : ; xm/.dx1/ : : : .dxm/ for MF .R/: WD WD Rm 2 30 Z.-Q.Chen, H. Wang and J. Xiong

Such a function Ff is called a monomial function on space MF .R/. Note that for such a monomial function Ff , N.f / Z N.f / @Ff ./ X Y f .x1; : : : ; xj 1; x; xj 1; : : : ; xN.f // .dxl / @.x/ D RN.f / 1 C j 1 l 1;l j D D 6D and 2 N.f / Z @ Ff ./ X f .x1; : : : ; xj 1; x; xj 1; : : : ; xk 1; y; @.y/@.x/ D RN.f / 2 C j;k 1; j k N.f / D 6D Y xk 1; : : : ; xN.f // .dxl /: C l 1;l j;k D 6D 2 m m For f C .R /, x .x1; : : : ; xm/ R , we deﬁne 2 b D 2 m XÂ @2 Ã L.m/f .x/ 1 a.x / b.x / ˛˛=2 f .x/ 2 k 2 k xk WD @xk C j j k 1 m D 1 X @2 .xk xj / f .x/ C 2 @xk@xj (4.11) j;k 1;j k D 6D m 2 Â m Ã X @ X ˛ ˛=2 jk f .x/ b.xk/ xk f .x/; D @xk@xj C j j j;k 1 k 1 D D where jk is deﬁned (1.9). Then by (1.2), we have for monomial function Ff on MF .R/ with N.f / m, D LFf ./ AFf ./ BFf ./ D C m 2 X F .m/ ./ F ./ D L f C 2 ˆjk f j;k 1; j k D 6D 2 m X F .m/ ./ F ./ F ./ D L f C 2 ˆjk f f j;k 1; j k D 6D 2 m.m 1/F ./ C 2 f 2 m .m/ X F.L f / F.ˆ f / F.f / D C 2 jk j;k 1; j k D 6D 2 m.m 1/F.f / C 2 1 2 L F.f / m.m 1/F.f /: DW C 2 Interacting superprocesses with discontinuous spatial motion 31

~~m 1 Here for j < k, ˆjkf is a function on R deﬁned by~~

ˆjkf .y/ ˆkj .y/ f .y1; : : : ; yj ; : : : ; yk 1; yj ; yk; : : : ; ym 1/ WD WD m 1 for y .y1; : : : ; ym 1/ R . D 2 2 In order to use Dawson–Kurtz’s duality method, we establish C C -regularity on the transition semigroup of L.m/ for some .0; 1/. Following [23], we 2 introduce the Hölder spaces on Rm and Œ0; T Rm as follows. Given .0; 1/, 2 deﬁne for f C.Rm/ 2 f .x/ f .y/ Œf sup j j WD x y x y 6D j j and for C.Œ0; T Rm/ 2 .s; x/ .t; x/ .s/; T sup j j; b c I WD s;t Œ0;T ; x Rm s t 2 2 j j .s; x/ .s; y/ .x/; T sup j j: b c I WD s Œ0;T ;x;y Rm x y 2 2 j j Pm For k .k1; : : : ; km/ with non-negative integer components, let k i 1 ki D k j j WD k @j j j m D and @ k k . For j 0; 1; 2 and .0; 1/, let C C .R / denote WD @ 1 x1 ::: @ m xm D 2 b the Banach space with the norm

X k X k f j sup @ f .s; x/ Œ@ f ; k k C WD m j j C k j .s;x/ Œ0;T R k j j jÄ 2 j jD j . =2/;2j m and C C C .Œ0; T R / the Banach space with the norm b X l k 2j ;T sup @s@x.s; x/ k k C WD m j j 2l k 2j .s;x/ Œ0;T R Cj jÄ 2 X l k @s@x .s/;.2j 2l k /=2 T C b c C j j I 0<2j 2l k <2 C j j X l k @s@x .x/; T : C b c I 2l k 2j Cj jD We also deﬁne the Banach spaces C 0;2.Œ0; T Rm/ and C 1;2.Œ0; T Rm/ b b of bounded C 0;2- and C 1;2-smooth functions on Œ0; T Rm, respectively, with norms X k 0;2 T sup @x.s; x/ k k I WD m j j k 2 .s;x/ Œ0;T R j jÄ 2 32 Z.-Q.Chen, H. Wang and J. Xiong and X l k 1;2 T sup @s@x.s; x/ : k k I WD m j j 2l k 2 .s;x/ Œ0;T R Cj jÄ 2 For any given integer m 1, let Z.m/ .z1; : : : ; zm/ be the process of m par- t D t t ticles given by (3.1). For the next theorem, the law of Z.m/ with initial starting m point from x R will be denoted as Px, and the mathematical expectation under 2 Px will be denoted as Ex.

Theorem 4.4. Assume that the diffusion matrix .ij /1 i;j m deﬁned by (1.9) is Ä Ä ˛ uniformly elliptic and bounded. Assume also that each ij and b are bounded j j and -Hölder continuous for some .0; 1 .2 ˛//. Let P m t 0 be the 2 ^ ¹ t I º transition semigroup for Z.m/, that is,

m h .m/ i m P f .x/ Ex f .Z / for t 0 and f B .R /: t WD t 2 b 2 m m 1 . =2/; 2 Then for every f Cb C .R /, Pt f .x/ as a function of .t; x/ is Cb C C m 2 2 m m on Œ0; T R for every T > 0. In particular, C C .R / is invariant under P b t for every t > 0 and m 1. .m/ The above theorem gives sufficient regularity for the semigroup Pt needed to apply Dawson–Kurtz’s duality method for the well-posedness of the martingale problem. To keep the flow of the argument for the uniqueness of the martingale problem, we postpone its proof into the next section. S 2 k Fix an arbitrary .0; 1 ˛ .2 ˛//. Let S k1 0 Cb C .R / (disjoint 2 20 ^ ^ D D .m/ union) with Cb C .R / R. We see from the proof of Theorem 4.4 that L 2 mWD coincides on Cb C .R / with the infinitesimal generator of the strong Markov .m/ process Z for the motion m particles given by (3.1). Thus L has the structure of the infinitesimal generator of an S-valued strong Markov process Y , whose dynamics involve two mechanisms as follows.

(a) Jump mechanism: Let Mt t 0 be a non-negative integer-valued cádlág ¹ I º Markov process with M0 0 and transition intensities qi;j such that D ¹ º 2 qi;i 1 qi;i i.i 1/ and qi;j 0 for all other pairs .i; j /: D D 2 D Thus Mt t 0 is just the well-known Kingman’s coalescent process. Let ¹ I º 0 0, M0 1 and k 1 k M0 be the sequence of jump times D C D 1 ¹ I Ä Ä º of Mt t 0 . Let S 1 k M0 be a sequence of random operators ¹ I º ¹ kI Ä Ä º which are conditionally independent given Mt t 0 and satisfy ¹ I º 1 P S ˚i;j M. / l ; 1 i j l: ¹ k D j k D º D l.l 1/ Ä ¤ Ä Interacting superprocesses with discontinuous spatial motion 33

(b) Spatial jump-diffusion semigroup: Let B denote the topological union of C .Rm/ m 1; 2; : : : endowed with supremum norm on each C .Rm/. ¹ b I D º b Then M M M k k 1 1 M0 Yt Pt k SkPk k 1 Sk 1 :::P2 1 S1P1 Y0; D k t < k 1; 0 k M0; Ä C Ä Ä deﬁnes a Markov process Y Yt t 0 . By Theorem 4.4, the process Y WD ¹ I º takes values in S B. Clearly, .Mt ;Yt / t 0 is also a Markov process. ¹ I º The duality relationship can be described as follows. Take any .0; 1/. Let us 2 m denote by D.L/ the set of all functions of the form Fm;f ./ f; with f 2 m D h i 2 C C .R /. If Xt t 0 is a solution of the .L; D.L//-martingale problem b ¹ I º with X0 0 on a probability space .; F ; P/, then, by the Feynman–Kac D formula (see [9]), we have Ä Â 2 Z t Ã m Mt E f; Xt Em;f Yt ; 0 exp Ms.Ms 1/ds (4.12) h i D h i 2 0

2 m for any t 0, f C C .R / and integer m 1, where the right hand side 2 b is the expectation taking on the probability space for which the dual process is 2 m deﬁned with giving M0 m and Y0 f C C .R /. From this, we see D D 2 b that the marginal distribution of X is uniquely determined and hence the law of X is unique (see, e.g., [17, Theorem 4.4.2] or [9, Theorem 2.1]). This proves the uniqueness of the martingale problem for L. We summarize these results in the following theorem.

Theorem 4.5. Assume that b; c Lip .R/ and h L2.R/ Lip.R/. For the 2 b 2 \ uniqueness for the martingale problem below, assume further that c is bounded 1 from below by a strictly positive constant and h L .R/. For any MF .R/, 2 2 the .L; ı/-martingale problem has a unique solution t with sample pathes in C.Œ0; /; MF .R//, which is a diffusion process and satisﬁes 1 Z tZ t ./ 0./ Xt ./ .x/M.dx; ds/ D C 0 R (4.13) Z t 1 ˛ ˛=2 2 a00 b ; s ds C 0 h C j j i for every t > 0 and C 2.R/, where W is a Brownian sheet, 2 b Z tZ Xt ./ h.y /0. /; s W.dy; ds/; t 0; WD 0 Rh i 34 Z.-Q.Chen, H. Wang and J. Xiong and M is a square-integrable martingale measure with

~~Z t 2 2 2 M./ t ; u du for every t > 0 and Cb .R/: h i D 0 h i 2 Here Z tZ Mt ./ .y/M.ds; dy/ WD 0 R is a square-integrable, continuous Ft -martingale, where ¹ º~~

~~Ft s.f /; Ms.f /; Xs.f /; f B .R/; s t : WD ¹ 2 b Ä º~~

~~Moreover Xt ./, Mt ./ are orthogonal square-integrable Ft -martingales for ¹ º every S.R/. 2~~

Proof. According to previous results, we only need to prove the continuity of t as a process taking values in MF .R/. Hence we only need to show that for any bounded continuous function f on R, ; t is continuous in t 0. However, h i this is just a simple application of Bakry–Emery’s result (see [1, Proposition 2] and the last section in Wang [33]).

~~2 .m/ 5 C C -regularity for the pseudo-differential operator L~~

~~2 In this section, we give the proof of the C C -regularity for the semigroup of the pseudo-differential operator L.m/ of (4.11).~~

Proof of Theorem 4.4. For 1 j m, let ej denote the unit vector in the positive Ä Ä2 m .m/ xj direction. Note that for f C .R / the inﬁnitesimal generator L given 2 b by (1.8) can be written as .m/ L Gm Im; D C where m 1 X @2f Gmf .x/ ij .x/ .x/ WD 2 @xi @xj i;j 1 D and m Z Â X ˛ Imf .x/ c˛ b.xj / f .x wej / f .x/ D j j R 0 C j 1 X¹ º D @ Ã 1 f .x/w1 w 1 1 ˛ dw: @xj ¹j jÄ º w j j C Interacting superprocesses with discontinuous spatial motion 35

Since ij is uniformly elliptic and Hölder continuous, it is known from Theo- 2 m rem 5.1 in [23, p. 320] that for every T > 0, every f C C .R / and every . =2/; 2 C .Œ0; T Rm/, the differential equation 2 b 8 @v.t; x/ m < Gmv.t; x/ .t; x/ for .t; x/ .0; 1 R ; @t D C 2 (5.1) : v.0; x/ f .x/ for x Rm; D 2 1 . =2/; 2 m has a unique solution v C C C .Œ0; T R /. Moreover, there is a con- 2 b stant c0.T / > 0 such that v 2 ;T c0.T / f 2 ;T : (5.2) k k C Ä k k C C k k 0 m m Let s s 0; Px ; x R be the diffusion on R with inﬁnitesimal gener- ¹ I 2 º m ator Gm, whose transition semigroup will be denoted as T t 0 . For any ¹ t I º t .0; T , by using Itô’s formula, we can conclude that Ms v.t s; s/ R s2 0 WD 0 C 0 .t r; r /dr, 0 s t, is a bounded Fs 0 s t -martingale under Px for Ä Ä ¹ º Ä Ä every x Rm, where F 0 s 0 denotes the ﬁltration generated by the diffusion 2 ¹ s I º process . Thus ÄZ t v.t; x/ ExŒM0 ExŒMt ExŒf .t / Ex .t r; r /dr D D D C 0 ÄZ t ExŒf .t / Ex .r; t r /dr : D C 0 In other words, Z t m m m v.t; x/ Tt f .x/ Tt s.s; /.x/ds for .t; x/ Œ0; T R : (5.3) D C 0 2

~~In fact, by [23, Chapter IV], the operator Gm has a fundamental solution p.t; x; y/ such that Z Z tZ v.t; x/ p.t; x; y/f .y/dy p.t s; x; y/.s; y/dyds: D Rm C 0 Rm~~

(It follows that p.t; x; y/ is the transition density function of t t 0 .) Deﬁne ¹ I º Z tZ v1.t; x/ p.t s; x; y/.s; y/dyds WD 0 Rm and Z v2.t; x/ p.t; x; y/f .y/dy: WD Rm 36 Z.-Q.Chen, H. Wang and J. Xiong

By estimates (13.1) and (14.11)–(14.12) in [23, Chapter IV], there is a constant c1 > 0 such that for every t 0, =2 v1 0;2 t c1t .x/; t L .Œ0;t Rm/ : (5.4) k k I Ä b c I C k k 1 2 m m For f C C .R /, deﬁne for .t; x/ Œ0; 1 R 2 b 2 m J0f .t; x/ T f .x/; D t Z t m Jkf .t; x/ Tt s.ImJk 1f .s; //.x/ds; k 1: D 0

0; 2 m @.t;x/ Let C .Œ0; 1 R / and for 1 j m, write j .t; x/ . For 2 b Ä Ä WD @xj each i j m, x; y Rm and w . 1; 1/, there is Œ0; 1 such that Ä Ä 2 2 2 ˇ ˇ .t; x wej / .t; x/ j .t; x/w C ˇ .t; y wej / .t; y/ j .t; y/w ˇ C ˇ ˇ ˇ j .t; x wej / j .t; y wej / j .t; x/ j .t; y/ ˇ w D C C j j ´ µ X k 2 X k min @x.t; / w ; @x.t; / x y w Ä k k1j j k k1 j jj j k 2 k 2 j jD j jD X k 2 @x.t; / x y w : (5.5) Ä k k1 j j j j k 2 j jD For each i j m, x; y Rm and w 1, by the mean-value theorem, Ä Ä 2 j j ˇ ˇ ˇ .t; x wej / .t; x/ .t; y wej / .t; y/ ˇ C C ® ¯ (5.6) min 2 j .t; / x y ; 4 .t; / : Ä k k1 j j k k1 Recall that .0; 1 .2 ˛// and Note 2 2 ^ in @ formula, Z .t; x wej / .t; x/ @x .t; x/w ˛=2.t; x/ c C j dw subscript xj ˛ 1 ˛ of the ﬁrst D w R 0< w <1 w C ¹ 2 W j j º j j integral: replace Z .t; x wej / .t; x/ colon by c˛ C 1 ˛ dw: C w R w 1 w C semi- ¹ 2 W j j º j j colon? We deduce from (5.5)–(5.6) and Lemma 3.1 that there is a constant c2 > 0 so that for every C 0;2.Œ0; 1 Rm/, 1 j m, and t .0; 1, 2 Ä Ä 2 ˛=2 X k m m xj .x/; t L .Œ0;t R / c2 @x L .Œ0;t R /: b c I C k k 1 Ä k k 1 k 2 j jÄ Interacting superprocesses with discontinuous spatial motion 37

Since the function r b.r/ ˛ is bounded, -Hölder continuous and 7! j j m X ˛ ˛=2 Im.t; x/ b.xj / .t; /.x/; D j j xj j 1 D there is a constant c3 > 0 such that for t .0; 1, 2 Im .x/; t Im L .Œ0;t Rm/ c3 0;2 t : b c I C k k 1 Ä k k I From this and (5.2)–(5.4), with c0 c0.1/ > 0, we have for t Œ0; 1, D 2 J0f 2 ;1 c0 f 2 ; k k C Ä k k C =2 =2 J1f 0;2 t c1c3t J0f 0;2 t c0c1c3t f 2 ; k k I Ä k k I Ä k k C and hence by induction,

~~k k k =2 Jkf 0;2 t c0c1 c3 t f 2 for k 2: k k I Ä k k C 1 1 1 2= Now take T0 min c c ; 1 . Then D 2 ¹ 1 3 º~~

~~X1 c0 Jkf 0;2 T0 f 2 2c0 f 2 : k k I Ä =2 k k C Ä k k C k 0 1 c1c3T0 D P 0;2 m Deﬁne u.t; x/ k1 0 Jkf .t; x/. Then u Cb .Œ0; T0 R / with WD D 2~~

u 0;2 T0 2c0 f 2 : k k I Ä k k C Since Z t m m u.t; x/ Tt f .x/ Tt s.Imu.s; //.x/ds; (5.7) D C 0 1;2 m we have u C .Œ0; T0 R / by Theorem 12 in [19, Chapter 1]. 2 =2; m We next show that Imu.s; /.x/ C .Œ0; T0 R /. To prove this, ﬁrst 2 b observe that for each 1 j m, Ä Ä i0 Z X u.s; x wej / u.s; x/ ˛=2u.s; /.x/ lim c dw xj ˛ C 1 ˛ D i0 w R 2 i < w 21 i w !1 i 1 C D ¹ 2 W j jÄ º j j Z u.s; x wej / u.s; x/ c˛ C 1 ˛ dw C w R w >1 w C ¹ 2 Wj j º j j i0 X lim c˛ Ii .s; x/ c˛I0.s; x/: DW i0 C !1 i 1 D 38 Z.-Q.Chen, H. Wang and J. Xiong

For 0 s t T0 and 1 i i0, Ä Ä Ä Ä Ä ˇZ Â ˇ Ii .s; x/ ˇ u.s; x wej / u.s; x/ j j D ˇ w R 2 i < w 21 i C Ã ˇ ¹ 2 W j jÄ º @ 1 ˇ u.s; x/w 1 ˛ dwˇ @xj w ˇ j j C .1 i/ Â Ã Z 2 X k 1 ˛ sup @xu.s; x/ w dw Ä Rm j j 2 i k 2 x j jD 2 2 ˛ Â Ã 2 1 .2 ˛/i X k 2 sup @xu.s; x/ D 2 ˛ m j j k 2 x R j jD 2 and so 2 ˛ 2 1 .2 ˛/i Ii .s; x/ Ii .t; x/ 2 2 u 1;2 T0 : j j Ä 2 ˛ k k I On the other hand, ˇZ ˇ Ii .s; x/ Ii .t; x/ ˇ u.s; x wej / u.t; x wej / j j D ˇ w R 2 i < w 21 i C C ˇ ¹ 2 W j jÄ º Á 1 ˇ .u.s; x/ u.t; x// dwˇ w 1 ˛ ˇ j j C .1 i/ Â Ã Z 2 1 ˛ 4 sup @su.s; x/ t s w dw m i Ä .s;x/ Œ0;T0 R j j j j 2 2 ˛ 4.1 2 / i˛ 2 u 1;2 T0 t s : Ä ˛ k k I j j

By the above two displays, there exists a positive constant c5 > 0, independent of m i 1, such that for every s; t Œ0; T0 and x R , 2 2 Á1 =2 Á =2 .2 ˛/i i˛ Ii .s; x/ Ii .t; x/ c5 2 2 t s j j Ä j j .2 ˛ /i =2 c52 t s : D j j By increasing the value of c5 if necessary, clearly we also have by the mean-value theorem

=2 m I0.s; x/ I0.t; x/ c5 t s c5 t s for s; t Œ0; T0 and x R : j j Ä j j Ä j j 2 2 Since < 2 ˛, we conclude that for each 1 j m, Ä Ä ˛=2 ˛=2 =2 m u.s; /.x/ u.t; /.x/ c6 t s for s; t Œ0; T0 and x R : j xj xj j Ä j j 2 2 Interacting superprocesses with discontinuous spatial motion 39

Pm ˛ ˛=2 This shows that the function Imu.s; /.x/ j 1 b.xj / xj .s; /.x/ is in =2; m D D j j Cb .Œ0; T0 R /. 1 =2;2 m Now we have from (5.2)–(5.3) and (5.7) that u C C C .Œ0; T0 R / 2 b and 8 @u.t; x/ .m/ m < Gmu.t; x/ Imu.t; x/ L u.t; x/ for .t; x/ .0; T0 R ; @t D C D 2 : u.0; x/ f .x/ for x Rm: D 2 .m/ For any t .0; T0, applying Itô’s formula to s u.t s; Zs / (see, e.g., the 2 7!.m/ calculation for (3.4)), we see that s u.t s; Zs / is a bounded martingale for 7! s Œ0; t. Thus 2 h .m/ i h .m/ i m u.t; x/ Ex u.0; Z / Ex f .Z / P f .x/ for t T0: D t D t D t Ä This together with the semigroup property of P m t 0 proves the theorem. ¹ t I º R Remark. Deﬁning .x/ R h.y x/h.y/dy, we have by Hölder’s inequality, R 2 WD R h.y/ dy. Moreover, for every x1; : : : ; xm; 1; : : : ; m R, k k1 Ä ¹ º m m !2 X Z X .xi xj /i j h.y xi /i dy 0: D i;j 1 R i 1 D D Thus if the function a.x/ c.x/2 is bounded between two strictly positive con- D stants, then .ij .x///1 i;j m is uniformly elliptic and bounded. The -Hölder condition on and bÄ˛ areÄ satisﬁed when c; b Lip .R/ and h L2.R/ j j 2 b 2 \ L1.R/ Lip.R/ for any .0; 1 ˛ .2 ˛//. \ 2 ^ ^

~~Acknowledgments. We would like to thank the anonymous referee for construc- tive suggestions which lead to an improvement of the paper.~~

~~Bibliography~~

[1] D. Bakry and M. Emery, Diffusions hypercontractives, Lecture Notes in Math. 1123 (1985), 177–206. [2] R. F. Bass, Regularity results for stable-like operators, J. Funct. Anal. 257 (2009), 2693–2722. [3] R. F. Bass and Z.-Q. Chen, Systems of equations driven by stable processes, Probab. Theory Related Fields 134 (2006), 175–214. [4] J. Bertoin, Lévy Processes, Cambridge University Press, Cambridge, 1996. 40 Z.-Q.Chen, H. Wang and J. Xiong

[5] N. Cho, Weak convergence of stochastic integrals driven by martingale measure, Stochastic Process. Appl. 59 (1995), 55–79. [6] P. Courrègr, Générateur inﬁnitésimal d’un semi-groupe de convolution sur Rn, et formule de Lévey-Khinchine, Bull. Sci. Math 88 (1964), 3–30. [7] D. A. Dawson, Inﬁnitely divisible random measures and superprocesses, in: Proc. 1990 Workshop on Stochastic Analysis and Related Topics, Silivri, Turkey, 1992. [8] D. A. Dawson, Measure-valued Markov processes, in: Lecture Notes in Math. 1541 (1993), 1–260. [9] D. A. Dawson and T. G. Kurtz, Application of duality to measure-valued diffusion processes, Lecture Notes in Control and Inform. Sci. 42 (1982), 177–191. [10] D. A. Dawson, Z. Li and H. Wang, Superprocesses with dependent spatial motion and general branching densities, Electron. J. Probab. 6(25) (2001), 1–33. [11] D. A. Dawson, J. Vaillancourt and H. Wang, Stochastic partial differential equations for a class of interacting measure-valued diffusions, Ann. Inst. Henri Poincaré Probab. Stat. 36 (2000), no. 2, 167–180. [12] E. B. Dynkin, Markov Processes, Springer-Verlag, Berlin, 1965. [13] E. B. Dynkin, An Introduction to Branching Measure-Valued Processes, CRM Mo- nograph Series 6, American Mathematical Society, Providence, RI, 1994. [14] E. B. Dynkin, S. E. Kuznetsov and A. V. Skorokhod, Branching measure-valued processes, Probab. Theory Related Fields 99 (1994), no. 1, 55–96. [15] N. El Karoui and S. Roelly, Propriétés de martingales, explosion et représentation de Lévy-Khintchine d’une classe de processus de branchement à valeurs mesures, Stochastic Process. Appl. 38 (1991), 239–266. [16] J. Engländer and D. Turaev, A scaling limit theorem for a class of superdiffusions, Ann. Probab. 30 (2002), no. 2, 683–722. [17] S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence, John Wiley & Sons, New York, 1986. [18] P.J. Fitzsimmons, Construction and regularity of measure-valued branching processes, Israel J. Math. 64 (1988), 337–361. [19] A. Friedman, Partial Differential Equations of Parabolic Type, Prentice-Hall Inc., Upper Saddle River, NJ, 1964. [20] N. Ikeda and S. Watanabe, Stochastic Differential Equations and Diffusion Proces- ses, North Holland, Amsterdam, 1981. [21] A. Jakubowski, On the Skorohod topology, Ann. Inst. Henri Poincaré Probab. Stat. 22 (1986), no. 3, 263–285. [22] A. Jakubowski, The almost sure Skorohod’s representation for subsequences in non- metric spaces, Theory Probab. Appl. 42 (1997), no. 2, 167–174. Interacting superprocesses with discontinuous spatial motion 41

[23] O. A. Ladyženskaja, V.A. Solonnikov and N. N. Ural’ceva, Linear and Quasilinear Equations of Parabolic Type, American Mathematical Society, Providence, RI, 1968. [24] Z. H. Li, H. Wang and J. Xiong, A degenerate stochastic partial differential equation for superprocesses with singular interaction, Probab. Theory Related Fields 130 (2004), 1–17. [25] Z. H. Li, H. Wang and J. Xiong, Conditional Log-Laplace functionals of immigration superprocesses with dependent spatial motion, Acta Appl. Math. 88 (2005), no. 2, 143–175. [26] P.A. Meyer, Un cours sur les intégrales stochastiques, Lecture Notes in Math. 511 (1976), 245–400.

[27] I. Mitoma, Tightness of probabilities on C.Œ0; 1; S 0/ and D.Œ0; 1; S 0/, Ann. Probab. 11 (1983), 989–999. [28] A. Negoro,A. and M. Tsuchiya, Stochastic processes and semigroups associate with degenerate Lévy generating operators, Stoch. and Stoch. Reports 26 (1989), 29–61. [29] R. Pinsky, Transience, recurrence and local extinction proporties of the support for supercritical ﬁnite measure-valued diffusions. Ann. Probab. 24 (1996), 237–267. [30] P. Protter, Stochastic Integration and Differential Equations, Stochastic Modelling and Applied Probability 21, Springer-Verlag, New York, 1990. [31] J. B. Walsh, An introduction to stochastic partial differential equations, Lecture Notes in Math. 1180 (1986), 265–439. [32] H. Wang, State classiﬁcation for a class of measure-valued branching diffusions in a Brownian medium, Probab. Theory Related Fields 109 (1997), 39–55. [33] H. Wang, A class of measure-valued branching diffusions in a random medium, Stoch. Anal. Appl. 16 (1998), no. 4, 753–786.

~~Received November 26, 2009; revised November 28, 2010.~~

Author information Zhen-Qing Chen, Department of Mathematics, University of Washington, Seattle, WA 98195, USA. E-mail: [email protected] Hao Wang, Department of Mathematics, University of Oregon, Eugene, OR 97403-1222, USA. E-mail: [email protected] Jie Xiong, Department of Mathematics, University of Tennessee, Knoxville, TN 37996-1300, USA. E-mail: [email protected]