Arxiv:1808.05010V3 [Math.PR] 22 May 2020 Upre Npr Yterb Rn 19-01-00356

arXiv:1808.05010v3 [math.PR] 22 May 2020 upre npr yteRB rn 19-01-00356. Grant RFBR funded the Engineering by part Data-Centric in on supported Programme the by sponsored theory. ergodic infinite inducing, chain, Markov cha embedded Markov entrance distribution, stationary measure, invariant 28D05. 60F05, 60G40, Mi upre yteESCgatE/031/ n elwhpa Fellowship a and EP/P003818/1 grant EPSRC the by supported is AM phrases. and words Key 2010 ..Daiyo nrneadei akvcan 11 13 8 2 inducing of method the via chains Markov chains exit Markov and exit Entrance and entrance of 3. chains Duality Markov exit and 2.3. Entrance duality their and notation 2.2. chains Basic Markov exit and 2.1. Entrance paper the of 2. Structure questions results related main and 1.3. the Motivation of description and 1.2. Introduction 1.1. Introduction 1. fteetotpso akvcan ntecs hnteiiilchain initial the when case the in chains Markov of types two these of akvchain Markov Abstract. xttmsfrom times exit complement nain esr.Udrcranrcrec-yeassumptions recurrence-type certain Under measure. invariant loyed,i aua a,tevrin fterslsaoe(pro above results the of versions the way, natural classical the a in yields, also hr esuyteMro hi foesot bv h eolev zero intere the above mostly overshoots are of We chain between Markov oscillates the measure. study Lebesgue we the where under started chains xlctfrua o nain esrso hs his hnw st we Then chains. these that of assuming ergodicity measures invariant and for formulas explicit rv eta ii hoe o h ubro ee rsig for crossings level increments. of of number variance the finite for and theorem mean limit central a prove ahmtc ujc Classification. Subject Mathematics NUIG N EE-RSIG FRNO WALKS RANDOM OF LEVEL-CROSSINGS AND INDUCING, u prahi ae ntetcnqeof technique the on based is approach Our egv plctost admwlsin walks random to applications give We TTOAYETAC AKVCHAINS, MARKOV ENTRANCE STATIONARY nue akvchains Markov induced A o akvchain Markov a For LKADRMIJATOVI ALEKSANDAR bandb sampling by obtained c iial,cnie the consider Similarly, . A c −∞ to ee rsig admwl,oesot nesot oa time local undershoot, overshoot, walk, random crossing, Level A n + and hspprpoie rmwr o nlsn nain measures invariant analysing for framework a provides paper This . ∞ Y eso htti hi segdc n s hsrsl to result this use and ergodic, is chain this that show We . stplgclyrcret reuil,adwa Feller. weak and irreducible, recurrent, topologically is Y . Y ihvle naPls pc,cnie the consider space, Polish a in values with rmr:6J0 05,3A0 eodr:6J5 60G10, 60J55, secondary: 37A50; 60G50, 60J10, Primary: Contents ttemmnswe tetr xdset fixed a enters it when moments the at xtMro chain Markov exit N LDSA VYSOTSKY VLADISLAV AND C ´ 1 R d inducing hc ergr s“ttoay Markov “stationary” as regard we which , n xtMro hi,idcdMro chain, Markov induced chain, Markov exit in, rmifiieegdcter.This theory. ergodic infinite from bandb sampling by obtained , yLodsRgse onain Vis VV Foundation. Register Lloyd’s by ( Y a be can lo admwl that walk random a of el admwlswt zero with walks random ie nteppr for paper) the in vided tdi ieso one, dimension in sted Y d hi uniqueness their udy h lnTrn Institute, Turing Alan The t a known a has transient A ,w give we ), Y entrance frno walk, random of rmits from σ tthe at -finite 9 8 7 6 2 2 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

3.1. Setup and notation 13 3.2. Invariant measures of general induced, entrance, and exit Markov chains 17 3.3. Existence and uniqueness for weak Feller chains 24 4. Applications to random walks in Rd 29 5. Futher results on level-crossings for one-dimensional random walks 32 5.1. The limit theorem for the number of level-crossings 32 5.2. Expected occupation times between level-crossings 36 Acknowledgements 37 Appendix A. Induced transformations in inﬁnite ergodic theory 37 References 42

1. Introduction

1.1. Introduction and description of the main results. Let S = (Sn)n 0 with Sn = S + X + ... + X be a non-degenerate random walk in Rd, where d 1, with≥ independent 0 1 n ≥ identically distributed (i.i.d.) increments X1,X2,... and the starting point S0 that is a random vector independent of the increments. Deﬁne the state space of the walk S, denoted by , as the minimal topologically closed subgroup of (Rd, +) containing the topological Z support of the distribution of X1. Assume throughout that S0 . Let λ be the normalized Haar measure on ( , +); in dimension one this means that λ([0∈, Z x)) = x for positive x . Z ∈ Z Consider the case d = 1 and assume that either EX1 =0 or EX1 does not exist. This is equivalent to assuming that the random walk S oscillates, that is lim sup Sn = lim inf Sn = + a.s. as n (see Section 4). In particular, S can be transient. Deﬁne− the crossing times∞ of the zero→ ∞level by := 0 and T0 n := inf k> n 1 : Sk 1 < 0,Sk 0 or Sk 1 0,Sk < 0 , n N, T { T − − ≥ − ≥ } ∈ and let

n := S n , n := S n 1, n N (1) O T U T − ∈ be the corresponding overshoots and undershoots; put = := S . It is easy to see, using O0 U0 0 that the n’s are stopping times, that the sequence := ( n)n 0 is a Markov chain. The T O O ≥ sequence := ( n)n 0 also is a Markov chain but this is far less intuitive (cf. Lemma 2.1). ThisU paperU was≥ motivated by our interest in stationarity and stability properties of the Markov chain of overshoots . In our companion paper [33] we essentially showed that this chain has an invariant measureO

c1 ½ π(dx) := ½[0, )(x)P(X1 > x)+ ( ,0)(x)P(X1 x) λ(dx), x , (2) 2 ∞ −∞ ≤ ∈ Z

where either c1 := 2/E X1 if E X1 < (and d = 1) or c1 := 1 otherwise. Note that π is ﬁnite if and only if E| X| < | , in| which∞ case by the assumption of oscillation we have | 1| ∞ EX1 =0 and so π is a probability. We found this measure in [33] computing it in a special case using an ergodic averaging argument and assuming that an invariant measure exists. Then we proved invariance of π in general case using quite interesting but yet complicated and apparently ad-hoc arguments STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 3

based on time-reversibility. The same approach of computing or even guessing and then proving invariance was used in a number of other works (e.g. [8, 28, 36], commented below) concerning stability of certain Markov chains. In all these cases this neither gives insights on the form of invariant measures nor shows how to find them. In particular, this does not explain why the Haar measure, which was introduced in [33] only to simplify the notation, appears in formula (2). Moreover, this approach does not allow one to prove uniqueness. A universal approach for proving uniqueness of the invariant measure is in establishing some type of distributional convergence of n towards π as n (when EX1 = 0) starting S from every x . In this paper we set thisO delicate problem→∞ aside. It is considered, under ∈ Z additional smoothness assumptions on the distribution of X1, in the companion paper [33] using the methods that are entirely different from the ones used here. However, we were not able to establish convergence in full generality. The main difficulty is that no standard criteria of convergence apply to : in particular, this chain in general is neither weak Feller (Remark 3.2) nor ψ-irreducible.O By the same reasoning, a priori it is unclear if the chain has a stationary distribution regardless of moment assumptions on S. O This paper presents a new approach which allows us to find invariant measures and obtain their uniqueness and ergodicity in much more general context than level-crossings of one-dimensional random walks. In order to proceed to this general setting, note that the chain of overshoots has periodic structure since its values at consecutive steps have different signs. Therefore, it suffices to consider the non-negative Markov chain O = (On)n 0 of overshoots at up-crossings defined ≥ by On := 2n ½(S0<0) for n 1 and starting at O0 := S0, and the analogous negative chain O − ≥ O↓ of overshoots at down-crossings. Similarly, define the negative-valued chain U =(Un)n 0 ≥ of undershoots at up-crossings given by Un := 2n ½(S0<0) for n 1 and U0 := S0. This chain played an important role in the proof of equalityU − (2) in [33]. ≥ Observe that the Markov chain of overshoots O at up-crossings above the zero level is obtained by sampling the one-dimensional random walk S at the moments it enters the set [0, ) from ( , 0). Similarly, for any Markov chain Y we can consider the entrance ∞ −∞ A Markov chain, denoted by Y → , constructed by sampling Y at the moments of entry into an arbitrary fixed set A from its complement Ac. We also consider the exit Markov chain, Ac c denoted by Y →, obtained by sampling Y at the exit times from A to A; the Markov property of this sequence is not obvious and we refer to Lemma 2.1 for its proof. In this [0, ) ( ,0) notation, we have O = S→ ∞ and U = S −∞ →. Note that alternating the values of the entrance and exit chains gives the Markov chain of overshoots over the boundary ∂A analogous to but we will not give it any consideration. We will showO (Theorems 2.1 and 3.1) that for any Borel set A in a Polish space and any time-homogeneous Markov chain Y that takes values in and has an invariant σX- A Ac X finite measure µ, the entrance chain Y → and the exit chain Y → have respective invariant measures

entr c exit c µ (dx)= P (Yˆ A )µ(dx) on A, µ c (dx)= P (Y A)µ(dx) on A , (3) A x 1 ∈ A x 1 ∈

where Yˆ is a Markov chain dual to Y with respect to µ and Px(Y0 = Yˆ0 = x) = 1, assuming c exit that Y visits both sets A and A infinitely often Px-a.s. for µAc -a.e. x, and the same holds 4 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY true for Yˆ (in particular, A and Ac are recurrent sets for Y and Yˆ ). We will also show that A Ac the chains Y → and Y → are recurrent and ergodic if so is Y when “started” under µ. The assumptions above are satisfied (Remark 3.1.b) if Y is recurrent started under µ and it can get from Ac to A. We stress that this not a requirement, that is Y can be transient. Ac c It turns out that the exit chain Y → of Y from A to A is dual to the entrance chain ˆ Ac ˆ c exit Y → of Y into A from A with respect to the measure µAc (Theorem 2.1). This immediately exit Ac entr implies that µAc is invariant for the exit chain Y →. This in turn yields invariance of µA A for the entrance chain Y → since formulas (3) are antisymmetric in the sense that their right-hand sides interchange if we swap the chain Y and the set A with the dual chain Yˆ and the complement set Ac. Therefore, we can concentrate on entrance Markov chains. Our results apply for two very wide classes of Markov chains with a known invariant measure, namely reversible chains and random walks on topological groups with the Haar measure. In this paper we will only consider applications with concrete examples for random walks on (Rd, +) (see Section 4). Let us briefly describe possible applications for the first class of Markov chains. Recall that a chain Y is reversible if it is self-dual with respect to entr a σ-finite measure µ, which has to be invariant. For such chains, formula (3) for µA is particularly simple since we can take Yˆ1 = Y1. The assumptions can be verified as follows. A simple necessary and sufficient condition for recurrence of countable reversible Markov chains is due to Lyons [30]. For transient chains other than random walks, we are aware of only one particular example with explicit characterization of all recurrent sets: by Gantert et al. [19, Theorem 1.7], a planar simple random walk conditioned on never hitting the origin visits any infinite subset of Z2 infinitely often a.s. Necessary and sufficient conditions for recurrence of a set for a countable state space Markov chain can be found in Bucy [9], Murdoch [34], and Menshikov et al. [31, Theorem 2.5.8]. Let us stress that the above simple probabilistic explanation of invariance of the measures in (3) neither clarifies how to find them and why they have the form given, nor allows us to prove their uniqueness. Therefore, we use an entirely different method. Our approach is built on inducing, a basic tool of ergodic theory, introduced by S. Kaku- tani in 1943. We need to use infinite ergodic theory since in many cases of interest the invariant measure µ of the chain Y is infinite. In particular, so is the Haar measure λ on the subgroup of Rd. Since λ is invariant for the random walk S and the dual of S with respect to λ isZ S, the first formula in (3) reads as − λentr(dx)= P(X x Ac)λ(dx). A 1 ∈ − This example provides all applications of inducing presented in this paper (see Section 4). Our main interest is in level-crossings of random walks in dimension one, corresponding to A = [0, ). We also briefly consider other choices of A, two of which are discussed below. First,± ∞ let A be such that both sets A and Ac have non-empty interior, and assume ⊂ Z entr that the walk S is topologically recurrent on , so d =1 or d = 2. Then the measure λA AZ is always invariant for the entrance chain S→ into A and is finite if A is bounded. Second, A is either the non-negative or the negative orthant in Rd. Assume that both events Sn 0 and Sn < 0 occur infinitely often a.s.; we always mean that inequalities between{ points≥ } in Rd {hold coordinate-wise.} Notice that in dimension one this assumption is STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 5

equivalent to oscillation of S. Let us again stress that S can be transient. Then the measure

π+(dx) := c1 ½[0, )d (x)(1 P(X1 x))λ(dx), x , (4) ∞ − ≤ ∈ Z entr [0, )d which satisfies π+ = c1λ[0, )d , is invariant for the entrance chain S→ ∞ into the non- ∞ negative orthant [0, )d. In particular, for d = 1 this means that π is invariant for the chain ∞ + O. Combining this with the analogous result for the chain O↓ of overshoots at down-crossings, entr 1 1 which is invariant under π := c1λ d , we obtain that the measure π = π+ + π is − ( ,0) 2 2 − invariant for the chain of overshoots −∞. Let us mention that distributions of the same form O as π+ in d = 1 appear on many occasions, as discussed in detail in [33, Sections 2.1 and 2.2]. Our further result (Theorem 3.2) implies that under the topological assumptions of recurrence, irreducibility, and weak Feller property of the chain Y and, essentially, non- emptiness of the interiors of the sets A and Ac, the questions of existence of an invariant measure, its ergodicity and uniqueness (up to a constant factor) in the class of locally finite A Ac Borel measures have the same answer for each of the chains Y , Y → , Y →. In particular, since the Haar measure is known to be ergodic and unique locally finite invariant measure of the topologically recurrent random walk S on , this yields uniqueness and ergodicity of the invariant measure π for the chain of overshootsZ . More generally (Theorem 4.2), the same entr A O holds for the measure λA and the chain Y → if we assume for simplicity that λ(∂A) = 0. Theorem 3.2 also is a useful tool for proving existence (Proposition 3.2) of a locally finite invariant measure of the weak Feller chain Y when its path empirical distributions are not tight and so the classical Bogolubov–Krylov theorem does not apply. This is based on Kac-type formulas of Proposition 3.1, which in a sense are inverse to those of Theorem 3.1 obtained by inducing. NB: after this paper was finished, we found the works by Lin [29] and Skorokhod [42] on invariant distributions of weak Feller topologically recurrent Markov chains, with a stronger existence result (see Proposition 3.3 and the preceding discussion). Our approach also provides a natural venue to study induced Markov chains (sometimes referred to as embedded chains), obtained by sampling a Markov chain when it belongs to an arbitrary fixed set. These chains are closely related to the entrance and exit chains, and some of our results can actually be obtained by sampling the bivariate chain Z on , formed by the pair of two consecutive values of Y , when Z belongs to the set Ac A. InducedX × X chains are discussed, for example, in Revuz [40, Chapters 2.4 and 3.2], where× they appear in the context of probabilistic potential theory. For completeness of exposition, we also give versions of all our statements on existence and uniqueness of invariant measures for general induced chains. Some of these results, stated in Parts 1 of respective assertions, essentially are not new but it is hard to extract them from the literature even for recurrent chains. Based on our experience, the use of inducing in probability theory is mostly reverse, being related to Kac-type formulas for stationary processes and even more specifically, for Markov chains but only under the omnipresent assumption of recurrence. We stress that our main results Theorems 2.1, 3.1 and Proposition 3.1 also apply to transient Markov chains. We are not aware of any applications of (direct) inducing in problems specifically related to random walks. In the context of level-crossings by one-dimensional random walks, the classical and universal tool is the Wiener–Hopf factorization. Among the vast literature on the topic, the works of Baxter [3], Borovkov [8], and Kemperman [26] are the most relevant 6 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY to the questions considered here. These papers rely on the Wiener–Hopf factorization (which does not yield much for our problem!), in contrast to our entirely different approach. As far as we know, fluctuation theory for random walks in Rd is limited to a single paper by Greenwood and Shaked [20], and our formulas for invariant measures such as (4) are the only explicit high-dimensional results available. The idea to study properties of random walks regarding them as “stationary” processes starting from the Haar measure is novel to us, and we have never seen it in the vast literature on random walks. We are not aware of any works concerning entrance and exit Markov chains of any type. We conclude the paper with a number of one-dimensional results on level-crossings of random walks presented in Section 5. The main result there is the limit theorem for the number of level-crossings motivated below in Section 1.2. Crucially, this theorem requires no assumption other than that the increments are zero-mean and have finite variance. We also present several formulas for expected occupation times between level-crossings. 1.2. Motivation and related questions. This section concerns only the one-dimensional case. There are several good reasons to study overshoots of random walks besides purely theoretical interest. First, there is connection to the local time. Namely, the Markov chain features in Perkins’s [38] definition of the local time of a random walk. There is no conventionalO definition of this notion, see Csörg˝oand Révész [13] for other versions. Let L := max k 0 : n (5) n { ≥ Tk ≤ } be the number of zero-level crossings of the walk by time n, and let ℓ0 be the local time at level 0 at time 1 of a standard Brownian motion. Perkins [38, Theorem 1.3] proved that for 2 2 a zero mean random walk S with finite variance σ := EX1 and starting at S0 = 0, one has Ln 1 d k σℓ0. (6) √n |O | n−→ k →∞ X=1 To the best of our knowledge, all the other limit theorems for the local time (under either definition) of a random walk with finite variance require additional smoothness assumptions on the distribution of increments. Under the above assumptions on the random walk, by ergodicity of the Markov chain 1 n O (Theorem 4.2), we have n k=1 k x π(dx) a.s. for π-a.e. starting point S0 = x ; we will show that this convergence|O |→ actuallyZ | holds| for every x. Hence (6) immediately gives∈ Z P R a limit theorem for the number of level crossings Ln divided by √n (Theorem 5.1). Under 2 the optimal moment assumption EX1 < , limit theorems of such type were first obtained in the early 1980s by A.N. Borodin, who∞ studied more general questions of convergence of additive functionals of consecutive steps of random walks; see Borodin and Ibragimov [7, Chapter V] and references therein. However, Borodin’s method assumes that the distribution of increments of the walk is either aperiodic integer-valued or has a square-integrable characteristic function, and hence it is (Kawata [25, Theorem 11.6.1]) absolutely continuous. We stress that our result, Theorem 5.1, does not require any smoothness assumptions. Second, the Markov chain O appeared in the study of the probabilities that the integrated random walk (S1 + ... + Sk)1 k n stays positive; see Vysotsky [46, 47]. The main idea of the approach of [46, 47] is≤ in≤ a) splitting the trajectory of the walk into consecutive STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 7

“cycles” between the up-crossing times; and b) using that for certain particular distributions of increments, e.g. in the case when the distribution P(X X > 0) is exponential, 1 ∈ ·| 1 the overshoots (On)n 1 are stationary (actually, i.i.d.) regardless of the starting point S0. The current paper was≥ originally motivated by the question whether this approach can be extended to general distributions of increments. Third, there is a close connection to so-called switching random walks. Deﬁne the switching ladder times

′ ′ inf k > n′ 1 : Sk S n−1 , if S n−1 0, 0′ := 0, n′ := { T − ≤ T } T ≥ n N. T T inf k > : S S ′ , if S ′ < 0, ∈ ( n′ 1 k n−1 n−1 { T − ≥ T } T

and the switching ladder heights Zn := S n′ , n 0. The random sequence Z = (Zn)n 0 belongs to a special type of Markov chainsT that≥ we call random walks with switch at zero,≥ whose distributions of increments depend only on the sign of the current position of the chain; the other authors call them oscillating random walks but this can be easily confused with oscillation in the sense used in Section 1.1. More precisely, the transition probabilities P (x,dy) of such a chain Y are of the form P (x,dy)= P (dy x) for x = 0 and P (0,dy)= sign x − 6 αP+(dy)+ (1 α)P (dy), where P+ and P are two probability distributions on R and − − − 1 α [0, 1]. In the symmetric case when P+(dy) = P ( dy) and α = , the sequence ∈ − − 2 ( Yn )n 0 is called a reflected random walk. | |Random≥ walks with switch were introduced by Kemperman [26], and then considered in a few works including that by Borovkov [8]. Reflected random walks received much more attention; see Peignéand Woess [37] for the most recent and comprehensive list of references and generalizations to processes of iterated i.i.d. random continuous mappings. Their relevance to the present paper is that the overshoots above zero level of the switching ladder heights chain Z coincide with those of the random walk S. Recently we learned about the unpublished work by Peignéand Woess [36] who found the invariant measure for a reflected random walk ( Yn )n 0 sampled at the moments of reflection at zero. In our terminology, this is the random| | sequence≥ of absolute values of non-strict overshoots above the zero level by Y . One can check using the Wiener–Hopf factorization that in the special case when Y is the switching ladder heights chain Z generated by a random walk S on = R with symmetrically distributed increments, the invariant measure of [36, Theorem 4.7]Z equals 2 π( ). The other notable fact is that, if EX1 = 0 and EX1 < , then π is an invariant distribution|·| for a random walk with switch at zero defined by P∞ = P(X X < 0), + 1 ∈ ·| 1 P = P(X1 X1 > 0), and α = 1; this can be shown using a stationary distribution for− Y found∈ in ·| [8]. We will use our inducing approach to explore these connections and show further relations to classical stationary distributions of renewal theory in the separate paper [48].

1.3. Structure of the paper. In Section 2 we carefully define the entrance and exit chains and use probabilistic arguments to check their Markov property and the crucial property of duality. In Section 3 we study stationarity of induced, entrance and exit chains using the idea of inducing from ergodic theory: in Section 3.1 we provide a self-contained setup needed to apply inducing in the context of Markov chains; in Section 3.2 we show its use 8 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY to find invariant measures for the three types of chains in question, obtained from a general Markov chain on a topological space; and in Section 3.3 we study existence and uniqueness of these invariant measures for a more specific class of weak Feller chains on Polish spaces. In Section 4 we give applications to the entrance chains sampled from random walks in arbitrary dimension, including the chains of overshoots above a level in dimension one. In Section 5 present further one-dimensional results on level-crossings of random walks, including the limit theorem for the number of level-crossings. We conclude the paper with the Appendix, where we give the basics on the use of inducing in infinite ergodic theory. We present them in a nearly self-contained way and with proofs to ensure assumptions more general than those we found in the literature.

2. Entrance and exit Markov chains and their duality 2.1. Basic notation. Throughout this paper will be a topological space. We always equip subsets of with the subspace (induced)X topology. All the measures on (and its subsets) consideredX here will be Borel measures, that is defined on the Borel σ-algebraX ( ). B X Let Y = (Yn)n 0 be a time-homogeneous Markov chain taking values in . By saying this, we assume that≥ Y is defined on some generic probability space (Ω, , P)X and Y has a transition kernel P on under P. We also assume that (Ω, ) is equippedF with a family of X F probability measures Px x such that: Y is a Markov chain with the transition kernel P { } ∈X under Px and Px(Y0 = x) = 1 for every x , and the function x Px(Y B) is measurable N0 ∈ X 7→ ∈ for any set B ( )⊗ , where N0 := N 0 . By the Ionescu Tulcea extension theorem (Kallenber [24,∈ Theorem B X 6.17]), such family∪{ of} measures always exists for any transition kernel on . For any Borel σ-finite measure µ on , denote Pµ := Px( )µ(dx). Then Y0 X X X · has “distribution” µ under Pµ, in which case we say that Y starts under µ. Although µ is not necessarily a probability, we prefer to (ab)use probabilistic notationR and terminology as above, instead of using respective notions of general measure theory. Denote by Ex and Eµ respective expectations (Lebesgue integrals) over Px and Pµ. We say that a transition kernel Pˆ on is dual to P with respect to a σ-finite measure µ on if the following equality of measuresX on ( ) ( ) is satisfied: X B X ⊗ B X µ(dx)P (x,dy)= µ(dy)Pˆ(y,dx), x,y . (7) ∈ X This equality is called the detailed balance condition. We will also use this definition of duality for Markov chains on subsets of of full measure µ. If is separable, there cannot be two distinct mod µ kernels dual to PX(see Revuz [40, LemmaX 4.7 in Chapter 2]); if is a Polish space, then a dual kernel always exists and is unique mod µ (see Section 3.1). TwoX Markov chains Y and Yˆ on are dual if so are their transition operators. In other words, X assuming for convenience that Yˆ0 = Y0, we have the time-reversal equality P ((Y ,Y ) B)= P ((Yˆ , Yˆ ) B), B ( ) ( ). µ 0 1 ∈ µ 1 0 ∈ ∈ B X ⊗ B X Note that this equality implies that the measure µ is invariant both chains Y and Yˆ , that is P (Y )= P (Yˆ )= µ. More generally, for any k 1 we have µ 1 ∈· µ 1 ∈· ≥ (k+1) P ((Y ,...,Y ) B)= P ((Yˆ ,..., Yˆ ) B), B ( )⊗ . (8) µ 0 k ∈ µ k 0 ∈ ∈ B X STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 9

2.2. Entrance and exit Markov chains. Fix a Borel set A . Deﬁne the entrance c A ⊂ X times to A from A of the chain Y by T0→ := 0 and A A c c Tn→ := inf k > Tn→ 1 : Yk 1 A ,Yk A on Yk A i.o.,Yk A i.o. , n N, { − − ∈ ∈ } { ∈ ∈ } ∈ and the sequences of entrances to A from Ac and exits from Ac to A for Y , respectively, by

A Ac c A A Yn→ := YTn→ and Yn → := YTn→ 1 on Yk A i.o.,Yk A i.o. , n N, − { ∈ ∈ } ∈ A Ac and Y0→ = Y0 → := Y0, where “i.o.” stands for “inﬁnitely often”. We will need to clarify for which initial values Y0 of the Markov chain Y the i.o. events occur. To this end, we deﬁne the set NA′,Y (in short, NA′ ) as follows:

N ′ := x : P (Y A′ i.o.) = 1 , A′ ( ). A ,Y { ∈ X x k ∈ } ∈ B X Conditions when the sets NA and NA NAc are non-empty will be discussed in Section 3. c ∩ c c Deﬁne the exit set of Y from A , denoted by (A )ex,Y (in short, Aex): Ac := x Ac : P (x, A) > 0 . (9) ex,Y { ∈ } A Ac The following result justiﬁes referring to the sequences Y → and Y → as entrance and exit Markov chains. Lemma 2.1. Let Y be a Markov chain taking values in a topological space . Let A be X A ⊂ X a Borel set such that NA NAc is non-empty. Then the entrance sequence Y → and the exit c A ∩ c c c sequence Y → are Markov chains on the respective sets A NA NA and Aex NA NA , with the probability transition kernels ∩ ∩ ∩ ∩ entr A P (x,dy) := P (Y → dy), x,y A N N c A x 1 ∈ ∈ ∩ A ∩ A and

exit Ac c P c (x,dy) := P (Y → dy)P (Y dz Y A), x,y A N N c . (10) A z 1 ∈ x 1 ∈ | 1 ∈ ∈ ex ∩ A ∩ A ZA A Ac We say that a Borel measure ν on is proper for the chains Y → and Y → if ν is Xc supported on NA NAc , i.e. ν((NA NAc ) ) = 0. We will start these chains only under their proper measures. ∩ ∩

Proof. We will use that for any A′ ( ), the set N ′ is absorbing for Y , in the sense that ∈ B X A P (Y N ′ )=1, x N ′ . (11) x 1 ∈ A ∈ A Indeed, for every x N ′ we have ∈ A 1= P (Y A′ i.o.) = P (Y A′ i.o. Y = y)P (Y dy)= P (Y A′ i.o.)P (x,dy). x k ∈ x k ∈ | 1 x 1 ∈ y k ∈ ZX ZX Hence P (Y A′ i.o.) = 1, i.e. y N ′ , for P (x, )-a.e. y. That is, P (Y N ′ ) = 1. y k ∈ ∈ A · x 1 ∈ A It follows from equality (11) that the non-empty set NA NAc is absorbing for Y . Then c ∩ clearly A NA NAc and A NA NAc are non-empty. Hence for every x NA NAc , the c ex ∩ A ∩ ∩A ∩ c c ∈ ∩ c sequences (Yn →)n 1 and (Yn→ )n 1 belong respectively to Aex NA NA and A NA NA , ≥ ≥ ∩ ∩A ∩ ∩ and thus have inﬁnitely many terms, Px-a.s. Then the sequence (Tn→ )n 1 of entrance times c ≥ to A from A is inﬁnite Px-a.s. This is a sequence of stopping times with respect to Y . Then 10 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

A A A it follows from the equality Y → = (YTn→ )n 0 that Y → is a Markov chain under Px. We omit the proof of this standard fact, which≥ is based on the strong Markov property of Y . entr A entr The formula for the transition kernel PA of Y → is evident; PA is a probability kernel on A NA NAc by (11). ∩ ∩ Ac A The Markov property of the exit sequence Y → is not evident since (Tn→ 1)n N are exit − ∈ ex not stopping times with respect to Y . We see from deﬁnition (10) that P c (in short, P c ) c A A c c A is a probability kernel on Aex NA NA by (11). To prove that Y → is a Markov chain on c c ∩ ∩ ex c Aex NA NA with the transition kernel PAc , it suﬃces to show that for any x NA NA , ∩ ∩ c ∈ ∩ integer n 2, and Borel sets B , B ,... A N N c , we have ≥ 1 2 ⊂ ex ∩ A ∩ A

Ac Ac Ac ex ex Px(Y1 → B1,...,Yn → Bn)= Px(Y1 → dx1) PAc (x1, dx2) ... PAc (xn 1, dxn). ∈ ∈ ∈ − ZB1 ZB2 ZBn A The proof is by induction. Let n = 2. Since T1→ is ﬁnite Px-a.s., it is true that

c c ∞ c A A A A Px(Y1 → B1,Y2 → B2)= Px(T1→ = k,Yk 1 B1,Y2 → B2) ∈ ∈ − ∈ ∈ k X=1 ∞ c A A = Px T1→ >k 1,Yk 1 B1,Yk A, Y2 → B2 − − ∈ ∈ ∈ k=1 X ∞ A = Px(T1→ >k 1,Yk 1 dx1) − − ∈ k=1 ZB1 X Ac A Px Yk A, Y2 → B2 Yk 1 = x1, T1→ >k 1 . × ∈ ∈ − −

By the Markov property of Y , for Px(Yk 1 )-a.e. x1 B1 and every k 1 we have − ∈· ∈ ≥ Ac A Ac Px Yk A, Y2 → B2 Yk 1 = x1, T1→ >k 1 = Px1 (Y1 A, Y2 → B2) ∈ ∈ − − ∈ ∈ Ac = P (Y → B )P (Y dz). z 1 ∈ 2 x1 1 ∈ ZA ex On the other hand, from deﬁnition (10) of PAc we see that

Ac ex P (Y → B )P (Y dz)= P (Y A)P c (x , B ). (12) z 1 ∈ 2 x1 1 ∈ x1 1 ∈ A 1 2 ZA Putting everything together, we obtain

c c ∞ A A A Px(Y1 → B1,Y2 → B2)= Px(T1→ >k 1,Yk 1 dx1)Px1 (Y1 A)P (x1, B2) ∈ ∈ − − ∈ ∈ k B1 X=1 Z Ac ex = P (Y → dx ) P c (x , dx ), (13) x 1 ∈ 1 A 1 2 ZB1 ZB2 as required in the case n = 2. This proves the basis of induction. STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 11

To prove the inductive step, we proceed exactly as above and arrive at Ac Ac P (Y → B ,...,Y → B ) x 1 ∈ 1 n+1 ∈ n+1 ∞ c c A A A = Px(T1→ >k 1,Yk 1 dx1) Pz(Y1 → B2,...,Yn → Bn+1) Px1 (Y1 dz). − − ∈ ∈ ∈ ∈ k B1 A X=1 Z Z c We now use the assumption of induction for the probability under A when z NA NA . c c c∈ ∩ Since Px1 (Y1 NA NA ) = 1 for every x1 NA NA because the set NA NA is absorbing for Y , we get∈ ∩ ∈ ∩ R ∩

Ac Ac Ac P (Y → B ,...,Y → B ) P (Y dz)= P (Y dz) f(x )P (Y → dx ), z 1 ∈ 2 n ∈ n+1 x1 1 ∈ x1 1 ∈ 2 z 1 ∈ 2 ZA ZA ZB2 where f is a non-negative measurable function on B2 given by

ex ex f(x2) := PAc (x2, dx3) ... PAc (xn, dxn+1). ZB3 ZBn+1 We claim that for any x B and any non-negative measurable function g on B , 1 ∈ 1 2 Ac ex P (Y dz) g(x )P (Y → dx )= P (Y A) g(x )P c (x , dx ). (14) x1 1 ∈ 2 z 1 ∈ 2 x1 1 ∈ 2 A 1 2 ZA ZB2 ZB2 ex Indeed, for indicator functions g this holds by definition (10) of PAc ; cf. (12). Hence, (14) holds for simple functions (i.e. finite linear combinations of indicator functions) by additivity of the three integrals in (14). Finally, since any non-negative measurable function g can be represented as pointwise limit of a pointwise non-decreasing sequence of simple functions, equality (14) follows from the monotone convergence theorem. Putting everything together and applying (14) with g = f establishes the inductive step (exactly as we obtained (13) applying (12) in the case n = 2). 2.3. Duality of entrance and exit Markov chains. The following assertion is crucial. Theorem 2.1. Let Y and Yˆ be Markov chains that take values in a topological space , are X dual with respect to a σ-finite invariant Borel measure µ, and satisfy Y0 = Yˆ0. Let A ( ) c c ∈ BAX be a set such that N := NA,Y NA ,Y NA,Yˆ NAc,Yˆ is non-empty. Then the exit chain Y → Ac∩ ∩ ∩ c and the entrance chain Yˆ → are dual with respect to the Borel measure on A given by exit c

µ˜ c (dx) := ½ (x)P (Y A)µ(dx), x A . A ,Y N x 1 ∈ ∈ Recall that there always exists a chain Yˆ dual to Y if is a Polish space (see Section 3.1). Note in passing that in the special case when Y is an oscillatingX random walk S on = R and A = [0, ), we actually have (see [33, Section 2.4]) a quite surprising represenZtation ∞ A Ac of the transition kernels of the chains S→ and S → (i.e. O and U) as products of two entr − − kernels reversible (self-dual) with respect to λA,S = π+; see the Introduction for the notation.

A A Corollary 2.1. The entrance chain Y → and the exit chain Yˆ → are dual with respect to exit the measure µ˜A,Yˆ on A. This follows if we apply Theorem 2.1 to the chain Yˆ and the set Ac instead of Y and A. 12 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

exit Ac Corollary 2.2. The measure µÃc,Y is invariant for the exit chain Y → and the measure exit A µÃ,Yˆ is invariant for the entrance chain Y → . The follows immediately from the dualities in Theorem 2.1 and Corollary 2.1. exit entr Corollary 2.3. The measures µAc and µA (defined in (3)) are invariant for the respective Ac A exit c entr c c chains Y → and Y → when µAc (N )= µA (N )=0 (e.g., this holds if µ(N )=0). exit exit exit c c This follows from Corollary 2.2 since the assumption yieldsµ Ã ,Y = µA andµ Ã,Yˆ = entr c exit entr µA . These inequalities hold if µ(N ) = 0 since µAc µ and µA µ. Note that in general, they may cease to be true, for example, if the set≪N is empty. ≪

Ac Ac Proof of Theorem 2.1. By Lemma 2.1, the sequences Y → and Yˆ → are Markov chains c c exit on the respective sets Aex,Y N and A N. Since the measureµ Ãc,Y is supported on Ac N, the duality stated∩ will follow once∩ we check the detailed balance condition ex,Y ∩ exit exit exit entr c µ˜ c (dx)P c (x,dy)=˜µ c (dy)Pˆ c (y,dx), x,y A N. A ,Y A A ,Y A ∈ ex,Y ∩ exit c By definition ofµ ˜ c , this amounts to showing that for any Borel sets B , B A N, A ,Y 1 2 ⊂ ex,Y ∩ exit entr P c (x, B )P (Y A)µ(dx)= Pˆ c (y, B )P (Y A)µ(dy). (15) A 2 x 1 ∈ A 1 y 1 ∈ ZB1 ZB2 Let us transform the l.h.s. The chain Yˆ starts under µ since Yˆ0 = Y0 by assumption. exit Then, using at first definition (10) of the transition operator PAc , we get

Ac LHS (15) = µ(dx) P (Y → B )P (Y dz) z 1 ∈ 2 x 1 ∈ ZB1 ZA ∞ k+m+1 k c m 1 = P (Y ) B (A N) (A N) − B (A N) µ n n=0 ∈ 1 × ∩ × ∩ × 2 × ∩ k,m=1 X ∞ k+m+1 c m 1 k = P (Yˆ ) (A N) B (A N) − (A N) B , µ n n=0 ∈ ∩ × 2 × ∩ × ∩ × 1 k,m=1 X where the third equality followed from duality of Y and Yˆ by (8), and in the second equality we used that the set N is absorbing for the chain Y by (11), the assumption B1 N, and A ⊂ the fact that T → is ﬁnite P -a.s. for every x N by N N N c . We transform the 2 x ∈ ⊂ A,Y ∩ A ,Y last sum above using the same reasoning applied to the dual chain Yˆ instead of Y :

entr ½ ½ ˆ ˆ ˆ c ˆ LHS (15) = Eµ[ A N (Y0) B2 (Y1)PA (Y1, B1)] ∩

entr ½ ½ ˆ c = Eµ[ A N (Y1) B2 (Y0)PA (Y0, B1)] = RHS (15), ∩ where in the second equality we again used duality of Y and Yˆ . This ﬁnishes the proof. Remark 2.1. Assuming that the transition kernel P of the chain Y admits a dual kernel Pˆ with respect to µ, deﬁne the entrance set into A of Y started under µ: A (µ) := x A : Pˆ(x, Ac) > 0 . (16) en,Y { ∈ } STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 13

It is readily seen from the time-reversal equality (see (8) for k = 1) that this set supports entr entr c the entrance measure µA , given by µA = Pµ(Y0 A ,Y1 A ), in the sense that entr entr ∈ ∈ ∩· µ (A A (µ)) = 0 and µ (A′) > 0 for every Borel set A′ A (µ) such that A \ en,Y A ⊂ en,Y µ(A′) > 0. This ensures that Aen,Y (µ) does not depend mod µ on the dual kernel Pˆ chosen. We now can simplify the assumptions of Corollary 2.3 using the equivalences

exit c entr c c c µ c (N )=0 µ (N )=0 µ(A N en ′ )= µ(A N ′ )=0 for Y ′ Y, Yˆ , A ⇔ A ⇔ ex∩ A ,Y en∩ Aex,Y ∈{ } c c where Aen = Aen,Y (µ) and Aex = (A )ex,Y . These equivalences follow rather straightfor- wardly and we will not prove them in full. The fact that the third condition implies the ﬁrst two is essentially shown in the proof of Part 2b of Theorem 3.1 below. Notice that the third condition is symmetric in the sense that it does no change if we ˆ c c substitute (Y, A) by (Y , A ). This simply interchanges the sets Aen and Aex since by (16), c c Aen,Y (µ)= Aex,Yˆ mod µ and Aex,Y = Aen,Yˆ (µ) mod µ. (17)

3. Entrance and exit Markov chains via the method of inducing In this section we study stationarity of general entrance and exit Markov chains using the results of inﬁnite ergodic theory. The application of these results is rather straightforward for recurrent Markov chains and amounts to working with measure preserving shifts on sequences. For transient chains, we need to introduce additional construction allowing the time to run backwards in order to let us work with invertible two-sided shifts on bilateral sequences. The simple time-reversal argument used in the proof of Theorem 2.1 suggests that this extension is indeed natural. For completeness of exposition, we give all results on entrance and exit chains together with analogous statements for closely related induced Markov chains. This addition is very natural within the context used, and it does not take a lot of eﬀort to provide it. The corresponding results (presented in Parts 1 of Theorems 3.1, 3.2 and Proposition 3.1) are not new, morally or essentially, but it is hardly possibly to provide matching references, especially due to the very general assumptions we use.

3.1. Setup and notation. Recall that Y is a Markov chain on the topological space . Y X Denote by Pµ := Pµ(Y ) the “distribution” of Y started under a Borel measure µ on the ∈·N0 Y Y space of sequences ( )⊗ , and denote by Eµ the “expectation” (integral) over Pµ . For the rest ofB SectionX 3.1 we assume that µ a σ-finite non-zero invariant measure of Y . N0 For any x , denote by xi the i-th coordinate of x. Similarly, for any non-empty set ∈ X I I N0, denote by xI the projection of x onto . Let θ be the one-sided shift operator on ⊂N0 X defined by (θx)i := xi+1 for i N0. Then θ is a measure preserving transformation of X N0 ∈ N0 Y the σ-finite measure space ( , ( )⊗ , Pµ ). XN0 B X For any set C ( )⊗ , the shift θ defines the first hitting time τC of C, the first

∈ B τXC τC ½X\C hitting mapping ϕC := θ of C,ϕ ˜C := θ · , and the induced shift θC := (ϕC ) C on C; k | see the Appendix for details. For any k 1 and B ( )⊗ , deﬁne the cylindrical set ≥ ∈ B X N0 CB := x :(x0,...,xk 1) B . { ∈ X − ∈ } 14 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

We will use the short notation τB′ := τCB , which matches in the case k = 1 the traditional probabilistic notation for the hitting time of the set B. For arbitrary k, we can think of τB′ k as of the hitting time of B by the multivariate Markov chain (Yn,...,Yn+k 1)n 0 on . A − ≥ X We can use the new notation to write the entrance chain Y → into a set A ( ) and Ac c ∈ B X the exit chain Y → from A , deﬁned in Section 2.2, as

Ac A n 1 c (Y →,Y → )= θ −c ϕ˜ c (Y ) , on Y A i.o.,Y A i.o. , n N. (18) n n CA ×A CA ×A 0,1 k k ◦ { } { ∈ ∈ } ∈ We will also consider the induced sequence Y A obtained by restricting the chain Y to A: A n 1 Y := θ − ϕ (Y ) on Y A i.o. , n N . (19) n CA ◦ CA 0 { k ∈ } ∈ 0 A It is easy to show, using the strong Markov property of Y , that Y is a Markov chain on A A NA (cf. Lemma 2.1). We will say that a Borel measure ν on is proper for Y if ∩ c X ν(NA) = 0; we will start induced chains only under their proper measures. The powerful idea of ergodic theory is that the induced shift θC is a measure preserving N0 Y transformation of the induced space (C, ( )⊗ C, (P ) ), under certain recurrence-type B X ∩ µ |C assumptions on Y and C; see Lemmas A.1, A.1′, and A.2. Below we introduce the definitions needed to apply these general results of ergodic theory in the context of Markov chains. We also refer the reader to Kaimanovich [23, Section 1] for a brief account of relevant results on invariant Markov shifts, and to Foguel [18] for a detailed one. The Markov chain Y is called recurrent starting under µ if for every Borel set B ⊂ X such that µ(B) < , we have Px(τB′ (Y ) < ) = 1 for µ-a.e. x B. Clearly, we can drop the assumption µ(∞B) < , as readily seen∞ from σ-additivity of∈µ. It follows easily from ∞ invariance of µ that this definition is equivalent to Px( Yn B i.o. )= 1 for µ-a.e. x B; see (48). Following Kaimanovich [23], we say that Y is transient{ ∈ starting} under µ if for every∈ B ( ) such that µ(B) < , we have P ( Y B i.o. )=0 for µ-a.e. x B. We stress ∈ B X ∞ x { n ∈ } ∈ that for transient Y , it may be that Pµ B ( Yn B i.o. ) > 0 when µ(B)= . There is the usual recurrence–transience dichotomy,| see{ Condition∈ } 5 in Lemma 3.1 below.∞ We say that Y is topologically recurrent if Px(τG′ (Y ) < ) = 1 for every open set G and every x G. Warning: this definition matches the∞ ergodic-theoretic one, while⊂ the X Markov chains∈ literature often defines topological recurrence by taking every x instead of every x G (with G = ∅). In certain cases these two definitions are equivalent;∈ X see (32). Furthermore,∈ we say6 that Y is ergodic starting under µ or, synonymously, µ is an ergodic Y invariant measure of Y , if the (Pµ -preserving) shift θ is ergodic. The chain Y is called irreducible starting under µ if every invariant set of Y is µ-trivial, that is for any A (X), c ∈ B the equality Px(Y1 A)= ½A(x) mod µ implies that either µ(A)=0or µ(A ) = 0. Warning: this shall not be confused∈ with µ-irreducibility. We say that Y is topologically irreducible if Px(τG′ (Y ) < ) > 0 for every x and every non-empty open set G . Let us give∞ necessary and sufficient∈ X conditions for recurrence and ergodicity⊂ X of Y . Lemma 3.1. Let Y be a Markov chain that takes values in a topological space and has a σ-finite invariant Borel measure µ. The following statements hold true. X N0 N0 Y 1) Y is recurrent starting under µ iff the shift θ on ( , ( )⊗ , P ) is conservative. X B X µ 2) Y is recurrent starting under µ iff there exists a sequence of sets Bn n 1 ( ) such { } ≥ ⊂ B X that = n 1Bn mod µ, and Pµ Bn (τB′ n (Y )= )=0 and µ(Bn) < for every n 1. X ∪ ≥ | ∞ ∞ ≥ STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 15

k 3) Y is recurrent starting under µ if for some k 1 there exists a set B ( )⊗ such ≥ ∈ B X that P (τ ′ (Y ) = )=0 and P ((Y ,...,Y ) B) < . In particular, this holds if µ B ∞ µ 1 k ∈ ∞ Px(τG′ (Y ) < )=1 for an open set G of finite measure µ and every x . 4) Y is ergodic and∞ recurrent iff it is irreducible⊂ X and recurrent, all properties starting∈ X under µ. 5) If Y is irreducible, then it is either recurrent or transient, all properties starting under µ. The assumption in Condition 3 is not vacuous since in general, a Borel measure on may be infinite on every non-empty open set. For example, if is separable, take a Xsum of δ-measures at the points of a dense countable subset of ;X see Lemma 3.3 below for conditions to exclude such pathologies. Condition 5 extendsX the standard recurrence– transience dichotomy for countable space chains. Proof. 1) For the direct implication, note that since µ is σ-finite, N0 can be exhausted X by countably many cylindrical sets CBn with bases Bn ( ) of finite measure. Each Y ∈ B X set has measure Pµ (CBn ) = µ(Bn) < and is recurrent for θ by recurrence of Y . Then θ is conservative by the Conditions for∞ conservativity from the Appendix. For the reverse implication, every measurable cylindrical set CB is recurrent for θ by conservativity of θ, hence Y is recurrent. 2) These are the necessary and sufficient conditions appeared in the proof of Condition 1. 3) This holds by Maharam’s recurrence theorem from the Appendix restated for Y using Condition 1. Here τ is finite PY -a.e. and PY (C )= P ((Y ,...,Y ) B) < . CB µ µ B µ 1 k ∈ ∞ 4) The direct implication holds since every θ-invariant cylindrical set CB with one- dimensional base B ( ) is PY -trivial. The reverse one is in [23, Proposition 1.7]. ∈ B X µ 5) This is stated in [23, Theorem 1.2]. Since there is neither formal proof nor exact reference given there, let us comment that this claim follows from Foguel [18, Chapter II]. In more detail, we have = C D, where C and D are respectively conservative and dissipative parts, which are disjointX and∪ measurable. It follows (see [18, p. 17]) from irreducibility of Y

that Px(Y1 C) = ½C (x) mod µ. Then either C = X mod µ, in which case Y is recurrent

by [18, Eq.∈ (2.4)], or D = X mod µ, in which case Y is transient by repeating the argument ½ after [18, Eq. (2.4)] (for any B ( ) such that µ(B) < , take f = ½B and u = BM , d ∈ B Xk ∞ ∞ where BM := x B : k=0 Pµ B (Y dx) M , and let M ). ∈ dµ | ∈ ≤ →∞ We will also considerP non-recurrent Markov chains, in which case we shall work with invertible measure preserving transformations on spaces of sequences. The shift θ on one- sided sequences in N0 is not invertible, therefore we shall extend time to negative integers. This corresponds toX the natural extension in ergodic theory. It can be constructed using a standard argument (see Doob [16, Chapter X.1]) based on Kolmogorov’s consistency theorem. However, this theorem may cease to hold if the space is not Polish (see Bogachev [6, Example 7.7.3]). Therefore, here we proceed differently. X Z Z Consider the space of two-sided sequences equipped with the σ-algebra ( )⊗ . ¯ X Z ¯ Z B X The two-sided shift θ on is given by (θx)i := xi+1, where x , i Z. It is invertible 1 X ∈ X Z∈ and θ¯− is measurable, since the collection of measurable sets B such that θ(B) is measurable is a σ-algebra which contains all cylindrical sets. ⊂ X Z Z Define the time-reversal operator R : , given by R(x)i := x i, where i Z, Z NX → X − ∈ for x . Denote the projection onto 0 by x+ := (x , x ,...), and define the mapping ∈ X X 0 1 16 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

N0 N N0 V : − by V (y) i := yi, where i N, for y . It is easy to see that R, V , X + → X − ∈ ∈ X and ( ) are measurable mappings. Note also that for every x , the restriction V C x 0 { 0} · ∈ X | 1 is bijective and its inverse is measurable by the same argument as we used above for θ¯− . Assume now that there exists a transition kernel Pˆ dual to P with respect to µ. This is always true when is Polish space, in which case a dual kernel is unique mod µ (we will X always denote it by Pˆ). Indeed, if µ is a probability measure, then this claim is nothing but the disintegration theorem combined with existence of regular conditional distributions for probability measures on Polish spaces; see Kallenberg [24, Theorems 6.3, 6.4, and A1.2] or Aaronson [1, Theorem 1.0.8]. This extends to σ-finite measures by σ-additivity. Recall that Z Z for a Polish space , we have ( ) ( )= ( ), ( )⊗ = ( ), etc. X B X ⊗ B X N0B X × X B X B X Denote by Pˆ the distribution on ( )⊗ of a Markov chain with the transition kernel x0 B X Pˆ and starting at x0 ; it exists by the Ionescu Tulcea theorem ([24, Theorem 6.17]). We say that the probability∈ X measure Y 1 Y Z P¯ := (Pˆ V − ) P on ( )⊗ , x0 x0 ◦ ⊗ x0 B X Z ( N) N0 is an extended law of Y starting at x , where ( )⊗ is understood as ( )⊗ − ( )⊗ . 0 B X B X ⊗B X We can assume that there is a sequence Yˆ =(Yˆn)n 0 of random elements of , defined ≥ X on the same measurable space (Ω, ) with Y , such that Yˆ0 = Y0 and for every x0 , a) Yˆ F ˆ ∈ X ˆ is a Markov chain with the transition kernel P under probability Px0 ; and b) Y and Y are N0 N0 independent, as random elements of ( , ( )⊗ ), under probability P . X B X x0 Then Yˆ is a Markov chain dual to Y with respect to µ. Moreover, Yˆ is recurrent starting under µ iff so is Y ; this follows from Conditions 1 and 2 of recurrence and Remark A.1.b. Z Z Indeed, we can define Y and Yˆ on the canonical space (Ω, )=( , ( )⊗ ) with + ˆ + Z P F ¯Y X B X ω = x by taking Y (ω)= x and Y (ω)= R(x) for x and x0 = Px0 for x0 . Then N0 ∈ X ∈ X for any x0 and any sets B1, B2 ( )⊗ C x0 , we have ∈ X ∈ B X ∩ { } Y + + Px (Yˆ B ,Y B )= Px (V (Yˆ ) V (B ),Y B )= P¯ V (R(x) ) V (B ), x B , 0 ∈ 1 ∈ 2 0 ∈ 1 ∈ 2 x0 ∈ 1 ∈ 2 where the first equality holds true and makes sense since the function V C x is bijective and { 0} | + + its inverse is measurable (hence V (B1) is measurable). By the equality x =(V (R(x) ), x ) ¯Y and the definition of the extended law Px0 , we get Y 1 Y Px (Yˆ B ,Y B )= P¯ (V (B ) B )=(Pˆx V − )(V (B )) P (B ). 0 ∈ 1 ∈ 2 x0 1 × 2 0 ◦ 1 · x0 2 N 1 Since B1 = x0 B1′ for some B1′ ( )⊗ , we have V − (V (B1)) = B1′ , hence ˆ 1 { } × ˆ ˆ∈ B X X × Px0 V − (V (B1)) = Px0 (B1) because Px0 is supported on C x0 . Therefore, we get { } Y Px (Yˆ B ,Y B )= Pˆx (B ) P (B ), 0 ∈ 1 ∈ 2 0 1 · x0 2 ˆ which implies independence of Y and Y under Px0 , as required. Furthermore, arguing as above, we can obtain Y P¯ = Px (V (Yˆ ),Y ) , x . x0 0 ∈· 0 ∈ X ¯Y Z Notice that the function x0 Px0 (B) is measurable for every B ( )⊗ since the collection of the sets B with this7→ property is a σ-algebra (this follows∈ B fromX the monotone STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 17

convergence theorem) which contains measurable rectangles. Then we can deﬁne

Y Z P¯ (B) := P (V (Yˆ ),Y ) B µ(dx ), B ( )⊗ , (20) µ x0 ∈ 0 ∈ B X ZX an extended law of Y starting under µ, which is simply the law of (V (Yˆ ),Y ) under Pµ. When is a Polish space, such extended law always exists and is unique since the same holds for Pˆ. X We now claim that the two-sided shift θ¯ is an invertible measure preserving transforma- Z Z ¯Y ¯ 1 tion of the σ-finite measure space ( , ( )⊗ , Pµ ). Then so is the measurable mapping θ− . ¯ X B X ¯Y It suffices to show that θ preserves the measure Pµ on cylindrical sets whose bases are measurable rectangles. To this end, it suffices to check that for any sets B0, B1,... ( ) and any integers 1 k n, we have ∈ B X ≤ ≤ Pµ(Y0 B0,...,Yn Bn)= Pµ(Yˆk B0,..., Yˆ1 Bk 1,Y0 Bk,...,Yn k Bn). (21) ∈ ∈ ∈ ∈ − ∈ − ∈

Denote f(x0) := Px0 (Y1 Bk+1,...,Yn k Bn) if k < n and f(x0) := 1 if k = n. Then ∈ − ∈

Pµ(Y0 B0,...,Yn Bn)= µ(x0) f(xk)Px0 (Y0 B0,...,Yk dxk) ∈ ∈ Bk ∈ ∈ ZX Z

= E [½(Y B ,...,Y B )f(Y )] µ 0 ∈ 0 k ∈ k k

= E [½(Yˆ B ,..., Yˆ B )f(Yˆ )] µ k ∈ 0 0 ∈ k 0 ˆ ˆ = Px0 (Yk B0,..., Y1 Bk 1)f(x0)µ(dx0), ∈ ∈ − ZBk

where in the first equality follows from the Markov property of Y (under Px0 ) and the third one follows from time-reversal equality (8). This yields required equality (21) by independence of Y and Yˆ under P for every x . x0 0 ∈ X 3.2. Invariant measures of general induced, entrance, and exit Markov chains. The proof of our first result shows that the method of inducing allowsus to compute invariant measures of the Markov chains mentioned. c c Recall that the exit set Aex of the chain Y from A was defined in (9), and the entrance set Aen of Y into A was defined, in the case when Y has a dual, in (16). Theorem 3.1. Let Y be a Markov chain that takes values in a topological space and has a σ-finite invariant Borel measure µ. Let A be a Borel set. X ⊂ X 1) Assume that µ(A) > 0. Then the induced chain Y A has a proper non-zero invariant Borel measure µA := µ A on A if either of the following conditions is true: a) Y is recurrent| starting under µ; ˆ b) P A (τ ′ (Y )= )=0 and the same holds (instead of Y ) for a Markov chain Y that is µ A ∞ dual to Y with respect to µ and satisfies Yˆ0 = Y0;

c) PµA (τA′ (Y )= )=0 and µ(A) < ; Moreover, if Y is∞ recurrent (resp. recurrent∞ and ergodic) starting under µ, then Y A is recurrent (resp. recurrent and ergodic) starting under µA. 18 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

A Ac 2) Assume that Pµ Ac (Y1 A) > 0. Then the entrance chain Y → and the exit chain Y → have respective proper| ∈ non-zero invariant Borel measures

entr exit c µA := Px(Y1 )µ(dx) on A and µAc (dx) := Px(Y1 A)µ(dx) on A , c ∈· ∈ ZA if either of the following conditions is true: a) Y is recurrent starting under µ; c b) Pµ Ac (τA′ en (Y ) = ) = Pµ Aen (τA′ (Y ) = )=0 and the same holds (instead of Y ) | ex ∞ | ex ∞ for a Markov chain Yˆ that is dual to Y with respect to µ and satisﬁes Yˆ0 = Y0; c c) Pµ Ac (τA′ (Y )= )= Pµ Ac (τA′ (Y )= )=0 and Pµ Ac (Y1 A) < . | ex ∞ | ex ex ∞ | ∈ ∞ A Moreover, if Y is recurrent (resp. recurrent and ergodic) starting under µ, then Y → and Ac entr exit Y → are recurrent (resp. recurrent and ergodic) starting respectively under µA and µAc .

Theorem 3.1 applies to recurrent chains merely if Pµ Ac (Y1 A) > 0, which means that Y can get from Ac to A. However, we stress that Y is not| required∈ to be recurrent; instead, there shall exist a dual chain Yˆ satisfying Assumptions 1b and 2b. The result of Part 1a is not new and it is especially well-known for ﬁnite µ. However, the best explicit reference we found, Revuz [40, Proposition 2.9 in Chapter 3], assumes that Y is Harris-recurrent (this is stronger than recurrence starting under µ). We do not know references for transient Y . For Part 2b, we essentially give a second proof of Theorem 2.1 using the inducing approach (Remark 2.1 in Section 2.3 explains the assumptions), which we will need later for Part 2b of Proposition 3.1.

Remark 3.1. The assumption of recurrence (in Parts a) is the strongest and the assumptions of Parts b are the weakest, in the following precise sense. a) Parts 1c and 2c are secondary but we stated them for the purpose of completeness (on practise, it can be difficult to check irreducibility of Y ). Indeed, if Y is recurrent starting under µ, then Assumptions 1c and 2c are clearly satisfied. In the opposite direction, if Y is irreducible starting under µ, then either of these assumptions implies, by Condition 5 in Section 3.1, that Y is recurrent starting under µ, since Y cannot be transient. entr b) Assume that is a Polish space. Then the measure µA , as defined in Theorem 3.1 in a more general setting,X has the form given by (3); either of Assumptions 1a or 1c implies Assumption 1b; and either of Assumptions 2a or 2c implies Assumption 2b. Indeed, on a Polish space, there always exists a Markov chain Yˆ dual to Y (see Sec- entr tion 3.1). Then the formula for µA simplifies immediately by the duality. Furthermore, if Y is recurrent starting under µ, then Assumption 1b for Y is satisfied, and Assumption 2b for Y holds by Condition 1 in Section 3.1. Also, if Y is recurrent starting under µ, then so is the dual chain Yˆ (see Section 3.1). Then Assumption 1b for Yˆ holds true as just shown. By the same reasoning applied for the set Ac instead of A, we see that Assumption 2b for ˆ c Y is satisfied since Pµ A (Y1 A ) > 0 by the duality and we can write the sets Aen = Aen,Y | c c ∈ ˆ and Aex = (A )ex,Y in terms of Y using equalities (17). Finally, Assumption 1c (resp. 2c) implies Assumption 1b (resp. 2b) by Remark A.1.b. STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 19

c) The results of Theorem 3.1 (as well as of Theorem 2.1 and Proposition 3.1) are purely measure-theoretic and can be restated for any σ-algebra on instead of ( ). The X B X assumptions of Parts a, b, c correspond respectively to those of Lemmas A.2, A.1, A.1′. Proof. 1) a) By Condition 1 in Section 3.1, recurrence of Y is equivalent to conservativity N0 N0 Y Y of the measure preserving shift θ on ( , ( )⊗ , P ). We have P (C )= µ(A) > 0 and X B X µ µ A by Lemma A.2, the induced shift θCA is a measure preserving conservative transformation N0 Y of the induced space (C , ( )⊗ C , (P ) A ). Then for any Borel set B A, A B X ∩ A µ |C ⊂ µ (B)= P (Y B)=(PY ) (C ) A µ 1 ∈ µ |CA B = (PY ) x N0 : θ (x) C = P (θ (Y )) B = P (Y A B), µ |CA ∈ X CA ∈ B µA CA 0 ∈ µA 1 ∈ where in the last equality we used definition (19) of Y A and the equality θ = (ϕ ) . CA CA CA A A Thus, the measure µA is invariant for the induced chain Y . This measure is proper for Y by implication (48) and the fact that (PY ) (τ = ) = 0, which holds by conservativity µ |CA CA ∞ of θ. Clearly, µA is non-zero since µ(A) > 0. A Recurrence of Y starting under µA follows trivially from that of Y starting under µ. It remains to obtain ergodicity of the induced chain from ergodicity and recurrence of Y . Use A Y A representation (19) to write the law of the induced chain Y starting under µA as PµA = Y 1 N0 2 (Pµ ) CA ψ− , where ψ : CA A is defined by ψ(x) := (x0, (θCA (x))0, (θCA (x))0,...). Note| ◦ that ψ(θ (x)) = θ→(ψ(x)) for every x C , implying CA ∈ A 1 1 1 1 N0 θ− (ψ− B)= ψ− (θ− B), B (A) . (22) CA ∈ B A N0 N0 N0 Y In particular, this yields that θ (restricted to A ) is measure preserving on (A , (A)⊗ , PµA ). A B We already know this since µA is an invariant measure for the chain Y and we have N0 N0 N0 (A)⊗ = ( )⊗ A , which follows from the definition of the induced topology B B X ∩ N N on A. To show ergodicity of θ on A 0 , consider an invariant set B (A) 0 , that is 1 Y A 1 1 1 Y ∈ B θ− B = B mod PµA or, equivalently, ψ− (θ− B)= ψ− B mod (Pµ ) CA . By (22), this gives 1 1 1 Y 1 | θC−A (ψ− B) = ψ− B mod (Pµ ) CA , meaning that ψ− B is an invariant set for θCA on CA. | 1 Y Since θCA is ergodic by Lemma A.2, the set ψ− B is (Pµ ) CA -trivial, implying that B is Y A | A PµA -trivial. This proves ergodicity of the induced Markov chain Y starting under µA. b) The two-sided shift θ¯ is an invertible measure preserving transformation of the mea- Z Z Y 1 sure space ( , ( )⊗ , P¯ ), and so is θ¯− . Denote C¯ := C . This is a cylindrical set in X B X µ A A Z with no constraints on negative coordinates. Hence P¯Y (C¯ ) = PY (C )= µ(A) > 0 and X µ A µ A ¯Y ¯ ¯ k ¯ ¯Y Z Pµ CA k 1θ− (CA) = Pµ (x : x0 A, τC¯A = ) \ ∪ ≥ ∈ X ∈ ∞ Y N0 = P x : x A, τ = = P (τ ′ (Y )= )=0. (23) µ ∈ X 0 ∈ CA ∞ µA A ∞ A In particular, by (48) this implies that the measure µA is proper for Y . Similarly, use representation (20) to get

¯Y ¯ ¯k ¯ ¯ Y Yˆ Pµ CA k 1θ (CA) = Pµ x : x0 A, x 1, x 2,... A = Pµ x : x0 A, τCA = =0. \ ∪ ≥ ∈ − − 6∈ ∈ ∞ 20 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

¯ ¯ Z ¯ ¯Y From Lemma A.1, the induced shift θC¯A is measure preserving on (CA, ( )⊗ CA, (Pµ ) C¯A ). Hence for any Borel set B A, B X ∩ | ⊂ Y Y Y µ (B)=(P¯ ) ¯ (C¯ )=(P¯ ) ¯ x : θ¯¯ (x) C¯ = (P ) x : θ (x) C . A µ |CA B µ |CA CA ∈ B µ |CA CA ∈ B A The last probability was already shown in (the proof of) Part 1a to be PµA (Y B). 1 ∈ c) We have PY (C ) > 0 and PY x C , τ = = 0 arguing as in Part 1b, and the µ A µ ∈ A CA ∞ claim follows as in Part 1a if we apply Lemma A.1′ instead of Lemma A.2. 2) a) We argue as above in Part 1a. The cylindrical set CAc A with two-dimensional × c Y c base A A satisﬁes Pµ (CA A) = Pµ Ac (Y1 A) > 0. The induced shift θCAc A on × | × × N0 Y ∈ c c c (CA A, ( )⊗ CA A, (Pµ ) CA ×A ) is measure preserving and conservative. In particular, × B X ∩ × | Y c c implication (48) and the equality (Pµ ) CA ×A (θCA ×A = ) = 0, which holds by conservativity of θ, give us | ∞

Y k c c c c c 0=(Pµ ) CAc A ( θ CA A i.o. )= Pµ( Y0 A ,Y1 A Yk A i.o.,Yk A i.o. ). | × { ∈ × } { ∈ ∈ }∩{ ∈ ∈ } c exit entr Hence, by the deﬁnitions of the sets NA, NA and the measures µAc , µA , we get c c 0= P (Y A (N N c ),Y A)+ P (Y A ,Y A (N N c )) µ 0 ∈ \ A ∩ A 1 ∈ µ 0 ∈ 1 ∈ \ A ∩ A exit c entr = µ c (A (N N c )) + µ (A (N N c )). A \ A ∩ A A \ A ∩ A exit entr Ac A Thus, both measures µAc and µA are proper for the chains Y → and Y →. They are non-zero by entr exit c c µA (A)= µAc (A )= Pµ(Y0 A ,Y1 A)= Pµ Ac (Y1 A) > 0. ∈ ∈ | ∈ We now give the key computation which shows the use of inducing in ﬁnding invariant measures. By Lemma A.2, for any measurable set B Ac A, ⊂ × Y Y N0 P ((Y ,Y ) B)=(P ) c (C )=(P ) c x : θ c (x) C µ 0 1 ∈ µ |CA ×A B µ |CA ×A ∈ X CA ×A ∈ B c P c = µ (Y0,Y1) A A, (θCA ×A (Y )) 0,1 B . ∈ × { } ∈

Combining this with representation (18) and the identity θCAc A =(ϕCAc A ) c , we obtain × × CA ×A c Ac A P ((Y ,Y ) B)= P (Y ,Y ) A A, (Y →,Y → ) B . µ 0 1 ∈ µ 0 1 ∈ × 2 2 ∈ Let us take B = Ac B , where B A is an arbitrary Borel set. Then × 1 1 ⊂ entr c A µ (B )= P (Y ,Y ) A A, Y → B A 1 µ 0 1 ∈ × 2 ∈ 1 A = µ(dx0) Px0 (Y2→ B1 Y1 = x1)Px0 (Y1 dx1) c ∈ | ∈ ZA ZA A = µ(dx0) Px1 (Y1→ B1)Px0 (Y1 dx1) c ∈ ∈ ZA ZA entr entr = PA (x1, B1)µA (dx1), (24) ZA where in the third equality we used the Markov property of Y and in the last one we used entr A entr that the measure µA is proper for the entrance chain Y → (its transition kernel is PA ). entr A Thus, µA is invariant for Y → . STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 21

Similarly, let us take B = B A, where B Ac is an arbitrary Borel set. Then 0 × 0 ⊂ exit c Ac µ c (B )= P (Y ,Y ) A A, Y → B A 0 µ 0 1 ∈ × 2 ∈ 0 Ac = µ(dx0) Px1 (Y1 → B0)Px0 (Y1 dx1) c ∈ ∈ ZA ZA Ac = Px0 (Y1 A)µ(dx0) Px1 (Y1 → B0)Px0 (Y1 dx1 Y1 A) c ∈ ∈ ∈ | ∈ ZAex ZA exit exit = PAc (x0, B0)µAc (dx0), (25) c ZAex exit where in the last equality we used formula (10) for the transition kernel PAc of the entrance Ac exit Ac exit chain Y → and the fact that the measure µAc is proper for Y →. Thus, µAc is invariant Ac for Y →. Recurrence of the entrance chain trivially follows from conservativity of the induced A entr c transformation θCA ×A if we argue as above in (24) to start Y → under µA . Similarly, the exit exit chain is recurrent starting under µAc ; cf. (25). c N N As for ergodicity, deﬁne the functions ψ0 : CAc A (A ) and ψ1 : CAc A A by × → × →

2 c ψi(x) := (xi, (θCAc A (x))i, (θCAc A (x))i,...), x CA A, i 0, 1 . × × ∈ × ∈{ } A A c The entrance chain (Yn→ )n 1 starts from Y1→ , which is Y1 on the event Y0 A ,Y1 A . Since for any Borel set B ≥A it is true that { ∈ ∈ } ⊂

entr c Y c µA (B)= Pµ(Y0 A ,Y1 B) = Pµ x CA A : x1 B , ∈ ∈ ∈ × ∈ A A entr Y 1 c we can write the law of (Yn→ )n 1 with Y1→ distributed according to µA as (Pµ ) CA ×A ψ1− . ≥ Ac Ac exit Y | ◦ 1 c Similarly, the law of the exit chain (Yn →)n 1 with Y1 → following µA is (Pµ ) CA ×A ψ0− . A A≥c | ◦ Then ergodicity of the chains Y → and Y → follows from that of Y exactly as in Part 1. ¯ c ¯Y ¯ c b) We prove as in Part 1b. The cylindrical set CA A satisﬁes Pµ (CA A)= Pµ Ac (Y1 × × | ∈ c c A) > 0. Arguing similarly to (23) and using that τA′ A(Y )= τA′ ex Aen (Y ) mod Pµ, we get × ×

Y k ¯ ¯ c ¯ ¯ c Pµ CA A k 1θ− (CA A) × \ ∪ ≥ × c = Pµ Y0 A ,Y1 A, τA′ c A(Y )= ∈ ∈ × ∞ = P (Y Ac ,Y A ,Y ,Y ,... Ac ) µ 0 ∈ ex 1 ∈ en 2 3 6∈ ex ∞ + P Y Ac ,Y A ,Y ,...,Y Ac ,Y Ac ,Y ,Y ,... A µ 0 ∈ ex 1 ∈ en 2 k 6∈ ex k+1 ∈ ex k+2 k+3 6∈ en k=1 X c Pµ Aen (τA′ (Y )= )+ Pµ Ac (τA′ en (Y )= )=0. (26) ≤ | ex ∞ ∞· | ex ∞ exit entr In particular, this implies (as in Part 1b) that both measures µAc and µA are proper for Ac A the chains Y → and Y →. 22 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

1 Further, by representation (20) and invariance of the right two-sided shift θ¯− , we get Y k ¯ ¯ c ¯ ¯ c Pµ CA A k 1θ (CA A) × \ ∪ ≥ × ¯Y Z c c c = Pµ x :(x0, x1) A A, (x 1, x0) A A, (x 2, x 1) A A, . . . ∈ X ∈ × − 6∈ × − − 6∈ × ¯Y Z c c c = Pµ x :(x0, x 1) A A , (x 1, x 2) A A , (x 2, x 3) A A ,... ∈ X − ∈ × − − 6∈ × − − 6∈ × ˆ ˆ c ˆ = Pµ Y0 A, Y1 A , τA′ Ac (Y )= =0, ∈ ∈ × ∞

c ˆ c ˆ where the last equality follows from τA′ A (Y )= τA′ en Aex (Y ) mod Pµ exactly as in (26). × × ¯¯ c By Lemma A.1, the induced two-sided shift θCA ×A is a measure preserving trans- Z Y Y ¯ c ¯ c ¯ c ¯ c ¯ formation of (CA A, ( )⊗ CA A, (Pµ ) C¯A ×A ). Then the equality (Pµ ) C¯A ×A (CB) = ¯Y ¯ × B¯ X ∩ × | c | (P ) ¯ c θ ¯ c CB holds for any measurable set B A A, and thus µ |CA ×A CA ×A ∈ ⊂ × Y P((Y ,Y ) B)=(P ) c θ c C . µ 0 1 ∈ µ |CA ×A CA ×A ∈ B exit entr This equality implies invariance of µAc and µA as required, as already shown in Part 2a. Y c Y c c) We have Pµ (CA A) > 0 and Pµ x CA A, τCAc A = = 0 arguing as in Part 2b, × ∈ × × ∞ and the claim follows as in Part 2a if we apply Lemma A.1′ instead of Lemma A.2. Notice that we could have proved Part 2 by applying the result of Part 1 to the bivariate chain Z = ((Yn,Yn+1))n 0, which takes values in the space (equipped with ( ) ( ), cf. Remark 3.1.c),≥ has an invariant measure P ((Y ,YX) × X), and satisﬁes theB relationX ⊗ B X µ 0 1 ∈· Ac A Ac A c Zn ×1 =(Yn →,Yn→ ) on Y0 A ,Y1 A , n N, (27) − { ∈ ∈ } ∈ following from (18). However, the proof of Part 2 we presented appears to be the most natural since the full trajectory of the Markov chain Y is already at our disposal due to the setup used. Moreover, our general proof allows other applications of inducing (to be considered in [48]), based on the use of stopping times which are more complicated than the A hitting times T → considered here, where reduction to a multivariate chain is not possible. We next present occupation time formulas for lifting invariant measures of the induced and entrance chains to recover the invariant measure of the underlying Markov chain. The assumptions of the following result are stronger than those of respective parts of Theorem 3.1. Proposition 3.1. Let Y be a Markov chain that takes values in a topological space and X has a σ-ﬁnite non-zero invariant measure µ. Let A ( ) be such that P (τ ′ (Y )= )=0. ∈ B X µ A ∞ 1) We have

τ ′ (Y ) 1 A − N0 P (Y E)= E ½ (Y ,Y ,...) E , E ( )⊗ , (28) µ ∈ µA  k k+1 ∈  ∈ B X k=0 X if either of the following conditions is true:  a) Y is recurrent starting under µ; ˆ ˆ b) there is a dual chain Y satisfying P A (τ ′ (Y )= )=0. µ A ∞ STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 23

2) We have

T →A 1 1 − N entr 0 P (Y E)= E ½ (Y ,Y ,...) E , E ( )⊗ , (29) µ ∈ µA  k k+1 ∈  ∈ B X k=0 X   if Pµ(τA′ c (Y )= )=0 and either of the following conditions is true: a) Y is recurrent∞ starting under µ;

ˆ ˆ c ˆ b) there is a dual chain Y satisfying Pµ Ac (τA′ en (Y )= )= Pµ Aen (τA′ (Y )= )=0. | ex ∞ | ex ∞ Thus, we can recover µ from µ or µentr using (28) or (29) with E = C for B ( ). A A B ∈ B X

Proof. 1) The equality P (τ ′ (Y )= ) = 0 ensures that µ(A) > 0 by µ A ∞

∞ ∞ 0 <µ( )= P (τ ′ (Y )= k) P (Y A)= µ(A). (30) X µ A ≤ µ k ∈ ∞· k k X=1 X=1 Thus, Assumptions 1a and 1b of the proposition are respectively stronger than Assump- tions 1a and 2b of Theorem 3.1. b) As we have seen in the proof of Part 1b of Theorem 3.1, the two-sided shift θ¯ is an Z Z ¯ Y invertible measure preserving transformation of ( , ( )⊗ , Pµ ) satisfying the assumptions X YB X Y of Lemma A.1. For the cylindrical set C¯ , we have P¯ (C¯ )= µ(A) > 0 and P¯ (τ ¯ = )= A µ A µ CA ∞ Pµ(τA′ (Y )= ) = 0. Thus, all the assumptions of Lemma A.3 are satisﬁed. From (52) and ∞ N0 (20) it follows that for any E ( )⊗ , ∈ B X

τC¯ (x) 1 τC (x) 1 A − A −

Y Y ¯k Y k Y

½ ¯ ¯ ½ ¯ ¯ Pµ (E)= Pµ (E)= (θ x E) Pµ (dx)= (θ x E) Pµ (dx). ¯  ∈   ∈  CA k CA k Z X=0 Z X=0     This implies (28) by the equality τCA = τA′ , as required. a) As we have seen in the proof of Part 1a of Theorem 3.1, the one-sided shift θ is a N0 N0 Y measure preserving conservative transformation of ( , ( )⊗ , Pµ ). For the cylindrical Y Y X B X set CA, we have Pµ (CA) = µ(A) > 0 and Pµ (τCA = ) = Pµ(τA′ (Y ) = ) = 0. Thus, all the assumptions of Lemmas A.2 and A.3 are satisﬁed∞ for the one-sided shift∞ θ and the set CA, and (28) follows directly from formula (52) as shown in the proof of Part 1b above. c c 2) Note that Pµ(Y0 A ,Y1 A) > 0. In fact, relation (30) applied for A instead of A ∈ ∈ c and equality P (τ ′ c (Y )= ) = 0 imply that µ(A ) > 0. Then the assertion follows from the µ A ∞ equality PµAc (τA′ (Y )= ) = 0 by an argument analogous to (30). Thus, Assumptions 2a and 2b of the proposition are∞ respectively stronger than Assumptions 2a and 2b of Theorem 3.1. Y c Then for the cylindrical set CAc A, we have Pµ (CAc A)= Pµ(Y0 A ,Y1 A) > 0 and Y × × ∈ ∈ P (τ c = ) = 0. The latter equality follows from the assumptions P (τ ′ (Y )= )=0 µ CA ×A ∞ µ A ∞ and P (τ ′ c (Y )= ) = 0 by the same argument as in (26). µ A ∞ 24 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

a) All the assumptions of Lemmas A.2 and A.3 are satisﬁed for the one-sided shift θ on N0 N0 Y c N0 ( , ( )⊗ , Pµ ) and the set CA A. Then for any E ( )⊗ , we have X B X × ∈ B X

τC c (x) ′ c A ×A τA ×A(Y )

Y k Y k ½ P (E)= ½(θ x E) P (dx)= E (θ Y E,Y A) µ(dx ), µ  ∈  µ x0 ∈ 1 ∈  0 Zc k=1 Zc k=1 CA ×A X A X     where in the ﬁrst equality we applied (52) and shifted the summation indices by one using Y invariance of (P ) c under the induced shift θ c . By the Markov property, we get µ |CA ×A CA ×A

′ c τA ×A(Y ) Y k Pµ (E)= µ(dx0) Ex1 ½(θ Y E) Px0 (Y1 dx1). c  ∈  ∈ A A k Z Z X=0 entr   This implies (29) by deﬁnition of µA , the fact that this measure is proper for the entrance A A c chain T → , and the equality T1→ = τA′ c A(Y )+1 on Y0 A, Yk A i.o.,Yk A i.o. . × { ∈ ∈ ∈ } ¯ Z Z ¯Y ¯ c b) The two-sided shift θ on ( , ( )⊗ , Pµ ) and the set CA A satisfy the assumptions × of Lemmas A.1 and A.3. Then equalityX B X (29) follows from a computation similar to those in the proofs of Parts 1b and 2a above.

3.3. Existence and uniqueness for weak Feller chains. In this section we apply the general results and ideas developed in Section 3.2 for a more specific class of topologically recurrent weak Feller Markov chains. For existence and uniqueness results on invariant measures under much stronger assumptions on Y , such as strong Feller or Harris properties or ψ-irreducibility, see Foguel [18, Chapters IV and VI] and Meyn and Tweedie [32]. We assume throughout that is a metric space. A Markov chain Y with values in is called weak Feller if its transitionX kernel P (x, ) is weakly continuous in x. A Borel Xmeasure on is called locally finite if every point of· admits an open neighbourhood of finite measure.X Such measures are finite on compactX sets. Also, they are σ-finite if is separable since in this case the space can be represented as a countable union of open ballsX of finite measure. Locally finite measures on Polish spaces are often called Radon. The main result of the section (supplemented by Proposition 3.3 below) is as follows. Theorem 3.2. Let Y be a topologically irreducible topologically recurrent weak Feller Markov chain that takes values in a Polish space . Let A be a Borel set with Int(A) = ∅. X ⊂ X 6 1) The mapping µ µA is a bijection between the sets of locally finite Borel invariant measures of the chain7→ Y on and of the induced chain Y A on A. X c 2) Assume that there exists an x Int(A ) such that Px(Y1 Int(A)) > 0. Then the entr ∈exit ∈ mappings µ µA and µ µAc (defined in (3)) are bijections between the sets of locally finite7→ Borel invariant7→ measures of the chain Y on and, respectively, of the A Ac c X entrance chain Y → on A and the exit chain Y → on A . When this paper was nearly finished, we found a work by Skorokhod with a uniqueness result [42, Theorem 3] very similar to Part 1 of our Theorem 3.2 but presented in a more STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 25

complicated way (for the purpose of proving existence, as explained below) and under the assumption that is locally compact. In this setting, the results are equivalent1. It is remarkableX that under the assumptions of Theorem 3.2, the chain Y may have two non-proportional invariant measures even if the space is compact! This is shown in the nice example of Carlsson [10, Theorem 1] whose assumptionsX are satisﬁed by Lemma (3.4); also see Skorokhod [42, Example 1]. Furthermore, the condition Px(Y1 Int(A)) > 0 for an x Int(Ac) excludes the case when the chain can enter A from Ac ∈only through ∂A. The weak∈ Feller property is required only to get surjectivity of the mappings. Note that we cannot use the duality of Section 2.3 to infer the result of Part 2 on the exit chain from that on the entrance chain since we cannot ensure that the dual chain Yˆ is weak Feller. The main use of Theorem 3.2 is when the initial chain Y has a known unique (up to multiplication by positive constant) invariant measure. Our main application, given below in Section 4, concerns random walks on Rd with the invariant Haar measure. On the other hand, Theorem 3.2 can be used to prove existence of an invariant measure for the chain Y . We will give a result in this direction (Proposition 3.2), which easily follows from the next statement. Recall that subsets of are equipped with the subspace topology. X Lemma 3.2. Under the respective assumptions of Theorem 3.2, the induced chain Y A and A the entrance chain Y → are weak Feller on A if P (Y ∂A)=0 for every x . x 1 ∈ ∈ X Corollary. For any topologically recurrent random walk S on R with continuous distribution of increments, the chain O of overshoots at up-crossings of the zero level is weak Feller. We postpone the proof of the lemma for a moment. The corollary follows immediately using that S is weak Feller by P (S )= P(x + X ) for x . x 1 ∈· 1 ∈· ∈ X A A Remark 3.2. In general, the chains Y and Y → may not be weak Feller. For example, consider a recurrent random walk S on R whose distribution of increments is continuous A except for an atom at 1. Take A = [0, ). Then P1(S1 =0)= P1(S1 = 0) > 0 but for any − A ∞ A 0 x< 1, the distribution of S1 is continuous and thus Px(S1 = 0) = 0. Similar examples ≤ A can be constructed for the chain O = S→ of overshoots. We get the following existence result combining Theorem 3.2 with Lemma 3.2 and the classical Bogolubov–Krylov theorem.

1 Let us explain this, for the purpose of exposing a useful “smoothing” trick of Skorokhod. Take a continuous non-zero function g on and put A := Cl( x : g(x) > 0 ). Let (αn)n≥0 be a sequence of X { ∈ X } i.i.d. random variables distributed uniformly over [0, 1] and independent with Y under Px for every x . A P A ∈ X Denote τg := min n 1 : αn g(Yn ) and define the transition kernel Pg(x, dy) := x(Yτg dy) { ≥ ≤ } A ∈ for x, y A, considered in [42, Theorem 3]. Define the Markov chain Z := (Zn)n≥0 := (αn, Yn )n≥0 on ′ := [0,∈1] A and put A′ := (a, x) ′ : a g(x) . X It is easy× to see that the chain{ Z∈on X the≤ Polish} space ′ satisfies the assumptions of Theorem 3.2. X Then da µA(dx) is an invariant locally finite measure for Z iff so is the measure (da µA(dx)) A′ for ⊗ A′ ⊗ | the induced chain Z . This yields, by integrating in a over [0, 1], that the mapping µA µg, given by 7→ µg(x) := g(x)µA(dx) for x A, is a bijection between the sets of locally finite measures invariant under the ∈ A transition kernels Pg and that of Y . Using Part 1 of Theorem 3.2 once again establishes [42, Theorem 3]. The proof in the other direction is by approximation argument. 26 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

Proposition 3.2. Let Y be a topologically recurrent topologically irreducible weak Feller Markov chain that takes values in a locally compact Polish space . Assume that there X exists a compact set A such that Int(A) = ∅ and Px(Y1 ∂A)=0 for every x . Then the chain Y has a⊂ locally X finite non-trivial6 invariant measure.∈ ∈ X This in turn yields, by Skorokhod’s “smoothing” trick explained in Footnote 1, a stronger existence result (which is not explicit in [42] but it was surely evident to Skorokhod, cf. [42, Lemma 4]). It was also obtained by Lin [29, Theorem 5.1] using functional-analytic approach. Proposition 3.3 (Lin [29]; Skorokhod [42]). Every topologically recurrent topologically irreducible weak Feller Markov chain that takes values in a locally compact Polish space has a locally finite non-zero invariant measure. Finally, we mention a more recent paper by Szarek [43] on existence of invariant probability measures for weak Feller chains on non-locally compact spaces. Before proceeding to the proof of Theorem 3.2, we give two simple auxiliary results. Lemma 3.3. Let Y be a topologically irreducible weak Feller Markov chain taking values in a metric space . An invariant Borel measure µ of Y is locally finite if and only if it is finite on some non-emptyX open set. Proof. The necessary condition is trivial. To prove the sufficient one, assume that G is a non-empty open subset of satisfying µ(G) < . By topological irreducibility of Y , for any x there exists an Xn = n(x) 1 such that∞ P (Y G) > 0. It follows by a simple ∈ X ≥ x n ∈ inductive argument that the n-step transition kernel Px(Yn ) is weakly continuous in x. Indeed, for any continuous bounded function f : R, we∈ have · X →

Exf(Yn)= Eyf(Yn 1)Px(Y1 dy), x − ∈ ∈ X ZX by the Chapman–Kolmogorov equation. The integrand is a continuous bounded function by assumption of induction, and so is the integral since Y is weak Feller. P 1 P Then there is an open neighbourhood Ux of x such that y(Yn G) 2 x(Yn G) for every y U . By invariance of µ, this gives ∈ ≥ ∈ ∈ x 1 >µ(G)= Py(Yn G)µ(dy) Py(Yn G)µ(dy) Px(Yn G)µ(Ux), (31) ∞ ∈ ≥ Ux ∈ ≥ 2 ∈ ZX Z implying ﬁniteness of µ(Ux), as required. Lemma 3.4. Let Y be a topologically irreducible topologically recurrent weak Feller Markov chain that takes values in a metric space . Then X P (τ ′ (Y ) < )=1 for every x and non-empty open G . (32) x G ∞ ∈ X ⊂ X Proof. As in the proof of Lemma 3.3, by topological irreducibility and weak Feller property of Y we can ﬁnd an open neighbourhood U of x such that infy U Py(τG′ (Y ) < ) > 0. The claim now follows by topological recurrence and the strong Markov∈ property of∞ the chain Y , which returns to U Px-a.s. STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 27

A A Proof of Theorem 3.2. First of all, the sets of invariant measures of the chains Y , Y → , Ac and Y → are non-empty since they contain the zero measure (an even non-zero ones, by Proposition 3.3). Let µ be a non-zero locally finite Borel invariant measure of the Markov chain Y . Then entr exit the measure µA is Borel and locally finite on A, and so are the measures µA on A and µAc c entr exit c on A as follows from the inequalities µA µA and µAc µA . Further, µ is σ-finite as a locally finite measure on a Polish space. By≤ choosing an open≤ set G in (32) of finite measure, we conclude that Y is recurrent starting under µ by Condition 3 in Section 3.1. Then entr exit Theorem 3.1 applies, and the measure µA, µA , and µAc are invariant for the respective A A Ac chains Y , Y → , and Y →. 1) Let us prove surjectivity of the mapping µ µA. Let ν be a locally finite non-zero Borel invariant measure of the induced chain Y A on7→A. As in (28) applied to one-dimensional cylindrical sets E = C , we can “lift” ν from A to a measure on : B X

τ ′ (Y ) 1 τ ′ (Y ) 1

A − A − ½ ν¯(B) := E ½(Y B) = E (Y B) ν(dy), B ( ). ν  k ∈  y k ∈  ∈ B X k A k X=0 Z X=0     Note the diﬀerence with equality (52) in Lemma A.3, where we considered measures on the N0 trajectory space instead of measures on as here. The crucial observation is that τA′ (Y ) is ﬁnite P -a.e., althoughX we do not require thatX ν µ as in Lemma A.3. In fact, we have ν ≪ A P (τ ′ (Y ) τ ′ (Y ) < )=1 for every y . y A ≤ Int(A) ∞ ∈ X Then by the same argument as in (53), from the equality P (τ ′ (Y )= ) = 0 we obtain ν A ∞

∞ ν¯(B)= P (Y B, τ ′ (Y ) >k), B ( ), ν k ∈ A ∈ B X k X=0 and then

∞ P (Y B)= P (Y B)¯ν(dy)= P (Y B)P (Y dy,τ ′ (Y ) >k) ν¯ 1 ∈ y 1 ∈ y 1 ∈ ν k ∈ A ZX k=0 ZX X′ τA(Y ) ∞

= P (Y B, τ ′ (Y ) k +1)= E ½(Y B) =ν ¯(B), (33) ν k+1 ∈ A ≥ ν k ∈  k k X=0 X=1   where the second to the last inequality is again analogous to (53) and in the last equality we A A ′ used the relation YτA(Y ) = Y1 and the assumed invariance of ν for the induced chain Y . Thus,ν ¯ is an invariant measure for the chain Y and it clearly satisfiesν ¯ A = ν. Since ν is locally finite in the subspace topology, we can find a set G Int(A) open in| this topology such that 0 < ν(G) < . By Lemma 3.3 this implies thatν ¯⊂is locally finite since G is also open in the topology of∞ . Thus, the mapping µ µ is surjective. X 7→ A 28 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

entr 2) We ﬁrst consider the mapping µ µA . To prove its surjectivity, let ν be a locally 7→ A ﬁnite non-zero Borel invariant measure of the entrance chain Y → on A. The Borel measure T →A 1 1 −

µ (B) := E ½(Y B) , B ( ), (34) 1 ν  k ∈  ∈ B X k X=0 is invariant for Y , which can be checked exactly as in the proof of Part 1 above. In fact, A A we have Y → = Y →A by deﬁnition of the entrance chain, where Pν-a.s. ﬁniteness of T → 1 T1 1 follows from the strong Markov property of Y combined with the equalities

P (τ ′ (Y ) τ ′ (Y ) < )= P (τ ′ c (Y ) τ ′ c (Y ) < )=1, y . y A ≤ Int(A) ∞ y A ≤ Int(A ) ∞ ∈ X Further, for any Borel set B A, we have ⊂ ∞ A Py(Y1 B)µ1(dy)= Py(Y1 B)Pν(Yk dy,T1→ >k) c ∈ c ∈ ∈ A k A Z X=0 Z ∞ A = P (Y B, T → = k + 1) ν k+1 ∈ 1 k=0 X A = P (Y → B)= ν(B). (35) ν 1 ∈ c By the assumption we have Px(Y1 Int(A)) > 0 for some x Int(A ), and it follows that there exists a z Int(A) such that∈ for any open set U satisfying∈ z U Int(A) we have P (Y U) > 0.∈ In fact, if this was not true, then every z Int(∈A) would⊂ admit x 1 ∈ ∈ an open neighbourhood Uz Int(A) such that Px(Y1 Uz) = 0. Hence the Borel measure P (Y Int(A)) is zero on⊂ compact sets, and by inner∈ regularity of finite Borel measures x 1 ∈ · ∩ on Polish spaces (Bogachev [6, Theorem 7.1.7]), we arrive at Px(Y1 Int(A)) = 0, which is a contradiction. ∈ By local finiteness of ν, choose an open set U satisfying z U Int(A) such that ∈ ⊂ ν(U) < . It holds Px(Y1 U) > 0 and by the weak Feller property of Y , we can find an open set∞U such that x U∈ Int(Ac) and P (Y U) 1 P (Y U) for every y U . By x ∈ x ⊂ y 1 ∈ ≥ 2 x 1 ∈ ∈ x (35) and exactly the same argument as in (31), this gives µ1(Ux) < . Hence the measure ∞entr µ1 on is locally finite by Lemma 3.3, and so the mapping µ µA is surjective. Its injectivityX follows immediately from Part 2a of Proposition 3.1. 7→ exit exit Now consider the mapping µ µAc . To prove its surjectivity, let ν be a locally finite 7→ Ac c non-zero Borel invariant measure of the exit chain Y → on A . Then the Borel measure exit A c ν := c P (Y Y A)ν (dy) on A is invariant for the entrance chain Y → from A to A y 1 ∈ ·| 1 ∈ A, and the measure µ1 introduced in (34) is invariant for the chain Y . Moreover, we have the followingR equality of Borel measures on Ac:

∞ A P (Y A)µ (dy)= P (Y A)P (Y dy,T → >k) y 1 ∈ 1 y 1 ∈ ν k ∈ 1 k=0 X Ac exit c = Pν(YT →A 1 dy)= Pν(Y1 →)= ν (dy), y A . (36) 1 − ∈ ∈ c Then, if x A is such that Px(Y1 Int(A)) > 0, by weak Fellerness of Y and local ﬁniteness of νexit∈ we can choose an open∈ set U such that x U Int(Ac), νexit(U) is ﬁnite, ∈ ⊂ STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 29

and P (Y Int(A)) 1 P (Y Int(A)) for every y U. By (36), this gives y 1 ∈ ≥ 2 x 1 ∈ ∈ νexit(dy) νexit(dy) 2νexit(U) µ (U)= < , 1 P (Y A) ≤ P (Y Int(A)) ≤ P (Y Int(A)) ∞ ZU y 1 ∈ ZU y 1 ∈ x 1 ∈ exit hence the measure µ on is locally finite by Lemma 3.3. So the mapping µ µ c is 1 X 7→ A surjective. Also, by the equality ν = Ac Py(Y1 )µ1(dy) of measures on A, ν is locally finite ∈· entr since µ1 is so, as we proved earlier. By the established injectivity of the mapping µ µA , R exit 7→ this implies injectivity of the mapping µ µ c . 7→ A Proof of Lemma 3.2. Consider the induced chain Y A. Note that if the set A is open, then this chain is well defined even without topological irreducibility of Y since Y returns to A A infinitely often by topological recurrence. We need to show that the mapping x Exf(Y1 ) is continuous on A for every continuous bounded function f on A. Define the7→ extension f¯ of f on by putting f¯ := 0 on Ac. Then for every x A, X ∈ ∞ A ¯ c Exf(Y1 )= Ex f(Yk)½(Y1,...,Yk 1 A ) , − ∈ k X=1 h i and by the dominated convergence theorem and the fact that Py(τA′ (Y ) < )=1for y A, it suffices to prove that every term is continuous on . We use a simple inductive∞ argument.∈ X ¯ It follows from the weak Feller property of Y that the first term Exf(Y1) is continuous at every x since the bounded function f¯ is continuous on (∂A)c and therefore continuous P (Y ∈)-a.s. X For every k 1, by the Chapman–Kolmogorov equation we have x 1 ∈· ≥ c c

¯ c ¯

½ ½ Ex f(Yk+1)½(Y1,...,Yk A ) = A (y) Ey f(Yk) (Y1,...,Yk 1 A ) Px(Y1 dy). ∈ · − ∈ ∈ h i ZX h i Now we see that the above expression is a continuous function of x. This follows from the weak Feller property of Y and Px(Y1 )-a.s. continuity (in y) of the integrand, whose second factor is continuous by assumption of∈· induction. A Similarly, the weak Feller property of the entrance chain Y → follows from the identity

∞ ∞ A ¯ c Exf(Y1→ )= Ex f(Yn+k)½(Y1,...,Yn 1 A, Yn,...,Yn+k 1 A ) − ∈ − ∈ n=1 k X X=1 h i A by induction on n using the continuity Exf(Y1 ) obtained above for the basis step n = 1. 4. Applications to random walks in Rd In this section we apply the ideas developed in Section 3 to random walks in arbitrary dimension. In particular, we answer our initial questions on stationarity properties of the chain of overshoots of a one-dimensional random walk over the zero level. Although the results of this section follow easily from those of Section 3, we state them as separate theorems. Recall that the state space of the random walk S in Rd, where d 1, was defined in the Introduction as the minimalZ closed subgroup of (Rd, +) containing≥ the topological support of the distribution of X . Let us normalize the Haar measure λ on such that 1 Z λ([0, x)) = λ′([0, x)) for any x 0 in , where λ′ is the Lebesgue measure on the linear hull lin( ) of and [0, x) := y ≥Rd : 0Z y < x ; we always mean that inequalities between Z Z { ∈ ≤ } 30 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY points in Rd hold coordinate-wise. In this section we assume w.l.o.g. that lin( ) has full dimension. We say that S is lattice if = Zd and non-lattice is otherwise. Z Clearly, λ is invariant for the walkZS on = . We can say more. X Z Lemma 4.1. Any topologically recurrent random walk S on its state space , where Rd and d 1, 2 , is recurrent and ergodic starting under λ, which is theZ unique (upZ to⊂ multiplication∈ { by constant)} locally finite Borel invariant measure of S on . Z Recall that topological recurrence of the random walk S on by definition means that Z P0(Sn G i.o.) = 1 for every open neighbourhood G of 0. Note that for such random walks,∈ this equality is in fact true for every non-empty⊂ Z open set G ; see Revuz [40, Proposition 3.4]. Combined with the results of Chung and Fuchs [11, Th⊂eorems Z 1, 3 and 4], this gives that topological recurrence of S on is equivalent to Z 1 lim sup dt = for all a> 0; it X1 r 1 [ a,a]d Re(1 rEe · ) ∞ → − Z − − the limit is always finite for d 3. For d = 1, the limit can be switched with the integral ≥ (Ornstein [35]). In particular, for d = 1 this integral diverges when EX1 = 0, and it may also diverge for arbitrarily heavy-tailed X1 (Shepp [41]). In dimension d = 2, S is topologically 2 recurrent on if EX1 = 0 and E X1 < (Chung and Lindvall [12]). For more general results on recurrenceZ of random walksk k on∞ locally compact abelian metrizable groups, see Revuz [40, Chapters 3.3 and 3.4]. Proof. The uniqueness is by Proposition I.45 in Guivarc’h et al. [21], which states that the right Haar measure on a locally compact Hausdorff topological group G with countable base is a unique invariant Radon Borel measure for any topologically recurrent right random walk on G such that no proper closed subgroup of G contains the support of the distribution of increments of the walk. To infer ergodicity, note that uniqueness of invariant measure implies irreducibility of S starting under λ. In fact, if there is a λ-non-trivial invariant set A ( ) of S, then the ∈ B Z locally finite measure ½Aλ is invariant for S, which contradicts the uniqueness. Further, by Revuz [40, Proposition 3.4], topological recurrence of S implies that Px(τG′ (S) < )=1 for every x and every non-empty open set G . Hence S is recurrent starting∞ under λ by Condition∈ Z 3 in Section 3.1. Therefore, S is⊂ ergodic Z by irreducibility and recurrence, all the three properties starting under λ (Kaimanovich [23, Proposition 1.7]).

We say that a Borel set A is massive for the random walk S if Px(τA′ (S) < )= 1 for λ-a.e. x . In particular,⊂ Z if S is topologically recurrent, then any Borel set∞ of positive measure∈λ Z is massive, as follows (Aaronson [1, Proposition 1.2.2]) from ergodicity and recurrence of S starting under λ (Lemma 4.1). If S is transient (i.e. not topologically recurrent), no set of finite measure can be massive. For walks on = Zd with d 3 2 Z ≥ satisfying EX1 = 0 and E X1 < , there is a necessary and sufficient condition for massiveness of a set, called Wiener’sk k test,∞ stated in terms of capacity, by Itôand McKean [22] and Uchiyama [45]. Easily verifiable sufficient conditions for massiveness in d = 3 are due to Doney [15]. For example, any “line” in Z3 is massive. Under the above assumptions, a set is massive for every such a walk if it is massive for a simple random walk, and so this STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 31

is a property of a set rather than of a walk. Apart from partial results of Greenwood and Shaked [20] (mentioned below) for convex cones with apex at the origin, we are not aware of any explicit results for non-lattice random walks. It appears (based on the estimates of Green’s function in Uchiyama [44, Section 8]) that such results should be fully analogous 2 to the lattice ones for walks with EX1 = 0 and E X1 < if the distribution of X1 has density with respect to the Lebesgue measure. Thek casek of heavy∞ -tailed random walks on Zd, including transient walks in dimensions d =1, 2, is considered by Bendikov and Cygan [4, 5]. Since A is massive for S if and only if A is massive for S, and the random walk S is dual to −S with respect to the measure λ (see, e.g. equality− (2.24) in [33]), Theorem− 3.1 immediately implies the following result. Theorem 4.1. Assume that the sets A, A, Ac, Ac are massive for a random walk S on its d − − entr c state space , where R and d 1. Then the measures λA (dx)= P(X1 x A )λ(dx) Zexit Z ⊂ ≥ c ∈ − A on A and λAc (dx)= P(X1 A x)λ(dx) on A are invariant for the entrance chain S→ Ac ∈ − and exit chain S →, respectively. d 2 Remark 4.1. If = Z with d 3, EX1 = 0 and E X1 < , then the assumptions on A and Ac in TheoremZ 4.1 are not≥ required since a setk isk massive∞ for S whenever it is massive− for a− simple random walk, which is self-dual. We do not know if such reduction is possible for arbitrary S. Let us discuss two particular cases. First, if the random walk S is topologically recurrent on , then the assumptions of Theorem 4.1 are satisﬁed for any λ-non-trivial Borel set A. Z d Second, A is of the form A = A′ , where A′ is a convex cone in R with apex at zero (and S may be transient). Here massiveness∩ Z of Ac and Ac follows from that of A and A. d − − In the case of the non-negative orthant A′ = [0, ) , which is of special interest, we have ∞ X x Ac = X (x A)c = X x A = X x a.s., { 1 ∈ − } { 1 ∈ − } { 1 6∈ − } { 1 6≤ } where in the last expression and below we mean that inequalities between points in Rd hold coordinate-wise. Combining this with the analogous expression for the negative orthant, we get the following.

Corollary 4.1. Assume that τ := τ ′ (0, )d (S) are ﬁnite P0-a.s. Then the measures ± ± ∞

entr entr ½

½ d d λ[0, )d = [0, ) (x)(1 P(X1 x))λ(dx) and λ( ,0)d = ( ,0) (x)(1 P(X1 > x))λ(dx) ∞ ∞ − ≤ −∞ −∞ − are invariant for the chains of entrances into [0, )d and ( , 0)d, respectively. ∞ −∞ entr entr Note that the measures λ d and λ d are always infinite if dim(lin( )) 2. [0, ) ( ,0) Z ≥ It is clear that the assumptions∞ of the−∞ corollary imply that the expectation of every (k) (1) (d) (k) + coordinate X1 of X1 = (X1 ,...,X1 ) is either 0 or does not exist, i.e. E(X1 ) = (k) + + E(X )− = + , where x := max x, 0 and x− := ( x) for a real x. In dimension 1 ∞ { } − one, where τ+ and τ are the first ascending and descending ladder times, this is actually an equivalence (cf.− Feller [17, Theorem XII.2.1] and Kesten [27, Corollary 3]). This is also equivalent to assuming that the one-dimensional random walk S oscillates, that is lim sup S = lim inf S = + a.s. as n . We are not aware of necessary and suffi- n − n ∞ → ∞ cient conditions for P0-a.s. finiteness of τ+ and τ in higher dimensions. By Greenwood and − 32 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

Shaked [20, Corollary 3], in any dimension a sufficient condition is ∞ 1 ∞ 1 P (S > 0) = P (S < 0)=+ . n 0 n n 0 n ∞ n=1 n=1 X X We now state a uniqueness result. Theorem 4.2. Let S be any topologically recurrent random walk on its state space , where Rd and d 1, 2 , and let A be any λ-non-trivial Borel set with λ(∂A)=0Z . Then Z ⊂ ∈{ } A ⊂ Z entr the entrance chain S→ is ergodic and recurrent starting under λA , which is the unique A (up to multiplication by constant) locally finite Borel invariant measure of S→ . The same Ac exit is true for S → starting under λAc . This follows from Theorems 3.1, 3.2 and Lemma 4.1 using that the transition kernel of any random walk is weak Feller by P (S )= P(x + X ); the inequality P c (S x 1 ∈· 1 ∈· λInt(A ) 1 ∈ Int(A)) > 0 holds true since otherwise Cl(A) would be a λ-non-trivial set invariant for S. The assumption λ(∂A) = 0 of the theorem can be relaxed but we prefer to avoid considering sets with “thick” boundary such as (R Q) [0, 1]. \1 ∩ 1 entr entr Recall that for d = 1 we have π = 2 π+ + 2 π with π+ = c1λ[0, ) and π = c1λ( ,0). − ∞ − −∞ Corollary 4.2. If a one-dimensional random walk S is topologically recurrent on , then Z the chains of overshoots O, O↓, and (defined in the Introduction) are ergodic and recurrent O starting respectively under their unique normalized invariant measures π+, π , and π. − Finally, we comment on stability of the “distribution” of the entrance chain into A. This entr question makes a probabilistic sense only if the measure λA is finite and therefore can be normalized to be a probability. For example, this is the case when A = [0, ), EX1 = 0 and d = 1 or the S is topologically recurrent on , A is bounded, and d 1,∞2 . In the former case, the question of stability is studied in ourZ companion paper [33].∈{ In the} latter case, it is reasonable to restrict the attention to convex and compact sets A. These are intervals for d = 1, considered in [33, Section 5.1]. It appears that convergence results in dimension d =2 can be obtained using exactly the same approach as in [33]. 5. Futher results on level-crossings for one-dimensional random walks Throughout this section the random walk S is one-dimensional.

5.1. The limit theorem for the number of level-crossings. Recall that Ln, deﬁned in (5), denotes the number of zero-level crossings of S by time n. Combining Theorem 4.2 on ergodicity of the chain of overshoots with a result by Perkins [38] on convergence of local times of random walks, we obtain the following central limit theorem for Ln. To the best of our knowledge, all other results of this type require some smoothness assumptions for the distribution of increments of the walk. 2 2 Theorem 5.1. For any random walk S such that EX1 = 0 and σ := EX1 (0, ), we have ∈ ∞ Ln σy lim Px y = 2Φ 1, x , y 0, n √n ≤ 2E X − ∈ Z ≥ →∞ | 1| where Φ denotes the distribution function of a standard normal random variable. STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 33

We will need the following auxiliary result, the law of large numbers for the chain . It does not follow directly from ergodicity of (stated in Corollary 4.2) since Birkhoﬀ’sO ergodic theorem implies convergence of the timeO averages only for π-a.e. x.

2 2 Proposition 5.1. Let S be any random walk such that EX1 = 0 and σ := EX1 (0, ). Then for every x , ∈ ∞ ∈ Z n 1 σ2 lim k = y π(dy)= , Px-a.s. (37) n n |O | | | 2E X →∞ k 1 X=1 ZZ | | Proof of Theorem 5.1. Denote by ℓ0 the local time at 0 at time 1 of a standard Brownian motion. By L´evy’s theorem, ℓ0 has the same distribution as the absolute value of a standard normal random variable. Combining this result with Theorem 1.3 by Perkins [38], we get

Ln 1 lim Px k y = 2Φ(y) 1, x =0, y 0; (38) n σ√n |O | ≤ − ≥ →∞ k X=1 note that since Perkins’s definition of crossing times is slightly different from the one of ours, his result shall be applied to the random walk S/σ. On the other hand, by Proposition 5.1, − Ln 1 σ2 lim k = , Px-a.s., x , (39) n L |O | 2E X ∈ Z →∞ n′ k 1 X=1 | | where Ln′ := Ln + ½(Ln = 0) and we used the fact that Px(limn Ln = ) = 1, which holds →∞ ∞ ′ true since S oscillates. Rewriting equality (38) using the identity 1 = Ln 1 and then √n √n · Ln′ combining it with (39) yields the assertion of Theorem 5.1 for x = 0 by Slutsky’s theorem. Furthermore, the results of Perkins actually imply (by Perkins [39]) that equality (38) remains valid, although this is not stated in [38, Theorem 1.3], if we replace x = 0 by x n ∈ Z for any sequence (xn)n 1 such that limn xn/√n = 0. In particular, we can take ≥ ⊂ Z →∞ xn x for an arbitrary x , which yields Theorem 5.1 in full by the above argument. ≡Let us explain in detail∈ Z the extension of (38) stated above. For x = 0, Theorem 1.3 of Perkins [38] is an immediate corollary to his Lemma 3.2 and Corollary 2.2. Our extension of (38) follows in exactly the same way if we let x in Lemma 3.2 be the nearstandard point in ∗R, the field of nonstandard real numbers, that corresponds to the sequence (xn)n 1, in which case st(x) = °x = 0, i.e. the standard part of x is 0. We referred to Cutland [14]≥ to digest the unusual notation and concepts of nonstandard analysis, which were used in [38] with no explanation. Proof of Proposition 5.1. Denote h := inf z : z > 0 ; then either = hZ if h > 0 or = R if h = 0. One can easily check that{ for∈π Z (defined} in (4) with c Z=2/E X ), Z + 1 | 1| 2 ∞ 2 ∞ yπ+(dy)= (y h/2)P(X1 >y)dy = (y h/2)P(X1 >y)dy (40) E X1 h − E X1 0 − ZZ | | Z | | Z and, similarly, 2 ∞ yπ (dy)= (y + h/2)P( X1 >y)dy. (41) − − E X1 0 − ZZ | | Z 34 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

Using that EX1 = 0 and integrating the above equality by parts, we find that the probability 1 1 2 measure π, which, recall, satisfies π = 2 π+ + 2 π , has the first absolute moment σ /(2E X1 ). Therefore, by Birkhoff’s ergodic theorem and ergodicity− of the chain of overshoots asserted| | in Corollary 4.2, the convergence in (37) holds true for π-a.e. x . We need toO prove this for every x . ∈ Z Denote∈ by Z supp π the topological support of π and by N the set of points x supp π that satisfy (37). We clearly have N = supp π in the lattice case h> 0, where is∈ discrete. In the non-lattice case h = 0, so far we only have that N is dense in supp π. ThisZ is because N has full measure π, hence N has full Lebesgue measure λ supp π, as readily seen from definition (2) of π. In order to prove (37), we need to show that| N = supp π, since the chain hits the support of π (which is a closed interval, possibly infinite) at the first step regardlessO of the starting point. Our argument goes as follows. Consider the random walk S′ := (Sn′ )n 0, where Sn′ = X1 + ... + Xn for n 1, starting ≥ ≥ at S′ := 0. Then P (S )= P (x + S′ , x + S′ ,...) . For real y ,y , define the functions 0 x ∈· 0 1 ∈· 1 2

g(y ,y ) := ½(y < 0,y 0 or y 0,y < 0), f(y ,y ) := y g(y ,y ). 1 2 1 2 ≥ 1 ≥ 2 1 2 | 2| 1 2 We claim that for any x supp π and ε (0, 1), there exists a y N such that ∈ ∈ ∈ n n k=1 f(y + Sk′ 1,y + Sk′ ) k=1 f(x + Sk′ 1, x + Sk′ ) lim sup n − n − ε, P-a.s. (42) n g(y + S′ ,y + S′ ) − g(x + S′ , x + S′ ) ≤ →∞ Pk=1 k 1 k Pk=1 k 1 k − − This will imply P that x N and hence proveP Proposition 5.1, since ∈ n 2 Ln 2 k=1 f(y + Sk′ 1,y + Sk′ ) σ 1 σ P lim n − = = Py lim k = =1, n g(y + S′ ,y + S′ ) 2E X1 n L |O | 2E X1 →∞ Pk=1 k 1 k →∞ n′ k=1 ! − | | X | |

P ½ where Ln′ = Ln + (Ln = 0) and the last equality holds by deﬁnition of the set N and the fact that Py(limn Ln = ) = 1, which is true because S oscillates. Thus, it remains to prove inequality (42).→∞ ∞ a1 a2 a1 a2 b1 From the identity = 1 for a1, a2, b1, b2 > 0 and the inequality b1 − b2 b1 − a1 · b2 1 a < 2 a 1 +2 b 1 for a> 0,b> 1 , we see that (42) will follow if we show that for − b | − | | − | 2 any x supp π and ε (0, σ2/(2E X )), there exists a y N such that P-a.s., ∈ ∈ | 1| ∈ n n k=1 f(x + Sk′ 1, x + Sk′ ) k=1 g(x + Sk′ 1, x + Sk′ ) εE X1 − − lim sup n 1 + n 1 | 2 |. n f(y + S′ ,y + S′ ) − g(y + S′ ,y + S′ ) − ≤ σ →∞ Pk=1 k 1 k Pk=1 k 1 k − − (43)

For any Pδ > 0, integer k 1, and any y PN such that x y δ, we have ≥ ∈ | − | ≤

g(x + Sk′ 1, x + Sk′ ) g(y + Sk′ 1,y + Sk′ ) ½( y + Sk′ 1 δ or y + Sk′ δ) | − − − | ≤ | − | ≤ | | ≤ and

f(x + Sk′ 1, x + Sk′ ) f(y + Sk′ 1,y + Sk′ ) | − − − |

δg(y + Sk′ 1,y + Sk′ )+( y + Sk′ + δ)½( y + Sk′ 1 δ or y + Sk′ δ). ≤ − | | | − | ≤ | | ≤ STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 35

This gives

n n ½ k=1 g(x + Sk′ 1, x + Sk′ ) k=1 ½( y + Sk′ 1 δ)+ ( y + Sk′ δ) − 1 | − | ≤ | | ≤ (44) n g(y + S ,y + S ) − ≤ n g(y + S ,y + S ) Pk=1 k′ 1 k′ P k=1 k′ 1 k′ − − and P P n k=1 f(x + Sk′ 1, x + Sk′ ) − 1 n f(y + S ,y + S ) − Pk=1 k′ 1 k′

n − ½ P k=1 δg(y + Sk′ 1,y + Sk′ )+( Xk +2δ)½( y + Sk′ 1 δ)+2δ ( y + Sk′ δ) − n| | | − | ≤ | | ≤ . (45) ≤ k=1 f(y + Sk′ 1,y + Sk′ ) P − By Lemma 3.3, the topologicallyP recurrent random walk S on = R is recurrent and ergodic starting under the Lebesgue measure λ. By Condition 1 inZ Section 3.1, recurrence of S starting under λ implies conservativity of the measure preserving one-sided shift θ on N0 N0 S (R , (R ), Pλ ). Therefore we can apply Hopf’s ratio ergodic theorem (see the Appendix) to theB ratios on the r.h.s.’s of (44) and (45). Let us explain in details, say, why n k=1 g(y + Sk′ 1,y + Sk′ ) E X1 P lim n − = | | =1, λ-a.e. y. (46) n 2 →∞ k=1 f(y + Sk′ 1,y + Sk′ ) σ /2 P − N0 Indeed, consider theP functions on R deﬁned by G(z) := g(z0, z1) and F (z) := f(z0, z1) for z =(z , z ,...) RN0 . Both functions are non-negative, non-zero, and PS-integrable by 0 1 ∈ λ 0 S ∞ Eλ G = Ez0 g(S0,S1)λ(dz0)= P(z0 + X1 0)dz0 + P(z0 + X1 < 0)dz0 = E X1 R ≥ 0 | | Z Z−∞ Z and

S Eλ F = Ez0 [ S1 g(S0,S1)]λ(dz0) R | | Z 0

∞ ½ = E[(z0 + X1)½(z0 + X1 0)]dz0 E[(z0 + X1) (z0 + X1 < 0)]dz0 ≥ − 0 Z−∞ Z ∞ 2

= E[( X z )½( X > z )]dz = E X /2, | 1| − 0 | 1| 0 0 | 1| Z0 where the last equality follows from Fubini’s theorem. Finally, we have

n 1 k S S k=0− G θ Eλ G Pλ lim sup n 1 ◦ S =0 n − F θk − E F 6 →∞ Pk=0 λ ! ◦ n k=1 g(y + Sk′ 1,y + Sk′ ) E X1 P P − = lim sup n |2 | =0 λ(dy), R n f(y + S′ ,y + S′ ) − σ /2 6 Z →∞ Pk=1 k 1 k − hence equality (46) follows from Hopf’s ratio P ergodic theorem. Similarly to (46), for every δ > 0, for λ-a.e. y the sum of the ratios on the r.h.s.’s of (44) and (45) converges P-a.s. as n to →∞ δE X +2δ(E X +2δ)+4δ2 4δ c(δ) := | 1| | 1| + . σ2/2 E X | 1| 36 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

Denote by Nδ the set of y where this P-a.s. convergence holds true. Choose a δ > 0 such that c(δ) <εE X /σ2. The Borel set N N has full measure λ and hence is dense in | 1| ∩ δ |supp π supp π. Therefore we can pick a y N Nδ that satisﬁes x y δ. Then inequality (43) follows from (44) and (45), as required.∈ ∩ | − | ≤

5.2. Expected occupation times between level-crossings. In the rest of the section we present several identities for occupation times, which are direct corollaries of Proposition 3.1 on general Markov chains. Deﬁne the ﬁrst up- and down-crossing times of the zero level as

T := inf k 1 : Sk 1 < 0,Sk 0 and T ↓ := inf k 1 : Sk 1 0,Sk < 0 . { ≥ − ≥ } { ≥ − ≥ } entr entr Recall that 1 = min T, T ↓ , π+ = c1λ[0, ) and π = c1λ( ,0) (see (4)). T { } ∞ − −∞ Proposition 5.2. For any random walk S that oscillates, for any Borel set B we have ⊂ Z ↓ T 1 T 1 1 1

− − T −

½ ½ c λ(B)= E ½(S B) = E (S B) =2E (S B) . 1 π+ k ∈ π− k ∈  π k ∈ "k # k " k # X=0 X=0 X=0 We have not seen these formulas in the random walks literature with exception of one particular case when S is a symmetric simple random walk. Here π+ = δ0 and the ﬁrst formula above is described by Feller [17, Section XII.2, Example b], who praised “the fantastic nature of this result”. In this case, the above formulas are similar to the identity

τ ′ (S) 1 {0} −

1= E ½(S = x) , x , 0 k  ∈ Z k X=0 which holds for any lattice recurrent random walk S and corresponds to A = 0 and { } E = C x in (28). By the same reasoning, we get a version of this result for non-lattice recurrent{ } walks, proved by Ornstein [35, Theorem 0.5]: for any Borel set A R of positive Lebesgue measure λ, ⊂

τ ′ (S) 1 A −

λ(B)= E ½(S B) λ(dx), B (R). x k ∈ ∈ B A k Z X=0 We stress again that in Proposition 5.2 we do not assume recurrence of S. Note also that the formulas in Proposition 5.2 are similar to the pre-zero-crossing occupation measure representation for the renewal measure of ladder heights of a random walk starting at zero (Asmussen [2, Theorem VIII.2.3b]), but in our case the walk starts diﬀerently.

Proof. The Markov chain S on , its invariant measure c1λ, and the set [0, ) satisfy the assumptions of Part 2b of PropositionZ 3.1; see Section 4. Then the ﬁrstZ equality ∩ ∞ follows entr from the fact that π+ = c1λ[0, ) and formula (29) applied to E = CB. The second equality is analogous. The third one∞ follows by applying the ﬁrst two to the sets B [0, ) and B ( , 0). ∩ ∞ ∩ −∞ STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 37

Further, for the number of up-crossings of arbitrary level a by time n 1, deﬁned as ≥ n 1 −

L↑ (a) := ½(S < a,S a), a , n i i+1 ≥ ∈ Z i=0 X we obtain the following surprising result.

Proposition 5.3. For any non-degenerate random walk S satisfying EX1 = 0, we have E L↑ (a)=1 and E L↑ (a)=1 for any a . π+ T π− T ∈ Z Thus, the expected number of up-crossings by the time T does not depend on the level (if S is started under π+ or π , i.e. at stationarity of either chain O or O↓), and therefore − equals 1 since LT↑ (0) = 1 by the deﬁnition of T .

N0 Proof. For the first equality, take E = x : x0 < a, x1 a and A = [0, ) in entr{ ∈ Z ≥ } Z1 ∩ ∞ (29), and use the facts that π+ = c1λ[0, ) and Pλ(S Ea) = Pλ(S E0) = c1− , where the ∞ ∈ ∈ first equality follows by shift-invariance of both the measure λ and the transition kernel of S. The second equality is analogous. From the idea that it is more natural to start the random walk from 0 rather than under π+, we can use Proposition 5.3 to find E0LT↑ (a) for two specific types of distributions of increments. We say that X1 has upward exponential distribution if the conditional distribution P(X > X > 0) is exponential. For every distribution of this type we have 1 ·| 1 π+( ) = P(X1 X1 > 0), which by Proposition 5.3 and the memoryless property of exponential· distributions∈ ·| easily implies (we omit the computations) that

P(X1 > 0) E L↑ (a)= + P(X a X > 0), a> 0. 0 T P(X = 0) 1 ≥ | 1 1 6 We say that X has upward skip-free distribution if P(X 1, 0, 1,... ) = 1. If the random 1 1 ∈{ − } walk S has such increments, then π+ = δ0 and thus E0LT↑ (a) = 1 for every real a. The main application of random walks with upward exponential distributions is in queuing theory, where they feature in the Lindley formula for the waiting times in GI/M/1 queues with exponential service times; see Asmussen [2, Section III.6]. The main application of random walks with skip-free distributions is in theory of branching processes; they also appear in queuing theory [2, Section III.6].

Acknowledgements We thank Wolfgang Woess for discussions and Vadim Kaimanovich for providing a reference to his extremely useful paper. We thank the anonymous referees for their comments and suggestions.

Appendix A. Induced transformations in infinite ergodic theory Here we present basic results on inducing for measure preserving transformations of infinite measure spaces; see Aaronson [1, Chapter 1] for the introduction to infinite ergodic theory. We present few variations of Kakutani’s classical results of 1943, mainly to cover 38 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY inducing on sets of infinite measure. To our surprise, we did not find references for the exact statements we need, and we prove them here. Let T be a transformation of some measurable space (X, ). For any set A , consider F ∈F the first hitting time τA of A and the first hitting mapping ϕA defined by τ (x) := inf n 1 : T nx A , x X and ϕ (x) := T τA(x)x, x τ < . (47) A { ≥ ∈ } ∈ A ∈{ A ∞} and the first return or induced mapping TA := (ϕA) A defined on A τA < . Put also n τÃ(x) | ∩{ ∞} τÃ(x) := inf n 0 : T x A andϕ Ã(x) := T x. All these mappings are measurable. From now{ ≥ on we assume∈ } that m is a measure on (X, ) and the transformation T is F measure preserving on (X, , m). We say that a set A is recurrent for T if τA is finite F k ∈F m-a.e. on A, that is A k 1T − A mod m, where mod m means true possibly except for a m-zero set. If A is recurrent⊂ ∪ ≥ for T , then from invariance of m it follows by simple induction that all iterations of the mapping ϕA are defined m-a.e. on A (see [1, Section 1.5]), that is m (τ = )=0 = m ( T k A i.o. c)=0, (48) |A A ∞ ⇒ |A { ∈ } where “i.o.” stands for “infinitely often”. The following result on induced transformations essentially is in [1, Proposition 1.5.3].

Lemma A.1′. Let T be a measure preserving transformation of a measure space (X, , m), and A be any set recurrent for T such that 0 < m(A) < . Then the induced mappingF T is a∈F measure preserving transformation of the induced space∞ (A, A, m ). A F ∩ |A Relaxing the condition m(A) < requires additional assumptions described in the next two statements. ∞ Lemma A.1. Let T be an invertible measure preserving transformation of a σ-finite mea- k sure space (X, , m). Let A be any set such that m(A) > 0, T A k 1 , k F ∈ F { } ≥ ⊂ F A k 1T A mod m, and A is recurrent for T . Then the induced mapping TA is a measure preserving⊂ ∪ ≥ transformation of the induced space (A, A, m ). F ∩ |A Remark A.1. a) If the mapping T is invertible and (X, ) is a standard measurable space, F 1 that is X is a Polish space and = (X) is its Borel σ-algebra, then the mapping T − F k B is always measurable (and hence T A k 1 ); see [1, Theorem 1.0.3]. In this case, k { } ≥ ⊂ F 1 the condition A k 1T A mod m means recurrence of A for the mapping T − , which preserves m. ⊂ ∪ ≥ 1 b) If T − is measurable (and hence measure preserving) and m(A) < , then A is recur- 1 ∞ k rent for T if and only if A is recurrent for T − , that is the assumptions A k 1T − A mod m k ⊂ ∪ ≥ and A k 1T A mod m are equivalent; see Kaimanovich [23, Proposition 1.3]. The⊂ following ∪ ≥ example shows what can go wrong if we impose only the former of the two assumptions for a set A of infinite measure: if T is the shift on Z equipped with the counting 1 measure and A = N, then TA is not measure preserving on A since TA− ( 1 ) = ∅. In this { } k example TA is not surjective mod m but it is so if we require that A k 1T A mod m. ⊂ ∪ ≥ c) The assumption of σ-finiteness of m can be relaxed to σ-finiteness of m A. The same is valid for Lemma A.2 below. Both claims can be verified easily by examining| the proofs. 1 Proofs of Lemmas A.1 and A.1′. We need to show that m(TA− B)= m(B) for any measurable set B A. By monotonicity of m, it suffices to prove this only for B of finite measure ⊂ STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 39

since m A is σ-ﬁnite. The rest is a standard argument (see the proof of [1, Proposition 1.5.3]), which we| present for convenience of the reader. Since the set A is recurrent for T , we have

∞ ∞ ∞ 1 n n n 1 k 1 m(TA− B)= m(A τA = n T − B)= m(A T − B k=1− T − A)= m(A T − Bn 1), ∩{ }∩ ∩ \∪ ∩ − n=1 n=1 n=1 X X X n n 1 k 1 where B0 := B and Bn := T − B k=0− T − A for n 1. The set T − Bn of ﬁnite measure is 1 \ ∪ ≥ 1 a disjoint union of A T − B and B , hence m(A T − B )= m(B ) m(B ). Then ∩ n n+1 ∩ n n − n+1 ∞ ∞ 1 1 m(TA− B)= m(A T − Bn 1)= m(Bn 1) m(Bn) = m(B) lim m(Bn), ∩ − − − − n n=1 n=1 →∞ X X 1 and this gives (only under recurrence of the set A for T !) m(T − B) m(B) and also A ≤ 1 m(T − B)= m(B) lim m(Bn)=0. (49) A n ⇐⇒ →∞ 1 Under the assumptions of Lemma A.1′, that is m(A) < , we also have m(TA− (A B)) 1 ∞ \ ≤ m(A B) since in the inequality m(TA− B) m(B) the set B can be any measurable subset of the\ set A of ﬁnite measure. Then ≤ 1 1 1 m(T − A) m(T − B)= m(A) m(T − B) m(A) m(B) A − A − A ≤ − 1 1 1 since TA− A = A mod m by recurrence of A. Thus m(TA− B) m(B), and m(B)= m(TA− B). Under the assumptions of Lemma A.1, by invertibility≥ of T we have n n 1 k n n n k n n k B = T − B − T − A = T − B [T − ( T A)] = T − (B ( T A)). (50) n \ ∪k=0 \ ∪k=1 \ ∪k=1 Hence n k k lim m(Bn) = lim m(B ( k=1T A)) = m(B ( k 1T A))=0, (51) n n ≥ →∞ →∞ \ ∪ \ ∪ 1 and so m(B)= m(TA− B) follows from (49).

We say that the transformation T is ergodic if its invariant σ-algebra T := A : 1 I c { ∈ F T − A = A mod m is m-trivial, i.e. for every A T either m(A)=0or m(A ) = 0. We say that T} is conservative if every measurable∈ I subset of X is recurrent for T . By Poincaré’s recurrence theorem, T is conservative if µ(X) < . ∞ Conditions for conservativity. A measure preserving transformation T of a σ-finite measure space (X, , m) is conservative iff there exists a sequence of sets Ak k 1 , all of F { } ≥ ⊂ F finite measure and recurrent for T , such that X = k 1Ak mod m. In particular, this holds k ∪ ≥ if X = k 1T − A mod m, i.e. τA < m-a.e., for some measurable set A of finite measure. ∪ ≥ ∞ The second assertion is known as Maharam’s recurrence theorem. Proof. The direct implication in the first assertion is trivial. For the reverse one, assume that there is a set A of positive measure that is not recurrent for T . Then so is n ∈ F A′ := A n∞=1T − A. Pick a k 1 such that m(Ak A′) > 0. By Lemma A.1′, the induced mapping\T ∪ is measure preserving≥ on the induced space∩ (A , A , m ) of finite measure. Ak k F ∩ k |Ak This mapping is conservative by Poincaré’s recurrence theorem, hence Ak A′ is a recurrent ∩ set for TAk , hence it is recurrent for T , which is a contradiction. 40 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

Next we give a version of Lemma A.1 for conservative transformations. The additional statement on ergodicity is in [1, Propositions 1.2.2 and 1.5.2]. Lemma A.2. Let T be a measure preserving conservative transformation of a σ-finite measure space (X, , m), and A be any set with m(A) > 0. Then T is a measure preserving F ∈F A conservative transformation of the induced space (A, A, m A). Moreover, if T is ergodic, k F ∩ | then TA is ergodic and X = k 1T − A mod m. ∪ ≥ Proof. First of all, TA is well defined since A is recurrent for T by the conservativity. The latter property of T trivially implies conservativity of TA; see [1, Proposition 1.5.1]. Since m 1 is σ-finite, by its monotonicity it suffices to check that m(TA− B)= m(B) for any measurable 1 set B A of finite positive measure. But TB− B = B mod m by conservativity of T , so 1⊂ n n 1 k m(TB− B)= m(B) and hence limn m(T − B k=0− T − B) = 0 by (49). Since B A, this n n 1 k →∞ \ ∪ 1 ⊂ gives limn m(T − B k=0− T − A) = 0, which by (49) implies m(TA− B)= m(B). →∞ \ ∪ k For invertible T , inducing can be reversed under additional assumption X = k 1T − A mod m using so-called suspensions (Kakutani towers). More generally, certain∪ invariant≥ measures of the induced transformation can be lifted to invariant measures of the original transformation, as follows. Denote m := m . A |A Lemma A.3. Let T be a measure preserving transformation of a σ-finite measure space (X, , m), and let A be any set recurrent for T such that m(A) > 0. Then for any σ-finiteF T -invariant measure∈ F ν on (A, A) such that ν m , the measure A F ∩ ≪ A τA(x) 1 − k

ν¯(B) := ½(T x B) ν(dx), B , (52)  ∈  ∈F A k Z X=0   k is invariant for T and satisﬁes ν¯ A = ν. Moreover, if X = k 1T − A mod m and the | ∪ ≥ assumptions of either Lemma A.1,A.1′, or A.2 are satisﬁed, then mA = m.

The assumption ν mA is imposed to ensure that τA is ﬁnite ν-a.e. on A. In the case of conservative T , equation≪ (52) with B = X is known as Kac’s formula.

Proof. The equalityν ¯ A = ν is trivial. Invariance ofν ¯ is standard; see the proof of [1, Proposition 1.5.7] or a similar| argument in (33) below. It remains to prove the last assertion. By monotonicity and σ-ﬁniteness of m, this can be checked only on sets of ﬁnite measure m. For any measurable set B X, ⊂ τA(x) 1 ∞ −

k ½ m (B)= ½(τ (x)= n) (T x B) m(dx) A  A × ∈  A n=1 k Z X X=0  n 1  ∞ − k

= ½(T x B, τ (x)= n) m(dx) ∈ A A "n=1 k # Z X X=0 ∞ k = m(A T − B τ >k ), (53) ∩ ∩{ A } k X=0 STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 41

and therefore, assuming that m(B) < , we get ∞ ∞ ∞ k k n 1 mA(B)= m(A T − B n=1T − A)= m(A B)+ m(A T − Bk′ 1), ∩ \ ∪ ∩ ∩ − k k X=0 X=1 k k n 1 where Bk′ := T − B n=0T − A for k 0. The set T − Bk′ has ﬁnite measure and it is a \ ∪ 1 ≥ 1 disjoint union of A T − B′ and B′ , hence m(A T − B′ )= m(B′ ) m(B′ ). Then the ∩ k k+1 ∩ k k − k+1 sequence m(Bk′ ) is decreasing, and

mA(B)= m(A B)+ m(B0′ ) lim m(Bk′ )= m(B) lim m(Bk′ ). (54) k k ∩ − →∞ − →∞ It remains to show that the limit in the above formula is zero. Let the assumptions of Lemma A.1 be satisfied. Then the mapping TA is invertible mod m. k Indeed, TA is surjective mod m by the assumption A k 1T A mod m of Lemma A.1. To ≥ ⊂ ∪ τA(x1) prove injectivity of TA, assume that TAx1 = TAx1 for some x1, x2 A. Then T x1 = τA(x2) (τA(x1) τA(x2))∈ T x . W.l.o.g., assume τ (x ) τ (x ). Then T − x = x , which is, by 2 A 1 ≥ A 2 1 2 definition of the first hitting time τA of A, possible only if τA(x1)= τA(x2). Hence x1 = x2. k Furthermore, we claim that X = k 1T A mod m. Then equality limk m(Bk′ )=0 ∪ ≥ →∞ follows exactly as limn m(Bn) = 0 followed above from equalities (50) and (51), and we →∞ get mA(B)= m(B) by (54), as required. To prove the claim, put 1 1 V (x) := T − (˜ϕ (x)), x τ˜ < ϕ˜− (T (A)). A A ∈{ A ∞} ∩ A A The mapping V is defined for m-a.e. x X because of the following: TA on A is invertible mod m; ∈k τÃ is finite mod m since X = k 1T − A mod m by assumption of Lemma A.3; and ∪ ≥ 1 1 k m τ˜ < ϕ˜− (T (A)) = m ϕ˜− (A T (A)) m ∞ T − (A T (A)) =0. { A ∞} \ A A A \ A ≤ ∪k=0 \ A Then for m-a.e. x, we have Tk(x)(V (x)) = x for a positive integer k(x) satisfying k(x) = k τA(V (x)) τÃ(x). Thus, equality X = k 1T A mod m holds, as claimed. − ∪ ≥ Let now the assumptions of either Lemma A.1′ or A.2 be satisfied. Then the transformation T is conservative. In the latter case this is by assumption. In the former case, this follows k by Maharam’s recurrence theorem since m(A) < and X = k 1T − A mod m by assump- ∞ ∪ ≥ tion of Lemma A.3. When m(A) < , the equality mA = m holds by [1, Lemma 1.5.4]. The following argument covers both cases∞ of finite and infinite m(A). (N) N n For any integer N 1, denote B := B ( T − A). Notice that for any k N, ≥ ∩ ∪n=1 ≥ we have τ (N) k N τ k , hence { B ≤ − }⊂{ A ≤ } k (N) k n T − (B ) T − A k N < τ (N) k , k N. \ ∪n=0 ⊂{ − B ≤ } ≥ Then for k N, ≥ k N n k n k (N) k n m(B′ )= m T − (B T − A) T − A + m T − (B ) T − A k \ ∪n=1 \ ∪n=1 \ ∪n=1 k N n mT − (B n=1T − A) + N sup m(τB(N) = n) ≤ \ ∪ n>k N − N n n (N) n 1 i (N) = m(B n=1T − A)+ N sup m(T − (B ) i=1− T − B ). \ ∪ n>k N \ ∪ − The first term in the last line can be made as small as necessary by choosing N to be large enough, and the second term vanishes as k for any fixed N by equivalence (49) applied →∞ 42 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY with B(N) substituted for A and B. Such application of (49) is possible since the set B(N) is recurrent for T because T is conservative, as explained above, and m(B(N)) m(B) < . ≤ ∞ Thus, limk m(Bk′ ) = 0, and by (54), this yields the required equality mA(B)= m(B). →∞ Finally, we recall the following classical result; see Zweimüller [49]. Hopf’s ratio ergodic theorem. Let T be a conservative ergodic measure preserving transformation of a σ-finite measure space (X, , m). Then for any functions f,g L1(X, , m) with non-zero g 0, F ∈ F ≥ n 1 k k=0− f T X fdm lim n 1 ◦ = , m-a.e. n − g T k gdm →∞ Pk=0 ◦ RX P ReferencesR [1] Jon Aaronson. An introduction to infinite ergodic theory. American Mathematical Society, Providence, RI, 1997. [2] Søren Asmussen. Applied probability and queues. Springer-Verlag, New York, second edition, 2003. [3] Glen Baxter. A two-dimensional operator identity with application to the change of sign in sums of random variables. Trans. Amer. Math. Soc., 96:210–221, 1960. [4] Alexander Bendikov and Wojciech Cygan. α-stable random walk has massive thorns. Colloq. Math., 138:105–130, 2015. [5] Alexander Bendikov and Wojciech Cygan. On massive sets for subordinated random walks. Math. Nachr., 288:841–853, 2015. [6] V. I. Bogachev. Measure theory. Vol. II. Springer-Verlag, Berlin, 2007. [7] A. N. Borodin and I. A. Ibragimov. Limit theorems for functionals of random walks. Proc. Steklov Inst. Math., 195, 1995. [8] A.A. Borovkov. A limit distribution for an oscillating random walk. In Summary of reports presented at sessions of the seminar on probability theory and mathematical statistics at the Mathematical Institute of the Siberian Section of the USSR Academy of Sciences, 1979, volume 25 of Theory Probab. Appl., pages 649–657. 1981. [9] R. S. Bucy. Recurrent sets. Ann. Math. Statist., 36:535–545, 1965. [10] Niclas Carlsson. Some notes on topological recurrence. Electron. Comm. Probab., 10:82–93, 2005. [11] K. L. Chung and W. H. J. Fuchs. On the distribution of values of sums of random variables. Mem. Amer. Math. Soc., No. 6:12, 1951. [12] Kai Lai Chung and Torgny Lindvall. On recurrence of a random walk in the plane. Proc. Amer. Math. Soc., 78:285–287, 1980. [13] M. Csörg˝ o and P. Révész. On strong invariance for local time of partial sums. Stochastic Process. Appl., 20:59–84, 1985. [14] Nigel J. Cutland. Nonstandard real analysis. In Nonstandard analysis (Edinburgh, 1996), volume 493, pages 51–76. Kluwer Acad. Publ., Dordrecht, 1997. [15] R. A. Doney. Recurrent and transient sets for 3-dimensional random walks. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 4:253–259, 1965. [16] J. L. Doob. Stochastic processes. John Wiley & Sons, Inc., New York; Chapman & Hall, Limited, London, 1953. [17] William Feller. An introduction to probability theory and its applications. Vol. II. Second edition. John Wiley & Sons, Inc., New York-London-Sydney, 1971. [18] Shaul R. Foguel. The ergodic theory of Markov processes. Van Nostrand Reinhold Co., New York- Toronto, Ont.-London, 1969. [19] Nina Gantert, Serguei Popov, and Marina Vachkovskaia. On the range of a two-dimensional conditioned simple random walk. Annales Henri Lebesgue. To appear. STATIONARY ENTRANCE MARKOV CHAINS, INDUCING, AND LEVEL-CROSSINGS 43

[20] Priscilla Greenwood and Moshe Shaked. Fluctuations of random walk in Rd and storage systems. Ad- vances in Appl. Probability, 9:566–587, 1977. [21] Yves Guivarc’h, Michael Keane, and Bernard Roynette. Marches aléatoires sur les groupes de Lie. Springer-Verlag, Berlin-New York, 1977. [22] Kiyosi Itôand H. P. McKean, Jr. Potentials and the random walk. Illinois J. Math., 4:119–132, 1960. [23] Vadim A. Kaimanovich. Ergodicity of harmonic invariant measures for the geodesic flow on hyperbolic spaces. J. Reine Angew. Math., 455:57–103, 1994. [24] Olav Kallenberg. Foundatioins of Modern Probability. Springer, New York, second edition, 2002. [25] Tatsuo Kawata. Fourier analysis in probability theory. Academic Press, New York-London, 1972. [26] J. H. B. Kemperman. The oscillating random walk. Stochastic Process. Appl., 2:1–29, 1974. [27] Harry Kesten. The limit points of a normalized random walk. Ann. Math. Statist., 41:1173–1205, 1970. [28] Frank B. Knight. On the absolute difference chains. Z. Wahrsch. Verw. Gebiete, 43:57–63, 1978. [29] M. Lin. Conservative Markov processes on a topological space. Israel J. Math., 8:165–186, 1970. [30] Terry Lyons. A simple criterion for transience of a reversible Markov chain. Ann. Probab., 11:393–402, 1983. [31] Mikhail Menshikov, Serguei Popov, and Andrew Wade. Non-homogeneous random walks. Cambridge University Press, Cambridge, 2017. [32] Sean Meyn and Richard L. Tweedie. Markov chains and stochastic stability. Cambridge University Press, Cambridge, second edition, 2009. [33] Aleksandar Mijatovićand Vladislav Vysotsky. Stability of overshoots of zero-mean random walks. Ac- cepted in Electron. J. Probab., 2020. [34] B. H. Murdoch. Wiener’s tests for atomic Markov chains. Illinois J. Math., pages 35–56, 1968. [35] Donald S. Ornstein. Random walks. I. Trans. Amer. Math. Soc., 138:1–43, 1969. [36] Marc Peignéand Wolfgang Woess. On recurrence of reflected random walk on the half-line. With an appendix on results of martin benda. 2006. Unpublished, arXiv:math/0612306. [37] Marc Peignéand Wolfgang Woess. Stochastic dynamical systems with weak contractivity properties I. Strong and local contractivity. Colloq. Math., 125:31–54, 2011. [38] E. Perkins. Weak invariance principles for local time. Z. Wahrscheinlichkeitstheorie verw. Gebiete, 60:437–451, 1982. [39] E. Perkins. Private communication. 2017. [40] D. Revuz. Markov chains. North-Holland Publishing Co., Amsterdam, second edition, 1984. [41] L. A. Shepp. Recurrent random walks with arbitrarily large steps. Bull. Amer. Math. Soc., 70:540–542, 1964. [42] A. V. Skorokhod. Topologically recurrent Markov chains. Ergodic properties. Teor. Veroyatnost. i Prime- nen., 31:641–650, 1986. [43] Tomasz Szarek. Feller processes on nonlocally compact spaces. Ann. Probab., 34:1849–1863, 2006. [44] Kôhei Uchiyama. Green’s functions for random walks on ZN . Proc. London Math. Soc. (3), 77:215–240, 1998. [45] Kôhei Uchiyama. Wiener’s test for random walks with mean zero and finite variance. Ann. Probab., 26:368–376, 1998. [46] Vladislav Vysotsky. On the probability that integrated random walks stay positive. Stochastic Process. Appl., 120:1178–1193, 2010. [47] Vladislav Vysotsky. Positivity of integrated random walks. Ann. Inst. Henri PoincaréProbab. Stat., 50(1):195–213, 2014. [48] Vladislav Vysotsky. Stability of random walks with switch at zero. In progress, 2020. [49] Roland Zweimüller. Hopf’s ratio ergodic theorem by inducing. Colloq. Math., 101:289–292, 2004. 44 ALEKSANDAR MIJATOVIC´ AND VLADISLAV VYSOTSKY

Aleksandar Mijatovic,´ Department of Statistcis, University of Warwick & The Alan Turing Institute E-mail address: [email protected] Vladislav Vysotsky, University of Sussex, Pevensey 2 Building, Falmer Campus, Brighton BN1 9QH, United Kingdom and St. Petersburg Department of Steklov Mathematical Insti- tute, Fontanka 27, 191011 St. Petersburg, Russia E-mail address: [email protected]