Markov Process Duality Jan M. Swart Luminy, October 23 and 24, 2013 Jan M. Swart Markov Process Duality Markov Chains S finite set. RS space of functions f : S R. ! S For probability kernel P = (P(x; y))x;y2S and f R define left and right multiplication as 2 X X Pf (x) := P(x; y)f (y) and fP(x) := f (y)P(y; x): y y (I do not distinguish row and column vectors.) Def Chain X = (Xk )k≥0 of S-valued r.v.'s is Markov chain with transition kernel P and state space S if S E f (Xk+1) (X0;:::; Xk ) = Pf (Xk ) a:s: (f R ) 2 P (X0;:::; Xk ) = (x0;:::; xk ) , = P[X0 = x0]P(x0; x1) P(xk−1; xk ): ··· µ µ µ Write P ; E for process with initial law µ = P [X0 ]. x δx x 2 · P := P with δx (y) := 1fx=yg. E similar. Jan M. Swart Markov Process Duality Markov Chains Set k µ k x µk := µP (x) = P [Xk = x] and fk := P f (x) = E [f (Xk )]: Then the forward and backward equations read µk+1 = µk P and fk+1 = Pfk : In particular π invariant law iff πP = π. Function h harmonic iff Ph = h h(Xk ) martingale. , Jan M. Swart Markov Process Duality Random mapping representation (Zk )k≥1 i.i.d. with common law ν, take values in (E; ). φ : S E S measurable E × ! P(x; y) = P[φ(x; Z1) = y]: Random mapping representation (E; ; ν; φ) always exists, highly non-unique. E X0 independent of (Zk )k≥1, then Xk := φ(Xk−1; Zk )(k 1) ≥ defines Markov chain with transition kernel P. Example if rand < 0:3 X = X + 1 else X = X 1 end − Jan M. Swart Markov Process Duality Continuous time Markov semigroup (Pt )t≥0 satisfies Ps Pt = Ps+t , limt#0 Pt = P0 = 1. Given by 1 X 1 P = e tG = tnG n; t n! n=0 where generator G satisfies G(x; y) 0 for x = y and P G(x; y) = 0. ≥ 6 y Def Process X = (Xt )t≥0 is Markov with semigroup (Pt )t≥0 and generator G if S E f (Xu) (Xs )0≤s≤t = Pu−t f (Xt ) a:s: (f R ): 2 2 P"(x; y) = 1fx=yg + "G(x; y) + O(" ) with G(x; y) jump rate. µ x µt := µPt (x) = P [Xt = x] and ft := Pt f (x) = E [f (Xt )] satisfy the forward and backward equations @ @ @t µt = µt G and @t ft = Gft : @ Also @t Pt = GPt = Pt G. Jan M. Swart Markov Process Duality Random mapping representation Write X Gf (x) = rm f (m(x)) f (x) − m2M with collection of maps m : S S and (rm)m2M nonnegative rates. M ! Let ∆ be a Poisson point subset of R with local intensity rmdt, and set M × ∆s;u := (m; t): s < t u f ≤ g =: (m1; tt );:::; (mn; tn) ; t1 < < tn: f g ··· Then Φs;u := mn m1 satisfy Φt;u Φs;t = Φs;u: ◦ · · · ◦ ◦ If X0 independent of ∆, then Xt := Φ0;t (X0)(t 0) ≥ Markov process with generator G. Jan M. Swart Markov Process Duality y I Pt = Qt , y I G = H . Duality X =(Xt )t≥0 Markov with state space S, generator G, semigroup (Pt )t≥0. Y =(Yt )t≥0 Markov with state space R, generator H, semigroup (Qt )t≥0. Def X and Y dual with duality function : S R R iff × ! x y E [ (Xt ; y)] = E [ (x; Yt )] (t 0): ≥ Implies more generally, if X and Y independent, then E[ (Xs ; Yt−s )] does not depend on s [0; t]: 2 Equivalent formulations (with Ay(x; y) := A(y; x)): X 0 0 X 0 0 I Pt (x; x ) (x ; y) = (x; y )Qt (y; y ), x0 y 0 Jan M. Swart Markov Process Duality y I G = H . Duality X =(Xt )t≥0 Markov with state space S, generator G, semigroup (Pt )t≥0. Y =(Yt )t≥0 Markov with state space R, generator H, semigroup (Qt )t≥0. Def X and Y dual with duality function : S R R iff × ! x y E [ (Xt ; y)] = E [ (x; Yt )] (t 0): ≥ Implies more generally, if X and Y independent, then E[ (Xs ; Yt−s )] does not depend on s [0; t]: 2 Equivalent formulations (with Ay(x; y) := A(y; x)): X 0 0 X 0 0 I Pt (x; x ) (x ; y) = (x; y )Qt (y; y ), x0 y 0 y I Pt = Qt , Jan M. Swart Markov Process Duality Duality X =(Xt )t≥0 Markov with state space S, generator G, semigroup (Pt )t≥0. Y =(Yt )t≥0 Markov with state space R, generator H, semigroup (Qt )t≥0. Def X and Y dual with duality function : S R R iff × ! x y E [ (Xt ; y)] = E [ (x; Yt )] (t 0): ≥ Implies more generally, if X and Y independent, then E[ (Xs ; Yt−s )] does not depend on s [0; t]: 2 Equivalent formulations (with Ay(x; y) := A(y; x)): X 0 0 X 0 0 I Pt (x; x ) (x ; y) = (x; y )Qt (y; y ), x0 y 0 y I Pt = Qt , y I G = H . Jan M. Swart Markov Process Duality Duality If the matrix is invertible, then y −1 Pt = Qt ; which relates the backward evolution of X to the forward evolution of Y . In general π invariant for Y π harmonic for X : ) y Proof Pt π = Qt π = (πQt ) = π. Similar: h harmonic for Y h invariant under right-multiplication with ) Pt (in particular, if h is a probabiity distribution, then it is an invariant law). Jan M. Swart Markov Process Duality Pathwise Duality Def Maps m : S S andm ^ : R R are dual w.r.t. if ! ! m(x); y = x; m^ (y) x; y: 8 Let X Gf (x) = rm f (m(x)) f (x) ; − m2M X Hf (y) = rm f (m ^ (y)) f (y) : − m2M Lemma For each t > 0, X and Y can be coupled such that (Xu)0≤u≤s and (Yu)0≤u≤t−s independent and Xs−; Yt−s a.s. does not depend on s [0; t]: 2 Jan M. Swart Markov Process Duality Pathwise Duality Proof Set ∆s−;u− := (m; t): s t < u f ≤ g =: (m1; tt );:::; (mn; tn) ; t1 < < tn: f g ··· Then Φ^ s−;u− :=m ^ 1 m^ n dual to Φs−;u− := mn m1: ◦ · · · ◦ ◦ · · · ◦ ^ For fixed t > 0, observe that Ys := Φ(t−s)−;t−(Y0)(s 0) Markov with generator H. Then ≥ Xs−; Yt−s = Φ0−;s−(X0); Φ^ s−;t−(Y0) = Φ0−;t−(X0); Y0 does not depend on s [0; t]. 2 Jan M. Swart Markov Process Duality A formal dual (S) := set of all subsets of S. For m : S S define mP −1 : (S) (S) by m−1(A) := x : m(!x) A inverse image. P !P f 2 g −1 Observe m dual to m w.r.t. (x; A) := 1fx2Ag: −1 (m(x); A) = 1fm(x)2Ag = 1fx2m−1(A)g = (x; m (A)): Consequence X dual to set-valued process with generator X X −1 f (A) = rm f (m (A)) f (A) : G − m2M Question Does the large ( (S) = 2jSj) space (Λ) contain any useful subspaces that are invariantjP underj the dynamicsP of ? X Jan M. Swart Markov Process Duality I x y and y x implies x = y, ≤ ≤ I x y z implies x z. ≤ ≤ ≤ I x y or y x for all x; y S; x = y. ≤ ≤ 2 6 −1 I A increasing m (A) increasing, ) −1 I A decreasing m (A) decreasing. ) Partial Order Recall that a partial order over S is a relation s.t. ≤ I x x, ≤ A partial order is a total order if Let S; S 0 be partially ordered. Then m : S S 0 is monotone if ! x y m(x) m(y): ≤ ) ≤ A set A S is increasing (decreasing) if 1A : S 0; 1 monotone (resp. ⊂ ! f g 1 1A monotone). − Observe m : S S monotone iff ! Jan M. Swart Markov Process Duality I x y z implies x z. ≤ ≤ ≤ I x y or y x for all x; y S; x = y. ≤ ≤ 2 6 −1 I A increasing m (A) increasing, ) −1 I A decreasing m (A) decreasing. ) Partial Order Recall that a partial order over S is a relation s.t. ≤ I x x, ≤ I x y and y x implies x = y, ≤ ≤ A partial order is a total order if Let S; S 0 be partially ordered. Then m : S S 0 is monotone if ! x y m(x) m(y): ≤ ) ≤ A set A S is increasing (decreasing) if 1A : S 0; 1 monotone (resp. ⊂ ! f g 1 1A monotone). − Observe m : S S monotone iff ! Jan M. Swart Markov Process Duality I x y or y x for all x; y S; x = y. ≤ ≤ 2 6 −1 I A increasing m (A) increasing, ) −1 I A decreasing m (A) decreasing. ) Partial Order Recall that a partial order over S is a relation s.t. ≤ I x x, ≤ I x y and y x implies x = y, ≤ ≤ I x y z implies x z. ≤ ≤ ≤ A partial order is a total order if Let S; S 0 be partially ordered. Then m : S S 0 is monotone if ! x y m(x) m(y): ≤ ) ≤ A set A S is increasing (decreasing) if 1A : S 0; 1 monotone (resp. ⊂ ! f g 1 1A monotone). − Observe m : S S monotone iff ! Jan M.