Stone Duality for Markov Processes Dexter Kozen∗, Kim G. Larseny, Radu Mardarey and Prakash Panangadenz ∗Computer Science Department, Cornell University, Ithaca, New York, USA yDepartment of Computer Science, University of Aalborg, Aalborg, Denmark zSchool of Computer Science, McGill University, Montreal, Canada Abstract instances. This allows us to use the Rasiowa–Sikorski lemma [5] to establish our results without needing to We define Aumann algebras, an algebraic analog assume Lindenbaum’s lemma. of probabilistic modal logic. An Aumann algebra con- Our key results are: sists of a Boolean algebra with operators modeling • a description of a new class of algebras that cap- probabilistic transitions. We prove a Stone-type dual- tures, in algebraic form, the probabilistic modal ity theorem between countable Aumann algebras and logics used for continuous-state Markov pro- countably-generated continuous-space Markov pro- cesses; cesses. Our results subsume existing results on com- • a version of the duality for countable algebras pleteness of probabilistic modal logics for Markov and a certain class of countably-generated Markov processes. processes; and • a complete axiomatization where the infinitary 1. Introduction axiom schemes have only uncountably many in- stances. For Markov processes, the natural logic is a simple The duality is represented in the diagram below. modal logic with probability bounds on the modalities. Here SMP stands for countably based Stone Markov It is therefore tempting to understand this logic alge- processes and AA for countable Aumann algebras. braically in the same way that Boolean algebras cap- The formal definitions are given in xx3–4. ture propositional reasoning and the Jonsson-Tarski [1] results give duality for algebras arising from modal A logics. SMP AAop In this paper, we develop a Stone-type duality for M continuous-space probabilistic transitions systems and a certain kind of algebra that we have named Aumann algebras. These are Boolean algebras with operators 1.1. A Technical Summary that behave like probabilistic modalities. Recent papers [2–4] have established completeness theorems and The duality theorem proved in this paper has some finite model theorems for similar logics. novel features that distinguish it from many others that A comparison with related work appears in x7. We have appeared in the literature. note here that we go beyond existing completeness We have avoided the assumption that every consis- results [2–4] in a number of ways. The strong com- tent set of formulas can be expanded to a maximal con- pleteness theorems of Goldblatt [3] use a powerful sistent set axioms [6] by using the Rasiowa–Sikorski infinitary axiom scheme with an uncountable set of lemma (whose proof uses the Baire category theorem) instances and establish the results contingent on the in the following way. In going from the algebra to the assumption that every consistent set of formulas can be dual Markov process, we look at ultrafilters that do expanded to a maximally consistent set (Lindenbaum’s not respect the infinitary axioms of Aumann algebras. lemma). In our version we show that this assumption We call these bad ultrafilters. We show that these can be proved. The key point is that we use differ- form a meager set (in the standard topological sense) ent infinitary axioms that have only countably many and can be removed without affecting the transition probabilities that we are trying to define. Countability on a measurable space M = (M; Σ) is a countably is essential here. In order to show that we do not additive set function µ :Σ ! R+. A measure is a affect the algebra of clopen sets by doing this, we probability measure if in addition µ(M) = 1. We use introduce a distinguished base of clopen sets in the ∆(M; Σ) to denote the set of probability measures on definition of Markov process, which has to satisfy (M; Σ). some conditions. We show that this forms an Aumann A fundamental fact that we use is about extending algebra. We are able to go from a Markov process set functions to measures. This is Theorem 11.3 of [7]. to an Aumann algebra by using this distinguished It says that a finitely additive and countably subadditive base. Morphisms of Markov processes are required to function on a field of sets can be uniquely extended to preserve distinguished base elements backwards; that a measure on the σ-algebra generated by the field. −1 is, if f : M!N and A 2 AN , then f (A) 2 AM. We can view ∆(M; Σ) as a measurable space by Thus we get Boolean algebra homomorphisms in the considering the σ-algebra generated by the sets fµ 2 dual for free. ∆(M; Σ) j µ(S) ≥ rg for S 2 Σ and r 2 [0; 1]. Removing bad points has the effect of destroying This is the least σ-algebra on ∆(M; Σ) such that all compactness of the resulting topological space. We maps µ 7! µ(S) : ∆(M; Σ) ! [0; 1] for S 2 Σ are introduce a new concept called saturation that takes measurable, where the real interval [0; 1] is endowed the place of compactness. The idea is that a saturated with the σ-algebra generated by all rational intervals. model has all the good ultrafilters. The Stone dual of an Every topological space has a natural σ-algebra Aumann algebra is saturated, because it is constructed associated with it, namely the one generated by the that way. However, it is possible to have a Markov open sets. This is called the Borel algebra of the space, process that is unsaturated but still represents the same and the measurable sets are called Borel sets. algebra. For example, we removed bad points and Recall that a topological space is said to be sep- could, in principle, remove a few more; as long as the arable if it contains a countable dense subset and remaining points are still dense, we have not changed second countable if its topology has a countable base. the algebra. One can saturate a model by a process Second countability implies separability, but not vice akin to compactification. We explicitly describe how versa in general; however, the two concepts coincide to do this below. for metric spaces. A Polish space is the topological space underlying a complete separable metric space. 2. Background An analytic space is a continuous image of a Polish space in another Polish space. More precisely, if X In this section we present background from measure and Y are Polish spaces and f : X ! Y is continuous, theory and topology. For proofs we refer the reader to then the image f(X) is an analytic space. Remarkably, [7] or [8]. We do not discuss the Stone duality theorem; one does not get a broader class by allowing f to be this is discussed elsewhere in this volume. merely measurable instead of continuous and by taking We use Q0 to denote the set Q \ [0; 1]. the image of a Borel subset of X instead of X. Measurable Spaces and Measures Analytic spaces enjoy remarkable properties that Let M be an arbitrary nonempty set. We assume that were crucial in proving the logical characterization of the basic notions like field of sets, σ-algebra, measur- bisimulation [9, 10]. We note that the completeness able set and measurable function are know. Similarly theorems proved in [2, 11, 12] were established for with topology, open and closed set and continuous Markov processes defined on analytic spaces. function and the Borel algebra of a topology. We The Baire Category Theorem use M ! N to denote the family of measurable The Baire category theorem is a topological result functionsJ fromK (M; Σ) to (N; Ω). with important applications in logic. It can be used to If Ω ⊆ 2M , the σ-algebra generated by Ω, denoted prove the Rasiowa–Sikorski lemma [5] that is central Ωσ, is the smallest σ-algebra containing Ω. for our paper. Let R+ = fr 2 R j r ≥ 0g. A nonnegative real- A subset D of a topological space X is dense if its valued function µ defined on a collection of sets (a set closure D is all of X. Equivalently, a dense set is one function) is finitely additive if µ(A[B) = µ(A)+µ(B) intersecting every nonempty open set. A set N ⊆ X whenever A \ B = ?. We say that µ is countably is nowhere dense if every nonempty open set contains S P subadditive if µ( i Ai) ≤ i µ(Ai) for a countable a nonempty open subset disjoint from N. A set is said family of measurable sets, and we say that µ is count- to be of the first category or meager if it is a countable P ably additive if µ([iAi) = i µ(Ai) for a countable union of nowhere dense sets. A basic fact that we pairwise-disjoint family of measurable sets. A measure use is that the boundary of an open set is nowhere dense. A Baire space is one in which the intersection domain. Two Markov processes M1; M2 are said to of countably many dense open sets is dense. For us, be bisimilar if there is a third Markov process M and the relevant fact is: every compact Hausdorff space is a span of zig-zags fi : M!Mi, i = 1; 2. Two states Baire. mi 2 supp(Mi), i = 1; 2, are said to be bisimilar if there exist a span of zig-zags f : M!M , i = 1; 2 Definition 1. Let B be a Boolean algebra and let T ⊆ i i and m 2 supp(M) such that m = f (m), i = 1; 2.