<<

arXiv:1704.04872v1 [cs.LO] 17 Apr 2017 ahde o ee otestates the to refer not does path and rbes(..[] 2,[].Awl-nw ehdi h one the is method well-known A [3]). [2], [1], (e.g. problems x srahbeb h path the by reachable is eoi lyr h nwrt h ecaiiypolmis problem reachability if the matter to no yes: answer again The player. demonic hcigpolm—sstefloig De hr xs a state exist initial there the “Does from following: path the problems—asks liven of checking example typical problem—a reachability The event. an yaglcaddmncpaes h gr 2 lutae an illustrates state (2) figure the The At players. example. demonic and angelic by ytma ntefiue() Here (1). figure the in as transition a system given are we that suppose ple, o,weetenx tt scoe oaodago state. good a avoid a to as modeled chosen be is can state system a next Such the where too, tt oeetal ec odstate. good contain often a systems real-world reach However, eventually to state an in controlled gelic is system the that fcagbacsmlto suidpeiul yteautho combinat the algebra suitable by previously a corecursive (studied in simulation answer coalgebraic an of find We ones). and probabilistic settings) nondeterministic for namely axiomatizat liveness, (in methods categorical for methods a proof proof seek well-known categorical we towards Concretely, coalgebras. step a least focus make have we therein Here studied Some coalgebras. methods of on proof theory the most in role analogously, central a play points” oehn go”eetal cus hl aeymastha occurs. means never safety “bad” while anything occurs, eventually “good” mean something Liveness specifications. c fixed-point more alternating of plex verification underlines that problem prototypical Backgrounds A. 2 ueosmtosaekonfrsc ieeschecking liveness such for known are methods Numerous nteeapeaoe eassumed we above, example the In Functions: Ranking 1) Abstract eicto flvns,mc ieta fsft,i a is safety, of that like much liveness, of Verification h nei lyrcnfrereaching force can player angelic the , accepting greatest → xdpitpoete vrdnmclssesmdldas modeled systems dynamical over properties fixed-point anr ecncos h next the choose can we manner: x 4 — et optrSine h nvriyo oy,Japan Tokyo, of University The Science, Computer Dept. → xdpitpoete iesft n bisimilarity. and safety like properties fixed-point ia coalgebras Final tt htrpeet good a represents that state x 5 → { sacasfirfr(non-)well-foundedness. for classifier a as rbntui qnighy urabenatsuki, x .I I. ask Urabe Natsuki 3 → x x 3 → NTRODUCTION 2 aeoia ieesChecking Liveness Categorical x or ∗ x sa exam- an As h etmv scoe ythe by chosen is move next the 1 x SSRsac Fellow Research JSPS 0 ). 0 x to → s“aeoia raetfixed greatest “categorical as 4 x scoe stescesrof successor the as chosen is x x yCrcrieAlgebras Corecursive by akn supermartingales ranking 4 1 2 x ”Tease syes: is answer The ?” an- and → 1 ∗ w-lyrgame two-player x is } 3 x @is.s.u-tokyo.ac.jp aaiHara Masaki → 5 x demonic . x x / 1  ÈÉ x x / ✉  0 1  ÈÉ x ✉  0 1 akn functions ranking o (by 1 o oeta the that Note . x x /  ÈÉ x x / → O 2 3  ÈÉ  ÈÉ O 2 3 o branching, o x 3 x x /  ÈÉ  ÈÉ s and rs) x x / played O 4 5  ÈÉ  ÈÉ o of ion → O 4 5 that s what om- ess ion (2) (1) (in x x ed 1 1 t eoi lyr ehave we player, demonic onstenme fsesfrom steps of number the bounds state eadeso h eoi lyrscoc,teaglcplay angelic the path choice, a player’s construct can demonic the of regardless farnigfunction ranking a of b o xml,i h w-lyrgm n() h akn func- ranking the (2), in game two-player tion in the impossible in is example, chain descending For infinite an and reached; is nue that ensures h tt space state the h olwn odtos i o ahnnacpigstate non-accepting each for (i) conditions: following the sn a using h nei lyr hr xssascesrstate successor a exists there player, angelic the nacpigsae h elfuddesof have well-foundedness we The state. accepting an ucini yial enda a as defined typically is function h ubro tp from steps of number the gales namely systems, probabilistic for known ahta osntvstayacpigsaea l (namely all at state accepting any visit not does → that path a 1 gr 3.I h lotsr ecaiiypolm ewant state we accepting problem, the reachability if almost-sure know the to In (3). figure osdrthe consider akn uemrigl safunction a is supermartingale ranking b rbblsi ytm.Atpcleapeis example typical the A systems. for also probabilistic problems checking liveness consider ε ahstate each h tt space state the elnmes htstse h olwn condition. following the satisfies that numbers, real Here ytmmksatasto from transition a makes system ∀ ( ′ ntePSi 3,tease sys huhteeexists there though yes, is answer the (3), in PTS the In . x fe rniin h xsec farnigsupermarting ranking a of existence The transition. a after x oinaaoost hto akn ucini also is function ranking of that to analogous notion A )RnigSupermartingales: Ranking 2) uhthat such x ) ∈ lotsr ecaiiyproblem reachability almost-sure 0 b ≥ y Prob( X 5,[] o xdpstv real positive fixed a For [6]. [5], → [ = ′ b akn function ranking ( \ of b x x x ( x Acc 0 ′ x 0 )+1 x x ∈ y i · · · → ainlIsiueo nomtc,Japan Informatics, of Institute National 7→ ) b rbblsi rniinsystem transition probabilistic ti nw that known is It . 1 → b . ′ X b > ( srahbefrom reachable is X x X 5 n i)frec o-cetn state non-accepting each for (ii) and ; h xetdvleof value expected the , ) ′ x x , ( ( ′ oteset the to u hsocr ihprobability with occurs this but , x oteset the to ) x < 1 x ) i eoe h rbblt ihwihthe which with probability the denotes +1 7→ = ≥ ∞ b [email protected] ) x 4.Fratopae ae ranking a game, two-player a For [4]. 0 cioHasuo Ichiro  b uhthat such 0 o each for x , mle htteepce au of value expected the that implies ( P → y x 2 ) [0 x N x 7→ ′ ≥ oa cetn tt sfinite is state accepting an to x , ∈ 1 ∞ ∞ 1 X x ··· · →· x b 4 soundness 0 ( x srahdwt probability with reached is ] Prob( x , = i to y nutvl,tevalue the Intuitively, . n can One fetne non-negative extended of b ′ oa cetn state. accepting an to eoea cetn state accepting an before 1 + ) 3 e us let : ( N x x b 7→ hteetal reaches eventually that b ′ ) > ε ′ hsmasta for that means This . {∞} ∪ ′ akn supermartin- ranking x : erae ya least at by decreases < 1 b → X o ahsuccessor each for x , : 0 N ∞ od:existence holds: x PS si the in as (PTS) X n( an , → 4 ′ scuilhere: crucial is ) htsatisfies that , 7→ → mle that, implies · 2 1 [0 x b ′ 3 ε , ′ ; N  ÈÉ ( ∞ -additive) ✉ x , O O uhthat such 0 x x 2 1 ∞ x y ). ′ 0 ] 5 1 ) from , from ,  fthe of 7→ b x + ( ale (3) N x of 2] er ε ) . functor F c : X → F X represents ( )Σ × {0, 1} deterministic automaton coalgebra-algebra homomorphism (|c|)r (see the diagram). P2( ) × {0, 1} two-player game Corecursive algebras have been previously used to describe D( ) × {0, 1} probabilistic transition system (PTS) general structured corecursion [13] (see also Rem. II.17). Our Fig. 1. Coalgebraic representations of transition systems. Here P and D use of them in this paper seems novel: r being corecursive denote the powerset and the distribution functors respectively (Def. II.15). means that the function Φc,r : f 7→ r◦Ff ◦c has a unique fixed point; in particular its least and greatest fixed points coincide1; (specifically it is no bigger than b′(x)/ε). From this it easily we find this feature of corecursive algebras suited for their use follows that an accepting state is visited almost surely. as categorical “classifiers” for (non-)well-foundedness. 3) Coalgebras and Algebras: This paper aims to under- 2) Modalities and Least Fixed-Point Properties: Liveness stand, in the categorical terms of (co)algebra, essences of properties such as reachability and termination are all instances liveness checking methods like ranking functions and ranking of least fixed-point properties: once a proper modality ♥σ is supermartingales. Coalgebras are commonly used for mod- fixed, the property in question is described by a least fixed- eling state-based dynamics in the categorical language (see point formula µu. ♥σu. The way we categorically formulate e.g. [7], [8]). Formally, for an endofunctor F over a category these constructs, as shown below, is nowadays standard (see C, an F -coalgebra is an arrow c of the type c : X → FX. e.g. [17], [18]). As the base category C we use Sets in this We can regard X as a state space, F as a specification paper (although extensions e.g. to Meas would not be hard). of the branching type, and c : X → FX as a transition • We fix a domain Ω ∈ C of truth values (e.g. Ω= {0, 1}), function. By changing the functor F we can represent various and a property over X ∈ C is an arrow u: X → Ω. kinds of transition types (see Fig. 1). It is also known that, • A (state-based, dynamical) system is a coalgebra c: X → using coalgebras, we can generalize various automata-theoretic FX for a suitable functor F : C → C. 2 notions and techniques (such as behavioral equivalence [9], • A modality ♥σ is interpreted as an F -algebra σ : F Ω → bisimulation [10] and simulation [11]) to various systems (e.g. Ω over Ω (see Example III.4 and Prop. IV.2 for examples). nondeterministic, probabilistic, and weighted ones). • Assuming some syntax is given, we should be able to A dual notion, i.e. an arrow of type a : FX → X, is known derive the interpretation ♥σϕ c of a modal formula from as an F -algebra. In this paper, it is used to capture properties ϕ c. In the current (purelyJ semantical)K framework this (or predicates) over a system represented as a coalgebra. JgoesK as follows. Given a property u: X → Ω, we define the property Φc,σ(u): X → Ω by the composite B. Contributions c F u σ We contribute a categorical axiomatization of “ranking Φc,σ(u) = X → FX → F Ω → Ω . functions” that is behind the well-known methods that we have • Assuming a suitable order structure ⊑ on Ω and ad- corecursive algebras Ω sketched. It combines: as value domains ditional monotonicity requirements, the correspondence (that are, like N , suited to detect well-foundedness) and lax ∞ Φ : ΩX → ΩX has the least fixed point. It is denoted homomorphisms (like in coalgebraic simulations [11], [12]). c,σ by µσ : X → Ω; intuitively it is the interpretation Based on the axiomatization we develop a general theory; our c µu.J♥σKu c of the formula µu. ♥σu in the system c. main result is soundness, i.e. that existence of a categorical J K ranking function indeed witnesses liveness (identified with Concrete examples are in §III-B. Another standard categorical modeling of a modality (see e.g. [19]) is by a predicate lifting, a least fixed-point property). We also exploit our general X F X theory and derive two new notions of “ranking functions” as i.e. a natural transformation σX : Ω ⇒ Ω . It corresponds instances. The two concrete definitions are new to the best of to our modeling via the Yoneda lemma; see e.g. [17]. F JµσKc our knowledge. 3) Ranking Functions, Categorically: F X / F Ω (4) Our modeling is summarized on the right: O We shall now briefly sketch our general theory, illustrating c =µ σ key notions and the backgrounds from which we derive them. the liveness property µσ c in question is  X / Ω J K JµσKc 1) Corecursive Algebras for (Non-)Well-Foundedness: In the least arrow (with respect to the order on ) that makes the square commute (note the subscript ). the (conventional) definition of a ranking function b : X → Ω =µ The liveness checking problem is then formulated as fol- N∞, well-foundedness of N = N∞ \ {∞} plays an impor- tant role as it ensures that no path can continue infinitely lows: given an arrow h: X → Ω, we would like to decide if (without hitting an accepting state). Similarly, for a ranking 1Examples abound in computer science—especially in — supermartingale, it is crucial that [0, ∞) = [0, ∞]\{∞} has no where similar coincidences play important roles. They include: limit-colimit infinite sequence that decreases everywhere at least by ε> 0. coincidence [14] and initial algebra-final coalgebra coincidence [15], [16]. In unifying the two notions, we need to categorically capture 2 We use the same functor F for coalgebras (systems) and algebras (modalities). This characterization is used in [17], [18] and also found in well-foundedness. many coalgebraic modal logic papers (e.g. [19]). However, for some examples, F (|c|)r Our answer comprises suitable use of core- F X ❴❴ / F R it comes more natural to use functors F and G together with a natural O transformation α F ⇒ G, and to model a system and a modality as cursive algebras [13]. An F -algebra r : c = r : (|c|)r  c : X → FX and σ : GΩ → Ω respectively. This modeling induces F R → R is said to be corecursive if from / αΩ σ X ❴❴❴ R our current one as σ and α together induce an F -algebra F Ω → GΩ → Ω an arbitrary coalgebra c : X → FX there exists a unique (cf. §L).

2 h ⊑Ω µσ c holds. Here ⊑Ω denotes the pointwise extension the ordinary order ≤ over R to [0, ∞] by regarding ∞ as the of theJ orderK on Ω. For example, let’s say we want to check greatest element. We write N∞ for N ∪ {∞}. For a function the assertion that a specific state x0 ∈ X satisfies the liveness ϕ : X → [0, 1], its support {x ∈ X | ϕ(x) > 0} is denoted property µσ c. In this case we would define the above by supp(ϕ). “assertion”J h:KX → Ω, where Ω= {0 ⊑Ω 1}, by: h(x0)=1 and h(x)=0 for all x 6= x0. A. Two-player Games and Ranking Functions Our categorical framework F X / F R / F Ω Our two-player games are played by an angelic player max of ranking function-based ver- O F b F q c r ⊑ σ ⊑ and a demonic player min. ification goes as follows. b  q  X / R / Ω (5) • We fix a ranking do- ⊒ ; Definition II.1 (two-player game). A (two-player) game struc- h main—the value domain ture is a triple G = (Xmax,Xmin, τ) of a Xmax of states for ranking functions—to be an algebra r : F R → R of the player max, a set Xmin of states of the player min, and together with a lax homomorphism q : R → Ω (from r a transition relation τ ⊆ Xmax × Xmin ∪ Xmin × Xmax. to σ, in the right square in (5)). The latter q identifies A strategy of the player max is a partial function α : ∗ r : F R → R as a “refinement” of the modality σ : F Ω → Xmax×(Xmin×Xmax) ⇀Xmin such that for each x0y1x1 ... ∗ Ω. A crucial requirement is that r is corecursive, making ynxn ∈ Xmax × (Xmin × Xmax) , if α(x0y1x1 ...ynxn) is it suited for detecting well-foundedness. defined then (xn, α(x0y1x1 ...ynxn)) ∈ τ. A strategy of the ∗ player min is a partial function β : Xmax ×(Xmin ×Xmax) × • A (categorical) ranking arrow for a coalgebra c: X → X ⇀ X such that for each x y x ...y x y ∈ FX is then defined to be a lax homomorphism b: X → min max 0 1 1 n−1 n−1 n X ×(X ×X )∗×X , if β(x y x ...y x y ) R, in the sense shown in the left square in (5). max min max min 0 1 1 n−1 n−1 n is defined then (yn,β(x0y1x1 ...yn−1xn−1yn)) ∈ τ. • Our soundness theorem says: given an assertion h: X → For x ∈ Xmax and a pair of strategies α and β of the players Ω, in order to establish h ⊑ µσ c, it suffices to find a max and min, respectively, the run induced by α and β from ranking arrow b: X → R suchJ thatK h ⊑ q ◦ b (see (5)) . α,β,x x is a possibly infinite sequence ρ = x0y1x1y2x2 ... that This way the problem of verifying a least fixed-point prop- is an element of the set erty is reduced to finding a witness b. Note that the requirement ∗ on the ranking arrow b—namely b ⊑ r ◦ Fb ◦ c—is local (it (Xmax × (Xmin × Xmax) × {⊥max}) ∗ only involves one-step transitions) and hence easy to check. ∪ (Xmax × (Xmin × Xmax) × Xmin × {⊥min}) ω Our main technical contribution is a proof for soundness. ∪ (Xmax × Xmin) There we argue in terms of inequalities between arrows— and is inductively defined as follows: for n =0, x = x; and much like in [11], [12]—relying on fundamental order- 0 for n> 0, theoretic results on fixed points (Knaster–Tarski and Cousot–

Cousot, see §II-D). Corecursiveness of r is crucial there. ⊥max (α(x0y1x1 ...yn−1xn−1) is undefined) yn = 4) Concrete Examples: Ranking functions for two-player α(x0y1x1 ...yn−1xn−1) (otherwise), and  games (§I-A) are easily seen to be an instance of our categor- ⊥min (β(x0y1x1 ...xn−1yn) is undefined) xn = ical notion, for suitable F, Ω and R. β(x0y1x1 ...xn−1yn) (otherwise) . Our second example in §I-A—additive ranking supermartin-  gales in the probabilistic setting—is itself not an example, The symbol ⊥max (resp. ⊥min) represents the end of the run however. Analyzing its reason we are led to a few variations of at max’s (resp. min’s) turn: it means the player got stuck. the definition, among which some seem new. We discuss these Once an initial state x and strategies α and β of the players variations: their relationship, advantages and disadvantages. max and min are fixed, a run ρα,β,x is determined. There are different ways to determine the “winner” of a run, including: C. Organization of this Paper max wins if min gets stuck; max wins if he does not get In §II we introduce preliminaries on: our running examples stuck; max wins if some specified states are visited infinitely (two-player games and PTSs); liveness checking methods for many times (the Buchi¨ condition), etc. In this paper where them; (co)algebras; and least and greatest fixed points. Our we focus on liveness, we choose the following (rather simple) main contribution is in §III, where our categorical develop- winning condition: the player max wins if an accepting state is ments are accompanied (for illustration) by concrete examples reached or the player min gets stuck. Studies of more complex from two-player games. The entailments of our general frame- conditions (like the B¨uchi condition) are left as future work. work in the probabilistic setting are described in §IV. Finally in §V we conclude. Definition II.2 (reaching set). Let G = (Xmax,Xmin, τ) be a Some details and proofs are deferred to the appendices. two-player game structure. We fix a set Acc ⊆ Xmax of ac- α,β,x cepting states. A run ρ = x0y1x1 ... on (Xmax,Xmin, τ) II. PRELIMINARIES is winning with respect to Acc for the player max if

In this paper [0, ∞) and [0, ∞] denote the sets {a ∈ R | • xn ∈ Acc for some n; or α,β,x a ≥ 0} and {a ∈ R | a ≥ 0}∪{∞} respectively. We extend • ρ is a finite sequence whose last letter is ⊥min .

3 We define the reaching set ReachG,Acc ⊆ Xmax by: Then it is a positional strategy such that for each strategy β of min, the run ρα,β,x is winning wrt. Acc for max. ∃α : strategy of max.   ∀β : strategy of min. ReachG,Acc = x ∈ X  . (6) B. Probabilistic Transition Systems and Ranking Supermartin- α,β,x ρ is winning wrt. Acc gales   Definition II.8 (PTS). A probabilistic transition system (PTS) Example II.3. We define a game struc- ✉x2 y3 I ✰U E is a pair M = (X, τ) of a set X and a transition function τ : ✓ ✰ x3 ☞ ture G = (Xmax,Xmin, τ) by Xmax = ✰  ÈÉ ☞  ÈÉ x4 ✓✓ ✰ W G X → DX. Here DX = {d : X → [0, 1] | x∈X d(x)=1} ✓ 0 1 ✴ {x0, x1, x2, x3, x4}, Xmin = {y0,y1,y2, U y I iy ✴✴ ✎✎ is the set of probability distributions over X. ✰ ✎ 2 and ✰ ✓ O y P y3} τ = {(x0,y0), (x0,y1), (x1,y1), ✰✰ ✓✓  ÈÉ ✓ x0 )  ÈÉ x1 Definition II.9 (reachability probability). Let M = (X, τ) be (x1,y2), (x3,y3), (y0, x2), (y1, x1), (y1, x2), a PTS. We fix a set Acc ⊆ X of accepting states. For each (y2, x3), (y2, x4)}. Let Acc = {x2}. The situation is shown on x ∈ X and n ∈ N, we define a value f (x) ∈ [0, 1] by: the right. Then the reaching set is ReachG,Acc = {x0, x2, x3}. n

Ranking functions. Suppose that we are given a game fn(x)= structure G = (Xmax,Xmin, τ) and a set Acc ⊆ Xmax of k−1 k ≤ n − 1,x0,...,xk ∈ X, accepting states, and want to prove that a state x is included in τ(x )(x ) x0 = x,xi ∈/ Acc (∀i ∈ [0,k − 1]), .  i i+1  ReachG,Acc.A ranking function is a standard proof method in i=0 and xk ∈ Acc X Y  such a setting. There are variations in the definition of ranking Note that if x ∈ Acc then f (x) = 1. As the sequence function [4], [20], [21]: in this paper we use the following.  n  is increasing for each , we can define a fn(x) n∈N x ∈ X Definition II.4 (ranking function). Let G = (Xmax,Xmin, τ) function f : X → [0, 1] by: be a game structure and Acc ⊆ Xmax. We fix an ordinal z  and let Ord≤z = {n | n ≤ z} be the set of ordinals smaller f(x) = lim fn(x) . n→∞ than or equal to z. A function b : Xmax → Ord≤z is called a ranking function (for G and Acc) if it satisfies The function f is called the reachability probability function with respect to M and Acc, and is denoted by ReachM,Acc. ′ miny : (x,y)∈τ supx′ : (y,x′)∈τ b(x ) +1 ≤ b(x) Here the value fn(x) ∈ [0, 1] is the probability with which ′ ′ an accepting state is reached within n steps from x. for each x ∈ Xmax\Acc, where b(x ) + 1b = min{b(x )+1, z} denotes addition truncated at z. Example II.10. We define a PTS M = ✉ x2 O c 1 b (X, τ) by X = {x0, x1, x2, x3}, and 1 1 1 The following well-known theorem states soundness, i.e. 1 1 2 2 2 τ(x0) = [x1 7→ 2 , x3 7→ 2 ], τ(x1) =  ÈÉ  ÈÉ that a ranking function witnesses reachability. 1 1 ; Zx1 x3 D c [x1 7→ , x2 7→ ], τ(x2) = [x2 7→ 1], ✻✻ 2 2 1 ✻ ✟✟1 2 ✻ ✟ 2 Theorem II.5 (soundness, see e.g. [4]). Let z be an ordinal, and τ(x3) = [x3 7→ 1]. Let Acc =  ÈÉ ✟ and let b : X → Ord≤z be a ranking function for G and Acc. {x2}. The situation is as shown on the x0 1 Then b(x) < z (i.e. b(x) 6= z) implies x ∈ ReachG,Acc. right. Then ReachM,Acc : X → [0, 1] assigns 2 to x0, 1 to x1 and x , and 0 to x . Example II.6. For the game in Example II.3, let z = ω and 2 3 Let us consider the almost-sure reachability problem for define a function b : X → Ord≤z by b(x0)=1, b(x2)=0 PTS. Given a PTS M = (X, τ), a set Acc ⊆ X of accepting and b(x1)= b(x3)= b(x4)= ω. Then b is a ranking function. states and an (initial) state x ∈ X, we want to prove that Hence by Thm. II.5, we have x0 ∈ ReachG,Acc. ReachM,Acc(x)=1. For this problem, a ranking function-like Completeness (the converse of Thm. II.5) does not hold. A notion called ranking supermartingale [5] is known. There are counterexample is given later in Example III.17. several variations in its definition. We follow the definition ∗ in [6]; a variation can be found in [23]. Remark II.7. A strategy α : Xmax × (Xmin × Xmax) ⇀ Xmin is said to be positional if its outcome depends only Definition II.11 (ε-additive ranking supermartingale). Let ′ on the last state of the input, i.e. xn = xn′ implies M = (X, τ) be a PTS and Acc ⊆ X be the set of accepting ′ ′ ′ ′ ′ α(x0y1 ...ynxn) = α(x0y1 ...yn′ xn′ ) . It is known that a states. Let ε> 0 be a real number. A function b : X → [0, ∞] positional strategy suffices as long as we consider reach- is an ε-additive ranking supermartingale (for M and Acc) if ing sets, i.e. the set ReachG,Acc in Def. II.2 is unchanged ′ ′ ′ ′ if we replace “∃α : strategy of max” in (6) with “∃α : x′∈supp(τ(x)) τ(x)(x ) · b (x ) + ε ≤ b (x) positional strategy of max” (see e.g. [22]). holds for P each x ∈ X \ Acc.  A ranking function allows us to synthesize such a positional ′ strategy. Let x ∈ Xmax and b : Xmax → Ord≤z be a ranking Intuitively an ε-additive ranking supermartingale b bounds the function s.t. b(x) < ∞. We define a strategy α for max by expected number of steps to accepting states: specifically it is no bigger than b′(x)/ε. ′ ′ ′ α(x0y1 ...ynxn) = arg miny : (xn,y)∈τ supx : (y,x )∈τ b(x ) .

4 Theorem II.12 ([6]). Let b′ : X → [0, ∞] be an ε-additive inverse image diagrams, recursive coalgebras—the categori- ranking supermartingale for M and Acc. Then b′(x) < ∞ cal dual of corecursive algebras used for general structured implies ReachM,Acc(x)=1. in [25]—are known to coincide with well-founded coalgebras [26], where well-foundedness is categorically mod- Example II.13. For the PTS in Example II.10, we define b′ : eled in terms of “inductive components.” For more general X → [0, ∞] by b′(x ) = ∞, b′(x )=2, b′(x )=0 and 0 1 2 categories, it is known that if a functor preserves monos then b′(x )= ∞. Then b′ is a 1-additive ranking supermartingale. 3 well-foundedness implies recursiveness, but its converse does Hence by Thm. II.12, we have Reach (x )=1. M,Acc 1 not necessarily hold [27]. The dual of this result, between C. Categorical Preliminaries corecursive and anti-founded algebras, is pursued in [13] but with limited success. We assume that readers are familiar with basic categorical In [28] the notion of co-founded part of an algebra is notions. For more details, see e.g. [24], [7]. introduced, with a main theorem that the co-founded part of an Definition II.14 ((co)algebra). Let F : C → C be an injectively structured corecursive algebra carries a final coal- endofunctor on a category C. An F -coalgebra is a pair (X,c) gebra. The result is used for characterizing a final coalgebra as of an object X in C and an arrow c of the type c : X → FX. that of suitable modal formulas. Despite its name, co-founded An F -algebra is a pair (A, a) of an object A in C and an parts have little to do with our current view of corecursive arrow a of the type a : F A → A. algebras here as well-foundedness classifiers. Discussions on other works on corecursive algebra are found In this paper we exclusively use the category Sets of sets in §L in the appendix. and functions as the base category C (although extensions Meas e.g. to would not be hard). We would be interested D. Verification of Least/Greatest Fixed-Point Properties in endofunctors composed by the following. The following results are fundamental in the studies of Definition II.15 (P, D and ( ) × C). The powerset functor fixed-point specifications. P : Sets → Sets is such that: Theorem II.18. Let L be a complete lattice, and f : L → L • for each X ∈ Sets, PX = {A ⊆ X}; and be a monotone function. • for each f : X → Y and A ∈PX, (Pf)(A)= {y ∈ Y | ∃x ∈ A. f(x)= y}. 1) (Knaster–Tarski) The set of prefixed points (i.e. those The (discrete) distribution functor D : Sets → Sets is: l ∈ L such that f(l) ⊑ l) forms a complete lattice. Moreover its least element is (not only a prefixed but) a • for each X ∈ Sets, DX = {d : X → [0, 1] | fixed point, that is, the least fixed point µf. d(x)=1}; and x∈X 2) (Cousot–Cousot [29]) Consider the (transfinite) se- • for each f : X → Y , δ ∈ DX and y ∈ Y , (Df)(δ)(y)= a P quence ⊥ ⊑ f(⊥) ⊑ ··· ⊑ f (⊥) ⊑ · · · where, for x∈f −1(y) δ(x). a b a limit ordinal a, we define f (⊥)= b

5 in a sharp contrast with those for safety, in which finding an Definition III.2 (Φc,a). Let F : Sets → Sets, c : X → FX invariant (i.e. a post-fixed point l in (KTν)) suffices. be a coalgebra and a : F A → A be an algebra. We define a X X The basic idea behind the current contribution—liveness function Φc,a : A → A by Φc,a(f)= a ◦ Ff ◦ c, that is, checking by combination of coalgebraic simulation and core- Ff F X / FA cursive algebra—can be laid out as follows. For verification f Φc,a O X / A 7−→ c a . it is convenient if we can rely on certificates whose con-  X A ! straints are locally checkable. Their examples include invari-   ants, various notions of (bi)simulation and a general notion Then corecursiveness (Def. II.16) is rephrased as follows: r : of coalgebraic simulation; they are all postfixed points in a F R → R is corecursive if and only if the function Φc,r has a suitable sense. They should thus be able to witness only gfp’s unique fixed point for each c : X → FX. (not lfp’s) in view of Cor. II.19. Here we leverage the lfp- Our categorical modeling of modality is as follows. gfp coincidence in corecursive algebras to make coalgebraic simulations witness lfp’s too. The lfp-gfp coincidence might Definition III.3 (a truth-value domain and an F -modality). seem a serious restriction but it is a common phenomenon A truth-value domain is a poset (Ω, ⊑Ω). If the order is clear in many “interesting” structures in computer science (as we from the context we simply write Ω. For a functor F : Sets → discussed at the end of §I-B3) Sets, an F -modality over the truth-value domain Ω is an F - algebra σ : F Ω → Ω. III. CATEGORICAL RANKING FUNCTIONS Example III.4. For two-player games (i.e. Fg-coalgebras) a Here we present our general categorical framework for natural truth-value domain is given by ({0, 1}, ≤) where 1 ranking function-based liveness checking. stands for “true.” On top of this domain a natural Fg-modality σg : Fg{0, 1}→{0, 1} is given as follows. A. Running Example: Two-Player Games In this section, in order to provide abstract notions with intu- 1 (t = 1) σg(Γ,t)= itions, we use two-player games (§II-A) as a running example. (maxA∈Γ mina∈A a (otherwise) 2 We use the functor Fg = P ( ) ×{0, 1} : Sets → Sets to 2 model them as coalgebras (§II-C, here g stands for “game”). Here, in (Γ,t) ∈ FgX = P X ×{0, 1}, t ∈{0, 1} indicates if the current state is accepting or not (t =1 if yes). The second Definition III.1. Given a Fg-coalgebra c : X → FgX, we c c c c case in the above definition of σg(Γ,t) reflects the intention define a game structure G = (Xmax,Xmin, τ ) and a set c c that, in Γ ∈P(PX), the first P is for the angelic player max’s Acc ⊆ Xmax of accepting states as follows: Xmax = X, c c ′ choice while the second P is for the demonic min’s. Xmin = PX, τ = {(x, A) | x ∈ X, A ∈ c1(x)}∪{(A, x ) | ′ c A ∈ PX, x ∈ A} and Acc = {x ∈ X | c2(x)=1}. Here Using an F -modality σ, liveness is categorically character- 2 we write c(x) = (c1(x),c2(x)) ∈P X ×{0, 1} for every x. ized as a least fixed-point property. Conversely, given a game structure G = (Xmax,Xmin, τ) F JµσK G,Acc Definition III.5 ( µσ c). Let (Ω, ⊑Ω) be a /c and a set Acc ⊆ Xmax, we define an Fg-coalgebra c : F XO F Ω G,Acc ′ truth-value domainJ andK σ : F Ω → Ω be a X → FgX as follows: X = Xmax and c (x) = ({{x ∈ c =µ σ ′ modality. We say that σ has least fixed points  Xmax | (y, x ) ∈ τ} | y ∈ Xmin, (x, y) ∈ τ},t) where t is 1 X / Ω if x ∈ Acc and 0 otherwise. if for each c : X → FX, the least fixed JµσKc X X point of Φc,σ : Ω → Ω (Def. III.2)—with respect to the The above two transformations constitute an embedding- pointwise extension of the order ⊑Ω—exists. The least fixed projection pair: games and Fg-coalgebra are almost equivalent; point is called the (coalgebraic) least fixed-point property in the former have additional freedom (in the choice of the set c specified by σ, and is denoted by µσ c : X → Ω. Xmin) that is however inessential. J K Throughout the rest of this section, each categorical notion Example III.6. The Fg-modality σg : Fg{0, 1}→ {0, 1} is accompanied by a concrete example in terms of two-player in Example III.4 has least fixed points (this follows from games. For readability, the details of these examples (they are Prop. III.8 later). For each coalgebra c : X → FgX, the all straightforward) are deferred to §A in the appendix. The least fixed-point property µσg c : X → {0, 1} is concretely other running example (PTSs) will be discussed later in §IV. described by: J K

B. Modalities and Least Fixed-Point Properties, Categorically 1 (x ∈ Reach c c ) µσ (x)= G ,Acc Towards a categorical framework in which a soundness g c J K (0 (otherwise). theorem is proved on the categorical level of abstraction, we need categorical modeling of modalities and least fixed-point Conversely, for each pair of a game structure G and a set properties. Our modeling here follows [17], [18]; it has been Acc, ReachG,Acc is described by µσg cG,Acc . See Prop. A.2 for sketched in §I-B2. a proof. This way we characterizeJ reachabilityK in two-player The following function is heavily used in our developments. games in categorical terms.

6 C. Ranking Domains and Ranking Arrows corecursiveness of r (Cond. 4): it makes r a refinement of σ As we described in §I-B3, we understand liveness checking that is suited for detecting well-foundedness. as the task of determining if h ⊑ µσ c, for a given assertion Definition III.11 (ranking arrows). Let (r, q, ⊑R) be a ranking h: X → Ω. Here we introduce ourJ categoricalK machinery for domain for F and σ; and c : X → FX be a coalgebra. An providing witnesses to such satisfaction of liveness. arrow b : X → R is called a (coalgebraic) ranking arrow for c For simplicity of arguments we assume the following. with respect to (r, q, ⊑R) if it satisfies b ⊑R Φc,r(b)= r◦Fb◦c (the square on the left in (7)). Assumption III.7. Let F : Sets → Sets. We assume that a truth-value domain (Ω, ⊑Ω) and an F -modality σ : F Ω → Ω Now we give a soundness theorem for (categorical) ranking over Ω satisfy the following conditions. arrows. This is the main theorem of this paper; its proof

1) The poset (Ω, ⊑Ω) is a complete lattice. demonstrates the role of the corecursiveness assumption. 2) For each F -coalgebra c : X → FX, the function Φc,σ : Theorem III.12 (soundness). Let (r, q, ⊑R) be a ranking do- ΩX → ΩX in Def. III.5 is monotone with respect to the main. Let c : X → FX be an F JµσKc pointwise extension of ⊑ . Ω F -coalgebra and b : X → R be F X / F R +/ F Ω O F b F q These assumptions are mild. For example, Cond. 2 is satisfied a ranking arrow for c (i.e. b ⊑ c r ⊑ σ ⊑ r ◦ Fb ◦ c). Then we have: b  q  if: F Ω has an order structure; σ : F Ω → Ω is monotone; X / R / Ω X F X 4 and the action FX, : Ω → (F Ω) of F on arrows is Ω q ◦ b ⊑ µσ . JµσK X Ω c c monotone, too. Cond. 1 in the above implies that Ω is a J K complete lattice. Thus we can construct a transfinite sequence Thus for liveness checking (i.e. for proving h ⊑ µσ c) it a suffices to find a ranking arrow b such that h ⊑ q ◦Jb. InK the ⊥Ω ⊑ Φc,σ(⊥Ω) ⊑···⊑ Φc,σ(⊥Ω) ⊑ · · · as in Thm. II.18.2, to obtain the least fixed point of Φc,σ as its limit. proof of the theorem we use the following generalization of Thm. II.18.2. It starts from a post-fixed point l (not from ⊥). Proposition III.8. Under the conditions in Asm. III.7, σ has least fixed points (in the sense of Def. III.5). Lemma III.13. Assume the conditions in Thm. II.18, and let l be a post-fixed point of f, i.e. l ⊑ f(l). Then we can define a Example III.9. The data Fg, σg for two-player games satisfy transfinite sequence l ⊑ f(l) ⊑···⊑ f a(l) ⊑ · · · in a similar the assumptions: see Prop. A.3 (in the appendix) for a proof. manner to Thm. II.18.2. The sequence eventually stabilizes and We are ready to introduce the key notions. its limit is a (not necessarily least) fixed point of f. Definition III.10 (ranking domains). We assume Asm. III.7. Proof (Thm. III.12). By Cond. 2a in Def. III.10, the poset X Let r : F R → R be an F -algebra, q : R → Ω (R , ⊑R) is a complete lattice. Moreover, by its definition, be an arrow, and ⊑R be a partial order on R. Note b : X → R is a post-fixed point of Φc,r. Hence together that for each set X, the order ⊑Ω (resp. ⊑R) extends with Cond. 2b, we can construct a transfinite sequence b ⊑R a to the one between functions X → Ω (resp. X → R) Φc,r(b) ⊑R ··· ⊑R Φc,r(b) ⊑R ··· as in Lem. III.13. By m in a pointwise manner. Lem. III.13, there exists an ordinal m such that Φc,r(b) is a F X / F R / F Ω m A triple (r, q, ⊑R) is called O F b F q fixed point of Φc,r. By its definition, we have b ⊑R Φc,r(b). c r ⊑ σ a ranking domain for F and σ ⊑ (7) Note here that r is assumed to be a corecursive algebra b  q  if the following conditions are X / R / Ω (Cond. 4 in Def. III.10). Hence Φc,σ has a unique fixed point; satisfied. it is denoted by (|c|)r : X → R. Then we have: 1) We have q ◦ r ⊑ σ ◦ F q between arrows F R → Ω (the m Ω b ⊑R Φc,r(b) = (|c|)r . (8) square on the right in (7)). By Cond. 2a in Def. III.10, RX F JµσKc 2) The same conditions as in Asm. III.7 hold for r, i.e. F (|c|)r & is a complete lattice. Hence we F X / F R / F Ω a) the poset (R, ⊑R) is a complete lattice; and O 3 F q a F b X can also define Φc,r(⊥R): X → c r ⊑ σ b) for each c : X → FX, the function Φc,r : R → b   X R for each ordinal a (here ⊥R )/ q / R (Def. III.2) is monotone. X X R 9 Ω denotes the least element in R ), (|c|)r 3) The function q : R → Ω is monotone (i.e. a ⊑R b ⇒ and by Thm. II.18.2, there exists JµσKc ′ q(a) ⊑Ω q(b)), strict (i.e. q(⊥R)= ⊥Ω) and continuous ′ m m such that Φc,r(⊥R) is also a fixed point of Φc,r. Hence ′ (i.e. for each subset K ⊆ R, we have q( a∈K a) = m Φc,r(⊥R) = (|c|)r. q(a)). a a∈K We shall now prove that q ◦ Φ (⊥R) ⊑ µσ c holds for 4) The algebra r : F R → R is corecursive. F c,r F each ordinal a. This is by transfinite inductionJ onK a. Cond. 2 in the definition ensures that the least fixed point of For a =0, we have: Φ arises from the approximation sequence in Thm. II.18.2. a c,r q ◦ Φc,r(⊥R)= q ◦⊥R (by definition) Cond. 3 ensures that this least fixed point is preserved by = ⊥ (by Cond. 3 in Def. III.10) q. In particular we insist on strictness—this is much like in Ω domain theory [30]. The most significant in Def. III.10 is the ⊑Ω µσ c . J K

7 For a successor ordinal a +1, we have: b) b : X → Ord≤z is a ranking function for a game structure G and a set Acc iff b is a ranking arrow for a+1 q ◦ Φ (⊥ ) G,Acc c,r R c wrt. (rg,z, qg,z, ⊑Ord). a = q ◦ r ◦ F (Φc,r(⊥R)) ◦ c (by definition) c) b(x) < z iff qg,z ◦ b(x)=1. a by Cond. 1 in Def. III.10 ⊑Ω σ ◦ F (q ◦ Φc,r(⊥R)) ◦ c ( ) Here recall the correspondence in Def. III.1. A formal state- ⊑Ω σ ◦ F ( µσ c) ◦ c (by IH and Asm. III.7.2) ment and its proof are found in Prop. A.7. J K = µσ c . ( µσ c is a fixed point) Combined with the characterization in Example III.6, we J K J K conclude that the conventional soundness result (Thm. II.5) is For a limit ordinal l, we have: an instance of our categorical soundness (Thm. III.12). l q ◦ Φc,r(⊥R) Remark III.16. Assume the conditions in Thm. III.12. As a = q a 0. Note that in the corresponding game

rg,z : FgOrd≤z → Ord≤z, qg,z : Ord≤z →{0, 1}, ⊑Ord structure Gc, all the choices are made by the player min. Then we have µσg c(xω)=1 because of well-foundedness of ω.   Ord as follows. However,J the uniqueK arrow (|c|)rg,ω : X → ≤ω such that (|c|) =Φ ((|c|) ) assigns, to each state x , the ordinal 1) Ord≤z = {a | a is an ordinal s.t. a ≤ z}, and rg,ω c,σg rg,ω a a. This means that qg,ω ◦ (|c|)rg,ω (xω) = qg,ω(ω)=0. Thus

0 (t = 1) qg,ω ◦ (|c|)rg,ω < µσg c. rg,z(Γ,t)= J K (minA∈Γ supa∈A(a +1) (otherwise) . Similarly, for every ordinal number z, we can construct an Fg-coalgebra whose reachability cannot be proved by the 2) z , and a for any a such that a z; qg,z( )=0 qg,z( )=1 < ranking domain rg,z : FgOrd≤z → Ord≤z. def. b 3) a ⊑Ord b ⇔ a ≥ b (note the directions of inequalities). By cardinality arguments we can show that sort of “com- Recall that a + 1 denotes min{a + 1, z}. The triple pleteness” holds in the example above, in the following sense: (r , q , ⊑Ord) is indeed a ranking domain (see Prop. A.4 in g,z g,z for every F -coalgebra c there exists an ordinal z such that the appendix). One can think of the above data as a classifier g b the reachability of c is provable by the ranking domain r . for (non-)well-foundedness: all the ordinals a < z are for g,z However, in this paper we use the term “completeness” in “well-founded” and the maximum ordinal z is for “non-well- a different sense in which we fix the domain R of ranking founded.” Observe that the map q acts accordingly. g,z functions in advance. We indeed have the following correspondences. Here is a categorical sufficient condition for completeness. a) b : X → Ord≤z is a (categorical) ranking arrow (Def. III.11) for an Fg-coalgebra c : X → FgX wrt. Proposition III.18 (a sufficient condition for completeness). c Ord (rg,z, qg,z, ⊑ ) iff b is a ranking function for G and Let (r, q, ⊑R) be a ranking domain, c : X → FX be an c Acc (in the conventional sense of Def. II.4). F -coalgebra, and (|c|)r : X → R be the unique arrow such

8 F JµσKc that (|c|) = r ◦ F (|c|) ◦ c. As- B. Known Variations: ε-Additive and α-Multiplicative Rank- r r F X / F R ,/ F Ω sume that we have the equality O F (|c|)r F q ing Supermartingales c = r = σ (|c|)r  q  q ◦ r = σ ◦ F q , (9) X / R /2 Ω The definition of ranking supermartingale that we have re- instead of an inequality, in the JµσKc viewed (ε-additive ones in Def. II.11) is not an instance of our categorical notion (Def. III.10). Specifically, its value domain square on the right. Then we have q ◦ (|c|)r = µσ c. J K (the interval [0, ∞] with a suitable Fp-algebraic structure) fails Intuitively, the equality (9) means that r approximates the to be corecursive. As a result, soundness of additive ranking modality σ in an adequate way. The result implies that, in supermartingale (Thm. II.12) cannot be directly proved using case h: X → Ω satisfies h ⊑ µσ c, the latter inequality our categorical soundness theorem (Thm. III.12). J K can always be witnessed by some ranking arrow (namely The following is an attempt to define a ranking domain for ). This is completeness of the proof method of categorical (|c|)r ε-additive supermartingales. Let us fix a real number ε > 0 ranking arrows. An example of a complete ranking domain ′ and define an Fp-algebra rp,ε : Fp[0, ∞] → [0, ∞], an arrow will be given in §IV-C. ′ qp : [0, ∞] → [0, 1] and a partial order ⊑[0,∞] over [0, ∞] as IV. CATEGORICAL RANKING ARROWS FOR follows. PROBABILISTIC TRANSITION SYSTEMS 1) For each (ψ,t) ∈ Fp[0, ∞]= D[0, ∞] ×{0, 1}, We shall now investigate what our categorical framework in §III entails in the probabilistic setting of §II-B. It turns out ′ 0 (t = 1) that the well-known definition of ranking supermartingale (ε- rp,ε(ψ,t)= a · ψ(a) + ε (otherwise) . additive ones in Def. II.11) is not an instance. Here we study ( a∈supp(ψ) some variations of the definition of ranking supermartingale; ′ P′  2) qp(∞)=0 and qp(a)=1 if a< ∞. two among them (distribution-valued and non-counting ones, def. that are new to our knowledge) exhibit the nice categorical 3) a ⊑[0,∞] b ⇔ a ≥ b (note the direction). properties in §III. We also discuss some relationships between Proposition IV.3. In this setting, for each c : X → FpX and those variations, showing that the soundness of ε-additive b′ : X → [0, ∞], we have the following. ranking supermartingales (Def. II.11) can nevertheless be ′ proved via the categorical arguments in §III. a) b is an ε-additive ranking supermartingale (Def. II.11) ′ ′ ′ ′ iff b satisfies b ⊑[0,∞] rp,ε ◦ Fpb ◦ c. A. Probabilistic Transition Systems as Coalgebras ′ ′ ′ b) b (x) 6= ∞ iff qp ◦ b (x)=1. To represent a PTS as a coalgebra, we use the functor Fp : ′ ′ Sets → Sets (p stands for “probability”) defined as follows. Therefore the triple (rp,ε, qp, ⊑[0,∞]) is suited for accom- modating ε-additive supermartingales in our categorical frame- Definition IV.1 (Fp). We let Fp = D( ) ×{0, 1}, where D work. Unfortunately it is not a ranking domain (Def. III.10). is the (discrete) distribution functor in Def. II.15. c ′ ′ For an Fp-coalgebra c : X → FpX, we define a PTS M = Proposition IV.4. The triple (rp,ε, qp, ⊑[0,∞]) introduced (Xc, τ c) and a set Accc ⊆ Xc of accepting states by: Xc = X, above satisfies the conditions of a ranking domain c c c τ (x) = c1(x) for each x ∈ X , and Acc = {x ∈ X | (Def. III.10), except for Cond. 4. c2(x)=1}. Here we write c(x) = (c1(x),c2(x)) ∈ DX × {0, 1} for every x. Example IV.5. We define a coalgebra Conversely, for a PTS M = (X, τ) and a set Acc ⊆ X, we c : X → FpX by: X = {x , x , x , x }, 0 1 2 3 1 1 M,Acc M,Acc 1 1 2  define an F -coalgebra c : X → F X by c (x) = c(x ) = ([x 7→ , x 7→ ], 0), c(x ) = 1 ✉ p p 0 1 2 2 2 1 2 x3 1 1 ? ❄_ 1 (τ(x),t) where t is 1 if x ∈ Acc and 0 otherwise. ([x1 7→ , x3 7→ ], 0), c(x2) = ([x3 7→ ⑧⑧ ❄❄ 2 2 -  ÈÉ ⑧x  ÈÉ x and . The ❄_ 1 ? 2 1], 0) c(x3) = ([x3 7→ 1], 1) ❄ ⑧ 1 These correspondences are indeed bijective. Analogously to 1 ❄ ⑧ 2 corresponding PTS is depicted on the right. 2  ÈÉ ⑧x0 Example III.6, we characterize reachability probabilities of a Let ε > 0. For this coalgebra, we define arrows b1,b2 : PTS as a coalgebraic least fixed-point property. 5 X → [0, ∞] by: b1(x0) = 2 ε, b1(x1)=2ε, b1(x2) = ε and Proposition IV.2. We define an Fp-modality σp : Fp[0, 1] → b1(x3)=0; and b2(x0) = ∞, b2(x1) = ∞, b2(x2) = ε and [0, 1] over the truth-value domain ([0, 1], ≤) as follows. Here b2(x3)=0. Both of these qualify as coalgebra-algebra homo- is the unit interval and is the usual order on it. ′ ′ [0, 1] ≤ morphisms from c to rp,ε. Therefore rp,ε is not corecursive. 1 (t = 1) σp(δ, t) = It is well-known that ε-additive supermartingales (Def. II.11) ( a∈supp(δ) a · δ(a) (otherwise) . witness positive almost-sure reachability [31], that is, the Note here that δ ∈ DP[0, 1] and t ∈{0, 1}. expected number of steps to accepting states is finite. This is Then σp satisfies Asm. III.7 and thus has least fixed points a property strictly stronger than almost-sure reachability (see (in the sense of Def. III.5). The lfp property µσp c coincides Example IV.6). It follows that ε-additive supermartingales are with the reachability probability function JReachK M c,Accc : not complete against almost-sure reachability. X → [0, 1] of the corresponding PTS (Def. II.9).

9 s MO z 1 Example IV.6. We define c : s  ÈÉ Definition IV.8 (distribution-valued ranking function). Let 1 ; O 1 O X → FpX by X = {x}∪{xi,j | 1 M = (X, τ) be a PTS and Acc ⊆ X. A distribution-valued 1 . }| N i  ÈÉ . 2i − 1 N i, j ∈ , 1 ≤ i, 1 ≤ j ≤ 2 }; O .O ranking function is b : X → D ∞ such that:  1 ...... and c(x) = (δ, 0), where δ(xi,j ) s  ÈÉ  ÈÉ ′ ′ 1 O O O ′ τ(x)(x ) · b(x ) [0,a − 1] ≥ b(x) [0,a] is if j = 0 and 0 otherwise, 1 1 1 { x ∈supp(τ) 2i  ÈÉ  ÈÉ  ÈÉ c(xi,j ) = ([xi,j+1 7→ 1], 0) for 1 ✴W G C for each x ∈ X \ Acc and a ∈ N . Here we let 2 ✴ ✎ 1 P ∞ i ✴ ✎ 4 1 ′ each i and j < 2 , and c(xi,2i ) = x  ÈÉ ✎ 2i b(x )([0, −1]) = 0. ([xi,2i 7→ 1], 1). The corresponding c By Thm. III.12 (soundness) we have the following: given a PTS M c and the set Acc of accepting states are shown on the right. This system when run from is clearly almost- PTS c: X → FpX and an “assertion” h: X → [0, 1], if there x N sure terminating; however the expected number of steps to exists a distribution-valued ranking function b: X → D ∞ accepting states is infinite. such that h ≤ qp ◦ b, then we can conclude that h ≤ ReachM c,Accc . Here ReachM c,Accc is given by reachability Another known variation of ranking supermartingales is probabilities and coincides with µσp c (Prop. IV.2). given by multiplicative ranking supermartingales [23]. Let We note that quantitative verificationJ K is possible using α ∈ (0, 1). A function b : X → [0, ∞] is an α-multiplicative distribution-valued ranking functions. For example an asser- ranking supermartingale if we have tion h: X → [0, 1] can be such that h(x)=1/2; by finding a suitable arrow b we conclude h ≤ ReachM c,Accc , that is, Acc ′ ′ ∀x ∈ X \ . x′∈supp(τ(x)) τ(x)(x ) · b(x ) ≤ α · b(x) , that the reachability probability from x is at least 1/2.3 Such a quantitative assertion cannot be verified using -additive and moreover thereP exists δ > 0 such that b(x) ≥ δ for each ε supermartingales: as in Thm. II.12 they only witness the x ∈ X\Acc. For this multiplicative variation, results analogous qualitative property of (positive) almost-sure reachability. to Prop. IV.3 and Prop. IV.4 hold (see §G in the appendix for the proofs). Example IV.9. We define a PTS M = ✉ x2 1 ; O x1  ÈÉ (X, τ) by X = {x0, x1, x2}, τ(x0) = 1 ; c 1 1 1 3 ✇✇ C. Distribution-Valued Ranking Functions 1 ✇ 1 1 [x0 7→ , x1 7→ , x2 7→ ], τ(x1) =  ÈÉ ✇ 3 3 3 3 3 ; Let us turn to possible instantiations of our categorical [x1 7→ 1] and τ(x2) = [x2 7→ 1]. Let x0 N framework in the current probabilistic setting. The first uses Acc = {x1}. The function b : X → D ∞ defined by b(x0)= i+1 DN∞, the set of distributions over extended natural numbers, [i 7→ 1/3 , ∞ 7→ 1/2], b(x1) = [0 7→ 1] and b(x2) = [∞ 7→ as a ranking domain (instead of [0, ∞]). In what follows, given 1] is a distribution-valued ranking function. This allows us to 1 1 a probability distribution γ over N∞ and a,b ∈ N, γ([a,b]) conclude ReachM,Acc(x0) ≥ (1 − 2 )= 2 . b ∞ denotes i=a γ(i) and γ([a, ∞)) denotes i=a ϕ(i). Finally we exhibit completeness of distribution-valued rank- ing functions. This is an immediate consequence of the cate- PropositionP IV.7. Recall that N∞ = N ∪P {∞} and DN∞ gorical result (Prop. III.18) and Prop. IV.7. collects all the distributions over N∞. We define an Fp-algebra N N N rp : FpD ∞ → D ∞, a function qp : D ∞ → [0, 1] and a Proposition IV.10. For each Fp-coalgebra c : X → FpX, partial order N over N as follows. ⊑D ∞ D ∞ there exists a (categorical) ranking arrow b : X → DN∞ 2 1) For each (Γ,t) ∈ FpDN∞ = D N∞ ×{0, 1}, such that qp ◦ b = µσp c. J K 1 (t = 1, a = 0) Example IV.11. In Example IV.6, the function b : X → DN∞ rp(Γ,t)(a)= 0 (t = 1,a> 0 or t = 0, a = 0) defined by (P Γ(γ) · γ(a − 1) (t = 0,a> 0) . γ∈supp(Γ) 1/2i (n = 2i,i> 0) b(x)(n)= and b(x )= δ i 0 (otherwise) i,j 2 −j 2) qp(ϕ)= ϕ [0, ∞) .  ′ def. ′ 3) N N . is a distribution-valued ranking function (here δ2i−j denotes ϕ ⊑D ∞ ϕ ⇔ ∀a ∈ . ϕ([0,a]) ≤ ϕ ([0,a]) a Dirac distribution). We have b(x) [0, ∞) = 1 and thus Then the triple (rp, qp, ⊑DN ) is a ranking domain. Moreover ∞ successfully verify almost-sure reachability from x. we have qp ◦ rp = σp ◦ Fpqp (cf. (9) in Prop. III.18).  In §IV-B we have argued that ε-additive ranking super- Intuitively, the value b(x)([0,a]) ∈ [0, 1] under-approximates martingales (Def. II.11) is not an instance of our categorical the probability with which an accepting state is reached from ranking arrow. It is nevertheless possible to prove its soundness x within a steps. The definition of ⊑ N , which is much D ∞ (Thm. II.5) using the categorical framework—specifically by like in probabilistic powerdomains (see e.g. [32]), also reflects showing that an ε-additive ranking supermartingale gives rise this intuition. Here the Dirac distribution δ (resp. δ ) is the 0 ∞ to a distribution-valued ranking function (Def. IV.8). Details greatest (resp. least) element. are found in §I in the appendix. The definition of ranking arrow (Def. III.11) instantiates to the following—as one sees by straightforward calculation— 3The problem solved here is more precisely the threshold reachability when we fix a ranking domain to be the one in Prop. IV.7. checking problem; and ranking function-based proof methods for the problem would be viable options especially when the state space X is infinite.

10 Similarly our framework can prove soundness of α- to be a γ-scaled non-counting ranking supermartingale. By γ multiplicative ranking supermartingales (see §J for the details). Cor. IV.14 we have ReachAcc(x0) ≥ 3−γ ; as this holds for any 1 γ ∈ [0, 1), by letting γ → 1, we conclude ReachAcc(x ) ≥ . D. γ-Scaled Non-Counting Ranking Supermartingales 0 2 The notion of distribution-valued ranking function exhibits We note that in general a scaling factor γ ∈ [0, 1) results pleasant properties like completeness and quantitative asser- in suboptimality of under-approximation of reachability prop- erties: in the last example γ is smaller than tion checking. A major drawback, however, is the complexity bγ (x0) = 3−γ the reachability probability 1/2. Such suboptimality is an of its value domain DN∞. In many realistic verification scenarios a ranking func- issue especially when we aim at qualitative verification of tion/supermartingale b would be synthesized as follows: the almost-sure reachability. In the last example we exercised an asymptotic argument in which we think of γ as a free variable function b is expressed in a predetermined template b~p (such as polynomials up-to a certain degree) in which some parameters and take the limit under γ → 1. This strategy can be employed for almost-sure reachability checking. ~p occur; the requirements on b~p translate to constraints on ~p; and one relies on some optimization solver (for SAT, LP, Finally, the following example demonstrates that application SDP, etc.) to solve the constraints. It significantly increases the of non-counting supermartingales is not limited to positive complexity of the workflow if the value b(x) is a distribution almost-sure reachability. N in D ∞ instead of an (extended) real number in [0, ∞]. Example IV.16. We define a PTS M = (X, τ) and Acc ⊆ X Here we present another probabilistic instantiation of the as in Example IV.6. For each γ, if we define bγ : X → [0, 1] 2i categorical framework. It takes values in the unit interval [0, 1]. ∞ γ 2i−j by bγ (x)= i=1 2i and bγ (xi,j )= γ for each i and j, Proposition IV.12. We fix a real number γ ∈ [0, 1). We define then it is a γ-scaled non-counting supermartingale. Hence we P an algebra rnc,γ : Fp[0, 1] → [0, 1] (here nc stands for “non- have ReachM,Acc(x) ≥ limγ→1 bγ(x)=1. counting”) as follows. Let us summarize the section. We presented four variations 1 (t = 1) of ranking supermartingales: ε-additive ones, α-multiplicative rnc,γ(ϕ, t)= γ · a∈supp(ϕ) a · ϕ(a) (otherwise) . ones, distribution-valued ranking functions and γ-scaled non-  P counting ranking supermartingales. The former two are known We further define qnc : [0, 1] → [0, 1] by qnc(a) = a. Then in the literature while the latter two seem to be new. The (rnc,γ, qnc, ≤), where ≤ is the usual on [0, 1], is a ranking known notions are not instances of our generic definition, domain with respect to the modality σp (Prop. IV.2). but their soundness can be derived via our generic theory Thus it makes sense to consider ranking arrows (Def. III.11) (see the end of §IV-C). Among the (seemingly) new notions, with respect to the ranking domain (rnc,γ , qnc, ≤). By straight- distribution-valued ranking functions enjoy nice properties like forward calculation, their definition unravels as follows. completeness (Prop. IV.10) and support of quantitative reason- ing (see Example IV.9)—at the cost of their complexity (they Definition IV.13 (γ-scaled non-counting ranking supermartin- take as values distributions in DN∞). Non-counting ranking gale). Let M = (X, τ) be a PTS and Acc ⊆ X. We fix a supermartingales (whose values are simply real numbers) are real number γ such that 0 ≤ γ < 1. A γ-scaled non-counting advantageous in quantitative reasoning (see Example IV.15) ranking supermartingale is a function b : X → [0, 1] such that and non-positive almost-sure termination, but the scaling factor for each x ∈ X \ Acc we have: γ in it leads to suboptimal approximation. Typically one needs ′ ′ to rely on asymptotic arguments to obtain sharp bounds. γ · x′∈X τ(x)(x ) · b(x ) ≥ b(x) . This notion ofP supermartingale seems new. The intuition is V. CONCLUSIONS AND FUTURE WORK that b(x) is a lower bound for the reachability probability. We have given a categorical account for liveness check- Note that, unlike for the other variations of supermartingales in ing: we identify the essence of ranking function-based proof this section (where we over-approximate the number of steps), methods as the combination of corecursive algebras (as value reachability probabilities should be under-approximated. domains) and lax homomorphisms; and for our notion of We obtain the following soundness result as an instance of ranking arrow a soundness theorem has been presented. Our Thm. III.12. Here a non-counting ranking supermartingale b leading examples have been two-player games and probabilis- itself gives lower bounds for reachability probabilities since tic transition systems; in the course of studying them we were qnc : [0, 1] → [0, 1] is the identity map. led to (seemingly) new variations of ranking martingales. Besides the concrete examples of “ranking functions” in this Corollary IV.14. Let M = (X, τ) be a PTS and b : X → paper, we wish to derive yet other concrete examples from [0, 1] be a γ-scaled non-counting ranking supermartingale. our categorical modeling, so that they provide novel proof Then we have Reach (x) ≥ b(x). Acc methods for various liveness properties. A possible direction Example IV.15. Consider the PTS M and Acc in Exam- towards this goal is discussed in §L in the appendix, moti- ple IV.9. For each γ ∈ [0, 1), we define bγ : X → [0, 1] by vated by categorical closure properties of corecursive algebras. γ bγ(x0) = 3−γ , bγ (x1)=1 and bγ(x2)=0. Then bγ is seen Some abstract categorical questions remain open, too, such as

11 characterization of the non-well-founded part—that represents [5] A. Chakarov and S. Sankaranarayanan, “Probabilistic program analysis failure of liveness properties—of a corecursive algebra. We are with martingales,” in Computer Aided Verification - 25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13-19, 2013. also interested in the relationship between these (expected) Proceedings, ser. Lecture Notes in Computer Science, N. Sharygina results/examples and productivity for coinductive datatypes and H. Veith, Eds., vol. 8044. Springer, 2013, pp. 511–526. [Online]. in functional programming [35]. Intuitively, the latter is a Available: http://dx.doi.org/10.1007/978-3-642-39799-8 34 [6] L. M. F. Fioriti and H. Hermanns, “Probabilistic termination: Soundness, property that any finite prefix of a coinductively defined data completeness, and compositionality,” in Proceedings of the 42nd Annual is obtained in finite time. ACM SIGPLAN-SIGACT Symposium on Principles of Programming We have used two-player games (systems with angelic and Languages, POPL 2015, Mumbai, India, January 15-17, 2015, S. K. Rajamani and D. Walker, Eds. ACM, 2015, pp. 489–501. [Online]. demonic transitions) and PTSs (systems with probabilistic Available: http://doi.acm.org/10.1145/2676726.2677001 transitions) as leading examples. A natural direction of future [7] B. Jacobs, Introduction to Coalgebra: Towards Mathematics of States work is to consider stochastic games, which involve angelic, and Observation, ser. Cambridge Tracts in Theoretical Computer Sci- ence. Cambridge University Press, 2016. demonic and probabilistic transitions [36]. [8] J. J. M. M. Rutten, “Universal coalgebra: a theory of systems,” Theor. In this paper we have focused on least fixed-point properties. Comp. Sci., vol. 249, pp. 3–80, 2000. Extension to nested fixed-point specifications—persistence, [9] A. Sokolova, “Probabilistic systems coalgebraically: A survey,” Theor. Comput. Sci., vol. 412, no. 38, pp. 5095–5110, 2011. [Online]. recurrence, and general fixed-point formulas—is important Available: http://dx.doi.org/10.1016/j.tcs.2011.05.008 future work. There we will need to categorically axiomatize [10] P. Aczel and N. P. Mendler, “A final coalgebra theorem,” in Category progress measures for parity games ([37]; see also [38]). Theory and Computer Science, Manchester, UK, September 5-8, 1989, Proceedings, ser. Lecture Notes in Computer Science, D. H. Pitt, D. E. Possibly relevant to this direction is our recent coalgebraic Rydeheard, P. Dybjer, A. M. Pitts, and A. Poign´e, Eds., vol. 389. modeling of B¨uchi and parity acceptance conditions [39]. Springer, 1989, pp. 357–365. Practical implications of the proposed framework (and con- [11] I. Hasuo, “Generic forward and backward simulations,” in CONCUR, ser. Lecture Notes in Computer Science, C. Baier and H. Hermanns, crete “ranking functions” derived thereby) shall be investi- Eds., vol. 4137. Springer, 2006, pp. 406–420. gated, too. We are especially interested in cyber-physical ap- [12] N. Urabe and I. Hasuo, “Coalgebraic infinite traces and Kleisli simu- plications in which state spaces are inherently infinite but often lations,” in Algebra and Coalgebra in Computer Science - 6th Interna- tional Conference, CALCO 2015, Nijmegen, Netherlands, June 24-26, allow succinct symbolic presentations (e.g. by polynomials). 2015. Proceedings, 2015. The work closely related to this direction is [23]. [13] V. Capretta, T. Uustalu, and V. Vene, “Corecursive algebras: A study of general structured corecursion,” in Formal Methods: Foundations and A categorical account on martingales is also found in Applications, 12th Brazilian Symposium on Formal Methods, SBMF recent [40], where a connection between two classic results— 2009, Gramado, Brazil, August 19-21, 2009, Revised Selected Papers, Kolmogorov’s extension theorem and Doob’s martingale con- ser. Lecture Notes in Computer Science, M. V. M. Oliveira and J. Woodcock, Eds., vol. 5902. Springer, 2009, pp. 84–100. [Online]. vergence theorem—is established in categorical terms. The Available: http://dx.doi.org/10.1007/978-3-642-10452-7 7 relationship between this work and ours shall be pursued, [14] M. B. Smyth and G. D. Plotkin, “The category theoretic solution of possibly centered around the notion of final sequence. recursive domain equations,” SIAM Journ. Comput., vol. 11, pp. 761– 783, 1982. [15] P. J. Freyd, “Algebraically complete categories,” in Como Conference on ACKNOWLEDGMENT Category Theory, ser. Lect. Notes Math., A. Carboni, M. C. Pedicchio, and G. Rosolini, Eds., no. 1488. Springer, Berlin, 1991, pp. 95–104. Thanks are due to Eugenia Sironi and anonymous referees [16] I. Hasuo, B. Jacobs, and A. Sokolova, “Generic trace semantics via coinduction,” Logical Methods in Computer Science, vol. 3, no. 4, 2007. for their useful comments. The authors are supported by ER- [17] I. Hasuo, “Generic weakest precondition semantics from monads ATO HASUO Metamathematics for Systems Design Project enriched with order,” Theor. Comput. Sci., vol. 604, pp. 2–29, 2015. (No. JPMJER1603), JST, and Grants-in-Aid No. 15KT0012 [Online]. Available: http://dx.doi.org/10.1016/j.tcs.2015.03.047 [18] W. Hino, H. Kobayashi, I. Hasuo, and B. Jacobs, “Healthiness & 15K11984, JSPS. Natsuki Urabe is supported by Grant-in- from duality,” in Proceedings of the 31st Annual ACM/IEEE Aid for JSPS Fellows (No. 16J08157). Symposium on Logic in Computer Science, LICS ’16, New York, NY, USA, July 5-8, 2016, M. Grohe, E. Koskinen, and N. Shankar, Eds. ACM, 2016, pp. 682–691. [Online]. Available: REFERENCES http://doi.acm.org/10.1145/2933575.2935319 [19] L. Schr¨oder and D. Pattinson, “PSPACE bounds for rank-1 modal [1] V. Schuppan and A. Biere, “Liveness checking as safety logics,” ACM Trans. Comput. Log., vol. 10, no. 2, 2009. checking for infinite state spaces,” Electr. Notes Theor. Comput. [20] C. Urban and A. Min´e, “An abstract domain to infer ordinal- Sci., vol. 149, no. 1, pp. 79–96, 2006. [Online]. Available: valued ranking functions,” in Programming Languages and Systems http://dx.doi.org/10.1016/j.entcs.2005.11.018 - 23rd European Symposium on Programming, ESOP 2014, Held [2] K. Claessen and N. S¨orensson, “A liveness checking algorithm that as Part of the European Joint Conferences on Theory and Practice counts,” in Formal Methods in Computer-Aided Design, FMCAD of Software, ETAPS 2014, Grenoble, France, April 5-13, 2014, 2012, Cambridge, UK, October 22-25, 2012, G. Cabodi and Proceedings, ser. Lecture Notes in Computer Science, Z. Shao, S. Singh, Eds. IEEE, 2012, pp. 52–59. [Online]. Available: Ed., vol. 8410. Springer, 2014, pp. 412–431. [Online]. Available: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6462555 http://dx.doi.org/10.1007/978-3-642-54833-8 22 [3] P. K. Nalla, R. K. Gajavelly, H. Mony, J. Baumgartner, and [21] Z. Manna and A. Pnueli, “Adequate proof principles for invariance R. Kanzelman, “Effective liveness verification using a transformation- and liveness properties of concurrent programs,” Sci. Comput. based framework,” in 2014 27th International Conference on VLSI Program., vol. 4, no. 3, pp. 257–289, 1984. [Online]. Available: Design and 2014 13th International Conference on Embedded Systems, http://dx.doi.org/10.1016/0167-6423(84)90003-0 Mumbai, India, January 5-9, 2014. IEEE Computer Society, 2014, pp. [22] E. Gr¨adel and I. Walukiewicz, “Positional determinacy of 74–79. [Online]. Available: http://dx.doi.org/10.1109/VLSID.2014.20 games with infinitely many priorities,” Logical Methods in [4] R. W. Floyd, “Assigning meanings to programs,” Proceedings of Computer Science, vol. 2, no. 4, 2006. [Online]. Available: Symposium on Applied Mathematics, vol. 19, pp. 19–32, 1967. http://dx.doi.org/10.2168/LMCS-2(4:6)2006

12 [23] A. N. Chakarov, “Deductive verification of infinite-state stochastic systems using martingales,” Ph.D. dissertation, University of Colorado, 2016. [24] S. Mac Lane, Categories for the working mathematician, 2nd ed. Springer-Verlag New York, 1998, vol. 5. [25] G. Osius, “Categorical set theory: A characterization of the category of sets,” Journal of Pure and Applied Algebra, vol. 4, no. 1, pp. 79 – 119, 1974. [Online]. Available: http://www.sciencedirect.com/science/article/pii/0022404974900322 [26] P. Taylor, Practical foundations of mathematics, ser. Cambridge studies in advanced mathematics. Cambridge, New York (N. Y.), Melbourne: Cambridge University Press, 1999. [Online]. Available: http://opac.inria.fr/record=b1095522 [27] J. Ad´amek, S. Milius, L. S. Moss, and L. Sousa, “Well-pointed coalgebras,” Logical Methods in Computer Science, vol. 9, no. 3, 2013. [Online]. Available: http://dx.doi.org/10.2168/LMCS-9(3:2)2013 [28] P. B. Levy, “Final coalgebras from corecursive algebras,” in 6th Conference on Algebra and Coalgebra in Computer Science, CALCO 2015, June 24-26, 2015, Nijmegen, The Netherlands, ser. LIPIcs, L. S. Moss and P. Sobocinski, Eds., vol. 35. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2015, pp. 221–237. [Online]. Available: http://dx.doi.org/10.4230/LIPIcs.CALCO.2015.221 [29] P. Cousot and R. Cousot, “Constructive versions of Tarski’s fixed point theorems,” Pacific Journal of Mathematics, vol. 82, no. 1, pp. 43–57, 1979. [30] G. Plotkin, “Notes on domains (pisa notes),” 1983, available at http://www.dcs.ed.ac.uk/home/gdp/publications/. [31] O. Bournez and F. Garnier, “Proving positive almost-sure termination,” in Term Rewriting and Applications, 16th International Conference, RTA 2005, Nara, Japan, April 19-21, 2005, Proceedings, ser. Lecture Notes in Computer Science, J. Giesl, Ed., vol. 3467. Springer, 2005, pp. 323–337. [Online]. Available: http://dx.doi.org/10.1007/978-3-540-32033-3 24 [32] N. Saheb-Djahromi, “Cpo’s of measures for nondeterminism,” Theor. Comput. Sci., vol. 12, pp. 19–37, 1980. [Online]. Available: http://dx.doi.org/10.1016/0304-3975(80)90003-1 [33] J. Ad´amek, M. Haddadi, and S. Milius, “Corecursive algebras, corecursive monads and bloom monads,” Logical Methods in Computer Science, vol. 10, no. 3, 2014. [Online]. Available: http://dx.doi.org/10.2168/LMCS-10(3:19)2014 [34] V. Capretta, T. Uustalu, and V. Vene, “Recursive coalgebras from comonads,” Inf. Comput., vol. 204, no. 4, pp. 437–468, 2006. [Online]. Available: http://dx.doi.org/10.1016/j.ic.2005.08.005 [35] A. Abel and B. Pientka, “Well-founded recursion with copatterns and sized types,” J. Funct. Program., vol. 26, p. e2, 2016. [Online]. Available: http://dx.doi.org/10.1017/S0956796816000022 [36] L. S. Shapley, “Stochastic games,” Proceedings of the National Academy of Sciences, vol. 39, no. 10, pp. 1095–1100, 1953. [Online]. Available: http://www.pnas.org/content/39/10/1095.short [37] M. Jurdzinski, “Small progress measures for solving parity games,” in STACS, ser. Lecture Notes in Computer Science, H. Reichel and S. Tison, Eds., vol. 1770. Springer, 2000, pp. 290–301. [38] I. Hasuo, S. Shimizu, and C. Cˆırstea, “Lattice-theoretic progress measures and coalgebraic model checking,” in Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20 - 22, 2016, R. Bodik and R. Majumdar, Eds. ACM, 2016, pp. 718– 732. [Online]. Available: http://doi.acm.org/10.1145/2837614.2837673 [39] N. Urabe, S. Shimizu, and I. Hasuo, “Coalgebraic trace semantics for B¨uchi and parity automata,” in 27th International Conference on Concurrency Theory, CONCUR 2016, August 23- 26, 2016, Qu´ebec City, Canada, ser. LIPIcs, J. Desharnais and R. Jagadeesan, Eds., vol. 59. Schloss Dagstuhl - Leibniz- Zentrum fuer Informatik, 2016, pp. 24:1–24:15. [Online]. Available: http://dx.doi.org/10.4230/LIPIcs.CONCUR.2016.24 [40] D. Kozen, “Kolmogorov extension, martingale convergence, and com- positionality of processes,” in Proc. 31st ACM-IEEE Symp. Logic in Computer Science (LICS 2016), N. Shankar, Ed. New York City: ACM, July 2016, pp. 692–699.

13 c APPENDIX (by def. of Gc and Acc ) A. Formal Discussions for Two-Player Games ⇔ ∃α : a strategy of max. ∀β : a strategy of min. α,β,x α,β,x Definition A.1 (σg). Let ({0, 1}, ≤) be a truth-value domain ρ is winning for max (by def. of ρ ) where ≤ denotes the usual order. We define an Fg-modality ⇔ f(x)=1 (by def. of f) . σg : Fg{0, 1}→{0, 1} by Hence f is a fixed point of Φc,σg . 1 (t = 1) It remains to show that f : X → {0, 1} is the least fixed σg(Γ,t)= ′ maxA∈Γ mina∈A a (otherwise) . point with respect to the pointwise extension of ≤. Let f : ( ′ X → {0, 1} be a fixed point of Φc,σg . To prove f ≤ f , it Proposition A.2. We define an Fg-modality σg : Fg{0, 1} → suffices to prove that f ′(x)=0 implies f(x)=0 for each {0, 1} as in Def. A.1. x ∈ X. ′ 1) The modality σg has least fixed points, and for each For each x ∈ X, we have: coalgebra c : X → F X, the corresponding least fixed- g f ′(x′)=0 point property µσg c : X →{0, 1} is given as follows: ′ ′ ′ J K ⇔ Φc,σg (f )(x )=0 (f is a fixed point of Φc,σg ) 1 (x ∈ Reach c c ) ′ ′ G ,Acc ⇔ (σg ◦ Fgf ◦ c)(x )=0 (by def. of Φ ) µσg c(x)= c,σg 0 (otherwise) . ′ ′ ′′ ′ ′′ J K ( ⇔ c2(x )=0, and ∀A ∈ c1(x ). ∃x ∈ A. f (x )=0

2) For a game structure G = (Xmax,Xmin, τ) and a set (by def. of σg and Fg) Acc ⊆ Xmax, we have: ⇔ x′ ∈/ Accc, and c ′ c 1 (x ∈ ReachG,Acc) ∀y ∈ Xmin s.t. (x ,y) ∈ τ . µσ G,Acc (x)= g c ′′ c s.t. ′′ c ′ ′′ J K (0 (otherwise) . ∃x ∈ Xmax (y, x ) ∈ τ .f (x )=0 (by def. of Gc and Accc) . Proof. We first prove (1). We define f : X →{0, 1} by ′ ′ ′ This means that if f (x )=0, then c2(x )=0 and for each 1 (x ∈ ReachGc,Accc ) ′ ′′ ′ ′′ f(x)= A ∈ c1(x ) there exists x ∈ A such that f (x )=0. Hence (0 (otherwise) . for each x ∈ X such that f ′(x)=0, we can define a strategy By definition of the least fixed-point property, it suffices to βx of the player min so that for each strategy α of the player max, the resulting run ρα,βx,x from x is not winning for max. show that f is the least fixed point of the function Φc,σ : g Therefore by the definition of f, we have f(x)=0. This {0, 1}X →{0, 1}X in Def. III.2. concludes the proof. We first show that f is a fixed point of Φ . For each c,σg The item (2) is proved in a similar way. x ∈ X, we have: 1) Details for Example III.9: Φc,σg (f)(x)=1 Proposition A.3. The modality σg : Fg{0, 1}→{0, 1} in ⇔ (σg ◦ Fgf ◦ c)(x)=1 (by def. of Φc,σg ) ′ ′ Def. A.1 satisfies Asm. III.7. ⇔ c2(x)=1, or ∃A ∈ c1(x). ∀x ∈ A. f(x )=1 Proof. It is easy to see that Cond. 1 is satisfied. By mono- (by def. of σg and Fg) tonicity of the functions max and min that are used in σg, ⇔ c2(x)=1, or Cond. 2 is satisfied. ∃A ∈ c (x). ∀x′ ∈ A. 1 2) Details of Example III.15: ∃α′ : a strategy of max. ∀β′ : a strategy of min. ′ ′ ′ Proposition A.4. We define an F -modality σ as in Def. A.1, ρα ,β ,x is winning for max (by def. of f) g g and fix an ordinal z. We define an Fg-algebra rg,z : ⇔ c2(x)=1, or FgOrd≤z → Ord≤z, a function qg,z : FgOrd≤z → {0, 1} Ord ∃A ∈ c1(x). and a partial order ⊑Ord over ≤z as follows. ′ 1) Ord = {a | a is an ordinal s.t. a ≤ z}, and r : ∃(αx′ )x′∈A : a family of strategies of max. ≤z g,z F Ord → Ord is defined by ∀x′ ∈ A. ∀β′ : a strategy of min. g ≤z ≤z α′ ,β′,x′ ρ x′ is winning for max 0 (t = 1) rg,z(Γ,t)= c ⇔ x ∈ Acc , or (minA∈Γ supa∈A(a +1) (otherwise) . c c ∃y ∈ Xmin s.t. (x, y) ∈ τ . Here a +1 denotes min{a +1, z}. ′ b ′ c ′ c 2) q : Ord →{0, 1} is defined by ∃(αx′ )x ∈Xmax s.t. (y,x )∈τ : a family of strategies of max. g,z ≤z ′ c ′ c ′ b ∀x ∈ Xmax s.t. (y, x ) ∈ τ . ∀β : a strategy of min. 0 (a = z) ′ ′ ′ a α ′ ,β ,x qg,z( )= ρ x is winning for max (1 (otherwise) .

14 def. 3) For a, b ∈ Ord≤z, a ⊑Ord b ⇔ a ≥ b (note the a +1= a +1. Hence we have: directions). f1b(x)= a Then the triple (rg,z, qg,z, ⊑Ord) is a ranking domain. ⇔ rg,z ◦ Fgf1 ◦ c(x)= a (f1 is a fixed point of Φc,rg,z ) The most difficult part of the proof is to prove that r is ′ ⇔ c2(x)=0 and min sup f1(x )= a − 1 a corecursive algebra (Cond. 4 in Def. III.10). We prove it A∈c1(x) x′∈A separately. To this end, we first prove the following sublemma. (by def. of rg,z and Fg) ′ Lemma A.5. The algebra Ord Ord satisfies ⇔ c2(x)=0 and min sup f2(x )= a − 1 (by IH) rg,z : Fg ≤z → ≤z A∈c1(x) x′∈A Cond. 2 in Def. III.10. ⇔ Φc,rg,z (f2)(x)= a (by def. of Φc,rg,z )

Proof. It is easy to see that Cond. 2a is satisfied. ⇔ f2(x)= a (f2 is a fixed point of Φc,rg,z ) . We prove that Cond. 2b is satisfied. Let b1,b2 : X → Ord≤z and assume that b1 ⊑Ord b2. Let x ∈ X. As 0 is Hence we have f1(x)= a iff f2(x)= a for each x ∈ X and Ord a < z. This immediately implies that f (x)= z iff f (x)= z the greatest element in ( ≤z, ⊑Ord), if Φc,rg,z (b2)(x)=0 1 2 for each x ∈ X. then we have Φc,rg,z (b1)(x) ⊑Ord Φc,rg,z (b2)(x). Ord Assume that Φc,rg,z (b2)(x) = n ∈ ≤z \{0}. Let 2 Proof (Prop. A.4). We prove that the conditions in Def. III.10 c(x) = (Γ,t) ∈ FgX = P X ×{0, 1}. Then by definition are satisfied. of rg,z and Fg, t =0 and moreover for all A ∈ Γ there exists Ord 2 x ∈ A such that b2(x) ≥ n. As b1 ⊑Ord b2, b2(x) ≥ n Cond. 1: Let (Γ,t) ∈ Fg ≤z = P rg,z ×{0, 1} and implies b1(x) ≥ n. Hence we have rg,z ◦ Fgb1(Γ,t) ≥ n, assume that σg ◦ Fgqg,z(Γ,t)=0. Then by the definitions of σg, qg,z and Fg, we have t =0 and ∀A ∈ Γ. z ∈ A. Hence by and this implies Φc,rg,z (b1)(x) ⊑Ord Φc,rg,z (b2)(x). Therefore Cond. 2b is satisfied. the definition of rg,z, we have rg,z(Γ,t)= z. Therefore by the definition of qg,z, we have qg,z ◦rg,z(Γ,t)=0. Hence Cond. 1 is satisfied. Ord Ord Lemma A.6. The algebra rg,z : Fg ≤z → ≤z is Cond. 2: Already proved in Lem. A.5 corecursive. Cond. 3: By the definition of qg,z : R →{0, 1}, for a1,a2 ∈ Ord , implies (here each Proof. Let c : X → FgX be an Fg-coalgebra. It suffices to ≤z a1 ≤ a2 qg,z(a1) ≥ qg,z(a2) ≤ denotes the usual order). Therefore qg,z is monotone. show that Φc,rg,z (Def. III.2) has a unique fixed point. X By its definition, qg,z(z)=0 and hence qg,z is strict. By Lem. A.5, the poset (Ord≤z , ⊑Ord) (here ⊑Ord For a subset K ⊆ Ord z, it is easy to see that denotes the pointwise extension of ⊑Ord over Ord≤z) is ≤ a complete lattice. Therefore we can construct a transfinite a∈K qg,z(a) ≤ qg,z a∈K a is satisfied. We prove the opposite direction. Assume that qg,z a = 1. Then sequence F  F  a∈K by the definition of qg,z, a = n for some n < z. a∈K F  By the definition of ⊑Ord, this implies a = n for some Ord Ord Ord Ord F ⊥ ≤z ⊑ Φc,rg,z (⊥ ≤z ) ⊑ ··· a ∈ K. Therefore we have qg,z(a) =1, and this implies a Ord Ord Ord . Hence ⊑ Φc,rg,z (⊥ ≤z ) ⊑ ··· a∈K qg,z(a) = 1 qg,z a∈K a ≤ a∈K qg,z(a) holds, and qg,z is continuous. F  F  F  as in Thm. II.18.2. By Thm. II.18.2, there exists an ordinal n Cond. 4: Already proved in Lem. A.6. n Ord Hence (r , q , ⊑Ord) is a ranking domain. such that Φc,rg,z (⊥ ≤z ) is the least fixed point of Φc,rg,z . g,z g,z It remains to show that this is the unique fixed point. Let Ord f1,f2 : X → Ord≤z be fixed points of Φc,r . We prove Proposition A.7. Let (rg,z, qg,z, ⊑ ) be the ranking domain g,z Ord f1(x) = a ⇔ f2(x) = a for each x ∈ X and a < z by in Prop. A.4. For each function b : X → ≤z, we have the transfinite induction on a. followings.

For the base case, we have: 1) Let c : X → FgX be an Fg-coalgebra. A function b : X → Ord≤z is a ranking arrow (Def. III.11) for c wrt. Ord if and only if is a ranking function f1(x)=0 (rg,z, qg,z, ⊑ ) b (Def. II.4) for Gc and Accc. ⇔ rg,z ◦ Fgf1 ◦ c(x)=0 (f1 is a fixed point of Φc,rg,z ) 2) Let G = (Xmax, xmin, τ) be a game structure and ⇔ c (x)=1 (by def. of rg,z and Fg) 2 Acc ⊆ Xmax be a set of accepting states. A function ⇔ rg,z ◦ Fgf2 ◦ c(x)=0 (by def. of Fg and rg,z) b : Xmax → Ord≤z is a ranking function for G and Acc if and only if b is a ranking arrow for cG,Acc wrt. ⇔ f2(x)=0 (f2 is a fixed point of Φc,rg,z ) . (rg,z, qg,z, ⊑Ord). 3) b(x) < z iff q ◦ b(x)=1. Let 0 < a < z and assume f(x) = a′ iff g(x) = a′ for g,z each x ∈ X and a′ < a. Note here that by a < z, we have Proof.

15 1: For x ∈ X such that c(x) = (Γ,t) we write c1(x) and D. Proof of Prop. IV.2 c2(x) for Γ and t respectively. We have: Proof. Recall that ReachAccc is defined as the function f : X → [0, 1] in Def. II.9. We first show that this f is a fixed b : X → Ord≤z is a ranking arrow for c point of Φc,σp . For x ∈ X, we have: ⇔ ∀x ∈ X. b(x) ⊑Ord rg,z ◦ Fgb ◦ c(x) (by Def. III.11) Φc,σp (f)(x) ⇔ ∀x ∈ X. rg,z ◦ Fgb ◦ c(x) ≤ b(x) = (σp ◦ Fpf ◦ c)(x) (by def. of Φc,σ ) (by the definition of ⊑Ord) p ′ 1 (c2(x)=1) ⇔ ∀x ∈ X. c2(x)=0 ⇒ min sup b(x ) +1 ≤ b(x) A∈c1(x) x′∈A ′ ′ =  c1(x)(x ) · f(x ) (c2(x)=0) (by the definitions of and  )  ′ b rg,z Fg x ∈supp(c1(x)) c X ⇔ ∀x ∈ Xmax. (by def. of σ and F )  p p x∈ / Accc ⇒ min sup b(x′) +1 ≤ b(x) c 1 (c (x) =1) y:(x,y)∈τ x′:(y,x′)∈τ c 2 ′ ′ c  c =  c1(x)(x ) · lim fn(x ) (c2(x) =0) (by the definitions ofbG and Acc ) n→∞  ′ c c x ∈supp(c1(x)) ⇔ b : X → Ord≤z is a ranking function for G and Acc X (by def. of f) (by Def. II.4) .  1 (c2(x) =1) ′ ′ Hence Cond. 1 is satisfied. =  lim c1(x)(x ) · fn(x ) (c2(x) =0) n→∞ 2: This is proved in a similar manner to Cond. 1.  ′ 1  x ∈supp(Xc (x)) 3: Immediate from the definition of qg,z. 1 (c (x)=1)  2 = (by def. of fn) lim fn+1(x) (c2(x)=0) (n→∞ = lim fn(x) (by def. of fn) n B. Proof of Lem.III.13 →∞ = f(x) (by def. of f) . The basic idea of the proof is similar to the proof of Hence f is a fixed point of Φ . Thm. II.18.2 in [29]. c,σp It remains to show that f is the least fixed point. Let f ′ : X → [0, 1] be a fixed point of Φc,σ . a p Proof. Let m be an ordinal such that |L| < |m|. As |{f (l) ∈ We prove f(x) ≤ f ′(x) for each x ∈ X. To this end, by a m , there exist ordinals a a′ such that a ′ L | ≤ }| ≤ |L| , f (l)= the definition of f, it suffices to prove fn(x) ≤ f (x) for a′ ′ f (l). Without loss of generality, we assume a < a . each x ∈ X and n ∈ N. We prove this by induction on n. By monotonicity of f and that l ⊑ f(l), we have For n =0, it is immediate from that f0(x)=0.

′ For n> 0, we have: f a(l)= f a+1(l)= ··· = f a (l) . fn(x) This concludes the proof. 1 (c2(x)=1) ′ ′ =  c1(x)(x ) · fn−1(x ) (c2(x)=0)  ′ x ∈supp(Xc1(x)) (by def. of f ) C. Proof of Prop.III.18  n 1 (c2(x)=1) Proof. By (|c|) = r◦F (|c|) ◦c F µσ r r J Kc ′ ′ ′ (by IH) ≤  c1(x)(x ) · f (x ) (c2(x)=0) and q ◦ r = σ ◦ F q, we have: FX / F R /+ F Ω O F q x′∈supp(c1(x)) F (|c|)r X c = r = σ ′ q ◦ (|c|)r =Φc,σ(q ◦ (|c|)r) . = (σp ◦ Fpf ◦ c)(x) (by def. of Fp and σp) (|c|)r  q   X / R 4/ Ω = f ′(x) (f ′ is a fixed point) . This means that q ◦ (|c|)r is a JµσKc ′ N fixed point of Φc,σ. As µσ c Hence we have fn(x) ≤ f (x) for each x ∈ X and n ∈ , ′ is the least fixed point ofJ Φc,σK (Def. III.5), we have and this implies f(x) ≤ f (x) for each x ∈ X. Therefore f is the least fixed point of Φc,σp .

q ◦ (|c|)r ⊒Ω µσ c . J K E. Proof of Prop. IV.3 Together with Thm. III.12, we have q ◦ (|c|)r = µσ c. J K Proof.

16 ′ ′ ′ 1: For each b : X → [0, ∞], we have: = rp,ε ◦ Fpb2(ϕ, t) (by def. of rp,ε)

′ =Φc,r′ (b )(x) (by def. of Φc,r′ ) . b is an ε-additive ranking supermartingale p,ε 2 p,ε ⇔ ∀x ∈ X. Therefore Cond. 2b is satisfied. ′ Cond. 3: By the definition of q , for each a1,a2 ∈ [0, ∞], c2(x)=0 p a ≥ a implies q′ (a ) ≤ q′ (a ) (here each ≤ denotes the ′ ′ ′ ′ 1 2 p 1 p 2 ⇒ c1(x)(x ) · b (x )+ ε ≤ b (x) ′ standard order). Hence qp is monotone. x′∈supp(c1(x)) ′ ′ X  By the definition, we have qp(∞)=0. Hence qp is strict. ′ (by Def. II.11) We prove that qp is continuous. Namely, for each subset ′ ′ ′ ′ K ⊆ [0, ∞], we prove ⇔ ∀x ∈ X. rp,ε ◦ Fpb ◦ c(x) ≤ b (x) (by def. of rp,ε) ⇔ b′ is a (categorical) ranking arrow wrt. (r′ , q′ , ⊑ ) ′ ′ p,ε p [0,∞] qp(a) = qp a . (by Def. III.11) . a∈K a∈K G  G  It is easy to prove . We prove the opposite Hence Cond. 1 holds. (LHS) ≤ (RHS) direction. To this end, by the definition of q′ , it suffices to 2: Immediate from the definition of q′ . p p prove that if the right-hand side is 1 then the left-hand side is also 1. We can prove it as follows (here denotes the F. Proof of Prop. IV.4 supremum with respect to ⊑[0,∞], and denotes the infimum with respect to the ordinary order over [0, 1]). F Proof. V ′ ′ Cond. 1: Let (ψ,t) ∈ Fp[0, ∞] = D[0, ∞] ×{0, 1}. Then qp a =1 ⇒ a< ∞ (by def. of qp) we have: a∈K a∈K G  G ′ ′ ⇒ a< ∞ (by def. of ⊑[0,∞]) qp ◦ rp,ε(ψ,t) a∈K q′ (0) (t = 1) ^ = p ⇒ ∃a ∈ K.a< ∞ q′ ( a · ψ(a)+ ε) (t = 0) ′ ′ ( p a∈supp(ψ) ⇒ ∃a ∈ K. qp(a)=1 (by def. of qp) ′ (by def. of rp,ε) ′ P ⇒ qp(a) =1 1 (t =1 or a · ψ(a)+ ε< ∞) a∈K = a∈supp(ψ) G  otherwise Hence q′ is continuous. (0 ( P) p ′ This concludes the proof. (by def. of qp) 1 (t =1 or ψ([0, ∞)) = 1) ≤ G. Multiplicative Ranking Supermartingale, Categorically (0 (otherwise) Definition A.8. Let M = (X, τ) be a PTS. Let α ∈ (0, 1). 1 (t = 1) ≤ A function b : X → [0, ∞] is an α-multiplicative ranking (ψ([0, ∞)) (otherwise) supermartingale if we have ′ ′ 1 (t = 1) ′ τ(x)(x ) · b(x ) ≤ α · b(x) = (by def. of q′ ) x ∈supp(τ(x)) 1 · Dq (ψ)(1) (otherwise) p ( p for each xP∈ X \ Acc, and moreover there exists δ > 0 such ′ = σp ◦ Fpqp(ψ,t) (by def. of σp) . that b(x) ≥ δ for each x ∈ X \ Acc.

′ ′ ′ Hence we have qp ◦ rp,ε ≤ σp ◦ Fpqp. The following is an attempt to define a ranking domain Cond. 2: It is easy to see that Cond. 2a is satisfied. for α-multiplicative supermartingales. Let us fix real numbers ′ We prove that Cond. 2b is satisfied. Let b1,b2 : X → [0, ∞] α ∈ (0, 1) and δ > 0, and define an Fp-algebra rpm,α : ′ and assume that b1 ⊑[0,∞] b2. For each x ∈ X such that Fp[αδ, ∞] → [αδ, ∞], an arrow qp : [0, ∞] → [0, 1] and a c(x) = (ϕ, t) ∈ FpX = DX ×{0, 1}, we have: partial order ⊑[αδ,∞] over [αδ, ∞] as follows. 1) For each (ψ,t) ∈ Fp[αδ, ∞]= D[αδ, ∞] ×{0, 1}, ′ Φc,rp,ε (b1)(x) ′ ′ ′ r (ψ,t)= = rp,ε ◦ Fpb1(ϕ, t) (by def. of Φc,rp,ε ) pm,α 1 (t = 1) αδ (t = 1) = 1 · a · ψ(a) (otherwise) . ( x∈supp(ϕ) ϕ(x) · b1(x)+ ε (t = 0) ( α a∈supp(ψ) ′ ′ ′ P (by def. of rp,ε) 2) qp(∞)=0 and Pqp(a)=1 if a< ∞. def. 1 (t = 1) 3) a ⊑ αδ, b ⇔ a ≥ b (note the direction). ≤ [ ∞] The following proposition is analogous to Prop. IV.3. ( x∈supp(ϕ) ϕ(x) · b2(x)+ ε (t = 1) P 17 Proposition A.9. In this setting, for each c : X → FpX, we Cond. 1: Let (ψ,t) ∈ Fp[αδ, ∞]= D[αδ, ∞]×{0, 1}. Then have the following. we have: ′ ′ ′ a) Let b : X → [αδ, ∞] and assume b ⊑[αδ,∞] rpm,α ◦ ′ ′ ′ ′ Fpb ◦ c. If we define b : X → [0, ∞] by b(x) = b (x), qp ◦ rpm,α(ψ,t) then b is an α-multiplicative ranking supermartingale. ′ qp(αδ) (t = 1) b) Let b : X → [0, ∞] be an α-multiplicative ranking = q′ ( 1 · a · ψ(a)) (t = 0) supermartingale such that b(x) ≥ δ if x ∈ X \ Acc. If ( p α a∈supp(ψ) ′ ′ ′ we define b : X → [αδ, ∞] by b (x) = max{αδ, b(x)}, P (by def. of rpm,α) ′ ′ ′ ′ then b satisfies b ⊑[αδ,∞] rpm,α ◦ Fpb ◦ c. 1 1 (t =1 or · a∈supp(ψ) a · ψ(a) < ∞) c) For each b′ : X → [0, ∞], we have b′(x) 6= ∞ iff = α ′ ′ (0 (otherwise) qp ◦ b (x)=1. P (by def. of q′ ) Proof. p 1 (t =1 or ψ([0, ∞)) = 1) a): Let x ∈ X \ Acc. Then we have: ≤ (0 (otherwise) τ(x)(x′) · b(x′) ′ 1 (t = 1) x ∈supp(Xτ(x)) ≤ ′ ′ ′ (ψ([0, ∞)) (otherwise) = c1(x)(x ) · b (x ) x′∈supp(c1(x)) 1 (t = 1) X ′ ′ ′ = (by def. of qp) = α · (rpm,α ◦ Fpb ◦ c)(x) (1 · Dqp(ψ)(1) (otherwise) ′ ′ ≤ α · b (x) = σp ◦ Fpqp(ψ,t) (by def. of σp) . = α · b(x) . ′ ′ ′ Hence we have qp ◦ rpm,α ≤ σp ◦ Fpqp. By the inequality above, we also have Cond. 2: It is easy to see that Cond. 2a is satisfied.

1 ′ ′ ′ We prove that Cond. 2b is satisfied. Let b1,b2 : X → b(x) ≥ c1(x)(x ) · b (x ) . α [αδ, ∞] and assume that b1 ⊑[αδ,∞] b2. For each x ∈ X such x′∈supp(c1(x)) X that c(x) = (ϕ, t) ∈ FpX = DX ×{0, 1}, we have: As b′(x′) ≥ αδ holds for each x′ ∈ X, we have b(x) ≥ δ. ′ Therefore b is an α-multiplicative ranking supermartingale. Φc,rpm,α (b1)(x) ′ b): For each x ∈ X, we have: ′ = rpm,α ◦ Fpb1(ϕ, t) (by def. of Φc,rpm,α ) ′ ′ (rpm,α ◦ Fpb ◦ c)(x) 1 (t = 1) = 1 αδ (c (x) =1) ( α · x∈supp(ϕ) ϕ(x) · b1(x) (t = 0) = 2 1 ′ ′ ′ (by def. of r′ ) ( α x′∈supp(c1(x)) c1(x)(x ) · b (x ) (c2(x) =0) P pm,α αδP (x ∈ Acc) 1 (t = 1) = ⊑[αδ,∞] 1 1 ′ ′ · ϕ(x) · b2(x) (t = 0) ( α x′∈supp(τ(x)) τ(x)(x ) · b(x ) (x ∈ X \ Acc) ( α x∈supp(ϕ) = r′ ◦ F b (ϕ, t) (by def. of r′ ) αδP (x ∈ Acc) pm,α p 2P pm,α ≤ ′ ′ 1 =Φc,rpm,α (b2)(x) (by def. of Φc,rpm,α ) . ( α · αb(x) (x ∈ X \ Acc) ′ ≤ b (x) . Therefore Cond. 2b is satisfied. ′ Cond. 3: It is proved in a similar manner to the proof of Therefore by the definition of ⊑[αδ,∞], we have b ⊑[αδ,∞] ′ ′ Prop. IV.4. rpm,α ◦ Fpb ◦ c. ′ c): Immediate from the definition of qp.

′ ′ Example A.11. We define a coalgebra c : X → FpX as Therefore the triple (rpm,α, qp, ⊑[αδ,∞]) is suited for accom- modating α-multiplicative supermartingales in our categorical in Example IV.5. We fix α ∈ (0, 1) and δ > 0. We assume framework. However it is not a ranking domain. The following that α > 1/2. For this coalgebra, we define arrows b1,b2 : 1 1 αδ αδ proposition is analogous to Prop. IV.4. X → [αδ, ∞] by: b1(x0) = 2 α 2α−1 + δ , b1(x1) = 2α−1 , b1(x2) = δ and b1(x3) = αδ; and b2(x0) = ∞, b2(x1) = Proposition A.10. The triple (r′ , q′ , ⊑ ) intro-  pm,α p [0,∞] ∞, b2(x2) = δ and b2(x3) = αδ. Both of these qualify as duced above satisfies the conditions of a ranking domain ′ coalgebra-algebra homomorphisms from c to rpm,α. Therefore (Def. III.10), except for Cond. 4. ′ rpm,α is not corecursive. Proof.

18 H. Proof of Prop. IV.7 We prove that Cond. 2b is satisfied. Let f1,f2 : X → DN∞ N We prove Prop. IV.7 in a similar manner to the proof of and assume f1 ⊑D ∞ f2. Let x ∈ X and assume that c(x)= . Prop. A.4: we first prove that rp is a corecursive algebra (ϕ, t) ∈ FpX separately. To this end, we first prove some lemmas. If t =1 then we have:

Lemma A.12. The order ⊑DN∞ in Prop. IV.7 is a partial rp ◦ Fpf1(f)(ϕ, t)= rp ◦ Fpf2(f)(ϕ, t)= δ0. order and DN∞ is a complete lattice with respect to this order. Therefore we have Φc,σp (f1)(x)=Φc,σp (f2)(x). Sublemma A.13. For every nondecreasing function G : N → Let t =0 and a ∈ N. If a =0, by the definition of rp, we [0, 1], there exists a unique distribution ϕ over N∞ such that have: ϕ([0,a]) = G(a) for each a ∈ N. rp ◦ Fpf1(f)(ϕ, t)([0,a]) = rp ◦ Fpf2(f)(ϕ, t)([0,a]) = 0 . Proof. We define a distribution ϕ over N∞ by If a> 0, we have: G(a) (a = 0) rp ◦ Fpf1(f)(ϕ, t)([0,a]) ϕ(a)= G(a) − G(a − 1) (0

N Proof (Lem. A.12). We first prove that ⊑D ∞ is a partial Proof. Let c : X → FpX be an Fp-coalgebra. It suffices order. Reflexivity and transitivity are immediate from those of X X to show that the function Φc,r : (DN∞) → (DN∞) ′ p N the standard order ≤ over [0, 1]. Assume that ϕ ⊑D ∞ ϕ and (Def. III.2) has a unique fixed point. ϕ′ ⊑ N ϕ. By the definition of ⊑ N , we have ϕ([0,a]) = X D ∞ D ∞ By Lem. A.14, the poset ((DN∞) , ⊑DN ) (here ⊑DN ′ ∞ ∞ ϕ ([0,a]) for each a ∈ [0, ∞]. Then by Sublem. A.13, we have denotes the pointwise extension) is a complete lattice. There- ′ ϕ = ϕ . Hence antisymmetry is also satisfied. fore we can construct a transfinite sequence We prove that each subset K ⊆ DN∞ has the least upper bound. We define G : N → [0, 1] by G(a) = sup ϕ([0,a]). ϕ∈K ⊥DN∞ ⊑DN∞ Φc,rp (⊥DN∞ ) ⊑DN∞ ··· Note that for each ϕ ∈ K, a ≤ b implies ϕ([0,a]) ≤ ϕ([0,b]). a ⊑DN Φ (⊥DN ) ⊑DN ··· Hence by the monotonicity of supremums, G is nondecreasing. ∞ c,rp ∞ ∞ Therefore by Sublem. A.13, there exists a unique distribution as in Thm. II.18.2. By Thm. II.18.2, there exists an ordinal n K K n N N ϕ ∈ D ∞ such that ϕ ([0,a]) = G(a) for each a ∈ [0, ∞]. such that Φc,rp (⊥D ∞ ) is the least fixed point of Φc,rp . We prove that ϕK is the least upper bound of K. It remains to show that this is a unique fixed point. Let N N Let ϕ ∈ K. For each a ∈ , we have: f1,f2 : X → D ∞ be fixed points of Φc,rp . We prove ′ K ϕ([0,a]) ≤ sup ϕ ([0,a]) = ϕ ([0,a]) . f1(x)(a)= f2(x)(a) (10) ϕ′∈K for each x ∈ X and a ∈ N. For each x ∈ X such that Hence by the definition of ⊑DN∞ , ϕ is an upper bound of K. Let ϕ′ be an upper bound of K. Then by the definition of (ϕ, t) = c(x), we write c1(x) and c2(x) for ϕ ∈ DX and ′ t ∈{0, 1} respectively. ⊑DN∞ , we have ϕ([0,a]) ≤ ϕ ([0,a]) for each ϕ ∈ K and a ∈ N. Therefore we have ϕK ([0,a]) ≤ ϕ′([0,a]), and this We first prove (10) for each x ∈ X such that c2(x)=1. K ′ For such x, we have: means ϕ ⊑DN∞ ϕ by the definition of ⊑DN∞ . Hence ϕ is the least upper bound of K. f1(x)(a)=Φc,rp (f1)(x) (f1 is a fixed point) In a similar manner, we can prove that each K ⊆ DN∞ has the greatest lower bound. = rp ◦ Fpf1 ◦ c(x) = δ0 Lemma A.14. The algebra rp : FpDN∞ → DN∞ in Prop IV.7 satisfies Cond. 2 in Def. III.10. = rp ◦ Fpf2 ◦ c(x) =Φ (f )(x) (f is a fixed point) Proof. It is already proved in Lem. A.12 that Cond. 2a is c,rp 2 2 satisfied. = g(x) .

19 Hence we have f1(x)(a)= f2(x)(a) for each a. = f2(x)([0, ∞)) It remains to prove (10) for each x ∈ X such that c (x)=0. 2 = qp ◦ f2(x) (by def. of qp) . We prove it by the induction on a. If a =0 then by the definition of rp, we have f1(x)(a) = Hence we have qp ◦ f1 ≤ qp ◦ f2, and therefore Cond. 3 was f2(x)(a)=0. proved. ′ ′ ′ ′ Let a > 0 and assume f1(x )(a ) = f2(x )(a ) for each By the definition of qp, we have qp(δ∞)=0. Therefore qp x′ ∈ X and a′

qp ◦ f1(x)= f1(x)([0, ∞)) (by def. of qp) pp(γ) = ε · a · γ(a) . = lim f1(x)([0,a]) a∈supp(γ) a→∞ X Then the following statements hold. ≤ lim f2(x)([0,a]) (by f1 ⊑DN f2) a→∞ ∞

20 ′ 1) qp ◦ pp ≤ qp (here ≤ denotes the pointwise extension 2) For each Fp-coalgebra c : X → FpX, the function ([0,∞]) X X N ′ over D ∞ ). Φc,rp,ε : [0, ∞] → [0, ∞] in Def. III.5 is monotone ′ ′ ′ ′ with respect to the pointwise extension of ⊑ . 2) Let b : X → [0, ∞] and assume b ⊑[0,∞] Φc,rp,ε (b ). [0,∞] N N As rp : FpD ∞ → D ∞ is corecursive (Prop. IV.7), Moreover the triple (rp : FpDN∞ → DN∞,pp : DN∞ → N there exists a unique function (|c|)rp : X → D ∞ such [0, ∞], ⊑DN∞ ) (see Prop. IV.7 and Lem. A.17) satisfies the

that (|c|)rp = Φc,rp ((|c|)rp ). For this function, we have following conditions. b′ ⊑ p ◦ (|c|) . ′ [0,∞] p rp 3) We have pp ◦ rp ⊒[0,∞] rp,ε ◦ Fppp between arrows F Jµσ K N F b′ F q p p c FpD ∞ → [0, ∞]. p p p ) / N /, /, 4) The following conditions are satisfied by rp: FpX FpD ∞ Fp[0, ∞] ′ Fp[0, 1] Fp(|c|) Fppp F q O rp p p a) the poset (DN∞, ⊑DN ) has the greatest element r ′ σ ∞ c = p rp,ε ⊑ p ′ ⊤ N ; (|c|)rp  pp  qp  D ∞ op X / DN∞ / [0, ∞] / [0, 1] N N 2 52 b) the poset (D ∞, ⊑D ∞ ) is ω -complete (i.e. for b′ qp N N JµσpKc each decreasing sequence ϕ0 ⊒D ∞ ϕ1 ⊒D ∞ in N , the infimum exists); ··· D ∞ di∈ω ϕi Sublemma A.18. For the function pp : DN∞ → [0, ∞] in c) for each c : X → FpX, the function Φc,rp : Lem. A.17, we have the following. X X (DN∞) → (DN∞) (Def. III.2) is monotone ′ and moreover ωop-continuous, i.e. for each de- pp ◦ rp = rp,ε ◦ Fppp (11) creasing sequence f0 ⊒DN f1 ⊒DN ··· in 2 ∞ ∞ Proof. Let (Γ,t) ∈ FpDN = D N ×{0, 1}. If t =1, then X ∞ ∞ (DN∞) wrt. the pointwise extension of ⊑DN , ′ ∞ by the definitions of rp and rp,ε, we have we have . Φc,rp di∈ω fi = di∈ω Φc,rp (fi) ′ 5) The function p : DN → [0, ∞] is monotone (i.e. pp ◦ rp(Γ,t) = rp,ε ◦ Fppp(Γ,t)=0 . p ∞  a ⊑DN∞ b ⇒ pp(a) ⊑[0,∞] pp(b)), top-preserving (i.e. Let t =0. Then we have: op pp(⊤DN∞ ) = ⊤[0,∞]) and ω -continuous (i.e. each decreasing sequence N N in N , pp ◦ rp(Γ,t) ϕ0 ⊒D ∞ ϕ1 ⊒D ∞ ··· D ∞ we have ). Note that in the pp(di∈ω ϕi)= di∈ω pp(ϕi) = ε · a · rp(Γ,t)(a) (by def. of pp) last equality, the infimum in the left-hand side is take a∈N∞\{0} X wrt. ⊑DN∞ while the one in the right-hand side is taken = ε · a · Γ(γ) · γ(a − 1) (by def. of rp) wrt. ⊑[0,∞]. N 6) The algebra r : F DN → DN is corecursive. a∈ X∞\{0} γ∈supp(Γ)X p p ∞ ∞

= ε · (a + 1) · Γ(γ) · γ(a) ′ Fpb N a∈ ∞ γ∈supp(Γ) , X X FpX / FpDN∞ / Fp[0, ∞] = ε · a · Γ(γ) · γ(a) O Fpb Fppp ′ c rp ⊒ r a∈N∞ γ∈supp(Γ) p,ε X X   b N pp + ε · Γ(γ) · γ(a) X / D ∞ /2 [0, ∞] a N ′ X∈ ∞ γ∈supp(Γ)X b = a · ε · Γ(γ) · γ(a) + ε Proof. N 1: Easy. a∈ ∞ γ∈supp(Γ)  X X 2: Already proved in Prop. IV.4. = Γ(γ) · ε · a · γ(a) + ε 3: Immediate from Sublem. A.18. γ∈supp(Γ) a∈N∞ X  X  4: It is easy to see that the Dirac distribution δ0 con- N = Γ(γ) · pp(γ) + ε (by def. of pp) centrated at 0 is the greatest element in (D ∞, ⊑DN∞ ). γ∈supp(Γ) Hence Cond. 4a is satisfied. Cond. 4b is immediate from that X   X ((DN ) , ⊑ N ) is a complete lattice (Lem. A.12). = b · Γ(γ′) + ε ∞ D ∞ We prove that Cond. 4c is satisfied. Monotonicity of Φc,rp ′ −1 b∈supp(Dpp(Γ)) γ ∈pp (b) op X X  is already proved in Prop. IV.7. We prove that Φc,rp is ω - = b · Dpp(Γ)(b) + ε continuous. To this end, it suffices to prove the following equality for each x ∈ X. b∈supp(XDpp(Γ)) ′  ′ = rp,ε ◦ Fppp(Γ,t) (by def. of rp,ε) . Φc,rp l fi (x) = l Φc,rp (fi)(x) (12) i∈ω i∈ω This concludes the proof.   ′ Note that on the right-hand side denotes the infimum with Sublemma A.19. The Fp-algebra r : Fp[0, ∞] → [0, ∞] in d p,ε respect to ⊑ N while that on the left-hand side denotes the Prop. IV.3 satisfies the following conditions. D ∞ infimum with respect to its pointwise extension. Let (δ, t) = 1) The poset ([0, ∞], ⊑[0,∞]) is a complete lattice. c(x).

21 If , then by the definition of , we have Assume . Then for all N, t =1 rp Φc,rp (f)(x)= di∈ω ϕi (∞) = C > 0 b ∈ δ for each f : X → DN . Hence we have (12). C 0 ∞ there existsib ∈ ω such that 1 − ϕi([0,b]) ≥ (c.f. the proof N  2 If t =0, for each a ∈ , we have: of Lem. A.12). Therefore for each b ∈ N, we have:

l Φc,rp (fi)(x) ([0,a]) l pp(ϕi) ≥ pp(ϕib ) i∈ω   i∈ω = inf Φc,rp (fi)(x)([0,a]) (see the proof of Lem. A.12) ∞ i∈ω = ε · a · ϕi(a) (by def. of pp)  = inf δ(x) · fi(x)([0,a − 1]) (by def. of rp) a=0 i∈ω X x∈supp(δ) X  ≥ ε · b · (1 − ϕ([0,b])) = δ(x) · inf fi(x)([0,a − 1]) C i∈ω ≥ ε · b · (1 − )) . x∈supp(δ) 2 X  (by the dominated convergence theorem) Hence we have = δ(x) · l fi (x)([0,a − 1]) pp ϕi = ∞ = pp(ϕi) . x∈supp(δ) i∈ω di∈ω di∈ω X  (see the proof of Lem. A.12)   Assume di∈ω ϕi (∞)=0. Then we have: =Φ f (x)([0,a]) (by def. of Φ ) . c,rp l i c,rp   i∈ω   pp l ϕi By Sublem. A.13, this implies (12). i∈ω   5: We first prove that pp : DN∞ → [0, ∞] is monotone. ∞ N by def. of Let ϕ1, ϕ2 ∈ D ∞ and assume that ϕ1 ⊑DN∞ ϕ2. = ε · a · l ϕi (a) ( pp) a=0 i∈ω If ϕ2(∞) = C > 0, then we have ϕ2([0,a]) ≤ 1 − C X   N ∞ for each a ∈ . By the definition of ⊑DN∞ , this implies = ε · 1 − ϕi ([0,a − 1]) ϕ1([0,a]) ≤ 1−C for each a, and therefore we have ϕ1(∞) > l a=1 i∈ω 0. Hence in this case, we have pp(ϕ1)= pp(ϕ2)= ∞. X   (as in the proof of monotonicity of pp) Let ϕ2(∞)=0. Then we have: ∞

pp(ϕ2) = ε · 1 − inf ϕi([0,a − 1]) i∈ω ∞ a=1 X  = ε · a · ϕ2(a) (by def. of pp) (see the proof of Lem. A.12) a=1 ∞ X n = sup ε · 1 − ϕi([0,a − 1]) i ω = lim ε · a · ϕ2([0,a]) − a · ϕ2([0,a − 1]) ∈ a=1 n→∞ X  a=1 = sup p (ϕ ) (as in the above) X n  p i i∈ω = lim ε · n · ϕ2([0,n]) − 1 · ϕ2([0,a − 1]) n→∞ = pp(ϕi) (by def. of ⊑[0,∞]) a=1 l n X  i∈ω

= lim ε · ϕ2([a,n]) n→∞ a=1 ∞ X Hence we have ωop-continuity. = ε · ϕ2([a, ∞]) 6: It is already proved in Lem. A.15. a=1 X ∞ Proof (Lem. A.17). = ε · 1 − ϕ2([0,a − 1]) 1: For each ϕ ∈ DN∞, we have: a=1 X∞  ′ qp ◦ pp(ϕ) ≤ ε · 1 − ϕ1([0,a − 1]) (by ϕ1 ⊑DN ϕ2) ∞ ′ a=1 = qp a · ϕ(a) (by def. of pp) X  as the transformation for a∈N = pp(ϕ1) ( ϕ2) . X  1 ( a∈N a · ϕ(a) < ∞) ′ Hence we have pp(ϕ1) ⊑[0,∞] pp(ϕ2) and therefore pp is = (by def. of q ) 0 (otherwise) p monotone. ( P By the definition of pp, pp(δ0) = 0. Hence pp is top- ≤ ϕ([0, ∞)) preserving. = qp(ϕ) (by def. of qp) . op We prove that pp is ω -continuous. Let ϕ0 ⊒DN∞ N ϕ1 ⊒DN∞ ··· be a decreasing sequence in D ∞. Hence Cond. 1 is satisfied.

22 ′ ′ 2: Lemma A.21. We define triples (rp,ε, qp, ⊑[0,∞]) and ′ Fp Jνr Kc ′ ′ p,ε (rpm,α, qp, ⊑[0,∞]) as in Prop. IV.3 and Prop. A.9 respectively. , ′ FpX / FpDN∞ / Fp[0, ∞] We define a function pp :[αδ, ∞] → [0, ∞] as follows. O Fp(|c|)rp Fpp r ′ c = p ⊒ rp,ε a ′ 1 (|c|)rp  p  pp(a)= ε · log +1 X / DN / [0, ∞] α δ ! ∞ 2 9   ′ Jνrp,εKc Let c : X → FpX be an Fp-coalgebra. Then we have: ′ ′′ ′′ ′′ b ′ ∃b : X → [αδ, ∞].b ⊑[αδ,∞] Φc,rpm,α (b ) By Sublem. A.19.1–2, the F -modality r′ : F [0, ∞] → ′ ′′ ′ ′′ p p,ε p ⇒ pp ◦ b ⊑[0,∞] Φc,r′ (pp ◦ b ) [0, ∞] satisfy dual conditions of Asm. III.7. Hence using the p,ε Fp′ dual statement of Prop. III.8, we can show that Φc,r′ : p p,ε Sublemma A.22. For the function F [αδ, ∞] / F [0, ∞] X X ′ ′ [0, ∞] → [0, ∞] has the greatest fixed point νr : r′ ⊑ r′ p,ε c pp :[αδ, ∞] → [0, ∞] in Lem. A.21, pm,α  p,ε  X → [0, ∞] with respect to the pointwise extensionJ of ⊑ K . ′ ′ ′ ′ [αδ, ∞] / [0, ∞] [0,∞] we have pp ◦rpm,α ⊑[0,∞] rp,ε ◦Fppp . ′ By the Knaster-Tarski theorem, we have pp

′ ′ Proof. Let (ψ,t) ∈ Fp[αδ, ∞] = D[αδ, ∞] ×{0, 1}. By the b ⊑[0,∞] νrp,ε c . (13) ′ ′ J K definition of ⊑[0,∞], it suffices to prove pp ◦ rpm,α(ψ,t) ≥ N ′ ′ By Sublem. A.19.3–6, the triple (rp : FpD ∞ → rp,ε ◦ Fppp(ψ,t). DN∞,pp : DN∞ → [0, ∞], ⊑[0,∞]) satisfies the dual con- If t =1, then we have: ditions of the axioms of a ranking domain (Def. III.10), ′ ′ X pp ◦ rpm,α(ψ,t) except that the length of a transfinite sequence in (DN∞) N ′ by def. of ′ is restricted to ω. Note here that (|c|)rp : X → D ∞ satisfies = pp(αδ) ( rpm,α) N by its definition. Therefore in a ′ (|c|)rp ⊒D ∞ Φc,rp ((|c|)rp ) = 0 (by def. of pp) similar manner to the proof of Thm. III.12, we can prove ′ ′ ′ = rp,ε(Dpp(ψ),t) (by def. of rp,ε) νr′ ⊑ p ◦ (|c|) . (14) ′ ′ p,ε c [0,∞] p rp = rp,ε ◦ Fppp(ψ,t) . J K ′ By (14) and (13), we have b ⊑[0,∞] pp ◦ (|c|)r. Let t =0. Note that a∈supp(ψ) ψ(a)=1. Hence we have: Proof (Prop. A.16). Immediate from Lem. A.17. ′ ′ pp ◦ rpm,α(ψ,t) P We can now prove the soundness of additive ranking su- 1 = p′ · a · ψ(a) (by def. of r′ ) permartingale (Thm. II.5) using our categorical framework as p α pm,α a∈supp(ψ) follows.  X  ′ ′ ′ 1 1 ∃b .b is an ε-additive ranking supermartingale and b (x) 6= ∞ = ε · log 1 · a · ψ(a) +1 α δ α ! Prop. IV.3 ′ ′ ′ ′ ′  a∈supp(ψ)  ⇔ ∃b .b ⊑[0,∞] Φc,r′ (b ) and q ◦ b (x)=1 X  p,ε p (by def. of p′ ) Prop. A.16 p ⇒ ∃b.b is a ranking arrow wrt. (r , q , ⊑ N ), and p p D ∞ a 1 qp ◦ b(x)=1 = ε · log · ψ(a) +2 α δ ! Thm. III.12  a∈supp(X ψ)  ⇒ µσp cG,Acc (x)=1  a Prop. IV.2 J K ≥ ε · (log 1 ) · ψ(a) +2 ⇔ Reach (x)=1 α M,Acc δ ! a∈supp(X ψ)  (log 1 ( ) is an upward convex function) J. Soundness of Multiplicative Ranking Supermartingale, α Categorically a = ε · (log 1 )+1 · ψ(a) + ε N α δ Proposition A.20. We define triples (rp, qp, ⊑D ∞ ) and a∈supp(ψ) ! ′ ′ X   (rpm,α, qp, ⊑[αδ,∞]) as in Prop. IV.7 and Prop. A.9 respec- ′ ′  ′ ′ = rp,ε ◦ Fppp(ϕ, t) (by def. of pp and rp,ε) . tively. Let c : X → FpX be an Fp-coalgebra. Then we have: ′′ ′′ ′′ This concludes the proof. ′ ∃b : X → [αδ, ∞].b ⊑[αδ,∞] Φc,rpm,α (b ) Proof (Lem. A.21). ⇒ ∃b : X → DN∞. ′ b ⊑ N Φ (b), and F b′′ F pp D ∞ c,rp FX / F [αδ, ∞] / F [0, ∞] ′ ′ O ∀x ∈ X. (qp ◦ b (x)=1 ⇒ qp ◦ b(x)=1) . ′ ′ c rpm,α ⊑ r ⊑  p,ε  This proposition is an immediate corollary of Prop. A.16 and / / X ′′ [αδ, ∞] ′ [0, ∞] the following lemma. b pp

23 ′′ ′′ Let b : X → [αδ, ∞] and assume that b ⊑[αδ,∞] is a complete lattice. Therefore we can construct a transfinite ′ ′′ Φc,rpm,α (b ). Then we have: sequence

′ ′′ a pp ◦ b ⊥[0,1] ≤ Φc,rnc,γ (⊥[0,1]) ≤ ··· ≤ Φc,rnc,γ (⊥[0,1]) ≤ · · · ⊑ p′ ◦ r′ ◦ F b′′ ◦ c (by the assumption) [0,∞] p pm,α p as in Thm. II.18.2. By Thm. II.18.2, there exists an ordinal ′ ′ ′′ ⊑[0,∞] rp,ε ◦ Fppp ◦ Fpb ◦ c (by Sublem. A.22) n n such that Φc,rnc,γ (⊥[0,1]) is the least fixed point of Φc,rnc,γ . ′ ′′ n ′ Let . =Φc,rp,ε (pp ◦ b ) (by definition) . f =Φc,rnc,γ (⊥[0,1]) It remains to show that f is the unique fixed point of Φc,r . This concludes the proof. nc,γ Let g : X → [0, 1] be a fixed point of Φc,rnc,γ . As f is the Proof (Prop. A.20). Immediate from Lem. A.21 and Prop. A.16. least fixed point, we have f(x) ≤ g(x) for each x ∈ X. We now define h : X → [0, 1] by h(x) = g(x) − f(x). Then we have: We can now prove the soundness of multiplicative ranking supermartingale using our categorical framework as follows. sup h(x) x∈X ∃b′′.b′′ is an α-multiplicative ranking supermartingale, and = sup g(x) − f(x) (by def. of h) ′′ x∈X b (x) 6= ∞  = sup Φc,rnc,γ (g)(x) − Φc,rnc,γ (f)(x) Prop. A.9 ′ ′ ′ ′ ′ x∈X ⇔ ∃b .b ⊑ Φc,r′ (b ) and q ◦ b (x)=1 [0,∞] p,ε p  Prop. A.16 (f and g are fixed points) N ⇒ ∃b.b is a ranking arrow wrt. (rp, qp, ⊑D ∞ ), and ′ ′ ≤ sup γ · c1(x)(x ) · g(x ) x X qp ◦ b(x)=1 ∈ x′ X X∈ Thm. III.12 ′ ′ ⇒ µσp cG,Acc (x)=1 − γ · c1(x)(x ) · f(x ) Prop. IV.2 J K x′∈X ⇔ ReachM,Acc(x)=1 X  (by def. of rnc,γ and that

c2(x)=1 ⇒ Φc,r(f)(x)=Φc,r(g)(x)=1) K. Proof of Prop. IV.12 ′ ′ ′ = γ · sup c1(x)(x ) · (g(x ) − f(x )) x X We prove that rnc,γ is a corecursive algebra separately. ∈ x′ X X∈ ′ ′ Lemma A.23. The algebra rnc,γ : Fp[0, 1] → [0, 1] in = γ · sup c1(x)(x ) · h(x ) (by def. of h) x∈X Prop IV.12 satisfies Cond. 2 in Def. III.10. x′∈X X ′ ′ Proof. It is easy to see that Cond. 2a is satisfied. ≤ γ · sup sup h(x ) (by c1(x)(x ) ≤ 1) x∈X x′∈X x′∈X We prove that Cond. 2b is satisfied. Let b1,b2 : X → [0, 1] X and assume that b1 ≤ b2. Let x ∈ X and assume. that c(x)= = γ · sup h(x) x∈X (δ, t) ∈ FpX = DX ×{0, 1}. Then we have: As 0 ≤ γ < 1, we have supx∈X h(x)=0, and this implies rnc,γ ◦ Fpb1(δ, t) f = g. This concludes the proof. 1 (t = 1) = Proof (Prop. IV.12). We prove that (rnc,γ, qnc, ≤) satisfies the ( x∈X δ(x) · b1(x) (t = 0) conditions in Def. III.10. Cond. 1: Let (δ, t) ∈ F [0, 1] = D[0, 1] ×{0, 1}. Then we P (by def. of Fp and rnc,γ) p have: 1 (t = 1) = (by b1 ≤ b2) qnc ◦ rnc,γ(δ, t) ( x∈X δ(x) · b2(x) (t = 0)

= rnc,γ ◦ Fpb2(δ, t) (by def. of rnc,γ and Fp) . qnc(1) (t = 1) P = (by def. of rnc,γ) q γ · a · δ(a) (t = 0) Therefore Cond. 2b is satisfied. ( nc a∈supp(δ) 1 P (t= 1) Lemma A.24. The algebra rnc,γ : Fp[0, 1] → [0, 1] in = (by def. of qnc) Prop. IV.12 is corecursive. (γ · a∈supp(δ) a · δ(a) (t = 0) 1P (t = 1) Proof. Let c : X → FpX be an Fp-coalgebra. For each x ∈ X ≤ (by γ < 1) such that c(x) = (δ, t) ∈ FpX, we write c1(x) and c2(x) for ( a∈supp(δ) a · δ(a) (t = 0) δ ∈ D[0, 1] and t ∈{0, 1} respectively. We prove that Φ c,rnc,γ = σpP(δ, t) (by def. of σp) has a unique fixed point. = σp ◦ Fpqnc(δ, t) (by def. of qnc) . We first show that Φc,rnc,γ has a fixed point. By Lem. A.23, the poset ([0, 1]X , ≤) (here ≤ denotes the pointwise extension) Hence Cond. 1 is satisfied.

24 Cond. 2: We have already proved in Lem. A.23. M. A Ranking Domain for Tree Automata Cond. 3: Immediate from qnc = id[0,1]. Notation A.26. We write hi for the empty sequence in N∗. Cond. 4: It is already proved in Lem. A.24. For w, w′ ∈ N∗ ∪ Nω, we write w  w′ if w is a prefix of w′. Hence (r , q , ≤) is a ranking domain. nc,γ p Definition A.27 (unlabeled tree). An unlabeled tree is a set D ⊆ N∗ that satisfies the following conditions. 1) The empty sequence is in D, i.e. hi ∈ D. L. Towards Further Examples 2) The set D is prefix-closed, i.e. ∀w ∈ N∗. ∀i ∈ N. wi ∈ . Now that we have a general categorical axiomatization of D ⇒ w ∈ D 3) The set D is downward-closed, i.e ∀w ∈ N∗. ∀i ∈ ranking functions (§III), we would like to exploit it in deriving N . further examples of “ranking functions” that are previously . ∀j ≤ i. wi ∈ D ⇒ wj ∈ T unknown, hoping that they will provide novel proof methods An unlabeled tree D is said to be finite-depth if it satisfies the for various liveness properties. In the previous section we following additional condition. derived two variations of ranking supermartingales. Towards 4) The set D has no strictly increasing sequence with further examples, here we indicate a possible direction. respect to , i.e. We can say that not many concrete examples are known ∀v ∈ Nω. |{w ∈ D | w  v}| < ∞ . of corecursive algebras. Nevertheless, the following “closure properties” can be used to derive new examples. We write Tree (resp. Treefin) for the set of all unlabeled trees (resp. unlabeled finite-depth trees). Lemma A.25. Let C C be a functor. F : → For an unlabeled tree D, we define a function branchD : 1) ([33]) Consider the well-known construction of the final D → N ∪ {∞} as follows. 2 ! F ! 2 F ! a sequence: 1 ← F 1 ← F 1 ← ··· ← F 1 ← · · ·, branchD(w) = max{i | wi ∈ D} . where we define F a1 for an arbitrary ordinal a (we a b The prefix order over Tree is a partial order  that is defined assume enough limits and let F 1 = limb

25 Notation A.32. Let c : X → Ft,ΣX be an Ft,Σ-coalgebra. For have: each x ∈ X where c(x) = (ξ,t), we write c1(x) and c2(x) n σt ◦ Ft,Σf((a, x1, x2,...),t) for ξ ∈ n∈N∪{ω} Σn × ( ) and t ∈{0, 1}, respectively. 1 (t =1 or ∀i.f(xi)=1) Definition` A.33 (run tree). Let c : X → F X be an F - = (by def. of σt) t,Σ t,Σ 0 (otherwise) coalgebra. We define a new ranked alphabet X × Σ = (X × ( Σ, | |) by |(x, a)| = |a|. For x ∈ X, an (X × Σ)-labeled tree 1 (t =1 or ∀i. RunTree (x ) is accepting) = c i (D,l) is called a run tree of c from x if it satisfies the following 0 (otherwise) ′ ( conditions (here for each w ∈ D where l(w) = (x ,a), l1(w) ′ (by def. of f) and l2(w) denote x and a respectively): 1 (RunTreec(x) is accepting) 1) l (hi)= x; and = 1 0 (otherwise) 2) for each w ∈ D, ( (by def. of RunTreec) = f(x) . c1(l1(w)) = l2(w),l1(w0),l1(w1),...) . Hence f is a fixed point of Φ . c,σt It remains to show that f is the least fixed point. Let f ′ : A run tree (D,l) is called accepting if it has no infinite X →{0, 1} be a fixed point of Φ . It suffices to prove that branch labeled only with non-accepting states, i.e. c,σt for each x ∈ X, f ′(x)=0 implies f(x)=0. Let x ∈ X and assume f ′(x) = 0. Let (D,l) = ω ∗ N ¬ ∃v ∈ N . ∀w ∈ N s.t. w  v. (w ∈ D ∧ c2(l(w)) = 0) . RunTreec(x). For each i ∈ , we inductively define xi ∈ X ′ and vi ∈ N so that f (xi)=0 for each i as follows.  ′ • For i =0, we let x0 = x. Then we have f (x0)= f(x)= Proposition A.34. For an Ft,Σ-coalgebra c : X → Ft,ΣX 0. and x ∈ X, there exists a unique run tree of c from x. • Let i ≥ 0 and assume that we have defined xj and ′ vj−1 for each j ≤ i so that f (xi)=0. Let c(xi) = Definition A.35 (RunTreec and Reachc). By Prop. A.34, we ′ ((ai, xi,1, xi,2,...),ti). As f is a fixed point of Φc,σt , write RunTreec(x) for the unique run tree of c from x. For an we have: Ft,Σ-coalgebra c : X → Ft,ΣX, we define a function Reachc : ′ ′ X →{0, 1} by 0= f (xi) = (σt ◦ Ft,Σf ◦ c)(xi) . (15)

By the definition σt, this means that there exists k such that f ′(x )=0. We let v = k and x = x . 1 (RunTreec(x) is accepting) i,k i i+1 i,k Reachc(x) = We can prove that x and v satisfy the followings. (0 (otherwise) . i i 1) ∀i. (v0 ...vi−1) ∈ D 2) ∀i. l(v0 ...vi−1)= xi Definition A.36. We define an Ft,Σ-modality σt : 3) ∀i. c2(l(v0 ...vi−1))=0 Ft,Σ{0, 1}→ {0, 1} over a truth-value domain ({0, 1}, ≤) Here (1) and (2) are immediate from the definition. The equa- by tion (15) implies that c2(l(v0 ...vi−1)) = c2(xi)= ti =0 for each i ∈ N. This means that RunTreec(x) is not accepting (see 1 (t =1 or ∀i.pi = 1) σt (a,p1,p2,...),t = Def. A.33), and therefore we have f(x)=0. This concludes (0 (otherwise) . the proof.  Proposition A.38. The Ft-modality σt : Ft{0, 1}→{0, 1} in

Proposition A.37. The Ft,Σ-modality σt : Ft,Σ{0, 1} → Def. A.36 satisfies conditions in Asm. III.7. {0, 1} in Def. A.36 has least fixed points. For an Ft,Σ- ⊥ Proposition A.39. Let Σ be a ranked alphabet and Treefin = coalgebra c : X → Ft,ΣX, the (coalgebraic) least fixed ⊥ Treefin ∪{⊥t}. We define an Ft,Σ-algebra rt : Ft,Σ(Treefin) → property µσt c : X → {0, 1} is given by the function ⊥ ⊥ Treefin, a function qt : Treefin → {0, 1} and a partial order Reachc : XJ →{K 0, 1} in Def. A.35. ⊥ ⊥ over Tree as follows. ⊑Treefin fin

Proof. We define f : X →{0, 1} by f = Reachc. It suffices ⊥ X 1) rt (a,D1,D2,...),t to prove that f is the least fixed-point of Φc,σt : (Treefin) → ⊥ X (Treefin) . hi  (t = 1) We first show that f is a fixed point of Φc,t. Let x ∈ X = combine(D1,D2,...) (t =0 and ∀i. Di 6= ⊥t) and assume that c(x) = ((a, x1, x2,...),t) ∈ Ft,ΣX. Then we ⊥t (t =0 and ∃i. Di = ⊥t)

 26 1 (D 6= ⊥t) Assume B ∈ Treefin. We prove that B is the greatest lower 2) qb(D)= (0 (D = ⊥t) . bound of K. Let D ∈ K. By the definition of B, we have def. D ⊆ B. As there exist no D1,D2 ∈ K that satisfy (16), we 3) D1 ⊑Tree⊥ D2 ⇔ D1 = ⊥t or (D1,D2 ∈ fin have branchD(w) ∈{0, branchB(w)} for each w ∈ D. Hence Treefin and D1  D2) (note the direction). we have B ⊑ ⊥ D, and therefore B is a lower bound of Treefin Then (r , q , ⊑ ⊥ ) is a ranking domain. t t Treefin K. It is immediate from its definition that B is the greatest ⊥ lower bound. Lemma A.40. The poset Tree ⊥ is a complete ( fin, ⊑Treefin ) lattice. Lemma A.41. The triple (r , q , ⊑ ⊥ ) in Prop. A.39 satis- t t Treefin ⊥ fies Cond. 2 in Def. III.10. Proof. Let K ⊆ Treefin. K has the least upper bound. We first prove that K has Proof. Cond. 2a is immediate from Lem. A.40. the least upper bound. If K ∩ Treefin = ∅, then ⊥t is the least It is easy to see that Cond. 2b is satisfied. upper bound of K. ′ ⊥ ⊥ Assume K ∩ Treefin 6= ∅. If K = K \ {⊥t} has the least Lemma A.42. The algebra rt : Ft,ΣTreefin → Treefin in upper bound, then it is also the least upper bound of K. Prop. A.39 is a corecursive algebra. We define A ⊆ N∗ as follows. Proof. It suffices to show that the function Φc,rt : ′ Tree ′ ′ ⊥ X ⊥ X A = {D ∈ fin | ∀D ∈ K .D  D} . (Treefin) → (Treefin) has a unique fixed point. ⊥ X ′ By Lem. A.41, the set ((Tree ) , ⊑ ⊥ ) where ⊑ ⊥ As {hi}  D[holds for each D ∈ K and Cond. 1–3 in fin Treefin Treefin denotes the pointwise extension is a complete lattice. There- Def. A.27 are preserved by the union, A ∈ Tree. We prove fore by Thm. II.18, the function Φ , which is monotone by that A is the least upper bound of K′. c,rt Lem. A.41, has the least fixed point. We first prove that A is an upper bound. Let D ∈ K′. We It remains to show that this is the unique fixed point. Let prove D ⊑ ⊥ A. To this end, by the definition of , it Treefin ′ ⊥ f,f : X → Treefin be fixed points of Φc,rt . suffices to prove A ⊆ D and branchA(w) ∈{0, branchD(w)} Tree⊥ for each w ∈ A. The former is immediate from the definition Let g : X → fin and assume g(x) = ⊥t. Let c(x) = ((a, x0, x1,...),t). As g is a fixed point of Φc,r , we have of A. The latter is satisfied because we have branchD′ (w) ∈ t ′ ′ ′ (rt ◦ Ft,Σg ◦ c)(x)= ⊥t. Hence we have t =0 and that there {0, branchD(w)} for each D such that D  D and w ∈ D . Hence is an upper bound of . We note that this also exists i such that g(xi)= ⊥t. A K 0 1 2 proves that is finite-depth. Therefore we can inductively define a sequence x x x ... ∈ A ω It is immediate from the definition of A that A is the least X so that: i upper bound. 1) g(x )= ⊥t; i K has the greatest lower bound. We prove that K has the 2) c2(x )=0 for each i ∈ ω; and 3) c (xi) = (ai, xi , xi ,...) then xi+1 = xi for some n. greatest lower bound. If ⊥t ∈ K, then d K = ⊥t. If K = ∅, 1 1 2 n then d K = {hi}. Conversely, we can prove that existence of a sequence 0 1 2 ω We assume ⊥t ∈/ K and K 6= ∅. For D1,D2 ∈ K, if there x x x ... ∈ X that satisfies the conditions above implies exists D3 ∈ K such that D1  D3 and D2  D3 then by the g(x)= ⊥t. ′ definition of , we have: Therefore we have: f(x)= ⊥t if and only if f (x)= ⊥t. We now prove that for each n ∈ N, w ∈ Nn and x ∈ X ∀w ∈ D1 ∩ D2. (branchD1 (w) 6=0 and branchD2 (w) 6= 0) ′ such that f(x) 6= ⊥t and f (x) 6= ⊥t, w ∈ f(x) if and only ′ ⇒ branchD1 (w)= branchD2 (w) . if w ∈ f (x) by the induction on n. For n =0, by Def. A.27, we have hi ∈ f(x) and hi ∈ f ′(x) Therefore if there exists D ,D ∈ K that satisfy 1 2 for each x. Nn ∃w ∈ D1 ∩ D2. branchD1 (w) 6=0, branchD2 (w) 6=0 Let n > 0 and assume that for each w ∈ and x ∈ X ′ such that f(x) 6= ⊥t and f (x) 6= ⊥t, we have w ∈ f(x) and branchD1 (w) 6= branchD2 (w), ′ (16) if and only if w ∈ f (x). Let x ∈ X and assume c(x) = m ((a, x0, x1,...),t) ∈ (Σ × ( ) ) ×{0, 1}. For each i ∈ N, then there does not exist D3 ∈ K such that D1  D3 and we have: D2  D3. This implies d K = ⊥t. Assume that there does not exist D1,D2 ∈ K that satisfy iw ∈ f(x) (16). We define N∗ by B ∈ ⇔ iw ∈ Φc,rt (f) (f is a fixed point) B = K. ⇔ i

27 Hence we have f(x) = f ′(x) for each x ∈ X. This concludes the proof.

Proof (Prop. A.39). We prove that ⊥ satisfies (rt, qt, ⊑Treefin ) conditions in Def. III.10. ⊥ 1: Let ((a,D0,D1,...),t) ∈ Ft,ΣTreefin. Then we have:

(qt ◦ rt) (a,D0,D1,...),t =1 ⇔ rt (a,D0,D1,...),t 6= ⊥t (by def. of qt) ⇔ t =1 or ∀i. Di 6= ⊥t (by def. of rt) ⇔ t =1 or ∀i. qt(Di)=1 (by def. of qt)

⇔ (σt ◦ Ft,Σqt) (a,D0,D1,...),t = 1 (by def. of σt) .

Hence we have qt◦ rt = σ ◦ Ft,Σqt.  2: We have already proved in Lem. A.41. 3: Monotonicity and strictness are immediate from the def- ⊥ inition of qt. We prove that qt is continuous. Let K ⊆ Treefin. Then we have:

qt(D)=0 D K G∈ ⇔ ∀D ∈ K. qt(D)=0

⇔ K = {⊥t} (by def. of qt)

⇔ K = ⊥t

⇔ qGt( K)=0 (by def. of qt) .

Hence qt is continuous.G 4: We have already proved in Lem. A.42.

Proposition A.43 (completeness). For an arbitrary Ft,Σ- coalgebra c : X → Ft,ΣX, there exists a ranking arrow ⊥ b : X → Treefin with respect to the ranking domain (r , q , ⊑ ⊥ ) in Prop. A.39 that satisfies the following. t t Treefin

qt ◦ b = µσt c J K Proof. Immediate from qt ◦ rt = σt ◦ Ft,Σqt (see the proof of Prop. A.39) and Prop. III.18.

28