Introduction to Model-Checking Theory and Practice

Beihang International Summer School 2018 Model Checking Linear Temporal Properties Model Checking

“Model checking is the method by which a desired behavioral property of a reactive system is verified over a given system (the model) through exhaustive enumeration (explicit or implicit ) of all the states reachable by the system and the behaviors that traverse through them” Amir Pnueli (2000) Basic Properties

• reachability (something is possible)

• invariant (something is always true) the gate is closed when the train cross the road

• safety (something bad never happen: ≡ ¬ 푅푒푎푐ℎ)

• liveness (something good may eventually happen) every process will (eventually) gain access to the printer

• Fairness (if it may happen ∞ often then it will happen) if messages cannot be lost ∞ often then I will received an ack Model-Checking

• We have seen how to use a formal specification language to model the system (⇒ as a graph, but also as a language ℒ)

• We have seen how to check basic properties on the reachability graph (and that SCC can be useful)

• What if we want to check more general properties? User-Defined Properties

• present before (we should see an 푎 before the first 푏) the latch must be locked before take-off

• infinitely often (something will always happen, ≠ 퐴푙푤푎푦푠) the timer should always be triggered

• leadsto (after every occurrence of 푎 there should be a 푏) there is an audible signal after the alarm is activated

• … see example of specification patterns here [Dwyer] Model-Checking

System +

Requirements informal

Description Description M of the of the 휙 behavior property

operational denotational formalization

verification Does 푀 ⊨ 휙 Does 푀 ≡ 퐺휙 yes / no / c. ex. present 푎 before 푏

• In every execution, action 푎 should occur before 푏 or there should be no 푏

푝1, 푝2, 푝6

푝0, 푝2 푝1, 푝2, 푝4

푝1, 푝5, 푝2 푝1, 푝3

present 푟푐푣 before 푎푐푘 Infinitely often 푎

• every “maximal execution” should have an unbounded number of 푎 equivalently: from every state, it is unavoidable to do an 푎

푝1, 푝2, 푝6

푝0, 푝2 푝1, 푝2, 푝4

푝1, 푝5, 푝2 푝1, 푝3

infinitely 푟푐푣 infinitely 푠푒푛푑 Infinitely often 푎

• every “maximal execution” should have an unbounded number of 푎 equivalently: from every state, it is unavoidable to do an 푎

푝1, 푝2, 푝6

Infinitely 푟푐푣 푝0, 푝2 푝1, 푝2, 푝4

푝1, 푝5, 푝2 푝1, 푝3 Infinitely often 푎

• every “maximal execution” should have an unbounded number of 푎 equivalently: from every state, it is unavoidable to do an 푎

푝1, 푝2, 푝6

Infinitely 푠푒푛푑 푝0, 푝2 푝1, 푝2, 푝4

푝1, 푝5, 푝2 푝1, 푝3 푎 leadsto 푏

• after every occurrence of action 푎 we should eventually find an occurrence of action 푏 or there is no 푎

푝1, 푝2, 푝6

푝0, 푝2 푝1, 푝2, 푝4

푝1, 푝5, 푝2 푝1, 푝3

푠푛푑 leadsto 푟푐푣 Model Checking Linear Temporal Properties using Language Inclusion Model-Checking

System +

Requirements informal

Description Description M of the of the 휙 behavior property

operational denotational formalization

verification Does 푀 ⊨ 휙 Does 푀 ≡ 퐺휙 yes / no / c. ex. Model-Checking

correct behavior

퐿(푀)

퐿 푀 ∩ 퐿 휙

counter-example 퐿 휙

Description Description M of the Does 퐿 푀 ⊆ 퐿(휙) ? of the 휙 behavior property present 푎 before 푏

• In every execution, action 푎 should occur before 푏 or there should be no 푏present 푎 before 푏

a = {a} 1 = {a, b, c, …} !a = {b, c, …}

• idea: use set of words (language) of infinite length that go through a “red state” infinitely often present 푎 before 푏

• In every execution, action 푎 should occur before 푏 or there should be no 푏present 푎 before 푏

a = {a} 1 = {a, b, c, …} !a = {b, c, …}

• Equivalently: find traces where “red places” are infinitely marked. present 푎 before 푏

푝1, 푝2, 푝6

푝0, 푝2 푝1, 푝2, 푝4

푝1, 푝5, 푝2 푝1, 푝3

02

00 01 present 푟푐푣 before 푎푐푘 12

10 13

14 11 Büchi Automata Büchi automata

• In here we consider automata with a set of final, or accepting states 퐹 ⊆ 푄 • we use “red states” to mean accepting • It is a finite state automata with a different acceptance condition. An (infinite) word is accepted if it corresponds to an infinite run that contains infinitely many accepting states

a = {a} 1 = {a, b, c, …} !a = {b, c, …} Example

푎. 푎. 푎. … 푎휔 is accepted 푎. 푎. 푏. 푎. 푏. 푎. … 푏. 푎 휔 is accepted 푎. 푎. 푏. 푏. 푏. … 푏 휔 is rejected 푎. 푏. 푎. 푏2. 푎. 푏3. … 푏푛. 푎 휔 is accepted Büchi automata

• To find if there exist a word that is not accepted by a Büchi automata, it is enough to find a cycle without accepting state we know how to do it (remember Tarjan’s algorithm) • Same thing if we want to test if ℒ = ∅ Model Checking Linear Temporal Properties using Language Inclusion a better idea Model-Checking

퐿(푀)

퐿 푀 ∩ 퐿 휙 퐿 ∖ 퐿 휙

퐿 휙

Description Description M of the Does 퐿 푀 ∖ 퐿 휙 = ∅ ? of the 휙 behavior property present 푎 before 푏

• We disprove the property if we find a trace where 푏 can be reached without firing action 푎

a = {a} 퐿 ¬ 휙 ≡ 1 = {a, b, c, …} !a = {b, c, …} present 푎 before 푏

푝1, 푝2, 푝6

푝0, 푝2 푝1, 푝2, 푝4

푝1, 푝5, 푝2 푝1, 푝3

present 푟푐푣 before 푎푐푘 Infinitely often 푎

• From every state, it is unavoidable to do an 푎

≡ 퐿 휙 Infinitely often 푎

• We disprove the property if we find a trace without a single 푎 (maybe after some other transitions fire)

≡ 퐿 ¬ 휙 Infinitely often 푎

10 12 infinitely 푟푐푣 11

10 Infinitely often 푎

100 010 001

001

100 010

100 푎 leadsto 푏

• after every 푎 we eventually find a 푏 We disprove the property if we find an 푎 followed by an infinite sequence (a SCC) without 푏

푠푛푑 leadsto 푟푐푣 What you need to remember

• We can check more complex properties using an automata-theoretic approach • It is often easier to try to disprove the property

퐴 푆푦푠 ⊗ 퐴 ¬휙 = ∅

• We need automata that accept infinite words → different acceptance criterion (Büchi-automaton) • There is a link with SCC (infinitely often ≈ cycle) Model Checking LTL, a Principled Approach Model-Checking

• We have seen how to use “language inclusion” (product of automata and search for an infinite path) to express temporal properties on a system

• What if we want to check more general properties? Is there a more friendly way to define temporal properties ?

• How can we derive an automaton from it Classical logic

• logic is the systematic study of the form of valid inference • Aristotle (322 BC) • Clarence Lewis (∼1910) for its actual form • A formula is valid iff it is true under every interpretation. • An argument form (or schema) is valid iff every argument is valid

wikipedia Non-classical logics

• Example of valid argument form (or schema) (퐴 ⇒ 퐵 ∧ 퐴) ⇒ 퐵 • Predicate logic has propositional variables (퐴, 퐵, …) and connectives (∧, ¬, ⇒, …) • There are also non-classical logics, such as modal logics, that extend logic with operators (modalities) expressing the fact that the truth may depends on the context examples are beliefs: I am certain vs it is possible; permissions (deontic logic): it is permissible vs it is obligatory; and time: always vs eventually Temporal logic

Amir Pnueli (1941—2009)

“In mathematics, logic is static. It deals with connections among entities that exist in the same time frame. When one designs a dynamic computer system that has to react to ever changing conditions, … one cannot design the system based on a static view. It is necessary to characterize and describe dynamic behaviors that connect entities, events, and reactions at different time points. Temporal Logic deals therefore with a dynamic view of the world that evolves over time." Atomic proposition

• We start by defining atomic propositions statements about the here and now ! • We assume a set of propositional variables {푝1, 푝2, … , 푝푛} • We will use the language of logic. Atomic Formulas are built from P. V. and from connectives

푎, 푏, … ∷= 푝 ¬푎 푎 ∧ 푏 | …

• e.g.: 푝1 ∧ (푝2 ∨ ¬푝3) 푝1 ⇒ 푝2 ≡ ¬푝1 ∨ 푝2 Atomic Prop.: event/state-based

• If we want to deal with events (transitions), we can choose atomic propositions 푎, 푏, … that corresponds to event names: • 푡1 • 푑푒푎푑

• We can also choose to deal with states (markings) • 푝1 + 2. 푝3 ≤ 4 푝1 ≡ (푝1 ≥ 1) • 퐹푖푟푎푏푙푒(푡) • 푑푒푎푑 • 푖푛푖푡푖푎푙 (`0)

(from now on we consider only “state-based” formulas) Linear Temporal Logic

• The logic has two main connectives 퐹 휙: reads “finally” (eventually) 휙 is true 퐺 휙: reads “globally” (always) 휙 is true

휙, 휓, … ∷= 푎 ¬휙 휙1 ∧ 휙2 퐹 휙 퐺 휙

• So you can write formulas such as:

퐺 (푝1 ⇒ 퐹 푝2 + 푝3 ≤ 푝4 ) ¬ 퐹 푑푒푎푑 ∨ 퐺 (¬푝2)

퐹 퐺 푝1 ∧ 퐺 퐹 푝2 퐺 ( 퐺 푡1 ∨ (퐺 푡2)) Linear Temporal Logic

• The logic has two main connectives 퐹 휙: reads “finally” (eventually) 휙 is true, also []휙 퐺 휙: reads “globally” (always) 휙 is true, also 〈〉휙

• There is another possible presentation based on two additional connectives 휙 푈 휓: reads “휙 holds until 휓” 푋휙 : reads “next” 휙 holds, also written ()휙 LTL—syntax equivalence

퐹 휙 〈 〉휙 finally 퐺 휙 휙 globally 푋 휙 휙 next 휙 푈 휓 휙 푈 휓 until ! 휙 ¬휙 negation

SPIN syntax selt syntax Pinyin name LTL—semantics

• LTL formulas are interpreted on (maximal) traces, 푤 = 푤0. 푤1. … . 푤푖. … for “state-based” properties, 푤푖 is a state ≡ the set of atomic propositions true in 푤푖 • We call 푤(푖) the 푖th element in 푤 • We use the notation 푤, 푖 ⊨ 휙 to say that 휙 holds for 푤 from position 푖

푤 ⊨ 휙 ⇔ 푤, 0 ⊨ 휙 satisfaction relation ⊨ LTL—semantics

푤 ⊨ 휙 ?

푤, 2 ⊨ ϕ … 푤, 푖 ⊨ ϕ … 푤, 푘 ⊨ ϕ

푤0 푤1 푤2 … 푤푖 … 푤푘 LTL—atomic propositions

• For atomic propositions, 푎, we can say whether it holds for 푤푖 or not (we write 푤푖 ⊨ 푎 if it holds). 푤, 푖 ⊨ 푎 iff 푤푖 ⊨ 푎

• For example, if 푤푖 is the marking (푝1, 푝2 .2, 푝4)— say (1, 2, 0, 1)—then we have that:

푤, 푖 ⊨ 푝2 푤, 푖 ⊨ (푝1 + 푝4 ≤ 푝2) 푤, 푖 ⊨ ¬(푝1 ∧ 푝3) LTL—other connectives

푤, 푖 ⊨ 휙 ∨ 휓 iff (푤, 푖 ⊨ 휙) or (푤, 푖 ⊨ 휓) 푤, 푖 ⊨ ¬휙 iff not 푤, 푖 ⊨ 휙 etc.

푤, 푖 ⊨ 〈 〉휙 iff ∃ j ≥ i . (푤, 푗 ⊨ 휙) 휙 holds at some “instant” 푗 in the future

푤, 푖 ⊨ [ ]휙 iff ∀푘 ≥ 푖 . (푤, 푘 ⊨ 휙) 휙 holds at all instants after 푖 LTL—semantics True : ⊤ False : ⊥ 푤 ⊨ 휙 ?

푤, 푖 ⊨ ϕ ⊥ ⊥ ⊥ … ⊥ ⊤ ⊥ … ⊥ 푤, 푖 ⊨ 휙 ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊥ …

푤0 푤1 푤2 … 푤푖 … 푤푘 LTL—semantics True : ⊤ False : ⊥ 푤 ⊨ [ ]휙 ?

푤, 푖 ⊨ ϕ ⊤ ⊤ ⊤ … ⊤ ⊥ ⊤ … ⊤ 푤, 푖 ⊨ [ ]휙 ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊤ …

푤0 푤1 푤2 … 푤푖 … 푤푘 LTL—other connectives

We can define the semantics for the two extra op. 푤, 푖 ⊨ ()휙 iff 푤, 푖 + 1 ⊨ 휙 휙 holds at the next “instant”

푤, 푖 ⊨ 휙 푈 휓 iff ∃푗 ≥ 푖 . (푤, 푗) ⊨ 휓 and ∀푘 ∈ [푖, 푗[ . 푤, 푘 ⊨ 휙 there is an instant 푗 in the future such that 휓 holds and 휙 holds until that time LTL—semantics True : ⊤ False : ⊥ 푤 ⊨ 휙 푈 휓 ?

푤, 푖 ⊨ ϕ ⊤ ⊤ ⊤ … ⊤ … 푤, 푖 ⊨ 휓 ⊥ ⊥ ⊥ ⊥ ⊥ ⊤ … 휙 푈 휓 ⊤ ⊤ ⊤ … ⊤ ⊤

푤0 푤1 푤2 … 푤푖 … 푤푘 LTL—other connectives

We can use the until connective to define F and G

휙 ≡ 푇푟푢푒 푈 휙

휙 ≡ ¬(푇푟푢푒 푈 (¬휙))

Actually, it is true that

휙 ≡ ¬ (¬휙)

Think De Morgan’s laws: ¬ 푎 ∨ 푏 ≡ ¬푎 ∧ ¬푏 LTL—other connectives

We can also use next to (recursively) define F and G

휙 ≡ 휙 ∨ 휙 ≡ 휇푋. (휙 ∨ 푋)

휙 ≡ 휙 ∧ 휙 ≡ 휇푋. (휙 ∧ 푋) 휙 푈 휓 ≡ 휇푋. (휓 ∨ 휙 ∧ 푋 )

휇푋. 푓(푋) means the “smallest fixpoint” for the functional 푓, i.e. 푉 such that 푓 푉 = 푉. LTL—semantics

• A property is true (it holds) for a trace 푤 if it is true “at the beginning” (푤, 0 ⊨ 휙)

• A property holds for a system if it is true for all its (maximal) trace • how many traces can there be ? LTL—semantics

We consider finite-state systems, hence a maximal trace either ends in a deadlock or it has a cycle

푠1 푠2 푠1 푠1 푠2 푠1 푠1 푠3 푠1 0 1 2 3 4 5 6 7 8

10 9

푠1 푠1 LTL—semantics

Example: 퐹 퐺 푠1 (that is 푠1)

0 1 2 3 4 5 6 7 8 … 푠1 = ⊤ ⊥ ⊤ ⊤ ⊥ ⊤ ⊤ ⊥ ⊤ ⊤ 푠1 = ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊤ ⊤ 푠1 = ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊤

푤 = 푠1 푠2 푠1 푠1 푠2 푠1 푠1 푠3 푠1 … 푠1 ∗ LTL—semantics

Example: 퐹 (푠3 ∨ 퐺 푠2) (that is (푠3 ∨ 푠2))

0 1 2 3 4 5 6 7 8 … 푠3 = ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊤ ⊥ ⊥ 푠2 = ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ 푠3 ∨ 푠2= ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊤ ⊥ ⊥ … = ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊤ ⊥ ⊥

푤 = 푠1 푠2 푠1 푠1 푠2 푠1 푠1 푠3 푠1 … 푠1 ∗ Exercise Check 푤, 푖 ⊨ 휙

푞 푞 푞 푝 ∅

0 1 2 3 4

0 1 2 3 4 5 … 푞 푈 푝 퐹 퐺 ¬푝 퐹(푞 푈 푝) 퐹¬(푞 푈 푝) ¬퐺(푞 푈 푝) ¬퐺¬(푞 푈 푝) Model Checking LTL specifications Example specification

• Mutual exclusion never more than one process can be in state working at any given time

• No starvation a process that wants to work (in state waiting) should eventually reach state working

• Bounded usage time a process in state working should eventually be idle Example specification

atomic prop. are: 푖푑푙푒푖, • Mutual exclusion 푤푎푖푡푖, and 푤표푟푘푖. ¬(푤표푟푘푖 ∧ 푤표푟푘푗)

• No starvation (푤푎푖푡푖 ⇒ 푤표푟푘푖) (푤푎푖푡푖 ⇒ 푤푎푖푡푖 푈 푤표푟푘푖 ) unnecessary

• Bounded usage time (푤표푟푘푖 ⇒ ¬푤표푟푘푖) Exercise Additional specification

We say that 휙 precedes 휓 holds for 푤, at 푘 (written 푤, 푘 ⊨ 휙 푃 휓) when:

∀푗 ≥ 푘 . 푤, 푗 ⊨ 휓 ⇒ ∃푖 ∈ 푘, 푗 . (푤, 푖 ⊨ 휙) that is, 푤 ⊨ 휙 푃 휓 as soon as:

∀푗. 푤, 푗 ⊨ 휓 ⇒ ∃푖 ≤ 푗. (푤, 푖 ⊨ 휙) can you express this new modality in LTL or should we add it to the logic ? Additional specification

We can write the following requirement as follows: “access to the critical section is allowed only to the workers that asked for it”

(¬푤표푟푘푖 ⇒ 푤푎푖푡푖 푃 푤표푟푘푖 ) that is, before working, process 푖 must have asked it. Could you express the stronger requirement that: “access to the critical section is granted in the order where workers asked for it” ? Model Checking using Tina selt tina > selt

• The tina toolbox has a LTL model-checker called selt • The program takes as input a reachability graph (either in AUT format, or in a compressed format called KTZ) • LTL formulas include: negation — implication => conjunction /\ disjunction \/ always [] eventually <> constants T (true), F (false), dead Some examples of formulas

[] (p1 /\ p2) ; p1 and p2 always true (everywhere) <> (p1 \/ p2) ; means either p1 or p2 is true in every trace [] (<> p) ; means p true infinitely often <> ([] p) ; means p will become always true How to use selt

1. use nd to draw/open a Petri net 2. use tool > reachability to generate the marking graph in compressed (ktz) format 3. you can either A. invoke selt directly from the nd window (right click then choose “model check LTL”) B. save in a file, say xx.ktz, and invoke “selt xx.ktz” in the command line 4. Every input must end with a semi-colon: “;” 5. When a property is false, a counter-example is printed see http://projects.laas.fr/tina/manuals/selt.html How to use selt

• Counter-examples can be replayed in the simulator (if it is already open) • There are several levels of details for printing the counter-examples: output fullproof ;

• To quit selt, simply enter “quit;” • There are other commands, go see: http://projects.laas.fr/tina/manuals/selt.html