Announcements

• Office hours Model Checking: – M 1:30pm-2:30pm An Introduction – W 5:30pm-6:30pm (after class) – and by appointment – ECOT 621 Meeting 2 • Moodle problems?

Fundamentals of Programming Languages CSCI 5535, Spring 2009

2

A Double Header Who are we again?

• Two lectures • We're going to find critical bugs in important – Model checking primer bits of software – Software model checking – using PL techniques ! – SLAM and BLAST (tools) • You’ll be enthusiastic about this • Some key players: – and thus want to learn the gritty details – Model checking Ed Clarke, Ken McMillan, Amir Pnueli – SLAM Tom Ball, Sriram Rajamani – BLAST Ranjit Jhala, Rupak Majumdar, Tom Henzinger

3 4

Take-Home Message Overarching Plan

• Model checking is the exhaustive Model Checking (today) exploration of the state space of a – Transition systems (i.e., models) system, typically to see if an error state – Temporal properties – Temporal logics: LTL and CTL is reachable . It produces concrete – Explicit-state model checking counterexamples . – Symbolic model checking • The state explosion problem refers to Counterexample Guided Abstraction Refinement the large number of states in the model. – Safety properties – Predicate abstraction “c2bp” • Temporal logic allows you to specify – Software model checking “bebop” properties with concepts like – Counterexample feasibility “newton” – Abstraction refinement weakest pre, thrm prv “eventually” and “always”. 5 6

1 Spoiler

• This stuff really works!

• Symbolic model checking is a massive Model Checking success in the model-checking field • SLAM took the PL world by storm There are complete courses in model checking – Spawned multiple copycat projects (see ECEN 5139, Prof. Somenzi). – Launched Microsoft’s Static Driver Verifier Model Checking by Edmund M. Clarke, Orna (released in the Windows DDK) Grumberg, and Doron A. Peled. Symbolic Model Checking by Ken McMillan. We will skim. 7

Keywords Verification vs. Falsification

Model checking is an automated technique • What is verification? Model checking verifies transition systems Model checking verifies temporal properties Model checking falsifies by generating counterexamples • What is falsification? A model checker is a program that checks if a (transition) system satisfies a (temporal) property

9 10

Verification vs. Falsification Temporal Properties

• An automated verification tool Temporal Property – can report that the system is verified (with a proof); A property with time-related operators such as – or that the system was not verified. “invariant ” or “ eventually ” Invariant( p) • When the system was not verified, it would be helpful to explain why. is true in a state if property p is true in every – Model checkers can output an error counterexample : a state on all execution paths starting at that state concrete execution scenario that demonstrates the error. G, AG, (“globally” or “box” or “forall”) • Can view a model checker as a falsification tool Eventually( p) – The main goal is to find bugs is true in a state if property p is true at some state • So what can we verify or falsify? on every execution path starting from that state F, AF, ♦ (“future” or “diamond” or “exists”) 11 12

2 Reachable States An Example Concurrent Program of the Example Program • A simple concurrent 10: while (true) { mutual exclusion program 11: wait(turn == 0); • Two processes execute // critical section t=0 t=1 Next: formalize asynchronously 12: work(); turn = 1; 10,20 10,20 this intuition … • There is a shared variable 13: } turn t=0 t=0 t=1 t=1 10,21 11,20 10,21 11,20 • Two processes use the || // concurrently with shared variable to ensure that they are not in the 20: while (true) { t=0 t=0 t=1 t=1 critical section at the 11,21 12,20 11,21 same time 21: wait(turn == 1); 10,22 • Can be viewed as a // critical section “fundamental” program: 22: work(); turn = 0; t=0 t=1 Each state is a valuation 12,21 any bigger concurrent one 23: } 11,22 of all the variables: would include this one turn and the two program 13 counters for two processes#1414

Analyzed System is a Transition System Example Properties of the Program

• Labeled transition system Also called a • “In all the reachable states (configurations) T = (S, I, R, L) Kripke Structure of the system, the two processes are never in the critical section at the same time ” – S = Set of states // standard FSM – I ⊆ S = Set of initial states // standard FSM –“pc1=12 ”, “ pc2=22 ” are atomic properties for being in the critical section – R ⊆ S × S = Transition relation // standard FSM – L: S → 2AP = Labeling function // this is new! • AP : Set of atomic propositions (e.g., “x=5”∈AP) – Atomic propositions capture basic properties •“Eventually the first process enters the – For software, atomic props depend on variable values critical section” – The labeling function labels each state with the set of propositions true in that state

15 16

For what? Example Properties of the Program Temporal Logics For expressing properties

• “In all the reachable states (configurations) There are four basic temporal operators: of the system, the two processes are never in • X p the critical section at the same time ” Next p , p holds in the next state –“pc1=12 ”, “ pc2=22 ” are atomic properties for being • G p in the critical section Globally p , p holds in every state, p is an invariant Invariant (¬¬¬(pc1=12 ∧∧∧ pc2=22 )) • F p Future p , p will hold in a future state, p holds eventually •“Eventually the first process enters the • p U q critical section” p Until q , assertion p will hold until q holds Eventually (pc1=12 ) • Precise meaning of these temporal operators are defined on execution paths 17 18

3 Execution Paths Paths and Predicates

• A path in a transition system is an infinite • We write sequence of states x p (s 0, s 1, s 2, ...), such that ∀i≥0. (si, s i+1) ∈ R • A path (s 0,s 1,s 2,...) is an execution path if s0 ∈ I “the path x makes the predicate p true” • Given a path x = (s 0, s 1, s 2, ...) – x is a path in a transition system th – xi denotes the i state: si i th – p is a temporal logic predicate – x denotes the i suffix: (si, s i+1, s i+2, ...) • In some temporal logics one can quantify paths starting from a state using path quantifiers • Example: – A : for all paths A x. x G(¬¬¬(pc1=12 ∧∧∧ pc2=22)) – E : there exists a path

19 20

Linear Time Logic (LTL) Linear Time Logic (LTL)

• LTL properties are constructed from atomic • LTL properties are constructed from atomic propositions in AP; logical operators ∧, ∨, ¬; propositions in AP; logical operators ∧, ∨, ¬; and temporal operators X, G, F, U. and temporal operators X, G, F, U. • The semantics of LTL is defined on paths: • The semantics of LTL is defined on paths: Given a path x: Given a path x: x p iff L(x 0, p) atomic prop x p iff L(x 0, p) atomic prop x X p iff x1 p next x X p iff x1 p next x F p iff x F p iff ∃i≥0. x i p future x G p iff x G p iff ∀i≥0. x i p globally x p U q iff x p U q iff ∃i≥0. x i q and ∀j

Satisfying Linear Time Logic Computation Tree Logic (CTL)

• In CTL temporal properties use path quantifiers : A : for all paths, E : there exists a path • Given a transition system T = (S, I, R, L) • The semantics of CTL is defined on states: and an LTL property p, T satisfies p if Given a state s all paths starting from all initial states I s p iff L(s, p) satisfy p s0 EX p iff ∃ a path (s 0, s 1, s 2, ...). s 1 p s0 AX p iff ∀ paths (s 0, s 1, s 2, ...). s 1 p

s0 EG p iff ∃ a path (s 0, s 1, s 2, ...). ∀i≥0. sii p s0 AG p iff ∀ paths (s 0, s 1, s 2, ...). ∀i≥0. si p … 23 24

4 Linear vs. Branching Time Recall the Example

• LTL is a linear time logic – When determining if a path satisfies an LTL formula we are only concerned with a single path t=0 t=1 • CTL is a branching time logic 10,20 10,20 – When determining if a state satisfies a CTL t=0 t=0 t=1 t=1 formula we are concerned with multiple paths 10,21 11,20 10,21 11,20 – In CTL the computation is instead viewed as a computation tree which contains all the paths t=0 t=0 t=1 t=1 – The computation tree is obtained by unrolling the 11,21 12,20 11,21 transition relation 10,22

The expressive powers of CTL and LTL are t=0 This is a labeled incomparable (LTL ⊆ CTL*, CTL ⊆ CTL*) t=1 12,21 11,22 transition system – Basic temporal properties can be expressed in both logics – Not in this lecture, sorry! (Take a class on Modal Logics) 25 26

Linear vs. Branching Time LTL Satisfiability Examples

t=0 Linear Time Branching Time t=0 10,20 View View 10,20 p does not hold p holds t=0 . . . 10,21 t=0 t=0 10,21 11,20 t=0 10,21 On this path: t=0 t=0 t=0 t=0 t=0 10,21 11,21 11,21 12,20 Holds Does Not Hold

11,21 … … …

t=0 11,21 A computation tree t=0 t=1 starting at state 12,21 10,20 … … t=0 (turn=0,pc1=10,pc2=20) 12,21 One path starting at state t=1 10,21 (turn=0,pc1=10,pc2=20)

… 27 28

LTL Satisfiability Examples LTL Satisfiability Examples

p does not hold p holds

. . . p does not hold p holds

. . . On this path: Holds Does Not Hold On this path: F p holds , G p does not hold , p does not hold , X p does not hold , X (X p) holds , X (X (X p)) does not hold

. . . On this path: F p holds, G p holds, p holds, X p holds, X (X p) holds, X (X (X p))) holds

29 30

5 p does not hold p does not hold CTL Satisfiability Examples p holds CTL Satisfiability Examples p holds s At state s: s At state s: Holds Does Not Hold Holds Does Not Hold

......

31 32

p does not hold CTL Satisfiability Examples p holds

s s s

...... At state s: At state s: At state s: EF p, EX (EX p), EF p, AF p, EF p, AF p, AF ( ¬¬¬p), ¬¬¬p holds EX (EX p), AG p, EG p, EX p, EG p, p holds Ex p, AX p, p holds AF p, AG p, AG ( ¬¬¬p), EX p, AG p, AG ( ¬¬¬p), EG ( ¬¬¬ p), EF ( ¬¬¬p), ¬¬¬ EG p, p does not hold AF ( p) does not hold does not hold#33 33

Model Checking Complexity State Space Explosion

• Given a transition system T = (S, I, R, L) and a • The complexity of model checking CTL formula f increases linearly with respect to the – One can check if a state of the transition system size of the transition system (|S| + |R|) satisfies the formula f in O(|f| × (|S| + |R|)) time • However, the size of the transition – Multiple depth first searches (one for each temporal operator) = explicit-state model checking system (|S| + |R|) is exponential in the number of variables and number of • Given a transition system T = (S, I, R, L) and concurrent processes an LTL formula f • This exponential increase in the state – One can check if the transition system satisfies the space is called the state space explosion |f| formula f in O(2 × (|S| + |R|)) time – Dealing with it is one of the major 35 challenges in model checking research 36

6 Algorithm: Pictorial Backward Fixed Point Temporal Properties = Fixpoints • States that satisfy AG(p) are all the states which are not in EF( ¬p) (= the states that can reach ¬p) Inverse Image of ¬p = EX( ¬p)

• Compute EF( ¬p) as the fixed point of Func: 2S → 2S Initial • • • ¬¬¬ pp states • Given Z ⊆ S, – Func(Z) = ¬p ∪ reach-in-one-step (Z) initial states that violate AG( p) ¬ ¬ – or Func(Z) = ¬p ∪ EX(Z) states that can reach p = EF( p) = initial states that satisfy = states that violate AG( p) • Actually, EF( ¬p) is the least -fixed point of Func EF(¬p) – smallest set Z such that Z = Func(Z) This fixed point computation can be used for: – to compute the least fixed point, start the iteration from Z= ∅, and apply the Func until you reach a fixed point • verification of EF(¬p) … and similar fixed points – This can be computed (unlike most other fixed points) handle the other cases • or falsification of AG(p) 37 38

Symbolic Model Checking Binary Decision Diagrams (BDDs)

• Symbolic model checking represent state sets • Efficient representation for boolean and the transition relation as Boolean logic formulas functions (a set can be viewed as a function) – Fixed point computations manipulate sets of states • Disjunction, conjunction complexity: at rather than individual states – Recall: we needed to compute EX(Z), but Z ⊆ S most quadratic • Fixed points can be computed by iteratively • Negation complexity: constant manipulating these formulas • Use an efficient data structure for • Equivalence checking complexity: manipulation of Boolean logic formulas constant or linear – Binary Decision Diagrams (BDDs) • Image computation complexity: can be • SMV (Symbolic Model Verifier) was the first CTL model checker to use BDDs exponential 39 40

Key Terms

• Counterexample guided abstraction Building Up To refinement (CEGAR) Software Model Checking – A successful software model-checking approach. Sometimes called “Iterative via Counterexample Guided Abstraction Refinement”. Abstraction Refinement • SLAM = The first CEGAR project/tool. – Developed at MSR There are easily dozens of papers. • Lazy Abstraction = CEGAR optimization We will skim. – Used in the BLAST tool from Berkeley.

42

7 What is Counterexample Guided Verification Abstraction Refinement (CEGAR)?

Verification by … Example ( ) { 1: do { Is this program correct? lock (); Model Checking? old = new; q = q->next; 2: if (q != NULL){ What does correct mean? 3: q->data = new; Theorem Proving? unlock (); new ++; } 4: } while (new != old); How do we determine if a 5: unlock (); program is correct? Dataflow Analysis or Program Analysis? return ; }

43 44

Verification by Model Checking Verification by Theorem Proving

Example ( ) { Example ( ) { 1: do { 1. (Finite State) Program 1: do { 1. Loop Invariants lock (); lock (); old = new; 2. State Transition Graph old = new; 2. Logical Formulas q = q->next; q = q->next; 2: if (q != NULL){ 3. Reachability 2: if (q != NULL){ 3. Check Validity 3: q->data = new; 3: q->data = new; unlock (); → unlock (); new ++; - Program Finite state model new ++; } - State explosion } Invariant: 4: } while (new != old); 4: } while (new != old); ¬¬¬lock ÆÆÆ new = old 5: unlock (); + State exploration 5: unlock (); return ; + Counterexamples return ; ÇÇÇ } } ¬¬¬lock ÆÆÆ new ≠≠≠ old Precise [SPIN,SMV,Bandera,JPF] 45 46

Verification by Theorem Proving Verification by Program Analysis

Example ( ) { Example ( ) { 1: do { 1. Loop Invariants 1: do { 1. Dataflow Facts lock (); lock (); old = new; 2. Logical Formulas old = new; 2. Constraint System q = q->next; q = q->next; 2: if (q != NULL){ 3. Check Validity 2: if (q != NULL){ 3. Solve Constraints 3: q->data = new; 3: q->data = new; unlock (); unlock (); new ++; - Loop invariants new ++; - Imprecision: fixed facts } - Multithreaded programs } + Abstraction 4: } while (new != old); 4: } while (new != old); 5: unlock (); + Behaviors encoded in logic 5: unlock (); + Type/Flow analyses return ; + Decision procedures return ; } } Precise [ESC, PCC] Scalable [Cqual, ESP] 47 48

8 Combining Strengths For Next Time

Theorem Proving Program Analysis • Read “Lazy Abstraction” - Need loop invariants - Imprecise – for the main ideas, ok to skim Sec. 7 (will find automatically) (will be precise) + Behaviors encoded in logic SLAM + Abstraction (used to refine abstraction) (will shrink the state + Theorem provers space we must explore) (used to compute successors, refine abstraction) Model Checking

- Finite-state model, state explosion (will find small good model) + State space exploration (used to get a path sensitive analysis) + Counterexamples (used to find relevant facts, refine abstraction) 49 50

9