A Tutorial on Automated Verification Tevfik Bultan

A Tutorial on Automated Verification

Tevfik Bultan

Who are these people and what do they have in common?

2007 Clarke, Edmund M2007 Emerson, E Allen2007 Sifakis, Joseph1996 Pnueli, Amir1991 Milner, Robin1980 Hoare, C. Antony R.1978 Floyd, Robert W1972 Dijkstra, E. W.

Outline

• Software’s Chronic Crisis• Temporal Logics and Model Checking Problem• Symbolic Model Checking• Automata Theoretic Model Checking • Software Verification Using Explicit State Model Checking

with Java Path Finder• Bounded Model Checking• Symbolic Software Model Checking with Predicate

Abstraction and Counter-Example Guided Abstraction Refinement

Software’s Chronic Crisis

Large software systems often:• Do not provide the desired functionality• Take too long to build• Cost too much to build• Require too much resources (time, space) to run• Cannot evolve to meet changing needs

– For every 6 large software projects that become operational, 2 of them are canceled

– On the average software development projects overshoot their schedule by half

– 3 quarters of the large systems do not provide required functionality

Software Failures

• There is a long list of failed software projects and software failures

• You can find a list of famous software bugs at:http://www5.in.tum.de/~huckle/bugse.html

• I will talk about two famous and interesting software bugs

Ariane 5 Failure

• A software bug caused European Space Agency’s Ariane 5 rocket to crash 40 seconds into its first flight in 1996 (cost: half billion dollars)

• The bug was caused because of a software component that was being reused from Ariane 4

• A software exception occurred during execution of a data conversion from 64-bit floating point to 16-bit signed integer value– The value was larger than 32,767, the largest integer storable in a

16 bit signed integer, and thus the conversion failed and an exception was raised by the program

• When the primary computer system failed due to this problem, the secondary system started running.– The secondary system was running the same software, so it failed

too!

Ariane 5 Failure

• The programmers for Ariane 4 had decided that this particular velocity figure would never be large enough to raise this exception. – Ariane 5 was a faster rocket than Ariane 4!

• The calculation containing the bug actually served no purpose once the rocket was in the air. – Engineers chose long ago, in an earlier version of the

Ariane rocket, to leave this function running for the first 40 seconds of flight to make it easy to restart the system in the event of a brief hold in the countdown.

• You can read the report of Ariane 5 failure at: http://www.ima.umn.edu/~arnold/disasters/ariane5rep.html

Mars Pathfinder

• Priority inversion occurs when – a thread that has higher priority is waiting for a resource held by

thread with a lower priority• Pathfinder contained a data bus shared among multiple threads and

protected by a mutex lock• Two threads that accessed the data bus were: a high-priority bus

management thread and a low-priority meteorological data gathering thread

• Yet another thread with medium-priority was a long running communications thread (which did not access the data bus)

• A few days into its mission, NASA’s Mars Pathfinder computer system started rebooting itself – Cause: Priority inversion during preemptive

priority scheduling of threads

Mars Pathfinder

• The scenario that caused the reboot was:– The meteorological data gathering thread accesses the bus and obtains the

mutex lock– While the meteorological data gathering thread is accessing the bus, an

interrupt causes the high-priority bus management thread to be scheduled– Bus management thread tries to access the bus and blocks on the mutex

lock– Scheduler starts running the meteorological thread again– Before the meteorological thread finishes its task yet another interrupt

occurs and the medium-priority (and long running) communications thread gets scheduled

– At this point high-priority bus management thread is waiting for the low-priority meteorological data gathering thread, and the low-priority meteorological data gathering thread is waiting for the medium-priority communications thread

– Since communications thread had long-running tasks, after a while a watchdog timer would go off and notice that the high-priority bus management thread has not been executed for some time and conclude that something was wrong and reboot the system

Software’s Chronic Crisis

• Software product size is increasing exponentially– faster, smaller, cheaper hardware

• Software is everywhere: from TV sets to cell-phones• Software is in safety-critical systems

– cars, airplanes, nuclear-power plants• We are seeing more of

– distributed systems– embedded systems– real-time systems

• These kinds of systems are harder to build• Software requirements change

– software evolves rather than being built

Summary

• Software’s chronic crisis: Development of large software systems is a challenging task– Large software systems often: Do not provide the

desired functionality; Take too long to build; Cost too much to build Require too much resources (time, space) to run; Cannot evolve to meet changing needs

What is this?

First Computer Bug

• In 1947, Grace Murray Hopper was working on the Harvard University Mark II Aiken Relay Calculator (a primitive computer).

• On the 9th of September, 1947, when the machine was experiencing problems, an investigation showed that there was a moth trapped between the points of Relay #70, in Panel F.

• The operators removed the moth and affixed it to the log. The entry reads: "First actual case of bug being found."

• The word went out that they had "debugged" the machine and the term "debugging a computer program" was born.

Can Model Checking Help

• The question is: Can the automated verification techniques we have been discussing be used in finding bugs in software systems?

• Today I will discuss some automated verification techniques that have been successful in identifying bugs

State of the art in automated verification: Model Checking• What is model checking?

– Automated verification technique– Focuses on bug finding rather than proving correctness– The basic idea is to exhaustively search for bugs in

software– Has many flavors

• Explicit-state model checking• Symbolic model checking• Bounded model checking

Model Checking Evolution

• Earlier model checkers had their own input specification languages– For example Spin, SMV

• This requires translation of the system to be verified to the input langauge of the model checker– Most of the time these translations are not automated

and use ad-hoc simplifications and abstractions• More recently several researchers developed tools for

model checking programs– These model checkers work directly on programs, i.e.,

their input language is a programming language– These model checkers use well-defined techniques for

restricting the state space or use automated abstraction techniques

Explicit-State Model Checking Programs

• Verisoft from Bell Labs– C programs, handles concurrency, bounded search,

bounded recursion. – Uses stateless search and partial order reduction.

• Java Path Finder (JPF) at NASA Ames– Explicit state model checking for Java programs,

bounded search, bounded recursion, handles concurrency.

– Uses techniques similar to the techniques used in Spin.

• CMC from Stanford for checking systems code written in C

Symbolic Model Checking of Programs

• CBMC – This is the bounded model checker we discussed earlier,

bounds the loop iterations and recursion depth.– Uses a SAT solver.

• SLAM project at Microsoft Research– Symbolic model checking for C programs. Can handle

unbounded recursion but does not handle concurrency. – Uses predicate abstraction and BDDs.

Beyond Model Checking

• Promising results obtained in the model checking area created a new interest in automated verification

• Nowadays, there is a wide spectrum of verification/analysis/testing techniques with varying levels of power and scalability– Bounded verification using SAT solvers– Symbolic execution using combinations of decision

procedures– Dynamic symbolic execution (aka concolic execution)– Various types of symbolic analysis: shape analysis,

string analysis, size analysis, etc.

What to Verify?

• Before we start talking about automated verification techniques, we need to identify what we want to verify

• It turns out that this is not a very simple question

• First we will discuss issues related to this question

Temporal Logics and Model Checking Problem

A Mutual Exclusion Protocol

Process 1:while (true) { out: a := true; turn := true; wait: await (!b or !turn); cs: a := false;}||Process 2:while (true) { out: b := true; turn := false; wait: await (!a or turn); cs: b := false;}

Two concurrently executing processes are trying to enter a critical section without violating mutual exclusion

Reactive Systems: A Very Simple Model

• We will use a very simple model for reactive systems

• A reactive system generates a set of execution paths

• An execution path is a concatenation of the states (configurations) of the system, starting from some initial state

• There is a transition relation which specifies the next-state relation, i.e., given a state what are the states that can follow that state

State Space

• The state space of a program can be captured by the valuations of the variables and the program counters

• For our example, we have– two program counters: pc1, pc2

domains of the program counters: {out, wait, cs}– three boolean variables: turn, a, b

boolean domain: {True, False}

• Each state of the program is a valuation of all the variables

State Space

• Each state can be written as a tuple (pc1,pc2,turn,a,b)

• Initial states: {(o,o,F,F,F), (o,o,F,F,T), (o,o,F,T,F), (o,o,F,T,T), (o,o,T,F,F), (o,o,T,F,T), (o,o,T,T,F), (o,o,T,T,T)}– initially: pc1=o and pc2=o

• How many states total?3 * 3 * 2 * 2 * 2 = 72exponential in the number of variables and the number of

concurrent components

Transition Relation

• Transition Relation specifies the next-state relation, i.e., given a state what are the states that can come immediately after that state

• For example, given the initial state (o,o,F,F,F) Process 1 can execute: out: a := true; turn := true;or Process 2 can execute:out: b := true; turn := false;

• If process 1 executes, the next state is (w,o,T,T,F)• If process 2 executes, the next state is (o,w,F,F,T)• So the state pairs ((o,o,F,F,F),(w,o,T,T,F)) and ((o,o,F,F,F),(o,w,F,F,T)) are included in the transition relation

Transition Relation

The transition relation is like a graph, edges represent the next-state relation

(o,o,F,F,F)

(o,w,F,F,T) (w,o,T,T,F)

(o,c,F,F,T) (w,w,T,T,T)

Transition System

• A transition system T = (S, I, R) consists of– a set of states S– a set of initial states I S– and a transition relation R S S

• A common assumption in model checking– R is total, i.e., for all s S, there exists s’ such that (s,s’) R

Execution Paths

• An execution path is an infinite sequence of statesx = s0, s1, s2, ...

such that s0 I and for all i 0, (si,si+1) R

Notation: For any path xxi denotes the i’th state on the path (i.e., si)

xi denotes the i’th suffix of the path (i.e., si, si+1, si+2, ... )

Execution Paths

A possible execution path:((o,o,F,F,F), (o,w,F,F,T), (o,c,F,F,T)) ( means repeat the above three states infinitely many times)

(o,o,F,F,F)

(o,w,F,F,T) (w,o,T,T,F)

(o,c,F,F,T) (w,w,T,T,T)

Temporal Logics

• Pnueli proposed using temporal logics for reasoning about the properties of reactive systems

• Temporal logics are a type of modal logics– Modal logics were developed to express modalities such

as “necessity” or “possibility”– Temporal logics focus on the modality of temporal

progression

• Temporal logics can be used to express, for example, that:– an assertion is an invariant (i.e., it is true all the time)– an assertion eventually becomes true (i.e., it will become

true sometime in the future)

Temporal Logics

• We will assume that there is a set of basic (atomic) properties called AP– These are used to write the basic (non-temporal)

assertions about the program– Examples: a=true, pc0=c, x=y+1

• We will use the usual boolean connectives: , ,

• We will also use four temporal operators:Invariant p : G p (aka p) (Globally)Eventually p : F p (aka p) (Future)Next p : X p (aka p) (neXt)p Until q : p U q

LTL Properties

p. . .

pp p p p p. . .

p. . .

pp p p q. . .

X p

G p

F p

p U q

Example Properties

mutual exclusion: G ( (pc1=c pc2=c))starvation freedom:

G(pc1=w F(pc1=c)) G(pc2=w F(pc2=c))

Given the execution path:x =((o,o,F,F,F), (o,w,F,F,T), (o,c,F,F,T))

x |= pc1=ox |= X (pc2=w)x |= F (pc2=c)x |= (turn) U (pc2=c b)x |= G ( (pc1=c pc2=c))x |= G(pc1=w F(pc1=c)) G(pc2=w F(pc2=c))

LTL Model Checking

• Given a transition system T and an LTL property p T |= p iff for all execution paths x in T, x |= p

For example:T |=? G ( (pc1=c pc2=c))T |=? G(pc1=w F(pc1=c)) G(pc2=w F(pc2=c))

Model checking problem: Given a transition system T and an LTL property p, determine if T is a model for p (i.e., if T |=p)

Complexity: (|S|+|R|) 2O(|f|)

Linear Time vs. Branching Time

• In linear time logics we look at the execution paths individually

• In branching time logics we view the computation as a tree– computation tree: unroll the transition relation

s2s1 s4s3

Transition System Execution Paths Computation Tree

s3

s4

s3

s3

s1

s2...

...

s3

s4

s3

s3

s1

s2

......

s3s4 s1...

......

s4 s1...

Computation Tree Logic (CTL)

• In CTL we quantify over the paths in the computation tree

• We use the same four temporal operators: X, G, F, U

• However we attach path quantifiers to these temporal operators:– A : for all paths– E : there exists a path

• We end up with eight temporal operators:– AX, EX, AG, EG, AF, EF, AU, EU

CTL Properties

s2s1 s4s3

Transition System Computation Tree

s4

s3

s3

s1

s2

......

s3s4 s1...

......

s4 s1...

p p p

p

p

p

s3 |= ps4 |= ps1 |= ps2 |= p

s3 |= EX ps3 |= EX ps3 |= AX ps3 |= AX ps3 |= EG ps3 |= EG ps3 |= AF ps3 |= EF ps3 |= AF p

p

p

CTL Model Checking

• Given a transition system T= (S, I, R) and a CTL property p T |= p iff for all initial state s I, s |= p

Model checking problem: Given a transition system T and a CTL property p, determine if T is a model for p (i.e., if T |=p)

Complexity: O(|f| (|S|+|R|))

For example:T |=? AG ( (pc1=c pc2=c))T |=? AG(pc1=w AF(pc1=c)) AG(pc2=w AF(pc2=c))

• Question: Are CTL and LTL equivalent?

CTL vs. LTL

• CTL and LTL are not equivalent

– There are properties that can be expressed in LTL but cannot be expressed in CTL

• For example: FG p

– There are properties that can be expressed in CTL but cannot be expressed in LTL

• For example: AG(EF p)

• Hence, expressive power of CTL and LTL are not comparable

Symbolic Model Checking

Temporal Properties Fixpoints [Emerson and Clarke 80]

Here are some interesting CTL equivalences:

AG p = p AX AG pEG p = p EX EG p

AF p = p AX AF pEF p = p EX EF p

p AU q = q (p AX (p AU q))p EU q = q (p EX (p EU q))

Note that we wrote the CTL temporal operators in terms of themselves and EX and AX operators

Fixpoint Characterizations

Fixpoint Characterization Equivalences

AG p = y . p AX y AG p = p AX AG pEG p = y . p EX y EG p = p EX EG p

AF p = y . p AX y AF p = p AX AF pEF p = y . p EX y EF p = p EX EF p

p AU q = y . q (p AX (y)) p AU q=q (p AX (p AU q))p EU q = y . q (p EX (y)) p EU q = q (p EX (p EU q))

Least Fixpoint

Given a monotonic function F, the least fixpoint y . F y is the limit of the following sequence (assuming F is -continuous):

, F , F2 , F3 , ...

If S is finite, then we can compute the least fixpoint using the above sequence

EF Fixpoint Computation

EF p = y . p EX y is the limit of the sequence:

, pEX , pEX(pEX ) , pEX(pEX(p EX )) , ...

which is equivalent to

, p, p EX p , p EX (p EX (p) ) , ...


s2s1 s4s3p

p

Start

1st iterationpEX = {s1,s4} EX()= {s1,s4} ={s1,s4}

2nd iterationpEX(pEX ) = {s1,s4} EX({s1,s4})= {s1,s4} {s3}={s1,s3,s4}

3rd iterationpEX(pEX(p EX )) = {s1,s4} EX({s1,s3,s4})= {s1,s4} {s2,s3,s4}={s1,s2,s3,s4}

4th iterationpEX(pEX(pEX(p EX ))) = {s1,s4} EX({s1,s2,s3,s4})= {s1,s4} {s1,s2,s3,s4} = {s1,s2,s3,s4}


• • •• • •pp

EF(EF(pp)) states that can reach states that can reach p p p p EX( EX(pp)) EX(EX( EX(EX(pp)) )) ......

EF(EF(pp))

Greatest Fixpoint

Given a monotonic function F, the greatest fixpoint y . F y is the limit of the following sequence (assuming F is -continuous):

S, F S, F2 S, F3 S, ...

If S is finite, then we can compute the greatest fixpoint using the above sequence

EG Fixpoint Computation

Similarly, EG p = y . p EX y is the limit of the sequence:

S, pEX S, pEX(p EX S) , pEX(p EX (p EX S)) , ...

which is equivalent to

S, p, p EX p , p EX (p EX (p) ) , ...


s2s1 s4s3

p p

p

StartS = {s1,s2,s3,s4}

1st iterationpEX S = {s1,s3,s4}EX({s1,s2,s3,s4})= {s1,s3,s4}{s1,s2,s3,s4}={s1,s3,s4}

2nd iterationpEX(pEX S) = {s1,s3,s4}EX({s1,s3,s4})= {s1,s3,s4}{s2,s3,s4}={s3,s4}

3rd iterationpEX(pEX(pEX S)) = {s1,s3,s4}EX({s3,s4})= {s1,s3,s4}{s2,s3,s4}={s3,s4}


• • •• • • EG(EG(p)p)

EG(EG(pp)) states that can avoid reaching states that can avoid reaching pp p p EX( EX(pp)) EX(EX( EX(EX(pp)))) ......

Symbolic Model Checking[McMillan et al. LICS 90]

• Basic idea: Represent sets of states and the transition relation as Boolean logic formulas

• Fixpoint computation becomes formula manipulation– pre-condition (EX) computation: Existential variable

elimination– conjunction (intersection), disjunction (union) and

negation (set difference), and equivalence check

• Use an efficient data structure for boolean logic formulas – Binary Decision Diagrams (BDDs)

Symbolic Pre-condition Computation

• Remember the functionEX : 2S 2S

which is defined as:EX(p) = { s | (s,s’) R and s’ p }

• We can symbolically compute pre as followsEX(p) V’ R p[V’ / V]– V : current-state boolean variables – V’ : next-state boolean variables– p[V’ / V] : rename variables in p by replacing current-

state variables with the corresponding next-state variables

V’ f : existentially quantify out all the variables in V’ from f

An Extremely Simple Example

Variables: x, y: boolean

Set of states:S = {(F,F), (F,T), (T,F), (T,T)}S True

Initial condition:I x y

Transition relation (negates one variable at a time):R x’=x y’=y x’=x y’=y (= means )

F,T

F,F

T,T

T,F


Given p x y, compute EX(p)

EX(p) V’ R p[V’ / V] V’ R x’ y’ V’ (x’=x y’=y x’=x y’=y ) x’ y’ V’ (x’=x y’=y) x’ y’ (x’=x y’=y) x’ y’ V’ x y x’ y’ x y x’ y’ x y x y

EX(x y) x y x yIn other words EX({(T,T)}) {(F,T), (T,F)}

F,T

F,F

T,T

T,F


Let’s compute compute EF(x y)

The fixpoint sequence isFalse, xy , xy EX(xy) , xy EX (xy EX(xy)) , ...If we do the EX computations, we get:False, x y , x y x y x y, True

EF(x y) TrueIn other words EF({(T,T)}) {(F,F),(F,T), (T,F),(T,T)}

F,T

F,F

T,T

T,F

0 1 2 3

1

2

3


• Based on our results, for our extremely simple transition system T=(S,I,R) we have

I EF(x y) hence: T |= EF(x y) (i.e., there exists a path from each initial state where

eventually x and y both become true at the same time)

I EX(x y) hence:T |= EX(x y) (i.e., there does not exist a path from each initial state where

in the next state x and y both become true)


• Let’s try one more property AF(x y)

• To check this property we first convert it to a formula which uses only the temporal operators in our basis:

AF(x y) EG((x y))

If we can find an initial state which satisfies EG((x y)), then we know that the transition system T, does not satisfy the property AF(x y)


Let’s compute compute EG((x y))

The fixpoint sequence isTrue, x y, (x y) EX(x y) , …

If we do the EX computations, we get:True, x y, x y,

EG((x y)) x ySince I EG((x y)) we conclude that T |= AF(x y)

F,T

F,F

T,T

T,F

0 1 2

0

1

Symbolic CTL Model Checking Algorithm

• Translate the formula to a formula which uses the basis – EX p, EG p, p EU q

• Atomic formulas can be interpreted directly on the state representation

• For EX p compute the precondition using existential variable elimination as we discussed

• For EG and EU compute the fixpoints iteratively

SMV [McMillan 93]

• BDD-based symbolic model checker • Finite state• Temporal logic: CTL• Focus: hardware verification

– Later applied to software specifications, protocols, etc.• SMV has its own input specification language

– concurrency: synchronous, asynchronous – shared variables– boolean and enumerated variables– bounded integer variables (binary encoding)

• SMV is not efficient for integers, but that can be fixed– fixed size arrays

SMV Language

• An SMV specification consists of a set of modules (one of them must be called main)

• Modules can have access to shared variables• Modules can be composed asynchronously using the process keyword

• Module behaviors can be specified using the ASSIGN statement which assigns values to next values of variables in parallel

• Module behaviors can also be specified using the TRANS statements which allow specification of the transition relation as a logic formula where next state values are identified using the next keyword

Example Mutual Exclusion Protocol

Process 1:while (true) { out: a := true; turn := true; wait: await (b = false or turn = false); cs: a := false;}||Process 2:while (true) { out: b := true; turn := false; wait: await (a = false or turn); cs: b := false;}


Example Mutual Exclusion Protocol in SMV

MODULE process1(a,b,turn)VAR pc: {out, wait, cs};ASSIGN init(pc) := out; next(pc) := case pc=out : wait; pc=wait & (!b | !turn) : cs; pc=cs : out; 1 : pc; esac; next(turn) := case pc=out : 1; 1 : turn; esac; next(a) := case pc=out : 1; pc=cs : 0; 1 : a; esac; next(b) := b;FAIRNESS running

MODULE process2(a,b,turn)VAR pc: {out, wait, cs};ASSIGN init(pc) := out; next(pc) := case pc=out : wait; pc=wait & (!a | turn) : cs; pc=cs : out; 1 : pc; esac; next(turn) := case pc=out : 0; 1 : turn; esac; next(b) := case pc=out : 1; pc=cs : 0; 1 : b; esac; next(a) := a;FAIRNESS running


MODULE mainVAR a : boolean; b : boolean; turn : boolean; p1 : process process1(a,b,turn); p2 : process process2(a,b,turn);SPEC AG(!(p1.pc=cs & p2.pc=cs)) -- AG(p1.pc=wait -> AF(p1.pc=cs)) & AG(p2.pc=wait -> AF(p2.pc=cs))

Here is the output when I run SMV on this example to check the mutual exclusion property

% smv mutex.smv-- specification AG (!(p1.pc = cs & p2.pc = cs)) is true

resources used:user time: 0.01 s, system time: 0 sBDD nodes allocated: 692Bytes allocated: 1245184BDD nodes representing transition relation: 143 + 6


The output for the starvation freedom property:

% smv mutex.smv-- specification AG (p1.pc = wait -> AF p1.pc = cs) & AG ... is true

resources used:user time: 0 s, system time: 0 sBDD nodes allocated: 1251Bytes allocated: 1245184BDD nodes representing transition relation: 143 + 6


Let’s insert an error

change pc=wait & (!b | !turn) : cs;

to pc=wait & (!b | turn) : cs;

% smv mutex.smv-- specification AG (!(p1.pc = cs & p2.pc = cs)) is false-- as demonstrated by the following execution sequencestate 1.1:a = 0b = 0turn = 0p1.pc = outp2.pc = out[stuttering]

state 1.2:[executing process p2]

state 1.3:b = 1p2.pc = wait[executing process p2]

state 1.4:p2.pc = cs[executing process p1]

state 1.5:a = 1turn = 1p1.pc = wait[executing process p1]

state 1.6:p1.pc = cs[stuttering]

resources used:user time: 0.01 s, system time: 0 sBDD nodes allocated: 1878Bytes allocated: 1245184BDD nodes representing transition relation: 143 + 6

Symbolic Model Checking with BDDs

• BDDs are used as a data structure for encoding trust sets of Boolean logic formulas in symbolic model checking

• One can use BDD-based symbolic model checking for any finite state system using a Boolean encoding of the state space and the transition relation

• Why are we using symbolic model checking?– We hope that the symbolic representations will be more

compact than the explicit state representation on the average

– In the worst case we may not gain anything

Problems with BDDs

• The BDD for the transition relation could be huge– Remember that the BDD could be exponential in the

number of disjuncts and conjuncts– Since we are using a Boolean encoding there could be a

large number of conjuncts and disjuncts • The EX computation could result in exponential blow-up

– Exponential in the number of existentially quantified variables

Heuristics

• Instead of computing a monolithic BDD for the whole transition system partition the transition relation in order to keep the BDD size small

• Use good variable ordering in order to keep the BDD sizes small– Use heuristics to find good variable orderings, – Use dynamic variable ordering heuristics that change the

variable ordering dynamically if the BDD size grows too much

• Use other data structures (such as multi-terminal decision diagrams)

Counter-Example Generation

• Remember: Given a transition system T= (S, I, R) and a CTL property p T |= p iff for all initial state s I, s |= p

• Verification vs. Falsification– Verification:

• Show: initial states truth set of p– Falsification:

• Find: a state initial states truth set of p• Generate a counter-example starting from that state

• The ability to find counter-examples is one of the biggest strengths of the model checkers

An Example

• • •• • •pp

EF(EF(pp)) states that can reach states that can reach p p p p EX( EX(pp)) EX(EX( EX(EX(pp)) )) ......

EF(EF(pp))

• We want to check the property AG(p) • We compute the fixpoint for EF(p)• We check if the intersection of the set of initial states I and

the truth set of EF(p) is empty– If it is not empty we generate a counter-example path

starting from the intersection

I

Generate a counter-examplepath starting from a state here

• In order to generate the counter-example path, savethe fixpoint iterations.• After the fixpoint computationconverges, do a second pass to generate the counter-example path.

• • •• • •

Automata Theoretic Model Checking

LTL Properties Büchi automata [Vardi and Wolper LICS 86]

• Büchi automata: Finite state automata that accept infinite strings– The better known variant of finite state automata accept

finite strings (used in lexical analysis for example)

• A Büchi automaton accepts a string when the corresponding run visits an accepting state infinitely often– Note that an infinite run never ends, so we cannot say

that an accepting run ends at an accepting state

• LTL properties can be translated to Büchi automata – The automaton accepts a path if and only if the path

satisfies the corresponding LTL property

LTL Properties Büchi automata

G p p ptrue

F p pptrue

G (F p) p

The size of the property automaton can be exponential in the size of the LTL formula (recall the complexity of LTL model checking)

p

p

p

Büchi Automata: Language Emptiness Check

• Given a Buchi automaton, one interesting question is: – Is the language accepted by the automaton empty?

• i.e., does it accept any string?• A Büchi automaton accepts a string when the

corresponding run visits an accepting state infinitely often• To check emptiness:

– Look for a cycle which contains an accepting state and is reachable from the initial state

• Find a strongly connected component that contains an accepting state, and is reachable from the initial state

– If no such cycle can be found the language accepted by the automaton is empty

LTL Model Checking

• Generate the property automaton from the negated LTL property

• Generate the product of the property automaton and the transition system

• Show that there is no accepting cycle in the product automaton (check language emptiness)– i.e., show that the intersection of the paths generated by

the transition system and the paths accepted by the (negated) property automaton is empty

• If there is a cycle, it corresponds to a counterexample behavior that demonstrates the bug

LTL Model Checking Example

G q

Each state is labeled with the propositions that hold in that state

Example transition systemProperty to be verified

Negation of the property G q F q

Property automaton for the negated property

q qtrue

2

1

3

p,q

q p

{q},{p,q}

, {p},{p},{q}, {p,q}

1 2

Equivalently

Transition System to Buchi Automaton Translation

2

1

3

Each state is labeled with the propositions that hold in that state

Example transition system Corresponding Buchi automaton

{p,q}

{p}

{q}

{p,q}

{q}

i

1

2 3

p,q

q p

{p,q}

{p}

{q}

{p,q}

{q}

{q},{p,q}

, {p},{p},{q}, {p,q}

1

2

3 4

1 2

Product automaton

{p,q}

{p}

{q}

{p,q}

1,1

2,1

3,1

4,2

{q}

3,2

{p}

Buchi automaton forthe transition system(every state is accepting)

Property AutomatonAccepting cycle:(1,1), (2,1), (3,1), ((4,2), (3,2))

Corresponds to a counter-examplepath for the property G q

SPIN [Holzmann 91, TSE 97]

• Explicit state model checker• Finite state• Temporal logic: LTL• Input language: PROMELA

– Asynchronous processes – Shared variables – Message passing through (bounded) communication

channels– Variables: boolean, char, integer (bounded), arrays

(fixed size)– Structured data types

SPIN

Verification in SPIN• Uses the LTL model checking approach• Constructs the product automaton on-the-fly

– It is possible to find an accepting cycle (i.e. a counter-example) without constructing the whole state space

• Uses a nested depth-first search algorithm to look for an accepting cycle

• Uses various heuristics to improve the efficiency of the nested depth first search:– partial order reduction– state compression

Example Mutual Exclusion Protocol

Process 1:while (true) { out: a := true; turn := true; wait: await (b = false or turn = false); cs: a := false;}||Process 2:while (true) { out: b := true; turn := false; wait: await (a = false or turn); cs: b := false;}


Example Mutual Exclusion Protocol in Promela

#define cs1 process1@cs#define cs2 process2@cs#define wait1 process1@wait#define wait2 process2@wait#define true 1#define false 0bool a;bool b;bool turn;proctype process1(){out: a = true; turn = true;wait: (b == false || turn == false);cs: a = false; goto out;}proctype process2(){out: b = true; turn = false;wait: (a == false || turn == true);cs: b = false; goto out;}init { run process1(); run process2()}

Property automaton generation% spin -f "! [] (! (cs1 && cs2))“

never { /* ! [] (! (cs1 && cs2)) */T0_init: if :: ((cs1) && (cs2)) -> goto accept_all :: (1) -> goto T0_init fi;accept_all: skip}

% spin -f "!([](wait1 -> <>(cs1)))“

never { /* !([](wait1 -> <>(cs1))) */T0_init: if :: ( !((cs1)) && (wait1) ) -> goto accept_S4 :: (1) -> goto T0_init fi;accept_S4: if :: (! ((cs1))) -> goto accept_S4 fi;}

Concatanate the generated never claims to the end of the specification file

• Input formula“[]” means G“<>” means F

• “spin –f” option generates a Buchi automaton for the input LTL formula

SPIN

• “spin –a mutex.spin” generates a C program “pan.c” from the specification file – This C program implements the on-the-fly nested-depth

first search algorithm– You compile “pan.c” and run it to the model checking

• Spin generates a counter-example trace if it finds out that a property is violated

%mutex -awarning: for p.o. reduction to be valid the never claim must be stutter-invariant(never claims generated from LTL formulae are stutter-invariant)(Spin Version 4.2.6 -- 27 October 2005) + Partial Order Reduction

Full statespace search for: never claim + assertion violations + (if within scope of claim) acceptance cycles + (fairness disabled) invalid end states - (disabled by never claim)

State-vector 28 byte, depth reached 33, errors: 0 22 states, stored 15 states, matched 37 transitions (= stored+matched) 0 atomic stepshash conflicts: 0 (resolved)

2.622 memory usage (Mbyte)

unreached in proctype process1 line 18, state 6, "-end-" (1 of 6 states)unreached in proctype process2 line 27, state 6, "-end-" (1 of 6 states)unreached in proctype :init: (0 of 3 states)

Problems/Heuristics for explicit state model checking

• State space explosion: Number of states can be exponential in the number of variables and concurrent components

• Heuristics used by Spin:– On the fly checking: use a depth first search that

computes the product of the property automaton and the transition relation while looking for a violation

– Bit-state hashing • do not store the full state information• might skip some unvisited states so it is not sound

– Partial order reduction • only explore a representative subset of interleavings

among the concurrent processes • this can be done in a sound manner

Software Verification Using Explicit State Model Checking with Java Path Finder

Java Path Finder

• Program checker for Java• Properties to be verified

– Properties can be specified as assertions• static checking of assertions

– It can also verify LTL properties• Implements both depth-first and breadth-first search and

looks for assertion violations statically• Uses static analysis techniques to improve the efficiency of

the search• Requires a complete Java program• It can only handle pure Java, it cannot handle native code

Java Path Finder, First Version

• First version – A translator from Java to PROMELA– Use SPIN for model checking

• Since SPIN cannot handle unbounded data– Restrict the program to finite domains

• A fixed number of objects from each class• Fixed bounds for array sizes

• Does not scale well if these fixed bounds are increased

• Java source code is required for translation

Java Path Finder, Current Version

• Current version of the JPF has its own virtual machine: JPF-JVM– Executes Java bytecode

• can handle pure Java but can not handle native code– Has its own garbage collection– Stores the visited states and stores current path – Offers some methods to the user to optimize verification

• Traversal algorithm – Traverses the state-graph of the program– Tells JPF-JVM to move forward, backward in the state space, and evaluate the assertion

• The rest of the slides are on the current version of JPF:W. Visser, K. Havelund, G. Brat, S. Park and F. Lerda. "Model

Checking Programs." Automated Software Engineering Journal Volume 10, Number 2, April 2003.

Storing the States

• JPF implements a depth-first search on the state space of the given Java program– To do depth first search we need to store the visited

states• There are also verification tools which use stateless

search such as Verisoft• The state of the program consists of

– information for each thread in the Java program• a stack of frames, one for each method called

– the static variables in classes• locks and fields for the classes

– the dynamic variables (fields) in objects• locks and fields for the objects

Storing States Efficiently

• Since different states can have common parts each state is divided to a set of components which are stored separately– locks, frames, fields

• Keep a pool for each component– A table of field values, lock values, frame values

• Instead of storing the value of a component in a state store an index at which the component is stored in the table in the state– The whole state becomes an integer vector

• JPF collapses states to integer vectors using this idea

State Space Explosion

• State space explosion if one of the major challenges in model checking

• The idea is to reduce the number of states that have to be visited during state space exploration

• Here are some approaches used to attack state space explosion– Symmetry reduction

• search equivalent states only once – Partial order reduction

• do not search thread interleavings that generate equivalent behavior

– Abstraction• Abstract parts of the state to reduce the size of the state

space

Symmetry Reduction

• Some states of the program may be equivalent– Equivalent states should be searched only once

• Some states may differ only in their memory layout, the order objects are created, etc. – these may not have any effect on the behavior of the

program• JPF makes sure that the order which the classes are

loaded does not effect the state – There is a canonical ordering of the classes in the

memory

Symmetry Reduction

• A similar problem occurs for location of dynamically allocated objects in the heap– If we store the memory location as the state, then we

can miss equivalent states which have different memory layouts

– JPF tries to remedy this problem by storing some information about the new statements that create an object and the number of times they are executed

Partial Order Reduction

• Statements of concurrently executing threads can generate many different interleavings– all these different interleavings are allowable behavior of

the program• A model checker has to check all possible interleavings that

the behavior of the program is correct in all cases – However different interleavings may generate equivalent

behaviors• In such cases it is sufficient to check just one interleaving

without exhausting all the possibilities– This is called partial order reduction

class S1 { int x;}class FirstTask extends Thread { public void run() { S1 s1; int x = 1; s1 = new S!(); x = 3;}}

class Main { public static void main(String[] args) { FirstTask task1 = new FirstTask(); SecondTask task2 = new SecondTask(); task1.statr(); task2.start();}}

class S2 { int y;}class SecondTask extends Thread { public void run() { S2 s2; int x = 1; s2 = new S2(); x = 3;}}

state space search generates 258 stateswith symmetry reduction: 105 stateswith partial order reduction: 68 stateswith symmetry reduction + partial order reduction : 38 states

Static Analysis

• JPF uses following static analysis techniques for reducing the state space:– slicing– partial evaluation

• Given a slicing criterion slicing reduces the size of a program by removing the parts of the program that have no effect on the slicing criterion– A slicing criterion could be a program point– Program slices are computed using dependency

analysis• Partial evaluation propagates constant values and simplifies

expressions

Abstraction vs. Restriction

• JPF also uses abstraction techniques such as predicate abstraction to reduce the state space

• Still, in order to check a program with JPF, typically, you need to restrict the domains of the variables, the sizes of the arrays, etc.

• Abstraction over approximates the program behavior– causes spurious counter-examples

• Restriction under approximates the program behavior– may result in missed errors

• If both under and over approximation techniques are used then the resulting verification technique is neither sound nor complete– However, it is still useful as a debugging tool and it is

helpful in finding bugs

JPF Java Modeling Primitives

• Atomicity (used to reduce the state space)– beginAtomic(), endAtomic()

• Nondeterminism (used to model non-determinism caused by abstraction)– int random(int); boolean randomBool(); Object randomObject(String cname);

• Properties (used to specify properties to be verified)– AssertTrue(boolean cond)

Annotated Java Code for a Reader-Writer Lock

import gov.nasa.arc.ase.jpf.jvm.Verify;class ReaderWriter {private int nr;private boolean busy;private Object Condr_enter;private Object Condw_enter;public ReaderWriter() { Verify.beginAtomic(); nr = 0; busy=false ; Condr_enter =new Object(); Condw_enter =new Object(); Verify.endAtomic();}public boolean read_exit(){ boolean result=false; synchronized(this){ nr = (nr - 1); result=true; } Verify.assertTrue(!busy || nr==0 ); return result;}

private boolean Guarded_r_enter(){ boolean result=false; synchronized(this){ if(!busy){nr = (nr +

1);result=true;}} return result;}public void read_enter(){ synchronized(Condr_enter){ while (! Guarded_r_enter()){

try{Condr_enter.wait();} catch(InterruptedException e){}

}} Verify.assertTrue(!busy || nr==0 );}private boolean Guarded_w_enter(){…}public void write_enter(){…}public boolean write_exit(){…}};

JPF Output>java gov.nasa.arc.ase.jpf.jvm.Main rwmain

JPF 2.1 - (C) 1999,2002 RIACS/NASA Ames Research CenterJVM 2.1 - (C) 1999,2002 RIACS/NASA Ames Research Center

Loading class gov.nasa.arc.ase.jpf.jvm.reflection.JavaLangObjectReflectionLoading class gov.nasa.arc.ase.jpf.jvm.reflection.JavaLangThreadReflection============================== No Errors Found==============================

-----------------------------------States visited : 36,999Transitions executed : 68,759Instructions executed: 213,462Maximum stack depth : 9,010Intermediate steps : 2,774Memory used : 22.1MBMemory used after gc : 14.49MBStorage memory : 7.33MBCollected objects : 51Mark and sweep runs : 55,302Execution time : 20.401sSpeed : 3,370tr/s-----------------------------------

Example Error Trace

1 error found: Deadlock======================== *** Path to error: ***========================

Steps to error: 2521Step #0 Thread #0Step #1 Thread #0 rwmain.java:4 ReaderWriter monitor=new ReaderWriter();Step #2 Thread #0 ReaderWriter.java:10 public ReaderWriter( ) {…Step #2519 Thread #2 ReaderWriter.java:71 while (! Guarded_w_enter()){Step #2520 Thread #2 ReaderWriter.java:73 Condw_enter.wait();

Bounded Model Checking


• Represent sets of states and the transition relation as Boolean logic formulas

• Instead of computing the fixpoints, unroll the transition relation up to certain fixed bound and search for violations of the property within that bound

• Transform this search to a Boolean satisfiability problem and solve it using a SAT solver

What Can We Guarantee?

• Note that in bounded model checking we are checking only for bounded paths (paths which have at most k+1 distinct states)– So if the property is violated by only paths with more

than k+1 distinct states, we would not find a counter-example using bounded model checking

– Hence if we do not find a counter-example using bounded model checking we are not sure that the property holds

• However, if we find a counter-example, then we are sure that the property is violated since the generated counter-example is never spurious (i.e., it is always a concrete counter-example)

Bounded Model Checking: Proving Correctness

• One can also show that given an LTL property f, if E f holds for a finite state transition system, then E f also holds for that transition system using bounded semantics for some bound k

• So if we keep increasing the bound, then we are guaranteed to find a path that satisfies the formula– And, if we do not find a path that satisfies the formula,

then we decide that the formula is not satisfied by the transition system

– Is there a problem here?

Proving Correctness

• We can modify the bounded model checking algorithm as follows:– Start from an initial bound. – If no counter-examples are found using the current

bound, increment the bound and try again.

• The problem is: We do not know when to stop

Proving Correctness

• If we can find a way to figure out when we should stop then we would be able to provide guarantee of correctness.

• There is a way to define a diameter of a transition system so that a property holds for the transition system if and only if it is not violated on a path bounded by the diameter.

• So if we do bounded model checking using the diameter of the system as our bound, then we can guarantee correctness if no counter-example is found.


• What are the differences between bounded model checking and BDD-based symbolic model checking?– In bounded model checking we are using a SAT solver

instead of a BDD library– In symbolic model checking we do not unroll the

transition relation as in bounded model checking – In bounded model checking we do not execute the

iterative fixpoint computations as in symbolic model checking

– In symbolic model checking for finite state systems both verification and falsification results are guaranteed

• In bounded model checking we can only guarantee the falsification results, in order to guarantee the verification results we need to know the diameter of the system


• A bounded model checker needs an efficient SAT solver– zChaff SAT solver is one of the most commonly used

ones

• Most SAT solvers require their input to be in Conjunctive Normal Form (CNF)– So the final formula has to be converted to CNF

• Similar to BDD-based symbolic model checking, bounded model checking was also first used for hardware verification

• More recently, it has been applied to verification of software

Bounded Model Checking for Software

CBMC is a bounded model checker for ANSI-C programs

• Handles function calls using inlining

• Unwinds the loops a fixed number of times

• Allows user input to be modeled using non-determinism– So that a program can be checked for a set of inputs

rather than a single input

• Allows specification of assertions which are checked using the bounded model checking

Loops

• Unwind the loop n times by duplicating the loop body n times– Each copy is guarded using an if statement that checks

the loop condition• At the end of the n repetitions an unwinding assertion is

added which is the negation of the loop condition– Hence if the loop iterates more than n times in some

execution, the unwinding assertion will be violated and we know that we need to increase the bound in order to guarantee correctness

• A similar strategy is used for recursive function calls– The recursion is unwound up to a certain bound and

then an assertion is generated stating that the recursion does not go any deeper

A Simple Loop Example

x=0;while (x < 2) { y=y+x; x++;}

x=0;if (x < 2) { y=y+x; x++;}if (x < 2) { y=y+x; x++;}if (x < 2) { y=y+x; x++;}

assert (! (x < 2))

Original code Unwinding the loop 3 times

Unwindingassertion:

From Code to SAT

• After eliminating loops and recursion, CBMC converts the input program to the static single assignment (SSA) form– In SSA each variable appears at the left hand side of an

assignment only once– This is a standard program transformation that is

performed by creating new variables• In the resulting program each variable is assigned a value

only once and all the branches are forward branches (there is no backward edge in the control flow graph)

• CBMC generates a Boolean logic formula from the program using bit vectors to represent variables

Another Simple Example

x=x+y;if (x!=1) x=2;else x++;assert(x<=3);

x1=x0+y0;if (x1!=1) x2=2;else x3=x1+1;x4=(x1!=1)?x2:x3;assert(x4<=3);

C x1=x0+y0 x2=2 x3=x1+1 (x1!=1 x4=x2 x1=1 x4=x3)P x4 <= 3

Check if C P is satisfiable, if it is then the assertion isviolated

C P is converted to boolean logic using a bit vector representation for the integer variables y0,x0,x1,x2,x3,x4

Original code Convert to static single assignment

Generate constraints

Bounded Verification Approaches

• What we have discussed above is bounded verification by bounding the number of steps of the execution.

• For this approach to work the variable domains also need to be bounded, otherwise we cannot convert the problems to boolean SAT

• Bounding the execution steps and bounding the data domain are two orthogonal approaches. – When people say bounded verification it may refer to

either of these– When people say bounded model checking it typically

refers to bounding the execution steps

Symbolic Software Model Checking with Predicate Abstraction and Counter-Example Guided Abstraction Refinement

Model Checking Programs Using Abstraction

• Program model checking tools generally rely on automated abstraction techniques to reduce the state space of the system such as: – Abstract interpretation– Predicate abstraction

• If the abstraction is conservative then, if there is no error in the abstracted program we can conclude that there is no error in the original program

• In general the problem is to construct a finite state model from the program such that the errors or absence of errors can be demonstrated on the finite state model– Model extraction problem

Model Checking Programs via Abstraction

• Bandera– A tool for extracting finite state models from programs – Uses various abstract domains to map the state space of

the program to a finite set of states via abstraction

• SLAM project at Microsoft Research– Symbolic model checking for C programs– Can handle unbounded recursion but does not handle

concurrency – Uses predicate abstraction, counter-example guided

abstraction refinement and BDDs

Abstraction (A simplified view)

• Abstraction is an effective tool in verification

• Given a transition system, we want to generate an abstract transition system which is easier to analyze

• However, we want to make sure that– If a property holds in the abstract transition system, it

also holds in the original (concrete) transition system


• How do we generate an abstract transition system?

• Merge states in the concrete transition system (based on some criteria)– This reduces the number of states, so it should be easier

to do verification

• Do not eliminate transitions– This will make sure that the paths in the abstract

transition system subsume the paths in the concrete transition system


• For every path in the concrete transition system, there is an equivalent path in the abstract transition system– If no path in the abstract transition system violate a

property, then no path in the concrete system can violate the property

• Using this reasoning we can verify ACTL, LTL and ACTL* properties in the abstract transition system– If the property holds on the abstract transition system,

we are sure that the property holds in the concrete transition system

– If the property does not hold in the abstract transition system, then we are not sure if the property holds or not in the concrete transition system


• If the property does not hold in the abstract transition system, what can we do?

• We can refine the abstract transition system (split some states that we merged)

• We have to make sure that the refined transition system is still an abstraction of the concrete transition system

• Then, we can recheck the property again on the refined transition system– If the property does not hold again, we can refine again

Predicate Abstraction

• An automated abstraction technique which can be used to reduce the state space of a program

• The basic idea in predicate abstraction is to remove some variables from the program by just keeping information about a set of predicates about them

• For example a predicate such as x = y maybe the only information necessary about variables x and y to determine the behavior of the program– In that case we can just store a boolean variable which

corresponds to the predicate x = y and remove variables x and y from the program

– Predicate abstraction is a technique for doing such abstractions automatically

Predicate Abstraction

• Given a program and a set of predicates, predicate abstraction abstracts the program so that only the information about the given predicates are preserved

• The abstracted program adds nondeterminism since in some cases it may not be possible to figure out what the next value of a predicate will be based on the predicates in the given set

• One needs an automated theorem prover to compute the abstraction

Predicate Abstraction, A Very Simple Example• Assume that we have two integer variables x,y

• We want to abstract the program using a single predicate “x=y”

• We will divide the states of the program to two: 1. The states where “x=y” is true2. The states where “x=y” is false, i.e., “xy”

• We will then merge all the states in the same set– This is an abstraction– Basically, we forget everything except the value of the

predicate “x=y”

Predicate Abstraction, A Very Simple Example• We will represent the predicate “x=y” as the boolean variable B in

the abstract program – “B=true” will mean “x=y” and – “B=false” will mean “xy”

• Assume that we want to abstract the following program which contains only one statement:

y := y+1

Predicate Abstraction, Step 1• Calculate preconditions based on the predicate

y := y + 1 {x = y}

y := y + 1 {x y} {x y + 1}

{x = y + 1}

precondition for B being false afterexecuting the statement y:=y+1

precondition for B being true afterexecuting the statement y:=y+1

Using our temporal logic notationwe can say something like:{x=y+1} AX{x=y}

Again, using our temporal logic notation:{x≠y+1} AX{x≠y}

Predicate Abstraction, Step 2• Use decision procedures to determine if the predicates

used for abstraction imply any of the preconditions

x = y x = y + 1 ? No

x y x = y + 1 ? No

x = y x y + 1 ? Yes

x y x y + 1 ? No

Predicate Abstraction, Step 3• Generate abstract code

IF B THEN B := false ELSE B := true | falsey := y + 1

Predicate abstraction wrt the predicate “x=y”

y := y + 1 {x = y}y := y + 1 {x y} {x y + 1}

{x = y + 1}

1) Computepreconditions

x = y x = y + 1 ? No x y x = y + 1 ? No x = y x y + 1 ? Yes x y x y + 1 ? No2) Check

implications

3) Generateabstract code

Model Checking Push-down Automata

A class of infinite state systems for which model checking is decidable

• Push-down automata: Finite state control + one stack• LTL model checking for push-down automata is decidable• This may sound like a theoretical result but it is the basis of

the approach used in SLAM toolkit for model checking C programs– A program with finite data domains which uses recursion

can be modeled as a pushdown automaton– A Boolean program generated by predicate abstraction

can be represented as a pushdown automaton

Predicate Abstraction + Model Checking Push Down Automata• Predicate abstraction combined with results on model

checking pushdown automata led to some promising tools– SLAM project at Microsoft Research for verification of C

programs– This tool is being used to verify device drivers at

Microsoft• The main idea:

– Use predicate abstraction to obtain finite state abstractions of a program

– A program with finite data domains and recursion can be modeled as a pushdown automaton

– Use results on model checking push-down automata to verify the abstracted (recursive) program

SLAM Toolkit

• SLAM toolkit was developed to find errors in windows device drivers– Examples in my slides are from the following paper:

• “The SLAM Toolkit”, Thomas Ball and Sriram K. Rajamani, CAV 2001

• Windows device drivers are required to interact with the windows kernel according to certain interface rules

• SLAM toolkit has an interface specification language called SLIC (Specification Language for Interface Checking) which is used for writing these interface rules

• The SLAM toolkit instruments the driver code with assertions based on these interface rules

A SLIC Specification for a Lock

• This specification states that KeAcquireSpinLock has to be called before KeReleaseSpinLock is called,

• and KeAcquireSpinLock cannot be called back to back before a KeReleaseSpinLock is called, and vice versa

state { enum { Unlocked=0, Locked=1 } state = Unlocked;}

KeAcquireSpinLock.return { if (state == Locked) abort; else state = Locked;}

KeReleaseSpinLock.return { if (state == Unlocked) abort; else state = Unlocked;}

SLIC specification:

A SLIC Specification for a Lock

state { enum { Unlocked=0, Locked=1 } state = Unlocked;}

KeAcquireSpinLock.return { if (state == Locked) abort; else state = Locked;}

KeReleaseSpinLock.return { if (state == Unlocked) abort; else state = Unlocked;}

enum { Unlocked=0, Locked=1 } state = Unlocked; void slic_abort() { SLIC_ERROR: ;}

void KeAcquireSpinLock_return() { if (state == Locked) slic_abort(); else state = Locked;}

void KeReleaseSpinLock_return { if (state == Unlocked) slic_abort(); else state = Unlocked;}

SLIC specification: Generated C Code:

An Example

void example() { do { KeAcquireSpinLock(); nPacketsOld = nPackets; req = devExt->WLHV; if(req && req->status){ devExt->WLHV = req->Next; KeReleaseSpinLock(); irp = req->irp; if(req->status > 0){ irp->IoS.Status = SUCCESS; irp->IoS.Info = req->Status; } else { irp->IoS.Status = FAIL; irp->IoS.Info = req->Status; } SmartDevFreeBlock(req); IoCompleteRequest(irp); nPackets++; } } while(nPackets!=nPacketsOld); KeReleaseSpinLock();}

void example() { do { KeAcquireSpinLock();A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; req = devExt->WLHV; if(req && req->status){ devExt->WLHV = req->Next; KeReleaseSpinLock();B: KeReleaseSpinLock_return(); irp = req->irp; if(req->status > 0){ irp->IoS.Status = SUCCESS; irp->IoS.Info = req->Status; } else { irp->IoS.Status = FAIL; irp->IoS.Info = req->Status; } SmartDevFreeBlock(req); IoCompleteRequest(irp); nPackets++; } } while(nPackets!=nPacketsOld); KeReleaseSpinLock();C: KeReleaseSpinLock_return(); }

Instrumented code

Boolean Programs

• After instrumenting the code, the SLAM toolkit converts the instrumented C program to a Boolean program using predicate abstraction

• The Boolean program consists of only Boolean variables– The Boolean variables in the Boolean program are the

predicates that are used during predicate abstraction

• The Boolean program can have unbounded recursion

Boolean Programsdecl {state==Locked}, {state==Unlocked} := F,T;

void slic_abort() begin SLIC_ERROR: skip;end

void KeAcquireSpinLock_return()begin if ({state==Locked}) slic_abort(); else {state==Locked}, {state==Unlocked} := T,F;end

void KeReleaseSpinLock_return() begin if ({state == Unlocked}) slic_abort(); else {state==Locked}, {state==Unlocked} := F,T;end

enum { Unlocked=0, Locked=1 } state = Unlocked; void slic_abort() { SLIC_ERROR: ;}

void KeAcquireSpinLock_return() { if (state == Locked) slic_abort(); else state = Locked;}

void KeReleaseSpinLock_return { if (state == Unlocked) slic_abort(); else state = Unlocked;}

C Code:

Boolean Program:

void example() { do { KeAcquireSpinLock();A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; req = devExt->WLHV; if(req && req->status){ devExt->WLHV = req->Next; KeReleaseSpinLock();B: KeReleaseSpinLock_return(); irp = req->irp; if(req->status > 0){ irp->IoS.Status = SUCCESS; irp->IoS.Info = req->Status; } else { irp->IoS.Status = FAIL; irp->IoS.Info = req->Status; } SmartDevFreeBlock(req); IoCompleteRequest(irp); nPackets++; } } while(nPackets!=nPacketsOld); KeReleaseSpinLock();C: KeReleaseSpinLock_return(); }

void example() begin do skip;A: KeAcquireSpinLock_return(); skip; if (*) then skip;B: KeReleaseSpinLock_return(); skip; if (*) then skip; else skip; fi skip; fi while (*); skip;C: KeReleaseSpinLock_return(); end

C Code:

Boolean Program:

Abstraction Preserves Correctness

• The Boolean program that is generated with predicate abstraction is non-deterministic. – Non-determinism is used to handle the cases where the

predicates used during predicate abstraction are not sufficient enough to determine which branch will be taken

• If we find no error in the generated abstract Boolean program then we are sure that there are no errors in the original program– The abstract Boolean program allows more behaviors

than the original program due to non-determinism.– Hence, if the abstract Boolean program is correct then

the original program is also correct.

Counter-Example Guided Abstraction Refinement

• However, if we find an error in the abstract Boolean program this does not mean that the original program is incorrect. – The erroneous behavior in the abstract Boolean program

could be an infeasible execution path that is caused by the non-determinism introduced during abstraction.

• Counter-example guided abstraction refinement is a technique used to iteratively refine the abstract program in order to remove the spurious counter-example traces


The basic idea in counter-example guided abstraction refinement is the following:

• First look for an error in the abstract program (if there are no errors, we can terminate since we know that the original program is correct)

• If there is an error in the abstract program, generate a counter-example path on the abstract program

• Check if the generated counter-example path is feasible using a theorem prover.

• If the generated path is infeasible add the predicate from the branch condition where an infeasible choice is made to the predicate set and generate a new abstract program.


void example() begin do skip;A: KeAcquireSpinLock_return(); skip; if (*) then skip;B: KeReleaseSpinLock_return(); skip; if (*) then skip; else skip; fi skip; fi while (*); skip;C: KeReleaseSpinLock_return(); end

Boolean Program:Refined Boolean Program:

void example() begin do skip;A: KeAcquireSpinLock_return(); b := T; if (*) then skip;B: KeReleaseSpinLock_return(); skip; if (*) then skip; else skip; fi b := b ? F : *; fi while (!b); skip;C: KeReleaseSpinLock_return(); end

(using the predicate (nPackets = npacketsOld))

the boolean variable b represents the predicate(nPackets = npacketsOld)


• Using counter-example guided abstraction refinement we are iteratively creating more an more refined abstractions

• This iterative abstraction refinement loop is not guaranteed to converge for infinite domains– This is not surprising since automated verification for

infinite domains is undecidable in general• The challenge in this approach is automatically choosing

the right set of predicates for abstraction refinement– This is similar to finding a loop invariant that is strong

enough to prove the property of interest

Documents

A Tutorial on Automated Verification Tevfik Bultan