1

Properties Incompleteness Evaluation by Functional Verification

IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007

2

Outline Introduction Background Methodology

Generation of faulty implementations Estimation of golden model incompleteness Incremental property coverage computation

Experimental results Conclusion

4

Verification Flow Based on Model Checking

5

Vacuum Cleaning vs. Property Coverage Evaluation

Vacuum cleaning Property coverage evaluation

P = { p1 , p2 , … , pn }

pi pi

pn+1

9

Background Kripke structure K = {S, S0 , R, L} FSM M = {I, O, S, s0 , R} Product machine MP = M1 XP M2 Retroactive network

Ιε

10

Methodology Overview

11

Why Properties will be incomplete?

Functional test plan

Design Verification

System specifications

Informal to formal

12

Methodology Overview

13

Static vs. Dynamic Static method

Formal verification Time-consuming Great effort in terms of memory resources Exhaustive verification response

Dynamic method ATPG & simulation

Lack of exhaustiveness Rapider than static method

15

Generation of Faulty Implementations Fault model and fault coverage for ATPG Define functional fault model

RTL level Bit coverage

Bit failure: stuck-at 0 or stuck-at 1 Condition failure: stuck-at true or stuck-at false Single fault: A faulty implementation is generated for

each fault Has been proved to be related to design errors

16

Detectable Faults

fi

0 1

0 0 0 011

Environment

18

Generation of Faulty Implementations(cont.) A non-optimized algorithm

If fail then f is ε-detectable Time-consuming and very likely state explosion

In this work: an approximation of the real set of ε-detectable

19

Methodology Overview

20

p-detectable and P-detectable

fi

0 1

0 0 0 011

Environment

pi

SAT

pi

UNSAT

P = { p1, p2, … , pn }

22

Property coverage

CP = 1 P is complete w.r.t. a specific fault model

Non-optimized algorithm

24

Witnesses and Counterexamples Witnesses

Existentially quantified CTL property

Counterexamples Universally quantified CTL property

25

Estimation of Golden Model Incompleteness(cont.) Witnesses and counterexamples

Tools can provide witnesses and counterexamples for CTL and LTL properties

Input witness and input counterexample

26

Witness Coverage Property coverage can be estimated by using

input witnesses From formal verification to dynamic method Under some conditions, CP = Cw

27

Proof of CP = Cw

Consider the safety and liveness properties separately Proof of theorem 5.6 (safety property):

fI

I I

, detable, is p-detectable for fail on

exist a finite counterexample (Def.5.1) holds on , is an input witness for (hypothesis)

Because is only temporal relations between

p P f f pp

ip i

p

PI and PO is a test sequence for (Def.4.1)i f

28

Proof of CP = Cw (cont.) wp-detectable and WP -detectable

29

Proof of CP = Cw (cont.)

( )det

detdet det

P

P

f Wf PW P

fI

I

( )det

, is p-detectableexist for on

w is witness for on (Theorem 5.6) is w-detectable

W-det

f Pp P f

w p

pff

30

Incremental Property Coverage Computation

32

Experimental ResultsTest vector