Improved Design Debugging using Maximum Satisfiability

Improved Design Debugging Improved Design Debugging using Maximum Satisfiabilityusing Maximum Satisfiability

University of TorontoUniversity of Toronto

Sean Safarpour, Hratch Mangassarian,

Andreas Veneris

Mark H. Liffiton,Karem A. Sakallah

University of MichiganUniversity of Michigan

Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat

OutlineOutline

Introduction & MotivationIntroduction & Motivation

Debugging with max-satDebugging with max-sat

Debugging with approximate max-satDebugging with approximate max-sat

ExperimentsExperiments

ConclusionConclusion



Verification is a major pain in VLSI Verification is a major pain in VLSI design cycledesign cycle check correctness against specscheck correctness against specs identify corner casesidentify corner cases evaluate test coverageevaluate test coverage

Automated solutions existAutomated solutions exist

When design fails, find error source When design fails, find error source manual manual debuggingdebugging process process time consumingtime consuming

pass?

specification

debug

specification

specificationspecification

design

verification

debug

yes

no

pass?



What is “debugging”What is “debugging” identifying error source/locations in designidentifying error source/locations in design i.e.i.e. lines of code in the RTL (Verilog/VHDL) lines of code in the RTL (Verilog/VHDL)

Debugging is a verification bottleneckDebugging is a verification bottleneck manual taskmanual task consumes up to 50% of verification effort consumes up to 50% of verification effort

Automated debugging or error localization Automated debugging or error localization finds source of error in circuit automaticallyfinds source of error in circuit automatically use information from verification (testbench, traces, etc.)use information from verification (testbench, traces, etc.) current techniques not practical in industrycurrent techniques not practical in industry



Why is debugging hard to automate?Why is debugging hard to automate? Problem Complexity grows O(nProblem Complexity grows O(n# errors# errors))

• many gate-level errors in high-level designsmany gate-level errors in high-level designs Algorithms work on gate/Boolean modelsAlgorithms work on gate/Boolean models

• circuits are big: > 100,000 gatescircuits are big: > 100,000 gates Algorithms must model sequential behaviorAlgorithms must model sequential behavior

• traces are long: > 1000s of clock cyclestraces are long: > 1000s of clock cycles• expensive using time frame expansionexpensive using time frame expansion

Debugging techniques over past decadesDebugging techniques over past decades Simulation, path tracing, BDDsSimulation, path tracing, BDDs SAT & QBF techniques: last 5 yearsSAT & QBF techniques: last 5 years

• promising resultspromising results



What are the inputs to automated debuggers?What are the inputs to automated debuggers? Erroneous design (one or multiple gate/high-level bugs)Erroneous design (one or multiple gate/high-level bugs) Simulation Trace: stimulus to observe erroneous behaviorSimulation Trace: stimulus to observe erroneous behavior Correct/Expected Response: trace provided from specsCorrect/Expected Response: trace provided from specs

Erroneousdesign

Erroneousdesign

AutomatedDebugger

potentialerror locations

AutomatedDebugger

potentialerror locations

Erroneousdesign

Simulation & Expected

traces


Introduction & MotivationIntroduction & Motivation Previous SAT & QBF techniquesPrevious SAT & QBF techniques

How can How can erroneous erroneous circuit be modified to satisfy circuit be modified to satisfy the the expectedexpected response under the given response under the given stimulusstimulus??

Insert instruments in circuit/CNF such that Insert instruments in circuit/CNF such that satisifying assignment => error locationssatisifying assignment => error locations

Find all error location tuples up to NFind all error location tuples up to N ObservationObservation

The union of the constraints is UNSATThe union of the constraints is UNSAT removing clauses corresponding error locations removing clauses corresponding error locations

makes the problem SATmakes the problem SAT• Maximum Satisfiability problemMaximum Satisfiability problem


Debugging exampleDebugging example

a

bc

d

e

011 1 0

Erroneous circuit

011

a

bc

d

e

Correct circuit

Simulation trace

Correct response

(!a+c)(!b+c)(a+b+!c)(c+!e)(d+!e)(!c+!d+e)

(!a)(b)(d)

(!e)= UNSAT

(!a+c)(!b+c)(a+b+!c)(c+!e)(d+!e)(!c+!d+e)

(!a)(b)(d)

(!e) = SAT



Removing clauses satisfies the formulaRemoving clauses satisfies the formula can only “remove” clauses corresponding can only “remove” clauses corresponding

to circuit constraintsto circuit constraints• identify bugs in gates (not stimulus/response)identify bugs in gates (not stimulus/response)

max-sat algorithms do exactly thatmax-sat algorithms do exactly that RequirementsRequirements

find all gate-level errors up to cardinality Nfind all gate-level errors up to cardinality N• must map clauses to gatesmust map clauses to gates• develop concept of error cardinalitydevelop concept of error cardinality



Maximal Satisfiable Subset (MSS)Maximal Satisfiable Subset (MSS) generalization of max-sat (maximum cardinality)generalization of max-sat (maximum cardinality) adding any extra clause makes formula UNSATadding any extra clause makes formula UNSAT

Minimal Correction Set (MCS)Minimal Correction Set (MCS) complement of MSScomplement of MSS

Can find all MCS of up to size Can find all MCS of up to size kk Can group clauses to approximate MCSesCan group clauses to approximate MCSes

groups ygroups y11, y, y22,..., y,..., ynn

every clause C becomes C’= (yevery clause C becomes C’= (y11 -> C) -> C)



How to debug gates instead of clauses?How to debug gates instead of clauses? define a group for each gate in circuitdefine a group for each gate in circuit

a

bc

d

e

011 1

(!a+c+!yA)(!b+c+!yA)(a+b+!c+!yA)(c+!e+!yB)(d+!e+!yB)(!c+!d+e+!yB)

(!a)(b)(d)

(!e)

can only reason about can only reason about satisfiablity at gate granularitysatisfiablity at gate granularity

How to get all solutions?How to get all solutions? block previous solutions and block previous solutions and

solve againsolve again cannot remove blocking cannot remove blocking

clausesclauses



What about sequential designs andWhat about sequential designs andmultiple vectors?multiple vectors? Time-frame expansion or Iterative Logic Array Time-frame expansion or Iterative Logic Array

unfold/replicate combinational circuit in timeunfold/replicate combinational circuit in time Multiple stimulus/response are used to increase Multiple stimulus/response are used to increase

debugging resolutiondebugging resolution All clauses derived from same combinational All clauses derived from same combinational

gate are in same groupgate are in same group


Replication ExampleReplication Example

a

bc

d

e 0

1a

b

c a

b

c a

b

c

d

e

d

e

d

e

1

1

1

1

2 2

2

2

2

3 3

3

3

3

(a+!c+!yA)(b+!c+!yA)(!a+!b+c+!yA)(c+!e+!yB)(d+!e+!yB)(!c+!d+e+!yB)

(a1+!c1+!yA1)(b1+!c1+!yA1)(!a1+!b1+c1+!yA1)(c1+!e1+!yB1)(d1+!e1+!yB1)(!c1+!d1+e1+!yB1)



1

1 11

0 0

11 1


Replication ExampleReplication Example

a

bc

d

e 0

1a

b

c a

b

c a

b

c

d

e

d

e

d

e

1

1

1

1

2 2

2

2

2

3 3

3

3

3

(a+!c+!yA)(b+!c+!yA)(!a+!b+c+!yA)(c+!e+!yB)(d+!e+!yB)(!c+!d+e+!yB)

(a1+!c1+!yA)(b1+!c1+!yA)(!a1+!b1+c1+!yA)(c1+!e1+!yB)(d1+!e1+!yB)(!c1+!d1+e1+!yB)



1

1 11

0 0

11 1



Can group clauses from single gate togetherCan group clauses from single gate together single group represents a gate-based errorsingle group represents a gate-based error

Can group “same” gates from different vectors Can group “same” gates from different vectors and different time frames togetherand different time frames together single group represents a gate-based error single group represents a gate-based error

anywhereanywhere What about other types of groupings?What about other types of groupings?

based on modules/functions (higher level)based on modules/functions (higher level) based on topologybased on topology arbitrarilyarbitrarily


Example based on topologyExample based on topology

group 1

group 2

group 3

group 4



Empirical results show: max-sat is very Empirical results show: max-sat is very efficient at finding approximate solutionsefficient at finding approximate solutions the larger the groups the fasterthe larger the groups the faster resolution vs. performance trade-offresolution vs. performance trade-off

SAT-based debugger is good for exact SAT-based debugger is good for exact solutions, not for approximatesolutions, not for approximate faster than max-sat for exact solutionsfaster than max-sat for exact solutions

Exploit the strengths of both approachesExploit the strengths of both approaches use approximate max-sat as preprocess/filteruse approximate max-sat as preprocess/filter use SAT-based debugging on filtered problemuse SAT-based debugging on filtered problem




Two step debugging framework with max-satTwo step debugging framework with max-sat divide and conquer approachdivide and conquer approach approximate max-sat finds coarse solutionsapproximate max-sat finds coarse solutions exact debugger refines solutionsexact debugger refines solutions

approximatemax-sat

debugger

exact debugger

erroneousdesign

vectors/traces

over-approximationof suspects

grouping

exact errorlocations


ExperimentsExperiments Circuits from ISCAS’85/89, ITC’99, opencores.orgCircuits from ISCAS’85/89, ITC’99, opencores.org error inserted: random function changeerror inserted: random function change

AND, OR, NAND, NOR, XOR swampedAND, OR, NAND, NOR, XOR swamped simulation to find errorsimulation to find error

captured stimulus and correct tracecaptured stimulus and correct trace 1GB memory limit, 1 hour timeout1GB memory limit, 1 hour timeout Clauses group by Clauses group by

gate gate replicationreplication circuit topologycircuit topology

max-sat solver from U of Michiganmax-sat solver from U of Michigan SAT-based debugger from U of Toronto SAT-based debugger from U of Toronto


ExperimentsExperiments How many clauses or gates to group togetherHow many clauses or gates to group together

bigger/fewer groups -> faster the max-sat portion, more suspectsbigger/fewer groups -> faster the max-sat portion, more suspects smaller/more groups -> few suspects, faster exact debuggersmaller/more groups -> few suspects, faster exact debugger

Empirical results show groups of 20 gates are effective Empirical results show groups of 20 gates are effective


ExperimentsExperiments

circuitcircuit # error # error locationlocation

debug debug

only (s)only (s)

max-sat (s)max-sat (s) suspects suspects eliminatedeliminated

exact exact

debug(s)debug(s)

speed-upspeed-up

mot-comb3mot-comb3 1616 357.67357.67 0.270.27 98.32 %98.32 % 0.470.47 483.34483.34

ac97ac97 44 [TO][TO] 2.222.22 99.63 %99.63 % 3.673.67 > 980> 980

dividerdivider 44 1160.391160.39 14.5814.58 99.01 %99.01 % 1.321.32 72.9872.98

b14b14 4545 1377.861377.86 11.1711.17 88.99 %88.99 % 50.7550.75 22.2522.25

rsdecoderrsdecoder 1111 572.68572.68 0.670.67 98.95 %98.95 % 0.650.65 433.85433.85

s38584s38584 3535 [MO][MO] 636.45636.45 98.26%98.26% 376.62376.62 > 3.59> 3.59

sample experimentssample experiments max-sat very fast on grouped problemmax-sat very fast on grouped problem exact debugger very fast on problem with eliminated suspectsexact debugger very fast on problem with eliminated suspects result: orders of magnitude fasterresult: orders of magnitude faster


ExperimentsExperiments Number of solved instances and runtimeNumber of solved instances and runtime

proposed framework against SAT-based debug onlyproposed framework against SAT-based debug only


ExperimentsExperiments Runtime comparison:Runtime comparison:

proposed framework against SAT-based debug onlyproposed framework against SAT-based debug only


ConclusionConclusion

SAT-based debugging is a natural SAT-based debugging is a natural max-sat problemmax-sat problem

formulate max-sat problem for circuitsformulate max-sat problem for circuits gate groupinggate grouping replication groupingreplication grouping find all error suspectsfind all error suspects

proposed framework combining strengths proposed framework combining strengths of max-sat and SAT-based debuggingof max-sat and SAT-based debugging order of magnitude improvements achievedorder of magnitude improvements achieved

Documents

Improved Design Debugging using Maximum Satisfiability