Improved Design Debugging Improved Design Debugging using Maximum Satisfiabilityusing Maximum Satisfiability
University of TorontoUniversity of Toronto
Sean Safarpour, Hratch Mangassarian,
Andreas Veneris
Mark H. Liffiton,Karem A. Sakallah
University of MichiganUniversity of Michigan
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
OutlineOutline
Introduction & MotivationIntroduction & Motivation
Debugging with max-satDebugging with max-sat
Debugging with approximate max-satDebugging with approximate max-sat
ExperimentsExperiments
ConclusionConclusion
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Introduction & MotivationIntroduction & Motivation
Verification is a major pain in VLSI Verification is a major pain in VLSI design cycledesign cycle check correctness against specscheck correctness against specs identify corner casesidentify corner cases evaluate test coverageevaluate test coverage
Automated solutions existAutomated solutions exist
When design fails, find error source When design fails, find error source manual manual debuggingdebugging process process time consumingtime consuming
pass?
specification
debug
specification
specificationspecification
design
verification
debug
yes
no
pass?
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Introduction & MotivationIntroduction & Motivation
What is “debugging”What is “debugging” identifying error source/locations in designidentifying error source/locations in design i.e.i.e. lines of code in the RTL (Verilog/VHDL) lines of code in the RTL (Verilog/VHDL)
Debugging is a verification bottleneckDebugging is a verification bottleneck manual taskmanual task consumes up to 50% of verification effort consumes up to 50% of verification effort
Automated debugging or error localization Automated debugging or error localization finds source of error in circuit automaticallyfinds source of error in circuit automatically use information from verification (testbench, traces, etc.)use information from verification (testbench, traces, etc.) current techniques not practical in industrycurrent techniques not practical in industry
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Introduction & MotivationIntroduction & Motivation
Why is debugging hard to automate?Why is debugging hard to automate? Problem Complexity grows O(nProblem Complexity grows O(n# errors# errors))
• many gate-level errors in high-level designsmany gate-level errors in high-level designs Algorithms work on gate/Boolean modelsAlgorithms work on gate/Boolean models
• circuits are big: > 100,000 gatescircuits are big: > 100,000 gates Algorithms must model sequential behaviorAlgorithms must model sequential behavior
• traces are long: > 1000s of clock cyclestraces are long: > 1000s of clock cycles• expensive using time frame expansionexpensive using time frame expansion
Debugging techniques over past decadesDebugging techniques over past decades Simulation, path tracing, BDDsSimulation, path tracing, BDDs SAT & QBF techniques: last 5 yearsSAT & QBF techniques: last 5 years
• promising resultspromising results
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Introduction & MotivationIntroduction & Motivation
What are the inputs to automated debuggers?What are the inputs to automated debuggers? Erroneous design (one or multiple gate/high-level bugs)Erroneous design (one or multiple gate/high-level bugs) Simulation Trace: stimulus to observe erroneous behaviorSimulation Trace: stimulus to observe erroneous behavior Correct/Expected Response: trace provided from specsCorrect/Expected Response: trace provided from specs
Erroneousdesign
Erroneousdesign
AutomatedDebugger
potentialerror locations
AutomatedDebugger
potentialerror locations
Erroneousdesign
Simulation & Expected
traces
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Introduction & MotivationIntroduction & Motivation Previous SAT & QBF techniquesPrevious SAT & QBF techniques
How can How can erroneous erroneous circuit be modified to satisfy circuit be modified to satisfy the the expectedexpected response under the given response under the given stimulusstimulus??
Insert instruments in circuit/CNF such that Insert instruments in circuit/CNF such that satisifying assignment => error locationssatisifying assignment => error locations
Find all error location tuples up to NFind all error location tuples up to N ObservationObservation
The union of the constraints is UNSATThe union of the constraints is UNSAT removing clauses corresponding error locations removing clauses corresponding error locations
makes the problem SATmakes the problem SAT• Maximum Satisfiability problemMaximum Satisfiability problem
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Debugging exampleDebugging example
a
bc
d
e
011 1 0
Erroneous circuit
011
a
bc
d
e
Correct circuit
Simulation trace
Correct response
(!a+c)(!b+c)(a+b+!c)(c+!e)(d+!e)(!c+!d+e)
(!a)(b)(d)
(!e)= UNSAT
(!a+c)(!b+c)(a+b+!c)(c+!e)(d+!e)(!c+!d+e)
(!a)(b)(d)
(!e) = SAT
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Debugging with max-satDebugging with max-sat
Removing clauses satisfies the formulaRemoving clauses satisfies the formula can only “remove” clauses corresponding can only “remove” clauses corresponding
to circuit constraintsto circuit constraints• identify bugs in gates (not stimulus/response)identify bugs in gates (not stimulus/response)
max-sat algorithms do exactly thatmax-sat algorithms do exactly that RequirementsRequirements
find all gate-level errors up to cardinality Nfind all gate-level errors up to cardinality N• must map clauses to gatesmust map clauses to gates• develop concept of error cardinalitydevelop concept of error cardinality
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Debugging with max-satDebugging with max-sat
Maximal Satisfiable Subset (MSS)Maximal Satisfiable Subset (MSS) generalization of max-sat (maximum cardinality)generalization of max-sat (maximum cardinality) adding any extra clause makes formula UNSATadding any extra clause makes formula UNSAT
Minimal Correction Set (MCS)Minimal Correction Set (MCS) complement of MSScomplement of MSS
Can find all MCS of up to size Can find all MCS of up to size kk Can group clauses to approximate MCSesCan group clauses to approximate MCSes
groups ygroups y11, y, y22,..., y,..., ynn
every clause C becomes C’= (yevery clause C becomes C’= (y11 -> C) -> C)
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Debugging with max-satDebugging with max-sat
How to debug gates instead of clauses?How to debug gates instead of clauses? define a group for each gate in circuitdefine a group for each gate in circuit
a
bc
d
e
011 1
(!a+c+!yA)(!b+c+!yA)(a+b+!c+!yA)(c+!e+!yB)(d+!e+!yB)(!c+!d+e+!yB)
(!a)(b)(d)
(!e)
can only reason about can only reason about satisfiablity at gate granularitysatisfiablity at gate granularity
How to get all solutions?How to get all solutions? block previous solutions and block previous solutions and
solve againsolve again cannot remove blocking cannot remove blocking
clausesclauses
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Debugging with max-satDebugging with max-sat
What about sequential designs andWhat about sequential designs andmultiple vectors?multiple vectors? Time-frame expansion or Iterative Logic Array Time-frame expansion or Iterative Logic Array
unfold/replicate combinational circuit in timeunfold/replicate combinational circuit in time Multiple stimulus/response are used to increase Multiple stimulus/response are used to increase
debugging resolutiondebugging resolution All clauses derived from same combinational All clauses derived from same combinational
gate are in same groupgate are in same group
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Replication ExampleReplication Example
a
bc
d
e 0
1a
b
c a
b
c a
b
c
d
e
d
e
d
e
1
1
1
1
2 2
2
2
2
3 3
3
3
3
(a+!c+!yA)(b+!c+!yA)(!a+!b+c+!yA)(c+!e+!yB)(d+!e+!yB)(!c+!d+e+!yB)
(a1+!c1+!yA1)(b1+!c1+!yA1)(!a1+!b1+c1+!yA1)(c1+!e1+!yB1)(d1+!e1+!yB1)(!c1+!d1+e1+!yB1)
(a2+!c2+!yA2)(b2+!c2+!yA2)(!a2+!b2+c2+!yA2)(c2+!e2+!yB2)(d2+!e2+!yB2)(!c2+!d2+e2+!yB2)
(a3+!c3+!yA3)(b3+!c3+!yA3)(!a3+!b3+c3+!yA3)(c3+!e3+!yB3)(d3+!e3+!yB3)(!c3+!d3+e3+!yB3)
1
1 11
0 0
11 1
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Replication ExampleReplication Example
a
bc
d
e 0
1a
b
c a
b
c a
b
c
d
e
d
e
d
e
1
1
1
1
2 2
2
2
2
3 3
3
3
3
(a+!c+!yA)(b+!c+!yA)(!a+!b+c+!yA)(c+!e+!yB)(d+!e+!yB)(!c+!d+e+!yB)
(a1+!c1+!yA)(b1+!c1+!yA)(!a1+!b1+c1+!yA)(c1+!e1+!yB)(d1+!e1+!yB)(!c1+!d1+e1+!yB)
(a2+!c2+!yA)(b2+!c2+!yA)(!a2+!b2+c2+!yA)(c2+!e2+!yB)(d2+!e2+!yB)(!c2+!d2+e2+!yB)
(a3+!c3+!yA)(b3+!c3+!yA)(!a3+!b3+c3+!yA)(c3+!e3+!yB)(d3+!e3+!yB)(!c3+!d3+e3+!yB)
1
1 11
0 0
11 1
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Debugging with approximate max-satDebugging with approximate max-sat
Can group clauses from single gate togetherCan group clauses from single gate together single group represents a gate-based errorsingle group represents a gate-based error
Can group “same” gates from different vectors Can group “same” gates from different vectors and different time frames togetherand different time frames together single group represents a gate-based error single group represents a gate-based error
anywhereanywhere What about other types of groupings?What about other types of groupings?
based on modules/functions (higher level)based on modules/functions (higher level) based on topologybased on topology arbitrarilyarbitrarily
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Example based on topologyExample based on topology
group 1
group 2
group 3
group 4
Debugging with approximate max-satDebugging with approximate max-sat
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Empirical results show: max-sat is very Empirical results show: max-sat is very efficient at finding approximate solutionsefficient at finding approximate solutions the larger the groups the fasterthe larger the groups the faster resolution vs. performance trade-offresolution vs. performance trade-off
SAT-based debugger is good for exact SAT-based debugger is good for exact solutions, not for approximatesolutions, not for approximate faster than max-sat for exact solutionsfaster than max-sat for exact solutions
Exploit the strengths of both approachesExploit the strengths of both approaches use approximate max-sat as preprocess/filteruse approximate max-sat as preprocess/filter use SAT-based debugging on filtered problemuse SAT-based debugging on filtered problem
Debugging with approximate max-satDebugging with approximate max-sat
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
Debugging with approximate max-satDebugging with approximate max-sat
Two step debugging framework with max-satTwo step debugging framework with max-sat divide and conquer approachdivide and conquer approach approximate max-sat finds coarse solutionsapproximate max-sat finds coarse solutions exact debugger refines solutionsexact debugger refines solutions
approximatemax-sat
debugger
exact debugger
erroneousdesign
vectors/traces
over-approximationof suspects
grouping
exact errorlocations
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
ExperimentsExperiments Circuits from ISCAS’85/89, ITC’99, opencores.orgCircuits from ISCAS’85/89, ITC’99, opencores.org error inserted: random function changeerror inserted: random function change
AND, OR, NAND, NOR, XOR swampedAND, OR, NAND, NOR, XOR swamped simulation to find errorsimulation to find error
captured stimulus and correct tracecaptured stimulus and correct trace 1GB memory limit, 1 hour timeout1GB memory limit, 1 hour timeout Clauses group by Clauses group by
gate gate replicationreplication circuit topologycircuit topology
max-sat solver from U of Michiganmax-sat solver from U of Michigan SAT-based debugger from U of Toronto SAT-based debugger from U of Toronto
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
ExperimentsExperiments How many clauses or gates to group togetherHow many clauses or gates to group together
bigger/fewer groups -> faster the max-sat portion, more suspectsbigger/fewer groups -> faster the max-sat portion, more suspects smaller/more groups -> few suspects, faster exact debuggersmaller/more groups -> few suspects, faster exact debugger
Empirical results show groups of 20 gates are effective Empirical results show groups of 20 gates are effective
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
ExperimentsExperiments
circuitcircuit # error # error locationlocation
debug debug
only (s)only (s)
max-sat (s)max-sat (s) suspects suspects eliminatedeliminated
exact exact
debug(s)debug(s)
speed-upspeed-up
mot-comb3mot-comb3 1616 357.67357.67 0.270.27 98.32 %98.32 % 0.470.47 483.34483.34
ac97ac97 44 [TO][TO] 2.222.22 99.63 %99.63 % 3.673.67 > 980> 980
dividerdivider 44 1160.391160.39 14.5814.58 99.01 %99.01 % 1.321.32 72.9872.98
b14b14 4545 1377.861377.86 11.1711.17 88.99 %88.99 % 50.7550.75 22.2522.25
rsdecoderrsdecoder 1111 572.68572.68 0.670.67 98.95 %98.95 % 0.650.65 433.85433.85
s38584s38584 3535 [MO][MO] 636.45636.45 98.26%98.26% 376.62376.62 > 3.59> 3.59
sample experimentssample experiments max-sat very fast on grouped problemmax-sat very fast on grouped problem exact debugger very fast on problem with eliminated suspectsexact debugger very fast on problem with eliminated suspects result: orders of magnitude fasterresult: orders of magnitude faster
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
ExperimentsExperiments Number of solved instances and runtimeNumber of solved instances and runtime
proposed framework against SAT-based debug onlyproposed framework against SAT-based debug only
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
ExperimentsExperiments Runtime comparison:Runtime comparison:
proposed framework against SAT-based debug onlyproposed framework against SAT-based debug only
Debug w max-satDebug w max-satIntro & MotivIntro & Motiv ConclusionConclusionExperimentsExperimentsDebug w approx. max-satDebug w approx. max-sat
ConclusionConclusion
SAT-based debugging is a natural SAT-based debugging is a natural max-sat problemmax-sat problem
formulate max-sat problem for circuitsformulate max-sat problem for circuits gate groupinggate grouping replication groupingreplication grouping find all error suspectsfind all error suspects
proposed framework combining strengths proposed framework combining strengths of max-sat and SAT-based debuggingof max-sat and SAT-based debugging order of magnitude improvements achievedorder of magnitude improvements achieved