Embed Size (px)
Citation preview
Stop Writing Assertions!Efficient Verification Methodology
Dave Whipp – DVClub SV, August 2008
NVIDIA Confidential
Todays Talk
An Overview of Methodology CreationMethodology as User Interface
Making ESL work as a Hardware Design FlowEvolution of a Interface Definition Language
Transaction Level Assertions
Transaction Level Debug
NVIDIA Confidential
A HW Development Flow
Big Paper Spec
Design
Verification
RTL
Testbench Checkers Tests
Scoreboard
Assertions
C Model
Synthesis
Directed
RandomBFMs,
TLMs
Clocks, ResetsAssertions
ISS Model
CoverageDebug
Formal
NVIDIA Confidential
User Model for Running Tests
Build Testbench
Generate Test
Run Predictor
Run DUT
Compare Behaviors
*
Part 1
Methodology Design
NVIDIA Confidential
Purpose of a Flow
Conceptual FrameworkWhere do things live?
How do things work?
Keep out of the wayDon’t make life difficult for people
Define MetaphorsSubtly influence future directions
Anchor for VariationGive people something to complain about
NVIDIA Confidential
Two Philosophies
ConformityEmphasize Standardization
DiversityEmphasize Innovation
Synergy or Conflict?
NVIDIA Confidential
Understanding Variation
To understand what to standardize:you need to understand what not to standardize
Personal Preferences
Technical Aspects of the Designs
Supporting Legacy
Seeking the Next Big Thing
NVIDIA Confidential
Personal Preferences
Emacs!
vi
NVIDIA Confidential
Personal Preferences
Choice of editor doesn’t affect othersAt least, not much
Choice of scripting language has greater impactBut is encapsulated
A script’s users don’t see the implementation language
Choice of HVL affects whole teamCan’t write “E” tests for a “Vera” testbench!
But a unit testbench isn’t seen by other units
A good flow will allow encapsulation of preferencesI can go to any unit and build & run its tests
Enables rapid localization of infrastructure issues
NVIDIA Confidential
Technological Differences
NVIDIA Confidential
Technical Characteristics
PCIE Video Frame
Buffer
Off-chip Memory
Graphics Pipe
NVIDIA Confidential
Reuse Vs Stagnation
Reuse considered GoodAvoid reinventing the wheel
Build on the shoulders of giants
Reuse invites InertiaReuse can propagate dependencies
Dependencies make things harder to change
Resistance to change is known as inertia
Inertia can lead to StagnationImproper reuse accumulates dependencies
Reused code that is not understood will bit-rot
To avoid stagnation, inject agitation
NVIDIA Confidential
Are Single Paradigm Projects Possible?
Project 1
Unit A
Unit B
Unit C
Project 2 Project 3
Unit D
Unit B
Unit C
Unit B
Paradigm 1
Paradigm 2
Paradigm 3
time
NVIDIA Confidential
Watch some Real Users
Image courtesy of DAZ Productions
NVIDIA Confidential
How to “Watch” Users
Meetings and Discussions
Coffee-Break Grousing
Bug Reports
Keep Track of Support Requestscreate FAQs
VNC (Remote Desktop)
Instrumentation
NVIDIA Confidential
Build Time Distribution: 10,000 per sample
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
> 1 hour
< 1 hour
< 30 min
< 15 min
< 10 min
< 5 min
< 2 min
< 1 min
< 30 sec
< 20 sec
NVIDIA Confidential
Build Time Distribution: 1000 per sample
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
> 1 hour
< 1 hour
< 30 min
< 15 min
< 10 min
< 5 min
< 2 min
< 1 min
< 30 sec
< 20 sec
Part 2
ESL for Transaction Design
NVIDIA Confidential
A HW Development Flow
Big Paper Spec
Design
Verification
RTL
Testbench Checkers Tests
Scoreboard
Assertions
C Model
Synthesis
Directed
RandomsBFMs,
TLMs
Clocks, ResetsAssertions
ISS Model
CoverageDebug
(BAD)
NVIDIA Confidential
A HW Development Flow
Big Paper Spec
Design
Verification
RTL
Testbench Checkers Tests
Scoreboard
Assertions
C Model
Synthesis
Directed
RandomsBFMs,
TLMs
Clocks, ResetsAssertions
ISS Model
CoverageDebug
(BAD)
NVIDIA Confidential
A HW Development Flow (Better)
RTL Scoreboards
Synthesis
Randoms
Clocks, Resets
Design
Verification
Testbench
Small Paper Spec
ISS Model ESL
C Model
Interfaces
Assertions
Directed Tests BFMs
TLMs
Validation
Assertions
Triage
Debug
Coverage
NVIDIA Confidential
Who Writes Assertions?
DesignersBottom Up Assumptions
Verification EngineersTop-down Intent
NVIDIA Confidential
Who Writes Assertions?
DesignersBottom Up Assumptions
Verification EngineersTop-down Intent
ArchitectsThe Specification
Top Down Assumptions
Bottom Up Intent
NVIDIA Confidential
Where to Write Assertions
The RTLInline
Bound
The TestbenchScoreboard
EnvironmentE.g. Post Process Log file
NVIDIA Confidential
Where to Write Assertions
The RTLInline
Bound
The TestbenchScoreboard
EnvironmentPost Process Log files
The SpecificationC Models (?)
NVIDIA Confidential
Where To Write Specification Assertions
FunctionalityModel
PerformanceModel
RTLDesign
NVIDIA Confidential
Where To Write Specification Assertions
TransactionModel
PerformanceModel
RTLDesign
CorrelationModel
ISS Model
Debug/TriageModel
NVIDIA Confidential
Where To Write Specification Assertions
PerformanceModel
RTLDesign
StructuralModel
TransactionModel
NVIDIA Confidential
Interfaces Vs State
Two approaches to comparing models:
Compare “Architectural State”Registers/flops within the design whose existence is required by the specification
Compare externally visible behaviorCompare interface traffic
B. F. Skinner?
NVIDIA Confidential
Birth of an IDL
Interface Description Language
Initially, a language just to define signals
Interface a2b
clock clk
down U valid 1
up U busy 1
down U cmd 24
down U data 32
NVIDIA Confidential
Evolution of an IDL
Quickly added flow-control protocol abstraction
Interface a2b
clock clk
flow valid_busy
down U cmd 24
down U data 32
From this we can generate:Testbench components (BFMs: producers, consumers)
Protocol Assertions
…
NVIDIA Confidential
Continued Evolution of an IDL
Separation of packet structure from interface
group SOP
down U cmd 24
group MOP
down U data 32
group EOP
down U checksum 32
Interface a2b
clock clk
flow valid_busy
packet SOP, MOP, EOP
NVIDIA Confidential
Transaction Assertions in YACC
valid_interface_traffic:
| valid_interface_traffic transaction;
transaction: begin middle end;
begin: SOP;
middle:
| middle MOP;
end: EOP
NVIDIA Confidential
Cycle Level Assertions in SVA
sequence valid_trans;(cmd==SOP)(##1 cmd != SOP && cmd != EOP) [*0:$]##1 cmd == EOP
endsequence
a_well_formed_transaction: assert @(posedge clk)cmd == SOP |-> sequence (valid_trans)
clk
cmd SOP EOPMOP MOP
NVIDIA Confidential
Transaction Level Assertions in SVA
sequence valid_transaction;
(cmd==SOP)
(##1 cmd != SOP && cmd != EOP) [*0:$]
##1 cmd == EOP
endsequence
clk
valid
cmd SOP EOPMOP MOP
NVIDIA Confidential
Transaction Level Assertions in SVA
event sample;
always @(posedge clk)
if (valid && ! busy) -> sample
assert @(sample)
cmd == SOP |-> sequence (valid_transaction)
clk
valid
busy
cmd SOP EOPMOP MOP
NVIDIA Confidential
Transaction Level Assertions in SVA
event sample;
always @(posedge clk)
if (valid && ! busy) -> sample
assert @(sample)
cmd == BEGIN |-> sequence (valid_transaction)
clk
valid
busy
cmd SOP EOPMOP MOP
NVIDIA Confidential
Grammer using IDL
assert “temporal representation”data SOP => past( EOP )
data MOP => past( SOP | MOP )
data EOP => past( SOP | MOP )
assert “bnf representation”bnf SOP
-> MOP ** past( SOP.length-1 :sample( SOP ) )
-> EOP
NVIDIA Confidential
Multi-Unit Assemblies
A EB C D F G
a2b b2c c2d d2e e2f f2g
A simple pipeline
NVIDIA Confidential
Multi-Unit Assemblies
A
E
B C D
F G
a2b b2c c2d
d2ee2f f2g
Simple rearrangement
NVIDIA Confidential
Multi-Unit Assemblies
A
E
B C D
F G
a2b b2c c2d
d2ee2f f2g
Identify units with similar behaviors
NVIDIA Confidential
Multi-Unit Assemblies
BE
CF
D
a2be
be2cf
cf2dd2be
cf2gG
A
Extract common behavior into unified components
be2cf === b2c + e2f
NVIDIA Confidential
Reusing Interface Definitions
A EB C D F G
D
GA
BE
CF
How to maximize reuse between these two architectures?
NVIDIA Confidential
Packets as Traffic Streams
group b2c
down U data 32
group e2f
down U data 32
Interface be2cf
clock clk
flow valid_credit
packet b2c, e2f
NVIDIA Confidential
Time Units of Temporal Expressions
Group b2cdown U value 4
assert value != past( value )
Group e2fdown U value 4
assert ( value == 0 ) => ( past( value ) != 0 )
Interface be2cfpacket b2c, e2f
assert b2c => ( b2c.value != past( b2c.value :sample(b2c) ) )
assert past( e2f && e2f.value == 0 ) => (b2c && b2c.value != 0)
NVIDIA Confidential
Example
Renderer Memory
NVIDIA Confidential
The Traffic
group mem_write
down U address 16
down U data 1
group sync
down U shape 2
enum SQUARE, CIRCLE, TRIANGE, BLANK
down U radius 3
NVIDIA Confidential
Accumulate Memory State
group mem_write
down U address 16
down U data 1
assign mem[ x = 0 .. 15 ][ y = 0 .. 15 ]
= past( data :sample( address == {x,y} ))
NVIDIA Confidential
Add Predicates
group_more mem_write
assign is_circle[ r = 0..7 ] = “&&”(
[ x = -8 .. 7 ]
[ y = -8 .. 7 ]
mem[ x+8 ][ y+8 ] == ( x**2 + y**2 <= r**2 )
)
NVIDIA Confidential
Define the interface
interface render2memory
clock mem_clk
flow valid_busy
packet sync, mem_write
assert “correct sync shape”
sync && sync.shape == CIRCLE
=> past( mem_write.is_circle[ sync.radius ] )
NVIDIA Confidential
Example
Renderer Memory
assert “correct sync shape”sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )
NVIDIA Confidential
Summary
Efficient VerificationUnderstand Variation
Encourage Diversityavoid anarchy
Encourage Standardizationavoid stagnation
Frontload The ScheduleProvide Verification with Executable Spec (ESL)
C Model, Correlation Model (pre validated)
Interfaces, Transactions (definitions for code generators)
Assertions, Testpoints (neutral language, pre validated)
NVIDIA Confidential
Summary
Architects should write assertionsValidated assertions are input to Verification
Assertions must be directly reusable across modelsManual recoding invites errors
Explicitly model the structure that is common to architectural models and to design
Tie assertions to these common points
NVIDIA Confidential
Threading Models
Thread Follows ResourceTraditional SystemC approach
Thread Follows TransactionThink “Multithreaded ISS”
Cycle-based ModelThe “Old” way of doing things
A non-threaded version of “thread follows resource”
Different models are convenient at various timesBut SystemC forces you to commit early to one approach
Therefore need extra code for transaction level debug
NVIDIA Confidential
70%
70% of schedule is Verification
What is correct figure
0%synthesize architectural models, constraints
Correct by construction!
100%synthesize verification models, constraints
no design needed!
NVIDIA Confidential
Transactions Vs Cycles
Data min_val (Addr a1, Addr a2){ Data d1 = mem_read(a1); Data d2 = mem_read(a2);
if (d1 < d2)return d1;
elsereturn d2;
}
Address
Data
a1
d1
Pipelined Bus
a2
d2
t1 t2 t3 t4 t5 t6
NVIDIA Confidential
Unit Level System Testing
Stage N-1
Stage N+1
Stage N
Graphics Pipeline (Transaction Model)
RTL Unit DIFFDIFF
NVIDIA Confidential
New Challenges
New Tools
New Platforms
New People
New Ideas
Refactoring
Testability
D.R.Y.
Avoiding Stagnation
