An Overview of Methodology CreationMethodology as User Interface
Making ESL work as a Hardware Design FlowEvolution of a Interface Definition LanguageTransaction Level AssertionsTransaction Level Debug
Presenter
Presentation Notes
Talk is 50 Minutes (hopefully slight less – need to trim the slides): Any one of these topics could fill this time, but I decided to go broad rather than deep
NVIDIA Confidential
A HW Development Flow
Big Paper Spec
Design
Verification
RTL
Testbench Checkers Tests
Scoreboard
Assertions
C Model
Synthesis
Directed
RandomBFMs,
TLMs
Clocks, ResetsAssertions
ISS Model
CoverageDebug
Formal
Presenter
Presentation Notes
Caricature of a flow – but probably quite familiar
NVIDIA Confidential
User Model for Running Tests
Build Testbench
Generate Test
Run Predictor
Run DUT
Compare Behaviors
*
Presenter
Presentation Notes
Standard flow for running tests: build the testbench exe (makefiles, etc.) then distribute multiple tests using LSF. Each test has 4 phases: create config files, etc; run the test on rtl and predictor; compare results. dotted-path from DUT (rtl) to predictor (model) allows “don’t cares” (e.g. cycle-accurate arbitration decisions) to be propagated: assertions check this data stream, so less risk of common-mode errors than a cycle-accurate model
Part 1
Methodology Design
NVIDIA Confidential
Purpose of a Flow
Conceptual FrameworkWhere do things live?How do things work?
Keep out of the wayDon’t make life difficult for people
Anchor for VariationGive people something to complain about
Presenter
Presentation Notes
Need examples of metaphors: “simulated annealing” Vs “shotgun and rifle”? – what are the consequences of these different metaphors? “predictor model” Vs “behavioral model”? – not really “metaphors” – just different names for the same thing
NVIDIA Confidential
Two Philosophies
ConformityEmphasize Standardization
DiversityEmphasize Innovation
Synergy or Conflict?
Presenter
Presentation Notes
Large Project Multiple Teams Long Legacy People move between projects benefit of familiar environment exposure to alternative approaches
NVIDIA Confidential
Understanding Variation
To understand what to standardize:you need to understand what not to standardize
Personal PreferencesTechnical Aspects of the DesignsSupporting LegacySeeking the Next Big Thing
NVIDIA Confidential
Personal Preferences
Emacs!
vi
NVIDIA Confidential
Personal Preferences
Choice of editor doesn’t affect othersAt least, not much
Choice of scripting language has greater impactBut is encapsulatedA script’s users don’t see the implementation language
Choice of HVL affects whole teamCan’t write “E” tests for a “Vera” testbench!But a unit testbench isn’t seen by other units
A good flow will allow encapsulation of preferencesI can go to any unit and build & run its testsEnables rapid localization of infrastructure issues
NVIDIA Confidential
Technological Differences
NVIDIA Confidential
Technical Characteristics
P C I E Video Frame
Buffer
Off-chip Memory
Graphics Pipe
Presenter
Presentation Notes
A chip has many units: graphics pipeline => data transaform; vido, memory more control-oriented. Pcie is basically a slow wire/router, data isn’t touched; graphics units transform data. In fullchip testing we see pretty pictures: units don’t see these pictures
NVIDIA Confidential
Reuse Vs Stagnation
Reuse considered GoodAvoid reinventing the wheelBuild on the shoulders of giants
Reuse invites InertiaReuse can propagate dependenciesDependencies make things harder to changeResistance to change is known as inertia
Inertia can lead to StagnationImproper reuse accumulates dependenciesReused code that is not understood will bit-rotTo avoid stagnation, inject agitation
Presenter
Presentation Notes
Is reuse good? Probably yes, but must recognize the dangers. Too much reuse without refactoring (cleanup) leads to problems
NVIDIA Confidential
Are Single Paradigm Projects Possible?
Project 1
Unit A
Unit B
Unit C
Project 2 Project 3
Unit D
Unit B
Unit C
Unit B
Paradigm 1
Paradigm 2
Paradigm 3
time
Presenter
Presentation Notes
As a result of reuse of units between projects, a requirement to have every unit use a common flow will either result in schedule-inefficiency, or else stagnation (nothing can ever change)
Lets return to this diagram I showed at the start of the talk ... Let’s label it “BAD”! Why Bad (After all, most of use recognize/use it)
NVIDIA Confidential
A HW Development Flow
Big Paper Spec
Design
Verification
RTL
Testbench Checkers Tests
Scoreboard
Assertions
C Model
Synthesis
Directed
RandomsBFMs,
TLMs
Clocks, ResetsAssertions
ISS Model
CoverageDebug
(BAD)
Presenter
Presentation Notes
70% of boxes are “verification” (14 of 20) – may marketing (and technical) presentations complain that “70% of schedule is in verification”: this flow may explain this statistic Not clear if “debug” live under “design”, or “verification”
NVIDIA Confidential
A HW Development Flow (Better)
RTL Scoreboards
Synthesis
Randoms
Clocks, Resets
Design
Verification
Testbench
Small Paper Spec
ISS Model ESL
C Model
Interfaces
Assertions
Directed Tests BFMs
TLMs
Validation
Assertions
Triage
Debug
Coverage
Presenter
Presentation Notes
30% of boxes are in “verification” : 7 of 22 ESL == Architectural Design “Triage” permits separation of verification from design “debug” “Interfaces” is the macro-arch netlist, plus transaction graph – that’s the second part of today’s talk “Validation” uses “Directed Tests”, but those tests are becoming increasingly high tech (graph walking) – all validation of assertions, and can be reused for verification.
NVIDIA Confidential
Who Writes Assertions?
DesignersBottom Up Assumptions
Verification EngineersTop-down Intent
Presenter
Presentation Notes
Traditionally, assertions are a verification tool: a testplan is created, and assertions assigned to be written either by design team, or by verif engineers Although it is reasonable for designers to add assertions for any assumptions they make, the intent of the design should be asserted by those who specify it. I.e. the Architects. Verification Engineers don’t need to be in the business of writing assertions (That’s obviously too strong a statement, but…).
NVIDIA Confidential
Who Writes Assertions?
DesignersBottom Up Assumptions
Verification EngineersTop-down Intent
ArchitectsThe SpecificationTop Down AssumptionsBottom Up Intent
Presenter
Presentation Notes
Traditionally, assertions are a verification tool: a testplan is created, and assertions assigned to be written either by design team, or by verif engineers Although it is reasonable for designers to add assertions for any assumptions they make, the intent of the design should be asserted by those who specify it. I.e. the Architects. Verification Engineers don’t need to be in the business of writing assertions (That’s obviously too strong a statement, but…).
NVIDIA Confidential
Where to Write Assertions
The RTLInlineBound
The TestbenchScoreboard
EnvironmentE.g. Post Process Log file
Presenter
Presentation Notes
RTL, Testbench, Environment are all really the same thing!
NVIDIA Confidential
Where to Write Assertions
The RTLInlineBound
The TestbenchScoreboard
EnvironmentPost Process Log files
The SpecificationC Models (?)
Presenter
Presentation Notes
In addition to assertions related to the verification environment, it is also possible to add assertions to the C models. Std C has the “assert” statement; and more advanced approaches allow assertions to be written that are temporal in nature.
NVIDIA Confidential
Where To Write Specification Assertions
FunctionalityModel
PerformanceModel
RTL Design
Presenter
Presentation Notes
The specification is rendered into an executable form in the various models written by the architects: a “function” model (e.g. UTF SystemC) and a “Performance” model (queuing). An obvious place to add assertions is in these models. But An assertion in these models is executable only when these models are being run, not when the rtl is being simulated/emulated/formally-verfied
NVIDIA Confidential
Where To Write Specification Assertions
TransactionModel
PerformanceModel
RTL Design
CorrelationModel
ISS Model
Debug/TriageModel
Presenter
Presentation Notes
A “Correlation Model” acts to comprehend transactions (events, context) in the various models – supports advanced debugging, etc. In generic terms, this model will be a graph. We want the nodes and arcs of this graph to be traceable to our other models. But what do the nodes and arcs represent? We might think that the structure that is important is the abstraction of processors and messages – Hoore’s CSP. But this is too much of an abstraction: we already have the UTF transaction/function; and the queuing model; for this. We need something closer to the RTL What about thinking of the structure as the netlist. Is this too close to the RTL? We actually want to end up with things between these two.
NVIDIA Confidential
Where To Write Specification Assertions
PerformanceModel
RTL Design
StructuralModel
TransactionModel
Presenter
Presentation Notes
A structural model is an attempt to pin down, up front, things that are common to the various different models. From the perspective of a full correlation framework, this is cheating. But It’s also efficient!
NVIDIA Confidential
Interfaces Vs State
Two approaches to comparing models:
Compare “Architectural State”Registers/flops within the design whose existence is required by the specification
Compare externally visible behaviorCompare interface trafficB. F. Skinner?
Presenter
Presentation Notes
BF Skinner is known for his “behaviorist” approach to human psychology. Most people think it’s too extreme … but it is the approach I’m going to use for this presentation
NVIDIA Confidential
Birth of an IDL
Interface Description Language
Initially, a language just to define signals
Interface a2bclock clkdown U valid 1up U busy 1down U cmd 24down U data 32
Presenter
Presentation Notes
Need to show history, as a way to avoid a tedious definition of the language as it currently exists – language defn is NOT the purpose of this talk! Initially just defines signals: was used to generate Verilog95 portlists, etc.
NVIDIA Confidential
Evolution of an IDL
Quickly added flow-control protocol abstraction
Interface a2bclock clkflow valid_busydown U cmd 24down U data 32
From this we can generate:Testbench components (BFMs: producers, consumers)Protocol Assertions…
Presenter
Presentation Notes
The “…” primarily refers to interaction with RTL generators
The formalization of accumulation is to define a “grammar” for the transaction that flow across it. The result of this will be to form a parse tree that represents the information that has flowed across the interface. We can then use powerful tree transformation operators to show that the “parse tree” of the rtl matches that of the fmodel. But “YACC” syntax is not particularly friendly, nor is it familiar to HW designers
So instead of using SW tools, we might try SVA. SVA is part of SystemVerilog, and so is potentially more familiar to designers than is YACC. But the SVA used by designers is generally rooted in a cycle-based level of abstraction This example shows a simple sequence: a “packet” is formed by a sequence of cycles that starts with a “begin” command, and is terminated by an “end”. SVA properties are very good at defining this type of sequence
But when we qualify the “cmd” signal, a simple cycle based description may no longer be appropriate. We could qualify each of the clauses in the sequence (e.g. “valid && cmd==BEGIN ##1 [->valid] ...”), but this would obscure the fact that the underlying sequence is unchanged by the qualifier
NVIDIA Confidential
Transaction Level Assertions in SVA
event sample;always @(posedge clk)
if (valid && ! busy) -> sampleassert @(sample)
cmd == SOP |-> sequence (valid_transaction)
clk
valid
busy
cmd SOP EOPMOP MOP
Presenter
Presentation Notes
A better approach (though one that many formal tools might not view so favorably) is to use transaction level assertions. In SVA, the “clock” definition is correctly known as a “sampling even”, and we can easily derive an event that is appropriate for the abstraction of the sequence. The downside here (other than the lack of tool support) is that using SVA to describe our transaction is tightly coupled to our RTL description of the interfaces. This sequence is equally valid in the world of C models, so it would be better to have a description of the sequences in a more agnostic medium.
NVIDIA Confidential
Transaction Level Assertions in SVA
event sample;always @(posedge clk)
if (valid && ! busy) -> sampleassert @(sample)
cmd == BEGIN |-> sequence (valid_transaction)
clk
valid
busy
cmd SOP EOPMOP MOP
Presenter
Presentation Notes
Note that there may be a problem here: The assertion as written does not constrain the value of “cmd”. But a valid-busy protocol may have a “data hold” property that requires us not to change anything while asserting “valid”. If using SFV tools (e.g. Magellan) then we would need additional constraints to ensure that an illegal command is not sent (i.e. the same assertion may need to appear more than once if one uses the sample-event approach)
Two alternative formalisms: The former is actually used by our tool; the latter didn’t (at the time) add sufficient value. Basically, people weren’t writing sufficiently complex assertions to justify the effort (we have a “state machine” mechanism that they use instead
NVIDIA Confidential
Multi-Unit Assemblies
A EB C D F Ga2b b2c c2d d2e e2f f2g
A simple pipeline
Presenter
Presentation Notes
This example makes explicit what it is that we want to define using the structural model. Start with an ideal pipeline. A set of processes (each may be millions of gates) propagate transactions down a simple pipeline. Imagine that each process has one input interface, and one output.
NVIDIA Confidential
Multi-Unit Assemblies
A
E
B C D
F G
a2b b2c c2d
d2ee2f f2g
Simple rearrangement
Presenter
Presentation Notes
...A simple rearrangement of the processes...
NVIDIA Confidential
Multi-Unit Assemblies
A
E
B C D
F G
a2b b2c c2d
d2ee2f f2g
Identify units with similar behaviors
Presenter
Presentation Notes
... A clever designer (or architect) realizes that the pipeline contains two groups of similar behavior ...
NVIDIA Confidential
Multi-Unit Assemblies
B E
C F
D
a2bebe2cf
cf2dd2be
cf2gG
A
Extract common behavior into unified components
be2cf === b2c + e2f
Presenter
Presentation Notes
... And so “simplifies” the design to allow reuse of the resources....
NVIDIA Confidential
Reusing Interface Definitions
A EB C D F G
D
GA
B E
C F
How to maximize reuse between these two architectures?
Presenter
Presentation Notes
After this simplification, do we really have a “different” architecture? The new architecture will be reflected in a new reference model, so both RTL and C-Model will share the new structure. But the muxing of data onto the new “be2cf” interface may lead to unavoidable differences in the ordering of transactions So our challenge is not just to show that the new architecture is the same as the old: it is to show that that two different descriptions of the new architecture are sufficiently similar. We will show that they are the same by mapping them back to the ideal pipeline that we started with. (of course, this is much easier when we take the time to sit down and work out what our ideal pipeline is before we start coding RTL)
enum SQUARE, CIRCLE, TRIANGE, BLANKdown U radius 3
NVIDIA Confidential
Accumulate Memory State
group mem_write
down U address 16down U data 1
assign mem[ x = 0 .. 15 ][ y = 0 .. 15 ]= past( data :sample( address == {x,y} ))
Presenter
Presentation Notes
Two things: Array == Function … keeps the language simple! Use “sample” option to define units of “past” … so the continuous assignment to the array (aka function ) does double-duty as state
NVIDIA Confidential
Add Predicates
group_more mem_write
assign is_circle[ r = 0..7 ] = “&&”([ x = -8 .. 7 ][ y = -8 .. 7 ]
Frontload The ScheduleProvide Verification with Executable Spec (ESL)
C Model, Correlation Model (pre validated)Interfaces, Transactions (definitions for code generators)Assertions, Testpoints (neutral language, pre validated)
Questions
http://dave.whipp.name/dv
Presenter
Presentation Notes
See the referenced web site for papers related to this talk.
Cycle-based ModelThe “Old” way of doing thingsA non-threaded version of “thread follows resource”
Different models are convenient at various timesBut SystemC forces you to commit early to one approachTherefore need extra code for transaction level debug
NVIDIA Confidential
70%
70% of schedule is Verification
What is correct figure
0%synthesize architectural models, constraintsCorrect by construction!
This slide is an introduction to some of the perceived weaknesses with current “transaction-oriented” tools. We’ll discuss this issue (&reset problem); don’t care management; and dynamic languages. Then I’ll mention Makepp. Define back-to-back read; animate to show required bus utilization Any HVL that claims to be transaction-based should support this basic concept. If it is to drive a BFM, we must be able to fully saturate the bus SystemC requires us to go to a “split-transaction” model, which negates benefits of threading model
NVIDIA Confidential
Unit Level System Testing
Stage N-1
Stage N+1
Stage N
Graphics Pipeline (Transaction Model)
RTL Unit DIFFDIFF
Presenter
Presentation Notes
C-model identifies all major interfaces between pipeline stages We are able to test units using system tests (simple combinatorics – and all the arrows point downstream) Capture input/outputs of units. Drive RTL with same stimuli. Compare Module-substitution is desired.
NVIDIA Confidential
New ChallengesNew ToolsNew PlatformsNew PeopleNew Ideas
RefactoringTestabilityD.R.Y.
Avoiding Stagnation
Presenter
Presentation Notes
Cut throat nature leads to rapid growth. Well ahead of Moore’s law. 1994 (When I was one year out of college) state of the art... Introduce terms: graphics pipe, triangles (polygons, vertices) -> pixel operations Very stylized, angular. Notice the Hair Similar technology on hand-held / Cell phones, today
