1

ADVANCED DIGITAL IC DESIGN(SESSION 6)(SESSION 6)

Digital VerificationBasic Concepts

Need for VerificationNeed for Verification2

Exponential increase in the complexity of ASIC implies need for sophisticated verification methods to be incorporated in the ASIC design process.

70% of time spent in verification

Catch the bug as early as possibleSo catch it in simulation – saves time and money.yImagine respin of a chip if the same bug is caught in Silicon – the famous FDIV bug in Intel

TradeoffsTradeoffs3

Building test benches involves Tradeoffs between:Performance – How fast we can test?Efficiency – Test coverage and fault grading?Extensibility – Evolution with the designy g

Design and verification go together.

What is verification?What is verification?4

A process that ensures conformance of a design to some predefined set of expectations.The expectations defined by the specifications

Types of VerificationTypes of Verification5

l f fFunctional verification of RTLGate-level simulation, to verify that the synthesized

l h h d f lnetlist matches the expected functionalityFormal Verification (equivalence checking) to make

h h l l l l hsure that the gate level netlist is equivalent to the RTL.Timing Verification, to verify that the design can run

d h ll l lat speed – this usually involves a static timing tool.

Functional VerificationFunctional Verification6

Run at RTL level and Gate Level too.It involves at RTL level

Specifying test casesCreating a test environmentgCreating the actual testsEnsure that all interesting cases are coveredg

At gate level in addition to above it involvesGates to RTL equivalence checking (Formal Verification) Gates-to-RTL equivalence checking (Formal Verification), reset conditions, clocks, race conditions etc. are checked.

Test BenchesTest Benches7

W fWith invention of HDLs like Verilog, simulators can compute the responses which could be manually t t dtested.Use of bus functional models which read stimulus f fil d ll d h i l d from files or procedurally created the stimulus and applied it to the DUT.Th b d l d h d h The bus models and the environment around the DUT are collectively known as the test bench.

Functional Verification FlowFunctional Verification Flow8

Design Specification

Strategy Phase

Verification Plan

Test Bench PhaseTest Bench Phase

Regression and CoverageRegression and CoveragePhase

Strategy PhaseStrategy Phase9

Identify Test Cases

Level of abstraction of E.g. Cycle accurate, packet acc rate WhatLevel of abstraction of

Test benchpacket accurate. What level of stimulus does it receive?

Stimulus GenerationPolicy

E.g. random, directed, directed y

Results monitoring

random

Real-time or gPolicy post-processing

Strategy Phase (continued)Strategy Phase (continued)10

Finding Interesting test casesThe exhaustive set of test cases for a million gate gdesign may be in billions!!! The challenge is in narrowing that test space to manageable and g p gpractical set without compromising the functionality

Regression and CoverageRegression and Coverage11

R iRegressionAbility to run tests periodically in batches, so the stimulus must be easily reproducible and pass/fail be automatically must be easily reproducible and pass/fail be automatically detected.

CCoverageHow much of the design has been tested.

Back annotationThe Verification engineer may look at coverage to modify The Verification engineer may look at coverage to modify or add more test cases. Get very close to 100% coverage is the objective.

Verification FlowsVerification Flows12

Depends on complexity of designUnit-level checks functionalityChip/system level checks the interconnection between different units.

Stimulus generation and result checking strategies Stimulus generation and result-checking strategies depend on the level of abstraction on which verification is doneverification is done.

Functions of the Test BenchFunctions of the Test Bench13

G h lGenerate the stimulusApply stimulus to the DUTCheck the results to verify that the test was successful –that is, the output of the DUT conformed to expectations.

Generating StimulusGenerating Stimulus14

Bi Sti l Binary Stimulus sequence of 0s and 1s driven into the inputs of the DUT

User Stimulusa directive from the user to a test bench

Finally translates to binary stimulusIf the binary stimulus is got from user-specified data then it is called directed testingIf the binary stimulus is generated randomly then it is called If the binary stimulus is generated randomly then it is called random testing

Checking Results( i l h ki )(visual checking)

15

Always usefulWorks for simple designsp gProne to human errorsN t it bl f l d i d b i tNot suitable for large design debug environment.

Checking Results( i f l )(Post-processing of logs )

16

This is good because it does not affect the simulation performance, as it is post simulation.Logging outputs at different levels

a very effective technique to locate faults without excessive y qlog sizes.

Difficulty is that the bug is located only after the y g ywhole test is completed.Necessary state information at time of the bug need Necessary state information at time of the bug need not be necessarily available.

Checking Results( l i i i )(real-time monitoring )

17

Implementing real-time monitors that check results in a fly.Useful for debugging, as it may flag an error as soon as it is detected and hence state information can be easily inferred.Slows down the simulationSlows down the simulation

Test casesTest cases18

Test cases for each test bench has to be identified. Three main ways for this:Test by featuresTest by interfacesTest by interfacesTest by corner cases

Test by FeaturesTest by Features19

De eloped b listing all the design feat res specified in the Developed by listing all the design features specified in the document for the DUT

a complete functionality list for the DUT.

Eliminating RedundancyThe major time-reducing factor in feature testing

Limitation:Missing interesting cases that are very hardware specificMissing interesting cases that are very hardware specific

Challenges:Ensure that the specification used to generate the test-by-features test plan is complete and accurateInvent the test cases that best verify the features being tested.

Testing by InterfacesTesting by Interfaces20

Best suited for protocol testingBest suited for protocol testing

Li i iLimitations:An interface test does not account for all possible states f th i t f d t F i t t of the interfaced components. For a given test sequence

the interfaced components may behave differently in different states.To overcome the limitations a use of test-by-features test plan in coordination with test-by-interfaces test plan is highly beneficial.

Testing by corner casesTesting by corner cases21

The boundary conditions of a DUT feature

The boundary conditions of the DUT yimplementation of a feature

Summary ofTest Plan Methodologies

T ti b F t T ti b I t f T ti bTesting by Features Testing by Interfaces Testing by corner cases

To Generate the test plan

List the documented feature of the DUT

List all combinations of transactions on all interfaces

Study the implementation of DUT features

Advantages of the Test Plan

Guarantees that advertised features are tested. Testing is not

Exposes transactions implied by protocols supported at interfaces

Uncovering test cases not easily inferred from

complete without full test-by-features coverage

but not explicitly named in the DUT feature list.

specifications or protocols.

Disadvantages of the Test Plan

Missing corner cases Missing DUT features that require a sequence of transactions

Requires detailed knowledge of DUT implementation

22

Which Test Plan to use?Which Test Plan to use?23

Use all three as one complements the other in advantages and disadvantages

All three methods helps achieve the ultimate goal All three methods helps achieve the ultimate goal for the verification engineer:

A F ll tested b g free designA Fully tested bug-free design

Test Generation SchemesTest Generation Schemes24

O l i li d h i i l Once test plan is listed, the next step is to generate stimulus to perform the test. Test generation schemes are the general methodologies used to generate this stimulus. The main schemes g gare:

ExhaustiveDirectedRandom and Directed RandomSliding window (sweep)Sliding window (sweep)

The Zeroth Law of Test Generation: Any generated test should y gsuccessfully run to completion and pass on an non-erroneous DUT

ReproducibilityReproducibility25

A must for both passing and failing testsDebugging failed testsRegression for passing testsReproducing random tests p g

the randoms are only pseudo-random (Verilog random facilities are also pseudo random in nature).

Overview ofi hTest Generation Schemes

26

Complete Test Space

Exhaustive Testing

Directed Testing

Random Testing

Directed random Testing Sliding

WindowWindow

Exhaustive TestingExhaustive Testing27

All possible input combinations coveredGenerated Algorithmicallyg yGuarantees complete test space coverageI ibl ith d d i d t hibiti l Impossible with modern designs due to prohibitively large test spaces

Directed TestingDirected Testing28

Testing of simple and completely unrelated features of a DUT is where this testing excels.Important test cases (Corner cases) have to kept in mind for Directed testing.gExpected results must be known in advance.

Random andi d d iDirected Random Testing

29

U thi th t dUnearthing the unexpectedThe most closest to a real-time in-field testRandom tests are longRandom tests are longLong Tests to be re-run on a bug being unearthed. They are necessary – say to fill up a long FIFO.A generic framework generating different tests based on random seeds and hence reproducible.Careful initialization of random tests finds bugs more quicklyCareful initialization of random tests finds bugs more quickly

starting at some known contextDirected Random testing needs:

Parameters and constraintsIntelligent test generators with feedback

Sliding WindowSliding Window30

Used in directed environment or random environmentSweep across parameters and transactions whose result depend upon ordering of them

sweep across all possible permutationsSweeping all parameters in a large design is out-of-reach and that is the biggest disadvantage.

Logging and Error CheckingLogging and Error Checking31

Human testing of log dumps and waveformsAutomated Checking

Running tests all throughout without human intervention is a big advantage.Regression tests requires automated checking facility

Log information – hierarchical dump – sufficient g penough to retrieve the context of a system

Formal VerificationFormal Verification32

How to ensure full coverageModel Checking: Define a property like fairness and ensure that a model confirms to this propertyEquivalence checking: Checks whether all models are equivalent functionally