verification and simulations

04/15/11 1

Verification and Simulation

04/15/11 2

Objectives

Outcomes

Understanding keys of verification and analyze a design Setup a simulation for a design Handling coverage based verification

Requirements

Understanding verilog styles for synthesis How to write a design

04/15/11 3

Agenda Verification

Goad and importance of verification Functions of verification Tools for verification

Simulation What is a testbench? How to write a Verilog testbench for simulation

Coverage driven verification Coverage metrics Coverage models Functional coverage implementation

04/15/11 4

Agenda Verification

Goad and important of verification Function of verification Tools of verification

Simulation What is testbench Write a module description for a Verilog testbench

Coverage driven verification Coverage metrics Coverage models Functional coverage implementation

04/15/11 5

Verification

Goal of verification: Demonstrate functional correctness of a design Attempt to find design errors Attempt to show that design implements specification

Importance of Verification Costs of design errors can be high

(think “Pentium Floating-Point Error” ~ $300M!) Verification consumes about 70% design effort in

current systems design

Verification - Reconvergence Model Verification checks a “transformation” for

correctness RTL Design and Coding Synthesis Physical Design

Reconvergence Model:

InitialSpecification

TransformationResult

Transformation

Transformation

Verification of RTL Design The Idea:

WrittenSpecification

RTL Code

RTL Coding

Verification

How it Really Works:

WrittenSpecification

RTL Code

RTL Coding

Verification

Interpretation

04/15/11 8

Functional Verification Approaches Black box

Verify using module I/O ports only No knowledge of implementation No access to internals

White box Verify using module I/O ports and internals Full knowledge of implementation Full access to internals during simulation

Gray box Verify using module I/O ports only Full knowledge of implementation No access to internals during simulation

Verification vs. Testing

Verification identifies design errors Does the design correctly implement the specification? Does it perform calculations correctly? Performed before manufacturing

Testing identifies manufacturing faults Does each chip function properly? Applied after manufacturing

Specification Netlist

Design

Verification

Manufacturing

Testing

Silicon

04/15/11 10

Verification Tools

Linting Tools Simulators Code Coverage Tools Formal Verification Tools Version Control Issue Tracking

04/15/11 11

Linting Tools

Key idea: check code for potential problems that are legal HDL but not desirable, e.g.: Latch inferences Wire size mismatches

Types of Linting Tools Commercial tools Code reviews - peer review

Limitations Sometimes report non-problems Can’t look beyond syntax - “spell checker” analogy

04/15/11 12

Code Coverage Tools

Key idea: check that all codes are simulated Check that all lines of code are exercised in simulation Check that all paths through conditionals are exercise

Coverage tool function Insert reporting code into HDL model Summarize coverage & report to user Key metric % coverage

Limitations 100% coverage difficult to accomplish No guarantee of correctness

04/15/11 13

Simulators Allow testing of system response to stimulus

Event-driven - including delay models Cycle-level - one evaluation per clock cycle

Simulation Tools Waveform viewer Testbenches - provide stimulus, check response 3rd party models - simulate existing designs

Full models Bus-functional models

Limitations of simulation Can’t be exhaustive for non-trivial designs Performance bottleneck

04/15/11 14

Other Verification Tools

Verification languages (e.g. e, Vera) Used to specify and generate testbenches Abstraction used to increase productivity

Revision control - used as in software engineering Formal Verification

Equivalence checking - prove that input, output are equivalent Model checking - Prove assertions concerning design properties,

e.g. Reachability of states Deadlock avoidance Completion of transaction in an interface

04/15/11 15

Agenda Verification

Goad and important of verification Function of verification Tools of verification

Simulation What is testbench Write a module description for a Verilog testbench

Coverage driven verification Coverage metrics Coverage models Functional coverage implementation

The Simulation Environment

This is a simplified picture of the overall simulation environment.

This section concentrates on testbench development strategies.

design source model libraries testbench source

file input:

stimulus,response

simulator

compile

simulate

file output:

test pats,reports

04/15/11 17

Testbench Concept A Verilog testbench (also called test fixture) is a virtual

“test bed” An upper-level hierarchical Verilog structure that applies input

stimulus to a Unit Under Test (UUT) and monitors the output to verify functionality

CNTR32_TB CNTR32module

module

CLK

Q_OUTD_INRSTLOAD

TC

04/15/11 18

Application of Testbench

Behavioral

Logic

RTL

Layout

Fewer details,verify designconcept

Technology-specific details, slower design entry and simulation

f

DFF AND_OR2

CLB_R5C5

CLB_R5C6

04/15/11 19

Verilog 2001 provides considerable flexibility for declaring and accessing files, along with various formats of data inputs and outputs

TESTBENCH

UUTmodule

module

Simulation I/O Flexibility

{ Verilog, VHDL, … }

04/15/11 20

Verilogmodules

Synthesis

Place & Route

Behavioral Simulation (Execute HDL source code )

Gate-Level Functional

(Execute structural .v (netlist))

Gate-Level Timing (Execute structural .v and .sdf )

Design Verification

Both the structural Verilog (.v ) and Standard Delay Format (.sdf ) files are produced by the Place & Route tool

04/15/11 21

Design with Testbenches: Typical Approach Develop a testbench for each module in design

Use for debugging when module design is created Use to check for errors when module design is changed Use to check synthesized result In software, this is known as a "unit test"

Testbenches are essential in large chip design Design team may include hundreds of people, who work on

different subsystems Testbenches allow semi-automatic checking when different

subsystems are changed Chip design groups do this with "simulation farms"

04/15/11 22

Instanceof

M2

Instanceof

M3

Instance ofM1

Definition of Module M4

Testbenches in Hierarchical Design Example Hierarchy

Module M4 - Top-level module Modules M1, M2, M3 - used as

instances in M4

Create testbenches forall modules M1, M2, M3, M4

What if we change M2? First run M2 testbench Next, run M4 testbench

04/15/11 23

Coding Testbenches in Verilog HDL

Module Instance:DeviceUnder

Verification(DUV)

Testbench Module

04/15/11 24

Testbench Approaches - Visual Inspection

Deviceunder

Verification(DUV)

StimulusGenerator

Testbench File

Waveform ViewerOR

Text Output

04/15/11 25

Testbench Approaches - Output Comparison

Deviceunder

Verification(DUV)

Testbench File

ReferenceModel

StimulusGenerator

OutputComparator

Error/Status Messages

“Gold”Vectors

04/15/11 26

Testbench Approaches - Self-Checking

Deviceunder

Verification(DUV)

StimulusGenerator

OutputSignals

Input Signals

Testbench File

OutputChecker

Error/StatusMessages

04/15/11 27

Testbench Example (1)Verilog Module of a Comparator

module compare (a, b, aeqb, agtb, altb); input [7:0] a, b; output aeqb, agtb, altb;

assign aeqb = (a == b);

assign agtb = (a > b);

assign altb = (a < b);endmodule

04/15/11 28

Testbench Example (2)Testbench Module`timescale 1 ns / 100 ps

module compare_bench; reg [7:0] a, b; wire aeqb, agtb, altb;

compare DUV(a,b,aeqb,agtb,altb);

initial begin a = 0; b = 0; #10 a = 1; b = 0; #10 a = 255; b = 5; #10 b = 255; #10 a = 127; #10 $stop(); end // initialendmodule

Testbench Example (3)Result Waveforms

Testbench Review

04/15/11 31

Testcases What is a Testcase? A testcase is the top-level structure that controls the verification environment

to accomplish a certain purpose: A testcase may be a directed simulation where the configuration,

input stimulus and expected response are carefully crafted andtimed to elicit the desired functionality out of the design underverification.

A testcase may also be a set of constraints to increase theprobability that the random generators in the verificationenvironment will create certain interesting input patterns.

A testcase may introduce synchronization between concurrentstimulus streams and interfaces to create a specific condition thatmust be observed by the DUT.

04/15/11 32

Stimulating Testcases

Random testcases should be as short as possible. Random testcases should use a biasing stimulus preamble to

reach deep interesting DUT states. Biasing stimulus shall be implemented by calling a virtual

method named bias_t() in the rvm_env::start_t() method Random testcases should be run in individual simulations There shall be a single testcase program construct in an

simulation

04/15/11 33

Verification of Large Chips

Create a verification plan which specifies Features necessaries for first-time success

Prioritization of features: essential vs. optional Which features should be exercised What the response should be

Testcases to exercise features Process for reporting and fixing bug

Implement testbenches for each testcase Report bugs & fix

04/15/11 34

Agenda Verification

Goad and important of verification Function of verification Tools of verification

Simulation What is testbench Write a module description for a Verilog testbench

Coverage driven verification Coverage metrics Coverage models Functional coverage implementation

04/15/11 35

COVERAGE-DRIVEN VERIFICATION Coverage is used as a confidence-building metric. Coverage measurements are done toward the end of the

verification process, when the bulk of the testcases have been written

04/15/11 36

COVERAGE METRICS

Coverage metrics are measures of collected coverage data against stated or implied goals, usually expressed as percentage

Coverage analysis or reporting tools compare the collected data, usually aggregated from multiple simulations or static analysis, against the goal for each coverage point

The coverage metrics for multiple coverage points is usually distilled into a single overall coverage metric using a weighted average

04/15/11 37

COVERAGE MODELS

A coverage model is composed of structural coverage and functional coverage target definitions

Structural coverage models are implicitly defined by the code used to implement the design Structural coverage includes line coverage, expression coverage,

togglecoverage and automatically extracted FSM coverage, assertion coverage (measures the number of vacuous, non-vacuous success and failures of assertions and the different paths and values used when evaluating them.)

Function Coverage Modeling typically details the individual testcases that must be written to verify a particular design

04/15/11 38

FUNCTIONAL COVERAGE IMPLEMENTATION

Two constructs are available to specify functional coverage: coverage groups and coverage properties. Coverage groups shall be used when covering data in the

verification environment.⨸ and the sampled data must be mapped into different

coverage points. Coverage properties should be used to specify

implementation-specific functional verification requirements and physical interface compliance requirements.

04/15/11 39

Comparing Approaches

Visual inspection Only practical for small designs Automatic support: Verilogger timing diagram editor

Output comparison Effective when a good reference model is available Used by ASIC foundries - “Gold” vectors are legal definition

of a “functional” chip

Output checking Most difficult to code Mandatory for large designs