Methodology and case study

Verification for FPGAs

Verification team leader

Avi Caspi

Company Overview

• Founded: 2007 in Tel Aviv

• Headquarter: Tel Aviv, Israel

• Offices: Israel: Management and R&D - 40 employees

Serbia: R&D - 30 employees

• Professional Services:

ASIC and FPGA design, verification and software

ASICs and IP Design

Design Verification

Architecture, RTL design, Verification, Backend, Post-silicon

support

Specman, SystemVerilog, UVM, VMM, OVM, Formal Verification,

C++ and SystemC

FPGA and System Design System architecture, FPGA design, Verification, LAB bring-up and

Software integration

Professional Services

Design and Verification

IPs

Design Soft IPs: CSI2, Serial Flash

Verification IPs: I2C, UART, CAN, SPI, QSPI, AXI, AHB, others

Virtualization (Modeling

and Acceleration)

Modeling using SystemC and C++, Hardware acceleration using

ZeBu, Paladium and Veloche

ASIC and FPGA companies – Historical view

RTL Design

Verification

Backend

GL Verification

TO

ASICs:

•Larger and complex

•Long development

cycle

•Errors are costly

RTL Design

Basic

simulation

LAB testing

QA

FPGAs:

•Small and simple

•Easy bugs fix

FPGA P&R

FPGA and ASIC get alike

Modern Verification

• Test Plan

• TLM – Transaction level modeling

• Self Checking Test Bench

• Stimulus generation – constraint random verification

• Functional Coverage

Test Plan

• Planning the verification - list of design features to verify

• Derived from specification docs

• Test plan includes :

– Block diagram description of the verification environment

– Description of each block

– Features list + coverage plan

– Test list

TLM – Transaction level modeling

• High level modeling of the design

• Easier system debug

BFM DUTTransaction

GeneratorMonitor

SB

1. RTL – register transfer level.2. Transaction level.

Self Checking Test Bench

• Expected transaction stored in a scoreboard

• Output data compared with the expected data,

• Error is reported in case of mismatch

BFM DUTTransaction

GeneratorMonitor

SB

Stimulus Generation

• Using random verification we generate many different tests from the

same code

• Small set of constraint random test scenarios can fulfill the coverage

goals

• Regression testing

The increasing design size and complexity

translates into an exponential increase in the

number of directed tests required for verification

Functional Coverage

• Random generation? So How can we know what was tested?

• Defines the verification goal

What is Achieved By Adding Verification to The Flow?

• Engineering:

– Specification and requirements definition

– Confidence in our design

– Shorten or eliminate the lab debugging

• Management:

– Confidence on schedule and clear vision on the progress of the project

– No/Less bugs on the client site

– Teams work in parallel

– Significant Improvement in TTM (time to market)

The traditional milestone “In The Lab”The new milestone - “Verification done”

Should FPGA teams adopt ASIC Verification approach?

ASIC Verification Approach:

• Modern Tools and methodology (e.g. UVM,ERM)

• Detailed verification plan with functional coverage definition - verification goal

• Automatic self checking test bench (monitors and score boards)

• Constraint random stimulus

• Full connectivity check to all registers, memories and IPs (CPU, interfaces, etc..)

• Regression testing

So… How Can We Improve Our FPGA Verification?

Should FPGA teams adopt ASIC Verification approach?

Yes!

But…

Few principles should be applied

Lab ramp up

• ASIC:

– After Tape out

– Finding critical bug in the lab is unacceptable

• FPGA:

– Lab testing can be done in parallel to verification

Integrated Interface (e.g. PCIe, SERDES )

• ASIC:

– Must check full connectivity

– Connect complex VIP and check the support for all

configuration supported by Spec

• FPGA:

– Integrated interface is part of the FPGA hardware

– Real interface connectivity will be checked in the Lab

– Interface can be bypassed for simpler verification IPs

CPU

• ASIC:

– Must check full connectivity

– Verify main flows – registers access, Interrupt handling , etc …

– Must test boot sequences – usually very long tests

– Simulation using SW

• FPGA:

– Integrated CPU can be verified in LAB using real SW

– CPU subsystem should be checked by connecting VIP and CPU

emulation sequences

– Self written CPU/controller must be fully verified, with exhaustive

random opcode generation – very hard to obtain in the lab

Memory Controller (e.g. DDR)

– Stress check to memory controller – very hard to check in the lab

DDR MODEL

EMI (External memory interface)

Axi VIP

#0

AXI VIP

#n

Client #0

sequence Client #n

sequence

SBSBSB

DDR IF

Monitor

Error Injection

Error injection examples:

– DDR return wrong data

– Error from sensors

– Packet with crc errors

Random Error injection in simulation can help us verify the system recovery from error

Failures can happen in almost every system, In lab it is sometimes

hard to simulate it.

Case Study 1

Registers

LOGIC

PCIe Local busAvalon

bus

matrix

Local bus

VIP

SerDesSerDes SerDesdata

VIP

data

VIPSerDes

• Altera FPGA for controlling RF antennas – LVDS data input/output• PCIe Gen2 registers and memory access.

Case Study 1 (Continue)

1. Verification flow using Questa (Mentor Graphics).2. Methodology UVM System verilog.3. Registers shadow using UVM regs connected to local bus

VIP.

Bugs in the labLab TestingDesign +

Verification

23 weeks4 month

Case Study 2

• IR Camera using XILINX spartan6 FPGA – DO-254.

Case Study 2 (Continue)

• Verification flow using VCS (Synopsys).• Methodology: VMM System verilog.• Reference model using Matlab simulator.

Bugs in the labLabVerification

0Ongoing3 Eng 1Y

Case Study 2 (Continue)

• Used block verification to check memory controller under full stress.

• Self written controller with ROM for opcodes:• Passed Lab testing – critical bugs were found in simulations.• Exhaustive testing for all opcodes.

• High level of failures checking verified on simulation – very hard to check in the lab.

Case Study 3

• Parallel computing – concept proof FPGA.• Verification flow using IUS (Cadence).• Methodology: Specman ERM.

Bugs in the labLabVerification

0Days2 Engineers 4M

Case Study 3 (Continue)

• High complexity logic calculations.• Reference model on Transaction layer for expected data.• Final results compared to Matlab simulator.

Conclusion

• Complex FPGAs need verification.

• ASIC verification principles should be adopt. But –

• FPGA verification should be a unique flow.

• Right verification approach saves TTM and effort in complex FPGA designs.