Introducing Obsidian Software and RAVEN-GCS for PowerPC

Introducing Obsidian Software and RAVEN-GCS for PowerPC

2

Obsidian Software

Founded in 1997 by 3 ex-AMDers

Located in Austin, TX

Bootstrapped Private Company

Focus: Processors & Verification

Made Inc.500’s List of Fastest Growing Companies

3

Understanding Your Verification Methodology

Describe your processor development activities.

• Type/schedule

What is your methodology for stimulus generation?

Are you using any verification languages?•

e

•

Vera

4

Do your current test generation tools have any

limitations?

• Limits on randomness?

• Coverage holes?

• Ease of use?

How many processor verification engineers will be on

the project?

Level of experience

Understanding Your Verification Methodology

5

Obsidian’s Mission

Random Test Generators• Standard Architectures

• Custom Architectures

• Simple Processors

Verification Services

6

Resource ExpenditureTask Tool/Solution Required Effort

Correctness Checking ISS Low (Reuse ISS from other projects)

Testbench Development Memory Model Low

Coverage Measurements Architectural

Implementation

Code

Moderate

Moderate

Moderate

Stimulus Generation Directed Tests

Random Test Generators

High

High

Debug Effort Engineers Looking at Waveforms

High

7

Common Customer Challenges

Scaling existing test generators for future designs

Challenge of staffing and maintaining in-house expertise

Test generators often need rewrites for new architecture features

Test generation development is often in the critical path

Focus should be on processor, not tool generation

Understanding the cost of generating an internal test generator

The last 10% is the most difficult and highest risk

8

What do Random Test Generators Do?

Reduce the Need for Directed Tests

Tests Full Range of the Architecture

Generate Directed-Random Tests

Allow Verification Engineers to Focus on

Failing Tests

9

Model of Operation

10

The RAVEN Solution

RAVEN® is not:• A test bench language (e, vera, perl) to write a Random Test

Generator• Used to wiggle pins on buses

RAVEN® is:• A tool to intelligently generate assembly instructions to provide

test stimulus for a microprocessor• A full featured processor specific random test generator• Under complete user control: intelligently direct tests into

difficult to reach areas• Extensible – users can add new instructions

11

Architecture

Architecture DescriptionExtracted from Headers orProcessor Documentation

ISSDynamically linked

shared object

Architectural LayerLinked in

Template File

generated by

GUI or text editor

XML

C API

C++ API(function overrides) ‏

XMLRAVEN Core

C++ libraries

CustomerCustomer or Obsidian

ObsidianGenerated

GUIXML

Legend

12

Business ModelStandard Processors Proprietary Processors

• ARMv(4-7), MIPS 32/64, X86

• Full featured generators for above cores licensed as run time licenses

• Obsidian maintains architecture layer

• Obsidian provides technical support thru tape-out including RAVEN bug fixes and work-arounds

• CISC, RISC, VLIW, SIMD, DSP

• Obsidian develops the architecture layer and delivers it as source code to customer

• Core technology licensed to customer as run time licenses

• On-going support of architecture layer can be done by customer or Obsidian

• Obsidian provides technical support for both RAVEN® core and development effort.

Verification Services : Obsidian can provide verification services at additional cost to increase verification productivity and reduce overall project risk. These services may include, training, test creation services, RAVEN customization, coverage services, and other RAVEN related activities.

13

Pricing ConsiderationsProduct or Service Description

RAVEN licensing• One year time based licensing• Quantity affects total price• Standard software support included in license price

Development services• Engineering resource billed as a result of RAVEN®

customization• Engineering resource billed as a result of custom

architecture layer development in RAVEN®-GCS

Verification Services • Test creation services, training, coverage related services, methodology integration, test planning

Intellectual Property • Instruction set simulators• Can be sold as time based license or source code

14

Relationship Solution Structure Engagement Activities Impact on Success

TrustedAdvisor

Same as below plus:Onsite RAVEN verification

engineerHigh level Methodology

consultant

Same as below plus:Work closely with team onsite to

maximize productivityEarly stage verification

planning to identify areas of focus for you and Obsidian

BestProactive services insure

success due to identification of key areas of focus and

dedication to maximum verification productivity

Strategic Partner

Same as below plus:Full time dedicated

development engineer for ongoing enhancement requests and feature development

Same as below plus:Interactive development support

for continued RAVEN enhancement in synch with your architecture changes and RAVEN productivity enhancement requests

BetterEnhanced development

support to bolster productivity at all phases in the design cycle

TechnologyProvider

RAVEN LicensingCPU Arch layer

developmentStd. Software SupportRAVEN training

Arch layer developmentISS IntegrationPhased product deliveryArch layer and RAVEN core

support

GoodHigh-end solution focused

specifically on your processor feature set

15

Engagement Model

Customer and Obsidian agree in principle to move forward together

Obsidian provides on-site 1-2 day tutorial for detailed | technical evaluation with following objectives:

• Provide in depth technical view of RAVEN

• Determine architecture challenges

• Gather requirements for test generator

Customer and Obsidian agree on proposal and SOW

16

Engagement Model

Phased delivery (3-9 months) ‏• Integrate core ISS from customer

• Deliver architecture layer in multiple phases

Provide upgrades and maintenance for core test generator as well as guidance for customer architecture development

Provide sustaining or peak development help for customer architecture layer as an option

17

In Summary

Collett International Research 2002 Data:• 39% First SI success

• 60% of initial failures caused by logical / functional problems•

82% -

Design errors•

47% -

Specification errors•

14% -

Imported IP problems• 50% - 70% of resources consumed in verification

18

In Summary

Verification is Usually on the Critical Path to SI

Failure Costs Ranges From Bad (lots of money) to Very Bad (lost market) ‏

• Spins

• A few months time to market

• Delay of next project

• Loss of confidence in the market

19

RTG Methodology

20

Overview of Methodology

Use random or mostly random templates to generate initial coverage

Define coverage points based on design knowledge

Write directed-random templates to hit missed coverage points

Review all templates and relax restrictions where possible

21

Step 1: Random Templates

There’s random and then there’s “random”• In an immature design some features will be disabled

• Some features have enables rather than sliders•

Enable exceptions•

Coprocessor present• Some control register bits need to be configured at reset

• Initial test may be single CPU/thread

• RAVEN architectural layer may need enhancement to configure some behaviors

22

Coverage Using Random Only Verification

Space

Expect about 80% coverage

Coveredbehaviors

23

Step 2: Define Coverage Points

Black-box• Coverage defined based on architectural definition• Coverage based on architectural specification from

block designers• Coverage of block interfaces• Should not be sensitive to implementation• Points are defined by verification engineers

White-box• Areas of greatest change or risk• Can be sensitive to implementation• Points are defined by designers

24

Distribution of Coverage Points Coverage

Points

Designknowledge andrisk assessmentdeterminesplacementand densityof coveragepoints

25

Randomtemplate coverage iscombined with directed testcoverage

Intersection of Coverage Points and Pure Random

26

Step 4: Review Templates

An engineer should examine all existing templates• Find any areas where the template is artificially restricted and

remove restrictions

• Increase the number of instructions per template where appropriate

• Make sure MP/MT behavior is enabled where appropriate

• Allow the template to be used as a random macro where appropriate

Engineer should not be the same as the person who originally created the template, if possible

27

RAVEN-GCS Introduction

28

RAVEN-GCS Features

Random test generator• Fully-random, semi-directed, and fully-directed

• Generator knows full state of the processor at each instruction boundary

•

Test files contain all register and memory updates•

Full definition of the initial register and memory image•

Full report of the final register and memory image• Never generates an invalid test

29

RAVEN-GCS Features

Easy to use• Intuitive GUI for template creation

• Templates can be run from the GUI or the command line

• User and developer manuals

• Simple and well-documented interface to simulator

• Can automatically place comments in test output

• Supports user-defined test sequences that can be used as macros

30

Expandable to all architectures• VLIW – any packet size

• Multi-processing and/or multi-threading

• Paging

• Exceptions

• CISC or RISC

• Cache support

• Floating point

• Predicated instruction execution

• Branch-not-taken code insertion

RAVEN-GCS Features

31

Biases or constraints for all supported features• Instruction

• Operands

• Addresses

• Number generation

• Operating modes

• Register values

RAVEN-GCS Features

32

Data files are used to define architecture• Much of the architecture can be modified without recompiling

• Features can be added as they become available in the RTL

• Easy to work around known RTL or simulator bugs

Test output in XML format• Easy to parse

• Source for converter for test output to customer specific format provided

•

Example provided for conversion to user-readable form

RAVEN-GCS Features

33

RAVEN – ISS interface

InstructionSet

Simulator

RAVEN Test

Generator

Register WriteMemory Write

Register Update

Memory Update

Step Simulator

Test Output

File

Template File

RAVEN generator reads template file• Populate internal event queue with requested behaviors

Generator picks instruction or event from top of event queue

• For example if the queue element is InstrTree, a specific instruction such as an “ADD register immediate” is picked

Generator picks operands/data values/memory addresses/etc based on current environment (settings and overrides) ‏

34

ISS-Interface

Generator writes values to uninitialized registers and data memory

Generator determines exact opcode of instruction and writes it to the simulator’s memory

Generator calls simulator Step()‏• Simulator returns memory and register updates caused by instruction

execution

Repeats steps 2 through 7 until end of event queue

Generator writes test output file including final memory and register image

35

RAVEN-GCS GUI

36

RAVEN-MP GUI

37

Instruction Overrides

Values for instruction fields may be generated randomly or may be overridden with specific values.

Double-clicking on a canvas element invokes the appropriate override pop-up window.

38

Test Output Files

Test output files are in XML formatConsist of the following parts:

• Initial state •

memory and registers•

completely defines the state of the processor before any instructions in the test have generated

•

register state can be used for hotloading

if supported by the testbench• Initialization sequence

•

used if hotloading

is not supported•

instructions from reset to create the state needed to execute the first instruction in the test

39

Test Output Files

Consist of the following parts• Instructions

•

sequence created from the canvas•

all memory and registers updated as a result of instruction execution are reported

•

the full state of the system is known at every instruction boundary• Final state

•

fully defines the state of memory and registers after completion

of the test

40

Architecture definition

41

Architecture defintion

Most of the architecture can be specified in data files• these are all xml files• many can be parsed directly from architecture definition files such as

manuals or simulator header files

Types of files• definition of instructions and instruction trees• definition of operands and operand groups• definition of special conditions

exceptionspipelining issuespacketization rules for VLIW

• display information for the GUI

42

Instruction tree definition

43

Instruction definition

44

Operand class is AddressingModeOperand Formula specifies how to determine address

Formula: NIA+(LI &lt;&lt; 2)‏NIA = next instruction addressLI = value of the LI operand (an immediate) ‏&lt; &lt; = “<<“

syntax due to xml restrictions=> New address = next instruction address + (immediate << 2)

Instruction definition

45

Handling non-determinism

46

Using RAVEN Register Output

RAVEN test output files contain a complete list of expected values for all registers that affect the behavior of the test.

This example shows a simple case with no unpredictable register fields

GP1 GP2 CR1

Initial State 0x01234567 0xabcdef0 0x35742278

Instr. 1 0x55327a8b --- ---

Instr. 4 --- --- 0x35743378

Instr. 22 --- 0xabcd7a1f ---

Instr. 55 0x00000000 0x55327a8f ---

Instr. 56 --- --- 0x25743378

Instr. 121 0x00000001 --- ---

Instr. 139 0x00000002 0x55327a90 ---

Final State 0x00000002 0x55327a90 0x25743378

47

Basic Register Checking

GP1

Initial State 0x01234567

Instr. 1 0x55327a8b

Instr. 4 ---

Instr. 22 ---

Instr. 55 0x00000000

Instr. 56 ---

Instr. 121 0x00000001

Instr. 139 0x00000002

Final State 0x00000002

List of expectedvalues from testoutput file

0x55327a8b

=

0x00000000

0x01234567

0x000000a1

pass

pass

pass

0x000000020x000000010x000000000x55327a8b0x01234567

fail

Actual valuesfrom RTL

=

=

=

48

Register Checking With Unpredictable Fields

With each reported register update, RAVEN also reports a valid mask

Compare logic needs to examine only those bits whose corresponding valid mask bit is set

The valid mask may change over the course of the test to reflect processor state

CR1 value CR1 mask

Initial State 0x35742278 0xffffffff

Instr. 1 --- ---

Instr. 4 0x35743378 0xfffffcff

Instr. 22 --- ---

Instr. 55 --- ---

Instr. 56 0x25743378 0xfffffcff

Instr. 121 --- ---

Instr. 139 --- ---

Final State 0x25743378 0xfffffcff

49

Using RAVEN Memory Output in an MP/MT Test

Each CPU reports all memory writes

Between zone semaphores, multiple writes to the same address results in non- deterministic memory

The memory final state is reported as bytes with non- deterministic bytes shown as Xs

Writes to Addr X CPU1 CPU2

Initial State 0x01234567

Instr. 1 0x01327a8b ---

Instr. 22 --- 0xabcd7a1f

Instr. 55 0x00000000 0x39327a8f

Instr. 56 --- ---

Instr. 121 0x55abf790 ---

Instr. 139 0x55327a90

Final State 0x55,0xXX (0xab/0x32),0xXX (0xf7/0x7a),

0x90

50

Memory Checking in an MP/MT Test

0x55abf7900x000000000x01327a8b0x01234567

0x55327a900xabcd7a1f0x01234567

0x01327a8b

0x00000000

0x01234567

0xabcd7a1f

Actual valuesfrom RTL

pass

pass

pass

Always compare againstthe last unmatched write

for each CPU

=

=

=

=

0x1234567

0x1327a8b or0xabcd7a1f

0x00000000 or0xabcd7a1f

0x55abf790 or0xabcd7a1f

0x55abf7900x000000000x01327a8b

0x55327a900xabcd7a1f

0x55abf7900x00000000

0x55327a900xabcd7a1f

0x55abf790 0x55327a900xabcd7a1f

CPU1 CPU2

passfail

51

Using Zone Semaphores

Zone semaphores divide instruction execution into known time fragments

• A shared memory location is used as a counter

• No CPU that shares the semaphore can advance to the next zone before the counter reaches zero

• Execution of all instructions within the zone must complete before advancing to the next zone

This may require synchronizing instructionsDoes not imply cache flush

Not all CPUs need to use all semaphores

52

Constraining non-determinism using SemaphoresCPU1 CPU2 CPU3 CPU4

0xaabbccdd 0xaabbccdd 0xaabbccdd 0xaabbccdd

0x9867a430 0x20080325 0x03195725 0x0f152b35

0xa9fc20d4 0xa9441901 0x2298afc7 0xffffff34

0x00001003 0x70073447 0x7e9a98e0 0x99a9c0d8

0x109000ff 0x70073448 0xacde3f54 0xa5aa0977

0xdcb9444a 0x70073449 0x00000000 0x00000000

0x8987564a 0x7007344a 0x7007344c 0x00000001

0xf0cd7aad 0x7007344b 0x10001000

0x32457699 0x7007344c 0xefffefff

0x75913052 0x7007344d 0x7091304d

= initial state= zone 1

= zone 2= zone 3

= zone 4

CPU1 implements all 4 semaphoresCPU2 implements all 4 semaphoresCPU3 implements 2 &4 onlyCPU4 implements 1, 2 & 4 only

CPU3 does no writes to this addressin zone 4

All 4 CPUS read and write the same semaphore here

53

Constraining Non-determinism using semaphores

CPU1 CPU2 CPU3 CPU40xaabbccdd 0xaabbccdd 0xaabbccdd 0xaabbccdd

0x9867a430 0x20080325 0x03195725 0x0f152b35

0xa9fc20d4 0xa9441901 0x2298afc7 0xffffff34

0x00001003 0x70073447 0x7e9a98e0 0x99a9c0d8

0x109000ff 0x70073448 0xacde3f54 0xa5aa0977

0xdcb9444a 0x70073449 0x00000000 0x00000000

0x8987564a 0x7007344a 0x7007344c 0x00000001

0xf0cd7aad 0x7007344b 0x10001000

0x32457699 0x7007344c 0xefffefff

0x75913052 0x7007344d 0x7091304c

At the end of zone 1:

9 possible values

At the end of zone 2:

0x8987564a or0x7007344b or0x7007344c or0x00000000

At the end of zone 3:6 possible values

At the end of zone 4:0x75913052 or0x7007344d or0x7091304d

Final memory reported as:0x75 or 0x700x91 or 0x070x30 or 0x340x52 or 0x4d or 0x4c