Using PSL for Assertions and Coverage at Analog Devices

Kelly LarsonAustin Design Center

August 17, 2005

—Analog Devices Confidential Information—

Topics Covered

Why the fuss?Uses of Assertions

Dynamic property checkingFunctional coverageFormal VerificationOther Uses

Conclusion

—Analog Devices Confidential Information—

Newand

Improved!!

—Analog Devices Confidential Information—

Why all the interest?

Assertions standards widely acceptedPSL (formally sugar)SVAOVL

Assertion languages are concise, well suitedMaintainable

Multiple VendorsMultiple ToolsMultiple Uses

—Analog Devices Confidential Information—

Who writes the assertions?

Block designersLow-level design assumptionsCoverage points of concern

Verification TeamInterfacesBehavioral

Is it described in the spec?Checkers, protocol

ArchitectsStarvation, liveness, bandwidth

—Analog Devices Confidential Information—

What are the uses?

Dynamic property checkingFunctional CoverageFormal VerificationOther uses

—Analog Devices Confidential Information—

Dynamic Property Checking

The GoodWorks greatNo detectible performance hit (so far)

The BadIntegration into environment may not be straight-forward

No ability to query status from within the simulationMust post-process

Debugging is hard.Poor visibilityExperimentationDebussy may help, eventually

Other observationsPSL by itself not sufficient.

Auxiliary code

—Analog Devices Confidential Information—

Functional Coverage

One of our stated metrics for tape-out is 100% functional coverage.

Functionality mainly derived from specification, identified in testplan.Further refined with input from testplan reviews and code coverage analysis.

PSL supports “cover” statement in addition to “assert”Much smarter than code coverage, though involves much more effort.

—Analog Devices Confidential Information—

What is a functional coverage check?

A coverage check may or may not have an associated property check within the same statement.

The “coverage” part will be on the left hand side of the implication operator.

Coverage checks need not ever fail.

Coverage checks have nothing to do with correct behavior of the system, only correct behavior of the tests.

—Analog Devices Confidential Information—

Global vs. Test Coverage

Tools are great at providing overall functional coverage data

Tables show all of the assertions hit during a simulation.Can concatenate results from multiple simulation runs.Can rank tests based on how much additional coverage they provide.

—Analog Devices Confidential Information—

Global vs. Test Coverage (cont.)

What if I want to focus on the behavior of a single test?

If my test doesn’t do what I want it to do, I’d like it to fail!

Current tools do not have support for using specific coverage points to affect individual simulation runs.

These capabilities need to be developed by the user.

—Analog Devices Confidential Information—

PSL For Property Checking vs. PSL For Test Coverage

PSL For Property CheckingRelies mainly on functionality supplied through tools.Will fail anytime wrong behavior is detected.Not tied to specific testcase.

PSL For Test CoverageRelies mainly on functionality customized in environment.Only fails for coverage when coverage point is enabled for specific testcase.

Much overlap exists because both are built upon the PSL language. Conceptually, however, they are very different things.

—Analog Devices Confidential Information—

Adding Capability – One Approach

Tools already provided ways of analyzing coverage globally, but we also wanted the ability to specify coverage points required for an individual test.

Test should fail if it doesn’t accomplish what it was written to do.Needed to fit in nicely with existing methodology.

Solution was to implement an additional post-processing step which was controlled by command-line options for each individual test.

+RequireAssert+IgnoreAssert+ProhibitAssert

—Analog Devices Confidential Information—

Test Robustness

Tests which are broken should always fail. This means more than simply having self-checks.Example from previous DSP project:

1. Chip supported four different bus ratios. These could be specified on the command line. If nothing specified, a default ratio was used.

2. Self-checking directed tests implemented, the majority used the default ratio.

3. Test behavior verified with waveform viewers, and added to regressions.

4. Product engineering wanted to use tests to verify silicon, but needed representation from all ratios. Command line options were changed to help balance the number of tests run in each ratio.

5. Tests still passed all self checks. Life was good? WRONG!In many cases the functionality being tested for completely went away. There was no way to translate the broken test into a failure, however, since everything still looked fine to the testcase.

—Analog Devices Confidential Information—

Formal Verification

Assertions make this path easierDynamic assertions used for formal analysisAdded constaints to formal analysis get verified as dynamic assertions

MisconceptionsThis is done late in project, after everything else

Earlier the betterPrimarily a verification tool

Possibly an even better design tool

—Analog Devices Confidential Information—

Assertion Tracking

Assertion tracking capability added to Austin environment.Enabled by a “-track” option with the simulation command.All assertion data for all tests run is stored in mySQL results database.Assertion coverage data is then browsable through a series of web cgi scripts.

—Analog Devices Confidential Information—

Assertion Tracking (cont.)

What is it good for? To answer questions like:Do we currently have a test for “feature x”? Which one?What types of bus activity does “test x” induce? What interrupts?I’d like to test for a variety of concurrent activity. I can describe the simultaneous events in PSL, do we have a test which covers this condition?I’ve written some randomization routines, does it induce the behavior I’m expecting?I’ve finished with my PSL checks, and all my directed tests. Do I have full functional coverage on all of the PSL I wrote?

—Analog Devices Confidential Information—

—Analog Devices Confidential Information—

—Analog Devices Confidential Information—

—Analog Devices Confidential Information—

Test Harvesting

Some tests are difficult to write because they involve trying todemonstrate interaction between signals that you do not have direct control over.

Example: Concurrent transactions from different system busses into a bus arbiter.

The conditions to test for can usually be easily described with PSL.Randomization can then be used to stimulate the device, the PSL coverage statements can be tracked, and tests with the desired behavior can be “harvested” from random test runs as a directed test.

More efficient than writing by hand.If a test breaks due to a design change, the same procedure can be repeated to “harvest” another testcase.

—Analog Devices Confidential Information—

Other Uses

Speedpath testingSimple scripts can be used to generate assertions which “fire”when certain speedpaths are hit.Once this is instrumented, the entire testsuite can be run to extract the best tests for hitting the desired speedpaths.

Clock analysisAgain, simple perl scripts generated assertions for firing when clocks in different areas of the chip were active.This enabled a detailed verification and analysis of different low power modes where clocks were dynamically enabled and disabled.

—Analog Devices Confidential Information—

Enhanced Visibility

Coverage assertions can be thought of as a way to verify waveforms in batch mode.PSL can be efficient way to communicate testing needs between design and verification engineers.PSL can be reviewed more easily during a final testplan review than the testcode itself.

—Analog Devices Confidential Information—

Our Future Goals & Strategy

All new checkers will be written in PSL.Easier to maintain.Easier to reuse.Allows leverage into formal verification effort.Provides base for tying tests to assertions for coverage.

With very few exceptions, all future directed tests will requireat least one assertion for coverage to determine the pass/fail of the test.Testplans reference assertions, and the assertion strategy, in addition to the directed tests and random test strategy.

—Analog Devices Confidential Information—

Summary

PSL and other assertion methods are crucial because they allow the same efforts to be leveraged in multiple ways.While many tools and methodologies are still fairly new, the value added is worth the effort today.

—Analog Devices Confidential Information—

More Information

Using PSL/Sugar for Formal and Dynamic VerificationBen Cohen, Srinivasan Venkataramanan, and Ajeetha Kumari

Using PSL for Functional Coverage WebinarKelly Larsonhttp://www.cadence.com/company/events/webinars.aspx

Accellera PSL 1.1 LRM (Language Reference Manual)http://www.accellera.org/

IEEE P1850http://www.eda.org/ieee-1850/

Cadence ABV documentsUnder docs/ directory of local Cadence tools (IUS) installation directorywriteabv.pdf - Simulation-Based Assertion Writing Tutorialabvguide.pdf - Simulation-Based Assertion Checking Guideabvtutorial.pdf - Simulation-Based Assertion Checking Tutorial

—Analog Devices Confidential Information—

Presented By:Kelly Larson

Austin Design Center

Analog Devices, Inc.6500 Riverplace Blvd.Bldg IV, Suite 300Austin, TX 78730PHONE 512-427-1094kelly.larson@analog.com