Automated On-Target Software Testing with … On-Target Software Testing with VectorCAST ... Software unit and integration testing focuses on isolating a ... Automated On-Target Software

Vector Software W H I T E P A P E R

© Vector Software, Inc. - All rights reserved.

VectorCAST.com

Automated On-Target Software Testing with VectorCAST

Introduction One of the challenges many projects face is how to quickly and easily test their embedded application on live hardware – either using a simulator or evaluation board (Processor-In-the-Loop: PIL) or using the production hardware (Hardware-In-the-Loop: HIL). PIL and HIL testing are especially important for safety-critical applications, in industries such as avionics, automotive, medical, and railway, where it is often a requirement for certification. Even for projects with less stringent testing requirements, testing in an environment that is as close as possible to the production environment will result in more errors being discovered. Hardware based testing presents many challenges, including: dealing with limited target resources, minimizing impacts on execution time, automating target communication mechanisms for data collection, and training teams to configure and troubleshoot target execution. Vector Software provides a family of tools that solves these problems by automating the test configuration and execution process so that engineers can focus on building and executing accurate and effective software tests. Specifically, VectorCAST helps with two key areas of target-based testing:

> Software unit and integration testing, to prove application correctness

> Code coverage analysis during testing, to prove test completeness This whitepaper will detail the benefits of Hardware based testing, and discuss how VectorCAST can be used with a variety of compilers, emulators, and target hardware.

Unit and Integration Testing Software unit and integration testing focuses on isolating a portion of the application code, typically a few C or C++ source files, or Ada packages, and thoroughly testing that sub-set to ensure it works properly and meets its low-level requirements. Code coverage analysis measures what portions of the application code have been executed. Code coverage is the single easiest way to quantify the thoroughness of your test activities allowing you to make an informed decision on how much testing is enough.



VectorCAST.com Page2

By using the VectorCAST family of products, you can easily create suites of tests for all of your

embedded targets. The built-in expected values for these tests formalize the correct behavior

of the application, and the code coverage reporting quantifies the completeness of testing. As

the number of tests increase over time, they can be automatically run on a regular basis in a

regression testing scenario, resulting in always available dashboard reports of your current test

status.

VectorCAST/C++ pass/fail results and code coverage viewer

Code Coverage Analysis during System Testing Thoroughness of testing is also a concern when it comes to system testing. The only reasonable metric for judging the adequacy of system testing is code coverage analysis. VectorCAST/Cover can be used to automatically instrument an application during build time. The resultant instrumented application will then capture code coverage metrics as normal system testing is performed. VectorCAST/Cover instrumentation has been highly optimized to support the size and timing constraints that are common in embedded environments. VectorCAST/Cover seamlessly integrates with a development team’s existing build tools. The coverage metrics can either be sent to the IDE I/O window or to an on-target serial port. The coverage from individual tests can be maintained independently within VectorCAST/Cover allowing for differential and aggregate coverage analysis.




VectorCAST and Cross-Compilers When VectorCAST/C++ or VectorCAST/Ada is combined with the VectorCAST Run-Time Support Package (RSP) completely automated Hardware based testing becomes a reality. VectorCAST/RSP provides an abstraction layer for the compiler and target hardware allowing for push-button test execution, regardless of the complexity of the process required to build and execute tests for your application. VectorCAST leverages the underlying compiler and emulator tools to support a wide variety of chip architectures, JTAG emulators, and development boards, providing the most sophisticated and seamless target testing environment possible. The VectorCAST tools are integrated with more than 30 different compiler families and hundreds of combinations of compilers, emulators, and hardware devices, including those from:

Green Hills

Wind River

IAR Systems

Open source (gcc/gnu)

Microsoft

Code Composer (TI)

Keil / ARM

CodeWarrior (Freescale)

AdaCore

Analog Devices Once the user selects an appropriate compiler and target configuration from the GUI, VectorCAST has built- in, all of the information it needs to compile and link its test harness using the chosen compiler. The VectorCAST target configurations also include an execution script that automatically connects to the debugger, loads the test harness, starts execution, and collects test results and coverage data. The VectorCAST compiler settings can be easily modified by the user to match specific build settings such as processor variants and optimization levels. VectorCAST makes these modifications simple by providing direct access to all configuration parameters. In addition to supporting automated test execution, the VectorCAST/RSP supports source code debugging via the underlying compiler debugger IDE, allowing test failures to be easily resolved. Because of the modular nature of the VectorCAST/RSP, support for new IDE versions and processors are easily added. Additionally, because the VectorCAST test cases exist in isolation from the compiler and target configuration, users can easily execute the same tests using any combination of compiler versions and microprocessors simply by creating a new compiler configuration node within VectorCAST.




Finally, for new designs, testing can initially be performed using a native compiler or instruction set simulator, and then transition to production hardware for final testing. There are 3 main capabilities of VectorCAST/RSP:

> Complete support for all C, C++ or Ada language extensions defined by the compiler

> Automatic downloading of the test harness executable to the target or simulator

> Automatic capture of the test results during execution Executing a test with VectorCAST is as simple as pressing a button.

Conclusion In conclusion, using the VectorCAST suite of tools together with a cross-compiler IDE provides an elegant solution to the challenges faced by organizations striving to increase the quality of their embedded application software. It provides push-button Hardware-In-the-Loop and Processor-In-the-Loop testing, allowing engineers to focus their time on writing good tests. VectorCAST supports pass/fail analysis during software unit and integration testing of C, C++ and Ada applications. In addition, it supports code coverage analysis during the entire software development lifecycle. Code coverage analysis is the simplest way to assess the thoroughness of testing.

About Vector Software Vector Software, Inc., is the leading independent provider of automated software testing tools for developers of safety and business critical embedded applications. Companies worldwide in automotive, aerospace, medical devices, industrial controls, rail, and other industries, rely on Vector Software’s VectorCAST™. By automating and managing the complex tasks associated with unit, integration, and system level testing, VectorCAST helps organizations accelerate the development and ensure the reliability of their embedded software applications.

Vector Software, Inc. 1351 South County Trail, Suite 310 East Greenwich, RI 02818 USA T: 401 398 7185 F: 401 398 7186 E: [email protected]

Vector Software Golden Cross House 8 Duncannon St. London WC2N 4JF, UK T: +44 203 178 6149 F: +44 20 7022 1651 E: [email protected]

Vector Software St. Töniser Str 2a 47906 Kempen Germany T: +49 2152 8088808 F: +49 2152 8088888 E: [email protected]

Vector Software Rm 403, Building 6, No.88 Daerwen Rd, Zhangjiang Hi-tech Park Pudong New Area Shanghai 201203 China T: 21- 3126 8126 F: 21-5132 8526 E: [email protected]

Documents

Automated On-Target Software Testing with … On-Target Software Testing with VectorCAST ... Software unit and integration testing focuses on isolating a ... Automated On-Target Software