Kristin Wortman Johns Hopkins University Applied Physics ...flightsoftware.jhuapl.edu/files/2012/FSW12_Wortman.pdf2012 FSW Workshop Nov 7 -9 7 . Red color – reuse potential (partial

Kristin Wortman Johns Hopkins University Applied Physics Laboratory

RBSP & SPP Acceptance Test Lead [email protected]

1 2012 FSW Workshop Nov 7-9 11/12/2012 9:22 AM

Overview of APL’s unmanned spacecraft software system test structure

Attributes of the Command and Data Handling (CDH) Flight Software (FSW) for NASA’s RBSP mission

Process followed for independent requirements verification of RBSP CDH FSW

Organization of the RBSP CDH FSW requirement and test specifications

Example of a RBSP test design and discuss its reuse potential

Discuss the mission dependencies of our test designs Plan for test reuse on future APL supported missions


3 2012 FSW Workshop Nov 7-9

Reference [1] 11/12/2012 9:22 AM

Considered part of our software development process… Independent Acceptance Test (IAT).

Independent test engineers follow a formal verification process.

Black box test design is driven by one or more functional requirement(s) to be verified.

Document test designs (Test Specifications) ◦ Requirements and test specifications are developed and

maintained using IBM Rational’s Dynamic Object Oriented Requirements Software (DOORS)

◦ Link test cases to the CDH FSW requirement specifications.

Goal is to achieve a high confidence level in flight software before release for Integration and Test and Mission Operations activities.


Considered a critical Computer Software Configuration Item (CSCI) that requires independent requirements verification (also referred to as Independent Acceptance Test)

CDH functionality implemented in 17 APL

developed applications to work with Goddard’s core Flight Executive (cFE) middleware

421 total functional requirements to be verified

by Independent Acceptance Test (IAT)

2012 FSW Workshop Nov 7-9 5 11/12/2012 9:22 AM

Criticality rating and verification method were available for each CDH FSW requirement

Two types of requirement verification tests: ◦ Functional (positive and negative tests) verify most critical requirements test steps are predetermined and peer reviewed scripted and repeatable test ◦ Exploratory (session-based tests) verify less critical requirements test steps evolved during session following a charter no scripts, not a repeatable test


2012 FSW Workshop Nov 7-9 7

Red color – reuse potential (partial or whole) of application and associated functional test case(s) * Requirements apply to all applications or verified in another application’s test case

11/12/2012 9:22 AM

CDH FSW Requirements Specification Reference [2]

CDH FSW Acceptance Test Specifications Reference [3]


CDH FSW Requirements Specification Reference [2]

CDH FSW Acceptance Test Specifications Reference [3]


2012 FSW Workshop Nov 7-9 10

Header(8 bits)

Segment Data Field(up to 1016 bytes)

Telecommand Transfer Frame

Header(5 bytes)

Frame Data Field(Up to 1017 bytes)

Error Cntl.(2 bytes)

Telecommand Codeblock #1

Data(56 bits)

Parity (7 bits) + 1 bit “0” fill




Data(56 bits)


Telecommand Codeblock #N

Data(56 bits)


CLTU

Start Sequence(16 bits)

Tail Sequence(64 bits)

Telecommand Codeblock #1 (64

bits)

TC Codeblock #2 (64 bits)

TC Codeblock #N (64 bits)

Telecommand Segment

PACKET GENERATION (Packet Layer)

SEGMENTATION(Segmentation Layer)

TRANSFER FRAME(Transfer Layer)

CODE BLOCKS (Coding Layer)

CLTU (Command Link Transmission Unit)

(Coding Layer)

FARM Control

Data(56 bits)

Version Number (2 bits)

Virtual Channel (6 bits)

Fram Seq # (8 bits)

Frame Length (10 bits)

Bypass Flag (1

bit)

Cmd Cntl Flag (1

bit)Spare (2 bits)

S/c ID (10 bits)

Seq Flags (2 bits)

MAP ID (6 bits)

Telecommand Packet

Primary Header (6 bytes)

Sec Hdr (SC 2 bytes, INST 4 bytes)

SC Packet data (up to 1008 bytes)INST Packet data (up to 246 bytes)

Version Number (3 bits)

Seq Flags (2 bits)

Packet Length (16 bits)

Seq Cnt (14 bits)

Type (1 bit)

Sec Hdr Flag

(1 bit)Ap ID

(11 bits)

Start Sequence - EB90HTail Sequence - C5C5 C5C5 C5C5 C579

Telecommand Codeblock #1 (64

bits)

Criticality (1 bit)

MET (32 bits)

Checksum (8 bits)

Function Code(7 bits)

Start Sequence – C0

Always 1

SC Sec Hdr

INST Sec Hdr

Reference [2] 11/12/2012 9:22 AM


Mission dependencies in the test design ◦ Use of mnemonics for command and telemetry definitions RBSP developed a tool to “find and replace” mnemonics in test

scripts as these mnemonics matured prior to launch ◦ Use of the RBSP test environment & tool to generate all

valid and invalid command files as test inputs ◦ Implementation of the test design using the scripting

language (JAS) which is supported by Telemetry West’s L3 InControl and is being used for the RBSP ground system

◦ Specific fields in the CCSDS Protocol (e.g. SC id, virtual channel) used for the telecommand transfer layer


Test tool usage and test environment setup procedures have been specified in a separate section of the RBSP ATS, which is referenced in test steps. ◦ Can be easily replaced in this section

High-level test design is reusable, details contained in the test steps will require minor changes (e.g. nomenclature used for command input files).

Test scripts can be reused if same ground system is used for a mission. ◦ Modify with script to “find and replace” command and

telemetry mnemonics ◦ Environment setup procedures


Two upcoming APL supported missions plan to use cFE architecture for CDH FSW

◦ Solar Probe Plus cFE architecture for CDH FSW Reuse of the common applications in CDH FSW Reuse of high-level functional requirements verification test

designs for common applications Reuse and modify test scripts used for RBSP and address the

mission dependencies

◦ Precision Tracking Space System cFE architecture for CDH FSW Reuse of the common applications in CDH FSW Reuse the high-level functional requirements verification

test designs for the common applications Rewrite the test scripts using different scripting language


Use command and telemetry descriptions rather than actual mnemonics in test designs.

Establish and follow conventions for writing

scripts to simplify replacement of test environment set up.

Automate, as much as possible, the verification steps. Establish and use a common test framework.


[1] Kristin Wortman, Management of Independent Software Acceptance Test in the Space Domain: A Practitioner’s View, IEEE Aerospace Conference, Big Sky, MT, 2012.

[2] JHU/APL Product Lifecycle Management, Document 7417-9604, RBSP Command and Data handling Flight Software Requirements Specification.

[3] JHU/APL Product Lifecycle Management, Document 7417-9493, RBSP Command and Data Handling Flight Software Acceptance Test Specification.



Documentation set ◦ Acceptance Test Plan (ATP) ◦ Acceptance Test Specifications (ATS) ◦ Test report ◦ Verification matrix

Process followed for requirement

verification ◦ Link one or more requirement to a test case ◦ Peer review process for ATP and ATS

Repeatable test design ◦ Regression test suite

Test records ◦ Artifacts ◦ Issues ◦ Test execution

Commercial tool set


Reference [1] 11/12/2012 9:22 AM

19

Reference [1] 11/12/2012 9:22 AM 2012 FSW Workshop Nov 7-9

Assigned to each requirement by the software lead and reviewed by the Mission Software System Engineer

Available in the C&DH requirements document

Method Description

Test * Verify with positive and negative tests

Inspection Inspect code used to implement the requirement

Demonstration Demonstrate the functionality Analysis Provide a logical argument that requirement

has been met * All C&DH requirements were “Test” verification method.


Reference [1] 11/12/2012 9:22 AM

Documents

Kristin Wortman Johns Hopkins University Applied Physics ...flightsoftware.jhuapl.edu/files/2012/FSW12_Wortman.pdf2012 FSW Workshop Nov 7 -9 7 . Red color – reuse potential (partial