SEDS Research GroupSchool of EECS, Washington State University Annual Reliability & Maintainability Symposium January 30, 2002 Frederick T. Sheldon and

SEDS Research Group School of EECS, Washington State University

Annual Reliability & Maintainability SymposiumJanuary 30, 2002

Frederick T. Sheldon and Hye Yeon Kim

Software Engineering for Dependable Systems (SEDS) Research Laboratory

School of Electrical Engineering and Computer Science

Washington State University

Validation of Guidance Control

Software Requirements Specification

for Reliability and Fault-Tolerance

SEDS Research Laboratory School of EECS, Washington State University

Overview Goal: Show the feasibility of this analysis approach

using a industrial strength SRS to ensure:Completeness and ConsistencyFault-tolerance

Specification Under StudyA NASA provided Guidance and Control Software (GCS)

development specification for the Viking Mars Lander. Analysis Approach

Using Zed to specify the data Using Statecharts : Statemate for dynamical analysis

Summary and Future study


Introduction

Why Requirements Specification?Cost, Time, and Effort

Defect detected phase Typical cost of correction

Requirements Specification $100 - $1,000

Coding/Unit Testing $1,000 or more

System Testing $7,000 - $8,000

Acceptance Testing $1,000 - $100,000

After Implementation Up to millions of dollars


Reliable Specification

Is Correct Complete, consistent and robust

Can the specification be trusted while minimizing the risk of costly errors?

How to analyze the specification to prevent the propagation of errors into the downstream activities?


Consistency and CompletenessCompleteness: The lack of ambiguity

Incomplete if …… the system behavior is not specified

precisely because the required behavior for some events or conditions is omitted or is subject to more than one interpretation.

ConsistencyThe Specification is free from conflicting

requirements and undesired nondeterminism.


Fault ToleranceFaults

A fault is a feature of a system that precludes it from operating according to its specification

– H. Ammar, B. Cukic, C. Fuhrman, and A. Mili, A comparative Analysis of HW and SW fault tolerance: Impact on software reliability engineering, 1999

Fault ToleranceThe ability to respond to unexpected system

failure (detection and mask/recover)


Guidance and Control Software

Software Requirements – GCS Dev. Spec.The system was designed to provide software

control of the embedded sensors and actuators of the Viking Mars Lander during the terminal decent (landing) phase.

ARSP (Altimeter Radar Sensor Processing)The ARSP module reads data provided by the

altimeter radar sensor to determine the lander’s altitude from the Mars surface.


Mars Lander trajectories


Velocity – Altitude Contour


EXTERNALRUN_PARAMETERS

SENSOR_OUTPUTGUIDANCE_STATE

TDLRSP.3

GSP.4

ARSP.2

ASP.1

TSP.5

TDSP.6



Zed Overview

Clarifying ambiguities

Identify assumptions

Correctness checking

Mathematical proofs

Giving an in-depth understanding of the

SRS


StatechartsVisual formalism: Diagrammatic in natureTestability is provided through symbolic

simulationPredevelopment evaluation through

Fault Injection

Statemate consists of …Activity chartStatecharts


Natural Language based SRS

Inherently ambiguous risking the possibility of multiple interpretations


Zed Schema


From NL to ZedDiscovered Ambiguities

The confusing definition of variable “Rotation”, and direction of the rotation.

The type assigned to the AR_COUNTER variable was unclear.

An undefined 3rd order polynomial.Where the AR_COUNTER should be

modified?


Statecharts Model: Activity chart


Statecharts Model: Statechart 1


Statecharts Model: Statechart 2


Some Theory …Set of Inputs

()

Set of Outputs

Unknowns ()

Known

Known Safe

Unsafe

AssumedSafe

Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators



Paradigms …

Software Failures:“Software does not fail - it just does not

perform as intended” Professor Nancy Leveson, MIT

Design and test for functionality:

Also specify what the system

should not do. . .

. . . then test it!


Some Theory… lets take a second look

Set of Inputs ()

Set of Outputs

Unknowns ()

Known

Known Safe

Unsafe

AssumedSafe

Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators

Fault Injection(added known)


Testing and Fault InjectionBy using symbolic simulation in

Statemate

Boundary TestingInput that is inside of the variable rangeInput that is outside of the variable range

Fault InjectionState variable alternationState transition redirection


Testing Results

ARSP Specification Test Input and Output

Variable Case 1 Case 2 Case 3 Case 4 Case 5

Input

FRAME_COUNTER 2 2 1 1 3

AR_STATUS - - [0, 0, 0, 0] - [0, 1, 0, 0]

AR_COUNTER -1 19900 -1 20000 -1

Output

AR_STATUS KP KP [1, 0, 0, 0] [0, -, -, -] [1, 0, 1, 0]

K_ALT KP KP [1, 1, 1, 1] [1, -, -, -] [0, 1, -, 1]

AR_ALTITUDE KP KP [*, -, -, -] [2000,-,-,-] KP


Detailed Testing Results - Case 1

Initial valuesFinal values

Initial variable values are being calculated based on the given equations.

Variable Case 1

Input

FRAME_COUNTER 2

AR_STATUS -

AR_COUNTER -1

Output

AR_STATUS [1, 0, 0, 0]

K_ALT [1, 1, 1, 1]

AR_ALTITUDE [2000, -, -, -]

[1, 1, 0, 0]

[1, 1, 1, 1]

[2000, 2000, -, -]


Fault Injection ResultsAltered state variable

FRAME_COUNTER AR_COUNTER AR_STATUS Fault injected State Case

1 Case

2 Case

3 Case

4 Case

5 Case

1 Case

2 Case

3 Case

4 Case

5 Case

1 Case

2 Case

3 Case

4 Case

5

CURRENT_STATE x x x x x x x x x x x x x x x

KEEP_PREVIOUS_VALUE b b b b b b b b b b b b b b b

CALCULATION b b b b b b b x x x b b x b x

ODD b b b b b b b x x x b b x b x

ESTIMATE_ALTITUDE b b b b b b b N/A b b b b N/A b b

CALCULATE_ALTITUDE b b b b b b b b x b b b b b b

KEEP_PREVIOUS b b b b b b b b b b b b b b b

DONE b b b b b b b b b b b b b b b

x incorrect outputs, b no defect


Detailed Fault Injection Results

Change FRAME_COUNTER at CURRENT_STATE from 2 to 3

Variable Case 1

Input

FRAME_COUNTER 2

AR_STATUS -

AR_COUNTER -1

Output

AR_STATUS [1, 0, 0, 0]

K_ALT [1, 1, 1, 1]

AR_ALTITUDE [2000, -, -, -]

[1/0, 1, 0, 0]

[1, 1, 1, 1]

[*, 2000, -, -]


SummaryInterpretation from NL to Zed

Clarifying ambiguities

Interpretation from Zed to StatechartsClarifying misinterpreted Zed specificationIterative process

Boundary Testing, Fault Injection analysisReveals weak point(s) in the systemFault Tolerance validation


ConclusionUsing this combination of FMs we could:

Clarify ambiguitiesValidate Correctness, Completeness, and ConsistencyValidate Fault tolerance features of the SRS

This approach enabled us to:Avoid the problems that result when incorrectly

specified artifacts force corrective rework.Minimize the risk of costly errors being propagated into

downstream activities


Future StudyBuild concrete translation rules between the

methods

Find an effective algorithm to automate the process

Validate the algorithm for the different (domain/ application specific) critical software requirements

In depth comparative study with other approaches

Documents

SEDS Research GroupSchool of EECS, Washington State University Annual Reliability & Maintainability Symposium January 30, 2002 Frederick T. Sheldon and