8/10/2019 Margaret Stringfellow

1/32

8/10/2019 Margaret Stringfellow

2/32

2

Outline

Safety Engineering and its application to Software

Safety Driven Design

The Process

Example: Martian Lander

Comparison to other Methods and Results

8/10/2019 Margaret Stringfellow

3/32

Software in Automotive

and Aerospace Systems

Lines of Code:

MER (Mars Rovers) 428,000

F-35 (Joint Strike Fighter) 5.7 million

Modern day car: 100 million

How can we be sure the software is safe? (Will not

cause a loss event?)

Testing?

Probability of sw failure is?

3

8/10/2019 Margaret Stringfellow

4/32

Safety Engineering

Broad Definition of Safety Loss Events (accident) can

be:

A car that wont start because of a Software Error in the

Computer (Recall!)A spacecraft that crashes into the surface of the planet

Hazard: System state that may permit an Accident

The purpose of Safety Engineering is to identify systemhazards and prevent systems from transitioning to anunsafe (hazardous) state.

4

8/10/2019 Margaret Stringfellow

5/32

STAMP Accident Model

Systems-Theoretic Accident Model and Processes(STAMP)

Accidents are not the last event in a chain of events

Accidents are the result ofthe inadequate control ofsystem state

Basic premise is to prevent accidents by enforcing safetyconstraints on system behavior (controlling hazardous

system states)

Safety is viewed as a control problem, not a failureproblem

5

8/10/2019 Margaret Stringfellow

6/32

System Safety

System accidents:

Catastrophic outcome arising from interactions betweenoperating components

Each component functions within an acceptableperformance range, or in the context of an appropriate

objective

Safety is Emergent

Safety must be Built-in From the Beginning Cheaper

More Effective

6

8/10/2019 Margaret Stringfellow

7/32

7

STAMP-Based Hazard Analysis (STPA)

Goals (same as any hazard analysis)

Identification of system hazards and related safety constraints

necessary to ensure acceptable riskAccumulation of information about how hazards can occur.

Use info to eliminate, mitigate and control hazards in system

design, development, manufacturing, and operations

8/10/2019 Margaret Stringfellow

8/32

8

Since hazardous states can be prevented through

appropriate control (enforcing safety constraints), this

hazard analysis method seeks to find instances of

Inadequate ControlInadequate control occurs when there are state transitions to

hazardous states

The commands or actions that lead to violation of safety

constraints:

Inadequate Control Actions

Controlling States

8/10/2019 Margaret Stringfellow

9/32

9

Inadequate Control Actions

Identify inadequate control actions

1.

A required control action is not provided or notfollowed

2.

An incorrect or unsafe control action is provided3.

A potentially correct control action is provided too

late or too early (at the wrong time)

4.

A correct control action is stopped too soon.

8/10/2019 Margaret Stringfellow

10/32

Control Structure

10

8/10/2019 Margaret Stringfellow

11/32

11

Controlled

Process

Inadequate

Sensor

Operation

Inadequate

Actuator

Operation

Process

Model

Wrong

Inadequate

Control

AlgorithmControl

Input

Wrong or

Missing

Feedback

Wrong or

Missing

Inadequate control

Commands

Process Input

Wrong or

Missing

Process Output

Wrong or Missing

Disturbances

Unidentified

or Out of

Range

Controller

Sensor(s)

Actuator(s)

Control Flaws and Generic Control Loop

8/10/2019 Margaret Stringfellow

12/32

12

How to Perform STPA

1.

High-level Hazard Analysis:

Indentify Accidents or Loss Events

Hazards

High-level Safety Constraints2.

Create and Analyze Control Structure to IdentifyInadequate Control Actions

3.

Identify Control Flaws

In the design4.

Change design to eliminate, mitigate, or controlpotentially unsafe control actions and behaviors.

Or accept

5.

Iterate

8/10/2019 Margaret Stringfellow

13/32

Design For Safety

Goals: To get safety designed into the system rather than

added on at the end.

Most hazard analyses can only be applied to systems thatalready exist.

FMEA

Hazop

Design for Safety attempts to get safety considerationsmade at the same time performance trades are made.

How? Use STPA to drive design decisions.

13

8/10/2019 Margaret Stringfellow

14/32

Process

Overview

Identify and Characterize theProblem to be Solved: System

Level Goals, Loss Events,

Hazards, Safety Constraints

and RequirementsUse STPA

(Inadequate

ControlActionsand Control

Flaws) to analyze

high-level design

and refine safety

constraints, orchange design.

Iterate.

14

Create Design

8/10/2019 Margaret Stringfellow

15/32

15

Characterize the Problem to be Solved

8/10/2019 Margaret Stringfellow

16/32

Simple Martian Lander Example:

System Characterization

Mission Goals

G1 Land on the surface of Mars and collect needed scientificdata.

G2 Transmit data back to Earth.

16

8/10/2019 Margaret Stringfellow

17/32

Loss Event, Hazard, Safety Constraints

"#$$ %&'()*Accident.1 Spacecraft experiences uncontrolled descent

into the surface of Mars and is consequently destroyed.

Hazard.1 Spacecraft comes in contact with the surface with an

impact greater than 100 N.

SafetyConstraint.1 The spacecraft must control its descent to the

surface of Mars so that its impact force is less than 100N.

SafetyConstraint.2 The spacecraft must be protected from impact

with the surface. Rationale: The spacecraft structure is susceptibleto damage even with gentle impacts and must have some type of

protection.

17

8/10/2019 Margaret Stringfellow

18/32

!"##"$% '()(* +(,-".(/(%0#1

The mission shall collect and analyze soil samples at XYZcoordinates.

Rationale: Scientists believe this location may contain ice and

discovering the presence on water on Mars is of great interest.

Customer-derived system design constraintsDC1. The mission must be carried out with existing technologies andspace exploration infrastructures as needed (i.e., technologies ratedat Technology Readiness Level TBD as defined by NASA).

Rationale: While technology development is expected to be anongoing activity of NASA, it is assumed to be beyond the mandate of

the mission

Customer programmatic constraints (e.g., budgets,etc.)

18

8/10/2019 Margaret Stringfellow

19/32

19

High Level Design

8/10/2019 Margaret Stringfellow

20/32

Design High-Level System Control

Structure

20

8/10/2019 Margaret Stringfellow

21/32

21

Create High-level Design

to Enforce Safety Constraints

SafetyConstraint.1: The spacecraft must control its

descent to the surface of Mars so that its impact force

is less than 100N.

Design Decision 1: Use Thrusters to Control Descent

rate of Spacecraft.

8/10/2019 Margaret Stringfellow

22/32

8/10/2019 Margaret Stringfellow

23/32

23

Perform 1stIteration of STPA

(How can constraints be violated?)

SafetyConstraint.1: The spacecraft must control its

descent to the surface of Mars so that its impact force

is less than 100N. .

ICA.1 Spacecraft descent control is not engaged.

ICA.2 Spacecraft descent control allows descent

velocity that are to fast.

ICA.3 Spacecraft descent control is activated too

late.ICA.4 Spacecraft descent control is de-activated too

soon.

8/10/2019 Margaret Stringfellow

24/32

8/10/2019 Margaret Stringfellow

25/32

8/10/2019 Margaret Stringfellow

26/32

8/10/2019 Margaret Stringfellow

27/32

Comparisons and Results

27

8/10/2019 Margaret Stringfellow

28/32

8/10/2019 Margaret Stringfellow

29/32

29

STPA Comparisons (2)

Concrete model (not just in head) Not physical structure (HAZOP) but control (functional)

structure

General model of inadequate control (based on control

theory)

HAZOP guidewords based on model of accidents being

caused by deviations in system variables

Includes HAZOP model but more general

Compared with TCAS II Fault Tree (MITRE)

STPA results more comprehensive

Included Ueberlingen accident

8/10/2019 Margaret Stringfellow

30/32

30

Ballistic Missile Defense System (BMDS)

Non-Advocate Safety Assessment using STPA

A layered defense to defeat all ranges of threats in allphases of flight (boost, mid-course, and terminal)

Made up of many existing systems (BMDS Element)

Early warning radars

Aegis

Ground-Based Midcourse Defense (GMD)

Command and Control Battle Management and

Communications (C2BMC) Others

MDA used STPA to evaluate the residual safety risk ofinadvertent launch prior to deployment and test

8/10/2019 Margaret Stringfellow

31/32

31

Results

Deployment and testing held up for 6 months because somany scenarios identified for inadvertent launch. In many ofthese scenarios:

All components were operating exactly as intended

Complexity of component interactions led to unanticipated

system behavior

STPA also identified component failures that could causeinadequate control (most analysis techniques consider onlythese failure events)

As changes are made to the system, the differences are

assessed by updating the control structure diagrams andassessment analysis templates.

Adopted as primary safety approach for BMDS

8/10/2019 Margaret Stringfellow

32/32