A Review of CAT II/III LAAS Integrity Requirements and their Antecedents Stanford GPS Laboratory Group Meeting 4 August 2006 Sam Pullen Stanford University

A Review of CAT II/III LAAS Integrity Requirements and their Antecedents

Stanford GPS Laboratory Group Meeting

4 August 2006

Sam Pullen

Stanford University

(with lots of help from Tim Murphy of Boeing)

4 August 2006 CAT II/III Integrity Requirements and Antecedents

2

English Word of the Day…

• Antecedent: (Webster online dictionary)1 : a substantive word, phrase, or clause whose denotation is referred to by a pronoun (as John in "Mary saw John and called to him"); broadly : a word or phrase replaced by a substitute grammar only

2 : the conditional element in a proposition (as if A in "if A, then B") grammar only

3 : the first term of a mathematical ratio rarely used

4 a : a preceding event, condition, or cause b plural : the significant events, conditions, and traits of one's earlier life very general

5 a : PREDECESSOR; especially : a model or stimulus for later developments b plural : ANCESTORS, PARENTS


3

Presentation Outline

• Review of LAAS Precision Approach Requirements

• Antecedents of these requirements:

– ICAO Annex 10 Requirements for ILS

– FAA AC 25.1309 and AC 120-28D wording

– FAA Hazard Risk Index table

– Total Aircraft Safety sub-allocation

• What should the “real” be, and how should it be derived?

– Some initial thoughts…

Precision Approach Requirements in Updated LAAS MASPS

(RTCA DO-245A, December 2004)


5

GBAS Service Level (GSL) Definitions

Table 1-1 (Section 1.5.1) of DO-245A

GSL Typical Operation(s) which may be Supported by this Level of Service

A Approach operations with vertical guidance (performance of APV-I designation)

B Approach operations with vertical guidance (performance of APV-II designation)

C Precision approach to lowest Category I minima

D Precision approach to lowest Category IIIb minima, when augmented with other airborne equipment

E Precision approach to lowest Category II/IIIa minima

F Precision approach to lowest Category IIIb minima


6

GSL Requirements Table

Table 2-1 (Section 2.3.1) of DO-245A

GSL

Accuracy Integrity Continuity

95% Lat. NSE

95% Vert. NSE

Pr(Loss of Integrity)

Time to Alert

LAL VALPr(Loss of Continuity)

A 16 m 20 m2 × 10-7 / 150

sec6 sec 40 m 50 m 8 × 10-6 / 15 sec

B 16 m 8 m2 × 10-7 / 150

sec6 sec 40 m 20 m 8 × 10-6 / 15 sec

C 16 m 4 m2 × 10-7 / 150

sec6 sec 40 m 10 m 8 × 10-6 / 15 sec

D 5 m 2.9 m10-9 / 15 s (vert.);

30 s (lat.)2 sec 17 m 10 m 8 × 10-6 / 15 sec

E 5 m 2.9 m10-9 / 15 s (vert.);

30 s (lat.)2 sec 17 m 10 m 4 × 10-6 / 15 sec

F 5 m 2.9 m10-9 / 15 s (vert.);

30 s (lat.)2 sec 17 m 10 m

2 × 10-6 / 15 s (vert.); 30 s (lat.)

Antecedents of Precision Approach Requirements

1: FAA Hazard Risk Index

Useful reference: Ch. 3 of FAA System Safety Handbook (12/30/00)

http://www.faa.gov/library/manuals/aviation/risk_management/ss_handbook/media/Chap3_1200.PDF


8

Cat III

FAA Risk Severity Classifications*

• Minor: failure condition which would not significantly reduce airplane safety, and which involve crew actions that are well within their capabilities

• Major: failure condition which would significantly: (a) Reduce safety margins or functional capabilities of airplane (b) Increase crew workload or conditions impairing crew efficiency (c) Some discomfort to occupants

• Severe Major (“Hazardous” in ATA, JAA): failure condition resulting in more severe consequences than Major: (a) Larger reduction in safety margins or functional airplane capabilities (b) Higher workload or physical distress such that the crew could not be relied upon to perform its tasks accurately or completely (c) Adverse effects on occupants

• Catastrophic: failure conditions which would prevent continued safe flight and landing (with probability 1)

* Taken from AC No. 25.1309-1A, AMJ 25.1309, SAE ARP4761 (JHUAPL summary)

Cat I


9

FAA Hazard Risk Index (HRI) Table

ConsequenceProb. Of Occurance

Catastrophic Hazardous Major Minor NoEffect

Frequent (>10-2) 1 3 6 10 21Reasonably Probable

(10-2 to 10-5)2 5 9 14 22

Remote (10-5 to 10-7) 4 8 13 17 23Extremely Remote

(10-7 to 10-9)7 12 16 19 24

Extremely Improbable(<10-9)

11 15 18 20 25

Hazard Risk Index Acceptance Criteria1-6 Unacceptable7-10 Undesirable11-18 Acceptable, but FAA review required19-25 Acceptable

Cat. I ILS caseCat. III ILS case

•Several versions exist, all with essentially the same meaning

•Source of this version: 1999 Johns Hopkins Applied Physics Laboratory “GPS Risk Assessment Study” final report

http://www.faa.gov/asd/international/GUIDANCE_MATL/Jhopkins.pdf


2: FAA Advisory Circulars Defining Certification and Airworthiness Criteria

• For AC 25.1309-1A, “System Design and Analysis,” 6/21/88:

http://www.airweb.faa.gov/Regulatory_and_Guidance_Library%5CrgAdvisoryCircular.nsf/0/50BFE03B65AF9EA3862569D100733174?OpenDocument

• For AC 120-28D, “Criteria for Approval of Category III Weather Minima for Takeoff, Landing, and Rollout,” 7/13/99:http://www.airweb.faa.gov/Regulatory_and_Guidance_Library%5CrgAdvisoryCircular.nsf/0/BBADA17DA0D0BBD1862569BA006F64D0?OpenDocument


11

Key Elements of AC 25.1309-1A

• AC 25.1309-1A is the primary basis for safety certification within the FAA

• AC 25.1309-1A specifies a “fail-safe” policy (quote):1) In any system or subsystem, the failure of any single element,

component, or connection during any one flight (e.g., brake release through ground deceleration to stop) should be assumed, regardless of its probability. Such single failures should not prevent continued safe flight and landing, or significantly reduce the capability of the airplane or the ability of the crew to cope with the resulting failure conditions.

2) Subsequent failures during the same flight, whether detected or latent, and combinations thereof, should also be assumed, unless their joint probability with the first failure is shown to be extremely improbable.

• AC 25.1309-1A defines the likelihood and severity terms found in the Hazard Risk Index

– Provides guidance as to what factors can be taken credit for in probability assessments and how this should be done

– Refers to RTCA DO-178 for software safety assurance guidance– More recent SAE standards (ARP 4754 and 4761) provide much more

detailed guidance on FAA safety-assurance methods


12

Summary of CAT III Airworthiness Requirements (Table from Tim Murphy of Boeing)

Condition Airworthiness Requirements Model

Related Success Criteria

1 AC 120-28DNominal Performance – App. 3, Section 6.3.1

Demonstrate equivalent or better performance under nominal conditions. (All variables varying across entire range). Meet 10-6 box

2 AC 120-28DPerformance with Malfunction – App. 3, Section 6.4.1

For all failures with probability > 10-9 demonstrate safe landing -> Land in box (with probability 1) – given environment and other variables ‘nominal’.

3 JAR AWO Subpart 1 – Performance DemonstrationLimit case conditions

Demonstrate performance when one of the variables is at its most critical value while the others vary in their expected manner – Land in defined box with 10-5 -> Conditional probability approach

Tim Murphy’s presentation is inside RTCA SC-159 WG-4 Archive File:

http://sc159.tc.faa.gov/wg4/060706/Jun072006.htm


13

CAT III Touchdown Zone (or “Box”)

Figure from Figure 3 of Tim Murphy’s requirements report to FAA: Boeing

Doc. # D6-83447-4, 10/19/05

Numbers taken from App. 3, Section 6 of FAA AC

120-28D

Additional “bank angle hazard”

requirement limits probability

of any part of wing or engine

touching ground to 10-7 or less


14

Translation of Touchdown Zone into Landing System Requirements

• Provided in ICAO Annex 10 for ILS (April 1985) – not available online

– Annex 10 was amended for MLS and is being amended for GBAS Amendment 79 is latest (?)

• Annex 10 specifies 95% accuracy limits and monitor limits in terms of ILS measurements (DDM)– Translation to LAAS required knowledge or assumption of

several non-obvious intermediate parameters

• In my understanding, ILS requirements in Annex 10 were designed around already-fielded ILS systems that were already deemed to be safe– CAT III guidance requirements were not much more strict main

difference was tighter, higher-reliability monitoring needed


3: Example Risk Allocations

Source: R.J. Kelly, J.M. Davis, “Required Navigation Performance (RNP) for Precision Approach and Landing with GNSS Application,” Navigation, Vol. 41, No. 1, Spring 1994, pp. 1 – 30.

http://www.ion.org/search/view_abstract.cfm?jp=j&idno=106


16

Breakdown of Worldwide Accident Causes: 1959 1990 (from ICAO Oct. 1990 Study)

• Total hull loss probability per flight (“mission”) as of 1990 = 1.87 × 10-6

• Current probability per commercial departure in U.S. = 2.2 × 10-7 (3-year rolling average last updated in March 2006)

− http://faa.gov/about/plans_reports/Performance/performancetargets/details/2041183F53565DDF.html


17

U.S. Accident Breakdown by Cause (2000-01)

20012000

From NSTB Annual Review of Aircraft Accident Data, 2000 and 2001; ARC 04/01; 06/01

http://www.ntsb.gov/publictn/A_Stat.htm


18

Semi-unofficial “Serious Accident” Risk Allocation (proposed in 1983 SAE paper†)

†D.L. Gilles, “The Effect of Regulation 25.1309 on Aircraft Design and Maintenance,” SAE Paper No. 831406, 1983.

Total Serious Accident RiskNumbers based on approximations of

observed accident history.

10-6 per flight hour

All Other Causes (human error, weather, etc.)

9 × 10-7 p. f. hr.90%10%

Aircraft System Failures (engines, control, avionics, etc.)

1 × 10-7 p. f. hr.

Assume 100 sepa-rate aircraft systems

Each individual system is allocated 1 × 10-9 p. f. hr. (or per flight).

Not subject to certification; thus

not broken down in detail here.

How should the “real” CAT II/III requirements (and other aviation safety requirements) be

determined (work in progress )?


20

Weaknesses in Current Safety Approach

1. No clear means to adapt safety requirements to continued improvement in overall aircraft safety• 10-9 requirement per individual aircraft system appears to be out-of-date

given that current overall serious accident risk is approaching 10-7 per flight

• 10-6 probability for landing in CAT III touchdown zone seems dated

2. No clear means to appropriately balance rare-event probabilities• 10-9 qualifies as “extremely improbable”, but 5 × 10-9 only qualifies as

“improbable” and must be treated as “latent” with probability 1 according to strict reading of AC 25.1309-1A

3. No means to “trade off” safety benefit vs. safety risk for new systems that, when working properly, reduce the risk of accidents caused by pilot/weather/ATC/etc. • Most new systems, including SBAS and GBAS, likely retire more

pilot/weather/ATC risk than they introduce due to the possibility of their own failure


21

FAA Safety Engineering Tries to Adapt

• FAA shows no interest in fundamentally changing current certification standards

• Instead, FAA reacts to accidents on a case-by-case basis and tries to change individual rules interpretations subtly and quietly

– New interpretations also apply to new systems, such as SBAS and GBAS

• Example 1: aircraft rolling out long and off runway (recent SWA 737 accident at Midway)

– FAA now promulgating requirements “clarification” mandating a specific 15% runway margin; see: http://aviationnow.com/avnow/news/channel_busav_story.jsp?id=news/FAA06196.xml


22

FAA Safety Engineering Tries to Adapt (2)

• Example 2: TWA 800 (July 1996) 747 explosion most likely caused by ignition of center fuel tank– NTSB accident report (August 2000):

http://www.ntsb.gov/publictn/2000/AAR0003.pdf

• Many small fuel-tank risk- reduction steps implemented under SFAR 88 beginning in 2001

• Major ignition-suppression retrofit proposed in Notice of Proposed Rule Making (NPRM; Nov. 2005)– http://dmses.dot.gov/docimages/pdf94/373450_web.pdf

• Lengthy technical and cost-benefit debate on this NPRM continues to this day; see:– http://dmses.dot.gov/docimages/pdf94/373645_web.pdf

– http://dmses.dot.gov/docimages/pdf95/389033_web.pdf


23

FAA Safety Engineering Tries to Adapt (3)(Continuation of Example 2: TWA 800 Accident)

• Previous certification of fuel tank safety relied on need for multiple triggering events to occur joint probability was below 10-9 per flight

• However, initiating event could lie undiscovered for many flights prior to being detected by periodic maintenance– New FAA “specific risk” concept requires that “knowable” latent

defects be treated as present with probability 1

– Thus, 10-9 mitigation argument no longer holds in this case

– Also, undetected latent failure could leave aircraft only one failure away from “catastrophic” incident

• FAA and manufacturers have been debating this application of “specific risk” since 2002; see:– https://www.faa.gov/regulations_policies/rulemaking/

committees/arac/minutes/media/TAE_OCT_05.pdf

– http://edocket.access.gpo.gov/2006/pdf/E6-4024.pdf


24

Summary

• A complex set of requirements and guidance documents links today’s CAT II/III landing requirements to overall FAA safety objectives

• As CAT II/III requirements are refined to be more “GBAS-specific,” re-thinking of the intent of the antecedents of these requirements is important

• FAA safety requirements evolution is limited in scope and is limited to “new” systems like SBAS and GBAS and response to external events, e.g., accidents

• Further changes to better reflect improved overall aircraft safety and safety contribution of newer systems would be desirable


25

Backup Slides Follow…


26

Integrity Requirement Definitions

• Integrity relates to the trust that can be placed in the information provided by the navigation system

• Misleading Information (MI) occurs when the true navigation error exceeds the appropriate alert limit (an unsafe condition) without annunciation

• Time-to-alert is the time from when an unsafe condition occurs to when the alarm message reaches the pilot (guidance system)

• A Loss of Integrity (LOI) event occurs when an unsafe condition occurs without annunciation for a time longer than the time-to-alert limit, given that the system predicts it is available


27

Notes to GSL Requirements Table

Section 2.3.1 of DO-245A

1. The values given for GNSS accuracy and alert limits are those required for the intended operation at the lowest height above threshold (HAT) where the GNSS guidance is relied upon.

2. The definition of the integrity requirement includes an alert limit and a time to alert, against which the requirement can be assessed.

3. The accuracy requirements include the nominal performance of a fault-free airborne subsystem.

4. The integrity requirements are specified in terms of a probability to be evaluated over a specified period. The duration of this period is intended to correspond to the most critical portion of an approach & landing for the operations the GSL is intended to support. Integrity risk includes the probability of latent failures, and the exposure time to these types of failures may exceed the specified period, therefore the requirement must apply during “any” period. Note that if the integrity requirements for GSL D-F are met, the integrity requirements for GSL A-C are also automatically met.

5. For these GSLs (D, E, and F), the combined lateral and vertical risk shall not exceed 1 × 10-9, where the risk for vertical applies over any 15 sec, and the risk for lateral applies over any 30 sec. The lateral period is longer because these GSLs are intended to support operations that require LAAS guidance during roll-out.

6. The time-to-alert (TTA) is the maximum time between the onset of a failure condition that affects the integrity of any information that could be applied by the airborne subsystem and the time that the alert indication is available at the output of the airborne subsystem, where the airborne subsystem is assumed to have zero latency. Compliance with the TTA requirement must include consideration of the probability of missed VDB messages by a fault-free airborne subsystem.


28

Actual “Hull Loss” Probability Breakdown (from October 1990 ICAO Study Data)

• Total final approach and landing risk (as of 1990) = 7.8 × 10-7 per flight (~ 42% of total risk!)

• Target level of safety (via “tunnel concept”) for final approach and landing = 0.2 × 10-7 per flight (~ 13% of total risk)

• Hazard due to loss of navigation system integrity is only a small part of the total “final approach and landing” risk

Documents

A Review of CAT II/III LAAS Integrity Requirements and their Antecedents Stanford GPS Laboratory Group Meeting 4 August 2006 Sam Pullen Stanford University