Accident Investigation and

Aircraft Hazard Areas in the Post-Columbia World

Paul D. Wilde, Ph.D., P.E.FAA/AST-4

Columbia Accident Investigator

Introduction

• I was an investigator for the Columbia Accident Investigation Board (CAIB).– At the CAIB, I investigated the technical cause and

the public safety issues.– The implications listed are derived from my CAIB

and other experience.

• Some things have changed since the CAIB, but some thing have not.– Aircraft Hazard Area (AHA) implementation has

evolved substantially (Murray AIAA 2010-1349)– Aircraft and space safety and investigation

paradigms remain vastly different.

Overview of CAIB Findings and Implications for Space Safety

• Space launches are risky

• Past success does not provide future success

• Standards and formal structure can help

• Independent technical authorities are valuable

• Be prepared for accidents

• Understand anomalies

• Don’t short cut formal processes

• Safety vigilance is challenging

Finding Implication

Space Vehicles Are Dangerous

• “Building rockets is hard. Part of the problem is that space travel is in its infancy.” CAIB Vol. 1 page 19

• “Building and launching rockets is a very dangerous business and will continue to be so for the foreseeable future” CAIB Vol. 1 page 19

• Rockets fail catastrophically 10-100 thousand times more often than commercial transport aircraft (per flight).

• Accidents should be expected; prepare plans for emergency response

• Prepare investigation and RTF plans, including interface to media and other orgs

• No presumption of safety: accidents usually stop all flights until cause is found

Evidence Implication

Independent Technical Authorities Are Valuable

• A compliance verification organization independent of operational program cited as key to success for Navy subs and nuclear reactors, and in Air Force launch verification.

• “Organizations that deal with high risk operations must always have a healthy fear of failure - operations must be proved safe rather than the other way around.” CAIB Vol. 1 page 190

• Independent compliance verification enhances the safety of complex technical systems

• Checks and balances promote communication (in-flow of new info, addressing minority opinions)

• Safety takes real effort

Evidence Implication

Public Safety

• Columbia break-up during re-entry clearly could have caused public casualties

• Lack of public casualties due to Columbia break-up was the expected outcome given the sparse population– P>1 serious injury was <50% (~10-30%)– Same accident over a major city expected to

produce a few public casualties

• Hypersonic ops late at night lowers risk– Roofs protect effectively from most debris

• Relatively high probability of failure makes “safe” for public difficult to verify

Risk to Aircraft Flying Near Columbia Break-up

• At the time of Columbia break-up, FAA was unaware of any hazard to aircraft.– TFR issued ~ 45 minutes afterward based on

radar detection of debris, media rept., etc.

• Post CAIB analysis by FAA showed aircraft PI ~ 0.001 to 0.01

• Post CAIB simulation illustrates the issue– Actual aircraft flight locations/trajectories – Blue dots are recovered debris locations – Statistical distribution of debris during fall– The view is from the southeast

• Green lines show County boundaries

8

Safety of Aircraft Flying Near Space Launch or Re-entry

• To provide safety and efficiency in US NAS, both pre-defined and real-time AHA are used.– AHA for planned debris (jettisoned stages)

– Break-up generally spreads debris over a large area; aircraft PI often exceeds 1E-6

– During exo-atmospheric flight, several minutes between break-up and debris reaching aircraft altitudes.

– Vulnerability of aircraft to such debris strikes is highly uncertain and under investigation.

BACK-UP

Sub-models for AHA Development

The last two (vulnerability and impact probability), plus the risk criteria for aircraft, have aspects that are necessarily unique to aircraft hazard

area analysis; all other sub-models are common with the debris risk analysis

Aircraft Grid & Trajectory Approaches to PI Estimate

• Grid approach– Assumes aircraft

continuously present in each grid cell

– Produces conservative results

• Specified trajectory

– Accounts for aircraft azimuth and limited dwell time in each cell

– More realistic PI is 2x to 7x lower

Airbus A300: Struck by a missile at 8,000 ft but landed safely 22 Nov 2003

Aircraft Vulnerability Modeling

See Wilde & Draper AIAA paper 2010-1542

Current Efforts Toward Higher Fidelity Aircraft Vulnerability Models (AVMs)

• FAA sponsored higher fidelity analysis using previously developed tools (e.g. military) and input data

• FAA impact testing to improve skin penetration eq., evaluate• Influence of obliquity, fragment density, distance from support,

etc.• Available results show

– Current penetration equation is conservative – 321-10 AVMs are excessively conservative, esp. for

“catastrophe”

V_Relative_Velocity_Fragment

V_Aircraft

V_Terminal_Velocity_Fragment

Elevation Angle

Public Safety Findings

• NASA should – Implement public risk acceptability policy

– Mitigate public risk from STS flight

– Study debris to improve risk estimates

• Collective public risk from space flight is small compared to civil aircraft operations.– Principle reason is huge number of aircraft

operations relative to launches.

• One in a million risk to individuals is a recognized benchmark for both and others

• Complete report at www.caib.us Vol.II D-16

Understand Anomalies

• O-ring blow-by and foam impacts were previously detected as anomalies

• The cause, effect, and limits of these anomalies were not understood

• “Engineers understood what was happening, but they never understood why.” CAIB Vol. 1 page 196

• Anomalies are often early warnings

• Successes do not prove problem solved or not dangerous

• Examine all data on anomalies separately and as a set

• Provide technical rigor in all requirements, rationales, validations

Evidence Implication

Formal Structure Can Help

• Formal documentation traces what was done to verify requirements were satisfied

• Formal structure can ensure that the burden of proof is on those saying it’s safe

• Formal structure identifies the responsible party

Implication Implication

• Formal standards help define what is an anomaly

• More uncertainty, justifies more attention and more caution

• Formal documents and peer reviews promote better decisions and help inform future generations

Informal Processes Are Not Effective

• Several informal attempts to obtain on-orbit imagery failed

• Lack of ground rules hampered engineering teams that evaluated the issues CAIB Vol. 1 page 200

• Management teams violated their own rules

• “When …analyses are condensed to fit on a…overhead slide, information is inevitably lost.” CAIB Vol. 1 page 191

• Clearly defined roles and rules improve effectiveness

• Design structure to promote communication

• Minority opinions should be addressed

• Communication needs to flow both up and down the chain of command

Evidence Implication