Upload
inara
View
31
Download
8
Tags:
Embed Size (px)
DESCRIPTION
GA JSC SAT and Working Group Processes Corey Stephens Co-Chair, GA JSC SAT GA JSC SAT Meeting March 22, 2011 Washington, DC. GA Joint Steering Committee. Evolve GA JSC to a CAST like Model Voluntary commitments Consensus decision-making Data driven risk management Implementation-focused - PowerPoint PPT Presentation
Citation preview
GA JSC SAT and Working GA JSC SAT and Working Group ProcessesGroup Processes
Corey StephensCo-Chair, GA JSC SAT
GA JSC SAT MeetingMarch 22, 2011
Washington, DC
GA Joint Steering Committee
• Evolve GA JSC to a CAST like Model– Voluntary commitments– Consensus decision-making– Data driven risk management– Implementation-focused
• The GA JSC is a means to… Focus Limited Government/Industry
Resources on Data Driven Risks and Solutions
What is CAST?
• Work began in 1997 after two significant accidents in 1996 (TWA 800 & ValueJet 592)
• CAST focus was set by:– White House Commission on Aviation Safety– The National Civil Aviation Review Commission
(NCARC)
• Opportunity for industry and government to focus resources on one primary aviation safety initiative
What is CAST?
Vision• Key aviation stakeholders acting cooperatively to lead the
world-wide aviation community to the highest levels of global commercial aviation safety by focusing on the right things.
Mission• Enable a continuous improvement framework built on
monitoring the effectiveness of implemented actions and modifying actions to achieve the goal.
Goal • Reduce the US commercial aviation fatal accident rate 80% by
2007 and • Maintain a continuous reduction in fatality risk in US and
International commercial aviation beyond 2007.
CAST Safety Strategy
Influence Safety Enhancements -
Worldwide
DataAnalysis
Set SafetyPriorities
Achieve consensus on
priorities Integrate into existing work and distribute
Implement Safety Enhancements -
U.S.
Agree onproblems and interventions
How CAST Works
CAST Safety StrategyOngoingAccident/
Incident/Studies
IncidentAnalysisProcess Emerging/
Changing Risk
Develop/ReviseEnhancements
& Metrics
10-28-05 CAST-064
CAST Plan
PerformanceTo PlanReview
Things to Watch
Industry/Government
Action
Safer System
Information on System Performance
Future Changes Analysis Process
Develop/ReviseEnhancements
& Metrics
Develop/ReviseEnhancements
& Metrics
Master Contributing
Factors
\
Approve TrainingProposal
5 8.25d
1/2/96 1/12/96
Select Accident/IncidentSets
1 15d
10/30/95 11/17/95
Review Accident/IncidentReports
2 5d
10/30/95 11/3/95
Analyze Accident/IncidentReports
3 5d
11/6/95 11/10/95
Develop PrioritizedIntervention Strategies
7 2h
1/10/96 1/10/96
Prepare Draft Report
6 3d
1/2/96 1/4/96
Revise Report
4 5d
11/13/95 11/17/95
Prepare FinalReport
8 1h
1/11/96 1/11/96
Approve FinalReport
9 1h
1/11/96 1/11/96
Select InterventionStrategies
10 1d
1/11/96 1/12/96
NTSB AccidentIncident Reports
21.3 Reports
Airclaims data
Implementation Strategy JSIT
Turbofans Installed on part 25 Aircraft
0
5
10
15
20
25
Un
cont
ain
ed
-d
isk
/spa
cers
Mul
ti -
com
mon
- en
v.
Ma
lfunc
tio
n +
cre
wer
ror
Un
cont
ain
ed
- bl
ade
s
Fire
(e.
g.,
un
der
co
wl)
Rev
erse
r
Mu
lti -
com
mo
n -
oth
er
Fue
lco
nta
m./
exh
aus
tion
Mu
lti -
unr
ela
ted
Cas
e ru
ptur
e
Cre
w e
rror
En
gin
e se
para
tion
Un
know
n
Mu
lti-r
ela
ted
Cow
l se
par
atio
n
CA
AM
Lev
el 3
an
d 4
Ev
ents
Level 4
Level 3
HistoricalData
Pareto PlotsJSAT
ASIAS data
Causal Analysis
CombinedThreat
Threat
Cause Cause Cause Cause
Accident
1.
2.
3.4.
5.
CAST Safety Analysis Process
Industry
JSAT
JSAT JSAT
Intervention Strategy
6.7.
Coordinated Plan
Measuring Progress to Goal
Industry
Government
SaferSkies
AvSP
5.3-23
• Safety enhancement development
• Master safety plan • Enhancement
effectiveness• Future areas of
study
• Data analyses
CAST
Joint Safety Analysis Teams (JSAT)
Joint Safety Implementation
Teams (JSIT)
Joint Implementation Measurement Data
Analysis Team (JIMDAT)
Commercial Aviation Safety Team (CAST)Commercial Aviation Safety Team (CAST)Commercial Aviation Safety Team (CAST)Commercial Aviation Safety Team (CAST)
CAST Accomplishments
• Forensic analysis of US and world accidents since 1987 (ongoing)
• Industry and government cooperative safety plan:– 72 Prioritized Safety Enhancements– 50 Complete and 22 underway– Projected 74% fatality risk reduction by 2020
• Development of proactive analytic processes for incident data
• CAST was the recipient of the 2008 Collier Trophy For achieving an unprecedented safety level in U.S.
commercial airline operations by reducing risk of a fatal airline accident by 83 percent, resulting in two consecutive years of no commercial scheduled airline fatalities
For this discussion…
• GA JSC = CAST
• Steering Committee = CAST ExCom
• Safety Analysis Team (SAT) = JIMDAT
• Working Groups (WGs) = JSAT/JSIT
GA JSC Groups and their CAST Counterparts
• Identify future areas of study/risk• Charter safety studies• Provide guidance and direction• Draw data from various areas• Develop a prioritized Safety Plan • Develop metrics to measure
effectiveness of safety solutions
• Data analyses• Safety enhancement• Mitigation development
• Strategic guidance • Management/Approval of Safety Plan• Provide direction• Membership Outreach• Provides linkage to ASIAS
Steering CommitteeCo-Chairs: Bruce Landsberg (AOPA/ASF) Tony Fazio (FAA/AVP)Government - FAA (AFS, AIR, ATO & ARP) - NASA (Research)
- NWSIndustry - GAMA, EAA, NBAA, NATA, & SAMA
Safety Analysis TeamCo-chairs: Corey Stephens (FAA) Jens Hennig (GAMA)Members: FAA, NTSB, AOPA, FSF, UAA, CGAR, FAST, NAFI, Insurance, Academia, SAFE
Working Groups(To include SMEs from various general
aviation segments, depending on study)
General Aviation Joint Steering Committee (GAJSC)
GA Safety Plan
GAJSC Safety Strategy
08-16-2011 GAJSC
GAJSC
SAT
WG
Time
OngoingFatal Accident
Studies
Accident Area
Proposed
Approves Priority / Assigns
Resources to WG
Detailed Accident Review and Propose
Mitigations
Approves Proposed
Mitigations
Amend Safety Plan
Develops Detailed Implementation
Plans (DIP)
Approves DIP & Assigns Industry
Government Responsibility
Industry Action
Government Action
Develop, Revise & Monitoring of Metrics
Monitor Effect
Identification of System Changes
General Aviation NAS Safety State
AviationSystem
EstablishSAT
FAA
NTSB
PilotsManufacturers
Academia
ASIASAccident Selection NASA
Evaluate Cost & Benefit
Review Proposed
Mitigations
GA JSC Working Group
Process
Step 1: Analysis
Typical CAST JSAT Membership
• ALPA/APA
• FAA (AIR, AFS, ASA, AAI, ATO)
• Airbus
• EASA
• ATA
• Transport Canada
• NASA
• Engine companies – (PW, GE, RR-Allison)
• Boeing
• RAA
• NACA
• AIA
• NATCA
5.5-24
GA JSC WG Process
CharterDevelopment
EstablishTeam
SelectData Set
ReviewData
IdentifyInterventionStrategies
AssignStandardProblem
Statements
Record Characteristics/
Indicators
DevelopEvent
Sequence
EvaluateIntervention
Effectiveness
PrioritizeInterventions
Technical Review &
ReportResults
IdentifyProblems
(what/why)
GlobalReview of
Characteristics/Indicators
EvaluateProblem Importance
Developed Event Sequence
• Facts and data• Pilot - controller voice events• Missed calls• Events that occurred or should have
• Time coded each event
# Time Event1015 21:53:28 ATC issued ATIS information Sierra: Ceiling
100’ overcast, 1/2 mile visibility and fog1016 21:53:28 F/O call 200’ above minimums1017 21:53:32 F/O calls ATC to report Marker Inbound1018 21:53:33 F/O call out 100’ above minimums1019 F/O fails to call out “runway not in sight” at the
minimums for the Decision Height
Develop Problem Statements
• Problem statements– What went wrong– Deficiency definition– Potential reason– Something which happened or didn’t happen
# Time Event/Data PointProblem (What)
Contributing Factors (Why)
1
8:53:00 Aircraft took off from Taipei Intl Airport
210:45:00 F/O briefed CAPT
on approach into
3
10:49:00 Capt gave very basic guidance to the F/O on aircraft control during approach and landing.
F/O was inexperienced; his actions were not commensurate with 1034 hours in type.
It is not normal practice at China Airlines for Capt and F/O to rotate takeoffs and landings. The FO is required to fly aircraft "in t/o and landing phases at least 3 times every 3 months" (3-28) (airline culture)
Sample Standard Problem StatementsCAST Examples
• 10 FLIGHTCREW – Failure of flight crew to follow established procedures (SOP)
• 39 AIRCRAFT EQUIPMENT – DESIGN NOT ERROR TOLERANT System design does not provide adequate redundancy to counteract errors or alerting of the effects of errors
• 44 FLIGHTCREW – Flight crew failure to recognize and correct unstable approach
• 100 REGULATORS – INSUFFICIENT AIR CARRIER OVERSIGHT . Insufficient regulatory oversight of air carrier operations including management and training practices
Identify Intervention Strategies
• Intervention strategies– Suggested solutions– Things to do to prevent or mitigate the problem– Etc.# Time Event/Data Point
Problem (What)
Contributing Factors (Why)
Standard Problem Statement
P1 A
1
8:53:00 Aircraft took off from Taipei Intl Airport
210:45:00 F/O briefed CAPT
on approach into
3
10:49:00 Capt gave very basic guidance to the F/O on aircraft control during approach and landing.
F/O was inexperienced; his actions were not commensurate with 1034 hours in type.
It is not normal practice at China Airlines for Capt and F/O to rotate takeoffs and landings. The FO is required to fly aircraft "in t/o and landing phases at least 3 times every 3 months" (3-28) (airline culture)
20 AIRLINE OPERATIONS - LACK OF TRAINING (FLIGHTCREW)
3 5
414 Airline operations – training failed to adequately develop FIRST OFFICER piloting skills. (SPS-20)
4 3
Intervention Effectiveness
• Power– Effectiveness of a specific intervention in reducing the likelihood that
a specific accident would have occurred (“Perfect World”)
• Confidence– Confidence that this specific intervention will have the desired effect
• Future Global Applicability– How well the intervention can be extrapolated to apply to a world-wide
fleet in the future
Effectiveness Rating Scales
POWER
This scale is to be used to judge the effectiveness of a specific intervention in reducing the likelihood that a specific accident would have occurred had the intervention been in place and operating as intended. (“perfect world”)
Hardly any effect
Slightly effective
Moderately effective
Quite effective
Highly effective
CONFIDENCE
This scale is to be used to define the level of confidence that you have that this specific intervention will have the desired effect.
Hardly any confidence
Slightly confident
Moderately confident
Quite confident
Highly confident
0 1 2 3 4 5 6
Not at all effective
Completely effective
0 1 2 3 4 5 6
Not at all confident
Completely confident
FUTURE GLOBAL APPLICABILITY
This scale is to be used to estimate how well the intervention can be extrapolated to apply to a world-wide fleet in the future.(for example: how often the situation it addresses occurs in accident scenarios; whether its impact is on present and future operations (equippage, traffic, regulatory differences); and whether it is applicable across airlines/airplanes/regions.
Hardly any applicable
Slightly applicable
Moderately applicable
Quite applicable
Highly applicable
0 1 2 3 4 5 6
Not at all applicable
Completely applicable
GA JSC Working Group
Process
Step 2: Implementation
GA JSC Feasibility Scales
• Technical • Financial • Operational • Schedule • Regulatory • Sociological
GA JSC Safety Enhancements
• Develop Safety Enhancements from Interventions
• Collect detailed resource information
• Prepare Detailed Implementation Plans (DIP’s)
GA JSC WG Reports
• Standard Problem Statements
• Interventions Prioritized
• Recommendations
• Detailed Implementation Plans (DIPs)
What’s a DIP?
SE 31
Loss of Control Joint Safety Implementation Team
Implementation Plan
for Training - Advanced Maneuvers
Statement of Work
Advanced Maneuvers Training (AMT) refers to training to prevent and recover from hazardous flight conditions outside of the normal flight envelope, such as, inflight upsets, stalls, ground proximity and wind shear escape maneuvers, and inappropriate energy state management conditions.
The purpose of this project is to collect and provide advanced maneuver training material and to encourage Part 121 operators to use these materials to implement advanced maneuver ground training and flight training using appropriate flight training equipment. Emphasis should be given to stall onset recognition and recovery, unusual attitudes, upset recoveries, effects of icing, energy awareness and management, and causal factors that can lead to loss of control. Additionally, research should be conducted to determine how existing flight simulation devices can be used effectively in AMT. Safety Enhancement: (SE-31) Pilots will be better trained to avoid and recover from excursions from normal flight and loss of control. Lead Organization for Overall Project Coordination (LOOPC): FAA, Flight Standards (AFS)
Score: 2007-(13.0) 2020-(13.0) 100%-(13.0) Resource Requirements: FAA AFS-400, Air Transport Association Training Committee, National Air Carrier Association (NACA), Regional Airline Association, manufacturers, pilot associations, Principal Operations Inspectors (POI’s), Directors of Safety, flight operations and training departments, NASA, aircraft manufacturers, flight simulation device manufacturers, training centers, existing training aids, and other materials.
GA JSC SAT (Safety Analysis Team)
ProcessSafety Plan Development
Develops a Prioritization Methodology (GA JSC SAT)
• Identifies the most effective solutions derived from all accident categories
• Considers effectiveness vs. resources
• Tests solutions against fatal and hull loss accidents
• Creates draft master strategic safety plan
• Identifies areas for future study/mitigation
Effectiveness
that an intervention has for reducing the accident rate if incorporated
Portion of world fleet with intervention implemented
( ),Accident Risk Reduction =
General Methodology for Calculating the Potential Benefit of a Safety Enhancing Intervention
Accident Date Location Aircraft Accident Description Portion Intervention Name
Class Type of EGPWS CFIT TRN
Description Accident Portion of World Fleet with Intervention Implemented by (2007)
Eliminated.600 .900
Intervention Effectiveness (%/100)
CFIT 1/2/1988 IZMIR, TURKEY 737 HIT MOUNTAIN ON APPROACH .657 .950 .226CFIT 2/8/1988 LUANDA, ANGOLA 707 HIT ANTENNA ON APPROACH .586 .800 .226CFIT 2/27/1988 KYRENIA MTS, CYPRUS727 HIT MOUNTAIN ON APPROACH .657 .950 .226CFIT 3/17/1988 CUCUTA, COLUMBIA 727 HIT MOUNTAIN DURING CLIMB .657 .950 .226CFIT 6/12/1988 POSADAS, ARGENTINA MD80 CRASHED ON FINAL APPROACH .203 .000 .226CFIT 7/21/1988 LAGOS, NIGERIA 707 CRASHED ON APPROACH .203 .000 .226CFIT 10/17/1988 ROME, ITALY 707 LANDED SHORT .203 .000 .226CFIT 10/19/1988 AHMEDABAD, INDIA 737 LANDED SHORT .586 .800 .226CFIT 2/8/1989 SANTA MARIA AZORES 707 TERRAIN IMPACT/DESCENT .657 .950 .226CFIT 2/19/1989 KUALA LUMPUR, MALAYSIA747 TERRAIN IMPACT/APPROACH .657 .950 .226CFIT 6/7/1989 PARAMARIBO, SURINAMEDC8 TERRAIN IMPACT/FINAL APPROACH.203 .000 .226CFIT 7/27/1989 TRIPOLI, LIBYA DC10 TERRAIN IMPACT/FINAL APPROACH.203 .000 .226CFIT 8/25/1989 ANKARA, TURKEY 727 HIT ILS ANT. ON TAKEOFF .000 .000 .000CFIT 10/21/1989 TEGUCIGALPA, HONDURA727 TERRAIN IMPACT/APPROACH .657 .950 .226CFIT 10/26/1989 HUALIEN, TAIWAN 737 TERRAIN IMPACT/DEPARTURE .657 .950 .226CFIT 2/14/1990 BANGALORE, INDIA A320 HIT SHORT (300 FT) .203 .000 .226CFIT 6/2/1990 UNALAKLEET, ALASKA 737 HIT HILL 7 MILES OUT IN FOG .657 .950 .226CFIT 11/14/1990 ZURICH, SWITZERLANDDC9 CRASHED 5 MILES SHORT .634 .900 .226CFIT 12/4/1990 NAIROBI, KENYA 707 HIT POWER LINE ON ILS FINAL .203 .000 .226CFIT 3/5/1991 MT LA AGUADA, VENEZUELADC9 HIT MOUNTAIN/APPROACH .657 .950 .226CFIT 8/16/1991 IMPHAL, INDIA 737 A/C HIT HILL 20 MILES OUT/APPROACH.657 .950 .226CFIT 1/20/1992 STRASBOURG, FRANCE A320 IMPACTED GROUND/FINAL APPROACH.586 .800 .226CFIT 2/15/1992 KANO, NIGERIA DC8 CFIT OUT OF PROCEDURE TRN-DARK.586 .800 .226CFIT 3/24/1992 ATHENS, GREECE 707 ABANDONED APPROACH-HIT MTN .657 .950 .226CFIT 6/22/1992 CRUZEIRO DO SUL, BRAZIL737 HIT SHORT,DRK NT,DISTRACTED .203 .000 .226CFIT 7/31/1992 KATMANDU, NEPAL A310 CFIT-HIT MTN-MISSED APPROACH .657 .950 .226CFIT 9/28/1992 KATMANDU, NEPAL A300 CRASHED SHORT DURING APPROACH.657 .950 .226CFIT 11/25/1992 KANO, NIGERIA 707 LANDED SHORT MISLEADING LIGHTS.538 .700 .226CFIT 11/26/1992 MANAUS, BRAZIL 707 HIT LIGHTS ON TO/RMLG COLLAPSE.000 .000 .000CFIT 4/26/1993 AURANGABAD, INDIA 737 HIT TRUCK AFTER TAKEOFF .000 .000 .000
Spreadsheet Example – Historical Airplane Accidents & Proposed Safety Enhancements – CAST Example
Basics of the Selection Spreadsheet
• EffectivenessEach safety enhancements is evaluated against each undesired condition in the set to
determine how effective the enhancement would be at eliminating these conditions if the enhancement were put in place.
• ImplementationImplementation level is based on the portion of the affected population with the
enhancement incorporated or predicted to be incorporated by a future date.
• Severity WeightingTo account for differences in severity or significance of the undesired conditions, a
weighting value can be entered so that the relative risk of the undesired conditions is realized.
• To account for differences in fatality risk associated with each accident in the data set, a severity value was applied. In this assessment, the severity value represented the portion of people onboard that perished in the given accident.
• Example: Comparison of two fatal accidents
757 CFIT accident, 98% perished. Weighting factor is .98
747 Turbulence accident, .6% perished. Weighting factor is .006
• Hypothetically assume an assessment showed that the chance of these accident occurring would have been reduced by proposed safety enhancements by 50%.
• The associated portion of fatality risk eliminated can be determined using the severity weighting factor as follows:
757 CFIT.98 x .5 = .49
747 Turbulence, .006 x .5 = .003
Severity Weighting OverviewCAST Example
Analysis Tool Output
• The spreadsheet output can be set up to show the The spreadsheet output can be set up to show the effect that an individual safety enhancement, or effect that an individual safety enhancement, or group of safety enhancements have on reducing group of safety enhancements have on reducing exposure to the undesired conditionexposure to the undesired condition..
SE1 SE2 SE3 SE1 &SE2
SE1 &SE3
Fatality RiskReduction
Robust CAST Methodology
• Detailed event sequence - problem identification from worldwide accidents and incidents
• Broad-based teams (45-50 specialists /team)
• Over 450 problem statements (contributing factors)
• Over 900 interventions proposed
• Analyzed for effectiveness and synergy- CAST Safety Enhancements
CAST Process Led to Integrated Strategic Safety Plan
• Part 121 or equivalent passenger and cargo operations studied
• Current CAST plan:
• 72 Prioritized Safety Enhancements
• 50 Complete and 22 underway
• Projected 74% fatality risk reduction by 2020
• Industry and Government implementing plan
Resource Cost Vs. Risk ReductionCAST Example
APPROVED PLAN
Completed + Plan (2007
Implementation Level)
Completed + Plan (2020
Implementation Level)
All JSIT Proposed Enhancements
(2020 Implementation
Level)
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Re
so
urc
e C
os
t ($
Mil
lio
ns
)
Risk Reduction
Total Cost in $ (Millions)
2007 2020
0%
25%
50%
75%
100%
Ris
k E
lim
ina
ted
by
Sa
fety
En
ha
nc
em
en
ts
Completed
$$
$$ $$ $$
$$
Do
llars
/Flt
. Cy
cle
Part 121 Aviation Industry Cost Due to Fatal/Hull Loss Accidents
100
80
60
40
20
0
Historical cost of accidents per flight cycle
74% Risk reductionSavings ~ $74/Flight Cycle
Or
~ $814 Million Dollars/Year
Cost of accident fatalities following implementation of the CAST plan @ 2020 levels
2020
Cost SavingsCAST Example
2007
What the GA JSC can accomplish
• GA accident and incident data drives direction of GA JSC activities
• GA JSC to charge the SAT with chartering study groups on specific topics
• Working groups of SMEs formed to identify risks and develop mitigations
• Mitigations are assessed and prioritized• A cooperative industry/government GA
safety plan is developed and implemented
GA JSC SAT & WGs - Moving Forward
• History shows focused action and introduction of new capabilities have led to accident risk reductions
• Joint industry and government teams working together to a common goal can further enhance the safety of our very safe aviation system
• Full implementation will require a coordinated effort between industry and government
• The GA JSC is moving forward to meet the challenge