Dynamic Data-Driven Avionics Systems: Inferring Failure Modes from Data Streams

Shigeru Imai, Alessandro Galli, and Carlos A. Varela Worldwide Computing Laboratory Rensselaer Polytechnic Institute

DDDAS Workshop at ICCS Reykjavik, Iceland, June 2015

1

Outline

2

Dynamic Data-Driven Avionics Systems PILOTS

(ProgrammIng Language for SpatiO-Temporal Streaming applications)

Tuninter 1153 Flight Accident Anomaly Detection Model

Evaluation Conclusion/Future work

PILOTS System

3

PILOTS* embodies Dynamic Data-Driven Avionics Systems

Aircraft sensors

PILOTS System

Avionics Application

Measured error

Corrected inputs

Corrected outputs

Identified failure Error Detection & Data Correction

(Mathematical function patterns used to identify failure modes)

Airplane Pilot/Cont

roller (User)

DDDAS Steering

Loop

*: ProgrammIng Language for spatiO-Temporal data Streaming applications

PILOTS Programming Language

4

Highly declarative

First class support for spatio-temporal input data selection Homogenize (often sparse)

existing data to application’s queries

closest, euclidean, interpolate methods

Error detection and correction Define error conditions as

error signatures

program Twice; inputs a(t) using closest(t); b(t) using closest(t); outputs o: b - 2*a at every 1 sec; errors e: b - 2*a; signatures S0 :e = 0 “Normal”; S1(K):e = 2*t+K “A failure”; S2(K):e =-2*t+K “B failure”; S3(K):e = K, abs(K) > 20 “Out-of-sync”; correct S1: a = b / 2; S2: b = a * 2; end

Error Detection Algorithm in PILOTS

5

… No error : S0

Error signatures

S1 SM

δ = <20, 3, … 5>

(2) Convert the distances to a mode likelihood vector

(3) Choose the best matching signature

(1) Compute L1 distance between e(t) and each signature

Measured error

t

e(t)

ω: window size

-1: unknown error

L = <3/20, 3/3, … 3/5> = < 0.15, 1.00, … 0.60>

Each element:

s If two or more satisfy significance threshold τ

Air France AF447

7/8/2015 6

Outline

7

Dynamic Data-Driven Avionics Systems PILOTS

(ProgrammIng Language for SpatiO-Temporal Streaming applications)

Tuninter 1153 Flight Accident Anomaly Detection Model

Evaluation Conclusion/Future work

Tuninter 1153 Flight Accident

8

Flight from Bari, Italy to Djerba, Tunisia on August 6th, 2005 ATR-72 ditched into the Mediterranean sea 16 of 39 people on board died

http://www.airdisaster.com/photos/ts-lbb/5.shtml “Final Accident Report for TS-LBB” http://www.ansv.it/cgi-bin/eng/FINAL%20REPORT%20ATR%2072.pdf

Bari, Italy

Palermo, Italy

x

Djerba, Tunisia

Actual route

Planned route

“Mayday” TV Series on Tuninter 1153 https://youtu.be/aCrZwctnNWo?t=1904

Initial Cause of the Accident

9

Incorrect fuel quantity indicator (FQI) installment FQI for ATR-72 was not working properly (LED failure) Technicians replaced the FQI with one designed for ATR-42

FQI showed 2,700 kg of fuel, but fuel actually weighed 550 kg Pilots did not realize data error eventually leading to fuel exhaustion

“Final Accident Report for TS-LBB” http://www.ansv.it/cgi-bin/eng/FINAL%20REPORT%20ATR%2072.pdf

Checking the Validity of Weight

10

Can we tell if the weight data is invalid from other independently-measured data streams?

Relationship between weight and airspeed The lighter, the faster. The heavier, the slower. Is it that simple? -- No

(Actual flight data recovered from flight data recorder (FDR) of the ATR-72 aircraft)

Model Creation Approaches

11

How to relate data streams mathematically? During cruise

phase, lift equals weight.

1. Use a model from aerodynamics theory Lift:

ρ (air density) is not available from flight data recorder (FDR)

2. Learn a model from the ATR-72 flight crew operating manual

All the required parameters are available from FDR Applicable to other ATR-72 flights

Overview of Our Approach

12

ATR-72 Crew

Operating Manual

Tuninter 1153

1st Cruise Phase Data

Tuninter 1153

2nd Cruise Phase Data

Evaluation Results

Model Creation Model Tuning Model Evaluation

Generally applicable to ATR-72 aircrafts

Tuned to the Tuninter 1153 flight

ATR-72 Flight Crew Operating Manual

13

Performance tables for take-off, climb, cruise, descent, and landing Focus on cruise this time

Weight x1000 [kg]

Altitude x100 [feet])

Temp. diff. to ISA [℃]

Airspeed [knot]

Model Creation

14

Approximate airspeed by 2nd-order polynomial regression Weight (w), altitude (h), delta ISA (tΔ), airspeed (va)

Correlation = 0.993

Model Tuning

15

Obtained equation not accurate enough Calibrate the equation with the 1st cruise phase

using true weight Evaluate with the 2nd cruise phase

Underweight Condition

16

Underweight condition Estimated airspeed ( ) is slower than the actual airspeed ( ) Assuming both va & h are correct, the monitored weight (w)

must be too heavy (thus, aircraft is underweight)

Assume no error:

Solve for w

True Weight Estimation

17

program WeightCheck; /* v_a : airspeed , w: weight , h: altitude */ inputs v_a , w, h(t) using closest (t); outputs corrected_w : w at every 10 sec; errors e: v_a - (6.4869E+01 + 1.4316E-02 * w + 6.6730E-03 * h + (-3.7716E-07) * w * h + (-2.4208E-07) * w * w + (-1.1730E-07) * h * h) + 2.59; signatures S0(K): e = K, -2 < K, K < 2 "Normal"; S1(K): e = K, 4.69 < K "Underweight"; correct S1: w = 3.34523E-12 * (sqrt(1.09278E+22 * h * h + (-1.65342E+27) * h + (-3.69137E+29) * v_a + 1.01119E+32) – 2.32868E+11 * h + 8.83906E+15); end

PILOTS Program

18

4.69 corresponds to 10% discrepancy in weight

Outline

19

Dynamic Data-Driven Avionics Systems PILOTS

(ProgrammIng Language for SpatiO-Temporal Streaming applications)

Tuninter 1153 Flight Accident Anomaly Detection Model

Evaluation Conclusion/Future work

Evaluation

20

Extract actual flight data from the accident report

Run WeightCheck PILOTS program for 2000~3500 sec of the flight data Including two cruise phases

Evaluate how accurately it detects/corrects errors in the weight

Evaluation period

Results

21

Underweight condition is detected 100% for the 2nd cruise phase

Big discrepancy (≈2000kg) between corrected and real weights during the 2nd cruise phase

Outline

22

Dynamic Data-Driven Avionics Systems PILOTS

(ProgrammIng Language for SpatiO-Temporal Streaming applications)

Tuninter 1153 Flight Accident Anomaly Detection Model

Evaluation Conclusions/Future work

Conclusions

23

Weight error detection/correction model for the Tuninter 1153 accident is proposed and evaluated using the PILOTS system

Proposed model works well for detecting anomaly conditions in weight (100% accuracy) during cruise phases, but corrected weight is not accurate enough

Complex Dependencies Between Data Streams

24

va

vg

vw

w fq

h

pw cf

T

(angle of attack, flaps, landing gear, pitch, roll, yaw)

Air France 447 Model

Tuninter 1153 Model

vg : ground speed vw : wind speed va : airspeed

fq : fuel quantity w : aircraft weight h : altitude T : temperature pw : engine power cf : aircraft configuration

(Short-term) Future Work

25

Model improvement for the Tuninter accident Revisit aerodynamics theory

Known constants : From data : From linear regression :

Assuming cruise flight

http://upload.wikimedia.org/wikipedia/commons/thumb/d/d1/Lift_curve.svg/300px-Lift_curve.svg.png

Coef

ficie

nt o

f Lift

(CL)

(Mid-term) Future Work

26

Expand PILOTS language into a DDDAS Model Learning Toolkit to include: Montecarlo simulation to learn model parameters from data. Kalman filters to reduce the impact of noise in data and enable

more robust models. Probabilistic (Bayesian) approach to continuously tune model

to data.

(Long-term) Future Work

PILOTS*1 System Left engine is damaged …

Avionics Application

Measured error

3D terrain data

Updated weather

Stochastic & Logic-based

Flight Scheduler (to be developed)

Aircraft sensors Corrected inputs

Corrected outputs

Identified failure

Failure & Recommended actions

We should land at airport X

immediately!

Airplane pilots

Safer Flight Assistant System External data inputs

*1: ProgrammIng Language for spatiO-Temporal data Streaming applications

Error Detection & Data Correction

(Mathematical function patterns used to identify failure modes)

Cloud-based Offline Data Analytics

(to be developed)

Internet

Information from other planes

Fundamental Developments for Safer Flight Assistant System

28

Quantitative spatial and temporal logic as a formalism: To enable reasoning about data streams that

associate values to specific points or intervals of space and time.

To enable geometric reasoning capabilities, in particular, trigonometric formulae to calculate with aircraft speeds, headings, range, and endurance.

Extensions to logic programming to support stochastic reasoning Language extensions to standard Horn clause-based

knowledge bases to incorporate probabilities. Special language support for spatial and temporal

data streams. Incremental reasoning algorithms to dynamically re-

compute logical queries efficiently as new data gets injected into the application.

Dynamic Data-Driven Flight Plan Adaptation Examples

If… then…

New pilot report: icing en route

New route

New winds aloft New altitude

New surface winds at destination

New airport

Imminent engine failure

Nearest airport

Ground speed and crosswind as functions of airspeed, wind, and runway heading

a Aircraft

w,x Wind, crosswind

r Runway

v Speed (horizontal)

α Direction

Scalable correlation analysis from hundreds of independently-measured sensor data streams Automating anomaly detection/correction model creation process

Cloud-based Offline Data Analytics

29

Aircraft sensor data streams

…

…

…

d1

d2

… Virtual Machines

d1 d2

d3 dN

dN

…

?

? ? ? ? Cloud Storage

d1 d2

d3 dN …

0.9

0.8 0.6

d1 ≈ c∙d2

Cost-Efficient High-Performance

Data Analytics

Open Source Software

Download PILOTS 0.2.4 at: http://wcl.cs.rpi.edu/pilots

Thanks! Questions?

Partial support from:

Air Force Office of Scientific Research DDDAS Program

Dr. Frederica Darema (AFOSR Grant No. FA9550-15-1-0214)

& Yamada Corporation Fellowship

MIT Press, June 2013

30

31

Extra Slides

Weight Airspeed relationship for Multiple Flight Levels

32

Analysis of Discrepancy

33

Proposed weight model is sensitive to airspeed change

+2.46knots

-1402kg

(16070.8, 181.94)

(14668.8, 184.4)

3280 sec 3140 sec

Analysis of discrepancy (cont.)

34

Actual weight: insensitive to airspeed change Corrected weight: sensitive to airspeed change