Departamento de Infraestructura, Sistemas Aeroespaciales y Aeropuertos DISAA - UPM May 2002 FAA / Eurocontrol AP2, 5 and 6 TIM/9 Are homogeneous performance

Departamento de Infraestructura,Sistemas Aeroespaciales

y Aeropuertos

DISAA - UPM May 2002FAA / Eurocontrol AP2, 5 and 6 TIM/9

Are homogeneous performance Are homogeneous performance metrics for the whole ATM metrics for the whole ATM system life cycle available?system life cycle available?

Dr. Professor Francisco Saez &

Professor Fernando GomezDepartment of Aeronautical Infrastructure, Air and Space Systems and Airports. Polythecnic University of Madrid

John N. Polky, Ph. D. Boeing Research & Technology Center, S.L.


y Aeropuertos


Objective of the workObjective of the work• Illustrate the existent gap between high level performance indicators

used normally for setting concept performance targets (ATM system capacity increase,etc... ) and low level metrics and indicators.

• Identify existing and recognised links.• Identify those areas where additional R&D effort is still required

for quantifying and modelling the various links and contribution between metrics.

ATM community objective:

To have available a measurement framework (an homogenous and coherent set of metrics) for the whole concept life cycle.


y Aeropuertos


Contextthe new concepts definition process

TOOHIGH LEVEL!

TOOLOW LEVEL!

TOO CONSTRAINED BY THE IMPLEMENTATION! Which are the

performances that really characterise the ATM

System?

ATM 2000+

SAFETY, CAPACITY,

ECONOMICS....

PRC

SAFETY, DELAY, ACCESS

FLEXIBILTY.

Controller workloadnumber of

conflicts, etc....

CCDTI suitabilitynumber of resolution

advisories, etc....


y Aeropuertos


Contextthe performance measurement framework?

Which is the relationship between low level

measurements and high levels performances?

The contribution of low level indicators to the high level indicators should be

quantified and unique mathematics relationships

should be established.

Which are the performances that really

characterise the ATM System?

R&D is required for Identify and develop the mayor gaps between High and Low level indicators

That work is fundamental for completing the overall

validation framework


y Aeropuertos


ATM SYSTEMATM SYSTEM

Local IndicatorLocal Indicator

ATMATM

sub systemsub system

GLOBAL indicatorGLOBAL indicator

?

High level performance areas vs. Local performance elements (I)

OutcomesResults for customers

OutputsProducts and services

ProcessesActivities to achieve outputs

InputsResources

GLOBAL GLOBAL INDICATORSINDICATORS

LOCAL LOCAL INDICATORSINDICATORS

•Focus on value of service provided to the user•Extension beyond traditional measures of capacity and delay

Modern approach to the Modern approach to the

ATM system performance measurementATM system performance measurement


y Aeropuertos


Complex and not characterise relationships between:

High level/Globalperformance objectives(areas, performance...)

Specific or Local performance indicators (factors, metrics,...)

High level/Globalperformance objectives are related to the Overall ATM systems performances

Specific or Local performance indicators are related to system components ( either airborne, ground systems, pilots, controllers, procedures,etc...

Contributes or not to achieve the target performance objectives

+ -

High level performance areas vs. Local performance elements (II)


y Aeropuertos


Examples:

High level/Globalperformance objectives(areas, performance...)

Specific or Local performance indicators (factors, metrics,...)

PRC/PRU:PRC/PRU:Performance area KPA DelayPerformance area KPA Delay

Factor that contributes to Factor that contributes to delay?:delay?:Controller workload:Controller workload:

Global Indicator KPIGlobal Indicator KPI•Total minutes of delay/Total number of flights•Capacity index (increase in movements for a given ATFM delay)

Local indicator :Local indicator :Percentage of time spent of the frequency

Number of aircraft handled.

High level performance areas vs. Local performance elements (III)


y Aeropuertos


Performance indicators framework elements

ATS perspective

Safety

...

Capacity

Efficiency

Cost effectiveness

Pilot’s workload

Equipment

Controller’s workload

...

Controller’s situational awareness

Separations standards

Conflicts

Delays

Stress

+-

Other stakeholders perspective

Pilot perspective

Final user perspective

+ -+30%

-5%

PerspectivesPerspectives ATM System performancesATM System performancesGlobal indicatorsGlobal indicators

Factors contributing to Factors contributing to ATM System performancesATM System performances

Local indicatorsLocal indicators


y Aeropuertos


Work to be done

Documentation and databases analysis

Approach definitionMethodology definition

Structure and framework definition

Gaps and R&D needsidentification

Metrics populationof the database

Feedback evaluation Framework rework and upgrade

Framework and supporting databasediffusion

PH

AS

E 1

PH

AS

E 2

PH

AS

E 3

R+D actionsLinks and relationships modelling

Continuous metrics assessment

ContinuosFramework rework and upgrade

PH

AS

E 4

Supporting database creation

Metrics populationof the database

Framework structure validity evaluation

Framework rework and upgrade

ON GOING W

ORK


y Aeropuertos


Performance indicators framework scheme

Low level /Local indicators or subsystem

performances

Stakeholders/participants

perspective or points of view

•ATS•Airlines•Airpots•Final users•----•----

Performances areas

categorisation•------•------•------•------•Delay

Metrics/measurements•------•------•------•------

•Total minutes of delay/Total number of flights

High level /Global ATM

Performance areas or

system performances

CODA

TORCH

PRC

ATM2000+

OCD

CAF/T Human factors

Atomatizacion

CNS

Separation standards

ATFM

Local Factors•------•------•------•------

•workload

Metrics/measurements•------•------•------•------

•Total minutes on the frecuency

Identify the missing contributionsIdentify the missing contributions

Develop a structured framework that allows interrelationships.Develop a structured framework that allows interrelationships.


y Aeropuertos


Developing a structured framework that allows interrelationships. (I)

High/global ATM system performance level

•An unique and structured framework of performances that allows or accommodate the various stakeholders views.•Each ones of the performances area will be characterised by their intrinsic characteristics and by the relationships with the others .•Global indicators are proposed for each one of the performance that are their intrinsic characteristics•Metric/units/measurements are compiled for each intrinsic area or global indicator•Studied sources:OCD, ATM2000+, PRC, CAFT, INTEGRA, TORCH, NAS

S A F E T Y

P R E D IC T A B IL ITY

D E L A Y A C C E S S F L E X IB IL ITY A V A IL A B IL ITY E Q U IT Y

C A P A C IT Y E N V IR O N M E N T

F L IG H T E F F IC E IN C Y C O S T O F T H E S E R V IC E

E F IC IE N C Y

A T M sys te m p e rfo rm a n ces

DELAY:

Stakeholder view relevant to:

UPPER

LOWER

COLATERALL Intrinsic characteristics

Metrics/measurements/units

Global indicators


y Aeropuertos


Developing a structured framework that allows interrelationships. (III)

subsys temperfo rm ances

hum an pe rfo rm ances

subsys tem + hum anperfo rm ances

dem and tra f f ic p ro f i lepe rfo rm ances

subsys tem + hum an + tra f f icpe rfo rm ances

S A F E T Y A V A IL A B IL ITY

C A P A C IT Y E N V IR O N M E N T

E F IC IE N C Y F L IG H T E F F IC E IN C Y

C O S T O F T H E S E R V IC E P R E D IC T A B IL ITY

A C C E S S E Q U IT Y

F L E X IB IL ITY D E L A Y

A T M sys te m p e rfo rm a n ces

ASSESSMENT OR MODELLING

Local Indicators


y Aeropuertos


Developing a structured framework that allows interrelationships. (IV)

Performance aggregation process•Once this low level characterisation is done performance aggregation models should be considered

D E L A Y

SU BSYSTE MELEM EN T X n

D E L A Y

SU BSYSTE MELEM EN T X 1

D E L A Y

SU BSYSTE MELEM EN T X 2

D ELA Y

ATM SYSTE M

ASSESSMENT OR MODELLING


y Aeropuertos


Gaps and R&D needs

identification

Final products

Measurement framework

Stakeholder views

Performancearea

Life cycle state

Agregation models

Subsystem under

analysis

Structured breakdown decomposition of factors s

contribution to each performance

Performanceroadmap

Relational database


y Aeropuertos


Evolution of the approaches towards performance metrics

User oriented view

ATM 2000+

•Safety•Capacity•Efficiency•Environment

Contributes to Characterise Operational improvements

Conceptual Level

No explicit measurements proposed

Do not reflect perspectives of the stakeholders.

PRCKPA

•Safety•Delay•Predictability•Access•Flexibility•Availability•Equity •Flight efficiency•Cost-effectiveness•Environment

Contributes to typify thereceived service

OCD

•Safety•Capacity•Efficiency•Environment•Security•Economic•Uniformity•Quality

CAF/T

•Delay•Predictability•Flexibility•Efficiency•Access•Cost of service

•Productivity•Capacity•Traffic•Safety


y Aeropuertos


ATM 2000+OCD

•Safety

•Capacity

•Efficiency

•Environment

PRC (KPA) / CAFT

Delay

Predictability

Access

Flexibility

Availability

Equity

Safety

Flight efficiencyCost-effectiveness

Environment


y Aeropuertos


CAF/T versus PRC approach

PRCKPA

•Delay

•Predictability

•Flexibility

•Flight efficiency

•Access

•Cost-effectiveness

•Availability

•Equity

•Environment

•Safety

CAF/T

•Delay

•Predictability

•Flexibility

•Efficiency

•Access

•Cost of service

•Productivity

•Capacity

•Traffic

•Safety

It accounts for the time in excess of the optimum time that it takes a user to complete an operation

It accounts for the abitly of the user to predict variation and build and maintain optimum flight schedules It accounts for the ability of

the system to accommodate changing user needs in real time and without penaltyIt accounts for the abitly of the

ATM system to allow the user to adopt the preferred flight profile (route and level) It accounts for the ability to

enter the ATC system and obtain services on demand

The value for the money that the user received from the ATS

Measures of an output over the cost to produce it

Number of operations using a resource in a specified period of time

Number of total operation (flight passengers)

Measures exposure of certain incidents

Measures the availability of critical ATM resources and services

Measures equity of treatment

Measures the conformance of air transport to environment regulations

Measures the conformance of air transport to specified safety targets

Common conceptual definitions and aims but different metrics


y Aeropuertos


FUSION and STRUCTURATION

European Performance Review System:CRCO,STATFORD, CFMU,

CODA, EATCHIP CIP monitoring

OCD

ATM2000+

MAEVACAF/T

PRCAirspace

Management handbook

R+D projects results

FAA performancemeasurement

database

SAFETYCAPACITY ENVIRONMENT EFICIENCY

DELAY

ACCESS

PREDICTABILITY

FLEXIBILITY

AVAILABLITY

EQUITY

FLIGHT EFFICIENCY

COST OF THE SERVICE

PRODUCTIVITY

Thirteen top ATM issues

Framework approach (I)


y Aeropuertos


Identifying factor that contribuyes to the ATM system performances

Example: Delay Performance Breakdown decomposition!Incremental approachMain assessed approaches:

PRCapproach

Initial delay attributes characterisationInitial delay breakdown decomposition

Initial Delay global indicators organisation

Initial identificationof factor and subsystem contributing to the total ATM system Delay

Proposal of aggregation models.

+

Integrated delay measurement framework•delay intrinsic attributes•delay indicators characterisation•breakdown decomposition•aggregation model

CAF/Tapproach

+ CFMUapproach

+ CODAapproach

+R&D

projectsapproach

+ Operation assessmentapproach


y Aeropuertos


Initial DELAY intrinsic attributes Abstracted from PRC

DELAYDELAY

DELAYDELAY DELAYDELAY DELAYDELAY

Delay key attributesMAGNITUDSEVERITYABSORPTION

Total delay/Total Nº of flights

Nº of delayed flights/Total Nº of flights

Total delay /Nº of delayed flights


DELAYDELAY

DELAYDELAY

Breakdown decompositionby contributions

(Phases of the flight operation, before and

during the flight)

DELAYDELAY

TD/Nº TF

Nº DF/Nº TF

TD/NºDFCausal Factor =Delay/cause

Delay key attribute extension

MAGNITUDSEVERITYABSORPTION


y Aeropuertos


Extended DELAY breakdown decomposition: Contributions to the total delay

T u rn a ro u n d D e layT A D

O ff- B lock D e layO S D

T a x i O u t d e layX ID

D e p artu re

D e p a rtu re T M A d e lay O n ro u te de lay A rriva l T M A

In f lig h t D e lay

O n S ta n d D e lay P u n tu a llity

A rriva l d e lay

G lo b a l D e lay

D e p artu re

T a x i O u t d e lay O n ro u te de lay A rriva l T M A T a x i In D e lay

In f lig h t D e lay

P u n tu a llity

A rriva l d e lay

G lo b a l D e lay

(PRC)

PRC+CODA Extension

d e m o ra p orco n tro l d e a flu en c ia

T u rn a ro u n d D e layT A D

O ff- B lock D e layO S D

T a x i O u t d e layX ID

D e p artu re

D e p a rtu re T M A d e lay O n ro ute de lay A rriva l T M A

In flig h t D e lay

O n S ta n d D e lay P u n tu a llity

A rriva l d e lay

G lo b a l D e layPRC+CODA+ Operational assessment views


y Aeropuertos


DELAYDELAY

DELAYDELAY

DT/Nº TV

Nº VD/Nº TV

DT/Nº VD


*___*___

_flightsofnumberTotaldelayofminutesTotal

MagnituteDelay

*___*____

_

flightsofnumberTotalflightsdelayedofnumberTotal

severityDelay

*____*___

_

flightsdelayedofnumberTotal

delayofminutesotalAbsorptionDelay

* Without repetition nor duplication

Initial Delay Aggregation Model


Breakdown decompositionby contributions

(Phases of the flight operation, before and during the flight)

Total delay/Total Nº of flights

Nº of delayed flights/Total Nº of flights

Total delay /Nº of delayed flights


y Aeropuertos


Mixed time references systems and origins (I)•Several time reference system are used for measuring delays:•There is a need of defining those references clearly and stabilised how to pass from one to another. Co-ordinates translation

PRC Defines :Optimum timesPreferred timesPlanned times

Scheduled timesActual times

Optimum times

Preferred times

Planned times

Scheduled times

Actual times

Event at Tn

Difference between optimum and preferred time

Difference between planned and preferred time

Difference between schedule and planned time

Difference between actual and schedule time

Difference between actual and optimum

Difference between actual and planned time

Delay can be calculate as difference between the various time reference as

convenience


y Aeropuertos


DELAYDELAY

place for R&D actions in the delay measurement framework?

DT/Nº TV

Nº VD/Nº TV

DT/Nº VD

CAPACITYCAPACITY

Delay/traffic curves as a measure of capacity variation

Capacity Index

Areas requiringfurther R+DCapacity


Causal Factor =Delay/cause

Specially attention should be pay on traffic patterns definition when relating delay and capacity.

?

•Lack of models for understanding the high level relationships between performance parameters.•Extension of the breakdown decomposition•Definition of proper mechanisms that relates the various time reference systems so mechanisms for translation are available.•Mapping delay metrics to the proposed structure and database population


y Aeropuertos


Conclussions.Conclussions.Project objectProject object

• To have available a measurement framework (an homogenous and coherent set of metrics) for the whole concept life cycle.

• To unify sources and stakeholders views.

• To be corrected and completed by ATM Community

• To be used by anyone that could be interested in.

Documents

Departamento de Infraestructura, Sistemas Aeroespaciales y Aeropuertos DISAA - UPM May 2002 FAA / Eurocontrol AP2, 5 and 6 TIM/9 Are homogeneous performance