025 ATFCM Performance and Declared Capacity

EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL EXPERIMENTAL CENTRE

ATFCM Performance and Declared Capacity Study

Work Interim Results (2006)

ATFM Ground Regulation Efficiency

EEC Note No. 20/2006

Project NCD-1-CD-FLOW

Issued: December 2006

The information contained in this document is the property of the EUROCONTROL Agency and no part should be reproduced in any form without the Agency’s permission.

The views expressed herein do not necessarily reflect the official views or policy of the Agency.

This document has been collated by mechanical means.

Should there be missing pages, please report to:

EUROCONTROL Experimental Centre Publications Office

B.P. 15 91222 BRETIGNY-SUR-ORGE Cedex

France

DOCUMENT CHANGE RECORD

The following table records the complete history of the successive editions of the present document.

EDITION DATE DESCRIPTION OF EVOLUTION MODIFICATIONS

0.1 October 15th2006 First Draft

0.2 December 2006 Final Note

Intentionally blank

REPORT DOCUMENTATION PAGE

Reference EEC Note No. 20/2006

Security Classification Unclassified

Originator EEC - NCD (Network Capacity and Demand management) Research Area

Originator (Corporate author) Name/Location : EUROCONTROL Experimental Centre B.P.15 F-91222 Brétigny-sur-Orge CEDEX FRANCE. Telephone: +33 (0) 1 69 88 75 00

Sponsor CFMU

Sponsor (Contract Authority) Name/Location EUROCONTROL Agency Rue de la Fusée, 96 B-1130 BRUXELLES Telephone: +32-(0)2-729 90 11

Title : ATFCM Performance and Declared Capacity Study - Work Interim Results (2006)

Authors

Marc Dalichampt (EEC) ADV Systems

Date 12/06

Pages 34

Figs 6

Tables

Annex 1

References 4

Project NCD-1-CD-FLOW

Sponsor Task No.

Period Year 2005

Distribution Statement : (a) Controlled by : Head of NCD (Network Capacity and Demand management) (b) Special Limitations (if any) : None (c) Copy to NTIS : No Descriptors (keywords): Uncertainty, Smoothing, ATFCM, ATFM, Indicators, CFMU, Performance, FMP, Flow, Capacity Abstract : Objective: This note presents the interim conclusions of a study launched in April 2006.The study aimed at proposing a list of ATFM performance indicators that reflect the variety of the ATFM users' objectives. The study was part of the yearly NCD "ATFCM Studies" work programme, in support of and funded by CFMU. Results: The notion of ATFM performance should be associated to a wide range of users' objectives which lies well over the traditional resolution of the hourly demand excesses problem. Other objectives are: Reduction of the traffic bunching problem, prevention from the forming of complex traffic clusters within the sectors, improvement of flows predictability, optimisation of the declared capacity. Different indicators are defined to assess whether the system is able or not to meet these objectives. In some cases, it is possible to observe that a given regulation can be very efficient for the achievement of one particular objective while providing modest results for the achievement of the other objectives. That is why, it is proposed to assess the performance by applying the whole set of indicators.

Intentionally blank

EUROCONTROL Experimental Centre ATFCM Performance and Declared Capacity Study

TABLE OF CONTENTS

1 INTRODUCTION 1 1.1 DOCUMENT RATIONALE................................................................................... 1 1.2 CONTEXT ....................................................................................................... 1 1.3 STRUCTURE OF THE DOCUMENT ...................................................................... 2 2 ATFM REGULATION EFFICIENCY: PERFORMANCE MEASURE ACCORDING TO USERS’ OBJECTIVE 3 2.1 OBJECTIVE N°1: IMPROVEMENT OF THE INCOMING TRAFFIC FLOWS SMOOTHING . 3 2.2 OBJECTIVE N°2: IMPROVEMENT OF FLOWS PREDICTABILITY............................... 4 2.3 OBJECTIVE N°3: OPTIMISATION OF CAPACITY USE............................................. 4 2.4 CONCLUSIONS................................................................................................ 5 3 PERFORMANCE INDICATORS 6 3.1 SMOOTHING INDICATORS ................................................................................ 6 3.2 UNCERTAINTY REDUCTION INDICATORS.......................................................... 14 4 ILLUSTRATIVE APPLICATION CASE 17 4.1 SMOOTHING PERFORMANCE APPLICATION CASE: ANALYSIS OF ONE REGULATION PERIOD (EUHL26A REGULATION - LFEUHL4 TV) .......................................... 17 4.2 UNCERTAINTY REDUCTION APPLICATION CASE: LFEUF4 TV (REIMS ACC) ANALYSIS, OVER ONE AIRAC CYCLE .............................................................. 20 5 APPROPRIATENESS OF THE USE OF ALL-FT FILES VS. THE USE OF SIMULATION RESULTS (TACOT) 23 5.1 RELEVANCE OF A STEP-BY-STEP SIMULATION OF THE SLOT ALLOCATION PLAN LIFE-CYCLE .................................................................................................. 23 5.2 SIMULATIONS RESULTS ................................................................................. 24 6 CONCLUSION & NEXT STEPS 29 6.1 CONCLUSION: A SEGMENTED ATFM GROUND REGULATION PERFORMANCE ASSESSMENT ............................................................................................... 29 6.2 NEXT STEPS................................................................................................. 30


List of Figures

Figure 1: Flow evolution curves.............................................................................................. 6 Figure 2: 2D view (average amplitude and duration of the traffic load fluctuations).......... 8 Figure 3: 2D view (average amplitude and duration of the traffic load fluctuations).......... 9 Figure 4: Flow evolution curves (measures of max. and average excesses) ........................ 11 Figure 5: Flow evolution curves (measure of the average deviation from the flow rate).. 12 Figure 6: Distributions of TV entries over the reg. period (measure of the time during

which TV entries are over a certain level) ......................................................... 13


GLOSSARY ACC Area Control Centre AIRAC Aeronautical Information, Regulation and Control AO Aircraft Operator ATC Air Traffic Control ATCO Air Traffic Controller ATFCM Air Traffic Flow and Capacity Management ATFM Air Traffic Flow Management ATM Air Traffic Management ATO Actual Time Over CASA Computer Assisted Slot Allocation CFMU Central Flow Management Unit CPR Correlated Position Report CTO Calculated Time Over CTOT Calculates Take-Off Time EEC EUROCONTROL Experimental Centre ETFMS Enhanced Tactical Flow Management System ETO Estimated Time Over FMD Flow Management Division (CFMU) Controllers FMP Flow Management Position (ACC) FPL Filed Flight Plan IFPS Integrated Initial Flight Plan Processing System NCD Network Capacity and Demand PRU Performance Regulatory Unit PRR Performance Review Report RTO Reference Time Over TMA Terminal Manoeuvring Area TV Traffic Volume WP Work Package


REFERENCES

EUROCONTROL / Performance Review Commission: Performance Review Reports: n°1 (1998) to n°8 (2005) EUROCONTROL Tasking Support for ATFM – Impact of uncertainty and smoothing on declared capacity – Interim results (November 2005) EUROCONTROL Tasking Support for ATFM – Impact of uncertainty and smoothing on declared capacity – General framework (November 2005) EUROCONTROL Tasking Support for ATFM – Intermediate note on ATFM efficiency indicators - ATFCM Performance and Declared Capacity study (July 2006)


1

1 INTRODUCTION

1.1 Document rationale

The present document was issued in the frame of the “ATFCM Performance and Declared Capacity study”, which has been launched by the EEC in April 2006. It provides the interim results drawn from the conclusions of the first two WPs which primarily intend to propose a list of ATFM performance indicators and to apply this set of indicators on some illustrative application cases.

The document investigates also the suitability of using the “FTFM1” and “RTFM2” profiles obtained from the processing of “after-the-fact” data, which is extracted from the CFMU “ALL-FT” files. The results provided by the use of the “ALL-FT” files are confronted to those obtained with the use of an ATFM fast time simulator (TACOT), when trying to assess the performances of the ATFM process and especially ground regulation one.

Indicators and application cases were presented and discussed with CFMU representatives in October 2006.

1.2 Context

The conclusions of a previous study, launched by the EEC in September 2005: “Study of the impact of ATFM uncertainty and smoothing performance on declared capacity” demonstrated that the notion of ATFM performance should be associated to a wide range of users’ objectives which lies well over the traditional resolution of the hourly demand excesses problem:

Ground regulation users’ secondary objectives:

• Reduction of the traffic bunching problem;

• Prevention from the forming of complex traffic clusters within the sectors;

• Reduction of the uncertainty (improvement of flows predictability); and

• Optimisation of the declared capacity.

Therefore, the ATFM performances should reflect the variety of the ATFM users’ various objectives and especially the ones of the FMPs and FMDs. The identification of the users’ objectives associated to ATFM performance has constituted the baseline for the instigation of the work on ATFM performances indicators. Once the objectives have been identified, a set of indicators has been proposed to assess the performances of the ground regulation process for each of the identified objectives. Those indicators have been further “tested” on sundry application cases.

1 The FTFM is the “initial” profile as it reflects the status of the demand before activation of the regulation plan. It is computed with the latest flight plan version, sent by each AO to the CFMU/IFPS. 2 The RTFM is the “regulated” profile as it reflects the status of the demand after activation of the regulation plan. It is computed with the latest ATFM slot (CTOT) issued to the AO, by the ground regulation system.


2

1.3 Structure of the document

The document is broken down into six sections and one annex:

Section 1 – Introduction – presents the purpose and structure of the present document;

Section 2 – ATFM regulation efficiency: Performance measure according to users’ objective – recalls the users’ objectives identified in the previous 2005 study and presents how the performance indicators are related to them;

Section 3 – Performance Indicators – presents the indicators proposed to give account of the system’s capability to fulfil the users’ objectives and traffic counting functions on which these indicators are based;

Section 4 – Illustrative application cases – presents the results, which could be obtained from the application of the indicators on a particular case;

Section 5 – Appropriateness of the use of ALL-FT files vs. the use of simulation results (TACOT) – assesses the suitability in using “after the fact” data extracted from ALL-FT files, as compared to the use of results provided by an ATFM fast-time simulator (TACOT), for the measurement of ATFM performances;

Section 6 - Conclusion and Next steps – presents the interim conclusions on the proposed performance assessment method, the next work steps and the associated requirements.

Annex 1 – ATFM smoothing performance assessment (methodology) – presents the methodology applied for the traffic data analysis and the building of the different observables of the ATFM slot allocation smoothing efficiency.


3

2 ATFM REGULATION EFFICIENCY: PERFORMANCE MEASURE ACCORDING TO USERS’ OBJECTIVE

As specified in the introduction of this document, our approach is essentially established on the account of a number of users’ objectives which stretches wider than the traditional basis upon which the ATFM performance is assessed (resolution of the hourly demand excesses problem).

In this document, the term “users” of the ATFM system refers to the direct beneficiaries of the ATFM ground regulation system, i.e. ATFM operational actors – FMPs, FMDs and controllers.

A recall of the users’ objectives and how the performance indicators relate to them is presented in the next paragraph.

2.1 Objective n°1: Improvement of the incoming traffic flows smoothing

Three distinct smoothing purposes <> Three measure parameters

In interviewing FMPs and FMDs, it appeared that the smoothing objective could be parted in three different sub-objectives:

Firstly, the traditional resolution of the hourly demand / capacity imbalance. This refers to the original mandate attributed to the ATFM regulation system and relies on a commitment vis-à-vis a number of movements per hour.

The corresponding performance indicators will therefore derive from the analysis of hourly flow counts.

Secondly, the reduction of the traffic bunching. The problem of bunching has much to deal with the punctual accumulation (bunching peaks) of traffic on controllers working positions, even in situations when the hourly traffic flow complies with the declared capacity. FMPs regularly exploit the traffic smoothing capability of the ground regulation process for the resolution of this most “unpleasant” situation for ATCOs, especially when the ATCOs are already working at the limits of the hourly sector capacity.

In that case, the corresponding performance indicator derives from the analysis of twenty minutes flow counts. This is consistent with operational figures a priori determinant in terms of ATCOs perception of demand pressure; such as the average time of virtual or effective presence of a flight into ATCO responsibility area, or the minimum time-window which the ATFM system offers to analyse the flow throughput.

Last but not least, the prevention from the forming of complex traffic clusters within the sectors. This last objective may be considered out of ATFM scope, and is normally addressed directly by the ACCs: FMPs, together with ATCOs collaboratively coordinate with upstream ACCs for obtaining a regular spacing of aircraft and preventing from the forming of complex traffic clusters within the sectors. However, we were reported that FMPs may also wish to improve in advance the regularity of the flows


4

throughput and use the “limit” smoothing capability of the ATFM system for this “à la marge” traffic smoothing objective.

The corresponding performance indicator derives from the analysis of instantaneous – 1 minute flow counts.

2.2 Objective n°2: Improvement of flows predictability

The second class of objective – Improvement of flows predictability – reflects a feeling regularly expressed by FMPs and controllers that they may request the implementation of ATFM ground regulations, even when the planned demand excesses are low, in order to improve the predictability of the incoming flows.

One of their primary concerns is to make sure that the uncertainty, conveyed by the stochastic nature of air transport, could be contained in between stable and reduced margins. To this regard, the slot allocation process is considered as a mean to improve flows predictability, since it imposes additional departure time restrictions3.

This commonly admitted explanation as to why the flows predictability is improved is assessed using a specific performance indicator accounting for the gain in time precision over TV entry when ATFM regulations are implemented.

From this perspective the predictability is measured in terms of individual flights time deviation between actual and planned times of penetration into a given airspace.

2.3 Objective n°3: Optimisation of capacity use

The last class of objective is an indirect, longer term objective that is out of the performance measurement scope presented in this document. It focuses on the direct results of ATFM ground regulation application (smoothing and predictability improvement) on the declared capacity over the years.

As a matter of fact, the calculation of the declared capacity could be sometimes an empirical process. A way of assessing a suitable value for the declared capacity of a given TV consists in issuing some ATFM ground regulations with gradual increased values. The process allowing the suitable declared capacity value could be explained as follows:

At first the FMP issues a low value for the declared capacity and if there is no ATCO report on over-delivery for some times, the FMP will gain confidence in the system and will increase gradually the declared capacity value to a point at which some ATCOs will complain on over-deliveries.

Therefore, the objective – that draws the link between the ATFM service performance and the capacity offer – could be assessed using a specific performance indicator accounting for the positive evolution of capacity figures, if not over the entire life cycle of the ATFM ground delay service, at least over months or years.

3 The regulations restrict departure times’ variations down to a 15 minutes tolerance whilst the tolerance applicable on non-regulated traffic is 30 minutes.


5

2.4 Conclusions

The performance of a given ATM ground regulation should reflect the objectives sought by the users when issuing the ground regulation. That is why, we propose to assess the overall performance of a ground regulation, by applying the whole set of indicators on this regulation. We will probably observe that a given regulation can be very efficient for the resolution of the bunching effect when providing modest results for the prevention of hourly flows over-deliveries. In other cases, we will maybe find that a given ground regulation has been issued just to reduce the uncertainty conveyed by the stochastic nature of air transport.

In any case, the performance of a given regulation will be assessed in applying the whole set of indicators reflecting all the users’ objectives.


6

3 PERFORMANCE INDICATORS

Performance indicators are derived from a model, which describes the different input databases, counting functions and observation parameters used for the computation of each indicator. The model is presented in Annex 1.

Performance indicators are presented in the following chapters.

3.1 Smoothing indicators

Flow evolution curves The ATFM activity is based on the notion of traffic deliveries (or traffic flows) throughput. Local ATFM service users (FMPs) specify a particular request on traffic flow throughput (flow rate) in order to meet a specific load requirement.

Therefore, the adopted indicators of performance will rest on flow evolutions curves.

The curves are obtained from CFMU data regulation records, and present the evolution, minute per minute, of TV entries during a certain period “ahead” (i.e. the number of aircraft flying over a sector entry point or landing at an airport in a specific unit of time).

The time period may be 60 minutes, 20 minutes, 1 minute, depending on the smoothing measured objective: “resolution of the hourly demand / capacity imbalance”, “reduction of the traffic bunching”, or “prevention from the forming of complex traffic clusters within the sectors”, respectively.

An example of flow evolution curves is presented below: The figure on the left presents the evolution of TV entries for the period elapsing from “t” to “t + 60”4 minutes, at three different anticipation levels: just before the activation of the reg. process (blue curve) and just after the activation (pink curve) of the reg. process, then at flights entry time into the regulated traffic volume (green curve). The requested hourly flow rate level (47) is pictured in black.

Figure 1: Flow evolution curves

4 A time index is represented on the (X axis). The origin (Index “0”) corresponds to the time of activation. Index “178” corresponds to the start of regulation period. The regulation process is therefore activated at anticipation “178” minutes, i.e. 2 h 58 min.

Regulation period


7

In order to build a practicable smoothing performance evaluation, the shapes of flow evolution curves have been characterised by a limited number of determinant figures, measuring:

Individual evaluation of the traffic load fluctuations amplitude and duration (analysis of the traffic fluctuations whatever the originating regulation(s)) ;

Scan of one regulation period (analysis of one particular regulation):

• Amplitude of the traffic excesses, over the regulation period;

• Average deviation from the flow rate (efficiency in the use of capacity);

• Dispersion around other references than the flow rate.

In the measures proposed herein, two levels of analysis are distinguished.

The first level of analysis – individual evaluation of the traffic load fluctuations – will correspond to a discrete characterisation of the flow evolution curve. The curve is indeed characterised by (amplitude, duration) pairs of measures, each pair being associated to a traffic load fluctuation. Given a regulation, these measures address each fluctuation individually, breaking the link between them (i.e. their common regulation period). Therefore, they are a good way to look into the overall protection of one to several regulations (e.g. protection of one TV over one AIRAC cycle), by aggregating results whatever their originating regulation.

The second level of analysis – scan of one regulation period – put the focus on one particular regulation period. Here, the indicators measure if the implementation of this regulation is in line with what the FMP was expecting, i.e. if it would allow the ATCO on his position to deal with the actual traffic and if the available capacity is efficiently used, over the regulation period.


8

Individual evaluation of the traffic load fluctuations. Indicator I1 (reduction of the duration and average amplitude of the fluctuations).

The primary purpose of ATFM service users (FMPs) in each ACC is to avoid situations of unbearable ATCO workload. The objective is the elimination of the uncontrolled loads of traffic (traffic peaks during significant periods of time).

Another objective is to make the most efficient use of the available capacity (deduced from the flow rates requests) by minimising capacity wastes (traffic shortfalls during significant periods of time), and to limit the burden (delays) imposed on airspace users induced by the implementation of regulations.

Therefore, ATFM smoothing performance will be firstly evaluated with regards to the reduction, in terms of duration and amplitude, of the traffic load fluctuations (“peaks” and “shortfalls”) around the flow rate.

In order to discriminate between the different fluctuations, we propose to segment the regulation period into series of shorter intervals, within which the flow evolution curve is continuously either strictly above (peak interval) or below (shortfall interval) the requested flow rate.

The result is a discrete characterisation of the flow evolution curve in terms of average amplitude of a fluctuation (positive or negative) and duration of the fluctuation.

Illustration n°1: results for one regulation.

A first illustration, showing the average amplitude and duration of the traffic load fluctuations for one single regulation (case of the example above presented – figure 1) is presented below5.

Figure 2: 2D view (average amplitude and duration of the traffic load fluctuations)

5 The average amplitude is measured in terms of number of aircraft above or below the flow rate and in proportion (%) to the flow rate.

Fluctuation of average amplitude +6 (+13%) and duration 108 minutes

Fluctuation of average amplitude -10 (-21%) and duration 106 min.

Flow Rate: 47


9

In the above figure, it could be shown that the blue curve (before activation of the reg. process) is represented by one dot showing one over-delivery of average amplitude 13% and one under-delivery of average amplitude 21%.

It could be also noticed that the pink curve (after activation of the reg. process) is very fluctuating, and close to the flow rate as being represented by several dots close to the “flow rate (i.e. 47)” axis.

Pink dots are inside a zone (in green) delimited by -10% and +10% below and above the flow rate. Then, it could be inferred that the regulation is “successful”, as the over-deliveries are fully eliminated according to a traditional7 tolerance margin accepted by the FMP (in case of hourly counts measurements).

Illustration n°2: aggregated results: several regulations (10 TVs, 6 days). As already mentioned, the 2D view presented can feature a wider array of points, whatever their originating regulation. Thus the use of the (amplitude, duration) pair of measures is a good way to look into the overall impact of a sample of regulations.

Figure 3: 2D view (average amplitude and duration of the traffic load fluctuations)

The aggregated results figure6 demonstrates clearly that regulations – most of the time – prevent or at least decrease substantially over-deliveries (most of the pink dots inside the -10% ; +10% green zone). However, some exceptions are recorded: 3 pink dots inside the {> 10% ; > 20 minutes}7 orange zone.

6 The “Y axis” evaluates the fluctuations in proportion to the flow rate. 7 The criteria are at the convenience of the ATFM service user. A {>10% ; > 20 minutes} zone would correspond to the outside zone of tolerance margin for the resolution of hourly over-deliveries, within which the FMPs / FMDs are traditionally satisfied with the regulation. Regarding the assessment of capacity wastes (under-deliveries), it could be inferred that the outside zone is symmetrical {i.e. <-10%; > 20 minutes} but this is to be confirmed and checked with the FMPs / FMDs.

12 situations with initial demand, exceeding in amplitude (> 10%) and duration (> 20 minutes)

3 situations with regulated demand, exceeding in amplitude (> 10%) and duration (> 20 minutes)

17 situations with significant initial under-deliveries (< -10%) and (> 20 minutes)

10 situations with regulated under-delivered demand (< -10%) and (> 20 minutes)


10

The observation of the blue zone {< - 10% ; > 20 minutes} depicts a more preocupaying situation, in terms of capacity efficiency use: still significant 10 under-deliveries records, revealing capacity wastes.

Thus, the use of the 2D views, as illustrated by the last presented figures are a good way to discriminate between the successes patterns (a “success” zone that would include the points for which the smoothing is successful according to a tolerance margin, in this case: between -10% and +10%) and exceptional or failure situations (the outside zones).

Then, a first indicator of performance (I1) could be the proportion of points inside the “success” zone, with respect to the points that are in the “outside” zone.

The reason for experimenting peaks of significant amplitude and duration, in zones outside the tolerance margin within which regulations are considered “successful” by the ATFM service user, are to be investigated using additional information recorded at the CFMU in heavy databases (Oplog files). In these files, all the relevant ATFM events, including regulation requests tracks are archived.

In many cases, sources of inefficiency situations are related to the system’s inertia (regulations have not been issued before the time required by CFMU to operate properly a regulation).


11

Scan of one regulation period: Indicators I2, I3 (prevention from over-deliveries)

Given one regulation or given several regulations, the 2D view (amplitude; duration – Indicator I1) is a very good way to look into the success or the failure of the TV(s) protection and use of the available capacity. This analysis can be done for several regulations, since each fluctuation is characterised individually.

Now, when trying to figure out if one particular regulation is in line with what the FMP was expecting; one has to figure out if the implementation of this specific regulation could allow the ATCO on his position to deal with the actual traffic.

To give account of this, the shape of the flow evolution curve can be characterised by the maximum peak and average traffic excesses records, over the regulation period. The two are complementary since an excess may not climax at the maximum, for a long time, which in turn mitigates the average excess record.

Then, another pair of indicators derives from the measures of the maximum of the flow evolution curve (I2) and of the average of the Flow evolution curve excesses (I3) over the regulation period.

In the illustration besides (case of hourly counts, before and after the activation of reg. process), the maximum amplitude of excesses is 59 (excess of 26%), while the average is 53 (excess of 13%), for the blue curve (before reg.). The corresponding values for the pink curve (after reg.) are 51 (excess of 9%) and 48 (excess of 2%), respectively.

In this example, it could be shown that the regulation was of interest in lowering the traffic excesses, over the period.

Figure 4: Flow evolution curves (measures of max. and average excesses)

Note: In the above figure, the shape of the flow evolution curve is evaluated in terms of numbers of aircraft and in proportion (%) to the flow rate (e.g. 53 aircraft corresponds to (53-47)/47, i.e. 13%)

Max (Before reg.): 59 (26%)

Average (Before reg.): 53 (13%)

Average (After reg.): 48 (2%)

Max (After reg.): 51 (9%)


12

Scan of one regulation period: Indicator I4 (capacity use) The other component of the ATFM smoothing performance relates to an efficient use of the achievable capacity offered by the ATCO (deduced from the flow rate requests).

Therefore, when looking at the performance of one single regulation, one has to look on the ability of the system to maintain the load as close as possible to the flow rate, over the regulation period. The system is all the more efficient as the average deviation from the flow rate is low.

Then, a fourth indicator derives from the measure of the average deviation (in absolute value) from the flow rate, over the regulation period.

In the illustration besides (case of hourly counts, before and after the activation of reg. process), the initial (blue curve) average deviation from the flow rate is equal to 8 (17%) whereas the system output a distribution with average deviation 1 (2%).

It could be shown the efficiency of the regulation in reshaping the initial distribution very close to the flow rate.

Figure 5: Flow evolution curves (measure of the average deviation from the flow rate)

Note: In the above figure, average deviations are evaluated in number of aircraft and in proportion to the flow rate (e.g. an average deviation of 8 aircraft corresponds to 8/47, i.e. 17%).

Average deviation from the flow rate

(Before regulation): 8 (17%)

Average deviation from the flow rate (After Regulation):

1 (2%)


13

Dispersion around other references than the flow rate, capacity management Indicator I5 (definition of a “tolerance” margin for over-deliveries)

With the last presented measures, the smoothing is characterised with respect to a “standard” reference, the requested flow rate.

In addition to the compliance with the requested flow rate, ATFM users may have different criteria against which their own evaluation of the system’s performance is made. Depending on the user, the “tolerance” margin within which a traffic excess remains acceptable could be for example of 0%, 10% or 20% above the flow rate. In the same way, the efficiency in the capacity use could be considered as achieved when the load is above a limit of -20%; -10% or 0% below the flow rate.

Therefore, the smoothing quality could be also evaluated when that natural reference (the flow rate) changes.

Then, a fifth indicator derives from the measure of the amount of time (expressed in terms of percentage of the total regulation time) during which the traffic flow has been over a certain level. That “certain level” is the new reference and is defined over a range of values centred on the optimum (the requested flow rate). Then it is another way of measuring the ability to keep the regulated traffic as close as possible to the flow rate.

In the illustration, the initial (blue curve) traffic excesses account for 47% of the total regulation time whereas the system output a distribution with excesses accounting for only 8% of the total regulation time.

When the reference changes (new reference: Flow Rate + 20%, i.e. 56), initial traffic excesses account for 5% of the total reg. time, while the system output a distribution with all the period below that reference (no excesses > 20%).

Figure 6: Distributions of TV entries over the reg. period (measure of the time during which TV entries are over a certain level)

Note: The ideal distribution would be a regulated curve confounded with the “Y axis”.

Flow Rate: 47

After regulation: % of reg. period with TV Entries > Flow Rate: 8%

Before regulation: % of reg. period with TV Entries > Flow Rate: 47%

New Reference: Flow Rate + 20%: 56

Before regulation: % of reg. period with TV Entries > Flow Rate + 20%: 5%


14

3.2 Uncertainty reduction indicators

Flight time deviations on arrival at the TV

The commonly admitted explanation as to why the predictability of incoming flows for a given TV is improved with ATFM ground regulation implementation is the application of additional time restrictions on the departing traffic, which squeeze operational drifts prior to take-off, and hence benefit to the subsequent flight segments, up to the TV in question.

Therefore, uncertainty reduction is here mirrored by the reduction in flight time deviations on arrival at the TV.

Flight time deviations are evaluated by comparing time estimates over TV entry points. The estimates are either an ETO8 in the case of non-regulated flight or a CTO9 in the case of regulated flight:

Non Regulated: The flight time deviation is measured by comparing ETO (Estimated Time Over) and ATO (Actual Time Over) on arrival at the TV;

Regulated: The flight time deviation is measured by comparing CTO (Calculated Time Over) and ATO (Actual Time Over) on arrival at the TV.

The methodology is based on a global evaluation – one studied TV over one to several AIRAC cycles (sample of analysis). Within the sample of analysis, the flights are sorted by call-sign and grouped in two subsets: regulated and non-regulated.

For each call-sign, a twofold measurement is performed for each subset:

1. Average flight time deviation over TV entry point (mean value);

2. Dispersion around the average (standard deviation).

Call-signs are then put to the test individually to determine whether the measurement is statistically meaningful. The “succeeding” call-signs are those for which significance is ensured with a confidence level of at least 95%, a standard value in statistics.

Significant call-signs are then correlated to figure out (if possible) a relationship between the average flight time deviation without regulation and the average flight time deviation with regulation. The type10 and coefficients of the relationship are the indicators of the impact of ATFM measures on uncertainty reduction (I6). A graphical illustration, recapping the main steps of the methodology, is presented below.

8 The ETO is the expected time of penetration within the TV, when the flight is not regulated. It is obtained from the latest flight plan sent by the AO to the CFMU. 9 The CTO is the expected time of penetration within the TV, when the flight is regulated. It is deduced from the latest slot (CTOT) issued to the AO. 10 Premininary results seem to derive a linear relationship (see § 4.2)


15

3/ Processing

4/ Analysis

2/ Reshuffle

1/ Collection TV

Day 1

TV

Day 2

TV

Day -

NO REG

REG

Call-sign 1 Call-sign 2 Call-sign -

TVCall-sign

NO REG ∆ = | ATO - ETO |

∆ = | ATO - CTO |

Movements

REG

∆ vs. occurrences

Distributions

Call-signs are processed one-by-one

Call-signs are tested one-by-one: are they statistically significant? [The scope extension is an important enabler of statistical significance]

Yes No

Mea

n RE

G

Mean NO REG

Significant call-signs

Non-significant call-signs

Correlation coefficient

5/ Integration

All movements are captured

through TV, over timeframe

Movements are sorted by: 1/ call-sign 2/ status (regulated or not)


16

1/ Data Collection:

The studied TV is put under the microscope over a significant amount of time, typically several AIRAC cycles. Every movement on record is taken into account within that scope. 2/ Data Reshuffle:

All movements sharing the same call-sign are clustered. Each cluster is partitioned: non-regulated movements vs. regulated movements (whatever the origin of the most penalising regulation). 3/ Data Processing:

For each cluster, the two subsets are the baseline samples from which the distributions of operational deviations are drawn:

Deviations from ETO11 for non-regulated movements;

Deviations from CTO12 for regulated movements. 4/ Data Analysis:

The two distributions are weighed one against the other to determine whether the cluster is statistically significant. Is it a meaningful indicator of how the call-sign behaves routinely, whether or not it is submitted to restrictions? Does it denote a deep-rooted operational change? A confidence level of 95% is retained to make a decision. Presumably: the broader the initial scope, the greater the number of significant call-signs. 5/ Data Integration:

Significant call-signs are “benchmarked” to try and set out a trend. In other words, if the correlation between them is strong enough, then a relationship can be drawn to model the impact of ATFM measures on predictability.

11 The ETO is the expected time of penetration within the TV, when the flight is not regulated. It is obtained from the latest flight plan sent by the AO to the CFMU. 12 The CTO is the expected time of penetration within the TV, when the flight is regulated. It is deduced from the latest slot (CTOT) issued to the AO.


17

4 ILLUSTRATIVE APPLICATION CASE This chapter presents the results of the application of the previous indicators on a particular case – Reims ACC, together with first indications that can be derived from the indicators analysis.

4.1 Smoothing performance application case: analysis of one regulation period (EUHL26A regulation - LFEUHL4 TV)

The case of the French North East ACC is interesting since located in an area with high concentration of traffic and complex flows crossing. A particular point on this case is the mix of cruising flows, of low complexity from an Air Traffic Controller viewpoint, with the climbing or descending flows from or to the airports of Zurich, Geneva, etc. which are of higher complexity.

Hence, in order to avoid the uncontrolled accumulation of complex aircraft, FMPs frequently request regulations applied to the complex flows, even when the planned demand hourly excesses are low. Here, the objective is more to tackle the bunching threat of the climbing or descending aircraft on the working positions, than to maintain the hourly load below a given barrier.

Because the resolution of the bunching problem is one of the intended objectives, the emphasis here is put on the assessment of that performance of the regulation system as well as on the (mandatory) performance of resolution of the hourly demand / capacity imbalance. As the illustration is focused on one single regulation case, the analysis is performed using the indicators I2, I3, I4 and I5.

First Objective: Resolution of the hourly demand / capacity imbalance

Analysis of the load evolution for initial (before activation of the reg. process) and “after regulation” (just after activation of the reg. process) traffic (60 min. counting step).

Maximum amplitude of over-deliveries (before reg.) (I2): 22%

Average amplitude of over-deliveries (I3) (After regulation): 0%

Average amplitude of over-deliveries (before reg.) (I3): 9%

% of reg. period with TV Entries > Flow Rate (I5): 56%

Flow Rate

Flow Rate


18

It could be noticed that the hourly situation is perfectly corrected (I2 and I3 after reg. – pink curve), albeit not critical initially (I2 and I3 before reg. – blue curve) with an average excess below 10%.

Second Objective: Reduction of the traffic bunching

Analysis of the load evolution for initial (before activation of the reg. process) and “after regulation” (just after activation of the reg. process) traffic (20 min. counting step) 20 min counts depict a more preoccupying situation, unveiling greater demand excesses (I2 and I3 before reg.) than expected with 1 h counts.

By the way, 20 min excesses and 1 h excess are not in phase. The regulation process proves to be efficient for the bunching objective; traffic is smoothened below 10%.

% of reg. period with TV Entries > Flow Rate + 10%: 1% (after reg.)

Maximum amplitude of over-deliveries (before reg.) (I2): 78%

Average amplitude of over-deliveries (I3) (after regulation): 4%

Average amplitude of over-deliveries (before reg.) (I3):

39%

Flow Rate


19

Synthesis table: 60 min. vs 20 min. counts

% 60 min. 20 min.

Before reg. After reg. Before reg. After reg.

I2 22 0 78 11

I3 9 0 39 4

I4 9 16 37 17

I5 (0%) 56 0 30 8

I5 (10%) 19 0 27 1

I5 (20%) 4 0 21 0

Summing up values is interesting to find trends. Measures indicate that:

From the (I2 ; I3) perspective (amplitude of the excesses), performance is perfect for the 60 min. case and very good for the 20 min. case (e.g. from 39% initial average excesses to 4%, while looking at the after reg. result);

Not surprisingly, from the I5 perspective, the performance is better, not to say perfect, for significant excess (> 20%) because smoothing first tackles those ones. All excesses > 20%, dealing with an hourly basis are cut down for both 60 min. and 20 min. bases;

On the other hand, from the (I4) perspective (use of capacity), the impact of the regulation is negative (from 9% initially to 16%, while looking at the after reg. result), in the case of 60 min. counts, but positive, when associated to the objective of bunching reduction (from 37% initially to 17%, while looking at the after reg. result).

Therefore, the indicators confirm the added-value of the system for the objective of bunching reduction. On the other hand, they point out a certain inefficiency of the regulation for the hourly situation. Even, if the hourly excesses are fully eliminated, the situation is not critical initially and capacity is wasted with the implementation of the regulation.

In the example presented, it could be noticed that the performance of a regulation should be assessed in applying the whole set of indicators, for the different count parameters reflecting the objectives sought by the users. In this regulation case, it could be seen that the regulation was issued for the reduction of the bunching essentially.


20

4.2 Uncertainty reduction application case: LFEUF4 TV (Reims ACC) analysis, over one AIRAC cycle13

As mentioned in the previous sections of this document, users’ expectations towards the ATFM service go beyond the improvement of the traffic load smoothing. Interviewed FMPs reported that, even when the planned demand excess is low, the decision can be made to implement an ATFM regulation in order to reduce the uncertainty on flights at entry into the sectors.

A particular point on this case is the LFEUF4 Reims ACC TV, feeding LFEUF sector (upper En Route sector, above FL 195), which was regulated 12 days out of 28 during the studied AIRAC cycle 265. Reims FMP has confirmed that the objective pursued when regulating that TV is also to maintain the uncertainty under control and hence to ensure that actual loads will remain consistent with the prediction.

In order to assess the uncertainty reduction, when the TV is regulated, a data mining process was launched, over the entire AIRAC cycle 265. The scan of the 28 days, allowed identifying about 400 call-signs with an average of 10 non-regulated and 5 regulated movements each.

For each call-sign, an analysis of the statistical significance of the results was performed (using statistical tests), in order to determine whether or not the results are representative of how the call-sign behaves routinely when it is regulated and when it is not. For some part, statistical significance depends on the size of the call-sign sample (either regulated or non-regulated samples).

Many “underrepresented” call-signs were therefore set aside so far, whereas the remainder is showcased hereafter. For these fifteen or so call-signs, the relevance of the results is secured since statistical significance is ensured with a high degree of confidence (95%)14.

Average deviation: Regulated movements vs. Non-regulated movements Each data point stands for a call-sign and pinpoints the average deviation, for non-regulated vis-à-vis regulated movements.

13 Application case: LFEUF4 TV over AIRAC cycle 265 14 As many call-signs are considered non eligible for the analysis, there is a need to consider a larger time frame for the analysis. Therefore, the next step is to collect data over six AIRAC cycles, to say the least. As more movements are pumped into the sample, more call-signs will acquire statistical significance, becoming meaningful data points in their own right. They should give further evidence on the trend obtained so far.


21

0

2

4

6

8

10

12

14

16

18

0 2 4 6 8 10 12 14 16 18

Non-regulated movement

Reg

ulat

ed m

ovem

ent

The data points lie on the same side of the pink solid line, pointing to a reduction of operational drifts when regulations are applied. Furthermore, they tend to flock to the green dashed line, profiling a linear relationship with a twofold reduction in deviation when the flight is submitted to restrictions.

Such preliminary results echo the initial assumptions15; they seem to derive a trend, to be confirmed by the consolidation of the study, which will address more TVs, over longer time periods.

Average deviation and Standard deviation reduction evidence

Uncertainty has different components. One of them is the average deviation of flight time estimates over TV entry, which seems to be half-reduced when regulations are applied. The other component of uncertainty is the dispersion of the values around the average deviation, which under a statistical approach, is referred as the standard deviation.

The standard deviation characterises uncertainty in terms of predictability and can be very small even with a significant average deviation.

Therefore; in order to assess the predictability improvement for regulated flights of the studied application case, a measure of the standard deviation for the call-signs has also been performed. Results are illustrated below.

15 Namely that uncertainty reduction seems to be half-reduced as the shortening of the ground tolerance window, which is twofold.


22

0

2

4

6

8

10

12

14

16

18

0 2 4 6 8 10 12 14 16 18

Mean

Stan

dard

dev

iatio

n

The same call-signs are spotted twice hereunder, in the non-regulated (diamonds) and regulated (triangles) modes. The standard deviation is exhibited in addition to the mean.

A slip towards the origin is noticeable for the whole cluster. The magnitude of the shift is a rough marker of uncertainty reduction. The attracting power of the origin points out the overall impact of ATFM measures in that sense. Ideally, a call-sign lying on the focal point would become more predictable, with no operational deviation.

Conclusion on the uncertainty reduction: linear relationship of slope 0.5

In the case study presented, the comparison between the average flight time deviation with regulation and the average flight time deviation without regulation derives a trend: a linear relationship of slope 0.5.

The present result needs to be confronted with an analysis on other TVs and longer periods so as to assess if this is a regular pattern and/or if it depends from the TVs characteristics such as e.g. the distances to the departure aerodromes.

The correlation coefficient will indicate the strength of the relationship.

Non-regulated movement

Regulated movement


23

5 APPROPRIATENESS OF THE USE OF ALL-FT FILES VS. THE USE OF SIMULATION RESULTS (TACOT)

The previously outlined indicators are herein run on data records to measure smoothing performance from an analytical perspective. However different options are available to shape the data into a purposeful observation model. They range from the static batch of “after-the-fact” flight models (ALL-FT-based)16 to the dynamic simulation of the evolution of the slot allocation plan.

The former is a fast-track (permanent availability of ALL-FT files) but partial way to measure performance, as the observation model does not strictly reflect the traffic load situation that FMP / FMD would have observed on the field.

The latter is a time-consuming solution, but it enriches considerably the performance assessment, and guarantees quality of the results since it provides the observables at different time events of the regulation life-cycle, when the FMP / FMD decides to activate, monitor, modify or cancel a regulation.

The following chapters present the results of a first set of simulations launched in order to “test” the quality of the observations obtained when using the ALL-FT model.

5.1 Relevance of a step-by-step simulation of the slot allocation plan life-cycle

Until now, our reports have presented preliminary results from the analysis of different observables, at three different statuses: “initial (FTFM)17”, “regulated (RTFM)18” and “actual (CTFM)19”. Those were obtained through the processing of “after-the fact” data, namely data extracted from CFMU “ALL-FT” files.

However, these views do not strictly reflect the traffic load situation that FMP / FMD would have observed on the field, at different time events of the regulation life-cycle, when deciding to activate, monitor, modify or cancel a regulation.

Indeed, flight plans are sent at different times by the AOs. Furthermore, slots are frozen at different times (function of the EOBT of each aircraft). Also, traffic departing from remote airports can be mixed with still on-ground traffic departing from closer sources, etc.

In particular, some doubts could be raised concerning the results provided by the analysis of the “regulated (RTFM)” vision. For instance, because of the “true revision process” (ETFMS dynamic process), slots that are not frozen yet can be improved. In addition, since the slots are frozen at different times (function of the EOBT of the aircraft), one slot that is already frozen, can be reallocated to another aircraft still on the ground, which hence benefits from the true revision process. The consequence is a risk of “counting” twice the slot that has been reallocated (aggregation of data, issued at different times).

16 “FTFM” & “RTFM” fields of the ALL_FT files 17 “Initial” reflects the status of the demand before activation of the regulation plan. It is computed with the latest flight plan version, sent by each AO to the CFMU/IFPS. 18 “Regulated” reflects the status of the demand after activation of the regulation plan. It is computed with the latest ATFM slot (CTOT) issued to the AO, by the ground regulation system. 19 “Actual” integrates the actual entry time of the flights in the regulated TV. It is computed with the Radar Data sent by ACCs to CFMU/ETFMS.


24

Therefore, the fast-time ATFM simulator – TACOT – has been used in order to rebuild the slot allocation plan observable, at different time events, and for our ATFM smoothing performance assessment model.

The model is presented in the annex 1.

The following section provides the results that were obtained so far.

5.2 Simulations results

A first set of simulations was launched in order to find out about the demand status before the implementation of a regulation, as well as to derive initial conclusions on the slot allocation process performance.

The simulation results were also confronted to results obtained with the processing of “ALL-FT” data so as to “test” the quality of the observations obtained when using that ALL-FT model.

First results: observation of the upcoming traffic load (initial demand), as seen by the user before a request [Pred-]

Observation of the drivers of the decision-making process

For one day of simulation and one TV under analysis, the flow evolution is depicted below, for the two observation models:

The one output by the simulator, at the following prediction time: just before the implementation of the regulation, herein, 113 minutes19 prior to the period start (Pred- curve): this is the demand, as figured out by the user when he or she decides to take action;

The one that is derived from the ALL-FT file (FTFM profile): this is the demand that is known “after the fact”, i.e. that integrates subsequent flight plan cancellations or later flight plan submissions.


25

It could be observed from the figure that the prediction simulated curve (Pred-) can present significant deviations from the ALL-FT (FTFM) one. It could be derived the following two trends:

An overestimation of the demand (FTFM < Pred-), for anticipation periods20 of up to 9 hours (540 minutes). For that this trend, our assumption (to be confirmed by further analyses) is that the impact of flight plans subsequent cancellations or modifications is predominant in the deviations observed between the prediction and the “after-the-fact” data;

An underestimation of the demand (FTFM > Pred-), for anticipation periods19 of more than 9 hours. For this trend, our assumption (to be confirmed by further analyses) is that a significant proportion of the flight plans has not been submitted yet, leading to the deviations observed between the prediction and the “after-the-fact” data.

As the time of events is getting closer, the two curves naturally reconcile one another.

Therefore, it could be shown from this example that the ALL-FT curve (FTFM) may deviate from the curve that the FMP would have observed on field, when deciding to create a regulation process.

In some cases, deviations are of significant amplitude, which lead to the following question: Could we estimate the appropriateness or inappropriateness of implementing ground regulations, only by the observation of the FTFM profile? In particular, a decision-making criterion which will recommend that ground regulations

20 A time index is represented on the (X axis). The origin (Index “0”) corresponds to the time where the user is taking the action of creation. Index “113” corresponds to the start of regulation period. The regulation process is therefore activated at anticipation “113” minutes, i.e. 1 h 53 min.

FTFM < Pred-

FTFM > Pred-


26

should not be activated for demand excesses lower than 10%, could be challenged if the observation is only based on the use of the ALL-FT files.

The response to this question leads to push further the investigations about the deviations between FTFM and prediction curves, and to launch an analysis for other regulation cases.

As a preliminary result from the analysis, we found in particular that for 33% of the studied sample of analysis (44 regulation cases); the FTFM curve was deviating more than 10% in average with respect to the prediction curve (Pred-).

Such result tends to confirm the issues raised by the using of “FTFM” curves only: frequent deviations of significant amplitude with what the FMP has observed on field, when deciding the regulation settings.

Second class of observables: the upcoming traffic load, as seen by the user just after a request [Pred+]

Smoothing performance of the Slot Allocation Process

Sample of analysis

They were 10 TVs under analysis, over 6 different days.

In order to get an initial overall evaluation of the ATFM regulations performance, we launched a first analysis by aggregating all the results corresponding to one specific indicator, derived from the measurement of “Traffic load fluctuations average amplitude”.

Paris CDG (4 TVs)

Madrid (1 TV)

Reims (2 TVs)

Lon. Gat. (1 TV)

Amsterd. (2 TVs)

Aggregated results (average amplitude and duration of the traffic load

fluctuations)


27

Results analysis

In the analysis of the results, two different scenarios have been considered:

1. Scenario Pred(+): evaluation of the full sample of analysis (6 days of simulation, 10 TVs), obtained with the data output by the simulator, at T+ prediction time, i.e. just after the activation of the user’s regulation request – distribution output by the system;

2. Scenario ALL_FT (RTFM): evaluation of the equivalent sample (6 same days, 10 TVs) obtained with the processing of ALL-FT data (RTFM – regulated profile), so as to measure the deviations in the results of this ALL-FT model with the results obtained by simulations.

Traffic load fluctuations average amplitude is evaluated in proportion to the flow rate (e.g. 0.05 means 5% above the flow rate, -0.05 means 5% below the flow rate).

An illustration is presented below. The curve represents the number of records (in proportion to the total number) comprised in a] x-1%; x%] interval (step of 1%).

The analysis clearly confirms that ALL-FT RTFM profiles under-estimate the performance of the system. In the ALL FT scenario, only 66% of the traffic load fluctuations average amplitude contained within an interval of ]-5%; 5%], whereas the simulation shows that 87% of the distribution output by the system is contained within that interval.

]-0.01; 0] interval: - Scenario Pred (+): 43%

]-0.01; 0] interval: - Scenario RTFM: 24%

]-0.05; 0.05] interval: Scenario Pred (+):

87%]-0.05; 0.05] interval: Scenario RTFM:

66%


28

When we strictly focus on the reference, i.e. the flow rate, the difference is equally important: 43% of the traffic load fluctuations average amplitude is contained within an interval of ]-1%; 0%] in the case of the Pred(+) scenario, 24% of the traffic load fluctuations average amplitude is contained within an interval of ]-1%; 0%] in the case of the ALL FT RTFM scenario.

It could be shown that the use of the RTFM files does not guarantee the quality of the results, in the analysis of the performance of the slot allocation process. As a matter of fact, RTFM data are altered by the real-time events occurring after T+ (as it takes into account the latest slot issued to the AO, which can be improved in function of subsequent real-time events, such as airborne deviations from the flight plan, etc.).

Therefore, performance should be better assessed through the observation of the slot allocation plan, rebuilt at T+, i.e. just after a user’s request.


29

6 CONCLUSION & NEXT STEPS

6.1 Conclusion: a segmented ATFM ground regulation performance assessment

As it has been demonstrated by the previous study and by the interviews conducted with the users of the ATFM ground regulation system, FMPs / FMDs have developed methods of using the ATFM regulation process to fulfil a number of various operational objectives that goes beyond the traditional objective of resolving the hourly demand / capacity imbalances.

ATFM service users will still look for the elimination of the hourly over-deliveries, but some will also look for increased traffic predictability, and others for the elimination of the bunching peaks. Assessing whether the system is able or not to meet these objectives therefore required to specify an appropriate ATFM performance assessment matrix of indicators.

That matrix of indicators21 has the following axes of measurements:

Traffic load fluctuations amplitude and duration, to measure whether or not the elimination of the uncontrolled loads of traffic (from an ATCO’s view point) is effective when a regulation is applied;

Average traffic load deviation from the flow rate, to measure whether or not the regulation has reshaped the initial distribution of traffic as closely as possible to the flow rate. This evaluates how the available capacity is efficiently used;

Dispersion around other references than the flow rate, to measure the performance of the system according to “customised” criteria, i.e. to the “tolerance” margin that the user has defined, while qualifying an excess as acceptable (this can be e.g. of 0%, 10% or 20% depending on the user);

Flight time estimates deviation over TV entry points, to measure the uncertainty reduction, and predictability improvement on the flight time estimates over TV entry, when the flight is regulated.

In the four above categories of indicators, the three first categories are referring to the smoothing performance of the ATFM ground regulation solution, and the fourth to the uncertainty reduction performance.

It is also relevant to notice that in the class of the smoothing indicators, the indicators are associated to different TV entry counting parameters depending on the smoothing objective that is measured: 60 minutes (hourly over-deliveries resolution), 20 minutes (traffic bunching reduction) or 1 minute (prevention from the forming of complex traffic clusters within the sectors).

In various cases, we could observe that a given regulation could be very efficient for one given objective, when providing modest results for another objective. Therefore, the regulation performance is to be assessed globally by the application of the whole set of indicators reflecting the various users’ objectives.

21 A full description of the indicators developed (incl. formulae) is available on a referenced report (Intermediate note on ATFM efficiency indicators).


30

6.2 Next steps

The previous indicators have all an operational significance, as they are representative of the added-value that the system is able to offer to the day-to-day work of ATCOs, and as a more indirect result, to the capacity that those controllers are ready to offer to the airspace users.

In particular, the present indicators set the scene for all the subsequent works that would need to be undertaken in the assessment of the ATFM performance.

In the case of traffic smoothing, we shall firstly assess the properly achievable capabilities of the slot allocation mechanism itself, by isolating the “external” source of alterations born from the reintegration of Real-Time events into the system’s functioning. In a second step, we would need to investigate into the “external” sources of alterations and assess their impact on the systems’ performance.

In the case of uncertainty, we shall extend the analysis to other TVs, in order to look at dependencies with the TV characteristics, such as the distances to the departure aerodromes.

In any case, an extrapolation of the results already obtained, from our first application of the indicators on a reduced set of cases, on a more extensive analysis of recorded regulation cases is required in order to guarantee the quality of our results.

It shall be borne in mind that this “extrapolation” will certainly require the use of simulators (TACOT, available in the EEC/NCD), in order to rebuild the slot allocation plans “life-cycle” and to make an assessment of the slot allocation plan evolution with regards to the events inherent to the functioning of the system (actions of creation, modification, cancellation of a reg. process and also the continuous adjustments of the plan triggered by the real-time events, e.g. airborne deviations to the flight plans, etc.).


31

ANNEX 1: ATFM SMOOTHING PERFO. ASSESSMENT (METHODOLOGY)

TRAFFIC DATA ANALYSIS - METHODOLOGY

The measure of the ATFM smoothing performance requires extracting and processing CFMU raw data from the archive database. A structuring model has therefore been developed in order to isolate the relevant data subsets for each regulation case.

The model is based on:

the selection of the appropriate observables for the building of a slot allocation plan observable;

an analytical model where different traffic pictures are drawn, for different statuses and/or at different key events of the slot allocation plan life cycle. The model indicates how the ATFM performance is evaluated according to these pictures and using the indicators.

1. Selection of the appropriate observables The analysis of ATFM ground regulation effectiveness requires studying the slot allocation plan “life-cycle”, i.e. for different statuses and/or at different key events, in order to make an assessment of the slot allocation plan evolution with regards to the regulation settings (flow rates, regulation periods).

In order to reconstitute the slot allocation plan that would have been observed on field and produce the results required for the analyses, we use an ATFM fast time simulator – TACOT, available in the EEC/NCD.

Note: In order to work without having to use a simulator, another option would be to build observables directly available from the processing of “after-the-fact” data, namely data extracted from CFMU “ALL-FT” files (initial22, regulated23 and actual24 flight profiles). Nevertheless, and at the present stage of our work, we have observed that the conclusions on the ATFM smoothing performance may be altered significantly by the use of the “ALL-FT” files (as described in the § 5 of this document).

22 “Initial” reflects the status of the demand before activation of the regulation plan. It is computed with the latest flight plan version, sent by each AO to the CFMU/IFPS. 23 “Regulated” reflects the status of the demand after activation of the regulation plan. It is computed with the latest ATFM slot (CTOT) issued to the AO, by the ground regulation system 24 “Actual” integrates the actual entry time of the flights in the regulated TV. It is computed with the Radar Data sent by ACCs to CFMU/ETFMS


32

• Time events:

The objective is to build the views that FMP / FMD would observe on the field, at different times events of the regulation life-cycle, when deciding to create, modify or cancel a regulation.

For each event – activation, modification or cancellation – two time events are considered:

Event T-: latest flight list version available in the system before the creation, modification or cancellation of the regulation process;

Event T+: flight list version output by the system, just after the creation, modification or cancellation of the regulation process.

The “T- view” reflects the status of the demand, at a time when a decision for regulating and for setting the regulation parameters is made by the FMP / FMD.

The “T+ view” reflects the status of the demand, output by the system and according to the regulation settings. At this time, ideally, the traffic load should be aligned with the flow rate(s) requested.

In order to identify the different time events, an analysis of regulation reports – extracted from the “Oplogs” files available within the CFMU archive database – is performed.

• Extraction of the relevant regulation observables: TACOT

In the EEC, TACOT is available for fast time ATFM simulations. TACOT integrates an implementation of the CASA slot allocation algorithm. We are able to generate, with this tool, slot lists at the different time events here above identified (T-; T+) and reconstitute the slot allocation plan, that would have been observed in operations by FMPs / FMDs.

The following data, as required for the analyses, is extracted from the simulations:

The “Flight List” status of each TV under study, and at the different times here above mentioned (T-, T+ for the processes of creation, modifications and cancellation).

The “Flight List” reports time estimates. For each flight of the TV, it gives the information related to the latest flight status, i.e. ETO25 if the flight is not regulated, eventually CTO26 if the flight is regulated and eventually ATO27 if the flight has already taken off.

The “Flight List” status of each TV under study, at the end of simulation day. Such “Flight List” is equivalent to the “CTFM” view (real flown profile) extracted from the processing of “ALL-FT” files.

25 The ETO is the expected time of penetration within the TV, when the flight is not regulated. It is obtained from flight plan sent by the AO to the CFMU. 26 The CTO is the expected time of penetration within the TV, when the flight is regulated. It is deduced from the slot (CTOT) allocated to the flight. 27 The ATO is the real time of penetration within the TV. It is obtained from radar data (CPR – Correlated Position Reports)


33

For each regulation case, these data aim to build different traffic pictures before and after an event of creation, modification or cancellation of the regulation process. The data obtained at the end of simulation day is used for the building of the real traffic pictures, as perceived by the controller on his working position.

2. Analytical model

The analytical model is granted on the processing of each “Flight List” (TACOT outputs), at the different time events here above identified: T-, T+. Each of these events triggers ATFM information updates (creation, modifications or cancellations).

Different traffic pictures are drawn before and after the time event, and compared to the regulation settings.

Slot Allocation System

(CASA)

Initial Reg. Settings (Creation)

Reg. parameters: Flow Rate, Period

Real time eventsReg. settings update

Flight Plans update (system inputs)

Real-time traffic (uncertainty)

Actual (ACT)

Pred - (T-) Pred + (T+)


34

Pred - (T-), Pred + (T+) and ACT (Actual) reflects predictions at different statuses and anticipation levels:

Pred - (T-): Flight List status before activation of a regulation process (creation / modification / cancellation);

Pred + (T+): Flight List status after activation of a regulation process (creation / modification / cancellation). It reflects the status of the demand after action of the system, according to the regulation settings;

ACT (ACT): Flight List status at the end of the day. It integrates the actual entry time of flights in the regulated TV.

The Real-time events are reintegrated into the system and are a source of alteration of the distribution initially output by the system (Pred + (T+)). Real-time events can be grouped into the following categories:

Reg. settings: The evolution of the regulation settings (subsequent modifications to the initial settings).

This category refers to all the subsequent modifications to initial reg. settings. They may have an incidence on the performance of the slot allocation system if the anticipation for the activation of such modifications is too short (system’s inertia to adapt traffic distributions to updated settings);

Flight plans: The uncertainty vis-à-vis the initial demand profile in the regulated TV.

Part of the uncertainty on the traffic loads also results from the late filing and sending of flight plans to the CFMU. Also, a significant proportion of the flight plans is subjected to subsequent modifications and cancellations, leading to modifications of the initial demand profile.

Real-time traffic: The real-time deviations from the flight plan (also evaluated through the analysis of flight time estimates on TV entry deviations, as detailed in the “uncertainty reduction” sections of this document).

The category refers to the operational deviations from the flight plan, inherent to the real-time execution of flights (pilot / ATC instructions).

Documents

025 ATFCM Performance and Declared Capacity