EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION
EUROCONTROL EXPERIMENTAL CENTRE
Gaze analysis and psychophysiological parameters: A tool for the design and the evaluation of Man-Machine Interfaces
Feasibility study
EEC Report N° 350
Project HRS-Z-05_PHYS
Issued: July 2000
The information contained in this document is the property of the EUROCONTROL Agency and no part shouldbe reproduced in any form without the Agency’s permission.
The views expressed herein do not necessarily reflect the official views or policy of the Agency.
EUROCONTROL
REPORT DOCUMENTATION PAGE
Reference:EEC Report N° 350
Security Classification:Unclassified
Originator:
EEC - IND
(INDependent Reseach)
Originator (Corporate Author) Name/Location:EUROCONTROL Experimental CentreCentre des Bois des BordesB.P.15F - 91222 Brétigny-sur-Orge CEDEXFRANCETelephone : +33 (0)1 69 88 75 00
Sponsor: Sponsor (Contract Authority) Name/Location:EUROCONTROL AgencyRue de la Fusée, 96B -1130 BRUXELLESTelephone : +32 (0)2 729 90 11
TITLE: GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS : A TOOL FOR THE DESIGNAND THE EVALUATION OF MAN-MACHINE INTERFACES
FEASIBILITY STUDY
Authors P. Cabon, B. Farbos
R. MollardLAA, Université R. Descartes Paris V
Date
07/2000
Pages
x + 55
Figures/Photos
26 / 7
Tables
1
Annex
1
References
19
EATCHIP TaskSpecification
-
Project
HRS-Z-05_PHYS
Task No. Sponsor
-
Period
1999
Distribution Statement:(a) Controlled by: Head of IND(b) Special Limitations None(c) Copy to NTIS: Yes
Descriptors (keywords):
ATC, human factors, HMI, gaze analysis, perceptual aspect, information processing, EEG, HRV, workload,stress, fatigue.
Abstract:This study demonstrates the value of gaze analysis with simultaneous physiological data recording for theassessment of the human-computer interface in air traffic control.The literature on visual scanning was reviewed. Two studies using different eye-tracking systems, one head-mounted (I VIEW) and one remote (ASL 504) were conducted at EEC. In each study, a TRACON IIsimulator was used to present controllers with a representative ATC task and a set of relevant electro-physiological measures (EEG, HRV etc..) was employed.This study shows that selected eye-view and physiological parameters may be used together to evaluatestrain, fatigue, operational strategies of controllers in highly interactive situations.Further studies should investigate the effect of changes in the nature and extent of automation of the ATCtask, such electronic stripping, flat displays etc.
This document has been collated by mechanical means. Should there be missing pages, please report to:
EUROCONTROL Experimental CentrePublications Office
Centre des Bois des BordesB.P. 15
91222 - BRETIGNY-SUR-ORGE CEDEXFrance
v
GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS :
A TOOL FOR THE DESIGN AND THE EVALUATION OF MAN-MACHINE INTERFACES
FEASIBILITY STUDY
By
P. Cabon, B. Farbos, R. Mollard LAA Université René Descartes - Paris V
SUMMARY
This study demonstrates the value of gaze analysis with simultaneous physiological datarecording for the assessment of the human- computer interface in air traffic control.
The literature on visual scanning was reviewed. Two studies using different eye-trackingsystems, one head-mounted (I VIEW) and one remote (ASL 504) were conducted at EEC.In each study, a TRACON II simulator was used to present controllers with a representativeATC task and a set of relevant electro-physiological measures (EEG, HRV etc…) wasemployed.
This study shows that selected eye-view and physiological parameters may be used togetherto evaluate strain, fatigue, operational strategies of controllers in highly interactive situations.
Further studies should investigate the effect of changes in the nature and extent of automationof ATC task, such electronc stripping, flat displays etc.. .
vi
vii
FOREWORD
The need for more ‘objective’ measures of the effects of Air Traffic Control on controllers haslong been apparent. Various studies have been undertaken at EEC on potential methods ofmeasuring these effects (EEC Reports n° 64, 164, 183, 226, 228, 323, 334 and 339).
The LAA (Laboratoire d’Anthropologie Appliquée, Université Paris V René Descartes) underthe direction of Professor Alex Coblentz, has carried out studies of fatigue, sleep disorder andassociated strain among Airline Pilots, Car Drivers and Air Traffic Controllers among others.
We therefore commissioned the LAA to undertake a series of feasibility studies on the use ofCortisol analysis, Electroencephalography, in particular Cortical Evoked Potential and otherrelated methods, which were reported in EEC Report n° 323 and in EEC Note n° 16/98. Thesestudies used a TRACON II autonomous simulator. LAA subsequently undertook the transferof these methods to real-time simulation, as reported in EEC Reports n° 334 and 339.
This report covers further TRACON II studies of eye movement recording methods, operatedin association with electrophysiological studies.
On behalf of the Director EEC, the experimenters for the LAA and the EEC Project Officerswish to thank the controllers and the EEC staff involved for their active and dedicatedco-operation.
Françoise CalooHugh David
EEC Project Officers.
viii
ix
TABLE OF CONTENTS
ABBREVIATIONS............................................................................................................................................... X
1 - INTRODUCTION AND AIMS - .................................................................................................................... 3
2 - LITERATURE REVIEW - ............................................................................................................................. 5
2.1 - THE EYE TRACKING TECHNIQUE -................................................................................................................ 52.2 - DATA PROCESSING METHODS - .................................................................................................................... 6
2.2.1 - Fixation chronology methods - ........................................................................................................... 62.2.2 - Statistical analysis of gaze in a specific area - ................................................................................... 82.2.3 - Analysis of fixation groups - ............................................................................................................... 92.2.4 Weighted Search Area (WSA) ............................................................................................................. 10
2.3 - APPLICATIONS OF VISUAL SCANNING IN ERGONOMICS - ............................................................................ 10
3 - SYNTHESIS OF THE IVIEW STUDY - ..................................................................................................... 14
4 - ASL STUDY - ................................................................................................................................................. 16
4.1 - AIMS - ....................................................................................................................................................... 164.2 - METHOD - ................................................................................................................................................. 16
4.2.1 - Controllers and sample - .................................................................................................................. 164.2.2 – Eye-tracking device description - .................................................................................................... 164.2.3 - Data collected - ................................................................................................................................ 18
4.3. - RESULTS - ................................................................................................................................................ 204.3.1 - Fixation frequency and duration - .................................................................................................... 204.3.2 - Physiological data -.......................................................................................................................... 214.3.3 - Controllers’ orders -......................................................................................................................... 224.3.4 - Perceived workload -........................................................................................................................ 23
5 - ERRORS AND VISUAL SCANNING ANALYZIS - ................................................................................. 24
5. 1 - THE IVIEW SYSTEM - .............................................................................................................................. 255.1.1 - Analyzis of errors - 5 minutes -......................................................................................................... 255.1.2 - Analyzis of errors - 2 minutes -......................................................................................................... 27
5.2 - THE ASL SYSTEM - ................................................................................................................................... 305.2.1 - Analyzis of errors - 5 minutes -......................................................................................................... 305.2.2 - Analyzis of errors - Weighted Search Area method -........................................................................ 32
6 - DISCUSSION AND CONCLUSION - ......................................................................................................... 41
7 - REFERENCES -............................................................................................................................................. 43
ANNEX................................................................................................................................................................. 45
FRENCH TRANLATION .................................................................................................................................. 49
Green pages: French translation of the foreword, the summary, the introduction, objectives, discussion and conclusion.Pages vertes : Traduction en langue française de l'avant-propos, du résumé, de l’introduction, des objectifs, de la discussion
et conclusion.
x
ABBREVIATIONS
ASL Applied Science Laboratories
Astrips Active Strips
ATC Air Traffic Controller
CRT Cathodic Ray Tube
CTTT Controllers Transition Trials Tools
EEC Eurocontrol Experimental Centre
EEG Electroencephalogram
EUROCONTROL European Organisation for the Safety of Air Navigation
HMI Human Machine Interface
HRV Heart Rate Variability
IVIEW System Video-based contact-free evaluation of gaze position
KeyB Key Board
LAA Laboratoire d'Anthropologie Appliquée
NAC Nippon Automatic Corporation
NASA National Aviation and Space Agency
NASA TLX Nasa Task Load Index
NOLDUS Noldus Information Technology Inc
OBSERVER video pro Tool for the study of human behavior
POPUP Window displayed in response to behavior mouse click
Pstrips Pending Strips
SMI Senso Motoric Instruments
TRACON Terminal Radar Approach Control ( Simulator )
WSA Weighted Search Area
1
A N T H R O P O L O G I E A P P L I Q U E E
45, rue des Saints-Pères 75270 PARIS Cedex 06Téléphone : 01 42 86 20 37- 01 42 86 20 41 -Télécopie : 01 42 61 53 80
E.mail : [email protected]
* * *
GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS :
A TOOL FOR THE DESIGN AND THE EVALUATION OF MAN-MACHINE
INTERFACES FEASIBILITY STUDY
* * *
DOC AA 407/00 June 2000
2
This study was conducted with the fruitful collaboration of Mrs Françoise Caloo and MrHugh David.The laboratory expressed their grateful acknowledgements to the Air Traffic Controllers whowere actively involved in these experiments :
Ms Andy BarffFranck DowlingPeter EriksenRoger GerreauPaul HaworthYann KermarquerBernard KerstenneFrançois VergneAntoine VidalJean Paul Zabka
3
1 - INTRODUCTION AND AIMS -
The introduction of new technologies in the field of Air Traffic Control leads to
deep changes in the Controllers’ working procedures and tools. Traditional radar
displays and paper strips are progressively being replaced by cathodic displays,
electronic strips, a mouse and a keyboard. These changes can affect the performance of
Controllers in different ways and it is crucial to ensure that these new tools are capable
of helping Controllers in managing traffic. In this context, the previous studies
conducted at the Eurocontrol Experimental Centre were intended to validate a
methodology to address some important issues for the development of these new tools.
This methodology was based on physiological and subjective data concerning workload,
stress and fatigue.
The present study aims to complete these evaluations with gaze analysis. In fact,
visual scanning is one of the first steps of information processing. However, a basic
distinction should be made between “looking” and “seeing”. Considering that one can
“see” without “looking” gaze analysis itself is not sufficient for the evaluation of
human-machine interfaces. This is the reason why it is necessary to develop a
comprehensible approach that should allow to take into account some perceptual
aspects as well as the information processing of Controllers. Therefore, gaze analysis
and physiological data can maximise the evaluation both in terms of the impact of a new
device on the Controller’s state as well as for the design of new interfaces. In particular,
the integration of new tools using windows makes the use of large displays
(2000X2000) necessary. Thus, the way information is presented to Controllers is
changing. Because of these changes, the Controllers’ visual strategies are changing
drastically. For example, depending on its position on the display some information
could be neglected. It is thus useful to get some objective evaluations of the impact of
these new technologies on Controllers.
The main aim of the feasibility study was to evaluate the relevance of a
methodology using both gaze analysis and physiological data to assess interfaces. With
this aim in view, simple simulations using the TRACON software were presented to 10
Controllers. TRACON has already been used in previous studies (Cabon et al, 1997) for
the investigation of several issues such as the use of psychophysiological measures for
the evaluation of workload, learning process, or stress… Brookings et al (1996) also
used it on a study on workload. There were two main advantages to using TRACON for
the purpose of this feasibility study : it is easy to learn for Controllers and various
scenarios can be setup. Obviously, the simulations played with TRACON cannot be
4
considered either as reality or as the complex real-time simulations played at
EUROCONTROL.
However, as the aim of this study is not to provide some results on TRACON itself
but on the physiological measures and gaze data, TRACON appears to be very
adapted to this context. Moreover, in previous studies Controllers felt the simulations to
be realistic regarding the normal control activities and they were highly motivated in
using the software.
After a literature review on the topic of visual scanning, this report presents a synthesis
of the two feasibility studies conducted at the Eurocontrol Experimental Centre (EEC).
In these two studies, two eye tracking devices were used. In the first study, a head-
mounted device (IVIEW) was used. Its main results are summarised. In the second
study, a remote device was tested (ASL 504). Thanks to technical improvements, the
ASL device has allowed a very precise analysis of errors. The methodology and the
main results are presented.
5
2 - LITERATURE REVIEW -
The aim of this review is to summarize:
- the eye tracking technique,
- the data processing methods,
- the main applications of this method in ergonomics, in particular in aeronautics.
2.1 - The eye tracking technique -
The eye tracking use the following components (figure 1):
- a video camera and an infrared light that tracks the eye movements (eye system),
- a video camera recording the scene (scene camera),
- a real-time data processing system calculating the gaze direction.
Figure 1
Description of the principle for the acquisition of gaze
(adapted from Rousseau et coll., 1994)
Scenecamera
Eye ofsubject
Processingsystem of
data
Scenepicture
Eye camera
Infrared
Eye picture
6
This device displays the location of gaze points on the scene observed by the subject.
The device can be either head-mounted (e.g. IVIEW) or remote (e.g. ASL 504).
Depending on the device used, a head movement’s compensation is achieved. This
compensation permits to process automatically the gaze co-ordinates.
2.2 - Data processing methods -
Main data obtained from the eye tracking are the number and the duration of
fixations on specified area of the visual field. Depending on objectives, 4 classes of data
processing methods can be distinguished:
- fixation chronology,
- statistical analysis of gaze in a specific area,
- analysis of fixation groups,
- weighted search area.
2.2.1 - Fixation chronology methods -
These methods are based on the analysis of fixation sequences. These sequences
can be presented in two ways:
- by indicating the fixation points routes (figure 2),
- by indicating the fixation points succession in a specified area of the visual field
(figure 3).
The first method provides a spatial information whereas the latter gives a
temporal information.
7
Figure 2
Illustration of gaze point during one session
(adapted from Menu et al., 1994)
Figure 3
Fixation time in one selected area during a reading task
(adapted from Menu et al., 1994)
Y c
oord
inat
es
Time of session (s)
X coordinates
Fixa
tion
time
(ms)
Time of session (s)
8
2.2.2 - Statistical analysis of gaze in a specific area -
These methods are based on the statistical analysis of the fixation distribution in a
specific area (figure 4). This method allows an hierarchical organization of the various
areas in terms of information search.
Figure 4
Description of fixation frequency for three sectors defined from driving simulation
(adapted from Laya, 1994)
0
20
40
60
80
100
Left In front Right
Freq
uenc
y fi
xatio
n (s
)
9
2.2.3 - Analysis of fixation groups -
Two kinds of methods can be distinguished:
- Modelling of transition probability from one area to another (Papin, 1981)
- Factor analysis (Rousseau et al, 1994 (figure 5) which allows the separation of
some sectors which are statistically different and the evaluation of the chronology
of fixations and the quantity of fixations.
The width of the arrows indicate the number of fixation sequence
Figure 5
Illustration of factor analysis realized from sequence of fixations on a car
(adapted from Rousseau et al, 1994)
Sect
or 2
Sector 3
Sector 4Sector 4
Sector 1
sect
or 5
0
20
40
60
80
100
1 2 3 4 5
Sector
Num
ber
of f
ixat
ion
sequ
ence
Sector
10
2.2.4 Weighted Search Area (WSA)
The WSA developped by Chia and Fang-Tsan (1997) is based on the integration
of fixation frequency and duration. It is based on the description by vectors of visual
fixations and on assignment of weight to each vector by its respective fixation time.
Hence, the aim of WSA is to describe concentrations of visual strategies and potentially
quantify the amount of visual load (figure 6). This method was applied in our study of
error analysis (section 5.2.2).
Figure 6
Illustration of Weighted Search Area method (WSA) - This method is based on the
description by vectors of frequency and time of fixations
(adapted from Chia and Tsang, 1997)
2.3 - Applications of visual scanning in ergonomics -
For the operator, visual scanning consists of directing his gaze toward
informative areas. The analysis of these different fixation areas permits the evaluation
of the strategy used by the operator in taking the information and of the efficiency of
this strategy regarding his activity.
This efficiency is influenced by various factors. Drury and Sinclair (1983)
showed that visual scanning is influenced by the task difficulty. In that study, task
difficulty was mainly related to the size, the contrast and the variety of the objects.
11
The temporal organization, i.e. the duration and the rhythm of an activity is also
an important factor which influences the visual scanning pattern and the performance
(Moraal, 1974). Togami (1984) showed that when the activity rhythm is high, the
fixation number and duration increase. They also observed that the increase of the
fixation duration is more efficient to improve performance than the increase of the
fixation number.
In addition, Saito (1972) showed that the higher is the temporal demand, the less
the visual scanning is large (i.e., less objects are inspected). Simon et al (1993) observed
that the visual scanning is more structured when the workload increases. These results
are confirmed by Tole et al (1982) on aircraft pilots. When the pilot has to cope with a
high workload, he has less time to look at various instruments and therefore adopt a less
random visual scanning strategy. Other studies conducted in simulators found the same
results (Krebs et al, 1977, Harris et al, 1986).
These results are consistent with the conclusions of Sperandio (1991). Operators
have to adapt their strategy to cope with the increase of task demand by using a more
“economic” visual strategy to optimize the information taking.
In the field of aeronautics, visual scanning behaviour was shown to be very useful
for the interface design. (Table 1) summarized some recent studies conducted in this
field.
12
AUTHORS OBJECTIVE SAMPLE DATA
COLLECTIONRESULTS
Bellendes et al.(1997)
Examination of differences inattention control betweenexperienced and novice pilots frommeasurement of visual scanning
- Twelve students pilots
- Twelve Flight instructors
- Flight simulator
- Number of fixations- Mean dwell time
- Novices tended to dwell for a longer time aneach instrument
- Information is extracted more efficiently byexperts from nearly all instruments anparticularly from the most information withinstrument
Itoh (1990) Examination of the differences inscanning behaviour under conditionsof high mental workload betweenpilots using and integrated displayand pilots using the earlier electro-mechanical display
- Five experimented pilots - Number of fixations- Fixation time
- The proportion of fixation duration increases inresponse to the increase of mental workload
- In terms of the ease of obtaining informationfrom instruments the integrated display usingthe CRT might be more preferable for pilotstask
Tole et al.(1982)
Examination of the relationshipbetween pilot visual scan ofinstruments and mental workload
- Six subjects varying in skill-level from non pilot to ahighly experienced NASAtest pilot
- Flight simulation
- Time course of eyefixations
- Dwell time of eachfixation
- Scanning behaviour may serve as an indicatorof both workload and skill.
-As a pilot’s skills develop, his visual scanningbehaviour is less as less affected by non visualincrements in workload.
- Pilots with a high experienced skill present adecrease of the percentage of long dwells.
David(1977)
Provide a detail description of threemethods as means for measuring theController’s eye movement and thenecessary : direct observation, videotranscription and eye tracker system
- Two teams consisted foreach by 6 Controllers
- Video- Number and durationof fixation
- Eye mark recording (NAC Mark V) is precisebut intrusive.
- Video-recording requires about four times thelength of observation: it may be biased byomission.
- Direct observation is very "cheap" but requiresfour times also the length of observation: insomme circumstances it may not be acceptableto the Controller.
Table 1
13
For Air Traffic Control, the understanding of controllers visual search behaviour
becomes more and more important with the prevalent use of computer screens and other
visual display devices. In fact, the evaluation of controllers’ visual scanning patterns could
help to improve the design of these new interfaces.
In 1975, Karsten et al. found already that controllers spend approximately 80% of
their time looking at the radar display, 13% at flight strips and 5% at input devices.
However, in a real time simulation, David (1977) showed that the number of fixations
directed at various devices were not necessarily indications of the relative efficiencies of
displays. This observation seems to be related to the system limitation (NAC). New
generation eyes tracking systems (e.g. ASL) now overcome these limitations
More recently, the research of Stein (1987), showed also from a real time ATC
simulation that the longest fixation duration’s were observed for aircraft compared to other
objects on the radarscope. For the authors, the inclusion of other objects and scene planes
could drastically reduce the average duration of the fixations on aircraft.
However, only a few studies have associated gaze data and others objectives or
subjectives data to evaluate the information processing related to perception.
Itoh (1992) observed, in pilots, with the measure of heart rate variability, no
difference in mental workload between pilots using integrated CRT displays (Boeing 767)
and pilots using earlier electromechanical displays (Boeing 747-300) (table 1). Regarding
the visual scanning, they noticed that when the workload increased, the fixation duration
for certain instruments increased. In addition, they observed a significant difference of
visual scanning between the two interfaces during abnormal situation (e.g. taking off and
landing with engine failures). In fact, the changes in scanning pattern for B767 were found
to be smaller than those of the B747 pilots. Based on these results, the authors concluded
that the application of CRT helps pilots to obtain sufficient information more easily than
electromechanical displays do, even under abnormal situations.
Only one study has used both psychophysiological and visual scanning
measurements in Air Traffic Controller (Brookings et al, 1996). Although, the Brookings’
study did not evaluated the fixation duration and number of fixations but the vertical blink
and horizontal saccade activity, their results indicated that « as the visual demands
increased, the controllers blinked less often so as not to miss important events on the radar
scope ». For the author, this interpretation is supported by other investigations reporting
decreased blink rates when operators were under conditions of increased visual load.
14
3 - SYNTHESIS OF THE IVIEW STUDY -
The first step of this study was intended to evaluate the interest of using eye tracking
technique and physiological data for the evaluation of various aspects in the context of
interface design (photos 1 and 2). In the context of this study, it was decided to test the
following hypothesis: the use of different methods for entering data, the keyboard or the
trackball, changes the visual scanning of Controllers as well as their attention and
workload. In fact, it is assumed that the use of the keyboard forced the Controller to look
frequently at the keyboard and, consequently, reduced his visual inspection on the radar
and his attention towards the information displayed. The experimental protocol allowed to
compare 4 conditions combining the use of a trackball and a keyboard and 2 levels of
traffic:
- keyboard and low level of traffic,
- keyboard and high level of traffic
- trackball and low level of traffic,
- trackball and high level of traffic.
Photo 1
15
Photo 2
Eight subjects were involved in this study. The data collected were: fixation time
and frequency on various items of the visual field, Heart Rate Variability for mental effort
evaluation, the electroencephalogram (EEG) for the assessment of attention, the NASA-
TLX for perceived workload and simulator data for the Controllers’action and
performance.
The comparison between the Keyboard conditions and the Trackball conditions
shows that:
- changes in fixations are more frequent in the keyboard conditions but fixations are
longer in the Trackball conditions,
- performance is better in the Trackball conditions,
- workload is not affected,
- attention is higher in the Trackball conditions.
The comparison of the two levels of traffic indicates that:
- gaze frequency and duration are not affected,
- workload is higher in the 30-aircraft condition,
- handover errors are increased with traffic.
Therefore this study showed that the use of both gaze data and physiological
measures are able to evaluate the impact of different interfaces.
16
4 - ASL STUDY -
4.1 - Aims -
The aim of this second study is twofold:
- to examine the advantages of the ASL 504 device compared to IVIEW,
- to study the interest of the data collected as regards error analysis.
4.2 - Method -
4.2.1 - Controllers and sample -
Four Controllers were involved in this study. Among these 4 controllers, 2 had
already participated in the previous IVIEW study. Each Controller was presented with 2
simulations on the TRACON. The two Controllers who were not involved in the previous
study were presented a traffic of 15 aircraft in 30 minutes (session 1) and then a traffic of
30 aircraft in 30 minutes (session 2). As they were already familiar with TRACON, the 2
other Controllers were presented only with two sessions with 30 aircraft. For this study,
only the trackball was used by the Controllers as the previous study had demonstrated that
the graphical interface (trackball) was better than the numerical interface (keyboard
condition).
4.2.2 – Eye-tracking device description -
The eye-tracking device used in this study is the ASL 504. Contrary to IVIEW, the
ASL 504 is not head-mounted but totally remote (photos 3 and 4). It is composed of an
eye camera located beside the TRACON display. An infrared light, emitted by the camera,
is reflected on the controllers’cornea. The only thing that the controller wears on his
forehead is a transmitter that gives his head’s position. This information is sent to an aerial
located behind the Controller’s head and processed by a computer that allows to the
camera to track the eyes thanks to a motor.
17
Photo 3
Photo 4
18
Therefore, this device takes into account all the controllers’ eyes movements (photo 5).
The method to determine the gaze position is the same as for IVIEW. Compared to IVIEW,
one of the main advantages of ASL is that it compensates the head movements and
therefore provides for each gaze its co-ordinates relative to the display. The TRACON
display was filmed using a video camera during all the sessions.
Photo 5
4.2.3 - Data collected -
The data collected were the same as those collected in the previous study:
- gaze: fixation frequency and duration,
- physiological: Heart Rate Variability (HRV) and the electroencephalogram (EEG),
- simulator data: Controllers’ actions and errors,
- subjective: workload (NASA-TLX).
19
Contrary to the IVEW study, the ASL 504 allowed to process the data automatically.
In the IVIEW study, the fixation points were obtained by reviewing the video recording at
a lower speed (5 times less than the normal speed) and by manually coding the gaze
position using the OBSERVER (photo 6). In the present study, the OBSERVER was used
to collect the Controllers’ actions (photo 7).
Photo 6
Photo 7
20
4.3. - Results -
The results were analyzed in two ways:
- a global analysis in which the two levels of traffic were compared,
- an analysis focused on the errors.
4.3.1 - Fixation frequency and duration -
Figures 7 and 8 show the fixation frequency and duration on the menu, the radar,
the strips and the communication window for the two levels of traffic. It can be observed
that for all the items, the fixations frequency and duration are more frequent in 15-aircraft
condition. However, the fixations duration on the radar are longer in the 30-aircraft
condition. Therefore, the Controllers changed their fixation more in the lowest traffic
condition and spent less time looking at the radar. This result can be explained by the fact
that in the lowest traffic condition, the Controllers used were those with the least
experience in TRACON while the 30-aircraft condition included the 2 controllers who had
been involved in the first study. This also related to the fact that the traffic increase the
controller tend to focus to only a few important elements of the display.
Figure 7 – Fixation frequency as a function of items and traffic levels.
0
2
4
6
8
10
12
14
16
Menu Radar Strips Com Off
Traffic level : 15 aircraft (n = 2 ATC)
Traffic level : 30 aircraft (n = 4 ATC)
Fix
atio
n (m
n)
21
Figure 8. Fixation duration as a function of items and traffic levels.
4.3.2 - Physiological data -
HRV and EEG results are presented in figure 9. The results show that the power in
the middle frequency is increased (i.e. the effort is increased) in the 30-aircraft condition.
EEG data indicate also an increase of attention in the 30 aircraft condition for the 4
Controllers.
Figure 9. Power in the theta frequency band of the EEG and in the middle frequency of the HeartRate Variability as a function of traffic.
0
10
20
30
40
50
60
Menu Radar Strips Com Off
Traffic level : 15 aircraft (n = 2 ATC)
Traffic level : 30 aircraft (n = 4 ATC)
150
160
170
180
190
200
210
220
230
T15 T300
10
20
30
40
50
60
70
80
90
100
T15 T30
Pow
er in
the
mid
dle
freq
uenc
y (m
n)
Pow
er in
the
The
ta b
and
(mn)
Fix
atio
n du
rati
on (
%/m
n)
22
4.3.3 - Controllers’ orders -
The number of Controllers’ orders divided by the number of aircraft (figure 10)
show that Controllers changed the aircraft heading more than their speed or altitude. This
result is consistent with the results obtained in the previous study. It can also be observed
that Controllers provide more orders per aircraft at the highest level of traffic in order to
cope with the task difficulty.
Figure 10. Frequency of controllers’ orders as a function of traffic
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
Altitude Speed Heading
Traffic level : 15 aircraft (n = 2 ATC)
Traffic level : 30 aircraft (n = 4 ATC)
Con
trol
lers
ord
ers
(mn)
23
4.3.4 - Perceived workload -
Figure 11 shows the results of the workload assessment after the two sessions for
the Controllers who were familiar and for the Controllers who were not familiar with
TRACON. In the first sessions, it can be observed that perceived workload is higher for
Controllers who were not familiar although they were only presented with 15 aircraft. In
the second session, the less experienced Controllers perceived a higher workload but with
less differences than in the first session.
Figure 11Assessment workload as a function of traffic.
Mentaldemand
Temporaldemand
Performance Effort Frustrationlevel
Physicaldemand
Mentaldemand
Temporaldemand
Performance Effort Frustrationlevel
Physicaldemand
Arb
itra
ry U
nits
Arb
itra
ry U
nits
100
80
60
40
20
0
100
80
60
40
20
0
Not experienced / 30 aircraft Experienced / 30 aircraft
Not experienced / 15 aircraft Experienced / 30 aircraft
24
5 - ERRORS AND VISUAL SCANNING ANALYZIS -
Three kinds of situations were analysed with the IVIEW and ASL systems:
separation conflicts, missed approaches and handover errors. The number of these errors
was determined with the TRACON software, at the end of each session. The time of errors
occurrence was determined with video recordings. Finally, each error’ analysis was
realised by taking into account gaze movements, physiological data and the conditions of
each session.
For the IVIEW system, the analyses were carried out in two ways:
- a mean was calculated for each set of data 5 minutes before Controller’ errors,
- data were processed per second, 2 minutes before errors.
With the ASL system, two kinds of analyses were also achieved:
- gaze movements were automatically analysed for 16 areas of the radar screen, 5
minutes before errors (figures 12 and 13),
- the application of the Chia and Fang-Tsan’s method: the Weighted Search Area.
Figure 12
Description of the areas analyzed with the Iview system
Radar
Communication
Strips
25
Figure 13
Description of the areas analyzed with the ASL system
5. 1 - The IVIEW system -
5.1.1 - Analyzis of errors - 5 minutes -
Five minutes before an error, the visual scanning of each Controller was determined
from this fixation frequency. The fixation frequency was calculated for the following
items: Radar, Keyboard, Astrips, Pstrips, Popup and Others (figure 14).
No significant difference was observed per item and error. Moreover, no effect of
the trackball-keyboard and traffic level conditions was observed on missed approach,
handover and separation conflict errors.
Nevertheless, two results can be noticed:
- the number of fixation decreased on the Keyboard, before a missed approach,
- the fixation frequency increased for the Astrips before a separation conflict error.
Radar
Communication
Strips
Radar Radar Radar
Radar Radar Radar Radar
Radar Radar Radar Radar
RadarRadarRadarRadar
26
Figure 14
Mean fixation frequency 5 minutes before missed approach, handover and
separation conflict errors
- Iview system -
0
50
100
150
200
250
Radar Keyb Astrips Pstrips Popup Others
Fix
atio
n fr
eque
ncy
Missed approach Handover Separation conflict
27
5.1.2 - Analyzis of errors - 2 minutes -
One subject’s duration and frequency of fixation, 2 minutes before a separation
conflict is presented on figure 15. Data obtained from these two gaze parameters were
processed per second.
Figure n 15
Description of duration and frequency fixation and the HRV of one Controller 2 minutes before a
separation conflict – Iview system -
00:00 00:15 00:30 00:45 01:00 01:15 01:30 01:45 02:00
popup
radar
pstrips
astrips
keyb
mouse
others
Time
-150-100
-500
50
100150
Var
iati
on o
f H
RV
(%
)
28
This data processing allowed to compare simultaneously the visual scanning of a
Controller and his workload determined by HRV.
No particular visual scanning was observed 2 minutes before the separation conflict.
Generally, when the workload was increasing we observed that duration of fixation was
increasing. The same trend was observed for the other kinds of errors: missed approach
(figure 16) and handover errors (figure 17).
Time before error
Figure 16
Description of the visual scanning and evolution of HRV of Controller n°3
before a missed approach - Iview system -
-150-100
-500
50
100150
Var
iati
on o
f H
RV
(%
)
p o p u p
r a d a r
p s t r ip s
a s t r ip s
k e y b
m o u s e
o t h e r s
02:00 01:45 01:30 01:15 01:00 00:45 00:30 00:15 00:00
29
Time before error
Figure 17
Description of the visual scanning and evolution of HRV of Controller n°4
before a handover error - Iview system -
popup
radar
pstrips
astrips
keyb
mouse
others
-200
-100
0
100
200
Var
iati
on o
f H
RV
(%
)
02:00 01:45 01:30 01:15 01:00 00:45 00:30 00:15 00:00
30
Moreover, the data showed a great inter-individual variability of visual scanning
before the occurrence of an error. Two hypotheses could explain this variability:
- every Controller will present different visual scanning patterns whatever the kind of
error,
- IVIEW did not allow to determine one particular visual scanning. Hence, it was
impossible to use the IVIEW’s data to determine sectors of the radar covered by the
Controller’s visual scanning. In fact, data processing with IVIEW system is extremely
time-consuming.
In conclusion, the IVIEW system did not allow a description of visual search pattern
before error. These results were associated with a limitation of the IVIEW system.
Nevertheless, the IVIEW data suggest a relation between workload and gaze patterns: the
higher the workload, the longer the fixation time. This increase in workload could be due
to an increase in visual load.
5.2 - The ASL system -
With the ASL system, the analysis of errors and visual scanning of Controllers was
faster than with the IVIEW system. Because ASL allows to compensate the head
movements an automatic processing was possible.
While the TRACON’s screen was divided into three big sectors with the IVIEW
system (cf figure 12) the ASL system allowed a more detailed processing. Therefore, the
radar was divided into 16 sectors (cf figure 13).
5.2.1 - Analyzis of errors - 5 minutes -
Figures 18 and 19 show the evolution of duration and number of fixation of one
Controller (n°9), 5 minutes before a separation conflict.
This presentation of results shows that the gaze zone is localized mainly into radar
sectors 4, 7, 8, 11 and 12. These zones correspond to pending strips and to an airport being
often associated with many missed approaches.
It can also be observed that before this conflict, the duration and the number of
fixation were high: their increase suggests that a Controller’s gaze is better focussed on this
area.
31
Therefore, this method provides a global and a quantitative information on the
Controller’s gaze on a period. However, it does not permit a detailed analyzis of visual
pattern just before the error as regards the place where the error occurred.
Figure 18
Fixation duration over the 16 radar sectors 5 minutes before separation conflict
Controllers n°9 - ASL system -
Figure 19
Fixation number over the 16 radar sectors 5 minutes before separation conflict
Controllers n°9 - ASL system -
Fix
atio
n n
um
ber
0
10
20
30
40
50
60
70
Radar sectors
Radar 1
Radar 2
Radar 3
Radar 4
Radar 5
Radar 6
Radar 7
Radar 8
Radar 9
Radar10
Radar 11
Radar 12
Radar 16
Radar 15
Radar 14
Radar 1313
1415
16
4
8
12
Radar 1
Radar 2
Radar 3
Radar 4
Radar 5
Radar 6
Radar 7
Radar 8
Radar 9
Radar10
Radar 11
Radar 12
Radar 16
Radar 15
Radar 14
Radar 130
10
20
30
40
50
60
70
80
90
Radar sectors
1314
1516
4
8
12
32
5.2.2 - Analyzis of errors - Weighted Search Area method -
In order to determine the evolution of duration and frequency of fixation before the
occurrence of an error, a new method reflecting the visual area covered by a subject was
applied.
This method was developed by (Chia-Fen and Fang-Tsan, 1997) and named
Weighted Search Area (WSA). Its principle is to determine visual fixations by vectors on
an X-Y co-ordinate system. Vectors are hence assigned by a weight from theirs fixation
duration. Through various procedures of adding vectors and rotating axes, the WSA and
overall fixation vector can be calculated. This methodology is used as a quantitative
measure for visual load where a large amount of visual attention is necessary.
Figures 20 and 21 show examples of the WSA graph calculated for 2 subjects on
the whole of their session. The description of the totality of gaze points is also proposed
simultaneously.
Figure 22 shows for the same Controller the WSA graph 15, 10 and 5 minutes
before a separation conflict. The results show a progressive decrease of Controller’s visual
scanning 10 and 5 minutes before error.
In addition, five minutes before this error, WSAs per minute show that a Controller
does not look at the area where the conflict occurred (figure 23). One minute before the
error the WSA showed that the gaze of Controller n°9 was focussed on a zone, outside the
conflict zone.
The same result was obtained for another Controller who made the same error
(figures 24 and 25).
At the opposite, during a missed approach, the use of the WSA method showed that
the Controller looked at the zone where the missed approach occurred (figure 26).
Therefore, these results suggest that some errors are due to visual perception
whereas others are related to the information processing. However, this errors classification
could be better analyzed with the confrontation and interviews of the Controller with his
gaze pattern after the simulation.
33
Gaze points Gaze points and associated WSA
Figure 20
Description of the gaze points and associated-WSA for Controllers n°9 on the whole session (30 aircraft per 30 minutes)
- ASL system -
34
Gaze points Gaze points and associated WSA
Figure 21
Description of the gaze points and associated-WSA for Controllers 11 on the whole session (30 aircraft per 30 minutes)
- ASL system -
35
Figure 22
Description of the WSA 15, 10 and 5 minutes before separation conflict
- Controllers 9, condition 30 aircraft per 30 minutes - ASL system
15 minutes before error10 minutes before error5 minutes before error
36
Figure 23
Description of the WSA each minute, 5 minutes before separation conflict
- Controllers 9, condition 30 aircraft per 30 minutes - ASL system
5 minute before error4 minute before error3 minute before error2 minute before error1 minute before error
Separation conflict
37
Figure 24
Description of the WSA 15, 10 and 5 minutes before separation conflict
- Controllers 11, condition 30 aircraft per 30 minutes - ASL system
15 minutes before error10 minutes before error5 minutes before error
38
Figure 25
Description of the WSA each minute, 5 minutes before separation conflict
- Controllers 11, condition 30 aircraft per 30 minutes - ASL system
Separation conflict
5 minute before error4 minute before error3 minute before error2 minute before error1 minute before error
39
Figure 26
Description of the WSA each minute, 5 minutes before a missed approach
- Controllers 10, condition 30 aircraft per 30 minutes - ASL system
Missed approach
5 minute before error4 minute before error3 minute before error2 minute before error1 minute before error
40
41
6 - DISCUSSION AND CONCLUSION -
This study was intended to evaluate the feasibility of using both physiological and
gaze data for the evaluation of Human-Machine Interface in the field of Air Traffic
Control. It was conducted in two steps:
- in a first step, a head-mounted eye-tracking device (IVIEW), was used to test the
differences in the visual scanning patterns of Controllers using a keyboard or a
trackball,
- in a second step, a remote eye tracking device compensating the head movements was
used (ASL 504). This step was more focused on the analysis of errors. The data
collected with IVIEW were also processed regarding errors.
The simulations were presented on the TRACON software. The first study has
involved 8 controllers, the second 4 controllers. Two controllers were involved in the two
studies. In addition to gaze parameters, physiological data (Heart Rate Variability for
mental effort, EEG for attention process) and subjective data (NASA-TLX for perceived
workload) were collected.
The results of the IVIEW study showed the relevance of the data collected to
evaluate the impact of the keyboard versus the trackball conditions. However, it has also
raised the limitations of the eye-tracking device related to the fact that it does not
compensate the head movements. Due to these limitations, the data processing was manual,
time-consuming and cannot be enough detailed to conduct a precise analysis.
Opposite, the ASL 504 tested in the second study has two main advantages. First, as
it is a remote device, it reduces the constraints for the operators. Secondly, as the eye
camera movements are linked to the head movements it compensates all the movements of
the operator. Thanks to this compensation, the data processing is automatic and thus is less
time-consuming and more precise than the IVIEW device. Moreover, the ASL 504 allows
to work on multi-display environment. The only limitation that was noticed concerns the
operators who wear spectacles. For some of them, the gaze data are lost during a few
seconds.
The ASL 504 was particularly useful for the error analzsis. In fact, it allowed to
evaluate the area of the radar covered by the controllers’ visual scanning in the minutes
preceding the occurrence of the errors. This analysis allows to distinguish two kinds of
errors: those related to a lack of visual perception of the area where the error occurred and
those due to the processing of the situation by the Controller. In the first kind of error, the
42
visual scanning area does not cover the point where the error occurred whereas in the
second type the visual scanning area covers this point. Therefore, for this latter, the cause
of the error would be more related to the cognitive process of the situation than to the
perception of the situation. This errors analysis using eye tracking should be confirmed on
a larger sample. Also, it should be useful to have a debriefing with the controller at the end
of the session in order to better interpret the cause of the error.
The IVIEW data regarding the error, although less detailed than those collected with
ASL suggest a relationship between HRV and visual scanning. Most of the time, in the
minutes preceding an error an increase of the power in the middle frequency of HRV (i.e.
an increase of mental effort) is observed while the fixation duration increase. This result is
consistent with those of Itoh et al (1990) who found that higher fixation duration is
observed when the workload increased. This result can be explained by the fact the
increase in task load imposed that the pilot and the Controller take information and
concentrate their attention on a limited amount of information.
In conclusion, the feasibility study presented in this report had demonstrated the
interest of using both physiological and gaze data in the context of the evaluation of a
specific HMI.
Therefore, it is proposed to transpose this method in more specific simulation as a
tool for the evaluation of new Controller ‘ workplace. In this context, the data collected
should be useful for the evaluation of the CTTT and Skytool concepts (Controllers
Transition Trials Tools). More specifically, this transposition will focus on the impact of
the automation of the Controllers’ workplace as well as on the use of electronic stripping
presented on flat displays. This study will allow to take into account different levels of
automation, from the traditional Controllers’ workplace to a highly automated HMI using
flat displays. This study would permit to provide some recommendations for the
optimization of the information displayed to the Controller.
43
7 - REFERENCES -
BELLENKES (A.H.) ; WICKENS (C.D.) ; KRAMER (A.F.).- Visual scanning and pilot
expertise: the role of attentional flexibility and mental model development.- Aviation, Space and
Environmental Medicine, 1997, vol. 68, n° 7, pp. 569-579.
BROOKINGS (J.B.) ; WILSON (G.F.) ; SWAIN (C.R.).- Psychophysiological responses
to changes in workload during simulated air traffic control. -Biological Psychology, 1996, 42,
pp. 361-377.
CABON (P.) ; MOLLARD (R.) ; COINTOT (B.) ; MARTEL (A.) ; BESLOT (P.).-
Elaboration of a Method for the Assessment of psychophysiological states of Air Traffic
Controllers in Simulation. EEC Report N° 323, 1997, 64 p.
CHIA (F.C.) ; FANG-TSAN (F).- A new method for describing search patterns and
quantifying visual load using eye movement data.- International Journal of Industrial
Ergonomics, 1997, 19, pp. 249-257.
DAVID (H.).- Measurement of Air Traffic Controllers’ eye movements in Real-Time
Simulation. EEC Report N° 187, 1977, 35 p.
DRURY (C.G.) ; SINCLAIR (M.A.).- Human and machine performance in an inspection
task.- Human factors, 1983, vol. 25, n°4, pp. 391-399.
HARRIS (R.L.) ; GLOVER (B.J.) ; SPADY (A.A.).- Analytical Techniques of Pilot
Scanning Behaviour and Their Applications.- NASA papers 2525, 1986.
ITOH (Y.) ; HAYASHI (Y.) ; TSUKUI (I.) ; SAITO (S.).- The ergonomic evaluation of
eye movement and mental workload in aircraft pilots.- Ergonomics, 1990, vol. 33, n° 6, pp. 719-
733.
KREBS (M.J.) ; WINGERT (J.W.) ; CUNNINGHAM (T.).- Exploration of an
Oculometer-Based Model of Pilot Workload.- NASA CR-145153, 1977.
44
LAYA (O.).- De la situation réelle à la simulation : exemples d’analyses de l’exploration
visuelle de l’automobiliste.- Editions Octares, Collection Colloques, 1994, pp. 69-92.
MENU (J.P.) ; PLANTIER (J.) ; VALOT (C.).- Les stratégies oculomotrices d’exploration
d’images satelliales.- Editions Octares, Collection Colloques, 1994, pp. 25-36.
MOORAL (J.).- Visual inspection of sheet steel : an ergonomic study in the Dutch steel
industry.- Le Travail Humain, 1974, vol. 31, n°1, pp. 35-52.
ROUSSEAU (F.) ; LOSLEVER (P.) ; ANGUE (J.C.).- L’exploration visuelle d’images
publicitaires.- L’analyse ergonomique du travail par l’étude de l’exploration visuelle.- Editions
Octares, Collection Colloques, 1994, pp. 37-56.
SAITO (M.).- Studies on bottles inspection task.- Journal of Science of Labour, 1972, vol.
48, n°8, pp. 475-495.
SIMON (P.) ; ROUSSEAU (F.) ; ANGUE (J.C.). Quantitative analysis of mental
workload influence on eye scanning movements.- IEEE/SMC’93 Conference, Systems
engineering in the service of humans, Le touquet – France, 1993.
TOGAMI (H.).- Affects on visual search performance of individual differences in fixation
time and number of fixations.- Ergonomics, 1984, vol. 27, n° 7, pp. 789-799.
TOLE (J.R.) ; SYTEPHENS (A.T.) ; HARRIS (R.L.) ; EPHRATH (A.R.).- Visual
scanning behavior and mental workload in aircraft pilots.- Aviation, Space and Environmental
Medicine, 1982, pp. 54-60.
SPERANDIO (J.C.).- La recherche en psychologie (Domaines et méthodes).- Edition
DUNOD, 1991, pp. 199-262.
STEIN (E.S.).- Where will all our air traffic controllers be in the year 2001 ? Proceeding
of the Fourth International Symposium on aviation, Columbus, pp. 195-201.
45
ANNEX
Detailed operating procedures forthe Weighted search area (wsa)
46
47
1 - Acquire the horizontal and vertical coordinates of each fixation point (Xij ; Yij)
from the eye movement data, where i = 1, 2, 3, 4 denote the four quadrants on a two-
dimensional plane and j = 1, 2, 3, …., n equal the total number of fixations taken by a
subject. These coordinate values correspond to the fixation vector extending from the
origin.
2 – Multiply each fixation vector by its respective fixation time tij, then the vector
becomes r V (t ij X ij ; t ijYij), where tij > 0.
3 – Derive the compound vector r V i (Xi ; Yi) by summing up all the component
vectors on each quadrant : Xi = tijXijj =1
n
∑ and Yi = tijYijj =1
n
∑ where i = 1, 2, 3, 4 denote the
four quadrants. The lenght (Li) of the compound vector on each quadrant equals Li =(Xi
2+Yi2). The angle () of the compound vector
r V i on each quadrant can be derived by
ß = tan-1Xi
Yi where i = 1, 2, 3, 4 denote four quadrants.
4 – Project all the r V i onto each axis and sum up by the following rules : L0(X+) =
L1cosß1 + L4cosß4, L0(X-) = L2cosß2 + L3cosß3, L0(Y+) = L1sinß1 + L2sinß2, L0(Y-) =
L3sinß3 + L4sinß4 where L0(X+) is the total projected length on the X axis for X>0, L0(X-)
is the total projected length on the X axis for X<0, L0(Y+) is the total projected length on
the Y axis for Y>0, L0(Y-) is the total projected length on the Y axis for Y<0.
5 – Rotate the horizontal and vertical axes, where 00 < K0 < 900 . Both axes would
be rotated K degrees by R times, where R = (900 / K) –1. Then, reproject the compound
vector r V i onto the new references axes. Though the reprojection, Lr(X+), Lr(X-), Lr(Y+),
Lr(Y-) can be derived where Lr(X+) is the total projected length on the X axis for X>0,
Lr(X-) is the total projected length on the X axis for X<0, Lr(Y+) is the total projected
length on the Y axis for Y>0, Lr(Y-) is the total projected length on the Y axis for Y<0, in
the r rotated coordinate system. Procedures would be repeated R times for r = 1 until r = R.
48
49
Traduction en langue française de l’Avant-propos, du Résumé, del’Introduction, des Objectifs, de la Discussion et Conclusion.
ANALYSE DU REGARD ET MESURES PSYCHOPHYSIOLOGIQUES:
UNE METHODE POUR LE DEVELOPPEMENT ET L’EVALUATION DESINTERFACES HOMME-MACHINE.
ETUDE DE FAISABILITE
par
P. Cabon, B. Farbos, R. MollardLAA Université René Descartes - Paris V
AVANT - PROPOS
Le besoin d'évaluer plus objectivement l'impact du contrôle aérien sur les contrôleurs est unepréoccupation ancienne. De nombreuses études ont été conduites au CEE pour déterminer lesméthodes d'investigations les plus adaptées pour en mesurer les effets (CEE Rapports n° 64, 164,183, 226, 228, 323, 334 et 339).
Le LAA (Laboratoire d'Anthropologie Appliquée de l'Université Paris V, René Descartes) amené, sous la direction du Pr Alex Coblentz, de nombreuses études sur la fatigue, les troubles dusommeil et autres troubles associés sur les pilotes de lignes, les conducteurs d'automobiles et lescontrôleurs aériens.
C'est pourquoi, nous avons demandé au LAA, de faire une série d'études de faisabilité surl'utilisation d'analyse du cortisol, d'électro-encéphalographie en particulier la technique du PEA(Potentiel Evoqué Auditif) ainsi que d'autres méthodes ad hoc et qui sont décrites dans unrapport CEE n° 323 et une note CEE n°16/98. Ces études ont été réalisées par le LAA sur unsimulateur autonome TRACON II et ultérieurement sur des simulations temps-réel et fait l'objetdes rapports CEE n° 334 et 339.
Ce rapport couvre des études complémentaires d'analyses des mouvements oculaires associées àdes mesures électro-physiologiques, également réalisées sur TRACON II.
Le Directeur du Centre Expérimental, les chercheurs du Laboratoire d'Anthropologie Appliquéeainsi que les chefs de projets tiennent à exprimer leurs remerciements aux contrôleurs et auxpersonnels du CEE pour leur étroite et dévouée collaboration.
Françoise CalooHugh David
EEC project officers
50
51
RESUME
Cette étude démontre la pertinence d'une méthode reposant à la fois sur l'analyse des regardscouplée à des données physiologiques pour évaluer des interfaces Homme-Machine dans ledomaine du contrôle de la circulation aérienne.
Une revue de la littérature sur le thème des mouvements du regard a précédé deux études qui ontété conduites au Centre Expérimental Eurocontrol. Deux systèmes ont été utilisés, dans lapremière étude un système ( I VIEW) posé sur la tête et dans la deuxième, un système déporté(ASL504). Un logiciel TRACON II permettant de présenter aux contrôleurs des tâches ATCreprésentatives et une sélection de mesures electro-physiologiques (EEG, HRV etc….) ontégalement, été utilisés pour ces deux études.
Cette étude conforte l'intérêt d'associer des paramètres physiologiques et d'analyse du regardpour évaluer la charge de travail, la fatigue, les stratégies opérationnelles des contrôleurs dans unenvironnement hautement interactif.
Des études complémentaires devraient être poursuivies pour analyser l'impact des changementsdans les tâches ATC induit par l'introduction et la généralisation de nouvelles technologiesinformatiques tels que: strips électroniques, écrans plats etc…
52
53
1 - INTRODUCTION ET OBJECTIFS -
L’introduction de nouvelles technologies dans le domaine du contrôle aérien
entraîne des changements importants dans les procédures de travail et dans les outils
utilisés par les contrôleurs. Les écrans radar traditionnels et les strips papiers sont
progressivement remplacés par des écrans cathodiques, des strips électroniques, une
souris,... Ces changements peuvent affecter la charge de travail et la performance des
contrôleurs. Il apparaît essentiel de s’assurer que ces nouvelles technologies soient
capables d’aider les contrôleurs à gérer un trafic en constante augmentation. Dans ce
contexte, les travaux menés par le Laboratoire d’Anthropologie Appliquée en collaboration
avec le Centre Expérimental d’Eurocontrol ont pour objectif de valider des méthodes
d’évaluations permettant d’étudier l’impact de ces nouvelles technologies sur les
contrôleurs. Ces méthodes reposent sur le recueil de données physiologiques et subjectives,
concernant la charge de travail, le stress, la fatigue et l’activité des contrôleurs.
La présente étude vise à compléter ces évaluations avec l’utilisation de l’analyse des
mouvements du regard. Elle a pour but de prendre en compte les aspects perceptifs et de
traitement de l’information lors de l’évaluation d’un nouvel outil ou pendant la
conception d’une nouvelle interface. L’intégration de nouveaux outils informatiques se
traduit en effet fréquemment par l’utilisation d’écrans de grande taille (2000X2000), ce qui
peut influer sur la présentation des informations par rapport aux postes de travail
traditionnels mais aussi modifier les stratégies visuelles des contrôleurs. En fonction de
leur localisation sur l’écran, certaines informations peuvent éventuellement être négligées
par les contrôleurs.
L’objectif principal de cette étude consiste à évaluer la pertinence d’une méthode,
reposant à la fois sur l’analyse des regards couplée à des données physiologiques, pour
évaluer des interfaces. La démarche retenue repose sur l’utilisation de simulations
simplifiées à partir du logiciel TRACON, déjà utilisé dans des études précédentes (Cabon
et coll., 1997) pour évaluer la charge de travail, les processus d’apprentissage et le stress
des contrôleurs. Deux campagnes d’essais ont été conduites au Centre Expérimental
d’Eurocontrol. Dans la première campagne, un système (IVIEW) posé sur la tête du
contrôleur a été utilisé. Dans la seconde, un système déporté a été testé (ASL504). Ce
dernier est équipé d’un dispositif de compensation des mouvements de la tête permettant
de réaliser des traitements plus précis sur les stratégies oculaires adoptées par les
contrôleurs. Ce rapport présente à la fois une revue de la littérature sur le thème des
mouvements du regard ainsi qu’une synthèse des principaux résultats obtenus au cours de
ces deux campagnes.
54
2 - DISCUSSION ET CONCLUSION -
La première campagne de mesure a été effectuée avec la participation de 8
contrôleurs, la seconde avec 4 contrôleurs. Deux contrôleurs ont été impliqués dans les
deux campagnes. En complément des paramètres de regard (nombre et fréquence de
fixations), des données physiologiques (variabilité cardiaque pour l’évaluation de l’effort
mental, l’EEG pour les processus attentionnels) et des données subjectives (NASA-TLX
pour la perception de l’effort) ont été recueillies.
Les résultats de l’étude « IVIEW » (première campagne de mesure) montrent
l’intérêt des données recueillies pour évaluer l’impact de l’utilisation du clavier comparé à
une boule roulante. Cette étude a également mis en évidence les limites d’IVIEW en raison
de l’absence de compensation des mouvements de la tête. De ce fait, le traitement des
données reste manuel, long et n’est pas suffisamment détaillé pour effectuer des analyses
précises. A l’opposé, le système ASL 504, évalué dans la seconde étude, présente deux
avantages principaux. Le premier est qu’il réduit les contraintes pour le contrôleur
puisqu’il n’est pas posé sur la tête. Le second a trait au dispositif de compensation des
mouvements de la tête qui permet d’effectuer un traitement automatique, beaucoup plus
rapide et précis qu’avec IVEW. Par ailleurs, ASL504 peut être utilisé pour évaluer des
postes de travail qui comportent plusieurs écrans. La seule difficulté technique rencontrée
concerne les contrôleurs qui portent des lunettes. Dans ce cas, les données ne peuvent pas
être exploitables sur de courtes périodes (quelques secondes) d’enregistrements.
Le système ASL s’est révélé particulièrement utile pour l’analyse des erreurs. En
effet, il permet d’évaluer précisément la localisation du regard sur l’écran dans les minutes
précédant l’erreur. Cette analyse a permis de distinguer deux types d’erreurs : celles liées à
une absence de perception visuelle de la zone où l’erreur a eu lieu et celles associées à une
absence de traitement de la situation. Dans le premier cas, les regards ne couvrent pas la
zone de l’erreur alors que dans le second, le contrôleur regardait cette zone. Néanmoins,
cette analyse d’erreur doit être confirmée sur un échantillon plus important. Un entretien
d’auto-confrontation avec le contrôleur à la fin de la session, permettrait également de
mieux interpréter ce type d’erreur, en particulier sur le plan de la stratégie adoptée par le
contrôleur.
Les données concernant les erreurs collectées lors des essais avec le système IVIEW
bien que moins détaillées que celles recueillies avec ASL, suggèrent une relation entre la
variabilité cardiaque et les mouvements du regard. Généralement, dans les minutes
précédant une erreur, une augmentation de la puissance dans la bande de fréquence
moyenne (c’est-à-dire une augmentation de l’effort mental) s’observe pour la variabilité
55
cardiaque alors que la durée des fixations augmente. Ces résultats confirment ceux de Itoh
et coll. (1990) qui ont observé que l’augmentation de la durée de fixation se trouve
associée à une augmentation de la charge de travail. Ce résultat peut être expliqué par le
fait que l’augmentation de la difficulté de la tâche entraîne une focalisation de l’attention
sur un nombre limité d’informations.
En conclusion, cette étude de faisabilité démontre l’intérêt d’utiliser à la fois des
données physiologiques et de regard dans le contexte de l’évaluation d’une interface
spécifique. Il est donc proposé de transposer cette méthode à des simulations plus
complexes comme outil d’évaluation de nouveaux postes de travail des contrôleurs. Dans
ce contexte, les données recueillies peuvent s’avérer utiles pour l’évaluation des concepts
CTTT (Controllers Transition Trials Tools) et Skytools. Cette transposition pourrait être
centrée sur l’impact de l’automatisation et de l’utilisation des strips électroniques sur la
charge de travail des contrôleurs.