The planning, execution and evaluation of field

exercises in civil protection

David Alexander University College London

Exercise: "an activity intended to further

preparations for emergency situations, test operations, policies, plans,

procedures or equipment, or demonstrate capacity to resond to a crisis."

(FEMA)

In an exercise one works as if one were in a real situation, but in response to a simulated event (yet with realistic actions).

Types of exercise:-

• tabletop exercise: with the aid of a hypothetical scenario

• command post exercise: to test a single activity, usually related to command and control

• field exercise: the most extensive and realistic.

• test of emergency procedures

Emergency procedures exercise: • mass evacuation • command procedures • rescue actions, etc.

Commmand post exercise: • 3-6 hours to test the system of communication and command.

Some reasons for holding a field exercise in emergency procedures (1):-

• to test and evaluate plans and procedures

• to show the weak points of emergency plans

• to train personnel in their roles and responsibilities

• to highlight and reduce inadequacies of personnel and equipment.

Some reasons for holding a field exercise in emergency procedures (2):-

• to clarify the division of roles and responsibilities among organisations

• to improve the performance of personnel and organisations

• to acquire support for emergency management on the part of politicians and the public.

• to improve co-ordination and communication among organisations

Some reasons for holding a field exercise in emergency procedures (3):-

• to prepare participants for a real emergency

• to identify education and training needs

• to improve the quality of the messages transmitted during an emergency

• to improve the utilisation of volunteers in emergency operations.

Typically, a field exercise takes 6 months to two years to plan and 8-48 hours to run.

It is carried out in real time and involves the typical stresses of the job.

Start by assessing capabilities:-

• agency's ability to conduct exercise

• designer's ability to create exercise

• when and where was the last exercise?

• what experience available in organisation?

• how much time is available to prepare the exercise? .

The "protagonists":-

• the general director of the exercise

• the steering committee of the exercise

• the chair of the steering committee

• the controller of the exercise

• the observers.

• the simulators

Start by assessing capabilities:-

• what personnel are available to develop the exercise?

• what skills do they have?

• what physical facilities available for conducting the exercise?

• what communications facilities available?

• what are the attitudes of chiefs and directors to the whole idea? .

Costs are incurred at every stage of developing and running an exercise:-

• staff salaries • consultants' costs and contract services • equipment and materials • printing and dissemination of materials • hire of facilities • logistical support.

Simulators

Exercise players

Exercise steering committee (and chair)

General Director

briefing

debriefing debriefing

briefing

briefing

Final report

Observer team

Some strategies:-

• inform participants of a window of time within in which the exercise will take place but not the exact timing until it happens

• base the exercise master document on a real event from the past or a credible future scenario for the area.

Putting the exercise into practice:-

Plan the field exercise - the activities developed for the exercise should test the emergency plan: the objectives should refer to a particular aspect of the plan that needs to be tested.

Carry out the field exercise - the objectives guide the exercise: all activities should be focussed on realising the objectives.

Putting the exercise into practice:-

Evaluate the field exercise - forms should be developed to help evaluate each objective, and to compare the results with the emergency plan, which will constitute the guidelines for evaluating the progress of the exercise.

Putting the exercise into practice:-

Planning the exercise:-

• estimate needs

• establish the scope of the exercise

• define the objectives

• write the narrative (the scenario) of the exercise.

Planning the exercise:-

• define the principal events and the expected actions

• write the emergency messages

• compile the master sequence of events list (MSEL)

• allocate tasks to participating personnel (simulators, observers, etc.).

evolution

development of the scenario evolution

time zero

formal evaluation of the outcome of the scenario

consequences at time n

Scenario methodology in emergency

planning

consequences at time 2

consequences at time 1

reference event

initial conditions

evaluation of the progress

of the scenario

historical analysis

hypothetical ingredients

The exercise controller:-

• has complete control of the exercise

• must ensure that the exercise is faithful to the emergency plan and to the programme compiled for the event

• can increase the number of messages sent to participants in order to increase the activity rate

• can slow down the exercise if too many things happen.

The exercise controller:-

• must try to solve any problems that occur

• stops the exercise when all the objectives have been met

• an exercise with more than one command centre will require more than one controller.

The simulators:-

• send the emergency messages

• it is better that they have no direct contact with other participants

• after the exercise, for information- gathering purposes, they should be given a full debriefing.

The main participants in the exercise:-

• should be given a full and detailed briefing on what is involved in the exercise

• need to know how the controller(s), simulators and observers are identified.

Exercise safety must be ensured:-

• every participant should monitor safety

• any participant can stop the exercise if a major safety issue arises

• safety concerns in the field and at control centres need to be assessed.

• appoint an exercise safety officer or monitor

The exercise evaluators need:-

• forms for evaluating the actions that take place during the exercise

• training as observers

• a debriefing meeting with the controller(s), simulators and observers

• to write a final report

The evaluation process is based on description, deduction and judgement.

Possible causes of lack of objectivity:-

• distractions when tiredness causes the observers to lose their concentration

• prejudices resulting from the personality preconceptions of an observer

• when the participants react to the presence of an observer

• when an observer judges all actions to be adequate.

Possible causes of lack of objectivity:-

• when an observer judges all activities to be neither particularly well done nor particularly badly done

• when an observer immediately forms the impression that things are going well and this persists throughout his or her period of observation

• when an observer believes he or she is obliged to find something inadequate

• when an observer has a preconception about how the exercise should go.

Evaluators need to avoid being:-

• too lenient

• fatigued and demotivated

• biased

• hypercritical

• affected by preconceptions.

The rules of exercise evaluation:-

• safety first and foremost

• actions, not people, are being evaluated

• do not interfere with course of exercise unless there is a safety problem

• do not coach exercise participants

• be as objective as possible.

Forms of evaluation:-

• describe actions in relation to the objectives of the exercise

• evaluate performance and infer quality

• document the activity

• collate data on observations and judgements

• review and conclude.

Post-exercise debriefing of evaluators:-

• information shared and perspectives compared

• suggestions made for improvements

• deficiencies noted

• training needs identified

• a general sense obtained of how things went.

simulator debriefing

participant debriefing

final report

evaluator debriefing

exercise conclusion

The ingredients of the final report:-

• an explanation of why the exercise was held

• a summary of the activities carried out during the exercise

• a summary of activities carried out before the exercise (meetings, training)

• a list of participating organisations

• the aims and objectives of the exercise.

The ingredients of the final report:-

• problems observed during play and recommendations for correcting them

• a description of the things that went well

• identification of particular training and educational needs

• other recommendations and comments

• any problems encountered in the development and activation of the exercise.

After Gebbie et al. (2005)

[x]