18
Contingency Exercises at Dexia Integrating Evacuation” – “Business Continuity” – “Crisis Mgt.” 1 March 23rd 2011 - Brussels Ludo.Jappens MBCI Dexia Bank Belgium [email protected]

Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Embed Size (px)

DESCRIPTION

Seminar: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia) during Infosecurity.be 2011

Citation preview

Page 1: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Contingency Exercises at Dexia

Integrating “Evacuation” – “Business Continuity” – “Crisis Mgt.”

1

March 23rd 2011 - Brussels

Ludo.Jappens MBCIDexia Bank Belgium

[email protected]

Page 2: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Agenda

� The Dexia Crisis Mgt. framework

� Components of the Contingency & BC plan� Actors vs. Volunteers� Command Control

Contingency exercising at Dexia Bank

2 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

� Command Control� Protocol, Scenario & Roles� Data driven approach� Awareness Campaign� Base principles integrated exercises

� Filmed report major incident exercise

� Mgt. buy in - Reporting & KPIs

Page 3: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Strategic

Tactical

CMT

What, Budget, Resources

Who, How, When, Where

Crisis Mgt. Team

Board of Directors (most concerned member)

Corporate Crisis Mgr.members (representing the impact domains)

CCM

��� �����

� (BoD)��

The Dexia Crisis Mgt. Framework

3 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Operational

DRP FCP BCP

HRRP CCP Execute

BCPBCP

Business Continuity Plan

Critical Business Activities: candidates

BC actors(coordination)

��

Qualified volunteers

Professional Staff(coordination)

HR services

Page 4: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

IDPBW/SIPPT

Tactical level

Facilities &Logistics

Responsible professional staff

Tactical actors vs. Operational volunteers

Evacuation “Flash point”

Impactdomain

Operational levelQualified volunteers

First Intervention Evacuation team

Reception desk at evacuated building

Provisional (first period)

4 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

HR - Medical Services

HR - Social Services

IDPBW – SIPPTPsychoSocial prevention

Communications

Human

Communications

Evacuation coordinator

“Flash point”

Medical support (1st aid)

PsychoSocial Relief

PsychoSocial After Care

Crisis Call Center operators

Page 5: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Contingency plan – Command Control

Strategic

Tactical

Facilities

CCM��

Internal crisis network

����

Extended crisis network

5 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Facilities & Logistics

Crisis Mgt. Dexia

Astrid (Belgian emergency telephone network)�Corporate Crisis Mgr.�Dispatching (planned : no reception)�3 Reception desks (DT, GI, PA)

Crisis Mgt. Government

Command Centre Government

Crisis Mgt. Government

Command Centre Government

“Flash point”

Evacuation coord.

Page 6: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Protocols – Scenario & Roles

� evacuated building

� assembly point

�DT

� registration points

welcome places

(business restaurants)

? major incidentY

Evac.coord

Evacuation

���� ����

Crisis Call Centre

���� Information Centre

���� Crisis Centre

Crisis Mgt

Very CriticalBusiness Activities (RTO 4H)

� evacuated building

relocation places BCP

Business Continuity

6 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

�OM

PA

�Martelaren Pl. Martyres

� GI

Badge traces enable statistics & determination “missing persons”

FlashPoint

Firstinterv.

Med

P-S

Page 7: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

� Critical Business Activities

�Understanding

the Organisation

� Business Impact Analysis(financial, human, operational impacts)

Internal vs. External Recovery

�Business Continuity CoordinatorsBusiness Relocation Coordinators

Integrated exercises – Business Continuity

(BS25999 - 2006) The BC Lifecycle

Good Practice guidelines (2010)

7 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

SI

Developing and

Implementing

a BCM Response

� Critical Business ActivitiesRTO: 4H, 2D, 7 D, …

CritBusAct

Persons

BCM

Programme

Management

�Determining

BCM

Strategy

Exercising,

Maintaining

& Auditing�

� Evacuation exerciseBCP: Very Critical BA (4H) included

Page 8: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

BCP

BCP operatorsvolunteers

Access

SAP

Staff membersOrgunits

ODBC

Registration

Physical accesses(badges)

Snapshots

� evacuation time

� Business resumption

� Staff memberssaved at other bldg.

Integrated exercises – Support tools

8 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

non registered persons KPIsprint BCP reports, welcome relocated staff

Page 9: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

� Exercise frequencyyearly, unannounced, minor incident but also 1 “major incident”.

� Confirmation transversal project groupCoordination: ORM – BC & Crisis Management.

� Minimal content exercises:� Evacuation, move to the assembly point, roll out coordination & communication structure (“vlampunt” & evacuation

coordinator).

� Information center: registration staff members in welcome building (major crisis) or at the moment of return in the left building (minor)

Awareness campaign - Directives (endorsed by BoD)

9 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

in welcome building (major crisis) or at the moment of return in the left building (minor)

� Start medical & psychosocial relief & crisis call center (technically)

� Activation crisis room in PA (primary) or DT (alternate)

� Fill in Business Continuity Plan roles (roll out = optional)

� Crisis Management training at Strategic & Tactical level� Active participation member BoD

� Reporting to Board of Directors (ORM) &Labor Union- Commission Evacuation (IDPBW)

Page 10: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Awareness campaign - Evacuation leaflet

Major incident

10 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

BCP

Page 11: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Awareness campaign – Lexicon major concepts

11 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Page 12: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Awareness campaign – Business Revue article

12 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Page 13: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

BCPVeryCBA

Crisis Mgt

Infocenter

Operationalvolunteers

Building#floors

type Staffmembers

Timing Announ-ced ?

Evac P-S Med CCC

DT 33

minor 3.300

Return to DT

April 27

2009

N Y - Registr.center(post)

Yactive

- - -

GI 12

minor 1.300

Return to GI

June11 2009

N Y - Registr.Centeractive

Yactive

Ypassive

Yactive

-

��������

Integrated exercises – Gradual implementation

Increasing degree of complexity

dealing room

13 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Return to GI active

PA12

minor 1.200

Return to PA

Sept21

2009

N Y - Registr.Centeractive

Yactive

Yactive

Yactive

-

OM6

major 453welcome buildingDT

Nov 19 2009

N Y Tacti-cal

teams

Registr.+ info

Yactive

Yactive

Yactive

-

dealing room

No data centers in working locations � no combined DRP & BCP incident

Since 2010 : every building cluster once a year

Most recent exercice: 1st of March 2011 - major for GI

Page 14: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

Awareness campaign - Filmed report

Exercise OM & PA Sept/Nov 2009

14 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Extended version (NL, FR, EN) 8 min.

Page 15: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

1. Identification Building Date Day Hour Period Type

DT (33 floors) 07/05/2010 Friday 9:35 normal minor

# persons SAP # persons In the building

% Occupation # BCP Business Units

# wildcardsBCP

# wildcards others

3.654 2.086 57 % 5 5 15

2 Exercise scope

BCP PsychoSocial relief Medical relief & “triage” place

Crisis call center technical

Crisisroom(Tactical BCP only)

Integrated Exercises – Reporting & KPIs DT (1)

15 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

# “missing persons”

- to examine: 234 (11,21%) (cfr. remark 2)

3. Evacuation statistics

Detection time Time alert Decision time Communication "building empty"

End of return Provokedwork delay

9:15 9:32 17 min

(cfr. remark 3)

10:03 11:00 285 man-days

96.615 €

First Intervention Team (FIT)

# in the building 133

# available at evacuation: 82

# floors without FIT: 7 (cfr. remark 1)

Activated Activated Activated Activated Activated

Positive Positive Positive Positive Positive

Page 16: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

4. Business Continuity Plan statistics

Dept # persons (C + A) Very CBA

in building DT

# persons (C + A)Very CBA

Relocation places

# desks Very CBA

at relocation place

BCP time to relocate

Requirement 4H !

Payments (DT) 44 OM 41 (93%) 100 % 33 min

Dexia Technology Services

(DT)

136 PA 105 (77%) 90 % 45 min

IT Development (DT) 168 GI 92 (55%) (remark 6) 100 % 1 u 16 min

Dexia SA (DT) 17 OM 8 (47%) (remark 6) 100 % 40 min

Dir Sales & Services (DT) 11 OM 6 (55%) (remark 6) 100 % NA (informed before

Integrated Exercises – Reporting & KPIs DT (2)

16 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Dir Sales & Services (DT) 11 OM 6 (55%) (remark 6) 100 % NA (informed before

alert)

1. Floors without First Intervention Team (FIT) member IDPBW

2. There probably exist unregistered ways to leave the building or some gates only generate an input registration FACM

3. The decision time (since the first alert) is too long. To many FITs had to be called to get the appropriate intervention IDPBW

4. Two fire elevators on three were not working “as designed” IDPBW

5. Some (new) staff members of the reception desk did not have the right education to use the application BCP & CM

6. Too many candidates/alternates for the very critical business activities did not move to the relocation building, as required by the BC

procedure.

BCP & CM

7. The evacuation procedures are not sufficiently known by the staff of the branches (just moved to the DT building) BCP & CM +

IDPBWW

5. Attention points and responsible departments

Page 17: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

1. Identification Building Date Day Hour Period Type

GI (12 fl.) 1/3/2011 Tuesday morning Normal major (���� DT / PA)

# persons SAP # persons In the building

% Occupation # BCP Business Units

# wildcardsBCP

# wildcards others

988 564 57 % 6 19 2

2 Scope – assessment exercise components

BCP PsychoSocial relief

Medical relief & “triage” place

Crisis call center Crisis communication

Crisis Mgt.

Integrated Exercises – Reporting & KPIs GI (1)

17 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

Registration Centre

# “non-registered persons”

- raw list: 108 (19 %) (cfr. remark 3, 7)

- reduced to 28 (5 %) with a post control

but 17 (3 %) (non Dexia) impossible to identify

3. Evacuation statistics

Detection time

Time alert Decision time

"building empty“Declare Major crisis

Arrival wellcome buildings (DT&PA)

End exerc.(DT & PA)

Staff at GI(return to GI)

Provokedwork delay

9:15 Broadcast 9:29

Siren 9.36

14 min

(cfr. rem 2)

9.49(20 min.)

10.20(51 min.)

10.46(1 H 17 min)

11.14(1 H 45 min.)

95 man-days 32.237 €

(minor 35 MD – 11.915 €)

First Intervention Team (FIT)

# assigned to building GI 32

# available at evacuation: 22 (69 %)

# floors without FIT: 2 (cfr. remark 1)

(probably because ongoing move that day)

Deployment BC plan

for Very Crit at PA

Activated at GI, PA and DT Activated for GI

low visibility in PA & DT

Call list CCC operators at

DT

Astrid, broadcasts,

Intranet, (rem 5)Activated at wellcome

buildings DT & PA

4 - good 4 - good 3 - average 4 - good 2 - poor 4 - good

3 - average4 - good

rem 6

Page 18: Filmed report of a comprehensive contingency exercise at Dexia Bank Belgium by Ludo Jappens (Dexia)

4. Business Continuity Plan statistics

Dept # persons (C + A) Very CBA

in building GI

# persons (C + A)Very CBA

Relocation place PA

# desks Very CBA

at relocation place

BCP time to relocate

Requirement 4H !

DBB-BSM (GI 01) 2 2 (100 %) 1 / 1 Call list 19 min – no operations

DBB-CPPR (GI 10) 16 3 (19 %) (remark 4) 3 / 3 Call list 28 min - no operations

DBB-CRM (GI 05/11) 15 4 (27 %) 3 / 3 Call list 28 min - no operations

DIC (GI 00) 1 0 (0 %) 0 / 1

DBB-FMR (GI 07) 6 6 (100 %) 3 / 3 Call list 24 min - no operations

DBB-OOFM (GI 03) 73 59 + 4 WC (86 %) 35 / 41 (rem 4) Call list 21 min - no operations

Integrated Exercises – Reporting & KPIs GI (2)

18 Integrated Exercises (Contingency Planning & BCP) March 23rd 2011 Ludo Jappens

DBB-OOFM (GI 03) 73 59 + 4 WC (86 %) 35 / 41 (rem 4) Call list 21 min - no operations

DBB-TFM (GI 01) 45 19 + 15 WC (75 %) 21 / 21 CL 19 min + techn. deployment SG

1. 2 Floors without First Intervention Team (FIT) member, possibly because of ongoing move Credits dept. IDPBW

2. The decision time (from the first alert to the final “go”) is too long (communication dispatching, FIT, evacuation Coordinator).

Review procedures by means of a structured walk-through.

IDPBW – FACM

3. Re-entry GI building through evacuation exits, interferes with registration centre goals ORM, FACM

4. Decreasing BCP awareness : BC actors did not go immediately to the relocation building.

In some cases it leads to very critical business activities that could not be filled in

ORM

5. Failure of both broadcasting & emergency signal systems, confusing the procedures and the triggering of consecutive actions.

Need for an Astrid telephone in the dispatching room (cost 6.000 €) to support communication with Evac. Coord & Crisis Mgr.

FACM

6. Cost-efficiency major crisis : goals can be achieved in a simulated minor incident exercise (with return) & structured walk-through exercise ORM

7. Registration centre: organisation at welcome building, 3 % of people (non Dexia) not identifiable, shift task to HR ? ORM – HRM - FACM

5. Attention points and responsible departments