Disaster recovery plan template€¦ · Web viewThis disaster recovery plan contains resources and procedures to be used in the event that a disaster occurs at one of [Company] locations

D

Disaster recovery plan templateMaine Rural Water Association

Written in conjunction with

AbstractThis template is designed to help you develop a complete and robust disaster recovery

plan for your facility. The goal is to have a complete set of guidelines in the event of catastrophe.

Cyber Policy Templates

Disaster Recovery Plan

Introduction

This plan contains resources that minimize the effects of a disaster. This plan serves as a guide for the [Company] IT department and management staff in the event of a disaster. [Company] will be able to quickly recover or maintain mission critical functions by using this plan.

Plan Overview

This disaster recovery plan contains resources and procedures to be used in the event that a disaster occurs at one of [Company] locations which results in the destruction or severe crippling of [Company] IT infrastructure. This plan is designed reduce the number of decisions which must be made during a disaster and restore operations as quickly as possible.

This plan must be updated regularly and when there are changes to the IT infrastructure. The accuracy of this document is vitally important to the speedy recovery of the organization. Due to the sensitive nature of the information in this plan, this document should be treated as confidential.

Plan Approval

This plan has been reviewed and approved by

------------------------------------------------ ------------------------[SIGN] [Date]

1


Plan Objectives

The primary objective of this plan is to ensure the timely recovery of [Company’s] critical IT systems in an orderly fashion, while simultaneously ensuring the safety of employees and minimizing the confusion of a disaster situation. The role of this plan is to document contact information, decisions, and procedures for responding to a disaster that involves IT systems, data, and services.

A disaster is the occurrence of any event that causes a significant disruption to IT capabilities. This plan may be used in the most severe kind of disaster, requiring the relocation of operations to an alternative site but may also be used as a reference for disasters that do not require relocation. The steps required to prepare for a disaster as well as the sequence of events that need to be followed immediately after a disaster are stated in this plan.

Some of the steps may be skipped or performed out of order due to unforeseen circumstances.The approach of this plan is general enough to be applied to any disaster situation regardless of the specific nature of the event.

The plan is organized around the concept of teams. Each team has specific duties and responsibilities. Each team has a leader and, depending on the size and capabilities of the staff, some personnel may be on multiple teams. Outside help from vendors and 3rd party support will also be enlisted to help during the recovery effort.

This plan is a living document that will be kept current by testing and review. Lessons learned must be integrated into the plan to ensure that mistakes are not repeated. As the IT environment changes, this plan will be adjusted and updated accordingly.

2


Disaster Declaration

The following employees are authorized to declare an IT systems disaster and also to signal the resumption of normal activity.

Name Title

Plan Activation

This plan shall be activated in response to internal or external incidents, or the eminent threat of an incident. Internal threats could include an extended loss of power, fire, sabotage, or other incidents that threaten the facility, equipment, or staff. External threats may include severe weather, civil unrest, or natural disasters. Authorized personnel will assess the severity and likely hood of the threat and initiate the plan.

Resumption of Normal Operation

When the incident has ended, facilities have been repaired and equipment has been repaired or replaced, the disaster recovery team will assess the situation and declare the disaster to be over and normal operations will resume.

3


DISASTER RECOVERY PHASES

The disaster recovery process consists of four phases. They are:Phase 1: Disaster AssessmentPhase 2: Disaster Recovery ActivationPhase 3: Alternate SitePhase 4: Return Home

DISASTER ASSESSMENT

The disaster assessment phase lasts from the first indication that an incident has occurred or is eminent until the disaster is under control and the damage can be assessed.

DISASTER RECOVERY ACTIVATION

This phase begins when the decision is made to move operations to a different location. The Disaster Recovery Management Team will meet at the command center and command the team members to perform their assigned duties. Resuming normal operations at a suitable location is the most important function of this phase. When normal operations resume this Phase is complete.

ALTERNATE SITE OPERATION

During this phase operations will be carried out at the alternate location and restoration of the primary location is performed.

RETURN

This phase involves reactivation of the primary location. A review of the disaster recovery process will be performed, and lessons learned will be integrated into the disaster recovery plan.

4


KEY DISASTER RECOVERY ACTIVITIES

Declaring a disaster means:1. Activating the recovery plan2. Notifying team leaders3. Notifying key management contacts4. Redirecting voice service to an alternate location5. Securing a new location for operations6. Ordering and configuring replacement equipment7. Configuring the network8. Reinstalling software and data9. Keeping management informed10. Keeping users informed11. Keeping the public informed

DISASTER DECISION TREE

Event DecisionLocation Destroyed Activate Disaster Recovery Plan

Location unusable for more than 2 day Activate Disaster Recovery Plan

Location unusable for less than 2 days Management Team andFacilities Team perform an assessment

Network Down Management Team andTech Support Team perform an assessment

Telephones System Down Management Team andTech Support Team perform an assessment

Environmental problems (A/C, power, etc.) Management Team andFacilities Team preform an assessment

5


DECISION MAKING FOR A DISASTER

Decision Point Action Action Action Action CategoryIncident Occurs

Activate Alerts Begin Evacuation

Ensure all employees evacuated

Meet in designated area

Initiation

Determine if incident is real

NO Recovery Plan Not activated

Return to Normal Operation

Evaluate evacuation

Determination

YES Switch call handling to an alternate location

Determination

Determine scope of damage

Small Scope,Minimal Damage

Begin cleaning and repairs, return

Return call handling

Return to normal operation

Short EvacuationRequired

Large Scope,Moderate Damage

Activate alternative site

Activate recovery team

Notify management and employees

Moderate to Severe damage to Infrastructure

Assess Recovery Time

Less than 30 days

Complete Repairs while operating at alternate site

Return to primary site


Moderate damage to Infrastructure

More than 30 days

Locate new facility

Order supplies and equipment


Severe damage to Infrastructure

6


RECOVERY TIME OBJECTIVES (RTO)

The following chart lists the estimated recovery time for major systems in the organization. These RTO’s should be considered best case scenarios.

System Description Recovery Goal in Days

RECOVERY POINT OBJECTIVES (RPO)

The following charts lists the estimated point in time to which a recovery can be made from a backup.The RPO for a weekly offsite backup would be up to 7 days

System & Backup type Recovery Point (Age of Data)

7


DISASTER RECOVERY COORDINATOR

The Disaster Recovery Coordinator is [INSERT NAME]

The Primary responsibilities of the Disaster Recovery Coordinator are:

Distribution of the Disaster Recovery Plan Training the Disaster Recovery Teams Testing of the Disaster Recovery Plan Evaluation of the Disaster Recovery Plan Tests Review, change and update the Disaster Recovery Plan

In a disaster situation, the Disaster Recovery Plan Coordinator will:

Facilitate communication between technical and non-technical staff Act as a Project Manager to coordinate the efforts of

Technical staffBusiness staffVendorsManagementOther personnel as needed

THE COMMAND CENTER & VITAL RECORDS

A command center must be established when a disaster is declared. It servers as an operational hub for all disaster recovery operations, and acts as a temporary office for team members. Companies that have successfully recovered from a disaster reported that establishing a command center was vital to the successful recovery effort. The command center should be stocked with: Paper, Post-it notes, Pens, Pencils Trash can White boards, Markers,Erasers Telephones, Printer, PCs

A small tool kit

8


COMMAND CENTER LOCATIONS

PRIMARY LOCATION

If the disaster permits, the primary location of the command center will be:[INSERT ADDRESS]

SECONDARY LOCATIONIf evacuation from the primary location is necessary, the command center will be located at: [INSERT ADDRESS]

VITAL RECORDS RETRIEVAL

Disaster recovery plans, software licenses and server installation media are stored [INSERT ADDRESS]

OVERVIEW OF WHAT IS STORED OFFSITE

Item Location1. An up to date copy of this plan

9


DISASTER RECOVERY MANAGEMENT TEAM

Sub-teams: Administration, Supplies and Public Relations

GENERAL RESPONSIBILITIES

The IT Disaster Recovery Management Team (MGMT) is responsible for the overall coordination of the disaster recovery process from an Information Technology Systems perspective. The other team leaders report to this team during a disaster. In addition to their management activities, members of this team will have administrative, supply, transportation, and public relations responsibilities during a disaster. Each of these responsibilities should be headed by a member of the MGMT team.

Assess the damage and if necessary, declare a disaster Coordinate efforts of all teams Secure financial backing for the recovery effort Approve all actions that were not planned Give strategic direction Be the liaison to upper management Expedite matters through all bureaucracy Provide counseling to those employees that request or require it

ADMINISTRATIVE RESPONSIBILITIES

The administrative function provides administrative support services to any team requiring this support.This includes the hiring of temporary help or the reassignment of other clerical personnel.

ACTIVITIES BY PHASE

ACTIVATION PHASE Notify all vendors and delivery services of change of address

PROCEDURES DURING ALL PHASES Process expense reports Account for the recovery costs Handle personnel problems

AFTER THE DISASTER Make recommendations on how the disaster recovery plan can be improved

10


SUPPLY RESPONSIBILITIES

SUPPLY OVERVIEW

The supply function is responsible for coordinating the purchase of all needed supplies during the disaster recovery period. Supplies include all computing equipment and supplies, office supplies such as paper and pencils, and office furnishings.

ACTIVITIES BY PHASE

ACTIVATION PHASE Purchase supplies required by the teams at the alternate site.

PROCEDURES DURING REMOTE OPERATION / REBUILD Order replacement supplies and expedite shipments Ongoing distribution of supplies

PROCEDURES DURING RETURN HOME PHASE Restock supplies at the restored site


PUBLIC RELATIONS RESPONSIBILITIES

PUBLIC RELATIONS OVERVIEW

The public relations function will pass appropriate information about the disaster and associated recovery process to the public and to employees. Every effort should be made to give these groups reason to believe that [COMPANY] is doing everything possible to minimize losses and to ensure a quick return to normalcy.

ACTIVITIES BY PHASE

ALL PHASES Ensure that employees do not talk to the media Control information released to the public and to employees Interface with city officials Publish internal newsletters Keep everyone aware of recovery progress

AFTER THE DISASTER Make recommendations on how the disaster recovery plan can be improve

11


MANAGEMENT TEAM CALL CHECKLIST

Team Leader Information

Name Telephone AssignmentPrimary Team Leader

Alternate Team Leader

Team Member Information

Name Telephone Assignment

TECH SUPPORT TEAM Sub-Teams: Hardware, Software, Network, Operations

HARDWARE RESPONSIBILITIES

ACTIVITIES BY PHASE

ACTIVATION PHASE

Determine scope of damage for servers and workstations Order appropriate equipment and supplies (coordinate and work with the Facilities Team for this activity)

PROCEDURES DURING REMOTE OPERATION / REBUILD PHASE

Set up servers and workstations Install software as necessary Restore data Install additional workstations as they arrive

12


PROCEDURES DURING RETURN HOME PHASE

Notify users Ensure data is backed up Relocate equipment

AFTER THE DISASTER

Make recommendations on how the disaster recovery plan can be improved

NETWORK RESPONSIBILITIES

ACTIVITIES BY PHASE

PROCEDURES DURING DISASTER RECOVERY ACTIVATION PHASE

Determine the requirements for voice and data communications Install the network including lines, routers, switches, controllers and other communications equipment at the alternate location Test the network

PROCEDURES DURING REMOTE OPERATION/ REBUILD PHASE

Operate the backup network When the replacement equipment arrives at the primary site, install it.

PROCEDURES DURING RELOCATION HOME PHASE

Support the primary site network Dismantle the alternate location data center network

AFTER THE DISASTER


13


SOFTWARE RESPONSIBILITIES

ACTIVITIES BY PHASE


Provide technical support to the other teams Build servers and workstations Reinstall and configure systems at the primary site Test the hardware and software Work with appropriate vendors to assist in recovery Verify that the systems are performing as expected

PROCEDURES DURING REMOTE OPERATION / REBUILD PHASE Provide technical support to the other teams Build servers and workstations Reinstall and configure systems at the primary site Test the hardware and software Work with appropriate vendors to assist in recovery Verify that the systems are performing as expected


Provide technical support to the other teams Verify that the system is performing as expected


OPERATIONS RESPONSIBILITIES

ACTIVITIES BY PHASE


Inventory and select the correct backup files Transport files to the alternate data center Assist all teams in restoring the production environment at the alternate site


Establish a production schedule at the alternate location Run the daily schedule at the alternate location Perform system and production backups at the alternate location Assist other teams in preparing the primary site Establish offsite storage at the alternate location

14



Perform system and production backups Inventory all files at the alternate site Transport all files from the alternate data center to the primary site

AFTER THE DISASTER


TECH SUPPORT TEAM CALL CHECKLIST






15


FACILITY TEAM

Sub-teams: Salvage Team, New Data Center and New Hardware Team

SALVAGE RESPONSIBILITIES

ACTIVITIES BY PHASE

PROCEDURES DURING DISASTER RECOVERY ACTIVATION PHASE Establish the command center Assist in the immediate salvage operations Contact insurance representatives Inventory all equipment primary site. If necessary, involve the vendors.

PROCEDURES DURING REMOTE OPERATION/DATA CENTER REBUILD PHASE

Salvage equipment and supplies Settle property claims with the insurance company Provide for security at the primary location

AFTER THE DISASTER


NEW DATA CENTER RESPONSIBILITIES

ACTIVITIES BY PHASE


Determine the requirements for a new location Work with contractors and City staff on the details Oversee the construction of the new location


Ensure that all controls are working as designed

AFTER THE DISASTER


16


NEW HARDWARE RESPONSIBILITIES

ACTIVITIES BY PHASE


Obtain a list of damaged and destroyed equipment

PROCEDURES DURING REMOTE OPERATION/DATA CENTER REBUILD PHASE

Determine what new hardware should be ordered Order new hardware Arrange for installation and testing of the new hardware

AFTER THE DISASTER


FACILITY TEAM CALL CHECKLIST






17


SEQUENTIAL LIST OF DISASTER RECOVERY TASKS

DISASTER ASSESSMENT PHASE

Task Number

Prior Task

Description Team X

A1

A2

DISASTER RECOVERY ACTIVATION PHASE

Task Number

Prior Task

Description Team X

B1

B2

18


ALTERNATE SITE OPERATION / REBUILD PHASE

Task Number

Prior Task

Description Team X

C1

C2

RETURN HOME PHASE

Task Number

Prior Task

Description Team X

D1

D2

SOFTWARE LICENSE KEYS/ACTIVATION CODES

Software License Key Notes

19


SERVER RECOVERY GENERAL TASK CHART

Task Number

Prior Task

Description Team X

S1

S2

NETWORK GENERAL TASK CHART

Task Number

Prior Task

Description Team X

N1

N2

VOICE RECOVERY

Task Number

Prior Task

Description Team X

V1

V2

20


PROCEDURES FOR FORWARDING CALLS TO ANOTHER LOCATION

[ENTER DETAILS HERE]

DISASTER RECOVERY PLAN UPDATE LOG

Section Description of Change Date

21


DISASTER RECOVERY TRAINING LOG

Name Signature Date

22


TEST SCRIPT TEST DATE: _ _/_ _/__ TEST # ____

Estimated Start Actual Start Finish Step Description

23


TEST EVALUATION

TEST DATE _______ TEST # ____

The Disaster Recovery Coordinator is responsible for coordinating the review and analysis of the test results and updating the plan accordingly. The test participants should document the test results immediately after the plan test. The Disaster Recovery Coordinator reviews the test results with the teams during a Postmortem meeting to discuss weaknesses and resolve problem areas. The Disaster Recovery Coordinator makes changes and updates to the plan accordingly.

1. Were the test objectives met?

2. What problems were encountered?

3. During the test, were there any deviations from the plan?

4. Were all of the materials used during the test retrieved from an offsite source? If not, what items from the data center or on-site offices were used?

24


PERSONNEL LISTING

This list should contain the contact information for all employees engaged in the disaster recovery effort.

Last First Team Home Phone Cell Phone

VENDOR LISTING

Company Name Contact Phone Number Email Vendor Type

25


DAMAGE ASSESSMENT AND SALVAGE CHECKLIST

This section contains checklists to help the Facilities and Hardware teams assess the damage to the systems and building. Once the assessment is complete, notify the Management Team of the results of the assessment, and coordinate salvage of equipment where possible.

A. Assess the requirement for physical security to minimize possible injury to unauthorized personsentering the facility of eliminate the potential for vandalism to [COMPANY] assets.Initials: __________ Date: __________ Time: __________

B. Utilizing the following checklist as a guideline, survey the systems and data center facilities to assess damage upon notification from the Management Team of the need for damage assessment.

1. Buildinga. Exteriorb. Interior

2. Computer Rooma. Wallsb. Ceilingc. Floor

3. Environmental/Controla. Electrical

i. UPSii. Transformersiii. Emergency/Building

b. HVACi. Air Handlingii. Air Conditioning

4. Fire Suppression5. Data Center Contents

a. Serversb. External Disk Drivesc. Backupd. Network Cablinge. Communicationsf. Workstationsg. Other Equipmenth. Spare Partsi. Documentation

26


The purpose of the above checklist is to provide a guide in the review and assessment of damage following a disaster to [COMPANY] facilities, the network and/or the data center. In using the checklist, the Damage Assessment and Salvage Teams must consider:

1. Is the area safe for employees or vendors to work in?

2. Can the equipment under examination function, and if so, at what percent of normal capacity?

3. What must be done to recover damaged equipment?

4. How long will it take to repair or replace the damaged equipment?

Initials: __________ Date: __________ Time: __________

C. Using the damage assessment, determine the estimated time to recover based on the followingguidelines.

Level I Minimal damage to facility and/or equipment. Estimated time to complete repairsis less than 4 hours.

Level II Moderate damage to facility and/or equipment. Estimated time to completerepairs is between 4 hours and 2 business days.

Level III Extensive damage to facility and/or equipment. Estimate time to complete repairsis greater than 2 business days.Initials: __________ Date: __________ Time: __________

27


DAMAGE ASSESSMENT AND SALVAGE LOG

Equipment Assessed

Type of Damage Comments Days to repair

Initials

28


EMERGENCY TELEPHONE NUMBER

Person or Organization Telephone

Local Police

State Police

FBI Regional Office

Environmental (Leaks, Spills)

Local Emergency Management Office

Ambulance

Fire Department

Electric Company

Telephone Company

Internet Service Provider

29

Documents

Disaster recovery plan template€¦ · Web viewThis disaster recovery plan contains resources and procedures to be used in the event that a disaster occurs at one of [Company] locations