Calculating Downtime Costs: How Much Should You Spend on DR?

1

CALCULATING DOWNTIME COSTS:How much should you spend on DR?

Paul Croteau – Enterprise Cloud StrategistRackspace Hosting

2

Agenda

Downtime: The NumbersBuilding Your CaseManaging ExpectationsReference ArchitecturesRoles & ResponsibilitiesQ&A

3

DowntimeThe Numbers

4

Outages Happen

59% of Fortune 500 companies

experience a minimum of 1.6 hours of

downtime per week, according to Dunn

& Bradstreet.

5

F500 2012 Hourly Loses

Total 2012 Revenue = $11.75T

Total 2012 Profit = $824B

• Ave. F500 Revenue = $23.5B

• Med. F500 Revenue = $10B

• Ave. F500 Profit = $1B

• Med. F500 Profit = $646M

6

F500 2012 Hourly Loses

Total 2012 Revenue = $11.75T

Total 2012 Profit = $824B

• Ave. F500 Revenue = $23.5B

• Med. F500 Revenue = $10B

• Ave. F500 Profit = $1B

• Med. F500 Profit = $646M

($2.7M/hr)

($1.2M/hr)

($122k/hr)

($74k/hr)

7

Minutes Matter• The average cost of data center downtime across industries:

approximately $5,600 per minute.

• For a partial data center outage, averaging 59 minutes in length, average costs were approximately $258,000.

• For total data center outages, which had an average recovery time of 134 minutes, average hourly costs were approximately $680,000.

• 93% of companies that lost their data for 10 days or more filed for bankruptcy within one year of the disaster, and 50% filed for bankruptcy immediately.

8

Humans Make Mistakes

Through 2015, 80% of outages impacting mission-critical services will be caused by people and process issues, and more than 50% of those outages will be caused by change/configuration/release integration and hand-off issues. – Gartner Research

9

Building Your CaseQuantifying Risk

10

Do You Have A Plan?

41% of SMBs surveyed said that putting together a Disaster Recovery plan never occurred to them.

Less than half of SMBs back up their data weekly or more frequently, and only 23% backup daily.

Backups are not enough! The goal of a backup is to enable data restoration. A DR plan helps quickly restore operations.

DR is a holistic strategy for restoring IT systems that powers business ops that includes people, process, policies and technology.

11

From minutes to weeksDowntime Perspective

How Resilient Is Your DR plan?– Device failure– Cabinet failure– Facility failure

Time To Recovery

12

Cost of Downtime Scenarios

Annual Revenue App/ProductivityAnnual Revenue $15,000,000 Annual Revenue $75,000,000

Percentage of Revenue from Online 90% Number of employees 400Average shopping hours per day 12 Annual revenue per employee $187,500

Annual total revenue hours 4380 Work hours per year (2000 hours/employee) 500,000Cost of downtime per hour $3,082 Employee revenue per hour $150

Hours of downtime 10Sales Lost Percentage employees affected by downtime 20%

Duration of Event (days) 4 Cost of downtime per hour $375,000 Hours of event 96

Expected visits generated 500,000 Event RevenueConversion rate (visits to purchase) 6% Expected Event Revenue $100,000

Average revenue per purchase $500 Event Duration (days) 3Revenue per event $15,000,000 Event hours 72

Cost of downtime per hour $156,250 Cost of downtime per hour $1,389

If you don’t know your actualcost of downtime,

you are wasting time.

13

Annual Revenue BasisCost of Downtime Scenarios

Annual Revenue

Annual Revenue $15,000,000

Percentage of Revenue from Online 90%

Average shopping hours per day 12

Annual total revenue hours 4380

Cost of downtime per hour $3,082

14

Cost of Downtime Scenarios

Event Revenue

Expected Event Revenue $1,000,000

Event Duration (days) 3

Event hours 72

Cost of downtime per hour $13,089

Single Event Revenue

15

Cost of Downtime Scenarios

Sales LostDuration of Event (days) 4

Hours of event 96Expected visits generated 500,000

Conversion rate (visits to purchase) 6%Average revenue per purchase $500

Revenue per event $15,000,000Cost of downtime per hour $156,250

Sales Lost

16

Cost of Downtime Scenarios

App/ProductivityAnnual Revenue $75,000,000

Number of employees 400Work hours per year (2000 hours/employee) 500,000

Employee revenue per hour $150 Annual revenue per employee $187,500

Hours of downtime 10Percentage employees affected by downtime 20%

Cost of downtime per hour $120,000

Productivity Basis

17

Get The Downtime Calculator

rackspace.com/dt-cost

18

ExpectationsImplication of Time

19

RPO / RTO

WeeklyBackup

WeeklyBackup

WeeklyBackup

20

RPO / RTO

WeeklyBackup

WeeklyBackup

WeeklyBackupOUTAGE

21

RPO / RTO

RPO

WeeklyBackup

WeeklyBackup

WeeklyBackupOUTAGE

22

RPO / RTO

RPO

WeeklyBackup

WeeklyBackup

WeeklyBackup

RTO

OUTAGE

23

RPO / RTO

RPO

WeeklyBackup

WeeklyBackup

WeeklyBackup

RTO

RecoveryCompleted

OUTAGE

24

RPO / RTO

Recovery Point ObjectiveHow much data is lost

Recovery Time ObjectiveHow long to recover

Weeks Days Hours Min Sec Sec Min Hours Days Weeks

25

RPO / RTO

Recovery Point ObjectiveHow much data is lost

Recovery Time ObjectiveHow long to recover

Weeks Days Hours Min Sec Sec Min Hours Days Weeks

Tape

PeriodicReplication

Snapshots

Replication Clustering

Snapshots

Tape Restore

26

RPO / RTO

Recovery Point ObjectiveHow much data is lost

Recovery Time ObjectiveHow long to recover

Weeks Days Hours Min Sec Sec Min Hours Days Weeks

Tape

PeriodicReplication

Snapshots

Replication Clustering

Snapshots

Tape Restore

CostImpact

27

RTO/RPO Cost ExpectationsHOT COLDWARM

RTO

RPO

Tier

• DNS Failover

• Array-based Replication

• Host-based Replication

• DB Replication (Transactional)

• DB Rep. (Log Shipping)

$$$ $$ $

0-24

2-6

0-24

4-24+

24-48+

1 2 3 4

0-2

• MBU (Disk)

• VM Replication

Price

• MBU (Tape)

• MBU (Offsite)

Elements of DR, not an end-to-end solution

Missing process, policies and procedures• GSLB

28

ArchitecturesDefined By Your Priorities

29

Designing for Redundancy

HA FirewallsHA Load Balancers

Private Cloud

DB Cluster

SharedStorageDedicated

Storage

Hypervisor

30

Designing for Geo-Redundancy

LUN

VM

vSphere-basedArray-based

Prod DR

31

I need backup!DR Site Requirements

How long must you depend on your DR site?

How do you define your DR site requirements?

DR = Insurance.

32

DR-specific Site

Prod DR

33

DR-specific Cold Site

Prod DR

34

DR-specific Cold Site

Prod DR

35

Staging as Warm DR

Prod DR

36

Staging as Warm DR

Prod DR

37

Staging as Warm DR

Prod DR

38

Staging as DR(expanded)

Prod DR

39

Roles/ResponsibilitiesShared

40

Leverage Expertise

Common questions from customers:

Who owns the overall DR strategy?

Who will design it?

Who is going to manage and monitor it?

Who will perform the failover?

41

Designing Your DR Strategy

Businesses own the strategy.

Vendors enable the strategy.

The strategy is unique to your needs.

Testing matters.

42

Prioritizing Content/Apps

How do you prioritize?

What are you protecting?– Business Operations– Revenue– Data– Customers– All of the above

43

Roles & Responsibilities

Role Responsibility

DR Plan Failover Plan / Run Book Business

“Pushing the failover button” Business

Failover Process Partner

Replication Applications Partner

Virtual Machine Partner

Database Partner

Guest OS Partner

Hypervisor Partner

Server Partner

Storage Partner

Network Partner

Data Center Partner

44

Testing

Companies don’t test their failover plan enough.

Some replication services charge per test: expensive

The failover/back process can be risky in production

Risk dictates extensive planning around every test

45

So, How Much Should You Spend On DR?

How much revenue will you lose?

How much else will you lose?

How much can you afford?

Based business decisions on fact, not emotion.

46

Summary

DR is Your Responsibility

Know Your Cost of Downtime

Prioritize Your Apps

Select The Right Tools

Select The Right Partner

47

Rackspace HostingA Diverse Portfolio

48

The Rackspace Portfolio

PRIVATECLOUD

PUBLICCLOUD

CUSTOMER

PREMISE

PARTNERDATA

CENTER

PRIVATECLOUD

PRIVATECLOUD

VIRTUALIZED

VMware

DEDICATED

BARE METAL

RACKSPACE DATA CENTER

Powered by Powered by Powered by Powered by Powered by

49

Q&A