41
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. John Fox, Principal Technical Product Manager Commvault Cloud Business Unit Shaown Nandi, VP, Head of Infrastructure and Cloud, Dow Jones News Corp. STG305 Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure

AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Embed Size (px)

Citation preview

Page 1: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

John Fox, Principal Technical Product Manager

Commvault Cloud Business Unit

Shaown Nandi, VP, Head of Infrastructure and Cloud, Dow Jones News Corp.

STG305

Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure

Page 2: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Agenda

•What are you looking for in the cloud?

•Key Considerations for DR to the Cloud

• Initial Protection — Data to the Cloud

•Seeding Cloud Instances

•Disaster! – DR Scenario with AWS

•DR Case Study using Commvault and AWS

•Protection in the Cloud – AWS Focused

•Key Takeaways

•Q & A2

Page 3: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

What are you looking for?

Page 4: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Key Considerations for DR to the Cloud

Page 5: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Key Considerations for DR to the Cloud

5

• Multiple Sources that are Interrelated. Physical, Virtual, different OSs and Applications

• Production is Dynamic – The DR Plan for today doesn't work Tomorrow

• Is this a full or Partial DR

• Prioritization – Core, Tier 1, Tier 2 and then Tier 3

• Should Work Versus Knowing it will Work

• Test, Test, Test

• Spot test key Applications – Virtual Application Testing

Page 6: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Getting Data to the Cloud: Step 1

Page 7: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Getting Data to the Cloud

Storage / offsite tape replacement

• AWS and 40+ other storage targets natively supported

• Combined local backup, archive with cloud offsite

• Short-term retention for fast local recoveries

• Dedupe, encryption pre-cloud transfer

• Distributed indexes — data movers are commodity resources, restore from anywhere

• Incremental forever — efficient WAN utilization

• Multiple accounts / account segmentation supported

• Data portability7

Page 8: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Not All Data Can Be Sent Over the Wire

• Large-scale datasets (petabytes)

• De-duplication baseline, or incompressible datasets(video, audio)

• Bandwidth $$

• Physical limitations and reliability of WAN links

8

Dataset Size

Link Size

1 GB 10 GB100 GB

1 TB 10 TB 100 TB 1 PB 10 PB

10 Mbit 14m 2.2 hrs 22.2 hrs 9.2 days 92.6 days – – –

100 Mbit 1m 20s 13m 20s 2.2 hrs 22.2 hrs 9.2 days 92.6 days – –

1 Gbit 8s 1m 20s 13m 20s 2.2 hrs 22.2 hrs 9.2 days 92.6 days –

10 Gbit 0.8s 8s 1m 20s 13m 20s 2.2 hrs 22.2 hrs 9.2 days 92.6 days

Page 9: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Achieving Petabyte Scale Data Transport

CLOUD LIBRARY SEEDING SUPPORT

• Not all datasets can be sent over WAN –physical transportation required for large scale transfers

• Parallelize as needed

• AWS transfer achieved = Transfer more than 1 petabyte per week

• Simplified Logistics

• End-to-end Custody chain

• De-dupe via Commvault

Page 10: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Amazon Storage Class Selection

Ensuring “fit-for-purpose”

• Amazon S3 - Hot

• S3-IA - Warm

• Amazon Glacier - Cold

• Cost models

• Ingress, egress

• Recall thresholds

• Special conditions

• Use cases

• Short- vs. long-term retention

• Backup vs. archive vs. compliance

• Data retrieval policies (Amazon Glacier)

10

Page 11: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Backup to the Cloud

Table stakes – S3, S3 IA, Amazon Glacier

11

Page 12: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

DR Time: Step 2

Page 13: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

How would you recover?

”My DR plan is a pile of tapes and hoping for the best.”

Page 14: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Workload Transformation

VMware Recovery to AWS

14

Page 15: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Replicate and Recover to the Cloud

15

Data Protection Operation is run against VMs to be replicated

After Protection Operation is Complete Changed Blocks are Shipped to Standby VMs

2

OR DASH Copy is Performed to Secondary Site

Then Changed Blocks are Overlaid on Replicated VMs

4

Primary Datacenter

Secondary Datacenter

From VMware to AWS

31

43

21

4

Page 16: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

DR to the Cloud

VMware recovery to AWS — Recovery and Replication

Let’s See it in Action!

16

Page 17: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

“Now that I’m in the cloud, I don’t need backup … right?”

Page 18: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Protection in Cloud: Step 3

Page 19: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Reasons to Back Up in the Cloud

Expect components to fail, design for it – don’t wait for it to happen

Data loss can happen

• Instances retire

• Applications can become corrupted

19

Page 20: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Commvault Cloud Protection

20

Page 21: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

What Should I Be Protecting?

Data classification matters — identifying valued data

Stateful machines

•Protect with VSA, BLB, FS, App agents

Stateless machines — “worker” nodes

•Where’s the valued data written out to? Protect that instead.

Data silos

•Object storage, Amazon EFS, Amazon EBS

21

Page 22: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Virtual Server Agent for AWS

Expanding Capabilities

22

Agent-less backup of an Amazon EC2 instance’s EBS volumes

Use agent-in-guest for Application Consistent protection, and as part of a comprehensive data protection strategy

Page 23: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

What’s New? - EBS Snapshot Management (VSA for AWS context)

• Snapshots auto-rotated based on # of Snaps, or Days (Retention)

• Extraction to S3 optimized snapshot storage model to reduce storage size

• Copy Management features (Multiple Copies, cross-account/providers)

• Known snapshot names (SP_2_10242_2266_1468318241) and Tags to differentiate between Commvault-created and other snapshot

25

Page 24: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Amazon RDS Support

Protecting Database-as-a-Service

26

Protect & Manage Databases -- whether they live in Amazon RDS, on EC2-instances, or both!

• Data protection for Databases running as a service within Amazon RDS

• Customer can protect Databases whether they reside in an EC2 instance, or in Amazon RDS

Supported mechanisms

• Application-consistent Snapshot backup Snap Restore, Snap Replicate

Page 25: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Keeping It Simple

Use the right tool for the jobVSA for Cloud Image: Agentless instance protection

Agent-in-Guest: App-consistent protection (File, SQL, Oracle, many more!), granular app-specific restore features

DBaaS: Snapshot orchestration (coordinate with other systems)

Cloud Snapshot Management: (Oracle on Linux, Linux FS)

27

Page 26: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Protection in Cloud

You Need to Protect once you Fail Over

Protection of AWS workloads

Let’s See it in Action!

28

Page 27: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Dow Jones / News Corp NAShaown Nandi – VP Head of Infrastructure and Cloud

Page 28: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)
Page 29: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

CLOUD TRANSFORMATION JOURNEY

• Save $100 Million annually

• Reduce from 50 data centers globally to 6

• 75% of computational power in the Cloud

AUDACIOUS GOALS SET IN FY14 BY NEWS CORP

Increase in AWS instances

400%

Dow Jones c loud compute

58%

Page 30: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

32

Dow Jones journey to the cloud with Commvault and AWS

1. Challenges

2. Preparation

3. Migration

4. Benefits

5. Cloud of the Future

Provision Cloud Resources Automated Recovery

Cloud Orchestration Workflow Automation

Page 31: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Dow Jones Legacy DR Operation and Challenges

Traditional DR Strategy Included:

• Management of over 30,000 tapes in cycle

• Poor RoI

• Offsite Storage Costs

• Unacceptable recovery SLA’s for mission critical applications

• Limited resiliency

33

Traditional DR Operations

Lowest CostSlowest Recovery

User Data Sets

Files, eMail, Mobile, Laptop, etc

Page 32: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Preparation & Planning

Leveraged Commvault and AWS Extensive Planning Resources

34

• Jointly Validated Reference Architectures for Commvault and AWS• Cloud Transformation Services with Commvault• Commvault Cloud Architecture Guide for AWS• AWS High Availability Best Practices

Commvault and AWS Jointly Validated Reference Architectures

Page 33: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Dow Jones Cloud Based Protection Solution – Real World ResultsResilient, Automated In Cloud Protection with Commvault

35

1. Started sending on-premisebackups to S3 and S3-IA –exploring Glacier

2. Ultimately protect over 4,000 instances in the cloud with Commvault

3. Investigating protected VM’s with Commvault directly into EC2

4. Building out full business continuity plan to include automated disaster recovery testing and failover.

Page 34: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Commvault Vision

Page 35: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Commvault Cloud Vision

•Common Management Platform

•Platforms

•On-premises

•Cloud subscriptions

•Policies

•APIs

•Applications

37

Page 36: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Commvault Ties together on-prem and cloud data strategies

Commvault Orchestrates the Enterprise

• Back up in the Cloud

• Back up to the Cloud

• Disaster Recovery to the Cloud

• Workload Portability

38

AWS and Commvault together combine to minimize networking, storage and

infrastructure costs, while providing the business a sound data protection and disaster

recovery strategy.

Page 37: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Takeaways

Page 38: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Key Takeaways

40

•Consider the Multiple Sources

•Keep the Plan Current

•Know what is going Where

•Bring up the key bits First!

•Test and know it Works

•DR doesn't need to be an Annual Event. Test Applications all year long.

Page 39: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Questions?

Don’t forget to visit us in Booth 1015

Page 40: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Thank you!

Page 41: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)

Remember to complete your evaluations!