Storage Fault Tolerance in Hyper-Converged Clouds running Red Hat OpenStack Platform

Asmita JagtapSenior Principal Software Engineer2nd May 2017

Stor

age

Faul

t Tol

eran

ce

High Availability of Application Data

Directly impacts Business Continuity

Outages are expensive!

Tier-1 workloads - the most demanding apps and that need protection

OpenStack Hyper-converged Environment

Cinder Storage Nodes

Network switch

Host OS Host OS

Nova Compute Nodes

VM VM VM VM VM VM VM VM

Hypervisor HypervisorHypervisor Hypervisor

Storage Faults in hyper-converged environments

Storage service

HDD

HDD

Storage service

Storage connectivity loss

Storage media failure Storage service dead

Host OS/hardware failure Network failure

Fault Tolerance Mechanisms

• Need extra storage for full copy of data

• No impact to I/O performance

• Zero RPO

Mirroring & Striping

• Need extra storage for erasure codes

• Some impact to I/O performance

• Zero RPO

Erasure Coding

• Need extra storage for snapshot data

• Indirect impact to I/O performance

• RPO up to last snapshot/backup

Snapshots &Backups

Backup

ComputePlane• RunningVMs• 1FullcopyofVM+deltachange

• OptimizedforrandomIOs• SetIOpoliciesforperformance

VM VM VM VM VM VM

HyperScale

FullImage Delta Delta

HyperScaleHyperScale

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

HyperScale

DataPlane• Point-in-TimecopiesofVMs

• Realtimefaultrecovery• RapidVMprovisioning

• “Off-hosting”– RunningVMsnotimpactedbyBackupoperations

• OptimizedforsequentialIOs

Introducing Data Plane!

VeritasTM HyperScale for OpenStack

Compute Nodes (Primary)

PeriodicdeltachangessyncedtotheDataNode

VM VM

Storage Fault handling in Veritas HyperScale

Compute Nodes (Primary)

VM VM VM VM VM VM

FullImage Delta Delta

HyperScaleHyperScale

Backup

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

VM VM

HyperScale

Storage service

HDD

HDD

Storage service

StorageHardwareFailureoncompute:

• RedirectI/Os toreflectiontarget• Storageownershipfailover• Restorereflectionfactor• LivemigrateVM

HyperScale

Storage Fault handling in Veritas HyperScale

Compute Nodes (Primary)

VM VM VM VM VM VM

FullImage Delta Delta

HyperScaleHyperScale

Backup

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

VM VM

HyperScale

Storage service

HDD

HDD

Storage service

StorageServiceFailureoncompute:

• RedirectI/Os toreflectiontarget• Storageownershipfailover• Servicerestart• Storageownershipfailback

HyperScale

Storage Fault handling in Veritas HyperScale

Compute Nodes (Primary)

VM VM VM VM VM VM

FullImage Delta Delta

HyperScaleHyperScale

Backup

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

VM VM

HyperScale

Storage service

HDD

HDD

Storage service

ComputeHostOS/hardwarefailure:

• Auto-evacuateallVMs• Restorereflectionfactor

HyperScale

Storage Fault handling in Veritas HyperScale

Compute Nodes (Primary)

VM VM VM VM VM VM

FullImage Delta Delta

HyperScaleHyperScale

Backup

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

VM VM

HyperScale

Storage service

HDD

HDD

Storage service

StorageFailureorHostOS/hardwareondatanode:

• NoimpacttoVMsoncompute• Datanodefailover• RestoreDNreflectionfactor

HyperScale

Storage Fault handling in Veritas HyperScale

Compute Nodes (Primary)

VM VM VM VM VM VM

FullImage Delta Delta

HyperScaleHyperScale

Backup

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

VM VM

HyperScale

Storage service

HDD

HDD

Storage service

Storageservicefailedondatanode:

• NoimpacttoVMsoncomputeDatanodefailover

• Servicerestart• Datanoderesync

HyperScale

Storage Fault handling in Veritas HyperScale

Compute Nodes (Primary)

VM VM VM VM VM VM

FullImage Delta Delta

HyperScaleHyperScale

Backup

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

VM VM

HyperScale

Storage service

HDD

HDD

Storage service

Networkfailure:• NoimpacttoVMsoncompute• Datanodeservicesunavailable• Restorenetwork• Re-enabledatanode

HyperScale

Storage Fault handling in Veritas HyperScale

Compute Nodes (Primary)

VM VM VM VM VM VM

HyperScale

FullImage Delta Delta

HyperScaleHyperScale

Backup

Data Nodes (Secondary)

HyperScale HyperScale

Network switch Periodic sync

VM VM

HyperScale

StorageFaultTolerancesummary:

Forhotdata,atcomputeplane• 1Fullcopyandreflecteddeltawrites• SpaceoptimizedusageofSSDtier

Forwarmdata,atdataplane• 1Fullcopyandincrementalsnapshots• InexpensivecommodityHDDtier

Forcolddata,atexternalbackupserver• Scheduledperiodicbackups• Noimpacttoprimarycompute

VeritasTM HyperScale for OpenStackSoftware-defined storage designed for OpenStack

Backup

Veritas HyperScale Value Props

SimplifiedStorage Management GUI

Compute andStorage Scalability

Predictable I/O Performance

No noisy neighbours!

Integrated Zero WindowBackups

Storage resiliency forall data tiers

Storage Fault Tolerance for Red Hat OSP in action!

THANK YOUplus.google.com/+RedHat

linkedin.com/company/red-hat

youtube.com/user/RedHatVideos

facebook.com/redhatinc

twitter.com/RedHatNews