Bulletproof Your Linux Environmentvox.veritas.com/legacyfs/online/veritasdata/SM B11.pdf · 2016-07-04 · •SAN aware load balancing key for high VM density •Proactive failover

Bulletproof Your Linux Environment

Chris Johnson

Senior Principal Technical Product Manager

Paul Belk

Director Technical Product Management

Bullet Proof Linux


• Scale out workloads for faster performance

• Fast failover

• Reduce Storage Consumption

Future Proof Scalability

• Deploy HA/DR without complex and hard to maintain scripts

• Enable non-disruptive maintenance operations

• Robust support for server virtualization

Enterprise Grade HA/DR

Production

DR

• Improve operational scale

• Reduce storage costs with freedom of choice

• Migrate storage online with no application impact

Maximize Data Center Investment

2

Reduce Complexity Automate High Availability and Disaster Recovery

Metropolitan HA (Stretch Cluster)

Wide-Area DR (Global Cluster)

Local HA • Recover faster

• Reduce reliance on personnel during an

incident

• Reduce operator error

• Provide comprehensive data and application

availability

• Simplify by using a single solution for:

– Local HA

– Campus/Metro DR

– Global DR

The Benefits Symantec High Availability and Disaster Recovery

SAP APP 1 APP 3 APP 2 SAP SAP APP 4

Synch Replication or Mirroring

APP 4 APP 1 APP 2 APP 3

Asynchronous Replication

Bulletproof Your Linux Environment 3

From full upgrade to rolling upgrade Reduced application downtime during upgrade


CFS

CVM

DMP LLT

GAB CFS

CVM

DMP LLT

GAB

HAD HAD

CFS*

CVM*

DMP* LLT*

GAB*

N1 N2

Old Version

New Version

CFS*

CVM*

DMP* LLT*

GAB*

HAD* HAD*

• Upgrade without any application downtime

• Two step process fully automated by Installer (CPI)

• Supported from 5.1 onwards for SFHA, CFS, CFSHA, SFRAC

Application Application

4

Simulate DR Failover: FireDrill

Test Your HA/DR Strategy With No Disruption

• Schedule and run actual application failovers at DR site

• Diagnose with built-in reporting

Primary Site

Volume Snapshot

Initiate Fire Drill Mount Snapshot

Replication

Test Application Reset

Secondary Site


Accelerate Recovery: Intelligent Monitoring Framework (IMF)

• Registration, Blocking, Notification and Remediation

– Agent registers resource with the AMF Kernel Driver and provides it resource specific information

– Agent blocks and awaits notification on resource status change

– Resource status change instantaneously relayed to the blocking agent for handling

• Enable extremely fast failure detection and remediation

• Significant reduction in system resource utilization thus devoting key resources for other processing

HAD

Agent Framework

Intelligent Monitoring Framework (IMF)

Resources

IMF: Agents loosely coupled with resources, wait on

the AMF driver for notification

Registering…

AMF Driver

Faulted


Maximize Availability Prevent Concurrencies

Node with Application Offline Node with Application Online

APP

Registering…

Application is online Application is Offline

VCS brings resource online

Agent registers with IMF

User attempts to bring app online outside of cluster

APP

IMF recognizes attempt

PCV prevents

• Works on Application Agent with VCS 6.0 • The only clustering technology with this feature • Preventing startup scripts is the future direction

• Prior to IMF-PCV, concurrency detection took up to 5 minutes

• No additional monitoring required as part of IMF framework


Why Fencing?

Prevent Downtime Eliminating Corruption Risk with Fencing

APP APP

•Need to restrict writes to current and verified nodes

SCSI-3 Based Fencing CPS Based Fencing

SCSI3 PR Disks

• SCSI3 disks for i/o fencing

• Maximum data protection

• Non SCSI3 fencing

• Virtualized environment

Coordination Point Servers


Data Corruption

APP APP

8

Prevent Downtime Control Which Node / Site Should Survive a Split-Brain

• Preferred fencing provides granular control to influence fencing outcome • Particularly useful for campus clusters and multi-node clusters where cluster

heartbeat goes over an actual network • Preference is configured as a ‘weight’ per node or service group

N1 (Active)

N2 (Standby)

N1 (Active)

N2 (Down)

Eliminate the Guess Work and Maximize Uptime


Multi-Tier Applications & Services

Database

Application

Web server

Billing

app app app

DB

FS IP

VVR

Service Group

VCS

App

IP

Service Group

VCS

Web

VM

Service Group

Web

VM

Service Group

ApplicationHA

Veritas Cluster Server HA/DR

Veritas Cluster Server HA

Symantec ApplicationHA

Veritas Operations Manager


Transition to Virtual Business Service Seamlessly

Database SG

Application Server SG

Web Server SG

Billing Virtual Business Service


Start/Stop Order

Inter-cluster fault policy

Database

Application

Web server

Billing

app app app

DB

FS IP

VVR

Service Group

VCS

App

IP

Service Group

VCS

Web

VM

Service Group

Web

VM

Service Group

ApplicationHA


Virtual Business Service Unparalleled Support Matrix

Operating Systems

Solaris

HP-UX

AIX

Linux

Windows

Virtualization

VMware

OracleVM, Zones

WPAR, LPAR

KVM

Hyper-V

High Availability

VCS 5.1, 6.0

VCS 5.1 SP1 (Unix)

VCS 5.1 SP2 (Win)

ApplicationHA 5.1 SP2

ApplicationHA 6.0

Databases

Oracle

Sybase

SAP

DB2

MySQL

Applications

Informatica

WebSphere

Microsoft Exchange

WebLogic

PeopleSoft

Replication

Veritas Replicator

Oracle Dataguard

EMC SRDF

NetApp SnapMirror



Robust Virtualization Support

13

VCS Guest Cluster for 24x7 Application Availability

24x7 Intelligent Monitoring by VCS

• Detailed application awareness

• Infrastructure/VM Availability

• Traditional clustering

Rapid Application Recovery

• Fast failover to standby VM

• Integrated with SFCFS

Minimize Application Downtime

• Protection against OS corruption

• No downtime for OS patching

Simplified Management

• App visibility through Veritas Operations Manager

• Fully supported with Virtual Business Services

VM1

OS

VM2

OS

HOST

SQL ORA

VM1

OS

VM2

OS

HOST

Web

VCS VCS

ORA


ApplicationHA for application visibility and management

Simplify Application Management

• Manage app dependencies

• Start/Stop/Status of in-guest app

Fault detection and remediation

• Detailed application monitoring

• Application restart to remediate faults; no App/VM failover

Easy Configuration

• Automatic application discovery

• App wizards for configuring HA

• No private interconnects

Simplified Management

• In-guest app visibility through Veritas Operations Manager

• Fully integrated with Virtual Business Services for multi-tier application management

VM2

OS

HOST

ORA

ApplicationHA

VM1

OS

SQL

VM3

OS

APP

ApplicationHA

ApplicationHA


Virtualization puts increased pressure on storage performance and availability


• SAN aware load balancing key for high VM density

• Proactive failover of storage connectivity during disaster

• Flexibility to choose ANY cost effective storage

• Optimized handling of ANY tier of storage

• Visibility into storage characteristics needed

• Data center wide I/O bottlenecks critical to isolate

Performance & Resiliency

Storage Heterogeneity

Visibility & Reporting

RAID10, SRDF, Thin , SSD drive

I/O bottlenecks to storage arrays!

Lack visibility into performance bottlenecks!

Getting locked-in by single array vendor

Am I using the right tier of Storage? Device naming complicated in ESX!

16

Veritas Dynamic Multi-Pathing (DMP) for VMware: Improved storage I/O performance and availability

• Improve performance

– SAN & workload aware load balancing

– Real-time I/O performance graphs

• Increase availability

– Proactive path failover

– Automated path recovery

• Improve storage visibility

– Enhanced device naming

– Extended attribute discovery & reporting

– Data center storage reporting via vCenter

• Utilize a single multi-pathing solution across physical and virtual with heterogeneous storage


Any X86 Server

VMware ESX

Hypervisor

ESX PSA Framework

MPP DMP

Guest Guest Guest Guest

17

VxDMP Components and Architecture

• vSphere / ESX Components

– Dynamic Multi-Pathing Core MPP components

– ASL / APM packages

• Management Interfaces

– vCenter integration for operations, visibility and reporting

– CLI Server for remote administration

• vCenter Functionality

– I/O policy management

– Path management (enable/disable)

– Performance graphs and tables

– DMP deployment statistics


Storage

ESX Servers

vCenter Server

VxDMP plug-in in vSphere client

SAN VxDMP

Remote CLI

LAN

18

KVM: SFCFSHA in the KVM Host


SFCFSHA in the Host:

•Replication of VM Images. •Highly available storage for virtual machines. •Simplified management of virtual machines. •Centralized Device Path Management

KVM Guest

RH6 KVM Host

KVM Guest

Shared Storage

Bo

ot

Dev

ice

App App

Dat

a D

evic

e

Bo

ot

Dev

ice

Dat

a D

evic

e

KVM Guest

KVM Guest

Bo

ot

Dev

ice

App App

Dat

a D

evic

e

Bo

ot

Dev

ice

Dat

a D

evic

e

SFCFSHA

RH6 KVM Host

19

KVM: SFHA in the Guest, DMP in the KVM Host


Using SFHA in the Guest Provides:

•Enhanced database performance. •Point-in-time copy features for data back-up, recovery, and processing. •Consistent Device Naming Using DMP in the host provides: •Centralized multi-pathing functionality. •Active/passive array high performance failover. • Centralized storage path management. .

KVM Guest

RH6 KVM Host

KVM Guest

Shared Storage

Bo

ot

Dev

ice

App App

Dat

a D

evic

e

Bo

ot

Dev

ice

Dat

a D

evic

e

KVM Guest

KVM Guest

Bo

ot

Dev

ice

App App

Dat

a D

evic

e

Bo

ot

Dev

ice

Dat

a D

evic

e

RH6 KVM Host

DMP DMP

Storage Foundation + DMP

20

KVM Storage Management (SF 6.0)


RHEL Host

RHEL Host

KVM KVM

RHEL Host

KVM

DMP

VM2

OS

SF

VM2

OS

SF

CVM/CFS

DMP

VM2

OS

SF

VM1

OS

SF

VM1

OS

DMP

• SF in the guest

– get consolidation benefits, keep

tools/processes unchanged

– enable P2V data migration with PDC

• DMP in the host

• CVM/CFS in the host

– present files (& filesnaps) as disks to guests

– enable live migration

– TP/R in the host

• Split stack (FS in guest)

• virtIO disk type support

• RHEL6

21

VCS High Availability in KVM Environment

In Guest Application Availability • Lean VCS footprint installed inside KVM Guest • No heartbeats, no additional network infrastructure • Start, Stop, Monitoring, Restart of Application • Support for Windows (6.0) and RHEL (5.1 SP1)

VM B

VM C

VM A

APP

KVM HOST

APP APP

VM Y

KVM HOST

VM X

APP APP

VM/HA without Application Awareness • VCS installed in KVM Host to form a cluster of KVM Hosts • Start, Stop, Monitoring, Failover of KVM Guest • No VCS footprint inside KVM Guest • Support for Windows (6.0) and RHEL (5.1 SP1)

VM B

VM C

KVM HOST

VM A

VM B

VM C

VM A

KVM HOST

VM/HA with Application Awareness • Lean VCS footprint in KVM Guest for Application Monitoring • VCS in KVM Host for KVM Guest & Infrastructure Monitoring • Application restart, VM failover (configurable levels) • Support for Windows (6.0) and RHEL (5.1 SP1)

VM B

VM C

KVM HOST

VM A

VM B

VM C

VM A

APP

KVM HOST


Bullet Proof Linux



• Fast failover







Production

DR





23


Improve Operational Scale

24

Veritas Operations Manager (VOM)

Finance VBS

AppHA AppHA AppHA


AppHA AppHA

Billing VBS

AppHA AppHA

HR VBS

VCS

VCS

VCS

CFS HA

SF HA

Finance Dept Billing Dept HRDept


Improve Operational Scalability Unified management across platforms

Manage Veritas Cluster •Configure Service groups using wizards

•Perform availability operations using simple menus

•Monitor clusters and generate alerts

Manage Storage Foundation •Manage Disk Groups, Volumes and File Systems using wizards

•Monitor and setup copy services •Monitor and control volume replication

VOM


Multi-Cluster Reports


Cluster uptime analysis reports

One chart with service group

state across all clusters

Ability to click on the graph and do batch operations on service groups

27

VOM

Cross Cluster Reports


Cross cluster failover summary

Date wise failover distribution

Service group wise failover distribution

28

VOM

Health Checks


POLICY THEME Sample health check to include

1. Ensure availability frozen service groups

2. Maximize utilization underutilized disks and file systems

3. Enforce best practices Cluster OS consistency

Pick from 50 built-in health checks

Ability to write custom

health checks

Run & schedule health checks with a simple click

Recommend creating policies around the following 3 themes

29

VOM

Create custom templates for policies

Use Customizable signatures to comply with best practices,

availability and utilization policies


VOM

Automate actions on faults

Configure rules to take action on fault

types


VOM

Exploiting ANY Storage: Modular Architecture to Optimize to Any Storage

• Array Support Library (ASL)

– ASL Module for each Array (Ex: VMAX, NTAP)

– Discovers Array Mode Types (A/A, A/P, ALUA)

– Discover array specific attributes (Tier, RAID, Repl)

• Array Policy Module (APM)

– HW specific plug in to DMP

– Dynamically loadable module

– Understands Array specific failover behavior

– Allows Non Disruptive Upgrades for Controllers

• Allows DMP to support 50+ array families & 1000+ array models


Any Storage

VMware ESX

Host Host Host Host

APM2

APM1

DMP

ASL1

ASL2

32

Improve operational efficiency and reduce complexity by using a single multi-pathing solution


Dynamic Multi-Pathing Enables Storage Flexibility on all Platforms

• Leverage extensive array support

• Standardize on single multi-pathing across storage management stacks

• Seamless migration from native multi-pathing to Dynamic Multi-Pathing

• Choose storage based on price / performance needs, avoiding lock-in

Reduce operational costs using a single multi-pathing solution

Veritas Cluster Server

Volume Manger

File System

DMP


DMP

Veritas Storage Foundation

ESX PSA Framework

MPP DMP for VMware

Veritas Application HA

Guest Guest Guest

VMware ESX

Guest

33


Faster Migrations

Change with Minimum Disruption

34

Migration Steps Cross Platform Migration

Migration using traditional methods (network copy)

1. Provision Storage

2. Set up for copy process

3. Stop Oracle instance

4. Start copy process

– Using NFS, rcp/scp or similar

5. Wait for data transfer

– Can take many hours or days

6. Start Oracle instance


Challenges with traditional methods: Requires double the storage

Require a high bandwidth network connection Will most likely require a long application outage to copy data

35

Migration Steps Cross Platform Migration

Migrating an Oracle Database between hardware platforms using Storage Foundation

1. Configure CDS

2. Stop Application

3. Validate file systems and convert byte order if necessary (to/from Linux)

4. Deport Disk Group

5. Import Disk Group on new host

6. Start Application


Perform Storage Migrations Online Over any distance


Migrate Arrays Locally Migrate Over Distance

• Wizard based configuration

• Configured during business Hours / Scheduled for non-Business hours

• Unattended Migration

• No Application Downtime

Veritas Storage Foundation SF, VCS

Synch/Asynch Replication

SF, VCS

• Performed Over IP without expensive Infrastructure

• Data Center Migration

• Allows for easy back out if issues detected

Utilize SmartMove to go Thin!

38




• Fast failover







Production

DR





39

CFS HA

Storage Foundation Cluster File System (SFCFS)


CVM

High-performance Low-overhead architecture Scalable architecture Sub-minute failover

Highly-available Robust, long-lived product Supported by entire Storage

Foundation software stack Local, campus, or WAN cluster options

SAN storage

From 1 to 64 nodes

256 TB Max FS

40

SFCFS Use Cases: Scale-Out Applications


Performance for Grid/Parallel Apps

• Minimal application overhead

• File access scales linearly as additional

nodes are added

• Single namespace for database copies

41

SFCFS

http://www-306.ibm.com/software/websphere

Maximize Availability / Minimize Complexity Eliminate Storage Failovers with CFS

Classic Clustering May Not Be Fast Enough Business Applications

Needing Maximum Uptime

Passive Server

Failed Server Active Server

Client

Active Server

Recovery Steps

• Detect failure • Un-mount file

system • Deport disk group • Import disk group • Mount file system • Start application • Clients reconnect

File System

Native clustering

File System

Native clustering


Failover as Fast as Application Restart

Maximize Availability / Virtualize Confidently Eliminate Storage Failovers with CFS

Failed Server

Client

Active Server





Veritas Cluster File System

Business Applications

Needing Maximum Uptime

Recovery Steps

• Detect failure • Un-mount file

system • Deport disk group • Import disk group • Mount file system • Start application • Clients reconnect


Hyper-V

43

http://www.linux-kvm.org/page/Main_Page


DB Clones

Compress De-Dupe

Compression Cloning & Snapshots De-duplication

Increase host storage efficiencies Compress where applicable Reduce storage consumption using

cloning and snapshot efficiently Reduce data duplication Enable thin provisioning & reclamation

Storage Foundation Cluster File System

Thin Reclamation

Store Information More Efficiently

44

http://www.iconfinder.com/icondetails/40763/128/





















• Fast failover







Production

DR





45

Documents

Bulletproof Your Linux Environmentvox.veritas.com/legacyfs/online/veritasdata/SM B11.pdf · 2016-07-04 · •SAN aware load balancing key for high VM density •Proactive failover