View
216
Download
1
Category
Preview:
Citation preview
Designing applications for highly-availability with Windows Server 2012 Failover ClusteringElden ChristensenPrincipal Program Manager Lead3-051
Failover Clustering broad strategyTarget “stateful” workloadsPlatform integrated with Windows serverIndustry-standard hardwarePossible to cluster an application or service with minimal changesFlexible plug-in model that enables a wide variety of scenariosEnable large ISV ecosystem extensionExtensible to allow high availability and disaster recovery in a single solution
Cluster scenariosFailover Clustering is a general purpose platformAlmost any application or service can be made highly available
Stateful applications that require high data integrity benefit the most from being cluster awareTransactional applications, such as SQL serverPersisting state in files: third-party storage/replication, such as Hyper-V
Stateless applications (such as web/IIS) traditionally use network load balancersNetwork load balancing (NLB) is a software load balancer in Windows server
Cluster workloadsFailover Clustering install base is primarily:
If you provide augmenting or integrated software, supporting cluster scenarios is a fundamental requirementClustering is also used by a wide variety of additional as well as 3rd party solutions
Hyper-V File serverSQL Exchange
Services provided for cluster-aware workloadsApplication plug-ins for customized health monitoring & lifecycle control (“resource DLL”)Flexible composition model for describing applications, dependencies, and failover groupsReplicated store for application descriptions, application meta-data, and cluster topology informationFlexible quorum policy to tailor redundancy and failure exposure to the high availability and DR requirements of the applicationLocation-independent client access points for client/server binding to hosted servicesHealth monitoring and watchdog services for failure detection of hardware and software componentsFailover service for placement of app/service components in response to hardware and software failures or automated management Distributed storage for safe sharing of storage devices, including a cluster file system (cluster shared volumes) and cluster integration of storage spacesAPIs and tools for automated management of cluster topology and failover capabilities
Typical Failover Cluster topologyClients
Public Network
Block level storage: Fibre Channel, iSCSI, FCoE,
SAS
Private network for redundancy
Shared storage commonly accessible
Failover Clustering in WS2012 Platform for the private cloudHyper-V high availabilityScalability (more nodes, more VMs)
StorageLower cost and better integrated with storage ecosystemInnovate cluster shared volumes
Flexible deploymentsFailover Clustering now available in standard edition SKU
New Win2012 Failover Clustering features
VM prioritization
and preemption
Faster validation
PowerShell integration with
Hyper-V & storage
8000 VMsEnforced
anti-affinityClustered
storage spaces
Storage space quorum
Bitlocker for CSV
AD-less cluster boot
CSV block cache
iSCSI network auto-detect
Node vote weight
Resource DLL upgrade
Dynamic quorum
Live migration queuing
Node maintenance
mode
Failback live migration
VM app monitoring
Clustered scheduled
tasks
CSV/VSS integration
IP failover in Azure IaaS
UI scalabilityVM intelligent
placementUI integration with Hyper-V
NetFT performance
filter
Sysprep deployment
UI integration with SM+
file/storage
VM storage migration UI
Scale-out file server
OS migration yool for
CSV/Hyper-V
Asymmetric storage
CSV 2.0 64 nodesDistributed
network nameWMI query
optimizationLive migration
integration
Cluster storageProvides an infrastructure for physically shared storageExclusive storage ownership with failoverShared access model
Data integrity based on single node ownershipGuarantees that a single server has ownership of a LUN• Prevents uncoordinated file system access• Resolves ownership disputes when nodes lose
all network communication
Storage hardware requirementsFailover Clusters use SCSI-3 persistent reservations to ensure ownership of a LUNProtects LUN from all other devices on the SANPrevents uncoordinated multi-initiator access, and the potential data corruption
Supports any block level storage typeFibre Channel, iSCSI, FCoE, Serial Attached SCSI (SAS)
Supported with any logo’d storage deviceCluster validation verifies requirementsIntegrated into Failover Clustering feature in Windows server
Device Identification VPD page 83h SCSI inquiry data of any of the following types:
• EUI-64 based – IDENTIFIER TYPE 2h• NAA – IDENTIFIER TYPE 3h• SCSI name string – IDENTIFIER TYPE 8h
Persistent Reservations• PERSISTENT RESERVE IN Read Keys (00h)• PERSISTENT RESERVE IN Read Reservation (01h)• PERSISTENT RESERVE OUT Reserve (01h)
• Scope: LU_SCOPE (0h)• Type: Write Exclusive – Registrants Only (5h)
• PERSISTENT RESERVE OUT Release (02h)• PERSISTENT RESERVE OUT Clear (03h) • PERSISTENT RESERVE OUT Preempt (04h)• PERSISTENT RESERVE OUT Register AND Ignore Existing Key (06h)• PERSISTENT RESERVE OUT Register (00h)
• Only for a Clustered Space in Windows Server 2012
Cluster storage requirementsThe SCSI command requirements for Failover Clustering are that the storage must support the following SCSI-3 SPC-3 compliant SCSI commands:
Prior to Win2008 R2, Failover Clustering implemented a “shared nothing” storage model Disk is owned by a single node at any one time, and only that node can perform I/O to it The application lives with the disk and takes a dependency on it (apps move with disks)
Share-nothing model
Shared storage
Only one node accesses a LUN
at a time
SAN
SQL FCI FailoverClients
Node A Node B
Shared LUN’s
HeartbeatSQL instance Passive
Node
Failure occurs
Reservation
broken
Instance fails over
and is available to clients
SQL instance
New reservation established
What is cluster shared volumesCluster shared volumes (CSV) is a clustered file system in Windows Server 2012Enables all servers in a Failover Cluster to access a common NTFS volumeProvides a layer of abstraction above NTFS
Provides applications complete abstraction with respect to which nodes actually own a LUNApplications can failover without requiring drive ownership changesNo dismounting and remounting of volumes• Faster failover times (aka. less downtime)
All nodes in the cluster have simultaneous read/write access to the same shared storageAll clustered file systems need a mechanism to orchestrate metadata updatesHistorically most clustered file systems have done it storage sideCSV synchronization is done server side • Avoids I/O interruptions
Cluster shared volumes (CSV) I/O synchronization overview
VHDVHDVHD
Shared Storage
Shared LUN
Read/Write
Metadata
How does CSV workCluster service selects single node to “own” each diskSimilar to shared-nothing storage modelEnforced with proprietary SCSI PR based arbitration and ownership algorithm
Disk ownership algorithm extended to permit I/O from other nodes in good stateCluster service creates CSV namespace on all nodesCSV file system routes I/O to the correct destinationDirect block-level to locally attached disk (“Direct I/O”)Redirected via SMB to owner node (“Redirected I/O”)
Direct RAW I/OCSV implements direct I/O by:
Pinning eligible files to prevent block reallocation and movement
Tracking the mapping of file extents to physical disk blocks
for each file
Translating file reads/writes (in terms of offset, length) into RAW I/O to disk blocks
Evolution of cluster shared volumesCSV introduced in Win2008 R2 for Hyper-V workloadFile system filter driver either performs direct I/O or redirects via SMB
CSV in Win2012Cluster file system (CSVFS)Support any workload (scale-out file server, etc.)Better integrated with storage stack (AV filters, VSS, VDS)Direct I/O in more scenarios (buffered, during backup)
CSV I/O architecture in Windows Server 2012CSVFS proxy file system
Disk
Volume Manager
NTFS
CSV File System Filter
Server / SMB
Node 2
Disk
MUP/RDBSS/
SMB
Node 1
MUP/RDBSS/
SMB
Node 3
Direct I/O
CSV VolumeMgr
CSV Proxy File System
Storage connection broken or not present
Coordination Node
CSV VolumeMgr
CSV Proxy File System
CSV VolumeMgr
CSV Proxy File System
CSVFS
Shared storage
LBFO/RDMA
Share VM
Share VM
Share VM
CSV component overview CSV filter driver (CSVFLT.sys)Mounted on Metadata Coordinator NodeBlocks access to the NTFS file system Co-ordinates metadata operations over SMB Filter Altitude - 404800
CSV proxy file system ( CSVFS.sys )Proxy file system on top of an underlying NTFS file system Mounted on every node including coordinatorPerformed Direct I/O to the physical disk. CSV volume manager ( CSVvBUS.sys )Responsible for CSV pseudo/virtual volumes Block-level IO redirector
Disk.sys
Volume Manager
NTFS
CSVFLT.sysSMB
Node 2 ( Coordinator )
Disk.sys
SMB
Node 1
Direct I/O
CSVvBUS.sys
CSVFS.sys
CSVvBUS.sys
CSVFS.sys
CSV file system
Shared storage
CSV Component
s
Windows Components
Share VM
Share VM
CSV–filtering optionsFile system filtersAttach on top of CSVFS.sysAttach to NTFS
Volume filtersFilters attach to CSVvBus.sys
Attaching legacy filters to NTFS stackCSV safeguards with Redirect IO mode
If attaching to MUP ignore CSV traffic to the coordinator node
Disk.sys
Volume Manager
NTFS
CSVFLT.sysSMB
Node 2 ( Coordinator )
Disk.sys
SMB
Node 1
Direct I/O
CSVvBUS.sys
CSVFS.sys
CSVvBUS.sys
CSVFS.sys
CSV File System
Shared Storage
CSV Component
s
Windows Component
s
File System Filters
File System Filterss
Volume Filters
Volume Filters
3RD Party
Share VM
Share VM
CSV proxy file system CSV enabled volumes appear as “CSVFS”NTFS file system under the coversEnables your application to discover when you are running on CSV and enable/disable as appropriate
CSV volumes have a “CSVFS” signature FLT_FILESYSTEM_TYPE as FLT_FSTYPE_CSVFS
Single namespaceConsistent view across the clusterSingle consistent file name spaceFiles have same name and path on any cluster node
Volumes exposed under “ClusterStorage” root directoryApplications can access the volumes from any nodes under the same pathVolumeX directory name can be renamed
File system API compatibility on CSVCSV appears as a local file system, but delivers some behavior of a remote file systemSide effects of how CSV redirects metadata operations over SMB
Some API semantic differences when running on CSVReview the API compatibility document at this link:http://www.microsoft.com/en-us/download/details.aspx?id=29043
FC
Shared storage optionsFlexible storage choices for building clusters
SAS RBOD
iSCSI FCoE
SAS JBOD
Shared storage
RAID HBA Software replication
Hardware replication
SMB
Data replication
3rd party software
replication solution
Application replication
Example: exchange
SQL AlwaysOn
Spaces
Win2012 changes for storage developers / OEM’sSCSI command requirementsPersistant reserve out register (00h) required for clustered spacesSCSI inquiry data (page 83h) support changed from recommended to requiredThe algorithm for how persistent reservation keys are generated has changed
• Moved from a per LUN key to a per Node key
Storage class resourcesStorage class resources will require support for new control codes to be compatible with GUI
• Such as: CLUSCTL_RESOURCE_TYPE_STORAGE_GET_RESOURCEID
Backup of cluster shared volumes story has changedCluster shared volumes file system identifierCSV volumes now appear as “CSVFS”, instead of NTFSCSVFS only supports file system mini-filter drivers (no legacy filter support)
Storage replication solutionsCluster resources are allowed in CSV resource groups to enable replication solutions
• Resource must support new CLUS_CHAR_COEXIST_IN_SHARED_VOLUME_GROUP characteristic
CSV backup improvements for requestor ISVs
Support for parallel backups On same or different CSV volumesOn same or different cluster nodes
Improved interoperabilityBackup applications / requestors are no longer required to be ‘CSV aware’Support for filter drivers to enable incremental backups and continuous data protection solutions
Improved I/O performanceDirect I/O mode for software snapshotsNon-disruptive backups
In-box SupportNative CSV backup support for Windows Server Backup
Non-disruptive backupsCSV volume ownership does not change during backup
Distributed SnapshotsNew infrastructure that enables a distributed app consistent snapshot creation across the cluster
Cluster programmatic interfaces (API)Cluster APINative APIMSCluster WMI providerRecommended management interfaceWMIv2 provider something we are looking at for the futureCluster automation server (MSClus) COM APIDeprecated in Windows Server 2008Optional component (FailoverCluster-AutomationServer) which is not installed by default
Cluster user interfaces (UI’s)Failover Cluster manager snap-inFailover Cluster manager integration points• Failover Cluster manager supports creating custom property pages for
resources
Cluster PowerShell cmdlet’s Can also be used as a programmatic interface
Cluster.exeDeprecated in Windows Server 2008 R2 (replaced by PowerShell as Cluster CLI)Optional component (FailoverCluster-CmdInterface) which is not installed by default
Systems Center Virtual Machine Manager (SCVMM)SCVMM 2012 can setup and deploy a cluster end-to-end
Considerations: deprecated components Cluster.exe Cluster.exe was deprecated in Win2008 R2 for PowerShell cmdlet’sOptional component in Win2012 which is uninstalled by default
Cluster automation server (MSClus) COM programmatic interfaceMSClus was deprecated in Win2008Optional component in Win2012 which is uninstalled by default
32-bit resource DLL support32-bit cluster resource DLL support was deprecated in Win2012
MSCluster WMIv1 providerThe MSCluster WMIv1 may be deprecated in vNext, to be replaced by a new cluster WMIv2 provider
Enabling applications for availability
Generic application
generic service
Good
Generic script
Better
Cluster resource DLL
Best
Generic serviceGeneric service resource – is a resource built into Failover Clustering which allows you to configure any service in service control manager for high availabilityAt a high level it orchestrates calling NET STOP / NET START across servers
Simple and easyEnables you to make a service highly available with no changesBasic health monitoringQueries service control manager (SCM) if the service is running
Good
Generic applicationGeneric application resource – resource built into Failover Clustering which enables any executable to be configured for high availabilitySimple and easyEnables you to make an app highly available with no changesBasic health monitoringMonitors the process IDDetects if the process crashes
Good
Generic scriptGeneric script resource consumes a VBScript that implements cluster entry pointsEnables better health monitoringMost importantly IsAlive/LooksAlive to validate app is healthy
Low development costSimple script
Higher customer deployment cost / complexityUser responsible for configuration
See the following link for more details on writing a scripthttp://msdn.microsoft.com/enus/library/windows/desktop/aa373089(v=vs.85).aspx
Better
Cluster entry points
Open
Online
LooksAlive
IsAlive
Offline
Close
Terminate
Initialize resource
Starts resource
Lightweight health check (every 5 sec by default)
Verbose health check (every 60 sec by default)
Stops resource
De-initialize of resource
Clean-up non-responsive resource
Generic script example GenScriptNotepad.vbs
Dim ProcessDim ProcessId
Function Open( ) On Error Resume Next Resource.LogInformation "Enter Open( )" If Resource.PropertyExists("CommandLine") = FALSE Then Resource.LogInformation "Creating property CommandLine" Resource.AddProperty "CommandLine" Else Resource.LogInformation "Property CommandLine exists" End if Resource.CommandLine = "Notepad" Set Process= GetObject("winmgmts:win32_process") Resource.LogInformation "Exit Open( )" Open = True ' SuccessEnd Function
Function Online( ) Resource.LogInformation "Enter Online( )" Process.Create Resource.CommandLine,,,ProcessId Resource.LogInformation "Exit Online( ) ProcessId = " & ProcessId Online = 0 ' SuccessEnd Function
Function Offline( ) On Error Resume Next Resource.LogInformation "Enter Offline( )" Terminate Resource.LogInformation "Exit Offline( )" Offline = Err.NumberEnd Function
Function Close( ) Resource.LogInformation "Enter Close( )" Resource.LogInformation "Exit Close( )" Close = 0 ' SuccessEnd Function
Function Terminate( ) On Error Resume Next Resource.LogInformation "Enter Terminate( )" Set Process = Nothing Set Process = GetObject("winmgmts:win32_process.Handle=" & ProcessId) Process.Terminate 1 Set Process = Nothing Resource.LogInformation "Exit Terminate( )" Terminate = 0End Function
Function LooksAlive( ) On Error Resume Next Set Process = GetObject("winmgmts:win32_process.Handle=" & ProcessId) If Err.Number <> 0 Or Process.ProcessId <> ProcessId Then LooksAlive = False Resource.LogInformation "LooksAlive failed. ProcessId: " & ProcessId & " Error: " & Err.Number & " " & Err.Description Else LooksAlive = True End if
Function IsAlive( ) On Error Resume Next Set Process = GetObject("winmgmts:win32_process.Handle=" & ProcessId) If Err.Number <> 0 Or Process.ProcessId <> ProcessId Then IsAlive = False Resource.LogInformation "IsAlive failed. ProcessId: " & ProcessId & " Error: " & Err.Number & " " & Err.Description Else IsAlive = True End ifEnd Function
Cluster resource DLLISV can write a custom resource DLL that plugs into Failover ClusteringVery flexible and customizableRich health monitoringHigher dev costTighter integration for a better customer experienceMulti-active/instanced applicationsSee this blog series for detailed information• Creating a cluster resource DLL (Part 1)
http://blogs.msdn.com/b/clustering/archive/2010/03/11/9976620.aspx • Creating a cluster resource DLL (Part 2)
http://blogs.msdn.com/b/clustering/archive/2010/03/30/9987135.aspx• Creating a cluster resource DLL (Part 3)
http://blogs.msdn.com/b/clustering/archive/2010/04/21/9999736.aspx• Creating a cluster resource DLL (Part 4)
http://blogs.msdn.com/b/clustering/archive/2010/08/18/10051655.aspx • Creating a cluster resource DLL (Part 5)
http://blogs.msdn.com/b/clustering/archive/2010/08/24/10053405.aspx
Best
Meeting customers availability expectations
High availability—failover is automatic and down is kept to a minimum Clients must reconnectContinuous availability—automatic failover with preservation of state. Failover is seamless to clients. For example, through TCP reconnects
High availability
Continuous availability
Branch Office
Leveraging Virtualization HA / DR• Failover Clustering can also make a Virtual Machine
HA• Delivers apps / service agnostic way to achieve HA
• Hyper-V Replica enables DR of a Virtual Machine• Delivers app / service agnostic way to achieve DR
Hyper-V Replica:Point-in-time replication
of VHDs for disaster recovery
VHDVHD
You can achieve an HA and DR strategy for free with no changes to your app!!
In review: session objectives and takeawaysFailover Clustering provides a platform to enable stateless applications to achieve high availabilityBasic support can be achieved with little to no changes to the applicationFailover Clustering can deliver the lowest total cost of ownership (TCO) solution
Writing Windows Server 2012 applications to leverage the file based shared storage
11/1/2012 12:00:00 PM
B92 Odyssey
Related sessions
• Cluster Product Team Bloghttp://blogs.msdn.com/b/clustering/
• Cluster Forums (to ask questions)http://social.technet.microsoft.com/forums/en-US/winserverClustering/threads/
Resources
Please submit session evals on the Build Windows 8 App or at http://aka.ms/BuildSessions
• Follow us on Twitter @WindowsAzure
• Get Started: www.windowsazure.com/build
Resources
Please submit session evals on the Build Windows 8 App or at http://aka.ms/BuildSessions
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Recommended