(DEP290) Windows 2003 Server™ New Storage features

Christophe DuboisStorage Consultant

Didier LalliWindows Wizard

Hewlett PackardHewlett Packard

Agenda

Understanding the Windows 2003 Server Storage Stack

Enhanced Driver Model STORport & iSCSI

MultiPath Driver Model (MPIO)

Volume Shadow Copy ServiceBackup, Transport, Restore

Virtual Disk Service

And what about Clusters?

HP/HP/OEMsOEMs

MicrosofMicrosoftt

Kern

el M

od

eK

ern

el M

od

e

File SystemsFile Systems

Volume SnapshotVolume Snapshot

Volume ManagementVolume Management

Miniport(sMiniport(s))

MiniportMiniport

PortPort

DiskDisk TapeTape ChangerChanger ClassClass

ScsiPortScsiPort StorPortStorPort IDEPortIDEPort

User

Mod

eU

ser

Mod

e

WMIWMI

RemovablRemovable Storage e Storage ManagerManager

(tape and (tape and optical optical media media

managemenmanagement)t)

Virtual Virtual Disk Disk

ServiceService

(RAID, disk (RAID, disk access, access,

Enclosures)Enclosures)

HW ProvidersHW Providers

Volume Volume Shadow Shadow

Copy Copy ServiceService

(Point-in-(Point-in-time copies)time copies)

SW ProviderSW Provider

Wri

ters

Wri

ters

RequestorsRequestorsW

rite

rsW

rite

rs

HW ProvidersHW Providers

SW ProviderSW Provider

iSCSI InitiatoriSCSI Initiator

Windows 2003 storage stack

iSCSIprtiSCSIprt

MPIOMPIO DSMDSM DSMDSM DSMDSM MPIOMPIO

Enhanced SAN Support STORport

Increases I/O performance: Removes workarounds for legacy devices

8-30% improvement in I/O rates, lower CPU utilizationSupports 255 I/Os per LUN

Adds functionality:Support for LUN Resets (SAN friendly!)

Good for FC nearline storage & ClusteringSupport for more Fibre Channel capabilities

Allows boot and tape devices on same data path as shared cluster disks

Support more devices (>2048 LUNs)

Microsoft iSCSI Support

Adds support for native iSCSI toWindows Server 2003, 2000 and XP

Initial iSCSI package will be “OOB” but is still considered part of Windows

Will be inbox for all future OS releases

Initial delivery vehicle will be Windows Download

iSCSI initiator package

iSNS Server package

iSCSI

What is it ?iSCSI is internet Small Computer System InterfaceInternet Protocol (IP)-based storage networking standard for linking data storage devices

Carries SCSI commands over IP networks

How does it work ?encapsulate SCSI cmds and data requests into IP packetscommands sent to SCSI/FC controller, and then to the SCSI/FC storage device. Generally via an iSCSI router

iSCSI cont.

What does it mean to me ?iSCSI can be used to transmit data over

local area networks

wide area networks

or the Internet

can enable location-independent data storage and retrieval

FC & IP Storage Network

iSCSIRouters

FibreFibreChannelChannel

Fibre ChannelStorage Systems

EthernetTCP/IP

Hosts with iSCSI drivers

Microsoft and HP Microsoft and HP deliverdeliver

this today!this today!

iSCSI vs. NAS?

Cannot really directly compare the twoNAS uses fileiSCSI uses blocks (like fiber channel)

Exchange is supported using iSCSI, NAS is currently notComplementary – NAS can be a front end for iSCSIiSCSI does not replace fileservers (or NAS)

Files are natural units of operation for end usersSecurity and sharing of files is easier

MULTIPATH DRIVER MODEL (MPIO)

What is it? The ability to use more than one physical path to access a storage device, providing improved system reliability and availability via fault tolerance and/or load balancing of the I/O traffic

Why is Microsoft doing this? Microsoft's Multi Path I/O delivers a standard and interoperable path for communication between storage products and Windows Servers

Enables multi-vendor storage configurations

MULTIPATH DRIVER MODEL (MPIO)

What does it mean to HP? HP is working closely with Microsoft to develop an MPIO solution for HP arrays. Secure Path may adopt this technology in future releases.

Vendor ships MPIO Solutions

Solutions available only from vendorsNot part of the operating system

Requires Device Specific Modules (DSMs)Each storage device has different behaviors and features

Not always possible to identify device down each path

i.e active/passive controller failover

Generic support is not possible without tradeoff

overview. . .overview. . .

Volume Shadow Copy Service

What is a Shadow Copy ?

Known in the industry as Snapshots, split-mirrors or Clones

Appears static, even though the original data is changing

Can be done at the file system level or LUN level in external arrays

SnapshotSnapshotAreaArea

Copy on WriteCopy on Write

Snapshot

Logical Logical VolumeVolume

Retrieve Original and Retrieve Original and Unchanged dataUnchanged data

Volume MapVolume Map Volume MapVolume Map

F:F:E:E:

Step 1Step 1

Step 2Step 2

SecondsSeconds

the result is an identical logical copy within secondsthe result is an identical logical copy within seconds

Logical Logical VolumeVolume

MirrorMirror LinkLink

Step 1Step 1

Step 2Step 2

40-140GB40-140GBHourHour

Clone: Split Mirror

ClonedClonedVolumeVolume

Block for Block Block for Block CopyCopy

Logical Logical VolumeVolume

MirrorMirrorLinkLink

Cloned Cloned VolumeVolume

XX

Logical Logical VolumeVolume

Step 3Step 3

SecondsSeconds

Identical CopiesIdentical CopiesAvailableAvailable

UnavailableUnavailable

ClonedClonedVolumeVolume

F:F:

E:E:

Logical Logical VolumeVolume

MirrorMirror LinkLink

Step 1Step 1

Step 2Step 2

Clone: SnapClone

ClonedClonedVolumeVolume

Copy on WriteCopy on Write

Logical Logical VolumeVolume

MirrorMirrorLinkLink

Logical Logical VolumeVolume

Step 3Step 3 Identical CopiesIdentical Copies

SecondsSeconds

F:F:

E:E:

AvailableAvailable

Background Block Background Block for Block Copyfor Block Copy ClonedCloned

VolumeVolume

ClonedClonedVolumeVolume

40-140GB40-140GBHourHour

Starts as a Snapshot and becomes a clone over timeStarts as a Snapshot and becomes a clone over time

XX

Introduction to VSS

Provides a framework on Windows 2003 Enterprise Edition and Data Center Edition

Simple and consistent method to create a point-in-time copy of a single volume or multiple volumes with control of the applications and stores

Provides unified view for backup applications in a heterogeneous SAN environment

Shadow Copy For BackupWhat problems does VSS solve?

For Backup Applications:No easy way for applications to describe their data to backup apps

Excessive number of APIs to deal with

For Data:Some transaction only exist in filesystem OS cache or application memory pages

An application can modify a file while it is being backed-up causing data corruption or a file open error

Current SAN management reality

Heterogeneous SANHeterogeneous SAN

Each array type within a SAN environment requires Each array type within a SAN environment requires it’s own management tool/APIit’s own management tool/API

HPQHPQtool/APItool/API

HPQHPQtool/APItool/API

EMCEMCtool/APItool/API

EMCEMCtool/APItool/API

IBMIBMtool/APItool/API

IBMIBMtool/APItool/API

HDSHDStool/APItool/API

HDSHDStool/APItool/API

Volume Shadow Copy Service

Heterogeneous SANHeterogeneous SAN

Enables a large heterogeneous SAN to be managed Enables a large heterogeneous SAN to be managed via ONE volume copy management solutionvia ONE volume copy management solution

Microsoft VSS frameworkMicrosoft VSS frameworkMicrosoft VSS frameworkMicrosoft VSS framework

VSS Framework

Within VSS there is the concept of a “Framework”.

The Framework can be compared to an air traffic control center - through which all coordination from the various components must travel through

VSS Components

Requestors: Apps that initiate the request to VSS to create a shadow copy of the volume(s)

Ex: Traditional backup application

Writers: Apps that change/manipulate data and respond to VSS signals

Ex: MS SQL Server, MS Exchange 2003

Providers: Interface to the disk array to create Volume Shadow copies on behalf of the VSS Framework

Ex: Hewlett-Packard EVA VSS Hardware Provider

Volume Volume Shadow Copy Shadow Copy

ServiceService

VSSVSSProvidersProviders

Backup Backup ApplicationApplication

SQLSQL

ExchangExchangee

WindowsWindowsServicesServices

AD, AD, Registry,Registry,

etc…etc…

WriterWriter

IISIIS•VeritasVeritas•LegatoLegato•HPHP•CommvaultCommvault•DantzDantz•UltrabacUltrabac•Others…Others…

Writer MetadataWriter Metadata

FilesFilesComponentsComponentsSettingsSettings

FilesFilesComponentsComponentsSettingsSettings

FilesFilesComponentsComponentsSettingsSettingsStateStateRestoreRestore

How does it work together ?

•Win2003 (volsnap.sys)Win2003 (volsnap.sys)•HP (EVA, VA, XP)HP (EVA, VA, XP)•EMC, HDSEMC, HDS•Others...Others...

F: F: Z: Z:

RequestorRequestor

Shadow Copy Transport

Enables you to easily transport data and volumes on your SAN

Create and export a shadow copy of your data volume or database

Import that shadow copy and data on a second server

Requires a provider for the SAN array

HP released compatible VSS hardware providers for EVA, VA, XP arrays

Exchange Server

Multiple LUNs shadow copied at single point-in-time with Multiple LUNs shadow copied at single point-in-time with data consistencydata consistency

EXC DBs100 GB

EXC Logs50 GB

EXC DBs ‘100 GB

EXC Logs ‘50 GB

Backup Server

Shadow Copy Transport

HP HP StorageWorksStorageWorks

EVAEVA

Shadow Copy Transport

What are the scenarios?Make a copy of your database for data mining on another server

Offload backups to another server

Make a copy of your production database for testing purposes

Keep a copy of your data around for fast recovery

Fast Recovery Solutions 2003

A requestor application focused at the application administrators (Exchange / SQL).Easy to use with a GUI and a Command Line Interface for scheduling options.Uses the VSS framework to create VSS LUNs for recovery, however these VSS LUNS could be used for other purposes (backup to tape, Exchange 2003 – Mail Box recovery…etc)Fast Recovery Solutions is tuned for the HP arrays for performance and features. Currently supports the EVA,XP and VA arrays.

Fast Recovery Solutions 2003

Fast Recovery Solutions 2003 will be available this fall. EVA and XP beta customer application process will start on August 18, 2003.

For additional details contact Rudy Castillo at rudy.castillo@hp.com

Shadow Copy Restore

New Feature for Windows 2003 ServerAllows users to see network folder contents at points-in-time in the pastUses system provider included with Windows Server 2003Supports Cluster configurationsReduces users’ dependence on busy administratorsNot a replacement for Archive or Business Recovery!

Empower users to undo their own mistakes

introduction. . .introduction. . .

Virtual Disk Service

VDS: What is it ?

VDS provides a single interface for managing storage on Windows 2003

VDS provides a vendor and technology neutral interface for logical volume management

VDS enables management of heterogeneous storage systems.

VDS leverages both client and provider APIs.

Focus is virtualization and innovation in hardware for auto-management

VDS ComponentsCommand Line

InterfacesDiskpart / Diskraid

ManagementApplication(s)

DiskManagement

- Hardware- MS functionality- 3rd party functionality

Virtual Disk Service

Software Providers - Basic Disk

- Dynamic Disk

Disks LUNs

Drives

SpindleSpindle

Hardware Provider(s)

VDS Hardware Provider

Operates on LUNs and drives

The physical hardware associated with a hardware provider is a “subsystem”

A subsystem can be internal or externalExample: hardware provider + PCI controller = internal subsystem

Example: hardware provider + Fiber channel array = external subsystem

HP released providers for EVA, XP, VA disk arrays

VDS Software Provider

Comes with OSDisk Manager Service in Windows 2003 uses VDS

Operates on LUNs and disks (JBOD)

Only host-based

No associated hardware

Note: VDS does NOT coordinate with applications on file systems like VSS

And what about Clusters?

Cluster setup

AD Integration

Majority Node Set Resources

Generic Script Resource DLL

WMI Support

And many other miscellaneous things I have to tell you…

Cluster Setup

Very different!Part of the OS install, just need to be configuredAll done from CluAdmin.exe

Step1: Add Cluster to turn standalone system into Single Node ClusterStep2: Add additional nodes

Wizards helps with checking the feasibility. No reboot requiredNode can be evicted as easily and join another cluster

Cluster Setup PM Demo

AD Integration

Virtual Server can be published in AD

Property of the Network Name resource in the virtual server

Also DNS registration can be enforced for IP address of virtual servers

AD Integration PM Demo

Majority Node Set Resource

Quite a big change!Quorum capable resource stored on each nodes local diskUse SMB hidden share to count votes

(1 server = 1 vote always!)

Changes “split-brain” logicnow needs (number of node configured / 2) + 1 to survive

Can build a cluster with no shared storageGood for “stretch cluster” coupled with storage-based data replication across geographically dispersed sites Can change cluster from Disk Quorum to MNS Quorum

Site 3Site 3Site 2Site 2Site 1Site 1

Dispersed Cluster Example

Node1 Node4Node2

Data

Node5Node3

Data

Quorum Quorum Quorum Quorum Quorum

Controller based ReplicationController based Replication

DisasterDisaster DisasterDisasterDisasterDisaster

Generic Script Resource DLL

New way to make application “Cluster Aware”Generic Script resource requires:

Main routine7 methods:

Open()Online()LooksAlive() Must return True or FalseIsAlive() Must return True or FalseOffline() Graceful shutdownClose() Terminate() Immediate shutdown

Functions can be empty!For example in %WinDir%\System32\inetsrv\

Clusweb.vbsClusftp.vbs

Generic Script Resource DLLPM Demo

FUNCTION LooksAlive( )

DIM objWmiProvider DIM objService DIM strServiceState SET objWmiProvider =

GetObject("winmgmts:/root/cimv2") SET objService =

objWmiProvider.get("win32_service='w3svc'") strServiceState = objService.state

IF ucase(strServiceState) = "RUNNING" THENLooksAlive = True

ELSELooksAlive = False

END IFEND FUNCTION

Function Open( ) REM put open code hereEnd FunctionFunction Online( ) REM put online code hereEnd FunctionFunction Offline( ) REM put offline code hereEnd FunctionFunction Close( ) REM put close code hereEnd FunctionFunction Terminate( ) REM put terminate code hereEnd FunctionFunction LooksAlive( ) REM put LooksAlive code here LooksAlive =End FunctionFunction IsAlive( ) REM put IsAlive code here IsAlive = End FunctionREM put main code here

WMI Support

New WMI ProviderOld Cluster API still exists but is in maintenance modeThe cluster WMI name space is: \\root\MSCluster (e.g: GetObject("winmgmts://./root/MSCluster")Exposes 36 classes:

2 super classes9 main classes17 association classes7 event classes2 reserved for future use

WMI Support PM Demo

SET Events = GetObject("winmgmts://./root/MSCluster"). _

ExecNotificationQuery( "Select * from _ MSCluster_EventStateChange" )

DO SET Item = Events.NextEvent WScript.Echo( DATE & " " & TIME & _ " EventNewState=" &

Item.EventNewState & _ " EventObjectName=" &

Item.EventObjectName & _ " EventObjectPath=" &

Item.EventObjectPath & _ " EventObjectType=" &

Item.EventObjectType & _ " EventTypeMajor=" &

Item.EventTypeMajor & _ " EventTypeMinor=" &

Item.EventTypeMinor )LOOP

=> Monitor State change of any event

wmic /namespace:\\root\mscluster /output:c:\resource.htm

path mscluster_resource get name,state /format:htable /every:300

=> Maintains a WEB page with resource information in a table format and refreshes every 5 minutes.

And many other miscellaneous things I have to tell you…

8 Nodes support64-bit supportMountpoints support Backup system state supportVolume Shadow Copy supportImproved Clustered Print QueuesPassword Change Utility MSDTC is now a real resourceMulticast support for heartbeat/regroup traffic

Oops, I forgot…

No more IIS, SMTP and NNTP supportSample Generic Resource VB script for IIS and FTP

No Dynamic Disks support

No real security settings

Still need nodes in same subnet

Community Resources

Community Resourceshttp://www.microsoft.com/communities/default.mspx

Most Valuable Professional (MVP)http://www.mvp.support.microsoft.com/

NewsgroupsConverse online with Microsoft Newsgroups, including Worldwidehttp://www.microsoft.com/communities/newsgroups/default.mspx

User GroupsMeet and learn with your peershttp://www.microsoft.com/communities/usergroups/default.mspx

evaluationsevaluations

© 2003 Microsoft Corporation. All rights reserved.© 2003 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.