Hyper-V Storage

Liang YangMathew John

MDC-B345

Session objectivesLearn more about• Hyper-V storage architecture• Our ongoing investments in storage• Deployment recommendations

Hyper-V storage architecture

SODA File ServerSODA File ServerPhysical Machine

VM Host

File Server

Hypervisor

App

File

CSV

NTFS/REFS

RDR

Volume

Partition

Spaces

SAS/SCSI IDE FC iSCSI

JBOD

SAN Target

FC SAN

Ethernet

SMB3.0RDMA

Multichannel

\\share

Stor VSP

FC VSP

VHDMP

VHD

VHDX

NPIV

Shared VHDXParser

VHDMP

VHDX

Shared VHDXFilter

CSV/NTFSSpaces

SASSRB

JBOD

SCSI over SMB3.0App

File

CSV

NTFSVolume & Partition

\\share

iSCSI

Synth FC

IDE

Synth SAS/SCSI*

Stor VSC

FC VSC

VMBus - Multichannel

Em

ula

ted

⇚ other cluster nodes

using SMB

⇚ other machines

Native VHD

De-Dup

⇚ other machines

Hyper-V Storage

iSCSI Target

Filter

Clustered Scale Out

Shared VHDXFilter

QoS Limit

QoS Reserve

Stor vDEV

Not Shown

• System Center

• Hyper-V Replica

• VSS

• ODX

• TRIM

• Tiering in Spaces

• Write back cache

• BitLocker

• CSV Cache

• Pass through disks

What do we get with this architecture?• Flexible & extensible• Resilient & reliable• Scalable & efficient• Cost effective• Simple!

Quick recapWindows Server 2012

Demo>1 Million IOPS

Liang Yang

How did we scale IO to each VM?• Pre-Windows Server 2012• 1 channel• 256 queue depth/SCSI controller• Fixed VP For IO Interrupt Handling

•Windows Server 2012• 1 channel/16 VPs/SCSI• 256 Queue Depth/Device/SCSI• Distributed IO interrupt handling Physical Machine

Parent Partition

VirtualStorage Stack

VM

VM

Device

Device

The default format for Virtual Hard DisksVHDX

Up To 64 TB

Larger Virtual Disks

MB Alignment

Large Sector

Support

Enhanced Perf

Larger Block Sizes

Internal Log

Enhanced Resiliency

Embed Custom

Metadata

User Defined

Metadata

DemoVHDX performance

Liang Yang

VHDX Perf: 1MB Sequential Writes

PassThru Fixed Dynamic Differencing0

200

400

600

800

1000

1200

1400

1600

1800 VHD VHDX Disk

MB/s

25% 25%

Queue Depth 16

VHDX Perf: 32KB Random Writes

PassThru Fixed Dynamic Differencing80000

90000

100000

110000

120000

130000

140000

150000

160000 VHD VHDX Disk

Queue Depth 16

IOPS

10%10%

VHDX Perf: Dynamic VHDX growth

Buffered IO 2MB Queue depth 1

Non-buffered IO 64KB Queue depth 64

Non Buffered IO 16MB Queue depth 48

0

200

400

600

800

1000

1200

1400 VHD VHDX

MB/s

100%

800%

Windows Server 2012 recommendation• Move away from PassThru drives• Start using Dynamic VHDX• Decouple storage and compute

Windows Server 2012 R2

Learn More: Hyper-V – What’s New in Windows Server 2012 R2 (MDC-B330)

VHDX online resize• Grow virtual disk size with no downtime• Shrink virtual disk size with no downtime• Safe Shrink

• VHDX only• SCSI virtual disk only• Boot drive resize with Generation 2 VMs

DemoVHDX online resize

Liang Yang

Storage Quality of Service* (QoS)• Limit – Maximum IOPS• Reserve – Minimum IOPS threshold alerts• Measure – New Storage attributes in VM

Metrics

DemoStorage QoS

Liang Yang

Storage QoS: Normalized IOPS• (IO) Size Matters

• Normalized IOPS only for Accounting

• Normalized IOPS accounted in terms of 8K IOPS• Example: One 8K I/O will be counted as 1 Normalized I/O• Example: One 4K I/O will be counted as 1 Normalized I/O• Example: One 10K I/O will be counted as 2 Normalized I/Os• Example: One 16K I/O will be counted as 2 Normalized I/Os

Host

VMStorage QoS Limits• Limits set per virtual disk

• StorVSP contains QoS limiter• Token bucket algorithm

Stor VSC

Stor VSP

QoS Limiter

VHDMP

QoS ReservesIO Balancer

Storage QoS Reserves• Minimums thresholds per virtual disk

• Threshold alerts

• VHDMP contains QoS IO Balancer

Host

VM

Stor VSC

Stor VSP

QoS Limiter

VHDMP

QoS ReservesIO Balancer

Storage QoS Metrics• Chargeback

• New storage metrics per virtual disk• Average Normalized IOPS (Averaged over 20s)• Average latency (Averaged over 20s)• Aggregate Data Written (between start and stop metric command)• Aggregate Data Read (between start and stop metric command)

Shared VHDX• Shared storage with VHDX• VM sees a shared Virtual SAS disk• Guest Clustering no longer bound to

storage topology

Learn More: Application Availability Strategies for the Private Cloud (MDC-B311)

Spaces: Tiered Storage• SSDs & HDDs in single

space

• Activity measured in background

• Hot data moved to SSD tier• Cold data moved to HDD

tier

• Assign files to storage tiersLearn More: Exceptional IOPS (MDC-B218)

Hyper-V Compute Nodes

Storage Space

HDD Tier

Cold Data

SSD Tier

Hot Data

400GB EMLC SAS SSD

4TB 7200RPM SAS HDD

Reads/Writes Accumulates Data Activity

Spaces: Write-Back Cache• Complements Tiering• Absorb short I/O bursts

• Seamless integration

Learn More: Exceptional IOPS (MDC-B218)

Hyper-V Compute Nodes

Storage Space

HDD Tier

Cold Data

SSD Tier & WBC

Hot Data

400GB EMLC SAS SSD

4TB 7200RPM SAS HDD

Reads/Writes Accumulates Data Activity

Live Deduplication*• Live VHDX/VHD

deduplication• VDI on Remote File Server

only• Supported on CSV

Scale-out File Server

Hyper-V VDI

VHD

SMB

DedupCSV

Learn More: Reduce Storage Costs with Data Deduplication (MDC-B342)

Recommendation

SODA File ServerSODA File ServerPhysical Machine

VM Host

File Server

Hypervisor

App

File

CSV

NTFS/REFS

RDR

Volume

Partition

Spaces

SAS/SCSI IDE FC iSCSI

JBOD

SAN Target

FC SAN

Ethernet

SMB3.0RDMA

Multichannel

\\share

Stor VSP

FC VSP

VHDMP

VHD

VHDX

NPIV

Shared VHDXParser

VHDMP

VHDX

Shared VHDXFilter

CSV/NTFSSpaces

SASSRB

JBOD

SCSI over SMB3.0App

File

CSV

NTFSVolume & Partition

\\share

iSCSI

Synth FC

IDE

Synth SAS/SCSI*

Stor VSC

FC VSC

VMBus - Multichannel

Em

ula

ted

⇚ other cluster nodes

using SMB

⇚ other machines

Native VHD

De-Dup

⇚ other machines

Hyper-V Storage

iSCSI Target

Filter

Clustered Scale Out

Shared VHDXFilter

QoS Limit

QoS Reserve

Stor vDEV

Not Shown

• System Center

• Hyper-V Replica

• VSS

• ODX

• TRIM

• Tiering in Spaces

• Write back cache

• BitLocker

• CSV Cache

• Pass through disks

SODA File ServerSODA File ServerPhysical Machine

VM Host

File Server

Hypervisor

File

NTFS/REFS

RDR

Volume

Partition

SAS/SCSI

SAN Target

FC SAN

SMB3.0RDMA

Multichannel

\\share

Stor VSP

VHDMP

Dynamic VHDX

Shared VHDXParser

VHDMP

VHDX

Shared VHDXFilter

CSV/NTFSSpaces

SASSRB

JBOD

SCSI over SMB3.0App

File

CSV

NTFSVolume & Partition

\\share

IDE

Synth SAS/SCSI*

Stor VSC

VMBus - Multichannel

Em

ula

ted

Native VHD

De-Dup

Hyper-V recommended greenfield deployment

Filter

Clustered Scale Out

QoS Limit

QoS Reserve

Stor vDEV

TakeawayWe take virtualization storage seriously, very seriously!

We will continue innovating to provide the best storage solution for you.

Related contentBreakout Sessions MDC-B311 Application Availability Strategies for the Private CloudMDC-B330 Hyper-V – What’s New in Windows Server 2012 R2MDC-B333 Software-Defined Storage in Windows Server R2 and System Center 2012 R2MDC-B335 Hyper-V over SMB Scenario (Overview, Configuration, and Performance)MDC-B342 Reduce Storage Costs with Data DeduplicationMDC-B344 Storage Management: Spanning the Enterprise to Low Cost Scalable SolutionsMDC-B345 Windows Server 2012 Hyper-V Storage PerformanceMDC-B349 Upgrading the Platform - How to Get There! Part 4: Your Fileservers and Storage OptionsMDC-B357 What’s New in System Center 2012 R2 - Virtual Machine Manager

Hands-on Labs MDC-H201 Implementing Storage Pools and Storage SpacesMDC-H303 Configuring Hyper-V over Highly Available SMB Storage

Find Me Later at the Virtualization Booth

Track resourcesLearn more about Windows Server 2012 R2 Preview, download the datasheet and evaluation bits on http://aka.ms/WS2012R2Learn more about System Center 2012 R2 Preview, download the datasheet and evaluation bits on http://aka.ms/SC2012R2

msdn

Resources for Developers

http://microsoft.com/msdn

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources

Sessions on Demand

http://channel9.msdn.com/Events/TechEd

Resources for IT Professionals

http://microsoft.com/technet

Complete an evaluation on CommNet and enter to win!

Evaluate this session

Scan this QR code to evaluate this session and be automatically entered in a drawing to win a prize

© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.