Application acceleration from the data storage perspective

© Copyright 2009-2010, Cambridge Computer Services, Inc. – All Rights Reserved

www.CambridgeComputer.com – 781-250-3000

Application Acceleration From the Data Storage

Perspective

New Advances in Caching and Solid State Storage

Lecturer:

Jacob Farmer, CTO

Cambridge Computer

2www.CambridgeComputer.comApplication Acceleration: New Advances in Caching and Solid State Storage

© Copyright 2010, Cambridge Computer Services, Inc. All rights reserved.

The Pain: Random Disk I/O Performance is not Easily Scalable

Random Disk I/O performance is not keeping pace with other

advancements such as processor and network I/O

• 15K drives are not that much faster than 10K!

• Traditional RAID arrays make it difficult to aggregate disk spindles

without host-based LVM software

Applications running in virtual servers are particularly

constrained by random I/O

• The virtualization environment randomizes traditionally sequential

workloads

• Booting and page file activity tend to share the same disk pools as

applications and thus compete with apps for disk I/O

Random v. Sequential I/O



The Relative Performance of Spinning Disk to CPU, Cache and RAM

1 lb

10 lb

100 lb

10,000,000 lb

CPU L2 Cache RAM Spinning Disk

4www.CambridgeComputer.com

What’s Changing?

The price of DRAM has declined to the point where a

meaningful capacity is affordable

• Traditional DRAM SSDs were very pricey, niche technologies

– 5 years ago – 20GB of SSD sold for about $500K

The emergence of enterprise-class flash technology

• Recent innovations allow flash-based SSD to achieve meaningful

performance with enterprise-class reliability.

• Flash offers new tiers in between DRAM SSD and 15K disk

We are nearing the tipping point where SSD goes viral.

• Higher demand accelerates the decline in price

• Commoditization inspires new software technologies that leverage

commodity hardware

Application Acceleration: New Advances in Caching and Solid State Storage



SSD Media Choices

• Equal performance for read and write

• No concern for duty cycle

• Requires uninterruptable power sourceDRAM

• Faster write speeds than MLC

• Lower power consumption than MLC

• Higher endurance than MLC

• Higher cost than MLC, but still cheaper than DRAMS

SLC FlashSingle-Level Cell

• Multiple bits occupy a flash memory cell, reducing costs

• Higher bit-error rate than SLC

• Lesser performance SLC

• Lesser endurance than SLC

• Software compensates for shortcomings in the media

MLC FlashMulti-Level Cell




Flash Memory Durability and Reliability

Flash memory does wears out over time

• MLC has a lower duty cycle than SLC

“Wear leveling”

• Firmware in the flash device keeps track of I/O requests to

each logical block and distributes I/O across flash cells

• As the device ages the wear leveling mechanism might

reduce performance.

– Especially true of devices designed for desktop and laptop usage

Confirm duty cycles with product manufacturer

• Most enterprise flash devices will outlive traditional hard

drives under specified use-cases





SSD Packaging: SSDs Come in Many Shapes and Sizes

Hard Drive Form-Factor with SATA, SAS, or FC interface

• Can be installed in server drive slots

• Can be installed in enterprise SAN enclosures

– Add a new high performance tier (“Tier Zero”) to your SAN array

PCIe Cards and PCIe External Appliances

• Looks like a hard drive to the local OS

• Lower latency than devices that use storage channels (FC, SATA, SAS)

• Suitable for applications that can use direct-attached storage

– PCIe switches are slowly being adopted by enterprise applications

SAN-Attached Appliances

• Great for traditional enterprise apps, especially when high availability clustering is involved

– Failover Clusters, Server Virtualization Platforms, Oracle RAC

DIMM Modules Installed on the Motherboard on the RAM Bus

• Extremely low latency

w w w . C a m b r I d g e C o m p u t e r . c o m 8

How do you size your RAID groups or disk arrays?

By Capacity?

By Performance?

By Guess Work?



Hard Drives = Units of Performance

Each hard drive (spindle) is like a little worker capable of

doing only so much work in a given day.

• Work is measured in IOPS – input/outputs per second.

• Faster hard drives deliver more IOPS than slower hard drives.

• Note: there is a diminishing return with increasing rotational speed.

Drive Type Typical Real-World IOPS



SSD Performance Comparison Approximate – 4 KB Random I/O

Drive Type Typical Real-World IOPS


Cost Per Gigabyte versusCost Per Disk I/O



Drive Type Cost per GB

Drive Type Cost per IOP

Notes: 320GB SSD PCIe Card 128GB SSD Hard Drive 450GB 15k Hard Disk

IOPS Assumptions 100,000 20,000 180

Costs $7,500 $10,000 $1,000


Taking Advantage of Solid State Storage



Applications for SSD

Any application or data set where the demand for random I/O

performance is high relative to the demand for capacity

• Databases: logs, temps, indexes, whole tables

– MS Exchange: logs and data

• Operating systems: boot images, page files

– Next Generation Virtual Desktop Solutions (VDI)

– Multi-host Platforms: VMware, Citrix

• Out-of-band file systems for metadata catalog

– Any file system where metadata can be isolated from file data

Modern advances enable cache and SSD for general purpose

computing

• Cache and SSD can be inserted anywhere along the storage I/O path

– Dynamic tiered storage, life cycle management, caching


How Do You Take Advantage of These New Storage Tiers?

These new SSD tiers offer performance and cost

characteristics that are wildly different from

traditional storage.

New tiers drive the demand for automated tiered

storage.

Automated Tiered StorageThe ability to scale performance and

capacity independently of each other


The Storage I/O Path – Plenty of Places to Insert Acceleration Logic



Out-of-band File System

Host-based storagemanagement software

Server Virtualization

SAN-based VolumeVirtualization Appliance

Disk Array Controller

Out-of-band Volume Level

Switch-based Virtualization

LAN-BasedAppliance

Ap plicatio n Ope ra ting System File System Volum e Ma nag er

Fabric

Hypervisor


Inserting Cache and SSD into the Storage I/O Path



LAN/WAN


Solid State Storage with MS Exchange


SSD in an Application: MS Exchange

Sizing disk I/O for MS Exchange

• ¼ to ½ I/O per active user

• Multiply by 4 for Blackberry/iPhone (any ActiveSync)

What do put on SSD

• The entire data set if it will fit

• EDBs holding mail for select power users

• Log files

– Big win for write-intensive mail systems

Exchange 2007 and 2010 offers HA clustering without the need for a shared storage SAN

• CCR – Continuous Cluster Replication

• DAG – Database Availability Groups

• Leverage the low-cost of direct-attached SSDs




MS Exchange : Clustering with Direct-Attached SSDs

Replication and HA clustering without the need for shared SAN storage • CCR – Cluster Continuous Replication (2007)

• DAG – Database Availability Groups (2010)



SSD SSD

LAN

Exchange CCR Cluster


MS Exchange: Direct-Attached SSDs for Log Files Only

Exchange logs are stored on

internal SSD

Exchange databases are

stored on traditional

enterprise SAN

• Snapshots

• Replication

• Provisioning



SAN

SSD


Solid State Storage for Databases


SSD for Databases

For small databases, put the entire database on SSD

Typical best practice is to determine which elements

of the database will benefit from SSD

• Usual suspects: Logs, Indexes, Temps, select tables

• Performance monitoring tools are available for all major

databases

– Example: Oracle StatPack

Use volume manager, intelligent SAN, or database

tools to migrate hotspots to SSD

• Oracle ASM allows seamless migration from one storage

device to another.




Adding SSD Device for Database Hotspots



SAN Fabric

RAM-Based

SSD

Appliance

Flash-Based

Array

Disk Array with

Big Cache or

Flash SSD Drives

RAM

Database Cluster


Database QoS with Operating System Resource Partitioning

Quality of service can be enforced by manipulating

prioritization of operating system resources

Specific queries or patterns of queries are given

priorities or assigned performance threshold

• Prioritization policies can be set for disk I/O as well as CPU

utilization

Renegade queries are throttled

• As opposed to running wild or being killed

Allows you to size hardware around normal

conditions rather than worst scenario




Hyper-V, vSphere and XenServer

Better Performance for Virtual

Environments



SSD with Clustered DAS Logical Volume Manager and VMWare

Add a redundant tier of

SSD to your VM

environment by installing

flash SSD cards into VM

hosts.

Server Virtualization Hosts with Internal SSD

SANSAN

“Clustered DAS” Software

Conventional

SAN Storage

SSD SSD


File Systems and NAS Acceleration


File System Acceleration

If metadata can be isolated from data, put the metadata on

SSD drives

• Especially useful for SAN-enabled file systems that are heavily

constrained by metadata

Some file systems (notably ZFS) can cache recently accessed

files in SSD drives.

• This is especially useful when creating very large ZFS stripe groups

Some file systems allow the journal to be stored separately

Maybe your whole file system will fit on SSD drives!

• Use SSDs with hard drive form factor in place of a hard drive.

A caching appliance in front of your NAS

• Any place where NFS/CIFS improvement is needed




Clustered Scale-Out NAS: Living the Dream!!!

Redundant SAN Fabric

RAM-Based

SSD

for Metadata

Tier 0

MLC Flash

Tier 1

15K Disk

Tier 2

SATA Disk

Clustered NAS with Distributed Coherent Cache

RAM RAM RAM RAM



A Clustered File System on IB with Internal SSD Drives

Infiniband Fabric

Application Servers and/or Compute Nodes

Servers with Flash SSD cards

and lots of RAM


Clustered In-Line NAS Cache with SSD and Tiered Storage



Local Area Network (1Gb, 10Gb, Infiniband, etc.)

File Server (NAS)

Clustered Caching Appliance with SSD and Tiered Storage

Use all SATA in your NAS

Retain snapshots, replication, backup, etc.


SAN Acceleration


Hot-spotting with SSD or DRAM Using SAN Volume Virtualization



NOTE: Virtualization layer is not drawn as redundant.

SAN

DISK CONTROLLERS

VIRTUALIZATION DEVICE

Internal SSD

External SSD


Transparent In-band SAN Cache



SAN


SAN Topology View of Inline Cache



SAN



SSD in Dynamically Tiered Array

SSD can be leveraged across the board with dynamic block-level tiering.

Hottest blocks or fresh new writes can stored on SSD and migrate to lower tiers when not in active use.

Makes it easy to deploy SSD without having yet one more tier to worry about.

• Secondary goal is to squeeze the

most I/O out of your disk pool.



SSD with Deduplication

Deduplication gets its greatest value when

• There is a lot of duplicate data

• The media is relatively expensive

New technologies allow block-level dedupe for

primary storage

Killer applications:

• Boot appliance for booting servers over the SAN

• Virtual desktop storage

• Source code libraries

• Oracle, Exchange


Conclusions

Solid state storage is the big game changer in the storage industry right now

• The potential for increased random disk I/O performance changes traditional assumptions about application performance

• The wide range of SSDs create more tiers and inspire demand for automating tiered storage

New advances allow SSD to be inserted into the existing storage infrastructure

• The challenge is to figure out the best way to insert SSD for any given application

The end-goal is to purchase capacity and performance independently of each other

• Grow capacity with larger, less expensive drives and grow performance with various types of SSDs



Education

Application acceleration from the data storage perspective