CloudStack storage subsystem introduction

Edison Su

● Agenda– CloudStack storage subsystem introduction

– New storage API introduction

Storages in CloudStack

● Primary storage– Root disks and data disks

● Secondary storage– Images

– Volumes and snapshots backup

Primary storage

● Different scopes– Local storage, cluster-wide storage, zone-wide

storage

● Different types– NFS/iSCSI/FC/Ceph

Life Cycle of a primary storage

Start Initialized ReadyInitializeAttach

Maintenance

Maintenance

Removed

Remove

UnM

aintenance

Features highlight

● Storage as a service– Storage tags

– Disk offering with tags

Features highlight

● Mix setup of different storages– Multiple scope of storages can be coexist in one

zone

– Multiple types of storages can be coexist in one zone

– Different hypervisors can be coexist in one zone

Secondary storage

● Different scopes– Region/zone wide

● Different types– Nfs/S3/Swift

What is secondary storage VM?

● Don't mislead by its name, it's more powerful than just dealing with secondary storage

● A bootstrap VM created by CloudStack● It's the place where storage provision

happened● VM's cpu/ram/network throughput is fine

grained controlled

Scaling storage provision

Mgt server

Mgt server

Mgt server

SSVM

SSVM

SSVM

SSVM

PrimaryStorage

SecondaryStorage

A typical setup

Cluster storage

storage

storage

Nfs Secondary storage

Cluster storage

storage

storage

Zone

Multiple zones setup

Clusters

storages

Nfs as cache storage

Zone BZone A

Nfs as cache storage

S3/Swift

storages

Clusters

What you can do with storage?

● Volumes– Create root/data volume on primary storage

– Import data volume

– Create template from volume

– Create snapshot from volume

What you can do with storage?

● Snapshot– Create volume snapshot on primary storage

– Backup snapshot into secondary storage

– Create template from snapshot

– Xenserver supports delta snapshot

What you can do with storage?

● Template– Import template

– Create vm from template

– Export template

– Copy template between zones

Part2: storage frameworkMgt server storage orchestration engine

ImageService

VolumeService

SnapshotService

DataMotionService

StoragePoolAllocator

DataStore plugin

DataStore plugin

DataStore plugin

Storage provision

Hypervisor agent

SSVM agent

Any other provision service

Storage box

Design considerations

● Separate orchestration and provision– Orchestration decides which storage to use, where

to conduct storage provision, what resources needed for storage provision

– Orchestration only working on the control panel

– Provision working on the data panel, actually talking to storage, either through hypervisor API or storage vendor's API, or common storage protocols.

Orchestration is complicated

● Different hypervisors, different storage solutions have different requirements, can work differently

● CloudStack has different plugins to hookup into CloudStack management server, change the way how to orchestrate storage related operations.

Orchestration is complicated

● For example, what's the requirements to download a template from s3 into primary storage?– For file system based storage, it's easy, directly

download from s3 to primary storage

– For block based storage, you may need to download template into a temporary file system, then import the file primary storage through hypervisor's API

– So during the orchestration, it decides for this storage, need a file based storage as cache storage, download it into cache storage, then import into primary storage. While for other storage, don't need this cache storage.

–

Orchestration is complicated

● For example, where is the place to create volume?– For KVM/Xenserver, one can login into hypervisor

host, do a lot of things to create a volume

– For Vmware, has to go through vCenter API

– If storage vendor has its own APIs on the control panel, it can be directly called on the mgt server

– If there are other storage services running out of cloudstack mgt server, cloudstack should be able to talk to them

Orchestration is complicated

● For example, what's the requirements for storage migration?– If hypervisor supports, one can directly call

hypervisor's API, no additional resource needed

– If no, then may need a temporary storage to copy the storage back and forth

– If other storage vendor has its own clever way to do storage migration, may have other requirements

Orchestration is complicated

● For example, what's the best way to take and backup snapshot?– Full snapshot vs delta snapshot

– Hypervisor based snapshot vs storage snapshot

– Do you need a temporary storage as cache storage, during snapshot backup?

What's the plugins for orchestration

● CloudStack 4.2 will provide a flexible storage framework for deep integration with different storage solutions

● One thing can't fit all, so we orchestrate differently● The extension points for orchestration:

– Pluggable UI– DataStoreDriver– DataMotionStrategy– SnapshotStrategy

● It's actively developed on object_store branch, should be merged into 4.2

Provision

● Mainly related to data motion– Download template from s3 to primary storage

– Back up snapshot from primary storage to s3

– Move disk from one storage to another

– Etc

● It's on the data path● Provision usually happened inside SSVM or on

the hypervisor host– SSVM is the place to scale, by default

Q & A