Deploying Swifton a Scale-out

File System

Dean HildebrandIBM Research

William OwenIBM

Thiago da SilvaRed Hat

Openstack Swift“A highly available, distributed, eventually consistent object/blob store.Organizations can use Swift to store lots of data efficiently, safely, and

cheaply.” source

Swift Architecture

Swift 2.0 - Storage PoliciesAbility to choose how and where to store

objectsHow: 2x or 3x replication, ECWhere: HDs, SSDs or a Scale-out File System

How is this accomplished?

"multiple rings"

Before...

After...

Why deploy a scale-outfilesystem as a backend for

Swift?

BenefitsAdd swift to an existingscale-out filesystemA single management planefor both file and objectstorage

Leverage filesystem featuresSnapshotsEncryption, compression,deduplicationAutomatic tieringbetween classes ofstorageTape Integration

but, what's missing?No easy way to access object data from the file system

This object:http://swift.example.com/v1/acct/cont/obj

is located here:/mnt/swift/z1device7/objects/63773/ba2/

f91d1e7550cd32822a17b00fa86d9ba2/1414045361.93852.data

Swift-on-FileA Swift DiskFile implementation used to store objects on scale-out

filesystems (e.g., GlusterFS, GPFS).Allows objects created using Swift API to be accessed as files andvice-versaMap URL to file pathDeployed as a Storage Policy

Now, this object:http://swift.example.com/v1/acct/cont/obj

is located here:/mnt/swiftonfile/acct/cont/obj

Use Case: Multiple protocolaccess

Use Case: Analytics

Problem: Separate Storage systems forobject ingest/distribution and analysis

Data duplication andmaintenance overheadData ConsistencyLack real-time analysis

Solution: Swift with Scale-out filesystemDecreased time to resultsRun MapReduce directlyon objects through nativeFile APINo waiting for datatransfer between storagesystems

Use Case: ScientificCollaboration

Problem: Secure and scalable sharing ofscientific data

Native object stores notsuited to scientific dataanalysisScale-out file systems lackglobal sharing capabilitiesReplicating large scientificdatasets problematic

Solution: Swift with Scale-out filesystemHigh-PerformanceScientific applicationsuse filesystem directly toachieve requiredbandwidthImmediately share resultsthrough SwiftSecure sharing withKeystone

To summarize, Swift-on-Fileenables you to:

Quickly deploy object storage on a scale-out filesystemA single system that allows you to access your data as both files andobjectsExtend an existing Swift cluster

What's next?Come hear more about it during the Swift design sessions on Friday

Come join us...Github: https://github.com/stackforge/swiftonfile

Launchpad: https://launchpad.net/swiftonfileirc: #swiftonfile