Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014

Ceph, Open Source, and the Path to Ubiquity in

StorageAACS – 2014 Apr 02

Copyright © 2014 by Inktank | Private and Confidential

WHO?

2

Patrick McGarry

Director, Community - Inktank

/. -> ALU -> P4 -> Inktank

scuttlemonkey

FOSS 4 LYFE!


AGENDA

3

INDUSTRY MUSINGS

INTRO TO CEPH

ARCHITECTURE

COMMUNITY

INKTANK CEPH ENTERPRISE


THE FORECAST

By 2020over 15 ZB of data will be stored.1.5 ZB are stored today.

4


THE PROBLEM

Existing systems don’t scale

Increasing cost and complexity

Need to invest in new platforms ahead of time

2010 2020

IT Storage Budget

Growth of data

5


THE SOLUTION

PAST: SCALE UP

FUTURE: SCALE OUT

6

INTRO TO CEPH


HISTORICAL TIMELINE

8

RHEL-OSP & RHEV Support FEB 2014

MAY 2012Launch of Inktank

OpenStack Integration 2011

2010Mainline Linux Kernel

Open Source 2006

2004 Project Starts at UCSC

Production Ready Ceph SEPT 2012

2012CloudStack Integration

OCT 2013Inktank Ceph Enterprise Launch

Xen Integration 2013


ARCHITECTURAL COMPONENTS

9

RGWA web services

gateway for object storage, compatible

with S3 and Swift

LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,

PHP)

RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors

RBDA reliable, fully-distributed block device with cloud

platform integration

CEPHFSA distributed file

system with POSIX semantics and

scale-out metadata management

APP HOST/VM CLIENT


CEPH UNIFIED STORAGE

FILE SYSTEM

BLOCK STORAGE

OBJECT STORAGE

Keystone

Geo-ReplicationNative API

10

Multi-tenant

S3 & Swift

OpenStack

Linux Kernel

iSCSI

Clones

Snapshots

CIFS/NFS

HDFS

Distributed Metadata

Linux Kernel

POSIX


TRADITIONAL STORAGE VS. CEPH

Single Purpose

TRADITIONALENTERPRISE STORAGE

Multi-Purpose, Unified

Hardware Distributed Software

Single Vendor Lock-in Open

Hard Scale Limit Exabyte Scale

11


CEPH WITH OPENSTACK

OPEN STACK

KEYSTONE API

SWIFT API CINDER APIGLAN

CE API

NOVAAPI

CEPH STORAGE CLUSTER(RADOS)

CEPH OBJECT GATEWAY

(RGW)

CEPH BLOCK DEVICE(RBD)

HYPERVISOR

(Qemu/KVM)

12


WEB APPLICATION STORAGE

WEB APPLICATION

APP SERVER

APP SERVER

APP SERVER

CEPH STORAGE CLUSTER(RADOS)

CEPH OBJECT GATEWAY

(RGW)

CEPH OBJECT GATEWAY(RGW)

13

APP SERVER

S3/Swift S3/Swift S3/Swift S3/Swift


MULTI-SITE OBJECT STORAGE

WEB APPLICATIONAPP

SERVER

CEPH OBJECT GATEWAY

(RGW)

14

CEPH STORAGE CLUSTER

(US-EAST)

WEB APPLICATIONAPP

SERVER

CEPH OBJECT GATEWAY

(RGW)

CEPH STORAGE CLUSTER

(EU-WEST)


ARCHITECTURE


ARCHITECTURAL COMPONENTS

16

RGWA web services

gateway for object storage, compatible

with S3 and Swift

LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,

PHP)

RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors

RBDA reliable, fully-distributed block device with cloud

platform integration

CEPHFSA distributed file

system with POSIX semantics and

scale-out metadata management

APP HOST/VM CLIENT

17

DISK

FS

DISK DISK

OSD

DISK DISK

OSD OSD OSD OSD

FS FS FSFSbtrfsxfsext4

MMM

17

18

M

M

M

HUMAN

18

19

Monitors:Maintain cluster mapProvide consensus for distributed decision-making

Must have an odd numberThese do not serve stored objects to clients

MOSDs:•One per disk (recommended)•At least three in a cluster•Serve stored objects to clients•Intelligently peer to perform replication tasks•Supports object classes

19

20

M

M

M

CLIENT

0110

datametadata

20

21

Metadata ServerManages metadata for a POSIX-compliant shared filesystem

Directory hierarchyFile metadata (owner, timestamps, mode, etc.)

Stores metadata in RADOSDoes not serve file data to clients

Only required for shared filesystem

21

CRUSH Pseudo-random placement algorithm Ensures even distribution Repeatable, deterministic Rule-based configuration Replica count Infrastructure topology Weighting

CRUSH

22

23

10 10 01 01 10 10 01 11 01 10

10 10 01 01 10 10 01 11 01 10

hash(object name) % num pg

CRUSH(pg, cluster state, rule set)

23

24

CLIENT

??

24

25

25

26

26

27

CLIENT

??

27

Ceph Developer Summit

29

• Recent: “Giant”

• March 04-05• wiki.ceph.com • Virtual

(irc, hangout, pad, blueprint, youtube)

• 2 days (soon to be 3?)

• Discuss all work

• Recruit for your projects!

New Contribute Page

30

• http://ceph.com/community/Contribute

• Source tree• Issues• Share

experiences• Standups• One-stop shop

New Ceph Wiki

31

Accepted as a mentoring organization 8 mentors from Inktank & Community http://ceph.com/gsoc2014/ Student applications being evaluated Hope to turn this into academic outreach

Google Summer of Code 2014

32

http://ceph.com/gsoc2014/

Ceph Days

33

• inktank.com/cephdays

• Recently:London, Frankfurt, NYC, Santa Clara

• Aggressive program

• Upcoming:Sunnyvale, Austin, Boston, Kuala Lumpur

Meetups

34

• Community organized

• World wide• Wiki• Ceph-

community• Goodies

available• Logistical

support• Drinkup to

tradeshow

We haven’t forgotten! Looking for potential founding members Especially important to keep the IP clean

Ceph Foundation

35

Coordinated Efforts

36

• Always need help

• CentOS SIG• OCP• Xen• Hadoop• OpenStack• CloudStack• Ganetti• Many more!


http://metrics.ceph.com

37

THE PRODUCT

Copyright © 2014 by Inktank | Private and Confidential39

INKTANK CEPH ENTERPRISEWHAT’S INSIDE?

Ceph Object and Ceph Block

Calamari

Enterprise Plugins (2014)

Support Services

Subscription-based

Priced on capacity

Single price for all protocols


Ceph Inktank Ceph Enterprise

Open-Source Ceph ✔ ✔

Object Storage ✔ ✔

Block Storage ✔ ✔

File System ✔

Management API ✔ ✔

Management GUI ✔

Hyper-V Support Q3 2014

SNMP Support Q3 2014

24x7 Support ✔

Bug Prioritization ✔

FEATURE COMPARISON

41


ROADMAPINKTANK CEPH ENTERPRISE

42

1.2 2.0

CEPH

CALAMARI

PLUGINS

Erasure Coding

RHEL7 Support

Cache Tiering

User Quotas

RADOS Management

Analytics Hosted/SaaS

SNMP, Hyper-V

Ceph 0.77 Firefly Ceph 0.87 “H-Release”

April 2014 September 2014

CephFS

Ubuntu 14.04 Support

VMware

HDFS Support

iSCSI

Intelligent Objects

QoS

2015


Emperor Giant

H

I

J

Inktank Ceph Enterprise v1.1 (Dumpling LTS until May 2015)

Inktank Ceph Enterprise v1.2 (Firefly LTS until November 2015)

RELEASE SCHEDULE

43

2013 2014 2015

Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2

FireflyDumpling


QUESTIONS?

44

THANK YOU!

Patrick McGarryDirector, Community

[email protected]/ceph

@scuttlemonkey@ceph

Technology

Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014