Embed Size (px)
Citation preview
PRESENTATION TITLE GOES HERE
Kinetic Open Storage Platform: Enabling Break-through Economics in Scale-out Object Storage
Ali Fenn & James Hughes Seagate Technology
2 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
2020: 7.3 Zettabytes
2010: 100 Exabytes
56% of total = in the
cloud
90% Unstructured data
6.5 Zettabytes of unstructured data stored in the cloud
3 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
If you stacked 4TB hard drives side
by side, they would circle the earth
1 inch
How big is 6.5 Zettabytes?
4 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
6.5 Zettabytes
Using today’s architecture, it could cost cloud data centers:
>$240,000,000,000/year in CAPEX + 1 Year OPEX
5 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
How do we get there?
HDFS
The Opportunity for Change
Where do we go from here?
Ecosystem = Open and Software-Defined
CEPH
6 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
1990s to 2000s
POSIX 1988
RAID 1986
NTFS 1993
XFS 1993
Server
Storage Application File System DB
POSIX File System Volume Manager Driver
Devices SAS Interface SMR, Mapping Cylinder, Head, Sector Drive HDA
Storage Server RAID Battery Backed RAM CACHE
FC
SAS
Look at Legacy ‘Opportunity’
7 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Standard Device Recording
Key/Value Interface
Ethernet Connectivity
Application File System DB
POSIX File System Volume Manager Driver
Devices SAS Interface SMR, Mapping Cylinder, Head, Sector Drive HDA
Storage Server RAID Battery Backed RAM CACHE
FC
SAS
Storage to Enable New World Use Cases
8 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Open Source Key/Value API and libraries Open Source Interface Specification Object storage software partners Systems partners
Storage now fully disaggregated from compute
The Kinetic Open Storage Platform
9 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
4K Sector Transitions
Shingled Magnetic Recording
Advanced Management
Data Security
Simplifying Storage Advancements
= Greater Agility
10 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Performance
Raw throughput IO utilization
Data streams to drive as written
Drive handles space mgmt. (no file system metadata)
Performance Opportunities
11 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
+
+
+
Deploying a Kinetic based architecture could deliver:
Up to 50% lower TCO
TCO Impact
12 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Single Drive – 1 Additional Chip to Start
13 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Libraries, API enable Applications
ProtoBuf TCP/IP/GbE
Proprietary to Seagate
GPL Standard
Proprietary to System
Vendor
• Application • Clustering
• Management
• Storage
• Interconnect
C++, Java, Python, Erlang, DIY
14 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Multiple Masters
ProtoBuf TCP/IP/GbE
Proprietary to Seagate
GPL Standard
Proprietary to System
Vendor
• Application • Clustering
• Management
• Storage
• Interconnect
15 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Multiple Drives, P2P Operations
ProtoBuf TCP/IP/GbE
Proprietary to Seagate
GPL Standard
Proprietary to System
Vendor
• Application • Clustering
• Management
• Storage
• Interconnect
16 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Goals of the Kinetic API
Data movement Get/put/delete/getnext/getprevious Versioned (== for success), options
Multiple masters Authentication/Integrity/Authorization
Cluster-able Simple cluster configuration version enforcement
3rd party copy Management
17 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
1
Management (System Vendor)
Configures the drive
Network Authorized clients
Monitors
Health Statistics Logs
Initiates recovery
Change cluster version 3rdPartyCopy
17
18 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Connector
▪ Connector re-pinned ▪ Two Ethernet connections
Standard HDD Form Factor
19 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
▪ No new ports ▪ Ethernet v.
SAS switch
System Implications
20 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Traditional Architecture
No impact to Data Center Networking
Kinetic Architecture
Network Implications
21 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Performance Opportunities
22 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Map of Operations
23 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Performance Expectations
Same normal performance expectations • Sequential Write: 50 MB/s • Random Write: 50 MB/s • Sequential Read: 50 MB/s • Random Read: 1.2x slower than traditional drives
24 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Write Performance Results
[PRELIMINARY]
25 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
25
Swift - Traditional
26 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
26
Swift - Kinetic
27 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
27
Swift - Kinetic
28 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
28
Basho Riak
29 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
29
Basho Riak - Kinetic
30 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
HDFS - Kinetic
30
31 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Scality Kinetic Model
• Direct data path from clients to kinetic drives
• Native support for file, object and block
• Geo distribution across multiple sites
• Mix of replication and erasure coding
• Geo distributed metadata cluster
32 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Kinetic Fits All Scale-Out Storage
Object Storage Cloud Storage, Cloud backup, Cloud Archive / Cold Storage (Open Stack Swift, S3, Riak CS)
Distributed File System Architectures
Hadoop Distributed File System (HDFS), Google File System (GFS), Ceph, Windows Distributed File System (DFS), FhGFS, GlusterFS, Lustre
Distributed Database and Memory Systems
No SQL: Cassandra, Voldemort, Riak Memory: Memcached
33 2014 Data Storage Innovation Conference. © Insert Your Company Name. All Rights Reserved.
Summary
The Kinetic Open Storage Platform:
• Lowers TCO • Disaggregates storage from compute • Improves performance • Increases innovation agility and efficiency
More info at: developers.seagate.com
