Copyright 2017 FUJITSU LIMITED
Achieving Best Price/Performance for Enterprise Grade SDS with SPDK
Paul von Stamwitz
0
Synopsis
The Enterprise Storage market is rapidly migrating to NVMe-based all-flash configurations.
However, CPU processing power is falling behind the performance gains in storage, creating inefficiencies in both performance and cost.
To address this, Fujitsu is using SPDK toward the goal of achieving both higher performance and an overall lower cost per IOP for mission-critical workloads.
Copyright 2017 FUJITSU LIMITED1
Agenda
Background
Why SPDK
How Fujitsu is using SPDK
What are we interested in
Copyright 2017 FUJITSU LIMITED2
Copyright 2017 FUJITSU LIMITED
ETERNUS protects customer assets forever and ensures continuous business operation
Accelerating data flows with flash storage
Effectively consolidating storage systems
Managing unpredictable data growth with software defined storage
Radically simplifying backup infrastructures
Fujitsu Storage ETERNUS lineup
All-flash system
ETERNUS AF
Hybrid RAID Storage
ETERNUS DX
Software defined hyper-scale storage
ETERNUS CD10000
Backup Appliances
ETERNUS CS
Tape Libraries
ETERNUS LT
3
Copyright 2017 FUJITSU LIMITED
Enterprise SANs transitioning to SDS
Advantages of SDS:
•Easy to take advantage of latest hardware features
•Easy to change hardware generation without changing operations
External storage configurations changing to AFA
Hybrid will continue to be a significant part of the overall market
Market Trends
All-flash arrays (AFAs)
Hybrid flash arrays (HFAs)
HDD only
Source: IDC, Oct. 2016
4
But... CPU/DRAM is the New BottleneckCPU/DRAM not keeping
pace with storage/network
What to do?
Throw more CPU/DRAM at the problem?
Can work
But adds cost
Make the CPU/DRAM more efficient
SPDK
•User space
•Polling
•Zero Copy?
Copyright 2017 FUJITSU LIMITED
$
5
SPDK is More Efficient
Copyright 2017 FUJITSU LIMITED
Linear scaling for 4 NVMe devices on 1 SPDK core for 4K random reads
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
1800000
2000000
1 2 4 8 16 32 64 128 256 512 1024
IOP
S
Queue Depth
1 DEV 2 DEV 4 DEV Kernel
6
Target Market
Use Cases
Mission critical, high availability
DB/OLTP acceleration
Business decision acceleration
Virtualized server infrastructure and VDI
Goals:
Ultimate performance
• Best in class IOPS/latency
Cost efficient
• NVMe – best in class $/IOP
• Further cost reduction via inexpensive SSDs or hybrid configurations
Copyright 2017 FUJITSU LIMITED7
Copyright 2017 FUJITSU LIMITED
Multiple Hardware Configurations on Same SPDK-based software
AFA Configuration
•More performance
SSDs & Nearline HDDsHybrid Configuration
•Comparable price withHDD systems
Scalability
•Scale out & Scale up
Connectivity
•Frontend: FC, iSCSI and NVM over Fabric
•Backend: NVMe & SAS
•Interconnect:Omni-Path, etc.
NVMeSSD
Linux/SPDK
Scale out
Scale Up
Scale out
Node 0
Node 0
Various Configurations by Common Architecture
NVMeSSD
Linux/SPDK
Node n
Linux/SPDK
SASSSD
NL HDD
Node n
Linux/SPDK
SASSSD
NL HDD
8
How Are We Using SPDK
Copyright 2017 FUJITSU LIMITED
Now
Future Interest
Drivers
Storage
Services
Storage
Protocols
iSCSI
Target
NVMe-oF*
Target
SCSI
vhost-scsi
Target
NVMe
NVMe Devices
Blobstore
NVMe-oF*
Initiator
Intel® QuickData
Technology Driver
Block Device Abstraction (BDEV)
Ceph
RBD
Linux
Async IO
Blob
bdevFujitsu
NVMe
NVMe* PCIe
Driver
Released
Q2’17
Pathfinding
vhost-blk
TargetObject
BlobFS
Integration
RocksDB
Ceph
9
Thread Model and DPDK
Minimal dependence on DPDK
Currently rely on rte_timer
Copyright 2017 FUJITSU LIMITED
SSD
SSD
SSD
SSD
SSD
SSD
SSD
SSD
SSD
SSD
SSD
SSD
Storage Services
+
Inter-Node Comm.
Host I/F
Host I/F
Dedicated threads
Thread pool
bdev channels
10
What Are We Interested In
Bdev
Optimize for NVMe
Need ability to support for SAS/SATA
Robust Error Recovery
Timeouts
Aborts
Monitoring
Performance
Health
Hot-plug
User space TCP
Future
Vhost and NVMeF (target)
Copyright 2017 FUJITSU LIMITED
Enterprise
11
Contributions
SPDK community contributions
30+ pull requests
20+ merges
•NVMe-to-SCSI error code translation
•Register callback in the event of a timeout
•Consistent naming
•Abort option in timeout callback
•Etc..
Copyright 2017 FUJITSU LIMITED12
What are we working on now
NVMe “passthrough” command
IOCTL like interface to obtain Health/SMART log pages
Performance Monitoring
PCIe Hot-plug
FW download
Copyright 2017 FUJITSU LIMITED13
Conclusion
SPDK is a compelling architecture for Enterprise Grade SDS solutions
Copyright 2017 FUJITSU LIMITED14
Copyright 2017 FUJITSU LIMITED
Thank you!
Questions?
15
Copyright 2017 FUJITSU LIMITED