Upload
ivan
View
39
Download
0
Tags:
Embed Size (px)
DESCRIPTION
19. April 2013. Toine Beckers. [email protected]. Big Data Evolution. Top500 – Storage, Nov 2012. Accelerating Accelerators. DDN is the leading provider of affordable, high-availability storage for the next generation of particle physics research. - PowerPoint PPT Presentation
Citation preview
ddn.com©2012 DataDirect Networks. All Rights Reserved.
19. April 2013
Toine [email protected]
Big Data Evolution
ddn.com©2012 DataDirect Networks. All Rights Reserved.
Top500 – Storage, Nov 2012
DDN proudly powers…
50% 5 Of the TOP10
55% 11 Of the TOP20
54% 27 Of the TOP50
50% 50 Of the TOP100 30% 148 Of the TOP500over 50% Of the TOP100 GB/sover 70% Of the Lustre Sitesover 65% Of the GPFS Sites
ddn.com©2012 DataDirect Networks. All Rights Reserved.
Accelerating Accelerators
DDN is the leading provider of affordable, high-availability storage for the next generation of particle physics research.
DDN Supplied over 40PB of Storage to the LHC Community in the last 4 years
ddn.com©2012 DataDirect Networks. All Rights Reserved.
LHC Customer Base
• Tier 0– CERN-LHCb (1*S2A9900 SFA10K,
100TB)
• Tier 1– SARA/NIKHEF (13*S2A9900, 6 PB)– KIT (10*S2A9900, 1*SFA10K, 20PB)– IN2P3 (7*DCS9550, 1.5PB)– PIC (2*S2A9900, 2.4PB)– INFN-CNAF (5*S2A9900, 1*SFA10K,
10PB)
• Tier 2– DESY (2*S2A9900, 2*SFA10K, 3PB)– NBI (1*S2A6620, 60*2TB)– INFN-PISA (2*S2A9900, 1*SFA12K,
1PB)– INFN-PADOVA (1*S2A9900, 240TB)– IFCA (1*S2A9900, 1.2PB)– TRIUMF (2*DCS9900, 0.6PB)– SFU (1*S2A9900, 1PB)– UNIV. ALBERTA (1*S2A9550, 100TB)– UNIV. VICTORIA (1*S2A9900, 500TB)– SCINET (2*S2A9900, 1PB)– McGill UNIV. (2*SFA10K, 1PB)
ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN Portfolio
5
• EXAScaler™• 10Ks of Clients• 1TB/s+, HSM• NFS, CIFS
• Storage Fusion Architecture Storage Appliances
• WOS® 2.5• 256 Billion Objects
• GeoReplicated• Cloud Foundation
• Mobile Cloud Access
• 40GB/s/1.7M IOPS• 1,680 Drives: 2 Racks• Embedded Computing
• SAS • SATA • SSD
• 12K
• DirectMon• Enterprise Platform
• Management
• 10GB/s, 600K IOPS• 60 Drives in 4U; 396 Drives in 20U
• Embedded Computing (tba)
• 7700
• Flexible Media Configuration
• Parallel File Storage
• Analytics
• Block
• Cloud • Storage
• GRIDScaler™• 1Ks of Clients• 1TB/s+, HSM• NFS, CIFS
• Storage Fusion Xcelerator (SFX) Flash Acceleration
• SFX • Read
• SFX • Write
• SFX • Context• Commit
• SFX • Instant• Commit
• Cloud • Tierin
g
• Filesystems, • customer applications
• Embedded systems
ddn.com©2012 DataDirect Networks. All Rights Reserved.
SS8460 – Highest Density Enclosure
• 84 Drives – SSD, SAS, SATA - in 4 rack units
• Up to 336 TB (84 x 4)
ddn.com©2012 DataDirect Networks. All Rights Reserved.
SFA12K-40 (Block Appliance)
Highly Parallelized SFA Storage Processing Engine
Active/Active Storage Design35-40GB/s Read & Write Speed
Up to 6.7PB of Disk2.4+ Million Burst IOPS
700K+ Random Spinning Disk IOPS1.7M Sustained Random SSD IOPS64GB+ Mirrored Cache (Protected)
RAID 1/5/6Intelligent Block Striping
DirectProtect™GUI, SNMP, CLI, API
16 x FDR IB Host-Ports8RU Height
• 240Gb/s• Cache Link
• 32-64GB High-Speed Cache
• 32-64GB High-Speed Cache
• SFA Interface Virtualization
• SFA Interface Virtualization
• 960Gb/s Internal SAS Storage Management Network
• 16 x FDR InfiniBand Host Ports
• SFA RAID 5,6
• RAID 5,6
• SFA RAID 1
• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • P• RAID 5,6
• Q• RAID 6
• 1 • 2 • 3 • 4
• 1 • 1m
• Q• RAID 6
• P• RAID 5,6
• Internal SAS Switching
• Internal SAS Switching
• 40 GB/s Read & Write Speed
ddn.com©2012 DataDirect Networks. All Rights Reserved. DDN Confidential
Big Data Solution Shapes
8
► Data separate from compute
► Data inside compute
► Data Separate from Compute
► Compute inside the Data
► Compute and Data all over the place
ddn.com©2012 DataDirect Networks. All Rights Reserved. DDN Confidential
Traditional Parallel Filesystems
9
► Data separate from compute
• Storage Fusion Architecture™ • [Core Storage S/W Engine]
• In-Storage Processing™ Engine & DMA Driver
•D
irect
Mon
™: I
nfra
stru
ctur
e M
anag
emen
t
• ‘Scaler File System Family
• Low-Latency Connect: FC, IB, Memory
• Interrupt-Free Storage Processing
• ReACT™ Adaptive Cache Technology
• DirectProtect™ Data Integrity Management
• Quality of Service Engine
• Storage Fusion Fabric™
• Storage Fusion Xcelerator (SFX)
ddn.com©2012 DataDirect Networks. All Rights Reserved. DDN Confidential
High Single Client Performance
10
► Data Separate from Compute
10
• 1-Trillion Row Big Data
Queries in less than
20s.
• Best Runtime
Ever for Drug Discovery, Warranty,
Risk Analytics
• Up to 570% faster FSI
back-testing and risk
management
ddn.com©2012 DataDirect Networks. All Rights Reserved. DDN Confidential
Move Compute to the Data
11
► Data inside compute
• Full Data Protection Offload
• Innovative I/O Node Data Pipelining
• End:End RDMA for Hadoop Shuffle
• 8+2 Data Protection w/ Real-Time I/O
• Hybrid HDD & SSD Configuration
• 300%+ Density; Flexible Scaling
ddn.com©2012 DataDirect Networks. All Rights Reserved. DDN Confidential
Embedded Systems
12
► Compute inside the Data
• Multi-core CPU Application Processor (AP)
Back-End Storage
Enclosures
•Fi
le S
erve
r
• Dedicated• I/O Bridge
• Multi-core CPU RAID Processor (RP)
• Memory Pointers• (Virtual Disks)
• Multi-Threaded Real-Time• RAID Engine, Hypervisor
• Dedicated• I/O Bridge
• Cache• Memory
Filesystem Clients
• High Speed Bus
•Fi
le S
erve
r
•Fi
le S
erve
r
• Application• Memory
• Virtual Disk• Block
Driver
• Dedicated PCI-e I/O
• ……
ddn.com©2012 DataDirect Networks. All Rights Reserved. DDN Confidential13
► Compute and Data all over the place
ddn.com©2012 DataDirect Networks. All Rights Reserved.
Simplified Data Lifecycle
• Process• Ingest
• Distribute• Store
14 06/2012
ddn.com©2012 DataDirect Networks. All Rights Reserved.15
WOS: WEB Object Store
► Not POSIX-based► Not RAID-based► No Spare Drives► No inode references, no FAT, no
extent lists► No more running fsck► No more volume management► Not based on single-site/box
architecture► 3 commands only:
PUT, GET, DELETE
ddn.com©2012 DataDirect Networks. All Rights Reserved. Confidential - To Be Shared Under NDA Only16
ddn.com©2012 DataDirect Networks. All Rights Reserved.
• Zone 2• Zone 1
WOS Puts & Gets
• App/Web Servers
• A file is uploaded to the application or web server.
• Application makes a call to the WOS client to store (PUT) a new object
• Application returns file to user.
• The WOS client stores the object on a node. Subsequent objects are automatically load balanced across the cloud.
• LAN/WAN
• Database
• The WOS client returns a unique Object ID which the application stores in lieu of a file path. The application registers this OID with the content database.
• Application makes a call to the WOS client to read (GET) the object. The unique Object ID is passed to the WOS client.
• A user needs to retrieve a file.
• The WOS client automatically determines what nodes have the requested object, retrieves the object from the lowest latency source, and rapidly returns it to the application.
• The system then replicates the data according to the WOS policy, in this case the file is replicated to Zone 2.
• OID = 5718a36143521602
• OID = 5718a36143521602
ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN | WOS®
18
•W
OS
Clu
ster
Man
agem
ent
• ObjectAssure™ Erasure Coding• Replication Engine
• WOS Policy Engine
• De-clustered Data Management
• Self-Healing Object Storage Clustering
• Latency-Aware Access Manager
• WOS Core• [Peer:Peer Object Storage]
iRODS
•C
onne
ctor
s
WOS API• C++, Python, Java, PHP, •HTTP, REST interfaces•PUT, GET, DELETE
NFS
• API-based• Integrate applications and
devices more robustly
• Policy driven• Manage truly via policy, rather
than micromanaging multiple layers of traditional filesystems
6/8/12
• Object Placement
• Global, Peer:Peer• Distribute data across 100s of
sites in one namespace
• Self-Healing• Intelligent Data Management
system recovers from failures rapidly and autonomously
CIFS
• S3 & WebDAV APIs
• IOS Smartphones and Tablets
• Multi-tenancy, Reporting and Billing
• Object ID Management
• Data Protection• Replicate and/or Erasure Coding
• Small files, large files, streaming files
• Low seek times to get data• WOS caching servers for massive
streaming data
ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN | WOS™ Deployment & Provisioning
DDN | WOS building blocks are easy to deploy & provision – in 10 minutes or less
• Provide power & network for the WOS Node• Assign IP address to WOS Node
& specify cluster name (“Acme WOS 1”)• Go to WOS Admin UI. WOS Node appears
in “Pending Nodes” List for that cluster
• San Francisco
• New York• London• Tokyo • Simply drag
new nodes to any zone to extend storage
• NoFS
• Drag & Drop the node into the desired zone• Assign replication policy (if needed)
• It’s that simple to add 90TB (30x3)• to your WOS cluster!
ddn.com©2012 DataDirect Networks. All Rights Reserved.
WOS Screenshots
Confidential - To Be Shared Under NDA Only20
ddn.com©2012 DataDirect Networks. All Rights Reserved.
• iRODS
• Server
• ICAT DB
• ..
• iRODS
• Server
• Site 3
• Site 2
• WOS Clust
er
• SFA 10K
• Site 1
• iRODS
• Server
• ICAT DB
• ..
• iRODS
• Server
• iRODS
• Server
• JBOD
• ICAT DB
• ..
• iRODS
• Server
Big Data ConnectediRODS Integration
Site 1
iRODSServer
ICAT DB
.. iRODSServer
iRODSServer
JBODICAT DB
.. iRODSServer
iRODSServer
ICAT DB
.. iRODSServer
Site 3
Site 2
WOS Cluster
SFA 10K
► Now: iRODS/WOS compound resource
► E-iRODS will bring a composable resources• Now iRODS uses WOS as a global storage system• iRODS Talks to all WOS IP addresses via REST
interface• WOS handles the replica management
► Interconnect Parallel Filesystem, Analytics and Cloud storage with Policy-based management
ddn.com©2012 DataDirect Networks. All Rights Reserved.
Enable High Speed Edge Computing with GRIDScaler-WOS Bridge
• Offload – Offload/archive GRIDScaler files to WOS to free up space & improve performance
• Distribute & Federate – Replicate & federate files across other sites for collaboration & disaster protection
• Collaborate – NFS users at remote sites can review & update files at local LAN speeds & share with GRIDScaler users
• GRIDScaler• GRIDSca
ler
ddn.com©2012 DataDirect Networks. All Rights Reserved.
Local Ingest Remote DistributionArchive & Distribution Use Case
• WS
• WOS Zone 1
• GPFS
• WOS Policy driven replication
• WOS REST
• or WOSLIB
• Samba or native CIFS
• Analysis
• Native GPFS or nfs
• WOS Zone 2
• WOS REST
or WOSLI
B
• Viewing App
• NFS or CIFS
• WOS Access NFS
WOS Zone 2
WOS REST or WOSLIB
Viewing App
NFS or CIFS
WOS Access NFS
• GPFS / WOS Access DB Sync
• WOS Access
• Ingest to GS for analysis, GS w/connector distributes to WOS for viewing/processing, GS to WOS DB Sync federates GS & WOS
• Compute Cluster
GS Ingest Process Flow, GS to WOS connector1. Raw data “A” ingested into GS via WS2. Analysis App processes raw sequencer data, writes
resultant file “B” to WOS via GS-WOS Bridge3. WOS Bridge DB synchronizes with WOS Access Sites 4. WOS Access DB syncs across sites which federates the
NFS view across all sites Users utilizing review and/or processing applications @ remote sites can access the resultant files.
• 2
• 1
• 2
• 1• 2
• 4
• 3
• 4
• GPFS –WOS DB Sync
• 3
• 3
• A• B
• B
• B
• Compute Cluster
• 4
ddn.com©2012 DataDirect Networks. All Rights Reserved.
Big Data Connected
Confidential - To Be Shared Under NDA Only24
► Use GPFS HSM features to create candidate file lists
► Fast data movements into object store via 10Gbe/IB
► Maintain metadata and file stubs in GPFS
► Data in WOS moves according to policy
ddn.com©2012 DataDirect Networks. All Rights Reserved.
WOS: Performance Comparison
-
10,000,000,000
20,000,000,000
30,000,000,000
40,000,000,000
50,000,000,000
60,000,000,000
• World’s Fastest
POSIX FS
• World’s Fastest
POSIX FS• (new:
2H11)
• Google!
• EMC: http://reg.cx/1P1E• Lustre; http://wiki.lustre.org/images/1/16/LUG08_Cray_HPCS.pdf• GPFS: http://www.spscicomp.org/ScicomP13/Presentations/IBM/GPFSGunda.pdf• Megastore: http://highscalability.com/blog/2011/1/11/google-megastore-3-billion-writes-and-20-billion-read-transa.html
EMC Atmos Lustre GPFS Megastore WOS 2.0Reads/Day 500,000,000 864,000,000 2,764,800,000 20,000,000,000 55,472,947,200
Writes/Day 500,000,000 864,000,000 2,764,800,000 3,000,000,000 23,113,728,000
ddn.com©2012 DataDirect Networks. All Rights Reserved.26
Summary
► Scaling Up and Out and Deep across the Big Data Landscape requires a range of building block shapes
► Managing the complexity of global distribution requires an alternative model for data management
► Connecting Big Data solutions is the next challenge
► More details: www.ddn.com
http://cern.ch/Computing.Seminars/2013/0131