Upload
redhatstorage
View
443
Download
1
Embed Size (px)
Citation preview
CONTAINERIZED STORAGE
Kapil Jain, Verizon Labs Narendra Narang, Red Hat
With special thanks to:Subhajit Mukherjee and Rehan Tahir at Verizon LabsJeff Darcy, Ben England, Shyam Ranganathan and Sayandeb Saha at Red Hat
Tuesday, January 19th 2016
Brief Bio: Narendra N. Narang - Sr. Cloud Storage Solutions Architect at Red
Hat - At Bloomberg for 11 years, where primary
focus was on storage and emerging technologies
- 17 years experience working in various information technology roles at New York financial institutions.
Brief Bio: Kapil Jain - Verizon Cloud platform team (storage) - Verizon Terremark - Verizon labs - Dell Equallogic SAN management team - Progress Relational Database SQL Engine Team
Agenda - Strategic Business initiatives - Technology Stack - FSaaS - Migrating to the cloud - Architecture - State of the Union - Business benefits - Proposed futures - The Red Hat Engagement
Verizon Labs- Center of excellence and
innovation - Platform for Internet Services.- Charter for the team
Strategic Business Initiatives - Cloud platform for internal applications
created on commodity hardware, open-source software
- Resilient, scalable and highly available platform.
- Services for enterprise applications - Migration path for existing applications - Achieve substantial cost savings over
existing alternatives
Key Technologies & Vendors
- CoreOS – open-source lightweight OS ideal for container environment
- EMC – Elastic Cloud Storage (ECS) for the object store
- MESOSPHERE – mesos orchestration tool for deploying apps in a cluster-computing platform
- Red Hat – elastic, distributed, replicated , shared file system.
FSAAS- Software Defined Storage (SDS)- Containerized micro-services.- Highly Available- Disaster tolerance- High Performance- Support Structure
Performance & Costs Scaling of SDS Architecture
Number of Storage Nodes
Rea
ds /
Writ
es T
hrou
ghpu
t (m
Bps
)
Software
Defi
ned S
cale-
out S
torag
e
( Glus
terFS)
Traditional Enterprise NAS Storage
Re-architecture of high performance NFS to scale-out achieves lower costs
Number of Storage Nodes
Tota
l Sto
rage
Cos
ts ($
)Trad
itiona
l Ente
rprise
NAS S
torag
e
Software DefinedScale-out Storage( GlusterFS)
Storage Performance Scalability Storage Costs Scalability
Migrating to the CloudMigrating from a traditional NAS environment to a SDS environment
- Characterization and segmentation of different NAS workloads on based on performance requirements
- Mapping those workloads to software-defined-storage ( GlusterFS)
- Comparing and validating the reliability of Red Hat Gluster Storage running on commodity off-the-shelf hardware with a traditional enterprise class NAS appliance
- Aligning dynamic scaling of the persistent/storage layer (shared filesystem) with the dynamic scaling requirements of the application workload.
To meet our objectives and overcome these challenges, we engaged Red Hat.
ETH/IB
app app app
Converged Computing Architecture: Mixed App and Storage Compute Workloads
App-only servers
Converged servers
app
app +
Storage-only servers
Storage stack imposes only3% - 10% load on compute processing
Applications and storage stacks can co-exist on same compute substrate. This achieves higher server utilization and lower operational costs across the cloud
Architecture
Containerized Docker containers running on commoditized converged hardware
State of the Union - Simplified Deployment via containers - Synchronous replication for high availability - Geo-Replicated for disaster recovery - Seamless upgrade and rollback - Dynamic scaling (up) of capacity as needed - Performance and health metrics collection via
container monitoring agents and sending alerts.
Examples of Usage and Scale.Volume Media Bricks Size Geo -
ReplicatedFile Store HDD 28 112 TB
and growing
Yes
Cache SSD 20 8 TB No
Geo-Rep Vehicle
HDD 12 48 TB Yes
Implementation Challenges- DNS to resolve Gluster container hostnames
- Persistence of Gluster metadata
- Gluster FUSE client optimization
- Geo-replication performance
Solved Challenges• New Technology - Trailblazing issues• Provisioning resources based on demand, not
projections• Unproven Software, production issues• Storage in docker environment is relatively
unsolved• Deploying and orchestration • Automated Seamless upgrade.
BUSINESS BENEFITS
- Leveraging stable proven technology in a new environment
- Higher Resource utilization, reducing costs and capex requirements
- Increase speed and reliability of product delivery, reducing time to market and increase speed of innovation
- Reduce cost and reliance on specialized hardware by running on commoditized hardware with open source software
- For disaster recovery, we have reduced the time to access backups from a couple of days to a couple of hours
- Reduced dependence on external clouds.
Proposed Futures- Data migration - Better Management of resources and gluster
volumes – Heketi ???
Engagement with Red Hat- Weekly Calls- Responsive and meaningful interactions- Developer Support - TAM engagement- Training and certification
Q&A