Upload
clinton-kitson
View
1.132
Download
3
Tags:
Embed Size (px)
Citation preview
EC2 Storage for DockerPersistent Containers the Right Way with Container Data Volumes
EMC {code} - @clintonskitson - emccode.github.io 1
Is anyone using containers for persistence today?
EMC {code} - @clintonskitson - emccode.github.io 4
ComposabilityThe Container Eco-System.. A Container Data Volume
EMC {code} - @clintonskitson - emccode.github.io 8
A Heterogeneous Stack and Granular Services
4 Multiple storage platforms, container runtimes, container OSs, container management platforms
4 Fast Bytes
4 Data Services
4 Data Mobility
4 Availability
EMC {code} - @clintonskitson - emccode.github.io 9
Want to use EBS volumes with container granularity?
EMC {code} - @clintonskitson - emccode.github.io 10
How are Docker Images, Graph, and Volume drivers related?--storage-driver= and --volume-driver=
EMC {code} - @clintonskitson - emccode.github.io 11
First, Big Thanks to the Devs!4 David Calavera @calavera
4 Brian Goff @cpuguy83
4 Arnaud Porterie @icecrime
4 Luke Marsden @lukemarsden
4 Michael Crosby @crosbymichael
4 Alexander Morozov @LK4D4mathEMC {code} - @clintonskitson - emccode.github.io 13
4 Jessie Frazelle @frazelledazzell
4 Tibor Vass @tiborvass
4 Jeff Lindsay @progrium
EMC {code} - @clintonskitson - emccode.github.io 14
EXPERIMENTAL (Docker 1.7)4 Volume Enhancements #13161
4 Refactor Volume Interface
4 Prepare for Volumes as 1st class citizens
4 Create a plugin client in Docker to request storage from a Volume Manager API
EMC {code} - @clintonskitson - emccode.github.io 15
Volume Manager4 Simple
4 Plumbing of Storage control plane
4 Advanced
4 Abstraction for storage, os, container runtimes
4 Guest storage introspection
4 Inter/intra-storage platform operationsEMC {code} - @clintonskitson - emccode.github.io 17
REX-Ray
github.com/emccode/rexray
Flocker
github.com/clusterhq/flocker
EMC {code} - @clintonskitson - emccode.github.io 18
Volume Mananger in the stackBlock is more difficult than NAS (NFS,CIFS) - NAS is globally advertised to a host or set of hosts - Block is targeted towards a host - Block can be surfaced as virtual hardware (below), and networked devices (aside) - Block and NAS can be managed for containers in similar ways with the right abstractions
EMC {code} - @clintonskitson - emccode.github.io 19
Linux serves as an OS that receives storage volumes as devices and must present these appropriately as mounts.
OS capabilities- Get mounts- Mount/Unmount- Format
EMC {code} - @clintonskitson - emccode.github.io 23
EBS volumes serve as the storage platform underneath the VMs, creating volumes and attaching SCSI devices on demand.
Storage capabilities- Create/Remove Volumes- Create Volumes from Scratch/Snapshot/Other Volume- Attach/Detach Volumes- Create/Remove Snapshot- Copy Snapshot between regions
EMC {code} - @clintonskitson - emccode.github.io 25
Introspection- rexray get-instance = Who am I?- rexray attach-volume = Attach to me.
Storage Options- --iops = supported IOPS- --volumetype = General Purpose SSD (gp2), Provisioned IOPS (io1), Magentic (standard)- --size = size in GB- --availabilityzone = target AZ
EMC {code} - @clintonskitson - emccode.github.io 26
Basic Storage Workflow- rexray new-volume --size=100 --volumetype=io1 --iops=1000 (storage adapter)- rexray attach-volume --volumeid=vol-ssss (storage adapter - introspection)- rexray format-device --devicename=/dev/xvd_ --fstype=ext4 (os adapter)- rexray mount-device --devicename=/dev/xvd_ --mountpoint=/mnt/test (os adapter)
EMC {code} - @clintonskitson - emccode.github.io 27
The Volume Manager combines both storage and os drivers to simplify management for container runtimes and others. Managing block storage management is simplified and aligns to NAS.
Volume Manager Capabilities- Create/Remove- Mount/Unmount- Path
EMC {code} - @clintonskitson - emccode.github.io 29
What Changes4 Old way
4 Advertise block/NFS storage to instance
4 Create FS (block)
4 Mount device to mount point /mnt/vol1
4 docker run -ti -v /mnt/vol1:/vol1 busybox
4 ..then undo it allEMC {code} - @clintonskitson - emccode.github.io 30
Hybrid of Storage/OS/Volume Workflow- rexray new-volume --volumename=testing10 --size=1- rexray mount-volume --volumename=testing10
EMC {code} - @clintonskitson - emccode.github.io 31
Advanced Storage Workflow - Copy Volume between AZ- rexray new-volume --volumename=testing20 --size=1 (host1)- rexray mount-volume --volumeid=vol-sss (host1)- touch /var/lib/docker/volumes/testing20/test- rexray get-volumemap (host2)- rexray get-volume --volumeid=vol-sss (host2)- rexray new-volume --volumename=testing20b --volumeid=vol-sss --availabilityzone=blah
EMC {code} - @clintonskitson - emccode.github.io 32
How does it work?- docker create - Create Volume- docker start - Mount Volume- docker stop - Unmount Volume- docker rm -v - Remove Container and Volume- docker run - Create, Mount, Unmount Volume- docker run --rm - Create, Mount, Remove Volume
EMC {code} - @clintonskitson - emccode.github.io 34
4 Commands get passed via VolumeDriverProxy
4 /usr/share/docker/plugins/rexray.spec (Unix Socket or TCP)
4 unix:///usr/share/docker/plugins/rexray.sock
4 tcp://127.0.0.1:8080
4 Currently only passes name of volume
4 Different instances can represent different profiles in the short term..
EMC {code} - @clintonskitson - emccode.github.io 35
4 echo -e "GET /Plugin.Activate HTTP/1.1\r\n" | socat unix-connect:/usr/share/docker/plugins/rexray.sock STDIO
4 /usr/src/go/bin/curl-unix-socket -v -X POST -d '{"Name":"test22"}\r\n' unix:///usr/share/docker/plugins/rexray.sock:/VolumeDriver.Create
4 /usr/src/go/bin/curl-unix-socket -v -X POST -d '{"Name":"test22"}\r\n' unix:///usr/share/docker/plugins/rexray.sock:/VolumeDriver.Mount
EMC {code} - @clintonskitson - emccode.github.io 36
4 New Docker way with Volume Drivers
4 docker run -ti --volume-driver=rexray -v volumeName:/externalVol busybox
4 docker run -ti --volume-driver=rexray -v volumeName2:/externalVol2 busybox
EMC {code} - @clintonskitson - emccode.github.io 37
Host Mobility- (host1) docker run -ti --volume-driver=rexray -v testing100:/testing100 busyboxtouch /testing100/testexit- (host2) docker run -ti --volume-driver=rexray -v testing100:/testing100 busyboxls /testing100/test
EMC {code} - @clintonskitson - emccode.github.io 38
4 What does this enable?
4 Run important things with data in containers
4 Composability of Volume features
4 Storage platform services are directly available to containers
4 Volumes can be easily moved between container hosts
EMC {code} - @clintonskitson - emccode.github.io 39
4 Container images remain non-persistent, while the container data volumes are persistent
4 Container OSs remain immutable
EMC {code} - @clintonskitson - emccode.github.io 40
4 How do you customize volumes in this experimental mode? != ideal
4 Integrate against storage platform separately
4 Create them ahead of time
4 Dedicate Unix sockets to storage profiles (size, io, type, az)
EMC {code} - @clintonskitson - emccode.github.io 41
Future- SSL and Authentication tokens- Storage profiles and settings- Volumes as 1st class citizens- Storage services- Volume updates- Asynchronous calls- Distributed volume manager- Direct to storage platforms (Docker -> EC2?)
EMC {code} - @clintonskitson - emccode.github.io 42
4 go get github.com/Soulou/curl-unix-socket
4 docker run --rm -it -v $GOPATH:/go -w /go/src/github.com/emccode/rexraycli golang:1.4.2-cross make release
4 docker run --rm -ti -e DOCKER_EXPERIMENTAL=true --privileged -v ~/go/src/github.com/docker/docker:/go/src/github.com/docker/docker dockercore/docker hack/make.sh binary
EMC {code} - @clintonskitson - emccode.github.io 44