Ceph, Docker, Heroku Slugs,CoreOS and Deis Overview

[EXTENDED]lorieri Nov/2014 @againstty0

Welcometothefuture

“Ceph's main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-

available”

● 100% distributed

● CephFS○ For POSIX sharing○ not really 100% (active-standby)

● LibRados

○ RBD■ For Blocks

○ Rados Gateway■ For REST Objects■ S3 and Swift compatible

CephFS

● Rados Gateway

$ s3cmd mb s3://mybucket

Bucket 's3://mybucket/' created

$ s3cmd put ~/myfile.txt s3://mybucket/ --acl-public

/Users/lalorie/myfile.txt -> s3://mybucket/myfile.txt [1 of 1]

15 of 15 100% in 0s 240.81 B/s done

$ curl mybucket.myradosgw.com/myfile.txt

myfile content

● RBD $ rbd create mypool/myimage --size 102400

$ sudo rbd map mypool/myimage --name client.admin

$ sudo mkfs.ext4 -m0 /dev/rbd/rbd/foo

$ sudo mount /dev/rbd/rbd/foo /media/disk1

● CephFS On ALL machines you want to share

$ mount -t ceph \10.0.0.1,10.0.0.2,10.0.0.3:/ /media/disk1

10.0.0.[1-3] are the metadata servers

“Build, Ship and RunAny App, Anywhere”

● 100% portable● Easy API for LXC● Public Repository● Lightweight, 1 process (ideal)● Layers (Union File System)

○ Shares read-only data○ Incremental

● Volumes (not layered volumes)

FROM ubuntu:12.04

RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.listRUN apt-get update && apt-get upgrade -yRUN apt-get install -y gcc make g++ build-essential libc6-dev tcl wgetRUN wget http://download.redis.io/redis-stable.tar.gz -O - | tar -xvzRUN (cd /redis-stable && make)RUN (cd /redis-stable && make test)RUN mkdir -p /redis-data

VOLUME ["/redis-data"]EXPOSE 6379

ENTRYPOINT ["/redis-stable/src/redis-server"]CMD ["--dir", "/redis-data"]

● Slugs

“A slug is a bundle of your source,fetched dependencies, the language

runtime, and compiled/generated outputof the build system - ready for execution.”

Your App

Git

Buildpacks Repo

$ git push

Builder

Code

Runtime

Slug

● Twelve Factor

“Who should read this document?Any developer building applications which run as a service. Ops engineers who deploy or manage

such applications.”

● III - Config

“Store config in the environment

- easy to change between deploys without changing any code;- [not] checked into the code repo accidentally;

- language- and OS-agnostic standard.”

● VI - Processes

“Execute the app as one or more stateless processes

Twelve-factor processes are stateless and share-nothing.”

● IX - Disposability

“app’s processes are disposable, meaning they can be started or stopped at a moment’s

notice”

● XI - Logs

“Treat logs as event streams

… streams of all running processes and backing services”

“Linux for Massive Server Deployments

CoreOS enables warehouse-scale computing on top of a minimal, modern

operating system.”

● 100% distributed● Lightweight● cloud-init for every boot● Automatic Updates

○ 2 boot partitions

Etcd

Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7

Locksmithd ConfdFleet

Systemd Systemd Systemd Systemd Systemd Systemd Systemd

Distributed database

Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7

Reboot Admin Configuration ManagerDistributed Init

New Init New Init New Init New Init New Init New Init New Init

Fleet

● Fleet

DistributedServices

[Unit]

Description=deis-controllerRequires=deis-store-volume.serviceAfter=deis-store-volume.service

[Service]

EnvironmentFile=/etc/environmentTimeoutStartSec=20m

ExecStartPre=/bin/sh -c "IMAGE=`/run/deis/bin/get_image /deis/controller` && docker history $IMAGE >/dev/null || docker pull $IMAGE"ExecStartPre=/bin/sh -c "docker inspect deis-controller >/dev/null && docker rm -f deis-controller || true"

ExecStart=/bin/sh -c "IMAGE=`/run/deis/bin/get_image /deis/controller` && docker run --name deis-controller --rm -p 8000:8000 -e EXTERNAL_PORT=8000 -e HOST=$COREOS_PRIVATE_IPV4 -v /var/run/fleet.sock:/var/run/fleet.sock -v /var/lib/deis/store:/data $IMAGE"

ExecStopPost=-/usr/bin/docker rm -f deis-controller

Restart=on-failureRestartSec=5

[Install]WantedBy=multi-user.target

● Fleet

DistributedServices

“Your PaaS. Your Rules.

An open source PaaS that makes it easy to deploy and manage applications on your

own servers. Deis builds upon Docker and CoreOS to provide a lightweight PaaS

with a Heroku-inspired workflow.”

● CoresOS + Docker + Ceph + Heroku● Twelve-Factor

○ for Deis: must be stateless (no wordpress)● Nginx Router + Wildcard DNS● First release using Ceph

○ more features coming soon● Limited

○ twelve-factor○ only HTTP port (non-http soon)○ must expose ONLY one port

● Installation:○ Install CoreOS and ssh keys then:

$ export DEISCTL_TUNNEL=coreos01

$ curl -sSL http://deis.io/deisctl/install.sh | sh

$ git clone https://github.com/deis/deis.git ; cd deis

$ deisctl config platform set \

domain=mylocalpaas.com

$ deisctl install platform && deisctl start platform

Installation will set the domain inthe distributed database (Etcd)

then load and run the“init” files (Fleet+Systemd)

into the cluster

Everything else happens by the help of twelve-factor

● Install client * at deis git directory

$ pip install ./client/

$ deis register http://mylocalpaas.com

$ deis keys:add

● Create a Docker App

# write some “code”

$ mkdir ~/myapp ; cd ~/myapp

$ git init

$ echo "Hello" > index.html

# create the Dockerfile

$ echo 'FROM myregistry/my-tiny-nginx

ADD ./index.html /usr/share/nginx/www/index.html

EXPOSE 80

ENTRYPOINT ["/usr/sbin/nginx"]

CMD ["-c","/etc/nginx/nginx.conf","-p","/etc/nginx","-g","daemon off;"]' \

> Dockerfile

# create the app

$ git commit -a -m "initial"

$ deis create myapp

● Deploy it

$ git push deis master

…

-----> Building Docker image

…

-----> Pushing image to private registry

…

-----> Launching...

done, myapp:v1 deployed to Deis

http://myapp.mylocalpaas.com

$ curl http://myapp.mylocalpaas.com

Hello

● Scale it

$ deis scale cmd=5

Scaling processes... but first, coffee!

..o

done in 25s

=== myapp Processes

--- cmd:

cmd.1 up (v13)

cmd.2 up (v13)

cmd.3 up (v13)

cmd.4 up (v13)

cmd.5 up (v13)

● Deploy a Heroku App

Available Buildpacks:

● Ruby● Nodejs● Java● Gradle● Grails● Play● Python● Clojure● PHP● Go● Meteorite● Perl● Scala● Dart● Nginx● Apache

● Create it $ git clone https://github.com/deis/example-ruby-sinatra.git

$ cd example-ruby-sinatra

$ deis create myappheroku

Creating application... done, created myappheroku

Git remote deis added

● Deploy it

(suppressed output)

$ git push deis master

-----> Ruby app detected

-----> Compiling Ruby/Rack

-----> Installing dependencies using 1.5.2

Using bundler (1.5.2)

Installing tilt (1.3.6)

Installing rack (1.5.2)

Installing rack-protection (1.5.0)

Installing sinatra (1.4.2)

Your bundle is complete!

-----> Compiled slug size is 12M

-----> Building Docker image

-----> Pushing image to private registry

-----> myappheroku deployed to Deis

http://myappheroku.mylocalpaas.com

$ curl -s http://myappheroku.mylocalpaas.com

Powered by Deis!

● using deis

deis open

● using deis

deis config

● using deis

deis run ( runs in an ephemeral container!)

● using deis

deis limits

● using deis

deis releases / deis rollbacks

● using deis

deis logs

● using deis

deis domains

● Advanced DebugExample

1 - Get the ENTRYPOINT and the CMD of your container

for buildpacks they looks like:

ENTRYPOINT [“/runner/init”]

CMD [“start”, “web”]

2 - Get (or Ask for) the address of the Deis Registry

Ex: 10.1.1.3:5000

* and if you have access to it :)

3 - Get the App’s version

$ deis releases

3 - Run it locally

$ docker run --name debug -d \

10.1.1.3:5000/example-ruby-sinatra:v7 \

/runner/init start web

3 - Then you can do things like run “docker exec”

$ docker exec -t -i debug /bin/bash

● Example:deploy,migrate orreuseyour appin anothersystem

using Chef/Puppet/Ansible to deploy an image from Deis

1 - Makes your automation creates a file with all environment variables on it:

/apps/myapp/environment

export DB=user@host:port

● Example:deploy,migrate orreuseyour appin anothersystem

using Chef/Puppet/Ansible to deploy an image from Deis

2 - Run a container from Deis:

- Use the host’s /apps/myapp directory as a

volume to /my

- Perhaps bind a local port to it

- Try to load the environment from the file

before run the ENTRYPOINT

● Example:deploy,migrate orreuseyour appin anothersystem

using Chef/Puppet/Ansible to deploy an image from Deis

docker run -d --name myapp \

-v /apps/myapp:/my \

-p 80:5000 \

10.1.1.3:5000/myapp:v14 \

bash -c \

‘true; source /my/environment; /runner/init start web’

● Example:deploy,migrate orreuseyour appin anothersystem

… or

make your automation create a Dockerfile:

FROM 10.1.1.3:5000/myapp:v14

ADD /apps/myapp /my

RUN true; source /my/environment;

EXPOSE 5000

ENTRYPOINT [“/runner/init”]’

CMD [“start”,”web”]

$ sudo docker build -t myapp:v14

$ docker run -d --name myapp myapp:v14

Thanks!

lorieri@againstty0