Orchestrating Docker with OpenStack

Nov 3rd, 2014Orchestrating Docker with OpenStack

ComputeMAGNUM

Containers as a Service

Project

SOLUM“Convert code into a managed application running on

an OpenStack cloud at the push of a button.”

FROM CODE TO MANAGED APP

Docker Docker

Key element of the Solum data plane

Applying HeatOrchestration for Docker API

DockerHeat Resource

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

Installing the plugin

git clone https://github.com/openstack/heat ln -sf $PWD/heat/heat/contrib/docker/plugin; \ /usr/lib/heat/docker"

echo “plugin_dirs=$PWD/heat/heat/contrib/docker/plugin” >> /etc/heat/heat.conf

DockerHeat Resource

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

DockerHeat Resource

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

1. Heat provides a Docker resource2. Docker resource communicates

directly to Docker3. Templates may glue Nova and

Docker resources4. Can deploy containers on top of

VMs or bare-metal instances.

Heat: Cirrosheat_template_version: 2013-05-23 description: Single compute instance running cirros in a Docker container. resources: my_instance: type: OS::Nova::Server properties: key_name: ewindisch_key image: ubuntu-precise flavor: m1.large user_data: #include https://get.docker.io my_docker_container: type: DockerInc::Docker::Container docker_endpoint: { get_attr: [my_instance, first_address] } image: cirros

Applying Heat

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

$ cat template.ymlheat_template_version: 2013-05-23 description: Single compute instance running cirros in a Docker container. resources: my_instance: type: OS::Nova::Server properties: key_name: ewindisch_key image: ubuntu-precise flavor: m1.large user_data: #include https://get.docker.io my_docker_container: type: DockerInc::Docker::Container docker_endpoint: { get_attr: [my_instance, first_address] } image: cirros

$ heat stack-create -f template.yml docker

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

Applying Heat

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

Heat API

VM

Docker

NovaNova resource

Docker resource

Container1

Container2

Container3

HOT

Heat: Dockenstackheat_template_version: 2013-05-23 description: Single compute instance running Tempest resources: my_instance: type: OS::Nova::Server properties: key_name: ewindisch_key image: ubuntu-precise flavor: m1.large user_data: #include https://get.docker.io my_docker_container: type: DockerInc::Docker::Container properties: docker_endpoint: { get_attr: [my_instance, first_address] } image: dockenstack privileged: true cmd: /opt/dockenstack/bin/tempest

heat_template_version: 2013-05-23 description: Two containers, one host with shared volumes resources: my_instance: type: OS::Nova::Server properties: key_name: ewindisch_key image: ubuntu-precise flavor: m1.large user_data: #include https://get.docker.io ftp_container: type: DockerInc::Docker::Container properties: docker_endpoint: { get_attr: [my_instance, first_address] } image: mikz/vsftpd ports: [ “21:21” ] volumes: [ “/ftp” ] name: “FTP” apache_container: type: DockerInc::Docker::Container properties: docker_endpoint: { get_attr: [my_instance, first_address] } image: fedora/apache ports: [ “80:80” ] volumes-from: “FTP” cmd: “rm -rf /var/www; ln -s /ftp /var/www; /run-apache.sh”

Resources: Heat

• http://blog.oddbit.com/2014/08/30/docker-plugin-for-openstack-he/

• http://techs.enovance.com/7104/multi-tenant-docker-with-openstack-heat

MAGNUM Containers as a Service

a new service of the OpenStack Compute program

The Containers TeamWorking Group of the Compute Program

The Containers TeamWorking Group of the Compute Program

• Operating underneath Compute program• Outlined a proposal for Magnum (Nova Mid-cycle)• Magnum would directly orchestrate containers• Would leverage all benefits and features unique to

containers.• It would be the “nova of containers”• It could use Nova to spawn instances to hold

containers.• Those instances may be VMs, Baremetal, or

Containers.

See Adrian Otto’s presentation:Containers for Multi-cloud Apps

Tomorrow: 17:20

Docker plugin for Nova

Nova Integration

Awesome PeopleIan Main (Red Hat) Chris Alfonso (Red Hat) Davanum ‘dims’ (IBM) ChangBo Guo Julien Vey (Numergy) Aaron Rosen (Nicera) Derek Higgins (Red Hat) Paul Czarkowski (Rackspace) Daniel Kuffner

Pedro R Marques (Juniper) Lars Kellogg-Stedman (Red_Hat) Sam Alba (Docker) & more…

What?Enables control of Docker via OpenStack:

• Nova API • Horizon UI

Supports: • launch • terminate • reboot • serial console • snapshot • Glance • Neutron • Pause/unpause

https://wiki.openstack.org/wiki/HypervisorSupportMatrix

Identity Crisis

Nova doesn’t…Link container networks

Pass environment variables Specify working directories

Create docker-volumes Share docker-volumes between containers

Arbitrary commands Arbitrary command-arguments

Pass devices

Nova is a machine abstraction, not a process one.

Docker doesn’t…

• Support mounting devices (unprivileged) • Live-migration is future-speak • Boot from block devices (natively - it’s possible…) • Support Glance natively • PCI pass-through

Havana & IcehouseImage Management

(at-release)

Havana & IcehouseImage Management

• docker-registry worked as a proxy

• Users had to upload through docker-registry.

• docker pulls images through the docker-registry proxy

(at-release)

Havana & IcehouseImage Management

(at-release)

• Glance was only used to provide visibility of Docker images for Nova.

Havana & IcehouseImage Management

(at-release)

• Glance was only used to provide visibility of Docker images for Nova.

• Users could not upload through Glance directly

Havana & IcehouseImage Management

(at-release)

• Glance was only used to provide visibility of Docker images for Nova.

• Users could not upload through Glance directly

• Making that work would require a special procedure for glance uploads.

Havana & IcehouseImage Management

(at-release)

so… we took out the docker-registry instead.

Just Enough Docker

Just Enough Docker

Just Enough Docker

• A subset of Nova features…

Just Enough Docker

• A subset of Nova features…• A subset of Docker features…

Just Enough Docker

• A subset of Nova features…• A subset of Docker features…• Enough for Nova to allow running

Docker-in-Docker.

Just Enough Docker

• A subset of Nova features…• A subset of Docker features…• Enough for Nova to allow running

Docker-in-Docker.• DinD retains most performance

benefits of Docker.

Just Enough Docker

• A subset of Nova features…• A subset of Docker features…• Enough for Nova to allow running

Docker-in-Docker.• DinD retains most performance

benefits of Docker.• DinD is Docker and everything

you love about Docker.

DockerOpenStackNova

novadocker

DockerOpenStackNova

novadockerDocker

DockerOpenStackNova

novadockerDocker

OpenStack API

Docker API

DockerOpenStackNova

novadockerDocker

OpenStack API

Docker API

Docker API

Docker

Docker

Kubernetes Heat

Mesos CloudFoundry

OpenShiftSolum

Magnum

nova-apineutron

nova-compute

VM

VM

docker

docker

Hypervisor

container

container

nova-apineutron

nova-apineutron

nova-apineutron

nova-compute

container

container

Docker

nova-apineutron

nova-compute

container

container

Docker

nova-compute

VM

VM

docker

docker

Hypervisor

container

container

nova-apineutron

Hybrid Nova configuration

nova-compute

container

container

Docker

nova-compute

VM

VM

docker

docker

Hypervisor

container

container

nova-apineutron

nova-compute

container

container

Docker

nova-compute

VM

VM

docker

docker

Hypervisor

container

container

nova-compute

Machine docker

Ironic

container

container

nova-apineutron

Hybrid Nova configuration + Ironic

nova-compute

container

container

Docker

nova-compute

VM

VM

docker

docker

Hypervisor

container

container

nova-compute

Machine docker

Ironic

container

container

nova-api

nova-compute

container docker

Docker

container

container

nova-api

nova-compute

container docker

Docker

container

container

nova-api

nova-compute

container docker

Docker

container

container

KubernetesHeat

Mesos

CloudFoundry

Magnum

Install the plugin

mkdir git-co; cd git-co"

git clone https://github.com/stackforge/nova-docker"

cd nova-driver"

python setup.py install

Configure Nova

Set in nova.conf:"

compute_driver=novadocker.virt.docker.DockerDriver"

docker pull cirros"docker save cirros | glance image-create \ --is-public=True \ --container-format=docker \ --disk-format=raw \ --name cirros

Putting an image into your repository

‘nova boot’

Networking

Nova Network

Please welcome:Ian Main

Testing - Running & Passing

- Get as many tests passing as possible.!- Now running 1726 tests, 0 failures.!- Turned off: volumes resizing & suspending rescue!! !migrations.

Testing - Running & Passing

Working Upstream

Working Upstream

• Added pause and unpause support for docker containers.• Well accepted into the Docker project.

• Dynamic device support needed for Cinder volumes.• First API that modifies running containers.• Docker community wants the user experience to be right.• It will land, just need to get it right

Cinder VolumesUse cases:!• Direct access to block device – not common.!• Mounting file systems.!

- Possible security issues.!- Different from VMs.!- Privileged containers.!- FUSE filesystem support through user namespaces.!

• PoC of boot from volume.

KILONova-Docker

KILO

- Cinder support

KILO

- Cinder support

- Security groups (merged)

KILO

- Cinder support

- Security groups (merged)

- docker-py (merged)

KILO

- Cinder support

- Security groups (merged)

- docker-py (merged)

- privileged containers

KILO

- Cinder support

- Security groups (merged)

- docker-py (merged)

- privileged containers

- more +2 contributors

KILO

Fix our Bugs!use our code…

Q & A

Eric Windisch <erw>@freenode @ewindisch

Ian Main <slower>@freenode