A Docker Provisioning Module for VCL - IBM .Docker is an open platform for developers and sysadmins

Embed Size (px)

Text of A Docker Provisioning Module for VCL - IBM .Docker is an open platform for developers and sysadmins

  • A Docker Provisioning Module

    for VCL

    Young-Hyun Oh Xianqing Yu

    ETI, IBM Systems Computer Science, NCSU

    ohy@us.ibm.com xyu6@ncsu.edu

    ICA CON 2015

    1

    mailto:ohy@us.ibm.commailto:xyu6@ncsu.edu

  • Agenda

    Introduction

    Apache Virtual Computing Lab (VCL)

    Docker

    VCL Docker Provisioning Module

    Summary

    2

  • 3

    Reference: http://vcl.ncsu.edu/help/general-information/how-it-works 3

    Introduction VCL (1)

    Apache VCL

    Apache Virtual Computing Lab (VCL) Open-source cloud computing platform with the primary goal of delivering dedicated,

    custom compute environments to users

    Widely used by many universities and colleges around the world

    Compute environment: simple virtual machine to complex HPC

    Provisioning Methods Bare Metal

    Computer Lab Machines

    Virtual Machines

    VMware ESXi 4.x/5.x

    KVM

    libvirt

    VirtualBox

    Cloud OpenStack (VCL 2.4)

    OpenNebula (VCL 2.4)

    IBM SoftLayer

    Docker (Container)

    http://www.cloud-watch.net/http://vcl.ncsu.edu/help/general-information/how-it-workshttp://vcl.ncsu.edu/help/general-information/how-it-workshttp://vcl.ncsu.edu/help/general-information/how-it-workshttp://vcl.ncsu.edu/help/general-information/how-it-workshttp://vcl.ncsu.edu/help/general-information/how-it-workshttp://vcl.ncsu.edu/help/general-information/how-it-workshttp://vcl.ncsu.edu/help/general-information/how-it-works

  • Compute Nodes

    Provisioning Engines

    Front-end web portal

    VCL Scheduler

    Database

    Management node

    VCL Daemon

    Physical

    Deployer

    Virtual

    Deployer

    New Deployer

    Module

    Image Library

    Physical Server Node Virtual Machine Node

    Computing Lab Node

    BareMatal Blade Node

    New Deployer Environment

    (Cloud Computing Resources)

    OpenStack (IaaS)

    IBM SoftLayer (IaaS)

    Virtual Machine Node

    Virtual Machine Node 4

    Introduction VCL (2)

    VCL components

  • 5

    Introduction Docker (1)

    What is Docker ? Docker is an open platform for developers and sysadmins to build, ship, and run

    distributed applications [www.docker.com]

    Docker allows users to automate the deployment of application into software containers.

    Features

    Light-Weight

    Based on Linux containers: A lightweight VM

    Uses Union File System to build layered file system; images (less space)

    Uses a copy-on-write file system to track changes (only save changes)

    Portable

    Can deploy any Linux distributions with kernel 3.8 or later

    Windows, OSX :- Virtual Box VM (boot2docker)

    Cloud (Amazon EC2, Google Compute Engine, IBM SoftLayer, Microsoft Azure, ...)

    Self-sufficient

    A container has everything it needs to run

    Minimal Base OS

    Docker

  • 6 Reference: https://www.docker.com/whatisdocker/

    Do

    cker

    Virtu

    al M

    ac

    hin

    es

    Introduction Docker (2)

    VMs vs Docker Containers

  • Introduction Docker (3)

    Docker components

    Docker client

    Use Command Line Interface (CLIs) or RESTful APIs to interact with Docker Server

    Docker server (daemon)

    Run as a root-privilege

    Use OSs system call interface

    Docker images

    Use Union File System

    Layered format

    Read-only layer

    Docker containers

    Launch from images

    An execution environment

    An image format

    A set of standard operations.

    Docker registries

    Docker HUB registry: maintained by Docker Inc.

    Private registry: maintained by an individual user

    Docker Daemon

    App A

    noVNC

    App B

    apache

    App C

    nginx

    Server

    Host OS

    Container Container Container

    Layer 2 (shared) Base Image

    Layer 1 (shared) bootfs

    Docker Registries

    Docker images push, pull

    Docker images push, pull

    Docker HUB Registries

    Docker Client

    (CLIs) / RESTful APIs

    Docker Client

  • 8

    Provisioning Engines

    User Interface (Web portal)

    VCL Scheduler

    Database

    Management node

    VCL Daemon

    Physical

    Deployer

    Virtual

    Deployer

    Docker

    Module

    VCL Docker

    CLI or

    Rest API

    Docker Daemon

    App A

    noVNC

    App B

    apache

    App C

    nginx

    Server

    Host OS

    Container Container Container

    Layer 2 (shared) Base Image

    Layer 1 (shared) bootfs

    Docker Registries

    Docker images push, pull

    VCL Docker Provisioning (1)

    VCL Docker Integration

    Why?

    VCL Docker provisioning module can provide fast start up and less overhead

    than other hypervisor-based VCL provisioning modules.

  • 9

    VCL Docker Provisioning (2)

    Demo.

    VCL 2.4 (released in May 2015)

    An Intel blade server with 24 CPUs and 46 GB memory

    RHEL 6.4 x86_64

    Docker 1.5

    Ubuntu 14.04 x86_64 (Kernel 3.8 or later)

    A VM with 2 CPUs and 4 GB memory

    Container image

    Ubuntu 14.04

    noVNC (HTML 5)

    SSH

    Docker Demo

    Docker Daemon

    H/W

    Host OS

    Docker & VCL

    Web Portal

    Database

    Mgmt. Node

    Docker Host VM

    Docker Local

    Registry

    Ubuntu

    1. User request Ubuntu 14.04

    2. Register

    3. Find image

    4. Pull image

    5. Load image to a container

    8. Receive access info.

    6. Return the access Information of container 7. Result

    9. Connect

    9

    http://152.46.17.174/vcl/http://152.46.17.174/vcl/http://152.46.17.174/vcl/

  • VCL Docker Integration

    VCL can support a container provisioning

    Docker module can provide faster startup and less overhead than other hypervisor-based VCL provisioning modules

    Docker Module will be included in VCL 2.5

    Future work

    Thorough performance analysis required between Docker and hypervisor-based VCL modules to demonstrate the effectiveness of Docker module

    10

    Summary

  • 11

  • References

    VCL

    http://vcl.apache.org/

    http://vcl.ncsu.edu/

    Docker

    http://www.docker.com/

    http://devops.com/blogs/devops-toolbox/docker-vs-vms/

    http://www.dockerbook.com

    https://issues.apache.org/jira/browse/VCL-174

    https://www.docker.com/whatisdocker/

    https://docs.docker.com/terms/layer/

    12

    http://vcl.apache.org/http://vcl.apache.org/http://vcl.ncsu.edu/http://vcl.ncsu.edu/http://www.openstack.org/http://www.openstack.org/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://devops.com/blogs/devops-toolbox/docker-vs-vms/http://www.dockerbook.com/http://www.dockerbook.com/https://issues.apache.org/jira/browse/VCL-174https://issues.apache.org/jira/browse/VCL-174https://issues.apache.org/jira/browse/VCL-174https://issues.apache.org/jira/browse/VCL-174https://www.docker.com/whatisdocker/https://www.docker.com/whatisdocker/https://docs.docker.com/terms/layer/https://docs.docker.com/terms/layer/