A Docker Provisioning Module

for VCL

Young-Hyun Oh Xianqing Yu

ETI, IBM Systems Computer Science, NCSU

ohy@us.ibm.com xyu6@ncsu.edu

ICA CON 2015

1

Agenda

Introduction

Apache Virtual Computing Lab (VCL)

Docker

VCL Docker Provisioning Module

Summary

2

3

Reference: http://vcl.ncsu.edu/help/general-information/how-it-works 3

Introduction – VCL (1)

Apache VCL……

Apache Virtual Computing Lab (VCL) Open-source cloud computing platform with the primary goal of delivering dedicated,

custom compute environments to users

Widely used by many universities and colleges around the world

Compute environment: simple virtual machine to complex HPC

Provisioning Methods Bare Metal

Computer Lab Machines

Virtual Machines

VMware ESXi 4.x/5.x

KVM

libvirt

VirtualBox

Cloud OpenStack (VCL 2.4)

OpenNebula (VCL 2.4)

IBM SoftLayer

Docker (Container)

Compute Nodes

Provisioning Engines

Front-end web portal

VCL Scheduler

Database

Management node

VCL Daemon

Physical

Deployer

Virtual

Deployer

New Deployer

Module

Image Library

Physical Server Node Virtual Machine Node

Computing Lab Node

BareMatal Blade Node

New Deployer Environment

(Cloud Computing Resources)

• OpenStack (IaaS)

• IBM SoftLayer (IaaS)

Virtual Machine Node

Virtual Machine Node 4

Introduction – VCL (2)

VCL components……

5

Introduction – Docker (1)

What is Docker ? Docker is an open platform for developers and sysadmins to build, ship, and run

distributed applications [www.docker.com]

Docker allows users to automate the deployment of application into software containers.

Features

Light-Weight

Based on Linux containers: A lightweight VM

Uses Union File System to build layered file system; images (less space)

Uses a copy-on-write file system to track changes (only save changes)

Portable

Can deploy any Linux distributions with kernel 3.8 or later

Windows, OSX :- Virtual Box VM (boot2docker)

Cloud (Amazon EC2, Google Compute Engine, IBM SoftLayer, Microsoft Azure, ...)

Self-sufficient

A container has everything it needs to run

Minimal Base OS

Docker……

6 Reference: https://www.docker.com/whatisdocker/

Do

cker

Virtu

al M

ac

hin

es

Introduction – Docker (2)

VMs vs Docker Containers……

Introduction – Docker (3)

Docker components……

Docker client

Use Command Line Interface (CLIs) or RESTful APIs to interact with Docker Server

Docker server (daemon)

Run as a root-privilege

Use OS’s system call interface

Docker images

Use Union File System

Layered format

Read-only layer

Docker containers

Launch from images

An execution environment

An image format

A set of standard operations.

Docker registries

Docker HUB registry: maintained by Docker Inc.

Private registry: maintained by an individual user

Docker Daemon

App A

noVNC

App B

apache

App C

nginx

Server

Host OS

Container Container Container

Layer 2 (shared) – Base Image

Layer 1 (shared) – bootfs

Docker Registries

Docker images push, pull

Docker images push, pull

Docker HUB Registries

Docker Client

(CLIs) / RESTful APIs

Docker Client

8

Provisioning Engines

User Interface (Web portal)

VCL Scheduler

Database

Management node

VCL Daemon

Physical

Deployer

Virtual

Deployer

Docker

Module

VCL Docker

CLI or

Rest API

Docker Daemon

App A

noVNC

App B

apache

App C

nginx

Server

Host OS

Container Container Container

Layer 2 (shared) – Base Image

Layer 1 (shared) – bootfs

Docker Registries

Docker images push, pull

VCL Docker Provisioning (1)

VCL Docker Integration……

Why?

VCL Docker provisioning module can provide fast start up and less overhead

than other hypervisor-based VCL provisioning modules.

9

VCL Docker Provisioning (2)

Demo.…

VCL 2.4 (released in May 2015)

An Intel blade server with 24 CPUs and 46 GB memory

RHEL 6.4 x86_64

Docker 1.5

Ubuntu 14.04 x86_64 (Kernel 3.8 or later)

A VM with 2 CPUs and 4 GB memory

Container image

Ubuntu 14.04

noVNC (HTML 5)

SSH

Docker Demo

Docker Daemon

H/W

Host OS

Docker & VCL

Web Portal

Database

Mgmt. Node

Docker Host – VM

Docker Local

Registry

Ubuntu

1. User request Ubuntu 14.04

2. Register

3. Find image

4. Pull image

5. Load image to a container

8. Receive access info.

6. Return the access Information of container 7. Result

9. Connect

9

VCL Docker Integration

VCL can support a container provisioning

Docker module can provide faster startup and less overhead than other hypervisor-based VCL provisioning modules

Docker Module will be included in VCL 2.5

Future work

Thorough performance analysis required between Docker and hypervisor-based VCL modules to demonstrate the effectiveness of Docker module

10

Summary……

11

References

VCL

http://vcl.apache.org/

http://vcl.ncsu.edu/

Docker

http://www.docker.com/

http://devops.com/blogs/devops-toolbox/docker-vs-vms/

http://www.dockerbook.com

https://issues.apache.org/jira/browse/VCL-174

https://www.docker.com/whatisdocker/

https://docs.docker.com/terms/layer/

12