Revolutionizing WSO2 PaaS with Kubernetes & App Factory

Imesh GunaratneSenior Technical Lead, WSO2

Dmitry SotnikovVice President - Cloud, WSO2

Agenda• An Introduction to Linux Containers• An Introduction to Docker• An Introduction to Kubernetes• Deploying WSO2 products on Kubernetes

– Carbon cluster discovery on Kubernetes– Monitoring server health – Artifact distribution– Propagating software updates – Load balancing with Nginx– Demo

Agenda• App Factory and WSO2 App Cloud

– Creating applications and uploading existing apps

– Hosting web applications and API backends– Using databases in the Cloud– Coding and debugging applications with Cloud

IDE or desktop tools– Application lifecycle management– Custom URLs– Teamwork and social development– Issue and task tracking

An Introduction to Linux Containers

Linux Containers

Linux Container Brief for IEEE WG P2302, Boden Russell

Kernel features used by Linux Containers

• Namespaces• cgroups

○ Limit, account & Isolate resource usage (CPU, memory, disk, etc)

• chroot○ Isolate access to disk

• AppArmor• SELinux• seccomp

- mnt (mount points, filesystems)

- pid (processes)- net (NICs, routing)- ipc (inter-process

communication)- uts (hostname)- user (user ids)

- Secure computing/ application sandboxing

- Security modules for controlling access to resources based on security policies

Linux Container Brief for IEEE WG P2302, Boden Russell

LXC - A Hypervisor for Linux Containers

LXC Components

https://linuxcontainers.org/lxc/introduction

• The liblxc library• Language bindings for the API:

○python3, python2○lua○Go○ruby○Haskell

• A set of standard tools to control the containers

• Distribution container templates

More Information on LXC

https://linuxcontainers.org

An Introduction to Docker

Virtual Machines Vs Docker

https://www.docker.com/what-docker

Introduction to Docker

• Started as an internal project within dotCloud

• Initial release in March 2013• Initially used libvirt, LXC &

systemd-nspawn• Introduced libcontainer: A

standard interface for Linux containers

https://en.wikipedia.org/wiki/Docker_(software)

Docker Ecosystem

Enterprise Docker, Adrien BLIND, Aurelien GABET, Arnaud MAZIN

DockerfileFROM debian:7.7MAINTAINER dev@wso2.org

ENV DEBIAN_FRONTEND noninteractiveENV JAVA_HOME /opt/jdk1.7.0_80

ADD packages/jdk-7u80-linux-x64.tar.gz /optCOPY packages/wso2esb-4.8.1.zip /opt/

WORKDIR /opt/RUN apt-get update && apt-get install -y zipRUN unzip /opt/wso2esb-4.8.1.zip

EXPOSE 9763 9443

ENTRYPOINT /opt/wso2esb-4.8.1/bin/wso2server.sh

Docker Image Structure

https://www.docker.com

An Introduction to Kubernetes

Why Container Clusters?

• Avoid single point of failure• Make horizontally scalable• Have more granular

management for distributed applications (microservices)

• Self healing systems

http://googlecloudplatform.blogspot.com/2015/01/what-makes-a-container-cluster.html[image ref] https://www.docker.com/what-docker

Kubernetes Architecture

Docker Host 1

Docker Host 2

Docker Host n

Physical Network

Controller Host

Kube

rnet

es A

PI

Overlay Network (SDN)

Sche

dule

r

Kubernetes Component Architecture

Kubernetes Pods

• A group of containers which can share resources and context

• Shared namespaces:○PID namespace (processes visible

globally)○network namespace (same IP and

port space)○IPC namespace (SystemV IPC or

POSIX)○UTS namespace (share a

hostname)

Pod

C1 C2 C3

http://kubernetes.io/v1.0/docs/user-guide/pods.html

Kubernetes Labels & Selectors• Labels are key/value pairs

attached to objects• Selectors use the label

key to find a collection of objects matched with the same value

○L1 -> ClusterID = C1○L2 -> ClusterID = C2

Pod 1

http://kubernetes.io/v1.0/docs/user-guide/labels.html

L1

Pod 2

L1

Pod 3

L1

Pod 3

L2

Pod 4

L2

Pod Template

Replicas = n

Kubernetes Replication Controllers

• Provides features for replicating pods:

○Auto-healing○Scaling○Rolling updates○Multiple release tracks Pod 1

http://kubernetes.io/v1.0/docs/user-guide/replication-controller.html

Pod 2 Pod n

Replication Controller

Node IP: 172.17.8.102

Port: 9443

Domain Name: service1

IP: 10.2.10.20Port: 9443

NodePort: 32001Protocol: TCP

Kubernetes Services• Nginx based load

balancing rules•Service types:

○ClusterIP (default)○NodePort○LoadBalancer

•Service discovery:○DNS○Environment variables

•Session Affinity

Pod 1

http://kubernetes.io/v1.0/docs/user-guide/services.html

Pod 2 Pod n

ServiceL1

L1 L1 L1

Node

• A distributed key value store

• Read and write values with curl and other HTTP libraries

• Store data in directories, similar to a file system

• Watch a key or directory for changes and react to the new values

https://coreos.com/etcd/

etcd

• A software defined network• Runs an agent, flanneld, on

each host• Uses etcd to store the

network configuration, allocated subnets, and auxiliary data (host ips)

https://github.com/coreos/flannel

flannel

Deploying WSO2 products on Kubernetes

Carbon Cluster Discovery on Kubernetes

Pod 1 Pod 2 Pod n

Service

Server 1 Server 2 Server n

Pod 1

Service

Server 1

Pod 1 Pod 2

Service

Server 1 Server 2

Hazelcast member initialization

Hazelcast member initialization

Hazelcast member initialization

1 2 n

WSO2 Carbon Reference Architecture for Kubernetes

Pod 1 Pod 2

Manager Cluster

Services

Pod 3 Pod 4 Pod n

Worker Cluster Services

Manager Replication Controller

Worker Replication Controller

Client

Gov Reg

Conf

Reg

User

Store

WSO2 ConfiguratorA python module written using Jinja2 template engine for configuring a product using a set of key/value pairs.

4

1 2

3

Deployment Workflow

Build Template Module

Kubernetes Membership

Scheme

Build Docker Image

Import DI to Kubernetes

Nodes

Application Artifacts

Template Module

Product Distribution

Docker Image (DI)

Configurator

JDK

Kernel & Security Patches

Templates

Deploy Kubernetes

ArtifactsKubernetes

Artifacts

3

1 2

Artifact Distribution/Software Updates

Build Docker Image

Import DI to Kubernetes

Nodes

Application Artifacts

Template Module

Product Distribution

Docker Image (DI)

ConfiguratorJDK Execute Rolling Update

Replication Controller

Monitoring Kubernetes Cluster Health

Monitoring Kubernetes Node/Pod Health

cAdvisor

Monitoring Kubernetes Cluster/Node/Pod Health

cAdvisor

Node 1

cAdvisor

Heapster

Node 2 Node n

Grafana

InfluxDB

Demo: Deploying WSO2 ESB on Kubernetes

WSO2 App Factory & App Cloud

WSO2 Platform

Application PaaS from WSO2

WSO2 Platform Deployment Options

Demo Time!

• App Factory and WSO2 App Cloud●Creating applications and uploading existing

apps●Hosting web applications and API backends●Using databases in the Cloud●Coding and debugging applications with Cloud

IDE or desktop tools●Application lifecycle management●Custom URLs●Teamwork and social development●Issue and task tracking

Announcing: ESB in App Cloud / App Factory

Demo Time!

• Create ESB project• Create Properties

pointing to endpoints

• Move through lifecycle and observe change in endpoints invoked

Questions

Thank You