Deploying WSO2 Middleware on Containers

Deploying WSO2 Middleware on ContainersLakmal WarusawithanaDirector Cloud Architecture, WSO2VP, Apache Stratos

Imesh GunaratneSoftware Architect, WSO2

Agenda• An Introduction to Docker and Docker Compose• Building WSO2 Docker Images• Deploying WSO2 Middleware on Docker with Docker Compose• An Introduction to Kubernetes• Deploying WSO2 middleware on Kubernetes• Kubernetes Vs OpenShift• Using Puppet and Hiera for Configuration Management• Automated WSO2 Carbon Cluster Discovery• An Introduction to Mesos DC/OS• Deploying WSO2 Middleware on Mesos DC/OS• CloudFoundry Current Limitations and the Future

An Introduction to Docker

What are containers?

https://sysadmincasts.com/episodes/31-introduction-to-docker















What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


What is Docker?


Docker File System


How to Install Docker?• For Linux

– wget -qO- https://get.docker.com/ | sh• For Mac

– https://docs.docker.com/engine/installation/mac/• For Windows

– https://docs.docker.com/engine/installation/windows/

https://docs.docker.com/engine/installation/mac/

https://docs.docker.com/engine/installation/mac/

https://docs.docker.com/engine/installation/windows/

https://docs.docker.com/engine/installation/windows/

Demo: Docker Basics

Cinema clapperboard icon source: http://www.flaticon.com/free-icon/graph_102534

Building WSO2 Docker Images

Building WSO2 Docker Images• Clone https://github.com/wso2/dockerfiles• Let's build WSO2 APIM Docker image.

– Navigate wso2am folder and follow the ReadMe.

https://github.com/wso2/dockerfiles

An Introduction to Docker Compose

An Introduction to Docker Compose• Compose is a tool for defining and running

multi-container Docker applications

• In compose yaml can define your application’s services

• Using a single command, you create and start all the services from your configuration

• To Install Docker Compose– https://docs.docker.com/compose/install/

https://docs.docker.com/compose/install/

https://docs.docker.com/compose/install/

Deploying WSO2 Middleware on Docker with Docker Compose

Deploying WSO2 Middleware on Docker with Docker Compose

• Clone https://github.com/lakwarus/wso2am-deployment-patterns

• Navigate to your patten• Follow the ReadMe

– docker-compose up

https://github.com/lakwarus/wso2am-deployment-pattens



Demo: Deploying WSO2 API-M on Docker with Docker Compose


An Introduction to Kubernetes

An Introduction to Kubernetes• An open source container cluster management system• Started by Google with decade and half experience on

running containers at scale (Borg)• Written in Go• Can be deployed on any Infrastructure/IaaS• Over 600 contributors

http://kubernetes.io/v1.0/docs/whatisk8s.html

Kubernetes High Level Architecture

Kubernetes Pods

● A group of containers which can share resources and context

● Shared namespaces:○ PID namespace (processes visible globally)○ network namespace (same IP and port

space)○ IPC namespace (SystemV IPC or POSIX)○ UTS namespace (share a hostname)

http://kubernetes.io/v1.0/docs/user-guide/pods.html

Pod

C1 C2 C3

Kubernetes Replication Controllers

● Provides features for replicating pods:○ Auto-healing○ Manual scaling○ Rolling updates○ Multiple release tracks

http://kubernetes.io/v1.0/docs/user-guide/replication-controller.html

Pod TemplateReplicas = n

Pod 1 Pod 2 Pod n

Replication Controller

Kubernetes Horizontal Pod Autoscaler

● Provides features for horizontally autoscaling pods○ CPU consumption based

http://kubernetes.io/v1.1/docs/user-guide/horizontal-pod-autoscaler.html

Pod TemplateReplicas = n

Pod 1 Pod 2 Pod n

Replication Controller

Horizontal Pod Autoscaler

Kubernetes Services

● Service types:○ ClusterIP (default)○ NodePort○ LoadBalancer

● Service discovery:○ DNS○ Environment variables

● Client IP based Session Affinity

http://kubernetes.io/v1.0/docs/user-guide/services.html

Node IP: 172.17.8.102

Port: 9443

Domain Name: service1IP: 10.2.10.20

Port: 9443NodePort: 32001

Protocol: TCP

Pod 1 Pod 2 Pod n

Service

L1

L1 L1 L1

Node

Overlay Network

● Unique IP addresses● Provides pod to pod networking● Options available:

○ Flannel (default)○ OpenVSwitch○ Weave○ Calico

http://kubernetes.io/v1.1/docs/admin/networking.html

WSO2 Carbon Reference Architecture 1

Pod 1 Pod 2

Manager Cluster Services

Pod 3 Pod 4 Pod n

Worker Cluster Services

Manager Replication Controller

Worker Replication Controller

Client

Gov Reg

Conf Reg

User Store

WSO2 Carbon Reference Architecture 2

Pod 1

Gateway Mgr Services

Pod 2 Pod 3

Gateway Worker Services

Client

Pod 4 Pod 5

Key Manager Services

Pod 6 Pod 7

Store Services

Gateway Mgr RC

Gateway Worker RC

Key Manager RC

Store RCGov Reg

Conf Reg

User Store

API-M DB

Pod 8 Pod 9

Publisher Services

Publisher RC

Setting Up Kubernetes

How to Setup Kubernetes• git clone https://github.com/imesh/kubernetes-vagrant-

setup• Follow the ReadMe

https://github.com/imesh/kubernetes-vagrant-setup



Deploying WSO2 Middleware on Kubernetes

Deploying WSO2 Middleware on Kubernetes

● WSO2 Kubernetes Artifacts Git Repository○ Kubernetes Membership Scheme○ Kubernetes Services○ Kubernetes Replication Controllers

https://github.com/wso2/kubernetes-artifacts

Demo: Deploying WSO2 API-M on Kubernetes


An Introduction to OpenShift

Introduction to OpenShift• Based on top of Docker containers and the

Kubernetes container cluster manager• OpenShift 3 adds

– Developer and operational centric tools to enable rapid application development

– Easy deployment and scaling– Long-term lifecycle maintenance for small and

large teams and applications.

https://www.openshift.com

An Introduction to Puppet + Hiera

An Introduction to Puppet

Prerequisites

Desired Server Runtime

Execute Puppet

Server Distribution

Configuration Data

Artifacts

Pupp

et M

odul

e

Puppet Master

How does it work?

Puppet Modules

http://www.slideshare.net/pk.habi/introduction-to-puppet-16586463

Puppet Agent (Server 1)



Puppet Agent (Server n)

1. Master-agent modePuppet Master

How does it work?

http://www.slideshare.net/pk.habi/introduction-to-puppet-16586463

2. Standalone mode

Puppet (Server 1)

puppet apply

Puppet Modules

Puppet (Server 2)

puppet apply

Puppet Modules

Puppet (Server n)

puppet apply

Puppet Modules

What is Hiera?• Hiera is a key/value lookup tool for configuration data.• It makes Puppet better by keeping site-specific data

out of the manifests.• Configuration data is stored in YAML files.• Avoids repetition by introducing common YAML files.

https://docs.puppet.com/hiera/

A Sample Hiera YAML File

https://github.com/wso2/puppet-modules

Deployment Automation Overview

WSO2 Server

ProfilesProfilesProfiles

Deployment Patterns

Deployment Patterns

Deployment Patterns

Puppet Module

Templates

Hiera Config Data

TemplatesTemplates

Hiera Config Data

Hiera Config Data

Hiera Config Hierarchy: Environments -> Products -> Versions -> Platforms -> PatternsDev | Test | Pre-

Prod | ProdVM | K8S | Mesos

WSO2 Hiera Config Data Hierarchy

https://github.com/wso2/puppet-modules/blob/master/hiera.yaml

Build WSO2 Docker Images Using Puppet

Build WSO2 Docker Images Using Puppet

• Download & extract latest WSO2 Puppet Modules:– https://github.com/wso2/puppet-modules/releases

• Export env variable PUPPET_HOME pointing to above folder• Copy JDK 1.7_80 distribution to:

– <PUPPET_HOME>/modules/wso2base/files

• Copy product distribution to:– <PUPPET_HOME>/modules/<product>/files

• Download & extract WSO2 Dockerfiles:– https://github.com/wso2/dockerfiles/releases

• Execute build.sh:– ./dockerfiles/wso2esb/build.sh -v 4.9.0 -r puppet

https://docs.puppet.com/hiera/

https://github.com/wso2/puppet-modules/releases

https://github.com/wso2/puppet-modules/releases

https://github.com/wso2/dockerfiles/releases

https://github.com/wso2/dockerfiles/releases

Demo: Building WSO2 ESB Docker Images with Puppet


Automated WSO2 Carbon Cluster Discovery

What does WSO2 Carbon Clustering Provides?

• Distributed Caching• Coordinator Election (Using distributed locks)• Cluster Communication• Implemented using Hazelcast

https://docs.wso2.com/display/CLUSTER44x/Overview

How Carbon Cluster Discovery Works?

Pod 1 Pod 2 Pod n

Service

Server 1 Server 2 Server n

Pod 1

Service

Server 1

Pod 1 Pod 2

Service

Server 1 Server 2

Hazelcast member initialization



1 2 n

Deploying WSO2 Middleware on Mesos DC/OS

An Introduction to Apache Mesos

Image source: Anton Kirillov, Data processing platforms architectures [1]

Mesos is an open-source cluster manager that was initially developed at the University of California, Berkeley in year 2009.

An Introduction to Mesos DC/OS

Cluster Manager

PaaS UI | CLI | Package RepositoryBatch Job Scheduler

Reference Architecture for Mesos DC/OS

C 1 C 2

Manager Marathon Applicatio

C 3 C 4 C n

Worker Marathon Application

Marathon Load Balancer

Gov Reg

Conf Reg

User Store

Client

Manager Marathon Application


Steps for Deploying WSO2 Middleware on Mesos DC/OS

1. Build Docker images for Mesos using Puppet:– git clone -b mesos-platform-dev https://github.com/wso2/puppet-modules.git– Copy prerequisites to [puppet-modules]:

• Oracle JDK• WSO2 product distributions• Mesos membership scheme (optional)

– git clone https://github.com/wso2/dockerfiles.git– bash [dockerfiles]/<product>/build.sh -v [version] -r puppet -s mesos -l [profiles]

https://github.com/wso2/puppet-modules.git

https://github.com/wso2/dockerfiles.git

Steps for Deploying WSO2 Middleware on Mesos DC/OS

2. Deploy Marathon applications:– git clone https://github.com/wso2-incubator/mesos-artifacts

– bash [mesos-artifacts]/<product>/deploy.sh

https://github.com/wso2-incubator/mesos-artifacts

Demo: Deploying WSO2 ESB on Mesos DC/OS


Deploying WSO2 Middleware on Cloud Foundry

An Introduction to Cloud Foundry

An Introduction to Cloud Foundry

Limitations in the Current Architecture

• A container can only expose one transport/port. • Only HTTP traffic can be routed via the router.• SSL get terminated at the router.• Internal routing rules cannot be defined.• Autoscaling can only be done based on CPU usage.

Warning icon source: http://www.flaticon.com/free-icon/warning-triangle_10190

Reference Architecture for Cloud Foundry

C 1 C 2

Manager Marathon Applicatio

C 3 C 4 C n


CF Router

Gov Reg

Conf Reg

User Store

Client

Manager CF Application

Worker CF Application

Deployment Architecture

Demo: Deploying WSO2 ESB on Cloud Foundry


Summary

Deploying WSO2 Middleware on Containers

Container Cluster ManagersContainer Runtime Composite

Deployments

Development Production

Questions/Feedback?

Thank You!

#WSO2ConEU

Share your feedback for this session

wso2con.com/app

Technology

Deploying WSO2 Middleware on Containers