Scale into Multi-Cloud with Containers

Scale into Multi Cloud with Containers

Imesh GunaratneSenior Technical Lead, WSO2

Agenda

• WSO2 Carbon Reference Architectures for Kubernetes• Implementing a Multi-Cloud with Kubernetes• WSO2 Private PaaS Multi-Cloud Solution• Kubernetes, WSO2 Private PaaS Feature Comparison• WSO2 Vision on Kubernetes

WSO2 Carbon Reference Architectures for Kubernetes

WSO2 Carbon Worker/Manager Reference Architecture for Kubernetes

Pod 1 Pod 2

Manager Cluster Services

Pod 3 Pod 4 Pod n

Worker Cluster Services

Manager Replication Controller

Worker Replication Controller

Client

Gov Reg

Conf Reg

User Store

WSO2 API-M Reference Architecture for Kubernetes

Pod 1

Gateway Mgr Services

Pod 2 Pod 3

Gateway Worker Services

Client

Pod 4 Pod 5

Key Manager Services

Pod 6 Pod 7

Store Services

Gateway Mgr RC

Gateway Worker RC

Key Manager RC

Store RCGov Reg

Conf Reg

User Store

API-M DB

A volume mount for synchronizing API artifacts

Pod 8 Pod 9

Publisher Services

Publisher RC

Implementing a Multi-Cloud with Kubernetes

Why do we need a Multi-Cloud?

● Capacity overflow○ Overflow from on-premise clusters to public when run out of

on-premise capacity● Sensitive workloads

○ Privacy-sensitive workloads to run on on-premise clusters● Vendor lock-in avoidance

○ Workloads to run across multiple cloud providers● High availability

○ Multiple availability zones within a single cloud provider

https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md

Kubernetes Multi-Cloud Reference Architecture 1 (monolithic)

IaaS Provider 1

Kubernetes Cluster 1

Load Balancer 1

Load Balancer 2

Load Balancer n

Central Load Balancer

N1 N2 N3 NnM N1 N2 N3 NnM N1 N2 N3 NnM

IaaS Provider n

Client

IaaS Provider 2

Kubernetes Multi-Cloud Reference Architecture 2 (decoupled & hierarchical)

IaaS Provider 1

Kubernetes Cluster 1

Load Balancer 1

Load Balancer 2

Load Balancer n

Central Load Balancer

N1 N2 N3 NnM

Kubernetes Cluster 2

N1 N2 N3 NnM

Kubernetes Cluster n

N1 N2 N3 NnM

IaaS Provider n

Client

Federation Controller

Core Challenges of Implementing a Multi-Cloud● Location affinity

○ Distributed applications ■ Strictly coupled■ Strictly decoupled■ Preferentially coupled

○ Can pods of a single distributed application partitioned across more than one cluster?

● Cross-cluster service discovery○ Use DNS

https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md

Core Challenges of Implementing a Multi-Cloud● Cross-cluster scheduling

○ Closely related to location affinity○ Have a layer on top of k8s

clusters to handle resource creation

○ Make policy-based decisions● Cross-cluster migration

○ Moving a distributed application from one k8s cluster to another

https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md

Kubernetes Cluster Federation Proposed Architecture (decoupled & hierarchical)

https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md

Ubernetes Control Plane

Kubernetes Cluster 1

GCE

Kubernetes Cluster 2

AWs

Kubernetes Cluster n

Private

Standard Kubernetes API

Ubernetes API

Policy Engine

Auth

Desired Federation

State

Migration Controllers

WSO2 Private PaaS Multi-Cloud Solution

+

WSO2 Private PaaS Architecture

Network Partitioning

IaaS Provider 1 IaaS Provider n

Region 1 Region 2

Zone A Zone B

Network Partition 1

Partition 1 Partition 2

Network Partition 2 Network Partition 3

Region 1

https://docs.wso2.com/display/PP410/Network+Partitions

Composite Application Model

https://docs.wso2.com/display/PP410/WSO2+API+Manager+as+a+Service

Gateway Mgt Cartridge

Gateway Worker Cartridge

Key Manager Cartridge

Publisher/Store Group

Gateway/KM Group

API-M App

Network Partition 1

Network Partition 2

Publisher Cartridge Store Cartridge

Deployment Policy

Autoscaling Policy Deployment

Policy

Autoscaling Policy

Runtime Model (Topology)

https://docs.wso2.com/display/PP410/WSO2+API+Manager+as+a+Service

API-M App

NP1 App Instance NP2 App Instance

Gateway Mgt Cluster

Gateway Worker Cluster Key Manager Cluster

Pub/Store Cluster

Gateway/KM Group

Gateway Mgt Cluster

Gateway Worker Cluster Key Manager Cluster

Pub/Store Cluster

Gateway/KM Group

WSO2 Private PaaS Multi-Cloud Solution

IaaS Provider 1

Kubernetes Cluster 1

Load Balancer 1

Load Balancer 2

Load Balancer n

Central Load Balancer

N1 N2 N3 NnM

Kubernetes Cluster 2

N1 N2 N3 NnM

Kubernetes Cluster n

N1 N2 N3 NnM

IaaS Provider n

Client

Kubernetes, WSO2 Private PaaS Feature Comparison

≅

Feature ComparisonKubernetes WSO2 Private PaaS

Carbon Cluster Discovery Available Available

Artifact Distribution Available (Rolling update) Available (Git based)

Centralized Logging Available (kubectl) Available (Thrift, DAS)

Monitoring & Metering Available (cAdvisor, heapster, grafana)

Available (DAS)

Multi-Tenancy Available (Namespaces) Available (Carbon)

Load Balancing Available (nginx) Available (nginx, haproxy, LVS, ec2)

Feature Comparison (cont.)Kubernetes WSO2 Private PaaS

Horizontal Pod Autoscaling

Available with v1.2.0-alpha.2

Available

Multi-Cloud Deployment A proposal available Available

Horizontal VM Autoscaling

Being discussed Available

Composite Application Model

Being discussed Available

WSO2 Vision on Kubernetes

WSO2 Vision on Kubernetes

● Contribute to Kubernetes on implementing:○ Autoscaling (Horizontal Pod/Node + Vertical Pod scaling)○ Kubernetes Cluster Federation (Ubernetes)

○ Composite Application Model (may be using TOSCA, CAMP, etc)

● Use Kubernetes in the future:○ As the WSO2 PaaS ○ As the Microservices solution platform○ Replace in JVM multi-tenancy using Kubernetes namespaces

Thank You