Last Updated: May. 2014

Committer and PMC member of Apache StratosSenior Software Engineer, WSO2

Lahiru Sandaruwan

Cloud Concepts

*

Agenda

• Some core concepts• Vertical/horizontal scaling techniques• Capacity planning• SLA Awareness• Autoscaling• Fault tolerance• Q&A, Discussion

*

Scalability

"The ability of the of a system to continue to operate correctly even when it is scaled to a larger size”

*

Availability

*

Availability

*

High Availability

A system that is designed for continuous operation in the event of a failure of one or more components. However, the system may display some degradation of service, but will continue to perform correctly.

High Availability: The proportion of time during which the service is accessible with reasonable response times should be close to 100%.

*

How to decide required scale (capacity) & availability?

• Average throughput (TPS)• Max throughput (TPS)• Monetary value of a transaction• Average loss & max loss per second of

downtime• Decide on how much to invest based on cost vs.

benefit tradeoff

*

Vertical Scaling

• Get the maximum out of each allocated JVM or resource

• Increase CPU size• Increase memory

*

Horizontal Scaling

*

Load Balancing

• Load balancing algorithms• Round robin• Weighted• Response based• Health check• Failover-only

*

Clustering for scalability

*

Clustering for availability

Group Communication Channel/State replication

*

Capacity Planning

Source: http://srinathsview.blogspot.com/2012/05/how-to-measure-performance-of-server.html

*

Capacity Planning

Throughput = number of completed requests / time to complete the requests

No. of servers = (projected max load * 1.3) / max throughput of one server

max throughput of one server = max throughput of that server for the slowest scenario in the set of use cases

*

Static membership

• Predefined members• Other (non-predefined) nodes cannot join

M2M1

M3 M4

Static group

N

*

Dynamic membership

• No predefined members• Nodes can join & leave

M2M1

M3 M4

Dynamic group

NJoin

*

Hybrid membership• Some predefined (well-known) members, and some

dynamic members• Nodes can join & leave• Membership revolves around the static members

Hybrid group

N

M2M1

M3 M4

Static members

M6M5

M7

Dynamic members

Join (IP, Port)

*

Multicast based membership management

N

M2

M1

M3

M4

Join (IP, Port)

*

Well-known Address (WKA) based membership management

Hybrid group

NWK2

WK1

WK3 WK4

Static members

M6M5

M7

Dynamic members

Join (IP, Port)

Notify

*

Multicast vs. WKA

Multicast WKA

All nodes should be in the same subnet Nodes can be in different networks

All nodes should be in the same multicast domainNo multicasting requirement

Multicasting should not be blocked

No fixed IP addresses or hosts required At least one well-known IP address or host required

Failure of any member does not affect membership discovery

New members can join with some WKA nodes down, but not if all WKA nodes are down

Does not work on IaaSs such as Amazon EC2 IaaS-friendly

Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure

*

● Solves performance, availability, and economic costs issues

● Currently the customer needs a considerable capacity planning

● Reduce cost of cloud: reduce economic cost and energy footprint

SLA Awareness

*

● Monitoring the cloud parameters● Trigger an event if a SLA violation happened● Acting upon the events

SLA Awareness: Solution

*

Auto-scaling

• Scale-out when load increases• Scale-in when load decreases• Always use the optimum amount of resources• Try out

• AWS ELB • Apache Stratos Load Balancer• WSO2 ELB

*

Auto-scaling – steady load

*

Auto-scaling – load increasing

*

Auto-scaling – load increasing

*

Auto-scaling – steady load

*

Auto-scaling – decreased load

*

Auto-scaling – decreased load

*

Auto-scaling – steady load

*

Autoscaling - Analysis & Results

*

Autoscaling - Analysis & Results

*

Single node

Availability

Cost

LOWEST

HIGHEST

*

Primary-secondary

Primary Secondary

Availability

Cost

LOWEST

HIGHEST

*

Primary-secondary, multiple LB

Primary Secondary

keepalived

Availability

Cost

LOWEST

HIGHEST

*

Active cluster, multiple LB

Active Active

keepalived

Active

Availability

Cost

LOWEST

HIGHEST

*

Multi-zone or multi-datacenter Deployment

Region X

Zone 1

Zone 2

Cloud Controller

Availability

Cost

LOWEST

HIGHEST

*

Multi-region deployment

Region X

Zone 1

Zone 2

Region Y

Zone 1

Zone 2

Availability

Cost

LOWEST

HIGHEST

*

Multiple IaaS (hybrid) DeploymentAvailability

Cost

LOWEST

HIGHEST

Private cloud (data center)

Zone 1

Zone 2

Amazon EC2

Zone 1

Zone 2

Rackspace Cloud

Zone 1

Zone 2

*

Sin

gle

Nod

e

Prim

ary-

Sec

onda

ry, s

ingl

e LB

Mul

ti-no

de a

ctiv

e cl

uste

r- S

ingl

e zo

ne

Mul

ti-re

gion

Prim

ary-

Sec

onda

ry,

with

mul

tiple

LB

s

Mul

ti-zo

ne Mul

ti-Ia

aS

Cost of Availability

*

Stratos Architecture

*

References

๏ http://www.cari-info.org/cari-2012/session%201/1B3.pdf๏ http://domino.research.ibm.com/library/cyberdig.

nsf/papers/16D9BBF221540A0785257784004FC33D๏ http://ieeexplore.ieee.org/xpl/login.jsp?

tp=&arnumber=5452504&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5452504

๏ http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5990687&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5990687

๏ http://www.slideshare.net/afkham_azeez/java-colombo-developing-highly-scalable-apps

- Azeez’s talk at Java Colombo

*

DISCUSSION

*

Thanks!