Intelligent Workload Factoring for A Hybrid Cloud Computing Model

Hui Zhang Guofei Jiang Haifeng Chen Kenji Yoshihira Akhilesh Saxena

NEC Laboratories AmericaPrinceton, NJ

July 10th, 2009

www.nec-labs.com

2

IT trends: Internet-based services and Cloud Computing

Trend on IT applications

– Adoption of service oriented architectures & Web 2.0 applications, e.g.

• Software as a Service

(SaaS)

• Mobile commerce

• Open collaboration

• Social networking

• Mashups

Trend on IT infrastructure

– Adoption of cloud computing architecture.

• Computations return to the data centers.

– Promise of management simplification, energy saving, space reduction, …

Blue Cloud

3

What is Cloud Computing?

4+ billion phones by 2010 [Source: Nokia]

Web 2.0-enabled PCs, TVs, etc.

Businesses, from startups to enterprises

An emerging computing paradigm– Data & services : Reside in massively scalable data centers

• Can be ubiquitously accessed from any connected devices over

the internet. The unique points to cloud computing users are the Elastic infrastructure and the Utility model: provision on demand, charge back on use.

[IBM]

4

Cloud Computing is not a reality yet for the majority “Little Investment In Cloud & Grid Computing for 2009.” “CIOs are looking primarily to tested, well-understood technologies

that can result in savings or increased business efficiencies whose support can be argued from a financial point of view” – a survey by Goldman Sachs & Co., July 2008.

Private cloud? Public cloud?Choose one,

please! Let me think about it.

•What about current application platform?•What about data privacy?•What about the performance?•Why the full package?

….

5

Local data center (small, dedicated)

A hybrid cloud computing infrastructure model

Remote cloud (large, pay per

use)

Dynamic Workload

IT customers can have the best Total Cost of Ownership (TCO) strategy with their applications running on a hybrid infrastructure – Local data center, small and fully utilized for best application performance.– Remote cloud, infinite scaling, use on demand and pay per use.

User requests

User requestsWorkload factoringWorkload factoring

6

The economic advantage of hybrid cloud computing model: a case study

To host Yahoo! Video website

workload

A local data center hosting 100%

workload

Hosting solution

Annual Cost ($$)

Cost on running a 790-servers data

center

A local data center:workload of 95% time

Amazon EC2: peak workload of 5% time

+

Amazon EC2 hosting

100% workload

Workload Factoring

US $ 1.384M

†

†

†: assume over-provisioning

over the peak load

Cost on running a 99-servers data

center

+US $ 7.43K

‡

‡

‡: only consider server cost. Amazon EC2 pricing: $0.10 per machine hour – Small Instance (Default).

Hybrid Cloud Computing architecture

Design goals 1.smoothing the workload dynamics in the base zone application platform and avoiding overloading scenarios through load redirection;

2.making trespassing zone application platform agile through load decomposition not only on the volume but also on the application data popularity.

(1) (2)(3)

Intelligent workload factoring: problem formulation

cutjt

jtccutsizeMin )()(

2,1

,...,2,1)1()(

tand

KkforCVW tk

tk

t

)( jtc

Problem statement:• Input:

– requests (r1, r2, …, rM).– data objects (d1,d2, …,dN).– request-data relationship

types (t1=(di,dj,…), t2=(dx,dy,…),…, tR)

• each request belongs to one of the R types

• Output: – Request partition schemes

(R1, R2,…, RK) and data partition schemes (D1,D2,…,DK ) for K locations.

• Problem: a fast online mechanism to make the optimal decision on request and data partition for minimal cross-location data communication overhead.

Solution: – fast data frequency estimation

• Graph model generation– greedy bi-section partition

• Hypergraph partition [Karypis99]

Loc. 1 Loc. 2d1

d3

d2

d5

d4

d6

A hypergraph partition problem model (NP-hard)

Where:

Subject to

request type i; # of requests for type-i;sum of the vertex weights in Location-k

Loc-i capacity of res. type t (1: storage, 2: computing)

jt

)( kt VW

tkC

The fast top-k data item detection algorithm

9

Time t0

Data popularity

Pold

Data popularity

Pnew

Design goal Starting at t0, reach an estimation accuracy on the top-k data items in Pnew

within the minimal time.

The key ideas leading to the detection speedup filtering out old popular data items in a new distribution filtering out unpopular data items in this distribution.

Speedup analysis of the fast top-k algorithm

Problem model– Formally, for a data item T, we define its actual request rate p(T) =

total requests to T/total requests .

– FastTopK will determine an estimate p’(T) such that with probability greater than α.

• We use Zα denote the percentile for the unit normal distribution. For example, if α = 99.75%, then Zα = 3.

Main speedup result– Define an amplification factor X for the rate change of a data item

before and after the historical topk-K filtering as

– Theorem 1: Let NCbefore be the number of samples required for basic

fastTopK, and NCfafter be the number of samples required for filtering

fastTopK

– Notation: X2 speedup of the detection process even with a X-factor on rate amplification due to historical information filtering.

))2

1)((),2

1)((()('

TpTpTp

)(

)(

Tp

TpX

before

after

2X

NN

CbeforeC

after

11

Fast and memory-efficient workload factoring scheme

“Base zone”

Arriving request

n

ny

“Trespassing zone”

Fast top-k data item detection scheme

end

end

“Base zone”

end

yPanic mode?

Does it belong to the top-k list?

12

A complete request dispatching process in hybrid cloud computing

Round-robin dispatching

Arriving request

Trespassing zone

n

end end

LWL

Base zoneWorkload factoring

Workload shaping

Available server?

Drop the request

Admit the request

drop admit

end

Drop the request

endy

Testbed setup

13

EC2 S3

load controller

a http request

request forwarding

Dispatching decision

http replyrtsp://streamServer_x//…

rtsp://streamServer_x//…

IWF

14

Workload factoring evaluation: incoming requests

t0

15

Workload factoring evaluation: results (I)

Workload factoring evaluation: results (II)

16

Base zoneserver capacity

Trespassing zone server capacity

17

Conclusions

We present the design of intelligent workload factoring, an enabling technology for hybrid cloud computing.– Targeting enterprise IT systems to adopt a hybrid cloud

computing model where a dedicated resource platform runs for hosting application base loads, and a separate and shared resource platform serves trespassing peak load of multiple applications.

The key points in our research work– Matching infrastructure elasticity with application agility is a

new cloud computing research topic. – Workload factoring is one general technology in boosting

application agility.• CDN load redirection is a special case.

18

Backup slides

19

Multi-application workload management

Multi-application workload management architecture