13
1 KPC-Toolbox KPC-Toolbox Demonstration Demonstration Eddy Zheng Zhang, Giuliano Casale, Evgenia Smirni Eddy Zheng Zhang, Giuliano Casale, Evgenia Smirni Computer Science Department Computer Science Department College of William & College of William & Mary Mary

KPC-Toolbox Demonstration

  • Upload
    ugo

  • View
    93

  • Download
    0

Embed Size (px)

DESCRIPTION

KPC-Toolbox Demonstration. Eddy Zheng Zhang, Giuliano Casale, Evgenia Smirni Computer Science Department College of William & Mary. What is KPC-Toolbox for?. KPC-Toolbox: MATLAB toolbox Workload Traces  Markovian Arrival Process (MAP) Why MAP? Very versatile - PowerPoint PPT Presentation

Citation preview

Page 1: KPC-Toolbox Demonstration

1

KPC-ToolboxKPC-Toolbox DemonstrationDemonstrationEddy Zheng Zhang, Giuliano Casale, Evgenia SmirniEddy Zheng Zhang, Giuliano Casale, Evgenia Smirni

Computer Science DepartmentComputer Science DepartmentCollege of William & College of William & MaryMary

Page 2: KPC-Toolbox Demonstration

2

What is KPC-Toolbox for?

KPC-Toolbox: MATLAB toolbox Workload Traces Markovian Arrival Process (MAP)

Why MAP? Very versatile High variabilityHigh variability & temporal dependence temporal dependence in Time SeriesTime Series Easily incorporated into queuing models

Friendly Interface Departure from previous Markovian fitting tools Fit the automatically (no manual tuning)

Page 3: KPC-Toolbox Demonstration

3

User Interface Requirement: Matlab installed

Input A trace of inter-event times Or a file that already stores the statistics of the trace

E.g., a file stores the moments, autocorrelations and etc

Help Information Type “help FunctionName”,

E.g., “help map_kpcfit” Website Keeps Up-To-Date Tool version

http://www.cs.wm.edu/MAPQN/kpctoolbox.html

Page 4: KPC-Toolbox Demonstration

4

A Simple Example of MAP Two state jumps

1 2 00 bbaa

ccdd

D1 =

D0 =-b-d

-a-c

Time:

a bc

d

I1 I2

I3

Background Jumps Jumps With Arrivals

Arrivals:

Page 5: KPC-Toolbox Demonstration

5

Challenges

How large is the MAP? MAP(n): determine n?

Which trace descriptors are important? Literature: Moments of interval times, lag-1 autocorrelation But, for long range dependentlong range dependent traces?

Need temporal dependencetemporal dependence descriptors

MAP Parameterization Construct MAP(n) with matrices D0 and D1 (2n2 – n entries)

Page 6: KPC-Toolbox Demonstration

6

Example: Important Trace Statistics

1

2

First, second, third moment and lag-1 autocorrelation accurately fit

The queuing prediction ability is not satisfactory!The queuing prediction ability is not satisfactory!

Seagate Web Server Trace Queue Prediction, 80% Utilization

Fit With MAP(2)

100 101 102 103 104 105 106 10710-4

10-3

10-2

10-1

100

Pr(Q

ueue

Len

gth

> X)

X [log]

Trace /M/1 MAP(2)/M/1

Page 7: KPC-Toolbox Demonstration

7

Example: Higher Order Statistics Matter

Much Better Results!Much Better Results!

Queuing Prediction, 80% Utilization

k ,....,, 21

1

2

3

4

……

… ……

… ……

13

14

15

16

Fit with MAP(16)

A grid of joint moments and a sequence of autocorrelations fitted, E[XiXi+kXi+k+h]

100 101 102 103 104 105 106 10710-4

10-3

10-2

10-1

100

Pr[Q

ueue

Len

gth

> X]

X [log]

Trace /M/1MAP(16)/M/1

Seagate Web Server Trace

Page 8: KPC-Toolbox Demonstration

8

Higher Order Correlations V.S. Moments Correlations capture sequence in the time series Correlations are very important

Summary: Matching up to the first three momentsfirst three moments is sufficient Matching higher order correlationshigher order correlations with priority

Fitting Guidelines

Ref: "KPC-Toolbox: Simple Yet Effective Trace Fitting Using Markovian Arrival Processes", G. Casale, E.Z. Zhang, E. Smirni, to appear in QEST’08

Page 9: KPC-Toolbox Demonstration

9

Challenge (1): Determine MAP Size

Definition: lag-k ACF coefficientk ACF coefficient

MAP(n) Property: Linear Recursive Relationship

of nn consecutive ACF coeffs

BIC Size Selection: Linear regression model on

estimated ACF coeffs BIC value assesses goodness of

model size

kMAP Size Selection - Seagate Trace

- 111600

- 111200

- 110800

- 110400

- 110000

0 10 20 30 40 50 60 70

MAP Size

BIC

MAP(8)

MAP(16) MAP(32)

Page 10: KPC-Toolbox Demonstration

10

Challenge (2): Trace Descriptor Matching Kronecker Product Composition (KPC)

KPC Properties: Composition of Statistics Moments are composed from moments of small MAPs

MAP Parameterization by KPC to Match Mean and SCV Exactly Higher order correlations as Close as Possible

Page 11: KPC-Toolbox Demonstration

11

KPC Tool Overview

TraceExtract Statistics

MomentsACF

Correlations…… Size

Selection

MAP(2) MAP(2) MAP(2) MAP(2)……

J = log2N MAP(2)s

MAP(N)MAP(N)

Size of MAPN

Optimization

KPC

This work is supported by NSF grants ITR-0428330 and CNS-0720699

Page 12: KPC-Toolbox Demonstration

12

Thank you!

Page 13: KPC-Toolbox Demonstration

13

What are higher order correlations?

Joint moments of a sequence of inter-arrival times in the time series

Which higher order correlations to fit in KPC? E[XiXi+jXi+j+k], where i can be arbitrary without loss of

generality, and [j,k] chose from a grid of values E.g., [10 100 1000 10000] × [10 100 1000 10000]

= {[10,10], [10,100], [10,10000], …}

Appendix