KPC-Toolbox Demonstration

1

KPC-ToolboxKPC-Toolbox DemonstrationDemonstrationEddy Zheng Zhang, Giuliano Casale, Evgenia SmirniEddy Zheng Zhang, Giuliano Casale, Evgenia Smirni

Computer Science DepartmentComputer Science DepartmentCollege of William & College of William & MaryMary

2

What is KPC-Toolbox for?

KPC-Toolbox: MATLAB toolbox Workload Traces Markovian Arrival Process (MAP)

Why MAP? Very versatile High variabilityHigh variability & temporal dependence temporal dependence in Time SeriesTime Series Easily incorporated into queuing models

Friendly Interface Departure from previous Markovian fitting tools Fit the automatically (no manual tuning)

3

User Interface Requirement: Matlab installed

Input A trace of inter-event times Or a file that already stores the statistics of the trace

E.g., a file stores the moments, autocorrelations and etc

Help Information Type “help FunctionName”,

E.g., “help map_kpcfit” Website Keeps Up-To-Date Tool version

http://www.cs.wm.edu/MAPQN/kpctoolbox.html

4

A Simple Example of MAP Two state jumps

1 2 00 bbaa

ccdd

D1 =

D0 =-b-d

-a-c

Time:

a bc

d

I1 I2

I3

Background Jumps Jumps With Arrivals

Arrivals:

5

Challenges

How large is the MAP? MAP(n): determine n?

Which trace descriptors are important? Literature: Moments of interval times, lag-1 autocorrelation But, for long range dependentlong range dependent traces?

Need temporal dependencetemporal dependence descriptors

MAP Parameterization Construct MAP(n) with matrices D0 and D1 (2n2 – n entries)

6

Example: Important Trace Statistics

1

2

First, second, third moment and lag-1 autocorrelation accurately fit

The queuing prediction ability is not satisfactory!The queuing prediction ability is not satisfactory!

Seagate Web Server Trace Queue Prediction, 80% Utilization

Fit With MAP(2)

100 101 102 103 104 105 106 10710-4

10-3

10-2

10-1

100

Pr(Q

ueue

Len

gth

> X)

X [log]

Trace /M/1 MAP(2)/M/1

7

Example: Higher Order Statistics Matter

Much Better Results!Much Better Results!

Queuing Prediction, 80% Utilization

k ,....,, 21

1

2

3

4

……

… ……

… ……

…

13

14

15

16

Fit with MAP(16)

A grid of joint moments and a sequence of autocorrelations fitted, E[XiXi+kXi+k+h]

100 101 102 103 104 105 106 10710-4

10-3

10-2

10-1

100

Pr[Q

ueue

Len

gth

> X]

X [log]

Trace /M/1MAP(16)/M/1

Seagate Web Server Trace

8

Higher Order Correlations V.S. Moments Correlations capture sequence in the time series Correlations are very important

Summary: Matching up to the first three momentsfirst three moments is sufficient Matching higher order correlationshigher order correlations with priority

Fitting Guidelines

Ref: "KPC-Toolbox: Simple Yet Effective Trace Fitting Using Markovian Arrival Processes", G. Casale, E.Z. Zhang, E. Smirni, to appear in QEST’08

9

Challenge (1): Determine MAP Size

Definition: lag-k ACF coefficientk ACF coefficient

MAP(n) Property: Linear Recursive Relationship

of nn consecutive ACF coeffs

BIC Size Selection: Linear regression model on

estimated ACF coeffs BIC value assesses goodness of

model size

kMAP Size Selection - Seagate Trace

- 111600

- 111200

- 110800

- 110400

- 110000

0 10 20 30 40 50 60 70

MAP Size

BIC

MAP(8)

MAP(16) MAP(32)

10

Challenge (2): Trace Descriptor Matching Kronecker Product Composition (KPC)

KPC Properties: Composition of Statistics Moments are composed from moments of small MAPs

MAP Parameterization by KPC to Match Mean and SCV Exactly Higher order correlations as Close as Possible

11

KPC Tool Overview

TraceExtract Statistics

MomentsACF

Correlations…… Size

Selection

MAP(2) MAP(2) MAP(2) MAP(2)……

J = log2N MAP(2)s

MAP(N)MAP(N)

Size of MAPN

Optimization

KPC

This work is supported by NSF grants ITR-0428330 and CNS-0720699

12

Thank you!

13

What are higher order correlations?

Joint moments of a sequence of inter-arrival times in the time series

Which higher order correlations to fit in KPC? E[XiXi+jXi+j+k], where i can be arbitrary without loss of

generality, and [j,k] chose from a grid of values E.g., [10 100 1000 10000] × [10 100 1000 10000]

= {[10,10], [10,100], [10,10000], …}

Appendix