Control Policies for HGI Performance in Resource Sharing

Control Policies for HGI Performance in ResourceSharing Networks

Amarjit BudhirajaDepartment of Statistics & Operations Research

University of North Carolina at Chapel Hill

Joint work with Dane Johnson

Institute for Mathematics and its Applications

Amarjit Budhiraja Department of Statistics & Operations Research University of North Carolina at Chapel Hill Joint work with Dane JohnsonControl Policies for HGI Performance in Resource Sharing NetworksInstitute for Mathematics and its Applications 1

/ 32

Background

Resource Sharing Networks – Massoulie and Roberts(2000).

Goal: Allocate capacities of various resources to the various job types in a ‘suitablemanner’.

One popular class of allocation schemes: Proportional Sharing [Kelly(1997), Kelly,Maulloo and Tan (1998), Kelly and Williams(2004), Kang, Kelly, Lee, Williams(2009)].

Harrison, Mandayam, Shah and Yang[HMSY(2014)] introduced: HierarchicalGreedy Ideal[HGI] performance.

HGI strives for two things:

Workload minimizationMinimization of holding cost subject to a given workload.

Advantages of HGI motivated policies illustrated in HMSY(2014) throughsimulation and formal analysis.


/ 32

Background

Open Problem[HMSY(2014)]: Formulate control policies for resource sharingnetwork that achieve HGI performance in the heavy traffic limit.

Main Result:

We formulate a set of sufficient conditions under which such policies exist.

These conditions cover all the examples studied in HMSY(2014), including the‘negative example’[Due to Srikant].

One of the main ingredient in implementation is identifying a certain viable rankingmap.

The policy is given in terms of suitable ‘thresholds’ and ‘safety stocks’.

Proof of convergence to HGI performance uses large deviation estimates of the formfirst introduced in Bell and Williams[(2001), (2005)].


/ 32

Resource Sharing Networks

J types of jobs and I resources for processing jobs.

An I × J dim. matrix K describes relations between jobs and resources.

Kij = 1 if job j requires processing from resource i . 0 otherwise.

Capacity of resource i is Ci .

Jobs of type j arrive with iid exponential inter arrival times {ηrj (k)}k∈N with ratesλrj . Each such job brings an exponentially distributed work {∆r

j (k)} with mean size1/µrj .

A job of type j is simultaneously processed by all associated resources at a rate xjdecided by the controller.

Rate allocations must satisfy capacity constraints: Kx ≤ C .

A job departs when the integrated flow rate assigned to it equals the size of the job.


/ 32

Example

I = 3, J = 6Resource 1: K11 = 1,K14 = 1,K16 = 1, K1j = 0 otherwise.Resource 2: K22 = 1,K24 = 1,K25 = 1,K26 = 1, K2j = 0 otherwise.Resource 3 K33 = 1,K35 = 1,K36 = 1, K3j = 0 otherwise.

C1 C2 C3

λ5

λ6

λ4

λ3λ2λ1


/ 32

Assumptions

Stability and Heavy Traffic:

C > K%r , where %rj = λrj /µrj .

limr→∞ λrj = λj > 0, limr→∞ µrj = µj > 0.

limr→∞ r(C − K%r ) = v∗ > 0.

Local Traffic:[Kang, Kelly, Lee, Williams(2009)]

For each resource i there is a unique job type j(i) that only uses resource i .


/ 32

State Equations and Control Policies

Basic Poisson Processes:

Arj (t) = max

{k :

k∑i=1

ηrj (i) ≤ t

}, S r

j (t) = max

{k :

k∑i=1

∆rj (i) ≤ t

}.

Rate Allocation Policy: A J-dimensional, absolutely continuous, nonnegative,non-decreasing, ‘non-anticipative’, stochastic process {B r (t)}

B rj (t) is the amount of type j work processed by time t.

Queue Length Process: J dimensional.

Qr (t) = qr + Ar (t)− S r (B r (t)) ≥ 0.

Capacity Utilization Process: I dimensional. T r (t) = KB r (t).

Require: T r (t) ≤ Ct.

Unused Capacity Process: I dimensional. I r (t) = Ct − T r (t).


/ 32

Performance Criteria

Let Qr (t) = Qr (r2t)/r be scaled Queue-length associated with a policy B r .

Consider two types of cost: Fix h ∈ RJ , h > 0.

Infinite horizon discounted cost:

J rD(B r , qr ).

=

∫ ∞0

e−θtE(h · Qr (t)

)dt.

Long-term cost per unit time:

J rE (B r , qr ).

= lim supT→∞

1

T

∫ T

0

E(h · Qr (t)

)dt.

A Difficult Goal: Construct simple form asymptotically optimal policies.

B r ,∗ is asymptotically optimal if for any other policy {B r}.

lim supr→∞

J r (B r ,∗, qr ) ≤ lim supr→∞

J r (B r , qr ).

A Less Ambitious Goal: [HMSY(2014)] Construct simple form policies that achieveHGI performance asymptotically.


/ 32

Equivalent Workload Formulation

Harrison(2000) , Harrison - Van Mieghem(1997)

Let G r = Kdiag(1/µr ) and consider the I -dimensional workload process

W r (t) = G r Qr (t).

Two step procedure for constructing asymp. opt. control

construct an asymptotically optimal workload process W r ,∗(t).construct an optimal way to distribute workload among various job-types.

First Step:C(w)

.= inf{h · q : q ≥ 0,Gq = w}, w ∈ RI

+.

Let W r ,∗ minimize the cost

lim supT→∞

1

T

∫ T

0E(C(W r (t))

)dt.

Second Step: Let q∗ : RI+ → RJ

+ be a continuous map s.t. h · q∗(w) = C(w). Then

Qr ,∗ ≈ q∗(W r ,∗).


/ 32

EWF: Asymptotic Formulation

State Equation:W (t) = w + ΛX (t)− v∗t + Y (t) ≥ 0

where Λ is I × J with rank I , X is a J-dimensional BM.

Here Y is the control process – nondecreasing, nonnegative, nonanticipative, RCLL.

Cost Function:

J(Y ,w) = lim supT→∞

1

T

∫ T

0E (C(W (t)))


/ 32

EWF: Asymptotic Formulation

In general optimal Y ∗ in the asymptotic formulation is hard to get.

Instead consider the Y that minimizes W coordinate wise.

The corresponding state process W is a I -dimensional RBM:

W (t) = Γ(w + ΛX − v∗ι)(t).

Let π be the unique stationary distribution of the RBM . Then the cost associatedwith Y is ∫

R+I

C(w)π(dw) = HGI.

if C is monotonic, Y is in fact optimal and HGI is the optimal cost in the EWF.


/ 32

Main Result

Restatement of HMSY Open Problem: Construct a simple form policy B r such that

limr→∞

J r (B r , qr ) = HGI =

∫R+I

C(w)π(dw).

Main Result: Under the assumption that there exists a ranking map for thejob-types, there is an explicit policy B r that achieves HGI asymptotically.

The policy is given in terms of certain thresholds/safety stocks of order rα (withα ∈ (0, 1/2)).

Proofs use large deviations estimates and Lyapunov function constructions.


/ 32

Main Result

In general finding a ranking map may be computationally hard but in many cases ittakes a simple explicit form.

In particular all examples in HMSY[2LLN, 3LLN, C3LN] are covered with a simpleand explicit ranking map.

The ‘negative’ example in HMSY also admits a simple ranking map.

All ‘linear networks’ [Massoulie and Roberts] are covered.

If C is monotonic then the result gives an asymptotically optimal policy.

There are non-monotonic C for which a (explicit) ranking exists and there aremonotonic C for which the ranking does not exist.


/ 32

Three Types of Jobs

A type j arriving job produces a holding cost h · ej = hj .

The resulting increase in workload vector is

µ−1j [K1,j ,K2,j , . . .KI ,j ]′ .= gj .

The best way to distribute this amount of workload among queues produces thecost C(gj). Note C(gj) ≤ hj .

If C(gj) < hj we should get rid of this job.

Such job types are referred to as primary. Collection of primary jobs denoted as Sp.


/ 32

Three Types of Jobs

All remaining jobs are called secondary, the collection of such jobs is Ss .

Jobs that require service from only one resource are always secondary.

Let Sm be collection of all secondary jobs that need more than one resource andS1 the jobs that require only one resource.

Ss = Sm ∪ S1.

Then jobs can be partitioned as Sp ∪ S1 ∪ Sm.


/ 32

A Final Assumption

Definition Let |Sm| = M. A ranking is a map R : {1, . . .M} → Sm with certainproperties.

Assumption There exists a ranking map R.

Given a ranking map, under our policy, the job-types have a hierarchy of the form:

Sp � S1 � R(M) � R(M − 1) · · · � R(1).

Recall that K% = C . Can interpret %j as the nominal capacity allocation to type jjobs.

The jobs higher in the hierarchy are ‘more expensive’ and, under the policy, will seemore than nominal allocation in a suitable sense.

A key consequence of the existence of a ranking is that it gives a simple formexplicit minimizer q∗(w) for the LP problem.


/ 32

Resource Allocation Policy

Let 0 < α < 1/2, 0 < c1 < c2.

τ j2l = inf{t ≥ τ j2l−1 : Qrj (t) ≥ c2r

α},

τ j2l+1 = inf{t ≥ τ j2l : Qrj (t) < c1r

α},

τodd is when queue is depleted and τeven is the next time it is stocked.

No processing between τodd and τeven: Let

E rj (t).

= 1{t ∈[τ j2l−1, τ

j2l

)for some l > 0}c

Rate allocation policy B r (t) =∫[0,t] x

r (s)ds satisfies

x rj (t) = y rj (t)E rj (t)


/ 32


δ.

=minj %j2J .

Stocked Job types:σr (t)

.={j : Qr

j (t) ≥ c2rα}

Resources associated with stocked job types:

ωr (t).

= {i : for some j ∈ σr (t),Kij = 1}.

Job types with rank higher than k associated with resource i .

ζki.

= { job types associated with resource i not in {R(1), . . .R(k)}}.


/ 32


Primary jobs. For j ∈ Sp

yj(t).

= [%j + δ]1{j∈σr (t)} + [%j −δ

J2M+3]1{j 6∈σr (t)}

Jobs in Sm. For k ∈ {1, . . .M}

yR(k)(t).

=

%R(k) − 2k−M−2δ, if ζki ∩ σr (t) 6= ∅ for all i ∈ NR(k)

%R(k) + 2k−M−2δ, if ζki ∩ σr (t) = ∅ for some i ∈ NR(k) and R(k) ∈ σr (t)

%R(k) − 2−k−M−2δ, if ζki ∩ σr (t) = ∅ for some i ∈ NR(k) and R(k) /∈ σr (t).

Jobs in S1. For j ∈ S1

yj(t).

= [Ci(j) −∑

l 6=j :Ki(j),l=1

yl(t)]1{i(j)∈$r (t)} + [%j − δ]1{i(j)6∈$r (t)}


/ 32

Resource Allocation Policy:Discussion

Primary jobs. For j ∈ Sp

yj(t).

= [%j + δ]1{j∈σr (t)} + [%j −δ

J2M+3]1{j 6∈σr (t)}

If the associated queue is stocked then it gets higher than nominal rate allocation andotherwise a lower than nominal allocation.


/ 32


Jobs in Sm. For k ∈ {1, . . .M}

yR(k)(t).

=

%R(k) − 2k−M−2δ, if ζki ∩ σr (t) 6= ∅ for all i ∈ NR(k)

%R(k) + 2k−M−2δ, if ζki ∩ σr (t) = ∅ for some i ∈ NR(k) and R(k) ∈ σr (t)

%R(k) − 2−k−M−2δ, if ζki ∩ σr (t) = ∅ for some i ∈ NR(k) and R(k) /∈ σr (t).

Consider j = R(M).

First line: Every associated resource has at least one job-type rated higher with a stockedqueue. Rate allocated to job-type R(M) is lower than nominal.

Second line: There is at least one associated resource such that none of its job-types thatare rated higher that R(M) has a stocked queue and the queue for job-type R(M) isstocked. Allocate a flow rate higher than nominal.

Third line: Allocate Lower than nominal flow rate if the queue R(M) is not stocked.


/ 32


Jobs in S1. For j ∈ S1

yj(t).

= [Ci(j) −∑

l 6=j :Ki(j),l=1

yl(t)]1{i(j)∈$r (t)} + [%j − δ]1{i(j)6∈$r (t)}

If the resource i(j) has some stocked queue, allocate job j all remaining capacity ofresource i(j).

If not, assign less than nominal allocation.


/ 32

Discussion of the Ranking Map

Example: I = 4, J = 7, µi = 1 for all i and h1 = h2 = h3 = h4 = 4, h5 = 6, h6 = 7,h7 = 13.

In this example Sp = ∅, Sm = {5, 6, 7}.

C1 C2 C3

λ6

λ7

λ5

λ3λ2λ1

C3

λ4

R(1) will be the least expensive job among {5, 6, 7}

In this example R(1) = 7.


/ 32

Discussion of the Ranking Maph1 = h2 = h3 = h4 = 4, h5 = 6, h6 = 7, h7 = 13

C1 C2 C31

111

C3

1

C1 C2 C31

1

C3

1

1

C1 C2 C3

1

1

1

C3

1

C1 C2 C3

1

1

1

1

C3

1


/ 32

Sufficient Conditions for Existence of the Ranking Map

Any network with Sm = ∅ trivially satisfies the condition.

Consider the network with I = 3, J = 6 and h1 = h2 = h3 = 1, h4 = h5 = h6 = 4. HereSm = ∅.

C1 C2 C3

λ5λ4

λ3λ2λ1

λ6


/ 32


Any network with Sm a singleton also trivially satisfies the condition.

In particular any linear network (e.g. 2LLN, 3LLN of HMSY) satisfies the condition.


/ 32


Let Nj denote the set of resources associated with job-type j .

For all j , k ∈ Sm, either Nj ⊂ Nk or Nk ⊂ Nj or Nj ∩ Nk = ∅.

C1 C2 C3λ7

λ5

λ3λ2λ1

C3

λ4

λ6


/ 32

Another Sufficient Condition for Existence of the RankingMap

All expensive jobs have the property that their every minimal cover is efficient.

The example C3LN of HMSY satisfies this condition. Here I = 3, J = 6 and hi = 1 forall i . A ranking map is given as R(1) = 6, R(2) = 4, R(3) = 5.

C1 C2 C3

λ5

λ6

λ4

λ3λ2λ1


/ 32

Another Sufficient Condition for Existence of the RankingMap


Example: I = 4, J = 8, hj = 1 for all j . Here R(8) = 1, R(5) = 2, R(6) = 3, R(7) = 4.

C1

C2

C3

λ5

λ3

λ2

λ1

C4

λ4

λ6

λ7

λ8


/ 32

A Second Sufficient Condition for Existence of the RankingMap


The ‘negative example’ of HMSY satisfies the sufficient condition. I = 6, J = 9, hj = 1.R(1) = 7, R(2) = 8, R(3) = 9.

C1

λ6λ3

λ4λ1λ4

C2

C3

C4

C5

C6

λ5λ2

λ9

λ8λ7


/ 32

An Example where Ranking Does not exist

Suppose I = 3, J = 6, µi = 1 for all i ,

h1 = h2 = h3 = 5, h4 = 7, h5 = 8, h6 = 11.

C1 C2 C3

λ5

λ6

λ4

λ3λ2λ1

Here Sm = {4, 5, 6}. Ranking does not exist.


/ 32

An Example where Ranking Does not exist

C1 C2 C3

λ5

λ6

λ4

λ3λ2λ1

Workload cost and its minimizer. The workload C for this example can be given explicitlyas follows. Let for w ∈ R3

+, w12.

= w1 ∧ w2, w23.

= w2 ∧ w3, w123.

= w1 ∧ w2 ∧ w3.

C(w).

=

5w2 + 2w1 + 3w3, if w2 ≥ w1 + w3

3w1 + 4w2 + 4w3, if w1 + w3 > w2 ≥ w1 ∨ w3

5(w1 + w2 + w3) + w123 − 3w12 − 2w23, if w1 ∨ w3 > w2

.

Optimal q∗(w)

q∗(w) =(0, w2 − w1 − w3, 0, w1, w3, 0), if w2 ≥ w1 + w3

(0, 0, 0, w2 − w3, w2 − w1, w1 + w3 − w2), if w1 + w3 > w2 ≥ w1 ∨ w3

(w1 − w12, w2 + w123 − w12 − w23, w3 − w23, w12 − w123, w23 − w123, w123), if w1 ∨ w3 > w2

.

Note that C is nondecreasing. In particular the HGI performance is also the optimal costin the associated BCP.


/ 32

Documents

Control Policies for HGI Performance in Resource Sharing