Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization

Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization

Longbo HuangMichael J. Neely

EE@USC

WiOpt 2009

1

*Sponsored in part by NSF Career CCF and DARPA IT-MANET Program

• Problem formulation• Backlog behavior under Quadratic Lyapunov

function based Algorithm (QLA): an example• General backlog behavior result of QLA for

general SNO problems • The Fast-QLA algorithm (FQLA)• Simulation results• Summary

Outline

2

Problem Description: A Network of r QueuesSlotted Time, t=0,1,2,…S(t) = Network State, Time-Varying, IID over slots (e.g. channel conditions, random arrivals, etc.)

x(t) = Control Action, chosen in some abstract set X(S(t))

(e.g. power/bandwidth allocation, routing)

(S(t), x(t)) costs: f(t)=f(S(t), x(t)) generates: Aj(t)=gj(S(t), x(t)) packets to queue j serves: μj(t)=bj(S(t), x(t)) packets in queue j

3

The stochastic problem: minimize: time average cost

subject to: queue stability.

[f(), g(), b() are only assumed to be non-negative, continuous, bounded]

Slotted Time, t=0,1,2,…S(t) = Network State, Time-Varying, IID over slots (e.g. channel conditions, random arrivals, etc.)

x(t) = Control Action, chosen in some abstract set X(S(t))

(e.g. power/bandwidth allocation, routing)

(S(t), x(t)) costs: f(t)=f(S(t), x(t)) generates: Aj(t)=gj(S(t), x(t)) packets to queue j serves: μj(t)=bj(S(t), x(t)) packets in queue j

4

The stochastic problem: minimize: time average cost

subject to: queue stability.

QLA achieves: [G-N-T FnT 06]

Avg. cost: fav <= f*av + O(1/V)

Avg. Backlog: Uav <= O(V)

[f(), g(), b() are only assumed to be non-negative, continuous, bounded]

Problem Description: A Network of r Queues

5

An Energy Minimization Example: The QLA algorithm

U1 U2

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)

The QLA algorithm (built on Backpressure):1. Compute the differentiable backlog Wii+1(t)=max[Ui(t)-Ui+1(t), 0],2. Choose (P1(t), …P5(t) that maximizes: Σi[Wii+1(t)μi(Pi(t)) -VPi(t)] =Σi[Wii+1(t) Si(t) –V]Pi(t) e.g., if S2(t)=2, then if W23(t)*2>V, we set P2(t)=1.

Link 2->3

Goal: allocate power to support the flow with min avg. energy expenditure, i.e.: Min: avg. ΣiPi s.t. Queue stabilityW23(t)

U2

U3

6

An Energy Minimization Example: Backlog under QLA

U1 U2

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)

Queue snapshot under QLA with V=100, first 100 slots:

U1

U2

U3

U4

U5

Goal: Min: avg. ΣiPi s.t. Queue stability

size

time

7


U1 U2

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)


U1

U2

U3

U4

U5

size

time


8


U1 U2

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)


U1

U2

U3

U4

U5

size

time


9


U1 U2

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)


U1

U2

U3

U4

U5

size

time


10


U1 U2

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)

Queue snapshot under QLA with V=100, (U1(t), U2(t)):

(500,400)

t=1:500k


11


U1 U2

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)

t=1:500k t=5k:500k

Queue snapshot under QLA with V=100, (U1(t), U2(t)):

(500,400)


General result: Backlog under QLA

€

q(UV* ) ≥ q(U) + L ||UV

* −U ||

12

Implications: (1) Delay under QLA is Θ(V), not just O(V); (2) The network stores a backlog vector ≈UV*.

Theorem 1: If q(U) satisfies C1:for some L>0 independent of V, then under QLA, in steady state, U(t) is mostly within O(log(V)) distance from UV* = Θ(V).

General result: Backlog under QLA

€

q(UV* ) ≥ q(U) + L ||UV

* −U ||

13

Implications: (1) Delay under QLA is Θ(V), not just O(V); (2) The network stores a backlog vector ≈UV*.

Let’s “subtract out” UV* from the network!

Replace most of the UV* data with Place-Holder bits

Theorem 1: If q(U) satisfies C1:for some L>0 independent of V, then under QLA, in steady state, U(t) is mostly within O(log(V)) distance from UV* = Θ(V).

Fast-QLA (FQLA): Using place-holder bits

A single queue example:

14

Start here

First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q for all t>=t0.(2) Let U(0)=Q, run QLA.



15

Start here

reduced

Advantage: delay reduced by Q, same utility performance.

First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q for all t>=t0.(2) Let U(0)=Q, run QLA.actual backlog



16

Start here

actual backlog ≈Θ(V)

Advantage: delay reduced by Q, same utility performance.

Problem: Q ≈ UV*-Θ(V), delay Θ(V).

reduced

First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q for all t>=t0.(2) Let U(0)=Q, run QLA.



17

FQLA idea: Choose # of place-holder bits Q such that backlog under QLA rarely goes below Q.

Problem: (1) U(t) will eventually get below Q, what to do?(2) How to ensure utility performance?



18

FQLA idea: Choose # of place-holder bits Q such that backlog under QLA rarely goes below Q.

Problem: (1) U(t) will eventually get below Q, what to do? (2) How to ensure utility performance?Answer: use virtual backlog process W(t) + careful pkt dropping



19

FQLA: (1) Choose # of place-holder bits Q such that backlog under QLA rarely goes below Q.

(2) Use a virtual backlog process W(t) with W(0)=Q to track the backlog that should have been generated by QLA.

(3) Obtain action by running QLA based on W(t), modify the action carefully.



20Modifying the action

If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.

If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].

This modification ensures: U(t) ≈ max[W(t)-Q, 0].

































26

Now choose: Q=max[UV*-log2(V), 0](1) ensures: Low delay: average U ≈ log2(V),(2) ensures W(t) rarely below Q, implying:Good utility & few pkt dropped: very few action modifications.

Modifying the action




Fast-QLA (FQLA): Performance

Theorem 2: If condition C1 in Theorem 1 holds, then we have under FQLA-Ideal:

€

U =O(log2(V ))

favFI = fav

* +O(1/V )

Pdrop =O(1/Vc0 log(V ))

27

Recall: under QLA:

€

U =Θ(V ), favQLA = fav

* +O(1/V )

28

Simulation

U1 U2

R(t)

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

Simulation parameters: - V= 50, 100, 200, 500, 1000, 2000,- Each with 5x106 slots,- UV*=(5V, 4V, 3V, 2V, V)T.

Backlog % of pkt dropped

29

Simulation

U1 U2

R(t)

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5


Sample (W1(t),W2(t)) process:V=1000, t=10000:110000

Note: W1(t)>Q1=4952 & W2(t)>Q2=3952

30

Simulation

U1 U2

R(t)

S1(t) S2(t) S3(t) S4(t) S5(t)

U3 U4 U5

Backlog % of pkt dropped

Quick comparisonV=1000, U QLA≈15V=15000U FQLA≈5log2(V)=25060 times better!


Summary

31

1. Under QLA, the backlog vector usually stays close to an “attractor” – the optimal Lagrange multiplier UV*.

2. FQLA subtracts out the Lagrange multiplier from the system induced by QLA by using place-holder bits to reduce delay.

Summary

Note: (1) Theorem 1 also holds when S(t) is Markovian, (2) FQLA-General for the case where UV* is not known, performance similar to FQLA-Ideal, (3) when q0(U) is “smooth”, we prove O(sqrt{V}) deviation bound, (4) The “Network Gravity” role of Lagrange multiplier.Details see ArXiv report 0904.3795

32

1. Under QLA, the backlog vector usually stays close to an “attractor” – the optimal Lagrange multiplier UV*.

2. FQLA subtracts out the Lagrange multiplier from the system induced by QLA by using place-holder bits to reduce delay.

Thank you !

Questions or Comments?

33

Documents

Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization