Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization
Longbo HuangMichael J. Neely
EE@USC
WiOpt 2009
1
*Sponsored in part by NSF Career CCF and DARPA IT-MANET Program
• Problem formulation• Backlog behavior under Quadratic Lyapunov
function based Algorithm (QLA): an example• General backlog behavior result of QLA for
general SNO problems • The Fast-QLA algorithm (FQLA)• Simulation results• Summary
Outline
2
Problem Description: A Network of r QueuesSlotted Time, t=0,1,2,…S(t) = Network State, Time-Varying, IID over slots (e.g. channel conditions, random arrivals, etc.)
x(t) = Control Action, chosen in some abstract set X(S(t))
(e.g. power/bandwidth allocation, routing)
(S(t), x(t)) costs: f(t)=f(S(t), x(t)) generates: Aj(t)=gj(S(t), x(t)) packets to queue j serves: μj(t)=bj(S(t), x(t)) packets in queue j
3
The stochastic problem: minimize: time average cost
subject to: queue stability.
[f(), g(), b() are only assumed to be non-negative, continuous, bounded]
Slotted Time, t=0,1,2,…S(t) = Network State, Time-Varying, IID over slots (e.g. channel conditions, random arrivals, etc.)
x(t) = Control Action, chosen in some abstract set X(S(t))
(e.g. power/bandwidth allocation, routing)
(S(t), x(t)) costs: f(t)=f(S(t), x(t)) generates: Aj(t)=gj(S(t), x(t)) packets to queue j serves: μj(t)=bj(S(t), x(t)) packets in queue j
4
The stochastic problem: minimize: time average cost
subject to: queue stability.
QLA achieves: [G-N-T FnT 06]
Avg. cost: fav <= f*av + O(1/V)
Avg. Backlog: Uav <= O(V)
[f(), g(), b() are only assumed to be non-negative, continuous, bounded]
Problem Description: A Network of r Queues
5
An Energy Minimization Example: The QLA algorithm
U1 U2
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)
The QLA algorithm (built on Backpressure):1. Compute the differentiable backlog Wii+1(t)=max[Ui(t)-Ui+1(t), 0],2. Choose (P1(t), …P5(t) that maximizes: Σi[Wii+1(t)μi(Pi(t)) -VPi(t)] =Σi[Wii+1(t) Si(t) –V]Pi(t) e.g., if S2(t)=2, then if W23(t)*2>V, we set P2(t)=1.
Link 2->3
Goal: allocate power to support the flow with min avg. energy expenditure, i.e.: Min: avg. ΣiPi s.t. Queue stabilityW23(t)
U2
U3
6
An Energy Minimization Example: Backlog under QLA
U1 U2
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)
Queue snapshot under QLA with V=100, first 100 slots:
U1
U2
U3
U4
U5
Goal: Min: avg. ΣiPi s.t. Queue stability
size
time
7
An Energy Minimization Example: Backlog under QLA
U1 U2
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)
Queue snapshot under QLA with V=100, first 500 slots:
U1
U2
U3
U4
U5
size
time
Goal: Min: avg. ΣiPi s.t. Queue stability
8
An Energy Minimization Example: Backlog under QLA
U1 U2
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)
Queue snapshot under QLA with V=100, first 1000 slots:
U1
U2
U3
U4
U5
size
time
Goal: Min: avg. ΣiPi s.t. Queue stability
9
An Energy Minimization Example: Backlog under QLA
U1 U2
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)
Queue snapshot under QLA with V=100, first 5000 slots:
U1
U2
U3
U4
U5
size
time
Goal: Min: avg. ΣiPi s.t. Queue stability
10
An Energy Minimization Example: Backlog under QLA
U1 U2
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)
Queue snapshot under QLA with V=100, (U1(t), U2(t)):
(500,400)
t=1:500k
Goal: Min: avg. ΣiPi s.t. Queue stability
11
An Energy Minimization Example: Backlog under QLA
U1 U2
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
R(t) μ1(t) μ2(t) μ3(t) μ4(t) μ5(t)
t=1:500k t=5k:500k
Queue snapshot under QLA with V=100, (U1(t), U2(t)):
(500,400)
Goal: Min: avg. ΣiPi s.t. Queue stability
General result: Backlog under QLA
€
q(UV* ) ≥ q(U) + L ||UV
* −U ||
12
Implications: (1) Delay under QLA is Θ(V), not just O(V); (2) The network stores a backlog vector ≈UV*.
Theorem 1: If q(U) satisfies C1:for some L>0 independent of V, then under QLA, in steady state, U(t) is mostly within O(log(V)) distance from UV* = Θ(V).
General result: Backlog under QLA
€
q(UV* ) ≥ q(U) + L ||UV
* −U ||
13
Implications: (1) Delay under QLA is Θ(V), not just O(V); (2) The network stores a backlog vector ≈UV*.
Let’s “subtract out” UV* from the network!
Replace most of the UV* data with Place-Holder bits
Theorem 1: If q(U) satisfies C1:for some L>0 independent of V, then under QLA, in steady state, U(t) is mostly within O(log(V)) distance from UV* = Θ(V).
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
14
Start here
First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q for all t>=t0.(2) Let U(0)=Q, run QLA.
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
15
Start here
reduced
Advantage: delay reduced by Q, same utility performance.
First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q for all t>=t0.(2) Let U(0)=Q, run QLA.actual backlog
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
16
Start here
actual backlog ≈Θ(V)
Advantage: delay reduced by Q, same utility performance.
Problem: Q ≈ UV*-Θ(V), delay Θ(V).
reduced
First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q for all t>=t0.(2) Let U(0)=Q, run QLA.
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
17
FQLA idea: Choose # of place-holder bits Q such that backlog under QLA rarely goes below Q.
Problem: (1) U(t) will eventually get below Q, what to do?(2) How to ensure utility performance?
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
18
FQLA idea: Choose # of place-holder bits Q such that backlog under QLA rarely goes below Q.
Problem: (1) U(t) will eventually get below Q, what to do? (2) How to ensure utility performance?Answer: use virtual backlog process W(t) + careful pkt dropping
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
19
FQLA: (1) Choose # of place-holder bits Q such that backlog under QLA rarely goes below Q.
(2) Use a virtual backlog process W(t) with W(0)=Q to track the backlog that should have been generated by QLA.
(3) Obtain action by running QLA based on W(t), modify the action carefully.
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
20Modifying the action
If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.
If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].
This modification ensures: U(t) ≈ max[W(t)-Q, 0].
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
21Modifying the action
If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.
If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].
This modification ensures: U(t) ≈ max[W(t)-Q, 0].
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
22Modifying the action
If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.
If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].
This modification ensures: U(t) ≈ max[W(t)-Q, 0].
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
23Modifying the action
If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.
If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].
This modification ensures: U(t) ≈ max[W(t)-Q, 0].
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
24Modifying the action
If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.
If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].
This modification ensures: U(t) ≈ max[W(t)-Q, 0].
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
25Modifying the action
If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.
If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].
This modification ensures: U(t) ≈ max[W(t)-Q, 0].
Fast-QLA (FQLA): Using place-holder bits
A single queue example:
26
Now choose: Q=max[UV*-log2(V), 0](1) ensures: Low delay: average U ≈ log2(V),(2) ensures W(t) rarely below Q, implying:Good utility & few pkt dropped: very few action modifications.
Modifying the action
If W(t)>=Q, same as QLA, admit A(t), serve μ(t), i.e., FQLA=QLA.
If W(t)<Q, serve μ(t), only admit: A’(t)=max[A(t)-(Q-W(t)), 0].
This modification ensures: U(t) ≈ max[W(t)-Q, 0].
Fast-QLA (FQLA): Performance
Theorem 2: If condition C1 in Theorem 1 holds, then we have under FQLA-Ideal:
€
U =O(log2(V ))
favFI = fav
* +O(1/V )
Pdrop =O(1/Vc0 log(V ))
27
Recall: under QLA:
€
U =Θ(V ), favQLA = fav
* +O(1/V )
28
Simulation
U1 U2
R(t)
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
Simulation parameters: - V= 50, 100, 200, 500, 1000, 2000,- Each with 5x106 slots,- UV*=(5V, 4V, 3V, 2V, V)T.
Backlog % of pkt dropped
29
Simulation
U1 U2
R(t)
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
Simulation parameters: - V= 50, 100, 200, 500, 1000, 2000,- Each with 5x106 slots,- UV*=(5V, 4V, 3V, 2V, V)T.
Sample (W1(t),W2(t)) process:V=1000, t=10000:110000
Note: W1(t)>Q1=4952 & W2(t)>Q2=3952
30
Simulation
U1 U2
R(t)
S1(t) S2(t) S3(t) S4(t) S5(t)
U3 U4 U5
Backlog % of pkt dropped
Quick comparisonV=1000, U QLA≈15V=15000U FQLA≈5log2(V)=25060 times better!
Simulation parameters: - V= 50, 100, 200, 500, 1000, 2000,- Each with 5x106 slots,- UV*=(5V, 4V, 3V, 2V, V)T.
Summary
31
1. Under QLA, the backlog vector usually stays close to an “attractor” – the optimal Lagrange multiplier UV*.
2. FQLA subtracts out the Lagrange multiplier from the system induced by QLA by using place-holder bits to reduce delay.
Summary
Note: (1) Theorem 1 also holds when S(t) is Markovian, (2) FQLA-General for the case where UV* is not known, performance similar to FQLA-Ideal, (3) when q0(U) is “smooth”, we prove O(sqrt{V}) deviation bound, (4) The “Network Gravity” role of Lagrange multiplier.Details see ArXiv report 0904.3795
32
1. Under QLA, the backlog vector usually stays close to an “attractor” – the optimal Lagrange multiplier UV*.
2. FQLA subtracts out the Lagrange multiplier from the system induced by QLA by using place-holder bits to reduce delay.
Thank you !
Questions or Comments?
33