Upload
avi-latner
View
191
Download
1
Embed Size (px)
Citation preview
Attention Allocation to Partially Attention Allocation to Partially Observable Heterogeneous Observable Heterogeneous Customers – with Imperfect Customers – with Imperfect
TreatmentTreatment
Avi LatnerAvi LatnerAdvisor: Prof. Gadi RabinowitzAdvisor: Prof. Gadi Rabinowitz
Possible model implementations:
• Machine maintenance• Allocation of medical staff
(NICU)• Law enforcement response
Resource allocation dilemmaResource allocation dilemma
server server server
stable unstableunstablei=1 i=5
unstablei=3 i=4
stablei=2
A single customer state transition diagramA single customer state transition diagram
stable
unstable
failuretreatmentαi
βi
γi*pi
γi*(1-pi)
∞
decision to treat
decision to
treat
αi – deterioration rateβi – “death” rateγi – treatment rate
Server Server Server
Who to Treat?
Actual state:
unstablei=1
stablei=5
unstablei=3
stablei=4
stablei=2
Perceived state:
unstablei=5
unstablei=1
unstablei=2
stablei=3
stablei=4
unstablei=3
stablei=5
Partial ObservabilityPartial Observability
perceived: stable
perceived: unstable
stab
leun
stab
le
stable stable
failure
treatment
unstable treatmentunstable
perceived: stable
perceived: unstable
stab
leun
stab
le
stable
failure
treatment
unstable treatment
stable
unstable
γi*pi
γi*(1-pi)
θi*(1-qi)
γi*qi
∞
αiαi
βi βi
decision to treat
decision to treat
decision to treat
decision to treat
φi
νi
ωi
ρi
Model dimensionsModel dimensions
customer server×2 ×1
customer server×5 ×2
customer server×20 ×4
8
192
≈5*108
Ω(2n)-O(3n)
Solution method: Solution method: the relaxed modelthe relaxed model
Semi-Markov Decision Process
Continuous Time
Markov Chain
who's next?
Service Rates:
)δi , ψi (i=1…n
The relaxed model:The relaxed model:Continuous Time Markov ChainContinuous Time Markov Chain
• Instead of the decision to treat a treatment rate is set for each customer:δi – treatment rate for “unstable”i – treatment rate for “stable”
• Treatment rate is exponentially distributed
• Full server utilization• The problem is formulized as a nonlinear
planning problem
Steady State EquationsSteady State Equationsiiiiiiiiiiiiiii pq 654321 )()(
iiiiiiii 324 )(
iiiiii 12 )(
)1()1()( 65413 iiiiiiiiiiiii pq
iiiiii 215
iiiiii 436
16
1
jij
And… steady state probabilitiesAnd… steady state probabilities1. π1 = -((γ2 θ2 (α + δ + ν) ω (β2 + β (δ + ρ + p ψ + ω) + p (ρ ψ + δ (ψ + ω)))) / EXP2); (8)
2. π2 = -((γ2 θ2 φ ω (β2+ β (δ + ρ + p ψ + ω) + p (ρ ψ + δ (ψ + ω)))) / EXP2;3. π3 = (γ θ (α2 (β + δ + ρ) - (-1 + q) (β + δ + ρ) (ν ψ + δ (φ + ψ)) + α (δ2 + β (δ + ν + ψ - q ψ) + ρ (ν + φ + ψ - q ψ) + δ (ν + ρ + φ - p φ + ψ - q ψ)))) / EXP1; 4. π4 = -
(1
*((γ θ ω (-1 - (γ2 θ (α + δ + ν) (θ + ψ) ω (β2 + β (δ + ρ + p ψ + ω) + p (ρ ψ + δ (ψ + ω)))) / EXP2 + (γ θ φ (α θ (γ + ψ) - γ (δ + θ) ω) (β2 + β (δ + ρ + p ψ + ω) + p (ρ ψ + δ (ψ + ω)))) / EXP2));5. π 5 = (γ2θ (-(δ φ) - (α + δ + ν) ψ) ω (β2 + β (δ + ρ + p ψ + ω) + p (ρ ψ + δ (ψ + ω)))) / EXP2;6. π6 = (θ (α2 (β ψ + ρ ψ + δ (ψ + ω)) - (-1 + q) (ν ψ + δ (φ + ψ)) (β ψ + ρ ψ + δ (ψ + ω)) + α (β (δ (φ + ψ) + ψ (ν + ψ - q ψ)) + (δ + ν + φ + ψ - q ψ) (ρ ψ + δ (ψ + ω)))) / EXP1;Where:EXP1 = (γ δ2 θ φ - q γ δ2 θ φ + γ δ θ ρ φ - q γ δ θ ρ φ + γ δ2 θ ψ + p γ δ2 θ ψ - q γ δ2 θ ψ + γ δ θ ν ψ + p γ δ θ ν ψ - q γ δ θ ν ψ + γ δ θ ρ ψ + p γ δ θ ρ ψ - q γ δ θ ρ ψ + γ θ ν ρ ψ + p γ θ ν ρ ψ - q γ θ ν ρ ψ + p γ δ2 φ ψ + p γ δ θ φ ψ + δ2 θ φ ψ - q δ2 θ φ ψ + p γ δ ρ φ ψ + p γ θ ρ φ ψ + δ θ ρ φ ψ - q δ θ ρ φ ψ + p γ δ2 ψ2 + δ2 θ ψ2 - q δ2 θ ψ2 + p γ δ ν ψ2 + δ θ ν ψ2 - q δ θ ν ψ2 + p γ δ ρ ψ2 + δ θ ρ ψ2 - q δ θ ρ ψ2 + p γ ν ρ ψ2 + θ ν ρ ψ2 - q θ ν ρ ψ2 + β2 γ (θ (ν + φ) + ν ψ + δ (θ + φ + ψ)) + p γ δ2 θ ω + p γ δ θ ν ω + p γ δ2 φ ω + γ δ θ φ ω + p γ δ θ φ ω - q γ δ θ φ ω + δ2 θ φ ω - q δ2 θ φ ω + p γ δ2 ψ ω + γ δ θ ψ ω - q γ δ θ ψ ω + δ2 θ ψ ω - q δ2 θ ψ ω + p γ δ ν ψ ω + γ θ ν ψ ω - q γ θ ν ψ ω + δ θ ν ψ ω - q δ θ ν ψ ω + α2 θ (δ ψ + ρ ψ + β (γ + ψ) + δ ω + γ (δ + ρ + ω)) + β ((1 - q) θ ψ (ν ψ + δ (φ + ψ)) + γ (δ2 (θ + φ + ψ) + ν ψ (ρ + p ψ + ω) + θ (φ (ρ + p ψ + ω) + ν (ρ + ψ + p ψ - q ψ + ω)) + δ (ν ψ + p φ ψ + p ψ2 + ρ (φ + ψ) + φ ω + ψ ω + θ (ν + ρ + 2 φ - q φ + ψ + p ψ - q ψ + ω)))) + α (β2 γ (θ + ψ) + θ (δ + ν + φ + ψ - q ψ) (ρ ψ + δ (ψ + ω)) + β (θ (δ (φ + ψ) + ψ (ν + ψ - q ψ)) + γ (δ (2 θ + ψ) + ψ (ρ + p ψ + ω) + θ (ν + ρ + φ + ψ + p ψ - q ψ + ω))) + γ (δ2 θ + p ρ ψ2 + θ (p φ ψ + ρ (φ + ψ + p ψ - q ψ) + φ ω + ψ ω - q ψ ω + ν (ρ + ω)) + δ (p ψ (ψ + ω) + θ (ν + ρ + φ - p φ + ψ + p ψ - q ψ + ω + p ω))))); andEXP2 = ((-α - δ - ν) ((q γ θ ψ - γ θ (α + φ + ψ)) ω (-(θ (β + δ + ρ) (γ + ψ)) - (γ + δ) θ ω) + γ2 θ (θ + ψ) ω (β2 + β (δ + ρ + p ψ + ω) + p (ρ ψ + δ (ψ + ω)))) - γ θ φ ((-(θ (β + δ + ρ) (γ + ψ)) - (γ + δ) θ ω) (-(α (β + p ψ)) + (q δ + ν) ω) - (α θ (γ + ψ) - γ (δ + θ) ω) (β2 + β (δ + ρ + p ψ + ω) + p (ρ ψ + δ (ψ + ω))));
perceived: stable
perceived: unstable
stable
failure
treatment
unstable treatment
stable
unstable decision to treat
decision to treat
decision
to treat
decis
ion
to tre
at
δi
δii
i
perceived: stable
perceived: unstable
stable treatment
treatment
stable
failure
unstable
Πi4
unstable
Πi3
βi βi
)],(),([, 43 iiiiiiiiiih
perceived: stable
perceived: unstable
failure
unstable
Πi4
stableΠi1
treatment
treatment
stable
Πi2
unstable
Πi3
i
iiii
i
iiiiiiig
)()(),( 4321
δi
i
i
δi
Nonlinear programming problemNonlinear programming problem
),(),(1
i
n
iihhMin
nini
Kgg
TS
i
i
i
n
ii
1010
),(),(
..
1
Solving the nonlinear problemSolving the nonlinear problem• Optimum point is found by Karush Kuhn Tucker
conditions• How much will a small addition of resource to
customer i decrease the failure rate:
• Distribute resources by the rule “who would contribute the most” until all the capacity is used
),(), ii
i
iii
i
i
g
hOR(g
h
-350
-300
-250
-200
-150
-100
-50
0
1
2
3
4
5
-350
-300
-250
-200
-150
-100
-50
0
Solving the nonlinear problem – Cont.Solving the nonlinear problem – Cont.
Solving the nonlinear problem – Cont.Solving the nonlinear problem – Cont.
δi\ii=00<i∞<i∞→
δi=0GROUP A
Immediatesolution
GROUP BOnedimensionalquery
-----
0<δi∞<GROUP C
Onedimensionalquery
GROUP DTwodimensionalquery
GROUP EOnedimensionalquery
δi∞→-----GROUP F
Onedimensionalquery
GROUP GImmediatesolution
Treatment rate for a “stable” customer
Trea
tmen
t rat
e fo
r an
“uns
tabl
e” c
usto
mer
-350
-300
-250
-200
-150
-100
-50
0
1
2
3
4
5
-350
-300
-250
-200
-150
-100
-50
0
GROUP G
GROUP G
GROUP C GROUP C
GROUP DGROUP D
GROUP AGROUP A
λ
Solving the nonlinear problem – Cont.Solving the nonlinear problem – Cont.
2 31 5 64
Sever
δ=0=30δ=240 δ=∞ =∞ δ∞=δ=110=0
FIFO By treatment rate
“Stable”
Unstable”"
Legend:
Service Priority PolicyService Priority Policy
2 31 5 64
שרת
δ∞=∞=δ∞= δ=240 =30δ=110ψ=50 δ∞=∞= δ=240 =30∞=
“Stable”
Unstable”"
Legend:
Service Priority PolicyService Priority Policy
Possible model implementations:
• Machine maintenance• Allocation of medical staff
(NICU)• Law enforcement response