View
218
Download
3
Embed Size (px)
Citation preview
1
A Generic Mean Field Convergence Result for Systems of Interacting Objects
From Micro to Macro
Jean-Yves Le Boudec, EPFL
Joint work with David McDonald, U. of Ottawaand Jochen Mundinger, EPFL
2
The full text of my talk is available in the proceedings of QEST 2007
The paper and this slide show are
also available from my web page
3
Contents
E.L.
1. Motivation
2. A Generic Model for a System of Interacting
Objects
3. Convergence to the Mean Field
4. Fast Simulation
5. Full Scale Example: A Reputation System
6. Outlook
4
Motivation
Find re-usable approximations of large scale systems
Examples from my fieldPerformance of UWB impulse radio : many sensors, each has a MAC layer stateAd-Hoc networkingReputation Systems
From microscopic description to macroscopic equations
Understand fluid approximation and mean field approximation
5
Example 1 : TCP/ECN
TCP connection n transmits at a rate 2 {s0, …, si, …, sI}
Queue length at router is R(t) With probability q(R(t))
connection i receives an Explicit Congestion Notification (ECN) in next time slot
When connection n does not receive an ECN, it increases its rate:
If rate == si, new rate := si+1 (i<I)
Else it decreases its rate:If rate == si, new rate := sd(i)
ECN router
queue length R(t)
ECN Feedback q(R(t))
N connections
1
n
N
The question is the behaviour when N is large
6
Microscopic Description Time is discrete Connection n runs one Markov chain XN
n(t);
The transition probabilities of the Markov chain XNn(t) depend on
global state R(t) (queue size)
Global state R(t) depends on states of all connectionslet MN
i(t) = nb of connections in state i at time t , C = service rate of
router
ECN received
no ECN received
7
Macroscopic Description The fluid approximation is often given as a simplification of the
previous model
Combined with
we have a macroscopic description of the system
In [17], Tinna. and Makowski show that it holds as large N asymptotics
8
The Mean Field Approximation Assume we want to analyze one TCP connection in detail We can keep the microscopic description for this TCP
connection, and use the fluid approximation for the others: We can call it fast simulation.
i.e. pretend XN1(t) (one connection) and R(t) (global resource) are
independent. This is similar to what is called the mean field approximation in physics
9
Another Example: Robot Swarm
N robots Robot has S = 2 possible states Transition for one robot depends
on this robot’s state + how many other robots are in search state
[11] uses the fluid approximation :
10
A few other Examples …
11
In these and other examples, some authors assume the validity of the fluid / mean field approximation and use the approximation to do performance evaluation, parameter identification, control… Never
again !
… while, in contrast, others spend most of the paper proving the derivation and validity of the approximations in their specific setting
papers in this latter class are intimidatingcost of proof of one approximation result ¼ 1 PhDand not re-usable
Proof of convergence to
Mean field
for TCP/ECN
12
Can we have answers of general applicability to:
When are the fluid approximation and the mean field approximation valid ?
Can we write them in a sound ( = mechanical) way ?
13
Contents
E.L.
1. Motivation
2. A Generic Model for a System of Interacting
Objects
3. Convergence to the Mean Field
4. Fast Simulation
5. Full Scale Example: A Reputation System
6. Outlook
14
Mean Field Interaction Model
A Generic Model, with generic results Does not cover all useful cases, but is a useful first step
Time is discrete N objects Every object has a state in .
Informally: object n evolves depending only onIts own stateA global resource whose evolution depends only on how many other objects are in each state
15
XNn(t) : state of object n at time t
MNi(t) = proportion of objects that are in state i MN is the “occupancy measure” ¼ the “mean field”
RN(t) = global resource =“history” of occupancy measure
Conditional to history up to time t, objects draws next state independent of each other according to
Model Assumptions
16
Two Mild Assumptions
1. Continuity of the integration function g()
2. For large N, the transition matrix K becomes independent of N and is continuous
17
TCP/ECN Example fits in this Framework
Intuitively satisfies the conditionsState of one connection depends only on buffer contentBuffer contents depends only on how many connections are in each state
Formally:One object = one TCP connectionState of one object = index i of sending rateRN(t) = total buffer occupancy / N
Function g() :
thus
g() is continuousAssumption 1 is satisfied
ECN router
queue length R(t)
ECN Feedback q(R(t))
N connections
1
n
N
18
TCP/ECN Example fits in this Framework
Transition matrix K
Let q(r) = proba of negative feedback when R==r
K is independent of N thus Assumption 2 is is satisfied if q() is continuous
ECN router
queue length R(t)
ECN Feedback q(R(t))
N connections
1
n
N
ECN received
no ECN received
19
A Multiclass Variant
Take same as previous TCP/ECN model but introduce multiclass
Aggressive connections, normal connection
State of an object = (c, i)c : classi : sending rate
Objects may change class or not
Also fits in our framework
Mean Field does not mean all objects are exchangeable !
20
Contents
E.L.
1. Motivation
2. A Generic Model for a System of Interacting
Objects
3. Convergence to the Mean Field
4. Fast Simulation
5. Full Scale Example: A Reputation System
6. Outlook
21
22
Practical Application : Derivation of the Fluid Approximation
The theorem replaces the stochastic system by a deterministic, dynamical system
This gives a method to write and justify the fluid approximation in the large N regime
Equation for the limiting occupancy measure can be rewritten as
where Ni(t) = N MNi(t) = number of objects in state i at time t
This recovers for example the result in [17]
23
Proof of Theorem
Based on The next theorem (fast simulation)A coupling argumentAn ad-hoc version of the strong law of large numbersThe Glivenko Cantelli lemma
24
Contents
E.L.
1. Motivation
2. A Generic Model for a System of Interacting
Objects
3. Convergence to the Mean Field
4. Fast Simulation
5. Full Scale Example: A Reputation System
6. Outlook
25
Fast Simulation / Analysis of One Object
Assume we are interested in one object in particularE.g. distribution of time until a TCP connection reaches maximum rate
For large N, since mean field convergence holds, one may do the mean field approximation and replace the set of other objects by the deterministic dynamical system
The next theorem says that, essentially, this is valid
26
Fast Simulation Algorithm
Returns next state for one objectWhen transition matrix is K
State of one specific object
This is the mean field independenceapproximation
Replace true value by deterministiclimit
27
Fast Simulation Result
28
Practical Application
This justifies the mean field approximation for the stochastic evolution of one object in the large N regime
Gives a method for fast simulation or analysisThe state space for Y1 has S states, instead of SN
29
Contents
E.L.
1. Motivation
2. A Generic Model for a System of Interacting
Objects
3. Convergence to the Mean Field
4. Fast Simulation
5. Full Scale Example: A Reputation System
6. Outlook
30
A Reputation System
My original motivation for this work Illustrates the complete set of steps, including a few modelling
tricks System
N objects = N peersPeers observe one subject and rate itRating is a number in (0,1)Direct observations and spreading of reputationConfirmation bias + forgetting
31
Operation of Reputation System: Forgetting
Zn(t) = reputation rating held by peer n
During a direct observation, subject is perceived as positive (with proba ) or negative (with proba 1-)
In case of direct positive observation
In case of direct negative observation
w is the forgetting factor, close to 1 (0.9 in next slides)
32
Confirmation Bias
Peer also read other peer ratings If overheard rating is z:
is the threshold of the confirmation bias
33
Liars and Honest Peers
Honest peer does as just explained Liar tries to bring the reputation down
Uses different strategies, see later
34
Initially: peers have Z=0, 0.5 or 1
= 0.9
Every time step: direct obs p=0.01, meet liar proba 0.30, meet honest proba 0.69
Example of exact simulation: N=100 peerswith maximal liars (always say Z=0)
ratingprop
orti
on o
f pe
ers
35
3 particular peers, one of each type
= 0.9
time
rating
36
Can we study the system with 106 users instead of 100 ?
37
The problem fits in our framework…
Assume discrete time At every time step a peer
Makes a direct observationOr overhears a liarOr overhears some honest peerOr does nothing
Object = honest peer
Assume first that liars use strategy 1: maximal lying (always say Z=0)
Transition of one honest peer depends onOwn stateDistribution of states of all other peers
=> Fits in our framework with memory R = occupancy measure M
38
Different Liar Strategies
Strategy 1 (maximal lying): liars always say Z= 0 Strategy 2 (infer): liar guesses your rating based on past
experienceTransition of one honest peer depends on
Own stateDistribution of states of all other peersWhat liars remember seeing in the past
=> Fits in our framework with memory R = occupancy measure of ratings at steps t and t-1
Strategy 3 (side information): liars know your rating and is as negative as you acceptnot realistic but serves as benchmark (worst case)
Similar to strategy 1, memory = occupancy measure M
39
We would like to apply the mean field convergence result to analyze very large N
But model has continuous state space
Discretize reputation ratings !Quantize Zn on ca. L bits; replace Zn by Xn = 2L ZN with
Issue: small increments due to “forgetting” coefficient w (e.g. w = 0.9) are set to 0
Solution: use random rounding; replace previous equation by
where RANDROUND(2.7) = 2 with proba 0.3 and 3 with proba 0.7 E(RANDROUND(x)) = x
40
Transition Matrix K
The transition matrix KN is straightforward but tedious to describe.
Unlike in the TCP/ECN example, it does depend on N
It contains terms such as : the proba that an indirect observation with a honest peer is with someone who has rating equal to k. This proba is equal to
It depends on N, but for large N it converges uniformly to MNk(t),
with no term in N
The limiting matrix K is polynomial in MNk(t), thus continuous,
thus assumption 2 is satisfied
Assumption 1 is trivially satisfied, by inspection
41
Therefore we can apply the theorem and derive the fluid approximation and the mean field approximation
Both are true in the limit N = 1
42
Limiting reputation ratings: 0.9 and 0.1
Discrete event simulation, N = 100 Fluid Approximation
Fast Simulation based on Mean Field Approximation
43
Fluid approximationCan be written using Theorem 4.1 Is a deterministic recurrence with state vector the memory number of dimensions is 2 L+1, where L = number of quantization bits for reputation values (e.g. L=8)
Mean Field Approximation = Fast SimulationSimulation of one Markov chain on state space with 2 L states, with time varying transition probability
44
Different Parameters (few liars)
Few liarsFinal ratings converge to true value
Phase transition
45
Different Initial Conditions
46
Liar Strategy 2(infer)
Liar Strategy 3(side information)
Peers starting after 512 time units
47
Modelling Locality with Multiclass Model
We can model spatial aspectsObject = honest peer ; state = (c, x) with
C = location (in a discrete set of locations)X = rating (same as before)
This allows to account for locality of interaction
48
Contents
E.L.
1. Motivation
2. A Generic Model for a System of Interacting
Objects
3. Convergence to the Mean Field
4. Fast Simulation
5. Full Scale Example: A Reputation System
6. Outlook
49
I have shown how a mean field convergence result can be used to write and validate
the fluid approximation = macroscopic descriptionthe mean field approximation = fast simulation (or analysis)
Applies to cases where objects interact such thatTransition depends on state of this object + current and past distribution of states of all other objectsNumber of objects is large compared to number of states of one object
Extensionsbirth and death of objects transitions that affect several objects simultaneouslygaussian approximations (central limit theorems)
Outlook
50
… thank you for your attention
E. L.