Stochastic modelling of urban structurehelper.ipam.ucla.edu/publications/elws4/elws4_14284.pdf · Stochastic modelling of urban structure Louis Ellam Department of Mathematics,

Stochastic modelling of urban structure

Louis Ellam

Department of Mathematics, Imperial College LondonThe Alan Turing Institute

https://iconicmath.org/

IPAM, UCLAUncertainty quantification for stochastic systems and applications

16-11-2017

Louis Ellam (Imperial, ATI) Urban Structure 03-07-2017 1 / 48


Louis Ellam Mark Girolami Greg Pavliotis Sir Alan WilsonICL & Turing ICL & Turing ICL UCL & Turing

Stochastic Modelling of Urban Structure, L. Ellam et al, Submitted (2017)Louis Ellam (Imperial, ATI) Urban Structure 03-07-2017 2 / 48

Overview

1 Motivation

2 Modelling urban structure (forward problem)

3 Inferring the utility function (inverse problem)

4 Conclusions and outlook


Motivation


Problem statement

What are the workings of cities and regions, and how will theyevolve over time?


Problem statement

Over half of the world’s population live in a city.

We should be interested in:

What is happening in the city;How the city is evolving; andHow can we enable a better quality of life.

Applications: planning, policy and decision making (e.g. retail, health, crime, education,transport etc.).

Requires an understanding of the underlying mechanisms and behaviours.

On going task of matching socio-economic theories with empirical evidence.

This talk: Improving insights into urban and regional systems with the development ofdata-driven mathematical models.


What can mathematics and statistics offer?

Mathematical models can represent socio-economic theories.

Can help explain the behaviour of complex systems.

Simulations may provide insights:

‘Flight simulators’ for urban and regional planners.‘What if’ forecasting capabilities.How does the system respond to change/initiatives?Which parameters/mechanisms are most important?What is the long term behaviour?

Mathematical modelling long history (> 50 years) in urban and regional analysis

Equilibrium values and dynamics of attractiveness terms in production-constrainedspatial-interaction models (Harris and A. Wilson, 1978).


Data-driven methodologies

Data sets are becoming routinely available:

Social media,Location tracking,Travel ticketing,Census data,Ad-hoc reports etc.

The range of statistical models available is arguably somewhat limited.

Ignoring either data or mathematical models seems unwise.

A structured approach that’s consistent with the available data is desirable.

Build upon well-established mathematical formalisms with this new found data (ratherthan throwing the baby out with the bath water).


Mathematical and statistical challenges

“All models are wrong; some models are useful.”

Urban and regional systems are complex in nature:Phase transitions; andMultimodality and rare events.

An emergent behaviour arises from the actions of many interacting individuals.

Seamless integration of mathematical models with data.

Quality of model vs. quality of data?

Uncertainty should be addressed in the modelling process:Model uncertainty and system fluctuations;Parameter uncertainty; andObservation error and bias.


Example: London retail system

Figure: London retail structure for 2008 (left) and 2012 (right). The locations of retail zones andresidential zones are red and blue, respectively. Sizes are in proportion to floorspace and spendingpower, respectively. N = 625 and M = 201.


Components of an urban and regional system

Three key features:Activities at location;Flows between locations (spatial interaction); andStructures that facilitate flows.

Broad range of applications:Retail: town centres, shopping centers, distribution centres;Health: surgeries, hospitals;Education: schools, colleges, universities;Transport: airports, car parks;Population: residence.

Spatial interaction of activities can be modelled with statistical averaging procedures(Boltzmann).

Evolution of structures relating to the activities can be modelled using differentialequations (Lotka-Volterra).

New approach: Expand on BLV models with Overdamped Langevin dynamics.Louis Ellam (Imperial, ATI) Urban Structure 03-07-2017 11 / 48

Modelling urban structure (forward problem)


Urban and regional systems

Urban structure:N origin zones and M destination zones.Origin quantities {Oi}Ni=1.Destination quantities {Dj}Mj=1.

Spatial interaction:Flows of activities at location are denoted {Tij}N,M

i,j , where Tij is the flow from zone i to j .


Modelling flows

Flows from origin zones:

Oi =M∑j=1

Tij , i = 1, . . . ,N.

Flows to destination zones (demand function):

Dj =N∑i=1

Tij , j = 1, . . . ,M.

Nomenclature: A singly-constrained system since {Oi}Ni=1 is fixed and {Dj}Mj=1 isundetermined.


Modelling flows

Fixed total benefit (or capacity):N∑i=1

M∑j=1

Tijxj = X ,

where xj is the attractiveness, of zone j .

Fixed total transport cost:N∑i=1

M∑j=1

Tijcij = C ,

where cij is the cost of transporting a unit from zone i to zone j .


A production-constrained spatial interaction model

The destination flows are obtained by maximizing an entropy function.

Dj =N∑i=1

Oiexp

(αxj − βcij

)∑Mk=1 exp

(αxk − βcik

) .xj is the attractiveness of j .

cij is the inconvenience of transporting from zone i to j .

α is the attractiveness scaling parameter.

β is the cost scaling parameter.

αxj − βcij is the net utility from transporting from zone i to j .

A statistical theory of spatial distribution models, A. Wilson, Transportation research (1967)Louis Ellam (Imperial, ATI) Urban Structure 03-07-2017 16 / 48

Equilibrium structures

Urban structure is a vector of sizes: w = {w1, . . . ,wN} ∈ RM>0.

The evolution of urban structure can be modelled by the Lotka-Volterra equations

dwj

dt= ε(Dj − κwj

)wj , j = 1, . . . ,M,

for some initial condition w(0) = w .

Equilibrium condition

N∑i=1

Oiexp

(α lnwj − βcij

)∑Mk=1 exp

(α lnwk − βcik

) = κwj , j = 1, . . . ,M.

Equilibrium values and dynamics of attractiveness terms in production-constrained spatial-interactionmodels. B. Harris A. Wilson, Environment and Planning (1978)


Stochastic modelling of urban structure

Work in terms of attractiveness x = {x1, . . . , xM} ∈ RM with wj(xj) = exp(xj).

The evolution of urban structure can be modelled by overdamped Langevin dynamics:

dx = −∇V (x)dt +√

2γ−1dB,

for some initial condition x(0) = x and standard M-dimensional Brownian motion B.

Equilibrium distribution

Urban structure is realization of the Boltzmann distribution

π(x) =1

Zexp

(− γV (x)

), Z :=

∫RM

exp(− γV (x)

)dx ,

specified by the potential function V : RM → R and ‘inverse-temperature’ γ > 0.

Stochastic Modelling of Urban Structure, L. Ellam et al, Submitted (2017)Louis Ellam (Imperial, ATI) Urban Structure 03-07-2017 18 / 48

Assumptions for the potential function

Interpretation of gradient structure (in terms of net supply/demand Πj):

−∂jV (x) = εΠj , j = 1, . . . ,M.

V (x) = VUtility(x)︸︷︷︸Demand

+VCost(x) + VAdditional(x)︸︷︷︸Supply

.

1 The potential function V ∈ C 2(RM ,R) is confining in that lim‖x‖→+∞ V (x) = +∞, and

e−γV (x) ∈ L1(RM), ∀ γ > 0.

2 There exists 0 < d < 1 such that

lim inf‖x‖→∞

(1− d)‖V (x)‖2 −∇2V (x) > 0.


Utility potential

For a singly-constrained system we have that:

−∂jVUtility(x) = εDj = ε

N∑i=1

vij(x)Oi ,

M∑j=1

vij(x) ≡ 1.

Can express weights vij in terms of utility functions uij :

−∂jVUtility(x) = ε

N∑i=1

ϕ(uij)∑Mk=1 ϕ(uik)

Oi . (1)

By inspection, we look for a potential function of the form

VUtility(x) = −εN∑i=1

Oi

{fi (x) ln

M∑j=1

ϕ(uij(x)

)}. (2)


Utility potential

Inserting Eq. (2) into Eq. (1), we obtain the requirement:

ϕ(uij(x)

)∑Mk=1 ϕ

(uik(x)

) =dfi (x)

dxjln

M∑j=1

ϕ(uij(x)

)+ fi (x)

dϕ(uik(x)

)dxj

( M∑k=1

ϕ(uik(x)

))−1

.

(3)

Eq. (3) is satisfied for ϕ(x) = exp(x),

uij(x) = αixj + βij ,

provided that each αi 6= 0 and that fi = α−1i .

We obtain:

VInflow(x) = −εN∑i=1

α−1i Oi ln

M∑j=1

exp(uij(x)

).


Cost and additional potentials

Cost potential (linear cost):

− ∂jVCost(x) = −εκwj(xj), j = 1, . . . ,M,

VCost(x) = εκM∑j=1

wj(xj).

Additional potential (constant support):

− ∂jVAdditional(x) = −εδj , j = 1, . . . ,M,

VAdditional(x) = ε

M∑j=1

δjxj .


Resulting Boltzmann distribution

π(x) =1

Zexp

(− γV (x)

), Z :=

∫RM

exp(− γV (x)

)dx ,

ε−1V (x) = −N∑i=1

α−1i Oi ln

M∑j=1

exp(uij(xj)

)︸︷︷︸

Utility (Spatial Interaction)

+κM∑j=1

wj(xj)︸︷︷︸Cost

−M∑j=1

δjxj︸︷︷︸Additional

.

uij(xj) = αxj − βcij is the net utility for a flow from zone i to j .

Potential assumptions satisfied.


Alternative derivation: random utility maximization

Individuals make choices to maximize their (random) utility:

Uij = uij + εij , εij ∼ Gumbel(0, 1).

The probability that individual from zone i makes choice j is:

Pr[Yij = 1

]= Pr

[∩j 6=k {Uij > Uik}

]=

exp(uij)∑Mk=1 exp(uij)

.

The expectation for all individuals gives the same demand flows as before:

Dj =N∑i=1

OiPr[Yij = 1

]=

N∑i=1

Oiexp

(αxj − βcij)

)∑Mk=1 exp

(αxj − βcik)

) .The utility potential is the (unscaled) expected welfare:

ε−1VUtility(x) = α−1N∑i=1

OiE[maxj

Uij ] + const = −α−1N∑i=1

Oi lnM∑j=1

exp(uij(xj)

)+ const.


Alternative derivation: maximum entropy method

Total welfare (surplus):

Eπ

[α−1

N∑i=1

Oi lnM∑j=1

exp(uij(xj)

)]= S ,

Total size (capacity):

Eπ[ M∑

j=1

wj(xj)

]= W ,

Expected attractiveness (benefit)

Eπ[ M∑

j=1

xj

]= X .

Then the Boltzmann distribution π(x) = Z−1 exp(− γV (x)

)is the maximum entropy

distribution (maximal uncertainty) subject to these constraints.


Sampling challenges

High-dimensionality (use gradient information)

Multi-modality (use a tempering approach)

Anisotropy (precondition)

α = 1.8, β = 1, δ = 0.5 α = 1.8, β = 0, δ = 0.1 α = 1.1, β = 10, δ = 0.2


Parallel tempering and Hamiltonian Monte Carlo

Joint probability distribution:

πpt(x1, . . . , xT ) =T∏t=1

πt(xt), πt(xt) = Z−1t exp(−γtV (x)), γ1 = γ.

HMC move steps Draw momentum pt ∼ N(0, 1), define Ht(xt , pt) = − lnπt(xt) + 12p

2t ,

obtain x ′t and p′t by simulating Hamiltonian dynamics with a volume preservingtime-integrator (e.g. leapfrog) and accept with probability

min

{1,H(xt , pt)− H(x ′t , p

′t)

}.

Exchange steps Propose neighbouring swaps xt 7→ xt+1, xt+1 7→ xt and accept withprobability

min

{1,πt(xt+1)πt+1(xt)

πt(xt)πt+1(xt+1)

}.


Airports in England

α = 0.5

α = 1.0

α = 2.0

Figure: Approximate draws from p(x |θ) using HMC combined with parallel tempering.


London Retail system

α = 0.5

α = 1.0

α = 2.0

Figure: Approximate draws from p(x |θ) using HMC combined with parallel tempering.


Overdamped Langevin dynamics

With the specification of V (X ), overdamped Langevin dynamics give the Harris andWilson model, plus multiplicative noise.

SDE Urban Retail Model

Floorspace dynamics is a stochastic process that satisfies the following Stratonovich SDE.

dWj

dt= εWj

(Dj − κWj + δj

)+ σWj ◦

dBj

dt,

where(B1, . . . ,BM

)Tis standard M-dimensional Brownian motion and σ =

√2/εγ > 0 is the

volatility parameter.

Fluctuations (missing dynamics) are modelled as Stratonovich noise.

The extra parameter δ to represents local economic stimulus to prevent zones fromcollapsing (needed for stability).

The Markov process converges exponentially fast to π(x) = Z−1 exp(−γV (x)).


Inferring the utility function (inverse problem)


A Bayesian model of urban structure

Inverse problem

Given observation data Y = (Y1, . . . ,YM), of log-sizes, infer the parameter valuesθ = (α, β)T ∈ R2

+ and latent variables X ∈ RM with low-order summary statistics:

EX ,θ|Y [h(X , θ)] =

∫h(X , θ)π(X , θ|Y )dXdθ.

Assumption (Data generating process)

Assume that the observations Y1, . . . ,YM are i.i.d. realizations of the following hierarchicalmodel:

Y1, . . . ,YM |X , σ ∼ N (X , σ2I ),

X |θ ∼ π(X |θ) ∝ exp(−U(X ; θ)),

θ ∼ π(θ).


Posterior distribution

Bayes’ rule

The joint posterior is given by

π(X , θ|Y ) =π(Y |X , θ)π(X , θ)

π(Y ), π(Y ) =

∫π(Y |X , θ)π(X , θ)dXdθ.

We have a hierarchical prior given by

π(X , θ) =π(θ) exp(−U(X ; θ))

Z (θ), Z (θ) =

∫exp(−U(X ; θ))dX .

The joint posterior is doubly-intractable

π(X , θ|Y ) =π(Y |X , θ) exp(−U(X ; θ))π(θ)

π(Y )Z (θ).


Russian roulette

On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods. AM Lyne, MGirolami, Y Atchade, H Strathmann, D Simpson. Statistical Science (2013)


Metropolis-within-Gibbs with block updates

We are interested in low-order summary statistics of the form

EX ,θ|Y [h(X , θ)] =

∫h(X , θ)π(X , θ|Y )dXdθ.

X and θ are highly coupled, so we use Metropolis-within-Gibbs with block updates.

X -update. Propose X ′ ∼ QX and accept with probability

min

{1,π(Y |X ′, θ) exp(−U(X ′; θ)) q(X |X ′)π(Y |X , θ) exp(−U(X ; θ)) q(X ′|X )

}.

θ-update. Propose θ′ ∼ Qθ and accept with probability

min

{1,π(Y |X , θ′)Z (θ) exp(−U(X ; θ′)) q(θ|θ′)π(Y |X , θ)Z (θ′) exp(−U(X ; θ)) q(θ′|θ)

}.

Unfortunately the ratio Z (θ)/Z (θ′) ratio is intractable!


Unbiased estimates of the normalizing constant

We can use the Pseudo-Marginal MCMC framework if we have an unbiased, positiveestimate of π(X |θ), denoted π(X |θ, u), satisfying

π(X |θ) =

∫π(X |θ, u)π(u|θ)du.

The Russian Roulette methodology gives an unbiased estimate of 1/Z :

S = V(0) +N∑i=1

V(i) − V(i−1)

Pr(N ≥ i),

provided that limi→∞ E[V(i)] = 1/Z .

Requires N estimates of 1/Z using path sampling e.g. annealed importance sampling orthermodynamic integration, for a random stopping time N.


Unbiased estimates of the normalizing constant

The Forward Coupling estimator (FCE) gives an unbiased estimate of 1/Z with lowervariance

The idea is to find two sequences of consistent estimators {V(i)}, {V(i)}, each with thesame distribution, such that V(i) and V(i−1) are“coupled”.

Coupling between V(i) and V(i−1) is introduced with a Markov chain that shares randomnumbers.

Then the unbiased estimate is given by

S := V(0) +N∑i=1

V(i) − V(i−1)

Pr(N ≥ i).

Markov Chain Truncation for Doubly-Intractable Inference, C. Wei, and I. Murray, AISTATS (2016)Louis Ellam (Imperial, ATI) Urban Structure 03-07-2017 37 / 48

Annealed importance sampling

Define the (log-gamma) density π0(X ) ∝ exp(−γU0(X )) by

U0(X ) = κ

M∑j=1

exp(Xj)− δM∑j=1

Xj .

We can sample from π0 exactly and easily.Define a temperature schedule 0 = t0 < t1 < · · · < tT = 1 with bridging distributions

πt = Z−1t π1−tt

0 πtt , for j = 1, . . . ,T .

Set w(n)0 = 1/N and X0 ∼ π0, for n = 1, . . . ,N. Then for t = 1, . . . ,T :

Draw X(n)t ∼ Kt(·|Xt−1), for n = 1, . . . ,N.

Set w(n)t = w

(n)t−1

πt(X

(n)t−1

)πt−1

(X

(n)t−1

) , for n = 1, . . . ,N.

Then Z = limN→∞∑N

n=1 w(n)T .


Bridging distributions

Figure: {πj}T=9j=1 from left to right, then top to bottom.


The signed measure problem

S can be negative when V(i) < V(i−1) for many i . This is known as the ‘sign problem’.

Rejecting when S is negative would introduce a bias.

Instead, we note that

E[h(X , θ)] =1

π(Y )

∫h(X , θ)π(Y |X , θ)π(X |θ, u)π(θ)π(u)dudθdX ,

=

∫h(X , θ)σ(X |θ, u)π(X , θ, u|Y )dudθdX∫

σ(X |θ, u)π(X , θ, u|Y )dudθdX,

where σ is the sign function and we have defined

π(X , θ, u|Y ) =π(Y |X , θ)|π(X |θ, u)|π(θ)π(u)∫

π(Y |X , θ)|π(X |θ, u)|π(θ)π(u)dudθdX.


Pseudo-marginal Markov chain Monte Carlo

We can sample from π(X , θ, u|Y ) using Metropolis-within-Gibbs with block updates.

X -update. Propose X ′ ∼ QX and accept with probability

min

{1,π(Y |X ′, θ) exp(−U(X ′; θ)) q(X |X ′)π(Y |X , θ) exp(−U(X ; θ)) q(X ′|X )

}.

(θ, u)-update. Propose (θ′, u′) ∼ Qθ,u and accept with probability

min

{1,π(Y |X , θ′)|S(θ, u)| exp(−U(X ; θ′))π(θ′)π(u′) q(θ|θ′)π(Y |X , θ)|S(θ′, u′)| exp(−U(X ; θ))π(θ)π(u)q(θ′|θ)

}.

Posterior expectations are estimated using

EX ,θ|Y [h(X , θ)] = limN→∞

∑Ni=1 h(Xi , θi )σ(Xi |θi , ui )∑N

i=1 σ(Xi |θi , ui ).


Airports in England

Figure: Left: Visualization of the posterior-marginals for the latent variables {xj}Mj=1 over a map ofEngland. Right: Visualization of the posterior demands.


The London Retail system

Figure: Left: Visualization of the posterior-marginals for the latent variables {xj}Mj=1 over a map ofLondon Right: Visualization of the posterior demands. Observation data obtained


Bayesian model comparison

Open question: Do consumers perceive distance linearly or logarithmically?

p(y |Hj) =

∫p(y |θj ,Hj)p(θj |Hj)dθj ,

p(y |θj ,Hj) =1

Z (θj ;Hj)

∫p(y |xj , θj ,Hj)q(xj |θj ,Hj)dxj ,

Z (θj ;Hj) =

∫q(xj |θj ,Hj)dxj .

Bayes’ factors

BF1,2 =p(y |H1)

p(y |H2).


Conclusions and outlook


Further work and extensions

Investigation of new data assimilation methodologies to calibrate models to data availableat different scales. For example:

Population data;Cost matrix; orTime dependent parameters.

Deployment of new methodology to a global-scale problem (non-Bayesian approaches?).

Extension of discrete-choice approach to other socio-economic phenomena e.g. crime.

Melding of data and models takes us beyond data analytics.


Summary

We have developed a novel stochastic model to simulate realistic configurations of urbanand regional structure.

Our model is an improvement on existing deterministic models in the literature, as weaccount for uncertainties arising in the modelling process.

We presented a Bayesian hierarchical model for urban and regional systems.

Our model can be used to infer the components of a utility function from observedstructure, rather than hidden flow data.

We have demonstrated our approach using an example of airports in England and retail inLondon.


Acknowledgment

EPSRC

Inference, Computation, and Numerics for Insights into Cities - ICONIC




Documents

Stochastic modelling of urban structurehelper.ipam.ucla.edu/publications/elws4/elws4_14284.pdf · Stochastic modelling of urban structure Louis Ellam Department of Mathematics,