Williams et.al. – Approximate Dynamic Programming for ...mitchell/Class/CS532M.2007W... · p) Jonatan Schroeder Approximate DP for Sensor Network Management. Problem Introduction

Problem IntroductionDynamic Programming Formulation

Project

Williams et.al. – Approximate DynamicProgramming for Communication-Constrained

Sensor Network ManagementAn Experimental Review and Visual Tool

Jonatan Schroeder

Project Outline – CPSC532MUniversity of British Columbia

Mar 17, 2008

Jonatan Schroeder Approximate DP for Sensor Network Management


Project

Outline

1 Problem Introduction

2 Dynamic Programming Formulation

3 Project

Based on: J. L. Williams, J. W. Fisher III, and A. S. Willsky. Approximatedynamic programming for communication-constrained sensor networkmanagement. IEEE Transactions on Signal Processing, 55(8):4300–4311,August 2007.



Project

Sensor Networks

Spatially distributed autonomous devicesSensor to monitor:

temperature, sound, pressurepollutantsexternal agentsbattlefield surveillance

Wireless communicationLimited energy



Project

Example



Project

Example



Project

Example



Project

Example



Project

Example



Project

Example



Project

Example



Project

The Problem

Identify the state (position, velocity) of the objectProbability Distribution Function (pdf)Estimate the object’s next state

Subset of sensors and a leader sensor

Objectives:Maximize the information estimation performanceMinimize the communication cost



Project

The Problem

Identify the state (position, velocity) of the objectProbability Distribution Function (pdf)Estimate the object’s next state

Subset of sensors and a leader sensorObjectives:

Maximize the information estimation performanceMinimize the communication cost



Project

Dynamic Programming Approach

Objective function: estimation performance orcommunication cost

Lagrangian relaxation

State: object actual positionPolicy: leader and sensor subset selectionRolling discrete horizon

In each step, N steps are calculated, and one is taken



Project

Object Dynamics

Object state: xk =

pxvxpyvy

Object dynamics: xk+1 = Fxk + wk

F =

1 T 0 00 1 0 00 0 1 T0 0 0 1

wk is a white Gaussian noise process



Project

Measurement Model

Measurement model: zsk = h(xk , s) + vs

k

h(xk , s) =a

‖Lxk − ys‖22 + b

vsk is a white Gaussian noise process

z0:k is the history of all measurements {z0, z1, . . . , zk}Estimation: in progress



Project

Communication

Cost (per bit) of direct communication: C̃ij ∝∥∥y i − y j

∥∥22

Cost considering path {i0, i1, . . . , inij}:

Cij =

nij∑k=1

C̃ik−1ik

Transmission of probabilistic model: Bp bitsTransmission of measurements: Bm bits



Project

Constrained Dynamic Programming

Conditional belief state: Xk , p (xk |z0:k−1) ∈ P(X )

Control: uk = (lk ,Sk ), lk ∈ S and Sk ⊂ SControl policy – time k : µk (Xk , lk−1)

Control policy set: πk = {µk , . . . , µk+N−1}



Project

Optimization Problem

minπ

E

[k+N−1∑

i=k

g (Xi , li−1, µi (Xi , li−1))

]

s.t.E

[k+N−1∑

i=k

G (Xi , li−1, µi (Xi , li−1))

]≤ M

Constraint addressed through Lagrangian relaxation:

Lk (Xk , lk−1, πk , λ) = E

[k+N−1∑

i=k

g (Xi , li−1, µi (Xi , li−1))

+ λ

(k+N−1∑

i=k

G (Xi , li−1, µi (Xi , li−1))−M

)]



Project

Optimization Problem

minπ

E

[k+N−1∑

i=k

g (Xi , li−1, µi (Xi , li−1))

]

s.t.E

[k+N−1∑

i=k

G (Xi , li−1, µi (Xi , li−1))

]≤ M

Constraint addressed through Lagrangian relaxation:

Lk (Xk , lk−1, πk , λ) = E

[k+N−1∑

i=k

g (Xi , li−1, µi (Xi , li−1))

+ λ

(k+N−1∑

i=k

G (Xi , li−1, µi (Xi , li−1))−M

)]Jonatan Schroeder Approximate DP for Sensor Network Management


Project

Dynamic Programming Function

Lk (Xk , lk−1, πk , λ) = E

[k+N−1∑

i=k

g (Xi , li−1, µi (Xi , li−1))

+ λ

(k+N−1∑

i=k

G (Xi , li−1, µi (Xi , li−1))−M

)]

JDk (Xk , lk−1, λ) = min

πkLk (Xk , lk−1, πk , λ)

JLk (Xk , lk−1) = max

λ≥0JD

k (Xk , lk−1, λ)



Project

Dynamic Programming Function

Lk (Xk , lk−1, πk , λ) = E

[k+N−1∑

i=k

g (Xi , li−1, µi (Xi , li−1))

+ λ

(k+N−1∑

i=k

G (Xi , li−1, µi (Xi , li−1))−M

)]

JDk (Xk , lk−1, λ) = min

πkLk (Xk , lk−1, πk , λ)

JLk (Xk , lk−1) = max

λ≥0JD

k (Xk , lk−1, λ)



Project

Complexity Evaluation

Ns sensorsNp samples for each policyN steps (horizon)Ns2Ns possible controlsNs2NsNp new total samplesO(NN

s 2NsNNNp )



Project

Linearized Gaussian Approximation

Considering the smoothness of zSkk

Function for entropy evaluation is simplifiedO(NN

s 2NsN)



Project

Greedy Sensor Subnet Selection

Each stage is divided in substagesA new problem is defined, algebraically equivalent to theoriginal problemSensors that increase the cost in a substage are discardedSensors are limited to a small neighbourhoodWorst-case complexity: O(NN3

s )



Project

Dynamic Programming Problem Implementation

Implementation of the problem described in this paperBoth communication contraint (CC) and informationconstraint (IC) approachesRepeat simulation resultsObtain extended resulsTool for visualizing:

Agents positionObject stateLeader and subset selection



Project

Williams et.al. – Approximate DynamicProgramming for Communication-Constrained

Sensor Network ManagementAn Experimental Review and Visual Tool

Jonatan Schroeder

Project Outline – CPSC532MUniversity of British Columbia

Mar 17, 2008


Documents

Williams et.al. – Approximate Dynamic Programming for ...mitchell/Class/CS532M.2007W... · p) Jonatan Schroeder Approximate DP for Sensor Network Management. Problem Introduction