Detection of Weak Radioactive Sources usingMobile Sensors with Positional Uncertainties

Indrajeet S Yadav, Herbert G Tanner

January 2, 2017

Abstract: This paper formulates an optimal steering law for mobilesensors with positional uncertainties using stochastic maximum principle andestablishes the effect of these uncertainties on the decision making accuracyof the sensor network. The goal is to decide if a target is carrying a weakradioactive source using a network of mobile sensors with noisy motion.The interplay between the mobility of a sensor and its decision making accu-racy has been well established for deterministic sensor motion using chernoffupper bounds on error probabilities as detection performance indicators. Itis shown that under certain simplifying assumptions, it is still possible touse deterministic bounds as performance indicators and solve stochastic op-timal control problem to derive the optimal steering law for mobile sensors.Further, the effect of uncertainties can be analyzed by varying diffusion co-efficient of underlying Stochastic Differential Equation representing sensordynamics. The numerical simulations indicate that the noisy sensor motioncould significantly reduce the decision making accuracy in a fixed time detec-tion test and is particularly detrimental to weak radiation sources then to thestronger sources.

1 Introduction

The upsurge in religious fundamentalism and terrorism in various parts ofthe world has increased the risk of sensitive nuclear technology being in thehands of non-state players. While acquiring the technology and materialto make a full-fledged nuclear device doesnt seem to be plausible, terrorist

1

groups could indeed acquire sensitive nuclear materials and mix it with con-ventional explosives to make what is called a “Dirty Bomb“; which, althoughnot as lethal a radiological threat as a conventional nuclear device, couldbe very disruptive to normal life and communal harmony. Security agenciesacross the globe are therefore putting greater emphasis on developing reli-able and efficient systems that can detect radioalogical threats and illlicitradioactive material transit. Byrd et al [4] advocates a multi-element ”sys-tem of systems” approach with the sensor based radiation detection systemas one of the important element. In this work we focus on increasing thereliability of a mobile sensor network assuming that some information aboutthe radiological threat is already available through other means.

Most detection systems deploy a distributed network of static sensorsand fuse their data together to make a decision. Nemzek et al [16] wereprobably the first to establish that the data fusion increases the Signal toNoise Ratio (SNR) which eventually results into improved detection perfor-mance. Many detection algorithms based on data fusion have been presentedsince then. Most of these algorithm couple detection and localization to-gether. These include deterministic solutions such as Maximum LikelihoodEstimator [7], Mean of Estimator [21], Iterative Pruning (ITP) [5]or prob-abilistic solutions like Least Square Estimation [14], Sequential ProbabilityRatio Test(SPRT) [10], [22] or [20] and Extended Kalman Filtering [8]. KMani Chandi et al [12] presents a very good summary of interdependenciesof various parameters critical to design a distributed sensor network.

The principle behind all these algorithms is to distinguish between thesignal (counts due to the source) from noise (background counts). By fus-ing the data together one hope to increase overall SNR, however, the mosteffective way to increase SNR is to move the sensor as close to the source aspossible. A limitation of static sensors is that they cannot be moved closer tothe source and therefore if the source is moving so that it crosses the coveragearea (area in which it can be detected reliably within a reasonable time) ofthe network before it gets detected or if the source is not in the coveragearea, the whole detection system is more or less ineffective. The problem isparticularly challenging in case of weak radioactive sources. Allowing limitedmobility to the sensors in such cases can overcome this limitation.

Not enough literature is available on detection or localization using mo-

2

bile sensors or when the soure is moving. Stephens Jr et al [25] considersdetection of a moving source when its motion is completely known. Brennanet al [3] presents a detection algorithm based on Bayesian techniques whenthe moving source’s trajectory is unknown. Ristic et al [24] presents an in-formation gain driven search for high intensity radioactive sources using amobile sensor, however, their goal was not to assess the effect of sensor mo-bility on detection or localization performance.

Pahlajani et al [17] were the first to study the effect of sensor mobility ondecision making accuracy of the sensor network. They considered nuclear de-tection as a hypothesis testing problem of deciding between a fixed intensitypoisson process (background radiaction) and a time-inhomogeneous poissonprocess (radiation due to the source). The detection and localiazton wereuncoupled in their approach and the asusmption was that the source motionis determined through other means and localization would be done using vi-sual sensors. We consider their assumptions reasonable and consistent withthe philosophy of “System of systems“approach [4] because relying only onnuclear sensors to do detection, localization and source intensity estimationwould require an extremely costly system whose efficiency may reduce dras-tically with the amount of shielding the stolen source has. Sun et al [26] usedthis mobility model to derive an optimal steering law for the mobile sensorsthat optimizes the detection performance.

This work is an extension of [17] and [26] in which we relax the assump-tion of a deterministic sensor motion. This is important because in practicaldetection scenarios motion of the sensor is never fully known due to inherentvariability and data measurement errors in IMU or GPS.

Specifically stated, the problem considerd in this paper is, a) derive anoptimal steering law for the sensor with stochastic motion and b) decide,at the end of a fixed time interval T , if the source is present. Because thedetection test is conducted at the end of time T when the sensor motion isfully realized, the decision part remain same as that presented by Pahlajaniet al [17]. The optimal control law can no longer be determined using opti-mal control techniques (as in [26]) owing to the state of the system being arandom variable now.

We used a Drift-Diffusion Model (DDM) based approach for modeling

3

the state of the sensor in which the constant diffusion term represents theuncertain element. DDMs have been considered as plausible models for deci-sion making tasks under the presence of uncertainty. [2] used DDMs to modelhuman behavioral data in Two-Alternative Forced-Choice (TAFC) tasks. [19]used DDMs to model stochastic evidence collectors and studied role of infor-mation centrality on the decision making accuray. Reverdy et al [23] used adrift diffusion model for robotic obstacle avoidance.

Modeling sensor motion in this way makes the position of the sensor (andtherefore the intensity of the observed poisson process due to the source) arandom variable with a time-varying mean. A complete mathematical treat-ment of deriving chernoff bounds for stochastic intensities with constant meanhas been presented by Hibey et al [9]. Deriving the bounds for stochasticintensities with time varying mean is an extremely challenging measure the-oretic problem and the advantages derived from such modeling in terms ofaccuracy of detection are unknown.

We therefore made an assumption that although weaker, the expressionsof deterministic bounds still represent a bound on true error probabilities instochastic case at low noise levels. This is justfied based on a seperate workof Pahlajani et al [18] which considers performance loss due to use of thedeterministic bounds when some parameters characterizing the observationprocess are not known exactly. A detailed discussion is presented in Section-4.

Equipped with the expressions for approximate bounds, we used Stochas-tic Maximum Principle (SMP) [1] to obtain the desired steering law by mini-mizing the expectation of the bound on probability of missed detection. SMPand Dynamic Programming Principle (DPP) are two most important tools forsolving stochastic optimal control problems. SMP has a distinct advantagethat it always gives necessary condition for optimization as against the suffi-cient condition given by Hamilton-Jacobi-Bellman(HJB) equations obtainedin DPP. Moreover, to arrive at the desired solution HJB equations searchfor full solution space and therefore computationally intensive. The cou-pled Forward-Backward Stochastic Differential Equations (FBSDEs)obtainedfrom SMP could be solved numerically using Milstein method [13], althoughin our case it was not required to solve them completely due to the simplify-ing assumptions. The sensor was moved as per the derived control law andsince all the parameters of the underlying SDEs are known, the stochastic

4

state of the sensor at each time point was obtained using Euler-Maruyamamethod [11]. Detection test was performed as per [17] at the end of specifiedtime interval.

The explicit contributions of this paper are:

• To the best of our knowledge this is the first attempt to solve a robotmotion optimization problem using stochastic maximum principle andnumerical solution of resulting FBSDEs. This framework can be ex-tended to solve various motion optimization problems under the pres-ence of uncertainties.

• It emphasizes the adverse effect of positional uncertainties on the deci-sion making accuracy of a moving sensor, particularly while detectinga weak radioactive source.

The remaining paper is organized as follows. Section II presents presents abackground of nuclear detection using mobile sensors when their motion iscompletely deterministic [17]. Section III presents a precise problem state-ment. Section IV presents the main stochastic optimal control methodologywe used, followed by results in section V and conclusions in section VI.

2 Background

This section presents a brief summary of the mobility model developed byPahalajani et al [17] for assessing the effect of sensor mobility on the decisionmaking accuracy of a sensor network.

2.1 Sensor Model

In the specific setting considered in this paper, we assume that we have in-formation about a target that could be carrying a radioactive point sourceof a specified intensity a (measured in neutron counts per seconds). Thetrajectory of this target can be specified as xt(t) ∈ R3. The target is to beclassified as either benign or radioactive within the time interval T using agroup of k mobile sensors (radiation counters). The trajectory of the sensori for i = 1, 2..., k is denoted by xi(t) ∈ R3.

5

The cumulative neutron counts observed by sensor i within time interval[0, t] is represented as Nt(i) and can occur due to the presence of an al-leged radioactive source or due to naturally occurring background radiation(henseforth referred to as background neutron activity). The intensity ofthe source perceived by the sensor is proportional to the inverse of squareddistance between the source and the sensor. For a sensor i ∈ 1, 2..., k of crosssection χi, the perceived intensity νi(t) due to the source (at a time instancet) is[]:

νi(t) =χia

2χi + ||(xt − xi)||2(1)

From a counter’s perspective the two signals (counts from the sourceand background) are identical and indistinguishable and so the detectionof a radioactive source is essentially distinguishing the signal (counts fromradioactive source) from the noise (counts due to background radioactivity).

2.2 Hypothesis Testing

Pahlajani et al [17] formulated it as a hypothesis testing problem in which thedecision is to choose between a Null hypothesis H0 (absence of the source)and an Alternate hypothesis H1(presence of the source) based on the ob-served neutron counts. They formulated a Neyman-pearson likelihood ratiotest(LRT) that essentially compares a global likelihood ratio LT accumulatedover a time interval [0, T ] against a suitably chosen threshold γ > 0, decidingH1 if LT ≥ γ, and H0 otherwise. Their setup forms the basis of our approachand is described below.

Let (Ω,F) is a measure space supporting a k-dimensional vector Nt

of counting processes which represents the counts observed at sensor i ∈1, 2, ...., k. The two hypothesis H0 and H1 corresponds to two distinct prob-ability measures in (Ω,F). Hypothesis H0 corresponds to a probability mea-sure P0 with respect to which Nt are independent Poisson processes withintensity βi(t) and H1 corresponds to a probability measure P1 with respectto which Nt are independent Poisson processes with intensity νi(t) + βi(t)respectively. The decision problem is to identify the correct probability mea-sure (P0 versus P1 ) based on the realization of process Nt. The intensitydue to the background βi(t) and due to the source νi(t) at the sensor i were

6

assumed to be continuous, strictly positive functions bounded within an in-terval.

Let FNt : 0 ≤ t ≤ T is the filtration generated by the process Nt. The

σ-field FNt thus represents the information generated by the totality of sensor

observations up to t = T . A test for deciding between hypotheses H0 and H1

on the basis of FNt observations can be thought of as a set A1 ∈ FN

t with thefollowing significance: if the outcome ω ∈ A1, decide H1; if ω ∈ A0 = Ω \A1,decide H0. For a test A1 ∈ FN

t , two types of errors might occur. A falsealarm occurs when the outcome ω ∈ A1 (i.e. decide H1) while H0 is the cor-rect hypothesis. This occurs with probability P0(A1). A miss occurs whenω ∈ Ω \ A1 (i.e. decide H0) while H1 is the correct hypothesis. This occurswith probability P1(Ω \ A1).

In this setting, the optimal test for deciding between H0 and H1 is anLRT obtained as follows [17]. For i ∈ 1, 2..k, let (τn(i) : n ≥ 1) denotes thejump times of Nt(i) and let

LT (i) = e−∫ T0 νi(s)ds

Nt(i)∏n=1

1 +νi(τn(i))

βi(τn(i))(2)

Assuming that P1 is absolutely continuous with P0; for a specific thresholdγ, the test

LT ≥ γ where LT =k∏i=1

LT (i) and γ ∈ R (3)

is optimal in the (Neyman-Pearson) sense that if A2 is any other test whoseprobability of false alarm P0(A2) ≤ P0(LT ≥ γ), then the probability of missfor test (3) is at least as low as that for A2, i.e. P1(LT < γ) ≤ P1(Ω \ A2).

2.3 Error Probability bounds for Deterministic SensorMotion

Exact Calculation of these two error probabilities require computationallyintensive Monte-Carlo simulations to determine the threshold γ for the like-lihood ratio test. [17] suggests use of chernoff bounds on error probabilitiesas surrogates for true error probabilities and derives these bounds in terms

7

of relative distance between the source and the sensor. For an ideal situationwhen the motion of the sensor and the source are completely deterministic,let ui represents the control input given to each sensor whose motion is gov-erned by differential equation xi = ui; the perceived source intensity (countrate) νi is a function of xi given by Equation-1. Since xi can be implicitlydetermined by the control input ui, νi is a functional operating on ui and wecan write it as νi(ui) or more generally as νi(u). For a scalar quantity

µi(u) = 1 +νi(u)

βi(4)

a scalar parameter p ∈ (0, 1) and η = log γ, the chernoff bounds onprobability of false alarm and missed detection can be given by:

PF ≤ exp

infp>0

[Λ(p)− pη]

, PM ≤ exp

infp<1

[Λ(p) + (1− p)η]

(5)

Where Λ(p) is specifically computable via

Λ(p) =k∑i=1

∫ T

0

[µi(s)p − pµi(s) + p− 1] βi(s)ds (6)

It is shown in [17] that if the bound on probability of false alarm is restrictedto an upper limit α, there exists a unique solution p∗ ∈ [0, 1] to the equationPF = α for which the tightest bound on probability of missed detectioncould be obtained and the exponent of bound on probability of false alarmand missed detection can be given by:

PF (u) =k∑i=1

∫ T

0

[p∗µi(s)p∗ log µi(s)− µi(s)p

∗+ 1]βi(s)ds = − logα (7)

PM(u) = logα + Λ′(p∗) where

Λ′(p∗) =

k∑i=1

∫ T

0

[µi(s)p∗ log µi(s)− µi(s) + 1]βi(s)ds

(8)

Where Λ′(p∗) = η is logarithm of the test threshold and is the derivative of

Λ(p∗) with respect to p∗. Note that the explicit dependency of µi on control

8

input(see Equation-4) is not shown for clarity. If the sensor motion i.e. µi(u)is known, the tightest bound on probability of missed detection for partic-ular probability of false alarm (α) can be determined by Equation-8. Thisis an indicator of detection performance or decision making accuracy of theindividual or network of sensor.

Conversely an optimal control problem can be formulated as follows. Findµi(u) that minimizes Λ

′(p∗) (a quantity proportional to the bound on proba-

bility of missed detection) for a given upper limit α on the bound on probabil-ity of false alarm. Jianxin et al [26] used Pontryagin’s maximum principle tosolve this problem and determined that the optimal control law is to close thegap between the source and the sensor as quickly as possible. Together [17]and [26] formulated the dependency of decision making accuracy on sensormobility and derived a control law that maximizes decision making accuracy,when the distance between the source and the sensor is known with certainty.

3 Problem statement

In this paper we relax the assumption of a perfectly deterministic sensormotion. Problem can be formally stated as follows. Given a network ofmobile sensors tasked to decide if a target is radioactive, establish the effectof positional uncertainties on the decision making accuracy of the sensors anddetermine a control law that maximizes the decision making accuracy. Asin [17], the decision making accuracy (detection performance) for the networkof sensors is defined as the probability of missed detection for a particularvalue probability of false alarm.

4 Stochastic Optimal Control

This section presents the derivation of an optimal control law for noisy mobilesensors using stochastic maximum principle.

4.1 Modeling Stochastic Sensor Dynamics

The uncertainties in the sensor motion have been modeled using two differentstochastic differential equations (SDEs). The full sensor motion has beendivided into two phases; 1) chasing phase in which the sensor chases the

9

target and 2) Hovering phase in which the sensor remain on top of the targetwith minimum control efforts. The dynamics of the sensor switches fromchasing phase to the hovering phase when the sensor is within a δ-ball fromthe target. If ui ∈ U and σi are the control input and constant diffusioncoefficient respectively for sensor i, W is the standard wiener process and kis a proportionality constant, sensor’s trajectory in two phases can be givenby:

dxi(t) =

ui(t)dt+ σidW when ||xt − xi||> δ

−kxi(t)− xt(t)dt+ σidW when ||xt − xt||≤ δ(9)

The uncertainties in the sensor motion have been modeled as additive whitenoise using constant diffusion coefficient. The objective is to maximize thedecision making accuracy by selecting an optimal control input ui(t) duringthe chase phase.

The control input is time-dependent during the chasing phase while thesensor motion in hovering phase is modeled by Ornstein-Uhlenbeck Stochas-tic Differential Equation (OU-SDE) [6]. During hovering phase the sensorcloses its gap with the target exponentially [6] and the rate of closing canbe controlled by appropriately selecting the proportionality constant k. Thisis analogous to switching to a PID controller in the hovering phase and theconstant k can be determined based on the parameters of the platform onwhich the moving sensor is mounted. The main advantage of using OU-SDEis that it minimize the variance in sensor position when the sensor is on topof the target. If the SDE with time dependent control input and constantdiffusion is used for full sensor motion, the variance in sensor position keepson increasing and is directly proportional to the time t [6]. By switching toOU equation the variance can be reduced exponentially to a very low value(depending on the value of k) [6]. As lowering the variance means reducingthe gap between the sensor and the source, it automatically results into abetter decision making accuracy.

4.2 Using Deterministic Bounds in Stochastic Case

Modeling the sensor motion using SDEs makes xi(t) a random variable whichultimately results into the variable νi and µi being random variables insteadof a smooth functions of time as they were in the deterministic case [17].

10

Hibey et al [9] presents stochastic bounds when the mean intensity of thecounting process is constant, however, in our case it is a time varying func-tion due to the mobility of the sensors. This makes exact computation ofstochastic bounds a complicated measure theoretic problem and thereforeinstead of deriving stochastic bounds we used deterministic bounds at theexpense of some sharpness. To this end, the following assumption has beenmade:

Assumption 1: Although weaker, the expressions of deterministic boundsstill represent a bound on true error probabilities in stochastic case at lownoise levels.

In the absence of any formal mathematical proof for this assumption, wejustify it heuristically as follows. In a separate work [18], Pahlajani et al es-tablished that the conservative approximation of decision performance givenby deterministic chernoff bounds vary smoothly with respect to small pertur-bations in underlying model parameters (source intensity, distance betweenthe source and the sensor). For us it means that small random fluctuations insensor position would not cause the bounds to change drastically and givena decent margin between true probabilities and deterministic bounds, samebound expressions can be used with a noisy sensor trajectory.

[17] indicate the availability of a wide margin between the true probabili-ties and their bounds. It can therefore be concluded that even in the presenceof small positional uncertainties, the deterministic bounds still bound the ac-tual probabilities. However, [18] does not provide any guidance on how smallthe “small perturbations“ should be in order to not blow up the bounds. Weused Monte-Carlo simulations as numerical evidence that for all the noiselevel studied in this paper, assumption-1 is indeed true.

Given that the intensity of the Poisson process is no longer a smoothfunction of time, following assumption have been made for βi(t) and νi(t):

Assumption 2: βi(t) and νi(t) are bounded random variables i.e. Pr(βmin ≤βi ≤ βmax) = 1 and Pr(νmin ≤ νi ≤ νmax) = 1

11

4.3 Stochastic Maximum Principle

The stochastic optimal control problem can be formulated as:

Minimize J(u) = E

k∑i=1

∫ T′

0

[µi(s)p log µi(s)− µi(s) + 1]βi(s)ds

Given :

PF (u) = E

k∑i=1

∫ T′

0

[pµi(s)p log µi(s)− µi(s)p + 1]βi(s)ds

= − logα

ui ∈ U and ||ui||≤ umax(10)

J(u) is the cost function same as Λ′(p) (see Equation-8). T

′is the time upto

which the sensor dynamics remain as defined by Equation-9a i.e. drift term isproportional to time dependent control input. Note that for an upper boundα on probability of false alarm, J(u) is proportional to the bound on prob-ability of missed detection. We use following results directly from [26]. Theapplicability of these results for stochastic case and associated modificationshas been discussed, interested reader may also refer to [26] for detailed proofs.

Lemma 1: For fixed u, expected value of PF is strictly increasing withp.Proof: p is not a random variable and so expected value of PF for a givenµ will be equal to PF itself and as per [], ∂PF

∂pis strictly positive when

µi ≥ 1, p ∈ 0, 1. Even though xi is a random variable, µi can neverbe < 1. Note that vector µ consists of k vectors µi, one for each sensori ∈ [1, 2..k] and each µi consists of countably many values µi calculated ateach time instant trajectory of sensor i.

Lemma 2: The functional mapping µ 7→ p,denoted φ, associates to eachµ a unique p.Proof: This proof is same as that given in [26].

Lemma 3: Consider the scalar p defined at a particular time instance

as p(t, µ) = φ(µ) +∑k

i=1

∂p∂µ

t(µi − µi) and PF µiµi is the second partial

12

derivative of PF wrt µi , then for all t1 ∈ [0, T ]:

∂p

∂µi

∣∣∣∣t=t1

=−βipµip−1 log µi + 1

2

PF µiµiσ2µidt

∆µi∑kj=1 pµj

p[log µj]2

Proof: This proof is considerably different than that used in [26] owing tothe fact that the scalar µi is a random variable now instead of a deterministicvariable. The second order term in Taylor expansion can not be neglected.Consider a function f(y) of a random variable y, its Taylor expansion abouta known value y0 is:

f(y) = f(y0) + fy(y0)(∆y) +1

2fyy(∆y)2 +R(y, y0)

Since y is a random variable, ∆y = y− y0 too is a random variable and con-sidering (∆y)2 as negligible because ∆y is small is equivalent to consideringvariance of y to be zero which is treating it like a deterministic variable [15].Therefore, E (∆y)2 > 0. The terms higher then second order would stillbe negligible. A more detailed discussion can be found in Chapter 7 of [15].The Taylor expansion of the function can then be:

f(y) ≈ f(y0) + fy(y0)(∆y) +1

2fyyE

(∆y)2

Where (∆y)2 has been replaced by its expectation, which is equivalent toreplacing the term with its “average“ value as an approximation. Anotherpossibility is to use, instead of E (∆y)2, some approximation limit of therandom variable (∆y)2 as time interval goes to zero. [15] (Chapter 4 and 10)shows that for infinitesimal intervals, the mean square limit of this term isdeterministic and is proportional to dt. This can be explained as follows. If yis a continuous time stochastic process that depends on a Wiener process (aswe expect µi to be as it is a function of x) and can be observed in continuoustime. We can partition interval [0, T

′] in which we are optimizing the sensor

motion into n equal parts of length h and can write ∆yk = akh+ σy∆Wk.

As n → ∞, h → 0 and ∆y = dy = σydW or (∆y)2 = (dy)2 = σ2y(dW )2.

However as (dW )2 = dt [15], (∆y)2 = (dy)2 = σ2ydt.

Let µ is a vector that contains µi’s for each sensor i ∈ [1, 2..k] and at alltime instances. Now consider a perturbed vector µ due to a small variation

13

in µ’s of sensor i at time instant t1. The Taylor expansion of integrand of PFis then:

PF (µ, p) = PF (µ, p) +∂PF∂µi

∣∣∣∣t=t1

∆µi +1

2

∂2PF∂µi2

∣∣∣∣t=t1

(∆µi)2

Since (∆µi)2 can be written as σ2

µidt, we get:

PF (µ, p) = PF (µ, p) + βip2µp−1

i log µi∆µi +1

2PF µiµiσ

2µidt

Total total variation in PF (due to both µ and p) can be obtained asfollows:

PF (µ, p)− PF (µ, p) = βip2µp−1

i log µi∆µi +1

2PF µiµiσ

2µidt+

∂PF∂p

∣∣∣∣t=t1

∆p

Given that ∆p =∑k

i=1∂p∂µ

∣∣∣∣t1

∆µi and that:

∂PF∂p

∣∣∣∣t=t1

= E

k∑j=1

pµjp[log µj]

2

=

k∑j=1

pµjp[log µj]

2

Note that for a given µj, expectation of the summation is equal to thesummation, It can be shown that

∂p

∂µi

∣∣∣∣t=t1

=−βipµip−1 log µi + 1

2

PF µiµiσ2µidt

∆µi∑kj=1 pµj

p[log µj]2

We are not ready to apply stochastic maximum principle and state ourmain result.Proposition 1: The solution of for sensor i ∈ 1, 2..k to the stochasticoptimal control problem (10) within the feasible control set U = u ∈ R3k :||ui||≤ umax is:

u =

umax

xt−xi||xt−xi|| ||xt − xi||> δ

Follow OU-SDE ||xt − xi||≤ δ

(11)

14

A detailed description of stochastic maximum principle taken from [1] is givenin ??ppendix-A.1. Here we only state the parts relevant to our problem. Thesensor dynamics is defined as per Equation-9a and the cost function is:

J(u) = E

k∑i=1

∫ T′

0

[µi(s)p log µi(s)− µi(s) + 1]βi(s)ds

Note that as per lemma 2, p = φ(u). Since J(u) is always finite, by

Fubini’s theorem:

J(u) = E

∫ T′

0

k∑i=1

[µi(s)p log µi(s)− µi(s) + 1]βi(s)ds

Along the optimal trajectory and optimal costate vector λ∗i , the Hamil-

tonian can be defined as:

H(x(t), u(t), λi(t)) =k∑i=1

[µpi log µi − µi + 1]βi +k∑i=1

λi(t)ui(t) (12)

Given that the diffusion coefficient σi for each sensor is a constant andthat the control input u(t) is not a function of x, the backward stochasticdifferential equation for the costate for an adapted square integrable processK can be defined as (see Appendix-A.1 for details):

−dλi(t) =∂

∂xi

k∑i=1

[µpi log µi − µi + 1]βi

dt+KdW

=∂

∂µi

k∑i=1

[µpi log µi − µi + 1]βi

∂µi∂xi

dt+KdW

=

k∑

j=1,i 6=j

µpj(log µj)2 ∂p

∂µiβj + [µpi (log µi)

2 ∂p

∂µi+ pµp−1

i log µi + µp−1i − 1]βi

∂µi∂xi

dt+KdW

=

k∑j=1

µpj(log µj)2βj−βipµp−1

i log µi + 12

PF µiµiσ2µidt

∆µi∑kj=1 µ

pj(log µj)2βj

+ pµp−1i log µi

∂µi∂xi

dt

+

(µp−1i − 1)βi

∂µi∂xi

dt+KdW

15

=

PF µiµiσ

2µi

2∆µi

∂µi∂xi

(dt)2 +

(µp−1i − 1)βi

∂µi∂xi

dt+KdW

For small time intervals (dt)2 goes to zero faster than ∆µi, first term in theequation tends to zero as dt→ 0 and the costate equation becomes:

dλi(t) = −

(µp−1i − 1)βi

∂µi∂xi

dt−KdW

The terminal boundary condition on the costate is λ(T′) = 0 since there is

no terminal cost. We therefore get a system of Forward-backward stochasticdifferential equations (FBSDEs), which along with the boundary conditionsare:

dxi(t) = u(t)dt+ σdW, x(0) = x0

dλi(t) = −

(µp−1i − 1)βi

∂µi∂xi

dt−KdW, λi(T′) = 0

(13)

∂µi∂xi

will be a normal derivative as per stochastic maximum principle formula-tion which can be easily obtained from Equation-1 and Equation-4, and canbe given as:

∂µi∂xi

=−2χia(||xt − xi||)

βi(2χi + ||xt − xi||2)2

Since ||xt−xi|| is always positive within [0, T′], ∂µi

∂xiwill always be a negative

quantity. In order to minimize the Hamiltonian, we need to find the signof λi(t). This can be done using Milstein’s method [13] described in detailin Appendix-A.2. However, a closer look reveal that due to our simplisticassumptions, full numerical solution of the FBSDEs are not required in ourcase. The reasoning is as follows.

Comparing the obtained system of FBSDEs to that in Appendix-A.2, itcan be shown that:

f(t, x, w) = 0⇒ b(t, x, w) = u(t) and g(t, x, w) = −

(µp−1i − 1)βi

∂µi∂xi

The semilinear partial differential equation obtained in variable w is:

∂w

∂t+ u(t)

∂w

∂xi+ σ2 ∂

2w

2∂x2i

−

(µp−1i − 1)βi

∂µi∂xi

= 0

16

The discretized solution w(tk, xj) of which at each time and space point willgive the costate λi. Given that w(T

′) = 0 (owing to the terminal condition

λi(T′)), the equation for w(tk, xj) (see Appendix-A.1) reveal that at time

interval N − 1, first two terms will be zero and sign of w(tN−1, xj) (i.e. signof λi(tN−1, xj)) depends on the sign of −

(µp−1

i − 1)βi∂µi∂xi

at tN−1.

Since µi ≥ 1 and p < 1, µp−1i − 1 < 0 and ∂µi

∂xi< 0; the sign of

−

(µp−1i − 1)βi

∂µi∂xi

will always be negative irrespective of the value spaceor time point.

The sign of w(tN−1, xj) will be negative for all space discretization pointsand so as the sign of w(tN−1, x) because it is just a linear interpolation oftwo space points. Since w(tN−1, x) for all x is negative, first two terms whilecalculating w(tN−2x) will be negative and third terms is always negative.Continuing this, it can be argued without even solving the FBSDEs thatthe sign of w and so λi at each space and time discretization point will benegative.

Thus in order to minimize the Hamiltonian in Equation-12, the controlinput u(t) = umax i.e. as maximum as possible. This means that the optimalcontrol strategy should be to close the gap between the sensor and the targetas quickly as possible. This is same as that obtained for the deterministiccase [26]. When the sensor is within δ region of the target, the sensor dynam-ics is switched to the OU-SDE and the sensor moves along with the target.This gives the optimal control law as per proposition-1:

u =

umax

xt−xi||xt−xi|| ||xt − xi||> δ

Follow OU-SDE ||xt − xi||≤ δ

4.4 Decision test and Effect of Uncertainties on Deci-sion Making

The detection part of the problem remain same as that in [17], this is becausethe likelihood ratio test for making the decision is performed at the endof time T at which the trajectory of the sensor is already observed and isno more stochastic. Note the difference between the time. Optimizationof sensor motion was performed within [0, T

′] before the sensor dynamics

17

switches to OU˙SDE while the likelihood ratio test is performed after T . Inbrief, the main steps for performing decision test are:

• Using the sensor dynamics defined by Equation-9, determine the tra-jectory of sensor i using maximum value of u and a given σ. This givesxi and µi for sensor i at each t ∈ [0, T ].

• Determine the value of parameter p ∈ [0, 1] for the given α usingEquation-7. If it is not solvable even at p = 1, the solution is notpossible within the specified time interval and T has to be increased.

• Given that Equation-7 has a solution, find the threshold (γ = exp Λ(p))for the detection test using Equation-6 and bound on probability ofmissed detection exp PM) using Equation-8.

• Perform likelihood ratio test as per Equation-3.

The effect of noise on decision making accuracy has been established byvarying the value of the diffusion coefficient σ in Equation- 9. In a fixedtime detection test, there is no control over the obtained value of the boundon probability of missed detection. It is governed by the optimal µi and p.However, we varied the end time and re-performed the test at each T in orderto determine at which T the bound on probability of missed detection reachesan acceptable value. Section 5 presents the results at different diffusioncoefficients and T .

4.5 Limits on Bounds

It is possible to derive an an upper and a lower limit on the deterministicbounds for a particular level of uncertainty (i.e. particular diffusion coef-ficient σ). This gives the most favorable and most detrimental effect ofpositional uncertainties. The tolerance interval is defined as:

If X is a continuous random variable with cumulative distribution functionFX(x) = P (X ≤ x) and E = (X1, X2, ..Xn) be a random sample from it, a(1− γ, 1− α) tolerance interval (LE , UE) satisfies the condition [27]:

PrE [(LE ≤ X ≤ UE |E) ≥ 1− γ] = 1− α (14)

Therefore the tolerance interval at any time instance is a statistical in-terval within which, with a certain confidence level (say (1 − α)%), a fixed

18

number (say (1 − γ)%) of xi would fall. For example, using 95% toleranceinterval, it is possible to bound 95% of xi of a moving sensor within twobounding values with 95% confidence. As per Equation-1 and Equation-4,these bounding xi’s will give bounding µi at each point which would be cor-responding to a best and a worst estimate of the performance bounds.

For a given tolerance factor ctf , variance V ar(xi) in xi and assuming thatthe sensor state xi is a normally distributed random variable, the toleranceinterval for xi is, [27]:

xi = xi ± ctf√V ar(xi) (15)

Tolerance factors which can be calculated using MATLAB algorithm dis-cussed in [27] for the normal distributions. A procedure for calculatinglimiting performance bounds without for stochastic case is as follows:

• Assuming sensor dynamics defined by Equation-9, find the variance inxi at each time point given by the equation [6]:

V ar(xi) =

σ2i t 0 ≤ t ≤ T

′(σ2i T′ − σ2

i

2k

)exp−2k(t−T ′ ) +

σ2i

2kT′ ≤ t ≤ T

T′

is the switching time and two variances are corresponding to twodifferent SDEs. .

• Use [27] to determine the tolerance factor ctf for a given diffusion co-efficient and a specified number of observations from an initial sample.Determine the maximum and minimum value of xi using Equation-15.

• Find the corresponding minimum and maximum values of µi usingEquation-1 and Equation-4.

• Since these limits on µi can be obtained for full trajectory of the sen-sor, the upper and lower limit of deterministic bounds can be easilycalculated from these µi.

• Note that when the sensor is on top of the sensor, both the upper andlower values of the xi will be less than xt which result into a lower µithan that obtained when xt − xi = 0. Therefore the upper limit on µishould be calculated assuming xi = xt so as to obtain maximum µi.

19

These bounds were calculated for each end time T and has been presentedin section-5. As expected, the stochastic approximation of the bounds fallwithin thee limiting values of the deterministic bounds, however, these limit-ing bounds are either overly conservative or overly optimistic and so cannotbe used for a realistic estimation of the performance.

5 Results

The optimal control law is similar to that obtained for the deterministic case,i.e. close the gap between the target and the sensor as quickly as possible.The likelihood ratio for the decision test can be obtained by combining like-lihood ratios of individual sensors (Equation-3). As the state and costateof individual sensor is uncoupled (Equation-13) (i.e. motion and positionaluncertainties of one sensor do not effect the other); it is possible to statewithout loss of generality that the effect of noise on the decision making ac-curacy of individual sensor will be indicative of effect of noise in k sensors ondecision making accuracy of the network. In other words, the dependencyof the detection performance on noise will be same irrespective of number ofsensors used.

Therefore, the numerical simulations have been performed for one dimen-sional case using one sensor. The results obtained will be applicable to anetwork of sensors and for 2 or 3 dimensional case. Domino neutron sensorD22S-20-D0010-V3 was used for numerical simulations. Experimentally ob-tained cross section of the sensor is χi = 0.045. The source was placed 100cmaway from the sensor and the sensor moves towards the source with a velocityof u = 50cm/sec. The bound on probability of false alarm α was 10−3. Thesensor switches dynamics at δ = 5cm and the constant k for OU-SDE wasassumed to be 10. The background count rate for the sensor was obtainedexperimentally and was 0.2 counts per minute(cpm) and the source intensitya was 20 cpm (100 times that of background).

Figure-1 shows the variation of the bound on probability of missed de-tection with varying end time T for a diffusion coefficient σ = 1.The de-terministic bound and its upper and lower limit are also The gray curvesindicate bounds for 500 different noise realizations. Clear difference betweenthe deterministic bound(black curve) and the approximate bound obtained

20

for noisy sensor(gray curves) is evident. Mean of gray curves is depicted bygreen curve. Note the difference in end time when black and green(or in gen-eral gray) curves approaches zero, or their difference at a particular end time.It indicates that for a fixed time test, decision based on deterministic boundscould lead to an underestimation of bound probability of missed detection.

Figure 1: Variation of Bound on Probability of Missed Detection with EndTime for a

βi= 100. Red and Magenta curve indicate upper and lower limit

on stochastic bounds. Black curve indicates the deterministic bound.

Figure-1 also indicate that using the limits on deterministic bounds couldeither lead to an underestimation or an overestimation. Although the differ-ence in bounds obtained for two different noise realizations my differ signif-icantly at the start, the bounds eventually converges. This is evident fromFigure-2 which shows that the standard deviation of the stochastic boundsgoes to 0 as end time increases. This indicates that using any of the noise re-alization to determine approximate sensor trajectory would results into samedecision making accuracy if sufficient time is allowed to make the decision.

21

Figure 2: Variation of Standard Deviation on Probability of Missed Detectionwith End Time for a

βi= 100.

Figure-3 show the variation of the mean of stochastic bounds on PM fordifferent diffusion coefficients. The deterministic bound is shown by blackcurve but the limits are not shown for clarity. As uncertainties in the sen-sor motion(diffusion coefficient) increases, the difference in deterministic andstochastic bounds increases and chances of underestimation of bound on PMincreases when using deterministic calculations. Figure-4 shows correspond-ing standard deviations at different diffusion coefficients. Spread in standarddeviation curve and the gap between deterministic and stochastic bounds onPM at higher noise level indicate that more time should be allowed to makesame quality decision as the uncertainty in sensor motion increases.

22

Figure 3: Variation of Bound on Probability of Missed Detection with EndTime for a

βi= 100 at different Diffusion Coefficients. Black curve indicates

Deterministic Bounds.

Figure 4: Variation of Standard Deviation on Probability of Missed Detectionwith End Time for a

βi= 100 at different Diffusion Coefficients.

23

Figure–5 and Figure-6 show variation of bounds with end time and theirstandard deviation for different source to background intensity ratios. Thedotted curves indicate corresponding deterministic bounds. The differencein deterministic bounds and their stochastic approximation increases as thesource intensity reduces while for high intensity sources deterministic boundand stochastic approximation are almost similar. The spread in standarddeviation curves also increases with reduces source intensity. Thus the un-certainty in motion is much more detrimental to the weak sources than tothe stronger sources.

Figure 5: Variation of Bound on Probability of Missed Detection with EndTime for different Source to background intensity ratio a

βiat σ = 1. Dotted

curves show corresponding Deterministic Bounds.

Monte-Carlo simulation results in Figure-7 depict that the actual proba-bilities are always bounded by the bounds and vary similarly to the bounds.

24

Figure 6: Variation of Standard Deviation on Probability of Missed Detectionwith End Time for a

βi= 100 at different Diffusion Coefficients.

6 Conclusions and Future Work

Under certain simplifying assumptions, the optimal strategy for a mobile sen-sor in presence of positional uncertanties remains the same as that in noisefree case i.e. close the gap between the source and the sensor as quickly aspossible. However, the uncertainties are detrimental to the detection perfor-mance particularly when the source to be detected is weak. An optimizationapproach was presented based on stochastic maximum principle and numer-ical solution of the resulting coupled stochastic differential equations. It waspossible to derive the control law without numerically solving the SDEs fora point mass system but the method can be easily extended to applicationswhere numerical solution is imperative to determine the control law. Monte-Carlo simulations established the ability of deterministic bounds to capturethe underlying true probabilities in stochastic case. Future work would in-volve accurate characterization of positional uncertainties in physical sensors,modeling their dynamics accurately and corroborate the simulation resultswith experiments.

25

References

[1] A. Bensoussan. Lectures on stochastic control. 972:1–62, 1982.

[2] R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen. Thephysics of optimal decision making: A formal analysis of models of per-formance in two-alternative forced-choice tasks. Psychological Review,113(4):700–765, October 2006.

[3] S. M. Brennan, A. M. Mielke, and D. C. Torney. Radioactive sourcedetection by sensor networks. IEEE Transactions on Nuclear Science,52(3):813–819, June 2005.

[4] R. C. Byrd, J. M. Moss, W. C. Priedhorsky, C. A. Pura, G. W. Richter,K. J. Saeger, W. R. Scarlett, S. C. Scott, and R. L. Wagner. Nucleardetection to prevent or defeat clandestine nuclear attack. IEEE SensorsJournal, 5(4):593–609, Aug 2005.

[5] Jren-Chit Chin, David K.Y. Yau, Nageswara S.V. Rao, Yong Yang,Chris Y.T. Ma, and Mallikarjun Shankar. Accurate localization of low-level radioactive source under noise and measurement errors. In Proceed-ings of the 6th ACM Conference on Embedded Network Sensor Systems,SenSys ’08, pages 183–196, New York, NY, USA, 2008. ACM.

[6] Crispin W. Gardiner. Handbook of stochastic methods : for physics,chemistry and the natural sciences. Springer series in synergetics.Springer, Berlin, 1983. Includes indexes.

[7] A. Gunatilaka, B. Ristic, and R. Gailis. On localisation of a radiologicalpoint source. In 2007 Information, Decision and Control, pages 236–241,Feb 2007.

[8] Ajith Gunatilaka, Ristic Branko, and Gailis Ralph. Radiological sourcelocalisation. Defence Science and Technology Organisation (Australia),2007.

[9] Joseph L. Hibey, Donald L. Snyder, and Jan H. van Schuppen. Error-probability bounds for continuous-time decision problems. IEEE Trans-actions on Information Theory, 24(5):608–622, 1978.

26

[10] K. D. Jarman, L. E. Smith, and D. K. Carlson. Sequential probabilityratio test for long-term radiation monitoring. IEEE Transactions onNuclear Science, 51(4):1662–1666, Aug 2004.

[11] Peter E. Kloeden and Eckhard Platen. Numerical solution of stochasticdifferential equations. Applications of mathematics. Springer, Berlin,New York, 1999.

[12] A. H. Liu, J. J. Bunn, and K. M. Chandy. An analysis of data fu-sion for radiation detection and localization. In 2010 13th InternationalConference on Information Fusion, pages 1–8, July 2010.

[13] G. N. Milstein and M. V. Tretyakov. Numerical algorithms for forward-backward stochastic differential equations. SIAM Journal on ScientificComputing, 28(2):561–582, 2006.

[14] K. R. Muske and J. W. Howse. Comparison of recursive estimationtechniques for position tracking radioactive sources. In Proceedings of the2001 American Control Conference. (Cat. No.01CH37148), volume 2,pages 1656–1660 vol.2, 2001.

[15] S.N. Neftci. An Introduction to the Mathematics of Financial Deriva-tives. Academic Press Advanced Finance. Elsevier Science, 2000.

[16] R. J. Nemzek, J. S. Dreicer, D. C. Torney, and T. T. Warnock. Dis-tributed sensor networks for detection of mobile radioactive sources.IEEE Transactions on Nuclear Science, 51(4):1693–1700, Aug 2004.

[17] Chetan D. Pahlajani, Jianxin Sun, Ioannis Poulakakis, and Herbert G.Tanner. Error probability bounds for nuclear detection: Improving accu-racy through controlled mobility. Automatica, 50(10):2470–2481, 2014.

[18] Chetan D. Pahlajani, Jianxin Sun, Ioannis Poulakakis, and Herbert G.Tanner. Performance bounds for mismatched decision schemes withpoisson process observations. Systems & Control Letters, 75:69–76, 2015.

[19] I. Poulakakis, G. F. Young, L. Scardovi, and N. E. Leonard. Informationcentrality and ordering of nodes for accuracy in noisy decision-makingnetworks. IEEE Transactions on Automatic Control, 61(4):1040–1045,April 2016.

27

[20] N. S. V. Rao, C. W. Glover, M. Shankar, Y. Yang, J. C. Chin, D. K. Y.Yau, C. Y. T. Ma, and S. Sahni. Improved sprt detection using localiza-tion with application to radiation sources. In 2009 12th InternationalConference on Information Fusion, pages 633–640, July 2009.

[21] N. S. V. Rao, M. Shankar, J. C. Chin, D. K. Y. Yau, C. Y. T. Ma,Y. Yang, J. C. Hou, X. Xu, and Sartaj Sahni. Localization under ran-dom measurements with application to radiation sources. In 2008 11thInternational Conference on Information Fusion, pages 1–8, June 2008.

[22] N. S. V. Rao, M. Shankar, J. C. Chin, D. K. Y. Yau, S. Srivathsan,S. S. Iyengar, Y. Yang, and J. C. Hou. Identification of low-level pointradiation sources using a sensor network. In 2008 International Confer-ence on Information Processing in Sensor Networks (ipsn 2008), pages493–504, April 2008.

[23] P. Reverdy, B. D. lhan, and D. E. Koditschek. A drift-diffusion model forrobotic obstacle avoidance. In 2015 IEEE/RSJ International Conferenceon Intelligent Robots and Systems (IROS), pages 6113–6120, Sept 2015.

[24] B. Ristic, A. Gunatilaka, and M. Rutten. An information gain drivensearch for a radioactive point source. In 2007 10th International Con-ference on Information Fusion, pages 1–8, July 2007.

[25] D. L. Stephens and A. J. Peurrung. Detection of moving radioactivesources using sensor networks. IEEE Transactions on Nuclear Science,51(5):2273–2278, Oct 2004.

[26] Jianxin Sun and Herbert G. Tanner. Constrained decision-makingfor low-count radiation detection by mobile sensors. Auton. Robots,39(4):519–536, 2015.

[27] Viktor Witkovsk. On the exact two-sided tolerance intervals for uni-variate normal distribution and linear regression. Austrian Journal ofStatistics, 43(4):279–292, 2014.

28

A Appendices

A.1 Stochastic Maximum Principle

The formulation of stochastic maximum principle is taken from [1]. Let, forany admissible control U , the system is represented by a general stochasticdifferential equation:

dx(t) = g(x(t), v(t), t)dt+ σ(x(t), t)dW (t), x(0) = x0

Where g(x(t), v(t), t) : Rn ×Rm × [0, T ]→ Rn and σ(x(t), t) : Rn × [0, T ]→L(Rn;Rn) satisfy:

• g is Borel measurable, continuously differentiable wrt x and v.

• |g(x, v, t)− g(x′, v, t)|≤ K(x− x′)

• |g(x, v, t)− g(x, v′, t)|≤ K(v − v′)

• |g(x, v, t) ≤ K1(|x|+|v|+1)

• σ is Borel measurable, continuously differentiable wrt x.

• |σ(x, t)− g(x′, t)|≤ K(x− x′)

• |σ(x, t)|≤ K1(|x|+1)

Let the cost functional for a finite horizon 0 < T <∞ can be given as:

J(v) = E

∫ T

0

l(x(t), v(t), t)dt+ h(x(T ))

Where l(x(t), v(t), t) is the running cost and h(x(T )) is the terminal cost.

l(x(t), v(t), t) : Rn ×Rm × [0, T ]→ Rn and h(x) are such that:

• l is Borel measurable, continuously differentiable wrt and v.

• |lx(x, v, t)|≤ C1(|x|+|v|+1)

• |lv(x, v, t)|≤ C2(|x|+|v|+1)

• l(0, 0, t) ∈ L∞(0, T )

29

• h(x) is continuously differentiable.

• |hx(x, v, t)|≤ C3(|x|+1)

The objective is to minimize J(v(.)) in U . If these assumptions hold truethan u(t) ∈ U is an optimal control that minimize J(u) a.e.t, a.s.,∀v ∈ Uadiff:

l(y(t), v, t

)+ λ(t)g

(y(t), v, t

)≥ l(y(t), u(t), t

)+ λ(t)g

(y(t), u(t), t

)(16)

Where y(t) is the optimal state and λ(t) is corresponding costate. The term

l(y(t), v, t

)+λ(t)g

(y(t), v, t

)is defined as the HamiltonianH

(x(t), v(t), λ(t), t

)of the system and above equation is corresponding to Stochastic Maximumprinciple. x(t) and λ(t) can be determined by solving the following coupledForward-Backward stochastic differential equations :

dx(t) = g(x(t), v(t), t

)dt+ σ

(x(t), t

)dW (t), x(0) = x0

−dλ(t) = g(x(t), v(t), λ(t), K(t), t

)dt+K(t)dW (t), λ(T ) = hx

(y(T )

)Where g(x(t), v(t), λ(t), K(t), t) is such that,

g(x(t), v(t), λ(t), K(t), t) =gTx (x(t), v(t), t)λ(t) + lx(x(t), v(t), t)− 〈σTxK(t)

A.2 Solution of FBSDEs

The optimal control u ∈ U can be obtained by solving the coupled forward-backward stochastic differential equations for state and costate. In theseequations, boundary conditions are given as initial condition for the statex(0) and the terminal condition for costate λ(T ). [13] presents a numericalsolution based on 4 point scheme of solving FBSDEs. This Appendix providean overview of their method.

If X = X(t) and a(t,X, Y ) are d-dimensional vectors; σ(t,X, Y ) is ad × n matrix; Y = Y (t), g(t,X, Y ) and ψ(x) are scalars; Z = Z(t) andf(t,X, Y ) are n-dimensional vectors and W (t) is an n-dimensional Ft≥0-adapted Wiener process on filtered probability space (Ω,F ,Ft, P ). Then,

30

there exists a unique Ft-adapted solution(X(t), Y (t), Z(t)

)to the coupled

FBSDEs,

dX = a(t,X, Y )dt+ σ(t,X, Y )dW (t), X(t0) = x

dY = −g(t,X, Y )dt− fT (t,X, Y )Zdt+ ZTdW (t), Y (T ) = ψ(X(T ))

(17)

under some appropriate smoothness and boundedness conditions on its coef-ficients.It is shown by [13] that the solution of the above system is connected with thesolution of the Cauchy problem for the semilinear partial differential equa-tion (PDE):

δw

δt+

d∑i=1

ai(t, x, u)δw

δxi+

1

2

d∑i,j=1

aij(t, x, u)δ2w

δxiδxj= −g(t, x, u)

−n∑k=1

fk(t, x, u)d∑i=1

σik(t, x, u)δw

δxi

For t < T, x ∈ Rd, w(T, x) = ψ(x) and aij :=∑d

k=1 σikσjk

Layer Method for solving PDE : For a one dimensional case, the par-tial differential equation converts into the form:

δw

δt+ b(t, x, u)

δw

δx+

1

2σ2(t, x, u)

δ2w

δx2+ g(t, x, u) = 0

such that t < T, x ∈ R, w(T, x) = ψ(x) and

b(t, x, y) := a(t, x, y) + f(t, x, y)σ(t, x, y)

If it can be shown that the coefficients b, σ, g and the function ψ are suffi-ciently smooth and that all these functions, together with their derivativesup to some order, are bounded on [t0, T ] × R × R; and that σ is boundedaway from zero: σ ≥ σ0, where σ0 is a positive constant. Then, there existsa unique and bounded solution u(t, x) to this problem.

31

For time and space discretization and constant h and κ:

T = tN > tN − 1 > ... > t0, h :=T − t0N

and xj = x0 + jκh

The approximate solution w(tk, x) can be determined by:

w(tN , x) = ψ(x)

w(tk, xj) =1

2w[tk+1, xj + hb(tk, xj, w(tk+1, xj))−

√hσ(tk, xj, w(tk+1, xj))]

+1

2w[tk+1, xj + hb(tk, xj, w(tk+1, xj)) +

√hσ(tk, xj, w(tk+1, xj))]

+hg(tk, xj, w(tk+1, xj))

w(tk, x) =xj+1 − xκh

w(tk, xj) +x− xjκh

w(tk, xj+1), xj ≤ x ≤ xj+1

j = 0,±1,±2.... and k = N − 1, N − 2...1, 0

(18)

Once w(tk, x) is determined, we can determine the partial derivation of wwith respect to x by:

∆u

∆x(tk, x) :=

u(tk, x+ γ√h)− u(tk, x− γ

√h)

2γ√h

For some γ > 0 close to the diffusion coefficient σ at the point (tk, x) and∆kW = W (tk + h)−W (tk).

The numerical solution of the stochastic differential equations can begiven now by:

X0 = x,

Xk+1 = Xk + a(tk, Xk, u(tk, Xk))h+ σ(tk, Xk, u(tk, Xk))∆kW, k = 0, 1, 2...

Yk = u(tk, Xk), Zk = σ(tk, Xk, Yk)∆u

∆x(tk, Xk)

(19)

32