Shortest paths through a reinforced random walk417959/FULLTEXT01.pdf · 2011. 5. 19. · describe a random walk model which can not be said to be solving the shortest path problem

U.U.D.M. Project Report 2011:8

Examensarbete i matematik, 30 hpHandledare och examinator: David J.T. Sumpter

Maj 2011

Department of MathematicsUppsala University

Shortest paths through a reinforced random walk

Dan Li

Acknowledgements

I would first like to thank Professor David Sumpter for his patient guidance, giving

demeanor, the many valuable lessons/opportunities, and friendship during my study. I would

like to thank Professor Sumpter, Anders Johansson, Qi Ma and Alma Kirlic at UU that have

helped me with many aspects of my study including intellectual discussions, advise, sharing

expertise, and friendship. I would like to thank the support of my family for their constant

source of encouragement, understanding, and love, without of which I could not have

completed this work.

II

Abstract

In this article, we first study random walks on special electric networks with unit

resistors and use four methods to solve Dirichlet problems in two dimensions. Then we study

random walks on general resistor networks and give probabilistic interpretations to the

qualities in electric network. In order to understand the movements of ants in an electric

network, we describe a random walk model where ants moving through the network do not

modify their path. After making a modification to this model by involving conductivity, we

build current reinforced random walk model since ants are made to respond the current they

generate. Here we allow ants to iteratively improve the path they take so that they can find the

shortest path. Two algorithms based on exponential distribution and Poisson distribution are

built and compared. We will look at the results about shortest path finding by random walks

and implement simulation of two algorithms for shortest path problems.

Keywords: shortest path problem, random walk, electric network, reinforced random

walk, exponential distribution, Poisson distribution

III

Contents

1. Introduction ............................................................................................ 1

2. Random walks on networks[12] ............................................................... 3

2.1 Random walks in two dimensions.................................................................... 3

2.1.1 Define problem in terms of particles walking in a network.............................................. 3

2.1.2 The payoff function: Probability of winning ...................................................................... 4

2.1.3 Correspondence with an electric network.......................................................................... 4

2.1.4 Harmonic functions in two dimensions; the uniqueness and maximum principle ......... 6

2.1.5 The Dirichlet problem .......................................................................................................... 7

2.2 The Monte Carlo solution................................................................................. 8

2.2.1 Monte Carlo Method ............................................................................................................ 8

2.2.2 Implementation on examples using method of Monte Carlo............................................ 9

2.3 The method of relaxations.............................................................................. 10

2.3.1 The method of relaxations ................................................................................................. 10

2.3.2 Implementation on examples using method of relaxations ............................................. 11

2.4 Exact solution by linear algebra ..................................................................... 12

2.5 Solution by the method of Markov chains ..................................................... 15

2.5.1 Some definitions[14][15]......................................................................................................... 15

2.5.2 Markov chain methods[14] .................................................................................................. 15

2.5.3 Implementation on examples using method of Markov chain........................................ 17

2.5.4 Summary ............................................................................................................................. 21

2.6 Random walks on more general networks ..................................................... 21

2.6.1 Random walks on a graph[12]............................................................................................. 21

2.6.2 Some language from graph theory[17] ............................................................................... 22

2.6.3 Probabilistic interpretation in electrical network............................................................ 22

2.6.4 Random walk model........................................................................................................... 24

3. Shortest path problems ......................................................................... 28

3.1 Ant colony optimization ................................................................................. 28

3.1.1 Ants’ foraging behavior and optimization ........................................................................ 29

3.1.2 Double bridge experiments ................................................................................................ 29

3.1.3 ACO for the Travelling Salesman Problem ...................................................................... 30

3.1.4 The Ant Colony Optimization Metaheuristic................................................................... 31

3.1.5 Ant System (AS) [3][20][21] ..................................................................................................... 32

3.1.6 Simple ant colony optimization (S-ACO) ......................................................................... 34

3.2 Current reinforced random walks................................................................... 35

3.2.1 Description of current reinforced random walks ............................................................ 35

3.2.2 Pheromone Rule and Ant Moving Rule ............................................................................ 36

3.2.3 A Simple Explanation ......................................................................................................... 37

3.2.4 Exponential distributed simulation................................................................................... 39

IV

V

3.2.5 Implementation in Matlab ................................................................................................. 40

3.3 An efficient algorithm based on Poisson distribution .................................... 46

3.3.1 Some knowledge of the Poisson process[14] ....................................................................... 46

3.3.2 Event-driven approaches ................................................................................................... 47

3.3.3 Poisson distributed simulation .......................................................................................... 48

3.3.4 Implementation in Matlab ................................................................................................. 49

4. Physarum solver ................................................................................... 56

4.1 Introduction to Physarum............................................................................... 56

4.2 Mathematical modeling of maze-solving by Physarum................................. 57

4.3 Biologically inspired adaptive network[27] ..................................................... 59

5. Appendix .............................................................................................. 62

References ................................................................................................ 76

Shortest paths through a reinforced random walk 1

1. Introduction

In our world, the emergence of complex behavior in a system consisting of simple,

interacting elements is among the most fascinating phenomena. We can find many

examples in every field of today’s scientific interest, ranging from the motion of animal

swarms[1] of biology, to the behavior of social groups[2]. Ant colonies, and more generally

social insect societies, are distributed systems that, in spite of the simplicity of their

individuals, present a highly structured social organization[3]. As a result of this

organization, ant colonies can accomplish complex tasks that in some cases far exceed the

individual capabilities of a single ant.

The field of “ant algorithms” [4] [5]studies models derived from the observation of real

ants’ behavior, and uses these models as a source of inspiration for the design of novel

algorithms for the solution of optimization and distributed control problems. There is a

problem faced by ants in an ant colony. How can a large number of individuals, none of

which have global information about the problem build a short connecting path between

food and the nest? Ants usually find their food sources by chance, for example, they reach

their destination in a stochastic way. In our model, the ants merely count on the local

information provided by chemical trail in order to guide themselves. They do not have

additional navigation or information processing capacities and are not subject to long-range

attracting forces to the food sources or to the nest. Ants coordinate their activities via

stigmergy[1] [6], a form of indirect communication mediated by modifications of the

environment. For example, a foraging ant[1] deposits a chemical on the ground which

increases the probability that other ants will follow the same path. We draw attention to this

specific collective phenomenon of trail formation[2]. Trails are adapted to the requirements

of their users. In the course of time, frequently used trails become more developed, making

them more attractive, whereas rarely used trails vanish again. These dynamical processes[7]

occur without any common planning or direct communication among the users.

How almost blind animals, like ants can manage to build the shortest paths from the

colony to the feeding sources and back? The collective behavior[8] is a form of behavior

where the more ants following a trail, the more attractive that trail becomes for being

followed. Pheromone trails[9] are the medium that used to communicate information among

individuals regarding paths, and to decide where to go. A moving ant lays pheromone in

varying quantities on the path. When an isolated ant moves at random, an ant encountering

a previously laid trail can detect it and decide to follow the path with high probability, thus

reinforcing the path with its own pheromone[11] . The probability with which an ant chooses


a path increases with the number of ants that chose the same path previously. The process is

characterized by a positive feedback loop. The trail formation model includes an equation

of motion, an equation for environmental changes, and an orientation relation.

The trails are used to connect the food sources with the nest to allow for a collective

exploitation of the food. In the case of ants, the markings named pheromones, which also

provide the basic orientation for foraging and homing of the animals[2]. Ants can find the

shortest way through a maze and even amoeba can minimize the length of their connections

between food sources. One way to see such problem solving is in terms of individuals

performing a random walk on a graph. As the individuals walk they reinforce the path they

have taken so others coming after them are more likely to take the same path[10]. Those

taking the shortest path will reinforce this most strongly. Under certain conditions, the path

created by such a process will converge to the shortest path. This project will look at results

about shortest path finding by random walks[12] and implement simulations of algorithm for

shortest path problems.

Ants are not unique in such shortest path problem. There are parallels between the

creation of pheromone trails in ants, the growth of tubes in Physarum and the construction

of trails by human pedestrians[2]. There is a sense in which nutrients, ants and humans all

perform a reinforced random walk within the network they create. Indeed, the algorithm

works in all three systems at some level. It is simple, being based on a reinforcement of

path (pheromone trail and slime mold’s tube) in response to traffic flow (ant traffic and

protoplasmic flow).

The article is structured as follows. First, describe random walks in networks[12] and

use different methods to find solution to the Dirichlet problem in two dimensions. Monte

Carlo method[12] is studied carefully. Random walks on general resistor networks are

studied and probabilistic interpretations to the qualities in electric network are given. Then,

describe a random walk model which can not be said to be solving the shortest path

problem. Based on this random walk model, the reinforced random walk model is built

after making a modification so that it is possible to make the random walkers moving

through the network modify their path. Discuss a mathematical result which shows that the

algorithm of reinforced random walk will, provided the algorithm is structured properly,

always converge to our optimal solution — the shortest path. In addition, I also study the

Ant Colony Optimization (ACO) [3] [6] and the Physarum Solver [13]. The algorithms of

reinforced random walk model based on exponential distribution and Poisson distribution

are stated and the simulations of algorithms are implemented using Matlab.


2. Random walks on networks[12]

In this chapter, random walks on finite networks will be studied. We follow the

monograph of Doyle and Snell [12] in what follows, giving different examples than

provided by them. We know that the connection between random walks and electrical

networks has been recognized for some time. We will establish the connection between the

electrical concepts of current and voltage and corresponding descriptive quantities of

random walks regarded as finite state Markov chains. Note that we call the random walker

as “particle” in electrical network. However, I would like to call the random walker as “ant”

in the random walk model of solving shortest paths problems.

2.1 Random walks in two dimensions

2.1.1 Define problem in terms of particles walking in a network

Let 21 2 1 2, : ,z z z z R

2 D

2 denote two-dimensional integer array (lattice).

Let , then the boundary of is D : 1cD y D x D x y . Note that

and are disjoint necessarily. A simple random walk starting at

D

D 2x is a stochastic

process, that is, a simple random walk is a random process where a walker starts at point x

and chooses one of its neighbors with equal probability and moves there at each time unit.

In the figures 2-1-1 and 2-1-2 below, we illustrate two such arrays. We consider the

case with boundary values 1 or 0 in the figures. The large dots represent boundary points;

those marked 0 indicate that the walker receive no payoff and those marked 1 indicate that

the walker gets payoff 1. We study a simple random walk on a two dimensional array. The

walker, starting at the interior point ,x a b , moves randomly to one of the four

neighboring points: , 1 1,y a b b2 1,y a , 3 , 1y a b or 4 , 1y a b . Each

alternative is chosen equally likely, i.e. with probability1

4. This random moving continues

until the walker reaches a boundary point, in which case he/she remains at this point, i.e.

the random walk is absorbed at the boundary points and obtain the payoff indicated.


Figure 2-1-1: Small arrays for a random walk Figure 2-1-2: Another arrays for a random walk

2.1.2 The payoff function: Probability of winning

Consider a random walk on one of the arrays in figures 2-1-1 and 2-1-2. Let p x be

the probability, starting at x , that the random walk is absorbed at a boundary point with

value 1. It is also the expected final payoff for starting point x . We regard p x as a

function defined on the points of the array. Using conditioning with respect to the next

point visited, it is easy to see that the function p x has the following fundamental

properties:

1. or 0p x 1p x if x is a boundary point with value 0 or 1, respectively.

2. For an interior point x , the value p x is the average of p y at the neighbours,

i.e.

. 1 2 3

1

4 4p x p y p y p y p y (2.1)

We can express (2.1) by stating that the function p is discrete-harmonic (at each interior

point). The function p x , will in other words be a solution the discrete Dirichlet problem

for the array: It is harmonic in the interior with specified values at the boundary.

2.1.3 Correspondence with an electric network

We consider an electric network depicted in figure 2-1-2. We assume that all resistors

have the same resistance. Voltage 1v x is established at the boundary points

1, 2, 4,5,9,13x and voltage 0v x , for 8,12,14x . We obtain this by shorting

the two set of points and applying a voltage source (battery) between them.


Figure 2-1-3: An electric network

If two points x and are connected by a resistance of magnitude , then the

current through the resistor equals

y r

,i x y v x v y r / by Ohm’s law. By

Kirchhoff’s current laws, the current flowing into a point x must be equal to the current

flowing out. For an interior point x , the current flowing into x equals

1 2 3 4.

y N x

v y v x

r

v y v x v y v x v y v x v y v x

r r r r

where is a shorthand for +u max ,0u and where 1 2 3 4: , , ,N x y y y y denotes the

four neighbors of x . The flow out of node x is similarly obtained as

y N x

v x v y

r

and equating both sides gives Kirchhoff’s current law in the form

y N x y N x

v y v x v x v y

r r

which is equivalent to

0

y N x

v x v y

r

(2.2)

since . Multiplying through by , and solving for gives + +a b b a a b r v x

1 2 3 4

1.

4v x v y v y v y v y

That is, the voltage function satisfies the same equation (2.1) as the payoff

function

v x

p x constructed before. It is also a fact that v x will have the same values at


the boundary points as p x if we identify the electrical network. It is natural to guess

that they are equal, and in order to deduce this, we will need a uniqueness principle for

discrete harmonic functions.

2.1.4 Harmonic functions in two dimensions; the uniqueness and maximum principle

We define harmonic functions for sets of lattice points in the plane (a lattice point is a

point in the plane with integer coordinates). Let be a finite set of lattice points

partitioned into two disjoint set

S

B and such that: (a) every vertex in has four

neighbours in and (b) every pair of points and in can be connected by a p

consisting of vertices in S , i.e. a sequence P x

D D

S u v

lx

S ath

1 2...x , 1x u and lx v , of points ix

in S , where, for all i , ix is a neighbour of 1ix . We refer to points in B as boundary

points and points in D as inte r points. rio

A function f is harmonic on S B D if it satisfies (2.1) for all points x in D , i.e.

the average equation

1.

| | y N x

f xN x

f y (2.3)

There is no restriction on the values of f x at the boundary points x in B .

As we have seen, the payoff function p x , defined in section 2.1.2, is harmonic

which follows by conditioning on the first steps. The voltage function v x is harmonic,

which follows by Kirchhoff’s Current Laws and Ohm’s Law.

The Uniqueness Principle for the Dirichlet problem asserts that there cannot be two

different harmonic functions having the same boundary values: If p x and are

two functions that are harmonic on

v x

S B D with boundary B and if

all z en

v z p z for

thB p x v r all x fo x S . This is a general fact about harmonic

functions. We will approach the Uniqueness Principle by way of the Maximum Principle

for harmonic functions. The Maximum Principle states that a harmonic function defined on

takes on its maximum value S max :M f x x S and its minimum value

minm f :x x S on the boundary B , i.e. points where the averaging equation (2.3)


does not hold. The argument for it is natural but we do not give the details: At interior

points f x equals the average of its neighbours so f x must either be smaller than

one of its neighbours or else all neighbours have the value f x . If f x M at one

interior point x then f z M for some z S , since we can connect x by a path to a

boundary vertex . The argument for the minimum value is symmetric. z

p x v x and To deduce the uniqueness principle, we note that if are harmonic in

then is also harmonic. This is due to equation (2.3) being a

homogenous linear equation and is readily verified by direct application. By assumption,

we have , for all in

S r x

r z

v

v

x

z

p x

0p z z B . But then the Maximum Principle states

that the maximum and minimum value of r x is zero which means that for all r x 0

x S and this means that p x v x x S for all .

2.1.5 The Dirichlet problem

In the following sections 2.2~ 2.5, we will show how the simple random walks in

figures 2-1-1 and 2-1-2 could be used to construct the solutions to a discrete version of the

classical Dirichlet problem, say discrete Dirichlet problem. It is necessary to define

harmonic function and Laplace operator in a discrete setting.

Suppose that is bounded. The Dirichlet problem is to find the unique

harmonic function : such that i) Laplace’s equation

2

D R

D

u 0u x x D , where

is the Laplace operator; ii) p xu x x D .

0u x If , is harmonic on if for every D2D u x D , we have .

Suppose that . The Laplace operator of is defined as 2u R u:

1 1

1 1

4 4e e

u x u x e

1e 1

4u xu x e u x

. This definition gives us

another view of Laplace operator, as the difference between the mean value of over the

neighbors of point

u

x and the value of at u x .


2.2 The Monte Carlo solution

We will generate approximate solution to a Dirichlet problem in two dimensions

before finding the exact solution. In this section, we present a Monte Carlo method that

using random walks for finding approximate solutions.

2.2.1 Monte Carlo Method

The Monte Carlo method can be used to calculate the scalar field satisfying Laplace’s

equation over a solution region with specified boundary conditions. It is basically an

application of the law of large numbers, since (an extension of) the argument above shows

that an absorbing random walk starting at x obtains the harmonic solution as the expected

final payoff. A given random walk is terminated when the boundary B of the solution

region is reached. Assume p z f z for boundary points z B are prescribed

boundary values or final payoffs. We want to compute or approximate the expected payoff

p x starting at the interior point x D . Assume that we can make random

experiments starting at

N

x and compute the empirical averages of the payoff

1, ,

z B

p x N N z p zN

where is the number of times the walk was absorbed at N z z B . By the Law of

Large Numbers we obtain that , p x N approach p x with probability one as

. N

Figure 2-2-1: The Matlab code in Appendix 2-2-1 calculates the scalar field at the centre of a square solution region

with boundary conditions on the bottom, right, top, and left boundaries, respectively.

Let us give an example and make the implementation using Monte Carlo method. In

figure 2-2-2, we consider the case with boundary values 1 or 0 as in our example. In this

case, the expected final payoff is the probability that the walker ends up at a boundary point


with value 1 (“success”) rather than a point of value 0 (“failure”). We start

random walks from each of the interior points and, for each

10,000N

x , count the number of

successes and estimate

n x

/p x n x N . The Matlab code is stated in Appendix 2-2-2.

Figure 2-2-2 The network and boundary values Figure 2-2-3 The nodes are denoted with integers

We get the outcome in Matlab:

We reorder the outcome and rewrite the solution in the form as figure 2-2-4:

1 1

0.837 0.837 1

1 0.838 0.516 0.372 0

1 0 0

Figure 2-2-4: Empirical frequency of success

2.2.2 Implementation on examples using method of Monte Carlo

We implement Monte Carlo simulation on our two examples which are shown in

figures 2-1-1 and 2-1-2 in section 2.1.1 and get the Monte Carlo results shown in figures

2-2-5 and 2-2-6. The Matlab code can be found in Appendix 2-2-3 (a) and 2-2-3 (b).

(a) For figure 2-1-1:


0 0

1 0.290 0.085 0

0 0.080 0.043 0

0 0

Figure 2-1-1: The network and boundary values Figure 2-2-5: Empirical frequency of success

(b) For figure 2-1-2:

1 1

1 0.830 0.789

1 0.877 0.502 0.326 0

1 0 0

1

Figure 2-1-2: The network and boundary values Figure 2-2-6: Empirical frequency of success

2.3 The method of relaxations

In this section, we will describe the method of relaxation. It is a more effective

method for finding approximate solutions to a discrete Dirichlet problem. This method was

originally developed as a way to get approximate solutions to continuous Dirichlet problem.

But we would like to replace the continuous problem with a discrete problem.

2.3.1 The method of relaxations

The method of relaxations[12] is a simple numerical scheme to get approximate

solutions to a discrete Dirichlet problem. We are looking for a function which has specified

boundary values, for which the value at any interior point is the average of the values at its

neighbours. The method goes like this. Pick up an interior point and adjust the value of the

function to be the average of the values at its neighbours. Then continue through the rest of

the interior points, updating every interior point by the average of its neighbours. After

running though all the interior points, the function will not be harmonic after only one

iteration of this process, although the function we get will be more harmonic than the

function we started with. If we iterate this averaging process, running all of the interior

points again and again, the resulting function will better and closely approximate the

solution to our Dirichlet problem.


2.3.2 Implementation on examples using method of relaxations

We will illustrate the method of relaxations in the case of the examples introduced in

section 2.1.1.

(a) For figure 2-1-1:

In figure 2-3-1, the computations are illustrated on our simple example (example in

figure 2-1-1). We first assign the value 0 to all interior points and keep the boundary points

fixed. Each iteration traverses all interior points in order. Note that we go from left to right

moving up each column replacing each value by the average of the four neighbouring

values. We print the results to three decimal places. The computations for this first iteration

of the method of relaxation are

0.250 1/ 4 1 0 0 0 ; 0.063 1/ 4 0.25 0 0 0 ;

0.063 1/ 4 0.25 0 0 0 ; 0.031 1/ 4 0.063 0.063 0 0 .

We continue the iteration and at iteration 5 and 6, we obtain the same results to three

decimal places. It is obvious that the values for the interior points have converged to the

required harmonic function.

Thus, the solution to the Dirichlet problem is precisely as (h) in figure 2-3-1.

0 0

1 0 0 0

0 0 0 0

0 0

0 0

1 0.250 0.063 0

0 0.063 0.031 0

0 0

(a) figure 2-1-1: The Dirichlet problem (b) Initial configuration (c) After one iteration

0 0

1 0.281 0.078 0

0 0.078 0.039 0

0 0

0 0

1 0.289 0.082 0

0 0.082 0.041 0

0 0

0 0

1 0.291 0.083 0

0 0.083 0.042 0

0 0

(d) After two iterations (e) After three iterations (f) After four iterations

0 0

1 0.292 0.084 0

0 0.084 0.042 0

0 0

0 0

1 0.292 0.084 0

0 0.084 0.042 0

0 0

(g) After five iterations (h) After six iterations

Figure 2-3-1 The relaxation method (for figure 2-1-1)


(b) For figure 2-1-2:

In figure 2-3-2, the computations are illustrated on a slightly larger example (example

in figure 2-1-2).

1 1

1 0 0 1

1 0 0 0 0

1 0 0

1 1

1 0.547 0.648 1

1 0.750 0.188 0.047 0

1 0 0

(a) figure 2-1-2: The Dirichlet problem (b) Initial configuration (c) After one iteration

1 1

1 0.749 0.749 1

1 0.797 0.348 0.249 0

1 0 0

1 1

1 0.802 0.776 1

1 0.837 0.459 0.302 0

1 0 0

1 1

1 0.817 0.782 1

1 0.865 0.492 0.310 0

1 0 0

(d) After two iterations (e) After three iterations (f) After four iterations

1 1

1 0.821 0.786 1

1 0.873 0.5 0.321 0

1 0 0

1 1

1 0.823 0.787 1

1 0.875 0.504 0.323 0

1 0 0

1 1

1 0.823 0.787 1

1 0.875 0.506 0.323 0

1 0 0

(g) After five iterations (h) After six iterations (i) After seven iterations

1 1

1 0.823 0.787 1

1 0.876 0.506 0.323 0

1 0 0

1 1

1 0.823 0.787 1

1 0.876 0.506 0.323 0

1 0 0

(j) After eight iterations (k) After nine iterations

Figure 2-3-2 The relaxation method (for figure 2-1-2)

After only nine iterations, we obtain the same result to three decimal places accuracy.

This only took a fraction of a second of computing time. We shall say that these results are

correct to three place accuracy. The Monte Carlo method took several seconds of

computing time and did not even give three place accuracy.

2.4 Exact solution by linear algebra

In this section, we will find an exact solution to a two dimensional Dirichlet problem.

We introduce a method by solving a system of linear equations.


(a) For figure 2-1-1,

Figure 2-1-1: The interior points are labelled a, b, c, d

By averaging property, we have

1

4

4

4

4

b ca

a db

a dc

b cd

x xx

x xx

x xx

x xx

After rewriting these equations in the matrix form, then we will get

1 1/ 4 1/ 4 0 1/ 4

1/ 4 1 0 1/ 4 0

1/ 4 0 1 1/ 4 0

0 1/ 4 1/ 4 1 0

a

b

c

d

x

x

x

x

For simplicity, we write it as Ax u . must have an inverse since there is a unique

solution. And we have

A

1x A u .

Carrying out this calculation, we will have

0.292

0.083

0.083

0.042

Calculated x

We compare this calculated solution with the approximate solutions found earlier:

0.290

0.085

0.080

0.043

Monte Carlo x

. .

0.292

0.084Re

0.084

0.042

laxed x


We can find that Monte Carlo approximations were fairly good because no error is

greater than 0.01. The relaxed approximations were good indeed since two values do not

show error and the errors of the other two values are only 0.001 (that is far less than 0.01).

(b) For figure 2-1-2,

Figure 2-1-2: The interior points have been labelled a, b, c, d, e

After labelling each interior point as , we represent the average property of

each interior point in terms of the following matrix form

, , , ,a b c d e

1 1/ 4 0 1/ 4 0 1/ 2

1/ 4 1 0 0 1/ 4 1/ 2

0 0 1 1/ 4 0 3/

1/ 4 0 1/ 4 1 1/ 4 0

0 1/ 4 0 1/ 4 1 0

a

b

c

d

e

x

x

x

x

x

4

.

There is a unique solution:

0.823

0.787

0.876

0.506

0.323

Calculated x

For comparison, the approximate solutions found earlier using two different method

are shown again

0.830

0.789

0.877

0.502

0.326

Monte Carlo x

. .

0.823

0.787

Re 0.876

0.506

0.323

laxed x

We see that the Monte Carlo approximations were fairly good because no error of the

simulation is greater than 0.01. The relaxed approximations were good indeed, because no

error occurs at all


2.5 Solution by the method of Markov chains

Solving an equation like can be interpreted as finding the fundamental matrix of

a Markov chain. In this section we will show how to find exact solution to the Dirichlet

problem by the method of Markov chains. It can be seen as a more complicated version of

the method of linear equations which has been stated above.

2.5.1 Some definitions[14][15]

We have a set of states 1 2, , , rS s s s

r

and a chance process that moves around

through these states. The process moves from the state to the state with the

probability . This process is called finite Markov chain. We can represent the transition

probabilities as an matrix P . This matrix is called transition matrix.

is

P

js

ijP

ijP r by

If we raise the transition matrix to its power as , the elements can be

stated as the probability that the chain that started in state will be in state after

steps. A Markov chain is a regular chain if some power of the transition matrix has no

zeros.

P nth nP

is

nijP

js n

A state with is called closed state or absorbing state. Once entered, cannot

be left. If a Markov chain has at least one absorbing state and if it is possible, not

necessarily in one step, to reach at least one absorbing state from any state, it is called an

Absorbing Markov chain. The states of an absorbing Markov chain that are not traps can be

called non-absorbing states. An absorbing Markov chain will finally end up in an absorbing

state if it is started in a non-absorbing state. We can denote by the matrix

i 1iip

B with elements

ijB which can be stated as the probability, starting at a non-absorbing state , that the

process ends at an absorbing state .

is

js

2.5.2 Markov chain[14] methods

We can find a way to solve our Dirichlet problem if we could find out the matrix B

by Markov chain techniques. In fact, the Dirichlet problem has a generalization in the

context of absorbing Markov chains so that it can be solved by Markov chain methods. We

will show this later.


We assume is an absorbing Markov chain which has absorbing states and

non-absorbing states. In order to let the absorbing states come first and the non-absorbing

states come last, we reorder the states then rewrite our transition matrix of dimension

as the canonical form

P

u v

P

u v by u v

0IP

R Q

,

where I is a identity matrix; is matrix with all elements 0. u by u 0 u by v

The matrix is called the fundamental matrix, it can be interpreted as

(Note that

1N I Q

2 ...I Q Q 1N I Q

I is not same as the identity matrix we stated

above, it is dimension here). The elements of the matrix represent the

expected number of times that the chain, starting at , will reach state before

absorption. In order to get the probability of ending up at a given absorbing state, we add

up the probabilities of going that given absorbing state from all the non-absorbing states

weighted by the number of times we expected to be in those non-absorbing states. So the

absorption probabilities

v by v ijN

iS

N

S j

B can be calculated by matrix formula 1B NR I Q R

.

Let us see a random walk example in figure 2-5-1 (a). We find the matrix B by the

Markov chain techniques which are shown in figure 2-5-1 (b) ~ (e):

(a) Random walks starts at 2, ends at 0 or 4 (b) The canonical form of the transition matrix

0 1/ 2 0

1/ 2 0 1/ 2

0 1/ 2 0

Q

1

3 / 2 1 1/ 2

1 2 1

1/ 2 1 3/ 2

N I Q

(c) Random walks reach non-absorbing states (d) The fundamental matrix


(e) The absorption probabilities matrix B

Figure 2-5-1 Markov chain methods

The absorbing Markov chain can be made to correspond to a Dirichlet problem in the

following way. The absorbing states can be seen as boundary states and the non-absorbing

states can be seen as interior states. Let B be the set of boundary states and the set of

interior states as we have stated in section 2.1.4. Let

D

f be a function which domain is the

state space of a Markov chain . For in set D , we have ijj

f i P fP i j . Then f

is a harmonic function for . If we represent P f as a column vector , f f is harmonic

if and only if . f=fP

We can write as , where represents the values of ff

ff

B

D

fB f at the boundary

points and fD represents the values of f at the interior points. We have

f f

f fB

D D

I

0

B Q

B

. If we let f fD BNR , then we see that

f1f f f f f fD B B B B B BR QN R

f =Bf

I QN R N NR Q I Q Q NR R N . Thus

D B gives a solution to . In Markov chain theory 1B NR I R

Q . Thus

1f fD BI Q

R . From this equation, we can see that the values of a harmonic function at

the boundary points determine the values of the function.

2.5.3 Implementation on examples using method of Markov chain

We will illustrate the method of Markov chain in the case of our two examples.

(a) For figure 2-1-1,


Figure 2-1-1 Figure 2-5-2

1) We regard this random walk as a Markov chain with states 1,2,3,…,12 in figure

2-5-2. The transition matrix is given by:

1 0 0 0 0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0 0 0 0

0 0 1 0 0 0 0 0 0 0 0 0

1/ 4 0 1/ 4 0 1/ 4 0 0 1/ 4 0 0 0 0

0 1/ 4 0 1/ 4 0 1/ 4 0 0 1/ 4 0 0 0

0 0 0 0 0 1 0 0 0 0 0 0

0 0 0 0 0 0 1 0 0 0 0 0

0 0 0 1/ 4 0 0 1/ 4 0 1/ 4 0 1/ 4 0

0 0 0 0 1/ 4 0 0 1/ 4 0 1/ 4 0 1/ 4

0 0 0 0 0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 0 0 0 1 0

0 0 0 0 0 0 0 0 0 0 0 1

P

2) We reorder the states so that the absorbing states come first and the non-absorbing

states come last. Then our transition matrix has the canonical form:

0IP

R Q

,

where R is the matrix of random walks reach absorbing states; is the matrix of

random walks reach non-absorbing states.

Q


1/ 4 0 1/ 4 0 0 0 0 0

0 1/ 4 0 1/ 4 0 0 0 0

0 0 0 1/ 4 0 0 1/ 4 0

0 0 0 0 0 1/ 4 0 1/ 4

R

, ,

0 1/ 4 1/ 4 0

1/ 4 0 0 1/ 4

1/ 4 0 0 1/ 4

0 1/ 4 1/ 4 0

Q

1

0.2917 0.0833 0.2917 0.1667 0 0.0417 0.0833 0.0417

0.0833 0.2917 0.0833 0.3333 0 0.0833 0.0417 0.0833

0.0833 0.0417 0.0833 0.3333 0 0.0833 0.2917 0.0833

0.0417 0.0833 0.0417 0.1667 0 0.2917 0.0833 0.2917

B I Q R

3) , , f

ff

B

D

1 2 3 6 7 10 11 12f f f f f f f f f 10 0 0 0 0 0 0T T

B

4

5

8

9

f 0.2917

f 0.0833f f

f 0.0833

f 0.0417

D BB

0.292

0.083.

0.083

0.042

Markov chains x

When starting at node 4, the expected payoff of the game is 0.292; when starting at

node 5 and node 8, the expected payoff is 0.083; when starting at node 9, the expected

payoff is 0.042.

(b) For figure 2-1-2,


Figure 2-1-2 Figure 2-5-3

The canonical form of the transition matrix is

1 0 0 0 0 0 0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0 0 0 0 0 0

0 0 1 0 0 0 0 0 0 0 0 0 0 0

0 0 0 1 0 0 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 1 0 0 0 0 0 0 0 0

0 0 0 0 0 0 1 0 0 0 0 0 0 0

0 0 0 0 0 0 0 1 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 0

1/ 4 1/ 4 1/ 4 0 0 0 0 0 0 0 0 1/ 4 0 0

0 1/ 4 0 1/ 4 0 0 0 0 0 0 0 1/ 4 1/ 4 0

0 0 0 0 1/ 4 0 0 0 0 1/ 4 1/ 4 0 0 1/ 4

0 0 0 0 0 1

P

/ 4 1/ 4 1/ 4 0 0 1/ 4 0 0 1/ 4

0 0 0 0 0 0 1/ 4 0 1/ 4 0 0 1/ 4 1/ 4 0

1/ 4 1/ 4 1/ 4 0 0 0 0 0 0

0 1/ 4 0 1/ 4 0 0 0 0 0

0 0 0 0 1/ 4 0 0 0 0

0 0 0 0 0 1/ 4 1/ 4 1/ 4 0

0 0 0 0 0 0 1/ 4 0 1/ 4

R

,

0 0 1/ 4 0 0

0 0 1/ 4 1/ 4 0

1/ 4 1/ 4 0 0 1/ 4

0 1/ 4 0 0 1/ 4

0 0 1/ 4 1/ 4 0

Q

1

0.2697 0.2921 0.2697 0.0225 0.0787 0.0112 0.0337 0.0112 0.0225

0.0225 0.3160 0.0225 0.2935 0.0899 0.0843 0.1278 0.0843 0.0435

0.0787 0.1685 0.0787 0.0899 0.3146 0.0449 0.1348 0.0449 0.0899

0.0112 0.0955 0.0112 0.0843 0.044

B I Q R

9 0.2921 0.3764 0.2921 0.0843

0.0225 0.0660 0.0225 0.0435 0.0899 0.0843 0.3778 0.0843 0.2935

1 2 4 5 8 9 12 13 14f f f f f f f f f f 111101010T T

B ,


3

6

7

10

11

f 0.8764

f 0.8230

f ff 0.5056

f 0.7865

f 0.3230

D BB

.

0.823

0.787

0.876

0.506

0.323

Markov chains x

2.5.4 Summary

In order to find the approximate solution to a Dirichlet problem in two dimensions, we

use two methods — Monte Carlo method in section 2.2 and relaxation method in section

2.3. In order to find an exact solution to two dimensional Dirichlet problem, we use the

method of linear equations in section 2.4 and method of Markov chains which may be

viewed as a more complicated version of the method of linear equations.

Since we consider the case in our example with boundary values 1 or 0, the expected

final payoff is the probability p x that the walker ends up at a boundary point with value

1. In addition, we have proved that p x v x in section 2.1.4. All the above four

alternative methods can determine the voltage and current in our networks.

2.6 Random walks on more general networks

So far, the networks we studied are special networks since they have unit resistors. In

this chapter we will introduce general resistor networks, and consider what it means to

carry out a random walk on such general network. In the random walk model, we will study

our electric network in terms of the random walkers, ants, performing a random walk on a

graph. We sometimes refer to the random walkers as “ants” or “particles” in electric

network.

2.6.1 Random walks on a graph[12]

Given a graph and a starting point, we select a neighbor of it at random, and move it to

this neighbor; then we select a neighbor of this new point at random, and move to it, etc.

The random sequence of points selected this way is a random walk on the graph.

There is not much difference between the theory of random walks on graphs and the

theory of finite Markov chains; every Markov chain can be viewed as random walk on a

directed graph, if we allow weighted edges. Similarly, time-reversible Markov chains can

be viewed as random walks on undirected graphs.


Random walks on more general, but finite graphs have received much attention. The

theory of random walks is very closely related to a number of other branches of graph

theory. Basic properties of a random walk are determined by the spectrum of the graph, and

also by electrical resistance of the electric network naturally associated with graphs.

2.6.2 Some language from graph theory[17]

A graph[14][16] is a finite collection of nodes with pairs of nodes connected by edges. It

is convenient to label the nodes using the integers. If it is possible to go between any two

nodes by moving along the edges, the graph can be called connected.

Let be a connected graph, where is the set of nodes (or vertices) and

is the set of edges. We assign to each edge

,G V E V

E xy a resistance xyr . The conductance of an

edge xy is 1/xyC

P

xyr . We define a random walk on to be a Markov chain with

transition matrix given by

G

xyxy

x

CP

C , where x xyC C

y .

2.6.3 Probabilistic interpretation in electrical network

Assume that a network of resistors are assigned to the edges of a connected graph. A

one-volt battery is applied across two points and , establishing a voltage and

. In this section, we will find the voltages

a b 1av

0bv xv and the currents xyi in the circuit. We

use the following probabilistic interpretations of electrical network theory.

a. The probabilistic interpretation of voltage:

As above we interpret the voltage as a “hitting probability”. If two points x and

are connected by a resistance of magnitude

y

xyr

, then by Ohm’s law, the current through the

resistor equals /xy x y xy xi v v r v v y Cxy . Note that we have xyi iyx . By

Kirchhoff’s Current Laws, the total current flowing into any point other than or is 0.

So that for

a b

0 ,xyy

i x a b . This will be true if x yy

v v C 0xy . And this equation

can also be written as /x xy y xyy y

v C v C x xy x yy

C C v xy yy

P vv . Thus the

voltages have the property that x xyy

v yP v for ,x a b . It means that voltage xv is

harmonic at all points except points and b . a


Let xp be the hitting probability that the walker starting at x will reach point

before point . Then

a

b xp is also harmonic at all points ,x a b . In addition, and v p

have the same boundary values and 1a av p 0b bv p . If we make and b

absorbing states, can be modified as an absorbing Markov chain

a

P P . And and v p

are both solutions to the Dirichlet problem for the Markov chain with the same boundary

values. Thus, . In conclusion, the voltage v p xv at any point x represents the

probability that a walker starting from x will return to before reaching b when a

unit voltage is applied between a and b , mak 1a

a

ing v and 0bv .

We could assume an arbitrage voltage instead of a unit voltage between points

and . We can replace the “hitting probability”

av a

b xp by the expected payoff in the game in

which the walker starting at x , is paid if is reached before . av a b

b. Probabilistic interpretation of current:

We first look at the process of electrical conduction before we interpret the current①:

positively charged particles enter the network at a source and walk randomly from

point to point until they reach a sink , where they leave the network. We hypothesize

that the current

a

b

xyi is proportional to the expected number of movements along the edge

from x to , where movements from to y y x are counted as negative.

Let xu be the expected number of visits to state x before reaching the sink . Then b

x y yu xy

u P for ,x a b and . We will have 0bu /x y xy x yy

u u P C C since

x xy xC P y yC P . It means /x x xCv u is harmonic for ,x a b , where xu can be

understood as “charge” and xC as “capacitance” in physics. We also have and

.

0bv

/a av u aC y x xy y xyxxy x yi v v xy

uC xy x

x y y

u u CC u

C C

xyP y yxu P

x

C u

C C represents

the current flows from x to y . Here, x xyu P is the expected number of times the walker

will go from x to y and is the expected number of times he/she will go from y yxu P y


to x . In this case, /x xv u Cx solves a discrete Neumann problem, that is,

. Thus x y xyy

v v C b

e

1

1

0

if x a

if x

otherwis

x v is harmonic on \ ,D V a b , i. e. outside

sources and sinks. Note that the uniqueness principle in this case holds “up to a constant”.

In conclusion, if there is a unit current flows into point and flows out of point ,

the current

a b

xyi is equal to the expected net number of times the walker, starting at and

wandering until he/she arrives at , will pass along the edge from

a

b to . yx

Note ①: Actually, the current interpretation makes most sense in relation to a Neumann

problem, instead of the Dirichlet problem so far we discussed.

c. Effective resistance and the escape probability:

a axxi iImpose a voltage at and av v a 0bv . A current will flow into the

circuit from the outside source. The current depends on the overall resistance. So we define

the effective resistance between and by effr a /a ar v ib eff . The effective

conductance is . When 1/f eff aC r /ef ai v 1av , we will have eff aC i , which means the

effective conductance equals the total current flowing into . This current is

, where

ai

y aC

a

ay y ay

P v C p

a a yi v v v a aC / 1y av Cay ay y

C C esc escp is

the probability, starting at , that the walk reaches before returning to . Hence,

, so that

a b a

ef a escC C pf eff

esca

Cp

C

a

s

.

2.6.4 Random walk model

In an electrical network we have studied above, the positively charged particles enter

the network at point and wander around until they arrived at point , where they leave

the network. The walker will keep on walking if he/she returns to before reaching b .

b

a

We can picture the random walker in a electrical network as ants foraging. Ants prefer

to take paths with short length and keep on walking until it reaches the target. Ants enter a

network at a source and they want to find the path through a network to a sink . We t


constantly inject ants into the source and take the ants away from the sink. The ants walk

independently throughout the network until they find the sink where they disappear.

In order to make the notations clear, I make a comparison table below. The table 2-6-1

shows the notations we will use in our ants’ random walk model in this section and the

following reinforced random walk model in section 3.2.

Table 2-6-1 Comparison between two random walkers

Random

walker

Interpretation

Electrical particles Ants

Voltage

A unit voltage applied between a

and b : 1av and 0bv

xv : The probability that particle

starting from x will return to a

before reaching b .

/x xv u C x . (Dual interpretation)

xu : Charge

xC : Capacitance

For the source s , 1sv and

for the sink t , 0tv .

iv : The probability that an ant

at node i arrives back at the

source s before arriving at

the sink t .

What the voltage tells us is

how far an ant has to go to get

to the sink t .

/i iv u Ci

: The expected number of iu

visits to state before i

reaching sink . t

Current

xyi : The expected net number of

times that a particle, starting at a and walking until it reaches b , will move along the edge from x

to y .

xy xy x yi C v v . (Dual

interpretation)

ijI : The expected rate at

which ants pass from i jto

minus the expected rate ants

pass from j to i .

What the current tells us is how many ants are moving in a particular direction.


Resistance

The effective resistance between

a and is b /eff a ar v i .

The effective conductance is

1/ /eff eff a aC r i v .

Each edge between and i j has

a resistance proportional to the

length of the connection . ij

Probability

Escape probability is the probability,

starting at , that the walker reaches a

b before returning to . a

1/

1/a

eff effesc

a ax E

C rp

C r

x

,

where is the set of nodes which aE

are connected to node . a

When an ant arrives at a node , i

the probability that it moves next

to node j , is

1/

1/i

ij

ikk E

P go fromi j

,

where is the set of nodes iE

which are connected to . i

We make some assumptions for the model: 1) In the graph, each edge between two

nodes and i j has a resistance, which is proportional to the length of the connection

edge (In the random walk model, we use to denote resistance); 2) Ants prefer to

travel down the edges with lower resistance; 3) Ants are constantly injected into the source,

and they are taken away once reach the sink. Thus, source is always occupied and sink

is always empty.

ij ij

s

t

When an ant arrives at a node , the probability that it moves next to node i j , is

1/

1/i

ij

ikk E

P go fromi j

(3.1)

where is the set of nodes which are connected to . An alternative but macroscopic

equivalent description: If we choose one edge of the network at each time step, the

probability that choosing edge is given as:

iE i

ij

1/

1/ij

ee E

(3.2)

where is the set of all edges in the graph. E


We swap the ants on the nodes of the chosen edge. If is empty and i j is occupied

then j becomes empty and becomes occupied. If both nodes are occupied or empty

then their status is unaltered.

i

Let’s construct a graph (figure 2-6-2) and let a number of ants move on this graph:

Figure 2-6-2: Flow of stacked ants between edge in a network ij

a) The node can be seen as a building which has n floors (levels). Each floor

(level) can only contain one ant. Each node contains a maximum of n ants;

b) We denote the number of ants on node i and j as iN and jN . Let the

ants place on top of each other from the first level, then /iN n can be seen as

the level to which the site i is filled, i.e. / 6 / 8iN n

in figure 2-6-2;

c) We pick one edge ij at random on each time step as in equation (3.2), then

pick a level at random and swap any of the ants at that level;

d) The probability that there is a change in the levels of ants on edge ij is

/i jN N n . If both nodes are occupied or empty on the level we pick, no flow

exists on that level;

Thus, the probability that there is a flow of ants between and i j on a given time

step is proportional to

i

ijij

N NI

j (3.3)


where iN N j can be think of as the voltage or potential difference. The current is equal

to the potential difference divided by the length of path of fixed resistance per unit length.

In this random walk model, the ants can not be said to be solving a problem because

they are not aware of the potential difference and current they are generating. However,

based on this model, we can make some modification to make it solve a problem. If we

make our ants respond to the current they generate, we can allow them to iteratively

improve the path they take through the network. The ants which take local information

about the flow of and number of ants around them can build solutions to problems such as

finding the shortest path between a source and a sink. We will describe the advanced model

— reinforced random walk model in section 3.2.

3. Shortest path problems

In section 2.6.4, we described random walk model where ants moving around the

network do not modify their path. But we can make a modification based on this model by

allowing pheromone conductivity. We call this new revised model current reinforced

random walk model. In new model, ants can update the conductivity in response to the

current they generate so that we can allow them to iteratively improve the path they take

through the network. The path that ants reinforce most strongly is the shortest path.

In this chapter, I first study Ant Colony Optimization. Although ant algorithms and

ACO can be applied to solve combinatorial optimization problems such as travelling

salesman problem, they can not solve our shortest path problem. In the second and third

part of this chapter, we will study current reinforced random walk on the graph. In chapter 4,

we will introduce Physarum solver which is the “mean-field” version of current reinforced

random walk.

3.1 Ant colony optimization

Ants have inspired a number of methods and techniques. The most studied and the

most successful among these techniques is the general purpose optimization technique

known as ant colony optimization (ACO) [6]. ACO is inspired by the foraging behavior of

ant colonies. The ants lay pheromone on the path they have taken in order to mark some

favorable path that should be followed by members coming after them. ACO exploits a

similar mechanism for solving optimization problems.


3.1.1 Ants’ foraging behavior and optimization

The visual perspective faculty of many ant species is only rudimentarily developed and

there are ant species that are completely blind. One of the problem studied was to

understand how these almost blind ants can establish the shortest paths leading from the

colony to the food sources and come back. An early research on ants’ behavior gave us an

important insight that most of the communication among individuals, or between

individuals and the environment, is based on the use of chemicals called pheromones

produced by the ants. The trail pheromone is a specific type of pheromone that plays a

particularly important role in social life of ants. A moving ant lays this kind of pheromone

on the ground thus marking the paths from food to the nest by a trail of this substance. By

sensing pheromone trails foragers can follow the path to food discovered by other ants. This

collective trail left by other is the inspiring source of ACO.

3.1.2 Double bridge experiments

The foraging behavior of ants is based on indirect communication mediated by

pheromones[3]. Ants deposit pheromones on the ground so that they form this way a

pheromone trail between food sources and the nest. Other ants can detect the pheromone

and tend to follow, probabilistically, paths where pheromone concentration is higher. Ants

are able to transport food to the nest in an effective way through this mechanism.

The pheromone laying and following behavior of ants has been investigated. One well

known experiment named “double bridge experiment[1] [6]”, using a double bridge

connecting a nest of a colony of ants to a food source. Experiments were ran by varying the

ration between the length of the two branches of the double bridge, where

was the length of the longer branch and

/lr s l

s the length of the shorter one.

In the first experiment the bridge had two branches of equal length (figure 3-1-1).

Initially, there is no pheromone on two branches so that the ants do not have a preference

and they randomly select any of the branches. Due to random fluctuation, one of the two

bridges presents a stronger pheromone concentration than the other after some time,

therefore, attracts more ants. This will lead to a further amount of pheromone on that bridge

so that the whole colony converges towards the use of the same single path after some time.

1r

In the second experiment, the length ration was set to 2r (figure 3-1-2), so that the

long bridge was twice as long as the short one. The stochastic fluctuations in the initial

choice of a bridge are much reduced. This mechanism plays an important role: the ants

choosing the short bridge by chance are the first to reach the nest. The short bridge receives


pheromone earlier than the long one so that this fact increases the probability that further

ants select it rather than the long one.

Figure 3-1-1: Branches have equal lengths Figure 3-1-2: Branches have different lengths

Pheromone evaporation is too slow which could favor the exploration of new paths:

the lifetime of pheromone is comparable to the duration of a trail, which means that the

pheromone evaporates too slowly to allow the ant colony to “forget” the suboptimal path to

which they converged so that the new and shorter one can be discovered and “learned”. [6]

3.1.3 ACO for the Travelling Salesman Problem

The TSP[3] [6][11][19] is the problem of a salesman who starting from his hometown,

wants to find a shortest tour that takes him through a given set of cities and then go back

home, visiting each city only once. The TSP can be represented by a complete weighted

graph with being the set of nodes representing the cities, and being

the set of edges representing connections between any two cities. The set fully connects

the components . The goal in the TSP is to find a minimum length Hamiltonian circuit of

the graph. Note that a Hamiltonian circuit is a closed path visiting each of the

,G V E

V

V E

E

n V

nodes of only once. G

ACO is an iterative algorithm. We simulate a number of ants moving on a graph. The

pheromone trail[3] is associated with every edge so that it can be detected and modified by

ants. Each ant builds a solution by walking from node to node on the graph. The ant will

not visit any node that it has visited. At each step of solution construction, an ant selects the

following node according to a stochastic mechanism based on the pheromone. In figure

3-1-3, the ant stands in node , the following node will be stochastically selected among

the unvisited ones. If

i

j has not been visited before, it can be selected with a probability

that is proportional to the pheromone associated with edge ,i j . We modify the pheromone

values at the end of an iteration based on the quality of solution constructed by ants so that

they could construct solutions similar to the best ones previously constructed.


Figure 3-1-3: An ant in city chooses the next city to visit as a function of the pheromone values iijD and of the

heuristic values on the edges connecting city to the cities 1/ ij i j the ant has not visited yet.

3.1.4 The Ant Colony Optimization Metaheuristic

A metaheuristic[3] is a general algorithm that could be used to different optimization

problems with relatively few modifications to make them adapted to a specific problem.

ACO is a metaheuristic in which a colony of ants cooperate in finding good solutions to

discrete optimization problems. Ant builds a solution by walking on the fully connected

construction graph ,CG V E . The set of all possible solution components is denoted by

. The graph can be obtained from in two ways: components may be represented by

nodes or by edges. Ants move from node to node along the edges and lay a certain amount

of pheromone on the components. The pheromone amount

C C

D depends on the quality of

the solution found by ants. The following ants use the pheromone information as a guide.

The ACO metaheuristic[3] is shown in Algorithm below. After initialization, the

metaheuristic iterates over three procedures. At each iteration, a number of solutions are

constructed by the ants; then these solutions are improved through a local search (optional);

finally the pheromone is updated.

Algorithm[3] [6]: The ACO Metaheuristic

Set parameters, initialize pheromone trails

while (termination condition not met)

Construct Ant Solutions

Apply Local Search % optional

Update Pheromones

End

(1) Construct Ant Solutions: The process of constructing ant solutions can be

considered as a walk on . Ants move by applying a stochastic local decision that ,CG V E


makes use of pheromone trails and heuristic information so that they build solutions to

optimization problem. Once an ant has built a solution, it evaluates the solution that will be

used in the Update Pheromones procedure to decide how much pheromone to lay.

(2) Apply Local Search: Once solutions have been constructed, it is common to

improve the solutions through a local search before updating the pheromone. However, this

procedure is optional.

(3) Update Pheromones: Updating the pheromone aims at increasing the pheromone

values associated with good solutions and decreasing those associated with bad solutions.[6]

This process can be achieved by two steps: (i) Increasing the pheromone levels: The deposit

of new pheromone increases the probability that those components which produced a good

solution will be used again by future ants. (ii) Decreasing pheromone values by pheromone

evaporation: Evaporation is a useful form of forgetting. It avoids a too rapid convergence of

the algorithm toward a suboptimal region.

3.1.5 Ant System (AS) [3][20][21]

On one tour, each ant leaves pheromone quantity given by , where is a

constant, is the length of its tour. So that more pheromone left on shorter tours and less

pheromone left on longer tours per unit length. In addition, the existing trails are reduced

by a factor named evaporation rate

/ kQ Q

k

before new pheromone is laid since pheromone

evaporation. At each iteration, the values of pheromone are updated by all the ants that

have built a solution itself

m

[1] [5]. The updating of the trail level associated with the edge

connecting to

ijD

i j , is updated as follows:

1

1m

kij ij ij

k

D D

D (3.4)

where

/ ,

0 ,k kij

Q if ant k used edge i j in its tourD

otherwise

, (3.5)

where

t iteration counter

0,1 evaporation rate, parameter of the reduction of ijD

m number of ants


kijD increase of trail level (the quantity of pheromone laid) on edge

caused by ant

,i j

k

ijD total increase of pheromone trail on edge ,i j

Q quantity of pheromone per tour laid by an ant, is a constant Q

k tour length of ant k

In the construction of a solution, ants select the following city to be visited through a

stochastic mechanism[11]. The higher the probability that a city is selected is, the more

intense the trail level leading to this city is[6]. The adapted behavior is based on the trail

levels and is regulated by a parameter ; the greedy behavior is based on the visibility and

is regulated by a parameter . When ant stays in city and it has so far constructed

the partial solution , the probability of going to city

k i

ps j is given by:

,

0 ,

pil

ij ij pijk

il ilc N sij

Dif c N s

Dp

otherwise

(3.6)

where

1

ijij

(3.7)

intensity of pheromone trail between cities and ijD i j

from city i visibility of city jij

parameter to regulate the influence of ijD

parameter to regulate the influence of ij

pN s set of cities that have not been visited yet; the set of feasible components

distance between cities and ij i j

Th ts have completed a tour. [5] The set of

citie ne

is selection process is repeated until all an

s to be visited is reduced by one city in each of iteration t . By keeping on visiting, o

city is left finally and it is selected with probability 1ijp . For each ant the length of the


tour generated is calculated and the best tour found so far is updated. [3] The next iteration

1t can be started based on updated pheromone levels.

Algorithm: The Ant System

Initialization

for 1t : number of iterations

for 1k : m

repeat until ant k has completed a tour

select the c ty i j to be visited next

with probability ij p given by equation (3.6)

calculate k of the tour generated by ant k

update ijD on l edges according to equation ( .4al 3 )

End

3.1.6 Simple ant colony optimization (S-ACO)

lks down an edge with probability According to equation (3.6), ant at node i wa ij

1/,

ij ijD

1/ij ijk

D

wher omone in the ed

(3.8)

e D is the concentration of pher ge connecting i to j , is the ij ij

length of the edge ij . Here we consider the simplest case 1 where the ants

employ the same resistance rule as in equation (3.1), i.e. the ity that an ant at n

i travelling to

probabil ode

j is

/

/ij ij

ij iji

D

D

(3.9)

rease in pr D t in the ant system is assumed to inc oportion to the number of ant on that ij

edge and to decrease with evaporation rate . The pheromone update rule is

1ij ij ij jiD t t t t t t D (3.10)

next ant travels from to where =1ij t if the i j . The probabi

determined by equation (3.9).

lity that an ant moves is


Though ants algorithms and Ant colony ptim zation can be applied to solve the

solve our shortest path problem

O i

combinatorial optimization problems like the travelling salesman problem, they can not

. It is necessary to introduce reinforced random walk model

whic

3.2.1 Description of current reinforced random walks

nt as in the description of ant stacks on each

node in section 2.6.4. Now, we allow the ants to modify the conductivity of the network

they

h will always converge to the optimal solution of the shortest path problem.

3.2 Current reinforced random walks

We keep the description of ant moveme

move in. Let ijD t be the conductivity of an edge at time t .

Figure 3-2-1: Ants are allowed to modify pheromone conductivity

The probability that an ant is moved from to i j is proportional to

i j ij

ij

N N D t

. Hence, an edge is chosen at each e step with probability proportional

to

tim

| |i j ij

ij

N N D t

.

from

We assume that every edge is independent and that the time until the next ant moves

to i j is exponentially distributed with rate:

i j ij

ijij

N N D t

I t (3.12)


The input and output currents are fixed to constants 0sI and 0tI . The time interval

t until the next edge of the network is updated is taken from an exponential distribution

with the mean value equals

1 (3.13)

i j e

e Ee

N N D t

We update the conductivity in response to current between and i j so that on the

chosen edge

ij :

1ij ij ijD t t t D t t (3.14)

where is , here, is the expected value of ij t q q ij t .

e exp v Th ected alue of ij t equals ij i j

ijE ij

D N Nq t q

.

This gives a “mean-field” equation as follows:

ij i j

ij i j

ijij

Nq t

D N ND q

dt t t

. (3.15)

On all other edges , we have

D N

ij ij ij ijij

dD D t t D tD

e 1eD t t t D t e . On each time step, the

conductivity decreases slightly, but the conductivity is increased if an ant crosses. This is a

curre m

3.2.2

nt reinforced rando walk.

I will implement the simulations based on Poisson process: Exponential distribution

and Poisson distribution.

Pheromone Rule and Ant Moving Rule

Figure 3-2-2


iN : Ant number

: Pheromone conductivity

: Resistance

: Chosen edge rate

To each path of the graph

ijD

ijl

ijq

,V E , we ij G associate a variable called

ph omone trails are read and written by the ants. Pheromone evaporation

can anism that avoids quick convergence of all the ants

toward a suboptimal path. The decrease in pheromone intensity favors the exploration of

di g the whole search process.

ach time step:

ijD

eromone trail. Pher

be seen as an exploration mech

fferent paths durin

Pheromone change at e

1 ,ij ij ijD t t t D t t 0 1

, 1 1 , 0D t D t t 1when t 1ij ij ij

1

0ij

Ant walks on this wayt

Ant walks on another way

Only one ant walks at each time step in this model.

Chosen edge rate:

0.001D ij

i j

ij ijij

N Nq D

ij

ij

q

qOne edge is chosen at each step with probability

j

. An ant moves from more ants

node to less ants node. Ant returns to start if it arrives at goal.

1, 1i i j j iN N N N N N j

1, 1i i j j iN N N N N N j

3.2.3 A Simple Explanation


Given:

1ij ijD

1 10N , 2 8N , 3 3N , 4 0N

Solution:

According to the edge choosing rate: i j

ij ijij

N Nq D

1 212 12

12

10 8 2N N

q D

1 313 13 10 3 7

N Nq D

13

2 424 24

24

8 0 8N N

q D

3 434 34

34

3 0 3N N

q D

2 323 23

23

8 3 5N N

q D

2 7 8 3 5 25ijq

5

25, 2

3

8 7

3 4

N

Edge 2-3 is chosen withN

2

4

8 7

10 11

N

N

8

25,Edge 2-4 is chosen with (node 4 is goal)


3.2.4 Exponential distributed simulation

ma ix shows the connected edges and lengths in the graph

% initial pheromone conductivity

tor f initial ant numbers on each node

% the expected value of

...G % the tr

...D

...N % vec o

...q ij

...inRate % the inflow rate in the source

outRat ...e % the outflow rate in the sink

% evaporation rate ...

……

— for

pick e

1:k S % iterate from 1 to S

ij i j

ij

Ddge at random with probabilistic rate

t N N

ck t ion 1

expij i j

ij

D t N N

pi from exponential distribut

% for each flip, transport one ant along edge ij from high to low

— if , an ant moves from to i jN N i j

1i iN t t N t

else

1j jN t t N t

ijD t t D t t ij ij

1i iN t t N t

1j jN t t N t

ji ji ijD t t D t t

—

,

— end

end

1 ,ij ijD t t t i j D t

t t t


— end

mentation in Matlab

In Monte Carlo methods, repeated experiments with stochastic transition components

are run to collect data about the statistics of interest. Based on Monte Carlo methods, we

will firs plement the Exponential distributed simulation in an simple graph, then in a

much more complicated graph. Matlab code could be found in the appendix 3-2(A) and

e graph with 4 nodes in figure 3-2-2, see the code in

App

3.2.5 Imple

t im

3-2(B).

(A) First, we implement on th

endix 3-2(A).

Figure 3-2-2

Show connection and length in a matrix:

1) Outcome:

0 1 6 0

1 0 2 4

6 2 0 1

0 4 1 0

G

2) We plot the number of ants on each node:


0 0.5 1 1.5 2 2.5

x 104

0

5

10

15

20

25

30

35

t

n

number of ant on each node,(number of node is4)

The deep blue line represents the number of ants at node 1; the green line represents

the n umber of ants at node 2; the red line represents the number of ants at node 3; the light

blue represents the number of ants at node 4. The number of ants on each node tends to be

equilibrium after iterations. It fluctuates near a value in an interval.

3) We plot the total amount of pheromone:

0 0.5 1 1.5 2 2.5

x 104

0

5

10

15

20

25

30

t

tota

lD

total amount of pheromone


The blue line represents the total quantity of pheromone. It tends to be equilibrium

after iterations. It fluctuates near a value in an interval.

4) Conclusion:

In the outcome matrix of conductivity , we find that elements and

(the elements in the red boxes) reinforce the pheromone. So there is

, as the green path in the graph below.

D 12D ,

The length of the shortest path is

23D

the shortest path

34D

1 2 3 4

12 23 34 1 2 1 4 .

(B) Now we implement on a much complicated graph with 10 nodes in figure 3-2-3.

See the code in Appendix 3-2(B).

Figure 3-2-3



0 2 0 0 0 0 1 0 0 0

2

0 2 0 3 0 0 0 0 0

0 2 0 4 0 0 0 0 0 0

0 0 4 0 0 1 0 0 0 6

0 3 0 0 0 1 5 0 0 0

0 0 0 1 1 0 0 0 1 2

1 0 0 0 5 0 0 4 0 0

0 0 0 0 0 0 4 0 3 0

0 0 0 0 0 1 0 3 0 2

0 0 0 6 0 2 0 0 2 0

G

1) Outcome:


Th axis represents the iteration time; the y e x axis represents the number of ants.

In the figure below, the traces with different colors represent the numbers of ants on


different nodes. The number of ants on each node tends to be equilibrium after a large

number of iterations. It fluctuates near a value in an interval.

0 0.5 1 1.5 2 2.5

x 104

0

10

20

30

40

50

60

t

n

number of ant on each node,(number of node is10)


0 0.5 1 1.5 2 2.5

x 104

0

5

10

15

20

25

40

30

35

t

tota

lD

total amount of pheromone


The blue line represents the total quantity of pheromone. It tends to be equilibrium

after iterations. It fluctuates near a value in an interval.

4) Conclusion:


reinforce the pheromone. The shortest path is ath

raph below. The length of the shortest path is

D 12D ,

10

6,10

25D ,

, as the green p

2 3

56D

1

6,10D

in the g

1 2 5 6

12 25 56 2 8 .


3.3

istributed simulation we have studied in section 3.2.4 and 3.2.5, we

picked up which follows exponential distribution[14]. In order to build a new algorithm,

we ma ber of ants going on the edge connecting node and

An efficient algorithm based on Poisson distribution

In exponential d

t

ke the num i j during a fixed

time interval follow Poisson distribution. It is desirable to know how these two algorithms

come and what the difference is between them, even it is interesting to study which one is

better. First, we introduce some knowledge of Poisson process in section 3.3.1. Then in

section 3.3.2, we will introduce event-driven approaches that gave inspiration in our two

algorithms. Finally, in the last two sections, we show our algorithm of Poisson simulation

and the implementation of it.

3.3.1 Some knowledge of the Poisson process[14]

The Poisson process is a collection : 0N t t

e t

of random variables, where

is the number of events that have occurred up to tim The number of events between

time and time is given as

N t

.

a b N a

e process. Consequences of

is a Poisson dist

N b

tim

N t

and has a Poisson distribution. The

Poisson process is a continuous- this definition include: i)

The probability di tion of ribution; ii) The probability

distribution of the waiting time until the next occurrence is an exponential distribution.

The exponential distribution is a family of continuous probability distributions. It

describes the time between events in a Poisson process, i.e. a process in which events occur

continuously and independently at a constant average rate. It describes the time for a

continuous process to change state. In the exponential distributed simulation of our CRRW

model, the edge is chosen by ants independe tic rate.

sen to update is taken from an

exponential distribution.

The Poisson distribution is a discrete probability distribution that expresses the

probability of a number of events occurring in a fixed period of time if these events occur

with a known average rate and independently of the time since the last event. The Poisson

distribution can also be used for the number of events in other specified intervals such as

distance, area or volume. One example that may be modeled as a Poisson distribution is the

number of phone calls arriving at a call centre per minute. Of course, in our CRRW model,

during a fixed time interval, the number of ants walking along the edge connecting to

stribu

ntly and with a probabilis The time

interval t until the next edge of the network is cho

i j


Pois models as a Poisson distribution. The notation of Poisson distribution is , where

the

If

is a positive real number, equal to the expected number of occurr r during

g

the number of arrivals in a given time interval

ences that occu

iven interval.

0, t follows the Poisson distribution,

twith mean , then the lengths of the inter-arrival times follow the exponential distribution,

with mean 1/ .

3.3.2 Event-driven approaches

Event-driven methods require explicit consideration of events. We need to consider

numbers of events that can occur, each causing the numbers in the relative classes to

increase or decrease by one, i.e. in the network ant chooses the edge at random with rate

| |i j ijN N D t. Result: 1i iN N and N

ij

j j i j

the next event, based on the cumu The time and numbers

to iterate the model through time (box 3-3-1).

Event-driven approaches: Gillespie’s Direct Algorithm

( if

( if This schem

lative rates of all possible events.

in each class are then updated according to which event is chosen.

1j jN

one updates.

i jN N

e estim

) or

ates the tim

This process is repeated

1i iN N

e until 1N N N N ); pheromand

[28]

1. Label all possible events as 1,..., nE E .

2. For each event determine the rate at which it occurs, 1,..., nR R .

3. The rate at which any event occurs is 1

n

total mmR R

.

4. The time until the next event is 1

1log

total

t randR

.

5. Generate a new random number . Set 2 2 totalrand P rand R .

6. Event p occurs if 1

1 1

p p

m mm m

R P R

7. The time is updated now, t t t , and event p is performed.

8. Return to Step 2.

Box 3-3-1: Gillespie’s direct algorithm


However, this method has low simulation times for large population ter

mber of different event types. We will use the simple SIS model to explain the algorithm

(in box 3-3-2).

sizes. Af

modification, a new algorithm comes up that is significantly faster when there are a large

nu

ve AEvent-driven approaches: Effecti lgorithm[28]

1. Let the time interval t , be “small” and fixed.

2. Let RM t TM t and represent the number of transmission and recovery events by

3. D

time t .

efine ,i i iM M t t M t i T R , then we will have

1| ,T

XYP M X Y t o t

N

, 1|RP M Y Y t o t .

t , the incr i4. For small ements M are a y Poiss hat: pproximatel on, so t

T

XYM Poisson

N

t

, RM Poisson Y t .

5. Then we can update:

T RX t t X t M M , T RY t t Y t M M .

6. Update the tim

. Return to step 4.

e:

t t t

Box 3-3-2: an lgo effective a rithm

From b tion of a orithm based

on Poisson distribution is much more efficient than that based on exponential distribution.

So that I wo ted algorithm of our reinforced random

walk model and implement the simulation.

3.3.3 Poisso

% the matrix shows the connected edges and lengths in the graph

ity

% vector of initial number of ants on each node

ox 3-3-1 and box 3-3-2, we get to know that the simula lg

uld like to introduce the Poisson distribu

n distributed simulation

...G

...D % initial pheromone conductiv

...N

... a , it should be less an zero

article, it is a fixed parameter which can be changed based

% on the sim

% ev poration rate th

...t

...t % fix the time interval

...Q % charge per p

ulation


—fo

outflow, ants go out of the sink

% for each flip, tra

r 1:ts S

...sN % inflow, ants come into the source

...tN %

nsport ants along edge ij from high to low

—for each ij

( , , ) , ,ijij

DP o

ij

P i t N ts i N ts j sign N ts i N ts jG

hat if )

% the number of ants going from node i to j during a fixed time interval follow

t % update the pheromone conductivity

—end

(Note t 0, 0ij ijG then P

t

% Poisson distribution

1ij ij ijD t D Q P

—for 1:k length G

% update the numbers of ants after all the ants’ currents are known

, ,: :,ts k sum P k sum P k 1,N ts k N

—end

—en

ulation in an simple graph, then in a

ould be found in the appendix 3-3 (C) and 3-3

(D). W

than the im ulation.

(C) First we im

Appendix 3-3(C).

t t t

d

3.3.4 Implementation in Matlab

We first implement the Poisson distributed sim

much more complicated graph. Matlab code c

e find that the implementation of Poisson distributed simulation is more efficient

plementation of exponential distributed sim

plement on a graph with 4 nodes in figure 3-2-2. See the code in


Figure 3-2-2


1) Outcome:

0 1 6 0

1 0 2 4

0 4 1 0

G

6 2 0 1

2) W ber of ants on each node:

e

e plot the num

axis represents the iteration time; the y x Th axis represents the number of ants.

umber of ants at node 1; the green line represents the

num ue

The deep blue line represents the n

ber of ants at node 2; the red line represents the number of ants at node 3; the light bl

represents the number of ants at node 4. The number of ants on each node tends to be

equilibrium after iterations. It fluctuates near a value in an interval.


0 0.5 1 1.5 2 2.5 3 3.5

x 104

0

1000

2000

3000

4000

10000

5000

6000

7000

8000

9000


0 0.5 1 1.5 2 2.5 3

x 104

0

1

2

3

4

10

5

6

7

8

9


The blue line represents the total quantity of pheromone. It tends to be equilibrium after iterations.

4) Conclusion:


reinforce the pheromone. So the shortest path is

graph below. The length of the shortest path is

D 12D ,

1 4

23D

, as the green path in the

34D

1 2 3 4

12 23 34 1 2 . It has the same

outcome as in exponential distributed simulation (A).

(D) We implement on a much complicated graph with 10 nodes in figure 3-2-3. See

the code in Appendix 3-3(C).

Figure 3-2-3



0 2 0 0 0 0 1 0 0 0 2 0 2 0 3 0 0 0 0 0

0 2 0 4 0 0 0 0 0 0

0 0 4 0 0 1 0 0 0 6

0 3 0 0 0 1 5 0 0 0

0 0 0 1 1 0 0 0 1 2

1 0 0 0 5 0 0 4 0 0

0 0 0 0 0 0 4 0 3 0

0

2 0 0 2 0

G

0 0 0 0 1 0 3 0 2

0 0 0 6 0

1) Outcome:


y The x axis represents the iteration time; the axis represents the number of ants.

In the figure below, the traces with different colors represent the numbers of ants on

different nodes. The number of ants on each node tends to be equilibrium after a large


number of iterations. It fluctuates near a value in an interval.

0 0.5 1 1.5 2 2.5 3 3.5

x 104

0

2000

4000

16000

6000

8000

10000

12000

14000


30

0 0.5 1 1.5 2 2.5 3

x 104

0

5

10

15

20

25


The blue line represents the total quantity of pheromone. It tends to be equilibrium after iterations. 4) Conclusion:


reinforce the pheromone. So the shortest path is

test path is

D

2 3

12D ,

10

25D ,

.

56D

The length of 6,10D

the shor

1 2 5 6

812 25 56 6,10 1 2 . It has the same outcome as in

exponential distributed simulation (B).

We can find that the implementation of Poisson simulation is much more efficient than

that of exponential simulation. Among the four implementations, (C) is efficient than (A)

and (D) is efficient than (B).


4. Physarum solver

Some organisms grow in the form of an interconnected network as part of their

foraging strategy to discover and exploit new resources. Such systems can adapt to their

environment continuously and balance the cost of producing an efficient network with the

consequences of limited failure in a competitive world. The slime mode Physarum

polycephalum is such an organism[27].

4.1 Introduction to Physarum

The Physarum Polycephalum[13][22] is a large, single-celled amoeboid organism[23][24]

with an intracellular structure consisting of a network of tubes[25], see the graph 4-1-1. This

organism forages for food resources in a woodland environment. During exploration, it

spreads with a relatively continuous foraging margin to maximize the area searched. The

large amoeboid plasmodium of Physarum is able to solve a maze and some other types of

geometrical puzzle, and is also able to connect multiple food locations via a smart network,

then successfully optimize survival tasks. The challenge is to extract a mathematical

algorithm for this natural computation. We could propose a mathematical model, and

consider how this primitive organism solves a shortest path finding problem[25] on the graph

by Physarum-like way[13][26].

4-1-1 Plasmodium

The Physarum solver is a simple, intuitive and highly localized mechanism for

solving shortest path problems which works as an adapting electrical network[17]. The

Physarum consists of a network of tubes. This network acts as an information highway

transporting chemical and physical signals and it also enables nutrients to circulate

throughout the organism. When food sources are presented to a starved plasmodium which


has spread over the whole surf

and concentrate on the food sources.

ace, parts of the organism are connected by only a few tubes

The path connecting these parts is the shortest path.

e

ze-solving by Physarum

d the Physarum solver[13] as a model of how Physarum

builds a minimal link between two food sources. We assume this slime mode Physarum

cons

Past experiments have shown two rules: first, open-ended tubes are likely to

disappear; second, the longer tube is likely to disappear when more tubes connect the sam

two food sources.

4.2 Mathematical modeling of ma

Tero et al. (2006) first describe

ists of a network of tubes. We illustrate this model using maze in Figure 4-2-1. Each

segment in the figure represents a section of tube.

In the graph, we make two nodes 1N and 2N be the food sources. Other nodes are

denoted as 3N , 4N , 5N and so on. There is also a rule: if there are more than one tubes

connect the same pair of nodes, we place the nodes in the center of the tubes.

4-2-1 Expression of the maze by graph( Figure from [13])

Ordinary nodes are indicated by squares and the nodes of the food sources are given by circles.

We have:

1) : the source 1N

2N : the sink

2) ijM : the tube section between nodes iN and jN

3) ijQ : the flux[26] go through the tube ijM


,ijij i j

ij

DQ p p

(4.1 )

: the length of the segment

where

ij ijM

p ip , j : the pressure at nodes and

4)

i jN N

ijI : the flow rate of protoplasm, it is given by Ohm’s law (equation (3.12)).

In equation (3.12), i j ij

ijij

n n D tI t

, note that we use ,i jn n here. in is the

number of ants at node iN and jn is the number of ants at nod . The e jN pressure ip is

proportional to n , i.e. iin

p .

5) D e ductivity of the pipe

i n

: th con

ij

| |

ij iji j

ij

ij ij

dD Dp ijp D

dt

Q D

(4.2)

This gives a deterministic ‘mean-field’ version of equation (3.14). We can replace

in equation (3.14) with its expected value, i.e. ij t ijQ , then obtain equation (4.2).

In equation (3.14), 1ij ij ijD t t t D t t

ijij

D t t D ij ijt tD t

t t ij ijdD

Dijdt dt

Here, we replace with its expected value ij t ij i j

ijij

D n nq dt q

.

ij i jijij

ij

D n ndDD q

d

□ We have

t

According to Kirchhoff’s Law, at each node j 1, 2j , the inflow and outflow must

be balanced as:

0 1,iji

j Q 2 (4.3)


for t The nodes of foo

network. At the food sources, equation (4.3) is modified by a source (or sink) term.

he source 1N and the sink 2N d sources drive the flow through the .

2ii

Q

1 0

0

0,

0,

iiQ Q

Q

(4.4)

is e f rc oing into the sink. Because the

time le f

shuttle streaming, the sources are taken to be constant. Note that is a constant.

)

and (4.4) as follows:

where 0Q th lux coming from the sou e and g

sca of network adaptation is an order of magnitude larger than the time scale o

0Q

The network Poisson equation for the pressure is derived from the equation (4.1), (4.3

0

0

1,

2,

0

iji j

i ij

Q jD

p p Q j

otherwise

(4.5)

as a basic pressure level. All2 0p ip ’s can be obtained by solving (4.5), and each

ijij i j

Dp p is also obtained.

ij

be

the sta

through the shortest path between the sink and the source remains, the other pipes are died

out then.

4.3 Biologically inspired adaptive network

al and social systems. They are a

critical part of the infrastructure needed to facilitate efficient movement of people, energy,

ation. However, m

design principles because the main motivation was to achieve high transport efficiency at

reasonable cost but with less em s tolerant to failure or

interruption. It is desirable to develop networks with greater intrinsic resilience.

e Physarum to develop a biologically inspired model for adaptive

netw grows in the form o

4-3-1) which has been subjected to successive rounds of evolutionary selection. And the

Q

In order to apply this dynamics to path-finding problem in a network, two nodes to

rt and the goal is to be the sink and the source of current 0Q , and only the pipes

[27]

Transport networks are ubiquitous in biologic

resources and inform ost networks have emerged without clear global

phasis on making system

We study th

ork development. Physarum f an interconnected network (figure


network is likely to have reached a point at which cost, efficiency and resilience are

combinatorial optimization problems. The Physarum networks showed charateristic

balanced appropriately. Such network is likely to yield reasonable solutions to

s

simi terms of cost, tra

tolerance. However, the Physarum networks self-organized without centralized control or

expl al

and simultaneous removal of redundant connections.

t the

cond

lar to those of the rail network in nsport efficiency, and fault

icit glob information by a process of selective reinforcement of preferred routes

We developed a mathematical model for adaptive network construction to emulate

this behavior. In order to describe the adaptation of tubular thickness, we assume tha

uctivity ijD changes in time according to the flux ijQ as follows:

ijij ij

dDf Q D

dt

f Q is an increasing function with 0 0f where . This equation implied that

conduc

segment. In the equation, the first term on the right side describes the expansion of tubes

in response to the flux; the second term represents the rate of the tube constriction, so

that in the absence of flow the tubes will gradually disappear.

tivity tends to decline exponentially, but is enhanced by the flux through the

f Q is given by / 1r r

f Q Q Q . r is a parameter controls the

0

( 0Q in eq

nonlinearity of feedback ( ). The selection of specific parameter combinations

uation (4.5)) and can give a network with features similar to those of both

ics of

network adaptability through ite

0r

r

Q

the Physarum system and the rail network.

The biological inspired mathematical model can capture the basic dynam

ration of local rules and produces solutions with

properties comparable to or better than those of real-world infrastructure networks. The

model has a number of parameters that allow adjustment of the benefit/cost ratio to

increase specific features such as fault tolerance or transport efficiency, while keeping

costs low.


4-3-1 Network formation in Physarum polycephalum (Figure from [27])

(A) At 0t , a small plasmodium of Physarum was placed at the location of Tokyo in a

experimental arena bounded by the Pacific coastline (white border) and supplemented w

additional food sources at each of the major cities in the region (white dots). The horizontal

of each panel is 17cm. (B -F) The plasmodium grow out from

n

ith

width

the initial food source with a

contiguous mar essively colonized each of the food s growing

margin, the s lium resolved into a netw od sources.

gin and progr ources. Behind the

preading myce ork of tubes interconnecting the fo

(This figure is from Tero’s paper [27])


5. Appendix

2-2-1 Page 8

imax=101; % number of x grid divisions

jmax=101; % number of y grid divisions

nmax=10000; % number of random walks

% boundary conditions

bottom_bc=1;

right_bc=2;

top_bc=3;

left_bc=4;

s=0; % initialize the scalar potential

for n=1:nmax

% begin the random walk at the centre of the grid

i=

t

elseif (r>=0.25)&&(r<0.5)

i=i-1; % step left

elseif (r>=0.5)&&(r<0.75)

j=j+1; % step up

else

j=j-1; % step down

end

end

% random walker reaches a side

if (j==1)&&(i~=1)&&(i~=imax)

s=s+bottom_bc;

elseif (j==jmax)&&(i~=1)&&(i~=imax)

s=s+top_bc;

elseif (i==1)&&(j~=1)&&(j~=jmax)

51; j=51;

% perform random walk until a boundary is reached

while(i~=1)&&(j~=1)&&(i~=imax)&&(j~=jmax)

r=rand;

if r<0.25

i=i+1; % step righ


s=s+left_b

elseif (i==imax)&&(j~

c;

=1)&&(j~=jmax)

ht_bc;

end

end

ndom walks

2-2-2

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Monte Carlo Solution

% describe the graph in a matrix, the internal points are 2,3,6,7,11; the boundary points are

% 1,4,5,8,9,10,12,13

%%% %%%%%%%%%%%%%%%%%%%%%%

G=z % initialize the matrix with all zero elements

G(1, ed

G(2,3)=1;G node 2 is connected with node 3 and node 6

G(3, cted with node 1, 2, 4, 7

G(4,

G(5,6)=1 e 5 is connected with node 6

G(6,2)=1;G(6,5)=1;G(6,7)=1;G(6,10)=1; % node 6 is connected with node 2, 5, 7, 10

G(7,3)=1 ; % node 7 is connected with node 3, 6, 8, 11

G(8,7)=1 ith node 7

G(9,10)= ith node 10

G(10,6)= node 10 is connected with node 6, 9, 11

G(11,7)=1 (11,13)=1; % node 11 is connected with node 7, 10,

% 12, 13

G(12,11)= with 11

G(13,11)=1; % node 13 is connected with 11

B=[1,4,5,8,9,10,12,13] % points with value 0

B1= value 1

%

I=zeros(1

v=ze

T=10000 n times

for t

s=s+rig

s=s/nmax % divide by the total number of ra

Page 9

%%%%%%%%%%%%%%%%%%%

eros(13);

3)=1; % node 1 and node 3 are connect

(2,6)=1; %

1)=1;G(3,2)=1;G(3,4)=1;G(3,7)=1; % node 3 is conne

3)=1; % node 4 is connected with node 3

; % nod

;G(7,6)=1;G(7,8)=1;G(7,11)=1

; % node 8 is connected w

1; % node 9 is connected w

1;G(10,9)=1;G(10,11)=1; %

;G(11,10)=1;G(11,12)=1;G

1; % node 12 is connected

[1,4,5,9,10] % points with

current

3);

ros(1,13);

; % iteratio

=1:T


for i

3);

while (sum(pos==B)==0) % check if hit boundary

r=rand; % choose a random number r

es are connected with i

p_choose_v=G(pos,:)/sum(G(pos,:)); % the probability of choosing

% random walk starting from node i

p_choose_v(p_choose_v>0);

= find(cumsum(p_choose)>r,1,'first');

5

)=1;G(8,9)=1;G(8,11)=1;

9,12)=1;

;

=1:13

pos=i;

I_step=zeros(1

num_connect_v=sum(G(pos,:)); % how many nod

p_ind=find(p_choose_v>0); % find the elements with positive probability

p_choose=

% the positive probability of choosing random walk starting from node i

chosen

chosen_v = p_ind(chosen);

I_step(pos,chosen_v)=I_step(pos,chosen_v)+1;

pos=chosen_v;

end

v(i)=v(i)+(sum(pos==B1));

I_step;

I=I+abs(I_step-I_step');

end

end

v=v/T

2-2-3(a) Page 10

G=zeros(12); % initialize the matrix with all zero elements

G(1,4)=1; % node 1 and node 4 are connected

G(2,5)=1; % node 2 is connected with node

G(3,4)=1;

G(4,1)=1;G(4,3)=1;G(4,5)=1;G(4,8)=1;

G(5,2)=1;G(5,4)=1;G(5,6)=1;G(5,9)=1;

G(6,5)=1;

G(7,8)=1;

G(8,4)=1;G(8,7

G(9,5)=1;G(9,8)=1;G(9,10)=1;G(

G(10,9)=1


G(11,8)=1;

G(12,9)=1;

% the internal points are 4,5.8,9.

v=zeros(1,12);

for i=1:12

% choose a random number r

um_connect_v=sum(G(pos,:)); % how many nodes are connected with i

um(G(pos,:)); % the probability of choosing random

% walks starting from node i

e_v>0); % find the element with positive probability

p_choose=p_choose_v(p_choose_v>0);

% the positive probability of choosing random walk starting from node i

chosen = find(cumsum(p_choose)>r,1,'first');


(pos,chosen_v)=I_step(pos,chosen_v)+1;

step;

;

% the boundary points are 1,2,3,6,7,10,11,12.

B=[1,2,3,6,7,10,11,12]

B1=[1] % point with score 1.

I=zeros(12);

T=10000; % starting T times random walks

for t=1:T

pos=i;

I_step=zeros(12);

while (sum(pos==B)==0) % check if hit boundary

r=rand;

n

p_choose_v = G(pos,:)/s

p_ind=find(p_choos

I_step

pos=chosen_v;

end


I_


end

end

v=v/T

2-2-3(b) Page 10

G=zeros(14)


G(1,3)=1;

(2,6)=1;

)=1;

1;

,6)=1;G(7,8)=1;G(7,11)=1;

;G(10,9)=1;G(10,11)=1;G(10,13)=1;

1,12)=1;G(11,14)=1;

B1=[1,2,4,5,9,13]; % points with score 1.

T=10000; %

(sum(pos==B)==0)

_connect_v=sum(G(pos,:));

(G(pos,:));

p_ind=find(p_choose_v>0);

p_choose=p_choose_v(p_choose_v>0);

chosen = find(cumsum(p_choose)>r,1,'first');


(pos,chosen_v)=I_step(pos,chosen_v)+1;

s=chosen_v;

G(2,3)=1; G

G(3,1)=1;G(3,2)=1;G(3,4)=1;G(3,7

G(4,1)=1;G(4,3)=1;G(4,5)=1;G(4,8)=1;

G(5,6)=1;

G(6,2)=1;G(6,5)=1;G(6,7)=1;G(6,10)=

G(7,3)=1;G(7

G(8,7)=1;

G(9,10)=1;

G(10,6)=1

G(11,7)=1;G(11,10)=1;G(1

G(12,11)=1;

G(13,10)=1;

G(14,11)=1;

% the internal points are 3,6,7,10,11.

% the boundary points are 1,2,4,5,8,9,12,13,14.

B=[1,2,4,5,8,9,12,13,14];

I=zeros(14);

v=zeros(1,14);

iteration times

for t=1:T

for i=1:14

pos=i;

I_step=zeros(14);

while

r=rand;

num

p_choose_v = G(pos,:)/sum

I_step

po


end


ge 40

%%%%%%%%%%%%%%%%%%%

ph G=(V,G), we first take an easy case. We have 4 nodes 1\2\3\4 in the graph.

the source and node 4 is the sink.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% show connection and length in a matrix

% the number of nodes

;

% set initial pheromone conductivity matrix

ial numbers of ants on each node

eromone that an ant deposits on this way during a

interval

1, avoiding a too rapid

I_step;


end

end

v=v/T

3-2 (A) Pa

%%%%%%%%%%%%%%%%%%%%%%%%

% Exponential distributed Simulation:

% Input gra

% Node 1 is

%%%%%%%

clear;

G=[0 1 6 0;

1 0 2 4;

6 2 0 1;

0 4 1 0];

N=length(G);

D=[0 1 1 0;

1 0 1 1

1 1 0 1;

0 1 1 0];

n(1,:)=[10,5,5,0]; % set init

q=0.1; % the amount of ph

% time

inRate=25; % the inflow for the source node

outRate=25; % the outflow for the sink node

lamda=0.5; % evaporation rate, should be less than

% convergence

t=0;

ts=1;


S=10000*floor(1/lamda); % iteration times

r v=1:N

T(u,v)=abs(n(ts,u)-n(ts,v)); % T is the matrix with the elements |

end

Gs=max(reshape(D.*T./G,1,N^2),0); % D.*T./G is the matrix with the elements

%

for s=1:S

T=zeros(N);

for u=1:N

fo

i jn n |

end

| |i j ijn n D

ij

=nansum(nansum(Gs)); % sG=

% ants coming in to the system

Gs=[Gs inRate outRate];

| |i j i

ij

n n D j

sG

Dt=(-1/sG)*log(rand); % exponentially distributed

G;

cumsum(Gs)<=r)+1;

n(ts,:);

n(ts+1,

else

j=ind-(i

flip an ant

if n(ts,i)>n(ts,j)

n(ts+1,i)=n(ts,i)-1;

t=t+Dt;

storet(ts)=t;

% choose one edge at random

r=rand*s

ind=sum(

n(ts+1,:)=

if (ind==N^2+1)

% ant coming in

1)=n(ts,1)+1;

elseif (ind==N^2+2)

n(ts+1,N)=max(0,n(ts,N)-1);

i=ceil(ind/N);

-1)*N;

%


n(ts+1,j)=n(ts,j)+1;

D(i,j)=D(i,j)+q; % pheromone laid by an ant

ts+1,i)=n(ts,i)+1;

j)=n(ts,j)-1;

end

-Dt*lamda)*D % quantity of pheromone after evaporation

=max(0.001,D).*(G>0); % the elements should not be less than zero

totalD(ts)=sum(sum(D));

plot(n)

nt on each node,(number of node is',num2str(N),')'])

l('t')

omone')

%%%%%%%%%%%%%%%%%%%%%%%

onential distributed Simulation:

There are ten nodes in the graph.

e and node 10 is the sink.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

10 is the sink.

else

n(

n(ts+1,

D(j,i)=D(j,i)+q;

D=(1

D

end

ts=ts+1;

end

figure(1)

grid on,box on

xlabel('t')

ylabel('n')

title(['number of a

figure(2)

plot(totalD)

grid on, box on

xlabe

ylabel('totalD')

title('total amount of pher

3-2 (B) Page 42

%%%%%%%%%%%%%%%%%%%

% Exp

% Input graph G=(V,G).

% Node 1 is the sourc

%%%%%%%%%%

clear;

% node 1 is the source and node


G=[0 2 0 0 0 0 1 0 0 0;

2 0 2 0 3 0 0 0 0 0;

0 2 0 4 0 0 0 0 0 0;

0 0 2 0]; % show connection and length in a matrix

f nodes

1 0 0 0 0 1 0 0 0;

1 0 0 0 0 0;

1 0 0 0 0 0 0;

0 0 1

0 1 1 0 0 0

1 0 0 0 1 1

0 0 0 0 0 0 1 0 1 0

0 1 0 1 0 1

1 0 0 1 0]; % set initial pheromone conductivity matrix

5,5,15,10,5,15,0]; % set initial numbers of ants on each node

% the amount of pheromone that an ant deposits on this way

% during a time interval

source node

outRate=25; % the outflow for the sink node

aporation rate, should be less than 1, avoiding a too rapid convergence

u=1:N

0 0 4 0 0 1 0 0 0 6

0 3 0 0 0 1 5 0 0 0

0 0 0 1 1 0 0 0 1 2

1 0 0 0 5 0 0 4 0 0

0 0 0 0 0 0 4 0 3 0

0 0 0 0 0 1 0 3 0 2

0 0 0 6 0 2

N=length(G); % the number o

D=[0

1 0 1 0

0 1 0

0 0 1 0 0 1 0

0 1 0 0

0 0 0 1

1 0 0 0 1 0 0 1 0 0

0 0 0 0

0 0 0 1 0

n(1,:)=[20,10,5,1

q=0.1;

inRate=25; % the inflow for the

lamda=0.5; % ev

t=0;

ts=1;

S=10000*floor(1/lamda); % iteration times

for s=1:S

T=zeros(N);

for

for v=1:N


T(u,v)=abs(n(ts,u)-n(ts,v));

.*T./G,1,N^2),0);

to the system

ate];

(Gs));

d); % exponentially distributed.

storet(ts)=t;

s)<=r)+1;

,1)+1;

)

x(0,n(ts,N)-1);

if n(ts,i)>n(ts,j)

else

n(ts+1,i)=n(ts,i)+1;

D(j,i)=D(j,i)+q;

t*lamda)*D % quantity of pheromone after evaporation

1,D).*(G>0); % the elements should not be less than zero

end

end

Gs=max(reshape(D

% ants coming in

Gs=[Gs inRate outR

sG=nansum(nansum

Dt=(-1/sG)*log(ran

t=t+Dt;

% choose one edge at random

r=rand*sG;

ind=sum(cumsum(G

n(ts+1,:)=n(ts,:);

if (ind==N^2+1)

% ant coming in

n(ts+1,1)=n(ts

elseif (ind==N^2+2

n(ts+1,N)=ma

else

i=ceil(ind/N);

j=ind-(i-1)*N;

% flip an ant

n(ts+1,i)=n(ts,i)-1;

n(ts+1,j)=n(ts,j)+1;

D(i,j)=D(i,j)+q; % pheromone laid by an ant

n(ts+1,j)=n(ts,j)-1;

end

D=(1-D

D=max(0.00


end

ts=ts+1;

(ts)=sum(sum(D));

t on each node,(number of node is',num2str(N),')'])

omone')

%%%%%%%%%%%%%%%%%%%%%%%

son distributed simulation in 4 nodes graph

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

urce and node 4 is the sink

% the number of nodes

totalD

end

figure(1)

plot(n)

grid on,box on

xlabel('t')

ylabel('n')

title(['number of an

figure(2)

plot(totalD)

grid on, box on

xlabel('t')

ylabel('totalD')

title('total amount of pher

3-3 (C) Page 49

%%%%%%%%%%%%%%%%%%%

% Pois

%%%%%%%%%%%%

clear;

% node 1 is the so

G=[0 1 6 0;

1 0 2 4;

6 2 0 1;

0 4 1 0]; % show connection and length in a matrix

N=length(G);

D=[0 1 1 0;

1 0 1 1;

1 1 0 1;

0 1 1 0]; % initial pheromone conductivity


n(1,:)=[10,7,3,0]; % set initial numbers of ants on each node

% evaporation rate, should be less than 1

% fix the time interval

% charge/particle

floor(1/lamda); % number of iteration

e(G)); % initial current matrix

r ts=1:S

t;

n(ts,1)+poissrnd(b);

ber of ants run in the sink is a Poisson distributed number

)=max(0,n(ts,N)-poissrnd(b));

ber of ants run away from the sink is also a Poisson distributed number

for j=1:N

i,j)==0)

end

(i,j)*Dt; % update the pheromone

ated after all the current is known

n(ts+1,k) = n(ts,k) - sum(P(k,:)) + sum(P(:,k));

0.001,D).*(G>0)

)=sum(sum(D));

lamda=0.3;

t=0;

ts=1;

Dt=0.02;

Q=0.01;

b=20;

S=10000*

P=zeros(siz

fo

t=t+D

n(ts,1)=

% the num

n(ts,N

% the num

for i=1:N

if (G(

P(i,j)=0;

else

P(i,j)=poissrnd(Dt*abs(n(ts,i)-n(ts,j))*D(i,j)/G(i,j))*sign(n(ts,i)-n(ts,j));

D(i,j)=D(i,j)*(1-Dt*lamda)+Q*P

end

end

for k=1:N

% the number of ants should be upd

end

D=max(

totalD(ts

end


3-3 (D) Page 52

clear;

e take 10 nodes, node 1 is the source and node 10 is the sink

0 4 0 0 0 0 0 0;

0 0 0 1 1 0 0 0 1 2

5 0 0 4 0 0

4 0 3 0

0

0

N=le nodes

D=[0

0 0 0;

;

1

0

0 1 0]; % set initial pheromone matrix

,10,5,15,5,15,10,5,15,0]; % the number of ants

mda=0.3; % evaporation rate, should be less than 1

t=0;

ts=1

% charge/particle

000*floor(1/lamda); % number of iteration

P=zeros(size(G)); % initial current matrix

% w

G=[0 2 0 0 0 0 1 0 0 0;

2 0 2 0 3 0 0 0 0 0;

0 2

0 0 4 0 0 1 0 0 0 6

0 3 0 0 0 1 5 0 0 0

1 0 0 0

0 0 0 0 0 0

0 0 0 0 1 0 3 0 2

0 0 6 0 2 0 0 2 0]; % show connection and length in a matrix

ngth(G); % the number of

1 0 0 0 0 1 0 0 0;

1 0 1 0 1 0 0

0 1 0 1 0 0 0 0 0 0

0 0 1 0 0 1 0 0 0 1

0 1 0 0 0 1 1 0 0 0

0 0 0 1 1 0 0 0 1

1 0 0 0 1 0 0 1 0 0

0 0 0 0 0 0 1 0 1

0 0 0 0 0 1 0 1 0 1

0 0 0 1 0 1 0

n(1,:)=[20

la

;

Dt=0.02; % fix the time interval

Q=0.01;

b=20;

S=10


for ts=1:S

t=t+Dt;

ts,1)=n(ts,1)+poissrnd(b);

number

,N)-poissrnd(b));

ants run away from the sink is also a Poisson distributed number

==0)

,j)=0;

,j)=poissrnd(Dt*abs(n(ts,i)-n(ts,j))*D(i,j)/G(i,j))*sign(n(ts,i)-n(ts,j));

t; % update the pheromone

f ants should be updated after all the current is known

(ts,k) - sum(P(k,:)) + sum(P(:,k));

*(G>0)

m(D));

n(

% the number of ants run in the sink is a Poisson distributed

n(ts,N)=max(0,n(ts

% the number of

for i=1:N

for j=1:N

if (G(i,j)

P(i

else

P(i

end

D(i,j)=D(i,j)*(1-Dt*lamda)+Q*P(i,j)*D

end

end

for k=1:N

% the number o

n(ts+1,k) = n

end

D=max(0.001,D).

totalD(ts)=sum(su

end


References

[1] o, and Guy Theraulaz. Swarm Intelligence: From Natural

39-46, 1999.

[2] Joachim Keltsch. Active walker model for the

,

997.

nd Thomas Stutzle. Ant Colony Optimization. The MIT Press, pages

76, 2004.

Maria Gambardella. Ant Colony System: A cooperative

Traveling Salesman Problem. IEEE TRANSACTIONS ON

RY, Vol. 1, No.1, 53-66, 1997.

putational study. University of Vienna, 1997.

ts as a computational intelligence technique. IEEE COMPUTATIONAL

LLIGENCE MAGAZINE, 1556-603X, 28-39, 2006.

] Daniel Merkle and Martin Middendorf. Modeling the dynamics of ant colony

volutionary Computation 10(3): 235-262, 2002.

[8] thnology,

Denebourg, S. Aron, S. Goss, and J. M. Pasteels. The self-organizing exploratory

havior, Vol. 3, 159, 1990.

enebourg, S. Aron and J. M. Pasteels. “How trail laying

and trial following can solve foraging problems for ant colonies,’’ in Behavioral

Mechanisms of Food Selection, R. N. Hughes Ed., NATO-ASI Series. Berlin:

Springer-Verlag, vol. G 20, 1990.

1] Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni. The ant system: Optimization

by a colony of cooperating agents. IEEE TRANSACTION ON SYSTEMS, MAN,

AND CYBERNETICS-PART B, 26(1): 29-41, 1996.

2] Peter G. Doyle and J. Laurie Snell. Random walks and electric networks. Mathematical

Association of America, Washington, DC, 1984.

3] Atsushi Tero, Ryo Kobayashi, and Toshiyuki Nakagaki. Physarum solver: A biologically inspired method of road-network navigation. Physica A: Statistical Mechanics and its Applications, 363(1): 115–119, 2006.

[14] David Stirzaker. Stochastic Processes &Models[M]. Oxford University Press, pages 50,

Eric Bonabeau, Marco Dorig

to Artificial Systems. Oxford University Press, pages 1-8, 14-16, 25-36,

Dirk Helbing, Frank Schweitzer, and

formation of human and animal trail systems. Physical Review E, vol. 56, 2527-2539

1

[3] Marco Dorigo a

1-34, 65-69, 70-

[4] Marco Dorigo and Luca

Learning Approach to the

EVOLUTIONA

[5] Bernd Bullnheimer and Richard F. Hartl. A new rank based version of the ant

system—a com

[6] Marco Dorigo, Mauro Birattari and Thomas Stutzle. Ant colony optimization:

artificial an

INTE

[7

optimization. E

J. L. Denebourg and S. Goss. Collective patterns and decision-making. E

Ecology and Evolution, vol. I, 295-311, 1989. [9] J. L.

pattern of the Argentine ant. Journal of Insect Be

[10] S. Goss, R. Beckers, J. L. D

[1

[1

[1


8, 2005.

. Reversibility and stochastic networks. John Wiley & Sons Ltd.,

[16] Laszlo Lovasz. Random

[18] Colorni, Marco Dorigo and Vittorio Maniezzo. Distributed Optimization by

Ant Colonies. In proceedings of ECAL91-

42, Paris.

[20] Marco Dorigo. Optimnico di Milano, Italy, 1992.

. . Rep. 91-016, 1991.

lasmic move

[23] oeboid organism. Nature

[24] T.. Flow-network

, 47–52,2001.

l., 244(4):

[28] d

s, pages 201~204, 2008.

71-76, 107-12

[15] Frank P. Kelly

Chichester, 1979. Wiley Series in Probability and Mathematical Statistics.

walks on graphs: a survey. YALEU/DCS/TR-1029, 1994.

[17] Kentaro Ito, Anders Johansson, Toshiyuki Nakagaki and Atsushi Tero. Convergence

Properties for the Physarum Solver. Arxiv preprint arXiv:1101.5249, 2011.

Alberto

European conference on artificial life,

Elsevier Publishing, 134-1

[19] E. L. Lawler, J. K. Lenstra, A.H.G. Rinnooy-Kan and D. B. Shmoys Eds.. The

travelling Salesman Problem. New York: Wiley, 1985.

ization, learning and natural algorithms (in Italian), Ph.D. dissertation. Dipartimento di Elettronica, Politec

[21] M. Dorigo, V. Maniezzo, and A. Colorni. Positive feedback as a search strategyDipartimento di Elettronica, Politecnico di Milano, Italy, Tech

[22] Kobayashi, R., Tero, A., Nakagaki, T.: Mathematical model for rhythmic protopment in the true slime mold. Journal of Mathematical Biology 53, 273–286,

2006. Nakagaki, T., Yamada, H., Toth, A.. Maze-solving by an am407, 470–470, 2000. Tero, A., Yumiki, K., Kobayashi, R., Saigusa, T., and Nakagaki,adaptation in physarum amoebae. Theory in Biosciences 127, 89–94, 2008.

[25] Nakagaki, T., Yamada, H., Toth, A.. Path finding by tube morphogenesis in an amoeboid organism. Biophysical Chemistry 92

[26] Toshiyuki Nakagaki, Atsushi Tero, and Ryo Kobayashi. A mathematical model for adaptive transport network in path finding by true slime mold. J. Theor. Bio553-564, 2007.

[27] Atsushi Tero, Seiji Takagi, Tetsu Saigusa, and Kentaro Ito. Rules for Biologically Inspired Adaptive Network Design. Science Vol. 327, 439-442, 2010.

Matt J. Keeling and Pejman Rohani. Modeling infectious diseases in humans an

animals. Princeton University Pres

Documents

Shortest paths through a reinforced random walk417959/FULLTEXT01.pdf · 2011. 5. 19. · describe a random walk model which can not be said to be solving the shortest path problem