Comparing the performance of the

network-based artificial immune system (AIS)

algorithm against a standard particle swarm

optimization (PSO) algorithm

In this report a network-based Artificial Immune System (AIS) algorithm is

applied to the Rosenbrock, Michalewicz, and Schwefels mathematical

optimization problems in 5, 10, and 15 dimensions each. The results are averages

across 75 samples and benchmarked against a standard particle swarm

optimization (PSO) algorithm.

Stuart Gordon Reid

Student number 10026942

Email: stuartgordonreid@gmail.com

Department of Computer Science

University of Pretoria

Page 1 of 22

Introduction and problem statement

Artificial immune system (AIS) algorithms are a relatively new paradigm in computational                     

intelligence inspired by the naturally occurring processes in the biological immune system.                     

Currently AIS offers a suite of five biologically inspired algorithms [1], namely:

1. The clonal selection algorithm

2. The negative selection algorithm

3. The artificial immune recognition algorithm

4. The immune network algorithm and

5. The dendritic cell algorithm

This assignment looks at the application of the network­based AIS algorithm to mathematical                       

function approximation problems. More specifically it looks at the following problems:

Mathematical optimization problem D = 5 D = 10 D = 15

Rosenbrock optimization problem 75 samples,2000 iterations,50 individuals

75 samples,2000 iterations,50 individuals

75 samples,2000 iterations,50 individuals

Michalewicz optimization problem 75 samples,2000 iterations,50 individuals

75 samples,2000 iterations,50 individuals

75 samples,2000 iterations,50 individuals

Schwefel optimization problem 75 samples,2000 iterations,50 individuals

75 samples,2000 iterations,50 individuals

75 samples,2000 iterations,50 individuals

The Ackley optimization problem was excluded from the study because the numerical results                       

produced by the algorithm were so small (such small exponents) that the graphs were                         

meaningless. The same is true for the optimization problems at dimensions less than 5.

The performance of the network­based AIS was benchmarked against the performance of a                       

standard Particle Swarm Optimization on the sample problem(s). The aim of the assignment is                         

to investigate the performance of the network­based AIS algorithms to function approximation.

Page 2 of 22

Table of contentsIntroduction and problem statementTable of contents1. Background Information

1.1. Function optimization1.2. Immune­inspired algorithms

1.2.1. Biological immune system1.2.2. Artificial immune system

1.3. Particle swarm optimization algorithms2. Design and Implementation

2.1. Algorithm implementation ~ class diagram2.2. Optimization implementation ~ class diagram2.3. Network­based artificial immune system pseudo­code algorithm

3. Results obtained3.1. Rosenbrock optimization problem

3.1.1. Rosenbrock Dimension 53.1.1.1. Fitness Function3.1.1.2. Standard Deviations

3.1.2. Rosenbrock Dimension 103.1.2.1. Fitness Function3.1.2.2. Standard Deviations

3.1.3. Rosenbrock Dimension 153.1.3.1. Fitness Function3.1.3.2. Standard Deviations

3.2. Michalewics optimization problem3.2.1. Michalewics Dimension 5

3.2.1.1. Fitness Function3.2.1.2. Standard Deviations

3.2.2. Michalewics Dimension 103.2.2.1. Fitness Function3.2.2.2. Standard Deviations

3.2.3. Michalewics Dimension 153.2.3.1. Fitness Function3.2.3.2. Standard Deviations

3.3. Schwefels optimization problem3.3.1. Schwefels Dimension 5

3.3.1.1. Fitness Function3.3.1.2. Standard Deviations

3.3.2. Schwefels Dimension 103.3.2.1. Fitness Function3.3.2.2. Standard Deviations

3.3.3. Schwefels Dimension 153.3.3.1. Fitness Function3.3.3.2. Standard Deviations

4. ConclusionsSources

Page 3 of 22

1. Background Information

This section aims to introduce the reader to relevant background information about function                       

optimization, immune­inspired algorithms, and particle swarm optimization algorithms.

1.1. Function optimization

Function optimization, otherwise known as mathematical optimization involves the maximization                 

or minimization of some objective function. Techniques for mathematical optimization can be                     

classified as either deterministic or stochastic. Computational intelligence offers a number of                     

algorithms that can be used to optimize mathematical functions. Such algorithms have shown                       

remarkable versatility when benchmarked against traditional techniques.

1.2. Immune-inspired algorithms

1.2.1. Biological immune system

The immune system consists of cells, molecules, and organs. Together these components form                       

a biological system that is capable of perceiving and combating cellular dysfunction. In other                         

words, the system can identify between good and bad cells in the body and act accordingly. The                               

interaction of the immune system allows the body to regulate itself and operate in a stable and                               

consistent state.

As mentioned, the immune system consists of a number of cells. One such cell is called the                               

T­Cell. T­Cells have receptor molecules that allow them each to recognize different peptide­MHC                       

combinations within the body. T­cells which recognize difference MHC combinations will then                     

respond through the secretion of lymphokines. These are recognized by B­Cells who are able to                           

differentiate between plasma cells (good) and antigen cells (bad). These B­Cells then bind to the                           

antigens they identify thereby neutralizing them using the antibodies.

Cells in the body are frequently selected to undergo mutation. Affinity maturation is process                         

through which the B­cells improve their response to antigens within the body after repeated                         

immunizations (iterations of exposure to non­exposure). Those cells which are better at picking                       

self (good) from non­self (bad) are cloned. This process is illustrated below

Page 4 of 22

The above diagram and related explanations are interpreted from the thesis entitled, ‘Artificial                       

Immune Systems: Part 1 Basic Theory and Applications’ by Leandro de Castro and Fernando                         

Von Zuben [2] published in December, 1999.

Page 5 of 22

1.2.2. Artificial immune system

Artificial immune systems are computational models inspired by the naturally occurring                   

processes in the biological immune system. As mentioned in the problem statement there are                         

five general algorithms stemming from the immune­based research field.

This assignment was specifically focussed around the immune network algorithm. The main                     

concept or strategy of the immune network algorithm is that better performing cells (ALC’s)                         

suppress weaker performing cells in their network and that any lymphocyte receptor within an                         

organism can be recognised by a subset of the total receptor repertoire. [3]

This concept was inspired by the work proposed by Jerne in 1974. The immune network theory                             

helped explain some of the observed emergent properties of the immune system, such as                         

learning and memory. Immune networks are often referred to as idiotypic networks. In his paper,                           

Jerne concluded that the immune system’s behaviour is as a result of it interacting with itself,                             

and from these interactions behaviour such as tolerance and memory emerge.

For more background information please visit one of the following resources:

1. AIS Web ­ http://www.artificial­immune­systems.org/ or

2. Clever Algorithms ­ http://www.cleveralgorithms.com/nature­inspired/immune.html

3. An overview of AIS ­ ftp.dca.fee.unicamp.br/pub/docs/vonzuben/tr_dca/trdca0199.pdf

4. Wikipedia page ­ http://en.wikipedia.org/wiki/Artificial_immune_system

5. Wikipedia page ­ http://en.wikipedia.org/wiki/Immune_network_theory

Page 6 of 22

1.3. Particle swarm optimization algorithms

PSO’s are population­based iterative search algorithms that manipulate a pool of potential                     

solutions, called particles, using mathematics borrowed from the field of dynamics to find                       

optima. In essence particles are ‘flown’ over hyper dimensional search spaces toward a global                         

optima. Such algorithms were originally derived from the behaviour of flocks of birds.

In the standard PSO algorithm a population, also referred to as a swarm, is initialized wherein                             

each particle represents a candidate solution to some optimization problem. For an                     

n­dimensional optimization problem, the candidate solution is represented as an n­dimensional                   

vector. During each iteration of the algorithm the global or neighbourhood best position is found                           

from the swarm. The best particle, called gbest, is then used in conjunction with the historical                             

personal best position(s) of each particle in the swarm to calculate a new position in the search                               

space for that particle. These two components are respectively called the social, and cognitive                         

components. In order to achieve this, an objective function (the function representing the quantity                         

being maximized or minimized) needs to be defined. The choice of objective function has a huge                             

impact on the effectiveness of the PSO algorithm. The below image illustrates the convergence                         

of particles in a PSO in a two dimensional search space towards at least a local minimum (note,                                 

this may not be the global minimum)1

1 This image can be found at https://www.groksolutions.com/static/images/technology/pso.jpg

Page 7 of 22

2. Design and Implementation

The implementation of the framework used to achieve the results discussed in the next section                           

was done in Java making use of standard packages.

2.1. Algorithm implementation ~ class diagram

The framework was divided into five distinct packages: algorithm, algorithm.ais, algorithm.pso,                   

optimization, and simulator. The class in the algorithm package and subpackages were defined                       

as follows. The algorithms were completely abstracted away from the optimization problems.

Page 8 of 22

2.2. Optimization implementation ~ class diagram

The class in the optimization package were defined as follows. The optimization problems were                         

defined as per those definitions provided by:http://www­optima.amp.i.kyoto­u.ac.jp/member/student/hedar/Hedar_files/TestGO_files/Page364.htm

The definition for Ackley's function did not function as expected, the output was always equal to                             

1.78 despite being correctly defined according the above website.

Page 9 of 22

2.3. Network-based artificial immune system pseudo-code algorithm

Below if the pseudo­code describing the network immune system algorithm as it is implemented                         

in the framework in the AIS_Algorithm class:

Input: Population_size, dimension, Number_clones, Number_random,Affinity_thresholdOutput: Solution_bestPopulation ← initalizePopulation(Population_size, dimension)While(!stoppingCondition())    EvaluatePopulation(Population)    Solution_best ← getBestSolution(Population)    Progeny ← None    For(cell : Population)

 Clones ← CreateClones(cell, Number_clones) for(clone : Clones)

 MutateRelativeToFitness(clone) end EvaluatePopulation(Clones) Progeny ← getBestSolution(Clones)

    end    SuppressLowAfinityCells(Progeny, AffinityThreshold)    Progency ← createRandomCells(Number_Random)    Population ← ProgenyEndSolution_best ← getBestSolution(Population)Return Solution_best

Page 10 of 22

3. Results obtained

The results presented in this section represented averages taken over 75 samples of the                         

simulation. For each problem and each dimension the graph showing the fitness functions is                         

presented followed by a graph illustrating the standard deviations of the results.

3.1. Rosenbrock optimization problem

3.1.1. Rosenbrock Dimension 5

3.1.1.1. Fitness Function

3.1.1.2. Standard Deviations

Page 11 of 22

3.1.2. Rosenbrock Dimension 10

3.1.2.1. Fitness Function

3.1.2.2. Standard Deviations

Page 12 of 22

3.1.3. Rosenbrock Dimension 15

3.1.3.1. Fitness Function

3.1.3.2. Standard Deviations

Page 13 of 22

3.2. Michalewics optimization problem

3.2.1. Michalewics Dimension 5

3.2.1.1. Fitness Function

3.2.1.2. Standard Deviations

Page 14 of 22

3.2.2. Michalewics Dimension 10

3.2.2.1. Fitness Function

3.2.2.2. Standard Deviations

Page 15 of 22

3.2.3. Michalewics Dimension 15

3.2.3.1. Fitness Function

3.2.3.2. Standard Deviations

Page 16 of 22

3.3. Schwefels optimization problem

3.3.1. Schwefels Dimension 5

3.3.1.1. Fitness Function

3.3.1.2. Standard Deviations

Page 17 of 22

3.3.2. Schwefels Dimension 10

3.3.2.1. Fitness Function

3.3.2.2. Standard Deviations

Page 18 of 22

3.3.3. Schwefels Dimension 15

3.3.3.1. Fitness Function

3.3.3.2. Standard Deviations

Page 19 of 22

4. Conclusions

From the results presented in the previous section two emergent trends can be identified and                           

interpreted. These two trends are that:

1. After 2000 iterations, the fitness of the global­best candidate solution found by the particle                         

swarm optimization is better than the global­best particle in the immune algorithm                     

population in the greater majority of cases

2. The rate of convergence achieved by the particle swarm optimization is much greater                       

than that of the immune algorithm as indicated by the higher standard deviations across                         

the fitnesses of individuals in the immune algorithm population

In the opinion of the author, this might be attributable to the formation of networks surrounding                             

local minima in the search space by the AIS algorithm. This is shown, graphically below:

As you can see in the diagram, networks form in the local minima. In order to ‘escape’ the local                                   

minima, the particles need to be mutated. However, in especially ruggest search landscapes this                         

mutation might be ineffective.

Page 20 of 22

In contrast, this is how the PSO algorithm would operate:

By comparison, a PSO can very easily “fly over” local minima that could trap individuals in the                               

AIS algorithms. The drawback of this is that the particles could easily converge onto one                           

sub­optimal solution. To mitigate this, the standard PSO algorithm used in this assignment                       

‘restarted’ particles once they were 99.8% similar to the global best position. This way diversity                           

can be maintained in the swarm.

The introduce diversity to the AIS algorithm, particles were also restarted if they were in the                             

bottom ten percent of the population in terms of fitness. Thereby making the results comparable.

All in all the particle swarm optimization algorithm outperformed the immune algorithm. That                       

having been said, the performance of these algorithms is highly dependent on their configuration                         

and parameter values. An immune algorithm optimized to overcome the aforementioned issues                     

may outperform a particle swarm optimization on certain classes of optimizations problems.

Page 21 of 22

Resources

Academic sources

[1] Clevel Algorithms: Nature­inspired programming recipes by Jason Brownlee, PhD. Overview of immune

algorithms at: http://www.cleveralgorithms.com/nature­inspired/immune.html

[2] Engelbrecht, A. P. (2007) “Introduction to Computational Intelligence, in Computational Intelligence: An

Introduction, Second Edition”, John Wiley & Sons, Ltd, Chichester, UK

Web-based sources

Code for network model shown in rubyhttp://www.cleveralgorithms.com/nature­inspired/immune.htmlhttp://www.cleveralgorithms.com/nature­inspired/immune/immune_network_algorithm.html

AIS Algorithms pseudo­code and background infohttp://www.artificial­immune­systems.org/people­new.shtml (for report)http://www.artificial­immune­systems.org/algorithms.shtml

Wiki pageshttp://en.wikipedia.org/wiki/Artificial_immune_systemhttp://en.wikipedia.org/wiki/Immune_network_theory

AIS overview paperftp://ftp.dca.fee.unicamp.br/pub/docs/vonzuben/tr_dca/trdca0199.pdf

Functionshttp://www­optima.amp.i.kyoto­u.ac.jp/member/student/hedar/Hedar_files/TestGO_files/Page2607.htmhttp://www­optima.amp.i.kyoto­u.ac.jp/member/student/hedar/Hedar_files/TestGO_files/Page2537.htmhttp://www­optima.amp.i.kyoto­u.ac.jp/member/student/hedar/Hedar_files/TestGO_files/Page295.htmhttp://www­optima.amp.i.kyoto­u.ac.jp/member/student/hedar/Hedar_files/TestGO_files/Page2530.htmhttp://www­optima.amp.i.kyoto­u.ac.jp/member/student/hedar/Hedar_files/TestGO_files/Page2376.htm

Page 22 of 22