Estimation of Distribution Algorithms (EDA)

Siddhartha K. ShakyaSchool of Computing.

The Robert Gordon UniversityAberdeen, UK

ss@comp.rgu.ac.uk

EDAs

• A novel paradigm in Evolutionary Algorithm

• Also known as Probabilistic model building Genetic Algorithms or Iterated density

• A probabilistic model based heuristic

• Motivated from the GA evolution

• More explicit evolution than the GA

Basic Concept of Solution and Fitness

Given a set of colours, GCP is to try and assign Colour to each nodes in such the way that neighbouring nodes will not have same colour

a

b de

f

c

Graph colouring Problem: An Example

Basic concept of a solution and Fitness

1

10

1

10

a

b de

f

c

1 0 0 1 1 1

a b c d e f

1

fitness

1

11

0

00

a

b de

f

c1 0 1 0 1 0 6

Solution

Representation of a solution as a chromosome

Given 2 colourBlack = 0White = 1

Chromosome and Fitness in GCP

• Chromosome: is a set of colours assigned to the nodes of graph. (there are other way of representing GCP in GA, such as order based representation).

• Fitness: is the number of correctly coloured nodes.

GA Iteration

1. Initialisation of a “parent” population

2. Evaluation

3. Crossover

4. Mutation

5. Replace parent with “child” population and go to step 2 until termination criteria satisfies

GA Iteration

1 0 1 1 0 1

0 0 1 0 1 1

1 0 1 0 1 1

0 1 0 0 1 1

Parent population

2

2

4

3

fitness1 0 1 0 1 1

0 1 0 0 1 1

0 1 0 0 1 1

1 0 1 1 0 1

Selected Solution

0 1 1 0 1 1

1 0 0 0 1 1

0 1 0 1 0 1

1 0 1 0 1 1

After Crossover

0 1 1 0 1 1

1 0 0 0 1 0

0 1 0 1 0 1

1 0 1 0 1 1

After mutation

1

2

6

4

fitness

Initialization Evaluation

SelectionCrossover

Mutation

Repeat iteration

ab d

e

f

c

Given 2 colours(0,1)

GA evolution

• Selection drives evolution towards better solutions by giving a high pressure to the selection of high-quality solutions

• Crossover and mutation (Variation operator) together ensures the exploration of the possible space of the promising solutions. Maintains the variation in the population.

Variation in GA Evolution

• Has its limitation

• Can recombine fit solution to produce more fit solution

• Also can disrupt good solution and converge in local optimum

Estimation of Distribution Algorithm (EDA)

• To overcome the negative effective of the crossover and mutation approach of variation, a probabilistic approach of variation has been proposed.

• Algorithm using such approach is known as EDA (or PMBGA)

GA to EDA

Simple GA framework

Selection

Crossover

Mutation

Evaluation

Initial Population

Selection

Probabilistic Model Building

Evaluation

EDA framework

Sampling Child Population

Initial Population

General Notation• EDA represents a solution as a set of value taken by a

set of random variable.

nXXXX ,...,, 21 nxxxx ,...,, 21Chromosome is a set of value taken by set of random

variables (Where each }1,0{ix for bit representation)

)()( iii xpsimplyorxXp is a univariate marginal distribution

)|()|( jijjii xxpsimplyorxXxXp is a conditional distribution

)()( xpsimplyorxXp is a joint probability distribution

1 0 1 1 0 1

Solution

1X 3X2X 4X 5X 6X

0 1 0 0 1 1

X

x

x

Estimation of Probability distribution

ii xX

i xpxp )()(

)()|().....,...,|(),...,|()( 13221 nnnnn xpxxpxxxpxxxpxp

)(

),()|(

j

jiji xp

xxpxxp

n

iixpxp

1)()(

ixUsually it is not possible to calculate the joint probability distribution, so it is estimated. For example, assuming all are independent of each other, the joint probability distribution becomes the product of simple univariate marginal distribution.

1 0 1 1 0 1

Solution

1X 3X2X 4X 5X 6X

0 1 0 0 1 1

X

x

x

Simple Univariate Estimation of Distribution Algorithm

)1()( iii XporxXp2

1

2

1

2

1

2

1

2

1

2

2

)0()( iii XporxXp2

1

2

1

2

1

2

1

2

1

2

0

Selection

Evaluation

Calculate univariate marginal probability

and sample Child Population

Initial Population

1 0 1 1 0 1

Solution

1X 3X2X 4X 5X 6X

0 1 0 0 1 1

X

x

x

Simple univariate EDA (UMDA)

1 0 1 1 0 1

0 0 1 0 1 1

1 0 1 0 1 1

0 1 0 0 1 1

Parent population

2

2

4

3

fitness1 0 1 0 1 1

0 1 0 0 1 1

0 1 0 0 1 1

1 0 1 1 0 1

Selected Solution

0 1 1 0 1 1

1 0 0 0 1 1

0 1 0 1 0 1

1 0 1 0 1 1

After mutation

1

2

6

4

fitness

Initialization Evaluation

Selection

Sampling

Repeat iteration

ab d

e

f

c

Given 2 colours(0,1)

4

2)1( iXp4

2

4

2

4

1

4

3

4

4

4

2

4

2

4

2

4

3

4

1

4

0)0( iXp

Estimation of

Distribution

n

iixpxp

1)()(

Build model

Calculate Distribution

Note

• It is not guaranteed that the above algorithm will give optimum solution for the graph colouring problem.

• The reason is obvious. – The chromosome representation of GCP has

dependency. i.e. node 1 taking black colour depends upon the colour of node 2.

– But univariate EDAs do not assume any dependency so it may fail.

• However, one could try

Complex Models• To tackle problems where there is dependency

between variables we need to consider more complex models.

• The extra model building step will be added to univariate EDA.

• Different algorithms has been purposed using different models

• They are categorised into three groups– Univariate EDA– Bivariate EDA– Multivariate EDA

Univariate EDA Model

Graphical representation of probability model assuming no dependency among variables. (UMDA, PBIL, cGA)

n

iixpxp

1)()(

x1

x2

x3

x4

x6

x5

x7

Bivariate EDA Model

Graphical representation of probability model assuming dependency of order two among variables.

a. Chain model (MMIC)

b. Tree model (COMIT)

c. Forest model(BMDA)

)()|(.).........|()|()(12121 nnn iiiiiii xpxxpxxpxxpxp

n

iji xxpxp

1)|()(

Multivariate EDA Model

Graphical representation of probability model considering multivariate dependency among variables.

a. Marginal product model (ECGA)

c. (BOA, EBNA)b. Triangular model (FDA)

Finding a probabilistic model• Task of finding a good probabilistic model

(finding the relationship between variable) is a optimization problem in itself.

• Most of the algorithm use Bayesian network to represent the probabilistic relationship.

• Two metric to measure the goodness of Bayesian Network.

– Bayesian Information Criterion (BIC) metric:– Bayesian-Dirichlet (BD) metric:

• Use greedy heuristic to find a good model.

• EDA is an active area of research for GA community

• EDAs are reported to solve GA hard problems, and also hard optimization optimisation problems like MAX SAT.

• Success and failure of EDAs depends upon the accuracy of the used Probabilistic model.

Summary

Links• http://cswww.essex.ac.uk/staff/zhang/MoldeBasedWeb/R

Group.htm (Research Groups working on EDAs)

• http://www.sc.ehu.es/ccwbayes/main.html (EDA homepage maintained by Intelligent system group).

Books• Larrañaga P., and Lozano J. A. (2001) Estimation of Distribution Algorithms:

A New Tool for Evolutionary Computation. Kluwer Academic Publishers, 2001.

• Pelikan, M., (2002). Bayesian optimization algorithm: From single level to hierarchy. Ph.D. thesis, University of Illinois at Urbana-Champaign, Urbana, IL. Also IlliGAL Report No. 2002023.