Parallel GAs

A Hitchiker’s guide to parallel GAs

3 key papers by Eric Cantu-Paz and David Golberg

Presented by :

Yann SEMETUniversite de Technologie de Compiegne

Parallel GAs

Our 3 papers

A survey of Parallel Genetic Algorithms, E. Cantu-Paz, 1998

On the scalability of Parallel GAs, E. Cantu-Paz & David Golberg, 1999

Markov Chain Models of Parallel GAs, E. Cantu-Paz, 2000

Parallel GAs

Roadmap

General knowledge Toward parallel GAs Categorization Key issues

Theory run for parameter sizing Bounding cases Gambler’s Ruin model Markov Model

Parallel GAs

A good nest

Illinois and parallelism

Illinois and GAs

An ideal nest for parallel GAs

Parallel GAs

Flynn’s taxonomy

SISD : your PC

SIMD : not really relevant

MISD : multi processors

MIMD : super computers

Parallel GAs

GAs : 2 levels of parallelism

Population-wise

Computation-wise

Parallel GAs

Taxonomy

Master-slave GAs

Fine-grained GAs

Coarse-grained GAs

Hierarchical GAs

Parallel GAs

Master and Slaves

Similar to a simple GA

Fitness/Communication tradeoff

Fitness distribution

Operator distribution

Efficient speedup

Parallel GAs

Fine-grained

One population

Limited spatial interaction

Critical parameter : radius

Matches massively parallel computer

A potential alternative to Coarse-Grained

Parallel GAs

Multiple-Deme

Key factors : Demes Migration Topology

Parallel GAs

Hierarchical

Three possibilities

The engineer’s choice

Parallel GAs

Non-Traditional GAs

ECO

GENITOR

mGAs, fmGAs

GP

Parallel GAs

Engineering Summary 1

Categorization

Market Map

Parallel GAs

Engineering Summary 2

Key parameters : Demes Migration Topology

Goal : remain panmictic

Parallel GAs

Milestone1

Up to now : Categorization Key issues

To come : Theoretical scaling Markov models

Parallel GAs

Theory Roadmap

Goal : calculate optimal parameters

From Master-Slave to : Multiple deme High Migration Dense topology

Markov models

Parallel GAs

Paramaters to be tuned

Populations : Size Number

Migration : Rate Frequency

Topology : Density Shape

Parallel GAs

Single Population 1

Assumptions Distributed population Modified operators to ensure panmictism

Why ? Straightforward and intuitive Close to Multiple-deme bounding case

Parallel GAs

Single Population 2

Computation time :

Optimal chunk size :

cf

p TPkP

nTT )1(

c

f

kT

nTP *

Parallel GAs

Multi Populations 1

2 Bounding cases : Lower Bound on migration and connectivity Upper bound : “deja vu)” but :

At most one migration per generation Picking the migrants

Parallel GAs

The Gambler’s ruin model

A random walk to absorbative barriers

Predicts solution quality

Yields population sizing

A conservative model

0

0

1

1

1 x

n

n

bb p

q

pq

pq

P

Parallel GAs

Multiple demes

Relaxation :

m

r

m

Q

mm

QP rr

2

ln

2

^

:

^^

Parallel GAs

Regular topologies

Over two epochs

1

^

1^

x

p

qP

bbbb

bbx

PP

PP

11

^

1

Parallel GAs

Optimal parameters

Deme size :

Optimal connectivity :c

xczz

nkk

d

2

^

12^^

2 22

3/2

0*

2

c

f

T

Tgn

Parallel GAs

Topology considerations

Efficiency depends on connectivity

Extented Neighborhoods

After several epochs :

nPnrcP bbdmbb

Parallel GAs

Derivation…

Dimensional analysis gives :

The GRM then gives :

1

1'

1'

mc

1'ln

1ln2

1'

1 0

^

n

qp

Pn

k

d

Parallel GAs

Derivation…

Optimal connectivity :

Optimal number of epochs obtained similarly

3/2

0*

'2

c

f

T

Tgn

Parallel GAs

The long run

At the end :

drnn

dnn 1'

11

max

r

Parallel GAs

Finally

Solving the time equation :

Similar to single population !

c

f

T

nTgr

1*

Parallel GAs

Markov Chains

Transient and closed states

M : transition matrix

N :fundamental matrix : T : time V : distribution A : Pbb on the long run

Parallel GAs

Upper Bounding case

Full migration

Dense topology

V is a binomial distribution

Parallel GAs

Arbitrary Migration

Different rates : might not converge

Yields more state

V is again binomial but with a disjunction

Parallel GAs

Arbitrary Topologies

Even more states !

Which demes actually converged ?

Each state is a binary string

Parallel GAs

Conclusions on Markov Chains

Predictive models

Panmictism on the long run

Prefer : High migration rates Dense topologies

Parallel GAs

General Summary

Categorization

Key issues

Parameter sizing

Accurate Predictive Models

Tradeoff Practical Guidelines

Parallel GAs

Discussion