GECCO'2007: Modeling Selection Pressure in XCS for Proportionate and Tournament Selection

Modeling Selection Pressure in XCS for Proportionate and

Tournament SelectionAlbert Orriols-Puig1,2 Kumara Sastry2

Pier Luca Lanzi1,3 David E. Goldberg2

Ester Bernadó-Mansilla1

1Research Group in Intelligent SystemsEnginyeria i Arquitectura La Salle, Ramon Llull University

2Illinois Genetic Algorithms LaboratoryDepartment of Industrial and Enterprise Systems Engineering

University of Illinois at Urbana Champaign

3Dipartamento di Elettronica e InformazionePolitecnico di Milano

Slide 2GRSI Enginyeria i Arquitectura la Salle

Motivation

Facetwise modeling to permit a successful understanding of complex systems (Goldberg, 2002)

– Model of generalization pressures of XCS (Butz, Kovacs, Lanzi, Wilson,04)

– Learning time bound (Butz, Goldberg & Lanzi, 04)

– Population size bound to guarantee niche support (Butz, Goldberg, Lanzi & Sastry, 07)


Motivation

Analysis of selection schemes in XCS: Proportionate vs. Tournament

– Tournament selection is more robust to parameter settings and noise than proportionate selection (Butz, Goldberg & Tharakunnel, 03) and (Butz, Sastry, Goldberg, 05)

– Proportionate selection is, at least, as robust as tournament selection if the appropriate fitness separation is used (Karbat, Bull & Odeh, 05)

In GA, these schemes were studied through the analysis of takeover time (Goldberg & Deb, 90; Goldberg, 02)


Aim

Model selection pressure in XCS through the analysis of the takeover time

– Consider that XCS has converged to an optimal solution

– Write differential equations that describe the change in proportion of the best individual

– Solve the equations and derive a closed form solution

– Validate the model empirically


Outline

1. Description of XCS

2. Modeling takeover time

3. Comparing the two models

4. Experimental validation

5. Modeling generality

6. Conclusions


Description of XCS

Representation:

– fixed-size rule-based representation

– Rule Parameters: Pk, εk, Fk, nk

Learning interaction:

– At each learning iteration Sample a new example

– Create the match set [M]

– Each classifier in [M] votes in the prediction array

– Select randomly an action and create the action set [A]


2. Modeling Takeover Time

3. Comparing the two Models

4. Experimental Validation

5. Modeling Generality

6. Conclusions


Description of XCS

Rule evaluation: reinforcement learning techniques.

– Prediction:

– Prediction error:

– Accuracy of the prediction:






6. Conclusions


Description of XCS

Rule evaluation: reinforcement learning techniques.

– Relative accuracy:

– Fitness computed as a windowed average of the accuracy:






6. Conclusions


Description of XCS

Rule discovering:

– Steady-state, niched GA

– Population-wide deletion

Proportionate selection

• Probability proportionate to rule’s fitness.

Tournament selection

• Selects τ percent of classifiers from [A]

• Selects the classifier with higher microclassifier fitness






6. Conclusions


Outline






6. Conclusions


Modeling Takeover Time

In GA:

– Usually, the fitness of an individual is constant

– Selection and replacement are performed over the whole population

In XCS:

– Fitness depends on the other rule’s fitness in the same niche

– Selection is niched-based, whilst deletion is population-wide






6. Conclusions


Modeling Takeover Time

Assumptions in our model– XCS has evolved a set of non-overlapping niches

– Simplified scenario: niche with two classifiers cl1 and cl2.

– cl1 is the best rule in the niche: k1 > k2

– Classifier clk has:

• prediction error εk

• fitness Fk

• numerosity nk

• microclassifier fitness fk

– cl1 and cl2 are equally general Same reproduction opportunities

– We assume niched deletion






6. Conclusions


Proportionate Selection

Fitness is an average of cl1 and cl2 respective accuracies

Then, the probability of selecting the best classifier cl1 is:

Probability of deletion: Pdel(clj) = nj/n where n=n1+n2

Num. ratio: nr = n2/n1

Accuracy ratio: ρ=k2/k1






6. Conclusions



Evolution of cl1 numerosity

1. The numerosity of cl1 increases if cl1 is selected by the GA and another classifier is selected to be deleted

2. The numerosity of cl1 decreases if cl1 is not selected by the GA but it is selected by the deletion operator

3. The numerosity of c1 remain de same otherwise.






6. Conclusions



Grouping the above equations we obtain

Rewritten in terms of proportion of classifiers cl1 in the niche:

Considering Pt+1 – Pt ≈ dp/dt

Pt = n1/n






6. Conclusions

Integrate: Initial proportion: P0 Final proportion: P



This gives us that the takeover time for proportionate selection is guided by the following expression:

P0: initial proportion of cl1 in the niche

P: final proportion of cl1 in the niche

ρ: accuracy ratio between cl2 and cl1

n: niche size

If ρ 1: trws ≈ ∞

If ρ 0:

A higher separation between fitness enables a higher ability in identifying accuraterules, as announced by Karbat, Bull & Odeh, 2005.

n






6. Conclusions


Tournament Selection

Assumptions

– Fixed tournament size s

– cl1 is the best classifier in the niche: f1 > f2 , that is, F1/n1 > F2 /n2






6. Conclusions



Evolution of cl1 numerosity

1. The numerosity of cl1 increases if cl1 participates in the tournament and another classifier is selected to be deleted

2. The numerosity of cl1 decreases if cl1 does not participate in the tournament but it is selected by the deletion operator

3. The numerosity of c1 remain de same otherwise.






6. Conclusions



This gives us that the takeover time for tournament selection is guided by the following expression:



s: tournament size

n: niche size

Tournament selection does not depend on the accuracy ratio between the best classifier and the others in the same [A], as pointed by Butz, Sastry & Goldberg, 2005

As s increases, this expression decreases

It does not depend on the individual fitness

For large s:






6. Conclusions


Outline






6. Conclusions


Proportionate vs. Tournament

Values of s and ρ for which both schemes result in the same takeover time. Require: t*RWS = t*TS

– We obtain

For P0 = 0.01 and P = 0.99






6. Conclusions


Outline






6. Conclusions


Design of Test Problems

Single-niche problem

– One niche with 2 classifiers:

• Highly accurate classifier cl1:

• Less accurate classifier cl2:

• Varying ρ we are changing the fitness separation between cl1 and cl2

– Population initialized with

• N · P0 copies of cl1

• N · (1 – P0) copies of cl2






6. Conclusions


Results on the Single-Niche Problem






6. Conclusions

Accuracy ratio: ρ= 0.01

RWS

Tournament s=9

Tournament s=3

Tournament s=2








6. Conclusions


RWSTournament s=9

Tournament s=3

Tournament s=2








6. Conclusions


RWS

Tournament s=9

Tournament s=3

Tournament s=2


Design of Test Problems

Multiple-niche problem

– Several niches with 1 maximally accurate classifier each niche.

– One over-general classifier that participates in all niches

– The population contains:

• N · P0 copies of maximally accurate classifiers

• N · (1 – P0) copies of the overgeneral classifier

– The problem violates two assumptions of the model

• Overlapping niches

• The size of the different niches differ from the population size

– Deletion can select any classifier in [P]






6. Conclusions


Results on the Multiple-Niche Problem

For small ρ the theory slightly underestimates the empirical takeover time The model of proportionate selection is accurate in general scenarios if the ratio of accuracies is small

In situations where there is a small proportion of the best classifier in one niche competing with other slightly inaccurate and overgeneral, the overgeneral may take over the population.

Further experiments, show that for ρ > 0.5, the best classifiers is removed from the population






6. Conclusions

RWS ρ = 0.01

RWS ρ = 0.20

RWS ρ = 0.30

RWS ρ = 0.40

RWS ρ = 0.50


Results on the Multiple-Niche Problem

For high s the theory slightly underestimates the empirical takeover time

The model of tournament selection is accurate in general scenarios if the tournament size is high enough

Only in the extreme case (s=2), the experiments strongly disagree with the theory






6. Conclusions

TS s = 2

TS s = 3

TS s = 9


Outline






6. Conclusions


Modeling Generality

Scenario

– The best classifier cl1 appears in the niche with probability 1

– cl2 appears in the niche with probability ρm






6. Conclusions



The takeover time for proportionate selection is guided by the following expression:



ρ: accuracy ratio between cl2 and cl1

ρm: occurrence probability of cl2

N: niche size

If cl1 is either more accurate or more general than cl2, cl1 will take over the population.






6. Conclusions



The takeover time for tournament selection is guided by the following expression:



s: tournament size

ρm: occurrence probability of cl2

n: niche size

For low ρm or high s the right-hand logarithm goes to zero, so that the takeover timemainly depends on P0 and P






6. Conclusions


Results of the Extended Model onthe one-niched Problem






6. Conclusions

RW

S

To

urn

amen

t


Outline






6. Conclusions


Conclusions

We derived theoretical models for proportionate and tournament under some assumptions

– Models are exact in very simple scenarios

– Models can qualitatively explain both selection schemes in more complicated scenarios

Models support that tournament is more robust (Butz, Sastry & Goldberg, 2005)

Fitness separation is essential to guarantee that the best classifier will take over the population in proportionate selection (Karbhat, Bull & Oates, 2005)

Models show that proportionate selection depends on fitness scaling. It may fail in domains where there are slightly inaccurate classifiers (real-world domains)






6. Conclusions

Modeling Selection Pressure in XCS for Proportionate and


Albert Orriols-Puig1,2 Kumara Sastry2

Pier Luca Lanzi2 David E. Goldberg2

Ester Bernadó-Mansilla1

1Research Group in Intelligent Systems

Enginyeria i Arquitectura La Salle, Ramon Llull University

2Illinois Genetic Algorithms LaboratoryDepartment of Industrial and Enterprise Systems Engineering

University of Illinois at Urbana Champaign

Education

GECCO'2007: Modeling Selection Pressure in XCS for Proportionate and Tournament Selection