View
213
Download
0
Category
Preview:
Citation preview
Fuzzy inference
20 26
Warm
17
Cold Hot
29
50
Partial
30
Cloudy Sunny
100
Fuzzyfication Implication
48
MediumLow High
Defuzzyfication
Knowledge base: defines rules and membership functions
Defuzzyfier: translates fuzzy outputs into crisp values
Fuzzyfier: translates crisp inputs into fuzzy values
Inference engine: applies reasoning to compute fuzzy outputs
DefuzzifierInferenceEngine
Fuzzifier
Rule base
Database
Knowledge base
FuzzyOutputFuzzy Output
CrispInputCrisp
Input
..
.. Logic Systems Laboratory − Swiss Federal Institute of Technology
Carlos Andres Pena Reyes
Fuzzy inference systems
Gas
LowOR
Pressure
Temp.
&
&
&
High
Hot
Cold
Low
ORHigh
.
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
.
Carlos Andrés Peña−Reyes
. .
Network−like view of a fuzzy system
Membership function values
Gas
OR
Hot
Cold
Low
High
Pressure
Temp.
&
Low
&
&
HighOR
. .Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
.
Operational parameters.
RulesConsequents
Weights
Gas
Pressure
Temp.
Antecedents
Low
OR
Hot
Cold
High
&
&
&
High
LowOR
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
.
.
Connective parameters
.
.
Low
{ Membership functionsRulesRelevant variables
Pressure
Temp.
Number of
OR
Hot
Cold
High
Low
Gas
High
&
&
OR
&
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
. .
. .
Structural parameters
Pressure
Defuzzification methodMembership function typesFuzzy operatorsReasoning mechanism&
&
&
Temp.
Low
High
OR
ORGas
Low
Hot
Cold
High
.
.
. Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
.
Logical parameters
Parameters of a fuzzy system
Relevant variables
Number of membership functions
Number of rules
Consequents of rules
Defuzzification method
Antecedents of rules
Operational
Connection
Fuzzy operatorsLogic
Class
Membership function types
Membership function values
Rule weights
Structural
Database
Rulebase
Knowledge base
Defuzzifier
Fuzzi- and defuzzifier
Inference engine
ComponentParameters
..
..
Carlos Andrés Peña-Reyes
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
Reasoning mechanism
If Tempearture is WARM then Ventilator is Low
If Temperature is HOT then Ventilator is Medium
InterpretabilityPrecision
Linguistic
... Let’s go to the lake!!!
If Temperature is VERY−HOT then Ventilator is High
If Temperature is HELLISH then Ventilator is Off, and...
If Temperature is COOL then Ventilator is Off
Numeric
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
.
Dual external nature
. .
.
Universal approximator
Numeric mapping: Crisp inputs / Crisp outputs
Uncertainty management: noise and low quality of data
Nonlinear behavior, but linearity not excluded
●
●
●
●
.
..
Carlos Andrés Peña-Reyes
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
.
Numeric issues
Number of elements: Compatible with human capabilities
●
●
●
●
●
Distinguishability: Each linguistic label has semantic meaning
Coverage: Any element belongs to at least one fuzzy set
Normalization: At least one element has unitary membership
Complementarity: For each element, the sum of memberships is one
Semantics: the study of meanings
0
1Cold Cool Warm
Temperature
Hot
0
1Cold
Temperature
HotCool Warm
..
.. Logic Systems Laboratory − Swiss Federal Institute of Technology
Carlos Andres Pena Reyes
Interpretability considerations: semantic criteria
●
●
Consistency: rules firing simultaneously must have similar consequents
Rule readability: small number of conditions in rule antecedents
Rule−base simplicity: Set of rules as small as possible
●
Completeness: for any input, at least one rule must fire●
Syntax: the way in which linguistic elements are put together
5R
0RR2
R
R
1R
R A
7R 9R
5
R
4R 6RR
1R 3R2R
RAR4
RB B
5
8
.
.. Logic Systems Laboratory − Swiss Federal Institute of Technology
Carlos Andres Pena Reyes
Interpretability considerations: syntactic criteria.
●
●
Linguistic labels shared by all rules
Normal, orthogonal membership functions
Default rule
●
●
Don’t care conditions
5 R
0R
A
RB
R
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology. .
. .
17 20 26 29
Cold Warm Hot
Strategies to satisfy interpretability criteria
What do you know about the modeled system?
Have you preferences or restrictions to the model?
How do you search?
Search methodi.e. do a well suited and/or well known technique exists?
i.e. what is predefined and what looked for?Search space
Constraintsi.e. do issues like size, speed, or simplicity matter?
..
..
The general modeling problem
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Database
Rule base
InferenceEngine Defuzzifier
some parameter pre−definition is thus required.The number of parameters is too high to perform a full search,
and, or, not, ...
f1, ... , fm
P1 P2 P3 P4
R1, ... , Rn
If V1 is Low AND ....
According with the searched parameters we can have:
System design.
Structural parameters:
Behavior learning.Connective parameters:
Operational parameters:
Logical parameters:
Knowledge tuning.
Structure learning.
..
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Knowledge base
Fuzzifier
. .
Search space in fuzzy modeling
"Classic" identification methods
Knowledge engineering
Evolutionary fuzzy modeling techniques
Neuro−fuzzy systems
Machine learning approaches
.
..
.
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
Search methods: Fuzzy modeling techniques
− Diagnostic: Overall performance, sensitivity, specificity
− Data mining: Completness, complexity.
− Availability: Continuity of explanations (time to provide them)− Interpretability: Allowed complexity.
Who is going to interact with the system?
− Control: Dynamic response, adaptability, robustness, etc.
− Speed: Real−time constraints, computing resources.
− Size: Available memory, computing platform.
− Classification: Classification performance, quadratic error.
What is the fuzzy system expected to do?
How is the system expected to do it?
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
.
Usual constraints in fuzzy modeling
. .
.
all rules share the same MFs
null and unity membership
Rule−specific MFs are not allowed
Orthogonal MFs with well defined
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
.
. .
.
Interpretability−related constraints
Fuzzy modeling: direct approach
This approach is also called knowledge engineering
Domainexpert
Knowledgeengineer
Fuzzymodel
Design loop
Validation loop
Domain expert
Fuzzy modeling: data-driven approaches
This approaches are also denominatedknowledge discovering
Fuzzymodel
Design loop
Validation loop
Domain data Building algorithm
Neuro−fuzzy systems
Fuzzy system
Identification−based
Constructive−learningFuzzy system
Fuzzy system
DataOperational
Connective
Structural
Structural
Connective
Logicdesign
algorithmEstimation
Operational
ANN−like training algorithm
Human
. .
. .
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology
Fuzzy modeling: some data−driven approaches
- Genome encodes values P, Q, and C
- Knowledge is tuned by evolution,
which searches for membership function values
- Rules of type:
if X1=Low and X2=Normal then Output = Ci
- Fixed rule base (completness)X2
X1
R9
R8R5
R4
R6
R7
R3
R1
R2
P3P2P1
Q2
Low Mid High
Q3
Big
Nor
mal
Sm
all
Q1
(3*P + 3*Q + 9*C) * 5 bits = 75 bits
..
..
Carlos Andrés Peña-Reyes
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
Evolutionary knowledge tuning (database)
Don’t care conditions and default rule
- Three main approches to evolutionary behavior learning
- Evolution can be used to find a minimal (or fixed size) rule base,
(i.e. fuzzy system)
- Two strategies for reducing this number:
- Number of rules exploses rapidly
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
RiR3
R2
Ra, Rb, Rc ...
R1
Iterative Rule Learning
Incremental construction ofthe knowledge base
Evolution finds the best rule
Carlos Andrés Peña-Reyes
R1
Michigan
Individual = One rule
Population = Rule base
.
R2
Pittsburgh
(rule base or knowledge base)Individual = Entire system
Population of systems
.
.
Ri
Evolutionary behavior learning (rule base)
R3
.
- Genome encodes rules: Antecedents and consecuents
- Evolution searches for a subset of N rules (fixed by the designer)
5 functions + 3 don’t care
- Space of 625 rules (1295 including don’t care conditions)
- 4 input variables, 5 membership functions per variable{Tiny, Small, Normal, Big, Huge}
IF V1 is Tiny AND ... AND V4 is Normal then Out = Huge
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
R2 .... Ri
Carlos Andrés Peña-Reyes
RnR1 ....
5 * 3 bits
15 bits
N rules * 15 bits
A1 A3 CA4A2
Evolutionary behavior learning: An example
. .
. .
Behavior learning
Evolutionary knowledge base learning
attributeFuzzy systemType of
valuesquantityUsual
type
Operational
Connective
class
Knowledge tuning 10 - 1000 Real-valued Database
Modeling
Symbolic Rule base10 - 1000
Parameter
- Computation requirements
- Parameter representation
- Size of the search space
- Tight interdependency
(Knowledge base = rule base + database)
Critical issues for applying evolution:
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña-Reyes
. .
. .
Evolutionary knowledge base learning
Sample ru;e: IF V1 is Low and V4 is High THEN Diagnostic is Benign
- A basic approach: Single population, single evolution
Example: Breast cancer diagnosis problem (Peña and Sipper 99)
- Genome encodes: Rule antecedents and membership function parameters
- A simple genetic algorithm searches for the knowledge base
- 9 inputs, 1 output, 2 membership functions per variable
Ai A9....A1 ....
9 antecedents * 2 bits
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
18 bits
P d
Carlos Andrés Peña-Reyes
3 bits 3 bits
....
6 bits
.... Ri .... RnV1 V2 Vi .... V9
.
9 Variables * 6 bits Nr rules * 18 bits
Low
dP
High
.
R2R1
.
.
Evolutionary knowledge base learning
1
Symbolic part: Rule baseNumeric part: Database
- A variation: Single population, double evolution
- The rule base is evolved using genetic programming
Example: Evolving fuzzy rule based classifiers with GA–P (García et al. 99)
- Genome encodes: Complete rule base and membership function parameters
- A simple genetic algorithm searches for the database
00 0 1 0 0 0 1 1 0
.. Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
. .
Carlos Andrés Peña-Reyes
X1
X2
Evolutionary knowledge base learning
Q3
Sm
all
Q1
Big
Low Mid High
Q2
Nor
mal R2
A fuzzy self-organizing map searches for P and Q values
Genome encodes rules: Antecedents and consequent
(J.-F. Philagor, student project SPG, 1999)Example: Breast cancer diagnosis- Evolution searches for a fixed-size rule base
- Hybrid learning: Evolved rule base, learned database
P1 P2 P3
R3R1
- Database is tuned using a neuro-fuzzy approach
R1 R2
.
Ri .... Rn....
A1 CA2 A9. . .
9 * 2 bits + 1 bit
19 bits
N rules * 19 bits
. .
.
Carlos Andrés Peña-Reyes
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
..
. .
The
test
Carlos Andrés Peña-Reyes
The
feat
ures
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
The
dat
abas
e
The Wisconsin Breast Cancer Database
.
. .
Carlos Andrés Peña-Reyes
... ...
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne... ...
else (Output is Malignant)
R1: if (V1 is Low) and (V2 is High) and ... and (V9 is Low) then (output is Benign)R2: if (V1 is Low) and (V2 is Low) and ... and (V9 is None) then (output is Benign)
...Malignant
P1+d1P1 P P2 +d +dP P2 2 9 9
.
9
Low
High
Low
LowLow
None
.....
.....
Benign
Benign
VV V1 2 9
Proposed Fuzzy System
.
.
.
Carlos Andrés Peña-Reyes
Ai = 0 or 3 (Variable not assigned)d = [1;8]
P = [1;8]
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
Ai = 1 (Benign)
d
Ai = 2 (Malignant)
.
3 bits 3 bits
P Ai A9....A1 ....
.... R1 R2 .... Ri .... RnV1 V2 Vi .... V9
9 antecedents * 2 bits
9 Variables * 6 bits Nr rules * 18 bits
6 bits 18 bits
Total genome length = 54 + 18Nr bits
Genome encoding
Fv : Number of variables
selection pressure to fine tune parameters
measures the interpretability
the most important performance measure
Fe : Quadratic error
Fc : Classification performance,
F = Fc + a* Fv + b*Fe
Carlos Andrés Peña-Reyes
.
. Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne .
.
Fitness function
97.36% (3)
97.80% (4.8)
97.80% (4.7)
97.07% (4)
96.65% (7)
96.35% (3)
Learned Boolean rules Evolved fuzzy rules
97.51% (3.4)
1
2
3
5
4
96.19% (1.8)
Setiono Tahawork (99)
97.21% (4)97.14% (4)
95.42% (2)
Results: Classification performance (Number of variables)
RulesPeña
Setiono (96) Liu (96)This
Sipper (98)Ghosh (97)
.
.
.
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña-Reyes
.
IF the clump of cells is not very thick,
ELSE the case is malignant.
THEN the case is benign;
AND nucleoli are not highly abnormal,
AND there are few bare nuclei,
AND the cell’s size is quite uniform,
The best single-rule system
Carlos Andrés Peña-Reyes
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne
Benign
Malignant
v1 v2
Low
v6
Low
v8
Low
. .
.
Low
.
2 Cooperative coevolutionA building algorithm:
1 Fuzzy systemsA system model:
Fuzzy
CoCo
Database
. .
. .
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology
Proposed approach: two elements
represented by different types of values,they can be decomposed in distinct components,
and which are very interdependent.
fuzzy modeling an These features render
COOPERATIVEadequate target for
COEVOLUTION
3 − 10Logical
The required solutions can be very complex,
Real−valued10 − 1000Knowledge tuningOperational (labels)
System design
Symbolic10 − 1000Behavior learningConnective (rules)
Integer5 − 20Structure learningStructural (size)
classParameter
typeModeling
numberUsual Type of
values
. .
. .
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology
Fuzzy modeling: a coevolvable problem
− Better search power− Lesser computational cost− More−flexible setup
Two evolutionary algorithmssearching for:
membership functions
and rules.
Advantages:− Divide−and−conquer strategy
Modification Modification
Selection
EvaluationEvaluation
Selection
Membership functions Rules
.
. .
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
.
Fuzzy CoCo:A cooperative coevolutionary approach to fuzzy modeling
to form fuzzy systems.2. Individuals are combined with cooperators
individual fitness is then calculated.3. These fuzzy systems are evaluated, and
both fitness−dependent andare selected from generation g−1
1. Cooperators for generation g
randomly
Rules
Fitness
Cooperators
Cooperators
Fitness
MFsRules
Gen
erat
ion
g
g−1
Selected cooperators
MFs
.
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
.
Fitness evaluation in Fuzzy CoCo
●
●
●
●
● Linguistic fitness: when used, increases selective pressure for interpretability
Normal, orthogonal membership functions: constrained representation
Shared membership functions: reinforced by the existence of a separate species
Default rule: guarantees complete coverage of the input space
Don’t care conditions: encourage shorter rules
R5 RA
R
0
B
R
0
1Cold Cool Warm
Temperature
Hot
..
.. Logic Systems Laboratory − Swiss Federal Institute of Technology
Carlos Andres Pena Reyes
Interpretability strategies in Fuzzy CoCo
5.9
3.0
3.0150 Virginica
Classes(1) setosa
(3) virginica(2) versicolor
(1) SL Sepal length(2) SW Sepal width(3) PL Petal length(4) PW Petal width
Features
2 Setosa
The
var
iabl
es
5.14.9
1 3.5 Setosa1.41.4
5.1
0.20.2
1.8
Case ClassSL SW PL PW
. Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
.
. .
The
dat
abas
e
Fisher’s Iris Data
.
..
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
.
Iris: Variable analysis
Iris proposed solution: Controller−type
Class ClassInputSubsystem
Fuzzy Subsystem
FuzzyInput
Fuzzy
CrispOutput
Output
InputCrisp Fuzzifier Defuzzifier
Knowledge base
Rule base
Database
EngineInference
The selection unit approximates it to the nearest class
The fuzzy subsystem estimates a continuous "class" value
estimation
Stair−function
Carlos Andrés Peña−Reyes
. .
.
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
.
1 2
21
1 2 3
3
3
... ... ...
R1: if (SL is A11) and (SW is A12) and (PL is A13) and (PW is A14) then (output is Class1)
R2: if (SL is A21) and (SW is A22) and (PL is A23) and (PW is A24) then (output is Class2)
Rn: if (SL is An1) and (SW is An2) and (PL is An3) and (PW is An4) then (output is Classn)
else (Output is Class0)
setosa
Logic Systems Laboratory − Swiss Federal Institute of Technology
...
Carlos Andres Pena Reyes
11 P21 P31 P12 P22 P32 P24P14
.
.
...
P
versicolor
.
virginica
High
Low
LowLow
None
.....
.....
Medium
SL SW PW
.
Iris controller−type: Proposed Fuzzy System
(setosa)
Iris proposed solution: Classifier−type
Threshold SubsystemClassInput
(versicolor)µ
µ
(virginica)µFuzzyInput
Fuzzy
CrispOutput
Output
InputCrisp Fuzzifier Defuzzifier
Knowledge base
Rule base
Database
EngineInference
Fuzzy Subsystem
The selection unit chooses the most active class
value for each classThe fuzzy subsystem estimates a continuous membership
Maximum and
. .
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
..
Yes
No
No
NoYes Yes
Yes Yes
Yes
YesNo YesNo
NoNoNo Yes
No
31 P24P14P
None
21P11P
Carlos Andres Pena Reyes
.Logic Systems Laboratory − Swiss Federal Institute of Technology
Low
.....
.....
versicolor
PW
setosa
.
virginicaversicolorsetosa
virginica
virginica
.
versicolorsetosa
Low
Medium
SL
... ... ... ... ...
R1: if (SL is A11) and ... and (PW is A14) then (setosa is Yes),(versicolor is No),(virginica is No)
R2: if (SL is A21) and ... and (PW is A24) then (setosa is No),(versicolor is Yes),(virginica is Yes)
Rn: if (SL is An1) and ... and (PW is An4) then (setosa is No),(versicolor is No),(virginica is Yes)
else (setosa is No),(versicolor is Yes),(virginica is No)
.
Iris classifier−type: Proposed Fuzzy System
Membership functions Rules (Controller/Classifier)
P
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
3 x 5 bits
P1 P3
55 5
P2
.
SWSL SW
.
.
Genome length = 60 bits
.
PL R1 ... Ri ... Rn Co
....A1 Ci
Nr rules * 19 bits
A4
4 * 2 bits
2/3 bits
2/3 bits
10/11 bits
4 Variables * 15 bits
Genome length = 10/11*Nr + 2/3 bits
Iris: the genomes
"Fit" cooperatorsRandom cooperators
1. Fitness function
Population size
Fv : Number of variables
1
1
{60, 70}500 + 100*Nr
{0.02, 0.05, 0.1}{0.1, 0.2}
{1, 2}
Maximum generationsCrossover probabilityMutation probabilityElitism rate
measures the interpretability
encourages not−so−bad errorsFm : 1 − mse (mean square error)
the most important
2. Fuzzy CoCo parameters
Fc : Classification performance,
F = Fc * Fm
b(Fc + a*Fv) * Fm{ b
.
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne .
. .
Iris: Fuzzy CoCo set−up
ICANNGA’01
8 97.3 % (2)
Iris results: classification (average rule)
Fuzzy CoCoNeurofuzzyConstructive Learning Methods
2 99.33% (2)
3 100 % (1.7)
4 98 % (2.6) 100 % (2.5)
5 100 % (3.3)
Wu (99)
RulesSimple GA
Shi et al (1999)FuGeNeSysRusso (1998)
Fuzzy CoCo
Hong (00) Hung (99) ICANNGA’01
2 98 % (1.5)
99.33% (2.3)3 96.2 % (4)
4 99.33% (2)97.4 % (4)
Rules
Con
trol
ler
Cla
ssifi
er
.
.
.
.Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
PWPLSWSL
PWPLSWSL
ClassPL PW
Class
SW
Class
SL
Class
.
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology ..
.
Iris controller−type: A three−rule system
setosa versic.
setosaPWPLSWSL
versic.setosa virgin.PWPLSL
virgin.
SL SW PL PW setosa versic. virgin.
virgin.
SW
versic.
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology
.
.
.
.
Iris classifier−type: A three−rule system
Fuzzy Subsystem
malignant
benign
DiagnosticInput Threshold SubsystemAppraisal
The
dat
abas
e
The
tes
t
..
..
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Breast cancer diagnosis: the WBCD problem
solu
tion
Pro
pose
d
RulesMembership functions
P d
6 bits
V9.... ....ViV2
Genome length = 54 bits
3 bits
9 Variables * 6 bits
Ai = 0 or 3 (None)
Ai = 1 (Low)
d = [1;8]
3 bits
Ci = 1 (Benign)P = [1;8]
.
. .
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
V1
Ai = 2 (Malignant)Ai = 2 (High)
.
Genome length = 19*Nr + 1 bits
Nr rules * 19 bits
19 bits 1 bit
1 bit9 * 2 bits
CiA9A1 ....
CoRn...Ri...R1
The genomes
"Fit" cooperatorsElitism rateMutation probabilityCrossover probabilityMaximum generations
Random cooperators
11000 + 100*Nr
[30−90]
{1,2,3,4}1
{0.1−0.6]
Fuzzy CoCo parameters
[0.02−0.3]
Population size
measures the interpretabilityFv : Number of variables
the most important performance measureFc : Classification performance,
F = Fc − a* Fv
Fitness function
. .
. .
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
Fuzzy CoCo set−up
98.98% (5)
98.83% (5)
98.83% (5)
98.68% (3)
98.54% (4)
98.54% (5)
97.36% (4)
97.73% (3.9)
97.91% (4.4)
98.12% (4.2)
98.18% (4.6)
98.18% (4.3)
97.36% (4.0)
98.25% (4.7)
Evolved fuzzy rulesLearned Boolean rules
Fuzzy CoCo − IEEE TFS 2001AIM 1999
Fuzzy−geneticNeuroRuleSetiono (2000)
Rules
97.36% (4)1
97.36% (3)2
97.80% (6)98.10% (4)3
97.80% (−)4
97.51% (−)98.24% (5)5
98.10% (−)6
97.95% (−)7
Best
97.07% (4)
Average
WBCD results: classification (longest rule)
. .
. .
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
Carlos Andrés Peña−Reyes
Classification rate = 98.54%
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne .
if (v1 is Low) and (v3 is Low) and (v5 is Low) then (output is Benign)
v
if (v1 is Low) and (v4 is Low) and (v6 is Low) and (v8 is Low) and (v9 is Low) then (output is Benign)
else (output is Malignant)
1 3
v
.
v6 v8
v
v94 v5
.
Two−rules evolved system
.
Five-rule systems: 900.000 fitness evaluations
Five-rule systems: 480.000 fitness evaluations
Single-rule systems: 352.000 fitness evaluations
32.000 * (1000 + 100Nr) {worst case, Ncr=3}
Single-rule systems: 500.000 fitness evaluations
200 * (2000 + 500Nr)Number of fitness evaluations = Np * Gmax
Fuzzy GA: Single population (Peña & Sipper 99)
Computing requirements
Fuzzy CoCo: Cooperative coevolution (CEC-2000)
Number of fitness evaluations = 2 * Np * Gmax * (Ncf + Ncr)
Carlos Andrés Peña-Reyes
Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne. .
. .
computer−assistedCOBRA system:
case interpretationreadingprotocol
mammogram
biopsy
recommendation329 benign (neg)187 malignant (pos)
516 readings{Database
Logic Systems Laboratory − Swiss Federal Institute of Technology
Carlos Andres Pena Reyes
. .
. .
The problem: mammography interpretation
input
Fuzzy system
Proposal
Reading form
appraisal
Diagnostic decision unit
BiopsyReading
Database
Web−based user interface
Malignancy
Carlos Andres Pena Reyes
Threshold unit
Logic Systems Laboratory − Swiss Federal Institute of Technology. .
. .
COBRA system: internal view
BinaryContinuousDiscrete
438
Variable type Number
.
. Logic Systems Laboratory − Swiss Federal Institute of Technology
Carlos Andres Pena Reyes
.
.
Understanding the database
not encoded
Continuous variables (e.g., V1):3 var. x 2 par. x 7 bits = 42 bits
Discrete variables (e.g., V3):8 var. x 2 par. x 4 bits = 64 bits
Binary variables (e.g., V2):
Total genome length = 106 bits
Logic Systems Laboratory − Swiss Federal Institute of Technology
.
. .
NoneLow
.
High
Carlos Andres Pena Reyes
Benign Malignant
V V15V3
P1 P15 P’15P’1 P3 P’3
Ri: if (v1 is Ai1) and (v2 is Ai2) and (v3 is Ai3) and ... and (v15 is Ai15) then (output is Ci)
DB
V1 Vi ........ V15V2
.....
Pi P’i
1
Genome encoding for linguistic labels
1 1
20
22
1
Clinical
120
11222222
if Sr = 0
if Sr = 1
+ Radiological
Total genome length = 20 x Nr +1
...Ri...R1
A3A2A1
RB
A15
Co
A11 A12
A9A8A7A4 A5 A6
Rn
Logic Systems Laboratory − Swiss Federal Institute of Technology
Ar2 ... Ar6 Sr CAc1 Ac2 Ac3 Ar1
Carlos Andres Pena Reyes
A10 A14
. .
. .
A13
Genome encoding for rules
Specificity TN
Basic fitness (Fbase)
Accuracy reinforcement
(note: done only if Accuracy > 0.7)
Performance measures and fitness function
αSensitivity + Specificity1 + α
1 + βFbase + Accuracyβ
SensitivityTP
TP + FN
TN + FP
AccuracyTP + TN
TP+TN+FP+FN
TPTP + FN
PPV
..
Carlos Andrés Peña−Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
..
0.82 0.83 0.84 0.85 0.86 0.87 0.88 0.89 0.9 0.91 0.920
5
10
15
20
25
2.502.22Vr
171712
Reff
0.9154
90.89780.8910Fitness
0.9109201510Nr
Best individual
2.4125
2.622.52Vr
15.7814.1512.039.17Reff
2.760.89340.87860.8754Fitness
25201510Nr
Average per class
2.59
2.70
0.8947
5
14
13
2
4
14
22
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology
.
..
.
Fuzzy CoCo results on 65 runs
395/516211/329184/187
Ratio
186/289412/516226/329186/187
Figure
76.55%60.93%
98.40%
Figure
64.36%79.84%68.69%99.47%
Measure
PPV
64.13%SpecificitySensitivity
184/302
Ratio
Accuracy
9−rule17−rule
. .
. .
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology
Performance of two selected systems
184/302395/516
Ratio
63.22%
Threshold = 2 Threshold = 3
SensitivitySpecificityAccuracyPPV
Measure
98.40%64.13%76.55%60.93%
Figure
184/187211/329
187/187208/329
76.55%60.71%
Figure
100.0%
395/516187/308
Ratio
Carlos Andres Pena Reyes
. .
.
Logic Systems Laboratory − Swiss Federal Institute of Technology
.
The 9−rule system with two different thresholds
.
. .
Carlos Andres Pena Reyes
Logic Systems Laboratory − Swiss Federal Institute of Technology.
COBRA system: reading form
Recommended