Ppsn Fuzzy

8/12/2019 Ppsn Fuzzy

1/10

Learning Fuzzy Rules with EvolutionaryAlgorithms an Analytic Approach

Jens Kroeske 1 , Adam Ghandar 2 , Zbigniew Michalewicz 3 , and Frank Neumann 4

1 School of Mathematics, University of Adelaide, Adelaide, SA 5005, Australia2 School of Computer Science, University of Adelaide, Adelaide, SA 5005, Australia

3 School of Computer Science, University of Adelaide, Adelaide, SA 5005, Australia;also at Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21,

01-237 Warsaw, Poland, and Polish-Japanese Institute of Information Technology, ul.Koszykowa 86, 02-008 Warsaw, Poland

4 Max-Planck-Institut f ur Informatik, 66123 Saarbr ucken, Germany

Abstract. This paper provides an analytical approach to fuzzy rulebase optimization. While most research in the area has been done ex-

perimentally, our theoretical considerations give new insights to the task.Using the symmetry that is inherent in our formulation, we show that theproblem of nding an optimal rule base can be reduced to solving a setof quadratic equations that generically have a one dimensional solutionspace. This alternate problem specication can enable new approachesfor rule base optimization.

1 Introduction

Fuzzy rule based solution representations combined with evolutionary algorithmsare a powerful real world problem solving technique, for example see [2, 5, 9, 15,

18,19]. Fuzzy logic provides benets in naturally representing real world quan-tities and relationships, fast controller adaptation, and a high capacity for so-lutions to be interpreted. The typical scenario involves using an evolutionaryalgorithm to nd optimum rule bases with respect to some application specicevaluation function, see [7,11,13].

A fuzzy rule is a causal statement that has an if -then format. The if part isa series of conjunctions describing properties of some linguistic variables usingfuzzy sets that, if observed, give rise to the then part. The then part is a valuethat reects the consequence given the case that the if part occurs in full. Arule base consists of several such rules and is able to be evaluated using fuzzyoperators to obtain a value given the (possibly partial) fullment of each rule.

Membership functions are a crucial part of the denition as they dene themappings to assign meaning to input data. They map crisp input observationsof linguistic variables to degrees of membership in some fuzzy sets to describeproperties of the linguistic variables. Suitable membership functions are designeddepending on the specic characteristics of the linguistic variables as well as pe-culiar properties related to their use in optimization systems. Triangular mem-bership functions are widely used primarily for the reasons described in [16].


2/10

Other common mappings include gaussian [11] and trapezoidal [8] member-ship functions. The functions are either predened or determined in part or com-pletely during an optimization process. A number of different techniques havebeen used for this task including statistical methods [7], heuristic approaches[2], and genetic and evolutionary algorithms [5, 9, 14,18]. Adjusting membershipfunctions during optimization is discussed in [9,20].

A nancial computational intelligence system for portfolio management isdescribed in [7]. Fuzzy rule bases are optimized in an evolutionary process tond rules for selecting stocks to trade. A rule base that could be produced usingthis system could look as follows:

If Price to Earnings Ratio is Extremely Low then rating = 0 .9 If Price Change is High and Double Moving Average Sell is Very High then

rating = 0 .4

The if part in this case species some nancial accounting measures (Price to

Earnings ratio) and technical indicators [1] used by nancial analysts; the outputof the rule base is a combined rating that allows stocks to be compared relative toeach other. In that system rule bases were evaluated in the evolutionary processusing a function based on a trading simulation.

The task of constructing rule base solutions includes determining rule state-ments, membership functions (including the number of distinct membership setsand their specic forms) and possible outputs. These parameters and the spec-ication of data structures for computational representation have a signicantimpact on the characteristics and performance of the optimization process. Pre-vious research in applications [8, 17, 1] has largely consisted and relied upon ex-perimental analysis and intutition for designs and parameter settings. This papertakes a theoretical approach to the analysis of a specic design of a fuzzy rulebase optimization system that has been used in a range of successful applications

[6,7, 11,13]; we utilize the symetries that are inherent in the formulation to gaininsight into the optimization. This leads to an interesting alternate viewpoint of the problem that may in turn lead to new approaches.

In particular, our formal denition and framework for the fuzzy rule baseturns the optimization problem into a smooth problem that can be analyzed an-alytically. This analysis reduces the problem to a system of quadratic equationswhose solution space has the surprising property that it generically contains awhole line. It should be possible to utilize this fact in the construction of fastand efficient solvers, which will be an important application of this research. Theapproach in this paper builds on experimental research presented in [7,6], butit should be noted that a number of other mechanisms have been proposed forencoding fuzzy rules [8].

The methods we consider could be used in an evaluation process where theerror is minimized with respect to tting rule bases to some training data in the context of the above example this would allow a system to learn ruleswith an output that is directly calculated from the data. For example a rulebase evaluated in this way could be used to forecast the probability that a stockhas positive price movement [10,12] in some future time period. A rule in such a


3/10

rule base could look like: If Price to Earnings Ratio is Extremely Low and Double Moving Average Buy is Very High then probability of positive price movement is0.75. In this case the training data set would be some historical stock marketdata similar to that used in [6, 7].

The structure of this paper is as follows: Section 2 contains the formal de-nitions for the analysis presented in Section 3. Section 4 concludes the paper.

2 Approach

In this section we introduce the formulation of the models used in the analy-sis, including the rule base solution representation, the rule base interpretationmethod and the evaluation function.

2.1 Rule Base Solution Representation and Interpretation

Let us introduce some precise denitions of what is meant by the rule base solu-tion representation. First of all, we are given L linguistic variables {A1 ,...,A L }.Each linguistic variable A i has M i linguistic descriptions {A i1 ,...,A iM i } that arerepresented by triangular membership functions ij , j = 1 ,...,M i . A fuzzy rulehas the form

If Ai 1 is A i 1j 1 and Ai 2 is A i 2j 2 and and A

i k is A i kj k then o, (1)

where i 1 , ...i k {1,...,L }, j k {1,...,M i k } and o [0, 1].A rule base is a set of several rules. Let us assume that we are given a rule

base consisting of n rules:

If Ai11 is A i

11

j 11and A i

12 is A i

12

j 12and and A i

1

k 1 is Ai 1k 1j 1k 1

then o1

If Ai21 is A i

21

j 21and A i

22 is A i

22

j 22and and A i

2

k 2 is Ai 2k 2j 2

k 2

then o2

......

If Ain1 is A i

n1

j n1 and Ai n2 is A i

n2

j n2 and and Ai nk n is A

i nk nj nk n

then on ,

where i ml {1,...,L } and j ml {1,...,M i ml }. Given a vector x RL of observed

values, whose components are values for the linguistic variables A1 ,...,A L , wecan evaluate the rule base as follows: the function describes the way the rulebase interprets data observations x to produce a single output value. This valuehas an application specic meaning and can be taken to be a real number (usuallynormalized to lie between zero and one). More precisely, is dened as follows:

: R L R

x =

x 1x 2

...x L

nm =1 o

m k ml =1

i mlj ml

(x iml )

nm =1 om

.


4/10

2.2 Evaluation Function

We consider an evaluation function (to minimize) that measures the error whentraining a rule base to t a given data set. This training data consists of a set{x i , y i }i =1 ...N , where each

x i =

x 1ix 2i...

x Li

is a vector that has as many components as there are linguistic variables, i.e. x i R L i = 1 ,...,N , and each yi is a real number, i.e. yi R i = 1 ,...,N . Thenthe evaluation function has the form

=N

i =1

((x i ) yi )2 (2)

=N

i =1

nj =1 a ij o

j

nj =1 oj

yi

2

, (3)

where

a sm =k m

l =1

iml

j ml(x i

ml

s ).

Our aim is to optimize the rules base in such a way that the evaluation function becomes minimal. This involves two separate problems. Firstly, the form of the

membership functions ij may be varied to obtain a better result. Secondly, therule base may be varied by choosing different rules or by varying the weights oi .

In this paper we will concentrate on the second problem, taking the form of themembership functions to be xed. For example, we can standardize the numberof membership functions for each linguistic variable Ai to be M i = 2n i 1 anddene

ij =

0 : x j 1

2 n i

2n i x + 1 j : x j 1

2 n i, j2 n i

2n i x + 1 + j : x j2 n i , j +1

2 n i

0 : x j +12 n i

for j = 1 , ..., 2n i 1 = M i . These functions are shown in Figure 1.Moreover, we can consider the number n of rules to be xed by either working

with a specic number of rules that we want to consider, or by taking n to bethe number of all possible rules (this number will be enormous, but each rulewhose optimal weight is zero, or sufficiently close to zero can just be ignored and


5/10

1

2 n i

2

2 n i 12 n i

12 n i

i1

i2

i3

iM i 1

iM i

Fig. 1. Membership Functions

most weights will be of that form), depending on the application. The resultingoptimization problem will be considered in 3.2.

3 Analysis

This section contains the detailed analysis of the problem described in Section 2.We rstly determine the maximum possible number of rules and then considerthe optimization problem for the evaluation function. As a result, we are ableto reduce the optimization problem to a system of equations (6), that has theremarkable property that it allows (generically) a one-dimensional solution space.This is the content of Theorem 1.

3.1 Search Space

The search space is the set of all potential rule base solutions. Let us rst of allcompute the maximum number of rules nmax that we can have. Each rule canbe written in the form

If A 1 is A 1j 1 and A2 is A 2j 2 and and A

L is A Lj L then o,

where in this case j i {0, 1,...,M i } and j i = 0 implies that the linguistic variableA i does not appear in the rule. Then we have

n max = ( M 1 + 1) (M 2 + 1) (M L + 1) 1.

Note that we have subtracted 1 to exclude the empty rule. If we include thepossible choices of weights oi with discretization oi {0, 1d , ..., 1}, then we havea system of

(d + 1) n max

possible rule bases.


6/10

3.2 Optimization Problem

In this subsection we will treat the optimization problem described in 2.2. We

have to take the training data {x i , y i }i =1 ...N and the various membership func-tions ij as given, so we can treat the various a ij as constants and simplify

(o) =N

i =1

nj =1 a ij o

j

nj =1 oj

yi

2

=N

i =1

nj =1 (a ij yi )

2 oj oj + 2 j


7/10

Therefore, if we assume that o1 = 0 or o2 = 0, then the optimal solution is

o1 = A22 A 21A 11 A 12 o2 .

This is a whole line that intersects zero in R 2 . This phenomena can be seenclearly in the following picture:

1.00.75

y 0.5

0.25

0.0

0.00.19

0.25

0.24

x

0.5

0.29

0.75

0.34

1.0

0.39

0.44

0.49

Fig. 2. Evaluation function for two rules

More than two rules If we have more than two rules, then the conditionsbecome

o q

= 0 n

i =1

n

j =1

(A iq A ij )oi oj = 0 , q = 1 ,...,n. (6)

Theorem 1. Generically, there exists a one-parameter family of solutions tothe system (6). Hence the space of extremal points for is a line in R n that passes through zero.

Proof. We will show that the n equations (6) are dependent, i.e. that we onlyneed to solve n 1 of these equations and the n -th equation then follows auto-


8/10

matically. For this purpose, we rewrite the system

n

i =1

n

j =1(A iq A ij )oi oj =

n

j = 1

j = q

oj (Aqq Aqj )oq + ( A jq A jj )oj

B qj+

n

j = 1

j = q

n

i = 1

i { q, j }

(A iq A ij )oi oj .

Note thatB qj = B jq .

Denote the q -th equation by E q . Using the equality above, we compute

n

k =1

ok E k =n

k =1

n

j = 1

j = q

B kj ok oj

=0+

n

k =1

n

j = 1

j = q

n

i = 1

i { q, j }

(A ik A ij )

C ijkoi oj ok

= 0 .

The last term vanishes due to the fact that the tensor C ijk is symmetric in the

index pair ( i, j ), symmetric in the index pair ( i, k ) and skew (i.e. anti-symmetric)in the index pair ( j, k ). Such a tensor has to vanish identically. It is hence suf-cient to solve (6) just for ( n 1) equations, the last equation is automaticallysatised.

Remark Given realistic training data, it is obvious that the extremal pointsthat lie in [0 , 1]n will be minimal.

4 Conclusions and Future Work

We have successfully reduced the problem of nding optimal weights oi for a rulebase (given an arbitrary set of training data points) to a system of n equations forn unknowns, where n is the number of rules. Moreover, we have shown that thespace of extremal points for the evaluation function is a line through the origin inR n . Hence a genetic algorithms will be able to nd an optimal solution in [0 , 1]n

using well-established and fast methods [3, 4]. The reason for this, somewhatsurprising, result lies in the specic form of our rule base formulation: not the


9/10

values of the weights themselves are important, but the relationship that theyhave with respect to each other. Mathematically speaking, the optimal solutiono is really an element of ( n 1)-dimensional projective space RP n 1 , rather thatan element of R n .

As a result, it is possible to use the analysis in this paper to design an op-timization process in which combinations of rules consisting of the if parts areselected and then evaluated using a fast algorithm to nd the optimal then parts(output weights) to produce a rule base. This would be benecial in a range of applications as mentioned in the introduction. For example, reducing the sizeof the search space by removing the assignment of output weights from thegeneral rule base search problem; or by xing an initial structure for the rules(using knowledge from the application domain or because of specic applica-tion requirements) that feasible solutions should contain and then redening thesearch objective to extend this set rather than using a free search this is forinstance a very useful feature in nancial applications [7]. As a part of this fur-

ther research, we will also examine, combinatorially and empirically, algorithmsand genotype representations that utilize the reduction in complexity that arisesfrom the analysis in this paper.

References

1. F. Allen and R. Karjalainen. Using genetic algorithms to nd technical tradingrules. Journal of Financial Economics , 51:245271, 1999.

2. A. Bai. Determining fuzzy membership functions with tabu search an applicationto control. Fuzzy Sets and Systems , 139(1):209225, 2003.

3. N. Courtois, L. Goubin, W. Meier, and J.-D. Tacier. Solving underdened systemsof multivariate quadratic equations. In PKC 02: Proceedings of the 5th Inter-national Workshop on Practice and Theory in Public Key Cryptosystems , pages

211227. Springer-Verlag, 2002.4. L. D. (ed). Handbook of genetic algorithms. 1991.5. A. Esmin and G. Lambert-Torres. Evolutionary computation based fuzzy mem-

bership functions optimization. In Systems, Man and Cybernetics ISIC IEEE In-ternational Conference on , pages 823828. IEEE, 2007.

6. A. Ghandar, Z. Michalewicz, M. Schmidt, T.-D. To, and R. Zurbruegg. A com-putational intelligence portfolio construction system for equity market trading. InCEC 2007: Proceedings of the 2007 IEEE Congress on Evolutionary Computation .IEEE Press, 2007.

7. A. Ghandar, Z. Michalewicz, M. Schmidt, T.-D. To, and R. Zurbruegg. Compu-tational intelligence for evolving trading rules. To appear in IEEE Transactions On Evolutionary Computation, Special Issue on Evolutionary Computation for Fi-nance and Economics , 2008.

8. J. Gomez. Evolution of fuzzy rule based classiers. In GECCO2004 Proceedings of the Genetic and Evolutionary Computation Conference , volume 3102, pages 11501161. Springer-Verlag, 2004.

9. N. R. Guo, T.-H. S. Li, and C.-L. Kuo. Design of hierarchical fuzzy model forclassication problem using gas. Comput. Ind. Eng. , 50(1):90104, 2006.

10. N. Jegadeesh and S. Titman. Protability of momentum strategies: An evaluationof alternative explanations. 56(2):699720, 2001.


10/10

11. R. Johnson, M. Melich, Z. Michalewicz, and M. Schmidt. Coevolutionary optimiza-tion of fuzzy logic intelligence for strategic decision support. IEEE Transactions on Evolutionary Computation , 9(6):682694, 2005.

12. M. Lo, A.W., H., and J. Wang. Foundations of technical analysis: Computationalalgorithms, statistical inference, and empirical implementation. 55(4):17051765,2000.

13. Z. Michalewicz, M. Schmidt, M. Michalewicz, and C. Chiriac. A decision-supportsystem based on computational intelligence: A case study. IEEE Intelligent Sys-tems , 20(4):4449, 2005.

14. M. Mucientes, L. Moreno, A. Bugarin, and S. Barro. Evolutionary learning of a fuzzy controller for wall-following behavior in mobile robotics. Soft Comput. ,10(10):881889, 2006.

15. B. Oduor and A. Anev. Evolving fuzzy systems. J. Comput. Small Coll. , 19(5):356357, 2004.

16. W. Pedrycz. Why triangular membership functions? Fuzzy Sets Syst. , 64(1):2130,1994.

17. E. H. Ruspini, P. P. Bonissone, and W. P. (eds.). Handbook of fuzzy computation.1998.

18. Y. Shi, R. Eberhart, and Y. Chen. Implementation of evolutionary fuzzy systems.IEEE Transactions on Fuzzy Systems , 7(2):109119, 1999.

19. H. Surmann and M. Maniadakis. Learning feed-forward and recurrent fuzzy sys-tems: a genetic approach. J. Syst. Archit. , 47(7):649662, 2001.

20. C.-J. Wu and G.-Y. Liu. A genetic approach for simultaneous design of membershipfunctions and fuzzy control rules. J. Intell. Robotics Syst. , 28(3):195211, 2000.

Documents

Ppsn Fuzzy