SEVENTEENTH NATIONAL RADIO SCIENCE CONFERENCE Feb. 22-24,2000, Minujiya University , Egypt.

NNFRM: NEURO-NEW FUZZY REASONING MODEL INTERPRETED AS GENERAL CASE OF FUZZY REASONING MODEL

prof: Dr. MAZHAR TAIEL Electrimi Engineering Dep"ent, faculty of engineering, A&xandria Universi& Egyp.

M0RQi)Zl)aED GAMAL ELDINAlYMEDABD ELMONEM AIexan&ia Petroleum Company, Aiexundria, Egypt.

ABSTRACT In this paper, an interpretation of the New Fuzy Reasoning Model (NFRM) iS developed. This

h t e r p r d o n makes the traditional Fuay Reasoning Model 0 a special case under certain conditions. In addition, a neural network is constructed to represent the NFRM. The proposed Neuro-new fuzzy reasoning model ("FRM) optimizes the parameters of the "RM by using the well-known backpropagation concept. The parameters to be optima4 are those of input "be f sh ip functions output membership fiurdion and relation matrix. The proposed "FRM is used to predict hture values of a chaotic time series, wfiich is considered a benchmark problem. It is shown that the proposed "FRM outperforms other modeling methods in prediction of this chaotic time series. The NNFRM used here has fewer adjustable parameters, than those used in other lnode4ingtechniques.

1. INTRODUCnON Fuzzy reasoning method and neural network are two modeling process approaches. They differ in the

way howledge is acquired, encoded internally, and represented. ~oughfuzzyreasoningmetbodsarewell suited for representing the imprecise human knowledge and reasonhg process, it can not learn &om examples. On the otber hand, neuralnetworkcanLearnadesirediaputoutputmappiogfi.omexamples,butitcan'ttenthe concepts or the rules of this process. A 4 network based fuzzysystemisasystetnthatusestheneural network to conshuct the f k z y system. A neural nehvorkbesed~systemisabletolearnrules,inputand output membership iimctions from examples. Many efforts have gone into the neural network implementation of fuzzy systems. This approach has been disused by number of researcbu?? [2-7]. R d y a new fiuty re%sonhg model @?RI@ was developed which turns out to be superior to conventional fuzzy reasoning model @RM) [I].

In this paper, an interpretation of NFzaM m a way that makes FRM a sjmdcaseundexcettain conditions was developed and a four layers neural network is constructed to implement the NFRM with the developed intetpretation. The proposed Neum-IGRM ("FRh@ uses the well-known concept of back propagation [8] to optimize the inputs and output membership W o n as well as to find the fuzzy relation matrix that relates the inputs to output. The initial values for all parametas may be initialized randomly or obtained from human experience.

11. FUZZY MODELS A. Fuzzy inference system

A block diagram representing a fiuzy inference system with crisp input and output is shown in figure 1. The

A fuzzification input intefiace, which transform the crisp inputs into degrees of match with linguistic

An inference mechanism, which perform the inference operation according to the rules to produce

A knowledge base that contains the definition values of inputs and output membership function in

fuzzy inference system consists of four functional blocks:

variable

output fuzzy variables fiom input fuay variables.

addition to the fuzzy ifthen-rules.

0

0

SEVENTEENTH NATIONAL RADIO SCIENCE CONFERENCE ' Feb. 22-24, 2000, MiniiJyu Utiiversity , Egypt.

A defkification output interface, which transforms the fuzzy result into a crisp output value.

Input crisp ..___

__ --__ 7 r-------

--..-A L--._I

Fuuification I lnfemce \ $Defuuification i-4 mechnismi 1 interface

4

i Knowledge j base

output -E.

crisp

Figure 1 ; Fuzzy inference system.

Assume X and Y to be input variables and Z to be the output variable, then define a set of N rules R, (If X is hand Y is B, then Z is C,) (1)

Where i =1,2 , N and A, ,Bl ,and C, are linguistic description terms of X,Y and Z respectively, which can be quantified by the fizzy subsets A*, ,B*, ,and C*, The N rules are all applied in parallel, so the sentence connective between them is "also" This conditional statement can be formalized in the form of fuzzy relation R(X,Y,Z) [9]

N

R=RIu Rzu U R,,= U RI (2)

RI = pA* I (x )@PB*, (Y)@ P c * I ( M x > Y A (3)

r=l

Where R, denotes the fuzzy relation between X,Y, and Z determined by the &I fuzzy rule namely,

Where @ is fuzzy implication operator [l], and k,( ) is the membership grade of the fuzzy subset s,* in the fuzzy linguistic set S,

Having established a fuzzy relation R(X,Y,Z) between input variables and output variable, the compositional rule of inference is then applied to infer the hzzy subset C for Z, given a fuzzy subset A and B for X and Y

Z=( AxB)oR( X, Y ,Z) (4)

Where o is a compositional operator For a complete set of possible fuzzy implication operators and fuzzy compositional operator, see [ 101

In this paper, the fuzzy implication operator is chosen to be algebraic product and the fizzy compositional operator to be algebraic sum.

In order to calculate the crisp value of the output linguistic variable Z, there are many defbzzification methods [9] In this paper, the center of area method of defizzification is chosen

%

B. The new fuzzy reasoning method and its interpretation The difference between the FRM and the NFRM is that elements in the fuzzy relation matnx R in the first

case may only take the value of either 1 or 0, in the second case these element may take any value in the interval [0, 11 In this paper, the interpretation of the NFRM is equivalent to another method called Analytical Hierarchy Process (AHP) for multiobjective evaluation [ I 11 In the AHP the elements of R matrix represents the weights that the decision maker gives to each of the linguistic terms of inputs that is

SEVENTEENTH NATtONAL RADIO SCIENCE CONFERENCE ~

Feb. 22-24 ,2000, Miitrrfya University ;Egypt.

nk

Where pk ( v l ) is the membership value of the number 1 linguistic description term of input variable k, nk is the number of linguistic terms of input variable k and i is the number of output linguistic variable. It is noted that in (5) the composition rule of inference is taken to be algebraic product instead of min in [ 1 11. Taking the fuzzy implication operator and fuzzy compositional operator as described above, and with the descried interpretation of NFRM; the FRM with the same operators is just a special case of the NFRM when wij in (5) takes the value of either 1 or 0.

To illustrate this consider the following rule If x is A1 and y is B1 then z is C1 (6)

This is a FRM rule and is illustrated with Mamdani [12] model of inference in figure 2. The general NFRM rule as interpreted in this paper is

If x is A1 with weight wI1 and y is Blwith weight wIz then z is C1 (7) This is illustrated in figure 3.

t P(x) .................................................... kx X

t I B1 t

Figure 2: FRM rule illustrated with Mamdani model

X

Figure 3: NFRM rule illustration.

It is apparent that if rl I and r12 are both set to 1 the NFEW rule (7) becomes the same as the FRM rule (6). However, both inference mechanisms are equivalent only if we chose the fuzzy operators as descried above.

Ill NNFRM: NEURO-NEW FUZZY REASONING MODEL A. NNFRM Architecture

The architecture of NNFRM is shown in figure 4 it consists of four layers. Layer1 : This is the input layer The nodes of this layer just transmit input values to the next layer directly That is

(8) I I I I v, = x, and y, = v,

where v,' is the net input to the ith-node in layer L, and yf- is the output i& node in layer L

SEVENTEENTH NA TIONAL RADIO SCIENCE CONFERENCE Feb. 22-24, 2000, Miizirfiya Uitiversity , Egypt.

Layer 2: The nodes of this layer act as membership hc t ions to express the linguistic terms of input variables. For a bell-shaped function, they are

where cij and oij are respectively the center or mean and width or variance of the bell-shaped function of the jth input linguistic term of input variable q.

Figure 4: The architecture of NNFRM.

Layer 3: The nodes of this layer are used to perform both preconditioning matching of fizzy logic rules and to integrate these rules to produce the output linguistic terms. The weights play the same role as in (5) ; that is

where m is the number of output term nodes, nk is the number of input term nodes corresponding to &, & is the output of node j in layer 2 that corresponds to input variable Q, and wjkI is the weight connecting that node to the i& node in layer 3.

Layer 4: The node of this layer acts as defitzziffier. If and a, are the center and width of bell-shaped membership function respectively then the following function can be used to simulate the center of area defkzification method [ 111

v4 vp = Cp,a,~,~ and y: =I

1 Ca,y13

B. Learning algorithm The learning algorithm uses the well-known backpropagation concept [9] to optimize the "FRM

parameters. The objective function to be minimized is defined as

where yd(t) is the desired output for pattern t and p is the number of pattems. According to the backpropagation method, there are two phases of learning the forward phase and the

backward phase. In the forward phase, the inputs are presented to layer 1 nodes and the output is calculated according to equations (8-11). In the backward phase the error of the output layer is calculated and is backpropagated to the previous layers. The updated parameters is then calculated according to

SEVENTEENTH NATIONAL RADIO SCIENCE CONFERENCE Feb. 22-24,2000, MinuJiya Universiiy , Egypt.

aE am

m(t + 1) = m(t) + q(--) (13)

where m is the parameter to be adjusted. For a hidden layer node j in layer 1 where 1 =1,2, ...,( number of layers -1)

(14)

define the value between practice as 6‘ .The backward phase could be summarized as follows:

1) Training the centers of output membership hnction

2) Training the widths of output membership hnction

3) The value of e3

4) Training the weights wJki

w j k , r ( t + l ) = w , k , t ( t + 1 ) + q e 3 Y : k

5 ) The value of 8’ n ( 3 )

e2 = - p 3 w , n , , I =1

6) Training the centers of input membership functions - 1

2 1 : C l k ( t -k 1) = C l k ( t + 1) - 178

’ 1.. *tk

7) Training the widths of input membership hnctions

The learning algorithm is implemented by a C++ program.

IV PERFORMANCE EVALUATION OF THE NNFRM In this section, the proposed “ F R M is used to predict future values of a chaotic time series The time

series used in the simulation is generated by the chaotic Mackey-Glass differential delay equation [2] defined below

0.2x(t - z)

1 + x’O(t - z) x ( t ) = ___- - O.lx(t)

The prediction of future values of this time series is a benchmark problem, which has been considered by a .number of researchers. The results obtained by those researchers , that can be found in [ 131, is compared to the proposed NNFRM results.

SEVENTEENTH A A TIOh%L RADIO SCIEh'CE COh'FERENCE . Feb. 22-24, 2000, Mirirrjiyu Uriiversitji , Egypt.

XI

X 2

X3

The time series value at integer points can be found in MATLAB fuzzy logic Toolbox [ 141 in fde called "mgdata.dat". The standard prediction data values and a method to solve it using Adaptive Neuro fuzzy inference system (ANFIS) [2] is found also in [14]. This method uses 1000 input-output data pairs of the following format:

[x(t-l8), x(t-12), x(t-6), x(t); x(t+6)] (23) wheret-118to 1117

The first 500 pairs (training data set) is used for training while remaining 500 pairs (checking data set) is used for validating the identified model. The number of membership functions assigned to each of the four input was set 2, and also the number of membership functions assigned to output was 2. Figure 5 shows the Mackey- Glass time series and predicted time series using " F R M . Figure 6 shows the input membership function and output membership function. Table 1 shows the R matrix. Table 2 shows a comparison between the proposed model and other models. The non-dimensional error index CNDEI) [2,13] is used to be compatible with the results listed in [ 131. The NDEI is defined as the root mean square error divided by standard deviation of the target series. However, since the " F R M depends on backpropagation method it may be needed to restart the program several times before reaching these results.

It is apparent From the comparison that the proposed " F R M outperfoms other modeling methods in prediction of this chaotic time series, except for the ANFlS model. The ANFlS model is based on the Sugeno- Takagi-kang fuzzy model, which is known to be difficult to build and time consuming [ 15,161. The NNFRM used here has 36 adjustable parameters, far fewer than those used ir. ANFIS (1G4). the cascade-correlation NN (693) and backpropagation MLP(about 540).

output 0.346914 1.000000

0.999814 0.564905

0.466910 1.000000

0.91521 1 0.635658

0.829935 0.997727

0.873895 0.818424

Auto regressive CAR) model 500 0 19

Cascaded-correlation NN 500 0.06 693 ~ Backpropagation h4LP 500 0.02 540 I I 6* order polynomial 500 004

Method Training cases

NNFRM 500

ANFIS 500

NDEI Number of adjustable

0.012 36

0.007 1 04

parameters

SEVENTEENTH NATIONAL RADIO SCIENCE CONFERENCE ml Feb. 22-24,2000 Mirirrfiya University Egypt.

V CONCLUSION In this paper, an interpretation of NFRM in a way that makes FRM a special case under certain

conditions is developed and a four layers neural network is constructed to implement t h e m w i t h t h e developed interpretation. The proposed Neuro-NFRM ("FRM) uses the well-known concept of back propagation to optimize the inputs and output membership function as well as to find the fuzzy relation matrix that relates the inputs to output The proposed NNFRM is used to predict fiture values of a chaotic time series, which considered a benchmark problem. It is shown that the proposed NNFRM outperforms other modeling methods in prediction of this chaotic time series. The NNFRM used here has fewer adjustable parameters, than those used in other modeling techniques.

REFERENCES [I] Daihee Park, Abraham Kandel and Gideon Langholz "Genetic-Based New fuzzy Reasoning Models with Application to control" IEEE Trans on Systems, Man, and Cybernetics, 24(01) 39-47, Jan 1994 [2] J -S Roger Jang "ANFIS Adaptive-network-based fuzzy inference systems" IEEE Trans on Systems, Man, and cybernetics, 23(03) 665-685, May 1993 [3] Chin-Teng Lin and George Lee "Neural-Network-Based Fuzzy Logic Control and Decision System" EEE Trans On Computer, 40(12),1320-1336, Dec 1991 [4] Yan-Qing Zhang and Abraham kandel "Compensatory Neurofuzzy Systems with Fast Learning Algorithms" IEEE Trans on Neural Networks, 9(01), 83-105, Jan 1998 [5] Kim C Ng and Mohan M Trivedi "A Neuro-Fuzzy Controller for Mobile Robot Navigation and Multirobot Convoying" IEEE Trans. on Systems, Man, and Cybernetics-PART B, 28(6),Dec 1998 [6] Chu Kwong Chak, Gang Feng afid Jian Ma "An Adaptive Fuzzy Neural Network for MIMO System Model Approximation in High-Dimensional spaces" IEEE Trans on Systems, Man, and Cybernetics-PART B, 28(3),June 1998 [7] Chin-Teng Lin and Ya-Ching Lu "A Neural Fuzzy System with Fuzzy Suprvised Leamning" IEEE Trans on Systems, Man, and Cybernetics-PART B, 26(5),0ct 1996 [8] Simon Hykin "Neural network A comprehensive Foundation I' Second Edition Macmillan 1999 [9] Witold Pedrycz, "Fuzzy Control and Fuzzy Systems" Second Edition, Research Studies Press LTD, 1993 [ 101 Chuen Chien Lee, "Fuzzy Logic Control Systems Fuzzy Logic Controller -Part I and Part 11" IEEE Trans on Systems, Man, and cybernetics, 20(2), MarcWApril 1998 [I I ] Toshiro Terano, Kiyoji Asai and Michio Sugeno "Applied Fuzzy Systems" AP PROFESSIONAL, 1989 [12] Ebrahim H Mamdani, "Application of fuzzy logic to approximate reasoning using linguistic synthesis",

IEEE Trans On Computer, 26( 12); 1182-1 191, Dec 1977 [ 131 J -S Roger Jarig, Chuen-Tsai Sun and Eiji Mizutani Neuro-Fuzzy and Soft Computing - a computational approach to learning and machine intelligence" Prentice Hall, 1997

[14] Fuzzy Logic Toolbox User's guide, COPYRIGHT 1995 - 1998 by The Mathworks, h a n d Fuzy Logic Toolbox for Use with MATLAB@ [15] L Wang and R Langari, "Complex system modeling via fuzzy logic", IEEE Trans Systems, Man, and Cybernetics, 26(2), Feb 1996 [16] Wael A Farag, Victor H Quintana, and German0 Lambert-Torres "A Genetic-Based Neuro-Fuzzy Approach for Modeling and Control of Dynamical Systems", IEEE Trans on Neural Networks, 9(5), Sept 1998

m[ SEVENTEENTH NATIONAL RADIO SCIENCE CONFERENCE Feb. 22-24,2000, Minirfiya University , Egypt.

1 3 1

Cl 1 0 0 200 300 400 900 600 700 S O 0 900 1 0 0 0

Figure 5: The Mackey-Glass time series (solid line) and predicted time series using " F R M (dotted line)

0.7

0.e

0 5

U 4

0 3 0

0 . 7

Figure 6: Inputs membership functions (above) and output membership functions.