Fuzzy vs Neural

7/28/2019 Fuzzy vs Neural

1/10

356 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 11, NO. 2, MARCH 2000

The Equivalence Between Fuzzy Logic Systems andFeedforward Neural Networks

Hong-Xing Li and C. L. Philip Chen

AbstractThis paper demonstrates that fuzzy logic systemsand feedforward neural networks are equivalent in essence.First, we introduce the concept of interpolation representationsof fuzzy logic systems and several important conclusions. Wethen define mathematical model for rectangular wave neuralnetworks and nonlinear neural networks. With this definition,we prove that nonlinear neural networks can be represented byrectangular wave neural networks. Based on this result, we provethe equivalence between fuzzy logic systems and feedforwardneural networks. This result provides us a very useful guidelinewhen we perform theoretical research and applications on fuzzylogic systems, neural networks, or neuro-fuzzy systems.

Index TermsEquivalence on fuzzy logic systems, fuzzy logic

systems, feedforward neural networks, interpolation representa-tion, rectangular wave neural networks.

I. INTRODUCTION

MANY researches focused on combining neural networksand fuzzy logic systems, such as neuro-fuzzy systems,or fuzzy neural networks [1][3], [10][16]. They have shown

valuable results undoubtedly. However, this paper takes a dif-

ferent approach to demonstrate the equivalent relationship be-

tween fuzzy logic systems and neural networks. Based on this

result, we wish to provide more significant theoretical result

on combining both systems. To provide this, we first introduce

representations of fuzzy logic systems and then we elaboratethe idea of interpolation representation of fuzzy logic systems

and provide new results under some weaker conditions. We first

prove rectangular wave activation function neural networks can

represent nonlinear neural networks. Based on this result, we

further prove that fuzzy logic systems and neural networks are

equivalent essentially under some restriction.

First of all, we introduce interpolation representation of fuzzy

logic systems. We prove that the antecedents of inference of a

fuzzy logic system are the base functions of interpolation and

the consequents of inference only relate to their peak values but

not to the shape of the membership functions.

Second, we define mathematical model of rectangular wave

neural networks, where their activation functions are rectan-

Manuscript received February 8, 1999; revised July 29, 1999 and November22, 1999. This work was supported by the National Natural Science Foundationof China, U.S. AFOSR Grant F49620-94-0277, and NSF-EIA-9601670.

H.-X. Li is with the Department of Mathematics, Beijing Normal Uni-versity, Beijing 100875, China (e-mail: [email protected]). He is currentlyon leave with Wright State University, Dayton, OH 45435 USA (e-mail:[email protected]).

C. L. P. Chen is with the Department of Computer Science and En-gineering, Wright State University, Dayton, OH 45435 USA (e-mail:[email protected]).

Publisher Item Identifier S 1045-9227(00)01743-4.

gular waveforms. Then we prove that a nonlinear neural net-

work can be represented by a rectangular wave neural network.

By means of this result, we prove the equivalence between

fuzzy logic systems and feedforward neural networks. This con-

clusion provides an important theoretical tool or basis for fuzzy

logic systems or neural networks, especially when combining

both of them.

Similar work can be found from [12] and [16]. Jang [12]

treated the system using radial basis function and Sugeno

model. The equivalence is found by finding the mapping

between Sugeno model and radial basis function without some

rigorous proof. Buckley et al. [16] focused on the equivalencefor fuzzy expert system. In this paper, we study the interpola-

tion and equivalence under weak condition, i.e., Kronecker's

property.

II. INTERPOLATION REPRESENTATIONS OF FUZZY LOGIC

SYSTEMS

This section reviews and elaborates the interpolation mech-

anism or representations of fuzzy logic systems. The interpo-

lation mechanism or representation of fuzzy logic systems is

discussed in detail in [1]. Here, we review, briefly, some main

issues under weaker restrictions as follows.

A. Some Necessary Concepts and Notations

Let us quickly recall fuzzy logic systems, taking a two in-

puts and one output system as an example. Let and be

the universes of input variables and , respectively, and be

the universe of output variable . Denote

and , where

and , and and

is the family of all fuzzy sets on and , respec-

tively. We regard and as linguistic variables, so that a

group of fuzzy inference rules is formed as follows:

If is and is then is (1)

where , and and

are called base variables.

According to the Mamdanian algorithm, the inference rela-

tion of the th inference rule is a fuzzy relation from

to , where

10459227/00$10.00 2000 IEEE
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


2/10


3/10
http://-/?-


4/10

LI AND CHEN: THE EQUIVALENCE BETWEEN FUZZY LOGIC SYSTEMS AND FEEDFORWARD NEURAL NETWORKS 359

approximately a binary piecewise interpolation function taking

for its base functions

(16)

Furthermore, when the system is normal and

is an equidistant partition, we have

(17)

Proof: In (2), after is replaced by

(18)

For given inputs and , we get

(19)

Similar to the proof of Lemma 2, we have

(20)

where

and . Let

and we get (16).

When the system is normal and is an

equidistant partition, it is easy to see that . So

(17) is also true.

E. Interpolation Representations of Weighted -Centroid

Algorithm

The weighted -centroid algorithm is proposed in [6],

which means the inference rules are joined by weighed or.Taking (18) for an example, we should have

(21)

where , and usually .

Lemma 5: Under the conditions in Lemma 2, there exists a

group of base functions such that

the weighted -centroid algorithm with two inputs and one

output is approximately a binary piecewise interpolation func-

tion taking for its base functions

(22)

where and

(23)

The proof is omitted for it is similar to the proof of lemma 4.

F. Interpolation Representation of the Simple

Inference Algorithm

The simple inference algorithm is proposed in [7] and [8]

which is thought by some people to be a kind of quick and

simple algorithm with respect to fuzzy inference. In fact, it

has been most widely used in neuro-fuzzy systems involving

learning mechanisms.

In the algorithm, the fuzzy sets representing inference con-sequents are replaced by numbers. For example, inference rules

with two inputs and one output are as follows:

If is and is then is (24)

Then for a given input , the response value is calcu-

lated in the following steps:

Step 1)

or

Step 2)

(25)

Lemma 6: The simple inference algorithm with two inputs

and one output is exactly (not approximately)a binary piecewise

interpolation. When

(26)

where

(27)

When and the system is normal

(28)

The proof is omitted for it is not difficult.

G. Interpolation Representation of Function

Inference Algorithm

The function inference algorithm is proposed in [9] which is

the generalization of the simple inference algorithm. We con-
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


5/10


6/10


which represents a fuzzy logic system. Naturally, the interpola-

tion representation of the system can be given by the following

expression:

This provides us a general guideline to construct models or al-gorithms for fuzzy logic systems.

III. RECTANGULAR WAVE NEURAL NETWORKS AND

NONLINEAR NEURAL NETWORKS

As two of basic tools for proving the equivalence of fuzzy

logic systems and feedforward neural networks, we define and

discuss rectangular wave neural networks and nonlinear neural

networks in this section. First, let us recall the basic structure of

a neuron. At present, the models of neurons most in use are the

units with multiinput and single output, for example, MP model

(i.e., McCullochPitts model, see Fig. 1).

It is well known that the relation between the inputs and out-

puts of the neuron is expressed as follows:

(43)

If we take , then ,

which means that the threshold value has been absorbed into

. We reuse the notation instead of and (43) is changed as

the following:

(44)

where in is regarded as a -ary function

Let . Clearly, is just the mathematical representa-

tion of the artificial neuron. Pay attention to the simple fact that

is the composition of and , where plays a role in the

synthesizing of -dimensional Euclidean space and plays a

role in activating signals. In fact, is just a kind of multifacto-

rial function (see [10]); so usually we call a space synthesizer

or a synthetic function of space. And is called a signal acti-vator or an activation function.

Now we should understand that a neuron can be interpreted

as a function of several variables (including unary functions)

and from mathematical viewpoint a neural network is regarded

as a certain combinatorial form of some functions of several

variables.

Definition 1: A neuron is called a nonlinear neuron if its ac-

tivation function is a nonlinear function. A neural network is

called a nonlinear neural network, if there at least exists one

nonlinear neuron.

The following definition shows an important concept in this

paper.

Fig. 1. MP model of artificial neurons.

Definition 2: A neuron is called a rectangular wave neuron if

itsactivation function is just a characteristic function of a certain

real number interval (including infinite intervals; see Fig. 2). A

neural network is called a rectangular wave neural network if

every neuronof theneural network is a rectangular wave neuron.

Usually for the sake of convenience, we do not distinguish

between a neuron and its activation function. So in Fig. 1, weuse the notation of the activation function of a neuron to stand

for the neuron.

It is worthwhile to mention that rectangular wave (pulsesignal) neural networks can be easily implemented by hard-

ware.

Lemma 8: A nonlinear neural network can be, approxi-

mately, represented by a rectangular wave neural network.

Proof: For a given nonlinear neural network, without loss

of generality, we take the neural network as in Fig. 3, where

and are the activation functions of these

neuron with nonlinear activation function. Also without loss of

generality, we can suppose that these activation functions are

all nonlinear and continuous functions.

According to the relation between the input and output of the

neural network, we have

(45)

Let have the universe (domain) ( is the

set of real numbers). For any , we create a partition of ,

denoted by , where , and

form a group of zero-order base functions, as

shown in Fig. 4

otherwise(46)

such that where .

This means that . So

(47)

For any and for any , in a similar

way, we can obtain zero-order interpolation functions

(48)
http://-/?-http://-/?-


7/10


Fig. 2. The activation function of a rectangular wave neuron.

such that , i.e., , where the

universe of is and the base function

is as in Fig. 5. Let

(49)

which is just the relation between the input and output of

the simple neural network shown in Fig. 6. We have shown

that . We now prove

that can approximate

within an arbitrary accuracy. In fact, we consider the following

expression:

, since is a continuous function, such

that, , when .

Let . Because of being

arbitrary, we can make . Thus

Fig. 3. A nonlinear neural network.

Fig. 4. Rectangle wave activation functions as the basic functions.

Fig. 5. Zero-order activation function g ( v ) .

So . Also because of being

arbitrary, we can take . Hence

. This means


8/10


Fig. 6. A rectangular wave neural network, where I ( x ) = 1 1 1 = I ( x ) = I ( x ) = I ( x ) = i d ( x ) .

, i.e., can approximate

within an arbitrary accuracy.

IV. THE EQUIVALENCE BETWEEN THE TWO SYSTEMS

Theorem: A fuzzy logic system is approximately equivalent

to a feedforward neural network.

Proof: Necessity: Arbitrarily given a fuzzy logic system,

for the convenience of the proof, we consider the case with

two-input and one-output. Without loss of generality, based on

the conclusions in Section II, the system can be regarded as an

interpolation function

Now we create a feedforward neural network shown in Fig. 7,where and are the neurons (regarded as functions of sev-

eral variables), holding

, i.e., considered as hyperbolic functions, and is

given as ; that is

Thus, the output of the network is as follows:

So . Clearly, the neural network is a non-

linear neural network. According to Lemma 8, it can be changedinto a rectangular wave neural network. This finishes the proof

of the necessity.

Sufficiency: Arbitrarily given a feedforward neural net-

works, without loss of generality, we consider the case with two

inputs and one output and three layers as an example shown in

Fig. 8.

We consider the output of the network

(50)

From (50) we should get a fuzzy logic system such that

is approximately the output of the fuzzy logic system. Using


9/10


Fig. 7. The three-layer feedforward neural network with two inputs and one output.

Fig. 8. An arbitrarily given neural network.

(50), because can be regarded an unary function (such that

there exist at most finite discontinuous points) and for any

, there exists a group of base functions

such that

where

otherwise

Thus

(51)


10/10


where is the characteristic function of the

set , (clearly, , for any ),

and

(52)If we form a group of fuzzy inference rules

If is then is (53)

for a fuzzy logic system, then (51) is just the interpolation rep-

resentation of the system according to the statement of Section

II-H. This finishes the proof of the sufficiency.

V. CONCLUSION REMARKS

We briefly summarize our main conclusions in this paper as

follows.

1) Fuzzy logic systems are equivalent to feedforward neural

networks, which means that, an arbitrarily given fuzzy

logic system can be approximately represented by a feed-forward neural network. Conversely, an arbitrarily given

feedforward neural network can be approximately repre-

sented by a fuzzy logic system. It is worth noting that here

approximately used by us means the approximation can

reach an arbitrarily given accuracy.

2) We define mathematical model of rectangular wave

neural networks and nonlinear neural networks. Then we

prove that nonlinear neural networks can be represented

by rectangular wave neural networks, which can be

implemented easily by hardware.

3) From Lemma 1 to Lemma 7, we discover a very impor-

tant and interesting conclusion: under the condition of

Kronecker's property (see Section II-A), the membership

functions defined on output variable universes do not take

effect in fuzzy logic systems but only their peak values do.

So, in many real applications, as long as the Kronecker's

property is satisfied, we do not need to care for the shapes

of these membership functions but only need to consider

whether their peak values are suitable.

REFERENCES

[1] H.-X. Li, The mathematical essence of fuzzy controls and fine fuzzycontrollers, in Advances in Machine Intelligence and Soft-Computing,P. P. Wang, Ed. Durham, NC: Bookwright, 1997, vol. IV, pp. 5574.

[2] W. A. Farag, V. H. Quintana, and G. Lambert-Torres, A genetic-based

neuro-fuzzy approach for modeling and control of dynamical systems,IEEE Trans. Neural Networks, vol. 9, pp. 756767, 1998.

[3] C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems: A Neural-FuzzySynergism to IntelligentSystems. Englewood Cliffs, NJ: Prentice-Hall,1996.

[4] M. Mizumoto, The improvement of fuzzy control algorithm, part 4:( + ; ) -centroid algorithm, in Proc. Fuzzy Syst. Theory, vol. 9, 1990, p.9. (in Japanese).

[5] P. Z. Wang and H.-X. Li, Fuzzy Information Processing and Fuzzy Com-puters. New York: Science, 1997.

[6] T. Terano, K. Asai, and M. Sugeno, Fuzzy Systems Theory and Its Ap-plications. New York: Academic, 1992.

[7] M. Sugeno, Fuzzy Control (in Japanese). Tokyo, Japan: Japan Ind.Press, 1988.

[8] T. Takagi and M. Sugeno, Fuzzy identification of systems and its ap-plications to modeling and control, IEEE Trans. Syst., Man, Cybern.,vol. SMC-15, pp. 1116, 1985.

[9] H.-X. Li, Multifactorial functions in fuzzy sets theory, Fuzzy SetsSyst., vol. 35, pp. 6984, 1990.

[10] B. Kosko,Neural Networks and Fuzzy Systems. EnglewoodCliffs, NJ:Prentice-Hall, 1992.

[11] M. Brown and C. Harris, Neurofuzzy Adaptive Modeling and Con-trol. Englewood Cliffs, NJ: Prentice-Hall, 1994.[12] J.-S. R. Jang and C.-T. Sun, Functional equivalence between radial

basis function networks and fuzzy inference systems, IEEE Trans.Neural Networks, vol. 4, pp. 156159, 1993.

[13] D. Nauck, F. Klawonn, and R. Kruse, Neuro-Fuzzy Systems. NewYork: Wiley, 1997.

[14] L. H. Tsoukalas and R. E. Uhrig, Fuzzy and Neural Approaches in En-gineering. New York: Wiley, 1997.

[15] J.-S. R. Jang, C.-T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Com-puting. Englewood Cliffs, NJ: Prentice-Hall, 1997.

[16] J. J. Buckley, Y. Hayashi, and E. Czogala, On the equivalence of neuralnets and fuzzy expert systems, Fuzzy Sets Syst., vol. 53, pp. 129134,1993.

Hong-Xing Li received the B.S. degree from NankaiUniversity, China, in 1978 and the Ph.D. degree inmathematics from Beijing Normal University, China,in 1993.

He has been a Professor atBeijing Normal Univer-sitysince 1994.He wasa visiting Professorat Depart-ment of Computer Science and Engineering, WrightState University, Dayton, OH, during 1998 to 1999.His research interests are fuzzy control theory, adap-tive fuzzy control, neural networks, and mathemat-ical theory of knowledge representation.

Dr. Li is on the editorial board ofJournal of Fuzzy Systems and Mathematics,and Journal of Systems Engineering.

C. L. Philip Chen (S88M88SM94) received

the B.S.E.E. degree from NTIT, Taiwan, in 1979, theM.S. degree from the University of Michigan, AnnArbor, in 1985, and the Ph.D. degree from PurdueUniversity, West Lafayette, IN, in Dec. 1988.

In 1988 to 1989, he was a Visiting AssistantProfessor at the School of Engineering and Tech-nology, Purdue University, Indianapolis, IN. SinceSeptember 1989, he has been at the ComputerScience and Engineering Department, Wright StateUniversity, Dayton, OH, where he is currently an

Associate Professor. He is also a Visiting Research Scientist, at the MaterialsDirectorate, Wright Laboratory, Wright-Patterson Air Force Base. He has beena Senior Research Fellow sponsored by National Research Council, NationalAcademy of Sciences. He also worked as a Research Faculty Fellow at NASAGlenn Research Center during summer of 1998 and 1999. He was on sabbaticalleave to Purdue University and Case Western Reserve University for Fall1996 to Fall 1997. His current research interests and projects include neural

networks, fuzzy-neural systems, neuro-fuzzy systems, intelligent systems,robotics, and CAD/CAM.Dr. Chen was a Conference Cochairman of the International Conference

on Artificial Neural Networks in Engineering in 1995 and 1996, a TutorialChairman for the International Conference on Neural Networks in 1994, theConference Cochairman of the Adaptive Distributed Parallel Computing in1996, a Program Committee of OAI Neural Networks Symposium in 1995, theIEEE International Conference on Robotics and Automation in 1996, and theIntational Conference on Intelligent Robotics and Systems (IROS) in 1998 and1999. He is a member of Eta Kappa Nu. He is the founding faculty advisor ofthe IEEE Computer Society Student Chapter at Wright State University and arecipient of the 1997 College Research Excellent Faculty Award.

Documents

Fuzzy vs Neural