-
7/24/2019 (758981491) Soft Computing.
1/287
LECTURE NOTICE. INTRODUCTION
TO SOFT COMPUTING
Dmitry Bystrov
-
7/24/2019 (758981491) Soft Computing.
2/287
2
Lecture Notice "Introduction to Soft Computing" are based on
Heikki Koivo "Soft Computing inDynamical Systems" and obert !uller
"Introduction to Neuro!u##y Systems" books$
!u##y logic systems c%apter describes t%e basic definitions of
fu##y set t%eory& i$e$& t%e basicnotions& t%e
properties of fu##y sets and operations on fu##y sets$ Some popular
constructions of
fu##y systems are presented in sections 2$2$' 2$2$($ )%e
appro*imation capability is described insection 2$(& and
section 2$+ summari#es t%e different interpretations of fu##y
sets$
Neural net,orks c%apter gives a s%ort introduction to neural
net,orks$ )%e basic concepts arepresented in ($'$ -fter t%at&
multilayer perceptron& radial basis function net,ork&
neurofu##ynet,orks and& finally& -N!IS .-daptive Neurofu##y
Inference System/ are revie,ed$ )%e c%apterends ,it% a discussion
about appro*imation capability$
0nified form of soft computing met%ods c%apter describes soft
computing met%ods in neuralnet,orks& i$e$& interpolation
net,orks& models for nonlinear mapping$ Some actual relations
as
posterior probability in !LS and neural net,ork is presented in
section +$+$ 1asic function selection
is described in section +$$
)raining of soft computing met%ods c%apter gives more detail
describes of soft computing trainingmet%ods in neural net,orks$ )%e
basic concepts are presented in section $'$ -fter t%at&
net,orkmapping& regulari#ation& t%e bias3variance tradeoff
and training met%ods in nuts%ell are revie,ed$
Conclusion c%apter summari#ed basic relations in soft
computing$
-
7/24/2019 (758981491) Soft Computing.
3/287
Contents
1. Introdution . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
'$' Soft Computing . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . '$'$' !u##y Logic . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 4'$'$2 Neural Net,orks . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . '5'$'$( 6robabilistic
easoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . ''
!. Fu""y Lo#i Systems . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . '+
2$' 1asic Concepts . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . '+2$'$' Set)%eoretical
7perations and 1asic Definitions . . . . . . . . . . . . '+2$'$2
!u##y elations . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 2(2$'$( )%e 8*tension 6rinciple . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 292$'$+
-ppro*imate easoning . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 292$'$ !u##y ules . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 2:2$'$9 !u##y
Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . ('2$'$4 !u##ifier and Defu##ifier . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . (2$'$; )%e
-
7/24/2019 (758981491) Soft Computing.
4/287
+$' Introduction . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .+$2 Interpolation Net,orks .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.+$( >odels for Nonlinear >apping . . . . . . . . . . . . . .
. . . . . . . . . . . . . .+$+ 6osterior 6robability in !LS and
Neural Net,orks . . . . . . . . . . . . . .+$ 1asis !unction
Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .
-. Tr%inin# o* So*t Com+utin# Met,ods . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
$' )%e 1asics . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .$2 Net,ork >apping . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.$( egulari#ation . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .$+ )%e 1ias3@ariance )radeoff . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .$ )raining
>et%ods in Nuts%ell . . . . . . . . . . . . . . . . . . . . . .
. . . . . . .
. Con&usion . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Re*erenes . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
4(
4(
44;5;;:5
:(
:+:4:;
'52'5+
'5
-
7/24/2019 (758981491) Soft Computing.
5/287
'54
-
7/24/2019 (758981491) Soft Computing.
6/287
C,%+ter 1
Introdution
No,adays& fuzzy logic& neurocomputing and probabilistic
reasoning %ave rooted in manyapplication areas .e*pert systems&
pattern recognition& system control& etc$/$ )%e first t,o
met%ods%ave received a lot of critiAue during t%eir e*istence .for
e*ample& see B>insky 6apert& ':9:C%eeseman& ':;9E/$
)%e critiAue %as been particularly strong ,it% fu##y logic& but
it %as ,eakenedas fu##y logic %as been successfully applied to
practical problems and its universal appro*imation
property %as been proven BKosko& '::2 ?ang& '::+ Castro
Delgado& '::9E$
-lt%oug% t%ese met%odologies seem to be different& t%ey %ave
many common features like t%e useof basis functions .fu##y logic
%as members%ip functions& neural net,orks %ave activation
functionsand probability regression use probability density
functions/ and t%e aim to estimate functions fromsample data or
%euristics$
)%e purpose of t%is Lecture Notice is to illustrate t%e use of
soft computing met%ods$ )%e emp%asisis put on t%e fu##y logic and
only t%e basics of neural net,orks and probabilistic met%ods ,ill
bedescribed$ )%e probabilistic met%ods are revie,ed only in t%e end
of t%is introductory c%apter$
)%is first c%apter describes t%e term soft computing and gives
an introduction to its basiccomponents$ )%e follo,ing c%apters go
deeper into t%ese components$ )%e 2 nd c%apter provides
adescription of fu##y logic systems and (rd c%apter describes
neural net,orks$ 0nified approac% tot%ese met%ods is presented in
c%apter + and common basics of learning met%ods in c%apter $C%apter
9 is devoted to conclusion$
1.1 So*t Com+utin#
)%e termsoft computing ,as proposed by t%e inventor of fu##y
logic& Lotfi -$ Fade%$ He describesit as follo,s BFade%&
'::+EG
Soft computing is a collection of methodologies that aim to
exploit the tolerance for imprecisionand uncertainty to achieve
tractability, robustness, and low solution cost. Its principal
constituents
are fuzzy logic, neurocomputing, and probabilistic reasoning.
Soft computing is likely to play an
increasingly important role in many application areas, including
software engineering. The role
model for soft computing is the human mind$
-
7/24/2019 (758981491) Soft Computing.
7/287
Soft computing is not precisely defined$ It consists of distinct
concepts and tec%niAues ,%ic% aim toovercome t%e difficulties
encountered in real ,orld problems$ )%ese problems result from t%e
factt%at our ,orld seems to be imprecise& uncertain and
difficult to categori#e$ !or e*ample& t%euncertainty in a
measured Auantity is due to in%erent variations in t%e measurement
process itself$
)%e uncertainty in a result is due to t%e combined and
accumulated effects of t%ese measurementuncertainties ,%ic% ,ere
used in t%e calculation of t%at result BKirkpatrick& '::2E$
In many cases t%e increase in precision and certainty can be
ac%ieved by a lot of ,ork and cost$Fade% gives as an e*ample t%e
travel salesman problem& in ,%ic% t%e computation time is a
functionof accuracy and it increases e*ponentially BFade%&
'::+E$
-not%er possible definition of soft computing is to consider it
as an antit%esis to t%e concept ofcomputer ,e no, %ave& ,%ic%
can be described ,it% all t%e adJectives suc% as %ard&
crisp& rigid&infle*ible and stupid$ -long t%is track&
one may see soft computing as an attempt to mimic naturalcreaturesG
plants& animals& %uman beings& ,%ic% are soft&
fle*ible& adaptive and clever$ In t%is sense
soft computing is t%e name of a family of problemsolving met%ods
t%at %ave analogy ,it%biological reasoning and problem solving
.sometimes referred to as cognitive computing/$ )%e basicmet%ods
included in cognitive computing are fu##y logic .!L/& neural
net,orks .NN/ and geneticalgorithms .
-
7/24/2019 (758981491) Soft Computing.
8/287
)%e main dissimilarity bet,een fu##y logic system .!LS/ and
neural net,ork is t%at !LS uses%euristic kno,ledge to form rules
and tunes t%ese rules using sample data& ,%ereas NN formsrules
based entirely on data$
Soft computing met%ods %ave been applied to many real,orld
problems$ -pplications can be foundin signal processing&
pattern recognition& Auality assurance and industrial
inspection& businessforecasting& speec% processing&
credit rating& adaptive process control& robotics
control& naturallanguage understanding& etc$ 6ossible ne,
application areas are programming languages& userfriendly
application interfaces& automatici#ed programming& computer
net,orks& databasemanagement& fault diagnostics and
information security BFade%& '::+E$
In many cases& good results %ave been ac%ieved by combining
different soft computing met%ods$)%e number of t%is kind of %ybrid
systems is gro,ing$ - very interesting combination is t%eneurofu##y
arc%itecture& in ,%ic% t%e good properties of bot% met%ods are
attempted to bringtoget%er$ >ost neurofu##y systems are fu##y
rulebased systems in ,%ic% tec%niAues of neural
net,orks are used for rule induction and calibration$ !u##y
logic may also be employed to improvet%e performance of
optimi#ation met%ods used ,it% neural net,orks$ !or e*ample& it
may controlt%e vibration of direction for searc%ing vector in Auasi
Ne,ton met%od BKa,arada Suito& '::9E$
Soft computing can also be seen as a foundation for t%e gro,ing
field of computational intelligence.CI/$ )%e difference bet,een
traditional artificial intelligence .-I/ and computational
intelligence ist%at -I is based on %ard computing ,%ereas CI is
based on soft computing$
Soft Computing is not ust a mixture of these ingredients, but a
discipline in which each constituentcontributes a distinct
methodology for addressing problems in its domain, in a
complementary
rather than competitive way. BFade%& '::+E$
1.1.1 Fu""y Lo#i
!uzzy set theory ,as developed by Lotfi -$ Fade% BFade%&
':9E& professor for computer science att%e 0niversity of
California in 1erkeley& to provide a mat%ematical tool for
dealing ,it% t%econcepts used in natural language .linguistic
variables/$ !u##y Logic is basically a multivalued logict%at allo,s
intermediate values to be defined bet,een conventional
evaluations$
Ho,ever& t%e story of fu##y logic started muc% more earlier
$ )o devise a concise t%eory of logic&and later
mat%ematics&"ristotleposited t%e socalled La,s of )%oug%t$ 7ne
of t%ese& t%e La, of
t%e 8*cluded >iddle& states t%at every proposition must
eit%er be True .T/ or!alse .F/$ 8ven ,%en#arminedesproposed t%e
first version of t%is la, .around +55 1efore C%rist/ t%ere ,ere
strong andimmediate obJectionsG for e*ample& $eraclitus
proposed t%at t%ings could be simultaneously Trueand not True$ It
,as #lato ,%o laid t%e foundation for ,%at ,ould become fu##y
logic& indicatingt%at t%ere ,as a t%ird region .beyond T and F/
,%ere t%ese opposites tumbled about$ - systematicalternative to t%e
bivalued logic of -ristotle ,as first proposed by %ukasiewicz
around ':25& ,%en%e described a t%reevalued logic& along
,it% t%e mat%ematics to accompany it$ )%e t%ird value %e
proposed can best be translated as t%e term possible& and %e
assigned it a numeric value bet,een Tand F$ 8ventually& %e
proposed an entire notation and a*iomatic system from ,%ic% %e
%oped toderive modern mat%ematics$
-
7/24/2019 (758981491) Soft Computing.
9/287
Later& %e e*plored fourvalued logics& fivevalued
logics& and t%en declared t%at in principle t%ere,as not%ing to
prevent t%e derivation of an infinitevalued logic$ukasie,ic# felt
t%at t%ree andinfinitevalued logics ,ere t%e most intriguing&
but %e ultimately settled on a fourvalued logic
because it seemed to be t%e most easily adaptable to
-ristotelian logic$ It s%ould be noted t%at&nuth
also proposed a t%reevalued logic similar to Lukasie,ic#=s&
from ,%ic% %e speculated t%atmat%ematics ,ould become even more
elegant t%an in traditional bivalued logic$ )%e notion of
aninfinitevalued logic ,as introduced in Fade%=s seminal ,ork !u##y
Sets ,%ere %e describedt%e mat%ematics of fu##y set t%eory& and
by e*tension fu##y logic$ )%is t%eory proposed makingt%e members%ip
function .or t%e values F and T/ operate over t%e range of real
numbers B5& 'E$
Ne, operations for t%e calculus of logic ,ere proposed& and
s%o,ed to be in principle at least agenerali#ation of classic
logic$ !u##y logic provides an inference morp%ology t%at
enablesappro*imate %uman reasoning capabilities to be applied to
kno,ledgebased systems$ )%e t%eoryof fu##y logic provides a
mat%ematical strengt% to capture t%e uncertainties associated ,it%
%umancognitive processes& suc% as t%inking and reasoning$ )%e
conventional approac%es to kno,ledge
representation lack t%e means for representating t%e meaning of
fu##y concepts$ -s a conseAuence&t%e approac%es based on first
order logic and classicalprobability t%eory do not provide an
appropriateconceptual frame,ork for dealing ,it% t%e representation
of commonsense kno,ledge& since suc%kno,ledge is by its nature
bot% le*ically imprecise and noncategorical$
)%e developement of fu##y logic ,as motivated in large measure
by t%e need for a conceptualframe,ork ,%ic% can address t%e issue
of uncertainty and le*ical imprecision$ Some of t%e
essentialc%aracteristics of fu##y logic relate to t%e follo,ing
.Fade%& '::2/G In fu##y logic& e*act reasoningis vie,ed as
a limiting case of appro*imate reasoning$ In fu##y logic&
everyt%ing is a matter ofdegree$ In fu##y logic& kno,ledge is
interpreted a collection of elastic or& eAuivalently&
fu##yconstraint on a collection of variables$ Inference is vie,ed
as a process of propagation of elastic
constraints$ -ny logical system can be fu##ified$ )%ere are t,o
main c%aracteristics of fu##ysystems t%at give t%em better
performance for specific applications$ !u##y systems are suitable
foruncertain or appro*imate reasoning& especially for t%e
system ,it% a mat%ematical model t%at isdifficult to derive$ !u##y
logic allo,s decision making ,it% estimated values under incomplete
oruncertain information$
)%eory %as been attacked several times during its e*istence$ !or
e*ample& in ':42 Fade%=s colleague$ 8$ Kalman .t%e inventor of
Kalman filter/ commented on t%e importance of fu##y
logicG'...(adeh)s proposal could be severely, fericiously, even
brutally criticized from a technical point of
view. This would be out of place here. *ut a blunt +uestion
remains Is (adeh presenting important
ideas or is he indulging in wishful thinking-...
)%e %eaviest critiAue %as been presented by probability
t%eoreticians and t%at is t%e reason ,%ymany fu##y logic aut%ors
.Kosko& Fade% and Klir/ %ave included t%e comparison
bet,een
probability and fu##y logic in t%eir publications$ !u##y
researc%ers try to separate fu##y logic fromprobability t%eory&
,%ereas some probability t%eoreticians consider fu##y logic a
probability indisguise$
C&%imG 6robability t%eory is t%e only correct ,ay of dealing
,it% uncertainty and t%at anyt%ing canbe done ,it% fu##y logic can
be done eAually ,ell t%roug% t%e use of probabilitybased
met%ods$-nd so fu##y sets are unnecessary for representing and
reasoning about uncertainty and vagueness
probability t%eory is all t%at is reAuired$ Close examination
shows that the fuzzy approaches have
-
7/24/2019 (758981491) Soft Computing.
10/287
exactly the same representation as the corresponding
probabilistic approach and include similar
calculi$ BC%eeseman& ':;9E$
O0etionG Classical probability t%eory is not sufficient to
e*press uncertainty encountered in e*pert
systems$ )%e main limitation is t%at it is based on t,ovalued
logic$ -n event eit%er occurs or doesnot occur t%ere is not%ing
bet,een t%em$ -not%er limitation is t%at in reality events are not
kno,n,it% sufficient precision to be represented as real numbers$
-s an e*ample considers a case in ,%ic%,e %ave been given
informationG
-n urn contains 25 balls of various si#es& several of ,%ic%
are large$
'/ne cannot express this within the framework of classical
theory or, if it can be done, it cannot be
done simply .Fade% to C%eeseman& in same book/$
-
7/24/2019 (758981491) Soft Computing.
11/287
Fi#ure 1.1 8*ample of a control problem
1.1.! Neur%& Net'or(s
)%e study of neural net,orks started by t%e publication of
>cCulloc% and 6itts B':+(E$ )%e single
layer net,orks& ,it% t%res%old activation functions&
,ere introduced by osenblatt B':92E$ )%esetypes of net,orks ,ere
called perceptrons$ In t%e ':95s it ,as e*perimentally s%o,n
t%at
perceptrons could solve many problems& but many problems
,%ic% did not seem to be more difficultcould not be solved$ )%ese
limitations of onelayer perceptron ,ere mat%ematically s%o,n
by>insky and 6apert in t%eir book #erceptron$ )%e result of t%is
publication ,as t%at t%e neuralnet,orks lost t%eir interestingness
for almost t,o decades$ In t%e mid':;5s&
backpropagationalgorit%m ,as reported by umel%art& Hinton&
and ?illiams B':;9E& ,%ic% revived t%e study ofneural net,orks$
)%e significance of t%is ne, algorit%m ,as t%at multilayer net,orks
could betrained by using it$
Neural net,ork makes an attempt to simulate %uman brain$ )%e
simulating is based on t%e presentkno,ledge of brain function&
and t%is kno,ledge is even at its best primitive$ So& it is not
absolutely,rong to claim t%at artificial neural net,orks probably
%ave no close relations%ip to operation of%uman brains$ )%e
operation of brain is believed to be based on simple basic elements
calledneurons ,%ic% are connected to eac% ot%er ,it% transmission
lines called a*ons and receptive linescalled dendrites .see !ig$
'$2/$ )%e learning may be based on t,o mec%anismsG t%e creation of
ne,connections& and t%e modification of connections$ 8ac%
neuron %as an activation level ,%ic%& incontrast to 1oolean
logic& ranges bet,een some minimum and ma*imum value$
-
7/24/2019 (758981491) Soft Computing.
12/287
Fi#ure 1.! Simple illustration of biological and artificial
neuron .perceptron/$
In artificial neural net,orks t%e inputs of t%e neuron are
combined in a linear ,ay ,it% different,eig%ts$ )%e result of t%is
combination is t%en fed into a nonlinear activation unit
.activationfunction/& ,%ic% can in its simplest form be a
t%res%old unit .see !ig$ '$2/$
Neural net,orks are often used to en%ance and optimi#e fu##y
logic based systems& e$g$& by giving
t%em a learning ability$ )%is learning ability is ac%ieved by
presenting a training set of differente*amples to t%e net,ork and
using learning algorit%m ,%ic% c%anges t%e ,eig%ts .or t%e
parametersof activation functions/ in suc% a ,ay t%at t%e net,ork
,ill reproduce a correct output ,it% t%ecorrect input values$ )%e
difficulty is %o, to guarantee generali#ation and to determine ,%en
t%enet,ork is sufficiently trained$
Neural net,orks offer nonlinearity& inputoutput mapping&
adaptivity and fault tolerance$Nonlinearity is a desired property
if t%e generator of input signal is in%erently nonlinear
BHaykin&'::+E$ )%e %ig% connectivity of t%e net,ork ensures
t%at t%e influence of errors in a fe, terms ,ill
be minor& ,%ic% ideally gives a %ig% fault tolerance$ .Note
t%at an ordinary seAuential computationmay be ruined by a single
bit error/$
1.1.$ Pro0%0i&isti Re%sonin#
-s fu##y set t%eory& t%e probability t%eory deals ,it% t%e
uncertainty& but usually t%e type ofuncertainty is different$
Stoc%astic uncertainty deals ,it% t%e uncertainty to,ard t%e
occurrence ofcertain event and t%is uncertainty is Auantified by a
degree of probability$ 6robability statements can
be combined ,it% ot%er statements using stoc%astic met%ods$
>ost kno,n is t%e 1ayesian calculusof conditional
probability$
-
7/24/2019 (758981491) Soft Computing.
13/287
6robabilistic reasoning includes genetic algorit%ms& belief
net,orks& c%aotic systems and parts oflearning t%eory
BFade%& '::+E$ .-lt%oug%& in t%is t%esis& emp%asis ,ill
be on fu##y logic systemsand neural net,orks& probabilistic
reasoning is included because of t%e consistency/$
Geneti %orit,ms2
-
7/24/2019 (758981491) Soft Computing.
14/287
#."/ #.*&"/ #."/.'$'/
,%ere #."*/ represents t%e conditional probability t%at
specifies t%e probability of t%e event "
given t%at t%e event * occurs$ )%e Auantity #.*/ is t%e
probability of * ,it% respect to t%e,%ole event space .prior
probability/$ #.*&"/ & is defined to be t%e probability
t%at bot% t%e
events " and * occur .Joint probability/$
- related approac% to 1ayesian probability is t%e
7empster0Shafer belief theory ,%ic% is alsoreferred to as
t%e7empster0Shafer theory of evidence .t%e latter is preferred
terminology/$ - centralconcept is t%e belief function$
De*inition 1.1. .belief/ Let 4 be a finite set and let m be a
function from subsets of 4 tononnegative reals suc% t%at
)%e belief is defined by
m . / 5 and (m.t/ '
T GT ,4
.'$2/
*el.S/ (m.T/
TGT,
S
.'$(/
Different belief functions can be combined by t%e7empster)s rule
of combination$
Be&ie* net'or(2
- belief net,ork may also be referred to as a *ayesian network
.preferred terminology/& or as acausal net,ork$ 1ayesian
net,ork is a model for representing uncertainty in our kno,ledge
and ituses probability t%eory as a tool to manage t%is uncertainty$
Net,ork %as a structure ,%ic% reflects
causal relations%ips& and a probability part ,%ic% reflects
t%e strengt%s of t%ese relations%ips$ 0serof net,ork must provide
observation based information about t%e value of a random
variable$
Net,ork t%en gives t%e probability of anot%er random
variable$
-
7/24/2019 (758981491) Soft Computing.
15/287
C,%+ter !
Fu""y Lo#i Systems
!.1 B%si Cone+ts
Since set t%eory forms a base for logic& ,e begin ,it% fu##y
set t%eory in order to pave t%e ,ay tofu##y logic and fu##y logic
systems$
!.1.1 Set3T,eoreti%& O+er%tions %nd B%si De*initions
In classical set t%eory t%e members%ip of element x of a set "
." is a crisp subset of universe4 / is defined by
."
.x/
5& if
'& if
x
"&
x#
"
.2$'/
)%e element eit%er belongs to t%e set or not$ In fu##y set
t%eory& t%e element can belong to t%e setpartially ,it% a
degree and t%e set does not %ave crisp boundaries$ )%at leads to
t%e follo,ingdefinition$
De*inition !.1.1 .fuzzy set, membership function/ Let 4 be a
nonempty set& for e*ample
4 8 n & and be called t%e universe of discourse$ - fu##y
setmembers%ip function
"%4 is c%aracteri#ed by t%e
."
G4 /B5&'E
.2$2/
,%ere ." .
x/
is a grade .degree/ of members%ip of x x in set "$
!rom t%e definition ,e can see t%at t%e fu##y set t%eory is a
generali#ed set t%eory t%at includes t%eclassical set t%eory as a
special case$ Since 5&'+#B5&'E & crisp sets are fu##y
sets$>embers%ip
function .2$2/ can also vie,ed as a distribution of trut% of a
variable$ In literature fu##y set " isoften presented as a set of
ordered pairsG
-
7/24/2019 (758981491) Soft Computing.
16/287
" 0x&. .x/1x#4
+
.2$(/
,%ere t%e first part determines t%e element and t%e second part
determines t%e grade of members%ip$
-not%er ,ay to describe fu##y set %as been presented by
Fade%& Dubois and 6rade$ If 4 is infinite&
t%e fu##y set " can be e*pressed by
" 2." .x/ 3x
4
If 4 is finite& " can be e*pressed in t%e form
.2$+/
" ."
.xi/ 3
x'
3$$$ 3."
.xn/ 3
xn
n
(." .xi / 3x
ii '
.2$/
Note2 Symbol 2 in .2$+/ %as not%ing to do ,it% integral .it
denotes an uncountable enumeration/
and 3 denotes a tuple$ )%e plus sign represents t%e union$ -lso
note t%at fu##y sets are members%ipfunctions$ Nevert%eless& ,e
may still use t%e set t%eoretic notations like " )* $ )%is is
t%e
name
of a fu##y set given by ." )
*
.,ill be defined later/$
E/%m+&e !.1.1
Discrete caseG.
"5$'3x' 35$+3x2 35$;3x( 3'$53x+ 35$;3x 35$+3x9 35$'3x4
' 4x 4
c h
& x #Bc 4h& hE
Continuous case G.
".x/
x 4ch
&x#Bc&c 3hE
5 & ot%er,ise
"
-
7/24/2019 (758981491) Soft Computing.
17/287
Fi#ure !.1 - discrete and continuous members%ip functions
De*inition !.1.! .support/ )%e support of a fu##y set " is t%e
crisp set t%at contains all elements
of " ,it% non#ero members%ip gradeG
supp ."/ x #
4
.. x/ 5
5+.2$9/
If t%e support is finite& it is called compact support$ If
t%e support of fu##y set" consists of only onepoint& it is
called afuzzy singleton$ If t%e members%ip grade of t%is fu##y
singleton is one&" is calleda crisp singleton 6Fimmermann&
':;7$
De*inition !.1.$ .core/ )%e core .nucleus& center/ of a
fu##y set " is defined by
core ."/ x#
4
. .x/
'+
.2$4/
De*inition !.1.) .height/ )%e %eig%t of a fu##y set " on 4 is
defined by
%gt ." / supx#4
." .x/
.2$;/
and " is called normal if %gt ."/ '& andsubnormal if %gt ."/
P '$
Note2Nonempty fu##y set can be normali#ed by dividing ." .
x/by sup x ." .x/ $
Normali#ing
"
"
-
7/24/2019 (758981491) Soft Computing.
18/287
of " can be regarded as a mapping from fu##y sets to possibility
distributionsGNormal ."/Q 8"
9."
Boslyn& '::+E$
T,e re&%tion 0et'een *u""y set mem0ers,i+ *untion .4
+ossi0i&ity distri0ution 8%nd
+ro0%0i&ity distri0ution p 2 )%e definition p " .x/ 9." .x/
could %old& if .is additivelynormal$
-dditively normal means %ere t%at t%e stoc%astic
normali#ation
2." .x/dx '
4
,ould %ave to be satisfied$ So it can be concluded t%at any
given fu##y set could define eit%er aprobability distribution or a
possibility distribution& depending on t%e properties of ..
1ot%probability distributions and possibility distributions are
special cases of fu##y sets$ -ll generaldistributions are in fact
fu##y sets Boslyn& '::+E$
De*inition !.1.- .convex fu##y set/ " fu##y set " is conve*
if
&x&y #4 and &:
#B5&'E
.2$:/
."
.:x 3.' 4:/y/ min.."
.x/& ."
.y//
De*inition !.1. .width of a conve* fu##y set/ )%e ,idt% of a
conve* fu##y set " is defined by
,idt% ."/ sup.supp ." / / inf.supp ."/ / .2$'5/
De*inition !.1.5 .;0cut/ )%e ;"cut of a fu##y set " is defined
by
;4cut."/ x #
4
." .2$''/
and it is called astrong ; 4cut & if is replaced by R$ ;
4cut is a generali#ed support$
Fi#ure !.! -n ; 4cut of a triangular fu##y number
.x/ ;+
-
7/24/2019 (758981491) Soft Computing.
19/287
De*inition !.1.6 .fuzzy partition/ " set of fu##y sets is called
fu##y partition if
&x#4
1"
(."i 'i'
.2$'2/
provided "i are nonempty and subsets of4$
De*inition !.1.7 .fuzzy number/ " fu##y set .subset of a real
line R/ is a fu##y number& if t%efu##y set is conve*&
normal& members%ip function is piece,ise continuous and t%e
core consists ofone value only$ )%e family of fu##y numbers is $ In
many situations people are only able to
c%aracteri#e numeric information imprecisely$ !or e*ample&
people use terms suc% as& about 555&
near #ero& or essentially bigger t%an 555$ )%ese are
e*amples of ,%at are called fuzzy numbers$0sing t%e t%eory of fu##y
subsets ,e can represent t%ese fu##y numbers as fu##y subsets of
t%e setof real numbers$
Fi#ure !.$ !u##y number
Fi#ure !.)Nonfu##y number
Note2 !u##y number is al,ays a fu##y set& but a fu##y set is
not al,ays a fu##y number$-n e*ample of fu##y number is Oabout '=
t%at is defined by ..x/ e*p.4
-
7/24/2019 (758981491) Soft Computing.
20/287
De*inition !.1.11 .980representation of fuzzy numbers/ -ny fu##y
number can be described by
9..a 4x/ 3; & x #Ba 4;& aE
." .x/
' &8..x 4b/ 3 / /B5&'E
are s%ape functions t%at are continuous and nonincreasing suc%
t%at 9.5/ 8.5/ '
and t%ey approac% #eroG limx />
4x
9.x/ 5&
4x2
limx />
8.x/ 5
!or e*ample&f .x/ e& f .x/ e and
f .x/ ma*.5$' 4x/
are suc% s%ape functions$ In
t%e follo,ing t%e classical set t%eoretic operations are
e*tended to fu##y sets$
De*inition !.1.1$ .set theoretic operations/
. 95 .empty set/
.49
'
.basic set& universe/
"*."
.x/ .*
.x/
" %*." .x/ ?.* .x/
-
7/24/2019 (758981491) Soft Computing.
21/287
&x#4
&x#4
.
i
dentity/
.subset%ood/
&x#4 G .")* .x/ ma*.." .x/& .* .
x//&x#4 G ."* .x/ min.." .x/& .* .
x//
.union/
.intersection/
&x#
4
G .4 .x/ ' 4." .x/
"
.complement/
0nion could also be represented by" )" .x&
ma*..
Same notation could also be used ,it% intersection and
complement$
.x/& .
*
.x///x#4
+
" is a subset of *)%e grap% of t%e universal fu##y
subset in 4 = [0, 10]$
Fi#ure !.- !u##y sets
T,eorem !.1.1 )%e follo,ing properties of set t%eory are
valid
" " .involution/
" )* * )
" " * *
"
." )*/ )C " ).*)C/ ." */ C "
.* C/
.commutativity/
.associativity/
" .* )C/ ." */ )."C/
" ).* C/ ." )*/ .")C/
.distributivity/
" )" "
" " "
.idempotence/
"
-
7/24/2019 (758981491) Soft Computing.
22/287
" )." */ "
" ." )*/ "
.absorption/
." )*/
." */
" *
" )
*
.De >organ=s la,s/
Proo*2 -bove properties can be proved by simple direct
calculations$ !or e*ample&
" ' 4 .' 4"/ " $
)%e la,s of contradiction " " /
.basis for many mat%ematical proofs/ and e*cluded
middle " )" 4
are not valid for fu##y sets$ Ho,ever& for t%e crisp sets
t%ey are valid$
1ertrand ussell commented t%e la, of e*cluded middle in t%e
follo,ing ,ayG
The law of excluded middle is true when precise symbols are
employed but it is not true when
symbols are vague, as, in fact, all symbols are.
-lso ot%er definitions for set operations are possible$ -n
alternative ,ellkno,n definition of unionand intersection is given
belo,$ 7perations are mostly selected relying on intuition or
empiricalresults in particular problem$De*inition !.1.1)
&x#4 G .")*
.x/ min.'& ."
.x/ 3.*
.
x//&x #4 G ."* .x/ ." .x/ !.* .x/
.union/
.intersection/
Intersection of t,o triangular fu##y numbers 0nion of t,o
triangular numbers
Fi#ure !. 7perations on fu##y sets
-
7/24/2019 (758981491) Soft Computing.
23/287
>ore generally& t%e term aggregation operator is used to
describe t%e union and intersectionoperators$ -ggregationoperators
can be divided into t,o groupsG compensatory and non0compensatory$
Noncompensatory intersection operators are called t0norms and union
operators arecalled t0conorms or s0norms$ Compensatory ones can
perform operations ,%ic% are some,%ere
bet,een intersection and union operations$De*inition !.1.1-
.t0norm& generali#ed intersection/ !unctionis a tnorm& or
triangular norm& if it satisfies t%e criteriaG
T GB5&'EB5&'E /B5&'E
.i/
.ii/T .x&y/ T .y&x/
T.T.x&y/&z/ T.x&T.y&z//
.symmetry/
.associativity/
.iii/ x ?z and y ?
implies T .x&y/ ?T.z&
/
.monotonicity/
.iv/ T .x&'/ x
.border condition&one identy/
-
7/24/2019 (758981491) Soft Computing.
24/287
)%e basic tnorms areG
! minimumG min.a&b/ mina&b+&
! Lukasie,is#G T .a&b/ ma*a3b 4'&5+
! productG T p .a& b/ ab
! ,eakGmina& b+ if ma*a& b+'
Tw
.a& b/ 5 ot%er,ise
! Hamac%er$
.a&b/ ab
3.'4/.a3b 4ab/
& 5
! Dubois and 6rade
ab7 .a&b/ &;#
.5&'/ma*a&b&;+
! ager
:p.a&b/ ' 4min'&p 6.'4
a/ p
3.' 3b/p 7+&p 55
! !rank
!:.a& b/
mina&b+
Tp
.a& b/
T9
.a&b/a b
if :
5
if :'
if :
>
'4log
:'
.: 4'/.: 4'/
ot%er,ise
:
4'
De*inition !.1.1 .t0conorm& generali#ed union/ !unctionis a
t conorm .snorm/ if it satisfies t%e criteriaG
.i/.iii/ Same as in t%e definition of tnorms
S GB5$'E B5$'E /
B5$'E
.iv/ S .x&5/ x
)%ere is a special relation bet,een tnorms and tconorms if t%ey
are conJugate to eac% ot%erG
S .x&y/ ' 4T .' 4x&' 4y/ $ -mong ot%er& minimum and
ma*imum operations satisfyt%is relation BDriankov et$ al$&
'::(E$ elation is not essential& but it is sometimes used to
select
suitable pair of operations$
.i/ S .x& y/
S .y&x/
.symmetry/
.ii/ S .x & S .y & z //
9
;
3
-
7/24/2019 (758981491) Soft Computing.
25/287
S . S.x &
y /& z / .associativity/
.iii/ S .x &y / ? S .x T &y T / if x ?x T andy ?yT
.monotonicity/
.iv/ S .x & 5 / x&
&x #65&'7
.#ero identy/
)%e basic tconorms areG
! ma*imumG ma*.a&b/ ma*a&b+
! Lukasie,ic#G
! probabilisticG
! strong
S9
.a&b/ mina3b&'+
Sp
.a&b/ a 3b 4ab
ST8/1;.a& b/ ma*a& b+ if min a& b+ 5
' ot%er,ise
! Hamac%er
$/8 . a & b /
a 3 b 4 . 2 4 / a b& 5
' 4 .' 4 / ab
! ager:/8
p.a& b/ min'&p a
p
3b p +&p 55
Note2 If t%e product .see def$ 2$'$'+/ is used as a tnorm&
t%e idempotence is not valid since.
".x/ ." .x/ ." .
x/$ -lso t%e distributivity is not valid& ,%ic% can be
verified by substituting
operations in def$ 2$'$'+ into distributivity la,$ Ho,ever&
t%e product function is gradually replacingt%e min operation in t%e
real ,orld applications$ It %as been s%o,n to perform better in
manysituations BDriankov et$ al$& '::( 1ro,n Harris&
'::+E$
!.1.! Fu""y Re&%tions
De*inition !.1.15 .fuzzy relation/ !u##y relation is
c%aracteri#ed by a function.
8G4' $$$4 m /
B5&'E
,%ere 4i are t%e universes of discourse and 4 ' $$$ 4 m
is t%e product space$ If ,e %ave t,o finite universes& t%e
fu##y relation can be presented as a matri*.fuzzy matrix/ ,%ose
elements are t%e intensities of t%e relation andR %as t%e
members%ip function
.8 .u&/ & ,%ere u #4' 2 $ ),o fu##y relations are
combined by a so called supUor
ma*min composition& ,%ic% ,ill be given in t%e definition
2$'$':$
-
7/24/2019 (758981491) Soft Computing.
26/287
Fi#ure !.5 S%ado,s of a fu##y relation
Note2 !u##y relations are fu##y sets& and so t%e operations
of fu##y sets .union& intersection& etc$/can be applied to
t%em$
E/%m+&e !.1.! Let t%e fu##y relation8 appro*imately eAual
correspond to t%e eAuality of t,onumbers$ )%e intensity of cell
8.u&/ of t%e follo,ing matri* can be interpreted as t%e degree
ofmembers%ip of t%e ordered pair in8$ )%e numbers to be compared
are '&2&(&++and (&+&&9+$
u V ' 2 ( +
( $9 $; ' $;8.u&/ + $+ $9 $; '
$2 $+ $9 $;9 $' $2 $+ $9
)%e matri* s%o,s& t%at t%e pair .+& +/ is appro*imately
eAual ,it% intensity ' and t%e pair .'& 9/ isappro*imately
eAual ,it% intensity 5$'$
De*inition !.1.16 .Cartesian product/ Letdefined
"i#4
i
be fu##y sets$ )%en t%e Cartesian product is
."'$$$"n
.x'&$$$&xn / min .'?i ?n "i .xi/+ .2$'/
Note2 min can be replaced by a more general tnorm$
-
7/24/2019 (758981491) Soft Computing.
27/287
Fi#ure !.6 Cartesian product of t,o fu##y sets is a fu##y
relation in 4 :
De*inition !.1.17 .sup0
-
7/24/2019 (758981491) Soft Computing.
28/287
)%e interpretation of e*ampleG x is small$ If x and y are
appro*imately eAual& y is more or lesssmall$
-
7/24/2019 (758981491) Soft Computing.
29/287
!.1.$ T,e E/tension Prini+&e
)%e e*tension principle is said to be one of t%e most important
tools in fu##y logic$ It gives means togenerali#e nonfu##y
concepts& e$g$& mat%ematical operations& to fu##y sets$
-ny fu##ifying
generali#ation must be consistent ,it% t%e crisp
cases$De*inition !.1.!8 .extension principle/ Let "'
&$$$&
"n
be fu##y sets& defined on4'$$$4n
and let
f be a function f G4 i $$$4 n /=
$ )%e e*tension of f operating on "' &$$$&"
n
gives a
members%ip function .fu##y set ! /
.!
./
supu'
&$$$&un#f
./ min.. ' .u' /&$$$& ."
.un// .2$';/
,%en t%e inverse of f e*ists$ 7t%er,ise definemapping$
.!
./ 5 $ !unction f is calledinducing
If t%e domain is eit%er discrete or compact& supmin can be
replaced by ma*min$ 7n continuousdomains supoperation and t%e
operation t%at satisfies criterion
x & y 5
Sw .x&y/ y & x 5 .2$':/
' & ot%er,ise
s%ould be used BDriankov et$ al$& '::(E$
!.1.) 9++ro/im%te Re%sonin#
De*inition !.1.!1 .linguistic variable, fuzzy variable/ "
linguistic variable is c%aracteri#ed by aAuintuple .= &T&4
&;&> /& ,%ere = denotes t%e symbolic name of a
linguistic variable&e$g$&
error& temperature$ T is t%e set of linguistic values t%at =
can take .term set of = /& e$g$&
Wnegative&zero&positiveX$ 4 is t%e domain over ,%ic% t%e
linguistic variable = takes its Auantitative values&
e$g$& a continuous interval B4'5&'5E $ Sometimes t%e
universe of discourse is used instead of 4 $; is a syntactic rule
,%ic% defines t%e elements .names of values/ of = and > is a
semantic
rule ,%ic% gives t%e interpretation of linguistic value in terms
of Auantitative elements$
)%e definition 2$'$2' says t%at linguistic variable may take
,ords in natural language .or numbers/
as its values$ )%ese ,ords are labels of fu##y sets$ !ig$ 2$:
describes t%e linguistic variablegrap%ically$
4'" n
-
7/24/2019 (758981491) Soft Computing.
30/287
De*inition !.1.!! .completeness/ )%e fu##y variable is said to
be complete if for eac% x #4t%ere e*ists a fu##y set " suc%
t%at
."
.x/ 55 .2$25/
Fi#ure !.7 Linguistic variable Ovelocity=
De*inition !.1.!$ .hedge/ " %edge is an operation ,%ic% modifies
t%e meaning of t%e termG
concentration .very/G
dilation .more or less/G
.con."/
.x/ .."
.x//
'
.dil ."/ .x/ . ." .x//
2
.2$2'/
.2$2'/
contrast intensificationG
.int."/ .x/
2..
.x// 2 & 5 ? ." .x /?
5 $
.2$2(/
' 4 2.' 4 . .x//2
& 5$ ."
.x/ ?'
If t%e term is not derived from some ot%er term by using
%edge& it is called a primary term$ If t%eprimary term Ocold=
is modified by t%e concentration& ,e get a ne, term Overy
cold=$
>ore generally& linguistic variable can be seen as a
micro language& ,%ic% includes conte*tfreegrammar and
attributedgrammar semantics$ Conte*tfree grammar determines
permissible values oflinguistic variable and attributedgrammar
semantics gives t%e possibility to add modifiers
.%edges/BFade%& '::2E$
In appro*imate reasoning& t%ere are t,o important inference
rules& a generalized modus ponens.6/ ,%ic% is a for,ard
reasoning rule and a generalized modus tollens .)/ ,%ic% is a
back,ard reasoning rule$ )%ey are defined in t%e follo,ing$
2
"
"
-
7/24/2019 (758981491) Soft Computing.
31/287
De*inition !.1.!) .;eneralized >odus #onens/ 6 is defined as
BFade%& '::2 Fimmermann&'::(E
premiseG x is "&
implication G If x is " & t%en y is *
conseAuence G y is *&
,%ere " &"&& * &*
&are fu##y sets and x and y are symbolic names for
obJects$
De*inition !.1.!- .;eneralized >odus Tollens/ ) is defined as
t%e follo,ing inference BFade%&'::2 Fimmermann& '::(E
premiseG y is *&
implicationG If x is " & t%en y is *
conseAuenceG x is "&
,%ere"&"=&*&*= are again fu##y sets$
-
7/24/2019 (758981491) Soft Computing.
32/287
6ropositional calculusG ."/*
.u&/ ."
.u/ 3."*
./
.2$(5/
-
7/24/2019 (758981491) Soft Computing.
33/287
'$ %uman e*perts provide rules2$ data drivenG rules are formed
by training met%ods($ combination of '$ and 2$
)%e first ,ay is t%e ideal case for fu##y systems$ -lt%oug% t%e
rules are not precise& t%ey containimportant information about
t%e system$ In practice %uman e*perts may not provide a
sufficientnumber of rules and especially in t%e case of comple*
systems t%e amount of kno,ledge may bevery small or even
none*istent$ )%us t%e second ,ay must be used instead of t%e first
one .providedt%e data is available/$ )%e t%ird ,ay is suited for
t%e cases ,%en some kno,ledge e*ists andsufficient amount of data
for training is available$ In t%is case fu##y rules got from
e*perts roug%lyappro*imate t%e be%avior of t%e system& and by
applying training t%is appro*imation is made more
precise$ ules provided by t%e e*pert=s form an initial point for
t%e training and t%us e*clude t%enecessity of random initiali#ation
and diminis% t%e risk of getting stuck in a local minimum.provided
t%e e*pert kno,ledge is good enoug%/$
It %as been s%o,n in B>ou#ouris& '::9E t%at linguistic
information is important in t%e absence ofsufficient numerical data
but it becomes less important as more data become available$
!u##y rules define t%e connection bet,een input and output fu##y
linguistic variables and t%ey canbe seen to act as associative
memories$ esembling inputs are converted to resembling outputs$ules
%ave a structure of t%e formG
IF .antecedent/ T:EN .conseAuent/ .2$((/
In more detail& t%e structure is
IF .x'
is"i 9ND
Z9ND
xis "i T:EN . yis
*
i
/ .2$(+/
i,%ere "i i and* are fu##y sets .t%ey define complete fu##y
partitions/ in 6 %R = %R&respectively$ Linguistic variable / is
a vector of dimension d in 6 ' $$$ 6
d
and linguistic
variable y #= $ @ector / is an input to t%e fu##y system and y
is an output of t%e fu##y system$
Note t%at
*can also be a singleton .conseAuence part becomesG $$$T:EN .y
is zi
//$ !urt%er& if
fu##y system is used as a classifier& t%e conseAuence part
becomesG $$$T:EN class is c$
- fu##y rule base consists of a collection of rules 8'&82
&$$$&8> +& ,%ere eac%
rule
8i
can be
considered to be of t%e form .2$(+/$ )%is does not cause a loss
of generality& since multiinputmultioutput .>I>7/ fu##y
logic system can al,ays be decomposed into a group of
multiinputsingleoutput .>IS7/ fu##y logic systems$ !urt%ermore
.2$(+/ includes t%e follo,ing types of rules as aspecial case
B?ang& '::+EG
!IF .x' is"
i9ND ; 9ND x is
i
"i
< T:EN .y is
i
*i/& ,%ere k d
ii!IF .. x' is "' 9ND ; 9ND xk is "k < OR .xk 3' is "k3' 9ND
; 9ND xd is "d // T:EN
.y is
' dd
' kk
-
7/24/2019 (758981491) Soft Computing.
34/287
!y is
!.y is
*i/
*i
*i / UNLESS .x is
i' 9ND $$$ 9ND xd is "d /
!nonfu##y rules
In control systems t%e production rules are of t%e form
IF Pprocess stateR T:EN Pcontrol actionR .2$(/
,%ere t%e Pprocess stateR part contains a description of t%e
process output at t%e kt% samplinginstant$ 0sually t%is description
contains values of error and c%angeoferror$ )%e Pcontrol
actionR
part describes t%e control output .c%angeincontrol/ ,%ic% s%ould
be produced given t%e particularPprocess stateR$ !or e*ample& a
fu##ified 6I controller is of t%e type .2$(/$ Important properties
fora set of rules are completeness& consistency and continuity$
)%ese are defined in t%e follo,ing$
De*inition !.1.!5 .completeness/ - rule base is said to be
complete if any combination of inputvalues results in an
appropriate output valueG
&/ #= G %gt.out./// R5 .2$(9/
De*inition !.1.!6 .inconsistency/ - rule base is inconsistent if
t%ere are t,o rules ,it% t%e same ruleantecedent but different rule
conseAuences$
)%is means t%at t,o rules t%at %ave t%e same antecedent map to
t,o nonoverlapping fu##y outputsets$ ?%en t%e output sets are
nonoverlapping& t%en t%ere is somet%ing ,rong ,it% t%e
outputvariables or t%e rules are inconsistent or discontinuous$
De*inition !.1.!7 .continuity/ - rule base is continuous if t%e
neig%boring rules do not %ave fu##youtput sets t%at %ave empty
intersection$
!.1. Fu""y In*erene
-s mentioned earlier& a fu##y I!)H8N rule is interpreted as
a fu##y implication$ )%e t,o mostcommon operation rules of fu##y
implications are!>inioperation rule .>amdani
implication/G
."/*
./4y min ." .//& .* .y/
+
.2$(4/
' "
i
-
7/24/2019 (758981491) Soft Computing.
35/287
Fi#ure !.18 Inference ,it% >amdani=s implication
operator!6roductoperation rule .Larsen implication/G
."/* ./4y ."
?/ fu##y sets in t%e form.2$+'/ or t%e union of > fu##y
sets
.* T .y/
.'T .y/ 3 $$$3
*
.> T .y/*.2$+2/
! )sukamoto implication$ -ll linguistic terms are supposed to
%ave monotonic members%ip
functions$ )%e firing levels of t%e rules& denoted by ;i
& i
'&2&
are computed by
;' "' .x5 / *' .y5 /& ;
2
"2 .x5 /
*2 .y 5 /
In t%is mode of reasoning t%e individual crisp control actions
z' andz 2 are computed from t%e
eAuations ;' C' .z' /& ;2 C2 .z2
/
and t%e overall crisp control action is e*pressed as
;z 3;z
z ' ' 2 253;
i$e$ z5 is computed by t%e discrete Centerof
-
7/24/2019 (758981491) Soft Computing.
37/287
! Sugeno implication . Sugeno and )akagi use t%e follo,ing
arc%itecture
R ' 2 if x isalso
" and y is *' t%en z' ;'x 3b'y
R 2 2 if x is "2 and y is *2 t%en z2 ;2x 3b2y*%t2 x is x5
ons.2
and y is y 5
z5
)%e firing levels of t%e rules are computed by ;' "' .x5 / *'
.y5 /& ;2
"2 .x5 / *2 .y5 /
t%en t%e individual rule outputs are derived from t%e
relations%ips z'
z;x 3b y
;'x53b'y5 &
;z3;z
z ' ' 2 2
and t%e crisp control action is e*pressed as;' 3;2
If ,e %ave n rules in our rulebase t%en t%e crisp control action
is computed as
n
z5 (i '; iz i
n
i' i
,%ere ;i denotes t%e firing level of t%e i t% rule& i
'&$$$& n
Fi#ure !.1$ Sample of Sugeno=s inference mec%anism
E/%m+&e !.1.) Let t%e input be a crisp value and let t%e
minioperation rule .2$(4/ be used ,it%.2$(:/$ ?%en .2$+2/ is
applied ,e get an output set illustrated in !ig$ 2$'+$
'
2 2 5 2 5
5
(;
-
7/24/2019 (758981491) Soft Computing.
38/287
$
IF .x' is
IF .x' is
'
' 9ND
29ND
x2 is
x2 is
"'
/ T:EN .y is
"2
/ T:EN .y is
*'
/
*2
/
Fi#ure !.1) -n e*ample of fu##y inference
!.1.5 Fu""i*ier %nd De*u""i*ier
)%e fuzzifier maps a crisp point / into a fu##y setfu##y set is
simply given by a singletonG
"& $ ?%en an input is a numerical value& t%e
."T .x/
'& if x xT.2$+(/
5& ot%er,ise
,%ere xT is t%e input$ !u##ifier of t%e form .2$+(/ is called a
singleton fuzzifier$ If t%e inputcontains noise it can be modeled
by using fu##y number$ Suc% fu##ifier could be called anonsingleton
fuzzifier$ 1ecause of simplicity& singleton fu##ifier is
preferred to nonsingletonfu##ifier$
"
"
'
2
2
-
7/24/2019 (758981491) Soft Computing.
39/287
Fi#ure !.1- !u#y logic controller Fi#ure !.1 !u##y singleton as
fu##ifier
)%e defuzzifier maps fu##y sets to a crisp point$ Several
defu##ification met%ods %ave beensuggested$ )%e follo,ing five are
t%e most commonG
!Center o* Gr%vity .Co
-
7/24/2019 (758981491) Soft Computing.
40/287
Fi#ure !.15 Defu##ification met%ods
Defu##ification met%ods can be compared by some criteria&
,%ic% mig%t be t%e continuity of t%eoutput and t%e computational
comple*ity$ H>& Co- and CoS produce continuous outputs$
)%esimplest and Auickest met%od of t%ese is t%e H> met%od and
for large problems it is t%e best c%oice$)%e fu##y systems using it
%ave a close relation to some ,ellkno,n interpolation met%ods .,e
,ill
return to t%is relation later/$
)%e ma*imum met%ods .>o>& !o>/ %ave been ,idely
used$ )%e underlying idea of >o> .,it%ma*min inference/ can
be e*plained as follo,s$ 8ac% input variable is divided into a
number ofintervals& ,%ic% means t%at t%e ,%ole input space is
divided into a large number of ddimensional
bo*es$ If a ne, input point is given& t%e corresponding
value for y is determined by finding ,%ic%bo* t%e point falls in
and t%en returning t%e average value of t%e correspondingyinterval
associated,it% t%at input bo*$ 1ecause of t%e piece,ise constant
output& >o> is inefficient for appro*imatingnonlinear
continuous functions$ Kosko %as s%o,n BKosko& '::4E t%at if
t%ere are many rules t%at firesimultaneously& t%e ma*imum
function tends to approac% a constant function$ )%is may cause
problems especially in control$
)%e property of CoS is t%at t%e s%ape of final members%ip
function used as a basis fordefu##ification ,ill resemble more and
more normal density function ,%en t%e number of functionsused in
summation gro,s$ Systems of t%is type t%at sum up t%e output
members%ip functions toform final output set are called additive
fuzzy systems$
!.1.6 T,e Gener%& Re&%tion 0et'een Pro0%0i&ity %nd
Fu""iness
)%e relation bet,een probability and fu##iness is per%aps best
described by Kosko in BKosko&
'::5E$ He uses ndimensional %ypercubes to illustrate t%is
relation and s%o,s t%at t%e probabilitydistributions occupy points
along t%e diagonal of .n 4 '/ dimensional %ypertetra%edron of
t%efu##y set .!ig$ 2$';/$ ?e notice t%at t%ere are far more fu##y
sets ,%ic% are not probabilitydistributions& and ,%ic% fill t%e
%ypercube$ In fu##y %ypercube& eac% a*is corresponds to
t%emembers%ip grade of a fu##y set$
oslyn %as compared fu##y logic& probability and possibility
in t%e basis of possibility t%eoryBoslyn& '::+E$ He claims&
t%at alt%oug% probability and possibility are logically
independent& t%eirmeasures bot% arise from DempsterS%afer
evidence t%eory as fu##y sets defined on random sets andt%eir
distributions are fu##y sets$
-
7/24/2019 (758981491) Soft Computing.
41/287
Fi#ure !.16 6robability in t%e fu##y 2dimensional %ypercube for
t,o sets$ In t%e diagonal line aret%e points t%at=s sum to unity$
.If t%e possibility distributions %ad been dra,n& t%ey ,ould be
t%eupper and t%e rig%t edge of t%e %ypercube/$
!.! Di**erent Fu""y Systems
!u##y system is a set of rules& ,%ic% converts inputs to
outputs$ 1y allo,ing partial members%ips& itis possible to
represent a smoot% transition from one rule to anot%er&
i$e$& to interpolate bet,een t%erules$ In t%e follo,ing t,o
commonly used fu##y systems& ,%ic% %ave importance in t%e
laterc%apters& are presented$
!.!.1 T%(%#i3Su#eno@s Fu""y System
In fu##y systems& t,o types of rules can be distinguis%edG
rules of t%e type presented earlier .2$(+/
called >amdani rules& and )akagiSugeno rules of t%e type
.2$++/$
IF .x' is "'9ND ; 9ND x is "
i/ T:EN y
if .x' &$$$&xd
/
.2$++/
,%ere t%e conseAuences of t%e fu##y rules are functions of t%e
input vectors /$ Inde* i means t%e
i t% rule and " s are fu##y sets$ Sugeno used linear
combinations of input values as output
functions yy c5
d
3(k '
ckx
k .2$+/
,%ere ck are realvalued parameters and specific for eac% rule$
!unction can also be nonlinear ,it%respect to inputs$ - ,eig%ted
sum is used as defu##ification met%odG
k
(y Jd
."
.xi/
y .// '
k
i '
d.2$+9/
(." .xi/
i' d
d
i
i
-
7/24/2019 (758981491) Soft Computing.
42/287
' i '
since it provides a smoot% interpolation of t%e local models$
)%is type of fu##y system %as been
successfully used in control problems& for e*ample in t%e
control of a model car BSugeno >urakami& ':;+E$ )%e output
of t%e system is a ,eig%ted sum of functions$ )%us .2$+9/
interpolatesbet,een different linear functions$
Fi#ure !.17 - function f defined by )SK model$ Das%ed lines
represent local models
!.!.! Mende&3A%n#@s Fu""y System
>endel?ang=s fu##y logic system is presented ask
(wld
.."l .xi /
l' i'y .// k d .2$+4/
(.."l.xi /l'
i
i'
,%ere wl
is a place of output singleton$ )%e members%ip function .l
.xi/
i
corresponds to t%e
input xi of t%e rule l $ In s%ort& t%e and connective in t%e
premise is carried out by a product and
i
"
-
7/24/2019 (758981491) Soft Computing.
43/287
defu##ification by t%e Heig%t >et%od .a special case of
Center of amdani type$ )%e fu##y basis function is of t%e form
d
.
! .x
i
/
i '
b .// k d .2$+;/
(..! .xi//
i ' i '
,%ere t%e denominator normali#es t%e product of members%ip
functions so t%at t%e sum of basisfunctions yields one at eac%
point$ !unctions .2$+;/ are called basis functions even t%oug% t%ey
are
not al,ays ort%ogonal$ )%e output of .2$+4/ eAuals a discrete
e*pectation of t%e k t%enpart setcentroids wi $ 1asis functions
could also be of t%e form
min. .xi /xi
i
#4
i &i '&$$$&d+
b .// k
+.2$+:/
(min .! .xi /xi #4 i &i '&$$$& di'
if miniinference rules ,ere used$ Ho,ever& modern fu##y
systems dispense ,it% min and ma*$)%ey consist of only
multiplies& additions& and divisions BKosko& '::4E$
System .2$+4/ can bee*pressed as a nonlinear mapping ,%ic% is
formed by a ,eig%ted sum
y .//k
(wi bi .// .2$5/i '
-lt%oug% .2$5/ does not look like a fu##y system& it can be
interpreted as suc%& because it is formedby using members%ip
functions& tnorms and tconorms$ In t%e follo,ing& t%e use
of fu##y system oft%e form .2$+4/ is Justified$
A,y sin#&eton *u""i*ier %nd sin#&eton out+ut mem0ers,i+
*untions
Singleton fu##ifier is simpler and more used t%an its
nonsingleton counterpart$ Ho,ever& it is
assumed t%at nonsingleton fu##ifier may ,ork better in a noisy
environment as it modelsuncertainty& imprecision or inaccuracy
in t%e inputs B?ang& ':+ ager& '::E$ If t%e
outputmembers%ip functions ,ere continuous& t%ey ,ould %ave to
be discreti#ed in defu##ification$ -s aconseAuence& tnorm
.product/ and tconorm .sum/ ,ould %ave to be calculated at every
discreti#ing
point B@ilJamaa& '::9E$
A,y +rodut *or %nd 3o+er%tion
)%e use of productrule is Justified by t%e a*ioms for
aggregation operations 6Klir& ':;;7$ ?it%aggregation operation
several fu##y sets are combined to produce a single set as
described earlier$-ggregation operation is defined by h GB5&'E
/B5&'E
and t%e a*ioms are
i
i
!
i
-
7/24/2019 (758981491) Soft Computing.
44/287
'$ h .8/5 and h .1/'
2$ h is monotonic nondecreasing in all its argumentsG If
$
xiy
i& t%en h.xi / h.yi /
)%e follo,ing additional a*ioms are not essential& but t%ey
are usually taken into account
'$ h is continuous2$ h is symmetric in all its arguments
)%e most significant advantage of product over minimum operator
is t%e fact t%at product retainsmore information$ )%is can be
seen& for e*ample& in t%e case of t,odimensional input
space$ ?%ent%e product operation is used& bot% inputs ,ill %ave
an effect on t%e output$ If t%e min operation isused& only one
input %as effect on t%e output$ It is e*tremely difficult to design
a fu##y system basedon ma*min operations suc% t%at t%e
interpolation properties of t%e resulting system can be
,ellunderstood$ It can be only kno,n t%at t%e value in t%e interior
of an interval lies bet,een t%e valuesat eit%er end 61ro,n
Harris& '::+7$
A,y :ei#,t Met,od *or de*u""i*i%tion
Driankov %as compared in BDriankov et$ al$& '::(E si* best
kno,n defu##ification met%ods$ )%eresult of t%is comparison is t%at
Heig%t >et%od& Center of Sums and Center of -rea satisfy
eAualnumber of criteria and more t%an t%e ot%er met%ods$ Criteria
are continuity& disambiguity .i$e$& can
produce a crisp output for every combination of output
members%ip functions/& plausibility .outputlies appro*imately
in t%e middle of support and %as a %ig% degree of members%ip/&
computationalcomple*ity and ,eig%t counting .,eig%t information is
not lost/$ )%ey all are& in fact& variations of
Center of
-
7/24/2019 (758981491) Soft Computing.
45/287
Out+ut %s ondition%& %ver%#e in s%&%r v%&ued FLS
?sin#&eton out+uts uses t%e volume or area =i of set *i and
its
center of mass$
local rule outputd
ii is a scalar rule ,eig%t$ )%e larger =
ii t%e more t%e global mapping acts like t%e
wi$ )%e S-> reduces to t%e Co< model if t%e relative rule
,eig%ts and volumes of
t%e t%en part sets are ignored .e$g$& set to some constant
values& usually to unity/$ It can be easilyseen t%at .2$+4/ is
a special case of .2$(/$
Kosko proved in BKosko& '::4E t%at all center of gravity
fu##y systems are probabilistic systems andt%ey compute a
conditional e*pectation .conditional average/ and t%us compute a
meansAuaredoptimal nonlinear estimator$ !u##y system can be
regarded as a modelfree statistical estimatorsince it does not use
mat%ematical model but appro*imates t%e system ,it% conditional
averageG
y .// Co
-
7/24/2019 (758981491) Soft Computing.
47/287
4y .// / 2
.2$/
2b. / &t /dt
2 .t 4c/
2
i
p .t /dt
and=
i. // 2bi . /4 t /dt
pi.// $
=i is t%e
Tvolume of output set i $
(=
Fi#ure !.!1 @olume of output setT
(2b . /4 t /dt
!.$ 9++ro/im%tion %+%0i&ity
0niversal appro*imator can appro*imate any continuous function
on compact sets to any degree ofaccuracy$ ?ang proved in B?ang&
'::2E t%e follo,ing important t%eoremG
T,eorem !.1.! .6niversal "pproximation Theorem/ !or any given
real continuous function g oncompact set 6 8d #and arbitrary
55& t%ere e*ists a fu##y logic system f in t%e form of
.2$+4/&,%ere t%e members%ip functions are bells%aped
-
7/24/2019 (758981491) Soft Computing.
48/287
f 2 . / / #: 4 f' . / /f 2 . / / #:
and cf' . / / #: & ,%ere c is an arbitrary real valued
coefficient$ )%is is easy to prove for t%e !LSof t%e form .2$+4/
,it% t%e
-
7/24/2019 (758981491) Soft Computing.
49/287
1uckley proved t%e universal appro*imation t%eorem for Sugeno
type of fu##y controllers andCastro Delgado proved it for fu##y
rule systems$ Castro Delgado s%o,ed t%at t%e fu##ysystems
c%aracteri#ed by '/ a fi*ed fu##y inference& 2/ a certain kind
of fu##y relation t%at satisfies asmoot% property& and (/ a
defu##ification met%od& are universal appro*imators$ )%ey also
s%o,ed
t%at t%ere e*ists an optimal fu##y system for a ,ide variety of
problems$ )%is result does not&%o,ever& say& %o, ,e can
design a fu##y system for a particular problem$ BCastro
Delgado&'::9E$
!.) Di**erent Inter+ret%tions o* Fu""y Sets
7ne of t%e main difficulties of fu##y logic %as been ,it% t%e
meaning of members%ip functions$Lack of consensus on t%at meaning
%as created some confusion$ Depending on t%e interpretation
offu##iness& t%e members%ip functions are c%osen based on t%at
interpretation$ )%e more detaileddescription of t%e different
interpretations can be found in B1ilgi[ )\rksen& '::4E$
!T,e &i(e&i,ood vie'G
Sources of fu##iness are errors in measurement .statistical
vie,point/& incomplete information andinterpersonal
contradictions$ If t%e information is perfect& t%ere cannot be
any fu##iness and t%emembers%ip functions are t%res%old functions$
Likeli%ood vie, can be e*plained by a follo,ingillustrative
e*ampleG
E/%m+&e !.1.- ?e %ave some imprecise measurement of room
temperature .,%ic% is ,it%in 25 2=degrees Celsius/$ >easurements
construct an error curve around 25 degrees$ If t%e
temperature lies in :5M of t%e cases in t%e Auanti#ation
interval& ,e ,ould say t%at t%e likeli%oodt%at t%e label&
for e*ample O,arm=& is assigned to t%e room temperature is
5$:G
.warm
.25/ #.warm x 25/ 5$:
>a* and min as proposed by Fade% are not al,ays supported for
t%e calculation of conJunction anddisJunction$
)%is is not t%e only ,ay to treat errors in measurement$ >ore
commonly t%ey are treated byconventional tec%niAues of statistics
.probabilistic 1ayesian inference is regarded as suc% atec%niAue/$
esearc%ers& ,%o represent likeli%ood vie,point& are
Hisdal& >abuc%i and )%omas$)%eir ,ritings can be found in
BHisdal& ':: >abuc%i& '::2 )%omas& '::E$
!R%ndom set vie'
>embers%ip functions are vie,ed %ori#ontally as a family of
level cuts& i$e$& ;cuts definedin .2$''/$ 8ac% levelset is
a set in t%e classical sense$ In t%is vie, t%e members%ip function
can bevie,ed as a uniformly distributed random set$ )o illustrate
t%is vie, t%e e*ample 2$'$ is modifiedand interpreted as follo,sG
:5M of t%e population defined an interval in t%e universe of
discourse fort%e set O,arm= ,%ic% containedx& and (5M defined
intervals ,%ic% did not contain x $
-
7/24/2019 (758981491) Soft Computing.
50/287
!Simi&%rity vie'
>embers%ip measures t%e degree of similarity of an element to
t%e set in Auestion$ )%ere is someprototype or perfect set&
,%ic% is used in comparison$ )%e comparison is made by calculating
t%e
distance bet,een t%e obJect and t%e prototype$ In t%is vie,point
members%ip function could be oft%e form BFysno& ':;'EG
..x/ ' .2$4/
' 3f .dx
/
,%ere dx is t%e distance of t%e obJect from t%e ideal andf is a
function of distance$ Similarity reAuiresa metric space in ,%ic% it
can be defined$ )%us t%is vie, %as connection to >easurement
)%eory$)%e rules in fu##y systems t%at represent similarity vie,
can be described as follo,sG
IF .x' is ' 9ND ; 9NDxd is pi
/ T:EN .y isz i / .2$;/
,%ic% differ from .2$(+/ by t%e fact t%at fu##y sets are
replaced by reference values p .prototypes/
and t%e members%ip function is t%e distance bet,een x andp
$
Interpretation of e*ample 2$'$G oom temperature is a,ay from t%e
prototypical set O,arm= to t%edegree 5$'$
!T,e me%surement vie'
)%e elements in set can be compared to eac% ot%er$ )%e
relations%ip ".x/ 5".y/ says t%at x ismore representative of t%e
set " t%an y $ elative values can be used to get information about
t%e
relative degree of "$
Interpretation of e*ample 2$'$G ?%en compared to ot%er
temperatures in set O,arm=& measuredroom temperature is ,armer
t%an some in t%at set$ - value of 5$: is attac%ed to it on some
scale$
-s contrary to likeli%ood vie,& ma* and min are Justified to
be used for conJunction and disJunctionB1ilgi[ )\rksen&
'::4E$
!.- Di**erent A%ys to *orm Fu""y Sets
)%is is a summary of met%ods described in B1ilgi[ )\rksen&
'::4E$
!Po&&in#2 )%e Auestion Do you agree t%at x is " ] is
stated to different individuals$ -n averageis taken to construct
t%e members%ip function$ -ns,ers are typically of yes3no type$
!Diret r%tin#2 Ho, " isx ] )%is approac% supposes t%at t%e
fu##iness arises from individualsubJective vagueness$ )%e person is
made to classify an obJect over and over again in time in suc%
a
p id
-
7/24/2019 (758981491) Soft Computing.
51/287
,ay t%at it is %ard for %e3s%e to remember t%e past ans,ers$ )%e
members%ip function is constructedby estimating t%e density
function$
!Reverse r%tin#2 )%e Auestion Identify x ,%ic% is " to t%e
degree ."
.x/ is stated toan
individual or a group of individuals$ )%e responses are recorded
and normally distributeddistributions are formed .mean and variance
are estimated/$
!Interv%& estim%tion2 )%e person is asked to give an
interval t%at describes best t%e "ness of x$ )%is is suited to
random set vie, of members%ip functions$
!Mem0ers,i+ e/em+&i*i%tion2 )o ,%at degree x is "] 6erson
may be told to dra, amembers%ip function t%at best describes
"$!P%ir'ise om+%rison2 ?%ic% is a better e*ample of "&
x' or x2 and by %o, muc%] )%e
results of comparisons could be used to fill a matri* of
relative ,eig%ts and t%e members%ip
function is found by taking t%e components of t%e eigenvector
corresponding to t%e ma*imumeigenvalue BC%ameau Santamarina&
':;4 Saaty& ':4+E$!C&usterin# met,ods2 >embers%ip
functions are constructed given a set of data$ 8uclidean normis
used to form clusters on data$!Neuro*u""y te,niues2Neural net,orks
are used to construct members%ip functions$
-n essential part of forming members%ip functions is t%e input
space partitioning$ In !ig$ 2$2( .a/ isa grid& ,%ic% is fi*ed
before%and and it does not c%ange later$
-
7/24/2019 (758981491) Soft Computing.
52/287
C,%+ter $
Neur%& Net'or(s
$.1 B%si Cone+ts
-rtificial neural net,orks .-NN/ %ave been used for t,o main
tasksG '/ function appro*imation and2/ classification problems$
Neural net,orks offer a general frame,ork for representing
nonlinear
mappings
or more generallyy^
f . /4 / .($'/
_kf
k. /& 1 .($2/
,%ere t%e form of t%e mapping is governed by a number of
adJustable parameters &,%ic% can bedetermined by using training
met%ods e*plained in t%e c%apter $
)%e classification problem can be regarded as a special case of
function appro*imation& in ,%ic% t%emapping is done from t%e
input space to a finite set of output classes$ )%e arrangement of
t%e
net,ork=s nodes .neurons/ and connections defines its
arc%itecture$ Some of t%e net,orkarc%itectures are better suited
for classification t%an for function appro*imation$ Suc% net,orks
areSelforgani#ing maps .S7>/ developed by Ko%onen B'::5E and
net,orks using learning vectorAuanti#ation .L@`/ algorit%ms$
"rtificial neural systems can be considered as simplified
mat%ematical models of brainlike systemsand t%ey function as
parallel distributed computing net,orks$ Ho,ever& in contrast
to conventionalcomputers& ,%ic% are programmed to perform
specific task& most neural net,orks must be taug%t&
ortrained$ )%ey can learn ne, associations& ne, functional
dependencies and ne, patterns$ -lt%oug%computers outperform bot%
biological and artificial neural systems for tasks based on precise
andfast arit%metic operations& artificial neural systems
represent t%e promising ne, generation ofinformation processing
net,orks$
)%e study of brainstyle computation %as its roots over 5 years
ago in t%e ,ork of >cCulloc% and6itts .':+(/ and slig%tly later
in Hebb=s famous /rganization of *ehavior .':+:/$ )%e early ,ork
inartificial intelligence ,as torn bet,een t%ose ,%o believed t%at
intelligent systems could best be
built on computers modeled after brains& and t%ose like
>insky and 6apert .':9:/ ,%o believed t%atintelligence ,as
fundamentally symbol processing of t%e kind readily modeled on t%e
von 1eumanncomputer$ !or a variety of reasons& t%e
symbolprocessing approac% became t%e dominant t%eme in
"rtifcial Intelligence in t%e ':45s$ Ho,ever& t%e ':;5s
s%o,ed a rebirt% in interest in neuralcomputingG
y
-
7/24/2019 (758981491) Soft Computing.
53/287
176! Hopfield provided t%e mat%ematical foundation for
understanding t%e dynamics of anmportant class of net,orks$176)
Ko%onen developed unsupervised learning net,orks for feature
mapping into regular arrays ofneurons$176 umel%art and
>cClelland introduced t%e backpropagation learning algorit%m for
comple*&multilayer net,orks$
1eginning in ':;9;4& many neural net,orks researc% programs
,ere initiated$ )%e list ofapplications t%at can be solved by
neural net,orks %as e*panded from small testsi#e e*amples tolarge
practical tasks$ @erylargescale integrated neural net,ork c%ips
%ave been fabricated$
In t%e long term& ,e could e*pect t%at artificial neural
systems ,ill be used in applications involvingvision&
speec%& decision making& and reasoning& but also as
signal processors suc% as filters&detectors& and Auality
control systems$
De*inition."rtificial neural systems, or neural networks, are
physical cellular systems which can
ac+uire, store, and utilize experiental knowledge.
)%e kno,ledge is in t%e form of stable states or mappings
embedded in net,orks t%at can berecalled in response to t%e
presentation of cues$
Fi#ure $.1 - multilayer feedfor,ard neural net,ork$
)%e basic processing elements of neural net,orks are called
artificial neurons& or simply neurons ornodes$
8ac% processing unit is c%aracteri#ed by an activity level
.representing t%e state of polari#ation of aneuron/& an output
value .representing t%e firing rate of t%e neuron/& a set of
input connections&.representing synapses on t%e cell and its
dendrite/& a bias value .representing an internal restinglevel
of t%e neuron/& and a set of output connections .representing a
neuron=s a*onal proJections/$
8ac% of t%ese aspects of t%e unit are represented mat%ematically
by real
numbers$
-
7/24/2019 (758981491) Soft Computing.
54/287
)%us& eac% connection %as an associated ,eig%t .synaptic
strengt%/ ,%ic% determines t%e effect oft%e incoming input on t%e
activation level of t%e unit$
)%e ,eig%ts may be positive .e*citatory/ or negative
.in%ibitory/$
Fi#ure $.! - processing element ,it% single output connection)%e
signal flo, from of neuron inputs& x
& is considered to be unidirectionalas indicated by
arro,s&
as is a neuron=s output signal flo,$ )%e neuron output signal is
given by t%e follo,ing relations%ip
o f .w&x / f .wTx/ n
f .(wx /'
,%ere w .w' &$$$&w
n
/T #R n is t%e ,eig%t vector$ )%efunction
f .wTx/ is often referred to as
an activation ?or transfer@ function$ Its domain is t%e set of
activation values& net& of t%e neuronmodel& ,e t%us
often use t%is function as f (net)$ )%e variable net is defined as
a scalar product oft%e ,eig%t and input vectors
net w&x
wTx w x 3$$$ 3w x
' ' n n
and in t%e simplest case t%e output value o is computed as
o f .net/
' if wTx
5 ot%er,ise
,%ere is called t%res%oldlevel and t%is type of node is called a
linear threshold unit$
Net,ork arc%itectures can be categori#ed to t%ree main typesG
feedforward networks& recurrentnetworks .feedback networks/
andself0organizing networks$ )%is classification of net,orks %as
been
proposed by Ko%onen B'::5E$ Net,ork is feedfor,ard if all of t%e
%idden and output neuronsreceive inputs from t%e preceding layer
only$ )%e input is presented to t%e input layer and it is
propagated for,ards t%roug% t%e net,ork$ 7utput never forms a
part of its o,n input$ ecurrentnet,ork %as at least one feedback
loop& i$e$& cyclic connection& ,%ic% means t%at at
least one of its
-
7/24/2019 (758981491) Soft Computing.
55/287
neurons feed its signal back to t%e inputs of all t%e ot%er
neurons$ )%e be%avior of suc% net,orksmay be e*tremely comple*$
Haykin divides net,orks into four classes BHaykin& '::+EG '/
single0layer feedforward networks& 2/
multilayer feedforward networks& (/ recurrent networks&
and +/ lattice structures$ - lattice net,orkis a feedfor,ard
net,ork& ,%ic% %as output neurons arranged in ro,s and
columns$
Layered net,orks are said to be fully connected if every node in
eac% layer is connected to all t%efollo,ing layer nodes$ If any of
t%e connections is missing& t%en net,ork is said to be
partiallyconnected$ 6artially connected net,orks can be formed if
some prior information about t%e problemis available and t%is
information supports t%e use of suc% a structure$
)%e follo,ing treatment of net,orks applies mainly to
feedfor,ard net,orks .single layer net,orks&>L6& 1!&
etc$/$ )%e designation nlayer net,ork refers to t%e number of
computational nodes ort%e number of ,eig%t connection layers$ )%us
t%e input node layer is not taken into account$
$.1.1 Sin#&e3&%yer Feed*or'%rd Net'or(s
)%e simplest c%oice of neural net,ork is t%e follo,ing ,eig%ted
sumd
y. // (wixii5
.($(/
,%ere d is t%e dimension of input space& x5 'and w5 is t%e
bias parameter$ Input vector / can
be considered as a set of activations of input layer$ In
classification problem .($(/ is called adiscriminant function&
because y .// 5 can be interpreted as a decision boundary$ ?eig%t
vector '
determines t%e orientation of decision plane and bias parameter
w5 determines t%e distance from
origin$ In regression problems t%e use of t%is kind of net,ork
is limited only .d 4'/ dimensional%yperplanes can be modeled$ -n
e*ample of singlelayer net,orks is a linear associative
memory&,%ic% associates an output vector ,it% an input
vector$
8*tension of .($(/ to several classes is straig%tfor,ardGd
yk .// (wkix
i
i5
.($+/
,%ere again x5 'and
wk5 is t%e bias parameter$ )%e connection from input i to output
k is
,eig%ted by a ,eig%t parameter wki $
-
7/24/2019 (758981491) Soft Computing.
56/287
Fi#ure $.$ )%e simplest neural net,orks$ Computation is done in
t%e second layer of nodes
!unctions of t%e form .($(/ can be generali#ed by using a
.monotonic/ linear or nonlinear activationfunction ,%ic% acts on
t%e ,eig%ted sum as
y./
/
d
g.(w
ix
i
/i 5
.($/
,%ere g .v/ is usually c%osen to be a t%res%old function&
piece,ise linear& logistic sigmoid&sigmoidal or %yperbolic
tangent function .tan%/$ )%e first neuron model ,as of t%is type
and ,as
proposed as early as in ':+5=s by >cCulloc% and 6itts$
!)%res%old function .step function/G
g .v/ L
' & if v 5
5 & if v 5 .($9/
!6iece,ise linear function .pseudolinear/
' & if
g .v/ v & ifv 5$
v 545$ .($4/
5 & if v ?45$
!Logistic sigmoid
g.v/'
' 3 e*p. 4v/
.($;/
De*inition $.1 - functionsatisfied
g G8 /
8
is called sigmoidal function& if t%e follo,ing criterion
is
limg.v/ 5v/4>
-
7/24/2019 (758981491) Soft Computing.
57/287
limg .v/ 'v/>
If v is small t%en g appro*imates a linear function and .($/
contains appro*imately a linearnet,ork as a special case$ !unctions
of t%e form .($/ ,%ic% %ave logistic sigmoids as
activationfunctions are used in statistics and are referred to as
logistic discrimination functions$ Sigmoid is a
popular c%oice because it is monotone& bounded&
nonlinear and %as a simple derivativeg T .v/ g .v/.' 4g .v// $7ne
,ay to generali#e linear discriminants is to use fi*ed nonlinear
basis functions
transform t%e input vector /before it is multiplied by t%e
,eig%t vectorGb
i.// ,%ic%
k
y .// (w
ib
ii 5
.// .($:/
,%ere t%e bias is included in t%e sum by defining b5 0/1$ !or
suitable c%oice of basis functions
.($:/ can appro*imate any continuous function to arbitrary
accuracy .t%ese basis functions could be&for e*ample&
radial basis functions& fu##y basis functions& splines&
etc$/$
If .($:/ is used as input to t%e t%res%old activation function
g& ,e get a perceptron$ osenblattproved for perceptrons t%at if
t%e vectors used to train t%e perceptron are from linearly
separableclasses& t%en t%e perceptron learning algorit%m
converges Bosenblatt& ':92E$
Fi#ure $.) )%e perceptron net,ork
-
7/24/2019 (758981491) Soft Computing.
58/287
$.! Mu&ti&%yer Pere+tron
Fi#ure $.- )%e multilayer perceptron net,ork$
)%e summing Junction of t%e %idden unit is obtained by t%e
follo,ing ,eig%ted linear combinationGdv
(wixi
i5
.($'5/
,%ere wi is a ,eig%t in t%e first layer .from input unit i to
%idden unit / andw
5 is t%e bias for
%idden unit $ )%e activation .output/ of %idden unit is t%en
obtained by
ohiddeng.v
/
.($''/
!or output of ,%ole net,ork t%e follo,ing activation is
constructedk
y g .(w ohidden/
i5
.($'2/
),olayered multilayer perceptron in !ig$ ($ can be represented
as a function by combining t%eprevious e*pressions in t%e form
k d
y g.(wg.(wixi//
.($'(/
i5 i5
)%e activation function for t%e output unit can be linear$ In
t%at case .($'(/ becomes a special caseof .($:/& in ,%ic% t%e
basis functions are
d
b g.(wixi/
i5
.($'+/
-
7/24/2019 (758981491) Soft Computing.
59/287
If t%e activation functions in t%e %idden layer are linear&
t%en suc% a net,ork can be converted intoan eAuivalent net,ork
,it%out %idden units by forming a single linear transformation of
t,osuccessive linear transformations B1is%op& '::9E$ So t%e
net,orks %aving nonlinear %idden unitactivation functions are
preferred$ NoteG In literature t%e eAuation for output of neuron
can also be
seen ,ritten as follo,s dy
kg.(wk
x
i'
4k/ .($'/
,%ere kis t%e bias parameter$ >at%ematically t%is is
eAuivalent to t%e former eAuations ,%ere t%e
bias ,as included in t%e summation$ ?e can al,ays set wk5
kand
x5 4'$ )%e sign OO
can& of course& be included in ,eig%t parameter and not
in t%e input$
>L6s are mainly used for functional appro*imation rat%er t%an
classification problems$ )%ey aregenerally unsuitable for modeling
functions ,it% significant local variations$ )%e
universalappro*imation t%eorem states t%at >L6 can appro*imate
any continuous function arbitrarily ,ell&alt%oug% it does not
provide indication about t%e comple*ity of >L6$ )%e
@apnikC%ervonenkisdimension d@C gives a roug% appro*imation about
t%e comple*ity$ -ccording to t%is principle& t%eamount of
training data s%ould be appro*imately ten times t%e
>L6$d
=C& or t%e number of ,eig%ts in
>L6s are suitable for %ig%dimensional function
appro*imation& if t%e desired function can beappro*imated by a
lo, number of ridge functions .>L6s employ ridge functions in
t%e %iddenlayer/$ )%ey may perform ,ell& alt%oug% t%e training
data %ave redundant inputs$
$.$ Funtion%& Lin( Net'or(
!unctional link neural net,orks B6ao& '::+E %ave t%e same
structure as >L6s but polynomial ortrigonometric functions are
used as activation functions in t%e %idden layer$ Nodes in t%e
input andoutput layer are linear and only t%e connections from
%idden layer to output layer are ,eig%ted$)%us& functional link
net,ork can be regarded as a special case of .($:/$ - successful
use of t%e
functional link net,orks reAuires a set of basis functions ,%ic%
can represent t%e desired function,it% a sufficient accuracy over
t%e input space$ )%e success of t%e net,ork depends mainly on
t%ebasis functions used$ )%e set of basis functions s%ould not
overparametri#e t%e net,ork$
$.) R%di%& B%sis Funtion Net'or(
Inter+o&%tion
adial basis functions .1!/ %ave originally been used in e*act
interpolation B6o,ell& ':;4E$ In
interpolation ,e %ave n data points x #8d
& and n real valuednumbers
ti #8 & ,%ere
i '&$$$& n $ )%e task is to determine a function s in
linear space suc% t%ats.xi/
ti
interpolation function is a linear combination of basis
functions
i '&$$$& n $ )%e
i
-
7/24/2019 (758981491) Soft Computing.
60/287
n
s .// (wibi
i'
./ / .($'9/
-s basis functions bi & radial basis functions of t%e form
.($'4/ are used$
bi.// 5 . / /
i/ .($'4/
,%ere 5 is mapping R 3 /R and t%e norm is a 8uclidean distance$
)%e norm could also bea>a%alanobis distance$ )%e follo,ing forms
%ave been considered as radial basis functions B6o,ell&
':;4EG'$ >ultiAuadric functionG 5.r /
.r 23c/
'2 & ,%ere c is positive constant .for e*ample& a
distancebet,een neig%boring knots/ and r#R$2$ 5.r /
r & ,%ere r 5 $
($ 5.r/ r2
+$ 5.r/ r(
$ 5.r/ e*p.4r 2 /
It %as been reported t%at t%e global basis functions may %ave
slig%tly better interpolation propertiest%an t%e local ones .
-
7/24/2019 (758981491) Soft Computing.
61/287
Re&%tion to s+&ines
-lso so called t%in plate splines %ave been used as radial basis
functions$ In contrast to
-
7/24/2019 (758981491) Soft Computing.
62/287
.// .($2(/
,%ere bi are radial basis functions$ 7t%er common c%oices for
radial basis functions are t%e Cauc%y
function& t%e inverse multiAuadric and t%e nonlocal
multiAuadric$ ?%en t%e basis function is
L6 net,ork& ,%ic% is partially connected and %as t%ree
layers of ,eig%ts$ Input layeris fully connected to t%e first
%idden layer& ,%ere t%e %idden nodes are arranged to t%e groups
of 2dnodes& ,%ere d is t%e dimension of t%e input space$
Sigmoidal functions are used as activationfunctions in bot% %idden
layers$ 8ac% of t%e second %idden layer neuron receives inputs only
fromt%e particular preceding layer node group$ )%us& t%e first
and t%e second %idden layers are partiallyconnected to eac% ot%er$
1is%op illustrates t%e appro*imation capability of t%is net,ork by
a t,odimensional e*ample B1is%op& '::9EG
7utput of a neuron& for e*ample of a neuron ' in !ig$
($9& is a function of t,o inputs and t%e s%apeof t%is function
is illustrated in !ig$ ($4 .a/$ -dding t,o suc% outputs& for
e*ample outputs ofneurons ' and 2& can produce a ridges%aped
function .!ig$ ($4 .b//$ -dding t,o ridges can give a
-
7/24/2019 (758981491) Soft Computing.
63/287
function of t%e form as in !ig$ ($4 .c/$ )%is is a result of
summation in node :$ )ransforming t%isbump function ,it% sigmoid
gives a locali#ed basis function and t%e final output y is a
linearcombination of t%ese basis functions .outputs of neurons :
and '5/$ Das%ed lines remind& t%at t%enumber of groups in t%e
first %idden layer and number of neurons in t%e second %idden layer
could
be different from 2$
Fi#ure $. )%ree net,ork layers
Fi#ure $.5 )%ree layer net,ork constructs locali#ed basis
functions& ,%ic% resemble
-
7/24/2019 (758981491) Soft Computing.
64/287
$.- Neuro*u""y Net'or(s
Motiv%tion2 Neural net,orks and fu##y systems try to emulate t%e
operation of %uman brain$Neural net,orks concentrate on t%e
structure of %uman brain& i$e$& on t%e %ard,are ,%ereas
fu##y
logic systems concentrate on soft,are$ )%is can be vie,ed
%ierarc%ically as if t%e >L6 ,ere att%e lo,est level trying to
simulate t%e basic functions and fu##y logic at a %ig%er level
trying tosimulate fu##y and symbolic reasoning. B?ang&
'::+E
Combining neural net,orks and fu##y systems in one unified
frame,ork %as become popular in t%elast fe, years$ -lt%oug% Lee Lee
,orked on neurofu##y as early as in ':4 BLee Lee& ':4E&more
effort ,as not put on t%e researc% until in early '::5=s$ 1erenJi
K%edkar& ang $S$$&)akagi Hayas%i& Nauck Kruse& ?ang
L$$& Kosko& Harris 1ro,n are researc%ers ,%o %ave%ad a
strong influence on t%is field$ )%ey all %ave publis%ed t%eir most
significant researc% results.concerning neurofu##y systems/ in t%e
'::5=s$
>ost of t%e approac%es are difficult to compare because t%eir
arc%itectures& learning met%ods& basisor activation
functions differ strongly$ Harris and 1ro,n in B1ro,n Harris&
'::+E separate t%eneurofu##y researc%ers into t,o sc%oolsG t%ose
,%o try to model %uman brain=s reasoning andlearning& and t%ose
,%o develop structures and algorit%ms for system modeling&
control& t%at is&speciali#ed applications$ )%e follo,ing
treatment of neurofu##y systems adopts more t%e latter
pproac% t%an t%e former algorit%ms for system modeling&
control& t%at is& speciali#ed applications$)%e follo,ing
treatment of neurofu##y systems adopts more t%e latter approac%
t%an t%e former$
$.-.1 Di**erent Neuro*u""y 9++ro%,es
Designation neurofu##y %as several different meanings$ Sometimes
neurofu##y is associated to%ybrid systems ,%ic% act on t,o distinct
subproblems$ In t%at case& neural net,ork is utili#ed in
t%efirst subproblem .e$g$& in signal processing/ and fu##y
logic is utili#ed in t%e second subproblem.e$g$& in reasoning
task/$ Normally& ,%en talking about t%e neurofu##y systems&
t%e link bet,eent%ese t,o soft computing met%ods is understood to
be stronger$ In t%e follo,ing lig%t is tried to s%edon t%e most
common different interpretations$
Fu""y &o#i in &e%rnin# %orit,ms
- common approac% i
