33
ANFIS: Adaptive Neuro-Fuzzy Inference Systems Adriano Cruz Mestrado NCE, IM, UFRJ Logica Nebulosa – p. 1/3

ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Embed Size (px)

Citation preview

Page 1: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

ANFIS: Adaptive Neuro-Fuzzy InferenceSystems

Adriano Cruz

Mestrado NCE, IM, UFRJ

Logica Nebulosa – p. 1/33

Page 2: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Summary

• Introduction• ANFIS Architecture• Hybrid Learning Algorithm• ANFIS as a Universal Approximatior• Simulation Examples

Logica Nebulosa – p. 2/33

Page 3: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Introduction

• ANFIS: Artificial Neuro-Fuzzy Inference Systems• ANFIS are a class of adaptive networks that are

funcionally equivalent to fuzzy inference systems.• ANFIS represent Sugeno e Tsukamoto fuzzy

models.• ANFIS uses a hybrid learning algorithm

Logica Nebulosa – p. 3/33

Page 4: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Sugeno Model

• Assume that the fuzzy inference system has twoinputs x and y and one output z.

• A first-order Sugeno fuzzy model has rules as thefollowing:

• Rule1:If x is A1 and y is B1, then f1 = p1x + q1y + r1

• Rule2:If x is A2 and y is B2, then f2 = p2x + q2y + r2

Logica Nebulosa – p. 4/33

Page 5: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Sugeno Model - I

xX

A1

A2

y

X

B1

B2

W1

W2

f1=p1x+q1y+r1 f2=p2x+q2y+r2 f=w1.f1+w2.f2

w1+w2

Y

Y

Logica Nebulosa – p. 5/33

Page 6: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

ANFIS Architecture

A1

A2

x

y

B1

B2

Prod

Prod

Norm

Norm

Layer1 Layer2 Layer3

W1

W2

x y

x y

Sum

W1f1

W1f2

f

Layer4 Layer5

Logica Nebulosa – p. 6/33

Page 7: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Layer 1 - I

• Ol,i is the output of the ith node of the layer l.

• Every node i in this layer is an adaptive node witha node functionO1,i = µAi

(x) for i = 1, 2, orO1,i = µBi−2

(x) for i = 3, 4

• x (or y) is the input node i and Ai (or Bi−2) is alinguistic label associated with this node

• Therefore O1,i is the membership grade of a fuzzyset (A1, A2, B1, B2).

Logica Nebulosa – p. 7/33

Page 8: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Layer 1 - II

• Typical membership function:

µA(x) =1

1 + |x−ci

ai

|2bi

• ai, bi, ci is the parameter set.• Parameters are referred to as premise parameters .

Logica Nebulosa – p. 8/33

Page 9: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Layer 2

• Every node in this layer is a fixed node labeledProd.

• The output is the product of all the incomingsignals.

• O2,i = wi = µAi(x) · µBi

(y), i = 1, 2

• Each node represents the fire strength of the rule• Any other T-norm operator that perform the AND

operator can be used

Logica Nebulosa – p. 9/33

Page 10: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Layer 3

• Every node in this layer is a fixed node labeledNorm.

• The ith node calculates the ratio of the ith rulet’sfiring strenght to the sum of all rulet’s firingstrengths.

• O3,i = wi = wi

w1+w2

, i = 1, 2

• Outputs are called normalized firing strengths .

Logica Nebulosa – p. 10/33

Page 11: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Layer 4

• Every node i in this layer is an adaptive node witha node function:

O4,1 = wifi = wi(px + qiy + ri)

• wi is the normalized firing strenght from layer 3.• {pi, qi, ri} is the parameter set of this node.• These are referred to as consequent parameters .

Logica Nebulosa – p. 11/33

Page 12: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Layer 5

• The single node in this layer is a fixed nodelabeled sum, which computes the overall output asthe summation of all incoming signals:

• overall output = O5,1 =∑

i wifi =∑

iwifi∑iwi

Logica Nebulosa – p. 12/33

Page 13: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Alternative Structures

• There are other structures

A1

A2

x

y

B1

B2

Prod

Prod

Layer1 Layer2 Layer3

W1

W2

x y

x y

Sum

W1f1

W1f2

f

Layer4 Layer5

W1f1+W2f2

/

Sum

Logica Nebulosa – p. 13/33

Page 14: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Learning Algorithm

Logica Nebulosa – p. 14/33

Page 15: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Hybrid Learning Algorithm - I

• The ANFIS can be trained by a hybrid learningalgorithm presented by Jang in the chapter 8 ofthe book.

• In the forward pass the algorithm usesleast-squares method to identify the consequentparameters on the layer 4.

• In the backward pass the errors are propagatedbackward and the premise parameters areupdated by gradient descent.

Logica Nebulosa – p. 15/33

Page 16: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Hybrid Learning Algorithm - II

Forward Pass Backward Pass

Premise Parameters Fixed Gradient Descent

Consequent Parameters Least-squares estimator Fixed

Signals Node outputs Error signals

Two passes in the hybrid learning algorithm for ANFIS.

Logica Nebulosa – p. 16/33

Page 17: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Universal Aproximator

Logica Nebulosa – p. 17/33

Page 18: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

ANFIS is a Universal Aproximator

• When the number of rules is not restricted, azero-order Sugeno model has unlimitedapproximation power for matching any nonlinearfunction arbitrarily well on a compact set.

• This can be proved using the Stone-Weierstrasstheorem.

• Let D be a compact space of N dimensions, andlet F be a set of continuous real-valued functionson D satisfying the following criteria:

Logica Nebulosa – p. 18/33

Page 19: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Stone-Weierstrauss theorem - I

• Let D be a compact space of N dimensions, andlet F be a set of continuous real-valued functionson D satisfying the following criteria:

Indentity function: The constant f(x) = 1 is in F .

Separability: For any two points x1 6= x2 in D, there is anf in F such that f(x1) 6= f(x2).

Algebraic closure: If f and g are any two functions in F ,then fg and af + bg are in F for any two realnumbers a and b.

Logica Nebulosa – p. 19/33

Page 20: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Stone-Weierstrauss theorem - II

• Then F is dense on C(D), the set of continuousreal-valued functions on D.

• For any ε > 0 and any function g in C(D), there is afunction f in F such that |g(x) − f(x)| < ε for allx ∈ D.

• The ANFIS satisfies all these requirements.

Logica Nebulosa – p. 20/33

Page 21: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Anfis and Matlab

Logica Nebulosa – p. 21/33

Page 22: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Matlab

• It is possible to use a graphics user interface• Command anfisedit.

• It is possible to use the command line interface orm-file programs.

• There are functions to generate, train, test and usethese systems.

Logica Nebulosa – p. 22/33

Page 23: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

ANFIS gui

Logica Nebulosa – p. 23/33

Page 24: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Applying

• Initializing• Training• Testing• Using

Logica Nebulosa – p. 24/33

Page 25: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Initializing - GENFIS1 - 1

• FIS = GENFIS1(DATA) generates asingle-output Sugeno-type fuzzy inference system(FIS) using a grid partition on the data (noclustering).

• FIS is used to provide initial conditions forposterior ANFIS training.

• DATA is a matrix with N+1 columns where the firstN columns contain data for each FIS input, andthe last column contains the output data.

Logica Nebulosa – p. 25/33

Page 26: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Initializing - GENFIS1 - 2

• By default GENFIS1 uses two ’gbellmf’ typemembership functions for each input.

• Each rule generated has one output membershipfunction, which is of type ’linear’ by default.

• It is possible to define these parameters usingFIS = GENFIS1(DATA, NUMMFS, INPUTMF,OUTPUTMF)

• fis = genfis1(data, [3 7],char(’pimf’, ’trimf’));

Logica Nebulosa – p. 26/33

Page 27: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Initializing - GENFIS1 - 3

data = [rand(10,1) 10*rand(10,1)-5 rand(10,1)];

fis = genfis1(data, [3 7], char(’pimf’,’trimf’));

[x,mf] = plotmf(fis,’input’,1);

subplot(2,1,1), plot(x,mf);

xlabel(’input 1 (pimf)’);

[x,mf] = plotmf(fis,’input’,2);

subplot(2,1,2), plot(x,mf);

xlabel(’input 2 (trimf)’);

Logica Nebulosa – p. 27/33

Page 28: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Initializing - GENFIS1 - 4

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

input 1 (pimf)

−2 −1 0 1 2 3 40

0.2

0.4

0.6

0.8

1

input 2 (trimf)

Logica Nebulosa – p. 28/33

Page 29: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Initializing - GENFIS2

• GENFIS2 generates a Sugeno-type FIS usingsubtractive clustering.

• GENFIS2 extracts a set of rules that models thedata behavior.

• The rule extraction method first determines thenumber of rules and antecedent membershipfunctions and then uses linear least squaresestimation to determine each rule’s consequentequations.

Logica Nebulosa – p. 29/33

Page 30: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Training

• ANFIS uses a hybrid learning algorithm to identifythe membership function parameters ofsingle-output, Sugeno type fuzzy inferencesystems (FIS).

• There are many ways of using this function.• Some examples:

• [FIS,ERROR] = ANFIS(TRNDATA)• [FIS,ERROR] =ANFIS(TRNDATA,INITFIS)

Logica Nebulosa – p. 30/33

Page 31: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Using

• EVALFIS evaluates a fuzzy inference system.• Y = EVALFIS(U,FIS) simulates the Fuzzy

Inference System FIS for the input data U andreturns the output data Y.

Logica Nebulosa – p. 31/33

Page 32: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

Example

• run exemplo06_03.m

Logica Nebulosa – p. 32/33

Page 33: ANFIS: Adaptive Neuro-Fuzzy Inference Systems · Introduction • ANFIS: Artificial Neuro-Fuzzy Inference Systems • ANFIS are a class of adaptive networks that are funcionally

The End

Logica Nebulosa – p. 33/33