4 NEURONSKE MREŽE I NJIHOVA PRIMENA

4.1 OSNOVE NEURONSKIH MREŽA

4.1.1 KONCEPT I STRUKTURA NEURONSKIH MREŽA

Struktura veštačkog neurona prikazana je na slici, gde su:

1 2, , ,

nx x x - ulazni signali neurona ( )f - nelinearna prenosna funkcija neurona

(aktivaciona funkcija)

1 2, , ,

nw w w - težinski koeficijenti

neurona

72

b - prag neurona

1

( ) ( )n

i i

i

y f u f w x b

- izlaz neurona

73

Često korišćene aktivacione funkcije:

sigmoidna funkcija

2

2( ) 1

1 xf x

e

logaritamska sigmoidna funkcija

1( )

1 xf x

e

Aktivacione funkcije u Matlabu su:

tansig(x),

logsig(),

satlin(),

hardlims(),

radbas(), …

74

Više veštačkih neurona se može povezati u jednu mrežu koja se naziva veštačka neuronska mreža (kraće neuronska mreža).

U zavisnosti od načina povezivanja neurona u mrežu, postoje različiti tipovi neuronskih mreža.

Mi ćemo se baviti isključivo sa neuronskim mrežama kod kojih se signali prostiru isključivo od ulaza ka izlazu, bez povratnog prenosa (feedforward neuronske (FFN) mreže).

FFN mreže se mogu koristiti za bilo koje preslikavanje ulaza u izlaze.

FFN mreža sa jednim skrivenim slojem u kome se nalazi dovoljan broj neurona pogodna je za fitovanje podataka.

Svaka neuronska mreža se mora obučiti, tj. moraju se podesiti težinski

koeficijenti i prag svakog neurona (1 2, , ,

nw w w i b) tako da mreža

obavlja neku unapred zadatu funkciju.

Kod FFN mreža greška obučavanja se prenosi od izlaza ka ulazu mreže, pa se zbog toga ove mreže nazivaju i back-propagation (BP) neuronske mreže.

75

Tipična struktura jedne BP neuronske mreže je prikazana na slici.

U mreži postoje tri sloja:

1. jedan ulazni sloj

2. jedan ili više skrivenih slojeva sa više neurona po sloju

3. jedan izlazni sloj sa jednim ili više neurona

Za izlazni sloj se obično usvaja poslednji skriveni sloj.

76

Neophodni koraci u kreiranju, obučavanju, validaciji i korišćenju neuronskih mreža:

1. Prikupljanje podataka

2. Kreiranje neuronske mreže

3. Konfigurisanje neuronske mreže

4. Inicijalizacija težina i pomeraja (pragova) mreže

5. Treniranje mreže

6. Validacija mreže

7. Korišćenje mreže

77

4.1.2 KREIRANJE NEURONSKE MREŽE

U Matlabu (Neural Networks Toolbox) se lako konstruiše FFN mreža koristeći funkciju feedforwardnet()

net = feedforwardnet(hiddenSizes, trainFcn)

hiddenSizes Vektor vrsta čiji su elementi veličine jednog ili više skrivenih

slojeva mreže (default = 10) trainFcn Trening funkcija (default = 'trainlm'): net Kreirana mreža

78

Training Function Algorithm

'trainlm' Levenberg-Marquardt

'trainbr' Bayesian Regularization

'trainbfg' BFGS Quasi-Newton

'trainrp' Resilient Backpropagation

'trainscg' Scaled Conjugate Gradient

'traincgb' Conjugate Gradient with Powell/Beale Restarts

'traincgf' Fletcher-Powell Conjugate Gradient

'traincgp' Polak-Ribiére Conjugate Gradient

'trainoss' One Step Secant

'traingdx' Variable Learning Rate Gradient Descent

'traingdm' Gradient Descent with Momentum

'traingd' Gradient Descent

79

Primer. Kreiranje neuronske mreže % Kreira mrežu sa jednim ulaznim slojem, % jednim skrivenim slojem od 10 neurona i % jednim izlaznim slojem od 1 neurona. net = feedforwardnet(10); net – predstavlja objekat FFN mreže

80

4.1.3 OBUČAVANJE NEURANSKE MREŽE

81

U Matlabu, neuronska mreža se trenira koristeći funkciju train()

net Newly trained network

tr Training record (epoch and perf)

net Network

X Network inputs

T Network targets (default = zeros)

Xi Initial input delay conditions (default = zeros)

Ai Initial layer delay conditions (default = zeros)

EW Error weights

82

Primer 1. Koristeći FFN mrežu fitovati podake generisane pomoću:

0.123 0.170.12 0.54 sin(1.23 ), 0,10x xt e e x x

% Ucitavanje podataka u radni prostor

x=0:.5:10;

t=0.12*exp(-0.213*x)+0.54*exp(-

0.17*x).*sin(1.23*x);

plot(x,t)

% Kreiranje mreze

net = feedforwardnet(10);

% Izbor aktivacionih funkcija u slojevima

net.layers{1}.transferFcn = 'tansig';

net.layers{2}.transferFcn = 'tansig';

% Obucavanje mreze

net = train(net,x,t);

% Prikaz seme mreze

83

view(net)

% Odredjuje izlaz mreze za novi ulaz

x0=[0:0.1:10];

y = sim(net,x0);

% Poredjenje rezultata

figure, plot(x,t,'o',x0,y)

legend('Podaci','Fitovanje')

% Parametri mreze

net.IW{1,1}

net.LW{2,1}'

net.b{1}

net.b{2}

84

SVOJSTVA NEURONSKIH MREŽA

ARCHITECTURE PROPERTIES

net.numInputs 0 or a positive integer Number of inputs.

net.numLayers 0 or a positive integer Number of layers.

net.numOutputs 0 or a positive integer (read only)

Number of network outputs according tonet.outputConnect.

net.numInputDelays 0 or a positive integer (read only)

Maximum input delay according to allnet.inputWeights{i,j}.delays.

STRUCTURE PROPERTIES

net.inputs numInputs-by-1 cell array net.inputs{i} is a structure defining input i.

net.layers numLayers-by-1 cell array net.layers{i} is a structure defining layer i.

net.layers{1}.transferFcn = 'tansig', 'logsig', ‘purelin’, ‘satlin’, ‘hardlims’, ‘ radbas’

net.biases numLayers-by-1 cell array If net.biasConnect(i) is 1, then net.biases{i} is a structure defining the bias for layer i.

net.inputWeights numLayers-by-numInputs cell array

If net.inputConnect(i,j) is 1, thennet.inputWeights{i,j} is a structure defining the weight to layer i from input j.

net.layerWeights numLayers-by-numLayers cell array

If net.layerConnect(i,j) is 1, thennet.layerWeights{i,j} is a structure defining the weight to layer i from layer j.

85

net.outputs 1-by-numLayers cell array If net.outputConnect(i) is 1, thennet.outputs{i} is a structure defining the network output from layer i.

FUNCTION PROPERTIES

net.adaptFcn Name of a network adaption function or ' '

net.initFcn Name of a network initialization function or ' '

net.performFcn Name of a network performance function or ' '

net.trainFcn Name of a network training function or ' '

PARAMETER PROPERTIES

net.adaptParam Network adaption parameters

net.initParam Network initialization parameters

net.performParam Network performance parameters

net.trainParam Network training parameters

WEIGHT AND BIAS VALUE PROPERTIES

net.IW numLayers-by-numInputs cell array of input weight values

net.LW numLayers-by-numLayers cell array of layer weight values

net.b numLayers-by-1 cell array of bias values

86

net.layers{i}.transferFcn = 'purelin';

net.layers{i}.transferFcn = 'logsig';

net.layers{i}.transferFcn = 'tansig';

87

4.2 FITIVANJE PODATAKA POMOĆU NEURONSKIH MREŽA

Neka smo u cilju analize zavisnosti ( )f x neke fizičke veličine y od druge fizičke

veličine, x izvršili niz merenja i dobili tabelu sa parovima izmerenih vrednosti posmatranih veličina:

Postupak formulisanja funkcije ˆ ( )f x , koja aproksimira nepoznatu, u opštem

slučaju nelinearnu zavisnost ( )f x , tako da odstupanja eksperimentalnih

vrednosti od računskih procena dobijenih iz nje:

ˆ( ), 1,2,3,...i i

e y f x i

budu u određenom smislu mala, naziva se fitovanje eksperimetalnih

podataka. Dakle odabranu funkciju ˆ ( )f x , koju nazivamo empirijska formula

prilagođavamo (fitujemo, od engleske reči fit) eksperimentalnim podacima.

Empirijsku formulu možemo implementirati i pomoću neuronske mreže.

88

U Neural networks Toolbox postoji specijalizovana neuronska mreža namenjena za fitovanje podataka. Ona se kreira pomoću funkcije fitnet().

Ulazno izlazni parametri su isti kao i kod funkcije feedforwardnet(). Postoje dva načina za fitovanje podataka korišćenjem funkcije fitnet().

1. Korišćenjem grafičkog interfejsa pozivom funkcije nnstart i biranjem Fitting app.

2. Korišćenjem programskog koda sa odgovarajućim funkcijama.

89

4.2.1 GRAFIČKI INTERFEJS ZA FITOVANJE PODATAKA KORIŠĆENJEM

NEURONSKIH MREŽA

1. Otvaramo grafički prozor nnstart i biramo opciju Fittinf app

90

91

92

1

2

93

94

95

1

2

96

1

2

97

1 2

3

98

1

99

Opsezi vrednosti grešaka. Širina opsega je 0.00003

2

Bro

j uzo

raka

za

tre

nir

anje

, val

idac

iju i

test

iran

je k

oji

leže

u p

oje

din

im o

pse

zim

a

100

3

101

1

2 3 4

5

102

1

2

1

1 ˆn

i i

i

MSE T Tn

103

2

104

105

1. SNIMANJE PROGRAMSKOG KODA U .M FAJL 2. SNIMANJE PROMENLJIVIH I OJEKATA U RADNI PROSTOR

106

4.2.2 FITOVANJE PODATAKA KORIŠĆENJEM PROGRAMSKOG KODA

Najjednostavniji način da se napiše programski kod za fitovanje podataka pomoću neuronske mreže jeste da se u grafičkom interfejsu, koji je prethodno objašnjen, izabere Simple Script u prozoru Save Results.

Nakon generisanja programskog koda, on se može snimiti kao m-skripta i koristiti za dalje izvršavanje.

Drugi način za generisanje programskog koda jeste direkto kucanje programskog koda u editor i snimanje u fajl sa ekstenzijom .m.

107

% Solve an Input-Output Fitting problem with a Neural Network

% Script generated by Neural Fitting app

% Created 27-Dec-2018 16:13:57

%

% This script assumes these variables are defined:

%

% simplefitInputs - input data.

% simplefitTargets - target data.

x = simplefitInputs;

t = simplefitTargets;

% Choose a Training Function

% For a list of all training functions type: help nntrain

% 'trainlm' is usually fastest.

% 'trainbr' takes longer but may be better for challenging

problems.

% 'trainscg' uses less memory. Suitable in low memory situations.

trainFcn = 'trainlm'; % Levenberg-Marquardt backpropagation.

% Create a Fitting Network

hiddenLayerSize = 10;

net = fitnet(hiddenLayerSize,trainFcn);

% Setup Division of Data for Training, Validation, Testing

108

net.divideParam.trainRatio = 70/100;

net.divideParam.valRatio = 15/100;

net.divideParam.testRatio = 15/100;

% Train the Network

[net,tr] = train(net,x,t);

% Test the Network

y = net(x);

e = gsubtract(t,y);

performance = perform(net,t,y)

% View the Network

view(net)

% Plots

% Uncomment these lines to enable various plots.

%figure, plotperform(tr)

%figure, plottrainstate(tr)

%figure, ploterrhist(e)

%figure, plotregression(t,y)

%figure, plotfit(net,x,t)

109

4.2.3 ODREĐIVANJE PARAMETARA NEURONSKE MREŽE

Izborom opcije Save results u dijalogu Save resulte unutar grafičkog interfejsa, u radnom prostoru snimaju se izabrane promenljive i objekti.

net je objekat koji predstavlja

sintetizovanu i obučenu neuronsku mrežu.

Parametri mreže: IW, LW, b

net.IW{1,1}

net.LW{2,1}'

net.b{1}

net.b{2}

110

111

4.3 MODELOVANJE SISTEMA POMOĆU NEURONSKIH MREŽA

Razlikujemo tri tipa modela sistema pomoću vremenskih serija:

1. NARX model: ( ) ( 1),..., ( ), ( 1),..., ( )y t f x t x t d y t y t d

2. NAR model: ( ) ( 1),..., ( )y t f y t y t d

3. NIO model: ( ) ( 1),..., ( )y t f x t x t d

gde su:

( )f nelinearna funkcija koja predstavlja model sistema, a

implementirana je pomoću neuronske mreže,

( )y t vrednost izlaza sistema u tekućem trenutku,

( 1),..., ( )x t x t d niz od d podataka dobijenih merenjem ulaznog

signala sistema u prethodnim trenucima vremena

( 1),..., ( )y t y t d niz od d podataka dobijenih merenjem izlaznog

signala sistema u prethodnim trenucima vremena

112

113

U Neural networks Toolbox postoji specijalizovana neuronska mreža namenjena za modelovanje sistema. Ona se kreira pomoću funkcije narxnet().

Postoje dva načina za fitovanje podataka korišćenjem funkcije fitnet():

1. Korišćenjem grafičkog interfejsa pozivom funkcije nnstart i biranjem Time series app.

2. Pisanjem programskog koda i snimanjem u .m fajl.

114

4.3.1 GRAFIČKI INTERFEJS ZA MODELOVANJE SISTEMA POMOĆU

NEURONSKIH MREŽA

Posmatramo NARX model:

( ) ( 1),..., ( ), ( 1),..., ( )y t f x t x t d y t y t d

Pokrećemo funkciju nnstart za pokretanje grafičkog interfejsa

115

116

117

118

119

120

121

122

123

124

Perfomance

125

Training State

126

Error Histogram

127

Regression

128

Time Series Response

129

Error Autocorrelation

130

Input-Error Cross-correlation

131

132

Promena mreže

133

Ponovno treniranje mreže

134

Nakon treniranja mreže

135

136

137

138

4.3.2 MODELOVANJE SISTEMA KORIŠĆENJEM PROGRAMSKOG KODA

Najedenostavniji način da se napiše programski kod za modelovanje sistema pomoću neuronske mreže jeste da se u grafičkom interfejsu, koji je prethodno objašnjen, izabere Simple Script u prozoru Save Results. nakon generisanja programskog koda, on se može snimiti kao m-skripta i koristiti za dalje izvršavanje. Drugi način za generisanje programskog koda jeste direkto kucanje programskog koda u editor i snimanje u fajl sa ekstenzijom .m.

139

Programski kod dobijen izaborom Simple Script u prozoru Save Results: % Solve an Autoregression Problem with External Input with a NARX Neural

Network

% Script generated by Neural Time Series app

% Created 27-Dec-2018 13:09:24

%

% This script assumes these variables are defined:

%

% exchangerInputs - input time series.

% exchangerTargets - feedback time series.

X = exchangerInputs;

T = exchangerTargets;

% Choose a Training Function

% For a list of all training functions type: help nntrain

% 'trainlm' is usually fastest.

% 'trainbr' takes longer but may be better for challenging problems.

% 'trainscg' uses less memory. Suitable in low memory situations.

trainFcn = 'trainlm'; % Levenberg-Marquardt backpropagation.

% Create a Nonlinear Autoregressive Network with External Input

inputDelays = 1:5;

feedbackDelays = 1:5;

hiddenLayerSize = 10;

net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize,'open',trainFcn);

% Prepare the Data for Training and Simulation

140

% The function PREPARETS prepares timeseries data for a particular network,

% shifting time by the minimum amount to fill input states and layer

% states. Using PREPARETS allows you to keep your original time series data

% unchanged, while easily customizing it for networks with differing

% numbers of delays, with open loop or closed loop feedback modes.

[x,xi,ai,t] = preparets(net,X,{},T);

% Setup Division of Data for Training, Validation, Testing

net.divideParam.trainRatio = 70/100;

net.divideParam.valRatio = 15/100;

net.divideParam.testRatio = 15/100;

% Train the Network

[net,tr] = train(net,x,t,xi,ai);

% Test the Network

y = net(x,xi,ai);

e = gsubtract(t,y);

performance = perform(net,t,y)

% View the Network

view(net)

% Plots

% Uncomment these lines to enable various plots.

%figure, plotperform(tr)

%figure, plottrainstate(tr)

%figure, ploterrhist(e)

%figure, plotregression(t,y)

%figure, plotresponse(t,y)

141

%figure, ploterrcorr(e)

%figure, plotinerrcorr(x,e)

% Closed Loop Network

% Use this network to do multi-step prediction.

% The function CLOSELOOP replaces the feedback input with a direct

% connection from the outout layer.

netc = closeloop(net);

netc.name = [net.name ' - Closed Loop'];

view(netc)

[xc,xic,aic,tc] = preparets(netc,X,{},T);

yc = netc(xc,xic,aic);

closedLoopPerformance = perform(net,tc,yc)

% Step-Ahead Prediction Network

% For some applications it helps to get the prediction a timestep early.

% The original network returns predicted y(t+1) at the same time it is

% given y(t+1). For some applications such as decision making, it would

% help to have predicted y(t+1) once y(t) is available, but before the

% actual y(t+1) occurs. The network can be made to return its output a

% timestep early by removing one delay so that its minimal tap delay is now

% 0 instead of 1. The new network returns the same outputs as the original

% network, but outputs are shifted left one timestep.

nets = removedelay(net);

nets.name = [net.name ' - Predict One Step Ahead'];

view(nets)

[xs,xis,ais,ts] = preparets(nets,X,{},T);

ys = nets(xs,xis,ais);

stepAheadPerformance = perform(nets,ts,ys)

142

4.3.3 ODREĐIVANJE PARAMETARA NEURONSKE MREŽE

Izborom opcije Save results u dijalogu Save resulte unutar grafičkog interfejsa, u radnom prostoru snimaju se izavrane promenljive i objekti.

net je objekat koji predstavlja sintetizovanu i

obučenu neuronsku mrežu.

Parametri mreže: IW, LW, b

net.IW{1,1}

net.IW{1,2}

net.LW{2,1}'

net.b{1}

net.b{2}

143

144

4.4 SKUPOVI PODATAKA KOJI DOLAZE SA NEURAL NETWORK TOOLBOX

The Neural Network Toolbox™ software contains a number of sample data sets that you can use to experiment with the functionality of the toolbox. To view the data sets that are available, use the following command:

help nndatasets

Neural Network Datasets ----------------------- Function Fitting, Function approximation and Curve fitting.

Function fitting is the process of training a neural network on a set of inputs in order to produce an associated set of target outputs. Once the neural network has fit the data, it forms a generalization of the input-output relationship and can be used to generate outputs for inputs it was not trained on. simplefit_dataset - Simple fitting dataset. abalone_dataset - Abalone shell rings dataset. bodyfat_dataset - Body fat percentage dataset. building_dataset - Building energy dataset. chemical_dataset - Chemical sensor dataset. cho_dataset - Cholesterol dataset. engine_dataset - Engine behavior dataset. vinyl_dataset - Vinyl bromide dataset. ----------

145

Pattern Recognition and Classification Pattern recognition is the process of training a neural network to assign the correct target classes to a set of input patterns. Once trained the network can be used to classify patterns it has not seen before. simpleclass_dataset - Simple pattern recognition dataset. cancer_dataset - Breast cancer dataset. crab_dataset - Crab gender dataset. glass_dataset - Glass chemical dataset. iris_dataset - Iris flower dataset. ovarian_dataset - Ovarian cancer dataset. thyroid_dataset - Thyroid function dataset. wine_dataset - Italian wines dataset. digitTrain4DArrayData - Synthetic handwritten digit dataset for training in form of 4-D array. digitTrainCellArrayData - Synthetic handwritten digit dataset for training in form of cell array. digitTest4DArrayData - Synthetic handwritten digit dataset for testing in form of 4-D array. digitTestCellArrayData - Synthetic handwritten digit dataset for testing in form of cell array. digitSmallCellArrayData - Subset of the synthetic handwritten digit dataset for training in form of cell array. ----------

146

Clustering, Feature extraction and Data dimension reduction Clustering is the process of training a neural network on patterns so that the network comes up with its own classifications according to pattern similarity and relative topology. This is useful for gaining insight into data, or simplifying it before further processing. simplecluster_dataset - Simple clustering dataset. The inputs of fitting or pattern recognition datasets may also clustered. ---------- Input-Output Time-Series Prediction, Forecasting, Dynamic modeling Nonlinear autoregression, System identification and Filtering Input-output time series problems consist of predicting the next value of one time series given another time series. Past values of both series (for best accuracy), or only one of the series (for a simpler system) may be used to predict the target series. simpleseries_dataset - Simple time series prediction dataset. simplenarx_dataset - Simple time series prediction dataset. exchanger_dataset - Heat exchanger dataset. maglev_dataset - Magnetic levitation dataset. ph_dataset - Solution PH dataset. pollution_dataset - Pollution mortality dataset.

147

refmodel_dataset - Reference model dataset robotarm_dataset - Robot arm dataset valve_dataset - Valve fluid flow dataset. ---------- Single Time-Series Prediction, Forecasting, Dynamic modeling, Nonlinear autoregression, System identification, and Filtering Single time series prediction involves predicting the next value of a time series given its past values. simplenar_dataset - Simple single series prediction dataset. chickenpox_dataset - Monthly chickenpox instances dataset. ice_dataset - Global ice volume dataset. laser_dataset - Chaotic far-infrared laser dataset. oil_dataset - Monthly oil price dataset. river_dataset - River flow dataset. solar_dataset - Sunspot activity dataset

Notice that all of the data sets have file names of the form name_dataset. Inside these files

will be the arrays nameInputs and nameTargets. You can load a data set into the workspace

with a command such as

148

load simplefit_dataset

This will load simplefitInputs and simplefitTargets into the workspace. If you want to

load the input and target arrays into different names, you can use a command such as

[x,t] = simplefit_dataset;

This will load the inputs and targets into the arrays x and t. You can get a description of a data

set with a command such as

help maglev_dataset

149