Upload
agatha-burns
View
220
Download
0
Embed Size (px)
Citation preview
Radial Basis Networks:An Implementation
of Adaptive Centers
Nivas Durairaj
ECE539 Final Project
Brief Description of RBF Networks
• Consists of 3 layers (input, hidden, output)• Input layer made up of nodes that connect
network to environment• At input of each neuron (hidden layer), distance
between neuron center & input vector is calculated
• Apply RBF (Gaussian bell function) to form output of the neurons.
• Output layer is linear and supplies response of network to activation function.
Project Overview
Purpose: Develop a Radial Basis Network with a supervised selection of centers
Question: Are there any disadvantages or advantages between a fixed center RBF network and an adaptive RBF network?
A RBF network with multiple outputs
Adaptation Formulas
RBF with supervised selection of centers require the following formulas:
)(
)()()1( 1 nw
nEnwnw
iii
1. Linear Weights (output layer)
2. Positions of centers (hidden layer)
)(
)()()1( 2 nt
nEntnt
iii
3. Spreads of centers (hidden layer)
)(
)()()1(
1311
n
nEnn
iii
W: 1x1
T: 1xm vector
: mxm matrix
M is the feature dimension
1 i
Programming
• Used Matlab to implement RBF Network with Adaptive Centers
• Sample code for calculation of linear weights given below:
%Calculation of linear weights weightdiff=0; for j=1:n g=exp(-0.5((x(j,:)-t(i,:)))*covinv(:,:,i)*((x(j,:)-t(i,:))')); weightdiff = weightdiff + e(j)*g; end w(i)=w(i) - (eta1*weightdiff);
)(
)()()1( 1 nw
nEnwnw
iii
Testing & Comparison
• Tested Adaptive Center RBF against Fixed Center RBF.
• Used data for three functions, namely sinusoidal, piecewise-linear, and polynomial functions.
• Made use of the cost function given below analyze differences between two networks
N
jjeE
1
2
2
1
M
iCijij
jjj
itxGwd
xFde
1
)(
)(*Cost Function
where
Sinusoidal Function Testing
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6-1.5
-1
-0.5
0
0.5
1RBF with Adaptive Centers
test samplesapproximated curvetrain samplesradial basis
Sinosoid Function Data
0
0.1
0.2
0.3
0.4
0.5
0.6
2 3 4 5 6 7
No. of Radial Basis Functions
Co
st F
un
ctio
n O
utp
ut
Fixed Center RBF Network
Adaptive Center RBF Network
For fewer radial basis functions, adaptive center RBF network seems to perform a bit better. However, after number of RBFs increase, results in cost function are negligible.
Piecewise Linear Function Testing
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5RBF with Adaptive Centers
test samplesapproximated curvetrain samplesradial basis
Piecewise-Linear Function Data Chart
0
0.0005
0.001
0.0015
0.002
0.0025
0.003
0.0035
0.004
0.0045
2 3 4 5 6 7 8 9 10
No. Of Radial Basis Functions
Co
st F
un
ctio
n O
utp
ut
Fixed Center RBF Network
Adaptive Center RBF Network
Adaptive center RBF network performed better till the number of radial basis functions reached 6. I found that at higher numbers of radial basis functions (9 and above), both RBF networks were providing similar approximations of piecewise-linear function.
Polynomial Function Testing
The adaptive center RBF network was clearly the winner in the approximation of the polynomial function. Differences in cost function for higher numbers of RBFs were too small for Excel to plot.
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1RBF with Adaptive Centers
test samplesapproximated curvetrain samplesradial basis
Polynomial Function Data Chart
0.00E+00
1.00E-04
2.00E-04
3.00E-04
4.00E-04
5.00E-04
6.00E-04
7.00E-04
8.00E-04
2 3 4 5 6
No. of Radial Basis Functions
Cos
t Fun
ctio
n O
utpu
ts
Fixed Center RBF Network
Adaptive Center RBF Network
Conclusion
• Results show RBF network with adaptive centers performs slightly better than fixed-center RBF.
• Advantage of Adaptive RBF: Performs better with fewer RBFs
• Disadvantage of Adaptive RBF: Takes longer to run.
• Unless situation is known, one cannot say with certainty that one model is better than other.