Upload
bomezzz-enterprises
View
228
Download
0
Embed Size (px)
Citation preview
8/3/2019 L17 My Neural Nw
1/32
241-320 Design Architecture and Engineeringfor Intelligent System
Suntorn Witosurapot
Contact Address:Phone: 074 287369 or
Email: [email protected]
January 2011
8/3/2019 L17 My Neural Nw
2/32
Lecture 17:
Introduction to Neural Networks
8/3/2019 L17 My Neural Nw
3/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 3
What Are Artificial Neural Networks?
An extremely simplified model of the brain
Essentially a function approximator
Transforms inputs into outputs to the best of its ability
8/3/2019 L17 My Neural Nw
4/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 4
What Are Artificial Neural Networks?
Composed of many neurons that co-operate toperform the desired function
8/3/2019 L17 My Neural Nw
5/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 5
What Are They Used For?
Classification
Pattern recognition, feature extraction, imagematching
Noise Reduction
Recognize patterns in the inputs and producenoiseless outputs
Prediction Extrapolation based on historical data
8/3/2019 L17 My Neural Nw
6/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 6
What Are They Used For?(cont.)
Examples: Predicting time-series in financial world () Fraud detection (- )
Object Separation () etc
Very useful in Data Mining
imitate a humans ability to learn from experience
better results are the hope(Adequately designed and trained NN can capture varied patterns)
Drawback models tend to be difficult to understand(Is it necessary to understand?)
8/3/2019 L17 My Neural Nw
7/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 7
What Are They Used For?(cont.)
is not applying some set formula, but balancing herexperience and knowledge of sale prices of similarhousesher knowledge about housing prices is not
static...fine tuning her calculation to fit the latest data
Real EstateAppraiser
8/3/2019 L17 My Neural Nw
8/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 8
Loan Prospector ()
A Neural Network is like a black box that knows how toprocess inputs to create a useful output. The calculation isquite complex and difficult to understand, yet the resultsare often useful
8/3/2019 L17 My Neural Nw
9/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 9
Neural Net Attractions
Ability to learn
NNs figure out how to perform their function on
their own
Determine their function based only upon sampleinputs
Ability to generalize
i.e. produce reasonable outputs for inputs it hasnot been taught how to deal with
8/3/2019 L17 My Neural Nw
10/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 10
Neural Net Limitations
Neural Nets are good for prediction and estimationwhen:
Inputs are well understood
Output is well understood Adequate examples are trained the neural net
Neural Nets are only as good as the training set usedto generate it. The resulting model is static and mustbe updated with more recent examples and retrainingfor it to stay relevant
8/3/2019 L17 My Neural Nw
11/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 11
Feed-Forward Neural Net Examples
One-way flow through the network from inputs to outputs
8/3/2019 L17 My Neural Nw
12/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 12
Feed-Forward Neural Net Examples(cont.)
8/3/2019 L17 My Neural Nw
13/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 13
A Biological neuron
cell andnucleus
Axon(Neurit)
Dendrits
Synapsis
Dendrits: (Input)Getting other activations
Axon: (Output ) forward the activation(from 1mm up to 1m long)
Synapse: transfer of activation:
to other cells, e.g. Dendrits of otherneurons
a cell has about 1.000 to 10.000
connections to other cells Cell Nucleus: (processing)
evaluation of activation
8/3/2019 L17 My Neural Nw
14/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 14
Natural vs. Artificial Neuron
cell andnucleus
Axon(Neurit)
Dendrits
Synapsis
j
jjii ownet
),( iiacti netfact
)(iouti
actfo
w1i w2i wji...
oi
net : input from the network
w : weight of a connectionact : activationfact : activation function :bias/thresholdfout : output function (mostly ID)o : output
8/3/2019 L17 My Neural Nw
15/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 15
Abstraction
Dendrits:weighted (real number) connections
Axon: output: real number
Synapse: ---(identity: output is directly forwarded)
Cell nucleus:unit contains simple functions
input = (many) real numbersprocessing = activation functionoutput = real number (~activation)
8/3/2019 L17 My Neural Nw
16/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 16
Artificial Neuron
8/3/2019 L17 My Neural Nw
17/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 17
Loan Prospector()
8/3/2019 L17 My Neural Nw
18/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 18
NN can have multiple output neurons
8/3/2019 L17 My Neural Nw
19/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 19
How does Neural Net work?
8/3/2019 L17 My Neural Nw
20/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 20
Common transfer functions
8/3/2019 L17 My Neural Nw
21/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 21
Where Do The Weights Come From?
The weights in a neural network are the most importantfactor in determining its function
Training - process of setting the best weights on the
edges connecting all the units in the network
Use the training set to calculate weights such that NNoutput is as close as possible to the desired output foras many of the examples in the training set as possible
8/3/2019 L17 My Neural Nw
22/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 22
Where Do The Weights Come From?(cont.)
Network
changingnetwork parameters
evaluationnetwork errorlearning
examples
8/3/2019 L17 My Neural Nw
23/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 23
Where Do The Weights Come From? (cont.)
There are two main types of training
Supervised Training
Supplies the neural network with inputs and the desired outputs
Response of the network to the inputs is measured
Adjust weights such that differences between desired andactual outputs are minimized
Unsupervised Training
Only supplies inputs
The neural network adjusts its own weights so that similar inputsshould generate the similar outputs
The network identifies the patterns and differences in theinputs without any external assistance
8/3/2019 L17 My Neural Nw
24/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 24
Example: Voice Recognition
Task: Learn to discriminate between twodifferent voices saying Hello
Data Sources
Steve Simpson
David Raubenheimer
Format Frequency distribution (60 bins)
Analogy: cochlea
8/3/2019 L17 My Neural Nw
25/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 25
Network architecture
Feed forward network
60 input (one for each frequency bin) 6 hidden
2 output (0-1 for Steve, 1-0 for David)
8/3/2019 L17 My Neural Nw
26/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 26
Presentingthe data
Steve
David
8/3/2019 L17 My Neural Nw
27/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 27
Presenting the data (untrained network)
Steve
David
0.43
0.26
0.73
0.55
8/3/2019 L17 My Neural Nw
28/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 28
Calculate error
Steve
David
0.430 = 0.43
0.261 = 0.74
0.731 = 0.27
0.550 = 0.55
8/3/2019 L17 My Neural Nw
29/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 29
Backprop error and adjust weights
Steve
David
0.430 = 0.43
0.261 = 0.74
0.731 = 0.27
0.550 = 0.55
1.17
0.82
8/3/2019 L17 My Neural Nw
30/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 30
Repeat process (sweep) for all training pairs Present data
Calculate error Backpropagate error Adjust weights
Repeat process multiple times
8/3/2019 L17 My Neural Nw
31/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks 31
Presenting the data (trained network)
Steve
David
0.01
0.99
0.99
0.01
8/3/2019 L17 My Neural Nw
32/32
241-320 Design Architecture &Engineering for Intelligent System Introduction to Neural Networks
32
Results Voice Recognition
Performance of trained network
Discrimination accuracy between knownHellos 100%
Discrimination accuracy between newHellos
100%
Summary:
Network has learnt to generalise from original data
Networks with different weight settings can havesame functionality
Trained networks concentrate on lower frequencies
Network is robust against non-functioning nodes