Upload
penelope-norton
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
B.Macukow 1
Neural Networks Neural Networks
Lecture 4
B.Macukow 2
McCulloch symbolismMcCulloch symbolism
The symbolism introduced by McCulloch at the basis of simplified Venn diagrams is very useful in the analysis of logical networks
Two areas X1 i X2 correspond to two argument logic function. Symbol X1 means the input signal x1 = 1, its complement - signal x1 = 0.The same for X2.
B.Macukow 3
We have four fragments denoted:
X1X2 , X1 ~X2 , ~ X1 X2 and ~ X1 ~ X2
McCulloch symbolismMcCulloch symbolism
B.Macukow 4
Instead of circles we can used crosses
McCulloch symbolismMcCulloch symbolism
B.Macukow 5
Symbolic notation – cross with dots
X1X2
X1X2= (X1 ~X2 ) ( ~ X1 X2 ) ( X1X2)
conjunction – AND operation
disjunction -
OR operation
McCulloch symbolismMcCulloch symbolism
B.Macukow 6
Symbolic notation – cross with dots
~X2
X1X2
negation –
NOT operation
implication
McCulloch symbolismMcCulloch symbolism
B.Macukow 7
The function depending of parameters
x1 x1x2 x2
x1 x2 ~x1 x2
McCulloch symbolismMcCulloch symbolism
B.Macukow 8
Operations performed
•
•
• •••
x1
x1
x2
(
(
x1 x1x2 x2 ~~(( ) )
)x3x2 x2 ~~ ~ )x3x2x1 ((([ x3x3 ]) ) x2 ~~
[x1 ~ ~ x2 x3( ) ]
x1 ~ )x3x2~~
McCulloch symbolismMcCulloch symbolism
B.Macukow 9
The whole algebra
•
•••
••
Proof:
••
x1 x2 x1x1 x2 ~~( ) x1 ~( )x2~
x2x1 x2 ~( ) x1 ~~( )x2~x1 x2 •
•
x2x1~•
~ x2x1~ ~ x1 •x2
McCulloch symbolismMcCulloch symbolism
B.Macukow 10
Analysis of the simple nets composed from logical neurons
x1
x4x3
x2
xout
McCulloch symbolismMcCulloch symbolism
B.Macukow 11
Simplified notation
x1
x4x3
x2
xout
McCulloch symbolismMcCulloch symbolism
B.Macukow 12
The middle cross denotes an operation performed on the two symbols on either side. For example, the operation below means the operation which is not entered in either symbol on the left or symbol on the right should be written down as the result.
••• •
•
••
•••
Proof •)x1 x2•
•(
x3
x1 x2•( )
x4
McCulloch symbolismMcCulloch symbolism
B.Macukow 13
x3 x2~ x4 ~ x1 x2
x3•
x4 ~~ x3 x4
~ ( (x2~ ) ) ~ ~ x1 x2
x2x1
x2 ( )x1 x2~
(( ) )x2 x1 x2 x2~
)x1 x2••
(
x3
x1 x2•( )
x4
McCulloch symbolismMcCulloch symbolism
B.Macukow 14
The net with the loops
McCulloch symbolismMcCulloch symbolism
B.Macukow 15
Threshold influence for the neuron reaction
McCulloch symbolismMcCulloch symbolism
B.Macukow 16
Applications of McCulloch symbols, operation on the symbols
McCulloch symbolismMcCulloch symbolism
B.Macukow 17
Use of McCulloch symbols to denote the function of a neuron
McCulloch symbolismMcCulloch symbolism
B.Macukow 18
Network for modeling the conditioned reflex network realized by a single unconditioned reflex (UR) and a
conditioned reflex (CR)
McCulloch symbolismMcCulloch symbolism
B.Macukow 19
Venn diagram and McCulloch symbols for three outputs. Unknown are marked by A, B and C.
McCulloch symbolism for three outputs
McCulloch symbolism for three outputs
B.Macukow 20
McCulloch symbolism four outputs
McCulloch symbolism four outputs
Venn diagram and McCulloch symbols for four outputs. Unknown are marked by A, B, C and D.
B.Macukow 21
Logical functions of two unknown
Logical functions of two unknown
Logical functions of two unknownLogical functions of two unknown
Function Formula Description Diagram 00 01 10 11
Const 1 1 (AB)(A~B)(~AB)(~A~B)
1 1 1 1
NAND ~(AB) (A~B)(~AB)(~A~B)
1 1 1 0
Implication AB (AB)(~AB)(~A~B)
1
1
0
1
Negation A ~A (~AB)(~A~B) 1 1 0 0
Function Formula Description Diagram 00 01 10 11
Implication BA (AB)(A~B)(~A~B)
1 0 1 1
Negation B ~B (A~B)(~A~B) 1 0 1 0
equivalence AB (AB) (~A~B) 1
0
0
1
NOR ~(AB) (~A~B) 1 0 0 0
Logical functions of two unknownLogical functions of two unknown
Function Formula Description Diagram 00 01 10 11
disjunction AB (AB)(A~B)
(~AB)
0 1 1 1
non-equivalence
~(AB) (A~B)(~AB) 0 1 1 0
B B (AB)(~AB) 0
1
0
1
negation of implication ~AB (~AB) 0 1 0 0
Logical functions of two unknownLogical functions of two unknown
Function Formula Description Diagram 00 01 10 11
A A (AB)(A~B) 0 0 1 1
negation of implication
A ~ B (A~B) 0 0 1 0
conjunction A B (AB) 0
0
0
1
constant 0 0 0 0 0 0
Logical functions of two unknownLogical functions of two unknown
B.Macukow 26
Neural NetworksNeural Networks
B.Macukow 27
Neural NetworksNeural Networks
At the beginning was the idea that it is enough to build the net of many randomly connected elements to get the model of the brain operation.
Question: how many element is necessary for the process of self organization ??
B.Macukow 28
Research of McCulloch, Lettvin, Maturana, Hartlin and Ratliff.
Research on the frog’s eye and specially on the compound eye of the horseshoe cram - Limulus.
Hubel and Wiesel research on the mammals visual system.
Some parts are constructed in the very special, regular way.
Neural NetworksNeural Networks
B.Macukow 29
Two – layers chain structure
Neural NetworksNeural Networks
B.Macukow 30
The input layer of photoreceptors and the layer of processing elements which will locate the possible changes in the excitation distribution.
Connection rule:
Each receptor cell is to excite one element (exactly below). In addition to the excitatory connections there are also inhibitory connections (for the simplicity - to the adjacent cells only) which reduce the signal to the neighbors.
Neural NetworksNeural Networks
B.Macukow 31
The inhibition range can differs.
This is known as the of lateral inhibition
Neural NetworksNeural Networks
B.Macukow 32
As can be easily seen the uniform excitation of the first layer will not excite the second layer. The excitatory and inhibitory signals will be balanced.
A step signal is a step change in the spatial distribution. The distribution of output signal is not a copy of the input signal distribution but is the convolution of the input signal and the weighting function.
Neural NetworksNeural Networks
B.Macukow 33
The point in which the step change occurs is exaggerated at each side by increasing and decreasing the signal resulting in the single signal at the point of the this step.
Neural NetworksNeural Networks
B.Macukow 34
Input Signals
Output Signals
1
Elements’ transfer function
Neural NetworksNeural Networks
B.Macukow 35
As you can see such a network gives the possibility to locate the point where the changes in the excitation were enough high (terminations, inflections, bends etc.).
From the neurophysiology we know on the existence of the opposite operation lateral excitation.
These nets allows to detect the points of crossing or branchings etc.
Neural NetworksNeural Networks
B.Macukow 36
The lateral inhibition rule can be realized be the one dimensional net with negative feedback
Attention: elements are nonlinear and the feedback loops make analysis difficult; such the networks can be non stable and the distribution of the input signals does not depends univocally from the input signals.
Neural NetworksNeural Networks
Another simple neural netsAnother simple neural nets