Soft Computing PDF

8/13/2019 Soft Computing PDF

1/103

SOFT COMPUTING


2/103

2

Human sciences

quantitative qualitative

numerical dataprecise objects

conventional logic

complicated mathematicscomputer models

non-numerical dataimprecise objects

approximate reasoning

interpretationmanual work

methodsare

methodsare

that means that means

often usedseparately

social sciencesbehavioral sciences

the humanitieseconomics

law

medicine

include


3/103

3

Traditional Approaches to Computerized Modeling

Mathematical models:Complicated, black

boxes, numbercrunching.

Rule-based systems(crisp & bivalent):

Large rule bases.


4/103

What is Hard Computing?

Hard computing, i.e., conventional computing, requires a

precisely stated analytical model and often a lot of

computation time.

Many analytical models are valid for ideal cases.

Real world problems exist in a non-ideal environment.

Premises and guiding principles of Hard Computing are

Precision, Certainty, and rigor. Many contemporary problems do not lend themselves to precise

solutions such asRecognition problems (handwriting, speech, objects,

images)

Mobile robot coordination, forecasting, combinatorial problems etc.


5/103

M

V

L

s(t) H(t)

md2/dt2(s(t)+Lsin(t) = H(t)md2/dt2(Lcos(t)) = V(t)-mgJd2/dt2= (LV(t)sin(t)-LH(t)cos(t)) = V(t)-mgMd2/dt2s(t) = (t)-H(t)-Fd/dts(t)


6/103

Soft Computing (SC)

mimic humanreasoning that

is oftenlinguistic and

approximate bynature

fuzzy systems, probabilistic reasoning,natural computing (evolutionary computing,

cellular automata, DNA computing,neurocomputing, immunocomputing,

swarm theory, etc.).

imprecisionlearning

uncertaintyoptimization

adaptive andintelligent systems

orcomputational

intelligence

mathematical orstatistical methods(hybrid methods)

aims to includes

also known as

can copewith

can be usedwith


7/103

What is Soft Computing ?

(adapted from L.A. Zadeh)

Soft computing differs from conventional (hard) computingin that, unlike hard computing, it is tolerant of imprecision,

uncertainty, partial truth, and approximation. In effect, the role

model for soft computing is the human mind.

Guiding Principles of Soft Computing The guiding principle of soft computing is:

Exploit the tolerance for imprecision, uncertainty, partial truth, and approximation

to achieve tractability, robustness and low solution cost.


8/103

As complexi ty r ises, precise

statements lose meaning and

meaningfu l statements loseprecision

-

Lotfi Zadeh


9/103

Premises of Soft Computing

The real world problems are pervasively imprecise and uncertain

Precision and certainty carry a cost


10/103

Techniques in Soft ComputingNeural networks

Fuzzy Logic

Genetic Algorithms in Evolutionary ComputationParticle swarm optimization

Neuro-Fuzzy systems


11/103

Unique Property of Soft computing Learning from experimental data

Soft computing techniques derive their power of generalization from

approximating or interpolating to produce outputs from previously unseeninputs by using outputs from previous learned inputs

Generalization is usually done in a high dimensional space.


12/103

Current Applications using Soft Computing Application of soft computing to handwriting recognition

Application of soft computing to automotive systems and

manufacturing

Application of soft computing to image processing and

data compression Application of soft computing to architecture

Application of soft computing to decision-support

systems

Application of soft computing to power systems

Neurofuzzy systems

Fuzzy logic contro


13/103

Future of Soft Computing(adapted from L.A. Zadeh)

Soft computing is likely to play an especially important role in science

and engineering, but eventually its influence may extend much

farther. Soft computing represents a significant paradigm shift in the aims of

computing

a shift which reflects the fact that the human mind,

unlike present day computers, possesses a remarkable ability to store

and process information which is pervasively imprecise, uncertain and

lacking in categoricity.


14/103

14

SC Applications: Control

Heavy industry(Matsushita, Siemens,Stora-Enso, Metso)

Home appliances (Canon,Sony, Goldstar, Siemens)

Automobiles (Nissan,Mitsubishi, Daimler-Chrysler, BMW,Volkswagen)

Space crafts (NASA)
http://web1.mitsubishi-motors.co.jp/inter/technology/technology.html


15/103

Fuzzy logic in Reality (Industrial Applications)

Efficient and stable control of car-engines

(Nissan)Simplified control of robots (Hirota,Fuji Electric, Toshiba, Omron)

Industrial control applications

(Aptronix, Omron, Meiden, Micom,

Mitsubishi, Nissin-Denki, Oku-Electronics)Archiving system for documents

(Mitsubishi Elec.)

Prediction system for early recognitionof earthquakes

(Bureau of Metrology, Japan)

Recognition of handwritten symbols withpocket computers (Sony)

Video cameras (Canon, Minolta)

Washing-machines (Matsushita,Hitatchi, Samsung)

Recognition of handwriting,objects, voice (Hitachi, HosaiUniv., Ricoh)

Efficiency for elevator control(Fujitec, Hitachi, Toshiba)

Positioning of wafer-steppers inthe production ofsemiconductors

(Canon)

Automatic control of dam gatesfor hydroelectric-power plants(Tokyo Electric Power)


16/103

Components of soft computing include:Neural networks(NN)Fuzzy logics(FL)Evolutionary computation(EC),

including:Evolutionary algorithms

Genetic algorithmsDifferential evolution

Metaheuristicand SwarmIntelligence

Ant colony optimizationBees algorithmsBat algorithmCuckoo searchHarmony searchFirefly algorithm

Artificial immune systemsParticle swarm optimization

Ideas about probabilityincluding:Bayesian network

Chaos theoryPerceptron
http://en.wikipedia.org/wiki/Neural_networkhttp://en.wikipedia.org/wiki/Fuzzy_logichttp://en.wikipedia.org/wiki/Evolutionary_computationhttp://en.wikipedia.org/wiki/Evolutionary_algorithmhttp://en.wikipedia.org/wiki/Genetic_algorithmhttp://en.wikipedia.org/wiki/Differential_evolutionhttp://en.wikipedia.org/wiki/Metaheuristichttp://en.wikipedia.org/wiki/Swarm_Intelligencehttp://en.wikipedia.org/wiki/Swarm_Intelligencehttp://en.wikipedia.org/wiki/Ant_colony_optimizationhttp://en.wikipedia.org/wiki/Bees_algorithmhttp://en.wikipedia.org/wiki/Bat_algorithmhttp://en.wikipedia.org/wiki/Cuckoo_searchhttp://en.wikipedia.org/wiki/Harmony_searchhttp://en.wikipedia.org/wiki/Firefly_algorithmhttp://en.wikipedia.org/wiki/Artificial_immune_systemhttp://en.wikipedia.org/wiki/Particle_swarm_optimizationhttp://en.wikipedia.org/wiki/Probabilityhttp://en.wikipedia.org/wiki/Bayesian_networkhttp://en.wikipedia.org/wiki/Chaos_theoryhttp://en.wikipedia.org/wiki/Perceptronhttp://en.wikipedia.org/wiki/Perceptronhttp://en.wikipedia.org/wiki/Chaos_theoryhttp://en.wikipedia.org/wiki/Bayesian_networkhttp://en.wikipedia.org/wiki/Probabilityhttp://en.wikipedia.org/wiki/Particle_swarm_optimizationhttp://en.wikipedia.org/wiki/Artificial_immune_systemhttp://en.wikipedia.org/wiki/Firefly_algorithmhttp://en.wikipedia.org/wiki/Harmony_searchhttp://en.wikipedia.org/wiki/Cuckoo_searchhttp://en.wikipedia.org/wiki/Bat_algorithmhttp://en.wikipedia.org/wiki/Bees_algorithmhttp://en.wikipedia.org/wiki/Ant_colony_optimizationhttp://en.wikipedia.org/wiki/Swarm_Intelligencehttp://en.wikipedia.org/wiki/Swarm_Intelligencehttp://en.wikipedia.org/wiki/Metaheuristichttp://en.wikipedia.org/wiki/Differential_evolutionhttp://en.wikipedia.org/wiki/Genetic_algorithmhttp://en.wikipedia.org/wiki/Evolutionary_algorithmhttp://en.wikipedia.org/wiki/Evolutionary_computationhttp://en.wikipedia.org/wiki/Fuzzy_logichttp://en.wikipedia.org/wiki/Neural_network


17/103

What Is A Neural Network?

The simplest definition of a neural network, more properlyreferred to as an 'artificialneural network (ANN), is providedby the inventor of one of the first neurocomputers, Dr.Robert Hecht-Nielsen.

He defines a neural network as: "...a computing system

made up of a number of simple, highly interconnectedprocessing elements, which process information by theirdynamic state response to external inputs.

In "Neural Network Primer: Part I" by Maureen Caudill, AI

Expert, Feb. 1989


18/103

Neural net work definations

In general form, a neural network is a machinethat is designed to model the way in which the

brain performs a particular task or function of

interest

a neural network is a system composed of many

simple processing elements operating in parallel

whose function is determined by network

structure, connection strengths, and theprocessing performed at computing elements

or nodes


19/103

A neural network is a massively parallel distributed

processor made up of simple processing units, whichhas a natural propensity for storing experiential

knowledge and making it available for use.

It resembles the brain in two respects:

Knowledge is acquired by the network through a

learning process.

Interneuron connection strengths known as synaptic

weights are used to store the knowledge


20/103

Applications

Non linearity.

A neural network, made up of aninterconnection of nonlinear neurons is

itself nonlinear.

Input-output mapping


21/103

Adaptivity; Neural networks have a builtin capacity to adapt their synaptic weightsto changes in the surrounding

environment.

Evidential response: Indicates the

confidence in decision made;


22/103

McCulloch and Pitts produced the first

neural network in 1943Many of the principles can still be seen

in neural networks of today


23/103

If the brain were so

simple that we could

understand it then wedbe so simple that we

couldntLyall Watson


24/103

Biological inspiration

Animals are able to react adaptively to changes in their external andinternal environment, and they use their nervous system to performthese behaviours.

An appropriate model/simulation of the nervous system should beable to produce similar responses and behaviours in artificialsystems.

The nervous system is build by relatively simple units, the neurons,so copying their behavior and functionality should be the solution.


25/103


Dendrites

Soma (cell body)

Axon


26/103


synapses

axondendrites

The information transmission happens at the synapses.


27/103

We are born with about 100 billion neurons

A neuron may connect to as many as 100,000 other neurons


28/103

Signals move via electrochemical

signals The synapses release a chemical

transmitterthe sum of which can

cause a threshold to be reachedcausing the neuron to fire

Synapses can be inhibitory or

excitatory


29/103

Comparison of Brains and Traditional

Computers

200 billion neurons, 32

trillion synapses Element size: 10-6m

Energy use: 25W

Processing speed: 100 Hz

Parallel, Distributed

Fault Tolerant

Learns: Yes

Intelligent/Conscious:Usually

1 billion bytes RAM but

trillions of bytes on disk Element size: 10-9 m

Energy watt: 30-90W (CPU)

Processing speed: 109 Hz

Serial, Centralized

Generally not Fault Tolerant

Learns: Some

Intelligent/Conscious:Generally No


30/103

Neurons in the Brain

Although heterogeneous, at a low levelthe brain is composed of neurons A neuron receives input from other neurons

(generally thousands) from its synapses

Inputs are approximately summed

When the input exceeds a threshold theneuron sends an electrical spike that travelsthat travels from the body, down the axon, tothe next neuron(s)


31/103


The spikes travelling along the axon of the pre-synaptic neurontrigger the release of neurotransmitter substances at the synapse.

The neurotransmitters cause excitation or inhibition in the

dendrite of the post-synaptic neuron.The integration of the excitatory and inhibitory signals mayproduce spikes in the post-synaptic neuron.

The contribution of the signals depends on the strength of thesynaptic connection.


32/103

McCulloch & Pitts (1943) are generally recognised as thedesigners of the first neural network

Many of their ideas still used today (e.g. many simple units

combine to give increased computational power and the idea of athreshold

Hebb (1949) developed the first learning rule (on the premisethat if two neurons were active at the same time the strength

between them should be increased

During the 50s and 60s many researchers worked on theperceptron amidst great excitement.

1969 saw the death of neural network research for about 15

yearsMinsky & Papert

Only in the mid 80s(Parker and LeCun) was interest revived (infact Werbos discovered algorithm in 1974


33/103


34/103

We are born with about 100 billion neurons

A neuron may connect to as many as 100,000 other neurons


35/103

Fault tolerance a neural net workimplemented in hardware form, has thepotential to be inherently fault tolerent,or

capable of robust computation VLSI Implement- ability. The massive

parallel nature of a neural network makes

it potentially fast for the computation ofcertain task.


36/103


37/103


38/103


39/103


40/103


41/103


42/103


43/103


44/103


45/103


46/103
http://en.wikipedia.org/wiki/File:Neural_network_example.svg


47/103

Simplified view of a feedforward artificial neural network
http://en.wikipedia.org/wiki/File:Neural_network_example.svghttp://en.wikipedia.org/wiki/File:Neural_network_example.svghttp://en.wikipedia.org/wiki/File:Neural_network_example.svghttp://en.wikipedia.org/wiki/File:Neural_network_example.svg


48/103


49/103


50/103

Network architecture

Feed forward network

60 input (one for each frequency bin)

6 hidden

2 output (0-1 for Steve, 1-0 for David)


51/103

Presenting the dataSteve

David


52/103

What is meant by learning in NN ?

Learning is a process by which the freeparameters of a neural network are adapted

through a process of stimulation by the

environment in which the network is

embedded.

The type of lerning is determined by the

manner in which the parameters changes takeplace.


53/103

Classification of Learning algorithms

Supervised Learning:Error Correction, Gradient descent :

LMS , Back PropStochastic (Boltzmann)

Re-inforcement learning

Unsupervised learningHebbianCompetitive

S i d l i A i ti l i i hi h th t k i


54/103

Supervised learning or Associative learning in which the network istrained by providing it with input and matching output patterns.These input-output pairs can be provided by an external teacher


55/103

Reinforcement Learning / Neurodynamic

Programming

In supervised learning we have assumed that there is a target output value for each


56/103

In supervised learning we have assumed that there is a target output value for eachinput value. However, in many situations, there is less detailed information available.In extreme situations, there is only a single bit of information after a long sequenceof inputs telling whether the output is right or wrong. Reinforcement learning isone method developed to deal with such situations.

Reinforcement learning (RL) is a kind of supervised learning in that some feedbackfrom the environment is given. However the feedback signal is only evaluative, notinstructive. Reinforcement learning is often called learning with a criticas opposedto learning with a teacher.

Humans learn by interacting with the environment. When a baby plays, it waves itsarms around, touches things, tastes things, etc. There is no explicit teacher but

there is a sensori-motor connection to its environment. Such a connectionprovides information about cause and effect, the consequence of actions, and whatto do to achieve goals.

Learning from interaction with our environment is a fundamental idea underlyingmost theories of learning. RL has rich roots in the psychology of animal learning,from where it gets its name.

The growing interest in RL comes in part from the desire to build intelligentsystems that must operate in dynamically changing real- world environments.

Robotics is the common example.


57/103

Unsupervised learning


58/103

Error correction method

Error correction is, as the name indicates, about correcting the

synaptical strengths according to the error in the neurons output.

Mathematically the definition of a neurons output error is given by

this equation:

This equation is one of the important ones, and is used frequentlythroughout this page. It is explained more thoroughly on the next

page, where also the time is involved.


59/103


60/103

The change of the neuron ks synaptic weight is determied by the

error of the neurons output, the associated synapses and a learning

rate. This rule is named The Widrow-Hoff-rule, or simply The

Delta-rule.

The larger error the larger change.

The input signals (synapses) xjalso determins the rate of learning.

If an input signal is small, it havent got much influence in the

neurons error. The associated synaptic weight, should likewise not

be changed much.

The learning rate is an auxillary value determining the rate of

learning. It can be set by the teacher to control the updates of the

synapticweights.


61/103

The update of neuron k's synaptical strengths is now very

simple. The future strength of synapse j in neuron k has to beequal to the present plus the given alteration of the synapse.

The synapses of neuron k are now altered and the result of thesame input should now be closer to the wanted than before.

P t L i


62/103

Perceptron Learning


63/103


64/103


65/103


66/103


67/103


68/103


69/103


70/103


71/103


72/103


73/103


74/103

Competitive learning


75/103

Competitive learning is a simple learning algorithm, which is veryuseful for classify set of input patterns. In competitive learningthere is only one active neuron at a time, and it's only the activeneuron that is trained.

Three basic elements of competitive learning rule

A set of neurons that all are same except for some randomlydistribute synaptic weights which therefore respond differently togiven set of input patters

A limit imposed on the strength of each neuron

Mechanism that permits the neuron to compete for the right torespond to a given subset of inputs such that only one outputneuron or only one neuron group is active at a time the neuron that

wins the competition is called a winner takes all neuron

Competitive learning


76/103


77/103

Actually this just means that all outputs is run through and the

neuron with the largest output, has an output at 1. The rest of theneurons will have 0 as output.

Hebbian learning


78/103

Donald O. Hebb presented a theory of behaviour

based as much as possible on the physiology of the

nervous system.

The most important concept to emerge from Hebb's work was his formal

statement (known as Hebb's postulate) of how learning could occur. Learning

was based on the modification of synaptic connections between neurons.

Specifically,

When an axon of cell A is near enough to excite a

cell B and repeatedly or persistently takes part in

firing it, some growth process or metabolic change

takes place in one or both cells such that A's

efficiency, as one of the cells firing B, is increased.


79/103

.

1. if two neurons on either side of synapse areactivate simultaneously then the strength of that

synapse is selectively increased.

2. If two neurons on either side of a synapse are

activated asynchronously, the that synapse is

selectively weakned or eliminated.

H bbi


80/103

Hebbian synapse

Time dependent mechanism. This mechanism refers to the fact that modification in a

hebbian synapse depend on the exact time of occurrence of the presynaptic and

post synaptic signals.

Local mechanism. The local available information is used by a Hebbian synapse to

produce local synaptic modification that is input specific.

Interactive mechanism. The occurrence of a change in a Hebbian synapse depends

on signals on both sides of synapse.(deterministic or stochastic)

conjunctional or correlation Mechanism;Conjuction( co-ocurrence) of presynaptic and post synaptic signals with in a short

interval of time is sufficient to produce the synaptic modification.

And the correlation between presynaptic and post synaptic signals viewed as being

responsible for synaptic changes.

S M d f


81/103

Synaptic Modifications

Hebbian:Hebbian synapse increases its strengthfor positively correlated presynaptic and

postsynaptic signals and decreases its strength

when these signals are either uncorrelated or

negatively correlated Anti Hebbian: strengthen the negatively

correlated signals

Non Hebbian: Does not involve Hebbianmechanism


82/103

The basic idea is that if two units j and k are active simultaneously, theirinterconnection must be strengthened. If j receives input from k, thesimplest version of Hebbian learning is described by

Application of the hebbian learning rule: the linear


83/103

associator

Take a network of neurons which are organized in two distinct layers (calledf and g). Neurons in layer f project to neurons in layer g but not vice versa.Each neuron in layer g is a linear unit, its output is the sum of its inputs(hence the name linear associator). In vector notations, this is equivalent towriting: g = a*fThe strength of the connection form presynaptic neuron f[j] to postsynpatic

neuron g[i] is given by a[i,j].This notation may seem strange (w[i,j] denotingthe connection going from neuron j to neuron i but is the most commonlyused). The activation of each neuron in the output layer is given by is sum ofweighted inputs. The strength of each connection is calculated from theproduct of the pre- and postsynaptic activities scaled by a learningrate(which determines how fast connection weights change).

wij= * g[i] * f[j]. in vector notations, this is equivalent to writing: w= *g* fT. We will calculate anexample in class!

The linear associator stores associations between a pattern of neural activationsin the input layer f and a pattern of activations in the output layer g Once the


84/103

in the input layer f and a pattern of activations in the output layer g. Once theassociations have been stored in the connection weights between layer f andlayer g, the pattern in layer g can be recalled by presentation of the inputpattern in layer f.

An auto-associator stores association within a layer of neurons. Once theassociations between the neural activities in a given pattern are stored in theconnection weights, the auto-associator can recall the storedpattern from a noisy or incomplete input pattern.


85/103

Boltzmann Learning(stochastic method)

Boltzmann learning is statistical in nature, and is derived from thefield of thermodynamics. It is similar to error-correction learningand is used during supervised training. In this algorithm, the state ofeach individual neuron, in addition to the system output, are takeninto account. In this respect, the Boltzmann learning rule issignificantly slower than the error-correction learning rule. Neuralnetworks that use Boltzmann learning are called Boltzmannmachines.

Boltzmann learning is similar to an error-correction learning rule, inthat an error signal is used to train the system in each iteration.However, instead of a direct difference between the result valueand the desired value, we take the difference between theprobability distributions of the system.


86/103

Boltzmann machine properties

Recurrent network with all nodes connected to each other

Nodes have binary outputs (either 0,1 or -1,1) Weights between the nodes are symmetric

Connection from a node to itself is not allowed

Nodes are updated asynchronously (i.e. nodes are selected at random)

The network can have hidden nodes

Learning can be supervised or unsupervised

Node activation is stochastic

The machine is characterized by the energy function E and value dependsupon particular states occupied by individual neurons where xjis the stateof j neuron and no self feed back


87/103

Activation algorithm

A node i is chosen at random for updating and the state of the node is setto 1 with a probability :

where is the energy difference from the ith unit being on or off. Thisdifference is given by derivation of the energy E with respect to the nodes

state and is therefor:

And T is a parameter which simulates the temperature of a physicalsystem, it is used with simulated annealing in the learning process.


88/103

Neurons in boltzmann machine can be of two

groups:Visible neurons and Hidden neurons There are two modes of operation: clamped

condition and free-running condition ( for both

neurons)

Where P+IJ is correlation under clamped condition between i and j neuronAnd p-ij is correlation under free running condition and i not equal to k

.


89/103

The network can be divided into to sets of nodes, a nonempty set ofvisible nodes V and a set of hidden nodes H. Visible nodes are those which

are the interface towards the environment or the outside world. Thehidden nodes used only for internal representation or modeling and doesnot interact with the outside world.

Supervised learning can be achieved by declaring some of the visible nodesas input nodes and some as output nodes. When we have no output nodesamong the visible nodes we have unsupervised learning.

The learning algorithm has two different phases. The clamping phase andthe free-running phase. In the clamping phase we lock or clampa set ofthe visible nodes to specific values given by a specific pattern. Nodes thatare clamped wont change their output value and then we sample thevalues of the hidden nodes using simulated annealing.

In the free-running phase no nodes are locked/clamped and we samplevalues for all the nodes using simulated annealing.

M B d L i


90/103

Memory Based Learning

Memory Based Learning (MBL) is a simple function approximationmethod whose roots go back at least to 1910. Training a memorybased learner is an almost trivial operation: just store each datapoint in memory (or a database). Making a prediction about theoutput that will result from some input attributes based on the data

is done by looking for similar points in memory, fitting a local modelto those points, and then making a prediction based on the model.There are four components that define a memory based learner: adistance metric, the number of nearest neighbors, a weightingfunction, and a local model. Each will be described in turn in the nextfour subsections followed by a discussion of multivariate

considerations and classification problems.

Memory based learning


91/103

Memory-based learning is actually a method of storing past experience. Its storesinput/output examples in a set of vectors for later use in a large memory grouping.

A memory-based system needs two things: criterion which defines the localneighborhood of inputs, and learning rules which are applied to the training examples inthat local neighborhood.

Essentially the system is given a set of examples and when given a set of inputs it triesto figure out which one it most closely resembles. Because of this system's skill inclassifying, even an infinite size learning set, only half of it would ever be used.

This type of system is exceptional at identifying outliers in a data set in order toapproximate or classify its category or function [4].

XN { x1, x2,xN} is said to be nearest neighbor of vectror Xtestif min d(xi, xtest) = d (xN, Xtest) where d (xN, Xtest) is the Euclidean distance between the vectors xi,

and xtest

Neural network applications


92/103

Neural network applications

Pattern association: There are two types of patternassociation, autoassociationand heteroassociation.

Autoassociation is about constantly showing the network a certain

pattern, the network should be able to store this pattern, and then when

a distorted image of the same pattern shows up, the task is to retrieve

the pattern.

Heteroassociationis different in the sense that it is supervised, and it is

a pairing of a set of input patterns and a set of output patterns. Mostly it

is about the retrieving phase, xkis a key pattern and ykis a memorised

pattern, the task is to make the network retrieve ykat the presentation of

xk.

Pattern

associator

Output vectorY

Input vector

Storage phase andRecall phase

Patterniti

Unsupervised network for

feature extraction

supervised network for

classifications


93/103

recognition

Pattern recognition is formally defined as the process where by a receivedpattern/ signal is assigned to one of a prescribed number of classes(categories)

m-dimensionalobservation space

q-dimensional featurespace

r-dimensional

decision space

Input pattern x Feature vector Y

Function Approximation


94/103

pp Function Approximation: The neural network can also be used to

approximate functions, this means that the network is able to receive an

input and a desired output, and then approximate the function that hasbeen used,

Let d = f(x)Let's say that F(x)is the real function used, the function that thenetwork comes up with, will be very close by: F(x) - f(x) < eWhere e is asmall positive number specifying the acceptable error. We now have afunction to decide which output matches the specific input. The difference e

and the network output y, sends out an error-signal, this is used to adjustthe free parameters, to make the difference disappear statistically.

The image above shows a two graphs, one is the f(x) function, the other theapproximated F(x) function.

F(x)-f(x)


95/103

System identification

Unknownsystem

NN

Inputvector Xi

di

Yi

ei

I


96/103

Inverse system

f(.)di

Yi

ei

Inversemodel

Inputvector Xi

X=f -(d)

System output

Model output

Control J= yk/ uj


97/103

Yei

Controller

Reference input

System outputPlant input

Plantdi

Unity feed back

Indirect learning: Using actual input output measurement on the plant Aneural network model is constructed to produce copy of it. This model isinturn used to provide an estimate of the Jacobian matrix J. The partialDerivatives constituting this jacobian matrix are subsequently used in errorcorrection learning algorithm for computing the adjustments to the freeparameters of the NN

j

Direct learning: The sign of the partial derivatives are generally known andusually remain constant over the dynamic range of the plant. This suggest thatwe may approximate these partial derivative by their individual signs. Theirabsolute values are given a distributed representation in the free parameter ofthe neural controller. The neural controller their by enabled to learn theadjustments to its free parameters directly from the plant

Filtering


98/103

Filtering

The term filter refers to a device or algorithm used to extractinformation about a prescribed quantity of interest from a set ofnoisy data. We as humans are also able to perform filtering, in manysituations we are forced to isolate one voice from hundreds ofothers, in order to get the information we need. Neural networkscan, just as the brain, be trained to remove noise from various

sources with great results.

Filter can be used to perform the following


99/103

p g

information processing tasks

Filtering: This task refers to the extraction of information atdiscrete time n using data measured up to and including time n

Smoothing: The information about the quantity of interest need notbe available at time n and data measured later time n can be used in

obtaining this information. Thus there is a delay in producing results.The results may be more accurate than filtering.

Prediction This task is the forecasting side of informationprocessing. The aim here is derive information about what the

quantity of interest will be like at some time n + n0in the future forsome no, by using data measured up to and including time n.

Nonlinear prediction


100/103

Nonlinear prediction

NN

X(n-T)

X(n-2T)

X(n-mT)

X(n)

Estimated value ofx(n)

Beam forming


101/103

Beam forming

Beamforming is a spatial form of filtering and is used todistinguish between the spatial properties of a targetsignal and back ground noise. The devise used to do thebeam forming is called beamformer

Beamforming is commonly used in radar and sonarsystems where the primary task is to detect and track atarget of interest in the combined presence of receivernoise and interfering signal( e. g. Jammers). This task iscomplicated by two factors

That target signal originates from an unknown direction

There is no a priori information available on the interferingsignals

Generalized sidelobe canceller


102/103

Generalized sidelobe canceller

w1

NNSignalBlocking

Matrix Ca

w2

wm

Desired responsed(n)

Out puty(n)

x(n)

u1(n)

u2(n)

um(n)


103/103

An array of antenna elements:

A linear combiner: Acts as spatial filter characterized by a radiation pattern

Signal blocking matrix:

The function of which is to cancel interference that leaks through thesidelobes of the radiation pattern of the spatial filter representing linearcombiner.

Neural network

Documents

Soft Computing PDF