• Theory

• Simulations

• Applications

• Theory

• Simulations

• Applications

• Analytic techniques modeled after the Analytic techniques modeled after the processes of learning in the cognitive processes of learning in the cognitive system and the neurological functions of system and the neurological functions of the brain the brain

• Capable of predicting new observations Capable of predicting new observations from other observations from other observations

• Analytic techniques modeled after the Analytic techniques modeled after the processes of learning in the cognitive processes of learning in the cognitive system and the neurological functions of system and the neurological functions of the brain the brain

• Capable of predicting new observations Capable of predicting new observations from other observations from other observations

• Information is received at the synapses Information is received at the synapses on its dendrites on its dendrites

• An electro-chemical transmission occurs An electro-chemical transmission occurs at the synapses at the synapses

• At the cell body, a summation of the At the cell body, a summation of the electric impulses takes place electric impulses takes place

• If summation meets a particular If summation meets a particular threshold, the neuron will send a signal threshold, the neuron will send a signal

• Information is received at the synapses Information is received at the synapses on its dendrites on its dendrites

• An electro-chemical transmission occurs An electro-chemical transmission occurs at the synapses at the synapses

• At the cell body, a summation of the At the cell body, a summation of the electric impulses takes place electric impulses takes place

• If summation meets a particular If summation meets a particular threshold, the neuron will send a signal threshold, the neuron will send a signal

• Receives a number of inputsReceives a number of inputs

• Each input has a weight that corresponds Each input has a weight that corresponds to synaptic efficacy in a biological neuronto synaptic efficacy in a biological neuron

• Weighted sum of the inputs is formed Weighted sum of the inputs is formed

• Activation if weighted sum meets the Activation if weighted sum meets the threshold valuethreshold value

• Receives a number of inputsReceives a number of inputs

• Each input has a weight that corresponds Each input has a weight that corresponds to synaptic efficacy in a biological neuronto synaptic efficacy in a biological neuron

• Weighted sum of the inputs is formed Weighted sum of the inputs is formed

• Activation if weighted sum meets the Activation if weighted sum meets the threshold valuethreshold value

• Extensively used for classification and regression problems

• Does not suffer the curse of dimensionality

• Valuable for solving problems with a large number of inputs

• Extensively used for classification and regression problems

• Does not suffer the curse of dimensionality

• Valuable for solving problems with a large number of inputs

• Maximize the margin between the Maximize the margin between the separating patterns using a hyperplaneseparating patterns using a hyperplane

• Margin of separation is the separation Margin of separation is the separation between the hyperplane and the closest between the hyperplane and the closest data pointdata point

• The goal is to find the optimal The goal is to find the optimal hyperplane to maximize the margin of hyperplane to maximize the margin of separationseparation

• Maximize the margin between the Maximize the margin between the separating patterns using a hyperplaneseparating patterns using a hyperplane

• Margin of separation is the separation Margin of separation is the separation between the hyperplane and the closest between the hyperplane and the closest data pointdata point

• The goal is to find the optimal The goal is to find the optimal hyperplane to maximize the margin of hyperplane to maximize the margin of separationseparation

• Support vectors are data points that lie directly on the decision boundary

• Support vectors serve a very important role in the operation of this algorithm

• Support vectors are data points that lie directly on the decision boundary

• Support vectors serve a very important role in the operation of this algorithm

• Slack variables consider the case of nonseparable patterns, and measure the deviation of the data points from the boundary of the region of separability

• Slack variables consider the case of nonseparable patterns, and measure the deviation of the data points from the boundary of the region of separability

• Given the training example:

• Find the optimum values of the weight vector and bias such that they satisfy the constraint:

• Given the training example:

• Find the optimum values of the weight vector and bias such that they satisfy the constraint:

Niii dx 1, Niii dx 1,

Niforbxwd iiT

i ,,2,11 Niforbxwd iiT

i ,,2,11

iallfori 0 iallfori 0

• Minimize the cost functional:

• The parameter C quantifies the trade-off between training error and system capacity

• Minimize the cost functional:

• The parameter C quantifies the trade-off between training error and system capacity

N

ii

T Cwww12

1,

N

ii

T Cwww12

1,

• The inner-product kernel may be used to construct the optimal hyperplane in the feature space without having to consider the feature space itself in explicit form

• The requirement on the construction of the kernel is that it satisfies Mercel’s theorem

• The inner-product kernel may be used to construct the optimal hyperplane in the feature space without having to consider the feature space itself in explicit form

• The requirement on the construction of the kernel is that it satisfies Mercel’s theorem

• polynomial learning machine:

• radial-basis function network kernel:

• two-layer perceptron kernel:

• polynomial learning machine:

• radial-basis function network kernel:

• two-layer perceptron kernel:

piT xx 1 piT xx 1

2

22

1exp ixx

2

22

1exp ixx

10tanh iT xx 10tanh iT xx

• Linearly Separable Data

• Nonlinearly Separable Data

• Polynomial Mapping

• Classification Example

• Linearly Separable Data

• Nonlinearly Separable Data

• Polynomial Mapping

• Classification Example

• The goal is to classify the class of an iris given two features - pedal length and pedal width

• The goal is to classify the class of an iris given two features - pedal length and pedal width

• Nonlinear Equalization

• Text Categorization

• 3D Object Recognition

• Nonlinear Equalization

• Text Categorization

• 3D Object Recognition

• Output of a channel is used as the input of the classifier

• Channel output is transformed into a pattern space, which is mapped into a higher-dimensional feature space

• Classifier matches a delayed version of the original signal

• Output of a channel is used as the input of the classifier

• Channel output is transformed into a pattern space, which is mapped into a higher-dimensional feature space

• Classifier matches a delayed version of the original signal

• Utilizing Information Retrieval Theory, word stems are used as representation units

• Each distinct word corresponds to a feature, with the frequency of occurrence as its value

• To avoid an unnecessarily large number of features, only words of a threshold frequency are considered, and “stop-words” (“and”, “or”, etc.) are ignored

• Utilizing Information Retrieval Theory, word stems are used as representation units

• Each distinct word corresponds to a feature, with the frequency of occurrence as its value

• To avoid an unnecessarily large number of features, only words of a threshold frequency are considered, and “stop-words” (“and”, “or”, etc.) are ignored

• Many views of an object are given to the SVM

• Types of features selected:– Shape of the object – Color of the object – Shape and the color of the object

• Many views of an object are given to the SVM

• Types of features selected:– Shape of the object – Color of the object – Shape and the color of the object

• Support vector machines method is an efficient, robust method for classification

• Advantages:– Small number of adjustable parameters – Doesn’t require prior information or

heuristic assumptions – Train with relatively small amounts of data

• Support vector machines method is an efficient, robust method for classification

• Advantages:– Small number of adjustable parameters – Doesn’t require prior information or

heuristic assumptions – Train with relatively small amounts of data