7/30/2019 A Support Vector Machines
1/2
A Support vector machines (SVMs) are a family of supervised
machine learning methods
developed in recent years. SVMs were first suggested by Vapnik
in the 1960s for classification.
The main idea behind the SVM algorithm is to map input vectors
into a feature space of higher
dimension, construct a linear decision surface (hyperplane), and
then optimize that hyperplane
for generalization. SVMs are used for classification,
regression, and ranking, and are used for
related tasks such information retrieval and optical character
recognition.The support vector machine (SVM)is a training algorithm
for learning classification and
regression rules from data. SVMs arose from statistical learning
theory; the aim being to solve
only the problem of interest without solving a more difficult
problem as an intermediate step.
SVMs are based on the structural risk minimization principle,
closely related to regularization
theory.
Two key elements in the implementation of SVM are the techniques
of mathematical
programming and kernel functions. The parameters are found by
solving a quadratic
programming problem with linear equality and inequality
constraints; rather than by solving a
non-convex, unconstrained optimization problem. The kernel forms
the basic parts of SVM and
the flexibility of kernel functions allows the SVM to search a
wide variety of hypothesis spaces.
The basic principles of an SVM are as follows1. The SVM performs
a nonlinear mapping of the input vectors (objects) from the input
space Rdinto a high dimensional feature space H; the mapping is
determined by a kernel function.
2. It finds a linear decision rule in the feature space H in the
form of an optimal separating plane,
which is the one that leaves the widest margin between the
decision boundary and the input
vectors mapped into H.
3. This plane is found by solving the following constrained
quadratic programming problem:
Maximize W
1 1 1
1,
2
n n n
i i j i j i j
i i j
y y K x x
under the constraints1
0n
i i
i
y
and
0 C for i=1,2,.., n wherei dx R are the training sample set
(TSS) vectors, and
1, 1iy the corresponding class labels. C is a constant needed
for non-separable classes;
,K u v is a kernel function.There are three types of support
vector machines:
Polynomial SVM:
, , 1d
K u v u v
This type of machine describes the mapping of an input space
into the feature space that
contains all products .....i i j j k k u v u v u v up to degree
d.
Radial Basis Function SVM:
2, expK u v u v
This type of machine forms a separation rule according to radial
hyperplanes.
Two-layer neural network SVM:
, tanh ,K u v w u v c This type of machine generates a kind of
two-layer neural network without back propagation.
