Support-Vector NetworksC Cortes and V Vapnik

12.04.26.(Tue)Computational Models of Intelligence

Joon Shik Kim

Introduction• The support-vector network is a new

learning machine for two-group clas-sification problems.

• Input vectors are non-linearly mapped to a very high dimension feature space.

• In this feature space a linear decision surface is constructed.

Graphical Description of SVM

Optimal Hyperplane Algorithm (1/2)

• The set of labeled training patterns

is said to be linearly separable if there exists a vector w and a scalar b such that the inequalities if if

1 1( , ),..., ( , ),l ly yx x { 1,1}iy

1b w x 1,iy 1i b w x 1.iy

Optimal Hyperplane Algorithm (2/2)

• The optimal hyperplane

• Distance is given by

0 0 0b w z

1 1min max| | | |y y

x w x ww w

0 00 0 0

2 2( , )| |

bw

ww w

Lagrangian (1/2)

1

1( , , ) [ ( ) 1]2

l

i i ii

L b y b

w w w x w

0 01

( , , ) | ( ) 0,l

i i ii

L b y

w ww w xw

0

( , , ) | 0.i

b b i iL b y

b

w

Lagrangian (2/2)

0 01

1( )2

l

ii

W

w w

1 1 1

1 .2

l l l

i i j i j i ji i j

y y

x x