1

Lagrangian Duality in SVMSrikumar RamalingamComputer Vision Reading Group

Oxford University11 Jan 2008

Slides taken fromhttp://www.stanford.edu/~boyd/cvxbook/

http://www.rpi.edu/~bennek/

2

Overview

Introduction• Convex Sets and functions

Lagrangian Duality

Duality in SVM

3

Affine sets, Convex Sets, Convex Hulls,Hyperplane, Halfspaces, AffineFunctions…

4

5

6

7

8

9

10

11

12

Convex functions, First order andsecond order conditions, preservingoperations…

13

14

15

16

17

18

19

20

21

22

23

Lagrangian Duality

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

Support Vector Machines (SVM)

Key Ideas:• “Maximize Margins”• “Do the Dual”• “Construct Kernels”

A methodology for inference based on Vapnik’sStatistical Learning Theory.

64

Best Linear Separator?

65

Best Linear Separator?

66

Best Linear Separator?

67

Find Closest Points in ConvexHulls

c

d

68

69

70

Best Linear Separator:Supporting Plane Method

71

Maximize margin usingquadratic program

72

Standard primal SVM 2-normformulation (Vapnik, 1996)

73

Linearly Inseparable Case

Convex Hulls Intersect!

Same argumentwon’t work.

74

75

76

Classical SVM (Vapnik 1996)

77

78

Thank You