Linear Programming: Data Fitting

Steve Gu

Mar 21, 2008

Outline

• Data fitting (Ref: pp 250)

• Data fitting using LP

Data fitting

X 1 2 3Y 2 5 8

0

1

2

3

4

5

6

7

8

9

0 0.5 1 1.5 2 2.5 3 3.5 X

Y

Data fitting

• How to fit the model to the following data set using Chebyshev criterion?

• Chebyshev’s criterion: minimize the largest absolute deviation:

y cx

X 1 2 3Y 2 5 8

( )i i ir y y x

Chebyshev Criterion

max ( )

: minimize

i i iy y x

Goal

A min max problem!

More Formal Statement

X 1 2 3Y 2 5 8

max

: min r

i i ir y cx

Goal

subject to:(2 ) 0

(2 ) 0

(5 2 ) 0

(5 2 ) 0

(8 3 ) 0

(8 3 ) 0

r c

r c

r c

r c

r c

r c

Interpret Geometrically

1 2 3 4 5 c

r

2

4

6

8

(2 ) 0

(2 ) 0

(5 2 ) 0

(5 2 ) 0

(8 3 ) 0

(8 3 ) 0

r c

r c

r c

r c

r c

r c

Feasible region

Interpret Geometrically

1 2 3 4 5 c

r

2

4

6

8

Feasible region

Optimal solution

(8 3 ) 0

(2 ) 0

2.5, 0.5

r c

r c

c r

Therefore, 2.5 is the optimal value of c and the residual error is 0.5

More general problem

• Given N points (X1,Y1),(X2,Y2),…,(XN,YN), How to fit the model y=cx using Chebyshev criterion?

max ( )

: minimize

i i iy y x

Goal

Data fitting using Linear Programming

min r, where r=maxi i iy cx

• What’s the Goal?

• What are the Unknowns?– r and c

• What are the constraints?– Compare to 3-points case:

(2 ) 0

(2 ) 0

(5 2 ) 0

(5 2 ) 0

(8 3 ) 0

(8 3 ) 0

r c

r c

r c

r c

r c

r c

Data fitting using Linear Programming

1 1

1 1

2 2

2 2

1 1

1 1

( ) 0

( ) 0

( ) 0

( ) 0

...

( ) 0

( ) 0

( ) 0

( ) 0

N N

N N

N N

N N

r y cx

r y cx

r y cx

r y cx

r y cx

r y cx

r y cx

r y cx

1 1

1 1

2 2

2 2

1 1

1 1

...

N N

N N

N N

N N

r x c y

r x c y

r x c y

r x c y

r x c y

r x c y

r x c y

r x c y

Data fitting using Linear Programming

1 1

1 1

2 2

2 2

1 1

1 1

...

N N

N N

N N

N N

r cx y

r cx y

r cx y

r cx y

r cx y

r cx y

r cx y

r cx y

1 1

1 1

1

1

1

1

N N

N N

x y

x y

r

c

x y

x y

Ax b

Data fitting using Linear Programming

1 1

1 1

1

1

, ,

1

1

1:

0

N N

N N

x y

x y

rA b x

c

x y

x y

Define f

constraints:

objective: max T

Ax b

f x

LP in MATLAB

• Use command:

– linprog

– Do type “help linprog” !

• Example:

– X=linprog(f,A,b) solves the linear programming problem: min f'*x subject to: A*x <= b

LP in MATLAB

Results: Fitting data using LP

Results: Fitting data using LP

c=1.0039r=18.1304

End: Q&A

Thanks