Bounded variables new

1

Outline relationship among topics secrets LP with upper bounds

by Simplex method basic feasible solution (BFS)

by Simplex method for bounded variables extended basic feasible solution (EBFS)

optimality conditions for bounded variables ideas of the proof

examples Example 1 for ideas but inexact Example 2 for the exact procedure

2

A Depot for Multiple Products multi-product by a fleet of trucks

depot

Possible Formulation: objective function

common constraints, e.g., trucks, DC capacity, etc.

network constraints for type-1 product



....

non-negativity constraints

3

A General Type of Optimization Problems

structure of many problems: network constraints: easy other constraints: hard

making use of the easy constraints to solve the problems solution methods: large-scale optimization

column generation, Lagrangian relaxation, Dantzig-Wolfe decomposition …

basis: linear programming, network optimization (and also non-linear optimization, integer optimization, combinatorial optimization)

objective function

network constraints

non-negativity constraints

hard constraints

4

Relationship of Solution Techniques

two directions of theoretical development for network programming from special structures of networks from linear programming

ideal: understanding development in both directions

linear prog.

network prog.

int. prog.non-linear prog.dynamic prog.

…

5

Relationship of Solution Techniquesminimum cost flow column generation, Dantzig-

Wolfe decomposition

Lagrangian relaxation

network algorithms

network simplex

shortest-path algorithms

simplex method

revised simplex method

non-linear optimization

linear algebra

6

Our Topics

simplex method for bounded variables linkage between LP and network simplex optimality conditions for minimum cost flow networks

minimum cost algorithms standard, and successive shortest path equivalence among network and LP optimality conditions

revised simplex column generation Dantzig-Wolfe decomposition Lagrangian relaxation

It takes more than one semester to cover these

topics in detail! We will only cover the ideas.

7

Secrets

8

The Most Beautiful …

linear algebra

9

Maybe the Most Beautiful of All…

algebraic properties

geometric properties

matrix properties

10

LP with Upper Bounds

upper bounds: common in network problems, e.g., an arc with finite capacity

quite some theory of network optimization being from LP

11

LP with Upper Bounds

Tmax. .s t

c xAx b

0 x u

incorporate the upper-bound constraints into the set of functional constraints and solve accordingly

12

To Solve LP with Upper Bounds

Tmax. .s t

c xAx b

0 x u

Tmax

. .s t

c xA b

xI u

0 x

In the simplex method the lower bound constraints 0 x do not appear in A.

Is it possible to work only with A even with upper-bound constraints?

Yes.

13

To Solve LP with Upper Bounds

Tmax. .s t

c xAx b

0 x u

Tmax

. .s t

c xA b

xI u

0 x

Amn, m n, of rank m basic feasible solution (BFS) x of LP, i.e.,

feasible: Ax b, 0 x basic

non-basic variables: (at least) n-m variables = 0 basic variables: m non-negative variables with linearly

independent columns

14

BFS for Standard LP Tmax. .s t

c xAx b0 x

Amn, m n, of rank m extended basic feasible solution ( EBFS ) x of LP

with bounded variables, i.e., feasible: Ax b, 0 x u basic solution

non-basic variables: (at least) n-m variables = 0, or = their upper bounds

Basic variables: m variables of the form 0 xi ui, with linearly independent columns

15

Extended Basic Feasible Solution of LP with Bounded Variables

Tmax. .s t

c xAx b

0 x u

Maximum Conditions: BFS x is maximal if 0 for all non-basic variable xj = 0

Minimum Conditions: BFS x is minimal if 0 for all non-basic variable xj = 0

intuition : increase of the objective function by unit increase in xj

maximum condition: no good to increase non-basic xj

minimum condition: no good to decrease non-basic xj

16

Optimality Conditions of Standard LP

jc

jc

jc

Maximum Conditions: EBFS x is maximal if 0 for all non-basic variable xj = 0, and

0 for all non-basic variable xj = uj

Minimum Conditions: EBFS x is minimal if 0 for all non-basic variable xj = 0, and


17

Optimality Conditions of LP with Bounded Variables

jcjc

jc

jc

18

How to Prove?

optimality conditions of the EBFS from duality theory and complementary slackness

conditions

19

General Idea

primal-dual pair

Theorem 1 (Complementary Slackness Conditions) if x primal feasible and y dual feasible then x primal optimal and y dual optimal iff

xj(yTAjcj) = 0 for all j, and yi(biAix) = 0 for all i

20

Complementary Slackness Conditions

Tmax. .s t

c xAx b0 x

T

T T

min

. .s t

b y

y A cy

primal-dual pair

Theorem 2 (Necessary and Sufficient Condition) if x primal feasible then x primal optimal iff there exists dual feasible

y such that x and y satisfy the Complementary Slackness Conditions

21

Complementary Slackness Conditions

Tmax. .s t

c xAx b0 x

T

T T

min

. .s t

b y

y A cy

by Theorem 2, primal feasible x and dual feasible (yT, T) are optimal iff xj(yTAj + j - cj ) = 0, j

yi(bi - Aix) = 0, i

j(uj - xj ) = 0, j

22

Complementary Slackness Conditions for LP with Bounded Variables

Tmax. .s t

c xAx bx u0 x

T T

T T T

min

. .s t

b y + u

y A + cy

optimality conditions of the EBFS from duality theory and complementary slackness

conditions ideas of the proof

given an EBFS x satisfying the upper-bound optimality conditions

then possible to find dual feasible variables (yT, T)T such that x and (yT, T)T satisfy the complementary slackness conditions

23

General Idea of the Proof

max 2x + 5y, min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

24

Example 1. Upper-Bound Constraints as Functional Constraints

25

Examples of LP with Bounded Variables

min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

max. value = 44 x* = 2 and y* = 8

26

Example 1. Upper-Bound Constraints as Functional Constraints

27

The following procedure is not exactly the Simplex Method for Bounded

Variables. It primarily brings out the ideas of the exact method.

y as the entering variable 2y + s1 = 20

y + s2 = 16 y 8

28

Example 1. Upper-Bound Constraints by Optimality Conditions of Bounded Variables

-5

min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

mark the non-basic variable y at its upper bound for y = 8

obj. fun.: -2x – 5y – z = 0 -2x - z = 40 eqt. (1): x + 2y + s1 = 20 x + s1 = 4

eqt. (2): 2x + y + s2 = 16 2x + s2 = 8

29


x as the entering variable x + s1 = 4

2x + s2 = 8 x 2

30


min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

for x at its upper bound 2, mark x, and obj. fun.: -2x – z = 40 -z = 44 eqt. (1): x + s1 = 4 s1 = 2

eqt. (2): 2x + s2 = 8 s2 = 4

31


min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

satisfying the optimality condition for bounded variables 0 for all non-basic variable xj = 0, and


z* = -44, with x* = 2 and y* = 832


jc

jc

in general, variables swapping among all sorts of status non-basic at 0 basic at 0 basic between 0 and upper bound basic at upper bound non-basic at upper bound

Simplex method for bounded variables: a special algorithm to record all possibilities

33

Example 1 Being Too Specific

34

The following example follows the exact procedure of the Simplex

Method for Bounded Variables.

max 3x1 + 5x2 + 2x3 min 3x1 5x2 2x3, s.t. x1 + x2 + 2x3 7, 2x1 + 4x2 + 3x3 15, 0 x1 4, 0 x2 3, 0 x3 3.

35

Example 2

potential entering variable: x2 bounded by upper bound 3 define = u2-x2 = 3-x2

36

Example 2 by Simplex Method for Bounded Variables

2x

min 3x1 5x2 2x3, s.t. x1 + x2 + 2x3 7, 2x1 + 4x2 + 3x3 15, 0 x1 4, 0 x2 3, 0 x3 3.

37


x1 as the (potential) entering variable

s2 as the leaving variable a pivot operation as in standard Simplex Method

38



which can be an entering variable? can s1 be a leaving variable? Yes

can x1 be a leaving variable? Yes39


2x


when = 1.25, x1 reaches its upper bound 4

replace x1 by and is a basic variable = 0 result

40


2x

1,x 1x

1 2 3 2

1 1 2 3 2

1 2 3 2 1

2 1.5 0.5 1.5( ) 2 1.5 0.5 1.5

2 1.5 0.5 1.5

x x x su x x x sx x x s u


. a “normal” pivot operation with aij < 0

41


2 1 entering and leaving x x


minimum z* = -20.75, x1

* = 4, x2* = 1.75, x3

* = 0

42



Technology

Bounded variables new