Dual simplex method

Submitted by:

Pooja jain

M.com second semester

Group A roll no. 37

Linear Programming

During the Second World War, the soviet army was probably more efficient thanks to Leonid

Kantorovich. In 1939, this mathematician and economist came up with a new mathematical

technic to solve linear programs, hence improving plans of production at lower costs. In 1975, he

became the only Nobel Prize winner in Economy from the USSR. After the war, methods of

linear programming kept secret got published, including the simplex method by the American

mathematician George Dantzig and the duality theory by the Hungarian-American

mathematician John von Neumann.

Applications in company daily planning rose, increasing efficiency thanks to linear programs.

Applications also include transport, supply chains, shift scheduling, telecommunications, yield

management, assignment, set partitioning, set covering, advertisement. I’ve also heard of DNA

analysis, and I personally use it for cake-cutting.

Let’s be smart robbers

Suppose we have just attacked a village and we want to steal as much as we can. There’s gold

and dollar bills. But we only have one knapsack with a limited volume and we know we can’t

take it all. So we’ll have to choose the volumes of each thing to steal. Also assume that we are

not strong enough to carry too much weight (I do play sports but I don’t go to the gym, so I have

a weak upper body…).

You face what is called a mathematical program or an optimization problem. You need to

maximize the value of your robbery by choosing volumes of gold and 1 dollar bills

with constraints on total volume and total weight. One volume of gold is worth more than one

volume of dollar bills but it’s also heavier.

The value of the robbery is called the objective function. It’s what we want to maximize or

minimize. Volumes of gold and bills are called the variables. Mathematically, we usually write

the mathematical program as follows.

Of course, we usually have complicated notations instead of words… But we’ll get to that!

So far, what we have here is a mathematical program. There are plenty of methods to solve a

mathematical program but none of them can guarantee for sure that we find the solution in a

reasonable time, especially if the number of variable is higher than 10. But in our case, we can

write our mathematical program as a linear program.

A linear program is a mathematical program with a linear objective function and linear

constraints.

In our case, the objective function is linear since, whenever you add a volume of any product, the

total value increases by the value of that volume of that product. That means that we can write

the objective function in the following way:

A linear constraint is a constraint that says that a linear function of variables must be higher,

lesser or equal to a constant. Now, our first constraint says that the sum of all volumes must be

less than some constant limit. This is obviously a linear constraint. Similarly, because the weight

of volumes is the sum of weights of each volume, the total weight is also a linear function.

Hence, the second constraint is also linear: our program is linear.

Cool! Are we done writing our linear program?

Almost. We forgot two constraints. So far volumes are assumed to be real numbers, but they

actually need to be non-negatives. I mean stealing -1 litre of gold… that has no meaning. So we

just need to add constraints that say that volumes are positive. Yet, these are linear constraint

too! So our program is totally linear:

Our problem has two variables. Now any two variables will not necessarily satisfy all the

constraints, in which case we will say that the solution is infeasible. For instance, if the volume

of gold is -1 and the volume of bills is 0, then the solution is infeasible because the

constraint is not satisfied. If our program had no constraints, any solution

would be feasible, but constraints modify the feasible set.

What’s interesting is how each constraint modify the feasible set. As a matter of fact, each

inequality constraint divides the space into two areas, one for solutions satisfying the constraint,

the other for solutions that don’t satisfy the constraint.

The feasible set is the white area. We can see

that they we have a finite number of extreme

points and that we have the other points inside

are points in-between the extreme points. A

theorem says that there always exists an

extreme point that is one of the points that

maximize the objective function.

It’s possible that other points also maximize

the objective function. Still, it means that if

we know all the extreme points of our feasible

set, we can easily solve our problem by testing

them all! And that’s not all! Any mathematical

program can be associated with a dual

program, which is another mathematical program that has interesting properties related to the

initial mathematical program. In the case of linear programming, the dual program has even

more interesting properties and many reasoning in this dual program enable major

improvements.

Duality of LP problems

Each LP problem (called as Primal in this context) is associated with its counterpart known as

Dual LP problem. Instead of primal, solving the dual LP problem is sometimes easier when a)

the dual has fewer constraints than primal (time required for solving LP problems is directly

affected by the number of constraints,) and b) the dual involves maximization of an objective

function (it may be possible to avoid artificial variables that otherwise would be used in a primal

minimization problem).

Duality in Linear Programming

In order to explain duality to, continuing with the example of the smart robber. Basically, the

smart robber wants to steal as much gold and dollar bills as he can. He is limited by the volume

of his backpack and the maximal weight he can carry. Now, let’s notice that we can write the

problem as follows.

The problem we have written here is what we call the primal linear program. The dual program

will totally change our understanding of the problem, and that’s why it’s so cool.

In the primal program, constraints had constant numbers on their right. These constant numbers

are our resources. They say what we are able to do concerning each constraint. The dual problem

consists in evaluating how much our combined resources are worth. If the offer meets the

demand, our resources will be just as much as their potentials, which is the worth of the robbery.

Let’s go more into details. In the dual problem we will attribute values to the resources (as in

“how much they’re worth”). These values are called the dual variables. In our case, we have two

constraints, so we will have 2 dual variables.

The first dual variable, let’s call it ValueVolume refers to the value of one unit of volume. The

second dual variable refers to the value of one unit of weight. Value Weight seems like a right

name for it.

Now, we can write the value of the robbery with these two new variables… Let’s see if we get

the same result.

But how are the values per volume and per weight determined?

If I wanted to sell my resources, potential buyers are going to minimize the value of my

resources. So their valuations are the minimum of the total value. But as a seller, I will argue that

each of my resource is worth a lot, because it enables the robbery of more gold and more bills.

Obviously, the values of resources depend on the actual values of gold and bills per volume.

Let’s have a thought about the value of gold (and then we’ll be able to apply the same reasoning

to bills). If the constraints enabled us to steal one more volume of gold, then incremental value of

the robbery would be at least the value of this one volume of gold, right? It could be more, if we

use the new constraints to steal something else than gold that’s worth more. What I’m saying is

that, if the total volume enabled us to steal one more unit of volume of gold, and if we could

carry one more unit of weight of one volume of gold, then the value of this incremental steal

would be at least the value of one more volume of gold. Let’s write it.

From this we deduce a constraint on dual variables. As we can see, any variable in the primal

problem is associated with a constraint in the dual problem and vice-versa.

We are almost done. Let’s notice the fact that if we increase the total volume, then we have more

possibilities for the primal variables, which are the volumes of stolen gold and bills. Therefore,

the value of a unit of volume cannot be negative. This adds two more constraints on the sign of

dual variables. Now, we’re done and we can write the dual problem.

The dual program of a linear program is a linear program! Let’s have a look at the feasible set.

It’s important to notice that the information

of the primal objective functions appear in

the dual feasible set. The most important

result is the strong duality property: optimal

values of the primal and dual problems

are the same. We can solve the primal

problem simply by solving the dual

problem! And sometimes, the dual problem

can be much more simple than the primal

one.

Primal-Dual relationships

Following points are important to be noted regarding primal-dual relationship:

Primal Dual

Maximization Minimization

Minimization Maximization

Ith

variable ith

constraint

jth

constraint jth

variable

xi >=0 Inequality sign of Ith

constraint:

<= if dual is maximization

>= if dual is minimization

Ith

variable unrestricted Ith

constraint with = sign jth

constraint with = sign jth

variable unrestricted

RHS of jth

constraint Cost coefficient associated with jth

variable in the objective function

Cost coefficient associated with ith

variable in the objective function

RHS of ith

constraint

Primal and Dual Simplex Methods

We’ll consider the smart robber problem, that is we can steal bills or gold, but the total stolen

volume is limited, as well as the total weight stolen.

The simplex method

Recall that, in a linear program, there is necessarily an extreme point that is the optimum.

So we could just list all the extreme points and choose the best one…

There are two problems with listing all the extreme points. First, there can be a lot of them,

especially when considering problems with millions of variables and hundreds of thousands

constraints. Second, finding an extreme point can be quite difficult as it involves solving a

system with all the constraints that may not lead to a feasible solution. Shortly said, listing all the

extreme points is a bad idea.

What the simplex method does is moving from extreme points to strictly better extreme

points until finding an optimal extreme points. This simple idea is considered as one of the

main breakthroughs of the 20th century in

computing!

At any extreme point, the simplex method seeks

for a next feasible constraint base to visit, which is

strictly better than the current constraint base. In

order to do that, it looks at the possible direction

where it can go to reach a next extreme point. The

figure shows what the simplex method would do in

our case:

The simplex method has become famous and has

been used a lot as it enabled the resolution of

problems with millions of variables and hundreds

of thousands of constraints in reasonable time. However, it faces problems in cases of

degeneracy.

Solving an example with simplex method:

Pancakes

3 cups Bisquick

1 cup Milk

2 Eggs

Serves: 6

Waffles

2 cups Bisquick

2 cup Milk

2 Eggs

Serves: 5

You have 24 cups of Bisquick, 18 cups of milk, and 20 eggs. If you want to feed as many people

as possible, how many batches of each should you make?

Step 1

All information about example

Resource Pancakes (p) Waffles (w) Constraints

Bisquick (cups) 3 2 24

Milk (cups) 1 2 18

Eggs 2 2 20

Serves 6 5

Objective Function Z= 6p+5w

Bisquick Constraint 3p+2w ≤ 24

Milk Constraint p+2w ≤ 18

Egg Constraint 2p+2w ≤ 20

Non-negativity conditions p,w ≤ 0

The first step of the simplex method requires that each inequality be converted into an equation.

”less than or equal to” inequalities are converted to equations by including slack variables.

Suppose S1 bisquick cups, S2 milk cups and S3 eggs remain unused in a week. The constraints

become;

3p+2w+S1 = 24

p+2w+S2 = 18

2p+2w+S3 = 20

As unused hours result in no profit, the slack variables can be included in the objective function

with zero coefficients:

Z= 6p+5w+0S1+0S2+0S3

The problem can now be considered as solving a system of 3 linear equations involving the 5

variables in such a way that Z has the maximum value;

Now, the system of linear equations can be written in matrix form. The initial tableau is;

Step 2

Table 1

Cj

6 5 0 0 0

Basic

variable Solution P w S1 S2 S3

0 S1 24 3 2 1 0 0

0 S2 18 1 2 0 1 0

0 S3 20 2 2 0 0 1

Zj 0 0 0 0 0 0

Cj-Zj - 6 5 0 0 0

Step 3

Select the pivot column (determine which variable to enter into the solution mix). Choose the

column with the “most positive” element in the objective function row.

Table 2

Cj

6 5 0 0 0

Basic

variable Solution P w S1 S2 S3

0 S1 24 3 2 1 0 0

0 S2 18 1 2 0 1 0

0 S3 20 2 2 0 0 1

Zj 0 0 0 0 0 0

Cj-Zj - 6 5 0 0 0

p should enter into the solution mix because each batch of p (pancake) serves 6 compared with

only 5 for each batch of w (waffle).

Step 4

Select the pivot row (determine which variable to replace in the solution mix). Divide the

solution element in each row by the corresponding element in the pivot column. The pivot row is

the row with the smallest non-negative result.

Table 3

Cj

6 5 0 0 0

Basic

variable Solution P w S1 S2 S3 Ratio

0 S1 24 3 2 1 0 0 8

0 S2 18 1 2 0 1 0 18

0 S3 20 2 2 0 0 1 10

Zj 0 0 0 0 0 0

Cj-Zj - 6 5 0 0 0

Pivot column

Pivot row

and 3 is pivot

number

S1 Should be replaced by p in the solution mix. 10 pancakes can be made with 20 unused eggs

and 18 pancakes can be made with 18 unused milk cups but only 8 pancakes can be made with

24 bisquick cups. Therefore we decide to make 8 tables.

Now calculate new values for the pivot row. Divide every number in the row by the pivot

number.

Table 4

Cj

6 5 0 0 0

Basic

variable Solution P w S1 S2 S3

6 p 8 1 2/3 1/3 0 0

0 S2 18 1 2 0 1 0

0 S3 20 2 2 0 0 1

Zj 0 0 0 0 0 0

Cj-Zj - 6 5 0 0 0

Use row operations to make all numbers in the pivot column equal to 0 except for the pivot

number which remains as 1.

Table 5

Cj

6 5 0 0 0

Basic

variable Solution P w S1 S2 S3

6 P 8 1 2/3 1/3 0 0

0 S2 10 0 4/3 -1/3 1 0

0 S3 4 0 2/3 -2/3 0 1

Zj 48 6 4 2 0 0

Cj-Zj - 0 1 -2 0 0

If 8 pancakes are made, then the unused milk cups are reduced by 8 cups (1 cup/pancake

multiplied by 8 pancakes); the value changes from 18 cups to 10 cups. similarly, unused eggs are

reduced by 16 (2 egg/pancake multiplied by 8 pancakes); the value changes from 20 eggs to 4

eggs . Making 8 pancakes results in the service being increased by 48; the value changes from 0

to 48.

Now repeat the steps until there are no positive numbers in the last row.

Select the new pivot column. w should enter into the solution mix.

Select the new pivot row. S3 should be replaced by w in the solution mix.

Table 6

Cj

6 5 0 0 0

Basic

variable Solution P w S1 S2 S3 Ratio

6 P 8 1 2/3 1/3 0 0 12

0 S2 10 0 4/3 -1/3 1 0 15/2

0 S3 4 0 2/3 -2/3 0 1 6

Zj 48 6 4 2 0 0

Cj-Zj - 0 1 -2 0 0

Calculate new values for the pivot row. Divide every number in the row by the pivot number.

R1/3

R2 - R1

R3 -2 R1

Use row operations to make all numbers in the pivot column equal to 0 except for the pivot

number.

Table 7

Cj

6 5 0 0 0

Basic

variable Solution P w S1 S2 S3

6 P 4 1 0 1 0 -1

0 S2 2 0 0 1 1 -2

5 W 6 0 1 -1 0 3/2

Zj 54 6 5 1 0 3/2

Cj-Zj - 0 0 -1 0 -3/2

If 6 batches of waffles are made, then the number of pancakes is reduced by 4 of batches

pancakes (2/3 pancake/waffle multiplied by 6 batches of waffles); the value changes from 8

batches of pancakes to 4 batches of pancakes. The replacement of 4 batches of pancakes by 6 of

batches waffles results in the serving being increased by 8; the value changes from 48 to 54.

As the last row contains no positive numbers, this solution gives the maximum value of Z.

The dual simplex

I’m now going to explain what’s happening in the dual as we apply the simplex method to the

primal. This will help us define another way of solving linear programs, known as the dual

simplex. In order to have interesting things to talk about, we’ll assume that the optimal solution

is to steal bills only. This gives us the following primal-dual spaces.

We must choose the worst better dual value because we need to keep the dual reduced costs

positive (if the dual program is a minimization program), so that when we reach a feasible dual

base, we’ll be at the optimum for sure.

Each pivot in the dual space should be done as the one we just did. From the dual orange dot,

only the dual yellow constraint isn’t valid, so we have to get it into the dual constraint base,

which leads us to choose the worst dual value between the yellow and pink dots. The pink dot

has a worse dual value, so we pivot towards it.

Then, from the dual pink dot, only the blue constraint isn’t satisfied, so we’ll get it into the dual

constraint base. We then have to choose between the cyan and the white dots, but, as the white

dot has a worse dual value, we cannot go there. So we’ll go to the cyan dot. The cyan dot is

feasible, thus it is the dual optimum.

Dual Simplex Method

Computationally, dual simplex method is same as simplex method. However, their approaches

are different from each other. Simplex method starts with a non-optimal but feasible solution

whereas dual simplex method starts with an optimal but infeasible solution.

Simplex method maintains the feasibility during successive iterations whereas dual simplex

method maintains the optimality. Steps involved in the dual simplex method are:

The procedure for forming the dual problem is summarized below:

Formation of the dual problem

Given the maximization problem with the problem constraints,

Step1. Use the coefficients and constants in the problem constraints and the objective function to

form a matrix A with the coefficients of the objective function in the last row.

Step2. Interchange the rows and columns of matrix AT, the transpose of A.

Step3. Use the rows of AT to form a minimization problem with ≥ problem constraints.

Forming the dual problem

Primal problem in

algebraic form

Maximise Z= 6p+5w

Subject to

3p+2w ≤ 24

p+2w ≤ 18

2p+2w ≤ 20

And p,w ≥ 0

Primal problem

2 2 24

1 2 18

2 2 20

6 5 1

Dual problem

2 1 2 6

2 2 2 5

24 18 20 1

Dual problem in

algebraic form

Minimise

Z= 24x+18y+20z

Subject to

3x+y+2z ≥ 6

2x+2y+2z ≥ 5

And x,y,z ≥ 0

Step4. Modified problem, as in step 3, is expressed in the form of a simplex tableau.

Table 8

Cj

24 18 20 0 0 M M

Basic

variable Solution X Y Z S1 S2 A1 A2 Ratio

M A1 6 3 1 2 -1 0 1 0 2

M A2 5 2 2 2 0 -1 0 1 5/2

Zj 11M 5M 3M 4M -M -M M M

Cj-Zj - 24-5M 18-3M 20-4M M M 0 0

Step5. Selection of exiting variable: The basic variable with the highest negative value is the

exiting variable. If there are two candidates for exiting variable, any one is selected. The row of

the selected exiting variable is marked as pivotal row.

Step6. Selection of entering variable: The column corresponding to minimum ratio is identified

as the pivotal column and associated decision variable is the entering variable.

Step7. Pivotal operation: Pivotal operation is exactly same as in the case of simplex method,

considering the pivotal element as the element at the intersection of pivotal row and pivotal

column.

Step8. Check for optimality: If all the basic variables have nonnegative values then the

optimum solution is reached. Otherwise, Steps 3 to 5 are repeated until the optimum is reached.

Table 9

Cj

24 18 20 0 0 M

Basic

variable Solution X Y Z S1 S2 A2 Ratio

24 X 2 1 1/2 2/3 -1/3 0 0 4

M A2 1 0 1 2/3 2/3 -1 1 5/2

Zj 48+M 24 12+M 16+2/3M -8+2/3M -M M

Cj-Zj - 0 6-M 4-2/3M 8+2/3 M M 0

Table 10

Cj

24 18 20 0 0

Basic

variable Solution X Y Z S1 S2

24 X 3/2 1 0 1/3 -2/3 1/2

18 Y 1 0 1 2/3 2/3 -1

Zj 54 24 18 20 -4 -6

Cj-Zj - 0 0 0 4 6

Economic interpretation of the dual

The optimal solution to primal problem has already been found and presented in table 7. If we

compare table 7 and 10, we observe that objective functions of the two tables assume identical

values, i.e., 54. We also note that values in the opportunity cost row under column S1 and S2 of

the optimal table of the dual are the same as values under the “solution” column in the optimal

table of the primal problem. We further observe that magnitude of variables x and y are exactly

the same as the entries (with sign changed) in the opportunity cost row under columns S1 and S3

of the optimal table of the primal problem.

Advantages

Knowledge of dual is useful because solution of a linear programming problem may be easier to

obtain through the dual than through the primal problem. For example, consider a primal

problem involving three products all of which have to be processed on eight machines. The

initial simplex table for this problem will have eight rows. Where the same problem can be

solved via its dual, for which the initial table will have only three rows.

There’s something else quite interesting about duality: It gives directly a sensitivity analysis.

Consider our primal problems. It’d be interesting to know how much more I could get, had my

knapsack been larger or my body stronger. This looks like a difficult question in the primal

program, but it’s very obvious in the dual program. If I can have one unit of volume more in the

knapsack, then the incremental worth of the robbery will be Value Volume.

Modern advances in linear programming theory establish the use of the dual simplex algorithm

as a powerful optimization tool. While the performance of the dual simplex was originally

considered to lag that of its most popular primal variant, the dual simplex is widely used today in

practice. One of the more popular applications of the algorithm includes large-scale mixed

integer programming, where row additions break primal feasibility, but typically, dual feasibility

remains intact.

All these interesting properties of the method, coupled with the fact that linear optimization is a

fundamental topic in operations research, ensure that advances in this area will continue to be

positioned at the forefront of the discipline.