Module 2 Dynamic Programming

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

M2-1 © 2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

Module 2Module 2

Dynamic Dynamic ProgrammingProgramming

Prepared by Lee Revere and John LargePrepared by Lee Revere and John Large



Learning ObjectivesLearning Objectives

Students will be able to:

• Understand the overall approach of

dynamic programming.

• Use dynamic programming to solve the

shortest-route problem.

• Develop dynamic programming stages.

• Describe important dynamic programming

terminology.

• Describe the use of dynamic programming

in solving knapsack problems.



Module OutlineModule Outline

M2.1 Introduction

M2.2 Shortest-Route Problem

Solved by Dynamic

Programming

M2.3 Dynamic Programming

Terminology

M2.4 Dynamic Programming

Notation

M2.5 Knapsack Problem



Dynamic ProgrammingDynamic Programming

• Dynamic programming is a quantitative analytic technique applied to large, complex problems that have sequences of decisions to be made.

• Dynamic programming divides problems into a number of decision stages; • the outcome of a decision at one stage affects

the decision at each of the next stages.

• The technique is useful in a large number of multi-period business problems, such as • smoothing production employment,

• allocating capital funds,

• allocating salespeople to marketing areas, and

• evaluating investment opportunities.



Dynamic ProgrammingDynamic Programmingvs. vs.

Linear ProgrammingLinear Programming

Dynamic programming differs from Dynamic programming differs from linear programming in two ways:linear programming in two ways:

1. First, there is no algorithm (like the simplex method) that can be programmed to solve all problems.

• Instead, dynamic programming is a technique that allows a difficult problem to be broken down into a sequence of easier sub-problems,

which are then evaluated by stages.



2. Second, linear programming is a method that gives single-stage (i.e., one-time period) solutions.

• Dynamic programming has the power to determine the optimal solution over a one-year time horizon by breaking the problem into 12 smaller one-month horizon problems and to solve each of these optimally. Hence, it uses a multistage approach.

Dynamic ProgrammingDynamic Programmingvs. vs.

Linear ProgrammingLinear Programming



Four Steps in Dynamic Four Steps in Dynamic ProgrammingProgramming

1. Divide the original problem into subproblems called stages.

2. Solve the last stage of the problem for all possible conditions or states.

3. Working backward from that last stage, solve each intermediate stage.

4. Obtain the optimal solution for the original problem by solving all stages sequentially.



Solving Types of Dynamic Solving Types of Dynamic Programming ProblemsProgramming Problems

The next slides will show how to solvetwo types of dynamic programmingproblems:

1. network2. non-network

• The Shortest-Route Problem is a network problem that can be solved by dynamic programming.

• The Knapsack Problem is an example of a non-network problem that can be solved using dynamic programming.



SHORTEST-ROUTE SHORTEST-ROUTE PROBLEM SOLVED BY PROBLEM SOLVED BY

DYNAMIC PROGRAMMINGDYNAMIC PROGRAMMING

George Yates is to travel from Rice, Georgia (1) to Dixieville, Georgia (7).

George wants to find the shortest route. But • there are small towns between Rice

and Dixieville. • The road map is on the next slide.

The circles (nodes) on the map represent cities such as Rice, Dixieville, Brown, and so on. The arrows (arcs) represent highways between the cities.



Dynamic ProgrammingDynamic ProgrammingGeorge YatesGeorge Yates

Dixieville

1

4

3

2

5

6

7

4 m

iles

10 miles

14 miles

2 miles

10 miles

6 miles

4 m

iles

2 m

iles

12 m

iles

5 milesRice

Lakecity Athens

Hope Georgetown

Brown

Figure M2.1



SHORTEST-ROUTE SHORTEST-ROUTE PROBLEM SOLVED BY PROBLEM SOLVED BY

DYNAMIC PROGRAMMINGDYNAMIC PROGRAMMING

• The mileage is indicated along each arc.

• Can solve this by inspection, but it is instructive seeing dynamic programming used here to show how to solve more complex problems.



George Yates George Yates Dynamic Programming Dynamic Programming

Step-1: Step-1:

First, divide the problem into sub-problems or stages.

Figure M2.2 (next slide) reveals the stages of this problem.

In dynamic programming, we usually start with the last part of the problem, Stage 1, and work backward to the beginning of the problem or network, which is Stage 3 in this problem.

Table M2.1 (second slide) summarizes the arcs and arc distances for each stage.



George YatesGeorge YatesStagesStages

Dixieville

1

4

3

2

5

6

7

4 mile

s

10 miles

14 miles

2 miles

10 miles

6 miles

4 m

iles

2 m

iles

12 m

iles

5 miles

Rice

Lakecity Athens

Hope Georgetown

Brown

Stage 1Stage 2Stage 3Figure M2.2



Table M2.1: Distance Along Table M2.1: Distance Along Each ArcEach Arc

ARCSTAGE ARC DISTANCE 1 5-7 14 6-7 2 2 4-5 10

3-5 123-6 62-5 42-6 10

3 1-4 41-3 51-2 2

Table M2.1



Step 2: Solve The Last Step 2: Solve The Last Stage – Stage 1Stage – Stage 1

• Next, solve Stage 1, the last part of the network. This is usually trivial.

• Find the shortest path to the end of the network: node 7 in this problem.

• The objective is to find the shortest distance to node 7.



Step 2: Solve The Last Step 2: Solve The Last Stage – Stage 1 Stage – Stage 1 continuedcontinued

• At Stage 1, the shortest paths, from nodes 5 and 6 to node 7 are the only paths. • Also note in Figure M2.3 (next slide)

that the minimum distances are enclosed in boxes by the entering nodes to stage 1, node 5 and node 6.



George YatesGeorge YatesStage 1Stage 1

1

4

3

2

5

6

7

4 m

iles

10 miles

14 miles

2 miles

10 miles

6 miles

4 m

iles

2 m

iles

12 m

iles

5 miles

14

2



Step 3: Moving Backwards Step 3: Moving Backwards Solving Intermediate Solving Intermediate

ProblemsProblems

• Moving backward, now solve for Stages 2 and 3.

• At Stage 2 use Figure M2.4. (next slide)




1

4

3

2

5

6

7

4 mile

s

10 miles

14 miles

2 miles

10 miles

6 miles

4 m

iles

2 m

iles

12 mile

s

5 miles

2

1424

12

8



Fig M2.4 (previous Fig M2.4 (previous slide) Analysisslide) Analysis

• If we are at node 4, the shortest and only route to node 7 is arcs 4–5 and 5–7.

• At node 3, the shortest route is arcs 3–6 and 6–7 with a total minimum distance of 8 miles.

• If we are at node 2, the shortest route is arcs 2–6 and 6–7 with a minimum total distance of 12 miles.

• The solution to Stage 3 can be completed using the network on the following slide.



1

4

3

2

5

6

7

4 mile

s

10 miles

14 miles

2 miles

10 miles

6 miles

4 m

iles

2 m

iles

12 m

iles

5 miles


2

14

12

24

813

Minimum Distance Minimum Distance to Node 7 to Node 7 from Node 1from Node 1



Step 4 : Final StepStep 4 : Final Step

• The final step is to find the optimal solution after all stages have been solved.

• To obtain the optimal solution at any stage, only consider the arcs and the optimal solution at the next stage.

• For Stage 3, we only have to consider the three arcs to Stage 2 (1–2, 1–3, and 1–4) and the optimal policies at Stage 2.



DYNAMIC PROGRAMMING DYNAMIC PROGRAMMING TERMINOLOGYTERMINOLOGY

1. Stage: a period or a logical sub-problem.

2. State variables: possible beginning situations or conditions of a stage. These have also been called the input variables.

3. Decision variables: alternatives or possible decisions that exist at each stage.

4. Decision criterion: a statement concerning the objective of the problem.



DYNAMIC PROGRAMMING DYNAMIC PROGRAMMING TERMINOLOGY TERMINOLOGY continuedcontinued

5. Optimal policy: a set of decision rules, developed as a result of the decision criteria, that gives optimal decisions for any entering condition at any stage.

6. Transformation: normally, an algebraic statement that reveals the relationship between stages.



Shortest Route Problem Shortest Route Problem Transformation CalculationTransformation Calculation

Distance from the beginning of a given stage to the last node

Distance from the beginning of a given stage to the last node

==Distance from the beginning of the previous stage to the last node

Distance from the beginning of the previous stage to the last node

++

Distance from the given stage to the previous stage

Distance from the given stage to the previous stage

In the shortest-route problem, the following transformation can be given:



Dynamic Programming Dynamic Programming NotationNotation

• In addition to terminology, mathematical notation can also be used to describe any dynamic programming problem.

• Here, an input, decision, output and return are specified for each stage.

• This helps to set up and solve the problem.

• Consider Stage 2 in the George Yates Dynamic Programming problem first discussed in Section M2.2. This stage can be represented by the diagram shown in two slides - Figure M2.7 (as could any given stage of a given dynamic programming problem).



Input, Decision, Output,Input, Decision, Output,and Return for Stage 2 inand Return for Stage 2 inGeorge Yates’s ProblemGeorge Yates’s Problem

• Please note that the input to one stage is also the output from another stage. e.g., the input to Stage 2, s2, is also the output from Stage 3 (see Figure M2.7 in the next slide).

• This leads us to the following equation:

s n −1 = 1 output from Stage n

sn = input to stage n (M2-1)

dn = decision at stage n (M2-2)

rn = return at stage n (M2-3)



Input, Decision, Output,Input, Decision, Output,and Return for Stage 2 inand Return for Stage 2 inGeorge Yates’s ProblemGeorge Yates’s Problem

Fig M2.7

Stage 2Input s2

Decision d2

Output s1Return

r2



Transformation Transformation FunctionFunction

• A transformation function allows us to go from one stage to another.

• The total return function allows us to keep track of profits and costs.

tn = transformation function at Stage n (M2-5)

Sn-1 = tn(Sn, dn) (M2-6)

fn = total return at Stage n (M2-7)



Dynamic ProgrammingDynamic ProgrammingKey EquationsKey Equations

sn Input to stage n

dn Decision at stage n

rn Return at stage n

sn-1 Input to stage n-1

tn Transformation function at

stage n

sn-1 = tn [sn dn] General

relationship between stages

fn Total return at stage n



KNAPSACK PROBLEMKNAPSACK PROBLEM

• The “knapsack problem” involves the maximization or minimization of a value, such as profits or costs.

• Like a linear programming problem, there are restrictions.

• Imagine a knapsack or pouch that can only hold a certain weight or volume. We can place different types of items in the

knapsack.

• Our objective is to place items in the knapsack to maximize total value without breaking

the knapsack because of too much weight or a similar restriction.



Types of Knapsack Types of Knapsack ProblemsProblems

• There are many kinds of problems that can be classified as knapsack problems. e.g., Choosing items to place in the cargo

compartment of an airplane and selecting which payloads to put on the next

NASA space shuttle. • The restriction can be volume, weight,

or both.

• Some scheduling problems are also knapsack problems. e.g., we may want to determine which jobs

to complete in the next two weeks. The two-week period is the knapsack, and

we want to load it with jobs in such a way as to maximize profits or minimize costs.

The restriction is the number of days or hours during the two-week period.



Examples of Knapsack Examples of Knapsack ProblemsProblems

Salesman has to touch 16 locations without wasting extra time and traveling haphazardly.

Traveling Salesman Problem



Example 2: ScenarioExample 2: Scenario

• Consider packing a knapsack for a picnic in the country. There are many different items that you could bring, but the knapsack is not big enough to contain all items.

• Decide what is appropriate for the trip and leave the not so important things behind.

• Evaluate the importance of different items for this situation.

• Bringing food may be deemed very important and bringing a DVD as not important, even if it can yield a pleasant day.



Scenario Scenario continuedcontinued

• In theory, values must be assigned to each item. Items must be rated in terms of profit and cost.

• Profit would be the importance of the item, while the cost is the amount of space it occupies in the knapsack. That is a multi-objective problem.

• You want to maximize the profit while minimizing the total cost. • To maximize profit would mean taking all

the items. However, this would exceed the capacity of the backpack.

• To minimize cost, we would take none of the items, but this would mean that we have no profit. You have to find the best compromise.



Considered ExampleConsidered Example

Backpack capacity (max cost) = 10Items: Cost: Profit:Flan 5 12

Gyros 3 10

Flatware 2 7

Blanket 4 10

Cups 1 6

DVD 4 2



Examine Some Possible Examine Some Possible Solutions..Solutions..

Item Total Cost Total Profit

Gyros, Flan, Flatware

10 29

Gyros, Flan, Flatware, Blanket

14 39

Flan, Blanket, Cups

10 28

Gyros, Cups, Flatware, Blanket

10 33



More on Knapsack ProblemsMore on Knapsack Problems

• The knapsack problem concerns many situations of resource allocation with financial constraints, for instance, • to select what things we should buy, given

a fixed budget.

• Everything has a cost and a profit, so seek the most value for a given cost.

• The term knapsack problem invokes the image of the backpacker who is constrained, by a fixed-size knapsack, to fill it only with the most useful items.



Knapsack Problem in Knapsack Problem in GraphicsGraphics



GLOSSARYGLOSSARY

• Decision Criterion. A statement concerning the objective of a dynamic programming problem.

• Decision Variable. The alternatives or possible decisions that exist at each stage of a dynamic programming problem.

• Dynamic Programming. A quantitative technique that works backward from the end of the problem to the beginning of the problem in determining the best decision for a number of interrelated decisions.



Glossary Glossary continuedcontinued

Optimal Policy. A set of decision rules, developed as a result of the decision criteria, that gives optimal decisions at any stage of a dynamic programming problem.

Stage. A logical sub-problem in a dynamic programming problem.

State Variable. A term used in dynamic programming to describe the possible beginning situations or conditions of a stage.

Transformation. An algebraic statement that shows the relationship between stages in a dynamic programming problem.

Documents

Module 2 Dynamic Programming