Chapter 3

AlgorithmsComplexity AnalysisSearch and Flow Decomposition Algorithms

2

Terminology

Problem: Class of models characterized by the mathematical form of objective function and constraints (e.g., linear program, network flow, shortest path, traveling salesman)

Algorithm: Sequence of steps to solve a problem (e.g., simplex method, interior point method)

Instance: Particular set of numerical data for a problem

1 2

1 2

1 2

1 2

max 3 2

s.t. 5 4 2

0

, 0

x x

x x

x x

x x

3

Terminology (cont.)

Solves: An algorithm solves a problem if it is guaranteed to find a solution to any instance of the problem.

Size: (of an instance) is an indication of the amount of space required to store all the data – for a network flow problem this is typically specified by

Estimates of the time it takes an algorithm to solve a problem

Empirical: How long it would take for typical, practical instances

Average-case: Expected time over all possible instances

Worst-case: Upper bound on the time required for any instance

Typically expressed by number of “steps” in terms of the instance size.

and n N m A

4

Worst-Case Complexity

An algorithm is said to run in O(f(n)) time if for some numbers c and n0, the time taken by the algorithm is at most cf(n) for all n n0.

• Measures the asymptotic growth rate of the running time

• Indicates only the most dominant term in the running time formula

• Ignores constants: assumes that each elementary mathematical operation takes a constant amount of time and we don’t care what that constant is!

5

Polynomial vs. Exponential Time

An algorithm is generally considered “good” if its worst-case complexity is a polynomial function of n and m

If the algorithm’s worst-case running time grows as a function that cannot be polynomially bounded, it is an exponential-time algorithm

Note that for any constants a and b, if n is sufficiently large then

Exponential-time algorithms can solve only small instances!

2 2 153,274, , log , log ,n nm n nm n m m

log2 , !,n nn n

n ba n

6

Comments• Polynomial-time algorithms are considered efficient while

exponential-time algorithms are considered inefficient; but• Worst-case complexity comparison may not reveal much

about the average case– Simplex method is exponential while Khachian’s ellipsoid algorithm

for LP is polynomial– But on typical problems, simplex is much faster!

• The class of problems– with polynomial-time solution algorithms is known as , – for which a given solution can be judged optimal or not in

polynomial time is , – for which even this recognition is impossible in polynomial time are

-complete.

• If you can show your problem is -complete, everyone will agree that it’s hard!

7

A Search Algorithm

Problem: Given a network G = (N, A), and a designated source node, s, find all the nodes that are reachable from s along directed paths.

During the algorithm, each node is either marked, i.e., known to be reachable, or unmarked – if its reachability is as yet unknown. An arc (i, j) is admissible if i is marked and j is unmarked.

When the procedure marks a new node, j, by examining an admissible arc (i, j), node i is called the predecessor of node j. A search tree is defined by the predecessors.

The algorithm stops when there are no admissible arcs. It will have traversed the marked nodes in a certain order

8

Data Storage and Assumptions

Assume that we have available the adjacency list A(i) of arcs emanating from each node i. The arcs are listed in A(i) in increasing order of their head nodes. The algorithm examines the arcs in A(i) in that sequence.

Refer to algorithm search in Figure 3.4.

If the LIST of marked nodes yet to be fully examined is a

- (FIFO) queue, it is a breadth-first search

- LIFO queue or stack, it is a depth-first search.

9

Topological Ordering

A labeling of nodes is a topological ordering if for every arc (i, j), i < j.

A directed network can be topologically ordered if and only if it is acyclic.

A topo ordering algorithm either finds such an ordering or detects a directed cycle.

Refer to algorithm topological ordering in Figure 3.8.

This time, LIST refers to the set of nodes with zero indegree; different “queue disciplines” for the LIST result in different topological orderings.

10

Flow Decomposition

Flows are usually defined on arcs: xij is the flow along (i, j).

Alternatively, they can be defined on paths and cycles. To see how to translate between the two descriptions, define an arc flow as a set of flows {xij} that satisfies:

Note that this is not necessarily a feasible solution for given node supplies and demands. The quantity e(i) is node i’s imbalance (excess if positive, deficit if negative) with these flows. If e(i) = -b(i) for all i, then the flows are feasible.

, for all ,

0 , for all ,

ij jij j

ij ij

x x e i i N

x u i j A

11

Path/Cycle Flows to Arc Flows

Let be the set of all paths and be the set of all cycles in the network. A path and cycle flow formulation has decision variables f(P) = the flow on path P, P , and f(W) = the flow on cycle W, W . For a path P, let

and define ij(W) similarly for a cycle W.

Then the total flow on arc (i, j) is given by:

1 if arc , is included in path

0 otherwiseij

i j PP

ij ij ijP W

x P f P W f W

P W

12

Arc Flows to Path/Cycle Flows

Flow Decomposition Theorem: Every path and cycle flow has a unique representation as nonnegative arc flows. Conversely, every nonnegative arc flow x can be represented as a path and cycle flow (not necessarily unique) such that:

(a) Every directed path with positive flow connects a deficit node to an excess node.

(b) At most n+m paths and cycles have nonzero flow; out of these, at most m cycles have nonzero flow.

We just “proved” the first statement. The proof of the converse is really the algorithm to convert arc flows to path/cycle flows.

13

Flow Decomposition AlgorithmFirst, find the path flows:

Repeat

0. Find a deficit node i0, set k := 0.

1. Find an arc (ik, ik+1) that carries a positive flow.

2. If ik+1 is not an excess node and not previously visited then set k := k+1 and return to step 1, else stop with a path P or cycle W from i0 to ik+1 with positive flow along each arc. For a path P, let

for all nodes ik and arcs (i, j) along P. For a cycle W, let

for all arcs (i, j) included in W.

Until all node imbalances are zero.

0

0 0

min , ,min : , ,

, ,

k ij

k k ij ij

f P e i e i x i j P

e i e i f P e i e i f P x x f P

min : , ,ij ij ijf W x i j W x x f W

14

Flow Decomposition Algorithm (cont.)Second, find any remaining cycle flows:While x > 0 begin

0. Set k := 0. Find a node i0 with at least one outgoing arc (ik, ik+1) with a postive flow.

1. If ik+1 has not been visited previously then set k := k+1 and return to step 1, else stop with a cycle W with positive flow along each arc. Let

for all arcs (i, j) included in W.end.The total flow is the sum of flows on the paths and cycles

identified. Algorithm runs in O(nm) time.

min : , ,ij ij ijf W x i j W x x f W