Source-Destination RoutingOptimal Strategies

Eric ChiEE228a, Fall 2002

Dept. of EECS, U.C. Berkeley

Basic Routing Problem

• Network with links of finite capacity• Connection requests for various node-pairs arrive one by

one• A decision is made to either

– deny the request or– admit the connection along a given route

• An admitted call simultaneously holds some capacity along all links along the route for some amount of time before departing

• Objective: Make decisions that minimize blocking probability

Approaches

• Suboptimal: Greedy algorithms– Always admit if there is space.– Choose good heuristics for where to place calls.

• Maximize spare capacity• Minimize “Interference”

• Optimal: Dynamic programming– Balances

• Immediate gains• Long term opportunity costs

Markov Decision Process

• State specified by a Markov Chain– Request arrivals are Poisson

– Calls holding times are exponentially distributed

• Rewards (Costs) associated with– Residing in a state

– Making a transition

• Transition probabilities depend on policies for a given state.

Discrete Time MDP

Bellman Principle of Optimality

• Given an optimal control for n steps to go, the last n-1 steps provide optimal control with n-1 steps to go.

• Example: Dijstkra’s Shortest Path Algorithm

Solving MDPs: Value Iteration

• Solve the fixed point equation.

Then

Solving MDPs: Policy Iteration

Example: Symmetric

Y/C

X/C

’

• Optimal Policy: Route to least loaded

Proof (Sketch)

• Prove that load balancing is optimal for any finite time to go n. (Monotone convergence allows us to take the limit.)

• Prove inductively that for all n, , a

Example: Unbalanced

Y/C

X/C

2

Example: Unbalanced

Y/C

X/C

’

• Optimal Policy: Route to lower link until full. If full route to top link.

Comparison

Example: Alternate Routing

• Policy A: Route up 1st, Route down 2nd

• Policy B: Route down 1st, Route up 2nd

Y/C

X/C

2

Comparison

• Two policies

Literature

• K. R. Krishnan and T. J. Ott, "State-dependent routing for telephone traffic: theory and results," in 25th IEEE Control and Decision Conf., Athens, Greece, Dec. 1986, pp. 2124-2128.

• A. Ephremides, P. Varaiya, and J. Walrand. A simple dynamic routing problem. IEEE Transactions on Automatic Control, 25(4):690-693, August 1980.

• R.J. Gibbon and F.P. Kelly. Dynamic routing in fully connected networks. IMA journal of Mathematical Control and Information, 7:77--111, 1990.

• Marbach, P., Mihatsch, M., Tsitsiklis, J.N., "Call admission control and routing in integrated service networks using neuro-dynamic programming ," IEEE J. Selected Areas in Comm., v. 18, n. 2, pp. 197--208, Feb. 2000.

• K. Kar, M. Kodialam, and T.V. Lakshman, “Minimum Interference Routing of Bandwidth Guaranteed Tunnels with Applications to MPLS Traffic Engineering,” IEEE JSAC, 1995, Special Issue on Advances in the Fundamentals of Networking, pp. 1128-36.