Putting BGP on the Right Path: A Case for Next-Hop

Routing

Michael Schapira

Joint work with Yaping Zhu and Jennifer Rexford

(Princeton University)

Once Upon a Time… Internet Inter-Network Routing: •Small network

•Single administrative entity NSFNET

•Shortest-path routing distance-vector routing

•Then....

Interdomain RoutingOver 35,000 Autonomous Systems (ASes)

Interdomain routing = routing between ASes– Border Gateway Protocol (BGP)

AT&T

Qwest

Comcast

Sprint

Today’s Path-Based Routing With BGP

•Complex!– configuration errors, software bugs, …

•Bad convergence!– persistent route oscillations, slow convergence, …

•Vulnerable to attacks!– malicious, economically-driven, inadvertent, …

•and more, and more, and more …– bad performance, clumsy traffic engineering, …

How Can We FixInterdomain Routing?

•One approach: add mechanisms to an already complex protocol– route flap damping, S-BGP, …

•Another approach: redesign interdomain routing from scratch– HLP, NIRA, pathlet routing, consensus routing, …

•Our approach: simplify BGP!

Background: Today’s Path-Based Routing With BGP

• AS i’s routing policy: ranking of simple routes from i to each

destination d export policy

• BGP is a path-vector protocolReceive

route updates from

neighbors

Choose single “best”route

(ranking)

Send route updates to neighbors

(export policy)

3 d

1

2

Background: Today’s Path-Based Routing With BGP

32d > 31d

Don’t export 2d

to 3

1, 2, I’m available

3, I’m using 1d

a stable state is reached

AS-PATH = the Route of All Evil

•AS-PATH: list of all ASes on path– originally meant for loop-detection

•The AS-PATH is to blame!– error-prone, software bugs– no/slow convergence– large attack surface– bad scalability, clumsy traffic engineering, bad

performance, …

Getting Off the AS-PATH•No way back to shortest-path

routing…

•Our proposal: next-hop routing–make routing decisions based solely on the “next hop”

–relegate the AS-PATH to its original role

Wish List• Loop freedom• Fast Convergence• Security• Incentive compatibility• Business policies• Good performance• Traffic engineering• Scalability• Simplicity

Expressiveness vs. Complexity

complexity

expressivenessnot expressive

enoughsufficientlyexpressive

extremelyexpressive

simple

too complex

shortest-pathrouting

next-hoprouting

BGP’spath-based

routing

Next-Hop Routing Rules!• Rule 1: use next-hop rankings

4 d

3

5

1

2

4 > 3541d > 53d > 542d

Next-Hop Routing Rules!• Rule 1: use next-hop rankings • Rule 2: prioritize current route

– to minimize path exploration [Godfrey-Caesar-Hagen-Singer-Shenker]

2

d

3

1

2=3Break ties in

favor of lower AS number

2=3Prioritize current route

Next-Hop Routing Rules!• Rule 1: use next-hop rankings • Rule 2: prioritize current route• Rule 3: consistently export

– to avoid disconnecting upstream nodes [Feigenbaum-S-Ramachandran]

3 d41

2

1 > 2,Export 32d, but not 31d,

to 4

1 > 2,Export 31d

to 4

Next-Hop Routing Rules!• Rule 1: use next-hop rankings • Rule 2: prioritize current route• Rule 3: consistently export

– Defn: Node i consistently exports w.r.t. neighbor j if there is some route R s.t. each route Q is exportable to j iff R ≤i Q.

– Defn: Node i consistently exports if it consistently exports with respect to each neighboring node j.

Next-Hop Routing Rules!• Rule 1: use next-hop rankings • Rule 2: prioritize current route• Rule 3: consistently export

• 3 deployment schemes– Configure today’s routers– Create new router configuration interface– Build new router software

Wish List Revisited

Wish List Revisited• Loop freedom• Fast convergence?• Security• Incentive compatibility [Feigenbaum-S-Ramachandran]

• Business policies• Good performance• Traffic engineering• Scalability?• Simplicity

Existence of Stable State•Existence of stable state not guaranteed even

with next-hop rankings (Rule 1) [Feamster-Johari-Balakrishnan]

•Thm: If the next-hop routing rules hold, then a stable state exists in the network.

• What about (fast!) convergence?

BGP OscillationsBGP not guaranteed to converge even with next-hop routing! [Griffin-Shepherd-Wilfong]

1

d

22 > d 1 > d

The Commercial Internet•ASes sign long-term contracts.

•Neighboring pairs of ASes have:– a customer-provider relationship– a peering relationship

peer providers

customerspeer

Gao-Rexford Framework• 3 simple conditions that are naturally

induced by the AS-business-hierarchy.– Topology condition, Preference condition, Export condition

• If the Gao-Rexford conditions hold, then BGP is guaranteed to converge to a stable state. [Gao-Rexford]

• But, this might require exponentially-many forwarding changes! [Syed-Rexford]

Fast BGP Convergence• Thm: In the Gao-Rexford framework,

next-hop routing convergence to a stable state involves at most O(L2) forwarding changes (L = # links).– all network topologies– all timings of AS activations and update

message arrivals– all initial routing states– all initial “beliefs”– implications for routing changes and number of

BGP updates

Simulations• C-BGP simulator. Cyclops AS-level topology,

Jan 1st 2010 (33,976 ASes, ~5000 non-stubs)

• Protocols: BGP, Prefer Recent Route (PRR), next-hop routing

• Metrics: # forwarding changes, # routing changes,# updates, AS-PATH length

• Events: prefix up, link failure, link recovery

• Methodology: 500 experiments, 10,000 vantage points (all non-stubs, 5000 stubs)

Simulation Results(# Forwarding Changes)

maximum number of BGP

forwarding changes > 20

maximum number of routing changes in next-hop routing

= 3maximum number of forwarding

changesin PRR = 10

Simulation Results(# Routing Changes)

maximum number of BGP routing changes

> 160

maximum number of routing changes in next-hop routing

< 20

maximum number of

routing changesin PRR > 40

Simulation Results(# BGP Updates, Non-Stub ASes)

maximum number of BGP updates > 6000

maximum number of

updates in next-hop routing <

300

maximum number of updates in PRR >

1000

Simulation Results(# Routing Changes, The 0.1%

Position)

Incentive Compatible Routing Configurations

2

d

3 1d > 2

3 > d > 1

2

Each node is getting its best feasible next-hop

Next-Hop Routing isIncentive Compatible

•Thm [Feigenbaum-Ramachandran-S]: In the Gao-Rexford framework, next-hop routing is incentive compatible.

(each node is guaranteed its best feasible next-hop)

Wish List Revisited• Loop freedom• Fast convergence• Security?• Incentive compatibility• Business policies• Good performance?• Traffic engineering?• Scalability• Simplicity

Limitations of Next-Hop Routing

•AS-PATH length•AS-avoiding policies•AS-name prepending•AS-PATH-based traffic engineering

Security, Performance,Traffic Engineering

•Still open research questions.

•Handled mostly outside the routing protocol.

•We argue that next-hop routing makes things mostly better.

Performance• Faster/better convergence than BGP.• much more scalable.• But…potential increase in path lengths.• b

– loosely correlated with performance (# routers, physical distance… throughput…).

– still, significant increase clearly undesirable!– Simulation results: same path length for 97-99%

of ASes; big increase only for ~0.1%.

Security• Reduces BGP’s attack surface (AS-PATH

length plays no role in routing decisions).

• More resilient to economically-driven attacks (incentive compatible).

• More resilient to misconfigurations (in progress)

• But… AS-avoiding policies impossible.– come with no guarantees. e2e?

Traffic Engineering•We discuss how traffic engineering

can be done without relying on the AS-PATH.– using different next-hop rankings for different

(groups of) prefixes– using the BGP communities attribute– …

Multipath Routing• Performance, security and traffic engineering

can greatly benefit from multipath routing.– multiple working paths– immediate response to failures– load balancing among multiple next-hops– …

• Next-hop routing lowers the barrier for making this a reality (work in progress).

Multipath Routing•Exploiting path diversity to

–realize the AS’s own objectives–customize route selection for neighboring ASes

•But... multipath routing is not scalable!– disseminate and store multiple routes

Multipath Routing is Not Scalable!

d

1

2

34

P1

P2

Q1

Q2

I’m using P1 and P2

I’m using Q1 and Q2

I’m using P1, P2, Q1 and Q2

From AS-PATH to AS-SET

•Next-hop routing is more amenable to multipath– nodes don’t care about entire paths– … other than for loop detection

•Don’t announce routes, announce sets!– set = union of ASes on all routes– BGP route aggregation

Neighbor-SpecificNext-Hop Routing

•Customizing route selection for neighbors– operational motivation [Kushman-Kandula-Katabi-Maggs]– economic motivation [Wang-S-Rexford]

C1

zC2

C3

d

?R1

R2

R3

Secure!

Short!

Cheap!

Neighbor-SpecificNext-Hop Routing

•Neighbor-Specific BGP [Wang-S-Rexford]– implementable using existing tools

•Results for convergence and incentive compatibility extend to multipath!

Conclusions andFuture Research

•BGP is far too complicated!

•New approach: simplify BGP– without compromising global and local goals!

•Directions for future research:– getting rid of the AS-PATH?– software / configuration complexity– more theoretical and experimental work

Thank YouThank You