SuperDREAM Testbed

Pete SiemsenAugust 2007

Why SuperDREAM?FRGP problems:

• Single Points of Failure– Routers – Level 3 switch– Member circuits– Commodity Internet

• Members can’t connect at multiple sites

Existing Single Points of Failure

• 1200 Larimer– Router, switch, ATM switch

• Level 3– switch/router – failure disconnects BiSON from Denver

• BPoP– switch/router

• Movaz boxes not addressed here

NLR Level 3 Qwest Abilene

BiSO

N

DREAM

FRGP logical view

Level 3 PoP1200 Larimer

Larimer switch

Larimer router

Larimer ATM switch

BPoP

BPoP switch/router

Level 3 switch/router

M20 routerFor ICG only

ICG

SuperDREAM goals

RedundancyAllow members to connect to two sites

Provide multiple paths to Commodity providers

Optimize

Service offeringsRouting

FRGP services

• Default service: access to other FRGP members and to FRGP peers (Comcast, RMIX)

• Commodity Internet + TransitRail• Abilene (I2)• NLR PacketNet

Proposed Solution: VRFs

• VRFs - Virtual Routing and Forwarding.

• Separate routing tables allows a router to partition services in a natural way.

• 2 possible models: per-service VRFs or per-permutation VRFs

Per-service VRF solution

• Every FRGP router will have just these VRFs:

– Members & peers VRF– A Commodity/TransitRail VRF– An NLR PacketNet VRF– An I2 VRF

Per-service VRFs…

Each member will have a VLAN and a BGP session for each service that they buy.

Per-permutation VRF solution

• Every router will have a VRF for each permutation of services bought by members

– Members & peers VRF– A Commodity/TransitRail VRF– An NLR VRF– An I2 VRF– An NLR/I2 VRF– A Commodity/Transitrail/NLR VRF– A Commodity/Transitrail/I2 VRF– A Commodity/Transitrail/NLR/I2 VRF

Solution trade-offs

Per service

Per permutation

Number of connections

Higher Lower

Memory usage Lower Higher

Rate limits Easier Harder

Statistics Easier Harder

Member complexity Higher Lower

Commodity redundancy

• Commodity ISPs will be spread across FRGP sites. Commodity access will no longer depend on the Larimer router.

• Commodity traffic will flow to the globally “best” ISP, not necessarily the ISP nearest to each member.

Rate limits (per-service)

Member traffic will rate limited at the member’s primary connection point. This greatly simplifies engineering.

Rate limits (per-permutation)

• Member traffic will be rate-limited at multiple points.

• We can’t apply a single rate limit to traffic on two separate routers.

• Multihomed members must have a primary and an idle backup per service.

Load-sharing

Level 3 Qwest

Level 3 Larimer

member

Constraints

• Maintain charging algorithm

• Have one engineering solution for everyone

• All members will do BGP

• Solution must fit in router memory

Open issues

• Implementation difficulty

• Unknown memory limits

• Engineering troubleshooting/debug

• MPLS vs. VLANs

• Cisco/Juniper interoperability

• Long-term FRGP 10G routing solution

TestBed