1/25/20051 CO internetworking (intra-domain + inter-layer) work in progress Malathi Veeraraghavan, Xuan Zheng, Zhanxiang Huang {mv5g, xuan, zh4c}@virginia.edu

1/25/2005 1

CO internetworking(intra-domain + inter-layer)

work in progress

Malathi Veeraraghavan, Xuan Zheng, Zhanxiang Huang{mv5g, xuan, zh4c}@virginia.edu

Jan. 25, 2005

1/25/2005 2


Terminology, questions• Problem description• Take a cue from CL internetworking• CO internetworking CHEETAH scenario (simple)

– Network-by-network setup– Continued setup

• CO internetworking: complex scenarios with MPLS, VLANs, SONET, WDM

• Partial CO segments intermixed with CL segments• Research problems, key ideas, conclusions

Malathi Veeraraghavan, Zhanxiang Huang, Xuan Zheng{mv5g, zh4c, xuan}@virginia.eduNov. 25, 2004

mailto:xuan%[email protected]

1/25/2005 3

Terminology

• A switch is a node in which all interfaces use the same type of multiplexing

• A gateway has interfaces that use different types of multiplexing Types of multiplexing

Circuit based(position-based) Packet based (CO)

• Different types of packets Connection-oriented IP (2205) VLAN (L2SC) MPLS (PSC-1)

SDM(space)

TDM(time)

WDM(freq.)

Note: there is no “CO Ethernet” multiplexing/switching

(FSC) (LSC)(TDM)

Network Internetwork

1/25/2005 4

Legend

G

H

IP

Host

Gateway (any type)

IP switch: CL and CO

S SONET switch

MPLS switch

VLAN switchV

M

E Ethernet switch(by default: CL)

1/25/2005 5

Questions• Difference between LSP encoding type, switching type and GPID

– Switching type: type of multiplexing used on the links of an end-to-end LSP. 3471 states “This field normally is consistent across all links of an LSP.”

• e.g., TDM, PSC-1– LSP encoding type: type of data carried on each link of the LSP 3471 says “A link

may support a set of encoding formats, where support means that a link is able to carry and switch a signal of one or more of these encoding formats depending on the resource availability and capacity of the link.” 3471: “The LSP Encoding Type represents the nature of the LSP, and not the nature of the links that the LSP traverses.” MRN document says “LSP Encoding Type (representing the nature of the link that the LSP traverses)” and says on a link terminating at a gateway that can perform PSC, TDM and WDM switching, the LSP encoding is lambda.

• Vijay confirmed my (and MRN) understanding that LSP encoding is below the switching level (actually all the way below) and GPID is above LSP

• Examples: MPLS switch with PoS link and GbE link. For the PoS link, LSP encoding should be SONET while switching type: PSC-1 and for the GbE link, LSP encoding should be Ethernet and switching type: PSC-1. Vijay says that MPLS specs don’t allow this sort of LSP to be set up. Both switching and LSP encoding type need to be the same across a “switch” for LSP setup!

– GPID: what’s carried on the LSP end-to-end

1/25/2005 6

Questions

• Nested vs. contiguous vs. stitched LSPs– Tom: if you don’t treat an LSP as an FA LSP and send

it out in IGP, then it may be regarded as a contiguous LSP. If there is label stacking, then it is a nested LSP. Unclear.

• Plain and hybrid nodes: switch and gateway– Chris: Plain node vs. hybrid node – from mrn document

– how does it relate to my “switch” and “gateway?” Plain node can also have multiple interface types (from a mux point of view) but only one mux type is enabled in a plain node at a time unlike in a hybrid node.

1/25/2005 7


• Terminology, questions Problem description• Take a cue from CL internetworking• CO internetworking CHEETAH scenario (simple)






1/25/2005 8

Initial problem

• Simplistic starting point in CHEETAH– “Optical Connectivity Service” – check to see if far end

has access to CHEETAH service

– Added need to find MAC address of far-end host

• Simplistic answer: Add “OCS available” TXT resource record to domain name in DNS server. Add MAC address associated with domain name again using TXT resource record

• See example in following two slides

1/25/2005 9

A simple scenario: Ethernet-SONET-Ethernet

End host 1 End host 2

Ethernet SONET EthernetSONET

SONETcloud

End host 1 needs to know:

• IP address of End host 2 to set up Ethernet-over-SONET circuit

• MAC address of End host 2 to encapsulate Ethernet frames

1/25/2005 10

DNS based solution for determining availability of optical connectivity and MAC address

• Use DNS server to obtain MAC address along with IP address of far-side end host.


Ethernet SONET EthernetSONET

SONETcloud

Local DNS server

Local DNS server

High-level DNS server

DNS hierarchy architecture

End host 2 registers itself in local DNS server with “OCS available” and its MAC address in TXT type

Resource Record (RR)

End host 1 finds End host 2’s IP address (using query type A) and

OCS availability and MAC address (using query type TXT)

If local DNS does not have the record for end host 2, it will obtain it through

the DNS hierarchy

1/25/2005 11

Why this approach to obtain MAC address?

• Avoid ARP broadcast going long-distance!• ARP is a fine address resolution scheme if kept local

– For performance reasons, let’s not add another “50ms” prop. delay just for ARP– Probably more important: avoid broadcast storms at Ethernet switches in both LANs

• DNS: natural solution for this mapping; it is already an “address resolution” server translating domain names to IP addresses. Using the DNS infrastructure to obtain MAC addresses (wide area) seems a natural extension.

• Implementation:– RSVP-TE client at end host 1 writes an entry in the IP routing table (route add) at end host 1

showing that to reach end host 2’s second NIC IP address, next hop is the same IP address. This removes need to place remote end hosts in same subnet

– RSVP-TE client at end host 1 writes an entry in the ARP table mapping end host 2’s second NIC IP address to obtained MAC address

– Comparable to switch fabric configuration actions at a switch– When an IP datagram is handed to the IP module of end host 1, it sees the destination-specific

routing entry in its table and checks ARP table. It finds the MAC address and can encapsulate the IP frame with Ethernet frame and send out without requiring an ARP.

1/25/2005 12

Initial problem description

• Three problems with above simplistic solution:– Can have CHEETAH-style network islands that are not

interconnected. Even if DNS query confirms “OCS available” for an end host it may not be on the same CHEETAH network as the querying host

– Internetwork scenarios: the MAC address to be returned could be that of a gateway, not the far-end host

– Partial CO internetworking impacts both “OCS available” and MAC issue.

1/25/2005 13

A more complex scenario: heterogeneous internetworking


Ethernet Ethernet

EthernetSwitch

EthernetSwitch

EthernetSwitch

EthernetSwitch

MPLS router 1 MPLS router 2VLAN 1

VLAN 2

L3 MPLStunnel

In this case, End host 1 needs to know:• MAC address of the Ethernet interface 1 on router 1 • This should be the destination MAC address in the frames sent by End host 1

12

1/25/2005 14

More complete problem(in context; rather than deconstructed)

• Answer four phases for operation of heterogeneous CO networks– how should topology/reachability/loading

conditions be advertised through routing protocols?

– how should paths be pre-computed?– what signaling parameters and values should be

used in call setup (Path and Resv messages) – what are the user-plane packet formats?

1/25/2005 15

CO internetworking

• Terminology, questions • Problem description Take a cue from CL internetworking• CO internetworking CHEETAH scenario (simple)




1/25/2005 16

Connectionless Ethernet-IP-Ethernet Scenario

ARP

MAC

ARP

MAC

End-host A’s IP routing table

Host A Host B

Gateway I Gateway II

Router A’s routing table Router B’s routing table

B Directly connectedB IP address of gateway I.1 interface

B Switch 3.1 interface

End-host A’s Internet to network address mapping table (ARP table)

I.1 interface IP addr. I.1 interface MAC addr.

Internet address Network 1 address

READ NOTES

G G

Ethernet packet-based multiplexing on all links

Switch 1

(Ethernet)

Switch 2

(Ethernet)

Network 1

E EH H

I.1

Switch 3

(IP)

IP multiplexing on all links (e.g. all PPP links)

Network 2

I.2 II.23.1 3.2

IPII.3

Ethernet packet-based multiplexing on all links

Network 3

Switch 4

(Ethernet)

Switch 5

(Ethernet)

E E

1/25/2005 17

CO internetworking

• Terminology, questions • Problem description• Take a cue from CL internetworking CO internetworking CHEETAH scenario


• CO internetworking: scenarios with MPLS, VLANs, SONET, WDM


1/25/2005 18

Cheetah Scenario:Dedicated Ethernet-SONET-Dedicated Ethernet

(actually SDM-TDM-SDM)Routing info distribution phases

GW1 GW2

H1 H2I.1 II.3I.2 II.23.1 3.2

GH H

G

VLSR VLSR

S

Network 1

TDM muxing

Internetwork: SDM muxing based network

GW1’s link state database

Router Link Sw. Cap.

GW2 SW1:3.2-II.2 TDM

GW2 II.3 FSC

SW1

(SONET)

LSA originated by GW2

Link Type Sw. Cap.

II.3 3 FSC

SW1:3.2-II.2 2 TDM

LSA originated by SW1

Link Type Sw. Cap.

GW:I.2-3.1 2 TDM

3.2-GW2:II.2 2 TDM

GW2 is a

gateway

GW1 is a

gateway

LSA originated by GW2

Link Type Sw. Cap.

GW1-GW2 Pretend FSC/TDM

GW1’s link state database



GW2 II.3 FSC

GW2 GW1-GW2 FSC/TDM

1/25/2005 19


(actually SDM-TDM-SDM)Path-computation phases

GW1’s Link State Database


GW2 GW1-GW2 FSC/TDM

GW2 II.3 FSC

… … …

GW1 GW2

H1 H2I.1 II.3I.2 II.23.1 3.2

GH H

G

VLSR VLSR

S

Network 1

TDM muxingInternetwork: SDM muxing based network

SW1

(SONET)

Dest Next hop CO type

H2 GW2 SDM

Dest Next hop CO type

GW2 SW1 TDM

Inner routing table

Outer routing table

CSPF

1/25/2005 20


(actually SDM-TDM-SDM)Signaling phase (network-by-network setup: nested LSPs)

GW1 GW2

PATH

H1H2

RESV

I.1 II.3I.2 II.23.1 3.2

GH H

G

READ NOTES

VLSR VLSR

S

Network 1

TDM muxingInternetwork: SDM muxing based network

Path Dest BW Sw. cap.

LSP enc. GPID

H1-GW1-GW2-H2 H2 Intserv Tspec FSC Fiber or Ethernet?

Ethertype

GW1-SW1-GW2 GW2 SONET Tspec TDM SONET/SDH SONET/SDH

Resv Stack of labelsH2-GW2-GW1-H1 H2 MAC (know to generate this because of GPID)

GW1-SW1-GW2 GW2 MAC

End-host A’s CO IP routing table (consulted by RSVP-TE client)

H2 GW1

SW1

(SONET)

1/25/2005 21


(actually SDM-TDM-SDM)User-plane: Data-flow phase

H1 H2I.1 II.3I.2 II.23.1 3.2

GH H

G

VLSR VLSR

S

Network 1

TDM muxing


H2MAC

H2IP

Data H2MAC

H2IP

DataSONETframe

H2MAC

H2IP

Data


H2 GW1

GW1 GW2SW1

(SONET)

1/25/2005 22


(actually SDM-TDM-SDM)(continued setup: also a nested LSP!)

PATH

H1 H2

RESV

I.1 II.3I.2 II.23.1 3.2

Network 1

Doesn’t seem to be a good solution; mismatchesin crossconnect rates (e.g., lambda);

gets complex if VCAT necessitates multiple LSPs

GH H

GS

READ NOTES

VLSR VLSR


GW1 GW2SW1

(SONET)


H2 GW1

1/25/2005 23

CO internetworking

• Terminology, questions • Problem description• Take a cue from CL internetworking• CO internetworking CHEETAH scenario (simple)


CO internetworking: complex scenarios with MPLS, VLANs, SONET, WDM


1/25/2005 24

Before we start with the scenarios

• Learn what current gateway implementations actually do

• Some background on OSPF, OSPF-TE and GMPLS extensions

• Key idea: Pretend links; allows for homogeneous LSPs to be setup in nested mode

• Outermost network: SDM, VLAN, IP– Two points

1/25/2005 25

User-plane capabilities(to support CO internetworking)

• Three types of gateways:– Summit Extreme

• SDM ↔ VLAN (untagged ports to tagged VLANs)– Cisco GSRs

• IP ↔ MPLS• VLAN ↔ MPLS• SDM ↔ VLAN (port mapped Ethernet over MPLS)

– Cisco 15454/Movaz• SDM ↔ SONET• SDM ↔ WDM• VLAN ↔ SONET? (Ethernet cards in 454)?• IP ↔ SONET? (ML series cards in 454)?

1/25/2005 26

User-plane capabilities(to support CO internetworking)

• Summit Extreme– Can crossconnect SDM to VLAN– Meaning place an untagged port (port mapped) to a

VLAN along with a tagged port (with a VLAN ID)– Switch is capable of popping on label (VLAN ID) and

popping it off if outgoing port is untagged– The form of mux/demux on each port is changeable on

a packet-by-packet basis – quite amazing! • on packet can come in w/o a VLAN tag requiring the switch to

forward it according to rules of its untagged VLAN setting• another packet can come in with a VLAN tag and need to be

switched accordingly.

1/25/2005 27

User-plane capabilities (to support CO internetworking)

• Cisco GSR– “L3 MPLS” – map packets of a certain flow to an

MPLS tunnel.– Ethernet over MPLS

• VLAN• Port mapped• VLAN rewrite

– This is simply a CO PS scenario where the label is changed. With a VLAN, the same label is used on all links unlike in the more generic CO PS where labels are only unique to a link

– With VLAN rewrite on the other side of a different type of network, the VLAN ID is changed (i.e. label).

1/25/2005 28

User-plane capabilities (to support CO internetworking)

• Cisco 15454 and Movaz box– Can map all frames arriving on a port to a

SONET circuit or wavelength• SDM ↔ SONET or WDM

• Unsure whether Ethernet cards are capable of processing VLAN IDs or IP header fields to then decide which ones to map to a certain SONET circuit or wavelength

1/25/2005 29

Background(what info does a switch/gateway have)

• OSPF (2328): – Summary LSAs, Router LSAs, Network LSAs

– Summary LSAs sent by area border routers with reachability information

• OSPF-TE (3630):– TE LSAs – top-level TLV: Link TLV

• ccamp extensions for GMPLS:– sub-TLVs that describe switching capability on link

identified in top-level Link TLV

1/25/2005 30

OSPF-TE (3630)

The following sub-TLVs of the Link TLV are defined:

1 - Link type (1 octet) 2 - Link ID (4 octets) 3 - Local interface IP address (4 octets) 4 - Remote interface IP address (4 octets) 5 - Traffic engineering metric (4 octets) 6 - Maximum bandwidth (4 octets) 7 - Maximum reservable bandwidth (4 octets) 8 - Unreserved bandwidth (32 octets) 9 - Administrative group (4 octets)

1/25/2005 31

ccamp extensions for GMPLS

Sub-TLV Type Length Name

11 8 Link Local/Remote Identifiers

14 4 Link Protection Type

15 variable Interface Switching Capability Descriptor

16 variable Shared Risk Link Group

1/25/2005 32

GSR capabilities and Link TLVs

• Cisco GSR is capable of– “L3 MPLS” – mapping a flow at the IP layer to

an MPLS LSP– Ethernet over MPLS:

• VLAN• Port mapped• VLAN rewrite – this is nothing but changing labels

– generically in CO PS networks the labels are unique only to each link; which means they are mapped from link to link on the end-to-end path.

1/25/2005 33

GSR capabilities and Link TLVs

• These capabilities of the GSRs got us wondering whether in addition to Link TLVs in TE LSAs we need some way of identifying the capabilities of the gateways– e.g., whether it supports VLAN over MPLS in addition to L3

MPLS• One answer:

– Don’t need another TLV– The gateways need to “pretend” they are interconnected by logical

links and advertise the switching capability (i.e., the types of multiplexing) available on these links

– If two LERs at the edge of an MPLS cloud can support VLAN over MPLS then they advertise a direct logical link between the LERs, which supports VLAN multiplexing

1/25/2005 34

Use routing data to ask for the right type of connection

• By spreading multiplexing capabilities on these “pretend” logical links, a gateway/switch can determine whether or not there is an end-to-end path with a certain type of multiplexing

• If the routing data is somehow wrong, a gateway/switch should simply reject a call if it receives a Path request for a certain type of connection that it does not support (e.g., a request for a TDM circuit via a WDM switch or a request for a VLAN connection to a gateway whose outgoing logical links to other gateways on an MPLS cloud do not support “VLAN over MPLS”

1/25/2005 35

Possible multiplexing schemes on the CO internetwork, i.e., the outermost network

• Three possibilities– SDM (Fiber switch capable)

– VLAN

– IP (Connection-oriented)

1/25/2005 36

Two points re. outermost network

• Presence of IP and Ethernet even with SDM outermost network

• Forget hierarchy

1/25/2005 37

Point 1: Presence of IP and Ethernet even with SDM outermost network

• Even when SDM is outermost network, user-data payload is carried in IP datagrams encapsulated in Ethernet frames– Reason 1: Use socket API in application programming

– which means IP datagram encapsulation is a given.

– Reason 2: Ethernet is the common NIC for end hosts; so Ethernet frame encapsulation is a given.

• Small overhead – hence ignored

1/25/2005 38

Point 2: Forget hierarchy

• Conventional thinking:– SDM is at the lowest level of the hierarchy

– Common view:• MPLS over SONET over WDM over fiber

• Contrary view here:– SDM (aka) fiber is outermost network!

• Any “multiplexing/switching layer” can ride on top of any other multiplexing/switching layer– Just depends on network topology

1/25/2005 39

Scenarios

• Scenario 1: SDM-(VLAN)-(MPLS)-(VLAN)-SDM– No VLAN capability at end host NICs; but these NICs

are connected to Ethernet switches with VLAN cap.– Gateways between VLAN and MPLS networks have

VLAN capability in Ethernet cards but these gateways have no support to carry VLAN frames on MPLS LSPs

• Gateways issue OSPF-TE LSAs indicating GW↔GW “pretend” links support FSC

• Also basic OSPF LSAs indicate availability of “pretend” links allowing for CO IP to the outermost network (internetwork)

– Make outermost network call setup be SDM

1/25/2005 40

End-to-end CO: Scenario 1SDM-(VLAN)-(MPLS)-(VLAN)-SDM

Routing info distribution phases

VLAN multiplexing VLAN multiplexing

H1 H2G M

H

VG

H

V GG

VLSR VLSR VLSR VLSR VLSRVLSR

Network 1 Network 2 Network 3

GW1

MPLS multiplexing

SW1 GW4GW3GW2 SW2 SW3

GW2’s LSA

Link Type Sw. Cap.

SW1-GW2 2 L2SC

GW2-SW2 2 PSC-1

GW1’s LSA

Link Type Sw. Cap.

GW1-SW1 2 L2SC

GW1-H1 3 FSC

SW1’s LSA

Link Type Sw. Cap.

GW1-SW1 2 L2SC

SW1-GW2 2 L2SC




GW2 II.3 FSC

GW2 is a

gateway

GW1 and

GW3 are gateways

GW2 and

GW4 are gateways

GW3 is a

gateway



GW2 GW1-GW2 FSC/L2SC

GW2 GW2-GW3 FSC/PSC-1


GW2’s LSA

Link Link type Sw. Cap.

SW1-GW2 2 L2SC

GW2-SW2 2 PSC-1

GW1-GW2 Pretend FSC/L2SC

GW2-GW3 Pretend FSC/PSC-1

1/25/2005 41

End-to-end CO: Scenario 1SDM-(VLAN)-(MPLS)-(VLAN)-SDMRouting path precomputation phases

GW1’s LS database

Router Link Link type Sw. Cap.

GW2 GW1-SW1 2 L2SC

GW2 GW2-SW2 2 PSC-1

GW2 GW2-GW3 Pretend FSC

GW2 GW1-GW2 Pretend FSC

… … … …


H1 H2G M

H

VG

H

V GG



GW1

MPLS multiplexing



Dest. Next hop Sw. Cap.

H2 GW2 FSC


GW2 SW1 L2SC

CSPF

Outer Routing table

Inner Routing table

1/25/2005 42


Signaling phase


PATH

RESV

H1 H2G M

H

VG

H

V GG



GW1

MPLS multiplexing



LSP enc. GPID

H1-GW1-GW2-GW3-GW4-H2 H2 Intserv Tspec FSC Fiber or Ethernet? Ethertype

GW1-SW1-GW2 GW2 Intserv Tspec L2SC Ethernet or packet? VLAN? Ethertype

GW2-SW2-GW3 GW3 Intserv Tspec PSC-1 Packet? Ethernet? Ethertype


Resv Stack of labelsH1-GW1-GW2-GW3-GW4-H2 H2 MAC (know to generate this because of GPID)

GW1-SW1-GW2 GW2 MAC + VLAN ID

GW2-SW2-GW3 (assuming Eth. links)

GW3 MAC + MPLS label



1/25/2005 43


User-plane: data flow phase (MPLS network links are both Ethernet)


H1 H2G M

H

VG

H

V GG


Network 1 Network 3

GW1

MPLS multiplexing


H2MAC

H2IP

Data

H2MAC

H2IP

Data

Network 2


H2MAC

H2IP

DataVLAN1 ID

GW2MAC

H2MAC

H2IP

DataMPLSLabel

GW3MAC

H2MAC

H2IP

DataVLAN1 ID

GW4MAC

1/25/2005 44


User-plane: data flow phase (MPLS network links are both PPP)


H1 H2G M

H

VG

H

V GG


Network 1 Network 3

GW1

MPLS multiplexing


H2MAC

H2IP

Data

H2MAC

H2IP

Data

Network 2


H2MAC

H2IP

DataVLAN1 ID

GW2MAC

H2MAC

H2IP

DataMPLSLabel

PPPheader

H2MAC

H2IP

DataVLAN1 ID

GW4MAC

1/25/2005 45


Signaling phaseMPLS network: first link is PPP and the second Ethernet


LSP enc. GPID

H1-GW1-GW2-GW3-GW4-H2 H2 Intserv Tspec FSC Fiber or Ethernet? Ethertype


GW2-SW2-GW3 GW3 Intserv Tspec PSC-1 Packet? Ethernet? Ethertype


Resv Stack of labelsH1-GW1-GW2-GW3-GW4-H2 H2 MAC (know to generate this because of GPID)


GW2-SW2-GW3 (assuming Eth. links) GW3 MAC + MPLS label



PATH

RESV

H1 H2G M

H

VG

H

V GG



GW1

MPLS multiplexing



1/25/2005 46

Scenarios contd.

• Scenario 2: SDM-(VLAN-(MPLS)-VLAN)-SDM– No VLAN capability at end host NICs; but these NICs

are connected to Ethernet switches with VLAN cap.– Gateways between VLAN and MPLS networks have

VLAN capability in Ethernet cards and these gateways have support to carry VLAN frames on MPLS LSPs

• Gateways issue OSPF-TE LSAs indicating GW↔GW “pretend” links support FSC and L2SC (VLAN)


– Make outermost network call setup be SDM

1/25/2005 47

End-to-end CO: Scenario 2SDM-(VLAN-(MPLS)-VLAN)-SDM



H1 H2G M

H

VG

H

V GG



GW1

MPLS multiplexing



GW1’s LSA

Link Type Sw. Cap.

GW1:H1 3 FSC

GW1-SW1 2 L2SC

SW1’s LSA

Link Type Sw. Cap.

GW1-SW1 2 L2SC

SW1-GW2 2 L2SC

Link Type Sw. Cap.

SW1-GW2 2 L2SC

GW2-SW2 2 PSC-1/L2SC



GW2 SW1-GW2 L2SC

GW2 GW2-SW2 PSC-1

… … …

GW2 is a

gateway

GW1 and

GW3 are

gateways

GW2 and

GW4 are

gateways

GW3 is a

gateway

GW2’s LSA

Link Type Sw. Cap.

GW1-GW2 Pretend FSC/L2SC

GW2-GW3 Pretend FSC/PSC-1/L2SC




GW4 GW4:H2 FSC

… … …

1/25/2005 48


Routing path precomputation phases


H1 H2G M

H

VG

H

V GG



GW1

MPLS multiplexing






GW4 GW4:H2 FSC

GW2 GW2-GW3 FSC/L2SC/PSC-1

… … …


H2 GW4 FSCCSPF

Outer

Routing tables


GW4 GW2 L2SC

InnerDest. Next hop Sw. Cap.

GW2 SW1 L2SC

Intermediate

1/25/2005 49


Signaling phase(MPLS network links are both Ethernet)


PATH

RESV

H1 H2G M

H

VG

H

V GG



GW1

MPLS multiplexing


Path Dest BW Sw. caps LSP enc. GPIDH1-GW1-GW4-H2 H2 Intserv Tspec FSC Fiber Ethertype

GW1-SW1-GW2-GW3-SW3-GW4 GW4 ? L2SC ? Ethertype

GW2-SW2-GW3 GW3 Intserv Tspec PSC-1 Packet Ethertype



GW1-SW1-GW2-GW3-SW3-GW4 GW2 MAC + VLAN ID

GW2-SW2-GW3 GW3 MAC + VLAN ID +MPLS label

1/25/2005 50


User-plane: data flow phase(MPLS network links are both Ethernet)


H1 H2G M

H

VG

H

V GG


Network 1 Network 3

GW1

MPLS multiplexing


H2MAC

H2IP

Data

H2MAC

H2IP

Data

Network 2

H2MAC

H2IP

DataVLAN1 ID

GW2MAC

H2MAC

H2IP

DataMPLSLabel

VLAN1 ID

GW3MAC

H2MAC

H2IP

DataVLAN1 ID

GW4MAC


1/25/2005 51

Scenarios contd.

• Scenario 3: VLAN-(MPLS)-VLAN– VLAN capability at end host NICs and these NICs are

connected to Ethernet switches with VLAN cap.– Gateways between VLAN and MPLS networks have

VLAN capability in Ethernet cards and these gateways have support to carry VLAN frames on MPLS LSPs



– Make outermost network call setup be VLAN

1/25/2005 52

End-to-end CO: Scenario 3 VLAN-(MPLS)-VLAN


VLAN multiplexing

H1 H2G M

H

VV

H

V VG


Network 2

SW1

MPLS multiplexing

SW2 SW5GW3GW2 SW3 SW4

GW2’s LSA

Link Type Sw. Cap.

GW2-SW3 2 PSC-1

SW2-GW2 2 L2SC

GW2’s LSA

Link Type Sw. Cap.

GW2-SW3 2 PSC-1

SW3-GW3 2 PSC-1

GW2’s LSA

Link Type Sw. Cap.

SW3-GW3 2 PSC-1

GW3-SW4 2 L2SC



GW3 SW3-GW3 PSC-1

GW3 GW3-SW4 L2SC

… … …

GW3 is a

gateway

GW2 is a

gateway

GW2’s LSA

Link Type Sw. Cap.

SW3-GW3 2 PSC-1

GW3-SW4 2 L2SC

GW2-GW3 Pretend L2SC




GW3 GW3-SW4 L2SC

… … …

1/25/2005 53


Routing path precomputation phases

VLAN multiplexing

H1 H2G M

H

VV

H

V G


Network 2

SW1

MPLS multiplexing

SW2 GW3GW2 SW3 SW4VSW5




GW3 GW3-SW4 L2SC

… … …


H2 GW3 FSCCSPF

Outer

Routing tables


GW3 SW3 PSC-1

1/25/2005 54



VLAN multiplexing

PATH

RESV

H1 H2G M

H

VV

H

V G


Network 2

SW1

MPLS multiplexing

SW2 GW3GW2 SW3 SW4

Path Dest BW Sw. caps LSP enc. GPIDH1-SW1-SW2-GW2-GW3-

SW4-GW4-H2H2 ? L2SC ? Ethertype


Resv Stack of labelsH1-SW1-SW2-GW2-GW3-SW4-GW4-

H2H2 MAC (know to generate this because of GPID)


VSW5

1/25/2005 55



VLAN multiplexing

H1 H2G M

H

VV

H

V G


Network 2

SW1

MPLS multiplexing

SW2 GW3GW2 SW3 SW4

H2MAC

H2IP

DataVLAN1 ID

GW2MAC

H2MAC

H2IP

DataMPLSLabel

VLAN1 ID

GW3MAC

H2MAC

H2IP

DataVLAN1 ID

H2MAC

VSW5

1/25/2005 56

Scenarios contd.

• Scenario 4: VLAN-(SONET)-VLAN– VLAN capability at end host NICs and these NICs are

connected to Ethernet switches with VLAN cap.

– Gateways between VLAN and SONET networks have VLAN capability in Ethernet cards and these gateways have support to carry VLAN frames on SONET circuits



– Make outermost network call setup be VLAN

– See notes

1/25/2005 57

End-to-end CO: Scenario 4 VLAN-(SONET)-VLAN


VLAN multiplexing

H1 H2G M

H

VV

H

V VG


Network 2

SW1

SONET multiplexing

SW2 SW5GW3GW2 SW3 SW4

GW2’s LSA

Link Type Sw. Cap.

GW2-SW3 2 TDM

SW2-GW2 2 L2SC

GW2’s LSA

Link Type Sw. Cap.

GW2-SW3 2 TDM

SW3-GW3 2 TDM

GW2’s LSA

Link Type Sw. Cap.

SW3-GW3 2 TDM

GW3-SW4 2 L2SC



GW3 SW3-GW3 TDM

GW3 GW3-SW4 L2SC

… … …

GW3 is a

gateway

GW2 is a

gateway

GW2’s LSA

Link Type Sw. Cap.

SW3-GW3 2 TDM

GW3-SW4 2 L2SC

GW2-GW3 Pretend FSC/TDM



GW3 GW2-GW3 FSC/TDM

GW3 GW3-SW4 L2SC

… … …

1/25/2005 58


Routing protocol and path precomputation phases

VLAN multiplexing

H1 H2G M

H

VV

H

V G


Network 2

SW1

SONET multiplexing

SW2 GW3GW2 SW3 SW4VSW5



GW3 GW2-GW3 FSC/TDM

GW3 GW3-SW4 L2SC

… … …


H2 GW3 FSCCSPF

Outer

Routing tables


GW3 SW3 TDM

1/25/2005 59


Signaling phase

PATH

RESV

H1 H2G S

H

VV

H

V G


Network 2

SW1

SONET multiplexing

SW2 GW3GW2 SW3 SW4

Path Dest BW Sw. caps LSP enc. GPIDH1-SW1-SW2-GW2-GW3-SW4-GW4-H2

H2 ? L2SC ? Ethertype


Resv Stack of labelsH1-SW1-SW2-GW2-GW3-SW4-

GW4-H2H2 MAC (know to generate this because of

GPID)

GW2-SW3-GW3 GW3 SONET label + VLAN ID

VLAN multiplexing

VSW5

1/25/2005 60


User-plane: data flow phase


H1 H2G S

H

VV

H

V G


Network 2

SW1

SONET multiplexing

SW2 GW3GW2 SW3 SW4

H2MAC

H2IP

DataSONETframe

VLAN1 ID

H2MAC

H2IP

DataVLAN1 ID

GW2MAC

H2MAC

H2IP

DataVLAN1 ID

H2MAC

VSW5

1/25/2005 61

For each scenario, answer the following questions

1. Routing protocol and path pre-computation phase:• What should have been advertised by OSPF-TE and what

computations should have been run by CSPF modules?

2. Signaling phase:• What types of objects are used in Path messages for the five

parameters and what values are set in the key fields of these objects?

• What labels are carried in the Resv messages? Which interface’s MAC address is necessary at sender?

3. User-plane:• Packet formats on each interface

1/25/2005 62

Why end host needs CO reachability information with type of CO service (slide i)

• In CL IP networks:– default setting: use IP subnet address to

determine whether destination is directly reachable or not?

– this allows sending end host to issue an ARP with IP address of destination host or IP address of gateway (typically just one – since there is only one type of internetwork CL service, aka IP)

1/25/2005 63

Why end host needs CO reachability information with type of CO service (slide ii)

• If the sending host has a manually set entry in its IP routing table for the destination host which is on a different subnet as well as an entry in the ARP table giving the MAC address of the destination (H2)– It can generate a frame with destination MAC address = H2 and

destination IP address = H2– The default gateway will not intercept the packet in some sort of

proxy mode and relay it to the destination– This is an application of the “end-to-end argument” – If the gateway did intercept such a packet, it becomes more like

Intelligent Networks– What will happen is that the default gateway will not accept the

packet since the destination MAC address does not match it’s own interface’s MAC address. Therefore the packet will just be dropped

1/25/2005 64

Why end host needs CO reachability information with type of CO service (slide iii)

• Applying similar reasoning– it is the responsibility of the end host to generate a Path

message requesting the “right” type of connection to a destination, i.e., one that is indeed available.

– if such a connection is not possible, the call should be rejected

• Sending end host has three options for the type of connection it can request– SDM– VLAN– CO IP

1/25/2005 65

Why end host needs CO reachability information with type of CO service (slide iv)

• This concept of the gateway not automatically changing the type of Path request holds at each hop.

• Example:– In scenario 1, if the VLAN-MPLS GW2 had not announced

availability of SDM multiplexing on its pretend link to the far-end GW on the MPLS network, the SDM-VLAN GW1 would not have issued the Path request of the SDM variety.

– By having OSPF-TE LSAs for pretend links, we can use the end-to-end argument in CO networks

– Without these LSAs, gateways would need to automatically convert the type of requests – makes it more IN like.

1/25/2005 66

Why end host needs CO reachability information with type of CO service (slide v)

• Implication– End hosts need to keep a CO routing table – How is this information learned at end hosts – LMP? Since end hosts don’t run OSPF

Destination IP addr(subnet or host)

Gateway(next-hop)

Type of connection(SDM, VLAN, CO IP)

1/25/2005 67

CO internetworking

• Terminology, questions • Problem description• Take a cue from CL internetworking• CO internetworking CHEETAH scenario (simple)



Partial CO segments intermixed with CL segments• Research problems, key ideas, conclusions

1/25/2005 68

Partial Connection Oriented: Scenario 1 Ethernet-MPLS-Ethernet Scenario


Ethernet packet-based multiplexing (E1)


GW1GW2

End-host A’s routing table

H1 H2


I.1 I.2 II.2II.3

3.1 3.2

SW3(MPLS)

EG MH

EEEH

G

VLSR VLSR

IP packet-based multiplexing

H2 GW1

Link Sw. Cap.

GW1-GW2 PSC-1

GW2’s advertised link TLVs

Link Sw. Cap.

GW1-GW2 PSC-1


1/25/2005 69


Signaling phase



GW1GW2

PATH

H1H2

tagging untaggingRESV


I.1 I.2 II.2 II.33.1 3.2

SW3(MPLS)

EG MH

EEEH

G

VLSR VLSR


H2 GW1

Ethernet packet-based multiplexing

Path Dest BW Sw. caps LSP enc. GPIDH1-SW2-GW2-H2 H2 Intserv Tspec PSC-1 Packet Ethertype



GW2-SW3-GW3 GW2 MAC + MPLS label

1/25/2005 70


User-plane: data flow phase(MPLS network links are both Ethernet)



GW1GW2

H1 H2


I.1 I.2 II.2 II.33.1 3.2

SW3(MPLS)

EG MH

EEEH

G

VLSR VLSRGW1MAC

H2IP

Data H2MAC

H2IP

DataH2IP

DataMPLSLabel

GW2MAC

1/25/2005 71

Partial Connection Oriented: Scenario 2 Ethernet-SONET-Ethernet Scenario




GW1GW2


H1 H2


I.1 I.2 II.2II.3

3.1 3.2

SW3(SONET)

EG MH

EEEH

G

VLSR VLSR


H2 GW1

Link Sw. Cap.

GW1-GW2 TDM


Link Sw. Cap.

GW1-GW2 TDM


1/25/2005 72


Signaling phase



GW1GW2

PATH

H1H2

tagging untaggingRESV


I.1 I.2 II.2 II.33.1 3.2

SW3(SONET)

EG MH

EEEH

G

VLSR VLSR


H2 GW1


Path Dest BW Sw. caps LSP enc. GPIDH1-SW2-GW2-H2 H2 Intserv Tspec PSC-1 Packet Ethertype



GW1-SW3-GW2 GW2 SONET label

1/25/2005 73


User-plane: data flow phase



GW1GW2

H1 H2


I.1 I.2 II.2 II.33.1 3.2

SW3(MPLS)

EG MH

EEEH

G

VLSR VLSRGW1MAC

H2IP

Data H2MAC

H2IP

DataH2IP

DataH2

MACSONETframe

1/25/2005 74


• Terminology, questions• Problem description• Take a cue from CL internetworking• CO internetworking CHEETAH scenario (simple)



• Partial CO segments intermixed with CL segments Research problems, key ideas, conclusions



1/25/2005 75

Research vs. eng. problems

• Problem statement– How to create connections (rate-guaranteed allocations)

across CO networks that support different types of multiplexing?

• Mostly engineering problems– What type of Path message objects to use, and what

values to use for the fields in these objects?

– How to declare “pretend” links to allow for path computation or next-hop

1/25/2005 76

“Pretend” links

• How many pretend links should be advertised to outside this network?

– N(N-1)/2 links, where N is number of GWs, e.g.,GW1↔GW2; GW1↔GW3.....

• What is the capacity for each of these pretend links? How should gateway-to-switch and switch-to-switch link capacity be divided between the pretend links before advertising – estimates?

• Actual routing of connections to make these pretend links real can be changed during setup.

• Switching capabilities on these links are easy – if GW1 supports VLAN over MPLS as well as Layer 3 MPLS, just advertise L2SC, SDM and whatever allows for 2205 RSVP (which is CO IP)

G

M

G

G G

G

MM

M M

GW1

GW3GW2

GW4

GW5

SW1SW5

SW4SW3

SW2Path

OSPF MPLS mux.

1/25/2005 77

Is there a research problem?

• First consider the research problems in this whole area of routing/signaling/user-plane protocols – in homogeneous networks

– There are two problems when deconstructed (i.e, context is removed)

1. routing problem deconstructs to the constrained shortest path computation in a graph problem

2. signaling – main aspect is bandwidth management – CAC; so here the research problem is how to bandwidth share? Accept/reject a call or provide less BW than asked for? Classes of service; fairness traded off against utilization; ULGM sharing; how to set UL and GMs?

1/25/2005 78

CSPF routing deconstructed problemin homogeneous network

• Example constraint: bandwidth• Problem: find SP from a-f with min. BW = 30Mbps

a

c

d

b

e

f2, 30Mbps

5, 100Mbps

1, 150Mbps1, 10Mbps

1, 50Mbps1, 500Mbps2, 50Mbps

1, 50Mbps

Link weight Available BW

• Answer: a-b-c-e-f: path weight = 2 + 5 + 1 + 1 = 9• Path a-b-d-f is shortest path with path weight = 2+1+1 = 4, but BW available is only

10Mbps, less than the required 30Mbps

1/25/2005 79

How does the addition of the heterogeneity dimension affect these two research problems?

Routing problem

a

c

d

b

e

f2, 30Mbps

1, 3200Gbps

1, 8000Gbps

5, 100Mbps

2, 10Gbps

1, 10Gbps

1, 10Gbps

1, 2.5Gbps

Link weightAvailable BW e.g., WDM switch

e.g., SONET switch

e.g., MPLS switch

Crossconnect granularity1Mbps

1Mbps51Mbps

10Gbps

• New constraint: node crossconnect granularity

• Problem: find SP from a-f with min. BW = 30Mbps

• Answer: a-b-d-f: path weight = 2 + 1 + 1 = 4, and 30Mbps is available, but it needs the setup of a 10Gbps circuit on b-d-f before 30Mbps can be assigned on this logical link.

• If the rule had been to tie up minimum extra bandwidth, then path would be a-b-c-e-f • Or split connection on small-granularity paths

1/25/2005 80

Bandwidth sharing deconstructed problemin homogeneous network

• Problem: Should node b accept the call, reject the call or assign it some BW lower than the requested 30Mbps– Simple Complete Sharing (CS) algorithm would say yes!– But from a fairness perspective (short-duration vs. long-duration, short-path

vs. long-path), node b may reject the call or give a lower BW level• Added dimension: to split call on different routes

a

c

d

b

e

f2, 30Mbps

5, 100Mbps

1, 150Mbps1, 10Mbps


1, 50Mbps

Request for 30Mbps connection

1/25/2005 81

Bandwidth sharing deconstructed problemin heterogeneous network

• Problem: – Tradeoff of fairness and utilization becomes more

difficult when these crossconnect granularities are considered

a

c

d

b

e

f2, 30Mbps

5, 100Mbps

1, 150Mbps1, 10Mbps


1, 50Mbps

Request for 30Mbps connection1Mbps

1Mbps51Mbps

10Gbps

1/25/2005 82

Hop-by-hop vs. source routing

• I think both can be supported even for QoS-based routing

• Industry seems to lean toward source routing

1/25/2005 83

Key ideas

• Use nested LSPs, each of a uniform multiplexing type (aka switching capability)

• Advertise “pretend links” to enable other gateways/switches to determine whether they can set up an LSP of a certain type

1/25/2005 84

Conclusions

• Is there a protocol to share reachability data from nearest CO switch/gateway to end host – use of LMP?

• Pretend links – how do nodes automatically recognize need to report these?– From OSPF-TE LSAs on interface switching capabilities, easy to

recognize gateways from switches– A gateway reports a pretend link to each other gateway that it can

reach with one form of multiplexing– Report current available BW as minimum available BW to each

gateway (ignore the fact that if a connection got set up to one gateway, the available BWs drop and hence available BW to another gateway will also drop)

1/25/2005 85

Conclusions for my original problem

• What should OCS do?– Useful to bring back MAC address of

destination end host?– Determine type of CO path available?

1/25/2005 86

What software modules need to be developed and deployede.g., SDM-(VLAN)-(MPLS)-(VLAN)-SDM


PATH

RESV

H1 H2M

H

VG

H

V G

VLANVLSR

Pretendlink advertiser

VLSR


GW1

MPLS multiplexing

SW1 GW4GW3Cisco

GSR

SW2 SW3


Identify what functionality each gateway box and custom-build gateway GMPLS engine for that box. Leverage software implemented in box.

OSPF-TE

RSVP-TE

Pretend link signaling module

See next slide for functional description of pretend link advertiser and pretend link signaling module.







OSPF LSA

Serves as an RSVP-TE client at an “end host” for this connection

OSPF-TE

RSVP-TE

Updateconn.table

VLANVLSR

A Cisco GSR has • RSVP-TE software as an end client, i.e., an MPLS LER• RSVP-TE

1/25/2005 87

Software for gateways• Pretend link advertiser:one per gateway - reads OSPF-TE database, recognize from

interfaces on all switches in network 2 that node GW3 is a gateway (links with multiple switching capabilities); this means a pretend link should be advertised between these two gateways. Determine what switching capabilities and rates to advertise for this pretend link based on user-plane gateway capabilities of the box. The pretend links should not be written into the connectivity table of the nodes unless there is software in the nodes to handle these links differently from regular links.

• Pretend-link signaling module: one per-gateway; if gateway has no RSVP-TE LER (edge) engine to act like an end host client RSVP-TE, then it serves this functionality for the gateway. If the gateway does have this built-in functionality, it simply serves to hold up Path messages headed for the pretend links and invoke the RSVP-TE client software through a CLI/TL1 command to initiate call setup. Thus, it receives RSVP-TE messages, determines if it requires pretend link setup, then issues commands to initiate the set up of the pretend link if not already setup. Then if the box has ability to integrate the newly setup LSP as a FA -LSP, then it transparently passes the SDM RSVP-TE path setup message to the RSVP-TE signaling module built into the gateway. The latter will make the GW2 RSVP-TE signaling module handle it and send one to the GW3 RSVP-TE signaling module; these modules act as the RSVP-TE end host clients for the intra-network connection or as the external trigger for the RSVP-TE client built into the node as an end host client (e.g., the GSR).

1/25/2005 88

End host first CO node discovery

• Define a protocol for end host software to broadcast a discover – see if DHCP can be used augmented with usage of some obscure field.

• Get multiple replies of which nodes have CO capability. • Copy routing data from these nodes to know which IP

addresses are reachable through what form of CO network: SDM, VLAN or IP.

• Need to deploy end host software for this purpose as well as software external to CO nodes to respond to end hosts queries.

• Limit this to an enterprise. If enterprise doesn’t purchase CO service, individual end hosts cannot. So the DHCP discovery of CO capability only spreads up to the WAN access router.

1/25/2005 89

MAC address software?

• If sending RSVP-TE client at end host asks for SDM or VLAN call with GPID as Ethernet, then receiving RSVP-TE client at end host responds with MAC address of itself in stacked label. – Question: can Resv message processing at switches simply pass this

along?

• If sending RSVP-TE client at end host asks for CO IP call (2205), then do not return MAC address.

• Software needed only at end hosts to handle MAC issue, provided intermediate switches simply pass on higher levels of label stacks.

• Test with SN16000 and Cisco GSR.• Test RSVP 2205 set up with GSR.

Documents

1/25/20051 CO internetworking (intra-domain + inter-layer) work in progress Malathi Veeraraghavan, Xuan Zheng, Zhanxiang Huang {mv5g, xuan, zh4c}@virginia.edu