MetroEthernet MetanoiaInc Next Gen Workshop 2007-07-17

8/13/2019 MetroEthernet MetanoiaInc Next Gen Workshop 2007-07-17

1/97

Metro Ethernet:

Understanding Key Underlying

Technologies

Copyright 2007

All Rights Reserved

Metanoia, [email protected]+1-888-641-0082http://www.metanoia-inc.com

Metano ia, Inc .Critical Systems Thinking
http://www.metanoia-inc.com/http://www.metanoia-inc.com/http://www.metanoia-inc.com/http://www.metanoia-inc.com/


2/97

Copyright 2007

All Rights Reserved

Metano ia, Inc.Critical Systems Thinking

Next-Generation Systems & Networks Workshop, 17thJuly. 2007, Bangalore, India 2

Who is Metanoia, Inc.?

Special ty techno logy con sultancyfounded in mid-2001, with HQ in Mountain View, California

Undertakes d eep-dive technical cons ult ingin telecom network, systems, software and chiparchitecture and design for clients across the world

Services have spanned4 cont inents, with clients in: North America, Europe, Asia, and Australia.

Principals provided services intechno logy strategies, architecture and design trade-of fs, productdevelopment, hardware/sof tware architecture, and know ledge enhancementto organizations thatinclude large equipment manufacturers, international, national and regional ISPs, premier metro/access

systems startups, network planning tool vendors, established software and technology houses andleading component and semiconductor vendors

Principals are technologis ts at the forefront of new developments, as leaders, creators,implementers, researchers, academics, strategists, and advisors in the US and abroad

Expertise spans Layer 1 through Layer 4, and wireline (optical, Ethernet, IP/ATM, SONET/SDH)through wireless (Wi-Fi, cross-layer design, Wi-Max, cellular data, 2.5-3G)

125+ man yearsof technology design and developm ent, and technology management experience,

having worked at leading global corporations, such as Apple, AOL Time Warner, BBN, Cisco, 3Com,Fujitsu, LSI Logic, Motorola, Tellabs, Siemens, Nokia, Tibco, and Qualcomm, and having workedat/consulted to corporates in the US and abroad for almost the last decade

70+ patents collectively issued/pending

Advanced graduate degrees from some of the most distinguished universities in the worldtheUniversity of California, Stanford University, Iowa State University, the University of Texas, theUniversity of Waterloo, and the Indian Institute of Technology


3/97

Copyright 2007

All Rights Reserved



Workshop Outline

Legacy networks & Ethernet over legacy networks Value propositions and business drivers

Ethernet over SDH/SONET

Metro Ethernet Forum (MEF)

MEF architecture

E-Line and E-LAN services

Native Ethernet as Carrier-class transport

Provider Bridges

Provider Backbone Bridges (PBB), Provider Backbone Transport (PBT)

MPLSan enabler for Ethernet services

Layer 2 VPNs: VPWS, VPLS, H-VPLS

Advanced concepts: traffic engineering, QoS, OAM, resilience

Conclusions


4/97

Ethernet over

Legacy Networks



5/97

Copyright 2007

All Rights Reserved



Issues with Legacy Networks

Low bandwidth

No flexibility to scale

High cost of installation

Slow provisioning

Bandwidth growth inflexible/non-linear Limited by multiplexing hierarchy

TDM-based access: inefficient for converged data


6/97

Copyright 2007

All Rights Reserved



Next-Generation SDH

NG ADM

NG ADM

NG ADM

Ethernet

Ethernet

Central

Office

Switch

Core

NetworkCustomer

NetworkSTM/4/16

RingCross

Connect

CustomerNetworkNG-SDH

NG-SDH

NG-SDH

Customer

Network

Customer

Network


7/97

Copyright 2007

All Rights Reserved



Ethernet-over-SDH

Framing protocol Encapsulates Ethernet frames in SDH payloads

Mapping of SDH payload to SDH channels

Vir tual concat.: for allocation of non-contiguous VCs

Flow control mechanism

Avoids packet drops due to speed mismatch between SDH and

Ethernet

Mechanism to increase/decrease allocated SDH bandwidth

Add or remove VCs


8/97

Copyright 2007

All Rights Reserved



Ethernet-over-SDH (contd)

Very popular in carriers with installed base of SDH rings E.g. BSNL in India

Good deployment choice when traffic primarily circuit

switched

Inefficient if major traffic is bursty packet-switched data

Solution: Carrier-class Ethernet!


9/97

Copyright 2007

All Rights Reserved



Metro Ethernet Value Propositions

Lower per-user provisioning costs Technically simple relative to TDM ckts.

Due to large installed base

Efficient and flexible transport

Wide range of speeds: 128 Kbps--10 Gbps

QoS capabilities

Ease of inter-working

Plug-and-play feature

Ubiquitous adoption

Thetechnology of choice in enterprise networks


10/97

Copyright 2007

All Rights Reserved



Ethernet Business Drivers

Business connectivity Storage networks

Data centers

Video conferencing

Residential services

Triple-play services (IPTV)

On-line gaming

High-speed Internet access

Wireless backhaul

Reduced cost, complexity for mobile operators


11/97

Metro Ethernet Services



12/97

Copyright 2007

All Rights Reserved



Metro Ethernet Forum (MEF)

Industry forum at forefront of Carrier Ethernetstandardization

Carrier Ethernet architecture

Ethernet services

Founded in 2001. Currently approx. 120 members

Technical Sub-committees

Architecture

Services

Protocols and Transport

Management


13/97

Copyright 2007

All Rights Reserved



MEN Architectural Components

13

End

User

Customer

NetworkMEN

Customer

NetworkEnd

User

S

T T

S

UNI Reference Point UNI Reference Point

Ethernet Virtual Connection

End-to-End Ethernet Flow

End user Interface End user Interface

Ethernet Flow Unidirectional stream of Ethernet frames

UNI Interface used to interconnect MEN subscriber to provider

EVC Defines association between UNI for delivering Ethernet flow across MEN


14/97

Copyright 2007

All Rights Reserved



Application Service

Layer(IP, MPLS, PDH, E1/E3, SDH)

Ethernet ServiceLayer

Transport Service

Layer(802.1, SONET/SDH, MPLS)

MEN Layer Model

MEN Layer Model


15/97

Copyright 2007

All Rights Reserved



MEF Services Definition Framework

Service Type Construct used to create broad range of services

Service Attributes

Defines characteristics of a service type

Attribute Parameters

Set of parameters with various options


16/97

Copyright 2007

All Rights Reserved



Service Types

E-Line Point-to-point Ethernet Virtual

Circuit (EVC)

E-LAN

Multipoint-to-multipointEthernet Virtual Circuit

16

EVC1

EVC2


17/97

Copyright 2007

All Rights Reserved



Service Attributes

Physical Interface

Medium, speed, mode, MAC layer

Traffic Parameters

CIR, CBS, PIR, MBS

QoS Parameters

Availability, delay, jitter, loss

Service Multiplexing

Multiple instances of EVCs on a given physical I/F

Bundling

Multiple VLAN IDs (VID) mapped to single EVC at UNI

M t i I


18/97

Copyright 2007

All Rights Reserved



Ethernet Services

Ethernet Private Line (EPL) Uses E-Line

Does not allow service multiplexing

High degree of transparency

Low delay, delay variation, and packet loss ratio

Ethernet Virtual Private Line (EVPL)

Uses E-Line

Allows for service multiplexing

Need not provide full transparency

M t i I


19/97

Copyright 2007

All Rights Reserved



Service Types and Ethernet Services

Service Types

E-Line

(p2p connectivity)E-LAN

(mp2mp connectivity)

Ethernet Private

Line (E-line)

Ethernet Virtual

Private Line (E-VPL)Ethernet Private

LAN (E-LAN)

Ethernet Virtual Private

LAN (E-VPLAN)

Ethernet Services


20/97

Native Ethernet as

Carrier-class Transport


M t i I


21/97

Copyright 2007

All Rights Reserved



Requirements for Carrier-class Ethernet

Scalability

Network should support millions of subscribers

Protection and restoration

50ms resilience

Quality-of-Service (QoS)

Ability to offer differentiated levels of service

Service Monitoring and Fault Management

Support for TDM traffic

Seamless integration with legacy networks

Metano ia Inc


22/97

Copyright 2007

All Rights Reserved



Ethernet Ring

Ethernet

Switch

Ethernet

Ethernet

Ethernet

Switch

Ethernet

Switch

Ethernet

Switch

1/10 GigabitEthernet Ring

Core

Network

Customer

Network

CustomerNetwork

Metano ia Inc


23/97

Copyright 2007

All Rights Reserved



Native Ethernet in Metro Access

How does one create the notion of a vir tual circu i t?

VLAN tagging with point-to-point VLAN

VLAN stacking

Outer tag service instance; Inner tag individual customer

802.1Q in 802.1Q (Q-in-Q) - IEEE 802.1ad

C-DA: Customer Destination MAC

C-SA: Customer Source MAC

C-TAG: IEEE 802.1q VLAN Tag

C-FCS: Customer FCS

S-TAG: IEEE 802.1ad S-VLAN Tag

C-DA C-TAGC-SA Client data FCSS-TAG

6bytes 6bytes 4bytes 4bytes 4bytes

Metano ia Inc

P id B id (IEEE 802 1 d)


24/97

Copyright 2007

All Rights Reserved



Customer

Network

Customer

Network

Customer

Network

24

Provider Bridge (IEEE 802.1ad)Architecture

CE: Customer Equipment

UNI: User-to-Network Interface

CES: Core Ethernet Switch/Bridge

P-VLAN: Provider VLAN

UNI-B

CES

CES

CE-A

UNI-A

UNI-C

CE-C

Spanning tree

CE-B

CES

Metano ia Inc


25/97

Copyright 2007

All Rights Reserved



Limitations of Provider Bridge Scalability

Limited to 4096 service instances

Core switches must al lMAC addresses

Broadcast storms ensue due to learning

MAC address tables explode!

Metano ia Inc


26/97

Copyright 2007

All Rights Reserved



Provider Backbone Bridging (802.1ah)

Encapsulate customer MAC with provider MAC at edge Edge switch adds 24-bit service tag (I-SID), not VLAN tag

Core switches need on ly learnedge switch MAC adds.

S-TAG: IEEE 802.1ad S-VLAN Tag

B-DA: IEEE 802.1ah Backbone Destination

B-SA: IEEE 802.1ah Backbone Source MAC

I-TAG: IEEE 802.1ah Service Tag

B-DA B-TAGB-SA I-TAG C-DA C-TAGC-SA Client data B-FCS

6bytes 6bytes 6bytes6bytes4bytes 5bytes 4bytes 4bytes

Metano ia Inc

P id B kb B id i (PBB)


27/97

Copyright 2007

All Rights Reserved



Provider Backbone Bridging (PBB)Architecture

CPE BCPE ACPE C

Provider backbonenetwork (802.1ah)

CPE BCPE A

802.1ad

CPE B

CPE B

802.1q

CPE C

Provider backbone

network (802.1ad)

CPE D

CPE D

CPE C

CPE A

Provider backbone

network (802.1ad)

Provider backbone

network (802.1ad)

Provider backbone

network (802.1ad)

Metano ia Inc


28/97

Copyright 2007

All Rights Reserved



Benefits of PBB

Scalability Addresses limitations of 4096 service instances

Robustness

Isolates provider network from broadcast storms

Security

Provider need switch frames onlyon provider addresses

Simplicity

Provider & customers can plan networks independently

Metano ia, Inc.


29/97

Copyright 2007

All Rights Reserved



Traffic Engineering in PBB

Via Multiple Spanning Tree Protocol (MSTP)

Maps a VLAN to ST or multiple VLANs to ST

Enables use of links that would otherwise be idle in ST

Eliminates wasted bandwidth but

Too slow for protection switching

Not suitable for complex mesh topologies

Difficult to predict QoS

Metano ia, Inc.

Ch ll ith All Eth t


30/97

Copyright 2007

All Rights Reserved



Challenges with an All-EthernetMetro Service

Restriction on # of customers

4096 VLANs!

Service monitoring

Scaling of Layer 2 backbone

Service provisioning

Carrying a VLAN is not a simple task!

Inter-working with legacy deployments

Need hyb r id archi tectures

Multiple L2 domains connected via IP/MPLS backbone

Metano ia, Inc.


31/97

Copyright 2007

All Rights Reserved

,Critical Systems Thinking


What Solutions do we Have?

Ethernet-based Architecture

Provider Bridge (802.1ad) in edge

Provider Backbone Transport (PBT) in Core

Hybrid Architecture

802.1ad in the edge Multiprotocol Label Switching (MPLS) in core

Metano ia, Inc.


32/97

Copyright 2007

All Rights Reserved



Provider Backbone Transport (PBT)

Connection-oriented, traffic-engineered Ethernet tunnels

Replaces spanning tree control plane with either a:

Management plane

External control plane

No learning !

Forwarding info. provided by management plane

Forwarding done on MAC + VID (60-bit) address

VID is not network global; however, MAC + VID is

B-MAC identifies destination

B-VID identifies per-destination alternate paths

Metano ia, Inc.


33/97

Copyright 2007

All Rights Reserved



Customer

NetworkCustomer

Network

33

PBT Architecture

Central TE Module

SA : PE1

DA : PE2

VLAN 22

SA : PE1

DA : PE2

VLAN 33

PE1PE2

Metano ia, Inc.


34/97

Copyright 2007

All Rights Reserved

Critical Systems Thinking


Benefits of PBT

No learning Eliminates undesirable broadcast storms

Resolves MAC flooding problem

Addresses scaling by forwarding on MAC + VID-highly scalable

Protection

Sets-up backup paths

50ms restoration possible

QoS support available

M t i I


35/97

MPLSAn Enabler forEthernet Services:

Fundamentals & Operations


Metano ia, Inc.


36/97

Copyright 2007

All Rights Reserved



Basic Concept of MPLS

Routing fills routing table

Signaling fills label forwarding table

DA Next hoprouter

N/wInt.

129.89.10.x 198.168.7.6 1

179.69.x.x 198.168.7.6 1

128.89.10.x

1

179.69.x.x

2

1

128.89.10.12

179.69.42.3

198.168.7.6

In

label

Out

labelAddress Prefix N/w

Int.

Advertises binding

Advertises binding

128.89.10.x5 1179.69.x.x7 2

Advertises bindings

128.89.10.x3 1179.69.x.x4 1

34

X

X

DA Next hoprouter

N/wInt.

129.89.10.x 129.89.10.1 1

179.69.x.x 179.69.42.3 2

Routing Table

In

label

Out


Int.

Label Table

R1 R2

R3

R4

Metano ia, Inc.


37/97

Copyright 2007

All Rights Reserved



Basic Concept of MPLS

128.89.10.x

1

179.69.x.x

2

1

128.89.10.12

179.69.42.3

198.168.7.6

In

label

Out


Int.

In

label

Out


Int.

128.89.10.x5 1

179.69.x.x7 2128.89.10.x3 1

179.69.x.x4 1

3

4

X

X

3

5

Packet arrives

DA=128.89.10.25

3Push

Label

5Pop

labelForward

packet

553

Swap

Label

R2R1

R3

R3 R4

Metano ia, Inc.

So what about MPLS Control and


38/97

Copyright 2007

All Rights Reserved



So what about MPLS Control andForwarding?

Supersetof conventional router control

Distribute info. via n/w layer routing protocols (OSPF, BGP, etc.)

Algos. to convert routing info. into forwarding table:

Create binding from FEC label

Assign & distribute labels to peer LSRs via signaling

Label switching forwarding table (or label information base LIB)

Forwarding algo = label swapping, independentof control

component (implementable in optimized H/W or S/W)

Control

Component

ForwardingComponent

First Subentry Second Subentry

(for multicast or load balancing)

Incoming Label

Map

Next hop label forwarding entry (NHFLE)

Outgoing label

Outgoing inf.

Next hop address

Outgoing label

Outgoing inf.

Next hop address

Incoming

Label

Metano ia, Inc.C iti l S t Thi ki What does a Label Represent? The


39/97

Copyright 2007

All Rights Reserved



What does a Label Represent? TheIssue of Label Granularity

Packets form Forwarding Equivalence Class (FEC)

Treated identical lyby participating routers

Assigned the samelabel

Membership in FEC must be determinable from IP header + other info. thatingress router has about the packet

Entities that may be grouped into an FEC are flexible. E.g. FEC could be:

Connection between two IP ports on two hosts or between IP hosts

Traffic headed for a particular network with same TOS bits

All destination networks with a certain prefix

Manually configured connection

Traffic belonging to a customer or department VLAN

Traffic of a given applicationvoice, video, plain data, management traffic

and many others

Metano ia, Inc.C iti l S t Thi ki


40/97

Copyright 2007

All Rights Reserved



Lets Recap: Elements of MPLS

Label Forwarding

Use data link addressing. E.g. ATM VPI/VCI, FR DLCI

Shim header between data link and IP header

Label Creation and Binding Label Assignment and Distribution

Ride piggyback on routing protocols, where possible (BGP)

Separate label distribution protocolRSVP, LDP

Variable

L2 header L3 IP headerMPLS shim

header

Higher Layers

4 bytes 20 bytes

LabelEXP/

CoS TTLS

20 bits 3 bits 8 bits

Data

Plane

Control

Plane

1 bit

Metano ia, Inc.Critical Systems ThinkingPrimary Label Assignment and


41/97

Copyright 2007

All Rights Reserved



Primary Label Assignment andDistribution Modes

4

33

Edge LSR

Edge LSR

Downstream-on-demand

with Independent Control

1 Requests

2

2Assignments

Edge LSR

2

35

6

Edge LSR

Downstream-on-demand

with Ordered Control

1 Requests

4

Assignments



42/97

Copyright 2007

All Rights Reserved



Advantages of MPLS

Original justification

Availability of fast, amortized, ATM hardware; emergence of H/W

forwarding engines has practically eliminated this

Current justifications

Separates forwarding from control, allowing Routing functionality to evolve independent lyof forwarding algorithm

MPLS to control non-packettechnologies: SONET/SDH ckts., lightpaths

Provides explicit, manageable IP routes

Enables pol icy rout ingand t raf fic engineer ing

Offers TE for Ethernet tunnels in metro-Ethernet environments

Facilitates scalable hierarchical routing



43/97

Copyright 2007

All Rights Reserved



The Utility of Hierarchical Label Switching

Core LSRs

Edge LSRs

Swapand Push Pop

Swap

Concept is similar to VLAN stacking in PBT we saw earlier



44/97

Copyright 2007

All Rights Reserved



Hierarchical Label Stacking/Switching

Inside a transit AS, each co re rou termust keep track of all

networks that might be reached through it

With hierarchical labels, only edge routersneed know whatnetworks might eventually be reached through them

A lltransit traffic can be made to tunnel through core routers

using LSPs with stacked labels

Metano ia, Inc.Critical Systems ThinkingExplicit Manageable Routes -- Policy


45/97

Copyright 2007

All Rights Reserved



Explicit Manageable Routes -- Policyrouting, Traffic engineering

Carriers want certaintraffic to go over certainroutes. Suchnetwork engineering:

Keeps network loads balanced

Enhances network stability and reliability

Enables better QoS and performance assurances

Allows carriers to meet customer SLAs

Constraint-based routing together with MPLS allows carriers to

Bind Ethernet tunnelsto an LSP,

Place (or ro ute)LSP over the desired sequence of LSRs in the n/w

TE tunnels are helpful for VPLS-based carrier Ethernet n/ws

Metano ia Inc


46/97

IP/MPLS-based Layer 2 VPNs




47/97

Copyright 2007

All Rights Reserved



L2 VPN Components

A

B

A

PE1 PE2

B

PE3

Routed

backbone

Emulated

LAN A

Emulated

LAN B

VC LSP

AC

What does the P1-PE2

connection really look like?



48/97

Copyright 2007

All Rights Reserved

y g


L2 VPN Component Details

PSN Tunnel

PWs

PE1 PE2

Emulated LAN

Interface

From CE

devices

PW Signaling

3

ForwarderBridgeModule

4

5

Emulated LAN

Instance

Routed backbone

with P routersFrom CE

devices

6

1 ACs 2



49/97

Copyright 2007

All Rights Reserved

y g


VPLS Network Overview

B

A

CE

B

A

CE

VSI

VSI

VSI

VSI

VSI

LAN Service

LAN Service

PW

(full mesh)

Tunnel

(full mesh)

L3/MPLS

Backbone

AC



50/97

Copyright 2007

All Rights Reserved Next-Generation Systems & Networks Workshop, 17thJuly. 2007, Bangalore, India 50

VPLS Protocols Involved

B

A

CE

B

CE

PE PE

EthernetSTP

MP-iBGP (PW) + RSVP-TE /LDP (tunnel)Targeted LDP (PW) + LDP (tunnel)

EthernetSTP

ControlPlane

Data

Plane

EthernetEthernet or

Ethernet in IP/

ATM/FR/SDH/

SONET

Ethernet/MPLS

Ethernet/IPSec

Ethernet/GRE

EthernetEthernet or

Ethernet in IP/

ATM/FR/SDH/

SONET

BGP/Targeted LDP

LSP or PSN Tunnel



51/97

Copyright 2007


Operational Characteristics of VPLS

Operational Requirement Realized Via

MAC address learning andswitching, work with 802.1p/qtags and VLANs

- VSI Forwarder- Bridge Module

Flooding pkts. with unknownsbroadcast, or multicast address

Frame replication on PWs

Provider edge signalinginformPE's to autoconfigure, and ofmembership, tunnelling

- Targeted LDP- BGP

VPLS membership discovery- BGP- Configuration

Inter-provider connectivity Globally unique VPLS ID

Metano ia, Inc.Critical Systems ThinkingData Plane: Flooding, Address


52/97

Copyright 2007


Data Plane: Flooding, AddressLearning and Forwarding

All address unknown frames (unicast, multicast, broadcast)

flooded over corresponding PWs to all relevant PEs only

B

A

CE

BA

CE

VSI

VSI

VSI

VSI

VSI

PE1PE2

PE3 PE4

PWs

Src. MAC = 09:10:01:45:00:AB

Dest. MAC = 08:00:69:02:01:FC1

?2

2

3

3



53/97

Copyright 2007


Address Learning

Layer 2 reachability directly learned in data plane

Use standard learning bridge functions for local MACs

PW-based association for remote MACs

Allow PE to determine from which physical port or LSP a given MAC

address came

VSI FIB keeps mapping between Ethernet MAC PW to use

Qualified Learning Unqualified Learning

- Each customer VLAN is its own

VPLS instance

- Has its own PW mesh and brdcast

domain

- All customer VLANs are part of

the same VPLS

- One PW mesh and single brdcast

domain



54/97

Copyright 2007


Address Learning Example

ACE

VSI

VSI

PE1 PE2

PE3

i/f1 i/f2i/f1

Dest.

MAC

VC

LabelOut I/FTunnel

1 Inbound

VC LSP Label = 1002

Outbound

VC LSP Label = 2001

Src. MAC = 08:AA:FC:01:10:DE (S1)

Dest. MAC = FF:FF:FF:FF:FF:FF (D1)

(broadcast)

2

Local Learning

3

4

Remote

Learning

S1 1002 i/f1-



55/97

Copyright 2007All Rights Reserved Next-Generation Systems & Networks Workshop, 17

thJuly. 2007, Bangalore, India 55

Forwarding and Encapsulation

Forwarding requires ability to Dynamically learn MAC addresses on

Physical ports

Pseudowire VCs (VC LSPs)

Forward/replicate pkts. across physical ports and VC LSPs

Encapsulation

PW header applied to Ethernet packet w/o preamble + FCS VLAN tag denoting customers VPLS instance can be stripped at

ingress, reapplied at egress

Metano ia, Inc.Critical Systems ThinkingTunnel and PW Topology and


56/97



u e a d opo ogy a dLoop Freedom

Full mesh of PW and tunnels deployed

Tunnels

Help transport the PW payload

Aggregate traffic from multiple PWs

Pseudowiresdemultiplex the L2 traffic traversing tunnels

A

CEB

ACE

VSI

VSI

VSI

VSI

VSI

PW

(full mesh)

Tunnel

(full mesh)

AC

Dest. MAC = 08:00:69:02:01:FC

PE1 PE2

PE3 PE4

?



57/97



Scaling VPLS: Hierarchical VPLS

Base VPLS requires full mesh of VC LSPs between PE routers

Adequate for PE routers in COmultiple customers aggregated

Inadequate for PE routers in MTU basements!

LSP explosion

Operational nightmare!

PE PE

PE

PEPE

MTU

MTU MTU

MTU

MTU



58/97



Hierarchical VPLS Advantages

Benefits

Simplifies signaling

Reduces pkt. replication

Simplifies MTU

Scalable inter-domain VPLS

Simplifies new site addition

PE PE

PE

PEPE

MTU

MTU MTU

MTU

MTU

Spoke

VCs

Hub PE

Core VC

LSP mesh

(VLL or Q-in-Q)

Metano ia, Inc.Critical Systems ThinkingHierarchical VPLS: Case Study for


59/97



ya Metro Region

100 MTUs; 10 customers/MTU; 2 VPLS/cust.; 100 stations/VPLS

VPLSs/MTU = 10x2 = 20

MACs/MTU = 20x100 = 2000

No hierarchy PE supports

2000 MACs

LDP/BGP sessions = (100x99)/2 x

20 = 245,000

Hierarchy (10 MTU/PE) PE supports

2000 x 10 = 20,000 MACs

LDP/BGP sessions = (10x9)/2 x 200 = 9000

# of spoke VLLs = 10 x 20 = 200

PE

PE

PEPE

MTU40

MTU1

MTU99MTU2

PEMTU 100

PEMTU3

CE

CE

CECE

MTU40

Hub PE

MTU91

MTU81MTU10

CE

MTU100

CE

MTU1

CEMTU31

CE

MTU90

PEPE

PE



60/97



Benefits of IP/MPLS-based L2 VPNs

Separation of administrative responsibilities

Migration from traditional L2 VPNs: seamless transport of Ethernet

services

Privacy of routing

Layer 3 independence

Less operational overhead

Ease of configuration (?)

Metano ia, Inc .


61/97

Advanced Features:Traffic Engineering,

Resilience, OAM, QoS


Metano ia, Inc .


62/97

Traffic Engineering Concepts

Copyright 2006All Rights Reserved




63/97



Constraint Based Routing

A class of routing systems that computes routes through a

network subject to a set of constraints and requirements

QoS-based Routing

Path of flows determined by

Knowledge of resource

availability in network

QoS requirements of flows

Policy-based Rou ting

Path/routing decision based

on administrative policy

Can be on-line or off-line



64/97



CB Routing System

Inputs

Flow/path attributes:

required b/w, hop count, ...

Resource attributes:

properties of nodes/links

Network topology & state

Outputs

Computed feasible path

Explicit route of the path

Constraint-Based

Routing Process

Attributes

Resources

Topology

Feasible Path

ERO {1,3,4,5}

1

3

4

5

2

Metano ia, Inc .


65/97

MPLS-based Resilience for the Metro





66/97



Fundamental Characteristics of RSVP

Allows apps. to signal QoS requests to n/w, and n/w to respond

with success or failure

Designed to transport

Classification info. (Sender_Template)

Allows flows with specific QoS reqs. to be recognized

Traffic specs of source/sender (Tspec)

QoS needs of receivers (Rspec)

Soft-state protocol Path/Resv transmitted periodically to refresh reservation

Refresh Reduction [RFC2961] has practically eliminated original

scalability concerns with use of soft state



67/97



Basic Operation of RSVP-TE

Path Message

Application for which RSVP

reservation is to be made

Identifies pkts. of the sender

Defines traffic output by sender

Request for label on this hop

Specific path to which flow isto be bound

LSP attributes for this sender

IP address of I/F that

transmitted Path Msg.

RSVP Header

SESSION

SENDER_TEMPLATE

SENDER_TSPEC

LABEL_REQUEST

SESSION_ATTRIBUTE

PHOP

ERO/RRO

Resv Message

Flow Descriptor

RSVP Header

SESSION

STYLE

LABEL

RRO

SENDER_TEMPLATE

NHOP

RSpec

Same as that in Path Msg.

Specifies senders that may

use the reserved resources

Label assigned to this hop

Record route taken by Path

QoS desired by receiver

Flow for which QoS is

desired

IP address of I/F originating

the Resv msg.

A B C D E

Path (Label_Req) Path (Label_Req)

Resv

Label=5Resv

Label=7

Resv

Label=49

Resv

Label=21

Metano ia, Inc.Critical Systems ThinkingFast Re-Route (FRR) using


68/97



LSP ID = L2

RSVP-TE

Rerouting is done when

A better path is available

Upon failure along LSP

Use SESSION Obj. & SE style

Tunnel uniquely identified by

Destination IP address

Tunnel ID

Ingress IP address

Tunnel ingress made to appear

as 2 different senders to the

RSVP session (via LSP ID)

Src

Rcvr

LSP ID = L1

On these links the

LSPs share resources

Tunnel ID in

Session Obj

Originates LSPs

with IDs 1 and 2

Here they are treated as di f ferent

LSPs within the same Session

LSPs 1 and 2 have a common SESSION Obj, but

a new LSP ID in the SENDER_TEMPLATE and a

different ERO (with possibly common hops)

Metano ia, Inc.Critical Systems ThinkingTE with Constraint-based Routing


69/97



in a Nutshell

Route Computation

Process(on-line (CSPF) or offline)

Enhanced IGP

Process(OSPF-TE)

Signaling Process

(RSVP-TE)Standard IGP

Process (OSPF)

Link State

Database

(LSDB)

Routing Table

(RIB)Computed

feasible path(ERO)

Operator Input

(Flow or LSPAttributes)

MPLS LSPs

(Label Info. Base)

TED

Forwarding

Info. Base (FIB)

LSP

Establishment Link Attribute

Modification

Output

Resource

Attributes

Network

Topology + State

Demand or Traffic drivenLSP path selection

Control driven route computation

and LSP path selection

CONTROL PLANE

DATA PLANE



70/97



How it All Fits Together

PE1

PE2

PE3

CE1

CE2

CE3

CE4

Last-mile Ethernet

PBB clouds

IP/MPLS Core

Pseudo-wires

Attachment circuits

-- Physical (PDH/SDN)

-- Logical (FR, ATM, VLANs, tunnels)

LSP Tunnels

Metano ia, Inc .


71/97

OAM: The Traditional Achilles Heel of

Ethernet





72/97



Why Ethernet OAM?

Current management protocols lack per-customer

granularity to handle Ethernet services

Most management protocols operate are point-to-point

Ethernet OAM can exploit multipoint capability

Link management required for last-mile connection

Similar to link mgt. in FR and ATM



73/97



Ethernet OAM Types

Service OAM

e2e connectivity and fault mgt. per service instance

Part of IEEE 802.1ag, CFM project

Link OAM

Monitoring & fault mgt of individual Ethernet link (physical/emulated)

Part of IEEE 802.3, Clause 57 (formerly 802.3ah (not to be confused

with 802.1ah))

Ethernet Local Mgt. Interface (E-LMI)

Configuration & operational provisioning of customer edge device

Part of MEF Standard MEF-16



74/97



Service OAM

Works on per-EVC basis

Independentof underlying transport technology

CFM messages

Continuity Check Message

Detects loss of service connectivity

Link Trace Message

Traces the path hop-by-hop (like IP traceroute)

Loopback Message Detects whether target point is reachable (like ICMP Ping)

AIS (Alarm Indication Signal) Message

Asynchronous notification to indicate fault



75/97



Link OAM

Discovery

Identifies devices at both ends of the link

Link Monitoring

Detects link faults

Statistics of packet errors

Remote Failure Indication

Conveys loss-of-signal indication to peers, due to poor SNR, power

failure, or other critical events

Remote Loopback

Determines quality of link during installation and troubleshooting



76/97



E-LMI

Provides local configuration & operational parameters to

customer edge

VLAN-EVC mapping

QoS profiles of EVC

Reduces configuration errors, improves performance

Dynamic EVC management

Metano ia, Inc .


77/97

Quality-of-Service: Ah! that elusive QoS



Metano ia, Inc.Critical Systems ThinkingMPLS and Quality-of-Service for

Eth t S i


78/97



Ethernet Services

MPLS supports (no textends) a packet-based QoS model

MPLS does notrun in hosts (only in metro/core routers)

QoS, however, is an end-to-endmechanism

MPLS helps carriers offer QoS-enabled services efficiently

Can support MEF QoS model via DiffServ QoS framework


Diff ti t d S i F k


79/97



Differentiated Services Framework

Traffic flows aggregated into small # of classes

Per-flow state is not required

More scalable than IntServ

EF

AF1x

AF2x

AF3x

AF4x

PriorityDrop Precedence

123

Class DSCP

001xx0

01xx10

1xxx10

11xx10

101110

Class encoded in IP header via

DiffServ Code Point (DSCP)

Edge router

Classifies packets to DifServ classes

DSCP identifies Per Hop Behavior(PHB)

Best Effort (BE)

Expedited Forwarding (EF)

Minimal delay & loss

Assured Forwarding (AF)

4 classes

3 drop precedences each

12 possibilities total

BE


Diff ti t d S i A hit t


80/97



Differentiated Services Architecture

Diffserv Domain

WFQ

Strict

Priority

EF

AF

BE

Core Functions

Queueing

Scheduling

Aggregate

PHBs

Colored packet

(marked DSCP)

Classifier Marker

Meter

Shaper

Traffic Conditioning

Edge Functions

Metano ia, Inc.Critical Systems ThinkingMPLS Support of DiffServ:

M i DSCP t LSP ( l b l )


81/97



Mapping DSCPs to LSPs (or labels)

Map DSCP EXP bits in MPLS shim header

6 DS bits (64 PHBs) and only 3 EXP bits (8 classes)!

Complete mapping is infeasible

For many practical cases, 8 PHBs may suffice

Results in an LSP called an E-LSP

Label EXP TTLSDSCP

6 bits

IP Header

DSCP

3 bitsDS byte

MPLS shim header

Metano ia, Inc.Critical Systems ThinkingMPLS Support of DiffServ:

M i DSCP t LSP ( l b l )


82/97



Mapping DSCPs to LSPs (or labels)

Map {PHB, FEC} MPLS Label

That is, provide the info. in the label itself!

Requires enhancing the label distribution protocols

Use EXP bits for drop precedence

That is to determine different PHBs of a PHB scheduling class

Label EXP TTLSDSCP

6 bits

DSCP

3 bitsDS byte

DS class dropprecedence

DS class: EF, AFx

IP Header MPLS shim header

Results in an LSP called an L-LSP

Metano ia, Inc .


83/97

Conclusions and Discussion



C l i


84/97



Conclusions

Ethernet poised to be dominant choice in metro networks

Reduces capex and opex for providers

Enables new revenue generating services

802.1ad provider bridge with OAM of 802.1ag a choice at the edge

Two architectures emerging for Ethernet in the metro core

Provider Backbone Transport (PBT)

IP/MPLS-based L2 VPNs

Metano ia, Inc .


85/97

Thank You!

Questions?



Glossary


86/97



Glossary

AC Attachment CircuitACL Access Control ListAF Assured ForwardingAPI Application Programming InterfaceAS Autonomous SystemATM Asynchronous Transfer ModeBA Behavior AggregateB-DA Backbone Destination AddressB-DA Backbone Source AddressBE Best EffortB-FCS Backbone Frame Check SequenceBGP Border Gateway ProtocolCBS Committed Burst SizeCE Customer Edge (router)CES Core Ethernet Switch/BridgeCFMCIR Committed Information RateCO Central OfficeDA Destination AddressDS DiffServ

DS DiffServDSCP DiffServ Code PointEF Expedited ForwardingE-LMI Ethernet-Local Management InterfaceE-LSP EXP mapped LSPEPL Ethernet Private LineERO Explicit Route ObjectE-UNI Ethernet UNIEVC Ethernet Virtual CircuitEVPL Ethernet Virtual Private LineEXP Experimental (EXP bits in MPLS "shim"header)EXP Experimental BitsFCS Frame Check SequenceFEC Forwarding Equivalence ClassFIB Forwarding Information BaseFR Frame RelayGR Graceful RestartH-QoS Hierarchical Quality-of-ServiceH-VPLS Hierarchical VPLSIPTV IP Television


Glossary


87/97



Glossary

L2 Layer 2 (Data Link Layer; MAC Layer)L3 Layer 3 (Network or IP Layer)LAN Local Area NetworkLDP Label Distribution ProtocolLER Label Edge RouterLIB Label Information BaseL-LSP Label inferred LSPLSP Label Switched PathLSR Label Switching RouterMAC Medium Access ControlMBS Maximum Burst SizeMEF Metro Ethernet ForumMEN Metro Ethernet ArchitectureMPLS Multi-Protocol Label SwitchingMSTP Multiple Shortest Path TreeMTU Multi-Tenant UnitNG Next GenerationNGN Next-Generation NetworkNNI Network Network InterfaceOAM Operations, Administration, and Management

OSPF Open Shortest Path FirstP Provider (router)PB Provider BridgingPBB Provider Backbone BridgingPBT Provider Backbone TransportPDH Pleisosynchronous Digital HierarchyPE Provider Edge (router)PHB Per Hop BehaviorPIR Peak Information RatePSN Packet Switching NetworkP-VLAN Provider VLANPW Pseudo-WireQoS Quality-of-ServiceRIB Routing Information BaseRSTP Rapid Spanning Tree Protocol

RSVP-TEResource Reservation Protocol - Traffic

Engineering (RSVP protocol with MPLS

traffic engineering extensions)SA Source AddressSDH Synchronous Digital HierarchySONET Synchronous Optical Network


Glossary


88/97



Glossary

SPT Shortest Path TreeST Spanning Tree ProtocolSTP Spanning Tree ProtocolTDM Time-Division MultiplexingTE Traffic EngineeringTM Traffic ManagementTTL Time to LiveUNI User Network InterfaceVCI Virtual Circuit IdentifierVFI Virtual Forwarding InstanceVID VLAN IdentifierVLAN Virtual LANVLAN Virtual LANVOQ Virtual Output QueueVPI Virtual Path IdentifierVPLS Virtual Private LAN ServiceVPN Virtual Private NetworkVPWS Virtual Private Wire ServiceVR Virtual Router

VRF Virtual Routing and ForwardingVSI Virtual Switching InstanceWFQ Weighted Fair Queuing


Readings and References (1)


89/97




MEF 4: Metro Ethernet Network Architecture Framework Part 1 Generic

Framework

MEF 6: Metro Ethernet Services Definition Phase 1

MEF 10.1: Metro Ethernet Services Attributes Phase 2

MEF 16: Ethernet Local Management Interface

IEEE 802.1d/q WG: Media Access Control (MAC) Bridges, IEEE 1998

IEEE 802.1s, Multiple Spanning Tree, IEEE 2002

IEEE 802.1ah, Provider Backbone Bridges, Work in Progress

Documents on the MEF and IEEE 802.1 and 802.3 WG web sites




90/97




L. Andersson and E. Rosen, Framework for Layer 2 Virtual Private

Networks (L2VPNs), RFC 4664, September 2006

K. Kompella and Y. Rekhter, Eds., Virtual Private LAN Service: Using

BGP for Autodiscovery and Signaling, RFC 4761, January 2007

V. Kompella and M. Lasserre, Eds., Virtual Private LAN Service: UsingLabel Distribution Protocol for Signaling, RFC 4762, January 2007

S. Bryant and P. Pate, Eds. Pseudo Wire Emulation Edge-to-Edge (PWE3)

Architecture, RFC 3985, March 2005

L. Martini et al, Eds., Pseudowire Setup and Maintenance Using the Label

Distribution Protocol (LDP), RFC 4447, April 2006

Documents on the L2 VPN, PWE3, MPLS, and CCAMP WGs of the IETF

Metano ia, Inc .C iti l S t Thi ki


91/97

Additional Slides


Metano ia, Inc.Critical Systems ThinkingLabel Assignment and Distribution

(control component)


92/97



(control component)

Downstream Upstream

Ordered Solicited (On Demand)Unsolicited

SolicitedUnsolicited

Independent Solicited (On Demand)Unsolicited

SolicitedUnsolicited

Directionfromwhich labels flow

Refers to whether LSR distributes

labels on demand or voluntarily

Whether LSR waits to hear from

its upstream/downstream nbrs.

before responding to a request

for label(s)

Label Retention: Liberal or Conservative

Whether LSR keeps labels from a neighbor

who is not currently the next hop for a FEC

Labels

Data

Labels

Data


A Word on Reservation Styles


93/97



A Word on Reservation Styles

Always chosen by the receiver

Two styles apply with RSVP-TE

Fixed Filter (FF)

Dist inctreservation for traffic

from each sender Needs uniqu e labelper sender

Shared Explicit (SE)

Commonresvn. for traffic from

the senders specified by rcvr.

May assign unique label/sender

Useful for p2p or mp2p LSPs

Distinct reservationper sender

S1

S3

Link (i,j)

Unique label/sender

S2

Common reservation

shared by all senders

S1

S3

Link (i,j)

Different senders may

have different labels

S2


LDP versus BGP Signaling


94/97



LDP versus BGP Signaling

LDP session full mesh b/ween PEs

PEs exchange labels directly

New PE reconfig. mesh at allPEs

FIB per VPLS per PE

RRs reduce full mesh to 2 sessions/PE

Cannot direct label mapping to a

specific peer need label ranges

New PE peering session only w/ RRs

BGP-based SignalingTargeted LDP

i-BGP

PE

PE

PE

PE

PERR

TargetedLDP

PE

PE

PE

PE

PE


L2 VPNS with BGP


95/97

Copyright 2007All Rights Reserved Next-Generation Systems & Networks Workshop, 17thJuly. 2007, Bangalore, India 95

L2 VPNS with BGP

Autodiscovery + signaling, together via BGP with RTs (per slide 74)

PE configured with its VPLS ID (if VPLS)

Transmits VPLD ID or identity of attached CEs to peer PEs

Includes demux value for each BGP NLRI (as a label range)

Selection algorithm allows each remote PE to pick correct label for

sending traffic to advertising PE

BGP NLRI for L2 VPNBGP NLRI for VPLS

Length (2 octets)

RD (8 octets)

VE ID (2 octets)

VE Block Offset (2 octets)

VE Block size (2 octets)

Label Base (3 octets)

Length (2 octets)

RD (8 octets)

CE ID (2 octets)

Label blk offset (2 octets)

Circuit Status Vector

Label Base (3 octets)


BGP-based L2 VPN (VPWS)


96/97



BGP-based L2 VPN (VPWS)

PE1

PE2

PE3

1003

3001

CE1

CE2

CE3

CE4

DLCI=[101, 102, , 120]

DLCI=[11,12,, 30]

IP/MPLS

Core

Label block offset=0

Label base = 3000

Label range = 20


Label base = 1000

Label range = 20

10311

12

3002

DLCI=[401, 402, , 420]


Label base = 2000

Label range = 20

403

2003


BGP-based L2 VPN (VPLS)


97/97

BGP-based L2 VPN (VPLS)

PE1

PE2

PE3

3001

CE1

CE2

CE3

CE4

IP/MPLS

Core


Label block size = 10

Label base = 3000

3002VE ID = 3