Embed Size (px)
Citation preview
Carrier Ethernet Services -The Future
Public Multi-VendorInteroperability Test
Singapore, November 2008
2
Carrier Ethernet World Congress 2008 Multi-Vendor Interoperability Test
EDITOR’S NOTE
Carrier Ethernet servicedeployments are rapidlytaking up speed in many AsiaPacific countries — CarrierEthernet solutions are wellpositioned for the large-scale,broadband service providernetworks in the region.Many (if not most) end-to-endCarrier Ethernet networksemploy equipment frommultiple vendors today.
Service providers typically implement a dual-vendorstrategy separately for core backbone, metro aggre-gation and access components. Robust interopera-bility across the board for all vendors and types ofequipment is an important cornerstone of CarrierEthernet’s success story.For the first time in Singapore, EANTC showcases theresults of a large-scale live interoperability testfocusing all aspects of end-to-end Carrier Ethernetnetworks.More than 30 engineers from ten vendors with over35 systems participated in the hotstaging preparation of this test atEANTC’s lab in Berlin, Germany,in August.The participating vendors verifiedmore than 20 test areas in any-to-any combinations in ten days.Carrier Ethernet implementationssupport more functions and covermore markets today — rangingfrom core to microwave toaccess, E-Lines to E-Tree and tripleplay to mobile backhaul.It was a great experience towitness the testing session, aunique get-together of manyleading players with one singlegoal: To improve multi-vendorinteroperability of advancedCarrier Ethernet implementations.Interestingly, market forces areoperating at full strength. Thisyear, we once again tested three transport technol-ogies in the test event’s metro/aggregation networks:MPLS, PBB-TE and T-MPLS. These three compete tosome extent — at our test, they all proved being wellsuited for the transport of Carrier Ethernet services.We noticed that service OAM support is becomingmandatory for aggregation and CPE devices. Inaddition we saw substantial advances in Ethernetmicrowave solutions.This white paper summarizes in detail the monumentaleffort that the participating vendors and EANTC teamunderwent. Enjoy the read.
INTRODUCTION
The interoperability event focused on the Future ofCarrier Ethernet Services. EANTC has beenconducting such events at the Carrier Ethernet WorldCongress in Europe since its inception in 2005 andwhile each previous event concentrated on specifictopics such as mobile backhaul or service creation,this event aimed to congregate the knowledge andexperience the industry gained in the last four yearsinto a single modern, converged network thus demon-strating the state of the art for all technologies that atier-one service provider is likely to encounter. Wetherefore tested:• Converged residential and business services
realized using E-Line, E-LAN and for the first timeE-Tree services
• The leading access, transport and aggregationtechnologies
• Microwave access and transport• Ethernet OAM: Fault management and perfor-
mance monitoring• High availability• Management and SLA reporting
EANTC’s interoperability showcasesare driven by three main goals:Technical – Through participation inthe event, vendors have the oppor-tunity to verify the interoperability oftheir devices and protocol implemen-tations against the majority of theindustry’s leading vendors.Marketing – The participants canshowcase the interoperability of theirlatest solutions on a unique, large-scale platform.Standards – When fundamental issuesare found during the hot staging eventEANTC reports the discoveries to thestandard bodies. These in turn updatethe standards.The test plan was created by EANTCbased on the test topics suggested byvendors, our service provider panel
and experience gained from previous events, and waslined up with recent IEEE, IETF, ITU-T and MEFstandards.
Service Provider Test Plan ReviewThe test plan was reviewed by a panel of globalservice providers in July this year. Their feedback andcomments were reflected in the final version of the testplan. EANTC and the participating vendors would liketo thank: COLT, GVT Brazil, PT TELKOM Indonesia,T-Systems and Metanoia Inc.
Carsten RossenhoevelManaging Director
TABLE OF CONTENTS
Participants and Devices..............3Network Design .........................3Test Results:Ethernet Service Types.................3Diverse Access Technologies ........4Diverse Transport ........................5MPLS Core.................................6E-NNI........................................6Ethernet OAM ............................8Resilience and Fault Detection ......9Acronyms ................................11References ...............................11
3
Participants and Devices
PARTICIPANTS AND DEVICES
NETWORK DESIGN
As in previous events we set off to construct a networkthat would allow all participating vendors to establishend-to-end Ethernet services with any of the othervendors. One of the central design considerations forthe network was to enable any device positioned inthe access network to reach any other access networkdevice regardless of the other device’s point ofattachment to the network.We also aimed to build a network that would lookfamiliar to service providers. It is perhaps unrealisticto expect that service providers will incorporate allcurrent transport technologies into their network.Nevertheless the familiar network domains are likelyto exist: access, aggregation, metro and core,regardless of the chosen transport technology. It isalso realistic to expect service providers to use MPLSin the core of their network.Looking at the network from a customer’s perspective,we used the following network areas:• Access: The devices that normally exist at the
customer premise were positioned here. We werelucky to see a diverse number of access technol-ogies for transporting Ethernet such as microwavelinks, copper, and fiber. These devices imple-mented the UNI-C construct as defined by theMEF.
• Aggregation: The aggregation area of a networkconsisted of a variety of solutions meant toaggregate customer premise devices. Thisincluded Provider Bridges and H-VPLS Multi-Tenant Unit Switches (MTU-s). When applicable
these devices performed the UNI-N role in thenetwork.
• Metro: Three different transport technologies wereused in each of the three metro area networks:MPLS, PBB-TE and T-MPLS. This allowed eachtransport technology to test its own resiliency andNetwork-to-Network Interface (NNI) solutions.
• Core: As stated above, IP/MPLS was used tosupport connectivity between the different metroarea networks in order to realize end-to-endservices. In addition, MPLS Layer 3 VPNs asdefined in RFC 4364 were tested in the core ofthe network.
The physical network topology presented here depictsthe roles of all the devices and their respectiveplacement in the network. Please note that many testsrequired logical connectivity between the devices,often at an end-to-end nature, which will be shown,where applicable, using logical topologies in eachtest section.
In the next sections of the white paper we describe thetest areas and results of the interoperability event. Thedocument generally follows the structure of the testplan.Please note that we use the term »tested« whenreporting on multi-vendor interoperability tests. Theterm »demonstrated« refers to scenarios where aservice or protocol was terminated by equipment froma single vendor on both ends.
ETHERNET SERVICE TYPES
The Metro Ethernet Forum (MEF) has defined threeEthernet service types in order to allow the industryand specifically the customers interested in theservices to have a common language to discuss suchEthernet based services. The three service types aredefined in terms of the Ethernet Virtual Connection(EVC) construct:• E-Line – Point-to-point EVC• E-LAN – Multipoint-to-multipoint EVC• E-Tree – Rooted-multipoint EVCWhile the E-Line service type provides a service toexactly two customer sites, the E-LAN and E-Treeservice types allow the connection of more than twocustomer sites. In contrast to the E-LAN service typewhich allows an any-to-any connectivity betweencustomer sites, E-Tree introduces two different roles forcustomer sites: leaf and root. An E-Tree service facili-tates communication between leaves and roots,however, leaves can not communicate with each otherdirectly. An E-Tree service implemented by a rooted-multipoint EVC can be used to provide multicast trafficdistribution and hub-and-spoke topologies (e.g. DSLcustomers to BRAS).In the test network we instantiated three specificdefinitions of service types: Ethernet Virtual PrivateLine (EVPL), Ethernet Virtual Private LAN (EVP-LAN),and Ethernet Virtual Private Tree (EVP-Tree). Allservices were configured manually in the network.Due to the large number of devices present at the hotstaging, this process was time consuming and proneto mistakes. A multi-vendor provisioning tool wouldhave been ideal for the testing and is recommended
VendorParticipatingDevices
Alcatel-Lucent 1850 TSS-405650 CPAM7450 ESS-67705 SAR7750 SR79500 MPR
ECI Telecom SR9705
Ericsson Marconi OMS 2400
Harris Stratex Networks Eclipse (Gigabit) Radio
Ixia XM2 IxNetwork
Juniper Networks M10iMX240MX480
Nortel Metro Ethernet RoutingSwitch (MERS) 8600
Spirent Communications Spirent TestCenter
Tejas Networks TJ2030
Tellabs 8830 Multiservice Router
4
Carrier Ethernet World Congress 2008 Multi-Vendor Interoperability Test
for any service provider planning to deploy CarrierEthernet services.The services created in the network were configuredin two ways:• EVCs that remained within the same metro area
network• EVCs that crossed the network coreThe sections below describe the services in thenetwork in detail.
E-TreeFor the first time at an EANTC interoperability event,an E-Tree service instantiation was established. OneEVP-Tree was configured with one root node withinthe MPLS metro area and leaves throughout allnetwork areas. The MEF defines an E-Tree service tobe a rooted Ethernet service where the roots are ableto communicate with all leaves, and all leaves areable to communicate with the roots, but not with eachother.This service utilized each metro technology in aunique way. The MPLS metro and core each used aseparate VPLS instance to create this service, usingdifferent split horizon groups to ensure that leaf UNIscould only communicate with the root UNI, but couldnot establish communication between each other. ThePBB-TE and T-MPLS metros respectively configuredPBB-TE trunks and TMCs to a static rule set in order topropagate the tree service.Devices included in this test were Alcatel-Lucent 7750SR7, ECI SR9705, Ericsson Marconi OMS 2400,Harris Stratex Eclipse, Juniper MX480, Nortel MERS8600, Tejas TJ2030, and Tellabs 8830.
E-LANOne EVP-LAN was configured in the network withcustomer ports in all three metro areas. Theconstruction of the EVP-LAN service used differentmechanisms in each metro area. These mechanismsare described in details in the diverse transportsection.
E-LineThe E-Line service type configured in the network usedVirtual LAN (VLAN) IDs to distinguish between thevarious services. In some cases, much like real worldnetworks, a switch positioned at the customer sitewould add a Service VLAN tag (S-VLAN) to theEthernet traffic provided by the customer, therefore,allowing the customer to maintain its private VLANaddressing scheme and separate the customer VLANspace from the provider’s.All vendor devices successfully participated increation of E-Line services. From the number of combi-nations tested, we are confident that an any-to-anycombination of endpoints is possible.
DIVERSE ACCESS TECHNOLOGIES
The different services in the test network used a varietyof access technologies to reach the simulated last milecustomer access device. Most services used fiber(multi-mode) and copper based Gigabit Ethernet.Several Ethernet access links comprised of twoEthernet links with a microwave signal in between.These systems are described in more detail below.
Microwave for Access andTransportIn recent years we have seen an increased interest inour interoperability events from vendors offeringmicrowave connectivity and network solutions.Microwave solutions alleviate the need to roll outphysical wire infrastructure and are especiallyprevalent in such areas as cellular backhaul,emerging markets, large corporation networks,hospitals, and mobile-fixed operators. This eventenjoyed the participation of the Alcatel-Lucent 9500MPR and Harris Stratex Eclipse.Since the radios rely on a signal through the air someweather events such as rain and heavy fog can causethe signal to degrade effectively decreasing the rangeor capacity of the link. Radio devices can recognizethe decrease in air-link capacity and some solutionscan distinguish which frames should be prioritizedand further transported versus which frames will bedropped. The Alcatel-Lucent 9500 MPR and HarrisStatex Eclipse showed this functionality by decreasingthe modulation scheme in Quadrature AmplitudeModulation (QAM) which caused traffic loss only to
MPLS
User Network Interface (UNI)
UNI-N Metro Device
PBB-TE
UNI-N Aggregation Device
UNI-C Access Device
UNI-C Microwave Access Device
T-MPLS
Figure 1: E-Line service creation
5
Diverse Transport
best effort frames and no or minimal loss to prioritizedtraffic with unaffected latency.Services relying on microwave equipment will need tobe made aware when the microwave signal is tooweak to transmit traffic. The link state propagationfunction disables the Ethernet link state for all portsassociated with the microwave link. This functionalitywas demonstrated by the Harris Stratex Eclipse. Thisdevice also showed the capability to propagate anincoming loss of signal on a tributary Ethernet portacross the microwave link and switching off theappropriate physical port on the other side of theradio connection.
DIVERSE TRANSPORT
The Carrier Ethernet architecture specified by the MEFis agnostic to the underlying technology used toprovide Carrier Ethernet services. The creation andsupport of such services is, however, an essentialcomponent of the interoperability test event. Mainlythree technologies compete for Carrier EthernetTransport: MPLS, PBB-TE and T-MPLS. During thisevent we had the opportunity to verify all threetechnologies. The following sections describe testresults for each technology in detail.
MPLSMPLS is defined in a set of protocols standardized bythe Internet Engineering Task Force (IETF) and the IP/MPLS Forum. MPLS is positioned to deliver layer 2and layer 3 services including Ethernet services asdefined by the MEF while being agnostic to the under-lying transport technology.The tests in this area were based on previousexperience gained from EANTC’s Carrier EthernetWorld Congress and MPLS World Congress interop-erability test events. The MPLS metro domain operatedindependently from the MPLS core network.The MPLS metro network was built solely for thepurpose of Carrier Ethernet services. Multipoint-to-multipoint services were facilitated with the creation ofa single Virtual Private LAN Services (VPLS) instanceutilizing both VPLS PEs and H-VPLS MTU switchesestablished between the following devices: Alcatel-Lucent 7450 ESS-6, ECI SR9705, Ixia XM2IxNetwork, Juniper MX240 and Juniper MX480, andTellabs 8830. This VPLS instance used LDP forsignaling statically configured peers as described inRFC 4762. These devices also established Ethernetpseudowires using LDP to facilitate point-to-pointEthernet services.In order to test the interoperability of VPLS implemen-tations which use BGP for signaling as described inRFC 4761, another separate VPLS instance wasconfigured. This was demonstrated with BGP-basedAuto-Discovery enabled by the Juniper MX240 andJuniper MX480. The Juniper MX480 performed aninterworking function between this BGP signaled VPLSdomain and an LDP signaled VPLS domain.
Provider Backbone BridgeTraffic Engineering (PBB-TE)One of the potential solutions to delivering MEFdefined services using Ethernet technologies only isthe IEEE defined Provider Backbone Bridge TrafficEngineering (PBB-TE). The technical specification isdefined in 802.1Qay which is working its waythrough the standard process and is in draft version3.0 at the time of the testing. The standard extends thefunctionality of the Provider Backbone Bridges(802.1ah) adding a connection-oriented forwardingmode by creating point-to-point trunks. These trunksdeliver resiliency mechanisms and a configurablelevel of performance.The vendors participating in the PBB-TE transportdomain included Ixia XM2 IxNetwork, Nortel MERS8600, and Tejas TJ2030.In the PBB-TE Metro network we were able to test theestablishment of E-Line, E-LAN, and E-Tree services.The establishment of E-Line services was straight-forward as we tested it in several previous events.E-LAN and E-Tree services creation was tested for thefirst time within the PBB-TE cloud. For the E-LANservice the Tejas TJ2030 switch established abridging instance per PBB-TE trunk and C-VLAN/S-VLAN IDs. The PBB-TE edge devices established atrunk towards this device for each particular UNI. Afew issues related to usage of different Ethertypevalues in CFM messages, padding, and different inter-pretation of CCM intervals were discovered in theinitial configuration phase of PBB-TE trunks, however,these issues were resolved quickly.In addition, the Nortel MERS 8600 and the SpirentTestCenter tested one of the latest additions toEthernet - Provider Link State Bridging (PLSB) – a pre-standard implementation of the IEEE 802.1aq(Shortest Path Bridging) which is in draft version 0.3.The protocol uses the IETF defined IS-IS protocol fordistributing Backbone MAC addresses and ServiceIDs of participating nodes across the network. Oncethe network topology has been learned, IS-IS is usedto establish loop-free multipoint-to-multipoint services.The forwarding plane uses PBB (802.1ah), howeversince the other devices in the PBB-TE network did notsupport PLSB the three Nortel MERS 8600 deviceswere able to use a re-encapsulation of either PBB-TEtrunks or VLAN tags to peer within the PBB-TEnetwork. The Nortel MERS 8600 devices and theSpirent TestCenter emulated nodes successfullylearned the appropriate B-MAC addresses, andforwarded the respective traffic accordingly.Tejas Networks demonstrated a logical Ethernet LANnetwork with an IEEE 802.1ad based Ethernet RingProtection Switching (ERPS). This ring based controlprotocol being standardized under ITU-T G.8032 is aprotection mechanism which offers carriers a deter-ministic sub-50 ms network convergence on a fiberfailure as opposed to the conventional loop-breakingmechanisms like Rapid Spanning Tree Protocol (RSTP).ERPS convergence time is independent to the numberof nodes in the network, thereby vastly enhancing thescalability of a carrier network. We measured failoverand restoration of <35 ms for the demonstrated ERPS.
6
Carrier Ethernet World Congress 2008 Multi-Vendor Interoperability Test
Transport MPLS (T-MPLS)This test marked the third T-MPLS interoperabilitytesting at EANTC. The following devices successfullyparticipated in the T-MPLS area during the event:Alcatel-Lucent TSS-40, Ericsson Marconi OMS 2400,and Ixia XM2 IxNetwork.The T-MPLS standards specify the networking layer forpacket transport networks based on MPLS data planeand designed for providing SONET/SDH-like OAMand resiliency for packet transport networks.Alcatel-Lucent and Ericsson successfully tested thecreation of E-Line, E-LAN and E-Tree services, the lastof which was a first at an EANTC interoperabilityevent. Both participants constructed T-MPLS paths(TMP) which are end-to-end tunnels that aggregateT-MPLS channels (TMC) representing the services. TheTMPs and TMCs were transported over differentphysical layer types including 1 Gbit Ethernet,10 Gbit Ethernet, ITU-T G.709, and SDH STM-16. TheAlcatel-Lucent 7705 SAR was used as a non-T-MPLSswitch in the aggregation area, interfacing to theT-MPLS domain by means of statically configuredMPLS labels.The E-LAN and E-Tree services were configured usinga multipoint architecture similar to VPLS. On oneparticular E-Line service, both Alcatel-Lucent 1850TSS-40 and Ericsson Marconi OMS 2400 were ableto successfully test Quality of Service (QoS) by distin-guishing between three different classes of servicewithin the same Ethernet service and only drop lowpriority traffic when interfaces were oversubscribed.Since the T-MPLS standards do not define a controlplane protocol, the T-MPLS connections betweenvendors were manually configured. Ericsson used twoproprietary management tools (ENEA and DiToNe) tosetup the T-MPLS network configuration on theirdevices.
MPLS CORE
Since MPLS is used by the majority of serviceproviders as core technology it is only logical thatwhen providers add Carrier Ethernet services to theirproduct offering the MPLS core will be used. Wefollowed this approach and used the MPLS core toconnect between three different Ethernet transportmetro areas. The core area was constructed using thefollowing edge devices, all of which successfullyestablished multiple VPLS domains and Virtual PrivateWire Services (VPWS) using LDP for various Ethernetservices: Alcatel-Lucent 7750 SR7, ECI SR9705,Juniper M10i, and Tellabs 8830.In addition to providing transport for Ethernet services,all edge devices in the core established an IP/MPLSL3VPN service using BGP (based on RFC 4364). TheAlcatel-Lucent 7750 SR7 terminated Ethernet pseudo-wires into this VPN providing the potential to offerlayer 3 services to customers which are not reachableotherwise.
EXTERNAL NETWORK TONETWORK INTERFACE (E-NNI)As we described above three different technologieswere used in the metro areas. The problem that everyservice provider then faces is to connect the metroarea with the existing network core. In our testnetwork, much like in most service provider networks,the core used MPLS for transport and services.Therefore, we required mechanisms to allow servicesoriginating on one metro area to cross the core andbe received on other metro areas.The following subsections describe the specificNetwork to Network Interface (E-NNI) solutions usedin the network.
MPLS Metro Connectivity to theCoreSeveral options exist to allow connectivity betweentwo MPLS areas. The preferred options were MPLSbased, but one option used IEEE 802.1ad ProviderBridging tags, or simply 802.1Q VLAN tags totransport services between the two areas. The LabelEdge Router (LER) in the MPLS core would strip theMPLS header from traffic before it forwarded theEthernet frames to the LER in the MPLS metro. TheS-Tag or VLAN tag would then signal to the MPLSmetro device which pseudowire to forward the framesonto. Devices which used this method included theAlcatel-Lucent 7450 ESS-6, ECI SR9705, JuniperM10i and Juniper MX480.The other option used to connect between administra-tively separated MPLS core and metros is referred toas pseudowire (PW) stitching. This involves thecreation of two pseudowires, one in each domain,and then interconnecting them either within onedevice, or with a third pseudowire between the twoedge devices. Vendors who took this approach chosethe latter. In this case two MPLS labels must besignaled: the inner label (PW label) signaled by LDP,and the transport label (PSN label) signaled by eithereBGP (IPv4+label) or LDP. To facilitate the trans-mission of LDP sessions, either a separate OSPF area
T-MPLS to MPLS-TP MigrationFollowing the approval of the first version of theITU recommendations on T-MPLS, the IETF andITU-T jointly agreed to work together to extendMPLS protocols to meet transport network requi-rements in order to ensure a smooth convergenceof MPLS-based packet transport technology. AJoint Working Group (JWT) was formed betweenthe IETF and the ITU to achieve mutual alignmentof requirements and protocols and to analyze thedifferent options for T-MPLS standard progress.On the basis of the JWT activity, it was agreedthat the future standardization work will focus ondefining a transport profile of MPLS (namedMPLS-TP) within IETF and in parallel aligning theexisting T-MPLS Recommendations within ITU-T tothe MPLS-TP work in IETF.At their Dublin meeting in July 2008, the IETF hasinitiated the work on MPLS-TP. Due to the factthat IETF MPLS-TP standard or drafts do not existyet, we tested the implementations based on theT-MPLS ITU-T Recommendations currently in forceand its relevant drafts.It is our intention also to include the first imple-mentations of MPLS-TP drafts at our next event.
7
External Network to Network Interface (E-NNI)
Met
ro/C
ore
Net
wor
kD
evic
e
Gig
abit
Ethe
rnet
Acc
ess
Dev
ice
Met
rone
twor
k
Acc
ess
netw
ork
Agg
rega
tion
Dev
ice
Net
wor
kA
reas
Conn
ectio
nTy
pes
E-N
NI
Cor
ene
twor
k
Wire
less
10G
bitG
709
STM
16
10G
bitE
th
Teste
r
Dev
ice
Types
Agg
rega
tion
netw
ork
Radi
oTr
ansm
issi
onD
evic
e
Alc
atel
-Luce
nt95
000
MPR
Spire
ntTe
stCen
ter
Spire
ntTe
stCen
ter
Teja
sTJ
2030
IXIA
XM2
IxN
etw
ork
Spire
ntTe
stCen
ter
IXIA
XM2
IxN
etw
ork
Gam
ing/
Vid
eoC
lient
s-E
-LAN
Applic
ation
Dem
onst
rations
Nor
tel
MER
S86
00
Teja
sTJ
2030
Alc
atel
-Luce
nt95
00M
PR
Teja
sTJ
2030
Alc
atel
-Luce
nt77
05SA
R
Alc
atel
-Luce
nt77
05SA
R
Eric
sson
Mar
coni
OM
S24
00
Eric
sson
Mar
coni
OM
S24
00
Gam
ing/
Vide
oC
lient
Har
risSt
rate
xEc
lipse
T-M
PLS
Met
ro
Eric
sson
Mar
coni
OM
S24
00
Har
risSt
rate
xEc
lipse
Gam
ing/
Vide
oC
lient
IXIA
XM2
IxN
etw
ork
Teja
sTJ
2030
ECIS
R970
5
Tella
bs88
30
Alc
atel
-Luce
nt77
50SR
7
Juni
per
M10
i
Juni
per
MX4
80
Tella
bs88
30
MPLS
Met
roIX
IAXM
2Ix
Net
wor
kJu
nipe
rM
X480
Alc
atel
-Luce
nt74
50ES
S-6
Gam
ing/
Vide
oC
lient
Har
risSt
rate
xEc
lipse
IXIA
XM2
IxN
etw
ork
Alc
atel
-Luce
nt18
50TS
S-40
MTU
-s
Juni
per
MX2
40
Gam
ing/
Vide
oC
lient
ECIS
R970
5
PBB-T
EM
etro
Spire
ntTe
stCen
ter
Nor
tel
MER
S86
00
Alc
atel
-Luce
nt18
50TS
S-40
Nor
tel
MER
S86
00
Phys
ical
Net
wor
kTo
polo
gy
8
Carrier Ethernet World Congress 2008 Multi-Vendor Interoperability Test
was enabled between the two edge devices or a staticroute was used. Juniper M10i and Juniper MX480took the PW stitching approach.
PBB-TE Connectivity to the CoreAs PBB-TE and 802.1ad are both part of the IEEEProvider Bridging domain of technologies, it is notsurprising that Provider Bridging tags were supportedacross the board in the PBB-TE metro domain. Allservices crossing the core into the PBB-TE cloud usedS-Tags (Service Tags) to distinguish each servicebetween a core edge router and a PBB-TE switch.These devices included ECI SR9705, Nortel MERS8600, Tejas TJ2030, and Tellabs 8830. One PBB-TEtrunk was configured between Nortel MERS 8600and Tejas TJ2030 and traversed the MPLS core.
T-MPLS Connectivity to the CoreTwo options were used to establish services over anMPLS core into a T-MPLS network. The first was to use802.1ad S-Tags, similarly to the MPLS metro.The second option used was pseudowire stitching. TheT-MPLS edge device terminated TMCs coming fromthe edges of the network and stitched them to anMPLS Ethernet pseudowire which was established withthe neighboring core edge device. This was doneusing LDP for both MPLS labels, which ran over aseparate OSPF area than the core. This option wastested between Alcatel-Lucent 1850 TSS-40 and theAlcatel-Lucent 7750 SR7, which also had someservices configured over statically configured MPLSpseudowires.
ETHERNET OAMEANTC interoperability events have integratedEthernet Operations, Administration and Management(OAM) testing since 2006. Several different protocolsfall under the category of Ethernet OAM. In this eventwe have tested Ethernet in the First Mile (EFM), andConnectivity Fault Management (CFM), standardizedby the IEEE under 802.3ah and 802.1ag respec-tively, Y.1731, standardized by ITU-T, and E-LMI,specified by MEF. Over the past years we have seena significant increase in support in this area – in thefirst event four vendors tested their CFM implementa-tions and five vendors tested their EFM code. At ourcurrent event almost all participating vendors testedtheir EFM and CFM implementations.Our service provider panel placed a high value onboth Ethernet OAM test areas and with the inclusionof both EFM and CFM in the new MEF 20 “UserNetwork Interface (UNI) Type 2 ImplementationAgreement” technical specifications a clearcontinuous need for testing has been established
Link OAMLink OAM is the name used by the Metro EthernetForum (MEF) to refer to clause 57 of the IEEE 802.3standards where OAM is defined for Ethernet in theFirst Mile. The protocol monitors the health and opera-tions of the UNI’s physical layer. The MEF requires theusage of Link OAM between the UNI-N and UNI-C
starting from UNI type 2.2 and recommends the useof the protocol starting from UNI type 2.1.The idea behind the protocol is to assist operators inlocalizing last mile physical layer errors. The mostlogical location in the network where the protocolcould be used is between the Provider Edge device (arouter or a switch) and the customer premises device.Both devices can identify themselves to each otherand communicate problems with each other. TheProvider Edge device can use a palette of tools toverify that it can reach the customer premises devicetherefore saving the provider the extreme cost ofsending engineers to the customer site to verifyphysical link issues.The following products successfully participated in theLink OAM tests discovering and setting each other inloopback mode: Alcatel-Lucent 7450 ESS-6, Ixia XM2IxNetwork, Juniper MX240, Spirent TestCenter, andTellabs 8830.We performed an additional test within this areaverifying that a customer premises device is able tonotify the Provider Edge device that the loss of signalis due to the customer premise device being shutdown. The message carried in the Link OAM frame isaptly called Dying Gasp. Juniper’s MX480 andMX240 demonstrated their ability to receive DyingGasp messages and notify the operator using CLIflags that the customer device has been shut down.
Service OAMThe IEEE 802.1ag standard defines end-to-endEthernet based OAM mechanisms which are referredto by the MEF as Service OAM. The support forService OAM is mandatory starting from UNI type2.1. In contrast to link OAM the major use of CFM forservice providers and enterprises is to verify connec-tivity across different management domains. A carriercan define one level for internal use while allowingtheir customers to verify end-to-end connectivity overthe network using a different CFM level.Since the 802.1ag standard has been published inDecember 2007 this has been the first interoperabilityevent where we could specify a finished version of thestandards for testing which simplified the testing.For this interoperability event we added a test at therequest of the participating vendors to the ServiceOAM area: Ethernet Remote Defect Indication(ETH-RDI). This message type, integrated into theContinuity Check Messages (CCMs), communicates toa remote Maintenance End Point (MEP) that a defectcondition has been encountered. When a defectcondition is encountered on a MEP, that MEP will setthe RDI bit in CCMs for all Maintenance Entity Group(MEG) levels affected. When the defect condition isrepaired or removed, the MEP will reset the RDI bit inthe appropriate CCMs. This message type is particu-larly useful to notify a remote MEP about MAC layerproblems, changes to the configurations and sucherror conditions as sudden unidirectional connectivity.The following vendors successfully tested ServiceOAM’s Continuity Check, Linktrace and Loopbackfeatures as well as Remote Defect Indication: Alcatel-Lucent 7450 ESS-6, ECI SR9705, Ixia XM2IxNetwork, Juniper MX240 and Juniper XM480,Nortel MERS 8600, Spirent TestCenter, andTellabs 8830.
9
Resilience and Fault Detection
Performance MonitoringThe ITU-T specification Y.1731 defines two messagetypes for calculating loss and delay between twoEthernet OAM endpoints. Loss MeasurementMessages (LMMs) are sent which should be replied toon arrival with Loss Measurement Replies (LMRs). Theinitiating device can make a calculation of averageloss by comparing the number of frames sent by theinitiating end with those received at the far end. Delayis measured in a similar way, calculating the time ittakes for a Delay Measurement Reply (DMR) to comeback for each Delay Measurement Message (DMM).Similar to CFM, this tool helps both customers to learnabout the service they are receiving, and operators tolearn about the service they are providing. Bothfunctionalities were demonstrated by the Nortel MERS8600.
RESILIENCE AND FAULTDETECTION
One of the key aspects of carrier grade transporttechnologies is SONET/SDH-like resiliency mecha-nisms and fault detection. Based on historicalprecedence, SONET/SDH’s 50 ms restorationcapabilities has been used as the measurement stickfor all transport technologies to follow and these days,with Voice over IP (VoIP) and Mobile Backhaul we seea real need for 50 ms restoration time.The sections to follow depict several failover andrestoration tests that were performed during the event.We tested each transport technology’s resilienceprotocols interoperability and augmented these bynative fault detection mechanisms (CFM for PBB-TEand T-MPLS and Bidirectional Fault Detection (BFD) forMPLS). We focused our Link Aggregation tests on theUNI therefore providing a complete end-to-end resil-ience and fault detection story.
Link AggregationSeveral physical Ethernet ports of Ethernet switchescan be bundled into a single logical Ethernetinterface. This capability is used to increase theamount of bandwidth and/or to provide a linkprotection mechanism. The capability is specified inIEEE 802.3 clause 43, and commonly known as802.3ad or just LAG (Link Aggregation Group). Inorder to create or change the members of a LAGgroup dynamically, Link Aggregation Control Protocol(LACP) has been standardized and is used betweentwo Ethernet devices providing LAG.
The MEF specified LAG as a protection mechanism onthe UNI. Devices supporting MEF UNI type 2.2 mustsupport LAG and LACP which led us to require, asopposed to previous years, that vendors participatingin this test will use LACP and not the static configu-ration of link aggregation groups.As the test was defined for devices that implementUNI functionality we separated the participants intoUNI-C and UNI-N devices. At the UNI-C were IxiaXM2 IxNetwork and Spirent TestCenter, while on theUNI-N Alcatel-Lucent 7450 ESS-6 and ECI SR9705.We first verified that the link aggregation groupswere established between the two devices and thenmeasured the failover time by pulling out of theprimary link, followed by a measurement of the resto-ration time by reconnecting the primary link. In almostall cases we measured failover time below 31 ms,and restoration time below 25 ms. In one case wemeasured failover time of 550 ms and around twoseconds restoration time.Apart from the UNI, Link Aggregation can be used inthe core of the network at the External Network toNetwork Interfaces (E-NNI) or to provide added linkcapacity. In addition to the LAG tests at the UNI, weperformed some LAG tests between directly connecteddevices within the metro and core area networks withthe following: Alcatel-Lucent 7750 SR7, EricssonMarconi OMS 2400, Juniper MX480, and SpirentTestCenter.
MPLS ProtectionSeveral resilience mechanisms exist in MPLS and mosthave been tested in previous EANTC interoperabilityevents. MPLS Fast Reroute (FRR) is one suchmechanism that was tested in 2006 and 2007 (thewhite papers are available on EANTC’s web site).MPLS Fast Reroute provides link and node protectionfor Label Switched Paths (LSPs), therefore, protectingthe services running within the LSPs. The followingdevices participated in the role of the Point of LocalRepair (PLR): Alcatel-Lucent 7450 ESS-6, Alcatel-Lucent 7750 SR7, and ECI SR9705. The Merge Point(MP) is the router which merges the backup andprimary segments of an MPLS tunnel. Alcatel-Lucent7450 ESS-6 and Tellabs 8830 both participated asMPs. Failover times reached as low as 19 ms. Theimportance of the test, however, was the protocolinteroperability between the PLRs and the MPstherefore we spent no time on trying to reconfigure thedevices for faster failover times.In two MPLS Fast Reroute test combinations the tunnelfailure was recognized by Bidirectional FaultDetection protocol (BFD). BFD allows the routers to
AccessAggregation
Access Device
Aggregation Device
LAG Members
Figure 2: LACP Test Setup
PLR Mid-Point MP
MPLS Metro/Core
MPLS switch Working tunnelProtection tunnel
Figure 3: MPLS Fast Reroute Test Setup
10
Carrier Ethernet World Congress 2008 Multi-Vendor Interoperability Test
detect failures of non-directly attached links and signalthe failure to its neighbors quicker than the IGP usedin the network. In both tests the BFD transmissioninterval was configured to 50 ms. The followingdevices were involved in BFD testing: ECI SR9705,Juniper MX480, and Tellabs 8830.
PBB-TE ProtectionResiliency in the PBB-TE domain was accomplished bydefining a protection PBB-TE trunk for a working trunk.In case of the single homed access devices theworking and protection trunks had the sameendpoints, but different mid-point PBB-TE switches. Incase of the dual homing, a single access device wasconnected to two different PBB-TE switches, so thatone endpoint of the working and protection trunkswas different.
In both cases, the detection was done by trunkendpoint switches using CFM (CFM is described in theService OAM section). In almost all cases the Conti-nuity Check Messages (CCM) transmission intervalwas configured to 10 milliseconds (ms), and in onecase to 3.3 ms.All vendors participating in the PBB-TE cloud wereable to show PBB-TE protection interoperability. Inmost cases the failover time was below 51 ms, andrestoration time below 5 ms. In one case the resto-ration time was 718 ms. The high restoration time wasexplained by differences between implementations.Some vendors shutdown traffic on a trunk when theyrevert over to the restored primary trunk and drop anytraffic still running on the backup trunk. Other vendorscontinue to receive traffic on both trunks, but onlytransmit on one trunk. This ensures that the traffic isstill received and reaches the destination.
T-MPLS ProtectionMultiple protection mechanisms exist in T-MPLS. BothT-MPLS path (T-MPLS path is equivalent to an MPLStunnel) and T-MPLS channel (a T-MPLS channel isequivalent to an MPLS pseudowire) can be protected.During this event we demonstrated only the per pathprotection mechanisms.The T-MPLS linear 1:1 and 1+1 path protectionschemes were demonstrated by three EricssonMarconi OMS 2400 devices for 10 Gigabit Ethernet,STM-16 and 10 Gigabit G.709 interfaces. Theprotection schemes required two paths to be build tothe same destination. In the 1:1 protection one path isdeclared as active and the other is set in backupmode. When the active path fails traffic is switched tothe backup path. The second scheme, 1+1 protection,
replicates all frames across both active and backuppaths such that when the primary path fails, thebackup path is already carrying the lost frames. Inalmost all cases Ericsson demonstrated the failoverand restoration times below 35 ms for 1+1 protectionand failover times below 30 ms for 1:1 protection.The restoration time for 1:1 protection was under 0.5ms. In one test run we measured 76 ms failover timeover 10 Gigabit Ethernet link for 1+1 protection.Alcatel-Lucent demonstrated with the TSS-40 devices apre-standard G.8132 T-MPLS ring protection imple-mentation over STM-16 interfaces. In its demonstrationAlcatel-Lucent showed failover times below 30 ms andrestoration time below 16 ms. The two vendorsalready showed T-MPLS protection interoperabilityduring EANTC’s Mobile Backhaul event in January2008 (report available on EANTC’s web site).The T-MPLS demonstration results are summarized inthe figure above.
MANAGEMENT
Alcatel-Lucent demonstrated the 5650 Control PlaneAssurance Manger (CPAM), a vendor agnostic IP/MPLS control plane management solution, whichprovided real-time IGP topology maps, control planeconfiguration, and a look into route advertisementswithin the layer 3 services. The tool was helpful sincemany configurations were being made by manydifferent operators, helping to quickly identify configu-ration issues.
PBB-TE Metro
Working pathProtection path
PBB-TE Switch
Figure 4: PBB-TE Protection Setup
Alcatel-LucentAlcatel-Lucent
T-MPLS Metro
T-MPLS switch
T-MPLSTSS-40 TSS-40G.8132 draft protection
T-MPLS linear1+1 Protection
10 Gbit G709STM-1610 Gbit Ethernet
Ericsson MarconiOMS 2400
Ericsson MarconiOMS 2400
Ericsson MarconiOMS 2400T-MPLS linear
1+1 ProtectionEricsson Marconi
OMS 2400Ericsson Marconi
OMS 2400
Ericsson MarconiOMS 2400
T-MPLS linear1:1 Protection
Ericsson MarconiOMS 2400
Ericsson MarconiOMS 2400
Ericsson MarconiOMS 2400
Working pathProtection path
Figure 5: T-MPLS Protection Results
11
Acronyms
EditorsThis document has been edited by Jambi Ganbar,Sergej Kaelberer, Jonathan Morin, Bionda Mueffke,Carsten Rossenhoevel and Gabriele Schrenk.
ACRONYMS
REFERENCES
Ethernet Services Attributes Phase 2 (MEF 10.1)Requirements and Framework for Ethernet ServiceProtection in Metro Ethernet Networks (MEF 2)Metro Ethernet Network Architecture Framework - Part 1:Generic Frame-work (MEF 4)Ethernet Service Definitions (MEF 6)User Network Interface (UNI) Requirements andFramework (MEF 11)User Network Interface (UNI) Type 1 ImplementationAgreement (MEF 13)Ethernet Local Management Interface (E-LMI) (MEF 16)User Network Interface (UNI) Type 2 ImplementationAgreement (MEF Ballot)Provider Bridges (IEEE 802.1ad-2005)Provider Backbone Bridges (IEEE 802.1ah, draft 4.2)Provider Backbone Bridges Traffic Engineering (IEEE802.1qay, draft 3)Shortest Path Bridging (IEEE 802.1aq)Virtual Bridged Local Area Networks (IEEE 802.1Q)Link Aggregation Control Protocol (IEEE 802.3ad)Carrier Sense Multiple Access with Collision Detection(CSMA/CD) Access Method and Physical Layer Specifica-tions. Amendment: Media Access Control Parameters,Physical Layers, and Management (IEEE Std 802.3ah-2004)Virtual Bridged Local Area Networks - Amendment 5:Connectivity Fault Management (IEEE 802.1ag-2007)Pseudowire Setup and Maintenance Using the Label Distri-bution Protocol (LDP) (RFC 4447)Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling (RFC 4761)Virtual Private LAN Service (VPLS) Using Label DistributionProtocol (LDP) Signaling (RFC 4762)Fast Reroute Extensions to RSVP-TE for LSP Tunnels (RFC4090)Architecture of Transport MPLS (T-MPLS) Layer Network(ITU-T G.8110.1)Interfaces for the Transport MPLS (T-MPLS) Hierarchy (ITU-TG.8112)Characteristics of Multi Protocol Label Switched (MPLS)Equipment Func-tional Blocks (ITU-T G.8121)BFD For MPLS LSPs (draft-ietf-bfd-mpls-06.txt)OAM Functions and Mechanisms for Ethernet BasedNetworks (ITU-T Y.1731)
Term Definition
AIS Alarm Indication Signal
B-MAC Backbone MAC
BFD Bidirectional Fault Detection
BGP Border Gateway Protocol
BRAS Broadband Remote Access Server
CCM Continuity Check Message
C-VLAN Customer Edge Virtual LAN
CFM Connectivity Fault Management
CPE Customer Premise Equipment
DMM Delay Measurement Message
DMR Delay Measurement Reply
DSL Digital Subscriber Line
E-NNI External Network-to-Network Interface
EFM Ethernet in the First Mile
ERPS Ethernet Ring Protection Switching
EVC Ethernet Virtual Connection
EVPL Ethernet Virtual Private Line
FRR Fast ReRoute
IGP Internal Gateway Protocol
IS-IS Intermediate System to Intermediate System
LACP Link Aggregation Control Protocol
LAG Link Aggregation Group
LDP Label Distribution Protocol
LER Label Edge Router
LMM Loss Measurement Message
LOS Loss Of Signal
LSP Label Switched Path
MAC Media Access Control
MEG Maintenance Entity Group
MEP Maintenance End Point
MPLS Multi-Protocol Label Switching
MPLS-TP MPLS Transport Profile
MTU-s Multi Tenant Unit Switch
OAM Operations, Administration and Mainte-nance
OSPF Open Shortest Path First
PBB Provider Backbone Bridge
PBB-TE Provider Backbone Bridge TrafficEngineering
PE Provider Edge
PLR Point of Local Repair
PLSB Provider Link State Bridging
PSN Packet Switched Network
PW PseudoWire
QAM Quadrature Amplitude Modulation
QoS Quality of Service
RDI Remote Defect Indication
RFC Request For Comments
RSTP Rapid Spanning Tree Protocol
RSVP-TE Resource reSerVation Protocol TrafficEngineering
S-Tag Service Tag
SLA Service Level Agreement
T-MPLS Transport MPLS
TMC T-MPLS Channel
TMP T-MPLS Path
UNI User-Network Interface
VLAN Virtual Local Area Network
VoIP Voice over IP
VPLS Virtual Private LAN Service
VPN Virtual Private Network
VPWS Virtual Private Wire Service
Term Definition
EANTC AGEuropean Advanced Networking Test Center
IIR Telecoms
Einsteinufer 1710587 Berlin, GermanyTel: +49 30 3180595-0Fax: +49 30 [email protected]
29 Bressenden PlaceLondon SW1E 5DR, UKTel: +44 20 7017 7483Fax: +44 20 7017 [email protected]
This report is copyright © 2008 EANTC AG. While every reasonable effort has been made to ensure accuracy andcompleteness of this publication, the authors assume no responsibility for the use of any information contained herein.All brand names and logos mentioned here are registered trademarks of their respective companies in the UnitedStates and other countries.v1.0
