CO/DCNetworkTransformation

DanielVoyerTechnicalFellowMarch2017

WhatisBellCanada?

• OldestinWorld(1880)– Wereallydidinventthephone

• LargestinCanada• Public• Multipleventures

– Wireline,Wireless,Media,Enterprise,etc.– Satellites,Sportsteams,

• SPTransformation– Culture– Processes– Technology

• Newmodeofoperations(cloud)• Newcompetition(OTT)• Newservices(NFV)• Agility

OurOrigins Network3.0

Network 3.0 is a journey to…Transform how Bell delivers the best customer experience with seamless access to a software-driven, cloud-based ecosystem

https://en.wikipedia.org/wiki/Bell_Canada

Challenges - Internet traffic is growing

• Internet grow exponentially

• Physical Networks are static and requires long cycle migration changes

Growing faster than we can adapt – and pay for …

• Hit TCAM limits

• August 2014 widespread outages

• Cost more money $$$

Source https://bgp.potaroo.net

Challenges– Bell’sOwnComplexity

• ManyindependentMPLSdomainstoday

– Longprovisioningcycles• Cantakeupto3-4weekswithtools(orlonger)toengineer

– NoE2ETrafficEngineering• Complexwithstateinthenetwork• Staticandhardcoded,it’salwayson

– NoE2EOAM• Notalwaysawarethattunnelsarefailing• Poorvisibilityofthestateofthetunnels• hopbyhoptroubleshooting

Without simple and efficient traffic engineering, how do you manage this

A Need for a New Architecture

NextGenerationRequirements

• Needstobeanindustrystandardratifiedbyglobalstandardsorganizations• Reusableinthecore/WAN,possiblyasthegluetobringallthenetworkstogether

• Software-programmable• LeveragenewCO/DCgreenfieldopportunitytotrysomethingnew• Providessolutionsforbothtransitionandendstate• Interoperabilitywiththebrownfieldandgreenfield• ImplicitECMPhandling

Before- TraditionalView

Access Metro IPCore

BigInternet

Metro Access

After– NetworkTransformation

Access Metro IPCoretransport

BigInternet

CO/DCTier1

CO/DCTier2 ...CO/DC

Tier1 MetroCO/DCTier2 Access

Architecture Central Offices Re-Design for network operators virtualization use cases

Content Hardcopy & VNF/CNFsCached Copy & VNF/CNF

Guiding principle required to accelerate execution and ensure evolution towards Agility

Bag of existing Protocols

NETCONF/YANGSSH

Next Gen. ProtocolsSRv6 SR (MPLS)PCEPISISBGP (TE, LS)IP OAMEthernet OAMEVPN

Reducing operations complexity§ Simpler automation§ Simpler to repair§ Simpler integration§ Foundation for service Orchestration

SimplifyStandardizeAutomateAbstract

ArchitectureChange- DrasticNetworkProtocolsReduction@Bell

Ethernet802.1Q, 802.1adIPv4PPPoEIPv6MPLSL2TPPWE3ISISOSPFRSVP-TELACPMC-LACP

MP-BGPLDPLDP-TEIP OAMMPLS OAMEthernet OAMSTPG.8032RADIUSSNMPSyslogNetflowSSH CLI/XML

Key enabler for

CO/DC– FabricPEtoCorePE– highlevel

DC Fabric Core

PELeaf

LeafvPE

Site ABCServer A

VM A

PE

The goal for network transformation is to move the complexity from core transport to the CO/DC and virtualize network components

The DC Fabric and Core Network seen as a common IP Network

Leverage existing Data Plane – MPLS E2ESimplify Control Plane - SR fabric in DC - good starting point

CO/DC– ArchitectureOverview

B-LEAF

A-LEAF

SPINE

A-LEAF LEAFLEAFLEAF LEAF

SPINE

B-LEAF

Metro Core

CoreIGPAdj.

Big Internet

TS

TSES

ES

P

P

IS-ISLDPinterop

CEPH vRPvCE vPE CEPH

PE

vRR

AccessOLT TOR TOR TORTOR

Key SR Points• Fabric underlay is ISIS and SR = SIMPLE• ECMP & SR for traffic engineering = FLEXIBLE• SRTE with IP Core network = AGILE• EVPN Overlay – L2/L3 services = AGILE & SIMPLE SR

MapServer

CO/DC Challenges Solved by SR• Classic DCI overlay is wrong for CO/DC, we need

better integration to leverage network assets• SR Solutions:

• Map server for interop w/ brownfield LDP PE• Dual Stacking of LDP & SR

Segment Routing Mapping Server is important for brownfield interaction

CO/DC– SR&LDPintermediatestate

B-LEAF

A-LEAF

SPINE

A-LEAF LEAFLEAFLEAF LEAF

SPINE

B-LEAF

Metro Core

CoreIGPAdj.

Big Internet

TS

TSES

ES

P

P

IS-ISLDPinterop

CEPH vRPvCE vPE CEPH

PE

vRR

AccessOLT TOR TOR TORTOR

SRMapServer

Classic MPLS Domain (LDP)

SR MPLS Domain

SR – LDP interop using SRMS

SRAbsoluteLabelAlgorithm• SegmentRoutingMappingServers&SRAllocation

– SRMSiscriticaltoanySRdeploymentwithbrownfieldinterop– ReuseoftheSRMSalgorithmtoassignLabelintheSRDomain– SRLabelarethenassignedwiththeIPloopbackprocesses

– PlantheSRdomainsperlabelrange• UseoffullSRGBblock:65k• IPCore8kblock• CO/DC– Tier1:4kblock• CO/DC– Tier2:2kBlock

To ensure SR uniqueness across all domains, we came up with the following SR Absolute Label Algorithm

SRGB_Base* + (first-SID-Index [Infra_underlay|IPVPN|Internet] + loopback-last-octet) = SR absolute Label

*The SRGB_Base start at 16k

Learnings– SRAllocationexamplewithabsolutelabel

ForagivenCO/DC_Awiththefollowingloopbackaddress,209.71.196.15/32

NextSteps

• SegmentRoutingfromthehost(HV,vPE,kernel,etc.)– Expandcontroller:SR-TEinDC,On-DemandNext-Hop

• SR_between_DC’s(coretransformationtoSR)

Host

Host

Core– ISIS/SRTEPCEP

Leaf/Spine/BorderLeafISIS+SRTE

BorderLeaf/Spine/LeafISIS+SRTE

HostEnd-to-EndSR

SRTE

SRTE

CO/DC CO/DCCoreTransport

Application-responsive networking

Use-Case:Scaled-OutvPE

Buildahighly-distributedvPE functionthatscaleslinearlywiththeCO/DCfabric.

• LimitedEast-WestTraffic• Hypervisoristhenewedge• Sameprotocolstack• Avoidthe“BigFatVNF”

• Follownetworkdisaggregation principlesandbuildusingopen,modular,replaceablecomponents.

• Samedesignprinciplescanbeappliedtootherhigh-throughputVNFs

Learnings

• Simplicitywinseverywhere– Capturelatentnetworkvalue– leverageexistingphysicalassetswithefficienton-demandTE– ReductionCAPEX/OPEXandincreaseAgility– Alotoflegacyprotocolscanberemoved(LDP,RSVP-TE,etc.)– MakeEngineering/Opshappy(deterministiclabels,labelreuse)

• Startsmall,findagreenfieldislandtointroducenewtechnologies

• Thehardpartisthebrownfieldtransition,becareful– SRvsNon-SRNodesInterop/Integration– SRhaslotsofoptions!

• SRGBplanningisimportant– Inourcase,wechosetoallocate64KlabelstoSRinsteadofdefault8K(LOTSofVM’s!)

• Workwithindustrystandards– Keepthevendorshonest

Do a lot more, with a lot less

UseCases– SRv6

SRv6- NFV- ServiceChaining

• SegmentRouting servicechaining:servicesareexpressedwithsegments– Flexible– Scalable– Stateless

Packets from are steered through a sequence of services on their way to the server.

Firewall IDS/IPS

DPI box

ClientServer

S1

S2

S3

D

SR policy:〈 S1, S2, S3, D 〉

https://tools.ietf.org/html/draft-filsfils-spring-srv6-network-programming-00

IPv6 ( A1::0, A3::A32 )

payloadIPv6 ( T1::0, V2::0 )

SRH { A3::A32, A4::0, A5::A76, A2::C4 }

SRv6- IntegratedNFV

• A3::A32means– AppinContainer32– @nodeA3::/64

• Stateless– NSHcreatesper-chainstateinthefabric

– SRdoesnot• AppisSRawareornot

1

2

4

V/64

3

T/64

4

3 App 32Container

Server 3

5 App 76VM

Server 5

IPv6 ( T1::0, V2::0 )payload

InnerheadercouldalsobeIPv4insteadofIPv6

SRv6- IntegratedNFV

• IntegratedwithunderlaySLA1

2

4

V/64

3

T/64

4

5 App 76VM

Server 5

3 App 32Container

Server 3IPv6 ( A1::0, A4::0 )

payloadIPv6 ( T1::0, V2::0 )

SRH { A3::A32, A4::0, A5::A76, A2::C4 }

SRv6- IntegratedNFV

1

2

4

V/64

3

T/64

4

5 App 76VM

Server 5

3 App 32Container

Server 3

• A5::A76means– AppinVM76– @nodeA5::/64

• Stateless– NSHcreatesper-chainstateinthefabric

– SRdoesnot

• AppisSRawareornot

IPv6 ( A1::0, A5::A76 )

payloadIPv6 ( T1::0, V2::0 )

SRH { A3::A32, A4::0, A5::A76, A2::C4 }

SRv6- IntegratedNFV

1

2

4

V/64

3

T/64

4

5 App 76VM

Server 5

3 App 32Container

Server 3• IntegratedwithOverlay

IPv6 ( A1::0, A2::C4 )

payloadIPv6 ( T1::0, V2::0 )

SRH { A3::A32, A4::0, A5::A76, A2::C4 }

IPv6 ( T1::0, V2::0 )payload

VNF hostF1::

IPv4Hdr SA=A.A.A.A,DA=B.B.B.BPayload

NFV– SRv6Function- END.AS– Staticproxy

END.ASfunctionboundtoSIDF1::A1- RemovesIPandSRheadersfromthepacket- SendsIPv4packetoutonIface 1

InboundpolicyonIface 2:insertSRH- SteersincomingIPv4packetsintoanSRv6Policy〈 …,F2::,…〉- SendspacketouttowardsthenewDA:F2::

IPv4Hdr SA=A.A.A.A,DA=B.B.B.BPayload

Endpoint to SR-unaware APP via static proxy

Per-chain static configuration

VPP

VNF1

Iface 1 Iface 2

SRHdrIPv6Hdr SA=E1::,DA=F1::A1

(…,F1::A1,…)SL=k

PayloadIPv4Hdr SA=A.A.A.A,DA=B.B.B.B

SRHdrIPv6Hdr SA=F1::,DA=F2::

(…,F2::,…)SL=k’

PayloadIPv4Hdr SA=A.A.A.A,DA=B.B.B.B

VNFprocessesaregularIPpacket

END.AScanalsobeusedwithencapsulatedIPv6traffic

VNF hostF1::

IPv4Hdr SA=A.A.A.A,DA=B.B.B.BPayload

NFVSRv6Function- END.AD– Dynamicproxy

END.ADfunctionboundtoSIDF1::A2- DecrementsSLandupdatesouterDA- CachesouterIPv6headerandextensions- RemovesouterIPv6headerandextensions- SendsIPv4packetoutonIface 1

InboundpolicyonreturningIface 2:restoreouterIP- RestorescachedIPv6headerandextensions- SendspacketouttowardstherestoredouterDA:F2::

IPv4Hdr SA=A.A.A.A,DA=B.B.B.BPayload

Endpoint to SR-unaware APP via dynamic proxy

Simple per-chain configurationOne (SID, returning iface) per chain

VPP

VNF1

Iface 1 Iface 2

SRHdrIPv6Hdr SA=E1::,DA=F1::A2

(…,F2::,F1::A2,…)SL=k

PayloadIPv4Hdr SA=A.A.A.A,DA=B.B.B.B

SRHdrIPv6Hdr SA=E1::,DA=F2::

(…,F2::,F1::A2,…)SL=k-1

PayloadIPv4Hdr SA=A.A.A.A,DA=B.B.B.B

VNFprocessesaregularIPv4packet

END.ADcanalsobeusedwithencapsulatedIPv6traffic

SRv6- Sprayuse-case

VPP1B1::

ASR2B2::

ASR3B3::

CMTS4B4::

CMTS5B5::

ContentAppA::

TV2andTV4arenotsubscribedtothischannel(M1)anddonot

recievethecontent

TV-1C1::

TV-3C3::

TV-5C5::

>VPP1: show sr spray policiesIn_iface SR Spray PolicyGE0/5/0 {B2::, B4::, M1}

{B3::, B5::, M1}Total SR spray policies: 1 T

T T

T

SRv6 domain (Unicast IPv6) Multicast domainMulticast domain

Flexible,SLA-enabled andEfficientcontentinjectionwithout multicastcore

IPv6 Hdr SA = A::, DA = B5::

PayloadSR Hdr ( M1, B5::, B3:: ) SL=1

IPv6 Hdr SA = A::, DA = B1::Payload

IPv6 Hdr SA = A::, DA = M1Payload

IPv6 Hdr SA = A::, DA = B3::

PayloadSR Hdr ( M1::, B5::, B3:: ) SL=2

IPv6 Hdr SA = A::, DA = B4::

PayloadSR Hdr ( M1, B4::, B2:: ) SL=1

IPv6 Hdr SA = A::, DA = B2::

PayloadSR Hdr ( M1::, B4::, B2:: ) SL=2

SeetheDemo

ThankYou