Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
ImplementationJuniperNetworksvMX atA2BInternetBy ErikBais– A2BInternet
|MORE-IP2017presentation 2016Page|2
Whatisourbusiness??
oRegistrationofIPaddressesandASnumbers
o IPTransitinvariousDutchdatacenters
o Internet(Fiber)Access&DatacenterNetworkServices
o24*7MonitoringandmanagementofBGPinfrastructure.
o SpecializedconsultancyforISPrelatedtopicslikevendorselections,networkdesign&implementation.
|MORE-IP2017presentation 2016Page|3
Currently inthe following Dutchdatacenters
|MORE-IP2017presentation 2016Page|4
Shortintro
oA2BInternetisaDutchnetworkprovider.§ Providingdatacenterconnectivityandinternetaccessonfiber.
oWeimplementedtheJuniperNetworksvMX solutionrecently§ http://newsroom.juniper.net/press-releases/a2b-internet-deploys-juniper-networks-vmx-as-the-first-virtual-network-function--nyse-jnpr-11g134000-001
oButthe realquestioniswhy govirtual??…
|MORE-IP2017presentation 2016Page|5
Casestudy online
|MORE-IP2017presentation 2016Page|6
Previoussetup
oA2BInternethasalways beenan ExtremeNetworksshop…
oWenoticed that the BGPconverge took too longwith the current DFZsize
o Some updates(especially onthe AMS-IXpeering switch)took waytoo long..§ Slowupdatesofannouncing some prefixes..§ Orevensloweraccepting certain routes..§ Droppingpeers under highBGPload....(AARGGHHH!!)§ And properfiltering,madethings evenworse …
o Limitation wasinasinglethreaded BGPprocess onadual-core CPU
|MORE-IP2017presentation 2016Page|7
Let’shavealookatthevMX
|MORE-IP2017presentation 2016Page|8
|MORE-IP2017presentation 2016Page|9
vMX ProductOverview
VCPVFP
Physical NICs Management traffic
Guest VM (Linux) Guest VM (FreeBSD)
Hypervisor: KVM, ESXi
Cores Memory
Bridge / vSwitch
Physical layerPCI P
ass
thro
ugh
SR-IO
V
Virt
IO
Virtual Control Plane (VCP)• JUNOS hosted in a VM. Offers all the capabilities
available in JUNOS• Management remains the same as physical MX• SMP capable
Virtual Forwarding Plane (VFP)• Virtualized Trio software forwarding plane. Feature
parity with physical MX. Utilizes Intel DPDK libraries• Multi-threaded SMP implementation allows for
elasticity• SR-IOV capable for high throughput • Can be hosted in VM or bare-metal
Orchestration• vMX instance can be orchestrated through OpenStack
Kilo HEAT templates• Package comes with scripts to launch vMX instance
|MORE-IP2017presentation 2016Page|10
Architectural Difference with Shipping NFX250-S2
RIOT VMXT
SwitchingHardware(CrossconnectNIC) NIC RAM SSD
12x1GE 2x10GE 1GE2x10GE(internal)
...
External
X86CPU
LinuxHostOSVFP(PFE) KVMHypervisor
L2_TVPBSDJunos(JCP)
VirtualMachine
JunosDeviceManager (JDM)Container
LinuxBridge
SwitchingHardware(PFE) NIC RAM SSD
12x1GE 2x10GE 1GE
...
External
LinuxHostOS
X86CPU
VirtualControlPlane(VCP)
VirtualMachine
vMX on NFX Native NFX
SameHW
DifferentSW
ApproachvMX
DCPFE LCMD
BCMD
LCMD
KVMHypervisor
LinuxBridge
VNF#2
VNF#3
VNF#N
…vSRX2.0
VNF#1
2x10GE(internal)
|MORE-IP2017presentation 2016Page|11
OurUsedHWkitlist
oWeselectedHPasourvendorforthevMX setup.
oTheusedkitperbox:§ 1*HPProliant DL360gen92xE5-2650v4,64GB,2xPSU§ 2*HP560SFP+10GbePCIe Intelbased82599§ 2*HP240GBSSDHotplug 2.5inchSFF
o InShort..Enoughcore’s,enoughmemory,stickwiththe‘tested/recommendedNIC’s‘andsomeSSD’sforquickerbootingifneeded..
|MORE-IP2017presentation 2016Page|12
VMXlicenses…
oThevMX licensesthatweusearetheAdvanceversion..10G
oThereare1Gband5Gbversionsaswell..
oWedidn’tneedL3VPNor4Mil.routes..(yet)
|MORE-IP2017presentation 2016Page|13
Implementation
oYouneedtofollowtheJuniperimplementationguide..
oYes..RTFM!!..§ http://forums.juniper.net/t5/Day-One-Books/Day-One-vMX-Up-and-Running/ba-p/289129
oTheimplementationisquitepickyinkernelandlibraryversions.
o StartwiththerecommendedUbuntuversion..(notthelatest)
o Skipany idea ofrunningthis onVmWare ..Use Ubuntu+KVM… <period>
|MORE-IP2017presentation 2016Page|14
/home/vmx/vmxlite/config/vmx.conf
o SelecttherightimagestouseinKVMforVMX..
§ #Configurationonthehostside- managementinterface,VMimagesetc.§ HOST:§ identifier :vmx1 #Maximum6characters§ host-management-interface:em1§ routing-engine-image :"/home/vmx/vmxlite/images/junos-vmx-x86-64-16.1R3.10.qcow2"§ routing-engine-hdd :"/home/vmx/vmxlite/images/vmxhdd.img"§ forwarding-engine-image :"/home/vmx/vmxlite/images/vFPC-20161019.img"
|MORE-IP2017presentation 2016Page|15
/home/vmx/vmxlite/config/vmx.confo #vREVMparameterso CONTROL_PLANE:o vcpus :2o memory-mb :4096 #<=- 4Gbisbetterthan2Gb.2Gbworks.o console_port:2211o
o interfaces :o - type :statico ipaddr :<privateIP>o macaddr :"0A:00:DD:B0:DE:0E"
|MORE-IP2017presentation 2016Page|16
/home/vmx/vmxlite/config/vmx.conf
o #vPFEVMparameterso FORWARDING_PLANE:o memory-mb :24576o vcpus :22o console_port:2212o device-type:sriov #<=- YouwantandNEEDSR-IOV...o
o interfaces :o - type :statico ipaddr :<privateIP>o macaddr :"0A:00:DD:B0:DE:10” #<=- StaticMAC’s...Beware!!
|MORE-IP2017presentation 2016Page|17
WhatisSR-IOV?AndwhydoIwantthis?
o Single-rootinput/outputvirtualization
oSR-IOV isa networkinterface thatallowstheisolationofthe PCIExpress resourcesformanageabilityandperformancereasons.AsinglephysicalPCIExpresscanbesharedona virtualenvironment usingtheSR-IOVspecification.
|MORE-IP2017presentation 2016Page|18
SR-IOVallowsfordedicatedaccesstotheNICbytheVM
|MORE-IP2017presentation 2016Page|19
SR-IOVtakeaway…
|MORE-IP2017presentation 2016Page|20
WhatisSR-IOV?AndwhydoIwantthis?
o Single-rootinput/outputvirtualization
oSR-IOV isa networkinterface thatallowstheisolationofthe PCIExpress resourcesformanageabilityandperformancereasons.AsinglephysicalPCIExpresscanbesharedona virtualenvironment usingtheSR-IOVspecification.
ohttps://www.youtube.com/watch?v=hRHsk8Nycdg - IntelSR-IOVExplanation
|MORE-IP2017presentation 2016Page|21
Performanceo Weusea10Gfortransitand2*10Gtoourinternalnetworkpertransitbox.
o PCIExpresscando:
§ Source:Intel- http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005811.html
PCIExpressImplementation EncodedDataRate UnencodedDataRate
x1 5Gb/sec 4 Gb/sec(0.5GB/sec) x4 20 Gb/sec 16 Gb/sec(2GB/sec) x8 40 Gb/sec 32 Gb/sec(4GB/sec) x16 80 Gb/sec 64 Gb/sec (8GB/sec)
Theoretical Maximum Bus Throughput:•PCI Express* (PCIe*) Theoretical Bi-Directional Bus Throughput.
|MORE-IP2017presentation 2016Page|22
TheIntelNIC
o Specifications:
§ HostInterface§ n PCIe BaseSpecification2.0(2.5GT/s)or(5GT/s)§ n Buswidth— x1,x2,x4,x8
oOurnetworkcardshave2*10GbSFP+..Andpercard40Gbps theoreticalthroughput..
o2NIC’sperbox..Whichleavesenoughroomforline-rateperformance…
|MORE-IP2017presentation 2016Page|23
Intothenetwork…oWestartedbymigratingIPv6ontothevMX’s beforeIPv4.
o OnceIPv6wasrunningwithoutanyissues....Wescheduledav4migrationintovMX.
o Transitsmigrationwasreallysimple..§ ThevMX’s feellikeyouareworkingonanactualJuniperMX.
oMigratingallthepeersonthepeeringbox,wasabitmorework.MostlyduetolegacypeerswithMD5orspecificroute-mapsforcertainpeers.
o Afullv4BGPtableloadisdonewithin4seconds!!§ NoneedtoprogramASICS/TCAM..Allroutesareusablewhenloaded..
|MORE-IP2017presentation 2016Page|24
Currentmaxbandwidthsincethemigration:
oAMS-IX10Glink:8.2GboTransit10Gblink:6.3Gb
oCPUusageAMS-IXrouter:
|MORE-IP2017presentation 2016Page|25
Futurepath?
o Itisalwayspossibletopastetheexactsameconfig intoaJuniperMX240orbigger...
oYoucanstarthere… and your development(automation)can be ported into anyother Junos environment.
oNextversions ofvMX will supportQSFP’s and 100GbNIC’s …
oAnd if you don’t likeit oroutgrow the setup,the hw can be re-used for other tasks …
|MORE-IP2017presentation 2016Page|26
Questions?