Year 2 Annual Report (Sept 1, 2016 — August 31, 2017) NSF Award ACI-1451050 IRNC: RXP – Pacific Wave Expansion Supporting SDX & Experimentation Submitted: July 19, 2017 Louis Fox (lfox@cenic.org) YearTwoGoals&Accomplishments ProjectgoalsThisawardfundsthedevelopmentofadistributedmulti-domainSDX(SoftwareDefinedeXchange)tosupportcollaborationamonginternationalresearchandeducationnetworksexperimentingwithSoftwareDefinedNetworksthroughoutthePacificRimregion.ThePacificWaveexchangewillsupportandenhancecollaborativeresearchintoastronomy,digitalmediaandcinematicarts,biologyandmedicine,computationsciences,earthsciences,gridcomputing,high-energyphysics,andoceanography. TheoverallobjectivesofthePacificWaveExpansionSupportingSDXandExperimentationaretoenhance,upgrade,andevolvetheproductionPacificWavedistributedopenexchangefabrictosupportmultiple100gigabit-per-secondconnectionsfromcountriesthroughtheAsia-Pacificareaandtodevelopasetofparallelfacilitiessupportingexperimentationininterconnectingmulti-domainSoftwareDefinedNetworksthroughadistributedexchange.PacificWavewillachievetheseobjectivesthroughengagementwiththe17majorinternationalresearchandeducationnetworkscurrentlyconnectedtothePacificWaveexchangeandthroughinformationandtechnologyexchangedwithotherIRNCawardeesandtheUSR&Ecommunity. Activities&Accomplishments:SoftwareDefinedNetworking/SoftwareDefinedExchange

• AcquiredanddeployedanSDXnodeinSunnyvale[CompletedMay2017]o DeployedSDXswitchatSunnyvaleo ConnectedSunnyvaleSDXtoLosAngelesSDXandSunnyvalePacWaveat100Gbps

• ContinueengagementwithSDNapplicationsofPacificWaveparticipants• Aimtoengageandsupporttestbeds,experimentation,andSDXcontrolframeworks• ONOSSDNsoftware[CompletedMarch2017]

o InstalledONOSsoftwareonLosAngelesHypervisorserver

2

o LogicallyconnectedONOScontrollertobothSunnyvaleandLosAngelesSDXswitches• CastorSDXsoftware[CompletedMay2017]

o CollaboratedwithCastordevelopersfromAustraliaandinstalledCastorsoftwareontoLosAngelesHypervisorserver

o IntegratedCastorintoONOSplatformandperformedinitialdata-planetestingusingLosAngelesSDXandLosAngelesPacWaveswitches

o DiscussedcollaborationwithStarlightandAARNeto DevelopedintegrationplansforCastorconnectivitytootherentitieso MadeplanstoidentifyandexplainintegrationplansinYear3Goals

Free-RangeRouting(FRR)software[CompletedJune2017]

o InstalledFRRoutingsoftwareonLosAngelesHypervisorserverthatwillfunctionastheBGProute-serverconnectedtotheCastorsoftwareandalsopeerwithotherIRNCentities

SpecificObjectives: 100GbpsconnectedperfSONARnodesOnegoalofPacificWave’sperfSONARdeploymenthasbeentodevelopperfSONARnodescapableofspeedsapproaching100Gbps.PacificWavehascompletedthedeploymentoftwo100GbpsconnectedperfSONARnodesinSeattleandLosAngeles.AspartofupgradingtheperfSONARtoolkitfromversion3.5to4.0,were-imagedthe100GbpsconnectednodeswithCentOS7totakeadvantageofnewfeatures,TCPenhancements,andtuningcapabilitiespresentinthe3.xLinuxkernel. Inadhoctestingbetweenthetwo100Gbpsconnectednodesweobtainedapeakthroughputof~45Gbps;inmesh-scheduledtesting,weobservedanaveragethroughputof~40Gbps.Inbothscenarios,weemployedmoderatesystemandNICtuningandusediperf3,TCP,single-streamoverthe~25mslatencybetweenthetwoendpoints. IncollaborationwithAARNet(TimRayner,WarrickMitchell,etal.)wesetupanenvironmenttoenablemoreaggressive,multi-streamtestingbetweenthePacificWave100GbpsconnectednodesinSeattleandLosAngelesandanAARNet100GbpsconnectednodeinCanberra,Australia.Inthisscenario,eachnodehadthreeseparateinstancesofiperf3runninginservermodeondistinctTCPports,witheachiperf3processpinnedtoaseparateCPUcore.Thetheorywasthatmultipleiperf3instancesshouldbeabletoachieveahigheraggregatethroughputthanasingleiperf3process—particularlyon40Gbpsor100Gbpsconnecteddevices.Resultsincludedapeakaggregatethroughputof53GbpsfromCanberratoSeattleandof49GbpsfromSeattletoCanberra. Weareintheprocessofacquiringa100GbpsconnectedperfSONARnodeforSunnyvaleaspartofourYear2deliverables,andexpecttodeployitinthesummerof2017,althoughitispossibletheworkwillextendintoYear3.

3

IncreasedinstrumentationofPacificWaveinfrastructureAttheinvitationofESnetandInternet2,oneofPacificWave’s10GbpsconnectedperfSONARnodesinLosAngelesparticipatedinabetaprogramofpScheduler—thereplacementschedulerunderperfSONAR4.0fortheseparatebwctldandowampdschedulerservices.ThisnodehassincebeenupgradedtotheperfSONAR4.0testpointbundlerunningonCentOS7. PacificWavealsoprovisioneda10GbpsconnectedperfSONARnodeforPacificWaveSunnyvaletoreplaceaserverborrowedfromCENIC’sCalRENnetwork.ThemotivationfortheequipmentloanwastobridgeagapinvisibilityandsupportduringSC16activities.TheIRNC-fundedSunnyvale10GbpsconnectedperfSONARnodehassincebeenintegratedintothePacificWaveperfSONARMaDDAsh. StaffalsoworkedwithInternet2tosetupregulartestingbetweenthePacificWave10GbpsconnectedperfSONARnodesinSeattleandLosAngelesandaselectionoffourInternet2PerformanceAssuranceService(PAS)perfSONARnodesacrosstheUStopology. Wecontinuedtoexplorepotentialframeworksbywhichtoenablethedynamicscheduling/provisioningof10GbpsconnectedperfSONARnodesforauser-specifiedcircuit(VLAN).AspartofourparticipationintheGLIFAutoGOLEactivities(seebelow),10GbpsconnectedperfSONARnodesinSeattle,Sunnyvale,andLosAngeleshavebeenmadeavailableviaNSI-basedorchestration/reservation.

perfSONARMaDDashDuringSeptember,2016,wemigratedthePacificWaveperfSONARMaDDashfromv1.3toabetaofv2.0.WeworkedwiththedeveloperstoidentifyandresolveapparentinteroperabilityandconfigurationchallengesinasubsetofMaDDashandperfSONARtoolkitinteractions,includingthetime-seriesgraphingfunctionality. RegulartestingamongPacificWaveperfSONARnodesacrosspublic,inter-exchangeVLANswasestablishedtoprovideadditionalvisibilityofperformance.PacificWavehasalsocontinuedtoexplorethepotentialforestablishingregulartestingamongPacificWaveIRNC-fundedperfSONARnodesandthoseofinterestedPacificWaveparticipantsandpartnernetworks. VirtualCustomerEquipment(VCE)SoftwareBegindevelopmentofVirtualCustomerEquipment(VCE)withGlobalNOC[Inprogress]

• VCEsoftwareconnectedtoswitchinCENICHQLabforinitialtesting• Basicfunctionality(showcommands)confirmedworking• Moreadvancedfunctionality(interface/serviceconfiguration)isbeingtestedanddebugged;

developmentisongoingCurrentlyworkingonmovingVCEsoftwarefromIU-controlledVM’sandontoCENIC-controlledVM’s

4

SignificantResults:AutoGOLE/NSIWecontinuedcollaborationwiththeGLIF-AutoGOLEworkinggroup,exploringserviceanddeliverymodelsforprovidingdynamic,interdomaincircuitservicesbasedonNSI-signaling.Aspartofalong-termtrialofprovidingdynamiccircuitservicesonPacificWave,weestablishedseparateNSIdomainsforthePacificWaveSeattle,Sunnyvale,andLosAngelesexchanges,withtheseelements:

• NORDUnet-developedOpenNSA,providingtheabstractionlayerbetweenthePacificWavetraditionalLayer2switchesandNSIorchestrationelements.

• Control-planepeeringwiththeNSI-aggregatorsmanagedbyESnet,Netherlight,andStarLight.• Data-planepeeringwithESnet,StarLight,andSINET,withinterestindevelopingfurtherpeering

withotherPacificWaveparticipants.

• A10GbpsconnectedperfSONARnodeavailableasaNSI-assignableresourceineachdomain:o urn:ogf:network:sttlwa.pacificwave.net:2016:topology:irnc-10g02.sttlwao urn:ogf:network:snvaca.pacificwave.net:2016:topology:irnc-10g01.snvacao urn:ogf:network:lsanca.pacificwave.net:2016:topology:irnc-10g02.lsanca

• perfSONARnodes,participatingintheAutoGOLE‘demo’rangeofVLANsIDs,1779-1799.

• ParticipationinapilotleadingtowardbroaderadoptionoftheRNP-developedMEICANwebUIfordynamiccircuitprovisioning:

o https://wiki.rnp.br/display/secipo/AutoGOLE+MEICAN+Pilot

• ExpandedstaffinvolvementinAutoGOLEactivities,withcontributorsfromtheengineeringgroupsofPNWGPandCENICaswellasrepresentationfromtheCENICNOC,andwithagoalofgainingfamiliaritywithMEICANwebUIandtheunderlyingelements.

KeyOutcomes&Accomplishments:DataTransferNodes(DTN)AspartofthecollaborationwiththePRP(PacificResearchPlatform),wereprovisionedacontributedFIONA(FlashI/ONetworkAppliance)with96TB(raw)storagefordeploymentasaDTNonPacificWaveLosAngeles.Thestoragesubsystemisbasedon16x8GBSAS2spindlesmanagedasaZFSpool.TheDTNwasinitiallyconnectedat20Gbps(organizedasa2x10GbpsLACP-bundle),andwehavesinceupgradedtheconnectivityto1x10Gbps+1x100Gbps.TheDTNisrunningCentOS7,underwhichwecantakeadvantageofTCPpacingofmultiplestreamstoreachhighereffective(aggregate)throughput. Weareintheprocessofacquiring100GbpsconnectedDTNsforPacificWaveSeattle,Sunnyvale,andLosAngeles.Goalsincludetakingadvantageofrecentadvancesinflash-basedstorage.Ourintentisto

5

provideflexibleaccesstoamodestamountofveryfastNVMe-basedstorageandalargeramountofstoragebasedonSAS3spindles. AsoftheendofJune,2017,weareinthefinalstageofvettingthehardwarespecificationsforthenewDTNswithequipmentproviders.Duetothechallengesofoptimizingthecomponentselection,thevettingprocesshasbeenlongerthaninitiallyexpected.FulldeploymentandintegrationoftheDTNsmayslipintoYear3activities. Wecontinuetoexploredata-transfertoolsetsandsupportingtechnologiesintheinterestofprovidingresearchersarangeofoptionsfromwhichtochoosethedata-movement/data-placementmethodologywhichbestfitstheirscienceworkflow.InadditiontoGridFTPandasaGlobusendpoint(w/GlobusProvider-’network'subscription),wehaveexploredmakingthePRP-contributedPWaveDTNinLosAngelesavailableusingApache-wget,SquidproxywithOAuth2.0,andssh(scp,ssh-ftp/sftp):

• Apache-wget&Squidproxy-OAuth2.0:WorkingwithAndrewHowardinsupportofresearchersmovinggenomicdatatoSingapore’sNSCC,wehavesetupaccesstotheDTNviaApache/wget,usinglocalaccounts.Onourroadmap,weintendtostandupanauthenticatedSquidproxy/cache,withOAuth2.0authentication.

• ssh(scp,sftp):WorkingwithwithNCMIR-UCSDandtheKisailuslabatUCR,aspartofexploring

movementofimagingdatabetweenthetwoUSfacilitiesandacrossthe‘MicroscopyWormhole’toresearchersinSingapore.

TheDTNparticipatesinthePRPGridFTPMaDDash,whichrepresentsdisk-to-diskperformancedataasthroughputusingthesametoolsasforvisualizingiperf/memory-to-memoryperformance.Underthecurrenttestregime,a10GBreferencefileistransferredamongparticipatingDTNs.Dependingupontheotherendpointinthepair-wisemeshtesting,wehaveobservedtheDTNtohaveanaveragedisk-to-diskperformanceof~14Gbps. WorkingwithGlobus,weestablishedtheDTNasa(public)GlobusManagedEndpoint,partofatrialofGlobus’ProviderSubscriptionService.ThisispartofourexplorationofmakingtheDTNavailableasaresourcefor‘live’researchdatamovement.Inthiscontext,thewrite-accesstotheDTNisobtainedthroughusingGlobus’AWS-basedorchestrationforfederatedauthentication.TheDTNisalsoavailableasapublicanonymoustest-pointforread-onlytransfersofasubsetofthereferencedatafilesprovidedbyESnet. TheDTNalsoparticipatesinourAutoGOLE/NSIactivitiesasaresourceinthePacificWaveLosAngelesNSIdomain:

urn:ogf:network:lsanca.pacificwave.net:2016:topology:dtn0.lsanca TheDTNparticipatesintheAutoGOLE‘demo’rangeofVLANsIDs,1779-1799.Inthiscontext,theDTNisprimarilyusefulasatest-pointforadhocdisk-to-diskperformancetesting.

6

IncollaborationwithAndrewHowardandinsupportofaSingAREN-initiatedrequestforassistancetheLosAngelesDTNwasusedfortransferringgenomicdata-setsbetweentheUSandresearchersatSingapore’sNSCC.ThisconfigurationalsowasusefulincomparingthroughputtoSingaporeofdata-setsonthePacificWaveDTNoverR&Epathsversusmovingthesamedata-setsfromAmazon’sAWS. ExplorationofanOpenflow1.0environmentWhileOpenflow1.0wasinitiallyconsideredatestbedfordevelopmentandexperimentation,afterfurtherexaminationanddiscussionwithotherparticipants,itwasdecidedthatutilizingOpenflowwasn’tapaththatmostentitieswantedtopursue.Assuch,wedecidedagainstexploringthistestbedenvironmentanyfurther. TrainingandprofessionaldevelopmentTwomembersofCENIC’scoreengineeringteam,JohnHessandWillBlack,haverotatedintotheprojectforanopportunitytolearnmoreaboutperfSONAR,DataTransferNodes,SDN/SDX,AutoGOLE,andNSI.BothhavebeencontributingtheirknowledgetotheprojectwhilelearningabouttheadvancedtechnologiesusedbyPacificWaveandtheparticipatingNationalResearchandEducationNetworks(NRENs).JohnisalsoactiveinthePacificResearchPlatform(PRP)project,andprovidesastrongtechnicalbridgebetweenthetwoprojects. Resultdisseminationtocommunitiesofinterest PacificWavestaffregularlyattendmeetingsrelatedtoR&EExchangePointswheretheyparticipateindiscussionsregardingthetechnologiesdeployedandunderconsiderationwithinPacificWave.Presentationsincludeupdatesabouttheprojectsand,asappropriate,typicallycontaina“lessonslearned”section. PacificWavealsohoststhebi-weeklyPIRENcallswithparticipantsfromCENIC,PNWGP,UH,AARNet,REANNZ,andGuam.Thesecallsareusedtodiscussitemsofmutualinterest,shareprojectplans,andreviewoutagesthatmayhaveimpactedparticipants.Year3Goals: SoftwareDefinedNetworking/SoftwareDefinedExchange(SDN/SDX) Forthesecondyearofthisproject,wespentmostofourtimegettingtheneededpiecesin-placefortesting.Forthethirdyear,wewillbeinapositiontostartutilizingthosepiecesquiterapidly.Belowisasummaryoftheintegrationplansforthisthirdyear. Initially,wewillconfigureaneBGPsessionbetweentheFRRrouteserverandpeersatAARNetandStarlightusinggeneralpurposeroutersatbothentities.AnyLayer2connectivitythatmaybeneededbetweentheFRRrouteserverandtheseentitieswillutilizeeithertheAutoGOLE/GLIFNSImechanismviatheMEICANtoolor,iftheNSImechanismisunavailable,traditionalLayer2VLAN’s.OncetheseBGP

7

sessionsareestablished,theroutesforthe100GbpsconnectedDTNhostswillbeadvertisedviatheseBGPpeerings.Assumingtherearenootherissues,oncetheseroutesareadvertised,thevariousDTNhostswillbeabletocommunicatewitheitherovertheSDXswitchesusingthesoftware-defineddata-planeforwardingpaththatwillbeprogrammedbytheONOSSDNsoftware.Usingthismodel,someamountoftheforwarding-planewillbeSDN-controlled,andsomewillutilizetraditionalnetworkingmethods.[WillbecompletedbySeptember2017] Afterwehaveconfirmedthisbasicfunctionalityisworking,wewillmovetosetupBGPsessionsbetweentheCastorroute-serversatthethreedifferententities.ThismodifiedconfigurationshouldexpandthesizeandscopeoftheSDXnetworkstoincludemoresoftware-controlledpieces.[WillbecompletedbyNovember2017] Followingallofthesetests,wewillbeinapositiontostartpeeringwithothergeneralPacificWaveparticipantsandleveragetheSDXnetworkevenforentitiesthataren’tusingsoftware-controllednetworks.[WillbecompletedbyFebruary2018] InadditionaltoexpandingthesizeoftheSDXnetwork,wewillcontinueimprovingthereliabilityoftheSDXnetworkbyalsoinstallingtheFRRsoftwareontotheSeattlehypervisorserver,alongwiththeSDNcontrollersalreadypresentinSeattle.[WillbecompletedbyNovember2017] VirtualCustomerEquipment(VCE)SoftwareFirstandforemost,ourprimarygoalistogettheVCEsoftwareworkingwellontheBrocadeintheCENICHQlab.Followingthis,inlateAugust,2017,weplantostartaproductiontrialusingtheLosAngelesPacWaveswitch.ThistrialwillbeaimedatensuringtheVCEfunctionsproperlyinaproductionenvironmentanddoesn’tcauseanyunintendedissues,whilealsoensuringthatcustomerconfigurationsarekeptlogicallyseparatefromoneanother. Aftertheseinitialproductiontrialsarecompleted,wewillmoveforwardinprovidingaccesstoPacWaveparticipants,initiallyonalimitedbasisbuteventuallywithfulldeployment.[InitiallimitedaccesscompletedbySeptember2017,fullydeployedbyDecember2017] AddadditionalcapacitybetweenSunnyvaleandLosAngeles Year3oftheprojectcallsforaddingasecond100GcircuitbetweenSunnyvaleandLosAngelestothePacificWave(andWRN)backbone. PacificWavetrafficpeakedataround85GbpsduringSuperComputing2016.ItisanticipatedthattrafficpeakforthisSuperComputingwillincreasefurtherin2017,witharequirementforanadditional100GbpsofcapacitytobeaddedbeforeSC2018. ThisupgradecallsforCENICtoacquireanotherpairof100GbpstranspondersfortheDWDMsystemonthisroute.CENICwillprocuretheCienatranspondersforthisrouteandmanagethedeploymenttobecompletedandoperationalbytheendofDecember2017.

8

WorkBreakdownSchedulefornew100Gcircuit,LosAngelestoSunnyvale

9/5/17 WorkwithCienaandInternet2onopticalsystemdesign 9/6/17 Confirm100GEportavailabilityonBrocadesinSunnyvaleandLosAngeles 9/18/17 RequestquotesforCienatranspondersfromatleast3resellers 10/2/17 Issuepurchaseorderfortransponders 11/20/17 CoordinatewithLevel3/Internet2forinstallationoftransponders 12/11/17 Completecircuittestingandplacecircuitintoproduction

DataTransferNodes(DTN)Aspartofincreasingourunderstandingofdata-movementworkflowsandtheperformanceofvarioustoolsets,weintendtoinstrumentthePacificWaveDTNstomeasureandcharacterizetrafficpatterns.Tstat(TCPSTatisticandAnalysisTool): http://tstat.polito.it/ Tstatisapassive‘sniffer’whichcangatherstatisticaldataofTCP(andUDP)trafficusingstandardlibrariesoncommodityserverhardware.AsweexpecttoabletorunTstatdirectlyonthePacificWaveDTNs,theutilityshouldprovidevaluableinsightintothebi-directionaltrafficflowsassociatedwithdata-movementactivities.Toaddressprivacyconcerns,TstatprovidesautilitytoanonymizethehostportionoftheIPaddress(IPv4andIPv6)inthecollectedflowdata. Tstat-collectedflowdataisavailableinseveralofoutputformats,includinganintegrationforRRDtool,allowingforvisualizationofperformanceindexesovertimewhilethemeasurementarchiveconsumesa(configurable)fixedamountofstorage.Weintendtomakesummarytrafficstatisticsavailablethroughawebinterfaceandtoexploreotherreportingandvisualizationmechanisms. TstathasbeenusedsuccessfullybyESnettocharacterizetrafficpatternsassociatedwiththeirfourpublicanonymousDTNs,aswellasbytheNetSageproject: http://internationalnetworks.iu.edu/projects/netsage/index.htmlPRODUCTSPresentationsproducedontheproject

Sep22,2016|NorduNet|DavidReese|“PacificWaveandPRP;BigNewsforBigData”Sep30,2016|TechExchange/GLIF|JohnHess|“PacificWaveSDN/SDXUpdate”Oct13,2016|e-ResearchAustralasia|CelesteAnderson|“OvercomingtheTyrannyofDistancein

21stCenturyResearch”Oct18,2016|CANS|JohnHess|“PRPmeasurement,operations,engagement”Feb21,2017|PRP|JohnHess|“PacificWaveSDN/SDXUpdate”Apr26,2017|Internet2|DavidReese|“WesternRegionNetwork”Jun01,2017|CUDI2017|JohnHess|“PacificResearchPlatform”

9

PARTICIPATINGORGANIZATIONSOrganizationsinvolvedaspartnersorcollaborators. AsiaPacificAdvancedNetwork(APAN) AttherecentAPANmeetinginNewDelhi,India,PacificandAPANexecutedanMOU,renewingandextendingtheirlong-standingrelationship.TheMOUcallsforthetwoorganizationstoworktogetherinpromotingtheirrespectiveandmutualobjectivesthroughcollaborationsbetweencustomers/members.ItalsocallsforjointworkreportingontheR&Euseofinterconnectionsbetweennetworksandencouraginguseoftheirnetworksinsupportingthedevelopmentofnext-generationinfrastructureandapplications. PacificIslandsResearchandEducationNetwork(PIREN) PacificWavecontinuestohostbe-weeklyconferencecallswithparticipantsfromPIREN(UniversityofHawaii),AARNet,REANNZ,and,newthisyearasaregularonthecalls,RommelHidalgo,representingtheUniversityofGuam.Thecallsareusedtodiscussjointactivitiesofinterest(suchasNetSageandperfSONAR),activitiesrelatedtotheAARNetlinks,andupcomingevents.ThecallsareleadbyCelesteAnderson,andnotessummarizingthecallsaredistributedtothePIRENmailinglist. PacificResearchPlatform(PRP) PacificWavecontinuestoprovidethePRPwithconnectivitytoregional,national,andinternationalparticipants.PacificWavestaffcontinuetoparticipateinweeklyPRPengineeringcallsaswellasbi-weeklyPRPscienceengagementcalls.PacificWaveissupportingPRPeffortstoexploretrafficengineeringanddata-movementtechnologiesbyprovidingparticipantswithmoreoptionsandtuningknobsforaffectingtheannouncementprofileoftheroutesfortheirScienceDMZresourcesaswellastodevelopthepotentialtosteerrelatedtrafficacrossspecifichigh-speed,inter-domainpaths(akaSuperChannels). PacificWavesuccessfullyfacilitatedarequestthroughARINfortheassignmentofanAutonomousSystemNumber(ASN)specificallyfortheuseofthePRP—PacificResearchPlatform/AS395889.PacificWaveisprovidinghypervisorVMresourcestosupportthePRPv2BGPpilot,whichisexaminingthepotentialofroute-serversoperatingaBGP‘view’ofPRP/AS395889toprovideaBGP-signaledcontrol-planeamongparticipants’networks.PacificWaveLayer2infrastructureprovidestheseparate,high-speeddata-planefordata-movement.

10

AmongthegoalsofthePRPv2BGPpilotistoinformtheprocessofassessingvariousmodelsforscalingfromaregionalanationaland/orglobalmodel,fromaPRPtoanNRP,orGRP:

PRPv2 BGP Pilot: route servers for control planehttp://pacificresearchplatform.org

V0.1.820170510

losa2-pw-sw-1

NOTE: this diagram represents a subset of sites, devices, and connections

rs01.lsanca

Quagga / Juniper vMXRoute Server

Sunnyvale

El Paso

PRPv2 BGP Peering Pilot SitePRP / AS395889 control planeData PlanePacific WaveCENIC CalREN HPR1x100G

Los Angeles

snvl2-pw-sw-1

icas-sttlwa01-03

Seattle

Chicago

Tokyo

Hawaii

Quagga / Juniper vMXRoute Server

rs01.snvaca

rs01.sttlwa

lax-hpr3

SDSC

MREN 8700Starlight/

MREN

NCSA

PRP / AS395889

Oahu

Data Plane

Control Plane

Quagga / Juniper vMXRoute Server

11

NetworkStartupResourceCenter(IRNC:NSRC) InMay,PacificWavestaffparticipatedina5-dayNSRCDirectEngineeringAssistance(DEA)engagementwiththeUniversityofGuam(UoG).ThegoalsoftheDEAincludeprovidingon-sitetrainingandengineeringtowarddevelopinglocalexpertiseovertimetonotonlybenefitthespecificinstitutionbutprovidearesourceforthesurroundingregionaswell.TheDEAteamconsistedofnetworkengineersandsystemadministratorsfromNSRC,UniversityofOregon,PacificWave(CENIC),UoG,andNorthernMarianasCollege(Saipan).Theteamcollaboratedonseveralactivities,including:

• EnhancingtheUoGcampuscoreandbordernetworksinpreparationforhigh-speedR&EtranspacificconnectivitythroughPIREN-coordinatedactivities;

• ReprovisioningconnectivityoftheUoGperfSONARnodefrom100Mbpsto1Gbps.TheperfSONARhadbeendonatedbytheNSRCaspartoftheinitialUoGDEAengagementinJuly,2016.TheperfSONARiscurrentlyparticipatinginregulartestingwiththeUniversityofHawaii,andisintendedtobeincludedinaperfSONARMaDDashhostedbyCENIC.

• InstallationofaPRP-donatedDTN.TheDTNisa‘FIONette’(alow-costversionoftheFlashI/ONetworkAppliancesdevelopedforthePRP),andisconfiguredwith1Gbpsconnectivityand2TB

Pending move / add / change

Berkeley

SDSU

UCR

v0.40.720170712

Pacific Research Platform v1http://pacificresearchplatform.org

PRP ParticipantPRP Layer3 / AS395889 (in development) Pacific Wave

CalREN HPR Layer3 / AS2153

UCSB

UCSF

UCI

UCM

svl-hpr3

lax-hpr3

losa2-pw-sw-1

snvl2-pw-sw-1

PNWGP

NASA Ames / NREN

UCD

Stanford

UCSC

Caltech

USC

UCLA

UCSD SDSC

wrn-denv-sw-3

Global Research Platformicas-sttlwa01-03

wrn-elpa-sw-1

ShinagawaTransPAC

Pacific Wave

tri-hpr3

riv-hpr3

wrn-albu-sw-3

NOTE: this diagram represents a subset of sites, devices, and connections

Internet2

Los Nettos

1x100G1x10G PNWGP Chicago

MREN 8700

Starlight/MREN

wrn-denv-sw-4

wrn-albu-sw-4

Otemachi

dtn0.lsanca

PRP - Pacific Wave DTN

UWU.Montana

UIC / EVL

UvA

UCAR / NCAR

FRGP

SURFnet

Sunnyvale

Los Angeles

Seattle

Chicago

Denver

Albuquerque

Riverside

Amsterdam

El Paso

Tokyo

Oahu

Hawaii

AARNet

Brisbane

U.Hawaii

UQ

KREONET

KISTI

NCSA

Daejeon

Sydney

Sacramento

ESnet

U.Utah

UEN

U.Chicago GDC

LBL

NERSC

oak-hpr3

Oakland

12

ofstorage.TheDTNisparticipatingasanodeinthePRPGridFTPdisk-to-diskthroughputtestingregime,andisintendedtoinformthedata-movementrequirementsforsupportingscienceworkflowsofUoG-basedresearchers,including: • MarineLaboratory(GuamEPSCoR-EcosystemsCollaboratoriumdatabaseforcoral

specimensandoceanographicdata)• WERI(WaterandEnvironmentalResearchInstitute)-ProfessorYumingWenisinvolved,and

wearemovingtheirGISlabelsewhereoncampus,whereotherGISresearcherscansharetheequipment.

• CLASS(CollegeofLiberalArtsandSocialSciences)GeographydepartmentGISLab,runbyDr.RominaKing.

• InstantiationofcoreITinfrastructureelementssuchasanNTPserver,ajump/bastionhost,aswellasreorganizingtheserversandsystemssupportingthecampusDNS.

IRNCfundedNetworkOperationsCenter(IRNC:NOC) PacificWavestaffarepartneringwiththeGlobalNOCtoexpandthemappingcapabilityforExchangePoints,circuits,andpeersbeyondthatoftheNSF-fundedIRNCactivities.TheworkwillallowExchangePointstoexportmeasurementdatatotheNOCthatcanbespecificallyclassifiedbaseduponthepoliciesoftheExchangePointanditsParticipant(seeIRNC:AMIformoreinformationonexporttags). ThemappingcapabilitieswerealsoproposedtotheGlobalNetworkArchitecture(GNA)TechnicalTeam,andinterestwasexpressedbytheGNAparticipatingexchangestojointhemappingefforttocreateaglobalmapofR&Eexchangepointsandrelatedinter-connections. IRNCfundedAdvancedMeasurementInfrastructure(IRNC:AMI) ToincreasethevisibilityandmonitoringofPacificWave’sresources,wedevelopedaninterfacetaggingschemethat,combinedwithSNMPpolling,willallowustoexportinterfacedatatotheIRNC:AMIcollector.Itwillalsoallowusgranularcontroloverwhichinterfaceswillexportutilizationandlink-statedatatopublic-facingwebsitesandwhichstillstaycompletelyprivate.WorkbeganinMarch,2016,andshouldbecompletedbySeptember,2017. Thetagdefinitionsare: • <notag> TheinterfacewillbeignoredbyNOCandAMIasifitdidnotexist.• [ns] Thedatawillbecollectedandheldprivate,andthelinkwillnotshowonmaps.• [ns-stat] Thedatawillbecollectedandheldprivate,andthelinkwillshowonmapswithan

“up-down”statusindicator.• [ns-exp] Thedatawillbecollectedandbevisiblepublicly,andthelinkwillshowonmapswith

trafficdata. GlobalLambdaIntegratedFacility(GLIF) TheGlobalLambdaIntegratedFacility(GLIF)hasbeenworkingtoidentifyitsfutureintheGlobal/InternationalExchangelandscapegiventherecentintroductionoftheGlobalNetworkArchitecture(GNA).PacificWavecontinuestoparticipateandtosupporttheGLIF,aswebelievethisiswhereexperimentationonandfuturedevelopmentofexchangepointtechnologiescanbedoneona

13

globalscale.ActivitiessuchasPacificWave’sownSDXexperimentationrequiremultipleparticipantstoexploreandtesttechnologiesthatcanlaterbedeployedinaproductionenvironment.PacificWavewillbepartneringwithStarLightandAARNetonsomedemonstrationsofSDXcapabilitiesattheupcomingGLIFmeetinginSydney,Australia,tobeheldSeptember25-27,2017. GlobalNetworkArchitecture(GNA) PacificWaveparticipationinGNAcontinuesthroughattendanceatallGNAtechnicalmeetings.PacificWaveiscurrentlyworkingonthefinaleditsofaprivacypolicythatwillbepublishedonthePacificWavewebsiteperGNArecommendations.AsmentionedunderIRNC:NOC,PacificWaveisworkingwiththeGlobalNOConamappingtoolthatwilldisplayGNAexchangepoints,circuits,andpeers.

IMPACTS Impactsonthedevelopmentoftheprincipaldivision(s)oftheproject Findings,results,techniquesorotherproductsthatweredevelopedorextendedfromtheproject,andmadeanimpact,orarelikelytomakeanimpactonthebaseofknowledge,theory,andresearchand/orpedagogicalmethodsintheprincipaldisciplinaryfield(s)oftheproject. PacificWaveisleadingthewayinthedeploymentofScienceDMZresourceswithinanExchangePoint.Duringthisprojectyear,PacificWavewilladdDataTransferNodes(DTNs)inSeattle,SanJose.andLosAngeles.TheoutcomeofthisexperimentcouldsignificantlyalterhowExchangePointsarebuiltandoperatedinthefuture.Inaddition,thePacificResearchPlatform(PRP)wasinitiallybuiltonthePacificWaveandWesternRegionNetwork(WRN)infrastructure.ThishasalreadyresultedinseveralinterestedorganizationsreachingouttoPacificWaveandWRNwithanexpressedinterestinexpandingtheinfrastructureandina“NationalResearchPlatform”meetingscheduledinBozeman,MTinAugust2017. Therealimpactiscreatingandsustainingtheinfrastructuretosupporttheworkcarriedoutbyothers.Insomeways,predictingwhenandifhighcapacitynetworksareneededisdifficult,butwhentheyareneeded,capacityneedstobeavailable.Wearepoisedtosupportmultiplelarge-scaleprojectswithverylargedatarequirements,suchastheLargeSynopticSkyTelescope(LSST)andtheSquareKilometerArray(SKA).Thereareincreasingneedstosupportlargedatatransfersforcollaborationsingenomics,physics,astronomy,andearthsciences,tonameafew. Whatistheimpactonotherdisciplines? Findings,results,ortechniquesthatweredevelopedorimproved,orotherproductsfromtheprojectthatmadeanimpactorarelikelytomakeanimpactonotherdisciplines. PacificWaveisworkingwithotherregionalandnationalresearchandeducationnetworkstodeploycomponentsofthenetwork.ExamplesofthisarethevarioussoftwarecomponentsintheSDN/SDXdeployment(ONOS,etc),andtheMEICANprovisioningtooldevelopedbyRNPandusedbyPacificWave.

14

Throughexperimentaldeploymentsofthesesoftwarestacksand/ortools,theproductsthemselvesareimprovedthroughfeedbackregardingproblemsreportedand/orfeaturesrequested.Thisprocessresultsinmorestablesolutionsforfutureproduction-qualitydemandsandwiderdeployment. Howtheprojectmadeanimpactorislikelytomakeanimpactonhumanresourcedevelopmentinscience,engineering,andtechnology. ThedeploymentofSoftwareDefinedNetworks/SoftwareDefinedExchanges(SDN/SDX)islikelytohaveanimpactonthetrainingandskillsrequiredoffuturenetworkengineers.Engineerswillnolongerjustberunningexperimentsonaparticularproduct,butwillalsorequiresignificantsoftwaredevelopmentskillsinordertodesign,build,operate,anddebugfutureadvancednetworks. Whatistheimpactonphysicalresourcesthatforminfrastructure? Waystheprojectmadeanimpact,orislikelytomakeanimpact,onphysicalresourcesthatforminfrastructure,includingfacilities,laboratories,orinstruments. Data-intensivescienceapplicationshavealwaysbeenmajordriversoftechnologybecausetheyencountertechnicalbarrierslongbeforetheselimitationsareexperiencedwithinotherapplicationdomains.Multiplesciencedomainsaremotivatingthecreationofnewadvancedcommunicationservicestomeettherequirementsofthemostdata-intensiveapplicationsintheworld,includingthoseinhigh-energyphysics,astronomy,computationalastrophysics,genomics,andcomputationalchemistry. Newservicesarerequiredtosupportlarge-volume,high-qualitydigitalmedia,3Ddigitalmedia,andultra-high-definitiondigitalmedia.Digitalmediamustbesupportedasacommon,ubiquitousservice,notasahighlyspecializedcapability. ThePacificWaveinfrastructureiscontinuouslyevolvingtomeetingthesegrowingdemands.TheadditionofAutoGOLE/NSIandSDN/SDXcapabilitieswerespecificallyselectedtosupportthedemandsofresearchlaboratoriesanddistributednetwork-connectedinstruments.Whatistheimpactoninstitutionalresourcesthatforminfrastructure?Ways,inwhichtheprojectmadeanimpact,orislikelytomakeanimpact,oninstitutionalresourcesthatforminfrastructure. PacificWave,workingwiththePacificResearchPlatform(PRP),isrefiningthedefinitionoftheinstitutionalScienceDMZandhowitinterconnectswith,andintegratesinto,theregional,nationalandinternationalinfrastructure.BridgingthegapbetweenthescientificresourceswithintheinstitutionalScienceDMZandthoseoftheircollaboratorshasledtotheconceptof‘thenetworkasaninstrument,’firstproposedbyGregBellofESNet. Creatinganeasilyaccessibleandconfigurablemethodforconstructingpoint-to-pointandmulti-pointvirtualnetworksacrossmanyboundarieswillgreatlyenhancetheabilityofeducationandresearchinstitutionstoservetheneedsoftheirresearchersandscientists.

15

Waysthattheprojectmadeanimpact,orislikelytomakeanimpact,oninformationresourcesthatforminfrastructure. PacificWaveisengagedwiththeotherIRNCfundedprojects(PIREN,AmLight,Starlight,NOC,andAMI)aspartofboththeoperationalandexperimentalaspectsofoperatingandmaintaininganexchangepoint.PacificWaveisalsoengagedonaninternationallevelwithorganizationsliketheAsiaPacificAdvancedNetwork(APAN),andisnowaregularparticipantandcontributortotheGlobalNetworkInfrastructure(GNA). Throughtheseefforts,PacificWavesharesourexperiencesandusesthatknowledgetohelpgrowandexpandtheresearchandeducationinfrastructureavailabletotheresearchandeducationcommunity. Theimpactontechnologytransfer PacificWave,asanexchangepoint,isessentiallyamodern-daytechnical“meetingplace”ofnetworksfromaroundtheworld.Networksconnecttoexchangepointsfortheexplicitpurposeofconnectingtoothernetworksconnectedtotheexchangepointfortheinterchangeofresearchandeducationtraffic.Becausetechnologydevelopmentmovesatsucharapidpace,thenetworksconnectedatanexchangepointareoftenatdifferentstagesofexperimentationanddevelopment.Thisleadstoacommon“lifting”ofnetworkstothemoreadvancedusesoftechnologywhileprovidingaprovinggroundfortheleading-edgeideasofnetworkresearchers. Theimpactonsocietybeyondscienceandtechnology Theabilityofresearchersandscientiststoflexiblyfashioninternationalcollaborationscreatesnewavenuestofurtherresearchinmanyareas.Asthearchitectureoftheexchangeevolvestoprovideeasilyconstructedpathwaysthroughmultiplenetworksandorganizationsbetweenparticipantsaroundtheglobe,thespeedwithwhichadhocinternetcollaborationscanbeformedtosolvereal-timeproblemsshouldgreatlyshortenthetimelineofnewdiscoveries.

CHANGES/PROBLEMSChangesinapproachandreasonsforchange ActualorAnticipatedproblemsordelaysandactionsorplanstoresolvethem CENICandPacificWavearecontinuingtomonitortherecentacquisitionofBrocadebyExtremeNetworks.AstheacquisitionisnotsettocloseuntiltheendofAugust,2017,therehavebeennoannouncementsregardingthefutureoftheBrocadeMLX-EproductlinedeployedthroughoutPacificWave.Untiltheseplansareannounced,wewillbecloselywatchingouravailablepoolofportsonthePacificWaveswitchesandwillonlypurchaseadditionalcapacity(hardware)onanas-neededbasis.