Upload
others
View
18
Download
0
Embed Size (px)
Citation preview
DistributedDatasetSynchronizationin
NamedDataNetworkingWentaoShangFinaldefense06/01/2017
1
ResearchProblem
• Distributedapplicationsrequireefficientsupportformulti-partycommunication• Multiplenodespublishandsharedata
• NamedDataNetworking(NDN)enablesnewwaystosupportmulti-partycommunicationthroughdatasetsynchronization(sync)• Leveragingdata-centricnetworkarchitecture• Withoutcentralizedserver
2
StateofAffairs
• AnumberofsyncprotocolshavebeendevelopedsincethestartoftheNDNproject• CCNx 0.8Sync;ChronoSync;iSync;CCNx 1.0Sync;RoundSync;pSync
• AnumberofexistingNDNapplicationsrunontopofsync• CCNx repo:replicateddatastorage• ChronoShare:distributedfilesharing• ChronoChat:server-lessgroupchat• NLSR:link-stateroutingprotocol• NDN-RTC:groupconferencing• Distributeddatacatalog• IoT pub-subsystem
3
ResearchObjectives
• UnderstandingthedesignspaceofNDNsync• SystematicexaminationofalltheexistingNDNsyncprotocols
• Designinganewsyncprotocol• Learningfromthedesigntradeoffsintheexistingprotocols• Supportingnewfunctionsnotofferedbytheexistingworks• Applyingmethodsdevelopedinthedistributedsystemsarea
4
NDNOverview
• Uniqueandsecuredbindingbetweennameandcontent• Namedata,andsecuredatadirectly
• Name-baseddataretrieval• Stateful Interest-Dataexchange• Secureddataenablesin-networkstorage
5
/ucla/cs/wentao
Fetch:/ucla/cs/wentao/slides/v5
Interest
Data
/ucla/cs/wentao/slides/v5
InterestFetch:/ucla/cs/wentao/slides/v5
NDNSyncforMulti-PartyCommunication
• Enableagroupofnodestopublishandconsumedatainashareddataset• Maintainaconsistentstateofthedatasetamongtheparticipants
• NDNprovidesuniquebindingbetweennameanddataàSynchronizingdataset=synchronizingthenamespaceofthedataset
• FullyutilizeNDN’sdata-centriccommunication• In-networkcaching• Multicastdatadelivery
6
SyncinNDN
7
NationalParkTouristA
TouristB
TouristC
…/road/X/hazard/road/Y/closed
Bulletinboard…
/road/X/hazard/road/Y/closed
Bulletinboard
…/road/X/hazard/road/Y/closed
Bulletinboard
SyncinNDN
8
NationalPark
Publishalertdata:“BearspottedatsiteZ”
TouristA
TouristB
…/road/X/hazard/road/Y/closed
Bulletinboard
TouristC
…/road/X/hazard/road/Y/closed
Bulletinboard
SyncinNDN
9
NationalParkTouristA
TouristB
…/road/X/hazard/road/Y/closed/site/Z/bear
Bulletinboard
Synchronize
…/road/X/hazard/road/Y/closed
Bulletinboard
TouristC
…/road/X/hazard/road/Y/closed
Bulletinboard
SyncinNDN
10
NationalParkTouristA
TouristB
…/road/X/hazard/road/Y/closed/site/Z/bear
Bulletinboard …/road/X/hazard/road/Y/closed/site/Z/bear
Bulletinboard
TouristC
…/road/X/hazard/road/Y/closed
Bulletinboard
SyncinNDN
11
NationalParkTouristA
TouristB
…/road/X/hazard/road/Y/closed/site/Z/bear
Bulletinboard …/road/X/hazard/road/Y/closed/site/Z/bear
Bulletinboard
TouristC
…/road/X/hazard/road/Y/closed/site/Z/bear
Bulletinboard
ComparingNDNSyncwithToday’sDataSynchronizationSolutions• TraditionalSynchronizationwithTCP/IPnetworking• Networkprovidespoint-to-pointcommunication• Datasetsynchronizationachievedattheapplicationlayer
• SyncinNDN• Networkprovidesdata-centriccommunication• Syncprotocolprovidesdatatransportservicefortheapplication• Becauseofdata-centricnature,NDNsyncdoesnotrequireallpartiesconnectedtoeachotherallthetime
12
Dataset
DesignSpaceofNDNSyncProtocols
13
CommonSyncProtocolFramework
14
/road/X/hazard/road/Y/closed/site/Z/bear
Summary
DatasetnamespaceGenerateaconcisesummaryofthedatasetnamespacetobecommunicatedbetweennodes
Update
Detectandreconcileinconsistencybyexchangingthesummaryperiodically
(Optionally)supportquicknotificationtoothernodeswhenpublishingnewdata
Summary
Summary
Summary
KeyDesignAspects
• Datasetnaming• Howtonamedataitemsintheshareddataset
• Namespacerepresentation• Howtoprovideanefficientsummaryofnamespace
• Statesynchronizationmechanism• HowtomakenodeslearnaboutchangesASAP• Howtodetectandreconcileinconsistencycausedbyvariousfactors
15
DesignChoicesinDatasetNaming
• Syncprotocolsynchronizesapplicationdatanamesdirectly• CCNx 0.8Sync;iSync;CCNx 1.0Sync
• Syncprotocolnamesdatabyeachproducersequentially• Encapsulateapplicationnamesifneeded• ChronoSync;RoundSync;pSync
16
/road/X/hazard/road/Y/closed/site/Z/bear
/TouristA/13:{/site/Z/bear}/TouristA/14:{/site/W/alert}/TouristB/55:{/road/X/harzard}/TouristB/56:{/road/Y/closed}
DesignChoicesinNamespaceRepresentation
• Enumeration• Losslesscompressioninthenamespace(ornocompression)• CCNx 1.0Sync
• Hashing• One-waycompressionofnamespace• CCNx Sync;ChronoSync;RoundSync
• InvertibleBloomFilter(IBF)• Storeandextractindividualnamehashes• iSync;pSync
17
DesignChoicesinStateSynchronizationMechanism• Long-livedInterest• NodesmaintainpendingInterestsinthenetworktosolicitchangesfromothers• ChronoSync;pSync;
• Notification-driven• Nodesinformothersaboutnewchanges• CCNx 1.0Sync;RoundSync;
• Periodicexchangeofdatasetsummary• Nodesexchangetheirstatesummaryperiodicallytodetectandreconcileinconsistency• CCNx 0.8Sync;iSync
18
EvolutionofExistingSyncProtocols
19
Useapplicationname Namedatasequentially
Enumeration
Hashing
IBF
Long-livedInterest
Notificationdriven
Enumeration
Hashing
IBF
CCNx 0.8Sync
CCNx 1.0Sync
iSync pSync
W.Shangetal.,“ASurveyofDistributedDatasetSynchronizationinNamedDataNetworking”,NDN-TR-0053,2017
Periodicexchange
Long-livedInterest
Notificationdriven
Periodicexchange
ChronoSyncRoundSync
CCNx 0.8Sync
• Summarizedatasetnamespaceusingcombinedhashesovertreestructure• SendInterestwithroothashperiodicallytorequestdifferenthash(es)• Takemultipleroundstoreconcilethedifferences
20
/
/road /site/Z/bear
/road/X/hazard /road/Y/closed
H3=Hash(/road/X/harzard) H4=Hash(/road/Y/closed)
H1=H3+H4
H2=Hash(/site/Z/bear)
H0=H1+H2RootAdvice Interest:H0’
RootAdvice reply:H0
NodeFetch Interest:H0
NodeFetch reply:H1,H2
NodeFetch Interest:H1
NodeFetch reply:H3,H4
…
iSync:ImprovingCCNx 0.8Sync
• UseInvertibleBloomFilter(IBF)tosummarizethenamespace• DetectdifferencesusingIBFsubtraction
• Reducethesynchronizationround-tripatthecostoflargernamespacerepresentation• ExchangeonlytheIBFdigest• NeedextraRTTtoretrievetheIBFcontent
• BothCCNx 0.8SyncandiSync synchronizeviaperiodicexchangeofstatesummary• Addadditionaldelaytolearningnewdata
21
/road/X/hazard/road/Y/closed/site/Z/bear
InvertibleBloomFilter
01de…478a…33fc…
Hash
IBF Digest
Hash
Store Extract
ChronoSync
• Namedatasequentially• Summarizethenamespacewithadigest• Maintainlong-livedInterestinthenetworktowaitfornextupdate• Need“excludefilter”toretrievesimultaneousupdatesbymultipleproducers
• Interestcarriesstatedigestforinconsistencydetection• Providea“recovery”mechanismaslastresortforrepairingstateconflict
22
…/TouristA/12/TouristA/13
…/TouristB/54/ToursitB/55
…/TouristC/29/TouristC/30
{/TouristA:13,/TouristB:55,/TouristC:30}
DigestHash
TouristA
TouristB
TouristC/park/sync/[Digest]
{/TouristB:56}
{/TouristC:31}
X
pSync:Pub-suboverSync
• TakethesequentialnameapproachfromChronoSync,IBFasrepresentationfromiSync• IBFstoresonlyeachnode’slatestseq#,sosizeisdeterminedbythegroupsize
• Eachconsumersendslong-livedInterestwitholdIBFtorequestupdatesfromaproducer• IBFprovidesspecificinformationabouttheconsumer’sstate• Producercanreplywithnewdatanamesdirectly
23
/producer/[BF]/[old-IBF]
/producer/[BF]/[old-IBF]/[new-IBF]{/site/Z/31}
SyncInterest
Reply
/road/X/13 /road/Y/55
Store
InvertibleBloomFilter
ExtractH1 H2 H3
Hash/site/Z/30
OtherSyncProtocols
• CCNx 1.0Sync:anotherfixtoCCNx Sync• Enumeratedatanamesinamanifestfile• Broadcastmanifestdigestwhenpublishingnewdata
• RoundSync:arevisiontoChronoSync• Reducebutnoteliminatethesimultaneouspublishingproblem
24
LessonsLearned
• Allowingsyncprotocoltonamedatasequentiallysimplifiesthedesign• Onlyneedtosynchronizethelatestsequencenumbers
• Notificationsshouldcarryspecificupdateinformation• Sothatrecipientscanfetchnewdatadirectly,withoutfurtherexchangetoidentifythenewdata
• Avoidusinglong-livedInteresttofetchnewupdates• Along-livedInterestcannotfetchmultipledataproducedatthesametime• Long-livedInterestsaddburdentonetworkinmaintainingInterestpathstate
25
VectorSync Protocol
26
SynchronizationwithManagedGroup
• Manydistributedapplicationsrequireexplicitgroupmembershipmanagement• Examples:• ResourcediscoveryinIoT networks• Routingprotocol
• Existingsyncprotocolsdonotsupportmembershipmanagement• Difficulttoremovedepartednodesfromthedatasetstate
27
/Home/Thermostat/Home/AC/Home/Heater/Home/MotionSensor
VectorSync DesignHighlights
• Maintainaconsistentlyorderedlistofgroupparticipants(calledaview)amongallparticipatingnodes• Utilizealeader-drivenprocesstosynchronizetheviewamongallnodes• Leveragesequentialdatasetnamingtosynchronizethedatasetusingversionvector• Adoptnotification-drivensynchronizationwithspecificupdateinfo
28D.Parkeretal.,“DetectionofMutualInconsistencyinDistributedSystems”,1983
VectorSync Overview
29
ApplicationLogic
SharedDataset
DatasetStateStatesynchronization
PublishData
NotifyofRemoteData
NDNNetwork
Dataretrieval
GroupMembershipInfo
Viewsynchronization
SyncNode
VectorSync Overview
30
Statesynchronization
PublishData
NotifyofRemoteData
NDNNetwork
Dataretrieval
Viewsynchronization
DatasetState
GroupMembershipInfo
SyncNode
ApplicationLogic
SharedDataset
DatasetNamespace
• Sequentialdatanaming• Usingsequencenumbers
• Vectorrepresentationofnamespace• Producernamesandorderingspecifiedinthemembershipinfo
31
Statevector
[17,240,153]
Membershipinfo
0:Thermostat1:AC2:MotionSensorNode
order
Shareddataset…
/…/Thermostat/16/…/Thermostat/17
…/…/AC/238/…/AC/239/…/AC/240
…/…/MotionSensor/151/…/MotionSesnor/152/…/MotionSensor/153
Thermostat:17 AC:240 MotionSensor:153
PublishingandSynchronizingData
32
• MulticastnotificationInterestcarriesexplicitinformationaboutchanges• Nodedatacarriesfullstatevectorofthepublisher,providescausalordering
MotionSensor
Thermostat
AC
Publish/…/AC/241 NotificationInterest:/home/bonjour/AC/241
Replywithstatevector
Interest:/…/AC/241
Data:/…/AC/154app message,vector=
[17,240,153]
DetectingandReconcilingInconsistency
33
[17,240,153]
Receivedvector:[18,242,151]
Localvector:
Join
[18,242,153]Updatedvector:
Membershipinfo
0:/…/Thermostat1:/…/AC2:/…/Heater
/…/Heater/152/…/Heater/153Missingdata:
UpdatethelocalvectorwiththeresultofJoinandretrievemissingdata
VectorSync Overview
34
ApplicationLogic
SharedDataset
DatasetStateStatesynchronization
PublishData
NotifyofRemoteData
NDNNetwork
Dataretrieval
GroupMembershipInfo
Viewsynchronization
SyncNode
Soft-stateMembership
• Nodesrefreshtheirmembershipbypublishingdata(inthedataset)• Authenticatedassertionofexistence
• Whenapplicationisidle,nodepublishes“heartbeat”dataperiodically• Enableperiodicexchangeofstatevectors
• Anodeisconsidered“gone”ifnodatareceivedfromitforcertainamountoftime• Heartbeatperiodandtimeoutvaluesdecidedbyapplication
35
Thermostat
AC
MotionSensor
…/…/Thermostat/56appdata/…/Thermostat/57appdata …
/…/MotionSensor/101heartbeat/…/MotionSensor/102appdata
…/…/AC/23appdata/…/AC/24heartbeat
Leader-drivenMembershipManagement
• Nodesselectaleadertomanagethegroup
• Leaderdefinesandpublishesitsviewofthegroup• Othernodesfollowtheleader’sview
• Leadermonitorsthegroupandcreatesnewviewwhenthemembershipchanges
• Viewsarenamedsequentiallyusingviewnumber• Leaderincreasesthenumberwhencreatingnewview
• Uponnetworkpartition,eachpartitionmayselectitsownleaderwhichcreatesitsownview• ViewID=(viewnumber,leadername)
36
0:Thermostat;1:Heater;2:AC;3:MotionSensor
(1,Thermostat)
0:Thermostat; 1:AC;2:MotionSensor
(2,Thermostat)
0:Thermostat1:MotionSensor
(3,Thermostat)0:AC(3,AC)
Selectingaleader
• Ifthecurrentleaderhasleft,theremainingnodescompetetobecomethenextleader viarandomleaderselection• Whenanodedetectsleaderdeparture,itstartsarandomwaittimer• Afterthetimergoesoff,thenodedeclaresitselftheleaderandcreatesanewview• Ifitnoticesanewviewbeforethetimergoesoff,itcancelsthetimerandjointhenewview
• Otherleaderselectionmechanismcanalsobeused• Usingpre-configuredpreferencelist
37
SynchronizingtheView
• Leadersignsandpublishesviewinfoasdatawhencreatingnewview• Name:/[multicast-prefix]/vinfo/[view-id]• Containsnamesandcertificatesofthemembers
• ViewIDcarriedinallnotificationInterests
• Nodefetchestheviewinfoifnoticingalargerviewnumber• Jointhenewviewafterreceivingtheviewinfo
• Nodekeepspublishingdatainitscurrentviewbeforetheviewsynchronizationfinishes
38
0:Thermostat,{cert}1:AC,{cert}2:Heater,{cert}3:MotionSensor,{cert}
Thermostat’skeySign
/home/bonjour/vinfo/(1,Thermostat)
ViewSynchronizationwithSingleLeader
39
View(4,Thermostat):{0:AC;1:MotionSensor;2:Thermostat;3:Heater}
ThermoStat
MotionSensor
Heater
AC
RemoveHeaterPublishviewinfofor
(5,Thermostat)
/home/bonjour/(5,Thermostat)/…
/home/bonjour/vinfo/(5,Thermostat)
/home/bonjour/vinfo/(5,Thermostat)
/home/bonjour/vinfo/(5,Thermostat)0:AC,{cert};1:MotionSensor,{cert};
2:Thermostat,{cert}
Moveto(5,Thermostat)
Moveto(5,Thermostat)
UpdatingStateAfterMembershipChange
40
0:AC,{cert}1:MotionSensor,{cert}2:Thermostat,{cert}3:Heater,{cert}
(4,Thermostat)
[17,240,153,98]
0:AC,{cert}1:MotionSensor,{cert}2:Thermostat,{cert}
(5,Thermostat)
[17,240,153]
RemoveHeater
ReconcilingMultipleViews
41
HeaterThermostatACMotionSensor
Inview(4,Heater)Inview(6,Thermostat)
/home/bonjour/vinfo/(4,Heater)
/home/bonjour/vinfo/(4,Heater)0:Heater,{cert}1:MotionSensor,{cert}
/home/bonjour/(7,Thermostat)/…
Publish:/home/bonjour/vinfo/(7,Thermostat)0:Thermostat,{cert};1:AC,{cert}
3:Heater,{cert}4:MotionSensor,{cert}
Fetch(7,Thermostat)viewinfo andmoveto(7,Thermostat)
Aftergrouppartitionheals,theleaderwithhigherviewnumberor“larger”leadernameisresponsibleformergingtheviews
/home/bonjour/(4,Heater)/…
SynchronizingStateAfterViewMerging
42
Thermostat AC Heater MotionSensor0:Thermostat1:AC
0:Heater1:MotionSensor
[24,19] [23,21] [98,120] [96,121]
0:Thermostat1:AC2:Heater3:MotionSensor
Synchronizeview
[24,19,0,0] [23,21,0,0] [0,0,98,120] [0,0,96,121]
[24,21,98,121]Synchronize
state
SecuringDatasetSynchronization
43
ApplicationTrustAnchor/Home
/Home/Thermostat /Home/AC /Home/Heater
0:/Thermostat,{cert}1:/AC,{cert}2:/Heater,{cert}
/Home/resource/disc/vinfo/(4,/AC)
“10:30AM,Temp:70F”Vector=[15,100,92]
/Home/Thermostat/15HEARTBEATVector=[60,148,135]
/Home/Heater/135
Sign
SignSign Sign
Summary
• Explicitgroupmembershiplist(theview)enablestheuseofversionvectorasaconciserepresentationofthedatasetnamespace
• Event-drivenotificationwithexplicitinformationallowsnodestoretrievenewdataimmediately
• KeepingconsistentviewamongallnodesbyincludingtheviewIDinallnotificationInterests• NodescanretrievetheviewinfoafterreceivingnewviewID
• Publishingallstateinfo(vector,membershiplist)asnamedandsecureddatausingwell-definednamingconvention
44
SimulationStudy
• Methodology• ConductexperimentsusingndnSIM (ns3-basednetworksimulator)• ComparewithChronoSync usingthesameapplication
• Metrics• Datadisseminationdelay:thetimeneededforanynodetoreceivethedataafteritispublished• Synchronizationdelay:thetimeneededfortheallnodesinthegrouptoreceivethedataafteritispublished• Networktraffic:totalnumberofInterestandDatapacketstransmittedinthenetwork
45https://github.com/named-data/ChronoSync
Scenario:smallcampusnetwork
BB1 BB2
DR4DR3
DR1 DR2
GW1 GW2 GW3 GW4 GW5
GW6GW7
Department ADepartment B
Department C
CampusBackbone
10Gbps/5ms
1Gbps/10ms
1Gbps/5ms
EN1 EN2 EN3 EN4 EN5 EN6
EN7EN8EN9EN10
1Gbps/2ms
46
• Applicationrunningon10edgenodesparticipatinginasyncgroup• RTTrange:8ms~68ms
• Twotypesoftraffic:• Lowdatarate:10saverageinter-arrivaltimeforeach• Highdatarate:1saverageinter-arrivaltimeforeach
DataDisseminationDelaywithNoPacketLoss
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
Data dissemination delay in VectorSync (ms)
CD
F
●●
●●●●●●●●●●● ● ●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●● ● ●●●
●●●●●●●●●●●●● ●●
●●●●●●●●●●
●●●●●●
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
Data dissemination delay in VectorSync (ms)
CD
F
●●●●●●●●●●●●●●●● ●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●
●●●●●
●● ●●●●
●●●●●●●
●●●●●●●●●●
47
~1.5*MinRTT
~ 1.5*MaxRTT
~1.5*MinRTT
~ 1.5*MaxRTT
DataRate=0.1pps DataRate=1pps
SynchronizationDelayinVectorSync
48
DataRate=0.1pps DataRate=1pps
HigherdatarateinthegroupenablesVectorSync torecoverfrompacketlossfasterbecausethestatevectorcarriedineachdataenablesinconsistencydetection
0 500 1000 1500 2000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in VectorSync (ms)
CD
F
●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●
●●●
●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●● ● ●●●●● ●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
Loss rate0%1%5%10%
0 500 1000 1500 2000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in VectorSync (ms)
CD
F
●●●●●●●●●
●●●
●●●●
●●
●●●●●●●
●●●
●●●●●●●
●●●●●●●●●
●●●
●●●●●●
●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●● ●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●● ● ●● ● ● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●● ● ● ● ● ●●
Loss rate0%1%5%10%
SynchronizationDelayunderLowDataRate(VectorSync vs.ChronoSync)
49
VectorSync isresilienttosimultaneousupdatebecauseexplicitnotificationallowsreceiverstofetchthenewdataimmediately
0 500 1000 1500 2000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in VectorSync (ms)
CD
F
●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●
●●●
●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●● ● ●●●●● ●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
Loss rate0%1%5%10%
0 500 1000 1500 2000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in ChronoSync (ms)
CD
F
●●
●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●
●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●● ●●●●●●●● ●●●●● ●●●●● ● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●Loss rate
0%1%5%10%
Duetosimutaneous data
NetworkTrafficunderLowDataRate(VectorSync vs.ChronoSync)
50
MainreasonforhigherInterestvolumeinChronoSync:• AdditionalmulticastSyncInterestfordetectingsimultaneousupdates• RecoveryInterestforrepairingconflictingstates
LossRate 0% 1% 5% 10%
PacketType Interest Data Interest Data Interest Data Interest Data
VectorSync 167k 134k 170k 134k 183k 132k 205k 129k
ChronoSync 321k 132k 359k 151k 436k 172k 455k 154k
Totalnumberofpacketsacrossalllinks
SynchronizationDelayunderHighDataRate
51
Underhighdatarate,ChronoSync invokes“recovery”mechanismfrequently,whichprovidessimilarinformationasthestatevectorinVectorSync
0 500 1000 1500 2000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in VectorSync (ms)
CD
F
●●●●●●●●●
●●●
●●●●
●●
●●●●●●●
●●●
●●●●●●●
●●●●●●●●●
●●●
●●●●●●
●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●● ●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●● ● ●● ● ● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●● ● ● ● ● ●●
Loss rate0%1%5%10%
0 500 1000 1500 2000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in ChronoSync (ms)
CD
F
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●● ● ● ●●● ● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●● ●●●● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●● ● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●● ●●● ● ●●●
Loss rate0%1%5%10%
NetworkTrafficunderHighDataRate
52
LossRate 0% 1% 5% 10%
PacketType Interest Data Interest Data Interest Data Interest Data
VectorSync 101k 82k 104k 82k 112k 80k 126k 79k
ChronoSync 1179k 482k 989k 417k 730k 300k 576k 215k
Totalnumberofpacketsacrossalllinks
• HightrafficvolumeinChronoSync duetofrequentinvocationofrecoverymechanism
• TrafficvolumeinVectorSync increasesonlyslightlyduetoInterestretransmission
Summary
• CarryingdatanameexplicitlyinthenotificationInterestenablespromptandefficientdatadissemination,eveninfaceofmultiplesimultaneousdataproducers
• Carryingstatevectorinnodes’dataenablesdetectingandreconcilinginconsistencyinthedatasetnamespace
• HighergroupdatarateenablesVectorSync torecoverfrompacketlossmorequickly
53
EvaluatingViewSynchronization
• Understandingthebehaviorofviewsynchronizationundernodejoining/leavingandpacketloss• Howfastthegroupreactstomembershipchanges• Messagingoverhead• Additionaldelay• Implicationsofprotocolparametersettings
• Resultstobeincludedinthedissertation
54
ComparisonwithExistingProtocols
55
Enumeration
Hashing
IBF
Enumeration
Hashing
IBF
CCNx 0.8Sync
CCNx 1.0Sync
iSync pSync
ChronoSync RoundSync
VectorSync
Long-livedInterest
Notificationdriven
Periodicexchange
Long-livedInterest
Notificationdriven
Periodicexchange
Useapplicationname Namedatasequentially
ComparisonwithExistingProtocols
56
Factors affectingInterestsize
FactorsaffectingDataContentsize InterestOverhead MinData
DisseminationRTT
CCNx 0.8Sync Nodehash Numberofchildnodes Periodic Depending onInterestperiod+treewalk
CCNx 1.0Sync Manifestdigest Total numberofnames Oneperupdate 2.5
iSync IBFdigest IBF(dependingonnumberofnewdata) Periodic Depending onInterest
period+3.5RTT
ChronoSync State digest(withexcludefilter)
Number ofnameswithnewsequencenumbers Long-livedInterest
Minis0.5; canbelongwithsimultaneousdata
publishing
RoundSync Rounddigest(withexcludefilter)
Numberofnameswithnewseq# inaround Two perupdate
Minis1.5;canbelongwithsimultaneousdata
publishing
pSync** IBF(+subscriptionlist) IBF +numberofnewnames Long-livedInterest 1.5
VectorSync Leadername+nodename State vector(small) Oneperupdate
(withheartbeat) 1.5
** pSync doesnotsupportgroupsync
Conclusion
57
Conclusion
• DistributeddatasetsynchronizationisanimportantabstractioninNDNforsupportingdistributedapplications
• Ourcomparativestudyoftheexistingsyncprotocolsidentifiedcommonsyncprotocolframeworkandexposeddifferenttradeoffsintheprotocoldesignchoices
• WedevelopedVectorSync,anewsyncprotocoldesignthatovercomethedrawbacksoftheexistingsyncprotocols• Explicitgroupmembershipmanagement• Explicitnewdatanotification• Detectingandreconcilingdatasetinconsistencyusingversionvector
58
FutureWorks
• Exploringdifferentgrouprendezvousmechanisms• DHT• Viralpropagation
• ApplyingVectorSync toNDNapplications• Routingprotocol• Distributedrepository
59
Publications
60
Ø NDN.JS:aJavascript ClientLibraryforNamedDataNetworking,Infocomm’13Ø NDNFS:anNDN-friendlyFileSystem,NDN-TR-0027,2014Ø MicroForwarder.js:anNDNForwarderExtensionforWebBrowsers,ICN’16
NDNApplications
Ø SecuringBuildingManagementSystemsUsingNamedDataNetworking,IEEENetwork,vol.28,no.3,2014Ø NDN-ACE:AccessControlforConstrainedEnvironmentsoverNamedDataNetworking,NDN-TR-0036,2015Ø ChallengesinIoT NetworkingviaTCP/IPArchitecture,NDN-TR-0038,2016Ø NamedDataNetworkingofThings,IoTDI’16Ø TheDesignandImplementationoftheNDNProtocolStackforRIOT-OS,Globecom’16ICNSRAWorkshopØ Breakingoutofthecloud:LocaltrustmanagementandrendezvousinNamedDataNetworkingofThings,IoTDI’17
IoT overNDN
Ø TheDesignofRoundSync Protocol, NDN-TR-0048,2017Ø ASurveyofDistributedDatasetSynchronizationinNamedDataNetworking,NDN-TR-0053,2017
NDNSync
BackupSlides
61
RoundSync:addressingissuesinChronoSync
• Dividedatapublishingintorounds• Decouplestatenotificationfromupdateretrieval• Stillneedspecialmechanismtoretrievemultipledatainasingleround
62
RN Dataset Digest
…
24 {/TouristA/15,/TouristB/60} D2425 {/TouristC/32} D25
/park/bbs/DATA/25
{/TouristC:32}
DataInterest
Reply
RoundLog /park/bbs/SYNC/[Digest]SyncInterest
JoiningtheGroup
63
MotionSensorThermostatAC Heater
Inview(8,Thermostat) Createview(0,Heater)
Publish:/home/bonjour/vinfo/(0,Heater)0:Heater,{cert}
/home/bonjour/(0,Heater)/Heater/1
/home/bonjour/vinfo/(0,Heater)
Merge(0,Heater)intothegroupandperformviewchange
Nodejoiningishandledinthesamewayasviewmerging
Scenario2:largeISPnetwork
• Randomlypick10“leaf”nodes• RTTrange:111ms~476ms
• Traffic:10sinter-arrivaltimefromeachnode
64N.Stringetal.,“MeasuringISPtopologieswithRocketfuel”,2004
DataDisseminationDelaywithNoPacketLoss
65
~1.5*MinRTT
<1.5*MaxRTTduetocaching
0 200 400 600 800 1000
0.0
0.2
0.4
0.6
0.8
1.0
Data dissemination delay in VectorSync (ms)
CD
F
SynchronizationDelay
0 5000 10000 15000 20000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in ChronoSync (ms)
CD
F
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●● ●
Loss rate0%1%5%10%
66
0 5000 10000 15000 20000
0.0
0.2
0.4
0.6
0.8
1.0
Data synchronization delay in VectorSync (ms)
CD
F
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●● ●●●●●● ● ●
Loss rate0%1%5%10%
NetworkTraffic
67
LossRate 0% 1% 5% 10%
PacketType Interest Data Interest Data Interest Data Interest Data
VectorSync 519k 185k 522k 185k 534k 184k 575k 187k
ChronoSync 1224k 248k 1346k 273k 1616k 317k 1647k 294k
Totalnumberofpacketsacrossalllinks
HightrafficvolumeinChronoSync dueto“recovery”mechanism
DatasetSnapshotandPermanentStorage
68
Motivation
• Problem:• VectorSync synchronizesamongactivemembers• Someapplicationsmaywanttokeepalldatapublishedinthehistory
• Solution:• Thegroupgeneratesa“snapshot”forthedatasetstate atthebeginningofeachview• Asnapshotisaversionvectorcoveringallthedatapublishedinthegroupbeforetheviewstarts
• A dedicatedrepocollectsdatabasedonthesnapshotandstorepermanently• Newnodesretrievewholedatasetfromtherepotobootstrap
69
GeneratingGroupSnapshot
• Beforesyncingdatainanewview,eachnodepublishesitslocalsnapshotintheshareddataset• Localsnapshotpacketspropagatedinthegroupviasync
• LeadercomputesthegroupsnapshotastheJoin oflocalsnapshotsfromall nodesinthecurrentview
70
Previousview:(6,/HVAC)[/Thermostat:35,/HVAC:103]
Previousview:(6,/HVAC)[/Thermostat:34,/HVAC:103]
/Thermostatlocalsnapshot /HVAClocalsnapshot /Lamplocalsnapshot
Previousview:(4,/Lamp),(6,/HVAC)[/Thermostat:35,/HVAC:103,/Lamp:55,/LampSwitch:20]
(7,/HVAC)groupsnapshot:
Previousview:(4,/Lamp)[/Lamp:55,/LampSwitch:20]
(7,/HVAC)0:/Thermostat1:/HVAC2:/Lamp
Summary
• Effectiveness• Synchronizinglatestseq#ensuresnodesdiscoverallmissingdata• Periodicheartbeathelpsrecoverfrompacketloss
• Efficiency• Explicitdatanamenotificationenablesfastersync• Simultaneouspublishingdonotinterfere
• Scalability• Membershipmanagementcontrolsstatevectorsize• Largegroupsmayadoptcompressedencoding(e.g.,IBF)
71