CSCI 325: Distributed Systemssprenkle/cs325/slides/00-intro.pdf · Sept 8, 2017 Sprenkle - CSCI 325 15 Source: Livinginternet.com “It seems reasonable to envision, for a time 10

1

CSCI325:DistributedSystems

ProfessorSprenkle

Objec?ves• Courseoverview• Overviewofdistributedsystems•  Introduc?ontoreadingresearchpapers

Sept8,2017 Sprenkle-CSCI325 2

2

DistributedSystems?• Whatisadistributedsystem?• Knowanyexamplesofdistributedsystems?

Ø Anyused?


DistributedSystems?• Whatisadistributedsystem?

Ø Collec?onsofindependent,networkedcomputersworkingtogether

• ExamplesofdistributedsystemsØ Networkedprinters,storageØ InternetØ Peer-to-peersystemsØ Gridcompu?ngØ GamesØ Sensornetworks


3


DistributedSystemsArchitectures• Twomainmodels

Ø Client-server• Mostcommon,andarguablythesimplest• Mul?-?erclient-server

Ø Varia?ononthesimpleclient-serverarchitectureØ Mul?plelevelsofcommunica?on

Ø Peer-to-peer• Allprocessesinvolvedinataskorac?vityplaysimilarroles


4

Client-ServerModel


Client

Client

Client

ClientClient

Server

request

request

requestresponse

response

response

Peer-to-PeerSystems

Peer

Peer

Peer

PeerPeer

request

request

request

reque

strequest

response

response

response

response

Sept8,2017 8Sprenkle-CSCI325

Connections between peers

5

Challenges• Whatarechallengesindealingwithdistributedsystems?


DistributedSystemsChallenges• Communica?on• Naming• Distribu?onofworkload• Distribu?ontransparency• Consistency• Handlingfailure• Security• Scaling


6

WhatThisCourseisAbout• Networkingfundamentals• Distributedsystems

Ø ChallengesofdistributedsystemsØ DesignprinciplesØ Learnhowtobuildlarge-scaledistributedsystems

• Severalprogrammingprojects• Emergingresearchissues

Ø Studyfundamentalresearchpapers• Life-skills

Ø Reading,wri?ng,discussion,presenta?on

Sept8,2017 Sprenkle-CSCI325 11Overall goal: Emphasize “why” and “how” over “what”

Bonus: OS

ALITTLEBITOFHISTORYWhatmadedistributedsystemspossible?


7

TheInternet•  Connec?onofcomputernetworksusingtheInternetProtocol(IP)Ø Allowsnetworkapplica?ons,e.g.,email,filetransfer,world

wideweb,remotelogin,…


Internet

VannevarBush•  EstablishedtheU.S.military/universityresearch

partnershipthatdevelopedARPANET•  Wrote1stvisionarydescrip?onofpoten?aluse

forinforma?ontechnologyØ  inspiredmanyofInternet'screators


Source: Livinginternet.com

Could you envision the WWW years before it existed?

“Consider a future device for individual use, which is a sort of mechanized private file and library. It needs a name, and to coin one at random, ‘memex’ will do. A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory. “It consists of a desk, and while it can presumably be operated from a distance, it is primarily the piece of furniture at which he works. On the top are slanting translucent screens, on which material can be projected for convenient reading. There is a keyboard, and sets of buttons and levers. Otherwise it looks like an ordinary desk.”

- Vannevar Bush, “As We May Think,” Atlantic Monthly, July 1945

8

J.C.R.Licklider•  JosephCarlRobnef“Lick”Lickliderdevelopedideaofuniversalnetwork

•  SpreadhisvisionthroughouttheIPTO(Informa?onProcessingTechniquesOffice)

•  Inspiredhissuccessorstorealizehisdreambycrea?ngARPANET


Source: Livinginternet.com

“It seems reasonable to envision, for a time 10 or 15 years hence, a ‘thinking center’ that will incorporate the functions of present-day libraries together with anticipated advances in information storage and retrieval.“The picture readily enlarges itself into a network of such centers, connected to one another by wide-band communication lines and to individual users by leased-wire services. In such a system, the speed of the computers would be balanced, and the cost of the gigantic memories and the sophisticated programs would be divided by the number of users.”�

- J.C.R. Licklider, Man-Computer Symbiosis, 1960.

Background• 1957:USSRlaunchesSputnik,firstar?ficialearthsatelliteØ U.S.respondsbyformingAdvancedResearchProjectsAgency(ARPA)

• 1962:Licklider’sGalac?cNetwork• 1966:MarillandRoberts(MIT)paper:“TowardaCoopera?veNetworkofTime-SharedComputers”Ø hfp://dl.acm.org/cita?on.cfm?id=1464336

• 1967:Roberts(MIT):ACMSOSP“Mul?pleComputerNetworksandIntercomputerCommunica?on”Ø hfp://dl.acm.org/cita?on.cfm?id=811680


9

1969InternetMap:ARPANET


1st assignment:draw today’s Internet

Stanford Research Institute

1st message: “LO” as in “Lo and Behold”(supposed to be “LOG” but failure!)From UCLA to SRIOct 29, 1969, 10:30 p.m.

SDSSigma7

http://www.nsf.gov/news/special_reports/nsf-net/1960s.jsp https://www.nsf.gov/news/special_reports/nsf-net/kleinrockvideopop.html


10

InternetTimeline


Year Milestone1971 Tomlinsondevelopsemailprogram,bighit1972 Telnet1973 FileTransferProtocol(FTP)

1974 TransmissionControlProtocol(TCP)

1978 TCPsplitintoTCPandIP(InternetProtocol)1979 USENET(newsgroup)established

1984 1000hostsconnectedtoInternet,DNSintroduced

1988 Internetwormbringsdown10%ofInternet1991 WAIS,Gopher,WWWreleased

InternetGrowthTrends


Year Hosts on Internet1977 1111981 2131983 5621984 1000

1986 50001987 10,0001989 100,000

1992 1,000,0002001 151-175 million2002 Over 200 million2014 1.01 billion

# of computers connected directly to the Internet increased at a yearly rate >37% across 21 years

https://www.internetsociety.org/sites/default/files/Global_Internet_Report_ 2014_0.pdf

11

Sta?s?csfromtheIITFReport“TheEmergingDigitalEconomy”*• Togetamarketof50Millionpeoplepar?cipa?ng:Ø Radio:38yearsØ TV:13yearsØ Internet:4years

• Aqeropentogeneralpublic• hfp://govinfo.library.unt.edu/ecommerce/EDEreprt.pdfØ ReleasedonApril15,1998


* Delivered to the President and the U.S. Public on April 15,1998 by Bill Daley, Secretary of Commerce and Chairman of the Information Infrastructure Task Force


12

COURSEINFO


MyResponsibili?es• Prepareuseful,interes?ngknowledge• Cometoclassprepared,on?me•  Interes?ng,relevant,andchallengingassignments

• Promptfeedbackonassignments


13

YourResponsibili?es• Cometoclassprepared,on?me,andPARTICIPATE

• Turninassignmentson$me• Whenyou’rehavingtrouble

Ø LookforhelpontheWeb• Find,adaptsolu?ons• Givecredittowhereyoufoundsolu?on,ifnovelenough

Ø Askmeforhelp!• Learn,absorb,synthesize

Ø ExtraCredit:takeittothenextlevelSept8,2017 Sprenkle-CSCI325 25

Textbook• Required:DistributedSystems,byvanSteenandTanenbaum,3rded.Ø Providesbackgroundforclassdiscussionsandprojects

Ø Availableonline

• Op?onal:DistributedSystems,ConceptsandDesigns,byCourlouis,Dollimore,Kindberg,5thed.


14

Grading• 17%Individualprogramming,reading,wri?ngassignments

• 20%Midtermexam• 33%Programmingprojects• 25%FinalProject

Ø Includingpaperandpresenta?onØ Startthinkingaboutpossibletopics

• 5%Par?cipa?onandafendanceØ Successofclassdependsonstudentpar?cipa?on


ProgrammingProjects• 3projectsspanningthesemester

Ø Hands-onconstruc?onofinteres?ngdistributedservices

Ø Approximately2.5weekstocompleteØ Workinteamsof2or3

• UseversioncontrolØ Startearly!


15

READINGRESEARCHPAPERS


WhattoLookForWhileReading• Overallproblem

Ø Howlarge/importantistheproblem?• Goals• Contribu?ons

Ø Keywords:new,novel• Technicalapproach

Ø Keyinsights(“leverage”,“u?lize”)• Evalua?on

Ø Answersallyourques?onsaboutapproach?•  Limita?ons

Ø Maynotbeageneral-purposesolu?onØ Checkassump?ons


16

SomeConcreteQues?ons•  StatementoftheProblem/Goals

Ø  Trytostatesuccinctlytheoverallproblembeingaddressedinthispaper.Ø  Whatpar?culargoalsdotheseresearchershaveinaddressingthis

problem?Ø  Whatcontribu?onaretheyseekingtomaketothestate-of-the-art?

•  TechnicalApproachØ  Whatisthekeyinsightofthisgroup'sapproachtotacklingthestated

problem?Whatistheiroverallapproach/strategytosolvingtheproblem?•  Discussion/Cri?que

Ø  Howdidtheresearchersevaluatetheirefforts?Ø  Whatconclusionsdidtheymakefromtheirevalua?onresults?Ø  Whatapplica?on/usefulbenefitdotheresearchers/youseeforthiswork?Ø  Whatlimita?onsdotheresearchersmen?onwiththeirapproach?Ø  Whataddi?onallimita?onsdoyouthinkthereare?Ø  Writeoneinteres?ngques?ontoponderwithregardtothispaperbeyond

contentunderstanding.


SEDA


WeproposeanewdesignforhighlyconcurrentInternetservices,whichwecallthestagedevent-drivenarchitecture(SEDA).SEDAisintendedtosupportmassiveconcurrencydemandsandsimplifytheconstruc?onofwell-condi?onedservices.InSEDA,applica?onsconsistofanetworkofevent-drivenstagesconnectedbyexplicitqueues.Thisarchitectureallowsservicestobewell-condi?onedtoload,preven?ngresourcesfrombeingovercommifedwhendemandexceedsservicecapacity.SEDAmakesuseofasetofdynamicresourcecontrollerstokeepstageswithintheiropera?ngregimedespitelargefluctua?onsinload.Wedescribeseveralcontrolmechanismsforautoma?ctuningandloadcondi?oning,includingthreadpoolsizing,eventbatching,andadap?veloadshedding.WepresenttheSEDAdesignandanimplementa?onofanInternetservicesplawormbasedonthisarchitecture.WeevaluatetheuseofSEDAthroughtwoapplica?ons:ahigh-performanceHTTPserverandapacketrouterfortheGnutellapeer-to-peerfilesharingnetwork.TheseresultsshowthatSEDAapplica?onsexhibithigherperformancethantradi?onalservicedesigns,andarerobusttohugevaria?onsinload.

17

SEDA• Problem/Goals

Ø Highlyconcurrentinternetsystems• Goal:well-behavedunderload

• TechnicalApproachØ Staged,event-drivenarchitecture(SEDA)Ø Automa?ctuning,loadcondi?oning

• DiscussionØ Evalua?on:UsedSEDAarchitectureforwebserver,P2Ppacketrouter• Measuredperformance,robustnesstoloadvaria?on


ReadingFeedback:Annota?ons• Perusall:Applica?onaccessiblethroughSakai

Ø Allowsyoutocommentonanar?clesuchthatallstudentsandprofessorcanviewthem

• Youwillbeexpectedtomakeacertainnumberofannota?onsoneachar?cleØ “Certainnumber”=5,typically

• Annota?onscanbeques?onsorcommentsØ mustbesubstan?ve

• Eachannota?onwillbegradedasØ 2:thoughwul;full-creditØ 1:par?al-creditØ 0:thoughtlessornotcomplete;nocredit


18

TODO• SetupPerusall,throughSakai• ExploreCourseWebPage• Checkout“WelcometotheMachine”

Ø ReviewingsometermsfromCSCI210(plusmaybemore)

• ReadE2EArgumentpaperforFridayØ Skimthroughonce,reviewsec?onheadingsØ 3hoursmaxØ Reviewpaper

• Write5annota?onsinPerusallØ Wed:Discusspaperandques?ons


Documents

CSCI 325: Distributed Systemssprenkle/cs325/slides/00-intro.pdf · Sept 8, 2017 Sprenkle - CSCI 325 15 Source: Livinginternet.com “It seems reasonable to envision, for a time 10