ConceptsandTechnologiesforDistributedSystemsand

BigDataProcessingGuidoSalvaneschi

1

People

GuidoSalvaneschi

PascalWeisenburgerMatthiasEichholz

[…]2

Motivation

• Distributedsystemsareubiquitous• Emails,WWW,…

• Areaisquicklychanging- sorryforthebuzzwords:)• Bigdataanalytics• Complexeventprocessing/streamprocessing• Cloudcomputing• Newprogrammingmodels(streams,actors)• ...

• Yet,manyconceptsremainthesame• Abstractionoverlow-leveldetails• Faulttolerance• Performance:throughput,latency• Asynchonouscommunication• ...

3

ConceptsandTechnologiesforDistributedSystemsandBigDataProcessing

• ProvideanoverviewofrecentdevelopmentindistributedsystemsandBigDataprocessing

• Focusonconcepts,notontechnology.• Technology wasdifferent5yearsago,willbedifferent in5years• Wewilllookattechnologyonlyasawaytobettergrasptheconcepts

• Selectedtopics• Differentspeakersforeachtopic+guestlectures• Dr.GuidoSalvaneschi,Dr.MichaelEichberg,Prof.PatrickEugster,Dr.AlessandroMargara,Prof.Philipp Haller,…

4

Aboutthiscourse

WhatthiscourseIS• Anintroduction onselected topics• …ondistributedsystemsandbigdataprocessing• Acourseaboutconcepts…andabitabouttechnologies

WhatitisNOT• Itisnot(only)acourseaboutrecenttrendsinBigData.• Itisnotasystematic courseondistributed systems

• DistributedSystems:Principles andParadigmsAndrewS.Tanenbaum,MaartenvanSteen

• Itisnotatutorial onhowtoprogrambigdatasystems

5

April12- Intro,motivationApril19- Introtobigdata,mapreduceApril26- hadoop,hdfsMay4- ComplexeventprocessingMay10 ComplexeventprocessingMay17– ConcurrentprogrammingMay24- Futures,actors,streamsMay31- Component-based architecturesJune7- RESTarchitecturesJune14- Sparkstreaming/Spark?June21- GeodistributionJune28- Security,ResourcemanagementJuly5– DBGuestlecture?July12– Guestlecture/Exampreparation

July26- Exam

Tentativecourseschedule

Dateandroomchange!11:40-13:20CETroomS101/A03

Bigdataandcomplexeventprocessing

Architecturesandprogrammingmodelsfordistr.systems

Cloudcomputingandadvancedtopicsinresourcemanagement

Subjecttochange!Seecoursewebsiteforupdates 6

Exercises

• Nogradedexercises• Exerciseswillbeprovidedaftersomelectures.• Solutionsdiscussedinthenextlectureifneeded

• Why• Getmoreconfidenton thetopicsofthelectures• Getafeeling thetypeofquestions thatcancomeup intheexam

• Examples• Readapaper• Answerquestionsbasedonthecontentofthelecture• Discussacasestudy• Smallcodingexercises

7

RegistrationandGrading

• RegisterinTucan forthecourse!

• Writtenexam– July26• 90mins

• Questionsaboutthetopicscoveredinthelectures• Simple“programming”tasks• Understandacodesnippet thatisrelevantforoneofthecoveredtopics• Completeacodesnippet

8

Resources

• Website(slides,exercises)• http://stg-tud.github.io/ctbd/• Thisistheofficialplaceslides,exercises,updatedschedule,etc.

• Forum• https://www.fachschaft.informatik.tu-darmstadt.de/forum/viewforum.php?f=580

• Pleaseusetheforumforyourquestions.Answerswillremainasareferenceforotherpeople.

• Pleaseaskassistantsforquestionsthatcannotbepostedontheforum• Butthink ifthisisreallythecase:)

9