DO-AS-CASPOTT: DevOps AutoScaler and Critical Artifact Spotter

PAGE1 |GRACEHOPPERCELEBRATION2016 | #GHC16PRESENTEDBYTHEANITABORGINSTITUTEANDTHEASSOCIATIONFORCOMPUTINGMACHINERY

#GHC16

2016FeedbackDriven Developmentof CloudApplicationsHariniGunabalan,TechnicalUniversityof Darmstadt,Germany@harinigunabalan



Motivation

Cloud


Motivation

Cloud

Developer

Operator

EndUsers


Motivation

Cloud

Developer

Operator

EndUsers

App App

Deploy app


Motivation

Cloud

Developer

Operator

EndUsers

App App

Deploy app

VMVM VM VM

Provision resources


Motivation

Cloud

Developer

Operator

EndUsers

App App

Deploy app

VMVM VM VM

Provision resources

UseApp


Cloud

Developer

Operator

EndUsers

App App

Deploy app

VMVM VM VM

Provision resources

UseApp

Notworking!

Motivation


Motivation

Cloud

Developer

Operator

EndUsers

App App

Deploy app

VMVM VM VM

Provision resources

UseApp

Where is the problem?Notworking!


Motivation

Cloud

Developer

Operator

EndUsers

App App

Deploy app

VMVM VM VM

Provision resources

UseApp


Maybe,Ishouldimprovecode?


Motivation

Cloud

Developer

Operator

EndUsers

App App

Deploy app

VMVM VM VM

Provision resources

UseApp


Maybe,Ishouldimprovecode?

Is the resourceprovisioningnotsufficient?


ProblemStatement


ProblemStatement– APMInformation

What is Feedback Driven Development?


ProblemStatement- FDD

Problem: How to automate Resource provisioning using the Run-time information

Problem: How to identify Source Code Issues (root cause analysis) using Run-time information








ProblemStatement- FDD









WhatisCASPOTT?CriticalArtifactSpotter.Identifiessourcecodeissuesandperformancehotspots.

Solution: Combination of plug-ins that support the Developer (CASPOTT) and Operator (Auto-scaler)

WhatisAuto-scaling?On-demandscaleoutandscaleindepending onLoad.


SystemDesign

ArchitectureofDO-AS-CASPOTT

17

CASPOTTComponents

18

FeedbackHandler- Preparestherawlogs

forfurtherprocessingbymappingthemtotheAPIformatoftheStatisticalAnalysisEngine

StatisticalAnalysisEngine- Analyzesandaggregates

thecollectedinformationcomingfromFeedbackHandlerbasedonpredefinedmetrics

StaticCodeAnalysis- Analyzesthecode

andrealizestheconnectionbetweenthefeedbackandthecode

Auto-scalerComponents

19


20


21


22


Implementation


CASPOTT

• ImplementedasanEclipsePlug-in

• Storerun-timeinformation inElasticSearch

• Displayhotspots toDeveloper

• DirectMapping tosourcecode


Component1:CloudMonitoring

Keymetricsmonitored• Averageresponse time(CF Logs)• Numberofincomingrequests(CF

Logs)• Numberofrunningappinstances

(CFAPI)• CPUutilization(CFAPI)• Memoryutilization(CFAPI)• Diskutilization(CFAPI)

Component2:Auto-scaling

26


• Modeldatatoidentifycorrelationbetweenmetrics• MostrealworlddataareMulti-dimensional.• Multiple-inputandMultiple-output(MIMO)models.Ø State-spaceModelØ PolynomialModelsØ ARXØ ARMAX

Component3:DataModeling

27


1. SetOutput=[ResponseTime]

2. SetInput=[Numberofincoming requests,Noofinstances,CPU%,memory,anddiskutilization]

DataModeling inMATLABSystemIdentificationToolbox:ImportData

28


MATLABSystemIdentificationToolbox– Datasetsplitting

29


• Modelestimation

• State-spaceModel• Polynomial

Models• ARX• ARMAX

• Validatethemodel

ModelEstimationandValidation

30


EstimatingStateSpacemodel

31


EstimatingARXandARMAXmodels

32


Evaluation

33

Image Source: https://www.usu.edu/ccampis/evaluation/


DeployGuestbookApplicationtoCloud

34


GenerateLoadonGuestbookapp

35


Scalingoccurs

36

Requestsperse

cond

No.ofinstances


ComparisonwithandwithouttheAuto-scaler

37

Requests

persecond

Average

response

time

Requests

persecond

Average

response

time

WithoutAuto-scaler,Maximumresponsetimeis750ms

WithAuto-scaler,Maximumresponsetimeis100ms


VideoDemoofAuto-scaler


Conculsion

39

Auto-scaler

Solved!ü CASPOTTü Auto-scalerü CorrelationModel to

maketheauto-scalersmart!

CASPOTT


• Metricparametertuning• Metriccombination&CustomMetrics• Determiningmetricthresholds• Testingwith differentapplications types - memory or database

intensive.• Improvedmodeling,collectionoflargersampledatasets

OpenChallengesandFuturework


References

Ø M.Httermann,DevOpsfordevelopers.Apress,2012.Ø J.Cito,P. Leitner,H.C.Gall,A.Dadashi,A.Keller,andA.Roth,“Runtimemetricmeetsdeveloper:buildingbettercloudapplications

usingfeedback,” in2015ACMInternationalSymposiumonNewIdeas,NewParadigms,andReflectionsonProgrammingandSoftware,pp.14–27,ACM,2015.

Ø “Amazonwebservices.”https://aws.amazon.com/,Accessed:08-Aug-2016.Ø “Newrelic.”https://newrelic.com/,Accessed:08-Aug-2016.Ø J.Cito,P. Leitner,T.Fritz,andH.C.Gall,“Themakingofcloudapplications:Anempiricalstudyonsoftwaredevelopmentforthe

cloud,” inProceedingsofthe201510th JointMeetingonFoundationsofSoftwareEngineering,pp.393–403,ACM,2015.Ø L.Ljung,Systemidentification.Springer,1998.Ø “Saphana cloudplatform.”https://hcp.sap.com/index.html,Accessed:08-Aug-2016.Ø S.R.Seelam,P.Dettori,P.Westerink,andB.B.Yang,“Polyglotapplicationautoscalingserviceforplatformasaservicecloud,” in

CloudEngineering(IC2E),2015 IEEEInternationalConferenceon,pp.84–91,IEEE,2015.Ø J.HumbleandD.Farley,Continuousdelivery:reliablesoftwarereleasesthroughbuild,test,anddeploymentautomation.Pearson

Education,2010.Ø http://docs.pivotal.io/pivotalcf/1-7/customizing/autoscale-configuration.htmlØ Cloudwaveproject- http://cloudwave-fp7.eu/

41


Thankyou

Supervisors:

Prof.Dr.–Ing.MiraMezini

Dr.–Ing.GuidoSalvaneschi


Appendix- StateSpaceModels

• Representedbydifferentialequationsandstatevariables.• Outputcanbepredictedforanyfuturetimeprovidedtheinput,output,anda

minimumsetofstatevariablesxi(t),areknown.dx/dt =Ax+Buy =Cx+Du

43


Appendix- Polynomialmodels- ARX&ARMAX

ARX:

• Autoregression - currentoutputdependsonthepastinputandoutputvalues.

• Considering auto regression andthe current inputs, ARXmodelcan be mathematicallydescribed as:A(z)y(t)=B(z)u(t- n)+e(t)

44

ARMAX:

• UnlikeARX,ARMAXconsiders stochasticdynamics.

• Betterforsystemswithextradisturbances

• IncludesbothAR(p)andMA(q)models.

• ARMAXisrepresentedmathematicallyasA(z)y(t)=B(z)u(t- n)+c(z)e(t)


Appendix:Polynomialmodels- ARX

• TheARXmodel toevaluatetheoutput isbasedonauto-regression.• Autoregressivemodel isamodelwhosecurrentoutputdependson thepastinput

andoutputvalues.Thegenericnotion todenoteauto-regressivemodeloforderp,AR(p) foravariableXis:

where cand iare constants and e(t)is the noise.Considering auto regression and theinputs,ARXmodel canbe mathematically described as:

A(z)y(t)=B(z)u(t - n)+e(t)wherey(t)istheoutput, u(t)istheinput,ande(t)isthenoise/errormeasuredintheoutput.A(z)andB(z)arepolynomialsof thespecifiedorderwithrespecttothebackwardshiftoperatorz-1.

45


Appendix:Polynomialmodels- ARMAX

UnliketheARXmodel,inARMAX,thestochasticdynamicsareconsidered.ARMAXmodelsarebetterforsystemswithmoredisturbances.Ingeneral,themovingaveragemodeloforderq,MA(q)isrepresentedinthebelownotation:

wheree(t-i)isthenoise/error.Thenotationfortheautoregressivemovingaverage(ARMA)modelisasbelow:

ThismodelincludesbothAR(p)andMA(q)models.BasedonthesethefollowingmathematicalequationfortheARMAXmodelcanbewrittenas:

A(z)y(t)=B(z)u(t- n)+c(z)e(t)where,y(t)istheoutput, u(t)istheinput,ande(t)isthenoise.A(z),B(z)andC(z)arepolynomials

ofspecifiedorderswithrespecttothebackwardshiftoperatorz-1

46


Appendix- Modeloutput

TheModelestimationoutput forStatespace,ARXandARMAXmodels.

47


Appendix:CASpott

FeedbackHandler- Queriestherawlogs- Preparestherawlogsfor

furtherprocessingbymappingthemtotheAPIformatoftheStatisticalAnalysisEngine

StatisticalAnalysisEngine- Analyzesandaggregatesthe

collectedinformationcomingfromFeedbackHandlerbasedonpredefinedmetrics

StaticCodeAnalysisEngine- Analyzesthecodeandrealizes

theconnectionbetween thefeedbackandthecode


Appendix- SystemFootprintforAuto-ScalerImplementation

49

Technology

DO-AS-CASPOTT: DevOps AutoScaler and Critical Artifact Spotter