Tutorial 5: Modern Web Applica5ons’ Reliability Engineeringblogs.ubc.ca/karthik/files/2016/10/Issre-tutorial-2016.pdf · Web Applica5ons: Challenge 35 S 2 S 0 S 1 S … S N S 4

Tutorial 5: Modern Web Applica5ons’ Reliability

Engineering

KarthikPa*abiramanElectricalandComputerEngineeringUniversityofBri>shColumbia(UBC)

1Copyright:KarthikPa*abiraman,2016

Introduc5ons

• WhoamI?•  AssociateprofessoratUnivofBri>shColumbia,Vancouver,Canada•  ResearchinterestsinsoMwarereliabilityandsecurity•  Beenworkingonwebapplica>ons’reliabilityforpastsixyears(from2010)

• Whoareyou?•  Name,affilia>on,andwebapplica>onexperience(ifany)

•  IEEEReliabilityCer>ficate(pleaseaddyournametosignupsheet)


Modern Web Applica5ons: Examples


Modern Web Applica5on: Amazon.com

Amazon’sownad

Menubar SearchFunc5on

ThirdpartygadgetadWeb2.0applica5onsallowrichUIfunc5onalitywithinasinglewebpage

Copyright:KarthikPa*abiraman,2016 4

Modern Web Applica5ons: JavaScript •  JavaScript:Implementa>onofECMAScriptstandard

•  Client-SideJavaScript:usedtodevelopwebapps•  Executesinclient’sbrowser–sendAJAXmessages• Responsibleforwebapplica>on’scorefunc>onality• Noteasytowritecodein–hasmany“evil”features

5 Copyright:KarthikPa*abiraman,2016 5

JavaScript: History

JavaScript(JS)hadto“looklikeJava”onlylessso,beJava’sdumbkidbrotherorboy-hostagesidekick.Plus,IhadtobedoneintendaysorsomethingworsethanJSwouldhavehappened–BrendanEich(InventorofJavaScript)

Copyright:KarthikPa*abiraman,2016

BriefHistoryofJavaScript(Source:TomBarker.com)

6

JavaScript: Prevalence •  97oftheAlexatop100websitesuseJavaScript•  Thousandsoflinesofcode,oMen>10,000

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

Google

YouTub

e

Yaho

o

Baidu

QQ

MSN

Amazon

Sina

WordP

ress

Ebay

Linked

In

Bing

MicrosoM

Yand

ex

163

mail.ru

PayPal

FC2

Flickr

IMDb

Apple

Livedo

or

BBC

Sohu

go.com

Soso

Youku

AOL

CNN

Med

iaFire

ESPN

MySpace

MegaU

pload

Mozilla

4shared

Adob

e

Abou

t

LiveJournal

Tumblr

GoDa

ddy

CNET

YieldM

anager

Sogou

Zedo

Ifeng

PirateBay

ImageShack

Weather

NYTimes

Nen

lix

Linesofcode


8

JavaScript:Most popular language

JavaScript:Top languages on GitHub


JavaScript and the Web

9

Full Stackclient server


Client-Side JavaScript

10

EASYTODEPLOY

Writecode

Openbrowser

Webappinac>on!



11

FlexibleprogrammingfeaturesàRichuserinterac5ons



12

Clicktoexpandtweet

Client-sideexecu5onàNoneedtocontacttheserverallthe5me!


Studies of JavaScript Web Applica5ons Performanceandparallelism:JSMeter[Ratanaworabhan-2010],[Richards-2009],[Fortuna-2011]

Reliability

?

SecurityandPrivacy:[Yue-2009],Gatekeeper[Guarnieri-2009],[Jang-2010]

performance

security

reliability

Goal:StudyandimprovethereliabilityofJavaScriptwebapplica5ons


Does Reliability MaKer ? •  SnapshotofiFeng.com:LeadingmediawebsiteinChina

anerroroccurredwhenprocessingthisdirec5ve


Analyzing JS Code: Challenges

15

JShaslooseseman>cs

Lackofstandardprogrammingstyle&

JSframeworks

Frequentcross-languageinterac>ons


Talk Outline

•  Mo>va>onandGoals

•  EmpiricalStudyofreliability

•  ReliabilityImprovements

•  ProgramUnderstanding

•  IDESupport

•  OtherWorkandFutureDirec>ons


Our Prior Work

•  EmpiricalstudybasedonConsoleErrorMessages:Alexatop100• Popularwebapplica5onsexperiencefourdis5nctJavaScripterrormessagesonaverageacrosstheirweb-pages[Ocariza-ISSRE’11]

• Manyerrorswerenon-determinis>cordependentoneventorder-hardtodeterminetherootcauseandimpactoftheseerrors

024681012141618

Google

YouTub

eYaho

oBa

idu

QQ

MSN

Am

azon

Sina

WordP

ress

Ebay

Linked

In

Bing

MicrosoM

Yand

ex

163

mail.ru

PayPal

FC2

Flickr

IMDb

Ap

ple

Livedo

or

BBC

Sohu

go.com

Soso

Youku

AOL

CNN

Med

iaFire

ESPN

MySpace

MegaU

pload

Mozilla

4shared

Adob

eAb

out

LiveJournal

Tumblr

GoDa

ddy

CNET

YieldM

anager

Sogou

Zedo

Ife

ng

PirateBay

ImageShack

Weather

NYTimes

Nen

lix

TotalDis5nctErrors

17

Empirical Study: Research Ques5ons • Whaterrors/mistakescauseJavaScriptfaults?

• WhatimpactdoJavaScriptfaultshave?• Howlongdoesittaketofixtheseerrors?

BugReportStudyof19popularandopensourceJavaScriptapplica5ons&libraries

-Overaspanof10years-Over500bugreports


Bug Report Study: Methodology

19

Collected502bugreportsfrom19webapplica>ons

Qualita>velyanalyzedandclassifiedbugreportsmanually

Aggregateddataforfurtheranalysis


Bug Report Study: Objects

EightJavaScriptWebApplica5ons

FourJavaScriptLibraries


Bug Report Study: Categories

IncorrectMethodParameterFault:UnexpectedorinvalidvaluepassedtoJSmethodorassignedtoJSpropertyDOM-RelatedFault:ThemethodisaDOMAPImethod-Accountforaroundtwo-thirdsofJavaScriptFaults


Bug Report Study: DOM

22

html

body head

scriptdiv p

Text:“Helloworld”

table

tr p

Wanttoretrievethiselement

22

Bug Report Study: DOM-Related Faults

23

div

id:elem

JavaScriptcode:

DOM:

var x = document.getElementById(“elem”); var x = document.getElementById(“elme”);

InexistentIDWillreturnnull

23

DOM-Related Fault: Example

var elem, retrievedStr = [Retrieved via XHR]; var dotsInStr = retrievedStr.split(".").length; if (dotsInStr == 0) {

var prefix = "id_"; elem = $("#" + prefix + retrievedStr);

} else {

elem = $(retrievedStr); } elem[0].focus()





} else {

elem = $(retrievedStr); } elem[0].focus() Retrievedstring

viaXHR





} else {


Findthenumberofdotsinthestring





} else {


Iftherearenodots,prepend“id_”tothestringandaccessitvia$().Otherwise,leaveitasis,andaccessitvia$().





} else {


Retrievedstringof“editor”wouldgohereeventhoughithasnodots,whichwoulderroneouslycause$()touseselector“editor”,whichdoesn’tmatchanyelements.UNDEFINEDEXCEPTION!





} else {


BUG:TheassignedvalueshouldberetrievedStr.split(“.”).length–1,aslength()alwaysreturnsat

least1.Copyright:KarthikPa*abiraman,2016 29

Bug Report Study: Impact •  ImpactTypes–BasedonBugzilla[ICSE’11]

•  Type1(lowestimpact),Type5(highestimpact)

0

20

40

60

80

100

120

140

Type1 Type2 Type3 Type4 Type5

Num

bero

fBugRep

orts

ImpactType

AllFaults DOM-RelatedFaultsOnly

80%ofhighestimpactfaultsareDOM-related


Bug Report Study: Fix Times •  TriageTime:Timeittooktoassignorcommentonthebug•  FixTime:Timeittooktofixthebugsinceitwastriaged

0

10

20

30

40

50

60

70

80

90

100

TriageTime FixTime

Average#ofDays

AllFaults

DOM-RelatedOnly

Non-DOM-RelatedOnly


Bug Report Study: Browser Specificity

17% 1%1%1%

76%

2% IE

Firefox

Chrome

Safari

Opera

Notbrowser-specific

MostJavaScriptfaultsarenotbrowser-specific


Bug Report Study: Summary

• Bugreportstudyof12applica5ons:JSfaults•  Over300bugreportsanalyzed;onlyfixedbugsconsidered

• DOM-relatedfaultsdominateJavaScriptfaults•  Responsiblefornearlytwo-thirdsofallJavaScriptfaults•  Responsiblefor80%ofhighestimpactfaults•  Take50%longer>metofixfordevelopers•  Majorityarenotspecifictowebbrowserplanorm

• Needrobustsolu5onsforDOM-relatedfaults•  Fixing,Understandingandwri>ngcorrectcode


Web Applica5ons: Exis5ng Techniques • AddgradualtypingtoJavaScript(e.g.,TypeScriptfromMS,DARTfromGoogle,FlowfromFacebook)

•  TypicallyignoretheDOMorprovideonlylimitedsupport

• Usehigher-levelprogrammingidiomsinJavaScript•  MVCFrameworks(e.g.,AngularJS)•  Func>onalReac>veProgramming(e.g.,RxJS)

• Detec5ngerrorsinwebapplica5ons:IgnoreDOM•  Racecondi>ons[Vechev-OOPSLA’13][Livshits-FSE’15]•  TypeCoercionErrors[Pradel–ICSE’15]


Web Applica5ons: Challenge

35

S2

S0

S1

S… SN

S4

S3

S5UserInput/UserAc>on/ServerSide

DOMishighlydynamic!

Web Applica5ons: Our Approach

36

DOM-RelatedFaults[ESEM’13][TSE]

Clema5s/ToChal/Shaand:Programcomprehension

[ICSE’14B][ECOOP’15]

[ICSE’16][TOSEM]

AutoFlox/Vejovis:Localiza>onandRepair[ICST’12]

[ICSE’14A][ICSE’15][STVR]

DOMPle5on/LED:DOMCode

comple>onandsynthesis[ASE’14]

[ASE’15]


Talk Outline





•  IDESupport



Web Applica5ons: Bug Fix PaKerns

ParameterModifica>on

27%

DOMElementValida>on

26%

MethodModifica>on

25%

MajorRefactoring/

Other22%

38

Changeisverysmall


Web Applica5ons: DOM-Related Faults

39

body

divid=“bar1”

divid=“bar2”

divid=“bar3”

1 function generateId(index) { 2 var prefix = "bar"; 3 var id = prefix + index; 4 return id; 5 } 6 7 function retrieveElement(index) { 8 var id = generateId(index); 9 var e = document.getElementById(id); 10 return e; 11 } 12 13 for (var i = 1; i <= 4; i++) { 14 var elem = retrieveElement(i); 15 elem.innerHTML = "Item #" + i; 16 }

Retrievetheelementwithindexi

Updateretrievedelement

Addthe“bar”prefixtotheID


Web Applica5ons: DOM-Related Faults

40

body

divid=“bar1”

divid=“bar2”

divid=“bar3”


Thisshouldbe“<“,not“<=“

Evaluatesto“bar4”in4thitera>on

NULLEXCEPTION!Copyright:KarthikPa*abiraman,2016

AutoFlox: Fault Localiza5on

41


FAILUREPOINT

ERRORPOINTTOREPAIR

OurGoal


AutoFlox: Fault Localiza5on

42


FAILUREPOINT

DOMMETHODCALL

ERRORPOINTTOREPAIR

AutoFLox[ICST’12][STVR’16]

Vejovis[ICSE’14]


Vejovis: Fault Repair

43


Evaluatesto“bar4”

“bar”

“4”

“4”comesfromiteratori

“bar1”,“bar2”,and“bar3”arealreadyaccountedforinpreviousitera>ons

“OFF-BY-ONE,SOREMOVELASTITERATIONOFFORLOOP”


AutoFlox and Vejovis: Results-1

44

Real-worldJSfaults

RESULT1:AutoFLoxsuccessfullylocalized100%ofthereal-worldfaults

RESULT2:Vejovissuccessfullyfoundrepairfor91%

ofbugs

20bugsanalyzedbyAutoFLox,and22bugsbyVejovis(fromapplica>onswith100LOCto11000LOC)


AutoFlox and Vejovis: Results-2

45

Performance

RESULT:Bothtoolsexecute1min.onaverage(worstcase90seconds)

6webapplica>onsforAutoFLoxand11webapplica>onsforVejovis,

rangingfrom100LOCto11000LOC


Automa5c Fault Detec5on: Background

46

MODEL VIEW CONTROLLER

MODELVARIABLES CONTROLLERFUNCTIONS

DEFINESUSES DEFINESUSES USES

MVCFrameworksDOMabstractedoutàDOM-relatedfaultsnotproblema5c!



47




Problem:Inconsistenciesbetweeniden>fiersandtypesinmodel,view,andcontroller



48




MVCFrameworks

OurSolu5on:Approachtoautoma>callydetectiden>fierandtypeinconsistencies


Aurebesh: AngularJS

49

300%increaseinAngularJSusageintheyear2015

ThemostpopularJSMVCframeworkinGitHub,StackOverfow,

andevenYouTube!

Automa5c Fault Detec5on: Methodology

50

MODELVIEW

CONTROLLER

MODEL VIEW

CONTROLLERVIEW

CONTROLLER

String:aBoolean:bObject:c

String:d

MODEL

Boolean:e

String:aBoolean:eString:foo()

Object:cNumber:foo()

Boolean:eString:bar()

String:bar()

String:fun()

String:dString:fun()


Automa5c Fault Detec5on: Methodology

51

MODEL

String:aBoolean:bObject:c

MODEL

String:d

MODEL

Boolean:e

VIEW

String:aBoolean:eString:foo()

CONTROLLER

Object:cNumber:foo()

VIEW

Boolean:eString:bar()

CONTROLLER

String:bar()

VIEW

String:dString:fun()

CONTROLLER

String:fun()

“e”isnotdefinedinmodel!Inconsistenttypes!


Automa5c Fault Detec5on: Results

52

TOOL:Aurebesh

Faultinjec>onexperiment

Aurebeshis96.1%accurate,withonlyone

falseposi>ve

Real-worldwebapplica>ons

Aurebeshdetected15previouslyundetectedbugs(5wereacknowledgedby

developers)


h*p://www.ece.ubc.ca/~frolino/projects/aurebesh

Aurebesh: Screenshot

53

INDEX.HTML(VIEW)

APP.JS(MODEL)

“beets”

“beers”

“beerForms”

“beets”isundefined!


Drawback of Aurebesh

54

Analyzed90MVCbugreports

Over40inconsistencycategories!

Aurebeshonlysupports4inconsistencycategories!


A Generalized Inconsistency Detector

55

Step1:InferImplicitConsistencyRules

Func>onCall

MemberExp Params

foo close String

Fromtargetapplica>on:

Func>onCall

MemberExp Params

bar close String



56

Step1:InferImplicitConsistencyRules

Func>onCall

MemberExp Params

foo close String


Func>onCall

MemberExp Params

bar close String

Fromsampleapplica>ons:

Func>onCall

MemberExp Params

foo close String

Func>onCall

MemberExp Params

foo close String



57

Step2:DetectRuleViola>ons

Func>onCall

MemberExp Params

foo close String


Func>onCall

MemberExp Params

bar close String

Fromsampleapplica>ons:

Func>onCall

MemberExp Params

foo close String

Func>onCall

MemberExp Params

foo close String


HoloCron: Generalized Inconsistency Detector

58

LearnsinconsistencypanernsforanyMVC-likeframework

Analyzed90MVCbugreports

35%ofinconsistenciesarecross-language


Generalized Detector: Results

59

Real-worldwebapplica>ons

Holocrondetected18previously

undetectedbugsfromMVCapplica>ons

15inconsistencycategories

5crosslanguageinconsistencies


Generalized Detector: Results

60

TOOL:Holocron

1outofevery2reportsareeitherrealbugsorcodesmells

CodeSmell=notabug,butmakescodemoredifficulttomaintain

Example:Givingthesamenameto

unrelatedvariables


Talk Outline





•  IDESupport



Understanding JavaScript Apps

62

Event-based Interactions

JS and DOMInteractions

Client/Server Interactions

Asynchronous Server Interactions

Clematis [ICSE’14]

Tochal [ECOOP’15]

Sahand [ICSE’16]


Clema5s: Mo5va5on

• Goal: Understand and visualize dependencies between JavaScript events and the DOM

• Challenge: Difficult to understand the dynamic behavior and the control flow of events

•  Event propagation due to the DOM •  Asynchronous events (e.g., timeouts) •  DOM state changes due to events

• Approach: Dynamically capture execution of JavaScript applications and convert it to a model


Clema5s: Approach

64

JavaScript

Transformation

Trace

Collection

Model

Visualization

Behavioral Model

Creation


Clema5s: Model Episodes

Timeout ID: 1XHR ID: 1

DOM Event(Click)Target: SubmitBtn

Source

Result

Trace

body

fieldset

text input

button div

informServer Anonymous

e email

isEmailValid

SubmissionHandler

email

XHR Event(Response)XHR ID: 1

Source Trace

body

fieldset

text input

button div

data

Anonymous Timing Event(TO Callback)TO ID: 1

Source Trace

body

fieldset

text input

button div

clearMsg

Result Result


Clema5s: Model Links

Timeout ID: 1XHR ID: 1

DOM Event(Click)Target: SubmitBtn

Source

Result

Trace

body

fieldset

text input

button div

informServer Anonymous

e email

isEmailValid

SubmissionHandler

email

XHR Event(Response)XHR ID: 1

Source Trace

body

fieldset

text input

button div

data

Anonymous Timing Event(TO Callback)TO ID: 1

Source Trace

body

fieldset

text input

button div

clearMsg

Result Result

emporalausal


Clema5s: Visualiza5on


Zoom Level 1

CLEMATIS: VISUALIZATION


Visualiza5on: Zoom Level 1

69

Zoom Level 2 Copyright:KarthikPa*abiraman,2016

Clema5s: User Experiment

• Par>cipants•  20soMwaredevelopersfromalargesoMwarecompanyinVancouver(theywereallwellversedinwebdevelopment)

•  Experimentalgroup:Clema>s•  Controlgroup:Chrome,Firefox,Firebug(anytoolofchoice)

• Procedure•  Tasks:controlflow,featureloca>on,DOMmuta>ons,…

• Datacollec>on:Taskcomple>ondura>on&accuracy


Task Improvement

T1 (39%é)

T2 (48%é)

T3 (68%é)

T4 (32%é)

Duration

Task Improvement

T1 (67%é)

T2 (41%é)

T3 (20%é)

T4 (68%é)

Accuracy

Task Descrip5on

T1 Followingcontrolflowinpresenceofasynchronousevents

T2 FindingDOMmuta>onscausedbyaDOMevent

T3 Loca>ngtheimplementa>onofamalfunc>oningfeature

T4 Detec>ngcontrolflowinpresenceofeventpropaga>on

Clema5s: Results


•  Freelyavailable:•  https://github.com/saltlab/clematis

• AbilitytovisualizeJavaScripteventsandDOMstates•  Nochangestoserversideorclientsidecode•  Causaldependenciesbetweeneventsincl.AJAXrequests•  DOMstatechangesandeventpropaga>onintheDOM

•  Significantlyimprovedtaskdura>onandaccuracycomparedtootherstate-of-the-arttools

Clema5s: Summary


ToChal: Change Impact Analysis (CIA)

• Software must continually change to adapt to the changing environment.

•  Goal: identifying parts of the program that are potentially affected by a change.

73

•  Hybrid of static and dynamic analyses


ToChal:ImpactthroughtheDOM

body

fieldset

div div

1 function checkPrice() {2 . . .3 var cad-price = $(‘#price_ca’).innerText();4 . . .5 }

6 function calculateTax() {7 $(‘.price’).each(function(index) {8 $(this).text(addTaxToPrice( $(this).text());9 });10 }

11 $(‘#price_ca’).bind(‘click’, checkPrice);id=price_caclass=price

43

2


ToChal: Approach

•  Sta>ccontrol-flowanddata-flowanalysis

• Analyzingthedynamicfeatures•  Dynamiccallgraph•  DOMinterac>ons•  Event-basedimpactpropaga>on•  XHRrela>ons

• Hybridmodelforimpactanalysis


Tochal: Tool Implementa5on

•  Tochal:opensource•  h*ps://github.com/saltlab/tochal

• Proxy(Java,JavaScript)•  Esprima,Estraverse,Escodegen,Muta>onSummary,WALA

• Client-side(GoogleChromeextension)•  ChromeDevTools


ToChal: User Experiment

• Ques>on:DoesTochalhelpdevelopersinprac>cetoperformchangeimpactanalysis?

• Design:•  12par>cipantsfromindustry•  4tasks:detec>ngandanalyzingchangeimpact•  Measured:taskcomple>ondura>onandaccuracy


ToChal: User Experiment Results

Task Improvement

T1 78%éé

Dura>on

Task Improvement

Total 223%éé

Accuracy

Task Descrip5on

T1 Findingthepoten>alimpactofaDOMelement

T2 Findingthepoten>alimpactofaJavaScriptfunc>on

T3 FindingaconflictaMermakinganewchange(noranking)

T4 FindingabuginJavaScriptcode


Understanding JavaScript Apps

79

Event-based Interactions

JS and DOMInteractions

Client/Server Interactions

Asynchronous Server Interactions

Clematis [ICSE’14]

Tochal [ECOOP’15]

Sahand [ICSE’16]


Node.js Challenges

• Asynchronous execution• Network communication• Scalability

•  Example: Callback hell

80

Little pyramidof doom

fs.readdir(source, function(err, files) { files.forEach(function(filename, fileIndex)

gm(source + filename).size(function(err, values) { widths.forEach(function(width, widthIndex) {

this.resize(w, h).write(newName, function(err) { })

})

})

})

}) // example from callbackhell.com Copyright:KarthikPa*abiraman,2016

Our Approach: Sahand

81

1. Instrument automatically

2. Trace full-stack execution

3. Infer a behavioural model

4. Visualize the modelInaAct Act

Act

ActSch

Application


Behavioral Model

82

NodesLifelines of function executions

Act

ActSch Ina Act

Act

(A)Synchronous client/server events

Links Time, Type, Direction

foo()

bar()

baz()

event

Act Ina Act


Real Behavioural Models Are Complex


Visualization

https://github.com/saltlab/sahand

84

Client-Side Analysis

Server-Side Analysis


Evaluation

Does using Sahand improve developers’ performance in program comprehension tasks?


Controlled Experiment

• Sahand’s effect on developers’ performance•  12 Participants• Object: full-stack JavaScript application


Results Highlight

87

●T1−Exp

T1−C

trl

T2−Exp

T2−C

trl

T3−Exp

T3−C

trl

Tot−E

xp

Tot−C

trl

0

20

40

60

80

100

Accuracy (%)

Using Sahand

3 times more accuracyIn the same time

Acc

urac

y

Tasks

Control group

Experimental (Sahand) group


Talk Outline





•  IDESupport



Domple5on: Mo5va5on

• Providecode-comple>onforDOM-JavaScriptinterac>ons

89

html

body head

scriptdiv p

Text:“Hello”

table

tr p

Wanttoretrieveelementwithid“elem”

var x = document.getElementById(“e lem”);Copyright:KarthikPa*abiraman,2016

Domple5on: Challenge

90

S2

S0

S1

S… SN

S4

S3

S5UserInput/UserAc>on/ServerSide

Poten>allyinfinitenumberofDOMstates!


Domple5on: Intui5on

DOMstatesexhibitpa*erns

91

HTML

BODY

DIVid=“container”

INPUTclass=“val1”value=“10”


BUTTONid=“add”

value=“Add”

SPANclass=“result”html=“Result:”

HTML

BODY

DIVid=“container”



BUTTONid=“add”

value=“Add”

SPANclass=“result”html=“Result:”

Pid=“true”html=“30”

S0 S1


Domple5on: Approach

92

DOMAnalysis(Phase1)

CodeAnalysis(Phase2)

Sugges>onGenera>on(Phase3)

WebappURL

JavaScriptCode

Sugges>ons

Executedonlyonceatbeginning

Executedevery>meCopyright:KarthikPa*abiraman,2016

Domple5on: Sugges5on Genera5on

93

DOMAnalysisOutput CodeAnalysisOutput

Sugges>onsCopyright:KarthikPa*abiraman,2016

Domple5on: Screenshot

94

ImplementedintheBracketsIDE


Domple5on: Evalua5on

• RQ1:DoDOMelementlocatorsforwebapplica>onsconverge,andifso,whatistheconvergencerate?

• RQ2:Howaccuratearethecode-comple>onsugges>onsprovidedbyDomple>on?

• RQ3:Howeffec>veitDomple>oninhelpingthewebdeveloperswithcodecomple>ontasks?


Domple5on: Convergence (RQ1)

��

��

��

� � � � � � � � � ��

��

��

��


Domple5on: Convergence (RQ1)

��

��

��

��

��

��

��

� ��

��

��

��

��

��

��

��


Domple5on: Accuracy (RQ2)

98

��

��

��

��

��

��

��

� ��

!��"�� #��$� %�� &��'��

�

��

�

��


Domple5on: Accuracy (RQ2)

99

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

�!

!�

"!

��


Domple5on: User Study (RQ3)

•  9Par>cipants•  4TasksGroup Descrip5on No.ofPar5cipants AverageTime Precision Recall

GroupA UsingDompleiton

5 1m43s 90.83% 97.5%

GroupB WithoutDomple>on

4 4m28s 76.25% 47.5%


LED: Mo5va5on

• DOM-JSinterac>onsisamajorsourceoferrors[ESEM’13]

• PerformedthroughCSSselectorsinJScode•  22%oftheCSSselectorsareusedtoselectmul5pleDOMelements•  35%oftheCSSselectorsareacombina5onofmul>pleatomicCSSselectors

• Goal:AutomatesynthesisofCSSselectors


LED: Main Idea

• AskdevelopertoprovideDOMelementsasposi>veandnega>veexamplesforselector

• Analyzedis>nguishingproper>esofelementsandgenerateconstraintsfortheproper>es

•  LeverageSATsolverstosolveconstraints•  Rankselectorsbasedon“goodness”criteria


LED: Input


LED: Output


LED: Evalua5on (Coverage)

Number of stylesheets

Nu

mb

er o

f se

lect

ors

per

sty

lesh

eet

Total Selectors Supported Selectors

0 50 100 150 200 250 300 350 400 4500.1

1

10

100

1k

10k

86%oftheCSSselectorsusedintop500webapplica>onsaresupportedbyLED

AnalyzedCSSselectorsusedinAlexa’stop500webapplica>ons


LED: Evalua5on (Accuracy)

•  InterceptedDOMAPIcalls• Wordpress,Gallery3andPhormer

• AnalyzedDOMelements

•  SynthesizedCSSselectors

•  98%Recalland92%Precision


LED: Evalua5on (Performance) • Average>me:0.22Seconds

• Max>me:0.5Seconds

•  Toolandvideoavailableat

hnp://ece.ubc.ca/~kbajaj/led.html


Talk Outline





•  IDESupport



Open Challenges

• Whataretechniquestomi>gateotherkindsof(non-DOM-related)JavaScriptfaults?

• Howcanwehelpprogrammerswriteerror-freeJavaScriptcodethroughIDEsupport?

• Whatkindsofframeworks/variantsofJavaScriptareouttheretomi>gatefaults?


Non-DOM-related Faults

• WhataboutfaultsthatdonotinvolveDOM-JSinterac>ons?•  Currentlyaccountforabout35%offaults,butmayincreaseaswemi>gateDOM-relatedfaults

•  Threerelatedefforts•  Non-DOMrelatedAPIfaults[FSE’14]•  Type-relatedfaults(Typedevil)[Berkeley’14]•  Racecondi>ons:Webracer[PLDI’14]


IDE Support for JavaScript

• Wri>ngJavaScriptischallenging•  VerypoorIDEsupportforJavaScript•  Fewtoolstounderstandwebapplica>ons

• Codecomple>onanddebuggingtools•  Approximatecallgraphconstruc>on[ICSE’13]•  Sta>cenforcementofpolicies[Livshits’09]•  RecordandReplay:Mugshot[NSDI’10]


TypeScript and JavaScript Frameworks

•  Type/formalismanalysisforJavaScript•  VerifiedJavaScriptseman>cs[Guha-ECOOP’10]•  GradualTyping[Swamy-POPL’14]•  Sta>canalysis[Moller-FSE’11]

•  Frameworksforconstruc>onJavaScriptApps•  Flapajax[Guha-OOPSLA’09]•  Arrows[Hicks-DLS’09]


Future Work

• Understandinglarge-scaleapplica5onsusingtracecompression

•  Currenttracesaretoolargeforcomprehension•  Useofalgorithmsinspiredbygenesequencematching

•  JavaScriptintheIoTSpace•  Understandingthesourcesoferrorsandhowtheyaffectthesystem

•  Targetedtechniquesforimprovingreliabilitysubjecttoresourceconstraints


Conclusions

•  JavaScriptisoneofthemostprominentlanguagestoday•  FiveyearsofresearchintostudyingandunderstandingJavaScriptapplica>ons•  Performedempiricalstudiestoiden>fysourcesofJavaScriptbugs•  BuilttoolsforimprovingthereliabilityandprogrammabilityofJavaScript•  Evaluatedtoolsusingreal-worldapplica>onsandcasestudies

• OpenChallenges•  Non-DOMrelatedfaults•  ScalableIDEsupport•  JavaScriptonIoT


FrolinOcarizaSabaAlimadadi Kar>kBajaj SheldonSequira

KarthikPa*abiraman AliMesbah

Documents

Tutorial 5: Modern Web Applica5ons’ Reliability Engineeringblogs.ubc.ca/karthik/files/2016/10/Issre-tutorial-2016.pdf · Web Applica5ons: Challenge 35 S 2 S 0 S 1 S … S N S 4