Principles of Software Construction: Objects, Design, and...

115-214

SchoolofComputerScience

PrinciplesofSoftwareConstruction:Objects,Design,andConcurrency(Part2:Designing(Sub-)Systems)

MoreAnalysisforFunctionalCorrectness

ChristianKästner BogdanVasilescu

215-214

Administrativa

• Reviewsessionsstartedyesterday– Therearestillmanyslotsforreviewsessions– Onceyousignedup,pleaselookuptheplaceofthereviewsessioninthecoursecalendarhttps://www.cs.cmu.edu/~ckaestne/15214/s2017/

– Ifthereareofficehoursandreviewsessionsinparallel,thesignupisforthereviewsession

• HomeworkisdueonThursday.

315-214

LearningGoals

• Basicunderstandingofrefactoring• Integratingunittestingintothedesignprocess• Understandingandapplyingcoveragemetricstoapproximatetestsuitequality;awarenessofthelimitations

• Basicunderstandingofthemechanismsandlimitationsofstaticanalysistools

• Characterizingassurancetechniquesintermsofsoundnessandcompleteness

415-214

Correctness?

515-214

SoftwareErrors• Functionalerrors• Performanceerrors• Deadlock• Raceconditions• Boundaryerrors• Bufferoverflow• Integrationerrors• Usabilityerrors• Robustnesserrors• Loaderrors

• Designdefects• Versioningand

configurationerrors• Hardwareerrors• Statemanagementerrors• Metadataerrors• Error-handlingerrors• Userinterfaceerrors• APIusageerrors• …

615-214

SoftwareErrors• Functionalerrors• Performanceerrors• Deadlock• Raceconditions• Boundaryerrors• Bufferoverflow• Integrationerrors• Usabilityerrors• Robustnesserrors• Loaderrors

• Designdefects• Versioningand

configurationerrors• Hardwareerrors• Statemanagementerrors• Metadataerrors• Error-handlingerrors• Userinterfaceerrors• APIusageerrors• …

715-214

CODESMELLS

815-214

BadSmells->DesignDefects

• BadSmellsindicatethatyourcodeisripeforrefactoring

• Refactoringisabouthowtochangecodebyapplyingrefactorings

• Badsmellsareaboutwhentomodifyit

915-214

BadSmells:Classification

• Topcrime:codeduplication• Class/methodorganization

– Largeclass,LongMethod,LongParameterList,LazyClass,DataClass,...

• Lackofloosecouplingorcohesion– InappropriateIntimacy,FeatureEnvy,DataClumps,...

• Toomuchortoolittledelegation– MessageChains,MiddleMan,...

• NonObject-Orientedcontrolordatastructures– SwitchStatements,PrimitiveObsession,...

• Other:Comments

1015-214

Codeduplication(1)

code

code

code

code

Class

Method 1

Method 2

Method 3

code

Class

Method 1

Method 2

Method X

MethodX();

Method 3MethodX();

MethodX();MethodX();

• Extract method

• Rename method

1115-214

Codeduplication(2)

code

Subclass A

Method codeMethod

Subclass BClass

Sameexpressionintwosiblingclasses:

• Samecode:Extractmethod+Pullupfield

• Similarcode:Extractmethod+FormTemplateMethod

• Differentalgorithm:Substitutealgorithm

1215-214

Codeduplication(3)

code

ClassA

MethodA codeMethodB

ClassB

1315-214

Codeduplication(3)ClassA

MethodA MethodB

ClassB

Sameexpressionintwounrelatedclasses:

• Extractclass

• Ifthemethodreallybelongsinoneofthetwoclasses,keepitthereandinvokeitfromtheotherclass

code

ClassX

MethodX

ClassX.MethodX(); ClassX.MethodX();

1415-214

Longmethod//700LOCpublic boolean foo() {

try {synchronized () {

if () {} else {}for () {

if () {if () {

if () {if ()?{

if () {for () {}

}}

} else {if () {

for () {if () {} else {}if () {} else {

if () {}

}if () {

if () {if () {

for () {}

}}

} else {}

}} else {

Source: http://thedailywtf.com/Articles/Coding-Like-the-Tour-de-France.aspx

• Rememberthis?

1515-214

Refactoringalongmethodvoid printOwing() {

Enumeration e = _orders.elements();double outstanding = 0.0;// Print banner System.out.println(“******************“);System.out.println(“***** Customer *****“); System.out.println(“******************“);// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

1615-214

Refactoringalongmethodvoid printOwing() {

Enumeration e = _orders.elements();double outstanding = 0.0;// Print banner System.out.println(“******************“);System.out.println(“***** Customer *****“); System.out.println(“******************“);// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

1715-214

Refactoringalongmethodvoid printOwing() {

Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

void printBanner(){System.out.println(“******************“);System.out.println(“***** Customer *****“); System.out.println(“******************“);

}

Extractmethod

CompileandtesttoseewhetherI'vebrokenanything

1815-214

Refactoringalongmethodvoid printOwing() {

Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

} void printBanner(){…}

1915-214

Refactoringalongmethodvoid printOwing() {

Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} printDetails(outstanding);

}void printBanner(){…}void printDetails(outstanding){

System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

Extractmethodusinglocalvariables

CompileandtesttoseewhetherI'vebrokenanything

2015-214

Refactoringalongmethodvoid printOwing() {

Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} printDetails(outstanding);

}void printBanner(){…}void printDetails(outstanding){

System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

2115-214

Refactoringalongmethodvoid printOwing() {

Enumeration e = _orders.elements();double outstanding = getOutstanding(); printBanner(); printDetails(outstanding);

}void printBanner(){…}void printDetails(outstanding){…}

double getOutstanding() { Enumeration e = _orders.elements();double result = 0.0; While (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); result += each.getAmount();

} return result;

}

Extractmethodreassigningalocalvariable

CompileandtesttoseewhetherI'vebrokenanything

2215-214

ManyMoreBadSmells

• Topcrime:codeduplication• Class/methodorganization

– Largeclass,LongMethod,LongParameterList,LazyClass,DataClass,...

• Lackofloosecouplingorcohesion– InappropriateIntimacy,FeatureEnvy,DataClumps,...

• Toomuchortoolittledelegation– MessageChains,MiddleMan,...

• NonObject-Orientedcontrolordatastructures– SwitchStatements,PrimitiveObsession,...

• Other:Comments

2315-214

BACKTOFUNCTIONALCORRECTNESS

2415-214

Reminder:FunctionalCorrectness

• Thecompilerensuresthatthetypesarecorrect(typechecking)– Prevents“MethodNotFound”and“CannotaddBooleantoInt”errorsatruntime

• Staticanalysistools(e.g.,FindBugs)recognizecertaincommonproblems– WarnsonpossibleNullPointerExceptions orforgettingtoclosefiles

• Howtoensurefunctionalcorrectnessofcontractsbeyond?

2515-214

FormalVerification

• Provingthecorrectnessofanimplementationwithrespecttoaformalspecification,usingformalmethodsofmathematics.

• Formallyprovethatallpossibleexecutionsofanimplementationfulfillthespecification

• Manualeffort;partialautomation;notautomaticallydecidable

2615-214

Testing

• Executingtheprogramwithselectedinputsinacontrolledenvironment(dynamicanalysis)

• Goals:– Revealbugs(maingoal)– Assessquality(hardtoquantify)– Clarifythespecification,documentation– Verifycontracts

"Testing shows the presence, not the absence of bugs

Edsger W. Dijkstra 1969

2715-214

TestingDecisions• Whotests?

– Developers– OtherDevelopers– SeparateQualityAssuranceTeam– Customers

• Whentotest?– Beforedevelopment– Duringdevelopment– Aftermilestones– Beforeshipping

• Whentostoptesting?

(More in 15-313)

2815-214

TEST-DRIVENDEVELOPMENT

2915-214

Test Driven Development

(CC BY-SA 3.0) Excirial

• Testsfirst!• Popular

agiletechnique• Writetestsas

specificationsbeforecode• Neverwritecodewithout

afailingtest

3015-214

Mobprogramming

• WriteaprogramusingTDDthattakesasinputacharacterA..Zandprintsadiamondpattern:

A AB BA

AB B

C C CB BA

Input:A Input:B Input:C

3115-214

Test Driven Development• Testsfirst!• Popular

agiletechnique• Writetestsas

specificationsbeforecode• Neverwritecodewithout

afailingtest• Claims:

• Designapproachtowardtestabledesign• Thinkaboutinterfacesfirst• Avoidwritingunneededcode• Higherproductquality(e.g.bettercode,lessdefects)• Highertestsuitequality• Higheroverallproductivity

(CC BY-SA 3.0) Excirial

3215-214

TESTCOVERAGE

3315-214

Howmuchtesting?

• Yougenerallycannottestallinputs– toomany,usuallyinfinite

• Butwhenitworks,exhaustivetestingisbest!

• Whentostoptesting?– inpractice,whenyourunoutofmoney

3415-214

Whatmakesagoodtestsuite?

• Provideshighconfidencethatcodeiscorrect• Short,clear,andnon-repetitious

– Moredifficultfortestsuitesthanregularcode– Realistically,testsuiteswilllookworse

• Canbefuntowriteifapproachedinthisspirit

3515-214

• Alsoknowasfuzztesting,torturetesting• Try“random”inputs,asmanyasyoucan

– Chooseinputstotickleinterestingcases– Knowledgeofimplementationhelpshere

• Seedrandomnumbergeneratorsotestsrepeatable• Successfulinsomedomains(parsers,networkissues,…)

– But,manytestsexecutesimilarpaths– But,oftenfindsonlysuperficialerrors

Blackbox:RandomInputsNextbestthingtoexhaustivetesting

3615-214

Blackbox testing

Blackbox:CoveringSpecifications

• Lookingatspecifications,notcode:

• Testrepresentativecase• Testboundarycondition• Testexceptionconditions• (Testinvalidcase)

3715-214

TextualSpecificationpublic int read(byte[] b, int off, int len) throws IOException

§ Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read. The number of bytes actually read is returned as an integer. This method blocks until input data is available, end of file is detected, or an exception is thrown.

§ If len is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at end of file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

§ The first byte read is stored into element b[off], the next one into b[off+1], and so on. The number of bytes read is, at most, equal to len. Let k be the number of bytes actually read; these bytes will be stored in elements b[off] through b[off+k-1], leaving elements b[off+k] through b[off+len-1] unaffected.

§ In every case, elements b[0] through b[off] and elements b[off+len] through b[b.length-1] are unaffected.

• Throws:§ IOException - If the first byte cannot be read for any reason other than end of file, or if

the input stream has been closed, or if some other I/O error occurs.§ NullPointerException - If b is null.§ IndexOutOfBoundsException - If off is negative, len is negative, or len is greater

than b.length - off

3815-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

3915-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

Will this statement get executed in a test?

Does it return the correct result?

4015-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

Could this array index be out of bounds?

Will this statement get executed in a test?

Does it return the correct result?

4115-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

Could this array index be out of bounds?

Does this return statement ever get reached?

Will this statement get executed in a test?

Does it return the correct result?

4215-214

Codecoveragemetrics

• Methodcoverage– coarse• Branchcoverage– fine• Pathcoverage– toofine

– Costishigh,valueislow– (Relatedtocyclomatic complexity)

4315-214

MethodCoverage

• Tryingtoexecuteeachmethodaspartofatleastonetest

• Doesthisguaranteecorrectness?

4415-214

StatementCoverage• Tryingtotestallpartsoftheimplementation• Executeeverystatementinatleastonetest

• Doesthisguaranteecorrectness?

4515-214

StructureofCodeFragmenttoTest

Flow chart diagram forjunit.samples.money.Money.equals

4615-214

StatementCoverage• Statementcoverage

– Whatportionofprogramstatements(nodes)aretouchedbytestcases

• Advantages– Testsuitesizelinearinsizeofcode– Coverageeasilyassessed

• Issues– Deadcodeisnotreached– Mayrequiresomesophisticationto

selectinputsets– Fault-toleranterror-handlingcode

maybedifficultto“touch”– Metric:Couldcreateincentiveto

removeerrorhandlers!

4715-214

BranchCoverage• Branchcoverage

– Whatportionofconditionbranchesarecoveredbytestcases?

– Or:Whatportionofrelationalexpressionsandvaluesarecoveredbytestcases?

• Conditiontesting(Tai)

– Multicondition coverage– allbooleancombinationsoftestsarecovered

• Advantages– Testsuitesizeandcontentderived

fromstructureofbooleanexpressions– Coverageeasilyassessed

• Issues– Deadcodeisnotreached– Fault-toleranterror-handlingcode

maybedifficultto“touch”

4815-214

PathCoverage• Pathcoverage

– Whatportionofallpossiblepathsthroughtheprogramarecoveredbytests?

– Looptesting:Considerrepresentativeandedgecases:

• Zero,one,twoiterations• Ifthereisaboundn:n-1,n,n+1iterations• Nestedloops/conditionalsfrominsideout

• Advantages– Bettercoverageoflogicalflows

• Disadvantages– Infinitenumberofpaths– Notallpathsarepossible,ornecessary

• Whatarethesignificantpaths?– Combinatorialexplosionincasesunless

carefulchoicesaremade• E.g.,sequenceofniftestscanyield

upto2^npossiblepaths– Assumptionthatprogramstructureisbasically

sound

4915-214

TestCoverageTooling

• Coverageassessmenttools– Trackexecutionofcodebytestcases

• Countvisitstostatements– Developreportswithrespecttospecificcoveragecriteria

– Instructioncoverage,linecoverage,branchcoverage

• Example:Cobertura andEclEmma forJUnit tests

5015-214

5115-214

Checkyourunderstanding

• Writetestcasestoachieve100%linecoveragebutnot100%branchcoverage

void foo(int a, int b) {if (a == b)

a = a * 2;if (a + b > 10)

return a - b;return a + b;

}

5215-214

Checkyourunderstanding

• Writetestcasestoachieve100%linecoverageandalso 100%branchcoverage

void foo(int a, int b) {if (a == b)

a = a * 2;if (a + b > 10)

return a - b;return a + b;

}

5315-214

Checkyourunderstanding

• Writetestcasestoachieve100%linecoverageand100%branchcoverageand 100%pathcoverage

void foo(int a, int b) {if (a == b)

a = a * 2;if (a + b > 10)

return a - b;return a + b;

}

5415-214

Coveragemetrics:usefulbutdangerous

• Cangivefalsesenseofsecurity• Examplesofwhatcoverageanalysiscouldmiss

– Datavalues– Concurrencyissues– raceconditionsetc.– Usabilityproblems– Customerrequirementsissues

• Highbranchcoverageisnot sufficient

5515-214

Testsuites– idealvs.real

• Idealtestsuites– Uncoverallerrorsincode– Test“non-functional”attributessuchasperformanceandsecurity

– Minimumsizeandcomplexity• RealtestSuites

– Uncoversomeportionoferrorsincode– Haveerrorsoftheirown– Arenonethelesspriceless

5615-214

STATICANALYSIS

5715-214

StupidBugs

public class CartesianPoint {private int x, y;int getX() { return this.x; }int getY() { return this.y; }public boolean equals(CartesianPoint that) {

return (this.getX()==that.getX()) && (this.getY() == that.getY());

}}

5815-214

Fin

dB

ug

s

5915-214

Stupid Subtle Bugspublic class Object {

public boolean equals(Object other) { … }

// other methods…}

public class CartesianPoint extends Object {private int x, y;int getX() { return this.x; }int getY() { return this.y; }public boolean equals(CartesianPoint that) {

return (this.getX()==that.getX()) && (this.getY() == that.getY());

}}

classes with no explicit superclass implicitly extendObject

can’t change argument type when overriding

This defines a different equalsmethod, rather than overriding Object.equals()

6015-214

Fixing the Bug

public class CartesianPoint {private int x, y;int getX() { return this.x; }int getY() { return this.y; }

@Overridepublic boolean equals(Object o) {

if (!(o instanceof CartesianPoint)return false;

CartesianPoint that = (CartesianPoint) o;

return (this.getX()==that.getX()) && (this.getY() == that.getY());

}}

Declare our intent to override;Compiler checks that we did it

Use the same argument type as the method we are overriding

Check if the argument is a CartesianPoint.Correctly returns false if o is null

Create a variable of the right type, initializing it with a cast

6115-214

Fin

dB

ug

s

6215-214

Ch

eckS

tyle

6315-214

StaticAnalysis• Analyzingcodewithoutexecutingit(automatedinspection)• Looksforbugpatterns• Attemptstoformallyverifyspecificaspects• Pointouttypicalbugsorstyleviolations

– NullPointerExceptions– IncorrectAPIuse– Forgettingtocloseafile/connection– Concurrencyissues– Andmany,manymore(over250inFindBugs)

• IntegratedintoIDEorbuildprocess• FindBugsandCheckStyleopensource,manycommercial

productsexist

6415-214

ExampleFindBugsBugPatterns• Correctequals()• Useof==• Closingstreams• Illegalcasts• Nullpointerdereference• Infiniteloops• Encapsulationproblems• Inconsistentsynchronization• InefficientStringuse• Deadstoretovariable

6515-214

Bugfinding

6615-214

Canyoufindthebug?

if (listeners == null)

listeners.remove(listener);

JDK1.6.0, b105, sun.awt.x11.XMSelection

6715-214

Wrongbooleanoperator

if (listeners != null)

listeners.remove(listener);

JDK1.6.0, b105, sun.awt.x11.XMSelection

6815-214

Canyoufindthebug?

public String sendMessage (User user, String body, Date time) {

return sendMessage(user, body, null);

}

public String sendMessage (User user, String body, Date time, List attachments) {

String xml = buildXML (body, attachments);

String response = sendMessage(user, xml);

return response;

}

6915-214

Infiniterecursiveloop

public String sendMessage (User user, String body, Date time) {

return sendMessage(user, body, null);

}

public String sendMessage (User user, String body, Date time, List attachments) {

String xml = buildXML (body, attachments);

String response = sendMessage(user, xml);

return response;

}

7015-214

Canyoufindthebug?

String aString = "bob";

b.replace('b', 'p');

if(b.equals("pop")){…}

7115-214

Methodignoresreturnvalue

String aString = "bob";

b = b.replace('b', 'p');

if(b.equals("pop")){…}

7215-214

Whatdoesthisprint?

Integer one = 1;Long addressTypeCode = 1L;

if (addressTypeCode.equals(one)) {System.out.println("equals");

} else {System.out.println("not equals");

}

7315-214

Whatdoesthisprint?

Integer one = 1;Long addressTypeCode = 1L;

if (addressTypeCode.equals(one)) {System.out.println("equals");

} else {System.out.println("not equals");

}

7415-214

ASIDE:FINDBUGS NULLPOINTERANALYSIS

Detector foo = null;foo.execute();

7515-214

FindBugs• Workson“.class”filescontainingbytecode

– Recall:Javasourcecodecompiledtobytecode;JVMexecutesbytecode

• Processingusingdifferentdetectors:– Independentofeachother– Maysharesomeresources(e.g.,controlflowgraph,dataflowanalysis)

– GOAL:Lowfalsepositives– Eachdetectorisdrivenbyasetofheuristics

• Output:bugpatterncode,sourcelinenumber,descriptivemessage(severity)

HIGHSEVERE RISK OF

PROGRAM FAILURE

MEDIUMELEVATED RISK OF PROGRAM FAILURE

LOWLOW RISK OF

PROGRAM FAILURE

7615-214

Nullpointerdereferencing• Findingsomenullpointerdereferencesrequiresophisticated

analysis:– Analyzingacrossmethodcalls,modelingcontentsofheapobjects

• Inpracticemanyexamplesofobvious nullpointerdereferences:– Valueswhicharealwaysnull– Valueswhicharenullonsomecontrolpath

• Howtodesignananalysistofindobviousnullpointerdereferences?– Idea:Lookforplaceswherevaluesareusedinasuspiciousway

From: https://www.cs.umd.edu/class/spring2005/cmsc433/lectures/findbugs.pdf

7715-214

SimpleAnalysis

Detector foo = null;foo.execute();

Dereferencing Null

Detector foo = new Detector(…);foo.execute();

Dereferencing NonNull

HIGHSEVERE RISK OF

PROGRAM FAILURE

J

7815-214

Ifonlyitwerethatsimple…

• Infeasiblepaths(falsepositives)

• Isamethod’sparameternull?

boolean b;if (p != null)

b = true;else

b = false;if (b)

p.f();

void foo(Object obj) {int x = obj.hashcode();…

}

7915-214

Dataflowanalysis

• Ateachpointinamethod,keeptrackofdataflowfacts– E.g.,whichlocalvariablesandstacklocationsmightcontainnull

• Symbolicallyexecutethemethod:– Modelinstructions– Modelcontrolflow– Untilafixedpointsolutionisreached

8015-214

Dataflowvalues

• Modelvaluesoflocalvariablesandstackoperandsusinglatticeofsymbolicvalues

• Whentwocontrolpathsmerge,usemeet operatortocombinevalues:

8115-214

Dataflowvalues

• Modelvaluesoflocalvariablesandstackoperandsusinglatticeofsymbolicvalues

• Whentwocontrolpathsmerge,usemeet operatortocombinevalues:

Null⬦Null=Null

8215-214

Dataflowvalues

• Modelvaluesoflocalvariablesandstackoperandsusinglatticeofsymbolicvalues

• Whentwocontrolpathsmerge,usemeet operatortocombinevalues:

Null⬦ NotNull=MaybeNull

8315-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

8415-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

8515-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = not nullz = not null

8615-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = not nullz = not null

8715-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = not nullz = not null

x = nully = nullz = null

8815-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = maybez = maybe

8915-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = null

9015-214

Null-pointerdataflowexamplex=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

z = uncertain

9115-214

Abstract Interpretation• Staticprogramanalysisisthesystematicexamination ofanabstractionofaprogram’sstatespace

• Abstraction– Don’ttrackeverything!(that’snormalinterpretation)– Trackanimportantabstraction

• Systematic– Ensureeverythingischeckedinthesameway

Details on how this works in 15-313

9215-214

COMPARINGQUALITYASSURANCESTRATEGIES

9315-214

Errorexists No errorexists

ErrorReported Truepositive(correctanalysisresult)

Falsepositive(annoying noise)

NoErrorReported Falsenegative(falseconfidence)

True negative(correctanalysisresult)

Sound Analysis: reports all defectsà no false negativestypically overapproximated

Complete Analysis:every reported defect is an actual defect à no false positivestypically underapproximated

9415-214

Checkyourunderstanding

• Whatisatrivialwaytoimplement:– asoundanalysis?– acompleteanalysis?

9515-214

DefectsreportedbySoundAnalysis

AllDefects

DefectsreportedbyCompleteAnalysis

UnsoundandIncompleteAnalysis

9615-214

Errorexists No errorexists

ErrorReported Truepositive(correctanalysisresult)

Falsepositive(annoying noise)

NoErrorReported Falsenegative(falseconfidence)

True negative(correctanalysisresult)

How does testing relate? And formal verification?

Sound Analysis: reports all defectsà no false negativestypically overapproximated

Complete Analysis:every reported defect is an actual defect à no false positivestypically underapproximated

9715-214

The Bad News: Rice's Theorem

• Everystaticanalysisisnecessarilyincompleteorunsoundorundecidable(ormultipleofthese)

• Eachapproachhasdifferenttradeoffs

"Any nontrivial property about the language recognized by a Turing machine is undecidable.“

Henry Gordon Rice, 1953

9815-214

Soundness/Completeness/PerformanceTradeoffs• Typecheckingdoescatchaspecificclassofproblems

(sound),butdoesnotfindallproblems• Compileroptimizationsmusterronthesafeside(only

performoptimizationswhensureit'scorrect;->complete)• Manypracticalbug-findingtoolsanalysesareunsoundand

incomplete– Catchtypicalproblems– Mayreportwarningsevenforcorrectcode– Maynotdetectallproblems

• Overwhelmingamountsoffalsenegativesmakeanalysisuseless

• Notall"bugs"needtobefixed

9915-214

Testing, Static Analysis, and Proofs• Testing

– Observableproperties– Verifyprogramforoneexecution– Manualdevelopmentwith

automatedregression– Mostpracticalapproachnow– Doesnotfindallproblems

(unsound)

• StaticAnalysis– Analysisofallpossibleexecutions– Specificissuesonlywith

conservativeapprox.andbugpatterns

– Toolsavailable,usefulforbugfinding

– Automated,butunsoundand/orincomplete

• Proofs(FormalVerification)– Anyprogramproperty– Verifyprogramforallexecutions– Manualdevelopmentwith

automatedproofcheckers– Practicalforsmallprograms,may

scaleupinthefuture– Soundandcomplete,butnot

automaticallydecidable

What strategy touse in your project?

10015-214

Take-HomeMessages

• Therearemanyformsofqualityassurance• Testingshouldbeintegratedintodevelopment

– possiblyeventestfirst• Variouscoveragemetricscanmoreorlessapproximatetestsuitequality

• Staticanalysistoolscandetectcertainpatternsofproblems

• Soundnessandcompletenesstocharacterizeanalyses