Learning about Bugs in Systems

Laune Harris, Le Hoang Anh, Guy Lichtman, Nikos Michalakis

Bugs and bug finding techniques

Bugs

• Bugs are software defects that cause– Incorrect behaviour– Unintended behaviour

• Bugs are expensive – Development costs– Maintenance costs– Repair/recovery costs– Downtime– Sometimes lethal

Bug causes

• Design flaws

• Errors in source

• Mistakes in trans. from design -> source

• Unforeseen interactions with users

• Unforeseen interactions with other components

Bug Taxonomy

• Resource Management Bugs– Memory

• Leak, stack smashing, access violations, NULL dereference, using uninitialised memory

– File/Socket • Leak, double close, etc.

Examples

• //stack buffer overun for sizes greater than 42   void stack_buf( void* src, int size ) {     char buffer[42];    memcpy( buffer, src, size );  }

• //null dereference• public class nullDeref {

    public static void main(String argv []) {    MyObject o = null;     BufferedReader keyb = new BufferedReader;     String inp = keyb.readLine();     if( inp != "quit" ){       o.construct( inp );     }     System.out.println(o.toString());   }

Bug Taxonomy cont’d

• Concurrent Bugs– Data race– Inconsistent synchronisation– Deadlock– Orphaned Threads– Livelock

Example

• //Data race• int cnt = 0;

• void thread1() {• int y = cnt;• cnt = y + 1;• }• void thread2() {• int y = cnt;• cnt = y + 1;• }

Bug Taxonomy

• Semantic Bugs– Bugs that contradict programmer intent

– Often specific to APIs and domains• Eg: not releasing an acquired lock

Examples

• /* from sys/kern/disk.c*/int sys_disk_request( u_int sn, struct Xn_name *xn_user, struct buf* reqbp, u_int k )    if( reqbp->bflags & B_SCSICMD )       return sys_disk_scsicmd( sn, k, reqbp );

• //SQL injection• public void authenticate(HttpServletRequest request){

    String username = request.getParameter("user");    java.sql.Statement stmt = con.createStatement();     String query = "select * from users where username = ’" + username + "’and password = ’" + pwd + "’";    stmt.execute(query); ... // process the result of SELECT   }

Bug Taxonomy cont’d

• Arithmetic bugs– Divide by 0, floating point errors, range wraps,

truncation errors

• Peformance bugs– Copy instead of mmap

• Misc– Off by one, Short circuiting

conditionals ,unhandled exceptions

Finding bugs

• Lexical Analysis (source code based0– Lint, splint, clint, jlint, etc

• Semantic Analysis (AST/IR based)– ASTLog, CodeQuest, JQuery, etc

• Dynamic Runtime Analysis– Purify, Valgrind, Dynamine

• Program Trace Analysis– PQL, PTQL

Bug patterns

• Bugs found by error patterns• Eg. Null dereference pattern

public class nullDeref {     public static void main(String argv []) {    MyObject o = null;     BufferedReader keyb = new BufferedReader;     String inp = keyb.readLine();     if( inp != "quit" ){       o.construct( inp );     }     System.out.println(o.toString());   }

Bug patterns

• CVS Logs– Check revised code for patterns

• Bug reports/databases– Match bug report to source code

• Statistical Anal. to extract patterns– Pattern A is correlated to bug X

• Inferred from domain/api

Bug indicators

• Associate with poorly written code– Sections with dead/redundant code may

indicate errors

• Associate with style violations– Eg: failing to put constant on left side of

assignment may be (but not necessarily) error

~Bug Patterns

• Infer models of proper behaviour. Violations flagged as possible bugs– Ammons et al: Mining Specifications– Lam, et al: Dynamine (revision history

mining)– Engler et al: Bugs as deviant behaviour

Case: Linux Kernel

Linux kernel overviewSources: www.coverity.com

www.kernel.org

Latest version: 2.6.15.4 $ uname –a /usr/src/kernels

Bug distribution

Linux kernels: 0.17 bugs per 1000 lines of codeCommercial software: 20-30 bugs per 1000 lines of code

Files and functions in linux kernel

Version 2.6.15.4 has 16,249 files

Bug types

• General bug types: (version 2.6.9)– Forward null - null pointer dereference

– Reverse null - null pointer dereference or spurious null check

– Dead code - unused code due to logic flaws

– Overrun static - buffer overrun in the stack

– Resource leak - leak of memory or other system resources

– Free - use of resources that are no longer available

– Null returns - null pointer dereference

– Negative returns - buffer overrrun using a negative offset in stack/heap – Reverse negative - negative returns or spurious check against a negative

value– Overrun dynamic - buffer overrun in the heap

Security defects

– Tainted scalar – unsafe usage of tainted scalars – Tainted string – unsafe usage of tainted string – User pointer – unsafe dereference of user-land pointers – String null – unsafe usage of tainted, potentially non-terminated

strings – String size – unsafe usage of tainted strings with a potentially

unbounded size

– Overrun static – buffer overrun on stack

– Overrun dynamic – buffer overrrun on heap

Future bugs

– Most of detected bugs have been fixed– Linux is growing, there are more and more

contributors– New code, new bugs– Distributed development could decrease

the overall security

Two Case Studies of Open Source Software Development:

Apache and Mozilla

AUDRIS MOCKUSAvaya Labs Research

ROY T FIELDINGDay Software

andJAMES D HERBSLEB

Carnegie Mellon University

• Apache httpd: open source http server• Mozilla: open source browser including

supporting tools (such as Bugzilla)• Compared to 5 commercial projects

related to telecommunications• Data extracted using scripts (with

manual intervention) from CVS, Bug tracking system and email lists

Apache Code Contribution

Mozilla Code Contribution

Mozilla Modules, Apache and Commercial Projects Comparison

Apache Fixes

Apache Defect Density

Mozilla, Apache and Commercial Defect Density

Summary

• Many developers (~400)• Small core contribute most of code

development (Apache: 15 contribute over 80%)

• High participation of non-core team regarding fixes (Apache: core responsible only for 66%)

• Relatively low defect density

ML related work and OS error characteristics

DynaMine• Problem:

– Want to find patterns whose violation causes errors– Want to find patterns for program understanding

• Technique:– Look at revision histories

• Crucial observation:

• Use data mining techniques to find methods that are often added at the same time

• Mining uses the Apriori alg to compute patterns.• Patterns found: method pairs, state machines, more complex

Things that are frequently checked in together often form a

pattern

Cluster Filtering

• Problem: when beta testing software, inputs that cause errors are sparse.

• Idea: cluster execution profiles and select test cases based on clusters.

• Non-faulty profiles will cluster together, so more likely to pick faulty profiles if sample clusters.

• Profile vector = function caller(ee) count (large vector).

Fault Invariant Classifier

• Finds latent code errors by learning over program executions

• Targeted programs:– Test inputs easy to generate, outputs hard

to compute (e.g. complex comp, GUI’s)

• 2 steps: create model and classification

OS error characteristics

• 2001 study showed:• Most OS errors are in drivers• Errors increase as functions become larger.• Distribution: few files have several errors, long tail

has just one or two• Bugs live on average 1.8 years between versions.• Errors cluster where programmer ignorance of

interface or system rules combines with copy-and-paste.

Discussion

• Extracting the right program properties is still an art – predicates, invariants, functions calls for dynamic

analysis– Cvs transactions for static analysis

• What do you need for training? Labels?• What features are good for learning bugs?• Do we need to run the code?