Transcript
Page 1: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Fault Localization Where is the root cause

Tho T. Quan SAVE research group Faculty of Computer Science and Engineering Hochiminh City University of Technology Hochiminh, Vietnam [email protected]

Page 2: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Outline

Introduction

Fault Localization: An Odyssey

Academic View

Demonstrations

Conclusion

Page 3: [Tho Quan] Fault Localization - Where is the root cause of a bug?

About SAVE

System Analysis and VErification

Founded in 2008, by Dr. Nguyen Hua Phung

Current leader: A/Prof. Quan Thanh Tho

www.cse.hcmut.edu.vn/~save

Page 4: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 5: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Research topics

Formal methods to design, verify and secure computer-based systems

Improvement of programming productivity and performance

Education and practical tools to assist teaching and practicing programming and software engineering

Page 6: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Research Directions

Polymorphism Virus

Mobile Security Checking

Petri Nets – PAT – Model Checking

Automatic Composition of Prototype

Software Testing

Fault Localization

Page 7: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Fault Localization: An Odyssey

Concept

Tour

Page 8: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Fault Localization - Concept

Intuition view

Program introduced abnormal behaviors

Test case failed

Where is the bugs

Approaches

Memory dump

Breakpoint setting

Print out intermediate values

Issue: Locate the suspicious code

Page 9: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Fault Localization - Concept

Fault Localization

Locate the code that may cause the bug

Based on testing results

Automatic or Semi-Automatic

Page 10: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Let the tour commence

Page 11: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 12: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 13: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 14: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 15: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 16: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 17: [Tho Quan] Fault Localization - Where is the root cause of a bug?

More insight analysis

SQL Fault localization

Expert-based Localization

Page 18: [Tho Quan] Fault Localization - Where is the root cause of a bug?

SQL inspection

Page 19: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 20: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 21: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Expert asset

Page 22: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 23: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Does it really work? - Testing with Siemens suite

Page 24: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Fault Localization: Under Academic View

Page 25: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Overview

Program

Test suite

Test result: - Passed/failed test cases - Program spectra

Fault localization techniques

The ranked list of elements based on suspicious degrees

The set of suspicious elements

The root cause

spectrum

model

proof

Page 26: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Fault Localization Techniques

Model-based

Proof-based

Spectrum-based

Page 27: [Tho Quan] Fault Localization - Where is the root cause of a bug?

FL techniques: Model-based

Produce a mathematical model of the program

Solve the model to find suspicious elements

Page 28: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 29: [Tho Quan] Fault Localization - Where is the root cause of a bug?

FL techniques: Proof-based

Produce an unsatisfied formula for an error

Find the elements in the formula which cause the formula satisfied when removed

Page 30: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 31: [Tho Quan] Fault Localization - Where is the root cause of a bug?

FL techniques: Spectrum-based

Based on spectra of testing result

Returned a ranked list of suspicious elements

Page 32: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 33: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Some Definitions

S(e): suspicious score of element e

Nef(e): failed executions involving e

Nnf(e): failed executions NOT involving e

Nep(e): passed executions involving e

Nnp(e): passed executions NOT involving e

Page 34: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Taratula, Ochiai and Jaccard

Page 35: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Tarantula Ochiai Jaccard

3 0.5 0.7 0.5

4 1 1 1

6 0 0 0

n = 1 n= -1

Page 36: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Tarantula Ochiai Jaccard

3 0.5 0.58 0.33

4 1 1 1

6 0 0 0

n = 1 n= -1 n = 0

Page 37: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 38: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Open Issues

Impact of test-case sets

Missing code problem

Page 39: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Test-case sets

Better strategy of test-case generation yields better fault localization performance

Page 40: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 41: [Tho Quan] Fault Localization - Where is the root cause of a bug?

0.0;1.0;2.0 1.0;4.0;3.0 2.0;2.0;3.0

Page 42: [Tho Quan] Fault Localization - Where is the root cause of a bug?

0.0;1.0;2.0 1.0;4.0;3.0 2.0;2.0;3.0 1.0;2.0;1.0

Page 43: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Missing code problem

Fault localization aims at finding suspicious code that causes error

When the logic error caused by missing some code no code

available to be “suspected”

Page 44: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 45: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Case Study: The PROVE system

Program Verification system

http://elearning.cse.hcmut.edu.vn/provegroup/index.jsp

Verify student programming works

Produce counter-example

Show execution path

Show suspected code

Page 46: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 47: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 48: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 49: [Tho Quan] Fault Localization - Where is the root cause of a bug?
Page 50: [Tho Quan] Fault Localization - Where is the root cause of a bug?

Conclusion

Fault localization is the next step of testing

Spectrum-based is deemed suitable to be applied in industry

Applied in an online educational tool at HCMUT

Chances for collaboration with industry


Recommended