Impact Analysis
Yury Chebiryak
University of SaarlandDepartment of Computer Science
2/96
Impact Analysis is often used to assess the effects of a
change after change has been made
is rarely used to predict the effect of change before it is instantiated
3/96
Techniques
ImpactAnalysis
TraceabilityAnalysis
DependencyAnalysis
4/96
Dependency-based IA
DependencyAnalysis
Program Slicing Call Graph
Static Dynamic
Whole Program Path-Based Dynamic Impact Analysis
James Law
Gregg Rothermel
6/96
PathImpact Dynamic Call-based Works on binaries Relatively low cost Relatively low overhead
7/96
Call Graph
M
A D
C E
F
G
B
8/96
Call Graph
M
A D
C E
F
G
B
B is changed
9/96
M
A D
C E
F
G
B
Call Graph
Assumption:change in B has a
potential impact on nodes reachable
from B
10/96
Execution Traces (ET)
M
A D
C E
F
G
B
Run M()
ET: M
11/96
Execution Traces
Run B()
ET: MB
M
A D
C E
F
G
B
12/96
Execution Traces
M
A D
C E
F
G
B
Return
ET: MBr
13/96
Execution Traces
ET: MBrA
M
A D
C E
F
G
B
Run A()
14/96
Execution Traces
Run C()
ET: MBrAC
M
A D
C E
F
G
B
15/96
Execution Traces
Run D()
ET: MBrACD
M
A D
C E
F
G
B
16/96
Execution Traces
Return
ET: MBrACDr
M
A D
C E
F
G
B
17/96
Execution Traces
Run E()
ET: MBrACDrE
M
A D
C E
F
G
B
18/96
Execution Traces
Return
ET: MBrACDrEr
M
A D
C E
F
G
B
19/96
Execution Traces
Return
ET: MBrACDrErr
M
A D
C E
F
G
B
20/96
Execution Traces
Return
ET: MBrACDrErrr
M
A D
C E
F
G
B
21/96
Execution Traces
M
A D
C E
F
G
B
Return
ET: MBrACDrErrrrx
22/96
Execution Traces II Single
Multiple
MBrACDrErrrrx
MBrACDrErrrrxMBGrrrrxMBCFrrrrx
Programs with loops => very long trace
23/96
SEQUITUR algorithm
Data compression algorithm by Larus: Online Created grammar reproduces trace exactly O(N) running time O(N) size in the worst case O(log N) size in the best case e.g. 2GB trace → 100 MB
24/96
SEQUITUR algorithmalgorithm SEQUITUR( S )input Execution Trace Soutput Grammar GGrammar GRule T1. for each token in S2. append token to end of production for T3. if duplicate digram appears4. if other occurrence is a rule g in G5. replace new digram with non-terminal of g.6. else7. form a new rule and replace duplicate8. digrams with the new non-terminal.9. if any rule in G is used only once10. remove the rule by substituting the production.11. return G
25/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
26/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → MBrACDrErrrrx
27/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → MBrACDrE x1 → rr
1 1
28/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → MBrACDrE11x1 → rr
29/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → MBrACDrE11xM1 → rr
30/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → rACDrE11x 1 → rr
MBMB
31/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → rACDrE11x 1 → rr2 → MB
22
32/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → 2rACDrE11x2 1 → rr2 → MB
33/96
SEQUITUR: example
ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx
T → 2rACDrE113G43CF4rx 1 → rr2 → MB3 → x24 → 1r
grammar => whole path DAG
34/96
Whole path DAG
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
35/96
PathImpact
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r x
UP()
E
36/96
PathImpact
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r x
UP()
E
XBACKWARD() FORWARD()
37/96
PathImpact
PathImpact(E): I {E} I = {}
Returns=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
38/96
PathImpact
PathImpact(E): up(T, E) I = {E}
Returns=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
39/96
PathImpact
PathImpact(E): forward(1) I = {E}
Returns=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
40/96
PathImpact
PathImpact(E): forward(r) I = {E}
Returns=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
41/96
PathImpact
PathImpact(E): Returns++ I = {E}
Returns=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
42/96
PathImpact
PathImpact(E): forward(r) I = {E}
Returns=1
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
43/96
PathImpact
PathImpact(E): Returns++ I = {E}
Returns=1
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
44/96
PathImpact
PathImpact(E): after forward(1) I = {E}
Returns=4
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
45/96
PathImpact
PathImpact(E): forward(3) -> x I = {E}
Returns=4
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
46/96
PathImpact
PathImpact(E): backward(r) I = {E}
Returns=4
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
Skip=0
47/96
PathImpact
PathImpact(E): Skip++ I = {E}
Returns=4Skip=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
48/96
PathImpact
PathImpact(E): backward(D)->Skip-- I = {E}
Returns=4Skip=1
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
49/96
PathImpact
PathImpact(E): backward(C)->I := I U {C},Returns--
I = {E}
Returns=4Skip=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
50/96
PathImpact
PathImpact(E): backward(A)->I := I U {A},Returns--
I = {E, C}
Returns=3Skip=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
51/96
PathImpact
PathImpact(E): backward(r)->Skip++ I = {E, C, A}
Returns=2Skip=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
52/96
PathImpact
PathImpact(E): after backward(2) I = {E, C, A, M}
Returns=1Skip=0
T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x
1 r r 2 M B 3 x 2 4 1 r
M A B C D GF r xE
Chianti: A Tool for ChangeImpact Analysis of Java Programs
Ophelia ChesleyXiaoxia Ren
Barbara G. RyderFenil ShahFrank Tip
54/96
Chianti: high-level concept
Inputs:
1) test suite T
2) original version of code
3) edited version of code
55/96
Chianti: high-level concept
1) set of interdependent atomic changes A
56/96
Chianti: high-level concept
1) set of interdependent atomic changes A
2) call graph for each test in T
57/96
Chianti: high-level concept
1) set of interdependent atomic changes A
2) call graph for each test in T
3) T´ T – potentially affected by cA
58/96
Chianti: high-level concept
1) set of interdependent atomic changes A
2) call graph for each test in T
3) T´ T – potentially affected by cA
4) for tiT´ determine A´A that may have affected ti
59/96
Chianti: high-level concept
1) set of interdependent atomic changes A
2) call graph for each test in T
3) T´ T – potentially affected by cA
4) for tiT´ determine A´A that may have affected ti
determine subset of methods affected by
a set of changes
isolate subset of changes that affect
a given test
60/96
Chianti: architectureSet of Unit Tests
Changedprogram P’
Originalprogram P
Call Graphof Tests
in P’
AffectedTests
AffectingChanges
Call GraphBuilder
CI analyzer
CHIANTI
Call Graphof Tests
in P
Atomic ChangeDecoder
AtomicChanges &
Dependencies
61/96
Chianti: architectureSet of Unit Tests
Changedprogram
P’
Originalprogram
P
Call Graphof Tests
in P’
AffectedTests
AffectingChanges
Call GraphBuilder
CI analyzer
CHIANTI
Call Graphof Tests
in P
Atomic ChangeDecoder
AtomicChanges &
Dependencies
62/96
Chianti: architectureSet of Unit Tests
Changedprogram P’
Originalprogram P
Call Graphof Tests
in P’
AffectedTests
AffectingChanges
Call GraphBuilder
CI analyzer
CHIANTI
Call Graphof Tests
in P
Atomic ChangeDecoder
AtomicChanges &
Dependencies
63/96
Example: original versionclass A{public A() {}public void foo (){}
}class B extends A{public B(){}public void foo(){}
}class C extends A{public C(){}
}
64/96
Example: changed version
class A{void foo (){}int x;
}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;
}class C extends A{void foo(){ x=18; }int z;
}
65/96
Example: testsclass Tests {public static void test1(){
A a = new A();a.foo();
}public static void test2(){
A a = new B();a.foo();
}public static void test3(){
A a = new C();a.foo();
}}
66/96
Review: Atomic Changes
Change Description
AC Add an empty class
DC Delete an empty class
AM Add an empty method
DM Delete an empty method
CM Change body of a method
LC Change virtual method lookup
AF Add a field
DF Delete a field
67/96
Review: Atomic Changes II
Change Description
CFI Change definition of an instance field initializer
CSFI Change definition of a static field initializer
AI Add an empty instance initializer
DI Delete an empty instance initializer
CI Change definition of an instance initializer
ASI Add an empty static initializer
DSI Delete an empty static initializer
CSI Change definition of a static initializer
68/96
Example: Step 1
class A{void foo (){}int x;
}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;
}class C extends A{void foo(){ x=18; }int z;
}
69/96
Example: Step 1
AMB.bar()
class A{void foo (){}int x;
}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;
}class C extends A{void foo(){ x=18; }int z;
}
CMB.foo()
AFB.y
CMB.bar()
70/96
Example: Step 1
class A{void foo (){}int x;
}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;
}class C extends A{void foo(){ x=18; }int z;
}
AFA.x
CMC.foo()
AMC.foo()
LCC, A.foo()
71/96
Example: Step 1
class A{void foo (){}int x;
}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;
}class C extends A{void foo(){ x=18; }int z;
}
AFC.z
72/96
Example: atomic changesAM
B.bar()AFA.x
CMC.foo()
AMC.foo()
LCC, A.foo()
CMB.foo()
AFB.y
CMB.bar()
AFC.z
73/96
Recall: high-level concept
1) set of interdependent atomic changes A
2) call graph for each test in T
3) T´ T – potentially affected by cA
4) for tiT´ determine A´A that may have affected ti
74/96
Example: affected tests
A test is affected if its call graph contains:
1) node that corresponds to CM or DM, or
2) edge that corresponds to LC
75/96
Example: affected tests
76/96
Example: test1
void test1(){ A a = new A(); a.foo();}
<A, A.foo()>Tests.test1()
A.A() A.foo()
77/96
Example: test2
void test2(){ A a = new B(); a.foo();}
Tests.test2()<B, A.foo()>
B.B() B.foo()
A.A()
CMB.foo()
78/96
Example: test3
void test3(){ A a = new C(); a.foo();}
Tests.test3()<C, A.foo()>
C.C() A.foo()
A.A()
LCC, A.foo()
79/96
Recall: high-level concept
1) set of interdependent atomic changes A
2) call graph for each test in T
3) T´ T – potentially affected by cA
4) for tiT´ determine A´A that may have affected ti
80/96
Example: affecting changes
The set of atomic changes that affect a given affected test includes:
1) all atomic changes for nodes with AM and CM,
2) atomic changes in LC category, and
3) their transitively prerequisite atomic changes
81/96
Example: affecting changes
82/96
Example: AffectingChanges(test2)
Tests.test2()<B, A.foo()>
B.B() B.foo()
A.A() B.bar()
CMB.foo()
CMB.bar()
83/96
Example: AffectingChanges(test2)
AMB.bar()
CMB.foo()
AFB.y
CMB.bar()
84/96
Example
AFC.z
This atomic change does not affect any of the tests
=> additional tests are needed.
Remember the atomic change:
85/96
Snapshot
86/96
Chianti: Evaluation
Number of atomic changes between each pair of Daikon versions.
87/96
Chianti: Evaluation
Categorization of the atomic changes.
88/96
Chianti: Evaluation
Percentage of affected tests for each of the Daikon versions.
89/96
Chianti: Evaluation
Average percentage of affecting changes, per affected test.
90/96
Summary
PathImpact dynamic – uses dynamic traces online works on binariesChianti uses dynamic call graphs applicable for OOP (e.g. Java) determines affecting changes for every test
91/96
References[1] Chesley, O., Dolby, J., Ren, X., Ryder, B., Shah, F., and Tip, F. Chianti:
A prototype change impact analysis tool for Java. Tech. Rep. DCS-TR-533, Rutgers University Department of Computer Science, September 2003.
[2] Chesley, O., Ren, X., Ryder, B., Shah, F., and Tip, F. Chianti: a tool for change impact analysis of java programs. In Proceedings of the 19th annual ACM SIGPLAN Conference on Object-oriented programming, systems, languages, and applications, pp. 432448.
[3] Law, J., Rothermel, G. Whole program Path-Based dynamic impact analysis. In Proc. of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon.
[4] Ryder, B., Tip, F. Change impact analysis for object-oriented programs. In Proceedings of the 2001 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, pp. 4653, June 2001, Snowbird, Utah, United States.
92/96
Thanks for listening.
Questions?