Combining Static and Dynamic Impact Analysis forLarge-scale Enterprise Systems

The 15th International Conference on Product-Focused Software ProcessImprovement, Helsinki, Finland.

Wen Chen, Alan Wassyng, Tom Maibaum

McMaster Centre for Software Certification (McSCert)Department of Computing and Software

McMaster UniversityHamilton, Ontario, Canada

December, 2014

1 / 29

Outline

1 Large-scale Enterprise SystemsIntroductionCharacteristicsChanges Are Inevitable

2 Conventional Impact AnalysisIntroductionStatic AnalysisDynamic Analysis

3 Combining Static and Dynamic Analysis

4 The Approach at a Glance

5 Empirical Study

6 References

2 / 29

Large-scale Enterprise SystemsIntroduction

Enterprise systems (ES) are large-scale application software packages thatsupport business processes, information flows, reporting, and dataanalytics in complex organizations. Types of ES include but not limit to:

Enterprise Resource Planning (ERP) Systems

Customer Relationship Management (CRM) Systems

Supply Chain Management (SCM) Systems

Example

Oracle E-Business Suite, SAP ERP, Red Hat JBoss

3 / 29

Large-scale Enterprise SystemsCharacteristics

Scalable.

Complex.

Critical.

Costly.

Example

Total number of MODULES in SAP ERP: 241.Total number of CLASSES in Oracle E-Biz: 230 thousand.Total number of METHODS in Oracle E-Biz: 4.6 million.Large companies can spend $50 million to $100 million on softwareupgrades.

4 / 29

Large-scale Enterprise SystemsChanges Are Inevitable

System upgrade

User requirement change

Environment change

Performance issue

Other customized changes

The latest IT Key Metrics Data from Gartner (gartner12, 2011) reportthat in 2011 some 16% of application support activity was devoted totechnical upgrades, rising to 24% in the banking and financial servicessector. While making changes, current blind testing strategy is veryexpensive and time-consuming, however, the actual effect of a change mayin fact be minimal.

5 / 29

Large-scale Enterprise SystemsChanges Are Inevitable

A well-defined change impact analysis is required to:

reduce risks of unintended changes

reduce costs

minimize human efforts

focus testing

help identify uncovered false negatives

6 / 29

Software Change Impact AnalysisIntroduction

Software Change: Operations {add, modify, delete...} on softwareentities {function, field, logic, module, database objects...}Change Impact Analysis: Estimates WHAT will be affected insoftware and related documentation if a proposed software change ismade (Bohner, 1996).

7 / 29

Software Change Impact AnalysisStatic Analysis

Static analysis is to

identify a subset of affected elements of the program by analysing thecode

abstract all possible software behaviors by graphs (call graph,dependency graph ...) or other static representations

Static analysis is safe and complete, but it often comes up with too largeimpact sets due to the over conservative assumptions: the actualdependencies may turn out to be considerably smaller than the possibleones. Additionally, it usually requires long execution time.

8 / 29

Software Change Impact AnalysisDynamic Analysis

Dynamic analysis is to

identify a subset of affected elements of the program by analysingruntime information

collect dynamic information such as: event traces, test coverages,executions in the fields

Dynamic analysis is precise and efficient, but it often comes up withincomplete analysis due to under-estimation.

9 / 29

Combining Static and Dynamic AnalysisAspect-oriented programming (AOP)

“The hierarchical modularity mechanisms in object-oriented languages areextremely powerful, but they are inherently unable to modularize allconcerns of interest in complex systems.” (Kiczales et al., 2001)“Aspect-oriented programming (AOP) does for concerns that are naturallycrosscutting what OOP does for concerns that are naturally hierarchical, itprovides language mechanisms that explicitly capture crosscuttingstructure.” (Kiczales et al., 2001)

10 / 29

Combining Static and Dynamic AnalysisAspectJ

AspectJ adds to Java a new concept, joint point, and some constructs:

pointcuts pick out certain joint points in the program flow;

After pointcuts pick out join points, we use advice to implementcrosscutting behaviour. Advice brings together a pointcut (to pick outjoin points) and a body of code (to run at each of those join points);

11 / 29

Combining Static and Dynamic AnalysisAspectJ

Inter-type declarations in AspectJ are declarations that cut acrossclasses and their hierarchies. They may declare members that cutacross multiple classes, or change the inheritance relationship betweenclasses;

The definition of aspects is very similar to classes, which wrap uppointcuts, advice, and inter-type declarations in a a modular unit ofcrosscutting implementation.

12 / 29

Combining Static and Dynamic AnalysisAspectJ Example Code

Figure : Aspect Trace

13 / 29

Combining Static and Dynamic AnalysisAspectJ Example Output Sample

Figure : Output Sample for MGPAPP.class

14 / 29

Combining Static and Dynamic AnalysisBenefits

integrates with our safe static analysis (Chen et al., 2013);

provides precise estimation of impacts;

works at bytecode level;

does not alter system behaviour in any ways;

saves efforts in learning the application logic;

efficient in both time and space (seconds/class and kilobytes/class).

15 / 29

The Approach at a GlanceAnalysis Overview

EnterpriseSystem

AtomicChanges(AC)

ChangeAnalysis

Changes(C)

StaticAnalysis

AccessDependencyGraph

DynamicAnalysis

DynamicImpacts(D)

ReverseSearch

StaticImpacts(S)

PotentialFalse-Positives(PO)

ReachabilityAnalysis

AliasAnalysis

subtract

ImpactSet(I)union

input

output

Figure : Analysis Process Overview

16 / 29

The Approach at a GlanceAnalysis Overview

Steps in our approach include(Chen et al., 2013) (Chen, Wassyng, &Maibaum, 2014):

(i) Static analysis to abstract a representation of the target programP. A full dependency graph G is built for the system at functionslevel.

(ii) Change analysis to identify direct and indirect changes. Theidentification of indirect changes may require String Analysis.

17 / 29

The Approach at a GlanceAnalysis Overview

(iii) Graph searching algorithm is employed to extract a static impactset S . The static impact set S is conservative but safe, we will becutting off false positives from within this set.

(iv) Instrumenting the program P to collect a dynamic impact set D.The dynamic impact set D contains real execution information thatwe should keep in the static impact set S .

18 / 29

The Approach at a GlanceAnalysis Overview

(v) Reachability analysis to filter out other unidentified paths indynamic analysis that are false positives. Paths taken into account inthis step are those that haven’t been executed in dynamic analysis buthave the potential of reaching a direct/indirect change. Paths filteredout in this analysis are considered as infeasible paths (mis-matchedcalls and returns).

(vi) Pointer/aliasing analysis to further filter out unidentified paths. Ifthere isn’t any variable along a particular path aliased to any variablewithin a changed method, this path can be regarded as a falsepositive. Different from the infeasible paths identified in reachabilityanalysis, paths filtered out in this analysis are feasible but won’t beaffected by the changes.

Note that, step (v) and step (vi) are continued research appeared in (Chenet al., 2014).

19 / 29

Empirical Study

Target system: Oracle E-Business Suite Version 11.5

Source of changes: Oracle patch # 5565583, 10107418, 14321241

Objective: identify the impact set of the patches

Physical environment: Quad core 3.2GHz CPU, 32GB RAM, 64-bitRed Hat Linux Enterprise version

20 / 29

Empirical StudyCont’d

Figure : Oracle E-Business Suite System Architecture

Modules: CRM, CSM, Financials, SCM, HRMS...

21 / 29

Empirical StudyCont’d

Oracle E-Business Suite V11.5:

Number of classes: 195’999

Number of entities (functions and fields): 3’157’947

Patches will be affecting both application tier and database tier.

Patches Size Number of direct changesPatch # 5565583 212MB 52’870

Patch # 10107418 10KB 0

Patch # 14321241 99MB 230’209

22 / 29

Empirical StudyEmpirical Results

Oracle E-Biz NumbersClasses 195’999

Entities 3’157’947

Static dependencies 18’387’466

Dynamic dependencies 8’200

Reduced dependencies afterreachability analysis and alias-ing analysis

11’521’769

23 / 29

Empirical StudyResults

Figure : Oracle E-Business Suite 11.5, Patch 5565583 and its impacts.

24 / 29

Empirical StudyEmpirical Results

Patches 5565583 10107418 14321241

Size 212MB 10KB 99MB

Number of direct changes 52’870 0 25’114

Affected functions 699’534 0 230’209

Affected functions % 22% 0% 7.3%

Affected top functions 160’800 0 69’971

Affected top functions % 5.1% 0% 2.2%

Static Analysis

0 16.3 66.3 88.3 hours

Dynamic Analysis Reachability and Alias Analysis

9.5 64.3 76.3 83.3

Build access dependency graph

Extract changesand computestatic impacts

Instrumentation Compute dynamicimpacts

Build CFGand supergraph

Compute aliasinginformation

Mapping

Gather results

Figure : Analysis Time for Patch #5565583

25 / 29

SummaryAchievements

We have developed a multi-tasking, aspect-oriented instrumentor toadequately instrument large-scale systems and collect traces atbytecode level.

We have successfully combined static analysis and dynamic analysis.Static analysis was used as the input to dynamic analysis, providing asafety guarantee whenever full potential impacts are needed.

We have empirically demonstrated the practical applicability of theimproved approach on a very large enterprise system involvinghundreds of thousands of classes. Such systems are perhaps 2 ordersof magnitude larger than the systems analyzed by other approaches.

26 / 29

SummaryFuture Work

Running time still needs to be improved;

The impacts identified by dynamic analysis was only a small portionof the static impacts (0.015%), though they were executed hundredsof thousands of times.

Need to include customized code.

27 / 29

Bibliography

Bohner, S. A. (1996). Software Change Impact Analysis. In Proceedingsof the 27th annual nasa goddard/ieee software engineering workshop(sew-27’02).

Chen, W., Iqbal, A., Abdrakhmanov, A., Parlar, J., George, C., Lawford,M., . . . Wassyng, A. (2013). Large-scale enterprise systems:Changes and impacts. In Enterprise information systems (Vol. 141,p. 274-290). Springer Berlin Heidelberg.

Chen, W., Wassyng, A., & Maibaum, T. (2014). Impact analysis viareachability and alias analysis. In U. Frank, P. Loucopoulos, . Pastor,& I. Petrounias (Eds.), The practice of enterprise modeling(Vol. 197, p. 261-270). Springer Berlin Heidelberg. Retrieved fromhttp://dx.doi.org/10.1007/978-3-662-45501-2 19 doi:10.1007/978-3-662-45501-2 19

IT Key Metrics Data 2012. (2011, December). Gartner, Inc.Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., & Griswold,

W. G. (2001). An overview of aspectj. In Ecoop 2001object-orientedprogramming (pp. 327–354). Springer. 28 / 29

Thank you!

29 / 29