Comparing the Decompositions Produced by Software ...bmitchell/research/icsm01pres.pdf · Produced by Software Clustering Algorithms using Similarity Measurements ... M1 M5 M2 M6

1

Comparing the DecompositionsProduced by Software Clustering

Algorithms using Similarity Measurements

2001 IEEE International Conference on Software Maintenance (ICSM'01).

Brian S. Mitchell & Spiros MancoridisMath & Computer Science, Drexel University

2Drexel University Software Engineering Research Group (SERG)http://serg.mcs.drexel.edu

Motivation

Using module dependencies when determining the similarity between two decompositionsis a good idea…


Clustering the Structure of a System (1)

Given the structure of a system…



The goal is to partition the system structure graphinto clusters…

The clusters shouldrepresent the

subsystems



But how do we know that the clustering result is good?


Ways to Evaluate Software Clustering Results…

Assess it against a mental modelAssess it against a benchmark standardTechniques:n Subjective Opinionsn Similarity Measurements

Given a software clustering result, we can:


Example: How “Similar” are these Decompositions?

M1

M5

M2

M6

M3

M4

M7

M8

M1

M5

M2

M6

M3

M4

M7

M8

Green Edges:Similarity still the same…

Red Edges:Not as similar…

Conclusions:Once we add the red edges the similarity between PA and PB decreases

PA

PB

Blue Edges:Similarity still the same…


Observations

Edges are important for determining the similarity between decompositionsExisting measurements don’t consider edges:n Precision / Recall (similarity)n MoJo (distance)Our idea: Use the edges to determine similarity


Research ObjectivesCreate new similarity measurements that use dependencies (edges) n EdgeSim (similarity)n MeCl (distance)

Evaluate the new similarity measurements against MoJo & Precision/RecallUse similarity measurements to support evaluation of software clustering results (see our WCRE’01 paper)


Example: How “Similar” are these Decompositions?

M1

M5

M2

M6

M3

M4

M7

M8

M1

M5

M2

M6

M3

M4

M7

M8

Add Green Edges:PR, MoJo, MeCl &EdgeSim unchanged.

Add Red Edges:PR, MoJo unchanged. EdgeSim, MeCl reduced.

PA

PB

Add Blue Edges:PR, MoJo, MeCl &EdgeSim unchanged.


Definitions

M1

M2

M3

M4

Internal/Intra-Edge: Edge within a cluster

External/Inter-Edge: Edge between two clusters


EdgeSim Example

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

c f g

hd e

i j

k l

MDG


EdgeSim Example

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

c f g

hd e

i j

k l

MDG

Step 1:Find Common Inter-

and Intra-Edges


EdgeSim Example

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

c f g

hd e

i j

k l

MDG

CommonEdge Weight

Total EdgeWeight

=10

19= 53%


MeCl Example

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

c f g

hd e

i j

k l

MDG


MeCl Example(A→B)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

A1 A3A2

B1 B2


MeCl Example(A1 B1)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

c

A1,1A1 A3A2

B1 B2

U


MeCl Example(A2 B1)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

A1,1 A2,1A1 A3A2

B1 B2

U


MeCl Example(A1 B2)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e

A1,1 A2,1

A1,2

A1 A3A2

B1 B2

U


MeCl Example(A2 B2)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e h

A1,1 A2,1

A1,2 A2,2

A1 A3A2

B1 B2

U


MeCl Example(A3 B2)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e h

k l

i j

A1,1 A2,1

A1,2 A2,2

A3,2

A1 A3A2

B1 B2

U


B1

B2

MeCl Example(A→B)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e h

k l

i j

A1,1 A2,1

A1,2 A2,2

A3,2

A1 A3A2

B1 B2


MeCl Example (A→B)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e h

k l

i j

A1,1 A2,1

A1,2 A2,2

A3,2

A1 A3A2

B1 B2

B1

B2

Newly Introduced Inter-Edges


MeCl Example(B→A)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e h

k l

i j

B1,1 B1,2

B2,1 B2,2

B2,3

A1 A3A2

B1 B2


A1 A2

A3

MeCl Example(B→A)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e h

k l

i j

B1,1 B1,2

B2,1 B2,2

B2,3

A1 A3A2

B1 B2


MeCl Example (B→A)

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

a b

cf g

d e h

k l

i j

B1,1 B1,2

B2,1 B2,2

B2,3

A1 A3A2

B1 B2

A1 A2

A3

Newly Introduced Inter-Edges


MeCl Calculation

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

A1 A3A2

B1 B2

Inter-Edges Introduced

MeCl(A→B):({b,e},{e,c},{g,h},{f,h})

MeCl(B→A):({e,i},{h,j},{b,f},{c,f},{h,e})

MeCl= 1 -maxW(MA→B, MB→A)

Total Edge Weight

MeCl = 1-5

19= 73.7%


Similarity Measurement RecapM1

M5

M2

M6

M3

M4

M7

M8

M1

M5

M2

M6

M3

M4

M7

M8

A1:

B1:

M1

M5

M2

M6

M3

M4

M7

M8

M1

M5

M2

M6

M3

M4

M7

M8

A2:

B2:

P1 P2

MoJo(P1) = MoJo(P2) = 87.5%

PR(P1) = PR(P2) = P:84.6%, R:68.7%, AVGPR=76.7%

Conclusion… P1 is equally similar to P2


Similarity Measurement RecapM1

M5

M2

M6

M3

M4

M7

M8

M1

M5

M2

M6

M3

M4

M7

M8

A1:

B1:

M1

M5

M2

M6

M3

M4

M7

M8

M1

M5

M2

M6

M3

M4

M7

M8

A2:

B2:

P1 P2

EdgeSim(P1)=77.8% EdgeSim(P2)=58.3%

MeCl(P1)=88.9% MeCl(P2)=66.7%

Conclusion… P1 is more similar than P2


Summary:EdgeSim & MeCl

EdgeSim:n Rewards clustering algorithms for

preserving the edge typesn Penalizes clustering algorithms for

changing the edge types

MeCl:n Rewards the clustering algorithm for

creating cohesive “subclusters”


Special Modules

i j

k la b

c

d e

f g

h

c

f

g

a

b

e

h

k

l

d

i

j

PA

PB

A1 A3A2

B1 B2

Omnipresent Modules:“Strong” Connection to other Modules

Library Modules:Always used by other modules, never use other modules

Isomorphic Modules:Modules equally connected to other subsystems


Special Modules

i

k la b

c f g

h

c

f

g

a

b

h

k

l

i

PA

PB

A1 A3A2

B1 B2

k

k

Omnipresent Modules:Removed

Library Modules:Removed

Isomorphic Modules:Replicated

Special Treatment of Special Treatment of Special Modules helps to Special Modules helps to determine the Similaritydetermine the Similarity

d

d


Case Study Overview

Clustering Algorithms

Similarity Evaluation Tool

Similarity Analysis

Precision/Recall

Source Codevoid main(){printf(“hello”);

}

Clustered Result

M1

M2

M3

M5M4

M6

M7 M8

MoJo

EdgeSim

MeClAverage, Variance, etc. based on 100 clustering runs…(4950 Evaluations)


Case Study ObservationsAll similarity measurements exhibit consistent behavior for the systems studied

Removal of “special” modules improved all similarity measurements Treating isomorphic modules specially only improved similarity slightlyEdgeSim and MeCl produced higher and less variable similarity values then Precision/Recall and MoJo

For all systems examined:If MeCl(SA) < MeCl(SB) then MoJo(SA) < MoJo(SB), PR(SA) < PR(SB), and EdgeSim(SA) < EdgeSim(SB)


Questions

Special Thanks To:n AT&T Researchn Sun Microsystemsn DARPAn NSFn US Army

Documents

Comparing the Decompositions Produced by Software ...bmitchell/research/icsm01pres.pdf · Produced by Software Clustering Algorithms using Similarity Measurements ... M1 M5 M2 M6