1. Software Evolution and Reengineering 2. Model Capture and …labs.cs.upt.ro/~oose/uploads/SQA/sqa-chapter5.pdf · 5. Refactoring and Restructuring Dr. Radu Marinescu 3 Textbooks

Dr. Radu Marinescu 1

Quality Assurance and Software Evolution


Course outline

1. Software Evolution and Reengineering

2. Model Capture and Design Extraction

3. Object-Oriented Harmony ... and Its Disharmonies

4. Detecting Disharmonies

5. Refactoring and Restructuring


Textbooks


1. Introduction

! Goals

! Terminology

! Why Reengineering ?

! Object-Oriented Legacy

! Lehman's Laws

! Typical Problems

! common symptoms

! architectural problems & refactorings opportunities

! Reverse and Reengineering

! Techniques

! Patterns


Goals of this course

We will try to convince you:

! There are object-oriented legacy systems too!

! Reverse engineering and reengineering are essential activities in the lifecycle

of any successful software system.

! And especially OO ones!

! There is a large set of lightweight tools and techniques to help you with the

quality assessment and the evolution of your software.

! Despite these tools and techniques, people must do the job and they

represent the most valuable resource.


What is a Legacy System ?

legacy

A sum of money, or a specified article, given to another by will;

anything handed down by an ancestor or predecessor. — Oxford

English Dictionary

! so, further evolution and development may be prohibitively expensive

A legacy system is a piece of

software that:

• you have inherited, and

• is valuable to

you.

Typical problems with legacy systems

are:

• original developers no longer available

• outdated development methods used

• extensive patches and modifications

• missing or outdated documentation


Software Maintenance - Cost

requirementdesign

codingtesting

delivery

x 1

x 5

x 10

x 20

x 200Relative Maintenance

Effort

Between 50% and 75%

of global effort is spent

on maintenance !

Relative Cost

of Fixing Mistakes

Solution ?

• Better requirements engineering

• Better software methods & tools(database schemas, CASE-tools, objects,

components, …)


Requirements Engineering ?

17.4% Corrective

(fixing reported errors)

18.2% Adaptive

(new platforms or OS)

60.3% Perfective

(new functionality)

The bulk of the maintenance cost is due to new functionality

! even with better requirements, it is hard to predict new functions

4.1% Other


Lehman's Laws

A classic study by Lehman and Belady [Lehm85a] identified several “laws” of system change.

Continuous Change

! A program that is used in a real-world environment must change, or become progressively less useful in that environment.

Increasing complexity

! As a program evolves, it becomes more complex, and extra resources are needed to preserve and simplify its structure.

These laws are still applicable…


! they do not come for free

What about Objects ?

Object-oriented legacy systems

! = successful OO systems whose architecture and design no longer responds

to changing requirements

Compared to traditional legacy systems

! The symptoms and the source of the problems are the same

! ravioli code instead of spaghetti code ;)

! The technical details and solutions may differ

OO techniques promise better

! flexibility,

! reusability,

! maintainability

! …


What about Components ?

Components are very "fragile" …

After a while one inevitably resorts to glue :)


How to deal with Legacy ?

New or changing requirements will gradually degrade original design

… unless extra development effort is spent to adapt the structure

New Functionality

Hack it in?

• duplicated code

• complex conditionals

• abusive inheritance

• large classes/methods

First …

• refactor

• restructure

• reengineer

Take a loan on your software

! pay back via reengineering

Investment for the future

! paid back during maintenance

Y N


Common Symptoms

Lack of Knowledge

! obsolete or no documentation

! departure of the original developers or users

! disappearance of inside knowledge about the system

! limited understanding of entire system

! missing tests

Process symptoms

! too long to turn things over to production! simple changes take too long

! need for constant bug fixes

! maintenance dependencies

! difficulties separating products

Code symptoms

• big build times

• duplicated code

• code smells


Common Problems

Architectural Problems! insufficient documentation

! non-existent or out-of-date

! improper layering! too few are too many layers

! lack of modularity! strong coupling

! duplicated functionality! similar functionality by separate

teams

Refactoring opportunities! misuse of inheritance

! code reuse vs polymorphism

! missing inheritance! duplication, case-

statements

! misplaced operations! operations outside classes

! violation of encapsulation! type-casting; C++

"friends"

! class abuse! classes as namespaces

! duplicated code! copy, paste & edit code


Some Terminology

“Forward Engineering is the traditional process of moving from high-level

abstractions and logical, implementation-independent designs to the physical

implementation of a system.”

“Reverse Engineering is the process of analyzing a subject system to identify the

system’s components and their interrelationships and create representations of

the system in another form or at a higher level of abstraction.”

“Reengineering ... is the examination and alteration of a subject system to

reconstitute it in a new form and the subsequent implementation of the new

form.”

— Chikofsky and Cross [in Arnold, 1993]


The Reengineering Life-Cycle

Requirements

Designs

Code

(0) requirement

analysis

(1) model

capture

(2) problem

detection (3) problem

resolution

(4) program transformation

• people centric

• lightweight


Goals of Reverse Engineering! Cope with complexity

! need techniques to understand large, complex systems

! Generate alternative views! automatically generate different ways to view systems

! Recover lost information! extract what changes have been made and why

! Synthesize higher abstractions! identify latent abstractions in software

! Facilitate reuse! detect candidate reusable artifacts and components

Chikofsky and Cross [in Arnold, 1993]


Reverse Engineering Techniques

! Redocumentation

!pretty printers

!diagram generators" e.g. Together

!cross-reference listing generators" e.g. IDEA, SNiFF+, Source Navigator

! Design recovery

!software metrics

!browsers, visualization tools

!static analyzers

!dynamic (trace) analyzers


Goals of Reengineering

! Unbundling! split a monolithic system into parts that can be separately marketed

! Performance! “first do it, then do it right, then do it fast”

" experience shows this is the right sequence!

! Design refinement! to improve maintainability, portability, etc.

! Port to other Platform! the architecture must distinguish the platform dependent modules

! Exploitation of New Technology! i.e., new language features, standards, libraries, etc.


A Picture is Worth a Thousand Words...

System Complexity View of ArgoUML


FAMOOS Project

55,000

60,000

180,000

350,000

2,000,000

2,500,000

pipeline planning

user interface

embedded switching

mail sorting

network management

space mission identify components

unbundle application

portability & scalability

improve modularity

increase flexibility

extract design

FAMOOS Case studies

LOCDomain Reengineering Goal

Different reengineering goals … but common themes and problems !


Reengineering Techniques

! Restructuring!automatic conversion from unstructured to structured code!source code translation

[Chikofsky and Cross93]

! Refactoring ! renaming/moving methods/classes etc.

[Fowler99]

! Data reengineering! integrating and centralizing multiple databases

!unifying multiple, inconsistent representations!upgrading data models

[Sommerville, ch 32]


Reverse engineering Patterns

Reverse engineering patterns

! encode expertise and trade-offs in " extracting design from source code,

" running systems and

" people.

! Even if design documents exist, they are typically out of sync

with reality.

Example: Refactor to Understand


Reengineering Patterns

Reengineering patterns

! encode expertise and trade-offs in transforming legacy code to " resolve problems that have emerged.

! These problems are typically not apparent in original design but are due to architectural drift as requirements evolve

Example: Move Behaviour Close to Data


Summary

! Software “maintenance” is really continuous development

! Object-oriented software also suffers from legacy symptoms

! Reengineering goals differ; symptoms don’t

! Common, lightweight techniques can be applied to keep

software healthy


Reverse Engineering


What and Why ?

Definition

Reverse Engineering is the process of analyzing a subject system! to identify the system’s components and their interrelationships and

! create representations of the system

" in another form or

" at a higher level of abstraction.

— Chikofsky & Cross, ’90

Motivation

Understanding other people’s code" newcomers in the team,

" code reviewing

" original developers left

Generating UML diagrams is NOT reverse engineering

... but it is a valuable support tool



(0) req. analysis

(1) model capture

issues

• scale

• speed

• accuracy

• politics

Requirements

Designs

Code

(0) requirement

analysis

(1) model

capture

(2) problem

detection(3) problem

resolution



I. First Contact: the Forces

! Where Do I Start?

! Legacy systems are large and complex

!Split the system into manageable pieces

! Time is scarce

!Apply lightweight techniques to assess feasibility and risks

! First impressions are dangerous

!Always double-check your sources


First Contact

System experts

Chat with the

Maintainers

Interview

during Demo

Talk with

developers

Talk with

end users

Talk about it

Verify what

you hear

Software System

Read All the Code

in One Hour

Do a Mock

Installation

Read it Compile it

Skim the

Documentation

Read

about it


II. Initial Understanding: the Forces

! Data is deceptive

!Always double-check your sources

! Understanding entails iteration

!Plan iteration and feedback loops

! Knowledge must be shared

! “Put the map on the wall”

! Teams need to communicate

! “Use their language”


Initial Understanding

understand !

higher-level model

Top down

Speculate about Design

Recover

design

Analyze the

Persistent Data

Study the

Exceptional Entities

Recover

database

Bottom up

Identify

problems


Analyze the Persistent Data

Problem: Which objects represent valuable data?

Solution: Analyze the database schema

! Prepare Model! tables ! classes; columns ! attributes

! primary keys " naming conventions + unique indices

! foreign keys (associations between classes)" be aware of synonyms and homonyms

! Incorporate Inheritance! one to one; rolled down; rolled up

! Incorporate Associations! check for foreign keys

! association classes (e.g. many-to-many associations)

! Verification! Data samples + SQL statements


Example: One To One

Patient

id: char(5)

insuranceID: char(7)

insurance: char(5)

Salesman

id: char(5)

company: char(40)

Person

id: char(5)

name: char(40)

addresss: char(60)

Patient

id: char(5)

insuranceID:

char(7)

insurance: char(5)

Salesman

id: char(5)

company: char(40)

Person

id: char(5)

name: char(40)

addresss: char(60)


Example: Rolled Down

Patient

id: char(5)

name: char(40)

addresss: char(60)

insuranceID: char(7)

insurance: char(5)

Salesman

id: char(5)

name: char(40)

addresss: char(60)

company: char(40)

Patient

id: char(5)

insuranceID:

char(7)

insurance: char(5)

Salesman

id: char(5)

company: char(40)

Person

id: char(5)

name: char(40)

addresss: char(60)


Example: Rolled Up

Person

id: char(5)

name: char(40)

addresss: char(60)

kind: integer

insuranceID: char(7) «optional»

insurance: char(5) «optional»

company: char(40) «optional»

Patient

id: char(5)

insuranceID:

char(7)

insurance: char(5)

Salesman

id: char(5)

company: char(40)

Person

id: char(5)

name: char(40)

addresss: char(60)


Study the Exceptional Entities

Problem: How can you quickly identify design problems?

Solution: Measure software entities and study the anomalous ones

! Use simple metrics

! Visualize metrics to get an overview

! Browse the code to get insight into the anomalies


Questions

! Which tools to use?

! Which metrics to collect?

! Which thresholds to apply

! How to interpret the results?

! How to identify anomalies quickly?

! Should I trust numbers?

! What about normal entities?


Understand Code

Overview of an OO System: Easier Said Than Done :-)

Let’s play a game...

!Want brief overview of the code of an OO system never seen before

!Want to find out how hard it will be to understand the code

Metric Value

LOC 35.000

NOM 3.600

NOC 380


Now, Do We REALLY Know Something? :-)

! Is it “normal” to have...! ...380 classes in a system with 3.600 methods?

! ...3.600 methods in a system with 35.000 lines of code?

#What means NORMAL?# i.e. how do we compare with other projects?

! What about the hierarchies ? What about coupling?

1. We need means of comparison. Thus, proportions are important!

2. Collect further relevant numbers; especially coupling and use of inheritance

Several questions remain unanswered...


! A metrics-based means to both describe and characterize the structure of an object-oriented system by quantifying its:

! complexity,

! coupling and

! usage of inheritance

! Measuring these 3 aspects at system level provides a comprehensive characterization of an entire system

Understand

The Overview Pyramid [Lanza,Marinescu 2006]


Understand

Overview Pyramid: Size and Complexity

NOP = No. Of Packages

NOC = No. Of Classes

NOM = No. Of Methods

LOC = Lines Of Code

CYCLO = Cyclomatic Number (summed up over all methods)


NOM = No. Of Methods

CALLS = No. Of Calls

FANOUT = No. Of Called Classes

Understand

Overview Pyramid: Coupling


ANDC = Average Number of Derived Classes AHH = Average Hierarchy Height

Understand

Overview Pyramid: Inheritance


close to AVERAGE

close to HIGH

close to LOW

! Interpretation based on a statistically relevant collection of data! collected for Java and C++

! over 80 systems

Understand

Overview Pyramid: Interpretation


Visualizing the Whole System

Polymetric Views

[Lanza,Ducasse 2003]Use simple metrics and layout algorithms.

(x,y) width

height colour

Visualize up to 5 metrics per node


A Picture is Worth a Thousand Words...

System Complexity View of ArgoUML


Initial Understanding (revisited)

Top down

Speculate about Design

Analyze the

Persistent Data

Study the

Exceptional Entities

understand !

higher-level model

Bottom up

ITERATION

Recover

design

Recover

database

Identify

problems


Detailed Model Capture



! Details matter

!Pay attention to the details!

! Design remains implicit

!Record design rationale when you discover it!

! Design evolves

! Important issues are reflected in changes to the code!

! Code only exposes static structure

!Study dynamic behavior to extract detailed design



Expose the design

& make sure it stays exposed

Tie Code and Questions

Refactor to Understand

Keep track of

your understanding

Expose design

Step through the Execution

Expose collaborations

• Use Your Tools

• Look for Key Methods

• Look for Constructor Calls

• Look for Template/Hook Methods

• Look for Super Calls

Look for the Contracts

Expose contracts

Learn from the Past

Expose evolution

Write Tests

to Understand


Tie Code and Questions

Problem: How do you keep track of your understanding?

Solution: Annotate the code

! List questions, hypotheses, tasks and observations.

! Identify yourself!

! Annotate as comments, or as methods


Refactor to Understand

Problem: How do you decipher cryptic code?

Solution: Refactor it till it makes sense

! Goal (for now) is to understand, not to reengineer

! Work with a copy of the code

! Refactoring requires an adequate test base

! If this is missing, Write Tests to Understand

! ...and tool support

! automatic refactorings

! Hints:

! Rename attributes to convey roles

! Rename methods and classes to reveal intent

! Remove duplicated code

! Replace condition branches by methods

! Define method bodies with same level of abstraction

! Needs tool support!


Look for the Contracts

Problem: Which contracts does a class support?

Solution: Look for common programming idioms, i.e. look for "customs" of using the interface of that class

! Look for “key methods”! Intention-revealing names

!Key parameter types!Recurring parameter types represent temporary associations

! Look for constructor calls

! Look for Template/Hook methods

! Look for super calls

! Use your tools!


Constructor Calls: Stored Result

! Identify part-whole relationships (refining associations)

!storing result of constructor in attribute ! part-whole relation

public class Employee {

private String _name = "";

private String _address = "";

public File[ ] files = { };

…

public class File {

private String _description = "";

private String _fileID = "";

…public void createFile (int position, String description, String identification)

{

files [position] = new File (description, identification);

}

Employee

_name

_address

File

_description

_fileID

1

*


Constructor Calls: "self" Argument

public class Person {

private String _name = "";

…

public class Marriage {

private Person _husband, _wife;

public Marriage (Person husband,

Person wife) {

_husband = husband;

_wife = wife;}

…in class Person:

Marriage marryWife (Person wife) {

return new Marriage (this, wife);

}

... acts as PART

Person

_name

…

Marriage

_husband

_wife

1

1

1

1

Person

_name

…

Marriage

_husband

_wife

1 1


Hook Methodspublic class SimpleDatabase {

...

protected Table fetchTable (String tableSpec) {

//tableSpec is a filename; parse it as

//a tab-separated table representation

...};

public class ProjectDatabase

extends SimpleDataBase {

...

protected Table fetchTable (String tableSpec) {

//tableSpec is a name of an SQLTable;

//return the result of SELECT * as a table

...};

SimpleDatabase

# fetchTable(tableSpec):

Table

ProjectDatabase

Hook Method


Template / Hook Methods

public class SimpleDatabase {

...

public void generateHTML

(String tableSpec,

HTMLRenderer aRenderer,

Stream outStream) {

Table table = this.fetchTable (tableSpec);

aRenderer.render (table, outStream);}

…};

public class HTMLRenderer {

...

public void render (Table table, Stream outStream) {

//write the contents of table on the given outStream

//using appropriate HTML tags

…}

SimpleDatabase

generateHTML(String,

HTMLRenderer,

Stream)

Template Method


Learn from the Past

Problem: How did the system get the way it is?

Solution: Compare versions to discover where code was removed

! Removed functionality is a sign of design evolution

! Use or develop appropriate tools

! Look for signs of:

!Unstable design — repeated growth and refactoring

!Mature design — growth, refactoring and stability


CodEVolver: The Evolution Matrix [Lanza]

Last Version

First Version

Major Leap

Removed Classes

TIME (Versions)

Growth Stagnation

Added Classes


Visualizing Classes Using Metrics

! Object-Oriented Programming is about “state”

and “behavior”:

! State is encoded using attributes

! Behavior is encoded with methods

! We visualize classes as rectangles using for width and height the following metrics:

! NOM (number of methods)

! NOA (number of attributes)

Foo

BigFoo(t)

Bar


Pulsar & Supernova

Pulsar: Repeated Modifications make it grow and shrink. System Hotspot: Every System Version requires changes.

Supernova: Sudden increase in size. Possible Reasons:• Massive shift of functionality towards a class.• Data holder class for which it is easy to grow.• Sleeper: Developers knew exactly what to fill in.


White Dwarf, Red Giant, Idle

White Dwarf: Lost the functionality it had and now trundles along without real meaning. Possibly dead code.

Red Giant: A permanent god class which is always very large.

Idle: Keeps size over several versions. Possibly dead code,possibly good code.


Dayfly & Persistent

Dayflies: Exists during only one or two versions. Perhaps an idea which was tried out and then dropped.

Persistent: Has the same lifespan as the whole system. Part of the original design. Perhaps holy dead code which no one dares to remove.


Conclusion

! Setting Direction + First Contact

! First Project Plan

! Initial Understanding + Detailed Model Capture

!Plan the work … and Work the plan

!Frequent and Short Iterations

! Issues

!scale

!speed vs. accuracy

!politics


Design Extraction


Outline

! Why Extracting Design? Why Uml?

! Basic UML Static Elements

! Interpreting UML

! Language Specific Issues

! Tracks For Extraction

! Extraction Techniques

! Conclusion


Goals of this Lecture

! Design is not code with boxes and arrows

! Design extraction is not trivial

! Design extraction should scale up

! Design extraction can be tool-supported but not fully automatize

! Give a critic view on hype: “we read your code and produce design”

Dr. Radu Marinescu

Why is Design Extraction Needed?

! Documentation inexistent, obsolete or too verbose

! Abstraction needed to understand applications

! Original programmers left

! Only the code is available

! Why UML?

!Standard" communication based on a common language

" can support documentation if we are precise about its interpretation

!Extensible" by means of stereotypes

69 Dr. Radu Marinescu

UML (Unified Modeling Language)

! successor of OOA&D methods of the late 80’s and early 90’s

! Unified Booch, Rumbaugh (OMT) and Jacobson

!standardized by OMG

! UML is a modeling language not a process

! UML defines:

!a notation" syntax of the modeling language

!a meta-model (eMOF)" a model that defines the “semantics” of a model

70


Outline


! Basic UML Static Elements

! Interpreting UML

! Language Specific Issues



! Conclusion


The Little Static UML



Requirements

Designs

Code

(0) requirement

analysis

(1) model

capture

(2b) problem


resolution

(4) Code Transformation

(1) model capture

(2) reverse engineering

issues

• Scale

• Beyond boxes and arrows

(2a) Reverse

Engineering


War story: “Company X is in trouble.”

! Their product is successful (they have 60% of the world

market).

! But:

!all the original developers left,

! there is no documentation at all,

! there is no comment in the code,

! the few comments are obsolete,

! there is no architectural description,...

! ... and they must change the product to take into account new

client requirements


Let’s Practice


Evaluation

! We should have heuristics to extract the design.

! Try to clean the previous solution you found

! Try some heuristics like removing:

!private information,

! remove association with non domain entities,

!simple constructors,

!destructors, operators


A Cleaner View


Roadmap


! Basic Uml Static Elements

! Interpreting UML



! Conclusion


Three Essential Questions

When we extract design we should be precise about:

1. What are we talking about? Design or implementation?

2. What are the conventions of interpretation that we are applying?

3. What is our goal? ! documentation (programmers)

! framework (users)! high-level views


Levels of Interpretation: Perspectives

! Fowler proposed 3 levels of interpretations called perspectives: ! conceptual

! specification

! implementation

! Three Perspectives:! Conceptual

" we draw a diagram that represents the concepts that are somehow related to the classes but there is often no direct mapping.

" "Essential perspective"

! Specification " we are looking at interfaces of software not implementation" types rather than classes. Types represent interfaces that may have many

implementations

! Implementation" implementation classes


Attributes in Perspective

! Syntax: ! visibility attributeName: attributeType = defaultValue

! Example: + name: String

! Conceptual: ! Customer name = Customer has a name

! Specification: ! Customer class is responsible to propose some way to query and set

the name

! Implementation: ! Customer has an attribute that represents its name

! Possible Refinements: Attribute Qualification ! Immutable: Value never change

! Read-only: Clients cannot change it


Operations in Perspective

! Syntax: ! visibility name (parameter-list) : return-type

! Ex: + public, # protected, - private

! Conceptual: ! principal functionality of the object. It is often described as a sentence

! Specification: ! public methods on a type

! Implementation: ! methods

! Possible Refinements: Method qualification: ! Query (does not change the state of an object)

! Cache (does cache the result of a computation),

! Derived Value (depends on the value of other values),

! Getter, Setter


Associations

Represent relationships between instances

! Each association has two roles: each role is a direction on the

association.

! a role can be explicitly named, labeled near the target class

! if not named from the target class and goes from a source class to a

target class

! a role has a multiplicity: 1, 0, 1..*, 4

! LineItems = " role of direction Order to OrderLines

! LineItems role = OrderLine role

! One Order has several OrderLines


Associations: Conceptual Perspective

Associations represent conceptual relationships between classes! An Order has to come from a single Customer.

! A Customer may make several Orders.

! Each Order has several OrderLines that refers to a single Product.

! A single Product may be referred to by several OrderLines.


Associations: Specification Perspective

Associations represent responsibilities! One or more methods of Customer should tell what Orders a given Customer

has made.

! Methods within Order will let me know which Customer placed a given Order and what Line Items compose an Order

Associations also implies responsibilities for updating the relationship, like:! specifying the Customer in the constructor for the Order

! add/removeOrder methods associated with Customer


Associations: Implementation Perspective

Different ways to implement an association

!class attribute, local variable, parameters

!collections

Implementation of aggregation and composition

!part-of relationship

!composition – contained objects are deleted/copied when parent object is deleted/copied


Arrows: NavigabilityConceptual

! no real sense

Specification! responsibility

" an Order has the responsibility to tell which Customer it is for but Customer don’t

Implementation ! dependencies

" an Order points to a Customer, an Customer doesn’t


Generalization

! Conceptual:! What is true for an instance of a supertype is true for a subtype.

" Corporate Customer is a Customer

! Specifications:! Interface of a subtype must include all elements from the interface of a

superclass

! Implementation:! Generalization semantics is not inheritance.

! But we should interpret it this way for representing extracted code.


Roadmap


! Basic Uml Static Elements

! Experimenting With Extraction

! Interpreting Uml


! Extracting of Intention

! Extraction of Interaction

! Conclusion


Association Extractions

Goal: Explicit references to domain classes

! Distinguish associations from attributes! Qualify as attributes only implementation attributes that are not

related to domain objects.

! Value objects -> attributes and not associations,

! Object by references -> associations" String name -- an attribute

" Order order -- an association

" Piece myPiece (in C++) -- composition

! Two classes possessing attributes on each other! an association with navigability at both ends


Association Extraction: Language Impact

! Attributes interpretation

! In C++ " Piece* myPiece " aggregation or association

" Piece& my Piece " aggregation or association

" Piece myPiece (copied so not shared) " composition

! In Java, C#

! Aggregation and composition is not easy to extract

" Piece myPiece " association or aggregation


Operation Extraction (1)

! You may not extract! accessor methods with the name of an attribute

! operators, non-public methods,

! simple instance creation methods " constructor with no parameters in Java

! methods already defined in superclass," they are inherited

! methods that are responsible for the initialization, printing of the objects

! Use company conventions to filter! Access to database,

! Calls for the UI,

! Naming patterns


Operation Extraction (2)

! If there are several methods with more or less the same intent

! If you want to know that the functionality exists, and not all the details! select the method with the smallest prefix

! If you want to know all the possibilities, but not all the ways you can invoke them! select the method with the more parameters

! If you want to focus on important methods! categorize methods according to the number of time they are referenced by clients

! Counterexample: a hook method is not often called but still important

! What is important to show: the creation Interface

! Non default constructors in Java or C++


Design Patterns as Documentation Elements

! Design Patterns reveal the intent ! so they are definitively appealing for supporting

documentation

! But...! Difficult to identify design patterns from the code

" What is the difference between a State and a Strategy from the code p.o.v

! Need somebody that knows

! Lack of support for code annotation so difficult to keep the use of patterns and the code evolution


Documenting Dynamic Behaviour

! Focusing only at static element structural elements (class, attribute, method) is limited, does not support: ! protocols description (message A call message B)

! describe the role that a class may play e.g., a mediator

! Calling relationships is well suited for ! method interrelationships

! class interrelationships

! UML proposes Interaction Diagrams ! Sequence Diagram or Collaboration Diagram


Sequence Diagrams

! A sequence diagram depicts a scenario by showing the interactions among a set of objects in temporal order.

! Objects (not classes!) are shown as vertical bars.

! Events or message dispatches are shown as horizontal (or slanted) arrows from the send to the receiver.

! Recall that a scenario describes a typical example of a use case, so conditionality is not expressed!

caller lifts receiver

dial tone begins

dial (1)

dial tone ends

dial (2)

dial (2)

ringing tone phone rings

answer phone

ringing stopstone stops

Caller Phone Line Callee


Statically Extracting Interactions

! Pros:

!Limited resources needed

!Do not require code instrumentation

! Cons:

!Need a good understanding of the system " state of the objects for conditional

" compilation state #ifdef...

" dynamic creation of objects

! Potential behavior not the real behavior

!Blur important scenario


Dynamically Extracting Interactions

! Pros:! Help to focus on a specific scenario

! Can be applied without deep understanding of the system

! Cons: ! Need reflective language support

" message passing control or code instrumentation (heavy)

! Storing retrieved information " may be huge

! For dealing with the huge amount of information! selection of the parts of the system that should be extracted, selection of the

functionality

! selection of the use cases

! filters should be defined " several classes as the same, several instance as the same...

! A simple approach:! open a special debugger that generates specific traces


Detecting Design Problems using Metrics


! Functions, that assign a precise numerical

value to

!Products (Software)

!Resources (Staff, Tools, Hardware)

!Processes (Software development).

What are Metrics?

Let’s see some examples...

!Definition: Let G(N, E) be a control flow graph (CFG). Cyclomatic complexity is defined as

v(G) = e – n + 2

where e = |E| and n = |N|.


Example 1: Cyclomatic Complexity [McCabe, 1976]

! Interpretation:

more cyclomatic complexity in a given function $ more branching $ Problems with clearness


0

1

2

3

4

5

6

7

8

10

9

11

12

13

Cyclomatic Complexity for Euclid algorithm

int euclid(int m, int n)

{ /*0*/

int r;

if(n > m) /*1*/

{

r = m; m = n; n = r; /*2,3,4:*/

} /*5*/

r = m % n; /*6*/

while(r != 0) /*7*/ {

m = n; n = r; r = m % n; /*8,9,10:*/

} /*11*/

return r; /*12*/

} /*13*/

Function entry

Function exit

Extra-nodes:exit from structured statement

v(G) = 15 – 14 + 2 = 3

3

01

4

12

5

11

6

7

2

8

9

10

13

14


Example 2: Weighted Method Count [Chidamber&Kemerer, 1994]

! Definition:

where ci = complexity of method mi

! Interpretation:

!High WMC # increased time and effort for maintenance

!High WMC # reduces the reuse probability for the class

BAD: Interpretation can’t directly lead to improvement action!

GOOD: Metric is configurable!


Example 3: Depth of Inheritance Tree [Chidamber & Kemerer, 1994]

! Definition:

DIT = depth of a class in the inheritance graph

! Interpretation:

!higher DIT # decreased understandability of the class

!higher DIT # class is more complex

!higher DIT # higher the potential reuse from the

BAD: only the potential and not the real impact is quantified

GOOD: Inheritance is measured


Example 4: Coupling Between Objects [Chidamber & Kemerer, 1994]

! Definition:

!CBO = number of classes to which a class is coupled

!coupling means calling methods and accessing

attributes of another class

! Interpretation:! higher CBO # design is not modular

! higher CBO # testing of a class is more complex

! higher CBO # class is probably more central in the economy of the system

BAD: Does not differentiate neither between types nor intensity of coupling

GOOD: Takes into account real dependencies not just declared ones!


! Definition: ! the relative number of method-pairs that access an attribute of the

class

! Example:

! Interpretation:!higher TCC # tighter the connection between the methods

! lower TCC # probably class implements more than one functionality

Example 5: Tight Class Cohesion [Bieman&Kang, 1995]

TCC = 2 / 10 = 0.2

GOOD: Interpretation can lead to improvement action!

GOOD: Ratio values allow comparison between systems!


Metrics in Quality Models

Quality Models to correlate quality criteria with

concrete measures

Factor-Criteria-Metricsmodel

[J.A. McCall, 1977]


REALLY ?!

Metrics Assess and Improve Quality!


Problem I: Mapping between Criteria and Metrics

How are Criteria Mapped onto Metrics?... based on principles, rules and heuristics of “good style” in design

Structuredness

Number ofMethods

Number ofParents

MethodLength

?

In FCM models the design principles, rules and heuristics that drive the mapping of criteria onto metrics are implicit !

# FCM models are hard to define and understand ... because we reason in terms of principles rather than numbers ... because we can only guess the principles used in the model


Problem II: Identify causes of poor quality

What do we expect from quality models?

- diagnosis, location and treatment hints for quality problems

In FCM models it is hard to find the proper treatement for quality problems because abnormal metric values are symptoms and not

causes of poor quality

MethodLength

Structuredness

Number ofMethods

Number ofParents

89 methods?

FCM locates classes and methods with abnormal metric values


Obstacles in Using Metrics

! Interpretation of metrics is hard

! issue of thresholds" need statistical data

!many confusing and redundant definitions hard to compare the results

" normalize!

! Applying metrics is hard!issue of granularity

" metrics need to be used in combination" quality models

" detection strategies

" polymetric views


Can Metrics Help Me ...

... in what I really care for? :->


What Is Interesting for Me as a Developer/Designer?

! Understand the Code!e.g. „insourced“ code!you are relocated to a new team

! Improve the Code !e.g. refactor the design to make it portable

!e.g. make my subsystem more flexible to a change of requirements

! Improve Myself! to know how and when to apply in a concrete case OO design

principles! ...and how to come out of OO design pitfalls

I want to have NOTHING TO DO with metrics! ;-)


close to AVERAGE

close to HIGH

close to LOW

! Interpretation based on a statistically relevant collection of data! collected for Java and C++

! over 80 systems

Understand

Overview Pyramid: Interpretation


Understand

Quickly “Reading” Classes [Lanza,Ducasse 2001]

! Visualization Technique!serves as code inspection technique! reduces the amount of code that must be read


Understand

Class Blueprint


Understand

Class Blueprint Example


Lesson Learnt...

HIDE METICS!


Where are We?

Can Metrics Help Me ...

in what I really care for?

! Understand the Code

! Overview Pyramid

! Evaluate & Improve Design

! Detection Strategies

! Improve Myself

! Continuous QA


Evaluate & Improve

Design Problems? Why Should I Care?

! Design problems are frequent

! legacy systems with high business value

!must maintain and enhance the system

! Design problems are expensive

!high effort required for maintenance and extension

! Design problems will always be there!

!at least because of time pressure.

!…but I believe also because of changing requirements


...you would change just a small design fragment and suddenly...

...33% of all the classes in the

system may require changes?!

Just Imagine that...


But there is hope!!Design rules, guidelines and heuristics

"Apply them and they will provide the desired quality"Break them and they will break your design!

Is There Any Hope?

“There is no silver bullet!” [Brooks84]

!Encapsulation? Inheritance? Polymorphism?

!Generated code? Powerful IDEs?

!NO! These are just mechanisms and tools

!Like in chess..."… just knowing the pieces and the moves is not enough


Evaluate & Improve

Structural Evaluation and Improvement

! Structural problems, not bugs!!analyze static structure, not dynamic behavior!Don’t need compile-ready code

! Deviations from high-quality design!e.g. “a class with 100 methods is probably a to complex abstraction and

it should be split”

! Design rules, guidelines and heuristics!Apply them and they will lead you to the desired design quality!Break them and they will break your design!

How to control when these design rules are broken?!

Use metrics, but in higher-level mechanisms

Dr. Radu Marinescu

Design rule example: God Class is bad!

! “God Classes tend to centralize the intelligence of the system,

doing ‘everything’ and using data from small data-classes” [Riel

1996]

! It generates problems:

!Maintainability, Reusability, Understandability

! Detection technique:

!Search for classes that access data from other classes

!From these classes look for complex and low cohesive classes

124

How to detect when a class is a God Class?!

Use multiple metrics, composed by logical operators


Metrics should be used in a goal-oriented fashion

! Define a Goal !How efficient is the ACME tool

! Formulate Questions!Who uses the ACME tool?

!How high is productivity/quality with/without the ACME tool?

! Find suitable Metrics!Percent of developers that use the ACME tool

!Experience with ACME

!Size, complexity, solidity , … of code

Goal-Question-Metric Approach [Basili&Rombach, 1988]

Dr. Radu Marinescu

Evaluate & Improve

Applying GQM to detect God Classes

DetectionTechnique Metrics-

basedRule

InformalRules

(design-related)

•Top-level classes in a design should

share work uniformly• Beware of classes with much

non-communicative behavior• Beware of classes that access directly data

from other classes

• large and complex classes• non-cohesive• access “foreign” data

WMCHigherEqual(50) and

TCCLessThan (0.33) and

ATFDHigherThan(2)

Select MetricsAnalysis

126


Evaluate & Improve

Detection Strategy is a…

.... generic mechanism for higher level metrics interpretation

!Based on filtering and composition

!Allows to define metrics-based rules to relate to design

FilteringReturn a subset of data based on thresholds

MetricsMeasure Composition

Compose metrics in a high level rule

Dr. Radu Marinescu

A God Class centralizes too much intelligence

ATFD > FEW

Class uses directly more than a

few attributes of other classes

WMC ! VERY HIGH

Functional complexity of the

class is very high

TCC < ONE THIRD

Class cohesion is low

AND GodClass

128

Dr. Radu Marinescu

An Envious Method is more interested in data from a handful of classes.

ATFD > FEW

Method uses directly more than

a few attributes of other classes

LAA < ONE THIRD

Method uses far more attributes

of other classes than its own

FDP ! FEW

The used "foreign" attributes

belong to very few other classes

AND Feature Envy

129 Dr. Radu Marinescu

Data Classes are dumb data holders.

WOC < ONE THIRD

Interface of class reveals data

rather than offering services

AND Data Class

Class reveals many attributes and is

not complex

130

Dr. Radu Marinescu

Data Classes are dumb data holders.

AND

OR

Class reveals many

attributes and is not

complex

NOAP + NOAM > FEW

More than a few public

data

WMC < HIGH

Complexity of class is not

high

NOAP + NOAM > MANY

Class has many public

data

WMC < VERY HIGH

Complexity of class is not

very high

AND

131 Dr. Radu Marinescu 132

Evaluate & Improve

Detectable Design Problems (to be extended)

[Lanza, Marinescu 2006]

5.3 God Class 81

ATFD > FEW

Class uses directly more than a

few attributes of other classes

WMC ! VERY HIGH

Functional complexity of the

class is very high

TCC < ONE THIRD

Class cohesion is low

AND GodClass

Fig. 5.2. The God Class detection strategy

1. Class uses directly more than a few attributes of other classes.Since ATFD measures how many foreign attributes are used bythe class, it is clear that the higher the ATFD value for a class, thehigher is the probability that a class is (or is about to become) aGod Class.

2. Functional complexity of the class is very high. This is ex-pressed using the WMC (Weighted Method Count) metric.

3. Class cohesion is low. As a God Class performs several distinctfunctionalities involving disjunct sets of attributes, this has a neg-ative impact on the class’s cohesion. The threshold indicates thatin the detected classes less than one-third of the method pairshave in common the usage of the same attribute.

The general design of ArgoUML is good enough so that we could not Exampleidentify a pure God Class i.e., a class controlling the flow of the appli-cation and concentrating all the crucial behavior, which would indi-cate a clear lack of object-oriented design. However, certain classes inArgoUML acts as a black hole attracting orphan functionalities. Suchclasses are also detected by the metrics presented above and are stilla design problem. A class of ArgoUML which clearly stands out is thehuge class ModelFacade (see Fig. 3.12). This class implements 453

Dr. Radu Marinescu

Evaluate & Improve

Meaningful Thresholds in Detection Strategies

133 Dr. Radu Marinescu 134

Evaluate & Improve

A Catalogue of Design “Disharmonies”

! For each design problem,

we provide

!Description

!Context

! Impact

!Detection Strategy

!Examples

!Refactoring


Learn

How Do I Improve Myself ?

Follow a clear and repeatable process...

Reason quality in terms of principles, patterns,design rules... not numbers!


Learn

Integrate QA in the Development Process

IBM Eclipse Innovation Award 2006


Learn

Even Quality Models Can Help!

Lack ofBridge

Data Classes

God Classes

Interface Segregat. Pr.

WMC

TCC

CIW

NOAM

AUF

NOPA

COC

ATFD

NOD

LR

Principles, rules, heuristicsquantified in Detection Strategies

Qualitydecomposed in Factors

communicates with

Factor-Strategy Q.M. [Marinescu,2004]


Instead of Conclusions...

DISCLAIMER:Metrics are not enough to understand and evaluate design!

Can we understand the beauty of a painting by… … measuring its frame or counting the colors ?

Object-Oriented Reengineering

© S. Demeyer, S.Ducasse, O. NierstraszLecture 8

Radu Marinescu

139

Quality-Driven Code Restructuring


Refactoring

Refactoring

! What is it?

! Why is it necessary?

! Examples

! Tool support

Refactoring Strategy

! Code Smells

! Examples of Cure

Conclusions

! Obstacle-driven Conclusions



Requirements

Designs

Code

(0) requirement

analysis

(1) model

capture

(2) problem


resolution



issues

• Tool support

• Failure proof


What is Refactoring?

The process of changing a software system in such a way that it does not alter the external behaviour of the code, yet improves its internal structure [Fowl99a]

A change to the system that leaves its behaviour unchanged, but enhances some non-functional quality - simplicity, flexibility, understandability, ... [Beck99a]


Typical Refactorings

Class Refactorings Method Refactorings Attribute Refactorings

add (sub)class to hierarchy

add method to class add variable to class

rename class rename method rename variable

remove class remove method remove variable

push method down push variable down

push method up push variable up

add parameter to method create accessors

move method to component

extract code in new method

These simple refactorings can be combined to provide bigger restructurings

such as the introduction of design patterns.


Why Refactoring?

Some argue that good design does not lead to code needing refactoring,

But in reality! Extremely difficult to get the design right the first time

! You cannot fully understand the problem domain

! You cannot really plan how the system will evolve in five years

! System becomes difficult to change

Refactoring helps you to

! Manipulate code in a safe environment (behavior preserving)

! Understand existing code

Grow, don’t build softwareF.P. Brooks jr


Refactoring and OO

Object-Oriented Programming

! emphasize the possibility of changes

! rapid development cycle

! incremental definition

However software evolves, grows and... dies if not taken care of

=> This is where Refactoring comes in


Rename Method: Do It Yourself


Rename Method

Rename Method (method, new_name)

Preconditions! no method exists with the signature implied by new_name in the inheritance

hierarchy that contains method

! [Java, C++] method is not a constructor

Postconditions! method has new_name

! relevant methods in the inheritance hierarchy have new_name

! invocations of changed method are updated to new_name

Other Considerations! Scope of the renaming


Which Refactoring Tools?

Refactoring

! Source-to-source program transformation

! Behaviour preserving

=> improve the program structure

Programming Environment

! Fast edit-compile-run cycles

! Integrated into your environment

! Support small-scale reverse engineering activities

=> convenient for “local” ameliorations

Regression Testing

! Repeating past tests! Tests require no user interaction

! Answer per test is yes / no

=> verify if improved structure does not damage previous work

Configuration & Version Management

! keep track of versions that represent project milestones

=> possibility to go back to previous version

Change Efficient Failure Proof


Conclusion: Tool Support

C++ Java

refactoring tools - (?) +

rapid edit-compile-run cycles - +-

reverse engineering facilities +- +-

regression testing + +

version & configuration management + +


Obstacles to Refactoring

Complexity• Changing design is hard

• Understanding code is hard

Possibility to introduce errors• Run tests if possible

• Build tests

Clean first Then add new functionality

Cultural Issues• Producing negative lines of code, what an idea!

• “We pay you to add new features, not to improve the code!”

• If it ain’t broke, don’t fix it• “We do not have a problem, this is our software!“


Obstacles to Refactoring

! Performance

• Refactoring may slow down the execution

• The secret to write fast software: Write tunable software first then tune it• Typically only 10% of your system consumes 90% of the resources so just

focus on 10 %.

• Refactorings help to localize the part that need change

• Refactorings help to concentrate the optimizations

! Development is always under time pressure

• Refactoring takes time

• Refactoring better right after a software release


Conclusion: Know-when & Know-how

! Know-when is as important as know-how

• Refactored designs are more complex

• Use “code smells” as symptoms

• Rule of the thumb: “Once and Only Once” (Kent Beck)

=> a thing stated more than once implies refactoring


Further Information

More about code smells and refactoring

• Book on refactoring [Fowl99a]http://www.refactoring.com

• Discussion site on code smellshttp://c2.com/cgi/wiki?CodeSmell

The presented tools

• Refactoring Browser (in VisualWorks Smalltalk)http://www.cincom.com/smalltalk(also available in several other Smalltalks)

• Java Refactorings in Eclipsehttp://www.eclipse.org(also available in other Java environments)


Curing Duplicated Code

In the same class

• Extract Method

Between two sibling subclasses

• Extract Method

• Push identical methods up to common superclass

• Form Template Method

Between unrelated class

• Create common superclass

• Extract Component (e.g., Strategy)

Say everything exactly onceKent Beck


Design

ConfigFile

Process

CsServer UCDLink EventReceiver TrapFilter

SNMPCollector

calls


Repeated Functionality

Subclasses of Process all contain repeated Code


Template Method Pattern

ClasstemplateMethod()

hookMethod1()

hookMethod2()

//Some common codehookMethod1();//Some more common codehookMethod2();//Even more common code

Either abstract or default implementation

SpecializedClass

hookMethod1()

hookMethod2()

Specialized behaviour,if needed


ConfigurableProcess

ReadConfigFiles()

hookMethod()

EventReceiver

hookMethod()

CsServer

if (statusOfProcess == Running) { throw ConfigFile::ProcessRunning(); } if ((statusOfProcess == Idle) || (statusOfProcess == Paused)) { hookMethod(); statusOfProcess = Configured; } else { string err("Config File already read!!"); throw ConfigFile::ConfigurationFileError(err.c_str()); }

if (pAffacade->ReadConfigs() != true){ pAffacade->ClearConfigs(); throw ConfigFile::ConfigurationFileError(); };


Process

ConfigurableProcess

UCDLink EventReceiver TrapFilter

SNMPCollector

CsServer

Duplication Resolved


Restructuring

! Most common situations

! Transform Conditionals to Polymorphism

! Transform Self Type Checks

! Transform Provider Type Checks


Forces

! Requirements change! so new classes and new method will have to be introduced

! Conditionals group all the variant in one place,! but make the change difficult

! Conditionals clutter logic

! Editing several classes and fixing case statements to introduce a new behavior is error prone


Overview

! Transform Self Type Checks ! eliminates conditionals over type information in a provider by introducing

new subclasses

! Transform Client Checks ! eliminates conditionals over client type information by introducing new

method to each provider classes

! Factor out State ! kind of Self Type Check

! Factor out Strategy ! kind of Self Type Check

! Introduce Null Object ! eliminates null test by introducing a Null Object

! Transform Conditionals into Registration! eliminates conditional by using a registration mechanism


Transform Self Type Checks

! Symptoms

!Simple extensions require many changes in conditional code

!Subclassing impossible without duplicating and updating conditional code

!Adding new case to conditional code

Am()

Client …case Text: this.doSomething()case Border: this.doOther()case D:


Detection

! Long methods with complex decision logic

! Look for attribute set in constructors but never changed

! Attributes to model type or finite set constants

! look for "enums" and attributes with type-related names

! Multiple methods switch on the same attribute

! grep switch ‘find . -name “*.cxx” -print’


Transformation


Pros/Cons/Difficulties

! Pros

! New behavior is easy to add and to understand" a new class

! No need to change different method to add a behavior

! All behaviors share a common interface

! Cons

! Behavior are dispersed into multiple but related abstractions

! More classes

! Difficulties

! Not always one to one mapping between cases and subclasses

! Clients may be changed to create instance of the right subclass" ...buy creational patterns might help


Transform Client Type Checks

! Clients explicit type checks

! Adding a new provider requires to change all the clients

! Clients are defining logic about providers

Ainit()

Clienta : Am()

switch (a.class)case B: a.init(); ((B) a).x();case C: a.init(); ((C)) a).y();Case D: ((D) a).z()

Bx()

Cinit()y()

Dz()

Dr. Radu Marinescu

Transformation


Detection

! Changing clients of method when new case added

! Attribute representing a type

! In Java: instanceof

!x.getClass() == y.getClass()

!x.getClass().getName().equals(….)



! Pros!The provider offers now a polymorphic interface that can be used

by other clients!A class represent one case!Clients are not responsible of provider logic

!Adding new case does not impact all clients

! Cons!Behavior is not grouped per method but per class

! Difficulties!Refactor the clients (Deprecate Obsolete Interfaces)

! Instance creation should not be a problem


When the Legacy Solution is the Solution

! Abstract Factory may need to check a type variable to know

which class to instantiate.

!For example streaming objects from a text file requires to know the type of the streamed object to recreate it

! If provider hierarchy is frozen

!Wrapping the classes could be a good migration strategies)

! Software that interfaces with non-oo libraries

!switch to simulate polymorphic calls


Factor Out Strategy

! Problem: How do you make a class whose behavior depends

on testing certain value more extensible

! Apply State Pattern

!Encapsulate the behavior and delegate using a polymorphic call


Transformation

AbstractStrategy

handleOperation()

Aoperation()

…strategy.handleOperation()…

StrategyXhandleOperation()

Aoperation()

…case X: …case Z: ….…

strategy

StrategyZhandleOperation()



! Pros!Behavior extension is well identified!Behavior using the extension is clearer

!Change behavior at run-time

! Cons!Namespace get cluterred!Yet another indirection

! Difficulties!Behavior can be difficult to convert and encapsulate (passing

parameter…)

Documents

1. Software Evolution and Reengineering 2. Model Capture and …labs.cs.upt.ro/~oose/uploads/SQA/sqa-chapter5.pdf · 5. Refactoring and Restructuring Dr. Radu Marinescu 3 Textbooks