1

Modeling Approach Comparison Criteria for

the CMA Workshop at MODELS 2012

Geri Georg1, Shaukat Ali

2, Betty H.C. Cheng

3, Benoit Combemale

4, Robert

France1, Jörg Kienzle

5, Jacques Klein

6, Philippe Lahire

7, Markus Luckey

8, Ana

Moreira9, Gunter Mussbacher

10

1Computer Science Department, Colorado State University, Fort Collins, Colorado, USA

2Simula Research Laboratory, Norway

3Dept. of Computer Science and Engineering, Michigan State University, East Lansing, Michigan, USA

4Ecole Supérieure d’Ingénieurs de Rennes, Université de Rennes, Rennes, France

5School of Computer Science, McGill University, Montreal, Canada

6Computer Science and Communications Research Unit, Université du Luxembourg, Luxembourg

7Département d’Informatique, Université de Nice - Sophia Antipolis, Nice, France 8Institut für Informatik, Universität Paderborn, Paderborn, Germany

9CITI/Departamento de Informática, FCT, Universidade Nova de Lisboa, Lisbon, Portugal

10Dept. of Systems and Computer Engineering, Carleton University, Ottawa, Canada

August, 2012

The comparison criteria described in this document represent a work-in-progress,

begun at the Bellairs AOM1 workshop in April 2011, continued at the first CMA2 workshop

in October 2011, and further discussed at the Bellairs AOM3 workshop in April/May 2012.

The purpose of the criteria is to provide a basis for understanding, analyzing, and comparing various modeling approaches. Two groups of criteria are presented: Criteria related to general

modeling dimensions (see Section 1) and those related to key modeling concepts (see Section 2).

The modeling dimensions criteria characterize a modeling approach by (i) the

development phases and activities for which it is most applicable and (ii) the languages or notations used. The key modeling concepts criteria are used to classify a modeling approach

in terms of the modeling building blocks, attributes, or qualities targeted for optimization or improvement by the approach.

These criteria are intended to play a particularly important role in situating existing

approaches in the current body of work, and in identifying the considerations that must be made when developing new modeling approaches. In order to fulfil their role, it is important

that the criteria be defined in terms that are widely and precisely understood by the modeling community. We have identified the following as central key modeling concepts: modularity,

units of encapsulation, composability, and composition operators. The key modeling concepts criteria can be applied to either an approach or to the models that are produced using the

approach. The focus of this document, however, is the assessment of modeling approaches, and not the models they produce. Furthermore, key modeling concepts can conceivably be

interpreted differently depending on which modeling dimension is being considered. While

we recognize this fact, we present the modeling dimensions and key modeling concepts in a more orthogonal view.

1 http://www.cs.mcgill.ca/~joerg/SEL/AOM_Bellairs_2011.html 2 http://cserg0.site.uottawa.ca/cma2011/ 3 http://www.cs.mcgill.ca/~joerg/SEL/AOM_Bellairs_2012.html

2

Where possible, the description of a modeling dimension or key concept contains

generally agreed-upon definitions. In cases where such definitions are not available, we have

included examples that demonstrate our intent. Each description is followed by a set of

questions that aim to assess a modeling approach against the modeling dimension or key

modeling concept.

Figure 1 Metamodel for Modeling Approaches

Figure 1 structures modeling dimensions and key modeling concepts discussed in this

document in a metamodel that can be used to characterize and instantiate modeling approaches. The main classes in the metamodel are ModelingApproach, Language,

EncapsulationUnit, and CompositionOperator. Each of the following sections and subsections

3

explains the applicable portion of the metamodel in more detail (indicated by sans serif italic font).

Several additional key modeling concepts have been identified but have not yet been

adequately discussed to be included in the metamodel. These key modeling concepts have

been placed into a “parking lot” (see Section 3). Our goal is to continue to refine this document in order to address all key modeling concepts with a similar level of detail.

1. Modeling Dimensions

1.1 General

Questions:

A. What is the name of the modeling approach?

B. Which documents related to the modeling approach (e.g., standards, publications,

technical documentation) and tools related to the modeling approach were used for this assessment? Provide references and version numbers wherever possible.

1.2 Phase/activity

Modeling approaches (ModelingApproach) may be applicable during certain phases of

software development (DevelopmentPhase, PhaseType), or for specific activities of software development (Activity, ActivityType). They may also be useful during multiple phases or for

multiple activities. For example, the early requirements phase focuses on stakeholder goals and the description and assessment of alternatives. During the architecture or high-level

design phase, an architecture is created that describes the overall system structure as well as general rules and constraints imposed on that structure. Examples of architectures are 3-tier,

layered, event-based, distributed client-server, distributed peer-to-peer, etc. Activities such as specification/modeling, analysis, validation, verification, and evolution tend to crosscut across

development phases, although their purpose and targets may differ from one phase to another.

For example, model checking can be used as a high-level design analysis technique, whereas

performance analysis may be targeted towards low-level design or even implementation

phases. Validation ensures that the system is what the stakeholder wants, while verification

ensures that the system is built as specified in an artifact. Trade-off analysis investigates the

effect of simultaneous changes in one or more key factors in a decision, design, or project [1].

Trade-off analysis offers the ability to choose between a set of alternatives, based on a set of potentially conflicting criteria.

Note that a modeling approach is applicable to a software development phase or a

specific activity only if the modeling approach directly supports the phase or activity, e.g.,

indirect support by transforming a model of the modeling approach into a different model that

does not belong to the modeling approach and is subsequently used for an activity during a phase does not count.

Furthermore, note that in the metamodel the development phases (/phase) addressed

by a modeling approach are a derived attribute, because the development phases can be deduced from the development phases of the activities of the modeling approach.

4

Questions:

A. During which software development phases is the modeling approach applicable?

� Early requirements

� Late requirements

� Architecture/High-level design

� Detailed/Low-level design

� Implementation

� Integration

� Deployment

� Other:

B. In which software development activities is the modeling approach applicable?

� Specification/Modeling – describe what is modeled and in which phase:

� Validation – describe what is validated and in which phase:

� Verification – describe what is verified and in which phase:

� Evolution – describe what is evolved and in which phase:

� Analysis – describe what is analyzed and in which phase:

� Trade-off Analysis – describe what trade-off is analyzed and in which phase:

� Other:

1.3 Language/notation

A model must be defined in a well-defined language or notation (Language). Models

may be expressed in a single, general-purpose language, but there are certainly situations that can benefit from the use of languages that are specific to the model viewpoint (e.g., a domain-

specific language to define security policies or other system-wide properties). A language for an approach may be able to express the functional elements of a model or it may be able to

specifically express quality or other non-functional attributes that are used as criteria to judge the quality of a system, rather than its specific behavior. This ability can aid the modeler

when analyzing specific quality attributes. An example of an approach for specifying and

measuring such quality attributes is one that uses the UML MARTE profile to model

performance qualities and then uses the model as a means to perform system design performance analysis.

A language of a modeling approach is based on a paradigm (SupportedParadigm),

which is defined as the theories, rules, concepts, abstractions, and practices that constrain how

activities and development artifacts are handled or represented. A paradigm imposes a

decomposition criterion, which is then used to decompose and analyze a problem into smaller

problems. Examples are the object-oriented, aspect-oriented, procedural, service-oriented,

5

component-based, feature-oriented, goal-oriented, and logic-based paradigms. The choice of

paradigm affects other concepts and dimensions, including tools. For example, in the object-

oriented paradigm, modules are classes/abstract data types, while in the procedural paradigm

modules are functional abstractions. Tools must be able to perform activities on these

modules.

A view (View, ViewType) is a projection of a model for a particular reason. For many

models, while both structural and behavioral specifications are part of the model, there can be

separate structural and behavioral views of the model, and these perspectives may be

described using different languages (e.g., the structural perspective may be described using

class diagrams while the behavioral perspective may be described using state machines or

activity diagrams). A modeling approach may also use specific languages and their associated

views for quality attributes such as performance, reliability, and safety to support focused modeling and analysis of these qualities. Furthermore, an intentional view shows the goals

that are to be achieved by a system for its stakeholders.

A language may be applicable or not applicable to a specific domain (Domain). Note

that in the metamodel the attribute for general purpose (/generalPurpose) of a language is a

derived attribute, because a language is general purpose if it does not have any supported or avoided domains (supportedDomain or avoidedDomain, respectively).

A language may use textual, or graphical, or both types of representational elements

(ConcreteSyntax).

A language may have formal semantics (Semantics), i.e., the language is based upon a

formal domain that is well-understood and that allows proofs to be performed, e.g.,

mathematical logic or Petri nets, or it is possible to map the language to mathematical logic

expressions, Petri nets, etc. A language with formal semantics is entirely expressed in

mathematical terms. Such a language is machine analyzable, but this does not mean that all

aspects of the language are machine analyzable (as there are theoretical limitations to theorem

proving and model-checking). A language with a rigorous semantics is a language with semantics expressed in a form that allows language statements to be mechanically analyzed in

a limited way (i.e., not all aspects of the language are formalized – just enough to perform the types of analysis needed). Executable modeling languages fall into this category as they can

be used to support simulations and testing, but one cannot use them to prove that all behaviors satisfy certain properties. Often, languages with a rigorous semantics are based on a well-

defined metamodel. While this is usually sufficient to qualify as being rigorous, a well-defined metamodel by itself is insufficient to qualify as being formal. Finally, a language that

is informal has none of the above characteristics (in particular, it is not machine analyzable).

Questions (Repeat B–H for each language identified in A):

A. List the language(s) used by the modeling approach:

B. What is the standardization status of the language?

� Language is a standard.

� Language is based on standard(s) – list standard(s):

� Language is not a standard and is not based on standard(s).

6

C. For which application domains is the language applicable?

� Language is specific to application domain(s) – list domain(s):

Which application domains should be avoided?

� Language is general purpose.

D. What are the paradigms supported by the language?

� Object-oriented

� Aspect-oriented

� Procedural

� Service-oriented

� Goal-oriented

� Component-based

� Feature-oriented

� Logic-based

� Other:

E. The semantics of the language is:

� Formal

� Rigorous

� None of the above (Informal)

F. If the language is formal or rigorous, it is assumed to be executable, or machine analyzable, or both. What applies to the language?

� Executable

� Analyzable – describe what can be analyzed (e.g., which quality attributes)

G. What views are supported by the language?

� Structural

� Behavioral

� Intentional

� Quality-based – describe which quality attributes are supported:

� Other:

H. What types of concrete syntax are used by the language?

� Textual

� Graphical

7

2. Key Modeling Concepts

2.1 Modularity/units of encapsulation/types of concerns

The Separation of Concerns principle states that a good way of managing the

complexity of developing an artifact associated with many different kinds of concerns, is to break it up into parts that can be handled in relative isolation. The parts may reflect one or

more concerns and thus proper separation of concerns allows developers to focus on a subset of concerns at any given time. Effective support for the Separation of Concerns principle can

lead to software systems that achieve required engineering quality factors such as robustness, adaptability, maintainability, and reusability. The ideal decision regarding which concerns

must be separated from each other requires finding the right compromise for satisfying multiple (quality) factors. A multi-dimensional decomposition, or as it is called multi-

dimensional separation of concerns [2], permits the clean separation of multiple, potentially

overlapping and interacting concerns simultaneously, with support for on-demand re-

modularization to encapsulate new concerns at any time. A language of a modeling approach,

however, is not required to support (or indeed may not need to support) the encapsulation of

crosscutting concerns through advanced decomposition and composition mechanisms, but

may rather be restricted to basic forms of composability suitable for non-crosscutting concerns.

Modularization [3] is decomposition into modules according to decomposition criteria

imposed by the paradigm. A module is a software unit of encapsulation (EncapsulationUnit) with well-defined interfaces which express the services that are provided by the module for

use by other modules [4]. A module promotes information hiding by separating its interfaces from its internal design (often referred to as its implementation). Examples of well-defined

interfaces are public/private OO-type interfaces, protocol state machines, or service-level agreements.

A language of a modeling approach encapsulates some concepts as units, in order to

easily reason about them, manipulate them, analyze them, etc. Examples of encapsulation units are activities, services, aspects, features, themes, concerns, classes, etc. For example, an

aspect is a crosscutting unit of encapsulation. A feature is (usually) a functional unit of encapsulation, as in a feature model. A concern is a functional or non-functional unit of

encapsulation. A theme [5] is an encapsulation of some functionality or crosscutting concern

in a system and is considered more general than an aspect. A class is the designated unit of

encapsulation in the object-oriented paradigm. A file may also be considered a unit of

encapsulation but only if it contains several other units, e.g., if the file contains several classes

and a composition mechanism uses the whole file as input. Furthermore, the units may differ

depending on the phase of software development. For example, units for a requirements phase

could be use cases, quality attributes, and stakeholders, units for design could be classes,

aspects, data definitions, and states, and units for implementation could be classes and packages.

Finally, a unit of encapsulation may be shown in one or more views (see section 1.3).

Questions (Repeat B–E for each unit of encapsulation identified in A):

A. List the unit(s) of encapsulation used by a language of the modeling approach:

B. If the modeling approach uses more than one language, to which language does the

unit of encapsulation belong?

8

C. Which software development phases and their activities does the unit of encapsulation support?

D. In which view(s) is the unit of encapsulation shown?

E. Which type of well-defined interface, if any, does the unit of encapsulation have?

� Public/private OO-type interface

� Protocol state machine

� Service level agreement

� Other:

� No well-defined interface

2.2 Composability/composition operators

Composition is the act of creating new modules/units of encapsulation by using existing ones (e.g., by putting together several units of encapsulation). In the context of this

document and assessment, the term composition applies to a wide array of operators (CompositionOperator) that combine existing units of encapsulation. Basic composition

operators enable the structuring of modules through traditional means (e.g., inheritance, aggregation and composition (as defined by UML), hierarchical decomposition (e.g., sub-

activity diagrams), as well as grouping mechanism (e.g., a package)). More advanced forms

of compositions are (i) the composition of crosscutting concerns through pattern matching, superimposition, aspect weaving, or other means and (ii) model transformations.

For a composition to occur, one or more units of encapsulation must first be identified as input to the composition. Input identification (InputSpecificationMethod) can be done

explicitly (e.g., class A inherits from class B), by pattern matching (e.g., a regular expression

identifies the model elements that need to be composed), or through a binding (i.e., a type of

explicit input identification that maps one model element to another model element). Second,

the composition operator needs to be defined. Implicit composition does not require the

specification of a specific composition operator (e.g., the RAM [6] weaver assumes that the

desired composition operator is “merge”). On the other hand, explicit composition requires the specification of a composition operator, e.g., add, replace, wrap, merge, etc.

A composition operator may also be classified by whether it supports symmetric or

asymmetric composition (InputInterpretation). A composition operator is typically applied to

two input models to create another model. If both inputs are of the same “type”, then

symmetric composition is probably supported, and in this case commutativity holds as the

order of the inputs does not matter. If the input models are of different “types” (e.g., class and

aspect), then asymmetric composition is probably supported by the approach, i.e., composition works if A is applied to B but does not work if B is applied to A.

Furthermore, composition may be syntax-based or semantics-based [7] (DomainType).

In syntax-based composition, the composition is based on syntactic references to the input

models. In the context of the composition of crosscutting concerns, this may lead to the well-

known fragile pointcut problem, where structural changes in the base concerns may invalidate

the compositions. This problem is tackled in semantics-based composition by relying on the

meaning of the input models and the relationships to be captured by the composition rather

than the structure of the input models or specific naming conventions.

9

Semantics-based composition may be applied to the identification of locations where

composition is supposed to occur (e.g., identification of semantically equivalent patterns) or

the composition itself (e.g., in simplifying complex results by recognizing redundant model

elements; an example is composing inheritance classes that also have a direct relation –

simple composition will result in multiple direct relations, all except the first of which are

redundant). The rules that govern semantics-based composition cannot be inferred by a

machine unless the formalization of an approach goes beyond its abstract grammar, e.g., if an execution semantics is formally described then behavior equivalence may be determined.

The precision of a composition operator may be deterministic, probabilistic, or fuzzy

[8] (PrecisionType). Probabilistic compositions are required if there are uncertainties in the

specification of problems. Here, it may be preferable to define and optimize the solutions and

the composition of solutions with respect to the probabilistic problem definitions [8]. Fuzzy

compositions are required if a problem can be solved in a number of alternative ways, and it is

not possible (or desired) to commit to a single alternative. Here, each alternative may be assigned to a fuzzy set that expresses the degree of relevancy of a solution. Special

composition operators are required, which can reason about fuzzy sets.

Regardless of whether basic or more advanced composition mechanisms are supported by a language, assessing the algebraic properties of binary composition operators

(BinaryCompositionOperator) such as commutativity, associativity, and transitivity makes it possible to predict what the result of a composition will be. A language treats models

uniformly if it facilitates decomposing a model into a set of cooperating, structurally compatible concerns, and it provides composition operators that manipulate such concerns

without affecting their compatibility. The uniform treatment of the concerns helps to fulfil the closure property in the models. If a composition operator applied to any member of a set

produces a model that is also in the set, then the set of models is closed under that

operation [8]. For example, a language defined by a metamodel is closed under the operation

if that operation applied to a model that conforms to the metamodel produces a model that

again conforms to the metamodel. However, the set could also be a set of metamodels. Or

quite differently, the set could be the set of regular languages and the operation might change

the regular language. If the set of regular languages is closed under that operation, then the

resulting language again is regular. The closure property is a way to standardize the

decomposition mechanism and composition operators that are provided by a language. In

addition, it helps to increase the abstraction level of concerns by forming a hierarchy of

concerns in which the concerns at the higher levels of the hierarchy abstract from their lower

level concerns.

Whenever several composition operators can be applied, one needs to consider

interactions between these operators. At a minimum, languages may support an explicit

ordering of composition operators (e.g., through precedence). A more detailed assessment of interactions, however, is deferred (see “parking lot” in Section 3).

Finally, some approaches allow units of encapsulation to be composed with the help

of a composition specification. A composition specification may be an intrinsic part of a unit

or may be encapsulated in its own unit. Hence, the modularity of the composition

specification is also of interest, i.e., whether or not it is separated from the units of encapsulation that are being composed.

Questions (Repeat C–L for each composition operator identified in B; A, B, M, N, and

O need to be answered only once):

10

A. If the modeling approach does not use composition operators to create new units

of encapsulation, how does it create new units of encapsulation from existing units of encapsulation?

B. List the composition operator(s) used by a language of the modeling approach:

C. To which language does the composition operator belong?

D. To which unit(s) of encapsulation can the composition operator be applied (i.e.,

what are the inputs of the composition operator)?

E. What is the mechanism for defining the inputs for the composition operator?

� Explicit

� Pattern Matching

� Binding

� Other:

F. What is the mechanism for defining the composition operator?

� Explicit

� Implicit – provide details in this case:

G. Is the composition operator:

� Symmetric

� Asymmetric

H. Is the composition operator:

� Syntax-based

� Semantics-based – provide details in this case:

I. Is the composition operator:

� Deterministic

� Probabilistic

� Fuzzy

� Other non-deterministic

J. If the composition operator is binary, what algebraic properties does the

composition operator provide?

� Commutativity

� Associativity

� Transitivity

� Other:

11

K. Does the composition operator produce models that are closed under the operator?

� Yes

� No

L. Can the composition operator be used to address crosscutting concerns?

� Yes

� No

M. Does a language of the modeling approach support an explicit ordering of composition operators?

� Yes – state which language(s) and explain how for each language:

� No

N. Is the composition specification itself separated from the specification of the unit

of encapsulation? Describe in detail how the composition specification is or is not

separated from the unit of encapsulation.

� Yes

� No

Details:

O. How is a composed model intended to be presented to the modeler by a tool?

� The composed model is intended to be shown with automatic layout.

� The composed model is intended to be shown without automatic layout.

� The original model is intended to be shown with annotations indicating the

composition.

� The composed model is intended to be not shown.

� Other:

� If the composed model is presented in a different way depending on the used

composition operator, state the composition operator and explain the different approach:

2.3 Traceability

Traceability is (i) the ability to chronologically interrelate uniquely identifiable

entities in a way that is verifiable or (ii) the ability to verify the history, location, or application of an entity (e.g., a concern, a process, or an artifact such as a requirements model

element or a module) by means of documented recorded identification [9]. Entities that are

traced can either be units of encapsulations or model elements (ModelElementType) within such units.

Traceability can be vertical or horizontal (TraceabilityType). Using requirements as an example, vertical traceability is the ability to trace requirements back and forth through the

various layers of development, e.g., through the associated life-cycle work products of

architecture specifications, detailed designs, code, unit test plans, integration test plans,

system test plans, etc. It is possible to generalize this definition of vertical traceability to refer

12

to abstraction levels above the requirements (e.g., system engineering or business process engineering) and below the requirements (e.g., architecture) [10].

Again using requirements as an example, horizontal traceability refers to the ability to

trace requirements back and forth to associated plans such as the project plan, quality

assurance plan, configuration management plan, risk management plan, etc. This definition

can also be generalized. Horizontal traceability may be required during any phase of software

development, e.g., at the design level (tracing across different design documents) and at levels above requirements (system engineering or product family domain analysis).

Some traceability links may cross the boundaries of software development phases, while other links remain within such boundaries.

From a composition point of view, traceability includes being able to identify

elements in the composed model with their sources in the original models. More traditional

uses of traceability include identification of refinements at lower levels of abstractions if not directly supported by the modeling language (e.g., through hierarchical decomposition).

It may be argued that traceability is the concern of tools, however, some entities may

be explicit in the modeling approach that are, or could be, targeted for traceability.

Furthermore, modeling concepts may exist in the languages of the modeling approach primarily for the purpose of supporting traceability.

Questions:

A. What types of traceability are supported?

(For each type, describe (i) the entities that are traced either in the modeling

approach (i.e., as part of the process of applying the modeling approach or in

models produced by the modeling approach), (ii) the purpose of the type of

traceability, and (iii) whether the type of traceability spans more than one software

development phase)

� Horizontal

� Vertical

� Other:

B. Which language concepts of the modeling approach exist primarily to support

traceability?

2.4 Mapping from the previous and to the next phase of software development

This property is about whether the concepts of one approach are compatible with

another approach that precedes or follows the approach. It is hence not purely a traceability issue but rather a compatibility issue. The property describes the support of a modeling

approach for the transition from one development phase to another. It may be necessary to coerce the concepts of one approach into those of another or to extend one approach to make

it interface properly with another approach. Mappings may exist at the level of units of encapsulation or at finer granularity, i.e., model elements within a unit of encapsulation

(ModelElementType). If a modeling approach supports more than one development phase, this question is about the handover to preceding and succeeding modeling approaches.

13

Section 2.3 on traceability should cover the links between more than one development phase within the same modeling approach.

Questions:

A. How does the modeling approach address moving from the “previous” phase of

software development and which model elements are connected?

B. How does the modeling approach address moving to the “next” phase of software

development and which model elements are connected?

2.5 Tool support

Features (Feature) of tools (Tool) automate formalized parts of one or more activities

within and across software development phases (e.g., requirements, design, code, verification & validation, deployment, runtime management, evolution).

Questions:

A. List the tool(s) that support the modeling approach:

B. Which features are supported by the tool(s) and by what degree?

Specification/Modeling: � Full � Partial � None

Application of Composition Operators: � Full � Partial � None

Incremental Composition: � Full � Partial � None

Analysis: � Full � Partial � None

Trade-Off Analysis: � Full � Partial � None

Validation: � Full � Partial � None

Verification: � Full � Partial � None

Evolution: � Full � Partial � None

Model Consistency: � Full � Partial � None

Other:

C. How is a composed model presented to the modeler by the tool(s)?

� The composed model is shown with automatic layout.

� The composed model is shown without automatic layout.

� The original model is shown with annotations indicating the composition.

� The composed model is not shown.

� Other:

14

D. How do tools help users understand relationships across units of encapsulation?

Which relationships are covered (e.g., unwanted interactions or conflicts, dependencies between units of encapsulation)?

� Unwanted interactions or conflicts – describe how the tool(s) help:

� Dependencies – describe how the tool(s) help:

� Other:

2.6 Empirical studies/evaluation

Empirical studies are broadly classified into controlled experiments, case studies, and surveys [11].

A case study is an in-depth observational study, whose purpose is to examine projects or evaluate a system under study. Case studies are mostly used to demonstrate the application

of an approach in a particular context. For example, in the context of software modeling, the

goal of a case study could be to demonstrate the applicability of modeling non-functional

behavior (e.g., security) of an industrial case study and reporting lessons learnt during this process.

Surveys are normally conducted to evaluate a tool, methodology, or approach, which

has been in use for a while. Usually, data is collected using questionnaires and interviews

from a representative population. Once the data is collected, it is analyzed using appropriate

descriptive statistics and possibly using statistical tests. Based on the analysis of results, conclusions are drawn and generalized.

A controlled experiment is performed in a controlled environment with the aim to

manipulate one or more variables and maintain other variables at fixed values to measure the

effect of change (outcome). Experiments usually have two or more treatments, whose

outcomes are to be compared. For example, in the context of software modeling, an example

of a controlled experiment could be to measure effort required by two approaches, which are

developed to solve a particular modeling problem, such as modeling non-functional properties.

Questions:

A. Briefly describe empirical studies or evaluations that have been performed for the modeling approach (provide references wherever possible).

B. How does the modeling approach measure the quality of a composition?

� Resulting model modularity

� Resulting model coupling/cohesion

� Other:

3. Additional Key Modeling Concepts (“parking lot”)

Several other potential comparison criteria have been identified but at this point have

not been adequately discussed to be included in the assessment. During the workshops we ranked these criteria in order of importance. They are, in that order: reusability, scalability,

15

inter-module dependency and interaction, abstraction, usability (readability,

understandability), ease of evolution, reduction of modeling effort, completeness, and finally expressiveness.

4. References

1. Business Dictionary, http://www.businessdictionary.com/definition/tradeoff-analysis.html,

2011.

2. Harold Ossher and P. Tarr: “Multi-Dimensional Separation of Concerns and the Hyperspace

Approach”. In Software Architectures and Component Technology, M. Aksit (Ed.), Kluwer

Academic Publishers, pp. 293–323, 2002.

3. David Parnas: “On the Criteria To Be Used in Decomposing Systems into Modules”. In

Communications of the ACM, vol. 15, pp. 1053–1058, 1972.

4. Paul Clements, Felix Bachmann, Len Bass, David Garlan, James Ivers, and Reed Little:

“Documenting Software Architectures: Views and Beyond”. Addison-Wesley Professional, 2002.

5. Siobhán Clarke and Elisa Baniassad: “Aspect-Oriented Analysis and Design: The Theme

Approach”. Addison-Wesley Professional, 2005.

6. Jörg Kienzle, Wisam Al Abed, and Jacques Klein: “Aspect-oriented multiview

modeling”. Proc. 8th ACM International Conference on Aspect-oriented Software Development (AOSD’09), ACM, pp. 87–98, March 2009,

doi:10.1145/1509239.1509252.

7. Ruzanna Chitchyan, Phil Greenwood, Americo Sampaio, Awais Rashid, Alessandro Garcia,

and Lyrene Fernandes da Silva: “Semantic vs. syntactic compositions in aspect-oriented

requirements engineering: an empirical study”. In Proceedings of the 8th ACM international

conference on Aspect-oriented software development, pp. 149–160, 2009.

8. Mehmet Aksit: “The 7 C's for Creating Living Software: A Research Perspective for Quality-

Oriented Software Engineering”. In Turkish Journal of Electrical Engineering and Computer

Sciences, vol. 12, pp. 61–95, no. 2, 2004.

9. ISO 9001:2008, Quality management systems – Requirements,

http://www.iso.org/iso/catalogue_detail?csnumber=46486

10. Tim Kasse: “Practical Insight into CMMI”, Artech House Publishers, ISBN: 1580536255, pp.

153–154, 2008.

11. Claes Wohlin, Per Runeson, and Martin Höst: “Experimentation in Software Engineering: An

Introduction”, Springer, 1999.