Information System Architecture Metrics: An Enterprise

Page 1

Information System Architecture Metrics: An Enterprise Engineering Evaluation Approach

André Vasconcelos, Pedro Sousa, José Tribolet CEO - Centro de Engenharia Organizacional,

INESC – Instituto de Engenharia de Sistemas e Computadores and

Department of Information Systems and Computer Engineering, Instituto Superior Técnico, Technical University of Lisbon, Portugal

E-mails: {andre.vasconcelos, pedro.sousa, jose.tribolet}@dei.ist.utl.pt

Abstract Although some important technological developments have been achieved during last decade, information systems still do not answer efficiently enough to the continuous demands that organizations are facing – causing a non-alignment between business and information technologies (IT) and therefore reducing organization competitive abilities. This paper proposes sixteen metrics for the Information System Architecture (ISA) evaluation, supported in an ISA modelling framework. The major goal of the metrics proposed is to assist the architect previewing the impact of his/her ISA design choices on the non-functional qualities of the Enterprise Information System (EIS), ensuring EIS better align with business needs. The metrics proposed are based on the research accomplished by other authors, from the knowledge in other more mature areas and on the authors experience on real world ISA evaluation projects. The metrics proposed are applied to an e-government project in order to support the definition of a suitable ISA for a set of business and technological requirements.

Key-words: Information System Architecture Metrics, Information System Architecture Evaluation, Enterprise Information System, CEO Framework, E-Government Project Evaluation.

1. Introduction Though Information System Architecture (ISA) is currently recognized as an essential step in the process of building Enterprise Information Systems (EIS) aligned with business needs, there are not tools that assist the Information System (IS) architect in accessing (during “design time”) the impact of his or her decisions on the global ISA qualities. Moreover, other ISA stakeholders that might have limited knowledge on ISA matters (as business people, software engineers, infra-structure experts) do not have simple methods or tools to quickly and automatically evaluate an ISA in respect to a set of desired IS qualities driven from the business context.

The authors’ research pretends to provide ISA stakeholders the tools for assessing ISA qualities ensuring EIS suitable to business needs.

Firstly, recognizing the need for a coherent way of representing ISA, in Vasconcelos et al. (2001), the authors proposed a set of Enterprise modelling primitives (the CEO Framework), extended later into an UML profile for ISA modelling – regarding information, application and technological information system concerns (Vasconcelos et al. 2003).

Afterward the ISA modelling framework have been tested in real world case studies (Vasconcelos et al. 2004a) and enriched considering other IS characteristics (Vasconcelos et al. 2004b) – this

Page 2

research step confirmed the need for tools capable of supporting the architect while building the ISA and quickly accessing his or her design choices.

More recently, considering that the evaluation topic is a quite mature issue on the software engineering domain, the authors classify several software evaluation approaches in order to consider its applicability for ISA evaluation and adapted some software metrics to the information system context (Vasconcelos et al. 2005).

In this paper the authors present theirs recently developments on ISA evaluation by proposing and explaining the foundation of a set of metrics for ISA evaluation.

The ISA modelling framework that supports the evaluation metrics is introduced in section 2. In section 3 the authors proposed a coherent set of ISA evaluation metrics, relating ISA qualities and ISA components. In section 4 the metrics proposed are applied to an e-government ISA project. The conclusions and future work are presented in section 5.

2. CEO Framework for ISA modelling The CEO Framework (Figure 1) aims at providing a formal way of describing business goals, processes, resources and information systems and the dependencies between them. It is composed of three separate levels, each of which provides adequate forms of representing the notions about the layer being described (Vasconcelos et al. 2001).

Figure 1. Goal / Process / System framework

In the first level, the aim is at describing the current set of goals that drive business. These goals must be achieved through one or more business process. The business processes are described at the second level and must exist in order to satisfy one or more goals. Besides serving goals, business processes interact with resources in order to do work and may be supported by information systems. The information systems layer aims at modelling the components of the system that support business.

The modelling language used to implement the CEO Framework was UML (Unified Modelling Language). As UML was initially designed to describe aspects of a software system, it had to be extended to more clearly identify and visualize the important concepts of business, namely by use of stereotypes – for further detail on UML extension mechanisms see OMG (2004). Due to size restrictions, we will not do a full presentation on the CEO Framework (for further reading, refer to Vasconcelos et al. (2001)). Figure 2 presents the UML metamodel defined for the CEO Framework.

Page 3

«goal»

Goal

Resource

«resource»«process»Process

«block»

Blocksupports

controls,executesproduces,consumes,uses, refines

«contradictory»

achieves

*

**

*

sends,receives

Figure 2 - UML Metamodel of the CEO Framework

In order to model ISA key concepts, the «Block» component was specialized. The key concepts for the Information System Architecture are:

Information Entity – person, place, physical thing or concept that is relevant in the business context;

IS Block – Application architecture main aim is on the functional components characterization. At application level, the IS Block (or Application Block) notion is the founding concept. IS Block is defined as the collection of mechanisms and operations organized in order to manipulate organization data.

IT Block – Technological architecture addresses a large variety of notions, caused, on the one hand, by the continuous technological evolutions and, on the other hand, by the need for different specialized IT architectural views – as security, hardware and software development architectures. In order to encapsulate this diversity, this framework uses the “IT Block” concept. IT Block is the infrastructure, application platform and technological/software component that realizes (or implement) an (or several) IS Block(s). IT Block defines three major sub-concepts:

- IT Infrastructure Block – represents the physical and infra-structural concepts existing in an ISA: the computational nodes (as servers, personal computers or mobile devices) and the non-computational nodes (as printers, network, etc.) that support application platforms.

- IT Platform Block – stands for the implementation of the services used in the IT application deployment.

- IT Application Block, the technological implementation of an IS Block. At this level is relevant to consider the kind of IT Application Block (namely presentation, logic, data and coordination block), and its “technological principles” (like if it is implemented using components, modules, OO principles, etc.), among other characteristics.

Service – is an aggregation of a set of operations provided by an architectural block. A generalization of the web service notion (W3C 2002). We consider three distinct services in an ISA:

- Business Service. A business service is a collection of operations provided by IS Blocks that support business processes.

- IS Service. The set of operations provided by an IS Block to others IS Blocks defines the IS service.

- IT Service. The technological services provided by application platforms are the IT services [Open 2001].

Operation, the abstract description of an action supported by a service. Thus, operations are the minor level concept relevant in an ISA.

Figure 3 describes how these high-level primitives are related, in a UML profile for ISA. For further detail please refer to Vasconcelos et al (2003).

Page 4

Figure 3. Information System CEO framework metamodel (Vasconcelos et al. 2003)

3. Information System Evaluation Metrics In this section the authors propose a set of ISA evaluation metrics. These metrics were defined based on the research accomplished by other authors (specialists in certain areas – e.g., security, scalability, portability), on the adaptation of the evaluation knowledge from other more mature areas (e.g., software engineering) and on the authors experience on real world ISA evaluation projects.

The authors argue that with these metrics the architect has a set of indicators on the impact of each of his or her decision during the process of building an ISA and, therefore, he or she will be better equipped to build EIS align with a set of desired qualities.

The ISA qualities mention in this section are based on Bass et al.(1998) and Clements (2002) software qualities as described in Vasconcelos et al. (2005).

The following template is used to describe the metrics proposed.

Table 1. ISA metric template

Acronym Metric Acronym

Name Metric Name

Computation Description on the metric algorithm or formula

Scale Scale of possible values for the metric

Architectural Levels

Architecture levels relevant for this metric

ISA Primitives and attributes

Architectural primitives and attributes used in the metric computation

ISA Qualities Enumeration of the “architectural qualities” related with the metric

Support Rational that supports the metric proposed and its relevance for measuring the ISA qualities

Example(s) Presentation of ISA evaluation simple examples by applying the proposed metric

Page 5

Acronym NPOS (or NPOSISA)

Name Average Number of Possible Operating Systems

Computation

The Average Number of Possible Operating Systems is computed by counting, on each application («IT Application Block»), the number of possible operating systems (families) and dividing it by the number of applications

»«#

»«#

1

BlocknApplicatioIT

NPOSNPOS

BlocknApplicatioIT

ii

ISA

∑== , where

NPOSi – is the number of possible operating systems families that the «IT Application Block»i supports #«IT Application Block» – is the number of «IT Application Block» in the ISA

Scale [ [+∞;1


Technological Architecture


Primitive: «IT Application Block» - Attribute: possible operating systems

ISA Qualities The Portability and Technical Interoperability of an EIS tend to increase with this metric

Support

The portability and Technical Interoperability in an ISA increase with the number of possible platforms where ISA components are able to operate (Sarkis and Sundarraj 2003, section 3.2.1). From a software engineering perspective, the portability of an operating system is a major indicator on an application portability (Roulo 1997); in the same way, the technical portability of an EIS, represented by an ISA, is measure by this metric as the average of the software applications’ («IT Application Block») portability.

Example(s)

Example ISA A

:My Application B

«IT Application Block»

Possible Operating Systems = {Linux}

:My Application A


Possible Operating Systems = {Linux, Windows, UNIX}

2213 == +

ISAPOS

Example ISA B

:My Application B


Possible Operating Systems = {Windows}

:My Application A


Possible Operating Systems = {Windows}

1211 == +

ISAPOS

Page 6

Acronym NDTIS (or NDTISISA)

Name Average Number of Distinct Technologies for IS Services

Computation

The Average Number of Distinct Technologies for Information System Services is computed by counting for each «IS Service» the number of «IT integration Services».

»«#

»«#»«#

1

ServiceIS

ServicenIntegratioITNDTIS

ServiceIS

ii

ISA

∑== , where:

#«IT Integration Service»i – is the number of «IT Integration Service» that implement the «IS Service» i #«IS Service» – is the number of «IS Service» in an ISA

Scale [ [+∞;1


Application Architecture and Technological Architecture


Primitive:: «IT Integration Service» - Attribute: technology

ISA Qualities The Portability and Technical Interoperability of an EIS tend to increase with this metric

Support

The technical interoperability of a software architecture increases by providing the same interface in different technologies (Sarkis and Sundarraj 2003, section 3.2.1). In the same way, with this metric the technical interoperability and portability of an EIS is analyzed as the average of the Technologies that each application interface provides.

Example(s)

Example ISA A

111==ISANDTIS

Example ISA B

313==ISANDTIS

Page 7

Acronym NIIE (or NIIEISA)

Name Average Number of (Different) Implementations of an Information Entity

Computation

The Average Number of (Different) Implementations of an Information Entity is computed by counting, for each «Information Entity», the number of possible implementations in «Low Level Information Entities».

»«#

»«#

1

EntitynInformatio

NLLIENIIE

EntitynInformatio

ii

ISA

∑== , where:

NLLIEi – is the number of «Low Level Information Entities» that are related to the «Information Entity»i by the «implements» relation #«Information Entity» – is the number of «Information Entities» in an ISA

Scale [ [+∞;1


Information Architecture


Primitive: «Low Level Information Entity» Primitive: «Information Entity»

ISA Qualities The Syntactic Interoperability of an ISA will increase by the decrease of this metric

Support

This metric measures the number of different implementations that exist for each information entity. According to Inmon (2000), for each information entity (“top level”) there might be other entities that implementing it (“low level information entity”). The existence of different «Low Level Information Entities» points to syntactic incompatibilities for that «Information Entity» (e.g., by using different formats or attributes in the implementation of the information entity).

Example(s)

Example ISA A «Information Entity»

Address

«Low Level Information Entity»

AddressTaxes


AddressSocial Security


AddressHealth Care

StreetDoor numberTownZip_Code_Part1Zip_Code_Part2Zip_Place_Designation

StreetZip_CodeTown

StreetDoor numberTownZip_CodeZip_Place_Designation

«implements»

«im

plem

ents

» «implements»

313==ISANIIE

Example ISA B

111==ISANIIE

Page 8

Acronym NSITPLB (or NSITPLBISA)

Name Average Number of stateful «IT Presentation Block» and «IT Logic Block»

Computation

The Average Number of stateful «IT Presentation Block» and «IT Logic Block» is computed counting the number of «IT Presentation Block» and «IT Logic Block» that its attribute “state” value is “stateful”.

»«#»Pr«##

LogicBlockITBlockesentationITSITPLBNSITPLBISA +

= ,

where:

#SITPLB – is the number of «IT Presentation Block» and «IT Logic Block» that its attribute “state” value is “stateful”. #«IT Presentation Block» – is the number of « IT Presentation Block » #«IT Logic Block» – is the number of « IT Logic Block »

Scale [0; 1]




Primitives: «IT Presentation Block», «IT Logic Block», «IT Data Block», «IT Coordination Block» Attribute: state

ISA Qualities The scalability of an ISA tends to increase with the decrease of this metric.

Support

The Scalability of an EIS is increased if business and presentation components do not keep the state (since it will be easier for implementing new parallel instances of these ISA components) – BEA (2006). The Scalability of an ISA tend to grow if the «IT Presentation Blocks» and the «IT Logic Blocks» do not preserve the application state (stateless) – the «IT Data Blocks» should be the ones to keep application state.

Example(s)

Example ISA A

:W e b S ite

« IT P re s e n ta tio n B lo c k »

s ta te = “s ta te le s s ”

:S to re L o g ic

« IT L o g ic B lo c k »

s ta te = “s ta te fu ll”

:S to re D a ta

« IT D a ta B lo c k »


21

111

=+

=ISANSITPLB

Example ISA B

:W e b S ite

« IT P re s e n ta t io n B lo c k »


:S to re L o g ic

« IT L o g ic B lo c k »


:S to re D a ta

« IT D a ta B lo c k »


011

0=

+=ISANSITPLB

Page 9

Acronym NSC (or NSCISA)

Name Average Number of security components

Computation

The Average Number of security components is computed counting all the «IT Blocks» which attribute “security” value is “YES”.

»«##

BlockITSITBNSCISA = , where:

#SITB – is the number of «IT Blocks» which attribute “security” value is “YES” #«IT Block» – is the number of « IT Block »

Scale [0; 1]




Primitive: «IT Block» Attribute: security

ISA Qualities The security of an ISA tends to increase with this metric increasing.

Support The ISA security is increased by putting security elements on it, as IDS, firewalls, etc (Rito 2004). This metric considers this fact.

Observations This simple to compute metric is provides a quick first overview on the potential (miss of) security of an ISA. However, this metric does not considers the role of the security components on the ISA.

Example(s)

Example ISA A

:Firewall

«Netwrok»

security = “yes”

:Data Server

«Server»

:Application Server

«Server»

31

=ISANSC

Example ISA B

:LAN

«Netwrok»

security = “no”

:Data Server

«Server»

:Application Server

«Server»

030==ISANSC

Page 10

Acronym NSCBITAB (or NSCBITABISA)

Name Average Number of security components between «IT Application Blocks»

Computation

The Average Number of security components between «IT Application Blocks» is computed counting, for each «IT Application Block», the minimum number of «IT Blocks», which attribute “security” value is “YES”, that are between that block and all the other «IT Application Blocks».

( )2

#

1

»«#

1

»«#

}{#

BlocknApplicatioIT

SITBMinNSCBITAB

BlocknApplicatioIT

i

BlocknApplicatioIT

jij

ISA

∑ ∑= =

⎥⎦

⎤⎢⎣

⎡

= , where:

Min{#SITBij}– is the minimum number of «IT Blocks» which attribute “security” value is “YES” that are between «IT Application Block»i and «IT Application Block»j. #«IT Application Block» – is the number of «IT Application Blocks»

Scale [ [+∞;0





ISA Qualities The Security of an ISA tends to increase with this metric.

Support The ISA security is increased by putting security elements on it, as IDS, firewalls, etc (Rito 2004). This metric, is not limited to counting the number of security components but it also considers, for each application component, the number of security components that isolate it from other components.

Example(s)

Example ISA A

: F i r e w a l l

« N e t w r o k »

s e c u r i t y = “ y e s ”

: F i r e w a l l

« N e t w r o k »


: P C« P e r s o n a l C o m p u t e r »

: S o u r c i n g I n t e r f a c e C l i e n t« I T A p p l i c a t i o n B l o c k »

: P C P l a t f o r m« I T P l a t f o r m »

: A p p l i c a t i o n S e r v e r« S e r v e r »

: W a r e h o u s e B u s i n e s s C o m p o n e n t« I T A p p l i c a t i o n B l o c k »

: J 2 E E A p p l i c a t i o n E n v i r o n m e n t« I T P l a t f o r m »

: D a t a S e r v e r« S e r v e r »

: W a r e h o u s e D a t a C o m p o n e n t« I T A p p l i c a t i o n B l o c k »

: D B M S« I T P l a t f o r m »

98

3)21()11()21(

2 =+++++

=NSCBITAB

Example ISA B

: F i r e w a l l

« N e t w r o k »


: P C« P e r s o n a l C o m p u t e r »

: S o u r c in g In t e r f a c e C l ie n t« IT A p p lic a t io n B lo c k »

: P C P la t f o r m« IT P la t fo r m »

: A p p l ic a t io n S e r v e r« S e r v e r »

: W a r e h o u s e B u s in e s s C o m p o n e n t« IT A p p l ic a t io n B lo c k »

: J 2 E E A p p l ic a t io n E n v ir o n m e n t« IT P la t fo rm »

: D a t a S e r v e r« S e r v e r »

: W a r e h o u s e D a t a C o m p o n e n t« IT A p p l ic a t io n B lo c k »

: D B M S« IT P la t fo rm »

94

3)10()10()11(

2 =+++++

=NSCBITAB

Page 11

Acronym LCOIS (or LCOIS ISA)

Name Average Lack of COhesion in «IS Blocks».

Computation

The Average Lack of Cohesion in «IS Blocks» is computed counting the number of sets of information entities that are used by distinct functionalities of the same application (provided by operations in «IS Blocks»).

»«#

##

1

BlockIS

LCOISLCOIS

BlockIS

ii

ISA

∑== , where:

#LCOISi – is the number of sets of «Information Entities» that are used by «operations» distinct of the «IS Block» i;. #«IS Block» – is the number of «IS Blocks»

Scale [ [+∞;0





ISA Qualities The security of an ISA tends to increase with this metric.

Support

This metric measures the correlation between application blocks and the information entities used in that application block. It is quantified by the average of the number of sets of information entities that are used by distinct operations of the same application.

Example(s)

Example ISA A

«IS Block»

:ERP

«Information Entity»

:Customer

«IS Service»:Manage

Customers

«Operation»:Create

Customer

C


:Employee

R

«IS Service»:Manage Suppliers


:Supplier C«Operation»

:Create Supplier

21

11=

+=ISALCOIS

Example ISA B

Page 12

«IS Block»

:SCM


:Customer


Customers


Customer

C


:Employee

R



:Supplier C«Operation»:Create Supplier

«IS Block»

:CRM

12

11=

+=ISALCOIS

Page 13

Acronym NOIS (or NOIS ISA)

Name Average Number of Operations in «IS Blocks»

Computation

The Average Number of Operations in «IS Blocks» is computed counting the number of operations on each «IS Block» divided be the number of «IS Blocks»

»«#

»«##

1»«

BlockIS

operationNOIS

BlockIS

iiISBlock

ISA

∑== , where:

#«operation»«IS Block»i – is the number of operations on «IS Block» i. #«IS Block» – is the number of «IS Block»

Scale [ [+∞;0


Application Architecture


Primitive: «IS Block» ; «operation»

ISA Qualities The modificability of an ISA tends to be reduced with the increase of this metric

Support The simplicity of adapt/alter operations in an ISA to new business demands is maximized when the impact of changing each operation is reduced to a certain application block («IS Block»). This metric measures this fact.

Example(s)

Example ISA A

«IS Block»

:ERP


Customers


Customer



Supplier

«Operation»:Search

Customer

313==ISANOIS

Example ISA B

«IS Block»

:SCM


Customers


Customer


«Operation»:Create Supplier

«IS Block»

:CRM


Users

«Operation»:Search

Customer

«IS Block»

:WebSite

13

111=

++=ISANOIS

Page 14

Acronym SCC (or SCCISA)

Name Average Service Cyclomatic Complexity

Computation

The Average Service Cyclomatic Complexity is computed considering the average, number of dependencies between «IS Blocks» subtracted by the number of «IS Blocks» that support the service, for each service.

»«#»sin«#

2»«#»sin«#

1

ServiceISServiceessBu

neSCC


iii

ISA +

+−=

∑+

= , where:

ei – is the number of dependencies between «IS Block» for the service i. ni – is the number of «IS Blocks» that support the service i. #«Business Service» – is the number of «Business Services» #«IS Service» – is the number of «IS Services»

Scale [ [+∞;1




Primitive: «IS Block» ; «Business Service»

ISA Qualities The complexity of an ISA tends to increase with this metric. The modificability of an ISA tends to decrease with the increase of this metric.

Support

Like McCabe (1976), for the software engineering area, considering that the higher the number of paths in a program, the higher its control flow complexity probably will be, in Vasconcelos et. al. (2005) is proposed a similar metric for evaluate the complexity of an ISA in the support of the business services – considering that the complexity, for each service, is measure by the difference between the number of dependencies and applications involved.

Example(s)

Example ISA A

:Patient Clinical

Management

«IS Block»

:CRM

«IS Block»

:Mobile Application

«IS Block»

:Hospital System

«IS Block»:Primary

Health Care System

1 2

3

4

5

5

67

«Business Service»

61

248=

+−=ISASCC

Example ISA B

:P a tie n t C lin ica l

M a n a g e m e n t

« IS B lo c k »

:E R P1

2

« B u s in e s s S e rv ic e »

31

212=

+−=ISASCC

Page 15

Acronym RS (or RSISA)

Name Average Response for a Service

Computation

The Average Response for a Service is computed by considering the average of the number of «IS Blocks» that might be used to support each «Service»:

»«#»sin«#

»«#»«#»sin«#

1


BlockISRS


ii

ISA +=

∑+

= , where:

#«IS Block»i – is the number of «IS Blocks» involved in supporting service i. #«Business Service» – is the number of «Business Services» #«IS Service» – is the number of «IS Services»

Scale [ [+∞;0




Primitives: «IS Block» ; «Business Service»

ISA Qualities The complexity of an ISA tends to increase with this metric

Support

Similar to the software metric “Response For a Class” – see Chidamber and Kemerer (1995) and Basili (1996) for further details – that computes the number of methods that can potentially be executed in response to a message received. In Vasconcelos et. al. (2005) this metric is proposed (Average Response for a Service) and it computes the number of «IS Blocks» that might be used to support a service. In recent researches Sousa, Pereira and Marques (2004) suggest that each business process should be supported by the less number of applications as possible – this is also measure by this metric.

Example(s)

Example ISA A

:Patient Clinical

Management

«IS Block»

:CRM

«IS Block»

:Mobile Application

«IS Block»

:Hospital System

«IS Block»:Primary

Health Care System

1 2

3

4

5

5

67


414==ISARS

Example ISA B

:Patient C lin ical

M anagem ent

«IS B lock»

:ER P1

2

«B usiness Service»

111==ISARS

Page 16

Acronym NE (or NEISA)

Name Number of Entities

Computation The Number of Entities (of an ISA) is computed by counting the number of «Information Entities».

Scale [ [+∞;0




Primitive: «Information entity»

ISA Qualities The Maintainability of an ISA tends to decrease with this metric increase.

Support According to Género, Poels and Piattini (2003), this metric is strongly related to the Maintaining of an ISA.

Example(s)

Example ISA A


Product


Customer


Store

< buys

Placed in>

Goes to>

NEISA=3

Example ISA B


Product


Customer

< buys

NEISA=2

Page 17

Acronym NR (or NRISA)

Name Number of Relations

Computation The Number of Relations of an Information Architecture is computed by counting the number of relations between «Information Entities»

Scale [ [+∞;0




Primitive: «Information entity»

ISA Qualities The Maintainability of an ISA tends to decrease with this metric increase

Support According to Género, Poels and Piattini (2003), this metric is strongly related to the Maintaining of an ISA.

Example(s)

Example ISA A


Product


Customer


Store

< buys

Placed in>

Goes to>

NRISA=3

Example ISA B


Product


Customer

< buys

NRISA=1

Page 18

Acronym CPSM (or CPSM ISA)

Name Critical Process - System Mismatch

Computation

The Critical Process - System Mismatch is computed by counting the number of critical business processes supported by «IS Blocks» that also support non-critical business processes and the number of non-critical business processes supported by «IS Blocks» that also support critical business processes

»«Pr#}{Pr#}{Pr#

ocessISBlockocessISBlockocessCPSM CNCNCC

ISA∈+∈

= ,

where:

}{Pr# NCC ISBlockocess ∈ – is the number of critical processes supported by «IS Blocks» that support other non-critical processes

}{Pr# CNC ISBlockocess ∈ – is the number of non-critical processes supported by «IS Blocks» that support other critical processes #«Process» – is the number of processes

Scale [ ]1;0


Business Architecture and Application Architecture


Primitive: - «IS Block» ; - «Process» - attribute: Critical ={Yes, No}

ISA Qualities Business-Application Alignment

Support As described in Sousa, Pereira and Marques (2004) the critical business processes should be supported by different applications than non-critical business processes.

Example(s)

Example ISA A

«IS Block»

:Production System

«Business Service»Manage

Production

:Assemble Car Components

«Process»

Critical = YES

«IS Block»

:HR System


Manage HR

:Update Vacations Timetable

«Process»

Critical = NO

02

00=

+=ISACPSM

Example ISA B

«IS Block»

:ERP

«Business Service»Manage

Production

:Assemble Car Components

«Process»

Critical = YES


Manage HR

:Update Vacations Timetable

«Process»

Critical = NO

12

11=

+=ISACPSM

Page 19

Acronym NAIE (or NAIE ISA)

Name Average Number of Applications per «Information Entity»

Computation

The Average Number of Applications per «Information Entity» is computed counting the average number of applications («IS Blocks») that through its «operations» support each «information entity».

{ }[ ]

»«#

»«»«##

1

EntitynInformatio

nEntityInformatioCUDoperationISBlocksNAIE

EntitynInformatio

ii

ISA

∑=

∃∈= ,

where:

{ }inEntityInformatioCUDoperationISBlocks »«»«# ∃∈ – is the number of «IS Blocks» in which exists an «operation» that CUD (Creates, Updates or Deletes) the «information entity» i. #«Information Entity» – is the number of «Information Entities»

Scale [ [+∞;0


Information Architecture and Application Architecture


Primitive: - «IS Block» - «Information entity» - «operation»

ISA Qualities Information Architecture – Application Architecture Alignment

Support According to Sousa, Pereira e Marques (2004) each information entity should be managed by a single application.

Example(s)

Example ISA A


:Customer


CustomerC «IS Block»

:CRM«Operation»:Update

CustomerRU

111==ISANAIE

Example ISA B


:Customer


CustomerC

«IS Block»

:StoreSystem

«Operation»:Update

CustomerRU

«IS Block»

:CRM

212==ISANAIE

Page 20

Acronym IESSM (or IESSM ISA)

Name Information Entity - System Security Mismatch

Computation

The Information Entity - System Security Mismatch is computed considering the number of information entities with high level security requirements supported in «IS Blocks» that also support information entities without high security requirements and the number of information entities with low level security requirements supported in «IS Blocks» that also support information entities with high level security requirements.

»«#}{#}{#

nEntityInformatioISBlocknEntityInformatioISBlocknEntityInformatio

IESSM SNSNSsISA

∈+∈=

, where:

}{# NSS ISBlocknEntityInformatio ∈ – is the number of «Information Entities» that its Security attribute value is {Yes} supported in «IS Blocks» that support other «Information Entities» which Security attribute value is {No}; where an «Information Entity» is “supported” by an «IS Block» if and only if exists at least one «operation» provided by the «IS Block» that CUD the «Information Entity».

}{# SNS ISBlocknEntityInformatio ∈ – is the number of «Information Entities» that its Security attribute value is {No} supported in «IS Blocks» that support other «Information Entities» which Security attribute value is {Yes}; where an «Information Entity» is “supported” by an «IS Block» if and only if exists at least one «operation» provided by the «IS Block» that CUD the «Information Entity». #«Information Entity» – is the number of information entities.

Scale [ ]1;0




Primitive: - «IS Block» - «Information entity»; Attribute: Security ={Yes, No} - «operation»


Support According to Sousa, Pereira e Marques (2004) applications should manage information entities of the same security level.

Example(s)

Example ISA A

«IS Block»

:ERP

«IS Service»:Manage Stocks

«Operation»:Update Stock

CRUD


PaymentsC

«Operation»

:Order Bank Payment


:Product

Security = NO


:Payment Order

Security = YES

12

11=

+=ISAIESSM

Page 21

Example ISA B «IS Block»

:Warehouse System


:Product

«IS Service»:Manage Stocks

«Operation»:Update Stock

CRUD


Payments


:Payment Order C

«Operation»

:Order Bank Payment

«IS Block»

:Financial System

Security = NO

Security = YES

02

00=

+=ISAIESSM

Page 22

Acronym NUIEA (or NUIEAISA)

Name Average Number of Unused Information Entity Attributes

Computation

The Average Number of Unused Information Entity Attributes is computed counting the number of attributes in information entities that are not used in any Read (R) «operation».

∑

∑ ∑

=

=

∈

=

∈

∀= »«#

1

»«#

1

#

1

#

»)«(

EntitynInformatio

ii

EntitynInformatio

i

nEntityInformatioattribute

jj

ISA


operationRNOTattributeNUIEA

i

, where:

»)«(

»«#

1

#

1operationRNOTattribute

EntitynInformatio

i


jj

i

∀∑ ∑=

∈

=

- is the total

number of attributes of «Information Entity» i that are not read (R) by any «operation»

∑=

∈»«#

1#EntitynInformatio

iinEntityInformatioattribute - is the total number of attributes in all

the «information entities» of the ISA #«Information Entity» – is the number «information entities»

Scale [ ]1;0




Primitive: - «Information entity» - «operation»


Support According to Sousa, Pereira e Marques (2004) all the information entities’ attributes should be read for, at least, a business process.

Example(s)

Example ISA A

«O peration»G et P ro d uct

L is tR (productN am e )

« In form ation E ntity»

P ro du ct

pro d uctN am e pro d uctC o lo r

21

=ISANUIEA

Example ISA B

R(productName, productColor)


Product

productName productColor

«Operation»Get Product

List

020==ISANUIEA

Page 23

4. An ISA Evaluation Case Study In this chapter we briefly describe a real case study where some of the previously proposed metrics are used in the process of building, analyzing and improving an ISA in a Portuguese e-government project1.

We start by describing the global project goals, in section 4.1, and then (in section 4.2) we focus on presenting how the CEO modelling framework and metrics supported the ISA definition for a project phase. In section 4.3 we present a short discussion on the case study results.

4.1. The Enterprise Life Cycle Project In order to improve government services for enterprises UMIC - Knowledge Society Agency (a Portuguese governmental organization) set out the Enterprise Life Cycle Project. This project’s major goals are:

1. Implement an “Electronic Enterprise Folder”, dematerializing and providing enterprise information that currently is disperse and sometimes incoherent in different government organizations, through the Internet.

2. Reengineer, improve and accelerate the Firm Start-up Process – that by the time of the project definition took, in average, between 27 and 65 days.

3. Reposition and reorganize the government departments in order to provide a best, cheaper and agile service to enterprises and entrepreneurs.

4. Implement the Enterprise Portal, the preferred channel for government services to Enterprises.

In the meantime of this project definition and kick-off, another initiative was implemented: The “On the Spot Firm”. This initiative makes possible for entrepreneurs to create a company in just one office (one-stop office) in a single day (currently the average time is 1 hour and 14 minutes).

Thus “The Enterprise Life Cycle Project” was split into two major phases; a first phase that pretends to implement the “Enterprise Portal” and to make the “On the Spot Firm offline process“ also available online (in the Enterprise Portal). The second phase is expected to achieve the other project goals (as implementing the “Electronic Enterprise Folder”, Repositioning and reorganizing the government departments and reengineering the Firm start-up Process). In this article we will focus only on the first phase of the project (since the second one is still starting).

4.2. Selecting the “right” ISA The “on the spot” firm start-up process major difference to the traditional company creation process is that the members of the future company may only choose the company name from a set of pre-approved firm names and a set of pre-approved association packs. Currently the entrepreneurs can create an “on the spot” company only on physical desks. This project first phase will also make available this process on the Internet – the process is described on Figure 4.

1 Some facts presented in this case study were changed for security and confidentiality reasons.

Page 24

Figure 4. On the Spot Firm Start-up process

Considering that an application that supports the “offline” on the spot company start-up process already exists, two major options were analyzed before implementing the company start-up process online. The first option (“Architecture A”) considered was to used the “On the spot firm” application to support the online creation of a company in the Enterprise Portal and use this application to directly (“point-to-point”) integrate with the other government departments’ information systems – see Figure 5.

«IS Block»

:Enterprise Portal


Online Firm Start-up

:Create Company

«Process»

«IS Block»

:On the spot Firm Application

«IS Service»

Create Firm

«IS Service»

Finance Firm Registration

«Business Service»Offline

on the spot Firm

Start-up

«IS Service»

Social Security Firm

Registration

«IS Service»

Economy Firm Registration

«IS Service»

Firm Commercial Registration

«IS Block»:SIRCOM (Commercial Registration

Information System)

«IS Block»:Finance

Information System

«IS Block»:Social Security

Information System

«IS Block»:Economy

Information System

:Update available Names for “on the

spot Firms”

«Process»

:Choose Firm Name

«Process»


Available Firm

Names List

Figure 5. ISA “A” for the online company creation process

Another option considered (ISA “B”) was to implement an integration/interoperability layer that would be globally responsible for the “company creation online business process”, integrating with the

Page 25

different applications (namely the “on the spot” application, for getting the available pre-approved firm names) – see Figure 6.

Figure 6. ISA “B” for the online company creation process

Both architectures were analyzed using some of the previously metrics.

For the Information Entity - System Security Mismatch metric applied to architecture A (Figure 7) (considering the presented simplified architecture, with only two information entities) we have:

12

11»«#

}{#}{#=

+=

∈+∈=

nEntityInformatioISBlocknEntityInformatioISBlocknEntityInformatioIESSM SNSNSs

A

And the Information Entity - System Security Mismatch metric, for (simplified) architecture B (Figure 8), value is:

02

00»«#

}{#}{#=

+=

∈+∈=

nEntityInformatioISBlocknEntityInformatioISBlocknEntityInformatioIESSM SNSNSs

B

For the Average Number of Applications per «Information Entity» metric, for ISA A we have:

{ }[ ]

23

212

»«#

»«»«##

1 =+

=∃∈

=∑=

EntitynInformatio


EntitynInformatio

ii

A

And for ISA B the Average Number of Applications per «Information Entity» metric is:

{ }[ ]1

211

»«#

»«»«##

1 =+

=∃∈

=∑=

EntitynInformatio


EntitynInformatio

ii

A

These metrics point that ISA B has a better align between its application and information architectures than ISA A.

Page 26

Figure 7. ISA A – Application-Information Architectures (partial view)

Figure 8. ISA B – Application-Information Architectures (partial view)

In terms of complexity, two metrics were used: The Average Response for a Service metric and the Average Service Cyclomatic Complexity metric.

The Average Response for a Service metric (considering only the three business services described in Figure 9 to Figure 14, for simplicity) for ISA A is:

Page 27

313

3562

»«#»sin«#

»«#»«#»sin«#

1 =++

=+

=∑

+

=


BlockISRS


ii

A

And for ISA B:

53

153

663»«#»sin«#

»«#»«#»sin«#

1 ==++

=+

=∑

+

=


BlockISRS


ii

B

For the Average Service Cyclomatic Complexity metric for ISA A, we have:

311

3)256()268()224(

»«#»sin«#

2»«#»sin«#

1 =+−++−++−

=+

+−=

∑+

=


neSCC


iii

A

And the Average Service Cyclomatic Complexity metric for ISA B is:

313

3)268()268()236(

»«#»sin«#

2»«#»sin«#

1 =+−++−++−

=+

+−=

∑+

=


neSCC


iii

B

Thus, these metrics indicate that ISA B is slightly more complex than ISA A.

Figure 9. Available Firm Names List Collaboration Diagram (ISA A)

Figure 10. Online Firm Start-up Collaboration Diagram (ISA A)

Page 28

Figure 11. Offline on the spot Firm Start-up Collaboration Diagram (ISA A)

Figure 12. Available Firm Names List Collaboration Diagram (ISA B)

Figure 13. Online Firm Start-up Collaboration Diagram (ISA B)

Page 29

Figure 14. Offline on the spot Firm Start-up Collaboration Diagram (ISA B)

In order to evaluate the security of both ISA, from a technological perspective, the technological architecture of ISA A and ISA B were modelled (Figure 15 and Figure 16) and two metrics were applied.

The Average Number of security components metric values for ISA A and ISA B are:

596

»«##

==BlockIT

SITBNSCA =10,17%

10211

»«##

==BlockIT

SITBNSCB =10,78%

And The Average Number of security components between «IT Application Blocks» metric values for ISA A and ISA B are:

( )93,0

1443+14+14+14+14+8+8+8+8+8+8+8+8+19

»«#

}{#

22

#

1

»«#

1==

⎥⎦

⎤⎢⎣

⎡

=∑ ∑= =

BlocknApplicatioIT

SITBMinNSCBITAB

BlocknApplicatioIT

i

BlocknApplicatioIT

jij

A

( )68,2

2479+79+79+79+63+63+63+63+63+63+90+69+69+69+69+53+53+53+53+53+53+53+53+64

»«#

}{#

22

#

1

»«#

1 ==⎥⎦

⎤⎢⎣

⎡

=∑ ∑= =

BlocknApplicatioIT

SITBMinNSCBITAB

BlocknApplicatioIT

i

BlocknApplicatioIT

jij

B

These metrics point that ISA B has a higher security level than ISA A.

Page 30

Figure 15. Technological Architecture A

Page 31

Figure 16. Technological Architecture B

4.3. Discussion The metrics used in the previous section point that:

- ISA A has a worst alignment between its application and information architecture than ISA B;

- ISA A is less complex than ISA B;

- ISA A is less secure than ISA B.

Considering that this phase of the project had a high level of pressure on its implementation timeframe, the complexity quality was considered of highest importance. Thus, since the Interoperability layer was not available, the ISA adopted for this first phase was ISA A.

However, the project is now going to the second phase with several business and technological enhancements – like online creation of companies with names formulated online by the citizen (instead of picking up from a set of pre-approved ones), Electronic Folder on all the information available on the enterprise (after creation), among others. For this next phase the team is going to implement an ISA similar to ISA B, previously described.

Page 32

5. Conclusions and Future Work In this paper the authors proposed a set of ISA evaluation metrics, namely: Average Number of Possible Operating Systems, Average Number of Distinct Technologies for IS Services, Average Number of (Different) Implementations of an Information Entity, Average Number of stateful «IT Presentation Block» and «IT Logic Block», Average Number of security components, Average Number of security components between «IT Application Blocks», Average Lack of COhesion in «IS Blocks», Average Number of Operations in «IS Blocks», Average Service Cyclomatic Complexity, Average Response for a Service, Number of Entities, Number of Relations, Critical Process - System Mismatch, Average Number of Applications per «Information Entity», Information Entity - System Security Mismatch and Average Number of Unused Information Entity Attributes.

Some of these metrics were applied in an e-government ISA evaluation project and revealed to be useful on the process of selecting the most appropriate ISA for a set of desired qualities.

With these metrics, as described in the case study in this paper, the architect has a set of indicators on the impact of each of his or her decision during the process of building an ISA.

However the authors recognized that much more testing on the metrics should be developed in order to assess its merit and significance. Currently, in other projects these metrics are being applied and improved.

The implementation of a tool for automatically evaluate ISA according to a set of qualities is also a planned future work.

Acknowledgements The case study research presented in this paper was possible thanks to the support of UMIC - Knowledge Society Agency.

References

Basili, V. R., L. C. Briand and W. L. Melo, A validation of object-oriented design metrics as quality indicators,"IEEE Trans. Software Eng. 22, 1996.

Bass, C., P. Clements, and R. Kazman, Software Architecture in Practice, Addison-Wesley, 1998

BEA, Scaling EJB Applications, available in http://edocs.bea.com/wle/tuning/tsejb.htm, access in May 2006.

Chidamber S., and C. Kemerer A metrics suite for object-oriented design. IEEE Trans. on Software Engineering, 20(6):476–493, 1995.

Clements,P., R. Kazman, and M.Klein, Evaluating Software Architectures: Methods and Case Studies, Addison-Wesley Professional , ISBN 02017048, 2002

Genero, M., G. Poels, and M. Piattini, Defining and Validating Metrics for Assessing the Maintainability of Entity-Relationship Diagrams, Working Paper, Faculteit Economie En Bedrijfskunde, 2003.

Inmon, W., Using The Generic Data Model , White Paper, 2000.

Page 33

McCabe, T. J.,`A Complexity Measure, IEEE Trans. Software Eng. 2, 1976.

OMG, Unified Modeling Language Specification, v 2.0, 2004, http://www.omg.org

Rito, A., Atributos de Qualidade, Presentation on the Software Architecture Course, Instituto Superior Técnico, Technical University of Lisbon, 2004 (in Portuguese).

Roulo, M., Java's three types of portability, available in http://www.javaworld.com/javaworld/jw-05-1997/jw-05-portability_p.html, 1997 (access in May 2006).

Sarkis, J. and R. Sundarraj 2003, Evaluating Componentized Enterprise Information Technologies: A Multiattribute Modeling Approach, Information Systems Frontiers 5:3, 303–319, Kluwer Academic Publishers, 2003.

Sousa, P., C. Pereira and J. Marques, Enterprise Architecture Alignment Heuristics, Journal 4, Microsoft Architects Journal, October 2004.

Vasconcelos, A., A. Caetano, J. Neves, P. Sinogas, R. Mendes, e J. Tribolet, A Framework for Modeling Strategy, Business Processes and Information Systems, 5th International Enterprise Distributed Object Computing Conference EDOC, Seatle, USA, 2001.

Vasconcelos, A., M. Silva, A. Fernandes, and J. Tribolet, An Information System Architectural Framework for Enterprise Application Integration, Proceedings of 37th Annual Hawaii International Conference On System Sciences (HICCS37), Hawaii, USA, January 2004b.

Vasconcelos, A., P. Sousa, and J. Tribolet, Information System Architectures, Proceedings of Business Excellence 2003, International Conference on Performance Measures, Benchmarking and Best Practices in New Economy, Guimarães, Portugal, June 2003.

Vasconcelos, A., P. Sousa, and J. Tribolet, Information System Architecture Evaluation: From Software to Enterprise Level Approaches, 12th European Conference On Information Technology Evaluation (ECITE 2005), Turku, Finland, September 2005.

Vasconcelos, A., R. Mendes, and J. Tribolet, Using Organizational Modeling to Evaluate Health Care IS/IT Projects, Proceedings of 37th Annual Hawaii International Conference On System Sciences (HICCS37), Hawaii, USA, January 2004a.

W3C, World Wide Web Consortium, Web Services, 2002, http://www.w3.org/2002/ws/

Documents

Information System Architecture Metrics: An Enterprise