Co-Evaluation of IT Value as an Activity for Effective Project Appraisal at Ex-Ante Stage

Dawit Asmelash Informatics Department

University of Pretoria Pretoria, South Africa

Dawit.asmelash@up.ac.za

Abstract-The business benefits of IT projects are

becoming the main determining factor for selecting

projects at the ex-ante justification stage. At this

stage, the identification and measurement of benefits

is usually abdicated to business management and IT

professionals support as technical advisors. However,

there is still on-going evidence that shows that

organisations have not been able to appropriately

evaluate IT benefits. This paper highlights the

importance of close collaboration between business

managers and IT managers for effective and

appropriate IT benefit evaluation at the ex-ante

justification stage. Activity theory was applied as

analytical model to understand and explain the

dynamics of the activity in pursuit of achieving

improved outcome. The activity analysis sees the IT

project benefit evaluation as a systemic entity that

consists of elements with shared motive for effective

identification and measurement of IT value. The

paper presents a case study in a large academic

institution to assess the nature of joint participation

of evaluators for effective IT project appraisal and to

identify the desired roles and responsibilities needed

for effective IT benefit evaluation.

Close collaboration and partnership between users

and IT professionals is shown to be a crucial

component in the justification process. The roles and

responsibilities of IT management exceeds beyond the

task of technical advisors. New roles and

responsibilities are proposed to resolve some of the

challenges faced with the current justification process

in the organization. The paper provides plausible

insights for IT project evaluation research and for

practitioners that aim to improve their benefit

evaluation.

Keyword-IT benefit evaluation, IT benefit evaluators, roles and responsibilities, ex-ante justification

I. INTRODUCTION

The established underlying business principle suggests that during the proposal and project definition stage (ex-ante justification stage), IT projects that have greater expected business benefits should be ranked high and selected for development to ensure success from IT investments [18,36,38]. However many organizations still struggle to evaluate benefits effectively. The recent

empirical investigations in different parts of the world show that less than 35% of the respondents claim to be successful in identifying available benefits for the projects and only 31 % believe they quantify benefits adequately [28,39,25].

The main reason that benefits are evaluated seems to gain project approval [39, 25]. The IT project proposals take a form of business cases with explicit statements showing how the IT project will contribute towards the achievement of the business goals through the changes it will enable in the business. These contributions are expected to be measurable or at least observable in terms of the business objective in the justification process [17, 26, 8].

Benefit evaluation has become a key business management issue. The users (business) is usually charged with the responsibility of identifying and measuring the business benefits as they are assumed to know better the business operation, the need for IT, and the outcomes of IT enabled change in their own business [18, 32].

In most cases, while business management defines business objectives and the expected improvements (changes), the IT management act as technical advisors at ex-ante justification stage [18, 6, 4]. The IT management's business analysts come to the fore during the requirement stage to analyse the project feasibility for development [12]. Often it is in the latter stages of IT projects initiation such as in requirement and specification stages that close collaboration between business management and IT management is seen to occur [30]. A close collaboration between IT and the business management, with clear and unambiguous roles and responsibilities is key requirement of effective IT governance [17].

Value delivery is one of the focus areas of IT governance. It is claimed that effective IT governance is the most important predicator of the value generated from IT. Despite high adoption rates of formal IT investment appraisal methodology in organizations, most organizations often fail to identify and measure benefits appropriately and as a result the evaluation may depend and reflect arbitrary values for benefits [31, 23, 7, 6]. These vague statements of benefits may lead to an unnecessary allocation of resources that may not support the organization strategy.

1

IT evaluation researchers continue attempting to resolve the problems and challenges faced in benefits evaluation [4]. There are on-going studies conducted to provide practical-oriented best practices and processes that are necessary to identify, create and optimise benefits in the organizations. One aspect of effective justification process is assigning and mandating the roles and responsibility of users and IT professionals for better execution of key activities during benefit evaluation [18]. There are studies advocating joint participation of business management and IT management in defining the project objectives in the justification process [18, 31]. With high rate of benefit leakages, there is a need to re-examine the role of IT management during the justification process to improve benefit evaluation. So far, little has been done to define explicitly the roles and responsibility of benefit evaluators at ex-ante stage.

II. THE CONTEXT

The organization is an academic institution - one of the leading universities in Africa, The university'S IT service department (IT management) has proposed "IT projects prioritization method" that is intended to be effective in ranking and selecting discretionary IT projects for development. The IT project prioritization method currently used is adopted from Gartner's project portfolio management processes [1], and Murphy's practical guide for achieving business value [27]. These are popular IT project selection and benefit evaluation approaches, and the method in the organization is intended to provide an adequate basis for IT investment decisions in the organization. The IT benefit evaluation is conducted to select IT projects for discretionary investment portfolio.

The business management as IT project proposers and evaluators are expected to identify and define their own business process that they expect to be improved through their IT requests. After specifying the business processes they are expected to provide the business process improvements and expected changes.

However, the actual IT projects justifications from business management show that many benefits go unrecognized or poorly identified and measured to enable appropriate decision on the requested systems. Consequently, there is a clear rationale to assess the main factors contributing to ineffective benefit evaluation. One factor identified is the relationship and communication gap between business management and IT management during the justification process, which becomes the focus for the research, and explored in the paper. More

specifically, the following two research objectives are established:

•

•

To assess the nature of joint participation of users and IT professionals in the justification process for effective IT project appraisal.

To identify desirable roles and responsibilities of IT professionals for effective IT benefits evaluation.

In addressing these objectives, it was envisaged that the paper makes an important contribution to the literature by providing insights on benefits evaluation practices in IT project justification at ex­ante stage

III. LITERATURE REVIEW

Business management have the responsibility of defending their claim used to justify the IT projects [36, 8]. When the roles, responsibilities, and accountability of the business management and IT management are unclear, IT management tends to determine which IT projects should be pursued and prioritized based on their limited business insights [18]. IT professionals as technical supporters with their protagonist the business as IT project evaluators has been widely accepted by many IT evaluation researchers [4, 1]. However, the process and the content of the evaluation extend far beyond the professional capacity of IT experts [21, 13].

Close collaboration between the IT management and other business management is crucial in the justification. ITGI [18] points out that "when partnership is absent, communication suffers, inefficiencies mount, synergies fail to emerge and the IT management engagement in the proposal become too late to contribute significant value". Therefore, benefit evaluation is best conducted if the evaluators comprise representatives of both the business and the IT professionals [38].

Effective evaluation means understanding and taking seriously the perspectives of individual stakeholders and interest groups [36]. Following this interpretive approach, the main purpose of evaluators is proposed to achieve understanding of the IT project's impact to organization and to legitimize the decisions for IT projects [1]. Serafeimidis & Smithson [33] and Walsham [38] also argue that evaluators need to have deeper understanding of the case to generate motivation and commitment of a wide range of stakeholder groups for change. Remenyi and Sherwood-Smith [33] further proposed that the objectives of IT

2

projects should come as a result of evolved ideas among business and IT professionals through an iterative dialectic process. However, in most organizations, the participation of IT business analyst begins at the feasibility study stage rather than the earlier stage of proposal and justification [12, 16, 18, 37].

Close communication and joint participation in the justification process is necessary, otherwise the true value of the projects may not be defined appropriately through the limited insight of a specific group of stakeholders that may dominate the process. The inadequacies of traditional evaluation methods tend to direct IT project evaluation towards interpretive approaches. The IT professionals as part of evaluators are expected to adapt new roles of being facilitators, learners, and communication agents to understand deeply the views, concerns and issues of stakeholders and to assess quantitatively or qualitatively the IT projects under evaluation [35, 33, 2, 15, 38]. Although the interpretive approaches are prescribed to assess whether the right project is selected for development, so far its acceptability to actual management practice remain elusive.

Assessing how the relationship between business management and IT management affects the benefit evaluation and identifying the necessary roles and responsibility is seen one of the requirements for effective IT benefit evaluation. Thus, to extend the quest for better benefit evaluation, benefit evaluators' roles and responsibility is the area III

which this research attempts to examine.

IV. THE EMPIRICAL RESEARCH

The aim of this section is to briefly describe the research methodology utilized.

A. Methodology

An explanatory case study with a qualitative research is seen necessary to investigate the contradictions that exist between users and IT management and to get a deeper understanding of their perceptions and actions in the process [29, 31]. Formal interviews with semi-structured questions were conducted to managers (users) from II business units that are involved in the benefit evaluation process. Observations and several informal discussions were also conducted with senior managers of IT service department to obtain a deeper understanding of the organization context and the IT project justification and selection process.

It is necessary for the analytical model to be based on a sound theoretical basis and Activity theory is seen appropriate for this specific research. Activity theory incorporates different aspects of human activity that arise from collective intentions and an activity is regarded as a systemic entity that comprises a number of elements which must fit together to achieve a desired outcome [9, 20], The basic unit of analysis in this research is the actions of users and IT management in identifying and measuring the IT benefits to come up with valid and justifiable IT value [10, 24]. The misfits within and between activities are the contradictions that deter the activity from achieving the desired outcome. The analytical model takes a collective actor's perspective and is applied as a tool for explanatory and developmental analysis. Therefore, in Activity theory terminology the first objective of this research become to investigate the contradictions that exist between users and IT management within IT benefit evaluation activity at the ex-ante justification stage. Activity theory is useful for analysing the different elements relationship in any activity [10, 24, 28]. Concepts and vocabularies from the theory are applied on the analysis of this research. A code descriptive qualitative data through the combination of hermeneutics process [14, 41] and content analysis [23, 21] is used as a complementary for the activity analysis.

V. ANALYSIS OF THE IT BENEFIT EVALUATION

ACTIVITY AT EX-ANTE STAGE

The analysis utilizes the concept of the zone of proximal development [44]. The concept entails that it is possible to plan future activity only through understanding the present activity [9]. Therefore, there are three phases in the analysis; (A) identifying and structuring the central activity, (8) analysing the present state of the activity, and (C) restructuring and developing a working activity.

A. The IT benefit evaluation activity elements

The central activity is the IT evaluation performed by the IT evaluators that attempt to define and justify the value of the IT systems they are requesting. The IT evaluation activity is performed to produce clear, understandable and justifiable IT values of the IT projects (requests). A proper identification and measurement of "IT benefits" is needed to construct the worth (value) of the IT projects. In justifying the IT projects, both the users and IT professionals are involved to achieve the activity outcome. The businesses (users) are expected to identify and measure the benefits, and it is the duty of the IT management to support the

3

business in the use of justification instrument (IT request form). The IT management do not participate in the actual benefits identification and measurement process but add the technical feasibility of the project in the justification form .

An illustration of the activity is shown in Figure 1. The users' action and goal is represented as UA and UG respectively, where else the IT management is represented as IA and IG.

Mode of

operation ."".---- - --- ..........

, �------------------------� , , ,

� , � ,

/ ' Colle�ve "

Object \ \

\ \

.. � I

I

Project Evaluation

Activity

Outcome

\ I�----------�I Means of Work -

\ I \

*Skills and expertise in respective fields I , * Justification form, etc. I

, Means of coordination and communication / / '*�ustification Form *Division of labour ;

, � , '

, " ' - --.... _ - -- --

Figure 1. IT benefit evaluation activity

B. The neighbouring activity - Project evaluation activity

The main activity in this distinct case is examining and comparing the IT value of each project (request) to make judgment of which request to be approved, deferred or rejected. The outcome of the IT benefit evaluation activity is the object in this activity. In evaluating the IT projects, the group of senior business managers that is composed of finance, strategic and operation departments will assess the financial implication, realization of value, and the strategic alignment of the requested IS systems. The IT group that consist of senor IT

managers will examine the infrastructure compatibility, the IS schema of the organization, the technical capability and so forth. These two groups collectively analyse the potential values of each IT projects, and make a decisions (select) the projects for development that are seen more beneficial to the organization. The activity is illustrated in Figure 2. Since the emphasis of this research is on the IT benefit evaluation activity, the next section will present the current state of the activity.

4

IT system (Potential project)

Means of coordination and communication

* Justification Form *Division of labour

Figure 2. Project Evaluation Activity

C. The history of the IT evaluation activity

A work activity is supposed to be systemic in nature to achieve a mode of operation. [9, 28]. In this phase, the elements jointly are analysed to detect the misfit within and between the various elements, the overall mode and network relations. These contradictions can manifest as tensions, defects, challenges and conflicts [10, 20, 24]. There are four types of contradiction identified in the activity; (1) inner contradiction within users (2) contradiction between users and IT management (3) contradiction on the object the activity, and (4) contradiction with neighbouring activity (on activity network).

1) Inner contradiction within the users: The high number of IT projects competing for allocated budget affected the respondents' perception towards the outcome of their requests. Most participants repetitively expressed low expectation towards their IT request to be approved. The response R I indicated that low expectation approval can be a contributing factor for poor IT evaluation. It became evident that (see Table I, R2) that the lack of adequate awareness of IT benefit concepts among users contributed for ineffective benefit identification and measurement. This implies that that business management should demonstrate responsibility and commitment to learn more how to justify.

TABLE I: INNER CONTRADICTIONS WITHIN USERS

Ref Text: Excerpts of Transcripts

RI [ ... J We had some proposal that were rejected and most of our HODs before they come up with IT needs, they believe that it might be rejected and some may not even think to fill in on the form [ ... J/Faculty C Manager}.

R2 The filling of the form is not clearly understood and clearly communicated to all of us. I attend the meeting held but since all faculty and department were invited ( ... ) there was no way for each unique departments know how exactly to fill, and what is expected [ ... J/Client Service Centre

Manager}.

2) Contradiction between users and IT management: There is a tension between the users and IT management because of lack of proper feedback and transparency reduces the effort of identifying and measuring benefits properly. The

Description (Text Analysis) Interpretation

There is low expectation of The benefits are not project approval among users and thoroughly identified it led users to recline from because there is no will to justification do so. Low expectation of

approval have negative effect on benefit i denti fi cati on

Almost half of the respondents do There is a need for more not know how to identify and presentations and training measure IT benefits. in justifying projects.

evaluators lost interest towards the justification process because of late responses and non­transparent reports on the status of their requests

5

TABLE 2: CONTRADICTION BETWEEN USERS AND IT MANAGEMENT

Ref Text: Excerpts of Transcripts

R3 [ ... J to give proper feedbacks why our requests fall short.. Because you sit sometimes and feel is it really worthwhile completing these forms ... because we never get proper feedback [ ... J (Faculty D Manager{.

3) Contradiction on object and motive: The users seem to focus on the problems and risks facing their business operation if the requested system is not provided to them; rather than making the benefits visible in the benefit evaluation activity. The act of filling the justification form was supposed to converting the "needs" and "problems" into benefits for project prioritization purpose. It

Description (Text Analysis) Interpretation

Most respondents do not get The lack of good formal feedback on time about communication on the the status of their request from IT outcome of the users organization request has affected their

will to justify and completing the form.

was observed that most respondents associate the justification form with expressing the IT "needs" and "requirements". This shows the level maturity in the organization with regards to evaluations based on benefits. As the next excerpts shows, the culturally advance object is different than the desired dominant object in the activity,

TABLE 3: CONTRADICTION ON OBJECT BETWEEN THE DOMINANT FORM AND CUL TURALL Y APPLIED

Ref Text: Excerpts of Transcripts

R4 I believe we focused on what is needed by the faculty and assume that it improve specific function within the faculty, Considering the form again, it is confusing what was required[ ... J/Faculty A Manager}

"But this jargon (expected process output) etc put In the form, .. it is not applicable for us, we need the application [IT system J and we requested to be delivered otherwise we will have a problem, they should understand that [ ... J

/Facility Management Manager}

4) Contradiction with the neighbouring activity: The response of Rl in table 1 reveals how low expectation of IT project approval among users has caused them change their behaviour. While the inner contradiction focus on the actions of individuals or groups within an activity, here the emphasis is on the link between the activities where the outcome of IT project evaluation activity causes friction and un-stabilised the benefit evaluation activity. The solution for Rl and R3 requires not only activity level analysis but organizational level analysis that goes beyond the boundary of this study and concerns with IT governance. However, it indicates that there is a contradiction among networked activities. The decision made in IT project evaluation activity and the actions of IT management affects the activity of IT benefit evaluation to achieve the outcome of delivering justifiable IT value.

A summary of the contradictions in the IT benefit evaluation is graphically illustrated in the following cause and effect diagram to depict and explain why the problem occurred. In the cause-and-effect

Description (Text Analysis) Interpretation

Although benefits evaluation is Most of the participants as performed only for IT project benefit evaluators do not justification, most of the business understood the full extent managers attempt to make the of IT benefit concept. risks more visible than the benefits.

diagram the "effect" is the problem identified in this research, which is "inappropriate IT benefit identification and evaluation". The diagram illustrates the relationship between the problem and the contributing factors to the problem. The major categories of the diagram are the main entities or elements of the IT evaluation activity. The "Benefit" and "Measurement" labelled categories describes the causes for the contradictions on identifying and measuring the benefits.

For example, under "Measurement" category, users overestimate in measurement due to fear of IT project rejection. The "People" category refers to users' inner contradiction, and the "Communication" category illustrates how the friction between the actors (users and IT group) affects the IT evaluation activity. The detailed levels of the causes are identified and organized on the related categories. For example under "Communication" category, the second cause "hesitant to justify" is due to users "lack of interest" in identifying benefits because there was "no proper feedback" from previous attempts. All

6

these factors contribute to negative effect on benefit evaluation activity and consequently in

Business

project evaluation activity.

Focus on risk, needs and requirement

2. Ove rsight benefits \111[-----........... in justification objective not ----IIi\.

specified Difficult at this stage A

;�: p�fr::ro��t;;

t .... _.\ 3. Intangibles are '\IlI:--"&!"-...... I--.. neglected

1. Flawed rate

2. No outcome indicators -......... -1"-

Overestimation 1. Actual benefits

cannot emerge

Poor contact /�i--""'; with IT

1. Make risks visible, but not benefits

Not attended presentations

No information

INAPPROPRATE IT BENEFITS EVALUATION

No feedback

limited funds sharing ---t-/

Jl41-..... 1Iioo-... 2, Lack of motivation to justi'fy

COMM U NICATON lack of Interest PEOPLE

Figure 3. Cause-and-Effect diagram showing identified factors contributing to inappropriate IT benefit evaluation.

D. Desirable improved mode

The current state of analysis has revealed the contradiction between the elements in the activity. Drawing from the findings, this phase will develop the desirable improved mode of operation. The users group confirmed the importance for IT personnel participation in their justification activities and implied additional roles and responsibilities to the IT group in the IT evaluation activity to achieve the outcome. The nature of the roles and responsibilities is shown in Table 4.

In Table 4 below, it becomes apparent from the responses from Rl that the current role of IT professional as technical expert remains as it is and crucial in the activity. It emphasizes the users' acknowledgment for more and close contact with IT experts to acquire technology knowledge. This

confirms the literature that IT organization supports as technological advisors. R2 shows while most of the users know that IT management can assist by providing the IT schema of the organization and the reality about the actual IT resources, it was observed that most users do not inquire about them during justification stage.

In regards to RJ, there is a change in the role of the IT experts, and that is they should also collaborate in identifying the benefits of the IT systems with users. This supports Remenyi and Sherwood-Smith [28] proposal that the objectives for IT projects should come as a result of evolved ideas among business and IT professionals. This entails that IT experts must jointly participate in recognizing the benefits at justification (ex -ante) stage which is uncommon practices in many cases [29].

7

TABLE 4: IT MANAGEMENTS ROLES AND RESPONSIBILITIES

Ref

Rl

R2

R3

R4

R5

R6

R7

Text: Excerpts of Transcripts

I think we need to improve our communication with IT service department, to get full advantage of IT to become more

competitive. We need to involve more of our departments in this regard. There might be many possibility we don't know that IT systems can make us achieve our goal [ ... J.IC/ient

Service Centre, Manager]

[ ... JI want to know what they can provide, so that when we request IT system, we know the limits [ ... J actually we are not updated [ ... J/Faculty B Manager]

But I think a Business analyst ..... there should be someone to share and guide us to identify the benefit and to fill the form [ ... J [Library Service]

[ .. J We need somebody from IT service department to sit down with us ... and make discussion, and they can advise us in some issues and improve the quality of the document in that way ... before we actually submit them [ ... J /Sports Services

Manaf(er/

[ ... J Listing the major benefits, improved operation efficiency . .. but what does it means to everybody,... I mean it is very easy to put stuff here , that sounds good.. anybody that express themselves well may look very impressive.. I can write a nice motivation and justification here, .... we going to have a good picture of the client, .. How does IT Service department knows that it is true? [ ... J/Faculty D Manager]

[ ... J especially before we submit the justification form, the IT people should be present to understand our ideas and needs

[ ... J/Facilities Management /Deputy Director]

IT personnel should be able to translate our request and our information needs and our technology needs to them [ ... J he will have a better idea and will be a sort of our defense [ ... J

/Client Service Center Manaf(er/

To give us the broader pictures as well as to explain to us why the request from other faculty is more important than us [ ... J/Faculty E Manager]

The contradiction related to users focusing on the risks and the problems (R4) of business operations as a means of justification than making the benefits visible in evaluation can be resolved through IT experts (or business analyst) participating in the reformulation of the risks into benefits. Although, there is a general presentation/demonstration held how to use the justification form, most suggested they need one to one interaction. This will address the contradiction between the cultural practice and dominant motivation. It also resolves the contradiction between the two activities where the outcomes of IT benefit evaluation become cumbersome to make the selection.

Some suggested that there should be a way of authenticating the benefit estimation. The respondents (RS) are concerned that others might put impressive figures that might reflect more than the actual value and it might affect the chance of

Description (Text Analysis)

Recognition of close cooperation IS needed. The users need to acq uire knowledge of IT trends to harness opportunities.

The users want to understand IT organization resources. The IT schema of the organization is not fully known by the users thus they need to know if their IT requests are acceptable. The users need to be informed and updated on the technical capacity of IT service so that they can refine their requests for approval.

The users have problem in identifying expected benefits from the IT systems. They need IT professionals to share ideas and collaborate to identify benefits The benefits expressed in the document (means of communication) may not be up to standard for decision making process .. (To reduce conflict III network of activities) The changes and Improvement rate enabled by IT system (the benefits) may be overestimated for approval purpose. There is a need for someone to authenticate the claims.

User department believe that the selection committee (them) do not know deeply the current users situation and IT personnel (he) must involve in the actual benefit identification process to enlighten them

There are frequent rejections of IT requests, the business want to learn why not and prepare better requests in the future. They need the IT personnel to provide feedback with explanation

Interpretation

Technological consultant

Resource Informant

Identifier

Coach

Inspectors

Advocate, defender

Agent of Transparency.

their requests to be selected. The IT professionals participation is needed to take the role of inspectors on the measurement of the value.

More than half of the participants indicate that they are not fully confident whether the selection committee clearly understood the need and the impact of the proposed IT project from the justification document alone. Some expressed that they would also prefer to demonstrate orally the justifications to clarify the nature and benefit of the IT project. It is evident from response (R6), the respondent seeks an advocate that can translate their needs in the subsequent activity (IT project evaluation activity). Not only the justification form is going to be the means of network between the activities but an IT personnel will be used to clearly convey the IT values to the decision makers.

8

Most of the respondents are discontent with the way the feedback of their IT request is conducted. The last response (R 7) is also concerned with contradiction between networks where in the dialectic relationship between activities, the outcome (selection) of [T project evaluation being not communicated well and on time to the users. The empirical evidence show that there is a need of change between the activities, in that the "means of network" or the link should be in form of IT personnel to be a liaison to act as transparency agent on the outcomes of the projects evaluation activities.

E. Restructure of activity based on dynamics in IT evaluation activity: Co-evaluation

The fmdings of this study confirm that IT management participation is crucial in the

justification process. [t is shown that the absence of [T management participation in benefit identification and measurement results in unreliable and inaccurate representation of IT projects benefits. Concepts drawn from the analysis will change the IT evaluation activity as it is shown in Figure 4. The changes are seen necessary to balance back the misfits caused by the contradictions. In this proposed roles of IT evaluators, there are two noticeable changes observed from the old activity. The [T management group is jointly executing the actions and goals that were abdicated only to users (see Table 5, R[ to R5). The second is experts from [T management also act as a means of network between the two activities (see Table 5, R6 and R7). The joint participation of users and IT management in identifying and measuring IT benefits is what this research call co-evaluation.

Means of Network: Professional from IT Management

Figure 4. Desired Activity of IT benefit evaluation with Project evaluation activity

VI. DISCUSSION AND RECOMMENDATION

This study shows that the IT management should consider participating actively in defining and measuring benefits in the IT benefit evaluation activity. In practice this can be achieved by assigning IT professional that is charged with the role of an informer, trainer, identifier, advisor, inspector, advocator and agent of transparency during the justification process. The [T personnel is not only expected to be technology oriented but must know well the business context, culture, and environment where the evaluation is carried out.

The activity analysis shows that [T personnel should be responsible in clarifying the [T requests

to the selection committee. This is expected to support the selection committee to minimize subjective decisions and make it more objective in selecting [T projects.

The business management defme the business objectives and the IT management collaborate in identifying and measuring the benefit of the proposed projects. The ValIT framework [17] recommends joint participation in benefit evaluation. This paper defines explicitly the needed role of IT management for practical application of benefit evaluation at ex-ante stage. The users and [T management are expected to co-evaluate quantitatively or qualitatively the benefits as seen appropriate by [T investment decision committee to

9

achieve legitimate decisions in allocating resources for IT projects that have greater potential benefits.

The justification process should not be confined to identifying and measuring benefits. In this research a more deterministic analysis has identified the imbalances in the activity and proposed the desired improvement to benefit evaluation. Further studies should be conducted to show how co-evaluation be more effective by incorporating more softer issues that include the views, issues, concerns and behaviours of all interested stakeholders by applying social, psychology, behavioural theories for deeper understanding, and to explore for new platforms for collaboration in dialectic approach suitable for IT benefits evaluation at ex-ante stage [3,38].

VII. CONCLUSION

This paper highlights the need for co-evaluation of business and IT management in benefit evaluation activity at the justification stage. Activity theory was applied to analyse and to identify the apparent contradiction within the IT evaluation activity and with its neighbouring activity. The need for close communication and collaboration is confirmed by business management's expectation from IT management. New roles of IT management in the evaluation process are expected to address some of the factors that contribute to ineffective benefit evaluation. This include lack of awareness of IT benefit concepts, lack of skill in benefit evaluation, lack of appropriate feedback and low expectation of approval that are associated with the collaboration and communication gap between business and IT management.

It is proposed that IT management to assign IT professionals to execute the roles proposed by the business management. The task of IT professional is more likely to be the function of a business analyst but with the added task of "IT benefits

REFERENCES

[1] Apfel, P. 2007. PPM processes: keep the people in and the complexity out. Gartner Symposium/ITxpo, Gartner Inc.

[2] Avgerou, C. 1995. Evaluating information systems by consultation and negotiation. International Journal of Information Management, 15: 427-436.

[3] Avolio J. Kahai S. & Surinder. 2003. Adding "E" to E Leadership: How it may impact your leadership. Organizational Dynamics, 31(4): 325-338.

analyst". In most organization, business analysts are usually involved at feasibility and requirement stage for the development of the projects. This research is recommending that they should participate at the project definition or initial ex-ante justification stage, and execute the proposed roles and responsibilities for effective benefit evaluation.

This research also indicates user's low expectation of approval has an implication for close communication not only between users and IT management but a close communication roles between IT management, users and the executives (selection committee) of the organization. This relates to IT governance in mandating the proposed roles and the responsibilities, portfolio management, strategy and resource allocation which requires not only group/activity level of analysis but organizational level in order to effectively resolve the challenges of benefit evaluation. Further research at organizational level analysis should be performed using institutional theory to complement the limitation of activity theory analysis.

In this research, there are two implication related to e-Ieadership for effective IT benefit evaluation. The first relates to IT management taking leadership responsibility on the business operations and not abdicating the roles to the business management alone. The IT management should take advantage of the existing technologies for collaborative and group evaluation that is necessary for joint participation or co-evaluation of IT benefit. The second related to the executives of the organization to be responsible in assigning the roles and responsibilities that are required for benefit evaluation, and to be informed and be responsive to the ingoing concerns of their human capital. Good leadership through information and communication technologies may facilitate and enhance co­evaluation and is necessary for effective IT benefit evaluation.

[4] Berghout, E. & Remenyi, D. 2005. The eleven years of the European Conference on IT Evaluation: retrospectives and perspectives for possible future research. The Electronic Journal of Information Systems Evaluation, 8(2): 81-98.

[5] Berman, J. 2007. Maximizing project value: defining, managing, and measuring for optimal return. New York: American Management Association.

[6] Cronk, F. & Fitzgerald, E.P. 2002. Constructing a theory of 'IS business value' from the literature. Electronic

10

Journal of Business Research methods, 1(1): 11-17.

[7] Counihan, A. , Finnegan P. & Sammon, D. 2002. Towards a framework for evaluating investments In data warehousing. Information Systems Journal, 12: 321-338.

[8] Dhillon, G. 2000. Interpreting key issues In IS/IT benefits management. Proceedings of the 33rd Hawaii International Conference on System Sciences. [Online]. Available: http://ieeexplore.ieee.org/stamp/stamp.jsp? tp=&arnumber=926926 [Cited 12 June 2010].

[9] Engestrom Y. 1999. Activity theory and individual and social transformation, in Y. Engestrom, R. Miettinen & R. Punamaki (eds). Perspectives of Activity Theory. , Cambridge University Press , 19:38.

[10] Engestrom Y. & Engestrom R. 1986. Developmental Work Research: the approach and Application. in Cleaning Work, Nordisk Pedagogik 12-15,.

[11] Farbey, B. , Land, F. & Targett, D. 1993 How to Assess Your IT Investment. A Study of Methods and Practice. Oxford: Butterworth Heinemann Ltd.

[12] Farbey, B. & Finkelstein, A. 200l. Evaluation in software engineering: ROJ,

but more than ROJ. [Online]. Available: http://eprints. ucl.ac. ukl7 43/1/ 1.6_ 00948546.pdf [Cited 12 March 2010].

[13] Frisk, E. 2007. Categorization and overview of IT evaluation perspective. A literature revIew. Proceedings of European conference on Information management and evaluation. Edited by D. Remenyi. Montepellier: Academic conferences Limited. pp. 159-170.

[14] Gadamer, H. 1994. Truth and method. In Weinsheimer, J. and Marshal, D.G. New York: Continuum. (Original work published 1975).

[15] Guba, E.G. & Lincoln, Y.S. 1989. Fourth Generation Evaluation. Newbury Park, CA: SAGE Publishing.

[16] Irani, Z. & Love, P.E.D. 2002. Developing a frame of reference for ex-ante IT/IS investment evaluation. European Journal of Information Systems, 11(1): 74-82.

[17] ITGI. 2007. Control Objectives for Information and related Technology (COBIT 4. 1). Illinois: IT Governance Institute.

[18] ITGI. 2008. The ValIT Framework 2. 0. Enterprise value: Governance of IT

Investments. Illinois: IT Governance Institute.

[19] Korpela, M., Soriyan. H. & Olufokunabi, K.C. 2000. Activity analysis as a method

for information systems development: General introduction and experiments from Nigeria and Finland, Scandinavian Journal of Information Systems, 12(1): 191-210.

[20] Korpela, M., Mursu, A. , Soriyan. H. & Olufokunabi, K.C. 2002. Information systems development as activity, Computer Supported Cooperative Work, 11(1-2)

[21] Kumar, R. 2005. Research Methodology.

A Step-by-Step Guide. 2nd ed. London: Sage Publication

[22] Lagsten, J. & Goldkuhl, G. 2008. Interpretative IS evaluation: results and uses. The Electronic Journal Information Systems Evaluation, 11 (2): 97-108.

[23] Leedy, P.O. & Ormrod, J.E. 2005. Practical Research: Planning and Design, 8th ed. New Jersey: Pearson Education.

[24] Leont'ev, A.N 1978. Activity, Consciousness and personality. Englewood Cliffs, NJ: Prentice Hall.

[25] Lin, c., Huang, Y. & Cheng, M. 2007. The adoption of IS/IT investment evaluation and benefits realization methodologies in service organizations: IT maturity paths and framework. Contemporary Management Research,

3(2): 173-194. [26] Lin, C. , Pervan, G. & McDermid, D. 2005.

IS/IT investment evaluation and benefits realization issues in Australia. Journal of Research and Practice in Information Technology, 37(3): 235-251.

[27] Murphy, T. 2002. Achieving business value from technology. A practical guide for today's executive. Gartnerpress: John Wiley & Sons

[28] Mursu, A. , Luukkonen, I. ,Toivanen, M .& Korpela, M. 2007. Activity theory in information systems research and practice: Theoretical underpinning for an information systems development model. IR Information Research, 12(3).

[29] Myers, M.D. 2009. Qualitative Research

in Business & Management. London: SAGE publication.

[30] Naidoo, R & Palk, W. 2010. Evaluating the IS/IT Benefits Management Practice in South African Organizations. Proceedings of the International Conference on Information Management and Evaluation, Cape Town, South Africa.

[31] Oats, B.J. 2006. Researching Information Systems and Computing. London: SAGE publication.

[32] Peppard, J. , Ward, J. & Daniel, E. 2007. Managing the Realization of Business

11

Benefits from IT Investments. MIS Quarterly Executive, 6(1): 1-11.

[33] Remenyi, D. & Sherwood-Smith, M. 1999. Maximise information systems value by continuous participative evaluation. Logistics Information Management, 12(1/2): 14-31.

[34] Serafeimidis, V. , & Smithson, S. 1998. Rethinking the approaches to information systems investment evaluation. Logistics Information Management, 12(1/2): 94-107.

[35] Serafeimidis, V. , & Smithson, S. 2003. Information systems evaluation as an organizational institution - experience from a case study. Information Systems

Journal, 13: 251- 274. [36] Symons, V.J. 1993. Evaluation and the

failure of control: information systems development in the processing company. Accounting, Management and Information Technology, 3: 51-76.

[37] Thorp, J. 2007. The Information Paradox: Realizing the Business Benefits of Information Technology, Revised edition. Toronto: Fujitsu Consulting.

[38] Walsham, G. 1995. Interpretive case studies in IS research: nature and method. European Journal of Information Systems, 4: 74-8l.

[39] Ward, J. & Daniel, E. 2006. Benefits Management: Delivering Value from ISIIT Investments. Chichester: John Wiley & Sons Ltd.

[40] Ward, J. , Hertogh, S. & Viaene. 2007. Managing benefits from IS/IT investments: an empirical investigating into current practice. Proceedings of the 40th Hawaii International Conference on System Sciences. [Online]. Available: http://www .computer. org/portal/web/ csd 1/ doiIl0.ll09IHICSS.2007.330 [Cited 16 July 2010].

[41] Webb P. & Pollard C. 2006. Demystifying a hermeneutic approach to IS research. Australasian Journal of Information Systems, 13(2): 31- 48.

[42] Yin, R. 2003. Case Study Research, Design and Methods. 3rd ed. London: SAGE Publications.

12