Evaluating Performance in Information Technology

EvaluatingPerformance inInformationTechnology

By

Marc J. Epstein

and

Adriana Rejc

MANAGEMENT ACCOUNTING GUIDELINE

Published by The Society of Management Accountants of Canada, the AmericanInstitute of Certified Public Accountants and The Chartered Institute ofManagement Accountants.

M A N A G E M E N T

S T R A T E G Y

M E A S U R E M E N T

Copyright © 2005 by The Society of Management Accountants of Canada (CMA Canada), the American Institute of CertifiedPublic Accountants, Inc. (AICPA) and The Chartered Institute of Management Accountants (CIMA). All Rights Reserved.

No part of this publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any means,without the prior written consent of the publisher or a licence from The Canadian Copyright Licensing Agency (AccessCopyright). For an Access Copyright Licence, visit www.accesscopyright.ca or call toll free to 1-800-893-5777.

ISBN: 1-55302-173-8

NOTICE TO READERS

The material contained in the Management Accounting Guideline Evaluating Performance in Information Technology is designed to provide illustrative information with respect to the subject matter covered. It does not establish standards or preferredpractices. This material has not been considered or acted upon by any senior or technical committees or the board of directorsof either the AICPA, CIMA or The Society of Management Accountants of Canada and does not represent an official opinionor position of either the AICPA, CIMA or The Society of Management Accountants of Canada.

3

INTRODUCTION

While some of the recent publications oninformation technology question thevalue creating role of IT in today’sbusiness environments (Carr, 2003 and2004), others assert that, with theeconomic recovery in many parts of theworld, innovation — especially ininformation technology — is becomingeven more critical to growth and highperformance.

With respect to the first, doubts aboutthe potential payoffs of IT investmentscan be traced to numerous IT projectsmade without the rigor of measurementof either the benefits or costs of such

investments. Decisions were made basedon compelling arguments and keeping upwith competitors resulting in billions ofdollars of wasted corporate assets.France Telecom, for example, announcedthat they spent 700 million euros onexternal IT services in 2003 which is afterslashing 138 million euros from the ITspending of the prior year (838 millioneuros). Almost every organization hasstories about unfulfilled promises aboutthe benefits of new ERP or IT systems. In the United States alone, annualexpenditure on IT now runs into trillionsof dollars, and approaches 50 percent ofnew capital investment for mostorganizations with little evidence to

E VA L UAT I N G P E R F O R M A N C E I N I N F O R M AT I O N T E C H N O L O G Y

CONTENTS EXECUTIVE SUMMARY

Though there has been significantdiscussion concerning the importance ofevaluating the payoffs of IT investment,there has been little guidance as to howto design or implement an appropriateperformance evaluation system. Thus,investments are often made without therigor of measurement of either thebenefits or costs of such investments.Typically costs are much higher thananticipated and the benefits are far lowerand harder to achieve.

Senior IT managers are also frustrated.They are convinced that, if measuredproperly, IT does create value.

This Guideline will develop an ITperformance measurement framework,articulate specific measures, describe thecausal relationship between variousdrivers and measures, and throughexamples, illustrate how companies canidentify and measure the payoffs of ITinvestments.

INTRODUCTION 3OBJECTIVES AND TARGET AUDIENCE 5ENSURING ACCOUNTABILITY IN

INFORMATION TECHNOLOGY 5IDENTIFYING THE OBJECTIVES

AND DRIVERS OF SUCCESSFUL INFORMATION TECHNOLOGY INVESTMENTS 6

DEVELOPING THE APPROPRIATE METRICS 15FURTHER SPECIFICITY ON CALCULATING

BENEFITS (OUTPUTS) FROM IT INVESTMENTS 20

MONITORING THE STRATEGIC IT OBJECTIVES, DRIVERS AND PERFORMANCE MEASURES 23

MEASURING THE DISRUPTION COSTS OF IT INITIATIVES 23

MANAGING IT INVESTMENT PROCESSES 24RECOGNIZING THE RISK ASSOCIATED

WITH IT INVESTMENTS 25ALIGNMENT OF THE IT CONTRIBUTION

MODEL WITH THE BALANCED SCORECARD, SHAREHOLDER ANALYSIS AND ROI 26

THE APPLICABILITY OF THE IT CONTRIBUTION MODEL AND MEASURES TO OTHER BUSINESS FUNCTIONS 30

GUIDANCE FOR MANAGERS 31BIBLIOGRAPHY 35

Page

S T R A T E G Y

M A N A G E M E N T

S T R A T E G Y


4

suggest that this expenditure has generated asatisfactory return (Murphy, 2002, Davenportand Prusak, 1997). Typically, the costs oftechnology are much higher than anticipated,the cost of conversion is also higher, whereasthe benefits are far lower and harder toachieve than expected. In addition, this ignoresthe very significant costs related to employeetime wasted and the disruption to personnel,operations, and the revenue stream of theorganization. While the claim that someorganizations collapsed because of ineffectiveIT policies may be an exaggeration, many ChiefExecutive Officers (CEOs) and business unitleaders view IT as a value destroyer or a costrather than a value creator implying itscorroding impact on the organization’scompetitive advantage.

On the other hand, the advocates of the boldand comprehensive new vision of howorganizations can use information technologyto create value believe IT matters more thanever, yet in a different way. By moving from anera of technology to an era of technologycapability, the focus has shifted from individualtechnologies to the benefits that can becreated with them. Because of this focus ontechnology capabilities, innovation is emergingnot just from technologists, but from the usersof the technology components whounderstand how to use IT to deliver higherlevels of organizational performance. By fillingthe gap between the rate of technologyinnovation and people’s understanding andability to use and implement the technology,organizations can use these innovations to leadto substantial improvements in theirperformance. Even for organizations that wereactually quietly making a big difference in theirmarkets by leveraging IT, it was still oftendifficult for these Chief Information Officers(CIOs) to quantify those results, and prove thebenefits. Then, when earnings declined, IT wasan easy target for cost cutting.

A primary reason for doubts about thepotential value organizations can derive fromexisting and future investments in IT is relatedto the absence of a proper methodology toevaluate the payoffs of IT investments. So far,there has been little guidance of how to designor implement an appropriate IT performanceevaluation system, i.e. how to identify anddocument the contribution of informationtechnology to high-performance organizations.Historically, organizations were driven by

enthusiastic managers who were over relyingon technology and did not demanddevelopment of the needed skills and themeasures to complete these analyses. Today,the financial managers and other decisionmakers want the IT requests to be framed in aROI or shareholder value format so that theycan be effectively compared with alternativepotential company investments. According tothe Forrester Report, 90% of executives makethe IT funding decisions based on the financialimpact of IT initiatives; however, the topchallenge in selecting which IT project to fundis the lack of objective data (Cameron et al.,2000). Senior IT managers are convinced thatthey do create value and believe that, ifmeasured properly and with adequate support,they would be significant profit centers fortheir organizations. But without adequateperformance evaluation systems, they havedifficulties proving the value adding role of ITand find themselves continually fighting for andjustifying the resources that are needed. CEOsand CFOs lack information to make wellinformed decisions on the payoffs of theseinvestments and, as a consequence, corporategoals seem to focus on reduction of the costsof IT rather than maximizing IT value creationactivities.

OBJECTIVES AND TARGET AUDIENCE

As IT managers must show the payoffs of ITinvestment to convince key executives thatthey should be strong supporters of IT efforts,a framework for evaluation of IT performanceis a significant need. Few things are moreconvincing to top executives than measurableresults. When a new project is proposed,additional funding is typically based solely onthe results anticipated from the project.Currently, IT executives do not have propertools to measure the payoffs of IT. Evenfinancial managers that have expertise inmanagement control and performancemeasurement, have not focused on thebenefits of IT and have not developed theappropriate measures. Consequently, thepayoffs of IT are not measured, ROI is notcalculated, and IT investments are notevaluated with the same rigor as othercorporate investments. The purpose of thisguideline is to provide a model and a selectionof measures for evaluating performance ininformation technology in both for-profit and

5

E VA L U AT I N G P E R F O R M A N C E I N I T

not-for-profit organizations to help CIOs betterjustify and evaluate their initiatives and aid CEOsand CFOs in making better resource allocationdecisions.

This Management Accounting Guideline’sobjectives are as follows:

● To develop a general model of key factorsfor organizational success in IT integration (ITContribution Model) that includes fourdimensions: the critical inputs and processesthat lead to success in IT outputs andultimately to overall organizational success(outcome).

● To articulate each of the key factors(antecedents and consequences of ITsuccess) as objectives to facilitate furtheroperationalization of the model.

● To outline the specific drivers of IT successbased on the objectives related to inputs,processes, outputs, and outcomes, andidentify the causal relationships between thedrivers.

● To provide the specific measures of ITperformance. Following the cause-and-effectrelationships between the drivers of ITsuccess, measures are developed to trackperformance of IT initiatives along the fourdimensions. The metrics can be used forboth IT project justification prior to its start(planning) as well as for evaluation aftercompletion (performance measurement).

● To provide examples of how to assignmonetary values to non-financial IT outputs(benefits). Although some benefits do notalways easily translate into short term profits,they should ultimately lead to either costsavings or increased revenues.

● To provide an example of how to calculatethe IT payoffs. Here, the guideline specificallyrecognizes the importance of measuring boththe total costs of an IT initiative — includinga range of different disruption costs — aswell as the benefits, and additionallyconsiders the risks associated with ITinvestments.

● To show how the IT Contribution Model isconsistent with other measurementframeworks such as the Balanced Scorecardand Shareholder Value Analysis.

The target audience of this guideline is theaccounting and finance professionals that dealwith the challenges of performancemeasurement and control in IT. Presented in a

systematic format, the guideline is also intendedto help CIOs, Chief Training Officers (CTOs)and senior IT managers better understand howinformation technology contributes to higherlevels of corporate performance, more easilyevaluate the profitability of IT investments, andmake better resource allocation decisions. Inaddition, it is also helpful for the CEOs, CFOs,and other decision makers that struggle toidentify, document, measure, and communicatethe short-term results and long-term impacts ofIT investments. This includes both cost savingsand value creation, and thus provides argumentsfor additional IT resources when appropriate.

ENSURING ACCOUNTABILITY ININFORMATION TECHNOLOGY

Typical large IT investments, E-commerceinvestments, and very large ERP systemimplementations are all faced with the samechallenges of demonstrating the value of theinvestment and historical difficulties in estimatingboth the revenues and total costs. This typicallyfalls on senior corporate and senior financialmanagers to evaluate the payoffs and makerecommendations on resource allocation —which are typically significant. But it is not onlygrowing budgets that make IT an importantexpenditure. The consequences of improper andineffective IT initiatives can be significant if weconsider the integration of the IT departmentinto the mainstream functions of theorganization. Also, IT infrastructure upgradesprovide vast potential to contain costs, enhanceproductivity, improve quality, spur innovation,and satisfy and retain customers.

With CEOs and CFOs demanding accountabilityfor the tremendous investment in informationtechnology, IT managers are required to ensureaccountability, calculate the return oninvestment, develop a value-added approach,and make a bottom-line contribution. At theheart of accountability is measurement andevaluation. IT managers faced with the challengeto demonstrate the validity of IT initiatives tothe business must find ways to measure andcommunicate the contribution of IT so thatviable existing initiatives are managedappropriately, new projects are only approvedwhere there is satisfactory return, and marginalor ineffective projects are revised or eliminated.To properly assess the payoffs of investments inIT, organizations must implementcomprehensive systems to evaluate the impactof IT initiatives on financial performance and the

trade-offs that ultimately must be made whenthere are competing organizational constraintsand numerous barriers to implementation.

However, with larger and more complex ITsolutions and with business process changestypically enabled by or driven by informationtechnology, IT projects should be viewed asbusiness change projects with IT componentsrather than stand-alone IT projects. That is,change management constitutes an importantpart of IT projects and the business value ofany IT investment must be clearly stated.

Business value is created through acombination of IT tools and businessprocesses, thus, there must be a cleardistinction between IT and businessaccountabilities. IT managers are accountableprimarily for successfully delivering theappropriate technologies, infrastructure, andtechnical support, while accountability fordelivering business results rests jointly with theIT function, the relevant business managers,and performance measurement professionals.The benefit realization process usuallytranscends departmental boundaries, andwhen designing an appropriate IT evaluationmodel, this should be considered.

IDENTIFYING THE OBJECTIVESAND DRIVERS OF SUCCESSFULINFORMATION TECHNOLOGYINVESTMENTS

The identification and measurement of theimpacts of IT investments are particularlydifficult as they are often linked to long time-horizons, a high level of uncertainty, andimpacts that cannot be easily quantified. But,measurement and evaluation systems can besimplified and implemented with manageablecost through early planning, and a change inphilosophy and attitude of both the IT staff andthose it serves.

In recent years, organizations have placedincreasing importance on the development ofperformance metrics to better measure andmanage IT programs and projects; however,few specific metrics have been proposed. Withthe high costs that are often associated with ITinvestments and the seemingly smallpercentage of IT projects that pay off, theimpression is often that the projects areflawed whereas it may be that it is theperformance measures that are flawed.Worse, the lack of performance metrics may

have also led in part to the lack of both actualand perceived accountability for IT operationsto its various stakeholders.

New IT performance measurement models

There have been some recent attempts tomeasure the value of IT. One authordeveloped a ratio called InformationProductivity (IP) which is simply the ratio ofthe Economic Value-Added (EVA) to the totalcost of information management (Strassmann,1999). With information technology being oneof the fastest growing components of the costof information management this metric isdesigned to reflect an organization’s success atconverting the cost of information managementinto profit.

Another proposal is to expand conventionalfinancial measurement, like return oninvestment and payback period, to aneBusiness context which is a whole-viewmeasurement of business performance acrossboth internal and external constituents(Cameron et al., 2000). By setting weightedeBusiness objectives relating to end-customersuccess, hyper-partnering efficiency, and multi-organization financial performance andapplying quantitative and qualitative impactmetrics, organizations can track a project’simpact on a given eBusiness objective.

In yet another approach, Intel has developed aBusiness Value Index (BVI) (Intel, 2003b,Curley, 2004). BVI is a component index offactors that impact the value of an ITinvestment. It evaluates IT investments alongthree vectors: IT business value, impact on theIT efficiency, and the financial attractiveness ofthe investment. All three factors use apredetermined set of defining criteria thatincludes customer need, business and technicalrisks, strategic fit, revenue potential, level ofrequired investment, and the amount ofinnovation and learning generated. Eachcriterion is weighted, and project managers orprogram owners score their projects againstthese criteria to produce total scores for eachof the three vectors. By graphically depicting thethree indices for each project, BVI methodologyprovides some decision support to managers tocompare and contrast investments, and thendetermine the investments that align best withtheir business priorities.

Though all of these approaches are helpful,specific tools for identification and

6

M A N A G E M E N T

S T R A T E G Y



measurement are necessary. (Additional relevantapproaches can be found in Tardugno et al. 2000,Remenyi et al. 2000, Murphy 2000, Devaraj andKohli 2002, Lutchen 2004, Weill and Ross 2004,Curley 2004 and Schubert 2004). This guidelineattempts to provide a format so thatorganizations can identify and measure the costsand benefits of IT. It provides a useful model andspecific metrics that will help organizationsmeasure the inputs, and processes, as well as thebenefits and value resulting from IT initiatives. Asmost contemporary approaches to performancemeasurement emphasize the alignment betweenstrategic objectives and measures — since it isdifficult to measure performance unless it is clearwhat an organization is trying to achieve — webegin with the necessary alignment betweeninformation technology and the corporatestrategy.

Designing the IT Contribution Model — convergence of corporate strategy and technology

Information technology increasingly serves as theengine of innovation and the foundation ofgrowth for organizations worldwide. Virtually noorganization can generate new products andservices, manage its investments, communicatewith suppliers, buyers, or employees, and extendits markets without IT. Business and IT areinseparable, with few decisions resting entirely inthe domain of either business or IT. Thedecision-making process surrounding ITinvestments, thus, must seek to expand ITcapabilities to satisfy future business needs aswell as current needs. Investment decisions mustconsider the strategic perspective if they are tobe effective. In fact, the alignment between thecorporate and IT strategy is crucial for tworeasons:

● A proper incorporation of IT strategy into thecorporate strategic plan allows for sufficientcapital and human resources to fund ITprojects and programs;

● Carefully planned and implemented ITinitiatives significantly contribute to successfulexecution of the corporate strategy.

The strategic alignment thus refers to thealignment of IT investment strategy with theachievement of the organization’s strategic goalsand objectives. The question to be asked is ‘Willthis IT investment help us achieve our corporatestrategic goals?’

In a recent survey, researchers found that the

state of IT and business alignment has improvedover the past three years resulting in increasedproductivity, cost savings, and customersatisfaction (Russell, 2004). On the other hand,two critical issues were underlined: IT budgetsare poorly aligned with the corporate strategy,implying there is a risk that mid-cycle budgetchanges may derail planned activities, and, thelarger goal of successful execution of corporatestrategy remains elusive.

The deficit in strategic effectiveness may largelybe attributed to poor communication andunderstanding of strategy at lower levels inorganizations. Few organizations actually have astrategy that is communicated to those who areexpected to execute it. If IT is an importantcontributor to strategic success, then a lack ofunderstanding as to how IT resources are beingused to execute strategy represents a missedopportunity for IT/strategy alignment. It isimportant to identify how IT can help makecorporate strategy execution possible and here,an active involvement of the CIO in the planningand execution of corporate strategies is crucial.In addition, IT managers, financial managers, andheads of Strategic Business Units (SBUs) need tocooperate throughout the entire implementationprocess. While budget practices may obstruct ITalignment with corporate strategic goals, asystematic IT contribution model can be used toshow how discretionary IT spending can supportstrategic business initiatives. The purpose ofdiscretionary spending is to achieve strategicalignment and enable sufficient flexibility torespond to new strategic challenges andopportunities that cannot wait until the nextround of the formal budgetary process. As arelatively large portion of all IT spending isdiscretionary, the lack of a well-structuredmethod for communicating the strategic role ofIT is a serious handicap for the organization.Without the benefit of a well understoodanalytical and communication tool, IT is left todefend its budget with little more than areference to poorly measured project-level ROIanalysis.

Value creation depends on accomplishingstrategic change: to exploit new opportunitiesand existing capabilities, to defend againstcompetitive threats, and to overcomedeficiencies. Since every change involves new,enhanced, or changed information technology,strategy execution cannot be accomplishedwithout IT. How the organization spends the ITresources can thus significantly affect the

7

organization’s ability to create value. Inparticular, the organization and financing of theIT function are potentially significant drivers ofIT alignment and effectiveness.

The IT Contribution Model — inputs,processes, outputs, and outcomes

Exhibit 1 describes the key factors forcorporate success in IT integration. It includesthe critical inputs and processes organizationsneed that lead to success in IT outputs(internal and external). However, IT successultimately must be measured by itscontribution to overall organizational success(such as profitability or shareholder value) thatis the ultimate outcome and measure ofsuccess. The IT Contribution Model can beequally applied to both for-profit as well asnot-for-profit organizations.

An organization’s IT success is dependent onvarious inputs. This includes its existingcorporate strategy, structure, and systems thatprovide both opportunities and constraints onIT initiatives. These, along with availableresources and the external environment, arecritical inputs that affect choices in theformulation and implementation of ITstrategies. Other factors, such as leadershipand IT strategy, IT structure, and IT systems(processes) also significantly impact theperformance and success of IT initiatives. Boththe inputs and processes impact on various IToutputs that can be classified as either internaloutputs such as improvement in productivity,time savings, increased utilization of capacities,improved quality, overall cost reduction, as

well as external outputs such as channeloptimization, customer acquisition, satisfaction,and loyalty, and overall value capture. If the ITstrategy formulation and implementation issuccessful, these outputs should ultimately berealized in improved overall corporateprofitability (outcome).

In Exhibit 2, inputs, processes, internal andexternal outputs, as well as outcomes of ITactivities are further articulated as ITobjectives.

After having identified specific IT objectivesalong all four dimensions, the drivers of ITsuccess (see Exhibit 3) must be determinedand the relevant measures developed. Criticaldrivers specify more precisely the keys to ITsuccess and the actions that managers musttake to improve the success of IT activitiesthat will ultimately impact on overall

organizational success. In practice, there arenumerous drivers of IT success and the onesoutlined in this guideline do not attempt tocover all choices. Various elements should beconsidered when developing IT strategy andspecific IT programs, and several managerialactions such as the use of management controllevers, performance measurement andmanagement systems should be employed toassure their successful implementation.

A careful and clear articulation of the mostinfluential drivers of IT success help determinethe causal relationships leading from theinputs to the processes and then flowing tothe desired outputs and outcomes. Causalrelationships between drivers within each ofthe four dimensions as well as between drivers

8

M A N A G E M E N T

S T R A T E G Y


Exhibit 1: IT Contribution Model: Antecedents and Consequences of IT Success

INPUTS PROCESSES OUTPUTS OUTCOMES

Feedback Loop

ExternalEnvironment

CorporateStrategy,

Structure, andSystems

Resources

Leadership

ITStrategy

IT Structure

IT Systems

IncreasedProductivityTime Savings

IncreasedCapacity

UtilizationImprovedQuality

Direct CostSavings

ChannelOptimization

CustomerAcquisition

CustomerLoyalty

ValueCapture

CorporateProfitability


in different dimensions are based on hypotheticalassumptions of causes and effects, i.e. leading andlagging elements. In practice, the notion ofleading vs. lagging elements should be thought ofas a continuum, as, for example, customer loyaltyleads to increased financial performance, but atthe same time lags the quality of products orservices. These hypothesized relationships mayneither be a perfect description of actualrelationships that are underlying the IT strategynor are they supposed to be constant throughtime. On the contrary, they need to becontinuously tested and revised. A clearunderstanding of the causal relationshipsunderlying the primary drivers of value is one ofthe most important determinants of the model’seffectiveness.

The causal linkage map of drivers is useful andimportant as it helps ensure all actions necessaryto achieve success are taken, that unnecessaryactions are not taken, and that all employeesunderstand their critical role in the success ofthe IT activities. Exhibit 3 illustrates a

comprehensive example of IT performancedrivers and the many causal relationshipsbetween them. In practice, there should befewer critical IT performance drivers and theillustration of the causality of IT performancedrivers less complex.

For example, if organizations align the corporateand IT strategy, then they will potentially obtainmore resources to spend on IT technology.More resources spent on IT technology canenable them to consolidate and standardize ITinfrastructure leading to improved IT processes,increased productivity and quality, and decreasedcosts. Similarly, if organizations devote moreresources to enhance IT products and services,they can increase delivery options leading tohigher customer satisfaction, sales, and revenues.

Identifying the inputs — objectives and drivers

When revising existing IT strategy or developinga new one, senior managers need to fully

9

Exhibit 2: IT Contribution Model: IT Performance Objectives

Outcomes Long-term Corporate Profitability/Organizational Success

Short-term Corporate Profitability/Organizational Success

Outputs: Value Capture: Increased IT profits

o External Customer Loyalty and Retention: Increased sales from existing customers

Customer Acquisition: Increased sales from new customers

Channel Optimization: Increased site traffic and sales

o Internal Direct Cost Savings: Reduced IT costs and other direct costs

Improved Quality: Reliable information, less inspections, lower cost of quality

Increased Capacity Utilization: Optimal use of existing resources

Time Savings: Shortened process cycles

Increased Productivity: Improvements in operational productivity

Processes IT Systems: Appropriate processes for effective implementation

IT Structure: Integration into business unit structure

IT Strategy: Coherent and aligned strategy

Leadership: Commitment and focus on IT initiatives

Inputs Resources: Adequate capital & people

Corporate Systems: Suitable training, information, processes, and culture

Corporate Structure: Appropriate corporate organizational structure

Corporate Strategy: Alignment with type of products offered, customers served, technology used, and competitive positioning

External Environment: Adapted to external forces

M A N A G E M E N T

S T R A T E G Y


10

evaluate the external environment and thecorporate resources, strategies, structures,and systems to determine whether the ITstrategic initiative fits well with theorganization’s existing internal and externalenvironment and predict its likelihood ofsuccess.

External environment represents the leastcontrollable of all input variables. The externalenvironment may include changes in theeconomy, technological forces, political-legalforces, socio-cultural forces as well as theindustry — competitors, customers, suppliers,and so forth — within which the organizationoperates. Thus, even though organizations mayhave coherent strategies, structures, andsystems as well as adequate corporate capitaland labor resources for IT investments,changes in the external environment canimportantly influence corporate decisionsregarding the IT organization and its success.CIOs, CTOs and IT managers should beconstantly engaged in environmental scanning

by monitoring for weak and strong signals,specifically in the field of technologicaldevelopment, as well as examining the mostimportant stakeholder groups both internaland external to the organization.

Corporate resources that are most criticalfor IT success relate to capital (financialresources) and people (employees). In manyorganizations, CEOs, and CFOs provideglowing remarks about IT’s important impacton the organization’s success, but whenorganizations are in financial difficulty, cuttingthe IT budget is one of the first moves. Somecorporate level executives see IT’s share ofenterprise revenue and expense growing at analarming rate and while this growth typicallyreflects the advancing strategic role oftechnology, many believe IT spending is aboveits optimal level and pressure managers toreduce overall IT spending. Not surprisingly, inmany IT departments project managers aremultitask professionals as they servesimultaneously as business analysts, project

Exhibit 3: IT Contribution Model:Causality of IT Performance Drivers

Outcomes

Outputs:external &internal

Processes

Inputs

Increased Corporate Profits/Organizational Success

Increased revenues Decreased costs

Expandedmarkets

Greater sales

New customeracquisition Customer satisfaction

and loyalty

Increasedproductivity

Production/serviceimprovements

Improvedquality

Time savings

Data handlingimprovements

New products online

Increased strategic alliances

Increased deliveryoptions

Enhanced IT productsand services

Improved ITprocesses

Optimal outsourcing of IT work

Consolidated andstandardized ITinfrastructure

IT literate managers

and employees

Aligned corporate and ITstrategy

IT spending (technology,processes, people)

External environment including competitor offerings

Needed IT knowledgeand skills


managers, and support managers — roles thateach require specific skills, and techniques. Workoverloading impacts on morale, project delivery,and customer satisfaction. Informationtechnology and human capital are importantingredients for value creation. To support newor expanded IT operations, it is thus necessaryto look at the people and capital that currentlyexist in the organization and assure sufficientfunding for necessary training and education andnew hires.

Corporate strategy and the alignment ofinformation technology is a constantly recurringtopic of interest within the IT community.Corporate strategic objectives drive thedefinition of business processes, but they alsodetermine technology architecture, and theneeded staff capabilities, so that informationtechnology supports the underlying businessprocesses. Although corporate strategy mayactually be modified as a result of thedevelopment of IT, corporate strategy remainsan important input to the formulation andimplementation of IT strategy.

Corporate structure that an organization hasplays a big part in determining the organization ofIT operations within it. Whether an organizationhas a large number of strategic business units ornetworks across a wide geographical area willsignificantly impact on many IT activities. Inorganizations with more decentralizedorganizational structures and with independentbusiness units, a more streamlined, standardizedinfrastructure is integral to the way applications,business processes and services are delivered.The technology infrastructure must be able tohandle enormous volumes of data from insideand outside the firewall. It must accommodatedata from a range of new sources and in a varietyof formats, and it must gather, store, andtransmit information dynamically toaccommodate the fast-paced decision making.Corporate structure thus represents animportant input to the IT strategy, structure,systems, and leadership necessary for IT success.

Corporate systems relating to the flow ofinformation, human resources, compensation,management control, performancemeasurement, and organizational culture willimpact the choice of IT strategy and implementation.Often, there are no formalized communicationchannels for infrastructure and systems support,and lack of a disciplined process for coaching andfeedback. Also, performance of IT departments is

largely undocumented. This can often be tracedto poor HR policies and management controlpractices. An organization’s corporate systemsare thus highly relevant in improving theimplementation of newly developed IT strategies.

Identifying the processes — objectives and drivers

Once the viability of IT initiatives is estimatedthrough proper evaluation of the externalenvironment and inputs available in theorganization, senior managers responsible forplanning and developing IT strategies andprograms can focus on the processes necessaryto drive superior IT performance. Both thecareful examination of inputs as well as effectiveuse of processes will determine the outputs andoutcomes.

Leadership of the organization must beknowledgeable about IT, committed to ITinitiatives, and aware of the impacts on existingorganizational culture and behavioral patternsthat may act as impediments to effectiveimplementation of new IT initiatives. Seniormanagers must provide full support to ITprograms and projects and communicate thatthese are supported widely throughout theorganization. They must help employees tounderstand the importance of IT initiatives,foresee the potential benefits of specific ITprograms and projects, and willingly embrace thenotion that IT initiatives will ultimately benefitthem as well as the organization as a whole. Ifunable to secure strong support for IT programsat the corporate level, though an IT programmay be installed effectively by technical staff orconsultants, it is unlikely to reach its potentialand success is improbable.

IT strategy must be consistent and aligned withcorporate strategy, structure, and systems andsupported by IT structure and systems. Theorganization’s choice of markets, customerexpectations, products and services as well asthe pricing policy will importantly influence ITdecisions. IT strategy is about acquiring,allocating, and managing the use of limited ITresources so that IT can deliver high valuethrough increased revenues, decreased costs andfacilitated transactions. For example, AFLAC, aGeorgia-based insurance company, achieved atrue competitive advantage and dominantposition in Japan by handling most of its businessonline. The company’s early investments inInternet technology enabled it to communicate

11

with, sell to, and service Japanese companiesand their employees. Technology enabled thecompany to settle and process product claimswithout costly agents and claims processors.As a result, AFLAC’s cost to acquire andprocess a new customer is $72 versus $120for its Japanese competitors (Suh, 2004). Thus,IT investment decisions should be strategic inthat they can create business value in the nearterm and prevent the risk of making baddecisions in the long term. By having a clearlyformulated strategy, IT managers can resistpressure from individual business units toinclude ad hoc components to meet theirspecial IT requirements.

IT structure will be impacted by severalcritical factors, such as the level of outsourcingappropriate for the organization, therelationships the organization wants todevelop with its partners, the existingcorporate organizational structure, and thelevel of desired IT integration, consolidation,and standardization. They all impact on theorganization of the IT department and impactdecisions related to software coordination,maintenance of legacy systems, and thecentralization or decentralization of the ITfunction. Partnering with other companies intheir supply chain significantly impacts ITstructure and creates powerful valuepropositions. Eniro, for example, NorthernEurope’s largest yellow pages company,collaborated closely with Microsoft andcreated a product development tool with thepotential to reduce Eniro’s productdevelopment lead time dramatically (Mesøyand Arlebäck, 2004). In recent years, many ITinfrastructures have evolved and grown in ahaphazard fashion either through systemsincluded in mergers and acquisitions or inresponse to the needs of a specific businessunit. As a consequence, these infrastructuresare often poorly integrated and inflexible,complex and underutilized. Instead, ITstructures should evolve as a result of thecareful examination of a comprehensiveinfrastructure strategy. Sainsbury’s, one of theUnited Kingdom’s largest grocery retailers, forinstance, faced with fierce competition anddeclining operating profits, sought to reducethe complexity and improve the effectivenessof its IT infrastructure. Based on a program ofstandardization and consolidation across arange of hardware, database, communications,and applications systems, IT service levels have

been improved dramatically while associatedcosts have been reduced (Kasamis and Nunn,2004). Choices about how an organizationdecides to design and implement its ITstructure and organize its IT function are keyissues that organizations must address.Specifically, a broader understanding of whatthe core organizational processes are and acareful analysis of both the short-term andpotential long-term benefits is needed to justifyall new IT alliances including outsourcing.

It is essential that IT systems such asspecialized HR practices for IT departments, ITtraining, performance measurement, andmanagement control are part of the processespertinent to IT. In many organizations, the gapbetween the rate of technology innovation andemployees’ skills and knowledge to productivelyuse these new innovations is growingpreventing IT efforts to realize its full potential.It is important to improve employee ITliteracy, employees’ understanding and abilityto use these new innovations, measureimprovements based on implementedinnovations, and establish proper compensationpolicies to stimulate employees to deploy theuse of IT. Consequently, results may extendfar beyond items such as lower costs, andtimelier customer service, to heightenedproductivity, and greater sales. Cisco, forexample, a leading producer of routers andother packages to help control traffic andinformation on intranets and on the internet,has found great benefits from hiring online,including faster filling of positions, and highercompetence of people employed (Epstein,2004).

Identifying the outputs — objectives and drivers

If the IT initiatives are well designed andexecuted and the causal relationships properlyspecified, the identified inputs and processesshould lead to improved performance inoutputs, and ultimately to increased corporatefinancial performance. Objectives and driversfor internal and external outputs arespecifically important as they foretell thecritical key success factors and what resultsshould be expected. A weak performance onthe output metrics should signal a need toreexamine the inputs and processes anddetermine whether they have been wronglyspecified, improperly placed in the causalrelationship scheme, or just poorly executed.

12

M A N A G E M E N T

S T R A T E G Y



As already mentioned, the overall outputs of ITinitiatives can be divided in two categories:

● Internal outputs relating to increasedproductivity, time savings, increased capacityutilization, improved quality, and direct costsavings;

● External outputs that cover a broad arrayof results with respect to channeloptimization, customer acquisition, loyalty,and retention, and overall value capture.

Increased productivity is one of the expectedimmediate benefits of new IT programs andprojects. Improvements in informationtechnology infrastructure, for example, in termsof fully integrated application systems allows forbetter access to databases, faster exchange ofinformation, reduced operating cycles, and soforth. Also, the standardization of IT workprocesses, segmentation of the work, and globaldispersion for greatest efficiency permitnumerous improvements. These include reuse ofapplications and technical architectures,automation of much of the delivery process, andcodification of methodologies so that they can berepeated, which all greatly increases productivity.Intel (2003c), for example, reports on the effectsof wireless mobile technology on employeeproductivity. Employees who were upgraded towireless notebook computers realized aproductivity gain of more than two hours perweek, which exceeds the cost of the upgrades inthe first year. In addition, wireless mobilityrapidly changed the way employees worked. Forexample, employees were able to makeproductive use of formerly wasted slices of timebetween larger tasks, as they could redistributetheir working time around professional andpersonal obligations.

Time savings are another potential immediateinternal output of new information technologies.When improved information systems integratethe new product development process, theinvestment payoff results in reduced time tomarket. By replacing existing productionmethods and logistics with an enhanced ITsupport system, throughput time can besignificantly reduced. Also, improvementsthrough online processes assist in activities suchas ordering, invoicing, tracking, payment, anddelivery. Critical business processes such asprocurement, sales, and customer relationshipmanagement can be increasingly automated,creating both time savings and, consequently,cost savings through reduced paper handling and

requirements for human intervention inadministration. Pirelli, a manufacturer ofautomobile tires, cables, and systems, forexample, integrated its IT systems with those ofits global customers and dealers, and was able tosignificantly reduce administrative activities, savetime, and reduce costs (Hill et al., 2004). In a2002 study, Intel found that it could save itsemployees at least 60 hours per year on tencommon office application tasks by upgrading ITsoftware. According to Intel, these gains directlysupport their ability to remain competitive byenabling employees to make decisions quicker,execute faster, add value, and respond to internaland external customers more quickly. (Intel,2003a)

Infrastructure enhancement can also enableorganizations to optimize the use of existingresources. Increased capacity utilization canresult from investments in operationsapplications, and telemetry. One company, forexample, is investing in telemetry to improvecontrol over the large private rail fleet thattransports its products. The solution involves theuse of satellite communications; GPS; and weight,temperature and impact sensors. This companylaunched its own tests, which revealed largebenefits through increased turns (journeys fromthe production facilities to the customers andback) on its rail cars (Kasamis and Nunn, 2004).

Improved quality specifically relates to ITinitiatives aimed at enhancing TQM activities thatultimately result in lower preventive andappraisal cost of quality, reduced demand forrework, and increased customer satisfaction.Also, the electronic information tends to be moreaccurate due to fewer transcription errors andcontribute to higher reliability and quality of data.

Direct cost savings relate primarily to thereduced IT expenses originating from theconsolidation, standardization, and integration ofIT applications, from the increased IT systems’security, and from the replacement of old ITsystems that require high maintenance. A largetelecommunications company, for example,consolidated and standardized its entire global ITenterprise — from the workplace to theunderlying network, including data, applicationsand servers. The effort generated significant costsavings and reduced annual IT expenses by 40percent (Kasamis and Nunn, 2004). The samesource reports of a large financial servicescompany which had been operating on acountry-based franchise model. By standardizing,

13

consolidating, and building out a centralinfrastructure services organization with itsown P&L responsibility, savings of nearly 20percent were realized. Direct cost savings mayalso originate from enhanced IT systemssecurity. For example, the trucking industrysuffers heavy losses from truck and cargotheft. By placing computers in the trucks, GPSinstallations, and biometric identificationsystems, the truck, its contents, and allpersonnel can be tracked and verified(Ferguson and Taylor, 2004). Similarly,insurance organizations currently spendhundreds of millions of dollars to gather andanalyze information on how customersoperate their vehicles. Then they use thisinformation to help them manage risk andoffer appropriate, targeted policy coverage.Using telemetry to supply insurance providerswith this information saves time and reducescosts (Osman, 2003).

The financial consequences of improvementsin internal outputs are reflected in cost savingsor, potentially, in increased sales. The externaloutputs, on the other hand, relate toachievements realized in the market.

Channel optimization can be accomplishedand exhibited through an array of items thatvary from increased site traffic to informationalgains and improvements in demand and supplychain relationships. By encouraging suppliersand customers to use the channels thatprovide the highest revenues and lowest cost,the enhanced information technology outputsare evident and the impact on corporateprofitability becomes clear. Nordstrom, forinstance, a high-end retailer, was an earlymover in luxury retailing online. The companyused e-commerce as a cost-cutting strategy,replacing the need to send out expensivecatalogs to potential customers. Offeringproducts that need to be examined in personbefore the sale is complete, Nordstromprovided particularly generous return policiesto overcome the hesitation to purchaseproducts online (Epstein, 2004).

Customer acquisition can significantly beincreased by creating and using new channelsthat provide customers with products andservices. Organizations, for example, thatmove more commerce to the Web canaccomplish expanded global coverage andexposure with a relatively minimal investment.Through online activities, organizations canoffer existing and new products and services

directly to their customers, thus approachingsegments whose buying interests otherwisemay not be as easily aroused.

Customer loyalty can be seen throughrepeat customers and is a consequence ofoverall satisfaction levels. Customersatisfaction may originate from an array ofitems, such as improved product and servicequality (products with correct specifications,deliveries to the right locations), deliveryspeed (on-time deliveries, time between ordertaken and delivery effected), as well as lowerprices. Information technology can significantlyimpact on all of these. The internet, forexample, permits organizations to connectwith their customers, partners, and suppliersin a virtual environment permitting increasedcommunication and a greater ability tointertwine the organization’s operations withthe external stakeholders’ needs. ITapplications supporting e-commerce activitiescan provide an important opportunity toimprove customer service levels andrelationships leading to increased customersatisfaction, loyalty and repeat purchases.Amazon, for example, has designed most of itsown functions, including the search system,customer accounts system, and ITinfrastructure. The design of Amazon’s site,characterized by its personalizationtechnology, greatly contributes to its highrepeat customer rate (Epstein, 2004).

Value creation is at the core of businessobjectives. As customers are better able tocompare the products and services offered tothem through internet, for example, they canmore easily compare alternatives andcompetitor offerings in an expedient manner.Also, with IT impacting on the operating costsand sales cost, the opportunity to offer lowerprices to customers is greatly increased leadingto the ability to be more competitive and driveincreased revenues. Organizations can alsocreate and capture additional value byproviding additional services to their customerssuch as customization and personalization.Well-integrated IT infrastructures supportorganizations in these efforts by securelystoring large amounts of data.

For organizations seeking to achieve highperformance, the role of informationtechnology has never been more vital. Theextraordinary abundance and easy accessibilityof information, the tools and technologiesused today to gather and share information,

14

M A N A G E M E N T

S T R A T E G Y



cheaper computing power and data storage, allgive organizations an unprecedented opportunityto use IT to create business value.

Identifying the outcomes — objectives and drivers

For IT initiatives to be of value, the intermediateoutputs — both internal and external — musteventually pay off in increased organizationalsuccess (corporate profits). Viewed simply,increased profitability can only be achievedthrough reduced costs or improved revenues.Thus, in order to prove that IT investments inprograms and projects were financially sound,the ultimate effect on corporate financialprofitability must be determined and the payoffsclearly documented.

DEVELOPING THE APPROPRIATEMETRICS

To closely monitor the cause and effectrelationships, appropriate metrics must bedeveloped. These metrics must be consistentwith and support the objectives and drivers ofsuccess. Metrics should be used to foster anunderstanding of corporate strategy andperformance drivers that will enhancecooperation between business units andstimulate a forward-thinking approach toachieving relevant objectives.

The starting point for developing the appropriatemetrics should be an overall agreement on thedrivers that are critical to achieving theobjectives in those areas in which theorganization must excel in order to ensure

success. Some drivers, such as innovation, aremore difficult to measure and the temptation isto avoid measurement. However, if they areconsidered to be crucial in demonstrating howIT can improve business success, they must beincorporated in the performance measurementsystem. It may be that non-financial performancemeasurement is more appropriate in such cases,but attempts should be made to measure asmany drivers as possible with monetary values.

The same is true for outputs. For example,improvements in quality may well be measuredby the percentage of high quality products, but itis more important to measure the dollars savedon less rework. Or, increased employeeproductivity can be measured by the percentageincrease in production output per employee; byassigning monetary values, additional sales basedon productivity improvements can bedetermined. Both the non-financial and financialmeasures, as long as they are expressedquantitatively, i.e. either in absolute orpercentage terms, are useful and allowcomparability and target setting. However,financial measurement is especially important asmanagers want to calculate ROI anddemonstrate IT payoff.

The list of metrics presented in Exhibits 4, 5, 6, 7,and 8 is not meant to be a comprehensive set ofIT performance measures. Rather, it is a selectionof some metrics that may be appropriate. Theright IT metrics are neither the same norrelevant for every organization, so, managersmust select those that most closely fit theirstrategy, and adapt or develop others.

15

M A N A G E M E N T

S T R A T E G Y


16

Exhibit 4: IT Contribution Model:Metrics for IT Investments - Inputs

Inputs Performance measures

Corporate Strategy o Organization’s competitive position (rating) within industryo Cost, development time, delivery time, quantity, price, and channels of

products offered relative to the competitors’o % of planned change in annual IT budget o Number of IT projects approved in the strategic plan o Direction of IT projects approved in the strategic plan (integration,

standardization, industrialization, innovation)

Corporate Structure o Number of Strategic Business Units (SBUs)o Level of empowerment to SBU and functional managerso Geographic diversity of production and sales

Corporate Systems o Employee stability indexo % of employees compensated based on individual or group performance o % of business processes documented and measuredo Dollars invested in training

Resources o Growth rate of IT spend per growth rate of direct total spendo Dollars available for IT infrastructure investmento Total IT cost per employeeo % of systems security budget to total IT budgeto Dollars available for IT staff training and developmento IT literacy of existing employeeso Average years of IT experience o % growth of IT employees per % growth of total employeeso Assessment of current organization technology and processes relative to

the competitors’

External Environment o Number of potential threats relating to IT from external environmento Assessment of competitor IT investmentso Assessment of customer needso Assessment of supplier needs and capabilities

17


Exhibit 5: IT Contribution Model:Metrics for IT Investments - Processes

Processes Performance measures

Leadership o % of senior executives’ time dedicated to ITo % of annual budget allocated to IT initiativeso % of CIO’s and IT managers’ bonus linked to IT profitabilityo % of senior managers that are “IT literate”o % of senior managers and employees that understand objectives

of IT initiatives Create & Execute o Number, cost, price, and perception of products and services offeredAppropriate online relative to the competitors’IT Strategies o Availability and planning for IT security features

o Planned cost of disruption of IT initiativeso Planned cost of risk related to IT projectso % of unplanned information system’s downtimeo % of discretionary spending decisions aligned with corporate

and business unit strategyo Planned costs, benefits, and profitability of IT projects

Design & Institute o % of IT expenses related to outsourcingProper IT Structure o Number of IT-business partnership projects

o Number of IT applications that are not fully integrated within the overall IT system

o % of systems developed/maintained outside the organizationo % of standardized hardware, databases, communications and

applications systemsDevelop & Implement o Number of IT professionals per employeeAppropriate o Dollars invested in IT skills and knowledgeIT Systems o % of corporate-level performance measures and rewards aligned

and linked to IT profitabilityo % IT employee turnovero % of IT staff with pay for performance compensationo % of projects’ evaluations based on ROI metricso Number of help desk incidents (by business unit)o Average help desk pick-up/resolution time o Level of abandoned callso Break/fix maintenance response/resolution timeo % of IT costs charged back to business units or functions

M A N A G E M E N T

S T R A T E G Y


18

Exhibit 6: IT Contribution Model:Metrics for IT Investments - Internal Outputs

Internal outputs Performance measures

Increased Productivity o % increase in production output per employeeo Dollar increase in sales based on productivity improvementso % increase in customer orders processed

Time savings o Reduction in online response timeo % reduction in operating cycleo % reduction in ordering, invoicing, tracking and payment o Average time required to fill and process customer ordero Timeliness of resolution of rejected orderso Timeliness in order deliverieso Dollars saved based on time savings

Increased Capacity Utilization o Fixed costs per unit of capacityo % increase in capacity utilizationo % change in the number of turns (transportation)o % of utilization of databases

Improved Quality o Dollars saved on prevention of poor qualityo Dollars saved on waste and scrapo Dollars saved on less reworko Reduction in customer grievanceso Dollars saved on cost of quality

Direct Cost Savings o % reduction in IT mandatory expenseso % reduction in operating expenseso % reduction in marketing costso Dollars saved on increased IT systems security (reduced IT

systems’ downtime, reduced incidence of frauds, etc.)o Time saved on disaster recovery/business continuity

19


Exhibit 7: IT Contribution Model:Metrics for IT Investments - External Outputs

External outputs Performance measuresChannel Optimization o Dollar value of activities completed through Web sites

o Site traffic (number of visits) and functionality of Web site(click through rate, stickiness)

o Hours of Web site downtime (in a year)o Partner and supplier satisfaction level ratings

Customer Acquisition o Number of new customers gained through IT innovationo Sales from new customers o % of customers using Web sites exclusivelyo % of visitors to Web site who are also buyers (reach)o Number of new customers in other channels informed through

Web siteo Number of created customer partnerships

Customer Loyalty o Average yearly sales per customero Sales from retained customers versus new customerso Customer satisfaction with IT activitieso Customer shopping cart abandonment rateso % of customer attritiono Ratio of new visitors to repeat visitorso Frequency of customer return visits to the Web site

Value Capture o Profitability of IT projectso Revenues generated through IT initiative (total revenue, IT

revenue, revenue per IT customer)o Average customer profitabilityo Cost and price indexes of selected products and services offered

to customers relative to the competitors’o Average of prices paid by customerso Number of new IT products and services introduced

Exhibit 8: IT Contribution Model:Metrics for IT Investments - Outcomes

Outcomes Performance measuresLong-term o % change in stock price attributable to IT initiativesCorporate o EVAProfitability/ o Earnings growthOrganizational o ROISuccess o ROA

Short-term o Cash flow growthCorporate o Value added per employeeProfitability/ o Revenue growthOrganizational o % in overall cost reductionSuccess

There is no precise rule for the right numberof metrics to include in measurement systems;however, including too many tends to distractmanagers from pursuing focused IT initiatives.A system that is overly complex leads tobureaucracy and confusion. Recentdevelopments in technology have meant thatIT systems are now able to provide a plethoraof statistics but, without care, this can lead toan overload of data with little meaningfulinformation. For this reason, it is important tofocus on the key indicators rather thanintroducing indicators for everything that canbe measured. There is a need for prioritizingand discipline. Where a new issue arises andfurther indicators are developed, these shouldreplace those that have become less importantrather than adding to the existing set.Generally, a complete IT performancemeasurement system should include no morethan twenty measures in total.

The measures should be controllable in thatemployees in the organization can actuallyinfluence improvement in the factor measured.Ideally, they should be complete in that themeasure sums up in one number thecontribution of all elements of performancethat matter. For example, customer loyaltymay be a summary measure of on-timedelivery and customized product.

FURTHER SPECIFICITY ON CALCULATING BENEFITS(OUTPUTS) FROM IT INVESTMENTS

The ultimate objective of almost all ITinvestments is to deliver business value. The ITContribution Model incorporates both thedrivers of business value (inputs andprocesses) as well as outputs and financialoutcomes and the relevant metrics forevaluating IT success. Since most organizationshave little experience in assigning monetaryvalues to IT outputs and the measurement ofIT payoffs, some additional examples andmetrics might be useful.

In the early days of computing, investmentswere made almost exclusively on the basis ofdirect financial benefits and these generallyrelated to direct cost savings. The opportunitiesfor such direct savings have now been greatlyreduced. However, they still do exist and

senior managers continue to look to this areaof justification of IT investment. On the otherhand, attempting to quantify longer term orindirect benefits through the use of non-financial measures of performance in the realmof end-user computing, particularly when itrelates to managers, has always been a majorchallenge. And, as the role of IT changes, non-financial measures of long-term benefits havebecome significant. These include improvedorganizational agility and communications,enhanced employee performance such asempowerment, more flexible workingconditions, safer environment, and higher jobsatisfaction. These longer term benefits mayalso stem from enhanced managementperformance through better and fasterinformation, improved decision supportcapability, or reduction in the number ofmeetings due to better information.Integration of IT systems, enhanced security ofIT systems, and improved supplierrelationships are also drivers of more indirect,longer term benefits.

Although these benefits do not always clearlytranslate into short-term profits, they shouldultimately lead to either cost savings orincreased revenues. Sometimes, the directrelationship between a specific action orprocess, such as better and faster information,and business value creation isn’t clear enoughto provide an easy calculation of the benefit’smonetary value. In such cases, additionalinquiry in terms of ‘How does thisimprovement specifically help you in yourwork?’ should be undertaken. It may be thatthe system supports increased throughput peremployee (increased productivity), saves time(time savings), helps optimize the use ofexisting resources (increased capacityutilization), or allows fewer mistakes(improved quality).

The transformation of these internal outputsto monetary terms is illustrated through anumber of short examples in Exhibit 9. SpecificIT initiatives are presented resulting in internaloutputs and then the relevant calculations tocapture the monetary value of realized benefitsare provided.

Referring again to Intel’s study of theproductivity of employees who were upgradedto wireless notebook computers (Intel,

20

M A N A G E M E N T

S T R A T E G Y



Exhibit 9:Calculating monetary benefits from IT investment

INTERNAL OUTPUTS EXAMPLE CALCULATION OF MONETARY BENEFITS

Increased Productivity

Time Savings

Increased Capacity Utilization

Improved Quality

Direct Cost Savings

An IT infrastructure upgradeimproves productivity of amanufacturing operator whose laborcosts are tied to operating hours.

Calculate the unit labor cost of the operation. Theincrease in output multiplied by the unit labor cost isthe added value of the IT upgrade.

IT project reduces labor involvementin IT programs.

IT project eliminates production anddelivery bottlenecks resulting inincreased on-time deliveries.

Time savings (in processing a salesorder, for example) can increaseopportunity for profit by allowingmore time for acquiring new clients.

Monetary savings are the hours saved multiplied bythe labor cost per hour plus benefits.

If the result is reduction in grievances, the averagecost per grievance provides a basis for estimatingthe benefits.

Additional time for new clients should increase sales- benefits can be calculated as additional sales(realized within additional time) minus marginal salesexpense.

Infrastructure enhancement (use ofsatellite communications) optimizesthe use of existing resources(transportation vehicles).

Benefits arise out of increased turns (journeys fromthe production facility to the customer and back)and equal to additional sales minus direct variablecosts.

IT project enhances TQM results:

● lower preventative and appraisal cost of quality

● less scrap and waste

● less rework

● lower customer dissatisfaction

● Benefits equal to the number of hours saved multiplied by the standard labor wage and adjusted with a benefits factor.

● Benefits are related to savings in costs of defective products (total cost incurred at the point the mistake is identified minus the salvage value of a defective product).

● Multiply labor hours saved by the labor cost per hour and add eliminated direct cost.

● Benefits equal to money saved based on reduced reclamations.

IT upgrade integrates purchasing andmanufacturing applications resulting inreduced operating costs.

Integration of the organization ITsystems with those of its globalcustomers and dealers results inextended information and services toits customers and trading partners.

An IT project designed to betterintegrate infrastructure allowing theorganization to more securely storeand move data at will.

An IT initiative aiming to increaseinformation system’s security.

A program of consolidation and stan-dardization across a range of hardware,database, communications and appli-cations systems reduces IT expenses.

Benefits equal to the amount of the cost savings(costs of handling materials, financing stock, etc.).

Benefits can be traced to reduction in administrativecosts.

Benefits arise from both the improved control overinformation (elimination of damage costs based oninformation abuse) and smooth information sharing(generating time savings).

Benefits arise both from the reduced hours ofsystem’s downtime (multiply the hours saved by theaverage labor hour productivity) as well as fromreduced fraud incidence (estimate the financialdamage caused by frauds).

Benefits equal to reduced IT expenses (cost ofoperating and maintaining the IT system).

21

2003c), the following approach to calculatingthe monetary benefits was taken. The financialvalue of increased worker productivity wasevaluated from two perspectives: the cost ofeach employee’s time, and the benefit to thecorporation of its employees’ output. The firstperspective is a straightforward calculationwhere the productivity value equals the hourssaved multiplied by employee cost. Based onthe employee tasks they chose to evaluate,total time savings were then calculatedamounting to 5% of a typical 40-hour week.Assuming that the benefit of upgradingworkers to the faster configuration is 5% ofthe employee’s total cost, a company with25,000 employees and a per-employee totalcost of 100,000 dollars would realizeapproximately 125 million dollars annualbenefit from the upgrade. The annual benefitper employee would be 5,000 dollars. But forthe company to be profitable, their return oninvestment in workers must exceed totalemployee cost. According to Intel, simply tovalue employees’ time at their annual cost rateundervalues their true output and the returnon the company’s investment. Identifying thevalue of gained employee time fromproductivity increases is a better assessment ofthe true value of an upgrade’s productivityenhancement. By that measure, upgrading tothe faster, more powerful configuration wouldrealize an annual increase of 2.5 millionemployee work hours. That is the equivalentof adding 1,250 employees to the payroll.

Generally, cost savings, traditionally appliedprimarily to staff displacement, can now betraced to employee overtime reduction, lessneed for specialized (and more expensive)staff, or reduced travel costs. All sources oftime savings, such as less searching forinformation, fewer phone calls, fewer queries,reduced idle time, and reduced orderturnaround time, lead to cost savings and thenpotentially to increased sales. Based onimproved quality control, cost savings takeindirect form by way of reduced reworking,fewer rejections at final inspection, fewermistakes in invoicing and delivery, fewercustomer returns, and reduced help deskrequirements. Further, they originate fromreduced capital and maintenance costs for newequipment, enhanced inventory controlsystems that lead to savings on cash flow,reduced inventory, floor space, and employeetime.

With respect to additional revenues, somesystems, for example, enable totally newproducts to be introduced, enable faster andmore focused product development, orprovide economic justification for so farunacceptable products. Improved assetutilization can also lead to potential increasesin production and consequently in revenues.

External outputs — channel optimization,customer acquisition, customer loyalty, andvalue capture — on the other hand, are moredirectly related to or already incorporatebusiness value creation. Thus, the translationof these benefits to monetary value shouldn’tbe a difficult task. However, with respect tocustomer satisfaction, acquisition, and loyalty,special importance needs to be placed on themetrics chosen. Customer approval ratingsbased on satisfaction surveys, for example, ismore of a leading indicator of customersatisfaction and represents a customer wishlist more than real requirements. Also, thenew customer acquisition rate can be animportant indicator, but organizations oftenneed to focus more on retention than onacquisition. Thus, the best indicators ofcustomer satisfaction may be the customerretention ratio, the ratio of serious customercomplaints to quantity of services andproducts provided, or the level of increasedspending per retained customer.

When measuring benefits, two more thingsshould be addressed:

● It is often of significant importance toinvolve the constituents to help identifymetrics that are important to them(Devaraj and Kohli, 2002). A goodillustration is the case of a customercomplaints system that greatly reduced thenumber of follow-up calls. This had theeffect of reducing productivity, as measuredby number of calls handled per operator,however, effectiveness in fact improved.Another example is an increase in marketshare that may actually imply theorganization is getting customers thecompetition doesn’t want. Reduction incustomer complaints could also meandissatisfied customers are not speaking upbecause they chose not to come back.Similarly, reduced reject rate may berelated to more defective parts gettingthrough. Thus, it is extremely important tocarefully examine the underlying causes for

22

M A N A G E M E N T

S T R A T E G Y



achieved outputs.

● Sometimes, it is difficult to correctly attributebenefits to individual IT initiatives, especially,when a number of different initiatives couldhave had an impact on the benefits gained.This is called business complexity noise. Forexample, sales might have increased in linewith expectations following the introductionof a Web site, but other marketing initiatives(e.g. price reductions, advertising campaign)may also have been introduced around thesame time, rendering difficult an accurateassessment of the Web site’s contribution. Insuch case, it is important to explain thepotential noise problems to thoseaccountable for benefits realization andattempt to identify the contributing factors.Though often imprecise, identifying andmeasuring the monetary benefits from ITinvestments is critical for managerial decisionmaking and resource allocation.

MONITORING THE STRATEGIC ITOBJECTIVES,DRIVERS AND PERFORMANCE MEASURES

The implementation of a measurement systemshould not be seen as a threat to or impositionon staff, rather as a mechanism to enhanceperformance and corporate learning. A properlydeveloped and implemented measurementsystem promotes productivity by focusingattention on the most important issues, tasks,and objectives of the organization.

However, performance measurement systemshave to be modified as circumstances change inthat strategic objectives are modified —according to the new strategy — drivers arerevised and new causal linkages among driversdetermined. As a consequence, performancemeasures should be adapted so that they remainrelevant and continue to reflect the issues ofimportance to the business. All too often,organizations are implementing new measures toreflect new priorities but fail to discard measuresreflecting old priorities resulting in uncorrelatedand inconsistent measures. Ideally, organizationswould implement systematic processes formanaging the evolution of their measurementsystems.

Performance measurement systems should alsobe dynamic in that performance measures thatcan be more easily manipulated or that lead todeviations from the planned results are replaced.For instance, an organization implementing a new

ERP system might set an on-time targetimprovement of 15%, encouraging promptshipments, but possibly at the expense of moreerroneous shipments that are not measured.Similarly, an organization, trying to justify a newERP system may aim at achieving a 20%reduction in inventory carrying costs; but alsocause a sharp increase in the number of shortshipments. Such behavior may exist becauseperformance measurement itself serves as apowerful motivation mechanism. Systems mustbe carefully designed to align strategy,performance measures, and incentives, so thatthe incentive pressures are not severe and themetrics are appropriate - multidimensional andbalanced. Otherwise, dysfunctional behavior,such as gaming, can result.

Sometimes, it is just the annual targets andreward systems that tie compensation toperformance that have to be changed to assureproper functioning of the performancemeasurement system. The problem of targetsetting is largely related to the lack of standardsfor judging the success of IT investments.Although most IT executives generally knowwhen a measure represents acceptableperformance, there have been only a few attemptsto develop measures, standards, and indices. Thisobstacle can be minimized as more organizationsadopt measurement and evaluation frameworksand metrics and report their results publicly orthrough benchmarking projects. Compensationsystems and performance measures used toreward IT managers’ and IT staff performancemust be carefully selected. If the evaluationincludes elements other than those they directlycontrol, the reward system will often causefrustration and de-motivate staff rather thanencourage better performance.

MEASURING THE DISRUPTIONCOSTS OF IT INITIATIVES

It is increasingly recognized that the disruptioncosts associated with the adoption of IT can bemore significant than the direct cost of the ITinitiative. Yet, it is the elusive nature of thesecosts that make their identification and controldifficult.

Disruption costs of IT initiatives are primarilycaused by the transformation from old to newwork practices, based on the impact of the newsystem. In the immediate pre- and post-implementation phases, productivity invariablydeclines for a period. On one hand, employees

23

may resist adapting to new procedures andguidelines; on the other, they go through alearning curve, while struggling to adapt tonew ways of working and doing business.Often, too, productivity is inhibited as the staffis learning the new system while perhapskeeping the old one operational. The fall inproductivity is also related to employees beingtrained and training others especially when ITtraining is provided during work hours.Disruption costs may also be associated with adecline in product and service quality.Frustration, lack of knowledge and skills, ormerely the ignorance of the IT applications’potentials may increase mistakes in invoicingand delivery, increase order and delivery timeand may lead to absenteeism. It may also bethat the new IT systems or applications arenot functioning properly, causing disruption inprocesses and demanding system supportfrom vendors.

Disruption costs are not simply restricted tohuman factors but encompass organizationalissues as well. Organizations with extensiveinfrastructures in place tend to change theirorganizational structures either by reducingthe number of management levels, redefiningthe roles of management functions, or throughbusiness process re-engineering. While theplanned ultimate result is increased efficiencyand effectiveness of the organization, theredesign of organizational functions, processes,and reporting structures often causes technicaldisruptions and breakdowns in service. Thesedisruption costs can be costly when coupledwith employees that are resisting change.Similar to the way benefits are assignedmonetary values as illustrated in Exhibit 9,disruption costs can be calculated byconsidering lost hours, decreased productivity,cost of impaired quality, or lost revenues.

In many cases, the disappointing results fromIT can be traced to poor management ofdisruption costs. With every IT initiative, therewill be a period of experimentation andexploration, followed by the productive phase.As with the introduction of breakthroughtechnological advances, benefits don’t occuruntil the technology matures and theemployees adapt work patterns and processes.Thus, those organizations that adapt faster willgain greater returns. So a key objective forevery enterprise is to shorten the time it takesto achieve effective acquisition and deploymentof IT investments.

MANAGING IT INVESTMENTPROCESSES

In practice, the demand for technology in theorganization always exceeds the capacity of theorganization to deliver. Thus, some prospectiveIT opportunities cannot be funded and onlythose opportunities that yield the greatestbenefits can be selected. The IT ContributionModel requires continuous reconsideration ofwhether the underlying assumptions of themodel actually work when applied to a specificIT initiative. As soon as there are indicesimplying that the results (outputs andoutcomes) may not be achieved, the IT initiativeshould be reconsidered and the projecteventually cancelled. Too often, organizationshave become psychologically and financiallycommitted to prior investments and find it hardto terminate projects with substantial front endand operational investments. Instead,businesslike thinking must drive both theimplementation of new IT initiatives and themanagement of IT projects in place. The focusshould be on delivering high-quality IT servicesand optimizing business value.

Managing the IT investment process thusincludes several issues:

● Managing IT investment processes typicallyoperate on the basis of stated objectives,which are achieved over a given time frame,with costs and deliverables identified atvarious stages. One should continuallyquestion the assumptions on which theoriginal investments are justified. Thismeans using analysis not just to reviewperformance against plans, but to questionthe validity of those plans based on thisknowledge and to revise the estimates.Eventually, if the likelihood of realizingexpected benefits is low, for example,because the product being developed is nolonger demanded in the organization,discontinuing the funding for the programshould be considered. Alternatively, underuncertainty, the project can be postponeduntil the risks are lower, or the scope andfunding of the program can be reduced.

● IT management should have a plan toperiodically conduct risk assessmentscovering the organization’s use ofinformation technology, including internalsystems and processes, outsourcedservices, and the use of third-partycommunications and services.

24

M A N A G E M E N T

S T R A T E G Y



● Results from an IT evaluation andmeasurement process should be widelycommunicated throughout the organization.In a well-executed IT initiative, most units ofthe organization will have some involvement.The evaluation and measurement of the ITprogram will be less successful if the resultsare not disseminated throughout theorganization to the many disparate areas thatboth affect and are affected by it. Themethodology of evaluating performance in ITas well as results should be monitoredregularly to identify areas of weakness,challenge the framework and IT plans andsystems in place, and develop new initiativesand metrics.

RECOGNIZING THE RISK ASSOCIATED WITH IT INVESTMENTS

With respect to the risk associated with ITinvestments, there are two important questions:

● How to address the risks related to actual orpotential IT investment in a balanced andcomplete way?

● To address the dilemma of over- or lack of ITinvestment, the question is whether to investor not and if so, how much?

Given the extent of IT project failure anddisappointment, it is hardly necessary toemphasize the need to include risk as animportant determinant of IT success. There areseveral areas of risk; however, organizationalrisks, project risks, staff risks and risks from theexternal environment are among the mostimportant (Murphy, 2002). Organizational risksinclude the risk of the project not being alignedwith business objectives, being incompatible withexisting organizational structures and systems, orlacking management support. Project risks relateto critical project management skills, size,complexity and duration of the project, impreciseor incomplete definition of the business problemand/or the proposed business solution, hardwareand vendor related risk, and more. Staff riskscomprise the level of user commitment as well asuser capabilities to exploit IT applications, and ITstaff stability. With respect to the externalenvironment, competitors’ actions, governmentlegislation and overall economic performance canimpact the IT implementation and potentialpayoffs. Certainly, the number of potential riskelements is even greater.

With respect to project risks, the failure to

carefully estimate the full costs of ITimplementation represents one of the mostprominent risks. Most organizations takeconsiderable care in quantifying the directfinancial implications such as the costs forhardware and software, installation andconfiguration costs, overhead, and training costs,and maintenance costs. However, these areprimarily front-end costs, which, over time, bearincreasingly little resemblance to the realoperating costs that can exceed by orders ofmagnitude the up-front expenditure. The fullcosts of IT implementation, often referred to asthe total cost of ownership, include both thedirect cost that can be attributed to theimplementation and operation of newtechnology, as well as indirect human andorganizational costs. As hardware pricescontinue to fall, IT related human andorganizational costs are rising. At the USDepartment of Defense, for every $1 spent onIT and associated equipment, a further $7 wasspent on ‘softer’ human and organizational issues(Strassmann, 1997). Human and organizationalcosts are often three to four times as high as‘direct’ project costs. Unfortunately, these costsare rarely fully budgeted for in IT investmentproposals, which may partially explain the ‘costcreep’. Better estimates of both long-term costsand benefits are necessary.

One of the largest indirect human costs is that ofmanagement time spent on integrating newsystems into current work practices, and therelated resources used for investigating thepotential of IT and experimenting with newinformation flows and modified reportingstructures. Other indirect human costs arerelated to employee motivation, employee time,and system support costs. Typical lifetimesupport costs are at least 400% of the originalpurchase price (Remenyi et al., 2000). It mayeven be that developing new employee skills andincreasing their flexibility and their overallcontribution towards the organization mayincrease pay scales and associated costs.

Indirect organizational costs, on the other hand,relate to the disruption costs, i.e. temporary lossin employee productivity and to the consequentfall in product and service quality. Indirectorganizational costs may also occur whenmanagement attempts to capitalize on the widerpotential of the system, organizationalrestructuring, or business process re-engineering. In the survey of 175 organizationsworldwide, only 35 percent of respondents

25

agreed they effectively managed the riskassociated with business change (Baxter,2004). To successfully invest in IT and managesubsequent risks one needs to undertake astructured approach, such as presented here,encompassing all potential risks and includingthe full costs and benefits of ITimplementation.

The other perspective of looking at riskassociated with the IT investment is how muchfunding should be committed to technologyinnovation. Funding may be used both toprovide access to information securely,balancing protection against business need, aswell as to create value. Studies offer a range ofnumbers, which are determined primarily byhow an organization defines innovation andtechnology. As a general guideline, however,top performing innovative organizationsroutinely invest about 40% of theirdiscretionary IT funds into upgrading existingtechnologies as well as into building newsystems and technology capabilities.Maintaining a consistent level of funding fortechnology innovation is critical for success. Itis also critical to attracting and retaining highquality IT professionals.

The most successful organizations do not leaveIT innovations to chance. It is important to setup a formal procedure that would smooth theway for the introduction and implementation ofemerging technologies. With a modelincorporating IT inputs, processes, and outputsthat lead to overall IT payoff and improvedcorporate profitability, organizations will likelyrely less on a reactive approach to adoption ofnew technologies or risk making costly,personality-driven choices. Forging IT-businesspartnerships with engineers and IT professionalscollaborating with the organization’s leaders,managers, and management accountingprofessionals is important.

ALIGNMENT OF THE IT CONTRIBUTION MODEL WITHTHE BALANCED SCORECARD,SHAREHOLDER VALUE ANALYSISAND ROI

The IT Contribution Model can be adaptedinto any management system that anorganization utilizes. It is compatible withmeasurement and management frameworkssuch as the Balanced Scorecard andShareholder Value Analysis that focus on a

better understanding of the causalrelationships and linkages within organizationsand the actions managers can implement toimprove both customer and corporateprofitability and drive increased value.Specifically, it can help determine, articulate,and measure the items in the BalancedScorecard. In fact, many new frameworks andtechniques have been developed inperformance measurement in the last 10 years.Improved measures of the payoffs of IT can besuccessfully utilized with any performancemeasurement or management system.

Balanced Scorecard for EvaluatingPerformance of IT

The Balanced Scorecard (Kaplan and Norton,1996, Epstein and Birchard, 1999) wasoriginally developed as an approach toperformance measurement that combinedtraditional financial measures with non-financialmeasures to provide managers with richer andmore relevant information about activities theywere managing. Later it became a strategicmanagement system to articulate, execute, andmonitor strategy using a mix of financial,customer, internal business process, andlearning and growth related measures. Financialperformance perspective focuses onshareholders’ interests and indicates whetheran organization’s strategy, implementation, andexecution are contributing to bottom-lineimprovement. The customer perspective includesspecific measures of the value propositionsthat the organization will deliver to customersin targeted market segments. In the internalbusiness process perspective, executives identifythe critical internal processes that will have thegreatest impact on customer satisfaction andretention as well as achieving an organization’sfinancial objectives. The learning and growthperspective identifies the infrastructure theorganization must build to create long-termgrowth and improvement. Thus, the BalancedScorecard brings together, in a singlemanagement report, many of the seeminglydisparate elements of an organization’scompetitive agenda.

Further analyses of the relationships betweenthe causes and effects of various managerialactions have been developed to specify therelationships and measures that lead tosuperior performance. These strategy maps(Kaplan and Norton, 2000 and 2004), oraction-profit linkages (Epstein and Westbrook,

26

M A N A G E M E N T

S T R A T E G Y



2001) articulate the causal relationships as seenin Exhibits 1 and 3. In this way and when used ina Balanced Scorecard, these chains of cause andeffect linking across the learning and growth,internal processes, and customer perspectives tofinancial results for shareholders are given visualform. Since the concept was introduced in 1992,balanced scorecards have been implemented atcorporate, strategic business unit, shared servicefunctions, and even individual levels. Notably, ithas also been adopted to evaluate board ofdirectors’ and individual board members’performance (Epstein and Roy, 2002).

Exhibit 10 displays the causal relationships

between different drivers of success in an ITBalanced Scorecard and can be compared toExhibit 3 on the IT Contribution Model. Again,the illustration of IT performance drivers servesas an example and should be viewed as atemplate to build upon.

All four perspectives connect in a chain of causeand effect: IT innovation and growth improveinternal business processes; internal businessprocesses improve customer satisfaction, bothinternal and external, which in turn leads toimproved financial performance. In other words,one category of measurement drivesperformance in the next. These drivers and

27

Exhibit 10: The Causal Relationships in the IT Balanced Scorecard

Financial

Customer(external & internal)

InternalProcesses

Learning &Growth

Earnings growth

Revenue growth Cost reduction

Greater sales

Customer loyalty

Customersatisfaction

Customeracquisition

Increased productivity

Improved qualityIT literate

managers andemployees

Improved IT processes

New IT products and services

IT systemsecurity

Consolidated andstandardized ITinfrastructure

Enhanced IT services

$ invested in IT, IT skillsand knowledge

IT employee stabilityindex

IT processes documentedand measured Pay for performance

compensation

subsequent measures should reinforce eachother, all contributing to measuring the impactof IT investments on the bottom line. Thefinancial perspective relating to the outcomesof the IT initiatives helps managers keep scorein the traditional sense. Many of theperformance measures included in thisperspective, such as overall cost reduction andrevenue growth, will be the same fromorganization to organization. Proceedingfurther down the causal linkage model, driversmay differ greatly but should always clearlyexpress the logic of if/then hypotheses. Forexample, if organizations document andmeasure IT processes, both in terms ofresources allocated and results achieved, theywill then have the necessary data to argue for

additional resources to be invested in ITtechnology, skills, and knowledge. As morefinancial resources are available for IT,organizations can introduce new IT productsand services, leading to acquiring newcustomers and satisfying the existing ones andleading to increased sales and profits.

To closely monitor these cause-and-effectrelationships, appropriate metrics must bedeveloped. Exhibit 11 provides a list of metricsfor the IT Balanced Scorecard along the fourperspectives. Both the objectives and metricsare consistent with the IT Contribution Model.This demonstrates how the IT payoff can beapplied for organizations that commonly utilizebalanced scorecards.

28

M A N A G E M E N T

S T R A T E G Y


29


Exhibit 11:Objectives and Metrics for the IT Balanced Scorecard

OBJECTIVES MEASURES

FinancialLong-term Corporate o % change in stock price attributable to IT initiativesProfitability/ o EVAOrganizational Success o Earnings growthShort-term Corporate o Revenue growthProfitability/ o % in overall cost reductionOrganizational Success

Customers (external)Value Capture o Average customer profitabilityCustomer Loyalty o Profitability of IT projects

o Customer satisfaction with IT activitieso % of customer attritiono Ratio of new visitors to repeat visitorso Frequency of customer return visits to the Web site

Customer Acquisition o Number of new customers gained through IT innovationo % of visitors to Web site who are also buyers (reach)o Number of created customer partnerships

Customers (internal)Improved Quality o Dollars saved on less rework

o Reduction in customer grievanceso Dollars saved on costs of quality

Increased Productivity o % increase in customer orders processed o % increase in production output per employee

Internal ProcessesIT Processes, Products and o Number of new IT products and services introducedServices o Average time required to respond to customer service requests made

through ITo Hours of Web site downtime (in a year)

Chargeback on Actual o % of IT costs allocated back to business units or functionsResource UsageIncreased Security o % of unplanned information system’s downtimeConsolidated, Standardized, o Number of IT applications that are not fully integrated within the and Streamlined IT overall IT systemInfrastructure o % of standardized hardware, databases, communications and

applications systemsLearning & Growth

Performance Measurement o % of projects’ evaluations based on ROI metricso % of IT staff with pay for performance compensationo % of IT processes documented and measured

Development of IT Skills and o $ invested in IT skills and knowledgeKnowledge o IT employee stability index

o Number of IT professionals per employee

Shareholder Value Analysis

IT’s contribution to creating business valueshould pay off in increased shareholder value.Shareholder value analysis is used in manyforms by numerous organizations formeasuring shareholder value and draws on theprinciples of discounted cash flow analysiscommonly used for purposes of investmentappraisal. It takes an investment centerapproach by deducting the cost of capital fromearnings for projects as well as organizations’or units’ analysis. The analysis presented in thisguideline is consistent with these approachesas it provides needed specificity to theidentification and measurement of the payoffsof investments in information technology.

Calculating the Return on Investment in IT

In most organizations, the difficulty incalculating an IT return on investment (ROI)weakened the case for technology investment.The measures to complete the analyses hadnot been developed. What was needed wasboth a framework for the analysis and themeasures of both the costs and benefits toeffectively calculate a ROI. This guidelineprovides a model that articulates the drivers ofROI and the causal relationships betweenthem. Specific metrics for IT inputs, processes,and outputs have also been developed toprovide IT managers and financial managerswith the necessary tools to calculate thereturn on investment in IT.

For the ROI calculation to be complete, bothbenefits and costs need to be carefullyaddressed. The challenge of calculating the ITROI primarily lies in determining projectbenefits. Few organizations really assess thebenefits of IT projects. A French study reportsthat only half of companies monitor theexpected benefits (Monnoyer, 2004). Yet, theconversion of non-financial data to monetaryunits to obtain benefits expressed in monetaryterms can and should be completed.Converting quantified data, such as output,quality, cost, and time is admittedly easier thanconverting soft data. Examples and subsequentdiscussion in this guideline have been providedto enhance this process. In many cases,subjective input is used to assign monetaryvalues to benefits derived from projects. It isimportant to recognize that benefits will occurfor a substantial time period subsequent tosystems implementation.

As many IT projects over-run their costprojections, considerable care must also begiven to estimation of costs. Amounts includedin the cost estimates of an IT initiative shouldbe based on the total lifetime costs of theproject. The total program (project) costsconsist of all related costs including planningthe IT investment, forecasting potential risks,development, and implementation includingemployee training. This also must includeoperations consisting of maintenance costs andthe necessary upgrades. In addition, disruptioncosts, both human and organizational, shouldbe included in the calculation.

In general, the ROI calculation should beperformed on a marginal-costing basis. Thus,only additional costs incurred by the newsystem should be included. In the instance of anew system, which can use available processingpower, the investment cost may be calculatedas a marginal cost of the processing powerneeded to run the additional application.Likewise only marginal benefits, i.e. new oradditional benefits, should be compared to thecosts. The ROI should be calculated prior toan IT project to estimate the potential costeffectiveness and after a project to measurethe results achieved. As the benefits of the ITinvestment will increase with time, ROI shouldnot be calculated just after the IT initiative isimplemented but yearly throughout thelifetime of the project. It then aids in thebudgeting, planning, and resource allocationprocesses along with fitting into a broadperformance evaluation and reward system.

THE APPLICABILITY OF THE ITCONTRIBUTION MODEL ANDMEASURES TO OTHER BUSINESSFUNCTIONS

Though it is important to focus this discussionon IT, it can be noted that the challenge inmeasuring performance in IT is similar to theproblems in other business functions. Humanresources managers, legal staff, environmentalquality managers, and R&D managers alsocomplain that they are unable to demonstratethe value creation benefits of their activitiesand are seen as cost centers rather than profitcenters and often have difficulty obtainingresources. These functions have recentlygrown in importance and there is increasedpressure for more accountability for resourcesused. Similar to the model and measures

30

M A N A G E M E N T

S T R A T E G Y


31


proposed for IT, other business functions thatproduce benefits that have long time horizonsand are often seen as difficult to measure, willfind it useful to more carefully identify andmeasure the payoffs of their activities. Thisprovides for improved resource allocations andimproved organizational decisions.

GUIDANCE FOR MANAGERS

In too many organizations, decision makersoverlook economic rationality in justifying ITexpenditures and instead acquire the best andmost recent technologies to outpace othersregardless of the results achieved. However, thepressure to remain competitive in a dynamic,global economy has forced many organizationsto consider the results-based approach wherethe question ‘Will we see a return oninvestment?’ is central. Instead of thinking onlyin terms of ‘Can we afford not to do it?’organizations are seeking the answer to thequestion ‘Can we afford to do it?’

Despite significant discussions in both themanagerial and academic literature concerning

the importance of evaluating the payoffs of ITinvestments, so far, there has been littleguidance of how to design or implement anappropriate IT performance evaluation system.On one hand, there was a shortage of relevantmetrics. On the other, even approaches such asthe Balanced Scorecard and Shareholder ValueAnalysis that do provide frameworks foranalysis and management, were insufficient perse. Increased specificity was necessary to model,measure, and manage the organizational linksthat operationalize these approaches.

This guideline provides an IT ContributionModel for a comprehensive evaluation of ITperformance. It describes the antecedents andconsequences of IT initiatives, determines keyIT related objectives, develops the drivers of ITsuccess and the causal relationships betweenthem, and provides numerous performancemeasures to track IT performance. Exhibit 12provides a summary of how the IT ContributionModel is developed and applied. Exhibit 13specifies more explicitly how an IT initiative ROIis calculated.

M A N A G E M E N T

S T R A T E G Y

M E A S U R E M E N TExhibit 12: IT Contribution Model:Guidance to Implementation

Corporate Strategy IT Strategy

IT Initiatives

Develop the IT Contribution Model

Preview Inputs, Processes, Outputs, and Outcomes of IT Initiative

Determine IT Initiative Objectives along All Four Dimensions

Determine Drivers of IT Initiative Success and Develop CausalRelationships between Them

Develop Performance Measures to Track the IT Initiative

Manage ITInitiative

BusinessStrategy

Exhibit 1(see page 6)



Exhibits 4-8(see page 14-17)

32

Outcomes Objectives Drivers MeasuresOutputs: Objectives Drivers Measures

o Externalo Internal

Processes Objectives Drivers MeasuresInputs Objectives Drivers Measures

Outcomes Long-term/Short-term Organizational Success ObjectivesOutputs: Value Capture, Customer Loyalty and Retention Objectives

o External Customer Acquisition, Channel Optimizationo Internal Direct Cost Savings, Improved Quality, Increased Objectives

Capacity Utilization, Time Savings, IncreaseProductivity

Processes IT Systems, IT Structure, IT Strategy, Leadership ObjectivesInputs Resources, Corporate Systems, Objectives

Corporate Structure, Corporate Strategy, External Environment

CalculateROI of ITInitiative


Exhibit 13: IT Contribution Model:Calculation of an IT Initiative ROI

33

1Calculate the Monetary Value of

IT Initiative Benefits

Outputs Benefits Monetary Value

Increased Productivity Increase in output (units produced, services offered) $...................

Time Savings Labor hours saved, machine hours saved, increased on-time deliveries reducing cost of grievances, etc. $...................

Increased Capacity Increase in output (units produced, services offered) $...................Utilization

Improved Quality Labor hours saved, machine hours saved, cost of quality reduced, increased on-time deliveries reducing cost of grievances etc. $...................

Direct Cost Savings Reduced IT expenses, reduced direct administrative and operating costs, reduced fraud incidence, reduced hours of IT downtime $...................

Channel Optimization, Increased sales from existing and new customers $................... Customer Acquisition and Loyalty, Value Creation

Total Benefits $...................

2Calculate the Total Costs of IT

Initiative

3 Calculate the IT Initiative ROI

Costs Value

Front-end Direct Costs Hardware, software, installation and configuration costs,of IT initiative overhead, training costs $...................

Disruption Costs Related Decline in labor productivity, hours lost because of IT training,to Human Factors decline in product and service quality, absenteeism, revenues lost $...................

Disruption Costs Related Technical disruptions, breakdowns in service, costs of systemto Organizational Factors support from vendors, organizational restructuring $...................

Costs of Risk Mitigation Development and implementation of IT performance framework $...................

Total Capital Costs $...................

Operating Costs of IT Initiative Direct IT operation costs, maintenance costs $...................

Total Operating Costs $...................

Total Benefits – Operating CostsROI = ------------------------------------------------------ * 100

Capital Costs (Investment)

M A N A G E M E N T

S T R A T E G Y


34

With the IT Contribution Model, managers canimplement a performance measurementsystem to more effectively evaluate theeffectiveness of IT investments, which can leadto dramatic improvements in decision making,corporate resource allocations, andperformance. To enhance the effectiveness ofthe IT Contribution Model, the following issuesneed to be considered specifically:

● A careful identification of the actual driversof IT success and a well elaborated schemeof causal relationships between them iscritical for the development of key ITperformance measures.

● Prior to the implementation of the model,baseline indicators for the specifiedperformance measures need to beestablished. A lack of information on theinitial status of the critical drivers of ITsuccess prevents drawing conclusions aboutthe actual benefits from IT initiatives aftertheir completion. With more historical datafrom both within the organization as well asfrom other organizations, benchmarks canbe established and used for the objectiveevaluation of results achieved.

● There is no precise rule for the rightnumber of performance measures includedin the model; however, the tendency shouldbe to follow the pattern of critical drivers ofsuccess along inputs, processes, outputs,and outcomes, and minimize the number ofindicators to measure them. With increasedmaturity of the implemented model and theorganization’s ability to use it effectively, themetrics may eventually increase in number.

● The IT Contribution Model is equallyapplicable to both large-sized as well assmall-sized for-profit and not-for-profitorganizations. However, smallerorganizations with less complex inputs andprocesses and relationships between themshould strive to delineate just the mostcritical drivers and thus develop a smallernumber of performance measures.

● A precise identification and measurement ofthe present and future marginal costs andbenefits of IT initiatives are fundamental fora comprehensive and objective calculationof ROI on IT initiatives. In particular,disruption costs associated with theadoption of IT initiatives require a thorough

evaluation as they are typically significant.

● To attain business value from an ITinitiative, a structured and ongoing carefulexamination of costs, benefits, and risksfrom the initial feasibility through post-implementation is needed. Even whenbusiness value is achieved, there is noguarantee that this value will be maintainedunless there is an ongoing attention to assettracking, usage data, total cost ofownership, and IT performancemeasurement.

● Specifically important is the role of CIOs inensuring that information technologycontributes to helping their companiesachieve high performance. CIOs mustactively participate in the strategic planningprocess and focus the IT discussion on therole of IT in creating business value byincreasing productivity, reducing cycletimes, lowering risks, and creating growth.

A properly implemented IT ContributionModel yields several results:

● IT managers will be able to demonstrate theimpact on corporate profitability and valuecreation from IT.

● It will assist IT managers as they evaluatethe trade-offs and decide which IT projectprovides the largest net benefit to bothshort-term financial performance as well asthe overall long-term success of theorganization.

● It will help CIOs, CTOs, CFOs and othersenior corporate and financial managers asthey develop an IT strategy to make overallcorporate resource allocations to supportthat strategy. They can rely on convincingevidence based on formal measurement andevaluation when making recommendationson these allocations. By having a clearpicture of the IT cause and effectrelationships they can monitor how the ITinitiatives are progressing and more fairlyevaluate their intermediate results.

● The IT staff will know better how well theyare performing, correct any deficiencies,and by seeing the results of their workdevelop an important sense of personalsatisfaction.

35


BIBLIOGRAPHY

Baxter, Andrew. 2004. IT Project BusinessCases Lack Ring of Confidence. Financial Times,July 7.

Cameron, Bobby, Meringer, Julie, Dawe,Christopher, and Jastrzembski, Emily. 2000.Measuring eBusiness Success. The ForresterReport (September).

Carr, Nicholas G. 2003. IT Doesn’t Matter.Harvard Business Review (May): 41-49.

Carr, Nicholas G. 2004. Does IT Matter?Information Technology and the Corrosion ofCompetitive Advantage. Boston: Harvard BusinessSchool Press.

Curley, Martin. 2004. Managing InformationTechnology for Business Value: Practical Strategiesfor IT and Business Managers. Hillsboro: IntelPress.

Davenport, Thomas H., and Prusak, Laurence.1997. Information Ecology: Mastering theInformation and Knowledge Environment. Oxford:Oxford University Press.

Devaraj, Sarv, and Kohli, Rajiv. 2002. The ITPayoff: Measuring the Business Value of InformationTechnology Investments. Upper Saddle River:Prentice-Hall.

Epstein, Marc J. 2004. Implementing E-CommerceStrategies: A Guide to Corporate Success After theDot.Com Bust. Westport: Praeger Publishers.

Epstein, Marc J., and Birchard, Bill. 1999. Applyingthe Balanced Scorecard. Management AccountingGuideline. Mississauga: The Society ofManagement Accountants of Canada.

Epstein, Marc J., and Westbrook, Robert A.2001. Linking Actions to Profits in StrategicDecision Making. MIT Sloan Management Review(Spring): 39-49.

Epstein, Marc J., and Roy, Marie-Josée. 2002.Measuring and Improving the Performance ofCorporate Boards. Management AccountingGuideline. Mississauga: The Society ofManagement Accountants of Canada.

Ferguson, Glover T., and Taylor, Stanton, J.2004. Innovation: Bold New World. Outlook: theJournal of High-Performance Business, SpecialEdition (May): 31-37.

Frank, Serena E. 2004. IT Transformations SpurEnterprise-wide Strategic Alignment. BalancedScorecard Report: Insight, Experience & Ideas for

Strategy-Focused Organizations. Boston: HarvardBusiness School Publishing, Balanced ScorecardCollaborative (March-April).

Gold, Robert S. 2004. Follow the Money: ITFinance and Strategic Alignment. BalancedScorecard Report: Insight, Experience & Ideas forStrategy-Focused Organizations. Boston: HarvardBusiness School Publishing, Balanced ScorecardCollaborative (March-April).

Hill, Jon, Plesko, David E., and Roby, Anthony.2004. Integration: Beyond Data andApplications. Outlook: the Journal of High-Performance Business, Special Edition (May): 17-23.

Intel Information Technology White Paper.2003a. Investing in IT for the Future: Intel IT’sStrategies to Reduce Costs, Manage Risks, and StayCompetitive (June).

Intel Information Technology White Paper.2003b. Managing IT Investments: Intel’s IT BusinessValue Metrics Program (August).

Intel Information Technology White Paper.2003c. Effects of Wireless Mobile Technology onEmployee Productivity: Wireless Mobility Changesthe Way Employees work (November).

Kaplan, Robert S., and Norton, David P. 1996.The Balanced Scorecard. Cambridge: HarvardBusiness School Press.

Kaplan, Robert S., and Norton, David P. 2000.The Strategy-Focused Organization: How BalancedScorecard Companies Thrive in the New BusinessEnvironment. Cambridge: Harvard BusinessSchool Press.

Kaplan, Robert S., and Norton, David P. 2004.Strategy Maps: Converting Intangible Assets intoTangible Outcomes. Cambridge: HarvardBusiness School Press.

Kasamis, Doug, and Nunn, Stephen. 2004.Infrastructure: Consolidate, Standardize,Streamline. Outlook: the Journal of High-Performance Business, Special Edition (May): 9-15.

Lutchen, Mark D. 2004. Managing IT as aBusiness: A Survival Guide for CEOs. Hoboken:John Wiley & Sons.

Mesøy, Tor, and Arlebäck, Jonas. 2004. FourHigh-performance Opportunities for CIOs.Outlook: Point of View (February).

Monnoyer, Erik. 2003. What CEOs Really Thinkabout IT. The McKinsey Quarterly, No. 3.

M A N A G E M E N T

S T R A T E G Y


36

Murphy, Tony. 2002. Achieving Business Value fromTechnology: A Practical Guide for Today’s Executive.Hoboken: John Wiley & Sons.

Osman, Joel A. 2003. Technology: WhenProducts Talk. Outlook: the Journal of High-Performance Business, No. 2 (May): 34-39.

Remenyi, Dan, Money, Arthur, and Sherwood-Smith, Michael. 2000. The Effective Measurementand Management of IT Costs and Benefits. Oxford:Butterworth-Heinemann.

Russell, Randall H. 2004. The State of IT andBusiness Alignment — 2003. Balanced ScorecardReport: Insight, Experience & Ideas for Strategy-Focused Organizations. Boston: Harvard BusinessSchool Publishing, Balanced ScorecardCollaborative (January-February).

Schubert, Karl D. 2004. CIO Survival Guide: TheRoles and Responsibilities of the Chief InformationOfficer. Hoboken: John Wiley & Sons.

Strassmann, Paul A.1997. The SquanderedComputer: Evaluating the Business Alignment ofInformation Technologies. New Canaan:Information Economic Press.

Strassmann, Paul A.1999. Information Productivity:Assessing the Information Management Costs of U.S.Industrial Corporations. New Canaan: InformationEconomic Press.

Suh, Bob. 2004. Create Value with Technology:Avoiding the Austerity Trap. Outlook: the Journalof High-Performance Business, No. 1 (February):54-61.

Tardugno, Anthony F., DiPasquale, Thomas R.,and Matthews, Robert E. 2000. IT Services: Costs,Metrics, Benchmarking, and Marketing. UpperSaddle River: Prentice Hall.

The Institute of Chartered Accountants inEngland and Wales. 2004. Chartech, The Newsand Information Bulletin for Members of the IT Faculty, Issue No. 128 (February).

Weill, Peter, and Ross, Jeanne W. 2004. IT Governance: How Top Performers Manage IT Decision Rights for Superior Results. Boston:Harvard Business School Press.

E VA L U A T I N G P E R F O R M A N C E I N I T

37

John Babiak, CMA, FCMAPresident Reynolds Food Packaging Canada Inc.Montreal, Quebec

Dennis C. Daly, CMAProfessor of Accounting Metropolitan State UniversityMinneapolis, Minnesota

Marc Labossiere, CMA, FCMALabossiere and AssociatesWinnipeg, Manitoba

William Langdon, CMA, FCMAVice President Knowledge ManagementCMA CanadaMississauga, Ontario

Stephanie Mayberry, MBADirector Corporate Financial Planning and AnalysisKeystone Mercy Health PlanPhiladelphia, Pennsylvania

John Morrow, CPAVice President, The New Finance American Institute of Certified ProfessionalAccountantsNew York, New York

Les Passuello, MBA, CPAControllerVision Service PlanRancho Cordova, California

Larry E. Russell, CPA, CITPOwnerAccounTecValencia, California

David Ryan, BSBA, CPA, CITPVice President Information ServicesArtromick International Inc.Columbus, Ohio

Robert C. Sweeting, BSc, PhD, FCAExecutive Head of Accounting and FinanceManchester Metropolitan University Business SchoolManchester, UK

Kenneth W. Witt, CPATechnical Manager, The New FinanceAmerican Institute of Certified ProfessionalAccountantsNew York, New York

This Management Accounting Guideline was prepared with the advice and counsel of:

For more information on other products available contact:

In Canada: The Society of Management Accountants of CanadaMississauga Executive CentreOne Robert Speck Parkway, Suite 1400Mississauga, ON L4Z 3M3 CanadaTel (905) 949 4200Fax (905) 949 0888www.cma-canada.org

In the U.S.A.: American Institute of Certified Public Accountants1211 Avenue of the AmericasNew York, NY 10036-8775 USATel (888) 777 7077Fax (800) 362 5066www.aicpa.orgVisit the AICPA store at www.cpa2biz.com

In the United Kingdom: The Chartered Institute of Management Accountants26 Chapter Street London SW1P 4NP United KingdomTel +44 (0)20 7663 5441Fax +44 (0)20 7663 5442www.cimaglobal.comEmail Technical Information Services at [email protected]

Documents

Evaluating Performance in Information Technology