Research article

Information technology project risk

management: bridging the gap between

research and practiceHazel Taylor1, Edward Artman2, Jill Palzkill Woelfer1

1Information School, University of Washington, Seattle, USA;2Department of Information Technology, City of Seattle, Seattle, USA

Correspondence:H Taylor, University of Washington Information School, Mary Gates Hall, Box 352840, Seattle, WA 98195-2840, USA.Tel: þ (206) 616 6110;Fax: þ (206) 616 5149;E-mail: hztaylor@uw.edu

AbstractThe gap between research and practice is strikingly evident in the area of informationtechnology (IT) project risk management. In spite of extensive research for over 30 yearsinto IT project risk factors resulting in normative guidance on IT project risk management,adoption of these risk management methods in practice is inconsistent. Managing risk inIT projects remains a key challenge for many organizations. We discuss barriers to theapplication of normative prescriptions, such as assessments of probability and impact ofrisk, and suggest a contingency approach, which addresses the uncertainties, complex-ities, and ambiguities of IT projects and enables early identification of high-risk projects.Specifically, in a case study, we examine how the project management office (PMO) atone organization has bridged the gap between research and practice, developinga contingency-based risk assessment process well founded on research knowledge ofproject dimensions related to project performance, while also being practical in itsimplementation. The PMO’s risk assessment process, and the risk spider chart that isthe primary tool in this assessment, has proven to be effective for surfacing inherent risk atthe early stages of IT projects, thereby enabling the recommendation of appropriatemanagement strategies. The PMO’s project risk assessment process is a model for otherorganizations striving to engage in effective and collaborative practices in order to improveproject outcomes. The case illustrates the importance of considering the practicalconstraints of the context of application in order to transform research findings intopractices that promote attainment of desired outcomes.Journal of Information Technology (2012) 27, 17–34. doi:10.1057/jit.2011.29Published online 4 October 2011Keywords: IT project risk management; contingency approach; project uncertainty; risk spider chart;project dimensions; research transfer

Introduction

As members of an applied, or professional, discipline,Information Systems (IS) scholars seek to advanceboth academic and practical knowledge in their field.

A key challenge, however, is to bridge the gap betweenresearch and practice and to ensure that practice is wellfounded on empirical findings. Even when a research focusis profoundly applied, such as in the information technol-ogy (IT) project management arena, the goal of advancingpractice with the benefits of research findings has often

been difficult to achieve. This paper demonstrates thesuccessful transfer of research to practice in the area of ITproject risk assessment.

In spite of extensive research for over 30 years into ITproject risk factors resulting in normative guidance on ITproject risk management, adoption of these risk manage-ment methods in practice is inconsistent (Bannerman,2008) and delivery of IT projects to required performancestandards remains an elusive target (Standish Group, 2005;

Journal of Information Technology (2012) 27, 17–34& 2012 JIT Palgrave Macmillan All rights reserved 0268-3962/12

palgrave-journals.com/jit/

Sauer et al., 2007). One key area that can drive improve-ments in IT project performance is the early identificationof high-risk projects (Pennington and Tuttle, 2007). Ifhigh-risk projects are identified early, then appropriate riskmanagement and oversight mechanisms can be implementedto mitigate the threats, and to ensure early and decisive actionon problems that arise. However, early identification ofhigh-risk projects poses many questions. What determines ahigh-risk project? Which risk factors should be considered?How can they best be evaluated? How can the organizationget a holistic picture of the risk profile of the project? Whatis the best way to manage risk on high-risk projects? Giventhe extensive body of research on IT project risk factors andrisk management approaches (see, e.g., Bannerman, 2008, fora detailed review of recent IT project risk managementresearch; and Schmidt et al., 2001, for one of the mostextensive surveys of IT risk factors), it might be assumed thatthese questions would be easily answered. It is somewhatsurprising, then, that there is very little evidence that researchknowledge on IT risk factors and risk management hasactually been applied in the workplace (Bannerman, 2008; deBakker et al., 2010). One of the biggest challenges still in theIT project domain is to convert our research understandingof IT risks and risk management into practical, usable toolsthat are easy to implement and effective in practice(Bannerman, 2008; de Bakker et al., 2010).

Our goal in this paper is to examine a successful instanceof the transformation of research knowledge on IT projectrisk management into a solution that takes into account theday-to-day exigencies of the practical situation. In parti-cular, we show how the Project Management Office (PMO)at a municipal government organization, CityOrg, taskedwith improving the success rate of key IT projects acrossthe organization, developed a risk assessment tool, wellfounded in research and also practical in its application.The risk assessment tool incorporates the extensive body ofresearch knowledge on IT risk and uncertainty, whileavoiding the practical implementation problems of tradi-tional risk management approaches by building on thecontingency approach to risk management (McFarlan,1981; Barki et al., 2001).

Our presentation of an interpretive revelatory orenlightening case study (Marcus, 1997; Yin, 2009) enablesus to make contributions to both practice and research. Inthe practice arena, the case study reported here demon-strates a substantial advance in addressing the research-practice gap in the IT project risk management area, byproviding a practical implementation of the contingencyapproach to risk management whose effectiveness inpractice has been demonstrated. From the researchperspective, by exploring how and why this organizationhas successfully utilized research knowledge on IT projectrisk management, we provide insight into what is required

if research findings are to be transformed into practicesthat promote attainment of desired outcomes.

We begin with a brief review of the literature on ITproject risk to set the scene for the case study. We thenpresent our method for examining the case study, anddescribe the development of the risk assessment tool overa period of 5 years. Finally, we discuss the significance ofthe case findings for application of research on IT projectrisk management in practice, and draw on researchutilization and knowledge transfer literature to explorereasons why research utilization was successful in this case.

Literature review: IT project riskThe body of research examining risk in IT projects spansover 30 years, with Alter and Ginzberg (1978), Boehm(1973, 1983), Brooks (1974), McFarlan (1981) and Zmud(1980) being among the early contributors establishinga foundation of research knowledge in the area. Insubsequent years, research interest in IT project riskdeveloped primarily in two directions, risk managementand risk factors, with a smaller group of researchersbuilding on McFarlan’s (1981) work to develop contingencyapproaches to project risk management.

In spite of this extensive and comprehensive body ofresearch on IT risk, there is considerable evidence that theresearch findings and recommendations are not beingapplied in practice (Pfleeger, 2000; Addison and Vallabh,2002; Kutsch and Hall, 2005; Taylor, 2005; Bannerman,2008; de Bakker et al., 2010). Both the risk factor and therisk management directions draw on models of decisionmaking based on probability and expected utility (Charette,1996; Pender, 2001; Ward and Chapman, 2003; Kutsch andHall, 2005), which are founded on assumptions that risksare discrete potential events and that their impact andprobability can be assessed with a reasonable degree ofconfidence. As we will explain, these decision-makingassumptions are key to understanding why prescriptionsfrom the risk management and risk factor strands ofresearch appear to be so difficult to apply in IT projects.

Risk management researchRisk management researchers have focused on the exam-ination of process models that provide prescriptions for riskmanagement (see, e.g., Boehm, 1991; Fairley, 1994; Charette,1996; Heemstra and Kusters, 1996; Powell and Klein, 1996;Keil et al., 1998; Barki et al., 2001; Simister, 2004), typicallyincluding variations on the four processes of risk identifica-tion, assessment, response planning, and monitoring, asshown in Figure 1. Similar process models also underpin thebest practice recommendations of practitioner organiza-tions such as the Project Management Institute’s PMBOKguide (2004) and the Association for Project Management’s

RISKIDENTIFICATION

RISK RESPONSE PLANNING– Risk Elimination– Risk Mitigation– Risk Transfer– Risk Acceptance– Contingent Action Planning

RISK MONITORING– Progress Feedback– Progress Analysis– Corrective Action

RISK ASSESSMENT– Risk Analysis– Risk Prioritization

Figure 1 Project risk-management processes.

IT risk management: research and practice H Taylor et al

18

APM Body of Knowledge (2006). As noted above, thesemodels are based on a characterization of risk as a potentialdiscrete event, with a non-zero probability of occurrenceand a quantifiable impact on the project. It is assumed thatspecific risks to a project can be identified, and that theirprobability and impact can be quantified. The recommen-dations also assume that project managers will, indeed,evaluate the probability and impact of each risk in order todevelop a risk management plan.

In practice, the assumption that project managers willfollow this decision-making process has been questioned byseveral researchers. For example, empirical studies in thegeneral management field of how managers handle risksuggest that they are typically insensitive to probabilityestimates of risk and focus on only a few aspects of risk in asituation at any given time (March and Shapira, 1987).Similarly, there is evidence that IT project managers focuson a few factors and largely ignore others (Moynihan,1997). Pablo (1999) observes that software developmentmanagers focus more on the impact of a possible riskyevent, and comparatively less on the likelihood of the eventor the extent to which it can be controlled. Such failureto consider the whole risk spectrum and uneven attentionbetween impact and probability of occurrence underminethe effectiveness of the impact–probability approach torisk management.

Risk factor researchAlthough the entire risk management sequence outlinedin Figure 1, of risk identification followed by risk analysisand risk response planning, is not often followed inpractice, the risk identification stage is commonly com-pleted (Raz et al., 2002; Voetsch et al., 2004; Taylor, 2005;de Bakker et al., 2010). This stage has been supportedthrough extensive work in the second strand of riskresearch, examining the range of risk factors that canimpact projects. The aim of risk factor researchers has beento develop complete and comprehensive checklists of riskfactors that should be considered when planning andmanaging an IT project. There is now a substantial body ofwork on the typical risk factors faced by software projectmanagers, and also the priorities placed on these riskfactors by managers (see, e.g., Alter and Ginzberg, 1978;Boehm, 1991; Barki et al., 1993; Heemstra and Kusters,1996; Sumner, 2000; Schmidt et al., 2001; Wallace et al.,2004; Taylor, 2006b). The risk checklists vary in detail andemphasis – for example, the Schmidt et al. list contains 53risk factors, while Moynihan identified 113 constructsgrouped into 22 themes – but the risks identified allgenerally fall within Taylor’s (2006b) categories of (i)project management risk; (ii) relationships risk; (iii)solution ambiguity risk; and (iv) environment risk.

From a practical application perspective, the use of acomprehensive risk factor checklist seems to be a helpfultool for project managers, both in terms of identifying keyrisks for a project and in mitigating omissions of potentialthreats. However, these checklists vary considerably in therisk factors on which they focus, raising questions aboutwhich list is most applicable for a given project (Moynihan,1997; Bannerman, 2008). Once a checklist is chosen, theremay be a tendency to assume it is complete, and therefore

to overlook possible risks specific to a given project that arenot included in the checklist (Powell and Klein, 1996).Further, identifying risks on a checklist is only the firststage in the recommended process for management of risk.Simply identifying possible risks is not a substitute foractually taking action on the risks, and a ‘checklistmentality’ approach can result in undue focus on processrather than on action (Pohlmann, 2003).

A further major weakness of the risk factor strand ofresearch lies in the assumption that project managers havecomplete, or even adequate, knowledge about which of themany risk factors might threaten their projects, and to whatextent those risk factors are present. In reality, IT projectmanagers face considerable uncertainty in determining thelikely extent of any risk factor identified as a potentialthreat, and, therefore, uncertainty about possible solut-ions, in terms of their cost and effectiveness (Pender, 2001;Ward and Chapman, 2003; Kutsch and Hall, 2005). Forexample, although most IT project managers would agreethat some degree of requirements risk is likely to occur inany IT project, it is difficult to decide whether therequirements uncertainty evident at the start of a projectis simply the typical level for an IT project, or if there areserious hidden problems that will only surface during thecourse of the project. In reality, the extent of therequirements uncertainty and its impact on the progressof the project are almost impossible to assess with anydegree of accuracy until the project is underway. Similarly,project managers rarely have the luxury of a generousbudget or schedule, but at what stage do tight budget andschedule targets actually become a risk to project success?Rather than being discrete events with a quantifiableprobability of occurrence, factors such as requirementsuncertainty and tight budgets exist on a continuum as partof any typical IT project landscape.

The failure of IT project managers to use the output ofrisk identification processes in subsequent detailed riskanalysis and response planning speaks to their uncertaintyabout whether, and to what extent, any given risk threatensan IT project. If the size and impact of the threat cannot beaccurately estimated, or if it is impossible to even anticipatecertain threats, then it is extremely difficult to decide whatrisk response action to take, and it is, perhaps, not sosurprising that project managers often do not carry out thefull risk assessment process at the beginning of theirprojects. In the face of such uncertainty, contingencyapproaches may be more appropriate.

Contingency approaches to project risk managementContingency approaches attack the risk problem from adifferent angle, by providing the project manager withdecision tools for deciding when to apply certain projectmanagement methods in order to achieve the best chance ofproject success. McFarlan (1981) was an early advocate ofcontingency approaches, recommending that risk resolu-tion strategies should be based on an assessment of theproject’s risk in terms of size, structure, and experiencewith technology. Similarly, Barki et al. (2001) argued thatthe degree of formal planning, internal integration, anduser participation should be matched to the level of riskexposure identified for a particular project, with high-risk

IT risk management: research and practice H Taylor et al

19

projects requiring higher levels of planning and oversight,and Shenhar (2001) proposed that project leaders shouldconsider the scope (or complexity) and technologicaluncertainty in the project when determining the bestapproaches for management and risk control.

More recently, some theorists have suggested that in theface of high levels of ambiguity, or ‘unforeseeableuncertainty’, project managers should not attempt to applytraditional risk management methods at all. Instead, theyshould operate on a basis of continuous learning andadaptation as changing situations unfold (Pich et al., 2002;Sommer and Loch, 2004). These researchers argue that,although traditional project risk management methodswork well in contexts where the project team can reason-ably foresee and understand potential threats, in situationswhere it is impossible to fully understand all relevantvariables and interactions, the traditional methods breakdown. In these circumstances they recommend an approachof constant environmental scanning to recognize anunforeseen event when it arises, combined with problemsolving and a willingness to modify policies in order toquickly develop an appropriate response. There is somelimited evidence that this approach is taken by experiencedIT project managers in practice. Taylor (2007) found thatexperienced IT project managers rely heavily on environ-mental scanning to pick up and learn from situational cuesthat inform adaptive responses to problems as they arise,rather than on planning actions in anticipation of possibleproblems.

The challenge from a practical perspective is how todecide when a project is ambiguous and/or complexenough to warrant a continuous learning and managementapproach, rather than the traditional probability-impactrisk assessment approach. Instead of asking about theprobability that a risk will occur in a project and its impactif it does occur, the question now becomes: Is this projectinherently risky? In this regard, we do know that certaindimensions of a project are related to project performance.Clearly, projects with higher levels of complexity, un-certainty, or criticality have higher inherent risk (Shenharet al., 2001; Howell et al., 2010). Size matters: project cost,project duration, the number of systems the project isconnected with, the number of people on the project team,and the number of outside vendors or suppliers involved inthe project are all negatively correlated with projectperformance (Martin et al., 2007). The experience of theproject manager (Standish Group, 2001; Sauer et al., 2007),the project management maturity of the organization(Herbsleb et al., 1997; Jiang et al., 2004; Subramanian et al.,2007), and the active involvement of key stakeholders,particularly the executive sponsor and end-users (Wallaceand Keil, 2004; Standish Group, 2005), are all positivelyrelated to performance. These dimensions can be readilyassessed on a simple low-medium-high scale at the beginningof a project, and do not require probability-impact estimates.Projects with more dimensions assessed at the high end ofconcern (e.g., high complexity or low project managerexperience) are likely to have higher inherent risk andrequire closer oversight, even if it is not clear exactly which ofthe 50 or so risks on typical checklists will actually apply.

There is a clear correspondence between these projectdimensions and the risk factor categories identified earlier,

as shown in Table 1. The difference is essentially one ofperspective: the risk factor research views each category ascontaining specific discrete risk events whose probabilityand impact can be determined, whereas the contingencyapproach emphasizes that all projects should be assessedalong the continuum of each dimension, resulting in adetermination of the overall riskiness of the project.

The current study: risk management and the contingencyapproachThe contingency approach, with its emphasis on evaluatingthe inherent overall risk of projects, and applying differentmanagement methods for projects with high complexityand ambiguity, could have important implications fororganizations and how they approach risk management andresource allocation for the projects in their portfolio.However, to date, few tools have been available to aidorganizations in the identification of projects with highlevels of ambiguity. Once such high-risk projects areidentified, following the contingency approach, organiza-tions can take a more detailed, environmental scanningapproach to their planning and oversight by ensuring, forexample, that the project manager has the requisitecontinuous learning management background. As there islittle research on how project managers might learn theseenvironmental scanning and response skills, and asexperienced project managers are associated with betterproject performance (Standish Group, 2005; Sauer et al.,2007), previous project experience is likely to be the best

Table 1 Correspondence between risk factor categories and project dimensionsassociated with project performance

Risk factor categories(after Taylor, 2006b)

Project dimensions associatedwith project performance

Project management risk Project manager experience(Standish Group, 2001;Sauer et al., 2007)Project management maturity(Herbsleb et al., 1997; Jianget al., 2004; Subramanian et al.,2007)

Relationships risk Key stakeholder involvement(Wallace and Keil, 2004;Standish Group, 2005)Size (number of project teammembers; number of outsidevendors and suppliers) (Martinet al., 2007)

Solution ambiguity risk Complexity and uncertainty(Shenhar et al., 2001; Howellet al., 2010)Size (number of connectingsystems) (Martin et al., 2007)

Environment risk Criticality (Shenhar et al., 2001;Howell et al., 2010)Size (cost and duration)(Martin et al., 2007)

IT risk management: research and practice H Taylor et al

20

guide here. In addition, other steps may be taken to reducethe overall risk for high ambiguity projects. For example,very large projects can be split into smaller projects(Sussman and Guinan, 1999), or the requirements specifi-cation stage can be split off into a separate project (Jianget al., 2002; Taylor, 2006a).

We now turn to the current study, and discuss theapproach of one organization to these challenges ofevaluating inherent risk in the projects in the organization’sportfolio and implementing appropriate levels of projectmanagement oversight. We examine the use of a riskassessment tool – the risk spider chart – by the PMO at alarge municipal government organization, CityOrg. Weshow how this tool enables CityOrg’s PMO to apply acontingency approach to determining levels of projectmanagement oversight, by providing a mechanism to assessa project on a number of dimensions to determine itsinherent risk.

MethodWe present a single, particularly enlightening case studythat is instrumental in providing insight to the issue ofearly identification of IT projects with high inherent riskand exemplifies successful transfer of research knowledgeon IT risk management into the practice arena. The valueof a single, in-depth case study is the insight that it canprovide into complex interactions in practice (Stake, 2000;Yin, 2009). Our attention was drawn to the case becauseof the PMO’s novel and successful approach to addressingissues of project oversight and risk management, and wewished to explore the impact of this approach.

We approached the case study interpretively, from anexploratory and collaborative practice research perspective(Zmud, 1998; Mathiassen, 2002). Collaborative practiceresearch requires attention to twin goals. From the practiceperspective, the aim is to draw on research foundations inorder to implement improvements in practice, whereasfrom the research perspective, the aim is to collect dataabout practice systematically and rigorously in order todevelop an understanding and interpretation of the practicein the light of research concepts and frameworks. As istypical of practice-driven research (Reynolds and Yetton,2007), our research team comprised both academiciansand practitioners, and the work reported here representsthe culmination of over 5 years of engagement. The authors’different roles as researchers and practitioners in theorganization provided a triangulation of researcher per-spectives on the case, in addition to the more typicaltriangulation that was achieved by seeking severalcomplementary sources and types of data (Miles andHuberman, 1994).

Both practice and research questions drove the study.From the practice perspective, the question was, quitesimply, how can CityOrg increase the success rate of itscomplex IT projects? In particular, it was conjectured thatimprovements in CityOrg’s project oversight and riskmanagement processes would result in improved projectsuccess rates. In addressing the practice question, thesecond author led CityOrg’s PMO through three practicalaction research cycles of diagnosis, planning, actionimplementation, and evaluation and reflection (Susman

and Evered, 1978; Creswell, 2008), over a period of 5 years.The first action research cycle began with a review ofrelevant project management literature and observationof CityOrg’s current risk management and project oversightprocesses, followed by the development and implementa-tion of a new risk assessment process. For each cycle afterthe first, feedback from participating project managers,review of uptake of recommendations arising from the riskassessment processes in the previous cycle, and considera-tion of subsequent project performance, all fed into thenext planning stage, together with fresh input from theresearch literature. During these action cycles, the riskassessment process has been refined and now incorporatesa risk spider chart that supports discussions of a project’sinherent risk and management approaches with the localproject manager.

From the research perspective, we were motivated by thesubstantial evidence of lack of effective uptake of riskmanagement research knowledge in IT projects in practice(Bannerman, 2008; de Bakker et al., 2010) and we soughtto understand how and why CityOrg’s risk managementinitiatives were effective. In particular, knowing thatCityOrg had tracked its project performance and coulddemonstrate improvements over time and that soundresearch evidence on IT project risks had been a key driverin the action research initiatives, a key question relatedto why these initiatives were successful – what wasdifferent? In order to support the research perspectivequestion, we examined literature on research knowledgetransfer and transformation to understand why theinitiatives in the first action research cycle, which reflectedmore traditional presentations of risk assessment, were lesssuccessful. By comparing the early and late action researchinitiatives in the light of knowledge transfer researchframeworks, we were able to shed light on the question ofwhy the transfer of research knowledge to practice in thisarea of IT project risk management was successful in thisinstance.

Data for analysis of the research perspective question wascollected from several sources, in order to providetriangulation of sources and data (Miles and Huberman,1994). Publically available documents were reviewed forbackground information on the events leading up to theestablishment of CityOrg’s PMO and its activities sinceestablishment. Organizational records provided historicaldata on the process improvement actions and recordsof project performance within the organization over the5-year time period of the action research cycles. The secondauthor provided detailed reflections and comments onthe development and refinement of the risk assessmenttool, and its use on over 100 projects through the actionresearch cycles. We examined in-depth detailed data on11 projects, in eight different departments, assessed withthe tool including project details, risk assessment andrecommendations, and outcomes (a summary of theseprojects is provided in the Appendix, Table A1). Finally, weconducted brief semi-structured interviews with the projectmanagers of those projects seeking feedback on theirexperiences with the risk assessment process and the riskspider chart.

Our final analysis of the overall action research processand the data collected followed an interpretive pattern,

IT risk management: research and practice H Taylor et al

21

iterating between the data and research literature on bothIT project risk management and research knowledgetransfer (Walsham, 2006). Our process followed three keystages (Miles and Huberman, 1994; Wolcott, 1994):description (i.e., summarizing what happened during theaction research cycles); analysis (i.e., systematically identi-fying key factors and relationships); and interpretation(i.e., iterating between theory and our descriptions andanalysis of data to draw interpretations and conclusions).As an exploratory and interpretive study, our conclusionsare propositional and provide the foundation for furtherinvestigations.

The case studyThe case organization is a large municipality, CityOrg,comprising about 34 departments and municipal offices.CityOrg has about 10,000 employees, and supportsan estimated 600,000 constituents and customers. Theorganization has a federal governance mode (Sambamurthyand Zmud, 1999), with most of the 34 departmentssupported by a centralized computing infrastructure andcentralized financial and personnel software applications,but each department being responsible and accountable forthe success of its unique business software applicationprojects. The Chief Technology Officer (CTO) leads thePMO and is jointly accountable with each department forthe success of its IT projects. Joint accountability increasesthe need for visibility and oversight of IT projects andthe need for collaboration and cooperation betweendepartments and the office of the CTO.

The impetus for establishing the PMO came from acritical IT project that ran substantially over schedule andbudget, garnering extensive negative media attentionregarding waste of money and public resources. Thus, in2001, reacting to the negative publicity, the CTO establishedthe PMO to implement project management processessupporting a set of core competencies that would facilitatethe completion of CityOrg’s IT projects on time, withinbudget, and according to performance requirements. ThePMO was initially staffed with a single senior, highly skilledproject manager, who had a track record of successfullydelivering required project performance and was wellversed in project management methodologies. In 2002, asecond staff member was added, with skills in projectauditing and rescuing troubled projects. The currentstaffing level for the PMO is three.

The development of the risk assessment processA primary goal of the PMO was to increase the success ratesof complex IT projects and provide project status visibility tohigh-level stakeholders and sponsors, and the initial policyrequiring project oversight on certain key projects wasestablished in 2001. At this stage, a very blunt contingencyapproach was taken, with the requirement for independentproject oversight being determined by the CTO, basedprimarily on the assumption that high-cost projects aremore risky and therefore require more centralized oversight.For all other projects, risk assessment was left to theindividual departments, where individual project managerseither did no risk assessment, or followed the traditionalimpact–probability approach to risk management. The

introduction of the oversight requirement for high-costprojects was not well received by most departments: ifindependent project oversight was mandated by the CTO,it was generally perceived by the project department as anon-value added expense, with only a few departmentsacknowledging some benefits of the process.

In spite of the initial resistance from most departments,the oversight policy was seen centrally as beneficial, and in2004, the PMO worked to expand the process by introdu-cing more formal risk profile reviews to determine thedegree of risk associated with all projects in CityOrg’sportfolio. The first step in developing the risk assessmentprocess was to determine how to classify the projects inthe portfolio, in order to decide what level of centralizedoversight was required. Four levels of oversight could beapplied: (i) No Oversight Required – the project departmentis solely responsible for project outcomes; (ii) DashboardReporting – a monthly written status report is prepared bythe department project manager and reviewed with PMOstaff; (iii) Checkpoint Reviews – an independent qualityassurance consultant conducts in-progress audits of theproject at key project milestones or phase exits; and (iv)Formal Quality Assurance – an independent qualityassurance consultant provides continuous review through-out the project life cycle. It was at this point that CityOrgbegan to move from a traditional impact–probabilityapproach toward a contingency approach of determininglevels of inherent risk in projects according to anassessment of a number of project dimensions. The PMOdrew on both the extensive experience of the founding PMOstaff and various research publications (including, e.g.,Standish Group, 2001, 2005 on IT project risks; andWysocki et al., 2000 on classifying projects) in order todetermine a set of measurable and defensible attributes thatcould be used to categorize projects according to theirlevels of inherent risk. (We discuss these attributes in moredetail in the next section.)

These risk profile reviews provided project departmentswith broad-based information about the characteristicsused by the PMO staff to make an expert-level judgmentabout the risk and corresponding appropriate oversightlevel for each project. Although some departments wel-comed this more structured approach, generally theassessments were met with mixed reviews from projectdepartments, and overall, some rather stiff resistance wasobserved. The risk profile findings were presented todepartments in a narrative format, and although the intentwas to determine overall project risk, the main focusremained on the assessment of individual attributes, withlittle emphasis on the holistic risk picture for the project.This presentation often resulted in challenges to the PMOfindings regarding the overall level of inherent risk, becausedepartment project managers focused on individual attri-butes, and argued about whether each attribute, individu-ally, was a threat to project success.

Reflecting on the departmental reactions to their over-sight decisions, PMO staff recognized the need to do moreto move the focus of the discussion onto a project’s overalldegree of uncertainty, in order to avoid a negative spiralof debate with project department staff about whether ornot specific individual risks existed. At this stage theyexperimented with different approaches for presenting their

IT risk management: research and practice H Taylor et al

22

analysis, with a goal of finding a more visually impactfulapproach that would provide a synthesis of the holistic riskpicture of the project. In 2007, drawing on ideas forgraphical models in Boehm and Turner (2004) and Wysocki(2001), the second author developed a risk spider chart (orradar diagram) that incorporated 12 dimensions that,together, could be used to assess a project’s overall risk andalso highlighted that these factors existed on a continuumfor all projects. The PMO continued to refine the risk spiderchart as they gained more experience with the process, andreflected on their assessments and the final outcomes ofcompleted projects. The current version, shown in Figure 2,includes 18 dimensions that together enable the PMO staffand the department project manager to build a detailedpicture of the overall risk of a project and particular areasthat may require close attention.

The chart provides a visual representation of aggregaterisk that is accessible and easy to discuss, providing

specific measurable points along a continuum for eachdimension and showing the points where the dimension(e.g., cost estimate) becomes a high, moderate, or lowconcern. The discussion with the department projectmanager became more focused on the overall risk for theproject, and when an individual factor or dimension wasexamined, the question was not whether or not it existed,but whether it was sufficient to be a threat and how best tomanage it. The introduction of the risk spider chartinto the assessment process minimized the challengesabout the reality of the risks in a given project, as thedimensions represent measurable attributes of all projects(Figure 2).

The use of the risk spider chart enabled the PMO staffto maintain the focus on assessing the overall inherentrisk of the project and appropriate managementapproaches instead of getting caught up in debatesabout the existence of particular risks. The visual

Level – 1: Simple, Low RiskLevel – 2: Moderate, Medium RiskLevel – 3: Complex, High Risk

Duration(Months)

Criticality

Cost Estimate

External Project /Process

Dependencies

Data ConversionComplexity

ApplicationInterface Complexity

Technology

Changes to BusinessProcesses/Rules

Team Size

Customization /Configuration

End-UserInvolvement

Span of Impact(# of Depts., Agencies,

External Orgs, etc.)

External Visibility

Internal Visibility

ExecutiveSupport Match

(Sponsor &OrganizationCommitment/Availability/Experience)

PMExperience

Match

ScopeUncertainty

PM & SDLCMethodologyMaturity Level

Life/Safety

Legal/RegulatoryMandate

KeyEnhancement

DiscretionaryProject/Maint.

Minor

36 +

24

12

9

6

>$10 M

$6 -10 M

$3 -6 M

$1M

$500K

$ 1 0 0 K

Provenin Dept

Proven in City,New inDept.

Proven inIndustry,New in

Dept/City

Newin Industry

Heavy

Heavy

Moderate

Moderate

Light

Light

None

NoneNone

Light & WellUnderstood

Heavy

4+

3

2

1

0

High

Certain

Minor

Moderate

Mayoral / CouncilPriority

Multiple DeptImpact

Direct Impact onCustomers

Indirect Impacton Customers

InternalDept. Only

Multi-DeptVisibility

1

2

3

4

5

> 5

5

10

20

15

25

50

100

3High

Medium

Low

High

Medium

Low

GoodFitMinor Gap

Other ExternalInterest

Low

ExternallyFunded

Moderate &Well

Understood

Not WellUnderstood

Dept.Dir.Acct.Ag.

Strong

$2M

Significant Gap

SignificantGap

Moderate Gap

ModerateGap

Minor Gap

0

1

2

MissionCritical

(# of Business &Technical Resources)

Figure 2 Example completed risk spider chart with 18 risk dimensions (for Project F in Appendix Table A1).

IT risk management: research and practice H Taylor et al

23

presentation and holistic approach were particularlyappreciated:

It [the spider chart] helped to synthesize the wholepicture to see all the risks together. [PM I]

It [the spider chart] made it easier to see the ‘red-flagged’areas. [PM J]

It [the spider chart] really helps show where the focusneeds to be. [PM C]

Additional project department support for the process grewas project managers realized they could leverage the riskassessment process to get additional support from theirown senior managers to resolve ongoing issues:

It would have been difficult to persuade management to addbudget for contingency funds without [the chart]. [PM H]

PM E used the chart to emphasize a point with executivesand commented:

It [the spider chart] had an influence on the decision tobreak up the project and helped them [the executives]focus on getting agreement on the scope.

As CityOrg’s project managers gain more experience withthe risk assessment process, it is becoming embedded intheir own personal project management methods:

It [the risk assessment process] has really helped improve[our] perspective and use of risk management disciplinesin [our] day-to-day- activities. The real test is, if it [thespider chart] gets posted on a wall as a ready reference,[then] it has value. I see a lot of these posted. [PM G]

We have adapted to use [the process] internally y, evenfor projects that aren’t under official oversight. [PM B]

Risk dimensionsAs shown in Table 2, the dimensions address the range offactors, identified earlier in the literature review, that havebeen linked to project performance: criticality; uncertainty;complexity; size; project management experience; andstakeholder involvement. These 18 dimensions provide ameasurable way of assessing the overall level of inherentrisk in a project, without assuming that risks are discreteevents, present or not, and without requiring any assess-ment of probability and impact of given factors. Instead,the risk assessment is underpinned by the knowledge thatthe high end of these dimensions is typically associatedwith poorer project performance. For example, instead ofattempting to assess the extent of a project’s requirementsuncertainty, and its impact and probability, CityOrg usesspecific dimensions of scope and technology uncertainty,and changes to business processes, evaluated on simple lowto high scales, as part of an overall assessment of the projectand then determines recommendations for an appropriatemanagement approach based on the overall inherent risk.

Criticality dimensionThe three attributes of criticality – safety/mission criticality,external visibility, and internal visibility – are used as ‘redflag’ attributes, because the consequences of adverse eventson these types of project are so severe (Shenhar, 2001; Howellet al., 2010). High levels on these attributes prompt a carefulassessment of other dimensions to ensure that the riskinessof the project is not compounded by problems in otherattributes. For example, projects with safety/mission criticalityassessments involving life and safety require high levels ofproject management maturity to ensure that all appropriatesafeguards are in place and to minimize the probability ofadverse consequences. Projects with safety/mission criticalityinvolving legal and regulatory mandates are frequently drivenby an externally imposed duration and inflexible scope, whichminimize project management trade-off capabilities. Projectswith high external visibility are of particular concern becauseof possible negative attention from the public, whereas highinternal visibility projects are often subject to internalpolitical attention. High levels on the criticality dimensiontypically lead to recommendations of the highest levelsof oversight, combined with efforts to ensure that the projectmanager has a proven track record with projects of this type.

Uncertainty dimensionThree aspects of uncertainty are considered: scope un-certainty, technology uncertainty, and changes to businessrules. High levels of both scope uncertainty and technologyuncertainty are frequently associated with poorer projectperformance (Shenhar, 2001), and it is often difficult toresolve these uncertainties until considerable progress onthe project has been made (Ward and Chapman, 2003).Moderate to high levels of changes to business processes/rules are associated with more difficulties and greaterresistance from users, and, again, the full extent of thesedifficulties is often not understood until well into theproject. For projects with high levels of uncertainty, thePMO works with the project manager to clarify scope andresolve technology and business uncertainties beforeexiting the initiation stage.

Complexity dimensionThe complexity attributes – customization/configuration,data conversion complexity, application interface complex-ity, external project/process dependencies, and span ofimpact – are related to the uncertainty attributes, in thathigh scores on complexity are also likely to be associatedwith increased uncertainty about scope and technology(Shenhar, 2001). The span of impact measures complexityin terms of the number of different organizations, suchas other departments, vendors, agencies, and other externalorganizations, that are involved in the project. Similarly,external project/process dependencies introduce the compli-cation of greater collaboration and coordination with otherproject teams and departments. A key challenge in the com-plexity area is accurate estimation of the effort required;these attributes are frequently initially underestimated bothbecause of lack of experience with the application and alsouncertainty about the extent of complexity of the variousattributes (Sommer and Loch, 2004).

IT risk management: research and practice H Taylor et al

24

Tab

le2

Ris

kspid

er

chart

dim

ensio

ns

Fa

ctor

Dim

ensi

onM

easu

rem

ent

Ra

nge

(low

est

con

cern

–hig

hes

tco

nce

rn)

Ra

tion

ale

Cri

tica

lity

(Sh

enh

ar,

2001

;H

ow

ell

eta

l.,

2010

)Sa

fety

/mis

sio

ncr

itic

alit

yM

ino

rm

ain

ten

ance

–li

fe/s

afet

yL

ife/

safe

typ

roje

cts

req

uir

em

ost

rigo

rou

sp

roje

ctm

anag

emen

td

isci

pli

nes

tom

inim

ize

adve

rse

con

seq

uen

ces

Inte

rnal

visi

bil

ity

Inte

rnal

dep

artm

ent

on

ly–

may

ora

lo

rci

tyco

un

cil

pri

ori

tyP

roje

cts

that

inte

rest

sen

ior

exec

uti

ves

are

mo

reli

kel

yto

be

sub

ject

top

oli

tica

lco

mp

lexi

ties

Ext

ern

alvi

sib

ilit

yL

ow

–d

irec

tim

pac

to

ncu

sto

mer

sP

roje

cts

that

are

visi

ble

toth

ege

ner

alp

ub

lic

can

attr

act

neg

ativ

ep

ub

lici

tyif

pro

ble

ms

aris

e

Un

cert

ain

ty(S

hen

har

,20

01;

War

dan

dC

hap

man

,20

03)

Sco

pe

un

cert

ain

tyH

igh

cert

ain

ty–

hig

hu

nce

rtai

nty

Incr

ease

sli

kel

iho

od

of

cost

/sch

edu

leo

ver-

run

san

dq

ual

ity

sho

rtfa

lls

Tec

hn

olo

gyu

nce

rtai

nty

Pro

ven

ind

epar

tmen

t–

new

inin

du

stry

New

,u

nfa

mil

iar

tech

no

logy

crea

tes

un

cert

ain

ty

Ch

ange

sto

bu

sin

ess

pro

cess

es/r

ule

sN

on

e–

hea

vyH

igh

-to

-mo

der

ate

leve

lso

fb

usi

nes

sch

ange

are

asso

ciat

edw

ith

mo

red

iffi

cult

ies

and

grea

ter

resi

stan

ce

Co

mp

lexi

ty(S

hen

har

,20

01)

Cu

sto

miz

atio

n/

con

figu

rati

on

No

ne

–h

eavy

Hig

her

leve

lsca

nb

eu

nd

er-e

stim

ated

and

oft

enin

volv

eu

nan

tici

pat

edp

rob

lem

s

Dat

aco

nve

rsio

nco

mp

lexi

tyN

on

e–

hea

vyF

req

uen

tly

un

der

esti

mat

edan

din

crea

ses

lik

elih

oo

do

fco

stan

dsc

hed

ule

ove

r-ru

ns

Ap

pli

cati

on

inte

rfac

eco

mp

lexi

tyL

ow

–h

igh

Nu

mb

eran

dco

mp

lexi

tyca

nb

eu

nd

er-e

stim

ated

and

oft

enin

volv

eu

nan

tici

pat

edp

rob

lem

s

Ext

ern

alp

roje

ct/p

roce

ssd

epen

den

cies

0–4+

Pro

ject

com

ple

xity

incr

ease

sas

nu

mb

ero

fd

epen

den

cies

incr

ease

s

Span

of

imp

act

1–4

5C

om

ple

xity

incr

ease

sas

nu

mb

ero

faf

fect

edo

rgan

izat

ion

sin

crea

ses:

use

rre

qu

irem

ents

bec

om

em

ore

div

ersi

fied

and

gain

ing

con

sen

sus

bec

om

esm

ore

dif

ficu

lt

Size

(Mar

tin

eta

l.,

2007

;Sa

uer

eta

l.,

2007

)D

ura

tio

n6

mo

nth

s–

36+

mo

nth

sR

isk

incr

ease

sas

du

rati

on

incr

ease

s

Co

stes

tim

ate

US$

100

tho

usa

nd

–4

US$

10m

illi

on

Pro

ject

succ

ess

dim

inis

hes

asco

stin

crea

ses

Tea

msi

ze5–

100

Org

aniz

atio

nal

and

com

mu

nic

atio

nco

mp

lexi

tyin

crea

ses

aste

amsi

ze(b

usi

nes

san

dte

chn

ical

reso

urc

es)

incr

ease

s

Pro

ject

man

agem

ent

mat

uri

ty(H

erb

sleb

eta

l.,

1997

;Ji

ang

eta

l.,

2004

;Sa

uer

eta

l.,

2007

;Su

bra

man

ian

eta

l.,

2007

)

Pro

ject

Man

ager

&Sy

stem

sD

evel

op

men

tL

ifec

ycle

(SD

LC

)M

eth

od

olo

gyM

atu

rity

Lev

el

3–0

Ris

kin

crea

ses

asm

atu

rity

leve

lsd

ecre

ase

Pro

ject

Man

ager

Exp

erie

nce

Mat

chG

oo

dfi

t–

sign

ific

ant

gap

Hig

hly

pro

fici

ent

pro

ject

man

ager

sre

du

cep

roje

ctri

sk

Stak

eho

lder

invo

lvem

ent

(Wal

lace

and

Kei

l,20

04;

Wal

lace

eta

l.,

2004

)

En

d-u

ser

invo

lvem

ent

Hig

h–

low

Ris

kin

crea

ses

asen

d-u

ser

invo

lvem

ent

dec

reas

es

Exe

cuti

vesu

pp

ort

mat

chSt

ron

g–

sign

ific

ant

gap

Stro

ng

invo

lvem

ent

red

uce

sp

roje

ctri

sk

IT risk management: research and practice H Taylor et al

25

Size dimensionProject size, as measured by the attributes of duration, costestimate, and team size, has been shown to be negativelyassociated with budget and quality performance (Martinet al., 2007; Sauer et al., 2007). There is overlap between thecomplexity and size dimensions, in that longer and costlierprojects tend to be more complex, and larger teamstypically involve greater communication and organizationalcomplexity. Thus, higher ratings on the size attributesare likely to be associated with higher complexity ratings,and, conversely, actions taken to reduce one of the sizeattributes, such as splitting a very long project into shortersub-projects, are likely to also result in reduced complexityratings.

Project management maturity dimensionBoth the project management maturity level of anorganization (Herbsleb et al., 1997; Jiang et al., 2004;Subramanian et al., 2007) and the project manager’sexperience (Standish Group, 2001; Sauer et al., 2007) havebeen linked to project performance. In CityOrg, the projectmanagement maturity level varies across departments,and while some departments have highly experiencedproject managers on their teams, others have relativelyinexperienced managers. The aim when evaluating thisdimension is to ensure that the department has assigneda project manager with sufficient experience for the type ofproject and that the project manager is able to select andscale a project management methodology well suited to thetype of project. High concern measures on the attributesin this dimension result in recommendations such asassigning a more experienced project manager or intro-ducing an experienced mentor into the process.

Stakeholder involvement dimensionGiven strong research evidence of the importance of keystakeholder involvement for project success (Wallace andKeil, 2004; Wallace et al., 2004), two aspects of stakeholderinvolvement are considered. Indicators of strong executivesupport match include the executive’s availability and activeinvolvement, and the adequacy of commitments forfunding and resources, as well as the extent of the sponsor’sexperience with projects of a similar scope and complexity.For the end-user involvement attribute, project plans areassessed to determine whether adequate involvement of keyend-users has been built in throughout all project stages,but especially the early requirements stages. Where gaps areidentified in involvement of either type of key stakeholder,remedial efforts are initiated to minimize the shortfalls.

Applications of the risk spider chartThe 12-point spider chart has been used in approximately70 projects, and the recently introduced 18-point versionhas been used in over 30 projects, most of which are stillunder way. A typical process involves the local projectmanager completing a questionnaire about the proposedproject. A PMO staff member then completes the un-certainty assessment of the project, and prepares the riskspider chart and oversight recommendation for reviewwith the local project manager. An example chart is shownabove in Figure 2, and Table A1 in the Appendix details

assessments for a representative selection of projects in thelast 2 years.

In addition to the oversight recommendations, PMO staffwork with the local project manager to suggest methods ofreducing the overall inherent level of risk. These methodsinclude splitting large projects into separate smallersub-projects (i.e., reduction in project size and complexity);assigning a more experienced project manager or bringingin a senior project manager to act as a guide and mentor(i.e., increasing the project management experience);implementing standard project management techniquessuch as labor tracking, budgeting, and detailed schedulesand work-plans (i.e., increasing the project managementmaturity level); for multi-department projects, taking stepsto obtain departmental buy-in to the project, includinga memorandum of agreement between departments and asteering committee of representatives across departments(i.e., addressing stakeholder involvement); and clarifyingscope, roles and responsibilities, reevaluating proposedsolutions, and holding risk workshops (i.e., addressinguncertainty). In addition, high-risk projects, with highlevels of criticality, are typically assigned the highest levelsof oversight requirement.

The PMO recommendations are not mandatory. Fromthe PMO’s perspective, some less successful projects haveresulted when their recommendations have not beenfollowed by the project department. However, even thoughthe PMO’s recommendations are not always adopted bythe project departments, CityOrg’s project performancehas improved, in terms of key project metrics. Of the 14projects carried out in 2006, 57% were completed within10% of budget, and 36% were completed within 10% of theplanned schedule. While there is still substantial variationfrom project to project, of the 46 projects carried out from2007 through the first half of 2010, 76% were completedwithin 10% of budget – a 19% improvement over the 2006benchmark – and 43% were completed within 10% ofplanned schedule – a 7% improvement over the 2006benchmark. These improvements cannot be attributed toany single factor, owing to concurrent efforts to improvemany aspects of project management practice within theorganization, but the enhanced visibility provided by therisk spider chart and corresponding growth in projectmanagement maturity and awareness of risk are likely tohave played a substantial role in project performanceimprovements. The oversight provisions have raiseddepartmental awareness of the importance of projectmanagement methodologies and performance monitoring,while the risk spider chart provides a focus for discussionand coaching, both with the project department sponsorsand the project managers:

I used the chart with the steering committee to showareas of risk we need help in resolving and helped thesteering committee understand how they can helpmanage obstacles. [PM C]

DiscussionReal life IT project management is messy, and CityOrg’sexperience is no exception. Faced with public concern

IT risk management: research and practice H Taylor et al

26

about their project performance, they launched manyinitiatives to try and improve project management acrossthe organization, including the introduction of a cen-tralized early risk assessment process for their ITprojects. Over the last 5 years, CityOrg has moved froma very blunt oversight approach based simply on projectcost to a much more nuanced method, which takes intoaccount a holistic view of a range of project dimensions,providing better support for early detection of riskyprojects and allowing for mitigation actions. CityOrg hasbeen able to demonstrate a steady improvement in keyproject metrics, and while there are likely many factorsthat have contributed to this improved performance wehave chosen to focus on one major initiative, the riskassessment process and associated risk spider chart,because of its originality and practicality, and its strongfoundation in IT risk management research.

In keeping with the twin research and practice goals ofcollaborative practice research, we begin by discussing theinsights that this case provides into overcoming the barriersto utilization of research knowledge in practice, and then turnour attention to the implications of the case for practice.Finally, in this section, we discuss limitations of the study.

Overcoming barriers to use of IT risk management researchknowledge in practiceUnderstanding how research can successfully advancepractice in a given field has been an on-going challengefor scholars in professional disciplines (Van de Ven,2007). Within the IS discipline, the relevance of researchhas received considerable attention over a number ofyears (Marcus, 1997; Benbasat and Zmud, 1999), althoughmost recently Straub and Ang (2011) have challengedlong-standing concerns that IS topics are not useful andthat IS knowledge transference is not occurring. Cer-tainly, in the IT project management arena, neither ofthese concerns seems to be applicable; the topics of ITproject management research are clearly very relevant,and the findings have been widely disseminated throughpractitioner-oriented outlets such as PM Networkr andhave been incorporated into best practice prescriptionspromoted by professional organizations such as theProject Management Institute in the United States andthe Association for Project Management in the UnitedKingdom. Yet we still have strong evidence that IT projectrisk management research findings are not being applied(Bannerman, 2008; de Bakker et al., 2010), suggesting thatthe problem is deeper than failure of researchers toinvestigate relevant topics or failure to transfer researchknowledge to practitioner-oriented media. Strangelythough, there has been little attention to the question ofwhy these findings and best practice prescriptions are notbeing applied in practice.

Researchers in other disciplines, such as managementand public policy, have argued that simply producingrelevant research is not enough; research knowledge mustbe transformed or reconstructed to meet the idiosyncrasiesand constraints of practice contexts in order to make itmanagerially useful (Nutley et al., 2003; Rasche andBehnam, 2009; Markides, 2011). Most typically, suchtransformation requires the active participation of practi-

tioners, who can convert research findings into actionthrough the operation of their expertise in particularsettings (Desforges, 2000; Markides, 2011). In addition,knowledge produced in the research context is oftenpresented in a prescriptive, text-based format that workswell in communicating to other academics, but is lesseffective than visual representation formats in conveyingmeaning and providing cognitive support to managers(Desforges, 2000; Worren et al., 2002). The case studydescribed in this paper illustrates how research knowledgetransformation can be achieved in practice through thesetwo key aspects – active participation of practitioners andvisual presentation of research knowledge.

First, the PMO staff at CityOrg were research-oriented, sothey began from a starting point based in the substantialbody of knowledge on IT project risk management andused a rigorous approach to developing and evaluating theinitiatives in each action research cycle. At the same time,the PMO brought a practitioner perspective, founded in thepractical realities of the organizational context, to theconceptualization of the problem and the evaluation of theoutcomes of each action research cycle. The practical actionresearch supported field experimentation in context andallowed an emerging understanding of the situationalconstraints that can obstruct effective application of anyproposed problem solution, resulting in the development ofa risk assessment process that was both research-based andable to be utilized effectively in the practice context.

In particular, the action research approach used heresurfaced a practical solution that addresses concerns that,in practice, managers tend to focus on only a few risks andto place more weight on impact than on probability ofoccurrence of these risks (Moynihan, 1997; Pablo, 1999). Asshown in Tables 1 and 2, the project dimensions used byCityOrg in their risk assessment process are associated withdifferent categories of risk from the research literature. Byfocusing on measurable dimensions of projects that areknown to be linked with project performance, CityOrgaddresses risks indirectly, based on the assumption thathigher measures on the dimensions are associated withhigher risk. This indirect approach allows CityOrg toensure that all risks are covered in the assessment withouthaving to identify individual risks. CityOrg’s process alsoshifts the focus away from the uncertain ground ofestimating probability and impact for each risk, and ontomeasurements of project dimensions that can be moreeasily and accurately determined even with the relativelyincomplete information typically available at the beginningof a project.

The second critical aspect of the case was the recognitionby the PMO staff of the need to develop a better means ofcommunicating their risk assessments to local projectmanagers and other stakeholders. A simple change inpresentation, from a text-based to a graphical format,played a major part here in making relevant research usefulin practice. Although radar diagrams such as CityOrg’sspider chart are of course not new, and have been promotedfor many years as an effective means of condensinginformation on many variables into an intuitively under-stood format (Tufte, 2001, is the classic work), thisparticular implementation for the purpose of holistic riskassessment is new and worthy of wider dissemination. The

IT risk management: research and practice H Taylor et al

27

risk spider chart, with its easily assimilated graphicalpresentation of risk assessments, provides a quick andimpactful reference and summary of the project character-istics, and acts as a boundary spanning object (Levina andVaast, 2005), enabling the PMO staff to develop effectiveworking relationships with local department projectmanagers. Project managers use the spider charts on anon-going basis, pinning them on their office walls as avisual reminder of the likely problem areas in their projectsand using them to aid communication with sponsors,steering committees, and other stakeholders.

Implications for practiceThe contingency approaches to IT project risk managementrecommended in the research literature have provided littleguidance for practical implementation. CityOrg’s approachdemonstrates how to take a research-based recommenda-tion to use contingency approaches to manage uncertaintyand turn it into a practical solution. The recommendedactions arising out of the risk assessment process revolvearound reducing uncertainty and complexity in theproposed projects. In particular, following the contingencyapproach, high-risk projects are subject to higher levels ofplanning and oversight, with project management strength-ened through the application of project managementmethodologies and the assignment of more experiencedmanagers and mentors. Although CityOrg’s PMO does notexplicitly advocate use of situational awareness approachesto project management, as recommended by researchersin the contingency strand of risk management research,their approach does set the stage for a continual awarenessof the high complexity and ambiguity dimensions.

The implementation of many concurrent initiatives toimprove project management practices at CityOrg pre-cludes the attribution of performance improvements to anyone of these initiatives. However, aspects of the successfulimplementation of risk management practices in this casecan provide insights that may help other organizations toapply research-supported risk management practices intheir IT projects. As de Bakker et al. (2010) note, manyof the risks in IT projects are epistemic rather than proba-bilistic, and risk mitigation decisions in such circumstancesdepend on the availability of sufficient information. Inaddition, each project is likely to have context-specificproblem areas that must be addressed. In these situations,traditional probability-impact risk management ap-proaches break down, but, as this CityOrg case shows,organizations do have practical alternatives to address andmanage risks in their projects.

The risk assessment process and risk spider chartdescribed in this case study comprise a useful toolkit fororganizations struggling to get to grips with their IT projectperformance. CityOrg’s typical process involves a PMOstaff member working with the local project manager at thestart of the project to capture details of various projectdimensions. These dimensions are well founded in researchas being significantly related to project performance, andare easy to assess at the early stages of any project. Theproject dimensions are used to determine oversightrecommendations for the project, and are displayed on arisk spider chart, which is used for discussions on

appropriate risk mitigation and management approacheswith the local project manager and with various projectstakeholders. The visual presentation on the risk spiderchart provides an easy-to-use and impactful display bothfor discussion and collaboration about the project chal-lenges and for quick reference throughout the course of theproject. Together, the assessment process and chart form apowerful tool for the application of the contingencyapproach to IT project risk management and deserve wideradoption in the business world.

LimitationsPractice-driven research such as this study, involving aseries of action research cycles at a single site, is clearlylimited in terms of its generalizability. Action research is anemergent process in a fluid and constantly changingenvironment. Each planned intervention changes thesituation under study and feeds into a new cycle ofevaluation and action. The research takes place on-site and,as with all field studies, opportunities to control exogenousvariables are limited or non-existent. The researcher is aparticipant in the process and hence the objectivity ofreporting may vary depending on the level of researcherinvolvement. We have taken a number of steps to mitigatethese limitations, using triangulation of researcher per-spectives to mitigate potential bias from our participantinvolvement in the action cycles and triangulation ofinformants and data sources to provide complementaryperspectives on the case. Viewing the findings in thecontext of two contrasting perspectives of risk managementtheory – probability-impact and contingency – provides atheoretical triangulation (Patton, 2002) to support ourinterpretation of the outcomes of the action cycles.

We do not claim any generalizability from a single case,as every organization will operate within a different set ofenvironmental and contextual constraints. However, we dobelieve that examining the circumstances of this case andinterpreting them within the framework of the riskliterature reviewed earlier can provide insight into thepractical realities of successful risk management andproject oversight of IT projects, and into issues related tosuccessful transfer of research knowledge into practice.Such insight can provide a foundation for further studies toelaborate on the challenges of utilizing research knowledgein practice, and can also provide practical suggestions forother organizations facing similar IT project risk manage-ment issues.

ConclusionIn this paper we have sought to address the gap between ITproject risk management research and practice by demon-strating how one organization has embedded researchknowledge in a practical and effective application. Insteadof attempting the traditional probability-impact method ofassessing project risks on a risk-by-risk basis, the PMO atCityOrg has taken a contingent and holistic approach,categorizing projects on their level of overall inherent riskby evaluating key dimensions known to be associated withproject success. Drawing on a foundation of researchknowledge and extensive project management experience,the PMO has developed a risk assessment process and

IT risk management: research and practice H Taylor et al

28

associated risk spider chart that have proven to be effectivetools in practice for surfacing inherent risk at the earlystages of IT projects and enabling appropriate managementstrategies to be recommended. The project risk assessmentprocess is a model for other organizations striving toengage in effective practices in order to improve projectoutcomes.

In summary, the case study reported here provides anillustration of how research can be effectively utilized inpractice. We believe the successful application of researchin this instance had three key features:

� Active participation of research-oriented practitionerswho have a detailed understanding of the constraints andambiguities of the practice context.

� Synthesis of an overwhelming checklist of risk factors,which are difficult to assess accurately at the start of aproject, into a manageable set of project dimensions thatcan be measured at the start.

� Presentation of the information in a format that allowseasy visualization of the interaction of individual details,displays information holistically, and supports discus-sion among multiple stakeholders.

It is essential for researchers interested in improvingpractitioners’ uptake of research findings to consider thepractical constraints of the context of application. Inparticular, it is important to recognize that researchfindings, often developed with 20–20 hindsight fromretrospective examination of facts, can only be effectivelyutilized in the workplace if they are transformed toincorporate the ambiguities, uncertainties, and incompleteinformation typically faced by practitioners attempting tomanage future scenarios.

AcknowledgementsWe would like to thank the City of Seattle, Department ofInformation Technology Project Management Center of Excellencefor their help and support in the development of this paper.

References

Addison, T. and Vallabh, S. (2002). Controlling Software Project Risks – An

Empirical Study of Methods Used by Experienced Project Managers, in

Proceedings of the Annual Conference of the South African Institute of

Computer Scientists and Information Technologists (SAICSIT) (Port

Elizabeth, South Africa, 16–18 September).

Alter, S. and Ginzberg, M. (1978). Managing Uncertainty in MIS

Implementation, Sloan Management Review 20(1): 23–31.

Association for Project Management. (2006). APM Body of Knowledge, 5th edn,

London: Association for Project Management.

Bannerman, P.L. (2008). Risk and Risk Management in Software Projects: A

reassessment, Journal of Systems and Software 81(12): 2118–2133.

Barki, H., Rivard, S. and Talbot, J. (1993). Toward an Assessment of Software

Development Risk, Journal of Management Information Systems 10(2):

203–225.

Barki, H., Rivard, S. and Talbot, J. (2001). An Integrative Contingency Model

of Software Project Risk Management, Journal of Management Information

Systems 17(4): 37–69.

Benbasat, I. and Zmud, R.W. (1999). Empirical Research in Information

Systems: The practice of relevance, MIS Quarterly 23(1): 3–16.

Boehm, B.W. (1973). Software and its Impact: A quantitative assessment,

Datamation 19(5): 48–59.

Boehm, B.W. (1983). Seven Basic Principles of Software Engineering, Journal

of Systems and Software 3(1): 3–24.

Boehm, B.W. (1991). Software Risk Management: Principles and practices, IEEE

Software 8(1): 32–41.

Boehm, B.W. and Turner, R. (2004). Balancing Agility and Discipline: A guide

for the perplexed, Boston: Addison-Wesley.

Brooks Jr., F.P. (1974). Mythical Man-Month, Datamation 20(12): 44–52.

Charette, R.N. (1996). The Mechanics of Managing IT Risk, Journal of

Information Technology 11(4): 373–378.

Creswell, J.W. (2008). Educational Research: Planning, conducting and

evaluating quantitative and qualitative research, 3rd edn, Upper Saddle

River, NJ: Pearson Merrill Prentice Hall.

de Bakker, K., Boonstra, A. and Wortmann, H. (2010). Does Risk Management

Contribute to IT Project Success? A Meta-Analysis of Empirical Evidence,

International Journal of Project Management 28(5): 493–503.

Desforges, C. (2000). Putting Educational Research to Use Through KnowledgeTransformation, Keynote lecture presented at the Further Education

Research Conference (Coventry, England, 12 December).

Fairley, R. (1994). Risk Management for Software Projects, IEEE Software 11(3):

57–67.

Heemstra, F.J. and Kusters, R.J. (1996). Dealing with Risk: A practical

approach, Journal of Information Technology 11(4): 333–346.

Herbsleb, J., Zubrow, D., Goldenson, D., Hayes, W. and Paulk, M. (1997).

Software Quality and the Capability Maturity Model, Communications of

the ACM 40(6): 30–40.

Howell, D., Windahl, C. and Seidel, R. (2010). A Project Contingency

Framework Based on Uncertainty and its Consequences, International

Journal of Project Management 28(3): 256–264.

Jiang, J.J., Klein, G. and Discenza, R. (2002). Pre-Project Partnering Impact

on an Information System Project, Project Team and Project Manager,

European Journal of Information Systems 11(2): 86–97.

Jiang, J.J., Klein, G., Hwang, H.-G., Huang, J. and Hung, S.Y. (2004). An

Exploration of the Relationship Between Software Development Process

Maturity and Project Performance, Information & Management 41(3):

29–288.

Keil, M., Cule, P., Lyytinen, K. and Schmidt, R. (1998). A Framework for

Identifying Software Project Risks, Communications of the ACM 41(11):

76–83.

Kutsch, E. and Hall, M. (2005). Intervening Conditions on the Management

of Project Risk: Dealing with uncertainty in information technology projects,

International Journal of Project Management 23(8): 591–599.

Levina, N. and Vaast, E. (2005). The Emergence of Boundary Spanning

Competence in Practice: Implications for implementation and use of

information systems, MIS Quarterly 29(2): 335–363.

March, J.G. and Shapira, Z. (1987). Managerial Perspectives on Risk and Risk

Taking, Management Science 33(11): 1404–1418.

Marcus, M.L. (1997). The Qualitative Difference in Information Systems Research

and Practice, in A. Lee, J. Liebenau and J.I. DeGross (eds.) Information Systems

and Qualitative Research, London: Chapman & Hall, pp. 11–27.

Markides, C. (2011). Crossing the Chasm: How to convert relevant research

into managerially useful research, Journal of Applied Behavioral Science

47(1): 121–134.

Martin, N.L., Pearson, J.M. and Furumo, K. (2007). IS Project Management:

Size, practices and the project management office, Journal of Computer

Information Systems 47(4): 52–60.

Mathiassen, L. (2002). Collaborative Practice Research, Information Technology

& People 15(4): 321–345.

McFarlan, F.W. (1981). Portfolio Approach to Information Systems, Harvard

Business Review 59(5): 142–150.

Miles, B.M. and Huberman, A.M. (1994). Qualitative Data Analysis: An

expanded sourcebook, 2nd edn, London: Sage.

Moynihan, T. (1997). How Experienced Project Managers Assess Risk, IEEE

Software 14(3): 35–41.

Nutley, S., Walter, I. and Davies, H.T.O. (2003). From Knowing to Doing: A

framework for understanding the evidence-into-practice agenda, Evaluation

9(2): 125–148.

Pablo, A.L. (1999). Managerial Risk Interpretations: Does industry make a

difference? Journal of Managerial Psychology 14(2): 92–107.

Patton, M.Q. (2002). Qualitative Research & Evaluation Methods, 3rd edn,

Thousand Oaks, CA: Sage.

Pender, S. (2001). Managing Incomplete Knowledge: Why risk management

is not sufficient, International Journal of Project Management 19(2):

79–87.

IT risk management: research and practice H Taylor et al

29

Pennington, R. and Tuttle, B. (2007). The Effects of Information Overload on

Software Project Risk Assessment, Decision Sciences 38(3): 489–526.

Pfleeger, S.L. (2000). Risky Business: What we have yet to learn about risk

management, Journal of Systems and Software 53(3): 265–273.

Pich, M.T., Loch, C.H. and De Meyer, A. (2002). On Uncertainty, Ambiguity,

and Complexity in Project Management, Management Science 48(8):

1008–1023.

Pohlmann, T. (2003). How Companies Govern their IT Spending, Cambridge,

MA: Forrester Research.

Powell, P.L. and Klein, J.H. (1996). Risk Management for Information Systems

Development, Journal of Information Technology 11(4): 309–319.

Project Management Institute. (2004). A Guide to the Project Management

Body of Knowledge (PMBOK Guide), 3rd edn, Newton Square, PA: Project

Management Institute.

Rasche, A. and Behnam, M. (2009). As if it were Relevant: A systems theoretical

perspective on the relation between science and practice, Journal of

Management Inquiry 18(3): 243–255.

Raz, T., Shenhar, A. and Dvir, D. (2002). Risk Management, Project Success,

and Technological Uncertainty, R & D Management 32(2): 101–109.

Reynolds, P. and Yetton, P. (2007). Building Theory from Practice:

Opportunities in IS Project Management, in AMCIS 2007 Proceedings.

Paper 428, http://aisnet.org/amcis2007/428.

Sambamurthy, V. and Zmud, R.W. (1999). Arrangements for Information

Technology Governance: A theory of multiple contingencies, MIS Quarterly

23(2): 261–290.

Sauer, C., Gemino, A. and Reich, B.H. (2007). The Impact of Size and Volatility

on IT Project Performance, Communications of the ACM 50(11): 79–84.

Schmidt, R., Lyytinen, K., Keil, M. and Cule, P. (2001). Identifying Software

Project Risks: An international Delphi study, Journal of Management

Information Systems 17(4): 5–36.

Shenhar, A.J. (2001). One Size Does Not Fit All Projects: Exploring classical

contingency domains, Management Science 47(3): 394–414.

Shenhar, A.J., Dvir, D., Levy, O. and Maltz, A.C. (2001). Project Success: A

multidimensional strategic concept, Long Range Planning 34(6): 699–725.

Simister, S.J. (2004). Qualitative and Quantitative Risk Management,

in P.W.G. Morris and J.K. Pinto (eds.) The Wiley Guide to Managing Projects,

Hokoben: John Wiley & Sons, pp. 30–47.

Sommer, S.C. and Loch, C.H. (2004). Selectionism and Learning in Projects with

Complexity and Unforeseeable Uncertainty, Management Science 50(10):

1334–1347.

Stake, R.E. (2000). Case Studies, in N.K. Denzin and Y.S. Lincoln (eds.)

Handbook of Qualitative Research, Thousand Oaks, CA: Sage, pp. 435–454.

Standish Group. (2001). Extreme CHAOS, West Yarmouth, MA: Standish Group

International.

Standish Group. (2005). Chaos Rising, West Yarmouth, MA: Standish Group

International.

Straub, D.W. and Ang, S. (2011). Rigor and Relevance in IS Research: Redefining

the debate and a call for future research, MIS Quarterly 35(1): iii–xi.

Subramanian, G.H., Jiang, J.J. and Klein, G. (2007). Software Quality and IS

Project Performance Improvements from Software Development Process

Maturity and IS Implementation Strategies, Journal of Systems and Software

80(4): 616–627.

Sumner, M. (2000). Risk Factors in Enterprise-Wide/ERP Projects, Journal of

Information Technology 15(4): 317–327.

Susman, G.I. and Evered, R.D. (1978). An Assessment of the Scientific Merits of

Action Research, Administrative Science Quarterly 23(4): 582–603.

Sussman, S.W. and Guinan, P.J. (1999). Antidotes for High Complexity and

Ambiguity in Software Development, Information & Management 36(1):

23–35.

Taylor, H. (2005). Congruence Between Risk Management Theory and Practice

in Hong Kong Vendor-Driven IT Projects, International Journal of Project

Management 23(6): 437–444.

Taylor, H. (2006a). Critical Risks in Outsourced IT Projects: The intractable

and the unforeseen, Communications of the ACM 49(11): 74–79.

Taylor, H. (2006b). Risk Management and Problem Resolution Strategies for IT

Projects: Prescription and practice, Project Management Journal 37(5): 49–63.

Taylor, H. (2007). An Examination of Decision-Making in IS Projects from

Rational and Naturalistic Perspectives, in ICIS 2007 Proceedings. Paper 30,

http://aisle.aisnet.org/icis2007/30.

Tufte, E.R. (2001). The Visual Display of Quantitative Information, Cheshire,

CT: Graphics Press.

Van de Ven, A.H. (2007). Engaged Scholarship: A guide for organizationaland social research, Oxford, UK: Oxford University Press.

Voetsch, R.J., Cioffi, D.F. and Anbari, F.T. (2004). Project Risk Management

Practices and their Association with Reported Project Success, Paper

presented at the IRNOP VI Conference, 25–27 August, Turku,

Finland.

Wallace, L. and Keil, M. (2004). Software Project Risks and their Effect on

Outcomes, Communications of the ACM 47(4): 68–73.

Wallace, L., Keil, M. and Rai, A. (2004). How Software Project Risk Affects

Project Performance: An investigation of the dimensions of risk and an

exploratory model, Decision Sciences 35(2): 289–321.

Walsham, G. (2006). Doing Interpretive Research, European Journal of

Information Systems 15(3): 320–330.

Ward, S. and Chapman, C. (2003). Transforming Project Risk Management

into Project Uncertainty Management, International Journal of Project

Mana-

gement 21(2): 97–105.

Wolcott, H.F. (1994). Transforming Qualitative Data: Description, analysis,

and interpretation, Thousand Oaks, CA: Sage.

Worren, N., Moore, K. and Elliott, R. (2002). When Theories Become Tools:

Toward a framework for pragmatic validity, Human Relations 55(10):

1227–1250.

Wysocki, R.K. (2001). Building Effective Project Teams, New York:

John Wiley & Sons.

Wysocki, R.K., Beck Jr., R. and Crane, D.B. (2000). Effective Project

Management, 2nd edn, New York: John Wiley & Sons.

Yin, R.K. (2009). Case Study Research: Design and methods, 4th edn,

Thousand Oaks, CA: Sage.

Zmud, R.W. (1980). Management of Large Software Development Efforts,

MIS Quarterly 4(2): 45–55.

Zmud, R.W. (1998). Conducting and Publishing Practice-Driven Research,

Paper presented at the IFIP Working Groups 8.2 and 8.6 Joint Working

Conference on Information Systems: Current issues and future changes,

10–13 December, Helsinki, Finland.

About the authorsHazel Taylor is an Assistant Professor at the InformationSchool, University of Washington, Seattle. She holds aPh.D. from Queensland University of Technology, Brisbane,Australia, and before joining the Information School, shetaught at the University of Waikato in New Zealand, and atthe Hong Kong University of Science and Technology. Herteaching and research focuses on IT project managementand risk management with an emphasis on tacit knowledgeand decision-making in these areas. Before her academiccareer, she worked in industry with manufacturing,construction, and government organizations, both as asystems manager and an IT project manager.

Edward Artman leads the City of Seattle’s InformationTechnology Project Management Center of Excellence, whichoversees large complex IS projects across the City on behalfof the CTO. He is a certified Project Management Profes-sional (PMP) and certified Scrum Master with over 20 years’experience in IT project management. He conducts indepen-dent project assessments and consults with project managersto improve utilization of project management practicesthat lead to sustainable project performance with successfuloutcomes. As a passionate practitioner of project managementbest practices, he has successfully managed a wide varietyof business and technology projects, and has extensiveexperience in recovering troubled IT projects. His industryexperience includes technology, distribution, retail, trans-portation, insurance, real estate, utilities, hospitality, andgovernment.

IT risk management: research and practice H Taylor et al

30

Jill Palzkill Woelfer is a Ph.D. student in InformationScience at the Information School. Jill has extensiveprofessional experience working in IT-related functionsin the medical products manufacturing sector. Since2008, she has worked as a research assistant for theInstitute for Innovation in Information Management onprojects focused on the learning and behavioral compe-

tencies of IT project managers, and on a projectregarding critical success factors for geographicallydispersed technology teams. Jill is an alumna of theExecutive Master of Science in Information Managementprogram, and also pursues research in the role ofinformation technologies in life skills development ofhomeless young people.

IT risk management: research and practice H Taylor et al

31

Tab

leA

1R

epre

senta

tive

sele

ction

of

pro

jects

evalu

ate

dw

ith

the

risk

spid

er

chart

pro

cess,

and

assessm

ents

,re

com

mendations,

and

actions

Pro

ject

Pro

ject

des

crip

tion

Ass

essm

ent

sum

ma

ryR

ecom

men

da

tion

sA

ctio

ns

Res

ult

s

AC

entr

alIT

imp

lem

enta

tio

no

fa

soft

war

e

too

lo

nal

lcl

ien

tw

ork

stat

ion

sac

ross

all

dep

artm

ents

,af

fect

ing

mo

reth

an10

,000

use

rs

Sign

ific

ant

risk

of

goin

go

ver

sch

edu

le

and

bu

dge

to

win

gto

hig

hsp

ano

fim

pac

t,

new

tech

no

logy

,an

db

ein

gle

db

ya

rela

tive

lyin

exp

erie

nce

dp

roje

ctm

anag

er

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

(i)

Get

dep

artm

enta

lb

uy-

inas

earl

yas

po

ssib

leto

min

imiz

eth

eri

sko

fgo

ing

ove

rsc

hed

ule

.

(ii)

Use

asi

gned

char

ter

agre

emen

tto

do

cum

ent

bu

y-in

.

(iii

)U

sea

lab

or-

trac

kin

gto

ol

totr

ack

inte

rnal

lab

or

tom

easu

reco

sts

and

pro

gres

sag

ain

stp

lan

ned

wo

rk

Bu

y-in

was

do

cum

ente

dvi

aa

sign

ed

char

ter,

bu

to

nly

afte

rle

ngt

hy

neg

oti

atio

ns.

Lab

or

trac

kin

gto

ol

was

imp

lem

ente

d

Sch

edu

led

ura

tio

nw

as26

0%o

ver

its

init

ial

esti

mat

e,b

ecau

seo

fti

me

tak

ento

get

agre

emen

to

nth

ep

roje

ctsc

hed

ule

fro

mm

ult

iple

dep

artm

ents

BN

ewb

usi

nes

sap

pli

cati

on

imp

lem

enta

tio

nin

asi

ngl

ed

epar

tmen

t

Sign

ific

ant

risk

of

losi

ng

focu

so

win

gto

very

lon

gd

ura

tio

nco

mb

ined

wit

hh

igh

bu

sin

ess

and

tech

nic

alu

nce

rtai

nty

and

com

ple

xity

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

(i)

Spli

tth

ep

roje

ctin

tom

ult

iple

smal

ler

pro

ject

sin

ord

erto

man

age

du

rati

on

and

allo

wfo

rp

ilo

tte

sto

fb

usi

nes

san

d

tech

no

logy

chan

ges.

(ii)

Sup

ple

men

tth

ete

amw

ith

am

ento

r

wit

hex

per

ien

cein

the

man

agem

ent

of

IT

pro

ject

s

Th

ep

roje

ctw

asb

rok

enin

toa

seri

eso

f

smal

ler

pro

ject

sst

arti

ng

off

wit

ha

pil

ot

effo

rtin

alo

w-r

isk

bu

sin

ess

area

.

Am

ore

exp

erie

nce

dp

roje

ctm

anag

erw

as

assi

gned

asa

men

tor

tow

ork

dir

ectl

y

wit

hth

ep

roje

ctm

anag

ero

nsp

ecif

ic

del

iver

able

s

Co

mp

lete

do

nti

me

and

on

bu

dge

t,

del

iver

ing

all

req

uir

edsc

op

e.

Wo

nan

ind

ust

ryaw

ard

for

exce

llen

cein

bu

sin

ess

per

form

ance

/str

ateg

icp

lan

nin

g

and

the

org

aniz

atio

n’s

ann

ual

awar

dfo

r

Pro

ject

Man

agem

ent

Exc

elle

nce

CR

epla

ceal

lm

anu

alw

ork

ord

er

man

agem

ent

pro

cess

esw

ith

auto

mat

ed

ente

rpri

se-l

evel

Co

mm

erci

al-O

ff-T

he-

Shel

f(C

OT

S)ap

pli

cati

on

usi

ng

asi

ngl

e

shar

edd

atab

ase

and

wo

rkfl

ow

app

lica

tio

nsy

stem

Seve

ral

hig

her

risk

attr

ibu

tes

that

cou

ld

com

pro

mis

eth

eo

utc

om

eo

fth

eef

fort

.

Ho

wev

er,

the

visi

bil

ity

of

the

pro

ject

was

larg

ely

con

tain

edto

the

dep

artm

ent.

Sco

pe

cou

ldb

ere

du

ced

ifn

eces

sary

to

off

set

un

exp

ecte

dco

stin

crea

ses.

Th

e

dep

artm

ent

did

no

th

ave

exp

erie

nce

wit

h

larg

eco

mp

lex

ITp

roje

cts

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

(i)

Co

nd

uct

ari

skid

enti

fica

tio

n

wo

rksh

op

toh

elp

exp

ose

wh

atth

ete

am

and

stak

eho

lder

sd

idn

ot

kn

ow

and

hel

p

alig

nst

akeh

old

ers

on

pit

fall

san

d

stra

tegi

esto

red

uce

risk

Rec

om

men

dat

ion

ado

pte

d.

Dep

artm

ent

hir

eda

kn

ow

led

geab

le

pro

ject

man

ager

togu

ide

them

Stil

lin

pro

gres

san

dp

osi

tio

ned

for

com

ple

tio

nah

ead

of

sch

edu

lean

du

nd

er

bu

dge

to

vera

ll

DC

han

gem

anu

alp

roce

sses

for

rece

ivin

g,

revi

ewin

g,an

dap

pro

vin

gd

ocu

men

tsan

d

pla

ns

too

nli

ne

sub

mis

sio

n,

revi

ewan

d

app

rova

l

Th

ep

roje

cth

adh

igh

leve

lso

fin

tern

al

and

exte

rnal

visi

bil

ity

and

an

um

ber

of

un

cert

ain

ties

that

wer

eli

kel

yto

lead

to

sch

edu

leex

pan

sio

n,

incr

ease

dco

st,

and

the

po

ten

tial

for

issu

esw

ith

cult

ura

l

chan

ges.

Th

ep

roje

ctsp

on

sor

and

pro

ject

man

ager

had

exp

erie

nce

on

pro

ject

so

f

sim

ilar

size

,sc

op

e,an

dco

mp

lexi

ty

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

Ow

ing

to

the

pro

ject

spo

nso

ran

dp

roje

ct

man

ager

’sex

per

ien

cean

dth

e

Dep

artm

ent’

ssu

cces

sw

ith

sim

ilar

pro

ject

s,n

osp

ecif

icre

com

men

dat

ion

s

wer

em

ade

Th

ep

roje

ctd

eliv

ered

all

sco

pe

and

succ

essf

ull

yac

hie

ved

the

pro

ject

ob

ject

ives

.T

he

pro

ject

was

del

iver

edo

n

bu

dge

t,b

ut

was

15w

eek

s(�

30%

)la

ter

than

pla

nn

ed,

du

eto

ak

eyre

sou

rce

bei

ng

div

erte

dto

ah

igh

erp

rio

rity

pro

ject

.A

sex

pec

ted

,so

me

of

the

in-

ho

use

dev

elo

pm

ent

was

mo

reco

mp

lex

and

too

klo

nge

rto

dev

elo

pth

an

ori

gin

ally

esti

mat

ed

App

endi

xIT risk management: research and practice H Taylor et al

32

EA

dd

enh

ance

men

tsan

dfu

nct

ion

alit

yto

a

pre

vio

usl

yim

ple

men

ted

Soft

war

eas

a

Serv

ice

(Saa

S)ap

pli

cati

on

that

was

dep

loye

dto

fulf

ill

ale

gal

man

dat

e

Th

isp

roje

ctex

hib

ited

am

od

erat

eri

sk

pro

file

wit

hh

igh

exte

rnal

and

inte

rnal

visi

bil

ity,

bro

adsp

ano

fim

pac

t,so

me

new

tech

no

logy

,b

usi

nes

sp

roce

ss

reen

gin

eeri

ng,

mo

der

ate

cost

and

du

rati

on

.So

me

asp

ects

wer

ecr

itic

alin

ord

erto

mee

tth

ele

gal

man

dat

e.U

ser

acce

pta

nce

of

the

pre

vio

us

rele

ase

of

the

app

lica

tio

nw

asch

alle

nge

d.

Som

eh

igh

-

risk

fact

ors

wer

eo

ffse

tb

yth

esp

on

sor’

s

and

the

team

’sp

rio

rex

per

ien

ce

imp

lem

enti

ng

the

init

ial

pro

ject

and

the

app

lica

tio

no

fle

sso

ns

lear

ned

fro

mth

at

pro

ject

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

(i)

Res

tru

ctu

reth

ep

roje

ctin

tosm

alle

r

chu

nk

sto

add

ress

exis

tin

gu

ser

issu

es

firs

t.

(ii)

Def

erd

eplo

ymen

to

fo

pti

on

al

fun

ctio

nal

ity

pen

din

gim

pro

ved

use

r

acce

pta

nce

of

the

alre

ady

dep

loye

d

app

lica

tio

n.

(iii

)D

evel

op

dis

tin

ctb

ud

gets

and

sch

edu

les

for

all

pro

ject

wo

rk.

(iv)

Cla

rify

role

san

dre

spo

nsi

bil

itie

sto

incr

ease

un

der

stan

din

gab

ou

to

wn

ersh

ip

and

acco

un

tab

ilit

y

Th

ere

com

men

dat

ion

sw

ere

no

tad

op

ted

du

eto

ala

cko

fb

uy-

infr

om

the

Exe

cuti

veSt

eeri

ng

Co

mm

itte

e.

Pro

ject

wo

rkan

dd

aily

op

erat

ion

alw

ork

and

bu

dge

tsb

ecam

ein

term

ingl

edso

it

bec

ame

dif

ficu

ltto

man

age

the

effo

rtas

a

pro

ject

.

Th

ese

pro

ble

ms

bec

ame

app

aren

tin

the

mo

nth

lyd

ash

bo

ard

rep

ort

ing

pro

cess

and

led

toth

ep

roje

ctb

ein

gra

ted

un

hea

lth

yan

dat

-ris

kb

yth

eP

MO

Th

ere

com

men

dat

ion

tore

stru

ctu

reth

e

pro

ject

was

ado

pte

daf

ter

the

Exe

cuti

ve

Co

mm

itte

efa

iled

toco

me

toag

reem

ent

on

the

sco

pe

and

det

aile

dro

les

and

resp

on

sib

ilit

ies

and

the

pro

ject

rece

ived

anat

-ris

kra

tin

gfr

om

the

PM

O.

Th

efi

rst

stag

eo

fth

ere

stru

ctu

red

pro

ject

has

pro

ceed

edsu

cces

sfu

lly

F (see

Fig

ure

2)

AC

ityO

rgd

epar

tmen

tw

ill

par

tner

wit

ha

Stat

eag

ency

top

rovi

de

aw

eb-b

ased

un

iver

sal

inte

rnet

po

rtal

that

mak

esit

easi

erfo

rin

div

idu

als

toap

ply

for

and

acce

ssa

vari

ety

of

vita

lse

rvic

esan

d

ben

efit

s

Th

ep

roje

ctex

hib

ited

ah

igh

-ris

kp

rofi

le.

Mu

cho

fth

ete

chn

ical

and

pro

ject

man

agem

ent

risk

wo

uld

be

ow

ned

by

Stat

e.T

he

Cit

yp

ort

ion

of

the

pro

ject

req

uir

edco

nsi

der

able

coll

abo

rati

on

wit

h

inte

rnal

Cit

yo

rgan

izat

ion

and

the

Cit

y

had

very

lim

ited

inp

ut

and

con

tro

lo

fth

e

fin

also

luti

on

.T

he

role

s,re

spo

nsi

bil

itie

s,

acco

un

tab

ilit

ies,

and

com

mit

men

tsw

ith

the

par

tner

agen

cyw

ere

vagu

e.If

the

pro

ject

fail

edto

ach

ieve

gran

t

req

uir

emen

ts,

gran

tfu

nd

ing

cou

ldb

eat

risk

and

the

dep

artm

ent

wo

uld

be

req

uir

edto

cove

rex

pen

ses

Mo

nth

lyd

ash

bo

ard

rep

ort

ing

and

ind

epen

den

to

vers

igh

to

fp

artn

erag

ency

.

(i)

Ass

ign

ap

roje

ctm

anag

erw

ho

has

exp

erie

nce

wit

hp

roje

cts

of

sim

ilar

size

and

com

ple

xity

.

(ii)

Imp

lem

ent

aM

emo

ran

du

mo

f

Agr

eem

ent

tocl

arif

yro

les,

resp

on

sib

ilit

ies,

com

mit

men

ts,

auth

ori

ties

and

acco

un

tab

ilit

ies

bet

wee

n

all

par

tner

s.

(iii

)C

lari

fysc

op

ean

dre

-est

imat

eco

st,

du

rati

on

,an

dri

sks

tob

esu

reth

e

con

stra

ined

bu

dge

tw

ill

be

suff

icie

nt

to

com

ple

teth

ep

roje

ct

All

reco

mm

end

atio

ns

ado

pte

dex

cep

t

ind

epen

den

to

vers

igh

to

fp

artn

erag

ency

Th

ep

roje

cth

asb

een

pla

ced

on

ho

ldb

y

the

spo

nso

rsu

nti

lth

eC

ity’

sp

arti

cip

atio

n

inth

ep

roje

ctis

mo

recl

earl

yd

efin

edan

d

they

are

con

fid

ent

the

par

tner

agen

cyca

n

fulf

ill

its

com

mit

men

ts

GR

epla

ceth

eC

ity’

sm

essa

gin

gan

d

cale

nd

arap

pli

cati

on

wit

ha

dif

fere

nt

tech

no

logy

.T

he

pro

ject

was

init

iate

do

ut

of

anea

rlie

rp

roje

ctth

atd

evel

op

edth

e

pro

ject

and

imp

lem

enta

tio

np

lan

s,

arch

itec

ture

des

ign

,an

dco

stes

tim

ate.

Th

ep

roje

ctre

pre

sen

tsa

sign

ific

ant

tech

nic

alsh

ift

of

ah

igh

lyvi

sib

le,m

issi

on

crit

ical

app

lica

tio

nu

sed

on

ad

aily

bas

is

by

Cit

yst

aff

Th

ep

roje

ctsh

ow

edse

vera

lh

igh

-ris

k

fact

ors

incr

itic

alar

eas

du

ela

rgel

yto

its

size

and

span

of

imp

act

and

ther

efo

re

pre

sen

ted

ah

igh

-ris

kp

rofi

le.

Alt

ho

ugh

a

pla

nth

atca

lled

for

stag

edd

eplo

ymen

to

f

the

app

lica

tio

nh

elp

edre

du

ceth

eri

sk,

stro

ng

pro

ject

man

agem

ent

dis

cip

lin

es

wo

uld

be

req

uir

edto

imp

rove

the

lik

elih

oo

do

fsu

cces

s.

Mo

nth

lyd

ash

bo

ard

rep

ort

ing

and

ind

epen

den

tp

roje

cto

vers

igh

tb

yan

ou

tsid

eco

nsu

ltan

t

Th

ep

roje

ctu

sed

hig

hly

exp

erie

nce

d

con

sult

ants

tod

evel

op

the

imp

lem

enta

tio

np

lan

Th

ep

roje

ctco

mp

lete

do

nti

me

and

28%

un

der

bu

dge

t,an

dd

eliv

ered

all

iden

tifi

ed

sco

pe

HD

epar

tmen

tal

mis

sio

ncr

itic

alp

roje

ctto

rep

lace

two

MS

Acc

ess-

bas

ed

app

lica

tio

ns

wit

ha

Co

mm

erci

alO

ff-t

he-

Shel

f(C

OT

S)w

eb-b

ased

soft

war

e

app

lica

tio

nto

imp

rove

the

dep

artm

ent’

s

abil

ity

toev

alu

ate

dat

aan

dtr

ack

bu

sin

ess

nee

ds.

Ori

gin

alp

rop

osa

lw

asto

imp

lem

ent

ap

re-r

elea

seve

rsio

no

fth

e

ven

do

rso

ftw

are

pac

kag

e,re

qu

irin

g

con

sid

erab

leco

nfi

gura

tio

n

Th

ep

roje

ctp

rese

nte

da

mo

der

ate

risk

pro

file

ow

ing

toit

sin

tern

alan

dex

tern

al

visi

bil

ity,

the

pro

po

sed

use

of

pre

-rel

ease

soft

war

eth

atw

ou

ldre

qu

ire

con

sid

erab

le

con

figu

rati

on

by

staf

fw

ho

hav

eli

ttle

exp

erie

nce

wit

hth

atty

pe

of

tech

no

logy

,

and

the

lon

gd

ura

tio

nan

dth

ep

roje

ct

man

ager

and

spo

nso

rsla

cko

fex

per

ien

ce

wit

hp

roje

cts

of

this

size

and

com

ple

xity

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

(i)

Co

nsi

der

solu

tio

ns

that

wer

eal

read

y

pro

ven

inth

ein

du

stry

.

(ii)

Ad

dan

exp

erie

nce

dad

viso

rto

the

team

togu

ide

them

thro

ugh

crea

tin

ga

Stat

emen

to

fW

ork

for

the

ven

do

ran

d

the

pla

nn

ing

stag

eo

fth

ep

roje

ct

Bo

thre

com

men

dat

ion

sw

ere

ado

pte

dT

he

pro

ject

isst

ill

inth

eP

lan

nin

gP

has

e.

Wit

hp

lan

nin

gan

dp

rocu

rem

ent

acti

viti

esu

nd

erw

ay,

the

pro

ject

curr

entl

y

exh

ibit

sa

hea

lth

yp

rofi

le

IT risk management: research and practice H Taylor et al

33

Tab

leA

1C

ontinued

Pro

ject

Pro

ject

des

crip

tion

Ass

essm

ent

sum

ma

ryR

ecom

men

da

tion

sA

ctio

ns

Res

ult

s

IC

ust

om

bu

ild

anem

plo

yee

self

-ser

vice

po

rtal

and

up

grad

ea

maj

or

Co

mm

erci

al-

Off

-Th

e-Sh

elf

(CO

TS)

app

lica

tio

nto

a

sup

po

rted

vers

ion

of

the

sam

eve

nd

or’

s

soft

war

e.T

he

app

lica

tio

nis

use

db

yal

l

Cit

yem

plo

yees

and

dep

artm

ents

Th

ep

roje

ctex

hib

ited

am

ediu

m-t

o-l

ow

ove

rall

risk

pro

file

.W

hil

eth

isp

roje

ct

was

hig

hly

visi

ble

acro

ssth

eC

ity,

it

mai

nta

ined

alo

wp

rofi

leex

tern

alto

the

Cit

y.A

gove

rnan

cest

ruct

ure

was

inp

lace

wit

hth

ree

key

stak

eho

lder

sas

Stee

rin

g

Co

mm

itte

em

emb

ers.

Th

ecu

sto

m

dev

elo

pm

ent

com

po

nen

tp

ose

dth

em

ost

sign

ific

ant

tech

nic

alri

skb

ut

wo

uld

re-

use

som

em

od

ule

sfr

om

ano

ther

app

lica

tio

nto

min

imiz

eri

sks.

Th

e

pro

ject

team

was

smal

lan

dw

asfa

mil

iar

wit

hth

ete

chn

olo

gyan

dea

rlie

ru

pgr

ade

app

roac

h.

Th

ep

roje

ctm

anag

eran

dte

am

had

succ

essf

ull

yp

erfo

rmed

earl

ier

up

grad

es.T

his

new

vers

ion

of

the

ven

do

r

soft

war

eh

adb

een

pro

ven

inth

em

ark

et

for

abo

ut

1ye

ar.

Ver

yli

ttle

cust

om

izat

ion

of

the

soft

war

ew

as

req

uir

ed

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

(i)

Ass

ign

on

eo

fth

eth

ree

key

spo

nso

rs

asth

e‘f

inal

auth

ori

ty.’

(ii)

Imp

lem

ent

sim

ple

pra

ctic

esfo

r

Pro

ject

Po

rtfo

lio

Man

agem

ent

and

Res

ou

rce

Man

agem

ent

toh

elp

red

uce

exp

osu

reto

on

goin

gis

sues

wit

hfr

equ

ent

un

sch

edu

led

wo

rk,

and

seri

ou

sre

sou

rce

con

ten

tio

n.

(iii

)D

evel

op

det

aile

dw

ork

pla

ns

that

pro

vid

eda

min

imu

mo

fa

90-d

aylo

ok

-

ahea

d,

and

mil

esto

ne

leve

lp

lan

sfo

rth

e

pro

ject

and

reso

urc

elo

adth

ese

wo

rk

pla

ns

Rec

om

men

dat

ion

s(i

)an

d(i

i)w

ere

no

t

ado

pte

d.

Th

e90

-day

loo

k-a

hea

dw

asad

op

ted

bu

t

on

lysp

ora

dic

ally

imp

lem

ente

d.

Use

of

reso

urc

ep

lan

nin

gw

asn

ot

ado

pte

d

Th

isp

roje

ctis

nea

rin

gco

mp

leti

on

bu

tis

38%

beh

ind

the

ori

gin

alsc

hed

ule

and

8%

beh

ind

are

-bas

elin

edsc

hed

ule

.T

he

dep

artm

ent

do

esn

ot

pra

ctic

eP

roje

ct

Po

rtfo

lio

Man

agem

ent

and

issu

bje

ctto

freq

uen

tan

du

nex

pec

ted

new

hig

h

pri

ori

typ

roje

cts

that

del

ayw

ork

-in

-

pro

gres

s.T

he

pro

ject

man

ager

sas

sign

ed

atth

eti

me

of

the

Ris

kP

rofi

leR

evie

wd

id

no

tco

me

on

toth

ep

roje

ctas

pla

nn

ed.

Th

ep

roje

ctw

asth

enas

sign

edto

an

inte

rnal

pro

ject

man

ager

JR

epla

cean

exis

tin

gte

chn

olo

gyp

latf

orm

bec

ause

the

old

pla

tfo

rmw

asn

olo

nge

r

sup

po

rted

by

the

ven

do

r,m

igra

tin

ga

po

rtio

no

fex

isti

ng

app

lica

tio

n

fun

ctio

nal

ity

fro

mth

eo

ldp

latf

orm

toth

e

new

erte

chn

olo

gysy

stem

Th

ep

roje

ctp

rese

nte

da

mo

der

ate

risk

pro

file

ow

ing

toth

ecr

itic

alit

yo

fth

e

fun

ctio

nal

ity

del

iver

ed,

the

lon

g

du

rati

on

,ci

tyw

ide

visi

bil

ity,

and

mu

lti-

dep

artm

ent

nat

ure

Mo

nth

lyd

ash

bo

ard

rep

ort

ing.

(i)

Ow

ing

toth

em

ult

i-d

epar

tmen

tn

atu

re

of

the

pro

ject

,fo

rma

stee

rin

gco

mm

itte

e

of

rep

rese

nta

tive

sfr

om

affe

cted

dep

artm

ents

top

rovi

de

inp

ut

toth

e

stra

tegi

cd

irec

tio

no

fth

ep

roje

ct

Rec

om

men

dat

ion

ado

pte

dT

he

pro

ject

isco

mp

lete

wit

hve

rygo

od

per

form

ance

on

sco

pe,

bu

dge

t,an

d

sch

edu

le.

All

ob

ject

ives

wer

eac

hie

ved

.

Aft

erad

just

men

tfo

rch

ange

ord

ers,

sco

pe

was

del

iver

edfo

r4.

6%le

ssth

an

pla

nn

edco

stan

dth

e16

-mo

nth

pro

ject

was

del

iver

ed4.

6%b

eyo

nd

the

pla

nn

ed

com

ple

tio

nd

ate

IT risk management: research and practice H Taylor et al

34