Using the Troubled Project Recovery Framework: Problem Recognition … · 2014-10-08 · success for these projects, we propose a staged framework for recovery and rehabilitation

Using the Troubled Project Recovery Framework: Problem Recognition and Decision toRecoverAuthor(s): Douglas Havelka and T. M. RajkumarReviewed work(s):Source: e-Service Journal, Vol. 5, No. 1 (Fall 2006), pp. 43-73Published by: Indiana University PressStable URL: http://www.jstor.org/stable/10.2979/ESJ.2006.5.1.43 .Accessed: 18/03/2012 21:45

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

Indiana University Press is collaborating with JSTOR to digitize, preserve and extend access to e-ServiceJournal.

http://www.jstor.org

http://www.jstor.org/action/showPublisher?publisherCode=iupress

http://www.jstor.org/stable/10.2979/ESJ.2006.5.1.43?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp

43@ 2006 e-Service Journal. All rights reserved. No copies of this work may be distributed in print or electronically without express written permission from Indiana University Press.

Using the Troubled Project Recovery Framework:

Problem Recognition and Decision to Recover�

Douglas Havelka

Miami University

T.M. Rajkumar

Miami University

ABSTRACT

Information system development projects continue to be delivered significantly behind sched-

ule, drastically over budget and without meeting specifications. To improve the likelihood of

success for these projects, we propose a staged framework for recovery and rehabilitation.

This framework is composed of four stages that include twelve steps. The four stages are

recognition, immediate recovery, sustained recovery and maturity. We performed a field

study to validate the first stage of the framework, the Recognition Stage. The results of the

study indicate support for the steps of the Recognition Stage and provide detailed “symptoms”

that could indicate trouble during an information system development project. These

symptoms were logically categorized into eleven areas of interest to project managers: Client/

Stakeholder, Goal, Meeting, Team, Task, Project, Project Management, Communication,

Management, Project, and Process.

Keywords: Projects, Project Management, Troubled Projects, Information Systems Development

INTRODUCTION

More than $255 billion is spent every year in the US on information systems development

projects (Standish Group, 2004). However, most information system (IS) development

�. Earlier versions of this paper were presented at the 2004 Americas Conference on Information Systems

and the First International Workshop on IT Project Management, Milwaukee, Wisconsin, USA, 2006.

e-Service Journal

44

projects are not successful in at least one of the key measures: over budget, over schedule,

or do not meet specifications. Some research has shown that only 34% of software devel-

opment meets the three success criteria and as many as 66% of software development

projects are “troubled.” This leads to an annual waste of nearly 50 billion dollars in IS

development projects, $38 billion in lost value and $�7 billion in cost overruns (CHAOS

Chronicles, 2004). Referring to the Standish Group’s Chaos report of 2005, Balestreto,

the CEO of the Project Management Institute, stated that nearly 56 percent of projects

saw cost overruns of more than 50 percent and 84 percent of projects had schedule over-

runs (Cost overrun? Call in a project manager, 2006).

Nearly half of all projects were reported to be challenged, i.e., they were over budget,

behind schedule, and the end product did not meet user requirements. A definition for

projects that are even more deeply troubled has also been suggested, i.e., a runaway project

is one where the schedule, cost, or functionality is twice as bad as what was sought (Glass,

2002). Additionally, projects that are canceled prior to completion, i.e., impaired projects,

make up about �6% of all projects. For the purpose of this study and as suggested by prior

research, the following measures are used as indicative of trouble: (�) being over schedule

by 30%, (2) being over budget by more than 30%, and (3) having an end product that

does not meet user requirements (Whittaker, �999). Regardless of the label used for trou-

bled projects or the rule for classifying a troubled project, it is clear that schedule, cost, and

functionality goals are not adequately achieved by IS project managers.

Some of the confusion regarding troubled projects is attributable to the varying defi-

nitions of project success (Belassi and Tuckel, �996; Klein and Jiang, 200�; Linberg, �999).

Different stakeholders within the project may view success and failure differently (Pintro

and Slevin, �988). IS professionals have deemed systems as successful even when a system

is 4�7% over cost and �93 % behind schedule (Linberg, �999), because their notion of suc-

cess was based upon the knowledge gained by the innovative nature of the solution and the

relationships built within the team. Such views, while deviant from normal perceptions of

project management success, indicate how different groups view success criteria differ-

ently. Other factors include the fact that IS professionals might view success from a narrow,

information technology perspective and miss the overall business and organizational envi-

ronment within which the software solution is being deployed (Day, 2000).

In addition, it appears that businesses do not seem to learn from past mistakes in

software development. They repeat the same mistakes over and over again and are habit-

ually in trouble (Tarek and Madnick, �990). A reason for this may be the fact that a ma-

jority of software development improvement is done at the project management level

rather than at the organizational level (McConnell, 2002). In a study that measured

project management maturity across different industries, including engineering con-

struction, information management and movement (telecommunication), information

systems development, and high tech products and services, information systems compa-

Using the Troubled Project Recovery Framework

45

nies ranked lowest in maturity, achieving only a mean of 3.07 on a scale of � to 5 (Ibbs

and Kwak, 2000). The Software Engineering Institute has developed a model called the

Capability Maturity Model which measures the software development capabilities of

companies (Jalote, �999). Many U.S. companies rank at Level � (initial level) or 2 (re-

peatable level), which are the lowest levels. This is indicative of the lack of standard,

consistent, and predictable project management processes.

To be considered a “recovery,” a project headed toward disaster or cancellation must

at least be able to overcome the initial problem situation and achieve at least average re-

sults (Caper Jones, �995a, �995b). Recovery may not always be possible. Early discovery

of the problem should lead to improved likelihood of success or recovery. It is proposed

that if trouble in projects is not discovered or is discovered and left unattended, the proj-

ect will have a higher likelihood of failure. We contend that in the software industry

problems are seldom detected early enough. Based on this belief we attempt to address

the following research question:

How can organizations identify trouble in an IS development project as early as

possible?

To address this question, we conducted a field study using a nominal group process

to identify symptoms of trouble in IS development projects. The rationale for the field

study is based on the Troubled Project Recovery Framework (Aiyer, Rajkumar, and

Havelka, 2005).

Different perspectives of troubled projects and recovery processes exist. Several dis-

tinct areas of study can inform our efforts to understand the issues confronted by manag-

ers when dealing with the identification and correction of problems as early as possible.

Research and results from the areas of software project management, de-escalation of

project commitment, and crisis management exist. All of these areas contribute to our

understanding of how to improve troubled projects and are the foundations for the Trou-

bled Project Recovery Framework.

Software Project Management

It has been proposed that both long-term and short-term recovery methods are important

(Connell, 2000) for IS project management success. The Capability Maturity Model and

ISO 900� that help all projects across the organization are examples of long-term methods.

These approaches have been found to be effective, but may be inappropriate or difficult to

implement for an ongoing troubled project. For a specific project in trouble, short-term re-

covery methods are needed. Connell suggests the following steps: an initial assessment is

used to identify the areas of trouble in the project and a plan is then developed; the purpose

and final objectives and goals of the project should be reviewed and agreed upon by all

e-Service Journal

46

major stakeholders; modest, stable, and achievable goals for imminent milestones should

be established; realistic schedules are set; the project is tracked with source control; auto-

matic builds are in place; and efficient processes are specified to track and fix bugs. Con-

nell’s method seems to place an emphasis on the programming as opposed to the overall

life cycle of the project. The Troubled Project Recovery Framework adds detailed steps

that project managers or sponsors could follow to these overall quality assurance and man-

agement approaches. Additionally, the Troubled Project Recovery Framework addresses

management and personnel issues as well as the coding-intensive focus of these models.

Lastly, the results of our framework and this study provide managers using CMM or

ISO900� types of programs additional detailed areas for analysis and review of projects.

De-escalation of Commitment

The decision to add resources to a failing or troubled project is referred to as the escala-

tion of commitment. Through de-escalation, troubled projects may be successfully

turned around or sensibly abandoned (Keil and Robey, �999). Some research has looked

at recovery processes from the de-escalation perspective and this research identifies the

following steps as necessary: problem recognition, reexamination, selecting alternative

courses of action, and implementing an exit strategy (Keil and Montealgre, 2000). By

following these steps, organizations should do a better job of supporting projects that are

more likely to succeed and ending projects that have a strong chance of failing. Alternate

scenarios to kill projects have been presented (Kapur, 200�). Kapur’s approach includes:

check for vital signs of the troubled project; meet with decision makers to obtain key

stakeholder and senior management buy-in before a decision to kill or recover the project;

if the decision is to recover, then the manager must create both a recovery plan outlining

the steps needed to bring the vital signs back to acceptable levels and a mechanism to

track project recovery. If the decision is to kill the project, the aim is to develop a cancel-

lation plan, obtain stakeholder approval to the plan, kill the project while causing the

least amount of harm and salvaging usable project components, and conduct a post mor-

tem analysis including team members and stakeholders to glean lessons from the project

so that the mistakes are not repeated again. In general, the de-escalation literature focuses

on how to cancel a project when appropriate and cut losses to the organization. Recover-

ing and rehabilitating the project is only a secondary focus and is not as well developed.

The Troubled Project Recovery Framework (TPRF) overlaps with some of the first steps

in these approaches, but provides more detailed procedures for identifying trouble and

taking immediate steps to correct, if that is the decision made. The TPRF focuses less on

how the decision to recover is made and more on how to achieve recovery. The results of

this study provide managers with more detailed information to identify and manage

trouble on IS projects earlier than they previously might have.


47

Crisis Management

Another area of related research is crisis management, which is the process of assessment,

response, mitigation, and relief of crisis situations (Seeger and Ulmer, 200�). Crisis man-

agement is defined as the immediate response to such events, recovery efforts, and miti-

gation and preparedness efforts to reduce the impact of future crises (Summary of a

Workshop on Information Technology Research for Crisis Management, �998). Immediate

response is dedicated to the immediate protection of life and property. It requires urgent

action and the use of coordinated resources and facilities not available to routine prob-

lems. Recovery efforts encompass both short-term activity intended to return vital sup-

port systems of the project to operation and longer-term activities designed to return

infrastructure systems to pre-disaster conditions.

These activities are directly analogous to the recognition and recovery stages of the

Troubled Project Recovery Framework (Aiyer et al., 2005). In troubled software projects,

immediate response might involve recognizing the problem and communicating to the

stakeholders the nature and magnitude of the problem. Recovery efforts include assess-

ing the problems and taking quick corrective action to restore the health of the project,

such as changing team composition when one member of the team creates conflicts

within the team (Brown, Malveau, McCormick III, and Mowbray, �998). Recovery also

includes long-term plans such as reorganizing and implementing a new base plan.

Mitigation includes steps such as documenting the problems, preparing measures

for corrective actions, and preparing specific actions to address each problem. Prepared-

ness is the process of risk management: identifying, analyzing, and elaborating the symp-

toms and creating contingency plans to mitigate the effects of the risk. For IS projects,

documenting the problems identified and the success or failure of corrective actions can

provide lessons learned for these processes and can be used to proactively prepare risk

management procedures to be used in the current and future projects.

Some research has directly focused on crises in IS projects (Boundy and Diamond,

�998). Again, a set of steps is suggested for dealing with the crisis: recognizing and ac-

knowledging the crisis including discussion of the crisis and consideration of solutions to

the problem; identifying the magnitude of the problem and communicating it to the rel-

evant stakeholders; ensuring that the problem is being handled by the right people within

the team including escalating the problem to higher levels; and applying additional re-

sources (such as a consultant) if needed and appropriate; continuing work on other parts

of the project for which people can be spared while keeping the resolution of the crisis as

priority number one; and ensuring that management and stakeholders are updated daily

with regard to the crisis.

The prior work on crisis management, especially in the IS area, is directly relevant to

the recognition and recovery stages of the TPRF. The TPRF recognizes the fact that there

e-Service Journal

48

are times when trouble, or crises, in IS projects is not readily apparent and that it would be

useful to have early warning signs or indicators that trouble is occurring. Once the trouble

is recognized, prescriptions from the crisis management literature could be used to help

address and mitigate the trouble. Overall, the TPRF spans a greater set of activities and

provides a bridge between the more focused crisis management and project de-escalation

literature to the broader software management literature. As can be seen throughout the

literature, a logical, structured approach with specified activities and tasks is recommended

to help recover (or kill) troubled projects. The Troubled Project Recovery Framework pro-

vides just such an approach for use in IS project management (Aiyer et al., 2005).

THE TROUBLED PROJECT RECOVERY FRAMEWORK (TPRF)

The Troubled Project Recovery Framework is composed of four main stages: (�) problem

recognition, (2) immediate recovery, (3) sustained recovery, and (4) maturity. This

framework was built with prior research as a foundation and is similar to the crisis man-

agement approach which has four main stages: immediate response, recovery, mitigation

and preparedness; and the four stages of de-escalation: recognition, reexamination, se-

lecting course of action, and exit strategy (Keil and Montealegre, 2000). The conceptu-

alization of the four-stage �2-step recovery framework is shown in Figure �. The four

main stages are each composed of two to four steps. A short discussion of the four stages

and the twelve steps follows.

Stage 1: Problem Recognition and Decision to Recover

Before any project can be recovered, the fact that the project is in trouble must be recog-

nized. A decision to recover is made and the recovery process begins. The steps of the first

Figure 1. Troubled Project Recovery Framework


49

stage include: (�) problem awareness, (2) admission of trouble and seeking help, (3) quick

assessment and review, and (4) decision to recover.

Step � is problem awareness. When projects are in trouble, it is expected they would

display symptoms of the trouble, similar to a patient displaying symptoms of a disease.

Many times some type of trigger or other significant event occurs that demands a re-

sponse or action. This may be in the form of an event, (e.g., a subcontractor quitting the

project) or in the form of a state (red flag) that needs attention (e.g., a missed milestone or

deliverable) that indicates things are not operating as they should. In addition, many or-

ganizations do not use good estimation tools or techniques and lack any kind of histori-

cal measures or metrics to make comparisons during project management (Caper Jones,

�995b). Therefore, projects may get into serious trouble without knowledge of the proj-

ect manager or other stakeholders until the projects are very late in development. These

symptoms may be identifiable and monitored to help project managers recognize and

acknowledge trouble earlier in a project’s life and thereby increase the likelihood of suc-

cessful recovery.

Step 2 is admitting that trouble exists and getting help. According to a survey by

the Center For Project Management (Kapur, 200�), many project managers are afraid of

being labeled as quitters or failures and only 20% have a process for identifying troubled

projects. This suggests that project managers may resist or ignore symptoms until it is too

late for recovery. Denial is an unconscious coping mechanism to block out and avoid

major changes that may have some pain associated with them (Kiechel, �993). Project

managers recognize trouble, but deny they are in trouble (Brady and DeMarco, �994)

and keep silent without reporting the bad news (Smith, Keil, and Depledge, 200�). Proj-

ect managers tend to hold on to a problem too long and do not ask for help, thinking it

makes them look bad (Boundy and Diamond, �998). Keil and Robey (200�) also iden-

tify a deaf effect, i.e., even when the trouble is reported, the manager may fail to hear it.

By ignoring the trouble report, managers may insulate themselves from dealing with the

problem.

Step 3 is to conduct a quick and honest assessment and review of the project status.

This assessment must be a quick review of the project execution and its true status. The

aim is to create as complete and accurate an assessment of the project as possible in a

short time frame. A check for the vital signs of the project must be conducted (Kapur,

200�). This includes a check of planned versus actual achievement on the schedule, re-

sources used, milestones met, and deliverables met. Also a comparison of estimated ver-

sus actual cost of the project to date must be made.

Step 4 is the decision to recover the project or to kill it. Once an assessment is

made, a decision as to whether the project should be recovered or canceled must then be

made. Once the decision to recover is made in consultation with upper management and

stakeholders, an internal commitment to recover must be made. Not only must a deci-

e-Service Journal

50

sion to recover be made, but one must truly believe that project recovery is possible. Buy-

in to the commitment must come from both stakeholders and managers.

Stage 2: Immediate Recovery Stage

In this stage, steps are taken to nurture the troubled project back to stability. The aim in

immediate recovery is to identify and isolate the critical problems and take immediate

corrective or mitigating action. The aim is not to revamp the complete plan. The two

steps in this stage are triage and treatment. These steps may occur in sequence or concur-

rently and iteratively within the recovery process. In the triage step, similar to triage in

major accidents or natural disasters, the most critical problems are identified, isolated,

and prioritized. The major problem may be identified in Stage � and additional problems

or issues may have been identified in the assessment step of Stage �. For example, a trigger

may have been the loss of the project manager and the assessment revealed several missed

milestone deliverables and cost overruns. The issues should be prioritized so that the

treatments in the next step occur appropriately. Closely linked to the triage step is the

treatment step. As the critical problems are identified and prioritized, a treatment action

is determined for each critical problem. For example, the sponsor may decide to forgo an

imminent milestone until further analysis can be performed. The aim is to stabilize the

project and bring the project to a stage from which a full recovery can be made.

Stage 3: Sustained Recovery

The long-term or sustained recovery process can be put in motion once the project is in a

stable stage. The steps include analyzing the project status and creating an issue list with

resolutions for addressing the issues, creating a revised baseline plan and obtaining man-

agement approval, followed by executing and monitoring the revised plan. Building on

the prior steps, there should be an initial inventory of issues that need to be analyzed and

appropriate action planned. By incorporating resolutions into the original project plan

and making further adjustments as necessary, a new baseline plan (revised master plan

for the project) is now created. This might involve a reduction in scope, reprioritization of

goals and objectives, and clarification of the objectives and expectations. A re-estimation

of tasks will be performed, a new schedule will be developed, and resources allocated to

the project accordingly. In addition, the mix of personnel and skills inventory present

should be compared to the project requirements and appropriate actions planned, such as

training in new technologies. The risks involved in the project should be re-evaluated

and plans for mitigating the risk must be established and evaluated. In essence, a com-

plete new baseline plan is drawn up. The revised plan and milestones become the metric

against which success for the project is now measured. The revised baseline must be put

through the normal business value approval process. Management and customer ap-


5�

proval for the new baseline must be obtained before proceeding further, i.e., a go-no go

decision must be made. After the revised baseline is established and approved, the plan

should be implemented and the status of the project continually monitored. Implement-

ing the plan may include adding or redeploying resources. This may be human resources

that may involve re-assigning, refocusing, or removing personnel on the team. Other

project resources, such as software development tools or equipment, may need to be pur-

chased or changed in some manner.

Stage 4: Maturity Stage

The final stage is the maturity stage. This stage focuses on the on-going management of

projects in general and is critical for increasing the return on investment in recovered proj-

ects. Post implementation reviews of projects to learn from successful and fruitless actions

are not standard practice (Brady and DeMarco, �994). Research indicates that failures

continue to recur in organizations for various reasons (Fowler and Walsh, �999). Organi-

zational learning should help the individual project manager accomplish three goals: de-

liver a successful project, deliver a series of successful projects, and build project management

capabilities (Kotnour, �999). The detailed steps (or activities) in this stage are: document-

ing the lessons learned from each project, propagating project management knowledge

throughout the organization, and implementing this knowledge in other projects.

Comparison of Framework with Other Existing Frameworks

Other recent research discusses project management using early warnings to help recover

projects (Nikander and Eloranta, 200�). Early warnings reflect probable problems, lead-

ing the decision-maker to identify the underlying root cause of the problem. A response

to the problem follows using standard risk management techniques. These approaches

focus primarily on our problem recognition and decision to recover stage.

The State of Michigan’s project management resource center and others discuss

steps to assess and recover troubled projects (“Assessment, recovery, transition,” 2007;

Sifri, 2003a, 2003b, 2004). Both frameworks appear similar and contain two major

phases: assessment and recovery. The project management life cycle is applied in each of

the phases. Developing an assessment charter, assessment plan and conducting assess-

ment activities are the steps in the assessment phase. The recovery phase includes devel-

oping the recovery plan and implementing the recovery. The activities outlined in the

assessment phase are similar to the actions needed to be done in step 3 and 4 of the

TPRF, and is compatible with the TPRF. TPRF explicitly recognizes the need to per-

form both a short term recovery and a sustained recovery. In addition, TPRF includes

both a problem recognition stage and a maturity stage. The maturity stage emphasizes

the need for organizational learning which is consistent with the project management

e-Service Journal

52

book of knowledge (PMBOK). Organizational learning is of paramount importance to a

comprehensive recovery strategy, since not learning from mistakes dooms us to repeat

them (Iacovou and Dexter, 2005).

THE FIELD STUDY

To assist project managers in applying the TPRF a field study was performed to identify

common symptoms that might cause or contribute to trouble during information sys-

tems development projects. The purpose of the study is to identify and categorize these

symptoms to allow managers to achieve the first stage of the proposed framework, i.e.

recognize and acknowledge that trouble exists. This, in turn, should lead to the later

stages where the set of symptoms could be used to improve, measure, or take corrective

action on projects that are in trouble or may be heading for trouble. These results should

be of interest to business managers and IS project managers dealing with the project

management process as well as to information systems researchers attempting to better

understand the IS development process.

Research Method

The field study consisted of a series of focus groups undergoing a nominal group process

(Delbecq, Van de Ven, and Gustafson, �982). An underlying assumption of the method

is that individuals who perform a task can provide valuable insight into the important

factors influencing their ability to achieve a high level of productivity when performing

the task. This method has been used successfully in several domains including systems

development (Havelka, 2002; Havelka, Arnold, and Sutton, �998; Havelka, Sutton, and

Arnold, 200�).

In the current study, this method was used to identify symptoms of trouble on IS

development projects. The nominal group technique used in the focus groups is a modi-

fication of procedures developed by prior research in the social sciences (Delbecq et al.,

�982). The technique consisted of five steps. All five steps were conducted during a single

meeting (of each group) that took approximately two hours. The technique is discussed

in the next paragraphs and summarized in Table �. Detailed instructions for conducting

the nominal group technique are available upon request.

The first step in the nominal group technique was a general discussion of the subject

being addressed by the group. In this case, the subject chosen for study was the identifica-

tion of symptoms that might indicate trouble on an IS development project. The objective

of the first step was to provide a common understanding of the subject and to determine

the scope of the topic under study. A general definition was given of the IS development

process, the tasks performed during the process, and the outputs of the process. The par-

ticipants were then given the opportunity to clarify their understanding of the subject.


53

After any questions were raised and discussed, group consensus was used to define the

process and the boundaries of the problem to be addressed in the subsequent steps.

The second step in the nominal group technique was a period of silent generation of

ideas. This was achieved by presenting a nominal question to be addressed by the group

and asking them to record their thoughts individually and silently on worksheets. The

objective of this step was to generate as many ideas as possible for later discussion. The

participants were encouraged to be creative and to interpret the question based on their

own experiences and knowledge. The question posed to the groups in this step was:

What symptoms can you identify that might indicate that an information system

development project is in trouble, i.e., it is over-budget, behind schedule, or not

meeting specifications?

The benefits of this silent, independent idea generation include: adequate time for

thinking and reflection, social facilitation (i.e., constructive tension by observing other

participants working), avoidance of interruptions, avoidance of prematurely focusing on

particular ideas, sufficient time for search and recall, avoidance of competition, avoidance

of status pressures, avoidance of conformity pressures, and avoidance of choosing between

ideas prematurely (Delbecq et al., �982). The question being asked should be general

Table 1. Nominal group technique used

Step �: The facilitator made general introductions of the participants, explained the purpose of the study and the meeting, presented definitions to be used by the subjects, explained the nominal group process to be performed, and introduced the question to be answered by the subjects. A definition of troubled projects was given and the research question was presented to the group.

Step 2: Each subject was then asked to silently generate as many of symptoms as possible.

Step 3: After �5 minutes, the facilitator began to write the symptoms on a white board or flip charts for all participants to view. The symptoms were elicited from the participants in a round-robin fashion until all the participants’ symptoms had been listed. Only questions related to clarifying the symptoms being listed were allowed at this point and no discussion of the merits or importance of the symptoms was allowed. Participants were encouraged to add to their lists as this step progressed.

Step 4: After all of the symptoms were listed, discussion of the symptoms for clarification of the items and distinction from one another was allowed. Again, discussion of the relative merits or importance of the indicators was discouraged.

Step 5: Each participant was then asked to identify on a worksheet those symptoms generated by the group that they considered “critical” to IS audit quality.

Step 6: Participants were asked to rank the symptoms that they identified in step 5 from most to least important.

Step 7: Participants were asked to fill out a questionnaire for demographic data. Partici-pants were invited to an informal venue for further debriefing and discussion.

e-Service Journal

54

enough that the participants interpret it based on their personal perspectives. To this end,

the leader of the meeting should take special care to avoid answering the question for the

group, thereby focusing the group on a particular viewpoint toward the question.

After the silent generation of ideas is complete, a round-robin recording of the ideas

generated is performed. The leader of the group prepares an aggregate list of the partici-

pants’ ideas by asking for one idea from each person on each round. The aggregate list

was prepared in written format and was presented so that all the participants could view

it. During this step, participants were encouraged to add new ideas to their lists. The

benefits of the round-robin recording in step 3 include: equal participation in the presen-

tation of ideas, increase in problem-mindedness, depersonalization of ideas from person-

alities, increase in the ability to deal with a large number of ideas, tolerance of conflicting

ideas, encouragement of ”hitchhiking” of ideas, and a written record and guide of the

ideas presented. It is thought that by sharing ideas and equalizing participation, the cre-

ativity of the group is increased (Delbecq et al., �982). The round-robin format of elicit-

ing the ideas also establishes an important group behavior. After two or three rounds,

each member in the group is an achieved participant. This sets the precedent for further

active participation.

After recording of the ideas was complete, each idea was discussed for clarification

and definition so that the group could come to a consensus as to the item’s meaning. Dis-

cussion of each idea was performed to share the logic behind the idea, the relative impor-

tance attached to the idea, and to air agreements and disagreements. The benefits of

discussing each idea include: opportunity for clarification and elimination of misunder-

standing, opportunity to provide logic for arguments and disagreements, and recording

of differences of opinion without undue argumentation (Delbecq et al., �982). To ac-

commodate these benefits, the leader should make it clear that disagreements regarding

the relative importance of each idea are acceptable and expected. With respect to this

step, it should be emphasized that the individual responsible for suggesting an idea is not

directly responsible for clarifying the idea. The clarification is a group task and not the

unilateral responsibility of the author of the idea (Delbecq et al., �982).

The final step in the nominal group technique was to gather demographic data

from the subjects and perform an informal de-briefing. Information regarding each sub-

ject’s project management experience, training, IT development experience, education

level, and other demographics were gathered using a questionnaire. The informal de-

briefing usually consists of a comparison of the subjects’ inputs and evaluations of the

group results and the process overall.

There were four focus groups; the first was used as a “pre-test” of the data gathering

method and to establish some baseline constructs. Each group met for approximately

two hours. Group I consisted of five subjects. At the time of the focus group, all the sub-

jects were employed as instructors for a large mid-western university and had significant


55

(>5 years each) experience as IS consultants for information services organizations. Group

II consisted of five subjects all of whom worked in the IS department of a large financial

services firm and had acted as the project manager for IS projects for a minimum of six

years (average = 8.4 years). Group III consisted of six subjects from the same IS depart-

ment as Group II. These subjects had at least three years of IS project management expe-

rience and averaged �0.� years of IS project management experience. Group IV consisted

of four subjects from two separate organizations, one a large information services firm

and the other a major telecommunications firm. These subjects had an average of four

years of IS project management experience. The details of the focus groups are summa-

rized in Table 2. The output of these focus groups included a list of symptoms from each

participant and a comprehensive, master list of symptoms for each group.

Field Study Results

The results of the field study will be presented as follows: a general discussion of the focus

group outputs is given and then a discussion of the symptoms identified is given within a

logical categorization of the symptoms. The four focus groups yielded a total of �57 sepa-

rate potential symptoms; the groups identified 39, 5�, 43, and 24 symptoms respectively.

After review, 49 of the symptoms were considered duplicates, leaving �08 unique symp-

toms. Of the �08 unique symptoms identified, only 7 were identified by all four of the

groups, 9 were identified by three of the four groups, 22 were identified by two of the

four groups, and 70 were identified by only one of the four groups.

A few earlier studies have specifically looked at early warning or trouble indicators in

projects. One recent study identifies a dominant dozen of early warning signs of IS project

failure (Kappelman, Mckeeman, and Zhang, 2006). That study specifically looked at

warnings that occured in the first 20 percent of the projects initial calendar. The dozen

warnings primarily were people related and process related factors. People related factors

included lack of top management support, weak project manager, no stakeholder involve-

Table 2. Background of focus group participants

Group Industry

Number of Members in Focus group

IS/Project Management Experience

I University 5>5 years as Information Systems

Consultants

IIFinancial

Services Firm5 >6 years as Project Managers

IIIFinancial

Services Firm6 >3 years as Project Managers

IVTelecommunications and Information Services Firm

4 >4 years as Project Managers

e-Service Journal

56

ment, weak commitment of team, team members lacking requisite knowledge, and over-

scheduling of subject matter experts. Process related indicators included lack of

documented requirements, change control, communication breakdown among stake

holders, resources assigned to a higher priority project, and no business case. In contrast,

other researchers looked for warnings through the entire project life cycle and identified

eleven major categories of early warnings in projects (Nikander and Eloranta, 200�).

These categories included personnel or project group, project manager and management,

project planning, project control and reporting, working within the project, communica-

tion, exposed by parties (stakeholders such as clients, CEO, suppliers), and documents.

Using the prior research as a guide, a set of potential logical categories for the symp-

toms was identified. This set of symptom categories was determined by having indepen-

dent judges (the researchers and colleagues) assign each of the symptoms identified to a

category. The researchers then finalized these categories collaboratively and allowed a

specific symptom to be assigned to multiple categories as appropriate. There were eleven

categories identified in the final model. The categories and the number of symptoms as-

signed to each category are presented in Table 3. Each of these categories is discussed

below and the associated symptoms are presented.

Client-Related Symptoms. The first category of symptoms identified contains symp-

toms that are all related to the client or other stakeholders of the project. These symptoms

are presented in Table 4. These symptoms are in one way dependent upon or driven by

the project’s client(s) or other stakeholders (excluding members of the IS department or

project team members). As can be seen by the symptoms identified, the root cause of

many of these symptoms appears to be related to the client’s commitment, buy-in, and

Table 3. Logical categories and number of related symptoms

Logical Category Number of Symptoms

�) Client or stakeholder-related symptoms �2

2) Project’s goal-related symptoms �0

3) Meeting symptoms 6

4) Team symptoms �3

5) Task symptoms 7

6) Project symptoms �2

7) Project Management symptoms 2�

8) Communication symptoms 5

9) Management symptoms 8

�0) Project portfolio symptoms 7

��) Process symptoms 7


57

support of the project. Also, management of client’s expectations and a clear specification

of the project’s goal would appear to be related to the client-related symptoms.

Project Goal-Related Symptoms. The focus group subjects identified ten symptoms

that are included as project goal-related symptoms (presented in Table 5). These symp-

toms are all characteristics or attributes of the project’s objectives and desirable outcomes.

It appears that these symptoms become relevant when there is a lack of clearly defined

purpose, scope, and agreed upon metrics for success for the project. These symptoms

may become more important when there are multiple clients with multiple expectations

from the project.

Meeting-Oriented Symptoms. A handful of symptoms centered on meetings (Table

6). These symptoms appear to indicate trouble in two ways: (�) evidence that team mem-

bers have “given-up” by no longer attending or participating in meetings and (2) an in-

crease in meeting activity with clients and management that indicates something out of

the ordinary is happening. Regardless, changes in the quality, quantity, and scheduling

of meeting may be useful metrics to gauge a project’s health.

Team-Related Symptoms. The team-related symptoms represent characteristics of

the team or team behaviors that may indicate trouble on the project. These symptoms

included items such as the skill-set of the team members as well as team morale and stress

level. These symptoms emphasize the importance of creating and managing a well-

Table 4. Client-related symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Loss or lack of project sponsorship

Sponsor demands end date before scope and schedule are set

Efforts to redefine project after specifications are set

Poorly defined requirements—sponsor continually adding requirements (scope creep)

Different user groups communicate different points of view—adding to the complexity of the project

Lack of subject matter expert or user involvement

Lack of commitment by major stakeholders

Client informs project manager that project is in trouble

Client does not understand their needs

Sponsor does not want to meet with team members

Unrealistic stakeholder expectations

Lack of willingness to compromise

e-Service Journal

58

blended, high-performance team. As Table 7 shows, there are four symptoms in this cat-

egory that were identified by all four groups: high team turnover, high stress levels,

depletion of resources, and morale. Clearly, these symptoms could be useful to IS project

managers as indicators of trouble or risks that may need special attention.

Task-Related Symptoms. Several symptoms were identified that could be considered

task-related. These symptoms were associated with specific tasks and activities that need

to be performed during software development or the assignment, documentation, or

completion of these tasks during the project (Table 8). In general, it appears that most of

these symptoms point to a lack of good project management. Issues with scoping and

task assignment are regularly the responsibility of the project manager. This suggests that

perhaps these symptoms should be monitored by the project’s client sponsor.

Project-Related Symptoms. The next category is made up of symptoms that are re-

lated to the project itself, that is, these are symptoms that are particular or unique to the

Table 5. Goal-related symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Lack of clear scope definition

Efforts to redefine project

A large number of change requests or requirements fluctuations within the project

Project does not meet business strategy

Recurring questions about the value of the project

Lack of success criteria

Lack of definition for quality

Client does not understand their needs

Lack of clearly articulated goals

Conflicting objectives of participants

Table 6. Meeting symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Lack of participation at team meetings

Reduced attendance at project meetings

Too many meetings

Increased meetings with clients to clarify problems

Senior management involvement at project meetings

Change in meeting frequency


59

specific project (Table 9). These symptoms include characteristics of the system being

developed, e.g., size or value, as well as symptoms related to the project itself, e.g., ade-

quate budget and resources. Some of these symptoms may also be considered project

management symptoms, such as use of a sound methodology.

Project Management Symptoms. A total of 2� symptoms were identified and pre-

sented in Table �0 that were assigned to the category labeled project management. These

Table 7. Team symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Team fighting

High team turnover

Lack of communication on team

Resources dragging their feet on tasks, hoping tasks will go away

Lack of critical skills

Lack of team buy-in

Different levels of expertise within the team leading to communication problems

Stress

Adding new resources

Depletion of resources (e.g., allocation to other projects)

Morale

Team does not understand requirements

Lack of willingness to compromise

Table 8. Task symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Limited, or lack of, documentation (e.g., not docu-menting tasks completed or technical specifics)

Consistent levels of overtime

Design problems discovered during code review

WBS (work breakdown structure) not assigned

Major defects identified during the inspection—changes in the requirements or solution

High amount of rework or defects

Training deferred or postponed

e-Service Journal

60

symptoms were directly associated with how the project is being managed or the project

management practices followed or established by the organization (or lack thereof). Sim-

ilar to the task-related symptoms, this category may include primarily items that client or

project manager’s supervisor should monitor. One of these symptoms, project status un-

known, was identified by all four groups. The fact that the project’s status is not known,

i.e., budget, schedule, or meeting specs, would be a clear indicator that trouble is pres-

ent.

Communication Symptoms. The next category was made up of symptoms that were

all related to communication (Table ��). This included communication among the team

members, between the team and user groups, and among the team, user groups and

management. Communication during systems development has been shown by previous

research to be critical for establishing correct system requirements and maintaining good

relationships between developers and users. From the symptoms listed, it appears that

communication would lead to errors and possibly re-work or incorrect features.

Management Symptoms. The next category includes symptoms that were all related

to the organization’s general management, including support of upper management and

management issues such as clearly defined roles and responsibilities (Table �2). The im-

portance of support and involvement of management in systems development projects

Table 9. Project symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Over allocation of resources (i.e., more work than a person can reasonably perform)

Constantly changing scope

Lack of budget in terms of people, time, and materials for the project

Failing to use any methodology in that the team or sponsor is not following the method

New technology that is unfamiliar to the team

Risk mitigation delayed until end of the project (i.e., all major symptoms appear at the end of the project)

Scope is too large, indicator of size of project

High CPI (cost/performance index) in that costs are higher than benefits

Lack of budget for training

Team feels lack of ownership

Lack of project value

Budget unknown


6�

Table 10. Project management symptoms

SymptomGroup I

Group II

Group III

Group IV

Project status not known due to poor status report meetings

Schedule slippages along critical path

Poor change management

Schedule of work not defined or poorly defined

Revisiting issues thought to be closed

Milestones are not met

Difficulty in assessing the impact of change requests

Variance of time estimates (high or low)

Constantly behind schedule

Deliverables are not met on schedule

Symptoms not identified

Poor estimates

No one to take ownership of issues

Lack of leadership initiatives

Late PM engagements

Key risk issues not addressed

Not able to define team resources up front

Poor management of outsourced resources

Wrong product or deliverable defined in requirements

Expenditures are over budget

Multitasking of resources

Table 11. Communication symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Lack of communication on team

Different levels of expertise within the team – communication problems

Miscommunication between user and team

Lack of “say” or “input”

Escalation increases

e-Service Journal

62

has also received attention in prior research. Management can impact IS project success

in many ways, from controlling resources to setting the tone that a project is or is not

important to the organization.

Project Portfolio Symptoms. The next category included was project portfolio symp-

toms (Table �3). These symptoms are related to how the organization manages its port-

folio of projects and where the project under consideration relates to other projects in the

portfolio. These symptoms impact a project in several ways, including projects doing re-

dundant work or competing for resources. In addition, the value each project may add

Table 12. Management symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Weak project sponsorship (i.e., lack of commitment)

Pressure from upper level management causes stress on team

Reduction in staff, yet project kept on original schedule (i.e., original milestones)

Micromanagement by upper management

Lack of confidence in management

Unclear roles and responsibilities

Inability to commit resources

Multicultural offshore teams with issues such as communications, time zone, cultural habits, language differences

Table 13. Project portfolio symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Difficulties in blending across organizational teams (i.e., teams and/or management from different areas do not identify the problems in the same manner)

Management and project not in sync

Project interdependency

Project has a low priority

No portfolio management in that two similar projects underway

Inability to prioritize project

Putting a project on hold while management works it out


63

toward organizational goals may change over time and affect the priority of one project

vis-à-vis another.

Process-Related Symptoms. The last category of symptoms identified are process-

related items (Table �4). These symptoms were primarily related to the IS development

or implementation process being used. These symptoms could be considered the project

manager’s responsibility also; however, most of the items do not appear to be directly

within the project manager’s control, e.g., no quality assurance process in place.

In summary, the results of the field study yielded a total of �08 unique symptoms.

The vast majority was identified by only one of the groups. Only seven symptoms were

identified by all four. The symptoms appear to fall into eleven distinct logical categories.

Potential implications and further exploration of these results are discussed below.

DISCUSSION

Overall, the results of the focus groups suggest that there are different types of “symp-

toms” that may present themselves during a project that could cause or indicate trouble.

Some of the symptoms identified by the groups appear to be symptoms or indicators of

trouble; however, it appears that other items suggested by the groups are clearly the cause

of trouble rather than a symptom of a problem, e.g., lack of management commitment.

To further explore this distinction, each logical category of symptoms is reviewed and the

items identified are further classified as either a likely cause of trouble or a symptom of a

problem (or trouble indicator). In addition, for each category, prescriptions are suggested

that should help project managers and team members address the cause and symptom.

Causes, Symptoms, and Prescriptions

Although the goal of this research project was to identify symptoms that would act as

early warnings for trouble on a project, many of the items identified by the subjects were

more akin to causes or descriptions of problems. In fact, some of the items identified

Table 14. Process symptoms

SymptomGroup

IGroup

IIGroup

IIIGroup

IV

Testing is rushed, limited, or eliminated

Automating process without analysis

No lessons learned from a previous failure

No quality assurance or evaluative process

Lack of definition for quality

Lack of documentation (i.e., results, planning, design)

No R&D prior to planning

e-Service Journal

64

could be considered both a cause and a symptom. For each category of items identified,

we discuss causes, symptoms, and prescriptions for corrective action.

Client-Related Causes, Symptoms, and Prescriptions. As presented in Table �5, the

causes of trouble related to clients are primarily associated with the need for information

from the clients and the on-going support in terms of resources and psychological com-

mitment. For a project to succeed, substantive information from the client (users and

sponsor) is necessary to build a system that performs the desired tasks in a way the clients

can understand. The symptoms identified reflect difficulties in communicating this in-

formation and the complexity involved with building systems that serve multiple user

groups. The prescriptions suggested focus on establishing a common understanding

among clients and the IS staff working on the project, which includes an evaluation of

the business value of continuing the project.

Goal-Related Causes, Symptoms, and Prescriptions. The causes of trouble associated

with the goals of the project (see Table �6) appear to fall into two general types: (�) the

lack of documentation or work to clearly define the purpose, goal, and objectives of the

project and (2) misunderstanding or conflict about the goals of the project. The symp-

toms all center on efforts to correct these shortcomings by redefining and changing the

work and persistent questions regarding the project’s value. The prescriptions suggested

focus on gaining consensus about the value and purpose of the project and specifying in as

much detail as possible the deliverables expected and then obtaining agreement for these.

Meeting Causes, Symptoms, and Prescriptions. The meeting related items all appear

to be symptoms of trouble rather than causes of problems (see Table �7). Some of the

causes of meeting symptoms may include any of the causes identified in other categories.

Table 15. Client/stakeholder causes—symptoms—prescriptions

Causes Symptoms Prescriptions

Loss or lack of project sponsorshipPoorly defined require-ments—sponsor continu-ally adding requirements (scope creep)Lack of subject matter expert or user involvementLack of commitment by major stakeholdersClient does not understand their needsUnrealistic stakeholder expectationsLack of willingness to compromise

•

•

•

•

•

•

•

Sponsor demands end date before scope and schedule are setEfforts to redefine project after specifications are setDifferent user groups communicate different points of view – adding to the complexity of the projectClient informs project manager that project is in troubleSponsor does not want to meet with team members

•

•

•

•

•

Determine specific problem, if necessaryEvaluate the value of continuing projectDefine the scope of the project and specify boundariesIdentify the stakeholders and end-users and solicit support, especially a sponsorReview project specifica-tions and requirements with sponsor and users, obtain signoffs

•

•

•

•

•


65

The prescriptions suggested focus on investigating the causes and reviewing assignments

and responsibilities.

Team Causes, Symptoms, and Prescriptions. The causes related to the development

team address communication, competence, and psychological issues (Table �8). In gen-

eral, the lack of communication or skill level impacts the ability of the team to perform

adequately. The symptoms reflect the team’s general frustration and low morale. The

prescriptions suggested focus on motivating and energizing the team and making changes

to the team membership if necessary.

Task Causes, Symptoms, and Prescriptions. The causes of trouble associated with

Table 16. Goal-related causes—symptoms—prescriptions


Lack of clear scope definitionProject does not meet business strategyLack of success criteriaLack of definition for qualityClient does not understand their needsLack of clearly articulated goalsConflicting objectives of participants

•

•

••

•

•

•

Efforts to redefine projectA large number of change requests or requirements fluctuations within the projectRecurring questions about the value of the project

••

•

•

Determine specific problem, if necessaryEvaluate the value of continuing project, determine business objectiveDefine the scope of the project and specify boundariesReview project specifica-tions and requirements with sponsor and users, obtain signoffsSpecify clear metrics for project and system success

•

•

•

•

•

Table 17. Meeting causes—symptoms—prescriptions


Lack of participation at team meetingsReduced attendance at project meetingsToo many meetingsIncreased meetings with clients to clarify problemsSenior management involvement at project meetingsChange in meeting frequency

•

•

••

•

•

Determine the specific problem, if necessaryInterview key staff to determine reasons for absence or low participa-tionReview project specifica-tions and requirements with sponsor and users—rework project schedule and deliverables if changed from original project approved

•

•

•

e-Service Journal

66

tasks, presented in Table �9, address the need to document each person’s responsibilities

and track their progress. Each task should have one individual responsible for its perfor-

mance, making sure that it is performed, not necessarily doing it themselves, with a dead-

line for the task. The symptoms indicated for this category focus on “extra” work that is

performed to complete tasks correctly. This is in the form of rework or overtime, but also

includes poor performance when work does not pass inspections or testing later in the

development process.

Project Causes, Symptoms, and Prescriptions. Table 20 presents the project related

causes, symptoms, and prescriptions. The causes hit upon several different themes in-

cluding: lack of resources, lack of structure in the form of methods and procedures,

Table 18. Team causes—symptoms—prescriptions


Lack of communication on teamLack of critical skillsLack of team buy-inDifferent levels of expertise within the team leading to communication problemsDepletion of resources (allocation to other projects)Team does not understand requirementsLack of willingness to compromise (users & IS)

•

•••

•

•

•

Team fightingHigh team turnoverResources dragging their feet on tasks, hoping tasks will go awayStressAdding new resourcesDepletion of resources (e.g., allocation to other projects)Morale

•••

•••

•

Determine specific problem, if necessaryInterview key staff to pinpoint issues (personal-ity conflicts, overworked)Change team membersEvaluated workloads and compensationPerform team building and communication exercisesReview project specifica-tions and requirements with sponsor and users, obtain signoffs

•

•

••

•

•

Table 19. Task causes—symptoms—prescriptions


Limited, or lack of, docu-mentation (i.e., not docu-menting tasks completed or technical specifics)WBS (work breakdown structure) not assigned

Consistent levels of overtimeDesign problems discovered during code reviewMajor defects identified during the inspection—changes in the requirements or solutionHigh amount of rework or defectsTraining deferred or postponed

Review project methods and practices with teamReview and update project management artifacts, e.g. the WBSEvaluate workloads and compensationReview project specifica-tions and requirements with analysis, design, coding, and testing personnel

•

•

•

•


67

vagueness about the project, and the use of new or unfamiliar technology. The symptoms

also reflect a variety of issues including overloading team members with tasks and delay-

ing dealing with problems. The prescriptions for project related problems include evalu-

ating the business case to assess risk versus return for the project.

Project Management Causes, Symptoms, and Prescriptions. The items associated

with project management (Table 2�) reflect common project management issues. The

causes focus on lack of control and the quality of management of resources. Symptoms are

related to the standard overarching project management metrics of time, cost, and work

performed. Prescriptions for project management would include establishing and follow-

ing project management best practices and perhaps changing the project manager.

Communication Causes, Symptoms, and Prescriptions. The communication category

causes are summed up as lack of proper communication among team members and the

clients (Table 22). The symptoms include the frequent escalation of issues to get clarifica-

tion or a decision from higher up in the hierarchy. Prescriptions include establishing for-

mal policies and procedures for information exchange among the key groups involved.

Management Causes, Symptoms, and Prescriptions. Causes associated with general

management included “too much” involvement from upper management, e.g., micro-

managing tasks, as well as “too little” involvement in the form of lack of commitment

and resources (Table 23). The symptoms reflect an inappropriate involvement in the day-

to-day tasks by management. The prescriptions include reviewing the project purpose

and reaffirming management’s support of the project.

Project Portfolio Causes, Symptoms, and Prescriptions. The project portfolio focuses

attention on the fact that one project is usually dependent upon other projects and systems

Table 20. Project causes—symptoms—prescriptions


Constantly changing scopeLack of budget in terms of people, time, and materials for the projectFailing to use any method-ology in that the team or sponsor is not following the methodNew technology that is unfamiliar to the teamScope is too large, indicator of size of projectLack of budget for trainingLack of project valueBudget unknown

••

•

•

•

•••

Over allocation of resources (i.e., more work than a person can reasonably perform)Risk mitigation delayed until end of the project (i.e., all major symptoms appear at the end of the project)High CPI (cost/perfor-mance index) in that costs are higher than benefitsTeam feels lack of owner-ship

•

•

•

•

Define the scope of the project and specify boundariesPerform financial analysis, risk assessment, and business case to determine benefits of continuing projectReview project methods and practices with teamDetermine budget and constraints—assess level of skill for technology being used

•

•

•

•

e-Service Journal

68

Table 21. Project management causes—symptoms—prescriptions


Poor change managementSchedule of work not defined (i.e., poorly defined schedule)Poor estimatesLack of leadership initia-tivesLate PM engagementsKey risk issues not addressedPoor management of outsourced resources

••

••

••

•

Project status not known – due to poor status report meetingsSchedule slippages—along critical pathRevisiting issues that you thought were closedMilestones are not metDifficulty in assessing the impact of change requestsVariance of time estimates (high or low)Constantly behind scheduleDeliverables are not met on scheduleSymptoms not identifiedNo one to take ownership of issuesNot able to define team resources up frontWrong product or deliver-able defined in require-mentsExpenditures are over budgetMultitasking of resources

•

•

•

••

•

••

••

•

•

•

•

Perform an audit of work performed versus planReview (and test) docu-mentation of work performedChange project managersPerform financial analysis, risk assessment, and business case to determine benefits of continuing projectReview project methods and practices with teamDetermine budget and constraints—assess level of skill for technology being usedReview project specifica-tions, requirements, budget with sponsor

•

•

••

•

•

•

Table 22. Communication causes—symptoms—prescriptions


Lack of communication on teamMiscommunication between user and team

•

•

Different levels of expertise within the team—commu-nication problemsLack of “say” or “input”Escalation increases

•

••

Establish policy and procedures to exchange information between users and IS staffInterview key staff to pinpoint issues (personal-ity conflicts, overworked)Perform team building & communication exercisesReview project specifica-tions and requirements with sponsor and users, obtain signoffs

•

•

•

•


69

(Table 24). Ensuring that these projects and their ensuing systems can be integrated is a

major consideration in today’s corporate environment. Symptoms reveal difficulties when

two or more projects are addressing the same or similar business objectives. Prescriptions for

addressing these issues include establishing a project portfolio or program management that

manages the entire systems development function for an organization and establishing

knowledge management practices that would allow sharing of information across projects.

Process Causes, Symptoms, and Prescriptions. Table 25 presents the causes associ-

ated with the process used for IS projects. The causes are all in some manner related to

having an established methodology or process for IS project management. None of the

items in this category appeared to be actual symptoms of trouble. The prescriptions sug-

gested emphasize the importance of establishing a professional IS project management

function and following a methodology using best practices.

Further study should be performed to follow-up on these causes, symptoms, and

prescriptions to identify potential patterns that could apply across projects and organiza-

tions for IS projects. Clearly, further research is needed to test whether an item is a cause

or symptom of trouble and the nature of the relationship between causes, symptoms,

problems, and corrective prescriptions.

The Common Set of Symptoms. The fact that only seven symptoms were identified by

all four of the groups suggests that there may be a small set of “critical” symptoms that

Table 23. Management causes—symptoms—prescriptions


Weak project sponsorship (i.e., lack of commitment)Pressure from upper level management causes stress on teamReduction in staff, yet project kept on original schedule (i.e., original milestones)Micromanagement by upper managementUnclear roles and responsi-bilitiesInability to commit resourcesMulticultural offshore teams with issues such as communications, time zone, cultural habits, language differences

•

•

•

•

•

•

•

Micromanagement by upper managementLack of confidence in management

•

•

Review project specifica-tions and requirements with sponsor and users, obtain signoffsEvaluate the value of continuing projectDefine the scope of the project and specify boundariesDetermine who the stakeholders and end-users are and solicit support, especially a sponsor

•

•

•

•

e-Service Journal

70

project managers and sponsors could track as early indicators of trouble. The seven com-

mon symptoms are:

�. Lack of clear scope definition

2. High team turnover

3. Stress

4. Depletion of resources (allocation to other projects)

5. Morale

6. Consistent levels of overtime

7. Project status not known – poor status report meetings

Table 24. Project portfolio causes—symptoms—prescriptions


Difficulties in blending across organizational teams (i.e., teams and/or manage-ment from different areas do not identify the problems in the same manner)Management and project not in syncProject interdependencyProject has a low priorityNo portfolio manage-ment—two projects doing similar things

•

•

•••

No portfolio management in that two similar projects underwayInability to prioritize projectPutting a project on hold while management works it out

•

•

•

Review project specifica-tions and requirements with sponsor and users, obtain signoffsEvaluate the value of continuing project—in context of organization project portfolioIdentify the stakeholders and end-users and solicit support, especially a sponsor

•

•

•

Table 25. Process causes—symptoms—prescriptions


Testing is rushed, limited, or eliminatedAutomating process without analysisNo lessons learned from a previous failureNo quality assurance or evaluative processLack of definition for qualityLack of documentation (i.e., results, planning, design)No R&D prior to planning

•

•

•

•

•

•

•

Follow an established methodology for develop-ment – including appropri-ate analysis and business process redesignExecute project manage-ment best practicesImplement project portfolio and knowledge management practices to improve organizational learning

•

•

•


7�

Of these seven items, only one appears to be only a cause rather than a symptom,

i.e., lack of clear scope definition. The remaining items do appear to be good indicators

of some type of trouble. High turnover of personnel on the team could be a sign that the

“rats are fleeing the sinking ship” and the loss of competent personnel makes the project

even more likely to have errors or poor quality. In addition, the project may get a bad

reputation among the IS staff, making it difficult to recruit personnel. Stress and low

morale would both be indicators of the workload and frustration associated with poorly

run projects. The depletion of resources could be a cause of trouble, but as an indicator of

trouble this may indicate that de-funding of the project is occurring. Consistent overtime

is an indicator of rework due to low quality, poor specifications, or changing require-

ments; all of which indicate problems. Not knowing the status of a project indicates that

the project manager is not getting the documentation regarding work performed from

the staff or that the project manager is inadequately updating and reporting progress.

Together these seven items may make a good set of metrics to be used on a project man-

agement or project sponsor’s dashboard.

CONCLUSION

Many information system projects run into trouble. Many can be recovered and rehabili-

tated to success. We outlined the TPRF and performed a field study to validate the first

phase of our framework and provided guidance for managers using the TPRF. The four

stages include recognition, immediate recovery, sustained recovery, and maturity. With-

out recognition, no recovery can take place. The field study identified a broad set of

symptoms that could present themselves during a project that project managers can use

to recognize trouble.

The diversity of symptoms identified may indicate that industry and organization

specific symptoms exist, and some symptoms may be more prevalent or more important

in certain industries or application domains. However, the results also suggest that there

may be some symptoms that are general and relevant to all IS projects. These questions

require additional research into the details of the causes and effects of trouble in IS devel-

opment projects. Additionally, more research is needed to identify successful solutions to

common problems.

REFERENCES

Aiyer, J., Rajkumar, T. M., and Havelka, D. “A Staged Framework for the Recovery and Rehabilitation of

Troubled IS Development Projects,” Project Management Journal, Research Quarterly (36:4), 2005,

pp. 32–43.

“Assessment, Recovery, Transition,” Michigan Department of Information Technology, Retrieved Febru-

ary �0, 2007 from http://www.michigan.gov/documents/ART_200504_�22595_7.pdf.

Belassi, W., and Tuckel, O. “A New Framework for Finding Critical Success/Failure Factors in Software

Development,” International Journal of Project Management (�4:3), �996, pp. �4�–�5�.

e-Service Journal

72

Boundy, G., and Diamond, M. “Keeping Crisis out of Project Management,” Computing Canada, 29,

�998, p. 32.

Brady, S., and DeMarco, T. “Management Aided Software Engineering,” IEEE Software, �994, pp. 25–

32.

Brown, W. J., Malveau, R. C., McCormick III, H. W., and Mowbray, T. J. AntiPatterns: Refactoring Soft-ware, Architectures and Projects in Crisis, John Wiley and Sons, New York, �998.

Caper Jones, D. “Patterns of Large Software Systems: Failures and Success,” IEEE Computer, �995a, pp.

86–87.

Caper Jones, D. Patterns of Software System Failure and Success, International Thompson Computer Press,

Boston, �995b.

CHAOS Chronicles, Research Reports from http://www.standishgroup.com, 2004.

Connell, C. “Healing Sick Software,” 2000, retrieved from http://www.chc-3.com.

“Cost overrun? Call in a Project Manager,” Business Times (4), 2006, retrieved September 2006 from

http://www.pmi.org/prod/groups/public/documents/info/ap_articles.asp.

Day, J. “Software Development as Organizational Conversation: Analogy as a Systems Intervention,” Sys-tems Research and Behavioral Science (�7), 2000, pp. 349–358.

Delbecq, A. L., Van de Ven, A. H., and Gustafson, D. H. “Guidelines for Conducting NGT Meetings,” in

D. R. Hampton, C. E. Summer and R. A. Webber (eds.), Organizational Behavior and the Practice of Management, 4th ed., Scott, Foresman, and Company, Glenview, Illinois, �982, pp. 279–298.

Fowler, A., and Walsh, M. “Conflicting Perceptions of Success and Failure in an Information Systems Proj-

ect,” International Journal of Project Management (�7:�), �999, pp. �–�0.

Glass, R. L. “Failure is Looking More like Success These Days,” IEEE Software, 2002, pp. �03–�04.

Havelka, D. “Requirements Determination: An Information Systems Specialist Perspective of Process

Quality,” Requirements Engineering Journal (6:4), 2002, pp. 220–236.

Havelka, D., Arnold, V., and Sutton, S. “A Methodology for Developing Measurement Criteria for Assur-

ance Services: An Application in Information Systems Assurance,” Auditing: A Journal of Practice and Theory (�7), �998.

Havelka, D., Sutton, S. G., and Arnold, V. “Information Systems Quality Assurance: The Effect of Users’ Ex-

periences on Quality Factor Perceptions,” Review of Business Information Systems (5:2), 200�, pp. 49–62.

Iacovou, C. L., and Dexter, A. S. “Surviving IT Project Cancellations,” Communications of the ACM (48:4),

2005, pp. 83–86.

Ibbs, C. W., and Kwak, Y. H. “Assessing Project Management Maturity,” Project Management Journal, 2000, pp. 32–43.

Jalote, P. CMM in Practice, Addison-Wesley, �999.

Kappelman, L. A., Mckeeman, R., and Zhang, L. “Early Warning Signs of IT Project Failure: The Domi-

nant Dozen,” Information Systems Management (3�:36), 2006.

Kapur, G. “How to Kill a Troubled Project,” 200�, retrieved from http://www.cioinsight.com/print_

article/0,3668,a=�932500.asp.

Keil, M., and Montealgre, R. “Cutting your Losses: Extricating your Organization Losses when a Big Proj-

ect Goes Awry,” Sloan Management Review, 2000, pp. 55–68.

Keil, M., and Robey, D. “Turning Around Troubled Software Projects: An Exploratory Study of the De-

escalation of Commitment to Failing Courses of Action,” Journal of Management Information Systems (�5:4), �999, pp. 63–87.

Keil, M., and Robey, D. “Blowing the Whistle on Troubles Software Development Projectsc” Communica-tions of the ACM (44:7), 200�, pp. 87–93.

Kiechel, W. “Facing up to Denial,” Fortune (�23), �993, p. �63.

Klein, G., and Jiang, J. J. “Seeking Consonance in Information Systems,” Journal of Systems and Software (56), 200�, pp. �95–202.


73

Kotnour, T. “A Learning Framework for Project Management,” Project Management Journal Research Quar-terly, �999, pp. 32–38.

Linberg, K. R. “Software Developer Perceptions about Software Project Failure; A Case Study,” The Jour-nal of Systems and Software (49), �999, pp. �77–�92.

McConnell, S. “The Business of Software Improvement,” IEEE Software, 2002, pp. 5–8.

Nikander, I. O., and Eloranta, E. “Project Management by Early Warnings,” International Journal of Proj-ect Management (�9), 200�, pp. 385–399.

Pintro, J. K., and Slevin, D. P. “Project Success: Definition and Measurement Techniques,” Project Man-agement Journal, Research Quarterly (�9:�), �988, pp. 68–72.

Seeger, and Ulmer. “Virtuous Responses to Organizational Crisis,” Journal of Business Ethics (3�), 200�,

pp. 369–376.

Sifri, G. “Assessing and Recovering Troubled Projects, Part �,” ESI Horizons (5:5), 2003a, pp. �–7.

Sifri, G. “Assessing and Recovering Troubled Projects, Part 2,” ESI Horizons (5:7), 2003b, pp. �–5.

Sifri, G. “Assessing and Recovering Troubled Projects, Part 3,” ESI Horizons (5:9), 2004, pp. �–5.

Smith, J., Keil, M., and Depledge, G. “Keeping Mum as the Project Goes Under,” Journal of Management Information Systems (�8:2), 200�, pp. �89–227.

Summary of a Workshop on Information Technology Research for Crisis Management, �998, retrieved from

(http://books.nap.edu/html/itr_crisis_mgmt/index.htm).

Tarek, A. M., and Madnick, S. “The Elusive Silver Lining: How we Fail to Learn from Project Manage-

ment Failures,” Sloan Management Review, �990, pp. 39–49.

Whittaker, B. “What Went Wrong? Unsuccessful Information Technology Projects,” Information Manage-ment and Computer Security (7:�), �999, pp. 23–29.

Documents

Using the Troubled Project Recovery Framework: Problem Recognition … · 2014-10-08 · success for these projects, we propose a staged framework for recovery and rehabilitation