“What in the World Were We thinking?” Managing Stakeholder … · 2019. 8. 15. · Transparent and Collaborative Project Estimation1 By Laura Zuber Quantitative Software Management,

PM World Journal What in the World Were We Thinking? Vol. II, Issue X – October 2013 Managing Stakeholder Expectations & Engagement www.pmworldjournal.net Second Edition

1 Laura Zuber

© 2013 Laura Zuber www.pmworldlibrary.net Page 1 of 13

“What in the World Were We thinking?” Managing Stakeholder Expectations and Engagement through

Transparent and Collaborative Project Estimation1

By Laura Zuber Quantitative Software Management, Inc.

Abstract

One of the biggest reasons IT projects overrun their schedules and budgets is that they commit to unrealistic schedule and budget expectations. Why? Because budgets and schedules are set before estimation analysis can be performed. Early in the life cycle very little information is available upon which to determine size and scope. Another problem is that project estimate assumptions are not well documented, updated, or communicated such that, stakeholders remain engaged as the project progresses, and project performance can be objectively evaluated at closeout. Reducing overruns and avoiding having to ask “What in the world were we thinking (when we committed to that project)?” begins with a transparent and collaborative estimation process. Stakeholders are willing to commit to defensible estimates. Producing defensible estimates requires processes and tools that promote communication and collaboration by all project stakeholders including the project team, executives, and customers. A defensible estimate is one that:

Is based upon known capabilities derived from historical projects

Incorporates quantified risk assessment of multiple estimate solutions that explore project goal tradeoffs

Promotes stakeholder contributions to increase their support and the chance of project success

This paper shows how providing project estimation and measurement data early and throughout the project life cycle, by role definitions, helps project managers engage stakeholders and manage expectations.

1 Second Editions are previously published papers that have continued relevance in today’s project

management world, or which were originally published in conference proceedings or in a language other than English. Original publication acknowledged; authors retain copyright. This paper was originally presented at the 7

th Annual UT Dallas Project Management Symposium in Richardson, Texas, USA in

August 2013. It is republished here with the permission of the author and UT Dallas.

http://www.pmworldjournal.net/

http://www.pmworldlibrary.net/


1 Laura Zuber


I. INTRODUCTION

A. State of Affairs Information Technology (IT), particularly software development, has a poor track record of success. Although the Software Engineering Institute published the Capability Maturity Model for Softwarei in the early 1990s, development processes remain immature. Project budget and schedule overruns are still the norm. Steve McConnell ii recently reported that although the most common project outcome is success, about one-third of all projects are cancelled or considered failures, and the average schedule overrun may approach 100 percent (Figure 1). One of the biggest reasons projects overrun their schedules and budgets is that they commit to unrealistic schedule and budget expectations. Schedule and budget expectations are set before an estimation analysis can be performed. Early in the life cycle very little information is available on which to determine scope and size. Uncertainty about productivity assumptions is high, as are resource availability and staffing strategies. Tom DeMarcoiii noted back in 1995 that “failure to deliver within our estimate is an estimating failure, not a production failure.” Very little has changed.

Figure 1. Rating of software project outcomes

B. Estimates Are Uncertain Because IT development is a process of discovery, project cost, schedule, scope, and quality must be estimated (development is not manufacturing). Requirements analysis and design are part of the product development life cycle, thus most of the work in determining what to do (scope) and how to do it come after the project has started. The problem is that projects commit to unrealistic schedule and budget expectations, due to little or no information about the size and scope or productivity. Yet the business reality is that projects must be estimated early in the life cycle to support business goals and strategic planning. What ends up happening is that project stakeholders agree to unrealistic commitments, hoping that change management, super qualified personnel,

Cancelled

Successful

Unsuccessful

Challenged




1 Laura Zuber


or some other factor will make the project successful. The project ends less than favorably, shareholders wonder that went wrong, and second-guess the decision.

III. PROJECT STAKEHOLDER MANAGEMENT

There are two effective approaches to solve this problem: 1) produce defensible estimates based upon known capabilities, and 2) engage stakeholders in the estimation process to reduce unrealistic expectations. These approaches implement two new processes documented in A Guide to the Project Management Body of Knowledge (PMBOK® Guide) – Fifth Edition iv:

13.2 Plan Stakeholder Management

13.3 Manage Stakeholder Engagement

According to PMBOK, “Manage Stakeholder Engagement is the process of communicating and working with stakeholders to meet their needs/expectations, address issues as they occur, and foster appropriate engagement in project activities throughout the project life cycle.v” The benefits to project managers are increased support, minimized resistance, and increased chance of project success. The two key activities that are part of the estimation process are:

Engaging stakeholders at appropriate project stages to obtain or confirm their

continued commitment to the success of the project

Managing stakeholder expectations through negotiation and communication,

ensuring project goals are achieved.

The early stage of project concept definition and estimation is a particularly important stage to engage stakeholders, because this is when stakeholders have the greatest influence and opportunity to effect decisions. The process of transparent and collaborative estimation described later in this paper implements several PMBOK process inputs, namely the methodologies used for stakeholder communication, identifying information and the persons or groups who will receive information, and historical information about previous projects. Engaging stakeholders in the estimation process increases the rigor of initial and subsequent estimates, because information upon which the estimate is based and its sources are open and verifiable. Presenting a recommended estimate, along with a thorough analysis of alternative estimates that show a range of potential outcomes, helps project managers anticipate stakeholders’ reactions to the project. Negotiations are focused on practical solutions that lead to realistic commitments, support data-driven decision-making, and keep stakeholders engaged as the project progresses.




1 Laura Zuber


III. DEFENSIBLE ESTIMATES Defensible estimates support risk analysis and risk management processes because they are based on historical data, either an organization’s own history, or industry databases. How much historical data is needed? Only four core metrics, defined by the Putnam Software Production Equationvi. The simple form of the software production equation is

The two big unknowns during estimation are size and productivity. These four metrics, plus defects (measure of quality) are what Putnam and Myers call Five Core Metrics vii, and they are all that is needed from each completed project to calibrate performance and provide a solid basis of estimation. support the intelligence to provide what stakeholders want – predictable operations. For example, the total number of story points delivered, total calendar months, and total effort expended, measured in person-hours, for the last three Agile projects for the financial division is all that is necessary to set realistic expectations for what the next Agile project will be able to achieve. This section presents the estimation project information to be shared with stakeholders. The next section describes a model for defining information access by role and responsibility. A. Estimation Terminology

Defining estimation terminology helps separate the estimation process from the business decision process. Organizations sometimes confuse the following terms and the business practices they represent:

Target – a goal, what we would like to do or achieve

Constraint – some internal or external limitation on what we are able to do

Estimate – a technical calculation of what we might be able to do at some level of scope, cost, schedule, staff, and probability

Commitment – a business decision made to select one estimate scenario and assign company resources to meet a target within some constraints

Plan – a set of calculated project tasks and activities that will give us some probability of meeting a commitment at a defined level of scope, budget, schedule, and staff.

Delivered Size is proportional to Effort over Time at some Productivity




1 Laura Zuber


Making a distinction between these parts of the process highlights areas where various stakeholders should be involved.

B. Life Cycle Management Estimation is not a process to be performed in isolation. It is a key input to project planning and business development strategy. However, it is important that the entire development life cycle is supported by data collection and analysis. For example, from a project management perspective, we can define four high-level life cycle management activities:

1. Feasibility – produces an initial rough order of magnitude (ROM) estimate using

high-level inputs like budget and time constraints and project scope

2. Estimation - creates more detailed estimates or refines ROM baseline plans

produced during Feasibility

3. Tracking & Replan - compare the project plan against project actuals and

generate a forecast to completion

4. Closeout - record the final schedule, effort, size and reliability (defect) information

and enter the project into your historical database to be used for future

estimation, benchmarking, and productivity analysis

These activities are merely a subset of project management processes. What ties these processes together is the common purpose to manage the life cycle using metrics collection and analysis. Engaging shareholders at the earliest point, Feasibility, sets expectations about what is possible based on project scope and the productivity of the project environment. Once the project passes Feasibility, assuming a “go” decision, the Estimation process objective is to gather as much detailed data on scope as possible to reduce uncertainty on assumptions and assess the relative merit of multiple solution alternatives. Tracking and Replanning promote effective stakeholder engagement and expectation management by providing timely project status reports and forecasts to complete, based upon actual milestone completion dates and product completion measures. The actual data collected during project execution phases feed into the final completed project historical repository, making Closeout a short, manageable process. Maintaining a consistent project Closeout process can be difficult, because team members move on to their next assignments as the life cycle winds down. Remember that team members are stakeholders. Obtaining their assessment of project successes, challenges, and facts about project events during a quick lessons learned meeting provide the insight necessary to interpret performance benchmark data. It also increases their buy-in, keeping them engaged in working hard to achieve business goals based on realism.




1 Laura Zuber


A. Create Estimate

Should we commit resources to this project? The basis of, or the thinking behind the answer to this question sets the stage for project success or failure. Every stakeholder has good intentions. Team members and project managers want to step up and deliver upon expectations of senior management. Senior management wants to achieve business goals and reduce risk. Customers have a vested interest in supporting the project team and removing obstacles. However, each stakeholder comes to the project with different perspectives, based upon different sources and interpretations of information about the project, coupled with personal and political objectives. The feasibility of a project is a business decision based upon the estimate of project benefits compared to cost and risks. For IT projects, this is a comparison of effort (cost) and time trade-off options to project targets and constraints. Creating an early Rough Order of Magnitude (ROM) estimate of size, historical productivity, and the most constraining factor of the current project (effort, cost, staff, or time) lets stakeholders quickly assess a project’s worth with minimal resources. Figure 2 shows a cumulative cost curve for the desired project goals, compared to a typical similar project from history.

A detailed estimate should be performed once requirements and design activities have progressed and needs are better understood.

Figure 2. ROM estimate shows project feasibility based upon history.

Subject matter experts are now better equipped to provide a detailed size estimate and adjust productivity predictions for known technical risks. The WBS is better defined and resource needs identified.




1 Laura Zuber


B. Validate With History What is reasonable? It is difficult to truly see whether an estimate is realistic until it is validated against history and weighed against other solution alternatives. Part of the performance benchmarking process is to create a series of project trend lines for each of the core metrics. Scatter plots of duration, effort, and defects (or quality) against size are used to perform regression analysis, calculating the average and standard deviations for each. Trend lines provide a visual means of showing how important project metrics stack up against typical values for similar projects of the same size. Coupling trend lines with completed project data points clearly shows the reasonableness of the current estimate. How does the project schedule compare? Values inside plus or minus one standard deviation are within the realm of possibility. Values outside this range approach the “Impossible Zone.” C. Assess the Risk To simplify this analysis for stakeholders, estimates can be compared to a single, “typical” project, presented by taking effort and duration from the average trend line. Shown in Figure 3 as the Balanced Risk Solution (circle), we see that the Current Solution (diamond) is aggressive on effort, that is it predicts less effort than the historical average for projects the same size. The statistical regions have been translated into a shaded region above the average (moderately risky), white around the average trend line (typical), and lighter shading below the average (moderately risky). Thus, the Current estimate effort is Risky. A detailed comparison between these solutions helps identify the driving forces, say staff size or scope that can be adjusted to bring the estimate in line with known capabilities and project goals.

Figure 3. Validate estimate against historical trends.




1 Laura Zuber


Figure 4 Risk Comparison Chart lists key project metrics on the left, along with values calculated for both the Current and Balanced Risk solutions. The risky values are all related to staff and effort. The estimate’s planned average staff of 7 FTEs (Full Time Equivalents) is less than one third of the typical project. Stakeholders can clearly see the difference, and are equipped to negotiate potential remedies; either reducing scope, extending the time line, or applying more resources.

Figure 4. Risk Comparison Chart. Current estimate’s risk assessed relative to the Balanced solution.

Left is Conservative, Middle is Typical, Right is Risky.

D. Evaluate the Alternatives More often than not project business targets are out of range. When the estimators do not have a repeatable estimation process based upon history, they have no grounds to refute unrealistic expectations. Nor are they able to engage diverse stakeholders from multiple disciplines to take advantage of their knowledge or ability to effect change. A metrics-based estimation process, however, allows quick and easy “what-if” analysis to explore the potential outcomes of alternative ways to run the project. Calculating the cost and risk for shorter or longer delivery times, fewer features, multiple releases, or larger teams fosters communication and negotiation among stakeholders. This process provides a framework for zeroing-in on the best estimate. Figure 5 Solution Comparison shows a simple bar chart of total effort for a range of potential outcomes. Similar charts for other core metrics quickly communicate the relative risk and reward of meeting targets and constraints, enabling stakeholders to make a commitment that supports the organization’s business goals.




1 Laura Zuber


Figure 5. Compare alternative estimate solutions core metrics (cost, effort, duration) to explore potential outcomes.

V. TRANSPARENT AND COLLABORATIVE ESTIMATION

Engaging stakeholders in any project management process requires planning, and a structure for sharing the right information with the right people at the right time. The basic building blocks for this structure are:

Organization Breakdown Structure (OBS)

Roles and Responsibilities

Access Permissions

Project Data

A. Organization Breakdown Structure Similar to a Work Breakdown Structure (WBS), the OBS defines the organization’s hierarchy for managing projects. Whether the defining split is by department, location, cost center, or other category, both projects and stakeholders may be assigned to an OBS node. The example in Figure 6 shows three nodes below the root; one by customer, and two by division. The Product Division is broken down further by application type. This is particularly common and useful for software and system development projects.




1 Laura Zuber


B. Roles, Responsibilities, and Permissions Defining clear roles and responsibilities helps stakeholders understand how they interact with the project team and one another, what activities to perform, and where they can make a contribution to the success of the project. Only a handful of roles need to be created. A simple way to define roles is to replicate levels of decision-making authority, adding functional expertise. Here are a few examples:

Executive – view only privilege across all nodes

Portfolio Manager – view only privileges for specified high-level nodes

Project Manager – primary responsibility; create projects and full access

privileges

Contributor – update privileges only

Stakeholder – view only privileges; limited project and/or node access.

Figure 6. Sample Organization Breakdown Structure.

The benefit of defining process roles is that they provide a low maintenance way to grant access to project data without having to keep up with individual stakeholder’s changing faces and positions across the organization. OBS nodes, roles, permissions, and stakeholder “users” work together to not only grant access to specific projects, but also to specify the extent to which data view and update authority is available. Figure 7 is a schematic of the conceptual architecture for collaborative estimation process participation, using four links.




1 Laura Zuber


Figure 7. Conceptual definition of shareholder project information access rights.

1): Projects are assigned to OBS nodes. Granting access to nodes above a particular node allows automatic roll-up of project data to support portfolio management and Project Management Office (PMO) functions. 2): Stakeholders are assigned to OBS nodes. This ensures that the right people have access to the right projects. 3): Stakeholders are assigned roles, such as Contributor. 4): Permissions are assigned to roles. Some stakeholders may require full access privileges to project data, such as the project manager and key technical staff. Others may contribute to the project by providing detailed sizing data, or performing what-if analysis. Senior managers and customers may be limited to view only access, so they remain informed and engaged, but cannot modify project records Because people wear many hats within an organization and may possess greater authority on different projects, stakeholders may be assigned multiple roles across multiple OBS nodes. For example, David is a Project Manager in the Aerospace Applications group within the Product Development Division. Since Dave is the software sizing expert, he is also assigned as Contributor in the other Product Division groups. There are two ways to implement this requirement: a) assign David as Contributor for each sibling OBS node, or b) assign David as Contributor to the parent OBS node. Option b only works if each project’s access is set to grant access to OBS nodes higher in the structure.




1 Laura Zuber


VI. Summary Process improvement initiatives are often underfunded, because the return on investment is difficult to demonstrate without data. Gathering a handful of metrics for completed projects and establishing a transparent and collaborative estimation process can result in huge savings simply by avoiding disaster. Taking on a risky project is a legitimate business decision under the right circumstances. Engaging stakeholders in the estimation process increases the chances that at the end of a project the question will not be “What in the world were we thinking?” but “What new opportunity is on the horizon?” References: i. Software Engineering Institute. Capability Maturity Model for Software, version 1.1,

Technical Report CMU/SEI-93-TR-024, February 1993.

ii. Steve McConnell, Steve. The Business Case for Better Software Practices, Construx Software webinar, April 2013, www.construx.com

iii. DeMarco, Tom. Why Does Software Cost So Much?, Dorset House Publishing, 1995. iv. Project Management Institute, A Guide to the Project Management Body of Knowledge

(PMBOK® Guide) – Fifth Edition, 2013, Project Management Institute, Inc. v. PMBOK® Guide, pg. 404. vi. Putnam, Lawrence H. and Ware Myers. Measures For Excellence, Reliable Software on

Time, within Budget, Yourdon Press computing series, 1992.

vii. Putnam, Lawrence H. and Ware Myers. Five Core Metrics, The Intelligence Behind Successful Software Management, Dorset House Publishing, 2003.



http://www.construx.com/


1 Laura Zuber


About the Author

Laura Zuber Texas, USA

Laura Zuber has 20 years of experience in software development consulting and training. She currently serves as a training instructor and customer support representative for QSM, Inc., a PMI Registered Education Provider. Prior to coming to QSM, Laura managed software development projects, and served as a senior software process improvement specialist at SAIC. She has performed process assessments, designed and implemented best practices, and co-lead the corporate metrics training program. Laura holds B.S. and M.S. degrees in Petroleum Engineering from Texas A&M University, where her master’s thesis was on risk analysis using advanced Monte Carlo simulation techniques. She enjoys singing, golfing, and running. Laura can be contacted at [email protected],



mailto:[email protected]

Documents

“What in the World Were We thinking?” Managing Stakeholder … · 2019. 8. 15. · Transparent and Collaborative Project Estimation1 By Laura Zuber Quantitative Software Management,