2010 IPM Day Metrics Kerzner

©2010 International Institute for Learning, Inc., All rights reserved. 1 Intelligence, Integrity and Innovation | www.iil.com | ©2010 International Institute for Learning, Inc., All rights reserved.

Thank you for joining us today. This webinar is brought to you by IIL – a global leader in:

Project, Program and Portfolio Management

Microsoft® Project and Project Server

Lean Six Sigma | Business Analysis

PRINCE2® | ITIL®

Project Management Metrics, Key Performance Indicators and Dashboards

©2010 International Institute for Learning, Inc., All rights reserved. 2 Intelligence, Integrity and Innovation | www.iil.com |

�� Material in this presentation has been taken from the following book:

�� Harold Kerzner, Project Management Metrics, Key Performance Indicators and Dashboards, John Wiley & Sons and IIL Co-publishers, 2011 publication release date.

Copyright


GRADE LEVEL COMPETENCY 1 Understand project metrics and key

performance indicators

2 Be able to identify and create project-specific metrics

3 Be able to track and report metrics on a project

4 Be able to measure and evaluate metrics

5 Be able to extract best practices from metrics

The Importance of Metrics Today


�� Good metrics are a means of continuous project health checks

�� Good metrics serve as early warning signs

�� Good metrics serve as risk triggers

A Stakeholder View of Metrics


�� Our disbelief in the value of using a project’s metrics (or at least the existing metrics)

�� Selecting the wrong metrics

�� Our fear of what project health checks and metrics may reveal

Why Projects Get into Trouble


The Metrics Value Spectrum

The Metrics Value Spectrum

Metrics are the Holy Grail, specifically time and cost metrics

Metrics do not work; focus should be on leadership, vision, strategy and prayers


�� Bad news: The surfacing of previously unknown or hidden issues could lead to loss of employment, demotions, or project cancellation.

�� Good news: Project health checks offer the greatest opportunity for early corrective action to save a potentially failing project. Health checks can also discover future opportunities.

Issues with Health Checks and Metrics


�� Health checks can be performed randomly when needed or periodically throughout various life cycle stages.

�� However, there are specific circumstances that indicate that a health check should be accomplished quickly. These include:

�� Significant scope creep �� Escalating costs accompanied by a deterioration in value and

benefits �� Schedule slippages that cannot be corrected �� Missed deadlines �� Poor morale accompanied by changes in key project

personnel

Timing of a Health Check


�� The person doing the health check does not understand the project or the corporate culture, thus wasting time.

�� The health check is too costly for the value we will get by performing it.

�� The health check ties up critical resources in interviews.

�� By the time we get the results from the health check, it is either too late to make changes or the nature of the project may have changed.

Misconceptions


New Success Criteria Key Performance Indicators

Dashboard Design

Governance

Measurement

New Developments in Project Management


�� Metrics are:

�� Objective, measurable attributes of project performance to assist in informed decision-making

�� A means of assessing the project’s health �� A means of assisting in the prediction of project

success and failure �� A means of capturing best practices and lessons

learned

Defining a Metric


�� Good metrics allow you to catch mistakes before they lead to other mistakes.

�� Metrics tell us if we are hitting the targets/milestones, getting better or getting worse.

�� Metrics are often ignored because they are hard to define and collect.

Using Metrics


�� To improve performance for the future

�� To improve future estimating

�� To validate the baselines

�� To improve client satisfaction

Benefits for Using Metrics


�� The larger and more complex the project, the greater the difficulty in measuring and determining true progress and success. Therefore, the larger and more complex the project, the greater the need for effective metrics.

Metrics and Project Complexity


�� You cannot effectively manage a project without having good metrics and reasonably accurate measurement capable of providing you with complete (or nearly complete) information.

Role of Metrics in Project Management


�� What variables to emphasize?

�� What variables to select?

�� How will it be measured?

�� How will it be reported?

�� How will it be viewed by the stakeholders?

The Confusion over Metrics


�� The selection of the metrics is often based upon who is doing the asking.

�� Each stakeholder may ask for a different set of metrics.

�� Offering your stakeholders all of the metrics in your library can open Pandora’s box.

The Selection of Metrics


�� Make sure that the metrics are worth collecting.

�� Make sure that we use what we collect.

�� Make sure that the metrics are informative.

�� Train the team in the use and value of metrics.

The Metric Selection Process


�� Customers may desire real-time reporting rather than periodic reporting, thus making some metrics inappropriate.

�� The cost and complexity of the measurement may make a metric inappropriate for use.

�� The metric does not fit well with the organizational process assets available for an accurate measurement.

�� Project funding limits may restrict the number of metrics that can be used.

�� Metrics must be periodically reviewed.

Not All Metrics Can Be Used


Defining and Selecting Metrics

PAST VIEW PRESENT VIEW

Metrics are fixed for the duration of the project

Metrics can change over the duration of the project

(Metric-Driven Project Management)

(Usually Time and Cost Metrics Only)


�� Has a need or a purpose

�� Provides useful information

�� Focuses toward a target

�� Can be measured with reasonable accuracy

�� Reflects the true status of the project

�� Supports proactive management

�� Assists in assessing the likelihood of success or failure

�� Is accepted by the stakeholders as a tool for informed decision-making

Characteristics of a Metric


�� Business-based or financial metrics are linked to long-term strategic and business objectives and usually do not change much once they are identified.

�� Project-based metrics can change for each project and each life-cycle phase. The metrics may be restricted by the project’s funding limitations.

Two General Categories of Metrics


�� Quantitative metrics (planning dollars or hours as a percentage of total labor)

�� Practical metrics (improved efficiencies)

�� Directional metrics (risk ratings getting better or worse)

�� Actionable metrics (affect change such as the number of unstaffed hours)

�� Financial metrics (profit margins, ROI, etc.)

�� Milestone metrics (number of work packages on time)

�� End result or success metrics (customer satisfaction)

Specific Types of Metrics


�� Too many:

�� Metric management steals time from other work �� Provides too much information such that

stakeholders cannot determine what information is critical

�� Provides information that has limited or no value �� Too few:

�� Fails to provide the right information �� Inability to make informed decisions

How Many Metrics are Needed?


�� There can be three information systems on a project:

1. One for the project manager 2. One for the project manager’s parent company 3. One for the stakeholders and the client

�� There can be a different set of metrics and KPIs in each of these information systems.

Metrics Must Support Information Systems


�� Collecting information and reporting:

�� Who will collect the information? �� When will the information be collected? �� When and how will the information be reported?

�� Measurements:

�� What should be measured? �� When should it be measured? �� How should it be measured? �� Who will perform the measurement?

Typical Questions Concerning Metrics


�� If it cannot be measured, then it cannot be managed.

General Rule


�� Metric measurements can be recorded as:

�� Numbers �� Percentages �� Dollars �� Counts �� Ratings (good, bad or neutral) �� Qualitative versus quantitative

Metric Measurements


The Metric/KPI Target Boundary Box

Risk of Project Failure

Unfavorable Expectation

Performance Target

Favorably Exceeding Target

Performance Characteristics

Superior

Normal

Caution

Urgent Attention

Target + 10%

Target - 10%

Target - 20%


�� Disagreements among the stakeholders

�� Mistrust among the stakeholders

�� Team members believe that metrics look like a spying machine that allows management and stakeholders to watch them closely

Causes for Lack of Support with Metrics


�� Metrics are an expensive and useless measurement

�� Metrics are costly to maintain and the benefits do not justify the cost

�� Metric measurements are a waste of productive time

Metric Naysayers


�� Although most companies use metrics for measurement, they seem to have a poor understanding of what constitutes a KPI, especially for projects.

Understanding the KPIs


�� The ultimate purpose of a KPI is measurement of items relevant to performance and to provide information on controllable factors appropriate for informed decision-making such that it leads to positive outcomes.

�� Good KPIs drive change but do not prescribe a course of action.

Purpose of a KPI


�� What gets measured gets done!

�� You never really understand anything fully unless it can be measured.

�� If the goal of a performance measurement system is to improve efficiency and effectiveness, then the KPI must reflect controllable factors.

�� There is no point in measuring a KPI if the users cannot change the outcome.

The Need for Better KPIs


�� Key = a major contributor to success or failure

�� Performance = measurable, quantifiable, adjustable and controllable elements

�� Indicators = reasonable representation of present and future performance

Dissecting the KPIs


�� KPIs can be selected using the following:

�� Predictive: able to predict the future of this trend �� Measurable: can be expressed quantitatively �� Actionable: triggers changes that may be necessary �� Relevant: the KPI is directly related to the success or

failure of the project �� Automated: reporting minimizes the chance of human

error �� Few in number: only what is necessary

Defining a KPI


Variable Business/Financial Project Focus Financial measurement Project performance

Intent Meeting strategic goals Meeting project objectives, milestones and deliverables

Reporting Monthly or quarterly Real-time data

Items to be looked at Profitability, market share, repeat business, number of

new customers, etc…

Adherence to competing constraints, validation and verification of performance

Length of use Decades of even longer Life of the project

Use of the data Information flow and changes to the strategy

Corrective action to maintain baselines

Target audience Executive management Stakeholders and working levels

Business vs. Project KPIs


�� Measurements that lead to motivation of the team

�� Measurements that lead to compliance with use of organizational process assets and alignment to business objectives

�� Measurements that lead to performance improvements and the capturing of lessons learned and best practices

High-Level Purposes of a KPI


�� Quantitative KPIs: numerical values

�� Practical KPIs: interfacing with company processes

�� Directional KPIs: getting better or worse

�� Actionable KPIs: effect change

�� Financial KPIs: performance measurements

�� Lagging KPIs: measure past performance

�� Leading KPIs: measure drivers for future performance

Categories of KPIs


�� Many things are measurable but not key to the project’s success. KPIs are key metrics.

�� It is important that the number of KPIs be limited so that everyone is focused on the same KPIs and understands them.

�� Too many KPIs may distract the project team from what is really important.

Improper Selection of a KPI


�� Too many KPIs can slow down projects due to excessive measurement and reporting requirements.

�� Too many metrics/KPIs can also blur one’s vision on actual performance. We tend to create too many rather than too few metrics.

�� Too few critical KPIs can likewise cause delays because of the lack of critical information.

Risks in the Number of KPIs


�� Anything can be measured but perfect measurements may be unrealistic. Therefore, it may be impossible to select a perfect set of KPIs.

KPI Measurement


�� Percent of work packages adhering to the schedule �� Percent of work packages adhering to the budget �� Number of assigned resources versus planned resources �� Percent of actual versus planned baselines completed to date �� Percent of actual versus planned best practices used �� Project complexity factor �� Customer satisfaction ratings �� Number of critical assumptions made �� Percent of critical assumptions that have changed �� Number of cost revisions �� Number of schedule revisions �� Number of scope change review meetings �� Number of critical constraints �� Percent of work packages with a critical risk designation �� Net operating margins

Typical KPI


SYMBOL MEANING

Data not entered

Dissatisfied

Satisfied

Delighted (©2010 MahindraSatyam. All Rights Reserved.)

MahindraSatyam’s Customer Delight Index (CDI)


�� It may be difficult to get customer and stakeholder agreement on the KPI

�� Must determine if the KPI data is in the system or needs to be collected

�� Must determine the cost, complexity, and timing for obtaining the data

�� May have to consider the risks of information system changes and/or obsolescence that can impact KPI data collection over the life of the project

Complexities with Identifying KPI


�� Dashboard designers must understand:

�� The end user’s needs �� How the dashboard will be used �� How the measurements will be made �� How often the measurements will be made �� How the dashboard will be updated �� How to maintain uniformity in design, if possible

Dashboards


�� Dashboards are communication tools to provide information at a glance �� Dashboard design begins with an understanding of the user’s needs

�� Dashboard design can be done with simple displays

�� Dashboard design can be done with simple tools

�� Use the fewest metrics necessary

�� Determine the fewest metrics that can be retained in memory

�� Using too many colors or sophisticated, complex metaphors leads to distractions

�� Limit metrics to a single screen

�� Perfection in design can never be achieved

�� Asking for assistance with the design effort is not an embarrassment

�� Monitor the health and user friendliness of the dashboard

Understanding Dashboards


Dashboard Dashboard Design


�� Factors that must be considered include:

�� Colors �� Positioning �� Brightness �� Orientation �� Saturation �� Size �� Texture �� Shape

Dashboard Design and Layout


�� Some rules exist for dashboard design and layout:

�� Rules for selecting the right artwork �� Rules for screen real estate �� Rules for artwork placement �� Rules for color selection �� Rules for accuracy of information

Dashboard Design and Layout


Rules of Colors


Standard Colors Emphasis Colors Rules of Colors

©2010 International Institute for Learning, Inc., All rights reserved. 53 Intelligence, Integrity and Innovation | www.iil.com | ©2010 International Institute for Learning, Inc., All rights reserved.

Documents

2010 IPM Day Metrics Kerzner