2017

Public Service Reform

Programme Office

Benefits Identification

and Tracking Guide

1

Introduction

This guide has been designed to support people leading change initiatives in the Education and

Training Sector to identify potential benefits, and to put

in place a process for measuring and tracking benefits

during and after the planning, development and

implementation of a change initiative.

Identifying and tracking the benefits that will be realised

by the Department of Education and Skills, the sector,

learners or other stakeholders as a result of a change

initiative is a key element of change management. Most

change management projects and programmes involve a

range of stakeholders working together to bring about a

change or an improvement. Almost all are relatively

complex in nature and require substantial investment of

time and money. It is important from the outset of any

change initiative to identify (and quantify where

possible) the potential benefits to one or more of the key

stakeholders.

The early planning process considers how the proposed

change initiative fits with strategic directions and

objectives articulated in the Education and Skills

Statement of Strategy while the benefits identification

process helps to inform the decision to invest or not in a particular initiative.

A benefit is “the measurable improvement resulting from an outcome which is perceived as an

advantage by a stakeholder” (Managing Successful Programmes).

Benefits Realisation is a more complex process and includes offsetting the investment against the

outcomes and benefits as part of post programme or project evaluation. Benefits Realisation is a key

element associated with the Public Spending Code and takes into account not only the benefits of a

particular change project or programme but the associated costs of implementation and other

environmental factors. See appendix 1.

This Guide and Benefits Register concentrates on benefits identification and the establishment of a tracking process. These processes are an integral part of the project management approach being introduced by the Public Service Reform Programme Office in the Department of Education and Skills.

Background The Public Service Reform Programme Office was established in the Department of Education and

Skills in late 2013. One of the objectives of the Office is to encourage the use of programme and

2

project management structures and governance to manage the planning and oversee the

implementation of the extensive change management programme.

Projects are usually employed when we want to do something outside of the day to day operations

or business as usual. Projects are designed to bring about a change or to improve something. Some

typical projects might include:

Improving a process and/or procedure;

Developing a new policy;

Implementation of a new policy (preparing for implementation);

Design and development of an ICT solution;

Designing a new organisational structure;

Mapping provision to improve our strategic approach to funding;

Developing a communications plan;

Changing a resource allocation model; or

Publishing a customer charter.

The project process can be used to define an altered state which we wish to achieve. The project

plan identifies the steps we need to take to bring about this altered state and the people and other

resources we need to involve. The plan is also used to monitor actual progress against the plan, high

lighting variances and so guiding corrective actions that may be required to deliver the project. By

definition projects should bring about an improvement i.e. there should be some identifiable and

measureable Benefit to at least one stakeholder for every project.

Once we begin to identify the change or improvement that we are planning we are beginning to

identify the possible Benefits that a project may bring either for our own organisation or for other

key stakeholders. This process can start with a few simple questions:

Why are we doing the project? What change are we attempting to bring about?

What are we going to get out of it? What are the improvements we are going to make?

Will other people see improvements as a result of the successful execution of this project?

With these few simple questions we are beginning to implement a Benefits realisation process.

3

Where the Benefits Management process fits Benefits Identification and Tracking is not a stand-alone process. It is an integral part of programme

and project management. Not all activities are projects but where a programme or project has been

identified the PSR Programme Office recommends that it should have a project plan. The benefits

we hope to achieve should be identified as part of the project planning process.

The PSR Programme Office templates associated with project planning and implementation are

available on request.

Why Track Benefits

Build a business case and weigh up the different options

Focus attention on the most important benefits to the organisation or the end user

Focus attention on the change to be brought about and get people thinking about the final

outcome

Anticipate dis-benefits and put plans in place to manage them

Increase the understanding of the change to be brought about by providing measures for

the planned changes

Provide information for post programme/project review and evaluation

Demonstrate the tangible impact of the project and support statement of return on

investment

The methodology within this guide The following is a guide to a simple methodology for identifying and tracking benefits that can be

applied to projects across the Department of Education and Skills, its agencies and the sector. This

guide concentrates on benefits and is a first step in standardising the approach to identifying,

tracking and reporting benefits across different projects. The guide concentrates attention at the

level of the individual project. If you are managing a series of projects or a programme, the benefits

register can be used to aggregate the benefits from across a group of projects.

It is worth mentioning at this early stage that we should also identify any possible dis-benefits. A dis-

benefit is where the change we are working to bring about has a negative impact on one or more

stakeholders. Sometimes we will proceed with a project not because there are no dis-benefits but

because the benefits out-weigh the dis-benefits.

The guide has been compiled by the Programme Manager in the Department of Education and Skills’

Public Service Reform Programme Office and is based on current best practice. A list of references

and websites that have been consulted in the preparation of this Guide are included in Appendix 5

for those who wish to pursue this topic in more detail.

4

Success factors

Identify potential benefits as early as possible in the project planning phase, ideally during

the development of the Business Case or, where a formal Business Case is not required, in

the Project Plan Template.

Ensure that project benefits are clear, concise and add value. Ensure that the benefits

identified relate to the organisation’s goals and objectives.

Make sure that for every benefit identified there is a named individual who will be

responsible for that particular benefit and a named individual with responsibility for

ensuring that the benefit is measured at intervals agreed in the Benefits Register (these

people may be known as the Benefit Owner and the Benefit Tracker, they may, but are not

necessarily, be the same person)

Measure benefits before, during and after the project. Benefits should be measured a)

during the planning phase (baseline), b) during the project lifecycle, c) after the project

closes and d) when the product of the project have been handed back to the business

owners.

The Benefits Register should be reviewed regularly and revised as necessary. Revisions might

include a) adding new benefits, b) deleting benefits that are no longer relevant, c) adjusting

targets

Use existing project governance arrangements such as the project steering group or

programme board to monitor, track and report on benefits.

What is a project?

A distinct piece of work, that has a beginning, middle and end that is designed to bring about

an improvement or a change

Projects are temporary (not necessarily short) in nature and non-routine in that they are

different from the day to day process or operation

Projects have a defined scope and set of deliverables which once achieved will mean the

project will close/finish

Projects are usually focused on a singular goal and have a clearly defined

purpose

A quick guide to success

5

The Process Key Stages Overview The Benefits management process contains 12 sequential steps divided into 5 process areas as

outlined below:

Each of these steps will now be explained in detail:

6

The Process Key Stages Detail

Identification

Ideally, the work of identifying benefits should be done with a group of stakeholders, early in the

planning process. For most projects this activity will take place during the development of the formal

Business Case or when compiling and agreeing the project plan. Introducing benefits identification

before a final decision on the project methodology or the nature of the intervention can help to

ensure that that the intervention that best achieves the benefits is adopted. It is important to agree

up front and early what should be measured. If you think that benefits are going to be measured

during the life of the project then it would also be advisable to agree with the stakeholder group

when these measures will take place. These measurement points can then be added to the project

plan and included, perhaps as milestones, in the project schedule (tasks and timeframes).

The Business Case or project plan will state clearly the purpose and objectives of the proposed

project and how the project is expected to contribute to the organisational strategy or change

programme. Early planning work also identifies the project deliverables. From this work it should be

possible to identify the overarching, high-level benefits that should result from the project.

If you are involved in an initiative that is already up and running but where no distinct benefits were

identified of specified as part of the business case or the planning process it should still be possible

to identify benefits from the existing documentation. What were the original objectives? What

change is to result from the implementation of the change initiative? It should be possible to work

back and retrospectively identify a set of high-level benefits.

7

Summary of Benefits Identification

Strategic Objective →→

Business Benefit →→

How will we measure or demonstrate that the business benefit has occurred? →→

What metric will we use? →→

Where will get the information from?

From the organisation’s Strategic Objectives

High level benefit that will result from the project

How will the cost be reduced? How will efficiency be improved? How will we know learners are more proficient in reading?

EXAMPLES

Focus on Core Mission: Introduce Shared Services

Reduced Cost Increased Efficiency Improved Customer Service

Number of people Volume of transactions Dedicated Customer service unit

No of FTEs No of transactions per FTE % Calls closed in less than 24 hours

Payroll Performance Indicator for Shared Service Centre Performance Indicator for Shared Service Centre

Improve learning outcomes for all learners

Increase level of reading proficiency

Targeted results in standardised tests

% of learners with ≥ level x reading proficiency

Results of standardised test

Step 1 Identify high level or Business Benefits from the

organisational level strategic objectives

The first step in the Benefits Management process is to identify the high-

level benefits relating to the purpose, objectives and deliverables of the project.

These benefits should directly relate to the organisational strategy or objectives of the change

programme. Think about outcomes. What are the benefits to the business that will be achieved

through the change being instigated? Whether or not all of these benefits will materialise is

dependent on how the project progresses, at this stage you are identifying the potential benefits.

Answering the following questions may help:

Why are we doing the project? What are the business drivers?

What are we going to get out of it? What are the strategic outcomes of the project?

What are the measurable benefits? And when might we obtain them?

8

In identifying high-level benefits it may be useful to look at the different areas a benefit might relate

to and to consider including benefits from more than one area (balanced score card approach). For

example benefits may relate to:

Policy or legal requirement (mandatory), the benefit is in enabling the organisation to fulfil a

policy objective or to comply with a legal requirement i.e. where there is no choice.

Quality of service, the benefit is to the customer (internal or external) often related to

efficiencies where a customer can get the type of information in a format that suits them or

perhaps where they can get a response to a query faster.

Process improvement (productivity or efficiency), may relate to internal processes to do with

improving decision making or management processes.

HR/Personnel, the benefit of a better motivated workforce that may lead to a number of

other benefits such as flexibility or increased productivity.

Revenue enhancement or acceleration, benefits that relate to reducing costs while

maintaining quality.

Examples of high-level benefits might be:

Increased level of customer satisfaction

Improved Quality of Service

Increased efficiency

Maximising Value for the Taxpayer

Increased stability to minimise risk of system failure

Compliance with new regulation on x or y

Increased staff motivation

Improved connectivity for schools

Increase level of reading proficiency

Step 2 Use the list of high-level benefits to identify the actual `

benefits that we will measure

Having agreed a set of high-level benefits the next step is to ask how will

we measure these benefits? What are the changes that the project will

bring about that will lead to measurable benefits? For some projects the measureable benefits will

be easy to identify. However for more complex projects there is a tool called a Benefits Map that can

help to structure this exercise. An outline and worked example of a Benefits Map is included in

Appendix 2.

Start by listing the high-level benefits. For example, a shared service project might have 3 key high-

level benefits:

Reduced cost

Increased efficiency

Improved service

For a more policy related project the benefits might be

Increased levels of reading proficiency in primary school children

Increased level of integration of ICT in teaching and learning

9

Take each one of these high-level benefits and ask the question “how can we measure the

achievement of these benefits”? For example to measure a reduction in cost we might look at how

the costs will be reduced:

Reduced cost per payee

Reduced number of staff employed in payroll

Reduced cost of technology (licensing/maintenance)

Policy related benefits may be harder to measure but might include:

Percentage of 8 year olds with level x or higher proficiency in reading

Percentage of teachers reporting daily use of ICT in the classroom

For “increase efficiency” we might ask “how will efficiency be improved?” this might include:

Increase the volume processed by each FTE (number of transactions processed per FTE)

Decrease cycle time (improved processing time)

Staff freed up to work on core mission activities

And for “improve service” we could ask “how will we know that service has improved”? This could

include:

Increased access to customer service

Decrease in response time

When identifying benefits in the early stages of project planning the Benefits Map can be used to

look at the features of the new system/policy which will underpin or enable the benefits. This

exercise can help in identifying or prioritising the development of different features or policy

elements. Again, using the shared services example, the features might include:

100% online processing

Standardised process

Dedicated customer service team

On-line customer support team

Single ICT platform and common operating system

Step 3 Prioritise and select the Benefits that you are going to

measure.

Be careful not to include too many Benefits. Remember, there can be quite

a lot of work associated with the baselining and measurement of each Benefit. Ranking (rank the

Benefits in order of priority based on how critical they are to the success of the project) may help

with this step. There is no hard and fast rule about how many Benefits to include but for smaller

projects one or two key Benefits may be all that is needed. Larger projects may include a greater

number but baselining and tracking more than 10 to 15 benefits could lead to a situation where the

time, effort and resources involved in measuring the benefit negatively impacts on the project

implementation or delivery. The number of metrics should be proportionate to the size, type and

duration of the project.

10

Step 4 Determine Benefit Owners and Trackers

At this point in the process you have selected a list of key Benefits that you

are going to measure before, during and after the life of the project. This

step in the process begins to populate the Benefits Register (Appendix 3) for your particular project.

Working with the project team and key stakeholders begin on the “Project Summary Sheet” and

complete the following fields:

Please note that for the purpose of this guide the Benefits register is being completed at the level of

the individual project. Programme Managers may also create programme level registers which will

aggregate information from the project registers.

Project Information: Project Title, project manager’s name, key executive/sponsor’s name, dates of project commencement and forecasted end date.

Benefit Number/Code: if your project fits within a bigger programme with other projects check with the programme manager if they are applying a particular numbering or coding system to the Benefits process. If not, keep it simple and apply a unique identifier to each Benefit.

The Benefit description: this should be a summary statement of the Benefit you are going to

measure - for example: decrease in volume of complaints; reduced turnaround time for

queries; increased volume of invoices processes per hour; reduction in number of FTE posts

operating y; reduced cost of running z; reduced time for end to end processing.

o If It is a dis-benefit it might be increased cost of x; impact on broadband capacity;

delay in introducing related system.

Benefit owner name. Who is responsible for overseeing the identification and tracking of the

Benefit? This person may not be directly involved in the activities associated with the Benefit

but it is their responsibility to ensure that the processes is implemented. Only one name

should appear as the sole owner. You may also wish to enter their position and contact

details.

o The Benefit Owner is often the project manager but this will change when the

project closes and the responsibility for the Benefits transfers from the project team

to the business.

Benefit Manager/Tracker name. Who is responsible for actually measuring and tracking the

Benefit? To measure a Benefit often data will need to be drawn from different sources. It is

the Benefit Manger/Tracker who is responsible for ensuring that the data is gathered and

recorded.

The same should be done for dis-benefits.

11

Step 5 Identify the Benefit Type.

You are asked to state (using the drop down menu in the Benefits Register) if

the Benefit is Financial or Non-Financial. As a general guide financial Benefits

are those that are realisable as cash - for example: reduction in pay bill

associated with a reduction in the number of FTEs involved in a process; avoided cost of ICT;

elimination of IT system hosting/support costs; increased uptime on an online system etc. Non-

financial Benefits are a little more difficult to quantify but might include levels of customer

satisfaction; organisation’s reputation; employee satisfaction, level of engagement/motivation etc.

Benefit Type Examples

Financial or realisable as cash

Increased uptime on online purchases of services Reduction in pay bill (number of FTEs) Avoided cost of ICT upgrade and other avoided costs Eliminated IT system hosting/support costs Reduction in cost of maintenance contracts Decrease in unit costs Increase in discounts available Decrease in deliver and handling costs

Non-financial often not realisable as cash and more difficult to quantify

Increased staff engagement and/or motivation Employee/workforce satisfaction Customer Service Reputation Increase in enrolments Increased levels of literacy and numeracy Labour market activation interventions Compliance Faster processing time resulting from reduction in manual processes Increased flexibility in responding to customer/client needs Enabling staff redeployment Risk reduction Increased safety and security Improved warranties Reduced administrative burden

Step 6 Identify how each Benefit will be measured (identify the

metrics).

How can the Benefit actually be measured, i.e. what metric will be used?

The important thing here is not to create too many metrics that become difficult to manage. Keep it

simple. Keeping in mind the Benefit category and type, what is the simplest measure to apply to a

particular Benefit? One measure per Benefit is probably adequate in most situations.

Some sample metrics are included in the table below. Make sure the metric you chose is appropriate

to the Benefit. For example if the Benefit is a reduction in the number of complaints received the

metric may simply be the number of complaints recorded per hour/day/week. If the Benefit is a

reduction in transaction time the metric could be the time taken from beginning to end of the

12

transaction. If the Benefit is an increase in customer satisfaction levels the metric may be the

number of people scoring the organisation above 8 out of 10 on the monthly/annual customer

satisfaction survey. The more tightly defined the Benefit the easier it is to identify the appropriate

metric.

The metric must also be reliable i.e. it will give you the same answer no matter who is calculating it.

The process of collecting the data relating to the measure must be standardised to ensure that the

data is collected in exactly the same way across single or multiple sites. It is the Benefit

Manager/Trackers job to ensure that this is the case.

Metrics can be categorised as being system metrics, people metrics or process metrics. The Benefits

Register doesn’t ask you to categorise your Benefits but it may be useful to consider whether you

have a good balance between system, people and process metrics for your project - keeping in mind

that you don’t need to have a very large number of metrics, particularly in smaller projects.

Categorisation is more important for programme managers who need to ensure a mix of Benefit

type and category across the different projects in a programme.

Some examples of metrics:

Examples of metrics:

People Number of FTEs required to run the process Staff satisfaction rating Salary costs

System Cost of x as a % of the operating budget Annual cost of software Annual cost of hardware End to end cycle time Number of legacy systems still in operation Post implementation licencing and support costs (Cost of licencing and System support as a % of total ICT cost)

Process Cost per transaction Transactional Processing time Number of queries per cycle Cycle time to hire new employee Number of calls processes per hour/day

Data Source

Where will the data come from each time we measure the Benefit? Where possible this should be

from an existing source for example an annual staff survey, financial statements etc. If the

information or source of the information will involve extensive calculations and gathering of

information from a variety of sources make sure that it is the most appropriate metric for measuring

the Benefit and consider how often it needs to be measured and the cost/impact involved.

13

Step 7 Measure the baseline.

Ideally Benefits will be measured before, during, and after the life of the

project. In order to create a benchmark against which to measure progress

towards the intended target, it is advisable that a baseline measurement is taken before the change

project commences. A baseline measurement is the measure of the current situation. For example, if

the Benefit is to reduce the number of calls processed per day, the baseline measure will be the

number of calls processed currently. The baseline measures is sometimes referred to as the “as is”.

There are different ways of measuring a baseline some of which are outlined in appendix 4.

If a baseline measurement cannot be taken before the project commences it should be taken as

early as possible after the project commences. Once the baseline figure has been agreed, add the

date and the measurement to the Baseline and Measurements section of the Benefits Register and

update the “status” column to state that this is the baseline stage.

Where is it not possible to measure a baseline, you can define appropriate and assumed baseline

values through close collaboration with key stakeholders. This may be a useful method where no

original baseline was recorded.

The baseline or “as is” figure may be a separate measurement undertaken as part of the baselining

phase of the project or it may come from existing financial data, survey, historical record or work

14

study etc. Any assumptions that are made in calculating the original baseline figure or any formula

used to cost a particular element should be documented and noted in the Benefits Register.

Example of formula used to calculate the cost of an FTE.

Total staff costs used for the baselining exercise and business case were calculated based

on the following:

Formula

Total Cost = (A) Staff FTE Cost +

(B) 10.75% Employer’s PRSI +

(C) 7% Pension Costs +

(D) 25% FTE Salary Uplift +

(E) Unique Payroll Overhead Costs +

(F) ICT Costs

Calculation of FTE costs

Total Staff Cost = ∑ [(Average Gross Salary Cost of each FTE x time spent on payroll

activity)*1.42751]

Average Gross Salary: This figure was worked out for each grade based on the current

salaries Circular issued by the Department of Public Expenditure and Reform. In order to

take into consideration the age profile of public servants, the salary cost used for the

business case was weighted 60:40 between the highest and the median salary costs per

grade.

25% Uplift: The Central Expenditure Evaluation Unit, Department of Public Expenditure

and Reform, estimates that an addition of 25% to gross salary cost is appropriate to reflect

overhead costs applicable to the generality of civil service situations for accommodation,

utilities, support and back-office staff, training, travel, and similar expenses.

Source: Shared Services Payroll Business Case ETB Sector

15

Step 8 Set a target for the Benefit.

What is the new state that you wish to bring about? You are asked to set

a target which should identify the specific, planned level of result to be

achieved within a stated timeframe. Targets may be expressed as quantity (now much), quality (how

good) or efficiency (least cost). The target represents the results that the programme or change

initiative is to bring about. Targets can be quantitative or qualitative. For example if you are

measuring the number of complaints received per week and you currently (baseline/as is) process

100 calls per week, what is the number of calls per week that you want to achieve through your

project? It may be 50 (reduction of 50%) or it may be 10 (reduction of 90%). Qualitative targets are

more descriptive and might include improvements in management practise or a list of the new

functions that a new system will introduce. For example, the target for the introduction of a new

online reporting function might be that by a named date 70% of all internal reports would be logged

online. Qualitative targets can include quality measures, levels of customer satisfaction, enhancing

the Department’s reputation etc.

This figure or new state which you hope to achieve is known as the “target”.

Working with the Benefit owner or other key stakeholders, identify an agreed target for the Benefit

and the date by which this target should be realised. Some Benefits may be quite immediate and

may be realised during the life of the project while others may take years before they are fully

realised. In the case of the latter it is the “business side” which must take over responsibility for

measuring and realising the longer term Benefits. “End Measurement Period” in the Benefits register

should be the date in the future at which Benefits measurement will cease.

In general the “target” set is the end point target. If a projects business case has identified

intermediary targets (targets set at intervals) the Benefits Register can be tailored by adding

columns to accommodate these intermediary targets. Remember also to adjust the “variance”

columns to show variance against intermediary target and variance against final target.

The targets set should be realistic in terms of being challenging but achievable within the resources

available. The targets will form the Key Performance Indicators (KPIs) for the project. Monitoring

during the project may show movement towards the achievement of the target or anticipated

benefit. For projects where the targets will not be achieved during the life of the project, these KPIs

transfer to post-project implementation phase. For example a shared services target set during the

project phase informs or becomes a KPI in the Shared Service Centre’s continuous improvement

process.

16

Step 9 Establish a monitoring and tracking process.

Agree what reports will be required and in what format and how

frequently will the Benefits need to be measured. When Programmes have

a range of different projects, it may be prudent and practical to agree an end year date for reporting

Benefits on all the different projects. This should be managed by the Programme Manager. It is

important to identify when you will stop measuring a particular Benefit.

When deciding the frequency of measurement for your project keep in mind the level of work

associated with the measurement activity and try to ensure that the measurement doesn’t become

over burdensome.

Step 10 Set a date for the first measurement for each Benefit.

Some Benefits will be realised quite quickly, while others may take years.

Use the column in the Benefits register to record the date of the first

intended measurement. Depending on the nature of the project and the associated Benefit, the first

measurement may be scheduled to take place during the life of the project. However for most small

scale projects Benefits will begin to occur (or be realised) after the project has closed and when the

new policy or process has been transferred back to the business unit with responsibility for

implementation. Complete the Register to include how often and at what intervals (every 6 months,

annually, bi-annually) subsequent measure will be required. This is important as you, as the project

manager, may not be the person responsible for organising these subsequent measurements.

At this point you should move from the Project Summary Sheet and create an Individual Benefit Sheet

for each Benefit that you wish to track.

Step 11 Carry out the initial and subsequent measurements and

report.

When the time comes to take the first (and subsequent) measurements

the person with responsibly for tracking the Benefits organises the

collection of data from the identified source. In some cases this will be a very simple exercise as the

data associated with the metric will already exist. However for other projects the measurement

activity will require a separate measurement process. Check the metric to be used and the data

source identified in the Register. Remember it is important that the metric is reliable i.e. it will give

you the same answer no matter who is calculating it. The process for collecting the data relating to

the measure should have been standardised when the metric was first chosen to ensure that the

data is collected in exactly the same way across single or multiple sites. It is the Benefit

Manager/Tracker’s job to ensure that this is the case.

17

Add the resulting measurement and date to the Benefits Register. For Non-financial Benefits it may

be appropriate to indicate the nature of the value i.e. Number of FTEs, % satisfaction rating etc.

Complete the “variance” field indicating the difference between the current measure and the target.

Use the Benefits Register to report to the steering group. At this stage you should also add any

information to explain the findings or the metric used. The two columns “Explanation” and “Actions”

should be completed before the steering group meeting and amended following discussion i.e. after

step 12.

Analyse the metrics and identify follow up activities

The steering group will discuss the measure, variance, metrics and targets. Following this discussion

they may decide that a change to the metric or measurement process is required or that more detail

is required to explain the measures/variance etc. The steering group may also:

Decide to adjust targets if appropriate

Check priorities and targets are still valid

Add any project level risks to the project risk register

Escalate any programme level risks or issues to the appropriate level of programme

governance (programme manager, steering group, programme board, reform board)

Set date for next measurement and review

Handover Benefits reporting to the business once the project is closed.

In many instances the Benefits will continue to be realised (and therefore should continue to be

measured) after the life of the project. Named individuals within the business should be responsible

for continuing to measure and report on the Benefits. Names should be recorded in the Benefits

register and in the project close document to ensure that there is clarity about who is responsible for

this process after the project closes.

For shared services projects Benefits are incorporated into Key Performance Indicators (KPIs) and

become an integral part of the continuing improvement processes associated with operating the

Shared Service Centre.

18

Step 12 After the Benefits have been realised, evaluate the

collective impact or success of the project as a whole.

For large scale projects an evaluation process forms part of the requirements of the Public Spending

Code, and the evaluation of the Benefits of a particular project will be done as part of the economic

evaluation where costs are compared to Benefits. Economic evaluations are key in measuring the

effectiveness of a project in addition to measuring the difference between the baselining the final

measurement after implementation, economic evaluations also look at what would have happened

in the absence of the project etc.

For smaller projects it is advisable that the Benefits realised should be reviewed against the original

business case or project plan and a final report should be prepared including:

Return on investment

Planned cost Benefit versus actual cost Benefit

Lessons learned

Opportunities for further yield

It would also be useful to include in the final evaluation report any lessons learned that could be

incorporated into this guide and/or the wider project management process.

19

Process Stage Checklist

Have you: Project Planning or Business Case

Identified a set of high-level benefits aligned to the organisations strategic objectives? Used the benefits map to identify which benefits it is appropriate to measure? Identified any dis-benefits? Prioritised the benefits and selected a small number that will be measured during and after the change initiative/project? Identified who is responsible for identifying and tracking the benefits? Added this information to the Benefits Register?

Identified the Benefit type? is it financial or non-financial Identified how each benefit will be measured? Where will the information come from? What metric will be used? Will the information be easy to collect?

Taken a baseline measurement? Have you used a point in time or an average or rolling average to measure the baseline? Set a target for each benefit to be measured against? Agreed the targets with the key stakeholders? Ensured that the targets set are realistic but challenging? Added the baseline measurement and the targets to the Benefits Register?

During the lifecycle of the project or change initiative

Established a process to track and monitor benefits that may occur during the life of the project? Set a date for the first and subsequent measurements to take place? Named the individual with responsibility for collecting the data and completing the Register? Carried out the initial measurements and recorded them in the benefits register? Revised the metrics/measures/targets to take account of learning from the first round of measurements? Added new benefits or dis-benefits to be baselined and measured?

After the project has closed and responsibility for ongoing implementation has transferred back to the business

Named the individual with responsibly for collecting the data and completing the Register (no longer a responsibility of the project team). Added any explanations/formulae etc. to the Benefits Register? Revised the metrics/measures/targets to take account of learning from this round of measurements? Evaluated the project against the original business case and examined return on investment, planned cost benefit versus actual cost benefit, lessons learned and opportunities for further yield?

20

Appendix 1: Public Spending Code

The Public Spending Code requires that all planned public spending adheres to a process that

includes appraisal (pre-spend), planning and design, implementation and post project evaluation.

The scope and breadth of the different appraisal, review and evaluation processes associated with

the Public Spending Code are scaled to reflect the size, spend and complexity of the different

projects. Simple assessments are required for projects with spending up to 500,000 euro. Projects

with a cost of .5 to 5m euro are expected to carry out pre-spend appraisal which looks at the

different options and their associated risks and costs. Larger scale projects are subject to more

detailed appraisal and evaluation. Spending in the region of 5 to 20m euro is subject to multi-criteria

analysis. Spending over 20m (capital and new current with annual spend of 5m+) requires full

economic appraisal such as cost benefit analysis or cost effectiveness analysis to be submitted to the

Central Expenditure Evaluation Unit (CEEU) in DPER before “approval in principle” is granted. An

overview of these appraisal and evaluation methods is included in the Public Spending Code.

The Public Spending Code requires that costs and benefits are identified in the Appraisal and

Planning and Design phases and are monitored throughout the project lifecycle. Details of the Public

Spending Code and the associated appraisal and evaluation methodologies can be found on

www.publicspendingcode.per.gov.ie

21

Appendix 2: Benefits Map Template

1. First ask the question: ‘what are the strategic or programme objectives?’ Strategic / organisational level Benefits are generally aligned with organisational strategy and corporate plans. Strategic Objectives for the project or programme usually describe what the organisation is seeking to achieve as a result of the business changes, or from a change/reform initiative.

2. Next, break the identified investment objectives down into business benefits (high level and intermediate if required) i.e. if these objectives were achieved, what would be the resultant benefits in the organisation or sector? These Benefits describe the actual operational improvement resulting from the project e.g. quicker access to information, improved financial management, faster turnaround times etc.

3. What evidence will prove that the business benefit has been achieved? It is these functional or operational Benefits that will be measured. Don’t include benefits that cannot be measured or that cannot be directly attributable to the changes being brought about by the project. Measurable dis-benefits should also be included at this stage.

4. It can help in the planning stages of a project to link the measurable benefits to the features or output of the project (features/outputs can also be described as enablers). This will also help to establish the relationship between a particular measure and a specific feature. This can help in understanding the impact on future benefits of changes to the planned features. For example a business benefit may be an increase in efficiency measured through an increase in the volume of transactions handled by an individual FTE but this efficiency may be completely depended on the project ensuring that all applications are made “on-line”.

22

Sample Shared Services Benefits Map

Focus on Core Mission: Introduce

Shared Services

Improve quality of service to

stakeholders

Improve value for money

Reduced Cost

Increased Efficiency

Improved Customer Service

Number of FTEs involved in processing

Strategic Objectives

Business Benefits

Measureable business changes

Features produced by the project that

will ENABLE the business changes

and benefits

Standardisation of processes

On line processing

Dedicated customer service unit

% Calls closed in under 24 hours

No. of transactions processed per FTE

Satisfaction level of customers surveyed

23

Appendix 3: Benefit Register: Project Summary Sheet and Individual Benefit Sheets. Excel templates are available on the intranet on the

Public Service Reform page

Project Summary Sheet

Project Title

Project Manager

Key Executive/sponsor

Date of Project Commencement

Benefits (Expand no. of rows as necessary) Forecast date of Project completion

Benefit No. /Code

Benefit description

Benefit Owner

Benefit Manager/Tracker

Benefit Type

Method of measurement/Metric

Data Source

Baseline measurement Target

Measurement frequency

End of measurement period

1 Financial

Dis-benefits (Expand No. of rows as necessary)

Dis-benefit No. /Code

Dis-benefit description

Dis-benefit Owner

Dis-benefit Manager/Tracker

Dis-benefit Type

Method of measurement/Metric

Data Source

Baseline measurement Target

Measurement frequency

End of measurement period

1 Financial

24

Individual Benefit Record Sheet Excel templates are available on the intranet on the Public Service Reform page

(Please maintain one sheet for each benefit identified)

Project Title 0

Project Manager 0

Benefit Owner 0

Benefit Manager/Tracker

0

Benefit Description 0

Benefit Code 1

Measurement Frequency 0

End of Measurement Period

00/01/1900

(Expand table as required)

Baseline and measurements

Measurement record Date Measure (Value) Target (Value) Variance Explanation Action required and by whom

Baseline 0

Measurement 1 0

Measurement 2 0

25

Individual Dis-benefit Record Sheet

Excel templates are available on the intranet on the Public Service Reform page (Please maintain one sheet for each benefit identified)

Project Title 0

Project Manager 0

Dis-benefit Owner 0

Dis-benefit Manager/Tracker

0

Dis-benefit Description

0

Dis-benefit Code 1

Measurement Frequency

0 End of Measurement Period

00/01/1900

(Expand table as required)

Baseline and measurements

Measurement record Date Measure (Value) Target (Value) Variance Explanation Action required and by whom

0

Measurement 1 0

Measurement 2 0

26

Appendix 4: Baseline

How do I measure a baseline?

Measurements from time and motion study (not recommended as it can be intrusive and individual

being timed can feel threatened or motivated to work faster to demonstrate their efficiency but this

can introduce bias or skewed results).

A time and motion study is typically used when the organisation is trying to establish the time taken

by an individual or a group of individuals to complete a set task. You deconstruct the task and

measure the time it takes to complete each component part of the overall process. You can then add

these tighter to establish how long it takes to complete the full process

Point in time measurement

Point in time measurement is achieved by taking the value measured at a certain moment in time as

being representative of a baseline value. Although this method of baseline assessment it is

acceptable, it is not the most accurate.

Average over a nominated time period

Data gathered over a period of time is likely to provide a better representative sample of a baseline

than that taken at a specific point in time. However, raw data should be available to determine the

time based averages and the time period for the baseline measurement should be agreed.

Rolling average measurement

Rolling averages use data from the most recent historical time period to determine baseline average

values. The average value is determined for the same fixed time span but the time spans from x

months prior to the latest period.

If recording or data collection systems are not in place to determine historical baseline values, the project manager may plan and design new processes to measure the data. Key considerations are: What data do we need? Will data need to be measured over a period of time? If so what period? How frequently will measurements be required? Will the design and implementation of these new processes have a cost to the project?

Ensure that all assumptions and formula used in calculating the Baseline are recorded in the Benefits Register.

27

Appendix 5: Some Further Reading and References used in the development of this

Guide

Department of Finance (Northern Ireland). Project Management Templates (Belfast: NICS Centre of

Excellence for Delivery) <https://www.finance-ni.gov.uk/publications/project-management-

templates> [accessed 14 June 2017].

Finance, Service and Innovation (New South Wales). Benefits Realisation Management Framework

<https://www.finance.nsw.gov.au/publication-and-resources/benefits-realisation-management-

framework> [accessed 14 June 2017].

Benefits Realisation Planning, PwC

USAID Centre for Development Information and Evaluation. “Establishing Performance Targets”,

Performance Monitoring and Evaluation, Number 8 (1996).

Department of Public Spending and Reform. Public Spending Code (Dublin: Central Expenditure

Evaluation Unit, 2012) <http://publicspendingcode.per.gov.ie/> [accessed 14 June 2017].

Byatt, Gareth & Hodgkinson, Jeff. “Realising Benefits – It’s what projects are for!”. PM Times (16 May

2012) < https://www.projecttimes.com/articles/realizing-benefits-its-what-projects-are-for.html>

[accessed 14 June 2017].

Jenner, Stephen & Breese, Richard. “The Theoretical Basis for Benefits Management – part of the

solution or part of the problem?” Managing Benefits Series, APMG-International

<http://www.stephenjenner.com/free-materials/> [accessed 14 June 2017].