Project Success 'Orange Paper

www.acando.co.uk

Acando White Paper“ Project Success”Insight

White Paper - Project Success v2.indd 1 08/10/2012 12:37

Acando - White Paper

This research paper looks at what the key contributing factors to a successful project are and whether there are lessons we can learn from this to make our projects more successful.The research was in the form of a questionnaire sent out to over 43,000 people involved in delivering projects. There were 4,451 respondents and it is these responses that make up the material presented in this paper. There are two appendices to this paper which explain in some more detail:

1. The demographics of questionnaire respondents.

2. Statistical signifi cance testing and how it was applied to the questionnaire results.

For those who are just interested in the survey results – one important piece of advice before we dive straight in. A survey is an inexact science to understand exactly what is contributing to project success. It can give trends rather than absolute defi nition and is comparative in its nature not absolute. This piece of research does not purport to have the answers to what creates a successful project. But it does highlight common factors amongst

successful projects and trends that are exhibited more commonly across successful projects.

Without any further ado – let’s dive straight in.

What is a successful project?

Before we can understand what contributes to a successful project, it’s fi rst interesting to see what defi nitions there are of successful projects. The survey presented a list of 10 criteria that are often seen as defi nitions of a successful project. These criteria were taken from other existing surveys on project success (The Standish Group, IT Toolkit, Project Smart, The Chaos Group and Modern Analyst). Respondents were asked to rank the 10 criteria 1 through to 10 where 1 is, in their view, the most important defi nition of a successful project, 2 the next, and so on.

The graph below shows the aggregated number one choices from all respondents. There are several interesting observations in this analysis.

• Only 5% of respondents thought a successful project was one which delivers in the agreed timescales.

• Only 5% of respondents thought a successful project was one which delivers within the agreed budget.

Together these are interesting. The amount of eff ort we put in to measuring time and cost on projects is huge. But they are perceived to be poor measures of project success.


0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

...Wouldn’t have had a better outcome if we did the project again

...Was enjoyed by everyone who was on the project

...Delivers more bene�t than cost

...Delivers the expected bene�ts

...Delivers all of the objectives

...The project sponsor thought it was a success

...Delivers within agreed timescales

...Delivers within budget

...Delivers most of the objectives

What defines Project Success?

Percentage of respondents


www.acando.co.uk

• The number one measure of project success is that the project delivers the expected benefi ts.

• The number two measure of project success is that the project achieves all of the objectives.

These are also interesting when one considers how little time is spent on projects measuring benefi ts and ensuring the project delivers objectives. Perhaps we should take a long hard look at whether the objectives are ones stated in order to ensure the funding for the project, or whether the objectives are actually delivered. Almost 60% of respondents had benefi ts or objectives in their number one defi nition of a successful project.

• 18% of respondents thought the number one measure of a successful project was that the project sponsor thought the project was successful.

I question whether this is people not wanting to confront the reality of whether their projects did deliver, or whether the benefi ts are not measured (or not capable of being measured) in which case a benefi t-based analysis of success just isn’t possible. It is interesting that such a subjective measure is the third most popular number one defi nition of a successful project.

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

...Wouldn’t have had a better outcome if we did the project again









What defines Project Success?


When we expand the data (chart above) to look at each respondent’s top two factors that defi ne project success, little changes. Benefi ts are still the run-away leaders, the subjective view of the project sponsor is still in third place and time and budget are still poor cousins to any of the top three.

We should carefully consider how we measure our projects. The analysis is suggesting that benefi ts and project objectives should be carefully measured and reported on as these are the most signifi cant defi nitions of a successful project.



The chart above shows the percentage of respondents that marked “delivers the expected benefi ts” as their number one defi nition of a successful project and compares that data to the diff erent level the respondent has in the project hierarchy. It is statistically signifi cant that the higher you are up the project hierarchy, the more convinced you are that the defi nition of whether a project was successful or not is whether or not it delivered the expected benefi ts.

Interestingly, for those tasked with delivering our projects, less than 40% of them have the same number one defi nition of a successful project.

There are two hypotheses here.

1. We are poor at communicating that the most important thing to focus on in delivering any project, is that it delivers the benefi ts that are expected. This poor communication creates a diff erent focus as you move down the project hierarchy which is going to contribute to projects ‘missing the mark’ as the ‘mark’ that is being focused on is diff erent from the ‘mark’ that those are paying for the project are focusing on.

2. As you move down the project hierarchy, you become more task focused and less benefi ts focused and this is expected and acceptable. As you move down the hierarchy, you are tasked with delivering a set of actions, not a set of benefi ts and therefore you are going to have more task and less benefi t focus. This is natural and it’s the responsibility of the top of the project hierarchy to ensure the tasks deliver the benefi ts.

Personally, I prefer the fi rst hypothesis. The second one sounds too much like an excuse for me. And the second one breaks down still further when you consider that less than 50% of those managing our project delivery functions have benefi ts as their number one defi nition of success.

In order to have more successful projects, everyone has to focus more on whether the project is delivering the benefi ts that it was expected to deliver.

0% 10% 20% 30% 40% 50% 60% 70% 80%

Support

Deliver

Manage

Pay For

Delivers the Expected Benefits




www.acando.co.uk

There are some other interesting demographic variations, of which the data below is one. The chart shows the number one defi nitionof a successful project from those respondents who have delivered projects for 15 years or more. In this group of people – and only in this group of people, this phenomenon is not seen in any other group – the project sponsor’s view of the project is the most important defi nition of whether the project was successful or not. When tested, this result is statistically signifi cant.

We can draw some interesting inferences from this.

• The more experienced project managers take care of the project sponsor’s view more so than others.

• The only way to survive in project management for 15 years or more is to make project sponsors happy and therefore it’s not unexpected that those with signifi cant experience have this bias to their defi nition of a successful project. Politics is at play.

• Those who have been managing projects for a long time have come to the realisation that benefi ts are rarely measured and therefore have elected for a defi nition that they can measure in preference to what they know is the ‘better’ measure but too infrequently applied.

We can’t be sure which of these inferences is true. The data doesn’t show us that.

0% 5% 10% 15% 20% 25% 30% 35%

...Wouldn’t have had a better outcome if we did the project...

Other (please add any other success criteria in to this box)









What is project success – 15+ years of experience




The fi nal demographic diff erence is shown in the graph below. This data shows the top two responses from respondents managing large projects. In this group, and only in this group, there is more of a focus on delivering all the objectives rather than delivering expected benefi ts. Again, we can only guess as to the reasons why but candidates such as; it’s almost impossible to measure benefi ts on large projects; I’m too far removed from benefi ts realisation to be able to judge whether this project has been a success; the project is too complicated to evaluate benefi ts objectively so objectives is a better measure of success; are all likely explanations.

In summary, delivering benefi ts/objectives are the runaway winners in the defi nition of a successful project, but this is not

universal. This focus dissipates the further we go down theproject hierarchy. There are also some demographic groups who have a preferential focus on the project sponsor (those delivering projects for 15 years or more) or a signifi cant objectives bias as opposed to benefi ts bias (those delivering our largest projects). We could all have a chance of delivering more successful projects if we kept a constant eye on making sure what we were doing was delivering the expected benefi ts and measured the benefi ts frequently and fairly throughout our project lifecycles.

0% 5% 10% 15% 20% 25% 30%

...Wouldn’t have had a better outcome if we did the project...

Other (please add any other success criteria in to this box)









What is project success - respondents from large projects




www.acando.co.uk

Technical Factors

We now move on from a defi nition of project success to a view of the technical factors that have a positive impact on creating a successful project. For this part of the survey, respondents were presented with a list of technical factors and asked to rate how important they were in contributing to a successful project. They could rate them as ‘crucial’, ‘very important’, ‘important’, ‘has little infl uence’, ‘somewhat unimportant’ or ‘irrelevant’. The factors available were the technical factors listed in the APM competence framework.In the chart above we show the top ten factors (out of 24 examined) that were rated as ‘crucial’ to ensuring a successful project.

Stakeholder management is the runaway winner and is according to our survey the most important factor to ensure the delivery of a successful project. Interestingly project planning is second in the list and this is despite the prevalence to action rather than thought that typifi es many projects in these austere times. But this is a view of what people thought the success contributors were, not how prevalent these were used on their projects – that comes later!

It’s as interesting as to what doesn’t appear in the top ten list as to what does appear. Budgeting and cost management isn’t in the top ten list – which goes hand in hand with its low showing in the defi nition of a successful project as well. But is counter to the import it plays in the actual delivery of many projects. It’s also interesting to see the omission of benefi ts management. This was universally recognised as the most important defi nition of a successful project, but does not appear in the top ten technical activities that should be carried out to ensure a successful project. This further supports the hypothesis we made earlier that although benefi ts management is recognised as a good defi nition of a successful project, the majority of us pay lip service to it when it comes to actually delivering our projects.

0% 10% 20% 30% 40% 50% 60%

Handover and closeout

De�nition

Issue management

Change control

Scope management

Requirements management

Business case

Risk management

Project planning

Stakeholder management

Top 10 Technical competences crucial to ensuring a successful project




Again there are some interesting variations across some of the demographics that we measured.The table above has columns representing a respondent’s position in the project hierarchy (see Appendix B for further defi nition). The cells in the table show the top fi ve technical tasks that each of these groups consider as the most important to delivering a successful project.

Every one of the groups had project planning and stakeholder management in their top fi ve (marked dark green). The only demographic group who did not have stakeholder management in their top fi ve were those involved in managing the smallest projects.

Three out of four of the groups had scope management and/or requirements management in their top fi ve (marked light green). This reinforces the importance of these activities in ensuring the successful delivery of a project.

But it is the diff erences again that give us cause for most commentary.

• Those lower down the project hierarchy have risk management in their top fi ve technical factors but they do not appear in the top fi ve of those higher up the hierarchy. Are we delegating risk management too low down? Or is this a consequence of those lower in the hierarchy being task focused and therefore risk management is naturally appearing higher on their agendas?

• Those responsible for managing project teams have change

control in their top fi ve which is not repeated anywhere else. It is however closely allied to defi nition which is in the top 5 of those paying for the projects. But it does seem like those paying for the projects think we should spend more eff ort accurately defi ning what we’re doing and those responsible for managing interpret that, or implement that, by having strict change control requirements. It seems those managing have a cure (change control) rather than a prevention (better project defi nition).

• And then we come back to benefi ts. Not only was this recognised as the most prevalent defi nition of a successful project (and more prevalent the higher up the hierarchy one goes), but it’s reinforced by those who pay for projects marking it as one of the fi ve most important technical tasks to ensure a successful project. But it doesn’t appear in any of the other groups’ top fi ve at all. This is only adding weight to the premise that benefi ts management is important and we just don’t do it.

The other important trend across the hierarchy demographic is the percentage of dispersion amongst the top fi ve technical factors that contribute to project success. At the ‘project support’ level, 65% of respondents have the same top fi ve criteria. At the ‘pay for’ level 93% of respondents have the same top fi ve criteria. This statistically signifi cant observation shows that as you move up the hierarchy, one is more certain of what contributes to project success but as you move down the hierarchy there is less agreement on what contributes to project success. It seems again that communication is not all it could be in our project environments.

Project Planning

Business Case

Risk Management

Stakeholder Mngt

Requirements Mngt

Project Planning

Risk Management

Scope Management

Stakeholder Mngt

Requirements Mngt

Change Control

Project Planning

Stakeholder Mngt

Requirements Mngt

Scope Management

Project Planning

Support Deliver Manage Pay for

Stakeholder Mngt

Scope Management

De�nition

Bene�ts Management



www.acando.co.uk

Behavioural Factors

We repeated the technical factors exercise with behavioural factors. Respondents were given the same 6 options and had to rate how important these behavioural competences were in contributing to a successful project. The factors available were the behavioural factors listed in the APM competence framework.The chart below shows the top ten factors rated as ‘crucial’ by the survey respondents. The top ten behavioural factors are universally rated as higher than all of the top ten technical factors demonstrating an appreciation that the behavioural aspects of project management are more likely to contribute to a successful project than the technical.

This raises an interesting question for those tasked with managing project functions in their companies – how many of your project managers have been on a formal confl ict management training course versus those that have been on a formal technical training course (e.g. PRINCE2)? I’m guessing most people have a prevalence to training on the technical – despite our respondent group being massively in agreement that behavioural factors contribute to project success signifi cantly more so than technical.When we amalgamate technical and behavioural factors, the top three factors are all from the behavioural stable – communication, teamwork and leadership.

They were no statistically signifi cant variations across any of the demographic groups on behavioural factors.

What’s missing?

It’s interesting what’s missing from any of these analyses. In none of the top ten have we seen project methodology or standardised tools and systems. A lot of organisations invest heavily in these two areas but they are not seen in our survey as important or contributing factors to a successful project. Why?

One hypothesis is that there is observational bias creeping in to the results here. And we built additional questions in to the survey to highlight obvious areas where we thought the bias may be.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Organisational roles

Organisational structure

Professionalism and ethics

Negotiation

Con�ict management

Project governanace

Project sponsorship

Leadership

Teamwork

Communication

Project Success Criteria - Behavioural Competences




There was a very specifi c question asked later about whether tools, methodologies and governance were contributing to making projects more successful, where two thirds of respondents agreed they did.

But only one quarter of respondents actually recognised that their projects used strong tools and methodologies. One hypothesis for why tools and methodologies were not rated higher as a contributing factor to success when presented with a long list is that people recognise their own projects don’t have them so they tend to automatically rate them lower (we’re very bad at admitting what we know we’re bad at).

Another hypothesis is that tools and methodologies are considered a prerequisite today and therefore the survey results focus on less obvious aspects. Unfortunately the data doesn’t validate this hypothesis (only 25% of respondents projects had good methodologies and tools) so it’s more likely to be the former hypothesis. The survey doesn’t give us a defi nitive answer one way or the other, but it does raise the interesting question.

Parting Thoughts

To fi nish off the analysis, here is a collection of other interesting observations that didn’t nicely fi t into any of the categories at which we’ve looked so far.

We’ve focused a lot throughout on benefi ts management. It’s crucial to the number one and number two defi nitions as to what a successful project is. It’s more accepted as important the higher up the project hierarchy we go. And it’s in the top fi ve technical competences for those paying for the projects. But as a whole it rated ‘crucial’ with only 20% of respondents and was 17th out of all 24 attributes examined. There seems to be a big mismatch between what we know defi nes a successful project and what we’re actually doing on our projects.

Over 60% of respondents thought there was too much emphasis placed on whether a project is successful or not. I found this result shocking and disturbing. It may be an acknowledgement that we don’t do benefi ts management well so we can’t easily answer the question whether a project was successful or not. Or it may simply be that those of us tasked with managing change in our organisations have lost our way and our raison d’être. This for me is an area that is worthy of further analysis but our study didn’t go further in to this topic.

0%

10%

20%

30%

40%

50%

60%

70%

Tools, Methodologies and Governance contribute to making projects more successful

Perc

enta

ge

of r

esp

on

den

ts

No

Yes

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

My projects use strong project tools and methodologies

Perc

enta

ge

of r

esp

on

den

ts

No

Yes

0%

10%

20%

30%

40%

50%

60%

70%

There is too much emphasis placed on whether a project is successful

Perc

enta

ge

of r

esp

on

den

ts

No

Yes

0%

10%

20%

30%

40%

50%

60%

Percentage of respondents rating benefits management as crucial to project success

Perc

enta

ge

of r

esp

on

den

ts

Crucial and very important

Crucial



www.acando.co.uk

There is little confi dence in the individual’s own ability to infl uence the project to a successful outcome. Most respondents thought they were not able to infl uence the project in this way. Thankfully, the more experienced you are in projects, the less you agree with this statement.

Even so, there is a distinct lack of either personal responsibility or a distinct lack of confi dence in ability to infl uence a project to a successful outcome.

Respondents were asked to rate what their successful projects had which they thought had contributed to them being successful. The top fi ve reasons are shown in the chart above. Communication is number one and this is also the number one behavioural competence believed to contribute to project success. A good level of alignment there. But interestingly ‘a strong executive sponsor’, ‘realism around objectives and timescales’ and ‘a project team with authority to make decisions’ didn’t appear in any of the other top ten lists we’ve looked at but are numbers 3, 4 and 5 in projects that have been successful. This too is worthy of further analysis but beyond the scope of the current study.

0%

10%

20%

30%

40%

50%

60%

All my projects will be successful because I will make sure they are

Perc

enta

ge

of r

esp

on

den

ts

NoYes

0%

10%

20%

30%

40%

50%

60%

My projects will be successful because I will make sure they are

Perc

enta

ge

of r

esp

on

den

ts

New Experienced Professional

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

E�ective communication

Well de�ned requirements

Realism around objectives and timescales

A strong executive sponsor

A project team with authority to make decisions

My successful projects had the following




And fi nally, in a nod to the wise (I couldn’t bring myself to write ‘older’), we recognise that as we gain more and more experience in delivering projects, they become easier to manage through to a successful conclusion.

And this is supported across the experience demographic with those who have more experience being even more in agreement with this statement. This is either self-gratifi cation, or a reality that experience truly does lend itself to improved chances of project success.

To Summarise

I’m not sure if the survey has produced more questions than answers, but there are some interesting observations:

• The number one defi nition of a successful project is one which delivers the expected benefi ts / objectives. But too few of us measure benefi ts or objectives. There is a dissipation of this belief the further down the project hierarchy you transcend.

• Of all the factors that contribute to a successful project, the behavioural factors are signifi cantly more important than the technical factors.

• Delivering a project on time and to budget were not seen as key to defi ning project success.

• Project practitioners are not confi dent of their ability to infl uence a project to a successful outcome.

• Experience counts for a lot.

• There were no signifi cant variations across industries in our respondent data. All industries showed the same trends as the aggregated trends presented in this paper (or where those trends were diff erent, they were not signifi cantly diff erent).

0%

20%

40%

60%

80%

100%

Projects are getting easier to deliver the more experience I get of managing projects

Perc

enta

ge

of r

esp

on

den

ts

No

Yes

70%

75%

80%

85%

90%

95%

Projects are becoming easier as I gain more experience

Perc

enta

ge

of r

esp

on

den

ts




www.acando.co.uk

Appendix A – Respondent DemographicsWe looked for 4 diff erent demographic groupings in the respondents and have analysed the data by these groupings searching for hypotheses that responses are somehow diff erent across diff erent groupings. The groupings we looked for were:

1. The industry into which the project is delivered. The hypothesis being that diff erent industries have diff erent success rates and diff erent criteria that contribute to success.

2. Respondent’s position in the project hierarchy. The hypothesis being that people at diff erent levels in the hierarchy have diff erent defi nitions of project success and diff erent ‘thresholds’ for a successful project.

3. The number of years the respondent has been a project manager. The hypothesis being that experience is an important contributor to project success.

4. The ‘size’ of projects managed. The hypothesis being that larger projects are less successful than smaller projects – as a project grows in size (and assumed growth in complexity), it becomes less successful.

For each of these demographics, the responses were ‘grouped’ to create 3 or 4 demographic groups that made analysis more manageable. The full demographic data and groupings are presented below.

Industry

There were respondents in every industry we measured. 40% of respondents were in the top four industries of Public Sector (1st), Software and Computer Services (2nd), Financial Services (3rd) and Telecommunications (4th). These industries where then grouped in to service industries, production industries, sales industries, third sector and ‘others’.

The distribution across these 5 groupings shows no one grouping being over-represented in our respondents.

Hierarchy

The majority of our respondents were Project Managers. There were very few respondents who were responsible for commissioning projects (budget holding business executives or Board Members) but there were some people in both of these categories.

0%

5%

10%

15%

20%

25%

30%

Industries respondents deliver projects in to

Perc

enta

ge

of r

esp

on

den

ts

ServicesSales

Production

Not for P

ro�tOther

0%

2%

4%

6%

8%

10%

12%

14%

Industry into which respondents deliver projects

Perc

enta

ge

of r

esp

on

den

ts

Financial ServicesRetail

Oil & Gas

Life Sciences

Public Sector

Third Sector

Aerospace & Defence

Automobile & Parts

Beverages

Chemicals

Constructio

n

Energy & Utilitie

s

Telecommunications

Food Producers

Forestry & Paper

Industrial Engineerin

g

Transporta

tion

Media

Mining

Real Estate

Software and Computer

Tobacco

Travel & Leisu

reOther

0%

5%

10%

15%

20%

25%

30%

Respondents position in project hierarchy

Perc

enta

ge

of r

esp

on

den

ts

Project Support /

Project ...

Programme Management O

�ce

Project Manager

Senior Project M

anager

Programmer Manager

Project / Programme Dire

ctor /...

Business

Executive who has .

..

Board Member



These were further grouped in to 4 distinct hierarchy levels shown below.

The only group under-represented in our responses were those who ‘pay for’ projects. It would be interesting to conduct further research in this demographic group to validate the responses we have received so far – but as always, getting data from this group is harder and more costly.

Experience

The data had good representation across the full length of experiences and approximately follows a normal distribution curve which is as expected. This data was further grouped as shown above.

Those ‘new’ to managing projects have less than fi ve years experience. The experienced grouping includes everyone between fi ve and twenty years experience and the professional grouping includes those with more than twenty years experience. Again the peak in the experienced column is expected.

Size

Size was defi ned as budget, team size and project duration in recognition that a low budget project can have 1000s of team members and therefore be ‘bigger’ than the budget itself shows. There is an expected ‘tailing off ’ of responses as project sizes grow and an unexpected peak at the end. The data has a large number of very large projects.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Respondents position in project hierarchy

Perc

enta

ge

of r

esp

on

den

ts

Support Deliver Manage Pay for0%

10%

20%

30%

40%

50%

60%

Number of years experience managing projects

Perc

enta

ge

of r

esp

on

den

ts


0%

5%

10%

15%

20%

25%

30%

Typical size of project managed

Perc

enta

ge

of r

esp

on

den

ts

Budget less

than 1m; ...

Budget less

than 5m; ...

Budget less

than 10m; ...

Budget less

than 25m; ...

Budget less

than 50m; ...

Budget less

than 100m; ...

Budget less

than 250m; ...

Budget over 2

50m; ...0%

5%

10%

15%

20%

25%

30%


Perc

enta

ge

of r

esp

on

den

ts

Less than 3 years

Between 3 and 5 years





Over 30 years



www.acando.co.uk

This data was further categorised as below.

Appendix B – Statistical Signifi cance TestingThe results from the data have been analysed to identify whether they are statistically signifi cant. This appendix does not intend to give a thorough treatment of statistical testing and the interested reader is encouraged to research further.All conclusions that have been marked as ‘statistically signifi cant’ in the body of the research report have been tested using

t-tests. In layman’s terms, it is testing whether the diff erences shown in the same data are statistically signifi cant or not. Because of the distribution of responses data can appear to have valid diff erences, but when compared to the whole sample, those diff erences can prove to be not statistically signifi cant. This is what we have tested for in the data presented here.We compared all signifi cance testing at the 99%, 95% and 90% confi dence levels and anything that proved to be statistically signifi cant at the 95% confi dence level has been marked as statistically signifi cant in our research body. Test results not passing the 95% confi dence test have not been marked as statistically signifi cant.

Again for the layman, if we have marked a result as statistically signifi cant in our research body, what we’re saying is that we have tested the diff erences in responses and the diff erence is not down to subtle diff erences in the entire population, the diff erences are statistically signifi cant.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%


Perc

enta

ge

of r

esp

on

den

ts

Small Medium Large

For more insight, comments and opinion visit our website www.acando.co.uk

About the AuthorPhil Jacklin is the Managing Director of Acando UK, a global consultancy providing project management services. He has managed consultancy fi rms for over 15 years, all providing project management services to blue chip clients across the globe. In this capacity Phil has provided governance across numerous projects, employed and recruited hundreds of project managers and advised many of his clients on how to improve their project management capabilities.


The only project management company to be accredited by the APM.

www.acando.co.uk

United Kingdom Head Offi ce

Oak HouseGround Floor, Sutton Quays Business ParkClifton RoadSutton WeaverWA7 3EH

Tel: +44 (0) 1928 796800Email: [email protected]

Delivering perfect projects around the world...

Manchester / London / Helsinki / Stockholm / Gothenburg / Malmö / Linköping / Ludvika / Falun / Västerås / Hamburg / Frankfurt / Düsseldorf / München / Stuttgart / Zurich / Trondheim / Oslo / Vantaa / Wilmington / Singapore
