Acosm 2010 Harold Van Heeringen V3

1

Challenges from a supplier point of view

Metrics questions in Request for Proposals

Harold van Heeringen

Sogeti Nederland

Senior metrics consultant / Cost engineer

Sizing, Estimating & Control

[email protected]

metrieken.sogeti.nl

@haroldveendam

ACOSM 2010ACOSM 201022

Company name and Presentation Title hereCompany name and Presentation Title here

Agenda

Request for proposals

Software estimation using function points

Typical questions in request for proposals (RFP’s)

Supplier challenges

Recommendations for the client organization.

Harold van Heeringen - Metrics Questions in RFPs


Company name and Presentation Title hereCompany name and Presentation Title hereHarold van Heeringen - Metrics Questions in RFPs

Trends in outsourcing?

Dutch government wastes billions of euros every year

on failed ICT projects



Request for Proposal (RFP) An invitation for suppliers, through a bidding process, to

submit a proposal on a specific product or service (source: wikipedia)

Information provided: Corporate information

Schedule of bidding process

Project Summary

Detailed overview of the project

Decision criteria




Demand issues (RFP sender) Provide the right information

Detailed

Up-to-date

Ask the right questions

Build a good decision model

Evaluate the proposals

Choose wisely




Supplier issues (RFP responder)

Can we:

Deliver the required functionality ?

Meet the technical and quality requirements ?

Within the time limits required ?

Answer all RFP questions ?

Estimate the project costs accurately ?

Score the best on the clients decision model ?

Support our claims with proof ?




Sogeti Sizing, Estimating & Control Division MD – fixed price/fixed date projects

Sizing, Estimating & Control 11 (COSMIC) Function Point Analysts

2 metrics consultants

Responsible for metrics part of a quotation. Size: FPA/COSMIC

Estimation: QSM / Sogeti tool / ISBSG

Scenarios !

Product: Methodical Estimation Report




Agenda




Supplier challenges





Software estimation using function points Function point analysis (NESMA / IFPUG / COSMIC)

Objective (ISO/IEC)

Repeatable

Verifiable

Quantifying the size om the functional user requirements Technology independent

Independent of implementation

Is a product measure, not a project measure

Doesn’t take into account any ‘non-functionals’




Project estimation

Size objectively measured Size = xxx function points

Estimation: Effort (hours) per activity

Duration (months) and milestones

Team size (average staff, peak staff)

Tools QSM SLIM suite

ISBSG repository release 11

Sogeti Estimating wizard




QSM SLIM Estimate



Generic Estimation Model

Need Software

Energy

Software

development

process

Waste

Time

Size Size

Defects

Effort

Duration

Defects

Productivity

Metric: Size

Function points

Metric: Size

Function Points

Metric: Effort

Number of hours

Manpower buildup

Peak staff

Metric: Duration

Number of weeks

Metric: Quality

Number of defects

Metric: Process productivity

Skills and experience team

Development environment

Complexity

Quality Management System

External influences



Agenda




Supplier challenges





RFP – Typical Metric Questions Q1 - What is your productivity for Java projects?

Q2 - How long do you need to build a .Net application of 500 FP?

Q3 - What is your price per function point for a 500 FP Oracle system?

Are these the right questions ??

Is the client organization capable of making the right decision based on the answers to these questions?




Agenda




Supplier challenges





time

Concept

Definition

High

Level

Design

Low level

Design

Realization

Requirements: complete and detailed?

Idea

Why What How

Size: Function Points

RFP

4x

3x

2x

1x

0.8x

0.5x

Project Rate1 42 33 14 15 16 27 48 49 510 5Average 3




time

Size

Size always grows !

RFP

Concept

Definition

High level

design

Low level

designRealisation

Idea

Challenge: Which size should we use in our estimation

and which size will the competitors use in their

estimation?


Why What How



The Software equation (Putnam)E

ffo

rt (

ma

nh

ou

rs)

Duration

Plan A: 6 months, 4.500 manhours

Plan B: 7 months, 2.400 manhours

Size/productivity

= effort 1/3 * duration 4/3




Same project, different durationE

ffo

rt (

ma

nh

ou

rs)

/ Pe

ak s

ize

Duration

Plan A

Duration: 6 months

Effort: 4.500 manhours

Peak staff : 5,8 fte

MTTD: 1,764 daysPlan B

Duration: 7 months

Effort: 2.400 manhours

Peak staff: 2,7 fte

MTTD: 2,816 days




Scenarios

Estimate / Business Case

Cost depending on Time-to-market

Example Scenario 1:

Duration: 5,5 months

Effort: 5.000 Mhr

Team size: 6,7 fte

Cost: € 430.000

Example Scenario 2:


Effort: 5.500 Mhr

Team size: 7,5 fte

Cost: € 480.000

Example Scenario 3:


Effort: 5.900 Mhr

Team size: 8,3 fte

Cost: € 530.000

Example Scenario 4:


Effort: 6.300 Mhr

Team size: 9,4 fte

Cost: € 620.000

Example Scenario 5:


Effort: 5.200 Mhr

Team size: 6,2 fte

Cost: € 400.000

Example Scenario 6:


Effort: 4.900 Mhr

Team size: 5,8 fte

Cost: € 380.000

Example Scenario 7:


Effort: 4.700 Mhr

Team size: 5,5 fte

Cost: € 360.000

Eff

ort

(m

an

ho

urs

)

Duration




Challenge supplierP

ric

e p

er

fun

cti

on

po

int

Duration

Plan A: 767 €/FP

Plan B: 452 €/FP

Answer: 380 €/FP ??

Client expectation

What is your price per function point for a 500 FP Oracle system?




Professionalism en realism

Expertise Use of function point analysis

Database with experience data

Repository with Benchmark data / tooling

Realisme Opportunism: ‘buying projects’

Commercial interests

A non-realistic bid is not in anyones interest !!




Cost of high and low estimates

Non- Lineair extra costs

Planning errors

Enlarging team more expensive but almost not

faster

Extra management attention / overhead

Stress: More defects, lower maintainability

Lineair extra Costs

Extra hours will be spent




A

Re

aliz

ati

on

(h

ou

rs)

5.000

15.000

CB

10.000

5.000 hours3.000 hours 7.000 hours

7.000

Estimation Result

B: Realistic

5.000 hours

7 months

Successful !

Efficient!

Successful !

Not efficient !

7.000 hours

11 months

A: Optimistic

3.000 hours

5 months

C: Pessimistic

7.000 hours

11 months

High and low estimates in practice

FAILS !!

10.000 hours

9 months

5.000 hours

7 months

Realistic estimation is the key to

succesful projects



Agenda




Supplier challenges

Recommendations for the RFP sender




Recommendations

Ask the right questions objective comparison, keeping other factors equal.

Assess the reality value of the quotation Establish a range in which the answer has to fall

Tools: for instance Galorath SEER, QSM SLIM or the ISBSG database

Demand objective evidence Experience data of the supplier




RFP Question should include

Metric to compare between competitors, for instance Productivity (hours/FP, Function points/hour, PI)

Costs (Price/FP)

Quality (defects per function point, Mean-time-to-defect (MTTD))

Technology for instance Java, Cobol, Oracle or MS.NET

Size (in Function Points or COSMIC FP)

Technical/ Functional Complexity for instance high/mediate/low

Phases/Activities to include for instance Technical Design, Coding, Unit testing, systems testing.

Duration required !!




Example of a good RFP Question

‘What is your productivity rate (hours/FP) for a moderately complex Java project of 500 function points and a duration of 20 weeks? Phases to include are technical design, coding, unit testing, systems testing and support of the user organization during the user acceptance test.’




Assess reality with ISBSG

ISBSG database R11 International Software Benchmarking Standards Group

R11: >5.200 projects ‘Best in Class’

Realistic range: 7,2 uur/FP – 11,6 uur/FP

Realistic range: 4,5 mnd - 9,5 mnd

ISBSG R11 hours/FP DurationNUMBER OF PROJECTS 24 24PERCENTILE 10% (P10) 3,5 3,3 mndPERCENTILE 25% (P25) 7,2 4,5 mndMEDIAN 8,4 6,0 mndPERCENTILE 75% (P75) 11,6 9,5 mndPERCENTILE 90% (P90) 19,6 12,2 mnd




Conclusions

Ask the right questions: Size, Cost, Productivity, Duration and Quality are highly

inter dependent

State the questions as specific as possible!

Evaluate bids Assess reality of proposals

Demand evidence of the supplier

Choose wisely Don’t go automatically for the cheapest supplier!




Next steps IWSM, Stuttgart (Nov. 10-12, 2010)

Workshop on proposing a standard framework for the use of software metrics in RFP management




staat voor resultaat

[email protected]

0031 88 6606600 +3165

metrieken.sogeti.nl

@haroldveendam


Documents

Acosm 2010 Harold Van Heeringen V3