Embed Size (px)
Citation preview
25.09.2014 • © PROMIS AS 1
Process innovation - Key Success Factor
in Government ProjectsRemi Hansen, PROMIS AS
Kristina L. Tangen, EVRY ASA
25.09.2014 • © PROMIS AS 2
Agenda
1. Introduction to the presenters and the case projects
2. Process innovations
• Backlog and business value innovations
• Test and approval innovations
• Planning and monitoring innovations
3. Contracting innovations
• Agile contracting - balanced contracts
• Target pricing, incentives and risk sharing
4. Q & A
25.09.2014 • © PROMIS AS 3
The Presenter – Kristina Lassen Tangen
• Computer Science from University of Oslo
• Certified Scrum Master
• Certified ISEB Practitioner
• More than 20 years experience from IT-business
• System developer
• Project and test Manager
• Head of test manager community
• Currently Chief Consultant in EVRY and head of test management community
• Experienced in Agile project – from 2003
25.09.2014 • © PROMIS AS 4
The Presenter – Remi Hansen
• B.Sc. In SW Engineering, M. Sc. in Industrial Economics
• More than 20 years experience from
the IT Consulting business
• System Developer (3 years)
• Project Manager and Business Consultant (14 years)
• Line manager (4 years)
• Certified Project Management Professional (PMP), PRINCE2 Practitioner,
IT Project Professional (ITPP), CSPO, ISTQB SW Testing Foundation and ITIL.
• Experience with agile projects started in 2002.
• Currently Senior PM in PROMIS, a leading provider of agile project
management services in Norway (www.promis.no)
25.09.2014 • © PROMIS AS 5
Some pricing model basics
25.09.2014 • © PROMIS AS 6
Background understanding
Contracting price models
• Time & Material:
• The Customer pays for all worked hours, disregarding productivity and
quality – all risk is on the Customer
• Fixed Price:
• The Customer pays the same amount, disregarding the hours needed to
produce the scope with agreed quality – all risk is on the Supplier
• Target Price:
• Customer and Supplier share benefits or losses in an agreed ratio
– if this ratio is 50%, the risk is evenly distributed between the parties
25.09.2014 • © PROMIS AS 7
The hourly rate curves –
comparing different price formats
25.09.2014 • © PROMIS AS 8
The case study projects
25.09.2014 • © PROMIS AS 9
Case study
• Lessons learned from 3 federal
government projects
• Over 1,200,000 hours of effort
• Not a scientific study
• 1st and 2nd hand information used
• Factual information is from public sources,
the rest is partly our personal opinions and
partly from sources listed
Photo (Flickr):
Okko Pyykkö
25.09.2014 • © PROMIS AS 10
Norwegian Public Service Pension Fund (SPK)
PERFORM project
• The largest and most important project ever
for the Public Service Pension fund
• Government funded
• About 800 000 project hours spent
and 100-180 persons involved
• Implements system support for managing new
pension reform
• Replacing legacy system due to outdated
technology
• Agile development methodology - Scrum
• Status: Completed – very successful, seen as
best practice in agile implementation
25.09.2014 • © PROMIS AS 11
Statnett
LARM Project
Statnett
• State Enterprise responsible for all high
voltage electricity transmission
and distribution in Norway
• Owns, operates and regulates
the national main grid
• Statnett co-ordinates supply and demand, and owns the main Norwegian power grid.
Photo: Statnett.no
25.09.2014 • © PROMIS AS 12
Statnett
LARM Project
• Community-critical system
• Ensuring the responsiveness of the national power grid
• Large project
• Interesting feature
• Very specialized domain – few users, but community critical operation
• Contract and sourcing setup
• Single supplier
• Target price with customer friendly profile
• Status:
• Ongoing (midway)
• Challenged – overruns and conflicting interests
Photo: Statnett.no
25.09.2014 • © PROMIS AS 13
The Norwegian Public Roads Administration
AUTOSYS project
The NPRA
• Responsible for the planning, construction and operation of the national and
county road networks, vehicle inspection and requirements, driver training and
licensing
Autosys project
• Replacing 30 year old mainframe system
• 5 releases in 3 years (overall 5 year project)
• SOA platform
• Size:
• 65 on supplier side, 35 on customer side
• 4 Scrum teams
• € 100 million project budget
• Status: Ongoing, challenged Photo: Vegvesen.no
25.09.2014 • © PROMIS AS 14
Case study projects summarized
• Large to very large projects in public
sector
• Very different domains and parts of the
state administration
• All Scrum based projects
• One successfully completed,
two ongoing and challenged
• All rely on IT suppliers governed by
contractual obligations
• All target price contracts, but with very
different customer / supplier balance
Photo (Flickr):
jardenberg
25.09.2014 • © PROMIS AS 15
Agenda
1. Introduction to the presenters and the case projects
2. Process innovations
• Backlog and business value innovations
• Test and approval innovations
• Planning and monitoring innovations
3. Contracting innovations
• Agile contracting - balanced contracts
• Target pricing, incentives and risk sharing
4. Q & A
25.09.2014 • © PROMIS AS 16
Process innovation: Business value and
product backlog innovation
25.09.2014 • © PROMIS AS 17
Case findings
Business value and product backlog issues
• Poor quality of the product backlog is devastating for the project
• Decision making: real (and lasting!) prioritization is difficult for inexperienced
P.O.s
• Responsibilities for different aspects of the product backlog is a source of
misunderstanding and controversy
25.09.2014 • © PROMIS AS 18
Recommendations
Good product backlog practice
• The customer must control the scope!
• A well organized P.O. Team supporting the Chief P.O. is needed
• Individual P.O.s must coordinate across domains – creating one coherent
backlog
• Appoint a Business Analyst to each sprint team = P.O.’s delegates to the
teams will relieve the load on the P.O.
Photo (Flickr):
Budzlife
25.09.2014 • © PROMIS AS 19
Product owner team
Development team 1 Development team 2 Development team 3
Product owner team
Product owner
25.09.2014 • © PROMIS AS 20
Recommendations
Good product backlog practice
• Continuously work with grooming the backlog – invest enough capacity to
clearly define User stories, and give priority
• Setting priorities takes a lot more effort than one should expect
• Put in place a method for assessing business value
• View the User story estimate as the budget for the task – budgeted effort is set
in the contract the scope must be controlled to fit that effort
25.09.2014 • © PROMIS AS 21
Business value and product backlog issues
Key Learning Points
• The quality of the backlog cannot be stressed enough –
the project success completely relies on it!
Ensure enough effort is spent grooming the backlog
• Use business value as basis for prioritization
• Set up a P.O. team as described and appoint a
Business Analyst to each scrum team
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 22
Process innovations –
Test and approval innovations
25.09.2014 • © PROMIS AS 23
System Hardening sprints
• 2-3 sprints after the development sprints to perform system test and complete
outstanding work (typically documentation and clean-up) – done in parallel with
solution description for the next delivery
• Is that a good idea?
There’s always some activities not completed. May give better preparation for and
smoother execution of Acceptance test
Real integration test will not be done along the way becomes a waterfall
approach, where testing is postponed (long time from programming to testing)
Risk mitigation is delayed
Very demanding for the project to do both system testing and solution design in
parallel
Probably not a good idea – include system testing in the development sprints
instead to minimize time from programming to testing
25.09.2014 • © PROMIS AS 24
Unit test
Integration test –Continuous integration
Functional System test and Integration test
Approval test
Acceptance test
Production test
Test strategy
• Different system test approaches taken – integration of test into development
seems to give the best result
Development cycle
Responsibility:
Development teams
25.09.2014 • © PROMIS AS 25
Test process - development
Define and execute
Functional system tests
Define and refine
functional test
conditions
Define and execute unit and
integration tests
Acceptance
criteria
Iteration n - 1 Iteration n Iteration n + 1
Execute System
Integration tests
25.09.2014 • © PROMIS AS 26
Recommendations
Integration of test and development tasks
• Business Analysts develop acceptance criteria for each user story before
coding
• TDD at all test levels
• The Test Manager should state quality requirements to development teams –
i.e. Definition of Done related to testing
• The Test Manager should have the authority to label user stories «Done»
• Run tests in fully integrated system
• There should be very limited need for the customer to perform separate tests –
rather verifying the test activities undertaken by the supplier
• Turn mindset of customer from traditional waterfall to agile
25.09.2014 • © PROMIS AS 27
Recommendations
Integration of test and development tasks
• Integrate test into Scrum teams
• Appoint one in each Scrum team as
Team Test Responsible
• Planning, design, test conditions,
development and test performed
collectively by the team
• Testers are no longer quality police
– instead valued members of the team
• Unit tests, integration tests, functional
system test and system integration
tests in construction sprints
25.09.2014 • © PROMIS AS 29
Integration of test and development tasks in the sprint teams
Key Learning Points
Maximize the testing done as part of the development
sprints
Appoint a test responsible in each sprint team
Empower the Test Manager to set process
requirements
Avoid pointless replication of tests on the Customer
side
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 30
Process innovations – Planning and
monitoring innovations
25.09.2014 • © PROMIS AS 31
Recommendations
Planning and monitoring
• A release master plan (product roadmap) – not just a product backlog
• Control gates: Book keeping for each sprint (earned value calculations etc) –
covered later
• Target price on user stories – not on release. Makes it possible to reprioritize
and move user stories in / out of scope
25.09.2014 • © PROMIS AS 32
Recommendations
Planning and monitoring – Release Master Plan
• The Release Master Plan is a tool for planning handover and implementation of
new functionality
• The Release Master Plan identifies the number of releases, their size, timing
and dependencies
• Guiding principle: Realization of business value!
25.09.2014 • © PROMIS AS 33
Recommendations
Planning and monitoring – Control gate at sprint completion
• Definition of done
• May be done in 2-3 working days following the sprint demo
• During these days:
• Functional verification by Product Owner and his / her team
• Checking code quality
• Checking architectural guidelines
• Checking test documentation
• Checking other documentation
• Contractual milestone
25.09.2014 • © PROMIS AS 34
The Anatomy of the Sprint
Sprint demo Sprint X
Sprint planning Sprint X+1 starts with decomposition
Sprint backlog for Sprint X+1 ready
Control gate:
1. Evaluation of Sprint X
2. Approval of sprint plan for Sprint X+1
3. Updating the risk matrix
4. Change control
5. Repatriation of agreed outstanding issues to the
product backlog
25.09.2014 • © PROMIS AS 35
Construction phase: The control gates
Blue line: Sprint team
Red line: Product Owner
Yellow line: Verification, Control gate and Definition of Done
Construction phaseSolution description
phase
25.09.2014 • © PROMIS AS 36
Planning and controlling considerations in CG
What was the productivity of the last sprint?
How was the product quality?
How is the risk picture evolving?
What can we improve – emphasis on learning and continuous improvements
25.09.2014 • © PROMIS AS 37
Planning and monitoring –
Key Learning Points
Use a Release Master Plan for planning and
communication
Use Earned Value Management – based on user story
estimates – not effort on activities
S-curve with planned value, earned value and actual
cost
Calculating productivity (Cost Performance Index) in
Control Gate process as a basis for forcasting
Execute a Control Gate process after each sprint for
control (progress / velocity, productivity, risk, etc.)
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 38
Agenda
1. Introduction to the presenters and the case projects
2. Process innovations
• Backlog and business value innovations
• Test and approval innovations
• Planning and monitoring innovations
3. Contracting innovations
• Agile contracting - balanced contracts
• Target pricing, incentives and risk sharing
4. Q & A
25.09.2014 • © PROMIS AS 39
Part II: Contracting innovations
25.09.2014 • © PROMIS AS 40
Why contracts?
• Public sector is engineered for cost control – «must» have a contract with
supplier obligations to have cost limit
• Any customer will want supplier buy-in – achieved through risk sharing /
incentives
Photo (Flickr):
Images_of_Money
25.09.2014 • © PROMIS AS 41
What price model is beneficiary in agile projects?
Target price is best suited because:
• Agile means close co-operation
• 50/50 sharing of incentives and sanctions
ensure the customer and supplier works
towards the same goal
• Suitable when requirements are not very
detailed
• Can be based on a less detailed basis than
traditional fixed price
Price model Average overrun
Time & Material based
(4 projects)
55%
Fixed price (5 projects) 33%
Target price (7 projects) 10%
Others (2 projects) 13%
Total (18 projects) 27%
• Fixed price assumes
requirements are described in
detail and that uncertainty is
low.
• T & M is suitable when scope is
not well defined and uncertainty
is high.
25.09.2014 • © PROMIS AS 42
Why Target Price Contracts with Risk Sharing?
• The Supplier is invited to commit to estimates with a higher degree of
uncertainty than in traditional fixed price contracts
• Support agile principle to avoid waste
• Reduced effort can be on initial specification, detailed design and planning
• The Customer may produce tender documents with high level requirements
• The Suppliers produce solution descriptions, estimated with a significant degree of
uncertainty
Both parties reduce work load in the initial bidding phases of the project
25.09.2014 • © PROMIS AS 43
Recommendations
Contracting
A balanced contract is necessary to ensure commitment and quality
• Implies 50 / 50 risk sharing with no cap
• Not agile to commit to up-front estimates for several years based on very
limited knowledge – risk for the supplier and ultimately the customer
• Include mechanisms to avoid commitment to estimates over a very long time
25.09.2014 • © PROMIS AS 44
PS2000 – a Target Price Contracting standard
• A Norwegian IT contracting standard, available in international English version
• Balanced contract, addressing issues seen in past projects (“best practice”)
• Iterative and agile version
• De facto standard for large agile projects in Norway
There is a well-proven agile contract available
http://www.dataforeningen.no/it-contract-standards.146223.no.html
Requirement
Analysis Acceptance and -
Completion phase
Detailed planning Analysis and design
Testing Development
Progression
Iterative
construction phase
CG1
CGn C
CG2
PMS 0 Contract Signing
Solution
Description
PMS 1 Approved Solution
Description
PMS 2 Delivery ready
for Acceptance
PMS 3 Accepted
Delivery
25.09.2014 • © PROMIS AS 45
In summary - Advantages with PS2000 Agile
• Describes how the project phases should be executed
• Loyal to the principles in Agile – roles, ceremonies and artifacts from Scrum
• Target price model, with a large variety of configurations reflecting project risks
Incentives for the supplier to deliver more functionality within agreed time
limits, but with satisfactory quality
• Fast and easy to complete annexes
• Suitable for repeating releases in a frame agreement for new development or
application maintenance
• The main success factor is that the parties agree on the process for managing
the Product Backlog, Sprints and Control Gates, with the corresponding roles
• Change control becomes lean – a signed Product Backlog after each sprint
documents the agreed changes
25.09.2014 • © PROMIS AS 46
Specific public sector challenges and contractual implications
Key Learning Points
Agile projects can be run under a contract, given that
the contract
Is balanced
Is engineered for agile execution
Lets scope be defined as late as possible
- preferably contracting each release as a separate
target price delivery
- ensuring estimates are as realistic as possible
PS2000 Agile is such a contract standard
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 47
Summary
25.09.2014 • © PROMIS AS 48
Summarizing some major issues
• Impression that all the projects would be agile – only one of them were
• Contractual constraints
• Cultural challenges – a fixed price mindset on the customer side
• Scrum does not make a project agile!
• Naive to expect Scrum / agile to solve every problem
• Even the most successful project had challenges in the first stage – but a
willingness to learn & adapt brought the project on the path of good practice
25.09.2014 • © PROMIS AS 49
Some smart moves in the PERFORM project
paving the way for success
• Gained experience with Scrum before starting the project
• Understood the value of a balanced contract
• Initiation phase on T&M price model
• The customer had own Scrum teams
in parallel with supplier teams
• Invested in good procedures and tools for
migration and configuration control
• Test as integrated part of development
• Hired external help for agile project management
coupled with active top management involvement
• The customer took overall responsibility for all processes – the main suppliers
had responsibility for construction of their parts for every release
• A continuous focus on improvement
Truly agile execution!
Photo (Flickr):
apdk
25.09.2014 • © PROMIS AS 50
9 defining features of agile projects
1. Business value is the most important criteria for quality and direction
2. Continuous prioritization of functionality based on cost / benefit
3. Close communication between the business side and developers
4. Short iterations to delivery of executable code
5. Frequent releases to production (integrated and tested)
6. Binding decisions taken as late as possible (‘Rolling Wave Planning’)
7. Evaluation, learning and improvement along the way
8. Autonomy: Self-organizing cross-functional teams
9. Avoid waste
25.09.2014 • © PROMIS AS 51
Defining features of agile projects
Feature Degree of
divergence in case
study projects
Covariance
between good agile
practice and
success
Business value focus High
Continuous prioritization High
Communication with business Medium-High
Short iteration – executable code Low
Frequent releases Low
Deferring decisions High
Learning Medium-High
Autonomy Low
Avoid waste High
25.09.2014 • © PROMIS AS 52
Public sector specifics?
• Wide variety of organizations, management and culture in public sector
– not possible to see the whole sector as one homogeneous group
• The recommendations given here applies equally to private sector projects
25.09.2014 • © PROMIS AS 53
References
• IT Project Professional certification curriculum - http://smidigeprosjekter.no/itpp
International launch of certification in 2013.
• Upcoming book – working title ‘Agile Contracting’
• PS2000 Agile Standard Contract
• http://www.dataforeningen.no/it-contract-standards.146223.no.html
• Complete English version - PS2000 Agile (PS2000 Agile, Maintenance, Framework,
Service Operations) ~ € 750
• PRINCE2
25.09.2014 • © PROMIS AS 54
Questions or comments?
Photo (Flickr):
Horia Varlan
25.09.2014 • © PROMIS AS 55
Thank you for the attention!
