Adopting Scaled Agile Framework (SAFe)The Good, the Bad, and the Ugly
James JanisseVP Connected Navigation System
Navigation Product Unit
TomTom @ a Glance
• Founded in Amsterdam in 1991
• 1 billion euros annual revenue
• 3,600 employees in 35 countries worldwide
• 75 million PNDs sold since 2004
• 3 million in dash navigation systems sold since 2009
• Our maps cover 116 countries reaching more than 4 billion people
• Real-time traffic service in 36 countries
Agenda
• Background
• Context setting
• The situation in 2012
• The resulting strategy
• Actions Taken 2012-Present
• The Good
• The Bad
• The Ugly
• Summary
Navigation Product Unit
4
Consumer devices
Smartphones & tablets
In-Dash systems
Internal Supplier
• 250 people in 40 scrum teams
• 10 locations & 8 countries
• 100% of the turn-by-turn navigation code
• 80% of the total solution code
Navigation Engines
User Interface
Situation in Early 2012
Strategy
People
Process
Technology
Product
Strategy
Observation Impact
Project orientation • Run and optimise the department as a professional services team
Stage-gate milestones process for “governance”
• Did not actually manage or mitigate risk as projects can fail a milestone or “pass with concessions” and keep going
• Does not enable portfolio planning or prioritisation between projects
Poor visibility and facts based decision making
• Actuals based on earned task hours• Forecast based on estimated hours remaining to complete
Strategy
People
Process
Technology
Product
ProductStrategy
People
Process
Technology
Product
Observation Impact
Each product and product extension is a branch of the mainline code
• 10+ branches of the mainline code• Create a “catch-up” project for every
branch to try to re-integrate (15-25% extra effort)
• “downstream” teams spend 3,4,5 months accepting the code and often changing it
Many projects working in all parts of the code with minimal module or component ownership
• Enormous effort, risk, delays, stress… to integrate project changes back into mainline
• Most projects break each other’s code
PeopleStrategy
People
Process
Technology
Product
Observation Impact
Resource managers and a matrixorganisation
• Who is accountable for what?• Aggressive and determined project
managers fight for their project and to get resources
Bring people to the requirement and form a project
• Start up cost and delays• Never worked together before• Spend time creating a way of working• Cannot estimate velocity for 2+ months• No historical estimate repository
ProcessStrategy
People
Process
Technology
Product
Observation Impact
Feature complete over quality
• Downstream teams receive a lot of defects, and have to resolve these themselves
• 3-5 months acceptance period required for downstream integration/ commercialisation team
Developers do not have to maintain their code
• Guess what happens?
KPIs are all Projectoriented: on-time, on-budget…
• No continuous improvement as each project is “different”
TechnologyStrategy
People
Process
Technology
Product
Observation Impact
Negligible automated testing
• Mostly manual testing• full regression requiring 3-4 calendar
weeks and a team of 4-5 people
No continuous integration
• Typically completed at the end of the project or every few months
• Integration often blocked by other project’s changes
Conclusions from 2012
1 August 201411
• Many products and releases are months-quarters late
• Quality is decreasing
• Costs are increasing
• Time-to-market is increasing
• Branching the code is evil
• Project orientation is wrong
• Poor Accountability
• Complexity is too high
• Significant Waste
• Insufficient information to make effective decisions
4 Key Objectives for 2012
1 August 201412
1) Reduce End-to-End Cycle Time
2) Reduce End-to-End Effort & Waste
3) Improve Quality
4) Clear accountability
5 Key Changes 2012-2014
1. Adopted “Scaled Agile Framework” by Leffingwell
2. Re-organised from Scrum Teams up
3. Adopted “High Assurance” S/W Requirements Model
4. Implemented Rally for Planning & Reporting
5. Automated testing with Continuous Integration
01/08/201413
#1: Adopted “Scaled Agile Framework”
1. Replace homegrown Feature Flow methodology with SAFe
2. SVP reads book cover to cover on his vacation
3. SVP buys book for CTO and other SVPs
4. SAFE Training
5. Trained
1. 75 Certified Scrum Masters &
2. 50 CPOs
6. Re-organised
01/08/201414
Re-organised from Scrum Teams Up
• Key assumption is value is only created by scrum teams
• Organise into Product clusters and component scrum teams
• One Agile Release Train per Product
• Every active/viable module/component is allocated to one and only one scrum team
• Adjust/supplement all scrum teams to have scrum master, developers, etc
• Everyone not in a scrum team is put in a backlog i.e. Project Managers, Resource Managers, Team leads….
• Design a thin/lean program support team to feed the scrum teams: Product Owner, Architect, Systems Team
• Design a thin/lean portfolio team
01/08/201415
Agenda
• Background
• The Good
• The Bad
• The Ugly
• Summary
You Decide
Decide to adopt SAFe
Stock = €3/share
You Decide
Decide to adopt SAFe
Stock = €3/share Stock is now €6.1 /share
01/08/2014
19
“There is no doubt in my mind that without SAFe and Rally we would not have launched this in only 140 days. It is also our best new product ever”
Information
Observation Impact
Single shared backlogavailable to all to view
• Improved transparency and info sharing
Historical repository of estimates and actuals
• Improves estimating by allowing historical comparisons
• Enables estimation accuracy analysis
Historical record of velocity and delivery reliability
• Visible to the whole business so that everyone can see the health and output of all teams
Stops academic debating • Forced consensus unless there is a smarter way
Product
Observation Impact
Simpler & consistentrelease schedule
• Makes customers lives easier to know that a new PSI is available on fixed and shared dates
Reduced cycle time • Get innovation out the door faster
Allows testing to go upstream
• Customer/downs-stream acceptancetesting can be embedded with each story or feature to ensure it is accepted at the end of the sprint vs waiting to accept it post PSI delivery
People
Observation Impact
Bring requirements to perpetual teams
• No start-up delay or effort• Lifecycle management guaranteed• Track record of working together• Self improving teams• Stop timesheet reporting
More sustainabledevelopment
• Teams select how many stories to bring into their release and sprints
Enables “bigger” picture view
• Release planning days• Release demo days• Scrum of Scrums
Process
Observation Impact
Bring requirements to perpetual teams
• Known way of working• More accurate planning and estimating• Tailored way of working per team
Eliminate project milestones and stage gates
• Budgeting is an annual activity to balance product/team spending
• Given budget, team size, and release dates are fixed, team just has to perpetually prioritise scope for maximum business value/PSI
Easy to add a new scrum team
• Adopt de facto processes, schedules, tools, methods…
Technology
Observation Impact
Automated testing • More thorough testing• More timely test results• Increases customer confidence
Continuous Integration
• Quicker feedback• Detect/prevent issues with each new submission
• No bottleneck at the end• Reduces cycle time as others stay up to date
Facts based qualityanalysis
• Statistical analysis of code quality available to compare across teams, products,
• Self-organising quality improvements
Agenda
• Background
• The Good
• The Bad
• The Ugly
• Summary
Information
Observation Impact
Normalised story points & t-Shirt sizes “suck”
• Seen as an invention by management with no value for the team itself
Everyone can see everyone’s backlog, priority, throughput…
• Everyone can second guess your prioritisation
• Everyone can second guess your estimates
Product
Observation Impact
Limited by the critical chain product/team
• Each scrum team can aim for maximum efficiency, velocity, quality but they are still part of an ART
Teaches the business that Agile = 100% predictable
• What happened to iterative development?
• What happened to incremental development?
• Don’t print features on the box at the start of the PSI
Enables the business to change direction/ strategy/ priorities often
• Let’s face it, too much Agility is just an inability to make choices and decisions i.e. chaos
People
Observation Impact
Decrease in employee and product satisfaction
• High energy and engagement BUT engineers feel early PSIs were incomplete as they were not feature parity
Organisational design • Incompatible with a matrix organisation
Career management? • Do you want a scrum master giving career guidance to everyone in their team?
Highlights inconsistencies in roles & levels
• Historical debt of having uncontrolled job grades, titles, levels, etc
• Issues with Jr. and Sr. Titles and levels• Issues with relative size of ARTs
People
Observation Impact
Shows week teams • Teams that normally staid below the radar which no one new what they did are suddenly very exposed to the daylight
Less role types and levels = less opportunities
• Less vertical career growth opportunities• What happens if you are on a second tier
ART?
Teams are not interchangeable
• People are prone to comparing velocities, cycle times without understanding non standard Story Point scale, tech debt, technology choices, dev ops issues…
Process
Observation Impact
Goes on forever without a break (HIP downtime)
• Projects used to have a nice slow start up and shut down phase, so cyclical rhythm
• Now work is harder and does not let up
Teams need to align on cadence/dates
• Teams that never had to coordinate their schedule suddenly have to adopt a common schedule for their ART
Teams need to align on basic Way of Working
• Teams that has their own home grown way of working suddenly need to standardise how they work
Technology
Observation Impact
Not all tooling is fit for purpose
• Effective portfolio and product management across multiple teams is not always supported
• Deep hierarchical backlog is hard to model• Rolling up reports above scrum teams can
be difficult/impossible with light weight scrum tools
Increase load on Dev Ops (testing and continuous integration)
• Continuous integration with expanding automated testing frameworks can get larger and larger and slower and slower
Agenda
• Background
• The Good
• The Bad
• The Ugly
• Summary
Information
Observation Impact
Shared information and transparency can lead to politics
• Teams can start mis-information campaigns to distract and deflect attention for their own poor performance
Product
Observation Impact
PSI does not mean release everything, all once, every time
• Potentially shippable does not mean that the business is relieved of having to make business decisions on risk/reward of each new feature.
People
Observation Impact
Some teams will resist
• Reject the need to be part of an ART• Reject the need to have common ways
of working…
Some people loose power
• Project managers loose scope, resourcing, and budget
• Resource managers are no longer needed
Levels and role consistency
• Can lead to a lot of unhappy employees, HR issues, administration…
Process
Observation Impact
Centralised decision making shifts to decentralised
• Evolving decisions to the lowest level threatens central portfolio level experts like architects and makes guiding independent teams hard
Fully defined transforms to barely sufficient
• Architects still love to make future proof architectural designs and plans
• UX want to make pixel perfect designs for all use cases
Sometimes one ARTruns over anothterART
• If a product or launch requires multiple ARTs, then the speed is limited to the slowest, typically system integration
Technology
Observation Impact
Very large Monolithiccode base cannot be continuously integrated
• What happens when the number of Integration slots available within a sprint does not allow each scrum team to check in daily, let alone weekly?
Very large Monolithiccode base cannot complete daily regression testing
• What happens when the automated regression cycle takes more than 16 hours (i.e. cannot be completed after hours)
Agenda
• Background
• The Good
• The Bad
• The Ugly
• Summary
4 Key Takeaways
1.People: Address career management and mentoring
2.Basic consistency within the organisation
• Agile means scrum teams decide and manage themselves 100% independently right?
3.Bigger Picture:
• Optimise for the Portfolio or ART • not for each scrum team
01 August 201439
4 Key Takeaways (cont’d)
4. Tooling:
• Deeper, broader, more complicated hierarchical capability tree
• added four more levels above a user story
• Active dependency tracking
• Tracking/Manage:
• Need to be able to roll up, drill down, lateral/diagonal slice & dicing
01 August 201440
Summary
The Good far out weighs the Bad and the Ugly
Questions?
01 August 201442
01 August 201443
01 August 201444
01 August 201445