Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Michael S. McCalla
An Outcome Measurement Model:
Is your Agile Adoption Moving the Needle?
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
National Champs
Title Town!
Michael S. McCalla CSP, PSM II, SA, PMI-ACP
• Enterprise Agile Coach
• 10 Years of Experience Assisting Organizations with Agile Transformation
• Broad Range of Experience: Coaching on Team, Program, Portfolio, and Enterprise Levels
• Activist in the Agile Community:
• Named to Reviewer Panel of the Agile Practice Guide, a collaborative effort between Agile
Alliance & PMI
• One of the original board members of the Agile Uprising, online community dedicated to
advancing the Agile Mindset
• Other community involvement includes Social Media Enthusiast, Author, Agile Philly
Coordinator/Mentor
• Data Nerd
• Problem Solving and Product Development is My Passion, Coaching is a Close Second
• President of Lean Agile Intelligence, an Agile Assessment, Improvement, and Measurement
Platform Assisting Organizations with Agile Adoption given their Desired Outcomes
The Evolution: How Did We Get Here?
A Positive Upbringing
Are you saying………..
Confident, Ambitious, Passionate………..
July 15, 2008
“Embarking on a Journey”
Fundamentals
Source: Ahmed-Sidky
Coaching Ineffectiveness
“That is not Agile”
“The Agile Principle
Says……”
“Lets Check the
Scrum Guide”
“You need to develop more
of an Agile Mindset” “Let them Self-Organize”
“Kanban vs. Scrum”REPEATED
FAILURE!
The Daily Bashing
What I Wasn’t Considering
Enterprise CultureCommand & Control
Leadership
Continuously Learning from the Failure
The Epiphany
The Lean Startup
“A true experiment
follows the scientific
method. It begins
with a clear
hypothesis.“
“Focuses on
working
backwards from
desired business
results instead of
forward from the
technology stack
Validated Learning
Fundamentals
Source: Ahmed-Sidky
Taking It Further
Taking It Further
Based of Context
Traditional Approach
Based of Context
I Wasn’t Alone
Source: Version One State of Agile 2016 Survey
A Different Approach
Based of Context
Changing the Conversation…….
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Outcome Driven Improvement & Measurement Model
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Outcome Driven Improvement & Measurement Model
Enterprise
Unit
Program
Team
Self
For Every Level of Today’s Agile Organization……
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Outcome Driven Improvement & Measurement Model
Enterprise
Unit
Program
Team
Self
For Every Level of Today’s Agile Organization……
Case Study:
The Payments unit of a Credit Card Company has ten cross functional teams just
adopting scrum. They are responsible for all payments applications in the
organization. Six months ago, they lost a number of key senior developers to an early
retirement package. They were immediately replaced by developers fresh out of
college. Since then there has been several priority one bugs which has led to missed
payments and manual intervention. The bugs were created as a result of new
enhancements introduced, and there new feature backlog is massive.
Management is starting to question the hires. The teams have been vocal that the
products lack documentation and are very fragile. They are unaware of the impacts
of their changes both upstream and downstream. QA has advised that the bugs
found were edge cases, and that is why they were not caught in manual testing.
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Enterprise
Unit
Program
Team
Self
• What outcome are we looking to achieve given
our context?
- Improved Reliability
• Do we have clear priorities and does it align to the
organization's goals for the year?
- Less Interruptions to the Customer
• What do we need to focus on to help us get
there?
- High Quality
• What are some of the team challenges?
- Lack of Experience and Documentation
• How are we going to measure that outcome?
- Escaped Defects
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Enterprise
Unit
Program
Team
Self
• What practices will help us achieve our desired outcomes?
1. Pair Programming
2. Continuous Integration Pipeline
3. Automated Tested
4. Slack
5. Agile Testing
• Who is best to tell us how the team is performing in these practices?
1. The Team!
• How are we performing in the practices that will lead to better
outcomes?
1. No Pairing
2. CI Pipeline only executed unit tests
3. Integration & Acceptance Tests were only partially automated
4. 100% utilized every sprint
5. Developers only responsible for development not testing
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Organization
Unit
Program
Team
Self
• What short term tactical goals can we set to help achieve
our outcome?
1. Identify Pairs, and pair 50% of the Time
2. Increase Number of Integration Tests and Automate
3. Add Integration Tests to CI Pipeline
• Do we have any impediments that are out of control
preventing us from improving in the practice?
1. Full Backlog of Features for Company wide projects
• What metrics can we use to measure our progress?
1. Leading: # Integration Tests, Integration Tests
Automated %, WIP, Acceptance Tests Automated %, Pair %
2. Lagging: Escaped Defects & Throughput
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Enterprise
Unit
Program
Team
Self
• Did we meet our tactical goals and do what we said we
were going to do?
- Look at Leading Indicators
• Is it helping us move the needle?
- Look at Lagging Indicators
• Do we have organization constraints that are slowing us
down?
- This is an opportunity to gather insights and determine
new objectives for another cycle of improvement
• Do we need to continue improving in these practices, or
should we try some others?
- Has our hypothesis proved to be true?
Measure: Tracking Progress of Leading IndicatorsMeasure
Strategic Objectives
Self-Assess
Tactical Goals
Track leading indicators to see if the team is progressing towards the goal
Measure: Track Progress of Lagging IndicatorsMeasure
Strategic Objectives
Self-Assess
Tactical Goals
Track lagging indicators to see if the tactical improvement is leading to desired outcomes
Measure: Track Organization PatternsMeasure
Strategic Objectives
Self-Assess
Tactical Goals
Look at patterns across the organization to identify possible organization constraints
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Enterprise
Unit
Program
Team
Self
Outcome Driven Improvement & Measurement Model: Enterprise Level
Real Life Case Study
Just Do It Advisors is mid-tier (5000 people) asset management boutique that has
been in existence for 85 years. They pride themselves on establishing personal
relationships with their clients and utilizing an active money management approach.
This strategy has enabled them to charge higher fees. However, over the past 6
years, market share and revenue has declined significantly, because many of their
clients have become deceased and their children have chosen to go elsewhere.
Feedback has been competitors offer a comparable service at a cheaper price, and
provide more transparency of funds and reporting via online portals.
They have a traditional silo-ed organization structure in which the IT organization has
always been seen as a cost center who keeps the lights on, and serves multiple
business units. Recently, they have been asked to deliver new applications, but
feedback was the deliveries did not meet the need or it took to long. They have
experienced with Scrum, but it is the typical “Scrum-But”
Taking It Further
Frameworks & Practices
Tactical Goals: Determine Leading Indicators
• Team and groups should proactively pick the metrics that will help them track their progress
towards their desired outcomes so they can baseline and incrementally measure progress
• Identify Leading Indicators that will help measure if you are on track to achieve your tactical goal
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Time to Market Customer
Satisfaction
Employee
Satisfaction
Innovation Responsiveness Reliability Predictability
Throughput
(Velocity)
NPS Happiness Index # of New Products
Released
Product/ Project
Lead Time
Test Coverage Story Point
Attainment
Build Time Value Points Team Health
Assessment
Bug Fixing vs New
Product
Development %
Feature Lead
Time
Builds Per Day Velocity Variation
Deployment Lead
Time
Cost of Delay Code Complexity
New Environment
Automation
Failed
Deployments
Measure: Determine Lagging Indicators
• Team and groups should proactively pick the metrics that will help them track their progress
towards their desired outcomes so they can baseline and incrementally measure progress
• Identify Lagging Indicators to see if your tactical improvements are moving the needle
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Time to Market Customer
Satisfaction
Employee
Satisfaction
Innovation Responsiveness Reliability Predictability
Number of
Releases
ROI Attrition New Customer % New
Project/Product
Cycle Time
Escaped Defects
in Prod
On Time Delivery
Rate
Project/Product
Cycle Time
Revenue Yearly Surveys New Markets
Entered
New Feature
Cycle Time
Availability
Feature Cycle
Time
Profit Number of
Production
Deployments
# of Prod
Deployments
Takeaway
Start with Outcomes and Context…..
And Gain Alignment…..
Contact Me
Website: www.leanagileintelligence.com
Email: [email protected]
Twitter: @MSMcCalla_1
LinkedIn: https://www.linkedin.com/in/michaelsmccalla/
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
• What is important to this organization right now?
Transitioning it’s Business Model to be more Digital
• What outcome are we looking to achieve?
Increased Customer Satisfaction with Innovative Products
• What are some of the team challenges?
Siloed Environment & Lack of Collaboration
• Do we have clear priorities?
New Products Trump Everything else
Enterprise
Unit
Program
Team
Self
• What practices will help us achieve our desired outcomes?
Practices that ensure we deliver the right things:
1. Stable Cross Functional Teams
2. Dedicated “True” Product Owners
3. Customer Engagement
4. Experiments and Prototyping
5. Lean Budgets
• How are we performing in the practices that will lead to
better outcomes?
Initially, really bad
• Who is best to tell us this information?
The Team; in this case Senior Leadership
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Enterprise
Unit
Program
Team
Self
• What short term tactical goals can we set to help achieve
our outcome?
1. Identify Product Owners from the Business
2. Get New Product Owners Training
3. Create 2 Cross Functional Teams for New Prod Dev
• Do we have any impediments that are out of control
preventing us from improving in the practice?
1. Product Owner Empowerment
• What metrics can we use to measure our progress?
1. Leading: # of Dependencies, % of Trained POs, New vs.
Existing Prod Dev %, Net Promoter Score (NPS)
2. Lagging: Retention, Referral, Revenue
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Enterprise
Unit
Program
Team
Self
• Did we meet our tactical goals and do what we said we
were going to do?
Look at Leading Indicators
• Is it helping us move the needle?
Look at Lagging Indicators
• Do we have organization constraints that are slowing
us down?
This is an opportunity to gather insights and determine
new objectives for another cycle of improvement
• Do we need to continue improving in these practices,
or should we try some others?
Has our hypothesis proved to be true?
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Enterprise
Unit
Program
Team
Self
Measure: Tracking Progress of Leading IndicatorsMeasure
Strategic Objectives
Self-Assess
Tactical Goals
Track leading indicators to see if the team is progressing towards the goal
Measure: Track Progress of Lagging IndicatorsMeasure
Strategic Objectives
Self-Assess
Tactical Goals
Track lagging indicators to see if the tactical improvement is leading to desired outcomes
Measure: Track Organization PatternsMeasure
Strategic Objectives
Self-Assess
Tactical Goals
Look at patterns across the organization to identify possible organization constraints
Contact Me
Website: www.leanagileintelligence.com
Email: [email protected]
Twitter: @MSMcCalla_1
LinkedIn: https://www.linkedin.com/in/michaelsmccalla/
Appendix
Did we meet our tactical goals and do what we said?
Is it helping us achieve the desired outcome?
Do we have organization constraints slowing
us down?
Outcome Driven Improvement and Measurement Model Canvas
What is important to this team right now?
What outcome are we looking to achieve?
What are some of the teams' challenges?
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
What short term tactical goals can we set in
those practices to help us achieve our desired
outcomes?
Do we have any impediments out of our control
preventing us from achieving those tactical
goals?
What metrics, both leading and lagging,
can we use to measure our progress?
What Agile and Lean practices will help us
achieve those outcomes?
Where could we go to better understand the practices
and if we are doing them correctly?
Who is best to assess our performance on them?
Tactical Goals: Determine Leading Indicators
• Team and groups should proactively pick the metrics that will help them track their progress
towards their desired outcomes so they can baseline and incrementally measure progress
• Identify Leading Indicators that will help measure if you are on track to achieve your tactical goal
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Time to Market Customer
Satisfaction
Employee
Satisfaction
Innovation Responsiveness Reliability Predictability
Throughput
(Velocity)
NPS Happiness Index # of New Products
Released
Product/ Project
Lead Time
Test Coverage Story Point
Attainment
Build Time Value Points Team Health
Assessment
Bug Fixing vs New
Product
Development %
Feature Lead
Time
Builds Per Day Velocity Variation
Deployment Lead
Time
Cost of Delay Code Complexity
New Environment
Automation
Failed
Deployments
Measure: Determine Lagging Indicators
• Team and groups should proactively pick the metrics that will help them track their progress
towards their desired outcomes so they can baseline and incrementally measure progress
• Identify Lagging Indicators to see if your tactical improvements are moving the needle
MeasureStrategic
Objectives
Self-Assess
Tactical Goals
Time to Market Customer
Satisfaction
Employee
Satisfaction
Innovation Responsiveness Reliability Predictability
Number of
Releases
ROI Attrition New Customer % New
Project/Product
Cycle Time
Escaped Defects
in Prod
On Time Delivery
Rate
Project/Product
Cycle Time
Revenue Yearly Surveys New Markets
Entered
New Feature
Cycle Time
Availability
Feature Cycle
Time
Profit Number of
Production
Deployments
# of Prod
Deployments
Agile Assessments
They Are Not……….• Maturity Models• Audits• Absolute
They Are • Another vehicle to help teams identify improvement
opportunities• An opportunity to have rich conversation with teammates• An opportunity to learn
3.45Agile Team Score
Why Assessments?
• Easy and Tangible Improvement Model for Teams to Grasp • Self Sustaining – Something that can be done by themselves when coaches
aren’t around or consultants walk out the door
• Self-Assessments• Invitation to Improve!• Everyone has a voice, and stake in the game• Empowerment to self organize and get to the next level of team
performance
• Provides Holistic View of Agility Across the Enterprise • Aggregated Results all the way up to the Enterprise Level• Data Analytics to identify patterns preventing Business Agility such as
coaching needs, investment opportunities, and organizations constraints