Embed Size (px)
Citation preview
Data-Informed Product Decisions Using Feedback to De-risk your RoadmapRichard White, Founder & CEO
1
2 © Copyright 2016 UserVoice. Confidential and proprietary information.
90% of product management is building the right things in the right order
Step 1 - Define Your GoalsIncrease Win-Rate? Reduce Churn?
Step 2 - Estimate Impact Against GoalsGoal A Goal B Effort Confidence Score Accounts % Account $ %
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Step 2 - Estimate Engineering EffortGoal A Goal B Effort Confidence Score Accounts % Account $ %
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Step 3 - Estimate ConfidenceGoal A Goal B Effort Confidence Score Accounts % Account $ %
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Step 4 - Score & RankGoal A Goal B Effort Confidence Score Accounts % Account $ %
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
(GOAL A + GOAL B) x CONFIDENCE / EFFORT = SCORE
(REVENUE LIFT + REFERRAL LIFT) x CONFIDENCE
EFFORT
GOAL A GOAL B
100% BEHAVIORAL 100% QUALITATIVE
100% BEHAVIORAL 100% QUALITATIVE
Most Companies
100% BEHAVIORAL 100% QUALITATIVE
Most Companies
BEHAVIORAL ANALYTICS
CUSTOMER FEEDBACK
Step 4 - Score & RankGoal A Goal B Effort Confidence Score Accounts % Account $ %
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
(GOAL A + GOAL B) x CONFIDENCE / EFFORT = SCORE
Step 5 - Connect to Customer FeedbackGoal A Goal B Effort Confidence Score Customers % Account $ %
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
De-risk #1: Do asks back up estimates?Goal A Goal B Effort Confidence Score Customers % Account $ %
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
De-risk #1: Do asks back up estimates?Goal A Goal B Effort Confidence Score Customers % Account $ %
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
Why are we so confident in the impact with so few customer asks?
Step 6 - Add in CRM DataGoal A Goal B Effort Confidence Score Customers % Customer $ %
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
Ah, that’s because these are our largest Enterprise customers!Are we overestimating the impact?
Step 7 - Add in CRM & NPS DataReduce Churn Goal B Effort Confidence Score
Detractor & Churned
Customers %
Detractor & Churned
Customer $ %
Feature C 1 1 1 50% 1.0 5 10% $100,000 10%
Feature B 0 2 2 75% 0.8 0 0% $0 0%
Feature D 2 1 4 90% 0.7 60 40% $650,000 50%
Feature A 0 2 2 50% 0.5 12 9% $90,000 5%
De-risk #2: Do asks back up impacts?Reduce Churn Goal B Effort Confidence Score
Detractor & Churned
Customers %
Detractor & Churned
Customers $ %
Feature C 1 1 1 50% 1.0 5 10% $100,000 10%
Feature B 0 2 2 75% 0.8 0 0% $0 0%
Feature D 2 1 4 90% 0.7 60 40% $650,000 50%
Feature A 0 2 2 50% 0.5 12 9% $90,000 5%
Are we sure of our estimate for C to Reduce Churn? What about A?
De-risk #3: Confirm with AccountsGoal A Goal B Effort Confidence Score Customers % Customers $ %
Feature C 1 1 1 50% 1.0 60 30% $600,000 38%
Feature B 0 2 2 75% 0.8 3 2% $300,000 19%
Feature D 2 1 4 90% 0.7 90 45% $900,000 56%
Feature A 0 2 2 50% 0.5 30 15% $300,000 19%
Before we build C let’s review the feedback from those 60 accounts, contact them to give feedback on solution mock ups, beta, etc.
TakeawaysUse a goal-based scoring framework for making product decisions
- Make sure that decision framework is consistent and understood across teams. AKA Show Your Work
- Use your database of feedback to de-risk functionality you put on at the top of the TODO list.
21 © Copyright 2016 UserVoice. Confidential and proprietary information.
For more details read the eBook
How do we build that comprehensive database of customer feedback that informs good product management?
Need to tap into both direct & indirect feedback channels
Need to tap into both direct & indirect feedback channels
Getting Feedback from Users
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
CAUSES- Lack of awareness - Poor promotion / visibility
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
CAUSES- Lack of awareness - Poor promotion / visibility- Too much work to give feedback
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
CAUSES- Lack of awareness - Poor promotion / visibility- Too much work to give feedback- No value for giver - “What’s in it for me”
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
CAUSES- Lack of awareness - Poor promotion / visibility- Too much work to give feedback- No value for giver - “What’s in it for me”
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
CAUSES- Lack of awareness - Poor promotion / visibility- Too much work to give feedback- No value for giver - “What’s in it for me”
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
SOLUTION- Promote that you’re listening - “We want your feedback
to help us improve the product. Click here”
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
SOLUTION- Promote that you’re listening - “We want your feedback
to help us improve the product. Click here”- Single question survey if possible
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
SOLUTION- Promote that you’re listening - “We want your feedback
to help us improve the product. Click here”- Single question survey if possible- Explain what the user gets in return - Ideally influence
and not prizes. Set expectations and close the loop.
Feedback directly from end-usersMethods: NPS, Idea Boards, In-app Solicitation
Key Challenge: Low Response Rates Feedback from only a small vocal minority
SOLUTION- Promote that you’re listening - “We want your feedback
to help us improve the product. Click here”- Single question survey if possible- Explain what the user gets in return - Ideally influence
and not prizes. Set expectations and close the loop.
Getting Feedback from Internal Teams
Feedback from customer teamsMethods: Meetings, Spreadsheets
Sales Feedback Success Feedback
1. This thing
2. That thing
3. Some other thing
1. Not even a thing
2. Big thing
3. Small thing
VS
Feedback from customer teamsMethods: Meetings, Spreadsheets
Key Challenges: Inability to Aggregate How can you compare between top asks? How do you merge with direct feedback?
Sales Feedback Success Feedback
1. This thing
2. That thing
3. Some other thing
1. Not even a thing
2. Big thing
3. Small thing
VS
Feedback from customer teamsMethods: Meetings, Spreadsheets
Key Challenges: Inability to Aggregate How can you compare between top asks? How do you merge with direct feedback?
Hard to Trust Second hand reports instead of primary sources.
Sales Feedback Success Feedback
1. This thing
2. That thing
3. Some other thing
1. Not even a thing
2. Big thing
3. Small thing
VS
Feedback from customer teamsMethods: Meetings, Spreadsheets
Key Challenges: Inability to Aggregate How can you compare between top asks? How do you merge with direct feedback?
Hard to Trust Second hand reports instead of primary sources.
Time Consuming & FrustratingFor PMs and customer facing teams.
Sales Feedback Success Feedback
1. This thing
2. That thing
3. Some other thing
1. Not even a thing
2. Big thing
3. Small thing
VS
Feedback from customer teamsMethod: Custom or 3rd party tool for logging feedback directly into system of record.
SOLUTION- Know originating end-user - Feedback through
teams instead of from.
Feedback from customer teamsMethod: Custom or 3rd party tool for logging feedback directly into system of record.
SOLUTION- Know originating end-user - Feedback through
teams instead of from.- Store primary source feedback - What did the
user say in their own words.
Feedback from customer teamsMethod: Custom or 3rd party tool for logging feedback directly into system of record.
SOLUTION- Know originating end-user - Feedback through
teams instead of from.- Store primary source feedback - What did the
user say in their own words.- Conserve limited Product Manager time -
Crowdsource organization to customer facing teams + Skip the weekly feedback meeting.
Putting it All Together
60%% of customers we have feedback from
Now all feedback regardless of channel can be aggregated in a single system
USER FEEDBACK IDEAS
PM team mines for problems to solve.
Organized into ideas by users or customer team
members.
Now all feedback regardless of channel can be aggregated in a single system
USER FEEDBACK IDEAS
PM team mines for problems to solve.
Organized into ideas by users or customer team
members.
FEATURES
Solutions spec’d by PM team to
address one or more ideas
Now all feedback regardless of channel can be aggregated in a single system
USER FEEDBACK IDEAS
PM team mines for problems to solve.
Organized into ideas by users or customer team
members.
FEATURES
Solutions spec’d by PM team to
address one or more ideas
CUSTOMERS
CRM data about customers
TakeawaysBuild a system of record for all feedback
- Use best practices to get comprehensive feedback both directly and through customer facing teams. Avoid anecdotes and small numbers.
- Make sure the data structure enables you to combine that feedback. Avoid death by a thousand spreadsheets.
- Invest early. Cost is lower than you think and the payoff grows over time.
Use a goal-based scoring framework for making product decisions
- Make sure that decision framework is consistent and understood across teams. AKA Show Your Work
- Use your database of feedback to de-risk functionality you put on at the top of the TODO list.
Inform & de-risk the product roadmap
Increase internal confidence in the roadmapExposing your decision-making framework and the underlying data makes generating buy from customer facing as well as engineering teams easy.
Focus PM timeOur survey found that PMs spend 20-25% of their time gathering and organizing feedback. Free them up to focus on higher order functions.
Develop more evangelist customersPeople are often blown away when companies close the loop on product feedback (even if many months later). Don’t discount this aspect of being customer-centric.
Benefits Thank You!FOR FURTHER READING
eBook - Build a Customer Feedback Machine
eBook - The Product Manager's Guide to Smart Product Roadmap Prioritization
webinar - Best Practices for Managing Customer Feedback at Scale
video - How UserVoice uses UserVoice
