Embed Size (px)
Citation preview
Data Driven Decision EngineEngineer + Data Science + Product Manager = More Power!
Dr. June Andrews
February 10, 2015
June Andrews Data Driven Decision Engine February 10, 2015 1 / 42
Professional Social Graph
June Andrews Data Driven Decision Engine February 10, 2015 2 / 42
LinkedIn’s Vision
June Andrews Data Driven Decision Engine February 10, 2015 3 / 42
LinkedIn for Members
June Andrews Data Driven Decision Engine February 10, 2015 4 / 42
LinkedIn for Customers
June Andrews Data Driven Decision Engine February 10, 2015 5 / 42
Diverse Product Portfolio
Figure : Over 40 products integrated into the homepage.
June Andrews Data Driven Decision Engine February 10, 2015 6 / 42
June Andrews
My RoleUnderstand:
User EcosystemEngineering & DataConstraintsVision
Recommend RoadmapsSupport Roadmaps
Figure : Which is better? Routes viaGoogle Maps
June Andrews Data Driven Decision Engine February 10, 2015 7 / 42
Roadmaps Worked On
June Andrews Data Driven Decision Engine February 10, 2015 8 / 42
Roadmap Process
1 Generate Ideas
2 Project SizingProject ImpactDetails for Maximal Impact
3 Project Development Playbook
4 Communicate Recommendations
5 Learn
NoteUse the same process for yearly, quarterly, internal, and externalroadmaps.
June Andrews Data Driven Decision Engine February 10, 2015 9 / 42
Generate Ideas
1 Generate Ideas
2 Project SizingProject ImpactDetails for Maximal Impact
3 Project Development Playbook
4 Communicate Recommendations
5 Learn
June Andrews Data Driven Decision Engine February 10, 2015 10 / 42
Sourcing Ideas
Source from Everywhere:External Inspiration: books, research, sociologists, etc.Brain storming sessions with PMs, Designers, & EngineersDeep Dive AnalysisFriends & Family
DetailsThis is the top of the funnel. Make it big.Limit how long you listen, not where you listen.Do not interject ’We tried that.’
June Andrews Data Driven Decision Engine February 10, 2015 11 / 42
Edit Ideas
Example (My Mother)Products can come from a personal space. Translate the example intosalient points.
Example (Generalize Products)Products can come from a product specific space. Generalizerequests for UI changes to the broader product base.
Example (Unify Products)When a particularly big change is desired it will appear as many smallsuggestions. Find the focus of what product wants change.
June Andrews Data Driven Decision Engine February 10, 2015 12 / 42
Support Visionary Goals
Figure : Line Ideas up with Vision, check for full coverage.
June Andrews Data Driven Decision Engine February 10, 2015 13 / 42
Batch Process Ideas
1 Generate Ideas
2 Project SizingProject ImpactDetails for Maximal Impact
3 Project Development Playbook
4 Communicate Recommendations
5 Learn
June Andrews Data Driven Decision Engine February 10, 2015 14 / 42
Idea Sizing - Back of the Envelope
GoalFor each idea find:
1 How many members involved2 How frequently involved3 Magnitude of involvement
Physics:
Force = Mass · AccelerationWork = Force · Distance
Products:
Impact = Number of People · ∆MetricWork = Impact · Product Cost
June Andrews Data Driven Decision Engine February 10, 2015 15 / 42
Idea Sizing - Back of the Envelope
User error propagation to temper expectations:
Ideal Impact = Number of People · ∆MetricEstimated Impact = 1
2 · Number of People · 12 · ∆Metric
Estimated Impact = 14 · Ideal Impact
NoteThese estimates will be checked after the product is built. Beconservative - under promise over deliver.
June Andrews Data Driven Decision Engine February 10, 2015 16 / 42
Batch Process Idea Sizing
Idea Success RatioFor every idea in production, there are ≈ 7 ideas that did not make thecut. A single roadmap involves 5 to 20 major projects.
Figure : By adding dimensions to Hadoop queries can batch process ideas.
June Andrews Data Driven Decision Engine February 10, 2015 17 / 42
Find the Giants
Find the Maximal ImpactA big idea is first seen from multiple angles as small ideas.
Figure : Glimpses of the big picture. The Godzilla!
June Andrews Data Driven Decision Engine February 10, 2015 18 / 42
Product Development Playbook
1 Generate Ideas
2 Project SizingProject ImpactDetails for Maximal Impact
3 Project Development Playbook
4 Communicate Recommendations
5 Learn
June Andrews Data Driven Decision Engine February 10, 2015 19 / 42
New Product
Quantity or QualityBlank page effect is quantity without quality.Diamond in the rough effect is quality without quantity.
Recommendation:Improve QualityImprove QuantityIterate
GoalLong Term Growth. Virality easily controls Quantity, Quality is hard.Spend 80 to 20 on quality to quantity.
June Andrews Data Driven Decision Engine February 10, 2015 20 / 42
Order Matters - Social Products
Figure : Fire burns outward as a ring. A metric of burn length increases untilthe fire burns out.
June Andrews Data Driven Decision Engine February 10, 2015 21 / 42
Fire Ring Examples
Figure : Farmville hit 80M users in 1 year. Google+ hit 25M users in 24 dayswith an average of 7 min per month per user.
Fatal FlawTurned on uncontrolled viral mechanisms before creating a solidmember experience.
June Andrews Data Driven Decision Engine February 10, 2015 22 / 42
Controlled Virality
Figure : Facebook hit 6M users at 1 year. Gmail spent 3 years as aninvitation only service.
Key Components1 Released in stages to new populations.2 Release delay allowed for quality improvements.3 You were hungry for it, before you could get it.
June Andrews Data Driven Decision Engine February 10, 2015 23 / 42
Order Matters - Local or Global
Local DomainQuestion is still Quantity or Quality.
Growth Mechanisms:Community ManagersPower Users or ElitesAll Social Viral Mechanisms
June Andrews Data Driven Decision Engine February 10, 2015 24 / 42
Order Matters - Local or Global?
Figure : Citysearch developed many reviewers with few reviews. ThomasBrothers spent 94 years being the expert map makers of the west coast.
Fatal FlawBalance. Citysearch went global before understanding local drivers.Thomas Brothers fought to stay local.
June Andrews Data Driven Decision Engine February 10, 2015 25 / 42
Order Matters - Controlled Local & Global
Figure : Both Yelp and Uber grow one city at a time.
Key ComponentsCommunity ManagersRewarded initial users with parties and discountsWord of Mouth Virality - slow and controlled
June Andrews Data Driven Decision Engine February 10, 2015 26 / 42
Order Matters - Established Product Shifts
Established DomainGoal is to protect power user base and create new opportunities
Growth Mechanisms:Grandfathering of old membersLayering of new and old productApp specialization
June Andrews Data Driven Decision Engine February 10, 2015 27 / 42
Shifts without Grandfathering
Figure : Netflix split DVD mailings from base subscriptions. Foursquareported checkins over to Swarm.
Fatal FlawFinal outcome is still to be seen. Power users provided copiousnegative feedback about having to adapt their experience.
June Andrews Data Driven Decision Engine February 10, 2015 28 / 42
Shifts with grandfathering
Figure : Pandora grandfathered in yearly contracts to their now monthlysubscription. Gmail’s introduction of tabs can be set to old experience.
Key ComponentsPositive messaging for power usersNotice of changes far in advanceExpanded opportunity for connecting with new members
June Andrews Data Driven Decision Engine February 10, 2015 29 / 42
Simulate Growth
Stochastic ProcessesCan accurately prodict a year out. Simulate changes in viralitycoefficients and engagement.
June Andrews Data Driven Decision Engine February 10, 2015 30 / 42
Humanize the Data
1 Generate Ideas
2 Project SizingProject ImpactDetails for Maximal Impact
3 Project Development Playbook
4 Communicate Recommendations
5 Learn
June Andrews Data Driven Decision Engine February 10, 2015 31 / 42
Member Base Perspectives
Figure : Data Driven & Intuitive perspectives of the member base.
June Andrews Data Driven Decision Engine February 10, 2015 32 / 42
Member Base Perspective
Figure : Perseption of member base change.
Members as DataAdvantage is well defined tracking for all members.Disadvantage is limited understanding of emotional impact.
June Andrews Data Driven Decision Engine February 10, 2015 33 / 42
Member Base Perspective
Train Intuitive ThinkingFind a manageable set of representative users.
Interview these members. UEX team.Case Study these members’ experiences and long term behavior
Figure : Use Data to train Intuitive Thinking.
June Andrews Data Driven Decision Engine February 10, 2015 34 / 42
Reflect
1 Generate Ideas
2 Project SizingProject ImpactDetails for Maximal Impact
3 Project Development Playbook
4 Communicate Recommendations
5 Learn
June Andrews Data Driven Decision Engine February 10, 2015 35 / 42
Reflect
RefineCompare Project Sizing estimates and launch resultsCompare Playbook with release strategyAdapt elements to work with the company you are atPreserve past ideas and sizing for future considerations
June Andrews Data Driven Decision Engine February 10, 2015 36 / 42
Recap - Generate Ideas
No voice is too small.
June Andrews Data Driven Decision Engine February 10, 2015 37 / 42
Recap - Size Opportunity
Bound the future.
June Andrews Data Driven Decision Engine February 10, 2015 38 / 42
Recap - Release Playbook
Build with balance and pivots.
Figure : Seahawks’ playbook did not include Lynch in the final 2 minutes.
June Andrews Data Driven Decision Engine February 10, 2015 39 / 42
Recap - Communicate
Make conclusions relatable and memorable.
Figure : Humanize the Data
June Andrews Data Driven Decision Engine February 10, 2015 40 / 42
Recap - Reflect
Tune the Data Driven Decision Engine!
Figure : It takes a village to run this engine.
June Andrews Data Driven Decision Engine February 10, 2015 41 / 42
We’re Hiring
Drive the Data Driven Decision Engine!
June Andrews Data Driven Decision Engine February 10, 2015 42 / 42
