ProductCamp Vancouver 2013

  • View
    408

  • Download
    1

Embed Size (px)

DESCRIPTION

Product management as a discipline is undergoing a transition towards data-driven decision making. As the cost of testing product feature ideas falls, the role of a product manager becomes one based on testing many feature ideas, and rolling out proven features. In this presentation we touch on some of the trends behind this paradigm shift, and consider some tools the product manager can use.

Transcript

  • 1. Lean Product Management The Value of Experiments @DaveSharrock ProductCampVancouver 2013Photo: ESAhttp://www.nbcnews.com/id/50677062/ns/technology_and_science-space reserved. Copyright 2007-2013 agile42 | We advise, train and coach companies building software www.agile42.com | All rights

2. Assembly Line Manufacturinghas a high cost of change http://www.archives.gov.on.ca/english/on-line-exhibits/d-day/big/big_03_airline_assembly.aspx Archives of Ontario, Reference Code: C 190-5-0-0-21agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 3. Inclusive thinking Feature usage in In software development highcost of change leads to inclusive enterprise softwarethinkingNeverRarelySometimesOftenAlways Any and every idea has to be 7%captured in the rst version of a13%requirements specication Creates waste - bloated45%documents, unwanted features16%and entitlement thinking19%Standish Group - CHAOS Report, 1995, 2002agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 4. Cost ofHypothesizedChangeRequirements Capture all possible needs Detail of Requirementsagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 5. Software Development has commoditized cost of change Object-oriented languagesjqueryRuby on Rails Coffeescript Continuous Deliveryagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 6. Cost of HypothesizedChange Requirements Evolving RequirementsCapture allpossible needs Emerging needs as development progressesDetail ofRequirementsagile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 7. Changing paradigms If change is cheap, requirementscan change continuously We can evolve our thinking aswe learn more about the productwe are building Maximizing Return-on-Investment means validating newfeatures early, before the totalinvestment cost has been spentagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 8. Cost of HypothesizedChange Requirements Evolving RequirementsValidated Requirements Capture allpossible needs Emerging needs as development progressesLean Startup experimentsDetail ofRequirementsagile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 9. Shifting Paradigms B.G.|A.G.I know that half of my advertising budget is wasted,but Im not surewhich half Lord Leverhulmehttp://www.marketingprofs.com/charts/2009/3161/more-smbs-use-online-media-than-traditional http://marketing.blogs.ie.edu/archives/2011/01/trends-in-online-advertising-in-2011-in-the-united-states.phpagile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 10. Validated Learning Understand the problem a feature will solve Decide how to ask if your customer cares Validate before investing complete costagile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 11. The Scientic MethodTheory Use the theory to make a predictionModify or changeyour theoryObserve Prediction Design anexperiment totest theRun the predictionexperiment Experimentagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 12. The Theory of General RelativityGradual acceptance ofGeneral Relativity over Einsteins Theory ofGeneral Relativity Newtonian Mechanics Einsteinsprediction(1907) Observations validated Light bends in a General Relativity gravitational eldWait for 1919 solarArthur Eddington eclipse to observeobserved that Light did whether or not light bend around the sunbends around the sun Solar Eclipse of 1919 http://thethoughtstash.wordpress.com/2011/01/03/how-eddington-demonstrated-that-einstein-was-right/agile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 13. Applied to an Online Experience User Model Use the modelto make apredictionModify or changeyour user model Validate PredictionDesign an experiment to test theRun thepredictionexperiment Experimentagile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 14. Designed by KISSmetricsagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 15. User Story (Journey) MappingElements of Customer JourneyDifferent Customer Your model forExperiencescustomer interaction covers each step of the customer Each row matches a differentjourneycustomer experienceStory Mapping Jeff Patton, all rights reserved, www.AgileProductDesign.com agile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 16. Vanity vs. Actionable MetricsVanity metrics (like web hits or number ofdownloads) which only serve todocument the current state of theproduct but offer no insight into how wegot here or what to do next. An actionable metric is one that ties specic and repeatable actions to observed results.agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 17. http://500hats.typepad.com/500blogs/2007/09/startup-metrics.htmlagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 18. DepersonalizeDecision Making HiPPOs rule the business world It is never about you, it cant beabout you Benchmarking is awesome Leverage competitive analysis Experimentation / testing rock Execution strategy: Transparency, standardization,looking outside in Be a slave to customercentricity (its about yourcustomers (internal & external))agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 19. I cant say it any better, HiPPOs rule the world, they over ruleyour data, they impose their opinions on you and yourcompany customers, they think they know best (sometimes theydo), their mere presence in a meeting prevents ideas fromcoming up.The solution to this problem is to depersonalize decisionmaking, simply dont make it about you or what you think. Gooutside, get context from other places. Include external orinternal benchmarks in your analysis. Get competitive data (weare at x% of zz metric and our competition is at x+9% of zzmetric).Be incessantly focussed on your company customers anddragging their voice to the table (for example viaexperimentation and testing or via open ended surveyquestions). Very few people, HiPPOs included, can argue with acustomers voice, the customer afterall is the queen / king! : ) Avinesh Kaushik, kaushik.netagile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 20. Interaction ModelBacklog ValidationExecutionTo Do DoingObservation Validated?Doing DeployedFor Who Idea #1.1The Is A That IdeaUnlike #1.2Our Product IdeaIdea #1.3Idea #1.5 Pathway #1.4#1IdeaIdea #1.6#2.1Idea Pathway #2.4 IdeaIdea#2#2.3#2.2 Level of Detail / Close to doneVisualize your User Interaction Model Always revisit validation tests for size The execution board is equivalent toand adjust it as you learn the teams task board, but at the IdeaCome up with ideas for each pathway Card levelPrioritize experimental testingSpend time on measurementinfrastructure (a test without ability tomeasure = an unvalidated feature) 21. How Much Effort on ValidationFeature usage in In 2009, Google ranapproximately 12,000 enterprise softwareexperiments, of which about NeverRarely10% led to business changes SometimesOftenAlways 7% Linus Pauling famously said, 13%"The best way to have a goodidea is to have a lot of ideas." 45% Focus on minimizing the 16%incremental cost of testing ideaswith your customers/users19%Standish Group - CHAOS Report, 1995, 2002agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright 2007-2013 22. Feedback loops makeall the difference In 1940, the chance of anAmerican being killed bylightning was about 1 in400,000.Today its 1 in 11 million. In November 2007, economists in the Survey of Professional Forecasters examining some 45,000 economic-data series foresaw less than a 1-in-500 chance of an economic meltdown as severe as the http://www.nytimes.com/2012/09/09/magazine/the-weatherman-is-not-a- one that would begin one moron.htmlmonth later.agile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007-2013 23. Lean Product Management The Value of Experiments Validate requirements trump everything Incremental cost of validation is falling Feedback means nothing without a modelThe race doesnt go to the one with the best idea,but to the one with the most tested ideasPhoto: ESAhttp://www.nbcnews.com/id/50677062/ns/technology_and_science-space reserved. Copyright 2007-2013 agile42 | We advise, train and coach companies building software www.agile42.com | All rights 24. Coming together is a beginning.Keeping together is progress.Working together is success.Henry Fordthank youdave.sharrock@agile42.comskype: dave.sharrock twitter: @davesharrockslides: slideshare.net/davesharrockagile42 | We advise, train and coach companies building softwarewww.agile42.com | All rights reserved. Copyright 2007 - 2009.