Product Management For Version 1 Products: Creating Something from Nothing

Copyright © 2009 YourVersion

Dan OlsenDan OlsenCEO, YourVersionCEO, YourVersionNov 2, 2009Nov 2, 2009

Creating Something from Nothing:Product Management for v1 Products

http://www.yourversion.com/

http://www.yourversion.com



What IWhat I’’m Coveringm Covering

Understanding customer needs

Prioritization and maximizing ROI on engineering resources

Validating your product concept

UI design

Usability testing

Will post slides to slideshare.net/dan_o


My BackgroundEducation

BS, Electrical Engineering, NorthwesternMS, Industrial Engineering, Virginia TechMBA, StanfordPHP, MySQL, JavaScript, XHTML, CSS, UI design

18 years of Product Management ExperienceManaged submarine design for 5 years5 years at Intuit, led Quicken Product ManagementLed Product Management at FriendsterOlsen Solutions LLC, PM consultant for startupsCEO & Founder of YourVersion, real‐time discovery startup


For a v1 Product,Which is More Important?

Qualitative Qualitative Information?Information?

Quantitative Quantitative Information?Information?

oror

“When you’re Small, start with Qual!”


How is developing a v1 Product Different from developing a Later Stage Product?

With a v1 stage product, you haveWAY MORE uncertainty about:Who your target customers really are

Which customer needs you should address

How to best meet those needs

What product design works best

These are qualitative learnings/decisions

Quantitative is also valuable (later)


What’s the Formulafor a Winning Product?

A product that:Meets customers’ needs

Is better than other alternatives

Is easy to use

Has a good value/price


Russians: pencil

NASA: space pen($1 M R&D cost)

Example:Ability to write in space (zero gravity)

Problem Space vs. Solution Space

Problem SpaceA customer problem, need, or benefit that the product should address

A product requirement

Solution SpaceA specific implementation to address the need or product requirement


Problem Space vs. Solution SpaceProduct Level

Problem Space(user benefit)

Solution Space(product)

TurboTax

TaxCut

Pen and paperPrepare

my taxes

File my taxes


Problem Space vs. Solution SpaceFeature Level

Problem Space(user benefit)

Solution Space(feature)

Gmail importerMake it easy

to share a link with my

friends

Allow me to reuse my

email contacts

Design#1

Design#2

Design#3

DesignPreview with checkboxes

User can edit before import

#1 No No

#2 Yes No

#3 Yes Yes


The Customer Benefits “Ladder”

HigherHigher‐‐levellevelbenefitbenefit

(more abstract)(more abstract)

Quicken makes it easy for me Quicken makes it easy for me to balance my checkbookto balance my checkbook

……which gives me a clear picture which gives me a clear picture of how much money I haveof how much money I have

……which makes me feel more in which makes me feel more in control of my financescontrol of my finances

……which means one less thing to which means one less thing to worry about in my hectic lifeworry about in my hectic life

LowerLower‐‐levellevelbenefitbenefit

(more specific)(more specific)


How Do You Prioritize User Benefits and Product Features?

Need a framework for prioritizationWhich user benefits should you address?

Which product features to build (or improve)?

Importance vs. SatisfactionImportance of user need (problem space)

Satisfaction with how well a product meets the user’s need (solution space)


High Importance + Low Satisfaction =Opportunity

Impo

rtance of U

ser N

eed

Impo

rtance of U

ser N

eed

User Satisfaction with Current AlternativesUser Satisfaction with Current Alternatives

CompetitiveMarketOpportunity

LowLow HighHigh

LowLow

HighHigh

Not Worth Going After


Kano Model: User Needs & Satisfaction

User SatisfactionUser Satisfaction

User DissatisfactionUser Dissatisfaction

Performance (more is better)

Delighter (wow)

NeedNeednot metnot met

NeedNeedfully metfully met

Must Have

Needs & features migrate over time


Importance vs. SatisfactionAsk Users to Rate for Each Feature

98

8784

8679 847055 80

7280

75

4150

55

60

65

70

75

80

85

90

95

100

40 50 60 70 80 90 100

Satisfaction

Impo

rtan

ce

Recommended reading:“What Customers Want” by Anthony Ulwick

BadBad

GreatGreat


Prioritization and Scope

Customer value is only half the equation

How much engineering effort will it take?

Need to consider value and effort (ROI)

Ruthlessly prioritize: rank order

Be deliberate about scope & keep it smallIt’s easy to try to do too much

Strategy = deciding what you’re NOT doing

Break features down into smaller chunks

Smaller scope → faster iterations → better


Prioritizing Product Ideas by ROIPrioritizing Product Ideas by ROI

Investment (developer‐weeks)

Return (V

alue

Created

)

Idea C

Idea B

Idea D

Idea A

Idea F

1

1

2

3

4

2 3 4

?


The Art of Estimating Customer Value for a Particular Product Idea

For any product idea, the Customer Value it creates is higher:The higher the importance of the user need

The lower the user satisfaction with the status quo

The higher the # or % of users to which the idea applies

The more frequently the product idea will be used by users


Have to Prioritize Across Multiple Dimensions At The Same TimeCu

stom

er Value

Custom

er Value

TimeTime

Customer Customer UnderstandingUnderstanding

Functionality

Quality

Ease of Use


Case Study: Product ValidationSummary

My consulting client, CEO of TrustedID, had an idea for a new product

Team: me, CEO, head of marketing, UI design consultant

4 weeks from 1st meeting to validated product concept

Paid prospective users $1,500 ($75 x 20)

1 round of iteration on product concept


Product Concept was a “marketing report”that let consumers more directly control the direct mail that they receiveConcept was fuzzy with various components, so we broke it into 2 different “flavors”:

#1 “Shield”: Service to reduce/stop junk mail#2 “Saver”: Opt in to receive money‐saving offersWithin each concept, got feedback on modules that mapped to a specific user benefit

Worked with UI designer to create paper mockups of pages for each concept (5 pages each)

Case Study: Product ValidationDeveloping Product Concept




Clustering Potential User Benefits to Create Product Concepts

Reduce Junk Mail

Find out what “they” know about you

Money Saving Offers

Compare Yourself to Others

Social Networking

Marketing Report

Marketing Score

Marketing Profile

Save Trees

“Shield” Concept “Saver” Concept


Telephone recruit of prospective customersWrote screener using intuition for psychographic segmentation

Wanted users who work full‐time & use internetFit for opt‐in concept: use coupons, Costco membershipFit for anti‐junk mail concept: use paper shredder, block caller ID

Recruiters used screener to recruitScheduled 3 groups of 2 or 3 people to discuss each concept for 90 minutesModerated each group through the paper mockups to hear their feedback

Case Study: Product ValidationRecruiting People


Case Study: Product ValidationFindings on Concepts &User Benefits

Reduce Junk Mail

Find out what “they” know about you

Money Saving Offers

Compare Yourself to Others

Social Networking

Marketing Report

Marketing Score

Marketing Profile

Save Trees

Legend

Strong appeal

Somewhat positive

Low appeal

“Shield” Concept “Saver” Concept


Case Study: Product ValidationLearnings from Research

Learned that “Shield” (anti‐junk mail) concept was stronger than “Saver”People didn’t like many of the “Saver” concept componentsLearned users’ concerns / questions about “Shield”conceptRefined “Shield” concept:

Removed irrelevant componentsImproved messaging to address user concerns / questions

Validated revised “Shield” concept with quick 2ndround of tests

No customer concernsClear willingness to pay



Case Study: Product ValidationSummary

4 weeks from 1st meeting to validated product conceptCost $1,500 to talk with 20 users ($75 each)1 round of iteration on product conceptIdentified winning concept that users are willing to pay $10/month forTrimmed away non‐valuable piecesYou can achieve similar results


User Benefits vs. Ease of Use

Q: If two products equally deliver the exact same user benefits, which product is better?

A: The product that’s easier to use

‘Ease of use’ provides benefitsSaves time

Reduces cognitive load

Reduces frustration

Not many companies excel at UI design

‘Ease of use’ can be differentiator


The UI Design IcebergThe UI Design Iceberg

VisualDesign

InteractionDesign

InformationArchitecture

ConceptualDesign

Recommended reading: Jesse James Garrett’s“Elements of User Experience” chart, free at www.jjg.net

What most people seeand react to

What good PMs and Designers think about


Elements of User Interface Design

Consists of Three Distinct Elements:Information Architecture

Structure and layout at both site and page levelHow site is structured (sitemap)How site information is organized (site layout)How each page is organized (page layout)

Interaction DesignHow user and product interact with one anotherUser flows (e.g., navigation across multiple pages)User input (e.g., controls and form design)

Visual Design“How it looks” vs. “What it is”, often called “chrome”Fonts, colors, graphical elements


The Value of Usability Testing

Critical to talk with customers 1‐on‐1Gain better understanding ofUsability issues with your productCustomer needs and problemsWhat alternatives customers are using,pros & cons of each, customer preferencesQA: use cases & bugs you haven’t seen

Really a “user learning” sessionMake test as real for user as possible


“Ramen” Usability for Startups

Anyone can do it!

Ingredients:1 user with their laptop

1 desk

1 person to conduct usability

Pen and paper

N optional observers

N+2 chairs


Typical Format for Customer Session

5 ‐ 10 min: Ask questions to understand user needs and solutions they currently use

30 ‐ 50 min: Usability testNon‐directed as much as possible

When necessary, direct user to attempt to perform a specific task

5 ‐ 10 min: Wrap‐upAnswer any user questions that came up

Point out/explain features you want to highlight


Dos & Don’ts of Conducting Usability

DoExplain to the user:

Their usability test will help improve the productNot to worry about hurting your feelings“Think Aloud Protocol”

Ask user to attempt the task, then be a fly on the wallAsk non‐leading, open‐ended questionsTake notes and review them afterwards for take‐aways

Don’tAsk leading questions“Help” the user or explain the UI (e.g., “click over here”)Respond to user frustration or questions (until test is over)Get defensiveBlame the user


And now,And now,for something completely for something completely

differentdifferent……

Quantitative!Quantitative!(finally)(finally)


Approaching Business as an Approaching Business as an Optimization ExerciseOptimization Exercise

Given reality as it exists today,Given reality as it exists today,

optimize our business resultsoptimize our business results

subject to our resource constraints.subject to our resource constraints.


Profit = Profit = RevenueRevenue ‐‐ CostCost

Unique VisitorsUnique Visitors x x Ad Revenue per VisitorAd Revenue per Visitor

Impressions/VisitorImpressions/Visitor x Effective CPM / 1000x Effective CPM / 1000

Visits/Visitor x Visits/Visitor x PageviewsPageviews/Visit x Impressions/PV/Visit x Impressions/PV

New VisitorsNew Visitors + Returning Visitors+ Returning Visitors

Invited VisitorsInvited Visitors + Uninvited Visitors+ Uninvited Visitors

# of Users Sending Invites x Invites Sent/User x Invite Conv# of Users Sending Invites x Invites Sent/User x Invite Conversion Rateersion Rate

Define the Equation of your BusinessDefine the Equation of your Business““Peeling the OnionPeeling the Onion””

Advertising Business Model:


How to Track Your MetricsTrack each metric as daily time series

Create ratios from primary metrics: X / YExample: How good is your registration page?Okay: # of registered users per dayBetter: registration conversion rate =

# registered users / # uniques to reg page

DateUnique Visitors

Page views

Ad Revenue

New User Sign‐ups …

4/24/08 10,100 29,600 25 490

4/25/08 10,500 27,100 24 480

…


Sample Signup Page Yield DataSample Signup Page Yield Data

Daily Signup Page Yield vs. TimeNew Registered Users divided by Unique Visitors to Signup Page

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10

Dai

ly S

ignu

p Pa

ge Y

ield

Changedmessaging

Added questionsto signup page

Started requiringregistration


Continuous Feedback andContinuous Feedback andImprovementImprovement


Adding Metrics and Feedback to your Product Process

PlanPlan

DesignDesign

DevelopDevelop

BusinessObjectives

ProductObjectives

Prioritized Feature List

Scoping

Requirements & Design

Code Test Launch

Site Level

Feature Level

OptimizeOptimize Metrics & User Feedback


Optimization through Iteration:Optimization through Iteration:Continuous ImprovementContinuous Improvement

Measurethe metric

Analyzethe metric

Identify top opportunitiesto improve

Design & develop the enhancement

Launch theenhancement

Learning

Gaining knowledge:

• Market

• Customer

• Domain

• Usability


v1 Product Management Cheat Sheetv1 Product Management Cheat Sheet

Truly understand your customer needs

Get clear on your value proposition

Prioritize your feature set based on ROI

Validate your product concept with users

Launch v1

Talk with users 1‐on‐1 and get feedback

Learn and iterate


Great way to stay on top of your interests

Real‐time discovery engineDiscovers new, relevant content tailored to your specific interests

News, Blogs, Tweets, Webpages, Videos, Products

Bookmark and share via email, Twitter, Facebook

Launched in Sep at TechCrunch50

Free iPhone app

Check it out at www.yourversion.com




Questions?@[email protected] www.yourversion.com




http://twitter.com/danolsen

mailto:[email protected]