Measure What Matters: Measurement & ROI for Hyper Island Master Class

MEASURE WHAT MATTERSunderstanding measurement & roi for digital experiences

@FARRAHBOSTIC

4 THINGS:WHY

WHATHOW

(AND HOW TO SELL IT)

1. WHY YOU SHOULD MEASURE

BINGO

KPI SIGN-UPSTIME ON SITE

CPM IMPRESSIONS

CPA CPC REACH & FREQUENCY LIKES ENGAGEMENT

LTV SENTIMENT ANALYSIS ROI PAGE

VIEWSFOLLOWS

MENTIONS CTR DOWNLOADS

A WORD OF CAUTION

I notice increasing reluctance on the part of marketing executives to use judgment; they are coming to rely too much on research, and they use it as a drunkard uses a lamp post: for support, rather than for illumination.

David Ogilvy

WHY DO METRICS MATTER?

1. Metrics reduce arguments based on opinion.

2. Metrics give you answers about what really works.

3. Metrics show you where you’re strong.

4. Metrics allow you to test anything you want.

5. Clients love metrics.

http://bokardo.com/talks/metrics-driven-design-sxsw.pdf



Now we can easily assess pre-existing demand among our target: how easy is

user acquisition going to be?

Spontaneous product comments and mentions reveal sentiment around them,

and help us overcome existing boundaries for product adoption.

Digital traces reveal important drivers of human behavior that we can use in shaping brand’s online

actions.

DETECT DEMANDEXPLORE CONSUMER

SENTIMENTUNDERSTAND CONTEXT OF

BRAND & PRODUCT MENTIONS

Digital makes it is easier to detect problems or possible areas for

improvement related products/service use.

UNCOVER CONSUMER MOTIVATION

DEFINE CONSUMERS’ MULTIPLE IDENTITIES

OBSERVE SOCIAL GESTURES & CONTENT INTERACTION

Understanding different identities our audience assumes lets us help them in their role-playing and is also useful in

targeting.

Our targets’ gift-giving of content gives us insights into where we can insert brand

into their social exchanges.

WHAT ELSE CAN MEASUREMENT HELP YOU DO?

via @andjelicaaa

THERE ARE ONLY 3 RULES OF MEASUREMENT

Your metrics will be as UNIQUE as your business.

If metrics aren’t ACTIONABLE, they aren’t worth using.

DON’T TRY TO MEASURE EVERYTHING.

MEASURE WHAT MATTERS... TO WHOM?

To your BUSINESS

To your CUSTOMER

IT’S EASY TO FOCUS ON VANITY METRICS

“Numbers that make us look good but don’t really

help make decisions.” Eric Ries,

The Lean Start-up

WHAT’S A VANITY METRIC?


A lot of the time, we measure for a show reel, instead of for business results.



WHAT MATTERS IS VALUE CREATION

To your BUSINESS

To your CUSTOMER

CONVERSION ENGAGEMENT

2. WHAT YOU SHOULD MEASURE

USE ‘PIRATE’ METRICSAcquisition: users from channels come to site, landing page, widget, etc

Activation: users enjoy 1st visit, "happy" user experience

Retention: users re-visit multiple times

Referral: users like experience enough to refer others via email, links, likes, blogs, widgets, word-of-mouth, etc

Revenue: users buy, download, sign-up

*

*

http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version



WHAT ARE SOME ‘VALUE’ METRICS?

To your BUSINESS

To your CUSTOMER


Acquisition - drawing people into the brand experience

Revenue - converting visitors into customers

Referral - converting customers into advocates

To your BUSINESS

CONVERSION

BEWARE THE CTR...

The main metric for Acquisition isn’t CTR or “Site Visits”...

It’s CPA: Cost Per Acquisition (e.g., how much did you spend to get a new user)

But this by itself is irrelevant

If the CPA is higher than the lifetime value (LTV) of a new user, you’re in trouble

Activation - people enjoyed the experience...

Retention - enough to come back more than once...

Referral - and recommended the experience to others

To your CUSTOMER

ENGAGEMENT

HOW DO YOU MEASURE ACTIVATION?

A net promoter of +50 or better is a great score.

But it’s not good enough on its own - you have to ask people why they gave that score

And then put that information to use

HOW DO YOU MEASURE RETENTION & REFERRAL?

Returning Visitors

Registered Users

Frequency

Time on Site

Daily Active Users

Likes, +1s, RTs

Mentions

Email referrals

VALUE CREATION = CUSTOMERS BECOME ADVOCATES

To your BUSINESS

To your CUSTOMER


3. HOW YOU SHOULD MEASURE

APPS & SITES HELP ANSWER QUESTIONS...

I NEVER FELT LIKE I WAS LEARNING

ANYTHING IMPORTANT

@37SIGNALS

WHICH ONE IS MORE SUCCESSFUL?

http://www.slideshare.net/stueccles/lean-startup-metrics



EXTENDED SIGN-UP FUNNEL


WHEN DOES SUCCESS HAPPEN?



COHORT METRICS - CREATED PROJECT

WHERE IS SUCCESS COMING FROM?




DON’T HAVE ACCESS TO GOOGLE ANALYTICS?

First - get access. :)

Second - try QuantCast: Gives high-level insight into traffic sources, frequency and volume to site, and basic demographic and socio-economic data about your visitors. And it’s free.

OR THERE’S CHARTBEAT

For $10/month, try ChartBeat. It provides real-time tracking and engagement metrics like user paths, time spent with content, and micro-actions like clicks, downloads, and scrolls that show whether people are actively engaging with the site content.

FACEBOOKDashboard for a brand page provides information on traffic, content, user activity.

And starts to give a sense of more than just likes & mentions - the ‘reach’ of the users who are liking & talking about you on Facebook.

FACEBOOKThe advertising dashboard has been updated to focus on social metrics - like ‘social reach’ (the absolute number of people who saw an ad with social context - names/pictures of friends who connected with the ad’s destination) and ‘connection’s (social actions triggered by an ad, like Likes, app installs, event RSVPs)

http://www.secondshares.com/2011/05/10/facebook-advertising-embracing-the-second-internet-ethos/



TWITTERTwitter has not yet released its Analytics tool... So in the meantime - you can use twittercounter.com and tools like hootsuite/tweetdeck to track mentions & conversations

REMEMBER RULE 3 - DON’T MEASURE EVERYTHING -

[WE’RE LOOKING FOR WHAT MAKES MEANINGFUL DIFFERENCES]

•Bullet

•Bullet

Main Idea

TITLE

•Bullet

•Bullet

Main Idea

TITLE

•Bullet

•Bullet

Main Idea

TITLE

GOOGLE BING

4. HOW DO YOU ‘SELL IT’

HOW TO TALK TO CLIENTS & COLLEAGUES ABOUT METRICS

Setting Goals Accountability Transparency

Assessing Effectiveness Optimizing Learning

Good measurement practices improve: client-agency relationships

insightdesignresults

SETTING REASONABLE KPI’S

What are the business objectives?

What are the campaign objectives?

What are the channels/touchpoints?

What is the campaign experience & message?

What do we expect people to do?

A CREDIT CARD BRAND

The business objective: add new cardmembers

The campaign objective:use social media to drive new cardmember sign-ups

The touchpoints:Facebook brand page + Foursquare integration + Mobile-optimized Apply site

A CREDIT CARD BRAND

What is the most important KPI/success metric?

What would you measure/track to determine if you are succeeding?

What experiments would you run to find the best route to success?

A CREDIT CARD BRAND

What matters is acquisition - new cardmembers.

Acquisition comes from qualified leads completing applications - the result of smaller conversions:

•A larger prospect pool

•Prospects to qualified leads

•Qualified leads to applicants

Revenue, Retention &

Referral matter too:

• Cardmembers to active spenders

• Cardmembers to advocates

THANK YOU!

Technology

Measure What Matters: Measurement & ROI for Hyper Island Master Class