Using analytics to evaluate digital communications strategies

Using analytics to evaluatedigital communication strategies

Dana Chinn

1. Developing measurement models 2 Basic site metrics2. Basic site metrics3. Some thoughts about external ratings and metrics4. Social media metrics

PUBD 526 – Public Diplomacy EvaluationApril 2012

Improving digital communication channels start with identifying what needs to changewith identifying what needs to change

Start here

not here

2

Our site has 5,000 monthly unique visitors.

Last Tuesday that story got 20,000 page views.

Our iPhone app was downloaded 10 000 times

We have 5 000 Twitter followers

Our iPhone app was downloaded 10,000 times.

We have 2,000 Likes on Facebook.

We have 5,000 Twitter followers.

3

Questions for a e-commerce company

Who came to our site? e.g., previous vs. new; high vs. low potential

How did they get here?

What did they look at?

Were they successful in getting what they wanted?

A simple e-commerce data story“Current and potential customers who typed in “t-shirts” in Google arrived on our t-shirts landing page.

1.5% of them made a purchase.”

4-- Corey Koberg, Cardinal Path

The questions for a content- or mission-based site are the same…

Who came to our site? e.g., previous vs. new; high vs. low potential

How did they get here?

What did they look at?

Were they successful in getting what they wanted?

h i h i l ll hi lik hi…so why is the typical story usually something like this?

Our site has 5,000 monthly unique visitors.Last Tuesday that story got 20,000 page views.y y g , p g

The average time spent on our site last week was 24 minutes.

We have 5 000 Twitter followers

Our iPhone app was downloaded 10,000 times.We have 2,000 fans on our Facebook page.

5

We have 5,000 Twitter followers.

ExamplePurpose of a community news programPurpose of a community news program

Increase civic engagement

in a community

by bringing stakeholders togetherthrough their shared need for through their shared need for

community news and information

6

Using data for decision-making is dependent first on a clearly defined target audience…

Alhambra “South LA”Boyle Heights

LA TimesCity of Alhambra City of Los Angeles

…and then on an equally well-defined multichannel strategy

7

Each channel reaches different audiences and hasunique functionality, so each needs its own measurement model.

A channel’s model – and metrics – should be developed based on its role vs. other channels.

Goal: What the org wants the channel to do

Key Performance Indicator: A metric crucial to the org’s survival

Target: The value of the KPI that will indicate success or failure

8

Segment: A group of visits or visitors, categorized by type and/or behavior, that is essential to reach the target

Sample measurement model for a website (eek!)

9

1. Establish program objectives.

What does your org want to do through all of its channels?What does your org want to do through all of its channels?

10

2. Establish – exactly – how each channel will contribute to each program objective.

What are the goals of your site? Why does it exist?

11



Getting a name and e-mail address is the mail address is the first indicator someone is engaging with you.

12

Audience info, obtained with permission, is perhaps the most important function of a site.



13

3. Decide which metrics will be the Key Performance Indicators for each channel goal. Establish the targets that will indicate

f ilsuccess – or failure.

Analyze audience segments: What type of audience behavior is ypaffecting the KPI?

Example: Example:

Site visitors who entered through search engines visited an average of once a week.

Is this good?

Yes – Our target was two times a month

No Our target was twice a No – Our target was twice a week! We just added additional resources to put new content up daily!

14

M b th t t ’ tti th itMaybe the content you’re putting up on the siteisn’t the content that people want!

Type of analysis used by J. Paul Getty Trust from “Web analytics success for government websites,” by Avinash Kaushik, Oct. 12, 2009



16



Nonprofits need to decide whether the number of

KPIs need to be specific to what the channel can actually do.

individuals is more of a priority than the total amount of

Example: A $20,000 sponsorship

money package will be sold through nondigitalfundraising

17

methods.

Sample measurement model for a website…

…repeat for e-mail, Facebook, Twitter...eek!

18

Web analytics is the analysis of data “to drive a continual improvement of the online experience…which translates into your desired outcomes.” y

Just one part of web analytics

19from Web Analytics 2.0 by Avinash Kaushik

Step 1: Understand the clickstream,Step 1: Understand the clickstream,or every action relevant to site goals

Behavioral research

What people did h hwhen they came to your site,

as captured by

an action taken on a keyboard or mouse

20

What actions indicate engagement?

Vi it l l Visit

R d/ i l

, regularly

Read/view content, a lot

Interact oftenInteract, often-- rate, print, vote, take a poll, click on an ad-- share, e-mail, comment, contribute

21

U i i it

Basic site metrics

Unique visitors

visit sites

and generatepage views

22

Total visits:One indicator of overall site performance One indicator of overall site performance

A i it i o ted A visit is counted

every time

someone comes to a site

Visits: the strongest metric available

An increase in visits is always good.-- More people are coming to your site.-- Returning people are coming more often.

A decrease in visits? Always bad.

23

Strong vs. weak metricsg

Strong metrics are useful tools that give clear indications that give clear indications

of what’s successful or not

Weak metrics…-- are conceptually flawed

“so what?” counts of things

c. Kyle Taylor

-- are technically flawedmetrics calculated byweb analytics systems c. Kyle Taylor

in ways that give unclear indications

…could be so misleadingthey could lead to bad decisions

24

Really weak metric #1: Unique visitors

A unique visitor is really a unique computer.

Unique visitors are either over-counted…

…or under-counted.

library, school, I t t

You’ll never know which or by how much.*

Internet cafe

25

* It doesn’t matter anyway….better to measure outcomes (did people do what you wanted?) than the number of people who came to your site.

An increase in page views can be good - or bad *

Really weak metric #2: Page views

An increase in page views can be good - or bad.*

Bad design navigation site architecture?Bad design, navigation, site architecture?Lots of page views, annoyed users

A redesign improved usability? Fewer page views happier users

?Fewer page views, happier users

Content that should be there but isn’t? Lots of page views, annoyed users?

Dynamic content? Fewer page views, happier users (probably)

* d ’ b (d d l d h d ) h

26

* It doesn’t matter anyway….better to measure outcomes (did people do what you wanted?) than the number of pages people went to when they came to your site.

Really weak metric #3: Time spent on site

An increase in average time spent on site can be good –or bad.*

Bad design, navigation, site architecture?g , g ,Lots of time spent, annoyed users

A redesign improved usability? Less time spent, happier users? p , pp

Technically flawed: Time spent is either over-counted or undercounted

?

* d ’ b (d d l d h d ) h

27

* It doesn’t matter anyway….better to measure outcomes (did people do what you wanted?) than how much time people spent on your site.

Wh di d t i When audiences - new and returning -come, are they staying?

Bounce rate percentof the landing page

Key Performance Indicator

of the landing pagewhere most visits start

“I came. I saw. I puked.”-- Avinash Kaushik on bounce rate

28A bounce: a visit with only one page view

The bounce rate of a landing page is much more actionable than the bounce rate of the entire site

Start by looking at the top landing page, or the page where most visits start

100%

Home page bounce rate: 43%51%

8,331

16,304 visits

Home page bounce rate: 43%visits started on content pages

57%4,547went to

43%3,426

left the site without going to another

pages

49%7,973

visitsat least one other page

to another page

started on the home page

Action: Let’s try

29

changing the home page

Three types of site metrics that can be usedto segment visitors by behaviorto segment visitors by behavior

1. Visitor acquisition: How did people get to the site? Is your marketing working?

Traffic sources: direct, referrals, search engines, campaigns (e.g., e-mail newsletters, ads)

2. Site behavior: What did they do once they got to the site?

Bounce rate of a landing page – did people leave after seeing only one page?Visits that included internal searchVisits that went to a particular type of content

3. Outcomes: Did people take the actions essential to the organization’s success?

Visit frequency and recency Sign-up for an e-mail newsletterBuy a benefit dinner ticketDonate; sign up for membership

Internal metrics External metricsfor

Strategic Planningfor

Marketing, Advertising

• Census data100% of all visitors, visits, page views, etc. in a site

• Panel, toolbar dataActivity from a sample of self-selected people. Usually not relevant for small sites.

• Analysis, decisions, actions, evaluation

• Marketing, trending, competitive analysis

• OmnitureGoogle AnalyticsWebTrendsetc

• comScoreNielsenCompeteetc.etc.

• Digital Analytics Association

etc.

• Interactive Advertising BureauBureau

31

Understand all of the implicationsof mapping metrics to goals

Increasing the number and percent of students with high SAT SAT scores…

…leads to a higher ranking by US News & World Report…

…which helps with donations, partnerships, faculty recruitment, etc.

Action: Centralize UG admissions decisions.

One result: Fewer minority One result: Fewer minority journalism majors coming from high school

Measurement models need to be reviewed regularly

Charities now need to optimize both “Financial Health” and “Accountability and Transparency” y p y

Why does Charity Navigator have ratings like this?

They’re not helpful to donorsdonors.

More importantly, they give no guidance to guidance to charities on what they should manage.

?

?

?

“No more newsletters mailed to you at h l i t

Is Liberty Hill’s site successful?

home….please register on the website, even if you’ve been a supporter for years ”for years.

KPI: Percent of print newsletter subscribers who newsletter subscribers who register

Target: 100%

Action needed based on analysis of data:

We need more [Eastside/media/women] to register.

Let’s try a [follow-up postcard/raffle/event].


Look at the site traffic trends after the flyer is trends after the flyer is mailed to each audience type.

home page bounce rate-- home page bounce rate

-- pop-up bounce rate

-- sign-up processg p p

Maybe you’re getting people to come to people to come to the site, but the site is losing them.


Is our site selling the number of tickets number of tickets, sponsorships, ads, etc. that we want?

Do other channels Do other channels work better for some items?

Analysis needed (multiple data sources needed):

-- Total sold last year from all sources, by time period, y p

-- Tickets (premier, standard); sponsorships (by type); ads (by type)

P t f i t d l -- Percent of registered people (by type) who buy tickets, sponsorships, ads


“Purchase your sponsorship, ticket and ads…online. Go to ticket and ads…online. Go to www.LibertyHill.org/dinner.”

Have different direct mail and e-mail campaigns for each audience segment; campaigns for each audience segment; have a different landing page with a unique URL (e.g., /campaign1; /campaign2) for each.

Sending people just to www.libertyhill.org

KPI: No. of tickets sold by campaign

is a wasted tracking opportunity!

Week of Jan 26 2012Sample e-mail newsletter metrics

Actions needed based on e-mail KPIs and ticket sales:

Week of Jan. 26, 2012

Delivered/sent: 970/1,269 (76% delivery rate)Target: 100% (indicates list quality)

We need to clean up our list.

Let’s try a different message for [environmentalists/past

Clicks/delivered: 36/970 (3.7% click-to-delivery rate)Indicates the relevancy of the e-mail content)

E-mail newsletter bounce rate: 78%* *Estimated - links need to be tagged to track all

program advertisers/individual seats].

*Estimated - links need to be tagged to track all traffic from a newsletter to the site


Where are we losing people in the purchase process? purchase process?

KPI: Percent and no. of visits that started with the dinner overview page and completed the four-step payment processp y p

Social media

Not only are the technologies new, but the metrics are as well.but the metrics are as well.

--Online Media and Marketing Association Metrics and Measurement program, June 2009

41

We know about “spray and pray” business models…

The social media ‘provide and pray’ model

Not having a purpose for social media efforts “….often leads to a worst practice we call ‘provide and pray.’

Leaders and managers provide access to a social technology, and

then pray that a community forms and that community interactions somehow lead to business value.

In most cases, adoption never really materializes; communities may form, but their activity is not considered valuable to the organization.”

“Social Media Success Is About Purpose (Not Technology),” by Anthony J. Bradley and Mark P. McDonald, Harvard Business Review, Nov. 1, 2011

Why should Why should organizations…

…have a Facebook page?

…tweet?

How important are either in achieving their goals?

“Effectively measuring social media,” Susan Etlinger, Altimeter Group webinar, April 2011

How important are either in achieving their goals? Are either of them essential, given

--a large part of the target audience may not be on Facebook and/or Twitter?on Facebook and/or Twitter?

--extremely limited resources?

Social media metrics are just as important as site metrics.

Typical (and rarely explicitly d) i l di bj ti expressed) social media objectives

for a news or nonprofit organization

1. Drive traffic to the websiteS i l di b d t d ti i h iSocial media as broadcast advertising mechanism

2 Engage 2. Engage -- current audiences-- new audiences it’s not getting elsewherenew audiences it s not getting elsewhere

Is this realistic?

Using a measurement model is even more important for social media!

“What matters is everything that happens after you post / tweet / participate….The ‘so what’ matters!”

1. Conversation: “Social means talk and listen and discuss. So why not measure that?”

2. Amplification: “The rate at which your followers take your content and share it through their co te t a d s a e t t oug t enetwork.”

3 Applause: “What does your audience like?”3. Applause: What does your audience like?

“Best Social Media Metrics,” by Avinash Kaushik, Oct. 10, 2011. Chart designed by Erik Ohlen

Understand how to measure Twitter,and you’ll understand how to measure social media

Content

FollowersFollowers

46not demographics or other typical mass media audience metrics

Twitalyzer – user centric TweetReach –tweet-centric

Overall impact of an accountLooks at tweets for keywords

Total reach = Number of unique people

Overall impact of an accountInfluenceReach – potentialReach – effective

Number of unique people following people who generated one or more tweets that contained the keyword

Number of:--Followers--Followers of followers

f h b f lly

Total exposure = Number of people (duplicated)

Number of

--References to the account by followers--Lists that include the accounttweets--Times the account is replied to --Times the account is retweeted Number of

Tweets with the keywordContributors

List of users who

Times the account is retweeted

generated the highest number of impressions (calculated with tweets, retweets, followers of the people who tweeted and people who tweeted and followers of the people who retweeted

“Twitalyzer and TweetReach – A Symbiotic Pairing for Twitter Analysis,” by Tim Wilson, March 8, 2011

Measurable tweets haveMeasurable tweets have…

1. A call to actionGo here…look…tell me

2 A link that you track with link (e g bit y) 2. A link that you track with link (e.g., bit.y) and web analytics tools

RT - retweet

3. #Hashtags and/or keywordsMT – modified tweetVia or HT – heard throughFavoriteLists

4. Topic or person-specific handles

120 or fewer characters not 140!

48

…120 or fewer characters, not 140!

Follower metrics – Avinash Kaushik“Retention is the key to growth.” DRAFT

Metric Tool Time period Analysis

Follower growth rate

Twitter Counter Over last three months

Compare rate against two others.

Relevant follower growth rate

Twitalyzer, others Over last three months

Estimate; sample

No. of unique, GraphEdge (paid) Over last three Compare against two legitimate followers; percent of overlap

months others.

Ch n ate Gain G hEd th 30 d h t Al l b Churn rate: Gain in followers vs. no.of unfollows

GraphEdge; others 30-day churn rate Also analyze number of relevant followers

Follower retweetpercent

Klout (velocity) Over last three months

Unique senders Klout (engagement)q ( g g )

Chart derived from “Actionable Twitter Analytics” Market Motive video

Interaction metrics – Avinash Kaushik


DRAFT

Click percent, by topic; top referrers

URL shorteners –bit.ly, tr.im

Weekly Put in context vs. news cycle. Analyze tweet volume.

No. of retweets, by topic

Twitalyzer,Retweetrank, Klout(reach), others

Weekly

No. of retweets Twitalyzer, Weekly Compare against per thousand followers, by topic

Retweetrank, others two others.

Replies sent per day

Twitter-friends Daily Are you havingconversations?

Replies received per day

Twitter-friends Daily Are peoplecommunicating with you?

I b d t i Kl t ( t)Inbound metrics per outbound message

Klout (engagement)


Outcome metrics – Avinash Kaushik


DRAFT

Visits and other site by “campaign”

Kl.am; Google Analytics campaign tracking

Need to determine campaign parameters. Set up custom GA

treport.

Sentiment or Emotion – What is your Twitter account saying

TweetPsych 23 dimensions such as learning, thinking, positive. Is your account account saying

about you? Is your account what you want for your brand and what you are trying to do?

Sentiment or Emotion – What are other people saying about

?

StatsIt

you?


Twitter metrics investment levels – Avinash Kaushik

From “Actionable Twitter Analytics” Market Motive video

Facebook Insights

Analyze trends in

• Posts• Posts

• People are Talking About This

• Weekly Total Reach

“…it is worse to post something that people do not react to, than to post nothing at all.”

“…a completely flat level of weekly reach”

“Each post is not really making any difference one way or another Th b f l h lki b hi b d i d ianother. The number of people who are talking about this brand is dropping.

“This is an indication that you are boring. This brand is likely doing the same few things over and over again, and

“You are slowly turning yourself into a commodity. It is just something people can follow every day, but you are not motivating

brand is likely doing the same few things over and over again, and people are getting bored with it….

j g p p y y, y gyour audience to act. You are not changing anything.”

“Beyond Facebook Analytics,” by Thomas Baekdal, Nov. 8, 2011

Be honest with metrics

Do 538 people REALLY “Like” this?this?

O d h j Or do they just want another sweepstakes entry?entry?

55

Audiences and actions differ by channel……so there are completely different metrics for each!

SOCIAL

And you need to report them all separately – you can’t add them up to get a total audience number

SITES MEDIA

Totals

MOBILE

1. Who? How many?In target audience?

2 f i i ?

? ? ? ? ? ?

3. What did they see?

2. No. of visits? How often? ? ? ? ? ? ?

? ? ? ? ? ?Did they get want they wanted?

4. Did they interact?

? ? ? ? ? ?

? ? ? ? ? ?

56

yWhat did they do?How much?

? ? ? ? ? ?

Evaluating effectiveness

1. Define a measurable audience.

2. Set specific goals across all channels; map metrics to goals. Segment. Measure

3. Set up each channel to measure specific actions

Optimize Analytics ReportAct

measure specific actions that indicate engagement and lead to outcomes

Analyze

Don’t forget about Voice of Customer

57