Upload
katie-paine
View
1.408
Download
5
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
People talk, We Listen
Tales from the Trenches, How Organizations are Measuring Value in Social MediaA presentation for Marketing Profs B2BKatie Delahaye PaineCEOKDPaine & Partners [email protected], IPR Measurement Commissionwww.instituteforpr.org
People talk, We Listen
Why Measure?
“The main reason to measure objectives is not so much to reward or punish
individual communications manager for success or failure as it is to learn from the
research whether a program should be continued as is, revised, or dropped in favor of another approach ”
James E. Grunig, Professor Emeritus, University of Maryland “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”
Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis
2
People talk, We Listen
Conquering your fears• If it’s not working, why
keep doing it?
I’m afraid that metrics will reveal that my program
isn’t working • If you’re deaf to the
conversation, only your enemies will hear it
I’m afraid of what I’ll hear
• It’s not about justifying, it’s about improving
I’m afraid I won’t be able to justify my
program/existence
• You should be fired for not showing any numbers
I’m afraid I’ll be fired for not showing the
right numbers
• Learn the language of business first, measurement will follow
I’m afraid to admit that I don’t know how to measure
People talk, We Listen
People talk, We Listen
MSM Online Social Media
Eyeball counti
ngHITS Engag
ement
A measurement timeline
People talk, We Listen
You can’t talk when the marketing department is talking sales, leads and revenueImpressions are impossible to count in social media Who cares about impressions when you can measure brand engagement ?Who cares about reach when you can measure revenue? Wouldn’t you rather be closer to your customers? Better measurement tools are available
Page 5
Old School PR 21st Century Role of PR
Social Media renders everything you know about measurement obsolete
People talk, We Listen
Why Traditional Metrics (AVEs) are like buggy whips
They both confuse activity with outcomeThe goal is to arrive at your destination, not to have a faster horse
They both were based on a flawed premise
Neither PR people nor horses respond well to beatingsComparing PR to Advertising is like comparing the Surrey With a Fringe On Top to the Space Shuttle – they serve different purposes
They both were rendered obsolete by technology
Page 6
People talk, We Listen
People talk, We Listen
12 Signs that it’s the end of measurement as we know it 1. The Dept of Defense considers Twittering and other
forms of social media critical to national security2. BMC Software measures communications
effectiveness based on contribution to EPS 3. BestBuy measures 85% lower turnover as a result of
its Blue Shirt community4. State Farm uses an internal blog to measurably
improve morale5. ASPCA correlates increases in on-line donations and
increased membership with its social media efforts6. HSUS generated $650,000 in new donations from an
on-line photo contest on Flickr7. NWF increased wildlife spotting as well as members
with its Twitter account 8. The Red Cross measures the effectiveness of Twitter
via lives saved and harm avoided9. ImmunizeBC measures success in terms of vaccines
given, awareness AND traffic 10. IBM receives more leads, sales and exposure from a
$500 podcast than it does from an ad11. 11 Mom’s turned around Walmart’s image and
delivered measureable increases in sales. 12. @comcastcares turned around Comcast’s customer
service reputation 13. A social media campaign for War Child delivered a
38% increase in donations and 300 new volunteers
People talk, We Listen
The Engagement Decision Tree
Awareness
Consideration
Preference Trial Purchas
e
FindObserv
e/Lurk
Participate
Engagement
Purchase/Act/Link/WOM
People talk, We Listen
The measurement fork in the road
Marketing/leads/sales
Reputation/relationships
To fix this Or get to this
People talk, We Listen
Goals drive metrics, metrics drive results
10
Reputation/
RelationshipsRelation
ship scores
Recommend-
ations
Positioning
Engagement
Sales
Engagement IndexCost per
customer
acquisition
Web analytics
Sales leads
Marketing Mix
Modeling
Goal
Metrics
People talk, We Listen
Listen, learn, change
2 4 4 4 2 26 5
2 4 2 2 22
8 85 9 9
9
24
16
27
10
20
15
4
2 13
2
4
30
5
2
12
16
17
0
10
20
30
40
50
60
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2007 2008
Positive
Neutral
Negative
Mentions
Tone of Conversation over time
People talk, We Listen
Negative coverage over time
Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr2006 2007 2008
0
5
10
15
20
25
21
2
5
21 1
4 42
1 12 2 2
1
10
9
4
1418
21
12
10
15
14
7
26
2
10
4
12
2
3 1
1
1
4
2
2
1
2
5
3
2
2
2
Entr
ies
People talk, We Listen
People talk, We Listen
Correlation exists between traffic to the ASPCA web site and the organization’s overall media
exposure
-
100,000
200,000
300,000
400,000
500,000
600,000
700,000
0
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
300,000,000
350,000,000
Web
Sit
e Vi
sito
rs
Expo
sure
Overall Exposure
Web Traffic
People talk, We Listen
Tying activity to development/marketing goals
0
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
300,000,000
350,000,000
Exposure
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
$1,600,000
$1,800,000
Donations
Overall exposure
Online donations
14
People talk, We Listen
What do you need to measure?Impact
(Outputs/Outtakes)
ROI (Outcom
es)
Did audience behavior change?
Did the right people show up?
Did your relationship change?
Did sales or revenue or profits increase?
Did you get the exposure you wanted?
Were your messages communicated
Did your relationships improve?
People talk, We Listen
Goals, Actions and Metrics Goal Action Output
Metric Outtake Metric
Outcome Metric
Recruitment
Twitter campaign
Number of tweets
% inclined to support% improvement in perception of organization
% increase in online requests for information % increase traffic to recruitment site
Increase in support
Flickr photo contest
Number of entrants
% likely to support the cause
Improvement in relationship scores
Message consistency
Blogger/Speaker training
% postings/articles containing one or more key messages
Share of positioning on key issues
% hearing message% believing message
% consistency in messaging between external and internal communicationsMarket shareRatio of on-message to off-message quotes
16
People talk, We Listen
The 7 steps to Social Media ROI
1. Define the “R” – Define the expected results?
2. Define the “I” -- What’s the investment?
3. Understand your audiences and what motivates them
4. Define the metrics (what you want to become)
5. Determine what you are benchmarking against
6. Pick a tool and undertake research7. Analyze results and glean insight,
take action, measure again
People talk, We Listen
People talk, We Listen
Step 1: Define the “R”
What return is expected?
What will be different when the program has launched?
What were you hired to do?
If you are celebrating complete 100% success a year from now, what is different about the organization?
If your department was eliminated, what would be different?
18
People talk, We Listen
Step 2: Define the “I”
What is the investment? PersonnelAgency compensationSenior Staff time Opportunity cost
19
People talk, We Listen
People talk, We Listen
Step 3: Define your audiences and how you impact them
There is no “audience.” There are multiple constituencies List every stakeholder
Where do they go for information? Don’t ask me, ask your customers. What’s important to them?What is the benefit of having a good relationship with that stakeholder group?
Understand your role in getting the audience to do what you want it to do
Raise awarenessIncrease preferenceIncrease engagement
20
People talk, We Listen
Step 4: Define your Key Performance Indicators (KPIs) carefully because you become what you measure
Cost savingsEfficiency
Cost per message communicatedCost per new lead/customer acquired
Productivity: Increase in employee engagement/moraleLower turnover/recruitment costs
Engagement: Ratio of posts to comments% of repeat visitors% of 5+min visitors% of registrations
Trust:Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)
Thought leadership: Share of quotesShare of opportunities
Message penetrationPositioning on key issuesImprovement in favorable/unfavorable ratioImprovement in Optimal Content Score (OCS)
21
People talk, We Listen
What makes a perfect communications KPI?
Gets you where you want to go (achieves corporate goals) Is actionable by individuals as well as departmentsContinuously improves your processes Is there when you need it
People talk, We Listen
Why an Optimal Content Score?
You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive:
Mentions of the brandKey messagesPositioningVisibility
Negative OmittedNegative toneNo key message
23
People talk, We Listen
How to calculate Optimal ContentQuality score +1 0 -1
Score Score ScoreTonality Positive 3 Neutral 0 Negative -3
Positioning Contains 2 Doesn't contain 0
Positions the competition favorably or positions Sargento negatively -2
Messaging Contains 3 partially contains 0
Does not contain or miscommunicates key message (neg mess) -1
Quotes Contains 1 Does not contain -1Competitive mention
Does not mention Competition 1
Competition mentioned prominently -3
Total Score 10 0 -10
Visibility Score+1 0 -1
Score Score Score
Brand Photo Contains 3 Doesn't contain 0Contains competitive photo -5
Dominance Focal point 3 Not a focal point -1Visibility Headline mention 2 Top -20 % of story 0 Minor mention -2Target publication Top Tier 2 2nd tier 0 Not on target list -2
Total Score 10 0 -10
Optimal Content Score
People talk, We Listen
Standard classifications of discussion
• Acknowledging receipt of information
• Advertising something• Answering a question• Asking a question• Augmenting a previous
post• Calling for action• Disclosing personal
information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise• Giving a heads up
• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll• Validating a position
People talk, We Listen
For all institutions, most postings were simply making an observation or distributing media.
Page 26
3
6
1
1
7
36
1
29
5
15
14
2
16
1
2
12
7
2
6
2
24
787
3
2
203
12
12
46
11
1
3
2
1
4
1
4
3
6
2
1
13
2
2
1
13
2
6
18
4
1
1
5
35
3
17
2
8
9
1
1
1
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Conversation Types
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
44.2%
6.5%
30.9%
49.5%
100.0%
100.0%
100.0%
1.6%
53.9%
100.0%
26.9%
23.1%
10.8%
38.7%
72.7%
10.9%
15.5%
46.1%
66.6%
27.3%
35.1%
39.7%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acknowledging receipt of information
Advertising Something
Answering a question
Asking a question
Augmenting a previous post
Calling for action
Disclosing personal information
Distributing media
Expressing criticism
Expressing support
Expressing surprise
Giving a heads-up
Giving a shout-out
Making a suggestion
Making an observation
Offering an opinion
Playing a game
Rallying support
Recruiting people
Showing dismay
Share of Engagement by Conversation Type - Institutional Blogs
Arizona State
Michigan State
Penn State
Purdue University
University of Michigan
cx
People talk, We Listen
Past PerformanceThink 3
PeerUnderdog nipping at your heelsStretch goal
Whatever keeps the C-suite up at night
Step 5: Define your benchmarks
27
People talk, We Listen
Internet dominated in 5 out of 6 months
Page 28
People talk, We Listen
Stanford dominates in broadcast coverage
Page 29
People talk, We Listen
Benchmarks put numbers in perspective
Twitter grade
Followers
Twitter rank
98.3
1295
43395
96.4
622
93718
99.99
15291
398
Shel Israel U of Penn Carnegie Mellon
Page 30
People talk, We Listen
MIT Berkeley see highest visibility in Social Media
Page 31
People talk, We Listen
Overview of Key Metrics
Bookmark.
Ext. Blogs
Inst. Blogs
YouTube
MSM
SOV 2% — 8% 9% 11% 7%
Popularity
230 bkmks
500/mo.
—20
links150k views
—
Engagement
59 cmts
1 day13
cmts2-12 cmts
2 cmts —
% Positive
20% 32% 54% 50% 15% 15%
% Negative
0% 0% 4% 0% 1% 2%
Strat. Mess.
40%† 18%† 42% 42%† 18% 38%
Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.
† Small base size. Findings are directional only.
People talk, We Listen
First: find out what already existsWeb analyticsCustomer Satisfaction dataCustomer loyalty data
Second: Decide what research is needed to give you the information you need:
Message content analysisRelationship surveys
Step 6: Conduct research (if necessary)
33
People talk, We Listen
Step5: Selecting a measurement tool based on your KPIs
Objective Metric Tool
Increase inquiries, web traffic, recruitment
% increase in traffic#s of clickthrus or downloads
Google Analytics, Omniture
Increase awareness/preference
% of audience preferring your brand to the competition
Survey Monkey, Zoomerang, PollDaddy, Vizu
Engage marketplace Conversation index greater than .8Rankings
Type pad, Technorati , Xinureturns
Communicate messages
% of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages
Media content analysis –Dashboards
% aware of or believing in key message
Survey Monkey, Zoomerang,Vizu, PollDaddy
34
People talk, We Listen
Your tool box needs:
1. A content source: Google News/Google BlogsTechnorati, Social Media Mention, Twazzup, Cyberalert, CustomScoop, e-WatchRadian 6, Techrigy, Visible TechnologiesRSS feedsSurvey Monkey/Zoomerang
35
People talk, We Listen
Your tool box also needs to include: 2. A way to analyze that content
Automated vs. Manual Census vs random sampleThe 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisionsDashboards to aggregate data
Tools:• Woopra• Net promoter score
• Hubspot Grader
• Xinureturns• Twinfluence• SPSS• Excel• Crimson Hexagon
• www.tealium.com
36
People talk, We Listen
Diversity dominates C-M discussions in Social Media
Page 37
People talk, We Listen
Your tool box also needs to include: 3. A way to measure
engagementThe conversation index=• Ratio of posts to
comments Relationship studiesThe engagement index
38
People talk, We Listen
People talk, We Listen
Managing engagement on your own property
% increase or decrease in unique visits In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email address
Courtesy of Eric Peterson
People talk, We Listen
Share of conversation vs share of engagement
Page 40
2
2
1
2
1
6
5
3
1
1
1
1
1
1
1
1
1
1
2
2
2
1
2
1
2
1
4
2
1
4
2
1
1
4
1
6
7
6
2
2
2
2
1
3
2
3
1
0 2 4 6 8 10 12 14 16 18 20
Faculty
Students
Research, Physical Sciences
Courses
Research, Earth Sciences
Projects, Non -Research
Financials
Alumni Topics
Research, Life Sciences
Staff
Admissions
Legal News
Other
Research, Agriculture
Policies
Institution, Overall
Campus Life
Research, Social Sciences
Share of Subject
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
15.3%
68.7%
100.0%
4.4%
33.3%
96.8%
28.6%
34.9%
12.5%
43.3%
28.6%
13.0%
38.3%
100.0%
23.6%
66.7%
6.3%
28.6%
20.8%
2.3%
95.6%
33.2%
5.8%
28.6%
100.0%
86.8%
13.0%
31.0%
22.1%
3.2%
71.4%
43.5%
18.8%
94.2%
56.7%
14.2%
13.2%
53.2%
28.4%
21.1%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Admissions
Alumni Topics
Campus Life
Community Relations
Courses
Events
Faculty
Financials
Institution, Overall
Inventions
Legal News
Other
Partnerships
Policies
Projects, Non - Research
Research, Agriculture
Research, Earth Sciences
Research, Life Sciences
Research, Other
Research, Physical Sciences
Research, Social Sciences
Staff
Students
Share of Engagement by Subject - ,External Blogs
Peer 1
Michigan State
Peer 2
Peer 3
Peer 4
People talk, We Listen
The vast majority of discussion in external blogs is neutral.
Page 41
23
29
12
14
20
5
8
4
1
4
0
5
10
15
20
25
30
University of Michigan Purdue University Penn State Michigan State Arizona State
Share of Tone
Negative
Neutral
Positive
71%
3%
29%
94%
83%
42%
58%
6%
14%
58%
42%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Arizona State Michigan State Penn State Purdue University University of Michigan
Share of Engagement by Tone - External Blogs
Negative
Neutral
Positive
People talk, We Listen
Aspects of relationships
Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship
42
People talk, We Listen
Components of a Relationship IndexControl mutuality
In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.
TrustThis organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it will do.
SatisfactionGenerally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.
CommitmentThere is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization more
Exchange relationshipEven though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.
Communal relationshipThis organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other people. (Reversed)
43
People talk, We Listen
How to implement relationship metrics
Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed
People talk, We Listen
Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, 13-month averageFigure out what worked and what didn’t workMove resources from what isn’t working to what is
Step 7: Analysis - -Research without insight is just trivia
45
People talk, We Listen
Thank You!
For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard:
www.themeasurementstandard.comFollow me on Twitter: KDPaineFind me on Facebook: Katie PaineReach me on Skype: KDPaine Or call me at 1-603-868-1550Check out our website www.kdpaine.com