Social Media Monitoring Finding Relationships A Pilot Study July 4, 2011

Your brand is social.

From toilet paper to luxury vehicles, social plays a part in the customer experience of your brand.

People have always talked about brands and experiences

Only now these views and experiences are being shared online

And subsequently...

Agencies and brands are starting to take social media seriously.

With just about everyone offering some form of monitoring and analysis.

We are starting to ask better questions. Should we be only watching for brand and product mentions? What issues matter to our brand? What is the value of our social media efforts?

But we are ignoring a key question.

?

But we are ignoring a key question.

Why are we throwing away all of the data?

Report 1

Typically, a baseline social media audit is created based on an analysis of brand names, competitors, and relevant issues.

Report 1 Report 2

The next report, for a different time period, is created from new data. If comparisons to the first report are made, they are aggregate or trend-based.

Report 1 Report 2 Report 3

With each subsequent social media report, the process remains the same.

We have lots of pretty charts and surface-level analysis.

Pretty charts. “More people spoke positively about kittens this week.”

0

500

1000

1500

Week 1 Week 2 Week 3 Week 4

Positive Negative

…and “analysis”. “Here are our top kitten influencers this month.”

Twitter name Kitten Tweets Klout Score @justinbieber 618 90,194,000 @aplusk 6 12 @britneyspears 42 315 @oprah 9 120

But are all these kitten lovers net new people every month? Is this just a one time spike because of something in the news? Are we building relationships with any of these people?

A Real-World Example We decided to use Nikon (geo-filtered to Canada) as a pilot case. We explored using both Radian6 and Sysomos MAP. We settled on Sysomos MAP to pull the data we needed due to a feature that allowed us to pull an individual’s raw tweets with no search filters applied.

Some Charts Most of the tools can give us access to data and clues that lead to insights, but the built-in charts aren’t that helpful on their own.

Influence in Sysomos Sysomos gives us a list of the top Twitter influencers based on an algorithmic authority score of 7/10 or higher, sorted by volume.

It falls apart… With this chart, if we dive deeper and look at these users, we discover nearly all are spam or of little value to the brand.

Are these truly the most passionate and influential Nikon brand advocates?

Influence in Radian6 Some tools, like Radian6, integrate Klout or other similar theoretical metrics of absolute influence. But our attempts to generate lists of people who care about a brand with these tools generates primarily spammers and big name web celebrities.

We need deeper insights. With web analytics and email marketing, we track unique and repeat visitors. And traditional CRM programs track preferences, purchases, and engagement over time.

Deeper insights in social. Why not track the same things on social media? Who are our true brand advocates and what can we learn about them beyond their interest in our brand? Do the people who are passionate about our brand have anything in common?

What about Social CRM? Some are looking at Social CRM to solve this problem.

Most CRM tools were designed with the assumption of a near-perfect signal-to-noise ratio. To be useful, there should be zero noise in the system.

These systems fall apart when you start throwing hundreds of thousands of users and tweets at them. e.g. There was an estimated 501,618 “Nikon” tweets in the past 6 months.

Social CRM

A new approach. What if we started to wonder about the people who are consistently talking about our brand or issue? What could we learn if we weren’t wiping the slate clean every time we run a new search?

A new approach. We believe these are people who are passionate about your brand. We call them the consistently engaged. They are NOT the most influential Klout or PeerIndex users. They are the people who have an ongoing relationship with your brand. Good, bad or otherwise.

Report 1 Report 2 Report 3

Simply put, we find the individuals who are consistently talking about and engaging with your brand across time periods.

Looking across periods.

Our hypothesis The consistently engaged who love your brand are: •  Findable •  Of value to the brand

(promotions, research, reach) •  Ignored by most brands

Looking across quarters. We extracted all tweets over a two-quarter period for Nikon and looked at two lists:

1.  Top Twitter users by volume (4+ mentions)

2.  Top Twitter users by volume (4+ mentions) who are also consistent across multiple quarters

Finding the consistently engaged users

@bdicroce

@caltek79

@CFN007

@cicychan

@Digital_zin

@dorisdays88

@ginette4

@jedgar

@JohnBiehler

@justinlee_

@jwsutts

@kinematicdigit

@KingKabuz

@Kosmatos

@Lalalalal8

@maryellenphotos

@missemcee

@missfish

@msjconnoly

@neek247

@dorisdays88

@jedgar

@justinlee_

@maryellenphotos

@msjconnolly

@neek247

@nortonphoto

@jonah_lewis

@quotetasticc

@RajaKalsi

@RDslva

@Redawna

@samobeid

@scottoakley

@ShaniceAshley_

@stephaniefusco

@vickiesbphoto

@wifewithknives

@ZtheWayfarer

@zeefred

Volume Consistent

The list on the left includes people with the most volume over the past six months, in regards to the brand in question. The list on the right includes people who consistently engaged with or mentioned the brand a minimum of four times in both quarters.

Finding the consistently engaged users

@bdicroce

@caltek79

@CFN007

@cicychan

@Digital_zin

@dorisdays88

@ginette4

@jedgar

@JohnBiehler

@justinlee_

@jwsutts

@kinematicdigit

@KingKabuz

@Kosmatos

@Lalalalal8

@maryellenphotos

@missemcee

@missfish

@msjconnoly

@neek247

@dorisdays88

@jedgar

@justinlee_

@maryellenphotos

@msjconnolly

@neek247

@nortonphoto

@jonah_lewis

@quotetasticc

@RajaKalsi

@RDslva

@Redawna

@samobeid

@scottoakley

@ShaniceAshley_

@stephaniefusco

@vickiesbphoto

@wifewithknives

@ZtheWayfarer

@zeefred

Volume Consistent

The green boxes show the people consistent between the two methods. If you are using the standard method (left side) then you are ignoring up to 70% of the people consistently engaging with your brand. We need to focus on the users who are consistent in order to find the true brand advocates.

Rethinking Influence The list of influencers most social media tools give us aren’t very meaningful as they tend to use the standard method on the left or just find the big name web celebrities. With our Nikon example, they ignore a huge segment of people who are passionate about Nikon but will never appear on any top list.

The Consistently Engaged Finding these people was just the first step. Now we need to understand: 1.  Who are these people? 2.  What are their other interests? 3.  Their relationship to our brand 4.  Their potential value

@dorisdays88

@jedgar

@justinlee_

@maryellenphotos

@msjconnolly

@neek247

@nortonphoto

@jonah_lewis

@quotetasticc

@RajaKalsi

@RDslva

@Redawna

@samobeid

@scottoakley

@ShaniceAshley_

@stephaniefusco

@vickiesbphoto

@wifewithknives

@ZtheWayfarer

@zeefred

Consistent

Understanding these people All social media monitoring tools start with a filter – a way to cross-section massive amounts of data (in this case a search for “Nikon”).

Analyzing this data to understand who these people are is critically flawed. It will only show us insights based on a limited set of their conversations – those that match our original search terms.

It’s akin to doing an ethnography study where we ignore all the things the individual does or says that don’t directly involve our product. Not particularly valid.

So we remove the filter. The “from:username” operator in Sysomos MAP allows us to pull all user tweets over any period of time (sample based in some cases). This is time consuming to do but not overly difficult. But now we have hundreds of thousands of lines of data. How do we analyze this?

Not in the SM tool, sadly The text analytics tools in Sysomos MAP quickly fall apart when the filter is removed and we are looking at everything a group of users say.

So we need to export all the activity and analyze it using a qualitative tagging methodology

Qualitative vs. Quantitative Research Quantitative content analysis is about counting content. It provides numbers that represent frequencies or occurrences. Qualitative content analysis is about the quality of content. It provides insights into recurring themes that can lead to the development of valuable insights and recommendations.

Source: Zhang, Y. , & Wildemuth, B. M. (2009). Qualitative analysis of content. In B. Wildemuth (Ed.), Applications of Social Research Methods to Questions in Information and Library

Qualitative vs. Quantitative Research Rather than reporting how many times a brand has been mentioned [quantitative], we can look at the quality of the message [qualitative]. Rather than reporting frequencies, the application of qualitative tools to social media data will allow us to gain insights from a data sample that is traditionally overlooked.

Conducting a qualitative analysis of all expressions from identified

users of interest.

Conducting a qualitative analysis

of isolated individual

expressions of interest.

Qualitative Analysis Approach 1.  Identify your area of interest. 2.  Identify goals: e.g., understanding what common characteristics

and interests of a specific customer population might exist. 3.  Translate your goals into a structured and hierarchical coding

frame (either inductively or deductively). 4.  Make both categories and individual codes as mutually exclusive

as possible. 5.  Create coding rules to ensure that multiple coders can

consistently make the same coding decisions. 6.  Test your coding frame by calculating inter-coder reliability (or

other fancy reliability method). 7.  Assign expressions (or whatever your agreed upon unit of

analysis might be) to codes. 8.  Analyse the coded data by individual category and code.

Compare codes, find relationships, uncover themes, discover insights and develop recommendations.

Source: Zhang, Y. , & Wildemuth, B. M. (2009). Qualitative analysis of content. In B. Wildemuth (Ed.), Applications of Social Research Methods to Questions in Information and Library

What did we find? For this pilot, we analyzed the top 20 consistently engaged which were responsible for over 20,000 social media mentions in the quarters analyzed. Only a partial analysis was performed to validate that the most consistently engaged individuals identified would be of value to the brand.

What did we find? Nearly all of our consistently engaged users sample self-identify as amateur or professional photographers. Most have a relationship with Nikon – either because they own Nikon products or discuss Nikon products and news. 1 out of 20 self-identified as a professional photographer and 14 out of 20 stated they own Nikon equipment.

Their relationship with Nikon Recently bought a Nikon D3100 and looking to purchase some additional lenses

Wants to buy a Canon 60D but has an interest in Nikon gear and often chats with Nikon users

Hobbyist looking to move up to pro-level Nikon gear

Owns a Nikon D300 and mostly uses older manual-focus lenses. Just bought first “modern” Nikon lens.

Angry with Nikon Canada about a poor warranty service experience

Professional Canon shooter but blogged about a positive experience shooting with Nikon equipment

Their other interests Users are more likely to talk about iPhones vs. BlackBerry devices

–  74% of all iPhone mentions were positive in nature

–  73% of BlackBerry mentions were positive in nature

iPhone users more likely to talk about photo related apps than BlackBerry users

–  Almost half of all iPhone mentions had to do with a photo-related app

–  Only 4% of BlackBerry mentions were about photo-related apps

iPhone vs. Blackberry

iPhone

BlackBerry

Their other interests The consistently engaged Nikon users over the time period analyzed were more likely to talk about Starbucks than any other coffee brand

–  87.5% of these mentions were positive in nature

–  The second mentioned brand was Tim Hortons, but all mentions were neutral or negative in nature

People spoke of coffee 1.7 times more often than tea. Of the tea mentions, DavidsTea was the most mentioned product/brand.

Coffee Mentions

Starbucks

Tim Hortons

Quick Recap 1.  The consistently engaged are

a better way to find the relationships you are building (or not building) in social.

2.  Finding the consistently engaged is simply a matter of not throwing the data away and paying attention to who is engaging with your brand across time periods.

3.  Deeper analysis can lead to better insights into who your brand advocates are, their needs and their interests.

The “Secret” Sauce

Our Methodology A)  Finding consistent users across two quarters 1.  Extract two time periods of data from Sysomos MAP.

Export. 2.  Merge data into a single spreadsheet with each

quarter in a separate column. 3.  Cross-reference columns to find users who are

consistent across quarters. 4.  De-dupe results in the third “result” column to get a

complete list of consistent users.

Our Methodology B)  Determining tweet volume for consistent users 1.  Merge Tweet content from both quarters into a single

speadsheet column 2.  Sort alphabetically and run subtotal based on

usernames. Sort subtotals by volume. 3.  Set a threshold (we used 4 tweets across quarters) 4.  Cross-reference with list (result of part A) of

consistent users across quarters. 5.  Manually remove any spammy users above threshold. 6.  Extract user list of top 20-30 “casually engaged.”

Our Methodology C)  Removing the filter and exporting all user tweets for

“casually engaged” list 1.  Using the “from:username” operator in Sysomos MAP,

extract all top 20-30 user tweets to CSV over same two quarter period (Export is limited to 5000 tweets so generally each user must be done separately)

2.  Merge resulting CSV files for each user into a single spreadsheet. Discard all data except tweet content.

3.  Set up qualitative tagging system and methodology. 4.  Import data in NVivo or similar qualitative analysis

tool.

Our Methodology D)  Running qualitative analysis to find insights about

users 1.  Identify an area of interest and goals. Translate these

into a structured coding frame. 2.  Tag the data according to the coding frame. 3.  Analyze the data based on the categories and codes.

Pull insights and findings. 4.  Put together recommendations based on key findings

Colophon

Sean Howard is a freelancer and Associate at Thornley Fallis and spends his life searching for what drives and identifies the most passionate online and offline.

Eric Portelance is a Strategist at Thornley Fallis and a regular podcaster on the show Attention Surplus. He’s passionate about using new technologies to build engaging online experiences.

Katie Charbonneau is an Account Coordinator at Thornley Fallis with an MSc in Media and Communications. She’s a data junkie with a keen interest in American politics.

Twitter: @passitalong Twitter: @eportelance