Upload
john-blossom
View
1.262
Download
2
Embed Size (px)
DESCRIPTION
Shore Communications Inc. President John Blossom overviews the state of text analytics applied ot social media and provides a case study of how real-time analysis of social media enabled Neurolingo to develop Sentibet, a demonstration of sports event forecasting that correctly predicted the 2012 SuperBowl results, as well as the NCAA college basketball finals and weeks of Premier League and Championship League football results.
Citation preview
SOC A M A ANA CSSOCIAL MEDIA TEXT ANALYTICSMINING VALUE FROM PREDICTIVE INSIGHTS
John Blossom, President, Shore Communications Inc.24 April 2012
Topicsp
“The Flood” – Drowning or Swimming in Social Media?
1
The Flood Drowning or Swimming in Social Media? Predictive Analytics – Understanding Focus & Intent Case Study – Sentibet.com Case Study Sentibet.com Lessons Learned
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Th Fl dThe Flood2
24 April 2012
Copyright © 2009-2010 Shore Communications Inc. - All Rights Reserved
The Flood: Global3
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
The Flood: Real-Time, All the Time,
12,233 tweets/second
4
12,233 tweets/secondfor NFL SuperBowl
vs. NYSE record: vs. NYSE record:121,257 messages/sec.(5 August 2011)
As in finance, the value of social media analysis isshifting towards seeing shifting towards seeing patterns that enable the forecasting of intents
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
The Flood: Adapting to Complexityp g p y5
Complex Word Analysis
P di ti M k
Simple/No Complex
Predictive Analytics
Market Sentiment
Simple/No Grammar/
lexicographic Analysis
Complex Grammar/
lexicographic Analysis
Events Triggers
Alerts Streams
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Simple Terms Matching
The Flood: The Social Media Gapp
Alerts
6
Alerts Search filters via search engines, platforms & tags
Events Triggers Events Triggers Dow Jones Enterprise, InsideView
Market Sentiment Market Sentiment Radian6, Attensity, etc.
Predictive Analytics Predictive Analytics Early days, most focused on statistical analysis of terms
clustering clustering
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
P di i A l iPredictive Analytics7
24 April 2012
Copyright © 2009-2010 Shore Communications Inc. - All Rights Reserved
Predictive Analytics and the Floody
Learning to swim in data, not drown in it
8
Learning to swim in data, not drown in it Infer a focus from which you can infer intent
Identify the focus of specific cohorts Identify the focus of specific cohorts Use to forecast intents based on lexicographic analysis
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Making Predictive Analytics Workg y
Understand both:
9
Understand both: Vocabulary Complete grammar/syntax Cohort Complete grammar/syntax
Understand it specific to: Cohorts Cohorts LanguageGeography/dialect
PlatformTopic
Interests
Topics Platforms Terms, jargons, idioms
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Formats, handles, syntax
Forecasting from Social Mediag
Predictive analytics
10
The more sophisticated Predictive analytics enables pre-hypothesis testing of intents based
The more sophisticated the analysis, the more accurate the insights g
on semantically extracted sentiment &
gfrom smaller samples, enabling analysis of
forecasting statements highly focused cohorts
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
C S d S ibCase Study: Sentibet.com11
24 April 2012
Copyright © 2009-2010 Shore Communications Inc. - All Rights Reserved
Sentibet.com – Forecasting Sportsg p
By Neurolingo, L.P.
12
By Neurolingo, L.P. Complex lexicographic
analysis of social mediaanalysis of social media Yields event-specific
forecasts based onforecasts based onfan message analysis
Identifies, feelings, wishes Identifies, feelings, wishesand predictions
Applies proprietary formula for overall forecast Applies proprietary formula for overall forecast
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Sentibet™ – how it worksBlogs Social
NetworkingFiles, DBs, Reports
Subscription Content
Web Content
Sentibet™ pulls content from any available source via Neurolingo platform
Extracted Information are Extracted Information are further processed and analyzed to d
Sporting event data are identified, extracted, combined
together, verified, sorted and categorized
further processed and analyzed to determine:• Forecast Type (e.g. Home, Draw, Away etc.)
• Forecast Details (e.g. Correct Score etc.)
• Sentiment Type (e.g.
determine:• Reposted information• Discussion information (e.g. reply to a former post etc.)
• Type of information (e.g. opinion, press releases etc.)
• Source Details (e.g.
Sporting event sentiment/opinion information
are identified and extracted
Prediction, Feeling, Wish etc.) demographics etc.)• Fan Details
Confidence Rate is applied, based on the extracted information and domain
specific rules
Sentiment Based Forecasting algorithm produces the final forecasting figures
Neurolingo Mnemosyne™ platform process & analyze content based on Sporting Language patterns (“Kanon” rules) & Sporting Specific & General language
resources
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Sentiment Based Forecasting algorithm produces the final forecasting figures
Analyses Topics, Cohorts, Platformsy p , ,
Event specific messages
14
Event-specific messages Parses Twitter jargon
Cl b ifi d Club-specific terms and jargonS t t ifi Sports outcome specific language and termsF ll t i d Full stemming and parsing of combined grammarsgrammars
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Accurate Parsing of Smaller Cohortsg
Accurate extraction of forecasting messages from
15
Accurate extraction of forecasting messages from smaller cohorts aids developing statistical significance
Weighting of user accuracy aids forecasts Weighting of user accuracy aids forecasts
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Social Media Provides Volume
Parsing Accuracy + Volume
16
Parsing Accuracy + Volume = Statistically significant results
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Sentibet vs. Actual Outcomes17
Forecasted wins: Forecasted wins: 2012
Super Bowl XLVI(92,000+ forecasting tweets)
2012 NCAA Basketball Finals(3,900+ forecasting tweets)
Highly predictive g y pongoing forecasts for Premier League and Championship League Football
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
L b L dLessons to be Learned18
24 April 2012
Copyright © 2009-2010 Shore Communications Inc. - All Rights Reserved
Sentiment vs. Forecastingg
What you want is not
19
2012 NCAA Finals Kentucky (H) vs Kansas (A) What you want is not always what you thinkor feel will happen!
2012 NCAA Finals - Kentucky (H) vs. Kansas (A)
pp Dangers in sentiment
analysis based onlyy yon emotion gauging
Focus on the rightmessages to get statistically significant forecasts Understanding complex grammars the key to success
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
In-House (Build) vs. Vendor (Buy)?( ) ( y)
Powerful lexicographic analysis requires special
20
Powerful lexicographic analysis requires special tools
Maintenance required for shifts in vocabularies, Maintenance required for shifts in vocabularies, jargons & grammars of cohorts, topics & platforms
Argues for outsourcing extraction and semantic Argues for outsourcing extraction and semantic analytics backbone platform and in-sourcing apps
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
Applicability for Many Sectorspp y y
Finance
21
Trading triggers based on data mining
Draw conclusions from stacks of reports
Sports Enrich social media offerings and offer improved odds-making input
Target team match-ups for media markets Target team match ups for media markets
Politics Interpret voter intents more accurately
Determine more rapidly when to adjust strategies
Health I t t h d di ti f id ti Interpret research and diagnostics for rapid response actions
24 April 2012
Copyright © 1999-2012 Shore Communications Inc. - All Rights Reserved
For Follow-Up…p
PHONE (+01) 203.293.8511
EMAIL [email protected]
WEB shore.com contentblogger.com secondwebbook.com
TWITTER/GOOGLE+ @jblossom / John BlossomPOST POST John Blossom
PresidentShore Communications Inc.4 Merritt LaneW t t CT 06880 USA
24 April 2012
Copyright © 1999-2010 Shore Communications Inc. ALL RIGHTS RESERVED 22
Westport, CT 06880 USA