Embed Size (px)
DESCRIPTION
Slide deck I used at several "Industrial Track" invited talks. (SIGIR 2013, #TAIA2013, RepLab 2013, ECML/PKDD 2013)
Citation preview
(Technological)Problems and Applications
of Opinion Analysis
Hugo ZaragozaDirector de Websays
Before…
After…
Our Vision
By 2020 every company, brand, institution and personality,
will need tools and services to understand and take control
of online public opinion about them.
They will require services to monitor, understand & act.
Example: Reputation Management
From a relaxed yearly Public Relations affair…
to a 24h. data-crunching speed-of-light business.
The Opportunities
• Analysis– Reputation Management / Crisis– Competitive Analysis– Topical Analysis– Political Analysis
• Prediction Action– Audience selection– Retargetting
The problem
Filtering: selecting “relevant” content
• It’s not search / ranking: we need to select the right content to count!
• Precision Problem (hard):– 1 in 4 “Mango” items are about – 1 in 10 “Camper” items are about
• Recall (really hard):– “Barcelona OR BCN” : 3-5M items per month.– What about “Ramblas, Barceloneta, RazMatazz”…
Query FormulationTargeted Crawling
Boiler Plate DetectionScraping / Structuing
FilteringTopic Modeling
IndexingCaching
Graph RankingStreaming Algorithms
HCI
Machine LearningNLP
Machine LearningNLP
Machine Learning
• (demo)
Center for Global DevelopmentSwiss Re
Hugo Zaragoza
Boiler-Plate, Scraping, Parsing…
Data needs to be “correct”.
Model Adaptation is crucial Every client is a pico-domain.
Model Adaption is crucial.
Start with almost not training data(but have human anlalysts)
Active Learning Heavy Priors
Example: Active Learning @ Websays
Lower Bound In yourQuality / Confidence
Priors and Confirmations: Filtering
24
Priors and Confirmations: Sentiment Rule Learning
26
Entity Analysis
27
Sentimnet Analysis
28
Topic Share
Crisis example
“Location” Semantic Word Cloud detects “Switzerland” as an outlier
SwitzerlandMadrid
Relative importance of topics
Some crisis go away, some come back!
Engagement: how much of the conversation has been motivated by the corporation?
Public Opinion Polling
Public Opinion Polling
Berlusconi sig-saws,Grillo never stopped growing!
Data is also content / entertainment.
Data is also content / entertainment.
Almost every aspect of this needs more science & engineering…
• Crawling, Parsing• Filtering• Clustering, Classification• Named Entities, Sentiment• Ranking (Graph Measures)
• IR + ML + NLP …+ ADAPTATION
Room for dramatic improvements!
Exploiting opinion dynamics to reach near-perfect sentiment classification
Correction Dynamics
