Sentiment Analysis in Twitter with Lightweight Discourse Analysis

Title Layout

Sentiment Analysis in Twitter with Lightweight Discourse Analysis

-Subhabrata Mukherjee & Pushpak Bhattacharyya

Presented By: Naveen Kumar

1Sentiment Analysis in Twitter with Lightweight Discourse Analysis

OutlineIntroduction to Sentiment AnalysisMotivationIntroduction : Approach Discourse RelationsDiscourse Relations Critical for Sentiment AnalysisSemantic Operators Influencing Discourse RelationsAlgorithm ProposedFeature VariablesSteps for different semantics operators and discourse relationsFeature Vector ClassificationEvaluationData Set StatisticsAccuracy MeasuresConclusion

Sentiment Analysis in Twitter with Lightweight Discourse Analysis2

2Sentiment Analysis in Twitter with Lightweight Discourse Analysis

What is Sentiment Analysis ?Classification on polarity of given text .Example : @abc Good to see your government is working for common peopleApproaches : Opinion Mining


Introduction to Sentiment Analysis:

Usage of Sentiment Analysis Applications : Brand Monitoring , Business Planning , Risk Analysis and many others

Sentiment analysis is much more than simplistically subtracting the number of negative words from the number of positive in a document or message in order to produce a score.


Introduction to Sentiment Analysis(Contd..) :


Fig : Approaches for designing Sentiment Analysis Application

Introduction to Sentiment Analysis (Contd..):Most Common Techniques Used for Sentiment Analysis:

POS : Finding adjectives as they are indicators for opinionTerm Presence and Frequency Opinion DictionariesNegations : not good is equivalent to badPoint-wise Mutual Information


Introduction to Sentiment Analysis (Contd..):Data expansion in recent years is one of the key factor which increased the role of Sentiment Analysis in business.

Looking ahead , we can see a true social democracy in different domains which harness the opinion of crowd rather than few experts


Motivation

Sentiment Analysis in Twitter with Lightweight Discourse Analysis8Example :Im quite excited about Tintin , despite not really liking original comics.

This sentence is positive although there is equal number of positive and negative words. Despite gives more weight to previous discourse segment.

Motivation

Sentiment Analysis in Twitter with Lightweight Discourse Analysis9Example :Think Ill stay with whole sci-fi shit but this time . classic movie

This sentence is positive although there is equal number of positive and negative words. But gives more weight to following discourse segment.


MotivationDiscourse elements in text alters the polarity of text Sentiment Analysis classification should also take discourse elements.This paper takes into consideration : connectives, modals, conditionals and negation in a sentence for polarity detectionIn this work bag of word model incorporates discourse information which impacts the sentiment of a sentence


Motivation : Traditional Approach in Discourse Analysis


Traditional approaches for discourse analysis are mostly based on Rhetorical Structure Theory (RTS) proposed by Mann (1988)But , most of these theories are focused on structured text.Tweet : @abc I am gonna party 2day .. wanna join?Limitations: Parsers and taggers do fail frequently in unstructured text like tweetsHeavy linguistic processes increases the processing time and slow down the entire application.

Introduction: ApproachThe approach deals with noisy and unstructured data as in Tweets

This works well even with the structured reviews and gives a better accuracy compared to state of the art system.


Introduction: ApproachApproach in this paper , identifies features for Sentiment Analysis classification which incorporates discourse information to give better sentiment classification accuracy

Features here can include bag of words, along with domain specific features like emoticons, hashtags, etc.


Discourse Relations Sentiment Analysis in Twitter with Lightweight Discourse Analysis15Coherently Structured Discourse is collection of sentences related to each other.Coherence RelationConjunctionCause-Effectbecause ; and soViolated Expectationsalthough; but; whileConditionIf(then) ; as long as ; while Similarityand ; (and) similarityContrastby contrast ; butTemporal Sequences(and) then ; first , second , ..before; after ; whileAttributionaccording to ; said ; claim that ; maintain that ; statedExamplefor example ; for instanceElaboration also ; futhermore ; in additionGeneralizationIn general

Table : Contentful Conjunctions used to illustrate Coherence Relations

Discourse Relations Critical for Sentiment Analysis Violated Expectation and Contrast : Violating expectation conjunctions oppose or refute the neighboring discourse segment.

Categorized in two : Conj_Fol and Conj_Prev Example : He wasnt prepared for the exam but he did pretty well

I loved rides at Disney World despite having acrophobia

The discourse segment following but should be given more weight. In Example 2, the discourse segment preceding despite should be given more weight.

Conclusive or Inferential Conjunctions : Set of conjunctions : Conj_infer. The discourse segment following them should be given more weight.

Eg : @User I wasnt much satisfied with ur so-called gud phone and subsequently decided to reject it.


Discourse Relations Critical for Sentiment Analysis (Contd..)

Conditionals :

The presence of if , tones down the final polarity as it introduces a hypothetical situation in the context.

Example : If Micromax improved its battery life , it wud hv been gr8 phone

Other Discourse Relations : Sentences having other discourse conjunctions which has no contrasting or conflicting information can be handled by bag of word approach.


Semantic Operators Influencing Discourse Relations

Example : You may like the movie despite the bad reviews

Here despite gives more weight to like

But may pulls down the weight as it creates uncertaininty


Semantic Operators Influencing Discourse Relations (Contd..)Sentiment Analysis in Twitter with Lightweight Discourse Analysis19RelationsAttributesConj_Folbut, however, nevertheless , otherwise , yet , still , nonethelessConj_Prevtill , until , despite , in spite , though , although Conj_Infer therefore , furthermore , consequently , thus , as a result , eventually , henceConditionals ifStrong_Modmight , could , can , would , mayWeak_Modshould , ought to , need not , shall , will , must Negnot, neither , never , no , nor

Table : Discourse Relations and Semantic Operators Essential for Sentiment Analysis

Semantic Operators Influencing Discourse Relations (Contd..)Sentiment Analysis in Twitter with Lightweight Discourse Analysis20

Modals :

Strong_Mod is the set of modals that express a higher degree of uncertainity in any situation. Eg: He may be a rising star

Weak_Mod is the set of modals that express lesser degree of uncertainty and more emphasis oncertain events or situations. Eg: Our civil service should work without interference.

Semantic Operators Influencing Discourse Relations (Contd..)

2. Negation :

The negation operator (Example: not, neither, nor, nothing etc.) inverts the sentiment of the word following it. Eg : I do not like Nokia but I like Samsung.


Algorithm Proposed : Discourse Relations and Semantic Operators are used to create feature vectors

Let user post R consist of m sentences si(i=1m) where si consist of n words wij (i=1m , j=1..n)


Algorithm Proposed (Cond..): Feature VariablesLet A be the set of all discourse relations as described before.Feature Variables : fij : Weight of the word in sentence , initialized to 1flipij : Indicates weather the polarity of word should be flipped or nothypij : Indicates the presence of conditional or strong modal in sentence

Iterate the algorithm for every sentence :for i = 1 m


Algorithm Proposed (Cond..): Step 1 Deals with conditional and strong modalsAlgo : for j =1..n hypij = 0; if wij == Conditional OR Strong _Mod :hypij = 1; end ifend for


Algorithm Proposed (Cond..): Step 2 Deals with Conj_Fol and Conj_InferAlgo : for j =1..n fij = 1; if wij == Conj_Fol OR Conj_Infer :for k=j+1 .. N and w != A fik + = 1; end for end ifend for


Algorithm Proposed (Cond..): Step 3 Deals with Conj_PrevAlgo : for j =1..n fij = Taken from Step 2; if wij == Conj_Prev :for k=1 .. j-1 and w != A fik + = 1; end for end ifend for


Algorithm Proposed (Cond..): Step 4 Deals with NegationAlgo : for j =1..n flipij = 1; if wij == Neg :for k=1 .. Neg_Window and w != (Conj_Prev OR Conj_Fol) flipi,j+k = -1; end for end ifend for


Feature Vector ClassificationWe devised a lexicon based system as well as a supervised system for feature vector

Lexicon Based Classification : Here :fij : weight of each word in sentenceflipij : indicates polarity of word should be flipped or nothypij : indicates the presence of conditional or strong_mod


Feature Vector Classification (Contd..)Supervised Classification : SVMs are used to classify the set of feature vectors {flipij, wij, fij and hypij}.Features used :N-gramsStop WordsFeature Weight Modal and Conditional IndicatorStemmingNegationEmoticons POSFeature Space


Evaluation : Data Sets Statistics Data Set 1 : 8507 tweets are collected based on a total of around 2000 different entities from 20 different domains.Four classes - positive, negative, objective-not-spam and objective-spam.

Data Set 2 : Hashtags #positive, #joy, #excited, #happy etc. are used to collect tweets bearing positive sentiment, Hashtags like #negative, #sad, #depressed, #gloomy, #disappointed etc. are used to collect negative tweets.

Data Set 3 : Travel Review Dataset contains 595 polarity-tagged documents for each of the positive and negative classes.Sentiment Analysis in Twitter with Lightweight Discourse Analysis30

Evaluation : Data Sets Statistics Base Line System : C-Feel-It (Joshi et al., 2011) Rule Based SystemThe only difference between the two systems is the handling of connectives , modals, conditionals and negationSentiment Analysis in Twitter with Lightweight Discourse Analysis31Manually Annotated DataSet 1#Positive#Negative#ObjectiveNotSpam#Objective#Total25481209275719938507Auto Annotated DataSet 2#Positive#NegativeTotal7348786615214

Table : Twitter Datasets 1 and 2 Statistics

Evaluation : Accuracy Measures Sentiment Analysis in Twitter with Lightweight Discourse Analysis32Table : Accuracy Comparision b/w Base Line System and Discourse System using Lexicon in Data Set 1 and 2 Accuracy Measure for 2 Class Classification : DataSet 1Base Line System Discourse System 68.5872.81Accuracy Measure for 3 Class Classification : DataSet 1Base Line System Discourse System 57.261.31Accuracy Measure for 2 Class Classification : DataSet 2Base Line System Discourse System 80.5584.91

Evaluation : Accuracy Measures Sentiment Analysis in Twitter with Lightweight Discourse Analysis33Fig : Accuracy Comparision b/w Base Line System and Discourse System using Lexicon in Data Set 1 and 2

Evaluation : Accuracy MeasuresSentiment Analysis in Twitter with Lightweight Discourse Analysis34Table : Accuracy Comparison b/w Base Line SVM System and Discourse SVM System in Data Set 1 and 2 Accuracy Measure for 2 Class Classification : DataSet 1Base Line System Discourse System 69.4970.75Accuracy Measure for 3 Class Classification : DataSet 1Base Line System Discourse System 63.1164.23Accuracy Measure for 2 Class Classification : DataSet 2Base Line System Discourse System 91.9993.01

Evaluation : Accuracy MeasuresSentiment Analysis in Twitter with Lightweight Discourse Analysis35Fig : Accuracy Comparison b/w Base Line SVM System and Discourse SVM System in Data Set 1 and 2

Evaluation : Accuracy MeasuresSentiment Analysis in Twitter with Lightweight Discourse Analysis36This evaluation is used to determine whether our discourse based approach performs well for structured text as wellSentiment Evaluation Criterion AccuracyBaseline Bag-of-Words Model69.62Bag-of-Words Model + Discourse71.78

Table : Accuracy Comparision between Bag of Word and Siscourse System using Lexicon in DataSet 3

Evaluation : Accuracy MeasuresSupervised Classification Approach under 4 scenarios :When only unigrams are usedWhen only sense of unigrams are usedWhen unigrams are used along with their senses (synset-ids)When unigrams are used with senses and discourse featuresSentiment Analysis in Twitter with Lightweight Discourse Analysis37Note : IWSD : Iterative Word Sense Disambiguation ((Khapra et al., 2010)SystemAccuracy(%)Only Unigrams84.9Only IWSD Sense of Unigrams85.48Unigrams + IWSD Sense of Unigrams86.08Unigrams + IWSD Sense of Unigrams + Discourse88.13

Table : Accuracy Comparison In Dataset 3

Evaluation : Accuracy MeasuresSentiment Analysis in Twitter with Lightweight Discourse Analysis38Note : IWSD : Iterative Word Sense Disambiguation ((Khapra et al., 2010)Figure: Accuracy Comparison In Dataset 3

ConclusionMost of the works on unstructured data uses bag of words approach , ignores discourse markers

Evaluation on different dataset show that incorporating discourse information improves the performance by 2-4%

The approach presented can deal with unstructured data as it doesnt need to parse the document.


Questions for You What feature variables are used in the algorithm for incorporating discourse information?

What is the advantage of the presented approach over traditional approaches ?Sentiment Analysis in Twitter with Lightweight Discourse Analysis40


Data & Analytics

Sentiment Analysis in Twitter with Lightweight Discourse Analysis