1
Unpredictability of elections with social media Social media users are not a representative unbiased sample of likely voters. Social media data is easily manipulated by spammers and propagandists . (In Twitter nobody knows you are a robot). “It works” is not enough : we need to know how it works. (Much) better (and subtle) sentiment analysis methods are needed. Limits of Electoral Predictions using Twitter Daniel Gayo-Avello (@PFCdgayo) Univ. of Oviedo (Spain) Panagiotis T. Metaxas (@takis_metaxas) Wellesley College (USA) Eni Mustafaraj (@enimust) Wellesley College (USA) 7.6% is a far cry from 2-3% MAE . Sentiment analysis is just slightly better than random classifier (36.9%). Sentiment analysis weakly correlates with user’s political preference. Promising results? NO! Political discourse in social media is becoming common practice. One interesting aspect of this is the possibility of pulsing the public's opinion about the elections. Allegedly, predicting electoral outcomes from social media data can be feasible a even simple. Positive results have been reported, but without an analysis on what principle enables them. Our work puts to test the purported predictive power of social media metrics against the 2010 US congressional elections. We found no correlation between the analysis results and the electoral outcomes, contradicting previous reports. We argue that one should not be accepting predictions about events using social media data as a black box. Instead, scholarly research should be accompanied by a model explaining the predictive power of social media, if there is one. of adult users engaged in electoral campaigns through OSNs in 2010. 22% Social media data have been used to successfully “predict the present” in a great variety of topics. Motivation Therefore… <flame> Social media data might predict everything but it is difficult. Corollary: Social media data can predict everything once the answer is known and data not fitting are ignored. </flame> YES WE CAN Williams & Gulati (Politics and Technology Review, 2008) Carr (Fast Company, 2010) Tumasjan et al. (ICWSM, 2010) NO WE CANNOT Goldstein & Rainey (L.A. Times 2010) Jungherr, Jürgens & Schoen (Social Science Computer Review, 2011) Maybe. Who knows? O’Connor et al. (ICWSM, 2010) Lui, Metaxas & Mustafaraj (e-Society Conference, 2011) Gayo-Avello (CACM, 2011) Can social media data predict elections? Can we replicate the results? 17.1% (Twitter volume) 7.6% (Twitter sentiment) 2-3% (professional polls) Winner predicted in only half of the races. Mean Absolute Error (MAE)

Limits of Electoral Predictions using Twitter

Embed Size (px)

Citation preview

Unpredictabilityof elections with

social media

Social media users are not a representative unbiased sample of likely voters.

Social media data is easily manipulated by

spammers and propagandists.

(In Twitter nobody knows

you are a robot).

“It works” is not enough: we need to know how it works.

(Much) better (and subtle) sentiment

analysis methods are needed.

Limits of Electoral Predictions using Twitter

Daniel Gayo-Avello (@PFCdgayo) – Univ. of Oviedo (Spain)Panagiotis T. Metaxas (@takis_metaxas) – Wellesley College (USA)

Eni Mustafaraj (@enimust) – Wellesley College (USA)

7.6% is a far cry from 2-3% MAE.

Sentiment analysis is just slightly better

than random classifier (36.9%).

Sentiment analysis weakly correlates with user’s political preference.

Promising results?

NO!Political discourse in social media is becoming common practice. One interesting aspect of this

is the possibility of pulsing the public's opinion about the elections. Allegedly, predicting

electoral outcomes from social media data can be feasible a even simple. Positive results have

been reported, but without an analysis on what principle enables them. Our work puts to test

the purported predictive power of social media metrics against the 2010 US congressional

elections. We found no correlation between the analysis results and the electoral outcomes,

contradicting previous reports. We argue that one should not be accepting predictions about

events using social media data as a black box. Instead, scholarly research should be

accompanied by a model explaining the predictive power of social media, if there is one.

of adult users engaged in electoral campaigns

through OSNs in 2010.22%Social media data have been used to successfully “predict the present” in

a great variety of topics.

Motivation

Therefore…<flame>

Social media data might predict everything

but it is difficult.

Corollary: Social media data can predict everything

once the answer is known and data not fitting are ignored.

</flame>

YES WE CANWilliams & Gulati

(Politics and Technology Review, 2008)Carr (Fast Company, 2010)

Tumasjan et al. (ICWSM, 2010)

NO WE CANNOTGoldstein & Rainey (L.A. Times 2010)

Jungherr, Jürgens & Schoen (Social Science Computer Review, 2011)

Maybe. Who knows?O’Connor et al. (ICWSM, 2010)

Lui, Metaxas & Mustafaraj(e-Society Conference, 2011)Gayo-Avello (CACM, 2011)

Can social media data predict elections?

Can we replicate the results?17.1% (Twitter volume)

7.6% (Twitter sentiment) 2-3% (professional polls)

Winner predicted in only half of the races.

Mean Absolute Error(MAE)