Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
ANALYSISOFTWEETSONDEMONETIZATIONININDIA
USINGSASENTERPRISEMINERPAPERNUMBER:135
JITALPATELandNARMADAPANNEERSELVAMMSINBUSINESSANALYTICS
OKLAHOMASTATEUNIVERSITY
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page1of18
ABSTRACT
Thecitizensofacountryoftenfacethebenefitsorthebruntofvariouspoliciesadoptedbythe
government. Social platforms then become sounding boards for them to express their
displeasuresorconcernsforthematterathand.OnesuchpolicythatsentTwitterintoafrenzy
wasthedemonetizationorderthatwasannouncedratherabruptlybytheIndianPrimeMinister
onNovember8th2016withoutany formofprior intimation topublic. In thispaper,wehave
analyzedthetweetsthathelpedusrecognizewhetherdemonetizationwasperceivedpositively
ornegativelybythecitizens.
About 15 days after the demonetization decisionwas announced, a dataset of 8,000 tweets
spanningtwodays,wascollectedfromapubliclyavailabledatasource.Usingthecommonlyused
termsinthetweetsandstudyingthestrengthoftheirrelationsusingconceptlinks,ingeneral,
showapositivefeedbacktothedemonetizationimplementationpolicy.Usingtextclusteringand
texttopictogrouppeoplewithsimilarthoughtsbasedonthetweets,revealthatdemonetization
waspositivelysupportedbyalargenumberofpeople.Thus,ouroverallanalysisshowedthata
vast majority of Indians accepted the demonetization policy positively while some of them
expressedtheirdispleasureoverit.
INTRODUCTION
Demonetizationistheactofstrippingacurrencyunitofitsstatusaslegaltender.Itoccurs
wheneverthereisachangeofnationalcurrency:Thecurrentformorformsofmoneyispulled
fromcirculationandretired,oftentobereplacedwithnewnotesorcoins.Sometimes,acountry
completelyreplacestheoldcurrencywithnewcurrency.Manyreasonscouldleadtothisactof
demonetizationofcurrencyinvariouscountriesallovertheworld.Forexample,todiscourage
cash-dependenteconomy,UnitedStatesdeclaredTheCoinageActof1873asa legal tender.
AccordingtotheAct,silverwasdemonetizedtofullyadopttogoldstandards.Anotherexample
wouldbeTheNationsofEuropeanUnion,whodemonetizedtheoldnationalcurrencies,suchas
Germanmark,Frenchfranc,andItalianliraandintroducedeurobillsandcoinsin2002.Themajor
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page2of18
reason for demonetization being trade purposes. Similarly, the Zimbabwean government
demonetizeditsdollarin2015,asameanstofightcountry’shyperinflation.(Staff,2017)
Themostrecentdemonetizationensuedin Indiaon8thNovember,2016.Themajorobjective
being,tobattleagainstcorruptionandcrime.TheIndianGovernment’sdecisiontodemonetize
all500and1000RupeenotesoftheMahatmaGandhiseriesasaformof legaltender.These
werethetwobiggestdenominationsinitscurrencysystemaccountingforabout86%ofIndia’s
circulatingcash.TheannouncementwasmadebythePrimeMinisterofIndia,Mr.NarendraModi
withnowarning,thatthosenoteswereworthlessandtheyhadtobeexchangedwiththenewly
introduced2000and500Rupeebillsbytheendoftheyear.(Staff,2017)
Publicopinion iseverythingandTwitter isanextensiveplatformtodiscoverthe latest
newsandworldevents. Therefore, analyzing tweetsondemonetization in Indiaandderiving
high-qualityinformationfromTextMiningisthescopeofthispaper.Oneoftheobjectivesofthis
paperis,determiningthemostcommonlyusedtermsinthetweetsandstudyingthestrengthof
theirrelationshipwiththeotherterms.Anothergoalofthispaper is,usingTextClusteringto
grouppeoplewithsimilarthoughtsbasedonthetweetsandextractingTextTopics.
LITERATUREREVIEW
Various research studies and peer-viewed articles show different effects of
demonetization.
“Sentiment Analysis of demonetization of 500 & 1000 rupee banknotes by Indian
Government”byPrabhsimranSingh,RavinderSinghSawhney,KaranjeetSinghKahlon. In this
paper, theyhaveanalyzed theeffectof implementingdemonetizationpolicyusing sentiment
analysis.AnalysisshowsthatlargeshareofIndianpeoplewashappywiththispolicy.Duringinitial
days,thesentimentwasmorenegativeduetohardshipsbutafterthereleaseofnewbanknotes,
overallsentimentofthepeoplebecamepositive.
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page3of18
MohasinA.Tamboli inhispaper, “Demonetization:RetrospectiveAnalysis,” concludes
thatallthegainsofDemonetizationarecontingentongovernment’sabilitytocontrolcorruption.
Ifcorruptionisnotcontrolled,alltheshorttermgainsofdemonetizationwillbelostandtheonly
thingthatwouldchangeforthebetterislookandfeelofnewcurrency.
“DemonetizationandCompleteFinancialInclusion”byS.VijayKumarandT.ShivaKumar,
alsodeterminesthatrewardsofdemonetizationareencouragingandinthelongterminterest
ofthecountrybutgovernmentsneedtoensuresmoothflowofcurrency.ItwilltakeIndiaten
stepsaheadandwillinfluencecorruption,electionsandterrorism.
Another paper by Dr. Pawan Kumar about “Demonetization and its impact on
EmploymentinIndia,”describesthattheemploymentscenarioisnotconduciveenoughtoface
anychallengeofdemonetizationofcurrency.Infact,thedecisionofdemonetizationwillfurther
destabilizethealreadyvolatilelabormarketinIndia.
DATAACCESS
The dataset for this paper contains 8,000 observations, that is the number of tweets
collectedover a spanof twodays and about 15 days after the demonetization decisionwas
announced.ThedatasetistakenfromKaggle(kaggle.com.2017).Thelinkstothemainsourceof
thedatasetisalsoprovidedinthedataset.Adatadictionaryforthevariablesusedintheanalysis
arestatedbelow.
Variable Role Description
ID ID Observationnumber
Text Text DemonetizationTweets
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page4of18
METHODOLOGY
1. TextImport
The dataset is available in the excel file format. Using EnterpriseMiner, the Excel file is
importedusingtheFileImportNode.Thefileimportnodeconvertstheexcelfileformatto
SASfileformat.SASformatfilecanbeusedforfurtheranalysis.Metadatanodeisusedto
restrictonlytheIDandTweetsasinputsfortheadvancestudy.
2. TextParsing
Afterimportingandselectingvariables,TextParsingnodeisusedtoidentifytermsandtheir
instancesinthedata,whichcontainstext.TheresultsofTextParsingnodedisplaythemost
frequentlyoccurringterm,thenumberofdocumentsithasoccurredin,andthetermsthat
arerarelyused.Usually,thetermsthatareusedmoderately,aretheonesthatarethemost
helpfulinexplorationandmodeling.
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page5of18
The most frequently used term is demonetization which has a total frequency of 7,534
(3,854+3,680) since it has appeared twice. It has appeared in total 7,224 (3,849+3,375)
documents. Similarly, the next most frequently used terms are app (1,686), support(1,086),
narendramodi(928),people(866),bank(767).
3. TextFiltering
AText Filter node is added to the Text Parsing node to eliminate the terms that occur leastnumberoftimesinallthedocumentsandtoperformspellchecktosuggestpotentialsynonyms.
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page6of18
The results of the Text Filtering node show that terms like https, t, s, and have, have been
droppedintheTermstable,astheydonotaddanymeaningtotheanalysis.OnlytheTermsthat
addmeaningtotheanalysisofdemonetizationareretained.
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page7of18
TheTextFilternode,autocorrectsthemisspelttermslike‘hav’iscorrectedto‘have’,‘tertorists’
iscorrectedto‘terrorists’and‘oppositio’iscorrectedto‘opposition’.
Thenodealso,groupsthesynonymstogetherbasedonthesynonymsfilethatisimportedorthe
terms are manually dragged and dropped into each other. For example, terms like
‘demonization‘, ‘demoditization‘, ‘demonitization‘, and ‘dmonetization‘ are all grouped into
‘demonetization‘bytheTextFilternode.
ConceptLinks
ConceptLink,displaysthetermsthatarehighlyassociatedwiththeselectedtermintheTerms
table.Theselectedtermissurroundedbythetermsthatcorrelatethestrongestwiththeselected
term.Thewidthofthelinkdepictsthestrengthofassociationbetweenthetwoterms.
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page8of18
The above concept link is for the highest frequency term, Demonetization. The term
demonetization is associatedwithbank (associatedwithdemonetizationof currency), survey
(associatedwithdemonetizationsurvey),vote(associatedwithvotingfordemonetizationsurvey
app),modisurvey(associatedwithdemonetizationsurveyappinitiatedbyModi),rt(associated
with retweets about demonetization), mounting misery (associated with misery due to
monetization)support (associatedwith thesupportofpeople fordemonetization)opposition
(associatedwithoppositionparty’sviewwithrespecttomonetization).Outofallthetermsthat
Demonetizationisassociatedwith,itisstronglyrelatedwithsupport,opposition,modisurvey
and vote. The associated terms depict positive (support, rt) as well as negative (opposition,
misery)responsesofdemonetization.
Demonetization
Support
Opposition
Bank
Survey
Vote
ModiSurvey
rt
MountingMisery
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page9of18
TheaboveconceptlinkisforthetermSupport,whichisstronglyassociatedwithDemonetization.
The important terms associated with the term support are demonetization strategy, party,
decision,narendramodi,demonetization,question,people,andfeedback.Thetermsupportis
equallystronglyassociatedwithalltheaboveterms.Thisconceptlinkdepictsthatthefeedback
ofmajoritypeopleandparty, is supporting thedemonetization strategyordecision takenby
NarendraModi.
Support
Feedback
DemonetizationStrategy
Decision
Party
Demonetization
NarendraModi
question
People
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page10of18
Theaboveconcept link is forthetermPeopleandthe importanttermsassociatedwith itare
demonetization strategy, support, decision, narendra modi, question, back, strategy, and
feedback.Thetermpeopleisstronglyassociatedonlywithstrategy,feedback,narendramodiand
decisionterms.Thisconceptlinkalso,depictsthatthefeedbackofmajoritypeopleistosupport
thedemonetizationstrategyordecisiontakenbyNarendraModi.
People
Feedback
Strategy
Decision
Back
DemonetizationStrategy
Narendra Modi
question
Support
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page11of18
TheaboveconceptlinkisforthetermFeedbackandtheimportanttermsassociatedwithitare
strategy, people, demonetization strategy, support, question, and demonetization. The term
feedbackisstronglyassociatedonlywithstrategy,people,demonetizationstrategy,support,and
questionterms.Thisconceptlinkalso,depictsthatthefeedbackofmajoritypeopleistosupport
thedemonetizationstrategy.
Feedback
People
Demonetization
DemonetizationStrategy
Strategy
question
Support
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page12of18
Theaboveconcept link is for the termResultand the important termsassociatedwith itare
demonetization,decide,bypolls,modi,demonetizationstrategy,rt,bright,andfuture.Theterm
resultisequallystronglyassociatedonlywithalltheaboveterms.Thisconceptlinkalso,depicts
thattheresultofthesurveyisthatmodi’sdemonetizationdecisionwillleadtoabrightfutureis
retweetedmanytimes.
4. TextClustering
ATextClusternode is added to the text filteringnode. The text clusteringnode clusters the
documentsintodisjointedsetsofdocumentsandreportsondescriptivetermsofthoseclusters.
Hierarchicalclusteringalgorithmisusedtogroupclustersintoatreehierarchy.Theapproachof
hierarchical clustering relies on singular value decomposition (SVD) to transform original
weighted,term-documentfrequencymatrixintoadensebutlowdimensionalrepresentation.
Result
Future
Bypolls
Decide
Demonetization
DemonetizationStrategy
Modi
rt
Bright
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page13of18
Theaboveplot,showsthedistancebetweentheclusters.All theclustersaredistributedwell
apart. Thepie chartbelow, shows thedistributionof thecluster frequencies.Apart from the
clusternumber14and clusternumber16 the frequenciesarewelldistributedamongall the
clusters.
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page14of18
Cluster
ID
DescriptiveTerms Frequency Percentage ExampleTweets
8 Demonetizationmove+
blackmoney+black+
government+govt+modi
+money+move
658 8% Demonetizationis
government’smoveagainst
blackmoney
13 Rt@shashitharoor+big+
cash+country+
demonetization+duty+
time+understand
774 10% ShashiTharoor’stweets
retweeted,thatPM’sduty
tounderstandandaddress
parliamentabout
demonetization
14 Thirdsuchincident+bank
+demonetization+lakh+
paytm+questionclearly
critical
1584 20% 50lakhcustomers
respondedin24hoursand
90%support
demonetization.Waspaytm
informedabout
demonetization?40lakhs
lootedfromabank,third
incidentsince
demonetization.
16 Oppositionmps+
pmoindiaaddress
parliame+
rt@drkumarvishwas+
1053 13% OppositionMPssupporting
demonetization.Demanding
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page15of18
rt@shashitharoor+
address+govt+line
affected
PMofIndiatoaddress
parliament.
19 Abov+atm+byelection+
people+state+support
atmsippatel
347 4% DelayreplenishingATMs.By
electiontwodaysafter
demonetizationhasproved
support.
20 Debate+decide+
demonetization+Gandhi
+impact+know+modi+
note
950 12% Easyexchangetonew2k
note.RahulGandhi
(opposition)issuewith
Demonetization.
22 Daughter’swedding+
fundshortage+daughter
+fund+life+prob+tweet
ends
442 6% Manendslifeduetofund
shortagefordaughter’s
wedding.
25 Watchbriefing+
demonetization+effect
+happiness+keep+
pmoindia+stuff
announcements
699 9% Happinessabout
demonetizationbut
announcementnotmadeat
parliament,sowatch
briefing
26 Modisurvey+arunjaitley
+blackmoney+decision+
flight+move+response
+survey
776 10% Modi’sdemonetization
decisionsurveyshowed
positiveresponse
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page16of18
Theabovetextclustersshowspositive,negativeaswellasneutraltweettextreviewanalysis.
Outofthetenclusters,sixclustersshowpositivereviews.
5. TextTopic
TextTopicnodeisconnectedtotheTextParsingnodetocombinethetermsintotopicssothat
theycouldbeanalyzed further.A listof topics iscreatedtoestablishcombinationsofwords,
whichcouldbeofinterestinanalyzing.
Topic Number
of
Terms
Numberof
Documents
ExampleTweets
Thirdsuchincident,third,
incident,loot,terrorist
10 542 40lakhslootedfromabank,third
incidentsincedemonetization.
Criticalquestion,
demonetizationedict,
edict,inform,rssurjewala
15 289 Waspaytminformedabout
demonetization?
Partypolitics,
nitishkumar,putting
16 552 Puttingnationoverpartypolitics,
supportdemonetizationbyPM
28 Demonetizationstrategy
hugesupport
demonetizationmove+
rt@modibharosa+back+
c-voter+narendramodi+
nation
717 9% Demonetizationstrategy
wassupportedonalarge
scale
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page17of18
nation,rt@modibharosa,
narendramodi
Hugesupport,huge,c-
voter,demonetization
move,nation
23 264 Hugesupportbypeopleandnationfor
demonetization
Daughter’swedding,
fund,shortage,daughter,
end,life
20 159 Manendslifeduetofundshortagefor
daughter’swedding
Ls,retain,byelecton,seat,
state
44 309 Byelectiontwodaysafter
demonetizationhasprovedsupport.
rt@drkumarvishwas,
demonetzation,people,
rt,huge,support
13 350 Hugesupportbypeoplefor
demonetization
Join,corruptfreeindia,
walk,nationalist,ma
31 189 Jointhewalktowardscorruptionfree
India
Theabovetexttopicsshowpositive,negativeaswellasneutraltweettextreviewanalysis.Out
oftheeighttopics,fivetopicsshowpositivereviews.
CONCLUSION
Radicalchangesarealwaysfacedwithresistanceofsomekind,caseinpointbeingthepolicyto
implement demonetization. The goal of this paper was to analyze the tweets by the Indian
citizensaboutdemonetizationandtheirperceptiontowardsthepolicy,byusingTextMining.The
results of Concept Links, in general, show a positive feedback to the demonetization
implementationpolicy.Similarly,theresultsofouranalysisoftweetsusingTextClusteringand
TextTopicsrevealthatdemonetizationwaspositivelysupportedbyalargenumberofpeople.
AnalysisofTweetsonDemonetizationinIndiaUsingSASEnterpriseMiner SESUG2016
Page18of18
OuroverallanalysisshowedthatavastmajorityofIndiansacceptedthedemonetizationpolicy
positivelywhilesomeofthemexpressedtheirdispleasureoverit.
BIBLIOGRAPHY
[1]SASInstituteInc.2012.GettingStartedwithSAS®TextMiner12.1.Cary,NC:SASInstituteInc.
https://support.sas.com/documentation/onlinedoc/txtminer/12.1/tmgs.pdf
[2]Staff,I.(2017).Demonetization.[online]Investopedia.Availableat:
http://www.investopedia.com/terms/d/demonetization.asp?ad=dirN&qo=investopediaSiteSear
ch&qsrc=0&o=40186[Accessed5Apr.2017].
[3] “Sentiment Analysis of demonetization of 500 & 1000 rupee banknotes by Indian
Government”byPrabhsimranSingh,RavinderSinghSawhney,KaranjeetSinghKahlon
[4]“Demonetization:RetrospectiveAnalysis,”byMohasinA.Tamboli
[5]“DemonetizationandCompleteFinancialInclusion”byS.VijayKumarandT.ShivaKumar
[6]“DemonetizationanditsimpactonEmploymentinIndia,”byDr.PawanKumar
[7] Kaggle.com. (2017). Demonetization in India Twitter Data | Kaggle. [online] Available at:
https://www.kaggle.com/arathee3/demonetization-in-india-twitter-data [Accessed 10 Feb.
2017].