The Linguistics of Spin: A Computational Linguist’s Forays into Social Science Philip Resnik University of Maryland AAAL 2012

The Linguistics of Spin:A Computational Linguist’s Forays into Social Science

Philip ResnikUniversity of Maryland

AAAL 2012

This talk includes a little computational social science experiment

• On your phone, go to go.reactlabs.org– Works on most smartphones

• Select AAAL 2012• React in real time to the talk

– Tap buttons to react (select, react)– Comments are posted on Twitter– See everyone’s comments in the

app

• See results at results.reactlabs.org

2

Computational linguists like data.

Computational linguists like building things.

3

Sometimes building things gets you useful data.

4

Sometimes it doesn’t.

Sure, Hal, Gingrich does have cool ideas about NASA. What do you think about his position on corporate tax rates?

5

The dream

Terry Winograd, Procedures as a Representation for Data in a Computer

Program for Understanding Natural Language, MIT doctoral dissertation,

August 1970.

How not to understand language

block n.1. a. A solid piece of a hard substance, … having one or more flat sides.b. Such a piece used as a construction member or as a support.c. Such a piece upon which chopping or cutting is done: a butcher's block.d. Such a piece upon which persons are beheaded.e. One of a set of small wooden or plastic pieces, such as a cube, bar, or cylinder, used as a building toy.f. Printing A large amount of text.g. Sports A starting block.2. A stand from which articles are displayed and sold at an auction: Many priceless antiques went on the block.3. A mold or form on which an item is shaped or displayed: a hat block.4. A substance, such as wood or stone, that has been prepared for engraving.

…

17. Psychology A sudden cessation of speech or a thought process without an immediate observable cause, sometimes considered a consequence of repression. Also called mental block.18. Slang The human head: threatened to knock my block off.19. A blockhead.

What about “block: a section of statements grouped together in a computer program”?

Terry Winograd, Procedures as a Representation for Data in a Computer

Program for Understanding Natural Language, MIT doctoral dissertation,

August 1970.

How not to understand language

A bit of history

Overall, the AI industry boomed from a few million dollars in 1980 to billions of dollars in 1988, including hundreds of companies... Soon after that came a period called the “AI Winter”, in which many companies fell by the wayside as they failed to deliver on extravagant promises.-- Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach, 3rd ed., Prentice-Hall, 2010.

10

speech recognition algorithm

Fast forward to today

11

12



Brendan O’Connor, Ramnath Balasubramanyan, Bryan R. Routledge, Noah A. Smith, From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series, Proceedings of the International AAAI Conference on Weblogs and Social Media, Washington, DC, May 2010.

14

So, how did that happen?

Computers are doing all sorts of useful things with language.

NLP researchers

Speech researchers

15

19

85

19

90

19

95

20

00

20

05

1983 — 1993“the return of empiricism… probabilistic models throughout speech and language processing”

2000— 2008 “the rise of machine learning”

Jurafsky and Martin (2009), Speech and Natural Language Processing

19

70

19

83

1970 — 1983“natural language understanding”

1994 — 1999 “the field comes together”

16

Percentage of “statistical NLP”

papers at the field’s top

conference

0

100

0

Graph adapted from Church, K. (2003) “Speech and Language Processing: Where have we been and where are we going,” Eurospeech, Geneva, Switzerland, Adding data from figures in Cardie and Mooney (1999).

17

DataData

Knowledge-based

resources

Knowledge-based

resources

Machine learningMachine learning

SystemSystemTest dataTest data

It’s all about learning from data

18

Social scientists have started figuring this out, too.

19

20

21

Jure Leskovec, Lars Backstrom, and Jon Kleinberg. 2009. Meme-tracking and the dynamics of the news cycle. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD '09). ACM, New York, NY, USA, 497-506.

As a linguist, though,

I’m interested in going a little deeper.

22

As a linguist, though,

I’m interested in going a little deeper.

23

Agenda settingWho is controlling what gets talked about?Work with Jordan Boyd-Graber and Viet-An Nguyen

Framing and “spin”How is language used to bias our interpretations?Work with Stephan Greene, Jordan Boyd-Graber, Viet-An Nguyen, Eric Hardisty

Agenda setting

24

McCombs, M; Shaw, D (1972). "The agenda-setting function of mass media". Public Opinion Quarterly 36 (2).

[The] press may not be successful much of the time in telling people what to think, but it is stunningly successful in telling its readers what to think about.

Cohen, B.C. (1963). The press and foreign policy. Princeton.

Lippmann, W. (1922). Public opinion. New York: Harcourt.

This, then, will be the clue to our inquiry. We shall assume that what each man does is based not on direct and certain knowledge, but on pictures made by himself or given to him.

In choosing and displaying news, editors, newsroom staff, and broadcasters play an important part in shaping political reality. Readers learn not only about a given issue, but also how much importance to attach to that issue from the amount of information in a news story and its position.

25

Meraz, S., 2011. The fight for “how to think”: Traditional media, social networks, and issue interpretation. Journalism, 12(1), p.107-127.

[Our] findings point to two significant trends: the growing power of social influence among partisan blog networks and the weakening influence of elite, traditional media as a singular power in influencing issue interpretation within networked political environments.

Agenda setting

05_03_02.txt.0002 BEGALA Good evening. Welcome to CROSSFIRE,coming to you live from the George Washington University in beautifuldowntown Washington, D.C. Tonight in the CROSSFIRE, the case ofthe Reverend Paul Shanley, the Roman Catholic priest facing childrape charges in Massachusetts. Should his superiors be heldresponsible? Also, Matt Drudge, founder of the Internet "DrudgeReport." Is he a right-wing muckraker, an Internet gossip or alegitimate journalist? We'll ask Drudge himself when we get himin the CROSSFIRE. First, flying the not-so-friendly skies, wouldyou feel safer if pilots were armed? One outspoken congressionalcritic is against having guns in the cockpit. We're going tointroduce her now. Please welcome, Eleanor Holmes Norton, theDemocratic delegate from the District of Columbia. Ms. Norton, thankyou. Welcome back.

05_03_02.txt.0003 CARLSON Now, Ms. Norton, the majority,the vast majority of commercial airline pilots are strongly in favorof carrying guns in the cockpit on commercial airliners. You'reagainst it. What do you as a delegate know about operating acommercial airliner that the majority of commercial airline pilotsdon't know

05_03_02.txt.0004 DELEGATE Well, I know whatTransportation Secretary Norm Mineta tells me, and I know whatHomeland Security Adviser Tom Ridge tells me, and they are againstit. And I think the reason they are against it is you don't wantthe guy who's flying one of these big busters up there also with agun in his hand trying to protect his plane. You want air marshalsto do that. You want flight attendants to understand how to protectthe cockpit. And you want the redundancies that we have built in,redundancy after redundancy, working for you. We are panicking theAmerican people. They say, oh my God, I thought they had thehearings, I thought they did that. Here come the pilots saying,oh no, they haven't. We've got to have guns.







Looking at just word counts often gives you

a mish-mash.

Can word frequencies tell you about the topics in a set of documents?

Topics and agendas in conversation











Bayesian topic models* discover the distinct topics interwoven in documents.

*Wikipedia: Topic Model; Blei et al. 2003

.03

.44

.00

.11


Any part of the conversation can be viewed as a

mixture of topics.

Well, I know what Transportation Secretary Norm Mineta tells me, and I know what Homeland Security Adviser Tom Ridge tells me, and they are against it. And I think the reason they are against it is you don't want the guy who's flying one of these big busters up there also with a gun in his hand trying to protect his plane. You want air marshals to do that. You want flight attendants to understand how to protect the cockpit. And you want the redundancies that we have built in, redundancy after redundancy, working for you. We are panicking the American people. They say, oh my God, I thought they had the hearings, I thought they did that. Here come the pilots saying, oh no, they haven't. We've got to have guns.

29

.03

.44

.00

.11

Topics and agendas

Well, I know what Transportation Secretary Norm Mineta tells me, and I know what Homeland Security Adviser Tom Ridge tells me, and they are against it. And I think the reason they are against it is you don't want the guy who's flying one of these big busters up there also with a gun in his hand trying to protect his plane. You want air marshals to do that. You want flight attendants to understand how to protect the cockpit. And you want the redundancies that we have built in, redundancy after redundancy, working for you. We are panicking the American people. They say, oh my God, I thought they had the hearings, I thought they did that. Here come the pilots saying, oh no, they haven't. We've got to have guns.

This means we can compare parts of

the conversation to see how similar

they are.

.02

.31

.01

.24

If a criminal overpowers a prison guard, the criminal has a prison guard, and that is all. If a terrorist overpowers a pilot, he has an airplane, he has a weapon of mass destruction. And I don't want just a front line of defense. But why not give them the same training we give our air marshals? Our -- 70 percent of our commercial pilots already have military experience, and allow them to be just another air marshal on every flight?

31

For every turn in the conversation, we know who the author is and what words they used.

We have some mix of topics from the previous turn.

For this new turn, we decide whether to keep the old mix of topics, or generate a new mix.

That choice depends on the author of this new turn and their tendency to change the topic.

Once we have the topic mix for this turn, we generate words according to that mixture of topics.

Then we do it all over again with the next turn.

How can we model people’s tendency to shift topics?There are algorithms for fitting this kind of model to a set of observed data.

V. Nguyen, J. Boyd-Graber, and P Resnik. SITS: A Hierarchical Nonparametric Model using Speaker Identity for Topic Segmentation in Multiparty Conversations. ACL 2012

32

If it’s a good model, it will fit the data well and the results will make sense.

If it’s not the right model, they won’t.

Model: detecting topic shifts

34

Model: detecting topic shifts

Model: topic shift tendency

35

36

Ifill, moderator: Terrible. Yes, she was constrained by the agreed debate rules. But she gave not the slightest sign of chafing against them or looking for ways to follow up the many unanswered questions or self-contradictory answers. This was the big news of the evening. Katie Couric, and for that matter Jim Lehrer, have never looked so good.

Palin: "Beat expectations." In every single answer, she was obviously trying to fit the talking points she had learned to the air time she had to fill, knowing she could do so with impunity from the moderator. But she did it with spunk and without any of the poleaxed moments she had displayed in previous questions. The worst holes in her answers - above all, about the Vice President's role, also either mishearing or ignoring the question about her "Achilles heel" - were concealed in ways they haven't been before.

37

Model: topic shift tendency

38

Framing and “spin”

Framing (Entman 1993): “to select some aspects of a perceived reality and make them more salient in a communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation”

(Cf. “spin”)

Lexical framing

Death tax

Pro-life

Killer whales

Estate tax

Pro-choice

Orcas

Eric Hardisty, Jordan Boyd-Graber, and Philip Resnik. Modeling Perspective using Adaptor Grammars. Empirical Methods in Natural Language Processing, 2010.

ObamacareObamacare

Does Reuters Spin in Favor of the Palestinians?

Israelis did X. Palestinians did X.

Claims made by pro-Israeli media watchdog group honestreporting.com

Syntactic framing

“eleven people were lost in an explosion and fire”

“explosion killed eleven workers”

___X____ verbed ___Y____X caused the event Y was affected by the event

Greene 2007, Greene and Resnik 2009

“The dishwasher broke yesterday.”

“He broke the dishwasher yesterday.”

A hypothesis about syntactic framing

Facial expressions

Emotion

Construal

Sentiment

Linguistic expression

Anger

Terrorists destroyed the bus

The bus exploded

CausationIntent

Change of State…

Negative

They dispatched the bus

Ekman (2002)

Syntactic framing

“eleven people were lost in an explosion and fire”

“explosion killed eleven workers”

___X____ verbed ___Y____X caused the event Y was affected by the event

Greene 2007, Greene and Resnik 2009

NP1 V’ed NP2

NP1 chose to do V

NP1 caused V to happen

NP2 underwent a change of state

NP2 is distinct from NP1

V had a defined endpoint

NP1 killed NP2NP1 broke NP2

Syntactic Packaging and Semantic Properties

NP2 broke

Connecting semantic properties to implicit sentiment

• Have one set of subjects rate the extent to which the semantic properties are exhibited by a particular syntactic realization of a sentence’s meaning.

• Have a different set of subjects rate implicit sentiment for the same sentence.

• Show that the semantic property ratings predict the implicit sentiment ratings.

Rating Semantic Properties for Different Event Encodings

Telicity

Punctuality

Subject-Object Individuation

Kinesis

Affectedness of Object

Volition

Agency

Suffocation kills 24-year-old womanMan suffocates 24-year-old woman

…etc. (13 semantic property ratings total)

Relationship between semantic properties and implicit sentiment

• Property ratings: averaged across 18 subjects• Sympathy ratings: averaged across 31 subjects• Some strong individual correlations

– Volition: r = -.776

– Sentience: r = -.764

– Kinesis/movement: r = -.751

• Multiple regressions for 24 sentences

– Using verb, volition, telicity: • R2 = .78 (p < .001), adjusted R2 = .74

Computational linguists like data.

Where can I find lots and lots of utterances involving agendas, framing, and spin?

51

Oh yeah, it’s 2012!

52

Oops, never mind.

53

54

January 7 debate

55

(1) Gingrich: And I think protecting and upholding [marriage] doesn’t mean you have to go out and make life miserable for others.

(2) Gingrich: … it does mean you make a distinction between a historic sacrament of enormous importance in our civilization and simply deciding it applies everywhere and it’s just a civil right. It’s not.

(3) Huntsman: I don’t feel that my relationship is at all threatened by civil unions.(4) Huntsman: And I believe in reciprocal beneficiary rights. I think they should be part of civil unions, as well.

Summing up

• Working with naturally occurring data on a large scale revolutionized computational linguistics.

• Social science has begun to catch on.

• There’s a lot of fun work to do.

56

57

Acknowledgments:

I’d like to especially thank Jordan Boyd-Graber,Amber Boydstun,Stephan Greene,Eric Hardisty, Viet-An Nguyen

for discussions and collaboration on the work described in this talk.

Thank you!

58

Additional slides

Rewind a second

A closer look…

*Note that Noah Smith did point out this ambiguity!

Perspectives and framing

• Lexical framing

Death tax

Pro-life

Killer whales

Estate tax

Pro-choice

Orcas

Lexical framing

• Approach: non-parametric Bayesian modeling*

*Hardisty, Boyd-Graber, Resnik 2010; in progress.

November 12, 2007 Edition 41 Is the PA beginning to resemble the SLA?

two Palestinian views

The PA cannot remain transitional much longerby Ghassan Khatib

The situation in Nablus can only further discredit the PA in the eyes of its own public and strengthen comparisons with the SLA.

two Israeli views

The danger is thereby Yossi Alpher

Any comparison between Abbas/Salam and Lahd is, to say the least, not flattering.

Unite or dissolve an interview with Eyad Sarraj

The Palestinians have lost the ability to govern themselves, to make war or to make peace.

The political context is totally differentby Dani Reshef The SLA sought to join the Lebanese army as a territorial brigade.

Lexical framing

Lexical framing

Analysis of pro- and anti-death penalty sites

79 NAME 's 46 NAME 's case 19 NAME 's attorney 15 NAME 's execution 13 NAME 's lawyers 8 upheld NAME 's conviction 8 fight (for) NAME 's life 8 facts (of ) NAME 's case 7 NAME 's life 7 NAME 's lawyer

176 NAME 's 23 NAME 's car 22 NAME 's home 22 NAME 's body 17 NAME 's murder 17 NAME 's house 16 NAME 's death 15 NAME 's mother 14 NAME 's appeal 13 NAME 's wife

Rewind a second

A closer look…

*Note that Noah Smith did point out this ambiguity!

67

[Candidates] can decide how much of their debate time to devote to the topics presented through moderator and audience questions and how much time to spend “straying” to other topics that might be more advantageous. . . .

However, straying from the topic of a moderator’s question can be a risky strategy. . . . If the moderator in a debate asks the candidate a direct question about a topic, the candidate would be ill-advised to ignore that topic altogether, no matter how much he or she might prefer not to talk about it, as doing so might be seen as “dodging” the question. Going off topic is thus a moderately costly signal, one that candidates have incentives to send only when the potential votes gained by shifting to the more favored topic outnumber the potential votes lost by bucking social protocol.

A. Boydstun, C. Phillips, and R. Glazier, It’s the Economy Again, Stupid: Agenda Control in the 2008 Presidential Debates, under review.

Time flies like an arrow.

68

Fruit flies like a banana.

I see a bird

pronnoun

verbnoun

pronnoun

nounverb

2×2×2×2 = 24 = 16

Time flies like an arrow.

Fruit flies like a banana.

Let’s start at the beginning

Documents

The Linguistics of Spin: A Computational Linguist’s Forays into Social Science Philip Resnik University of Maryland AAAL 2012