Pictures and Words

Pictures and Words

Behavioral Decision Making: Building Approaches from Laboratory Insights

October 31th 2011

Elinor Amit

How do we think about things?...

The 3 main approaches:

• Words (inner speech) (e.g., Vygotsky, 1934; Oppenheim & Dell, 2010)

• Pictures (mental imagery) (e.g., Kosslyn et al)

• Abstract, a-modal representations (e.g., Caramazza, Hillis, Rapp, & Romani, 1990; Lambon Ralph, Graham, Patterson, & Hodges, 1999; Rapp, Hillis, & Caramazza, 1993; Tyler & Moss, 2001; Mahon & Caramazza, 2008)

How do we think about things?...

The 3 main approaches:

• Words (inner speech) (e.g., Vygotsky, 1934; Oppenheim & Dell, 2010)

• Pictures (mental imagery) (e.g., Kosslyn et al)

• Abstract, a-modal representations (e.g., Caramazza, Hillis, Rapp, & Romani, 1990; Lambon Ralph, Graham, Patterson, & Hodges, 1999; Rapp, Hillis, & Caramazza, 1993; Tyler & Moss, 2001; Mahon & Caramazza, 2008)

“the two most vivid forms of working memory are mental images... and snatches of inner speech”

Pinker, 2008

Apple

Red apple

Apple

Red apple

Differences between words and pictures

Red apple

Medium

words pictures

Dissimilar to the object:

Cognized

Arbitrary

Abstract

Gist

Simple

Similar to the object:

Perceived

Not arbitrary

Concrete

Peripheral

Complex

- The association between medium and psychological distance

- The implications of the medium of representation to moral judgment

We will discuss today…

- The association between medium and psychological distance

- The implications of the medium of representation to moral judgment

We will discuss today…

The medium/morality hypothesis

Proximal event Visual representation

Distal event Verbal representation

It depends…

Amit, Algom, & Trope, 2009

How do we think about things?...

What is psychological distance?

The distance between an individual and a target.

• Subjective• Egocentric

What is psychological distance?• Temporal: How much time separates between the individual’s present and the target event

• Spatial: How far in space is the event from the individual

• Social: How different is the social target from the individual

Bar Anan, Liberman & Trope, 2006

Why is medium associated with distance?

• Pictures and words serves different cognitive functions:

• Words preserve the invariant & essential properties of the item for a distal use

• Pictures preserve the stimulus in details for an immediate use.

Construal Level Theory (CLT) Mental construal processes serve to traverse psychological distances and switch between perspectives.

Trope & Liberman, 2003; 2010

Distal events not so much information =>

Proximal events there is information =>

abstract representation, entailing the essence, invariant features of the referent.

concrete, subordinate representation.

Main hypothesisPictures and words are associated with psychological

distance:

• People elect to represent close targets in pictures and distal targets in words.

• Pictures impart a feeling of proximity, whereas words impart a feeling of distance.

• cognitive processing: - Identification - Categorization - Selective attention - Memory

• Social cognition: - Interpersonal communication - Moral judgment

• Neural correlates

• cognitive processing: - Identification - Categorization - Selective attention - Memory

• Social cognition: - Interpersonal communication - Moral judgment

• Neural correlates

Speeded Identification: Social distance

football

soccer

Amit, Algom, & Trope, 2009, Jep: General

Socially proximal

Socially distal

Medium: pictures, wordsDistance: socially near, far Task: speeded identification

Speeded Identification: Social distance

570

590

610

630

650

picture word

Reaction Time (ms)

proximal

distal

F(1,13)=7.63, p < .05Amit, Algom, & Trope, 2009, Jep: General

Speeded Identification: Spatial distance

Medium: pictures, wordsDistance: spatially near, far Task: speeded identification

Amit, Algom, & Trope, 2009, Jep: General

Speeded Identification: Spatial distance

Amit, Algom, & Trope, 2009, Jep: General

600

620

640

660

680

700

picture word

Reaction Time (ms)

proximal

distal

F(1,15)=6.3, p < .05

Amit, Rim, Greene, & Trope, in prep

+

TomorrowCHAIR

+

10 yearsAPPLE

+

20 sec

1 sec

Mental travel

Exp.1: event related design, 11 subjectsExp.2: block design, 10 subjects

near>far

N=10, p =.001

calcarine

far>near

BA 21 L

Language regions Visual regions

Fusiform L

BA47L

precuneus

Fusiform R

Mental travel (Harvard sample)

Interim conclusion

Pictures are associated with proximity, whereas words are associated with distance.

- This association influence performance in various cognitive tasks (e.g., identification)

- People spontaneously elect to represent proximal things visually (embodied cognition), and distal things verbally.

Implications of the medium/distance association for public policy

paths

cost

air pollution

convenience for costumers

pressure from the government

bus driver’s union demands

location of businesses

existing routes

location of main public services

Walk a lotWait a little

Walk a littleWait a lot?

- The association between medium and psychological distance

- The implications of the medium of representation to moral judgment

We will discuss today…

- The association between medium and psychological distance

- The implications of the medium of representation to moral judgment

We will discuss today…

The medium/morality hypothesis

Proximal event Visual representation

Distal event Verbal representation

The medium/distance hypothesis

The medium/morality hypothesis

Proximal eventVisual representation

Distal eventVerbal representation

The medium/distance hypothesis

The medium/morality hypothesis

Proximal eventVisual representation

Distal eventVerbal representation

The medium/distance hypothesis

If you see something

Do something!

Implications for behavior…

• Pictures “emergency” reaction

• Words not so much

Visual processing

High emotionalreaction

Verbal processing

Low emotionalreaction

5 >1

Psychologically proximal

Psychologically distal

Visual processing

High emotionalreaction

Verbal processing

Low emotionalreaction

5 >1

Psychologically proximal

Psychologically distal

Emotionally-drivenjudgments

“cognitive”-drivenjudgments

Dual-Process Moral Cognition

5 >1 Yes

?

Dual-Process Moral Cognition

5 >1

No

Yes

?

XGreene et al, Cognition, 2009

Visual processing deontological judgments (rely on emotional reactions)

Verbal processing utilitarian judgments (rely on “controlled” reaction)

Predictions

Another rationale for the medium/moral judgment prediction

Pictures thinking about means (consistent with deontological moral reasoning)

Words thinking about end-goals (consistent with utilitarian moral reasoning)

Exp 1: How media preferences are related to moral judgments?

L.A. Times Wall Street journal

r(108) = 0.233, p = 0.011*After controlling for level of education: r(107) = .22, p = 0.018*

Exp 1: How media preferences are related to moral judgments?

More utilitarian

R(170) = -0.18, p = 0.017*After controlling for education: r(169) = -0.18, p = 0.01**

Hours/day of TV watching

Exp 1: How media preferences are related to moral judgments?

More utilitarian

Conclusions from Experiment 1:

Media consumption preferences are associated with moral judgments tendencies:

Visual deontological styleVerbal utilitarian style

* Cannot be accounted for by education

Exp 2: How cognitive style is related to moral judgments?

Amit & Greene, under revision

Verbal - Visual Accuracy

utilitarian

N = 50r = .35p = .01

R(50)=.346, p < .01

Exp 2: How cognitive style is related to moral judgments?

More utilitarian

Conclusions from Experiment 2:

Cognitive style is associated with moral judgments tendencies:

Visual style deontological Verbal style utilitarian

Amit & Greene, under revision

Enemy soldiers have taken over your village….

Is it appropriate to smother your baby?

2.5 secno…..yes

5 sec

Exp 3: How visual and verbal interference affect moral judgments?

5 sec

Amit & Greene, under revision

You are standing near a footbridge…

Is it appropriate to smother your baby?

no…..yes

Condition 1 Condition 2

Prediction:

• pictorial interference prevents representing the dilemma visually, thus leads to more utilitarian judgment.

• Verbal interference prevents representing the dilemma verbally, thus leads to more deontological judgment.

Amit & Greene, under revision

Interference type

* n.s

Exp 3: How visual and verbal interference affect moral judgments?

More utilitarian

Amit & Greene, under revision

Conclusions from Experiment 3:

Visual imagery creates more deontological judgments.

No effect for words (compared to control) – suggests that the default mode of thinking about moral dilemmas is visual.

Amit & Greene, under revision

Exp. 4

• Subjects read mean vs. side dilemma. • Than made moral permissibility judgment• And reported whether they imagined the one

to be killed or the five to be saved.

F(1,331) = 15.1, p < .0001

What did you imagine more?

The 5

The one

Dilemma(mean, side)

imagery

Moral judgment

P = .000*** P = .001**

P = .000***

Sobel = -2.04, p = .004

Take-home messageIntriguing implications for the differences between

pictures and words…- People think about proximal things in pictures

and about distal things in words- They think about “means” in pictures and about

“end-goals” in words- Thinking in pictures lead to deontological moral

judgments, and thinking in words lead to utilitarian moral judgments.

Thanks

• Joshua Greene• Yaacov Trope• Daniel Algom• Galit Yovel• Evelina Fedorenko• SoYon Rim• Georg Halbeisen

•Nobuhito Abe•Alek Chakroff•Fiery Cushman•Joe Paxton•Steven Frenkland•David Rand•Regan Bernhard•Ryan Halprin•Sara Gottlieb•Allison Gofman•Rebecca Fine•Warren Winter•Paul Lively

Thanks!