1 Moral Theory Meets Cognitive Science How the Cognitive Sciences Can Transform Traditional Debates Stephen Stich Dept. of Philosophy & Center for Cognitive

1

Moral Theory Meets Cognitive Science

How the Cognitive Sciences Can Transform Traditional Debates

Stephen StichDept. of Philosophy

& Center for Cognitive ScienceRutgers University

[email protected]

Jean Nicod

Lectures

2007

2

Lecture 4

Stephen StichDaniel Kelly

Joshua Knobe

Debunking Moral IntuitionA Hodgepodge of Multipurpose Kludges

Jean Nicod

Lectures

2007

3

Lecture 4

Stephen StichJoshua Knobe Daniel Kelly

Debunking Moral IntuitionA Hodgepodge of Multipurpose Kludges

Jean Nicod

Lectures

2007

4

Introduction

Philosophers – and more recently cognitive scientists – have offered many accounts of the psychological mechanisms & processespsychological mechanisms & processes underlying intuitive moral judgmentintuitive moral judgment

Moral philosophers have always insisted that sometimessometimes the outputs of those processes – people’s “moral intuitions”“moral intuitions” – are not to be not to be trusted trusted though they disagree about whenwhen skepticism is skepticism is

warrantedwarranted

5

Introduction

Our goal in this talk is to sketch a newly newly emerging perspectiveemerging perspective on the mechanisms underlying moral intuition …

and to explore its implicationsimplications for the hotly debated issue of whether and when intuitions whether and when intuitions should be relied onshould be relied on

6

Introduction

Philosophers have typically assumed that those Philosophers have typically assumed that those mechanisms were mechanisms were well designedwell designed for … for … somethingsomething

But we now have reasons to think that But we now have reasons to think that many of many of theses mechanismstheses mechanisms are are not well designed not well designed for for ANYTHINGANYTHING

7

Introduction

Moral Psychology is a Moral Psychology is a KludgeKludge

A hodgepodge of multipurpose A hodgepodge of multipurpose kludges!kludges!

8

Introduction

Before explaining and defending this claim it Before explaining and defending this claim it will be useful to consider some of the will be useful to consider some of the reasonsreasons that philosophers – both classic & that philosophers – both classic & contemporary – have offered contemporary – have offered for discounting for discounting moral intuitionsmoral intuitions

9

Philosophical Background

When should we be skeptical about When should we be skeptical about moral intuitions?moral intuitions?

The “Moral SenseMoral Sense” & “Ideal ObserverIdeal Observer” traditions

Reflective EquilibriumReflective Equilibrium

Evolutionary argumentsEvolutionary arguments debunking intuition

10



Ideal observer theorists maintain that our moral intuitions are correctintuitions are correct (or justified) when made under when made under ideal conditionsideal conditions

When conditions are not idealnot ideal – e.g. when we have false beliefs about relevant non-moral matters, or we are irrational – our intuitions are not to be trustedintuitions are not to be trusted

11



For HutchesonHutcheson – an important precursor of this tradition – moral judgments are the product of a “moral “moral sense”sense” implanted in us by “the Author of Nature”“the Author of Nature”

Thus it can be relied uponcan be relied upon when doing its job properly

But, like other senses, it can misleadmislead when conditions are unfavorableunfavorable

12


Reflective Equilibrium Reflective Equilibrium

Rawls’ “Decision Procedure for Ethics” (1951)

Narrow Reflective EquilibriumNarrow Reflective Equilibrium

Bring intuitions about

particular cases

moral principles

into accord

To do this, sometimes an intuition about a particular case must be rejectedrejected

13


Wide Reflective EquilibriumWide Reflective Equilibrium

Bring intuitions about

particular cases

moral principles

into accord with the rest of our beliefsthe rest of our beliefs

including beliefs about scientific matters, history, politics – even metaphysics & semantics

Even more of our intuitionsEven more of our intuitions about particular cases will have to be rejectedrejected


Evolutionary argumentsEvolutionary arguments debunking intuition

Perhaps the most influential writer in this tradition is Peter Singer

Updated in “Ethics & Intuition (2005)

The

ExpandingCircle

Ethics and Sociobiology

Peter Singer

FARRAR, STRAUS & GIROUXNew York

1981

15


In The Expanding Circle, Singer focuses on nepotistic intuitionsnepotistic intuitions which maintain that, in various domains, we ought to value the welfare of our kinkin and tribesmentribesmen more than the welfare of people outside these circles

The psychological processes leading to judgments of this sort were adaptive in ancestral environmentsadaptive in ancestral environments (and perhaps they still are)

But once we see why we have these nepotistic & why we have these nepotistic & tribal intuitionstribal intuitions, Singer suggests, we can also see that there is no good reason to use themthere is no good reason to use them in a “decision procedure for ethics”

16


In “Ethics and Intuition” (2005) Singer develops the argument by focusing on the sort of “trolley problems” “trolley problems” that have loomed large in recent philosophical and empirical studies

17


Singer (following Greene) maintains that the neuroscientific evidenceneuroscientific evidence suggests that intuitions about the “footbridge” case are the result of our emotional reactionemotional reaction to cases in which harm is caused by the sort of harm is caused by the sort of interaction that would have occurred in interaction that would have occurred in ancestral environmentsancestral environments

18


“The salient feature that explains our different intuitive judgments concerning the two cases is that the footbridge case is the kind of situation that was likely to arise during the eons of time over which we were evolving; whereas the standard trolley case describes a way of bringing about someone’s death that has only been possible in the past century or two…. But what is But what is the moral salience of the fact that I have killed someone in the moral salience of the fact that I have killed someone in a way that was possible a million years ago, rather than in a way that was possible a million years ago, rather than in a way that became possible only two hundred years ago? a way that became possible only two hundred years ago? I would answer: none….I would answer: none….

19


““At [a] more general level …this … At [a] more general level …this … casts serious doubt on the casts serious doubt on the method of reflective equilibriummethod of reflective equilibrium. There is little point in . There is little point in constructing a moral theory designed to match considered constructing a moral theory designed to match considered moral judgments that themselves stem from our evolved moral judgments that themselves stem from our evolved responses to the situations in which we and our ancestors responses to the situations in which we and our ancestors lived during the period of our evolution as social mammals, lived during the period of our evolution as social mammals, primates, and finally, human beings. We should, with our primates, and finally, human beings. We should, with our current powers of reasoning and our rapidly changing current powers of reasoning and our rapidly changing circumstances, be able to do better than that.” (348)”circumstances, be able to do better than that.” (348)”

What I am saying, in brief, is this. Advances in our understanding of ethics … undermine some conceptions of doing ethicsundermine some conceptions of doing ethics …. Those conceptions of ethics tend to be too respectful Those conceptions of ethics tend to be too respectful of our intuitions. Our better understanding of ethics gives of our intuitions. Our better understanding of ethics gives us grounds for being less respectful of them.us grounds for being less respectful of them.” (349)

20


We agree with Singer’s skepticism about intuition

But we also think his skepticism skepticism is

not radical enough!not radical enough!

21


AssumptionsAssumptions that SingerSinger and the friends of the friends of intuition intuition shareshare:

The psychological system underlying our moral intuitions is well designedwell designed

Thus there is some point point to – or reason forreason for – the intuitive moral judgments people make when the system is working properly

Though Singer (unlike the friends of intuition) insists that the function the system is designed for is of dubious dubious moral importancemoral importance, and thus that the intuitions are not to not to be taken seriouslybe taken seriously

22


We believe that the engine of moral intuition is not is not well designedwell designed at all

Far from being the sort of “elegant machine”“elegant machine” celebrated in the writings of some evolutionary psychologists, we think that it is a kludgekludge

a cluster of mechanisms cobbled together rather cobbled together rather awkwardlyawkwardly from bits of mental machinery most of which were designed for functions that have designed for functions that have noting to do with moralitynoting to do with morality

23


To use a term that may be more common in Paris, we maintain that the engine of moral intuition is the result of bricolagebricolage

François Jacob Claude Lévi-Strauss

24


This explains many of the quirksquirks of moral intuition …

And provides yet another reason to be skepticalskeptical of their use in moral deliberationuse in moral deliberation

25

Overview of the Rest of the Talk

26

Overview of the Rest of the Talk

Two examples of the “kludginess” of the mechanisms underlying moral intuition Dan Kelly’sDan Kelly’s work on Moral DisgustMoral Disgust Joshua Knobe’sJoshua Knobe’s work on intentionality intentionality

judgments & unconscious moral unconscious moral judgmentsjudgments

From kludginess to skepticismFrom kludginess to skepticism

27

Kelly on Disgust

KellyKelly has constructed a rich, nuanced, empirically supported account of the psychological psychological mechanisms underlying the mechanisms underlying the uniquely human disgust uniquely human disgust systemsystem and how that system how that system evolvedevolved

In this talk I’ll only have time to for a brief sketch of two central themes

Daniel Kelly

28

Kelly on Disgust

The Entanglement ThesisThe Entanglement Thesis Disgust is itself a kludgeDisgust is itself a kludge – a uniquely

human emotion produced by the merger of two distinct systems

The Co-Optation ThesisThe Co-Optation Thesis After the merger, disgust was co-optedco-opted by

the norm systemthe norm system the ethnic boundary systemthe ethnic boundary system

which were central elements in the emergence of human ultra-socialityhuman ultra-sociality

29

Kelly on Disgust

Kelly assembles a vast array of evidencevast array of evidence for these theses, drawn from neuroscience social psychology cognitive psychology developmental psychology evolutionary psychology gene-culture co-evolution theory

As usual, the devil is in the detailsthe devil is in the details So I join Paul Rozin in urging that you read the

work as it appears in print

30

Kelly on Disgust The Entanglement Thesis

Disgust exhibits a puzzling array of

elicitorselicitors

which evoke an equally puzzling cluster of

responsesresponses

31


Elicitors include FoodsFoods: dog meat, grubs, insects

32


Elicitors include FoodsFoods: dog meat, grubs, insects Substances associated with the bodybody: feces, vomit,

spit Organic decayOrganic decay People and objects associated with illnessillness: a shirt

once worn by a person with leprosy Sexual practicesSexual practices: necrophilia, incest Some moral transgressionsmoral transgressions & transgressorsors: rape,

torture, child molestation Members of low status outgroupsoutgroups: untouchables,

Jews

33




once worn by a person with leprosy Sexual practicesSexual practices: necrophilia, incest Some moral transgressionsmoral transgressions & transgressors: rape,


Jews

Some elicitors are pan-cultural

34






Jews

Others are culturally local(or idiosyncratic)

35


The disgust response includes Gape face (occasionally accompanied by retching) Feeling of nausea Sense oral incorporation

Quick withdrawal A more sustained & cognitive sense of

offensiveness A more sustained & cognitive sense of

contamination

36


How are all of these connectedconnected?

The Entanglement ThesisThe Entanglement Thesis maintains that the human emotion of disgust is the result of the fusionfusion of two distinct mechanisms each of which has homologous counterparts in

other species though they have combined only in humansonly in humans

37


One mechanism (“the poison avoidance the poison avoidance mechanismmechanism”) is directly linked to digestiondigestion It evolved to regulate food intakeregulate food intake and protect the gut

against ingested substances that are poisonous or poisonous or otherwise harmfulotherwise harmful

It was designed to expel substancesdesigned to expel substances entering the gastro-intestinal system via the mouth

And to acquire new elicitorsnew elicitors very quickly As John Garcia famously demonstrated, ingested

substances that induce gut-based distress often generate acquired aversions

38


The other mechanism (“the parasite avoidance the parasite avoidance mechanismmechanism”) Evolved to protect against infection from pathogens pathogens

and parasitesand parasites, by avoiding them Not specific to ingestion, but serves to guard against

coming into close physical proximityclose physical proximity with infectious infectious agentsagents

This involves avoiding not only visible pathogens and visible pathogens and parasitesparasites, but also places, substances and other places, substances and other organismsorganisms that might be harboring them

39





contamination

These elements of the disgust responseresponse aretraceable to the poison avoidance system

40





contamination

and these are traceable to the parasite avoidance poison system

41






Jews

These elicitorselicitors are traceable to the poison avoidance system

42






Jews

and these are traceable to the parasite avoidance system

43


One bit of evidence supporting the Entanglement Thesis is that different components of that response are on different developmental schedulesdifferent developmental schedules Distaste & gape are present within the first year of life Contamination sensitivity emerges significantly later

Once the full system in in place, the components of the response are produced together – they form a nomological clusternomological cluster Any elicitor of disgust will reliably produce all or most of

those clustered components

44


A puzzleA puzzle: Why should the sight of a festering sore or a person

with leprosy evoke a gape face and a feeling of nausea?

The solutionThe solution: Disgust is a kludgekludge!

But it is kludge with features that could be readily co-opted and put to other uses as humans began living in larger groups and human ultrasocialityultrasociality emerged

45

Kelly on Disgust The Co-Optation Thesis

46


The Gape Face as a SignalThe Gape Face as a Signal As group size increased, there was an increasing need

for a perspicuous signalperspicuous signal warning of dangerous dangerous foods and risk of infectious diseasefoods and risk of infectious disease

In humans, the face and facial expressions provide a rich source of such social information

The gape facegape face, which clearly has roots in the facial motions that accompany retching, was co-opted as a co-opted as a signalsignal, warning others not just against toxic foodstoxic foods, but also against the presence of parasites and parasites and contagious pathogenscontagious pathogens

47


Co-Optation by the Norm SystemCo-Optation by the Norm System As group size increased, there was increased need for

complex social social coordinationcoordination

The norm systemnorm system – whose structure we considered briefly in the 2nd Lecture – played an important role in facilitating this co-ordination

And the disgust system had features that made it an obvious candidate to be co-opted by the norm system as it evolved

48


The S&S model suggests that compliance motivation & punitive motivation are linked to “the emotion system”

49


Acquisition Mechanism

Execution Mechanism

identi

fy n

orm

im

plic

ati

ng

behavio

r

infe

r co

nte

nts

of

norm

ati

ve r

ule

s

compliance

motivation

punitive motivati

on

emotion

systemRule-related

reasoning capacity

norm data base

r1---------- r2---------- r3---------- …… rn----------

ProximalCues in

Environment

judgment

other emotion triggers

beliefs

explicit reasonin

g

post-hoc justificatio

n

50


But psychological & neurological evidence indicates that there are several separate emotion systemsseveral separate emotion systems – the disgust system being one of them

51



Execution Mechanism

identi

fy n

orm

im

plic

ati

ng

behavio

r

infe

r co

nte

nts

of

norm

ati

ve r

ule

s

compliance

motivation

punitive motivati

on

DISGUST

Rule-related

reasoning capacity

norm data base

r1---------- r2---------- r3---------- …… rn----------

ProximalCues in

Environment

judgment


beliefs

explicit reasonin

g


n

other emotion

s

52


Disgust is a natural candidate to provide both compliance & punitive motivationcompliance & punitive motivation for norms that involve intrinsically disgusting matters, like the disposal of corpses & bodily wastes, and other activities that are antecedently salient to the disgust system, like eating practices

ComplianceCompliance is motivated by making norm violating norm violating behavior disgustingbehavior disgusting & thus aversive

Punitive motivationPunitive motivation is provided because the violator violator is considered dirty and contaminatedis considered dirty and contaminated and is avoided or shunned

53


Execution Mechanism

identi

fy n

orm

im

plic

ati

ng

behavio

r

infe

r co

nte

nts

of

norm

ati

ve r

ule

s

compliance

motivation

punitive motivati

on

DISGUST

Rule-related

reasoning capacity

norm data base

r1---------- r2---------- r3---------- …… rn----------

ProximalCues in

Environment

judgment


beliefs

explicit reasonin

g


n

other emotion

s


54


The norm system is thus a kludge built with kludgy kludge built with kludgy partsparts Not surprisingly, this can lead to some very quirky and quirky and

disturbing behaviordisturbing behavior

Several recent studies have focused on the fact that the disgust systemdisgust system can be triggeredtriggered by many things that have nothing to do with normsnothing to do with norms

but even when triggered by these non-moral itemstriggered by these non-moral items, the disgust system can have dramatic and persistent dramatic and persistent influence on a person’s judgments about moral influence on a person’s judgments about moral issuesissues

55



Execution Mechanism

identi

fy n

orm

im

plic

ati

ng

behavio

r

infe

r co

nte

nts

of

norm

ati

ve r

ule

s

compliance

motivation

punitive motivati

on

DISGUST

Rule-related

reasoning capacity

norm data base

r1---------- r2---------- r3---------- …… rn----------

ProximalCues in

Environment

judgment


beliefs

explicit reasonin

g


n

other emotion

s

56


Wheatley & HaidtWheatley & Haidt have shown that when participants are hypnotically inducedhypnotically induced to feel a brief pang of disgustdisgust when they encounter the work “often”“often” and then presented with the following scenario

“Dan is a student council representative at his school. This semester he is in charge of scheduling discussions about academic issues. He oftenoften picks topics that appeal to both professors and students in order to stimulate discussion.”

many judge that Dan is doing something wrongsomething wrong!

57


SchnallSchnall et al. have shown participants make more severe moral judgments when the judgments are made in a disgusting office:

greasy pizza boxes sticky chair a dried up smoothie a chewed up pen

58


Other studies have focused on prima facie irrational downstream consequencesdownstream consequences of the disgust system being triggered in moral deliberation

59



Execution Mechanism

identi

fy n

orm

im

plic

ati

ng

behavio

r

infe

r co

nte

nts

of

norm

ati

ve r

ule

s

compliance

motivation

punitive motivati

on

DISGUST

Rule-related

reasoning capacity

norm data base

r1---------- r2---------- r3---------- …… rn----------

ProximalCues in

Environment

judgment


beliefs

explicit reasonin

g


n

other emotion

s

Downstream consequenc

es

60


The Lady Macbeth EffectThe Lady Macbeth Effect

Zhong & LiljenquistZhong & Liljenquist have shown that recalling an unethical deedunethical deed increased the desire for products related to cleansing, like antiseptic wipes

And that cleaning one’s handscleaning one’s hands after describing a past unethical deed reduced moral emotionsreduced moral emotions like guilt & shame

and also reduced the likelihood that participants reduced the likelihood that participants would volunteer to helpwould volunteer to help a desperate graduate student!

61


The Lady Macbeth EffectThe Lady Macbeth Effect

Schnall et al. Schnall et al. (unpublished) compared judgments about moral severity in two groups of participants

One group had just used an alcohol-based cleansingcleansing gel on their hands

The other group had just used an ordinary, non-non-cleansingcleansing hand cream

The moral judgments of those using the cleansing gelcleansing gel were significantly less severe!significantly less severe!

62


Ethnic Boundary MarkersEthnic Boundary Markers Boyd & RichersonBoyd & Richerson & their students have argued that

another crucial step in the development of human ultra-sociality was the emergence of mechanisms which allow people to recognize members of their own tribe or “ethnie”

This is important because in-group members share beliefs & norms norms that facilitate coordinationfacilitate coordination

63


Since different cuisinescuisines & eating practiceseating practices are one of the more visible correlates of ethnie membership, and since disgust is heavily involved in regulating food intake, disgust was a natural candidate to be co-opted by the emerging system of ethnic identification

Eating practices of out-groups and other readily detectable signs of out-groupsigns of out-group membership came to came to evoke disgustevoke disgust

And disgust came to provided a significant part of the motivation motivation to avoid out-group members

64


Though the evolutionary function of the ethnic boundary marker system was to facilitate cooperation by keeping groups apart, the kludgy solutionkludgy solution to this problem has some unfortunate consequences

Out-group members are not simply avoided, they are also considered offensive & contaminatingoffensive & contaminating

People who embrace different norms are often felt to be disgusting and sub-human! disgusting and sub-human!

65

Kludge Meets Kass

66

Kludge Meets Kass

Leon Kass, M.D., Ph.D.

Conservative bio-ethicist

Chairman of the U. S. A. President's Council on Bioethics from 2002 to 2005

67

Kludge Meets Kass In his book, Life, Liberty & the Defense of Dignity (2002),

there is a chapter called “The Wisdom of Repugnance”“The Wisdom of Repugnance”

Kass maintains that

"in crucial cases...repugnance is the emotional expression of deep wisdomdeep wisdom, beyond reason's power fully to articulate it.”

“In this age in which everything is held to be permissible so long as it is freely done, and in which our bodies are regarded as mere instruments of our autonomous rational will, repugnance may be the only voice left that speaks up to defend the core of our humanity. Shallow Shallow are the souls that have forgotten how to shudderare the souls that have forgotten how to shudder."

68

Kludge Meets Kass

The claims play a central role in Kass’ critique of human human cloningcloning

Others have adopted the idea to argue against abortionabortion, pornographypornography & same-sex marriagesame-sex marriage

69

Kludge Meets Kass

Some philosophers, most notably Martha Nussbaum, have challenged Kass, arguing that disgust should be discounted in moral & legal deliberation because (roughly) it reminds us of our animal origins

70

Kludge Meets Kass

I think Kelly’s work offers a far more

plausibleplausible &

powerfulpowerful

critiquecritique

71

Kludge Meets Kass

There is no reason to think there is

wisdom in repugnancewisdom in repugnance

because

Disgust is a KludgeDisgust is a Kludge

and the psychological system that bases moral judgments on disgust is a

Kludge twice over!Kludge twice over!

72

Kludge Meets Kass

Anti-Jewish Nazi propaganda often invoked the imagery and language of disgust, purity, contamination & dehumanization very flagrantly

A poster advertising the film The Eternal Jew

Hitler described “the Jew” as “a maggot in a festering abscess, hidden away inside the clean and healthy body of the nation”

73

Knobe on Norms and Intentional Action

My second example draws some elegant and exciting work by Joshua Knobe which demonstrates the way in which unconscious moral judgments – judgments which an agent may explicitly reject – can nonetheless have significant impact on a range of morally relevant intuitions

74


In his new book, Kluge, Gary Gary MarcusMarcus argues that more recently evolved, computationally slow and consciously accessible mental processes – “System 2 Processes”“System 2 Processes” in the currently fashionable jargon – were grafted onto older (System 1) psychological systems designed for quite different purposes

The resulting kludgy architecturekludgy architecture accounts for many of the quirks and quirks and shortcomingsshortcomings that contemporary cognitive science has discovered

75


I think that Knobe’s workKnobe’s work provides an important & disquieting illustration of this phenomenon in the moral domainin the moral domain

76


The story begins with “the side effect effect”“the side effect effect” (aka the Knobe effect) – one of best known and most surprising finding in the emerging field of experimental philosophy

Knobe (2003) reports an experiment in which participants were presented with a pair of almost identical vignettes

77


The vice-president of a company went to the chairman of the board and said, ‘We are thinking of starting a new program. It will help us increase profits, but it will also harmharm [helphelp] the environment.’

The chairman of the board answered, ‘I don’t care at all about harmingharming [helpinghelping] the environment. I just want to make as much profit as I can. Let’s start the new program.’

They started the new program. Sure enough, the environment was harmed harmed [helpedhelped].

78


In the harmharm case, participants were asked how much blameblame the chairman deserved (on a scale from 0 – 6) and whether he intentionallyintentionally harmed the environment

In the helphelp case, participants were asked how much praisepraise the chairman deserved (on a scale from 0 – 6) and whether he intentionallyintentionally helped the environment

In the harmharm case, 82% said the chairman brought about the side-effect intentionally

In the helphelp case, 77% said the chairman did notdid not bring about the side-effect intentionally

79


Knobe’s initial hypothesisinitial hypothesis was that people’s moral people’s moral assessment of the side-effectassessment of the side-effect plays a substantial role in determining whether they are willing to say that the side-effect was brought about intentionallyintentionally A judgment that the side-effect is morally badthe side-effect is morally bad makes

it more likely that it will be judged to be intentionalintentional

Though this seems incompatible incompatible with the widespread idea that judgments of intentionality are judgments about a purely factual mattera purely factual matter, it does have an an obvious rationaleobvious rationale since judgments about whether an action is intentional play a central role in determining whether an agent deserves praise or blame

80


Subsequent research showed that, if the hypothesis is understood as a claim about the effect of moral the effect of moral judgments that people consciously makejudgments that people consciously make, this hypothesis is mistaken mistaken

The problem emerges clearly in study Knobe ran in collaboration with David Pizarro & Paul Bloom

81


Liberal university studentsLiberal university students were given Knobe-style Knobe-style vignettesvignettes in which an advertising executive approves an ad campaign which has the side-effectside-effect of

encouraging interracial sex encouraging interracial sex or placing gardenias in one’s office

82


NoneNone of the participants judged that inter-inter-racial sexracial sex (or placing gardenias) is morally morally wrongwrong

But participants were much more inclined to say that the executive intentionally intentionally encouraged interracial sexencouraged interracial sex

Explicit moral judgmentsExplicit moral judgments cannot explain the difference in judgments about the intention-intention-ality ality of the side-effects

83


However, (following Pizarro & Bloom) Knobe has recently proposed that perhaps participants were making non-conscious normative judgmentsnon-conscious normative judgments that the behavior in question violates a normthe behavior in question violates a norm that is made salient by the question or situation, even if it is a norm that they explicitly rejectexplicitly reject

84


The picture Knobe now proposes looks like this:

“In reaching a conscious moral judgmentconscious moral judgment, we can consider a variety of different moral norms, weigh these norms against each other, perhaps even determine that some of the norms are themselves unjustified.”

85


Non-conscious moral judgmentsNon-conscious moral judgments are formed through a much simpler (system-1 style) process

They are formed extremely quicklyextremely quickly and therefore involve very shallow processinginvolve very shallow processing

In generating a non-conscious moral judgment, the the only norms we consider are the ones that first come only norms we consider are the ones that first come to mindto mind. We do not searchsearch for additional norms; we do not weighweigh norms against each other; we do not ask whether any of the norms might themselves be unjustifiedunjustified.

86


Instead, we simply determine whether the behavior in the behavior in question violates any of the norms question violates any of the norms in the very limited set we are considering

If it does, we classify it as a transgressiontransgression. It is this this judgment judgment as to whether or not the behavior is a transgression that then influences our intuitions influences our intuitions about intentionalabout intentional action.

87


The theory predicts that the most salient norms the most salient norms evoked by a given caseevoked by a given case will be the ones used to in making intentionality judgments, even if subsequent reflection leads the agent to think that there is nothing wrong with violating the norm – or that doing so would be a very good thing.

Here is a vignette that Knobe has recently used to test this idea

88


In Nazi Germany, there was a law called the ‘racial identification law.’ The purpose of the law was to help identify people of certain races so that they could be rounded up and sent to concentration camps. Shortly after this law was passed, the CEO of a small corporation decided to make certain organizational changes. The Vice-President of the corporation said: “By making those changes, you’ll definitely be increasing our profits. But you’ll also be violatingviolating [fulfillingfulfilling] the requirements of the racial identification law.” The CEO said: “Look, I know that I’ll be violatingviolating [fulfillingfulfilling] the requirements of the law, but I don’t care one bit about that. All I care about is making as much profit as I can. Let’s make those organizational changes!” As soon as the CEO gave this order, the corporation began making the organizational changes. 81%81% of subjects in the violateviolate condition said that he violated

the requirements intentionally; 30%30% of subjects in the fulfillfulfill condition said that he fulfilled the requirements intentionally.

89


Knobe’s theory is certainly not the last wordnot the last word on how how intentionality judgments are generatedintentionality judgments are generated His work has inspired dozens of other researchers

there are many studies I have not mentioned and many others are underway

90


However, IFIF Knobe’s theory is on the right track, then intentionality judgmentsintentionality judgments are a product of a kludgy kludgy architecturearchitecture which can be influenced by norms and judgments which the agent is not aware ofis not aware of, and does not endorsedoes not endorse

This raises serious questionsraises serious questions about the useuse of those judgments in further moral deliberationmoral deliberation, or in the lawlaw

91

From Kludginess to SkepticismFrom Kludginess to Skepticism

Both Kelly’s & Knobe’s work support the hypothesis that motivates this talk

The psychological mechanism underlying moral intuition is

A Hodgepodge of Multipurpose KludgesA Hodgepodge of Multipurpose Kludges

92


Suppose that’s right. What should we conclude about What should we conclude about moral intuition?moral intuition?

The answer is NOTNOT that all moral intuition should be all moral intuition should be rejectedrejected

nor even that intuitions that are closely tied to kludgy features of the mind should be rejected

For, as Shaun Nichols has argued, some of the most admirable featuresadmirable features of the cultural evolution of normscultural evolution of norms – including the increased scope and acceptance of increased scope and acceptance of norms prohibiting physical harmnorms prohibiting physical harm – are the products of kludgy design

93


Rather, I suggest, the right conclusion to draw is that ALL moral intuitionsALL moral intuitions should be viewed with a healthy dose of skepticismskepticism

The mechanisms that give rise to them may not have may not have been well designed to do anythingbeen well designed to do anything

So we should be skeptical about moral intuitionsskeptical about moral intuitions for roughly the same reason that we should be skeptical skeptical of the output of a kludgy piece of computer of the output of a kludgy piece of computer softwaresoftware

94


Compare and ContrastCompare and Contrast

The friends of intuitionThe friends of intuition (e.g. moral sense theorists) think the system producing them is well designed for morally well designed for morally admirable goalsadmirable goals

though it can sometimes misfireit can sometimes misfire when conditions are unfavorable

Previous enemies of intuitionPrevious enemies of intuition (e.g. Singer) think the system producing them has been well designedwell designed for morally morally problematic goalsproblematic goals

We believe that the system producing them is a kludgekludge – much of it has not been well designed at all!has not been well designed at all!

95


But if we should be skeptical about all intuition, how can we go about making how can we go about making moral decisions?moral decisions?

That’s a BIG questionBIG question & a HARD oneHARD one. Perhaps I’ll be able to suggest an answer …

96


…the next time I come to Paris

Documents

1 Moral Theory Meets Cognitive Science How the Cognitive Sciences Can Transform Traditional Debates Stephen Stich Dept. of Philosophy & Center for Cognitive