1

CAUSALMECHANISMS:APOTENTIALTOOLFORECONOMICPOLICY?

ACASESTUDYAPPROACH

ResearchMasterThesis

FelixJoachimMatthiasdenOttolander

(342598)

Supervisor:Prof.Dr.J.J.Vromen

Advisor:Dr.J.W.Hengstmengel

Dateofcompletion:01.07.2016

NumberofECTS:30

Wordcount:29.235

MA,ResearchMasterPhilosophyandEconomics

ErasmusInstituteforPhilosophyandEconomics(EIPE)

FacultyofPhilosophy

ErasmusUniversityRotterdam

2

Arunonthebank(Berlin,13July1931).

3

PREFACE

“Thecurioustaskofeconomicsistodemonstratetomenhowlittletheyreallyknowaboutwhattheyimaginetheycandesign.”

F.A.vonHayek,TheFatalConceit:TheErrorsofSocialism(1988)

Bankrunsareaneconomist’sworstnightmare.Liquiditydriesup,banks–literally–closetheirdoors,theeconomystagnates,unemploymentrises,publicorderdeteriorates,andpoliticiansareoustedfromoffice.Allthissoundspainfullyfamiliar.Thepastdecadehaswitnessedthebiggestbankruninmodernhistory,aswellasseveralnear-bankruns,withdevastatingeffects.Despitethe fact that the general causes arewell understood, economistshave ahard timepreventinghistorytorepeatitself.Yes,theyhavetriedtominimizethecostsfortaxpayersbymovingfrombailingoutbankstobailinginshareholders.Moreover,therehasbeenarecenteffortinsettingupabankingunioninEuropewiththeaimoferadicatingbankrunsonceandforall.

Althoughwemayunderstandhowbankrunsevolve,itremainsverydifficulttoeffectivelycontrolthefactorsthatplayaroleinthisprocess.Causalmechanisms,showingtheentitiesandprocesses that lead up to certain events, have proven useful for economists, sociologists andhistoriansintheexplanationofphenomena.Thisthesis,asthetitlesuggests,askswhethercausalmechanismscanalsobeusefulwithinthedomainofeconomicpolicy.Sincetheconnotationof‘mechanism’ ineconomicsdifferssignificantly fromitsphilosophicalconceptualisation,agreatdealofattentionwillbedevotedtobridgethisgap.More importantly, the invocationofcausalmechanismsforpolicypurposesraisesseveralmethodologicalissues,towhichthisthesisaimstocontribute.Aswithasizableprojectlikethis,somewordsofgratitudeareinorder.Firstofall,IwouldliketosincerelythankJackVromen,mysupervisor,fortakingthetimetoprovidesharpandinsightfulcommentsonearlierdrafts.ThanksalsotomyadvisorJoostHengstmengelwhomanagedtoreadandcommentonmythesisinjustoneday,whichsignificantlyimproveditsstructure,clarityandreadability.Thoughonlyinvolvedintheverybeginningofthesupervisionprocess,AttiliaRuzzene,myex-supervisorandpersonal tutor,hasbeenagreatsourceof inspiration. In thesamevein,manythankstoPhilippeVerreault-JulienandJulianReisswhoweresogeneroustohavealookatmy initial thesis proposal. I would also like to thank the organizers of the OZSW GraduateConferenceinTheoreticalPhilosophy(April,2016),inparticularKorayKaraca,whograntedtheopportunitytopresentmythesisproposalandprovidedusefulcommentsaswell. Onadifferentnote,thankstomygoodfriendandfellowcampustrotterLucvandeVenwithwhomIenjoyednumerouslunchesanddinnersthepastfewyears.Lastly,aspecialshout-outtoKeesKraaijeveldandMarlotvanderStoel–myroommateandgirlfriend,respectively–who,asnon-philosophers,hadtoenduretheinevitableconsequencesofsomeonedoingatwo-yearMaster’sprogrammeinanalyticalphilosophy.

4

TABLEOFCONTENTS

PREFACE 3

INTRODUCTION 5

A. GETTINGSTARTED:THESELF-FULFILLINGPROPHECY 5B. MAINARGUMENTS 8C. OUTLOOKONCASESTUDIES 9

CHAPTERONE:THEMEANINGANDFUNCTIONOFMECHANISMS 10

1.1. COLEMAN’SBOAT 111.2. ALTERNATIVEFUNCTIONSOFMECHANISMS 141.2.1. PREDICTION 151.2.2. CONTROL 181.2.3. MECHANISMSANDCONTROL 201.3. THEPROBLEMOFEXTERNALVALIDITY 211.3.1. MECHANISM-BASEDEXTRAPOLATION 221.3.2. STRUCTURE-ALTERINGINTERVENTIONS 241.4. ALTERNATIVEPERSPECTIVE:EVIDENTIALRELEVANCE 261.4.1. FFROMEFFICACYTOEFFECTIVENESS 271.4.2. CAUSALSCENARIOS 291.4.3. ANOTEONINTERPRETINGMECHANISMS 31

CHAPTERTWO:MECHANISMSANDAUCTIONPOLICY 32

2.1. THEFCCAUCTIONS 322.2. THENEEDFORAFULLERPICTURE 342.3. EXTERNALVALIDITY 372.3.1. MECHANISMDESIGN 382.3.2. MECHANISMDESIGNANDSTRUCTURE-ALTERINGINTERVENTIONS 402.4. EVIDENTIALRELEVANCE 432.4.1. AFOCUSONRELEVANCE 432.4.2. CAUSALSCENARIOS 442.4.3. CAUSALSCENARIOSANDMODULARITY 462.4.4. IT’SALLABOUTSTABILITY 48

CHAPTERTHREE:MECHANISMSANDBEHAVIOURALPOLICY 51

3.1. SAVEMORETOMORROWTM 513.2. EXTERNALVALIDITY 543.2.1. BACKGROUNDCONDITIONS 543.2.2. MODULARITY:BITINGTHEBULLET? 573.3. EVIDENTIALRELEVANCE 603.3.1. WELFARE 603.3.2. CAUSALSCENARIOSRECONSIDERED 623.3.3. JUSTIFYINGINTERVENTIONS 64

CONCLUSION 66

BIBLIOGRAPHY 71

5

INTRODUCTION

TheeconomicandfinancialturmoilinGreecehasbeenunfoldingforoversixyearsnow.Despite

the relatively calmness of the past year, Greece’s systemic problemsof low economic growth,

risingpublicdebt andaweakbanking sectorhave recently resurfaced. Financeministers and

otherpolicymakersfromtheEurozonearecurrentlyreviewingtheprogressmadebyGreecewith

respect to its third bailout programme, which was agreed upon in July 2015 after weeks of

uncertaintyandspeculation.Asimilarscenarionowseemstohavebeensetinmotion,formostof

theelementsthatplayedaroleinGreece’s“hotsummer”1haveunfortunatelyremainedinplace.

Clearly, thisposesasevere threat forEuropeanpolicymakers.Theirprimaryaim is to

bring back economic and financial stability to Greece: sustainable public finances, renewed

growthinGrossDomesticProduct(GDP)andawell-functioningbankingsector.Mostimportant

ofall,policymakerswanttopreventGreekbanksfromcollapsingduetouncertaintyabouttheir

abilitytomeetdepositors’shorttermdemands.Overthepastsixyears,therehavebeennumerous

institutionsandevents involved inthebuild-upofGreece’sproblems.Akeysetofevents took

placeattheendofJuneandbeginningofJuly2015,whenGreeceexperiencedarunonsomeofits

largestbanks.IfpolicymakerswanttosucceedintheirattempttokeepGreekbanksoperational

intimesofcrisis,thenitisimperativetheyunderstandthecausalmechanismofabankrun.

A. GETTINGSTARTED:THESELF-FULFILLINGPROPHECY

The phenomenon of a bank run has been thoroughly studied by both economists and social

scientists.Bankrunsarecommonlyunderstoodtooccurwheninitialbeliefsabouttheinsolvency

ofaparticularbankorgroupofbanksultimatelyleadtoalargeamountofpanickeddepositors

tryingtotakeuptheirfunds.Thefactthatsomedepositorsaremakingsubstantialwithdrawals

promptotherstodosoaswelloutoffearthattheymightbeleftempty-handedwhenthebank

eventuallyrunsofoutofmoney.Crucially,thefinancialpositionofthebankisfurtherweakened

bythecontinuousflowofwithdrawals–whethertheinitialworrieswerejustifiedornot.This

downwardspiralculminatesinaliteral‘runonthebank’,whereanxiousdepositorsqueueinfront

ofbankofficesandATMstryingtosalvagewhattheycan(seepage2).

RobertMerton(1948)hasbeenoneofthefirstsociologiststoformulateanexplanation

fortheexistence(andpersistence)ofbankruns.Accordingtohisinfluentialaccount,abankrun

1SilviaMerler(2015),anAffiliateFellowattheBruegelthink-tankinBrussels,wasoneofthefirstwhohadanticipatedfurtherfinancialproblemsinGreecelastsummer.

6

isessentiallyanexpressionofaself-fulfillingprophecy.Bydrawingonabasictheoreminsocial

science – “Ifmendefine situations as real, they are real in their consequences.” (ibid.: 193) –

Mertonclaimsthatpeopledonotonlyrespondtotheobjectivefeaturesofagivensituation,but

also to the subjective meaning they ascribe to that situation. That is, depositors might be

prompted to believe that a particular bank has serious solvency issues due to some general

rumourorpreviouswithdrawalsbyothers.Whatmattersforthemisnotwhethertherumouris

actuallytrue,butwhatthepossibleconsequencescouldbeincasepeoplebelievetherumourto

betrue.

Infact,aself-fulfillingprophecyisalwaysfalseinitially,inthesensethattheanticipated

consequenceswouldnotcomeaboutifpeopledidnotactonthebasisoftheirbeliefs.Inother

words,theinitialfalsebeliefsconcerningasituationtriggeractionsbypeoplethatinturnmake

those beliefs come true. This shows us that certain false beliefs – in the form of rumours,

speculationsorwhathaveyou– can “becomean integralpart of the situationand thus affect

subsequentdevelopments”(ibid.:195).Sinceaself-fulfillingprophecyofteninvolvesarangeof

differentcausalvariableswithinatemporalstructure,ishasbeendescribedasapropercausal

mechanism.

The characterisation of the self-fulfilling prophecy as a causalmechanism has become

evidentinGreeceoverthecourseoflastsummer.ThestorystartedwithAlexisTsipras,Greece’s

primeminster,announcingareferendumtobeheldonJuly5aboutanadditionalbailoutproposal

by theTroika2. This announcementwas followedby thedecisionof theTroika to suspend its

negotiationswiththeGreekgovernmentandlettheexistingbailoutprogrammeexpire.Moreover,

theECBdecidednottoincreaseitsemergencyfundingtotheGreekbankingsector.Duetosevere

uncertaintyaboutthefutureoftheeuroasaviablecurrencyforGreece,depositorsincreasedtheir

withdrawals and began to hoard large amounts of cash. 3 In order to avoid widespread

bankruptcies,theECBinitiatedacoupleoflast-resortmeasures:capitalcontrols,anationalbank

holidayandatemporaryclosureofGreekstockmarkets.Thebankrunfinallysubsidedwhenthe

GreekgovernmentandtheTroikaagreeduponathirdbailoutprogrammeonJuly12.

Now that another hot summer is looming forGreece, the causalmechanismof a self-fulfilling

prophecyisbecomingrelevantagainforpolicymakers.Assumingtheydonotwanttorepeatlast

year’sscenario,policymakersareconcernedwiththequestionwhenandwheretointerveneina

potentiallysimilarchainofevents.Incaseofabankrun,itisimportantforpolicymakerstoknow

atwhatstagetheycanstopthedestructiveprocessfromunfoldingfurther.Forinstance,given

2Thistermreferstothethreemaincreditorinstitutions–theEuropeanCommission(EC),theEuropeanCentralBank(ECB)andtheInternationalMonetaryFund(IMF)–thatjointlyconductthenegotiationswithGreecewithregardtoitsconditionalbailoutpayments.3FormoredetailsaboutthedepositoutflowsfromGreekbanks,seeHarari(2015:17).

7

thatthedecisionbytheECBnottoincreaseitsemergencyfundingtotheGreekbankingsector

contributed to increased uncertainty and withdrawals, the ECB could decide to increase its

emergencyfundingwhenitfindsitselfinasimilarsituationinthefuture.

More importantly, policy makers want to know how to intervene on a particular

developmenteffectively.Knowingatwhatstageofabankruntointerveneisoftennotenough;in

addition, policymakers need to have evidence aboutwhat kind of intervention is likely to be

effective. These kinds of considerations are often context-dependent, which means that the

effectivenessofpotentialinterventionslargelydependsonthespecificconditionsofthesituation

athand.Whenconsideringtheoptiontoincreaseemergencyfundingincaseofstagnatingbailout

negotiations,policymakersof theECBwillhave toassesswhether thisoptionwill lead to the

desiredoutcome–i.e.stoppingtheformationofabankrun.Thisparticularinterventionmaybe

effectivefortheaimofstoppingabankrun,butitmightbelesseffectivefordifferentpurposes.

Indeed,thedecisionoftheECBnottoincreaseitsemergencyfundingwasaimedatforcingthe

Greekgovernmenttorejointhebailoutnegotiations.

This thesis aims to defend the claim thatmechanisms4are a useful tool for economic policy-

making.Theexampleofthe2015bankruninGreeceillustratesinwhatsensemechanismscould

be useful for economic policy makers. However, the concept of mechanisms has received

relatively little attention within the realm of economic policy. Instead, most philosophers of

science and social scientists have focused on whether mechanisms can contribute to the

explanation of social and economic phenomena. They have been mainly concerned with the

explanatorypowerofmechanisms,whichhasresultedinaviewofmechanismsasanexplicittool

forcausalinquiry.5

Although explanation of economic phenomena is an important aim for the academic

community of economists, philosophers of science and social scientists in general, it is less

relevantforpolicymakers.Withrespecttotheearlierbankrunexample,policymakerscanclearly

benefit from the causal knowledge that mechanisms are able to provide: the self-fulfilling

prophecytellsthemthatqueuingdepositorsarelikelytoactonthebasisofsomeinitialfalsebelief

aboutbankinsolvency.Yetwhatpolicymakerscareaboutevenmoreiswhethertheycancontrol

a certain situationwith the tools at their disposal. Prima facie, mechanisms such as the self-

fulfillingprophecyappear tobe relevant forpolicymakers in this sensebecause theyprovide

4Fromthispointonwards,Iwillreferto‘mechanisms’withoutmentioningthe‘causal’componentexplicitly.Aswillbeexplained inchapterone, thereasonis that the interpretationofmechanismswithrespecttoexplanationislessimportantforthepurposesofpolicymakers,whicharemoreconcernedwithcontrollingeconomicphenomenainstead.5PeterHedströmandPetri Ylikoski (2010) review someof themost important contributions to causalmechanismsinthesocialsciences.Thisliteraturewillbeelaborateduponintheearlysectionsofchapterone.

8

themwith knowledge about how to intervene. If, for instance, policymakers could somehow

preventrumoursaboutbankinsolvencyfromspreading,thentheformationofabankrunmight

beeffectivelyavoided.

B. MAINARGUMENTS

Theconceptualisationofmechanismswithrespecttopolicy-makingwillbethecentralthemeof

thefirstchapter.Bydrawingonexamplesfromsociologyandeconomics,itwillbecomeclearwhat

mechanismsgenerally look like, andwhat rolemechanismsplay in thedifferentdesiderataof

science:explanation,predictionandcontrol.Whileexplanationandpredictionareimportantaims

foreconomicsingeneral,theyarelessrelevantfromapolicypointofview.

After sorting out the different functions of mechanisms, the two main arguments in

defence of using mechanisms for policy purposes will be introduced. The first refers to the

methodological problem of external validity, which reflects the difficulty of exporting causal

relationshipsoutsidetheirartificialenvironments.Accordingtosomeaccounts6,mechanismsare

able to resolve the problem of external validity by specifying the similarity in background

conditionsbetweentheartificialandtargetenvironments.Thesebackgroundconditionsarethen

incorporatedintotheinterventionsofpolicymakerssoastomakethemmoreeffective.Theclaim

thatmechanismsareactuallyabletosupporttheeffectivenessofinterventions–alsoreferredto

asmechanism-basedextrapolation–is,however,rathercontroversial:interventionsarelikelyto

alterthecausalstructuretheywishtoexploit.HereIwillmaketwoclaims:one,mechanismsplay

animportantroleinthedesignandimplementationofpolicyinterventions;second,mechanism

design, though susceptible to structure-altering interventions, can be a suitable extension of

mechanism-based extrapolation, and should thus be taken seriously by the economic policy-

makingcommunity.

Thesecondargumentrelatestoevidentialrelevance.Fromthisalternativeperspective,

mechanismshavetheabilitytoprovideapreliminaryunderstandingoftheevidencethatcould

berelevantfortheeffectivenessofpolicyinterventions.Heretheinterpretationofmechanismsas

causal scenarios will be introduced, where each causal scenario starts with the proposed

interventionandendswiththedesiredoutcome,andisconsideredplausibleaccordingtosome

basictheory,generalprinciple,orwidely-heldpublicopinion.Specifically,Iwillmaketwoclaims

withrespecttomechanismsandevidentialrelevance:one,policymakersadopttheperspective

of evidential relevance since they take the aims of interventions as their point of departure;

second,theyproceedinestablishingevidentialrelevancebyconstructingandevaluatingdifferent

6MostnotablythatofFrancescoGuala(2005,2011)andDanielSteel(2008).

9

causal scenarios. The scenarios themselves do not act as evidence for the effectiveness of

interventions;rather,theyareatoolforpolicymakerstobeingtheirsearchforrelevantevidence

inapreliminarymanner.

C. OUTLOOKONCASESTUDIES

Asmentionedalready,towhatextentmechanismscanactuallybeusefulforpolicymakersdiffers

percase.Forthisreason,thepotentialofmechanismsforpolicyinterventionswillbeassessed

withthehelpoftwocasestudies.Chaptertwodealswithmechanismsinthedomainofauction

policy.HereIwilldiscussoneprominentcaseinwhichgametheoryhasbeensuccessfullyapplied

to the design of real auctions, namely the 1994 Federal Communications Commission (FCC)

spectrumauctionsintheUnitedStates.Thechoiceforthisparticularcasestudyrestsonthefact

thattheauctionsinvolvedamultibillion-dollarbusiness,whichnotonlyhadasignificantimpact

onpublicfinancesbutalsoaffectedtheeconomicperformanceoftheClintonadministration.More

importantly, the FCC auctions can be characterised as a kind of economic engineering where

knowledgeofmechanismswasusedtodesignanewmechanismthatworkedwellintheactual

contextofthespectrumauctions.Inaddition,severalphilosophersofscience7haveextensively

engagedinthedebatearoundauctiondesignanditspolicyimplicationsoverthepastfewdecades,

whichprovidesamplematerialtoadvanceourassessmentofmechanismsforpolicy-making.

Chapterthreeisconcernedwithmechanismsinthedomainofbehaviouralpolicy.Here

thecasestudyconsistsoftheSaveMoreTomorrowTM(SMT)pensionplanasdescribedbyRichard

Thaler and Shlomo Benartzi (2004, 2013). This particular intervention intended to increase

savings contribution rates amongst employees by drawing on insights from behavioural

economics.AlthoughtheSMTplanwassatisfactoryintermsofraisingcontributionrates,itwas

not clearhow the intervention actuallymade a difference – i.e. throughwhatmechanism the

resultswereobtained.Sincepoliciesthatusebehaviouralinsights–alsoreferredtoas‘nudging’8

–arequicklygainingpopularity,investigatingtheroleofmechanismswillprovehelpfulinthis

regard.AccordingtoTillGrüne-Yanoff(2015),behaviouralpolicy(astheSMTpensionplan)needs

mechanisticevidencebecause,withoutspecifyingtheoperatingmechanism,policymakerscannot

sufficientlyjustifytheirinterventions.Thequestionthenis,ofcourse,whichmechanismwillcount

assufficienttothejustificationofinterventions.Oneoptionwouldbeforpolicymakerstojustify

interventionsaccordingtothewelfareeffectsinferredbymechanisms.

7Mostnotably,FrancescoGuala (2005,2011),AnnaAlexandrova(2006),AnnaAlexandrovaandRobertNorthcott(2009),andAlvinRoth(2002).8Foraseminalcontributiontotheconceptofnudging,seeThalerandSunstein(2008).

10

CHAPTERONE

THEMEANINGANDFUNCTIONOFMECHANISMS

Overthepastfewdecades,philosophersofscienceandsocialscientistshavedebatedaboutthe

roleofmechanisms9asamethodofinquiry.Manyearlycontributionshavefocusedondelineating

a proper definition of amechanism,which has resulted in an extensive and rather confusing

collectionofdefinitions.For instance,HedströmandYlikoski (2010:51) listno less thannine

differentdefinitionsofamechanismputforwardinthesocialsciences.10Mechanisms,itseems,

comeinavarietyofshapesandsizes.

Tomakesomesenseofthisconceptualmess,themanydefinitionsofamechanismcanbe

separated into two groups: horizontal mechanisms and vertical mechanisms. The horizontal

interpretation of mechanisms describes the intermediate causal processes that take place

betweena certain cause andeffect.Oneof themostprominent and relatively straightforward

horizontaldefinitionshasbeenput forwardbyDaniel Little,whodescribes amechanismas a

“series of events governed by law-like regularities that lead from the explanans to the

explanandum”(1991:56).Accordingtohisaccount,acausalanalysisofacertainphenomenonis

constitutedbyamechanismthatidentifiesall(probabilistic)conditionstobecausallyrelevantto

theoccurrenceofaneffect.Inthissense,amechanismpresentsthepathwaythroughwhichaset

of different causes consecutively lead to the observed effect. An example of a horizontal

mechanismispresentedinfigure1.

The vertical interpretation of mechanisms, on the other hand, is often more complex

because italsoexplicitlydealswithcausalprocessesthatoperateondifferent levelsofreality.

Besides listing the causally relevant events in the order they supposedly occur, a vertical

mechanismalsodescribeshowphenomenacanbeexplainedby,sotosay,zoominginorouton

certain causal processes. This reductionist approach is supposed to explain the occurrence of

phenomena more accurately because it shows how micro-level variables are responsible for

macro-levelvariables.Whataverticalmechanismlooks likeandhowthisexplanatoryprocess

goesaboutwillbediscussedinthenextsection.

9Throughoutthischapterandthefollowingchapters,Iusetheterms‘mechanisms’and‘mechanisticmodels’interchangeably.Mechanismsare,essentially,atypeofmodelthatusesassumptionstodescribe,explain,predictandcontrolphenomena.10Someof these definitions specifically refer to ‘socialmechanisms’ (Hedström& Swedberg, 1998), i.e.mechanismsdealingwiththeexplanationofsocialphenomena,suchasrevolutions,wars,etc.Sincethisthesisislessconcernedwithexplanationandfocusesmoreoneconomicphenomena,Iwillnotelaborateonthisparticularterminology.

11

Fornow,itisimportanttoemphasizethatbothgroupsofmechanismsarerelevanttothe

topicofthisthesis.Inwhatfollows,Iwillarguethatpolicymakerscanusemechanisms–whether

they are horizontal or vertical, or some combination of the two – to better control economic

phenomena.Whatmattershereisnotwhetheronegroupofmechanismsismoreorlessuseful

than the other in terms of explanation, but ifmechanisms in general can be useful for policy

purposes.Inthissense,Idonotwishtoendorseoneexplicitdefinition(orgroupofdefinitions)

ofmechanisms,fortheaimofthisthesisisnottoengageinaconceptualdiscussionassummarized

byHedströmandYlikoski(2010).Rather,itsapproachismethodological:towhatextent,ifatall,

canmechanismsbeusefulwithintheeconomicpolicy-makingprocess?

1.1. COLEMAN’SBOAT

Toillustratethegeneralstructureandoperationalisationofmechanisms,itisusefultointroduce

aclassicexamplebythefamoussociologistMaxWeber.Inhisprincipalthesis,TheProtestantEthic

and the Spirit of Capitalism (1905/1930), Weber claims that the religious doctrine of

Protestantismischieflyresponsiblefortheriseofourmoderncapitalisteconomicsystem.The

causalrelationshipisinthiscaseofamacro-to-macrotype:onemacrovariable(Protestantism)

causes another macro variable (capitalism) to occur. Figure 2 shows this particular causal

relationshipinasimplemechanisticmodel.

Presentedinthisway,amechanismdoesnotseemtobeanydifferentfromthecovering-

lawmodelofexplanationadvocatedbyCarlHempel(1942/2011).Inhisaccount,toexplainthe

occurrenceofaphenomenonentailsreferringtoageneralcausallaw.Foranexplanationtobe

satisfactory,itmustspecifyboththegenerallawandtheconditionsthatmakethelawapplicable

Figure1:Ahorizontalmechanism(Little,1995).

12

in the particular case. With respect to Weber’s thesis, the occurrence of capitalism could be

explainedbythegenerallawthatpurportedlyexistsbetweenreligionandeconomicsystems.

YetHempeladmitsthattheexistenceofdeterministiclawsishighlyunlikelyinthesocial

sciences. Instead,whatatmostcanbe invokedare lawsofaprobabilisticnature, i.e. lawsthat

state the probability with which a particular phenomenon will come about given certain

backgroundconditions.Oftenthereismerelyastatisticalassociationbetweentwophenomenaof

interest,inwhichcase“thespecificexplanationwilloffernomoreinsightsthanthelawitselfand

willusuallyonlysuggestthatarelationshipislikelytoexist,butitwillgivenoclueastowhythis

is likely to be the case” (Hedström & Swedberg, 1998: 8, original italics). For this reason,

mechanismsneedtogobeyondthemacro-to-macrotypeofcausalexplanation.

Oneviableextensionisformechanismstoincludecausalrelationshipsonthemicrolevel

aswell.Thisapproachisbroadlydefinedasmethodologicalindividualism,whichaimstoexplain

macro-levelphenomenaintermsofmicro-levelentitiessuchasindividualbehaviour.Thecausal

relationshipisnowofamacro-micro-macrotype:amacrovariable(Protestantism)isreducedto

amicrovariable(individualvalues),thatcausesanothermicrovariable(orientationstoeconomic

behaviour), which in turn is transformed back into a macro variable (capitalism). Figure 3

capturesthisextendedmechanisticmodelofWeber’sthesis.

Figure2:Macro-to-macrocausalrelationshipbetweenProtestantismandcapitalism.

Figure3:Macro-micro-macrocausalrelationshipbetweenProtestantismandcapitalism.ThisgeneralcausalstructureisreferredtoasColeman’sBoat,afteritsoriginatorJamesColeman(1986).

13

Thistypeofexplanation,basedonmethodologicalindividualism11,hasbeenproposedby

the sociologist JamesColeman (1986)andhas since thenaptlybeencalled ‘Coleman’sBoat’.12

Althoughtherearearguablymanydifferenttheoreticalmechanismsavailableforthepurposeof

explainingsocialandeconomicphenomena,Coleman’sBoatisagoodplacetostartourdiscussion

ofmechanismsingeneral.Beforetheanalysisturnstothepotentialofmechanismswithrespect

topolicy-making,itisimportanttopayabitmoreattentiontohowmechanismscanbeusedasan

explanatorytool.Thereasonisthatmostmechanisticaccountsdevelopedinthesocialsciences

sofarhaveaimedatacquiringabetterexplanationofthephenomenaunderinvestigation.Ifpolicy

makerswanttogainageneralunderstandingofwhatmechanismsareandwhattheycandofor

them,thentheyatleasthavetoseeoneofthefunctionsofmechanisms–i.e.thedesideratumof

explanation–inaction.

AccordingtoHedström&Swedberg(2010),theideaofmechanisticexplanationshaspartlyarisen

due to the shortcomings of Hempel’s covering-law model. Granting this is true, how is a

mechanism like Coleman’s Boat able to determine why it is likely that a causal relationship

betweentwoormorevariablesexists?Essentially,amechanismprovidesanexplanationofhow

acertaineffectcouldhavecomeabout.Thisimpliesthattheeffectinquestioncanbeproducedby

a number of different mechanisms, known or unknown to the investigator. To increase the

plausibility of one mechanism over another, empirical evidence about the assumed entities,

activities and relationships must be collected and systematically analysed. Thus, mechanistic

explanationstrytodescribecausalprocessesasaccuratelyaspossible,whilebeingselectiveabout

thedifferentaspectsofthoseprocessesandbydisregardingirrelevantdetails.

This is precisely what Coleman tries to achieve in his analysis of Weber’s thesis. By

selectingtwocausallyrelevantvariablesonamicrolevel–individualvaluesandorientationsto

economic behaviour – the causal relationship between Protestantism and capitalism is

presumablyexplainedmoreaccurately.Inshort,Protestantreligiousdoctrineplacesthevalues

ofworldly possessions and soberness upon individuals in society; these values then lead to a

positiveattitudetowardsprivateenterprisesandhardwork;whicheventuallymanifestsitselfin

a capitalistmodeof production and consumptionwithin society. So there are threeprocesses

activeinthisparticularmechanism:macro-to-micro(arrow2infigure3),micro-to-micro(arrow

1)andmicro-to-macro(arrow3).

11 Although Daniel Little thinks that Coleman’s Boat presupposes some form of methodologicalindividualism,PetriYlikoski(forthcoming)arguesthisviewismistaken.Anyway,thispointisnotrelevanttoourdiscussionsinceColeman’sBoatismerelyusedasanillustrationofwhatmechanismsineconomicsmaylooklikeandhowtheygenerallyoperate.12Coleman’sBoat,alsoreferredtoastheColeman’sdiagramandColeman’sbathtub,hasbecomeoneofthemostfamoustheoreticalmechanismsinsociology.Afterintroducingitinhis1986paper,ColemanuseditextensivelyinhismagnumopusFoundationsofSocialTheory(1990).

14

There are, of course, many other macro-level variables besides Protestant religious

doctrinethatcould,intheory,explaintheoccurrenceofacapitalisteconomicsystem,suchasan

abundanceofnaturalresourcesandlabour.13Nevertheless,theinclusionofmicro-levelvariables

givesstrengthtothecausalclaimbecauseitelucidateshowacertaincauseleadstotheobserved

effect.Anotherallegedadvantageofmechanisticexplanationsisthefactthatitsassumptionscan

beempiricallytested.Themicro-to-microcausalrelationshipthatindividualreligiousvalueslead

to orientations to capitalist behaviour, for instance, can be validated by sociological or

anthropological research that studies the professions and general economic behaviour of

Protestantcommunities.Sincearangeofpsychologicalvariablesoperateatthemicrolevel,more

precisemethodological tools – such as experiments –will be available in order to test causal

hypotheses.Thisisnottosaythatstudyingthesekindsofmicro-levelhypotheseswillbeeasyand

willalwaysbringaboutsatisfactoryresults,buttheyaregenerallyconsideredtobemorereliable

thanmeremacro-levelexplanations.

Although includingmicro-level explanationsmight be a reliablemethod in supporting

macro-levelexplanations,theyalsointroducenewproblems.Onekeyproblemisthatoncemicro-

levelexplanationsareestablished,itisdifficulttotransformtheireffectsbacktothemacrolevel.

In other words, the micro-to-macro process of a mechanism (arrow 3) often proves to be

insufficient in explaining how some individual causal relationship results in a collective

phenomenon.This isalsothecasewithWeber’saccount, inwhichhefailstoshow“howthese

individualorientationscombinedtoproducethestructureofeconomicorganisationthatwecall

capitalism(ifinfacttheydidincombinationproducethiseffect)”(Coleman,1986:1323).Thus,

whethertoincludemicrovariableswhenexplainingmacro-levelphenomenanotonlydependson

thestrengthofthemicro-levelexplanationitself,butalsoontheabilityoftransformingthelatter

backintothemacrophenomenonunderinvestigation.

1.2. ALTERNATIVEFUNCTIONSOFMECHANISMS

Sincetheintroductionofmechanismsinthesocialsciences,includingeconomics,theiraimhas

mainlybeen toenhance theexplanationofphenomena.Forexample, themechanismofaself-

fulfilling prophecy has helped to explain how bank runs develop: false beliefs about bank

insolvency trigger withdrawals by depositors, which eventually justify those initial beliefs.

13Forexample,theriseofChinaasaneconomicpowercouldbeseenasacounterexampletoWeber’sthesisbecauseithasdevelopedalargelycapitalisteconomicsystemdespitethealmostcompletelackofProtestantreligiousdoctrine,oranyotherreligiousdoctrineforthatmatter.AmoreplausiblemechanisticexplanationforthedevelopmentofChinesecapitalismwouldfeature,amongothervariables,anabundantsupplyofnaturalresourcesandcheaplabour.

15

Likewise,thecausalstructureofColeman’sBoathasincreasedourunderstandingofthecausal

relationship between religion and capitalism by incorporating macro- and micro levels of

explanation.

Despitethefactthatexplanationisanimportantaim–orfunction,asIwillrefertoit–of

mechanisms,itisnottheonlyonethatisrelevanttotheuseofmechanisms.Infact,therearetwo

otherfunctionsthataredistinctfrom,andatleastasequallyimportantas,explanation.Thesenon-

explanatoryfunctionsneedtobedistinguishedinordertoassesstheusefulnessofmechanisms

forpolicymakers. In thissection,someof theconceptualdifferencesbetweenthe functionsof

explanation,predictionandcontrolwithrespecttomechanismswillbediscussed.Mostattention

willbedevotedtothefunctionofcontrolbecausepolicymakers,asillustratedintheintroduction,

aremostlyinterestedinhowtheycaneffectivelyinterveneuponeconomicphenomena.14

1.2.1. PREDICTION

The first alternative function of mechanisms that will be discussed is prediction. Within

economics, theaimofpredicting the futurevaluesof certainvariables isof great significance:

correctly predicting next month’s stock market index hugely benefits traders and investors;

entrepreneurswillonlystayinbusinessiftheycanaccuratelypredictfutureconsumerdemand

fortheirproductsandservices;duringelectionperiods,politiciansheavilyrelyonGDPforecasts

thatshowhowtheirpolicyproposalswill impacttheeconomy.Admittedly,correctpredictions

arenotoriouslydifficulttocomebyineconomics,butitisclearlyworthwhileforeconomistsand

policymakerstopursuethisaimnonetheless.

Fromamethodologicalpointofview,predictionhasbeenthedominantdesideratumof

economic theory since at least the latter half of the 20th century. In his seminal paper The

MethodologyofPositiveEconomics(1953/2008),MiltonFriedmanarguesthatthe“ultimategoal

of a positive science is the development of a ‘theory’ or ‘hypothesis’ that yields valid and

meaningful(i.e.nottruistic)predictionsaboutphenomenanotyetobserved”(ibid:148).Sowhat

makesaneconomicmodelsuccessfuliswhetheritcanprovideaccuratepredictionsaboutfuture

phenomena,notwhetheritsufficientlyexplainstheoccurrenceofobservedphenomena.Inthis

sense,economistsaremostlyconcernediftheirmodelscanpredictsomephenomenonofinterest,

whileremainingagnosticabouthowtheirmodelsgenerallydoso.

14Besidestheeffectivenessofinterventions,policymakersalsocareaboutavarietyofotherconsiderations,suchascostsandwhetherinterventionsaremorallyacceptable.Thisthesisfocusesonthemethodologicalissues related to the function of control and therefore remains agnostic about these other kinds ofconsiderationsthatpolicymakersmighthave.

16

This implies that theassumptions incorporated ineconomicmodelsdonotnecessarily

havetoberealisticinordertoobtainaccuratepredictions.Infact,

“Truly important and significant hypotheseswill be found to have ‘assumptions’ that are

wildlyinaccuratedescriptiverepresentationsofreality,and,ingeneral,themoresignificant

thetheory,themoreunrealistictheassumptions.…Tobeimportant,therefore,ahypothesis

mustbedescriptivelyfalseinitsassumptions;ittakesaccountof,andaccountsfor,noneof

themanyotherattendantcircumstances,sinceitsverysuccessshowsthemtobeirrelevant

forthephenomenatobeexplained.”(ibid.:153).

According to Friedman, the question whether the assumptions of a model are realistic is

completelyirrelevant.Instead,whatmattersistowhatextenttheassumptionsresultinaccurate

predictions.Touseasimpleexample:theassumptionofunlimitedhumanrationalityisobviously

unrealistic, but it is allowed to be included in the model because it enables (more) accurate

predictionsofhumanbehaviour.Asaresult,theappropriatenessofatheorycanonlybejudged

accordingtoitspredictivecapabilities.

Theclaimthatassumptionscanbedescriptivelyfalseformodelstopredictwellturnsout

toberatherproblematicinthecaseofmechanisticmodels.Recallthatmechanismsprimarilyaim

to describe how phenomena come about, i.e. through what causal processes. Coleman’s Boat

becameinfluentialpreciselybecauseitcouldshowinwhatwaymicro-levelentitiesdetermined

macro-levelphenomena.Theoftencomplexstructureofmechanisticmodels15isaresultoftheir

aimtospecifythemostsalientcausalvariablesthatleadtosomeparticulareffect.Thepresumed

advantageofmechanismsintermsofexplanatorypowerislargelybasedontheirabilitytoexhibit

anempiricallyverifiablechainofcausalclaims.

Now if we accept Friedman’s position, which a substantial part of the economics

communityhasdone, then themain advantageofmechanismsbecomes largelyundone.More

specifically, by embracing the idea that predictions can be successful when the underlying

assumptions are unrealistic, it makes little sense to empirically test whether the causal

relationshipspostulatedbymechanismsareactuallyrealistic.Infact,inquiringaboutmechanisms

wouldnotbeusefulatallbecausewhatmattersisthatpredictionsareaccurate,notwhytheyare.

Toillustratethispoint,letusextendoursimpleexampleofunlimitedhumanrationality

asanaccuratepredictorofeconomicbehaviour.Besidesperfectrationality,let’sassumethereare

twomoreunrealisticassumptionsthatsupportthesuccessfulpredictionofeconomicbehaviour,

15DanielLittle’s(1995:33)extensionofthecausalmechanismresponsibleforrevolutionisagoodexampleofhowmechanismscanquicklyadoptacomplexstructure,withmanydifferentprobabilisticrelationshipsoperatingatthesametime.

17

namelyperfectinformationandfullyflexiblemarketprices.Onecould,inprinciple,constructand

test a hypothetical mechanism that somehow links the three assumptions to the predicted

economicbehaviour,suchasthreeseparatecausalrelationshipsthatequallyinfluencetheeffect

(seefigure4).

Oronecouldrearrangetheorderoftheassumptionsinthemechanismastoshowthat,

forinstance,perfectinformationcausesperfectrationality,whichtogetherwiththeassumption

of fully flexiblemarket prices predicts economic behaviour. The point is that it is simply not

importanttoknowhowpredictionsactuallycameabout.Rather,itisonlyrelevanttoknowthata

setof (unrealistic)assumptionscontributed to thepredictionofaparticularphenomenon.We

couldtrytoempiricallyverifytheseassumptionsaspostulatedbysomekindofmechanismbut

thiswouldnotbenefitthepredictioninanymeaningfulway.

Thus,itseemsthatmechanismsarenotveryusefulwhenitcomestopredictingeconomic

phenomena.Ifsuccessfulpredictionrequiresunrealisticassumptions,asFriedmansuggests,then

mechanisticmodelsstatingtheorderandinteractionsoftheseassumptionspossesslittleadded

value. Mechanisms might perform better at explaining phenomena, with their potential to

empirically test the assumptions of causal relationships, but it seems this ability cannot be

exploitedforpredictivepurposes.

At this point, however, one could object to this conclusion by stating that explanation and

predictionare the twosidesof thesamecoin.Theso-calledSymmetryThesis (Hempel,1958)

holds that once a phenomenon is adequately explained, its occurrence can also bepotentially

predicted; similarly, the adequate prediction of a phenomenon implies its observation can be

potentiallyexplained.SoHempelbelievedeveryadequateexplanationtobeapotentialprediction,

andviceversa. Inthissense,explanatorymechanismsautomaticallyhavepredictivepowersas

well:assumingColeman’sBoatadequatelyexplainstheriseofcapitalism,itisalsoabletopredict

futurecapitalisteconomicsystemsbasedonthesameanalysis.

Figure4:Simpleexampleofanonsensicalmechanismthatshowshowasetofunrealisticassumptionspredictaphenomenon(economicbehaviour).

18

Unfortunately,theSymmetryThesishasbeenchargedwithnumerouscounter-examples16

andhassubsequentlygoneoutoffashion.Theargumentthatmechanismshavepredictivepower

because theyarecapableofprovidingadequateexplanations issimply false, since thiskindof

reasoning is subject to the same criticism raised against the Symmetry Thesis. The fact that

Coleman’sBoatcanadequatelyexplaintheriseofcapitalismdoesnotmeanitcanatthesametime

predict all instances of capitalist economic systems, as the development of capitalism in

contemporary China nicely illustrates.17Therefore, the objection towardsmechanisms’ lack of

predictivepowerbasedontheSymmetryThesisisunwarranted.

Sofar,twoconclusionswithrespecttothefunctionofpredictionformechanismscanbe

drawn. First, knowledge ofmechanisms does not benefit the prediction of phenomena in any

significantwaybecauseeconomistsandpolicymakersarenotparticularlyinterestedinwhytheir

predictionswillbeaccurate.18Instead,theyareconcernedwhethercertainpredictionswillhold,

while disregarding the possibility that the unrealistic assumptions could be characterized by

some mechanistic model. Secondly, even when mechanistic models adequately explain

phenomenathisdoesnotimplytheyhavepredictivepoweraswell,forthecriticisedSymmetry

Thesisdoesnotholdinthecaseofmechanismseither.

1.2.2. CONTROL

Nowthatthefunctionsofexplanationandpredictionhavebeendiscussedwecanmoveontothe

thirdfunctionofmechanisms,namelycontrol.Thepreviousdiscussionaboutpredictionwillhelp

clarifyhowmechanismsrelatetothecontrolofphenomena.Thisparticularfunctionisthecentral

themebywhichthepotentialofmechanismsforpolicymakerswillbeassessed.Toemphasize,

policymakersaremainlyconcernedwithquestionsabouthowtheycaneffectivelyinterveneupon

causalprocessesinordertobringaboutsomepreferredoutcome.Morespecifically,theywantto

knowhowtomanipulatecausalvariablessothataparticulartargetvariabletakesonaspecific

value.

Thismeansinterventionsbypolicymakersareoftentiedtosomespecificaim,whichis

notonlybeingabletocontrolatargetvariableinitselfbutalsotocontrolitinsomepreferredway.

Forexample,inthecaseofabankrun,policymakersnotonlywanttoknowwhethertheycan

influence theoutflowofbankdeposits.Rather, theywant tobe sure that their intervention in

16See,forinstance,Scriven(1962).17Seefootnote13.18Ofcourse,itwouldbeniceforpolicymakerstoknowabouttheexactcompositionofthecausalvariablesthat have contributed to their correct prediction. This is certainly useful information since the samecompositioncanbeusedinordertoderivesimilarcorrectpredictionsinthefuture.Yetinmostcasesitismoreimportantforpolicymakerstotrusttheirpredictionsintermsofaccuracy.

19

termsofdailylimitsonwithdrawalsindeedreducestheoutflowofcapital.Thus,thefunctionof

controloftentakesonahighlyconcrete,context-specificform.

Explanation obviously plays a role in this process but it must, like prediction, be

distinguishedfromcontrol.Basically,toexplainacertainphenomenonbywayofamechanismis

tolookbackanddeterminewhatcausalvariableswerejointlyresponsiblefortheoccurrenceof

thephenomenon.Oncethisisdone,thepolicymakerhaspresumably,tosomeextent,increased

hiscausalknowledgeofthephenomenonofinterest:thefactthatabankrunissetinmotionby

themechanismofaself-fulfillingprophecyis,inandofitself,veryimportantforpolicymakersto

beawareof.Causalknowledgeofthismechanistictypethenactsasthestartingpointforfurther

inquiriesuponwhereandhowtointerveneuponthecausalprocessesinordertocontroltheir

outcome insomedesirableway.Mechanisticexplanation is thusan importantbutpreliminary

stepintheaimofpolicymakerstocontrolphenomena.

Whereaspredictionisafunctionthatinvolvesatwo-wayrelationshipbetweenasetof

causalvariablesandasetof targetvariables, the functionofcontrol isessentiallya three-way

relationshipbetweenasetof(candidate)causalvariables,asetoffeasibleinterventionsandaset

of targetvariables.Control can thereforebe seenasa specialkindofprediction,wherepolicy

makersdonotpassivelyobservetheoutcomeofsomesetofcausalvariablesbutinadditiontry

toactively–i.e.bysomeinterventionorsequenceofinterventions–bringaboutsomevalueof

thetargetvariable.Whilepredictingthefuturevalueoftargetvariablesishighlysignificantfor

economistsandpolicymakers,beingable tocontrol thevalueof targetvariables isevenmore

significant for quite obvious reasons. With respect to macroeconomics, policy makers are

continuallytryingtocontrolarangeofdifferentvariables,suchasinterestrates,inflationrates

andunemploymentrates.Moreover,ourbankrunexamplehasshownthatcontrollingtheamount

ofdepositwithdrawalsiscrucialformaintainingfinancialstabilitywithintheeconomy.

Thefunctionofcontrolisalsoveryimportantinmicroeconomics,wheretheconsequences

forindividualsandinstitutionsareofprimaryconcern.Herepolicymakersdonotonlywantto

explainorpredictcertainbehaviour,butratherwanttocontrolbehaviouralprocessessoasto

guaranteeadesiredoutcome.Theseoutcomesmaybeexpressed in termsof fair andefficient

behaviour by individuals, or some sufficient amount of government revenue.19 Alternatively,

policymakersmightwishtooptimisepeople’sdecisionswithregardtosavingforretirement.20

Thereareclearlynumerousinstances,inbothmacro-andmicroeconomics,wherepolicymakers

engageinthedesignofinterventionsbywhichtheyaimtobringaboutsomedesirableoutcome.

19Chapter twowilldiscussonespecificcase inwhich theaimofpolicymakerswas todesigna fairandefficientsetofauctionsthatwouldgeneratesufficientgovernmentrevenue.20Likewise,chapterthreewilldiscussonespecificcaseinwhichtheaimofpolicymakerswastoinfluenceemployeestoincreasetheirsavingscontributionrates.

20

1.2.3. MECHANISMSANDCONTROL

Thebasic intuitionbehind the functionofmechanisms for control is that they canhelppolicy

makersidentifywhereandhowtointerveneuponcausalprocessesinordertobringaboutsome

preferredoutcome. Inthis thesis, Iwilldefendtheclaimthatmechanismsareauseful tool for

policy makers by way of two main arguments. The first argument has to do with the

methodologicalproblemofexternalvalidity.Thisproblembasically refers to the fact that it is

oftenverydifficult,ifnotimpossible,toextrapolatecausalrelationshipsobtainedinonecontext

toanother.Mechanismsare–orsoitisclaimedbysomemethodologists–abletoovercomethis

problembecausetheyshowhowthecausalrelationshipsactuallyoperateindifferentcontexts.

Supposedly,themechanisticsolutiontotheproblemofexternalvalidityliesinitsabilitytospecify

thenecessarybackgroundconditionsthatenabletheextrapolationofsomecausalrelationshipto

thetargetenvironment.

Largelybasedontheprincipleofmechanism-basedextrapolation,thereisoneparticularly

interestingapproach:mechanismdesign.Herepolicymakersdonotonlyfocusonextrapolating

aparticulartheoreticalorexperimentalresultbyinvokingmechanisms,butinsteadtrytodesign

a new kind ofmechanism that best fits the conditions of the target environment. During this

procedure,policymakersstudymanydifferentmechanismsandintegratethemostusefulaspects

–beitbitsoftheory,someexperimentalresultorasalientbackgroundcondition–intotheirfinal

design.Thisway,theuseofmechanismsforpolicypurposescanbeseenasakindofeconomic

engineering,inwhichtheneedforextrapolationiscombinedwithmoreinductiveelements.The

nextsectionwillfurtherelaborateontheprocedureofdesigningmechanismsinthisregard.

Thesecondargument ismoregeneralandcomesdowntotheabilityofmechanismsto

provideapreliminaryunderstandingoftheevidencethatcouldberelevantfortheeffectiveness

ofpolicy interventions.Althoughevidenceforefficacy is important, itsrelevanceforthetarget

contextneedstobedeterminedfirst.21Essentially,drawingontheefficacyofcausalvariablesin

some artificial environment is only one piece of evidence for policy hypotheses; what is also

neededisknowledgeoftherelevanceoftheseefficaciesforthepolicy(target)environment.While

atheoristorexperimenterasksinwhatcontexthisobtainedresultsmightberelevant,apolicy

makerismoreinterestedinwhatkindofevidencemightberelevantforhisparticularpolicyaim.

Unfortunately,thelatterperspectivehasbeenrelativelyignoredinthedebateaboutmechanisms

withintherealmofpolicy-making.

Onewaytoaccountforthisperspectiveistointerpretmechanismsascausalscenarios,

whichpolicymakerssetupinordertogatherevidencethatispotentiallyrelevantforthepolicy

21Section1.4willelaborateonthedifferencebetweenevidenceforefficacyandevidenceforeffectiveness,andwhatrolethisdifferenceplaysinestablishingevidentialrelevance.

21

aimathand.Eachscenario–startingwithaproposedinterventionandendingwiththedesired

outcome–hastobeplausibleaccordingtosomebasictheoryorpublicopinion.Tobeclear,the

mechanisms (causal scenarios) themselves do not act as evidence for the effectiveness of

interventions; rather, theyarea tool forpolicymakers tosearch forevidenceandevaluate its

relevance inapreliminarymanner.Once theevidential relevancehasbeenestablished,policy

makerscanthenempiricallytestthecredibilityofthisevidenceinthetargetcontext.

Theremainderofthischapterwillintroducethegeneralfeaturesofexternalvalidityand

evidentialrelevancewithrespecttomechanisms,afterwhichthecasestudiesinthesubsequent

chapterswillfurthersubstantiatethearguments.

1.3. THEPROBLEMOFEXTERNALVALIDITY

This sectionwill dealwith some of themain aspects ofmechanismswith respect to external

validity,focusingonwhyexternalvalidityisviewedasaseriousmethodologicalproblemandhow

mechanisms are supposedly able to resolve this issue. Most importantly, the procedure of

mechanismdesignwillbegivenspecialattentionas itappearstobeapromisingapproachfor

policymakers.22Thenextsectionwillintroducethenotionofevidentialrelevanceandwillshow

inwhatsensemechanismsindicatewhichevidencecouldberelevanttotheaimsofpolicymakers.

Externalvalidity–or‘extrapolation’asDanielSteel(2008)framesit–hasbeenacentral

methodologicalproblemforexperimentaleconomics.Tounderstandtheproblematicaspectsof

externalvalidity it isusefultocontrast itwith internalvalidity,whichindicatesthatthecausal

relationshipsfoundwithinanexperimentalsetting(suchasinalaboratory)areindeedvalid.That

is, internal validity is established when the causality of an experimental result is properly

understoodbytheinvestigator.Somewhatmoreformally:theeffectEofanexperimentXcanbe

consideredinternallyvalidiftheproductionofEisattributedtoacause(orsetofcauses)C,and

CreallyisresponsibleforEinX.Therearenumerousproblemsrelatedtointernalvaliditytoo,but

thesearenotrelevantforourcurrentdiscussionofmechanisms.23Fortheproblemofexternal

validitytooccur,onemustassumeanexperimentalresulttobeinternallyvalid:itdoesnotmake

muchsenseto investigatewhetheraresult isvalidoutsidetheexperimentalsettingunlesswe

havereasontobelievethatitistherein.

22Foronething,twoNobelprizesineconomicshavebeenawardedtoworksonmechanismdesign:onetoLeonidHurwicz,EricMaskinandRogerMyerson(in2007);andtheothertoAlvinRothandLloydShapley(in2012).23Onekeyprobleminexperimentalsettingsisthatofconfoundingfactors,whereitisnotclearifacausalfactorleadstotheobservedeffectorifitinfluencestheeffectviasomeother(unobserved)causalfactor.

22

Whatisproblematicaboutthevalidityofexperimentalresultsineconomics,then,isthat

theyarehardlyeverobtainedincontextsoutsidethelaboratory.InhisbookAcrosstheBoundaries

(2008),Steeldefinesextrapolationas“thechallengeoftransferringcausalgeneralizationsfrom

one context to another when homogeneity cannot be presumed” (ibid.: 3). The absence of

homogeneitymeansthatthepopulationtowhichtheexperimentalresultisextrapolateddiffers

with respect to characteristics thataffect theoriginallyobtainedcausal relationships. Inother

words,transferringcausalknowledgefromonesituationtoanotheroftendoesnotworkbecause

thecomplexitiesofthelattermakeitunsurewhetherthecausalrelationshipswillhold.Thefact

that C causes E in one experimental setting, which may include highly stringent background

conditionsupon theagentsandmaterials involved,doesnotnecessarilymean thatCwill also

causeEinanothersettingwherethebackgroundconditionsmightbeverydifferent.

Astraightforwardwaytoillustratestheproblemofexternalvalidityistheexampleofthe

BangladeshIntegratedNutritionPolicy(BINP).24BasedonthepreviouslysuccessfulTamilNadu

Integrated Nutrition Project (TINP), the aim of the BINP was to decrease the amount of

malnourished children in Bangladesh. A focal point of both policies was to educate pregnant

womenandyoungmotherstobetternourishtheirchildrenaswellasthemselves.Althoughthe

twopolicieswereverysimilarintheirsetup,theenvironmentsinwhichtheywereimplemented

differed in several important respects. One aspect proved to be particularly significant in the

failureoftheBINPtodecreasemalnutritionamongchildren:itisnotthemotherbutthepaternal

grandmotherwhousuallytakescareofchildreninBangladesh.Whilemotherswereeffectively

targetedbytheTINP,inthecaseoftheBINPtheinterventiondidnotachievethesameresults.

Theeducationofmothersmayleadtoadecreaseinmalnutritionamongchildreninonecase,but

it may not do so when similar educational programmes are introduced elsewhere. Thus, the

exampleshowsthateventhoughknowledgeofacausalrelationshipmaybevalidinoneparticular

environment,itmayceasetoexistwhenappliedtoadifferentenvironment.

1.3.1. MECHANISM-BASEDEXTRAPOLATION

Giventheincreasingrelianceofeconomicpoliciesonexperimentalresults–generallyreferredto

as‘evidence-basedpolicy’byscholarssuchasNancyCartwrightandJulianReiss–itisimperative

forpolicymakerstobeawareoftheproblemofexternalvalidityand,evenmoreimportantly,to

knowhowtomitigateitsmainimplications.Oneprominentproposaltoovercomethisproblem

makesuseofmechanisms,orwhatSteelcalls‘mechanism-basedextrapolation’(Steel,2008:85).

WhileSteelisquitescepticalabouttheabilityofmechanismstoenablesuccessfulextrapolation

24ThisexampleisborrowedfromCartwrightandHardie(2012).

23

ineconomics,ourdiscussionwillfirstpresentthemainargumentinfavourofmechanism-based

extrapolation.

Over the past decade, Francesco Guala (2005, 2011) has been one of the key proponents of

mechanismsinappliedeconomics.Ingeneral,hearguesthatmechanismsareabletobridgethe

gapbetweenexperimentalresultsontheonehand,andtheirapplicationinrealworldsituations

on theother.This isdonebywayof specifying the similaritiesbetween theexperimental and

targetpopulations.Mechanisms establish these similarities in termsofbackground conditions,

whichtogetherindicatethelikelinessofacausalrelationshipstoholdinthetargetenvironment.

For example, knowledge of a mechanism that included the social norms within Bangladesh’s

societywouldhaveincreasedthechancesofsuccessfortheBINP.Ifpolicymakerswouldhave

beenawareof the fact that thesimilaritiesbetweenthetwopopulations(i.e. theexperimental

populationoftheTINPandthetargetpopulationoftheBINP)werelimitedwithrespecttofamily

structure and children’s care, then they would have been able to adapt their intervention

accordingly.

Toclarifythispoint,assumetherewasonlysomepieceofabstracttheory–withoutany

empirical evidence whatsoever – that described the positive causal relationship between the

educationofmothersandnutritionofchildrenindevelopingcountries.Inthissituation,policy

makersmustfirstknowwhetherthiscausalrelationshipwillbeconfirmedempiricallyatall.They

canbegintheirinquirybydevisinganexperimentinwhichthesupposedcausalrelationshipis

puttothetest.Fromtheseexperimentalresults,whichturnedoutoverwhelminglypositiveinthe

caseoftheTINP,policymakersthenhavesomereasontobelievethatthecausalrelationshipwill

alsoholdinthecontextoftheirinterest(i.e.Bangladesh).

However,itisherethatpolicymakershavetopayattentiontothedissimilaritiesbetween

theexperimentalandthetargetcontexts.Inthiscase,forinstance,backgroundknowledgesuch

associalnormsplayedanimportantroleinextrapolatingtheresultsoftheTINPtothecontextof

Bangladesh.Thesekindsofbackgroundconditionsaboutthetargetcontextarecrucialforpolicy

makers,astheydeterminewhethertheirinterventionswillprovesuccessfulornot.Sometimes,

knowledgeofbackgroundconditionshastobeobtainedfromawiderangeofsources,suchas

otherscientificdisciplines.25

ForGuala,themainadvantageofusingmechanismsforeconomicpolicyistheintegration

oftheoretical,experimentalandbackgroundknowledgeintoaunique,newmechanismthatisable

toperformeffectivelyinthetargetenvironment.Hedescribesthedesignofmechanismsas“an

enterprisebetween theoretical and applied economics,which requires first stating clearly the

25 In case of the BINP, “identification of the ‘mother-in-law’ effect came from reading anthropologicalliterature”(White,2009;citedbyCartwright2012:988).

24

goalstobeachievedbythemechanism,andthenfindingthebestmeanstoachievethemgiventhe

circumstances” (Guala, 2005: 164). So, once the aim of a particular intervention has been

determined,policymakersneedtostudydifferentmechanismsthatcouldpotentiallybeusefulto

achieve thisaim.Essentially, theprocedureofmechanismdesignmakesuseofmanydifferent

kindsofknowledge, includinghypothesesaboutmechanismsthatpurportedlyexist inthereal

worldandcouldpotentiallybeusedforextrapolation.

Yet where it differs from traditional mechanism-based extrapolation is in its creative

aspect, i.e. in the construction (or engineering) of awhollynewmechanism that didnot exist

before.AccordingtoGuala,policymakersadaptthemechanismsthatpresumablyexistinthereal

worldsoastomakethemoperationalinthetargetenvironment;theytakewhattheyneedfrom

eachpurportedmechanism, so to say,and integrate it into theirmechanismdesign.What this

procedure looks like and what role different kinds of knowledge – theory, experiments,

backgroundconditions–playwillbecomeclearinthediscussionofthetwocasestudies.

1.3.2. STRUCTURE-ALTERINGINTERVENTIONS

At this point, let us shortly discuss one of the main criticisms that have been raised against

mechanism-based extrapolation. As mentioned before, Daniel Steel is, on average, rather

pessimistic about thepotentialofmechanisms inextrapolatingexperimental results.Hismain

pointofcritiquehas todowith thestructure-alteringnatureof interventions,which isclosely

relatedtothefamousLucasCritique(1983)ineconomicmethodology.Lucasbasicallyarguesthat

manypolicy interventionswill prove ineffective because they tend to change the institutional

structureuponwhichtheinitialcausalrelationshipsarebased.Withrespecttoeconometricpolicy

interventions,heclaimsthat:

“Given that the structure of an econometric model consists of optimal decision rules of

economic agents, and that optimaldecision rules vary systematicallywith changes in the

structureofseriesrelevanttothedecisionmaker, itfollowsthatanychangeinpolicywill

systematicallyalterthestructureofeconometricmodels.”(ibid.:279).

AccordingtoSteel,sincemechanismsarecausalstructuresthatoperateondifferentlevels

of abstraction, including the macro or institutional level (see figure 3, for instance), policy

interventionsthatdrawuponknowledgeofmechanismsoftenresultinchangingtheinitialcausal

relationships.Inotherwords,interventionsbywayofmechanism-basedextrapolationwilllead

toasignificantchangeintheconditionsthathadcontributedtothe(internal)validityofthecausal

25

relationships in the first place. Yet what exactly makes these causal relationships change in

responsetointerventionsbypolicymakers?

Thisquestioncanbeansweredbylookingattheconceptofmodularity,whichrequires

interventionsupononecomponentofthemechanismtoleavetheothercomponentsunaltered.

Thatis,ifaninterventionismodularthenitonlyaffectsthatpartofthemechanismforwhichit

wasintended;aninterventionissaidtoviolatemodularityifitaltersother(unintended)partsof

themechanismaswell.ThisinterpretationbySteelofwhatinterventionsaresupposedtodoand

whatnot,soundsratheridealistic.Indeed,thistypeofinterventionhasbeenaptlyreferredtoas

‘idealinterventions’bySteel,aswellasbyotherscholarsusingsimilarconnotations.26

OriginallydevelopedbyJamesWoodward(2003:98–99),anidealinterventionisformally

definedaccordingtothefollowingfiveconditions:

1. [Intervention]Icauses[variable]X;

2. IactsasaswitchforallothervariablesthatcauseX;

3. AnydirectedpathfromIto[theeffect]YgoesthroughX;

4. Iis(statistically)independentofanyvariableZthatcausesYandthatisonadirectedpaththat

doesnotgothroughX;

5. IdoesnotaltertherelationshipbetweenYandanyofitscausesZthatarenotonanydirected

path(shouldsuchapathexist)fromXtoY.

Takentogether,theconditionsforidealinterventionsareverydemanding,andforgoodreason:

policymakerscanonlyknowwhethertheirinterventionsareactuallyeffectiveiftheydonotalter

thecausalrelationshipstheywishtoexploit.Mostofall,policymakersneedastableconnection

betweenthevariabletheymanipulateontheonehand,andthevariabletheyintendtoinfluence

ontheother.Sincethestabilityofthisconnectiondependsonthebackgroundconditionsthatare

present,interventionsthatchangetheseconditionstendtomakethecausalrelationshipundone.

Therefore,mechanism-basedextrapolationbecomessomewhatself-defeatingand,consequently,

theproblemofexternalvalidityremains.

Given that modularity is a serious issue for policy makers, it is hard to see how

mechanismscanbeused for the taskofextrapolation. It seems that theonlyway to solve the

problemofexternalvalidityistoconductidealinterventions,i.e.interventionsthatdonotviolate

modularity. The question is then: how can policy makers make sure their interventions are

modular?Moreimportantlyforuswouldbethequestionwhatrole,ifatall,mechanismsplayin

this task. Without going into detail here, the issue depends on one’s interpretation of what

26 Julian Reiss, for instance, speaks of “hypothetical interventions” having a number of very idealisedproperties(2008:162).

26

mechanismsaresupposedtodo.Viewedfromthemechanism-designinterpretationputforward

byGuala,mechanism-basedextrapolationisonlyonepartofthestory.Inthissense,mechanisms

arestudiednotmerelyforextrapolatingsometheoreticalorexperimentalresultbutprimarilyto

provideusefulinputforthedesignofanew,moreappropriatemechanism.Heretheeffectiveness

ofaninterventionlargelyreliesonhowthismechanismisdesigned,inwhichtheissueofexternal

validity–andthustheissueofmodularity–mightbelesspressing.

1.4. ALTERNATIVEPERSPECTIVE:EVIDENTIALRELEVANCE

Sofar,thepotentialofmechanismsforpolicy-makinghasbeendiscussedinlightoftheproblem

ofexternalvalidity.Moreprecisely,theargumentthatmechanismscansupportextrapolationhas

been critically assessed. Despite numerous publications that have contributed to this debate,

whethermechanismstrulysolvetheproblemofexternalvalidityremainsahighlycontestedissue

withinthephilosophyofsocialscience.Forthisreason,itisfruitfultoalsolookatthepotentialof

mechanismsforeconomicpolicyfromanotherperspective.

This section will introduce the second argument in defence of the usefulness of

mechanisms for policy purposes. This argument has to do with the ability of mechanisms to

provideapreliminaryunderstandingoftheevidencethatcouldberelevantfortheeffectiveness

ofpolicyinterventions.Framedinthisway,theargumentissomewhatmoregeneralinthatitdoes

notrespondtoonespecificproblem(thatofexternalvalidity)directly.Thoughitshouldbeseen

as complementary to the previous argument about external validity, whereby this second

argument addresses an important issue that has been relatively ignored in the debate about

mechanismswithintherealmofpolicy.

Sowhatisthisneglectedissueandwhyisitimportant?Fundamentally,ithastodowith

the relevance of causal knowledge for the purposes of policy-making. In the process of

establishingwhethersomecausalrelationshipisrelevantforaparticularpolicyhypothesis,the

economic theorist or experimenter adopts the perspective that takes his theoretical or

experimentalresultaspointofdeparture.Putdifferently, thetheoristorexperimentertriesto

answerthequestion‘inwhichcontextsaremyobtainedresultsrelevant?’.Whilethisperspective

is of course completely legitimate, it differs from that of policymakers in a crucialway.They

basically adopt the opposite perspective, answering the question ‘what kind of evidence is

relevantformypolicyhypothesis?’.Thesetwoperspectivesmightappearrathersimilaratfirst

sight, but it will become clear that their differences have important consequences for our

assessmentofmechanisms.

27

1.4.1. FFROMEFFICACYTOEFFECTIVENESSTomake sense of these different perspectives it helps to become familiar with an important

methodologicaldistinction.NancyCartwrighthasmademanyvaluablecontributionstothetopic

ofevidence-basedpolicy,whereshehasbothadvocatedtheusefulnessofmechanismsforpolicy

makers as well as expressing concern about the true potential of mechanisms in supporting

evidentialrelevance.27Aboveall,shemakesoneparticularlyinterestingconceptualdistinctionin

thisregard,namelyefficacyasopposedtoeffectiveness,whereefficacyis“theabilityofatreatment

toproducebenefitifappliedideally”andeffectivenessis“thebenefitthatactuallyoccurswhena

treatmentisusedinpractice”(Andrews,1999;citedinCartwright2009c:187–188).

The first concept, efficacy, is similar to that of internal validity: it indicateswhether a

causalrelationshipobtainedinanexperimentalsettingisgenuine.Ormoreprecisely,efficacyis

establishedwhenCcausesEiftheproductionofEisattributedtoCamongthepopulationinX.

Oneprominentmethodforgatheringevidenceforefficacyisarandomisedcontrolledtrial(RCT).

Originally based on John Stuart Mill’s method of difference (1843), RCTs engage in causal

inferencebydividingtheexperimentalpopulationintotwogroups:thetreatmentgroupandthe

controlgroup.Thetreatmentgroupisactuallyexposedtothecausalvariablethatispresumedto

makeasignificantdifferencetothephenomenonunderinvestigation.

Incontrast,thecontrolgroupis,unknowingly,notexposedtothesamecausalinfluence.

Thereasonforhavingtwoseparategroupsisthatanexperimentalsetuplikethiswillmitigatethe

problemofconfounders.Sincecompositionofbothgroupsissupposedtoberandom,thismeans

that“thedistributionofcausalfactorsotherthantheoneinquestionbetweenthetwogroupsis

(near enough) identical” (Cartwright&Hardie, 2012: 33). That is, the randomization element

impliesthattheinfluenceofconfoundingvariablesonthecausalrelationshipofinterestisevenly

distributed and therefore rendered insignificant. As a result, RCTs have been successfully

conducted in a wide range of scientific domains, most notably medicine. 28 Disregarding the

legitimateconcernthatRCTsmightnotbecompletelyabletoresolvetheproblemofconfounders,

theyaregenerallyconsideredtobeasuitablemethodforestablishingefficacy.

27 For instance, in her book Evidence-based Policy (2012) Cartwright, together with Jeremy Hardie,extensivelydealswiththeproblemofexternalvalidityspecificallyforthepurposesofpolicymakers.Itisimportant to note, however, that she does not explicitly refer to ‘mechanisms’ in most of her work.Nevertheless,herdiscussionofissuesrelatedtoexternalvalidity–suchasefficacyversuseffectiveness–cantoalargeextentbeusedfortheassessmentofmechanismsindifferentpolicycontexts.28Clarkeetal.(2014)discussthemanyproclaimedsuccesses,andconsequentpopularity,ofRCTswithinthefieldofevidence-basedmedicine,whichisdefinedas“theconscientious,explicit,andjudicioususeofcurrentbestevidenceinmakingdecisionsaboutthecareofindividualpatients”(ibid.:339).TheychallengethehighpositionofRCTsinthegeneralhierarchyofevidencebyclaimingthatmechanisticevidenceshouldbetreatedascomplementarytotheevidenceobtainedbyRCTs(andothertypesofstatisticaltrials,suchascohortstudies,caseseries,etc.).

28

RCTshavealsobeensuccessfullyconductedineconomics,astheexampleoftheTennessee

class-size reduction programme has shown. 29 In short, the 1985 STAR project, which was

designed as an RCT, concluded that pupils in smaller classes performed better than those in

relatively larger ones. The RCTwas able to confirm the positive causal relationship between

student performance and class size – albeit in only one particular context. This conclusion,

especiallywithrespecttominoritychildrenwhobenefittedmostfromthereductioninclasssize,

fittednicelywith theopinionsof politicians andotherpolicymakers. Surely this evidence for

efficacycouldbeusedforsimilarpolicieselsewhere,orsotheythought.

Unfortunately,thishasprovednottobethecase.Theresultsthatpolicymakershadhoped

forstayedoutwhenasimilarclass-sizereductionprogrammewasintroducedsomeyearslaterin

California. Cartwright (2009b) lists two plausible explanations for this failure. First, the

implementationoftheprogrammeswasdifferentineachcase.TheprogrammeinCaliforniawas

rolledoutinashortperiodoftime,whichcreatedasuddendemandforadditionalteachersand

classrooms. Consequently, large numbers of poorly qualified teachers were hired and many

classeswereorganizedinroomsthatwereinappropriateasa learningenvironment.Secondly,

thedistributionofconfoundingfactorsislikelytohavebeendifferentinCaliforniaascompared

toTennessee. It isplausiblethatparentswhoactivelyengagewiththeirchildren’seducational

development–bypracticingreadingcomprehensionathome,forinstance–areinclinedtosend

themtoschoolsthathavesmallerclasses.Thisway,acommoncauseexistedbetweenclasssize

andreadingperformanceinthecaseofCalifornia;aconditionthatwasnottakenupintheoriginal

RCTconductedinTennessee.

Thisexampleshows,muchlikethecaseoftheTINPandtheBINP,thatestablishingefficacy

issimplynotenoughifpolicymakerswanttoexploititsresultsinothercontexts.Whatisrequired

inaddition is theextrapolationof theexperimentalresults toseeminglysimilarsituations,but

with often very different populations andbackground conditions. Indeed, this is precisely the

issueofexternalvalidity:howdopolicymakersmakesurethatefficacy(orinternalvalidity)also

holds outside the experimental setting? As the previous section (1.3) has already shown,

mechanism-basedextrapolationcouldbeoneusefultoolforthisaim,asmechanismsspecifythe

necessarybackgroundconditionsfortheinterventiontobeeffective.Understoodinthissense,

mechanismscanbeusedtomovefromefficacytoeffectiveness;from‘itworkedthere’to‘itwill

workhere’.

29Again,thisexampleisborrowedfromCartwrightandHardie(2012).

29

1.4.2. CAUSALSCENARIOS

However, the crucial aspect that ismissingherehas to dowith the relevance of the evidence

availabletopolicymakers.Evidenceforefficacy,asestablishedbyRCTs,canbevaluabletothe

experimenterandotherscholarsinhisparticularfield,butitisnotnecessarilyrelevantforpolicy

makerswhoareoftenconcernedwithdifferenthypotheses.AccordingtoCartwright,besidesthe

requirementofevidenceforapolicyhypothesistobecredible–i.e.evidencethatislikelytobe

true–italsoneedstoberelevant.Itisnotsufficientforpolicymakerstoknowthatevidencefor

acertainefficacyclaimistrueinsomeexperimentalsetting;rather,inaddition,theyrequirethat

evidencetoberelevantforthepolicyaimathand.

Fairenough,buthowdopolicymakersknowwhetheraparticularpieceofevidencefor

efficacy is actually relevant?Amore appropriate formulation of this questionwould be:what

evidence(forefficacy) isrelevantforthepolicyhypothesis?Thisway,theusualperspectiveof

moving fromefficacy to effectiveness is switchedaround:policymakersdonot startwith the

extrapolationofexperimentalresultsbutinsteadfocusonthecriteriaforapolicy’seffectiveness.

Todothis,Cartwrightproposesthatpolicymakers“begintoconstructavarietyofdifferentcausal

scenarios[aboutwhatwillhappenwhenaninterventionisconducted],somemoreplausibleor

more probable than others” (Cartwright, 2009b: 135). These scenarios would start with the

interventionandendwithitspreferredoutcome,basedonseveralintermediatecausalsteps.For

example, when the outcome is improved reading performance by Californian students, one

plausible scenario could indeed involve a class-size reduction programme that has reading

performanceasitsultimateeffect(seefigure5).Thisscenarioassumesclasssizetoinfluencehow

muchpersonalattentionstudentsreceivefromtheirteachers,which,togetherwiththeintrinsic

abilityofstudents,determinereadingperformance.Inthiscase,reducingclasssizeseemstobe

aneffectiveinterventioninordertoimprovestudents’readingperformance.

Therecouldbe,ofcourse,manyotherplausiblescenariosthatpolicymakersmighttake

into account when they deliberate about potential interventions. For instance, introducing

Figure5:Sketchofoneplausiblemechanismbetweenpersonalattentionandintrinsicability(ascauses)ontheonehand,andreadingperformance(astheeffect)ontheother.Inthisscenario,interveningontheaverageclasssizeofstudentswouldpresumablylead–viamorepersonalattention–toanincreaseinreadingperformance.

30

mandatoryreadingassignmentsthroughouttheschoolyearmightenhancetheintrinsiclearning

abilitiesof students,whichcould in turn lead tobetter readingperformance(see figure6).As

mentionedalready,theproblemforpolicymakersisnotthedifficultyinconstructingplausible

mechanisms,butthatthereareoften(too)manydifferentplausiblemechanismsavailable–all

relevant,tosomeextent,totheparticularpolicyaim.Evaluatingtheplausibilityofmechanismsis

not just about yes-or-no questions, but about how plausible certainmechanisms actually are.

Answeringtheselatterkindsofquestionslargelydependsonthecontextinwhichpolicymakers

operate,asthecasestudiesinthefollowingchapterswillillustrate.

Fornow,supposepolicymakershaveidentifiedamechanismasinfigure5,wherethey

have some reason to believe that reducing class size will have a positive effect on reading

performance.30This belief couldbe reasonablybased, for instance, onpopular opinion among

parentsandpoliticians.Asanextstep,policymakerscouldusetheevidenceforefficacybetween

class-size reduction and reading performance obtained in the STAR project in order to

substantiatethisbelief.WithoutblindlyrelyingontheresultsoftheRCTconductedinTennessee,

policy makers are now able to account for different causal variables, as well as background

conditions,thatmayhaveaneffectonthetargetvariable.Whileit isplausiblethatinthiscase

reducingclasssizewillenhancereadingperformancebasedontheRCTresults,itmightbejustas

plausiblethatintroducing(extra)mandatoryreadingassignmentswillsignificantlycontributeto

thesameoutcome.Ortherecouldexistsomeothermechanismwithanevengreaterplausibility

thathasyettobeidentified.

Nonetheless, themainadvantage is thatpolicymakers candevelop their interventions

more carefully, and thereforemore effectively, due to their preliminary understanding of the

evidence that could be relevant for the effectiveness of policy interventions. In this sense,

30It is important to note that in this example policymakers have identified this particularmechanismindependentofanyevidenceforefficacybetweenclass-sizereductionsandreadingperformance.Althoughtheevidenceforefficacybetweenthetwovariablescouldplayarolelateroninthepolicy-makingprocess,itdoesnothavetoplayaleadingrole(suchasintheTennessee-Californiaexample).

Figure6:Sketchofanalternativemechanismbetweenpersonalattentionandintrinsicability(ascauses)ontheonehand,andreadingperformance(astheeffect)ontheother.Inthisscenario,interveningontheamountofreadingassignmentsmightalsolead–viaenhancedintrinsicability–toincreasedreadingperformance.

31

mechanisms show policymakerswhich variables and background conditions are likely to be

relevant.Whethertheevidenceactuallyisrelevanthastodeterminedempirically.Thus,within

some specific context, mechanisms are able to inform policy makers about the relevance of

differentkindsofevidence–including,butnotrestrictedto,evidenceforefficacy.

1.4.3. ANOTEONINTERPRETINGMECHANISMS

Toroundupthediscussionfornow,letmeemphasizethemaindifferencesininterpretingtheuse

of mechanisms for policy-making. Roughly speaking, there are three such interpretations:

mechanism-based extrapolation,mechanismdesign, andmechanisms as causal scenarios. The

firsttwopartlyoverlapwhilethethirdoneshouldbeseenasastand-alonealternative.

With respect to mechanism-based extrapolation, it is claimed that mechanisms can

supporttheextrapolationofexperimentalresultstoothercontextsbecausetheyshowhowcausal

relationshipsoperateacrossdifferentcontexts.Moreprecisely,mechanismsareabletoindicate

the similarity inbackgroundconditionsbetween theartificial and target environments,which

favours extrapolation in case the similarities are strong. This way, mechanisms can help

determinewhetheraninterventionislikelytobesuccessfulornot.

Guala (2005)endorsesmechanism-basedextrapolationbutaddsanotherdimension to

theuseofmechanisms,namelythedesignofmechanisms.Inthissense,policymakersstillaimto

extrapolate theoretical and experimental results to non-artificial environments by invoking

mechanisms.Yetmechanism-basedextrapolationisnowusedasameansforthedesignofanew

mechanism, one that enhances the objectives of interventions as best as possible. In short,

(purported) mechanisms are used to provide theoretical, experimental, and background

knowledgethatcanbeintegratedintoanew,specificallydesignedmechanism.

Theinterpretationofmechanismsascausalscenarios,however,isbasedonacompletely

differentlineofreasoning.Whereasmechanism-basedextrapolationandmechanismdesigntry

to move from efficacy to effectiveness, mechanisms as causal scenarios adopt the opposite

perspective.Thatis,policymakerstaketheaimofaninterventionaspointofdepartureandsee

whatkindofevidencecouldberelevanttothisaim.Theevidencemightincludeefficacyclaims

obtained insomeother (non-target)context,but this isoftenonlyasmall fractionof the total

evidenceavailable.Toacquireapreliminaryunderstandingoftheevidencethatcouldberelevant,

policymakersconstructdifferentcausalscenariosthatplausiblyconnecttheinterventionwith

the preferred outcome. This way, their approach is not one of ‘what worked there will also

(hopefully)workhere’butrather‘whatwillworkhere,giventhepolicyobjectives?’.

32

CHAPTERTWO

MECHANISMSANDAUCTIONPOLICY

When discussing the potential of mechanisms for economic policy, at some point one has to

transferthephilosophicalconcepts,arguments,problemsandsolutionstomorepracticalcontexts.

Ifwewishtounderstandwhymechanismscanbeausefultoolforpolicy-making,weneedtosee

how mechanisms actually operate in different policy domains. In this chapter, the use of

mechanisms in one such domain will be illustrated, namely the domain of auction policy. As

mentionedintheintroduction,thischapterwilldealwithonespecificcasestudyofrealauctions.

Thereasonforthisparticularcasestudyisthreefold:first,theauctionsinvolvedamultibillion-

dollar business, which not only had a significant impact on public finances but also on the

administration’seconomicperformance.Moreimportantly,theauctionscanbecharacterisedas

a kind of economic engineering where knowledge of mechanisms is used to design a new

mechanism thatworkswell in the policy context. In addition, the philosophical literature has

extensivelyengagedinthedebatearoundauctiondesignanditspolicyimplicationsoverthepast

fewdecades,whichprovidesamplematerialtoadvanceourassessmentofmechanismsforpolicy-

making.

Beforewedelveintothedetailsofthecasestudyrelatedtoauctionpolicy,theapproach

tothiscasestudy(andofthenextinchapterthree)mustbeclear.Thefollowingcasestudywill

beanalysedaccordingtothetwoperspectives–externalvalidityandevidentialrelevance–as

introducedinthepreviouschapter.Forinstance,howdidmechanism-basedextrapolationinthis

case supposedly solve the problem of external validity? What kind of role did theories,

experimentsandbackgroundconditionsplayinthedesignoftheauctions?Moreover,inwhatway

can mechanisms establish evidential relevance for the effectiveness of the auctions? Which

criteriawereused in thisprocess, andhowwere thesedetermined?From these twodifferent

perspectives,thecasestudywillbeabletosupporttheclaim,orsoIwillargue,thatmechanisms

canbeausefultoolforpolicy-making.

2.1. THEFCCAUCTIONS

Oneofthemostcelebratedapplicationsofgametheoryandexperimentaleconomicstoareal-

worldphenomenonwasasetofauctionsconductedbytheFederalCommunicationsCommission

33

(FCC),anagencyoftheUnitedStatesgovernment,in1994.31Thestageforthissuccesswasalready

set in the 1980’s when awave of decentralization hit the US economy. A new economic and

politicalparadigmwasestablishedthatreplacedcentralized,bureaucraticsystemsofallocation

withmoremarket-drivenprocesses.Theoldwayofallocatinglicensestouseradiospectrumwas

doneviaadministrativehearings,inwhicheachapplicanthadtoconvincetheFCCoftheir(public)

interestinobtainingthelicenses.Thisprocesswasslow,nontransparantandinefficientbecause

itallocatedthe licenses for free insteadofsellingthemfortheirmarketvalue.Aftertryingout

different alternatives, such as lotteries, the FCC turned to game theorists and experimental

economiststohelpdesignanefficientsetofauctions.

AlicenceforwirelessPersonalCommunicationSystems(PCS)providestherighttouse

someportionofthespectrumforradiocommunication(telephones,faxingmachines,etc.).The

primaryaimoftheFCCauctionswastoachieveanefficientallocationofthelicenses,whichmeant

selling the licenses to companies who valued them the most, thereby generating substantial

government revenue. Additionally, monopolies had to be prevented while relatively small

companies,andminority-ownedandwomen-ownedcompaniesweretobepromoted.Although

theseadditionalaimswerecontroversial,andtheirsuccessthereforeratherambiguous32,theFCC

auctions clearly succeeded in achieving itsmainobjective: a seriesof sevenauctionsbetween

1994and1996attractedmanybidders,allocatedthousandsoflicencesandraisedmorethan$20

billioningovernmentrevenues(Cramton,1998).

ThedesignersoftheFCCauctionstriedtousescientificknowledgeforaspecificpolicy

aim.Theteamincludedgametheorists,experimentalists,softwareengineers,lawyersandpolicy

makers who all worked together to create a procedure that could make the theory and

experiments work well in their target context. One way they did this was to use ‘testbed’

experimentsthatidentifiedimportantdetailsaboutthebehaviourofparticipants.CharlesPlott,a

prominentCaltechexperimentalist,definesthiskindofexperimentasfollows:

“Anexperimental ‘testbed’ is a simpleworkingprototypeof aprocess that is going tobe

employed in a complex environment. The creation of the prototype and the study of its

operation provides a joining of theory, observation, and the practical aspects of

implementation,inordertocreatesomethingthatworks.”(1996:1)

31Henceforth,Iwillrefertothisparticularsetofauctionsasthe‘FCCauctions’.32 For instance, the initial aims to promote minority-owned and women-owned companies (positivediscrimination)inthetelecommunicationsindustrywerewithdrawnbyorderoftheSupremeCourtin1995(McAfee&McMillan,1996:167).Nik-Khah(2008)arguesthatinfacttheauctions’onlysuccesswaslargegovernmentrevenue.

34

Forinstance,agroupoftrainedstudentswashiredtokeeptrackofthesoftwareproblems(bugs)

that arose during small-scale experiments. They focused on detecting the effects of minor

deviationsfromtherulesthatwereincludedinthedesignoftheauctions,suchasinvestigating

whathappensifparticipantsloginfrommultiplelocationsatthesametime.Itturnedoutthat

such apparently small variations in the auction design could have a significant impact on the

actualoutcomewhenimplementedinthetargetenvironment.

The actual FCC auctions were therefore designed according to theoretical insights –

especiallyfromgametheory–experimentalevidence,andasubstantialamountofbackground

knowledge.Theprocedureofmechanismdesigneffectivelyintegratesthesedifferentepistemic

sourcesinordertoconstructanewmechanismthatisabletobringaboutthepreferredoutcome.

IncaseoftheFCCauctions,policymakersstartedwithabstractgametheoreticalknowledgeand

proceeded to a more concrete design by incorporating the results of specifically conducted

experimentsandbystudyingsomeimportantbackgroundconditionsoftheirtargetcontext.This

so-called‘mechanismview’isquiteuniquebecauseitdiffersfromthemoretraditionalviewof

scientificknowledge,inwhichspontaneouslyoccurringphenomenaareexplainedbydeveloping

some kind of theory. While economists normally move from the study of phenomena to the

construction of theoreticmodels, policymakers aremore inclined to proceed in the opposite

direction(Guala,2005).

2.2. THENEEDFORAFULLERPICTURE

Inordertosubstantiatethepointthatpolicymakerstendtostudyphenomenaasawhole,this

sectionwillshowhowtheydidsointhecaseoftheFCCauctionsandinwhatsensethisprocess

differedfromthemoretraditionalviewineconomics.Forthispurpose,Iwilldrawontheworkof

AnnaAlexandrova(2006),whoarguesthatpolicymakersdonotjuststudycausalrelationships

inisolationbutinsteadinquireaboutphenomenatakenasawhole.Byengaginginthisparticular

discussion,thesubsequentargumentsrelatingtoexternalvalidityandevidentialrelevancewill

becomemoreconvincing.

SoinwhatsensedidthepolicymakerswhowereinvolvedintheFCCauctionsstudythe

phenomenonofspectrumauctions‘asawhole’?Whencriticisingthetraditionalviewofcausality

employed ineconomics–referredtoas themethodof isolationor theanalyticmethod,which

originatesinJohnStuartMill’saccountofcausaltendencies(1843)–Alexandrovadistinguishes

betweenderivationfacilitatorsandsituationdefiners.Bothtypesofassumptionsarenecessaryto

establish some general causal relationship. On the one hand, derivation facilitators are

35

assumptionsthatenableacertaindeductiontogothrough.Theyrepresenttheabstractaspectsof

atheoreticmodelthatfacilitate(mathematical)derivation.

OneexampleofaderivationfacilitatorAlexandrovamentionsistheassumptionthatutility

functionsshouldbetwice-differentiableinthegametheoreticmodelofaprivatevalueauction

withtwobidders.Withoutthisassumptionthemodelcannotpredictthevaluationsofthebidsin

equilibrium,wherebothbidders shouldbidhalf their valuation.33Thegeneral result thatbids

tendtobelowerunderafirst-pricerule(i.e.eachbidderonlysubmitsonce)andsealedbidding

(i.e.eachbidderdoesnotknowtheother’svaluation)provedtobeusefulforpolicymakers.Most

importantly,itshowedthemthatafirst-pricesealed-bidauctionwouldleadtobidsbelowtrue

valuation,whichwouldmeanlowergovernmentrevenuesifthistypeofauctionwasimplemented.

SincegovernmentrevenuewasoneoftheprimaryaimsoftheFCC,policymakerssubsequently

rejectedthistypeofauctionframework.

Situation definers, according to Alexandrova, “set the structure of the situation under

consideration” (2006: 181). That is, they depict the context of the phenomenon under

investigation so as to render the actual operation of causal relationships intelligible. These

empiricalfeaturestellthepolicymakerwhereandwhencertaincausalrelationshipswillhold.

Withrespecttothegametheoreticalmodeloffirst-pricesealed-bidauctions,situationdefiners

specify the bidding rules, the number of bidders, and the nature of their reasoning. These

assumptionsneednotbecriticaltoobtaintheresult–sincethesameresultwillbeobtainedwhen

therearemorethantwobidders,forinstance–buttheymayneverthelessbeimportantforpolicy

makerstomakesenseofaderivation.Forwithoutsituationdefiners,thederivationfacilitatorsin

agametheoreticalmodel“donottellushowtotranslatethestatementsaboutentitiesinamodel

constrainedbymathematicsintostatementsaboutnaturallyoccurringentitiesandpropertiesin

theworld”(ibid.:183).

Despitethefactthatbothderivationfacilitatorsandsituationdefinersarenecessarywhen

studying causal relationships in isolation, the presence of the former becomes problematic in

policycontexts.Thereasonisthatderivationfacilitatorsareoftenhighlyabstractassumptions

thatsimplydonotapplytothetargetenvironmentsinwhichpolicymakersareinterested.Even

though somederivation facilitatorsmight sound like entities that exist in the realworld (like

utilityfunctions),policymakerscannottreatthemassuchbecausetheydonotknowwhetherthe

assumptions actually hold in practice. For example, due to the mere presence of derivation

facilitators,models“canfailtodescribeevenonecontextinwhich[a]first-priceauctionleadsto

bidsbelowtruevaluation”(ibid.:182).Thus,gametheoreticalmodels,whichoftenincludeahigh

degreeofderivationfacilitators,onlyplayedalimitedroleinthedesignoftheFCCauctions.

33ForamoredetailedexpositionofthisparticulargametheoreticmodelthatwasusedinthedesignoftheFCCauctions,seeAlexandrova(2006:174–175).

36

So,yes,policymakersmadeuseofgametheorywhendesigningtheFCCauctionsbutonly

asabodyofknowledgesuggestinggeneralissuestobeawareof.Theydidnotstudygametheory

todetectanycausalrelationshipsthatcouldbeuseddirectlyintheirauctiondesign,sincenosuch

theoriesexistedforthespecificcontextofthespectrumauctions.Rather,theytreatedtheoryasa

goodwaytostarttheirauctiondesign,withouthopingtofindonekindoftheoreticalframework

thatwouldapplyperfectly.Thechallengeforpolicymakerswastofigureoutwhichtheoriescould

beusefulforthetaskathand.ThismainlydependedontheaimoftheFCCauctions,namelyto

generate a substantial amount of government revenue. From this perspective, the theoretical

resultofafirst-pricesealed-bidauctionleadingtorelativelylowbiddingappearedtoberelevant.

However,therelevanceofthisparticulartheorywaslimitedduetoitsambiguousresults

withregardtooneempiricalfeatureoftheFCCauctions:theexistenceofcomplementarities.The

problemwasthatthespectrumauctionsfeaturedstrongcomplementarities–i.e.thevalueofone

licensedependedonwhatotherlicensesaparticipantowned–whichmadethetheoreticalresult

obsolete. Put differently, given the existence of complementarities, the causal relationship

betweenfirst-pricesealed-bidauctionsandlowbiddingbehaviourcouldnolongerbereliedon.

Onecouldassume,forinstance,thatbidsforlicenseswithcomplementaryvaluewillbehigher

thanforthosewithout.However,openauctionsinthepresenceofcomplementaritiesmayalso

reintroducethesocalled‘winner’scurse’,whichisthetendencyforthewinningbidinasealed-

bidauctiontobetheonethatmostoverestimatesthetruevalueofanobject.Thepointwasthat

policy makers simply did not know how bidders would behave when an auction involves

complementarities.

Inresponsetotheseproblems,policymakersdidno longeradheretothemethodof isolation;

instead, theystudiedthephenomenonofspectrumauctions takenasawhole.Thisalternative

methodismoresensitivetotheempiricalfeaturesofagivencontext.AccordingtoAlexandrova:

“Themethod[intheFCCcase]wasdifferentfromthemethodofisolationbecauseratherthan

hunting after knowledge about the behaviour of tendencies in isolation from others, the

auctiondesignersinsteadsoughttofindoutfactsaboutonematerialsystemasawhole.They

remainedagnosticaboutallbut themostgeneral featuresof the tendenciesatwork.The

process by which these facts were established was a mixture of modelling and

experimentation,wheretheformerprovidedonlyindicationsofpossiblecausalrelationsand

thelatterrevealedamaterialimplementationofthedesirableeffectswithintheenvironment

oftheauction.”(2006:186)

37

By studying the details of a given phenomenon, policy makers tried to identify at least one

concreteenvironmentwherethedesignwouldsatisfytheiraims.Besidesusinggametheoryina

preliminary sense, they mainly relied on local, empirical knowledge of the target context.

Experimentaltestbedsfunctionedasaparticularlyimportantsourceofknowledgeinthisprocess

because these kinds of experiments showed towhat extent causal relationships derived from

theorycouldactuallyberealized.Forinstance,thepresenceofcomplementaritiesprovedtobea

significant hurdle in the design of the auctions and could only be solved by running a set of

experimentsthatincorporatedthisspecificcondition.Withthehelpoftheseresults,theactual

auctionsconductedbytheFCCincludedrulesthatforcedparticipantstocontinuebiddinginorder

tomaintain their eligibility. These rules effectively prevented a process of slow and cautious

bidding,whichenhancedtheefficiencyoftheauctions.

2.3. EXTERNALVALIDITY

Afterhavingclarifiedthatpolicymakersstudiedthephenomenonofspectrumauctionsasawhole,

letusturntotheproblemofexternalvalidity.Thissectionwillbeconcernedwithtwoaspectsof

externalvaliditywithrespecttotheFCCauctions:first,Iwillarguethatmechanismsplayedan

importantrole in thedesignand implementationof theauctions;andsecond, Iwillarguethat

mechanism design, though susceptible to structure-altering interventions, can be a suitable

extensionofmechanism-basedextrapolation,andshouldthusbetakenseriouslybytheeconomic

policy-makingcommunity.

Recall that Guala considers the design of a mechanism as “an enterprise between

theoreticalandappliedeconomics,whichrequiresfirststatingclearlythegoalstobeachievedby

themechanism,andthenfindingthebestmeanstoachievethemgiventhecircumstances”(2005:

164).Thefirstandmostimportantstepintheprocessofdesigningmechanismsisdetermining

theaimoftheintervention.Thatis,policymakersneedtobeclearaboutwhattheinterventionis

supposedtoachieve.AsfortheFCCauctions,theprimaryaimwastoraisesubstantialgovernment

revenue, aswell as some additional objectives such as the prevention ofmonopolies and the

promotionofminority-andwomen-ownedcompanies.

Thenextstepistoconstructthemosteffectivemechanismforthetaskathand,whichis

wheretherealworkbegins.Herepolicymakersmakeuseofthreedistinctsourcesofknowledge:

theory,experimentsandbackgroundconditions.34Thevalueofmechanismdesignisthatpolicy

34Admittedly,insomecasesthesesourcesofknowledgeoverlap:anexperimentmightmakeuseofcertainbackgroundconditions,orapieceoftheorymaybetestedbywayofexperiments.Nevertheless,toseethevalueofmechanismdesignforpolicy-makingitisimportanttodistinguishbetweenthesedifferentkindsofknowledge.

38

makers are able to integrate these different kinds of knowledge into concrete, effective

interventions.Mechanismdesignisthereforeacomplex,context-specificprocedurethatoutlines

howaninterventionaffectsthephenomenonunderinvestigation.Essentially,mechanismdesign

makesuseofmanydifferentkindsofknowledge, includinghypothesesaboutmechanismsthat

purportedly exist in the real world and could potentially be used for extrapolation. These

hypothesesformtheinitialbuildingblocksofthemechanismdesign,inwhichsomepartsofthe

(purported)mechanismsareusedwhileothersarenot:bitsoftheory,experimentalresultsand

background conditions are carefully selected and modified to fit the target environment.

Eventually,anewmechanismisdesigned–basedonthestudyofmanydifferentmechanisms–to

maketheinterventionworkeffectively.Letusnowseehowthisprocedureactuallytookplacein

caseoftheFCCauctions.

2.3.1. MECHANISMDESIGN

Firstupistheroleoftheoryinthedesignofanappropriatemechanism.Inhighlyappliedcases

suchastheFCCauctions,theprevioussectionhasalreadyshownthattheoryplayedarelatively

minor role because it often proved to be incomplete in the target context. In other words,

theoretical knowledge about the causal relationships operating in the specific context of the

spectrumauctionswasonlypartiallyavailable,atbest.Lackingacomprehensivetheoryofthese

causalprocesses,policymakershadtorelyon“piecemealtheoreticalinsights”(Guala,2005:169)

whendesigningtheFCCauctions.Theexampleofthefirst-pricesealed-bidauctionsprovesthis

point,sincethisparticulartheoreticalmodelcouldonlybeusedasaroughapproximationofwhat

actuallyhappens inreal-worldsituations.Thus, toput itphilosophically, theory isanecessary

conditionforthedesignofmechanismsbutitishardlyasufficientone.

Apartfromdrawingonabstracttheory,policymakersalsogainedagreatdealofunderstanding

fromconductingexperiments.Infact,theupshotofGuala’sdiscussionoftheFCCauctionsisthat

experiments were in large part responsible for its success. In particular, the introduction of

testbedexperimentshadledtoseveralsignificantresults,rangingfromtheefficiencyofdifferent

auctiondesignstothelikelyconsequencesofsoftwarebugs.Testbedexperimentswereusedin

severalwaysthroughouttheFCC’sauctiondesignprocedure.

First,theywereinstrumentalinchoosingoneauctiondesignoveranotherwithrespectto

their fundamentalproperties.Aswiththecomparative testbetweenasimultaneousascending

auction and a combinatorial sealed-bid auction, the former came out on top because the

experimenters noticed that the implementation of the latter raised too many practical

39

complications.These initial testsprovidedpolicymakerswith the firstoperationaldetailsand

potentialproblemsofthedifferentauctiondesigns.

Secondly, testbed experiments “wereused to transform the abstract design into a real

processthatcouldperformtherequiredtaskreliablyintheenvironmentinwhichtheauctionwas

to be implemented” (ibid.: 175). In this sense, the experiments can be seen as some sort of

economicengineering:ratherthantestingsomeexistingtheoreticalmodelofauctions,thetestbed

experimentsinsteadsupportedthedevelopmentofcompletelynewtypesofauctions.Theissue

wasnotaboutdiscardingonetheoryoranother,butwhethertheconcretepropertiesofagiven

auction designwould actually contribute to the aims of policymakers. In order to determine

which auctiondesignperformedbest, policymakers had to discover new causal processes or

sometimesconstructthemfromscratch–hencethenotionofengineering.Forexample,sinceno

gametheoreticalmodelofacombinatorialsealed-bidauctionexisted,policymakershadtorely

onexperimentsinordertoseewhetherthistypeofauctionwouldservetheiraims.Evenifthere

wouldhavebeensomemodelthat(intheory)produceddesirableresults,itcouldhavebeenvery

difficulttomakeitoperationalduetopracticalconstraintssuchastime,money,etc.Infact,lack

oftimewasoneofthereasonswhypolicymakersdidnotchoosetoimplementacombinatorial

sealed-bidauctiondesign.Therefore,itisclearthatpolicymakersbenefittedfromusingtestbed

experimentsindesigningtheFCCauctions.

Thirdly, the experience gained during the experimental testbeds proved valuable for

extrapolatingtheresultstoreal-worldsituations.InOctober1994,theteamofexperimentalists

monitored a real auction to infer whether the favourable test results of the simultaneous

ascendingauctionwouldalsoobtainoutsidethelaboratory.Akeyelementinthissuccessfulcase

ofextrapolationwasthestrongsimilaritybetweentheartificialsettingofthetestbedexperiments

andtheactualenvironmentoftheFCCauctions.Thetestbedexperimentshadbeenconstructed

(orengineered)soastomimictheempiricalfeaturesoftherealauctionsascloselyaspossible:

thesamenumberofbiddersandlicenses,theexistenceofcomplementarities,anequalnumberof

rounds,etc.Thisway,bothsituations(artificialandreal)couldbeeasilycompared,andpolicy

makerswereabletoconcludethatasimultaneousascendingauctiondesignbestservedtheiraims.

ThelastkindofknowledgethatpolicymakersmadeuseofinthedesignoftheFCCauctionswere

backgroundconditions.AccordingtoGuala,“theexternalvaliditystepwasbasedonacomparison

between laboratoryandreal-worldevidence, andstoodona stockof ‘backgroundknowledge’

aimedatmakingtheinferenceasstrongaspossible”(ibid.:183).Whatmadetheextrapolationof

theexperimentaltestbedresultstotheactualauctionssosuccessfulwastheamountofcontextual

information thatwas incorporated into theexperiments.Knowledgeofbackgroundconditions

addedagreatdealoffleshtootherwisebare-bonedauctiondesigns,sotospeak.

40

ToemphasizetheimportanceofbackgroundconditionsinthecaseoftheFCCauctions,let

usturntooneofthedetailsofthecontinuousascendingauctiondesign:eligibility.Abidderissaid

tobeeligibleifshe(1)hasmadeaninitialdepositbasedonthetotalnumberoflicensesshewants

tocompeteforand(2)herbidiseitherthestandinghighbidorishigherthantheminimumbid

allowed.Theideaofeligibilitywasintroducedtopreventtheparticipationofbiddersthatwere

notgenuinelyinterestedinobtaininganyspectrumlicenses,andtoregulatethedurationofthe

auctions.Since theFCCwasconcernedabout thecosts thatprolongedauctionswould incur, it

wantedtofindawaytospeeduptheauctionprocess.Thisparticularbackgroundcondition,along

with many others, guided policy makers in the auction design and eventually contributed –

throughtheintroductionofeligibilityrules–toitssuccessfulimplementation.

The abovediscussionof the three kinds of knowledgeusedbypolicymakershas shownhow

mechanisms played an important role in the design and implementation of the FCC auctions.

Before moving to a common critique against mechanism-based extrapolation, one feature of

mechanismdesignneedstobeemphasizedatthispoint.Mechanismdesignrelatestotheproblem

of external validity in an intricate way: policy makers extrapolate certain theoretical and

experimentalresults toa targetcontextby integrating themost importantpartsof thetheory,

experimentsandbackgroundconditionsthatareobtainedthroughthestudyofmechanisms.Here

extrapolationdoesnotproceedbytakingoneparticularpieceoftheoryorexperimentalresult

and applying it to a target context (as in the original interpretation of mechanism-based

extrapolation). Rather, it combines some basic theoretical insights with a set of specific

experimentalresultsandthemostsalientbackgroundconditionsinordertoconstructaneffective

intervention.Thepointisthatmechanismdesign,forittobesuccessful,isaninherentlycomplex

and context-dependent procedure. The careful and detailed design of the FCC auctions is a

foremostexampleofhowbitsofgametheory,experimentalbiddingbehaviour,andbackground

conditionsofthespectrumauctionswereusedtoservetheaimsofpolicymakers.

2.3.2. MECHANISMDESIGNANDSTRUCTURE-ALTERINGINTERVENTIONS

TheFCCauctionsarefrequentlyhailedasasuccessstoryofmechanismdesign,butcanonereally

drawanygeneralconclusionsfromthisparticularcase?Sincetheprocedureofmechanismdesign

partlyreliesonmechanism-basedextrapolation,wehavetotakethestructure-alteringnatureof

interventionsintoaccounthere,too.Asintroducedinchapterone,thispointofcritiqueisraised

bySteel(2008)whoisratherscepticalabouttheideaofmechanism-basedextrapolationwithin

economics.Morespecifically,hearguesthat:

41

“Causal relationships typically depend upon background conditions too numerous and

complex to fullyandexplicitly incorporate intoamodel.Consequently, interventions that

changesuchbackgroundconditionsmaybestructure-altering.Even if thegeneralizations

accuratelydescribedthecausalrelationshipundertheoriginalsetofbackgroundconditions,

it might be an inaccurate representation of that relationship in the new circumstances

broughtaboutbytheintervention.”(ibid.:155)

So, to say that an intervention is structure-altering means it changes the background

conditionsthatsupportthecausalrelationshipexhibitedbythemechanism.Asaresult,thecausal

relationshippolicymakerswish toextrapolatemayno longeroperatedueto the intervention.

Modularityisahelpfulconceptheresinceitrequiresaninterventiontoonlychangethatpartof

themechanismforwhichitwasintended.Forifaninterventionchangesotherpartsaswell,then

policymakerscannotknowforsurewhetheritwasactuallytheirinterventionthathadaneffect

(if at all) on the target variable. Therefore, the violation of modularity often results in the

breakdownofcausalrelationshipsandrendersinterventionsineffective.

Unfortunately, the issue of structure-altering interventions cannot be ignored when

discussingthepotentialofmechanismdesignwithinpolicy-makingcontexts.Infact,thedesignof

theFCCauctionssufferedfromquitesomeinterventionswithsignificantsideeffects.Oneexample

relates to the eligibility condition in the continuous ascending auction design. The supposed

problemherewasthatintroducingeligibilitydidnotjustneatlyaffectthedurationoftheauction

without having other unintended consequences, too.Whereas therewas no theory that could

informpolicymakersabouthowlongacontinuousascendingauctionwouldgoonfor,theyturned

toexperimentsthattestedtheeffectofdifferentbackgroundconditionsonthedurationof the

auctions.Inadditiontotheeligibilitycondition,policymakersalsoimposedlargebidincrements

soastoidentifythewinningbidderquickly.Yettheexperimenters“observedthatbigincrements

sometimeseliminatedbidderstooquickly,causingtheireligibilitytodropandthereforecreating

a ‘demand killing’ effect” (Plott, 1997: 633). It turned out that intervening upon a continuous

ascendingauctionby imposing largebid increments indeed shortened itsduration,but it also

impededtheeligibilityconditionfromoperatingproperly.

Interestingly,thisexampleillustratesasevereweaknessaswellwasapotentialstrength

in the mechanism-design account put forward by Guala (2005, 2011). On the one hand,

interventionsuponbackgroundconditionsoftenleadtoambiguousresultsbecauseofinteraction

effects:theeligibilityconditionandthebidincrementcondition,operatingatthesametime,had

adifferenteffectontheefficiencyoftheauctionsthanwhenappliedseparately.Moreworryingly,

theintroductionoflargebidincrementsnullifiedthepositiveeffectsoftheeligibilitycondition.

Thesekindsofinterventionsmayraisemorequestionsthananswers:giventheinteractioneffects,

42

whichconditionismoreimportantforincreasinggovernmentrevenue–eligibilityorlargebid

increments?Moregenerally,arethereanyotherbackgroundconditionsthatincreasegovernment

revenue,butdosounambiguously?

On the other hand, the question whether interventions are susceptible to interaction

effects can only be answered by conducting (more) testbed experiments. For instance, the

interactioneffectbetweentheeligibilityandincrementconditionscould,accordingtoPlott,only

be made sense of by further empirical testing: “The complex ways the rules [background

conditions] interact,and thepresenceofambiguities,donotbecomeevidentuntilone tries to

actually implement the rules in an operational environment” (1997: 628). In response to this

particular interaction effect, the experimenters made some additional adjustments to the

continuousascendingauctiondesign,suchaslongerormorefrequentbiddingrounds.Eventually,

theyconcludedthatmorefrequentroundsdidnotsignificantlyaffecttheefficiencyoftheauction.

Ofcourse,therewillalwaysbeariskthatstructure-alteringinterventionswilldomoreharmthan

goodtothedesignofamechanism.Thispointseemshardtodeny.Interactioneffectsmightbe

resolved by engaging in (even) more sophisticated experiments but this is by no means

guaranteed.Besides,thisstrategyislikelytobealengthyandcostlyendeavourforpolicymakers.

Nevertheless, the FCC auctions have proved the advantage of mechanism design over

mechanism-based extrapolation within the domain of auction policy. On a charitable

interpretation,mechanismdesign isable toaccount for themostproblematicconsequencesof

structure-alteringinterventionsdueofitscleveruseofexperiments.Tobefair,thisinterpretation

isnottoofarofffromSteel’sgeneralpositiononmechanism-basedextrapolation,inwhichhesees

“no reason in principle thatmechanism-based extrapolation cannot be successfully utilized in

socialscience”(2008:150,italicsadded).Perhapstheprocedureofmechanismdesignisasuitable

extension of mechanism-based extrapolation, where the background conditions of the target

environmentarefirstimportedintotheexperimentalsetupbeforethe(experimental)resultsare

extrapolated back to the realworld. Guala’s advice on economic engineering – “if you cannot

exportsomelaboratoryconditionsintotherealworld,youhadbettermakesurethattherelevant

aspectsof therealworldare imported intothe lab”(2005:189)–mayproveuseful forpolicy

makersinotherpolicydomains,too.

43

2.4. EVIDENTIALRELEVANCE

Untilnow,theFCCauctionshavebeendiscussedwithrespecttotheproblemofexternalvalidity.

Essentially, this problem can be seen as policy makers trying to move from efficacy to

effectiveness:giventhefactthatacausalrelationshipholdsinoneplace,howdowemakesureit

alsoworkswherewewantitto?Inthisregard,Ihavearguedthat,first,mechanismsplayedan

importantroleinthedesignandimplementationoftheFCCauctions;andsecond,thatmechanism

design, though susceptible to structure-altering interventions, can be a suitable extension of

mechanism-based extrapolation, and should thus be taken seriously by the economic policy-

makingcommunity.

However, moving from efficacy in artificial environments to effectiveness in policy

contextsistoadoptthewrongperspective.Aswasalreadyremarkedinchapterone,thecrucial

aspectthatismissingherehastodowithevidentialrelevance.Thatis,bytakingsomeefficacious

result as point of departure in their inquiry, policy makers tend to overlook other kinds of

evidence that could be relevant to their aims. For this reason, policy makers should be less

concernedwith whether a particular efficacious result is also operative in the target context

(basically the issueofexternalvalidity). Instead, theyshould focusonwhatkindofevidence–

whichmayincludebutisnotrestrictedtoevidenceforefficacy–isactuallyrelevanttotheirpolicy

hypotheses.Thisalternativeperspectiveinvolvesawhollydifferentuseofmechanisms,namely

ascausalscenariosshowingtheintermediateprocessesbetweenaninterventionanditsdesired

outcome.

Inthissection,Iwillargueforthreethings.First,Iwillshowthatpolicymakersinvolved

in the FCC auctions adopted the right perspective as they focused on establishing evidential

relevance.Second,theyproceededinthistaskbyconstructingcausalscenariosthatprovideda

preliminaryunderstandingoftheevidencethatcouldberelevantfortheeffectivenessofpolicy

interventions.Third,sincethemovefromefficacytoeffectivenessisreversed,theissuesrelated

toexternalvalidityappeartobelesspressingfromapolicypointofview.

2.4.1. AFOCUSONRELEVANCE

Asdiscussedearlierinthischapter(section2.2.),policymakersinvolvedintheFCCauctionsdid

notmerelystudycausaltendenciesinisolation,butinsteadstudiedthephenomenonofspectrum

auctionsasawhole.Morespecifically,theydidnotonlyuseinsightsfromgametheorybutalso

drewuponarangeofexperimentalresultsandbackgroundknowledgefortheirauctiondesign.

For instance, themodelofa first-pricesealed-bidauctiondidnot fitwellwiththeexistenceof

complementaritiesinthereal-worldspectrumauctions.Thepreciseeffectofcomplementarities

44

couldonlybelearnedthroughtheuseofexperimentaltestbeds,whichalsotestedtheeffectsof

otherbackgroundconditionsandtheirinteractions.

Inthissense,policymakerstreatedevidenceforefficacyasonlyoneofthe(many)kinds

ofevidencethatmightberelevanttotheirinterventions.Theirprimaryaimwastobringaboutan

efficient allocation of spectrum licenses and to raise substantial government revenue, which

featuredastheirpointofdeparture.Thatis,everypieceofevidence–eitheragametheoretical

model,someexperimentalresultorasalientbackgroundcondition–wasevaluated in lightof

whether itcontributedtotheefficiencyof theauctionsandtheUStreasury.Sowhatmattered

mostwastherelevanceof theevidencewithrespect to thespecificaimsofpolicymakers; the

issue of whether the evidence was also credible had to be taken up after its relevance was

established.

Right from the start, the FCC informed the group of game theorists, experimentalists,

softwareengineersandlawyersaboutitsobjectives.Theseobjectivesactedasconstraintsforthe

kindsofevidencethatweretobegatheredduringtheauctiondesignprocedure.Inprinciple,any

evidencethatwasproducedinthisprocedurewasallowedaslongasitsupportedtheaimsofthe

FCC. For example, the first-bid sealed-price auctionmodelwas brought in as evidence for its

contributiontothewinner’scurse.Thisparticularpieceoftheoreticalevidencewasonlyrelevant

totheFCCbecauseitsuggestedthatclosedauctionswouldreducegovernmentrevenue.Dueto

thisevidence,policymakersknewaclosedauctionwouldnotservetheiraimswellandoptedfor

anopenframeworkinstead.35Fromapurelyscientificpointofview,itisclearlyvaluabletoknow

that a causal relationship between closed auctions and the winner’s curse exists. For policy

makers,though,thisknowledgealsoneedsberelevanttotheirspecificpolicyaims,whichinthis

caseitwas.

2.4.2. CAUSALSCENARIOS

Howthen,couldpolicymakersinquireabouttherelevanceofdifferentkindsofevidence?Thisis

where the interpretation of mechanisms as causal scenarios comes in: mechanism provide a

preliminaryunderstandingofthecausalrelationshipsthatcouldberelevanttothepolicyaimsat

hand. Policy makers draw on different kinds of theoretical, experimental, and background

knowledge in constructing these causal scenarios. At least, the scenarios need to be plausible

35AnnaAlexandrovaandRobertNorthcott (2009)note that thedecisionbypolicymakers tochooseanopen auction framework for their final design required “a judgment call rather than a neat theoreticaldemonstration” (ibid.: 317). This does not only show that the ultimate auction design included manydifferentkindsofevidence,butalsothatmechanismscansometimesmerelyindicatethatsomepieceofevidenceisnotlikelytoberelevant.Nevertheless,thislimitedroleofmechanismsmightstillproduceusefulinsightsforpolicymakerssincetheyareabletoeliminateirrelevantevidence.

45

accordingtosomebasictheory,generalprinciple,orwidely-heldpublicopinion.Eachscenario

startswiththeproposedinterventionandendswithitseffectonthepreferredoutcome.

IntheFCCcase,wehaveseenthatpolicymakerstooktheaimsofthespectrumauctions–

i.e.enhancing theefficiencyof thespectrumauctionsand increasinggovernmentrevenue–as

theirpointofdeparture.Thisformedtheoutcomeofthedifferentcausalscenarios;theend,soto

say,oftheintermediatecausalprocessesthatareassumedtooperatebetweentheintervention

anditseffectuponthetargetvariable.Inthissense,causalscenariosaddressthequestion‘what

happens tooutcomeY ifwedo intervention I on variableX in target contextT?’Thepossible

answerstothisquestioncanbenumerousandcomplex,buttheyareguidedbythespecificaims

oftheintervention.

ToillustratehowmechanismswereusedascausalscenariosduringthedesignoftheFCC

auctions, let us return to the example of open and closed auctions in the presence of

complementarities.Asexplainedearlier,policymakerscouldnotrelyontheoryalonetoguide

theirdecisiononthematter.Toproceed,theyconductedarangeofexperimentsthattestedthe

effectsofcomplementaritiesuponthebiddingbehaviourofparticipants.Theproblem,however,

wasthattheexperimentershadnoinitialideaabouthowthisshouldbedone:“Howthis[winner’s

curse]mightworkoutwhenthere[are]complementarities…issimplyunknown.Noexperiments

havebeenconductedthatprovideanassessmentofwhatthedimensionsoftheproblemmight

be”(Plott,1997:626).

Withoutanyleadsonwhattolookfor,experimenterssetupdifferentcausalscenariosthat

showedhowthepresenceofcomplementaritiesmightaffectbiddingbehaviourand,consequently,

government revenue. One such scenario is presented in figure 7, which shows how

complementaritiescouldleadtoareductioninthewinner’scurseviaatwo-roundsprocedure:

licenses are initially auctioned in packages and later on an individual basis. This would,

presumably, result in an efficient auction design because licenses are allocated to the highest

bidderineitherthefirstorthesecondround,dependingonwhetherthelicenseisworthmoreto

thebuyeronanindividualbasisoraspartofapackage.Despitethefactthatnotheoreticalor

experimentalevidenceexistedforthisscenarioinvolvingcomplementarities,policymakerswere

stillabletoacquirearudimentaryunderstandingofhowtheintroductionofcomplementarities

couldaffecttheiraimsbyconstructingthesekindsofcausalscenarios.Thisparticularscenario

showed,forinstance,thatgovernmentrevenuewillbehigherwhenbiddersvaluetheaggregation

of licenses more than licenses sold individually. Whether these causal relationships actually

existedinthecontextoftheFCCauctionshadtobestudiedempirically,butpolicymakersatleast

hadaroughideaabouttheoperationofauctionsthatincludedcomplementarities.

46

Arguably,policymakerscouldhavecomeupwithawholelistofdifferentscenarios,each

portraying some plausible chain of causal processes between the introduction of

complementarities and government revenue. Interpreted as a causal scenario, a mechanism

shouldbeseenasarudimentaryrepresentationofthevariablesandbackgroundconditionsthat

couldbe relevant to theeffectivenessofan intervention; itdoesnotdeterminewhichof these

actuallyarerelevant.Theactualrelevanceofevidenceforanintervention’seffectivenesswillhave

to be established empirically. Thus, the usefulness of mechanisms in establishing evidential

relevanceislimitedinthesensethattheyprimarilyplayaroleintheearlystagesofthepolicy-

makingprocess,wherepolicymakersknowlittletonothingaboutthephenomenaofinterest.

2.4.3. CAUSALSCENARIOSANDMODULARITY

Theuseofmechanismsinpolicycontextsbywayofconstructingcausalscenariosdiffersfromits

use in extrapolating some piece of evidence from an artificial environment to the realworld.

Mechanism-based extrapolation and mechanism design, as put forward by Guala (2005) and

criticisedbySteel(2008),concernsknowledgeaboutcausalrelationshipsthatpurportedlyexist

intherealworld.Byinvokingmechanism,policymakerstrytospecifythenecessarybackground

conditions so as to make the extrapolation succeed. In addition, policy makers modify the

mechanisms–i.e.thetheory,experimentalresultsandbackgroundconditionsexhibitedbythe

differentmechanisms–inordertodesignanewmechanismthatworksbestinaspecificpolicy

context.Thisparticularmechanismislikelytonothaveexistedbeforesinceitwascreatedtosuit

thespecificpurposesofpolicymakers,asthefinaldesignoftheFCCauctionsaptlyshows.

Mechanisms as causal scenarios, on the otherhand, arenot necessarily basedon real-

worldentitieswithsupposedcausaltendencies.Themechanismsdoneedtobeplausibletosome

extent,accordingtoabasictheoryorwidely-heldopinionamongpoliticiansandthepublicatlarge.

Forexample,thescenarioinwhichpackagebiddingwouldprecedeindividualbidding(seefigure

7),whichwouldresultinhighergovernmentrevenue,isbasedontheprincipleof‘thewholebeing

Figure7:Acausalscenariobetweentheintroductionofcomplementaritiesandgovernmentrevenue:policymakersintervenebyincorporatingtheexistenceofcomplementaritieswithintheauctiondesign.Presumably,auctioningthelicensesaspackagesfirstwouldresultinhighergovernmentrevenuebecausebiddersaresupposedtovaluetheaggregationoflicensesmorethanlicensessoldindividually.

47

greater than the sum of its parts’: the aggregation of licenses will be more advantageous to

participantsthantheacquisitionofindividuallicenses.Thisprinciple,itturnedout,provedtobe

ausefulstartingpointforpolicymakersinmakingsenseoftheeffectofcomplementaritiesupon

biddingbehaviour.

So,whatdoesthisalternativeuseofmechanismsmeanfortheproblemofexternalvalidity?

Sincethemovefromefficacytoeffectivenessisreversed,asCartwright(2009b)advocates,the

issuewhethermechanisms are able to support extrapolation appears tobe lesspressing. The

reasonisthatsomeotherkindofevidence,whichmightnotbeefficaciousatall,couldbemore

relevanttotheeffectivenessofanintervention.Forexample,ithasbeenproved,boththeoretically

and empirically, that open auctions tend to reduce the winner’s curse. Yet this particular

efficacious result by itself is not very relevant for the FCC because it cannot account for the

presenceofcomplementarities.Indeed,evidencethatdescribestheeffectofcomplementarities

upongovernmentrevenue,at leastinsomeplausibleway, ispresumablymoreusefultopolicy

makers than a confirmed causal tendency in isolation.36Although evidence for efficacy canbe

important,“efficacyisonlyonesmallpieceofonekindofevidence(ibid.:133).

Now, focusing on the effectiveness of interventions instead, let us turn to the issue of

modularity.AccordingtoCartwright:

“Modularity thus secures a sure connection between causality and predictability under

manipulation[intervention].Buthowsatisfyingisthisconnection?Infact,itwillshowusto

useagivencausal relation forvery fewpolicymanipulations.That isbecause thekindof

manipulationsunderwhichitguaranteesinvariance–andhencepredictabilityfromthelaws

ofthesystem–areveryspecial.Theyarejustthekindsof‘surgicalincisions’thatwewould

demandinacontrolledexperiment,andtheseareveryunlikerealpolicychanges.”(2009a:

414)

Thenotionofa“surgical incision”relatestoourearlierdiscussionaboutthestructure-altering

natureofinterventions,whichremainsproblematicinmanypolicycontexts.Amajorworryisthat

interventionswill eliminate the causal relationship policymakerswish to exploit; due to the

intervention,therelationshipsarelikelytobecomespuriousormerelyreflectcorrelations.Infact,

“whenwedomanipulatesomefactor[whichisquestionableinitself]wegenerallyfindourselves

changingfarmorethanthatsinglefactoranditsdirectconsequences.Weusuallyendupchanging

36Ofcourse,evidenceforefficacymaybecomeusefulinmoreadvancedstagesofthepolicy-makingprocess.Supposingthereexistsagenuinecausalrelationshipbetweencomplementaritiesandgovernmentrevenue,policymakerscanthenclearlyusethisevidencetosupporttheirauctiondesign.Whenthiskindofevidencefor efficacy is lacking, as is often the case in policy contexts, they can, initially, only rely on the causalscenariosthatlinkcomplementaritieswithgovernmentrevenueinsomeplausiblemanner.

48

anumberofotherfactorsrelevanttotheeffectandveryoftenchangetheveryprinciplesbywhich

thesefactorsoperateaswell”(ibid.:415,italicsadded).Asaresult,policymakersoftencannotbe

surewhethertheirinterventionswillbeeffective.

2.4.4. IT’SALLABOUTSTABILITY

Interpretingmechanismsascausalscenarioscannotcompletelyresolvetheissueofmodularity,

for no matter what kind of evidence policy makers intend to use, there has to be a stable

connectionbetweenthepolicyandtargetvariable.Thisdoesnotmean,however,thateverystable

connectionhastobeofacausalnature.Putdifferently,thesuppositionthatcausalknowledgeis

inherently better than knowledge of spurious relationships or correlations is dubious from a

policypointofview.Toseewhyspuriousrelationshipsorcorrelationsmightalsobeusefulfor

policymakers,consideranabstractmechanismlikethatoffigure8.Ifweassumearelationship

tobecausal,then,accordingtothemodularityrequirement,thereisalwayssomeintervention

that leaves the relationship unaltered. This is then equally true for a spurious relationship

betweenthejointeffectsofcommoncause,foraninterventionupontheonewillleavetheother

unchanged: interveningon𝛼 will affect𝓍 via𝓏 ,while also affecting𝛾and leaving the spurious

relationship between𝓏 and 𝛾 intact (Cartwright, 2009a). Therefore, interventions based on

spurious relationships (or correlations)may be just as effective as those based on genuinely

causalrelationships,ifnotmore.

AccordingtoJulianReiss,“whatmattersforpolicyisthestableassociationbetweenthe

policy variable and the target, not the reasonwhy the correlation is stable” (2008: 163–164,

originalitalics).Thereisnothingwrongwiththefactthatthecommoncause𝓏contributesto𝛾

fromapolicypointofview.Ofcourse,itwouldbeamistaketosaythat𝓏explainstheoccurrence

Figure8:Anabstractmechanismthatshowshowaninterventionuponaspuriousrelationship(𝛼affecting𝓍via𝓏)willalsoaffectyandleavethislastrelationshipintact.Effectiveinterventions,therefore,neednotbebasedoncausalrelationshipsperse.

49

of𝛾but,asdiscussedinchapterone,policymakersdonotcaretoomuchforexplanationanyway.

What they do care about is how policy variables can be arranged so that they affect a target

variable in a stable and reliable manner. So, whether the effectiveness of an intervention is

broughtaboutbycausalornon-causal(spuriousrelationshipsorcorrelations)relationships is

not too important. Both types of relationships can be seen as relevant evidence for policy

hypothesesas longas theyare stable.FollowingCartwrightandReiss, it seems that theuseof

mechanisms as causal scenarios is not necessarily constrained to knowledge of causal

relationships,butmayalsoincludenon-causalknowledge.

With respect to the FCC auctions, a number of interventions were actually based on

mechanisms that partly exhibited non-causal relationships. The introduction of

complementarities,forexample,leadtothebreakdownofthecausalrelationshipbetweenopen

auctionsandthewinner’scurse.Sincepolicymakershadnoevidenceastohowopenauctions

wouldaffectgovernmentrevenueinthepresenceofcomplementarities,theyinitiallyhadtorely

onscenariosthatshowedhowcomplementaritiesmightaffectgovernmentrevenue(suchasin

figure7).Thisparticularmechanismwasnotconstructedbywayofcausalrelationshipsperse,

forwhether causality really existed had to be determined empirically.Without the benefit of

hindsight,policymakersreasonedthatcomplementaritiescouldplausiblyincreasegovernment

revenueaccordingtotheprincipleofsynergy.Asaresult,thismechanismdidnotincludegenuine

causalrelationships;infact,thiswouldhavebeenunlikelyduetoitshypotheticalnature.37

To close our discussion of evidential relevance with respect to the FCC auctions, let me

reformulate theuseofmechanismsascausalscenarios: theconstructionof (causal)scenarios,

basedonsomeplausibletheoryorprinciple,providedapreliminaryunderstandingofthe(causal)

relationshipsrelevanttotheaimofthespectrumauctions.Again,thescenarioswereusedasa

tooltoindicatewhichevidencecouldberelevant;theactualcredibilityoftheevidencewithinthe

contextofthespectrumauctionshadtobecheckedafterwardsusingempiricalmethods.While

some of the scenariosmay have included genuine causal relationships, others clearly did not

(figure7).Sincetheeffectivenessofaninterventioninaspecificenvironment–asopposedtothe

evidence for efficacy elsewhere – counts as the most important criteria for policy makers, a

scenariothatreflectsspuriousrelationshipsormerecorrelationsbetweenvariablescouldbejust

asusefulasapurelycausalscenario.

Althoughmechanisms, interpreted as (causal) scenarios, included non-causal features,

thesespuriousrelationshipsandcorrelationsstillhadtobestableinordertobeofrealuseto

37 Other plausible scenarios with respect to the relationship between open auctions and governmentrevenuemayincludecausalelementsbut,ingeneral,theydonothaveto.Generallyspeaking,thenotionof‘scenario’impliesthatthevariablesneednotbecausallyconnectedbecausetheircausalityisapossibilitytobeconfirmedempirically,notagivenapriori.

50

policymakers.Inshort:whiletherelationshipsincorporatedintothescenariosneednotbecausal,

stability is nonetheless a necessary condition for the effectiveness of interventions. It is

imperative,then,thatpolicymakersareawareofthisconditionand,wheneverpossible,tryto

accountforviolationsofmodularityfurtheroninthepolicy-makingprocess.

51

CHAPTERTHREE

MECHANISMSANDBEHAVIOURALPOLICY

Thepreviouschapterdiscussedtheuseofmechanismsinthedomainofauctionpolicy.Tofurther

substantiatetheargumentsinfavourofusingmechanismsforpolicy-makingthischapterwilldeal

with another policy domain, namely the domain of behavioural policy. More specifically, the

potentialofmechanismswillbeassessedwithrespecttointerventionsthatarebasedoninsights

frombehaviouraleconomics.Aswithauctionpolicy,thiswillbedonebywayofacasestudy:the

SaveMoreTomorrowTM(SMT)pensionplandevelopedbyRichardThalerandShlomoBenartzi

(2004,2013).Thisparticularinterventionshouldbeseenasanexampleofnudging,whichisa

relativelyrecentmovementwithinthebehaviouralpolicydomainthattriestoinducepeopleto

makebetterchoicesbychangingthechoicearchitecturetheyface.Thejustificationofbehavioural

policies,includingtheSMTpensionplan,remainsrathercontroversial.Therefore,investigating

theroleofmechanismswillprovehelpfulinthisregard.

SimilartotheFCCauctions,theusefulnessofmechanismswithrespecttotheSMTpension

planwillbediscussedfromtheperspectivesofexternalvalidityandevidentialrelevance.Forthis

purpose, Iwill largelydrawon theworkofTillGrüne-Yanoff (2015)whoadvocates, amongst

other things, thatknowledgeofmechanisms isnecessary for the justificationof interventions.

With regard to external validity, I will argue that whether interventions are judged efficient

dependsonwhichmechanismisusedbypolicymakers.Whileusingonemechanismtoexplain

howaninterventionoperatesmayresultintheviolationofmodularity–whichwouldrenderthe

interventionineffective–usinganothermechanismmightresultinareliableextrapolation.Asfor

evidential relevance, the evaluation of different mechanisms, interpreted as causal scenarios,

supports the identification of the relevant evidence for policy hypotheses. Although there are

oftenmanymechanismsavailable,policymakersnonethelesshavetosearchfortheminorderto

justifytheirinterventions.

3.1. SAVEMORETOMORROWTMSince the1980’s,principles frompsychologyhave started topenetrateeconomics.Apart from

developingnormativetheories,suchasrationalchoicetheory(RCT)38,economicsnowalsodeals

38Nottobeconfusedwitharandomisedcontrolledtrial,whichisalsoabbreviatedas‘RCT’.Throughoutthischapter,theabbreviation‘RCT’willrefertorationalchoicetheory–ifnotindicatedotherwise.

52

with descriptive and prescriptive theories. 39 Descriptive theories try to model the actual

behaviourofpeople,emphasizingtheirfrequentdeparturefromnormativetheories.Forexample,

thecontributionstoprospecttheory(Kahneman&Tversky,1979)andregrettheory(Loomes&

Sugden,1982)haveexposedtheanomaliesofpeople’sbehaviourwithregardtoRCT.Prescriptive

theories,then,areattemptstoimproveindividual’sdecision-makingandbringtheirbehaviour

closertothenormativeideal.TheSMTpensionplanisaforemostexampleoftheapplicationof

prescriptivetheory,wherepolicymakersandpolicy-orientedeconomistshavetriedtoincrease

savingscontributionratesamongstemployees.Theirbasicideawas“togiveworkerstheoption

ofcommittingthemselvesnowtoincreasingtheirsavingsratelater,eachtimetheygetaraise”

(Thaler&Benartzi,2004:166).

Theraisond’êtreoftheSMTpensionplanwasthetroublingobservationthathouseholds

typicallyfailtosaveenoughforretirement.Moreprecisely,householdstendtosavelessthanthe

life cycle theory of saving would suggest, which assumes people to optimize their expected

consumption level – i.e. they intend to smooth consumption over the course of their lives.

Admittedly,thistaskisadifficultone,evenforthosewhohaveaneconomicsbackground.Many

peoplealsolacktheself-controltocommittoahighersavingscontributionrate.Athirdissueis

procrastination,whichresultsinastatus-quobias:oncepeopleareenrolledintoacertainpension

plantheyarelikelytoremaininitforaconsiderableperiodoftimeduetoinertiaornaivety.

Toovercome theseproblems,Thaler andBenartzi (2004, 2013) included four rules in

theirdesignoftheSMTpensionplan:

1. Earlysign-upcall.Employeesareapproachedbycompanypersonnelaboutsigningupfor

theSMTplanwayaheadoftheirfirstscheduledpayrise.Sinceemployeestendtovalue

increased saving in the future more than higher savings contributions rates now

(hyperbolicdiscounting), thetime lagbetweenthesign-update(incasetheydecideto

signup) and the firstpay rise shouldbe as longaspossible.Thisway, employeeswill

perceivetheplantobemoreattractivebecauseitsconsequencesarenotimmediate.

2. Automatic escalation of the savings contribution rate. The savings contribution rate of

employees is increasedaftereachpayrise.Accordingtothebehaviouralmechanismof

loss aversion, people have the tendency to weigh losses more heavily than gains.

Consequently,employeesarereluctanttoincreasetheirsavingscontributionratebecause

theyperceivethisasalossindisposableincome.Increasingthesavingscontributionrate

39Thedistinctionbetweennormative,descriptive,andprescriptivetheorieshasbeenproposedbyHowardRaiffa(1982).

53

just after every pay rise mitigates the effect of loss aversion since employees’ higher

incomecompensatesfortheirincreasedamountofsavings.

3. Cap on maximum contribution rate. The savings contribution rate of employees is

graduallyincreaseduntilitreachesapre-setmaximum.Thisway,policymakersexploit

the states-quo bias of employees, which acts as a pull factor in keeping participants

enrolledintheplan.

4. Possibilitytoopt-out.KnowingthattheycanalwaysquittheSMTplan,employeeswillbe

more comfortable about singing up. In effect, this option increases the chance that

employeeswillactuallyenrolandstartaccumulatingmoresavings.

So,didtheSMTpensionplanactuallysucceedinitsobjectiveofincreasingtotalretirement

savingsamongstemployees?ThefirstimplementationataUSmid-sizemanufacturingcompany

generatedpromisingresults:78%oftheemployeesthatwereofferedtheSMTplandecidedto

signup,andsubsequentlyincreasedtheirsavingscontributionratefrom3.5%to13.6%inabout

fouryears(Thaler&Benartzi,2004).Sincethen,theadoptionoftheSMTpensionplan–along

with similar programmes using automatic enrolment 40 and automatic escalation rules – has

grownrapidly.ThalerandBenartzi(2013)reportthat56%ofemployersautomaticallyenroltheir

employees in a pension plan and, more significantly, 51% of them do so using an automatic

escalationscheme.Basedonconservativeassumptions,estimationsshowthattheSMTtypeof

pensionplanboostedannualsavingsby$7.4billionsinceitsinception.

Despite its resounding success, the SMTpensionplanwas not designed on theoretical

groundsalone.Althoughimportantinsightsfrombehaviouraleconomicswereavailable,thesedid

notindicatetheidealleveloftheparameterstobeincludedinthedesign.Forinstance,whileloss

aversionsuggestedthatanincreaseinthesavingscontributionrateshouldshortlyfollowapay

rise, itdidnot tellpolicymakersexactlyhowhigh the increasemustbe. Inaddition, then, the

group of behavioural economists, financial consultants and policy makers had to rely on the

specificbackgroundconditionsoftheenvironmentinwhichtheplanwastobeimplemented.With

respecttotheUSmanufacturingcompany,theseincludedthepreferencesofbluecollarworkers

inreceivingadvicefromafinancialconsultant.Whentheywereconfrontedwiththeconsultant’s

recommendation straight away theywouldbecomeunresponsive; only afterhavingdiscussed

theirfinancialsituationcouldanappropriaterateincreasebedetermined.TheinitialSMTplan,

40BasedontheoriginalSMTplan,somepensionplansincludeautomaticenrolmentwhereemployeesareautomaticallysignedupunlesstheydeliberatelydecidetooptout.Thisfurtherincreasesparticipationand,inturn,contributionrates.

54

basedonbehaviouraltheoryandparticularbackgroundconditions,wasthentestedempirically

bywayofsubsequentimplementations.41Thus,thesuccessoftheSMTpensionplanlargelyrested

on the intricate combination of theoretical, experimental, and background knowledge – three

sourcesofknowledgethatare,aswehaveseeninthepreviouschapter,crucialtothedesignof

anyeffectiveintervention.

3.2. EXTERNALVALIDITY

Theproblemof external validity is a recurring issuewithin applied economics that cannotbe

understoodproperlybyreferringtojustonecasestudy.Thissection,therefore,hasthetaskof

analysinghowtheSMTpensionplanrelatestomechanism-basedextrapolation.Thatis,towhat

extent did the intervention of the SMTplanmade use ofmechanisms in order to improve its

effectiveness?With respect to this case, Iwill argue that its effectiveness depended onwhich

mechanismswereusedbypolicymakers.Whilemechanismsare,inprinciple,abletoidentifythe

necessarybackgroundconditionsforextrapolationtoworkwell,theymaynotalwayssucceedin

doing so. Moreover, whereas an intervention might violate modularity according to one

mechanism,itmaynotdosoaccordingtoanother.

3.2.1. BACKGROUNDCONDITIONS

AccordingtoSteel(2008),mechanism-basedextrapolationhastwomainrequirements.First,the

background conditions between the artificial (experimental or otherwise) and the target

environment need to be as similar as possible. Policy makers can only conduct effective

interventions if theypossesssufficientbackgroundknowledgeabouttheirtargetenvironment,

which is provided by inquiring upon differentmechanisms. Thisway, eachmechanismpartly

showshowan interventionthatworked inonecontextwillalsobeoperative inthecontextof

interest. Inotherwords,mechanisms indicatewhichbackground conditions arenecessary for

somepieceofevidence(forefficacy)totraveltootherpolicycontexts.

Theidentificationofthenecessarybackgroundconditionscanbe,ofcourse,adifficultand

time-consuming task for policy makers. More often than not there are numerous possible

mechanismsthatcouldbeusedtosupporttheeffectivenessofinterventions.AlthoughThalerand

Benartzi(2004)mentionacoupleofmechanismsthatcouldplayaroleinthesuccessoftheSMT

41 Thaler and Benartzi (2004) describe two other implementations of the SMT pension plan. Eachimplementationgeneratednewinsights,suchasthefactthat“linkingsavingsincreasestopayincreases,whiledesirable,maynotbeessential”(ibid.:179).

55

plan–specificallyprocrastinationand loss-aversion–therearearguablymanymorepotential

mechanisms to choose from. In this sense, “the problem is not that there are nomechanistic

modelsassociatedwiththesepolicyproposals[liketheSMTpensionplan],butratherthatthere

are toomany–and that there ishardlyeveranyevidenceprovided tochoosebetween them”

(Grüne-Yanoff,2015:5).

However,followingGuala(2005),theevaluationofmechanismsisguidedbywayofthe

explicitaimsofinterventions.Recallthatbeforepolicymakerssetouttostudythemechanisms,

wherebytheywishtoacquirethenecessarybackgroundconditions,theyfirstspecifytheaimof

the intervention in concrete terms. This helps to focus their attention uponmechanisms that

wouldpotentiallycontributetothepolicyobjectives,whiledisregardingthosewhopresumably

donot(directly)enhancetheaimathandormerelyproduceambiguousresults.

AsfortheSMTpensionplan,theobjectivewasstraightforward:toincreasetotalsavings

ofemployees.Heresomemechanisms,primafacie,appearedtobemorepromisingthanothers.

Those including behavioural insights such as procrastination and loss-aversion were clearly

worthinvestigatingbecausetheywerealreadypresentedbytheacademiccommunityaslikely

causes for theobservedbehaviourof lowsavings.Amechanism involvingprocrastination, for

instance,resultsinastatus-quobias,whichpositivelyaffectsemployeeenrolment.Inturn,higher

(and longer) enrolment may significantly contribute to the total savings by employees. Now,

whether this effect will actually materialise depends on the background conditions that the

mechanismassumestobeinplace.Thestatus-quobiasofemployeeswillonlybeobtainedifthe

theyreallyareinertwhenitcomestothinkingaboutsavingforretirement.Withoutthecondition

ofinertia,theinterventionwillbelesseffectivebecauseemployeesmaybemoreinclinedtoquit

the SMT plan at any given moment. Therefore, the inertia of employees turned out to be a

necessarybackgroundconditionforthesuccessoftheSMTpensionplan(seerule3oftheSMT

planinsection3.1).42

Incontrast,amechanisminvolvingtherecommendationeffect–thecausalprocesswhere

apolicymakerrecommendsapensionplantoemployeesthroughautomaticenrolment–would

notresultinastatus-quobias,andthusdoesnotassumeemployeestobeinert.Foriftheywere,

thentherecommendationeffectwouldbecomeobsolete.Accordingtothismechanism,employees

donotenrolinapensionplanbecausetheyaresimplyignorantaboutthewholeidea,butinstead

they judge it positively due to the policymaker’s implicit recommendation. In this sense, the

42Admittedly,thedistinctionbetweenamechanismanditsbackgroundconditionscanbequitevagueandopen-ended.Here, the background condition of inertia is supposed to be a part of the procrastination-mechanism in the sense that it enables the development of status-quo bias. Yet inertia could also be aseparatemechanismincludingsimilarpsychologicalprocessesbutinasomewhatdifferentorder.Evenifwewouldconsiderinertiatobeamechanismitself,Grüne-Yanoffnotesnoteverybodywouldagreeonthis–see,forinstance,BergandGigerenzer(2010).

56

background condition of inertia does not appear to be necessary for the SMT pension plan’s

effectiveness.

This example illustrates how mechanisms can identify the necessary background

conditions, but it far from proves that mechanisms will always do so. Whereas the

procrastination-mechanismisabletoshowwhytheconditionofinertiaisimportantforapension

planaimedat increasing total savings, the recommendation-mechanismgivespolicymakers a

reason to believe the opposite. How, then, do policy makers decide on whether inertia is a

necessarybackgroundconditionornot?Analogously,whichmechanismactuallyoperateshere,

procrastinationorrecommendation?

These kinds of questions go back to a central concern with respect to mechanism-based

extrapolation,namelythatknowingwhichmechanismoperatesinthetargetenvironmentwould

nolongerrequireextrapolation.Steelaptlycallsthisconcerntheextrapolator’scircle(2008:94–

96).Hearguesthattheappropriatenessofamechanismcanonlybedeterminedifonealready

knows that the extrapolation will be successful; since it is unclear whether extrapolation is

possibleatall,justifyingtheoperationofamechanismremainsproblematic.

Ifpolicymakersreallyonlycareabouttheeffectivenessofan intervention inaspecific

environment, then they might as well leave extrapolation for what it is and focus on simple

induction. When designing the SMT pension plan, for example, policy makers could put the

insightsfrombehaviouraleconomicsasideandinquireaboutthecausalprocesseswithrespectto

thesavingbehaviourofemployeesempirically.Especiallywhenmechanism-basedextrapolation

leads to contradictory results – as the example of the procrastination and recommendation

mechanismsshows–woulditnotbebettertorefrainfromtheenterprisealtogether?

Unsurprisingly, it would not. Despite the call for more inductive inference within

economics43,theproblemofexternalvalidity,andthustheneedforextrapolation,wouldnotgo

awaycompletely.Sincethereisoftenaconsiderabletimelagbetweenfieldexperimentsandthe

implementationtheresultsderivedfromtheseexperiments,thetargetenvironmentislikelyto

undergosignificantchangesintermsofdemographic,technological,andpoliticalconditions.For

policymakers,thisproblemisparticularlyseverebecausetheyoftendealwithrapidly-changing

environments(considerthehighturnoverratesofemployeesatlargecompanies,forinstance).

Perhaps, then, it iswrong forpolicymakers to look for that single, uniquemechanism

‘somewhere out there’which happens to satisfactorily extrapolate a theoretical/experimental

resulttotheenvironmentofinterest.Instead,Iclaimthatpolicymakershavemoretogainfrom

mechanismdesign,whichcanbeseenasaprocedurethatcombinestheneedforextrapolation

43See,forinstance,LevittandList(2009).

57

withmore inductivemethods. Ineffect,policymakersstudyanarrayofmechanismsand take

whattheyneedfromeachoftheminordertofurtherdeveloptheirintervention.Someofthese

mechanismsmay provemore useful than others but for an intervention to work well, many

differentmechanismsmustbestudied.Eventually,thisprocedureresultsinthedesignofanew

mechanism, one that has integrated themost useful aspects – including theory, experimental

results,andbackgroundconditions–ofallthemechanismsthathavebeenevaluatedbypolicy

makers.44

3.2.2. MODULARITY:BITINGTHEBULLET?

Thesecondrequirementofmechanism-basedextrapolationismodularity:aninterventionshould

onlyaffect thosepartsof themechanismforwhich itwas intended.AccordingtoSteel(2008),

interventionswillonlybeeffectiveiftheydonotalterthecausalstructureofamechanismused

forextrapolation. Ingeneral, structure-altering interventionscanbeproblematicbecause they

leadtoabreakdowninthecausalrelationshipspolicymakerswishtoexploit.Thediscussionof

theFCCauctionsinthepreviouschapterhasalreadyshownusthatmodularityremainsathorny

issue.Isthereanotherwayout?

Grüne-Yanoff (2015) claims that interventions may or may not violate modularity,

depending on which mechanism policy makers use. Some interventions can be considered

structure-altering because they do not switch off all other causal tendencies on the variable

intervenedupon.45In this situation, the intervention isbutoneof thecausal tendenciesactive

uponthepolicyvariable.Lackingtheassumptionofceterisparibus,interventionsmightchange

thecausalrelationshipsthatpurportedlyexistbetweenthepolicyandtargetvariable.Asaresult,

policy makers cannot know for sure whether their intervention actually contributes to the

preferredoutcome.

Toseeifinterventionsareindeedstructure-alteringinthesensedescribedabove,letus

return to the SMTpensionplan. The first rule of the SMTplanwas based on the behavioural

conceptofhyperbolicdiscounting,wherepeopletendtovaluelowerconsumptioninthefuture

(duetoanincreasingsavingscontributionrate)morethanlowerconsumptionnow.Accordingto

44Althoughtheapproachofeconomicengineeringlooksasifitbegsthequestion–usingmechanismstoconstruct mechanisms – but in fact it does not. For the approach does not use hypotheses about theexistenceofmechanismsaspremise in itsargument thatmechanismsexist. Instead,policymakers takethoseaspectsofthepurportedmechanisms(beitsomepieceoftheory,acertainexperimentalresult,orasalientbackgroundcondition)andmodifythemtofitthetargetenvironment.Theseaspects,then,togetherformanewmechanism,especiallydesignedfortheaimsoftheinterventionathand.45 This critique corresponds to the second condition of an ideal intervention in Woodward’s (2003)framework.Seesection1.3.2.

58

Grüne-Yanoff,therearetwopotentialmechanismsthatcouldindicatehowintroducingatemporal

difference between signing up to the pension plan and the first increase in the savings

contributionrateisdeemedeffective.Thefirstmechanismisrelatedtovisceralfactors–suchas

hunger,thirstorotherkindsofphysicalcravings–whichincreaseinstrengthastheirexpected

satisfactionnearsthepresent.As influencesonbehaviour, theycontrastwithpeople’srational

deliberationsintermsofself-interest.Whileemployeesmightwanttosavemoreforretirement

becausetheybelieveitwouldarationalthingtodo,theprospectofgivingupcurrentconsumption

isamplifiedthroughvisceralfactors.Consequently,accordingtothismechanism,employeesare

morelikelytoincreasetheirsavingscontributionrateonlyifpresentedwiththeoptionofdoing

sointhefuture.

The second mechanism proceeds from the distinction between certain and uncertain

outcomes.Ingeneral,peopletendtoprefertheformeroverthelatterduetothepossibilityofnot

gettingarewardthat isacquiredinthefuture.Employeesthatchoosetogiveupconsumption

nowformoresavingslateraretradingacertainoutcomeforanuncertainone,whichisachoice

thatnotmanypeoplewouldfindattractive.TheSMTpensionplan,however,effectivelymitigated

the influence of this distinction by changing the choice architecture faced by employees. As

opposed to increasing one’s savings contribution rate now, doing so in, say, sixmonths’ time

meansemployeesaretradingoneuncertainoutcomeforanother.Thiswouldmaketheprospect

ofahighersavingscontributionrateinthefuture(andthuslowerconsumption)relativelymore

attractivebecauseitdoesnotcompetewithacertainoutcome.

Nowlet’sseehowthesetwomechanismswouldfarewithrespecttotherequirementof

modularityinthecaseoftheSMTpensionplan.Judgingfromtheuncertainty-mechanism,thefirst

ruleoftheplanmayviolatemodularitybecauseitdoesnotblockallothercausalinfluenceson

uncertainty.Forexample,

“aclumsycommunicationandimplementationoftheSaveMoreTomorrowTMplanmightalso

create the impression that retirement plans are changed haphazardly, letting employees

revisetheiruncertaintyjudgmentsabouttheirabilitytoretrievefunds.Suchanincreasein

uncertainty, causedby the implementation,mightwell erase anypositive effects that the

policywouldotherwisehavehadoncontributionsrates.”(Grüne-Yanoff,2015:11)

AlthoughtheSMTplanislikelytoincreasetotalsavingsbyemployeesthroughthecleveruseof

differentialuncertaintyjudgments,itseffectivenessmaybeundoneiftheremaininginfluenceson

uncertainty – such as communicative flaws – are significant. If this is indeed the case, then

modularitywouldbeviolatedandtheusefulnessofthismechanismcouldbequestioned.

59

However,thesameinterventiondoesnotviolatemodularitywhenjudgedwiththehelpof

adifferentmechanism.Usingthemechanismofvisceralfactorsaslineofreasoninginstead,flaws

in the communication and implementation of the SMT plan will, ceteris paribus, have no

complicatingeffectonthesuccessoftheintervention.Herethetemporaldifferencebetweensign-

updateandstartdateoftheSMTplanisbasedontheideathatvisceralfactorswillbeweaker

(andemployees thereforemore rational)whenemployees are able todefer increases in their

savingscontributionrate.Giventhemechanismofvisceralfactors,whethertheSMTpensionplan

iscommunicatedandimplementedproperlydoesnotmattermuch,fordifferentialuncertainty

judgmentsarenotassumedtoplayasignificantrole.

Whendiscussinganintervention’spotentialviolationofmodularity,Grüne-Yanoff(2015)adds

twomorecomplicationstotheargumentthatitdependsonwhichmechanismpolicymakersuse.

Namely,evenifaninterventioninitiallydoesnotviolatemodularity,itmaydosoinalaterstage

orafterrepeatedimplementation.Withregardtothemechanismofvisceralfactors,forinstance,

thepositiveeffectoftemporaldifferencemightwearoffwhentheSMTplaniswellunderway:

employeesanticipate thebenefitsof theplanas theynearretirementage,whichmayenhance

theirdesireforhigherconsumptionnow(andthusalowersavingscontributionrate).Atsome

point,employees’positiveattitudetowardssaving inthe futureasopposedtosavingnowwill

becomeblurredbecausetheirperceptionsof‘thefuture’and‘now’graduallymergeintoone.46

Similarly, an intervention may violate modularity only after it has been repeatedly

implemented.Forexample,ThalerandBenartzi(2004)reportthatthelinkbetweencontribution

rateincreasesandpayrises–basedonthemechanismofloss-aversion(seerule3oftheSMTplan)

–appearednottobe“essential”(ibid.:179)afterthreeconsecutiveimplementations.Thatis,the

SMTplanbecamelesseffectivebecauseitsrepeatedimplementationhadstructurallyalteredthe

causalrelationshiprelatedtopeople’sperceptionofgainsandlosses.Theinterventionbasedon

themechanismofloss-aversion,therefore,hadresultedinbreakingdownthecausalrelationship

itintendedtoexploit.

These complications are valid and reveal further shortcomings of mechanism-based

extrapolation.Nointerventionbypolicymakerswillbeeffectiveindefinitely.Theenvironments

in which interventions take place are often highly complex, which increases their chances of

violatingmodularity.Dependingonwhichmechanismisusedbypolicymakers,interventionscan

46 In their analysis, Thaler and Benartzi (2004) find that the replacement ratio (the percentage of anemployee’sincomethatispaidoutbyapensionplanuponretirement)ofemployeeswhocommittoaSMT-typeofpensionplanearlyonaresignificantlyhigherthanforthosejoiningatanolderage.Quiteobviously,increasing one’s savings contribution rate is more attractive for younger employees. Once employeesbecomeolder, theymaydecidetodropoutof theplanbecause itsrelative impactontheirtotalsavingsbecomessmaller.Hence,theurgeofhavingahigherpresentconsumptionlevelbecomesstronger.

60

indeedbeconsideredmodular.Yeteven(initial)modularinterventionsmaybecomeineffective

overtimeorafterrepeatedimplementation.Theproblemisthatpolicymakersoftenhavevery

littleevidencetochoosebetweendifferentmechanismsonthebasisofmodularity.Thatis,itis

verydifficultforpolicymakerstojustifyaninterventionbyreferringtoaparticularmechanism

becausetheycannotknowforsurethatmodularitywillnotbeviolated.Theupshot,then,isthat

policy makers have to study many different mechanisms, and regularly check whether their

interventionsdonotviolatemodularity.

3.3. EVIDENTIALRELEVANCE

Althoughmechanism-basedextrapolationcan,inprinciple,beusedtoovercometheproblemof

externalvalidity,theissueoffindingtherightmechanism(ormechanisms)forthejobremains

pertinent.Grüne-Yanoff(2015)arguesthatinterventionscanbeconsideredeffective–i.e.they

identifythenecessarybackgroundconditionsanddonotviolatemodularity–dependingonwhich

mechanismspolicymakersuse.Giventhatthereareoftenmanypotentialmechanismstochoose

from,howdopolicymakersdecideuponwhichmechanismstousefortheirinterventions?Inthis

section,Iwilldiscussonepossibleapproachtothisproblem,namelythatofevidentialrelevance.

Morespecifically,Iwilldefendthefollowingtwoclaims.First,policymakersinthecaseoftheSMT

pensionplanadoptedtheperspectiveofevidentialrelevance–whatevidenceisrelevanttothe

policyhypothesis?Secondly,theyproceededinthistaskbyusingmechanismsascausalscenarios.

ThesescenariosindicatedwhichevidencewaslikelytoberelevantfortheaimsoftheSMTplan,

andwhichwasnot.Ineffect,mechanismsfunctionedasafilterforwelfarejudgmentsaboutthe

consequencesoftheSMTplan.

3.3.1. WELFARE

InhisdiscussionoftheSMTpensionplan,Grüne-Yanoffendorsesthedistinctionbetweenefficacy

andeffectivenessputforwardbyCartwright(2009c).ThepolicymakersinvolvedintheSMTplan

weremostly interested in theeffectiveness,or,moreprecisely, thebenefit thatwouldactually

occurwhen theplanwas implemented. Inparticular, theywanted toknowhow theSMTplan

wouldimpactthewelfareofemployeeswithrespecttosavings.Inthissense,asubjectivewelfare

criterion in terms of welfare-promoting decisions was used to evaluate the effectiveness of

differentpensionplans.Thiscriterionwas in linewiththegeneralaimofnudging,which is to

improvepeople’sdecision-makingbychangingthechoicearchitecturetheyface.

61

The aimof the SMTplanwas to increase total savings by employees. Aswith the FCC

auctions, this aim featured as the point of departure for policy makers. This meant that all

gathered evidence – a behaviouralmodel, some experimental result, or a certain background

condition–wasevaluatedaccordingtoitspotentialcontributiontoincreasingtotalsavings.While

someevidenceonlycontributedindirectly,suchasthebackgroundconditionofinertia,thiskind

of evidence was highly relevant nonetheless. Other evidence, however, proved to be far less

relevanttothisaim,likethelifecycletheoryofsaving.Eitherway,itwascrucialforpolicymakers

todeterminetherelevanceoftheevidencewithrespecttoincreasingretirementsavingsearlyon

inthepolicy-makingprocess.Whethertherelevantevidencealsoturnedouttobecrediblehadto

bedeterminedatalaterstage.

Recallthatevidenceforefficacy,thoughimportant,“isonlyonesmallpieceofonekindof

evidence” (Cartwright, 2009b: 133). From a policy point of view, knowing that a variable is

efficaciousinsomeexperimental(ideal)environmentisnice,butitisnotparticularlyhelpful.As

discussedinsection3.2.1,beingabletoextrapolateevidenceforefficacytothetargetenvironment

requires the specification of all the necessary background conditions as well as checking the

modularityofinterventions.Itwasshownthatmechanism-basedextrapolationcanbedonebut

thatitssuccessisbynomeansguaranteed.Byfocusinglessonevidenceforefficacyandmoreon

evidentialrelevance,policymakersareabletodevelopmoreeffectiveinterventionsbecauseeach

pieceofevidencethattheyplantouseissupposedtoberelevant,thoughindifferentdegrees,to

theaimathand.

Of course, evidence for efficacymight bepart of the relevant evidence for a particular

policyaim,too.Thebehaviouralconceptsofstatus-quobiasandlossaversion,forexample,were

bothrelevanttoandefficaciousinthetargetenvironment.Yetitisonlypossibleforpolicymakers

toknowthatanefficacyclaimisapplicabletothetargetenvironmentifitisalsorelevant.Inshort:

whileevidenceforefficacymayalsoberelevantevidence,apieceofrelevantevidenceneednot

beefficacious.Thatis,itisnotonlyevidenceforefficacythatcanberelevanttoapolicyhypothesis;

moreoftenthannotweneedadditionalevidence,whichdoesnothavetobeefficaciousperse.In

thecaseoftheSMTpensionplan,policymakersdidnotrestrictthemselvestoonlyusingevidence

forefficacy.Rather,theyincludedmanydifferentkindsofevidenceintheirplan–suchastheory,

experimentalresults,andbackgroundknowledgeaboutthetargetenvironment.Thus,whatmade

theSMTplansuccessfulwasnotmerelythestudyofevidenceforefficacy,buttheevaluationofall

therelevantevidence.

62

3.3.2. CAUSALSCENARIOSRECONSIDERED

Mechanisms, interpreted as causal scenarios, can be useful for policy makers in order to

determinetherelevanceofdifferentkindsofevidence.47GrüneYanoff(2015)distinguishesthree

aspectsofanintervention’seffectiveness:robustness,persistenceandwelfareeffects.Allthree

aspectsareclearlyrelatedtotheproblemofexternalvalidity,buttheycanalso–aswillbedone

below–betakenasastartingpointforthedevelopmentofinterventions.Sincethewelfareeffects

ofinterventionsmattergreatlyforanypolicy,thesefunctionasthemaincriterionforestablishing

evidentialrelevance.Giventhiswelfarecriterion,policymakersthenconstructavarietyofcausal

scenarios, where each scenario indicates how certain variables and background conditions

supposedlycontributetoimprovingwelfare.Inthisprocess,policymakersmaybasethecausal

scenariosonsomebasictheory,generalprinciple,orwidely-heldpublicopinion.Whilethelistof

plausiblescenariosispotentiallyendless,itisatleastguidedbythewelfare-improvementthatthe

interventionintendstobringabout.

ToillustratehowmechanismswereusedascausalscenariosduringthedevelopmentoftheSMT

pensionplan,letusreturntotheexampleofintroducingatemporaldifferencebetweenthesign-

updateandthefirstsavingscontributionrateincrease(rule1intheSMTplan).Asdiscussedin

section3.2.2,therearetwomechanismsthatcouldpotentiallybeusedtosafeguardmodularity:

themechanismrelatedtovisceralfactorsandtheuncertainty-mechanism.Itwasshownthat,in

thiscase, themechanismrelatedtovisceral factorsmadesure the interventiondidnotviolate

modularity.

Nowlet’sshiftourattentionawayfromissuesofrobustnessandpersistence,andfocuson

thepotentialwelfareeffectsofinterventions.NudgesliketheSMTpensionplanaresupposedto

improvethewelfareofemployeesbyhelpingthemavoidmanipulativeforces.Usingthevisceral

factorsmechanism, the temporal difference induces people to behave according to their self-

interests.AccordingtoGrüne-Yanoff,“ithelpsfreepeople’schoicesfromvisceralinfluences,and

insteadallowspeopletochoosesoastosatisfytheirconsistentandwell-informedpreferences”

(2015:17).Moreconcretely,employeesaremore likely tomakerationaldecisionsconcerning

savingforretirementwhentheyarenotexposedtovisceral(irrational)factors.Theirbehaviour

broughtaboutbytheSMTplanconstitutesawelfare improvementbecausetheyareno longer

manipulatedthroughshort-termthinking.

Incontrast,theSMTpensionplandoesnotconstituteawelfareimprovementaccordingto

theuncertainty-mechanism.Whenviewedfromthisperspective,onecouldarguethatpreferring

47Assuch,mechanismsthemselvesdonotactasevidence for theeffectivenessof interventions.Rather,theyareatoolforpolicymakerstosearchforevidenceandevaluateitsrelevanceinapreliminarymanner.

63

relatively less consumption now to more consumption in the future is compatible with the

rationalityofemployees.Thereasonisthatsoonerconsumptionisjudgedaslessuncertainthan

postponed consumption. Since it is perfectly rational to prefer a certain outcome over an

uncertain one, thewelfare of employeeswould not be enhanced by changing the option of a

certainoutcomeintooneofuncertainty.Consequently,thesamebehaviourbroughtaboutbythe

SMT plan would not constitute a welfare improvement because, under the uncertainty-

mechanism, it is assumed there is no manipulative force present. Whereas this particular

intervention is considered tobewelfare-improvingwith respect to themechanismof visceral

factors, the same intervention cannot be judged as such when viewed from the uncertainty-

mechanism.48

Thetwomechanisminthepreviousexamplecanbeseenascausalscenariosthatshowed

whetherandhowaninterventionactuallyaffectedwelfare.Thefirststepinconstructingcausal

scenarioswasforpolicymakerstospecifythepreferredoutcomethateveryscenariomustarrive

at.Thiswasdonebyreferringtothesubjectivewelfarecriterion,whichjudgedbehaviourtobe

welfare-improvingifbasedonrationalpreferences. Intheexampleabove,thecausalscenarios

were supposed to improve the decision-making of employees with respect to saving for

retirement.

From this, policymakers proceeded by setting up different causal scenarios including

variables andbackground conditions thatwouldpresumably contribute to improvingwelfare.

Thesecausalvariablesandbackgroundconditionswerederivedfromtheorieslooselyrelatedto

the policy context, such as RCT and other behavioural theories.With the help of some basic

theoretical insights,policymakerswereableto formulateconcrete interventions.Oneof these

interventionswastheintroductionofatemporaldifferencebetweenthesign-updateandthefirst

increase inthesavingscontributionrate,where itsgeneral ideawasbasedonthebehavioural

conceptofhyperbolicdiscounting.

Oncethepreferredoutcomeandthecausalscenarioshadbeenspecified,policymakers

evaluated whether the scenarios actually had the desired effect. In this case, only the causal

scenarioinvolvingvisceralfactorswouldconstituteawelfareimprovement.Thecausalscenario

ofuncertainty,thoughplausible,wasnotconsideredtoimprovewelfarebecauseitdidnotavoid

a behavioural manipulation. Given the specific aim of policy makers, the construction and

evaluation of causal scenarios enabled them to determine the relevance of the evidence in a

preliminarymanner:theruleofhavingatemporaldifferenceintheSMTplanwasrelevantfor

improvingwelfarewhenviewed fromthecausalscenario involvingvisceral factors,but itwas

48Itisimportanttoemphasizeherethat,wheninterpretingmechanismsascausalscenarios,policymakersarenotlookingforthatonemechanismactuallyoperatinginthetargetenvironment(aswithmechanism-based extrapolation). Rather, they evaluate several different mechanisms (scenarios) and then decidewhichonetouseforthejustificationoftheintervention.Moreonthisbelow.

64

muchlessrelevantwhenjudgedbythecausalscenariobasedonuncertainty.Thus,althoughthe

SMTplanappearedtobewelfare-improvinginonesense(visceralfactors),itssupposedrelevance

towelfarewasweakeneduponfurtherinquiry(uncertainty).

Ultimately, then, policy makers had to make a judgment call whether introducing a

temporaldifferenceactuallyconstitutedawelfare-improvementornot.Theirfinaljudgmentwas

supported by causal scenarios that indicated which kinds of evidence could be relevant to

improvingwelfare.Thesescenariosinformedpolicymakersaboutthepreliminaryrelevanceof

the evidence – be it theory, experimental results or background conditions. Yet,whether this

evidenceactuallywasrelevantremainedanopenquestionandhadtobedeterminedempirically.

Incaseof theSMTpensionplan,policymakersdecidedthat introducingatemporaldifference

wouldconstituteawelfareimprovementforemployees.Inasense,theyusedcausalscenariosas

afilterforjudgingthewelfareeffectsoftheirproposedintervention:thefirstruleoftheSMTplan

couldbejustifiedaccordingtothecausalscenarioinvolvingvisceralfactors.Usinganotherfilter,

however,wouldhaveledtotherejectionoftheinterventionsince,underthecausalscenariowith

regardtouncertainty,thetemporaldifferencewouldnothavebeenjustified.

3.3.3. JUSTIFYINGINTERVENTIONS

Thisbringsustothequestionofwhenaninterventionissufficientlyjustified.Grüne-Yanofftries

tomakesenseofthisdifficultissuebyproposingthefollowingsufficiencyprinciple:

“Apolicy[intervention] isbasedonsufficientmechanisticevidence if it takesallavailable

mechanisticevidence49intoaccount,whereavailabilityisconstrainedbycurrenttheoretical

andtechnologicalfeasibility.Ifinformationofthissortdoesnotenterthediscussionatall,

thesepoliciescannotandshouldnotbedescribedas‘evidence-based’.”(2015:18)

Mechanisms–interpretedascausalscenarios–can,ingeneral,beusedtojustifyanintervention.50

Ifthereisoneparticularmechanismthatindicatesaninterventiontobewelfare-improving,and

49Inhispaper,Grüne-Yanoffrefersto‘mechanisticevidence’whenhediscussestheroleofmechanismsinbehaviouralpolicy.Here,Iinterprettheuseofmechanisticevidencebypolicymakersasbeingsimilartothe construction and evaluation of different causal scenarios: just like there can be many potentialmechanismsthatpolicymakerscanuseforextrapolatingsomeresultfromanartificialenvironmenttotheoneofinterest,therecanalsobemanycausalscenariosthatcouldplausiblyindicatewhatevidencecouldberelevanttothepolicyaim.50Discussinganotherinterestingcasestudy,MichiruNagatsu(2015)convincinglydefendsthejustificationof nudgepolicies by referring tomechanisms.Althoughhedoesnot explicitly interpretmechanisms ascausalscenarios,anattemptcouldbemadetointegratehisargumentswiththeinterpretationpresentedinthisthesis.Forthesakeoffocus(andspace),Iwillnotdosohere.

65

there arenoothermechanisms that provide relevant evidence in theoppositedirection, then

policymakerscanbeconfidenttoinvokethismechanismforthejustificationoftheirintervention.

Attheveryleastthen,everyinterventionshouldbebasedonevidenceputforwardbysomekind

ofmechanism.

Yet,asheadmitsinthesamepassage,itisoftenverydifficulttoobtainsufficientevidence

thatfullyjustifiesaninterventionbyinvokingmechanisms.Unfortunately,policymakershaveto

do with partial evidence in favour of a particular intervention, where mechanisms can only

provide “qualitative information regarding factors upon which the policy’s effectiveness and

welfare-propertiesislikelytodepend”(ibid:18).Eveninsituationswheremechanismsindicate

evidencenottoberelevanttothepolicyaim,itwillbeusefulforpolicymakerstobeawareofthe

reasons why an intervention might not be justified. For these reasons will, in themselves,

contributetothejustificationoftheintervention(albeitinanegativeway).

In order to enhance the justification of interventions, mechanisms need to be further

differentiated, accounting for different theories, empirical data and background conditions.

AccordingtoCartwrightandHardie(2012),“todecideabout[the]effectiveness[ofinterventions]

requiresanopen-endedprocessofthinkingthatisinevitablycontextualandcannotbereduced

torules”(ibid.:11).Thisinducesacertainmodestyamongstpolicymakersgiventhattheycan

hardlyever completely justify their interventions, as theSMTpensionplanaptly illustrates. If

nothing else, mechanisms caution policy makers against the premature implementation of

proposed interventions. Thus, the process of establishing evidential relevance by invoking

mechanismsmightyieldnoinitialresult,maybeverycostly,orcouldtakeconsiderabletime.But

searchonemust.

66

CONCLUSION

Inthisthesis,Ihavedefendedtheclaimthatmechanismsareausefultoolforeconomicpolicy-

making. More specifically, the potential of mechanisms for the function of control has been

assessed according to two main arguments: external validity and evidential relevance. To

conclude this thesis, Iwill restatemy claimswith respect to these two arguments andbriefly

summarizethemostimportantfindingsobtainedthroughthediscussionoftheprevioustwocase

studies.Finally,somesuggestionsforfutureresearchwillbeprovided.

Theproblemofexternalvalidityisapressingandrecurringissuewithinappliedeconomics.More

often than not, theoretical and experimental results do not hold outside the artificial

environments in which they were originally obtained. This is problematic for policy makers

becausetheyneedtoknowwhethercertaincausalrelationshipsfoundinonecontextwillalso

operate in a target context. Put differently, if policy makers want to conduct effective

interventions,thentheyhavetobeabletorelyoncausalrelationshipsthatcantotraveltoother

environments, too. One supposed solution to this problem ismechanism-based extrapolation,

where a mechanism is used to specify the similarity in background conditions between the

artificial and the target environment inorder to successfully extrapolate a causal relationship

fromtheformertothelatter.Thisway,policymakerswillknowthataresultobtained‘there’will

alsohold‘here’,forthemechanismindicateswhetherallthenecessarybackgroundconditionsare

inplace.

Withrespecttomechanism-basedextrapolation,Ihavearguedthatmechanismsplayan

importantroleinthedesignandimplementationofpolicyinterventions.FollowingGuala(2005),

themainadvantageofusingmechanismsforpolicypurposesistheabilitytointegratetheoretical,

experimental,andbackgroundknowledgeintoconcrete,effectiveinterventions.Thatiswhyhe

speaksof ‘mechanismdesign’:policymakerstrytodesignamechanismthatworksbest inthe

targetcontextbyfirstspecifyingtheaimoftheintervention,andthenstudyingmanydifferent

mechanismswhichareabletocontributetothisaim.Inthisprocedure,onlythemostusefulparts

ofthetheory,experimentsandbackgroundconditionsareincludedintothefinaldesign.TheFCC

auctions have aptly illustrated this procedure, inwhich policymakerswere able to integrate

insights from game theory and experimental (testbed) results with the specific background

conditionsofthespectrumauctions.

Using mechanism-based extrapolation for policy-making should be seen a kind of

economicengineering:policymakersmodifymechanismsthatpurportedlyexistintherealworld

so as tomake their interventionswork effectively. In this sense, policymakers do not try to

67

extrapolateoneparticularcausalrelationshipbyinvokingitsunderlyingmechanism,butinstead

combine knowledge of many different mechanisms related to the phenomenon under

investigation.Theystudyanarrayofmechanismsandtakewhattheyneedfromthem,sotosay,

inordertofurtherdeveloptheirintervention.

The testbed experiments used in the FCC auctions are a good example of what this

procedurelookslikeinpractice.Heretheexperimentsmimickedthebackgroundconditionsof

theactualspectrumauctionsascloselyaspossible,whichenabledpolicymakerstocomparethe

resultsobtainedintheartificialenvironmentwiththeiroperationinthetargetenvironmentina

straightforwardmanner.Ineffect,policymakersfirstmovefromtherealworldtothelaboratory

and then back to the field again. Since extrapolating some causal relationship found under

artificial conditions to the real world is often very difficult, policy makers can benefit from

importing the most important empirical features of the phenomenon of interest into the

experimental setup. Eventually, this procedure leads to the development of a new type of

mechanism,whichisspecificallycreatedtoenhancetheaimsofpolicymakers.

Thereis,however,oneparticularlyimportantobjectionagainstmechanism-basedextrapolation

totakeintoaccount.Namely,interventionsthatdrawonknowledgeofmechanismscanalterthe

structureof thecausal relationshipspolicymakerswish toexploit.On this, Ihaveargued that

mechanism design, though susceptible to structure-altering interventions, can be a suitable

extensionofmechanism-basedextrapolation,andshouldthusbetakenseriouslybytheeconomic

policy-makingcommunity.Althoughinterventionsthatadheretothedemandsofmodularityare

veryhardtocomebyinpolicycontexts,theFCCauctionshaveprovenitispossible:theextensive

useofexperimentsplayedacrucialroleinthisregard.

Admittedly,theprocedureofmechanismdesignmaybeverycostlyandtime-consuming.

AmorefruitfulwaytodealwiththeissueofmodularityisproposedbyGrüne-Yanoff(2015),who

claimsinterventionsmayormaynotviolatemodularitydependingonwhichmechanismpolicy

makersuse.TheSMTpensionplan,forinstance,canbeconsideredstructure-alteringaccording

to the uncertainty-mechanism, while the same conclusion does not hold when using the

mechanismofvisceralfactors.Ifpolicymakersmanagetoidentifyamechanismthroughwhich

theirinterventiondoesnotviolatemodularity,thenthejustificationofthatinterventioncanbe

enhanced.Yettocomplicatemattersevenmore,interventionsmightalsoviolatemodularityina

later stage or after repeated implementation. Therefore, the only way to make sure that an

intervention does not violate modularity is for policy makers to evaluate many different

mechanismsandtodosoregularly.

68

The potential of mechanisms for policy purposes can be understood differently, and more

appropriately,intermsofevidentialrelevance.Thatis,mechanismshavetheabilitytoprovidea

preliminaryunderstandingoftheevidencethatcouldberelevantfortheeffectivenessofpolicy

interventions.Whereasthetheoristorexperimentertriestoanswerthequestion‘wherearemy

obtainedresultsrelevant?’,thepolicymakerisprimarilyconcernedwiththequestion‘whatkind

ofevidenceisrelevanttomypolicyhypothesis?’.Sotheconventionalperspectiveofmovingfrom

efficacytoeffectivenessisreversedwithinthedomainsofevidence-basedpolicy.

With respect to evidential relevance, I have argued that policy makers adopt this

alternative perspective since they take the aims of interventions as their point of departure.

Evidence for efficacy, established by experimental methods such as RCTs, can be valuable to

scientistsbutthiskindofevidenceisnotnecessarilyrelevanttopolicymakers.Inprinciple,any

kind of evidence can be useful for policymakers as long as it is relevant to the aims of their

interventions;beforeevidenceisdeemedcredible(i.e.efficacious)inthetargetenvironmentit

mustbeconsideredrelevantfirst.BothinthecaseoftheFCCauctionsaswiththeSMTpension

plandidpolicymakersfocusontherelevanceoftheevidencefortheinterventions’objectives,

where government revenue was a focal point in the former and total retirement savings of

employeesinthelatter.Whilesomeevidenceprovedtobehighlyrelevanttotheaimsofthese

particularinterventions,otherevidenceturnedoutbefarlessrelevant.Eitherway,establishing

therelevanceofevidencewasanecessaryfirststepinthedevelopmentofeffectiveinterventions.

As I have subsequently argued, policy makers can inquire upon evidential relevance

through the construction and evaluation of causal scenarios. Interpreted in this sense,

mechanismsprovideapreliminaryunderstandingofthecausalrelationshipsthatarelikelytobe

relevantforagivenpolicyhypothesis.Eachscenariostartswithaproposedinterventionandends

with the desired outcome, showing the intermediate causal processes along the way. Policy

makers draw on different kinds of theoretical, experimental, and background knowledge in

constructingthesecausalscenarios.Everyscenariohastobeplausibleaccordingtosomebasic

theory,generalprinciple,orcommonlyheldopinion.Forinstance,thepackagebiddingprocedure

during theFCCauctionswas initiallybasedon theprincipleof synergy. Similarly,behavioural

insightssuchashyperbolicdiscountingwereusedasafoundationfortheconstructionofcausal

scenarioswithrespecttotheSMTpensionplan.

Interestingly, the interpretation of mechanisms as causal scenarios has important

consequences for the problem of external validity. Given that the efficacy–effectiveness

perspective is reversed, the task of extrapolating some efficacious result from one context to

another becomes less pressing from a policy point of view. As for the issue of modularity,

mechanisms do not necessarily have to reflect genuinely causal relationships; they may also

reflectspuriousrelationshipsormerecorrelations.However,interpretingmechanismsascausal

69

scenarioscannotcompletelyresolvetheissueofmodularity,fornomatterwhatkindofevidence

policymakers intendtouse, therehastobeastableconnectionbetweenthepolicyandtarget

variable. If there is no stability, then policy makers cannot know for sure whether their

interventionisactuallyeffective.Thus,theconstructionofscenariosrequiresstablerelationships

thatneednotbecausal.

Whenconstructing(causal)scenarios,policymakersareleftwiththequestionastowhich

evidence actually is relevant (and credible) for their policy objectives. There are oftenmany

plausiblescenariosavailablethatcouldjustifyanintervention.Forthisreason,itisverydifficult

forpolicymakerstosufficiently justifyan interventionby invokingmechanisms.Nevertheless,

evenifamechanismindicatesaparticularpieceofevidencenottoberelevantfortheaimathand,

itwillbeuseful forpolicymakers tobeawareofreasonswhytheir interventionmightnotbe

justified.Thisway,policymakersarecautionedagainsttheprematureimplementationoftheir

interventions. Ultimately, then, they have tomake a judgment call with respect to the actual

relevance of the evidence presented. This kind of judgement can be supported by the further

differentiationofmechanisms,orsoIhaveargued.

Finally,letmemaketwosuggestionsforfutureresearchwithregardtotheuseofmechanismsfor

policy-making.First,theroleofexperiments–especiallyits ‘testbed’version–inthedesignof

mechanisms appears to be particularly interesting for policymakers. The success of the FCC

auctionscanlargelybeattributedtothecleveruseofexperiments:thebackgroundconditionsof

thespectrumauctionswerefirstimportedintothelaboratory,afterwhichtheresultswereused

in thedesignof the auctions. This procedure combines extrapolationwith inductivemethods,

whichisapromisingapproachforpolicymakersgiventhecomplexityofthecontextsinwhich

theyusuallyoperate.Onewaytofurtherimprovethisprocedurewouldbetospecifythetypeof

backgroundconditionsthataretobe incorporated intotheexperiments.Sincethepotentialof

experimentaleconomicsisnaturallyconstrainedbyethicalaswellaspracticalconsiderations,it

needstobeclearwhenandinwhichcontextsexperimentsareactuallyfeasible.

Secondly, the interpretation of mechanisms as plausible causal scenarios needs to be

conceptualisedmorerigourously.Toillustrate,Cartwrightrefersto“causalscenarios”(2009b),

“stories” (2009a) and “causal chains” (2012) in various pieces of her work. Despite these

somewhatsimilarformulations,allbutthegeneralideaofcausalscenariosisstillprettyunclear.

Itisoftenmentionedasjustonepossiblealternativetoextrapolation,ormerelyasasupplement

to randomised controlled trials. Instead, it would be fruitful to approach the idea of causal

scenariosmoreexplicitlybydiscussing,forinstance,howthesescenariosshouldbeconstructed

70

andinwhatphaseofthepolicy-makingprocessthiscanbestbedone.51Tobefair,Cartwrightand

Hardie(2012:175–178)dobrieflydiscussacoupleofwaysinwhichcausalprocessescouldbe

represented, eachwith its own advantages and drawbacks. The literature onmechanisms for

policy-makingwouldgreatlybenefitfrommoreelaboratediscussionsinthisregard.

51Forexample,RaoulGervaisandErikWeber(2015)discusstheroleof‘orientationexperiments’inthediscoveryofmechanismswithrespecttothenaturalsciences.Theyclaimthat“orientationexperimentsarea special type of intervention experiments used to provide evidence for or against a qualitativecharacterizationofamechanism”(ibid.:47).Intheso-called‘orientation-phase’,“oneormoremechanismsketchesaboutthequalitativecharacterofthemechanismresponsiblefortheexplanandum-phenomenonareproposed”andscientistssubsequently“gatherevidencefororagainstthesemechanismsketches”(ibid.:49).Asimilarinquirywithrespecttothesocialsciences(especiallyeconomics)couldproveusefulforpolicymakers.

71

BIBLIOGRAPHYAlexandrova,A.(2006).ConnectingEconomicModelstotheRealWorld:GameTheoryandthe

FCCSpectrumAuctions.PhilosophyoftheSocialSciences,36(2),173–192.Alexandrova,A.&Northcott,R.(2009).ProgressinEconomics:LessonsfromtheSpectrum

Auctions.InH.Kincaid&D.Ross(Eds.),TheOxfordHandbookofPhilosophyofEconomics(pp.306–336).OxfordUniversityPress.

Andrews,G.(1999).Efficacy,Effectiveness,andEfficiencyinMentalHealthServiceDelivery.

AustralianandNewZealandJournalofPsychiatry,33,316–322.Berg,N.&Gigerenzer,G.(2010).As-ifBehavioralEconomics:NeoclassicalEconomicsin

Disguise?HistoryofEconomicIdeas,18,133–166.Cartwright,N.(2009a).Causality,Invariance,andPolicy.InH.Kincaid&D.Ross(Eds.),The

OxfordHandbookofPhilosophyofEconomics(pp.410–423).OxfordUniversityPress.Cartwright,N.(2009b).Evidence-basedPolicy:What’stobeDoneAboutRelevance?

PhilosophicalStudies,143(1),127–136.Cartwright,N.(2009c).WhatisThisThingCalled“Efficacy”?InC.Mantzavinos(Ed.),Philosophy

oftheSocialSciences(pp.185–206).CambridgeUniversityPress.Cartwright,N.(2012).WillThisPolicyWorkforYou?PredictingEffectivenessBetter:How

PhilosophyHelps.PhilosophyofScience,79(5),973–989.Cartwright,N.&Hardie,J.(2012).Evidence-BasedPolicy.OxfordUniversityPress.Clarke,B.etal.(2014).MechanismsandtheEvidenceHierarchy.Topoi,33(2),339–360.Coleman,J.S.(1986).SocialTheory,SocialResearch,andaTheoryofAction.AmericanJournalof

Sociology,91(6),1309–1335.Coleman,J.S.(1990).FoundationsofSocialTheory.Cambridge:BelknapPress.Cramton,P.C.(1998).TheEfficiencyoftheFCCSpectrumAuctions.JournalofLawand

Economics,41,727–736.Friedman,M.(1953/2008).TheMethodologyofPositiveEconomics.InD.Hausman(Ed.),The

PhilosophyofEconomics(3rded.,pp.145–178).CambridgeUniversityPress.Gervais,R.&Weber,E.(2015).TheRoleofOrientationExperimentsinDiscoveringMechanisms.

StudiesinHistoryandPhilosophyofScience,54,46–55.Grüne-Yanoff,T.(2015).WhyBehaviouralPolicyNeedsMechanisticEvidence.Economicsand

Philosophy,1–21.Guala,F.(2005).TheMethodologyofExperimentalEconomics.CambridgeUniversityPress.Guala,F.(2011).BuildingEconomicMachines.InD.Steel&F.Guala(Eds.),ThePhilosophyof

SocialScienceReader(pp.262–278).NewYork:Routledge.

72

Harari,D.(2015).GreekDebtCrisis:BackgroundandDevelopmentsin2015.HouseofCommonsLibrary.Retrievedfromhttp://www.parliament.uk/commons-library

Hayek,F.A.(1988).TheFatalConceit:TheErrorsofSocialism.Routledge.Hedström,P.&Swedberg,R.(1998).SocialMechanisms:AnIntroductoryEssay.InP.Hedström

&R.Swedberg(Eds.),SocialMechanisms.AnAnalyticalApproachtoSocialTheory(pp.1–30).CambridgeUniversityPress.

Hedström,P.&Ylikoski,P.(2010).CausalMechanismsintheSocialSciences.TheAnnualReview

ofSociology,36,49–67.Hempel,C.G.(1958).TheTheoretician’sDilemma:AStudyintheLogicofTheoryConstruction.

MinnesotaStudiesinthePhilosophyofScience,2,173–226.Hempel,C.G.(1942/2011).TheFunctionofGeneralLawsinHistory.InD.Steel&F.Guala(Eds.),

ThePhilosophyofSocialScienceReader(pp.68–77).NewYork:Routledge.Kahneman,D.&Tversky,A.(1979).ProspectTheory:AnAnalysisofDecisionUnderRisk.

Econometrica,47(2),263–292.Levitt,S.D.&List,J.A.(2009).FieldExperimentsinEconomics:ThePast,thePresent,andthe

Future.EuropeanEconomicReview,53(1),1–18.Little,D.(1991).VarietiesofSocialExplanation.WestviewPress.Little,D.(1995).CausalExplanationintheSocialSciences.TheSouthernJournalofPhilosophy,

34,31–56.Loomes,G.&Sugden,R.(1982).RegretTheory:AnAlternativeTheoryofRationalChoiceUnder

Uncertainty.TheEconomicJournal,92(368),805–824.Lucas,R.(1983).EconometricPolicyEvaluation:ACritique.InK.Brunner&A.Meltzer(Eds.),

Theory,Policy,Institutions:PapersfromtheCarnegie-RochesterConferenceSeriesonPublicPolicy(pp.257–284).Elsevier.

McAfee,P.&McMillan,J.(1996).AnalyzingtheAirwavesAuction.JournalofEconomic

Perspectives,10(1),159–175.Merler,S.(2015).Greece’sHotSummer.Brussels.Retrievedfrom

http://bruegel.org/2015/05/greeces-hot-summer/Merton,R.(1948).TheSelf-FulfillingProphecy.TheAntiochReview,8(2),193–210.Mill,J.S.(1843).ASystemofLogic.London:Parker.Nagatsu,M.(2015).SocialNudges:TheirMechanismsandJustification.ReviewofPhilosophyand

Psychology,6(3),481–494.Nik-Khah,E.(2008).ATaleofTwoAuctions.JournalofInstitutionalEconomics,4(1),73–97.Plott,C.R.(1996).LaboratoryExperimentalTestbeds:ApplicationtothePCSAuction(No.957).Plott,C.R.(1997).LaboratoryExperimentalTestbeds:ApplicationtothePCSAuction.Journalof

73

EconomicsandManagementStrategy,6,605–638.Raiffa,H.(1982).TheArtandScienceofNegotiation.Cambridge,MA:HarvardUniversityPress.Reiss,J.(2008).ErrorinEconomics:TowardsamoreEvidence-basedMethodology.Routledge.Roth,A.E.(2002).TheEconomistasEngineer:GameTheory,Experimentation,andComputation

asToolsforDesignEconomics.Econometrica,70(4),1341–1378.Scriven,M.(1962).Explanation,Predictions,andLaws.InH.Feigl&G.Maxwell(Eds.),Scientific

Explanation,Space,andTime:MinnesotaStudiesinthePhilosophyofScience(3rded.,pp.170–230).UniversityofMinnesotaPress.

Steel,D.(2008).AcrosstheBoundaries.OxfordUniversityPress.Thaler,R.H.&Benartzi,S.(2004).SaveMoreTomorrowTM:UsingBahavioralEconomicsto

IncreaseEmployeeSaving.JournalofPoliticalEconomy,112,164–187.Thaler,R.H.&Benartzi,S.(2013).BehavioralEconomicsandtheRetirementSavingsCrisis.

ScienceMagazine,339,1152–3.Thaler,R.H.&Sunstein,C.R.(2008).Nudge:ImprovingDecisionsAboutHealth,Wealthand

Happiness.YaleUniversityPress.Weber,M.(1905/1930).TheProtestantEthicandtheSpiritofCapitalism.Routledge.White,H.(2009).Theory-BasedImpactEvaluation:PrinciplesandPractice(No.3).NewDelhi.Woodward,J.(2003).MakingThingsHappen.OxfordUniversityPress.Ylikoski,P.(forthcoming).ThinkingwiththeColemanBoat.