CHAPTER 1

The Science in Social Science

1.1 INTRoDucTION

TRIS BOOK is 1bout research in the social sciences. Our goal is practical: designing research that will produce valid inferences about social and politicallife. We focus on political science, but OUT argurnent applies to other disciplines such as sociology, anthropology; history, econornics, and psychology and to nondisciplinary areas of study such as legal evidence, education research, and clinical reasoning.

This is neither a work in the philosophy of the social sciences nor a guide to specific research tasks such as the design of sUTveys, conduct of fie1d work, or analysis of statistical data. Rather, this is a book about research design: how to pose questions and fashion scholarly research to make valid descriptive and causal inferences. As such, it occupies a middle ground between abstract philosophical debates and the hands­on techniques of the researcher and focuses on the essential logic un­derlying all social scientific research.

1.1.1 Two Styles of Research, One Logic of Inference

Our main goal is to connect the traditions of what are conventionally denoted "quantitative" and "qualitative" research by applying a uni­fied logic of inference to both. The two traditions appear quite differ­ent; indeed they sometimes seem to be at war. Our view is that these differences are mainly ones of style and specific technique. The same underlying logic provides the framework for each research approach. This logic tends to be explicated and formalized clearly in discussions of quantitative research methods. But the same logic of inference un­derlies the best qualitative research, and all qualitative and quantita­tive researchers would benefit by more explicit attention to this logic in the course of designing research.

The styles of quantitative and qualitative research are very different. Quantitative research uses numbers and statistical methods. It tends to be based on numerical measurements of specific aspects of phenom­ena; it abstracts from particular instances to seek general description or to test causal hypotheses; it seeks measurements and analyses that are easily replicable by other researchers.

4 . The Science in Social Science

h' ontrast covers a wide range of approaches, Qualitative researc ,m c , hes relies on numerical mea-

but by definition, nonehof ~e~e~pFor~~~us on one or a small number surements. Such work as en. depth analysis of historical ma­of cases, to use intensive inter~l~ws o~ to be concerned with a rounded terials, to be discursive in met o ,an t or unit Even though they

h' ount of sorne even . or compre enSIve acc f qualitative researchers generally un-have a small number o ca~nfes, t' from their studies. Sometimes

ounts of I orma lOn d earth enormous am .. linked with area or case stu _ this kind of work in .the social :~~:-~::~ent, decision, institution, loca­ies where the focus lS on ~ pa. A' also the case with quantitative tion, issue, or piece of l~gls~ahor:' so~ant in its own right: a major research, the l~stance 15 ~ ~na :! or decision, or a world crisis. Why change in a nahon, an electlO , J dd ly I'n 1989

7 More gener-

. ollapse so su en . did the East German regIme c . t gimes of Eastern Europe col-ally, why did almost ~11 the COtn:::~:r r~ot always, the event m~~ be lapse in 1989? SometImes, bu . 1 r of event such as a politIcal chosen as an exemplar of a partlcu ~r {P~ommuni~ to reject a waste revolution or the decision o~ a I:artlcuf ar k's linked to area studies

. c: t' es thlS kmd o wor 1 f h disposal sIte. _o:n

e 1m. d culture of a particular part o t e

where the focus 15 on the hlstory an t' analyzed dosely and in full world. The particular place or even 15

detail. .. 1 cientists have debated the merits of For several decades, .p~hhca s. es area studies versus comparative

case studies ver~us ~~a;;stIc~. St~l ~litics using quantitative methods studies, and "sclenhÍlc s~u l~S PI' rich textual and contex-versus "historical" inVeStIgahon~'::t?v~~e~:archers believe that sys­tual understanding. Son:

e .qu~n lId to truth in the social sci­tematic statistical analysls. lS .t e on y TOha hemently disagree. This

t f quahtatlVe researc ve . 1 ences. Advoca ~s . o livel debate; but unfortunately, lt a so difference of opm~on le~ds to . to Ya quantitative-systematic-general­bifurcates the sOCIal sCI~nc~s mh 'stic-discursive branch. As the . . h d qualItatlve- umam . f f lzmg branc an a o histicated in the analysls o sta 15-former becomes m?re a~~ :~~~etle5s comprehensible to those who tical data (and. the1r wo . es) the latter becomes more and more have not studled the techmqu

f 'h lyses to the seemingly non-

d f th . relevance o suc ana convince o e Ir '. t . which its practitioners are replicable and nongenerahzable even s m

interested. . b k' t show that the differences between A major purpose of th1S . o~ lS tr~ditions are only stylistic and are

the quantitative and quahtahv~ l unimportant. AH good research methodologicaHy an~ sub~a~:I~:!t understood-to derive fro:n ~he can be understood---:mde: ' B th quantitative and quahtatIve """n'lP llooerlving- 10glC of mference. o

Introduction . 5

research can be systematic and scientific. Historical research can be an­alytical, seeking to evaluate alternative explanations through a process of valid causal inference. History, or historical sociology, is not incom­patible with social science (Skocpol 1984: 374-86).

Breaking down these barriers requires that we begin by questioning the very concept of "qualitative" research. We have used the term in Our title to signal our subject matter, not to imply that "qualitative" research isfundamentally different from "quantitative" research, ex­cept in style~

Most resea~ch does not fit clearly into one category or the other. The best often combines features of each. In the same research project, some data may be collected that is amenable to statistical analysis, while other equally significant informabon is not. Patterns and trends in social, political, or econornic behavior are more readily subjected to quantitative analysis than is the flow of ideas among people or the difference made by exceptional individualleadership. If we are to un­derstand the rapidly changing sodal world, we will need to indude information that cannot be easi1y quantified as well as that which can. Furthermore, alI social sdence requires comparison, which entails judgments of which phenomena are "more" or "less" alike in degree (Le., quantitative differences) or in kind (i.e., qualitative differences).

Two excellent recent studies exemplify this point. In Coercive Cooper­atian (1992), Lisa L. Martín sought to explain the degree of interna­tional cooperation on econornic sanctions by quantitatively analyzing ninety-nine cases of attempted economic sanctions from the post­World War TI era. Although this quantitative analysis yielded mueh valuable information, certain causal inferences suggested by the data were ambiguous; henee, Martín carried out six detailed case studies of sanctions episodes in an attempt to gather more evidence relevant to her causal inference. For Making Demacracy Work (1993), Robert D. Put­nam and his colleagues interviewed 112 ltalian regional councillors in 1970,194 in 1976, and 234 in 1981-1982, and 115 community leaders in 1976 and 118 in 1981-1982. They also sent a mail guestionnaire to over 500 Community leaders throughout the country in 1983. Four nation­wide mass surveys were undertaken especially!pr this study. Never­the1ess, between 1976 and 1989 Putnam and hiscolleagues condueted detailed case studies of the politics of six regions~eking to satisfy the "interocular trauma tic test," the investigators "gained an intima te knowledge of the internal political maneuvering and personalitíes that have animated regional politics Over the last two decades" (Putnam 1993:190).

The lessons of these efforts should be c1ear: neither guantitative nor qualitative research is superior to the other, regardless of the research

address-ed. Since mdnv nI sntefC::1t h~~ 50GJl SC1-

t;:ntists canno! be meaningfuHy fonnul<1ted in \\'i)V~ th"t permil statístl· cal of witn quantitdtive dnta,. ,Vi.> do flor wish to encouragt' the exdusive use of quantitative ri'Clmiques, VVE' are not tr."-

tú aU social sdentists out oI thE' library and into the o)mputt~r center, 01" to replace idiosyncratie eonversations v·;ith struc!ured ínter­vrevo/5. \v!:' ,1rgue that nonstatistieal researen h'il! prQduee more reliable result.'i if researehers pay attentíon to the rules of st'ÍenU{k in-fercnce-·-rules thuí are MJmenmes more dearly stated in the style or

Vr."r1<o;,"IV defined stntisticili melhods that under­,;¡bstract fomlai modeIs applkable

to aH kínds nf rt:seardl, even that ror whkh vari"bles carU10t bt> meil-sured quanhtativdy, The very i'lnd even unrcil1istic, nah,lfe oí statistical modeLs 15 vI/har lTUIkes the rules ni inierenc,,' shine through so de"rly,

The nlles oí inference that \<\,'12' di::;cuss are not reievant to all íssues that are of lo social sdentisi5L Many nI' ¡he most irnn<,rL"1

pohtical life-abour 5uch as agency, ob-dtizenship. and the proper reiation-

betvveen national socicties anó intemational po!itics--are phi1ü-mthcr than But the rules are rdevant to alI n.:seardt

lhe goal is to learn fads about the real worM. jnda~, thc dis­tinctive chara<:tenstic that sets soda! setenee apart from casual ooser-vation is tIMt social saenee seeks to arnve ay valíd i21ferenc('S bv toe

of weU-estaolisheu pmcedures oí inquíry. Our foens researd\ means that we sidestep lssues in the

sdence as ''.'en as controversies the role of the nature and existence oi tmth, ilHU fl'.'-

!ated We asstmw tnat ir is possible to have sorne knc)\vledge oi toe external world but thar 5uen knowk>d~e Í5 uneertaín.

nothing in our set of ruk:s tnat we mU3t run eXfX:~rímel1t Uf sueh a or coHect <1.11 relevant w!? can maK€ valió sdentifk inferences. An impor-

tant topic is worth studying even H very !ittl.e infoffi1atíon ís avaHable. The rt~ult oí uny research design in ¡his siruation wiH be

ul1c€rtain condusíons. out so long as we honestly report our chis kind of study can oe very usefuL Umited intt'1rmation

15 often a na~t.:ss.'lry feature of sodal inquiry. Because lhe social worl.d tllat us understand those r~

to understanó them neous¡y~ even ,,{!ten "bout cm cnndusions ís high. gene}' of a problem mar be so tilat data gathered by the userul sdentific metnods mig.ht oe ohsolete befúre it can be accum~ lated. If a distraught person ís nmning al HS swínging an ax, admi

i f lntroductíon

a.n

tüm;¡ke

inforrn,1t1011 about the Dhen,,,, .. ,,,,,, afe ,,{ten

S TI.e Scúmce in Social Scíence

able t~) scientific n:~~"2n.-'(h, but thc ~}c\:unluidti$,.H) oi ú~cts úhH1t-' is no·t sufü~ ('ÍtnL Facts. C.JD be f.'c1Ht?cted \lf quantit;¡tí\'l? r-t>~ean::her:s~ more ot" ¡ .. 'SS systematícaliy. ano lhe fünm.c:r is heiter than th,,· ¡aHeL huí OUT particular defínitíon (>f sóelKe rtC(lUlres the addítíonal ster llf iluempting tn infer the immedíatc d,11a lo :;omething brodder

\hat 1:; not directiy ubservoo, Thar somethjng may involve j¡¡

f,:rcnce-.using ob!ierv¡¡tions frmn the world to learn abour other unob­servoo facts. Or that sümething .may ínvolve (IlUsa! il!faCl1(c-leamíng abnut causal effects from tue data observed. 'Hle domain of inferenn~ can be restricted in spacte and behilvíor in American election,; sinee 1%0, so<.ial me)'vemt:nts ín F;¿¡stem Europe since 198t:)-··or ít can be extensíve--h1.llnan beha\'ior sim:e the ¡nventíon of In either case,lhe mark of sdentifk research Js (he oi rnak-

ing infereno2s that go the ol;!sen',ülüus coHectoo. :t The pmcedu:res a.re public. Scientific resean:h uses co(hfieó,< and

methods lo and data whooe reliablHty can there-

fore be assessed, Much social research in ¡he qualítdtín:, follows fe\Ver ruk'S oE research proCedufl' or üf ¡nf.:rence. As Robert K Merton ül94911%8:71-(2) pui il, "'1'he of qualilatíve ,lata üfren resides in a unfathom¡lble

and inélfabIe ' . is pub-

líe, no! prívate." /'vterton' s statemeni is not [roi' oí al! still tme oí sorne but

¡-mmy had no method-snmetimes as íf the use uf ex-methüds wouki díminÁsh theÍr Nevertheless they cannot

but U$e some methnd. St .. ,mehow ask ques-

tions, ínfer informatiun alx>u! 'rhe w()rld fmm thes€ observatíons, "ud

m,,!ke ínfererKi::s "DOU! cause ¡¡nd eH!;'cL H the method and

sea rcher' s observa tíons "nd ¡níerenee" are ¡dt the oí what tvas dúne. \Ve cannot

ev,tluate lhe (lf seit:ctivn Iha! wen.' used to n:cord übservatíons,

the ways Íll whíCÍl observathm" \vere and tbe \vhích (ondusions Viere drawll, '.Ve (annat ¡eara from th",ir mcthods l'r n:u"","<:

their TL"$ults. SucÍl research is l10t ,1 "el. V\'hether or l10t it makes

":",,un,,"-,' ít ¡s no! a contril::ll.!tÍOH to soda! sóen..:"" explidr or n.oi·-hiwt' Hrnitatíons. Tht: ¿¡dvan-

tage oE b ¡har those limitatlons can b(' understood M\d, if 1'0&­

síDle, addressed. in addition, the methLx1s can be and shared. This

protess aHows n:::;:;ean::h fl.>sults to br ers ;md research :;tudíes in b,' F""I·;,-"¡,,.,'¡

,1, The (cmdusions are uncertain.

prneess, tts the world tha!

lí'search­and scholars lo learn,

an d¡Ü3 tu learn abtYllt

I f

In!roduef1on 9

reasonabit.~ ~;'Stírnat( oE trnct~rtainty, (~ ,d the ret1I \\'odd or 011

¡nferenel! abou! <1 c;nb;d dÚ't'l in l'he f1'\11 world le; unintcrpretabl€, A re­~¿\~;ljcher \vho L"ül~ tn Lhvt! tht~ issuc (d un(ertainiY i...; FÜner a~~t~rf­

íng tlmt he' O[ she knows everything perfeetl\' 'tl[ [ha! hé "l!' ~¡~e ~:,~ ';1:1

idea how certain or UH,'ért1ín the results "re. Etther wav, inferences wlíh­ouí uncertai.nty eshHlales ¿¡re not science ,);. we define 'jI.

4. Toe content ís the method. Fina!!;', scientífic research "oh,,'l'('s to " $el ('jf

mIes (lf inference on v",hích íts vaJiditv Explicating the mOSl im­portan! nJles is a task of dlis oool<.': The n'l1!eni' of ",\"1C!1<:\"< ¡s

the meihods and rules. lwt the ~ubject m,lttef, "ínce \A;P can use these methuc]:. ro This W,1S

over a C,'nftlrv agu when Kar! Pcarson UtlQ1: lb} e).rl;'l!m~,j lh,ll "¡lw fl,'ld 01 scienc" b unl.ímíted; its mah"ria¡ í" endless; en:!\' gn>uP of natura! !10mena, l'verv of 5r'cÍi,! lite .. (·very s!age nf i;ll.~t nr ' me'nt ig material f(\1' science. Th{· of ,¡¡l sdence consists "kme in It;:

merhod, flot in íts materíaL"

Thes" (our íeatures 01' science have il further imnl¡·c·'tí('l'· "\'J' ' .. ""'" ~t ~ L,., r ~A. -' l~ ,J A' "u.\,..J:,. u

lis v€'st ís il S(lcil1i . rL:searcher 01' teitm oi reseilrchers lilbors undel' hmitiltions uf aod a.nd mistakes are umnroidablc, yet such errors .vin be poínted out others. LTn -

the social ch,:¡r.Kter 01' scíence can be sinee it means that out" "york need no! to be an im-portnnt contributinn-·whether tu [he d~"",'''¡''''

lo s" to redirect) the concems uf

scholars and uses methods tI.> arrive al ínfc'f' ences lhat are ccmsis!ent w1th rules oí sdenee and the tnformation ,1t OUf it is t) '~k ,'" A 1 1 1. ¡,,<1 "i:: a ü.ll1lnDutlOl1. i~nu t,c contrfbutíun uí even a minO!' ;¡rtide is than ¡!taL P( ¿he wnrk" ¡hat

fOft'ver in 2 desk Jr¡nver or v\'íthin tht' cnnfínes üf a compu!er.

1,1.3 Sdl'l1ce ami

Sodal sdence constitutt:.'s an aUcmpt to make sense ~!f s{}Clal sítth!u.ons that we as more 'Ir ¡"ss \<'~, ¡''''L>. J h) h . , ~,.. vv,. ".C U " Oh'"

ever, tÍlai wha! \Ve perct:Í\"'" as complexlty is not entirelv inllerent in phenomen¡¡: tIJe vvorld is l10t naturallv divid'" I'n"" ' J -' ~'U" aHú cnrn-

.1~t rrldr> . .-lrÜ~? uf thi' in)~XH'L'¡}'d ~¡:f ,,+{~ienhh(' ln.ft.::ren('~\

lnd{vtL rrUJst ds~t·;-t' Ú1t~t ~·t l\,lrtrpit~t(~ e\h~}tJsti\'e in·'

ís ímpossible, even in prindplt'.

• 1 {} , The s,.-imct' in Social Sdence

pIe, ~('ts or events. On the cüntr.lr\. the p,-,rceíved complexíty of a.si.tu­aílün depends in part on lit)\"; weil we úHi simplir).. ,m..:l ,1U~ capudty tu simplify depends on whether we can spearv outeomes ano explanatory vi1ri<~bk>5 in a coherent \-vay. Ha\'1.r:g. mort'~ obs:rvatlOns mav iissist us in lhis but is usually lt1Sl.üÚoenL 1 hus ccmpicx-ity'; is ¡:mrtiy nmefitümal an ¡he state (lf our theory., . .

Scíentific methuds can be as valtl<lbie fOT mtnnslCally eomplex events as for simpler onCS. Complexity is Hkely lo make OUT inferences less certain but should nof make them any 1e5..'> scientifk. Um:'ertainty and limited data shou1d not cauSE' us ro abandon scientific reSE'arch. On ¡he contrarv: the biggest payoff for using the rules of scientifk ín­ference CKcun; 'predsely \vhen data are limited, observation tools ~re

aTe undear, and rclatíonships are uncertatn. vVith dear and data" method may be lest? ímportant, since c'l/en partial1y mies of inference may producE' ansvlers that are rol.lghly corred,

sorne complex, and in some scnsc tmiquc f evenís vvíth enormous ramíficatÍons, The eollapsc of the Roman Empire, the Freneh Revolut1.on., the American Civil War, World Wal' 1, the 11010-caust and the of Germanl' In 199(\ (.re al! examples of such events.. TIlese events seem to be the resul.t of complex interachons oí many forces whose conjuneture crudal lo the event having taken plan:, That is, índependently caused sequences of events ~nd forces at a glven plaee and time, their interaction appeanng to brinO' about the events fFfirschman 1970). Ftlrther-

b more. it IS oiten diJ{kult to tht~e evenís ''''ere inevHable produds of l;¡r,ye··<;e;ale

on f,·",,,,,vn of our

5eems to have played a outside the providt>d crocial links in thc of events.

chance oHen 01' the theorv

Oue \vav to understand sueh events i5 by """,'k:'j",,,,

com:cptuaíiúng eaeh case as a member of a das:; events ahout which meaningful ge~eralization5 can be made, This method often ",,~orks \Vdl ter ordinarv v\'ars or revolutions, but some wars and revolutlOns, being much mo~e extreme than others, are "outHers" in the stanstkal di.stribution. Furthermore, notable early wars or l'eVO!utlOns may exert

írnpact on subsetluent events of the same dass-we think of tite Freneh Re\'olubon-that cauhon is necessarl' in comparing them \vith their successors, which rnay be to sume ext~nt ¡he product of ímit.'ltion the dass oí E'venís can be usetu1, but ir is not appropriate.

Another wa\' scientifkaHv \vith rare' large-scale events is to engage in ~ounterfactual anall's!s:' "the mental construction oí a

lntroduction . 11

course of \?yents ,'.'hich Ís altered through modifications in one or more 'conditiolls'" ü\·'ebt'f [190Sj 19'19:173), The applicatitm oí t11is idea in a sy5tematic, scientific wa)' is illustrated in a particularl)' ¿~xtreme ex­ample of arare event fmm geology and evolutionary biolngy, both historically oriented natural sciences. Stephen J. Goulcl has suggestt>d that oue way ro distinguísh systematic features oi evolution from sto­chastic, chance events mal' be to imagine what the wodd \VouId be like íf aIl conditions up to a spedfic p¡.)int were fixe-i and then the rest of history vvere remu. He coutends that if it were possible to "replay the tape off lite," to leí evolution ocrur again fmm the beginníng, the world's mganísms toda)' would be a completely different íGould 1989a),

A unique event on vv'lüch students of evolution have recentlv [0-

cused is [he sudden extinct1.on of tile dinosaurs 65 million vcars' agt1, GouIel 0989a:318} "\ve mus! assume thar consciousn~~ss wo~ld not have evolved on our planet jf a cosmic catastrophe had not daimed the dino&'1urs as vktims," If this statement is truc, the extinc­bon oi the dinosaurs was as imp¡.,rtant as any historÍ(al event rOl'

human beings; however, dinosaur extinction dtx>s nor fal! neatlv into a dass oi events fhat eould be studied in a systematíc, eompa~ative fasnion througn the applk"ltíon of laws in a straightforward way.

Nevcrtheless, dinosaur cxt1netion can be studied after-native hypotheses can b(~ developed and tesied with respect ro their observable ímplicdtions, Orw hypothesis to account for dinosaur ex­tinctiol1, developed by Luis Alvarez and eol1aborators ai Berkelev in the late 1970s (VV, Alvarez and Asaro, a cosmk coHi¿ion: a meteorite crashed rnto the earth ¡lt aboul 72JX10 kilorneters an nouL creating a blast greater than that from a fuH-scale nuclear \Var. H this hypothesis is corred,. ít would have the observable that iriditlm element ('omrnon inmeteorites but .are on should be found in the oí the eartn's crust tha! corresponds to st.>diment lnid clown míllion years indeed, ihe discoverv 01' iridium at predkted in the earth been t~ken as parti¡~J confirming evidence fer (he Although this is un unambígu-ously unique event., there are rmmy other observable implicatíons. For one example" it shouJd be possible lo find the metorite's crater some-where 01'1 Earth (and severai candidates haye been foundl.

The issue of the cause{s) oí dinosaur extinchon rernains unresoh' .. 'C1, although the has genel'ated mueh valuable research. For

.~ l-k)t\-'e\d&f .. an aHerrL~tiVt:· that t;~);tinctÚ.H\ \\'d~ (\1tL~~'J t:"v \:',)kanic crup .. tklrLSf !;>; 31S<J Cf)t'tsÜHent. t..,,'ith the preS{~nú,~ ot tridiun1, dnd g<-"\~ms rl1or~ l,)A)!1Sb!ent th~rt !he meteuri!e hypoth(>$ls wíth the findil1g that aH the specic'5 extim1imb did !lO! (KCUr

simultarwously.

12 . The Scicl1C'e in Sodal St.ience

our pnrposes, the pnint nf thi~ example t..;; that sClentific gc)neraliza­tkms dre usehJl in . CYeH highly t1!1u:;udl evento; th;l! do no( tal]

¡ntp i1 large ditss nf evenls. The AlvilR'J: hypothesis cannot be test!:\:! wíth reference to a ser uf common evento" buí ii does ha\'e ohsen:abte imp!ícatiol1s for orher phenomena th"t c;m be cv"luilred. We should nüle .. hmvever, tha! a hypothesis is no! considered a reasonahly ct~rtain explanation until it has been evaluated empiricaHy and passed a num­ber or demanding tests. At a minimum, iis implications musí be con­sistent \vim OUT knowledge of (he externa! wodd; at l:x:st, ir shouid preclíct what 1mre Lakatns 097m refers lo as "new facts," th"t those fornlerlv unobserved.

TI1e poíut is ¡!tal even apparently t~venl" such as dinosaur extioction can he sfudiL-a sdentiiícalIy 'Ve p<ly attention ro impnw-

theory, dala, and om use ni the data. lmprovíng OUT theory through conceptual darification and specification of var¡"lbles can generare more observable implications and e'len test causal thcurtes uf uníque events sueh as dinosaur extindion. lmpnwing our data al­JO\\'s us to observe more of these observable implications, ami imprü"­ing our use oí daí.) permits more oi these implíeatinns ro be extrac'ied from existing data. That a set of events to Di' studíed is highly complex dnes not render careful resean:::h irrc!evant \\fhether ,ve study milny phenomena or fe\v-or even one-the study \vil! be improved ¡f we eoned data on ilS manv observable oE our thenry as

L2

""¡P""'" research at lis best is a creative process of and within a weH-estab1isned struttUTe oí scient¡fk

ftlr a metharucal process of U"""''''",,,, the seholar musí nave tue ""AH"'''"'

and evalua­o! mind ro

oi looking al the \vorkt ro ask new questions, ro revÍse research desüms appropriately, fu,d then ro coHect more data of a diiferent l:ype than inlel1ded. However, if !he researcher's findings are to be valíd and aceepted by scholars in this field, aH these revisions ana recol1siderations rrlUs:t take accordi.ng to expíicit

consistent vvith the rules of A dyl1amic process ínquiry necu"TS \'vithin a stable structun" oi rules. &xial sdentisls often research \vith a considered dt,sign, co1-

lect some data, and drav."' condusions. But mis process 1S rarel}' a smooth une and ís not !x"'St done in this ordcr: conclusions rareiy foHm'\' easiiy from a resean:::h design afld data coHected in ateor-

:-'1a¡or Components of Research Design 13

dance \\,ith lt. L)r1Ce an tr1\'(~stigJtor hL15 c(,He~:ted dat.;) ~1S. rfOYided ~~y ti researcb he or shc \\~dl o.ften find an in1perh:ct tit aUl,ong U1C

main research quesbons, the tbcory dnd tIle data at hand. j\t thlS o,tage. researchers oiten bet~ome discouraged. They mistakenty hdie"e that other scientists find dose, immedi"ie fi!s between data ,md re­

Thís perception is fiue to the faet that ínvestígators often take do"\,vn t:he scaffolding after puttíng up their íntellet~tual buildings, lea\'­ing iittlc trace oí the agony ana uneerta.inty of con5tructiO!:, Thus tn.e pro<:ess of inquir\' seems more met~hamcaJ ;md cut-and-dned than Ir acroaUy 15.

Some of OUT advice is direded toward researchers who ané' tr:ring to make cmlnections l:x:tween th0)í\/ ;.md data, At times, they can design more data-collectinn proc01ures in order tn evaluare a theory al other times, the)' can tlSe the data tht'v h;:rve and rt,:'Cast a tlteÓretical questiün (or cven pose an t.'ntírely diff('renr question that was not originany foresl'Cn) to produce a more importan! reseaTeh project. The ¡f it adheres to rules of 'wiH stiH be sdentific and produce relíable inferences aaour the vvorld.

VVherever possibJe, researchers should also improve their research

dm.;i<'llS bdore conductin<> anv field research. I'lO\vever, data has a '-.. 17 o ~ h .

disciplining thoughL lt is exrremdy wmmon lo find t at me rescarch dl'sign fans apart ,vh('n the very first obscrvat1on5 are

co!lel::ted--lt is not tnat the throry i5 \vrong bui that the data are no! suited to answeling the qUi .. '5tiOns originally posed. Understanding from the outset vvnat can and \vhat cannot be done ar this later stnge can he:!p the rescan:::her al least 50me of the problems \vhen first designing the research.

For ¡malytical purpose5, "ve divide al! rese<1rch ínto four the research thl:' IheoTY, the da!ll, the use [he

düfll. These components are nor usual1y separately ¿¡ud scholars do not aUend to them in an)' order. In fact., for qualitative researehers who begin their neld work bdore choosing a precise research question, data comes follO\VL-a by the ot~ers. Hmvever, this particular breakdow'n, \\'hkh we explain in sect:ons 1.2.1--1,2.4, ís particularly usel:ul for understanding the nature (~l. rt.'" sean:::h designs. In urder to darify precisely wnat cvuld he done tí re­sources were redireded t our ad,dce in the remainder of this sedion as..sumes that researchers have unlimited time and resources. OE course, in any actual research sltuahon, OT\t? must always make eom­promises. W~ believe that understandíng the advice in the four cat~,,­gories that fol1ow will help n."Sean:hers mi1ke these compmmlses :n 5uch a way as ro improvE' their research most, even \vhen 0"1

fad thdr researeh 15 subject to external constraints.

The SOalce in SoCÍal Sdence

121

Ihroughout thís book, \Ve con.síder what to do once v;e ídt:'ntíÍ\' ¡bt'

object of researen, Givcl1 a rescaren questi(lT), what art: tlw \va~'s ro conciue! tbat rescaTen 5(1 that We can obtilin "dUd explanatíons nE ~Ociá¡ and politic,11 plwnümcn" 7 ()m Jiscussion begins with d research qU("S­

tlOn and then proceeds in the stages oÍ desÍgníng ilod eonductin'" the ~ L. t"'f

rest,~arch, But ''lhere do rescaTen yuestiol1s originate? How dOes a scholar choose tn/:' tapie for There is no simple ans\ver to thís question. Like others, K"d Popper (1968:32) has argued that "tltere ís no suen thing as a logieal methud oi hi1ving !lew ideas ... , eontalns 'an irrahona! element' m " 'creat1ve intuition.' u fhe nlies ni

at (he earliest of tne I'eseafch process are les" th"n are the rules ror otñeI' rL'Se;lrch adivines. filere are texts on Jc­signing laboratory expe-r¡!t1I::'nts un social statistical criteria 1m

drawing a {or " sun'cy oi attitudes 011 pubhc policy, ana man­ua!s on eonducting participanl observaoül1 of a buream::ratk offke, But there is no mJe foI' 'whích researeh pwject tu conduct. nor if we should decide to w()rk, are theI'e rules governing \vhere ,ve shüuld conduct it

"Ve can pro pose ways to select a sample oi comrmmities in order ro the impact Di alternative educational polkies, or ways to coneep"

ethnic conflict in a t11anner conduclve to the forrnulation and oE as to its inddence. Hur there are no rules tnat tell

educatíonal poliey or ethnic conflict. ¡n terrns off sodal sdenee tnere are bet!er ana \VOTSe '¡.vavs ro the

of ¡he Easl German govemment in 1989iust as -thert' are hetter and worse ways ro study the be!\'\'een i\ C;1n .. ",.-."

tion on laxes ~md the lik.::lihCK)(¡ ot electoral suecess, Sur there 15 no \vay to dettlrrnine whether ir ls better lo the of the East

reginw 01" the role ni taxes in U.5. electoral politícs, Ihe specific topk that a social "cienns! studíes may nave ¡} personal

and idiosyncratic lt is no acddent that reseárch on particular ~'oups 1S likely to be pion(~ET(,,(] by people oi that group: 'vmnen have oHlm led the ,vay in the history of women, blacks in the historv oi blaeks, immigrants in the oí immigration. Topícs ma)' aJ~) be mfiuene€l.i by and Vah.1é'S. The student ()f ¡hird-world polines ¡S likeJy to hdve a dL'Sire for trave! and a tolerance for difficult living condítions tnan the sl:udent oí' mngres­"ioIlal polky m¿lking; the annlyst of intemational eoot't'ration '~ay nave a particular distaste for violtmt conflícL '

These personal ¡ma ,'alm,'5 "iten provide lile motivation

Majen Components of Rescarch Design 15

ro beemnc J sZlCial scientist ;:lnd, ¡ah'!', ro cho()se a particular rt~Sblrch questiOlL As such, thc\' 111.1)' ((\nstltuk ¡he ",hll" ~f\b,mS [zlr

in: ¡} particular rescarch pmject-and approprít'ltdy so, But, ll(' matter ho\\' persona! or idiosyncratic the re"son;; tor choosing ¡; topic, the methods of "óenc(' and rules of infere!1cc díscu~sed in thís book wíll help schülars .. h;'vise more poweriul rescareh dtesígns, Fmm ti,," spective oí a potential mntribution to social personal rei1-sons are neilher necessary nor sufficient justíficatítll1s for the choice oi a tüpíc. In most cases, they should not appt.>ar in om senolady writings. Toput ít most direcHy but quitt;> ¡ndelicatelj~ no 011':: nm~

what we think-the schnlady communitv caH:S H'hat \Ve can aemonstrilte.

rules fUi choosíng ,) c!G not exisÍ, th<c'tc are individual detern1ining thc

of a research ¿'nterprise io ¡he scholarly community. IdeillI:v, aH rt.'Semch in the sodal sdences should [\'.'0 critt'ria, FirsC a rcsearch 5!n)uld pr)5C a tha! i~ in the rC;J!

The topk should be conset.¡uential inr política!, sociaL or eco­l1ümÍC ior understandíng something that affects rnany ¡x'oplc's Uve's, or for understanding and predíctíng eVl:'nts that might be harnlful or beneficíaJ Shively 1990:15). Second, a j-""''',,,.,,,,, project should makc a mnfrióufiul1 tu un flm: out c0llecfin' ftilfíans af the woríd. Tnis l¡¡Her crlterion does out imply !hat al! n>seareh that mntribl.ltes lo our stock of soda! sdenee natioos in tact ¡¡ims at makíng Gms;¡l inferences, Snmetimes the statc oí knO\vledge in ;¡ field is sl1ch lhM tnl1ch ilnd description ls nt"f'\led bdore \Ve can take on the ehúIlenge of nation. Often ¡he úmtributíon of a pl\)¡t:'ct wil! be descríptivc ínierence. Sumetimcs the may no! c"cn be descriptl\?e ¡nft~rence bui rathel' will t'X' the dos!:' oDser\'ation of partículilf evcnts <JI' ¡he 5um­mar)' of his!orkal detaíL meet OUf second críterion because are prerequisites to

Our firsi criterion directs our attention to tl10 rteal worle! of poHtics ,1nd social pbmomeoa and to rile current and historical re('ord (lf tht:, ('vents and problems Ihat Jives. VVhether el rescaTen question meets this eritt:'rion is a sodetal judgment. The "ec· ond criterion direets our attention Í<) the literature uf sderKe, to ¡he inteHeetual puzzlcs m)t püsed, lo th;,t re-maín to be ,1l1d to rhe scientific theorics ¿md melhods 2\'ajlablc lo süIn' them.

16 . The Sci¡'7¡ce in Social Sdeu<."'e

mt~ets OUT first criterÍon. Ten major wars during tlw last fOUT hundred llave killed almost thir:y mi1lion peop1l,' tLevy 19S5:372j; snme

"limltt.>d \'\'iUS," such as those between the Uníted States and North Vietnam and between lran nnd fraq, nave eaeh dainK>t1 over a míllion

and nuclear war, were it io oeeur,. cOUld km billions of hum,m being5. Polit.ical mismanngement, both domestk and international, has led tú economic prívation on a global basís-as in the 19305-as well as to regional and local depressl0n, as evidenced by the t:ragic expe­riences of much of Africa and Latin Amerka during the 19805. In generaL emss-nation¡11 variation in political institutions lB associated vdth great variation in the conditlons of ordinarv human Iife, whieh are reHectcd in differences in lite and infant mortal1ty be­tween countrít.>s with similar le veIs of economic deveJopment (Rus5ett 1978:913-28), Within the lJnÍtL't.1 programs designed to aHeviate po v ert Y or social disorganization seem to have varied greatly in their efficacy; It camwt be doubted that research \vhich contributes even marginaHy to an understanding üf these ls5ue5 is ímportant.

\Vhile SOt-:iaJ sclentists have an abundanee of sígnific.cmt questions that can be im/estigated, the r001S fnr understanding them are &Caree aOO ra!:her crude. Much has been written about vvar or mísery that adds 1mIe lO the underst.md.ing uf these issues because it raíls either to describe tht'Se phenomena systematically or to makl~ valid causal Oí descriptive inferences. BriHiilnt insights can contribute to un-derstanding by yielding new bUI brilliance is not a method of .1\11 hypotheses need h) be eva1u-at€'d empirically befor(' Gm

This book offers no advice on """'"'.V.<HH

ever, is to the importanee eonstitutes a contribunon to

a contribution ro knowledge. brillíanL What it can do, how­cnnducting researcn so lhaí tt

OuT secand l"riterion lor a research question, "'making ti

contribution/' means exp¡icitly a research des¡gn \vithin the framework oí the existing sooalSclentific literature, This ensures that the investigator understand the "sta te of the art" and minimizes !:he dü'lnce or duplicating what has aIready b(~en done. lt also that the work done \,;'ilJ be to thu5 success oi' {he of scholars h,ken as a v\'1101e.

contribution ro the líterature can be done in mi1ny diiferent \"'<'a1's. We líst a few oí the here:

1. Choose a

contribution.

ivlajor Cornpüncnts of Research Desígn 17

~ C:-h<)t)st.'> dn hyp~)tht~sis in the htC'Taturt~ that \-\.~(~

\{;f one \.n~ h.::'¡ievt' has nüt l:wen whether it ís inJeed fais,e or vfhetlwf s"me otht:'f theor\' l.'i n'frl'CL

3. Atternpt ro resolve or pren.·id .. further evídenct' ni one síde ni a contnr verSy in the rUeratun.::.it-perhaps dt~n1iH1str~1~} that the (pntro\'crsy i/'/3S

unfounded fmm the sta.rt. 4. Design research to iHurninate nr evall1ate unquestioned assumptions in

¡he lítemrure. 5. A.rb"U€ that an importallt topie has heen overlooked in the literature and

then pro~"t~d to cpnITibut:e a systematíe studv 1'0 the area. 6. Show tha! Iheorit'S 01' evidence for sorne purpose íll un\é' lih~ra-

ture could he applít>d in another literJture tu solve "n but appar-

cntfy unrebtL4:1 problern.

Focusing too mueh on making a r:ontríbution to a litera-ture without some attention to topics that have real-world importance nms the fisk of desn.mding ln politicaUy Con-versely, attention to the curren! política! to i;:,;-sues of the amenabi¡ity of a subject to syslem'ltlc study within tht.? framework a body of social sCÍeI1ce knowledge leads to careless .. vork that adds litile to our deeper understanding.

Our two erireda for choosing research questions are not in opposition to one another. In nm, understanding '''TOrId phenomena is enhanced by tile generatiol1 and evaÍuation explanatory hypotheses through the use of the method. But in the shorf term, t11ere mav be a contradiction betvveen use-furness and kmg-term sci~ntifíc VeliLle. For instanee, \1ankiw (990) poínt's out that maaoeconomic theory 2nd applit'd macroeconomics divergeti sharply during the 1970s ,md 19805: modds that had been ShtH\'Il to be lheoretical1v incoherenr "it~re still llSt'd ro forecast the direction oí the ES. • ,vhile the rte,,, tht:oreticaI de-

to correcí tuese flaws remained speculative and vI/ere not suffi­dentIy refined to make acrurate pretHetíons.

The criteda of practica! applicabiIity to the real world and tion ro to one another when a

,dI begin with " rea!­¡ht:> ¡hrear uf

l:}t~twf"{'>n men and 'VOlTH"I1.. ¡he lransition lO de­Others may stalt \vílh ,.n

thé soda! science mcrature: .1 coniradiction between studif's of under llr

an ínconsistency between thcories \oting and recen! election outcÓmt>s. The distinct10n hehveen rhe criteria oí (ourse,

18 The Setene!' in Social ':Xiencl:'

not hard ilDd bsL Snme rese,y¡ch questim15 Siltisty both criter;a tn)!1) the but in n.:.~st.~Jr('h, ft..:searchers oJtcn nearer (me than the other. 4

Whercver it begins, the ¡m)ccss of dt?sígníng rescareh ro ¡mswer a spedfic questlon 5hou1d mc>ve h)\\'ard the satlsfaction of OUT two c.ri­teri,L Ana obviously our dírt'ction of movernent wíll depend on wlwre ,ve start. Lf we are motivated by a sodal sdentific puzzle, we mus! ask ho\v io make thaL research topie more relevant lo reaJ-\Norld topics of signíficance-for instanee, how might ¡aboratory experiments beUer illuminate real-world strategic choices by poIitica¡ decísion-makers nr, what behavÍoraJ cousequences might tbe theory have. lf ,ve vvith a rea!-world \ve should ask huw that problem can be studied \vith modem scientifíc methods so that ir contribuíes to the stock oí sodal sdencE' Ir be i.hilt VI'e w¡¡¡ decide tha! too far from (me criteion or ¡he jo; not the mn5r truitfu! approach. Laboratory eX¡'?erimenters may argue tnat the &earen fnr externa! refer· en!s 1S premature and rhat mure progres" \vHi be made by refining the­urj' ana nlethoJ in the more controlled environment oi the laboratory. i\nd i.n terms of a long-h:,m1 rescareh tney may be righL

the schoLar mot1va!E'd a real-,vorJd pmblem rnay argue that aecurate description 15 needL,(l before moving to explanatiOl'L And sueh a researrher mav also be right Ac"Curate description ís an impor-tant step in researrh pmgrams.

in either case, a research ana íf possibJe a specifk researcn pmjecr, should aim lo our two aiter¡a: it snould deal witn d

rt'al-world to contribute. or to i1 ¡iterarure. Si.no: OUf main concem in

this b(,ok is research more scientiflc. we wHl who sta.rts wilh the "real .. v,'orld" pt'Y­

of rathel" ¡han wilh

literature. it 15 ,,;ssentlal to devise ti workable plan fOi

fhal (anno! bt into t1 re5carch ¡Ji Ciwsal 3hould be

¡ha! will make nO contri-

19

Eter;):url' ",houkl ~in1jL1r1v l~e (h-dnht~d, f'L1\'­ing cho:sen a \\'« ente!' il "n/ith tht:: litcrál.:U[\.:',

What questínns oí interest to U5 have aln,;\ldy been aH5wert.Mj? Hüw can we imd refine out' question su ¡hal lt seeu1S capable o) .. ~,."-m·,,d witn ¡he iun!s ;lYdildb1t<> \-Vl: ma\' "tart v;ith a lS511t',

but we wíH have to curne te h"th ,,¡th ¡he literature uf science and ¡he problems ni inference.

the such an oxymown sh,mld no\ be caBed ,1

1991:4; see a150 \Vmx..15 ,md Wallnn 1982). 'file development ot a as tlw fir:'it 01

It 50m.etimes comes in but it ne.d nor. in fac:t., we cannot a theof)' ,vitlwut of prior work 011 tlle

and the 'lince even tIle re"earel! don \vould \VhiÜeVer d!110unt oí' data has bL"CU there are sorne ate ilnd the usefulncss oE a theorv. VVé of these here but SilYe ¡¡ more detailed discussio)l ior

choose theoríes thdt could be wmnf!;. indL't'tL more 1" learned from theories thar are wrong rhan frO!'n theories th"t ,¡re $(1 lhar could not bto' \vrnng I;;~ven in : \'Ve need ro

a din.:'Ct answer ro ti",.' VVhdt ('vidence COl1\'lm::e us iha! Wt' are H lhere 1S no anf-'wer ro thi;~ then v;e du no! have a theon,r

;::hnüse one that is capil-as Thjs

choice váll allenA' more tests nI' the \víth more d"ra ¡¡nd a of will the al risk üf falsifi;:d nwre tunes,

"nd wiIi make it L"o'~"'~''':: to cullect data 50 a~ ro tmíld r'nr the

20 . TIw Sdcna: ín Socia! Sdence

Tllin.1, in the~)rj~"::--:, 1~~~ ~~s cnnCft.:te ¡)S possible~ \~Jguely st<lled theor!e,; ,md hypüthesl':-' SelY,.' ll\l purpnst' bUi hh)biuscah?:. The­Orlt'S th,1t are static't.i precise!y ilnd make specific prtxiictions can be shown niore ro be \VTong ana dre therefore beUer.

Sorne researchers recommend follmving the principIe of "parsi­rnonv." LJnfortunatelvI the word has bcen used in so manv ways in casllál conversatinn ~nd schülariy \vritings that the principie ha~ be­come obscured (see Sober [lQ881 for i\ complete discllssion). The dear­est definition ol parsimony V'<1S gh'en by Jeffre~'s (]961:-t7): "Simple theories nave highef prior probabililies."7 Parsimony is thcrefore a iudgment, or even assurnption, about ¡he nature of the ,vorld: í! ís as-sumed 1(1 be simple, The of theori",s thar a simple \yorld is a rule thar applies in situations ~'\'her(' there is " high degree oi certaínty ¡hat the workl is indet'Xi simple, Scholnrs in physics seern to find parsímony appropriate, but ¡hose in biolog.\r often think oi ir as In tlw soóal some forcefully ddend par-simony in theír subfields Zdincr 1984J, but we believe ít ís occasionaJ!r appropriate. Gi\'en the precise ddinition ni parsimony as an assumption about lhe .,'ürld, ,>,ce should never insist on parsirnony as n uf !ht'orit~s, but it i5 usdul in ¡hose situations wherc \\'e ha,,€' sorne oí tIte símpHcity of the 'levorld 1-ve an'

Our poínt is that we do nor advise researchers to se\:~k parsilTlony as an essential good, sínce there seems HUle reason lo adopi iI: unless ,ve alrl'adv know ajo! about a sUbiecL We do no! even need tn dvoid since ít is directl.v implied by [he maxim as aH our

evidence relative to the eS¡:lgi:Hell can ¡ead ro what \ve call

reS{~arch (see SL'Ctlon 4,1), out [hese are prob· lems nf research and not assumptions abour the world. .

/\.11 our advice ¡hus far applies ir we han: no! coHected our data and begun imy Hmvever, ifwe have gathered the

\ve can certain!y use these mies to modify Ollr and gather He\V data, and thus new observabie of the new

Of CTlUTse, this process ¡s time consurning, and \'\'ilsteful nf the d¿üa collected. VVhat then about the

situatíon where our IS in nbvinus net'tÍ oÍ but \ve cannot afford in collect additíonal data? Thís sltualion-íl1 \vhích

. This has comt' (o l:w KnUWl1 dS the cnncept ís similar tú CXcanú razor.

Maior Components (lf Researdt Design . 21,

/\11\ schobr CHl úHl\C up \\'íth ,1 "plau:.iblc" 111,,> for <"In\' ~et (li data alter the ¡"eL tti di) so (k'm()nstfdtc~ 1k)(h-

about lhe verad!)' uf the theorv, Th0 thcory wil! fi¡ [he data still Inav ¡x:' wildh,' wrong-índeed, demonstrahh' h'mng ,,·¡th other d:lta. llurrul1 al'(' verv aí pattdl):'

t very at recognizing nonpattems. (Müs! oi' us c':en se\: .,',._~ ... ', .... ," in 'random ink b¡ots~J Ad hoc adjustmen!s in a theory th<"lt

unt fit existing data IYtUst be usted rarely imd \,,:ith considerable

discipline.8 . .

rilere is stiH the problem of Wh,lt lo do when ,ve hilVe fnushed Otlr

coHe,~tiol1 and and wish tn work on improving ¿¡ theory, In this sittwtion. ",,'e fpl10vdng !wn rules: if our pre-",',,-.>,,,,,, 1S condítional on variables and we ilre willing to drop oue oi the condití¡)ns, ,ve may dü so, For example, if vve hypothesizt"C1 oricrinallv tha! demoCTatic countríes with social vvettare te;;'s do' not fíght ench other, ir \úmld be 1:0 extend that hvporhesis lo aU mndcm dcrnonades and thus !:'valuatc our theor)' against more cases and íncrease its ehances ot being .. '

püint is th¡lt <lfter the we mav modi!y our 111 a 'way that make$ it apply to a of phenomena. SinoS' such "n aJteration in our thesís it mon, fuUv to modifíca-oon in l:his direction should not lt~ad to ad hili: that merely to "sdve" dr1

ro phenomena that llave The opposít{' practke, ínappropríate, AHer ob-

the data, we should add a restridi\'t: conditíon and ~~w,.·""A as j¡ our th,,'ory, with ihat qualifkatiol1,. has been shown

to be corred, lf our oríginal" \vas that modern do not fight \vars wílh one al10ther due to their conslitutional ít \vould be less found lo ,mr restrict the pm¡:X1sition to dernocracies with advanced

once íf has bccu ¡he data tlmi such ti

!o make PUl' corred, 01' suppose that \vas that ren>lutions only occur under

oi severe econornk bUÍ we find that this ís not true in one oí our case studies, In sítualioll ¡¡ wnuJd flor be

such as, rcvolutlons never uelllr \vhen ¡he milítarv Ís the

ro ¿¡dd general uds (lf , ... rn"npntv

ershíp is "P1'''''''''''''' the f.'Conomv is bast>d" OH J smiill

- H \\"(1- thn'(' (hos~;n ~1 k~pk' <Ji n\11·\\'orki irnpnr:~uK't' ~~nJ '~H' ~l(W \x,'h~~:h rnake~ :'-'l~~1H" (ontribution ro ii scholadv htCTJttlrt~t the ~;ü('~al natuH::: 4,)1 dliH..leU1hl VI- HI corn::\ct thÚ1 snu­

¡¡tioa: $Olnt,'tme wil! rq:¡j¡~i11e our ,,'¡lh Mw!her ;¡eí 01 ¿¡¡ti! ,md uelno!l"t'r;¡!e UM!

we wefe wrong,

,'an~l111e'-clirn~1te i5 \\-:1DT'L Sucb ,1 f{~rr~1uL1ti~)n í~ Iil(lfC!Y ~) _misleading) Vi/ay of'my b correct, e:xcept in (()Un-

Since we nave discovered that our theory is lncorrect fÓÍ"cmmtrv x, ji dQ€s nor help to fum thís ialsífication into a spuríous generaiization. WiLhemt éforts to cnl1ect Dew we wil! han? no admis$ble evidence to support rhe new versíon (lf tht, theory.

So OUT bask rule "vith resped to altering our theorv after observing fue data is: me am ma/(¿' the theof!! les;; rcstridivc (50 tf1l1t ii nruers a broadcr nmge pltenomeml üud ís expDst'd lo mure Dpporhmities {or fl115ificatimrJ. [mI me s!u:ruld flof maite rt more rc::;tricfi,w WitilOut CO!!ecfing new data fo test the neu' Vi?fSiOft the tl/f;ory. lf >ve cannot collect additíonal data, then we are stuek; aud .ve do not pmpose any magica!way of getting un­stuck. At some point, dedding thnt \Ve are wrong lS indeed, nega-uve tindings can be valuable for a scholariy literature. Who vvould not prefer (mE' finding o',[er any number nE flimsy positive findings based on ad. boe theories?

!',4oreover. if \Ve are wrnng, we !'leed not stop \<vriting after admitting defeat, V'v'e muy acid a section to our artlde or a chapter lO our book about tuture empinen! resean::h and curren! rncclH'ticaI SpeCUL"tion. ín this contexto we have consíderab!y more fn.'Cdom. We ma)' 5uggest ad­ditional conditions that be attnchea tú our theorv, ii we beileve they mighr solve the probIem¡ propuse a modification uf

p",id·",.,n thcüry nr pmpose a range of entirely different theo­lve cannot conduae anything ,vHh a great deal of

o?rtainty perhaps iha! lhe ,ve stated nt the outset is out \\'e do have Ihe of !lev\" researen

ü:r data-cnllection that could be used to dc'cide v\'!lcther our speculations iire COlTié'ct These can be very in sug­oP;:.ti'"v areas vvhere futun: researchers can !oo!c

as \Ve discllssed sodal sdence aoes not to rules: the need tor sometimes mandates

that lhe textbook be discarded f And dar" can discipline thought. llenee rescarchers wm somebmes, after data, have i.nspí­rat¡ons abour loen-v they should have consfmcted the theorv in the first

Such a moditkation, even if restrichve, may be \vorthwhHe iE we can convinee oUrSt'fVes ánd nthers l:hat mcldífying the theory in the \vay that We' propuse is h'C could have done beiore vve co1-¡ecred the data ¡f 1ge had it, Buí unhl tt.'sted v'(üh 11(,,1' Jata. the status uf such a \ViH remain ver\' tmcertain. and ir should be labelcd 35 such.

Onc COlbcqucnce off ¡hes"! mies Í" !hát ¡jiten \tT\ useiuL "S[!l:"i. m 't's«'úrch \-';hctt~ datá must be

or other l1h?¡ms. Preliminary datil­questíons or modHy thc

"Data" are svstematicaHy collected dements of ínformation ~bout the world, They 'can be qualiíative or quantitative in style. Some~lmt'S d,lta are colleded to evaluate a ver}' specífic theory, but nor sO ínt:-equently, scholars col1ect data behm.' knowing predsely \.vhat they are mterested in finding out Moreov.:'r. cven if dJtil ,me co!1edea ro evaluate a ~pe-'f' - h"nt~thm;;l'" ma\' ultimatelv be interf'sted in questlíms oH: 'r-' ", .. ::>, • •

that h~d nor üccurred to them prevÍnus!y. In either case-,when data afe gathered for a purpose or

when data are used for some purpose not dearly in mina \vhen the)' were gathereLl-certain rules \'\tíll impmve the quallty 01 those data. ln princtple, Wt' can think aDOU! these ~ules fo~ ¡mpm\·ín.g ebta, [mm the rules in section 1.2.2 fUf llnpmvmg theory. ln . am data-colh."Ctíon efforl requires some (lE thcory, just as formulat-lng any sorne data (see . . 19(4): ' .

Our first ¡¡nd most important tor unprovmg data LJtIahty i5: record and fi'lmrt the ;mxess whid¡ ¡he data are \Nithoul this information we cannot determine lvhether standard pnxe-dures in analyzing the data \viH produce biasea inferen~es. On1y by knovdng the process by whích the data were generated W1Jl ,ve be able tú - valid nr causal inferences. In a opinion polI. recording the data-generation pmcess t!,at ,ve kmnv the E'x.¿¡ct method v\'hich the v;as dr:nvn and ti).:

dile questiofls that were asked, In a qualítative {',lse studji, reporting the mies ""hiel: v;e choose the smaH num-ber nf cases for IS critica!. We addlt10nilJ in d1apter 6 for case in qualitative but even more im-portant than choosing a good method is being C~rel111 lo record dnd report ,\-'hatt'ver method was lIsed and aH tl1(> inrormatlOn neces'k1fy

for someone else lO ¡¡PpJy it'l in section 1.2,2 \ve for theories th,ü are df

24 The S.:it'lVf in Soda! 'Sci('!Kt'

()Uf

data qunHty is in un!{"T l~etfL'r .f.{, e;\¡h-i{¡!t~ t; L'c:lI~'(t di.1'fa eH a,o.: lJhIU'i!

of its (ll'St'TUablt, . This me3rtS cullectíng as muci~ aata in as nMn\' diverse wnh',ts as possible. EdCh addit1c;nal ímpli­cat1cm of our th00ry which wc "bsern! pw\-kh:s aHotlwr cnntcxt in which to 1..'\'all.1ilte Í(s The more ObSi:'fVilblt, implicatkms which ilré hmnd to be COI1SLstent \-virh the theory, the more powerful the explanatíol1 ami tbe more ccrialn the results.

When adding ddta on ne\\' obseJ'yable irnplícations oi a theorv, vI/té' can la) coHect more obscrvations on the same dependent variabíe, or lb} record addithm<ll dependent vari"bles, VVe can, for inst.mce, dis-

to shorter time perinds nr ,1reas, We can collect ínt'orrnatüm on dependen! variables of dire{~t ínterest:

lf the results dre as t1w pn:dicb, we wíll hin:e more confidence in thE'

For cOTlsider the fational tneory: potentíal ínití-ators ot \varfare calculate ¡he cos!s and bendits of attacking other sta tes" and rilese cak'ulatlofis can be influenced bv cn:dible threats oí reíal¡.,tiofi, The most dirt'C't test oi tnis ¡heorv ~"ould be ro asscss

threats ol ,var, dedsions to atta~k are assocíakd h'Hh suen factors as (he oí militarv forces behveen the pütential ato t.,cleer and the defendt'r nr tht' ính,rests ai "take fm ¡he def~nder (Huth 1(88), even cases in v.;hích threats Me

íssued corLstifutes a set oJ implimtions of the theorv, thev ¡,¡re on1y pan oE the observatiollS thi'lt cnuld be gathered (¡ul¿i use¡i "lone mal' ¡cad to selectíoll :;inee sítuatíons in Vd1ich ¡hreóts tnemsclvcs are deterred vn)Uld be exduded ITorn lhe data seL 1-lenee it might be tvorthwhile aIs(, to colled data Oll an additinnal de'PE~n(lel1t variable a ser nf bdsed on a measurement of whether incentives tü do so,

Insofar as suffÍLlent datd ()fl dcterrenee in intern¡¡tíonal """"<"'''_~ is it could al50 be helpfuI to test a different nne wiih similar motivational rOl" a differenl dept:'udent variable under different conditions ,,,híeh ís still an implication of the sume For ex-r""'TH1'li"''H to see undcr símul,lted "threats" are dc-t0rred rather ¡han iKcentudted power and firm b,lrgaíning

(Jr \n~ coulJ exam!r'le v-'hether other actors in e sit:. sueh as flrms eompeting for milrket share or 01'-

{;:¡rnilies (or USt' deterrence ane! 110\'1/ successful are undel' rndt'ed, econo' misís working in the fieid of industrial orgilni7A1t!on have used nor¡-

\1ajC'f ('ompont'TIts nf l{t>seilrch Design 25

~> v fr'l ...... ) , ~:)n \\'hi·ch deterrt:n()'< C(}(lper,h.l \' t: ~.,. ~!.. l. " , .

sludy 5uch pmblcms 35 cnef\' mio market> dmi

(Fud~nbt>rg and TimJe jqK,n, Ci\'en l.he elose sim¡¡ilr1ty ¡'t>tween. tite theories, empírical e\"idence supportíng game lheorv' s predKholls about firm behavior wauId ínCTCa5t' ¡he phnlsibilitv oí reJ,ltt'cl hypoth­t.>SeS aboul stilte behavíor in íntemation,)j politícs, Uncertainty \A:ould rema in abuut the apphcabílity of concluskms tmm Ol1e domaín to an­otiler, but the issue is important ennugh lo \<V<1rrant attempts lo g,1111

inSight and evidence \vherever they can be found. , Obvioush~ In C(~llect data forever without doing anv arhlJysis wouki

precJude rather than faciJítate complctioll oí useful rescarch, In PfiK­rice, iimited time and resoufees wil! ahvays cOHsl:rúín datú~wlledion efforts, AHhllugh more informatiol1, addítional eas,:;,s, extra ínlcrvic\''''s, anothcr afia other relevant fonns of data collection \vil! al-

improve rhe oí OUT inferencps to sorne dcgree, prornis-l'lOtential ;;chola!':: can Ix' ruined bv too much inforrnatÍon as

as too little. Insisting on rcadíng another bnok or slill one more data sd wit110ut ~'ver \vriting ti w(lrd ís a prescriptíon fur

unproductive. OUT third guideline is: !lltlximi::c tile oí' om mc:vUn'111cn!,;, 'v'a

liditv refcrs ro what we think \-ve are measuring, The unlL·m-pioyment rate nM)' be a indiz-ator of the state uf the economy, !:lut the hvo are not In 1t is cask'Sf lO mdxirnize vahd·

ildhering to the and nol alknving unobserved or unmeasur-in ¡he wa,', ff an informant re~ronds lo our queshO!l

th~n we knO\\' he Si/id tilat he was Oí that, \ve have a ';aJid 111CaSurement VA1dt he mCimt ¡s rm altogdher different that canllo! be measured ,,,¡th <1

high (lf confidence, For in countríes w1th mdy be a W2y oí

poiitkal staternent fm sorne rol" it is <l ,vav nf donft kno\'\',"

Our fourth gu¡dehne is: msurc tlwf daia-cof1eclion mcillOds ¡¡re rdiaMe. ReliabHitv mean!; that app1ving the same in the same WiW

wm' the S<1~,e me,lStm:. \'\/11en ;¡ relíable pmcedure is applied at ¡imL'S and nothing h<1s in the meal1tlme to the "¡me" state oí the object \NE' ,lrc the san,e result 'will be obscrved, Rd¡"blc measures dls<'

\Ve can cht."Ck Z~Uf~f+;\.~5 b\' HH!d<:;,t.H'ing th(~ ".1nh:·~ q\ktntlty

wheth1.. .. r th{~ !n(\l;-;U.r('~ .-1-rt' tht~ <a11L Su~ctinH.>~ thi~ ~-t't:'rn:-..~ -C;k";~:< ~u·ch :,:"l~ thl> ~H11C qHt.'''..t¡n!'', .1t J¡tÚ:H:nt ;l~th"> ,juring

iH'h...\.-~ lHay ind~.h'·n\.\.-' th",' tu ~n V;'t~ nt>k~d in be (,an~fú!

26 . Hle SCÍi:nce in Social Science

suJts when applied by dlffert'nt resean::hcrs .. and this outcome F-~end~-, ot coursc" u¡Jon theTc bcing c'plicit procedurE's that can be fol­lo\\'ed. i1

Our final guíddine Ís: al! data ami should, as possible be Replicability ilpplies not cmly lo data, so that we 'can ~ whether oul' mensures ilre rehable, but to rile cntíre reasonin<> process used in producing condusíorL". On the basis of our resea:rch ~eportf ne'v resean::her should be able to duplicate OUT data and trace the ?y v\'hich we reached OUT condusions, Replicability ís important even' tí ~~: :me actu~lly repHcates our study~ On!y by reporting the study in surÍloent detall so thilt it can be repikatcd ¡s jt possible to cva!uaie the

foUO\-'i'ed i1nd methods used, Replkability of data may be difficult or impossible in some kinds of

rL"Search: intervieweL"S may die or disappcar, and dired observations (jf re~I-"~o..r1d E'vents by wítnes~"S or partidpants cannot be repeated. Rephcabtbty has also come to mean diHerent things in different re­sea,n::h traditions: In, quantitatl've resean::h, scholars tocus on repH­cahng I'he analys!s arter starting \vith the &ame data, As an\,one ;,vha has ever tTied tu the quantitativc rt."Sults ni even . publisht>d \vorks knows 'l/veR 1t 15 usually a lot harder than it should be and always more valuable than it seems at the outset {see De\vald et al. 1986 on repHcation in quantiti1tive research)'

The <1na10g)'" in traditkmal qualitative research ís provided bv foot­notes and biblingraphic essays. Using these tooIs, succeeding < •

should be ,lble lo locate the SOUrces used in published vvork and tnake their m"<.'n evaÍllations of the inf0'rences daimed from this information. For fesearch based on díred replication is more diffkult. On0' $Cholar could borro,,,, another's field notL"S or tape recorded inter­VleiVS to Sf'e whel~er lhev support the condusions made by the origi~ nal llWestlbBtor. Smce so much oí rhe data in field reseal'ch involve con:'erSr1ti~ms, ímpressions, and othel' unrecoroed partkipatory inful'- .. mahon, thlS of results lLsing {he S .. lme data is not often done, However, some important advances might be adtieved ir more schol-ars tried this type of i1nd it '\'GuId pmbably also encourage ' others lO keep more neld notes, Occasionallv, an entire re- :

induding has been replicated. Since we cannot quite

back in the replicati~m cal1not be but can be dH.ldl,ne nonethe!ess, Perhaps the most extensíve replkation of

/\'n is t hs,~ US(I ni tTh'nv Lh~~ n {'~IH~ c¡Y.1t,"r to t>'<Jruft !ron, tr;¡n'"-"("'ripts ,lt intervlev,/& fr {lV(?

_ how "fkn tlw same judgme!1L H they do no! prodw,:e reliaol" mt'a-5llnL'S, thel1 "le ca!! m¡¡ke the cnding mIes more preciS<., ,md try agaín, E,·entually; ¡¡ se!:

al mIes CAn often be gt'neratoo so tila! ¡he app!kation üf !ht~ same procedure by diJ'fere!.lt roders will yield the same result. '

'~

Major Compünents oí Resean::h Desígn 27

','c sh:dy is thc ",-wiologic11 study ofMiddlett'wl1, Indiana, by Roberl and Hden LV1KL Tht,ír fir;;t "I\1iddlettl\-vn" 5tudv \\';1::­

. in 192Y ;md was ;-{'p¡katt~d in a book publislwd Ín' 1937. fifty yearr; after the original stmlv, a long series ni books and

are published !hat n:plicatt' these NigínúJ studies bee t~t aL,1983a, 1953b ;md tlle dtatiüns thereÍnJ, AH qualítative

net.--d not be this extensíve, but this major resean:h projed serve as an exemplar for 1Nhat is possiblc,

research should attempt io achíe,-e as mudo replicability as pos­SChOJ,'fS should alvvays record tbe exad methods, mIes, and pro­

used tn gather ínformatíon and draw inlerences so that an" can do the <¡arlle thing and draw (one hopesJ the same

sion. Replicability also means that scholars \\'ho use unpub­or private records shnuld endeavor lo ensure t11dt tuture seho!-

wiH have access to the material on similar terms; taking advantagc access withour seekíng access for otheTs predudes repli­

cation and calls into ques!ion the scicntific qualítv ¡jf the \-York. Usuallv our work wiH not be replicated, but we have the responsibilíty lO ilt:t

H someone may .vish to do so. Even if tIte work is not replkated, providing the rnateríals for such repiication will endble readers io un­derstand and evaluate \vhat '.'ve have done,

Data

data pwblems by !lew ano better data ís almos! "l-an improvement on trying to use t1awed data in heHer

the former appmach is nut always possible. Soda 1 scientists nften find \víth el,lta and líttle ellanee to ilr1ythíng better; thus, han: to make the bt~t of what the)'

lmprovíng tite use oí previousJy coIlected data 1S ¡he lTlai.n topíc nmght in 011 statistkal methods and ís, indee:::L tite chid contri­bution oE Ínferentii;11 statístics lo th(' sodal sdences. The precepts on this topic that are so dear ¡n ¡he stud¡! of inferentíal statistics al::>o apply io quaIHative research. Thl~ remainder (lf this book deals wíth these precepts more hlllv, f-'lere .ve merely <1 brief outline oí tne guiddines lor ¡mpm~'ing ¡he use oí' collected data,

First, whenever possible, \\'e shouid use data to gerwrate ínterences that a.re "ul1biased:' th"l corred cm average. To understand this very idea fmm staiístical imagine,. the S<1m¿

methodology (in quan!ita!ive or qualitatín" research) for analyzing and drawing condusions {mm data across Inimv data sets. Because of smaU errors in the data or in the application oí the procedure, a single appliú'1tion of this methodology would probably never be exactly c~r-

\-\"111 be ('()rrt}ct \\d"h,~n tdkvll ~1:~ an aver-a¡-'Pllcdhr'''''' ........ ,'"·,',, ji nt) single appH,,-atifHl ü; correcL

not systematica!lv tilt Ih" oukom<..' in 0111: dírectíou

.-h'",'¡-,.,,,· unmas0d í¡¡ferentes depends. (){ Cllurse., br,tll un ¡-he nal cüUectioT1 uf the dat;¡ ami ib later U:-'10; and, ilS \\<e pointt::d out be­fore, ir is always Oest t,l anticípate problems befoH: dala COHectiOl1 be-

HOVVt,ver, \.\'e mcntion thest' issues brietlv here becauS(' when using the data, we need to be particularly caref~¡l to anal}'ze whether sources oC bias were ovel"iooked during dala collectinn. One such source., wnich C<ln 1e .. ti lo biased infenmres, is ¡hat o( seledion bias: choosing Ub:il~rvaLiüns in d manneT that syslemabcally distorts the

fmm \vhich were drawn. Although M1 obvious exam-pie is delíberately cases whkh support our rheory, selec-tkm bias calO occur in much more suotlt' wavs. Another diffícultv can resdt from omítted variable bías, w!üch rcfe~s lo the exdusion of :some control variable that míght infiuence a seeming causal comwctioll be­tween Ul,lr explanatory variables and ¡b,lt \vhkh \-ve ",'aní to explaín. VVe (ÍiSCU5S ¡hese ilnd numerou5 other pítfalls in prool1dng unbiased jnfert.~nct..'S in chapters 2-6.

The second 15 based 0n the statisticaJ 01 "'effi-. ¡¡n efficient use of data im'olVi .. 'S maximizim: the information

llsed for or Ci'lusal infcrence. ?vtaximízi~g re-nor onl)' using aJI our d¡:¡ta, but aIso usíng aH th~' relevant infor-

mllt10n in toe data lo ínferences, For eXilmple, if tlw data artO' into smaE \Ve should use íl tila!:

Ine smal ler will llaye of assodated with thenL bul if toe',' are,

in ob5erv"ble implications of the thenn,", fhe¡.- "vilÍ con-¡ .. ,in sorne inforrnation whiell can be brought lo oear 'on th~ inference

1.3 TJ1E\l1"S Of Tms VOl.l'ME

vVe conclude t!ti" o\'ervít'\v ,,,,,~~f,"~ thernes in

73J

In thís wh¡le, lrH:st 11,,\1:' fínd Ji the

¡he {OUT importan! h,¡ve discussed here

!1m! {fútil

to be worth ..

Theme:; uf Thb \tdunw

_h1b,,·t1,H facts. In !:ion, as Vh~n ilS hmv data dísdphne~ . " 11en:, we want tu strcss thnt tlwnrv "nd em¡:nncal rcse.1n::h must be ttght!y COI1-

nected. ¡har ddes real ViOr}; fOT us lus , . rOl' em­pírica! inyestígation;. no . . im'l-:tig,iltíon OH,' t)t'Sllcct:s~tuI,,~,¡th" out ro gtllde lis chOICe oi qUt'stmns. TI1eoI} ,md data ,.O!l:,ttOD are büth i1spects oí the bv which \Ve s0\7k to dt"'-"1de whether (l tneory should be provisíonalJy viewi;'d inle or ¡dlse,~5ublect as it 15 in both cases to the unc\~rtainÍy that charit(terízes all inlerencl:

We should ilsk nI' anv t!wo¡"v: \Vhat are íts observable lmphcatlOl1s:' We z¡sk abuu! ¡my en~piri(\11 Are ¡he (,bsen'a-nons rele\'i1J1t ro ihe imrlications of our and, Íí sn, v,h,lt do the)'

liS ro ínier aboul the (orrectness of the thc~(}rv? In ilnv ""-,,,,~n, .. studv, the implícations of the ilnd Hit' obsen'iltion uf fads need tU l~,esh vdtn one another: sociúl sderKe condusíons cannot be rdiable ii are nol based on ano data in

one another ,me! ilnd ímplications of a theorv.

The :;cholar who seuTches fol" is ence: as t!Wcfl as ,,,,,;;,;;,IlI,'

Sde!KC seeks (o inGeilSe ti", ro the inrormaüon used in the

what ai first to be a callsfdvariabJe nr il few variables, the is ven: une Of '" variables ,ve al50 han:' ¡he social sciences in dna even more so in !)i:lrtirular areas. Thís may be becanse do not know how to íncrease

it or becanse n"tllre flot ro be in d cmwenicnt fash~,

ion ur for both (Jf these reasons. Ancas are oiten titose in \Nhích

,1 h\,.lst nf ú hule. in such ('"se<;, uur

with more

ol' il.ny­Y3riab!es: \Vi;' use a lot in

"buuid bc to

There ¡¡fe ,',Hl0tlS W<1,'" in VI/hic!, we edIl innc;¡~'t: ,¡ur (lver ,1 n'St>1tch Tl~e IV"V is t" íncn'éhC Hit' lHitnbvr oí

:Ir r",':,;::: (,t

1hose }\s w" have dcscribed "00ve, ¡his 13Sk CDn invo!vc

The Scicm::e in Social Science

n) írnprovíng the theorv so that it has me>fC observable ímp1ic;1tions, (2) impn)ving the data so more of these implications are indced oh­SerVE.'t.1 and used to eva}uate the theory, and (3) impnwlng the use of the datD 50 that more oí these implicDtions are extracted frcon) exístíng dat,. None of these, nor the general concept nf maximizing lcverag; are lhe same as the concept oí parsimony, whích, as we explained in section 1.2,2, is an assumption about the nature of the \vorkl rather than a mle for designing research.

Maximizing leverage is so important and so genera] that we sfmngly recummend {hal researc}¡crs routínely li::;f aH po:;sibfe obst'rmbíe ímpJicatitms

thtir tila! be obscrced in t/reir data Ot in o¡ha data. It may be possíble to test some of tbese new implications in the original data set-as long as the implícation does no! "come oul of" the data but is a hypothesis índependentJy suggested by ibe theory or a differ­ent data S{~t. But it is beHer stilJ to fUffi to other data. Thus \ve should also consider implkations that might appe,u in (lther data-suco as dilta about other units, data about other of the units under study, dala from different of aggreganon, and data fttJm orher time such as predictions ab::mt the near futul'\c>'-and evaluate the hypo!hesis in those seUings. The more evidence 'Ne can find in varied crmlexts, the more powerful our explanation become5, and the more ccmfidence we and others should in aur

At first thought some researchers ma)' object to the idea oi collect-. ob::rvable implications aH)' 5OUr-ü~ 01" at any level 01' agsrega-

tWl1 difierent fmm that for whkh the theorv was For exam-Ut'berson (985) to qualitative ~t'search the statistkal idea

"n" .. ,.,,,",",,,h, dala to make in-wam cross-lE'\'el

lha! \ve can use data lo make íncorred about Índiv¡duaIs: if we are interestetl in then

individual'> is generalhi a better if we can obtain datn, t'''Íovvever, ir the infer~nce "ve seek tn make is more rilan a

C<l.st hypothesis, our theory may implicaríons at man~v ieveis (lf analysis, and we wiH oHen be able to us .. : data fmm aH these leveIs to provide some our theory Thus, even ir Vv'te an: priman!y interested in an leve! ni \ve cm

~~. Tl1t~ ". ~ 15 bttG~U$t.~ tbt.~ process> of rCds{)ning freHn ilg,greb~lle- ro mdJ\,'¡dUal-h:ve! proC('l'St5 15 Heitlwr nllr .1

b $.1n unhWH.Hl(lte choüx~ of \vürd in dt:~-cribt' tl-u:- h:!\'f~l nf R~ )bin;fYt1 ;': 0~ n (:oflclUÓt'"d in hb (~rigintü artidt! ággregate

ft'l n:'\'t"--nn: :1bout individuab bit qUiHititatl'd: sil-cía; scicntb-ts and statisti .. d,m~ nn\\', re~'úgnize Iha! süme íniofmatÍon ,¡bout índiFídu"ls dOt,$ "xi::;! ,'1 ,'ggTL~ gate levds <JI ,lna])'S!::;. <Uld numy metlwds of unbíased "ecoJogkal" inferenre have been develúlx"<l,

Themes of This Vo!ume . 31

, t'Jtt€'fl g,ün len::r"gt' alxml uur then.)' > bv ;lt t[¡Ie' dat.:l ihese othe. levds, example, if h'C develop a tneory k) explain n:nJlutions, ,ve

suui.'llulook for observabk" ímplícatjons of that theory n01 only in over­aU outcm:nlC'S but also such phenom12'na as the r~:sponses to in-depth intenriews 01' revolutionaries, the reac¡ions al' peoplc in smaH commu­niHes in minor parts nf the country, and offidal stalemenl<; by partv leaders. We should be wiHing lo take whatever inJormatíon \ve can acquire 50 long as it helps liS leam about Lile veracHy of OUT theory~ lf '\.\"€ can k>st our theory by examining outcomes of revolutions, fine. But ~l most cases ver)' littl~ ínformatíon exÍsts al: tha! ¡evel, perhaps just ene or a few observatíons, and theír values are rarely unamhíguous or measured without error, Manv different theoríes are consístent \v1th the existence nf a revolution. Únly by deeper in the prcsent case, or bringing in reievant informatíon exishng in olher ca5{;."5, i5 it possibie ro distinguish among prcviousiy indistinguishable IhíJories.

Tile only issue in USillg inJürmatíon ai other level" and from other 50UTces to study a theory designed at nn aggregate level is whether these new observations contain SO!Ht' ínJormation that is relevant to evaluating implkations of OUT theory. If these new observations help to test 01.11" thenrv, lhev should be llsea even if thev are not the implíca­Hons oí greatesÚnh=r~sL Por example, we may m;t care at aH about tlle views of revolutionaries, but if their ansvvers to our questions are coo­sistent with our theon' of re'volutlons, then the theorv itself wil! be more líkely to be ('(lrrl~i, and ¡he colIection of additim~al information \,,'ill have !:leen usefuL 1n ill1 observation at the must

(X'Ctunmce ni a predicted revolution. foI' one observed implkation of the theory, and be­

cause oí the small amoun! 01' infom,ation in ít, ¡t should not L1€ pn'\·i­leged OVer other observable implicatíons. \Ve need io colled infoffilil­non 011 as many observable ímplications of our theory as possible.

133 Rc¡;orting

AH knowiedge and aH quantitative and in Cjualitative research-is unceríain. measurement is as ís quant¡tativ€, but the sources of error may ditEer. TIle qualitative in­terviewer conductíng a long, in,depth intervie,y with a respondent , ... hose background he has studied is less l1kely to mismeasure the sub­ject's real polítical than IS d researcher conduding a strudured interview with a randomly selecled respondent abüut 'whom he knows nothing. (Although the opposíte is aIso possible if, for instan ce, he reties too heavily on an infurrnant who is not trust-

~'1jrUwJHow<:ver, the ;:¡Uiq'V researdwr í:; h':,;s likelv te· generalize inapFoprjah;~fjt (rom the CáSeO' int¿n·iewed to th\! broader popttÍ<!lticm than 15 the in-depth ff'Se.1fCneL Neither ís hnmune from ~ uncertainties of measurement or tht, undt'dyíng pmbabilistíc na-túr€' oí the ,,,mId.

AH good social sdentists··-whetner in [he quanlitatin: or qu¡"litative traditíons-report estimates oí rhe uncertaintv of their inferences. Per­haps the single mos! seríous problem v\'Íth q~¡alí!atiVt' researrn in po .. Jitic .. l ::.cience ís the pervdsive failure lo prm-ide reasonab!e estimates (lE rhe ni the investigator's inferences (see King 1990), We

in almos! anv situation, no matter how lim­ited the follml/ing the n¡i~s in this but \VE' shouid avoíd forging s,veeping condusions from weak data. The roinr ís not !hilt relíab!e inferCl1ces are in but racher that "ve snould report a reasonab!e 0stimate oI ¡·he

we have in ead¡ of our lnferences. Neustadt and rv1ay ue"u,,,, vdth <11'(',15 in which quantitative estimares

are propose a uscfuJ method nf encouraging pt,lkymakers C\<vho are üÜen faeed witl1 the necessitv oi condusions about \-v'ha! to follow out of inadequate data) tn the uncertaintv ol their mndusions, They ask "How mucn of \Tour o,,\'n monev ,vHuid you wager on ie' Thís ~lakes sense ¿'s ~s vve also as!.::. ":At what odds?"

1.3.·1 El]:!' a St}(.'·iaf Srienti~t.' ¡md RÍ{YIJ Hypothest's

The of caus.c¡l inferencl's means that good scíentists them. VVhen ¡(lId l\ ca uses B, someonc v'lllo

social sden tist" asks that cormection is a true causalone. H ¡s eas}' to ask stlch qUe5ticms abüut the research of others, but it 1S more important tu ask them about our own research. There are man}' reasons lvhy \Ve might be skeptical nf a causal ilcconnL plausible though it sound at f¡¡-si We rcad in the nevvspaper that rhe Japanese e,]1 !ess red meat and have fewer heart attack.s than Ameri· can$. This observation alome is il1ten~stil1g. in additiol1, rl.E' explana­han-too much leads to the high Tate of heart d¡sease in the United 5tatt's-is plausible. The skepticaJ social scientist asks about the accuraey of the data (how do we know ;lbour e¡:¡ting hilbits? ,vha! sam­pIe vvas used? are heart attacKs dassífí",'<.l similarlv in ]apan and the Unik'll States so that we Jre comparing sÍmihlr ph~nomena?i- As-sum­ing that the data are accurah.', lvnat else mlght explain the effeds: Are tl'\eI'e orher variables (other dietary difference5, genetic features f Há'-

Themcs of Thís \'ülume ;;3

ín-

ti\' cau:-~' dnd <.'ftecL' rt is h¿¡rd tu imagine he,," not a hear! aUt1ck might ,-aUSe c,ne lO ea! less red mea! bui ít ís . Pernar::; penpk lose th",ir appctile for h<lmburgers and

in lf this were tt-\\: case, tbose W!lO did not 11,1\,<: " heart atiack whatever reason) would live longer "nd ei1t less mí'at This fad

roduce the "ame rdatiol1ship that lcd the H..'St7drchers lo con­

Hwi mear \Vas the cuiprít in heart is not our purpose lO caH su eh m~>(iíci11 studics ínin questíon-

we wish mere!)' to illuslrate huw scientists appnhlch t¡w oi causal inferen~e: with ilnd il cunL'ern (or alternati\:c

that n1dV h,wt' been overlooked. Cau~al inferenct' thu~ . beromes a ' condusion becomes the occ2tsion ror further research tu refine ilnd h:st iL Through succeSSIV\"

!o come dO~'r i:lnd doser !o accurate ínfercnce.

Descriptive Inference

SOCIAL SCIENCE RESEARCH, whether quantitanve 01' qualitative, in­volves thc dual go¡ds of descríbing ilnd explaining. Sorne scholars set out to describe the \''lorld; others to t'xplain. Eaeh LS essentíal. \Ve C3n­no! construd mcaningful causal explanations without

dcscription, in ¡um, losz's mos! of it:;, inlerest unless linked to some causal rdationsnips. Description often comes fírst; it is hard tu dcvelop explanations before \ve know something abour the \",orld <lnd what needs ro be explained cm the of \""hat But the relanonship behveen descripticm and explanatíon is interactive. Süme­times nur explanatíons lead us lo ¡ook for déscriptions of different parts 01' the \vorld; conve1'sdy, our descriptions milV lcad to nev" causal explanations. .

and expianation bt)th depend upon rules of sdentific ~nference. In this we toens 00 description .md descripti\'c in-1e1'ence. Description ls far frorn mt."C'nanical 01' unpmblemat:k since ir involves selection from the ¡nfiníte number of fads tnat could be rc.~ corded. There dre several fundamental of scientifk description. One ís that it invol\'!;:'S part of the descriptíve task is to infer informatlo!1 about unobserved facts [rom tnt' facts ,';'e have Another involves dislingl.lishing thar whkh is arle abtmt ¡he observed and th"t whicÍ1 is n(Jnsvstcm¡ltic.

As ShOllId be we ,vith who denl GTi1te "mere'" description. Even ¡f explal1ntion-connectíng causes an~1 the ultimate goa.l, descrjption a central role in an explanation, and it J.s fundamentaHy important in and nI" ítseH. It ¡s not description ver­sus explanatian tha! distinguishes scientífk research from other re­search; ii lS whether systematk is conducted accanlinn to vaIid pmcedures. Inference, 'ivnether descriptíve or causal, qmmtita-tlve or qualitative .. ls the nltimnte uf al! social sdence. tematicaHy coHecting fads ls a very importm1t endca\,,'or witho1.l1 t-vhkh sdence ,v(mld noí: be possible but whkh deleS not by HseU con-stitute Good ardtlval work or wdl-donc summaries of híston-cal mil)! mak€' gCh.Xl hbh)r)~ but neither are suffkient to constitute sodal science.

In this cnapte1', we distinguish description--the coUection oi faets­fmm descripthre inference:. In section 2.1 we discuss the relationship

_ -Genera! Knowledge ,md Particular Facts . 35

nl scnolar::;hip: discovering knowledge and particular fads. \Ve are then

to explai.n ind

morl' detail tite concel~t of inference ín s"ction 2.2.. Our approach in thz' i'vmainder of the bOOK is ro present ideas both vcrbaHv and thnmgh \'ery simple algebnlk mo{iel::; 01' rescarch. In

. 2.~ we nmsíder ¡he nature oí these models. \Ve then discuss . <l>I'",UM"

irR"'ldels for data coHection, for summarizing histoncal dc,taiL aud lor descripti\'e inferencf' in secnons 2.4, and 2.6, respedivdy. Fínall~', we provide StlmE' spt'Cific critería for iudging descriptive infen:nccs in

"'l~ !,/"",,"'",\,!H .:..../ •

2.1 GENERAL K!\lOWUmGE AND I'A1U1U.iLAR FACTS

TI'le world tha! sodal scientists study ¡s madi'! up of particulars: indi­vidual v<Jters, partlcular government specifk tri bes, groups, shltes, and natíons. Good sodaJ science attempts tú SO beyond titese particulars to more general knO\vledge. Genem!¡za­tion, hü\vever, does no! elíminate the impnrtance oi the particular. in mct, the very purrose oí müvíng fmm the partícuhlr tü the is ro imprüve our understandÍJlg of both. The specific entlties of ¡he soda ( \.·vorld-"()f, more preciseÍy., spedfic facts about these entíties­provide the basis nI! 'which genernlizations must rest. In addition., ,ve almost leam more aoout a spt:cific case: by studying more gen­eral condusions. If we wísh to kmn\' \ .... hv the foreign minister of Brazil

it ,vin help to leam wh~' othe~ ministers in Brazit fnreign ministers in oiher cmmtrit~S have resígned, or \vhy peopll'

in general resign frm11. or I;'ven nongovernmental joos. E:1.ch oí these v,·m lIS understand different of facts ano principies of human behavior, but they are very important even ir our one and only goal is to understand the most n:.'\.--:cnt BrazBiilD f<treign minister resigned. For eXitmple, by' st"lJdying other ministers, we rnight learn that al! the ministers in Brazi.l resigned to protest the ac:nons of the presidenL something ,ve m.ight nüt have 1'('aiized ex­amining onl}' tbe actions of the foreign minister.

Sorne scKÍal science resenren tries lo say sOJ1.1.ething about a dass of events or tmlts vdthout snying anything in particular about a specific event 01' unit. Studies nf v'oti~g b~havi~r using mass L'Xplain the oecisiol1s of pi..'{)ple in general, no! the vote of an]' particular indh-'iduaL Stlldies of finance explain the of monev on electoral outeomes across di! districts. 1'",lost such ;tudies ,vould nui mcnHon the Seventh Congressional Dístrict in Pennsylvanía (}[ any other district except perhaps, in passing 01' ¿lS

exceptions to a general rule. These studies foilow the injunction of

36 Descríptjn~ InferenCt.'

PrZt)V~t()r~ki and Ir'une zlYH]i: ("'Jnrun{::L,~ F)T'()!>cr ndn1~,:~v

thOllgh tites" "ludie" !:1ilV nor sL'ck hl UIhJ,:rst;¡nJ any particular trIce they should not 19nürt.'--;¡s sometimcs ís unfor!unatelv done rhis tradition-:-,the requirement th¡!t rhe facts abouí ¡he \',l~inus dis~ tncts that go mío tht~ general analysís musí bt' ilccuratt',

Other research tries t:) tell tl" SilIllethíng abou! a partícular stance. H focuses on the Frenen Revolution or sorne other event and attemprs to pmvídc an expbn"tion of hu\\' or vvhv event came aDout. Resedrch in this tradition \v(mld be . cerrain!y unínteresting ro most of fhe usu<1l re"ders of such .. ,,"A~,~_ without names. A politícal scientist wrile dfcctívelv

uf ilcmss the set of -\,,:·ithout.!(lo~íng M specific districts or candíd,ltes bu! ¡m"gine Rnbert Caro s d¡SCtbSll1!1 09B3l ¡¡i ¡he 1948 Senare fiKC in Texas w1th­out , Johnsnn and Cok" 5tt'venson.! Particular events such as th,,> fn:nch Revolufion or the Democratic Senate primnrv in Texas in 1114S m,,}! inaced be of intrinsk interest: ¡he}' our ~uriosih~ and lt t,hey' .vere prenmditions for event'i (such as the N~pole~ OJ1!C Wa:!"S or presidenc~'), \ve ma}' n0<.."(1 to know about them t:} unders!imd those {"rer l'n~nts. knnw!edge about revo¡tl~ hon, ,rebeIlion, or ~iviJ 1NJr in \'\'111 pmvide invaiuable informa~ ~1(ln !Or any more rocused study of ¡he cau5l.S oí' the French Revolution 111 particular.

We 'will consider these issues daimed alternath'(:' tn m!en:l1CI! bectll m 2. l. ¡ 1, Ihe of uniquencss ¡¡lid complcxÍly oí the oi and thc area of case grudíes (section 2J .3).

In the .human some historienl and anthmpological rcseareh-ers darm to seek knovv!edge through what thev caH "in-terpretatinn." $l~ck ilccur;~te $umn~arjes nI' historícal de-taiL They also seek !o place the 0\;ents they descríb,e in an infelli"ible c,ont~xt within vvhkh the ()f acti¿ms bt:comes b .As FerE'}ohn COldstein and Keohmwl has ;.vnflen, "We want

l Nnr «lB t\'€, díSD1b5 Caro a~ ~t}n)(~üne in anothuf busincss: d

differ~; frnrn that {t[ tnf: SiK'íld scit"ntJ.st l'h~ %york addn:sse~ S0111e (ji" the Sárne

~rentf5t \Ydtltt.1: \Vh;H h\ads h ~ ~\lH.~t't.h:;'S ,~r faHuft' in ~Ui. dection . iVbat f~; tile r01f~ ~H' rno0ey <.UhJ {¡n,;;nce in t:ll'-([nfdl '-'th'(.e<..,~: \Vh.Jt

mntwat0; ",:mpaign cO!1rríbu!o!'3' TI1<' disnJssíon íOCU5eS (In ,1 parliculi\f (anélida(y in J

pa~('1J¡ar alstncL bu! the ~ubj<~t maHer .:tl1d the ovedar with st,\l~dilrd p{)htu::al scie,lR~,

Ct'flt'ral Knowlt;:'{ige anJ Particular Facb

thcpnes lo c;lU~:ll o¡' eh'nts <'in ,KCOU¡'\t oí the n2:¡"i,lES ftlf nr ot sudal ih.:t10n,

to know no! onlv ,""llar ca1.lsed the agent to pedorro some act the rN~~ms fnr taking the adion," Geertz OQ73:17)

",,"rites that "it is not in nur ínterest tn blead, human behaYÍor ¡:m,1perties tha! inkr~~st us beiore :~e begin ~o e>.amíne it." ,

\\'ho "mterpret"tlOfI seek to IlIummate the m-aspects human Dehavior by empl()yin~ Vcrstdwl1 ("em-

understanding the meaning (lf actíons and interactions fmm o'\'\'n points oí vi!.',>v" ! 1975:81 n, interpretívísts

o explaín lhe for intentional adínn in .< ro the se! of in vl/hien if ¡g embedded. llwy Jbo

<-<Thf' m()st obvious are ({}'"' scope: al1 interprelative "ccoua! províde ma'\ll11i1!

nI intelligibíli!'y tn a set of sodal pradíces, and an interp:c-accouni of a set of should b;¿, VV¡! h practíct's or tradHions üf ¡he {~loonI97S: i , .

PerItaps the single most important operatlCmal recommenLÍatlOn 01' ís that should leam a dea! abnut a

prior to fonnu!¡üing Für only \Nilh a deep immersion and understanding of a subject can a

the hypotheses. Far of workíng-dass black

hím~

!hat that

38 . D€scriptive Interence

lt is ..-:w(Ía! to UIh::lerst¿¡n~j ,1 culture' deeplv bdurc fürmulahni'; hvpoth­ese~ or de~ígn¡ng a sysh~nMtic research project ro fínd ;m fl¡~S\\:e~, \Ve only \'lÍsh tu Bdd that evaJuatillg the v¿r<Kity of daíms based on meth­ods such a~ partk1p~mt observation can bt' accomplished through lile ot SClt'lltlÍlC vvhich ,ve describe, Finding the right .Jns\vers lo th<: \vrong questions b a futile ~Kti\'ity. lnterprewtion ba~(.':d on . is often a rkh souree oi ínsightful hypotheses. For in~ stance, RIChard FeMo':,; dose obserl/atlons of Congress (Fenno 1978), madt: Ihr~:lUgh ~vhat he caHs "soaking and poking/' llave made major n)ntnbuhons ti} the s~dy ()f that ínstitution, parüeularly by helping to fram~ bener que~tíor:s ior r:~gE'arch. "Soaking and pokíng," Putnam in a study oi lti'lhan reglOns (1993:12), ¡he """""'H~"h to marina te in the mümti,}¡} of an institution--to experience irs customs and its successes and its as those who Uve ~t every day do. This ímmersion sharpens OUT intuitions and pnwidti.'S mnumerabJe about no\\' thc institution tits and how it adapts to íts envimnment." [\ny dcfinition oi science thal dnes no! in~ dude ro(lm for ideas tne gem:ratio¡¡ of hypotheses 15 as fooI-ish as an account that does not eare aboul truth.

Yet on<:<.: . have been tormulated, demonstrating their cor-1"ectness ('Yllh ¡m estima te of uncertaintv) valid scientific in-ferenc(.'S. The procedures for inference foÍlowed bV ' sodal "","."L"'''''''', furthermon~, must incorporare the sa~e standards as those' followed other i:md researrhers. TIlat .~hile that insightfui ¡nterpretn~ tlOn or other we aIso ¡nsís! that sdence b essential foI' accura!e inteI'pretation. H we could under-stand human behnvior on1y through we wouid never be abje lo our hypotheses or c,'i.dence for them

our experience. Ou1" condusíons \vould never go bevond the status of untest:ed and our interpretations '~vouId remain personal rather than sCÍentific.

. ?ne ,of th~ best and most famous in t.he interpretative tra~ dlhon 115 Chfford Geertz'15 analysis of GHbert discussion oE the difference a h.vitch and a wink Geertz (1973:6) '\Tires

Consider ... ¡wo boys rapidly contractíng the of their C'ves. In ooe, thís ís ;m invohmtary twitch; ín the other, a conspiratorial s¡gn~I to a frwnd. 11)(> hyo movements are, as movements, ¡dennes.!; from an r.am-a­«¡mera, "plwnomenalístic" nbsef"ation oí thCIl1 ilIon.:, one could flor tdl which Wib twiren ami whkh \VilS ,,,,in k, Of indeed \vhether both or either was twitch or 'wínk. Yet the difference, hmvever tmphotogmphable, jx ....

General Knm'\'ledge ,md Partímlar Facts . 39

, "f*~:een a t\'\.Y1tch ::nd ..1: \\'1'nk is \-ast~ .;1$ anvont~ unfc~rtunate en()u~h ü) h{:tvc , t,he fírst tak.t~n Íllf ¡he s~;(ond knüws. '1"11,; \dnkt'f i;,. ,me:! <:.:~,,-1.~ communicating in a and "recial way: (l '; delibt'r;ltely, C.l to

:;;omevnc in Filrticu!;.¡L (3) lO ímp;¡rt d parlicul,lf mt.'ssage. (4) ilCC()!\:líng lO ,1

establisht.'d mde. and (5) witllout 01 Ilw rest "f thc Gnu'

As Ryle points out, dw winker has done 1\\'0 things, contracted ni!' eyelids and \vinked, .",hilc the t;vitcher h"s done unl)' one, cuntracted. his

'~clids. )10m un purpüse v,hen there exisls a public code in ''''hich doíng s(' counts as d conspimtnri;¡J signa] ¡C' wlnking.

an importmt conceptual poínt. Without th0 con' " given meaning by a uf communication, thc

study of "t'yelid contracting by human be­\yould be rneaníngles5 for studenís of social relalions. In Ibis ex-

limpIe, che vvhích fmm months of "soaking and fUg" imd cultural 15 to the propeT qucstion oi

;whether eyelid conudetion even could be "hvitches" or ".vin"s." The m.agnifkent importance of interpretdtion by this is

it provides new of looking al tite world-ne\v concepts lo considered and hypothe:St.'S to be evall..lated. Without deep immer­

\",12 might not even think of the right theories to n""""",nt example, if we did not think oí the dífference

n"'l»t"i'I"í"n twich(,"S and winks,. everything \vemld be losL l.f interpreta-tlCI'fl--Clr an:vthing t:!.se--helps us arrive ai ne,,\! coneepts or hypothe-

then it is ando interpretabon, and similar hil\'e been proven

made" rdevane theoretkal such as that behveen the researcher thcn need15 to cuallmfe ¡he hypothe­

It 1S in su,:h evalua!ion !.hat the logk oí . sc::lentific inference is That lhe best >';ay uf determin-

t.~e meaning of eyelíd contrdctions is through tite meth­descrihed in this book If distinguishing a tw1tch from wink were

pivota.!, we could desih"1\ a research ro do so. lf, for mstance, \Ve belJeve that particular contractions are winks im-bued W1th meaning, then other similar ínstances musí also be since a soph::lsticated device such as ¡his (a

once developed, 15 used Given this record instance ín \\'hieh this actor":; eyelid

contmcts. observE' vvhether tht: (,ther actor is !ooking al the right time, and ,,\'hether he rt:.'Sponds. V ... re cC'uld E'ven a series o.t ex­periments to SL,{~ if individuals in th1S culture are accustomed ro com­municating in this fashion, Understanding the culture, carefully de-

40 O('Scriptive Inference

~..:rib¡ng the t'\'énL i1nd ¡',winf.' il

tions ,viIl aH help us ask the rjght and C1.'e11 gíve us tional confidence in out condusions. Bu! onlv \vith tlw methods of entifíc .ínference H'íll we be able té' eY,1¡ua!~ the h\:püthesis and hthether 1! is corred.

Geertz's wink irÜt~rpret<1tion is best expressed as d causal hvhich we define pre·císely in seLi:ton 3.1); the hypothetical ~ fed oí the wink on the other polHical ador is tite other aetor's

¡he eyeHd contradion minus his response ii tlwrc were no ment {a lid no other lE the eyelid contraction were a \vink, Cimsi11 effect would be if ir \Vere a twitch, the causal fect h'ould bt' Z('TO. if v/e decided tí' estímate this c¿)Usa! thus nnd out whelher it \>vas i'l vvinK nf a tvl/irchJ, al! ¡he problems inicrenc\:" disellssed ilt in the rest uf botik would nE'ed in underslood it \ve \vefe to arrive at tite best ínft'rL'nce with rf><,n,'f't

[he lnterpretatiofl ni lhe oDserved Qeha,'ior. lf \\'ha¡ w\:" interpret 'lS winks wen, ¡llvoluntarv

(iUI" to derÍve causar abt,ut evelid contractíün the bnsis or i'l thL"Ory nf voluntary sodal intem~tíon ívould Dt' tinely ,ve would noi be ab!e to ,." ... ,,.,,,,.,,¡, know íLl

DL'Signing reseiln-::h lo winKs and tn be ~ , part of most polítkilI ;;dene!:' ,.p,,,,,,,,,,.,-,, methodologícaI lssue arises in much oi the ka! sdentists work We are often

polie)' d!:'dsion makers send messages ro eacn otiler, il 3 3 poínt <l statement dt appeallng to a dt.lIncstic audience? of cultura! norm$, nmycntions in and of ¡he hislory' particular adors, as wen as dose obsen'ation 01' • or the comrrnmicatioTl, \\fill aH help us make such 3n Or consider the puzzle in res€arch: Voters in Unitet1 seem to be by fll't tunling out O(

polI:;, But lvhat the lOw turnout mean? Does it reHect alienation \vlth the poiitical system: A ing v\·ith ¡he costs being dates or recent campaigns? Could rhe minimum age of 01' il

For dw;¡;:¡k<:' tha! \Ve e[l!lId dl1 Cj¡tit'fL'nt tl1t;l'!'V in which M\ c\'dtd <ou!l'action Vi,l!' nnt ,1 wink but stm had i\ Glusal eifa:! Oil nther a<:ton>. rol' e'clmp¡e, the twítch <,ould bave ht~~n mi:;mtl:rpreled. lf ,ve WHe a!S\) mlefL>sted in wrt.'ther ¡he wi¡h ¡he eydid úmtractÍfm inleuJcd tu wink, We' would ru:.~'ti to look for othH conseqUí..'Ilces of thís 5iHne theurj',

Generdl Knowkdge and P,uticular Facts . .+1

g tu {hl~nl h) the puH~;? The dt:<.Ai~i~)n of J ~jti?t~n nt..)t tt~ \'\~tt' .

.,\".ính or.l di}11nmatíc messóge. can ¡¡kan man\' ihings, Th,· "o­n:S6,feher shuuld alv:.lvS work hard to as!.: thc ¡ight que,;·

d then careful!v design scientiric research In fínd llUi what lhe . s ad did in fad mean.

\\'ould also like lo brídly <,(kiress the extreme daims (lÍ ,1 few nf ín!erpretation who argue ¡ha! lhe ~oill oí some n'setm-:-h

to be feclings <ind meanings with no observi1blc conscquences, ~ liaren)' a fair characteriza!ion of a11 but a small rnínority oí re­

in ,this traditinn, out the d.1ims Me mad\:' suffkít'l1tJy Ú'rú'­hat they seem worth addressing cAplk:itly. Likt' the over-enthu­daims oi CMI} Píisitivísrs, \vho tuok the untenahlc ¡har

,-"'rh-,pnt,,, had no in scientific rescilfch. For exam-

on that which ís (w"rt ,md maníft~"t('d ín observable' aets is l\ilh'l" in ~y Ih,,' le3sL The tn

Ihen, is lu tmdl'r-¡11M the actor'" ¿¡et has for him.

u<!,,,tl',,,,, mal' be corree! that 50cld! sdentists who f(){,'us on nvert a lOI, buí hOVl are we to kncnv if we

see? for exmnple, if two theorles uf self-conception have identi-' nMnifestations, then 110 observer will have sufiicient in'

obser\'atiot1, quantHica­

,md methods are i¡)adequate to the task dístinguishing h'\'o theoríes wí¡hoUl diHering

omsl'..'tluences, On the otIter if the hvo theoríes l1a\'c some manifestations that diJfer, then rhe rneth()(ls "ve de-scribe in this booK pmvíde ro dístinguish behveen them,

In practicE', ethnogmphers (¡md aH other sodal scientists) do Jook for observable behavior in order ti) among their 1heo-nes. mal' lmmeTse themselves in the buí they al! rely on various forms of o[,sert'afÍl1ll. Any further "understanding" of the cul­tural context comes din'Ctly fmm thc'Se or other comparable observa-tion!'>. re]evant observations ¡s not On the con-

the appropriate observatíons 15 pt~rlMpS most diJfi-cult part of a resemeh pmjed t especiaHy (and necessarily) for those ureas of inquiry traditionally dmninated. by qua li t., ti ve research.

Dt.'$Criptive lnference

2,11

Some qu,,!ítatívelv nrit::,nted resea.n:'hers 'Nould the positíon general knowledge 15 either necessdry or useful (perhap5 eyen bId as toe IXl.sb fUf ,1 en:nt TheÍr positi tha! the events or uníts stuJyare "unique," in Ont' 5<:.'ns<:\ !hey right. There was onIr orw French Revolution and there ¡s onI)' Thailand, And no on0 who has reaJ the biographkal aecounts oI' li\/eJ thmugh the 19605 can Joubt tbe fact tbat there was on!v Lyndon 8. Johnsnn. But ¡hey go furiher Explanatinn, accordi their position, is limited to ¡har unique event or unit: not \vhy tions happen, but vd,}' the French Revolution happened; not ,vhy ITlocratization sonw!imes seems ro bu! iArhy it lélgs in Thailand;

candidales win, but LBJ wun in 1948 or 1964. ""'c",,,.,

"Uniqueness," however, ls a mis!eadíng term, The Frenen tion and Thailand and LB! are, indeed, unique, AH phenomena, events, are [11 sonw sense The Frencn Revolution was; but 50 was ¡he eiectíon in the Seventh Dist,kt

in 1988 and so "vas Lile voUng decision of every one the mi1lions of voters \\'no voied in the presidential dectlon tha! V¡evved holisticaI1y, every aspect oisocia! and connech.~ in some to c\'ents. inherent ís part ir drl.('s not dístinguísh sltuations amenable to sdentHic

about whkh dre no! ,",U:;;'¡iVI't:,

shmved in theories of dinosaur extinctíon in chapter Leven. studied paying attention to

n,."c,,\",."~""'''' implkations (if theoríes to ao::ount io! them. The real that tbe ¡ssue 01' uniqueness mises is tbe problem

oí complexity. The point 1S not \vnether events are inl1erently unique. but \vhether tne features (JI' social reality tliar ,ve want to unde:r:­stand can be abstrac.ied fmm a mass 01' f,lds, One of the first and mosl: difficult tasks oi in the social sciences is this ad of simplificll­tÍfm. H is il tasI<. that makes as vulnerable tí) tIle cntidsm of oversimpli-

and 01 omítting sígnificant oE tite situation. Neverthe-such simplkatlon i5 inevitable al! researclleTs. Simplification

has been ao oi every knovvn \york--qwmtita­tive and qualitatlve, zmthmpological and economic in the sodal sd­en(::i~ ¡}nd in the natural and physical scíences-and \v111 pmbably aI-

General Knüwledge amI I\¡rticu!ar Fa"is ' 43

simplífy. lm:leed , rhe

] L~ ~till nd tf¡ilf in tttc , bcttiX'cn thi~ tínckc<;:! d,'.;,-npti<m~ ,md tite tlW~t ab,;: mct . mi'! )Jü dt'SCT'iption, no matter how tl'uck. dnd no ex-

1.. ¡ t f'> .... ors jnlo tt n,'l1:W:; nO matter uO\\' many ('xp ana ory oC. ~" '"

capturíng the full "bJoomíng an~l buzzm~ re¡, lit y oi' lhi:

is no choíe'e bu! to 5ystematlt. sr.,"}) to useful As "n eronomíc histonan has t ' ¡'} t11!' "xtreme oí MISll:' on " .... ".

p --¡enn' 1S the H:'TV ~t .. .. "

tu ihc ¡¡lmiesssness (lf balladcl'rs" p(lssible, símplify their » _

attain <111 of toe ridmess or.. and euI-sdentísts mav use cmly a few oí the hlstt)r~' of stJmc " '1' . l'nf"rt'l""'"'' N· r'~\'''rthf'less, rich, unstrw.::tured even,s 1 • ,- .",;0, , '" '- ....,

thc historie,.! and cultural context nf the phenomena v;ant tu deal in a simplified and sdentifíe' is usu-

avoiding tha! afe simply v,:nmg, Fe1-v trust the of a about revolu-

elections if that knew little and Reyolution o, the

belie\'e th"t.< where (u",,,p'r;;¡¡ and

;:l\'!,eSE;E'S of evenrs as \veH as "b(lUt

ant to be nnd her mav ,/arv trorn

bu! both are likeiy in' be bdng opposed to tJ¡p l}L'~t ti

Much 01' what politkal scientists do is describe events svstematícallv. care ,1bout lhe

ln-filc

Union, thl? n.'actions "uf the in wuntries to the , ized \var to drive lraq from Kuwóit, the resu!ts (lE the lm~st CO~-

• .' l. l' '~" del tüc A1'd ·h<>v relv on nohhca] so-gresslonal eiectlons In tr1e ,r'lhe ,J a ",,y, ,rt, <") " . 1:"

cntis!s fUf th,)Í tened d more L,1Il1F'reiwnsin~ d\Vart'ne$$ the reiatiol1ship bd\\'een these ana other reh?,'dIÜ ('venb--en nuy i.md historicaI·~···tl1dn is found in journalistic accounts. Our scriptions oi events should be as pn>cise and systemdtíc as This means thiÜ ,vhen ,-\Ce are "ble to find valid quantitalivc 111

(Ji wndt \Ve ,vant to ,ve should use them: Whilt proportio Soviet níticize gnvernment polky? 'v\lhat do pubik km in Jordan and Egypt n:veilI about Jordanian and attHudes tO\.vara the Gulf \var? WhiÜ pern>r1tilge of con¡;ressíonaI cumben!s wen: reeleded: "

H precisíol1 f ir cines noi neCf'SS,ll accuracy, SÍlKt' qllimtitative indixes that do nol

dosely lO the ('oncepts or e\.·enb tha[ '\-ve pur¡:x)rt [O measun: can lo seríous nk>i1SUrerrH'l1! error and probJems Enr ctlusnl inference section 5, l.!. Similarly, tlwre dre rnore and less precise ways ro des evenls tha! cannot be quantifietl Dí::;;:iplíned qualitativc rL"St'il

try lo ana!yze constítutiOl1S and L.1\vS ra!her ¡han n";'n""

porl \vnat observers abmJt them. In doing case studíes ot ment researdlers dsk ¡heir ínformants trenchanL well-spec questiuns to \vhkh ans\vers 'Nm be relntívelr unambiguous imd

ioHmv 1.1 p on oH-hano n?marks made b~' i.1.t1 • '

relevan! Case siudll.:,'S are ~ssentia¡ for scription, "nd are, fundamental to stxi¡t! sdence. H 1S less !o Stc'!:k lo ,·vhat we have not described ",,,ith a

of precision, oí comp!ex events ¡s no

(Ir ínt-ernational

zmd cmmter others' 1s ¿¡ne! expectatitms may playas importi1nt a parí in fnr sta te benirvíor, 1\ pllrpnrted

oí world th8f ássumes ¡he absenn; interaction fp,Ktions \viH be rmJd. ¡¡;ss usdul lhxm a carefu!

OH evcnts that we have reason to believe

interconn\!cted,

f th~ otten (lverlooked "d.\'antages (lf the in-depth is that the dl'\'elopmen!: 01' gnnd is (:"W'

lo z:ood dt:S(T¡ptitm r,1tht'f than \<;ith íl. Fr;,m-e s!';dv Jmund 'an "",plan"ípfV t¡ucstkm mdV kód lO more

r~leYant desGiptiun, en,!! if the studv 15 ultinhlhA:' in its attempt lo províde e\'Cn a single yalid infereI1Ct'.

ve case sh¡dies \ve argm~, yíeld \'a!íd cilusal ínfer-in thc Test oí this bnok are used,

the\' oftcn do not meet the stan-

!\!mpÍlasue here so muer. as it 1S a w,y\, of non in descriptivc case in su eh a vvny that ít

~~The HterJhJh.~ un (On¡p~lr\ltivc l,:.-'¡:;t' ~~tthjh.~ is \'(.1sL Sorn{'> <.Jf tht.' b~;~t ~lckhtú.~nnj \vorks

E.:kstt'in (1975), L.i¡ph"r! (19711. ,md Colli<>r (lqqn.

46 Descriptive lnference

bt: used [,Ir desGíptin: ('jí "::cn¡s,ll in{en"l1('l', i\luch \'úluablc ,,' "bout doing con,}';lr¡ÜlVe t'd"'e often igmxl!d,

2.2 lNFElU,:'\;CE: TllE SCIENTfFlC Pt:RPOSf 01'

DATA COLLECTION

lnference is the process oE using the facts \Ne knmv to leam aoou! we do not kmnv. The facts \Ve do no! knmv are the subj~:cts oE research questions, theories, anel hypotheses, The fact::; .ve do form our «(Iuant:ítative or data or ol-;sef\'iltions,

In partkular vve must sornehmv avoid being oven'\'

the mas5ivc Di and actual obs0rvanons rhe world, solution to that problem lk>s .... ..,,,'-',""'1'\1' the search knmvledge. That the bt.>st sdennfic way organize fact" is as implkations of some (Ji'

sis, SLientific simpliHcation involve5 tIte productive choice of a (or hypothesis! io ('\-'aJuare: the theory tnen guides U5 lo the of those [¡¡ets that are implieation:; oI tneo!): Organizing facts in oí observabie implications ol a tl1ro!)' pn1duef."S tant and beneJidal results in designing and conducting \-vith tnis criterion for the sek-ction of fae!s, we can qukkly that more oDservations of the implkations of a theon' \viH onlv evaltmting tne in question, Since more info~1ation o(tlús cannot hure such data are neVi"r and the pnlCt'SS oí

Viii;' need not have a nor musr our theof\r remain throughout ad. As \'\'ith and tlle egg, some is befare dat" colh:dicm and some data are Í){ftore any

l;"'vtruv,l/c O}1 research teH us tbat we use OUT data lo 'test E:mm the dala may be as important " goal

evaluating prior theories and h:'Pútneses, Such iearning involves organizing our data into observabIe implications of the ne\ This is (omrnon early in many researdl

somepreliminary data have t1€en collectt't1.; after the data coUa'tion Hum eontllmes in ordef to evaluate

new throry; VVe should ah','itvs trv to continue to colled after the reorganizatinn in ord:~!' lo 'test thc ne .. \' ill1d thus a using ¡he same dala to evaluare the that ¡ve usted lo develop

.¡ For Cnon,bs O'it4; ÓemO!1stratt;,'C! that vírhUlily ever)' tlSt'ful d;}t'a-(n¡¡'l>ct~

1 nferen ce 47

en1pha:-;.is un ta(-t~ dS obSCr\\lhlc \.)1

S makes the cummon gmtmd betweeIl the quantiLltive and styles ni research much clearer In f"cl, once we get past G;ses or units or records in the usual ,'en' narrO\v or even

sense, W€ fealize that most studi2S poientii\lly pro-very lurge number ot observable irnplícations fOí the theüries

- ated, yet many ni these observations ma)' be overlooked by Organizing tht: data into a list of [he obsen-

ilLctlJlV"" oE a thenry thus helps rcyeal tnt: e&.sentía! sdentifk of rí\uch qualHatin' rcsearch, In a sense, we are asking the

\'Irnn 1:;; studving a evenr-il aSK: "H nw

me out the wa}' it die!, ",hat else world?" These addítional obs,,'fvahle

dedsions., bu! thev migh! also be found in other dedsion being shKlíed: fnr when ii \Yas hmv ít

E:, ht.)w it \-vas jus!ified, Tne cnKj,,! maxim ro guíde b'Jth Hit'-and data is: search fOf mOl"{' observable impJka-

ir L'i productive to be observed,

ítems ipr vvhích data

eme additíonal datum v,-m to evaluate a then to

time, and díort constraintsJ it i5 wor!h or otller observ"tíon might be 'abie ot this {or sorne pther rdc\'aniJ be obvlous thar it v'l'ill not

parí of the simplification data 1nlo observable implications of a the data. VVe can think ilhout the r,,1;\-' materia! of real-

into "dasses" thar afe made up (jf "uníts" or v·,rhkh are" in tum, made uF' of "attríbules" or al'

N The da.ss might be "\'pters"; the units might be él sample vorers" in s('vera ¡ distrícts: and ¡he attríbutes or

Desoiptivc InferelKC

runnel tt?sts, ít wuuId be lrrde\<anL l¡ü\v~'Ycr, sine'e L'Ydl

dust can (au~e aplane to V"lcígh more and tbus use more e ¡'uel, modds of this sort áH;: importanl to ¡he aidime industry ilnd bet'n huilt (and swed millions oí doH.lrsL

AH models range beh\<een restrictiy{: ami unrt'stríctive '\ ' stridive models are dearer, more par::iimonious! ilnd more but they are also h:ss re¿distíc lunless the '.vorlJ is ous), Modeb whích Me unrestrictive are detailed, contextua!, rnore realistk, bu! lhey dre also leso, cIcar and harder tl1 estímate precision (sce King19B9: section ') VVhere on ¡his continu dwose to G.mstruc! a mt.xJd ,,1',"""'1,n,'!C

be anJ on the

C01.mt will actual judidal tiOl1, the induded,

Vv'hHe

be an abstl'adion or Sillee undersl:auding book ís as nmch ivhat is lef!

researchers often use verbal in our discussíon

verbal models, JUSI as \vith models of studies of the French Revoiutton, our research not be confllsed \vith are dear statements of

otherwise.

In addition, we often us to dÍscover íde,ls that we vvould no! have

vVe assume rhar readers have had no braic

in tl1ese "nd Just because quantitaJive are probably more familiar our dnes not mean that are any h¿tter al the of scientific inference, these models do twt more to than ro qua1ihüive r,""f.'p,}'{'!>

the modeh~ are useful abstracticms (li ine fl>search to ;vhich th(>y applif.:'(l, 1'0 ease their intmauction, '.ve introduce aH i'\Igebraic wíth ,'crba! followed abo'>:

notation. skipped without

:\ Form.1¡ \Jodd uf Data Cóne,,'tion < SI

Ul DAT,·\ C(}LLfC,'TlU'.

pn!st:ntation pi descriptive ami CllLsal ínfér-(l primary ni social "cience fl~se"rch-we 'NílI d(c-

ior ¡he dMa tü be cdiectt'd dnd for these model is quite simple, but it is a pt)werrul [.)ol for analv/ing oi inferencc, Our aigebr<lic model will not be <l" formal as

hui makes our ideas dearer and casier ro collcdiali, \ve reft.'f to a v:idl' range oi indud­

partidpant obs(:f'\'atí{)ft lnlensín' intervicws, survevs, historv recorded 1'rom SG:.llTeS, ran-

"v,,>",,,rnnn' - Z'onlent and anv other vu¡¡' al!

h07C the data IUt'rt~ i~n:i.lted {Uhl lv.r ¡:¿: ¡ce carne ro of information tllar we should

observable impHcatíons of our ¡llenIT. !t muv help us n¿w research question, but it \vi!l be nf no \.1st: in

CjUt?StiOl1 if H is not an ohservable ímplicatiol1 nI' the ques" eek 10 answer,

data \vith ¡¡¡lit::; .. and llb:'.;{TI)aticrn~, 011e ex­annual income of eaeh oi tour people, The data might be simpJy four numbers: , 521. ana

ihe more case, wc could label the ¡ncome or four 1, 4) as ltL, .'12, }!:\, iind JM- One variable

tor t\VO unstructured intervÍl'\\'S takt' on the Yalt1E.'S "par-," "cooperan"e," or "íntransigent," ,:ma mighí he labeled .lit

In these cxamples, the '1\7riaNc 1S ti; the U/lits are the índividual ¡he are the values nf tbe ,'ariablcs fuI' eiich unít

for d ollars Uf oí The y Ís a

suppose we ;lrc ínterested Ín since 1945, Before \ve colled om data, we nc¿d ro decide ",hat

VVe could seek to understand the size issu€ area Of

nil.útionl in 1990; changes in the size o!' intemationa! activity sinee 1945; or cnanges in the size distribution of

Descriptíve lnference

intern~1hon~1l organi?atiol1élf dcti\Ylty ~ínt~e 1t~tt3" Vdriablc5 "divity (uuld índudc the number C>f únmtries

ing to íntem"tionaJ orgimizatinn:, ai a g¡n~n time, tne number of by infi'rnatíonaJ organ:z¡¡¡¡,ms, Uf the sizv:,; 01' budgets

staffs, In these the uní!::; oi ,1Ila.lvsis woukl indllde ' tional orgimizatiüns, issue Meas, cmmtry rnemberships, and time ods su eh as years, five-Yt'ar periods, or decades. At the data slage. no formal mIes <1pply as to \vliar "ariab!es tu colket how unlls ¡here snould be, whether the units must outnllmb"'f tne

or how "veH varía bies shonld be measured. The oHlv rule j judgT1wnt as 10 ,",vllar \yill pmve ro be ímpurtimL \Vhc!~ \l/t'

clearer ide,) nI' now the dat.a will be used, the rule becomes fínd I'mmv implkatitms {lf a theorv as As \Ve sized in chapter i, rese¡1rch ca;, be used both tu priori hypotheses or to not previotlsly but if ¡he latter l<eh' data must bt' coHected el/aJuate these

!t shnuld be ver}' dear {mm our disn.lssiol1 thal most 'works "case studics" h,1Vt' l1umcrotls \'ariables measured over manv !ypes of uníts. Although re~arch rarelv uses m~m,~ handful oí cases, the total nmnber 01 observations 15 general. mense. H ís therefore t."Ssent1¡,¡ to between !.he number cases and the number of observations. fnrmer mav be of soma

but onl\' the l,ttter ís oí .

of the number of obsen'alions comes fmm \vhere n IS ¡he r1umber of persons to be

\ve apply it much more genera!!)'. lndeed, \)ur defin:ition oí an vation" coincides vvith Hanv 0975:85} defi \'\'hat he c"Hs a "caste." As Ecksteil; argues, NA studv nf six dection.s in Britaín mev but need n~)t be, an 11 "" { studv. It dlso bt:~ an n '" 6 . H can diso be al1 tI := 120,000,000 st~d''V. It

,,11 \vhether the subject (lf 1S svstems, eJecÍions voten;." The "ambiguity atxmt what constitutes a~ 'individual'

on be dispelJed not lookíl1g af concrete entities but the measurt's made of fhem. On this a. 'case' can be defined

as a [or <md interpret only a gle measure on any pertinent variable." The onl)' difference in usage is that since Eckstein's artide, schola.rs have continued to use

Summ .. rizing HistorÍCal Detdíl 53

to refe!' ü) J. {un case stlldy, \\~hh~'h ~bB has i1 faidy ilnr~re­Tht'ret'el't', wherever poc;sible vve use lhe \",¡ni "~cas\'O

t'l'riters du ~md n:..'Servc ¡he WÜfd "obseryation" tu rdt;'r lo of one 01' more \';¡ríabk's on exactlv one unit 1pt in the rt'sí uf this d,i1pter tn shp\\, Ix),,,, . ¡11,:(' nd unHs can inemase the daríty pi nur thmkmg abou! re­

dl:~iO'n even when i! may be jDilppropriate ro on quan­me,~Ul'es to summarize the inform,ltíon at our disposaLTlw

\ye is; tI O\;' CilH ,ve milke de.;eriptíve infercnces alxmt aS it really \<\'i'lS" without getting ¡ost in a sea (,í ílTelevant

. how can \\'0 S(lrt oul ¡he essential from the

data are the fírst in aoy :mm-

of the data. Summarícs JesL'1ibe \Vh¡lt mil}' be il CUHount but are not dínxt!v rdated to infcrenee. Sinee \\'e are ulti-

in generalíz:1.títln and exphm;11ioll, ;1 summarv of the to be explained is usually J good pl;,ce to start but is not a suffi-

oí sodal sdence seholarship. is necessary. We ean l1ever tell

()f events; ít would be whkh events "vere o.m"tnKf

his be irreleva¡¡t h,

h"lr or whe!1wr he friz'd tO

Cood hisloricaJ writlng aHhough it mil\' Hot be verbal s~mrmi1ry nf " weHer uf historícal

SUmJm¿ln;él!l,g historieal detail is a stab­oí data in abbrevi¡ited formo liS

¡he appropriilte of the data .in a For example, Orle statlstic ¡5 t:he mean, or a\'crage:

i!:= ¡¡ + y: + .. , +

, Form,,11v, for " 5<?t oí' n units un whích il \\lri,ll:>1<' lf is me;\sUfL'Cl (V;" ,,1¡,el. ,.1 ~tdt¡st¡C Ir 15 a rea!-\'~lut'{l íunctinn déined as ¡'ol!ow!>: h '" [¡(!I'.) '"

54 . Descriptive Inferem:e

;"vht):!'t" 2:~! '.,.th is a convenient 'lvay (JI u~ + t/~ + !f~ ,~.

oth~:r statístk is {he Jabt'led U",y,:

ihe &,mple mean of the tour íncomes 1mm the example in $22,000, 521 and 554,292) is 526,573, The sampie

mum ls $...114,292. \:Ve Gm summanze the originai data C'onta numbers "vith these two numbers representing ¡he sample maximum. \Ve can "iso other :xlmple characteristics, th(~ minimum. mode" or \/ariara:e.

Each summary in this modd reduces aH the data this simple example, or our knmvledge of sorne asped of Eu history in the other! ro a single number, Communicat!ng v;ith . ríes is often easícr and more meaníngful lo a reader than

data. Of course, íf \Ve had ~;nJv four numl..Jt:rs in a data then ít would make Hule sense h) use five different summaries;

¡he four original numbers vvould be simpler, Interpreting a tistic is generaHy e¿¡sicr ¡han understandíng tht.: entire d"ta sd" out

onlya few. lO5t: information dt>scribing a Iarge sel of numbers

WhaJ rules govern the summary nr historkal detail? The first thilt summarlt'S should OH the ou/comes ¡hat TUl' !I.'ish [o

H we were intt.:rested in the 01 the ,werage tional v;e would not be l,vise ro focus 011 the Uni

but if vve were nmcemed about the size distributkm of l1at:ional from big ro :;ma11, ¡he United

be one of the unirs nn "vaich Unitcd is not a "01--'n~"".·,,¡'"

tant o:n0. In s!i:ltistícal terms .. tu ganiza!ion, \ve wouid exanline mean bershíps, etc.), but to undersi;md the to exarnine the varial1CC. A st·rorh.l, obvious precept is

mus! the al aur In quan terms, ihis rule means thar we should ahvavs use [ewer summary tistics than units in tht: original data, othen<;~ise, ,ve rouJd as easily sent aH the origina.! data \vithout summarv at all:' OUT -

should a150 he sufficicnHy simple tha! it can be" understoüd by . No phenomcnon can he summanzed perfecHy, so stal;d

adeqtl.1cv mus! depend on our purposes and on tht: imdiE'nce

e Thí~ püint is dosdy rd"teo to tlw cnnn:'pt of índetermÍnan! research we díscus& in section 4. L

we

Dcsuipün: Inierence 35

paper nn t\'-~Jr~ and aBian(t.>:; Íj:clud~ l,jatd in-obs(~r\'ati()n~, In Sl1ch ,1 p,-,¡:,cr summanes ot the d;Ü,1

numbers might be justified; ('\/en for an expert te indicators míght be incomprehensible \~'íthout sume iur-rv. For a ¡",eture ün the to an under¡,;raduatt' dass, might be 5uperíur.

'e inr'erenc(; 1S th(' prncess ol1.mderst<mdinf.~ ,111

011 the hasis oí a sd uf oDsery;1tí\ms, For in ,'ilrÍ;üions in the distrie! "ole for Lilbnm. and Social Demoaatic in 13rítaÍl1 in

hav\;' some to ho\ven,e observe is 650 district eJections tn the House oí Com-

in thilt \ve might think Ihat we \\'01'0 direct!y tht' dec-

of the COl1servatil(es bj recording theír share of the vote t and their oyeraH "Dare ot seats, But a c0rt .. in degrce oí ran­or unpredíctability is inhercnt in politics, as in al! ni sociallífe

i of scientific inql.liry.7 5uppose that in a sudden fit of absent-m:ss (or in deference to sodal 1he Brítish Parlian,ent

to electlol1s en~rv 1979 and suppose thal ¡hese elediOt~s were indep(~ndent nf one another. Even

"""!p,,¡,,',nn support foi' rhe Conservatives rernaíned constan!, replkation \Vnuld no! produce the same flumber of votes

in each dístrícL The \\'eather

cl(enrs might happen In the ínternational nr SGm­

might reaeh the mdSS media; even íf these had no long-term "".hUU"',,"' the'í' could aífect the weeklv results, Thus, nlJmer<)Us¡ l'l"iUIFatr)r\i events cuuld effed slightly diffe~ent sets ni election rdl.lms.

.," Our observatioI1 uf any one election v/01.!ld not be a perfect mensure oí" 5trength after aH.

As another example, suppüse we ilIt:: ính:rested in the of con-Bíct between lsradís (polke and rcsidents) and Palestiníans in commu­nitres on the IsmeH-rn:cupíed 'vVest Bank of the Jordan River. Offidal repons by both sídes seem suspect nr are censort-d, so ,ve decide to conduct our own study .. ve (',m ascertain the level conflíct in diffcrcnt communities íntensh'e in!er:ieh's or

Dest.TÍptive lnfurence

tion in farnHy (ir group ~?\-ent~, If \Vl~ do this tor ~) \\'eek in eZlcn mtmíh', uur condusiol1s "bout the h'\'d uf l:on¡'!ict ll1 ('a eh onc wíll a {unction in part of \Vhdtevef d1dnct' events occur the \veek we pen to vísíL tven if \ve conduct tlll' study o\'er a veal', \\'e stilI wiH perf,,'ctiy knm." tht.' true lt'VE,j nf contlict. \.'\'('n tlwugh nur unccrtainty abl'ut ít \vill dn1p,

In these examples, tfte variance in lhe Conservittíve vote 3CHlSS

tricts or the variance in conflírt between \Vest Bank communities be conn:ptualized as arising from tiNO separare fadors: sysfematic twnsystemafic differences, diHerences in our voter ínelude fundamental and chMilcteristks uf the d such as differences in in il1come, in campaígn or in traditional for cach of the In hypothetkal weekly

{1f the sanw would per-but the nnnsvtematic diffen'nCí~ such as rurmmt variations due

the ,ye3!her, ,,-'o¡11d vary In our \'\iest Bank dif-' \v¡mld indude the deep cultural differcnces bet\veen Isradis

and Palestinians, mutual kmrwledge of e<1ch nther, and pattems ni resídential housing 11' '¡.ve could star! our vatlol1 week a dozt'n differen! times, thesc ""'CH''''''

tween communities would contim.l0 to ;tffL'Ct the observed level of 1201\-

mct differences, stlch as terrorist incidents or instances of Israelí brut;,Iíty, WQuId noi be predktable ilrHl \vemld nnly aHect the \vt"ek in whieh they happened to oceur, \"'/1tl1

inferentia! techniqth's .. vve can leam about tne na:-differences even with the arnbiguíty that occurs in

one ser ni' real data due to nunsvstem.1t¡c, or differences, {he

is no! more and our attention shnuld not bl'

''''''P\,',,,,. distinguishing be!v\'een ¡he 1:1"'015 an essentíal task oE sodal sdenc€, One to think about ínfer-enee is to regard the data set \ve compile as one oi many possible data as the actual 1979 British elt~Liion retums constitute only one of many sel:; of results for dífterent hypothetkal

on whkh electicms coulel Í1ilve been or as our one ,veek of observation in eme small Ís one of manv possible weeks,

In inferem:e, we seek lO understand rhe dl~gn~e tü whieh our observations reflett either or outliers, tíild the 1979 British eiections üü:uITcd during a tlu epidemic ¡hilr thmugh 1.vorking-dass houses but tended to span' the rkh, our obser­vations might be rather pOOl" measures of underlying Conservatlve

Des.:riptive lnú:,rence 57

be(\lUSC thl' nt)n:~\'sh:mdti<'> chan('c eÍt:D1t'ct in thf' tend ro ()','erwhdm Uf disí;1r; tlw ~YSh~mdti(' ek~l1wnL jf our w0ek bad nCluned immt'jí;:¡tdy atter ihe Israelí ínvúsÍ(m Leoan01L \\"0 wixlld similarlv ~o¡ t";ped result~ that are

ve o! whi1t USUdlly happt:ns on thc West Bank. polítical .vorld Ís theoreli('¡,l!y c"pable nf producing multíple

ets tor everv pmblem but does no! foHow Hw n(~eds oí sdenlists, \lVe are usuallv on11' fmtuna!e enough In übsern' (1ne d,:¡t,l, ror purposcs of a modeL we wiii lel thís une set ni data bl'

eme \'ariablc lf {::oa\', the vote fur Labor) mcasured o\'t~r t1 ::: 650 tmHs ¡districts): ;¡¡, ,11;, ' , , , yry (for example, v: rníght bl,'

peoph: \'oting fpr Llbor in Jístríct ¡), Tlw set DÍ ()h~i'rZIt1ii,});~ we ¡abe! Ir i5 a nvfi:cd ¡mriablc. lt~ v(¡lues \far}' UH'f the 11 umb,

a:ddítion, wc: dcfínl' '¡ as a rrmdom i'firi;)/ilc oenmse ít hlríes ran­aeros:; hypothetkaJ rcp!ícatíons of the same electinn, Thus, ys is

the number of people v()tíng for Labor in dístrict S, and Ye; is tlle Tan­dom \.'aTí¡¡ble representing ¡he vote acros~ many h:T1o¡hetical cledions tilar could havE' heen heId in dislrid S l.mder essentially the Silme (on­

•• ¡¡finans, The observed votes for ¡he L<1bor par!y in th", une \ve observe, 1f¡, 1t2, ' ' , , l/I!' díffef ilcmss constítuendes because of sys('em-ane ilnd ;¿m¡iom faetofS, Tha! to distinguish the t\vo forms (lf "vilrí-

" ",ve otten use the ¡emi r,',lli::cd ,'{"iahle !o refer to ,¡I and nmdorn """FIt1,"¡,> to rcfer ro y

same lo our qu"llrative We wüuld han' no nr dcstre of quantifvíng the h?\'el of tension be-tViietm Israelís and in part becatlse ís a catee! iSSllC that 1nv01v0s the feelings of numerous indidduals, o1"g,\-nizational C'onflicts, ;md othcr fcatuft's, in ¡nis sit.uation, li'; ís a realíz(x'l variable which stilnds ¡he total cor¡-Hict observed ,:h;rim; PUl' \vcck in ¡he liflh cnmmunitv, sav El·Bireh," The random Yari<1bl~: y; represents both w!1al ,ve obs~;rve'ín Ei-Bin:h and whaL Vil!: couid have observed; the randomness comes fronl tIte variation in chanee evcn!s OVé'r the vveck::o ,VE' Cdldd have eh osen lo observe,'!

One uf the "111-dmn variilblcs bu! st,md,ud, ¡ermi-

although in distinguísh {mm nonsystematíc components in OUT data, in il case u'e vdsh lo

the san1e to f1Il th(~ ~}ther n~mn'tunitic's \-\~(> nligbt :-;tudv a Noh:' thnt tht~ rand~'~nn(~s i~ nnt exactiv n\'cr Jitfer-ent ~lctuai \\'et~k"L ~,>jrH.:t,~ bt.,th

ch¡:¡m:c eVPl15 ,¡na ~y,;tem<ltil: Jifferenef'S migh! accmm! ror ntserved Jiff",rellce", We therefor", <:re¡¡!e ¡he more ideal ;:.im,,!ioí! in which v.'e imagíne nnming ¡he wnrld ag"in wlth sys!em<ltíc feature':l hcld cünstant "na cllanee factors "¡¡,,wed tu \'¡wy.

58 Descriptíve lnferenn"

t;:¡kv ~1 r.)~ldCH~1 \'~~rjdi"~-e Jrtd e\tT~h.v: it" ~\'~h:n1~)ti( %t\1turt:\sJ Fur ~;?xa we l11ight ",ish ti' klK'\\' the ralue of the Labor \'ote

clistrict :; ({he average Labor q1te '(5 across " 1"r<2;e number oí h thdic11 eie~~tions in this ,:!istridl. SÍlwe thí~ b a ~v:4'~matic fe,1ture o'f

dectoral the ex:pected vaIne is of considerable 1:n. terest to sod"l scientists, In contr;;1st, the Lloor vote in one nb;;erved election, ys, is off considerably less long-term inten:st since it 1;; a tion of fea tu res mI;! random error. it1

The py'nt>r!p,1 \'¡j!ue tone fedture oí the titth VVt'st Bank El-Bireh, is

where n,} is the \.','!U('

acTOSS an jnfinite number oí hvpothetkal observe in ::;, El·Bireh, The mu ,vith a 5) the answer to tlw Cillcul;:¡lion (nlevd oí' conmct between Palestinians ano Israelí s) far community 5. Ihis is of om model {or 3 fea­tun, of the Tandom variable y" One might use file observai lever

,lf"" as al1 l'Stimate uf fi5' but beca use }j-, canfains man)' chance elements with information aboul ihis l:'>t~tt"'r

estímators .i\nother

exist bee sedion 2,.7}, featurt~ off these random \',¡riables

h, knO\'\,' 15 the h~\'d of cnnHíct in ¡he iTiynlY;c West Bank

One estím,1tor of Ji might bt.~ tht: average oi the observed leve15 uf con­flk! aeross aH the communities studit't1, y, but other estimators tm tMs

feature exist, too' {Note that the &anle smnmary uf data in our discussÍ(ll1 of for the 1'ur1'os0 oi inference.) Other ",,,,,,,,,,,,..,::tt'Ir

oí the random varulbles indude [he variance ami i1 vanetv of causa! introduced in sec!.ion 3,1. .

StiH anoHwr feature 01' these randnm vnnables that might be oi interes! is th0 variation in the level of nmtlict v:ithín a commu-

, ~ ()f CülLl~;f!" ti" tna \' bt> nf tn<!lendous H1ten~~t h:~ the peonh.~ t!1 dtstnct ::; h"1¡ tito! \'~~af<

ill\cJ thus ooln Ule mndom .Hhi sys!emalí..: compün<,nt~ (ji Ihí" <,veni be \~ur!h '''"''''''.'>''1<', Nt'verthde5S" \Ve shouid try lo Ihe mndnm ¡rom Ü'w WS-

Lkscriptive lnfenmce

the Sdme random variablt~l produce divergen¡ results, This the size of th0 nonsysh?n1<1tic comJ)t.1nenL

con11-:1unÍt\V t he \~ari:H'('0 (jnsreatj

¡23l

¡"tter ,knotes lhe n:sult ni t(J the random m a \Ves!

be ,,,'orse. ln an\' l'\'cnt, both researchcrs,

these isstles we distin¡::ui¡.;h hvo me""s of nmdom \'aríatíonn Thest' two ¿¡re extremes on

Although l1umbers of sdmlars can be !cmnd are comfor!able wirh each extn!lTk', mo¡.;r scíentists have

somewhere bet\vt"t'J1 the two.

ín variul1:S pdrts of th~·

::: A I)ctermi¡¡is!1c V'iúrld, Random vilrÍi¡tíol1 i" ¡ha[ p{\rtíon ni Hit: workl ¡úr which we 11,H'" En The division hctwz~t'n

abe ;md S[Odla¡;tíc \','1iatíml ís and dppend5 on n:l1at ,'aríables are ilv¡¡¡¡ahli;;' (l!1J indm1<,d in the Cíven tlw

variable;;, ti"" W<.lrlJ ;s

In the in ..

í: ~'t,~ (lY'91b} ror iH1 \:~L~boftltú~n cd' thl~ dhti!.Í\~tHHL ¡~ Ecnnomists tend t~} t~" ~.Jü~~r tí.' r1:ers;1{.·('bvt~ i, t'\' ht'lV.1S ':.>tzüisÜ!..-unt< dr=t3 t'lu~{;"r lO Pt.'r~

5p¿Cti\'(~ 2. 1 is aJs{'~ cornmo.n in the filAd ~)t engu1(h¿ring G1B~"d "<Judit Y control/" nave even dehaloo ¡hb di"tinction in tl:w lídd oí 'l"antum m€'Chanícs, Early propon{~ntsof P,:;'rspective:2 $ubscríbed to lhe "hidden variable t]wmy"

6ü Descriptive lnfcrcnce

i1: \\«~ tlSSUnlt>: under \'dri¡¡bIe~ femain UnkJH''''n,

!cl1\X' o<.'\:uro- ',\'hen ttwse unknO"'H explilneltorv "¡¡fiables in tiye 2 becnme tlle for the r¡md(1m vJriatíon in tive 1, Bec;w:;e nI tlw lack oí dltV ObSCiyable ímplic1tions with which to distinguish bet'ween t!wm, d choice bdh'een !lw two pCispecti >

d('pend~ 011 bith <Ir beJk( r,llnier ¡han on empírica! vCfüication, As iJnothcr ('xumple, wilh buth distinguíshíng whether

el partÍcular 01' sodal l"Vént IS the resuli of a

From tJw t~f¡'t'd ,,5

ser d da!a (nr t','en anuther case) to check fur the etfect eH' pilttt'n1, il is nory dífficult lO makt' ¡he judgment

From [he e\Jremc version of 2, ,ve can do no mort' th"n: :m l'vvnt ,1;, ",tochastíc (lr sys~

or írrelevant. A more realistic versíon 01 thís per­! 's corred nr i!1correct attribu!:ion of a

but it alkl\vs us some latitudt' in

what ,vil! remain aH observal:ions being tht:n tn cvidenc0 rcsul! torces,

random occurrence or ::>uC'p,'u f"r future research,

anei ti ve rest'archers. research is (ltten but mos! use as sodal sdence \l:Ílen ir Is ,lIso

thc randOlTl váriables from which observations are ate'el nnd to lo estímate their

the lüstoriG~¡ data coIledíons. Indef'ti, (me lTlark. of a

,,·¡1\.'~l~t the n...,>sohtuon uf tht> rntmt":rnus :rcrn.¡ú!ün.g ("(Hlt.r¡H . .'hrth.1n~ nf thb i!llfxH"LJrü

,md !lo; jo. ¡he' nature ni thc wMkl. H,w:ever, lhb dispuh~ in ks., ~,lthough uS('11 ro mw:h pi Ih", oi ~Kii\J ~:icnn~, is unllk.:'ly !o thc off iruen:~m't~ or prilctice of resEiltch in the sodal scienres.

Descriptivc lnfercncc

klHd uf Si)c1~)1 :::.cicncc re~e~1rch ~h()tdd :sab~ty the l-h1~il: In th~~ bnoky FinJing l~\~il..icncc uf

vvith some kinds oí <,~\'¡denct', bu! ir is 11\)

As a11 ._<tn~", suppnse th¡ü we aH! ínterestéd in ¡he ouiconH:S (lE CS-SovieT

betwet:?n ¡ 955 "nd 1 q9{) OUT ultim,lte purpuse ís to question: un .. :h~r whi1! conditions .md to \yhat t"xt¿'nt

summits lcad to in(reibed ('ooper¿ltitm? Answering thal a number oí díificuH iSSU0S of causal

nrnong a ~et c~f ·\-Vt.~ rest fi ('t

<,venle; Oí a cnmbinatínn of these measurement whicn summits \Ven' íoHoWf'J inncilsed su-

about tht: nm­in

and [he l'xtent !o lvhich Gmdítü1l1s in

<>v,..,,,,rt,.,hn1l,,, on both

have bt'i?l1 fulJíIled, rl~'''''·Hn"a leve! of in ea eh y'ear, and [O assodate it

¡he pres.ence or absence uf " surnmit as ,vdl as \,'ilh our (líher ,

V'/hat ,ve Obsc'íve (even our índices of ,.""""."",...,h",

tbe VCc1rs sl,llTlmit med­, \>vhether thc ~tlmmits and

ft'lated lO unIÓ' a¡¡{lther, ít could be rhaz the a::is(KidtÍun

randomne~s ciue tn funda­nr bad luck under

lmidentified explanatory variables sl1ch tmídentified variables incJude 'Neilther fluchwtions

the Soviet shifts in the b¡ll-aH of ,.vhich could d(Cnun:t ior in

If identified, these variables are altemati'le viuiabies that could be collect{"(l Of

bt'hn\'~ 1he ~S5Ul' ~s :!ktt of "",1,,,,,,,,,,,

i':(lt.lld k~lJ tn the CfH1V("ning üf surnn'lit ¡)1t:'etlngs" ~n ·~·\.'hk:h

(',1St', instead oí 5ummil mt'\?ting:' explainíng cooperatíon, antídp<ltcd cCJofX'fa¡¡on \vnuki t'xplain adual C'oojJ<'r;¡tion--han:lly ;¡ startling nnd¡ng if adors are ratínnaE

62 Desaiptíve Inferenct'

to nS5CSS tla:.ir Ínt'luence on rhe SUm!Ttlt outC{1nle~ lf un.idl~nhhecL t

variables may be trt,¿1tt"d dS e\'ents that (ould for the observed high Ot'gree of sllperpower cooperation. Tb evidence the possihílÍl:' th¡ü random 0vents ümidentified

variables! aeCollnt for lh.: observed coopc:ratiol1, we look al many orher vea ro,. Sine/! nmdom events ana procesS{'S definirían no! ¡hev v,·m bl' extremelv unlike1v t

differential cooperation in y'(;:.ns with ana w¡tl{~mt supe~po"ver mil". Once ilgain, ,,\"\:: are lea to the conc!usion that only repe¡lted in different CO!'Ü('.xts in ¡lüs (dSe) enable us to decide whe!:h

a Liue lo lhe transienl: or randorn proct'sses,

fmm nonsysternatk pmcessL'S i5 uf social a flu epidemic thdl

voters more ht,>;wihr than middle-dass ones is an eVf!nt that in one , replícatio

!-he 1979 dection would dccn:;lse the L~1bo;- vote. Bu! a persi.5tent ¡em (Ji" dass dífferences ín the incídence (lE a disab!íng illness woui

effect the an'rage level ()f Labor

ba5is of the víctor's or an accidental a debate a randorn factor that couid have

fected the likeHhood of bt?tween the USSR and the States rile Cold War. But ¡f thc' most effedive to vote!'s had be€ll Üw c0i1s1srent vktories of il factor

\v'hen

constants, appea15 ma]' behavior, but that fad does

mean thi'it campaign do 110t It 15 the off appeals 011 an electíon outcome thar Ís consta11t-~{)r, if is variable, it is VVllcn

Dad wCJ.ther ah\',H's Jc,ld~ lO f,,'wer votes fuI' favoring conciliatory ¡:miidcs) ..

In short, surnrnanzing historienl ís an important ,,' ,t.;>rl'Y'"u1

¡udging Descriplíve !nferences

our \.1It1, but ,ve mu~~ ,'tfs.c rni1k~" dtC\"Scrip-disting111shing bet\\'een ranLl(}I11 ~Jrtd pht,"~

. Knowing wlut happened on a occasi¡lD is no!' suHkicnt Jf ~T\"': ~nakf HO iD fxtratt filt"

CRliERJ.\ ¡:Ol{ !l:DG¡~G DESCRlI'TlYE l'iFERENCE5

s .. :dion, \Ve introduce threc explicit criteri,j th,ü are com­in statistics for judging methnds uf infcrences-

ami fileh rclíes on ¡he r;mdom-introduccd in st:díon 2.6 hu! has d.in'cj and

for and

\"Ve save research that is

"kme for toe remainder oi this

off infer!:'IKt? imd l,ve wi!! esb-are sometÍlTh.'S t(lO and smnetinles too smalL A.eross 3-

do ,.ve the ansvver 1m !1Z'emgc 7 If " ís said tu be unbiased. Thís prop­about ho\\' far removed fmm ¡he

()f the methnd might hut correct

oí

'. times one vvav~ sometimes the other. Bias occurs when there is a sys­tema tic error in the measure Ihar shifts the estimalc mure in une direc­Hon than <1l1other ove1' a sd of LE in our of conHict in West leaders h3d cn'dted

the studv"s results lo fur\ht:f theír llien the leve! 'oi conflict 'Ne observe in every cornmunity \-\,(luld be biascd tOlA'ard greater confIíct, on average_ If tne n'plications of our

hdped O!1{~ side "nel nul the othL'r fuI' instdnCt~. C,msen'd!1,'l"S wen' more nductant lo n,[c <'11

l"e¡¡"OHs!. (JI' oul" cstim,1tes mígh! bt' bilS€d fmm '/ote cmmters \\'no favor one pany Gn~r the

Ir, hnwe\'er, tl1(' rep!icated dections \".'ere Í1eld on ",,¡rious days in á mannel" Lmreldted to rhe variable we are ínterested in., any measl.lremen! v\'ould flor 01' al1o!her might {a'/or (me counts dne h, r"ndom of estima tes wl1uld bE' unbía"ed.

lr ¡he Britlsh electínns ;Y'::,re n,ethnd

intü the eIectinn c'"d,·.w,"

círcumstances [ha! jnduded tnt:'sc on the that 13 and cines mi! jusi exfst

¡he data alone. H makL"S littie sens,e ro &lV thar a data set evel1 though it b-e fiHed y,'¡¡h many errors.

In this we our ddinition of " tistical bias" in ;111 estimafor from bias" in an ch'fmnJ

of ¡he larter are to '\1ote--:} no! uncomm\.m ,"ubstanbn' bias uf

electoral As '.ve mav \vi3h to estimare ¡he vote oí the actual electora! ""'er,;""

but \Ve

are bias in the \Vhichever mean "ve are !lnve a sta!Ísticalh' unbiased estlmator,

ihat \ove use ror (1,;'<;crU,f1VP 111tl",'PI1ICF><;

estimilte~ tnat are ''''''+'.''''''''''''"-''

snnre up

to find <'5t1m211,,"3 tnat are VVciner'5 uf education and

labor in India 0991}. In tryíng to expIain the ¡evel of commitmel1t to compulsoJ'y education in india a)mpan:~i ta toat in atoer countrk"S,"

=/1

th,,"d ~1ttenda.n('e \\'a~ v,,~ert::). then 1i~tl~d J~ hH' ~p\.'cn

attcndal1i::e WdS f,H lll1C ,Ll\< CJoser "cruíÍnv figuI\: to l)t~ tTHKh hn\'('f.

for IV('

ti' use tlw i1'ú>[age il" da!;}, Ci ¡", lhe

{ur th~: ,1\'e[-

i> ¡" ;m estimatm pi u, ís a eXBmplt' 'titan ,1PP<':;US in formnl s\;\tlsti..::':' lc,¡s, bu!

are lht~ s¡¡me.}

1

Dt?scrlF'tivc 1 nrerence

V\'t: U~tL1Hy d{J not ha ve' an dpp(lrtunity to ap~:i:~Y out' l~tirnator ID' nmnGer uí essentialIy identíG1J applications,

ness is oí ínterest. but we would ¡íke mure cunfídt!.lKE' thi1t the estin-ubé' \ve 15 ck'St: ro the right 011E', Efficíency provídL"S a vva dis!inguishing amung unbíased estimators, ¡ndeed, Hw dfícíency ríon can "Iso hélp distinguish among "lterrlatjve e:>t¡mator~ sm,,!! amount uf t'ias, tAn estimator \vitn a bias ShCl\lld be roh:d out ('ven \Ylthour irs

is d rt'Íative c'nnr'PT>r

\"ilriance of lhe t:stimator across biased estím¡¡Í\Jrs, tlh> tht~

ter) tite esnmatoL A 5m,,11 variance is better becilllse our one esti ,vil! ¡)(~ clüser to the troc vahw. We are nui

prnr,plnr, for aH estim,:¡íor wíth il ,mee in !h15 situatíon wiH make it that the estimatl::' near the tfue vallle most of the estímates ,vouJd bt' dllstered armmd the v,'mn~ \alm:L As vve describe bdow, we a

in the case oi a SIl'hin (;im(}Unt of and \Vé

\.A/e are interes!ed in """.U'''« Palestlnians and lsr,lelis in

mid simíiar obst:'rvatíons f¡ve conununitíes, I! be ob,'íous t!u! <1 re better (ha 11 a into enen pi the We \vil!

enablc:s as ro "' Ji estimator 01' u with

aV{~fi1ge level 01' conmct found {rom studies in difft~rent communities on [he \Ves! Ba.nk. H

both estírnalors ¡ne unbiased !f the same mndel applies, the. estimator has a vmíance pf V(I' :~: al. That

,ve vrould have chosen ""har .ve thought \ .. !as a "typícal"

¡H)\\"l~\'t:L bz..' Jf;·l;.,,~.,t\:'d b\' rtHl."-..:(H11 \.'~u¡ai·';k\ ..... Ihv J.ndn~t'

'25, th~lt !S~ the \.-~~1r-í{3nCt> Uf ~h~" ,~·<:1rn-

ll~tin~atür j~ hn1trs

(Le" less diicient) thitn tlw estím,l!c when 1; = 2:" HelKe, ve the (}bvipu~ re~uH that Hl0Ti: obSer\\lhlH15 a:r¿~ bettt.~L

are tht' conJitions undcr whkh a more d{~tailed communitv would ilS

That is, although W(' should mon, observations tlw resources necessary te> cülJed

con-

being ('qua!. our ,}na¡y~is sor)';v" tl\iÜ the rmm: (lbs,~r-becdllse tand thus In

b sllen th,,! as the iltHl,b"f oí obsen'.,­dt'Creases to zero, and :he estimare to estima te d

thar

fur no otJwr re3S0n, than thilt we make corrE'ctíons for such l'dctorS

number of obsen:ationsl. \\l(, are then faced \",ith d tTade' th"t has additíonal nbservations internal tú cases in \vhich eaeh contaíns une ob-

oi on1)' (lnc lo our 15-observatínn re-

vvhich are easier to understand we could firsr sdect our communit)' very

of thc

the others, VVe míght ask él fe\v residen!s or look nt neW5-",->,."".-1'", io see whdher it .vas <1n average community or \<\'het!wr

bid~<i ~~~thn.üor ¡l[ /L huí it !~ nlcz~n~d5t-enr b\~\,-/~USf~ ,l::' H'i(~ numher nf urdts ¡n.:n.\l<;c .. thl:;':' cshrnatox dt~'S Hui irnprovc (OT in.:k:-t\:l change id aJO. An cst1rnah)t' C'ln dtS-t) tR" \~(H~s~stent out hk'lSt>d, F0f examplt\, y,-, S/n is biasezt hut ü is consisten! beG1U5f' Sh¡ ¡",comes 70m

as H appmaches ínfínity,

dnd tht'l1 \\"t:' rnj~i1t

e5timate d the tht, most difiícult pdrt nf the h'':: be \>\.~ry c¡¡refu! that bta~ Joc~ nt)t i..Tet~p in. ()ncc \ve ~~rE.' rc'"""nn,,,,~,,,*#,

confident that bí,,:; is \Ve could ¡''''cu" on íncreasing Tu do thís, we rnight man:,' \\'t.'eks in the community

ducting rmmenms studies, Vv'e couJd in!erview commu ¡e¿¡deT";, ordinarv

in a pf ¡he \vorld. further that there is a choice 01' tWt1

il case or a iictatistical countries of the !t \vould seem better to Bu! Jet US Sil)' that ro ít is necessal'v (ful' H:ilSl:.l l1S1 h.l use data to a UN agene)' {rorn tl1\:' emrncnts. Th('5e lwmbel's Me known tn have little re!ationship actual of since tht:,' WFfe in the Office imd based cm ccl11sideriltions oí

that \ve tuuld. make the cürrectioHs to the thiÜ \\'ould ,ve choose?

hvo 01' enree. Or 'c\'C

data best <lBs.ver our To take still another example, suppoS(' \Ve ¡¡re studying the Eur<r

pean Community and \Van! to estímate the degree oí tion of an th(;' ('n ti re that wiH I'esult

of i'v!inisters. We number oi mies forma!!y

industrial sector in ende thcse rules in terms of their strin-gene)', ;mJ then estim"lc the '1V\crage uf a nll,,, H vV\.' giüher data on HIO rules \vith sÍmilm a stríngency, the vanal1ce of out'

!udging Des.::ríptín: ¡l1fercno:~ 69

ir thc ruh:~ are ;,~:It1ü:(L L'nduubh:~Ji\:< thi:~ bt." ~1 bt~tt(~r nlt<l~

d<1ta un une rule as ¡he \"stim,ltor f()r strin.-as a v,:holt~.

wen,[, ihis prücedurc tu lhe retll

us h) lht' f\)rHul rule as

th\:.' Sic'I.:tnr under Íl1\'t'stigatioIl of rule ,lpplicatÍf)n, !kl\\'eVt'r.

varí,.tll1n in the extent lO whícÍl nominal rules He!Kc,. measuH.'S uf forma! rules might be

are actually en­bí,

ínsLmcc\ in fanJr oí' in a case, \\ t.' ,.,.'(luid {aee the bias-effídencv traJe-,1fT once ¡¡¡olÍ

múk,' 5ense io Ca!T\' "\JI thn'c uI' fnur ínteltsiv,' (,be studics oí ímp!enwnliltíon lo' the hctwecn formal and actual Onc, would be tu suh-an e:-;t1milt01' based 011 tlwsc fhree or four G1Sl~-¡es", bíased

les;; efficient-for the cstímate', bi1sed cm ltlCl G\:·+~S. il(1wi?vvr. ¡¡ he more crt'ative, if tu use (he íntensiVt,

work lor the three or rour cases to corred tht.' bi.as ni om ¡ OU-GlSe ind ¡­cator, and then In use d COH(y('ted version (ir ihe J nO-case indicdtor áS our e:-;timator. In this \'>i0 \-,-'oule! be

o11r intensive case stuJies \.vith think should bt: follo\.veJ muen more

sodal s,:íerKe, for case stud ¡es m.ldc by ¡hose v\' hu kmn\' a

\\'orld h'el! b onen t h,,' one in tl-n: studies mav upün num1:x'rs lhi1! are no!

¡he ni'\Í\'e researcheI' \In dará base (who be unáwar2 of the in which dectíon statishcs dfe in

a partinlbT h,cale i'md dssumes" thar thev han' som<:' reó] re!atkmship to the votes JS

the materi¿¡Is and !11JV 1,1(' abk lo com::ctions, ln sedions W(' wíll

how sueh choices míght be mih,1e more Our fonnal oí this in ¡he bOl< below shows pn.:-

how to decide wha! the results oí the trade-oH are in the e)(<1I1'I­

pIe of British electoral constituencies. The decisíon in anv w¡jJ be beHer when logic lik0 thar sho\\"n in ¡he

this lssue will ,llmost

\vorth more thal sometí mes exist betwt~en bld.& imd The mean the first two observatíons in any larger set nf unbiased obser\'ations 15

Descriptive InfercJ1ce

Formal Effkiency~omparisons, The L,rianct: ul t11<' ::,anlple y' 1S (:h::noh;~d as l'CY), and H1t' ruies for cakulating v,¡riances d"m \:"r1aI:>1.::s in the sirnple Cil9' uf fimd(im ~(,mt'líllg thi?

if \ve dssume thni r¡w ,'"ridrlü' ¡lCroSS hypothetícal li.catíon uf each distríct eh'dinn ís lhe same as t'verv ()ther dl1d is delh,teJ OC. tlwn tlw variancf' ni ¡he samp{e mean is

., ,:-.: (¡-

cstimator has v !;'sttmatm has \'ariance Un!ess ¡,ve

random-t'rror correctiüns to reduce the variance the statistkal

rut it doe'S n:ducc If \ye ¿id nüt ¡d",) u::,,,, the dficiency criterion, we \vould

no formal criteria for choosíng one e5timat~)r ove1' the other. Stippose "re are interesred in \vhether the Democrats ¡,youM

¡ucigíng Descriptíve lnferenct's 71

fandol1l sdectiort we Ch(1üSe survev rcspondt'nts inm1 ai! "dulí of v:hích has ¡m ot sdl,ctio!l,l Sup-

. , t some!.mt: else a!so did a similar with 1 cÍliYd'l:<

we indude these additional observaticms wíth ours tu crCalt' a based on 1,020 respondents? H the ne\\' observations

andomJv sdeded, just as the first tVH.'n!V,. it should be an eas)' to ¡ne'lude the additionaJ data \vitb mus: with tbe new obser­{he estimator is stm unbiased and no"v mueh mOfe efficienL

.;uppose tha! 990 nf ¡he 1 ,non nic~V\' obsen',üi(íns \VeTe dr"wn from the US i1nd ¡he otiler ten \Vere

alíe Dlembers oí Congres5 \vho Wt;[Z; induded in [he random had been dri'nvn. further

we found out th"t ¡hes!: additional observations were induded í.l1 data but did Bol know which OHes \'\'ere ,md thus could Bot

tht~m, '.Ve novv know a tha¡ Mí cstimator bibed on respondents would produce a overestimate of tbe likeli-i::haf a Deuwcrilt vvould '.vi n the natiomvide vote, Thus, induding

additiunal observations would bias the overaH esti-but it \vould also substantiaHy its effidency, V,,'helner uld ¡ndude the observations on whether the

in bias i5 the incrense in slatisbe"l oHh"u",,,\{"L'

it secms dear that Hw estímator b,1sed on the wil] produce cstimates

fre<.juently than the esUm3tor bascd OH

The bia s í n trl'd uced wouId be small samplc estimalor even though in \ve would proba-

addition, v,'e know the direction ni the bL:lS Ín this and ('{luId even partíally corree! for ¡U

ff quant:itatíve data are avaÍlable and we are ab1e to ¡'"r-such pwbiems as these, \ve can usuaIl;," mak" a dear decisíon,

enm if the qualitative nature of the rest:'arch makes e\',1I'uat­ihis trad("-of{ difficult or ímpossible, understanding H should help

us make more reHable inferences.

Fonnal Comparisons oi Bias and Effidenry. Considí:'r t\VO estima-10m, one a bv someone ,y¡th il vvlto i5 ¡hereÚ're that \Ve

bdieve is unbiased out less effícient ana is done by an im-partial investigator, As a fnrrnai O1o(leÍ of this examplc,. SUpp05t: "ve \'vish tn L'Stimafe,u and the large-n study produces estimator d:

72

( ::: r + y: '1

i !

where di5tricts 1 dnd 2 are iH'erage ilnd ::') ::::: 11,

VVhkh estirndtor ",h(luld \.ve

üus or twst éstímator is nol

Flrst, vve wii! asscss bias, We c:m 5ho\\' tha¡ the first cstimatnr <1 i$ biaseo ro the usu,11 calculatinn:

FU,I::: ,L~ - o.or \11

¡¡ .:::....,; , ,

v"',,, cm ¿,bo sho\\' ¡har ¡he senmd estimdü,r < 15 unbiased lar cakulatíon:

/1 +,u c:::

d ~hni-

Judgínf.; Dc'S("Tiptive lniercnces 73

the t}t'tc,rt:-:- ()] ~\ur ~n'L.üi-·'}i ~tt.hj\'> :~in\'t.l i~ l~

U.nbÜ15ed, ()n d\'eri1~e~ n(ro~s an lniirúte nUD1ber s,.1f hy}·hJtht.~th\Jl n .. ~p~ ljciltiül1S, h.lr ¡he im'éstígi!tO!' wíth a prtYünccptiol1, d would tlw >,/1'ong iU1S\ú;'!', albdt ,mI\- Sil, E"timah,r " wouJd ttw

>.',>-1',/"''''''''-'.' critt'riO!1 leHs a difIen:nt sh lry To late the variance of each t'stilTldtor:

-;l;,Vr - tU)l JI'''''''

1

:;:: a'~/!l

This "ariance is the Silme as ¡he vilridnü' n( ¡he silmpJc mean be­cause (Un unes not ¡has zt'rt! vilri:mcd aeross "amplle'i, Sími,·

vve cakuI::te the varidlKE' of e as

¡eSE <'!fkient than ti beClUSl' ::::,/2 ls 325 Thís should be intuitivelv dear dE

lhe ínlormiltion in tlw data set. VVhich shotikl we choose'J Estlnutor ti IS biast'd bu! mnre effkient

1;.S unbld~cd bu! it~s:-- et"fic}cnL !n this p¿1rticuL1r ~:,Jsef ~~,",h"h estímator d. \Ve \vClu1d thus be wíllíng to

sinee the sacrifi<.:e io; [,lirlv smali )' ;~ ,< -u,

more estím,ltur. ;\t sorne pOln t tol' a Ii ttJe bias sÍnce we end up guaranteeíng ¡hd! estímates v;il! Dt' farther froln the truth. The formal \vay to tb!.? biaso{'.ffíciency trade-off is to . calculare the lIlean squan.' error (i'vlSE), \vhiel< ís (l co~binati{)n oi bias> and díidency, If g is an estimator for sorne pal'ameter ;' Une Gfl'ek leHer t.-lSE is ddined as

:::::: -+

=: varianee + bias

square error ís Hms the sum of tht' variance dnd the (set~ l The idea ís tu chouse the estímator

\-vith the minimum mean square errOr sínce it shows precisely how an cst1matur wíth sume bias can be preferred ¡f ít has a smalle"r vnri­anee,

Por our eX2lJ'nn'lp tne t.vo a 1<:' as follows:

-+ (10001

and

Thus, fol' most values (It (jc, :\'iSHd) <: .lnd we ,vouId d as an estimator lo c.

In tne'l"n', \ve should unbii1sed estimates thut are as use as muen

rCil1 reseuren sÍ!11ations ,ve trade-off beh'lleen bias and is quite sillient

this

Causality and Causal lnference

\,rE HAVE OISCLSSED two 01' sodal sdt.'IKC rCSl.'iH'ch: SUI11m,1riY-.lnd dcseriplhe inferelKCS

the workl ¡nío systt'mJtk dnd nn!1sysh:nvItíc cnmpOHcnts 2.6), M.lny students oí sodal and polítical

stop aL Ihis püint, esc!1ev"'¡ng causal statements and Dskíng th('ir se-¡e('ted and wcll-ürdered facts ro !or tnemseln:s,"

Like historians, :5üCÍiJi scientists m,>ed in summarize histqrÍCal detaíl and ro make descriptive inferences. ror SO!1¡C socia! scientifk

b witll,)Ut causal lflfert'l1d'>, Th,ll just as causal inference 15 ímpossible \vithout ¡nfer-

e.nce, descriptive inference ilIone 15 oEten anei incomplete, To say this., however, is nuí to daim tllilt a11 social scientists musí, in al! úi their 'work, seek tú (1I::,v1se causal üf the """~''''''''''"Y>,'''TU

studv. Sometimt.'s causal inference rs too diffícult; ¡n mí1nV orher , ínference ís the ultimate (lE the l'~search

endea vúr. Of course, we should \vhctht'r thc

of a n:.~earch pmject or Many sodal sdentists are uneomfortable w¡th causal inference, are so Vfa!"v oí the that "corrdation IS nlJt causation" thilt thev \vill noi state causa! hypothL'Ses or dra,'\' causal tn [hei, as "studylng assodation and not causJtion," Others m<1h: causal statements with case, labeling l.l!1cvJlualvd or ulations as "explanatíons" on the hasi" ni indeterminate research "igILS,l We believ(; lhat each of these positions evades the pmblem 01' causal Ínlerence.

~ In vlew' oí' st'Hn(;~ $Ü~~Hl] '::<L'ienh~üs' prt:~feren(·~: Ú:n pver "mere dt:'~(Tlrtiün/~ it 15 not surprising thi){ students üf events ~~'<::t.4·;, to dress thcir \\'nrk in th~ tr.1PFlng~ of _¡t1rg~'rn; ~}thEr\vise< iJ:--. inferltí f

\\"ork. r\t its cure, reoJ 1;.: aasf'd (:in (-¡:H .. b31 jnferen(0s. V\'e rt~gard rnent~ in the hterature ar,out H nOf\CciU9","d 1n

itIl GlS'-""5, th("~e ~lfgun'lents arf" ~\r ;~n;:,

~n{,\)nsish.*t)L H St~iaI s(:i(·nh~t,-¡' f.áhlres to ('xplaln ~H'{~ nt)t due ro p-uor n.~'Se~lrt'h 01' lat k 0i ímilginiltí0t\, bul rathef lo lhe nil!ure uf lh,' Jitílw!t hut ¡ha! ti".?\, ,\re examlning, ~uch oi ínferiority are C(,o<l de!:'criptíon oí ímportarH "venls ís helter than bad explant1tiün nf anything.

76 . Causality and Cau&.l Infc.rence

tigatiOl1 eHh!!r renders the rC~Cdh'h írrdc\'imt ur u~disdpljnL>d PV th .. , mIes uf scll.'ntifk interencL Our uncerl¿¡inty' aboui cnusa! will never De climindred, Bul this uncertaint)' .. sh()uld nol th;ü we "void a!:tcmph ,lt causal infewl1{:t~, Rúther we should dnl\'; caus¿1i ¡nierences wh(~rt' they set~m appropriate bu! ,1!sO prO\ide the rcacler with the best and most honesl estimate of t.mcertaínty of tÍ1dt inference. rt is dppropriate tn be bold in dra\'v Gntsal inierence;; as long as we are c,ultious in ddailing the u I;ünty of the il1ference, H is importdnt further, thnt GlUsa! hypot be as as possíbk the mies of

4-6 ís to cIrcUmStances under v.'hieh causal inÍf:rence is appropriate and mak" ¡¡ for qualita!ive n::sean:hers to incrcasé [he ihal: thcir re<;t?ilrch wil! reliable evidence <lbout their

in sl'ction 3.1 vve appro-íor qualitative and tlwn in section 3,2 \Ve sf'vc-ral ahl'rnatíve notlons oE catlsality in the Hterature ana

dL'nHmstrate that thev do not cont1kt \vHh "ur more fundamental defi-nitíon. In s('Ction 3.3 \\'(: disL'USS rhe assumptíons abour "vorld and the to make reliabk' GHlsal inrerences.

(Tí teria we '~"'" "".".,,,,,,,,,'t

",ve cnndude strw:t GlUsal

3.1 DEflNING CAUSAUT)

\Ve define of the d¡lGl uSE·d ro learn about

,15 a theurcfica!

"h'Y,""'"" fmm our data, iFor prob!cms of causal see chapters 4-6.) In sedion 3,1,1 h'e give om definition or

causa!ity in full ddail, \vith a sim,ple quantitatíve example, and in secnon 3,1.2 \ve revisit our definltion altmg wÍth a more sophisti­cated

3.],l rile

Our theorerÍl:al ddÍnition of lTl0st ,md lo a As sectipn"\ d unit ¡s une o{ tlw many elements to be obsefved in a study, sm:h as a person, country, year" or

lOUf point t}f departl1!'€ in thí" ,,('dio!'! is Holhmd's ;.rtide 0986) un causa!ity ana

//

:;;tatus [elf <l Dt'mnuatic c:mdid"te iúr tlw LS, House ni on tht> rr\)porbnn n( \Voh~~ H1is (t1nJid~tte re()?i\<t~~,

il Democrati<: l:alldídate thv , Ld th,.' \'aridble be th~' Democratk proportíon of eh\.'

fOí the House, The kev caw"ll explan,ltnry \'ilri"ble Ís then dichot­cHher lhe DemcKrat is ;111 incumbenl ur IwL (Fm simplídty out this 5edion, vve nni)' (onsider districts where tht' ¡~epub!i"

candidate los: the lasí cledion.J

varÍ<.bl¡," varia bles"

dividl' the

is sOlTletímes ('alled the "outCO!l1C are ofkn referrpd to as "índepcn­

variablt,s into th!:' variable" caned the "causé"

thl' ''('ontrol variables," causal \'driabk (tU two or more \',¡llll'S, which are oflcn denoted "trt't1l!rWl1f

and "control group," No"" consíder onlv the Fourth Dístrid in \;e\\, Yurk

ill1d imagine aH elecÚo!l in ISlqg with a Demelt:ratk innuub':'IÜ ami one Rcpublkan {nonincumbentl thc Democratic can-didate receh,'ed fréKtion pf the vote in !his election nlw '* denotes the Díslríc.i in Nt'w York imd Ihe to the fae! that rhe Demn(Tat is <1n ,'d b thel1 a valuc of tlw dep",ndent variabit>. ':ro ddinc the causal eHect Ll tI:nw"tínrl ljlhmtitv},

imagine that \Ve go back in time lo ¡he siart oí the electiün ana rernains the same. !hat ¡he lJenKKratic in"

no! lo mn fOl" re"t.~lcct'i(ln ilnd [he lJemuCfatk nominales another candidate the winner lit' the primary eledionL \Ve denote the tradinn of th" ,,(,te that the Dernocratic (non'· innllTtbentl candidate would rece!vc' cratic candidate \\'ho is a Non"lncumbent),'

This conditíon IS (he essence behind this definitíon causalíty, and the difierence betwi'cn the adual vote dnd the likdv

whi1t he G1Ils '·Rubin'" \4udd" Hnlbnd bi1SC5 hb; hit';1~ "n (he work ol' n\!men'U~ :,eh..,;, ¿H~S, Lx}n:lld Rubin's 097-L F~7b? \.v()rk nn the subj"ect \Vdt..: Hh'''':! n11tncd¡(11d~' rl~lc\'.;lnt t~ut

he ,,10.0 dks Ari',totk ¡A,,~ke, Hume. Míf), INI:: e'xlt'nJ H"lbnJ's ddinítion oí ;l ,<;¡uSll

s~cn nt?"(t'SSJfY ;,;,Ü¡Ct.."> nu c'\~~tín!;

C~rangeL Fl~her( iUl,d ülh*2>rs, b.\' u(.,lng sonh~Á id'f~~tf

N \A,,\~ f"nUll(l thl~ t,,"x.k'n-

rt~pc-ct to a unii ¿in.,! ~hn ,,-üh"v/~n~\ nnc to partHú,n (~1tt""a¡ cíft'>('t:>.

"!id non"YSl'em,ük cllmpilnenb, 5ee (~e¡m¡¡n and Kíng 04<Jül rOl' detaíls oí thís example. M"rc ! and IV (MI

"tana for tn .. -"treMmeot" anO "c'ontrn!" group 0f inf ,\ny two treatm~'r1ts f.'xperiment;¡!ly

Ca usa lit Y and Causa! lnft?rence

\"uh:t 1:1 this ,,:c~llntt.~r!J('tua] sllu;ltipn {JJ(~) :;-.. ~'hf' \~.auS!1~ t~i(~:.('t

Qt:.llo\v. \·Ve ITlust l)t~ y,pry ('a-fetul in although they are obviau"ly COtmter tú the

they musí he reasonabie dmi it should be possibl,: fUf the w tua] 0\'ent to nave occurred under precisdy staü?d círcumstam: key part uf defíning the appnlpriate count"rfadual eondition is fying what \ve are holding constan! "",hile \Ve aTe ['1'!An,m'

the \'aJue of the treatment variable. In the present example, causal (or treatment) variable is íncumbencv status, and it fmm "írKmnbent" ro "non-incumnent'" During this

We hold constant up to the moment the '-"'H'''''''' nomination dedslon-the relatíve of the Demt>-

crats and Republicans in elections in thís district, the llature oí the nomínatton ¡:'1í(l('ess, tht: (lf the triet, and the economk ilnd políticaJ l70t cnnt.nl for nI ¡he

and 01' tu€' else that folkHvs the party nomínauo!1.

Cm!ScqllCt1aS of our treatment ot indude name

Si) lorth. H ,Ve díd and hence

and as a total. In faer,

could milke one fect at 311.4

f'vlore tlw causal effcd of in ¡he rClUrth Dístnct in Ne\v proportion of the vote re.::eived the Democratk

cand.id,üe that is aHributable to th<~ difference behveen that \ví!l become deilr <:.nnr'nv

admlnistered in fact ni in Oí COUise, the dedskm In c~lJ (HIt: value ni an .'Yl,H"",,-¡or" variable a (rcalmen! ,md the orher <l cüntrd is ii th!s "",,,,,,,,,,,c used al al!.

4 ,lon E1"t¡;r n983:34-36) has daimetl "¡he rtw'"nllrw'

counterÍactual ~t.,.ltenH!ntsF tn ((",m!s for t""th Ih" ilppanmi eXiPWn;¡rOl,

Elster is

cü'!mt.>r'Ü'i<'!n,,¡ ~t~h~n1erüs '~h"Yv{~ ~n irrliporLlnt ro~(' in (,Ju~kd

rh.'llce Elsk','" argument 15 mOr(' ('(lijen!. we thínk. as i) se! ni valuilble against careJ",s$ use of count~,rl¡1Ctuil¡S lhan as d critique of their fundanwntal deílnitíonal ímpm" mnl.'e in causal rea,solúng.

Ddining C1USi1nty 79

Causal Effect fm unit i) ,:;c ¡¡Í ...

onlv in t1worv since in til1\' on" r,,'al \Ve might observe eifhcr pr or l~eíthef<. hut n~~ver oolh.

chis simple definítion of GlUsallty demonstrates tha! \Ve Ci1n hope to know a causal effect fm certain. t:Í\,Iland 09So} refers lo

problem as Ihe fundamental l:aw:a! ,2nd it is ín-n'¡"'l'1>':I17,'H problem since no I1Mtter hüw the

no matter ho\<: much data \Ve no matter !ww pt'rcepti\'c obsern,rs" no mi1tter lIow dili¡;t'rlt the reseafch and nn

n,atter hmv muen control \\0 we will never knovv infen~nce for certain. most nf the empiric,11 issues oí desígns !ha! \ve discué:s in thís book ínvolH' this

problem, and mosl uf OUf constitutc aHcmprs to it.

Our working definition oí differs fmm Holland's, sinee in ,\i."rtion 2.6 we han: ilr~ued that soda! sdence needs to parti-

tite world into systematic and nonsvsternatic components, and detinition dc'{;'s not make thí...<; disunction dearJ)'~" lo see the

oE this partitioning, thínK abúut !'"ha! \.vould if \Ve rtCnm ¡he! CJ98 eiection campaign in the Four!h District in Nev" ,vith a DemocnHic íncumbent and ,A

sHgh!!y different total vote would fea-tures oí e]ectlon

canojda!e's rall\" or on election We can therefnre

rhe v,llues of the [k'molratic this same eiectíon.

OH id en-featurt"S might indude ¿l

Uf positíon on an íssue. ¿n had \.veilther during one

or thé results oi some investigatlve n ":ariabJe that would

vote acmss hypothetical replications oí'

:-:- Vie can for district ·1 t'tv h4"" ÚTf h'1 the 'Th(' reason for this lS thM Ho![¡md j" a sta!ísbda!1 who conws V(~ry dose lo

f'",'"'''''''''',''' 2" rand~jn:1 vari~)ti0n, \vhich is d1..~~Tit"X.~i in secücrn íL,,"t"'<'nti'cm ut' tht~ ~·:-tt~lb~b\.~':lr Sdfutú.}n~·· tu the rm+,[,,'f'I1 (d CiltlSill inte!1.~nc(\ h(~

hui ibb d~úrüt¡{~n b dh~Y.!t u<;in~ dUh·),rent unib ti'> ;.:.,(,h:l' thc' FUnttn1k'nL)~ Pr, }h~(>rn ~H."",L'fH:i f"t'taining tne definition nf ctluSdHty in ju~t nne, in his t/~~hH) !.:~ptEÜpr ~'1Veniges

ov\~r tlnits~ tvhereas ours úlC5¡;,:ribt,~ tlvt:~rages O\'e"r oI' tne ;;anlt~ experimt.>nt fur just a single un!t (se~' Holland 19Só;947).

I ,

;\5 rott:d ,1bO\"(~ (S()¡C ~eCt1{n1 ~ ,t ':), t l'ü~ '\:;~ l~

\'<Jliablc" :ÜnCL' it hilS il '-'<inaNes not

Demo(Tiltk candidate as votes that ""oHld be rcceived in era tic noníncumbent as

V\'t" nm .. : ddim,' the miliÍom owsaf ior dístrict 4 ,1') the bC!\Yt'en thest? t\Yo r,mdom variables,

wc switch notation from district 'f LO unít í:

is a causa! dfect that varíes 0\'\'1'

but «Iso m,mv ff \Ve could obser\'\' ¡,,\,oc

{,ict .f al ¡he 5ame time, D€nlOcrntic incumber1l' rcalized c¡msa!

we ca11nol the re{lli;¿¡~d causal effed in

urwl'serued reaHzi:ltinn of the randüm causa! effect 3.1 nther ,Kf('5S m<1n)' of the same tion in dístrkt 4 lNith d Democratk incumbent,. and anoss manv !hetkal of same dectinn but w;1h a Democriitic

renli7ed ciiusal dfect becnmes a nmdom

il makes our definihon of iltic

As W\~ in mur<' d,'íail in s¡~d:inn 22,. this úm L,,~ (ontu"ing. A "'ran-dmn vari."b!c'· cpnl;¡ins S<lmc (~om¡:XWlent and t'lms is nHt tw~

\rn"""" .... " this H".,","""_'

peHn FGll10mic ur tlw um:erLlin úlnSÚlUelKC nf ,1

change in ",lector.1l Systt'HL Tlw !O (lar repreSt'ntatícnl 15 tlMi e"eh of ¡hese "random" procesl,cs have systemalíc :md pmbabllistic components,

Definin¡::, Cm~a!ity

, ' E"1fen: .. :n(l~~ rIH .. <l!1:1 ,JfhJ \'arT,l.H(,·e:-; an,,' dl:---~} ~\-~-

ft:;Bturc~ '-.:f {"¡)nd0111 \:arlables {~~,~ in ~eLti~Hl 2.2}, Se'--.:ond~~', it us to p¡lrtitíon a cHl~<11 inferenct' pwblem into

tic c"mpOlwnts, Althnugh mi1l1V ,'arí.1bl..:' be o( interesL the most reln':mt fpr dur run,

ex.u1Tr'le ís ¡¡w ?l/can (dH:"i¡[

bV this, v,'e return lo our NcV\' Ymk dl'ctlon ~ tnat thí:' \'ariable rdcrs to ¡he vote ír,Ktion r<:cclved

nUBlO\'r the Democrat (inüm1bent ur nuníncumbentJ iKt\ISS "

replications (,f the Silmc clcdínn. vVe dcfílh' oí this random \utc imdíon

cffcc! of íncmnbenc\' ín unit i is ti

fffect tiJf unir i)

. .l ce

v,,'here in the first Hne uf this eqmttloH, 1) (beta) lo this causal effect. In tÍle second 'we indiGlte ¡har tbe mcan [Oí unit ¡ is just the mean '-<llue) of tite randüm «nd in the [hiTd and fourth Unes \ve show no\\' ro cakul,üe ¡nI:' mean, The last Hne 15 ;mother ,vay vvriting thc diíference in ¡he mean!:' (,l' lhe t\\'o sets oE e!ectíons, <1\'cr;¡ge ,,( lile díffcn:na! behvt:ell t\VO random variables the d¡flen:no: oí tht:c To summarize in words: rhe (i1!iSi,l is [he bdaten¡ the temafic component of obsenlfltüms madc whcn lhe e:rplanaf"ry nniablc

82 . Causality ilnd Causa! Infen:~¡Ke

,"1ftC ¡·;;lue anJ I1::' "::;A>~?~~!;:;Jtic

u{lr}tibl{ ((!k{'~ L?! {ohJther ;\"7lru.,',

'fhe 1.1st línt: or' 3.3 i5 similar lO ~~qu"tíon }, 1, dnd as rhe Fundamental oí C:ms"l Jnference sUB exists in this ¡,,(ion. lndeceL the this v<av IS 0ven m,lre ble because t:ven if we could gd amund the F¡~ndament¿¡l Fmblem a reaHzed ca1.lsal we would still have al! the usuaÍ pmblems inference, induding the pmblem of sepRrating out systematic !1onsystematic cOHlpllnents of the fandom causal eHect From he \Ve use HoUand's phrase, the Fundamental Froblem uf Causal ,-"nce, tú tefer h¡ lht' (ha! he idc.'lltified a:, ereN tE to rhese dard 01' vvhích \\'0 llave added to his In the hox on pagt' we prov;de a more genera! notat:ion 1m

\,,;hicl~ V.'m prove useful thrtlughout tlw í!;'st oí thís hook. oíher of thesc 1'0ndom causal effects

he nf inteF'st in v"riol!" cirnlmsti1nn's, Fur wis kno'.\' {he varíancc in the (rt'úlizcd) causal effeds uf '

status on Democratic vote in unít i, as \vith the variance ¡he vote itsejf that we described ln equation 23 in section 2,6, To latE' the YilrialKE' oE the causal we

(vari;,U1ce of the G!Usal effect in unít i1 ~:

in fOí tl)(., varianct> , ), new íncumbenis ,,'¡sh LO know the variation in the Glusal dice! oi incumbene)! so can hnw 'wJ!l be to that of ,.,ro""',,

from underst,md that Ihis vMlanu.: in oí' ¡he ,',>,orld and is not ,m,.'Ú~"~

3J.~ Ji

\Ve our precise definitjon of in section 3,1. sume of the in that 5c{.-:-tion are subt1e and quite soprusticated,· lNt~ iHustrated oul' \yith a running example fron': quantitative i\?search. This heJ¡''Xc·d as communícaie tile con.,

\ve wished to stress '\vlthout aIso having ro aUend to the c()ntex~ detilil 2nd that L good qualitativ.l:

rescaTe;', In thís our ddinítion oE cilus,,11-ity again, but this tirne vta a quaHtative exarnple.

Poütical scientists \vould lcaro a lot if they could rE'run history 'l\tith everything consh'mt save for one investigator-controlled explanatory

Detlning C'lUsaJitv ' 1'3

, F\.''I'( eX~1mrle/ une \)f tht} lnai,,~r l'lucshcH1> th.;Jt tdl.,.~\;~~ thp~e in"" \,,~ith pülitics and has to cid '\\'Hh the Ct~ns0.quence~

particubr law or passc,; a tax biU ¡hal is 1n-to have a particubr to ¡xlrtieular invest-

increase revenue by a certain arnounL and Cl'!1sumption Does it ha\'~ this dfect? V'le can ot-sene wh,lt after

, is pas..';(~ to sce if the íntended o.lflscquences ap¡:H'ur; bui even do, ít is never certaln that they rC5ull (rom the 1<1\V, The change

.. ""·,·,~,,,,·t políC)l might hav\': hilppened lf we could rerun 'wíth and 'wíthout the rte\v reguiation" \Ve ,yould h,we

more lt~\'eragc in estimating the causal E'Hect of t.his la\\',

we Gmnot do !his, Hut ¡he l.\'íll research tC)

us dl1 approximate ans'wer to our nm\' rhe !oHowlng extended

In the vVi1ke (lf the coHapse uf the Soviet in rhe ex-Soviet republics and in Eastem

uted ne\\' governmental fonns. realize-in a political (~xper¡ment they dre ¡ntroducing

constitutlons, nmstítutions that chey wil! have the intended oi creaung stable democratk s:<'st~rns. One uf the constítutionnI

:rh'Oh~es is betvveen nnd p1'esídential iorms ni go\'(~rn-V'/h1Ch 1S more líkely tn lead lo il srabl€' ¡:.; the

of consíderable debate among schol¡¡rs in the fíeld (Um:i 'tz 19L/:'; Ujphart '!'}93), The debate is not lile leas! be-

(lf tÍ1e numerous of svs-. fems and the of that migh!

and interact 'Nith Hüs choice ísw:h as the nalure 01 rhe elec-Ir Ís not om ro a !horough oí

lilese choices buí rather a n~rsion of the choice in order (O define a causal effec! in the context of this example,

,'>'e highlíght the distinction between systematíc and \1on-svstematic of a causal effed . . The debate about presidentíal versus parlíamentary' 1n-

irolve:s varled fearures of the two s\'sterns. We wiH focus on two: the extent to \vhich each represents the varied inte1'ests oE the dH-zenry and encourages strong and dccisÍve The is th,:¡t pa1'liamentary do ¡) better of n:prcsenting the tull r¡mge of sodetal group5 ¿¡nd ínterests in the 51nce there aH' many seats to bí:~ filled, and can be fmee! repre-St,'ntath.·f.:s clte'cred fmm v,1rÍnus gnmp5 !n cnntrdst, the all-or-nothinf; Chari:K1er 01 presídentia! mean:; (hat som(' groups \vin feel left out of tIte govemment, be disaffected, ,1nd cause greatcr in.<;tability. On th,,> othel' hand, parliamentary systems---especiaHy if they adequately represent th!.' fun range of social groups and interests--arE' Hkeiy to be

84

id ch(1r~)cteristics" h)\), (\H1 h:,Jd tc~ di~{1fft7(tit)n Jrh:1 ~n~.t:ibdity,'\

JI1\.' purros,,' uf this ",cdion 15 to d precise ,ü"-I'!.,."t-i a causal dfe'ct Tp d" S,L ¡hat we could íns!itute a

degree oí demoCTalic stabílíty {perhaps ,1I:'1ual survivilJ or dernocracy, attempted mups, Of other indícators oí instabílity}, ¡'he Silnie (ountry and dI the samc time, ínstítute a presidential aiso íts stabílity over the sm1\C period \vith the Sil sures, The /'calí::ed mus,,! would be tht: difference bdween gree of observed undeT ¿~ a The

m;:my

oí the realized l~)king ¡he av

nf this pmblem to out ilnd Jeavcs the lT\t'dn causal dfect to ¡nelude onlv ""ch-" .. , tures. Svsternatic ft:atures include indedsivencss ",,,,,,,t,'n-, or disaffection among minorities in a temnJic tealures mí;;1ü ¡ndude t11e sud den ni él president ¡hnw¡s the ínto chaos, TIw taller event would not

it \-','ould appear in one oí t:he

(me oi these ,,,eh,,,,,,, would be ínlercsled in íts mean causal dfect on

stabilítjt; however, ¡his om: one uf this Given this situiltion, pol:itkal

may be interested in more than the causal dfecL They \ds11 ro understand tvhat tite maxímum and minimum causal or at least the paritmec oE the causal might be. For mav be tlm! reduces dcmocratic stabilitv on in

'Thc~\! ciis!Índions JR i hemsdv('s dvhated. ~m1V Jfgue lha! ,1

can du ti bettt~r jo.L~< And pth(~r:~ t)rgu~ that

nn"<l,:w,r ~·~ppt.\1 r~ iD be .,1 n"n~,_ q,'m:mc

oí svsk'm, On tllC other h,1i1d, the \'uhwr;;bílíiy oí systell1~ tu !he' v¡¡ganes of tiw Jwalth ami person¡tlity oi ,; single individual ís a ¡¡tic efk"Ci ¡hal r¡;&;e; the likt'lihood th;ü j(1fHi' nOl1s}'stl'matk featufe 'wíll ,'pF'Ii'ar,

in tl1jS \.~ircun1~LH1(t\ it D1av be thdt

noliticdl le"der;.; \'voukl h, chousc 3rt o¡."\tlon ¡h"t r sHghtlv lt.~->:-- .stabllít\~ \'H) il\Vl~ragt} but h .. 1S a Icnver \'ar1'­

<-,' 'and thU5 minirrd:lcs tht' i.:hance ni a ddsa~truu::.

.1.1. \l,/e defincd in terrn~ of ?l

-is lhe diffcrcnce bd"'l.'ert the variable \\--!len t!w

In this sedion, we use (lur ddínitinl! oí ", ro ' ___ . '{, dnd appi1rcnHy (()mpllCi1tmg H.1ea:-, \'V\'

pv nrher aboué '\:allsal dl'hJ

"fe a b t11c diec! \1f " lWW

on n:ducíng the Uníted curren! acccmnt defícit \víth lo OUl' definition uf the eHect herc is

curren! aC(Cllmt defídt \vith thc taA

N;n~, ... ",rp(1 to the same si!uation (ilt the s,1m,,~ time and for me v/ith the exo.:,ption th<'lr the trealy ,vas not in effecL

causal mechanism her€' would indude, in tum, the and rntification of lhe tax ul the event,

mee!ings of the r(:le,'ant actors vvithü: majo: multination,11 , ,a

actions ro reduce the!r total iax burJe~l (such as its transfer pricing mIes Uf mO\'tng manubduring plants between countríesJ, further actíons bv Giher _ and workers to lnke 01' the rnovements ot .. c1nd labor, bt"

<, - h'¡" t'n-}l "'te'" on th\" "1"1, hveen countries, and su un, unhl \ú,' ¡vai.~ ",e d , L,,, - ''', ", ,mee of payments Det'\ve-en ¡he United Siates ;m~ Japan. ,

From the standpoint of pmce~sl>S through WhlCh causahty &n E'mphasis on causal mechanisms makes inhlÍtive sense: an)' coher-

86 CausaIity and Causal lnferenct'

C,Jus.al nl(~chanisrn~ i~ v\·a:·~ of dc\in¿; ern anah/ses. H has b,,~t~n in dl1ferent forms, "pro.:ess íng" ttvhich Wté' disellss in sectiuu "historie,,! anall'sls:' and taBea case stuJies:' of the derails of w('ll-done case involve iJent:ifying these causal mechanisms.

However, identifyíng the causal mechanisms rE'(juires C<lUS¿tl

enee, tlsing the methods díscuss{'d belcn\'. That is, to demonstrate causal status ni ('aeh pütential linkage in su eh a positcd mecha the \,>,ouid have ro define and rhen estímah' ¡he causal

OUT more' oi ca fUf ci)ch link in the ch,iÍn ni causa! evenís.

our ddinition uf prior lo Lhe id",!! tion 01' causa! nlL'Ch,misms, Furthennore, there exísts in the cia! sciences an infinity oE G1Usai between any hvo línks in chain oí CHIS,! mechanisms. lf we that an causes a dependent a "causal mechanisms" approach re{luire 115 lo ídentífy a lisí oí' eausallinks betvveen ¡he two This defínition would {liso us tn a series of caus<11 ages, to causdlit:v ror each pair 01' conseculive variables in s{~quence, ¿¡nd to the behveen anv twü of these <1bles and the coml<.~·t¡on5 (lf' variables. This

al no time it definition of causalitv rOl' au\' one Cduse and Ol1e

In our of the effect of a' , versus parliam

ano, in rurn, couId monitor the attitudt'S or differ under tbe tvvo experimental oi rile under eaeh sj'stem.

¡he h;'Tothesized under a ".,.'oc"",,,,,

[he plhieh a cause has its for ,1 i.HId is a verv userul opcral'iol1aJ

procedure. ldentifying causa! mecnanisms can ;'omet:ime~ gl'le u$ . ¡nore over a tlieory by makíng observatlons at él

Altemative Definítions oE Causality

t1t""\\" causal to \"<c should not 0:)11[115(' a d",fíniÜp!1 oi causality wirh the ntm.:idinítionaL albeit (,fb.:'n

. operi1tíonal pmeedure uf idl'ntifying c.:H1~a¡ nwch,misms

''lv1ultipk Causalily"

Chark'S Ragin, in a recent v\'ork argul"S for il methodol-~"ith many explan,ünrv variahles dnd fe\\' e>bservatÍpn5 in order (me can t;;lke ínto account what he calls "multiple GlUSiÜíl'n." That

js, "The phenomemm under has altemiltive JdcTmi-nants--\vhat \·Hll OH4:'\1 rdcm~d tu as thte' ot nf causes.'" This 15 tlw pmblem referred f\¡ ilS gen-

1982:1n In slruations cau-these dutnors argue tha! Hie S<1me outcome can bt' GlUst'd

combinations ni difieren! variables, ¡¡l Under nmdHions in vvhich díHerent vdri"bles cm ¡1V

t.hese variables ha\'(' cilusdl status, is correct tilar some statlstíL<,l m~)(1JelS (or ndE'\'anl qualítative researen investígator lo ¡lw existence of "multipie

could fail to alert an .. but

models can handle situations líke tbes,,' (50me oí discussesJ, the fundamenta! features oí "multipie are

compatible ,·vith out' deHnition oí Gmsillit!,. They are ,11so no d¡fh~rent qUilntit¡¡ti\'e Ihan resean:h. Thc ¡de;) cont;lins no ne\\,'

reatures or fnr ccmsíder the hy-pothesis th<l! a level nt income nona1 attainment ünd educated both 15 insuffident. In this case, ;ve need io compare causal variable: respondents who h.nve high educdtíonal attaÍmnent ano highly educated parents, the t\VO wllo have one but not the other, and the ¿;TouP wjth mdther. tht~ concept oi cau-sation" puts greater demands on our data since we no';- cat-

variable t.~itht~r nt..'C(~"

t'l~ )\\ '~~\ 'c-rf th ~s. btXi:1U5t:" thl~ diFtüxiú.rn bt~t\.Vt~pn fll\;:''lSSary and suffkient con-

3i'./,n,,,',u< ",hen we aJlrwv for tlw ,,,,,,h;,,~ NCllnsídcr the cLúrn th;,.)t F>nf>¡t UHnrnUn1(dtJun ún1\.~ng surcr··

P(l\VcT~ Juhng c["jsis jn\7i~<ts0S fht~ hhcEhOt)d oi \.v~H·, This ~~ (J daárt jt iJt.'D·

tiCk>;; ,1 G\lh,¡1 Vilrí,¡!:Jle (¡:><1()r (ommnnÍ(i\linn) ,¡no ilSSHts ¡h¡¡{ thi$ vilri;¡bl,' ín.::n::;:¡;,vs (n .. , probabilit\, of il outn:mw (w,u). It cannü! be tr;ms!at('(l into a daim "prmt tl:w m~:-J?ssaryani! conuitíons [ur "",,1', however; i! is iITcducibly

(ji {lUT C<lUSal \~drL¡l'\lc~. tYLH

definitlon of (>,)lv··~~'·dl1·V, For our income for the S<lm~ pt'r,,,,P1L al

cxperit:ncing c¡)ch Di the four conditions, 13ut what happens if different GlLlsal

vdlues üI rhe dependen! variable" FUf e\¿¡mple, \vhether o, not (me gradudted from cilusal variable in a popula!ion oí ',<'b",r,,'

both groups ((luId quite earn the Silme inconw pendent variableL Om' n,ason might be that this

attendance') ha~

nf irlúlme íOf ¡hose eduG.!ed and earn" particular lt.'vel oí income

who no educatíon might their {OUT VCM!> oí údditYOH'

on thE' In thís situabon \vouldn't \ve iJL~ lea lo conclua educatlon" has no causal 011 ¡ncome levpls

>-'.'no win become 1yorki2'TS?

OUT' definition 0f

che condiríof1, In the

son bad; in time tour

ínstead nf jn Uf her income " levels uf income rOT this une causal dfect in ¡he 'Ve have imagined that causal effcct is zero, But this croes no! me"n that "college t'.~dtJcati{}n no on N that the average '

{1 j and (2) is 2em, In therc ís "tIte causal effl'Ct of (:nHeQc

no! define a causal diee! without al lcast teVO

tlons nect:! nü! be rhe hvo Usted !'tere, but

of CdUS11 condí!ions ls lo lIitte vd!h someone \vithmll '" but with thc same ¡ob seniority as the college graduate. In one sen;:;€', this is u since the non-L"ollege graduate wou!d have to do something for

nut lhtil thi.~ per~(\n h3d a ditft'~Tt~nt irrt"le\'~H1t jnb ~ur

Put diff(~rentlVt thi~ dlternatlve (\yunh:rt'ácfual 1S tht,:~ eff€ct edm~atí()n cll~lparcd to that ni mme, wilh seníoritv bdd

FailuTe lo hold :icniority consLmt in the h~',\ e,lUsal ümJi­would CdUS(~ any 1\,se.1[cl1 Jesign t,l yíeld esHm"ües uf pur i]rst

ínsh;,)d uf ¡!'tis revised ont', H the laHer wefe ¡he

intluenced caus"l fadors do",s not milh~ pmblematic T!'tt'

ver\' úHnmnn situatinns is h) define tlll' makiflO un edeh eiluSill effet't ven; VVe demonstrilte in b r ~

5 that researchefs need flor "all" causal on a variable to pnwide estimates uf the Orle causal efft'ct of in-

(even ít that lVt>n,," A resean:Jwr can hiell'; nn ,miv the {}ne effeet nI interesÉ. estdblish firm and tlwn m,)"e on t(1

chal nI;}\' be oí ¡nieres! (see ~t'd¡(lns :;,2 imd ,:::;:1). l'

bctwecn wh,,[ he refer,; to

\vhkh díffer when an tn wltt'n it is dCcrC,1Sl.'d.

¡le 15 ínter·

tne CilU'i<li infllH::nce üf x; tan \'dríabltd un '¡ jil

dependent varí.1ble!. tor (>ne has a!so lo considc'f wht:ther shitb lo a V¡¡!U0 nf Xl rrüm either dír0 ... :tion h.1v0 the same ctlllsequences íor y,. , ¡f tht, causa! relariol1snip bet1.vcen Al ~,m

\ve kncn·v ft 1S r,,:llh~r than detenninisHc t~1en(c.< \.ye ~lbo di~;gn~'t~ vdth pn'mise 0987: ¡:;) th,,¡ ni ¡he cnm-

pHrath/t~ rn('t+tnd are nnt ,,~fHl(e}\'t-'i.l in rhenoml~nün r~ examirH:'tl ~lnd tlcc-nunted t"nr ir h:~ l"ut·· kx::t i1 \x~nsn~ uf intuffl\,,"ltíon on ev,,'ry in~L~:uh.:·t· ilnd t~\·er:' rt.~nrtutdtlnn

¡¡oc! cümbírh1ü,m of "al m.'!, ,,1' Üw expi;m,,(ory Filrí¡¡blt>s., lhe wor1<:l stíl! wnuki h¡¡n~ pn>­cluero ¡hes<' d,lta lo some probabi!í"tk pro,,;;ss (,b defhwd in sec!i"n 2,6), Thb

1I

90 Causality and Causal Iníerem::e

As ,111 (1f Lícbt'rsnn's point, imagine th,.t the Fourth gression,11 Distrid in l\f;'w York had no irh"umbt:'nt in 1998 ilnd Ihilí Democratk candidate n.::ceivt'd SS perú'nt of the vote. would definE' thE' causal effc'Ct uf incumbenc',' as lhe incn:asc vote if the vv'inning Democrat in 199B runs as an incumbcnt in ¡he e!edion in the vcar 2000, This cffed vvould bt, dsymmctrk" if the sence of au inn;mbent i.n the subsequent e1edion (in ycar :::002) the vote to rdum to 55 TIw eH0i:1 might be "asymmetric" íur example, tht.' íncmnbtmt Dcmocr¡¡t raised moneyand íntpro\'t:'d Demtx:ratic campdign as ,1 fesuIt, if nn incu v,:ere in 2002, ¡he Democrabc candídilte might than 55 of the voh::,

UebersoI1's b dever "ud ver)! \'icw" nis does nüt cCHls!itute a pHes causal

derined 011 the bas¡s of E'vents occurring in the 1998 election in the fourth Disfrkt in Nt~w York Om d ís the difference in the cmnponent of the vote in t!lis d \vith an íncul1.bent in election and wíthüut "n incumbent in same election, and distrkt.

In contrast, LieberS\)!ú,; involves no hypc)thetical aud therefore Cdnnot be d causal detlnitiou, onlv '.vhat would actuaHv o('cur if the tv,'¿ real t:,lections .!Tom n~n1ilK"1nnbent tü to nonincumbent in hvo other élections. empiriGll this wüuld ínvolve mrmerous problem" of infenmce, We cuss manv oí these problems of causal inference in chapters 4-tL in present e~ample, \ve might ask \vhether the estimaled larO'er on1v beeause we failed to account for a number of

él , h ' registerL,<1 cítízens in th~ fourth District Dr, did t e surge m su for irte Democrat in the elec'1ion in \vhkh <¡he or he ,va" an iDcum

~l10t,¡-it.\1 n as d

the('n.~ticd~ nr m¿lking infen~nt'{:~; tol(" .. lrn fn;~rn d/da tht'" S:lmt)

di 5cwnlifk rnt'erem:e th,1¡ we di~tlSS in ¡!lis hOOK, However, hi\; apprtMch can "ti!¡ "<11uabli:: ,15 ,1 lo:m oí foro1al ¡heory bee sectíon ~5,2): it en¡¡bk'5 ¡he

a imd Hs ímpiic,'ltion; in a way tha! be muen more out iL

01

than it shtHJld be\.~J,lu~C \~' .. ,' n~/,-'tl~~~1:-:1y JiS(tlrded di~tri~~ts tlw DemoCTdt lo"! dlé' tirst elccií,'n:

us Líebers(1{l's conú'pts \}[ "svmmetrical'" and "asvmmctri('al" c;v are imDortant to ~'onsider 'in the cm1tt'xt ot (,du~a¡ inÚ-rencc,

, , ,

, ,they slu,')t,lld l1üt be confust'J with a thülretical detinítílm uí in sechon 3,1,

fon ESTJ:'vJATf"<¡G

do Wt' avoid the FundamentdJ Prublem o( Causal lnfcnmce aBd ih", problem l)t {!'Pm nons\'st.t,'mdtk \:omplJ"

The iuU ¡¡nswer lo ¡his question wiH consume bu! we providc .In pverv¡f;w herc (1f wh¡¡i 1" in ícrms üf ¡he rwo

'bIt: assumptíons !hi1t enaHe tlS 10 arounJ ¡ht~ fundi1mcntal km 'n1('Sc dTe IiIJít \\'E' discuss ¡n seclínn 33,1 )

and tDJlditional {~{?ction 3.3,2), l'llt;;~se l1ke any cther attempt to círcunwent the Fundamental Problem uf C¡usallnft'f­enee" ahvays im'olve Sllmc untes!able ¡¡ssumptíons, H ís ¡he viiit\' ni aH fesearchers tn make the substantive implications of this

" sppt in their r("Sé¡uch extn:mely c!eilr and visible to read­Causal inferences should no! appe<1f like , TIl\? ibsumptions ami slmuld be wi!h wh,,!ever síde

cm be mustered, bul it mus! be

wíth yalu<-"S of our ,'ariabl" eaeh time-as a tTue ::;olu-

tinn lu !lit' Fundamental Problcrn oí Causa! lnterence woukl we can a!tempt to makc " second-bcst experíment in two different ¡!!lit:; that are" Two ¿mi!:> ,1fe hamogew?eus when fhr ,'(¡{ues ¡¡f [he each mlit are Ihe same <rilen ¡lur ;lt/rú¡ble itlkrs OH fl

oalue. (That is, and For example, if In' nbserve (an incumbent) in district 1 ,uld X-::::: O {no incumbenO in district 2, <111

assumption (Jf unIt rneans that ,ve can US{' the observed pmpnrnons nt [he votl' in !\vo sepdriltc districts fnr about thc causal effect ¡3.. ,vhieh we dSSUlne is the same in both dístrids, Fnr a dat1 set \vith unit ís the that al! units wíth the ;,ame v,11u<:' (jf the variables have tl1.::' 5."lme expeeted v,1Jue of ¡he depenJent varí,lble, course., thb is unl)' an assumption and ít can be wrong: thc hvo districts might differ ín