Upload
andrea-carancini
View
216
Download
0
Embed Size (px)
Citation preview
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
1/36
CountingDroneStrikeDeaths
October2012
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
2/36
2 COLUMBIAHUMANRIGHTSCLINIC
Acknowledgements
ThisreportwasproducedbystudentsandstaffattheHumanRightsClinicatColumbiaLawSchool.TheprimaryauthorandresearcherwasChantalGrut,L.L.M.12,withcontributionsbyNaureenShah,Acting
DirectoroftheHumanRightsClinicandAssociateDirectoroftheCounterterrorismandHumanRights
Project,HumanRightsInstituteatColumbiaLawSchool.Additionalresearchandeditingwereprovided
byLauraONeill,J.D.13,SarahMechlovitzSaadoun,J.D.14,ZoeHutchinson,L.L.M.12andDaniel
Mul,J.D.13.GretaMoseson,programcoordinatorattheHumanRightsInstituteatColumbiaLaw
School,andAshleyKinseth,J.D.13,providedmediaoutreachsupport.
TheColumbiaLawSchoolHumanRightsClinicandtheColumbiaLawSchoolHumanRightsInstituteare
gratefultotheOpenSocietyFoundationsandBullittFoundationfortheirfinancialsupportofthe
InstitutesCounterterrorismandHumanRightsProject,andtoColumbiaLawSchoolforitsongoing
support.
Copyright2012
HumanRightsClinicatColumbiaLawSchool
Allrightsreserved
----------------------------------------------------------
PrintedintheUnitedStatesofAmerica.
Copiesofthisreportareavailablefordownloadat:
http://web.law.columbia.edu/human-rights-institute
HumanRightsClinic,ColumbiaLawSchool
JeromeGreeneHall
435W.116th
St.
NewYork,NewYork10027-7297
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
3/36
COLUMBIAHUMANRIGHTSCLINIC 3
CountingDroneStrikeDeaths
Summary&Recommendations...................................................................................................................4
Recommendations.......................................................................................................................................7
I.Introduction:DroneDeathEstimatesasFalseAssuranceinPublicDebate.............................................9
II.OurMethodology...................................................................................................................................11
III.WeaknessesofMediaReportsasaMetricforDroneStrikeDeaths....................................................14
IV.OurRecountandEstimateofDroneStrikeDeaths..............................................................................20
V.ComparisonoftheTrackingOrganizationEstimates.............................................................................27
VI.U.S.GovernmentsResponsibilitytoAccountforDroneStrikeDeaths...............................................34
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
4/36
4 COLUMBIAHUMANRIGHTSCLINIC
Summary&RecommendationsPopulardebateonU.S.dronestrikesoftencentersonhowmanyindividualsarekilled,andwhichoftwo
categoriestheindividualskilledfallintomilitantorcivilian.U.S.officialsemphasizetheprecisionof
dronetechnologyandcontendthatextremelyfewcivilianshavebeenkilled.Yetothershavequestioned
theseclaimsandstatedthatthereisevidencetosuggestthatdeaths,andciviliandeathsinparticular,
aremuchhigherthanU.S.officialsadmit.
TheuncertaintyaboutciviliandeathsislargelyduetotheU.S.governmentsresistancetoopenly
providinginformationaboutstrikes.Intheabsenceofofficialdata,themostcommonsourcefordrone
strikecasualtyfiguresisnewsreportsaboutparticularstrikes.Someorganizationshavecataloguedand
aggregatedthesenewsreportstoprovideoverallestimatesofthetotalnumberofindividualskilled,
includingthenumberofmilitantsversuscivilians.TheBureauofInvestigativeJournalism,LongWar
JournalandNewAmericaFoundation(trackingorganizations)areamongthemostinfluentialofsuch
organizations,andtheirworkhasinmanyinstancescatalyzeddebateabouttheeffectivenessand
humanitariancostofstrikes.
Weareconcernedaboutoverrelianceonthetrackingorganizationsestimatesofdronestrikecasualties,
althoughwefindtheestimatesvaluableandagoodfaitheffort.Theestimatesreflectanechochamber
ofsorts:thetrackingorganizationscollectnewsreportsofparticularstrikesandmakeanestimateof
whoiskilledbasedonthem;theseestimatesarethenregularlycitedandrepeatedinsubsequentnews
storiesandmediaanalysispieces.
Inthelimitedpublicdebateondrones,thetrackingorganizationsestimatessubstituteforhardfacts
andinformationthatoughttobeprovidedbytheU.S.government.Wethepublic,theanalystsand
experts,andthepolicymakersstilldonotknowthetrueimpactorhumanitariancostofdrones;the
estimates,thoughwell-intended,mayprovidefalseassurancethatweknowthecostsandcanfairlyassesswhethertocontinuedronestrikes.Furthermore,wherethetrackingorganizationsestimates
significantlyundercountthenumberofcivilianskilledbydronestrikes,theymaydistortourperceptions
andprovidefalsejustificationtopolicymakerswhowanttoexpanddronestrikestonewlocations,and
againstnewgroups.
Becausethestakesarehigh,theColumbiaHumanRightsClinicsetouttothoroughlyexaminethedata
andmethodologyofthethreetrackingorganizations.Ourfindingsaretwo-fold.First,despitethestrong
effortsofthetrackingorganizations,theirestimatesofciviliancasualtiesarehampered
methodologicallyandpractically.Twooftheorganizations,accordingtoourindependentreviewofthe
mediasourcesavailable,significantlyandconsistentlyunderestimatedthepotentialnumberofcivilians
killedinPakistanduringtheyear2011.Second,whilesomeoftheflawsweidentifycanbefixed,others
areinherenttotheprocessandtheseinherentflawsunderscorethattheU.S.governmenthasthe
responsibilitytostepinanddescribeitsownaccountingontheciviliancasualtyquestion.Thetracking
organizationsprovideimportantinformation,butinlightofthemethodologicalandpracticallimitations
weidentify,theirestimatesareaninadequateanddangeroussubstituteforofficialgovernment
estimatesandinformationregardingciviliandeaths.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
5/36
COLUMBIAHUMANRIGHTSCLINIC 5
Wenotethatsomeofourconclusionsdescribedbelowarecorroboratedbyotherstudies,inparticular,
aSeptember2012reportbythehumanrightsclinicsatNYUSchoolofLawandStanfordLawSchoolthat
examinedadistinctandmorerecentdataset.Takentogether,thevariousstudiesunderscoretheneed
foraU.S.governmentaccountingofdronestrikedeaths,aswellasgreatercarebymediaoutletsand
observersindescribingtheimpactofdronestrikes.
FlawsinMediaReportingonDroneStrikeDeaths:Weanalyzedthetrackingorganizationsdata
collectionforstrikesinPakistanduring2011andfoundthatwhiletheirestimatesareuseful,they
necessarilyreflectthebiasesandflawsoftheirmediareportsources,i.e.,thenewsstoriesabout
particularstrikeswhichtheyaggregatetoarriveattheirownestimates.Mediacoverageofdronestrikes
isinconsistent,anditislikelythatsomedeathsandevenentirestrikesarenotcapturedbytracking
organizations,particularlytotheextenttheyrelyonEnglish-languagemediasources.Themediareports
oftenrelyonverylimitedsources,inparticularthewordofanonymousPakistaniofficialswhowillnot
puttheirnamestoastatement.Thereisnostandarddefinitionthatthemediasourcesusetocategorize
apersonasmilitantoracivilian,norastandardizedmeasurebywhichthemediasourcesweighand
corroboratetheirinformation.
OurRecount:Examiningthesamemediareportsthatthetrackingorganizationscited,wefounda
significantlyhighernumberofreporteddeathsoverallandciviliandeathsinparticularthanisreflected
inthecountsoftheNewAmericaFoundationandLongWarJournal;ourcountwassimilartothatofthe
BureauofInvestigativeJournalism.Ofthethreeorganizations,onlytheBureauhasconsistently
purportedtoactivelytrack civiliancasualtiesasopposedtofocusingonprovidinganestimateofthe
overallnumberofindividualskilled.Nevertheless,newsanalysisandpoliticalcommentaryfrequently
citeNewAmericaFoundationandLongWarJournalsnumbersinmakingconclusionsabouttheimpact
ofdronestrikesonciviliansandlocalcommunities.Exclusiveorheavyrelianceonthecasualtycountsof
thesetwoorganizationsisnotappropriatebecauseofthesignificantmethodologicalflawsweidentify.WhilewedonotagreewiththeBureausanalysisofmediasourcesinallcases,itappearstohavea
moremethodologicallysoundcountofciviliancasualties,commensuratewithitsspecialfocusonthat
issue.
Ourrecountfoundreportsofbetween72and155civilianskilledin2011Pakistandronestrikes,with52
ofthereportedlyciviliandeadidentifiedbynamearelativelystrongindicatorofreliability .By
comparison,NewAmericaFoundationscountisjust3to9civilianskilledduringthisperiod;LongWar
Journalscountis30civilianskilled.Inpercentageterms,andbasedontheirandourminimumnumbers,
wecounted2300percentmoreciviliancasualtiesthantheNewAmericaFoundation,and140percent
moreciviliancasualtiesthanNewAmericascivilianandunknowncasualtycountscombined.We
counted140percentmoreminimumciviliancasualtiesthantheLongWarJournal.TheBureauof
InvestigativeJournalismscountofbetween68to157civilianskilledinPakistanduring2011isclosestto
ourown:wecountedonly5.9percentmoreminimumciviliancasualties.Wedescribeourstandardsfor
therecountbelow.
Itisimportanttonotethatdespitethegreatcarewetookinreviewingthedataandoriginalmedia
sources,ourrecountdoesnotpurporttobereliableoranaccurateindicatoroftheactualnumberof
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
6/36
6 COLUMBIAHUMANRIGHTSCLINIC
civilianormilitantcasualtiesofU.S.dronestrikes.Rather,ourrecountreflectstheextenttowhich
civiliancasualtiesthatarecrediblyreportedarenonethelessmissedbythetrackingorganizations;it
underscoresthatthepublicandpolicymakersdonothavecompleteinformationaboutthe
humanitariancostofdronestrikes.
DiscrepanciesinTrackingOrganizationsEstimates:Incomparingthetrackingorganizationsstudies,wefoundthattheirmethodologiesvary,leadingeachofthemtoarriveatdifferentestimatesofthenumber
andidentityofindividualskilled.Weidentifiedfourprincipalreasonsforthediscrepanciesintracking
organizationsoverallcasualtycountsandcountsperstrike:(1)thenumberofmediasourcesthe
trackingorganizationsreliedon;(2)theBureauofInvestigativeJournalismsuseofnon-mediasources;
(3)theNewAmericaFoundationandLongWarJournalsdecisionnottoupdatefiguresinsomecases
wheremediasourceschangetheircasualtycountsinsubsequentreports;and(4)thetracking
organizationsdecisionsnottoincludesomedatafromsomeoftheirownlistedsources.
U.S.GovernmentsResponsibilitytoProvideEstimates:Thediscrepanciesincountsbythetracking
organizationscredibleandwell-resourcedinstitutionsunderscorethedifficultyofgainingan
accurateunderstandingoftheimpactofdronestrikesfrommediareportsalone.Thepublicandsome
policymakersarecompelledtorelyontheseestimatestojudgetheimpactofdronestrikesbecausethe
U.S.governmenthasnotofficiallyprovidedinformationondronestrikedeaths.Whiletoutingthe
successofthedroneprogramandparticularhigh-profilestrikes,U.S.officialshaveavoidedproviding
specificsandcitednationalsecurity.ThepublichasnoinformationonhowandwhethertheU.S.
tracksandinvestigatespotentialciviliandeaths.
Yet,inotherconflictsettingssuchasAfghanistan,U.S.officialshaveprovidedsomeofthisinformation
withoutcompromisingU.S.security.AstheU.S.governmentanticipatesthecontinuedandexpanded
useoflethaldronetechnology,itowesthepublicagenuineassessmentoftheimpactofdronestrikes,
includingtheeffectsonlocalcivilianpopulations.TheU.S.governmentshould,totheextentpracticable,
provideitsownestimateofthenumberofindividualskilledandtheiridentities.Moreover,insteadof
seekingtodiscrediton-the-groundreportingbyjournalistsandhumanrightsgroupsthatputsforward
evidenceofciviliancasualties,theU.S.governmentshouldinvestigateandaddressthereportedstrikes
andcasualties.
Media&TrackingOrganizationsRoles:Weencouragemediaandobserverstoreconsiderthewaythey
addressthequestionofwhoisbeingkilledindronestrikes.Theyshouldqualifytheiruseofestimates
providedbythetrackingorganizations.Theseestimatesarenotactualbodycountstheyarereports
fromaregionwhereevenseasonedjournalistsandinvestigatorssufferfromlimitedaccess,andwhere
witnessstatementsandofficialsexplanationsmaybebiased.Whenmediaandcommentatorsuse
civiliancasualtyestimates,theyshouldacknowledgetheselimitationsandtheambiguityoftermslike
militant.Likewise,trackingorganizationsshouldacknowledgethelimitationsoftheirstudies,and
addressthemethodologicalproblemswehaveidentified.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
7/36
COLUMBIAHUMANRIGHTSCLINIC 7
Recommendations
TOTHEU.S.GOVERNMENT
1.
Totheextentpracticable,collectandreleaseestimatesregardingpastdronestrikecasualtiesandprovideinformationonnewstrikesafteroperationsarecompleted,includingthefollowing
information:
a. ThetotalnumberofindividualskilledorinjuredinU.S.dronestrikesoutsideofAfghanistan
b. ThetotalnumberofindividualskilledwhomtheU.S.identifiesascombatantsorpersonssubjecttodirectattackunderU.S.legalstandardsorU.S.interpretationsof
humanitarianlaw.
c. ThetotalnumberofindividualskilledwhomtheU.S.identifiesasnotsubjecttodirectattackunderU.S.legalstandards,including:civilianskilledcollaterally(i.e.killedinan
attackonamilitaryobjective,andnotidentifiedasaciviliansdirectlyparticipatinginhostilities);civilianskilledmistakenly(e.g.killedbasedonmistakenidentity);or
individualsotherwiseaffordedprotectionfromdirectattackunderU.S.legalstandards
orU.S.interpretationsofhumanitarianlaw.
d. Thetotalnumberofdronestrikesandthelocationanddateofeachdronestrike.2. DisclosethelegalstandardsanddefinitionstheU.S.governmentusesincategorizingthe
individualskilledasmilitarytargetsorindividualssubjecttodirectattack;civilianssubjectto
directattack(e.g.civiliansdirectlyparticipatinginhostilities);civiliansentitledtoprotectionand
killedcollaterally.
3. Investigatecrediblereportsofciviliandeathbymediaandnongovernmentalorganizations,andmakeamendstofamiliesandlocalcommunitieswhereappropriate.
TOTRACKINGORGANIZATIONS
1. Publishestimateswithaprominentdisclaimer,includinginregardtothesourcesforitsdataset,thevarianceofreportingamongnewsoutletsandovertimeandthepotentialbiasesofprimary
sourcesonwhichidentificationofmilitantsversusciviliansarebased.
2. Fororganizationsthatrelyonalimitedsetofmediasources,expandtherangeofmediasourcesincataloguingdeaths;wheremoremediareportsareunavailable,qualifyestimatesas
incompleteoruncorroborated.
3. IndescribingthefindingsofmediareportsthatciteunnamedPakistaniofficials,acknowledgethepotentialpoliticalbiasorusequalifyinglanguagesuchasallegedandattributivelanguage
suchasaccordingto.Whereadecisionismadetodiscardcertainreportedfiguresinany
particularstrikeestimate,indicatewhichreportwasdiscardedandexplainwhy.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
8/36
8 COLUMBIAHUMANRIGHTSCLINIC
TOMEDIAANDCOMMENTATORS
1. Whendescribingtheoverallnumberandidentityofindividualskilledbydronestrikes:a. ReportedDeaths:Acknowledgethatinformationaboutdronestrikecasualtiesis
limitedasageneralmatter,anddescribethetrackingorganizationsestimatesas
collationsofreporteddeathsinacontextwherevirtuallynomediareportsarebasedoninformationgatheredinsidetheregionfirsthandorabletobeverifiedbythemedia
organizationitself,andwheremediareportssometimesrelyonbiasedsources,e.g.,
anonymousgovernmentofficials.
b. DifferentEstimates:Whencitingasingletrackingorganizationsestimateofthenumberoridentityofindividualskilled,acknowledgewheretherearediscrepanciesbetween
thatestimateandestimatesbyotherorganizations.
2. Whenreportingonparticularstrikes:a. Limitedreporting:Whereappropriate,acknowledgelimitsofreportingandinformation
aboutastrike,e.g.,thatinformationprovidedbylocalsourcescouldnotbeverifieddue
tolimitedaccesstotheregion.b. MilitantDeaths:
i. Regardingestimatesofthenumberofmilitantsversuscivilianskilled,acknowledgethatthedeterminationofwhetheracasualtyiscategorizedas
militantorcivilianisambiguousandcontroversial,e.g.,byusingtheterm
alleged.
ii. Inrecognitionofitsambiguousandcontroversialcharacter,avoidusingthewordmilitantunlessquotingagovernmentofficial;usemorespecific
identifierswherepossible.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
9/36
COLUMBIAHUMANRIGHTSCLINIC 9
I.Introduction:DroneDeathEstimatesasFalseAssurancein
PublicDebateDebateaboutlethaldroneoperationsbytheUnitedStatesoftencentersonthenumberofresulting
casualties,andwhichoftwocategoriesindividualskilledfallintomilitantorcivilian.Inexplainingand
justifyingtheexpansionofU.S.dronestrikesbeyondAfghanistan,U.S.officialsemphasizetheprecisioncapabilitiesofdronetechnologyandlimitednumberofciviliancasualties.
1Yetothershavequestioned
theseclaimsandstatedthatthereisevidencetosuggestthatdeaths,andciviliandeathsinparticular,
aremuchhigherthanU.S.officialsadmit.2
AlthoughtheObamaadministrationhasrecentlyacknowledgeditscovertdronestrikecampaignsin
PakistanandYemen,itcontinuestoavoiddisclosureofbasicdetailsincludingthenumberandidentity
ofindividualskilled.BecauseofU.S.governmentsecrecy,themostcommonsourceforcasualtyfigures
ismediareportsfrominternationalandlocalnewsoutlets.Mediareportsofparticularstrikesare
publishedbyarangeofoutlets:prominentWesternnewspapers,internationalwireservices,regional
newsagenciesandPakistaninewspapers.NewAmericaFoundation,LongWarJournalandBureauofInvestigativeJournalism(trackingorganizations)areamonganumberofotherorganizationsthattrack
thesemediareportsand,catalogingthereports,provideestimatesofthetotalnumberofindividuals
killed,includingthenumberofmilitantskilled.
Overthelastyearespecially,thetrackingorganizationsestimateshavepermeatedandsignificantly
impacteddebateondronestrikes.Newsanalysisandcommentarybypoliticalobserversandexperts
oftencitetheseestimates.Evenhumanrightsexperts,suchastheU.N.SpecialRapporteuron
1PresidentObamahasdescribeddronestrikesasprecise,precisionstrikesagainstAlQaedaandtheir
affiliates.SeeChristiParsonsandMichaelA.Memoli,ObamaOpensUpaboutDroneStrikesinPakistan,L.A.Times,January31,2012(quotingObamainavirtualinterviewonGoogle+and
YouTube).Obamaadministrationofficialshaverepeatedlyemphasizedthatdroneandotheradvanced
technologycanensurethatthebestintelligenceisavailableforplanningandcarryingoutoperations
withtheresultthattheriskofciviliancasualtiescanbeminimizedoravoidedaltogether.Various
administrationofficialshavespokeninnearlyverbatimterms.SeeAttorneyGeneralEricHolder,
AttorneyGeneralEricHolderSpeaksatNorthwesternUniversitySchoolofLaw(Mar.5,2012)available
athttp://www.justice.gov/iso/opa/ag/speeches/2012/ag-speech-1203051.html/ (Infact,theuseof
advancedweaponsmayhelptoensurethatthebestintelligenceisavailableforplanningandcarrying
outoperations,andthattheriskofciviliancasualtiescanbeminimizedoravoidedaltogether.);Harold
Koh,LegalAdviser,U.S.DepartmentofState,SpeechattheAnnualMeetingoftheAmericanSocietyof
InternationalLaw,(Mar.30,2010)availableathttp://www.state.gov/s/l/releases/remarks/139119.htm(Indeed,usingsuchadvancedtechnologiescanensureboththatthebestintelligenceisavailablefor
planningoperations,andthatciviliancasualtiesareminimizedincarryingoutsuchoperations.);Jeh
Johnson,NationalSecurityLaw,LawyersandLawyeringintheObamaAdministration,RemarksatYale
LawSchool(Feb.22,2012),availableatwww.lawfareblog.com/2012/02/jeh-johnson-speech-at-yale-
law-school/(Advancedtechnologycanensureboththatthebestintelligenceisavailableforplanning
operations,andthatciviliancasualtiesareminimizedincarryingoutsuchoperations.).
2See,e.g,ScottShane,C.I.A.IsDisputedonCivilianTollinDroneStrikes,N.Y.Times(Aug.11,2011).
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
10/36
1
COLUMBIAHUMANRIGHTSCLINIC
extrajudicial,summary,orarbitraryexecution,refertotheestimates.3Scholarshipandthinktank
studiestaketrackingorganizationsestimatesasastartingpointforfurtheranalysis.4
OntheirwebsitesandininterviewsandexchangeswithColumbia,noneofthetrackingorganizations
researcherspresenttheirestimatesasdefinitive.Webelievethetrackingorganizationsareengagedina
goodfaithefforttoinformandencouragepublicdebate,andhavenotsetouttoprovideconclusiveanswers.Nevertheless,intheabsenceofotherinformation,theseestimatestransmuteintosomething
likeactualbodycounts.Amongconcernedpolicymakersandobservers,theestimatesaresometimes
discussedasthoughtheywerebasedoncredible,corroboratedandfirsthandresearchratherthana
compilationofnewsarticlesandothermaterialthatonlyrarelyincludeson-the-groundresearch.
Theproblemisthatastheestimatesareassimilatedintofact,theythreatentobecomewhateverybody
knowsabouttheU.S.covertdronestrikesprogram .Theestimatesprovideadangerousassurance:the
humantollissomethingwehaveidentified,debatedandconsidered.Ifweknowwhoandhowmany
peoplewehavekilled,callstoexamineanddeliberateonthedroneprogramandcallstoenditlose
theirurgency.Wemaycometofalselybelievethatcovertdronestrikesareanopensecretwhenin
fact,theU.S.governmentcontinuestoresistdisclosureofbasicandimportantinformationaboutthe
dronestrikesprogram.Moreover,wherethetrackingorganizationsestimatessignificantlyundercount
thenumberofcivilianskilledbydronestrikes,theymaydistortourperceptionsandprovidefalse
justificationtopolicymakerswhowanttoexpanddronestrikestonewlocations,andagainstnew
groups.5
OurstudyunderscorestheneedforresponsibleengagementbytheU.S.governmentontheissueof
dronestrikes,andinparticular,theimpactonlocalcivilianpopulations.Itbuildsontheworkofscholars
andobserverswho,basedontheirfamiliaritywithreportingprocessesandlocaldynamics,haveforthe
pastfewyearsquestionedthereliabilityofdronestrikedeathestimates.6Whiletheestimatesof
3ChristofHeyns,SpecialRapporteuronextrajudicial,summaryorarbitraryexecutions,Follow-upto
countryrecommendationsUnitedStatesofAmerica,30March2012,A/HRC/20/22/Add.3,atpara.
81.4See,e.g.,PatrickB.Johnson&AnoopSarbahi,"TheImpactofU.S.DroneStrikesonTerrorismin
Pakistan,"(February25,2012),availableat http://patrickjohnston.info/materials/drones.pdf5
Foradiscussionofdronestrikesandtheexpansionofgroupsandindividualsofwhomaybetargeted,seeColumbiaHumanRightsClinic&CenterforCiviliansinConflict, TheCivilianImpactofDrones:
UnexaminedCosts,UnansweredQuestions67-77(2012).6Thisreliancehassparkedcritiquesbyjournalistsandscholarsskepticalofthereliabilityofthetracking
organizationscountsandmediareports.See,e.g.,ConorFriedersdorf,FlawedAnalysisofDroneStrike
DataisMisleadingAmericans,TheAtlantic(Blog)(July18,2012);ChaseMadar,SearchforaMethod,
TheNewInquiry.com(July2,2012);MuhammadIdreesAhmad,TheMagicalRealismofBodyCounts,
Aljazeera.com(June13,2011);NYUCenterforHumanRightsandGlobalJustice&StanfordInternational
HumanRightsClinic,LivingUnderDrones43-54(2012).
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
11/36
COLUMBIAHUMANRIGHTSCLINIC 1
Pakistan-basedtrackingorganizationsareoutsideofthescopeofthisreport,theiruseofawiderrange
ofsources,suchaslocalmediareportsinlocallanguagesandhospitalrecords,isinstructive.7
II.OurMethodology
ThisreportbuildsonpreviousstudiesbytheColumbiaHumanRightsClinicandHumanRightsInstitute
regardingtheimpactofdronestrikesonlocalcommunitiesandtheambiguityinU.S.legalstandards
regardingwhomaybetargetedlethally.8
Inthisreport,wesetouttoexaminetheassumptionsandaccuracyofinformationaboutdronestrike
deaths,includingtheidentityoftheindividualskilled.NewAmericaFoundation,LongWarJournaland
BureauofInvestigativeJournalismeacharriveatdifferentestimatesofthetotalnumbersofindividuals
killedandtheiridentities.Tofindoutwhy,wereviewedeachorganizationsdataforallstrikesin
Pakistanin2011,includingthemediareportstheyreferenced,andindependentlycountedreported
casualties.OurstudyconsistedofallreporteddronestrikesinPakistanin2011asignificantand
discretedatasetthatallowsustocompareourresultstothoseoftheotherorganizations,strike-by-strike,andinend-of-yearstatistics.Ourcompletestrike-by-strikecomparison,analysisanddatasetis
availableonline.
Inadditiontotheanalysisandcritiqueofmethodsinthisreport,werecountedthereportedcasualties
foreachstrikeusingeachtrackingorganizationsownsourcesandarrivedatourownestimates.We
usedonlythesourceslistedbytheorganizations,whicharehyperlinkedontheirwebsites(orinthecase
ofNewAmerica,providedinaseparatePDFdocumentwhichishyperlinkedontheirwebsite).Wedid
notseekoutadditionalsources,butwhereahyperlinkwasbrokenandappearedtobethesourceofan
organizationsupperorlowercasualtyfigure,wehavetriedtore-sourcethearticle.
Wegaveeachorganizationanopportunitytoreviewandrespondtoourdataandfindings.BillRoggioof
theLongWarJournalspokewithusextensivelyabouttheorganizationsmethodology;theBureauof
7Wedonotfocusonthesestudiesinpartbecauseinternationalmediararelycitetheirfigures,butalso
becausetheircountsarebasedonUrduandotherlanguagesourcesthatwedonothavetheresources
toverify.Thethreemostwell-knownPakistantrackingorganizationsarePakistanInstituteforPeace
Studies,PakistanBodyCountandConflictMonitoringCenter.Accordingtoitswebsite,thePakistan
InstituteforPeaceStudiesConflictandSecurityDatabaseteammonitorsaround100localandnational
newspapers,magazinesandjournalsonadailybasis,relyingheavilyonregionalcorrespondentsto
cross-checkinformationandseekfurtherdetailsofissuescoveredbythemedia.SeePakInstitutefor
PeaceStudies,DigitalDatabaseonConflictandSecurity,availableathttp://www.san-pips.com/app/database/index.php?id=3(subscriptiononly);EveryCasualty,InternationalPractitioners
Network,PakInstituteforPeaceStudies,availableathttp://www.everycasualty.org/practitioners/pips
seealsohttp://san-pips.com/index.php?action=reports&id=tml3 ;PakistanBodyCount,availableat
http://pakistanbodycount.org/drone_attack;ConflictMonitoringCenter,availableat
http://cmcpk.wordpress.com/drone-attacks-in-pakistan/.8SeeColumbiaHumanRightsClinic&CenterforCiviliansinConflict, TheCivilianImpactofDrones:
UnexaminedCosts,UnansweredQuestions(2012);HumanRightsInstitute,ColumbiaLawSchool,
TargetingOperationswithDroneTechnology:HumanitarianLawImplications(March25,2011).
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
12/36
1
COLUMBIAHUMANRIGHTSCLINIC
InvestigativeJournalismsChrisWoodsreviewedandcommentedonourdatafindingsandspokewith
usextensivelyabouttheBureausmethodology.Wearegratefulfortheirinsightsandcomments.
DespitenumerousattemptstodiscussourfindingswithNewAmericaFoundation,theydeclinedto
comment,butdidsubstantiallyaltertheirdataafterweprovidedourdetailedstrikecounts.9
TheonlyorganizationthatchosetoresponddirectlytousabouteachstrikecountwherewereachedadifferentresultwastheBureauofInvestigativeJournalism.Someinitialdifferencesbetweenourcounts
havebeeneliminatedthroughthisprocess,astheBureauchangeditsdatabasedonsomeofourwork,
andviceversa.Incaseswherewecontinuetodisagree,wehaveincorporatedtheBureausspecific
responsesinourdataset.
Toarriveatourindependentrecount,wetookintoaccountthelowestandhighestreportednumbers
presentedbylistedmediasources,exceptincaseswhereweweresatisfiedthatcertainfigureshad
beensupersededbylaterreportsor,inafewcases,wheretheycontainedevidenttypographical
errors.10
Asdiscussedbelow,webelievethatthisistheappropriateandmostaccurateapproach.We
havetriedtodiscardoutdatednumbersveryconservatively,doingsoonlywhenthereisaclear
differenceovertimeandwherethesamenewsagenciespublishupdatedfigures.Inourview,asthe
majorityofmediareportsprovidesolittleinformationandrelyonsofewsources,itisararecasewhere
areportedfigurecanbeconfidentlydiscardedonanyotherbasis.Wehavetriedtobeastransparentas
possibleabouthowwehavereachedourcountanditscomparisontoallofthereportedfigureswithin
thestrike-by-strikeanalysis.
Incaseswherewehadconcernswiththemediasources,wehavetakenalowerfigureof0.Incases
wherethereareconflictingreportsabouttheidentityoftheindividualskilledaseithermilitantsor
civilians,ourfiguresreflectbothpossibilitiesforexample,if5peoplewerereportedkilled,wetakea
lowerfigureof0andanupperfigureof5inbothcategories.11
Itisimportanttonotethatdespitethegreatcarewetookinreviewingthedataandoriginalmedia
sources,ourestimatesdonotpurporttobereliableoraccurateindicatorsoftheactualnumberof
civilianormilitantcasualtiesofU.S.dronestrikes.TheU.S.governmentshouldprovidesuchfigures
9InanemailresponseonJuly30,2012,PeterBergenoftheNewAmericaFoundationdeclinedtospeak
withus,stating:Wepublishallofourdataandupdateitcontinuouslyonourwebsiteandsoany
questionsyoumighthaveshouldbeansweredbywhatisonthesite.Followingseveralrepeated
inquiries,inanemailonAugust12,2012,JenniferRowlandoftheNewAmericaFoundationaskedfor
ourwrittenfindingsandonAugust15,2012indicatedtheywereunderreview.Despiterepeated
attemptsatcontactbyemailandphone,NewAmericahasnotrespondedwithcommentsorotherinformation.However,inlateAugust2012,NewAmericasignificantlychangedthewaythatitsdatasets
arerecordedandchangedanumberofindividualstrikeaccounts,perthespecificdatadiscrepanciesour
findingshadidentified.10
Ourrecountsupperandlowerfiguresarebasedonlyonmediasources,allofwhicharelistedas
footnotesinourdataset.Themediasourcesdonotincludeunknownblogsorwebsitespurportedlyrun
bymilitantgroups.11
Thisisalsothereasonwhyourminimumcivilianandmilitantcasualtyfiguresdonotadduptoour
minimumfigurefortotalkilled.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
13/36
COLUMBIAHUMANRIGHTSCLINIC 1
andclarifyitsstandardsforthepermissibilityofdirecttargeting.Rather,ourrecountreflectstheextent
towhichciviliancasualtieswhicharecrediblyreportedarenonethelessmissedbythetracking
organizations;itunderscoresthatthepublicandpolicymakersdonothavecompleteinformationabout
thehumanitariancostofdronestrikes.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
14/36
1
COLUMBIAHUMANRIGHTSCLINIC
III.WeaknessesofMediaReportsasaMetricforDroneStrike
DeathsMediareportsareatpresentthebestandperhapsonlywaytogetapictureofthefrequencyandscope
ofU.S.dronestrikesoutsideofAfghanistanandanideaoftheoverallnumberofpeoplebeingkilledby
them.However,therearesystematicflawsandbiasesinthereporting.Whileinsomerarecasesthereis
in-depthreportingonthecircumstancesofparticularstrikes,thisisunfortunatelyfarfromthenorm.
LimitedMediaAccessandFewDetails
Mediareportsofparticulardronestrikesarebasedonlimitedreporting,withthesamefewjournalists
andnewsoutletsprovidingthesamematerialstomultiplewireagenciesandnationalorinternational
press.Moreover,mediareportsareusuallybasedonlimitedon-the-groundinvestigation(withthe
exceptionofsomePakistaninewspapers,whichwenotebelow).Widerandmorein-depthreportingis
typicallyreservedforcaseswhereahigh-levelmilitantleaderhasbeenreportedkilled,orcaseswithan
unusuallyhighnumberofoverallandreportedlyciviliancasualties.Reportsoftenprovidenomore
informationthanthelocationofthestrike,theallegedorapparenttarget(suchasacompoundora
vehicle),thenumberofpeoplereporteddeadandanofficialclaimthatthosekilledweremilitants.
Whilethepublicandpolicymakersmayviewinternationalmediareportsasespeciallycredible,these
outletsincludingtheNewYorkTimes,theBBCandwireservicessuchasReutersandtheAssociated
Pressaregenerallyunabletoaccessthetribalareaswheredronestrikesareoccurring,makingthem
reliantonlocaljournalistsorstringersfortheirreporting.Internationalmediasometimesreportthe
numberandidentityofthosekilledbasedonthereportsofseverallocalstringers.However,the
stringersthemselvesmaybeunabletogototheareawhereadronestrikeactuallyoccurredduetosecurityissues,makingtheirreportssubstantiallydependentonthewordofahandfuloflocalofficials
aproblemweexplorefurtherbelow.Evenwhenstringersareabletoreachtheareaswheredrone
strikesoccurredandconducttheirownon-the-groundreporting,resultingreportsmayfailtoreflect
possibleciviliancasualties;ifonlyonestringerfindsevidenceofciviliancasualties,comparedtothreeor
fourotherstringerswhodonot,internationalmediawillordinarilynotreportthepossibilityofcivilian
casualties.12
Ontheotherhand,reportsofciviliancasualtiesbasedonstatementsofunnamedwitnessesorlocal
villagersmaybeflawed.Soonafterastrike,localmilitantgroupsmaymovethebodiesofindividuals
killed,forburialortoobscuretheidentityofthosekilled.Witnessesmayfearretaliationbylocalmilitant
groupsiftheydiscusswhatoccurred;itispossiblethatlocalgovernmentofficials,covertagents,or
12
SeeAsiaPacificForumWBAI99.5FM,KillerDrones:CountingtheHumanCosts(Interviewwith
journalistMadihaTahir),audioavailableathttp://www.asiapacificforum.organdtranscriptonfilewith
ColumbiaHumanRightsClinic;seealsoNYUCenterforHumanRightsandGlobalJustice&Stanford
InternationalHumanRightsClinic,LivingUnderDrones37-38(2012).
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
15/36
COLUMBIAHUMANRIGHTSCLINIC 1
militantgroupscouldinfluencewitnesseseitheragainstprovidinginformationatall,ortoward
providingmisinformationaboutwhoiskilledalthoughwedonotknowofreportstothiseffect.
WhoDefinesMilitantsversusCivilians?
Reportingondronestrikecasualties,andthetrackingofthosereports,typicallydividethedeadintwo
categories:militantsandcivilians.Thehiddenbiasofthiscategorizationstemsfromtheambiguity
oftheterms:theyarenotdefinedbytheU.S.government,thoughU.S.officialsusethem;theterms
soundvaguelylegal,althoughtheyonlylooselytracklegalandscholarlydebatesaboutwhomaybe
lawfullytargeted.Withoutauniversallyacceptedorstandardizeddefinitionfortheseterms,
categorizationofmilitantandciviliandeathsisbiasedbythedefinitionoftheindividualstowhom
mediareportsciteforidentifyingthedead.WhethertheseprimarysourcesareunnamedU.S.and
Pakistaniofficials,orunnamedlocalvillagersandwitnesses,identificationofthosekilledasmilitants
orciviliansislikelydrivenbypoliticalinterests,andcoloredbytheperspectiveandexperiencesofthe
source.Asthesetermsappearinmediareportsandthetrackingorganizationstudies,theymightbebetterunderstoodasmoralcategoriesofwhoshouldandshouldnotbekilled.Theyare,tothatextent,
inherentlylimitedandbiased.13
Ourreviewofthemediareportsandtrackingorganizationstudiessuggeststhattheyregularlyuse
civiliantorefertoapersonforwhomthereisnoallegationofaffiliationwithamilitantgroup,and
militantforapersonforwhomthereis.Ascolloquialexpressions,theseunderstandingsappear
intuitive.Thus,inmediaandtrackingorganizationreports,namedandwell-knownmilitantgroup
leadersarecategorizedasmilitantsandyoungchildrenandwomenareoftencategorizedas
civilians.
Thedistinctionbetweenmilitantandcivilianismoremuddledandcontroversial,however,than
theseexamplesreflect.14
Theslippagebetweenmilitantandcivilianascolloquialtermsandlegal
termscausesconfusionandmaybeareasonfordifferentaccountingofwhoisacivilian.Asalegal
matter,undersomecircumstancesciviliansmaylawfullybetargetedperhapsleadingU.S.and
Pakistanigovernmentsourcestorefertosuchciviliansasmilitants.15
Thecircumstancesunderwhich
13
Otherobservershavealsomadethisconclusion.See,e.g.,NYUCenterforHumanRightsandGlobal
Justice&StanfordInternationalHumanRightsClinic, LivingUnderDrones30-31(2012).14
Eventheexamplesofmilitantleadersandyoungchildrenandwomencarrycontroversy.Some
commentatorswouldarguethatmilitantleaderscannotbetargetedwhiletheyarenotdirectly
participatinginhostilities.Moreover,civilianwomenandchildrencouldbetargetableunderhumanitarianlaw,althoughyoungerchildrencouldpresumablynotformtheaffiliationsorintentions
necessarytoloseprotectionascivilians.SeeAvrilMcDonald,TheChallengestoInternational
HumanitarianLawandthePrinciplesofDistinctionandProtectionfromtheIncreasedParticipationof
CiviliansinHostilities,AsserInstitute(April2004),availableat
http://www.asser.nl/default.aspx?site_id=9&level1=13337&level2=13379 ;infranote15.15
Theinternationalhumanitarianlawprincipleofdistinctionrequirespartiestoaconflicttodistinguish
inattackbetweencombatants,asdefinedinArticle4oftheThirdGenevaConvention,andcivilians.In
aninternationalarmedconflict,individualswhoarenotmembersofthearmedforcesareciviliansand
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
16/36
1
COLUMBIAHUMANRIGHTSCLINIC
civiliansloseprotectionunderhumanitarianlawandbecomesubjecttodirectattackisamatterofhotly
contesteddebateamonglawyersandscholarsintheU.S.andinternationally.16
Governmentsources
mightprovidelowerciviliancasualtyestimatesthanlocalwitnessesbecauseinternalgovernment
standardspermittargetingofanexpansiverangeofcivilianssuchascivilianswho,thoughnot
membersofamilitantgroup,aresuspectedofsomeaffiliationorofprovidingsomematerialsupportto
militantgroupssometimesbeyondconventionalinterpretationsofhumanitarianlaw.
Yetthereareassumptionsandbiasesatplayindefiningsuchindividualsasmilitantsorinsistingthey
arecivilians:ontheonehand,governmentsmayjustifytargetingbasedonanindividualsprovisionof
suppliestoalocalmilitantgroup,whileontheotherhandlocalwitnessesmightcharacterizethe
individualasinnocentandnotethatthesupplieswerefoodormedicine,orprovidedonlyunder
duress.17
Likewise,governmentsmayjustifytargetinglocalgroupswhoaremeetingormixingwith
individualsidentifiedasmilitants;yetlocalwitnessesandobserversmightcharacterizethosemeetings
andtalksasattemptsatreconciliationorpeace-building,anddecrydronestrikesfortargetingcivilians.
areentitledtoprotectionagainstdirectattack.Inanon-internationalarmedconflict,acustomaryrule
ofdistinctionapplies,whichisformulatedinsimilarterms.SeeGenevaConventionRelativetothe
TreatmentofPrisonersofWar,Art.4,Aug.12,1949,6U.S.T.3316,75U.N.T.S.135(defining
combatant);ProtocolAdditionaltotheGenevaConventionsof12August1949,andrelatingtothe
ProtectionofVictimsofInternationalArmedConflicts(AdditionalProtocolI),art.50-51,June8,1977,
1125U.N.T.S.3,23[hereinafterAdditionalProtocolI];Rules1and3,ICRC,CustomaryInternational
HumanitarianLawDatabase,http://www.icrc.org/customary-ihl/eng/docs/home [hereinafterICRC,
CustomaryLawDatabase].TheU.S.isnotpartytoAdditionalProtocolI,butregardssomeofits
provisionsascustomarylaw.Inthisreport,werefertocustomarylawasrecognizedbyICRCsstudyon
customarylaw,althoughitsviewsdonotalwaysreflectthoseoftheU.S.government.Thereis
nonethelesssubstantialdebateabouthowtocategorizeindividualswhomaybetargetedbydrone
strikes,asmembersofnon-statearmedgroupsorciviliansdirectlyparticipatinginhostilities.16
Amongtheareasofdebate,twoofthemosthighlycontestedissuesare:(1)whomaybetargetedas
directlyparticipatinginhostilitiesandforhowlong;and(2)whomaybetargetedasfulfillinga
continuouscombatantfunction,astatusbywhichmembersoforganizedarmedgroupsceasetobe
civiliansandloseprotectionagainstdirectattack.Forabriefsummaryofpositionsandcontroversies,
seeHumanRightsInstitute,ColumbiaLawSchool, TargetingOperationswithDroneTechnology:
HumanitarianLawImplications(March25,2011),15-23availableat
http://www.law.columbia.edu/ipimages/Human_Rights_Institute/BackgroundNoteASILColumbia.pdf
(discussingandreferencing,interalia,ICRC,InterpretiveGuidanceontheNotionofDirectParticipation
inHostilitiesUnderInternationalHumanitarianLaw(NilsMelzered.,2009);MichaelSchmitt,
DeconstructingDirectParticipationinHostilities:TheConstitutiveElements,42N.Y.U.J.IntlL.&Pol.
697,699(2010);5RyanGoodman&DerekJinks,TheICRCInterpretiveGuidanceontheNotionofDirectParticipationinHostilitiesunderInternationalHumanitarianLaw:anIntroductiontotheForum,42
N.Y.U.J.IntlL.&Pol.637,640(2010);KennethWatkin,Opportunitylost:organizedarmedgroupsand
theICRCDirectParticipationinHostilitiesInterpretiveGuidance,42N.Y.U.J.IntlL.&Pol.640,692
(2010);NilsMelzer,KeepingtheBalanceBetweenMilitaryNecessityandHumanity:AResponsetoFour
CritiquesontheICRCsInterpretiveGuidanceontheNotionofDirectParticipationinHostilities,N.Y.U.
J.IntlL.&Pol.831,833(2010)).17
SeeColumbiaHumanRightsClinic&CenterforCiviliansinConflict, TheCivilianImpactofDrones:
UnexaminedCosts,UnansweredQuestions41-42(2012).
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
17/36
COLUMBIAHUMANRIGHTSCLINIC 1
Ourreviewofmediareportsfrom2011suggeststhat,beyondthechildrenandnamedmilitantleaders,
thereisrarelyenoughinformationprovidedinmediareportsforanoutsiderincludingatracking
organizationtoindependentlyassesswhethertheuseofthemilitantorcivilianlabelisaccurate,
accordingtoitsownstandardsorpresumedlegalstandards.
ArareexampleofdeepermediareportingontheidentitiesofthosekilledfollowedastrikeonBarmal,
SouthWaziristan,onJuly12,2011,18
butthegreaterlevelofdetailunderscoresourpointthatthelabels
militantandcivilianmayturnontheperspectiveoftheprimarysources,andaredifficultfortracking
organizationstoindependentlyassess.Mediareportscitedanonymousofficialsasindicatingthestrike
killedmilitants.PakistanTodayprovidedfurtherdetail,stating5youngsters,associatedwiththe
HaqqaniNetworkwerekilled,anditnamedthreeofthem.DigitalJournalreportedthatastatementin
anUrdudailyfromoneofthedeceasedsuniversityprofessorsdescribedhimasaninnocent,intelligent
studentwhohadcountlessgoodqualitiesandwhodevotedallhistimetohisstudies.Thestatement
fromtheteacherdescribingonestudentasinnocentisnot,inourview,enoughtoamounttoareport
thathewasacivilian.Ontheotherhand,whilethesearequiteclearlymediareportsofmilitant
deaths,wedonotknowwhatitmeanstobeayoungster(under18andthusachild?)whoisassociatedwiththeHaqqaniNetworkinWaziristantheextenttowhichtheassociationincludes
actualinvolvementinmilitantactivities,orbehaviorwhichwouldmakehimdirectlytargetableunder
unknownU.S.standards.
MediaReportsRelianceonUnnamedPakistaniGovernmentSources
Mediareportsprovideaweakbasisfordeterminingtheidentityofthosekilledbecausetheytypically
categorizethedeadasmilitantsbycitingunnamedPakistaniofficials,althoughinsomecasesthey
corroboratestatementsbyofficialswithstatementsbyanonymousresidents.Wedonotknowwhothe
unnamedPakistaniofficialsare,althoughobserversbelievetheyarePakistaniArmyofficials.What
definitiontheseofficialsusetocategorizeapersonasamilitantorcivilianisunknown.Nordoweknow
howthePakistaniArmyconfirmssuchdeathsorthequalityofinformationitisabletorelyon,giventhe
limitedaccessibilityofsomeofthetribalregionstoeventheArmy.CriticsofthePakistanimilitaryargue
thatitdoesnotconducton-the-groundinvestigationsbeforeissuingontheconditionofanonymity
announcementsthatthedeadaremilitants;thesecriticscontendthatitisplausibletheArmyhasa
politicalinterestincategorizingasmanyofthosekilledindronestrikesasmilitants.19
Beyondgovernmentsources,mediareportsrarelyprovideanyadditionalidentifyinginformationabout
thedeadthatwouldenablethereaderortrackingorganizationstoreachtheirownjudgmentonthe
matter.Ourreviewofthe2011dataforPakistanshowsthatoutof330to575militantsreporteddead,
forbetween100to219individualsidentifiedasmilitants,thesolesourcemediareportsprovideis
anonymousofficials.Inthecaseof36to71ofthosedead,absolutelynofurtheridentifyinginformation
isprovidedbeyondtheclaimthattheyweremilitantsorallegedmilitants.Fortheremaining64to148
dead,someidentifyinginformationisprovided,suchasanallegedconnectiontoaparticularmilitant
18
SeeColumbiaDataset,Strike46.19
See,e.g.,MuhammadIdreesAhmad,GunboatsandgurkhasintheAmericanImperium,
AlJazeera.com(July14,2012).
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
18/36
1
COLUMBIAHUMANRIGHTSCLINIC
grouporleader.20
Oftentheadditionalidentifyinginformationissimplyaclaimthatthedeceasedwere
foreign,whichmightconflictwithotherreportsclaimingtheywerelocal.
Thetrendofheavyrelianceonanonymousofficialsisnotlimitedtothe2011dataset.NewYork
UniversitySchoolofLawsGlobalJusticeClinicandStanfordsInternationalHumanRightsandConflict
ResolutionClinicanalyzedthearticlesreliedonbytheNewAmericaFoundationforitsdataondrone
strikesinPakistanfromJanuarytoJuly2012.NYUsanalysisfoundthatin74percentofthearticles,the
onlysourceforthenumberofmilitantskilledwasanonymousgovernmentofficials(almostalways
unnamedPakistaniofficials).21
InconsistentReportingAmongMediaReportsandtheImplicationforTrackingOrganizationCounts
Inthemajorityofstrikereportsthatweanalyzed,diversedeathcountsarereportedamongdifferent
sources.Inconsistenciesastothenumberofdead,theidentityofthedead,theobjecttargeted,and
eventhelocationofastrikearenotuncommon.
Amongreputablenewsorganizations,thereportednumberofdeadoftendiverges.Sometimesdifferent
reportednumbersreflectthedeathcountchangingovertimeasmoreinformationbecomesavailable,
butmoreoftenitappearsthatfiguresaresimplyconflicting.Sometimesthedifferenceissignificant,but
evenwherethedifferencesaresmall,thetotalsaddup.Tovaryingextents,thetrackingorganizations
choicesofapoolofsourcesskewedtheircountsupwardordownward.
Ofthethreetrackingorganizationswhosedataweanalyzed,theBureauofInvestigativeJournalism
typicallycitesthelargestnumberofsources.TheBureauisfollowedbytheNewAmericaFoundation,
withtheLongWarJournaltypicallyrelyingonthefewestsources,oftenonlyoneortwoarticles.The
divergencesinmediareportsandvaryingmediareportpoolsizesisonereasonwhytheBureauhasthe
highestuppercasualtyfigures,followedbyNewAmericaandthenbytheLongWarJournal.22
Interms
ofindividualstrikes,therelianceonalargernumberofsourcesbytheBureauandNewAmericaoften
leadstotheincorporationoflower,aswellashigherfigures.Themostsignificantdifferencethatresults
howeverisamuchhigherupperfigure,andawidergulfbetweenupperandlowerfigures.
Thedivergencesinmediareportsandtherelationshipbetweenthenumberofsourcesandhigher
casualtyfiguresunderscoresthatmediatrackingorganizationsshoulduseawiderangeofcredibleand
reliablemediaincataloguingdeaths.Moreover,whereitisnotpracticallyfeasiblefortracking
20SeeCountsofMilitantsReportedKilled(Pakistan2011)(chart), infraat22.21
SeeNYUCenterforHumanRightsandGlobalJustice&StanfordInternationalHumanRightsClinic,
LivingUnderDrones43-54(2012);seealsoConorFriedersdorf,FlawedAnalysisofDroneStrikeDatais
MisleadingAmericans,TheAtlantic(Blog)(July18,2012),
http://www.theatlantic.com/politics/archive/2012/07/flawed-analysis-of-drone-strike-data-is-
misleading-americans/259836/ .22
SeeColumbiaDataset,Strike4,Strike11,Strike12,Strike15,Strike42,forexamplesofcasesinwhich
relianceonlimitedsourcesbysomeorganizationswasthecauseofvaryingcasualtycounts.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
19/36
COLUMBIAHUMANRIGHTSCLINIC 1
organizationstocollateallrelevantreports,theirstatisticsonreporteddeathsareincompleteand
shouldbeprovidedwithaqualificationthattheydonotrepresentallreports.
Moreover,relianceonEnglish-languagesourcessignificantlylimitsthenumberofreportsthatcanbe
accountedforbythetrackingorganizationsanalyzed,aswellasforourowncount.23
Thisisparticularly
relevantinacontextwherelocalmediaoutletsinnon-Englishspeakingregionsmaybemorewillingandabletoinvestigatetheparticularstrikes.BillRoggio,ManagingEditoroftheLongWarJournal,toldus
thatnecessaryrelianceonEnglish-languagesourcesisahindranceinhiswork.24
Afurtherlimitationofmediatrackingingeneralisthatreportingoneverysingleallegeddroneattack,
andfollowingupontheseattacksasdeathcountschangeovertime,isnotnecessarilythepriorityof
journalistsandmediaoutlets(particularlyEnglishlanguageasopposedtomorelocalones).In
particularlycontroversialorimportantcases,wherestrikeshaveallegedlykilledalargenumberof
civiliansorimportantmilitantleaders,mediacoverageismoreextensive.
UncountedStrikes
Diversereportingillustratesnotonlyhoweasyitcanbetomisssomereportedfigures,butitmayalso
bethecasethatentirestrikesgounreported,orthatreportsofstrikesareoverlooked.
TheBureauofInvestigativeJournalismlistsanumberofcaseswhichmayormaynotbedronestrikes,
forwhichasinglesourceisavailable.WhilethesestrikesarelistedintheBureausdata,thefiguresare
notincorporatedintotheBureauscasualtycount.25
Forexample,TheNews,areputablePakistani
source,reportedthatasecondstriketookplaceonAugust22,2011,killinganArabfamily.Itistheonly
identifiedsourcetohavereportedonthestrike.Inanothercase,UrduTVstationAjjreportedthata
strikeonSeptember4,2011killedupto7allegedmilitants. 26TheonlyidentifiedsourceisChineseoutletXinhua,whichregularlycitesforeignlanguagemediareportsinitsownEnglish-languagereporting.
27
Thedecisionnottocountthesesingle-sourcestrikesincasualtycountsisasensibleone,reflecting
concernaboutthefailureofthemediatomorewidelyreportthestrike.Wecannottellwhyparticular
strikesarenotmorewidelyreportedininternationalandPakistaniEnglish-languagemedia,andwhileit
maybeareflectiononthecredibilityofthereport,itmayalsobeareflectionontheprioritiesofthe
mediainanyparticularnewscycle,oronthedifficultiesofreportingfromsuchremoteregions.Assuch,
itmaywellbethatafamilywaskilledbyastrikeinAugust,andthattheirdeathshavegone
unaccountedforinthecasualtyfigures.
23
However,theBureauhascitedtonewsreportsinotherlanguages,includingUrdu.24
ColumbiaHumanRightsClinicphoneinterviewwithBillRoggio(July23,2012).25
BureauofInvestigativeJournalism,CovertUSStrikesinPakistan,YemenandSomaliaour
methodology,http://www.thebureauinvestigates.com/2011/08/10/pakistan-drone-strikes-the-
methodology2/(lastvisitedOctober9,2012).26
StrikeOb240cintheBureausdataset.27
StrikeOb240ciintheBureausdataset.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
20/36
2
COLUMBIAHUMANRIGHTSCLINIC
Thepotentialforentirestrikestobemissedisalsoillustratedbythedelayedreportingofastrikeon
October26,2011.ReportsofthestrikedidnotsurfaceuntilOctober28,withReutersreporting:
SecurityofficialsandTalibansourcessaidtheremotelocationofthesuspectedstrikepreventeditfrom
beingreportedearlier,andreportsofthestrikeonlysurfacedwhentheinjuredarrivedatahospitalin
MirAli.28
Thestrike,whichreportedlykilledbetween13-22people,includingaseniorTalibanmember,
hadbeenmissedentirelybytheNewAmericaFoundation,buttheyaddedittotheirdatasetafterwe
providedthemourfindings.29
Whileinsomecases,suchaswiththemissedOctober26strike,amorerigorousmethodologymay
improvetheestimatesoftrackingorganizations,apersistentproblemisthebiasesofmediasourceswe
describedabove.Takentogether,theseflawsmakeaggregationofmediareportsaninadequatemetric
fordeterminingthenumberandidentityofindividualskilled.Whileon-the-groundandin-depth
reportingarealwayslimitedinwarzones,weareconcernedthatthepublicandpolicymakersmayfailto
recognizethatestimatesofdronestrikesdeaths,whileprovidedingoodfaith,areincompleteandmay
failtoreflectthetruehumanitariancostofstrikes.
IV.OurRecountofDroneStrikeDeathsWeconductedanindependentreviewofthemediasourcesreliedonbytrackingorganizationsand
arrivedatdifferentcountsontheoverallnumberofindividualskilledandtheiridentities.The
discrepancieswedescribebetweenourcountandthatofthetrackingorganizationsarenotmeantto
impugntheeffortsoftheorganizations.Rather,ourrecountreflectstheextenttowhichcivilian
casualtieswhicharecrediblyreportedmaynonethelessbemissedbythetrackingorganizations;it
underscoresthatthepublicandpolicymakersdonothavecompleteinformationaboutthe
humanitariancostofdronestrikesandshouldnotusetheestimatestojustifyanexpansionor
continuationofdronestrikes.
Basedonaclosereviewofeachtrackingorganizationsdataandmediasources,wecametodifferent
countsofthetotalnumberofindividualskilled,thenumberofallegedcivilianskilled,andthenumberof
allegedmilitantskilled.Whenwecomparedeachofthetrackingorganizationsownoverallestimates
withtheirindividualstrikecounts,wefoundsomediscrepancies.Moreover,whenwereviewedthe
originalmediasourcesthetrackingorganizationsreliedupon,wecametodifferentcounts;ourcounts
weresimilartothatoftheBureauofInvestigativeJournalism,butourcountofreportedcivilian
casualtiesinparticularwassignificantlyhigherthanthecountsbyNewAmericaortheLongWarJournal.
Belowweexplainfurtherthecausesofdifferencesbetweenthethreetrackingorganizationsanalyzed,
andourownwork.Wehavealsoendeavoredtobeastransparentaspossibleaboutourmethodologyforeachindividualstrikeinourstrike-by-strikeanalysis.
28
JibranAhmad,SuspectedU.S.dronekills13PakistaniTaliban,Reuters,Oct.28,2011.29
ColumbiaDataset,Strike67.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
21/36
COLUMBIAHUMANRIGHTSCLINIC 2
ComparingTrackingOrganizationsOverallNumbersandStrike-by-StrikeTallies
Whenweconductedastrike-by-strikereviewbasedonthetrackingorganizationslistedcountsfor
Pakistan2011,weinsomecasescametodifferentoverallcountsofdeathsthantheorganizations
reportedastheiroverallestimates.Belowwelistourrecountoftheorganizationsinformation,with
theirtotalcountsinparentheses:
TheBureauofInvestigativeJournalismo Totalkilled:CountingtheBureausownfiguresonastrike-by-strikebasis,wecounteda
rangeof473to669totalindividualskilled(comparedtotheBureausowntotalcountof
473-663)
o Civilians:Arangeof68to157civilianskilled(theBureaureportsanupperrangeof154civilians)
NewAmericaFoundation:o Totalkilled:Arangeof366to599totalindividualskilled(consistentwithNewAmericas
owntally)o Militants:331to524militantskilled(NewAmericareportsarangeof336to535)o Civilians:Arangeof3to9civilianskilled(consistentwithNewAmericasowntally)o Unknown:Arangeof32to66unknownskilled(NewAmericareportsarangeof27to
58)
TheLongWarJournal:o Totalkilled:Arangeof438to478totalindividualskilled(LongWarJournalreports435
total)
o Militants:Arangeof421to475militantskilled(LongWarJournalreports405)o Civilians:Arangeof3to38civilianskilled(LongWarJournalreports30)
InthecaseoftheBureauofInvestigativeJournalismandtheNewAmericaFoundation,thedifferences
hereareveryminor,applyonlytothemaximumfigures,andarelikelytheresultofasmallcounting
errorinthecompilationofdata.30
ThedifferencesintheLongWarJournalstotalsaremoresignificant,
andarelikelytheresultoftheJournalslessextensiveformofreporting.WhereasNewAmericaandthe
Bureaulistastrike-by-strikecountontheirwebsite,ourstrike-by-strikefiguresfortheJournalcome
fromseparatearticlespublishedoneachstrike.BillRoggiooftheLongWarJournaltoldusthat
generally,theJournalstotalstatisticsreflectwhatispublishedinthearticles,butonoccasionthe
figuresmightbeupdatedwithoutpublishinganewarticle.Wefoundthatinsomecases,anarticles
conclusiononthenumberoridentityofthedeadwasambiguousourtallyoftheJournalsfigures
reflectsthatambiguity,whereastheJournalsownsingle-figuretotalsdonot.31
30
Wewerenotabletodeterminethesourceofthecountingerror.31
ThemostsignificantexampleisthestrikeonMarch17,2011,listedasstrike21inourdataset.The
Journalputsreporteddeathsatbetween14and35,andnotesthataccountsdifferedastotheidentity
ofthedeceased(aseithermilitantsorcivilians).Assuch,ourtallyfortheJournalis14-35totaldeaths,0-
35militants,and0-35civilians.Weconsiderthistobethemostaccurateportrayalofthecontentofthe
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
22/36
2
COLUMBIAHUMANRIGHTSCLINIC
article,howeveritisclearlynotcommensuratewiththeJournalsowntotaltally,whichincludesonly
singlefigures,andinparticular,atotalciviliancasualtyfigureof30fortheentireyear.
0
100
200
300
400
500
600
700
800
Bureauof
Invesgave
Journalism
NewAmerica
Foundaon
LongWar
Journal
OurCount
Maximumcount
Minimumcount
TotalReportedKilled(Pakistan2011)
0
100
200
300
400
500
600
700
NewAmerica
Foundaon
LongWar
Journal
OurCount Militants
Strongly
Idenfied
Maximum
Minimum
CountsofMilitantsReportedKilled(Pakistan2011)
NB:TheBureauofInvesgaveJournalismdoesnotspecifically
trackamilitantcasualtycount
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
23/36
COLUMBIAHUMANRIGHTSCLINIC 2
ComparingOurIndependentCountandCriteriatotheTrackingOrganizationsCounts
Afterrecalculatingthetrackingorganizationsestimatesbasedonastrike-by-strikereview,wereviewed
themediasourcesthetrackingorganizationsreliedupon,andcametoanindependentcountofindividualskilled:
o Total:456to661totalkilledo Allegedmilitants:330to575o Allegedcivilians:72to155
Usingtheminimumcasualtyfigures,wecounted2300percentmoreciviliancasualtiesthanwere
reportedbytheNewAmericaFoundationasciviliancasualties;wecounted140percentmore
civiliancasualtiesthanwerereportedbyNewAmericaFoundationaseithercivilianorunknown
casualties;wecounted140percentmoreciviliancasualtiesthantheLongWarJournalreported;andwe
counted5.9percentmoreciviliancasualtiesthantheBureauofInvestigativeJournalism,towhichour
countwassubstantiallysimilar.
Ourrange,andthediscrepancybetweenourestimatesandthatofthetrackingorganizations,partly
reflectsthedifficultofassessingtheidentityofthosekilledbasedonmediareports.Belowwesetout
ourcriteriaforthisassessment,butwerecognizethesearesubjectivedeterminationsandthatevery
countwillvary;thisinevitablevarianceunderscorestheinherentuncertaintyaboutdronestrikedeath
020
40
60
80
100
120
140
160
180
Bureauof
Investigative
Journalism
New
America
Foundation
LongWar
Journal
OurCount Civilians
Strongly
Identified
Maximum
Minimum
Counts ofCiviliansReportedKilled(Pakistan2011)
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
24/36
2
COLUMBIAHUMANRIGHTSCLINIC
estimatesbasedonmediareports,andthattheU.S.governmenthasultimateresponsibilityfor
accountingforciviliancasualtiesandprovidinganofficialestimate.
CriteriaforIdentifyingCiviliansandMilitantinEstimates
Aswehavenoted,toooftenthemediareportsonwhichtrackingorganizationsbasetheircasualty
countsareweak,bothbecauseoftheambiguityofthetermsmilitantandcivilianandthelimitationsofthemediareportsprimarysources.Wereviewedthe2011mediareportscitedbytracking
organizationscloselytodeterminetherelativestrengthoftheidentificationsofmilitantsand
civilians.Ourownanalysisiscircumscribedbytheambiguityoftheseterms,yetwehavetriedto
isolaterelativestrengthsandweaknessesbasedonthefollowingcriteria:
Strongidentificationwherethedeceasedareindividuallyidentifiedbyname,and/orwherethereportedidentificationofthedeceasediscorroboratedbyanindependentinvestigation(the
on-the-groundinvestigationconductedbytheAssociatedPressandpublishedinFebruary2012,
andtheprimaryinvestigationsoftheBureauofInvestigativeJournalismsresearchersin
Waziristan.)
Mediumidentificationwheretherearemultipleoriginalsourcesfortheidentificationofthedead.Forexample,bothanonymousofficialsandalocalresident.
Weakidentificationwherethereisonlyonesourcefortheidentification.Forthispurposewetreatmultipleanonymousofficialsasonesource,andpluralunnamedresidentsasonesource.
Whilewehavetreatedidentificationbynameasastrongidentification,notallsuchcasesareequal.In
somecases,thenameofanotherwiseunknownpersonisallthatisreported.Inothercases,suchasthe
March17strike,theidentificationofciviliansbothbynameandbyanunusuallywidearrayoforiginal
sources,astrongnarrativeexplainingthecircumstancesofhowsomanyciviliansapparentlycametobe
attacked,andreportingofmassprotestsagainstciviliandeathsasaresultoftheattack,provideforan
evenmoreconvincingidentificationofciviliandeaths,particularlyinabsenceofareasonablecounter-
narrative.
Whilemostnamedmilitantsareleaders,thenamedmilitantskilledalsoincludethethreenamed
youngstersfromtheJuly12strikediscussedabove.Eventhestrongestmilitantidentifications,
identifyingnamedandknownmilitantleaders,arefallible.Thishasbeenproveninnumerousinstances
wherethereporteddeadhavelatersurfacedaliveandwell.32
32
AnotableexampleisHakeemullahMehsud,theleaderofthePakistaniTalibanwhowasincorrectly
reporteddeadonanumberofoccasions.SeeBillRoggio,HakeemullahMehsudnotdeadyet,The
LongWarJournal,April29,2010
http://www.longwarjournal.org/archives/2010/04/a_senior_pakistani_i.php.SeealsoBillRoggio,
HakeemullahMehsudrumoredkilledinrecentdronestrike,TheLongWarJournal,January15,2012,
availableathttp://www.longwarjournal.org/archives/2012/01/hameemullah_mehsud_r.php.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
25/36
COLUMBIAHUMANRIGHTSCLINIC 2
Recognizingthatsuchcategorizationsaremoreanartthanascience,wehavetriedtobeastransparent
aspossibleastohowwehavecategorizedeachstrikeinourstrike-by-strikeanalysis.Wehavealso
brokensomeofthefiguresdownintomoreobjectivesubparts33
:
StronglyIdentifiedMilitants:Total105outof330-575(32%oftheminimummilitantcount)o 34areidentifiedbynamethedeathsof23areunconfirmed,whilethereissomelevel
ofconfirmationfortheremaining11deaths
o 71ofthedeceasedsreportedidentificationasamilitantiscorroboratedbyanindependenton-the-groundinvestigation
StronglyIdentifiedCivilians:Total56outof72-155(78%oftheminimumciviliancount)o 52areidentifiedbynameo 16ofthedeceasedsreportedidentificationasacivilianiscorroboratedbyan
independenton-the-groundinvestigation
o Thisincludes12individualswhoarebothidentifiedbynameandwhosecivilianidentityiscorroboratedbyindependentinvestigation
WeaklyIdentifiedMilitants:Total113-232outof330-575o 100-219areidentifiedasmilitantssolelybyanonymousofficials.Ofthese:
Absolutelynoidentifyinginformationisprovidedfor36-71ofthedeceased Someidentifyinginformationisprovidedfortheremaining64-148(forexample,
thattheywereforeign,alinktoaparticularmilitantgroup)
o 13areidentifiedbyadifferentsingleoriginalsource,ortheoriginalsourceisnotapparentinreports
WeaklyIdentifiedCivilians:Total2-73outof72-155o 2-12areidentifiedascivilianssolelybytheBureauofInvestigativeJournalismsprimary
research,incaseswherethereareconflictingaccounts
o 0-53areidentifiedascivilianssolelybyaresidentorlocalsourceso 0-8areidentifiedascivilianssolelybyananonymousofficial
Thesebreakdownsarerepresentedinthegraphsbelow,asaportionoftheminimumfiguresfortotal
militantsandtotalcivilianskilled.Strongidentificationsarerepresentedinred,andweak
identificationsarerepresentedinblue.
33
Ourdatasetandouranalysisofthestrengthofexistingreportsdoesnotincorporatetheinvestigationsreported
inNYUCenterforHumanRightsandGlobalJustice&StanfordInternationalHumanRightsClinic,LivingUnder
Drones57-66(2012),availableatwww.livingunderdrones.org.Thisreport,releasedafterthefinalizationofour
data,providesfurtherrelevanttestimonyrelatingtothestrikesonMarch17andJune15,andiscitedbythe
BureauofInvestigativeJournalism.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
26/36
2
COLUMBIAHUMANRIGHTSCLINIC
Solesource:anonymous
officials,noidenfying
informaon
Solesource:anonymous
officials,someidenfying
informaon
Othersinglesource
idenficaonorsource
unclear
Idenfiedbyname,death
unconfirmed
Idenfiedbyname,death
confirmed
TotalMilitantsKilled:StrengthofIdenficaon
Basedonminimumcasualtyfigures
TotalCiviliansKilled:StrengthofIdenificaon
Idenfiedbyname
Bothidenfiedbynameand
reportscorroboratedby
independentinvesgaon
Reportedidenty
corroboratedbyindependent
invesgaon
Weakidenficaon:basedon
Bureauresearchbut
conflicngwithotherreports
OtherBasedonminimumcasualtyfigures
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
27/36
COLUMBIAHUMANRIGHTSCLINIC 2
V.ComparisonoftheTrackingOrganizationEstimatesWecomparedthecasualtytrackingconductedbytheBureauofInvestigativeJournalism,theNew
AmericaFoundation,andtheLongWarJournal,fordronestrikesinPakistanin2011,withtheprimary
motivationofunderstandingthequitestarkdifferencesbetweeneachorganizationsresults.The
influencethesetrackingorganizationshavemakesitcrucialtounderstandtheirmethodologiesandthe
reasonswhytheyarriveatdifferentestimates.Wheretrackingorganizationssignificantlyunderestimate
thenumberofcivilianskilled,theymaydistortourperceptionsandprovidefalsejustificationto
policymakerswhowanttoexpanddronestrikestonewlocations,andagainstnewgroups.
Wegaveeachorganizationanopportunitytorespondonastrike-by-strikebasisinthosecaseswhere
ourconclusiononcasualtyfiguresdifferedfromtheirs.Asnotedabove,theNewAmericaFoundation
declinedtospeakwithus,butdidsignificantlychangetheirdataafterreceivingourcritiques.34
Bill
RoggiooftheLongWarJournalspoketousandprovidedmanyinsightsintohiswork,butdidnot
responddirectlytoourstrike-by-strikedifferences.TheBureauofInvestigativeJournalismdidrespond
andengagedinadialoguewithusonparticularstrikes.Inanumberofcases,theBureausfeedback
causedustorevisitouranalysis,ortheBureauagreedwithusanddecidedtochangetheirowndatato
takeaccountofpointsthatweraised.
TrackingCivilianCasualties
Incomparingeachorganizationsworkwithregardstociviliancasualtiesinparticular,itisimportantto
notethattheBureauofInvestigativeJournalismistheonlyorganizationtohaveconsistentlyand
activelytrackedciviliancasualties.
WhiletheLongWarJournaldoesprovideaciviliancasualtycount,BillRoggionotedthatthisisnotthe
Journalsprimaryfocus.RatherthefocusandexpertiseoftheLongWarJournalisinpatternsof
operationsthattheU.S.isconductingthisprogram,whotheyretargeting,wheretheyretargeting,
whoseareasarebeingtargeted,whichTalibancommanderhaspurviewoverthoseareas.Thisis
clearlyreflectedintheJournalsworkunliketheBureauandNewAmerica,theLongWarJournaldoes
notpublishastrike-by-strikelistofcasualties,whichmakestheirtotalcasualtycountsomewhatless
transparent.Itdoeshoweverpublishseparatearticlestrackingeachstrike,inwhichagreatdealof
contextualinformationisprovidedtohelpthereaderunderstandtheplaceofeachstrikewithinwider
U.S.operations.TheJournalalsodoesexceptionalworkintrackingtheallegeddeathsofmilitant
leaders.
UntillateAugust2012,theNewAmericaFoundationsdatasetdidnotincludeacategoryforcivilians
andtrackedonlythedeathsofmilitants,militantleaders,andothers.Theothercategorywas
ambiguousandNewAmericascalculusforitwasunclear;inallofthePakistan2011strikeswherethe
NewAmericaFoundationhadcountedotherdeaths,wefoundthattherewereinfactreportedcivilian
deaths.InlateAugust2012,NewAmericareplacedtheothercategorywithcivilianandunknown
categories.Thisisapositivedevelopment,makingthedatasetmoretransparent.AccordingtoNew
34
WhetherthiswasacontributingcausetoNewAmericaschangesisunknown.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
28/36
2
COLUMBIAHUMANRIGHTSCLINIC
America,theunknowncategoryisusedifthevariousmediareportsaresocontradictorythat[New
Americais]notcomfortabledrawingaconclusion.35
NewAmericasnewcivilianestimateissurprisinglylowcomparedtoitsoldothercountalow-
rangefigureofonly3civilianskilledcomparedtoalow-rangefigure16otherskilledinPakistanin
2011.AlsoperplexingisthatNewAmericasnewciviliancountisonlyonepercentofitstotaldeathcountfortheyear,whereasithaduntilrecentlyclaimedaciviliandeathratein2011of7percent.
36The
shiftingclassificationofcasualtiesandtheambiguousdataunderscorethelackofstandardizationacross
trackingorganizationcounts,andthefallibilityoftheseestimatesasagaugeonthetruenumberof
civiliandeaths.
ReasonsforDiscrepanciesinTrackingOrganizationsCasualtyEstimates
1.Thenumberofmediasourcesreliedon
Asdiscussedabove,thewidertherangeofcrediblemediasourcesreliedupon,thewidertherecorded
reportedcasualtyfigureswillbe.Inmanycases,oneorganizationhasalowerminimumand/orhighermaximumcasualtyfigurethantheotherssimplybecauseitcitedtoanadditionalarticleprovidingthat
higherorlowernumber.Asdiscussedabove,thediversityandinconsistencyinthereportingondrone
casualtiesinPakistanmakestheentirebusinessofmediatrackingdifficult,asthereisalwaysthe
possibilitythatanothercrediblearticlewithanagaindifferentcasualtyfigureisoutthere.Wehave
pointedouttotrackingorganizationswheretheyaremissinganadditionalsourcewhichwouldjustifya
lowerorhighercasualtyfigure.TheBureauofInvestigativeJournalismwastheonlyorganizationto
respondtothisinformation,includingbyincorporatingouradditionalsourceandupdatingtheirfigures.
Inourview,thewiderthenumberofcrediblesourcesreliedupon,themoreaccuratethecasualtycount
isasafigurewhichisintendedtoreflectreportsofdeaths.Inlightofthesparsecontentandlimitedsourcesreliedoninreports,thereisrarelyenoughinformationforatrackingorganizationtobeableto
discardareport,exceptinthecaseofupdatedfigures.
2.TheBureausnon-mediasources
TheBureauhasundertakentwofieldinvestigationsintopossibleciviliancasualtiesinPakistan.This
primaryresearchisoneoftheBureaussourcesforitscasualtyfigures,butisnotincorporatedbyeither
theNewAmericaFoundationortheLongWarJournal.ThefailuretotaketheBureausownresearch
intoaccountisonecauseofdiscrepanciesbetweentheBureausfiguresontheonehand,andNew
AmericaandtheJournalsontheother.Interestingly,theonlyorganizationtociteanyoftheothers
workistheBureau,whichoccasionallycitestoLongWarJournalarticles.
35
NewAmericaFoundation,TheYearoftheDrone:Methodologyavailableat
http://counterterrorism.newamerica.net/drones/methodology.36
PeterBergenandJenniferRowland,CIAdronewarinPakistaninsharpdecline,CNN,Mar.28,2012,
availableathttp://edition.cnn.com/2012/03/27/opinion/bergen-drone-
decline/index.html?iref=allsearch.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
29/36
COLUMBIAHUMANRIGHTSCLINIC 2
TheBureausresearchersundertookfieldinvestigationsinWaziristan,wherealmostalldronestrikesin
Pakistanin2011tookplace.Journalistsdonottypicallyhaveaccesstothearea,andvirtuallynomedia
reportsarebasedoninformationgatheredinsidetheregionfirsthandorareabletobeverifiedbythe
mediaorganizationitself.Assuch,theBureauson-the-groundresearchisparticularlyusefulasatoolfor
corroboratinginformationpresentedinmediareports.
InlightoftheBureausaccesstotheregion,theirprimaryresearchisinourviewatleastasreliableand
credibleasthestandardmediareportswhichallthreeorganizationsrelyon,ifnotsignificantlymoreso.
Inanumberofcases,theBureausresearchershavealsobeenabletoprovideawealthofinformation
regardingtheidentityofthedead,includingnamesandtribes.Thislevelofinformationisrareinmedia
reportsexceptincaseswhereanimportantmilitantleaderhasallegedlybeenkilled.
Inlightoftheabove,wehaveincludedtheBureausprimaryresearchinourowncount.Asdiscussed
above,heretheBureausresearchcorroboratesthereportedidentityofthedead,wehavetreatedthat
identificationasparticularlystrong.
TheBureauhasalsocitedothernon-mediasourcesthatarenotincorporatedbytheNewAmerica
FoundationortheLongWarJournal,suchaslegaldocumentsfiledonbehalfoftheallegedcivilian
victimsofdroneattacks.Suchdocumentsaresometimesabletoprovidemoredetailedinformation,
includingthenamesofthedeceased,thanisavailableinmediareports.Whileitmightbearguedthat
thesedocumentsarebiased,havingbeenfiledonbehalfofapartytolitigation,theysometimesprovide
additionalcorroboratingdetailssometimes,farmoreinformationthanprovidedinmediareportsthat
citeunnamedanresidentorofficial.
3.Theproblemofupdatingfigures
Itisnotunusualfordeathcountstochangeovertime.Thisistobeexpectedinanyinstanceofbombing,
asmoreinformationbecomesavailable,asmorebodiesarepulledfromtherubble,orasinitialsurvivors
diefrominjuries.
Giventhecommoninconsistenciesinthereportingondronecasualties,andthefactthateachstrikeis
notcommonlyrevisitedwithanumberofupdatedarticlesbythesamemediaoutlet,itisoftenvery
difficulttodeterminewhetherdifferencesinreportingindicatethatlaterinformationhasbeenupdated,
orwhetherthedifferencesarejustanotherinstanceofinconsistentreports.Giventhedifficultyof
determiningthisissue,wehavetriedtoapproachitconservatively.
Theorganizationsanalyzedappeartotreatthisissuedifferently,whichisoneofthecausesofthe
discrepanciesbetweentheirfigures.Inanumberofcaseswehavealsoapproachedtheissuedifferently
totheorganizationsanalyzed,leadingtosomediscrepanciesbetweenourowncountandtheirs.Dueto
alackofclarityinthereporting,thereissomeroomforreasonabledisagreementonhowthismatteris
treated.Infairnesstotheorganizationsanalyzed,wealsohavetheadvantageofcomingtothisexercise
atalaterpointintime.Weappreciatethatmediatrackingofcasualtiesisanextremelytime-intensive,
andonemaynothavetheresourcestotrackallnewstrikeswhilecontinuingtorevisitalloftheold
ones.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
30/36
3
COLUMBIAHUMANRIGHTSCLINIC
Eachsuchincidentisdiscussedonacase-by-casebasisinourstrike-by-strikeanalysis,butingeneral
inconsistentupdatingaddsanotherlayerofambiguitytothedatacurrentlyinformingpublicdebateon
drones.Ingeneral,however,theBureauofInvestigativeJournalismappearstoupdateitscountsbased
onlaterreportsmoreconsistentlythantheotherorganizations.Insomecases,theBureauhastreated
earlierfiguresassupersededwherewehaveconservativelydecidedtocontinuetotreatthosefiguresas
relevant.37
TheNewAmericaFoundationandtheLongWarJournalhaveinsomecasescontinuedtousefigures
which,inourview,havebeensuperseded.BillRoggiotoldusthatoccasionally,theLongWarJournal
mightupdateitstotalfigureswithoutpublishinganewarticleonthestrikewhichthenewfigurecomes
from.Asdiscussedabove,wehavebasedouranalysisoftheLongWarJournalsfiguresonitsarticles,
andforthatreasonwecannotbecompletelysurethatanyindividualfigurehasnotbeenupdated.Ina
numberofcases,theLongWarJournalsarticleshavecitedfigureswhichwebelievehaveclearlybeen
superseded.38
Forexample,forastrikeinNorthWaziristanonJune15,2011,theLongWarJournalreliesonanow
deadlinktoDawnforitscountof5militantskilled.HoweverDawnreportedagainonthestrikeonJune
17,identifying4civilianswhowerereportedlykilled.
PriortoitsrecentdataoverhaulinAugust2012,afailuretoupdateitsfigureswasalsoamajorproblem
withtheNewAmericaFoundationscasualtycount.TheBureauofInvestigativeJournalismdescribed
NewAmericasdataduringthistimeasbeingofasnapshotnaturethatonlytakesintoaccountafew
articlesonthedayofthestrike.39
Sinceupdatingitsdataanditswebsite,NewAmericahasmade
significantimprovementsinthisrespect.40
Howeverdespiteupdatingitsdata,thereareafewinstances
inwhichNewAmericacontinuestousefigureswhichwebelievehavebeensuperseded.41
37
Seeforexample,intheColumbiaDataset:Strike10,February20,2011;Strike18,March13,2011;Strike36,June8,2011;Strike41,June27,2011;Strike58,September11,2011;Strike74,November16,
2011.38
SeeforexampleintheColumbiaDataset:Strike11,February21,2011;Strike21,March17,2011;
Strike24,May6,2011;Strike73,November15,2011.39
ChrisWoodsAnalysis:CNNexpertsciviliandronedeathnumbersdontaddupTheBureauof
InvestigativeJournalism,July17,2012,availableat
http://www.thebureauinvestigates.com/2012/07/17/analysis-cnn-experts-civilian-drone-death-
numbers-dont-add-up/40
ButseeNYUCenterforHumanRightsandGlobalJustice&StanfordInternationalHumanRightsClinic,
LivingUnderDrones48(statingthatNewAmericaFoundationsstrikedatadonotappeartobe
updatedregularlytoincludethemostup-to-dateinformationaboutthenumberandidentitiesofvictimskilledindronestrikes). 41
SeeintheColumbiaDataset:Strike11,February21,2011;seealsoStrike21,March17,2011.Itis
puzzlingthatNewAmericadoesnotincorporatetheAPsindependentinvestigationfromFebruary2012
initslistofsources,althoughAssociatedPresssfigurefitsroughlywithinNewAmericasrange.Asone
oftheveryfewinstancesofonthegroundinvestigation,AssociatedPresssreportis,inourview,a
particularlyvaluableandcrediblesource.InaCNNarticlebeforeitsdataupdate,PeterBergen
comparedNewAmericasfigurestoAssociatedPresssacuriousapproachforamediatracking
organizationwhichonemightexpecttoincorporatesuchreports.SeePeterBergenandJennifer
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
31/36
COLUMBIAHUMANRIGHTSCLINIC 3
4.DisregardingSources
Insomecases,theorganizationsfiguresdonotreflecttherangereportedinitsownlistedsources.
Wherelowerfiguresinarangearedisregardedbytheorganizationcitingthem,weassumethisis
becausetheorganizationcitingthemhastreatedthosefiguresassuperseded,amatterwhichweaddressabove.Disregardingupperfiguresishowever,inourview,slightlyhardertoexplain.
Insomecasesitmaysimplybeanerror.Thiswasthecasewiththeoneinstanceinwhichwefoundthe
Bureauhadnotincorporatedtheupperfigurefromoneofitslistedsources,asimplemistakewhichthe
Bureauthenrectified.
ThefailuretoincludesomefiguresishoweverdeliberateinthecaseoftheLongWarJournal,which
doesnotcountarangeofcasualtyfigures,withanupperandlowercount,butratherprovidesasingle
figure.BillRoggioexplainedtousthatwherethereisaninconsistencybetweenthesourcesthathe
cites,hetakesthefigurethathebelievesisthemostreliable.42
InsomeinstancestheNewAmericaFoundationhasalsofailedtoincorporateanupperfigurereported
initssources.43
SinceNewAmericaordinarilyreportsarangeofmediafigures,thereasonforthese
omissionsisnotalwaysclear.NewAmericaindicatesinitsmethodologythatitwillonlycountreported
civiliandeathsiftheyarereportedinmorethanonemediasource.44
Forreasonsdiscussedabove,inour
viewthisisnotappropriate.Asreportsofdronestrikesarebasedonsofeworiginalsources,the
reportingiserratic,andthebasisforidentificationisgenerallyextremelyweak,itisnotnecessarily
sensibletotreataclaimasreliablebasedsolelyonthenumberofoutletsitisrepeatedin.Forexample,
whilethesameofficialsmaybequotedinthewires,perhapsonlyonepaperwillquotearesidentor
afamilymember.Asofficialsarethemostcommonlycitedsource,thisalsoskewssources.For
example,inregardtotheMarch17strike,NewAmericalistsonlyavarietyofofficialsastheoriginal
source,whereasourownanalysisidentifiesanexceptionallylargeanddiversearrayofsources.45
Rowland,Civiliancasualtiesplummetindronestrikes,CNN,July14,2012,availableat
http://edition.cnn.com/2012/07/13/opinion/bergen-civilian-casualties/index.html. 42
Mr.RoggiogaveoneexampleofXinhuareporting11militantskilled,andAgenceFrancePresse(AFP)
reporting12militantskilled.AstheAFPreportwaspublishedalittlelaterandhetrustedAFPs
informationmore,heusedtheAFPnumber.ColumbiaHumanRightsClinicphoneinterviewwithBill
Roggio(July23,2012).43
SeeforexampleintheColumbiaDataset:Strike7,January23,2011;Strike28,May16,2011(failure
toincorporateCNNsupperfigureof10,althoughperhapstreatingitasatotalfortwostrikesontheday).44
NewAmericaFoundation,TheYearoftheDrone:Methodologyavailableat
http://counterterrorism.newamerica.net/drones/methodology45
Including:anonymousofficials;namedofficials(includingArmyChiefofStaffGen.AshfaqKayani,
AmbassadorHussainHaqqani,ForeignSecretarySalmanBashir,GovernorMasoodKasur,PrimeMinister
Gilani,MemberofNationalAssemblyKamranKhan);theTaliban;unnamedandanonymous
locals/tribesmen;namedlocals/tribesmen(farmerGulAhmed,SamiullahKhan);Bureauinvestigation;
legaldocumentsandcourtproceedingslodgedonbehalfofvictims.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
32/36
3
COLUMBIAHUMANRIGHTSCLINIC
Inanotherexample,NewAmericahadinitiallyincludedanumberofreferencestothereputedAfghan
publicationPajhwok.WhilePajhwokhadreportedciviliancasualties,NewAmericahadnotincorporated
theseclaimsintoitsdata.Inthemostrecentversionofthedataset,allofthereferencestoPajhwok
havesimplybeendeleted.
Moreover,insomecaseswheretherearemultiplestrikesononeday,NewAmericaappearstotreatthe
factthatsomemediaoutletshavenotreportedonallofthestrikesasevidencethatnotallofthestrikes
occurred.Forexample,fortwostrikesonJuly11,2011,mediareportsindicatethatatleast5people
werekilledinonestrike,andatleast10inanotherstrike.However,NewAmericatakesaminimum
countof7forbothstrikestogether,basedonaBBCreport.TheBBCreport,though,refersonlytoone
ofthetwostrikes:anattackonavehicle,andnotanadditionalattackonacompound.Assuch,New
Americasdataeitherintentionallyormistakenlyconflatesthetwostrikes,withtheeffectthatits
minimumcasualtycountmissesanentirestrike.IfNewAmericasomissionwasdeliberateandbasedon
theBBCsfailuretoreportthesecondstrike,itwouldnotinourviewbeappropriate;mostindividual
mediaoutletsdonotreportoneverysinglestrike.
Inourview,accountingforarangeofcasualtyfigures,asisdonebytheBureauofInvestigative
JournalismandgenerallybytheNewAmericaFoundation,isamoreaccurateandinformativewayof
trackingreportedcasualtiesthantheLongWarJournalsapproach.Whatisbeingtrackedisreportsof
deaths,andwherecrediblereportspresentdiverseresultsthatshouldbereflected.
Ifupperorlowerfiguresarediscardedandsourcesareusedselectivelyforanyreasonotherthanthe
existenceofupdatedinformation,thesereasonsneedtobemadeclearandexplicitinthedataandwith
thefinalcounts,aswhatisbeingcountedisthensomethingmorerefinedthanreporteddeathswhich
ishowthestatisticsarepresented.
4. NewAmericaFoundationsunknownandciviliancategoriesAsnotedabove,NewAmericaincludesanunknowncategoryincaseswherethereareconflicting
reportsabouttheidentityofthedead.Thisisarecentdevelopmentthatwasaddedafterwehad
completedourown2011analysis.Ourownapproachtoconflictingreportshasbeentotakealower
figureof0inboththecivilianandmilitantcounts.
Inourview,NewAmericasapproachinthisrespectisareasonableandmethodologicallysoundone.
Howeverinpractice,ourviewisthattheuseoftheunknowncategoryintheMarch17strikehas
unrealisticallyskewedtheciviliandeathcountdownwards.Althoughthenumbersaredifficultto
preciselydetermineinthatcase,reportsare,inourview,clearthatalargenumberofciviliansandsome
militantsdied.Inourview,24civiliansfromthisstrikealoneareparticularlystronglyidentified.
TheuseoftheunknowncategoryalsositssomewhatuncomfortablyalongsideNewAmericas
methodologyofnotcountingreportedciviliandeathsunlesstheyarefoundinmorethanone(English
language)newsoutlet.Assuch,whiletheunknowncategoryisusedincaseswherewewould
categorizetheidentificationofciviliansasstrong,anumberofreportsofciviliandeathsarenot
accountedforineithertheupper-civiliancasualtyfigureortheunknowncount.Thisisamatterof
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
33/36
COLUMBIAHUMANRIGHTSCLINIC 3
methodologicaldifferenceinopinion,butinourviewthefinalresultisnotanentirelyaccurateportrayal
ofwhatispresentedinthemedia.46
ProblematicCivilianCasualtyEstimates
Ourclosereviewofthedatasuggestsmethodologicalproblemsintheciviliancasualtytrackingofthe
NewAmericaFoundationandtheLongWarJournal.Whileinmanycasesthereisroomforreasonable
disagreementonthemethodologicalapproachtaken,thetotaleffectofnumerousproblemswefound
resultsinsignificantundercountingofciviliancasualtiesinparticular.Asdiscussed,sincethepurported
focusofLongWarJournalisnotciviliancasualtytracking,thisisperhapsnotsurprising.
Theseverityoftheundercountisapparentwhenwecompareourfindingsofstronglyidentified
civiliandeathstoNewAmericaandLongWarJournalsfindingsofciviliandeathsgenerally.Wehave
assessed52ofthereporteddeadasstronglyidentifiedascivilians;theyareidentifiedbynameinmedia
reports.Ourfigureissignificantlyhigherthanthe totalciviliandeathcountsofNewAmericaandthe
LongWarJournal,at3-9and30,respectively.TheirfiguresdonotevencapturetherangeofreportedciviliandeadinonesinglestrikeonMarch17,2011oneforwhichthereisanexceptionallystrongcase
ofnumerousciviliandeaths.
Interestingly,thesameseverityofundercountingcomparedtostrengthofreportingisnotapparentin
themilitantcasualtycounts.Themilitantcasualtycountsrelyheavilyonwhatwehaveassessedasvery
weakreportedidentifications.NewAmericaandtheLongWarJournalslowermilitantcountishigher
thanourown,andsignificantlyhigherthanourcountofstronglyidentifiedmilitants.Whilethelatteris
tobeexpected,itcontrastssharplywiththecomparisontoourciviliancasualtystatistics.
46
SeeforexampletheNewAmericaFoundationsdataandreportingofthefollowingstrikes(allin
2011):7JanuaryGharLaley;21FebruarySpalga;8MarchLandidog;27JuneMantoi;1AugustAzam
Warsak;16AugustMiranshah.
7/31/2019 COLUMBIA LAW SCHOOL - Human Rights Clinic: COUNTING DRONE STRIKE DEATHS
34/36
3
COLUMBIAHUMANRIGHTSCLINIC
VI.U.S.GovernmentsResponsibilitytoAccountforDrone
StrikeDeathsThediscrepanciesincountsbythetrackingorganizationscredibleandwell-resourcedinstitutions
underscorethedifficultyofgaininganaccurateunderstandingoftheimpactofdronestrikesfrommedia
reportsalone.Thepublicandsomepolicymakersarecompelledtorelyontheseestimatestojudgethe
impactofdronestrikesbecausetheU.S.governmenthasnotofficiallyprovidedanyinformationon
dronestrikedeaths.Whiletoutingthesuccessofthedroneprogramandparticularhigh-profilestrikes,
U.S.officialshaveavoidedprovidingspecificsandcitednationalsecurity.Indeed,althoughithas
acknowledgedoperationsinPakistanandYemenasageneralmatter,ithasrefusedtoofficially
acknowledgetheexistenceofitsdronesprogramincourtoropensessionsofCongressforeclosing
effectivelitigationandpreventinginformedpublicdebate.47
Ontheotherhand,theCIAhasaggressively
fendedoffcriticismthroughanonymousleakstothepressaforuminwhichitsclaimscannotbe
activelyquestioned.
ThepublichasnoinformationonhowandwhethertheU.S.tracksandinvestigatespotentialcivilian
deaths.Yet,inotherconflictsettingssuchasAfghanistan,U.S.officialshaveprovidedsomeofthis
information,withoutcompromisingU.S.security.48
AstheU.S.andothergo