University of California, Hastings College of the LawUC Hastings Scholarship Repository

Faculty Scholarship

2012

A New Look at Dismissal Rates in Federal CivilCasesScott DodsonUC Hastings College of the Law, dodsons@uchastings.edu

Follow this and additional works at: http://repository.uchastings.edu/faculty_scholarship

This Article is brought to you for free and open access by UC Hastings Scholarship Repository. It has been accepted for inclusion in Faculty Scholarshipby an authorized administrator of UC Hastings Scholarship Repository. For more information, please contact marcusc@uchastings.edu.

Recommended CitationScott Dodson, A New Look at Dismissal Rates in Federal Civil Cases, 96 Judicature 127 (2012).Available at: http://repository.uchastings.edu/faculty_scholarship/1396

A NEW LOOK MSpncly exam i onit e ;-of the icbz a-"oby dcIsions cot it p

by SCOTT DODSON

For most of the last seventy years,the Supreme Court has interpretedthe Federal Rules of Civil Proce-dure to impose on federal-court liti-gants a civil pleading regime callednotice pleading. Under this regime,a civil claim may be dismissed underRule 12(b)(6) only if legally insuffi-cient-only if the claim could not bemaintained under any set of facts.The pleading of facts was necessaryonly to provide "fair notice" of theclaim to the defendant.'

Beginning in 2007, the SupremeCourt abruptly changed course byissuing Bell Atlantic Corp. v. Twomblyand Ashcroft v. Iqball which impose anew factual-sufficiency requirement.Under these cases, a court should dis-regard all "conclusory" allegations(whether of law or fact) and assess

the "plausibility" of the claim usingthe judge's own 'judicial experienceand common sense." If the judgedetermines that the complaint doesnot supply sufficient facts to allowthe claim to cross the threshold fromthe conceivable to the plausible, thenthe claim should be dismissed underRule 12(b)(6).

Such a sea change in pleading stan-dards immediately set off alarms inprocedural circles. Serious doctrinal,normative, and institutional discus-sions immediately ensued. But manyof those discussions were based, inpart, on empirical speculation ofwhat real effects the new pleadingsdecisions were having on civil claimsin the federal district courts.

Commentators obliged by studyingthe cases' effect. However, because

of coding and collection difficul-ties, those studies have approacheddata with two problematic meth-odological choices. The first is thatstudies have tended to code wholecases rather than claims, leading tothe ambiguous coding category of"mixed" dismissals and to problems

I am indebted to Joe Cecil, Eric Kades. SarahStafford, and Tom Wilging for reviewing orcommenting on my study and its development.This Article benefited from comment-, reciivedat presentations at the 2011 SEALS annual con-ference and at William & Mary Law School. Ialso am grateful for the immensely valuablecomments of two anonymous referees Manythanks to the student research assistants whohelped compile and analyze the data, includingMatt Beard, Andrew Grindrod, Travis GunnAntonia Miller, Bill Novick, Chris Sickles, andSam Zimmerman.

. Conleyv. Gthson, 355 U.S. 41, 45-46 (1957)_2. 550 U.S. 544 (2007).3. 556 U.S 662 (2009).

in characterizing the nature of thedispute. The second is that studieshave failed to distinguish betweenlegal sufficiency and factual suffi-ciency. These methodological choicespotentially mask important detailabout the effects of the pleadingschanges.

This paper begins to fill in thatdetail. I compiled an original datasetof district court opinions and codedeach claim-rather than wholecase-subject to an adjudicated Rule12(b)(6) motion. For each claim, Ialso determined whether the courtresolved the motion on groundsof legal or factual sufficiency. Thismethodology opened an unprec-edented level of granularity in thedata.

The data reveal statistically signif-icant increases in the dismissal rateoverall and in a number of subsets ofclaims. Prior studies based on casesrather than claims have consistentlyfound modest increases in the dis-missal rate but have differed in theirfindings of statistical significance,My findings, based on claims ratherthan cases, suggest that the priorstudies' case-based coding choicesmay mask some significance.

I also find an Increase in theprevalence and effectiveness offactual-insufficiency arguments fordismissal. Perhaps surprisingly Ifind a decrease in the prevalence andeffectiveness of legal-insufficiencyarguments for dismissal. These dataand insights on the rationales of dis-missals are new to the literature andsuggest that TWombly and lqbol areaffecting both the strategy employedby movants and the rationale fordeciding motions to dismiss,

Overall, I find evidence thatTwombly and lqbal are affectingpleading-stage dismissals in federaldistrict courts in a variety of impor-tant ways not adequately captured orreflected by prior studies.

This article proceeds as follows:First I offer background on pleadingand the previous studies, isolatingsome of their deficiencies and dem-onstrating the need for additionalstudy. Then I outline my methodologyfollowed by results. Last I analyze

the methodology and results, explor-ing possible areas for further study.

Civil pleading standardsin federal district courtRule 8(a)(2), which governs most civilpleadings in federal court, requiresonly "a short and plain statement ofthe claim showing that the pleaderis entitled to relief,' The rule wasdesigned to relax pleading stric-tures, requiring only a viable legalclaim (legal sufficiency) and enoughdescription of the circumstances tonotify the defendant of the generalnature of the dispute (notice).'

Rule 12 provides mechanisms fortesting a complaint's sufficiency.Rule 12(b)(6) addresses legal insuf-ficiency. It allows a party to moveto dismiss a complaint for failure tostate a claim upon which relief can begranted.' In the seminal case Conley. Gibson, the Court stated:

In appraising the sufficiency ofthe complaint we follow, of course,the accepted rule that a complaintshould not be dismissed [under Rule12(b)(6)] for failure to state a claimunless it appears beyond doubt thatthe plaintiff can prove no set of factsin support of his claim which wouldentitle him to relief.

Under this standard, a federalcourt could dismiss a civil claimunder Rule 12(b)(6) only if legallyinsufficient-only if the claim couldnot be maintained under any set offacts. If the law does not recognizethe claim or prohibits it under thecircumstances alleged, then relief

4. FED . CIV. P 8(a)(21.5. Charles E. Clark, Simplfied Pleading, 2 F R.D.

456,460-61 (1943).6. PEo. R. C. P. 12(b)[6).7.355 U.S. 41,45-46 (1957).8. Swlerklewlcr v, Sarema NA.. 534 U.S. 506,

514(2002).9. FED. R. CiV. P. 12(e).10. See, e.g, Robert G. Bone, Plausibility Plead-

ing Revisited and Revised:A Comment on Ashcroftv. Iqbal. 85 NoTE 4 Ds L. REV. 849,865 (2010);Stephen B. Burbank, Pleading and the Dilem-mas of "General Rules," 2009 Wis. L. REv. 535.550-51; Kevin M. Clermont & Stephen C. Yeazel,inventing Tests. Destabilizing Systems. 95 IOWAI. Rev 821, 837-38 (2010); Wendy GerwickCouture, Conley v. Gibson's "No Set of Facts" TestNeither Cancer Nor Cure, 114 PENN ST. L. REYPari' STATm 19, 29 (2010); Richard L. Marcus,The Revival of Fact Pleading Under the FederalRules of Civil Procedure, 86 CoLwm. L Rev. 433,

cannot be granted even if all of theplaintiff's allegations are true. Forexample, a complaint that assertsa federal claim for discriminationon the basis of baldness should bedismissed as legally insufficientbecause the law does not recognizesuch a claim and provides no relieffor it, even if every fact the plaintiffalleges is true.

Conley also imposed a require-ment that the complaint provide "fairnotice" of the claim to the defendant.Any lack of notice was remedied notby dismissal but by a more definitestatement under Rule 12(e),' whichpermits a court to order the plaintiffto file a more definite statement if thecomplaint "is so vague or ambiguousthat the [defendant] cannot reason-ably prepare a response."'

Thus, under Conley, and as under-stood by most commentators, Rule12(b)(6) was for legal insufficiencyonly; factual insufficiency wastied to a lack of notice and remedi-able by a more definite statement."0

By contrast, weak or implausibleclaims were largely unremediableat the pleading stage; those quint-essentially merits-based defectswere relegated to later stages, suchas summary judgment, which weredesigned to test the factual meritori-ousness of claims after the opportu-nity for discovery."

Many lower courts have resistedthe liberality of notice pleading byapplying stricter tests of factualinsufficiency." judges of thosecourts feared that Rule 12(b)(6) was

438 (1986); Arthur R. Miller, From Conley toTwombly to lqbal. A Double Play on the FederalRAles of CIvil Procedure, 60 DuE 1.], 1. 18 wi59(2010); Emily Sherwin. TheStoryofConley: Pree-edent by Accident, in CIVIL PROCEDURE STORIES295, 316 n.83 (Kevin M. Clermont ed., 2d ed.2008); A. Benjamin Spencer, UnderstandingPleoding Doctrine, 108 Mic. L, Rv. 1, 20 (2009);Adam N. Steinman, The Pleading Problem, 62STAN. L. Rev. 1293, 1321 (2010).

11. FED. R. Civ. P. 56.12. Christopher M. Pairman, The Myth of

Notice Pleading. 45 ARitz L. REV. 987 (2003);Christopher M4. Fairman. Heightened Pleading.81 TEX. L. REV. 551 (2002); Richard L, Marcus.The Puzzling Persistence of Pleading Practice,76 Tax, L. Rev. 1749 (1998); Marcus, supra note10; Stephen N. Subrin, How Equity ConqueredCommon Law: The Federal Rules ofCivil Procedurein Historical Perspective, 135 U. PA. L. REv. 909.983-84 (1987).

J

toothless, and some commentatorshave credited that fear. Neverthe-less, during much of the last sev-enty-five years, the Supreme Courtrepeatedly endorsed the lax struc-ture described above.Y

In 2007, however, the Court sur-prised everyone with Bell Atlan-tic Corp. v. 79ombly.ts There, theCourt held that a complaint alleg-ing a federal antitrust conspiracy,supported only by allegations ofconscious parallel conduct, failedRule 8(a)(2) and should be dismissedunder Rule 12(b)(6). Undeniably, theclaim was legally sufficient: federallaw does prohibit conspiracies torestrain trade,"' and the plaintiffsdid not allege any facts that wouldhave precluded liability. And thecomplaint seemingly provided ade-

13. Geoffrey C. HazArd. jr.. From Whom NoSecrets are Hid, 76 Tex. L. Rev. 1665, 1685(1998) ("Literal compliance with Conley v. Gibsoncould consist simply of giving the names ofthe plaintiff and the defendant, and asking forjudgment."); Mark Hermann & James M. Beck.Opening Statement, Plausible Denial: Should Con-gress Overrule Twombly and Iqball, 158 U. PA.L. Rrv. PENNumeA 141, 143 (2009) ("Takenliterally. the Conley dictum could make it timpos-sible for a defendant to win a motion ti dismiss,thus rendering Federal Rule of Civil Procedure12 a nullity."), available at http://www.pen-numbra.com/debates/pdfs/PlausbleDenial.pdf;Rebecca Love Kourlis etal, Reinvigoroting Plead-ings, 87 DaNy1 U. L, Rev. 245. 252 (2010) ("Con-fey's 'no set of facts' standard did not appear torequire the recitation of any facts atthe pleadingstage."); William M. Janssen, lqbal "Plausibility"in Pharmaceutical and Medical Device Litigation,71 LA. L. REv. 541, 54849 (2011) (calling Rule 8"toothless").

14. See Swierkiewicz v. Sorema N.A., 534 U.S.506 (2002); Leatherman v. Tarrant C'ty Narcot-ics Intelligence & Coordination Unit, 507 U.S. 163(1993); Schener v. Rhodes 416 U.S. 232 (1974);Conley v. Gibson, 355 U.S. 41 (1957). Some com-mentators, looking at other cases, see moreambiguity in the Court's pleading doctrine. SeeEdward i. Cooper, Kfig Arthur Confronts 7WqyPleading, 90 OR. L. Rev, 955, 963-64 & n.27(2012); ScorT DODSON, SLAMMING THE FEDERALCOURTHOUSE Doons? New PLEADING IN THETWENTY-FIRsT CearutY (Oxford Univ. Press,forthcoming 2013),

15. 550 U.S. 544 (2007).16. 15 U.S.C. § 1.17. 7veombly, 550 U.S. at 557 (acknowledg-

Ing that the allegations were "consistent with"liability). But see Allan Ides, Bell Atlantic andthe Principle of Substantive Sufficiency UnderFederal Rule of Civil Procedure 8(a)[2]: Towarda Structured Approach to Federal Pleading Prmc-tice, 243 F.R.D. 604, 620 (2007) (contending thatthe plaintiffs peladed themselves out of court).

18. Thvombly 550 U.S. at 565 n.10: Id. at 588 n.0(Stevens, J., dissenting).

19. Id. at 556-57.20. Ashcroft v. ibat, 556 U.S. 662, 677-80

(2009).

quate notice of the claim and itsgrounds." Nevertheless, the Courtdismissed the complaint becauseIt failed to allege sufficient factsshowing a "plausible" entitlement torelief." Crucially, TiWombly does noteliminate the old requirements oflegal sufficiency and notice; rather, itadds a new dimension of factual suf-ficiency.

In 2009, the Court confirmed andexpanded Twombly in Ashcroft vlqbal2se The Court made three signifi-cant holdings. First, judges may dis-regard all "conclusory" allegationswhen deciding a motion to dismiss.2

This was a reversal of prior pleadingdoctrine, which required courts tocredit all factual allegations unlesswholly incredible and to evaluate allfactual inferences In the light most

21. Id22. See 58 Cu.i4aes ALAN WiaowT & ARTHuR

R. MiLLER, FED- PRAc & PRoc. § 1357 (3d ed.2004) ("For purposes of the motion to dismiss,(1) the complaint is construed in the light mostfavorable to plaintiff, (2) and its alliegations aretaken as true._. Basically what this means,.is [the court] will accept the pleader's descrip-tion of what happened to him or her along withany conclusions that can reasonably be drawntherefrom.").

23. Iqbal 556 U.S. at 684-86.24. Id. at 678.25. Evambly, 550 U.S. at 562.26. Four studies consider the effects of

'vembly alone. See Kendall W. Hannon, Note,Much Ado About Avombly?A Study on the Impactof Bell Atlantic Corp. v. Twombly on IZ(b)[6)Motions 83 NOTRE DAME L. Rev. 1811 (2008)Joseph A. Seiner, The Trouble with Twombly:A Proposed Pleading Standard [or EmploymentDtscrimination Cases, 2009 U. ILL. L. REv. 1011;Joseph A. Seiner, Pleading Disablity, 51 B.C.L. REV. 95 (2010); William it.l. Hubbard, TheProblem ofMeasuring Legal Change, with Applica-tion to Bell Atlantic v. Twombly (draft of June 14,2011). 'that limitation is problematic becauseit was not until lqlbal that the new pleadingregime was settled as meaningful and trans-substantive. The results of these studies mightbe as Indicative of the confusion surround'Ing the lWambly standard as of the changingrate of dismissal under the new standard. Itis true that some uncertainty persists afterlqbal, see Alex Reinert, Pleading as Information-Forcing, 75 L. & CoNrEMp. PRoss. 1, 2, 16 (2012)(documenting disparate applications of Iqbal),but Iqbal resolved many of TiWembl's broaderambiguities.

27. Patricia W. Itatamyar Moore, An UpdatedQuantitative Study of Ilqbal's Impact on 12(b)(6)Motions, 46 U. RiCa. L. Rev. 603 (2011); PatriciaW. Hatamyar. The Too of Pleading: Do Tw'omblyand lqbal Matter Empirically?, 59 AM. U. L. Rev553 (2010).

28- Hatamyar, supro note 27, at 589-96.29. d. at 621-22.30. Moore, supra note 27, at 605.31. Id. at 605.32. Id. at 618, 622-23.

favorable to the pleader.2 Second,the Court confirmed that Twomblysets a transsubstantive pleadingstandard that requires the pleader tostate-using nonconclusory factualallegations-a claim that is "plau-sible."2 3 Third, the Court held thatjudges should assess plausibility bydrawing upon "judicial experienceand common sense."' The purpose ofthis new pleading standard is, in theCourt's words, to screen out factuallyweak or meritless claims before theyimpose costs on defendants and thejudicial system."

To determine whether theintended effect was occurring, com-mentators conducted a number ofempirical or quasi-empirical studiesof Twombly and lqbal.u Althougheach has its flaws, and although theyspan a variety of methodologicalapproaches, they are relatively con-sistent in finding a modest increasein the grant rate of dismissal motionsfiled.

For example, Professor PatriciaHatamyar conducted two post-Iqbalstudies based on Westlaw opinions.27

She coded by case and whole-motionresult (i.e., grant, deny, or mixed).2

In her first study, Hatanyar found astatistically significant increase ingrant rates, from 46% (pre-Twombly)to 56% (post-lqbal), and a statisti-cally significant decrease in denialrates, from 26% to 18%. The bottomline was that the odds of a motion todismiss being granted or granted inpart were 1.5 times greater underTwombly and Iqbal than under thepre-Twombly regime, holding allother variables constant. The stron-gest predictor of this variation waspro se status.2 In her second study,which amplified the post-lqbaldata,"she found that the rate of motionsgranted without leave to amendincreased by a factor of 1.67 underlqbal, and that the rate of grants withleave to amend increased by a factorof 5.9 times (both significant at the95% confidence interval)." Exclud-ing pro se constitutional civil-rightscases still showed an increase in dis-missal rates, but only to the 93% con-fidence interval32

Recently, the Federal ludicial

t J _-

Center published a study using civildockets rather than Westlaw data-bases. Like Hatamyar's studies, theIC coded by case and whole-motionresult (grant, grant in part, deny).The FJC excluded all prisoner andpro se cases.? The FJC included casesfiled under pleading strandardsother than Rule 8(a)(2), namely,financial-instrument cases. Exclud-ing the financial-instrument casesresults in an overall t ' +grant-or-grant-in-part rate increaseof 5% post-lqbal N TW(from 66% to 71%),which is significant COUto the 95% con-fidence interval,though this figureis not regressed.Breaking down thedata by type of caserevealed increases in the dismissalrate across all types, and a substan-tial increase in civil-rights cases, 4

although the smallish Ns did not gen-erate statistical significance."

Methodologically, all these studieshave two common features. First,the studies' unit of analysis is awhole case. Yet a large percentage ofcases have multiple claims. Coding amulti-claim case's "type" necessarilyinvolves either reliance on classifi-cation by others (i.e., the plaintiff orthe clerk) or difficult judgment callsabout the relative importance of oneclaim over others, and prior studieshave conceded the messiness of suchclassifications.' Further, becausedecisions on motions to dismissoperate at the claim level rather thanthe case level, coding results by casenecessarily requires the ambigu-ous category of "mixed" decisions,in which at least one but not all theclaims were dismissed. Coding byclaim could provide a clearer pictureof how the new pleading standard isoperating.

The second common methodologi-cal feature is that existing studiesfail to distinguish between decisionsbased on factual sufficiency anddecisions based on legal sufficiency.Neither Twombly nor lqbal disturbs

the legal-sufficiency standard that limit the searches only to cases citingRule 12(b)(6) has always tested. the permissive Conley or the restric-Rather, TWombly and tqbal impose tive lWombly, as other studies did.a new factual-sufficiency standard. Instead, I used those terms only inInclusion of a relatively constant the disjunctive, to capture cases thatsubset of legal-insufficiency chal- mayhaveusedthem asa proxy forthelenges could overwhelm and mask motion-to-dismiss standard insteadthe significance of any increase in of referencing Rule 12 or "failure todismissals for factual insufficiency. state a claim." In addition, I includedThus, distinguishing between the the term 'Conley' in the post-iqbc

search. I therebyhoped to reduce the

4m o + -tin* risk ofanysample-skewing effect frompotentially loaded

DMBLYAND IBAL THE SUPREME search termsb

Finally, I used aRT USHERED IN A SEA CHANGE full twelve-month

period in eachIN PLtAlIING 1US NDARDS.

A A " A< 1, w,**~

bases for the outcomes is a crucial excluded allomission." Coding decisions by type circuit opinioof sufficiency challenged (factual or courts. Distrilegal) could better isolate the effect front line andof the new pleading standard. ence with mot

This study undertakes to fill ing appellatesome of these details left out of prior skewed the resstudies. only grants of

appealable. FiMathodology trict courts folI studied the impact of TWombly and as Supreme CIqbal on dismissal rates in federal was relativelydistrict courts using an original da-d, and so tdataset with an eye toward exploring among the crthe detail left out of previous studies. To control forI began with broadly permissive In circuit law, Icircuit-specific Westlaw searches Using a rain each of the "FedX-all" databases tor, I selected I(where aX" is each circuit)." I did not search list, TI

100 pre-o

33. Joe S. Cecil et aL. Motions to Dismiss fbrFaiure to State a Claim After (qbal 5-6 (Fed. Jud.Ctr 2011).

34 I. at 13.35. Id. at 14, 19 21. The p values in some

categories were very close to conventional sta-tistical significance levels, kading one com-mentator to challenge the F)C's conclusions thatthe increases found were not meaningful SeeLonny Hoffman. Twombly and lqbar's MeasureAn Assessment of the Federal Judicial CentersStudy of Mations to Dismiss, 6 FED. CTS. L. REV. 1,24-26 (2012).

36. Cecil supra note 33. at 3 (noting idosyn-cratic coding practices by clerks); Hiatamyar,supra note 27 (noting the difficulties of codingfor 'type" of case); cf. Christina .. Boyd et al.,Building a Taxonomy of Litigation. Clusters of

search to avoid anyseasonal biases."

Each searchgenerated a list ofcases, and I first

Supreme Court andns to isolate districtct courts are on thehave the most experi-ons to dismiss. Includ-decisions may haveults because generallydismissal motions areurther, although dis-low circuit law as wellourt decisions, Iqbalclear on the legal stan-here is little differencecuits on New Pleading.any latent differencescoded for circuit.

idom-number genera-00 opinions from eachhus, I ended up withbly and 100 post-Iqbal

Causes of Action in Federal Complaints (draft)(questioning the efficacy of characterizing casesby traditional grouping suppositions).

37 Kevin M. Clermont, Three Myths AboutTwombly-lqbaL 45 WAKE FOREST L, REv. 1337,1367 n.140 (2010).

38. Pre45vWambly dismiss! /p (rule-12 12(b)(6) conley (fail! /3 state /3 claim)) & da(aft21/5/2006) & da(bef 21/5/2007). Post-Ilqbal:dismiss! /p (rule-12 12(b)(6) conley twonblyiqbal (fail! /3 state /3 claim)) & da(aft18/5/2009).

39. Clermont & Yeazell, supra note 10, at 839n.66.

40. See Moore, supra note 27, at 635 (finding adental rate of 30% in the first six monthsof 2006and a 37% denial rate in the last six months of2006).

Category DesignationsCircuit 1, 2, 3,4, 5, 6, 7, 8, 9, 10, 11, DC, fed

Published Opinion Yes, i

Po Se Claimant Yes, ic

Outcome ) smissed, Wt Diomissedi AR vi 41

district-court opinions from eachcircuit.

I then read each case and excludedopinions that did not resolve aRule 12(b)(6) motion to dismissunder the Rule 8(a)(2) standard. Thiswas an important step because manycases that cited to Conley or Twomblyor lqbal or that used 12(b)(6) ter-minology actually did not resolve amotion to dismiss for failure to statea claim. The intentional breadth ofmy search terms assured a relativelyhigh number of false hits, making thereading stage important for catchingand excluding them.

In this reading stage, I discardedopinions resolving only motions forsummary judgment (which pertainsto a review of the evidence ratherthan of the pleadings), Rule 12(c)motions, jurisdictional dismissalsunder Rule 12(b){ 1) or (b)(2), venuedismissals, and motions to dismissonly on heightened pleading groundssuch as Rule 9(b) or the PSLRA. Thereason I disregarded each of these isbecause they potentiallywere subjectto a different dismissal or pleadingstandard. The standards of Twomblyand Jqbal may apply beyond merits

41. These acronyms stand for decisions basedon the In Forma Pauperis statute, 28 US.C.§ $915A. or the Prisoner Litigatfon Retforr Act,Pub. L. No. 104-134, both of which permit sasponte dismissal of claims for failure to state aclaim upon which relief can be granted.

42. See, eg. Stephen J. Choi & G. Mitu Gulati,Bias in Judicial Dedslans: A Window lat theBehavior of Judges, 37 ). LEGAL STuD. 87, 94(2008).

pleading under Rule 8(a)(2), but thecourts have not resolved so defini-tively (and, in fact, many courts haverefused to so apply them). Accord-ingly, to maintain a pure sample, Iexcluded all non-Rule 8(a)(2) claims.

I then turned to coding. I did notcode based on whole case, as otherstudies have done: rather, I codedeach Rule 8(a)(2) claim decided ona Rule 12(b)(6) motion to dismiss.Thus, I was not confined to difficultcharacterizations of grouped claims,such as classifying the type of awhole case. Nor did I have the ambig-uous category of "mixed" decisions.Table 1 describes the coding schemeused for each claim.

A few notes about my coding. Iused the political-party affiliation ofthe judge's appointing president tocode the judge's political atTiliation.There are some problems with thisproxy, but the proxy seems to be anacceptable one in most studies usingsimilar measurements.'

I designated a claim as a publishedopinion if it was published or slatedto be published in an official reporter,as indicated by Westlaw. All otherswere coded as unpublished.

I coded as "no" in the Pro Se Claim-ant category claimants who litigatedon their own behalf but who identi-fied themselves as attorneys or whowere corporations using in-housecounsel.

I coded for claim based on severalgroupings. "Civil Rights" included§ 1983 and Bivens actions against

public officials and entities, mostprisoner-government litigation, andhabeas corpus petitions. "Employ-ment Discrimination" included TitleVII, ADA, ADEA, and other like claimsagainst private defendants. "Tort"included intentional torts, medicalmalpractice, negligence, FTCA, andother common-law and statutorytorts. "Contract" included breachof contract, breach of implied war-ranties, indemnification, and othersimilar kinds ofcontract or quasi-con-tract claims. "IP" included statutoryand common-law claims commonlyassociated with intellectual propertydisputes. "Other" included all otherclaims, including antitrust, ERISA,RICO, and environmental statutes.

I included as "dismissed" claimsthat were found insufficientlypleaded but that were technicallynot dismissed in the opinion itself.These circumstances came up in pri-marily two types of circumstances:when a district court found a claimdismissable but nevertheless per-mitted the plaintiff an opportunityto replead, and when a magistratejudge recommended dismissal butthe district court's adoption of therecommendation was not attachedto the magistrate's opinion. Althoughthese opinions did not technicallyresult in an immediate dismissal,they functionally represent a judicialfinding that a pleading is insufficientunder Rule 12(b)(6) and thereforeshould be coded with dismissals. Forthe same reason, I coded magistraterecommendations that a motionbe denied as "not dismissed," eventhough it is possible that the districtjudge subsequently disagreed.

I coded rationale as "Fact" if thecourt determined that the complaintwas factually sufficient or insuffi-cient, "Law" if the court determinedthat the complaint was legally suffi-cient or insufficient, and "Both" if thecourt decided on both bases. The dis-tinction between fact-based and law-based decisions was generally clearfrom the opinions.

My methodology has two potentialweaknesses. The first is that it relieson Westlaw databases of judicialopinions, which overrepresent pub-

lished decisions and thus may reflectselection bias in the judge's decisionto publish a case." I will have moreto say about this potential weaknessin the final section of the paper.

The second weakness is that Idid not code for the presence of anamended complaint, partly becauseof the difficulty of following a partic-ular claim through any amendmentprocess. The FJC recently concludedthat the presence of an amendedcomplaint is correlative with grantrates. 44 I leave it to others to deter-mine how much this potential weak-ness affects my findings.

ResultsMy results show an overall increasein the dismissal rate (as a functionof motions) of all claims from 73.3%pre-lWombly to 77.2% post-lqbal.This 4% increase was significant tothe 99% confidence interval. Theoverall dismissal rate in each cat-egory increased after Jqbal, in mostcases significantly. Table 2 sets outthe results for all claims using a two-tailed test, with significance mea-sured at 95% by *, and 99% by **.

The statistical significance of the "Allclaims" differential in Table 2 holds upusing a multivariate probit regressionanalysis controlling for all other vari-ables (dummies used for Other Claim,Democratic Judge, Eleventh Circuit,Represented, Not PLR.A, and Unpub-lished; Z=2.96, statistically significant tothe 99% confidence interval). This addsto the robustness of the two-tailed testsabove, suggesting that the significanceof the differences in overall dismissalrates of all claims pre-Y1,vombty and post-lqbal are not due to changes in the dis-tributions of types of cases, litigants, orjudges.

In addition to coding by claim insteadof by case, a primary innovation of mystudy is its coding of the rationale ofthe opinion as based on factual or legalsufficiency. The data reveal that therationale for dismissals is more heavilyweighted toward factual insufficiencyafter Iqbal. The data show that thefactual-insufficiency dismissal rate,as a percentage of dismissals, hasincreased in all categories of cases,and significantly so in most. Table 3

All claims

Unpubsle

fton-i FP/P

Represente

Pro-Tw

1924

d pin ins 1585

RA 1352

d 935

Ail claims

Unpublished opinions

4,

Non-IFPIPLRA

Represented

omhlyrant Rate

733%

75.0%

72.0%

65.2%

Pre lmenitN Rate

1411 34.5%

1188 35.4%

973 28.2%

610 25.4%

sets out the data, using a two-tailedtest, with significance measured at95% by *, and 99% by **.

Predictably, the rate of dismissal forfactual insufficiency as a function of allmotions has also increased after Iqbal.In other words, for any given claimsubject to a motion to dismiss, the like-lihood that the claim will be dismissedfor factual insufficiency is higher afterJqbal. This is true, and statistically sig-nificant (using a two-tailed test, wheresignificance is measured at 95% by* and 99% by **) for all categories ofclaims. Table 4 sets out those data.

As was the case in Table 2, the sta-tistical significance of the "All claims"differential in Table 4 holds up using amultivariate probit regression analysiscontrolling for all other variables, withZ=5.36, statistically significant to the99% confidence interval.

Perhaps surprisingly, the rate of dis-missal for legal insufficiency as a func-tion of motions has decreased after Iqbal

Post- khaN Grant Rate

1979 77 2%

1602 77.8%

1426 73.5%

961 67.0%

Post-lgbalN Rate

1528 479%

1247 49.6%

1048 46.8%

644 44.9%

z Vale2J17"

1.88

0.86

1.74

ZYalue733**

3.57"

4, 154.

for most categories. Table 5 provides dataon these changes using a two-tailed test,where significance at 99% is denoted **.

As was the case in Tables 2 and4, the statistical significance of the"All claims" differential in Table 5holds up using a multivariate probitregression analysis controlling forall other variables, with Z=-6.25, sta-tistically significant to the 99% con-fidence interval.

Note that adding the percent-ages in Tables 4 and 5 producestotals that exceed the percentages in

43. Stephen B. Burbank. Vanishing D ials andStmmaryJudgment In Federal Civil Cases. DriftingToward Bethlehem or Gomorrah?, 1 J. Eminpu-CAL LEGAL STo. 591. 604 (2004); Kevin M.Clermont & Theodore Eisenberg, CAFA Judicata:A T(de of Waste and Politics, 156 U. PA. L. Ray.1553, 1558-60 (2008); see also Brian N. Lizotte,Publish or Perish. The Electronic Availability ofSummary Judgments by Eight District Coures,2007 Wis. L, Ray. 107, 130 (showing a selec-tion bias In publcation of summary-judgmentopinions).

44. Cecil, supra note 33. at 15, 29,

ctual n uffic ency Dism (0 (as a function old smissals)

AW Ove ' missl Rate- (as a functiont of motions )

34 i 3 i Pf, y

Pre-wemtyII Rate

All caims 1924 25.3%

Urpubhished opinions 1585 26.5%

Non-IFP/PLRA 1352 20.3%

Represented 935 186%

All claims

Unpublis

Non-1FP/

Represen

Pre- 1 mblyN Rate

1924 50,2%

hed oimons 1585 50.3%

PLRA 1352 52.3%

ted 935 56.1%

Table 2; that is because a few claimswere resolved on both law and factgrounds, leading to their inclusion inboth Table 4 and Table 5.

Analysis and implicationsIn this section, I highlight and discusssome of the more relevant results ofthe study

Pre-Twombly Legal-insufficiencyDismissals Were RoutineOne key result is that, even beforeTIombly, motions to dismiss weresuccessful more than 73% of thetime, and no category revealed agrant rate lower than 65%. Theseresults are consistent with otherstudies of pre-71vembly grant rates.'

45. Civil Case Processing in the Federal DistrictCourts: A 21st Century Analysis 47-48 (IAALS2009) (finding more than 7096 of all Rule 12motions to be granted at least In part).

46.CeciL supra note 33, at 10 n.21.47, See supra note 12.

Post-IqtelN Rate

1979 37.0%

1602 3836%

1426 34-A%

961 30.1%

P ast-e081II Rate

1979

1802

1426

961

432%

42.4%

421%

39.4%

Z value

7.27"

,S1?64882?A

Z Value-4.33"~

-440"

-6.35"

-7.27"~

These percentages are a func-tion of motions filed, so the datado not reflect what the overall dis-missal rate is of claims filed. Previ-ous studies, however, have tended toshow a motion-filing rate of around15% pre-Twombly." Based on thesefigures, one could roughly esti-mate that approximately 10% of allclaims filed were dismissed underRule 12(b)(6) under the pre-MIvomblyregime.

This dismissal rate underminesthe contention that the pre-Twomblystandard for testing pleadings wasa toothless one. Contrary to thatpopular misconception, any numberof relatively common legal deficien-cies could justify dismissal. Perhapsthe claim is preempted by federallaw. Perhaps the complaint miscon-strues the scope of the law govern-ing the claim. Perhaps the plaintiffalleges specific facts that, as a matterof law, preclude the claim.

FactBasd Dsmissal Rate (as a function of motions)

Law d Dism al Rate (as a function of mot ons)

These are not necessarily rare orunusual defects, nor are they neces-sarily the result of poor lawyering.The scope of the law is often unclearbut important, and therefore highlycontested. Whether the Constitu-tion protects flag burning, whatlegal standard applies to gender dis-crimination under the Equal Protec-tion Clause, whether Section 1983requires exhaustion, whether TitleVII applies extra-territorially-these are important legal ques-tions that are presented best by awell-crafted complaint and decidedby a well-prepared judge, oftenwithout needing any further factualenhancement. A legal-sufficiencychallenge, even before 7wombly,tees these questions up for judicialdecision. For this reason, it is unsur-prising that the data indicate thatthe legal-sufficiency requirementof Rule 12(b)(6) erects a useful andworkable mechanism for resolv-ing-often by way of dismissal-questions of law.

Pre-Twombly Factual-InsufficiencyDismissals Were RoutineThe data show that courts decidingmotions to dismiss under the pre-Twombly standard often dismissedfor factual insufficiency (more than1/3 of all pre-Twombly dismissals,and more than 1/4 of all pre-Tiomblyclaims facing a motion) even thoughsuch dismissals generally were notpermitted by existing law. From the1980s into the early 2000s, severalpapers documented the practice bysome lower courts of requiring amore stringent factual-sufficiencystandard than then-prevailingSupreme Court precedent allowed."'The results of my study lend empiri-cal support to those observations andsuggest that the practice persisted inthe lower courts even after repeatedand emphatic denunciations by theSupreme Court.

Twombly and Iqbal Have ChangedPleadingSome have argued that Twomblyand lqbal did not change pleading,either doctrinally or in practice,or that they did not change it very

much." The data undermine this enables those courts to dismiss fac- suppose that these cases would showargument, though how much is cer- tual-insufficiency claims that they the inverse relationship becausetainly debatable. had been uncomfortably squeezing courts often screen IFP/PLRA com-

Table 2 shows essentially single- into a legal-insufficiency rationale)0 plaints without adversarial Inputdigit increases in the dismissal rate In other words, New Pleading may be from the defendant" and pro se(as a function of motions) overall and legitimizing a practice by some dis- cases with leniency in pleading suf-across all categories. The increases trict judges to dismiss claims that, ficieucy.3 These factors might leadoverall and in a number of categories while not technically legally insuf- one to surmise that dismissal ratesare statistically significant. These ficient, struck the judge as so doubt- in these cases ought to he lowerresults add con- than other cases.firmnatary support iBut my data revealto other studies, instead that thesewhich have found cases exhibit rela-single-digit but Dism iSSAL RATES OF ALL CLAIMS tively high dismissalsignificant or rates even beforenearw-hsignifilcant not tcnica l l iincreases in the cent sruc th ea doubtoverall dismissal TWOMBLYAND AGAIN AFTER. IQBAL. facing a motionrate of cases after were dismissed) andIqn These results strongly significantalso suggest that -,k - 707 -0 * .-4 AA increases after lqbal.studies finding These results couldnon-significant be explained by theincreases, such as the FICs study, ful or unlikely that the judge thought meaningful assistance thatattorneysjust didn't have enough data points justice might be served by dismiss- provide in drafting complaints, orto generate significance. The data ing them anyway. perhaps by the absence of meaning-appearto support the conclusion that Whatever the justification, the ful attorney-selection mechanisms

wombly and lqbal are affecting the iresults have some implications for at the filing stages.' 4 Regardless, thedismissal rate of claims and cases, further study. Ififor example, dismiss data strongly suggest that any studyat least as a function of motions, in als based on legal insufficiency are of pleading effects that omits suchfederal court. generally without leave to amend but cases is incomplete.

Pleading has changed within dis- dismissals for factual insufficiencymissals as well. Table 3 indicates are generally with leave to amend, Westlaw Databasesthat factual insufficiency now rivals then it is possble that wiqP, to May Be Probativelegal insuficiency as the dominant the extent it shifts the rationale for Westlaw-based datasets have beenjustification for dismissal. I can offer dismissals in favor of factual insuffi- criticized as unrepresentative ofthree explanations, all of which may ciency, may ultimately give plaintiffs dockets generally. Westlaw data-in fact be at play here. One is that more opportunities to amend their bases include all published opinions,the motion-filing rate has increased, complaints, plus some-but not all-unpublishedand that that increase is primarily More study is necessary to bear opinions. It has been hypothesizedor even exclusively attributable to these theories out. The more general that the unpublished set of opinionsmotions relying on factual insuf- story here is that factual-insuff-ficiency, such that factual-insuffi- ciency dismissals are up and dis- 481, See, e. Steinmn. supro note 10.ciency dismissals now appear as a missals overall are up. The overall 49. See Scott Dodson, Pleading Standards After

aell Atlantic Corp. v. Twonbly, 93 VA. l. Rt, INhigher percentage even the abso- increase Is modest-only a single I 15 142 (2007) (predicting this effect).

lute numbers of legal-insufficiency digit-but statistically significant. 50. See Mart Moller, Procedu~re's Ambiguty, 86

dismissals has remained constant. These data support the anecdotal INs. Les 645667 (2011).5a See Reinert, slpra note 26, at of2 (survey.

Another likely explanation is that commentary that Twombly and Iqbal trig commentary); see also Has the Supreme Court

movants are strategically switch- have changed pleading standards Limited Americans' Access to Courts? Hearingmefsre the l Comm. n theludiciaryT 111th icng

Ing legal theories from a perceived and are having an effect in the lower 9l0s (prepared statement of Stephen B.

weak legal-insufficiency argument courts." Burbank) (appending a list of post-Tworbly disto esal opinions in which the udge remarkediniet the complaint would have survived before

insufficiency argument.49 A third Prisoner, Pauperis, and Pro Se Twombl).possible explanation is that some Litigants Are Affected Most 52. 28 u.s.c. § 191A(b)l), 1997e(c)(1)

53e Erickson v. Pvdus 551 U.S. 89,93 (2007).pre-Womly courts were stretching The disaggregated data suggest 54. Cf Theodore sfsenber & Stewart j.the legal-insufficiency standard to that prisoner, pauperis, and pro se Schwab, The Realfty of Constitutional Tort Ltf-

a lgtion, 72 CORNELL L. Rr n 641, 681 (987aher(finding a correlation between representation

tionably so, and that Twiqbal now dismissal-rate increases. One might and success).

0 i E 'I) iE, ; E 17,; Y 3,

has a greater percentage of denialsof motions because such decisionsare generally not appealable andthus have less need to be reduced toa published decision.-" If so, then adataset that includes only a subset ofunpublished decisions would excludea disproportionate number of denialsand thus tend to overstate the grantrate of motions.A

That theory could be true, but it isnot supported by my data. BecauseWestlaw captures some unpublishedopinions, I was able to compare dis-missals in unpublished and publishedsubsets. Table 2 suggests that unpub-lished opinions reflect higher grantrates than published opinions, bothbefore Twombly and after lqbal, andthe probit analysis suggests that anytemporal variation in the publica-tion status did not affect the overalldismissal rates. My results thereforelend some support to other studiesthat rely on Westlaw databases, and

55. Cecil, supr note 33, at 2.56. It is worth noting that many commenta-

tars have defended the utility of published-onlydatasets despite their potential for underrep-resentation. See, e.g., Clermont & Eisenberg,supra note 42, at 1560; Theodore Eisenberg &Geoffrey Miller, The Role of Opt-Outs and Objec-tors in Class Action Litigation: Theoretical andEmpirical issues, 57 VAxn. L. REV. 1529, 1542n.59 (2004): Theodore Eisenberg & Sheri LynnJohnson, The Ffects of intent. Do We Know HowLegal Standards Work? 76 CORNELL 1.. Rev. 1151,1195 (1991).

57 One researcher found nearly identical dis-missal-rate Increases among Westlaw-only andPACER datasets when filtered for representedparties. See Patricia Hatamyar, Thoughts on theFederal judiciary Committee's Study of 12(h)(6)MotionsAfteriqbal, Civit PRocEouRE & FEDERALCOURTS BLOG (July 11.2011). available at http://lawprofessors.typepad.com/civpro/2011/07/thoughts-on-the-federal-judiciary-committees-study-of-12b6-motions-after-iqbalhtm).

58. A related selection issue is Westlawcapture of unpublished decisions. Westlaw cap.tures some, but not all, unpublished opinionsthat are Issued, and the rate of capture varieswidely by district. Cecil, supra note 33, at37 n.47,It has been reported, however, that Westlawincludes evcry opinion it can find unless askedspecifically not to. Id. at 27. it does not seemto me that this selection criterion Is biasedagainst capturing unpublished denials. Accord-ingly, absent some evidence of a selection biasin Westlaw capture of unpublished opinions,my data strongly suggest that 7tetqbal affectsdismissal rates across the publication divide,

59. Cermont, supro note 37, at 1365.60. On defendant-selection effects, see Jonah

H. Gelbach, Note, Locking the Doors to Discovery?Assessig the Effects of Twombly and lqbal onAccess to Discovery, 121 YALE L.l. 2270(2012). Onplaintiff-selection effects, see Kevin M. Clermont& Stewart ). Schwab, Employment Discrimination

they undermine the criticism thatWestlaw-database studies overstateany 7Wiqbal effect.

These results might have ratio-nal explanations. Let's assume thatjudges ordinarily do not publishdenials of motions to dismiss becausethey often neither are appealable norpresent novel questions of law. Underthe Conley regime, then, one mightexpect only a small percentage ofdenials to be published. In the wakeof Ewombly and lqbal, however, a fargreater percentage of denials wouldbe published because Wombly andlqbol were new, important, and poten-tially unclear cases, and thus motionsto dismiss were more likely to presentnovel questions of law. Relying onlyon published cases, then, potentiallyoverestimates the grant rate underConley and reduces the differential indismissal rates. Alternatively, if dis-missals of prisoner, pauperis, and prose cases are disproportionately more

Plaintiffs in Federal Court From Bad to Worse?, 3FiARV, L. & Pogv REv. 103 (2009); Scott Dodson,New Pleading, New Discovery, 109 Mica. L. REV.53, 72-86 (2010): Scott Dodson, Federal Pleadingand State Presuit Discovery. 14 LEwiS & CLARKL. REv. 43 (2010). For seminal studies of party-selection effects, see Robert G. Bone, ModelingFrivolous Sutits, 145 U. PA. L. REV. 519, 522-29(1997); George L. Priest & Benjamin Klein, TheSelection of Disputes for Litigation. 13 1. LEGALSTO. 1 (1984).

61. See Joe S. Cecil et al.. Update on Resolutionof Rule 12(bj{6) Motions Granted with Leave toAmend (Fed. judicial Ctr. 2011).

62. For an attempt to study this feature, seeAlexander A. Reinert, The Costs of HeightenedPleading, 86 INo. L.). 119 (2011); ct. Stephen I.Choi et al.. The Screening Effect of the PrivateSecurities Litigation Reform Act, 6 ). EMPIRICALL. STun. 35 (2009) (studying this feature In thecontext of the PSLKA).

63. Bone. supra note 10, at 879 & n.141(predicting an increase In motions and plead-ing practice, and arguing that these 'processcosts' should be part of any assessment); Idat 878 (articulating the social cost of errone-ous dismissals); Edward Brunet, The Primacyof Private Attorney General Cforcement in theUnited States. IND. J. ALL. DisP. RasoL. (forth-coming Z012) (positing regulatory-deterrrencecosts of Twflbal); Cecil. supra note 33, at 8-10(finding a statistically significant increase inRule 12(b)(6) motions filed after lal); Cler-mont & Yeazell, supra note 10 (describing costsof doctrinal destabilization); Emery G. Lee IlII &Thomas E. WIllging. Attorney Satisfaction withthe Federal Rules of Civil Procedure 12 (Fed.judicial Ctr. 2010) (publishing survey resultsrevealing plaintiff attorneys to be adding morefacts to complaints as a result of Twombly andlqbal); Miller, supra note 10, at 67-68 (predict-Ing that plaintiffs will have to spend more timeand effort investigating clalims and filing moredetailed complaints).

likely to generate unpublished opin-ions, yet they also are disproportion-ately more likely to show increasedrates of dismissal after Iqbal, thenstudying only published opinionsis likely to understate the effects ofTwiqbal on dismissal rates." In sum,even if Westlaw databases are notrepresentative of all decisions, it isnot at all clear that they overrepre-sent dismissals."

ConclusionAs others have pointed out, the mostaccurate empirical tests on the effectof TWqbal would be very difficult todevelop." Defendant-selection andplaintiff-selection effects may be farmore meaningful than the dismissaleffect."e In addition, the dismissaleffect may be mitigated by oppor-tunities for amendment."' Also, thedismissal effect studied here revealsnothing about the merit of the casesaffected.62 Finally, the dismissaleffect is but one factor in a host ofnormatively relevant 7iviqbal effectsyet to be fully studied."

This paper speaks directly to noneof these questions. Instead, it strivesto add knowledge and understandingto how TWombly and lqbal are affect-ing dismissal rates. In particular, itadds granularity at the claim leveland in the rationale for decisions onmotions. The results are not insig-nificant. The claim-based results addto the robustness of the case-basedfindings of other studies. And thedisaggregation of legal-sufficiencyand factual-sufficiency rationalesopens a new window into the effectsof Twombly and Iqbal. *

I SCOTT DODSONis Professor of Law, University ofCalifornia Hastings College of the Law.(dodsons@uchastings.edu)

Nfl, r4l"N