Are All Patent Examiners Equal?: The Impact of Examiner

Characteristics on Patent Statistics & Litigation Outcomes

Iain Cockburn, Boston University & NBER

Sam Kortum, U Minnesota & NBER

Scott Stern, Northwestern, Brookings & NBER

Why look at examiners?

PTO examination process is fundamental to the IP system, but its relationship to policy proposals and economic assessment are poorly understood

Research on the “PTO production function” can help to– Evaluate policy initiatives aimed at PTO operations– Inform academic research on economic impact of IP

Differences among examiners may be an important driver of patent quality and breadth

Rules versus Discretion

The patent examination process is highly structured, but some discretion is necessary

Despite careful monitoring and detailed rules (MPEP) examiners use prior knowledge and individual judgment to search prior art, determine scope of allowed claims, etc.

– Exercise of discretion may have consequences for the scope of intellectual property rights

Impact of discretion may depend on examiners’ experience, specialization, and training

Methodology

Build qualitative understanding of examination process

Identify examiners responsible for patents whose validity is reviewed by CAFC (1997-2000)

Evaluate relationship between CAFC-tested patent statistics and examiner characteristics

Test whether examiner characteristics are associated with CAFC validity findings

Key Qualitative Findings

“There are as many patent offices as there are patent examiners” Extensive “apprenticeship” system w/ mentoring of junior examiners Prior art search technology and method varies across…

– art areas – time

Examiner job design varies in the degree of…– technological specialization– collaborative interaction

“All applications are created equal”

… potential for idiosyncratic exercise of discretion

Examiner heterogeneity: Qualitative Insights

May result in persistent variation in– breadth of claims allowed (“generosity”)– approval times– degree of “self-citation”

But…link to patent statistics or litigation outcomes claimed to be “noisy”

– “Finding a needle in a haystack”

Data

182 patents where validity tested by CAFC (1997-2000)– 50% found invalid, more likely for

computer/communication, less likely for drugs/medical

196 examiners on these patents, responsible for ~290,000 patents 1976-2000

Distribution of Examiner Experience

Distribution of Technological Specialization

Distribution of Examiner Cites / Patent

ANOVA tests

Examiner “id” accounts for 8-10% of variation in

– Approval times– Citations Made– Citations Received

Strongly statistically significant even after controlling for technology class and application year

Do Observable Examiner Characteristics Impact Patent Statistics?

Citations on CAFC-tested patents are strongly associated with Examiner Cites per Patent

Even after inclusion of detailed technology and cohort fixed effects

Approval time on CAFC-tested patents uncorrelated with examiner characteristics

Nuanced relationship with generality and originality of “forward” citation

Do Examiner CharacteristicsImpact Litigation Outcomes?

How do CAFC Validity Patents Compare to “Average” Patents?

  CAFC SAMPLE

TYPICAL PATENT

 

Claims 20.5 9-14

Citations Received

14.0 6-8

Citations Made

16.7 6-8

Approval Time 2.21 1.76-2.05

CAFC Patents have more claims, cite more prior art,are more highly cited, and are associated with slightly longer approval times.

Do Examiner Characteristics Impact CAFC Validity Decisions?

Probability of patent being found invalid

– NOT related to examiner’s experience, recent workload, propensity to self-cite, approval time

– Significantly higher when examiner has higher “lifetime” average level of citations received per patent

(controlling for technology class and application year)

Do Examiner Characteristics Impact CAFC Validity Decisions?

INVALID VALID

EXAMINER

CITES / PAT 6.89 5.72

Are CAFC rulings sensitive to “over-generous” examiners?

YES!– Econometric procedure links probability of invalidity to that part of

“forward” citations received which is attributable to the examiner’s propensity to allow patents which attract many cites

So, if….– citations received is an indicator of breadth of the patent AND– exercise of discretion by examiners results in variation in the

average # of citations that “their” patents receive

One increase in Examiner Cites / Patent associated with a 25% decrease in the probability of validity

Origins of “Generosity”

Differences in information (access to prior art) The Apprenticeship system Consistency across patents of a given examiner, not

necessarily consistency across examiners– Level of review not dependent on social or economic value of

invention

Effort: easiest thing to do is “allow” patents Incentives

Consequences of “Generosity”

Process by which application are allocated to examiners is particularly important

Specialization induces variation in the “size” of the rights awarded

– particularly of concern for immature technologies

Potential burdens on – the judicial system – private intellectual property contracting

Implications

Training, mentoring and monitoring are at the core of preserving “not too much, not too little”

Should level of review depend on the potential economic and social value of application?

CAFC rulings are responsive to exercise of examiner discretion

Conclusions

Examiners are heterogenous

Preliminary evidence does not support naïve hypotheses about negative impact of inexperience

Courts tend to invalidate patents associated with “generous” examiners

– Important role for judicial review in “trimming” discretion