The Contribution of The Contribution of Research Methods to Research Methods to Management TheoryManagement Theory

Brian Boyd

Arizona State University

www.briankboyd.com

RoadmapRoadmap

Where we stand: Paradigms and consensus

The role of methods: A bibliometric analysis

What we can do: Opportunities for using methods to enhance the impact of management studies

Where We Stand: Where We Stand: Paradigms and ConsensusParadigms and Consensus

Kuhn’s Paradigm Kuhn’s Paradigm ModelModel

A paradigm is a shared view of a given domain. “Shared view” is represented by agreement (or

consensus) on both theory and methods Some disciplines are more advanced than

others Evolutionary processes are based on the

accretion/disconfirmation of theories and empirical evidence

Stage of a field’s development affects how research is done

Stages of DevelopmentStages of Development

Consensus IndicatorsConsensus Indicators

Scholarly productivity Dissertation length Journal acceptance rates Time to process reviews Number of reviewers per article Extent of manuscript revisionOverall, management and social Overall, management and social

sciences fall below the hard sciences sciences fall below the hard sciences on each of these metrics.on each of these metrics.

YoungYoung MatureMature

Weak consensus Poor agreement on

quality

Strong consensus on methods, problems, and solutions

The The Evolutionary Evolutionary

ChallengeChallenge

An ExampleAn ExampleQuestion 1:

(a) The world is generally knowable

(b) The world is socially constructed

Question 2:

a) Only one type of methodology (epistemology) is needed to study phenomena

b) A range of methodologies are needed to study phenomena

Podsakoff, Shen and Podsakoff (2006, RMISM):

Vast majority of strategic management constructed mis-defined as reflective indicators, when they are in fact formative indicators

Edwards (2011, ORM):

“The presumed viability of formative measurement is a fallacy (p.370).”

How do less-developed How do less-developed paradigms create paradigms create

knowledge?knowledge?

Mimetic imitation of more advanced fields

Richard Feynman (1974): “cargo cult” science

Total Articles PublishedTotal Articles Published

0

5

10

15

20

25

30

35

40

45

None 1 to 2 3 to 4 5 to 10 11 to 20 21 to 50 50 plus

ManagementOther BusinessAll facultyPercent

Articles

χχ2 2 (p=.01) tests confirm that (p=.01) tests confirm that Management distributions Management distributions are different from both are different from both other categories for all other categories for all comparisons.comparisons.

HERI data on 34K full-time university profs; 2.3K from b-schools

Total Articles Published Last 2 Total Articles Published Last 2 YearsYears

0

10

20

30

40

50

60

None 1 to 2 3 to 4 5 to 10 11 to 20 21 to 50 50 plus

ManagementOther BusinessAll facultyPercent

Articles HERI data on 34K full-time university profs; 2.3K from b-schools

Hours Per Week Hours Per Week Spent on ResearchSpent on Research

0

5

10

15

20

25

30

35

40

None 1 to 4 5 to 8 9 to 12 13 to 16 17 to 20 21 plus

ManagementOther BusinessAll facultyPercent

Hours HERI data on 34K full-time university profs; 2.3K from b-schools

Productivity Benchmarks:Productivity Benchmarks:MacMillan PublicationsMacMillan Publications

0

2

4

6

8

10

12

14

0 1 2 3 4 5 6 7 8 9 10

75th ile

90th ile

95th ile

99th ile

Years post-Ph.D.

Cu

mul

ativ

eP

ubl

icat

ions

Data: 945 US strategy facultySource: Boyd, Finkelstein & Gove 2005 SMJ

The Role of Methods: The Role of Methods: A Bibliometric AnalysisA Bibliometric Analysis

Do Methods Contribute to Article Do Methods Contribute to Article Impact?Impact?

Beyer et al (1995 AMJ): Sophistication of statistical analysis unrelated to revise and resubmit or final acceptance decisions

Boyd, Gove & Hitt (2005 SMJ): While weak measurement is endemic to strategic management, quality of measurement unrelated to subsequent article citation

Bergh et al (2006 SMJ): Use of primary data, control for internal validity threats, and more powerful samples positively related to subsequent citation

What we can do: What we can do: Using methods to enhance the impact of Using methods to enhance the impact of

management studies management studies

Option 1: Different theories offer distinct causal chains

Option 2: Build a bigger toolbox Option 3: Extend theory through search

for artifacts

Option 1: Option 1: Different theories offer distinct Different theories offer distinct

causal chainscausal chains

SEM: The Problem of SEM: The Problem of Equivalent ModelsEquivalent Models

SEM appealSEM appeal Concurrent test of

multiple relationships Both global and local

tests of hypotheses Global fit measures

enable comparison of competing hypotheses

Equivalent modelsEquivalent models Many configurations

can yield identical fit Rarely addressed in

published studies Completely different

patterns of covariation and causation

Henley et al, ORM 2006Henley et al, ORM 2006Replication of Stimpert & Duhaime 1997Replication of Stimpert & Duhaime 1997

IndustryProfitability

R&DExpenditures

FirmDiversification

CapitalInvestment

SBUEffectiveness

Hypothesized Hypothesized ModelModel

Henley et al, ORM 2006Henley et al, ORM 2006Replication of Stimpert & Duhaime 1997Replication of Stimpert & Duhaime 1997

IndustryProfitability

R&DExpenditures

FirmDiversification

CapitalInvestment

SBUEffectiveness

Reverse Causality Reverse Causality ModelModel

Henley et al, ORM 2006Henley et al, ORM 2006Replication of Stimpert & Duhaime 1997Replication of Stimpert & Duhaime 1997

IndustryProfitability

R&DExpenditures

FirmDiversification

CapitalInvestment

SBUEffectiveness

Covariation ModelCovariation Model

Each model has a Each model has a unique logic, but unique logic, but

identical chi-square, identical chi-square, GFI, CFI, and so onGFI, CFI, and so on

Content Analysis by Henley Content Analysis by Henley and Colleagues and Colleagues

Reviewed 109 SEM articles Few alternate or equivalent models

addressed in papers 79% of articles had at least one

equivalent model Causality reduced to covariation in 71%

of studies Causality reversed in 48% of studies

EquivalentEquivalent models follow models follow a very specific set of a very specific set of

criteria.criteria.

Much broader range of Much broader range of alternatealternate models models

available.available.

Direct Comparison of Direct Comparison of Competing LogicsCompeting Logics

Quality

Prominence

Salaries

R2 = .11

R2 = .36

0.33*

Faculty Exp

Degree Prestige

Faculty Pubs

Media Ranking

GMAT

R2 = .69

0.03

-.01

-.51*

0.17*

0.18*

0.59*

0.13*Quality

GMAT

MediaRank

FacPubs

DegPrestige

Reputation

Prominence

Salaries

R2 = .79

R2 = .45

0.96(12.1)

0.66(8.2)

0.37(3.4)

-.94(10.5)

1.0

0.92(11.2)

0.93(11.1)

Faculty Exp

-.20(2.9)

0.05(0.1)Source: Rindova, et al 2005 AMJ

Source: Boyd, Bergh & Ketchen 2010 JOM

Additional ConfigurationsAdditional Configurations

Quality

GMAT

MediaRank

FacPubs

DegPrestige

Reputation

Salaries

Faculty ExpProminence

(a) Prominence as part of reputation

Quality

GMAT

MediaRank

FacPubs

DegPrestige

ReputationProminence

Salaries

Faculty Exp

(b) Direct effect on salaries

Quality

GMAT

MediaRank

FacPubs

DegPrestige

Reputation

Elite

Faculty Exp

Prominence

Salaries

(c) Prominence and salaries as common dimension

Comparison of ModelsComparison of Models

Hypothesized model fit better than Rindova on all eight indicators

Alternative A – comparable to Rindova on all eight indicators

Alternative B – Best fit on four indicators, but less parsimonious and no significant improvement on 2

Alternative C – Better fit than Rindova, worse than hypothesized or Model b

Option 2: Build a bigger Option 2: Build a bigger toolboxtoolbox

Diversity in Tool Use: The Case of Diversity in Tool Use: The Case of Contingency ModelingContingency Modeling

0

10

20

30

40

50

60

70

80

90

Art

icle

s / y

ear

Total

Empirical

Contingency

The good news…

0%

20%

40%

60%

80%

100%

Overall 1980s 1990s 2000s

Covariation

Profile

Gestalt

Matching

Mediation

Subgroup

Interaction

Other Empirical

Conceptual

The bad news…

Data: 1,715 SMJ articles, 1980 - 2009Source: Boyd, et al 2012 JOM

Implications of Contingency Implications of Contingency Content AnalysisContent Analysis

Strategic management researchers increasingly likely to test contingency hypotheses. But,

Interaction has increasingly become “the” way to test contingency effects.

These findings echo trends in training of strategic management faculty (Shook, et al., SMJ 2003):- “…doctoral training may not be keeping pace with data analytic

trends and future research needs…most scholars limit themselves to research opportunities that fit with their narrow skill set (p.1236).”

- Few faculty develop additional data analytic skills post-Ph.D.

Option 3: Extend theory Option 3: Extend theory through search for artifactsthrough search for artifacts

ExamplesExamples

Agency theoryAgency theory Amihud and Lev (1981):

Agency factors a main driver of M&A activity

Lane, Cannella & Lubatkin (1998): Contrary position

Boyd, Gove & Hitt (2005b): Both theories relevant, attenuation is primary explanation for divergent results

Strategic consensusStrategic consensus Tests of dynamism as a

moderator of the consensus – performance link have reported weak and inconsistent results

Gonzalez-Benito et al (in press) use IO econ theory to:- Unpack consensus into

discrete components- Develop a mediated

moderation framework

A Plug for A Plug for Organizational Organizational

Research MethodsResearch MethodsORM publishes:ORM publishes:

- Essays on methods

- Debates

- Teachers’ Corner

- Methods reviews

- Book and software evaluations

2009 JCR:2009 JCR:- 16/112 Management

- 7/63 Applied Psych

- 2.471 Impact score

Special issue calls for papers:

Construct measurement in strategic management (due 12/1)

Research design (see July issue for CFP)

Links to Cited ArticlesLinks to Cited Articles

Boyd, Finkelstein & Gove (2005) SMJ Boyd, Gove & Hitt (2005a) SMJ Boyd, Gove & Hitt (2005b) SMJ Boyd, Bergh & Ketchen (2010) Boyd, Haynes, Hitt, Bergh & Ketchen

(2012) JOM Gonzalez-Benito, Aguinis

, Boyd & Suarez-Gonzalez (2011) JOM

Back-Up Slides

Productivity Benchmarks:Productivity Benchmarks:MacMillan ‘Outstanding’ MacMillan ‘Outstanding’

PublicationsPublications

0

2

4

6

8

10

12

14

0 1 2 3 4 5 6 7 8 9 10

75th ile

90th ile

95th ile

99th ile

Years post-Ph.D.

Cu

mul

ativ

eP

ubl

icat

ions

Productivity Benchmarks:Productivity Benchmarks:CitationsCitations

0

20

40

60

80

100

120

140

160

0 1 2 3 4 5 6 7 8 9 10

75th ile

90th ile

95th ile

99th ile

Years post-Ph.D.

Cu

mul

ativ

eC

itat

ion

s

Publications

Cites

Precocity

Ability

Prestige

Lead

A-Tier

GMAT

Boards

Gourman

Impact

Cumulative PublicationsCumulative Publications

5 6 7 8 9 10

N for year 771 728 685 644 610 565

Path from

Precocity 1 .35*** .35*** .32*** .39*** .52*** .36***

Ability 2 .21** .21** .22** .11 .06 .12

Prestige 3 .13*** .14*** .13** .14** .13** .06

CED 0.31 0.30 0.28 0.26 0.25 0.21

Cumulative CitationsCumulative Citations

5 6 7 8 9 10

N for year 771 728 685 644 610 565

Path from

Precocity 1 .37*** -.15** -.04 .05 -.06 -.06

Ability 2 -.01 .28*** .16** .09 .09 .18*

Prestige 3 .03 .04 .09* .02 .03 .02

Publications 1

.39*** .68*** .63*** .62*** .75*** .70***

CED 0.49 0.60 0.51 0.50 0.61 0.58

Impact (cites/article)Impact (cites/article)

5 6 7 8 9 10

N for year 771 728 685 644 610 565

Path from

Precocity 1 -.23*** -.13** -.10+ -.22** -.35*** -.22***

Ability 2 .40*** .27*** .24*** .33*** .53*** .48***

Prestige 3 .01 .04 .01 .04 .01 .04

Publications 1

-.30*** -.31*** -.28*** -.05 -.11* -.19***

Citations 2

.91*** .89*** .83*** .69*** .74*** .66***

CED .63 .59 .52 .49 .52 .47

Lotka’s LawLotka’s Law

Lotka (1926): The number of persons making 2 contributions is about one-fourth of those making one; the number making three contributions is about one-ninth, etc.; the number making n contributions is about 1/n2 of making one.

Inverse-square law validated in many other disciplines; untested in management

0%

10%

20%

30%

40%

50%

60%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Articles

% o

f Aut

hors

Current Study Lotka's Inverse-Square Law

Results of Lotka’s Law Results of Lotka’s Law Study 2 SampleStudy 2 Sample

Research Outcome ClustersResearch Outcome Clusters

0

1

2

3

4

5

6

0.4 0.8 1.2 1.6 2 2.4 2.8

Log of publications

Lo

g o

f ci

tatio

ns

Minors

Coase-Major Hybrid

Coases

Grinders

Stars

Majors