Organizational Research Methods 2013 Hamann 67 87

7/21/2019 Organizational Research Methods 2013 Hamann 67 87

1/21

Feature Topic: Construct Measurement in Strategic Management

Exploring the Dimensions of

Organizational Performance:A Construct Validity Study

P. Maik Hamann1, Frank Schiemann1,2,

Lucia Bellora1, and Thomas W. Guenther1

Abstract

Organizational performance is a fundamental construct in strategic management. Recently,

researchers proposed a framework for organizational performance that includes three dimensions:

accounting returns, growth, and stock market performance. We test the construct validity of

indicators of these dimensions by examining reliability, convergent validity, discriminant validity, and

nomological validity. We conduct a confirmatory factor analysis with 19 analytically derived indi-

cators on a sample of 37,262 firm-years for 4,868 listed U.S. organizations from 1990 to 2010. Our

results provide evidence of four, rather than three, organizational performance dimensions. Stock

market performance and growth are confirmed as separate dimensions, whereas accounting returns

must be decomposed into profitability and liquidity dimensions. Robustness analyses indicatestability of our inferences for three dissimilar industries and for a period of 21 years but reveal that

organizational performance dimensions underlie dynamics during years in which environmental

instability is high. Our study provides an initial contribution to the clarification of the important orga-

nizational performance construct by defining four dimensions and validating indicators for each

dimension. Thus, we provide essential groundwork for the measurement of organizational perfor-

mance in future empirical studies.

Keywords

factor analysis, quantitative research, reliability and validity, measurement models, organizational

performance

Organizational performance (OP) is fundamental to strategic management research. Research in this

field builds on the assumption that strategy influences OP (Lubatkin & Shrieves, 1986). Furthermore,

1Faculty of Business Management and Economics, Technische Universitat Dresden, Dresden, Germany2School of Business, Economics, and Social Science, University of Hamburg, Hamburg, Germany

Supplementary material for this article is available on the journals website at http://orm.sagepub.com/supplemental.

Corresponding Author:

P. Maik Hamann, Technische Universitat Dresden (TU Dresden), Faculty of Business Management and Economics, Chair of

Business Management especially Management Accounting and Control, D-01062 Dresden, Germany.

Email: [email protected]

Organizational Research Methods

16(1) 67-87

The Author(s) 2013

Reprints and permission:

sagepub.com/journalsPermissions.nav

DOI: 10.1177/1094428112470007orm.sagepub.com

by guest on February 26, 2015orm.sagepub.comDownloaded from
http://www.sagepub.com/journalsPermissions.navhttp://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://www.sagepub.com/journalsPermissions.nav


2/21

OP is the most common concept addressed in empirical studies in this field; for example, 28% of 439

empirical articles reviewed by March and Sutton (1997) and 29% of 722 articles reviewed by Richard,

Devinney, Yip, and Johnson (2009) include OP in their research design.

The OP construct refers to the phenomenon in which some organizations are more successful than

others. Aconstructis a conceptual term that researchers define to describe a real phenomenon and is

unobservable by nature (Edwards & Bagozzi, 2000). Consequently, OP is subject to the problem of

unobservables in strategic management research (Godfrey & Hill, 1995, p. 519). This problem is

best described in reference to the predictive validity framework (PVF). The PVF includes two levels:

the conceptual level and the operational level (Bisbe, Batista-Foguet, & Chenhall, 2007). At the con-

ceptual level, theories explain relationships between constructs through propositions. Subsequently,

these propositions are empirically tested at the operational level, at which researchers apply indica-

tors to measure a construct. Indicators are observed scores or quantified records (Edwards &

Bagozzi, 2000). The link between the two levels (i.e., between constructs and their indicators) is

crucial to advances in theoretical relationships between constructs. Only if this link is rigorously

established can empirical findings at the operational level be used to test theoretical propositionsinvolving unobservables at the conceptual level. This link is established by examining construct

validity. Construct validity reflects the correspondence between a construct and a measure taken

as evidence of the construct (Edwards, 2003, p. 329). Construct validity encompasses four criteria:

reliability, convergent validity, discriminant validity, and nomological validity (Schwab, 2005).

Paradoxically, in the past, a majority of strategic management researchers regarded construct validity

and the measurement of constructs as low-priority topics (Boyd, Gove, & Hitt, 2005). Consequently,

unobservables (e.g., OP) have often been measured by single indicators whose construct validity has

rarely been assessed. From the PVF, it follows that related theoretical inferences from such studies are

seriously undermined (Combs, Crook, & Shook, 2005; Starbuck, 2004; Venkatraman & Grant, 1986).

Because of its importance for strategic management research, a growing number of studies exam-ine the measurement of OP. These studies are shown in Table 1 and encompass two groups: (a) fac-

tor analyses of the dimensionality of OP (Devinney, Yip, & Johnson, 2010; Fryxell & Barton, 1990;

Rowe & Morrow, 1999; Venkatraman & Ramanujam, 1987) and (b) reviews of the OP measurement

practices used in strategic management research (Murphy, Trailer, & Hill, 1996; Richard et al.,

2009; Tosi, Werner, Katz, & Gomez-Mejia, 2000). The first group of studies provides evidence

of the multidimensionality of OP. However, these studies disagree on the number of OP dimensions

and do not systematically examine the construct validity of indicators that measure these dimen-

sions. Reviews of OP measurement practice provide evidence that empirical studies in strategic

management research employ a plethora of different and unrelated indicators (Murphy et al.,

1996); for example, Richard et al. (2009) reviewed 213 studies and identified 207 different OP

indicators. In this review, 49% of the studies measure OP with a single indicator despite the

multidimensional nature of OP, and 52% of the studies employ only cross-sectional data sets. How-

ever, none of the aforementioned studies develop a framework of the dimensions of OP at the

conceptual level or examine the construct validity of OP indicators based on such a framework.

Combs et al. (2005) directly address the first gap in the literature and develop a framework of the

OP dimensions based on a synthesis of prior studies that focus on OP dimensions and a review of OP

measurement practices. They divide OP into three dimensions: accounting returns, stock market

performance, and growth. Subsequently, they test the OP framework by conducting a confirmatory

factor analysis (CFA) based on a correlation matrix of five OP indicators derived from a meta-

analysis. Despite the significant contribution made by Combs et al., their study has three limitations.

First, Combs et al. do not offer clear definitions of the OP dimensions. Specification of the concep-tual domain and clear definitions of constructs are prerequisites for construct validity (Schwab,

2005). Second, the CFA with three factors and five OP indicators does not satisfy the two-

indicator rule of model identification (Kline, 2011). Consequently, Combs et al. offer only

68 Organizational Research Methods 16(1)

http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/


3/21

Table1

.PreviousStudiesThatExaminetheDim

ensionsofOrganizationalPerformance.

Study

Numberof

Dimensions

Number

of

Indicators

Numberof

Studies/

samplesize

Method

DimensionsofO

rganizationalPerformance

Operational

Performance

Reviewsand

meta-analyticstudies

Combs

,Crook,andShook

(2005)a

3

5

Notreported

(238studies)

Narrativereviewand

meta-analytic

al

CFA

Accountingreturns

Growth

Stockmarket

Operational

performance

Tosi,Werner,Katz,

andGomez-M

ejia(2000)a

8

30

Notreported

(137studies)

Meta-analyticalEFA

Absolutefinancial

performance

Changein

financial

performance

Stock

performance

Internal

performance

indicators

Returnonequitys

hort

term

Marketreturn

Returnonequitylongterm

Richard,D

evinney,Y

ip,a

nd

Johnson(2009)

3

n/a

213

Narrativereview

Financialperformance

Shareholder

return

Productmarket

performance

Murphy,Trailer,

andHill(1996

)b

4

n/a

52

Narrativereview

Efficiency

Size

Liquidity

Profit

Studiesusingpr

imaryorsecondarydata

Devinney,Yip,andJohnson

(2010)

4

10

Notreported

EFA

Accountingmeasure

Salesmeasures

(salesgrowth)

Marketvalue

Cashflow/profitability

dimension

RoweandMorrow(1999)

3

10

311(2

,398

firm-years)

CFA

Financial(accounting)

Stockmarket

Subjective

reputation

rating

Murphyetal.(1

996)b

9(4)

19(8)

995

(586)

PCA (C

FA)

Liquidity

Salesmeasures

(salesgrowth)

Size

Profitability

Profitgrowth

Salesefficiency

Incomeefficiency

Absoluteincome

Employeeefficiency

FryxellandBarto

n(1990)

2

4

168

CFA

Accounting-basedmeasures

Market-based

measures

VenkatramanandRamanujam

(1987)

3

3

86

MTMMandCFA

Profitability

Salesgrowth

Profitgrowth

Note:WeallocatethedimensionsoforganizationalperformancetotheframeworkofCombsetal.(

2005),whichisshowninboldfa

ce.T

hisframeworkalsoseparatesope

rationalperformanceand

organizationalpe

rformance.C

FA

confirmatoryfacto

ranalysis;EFA

exploratoryfactora

nalysis;PCA

principalcomponentsanalysis;MTMM

multitrait-multimethodmatrix.

aCombsetal.(2

005)andTosietal.(

2000)onlyreporttheoverallnumberofprimarystudie

sthattheyuseintheirreviews.Thisn

umberisprovidedinparentheses.

bMurphyetal.(1

996)conductanarrativereviewandan

empiricalanalysisthatisbasedonther

esultsoftheirreview.C

onsequently,w

eincludethisstudyinbothlists.F

urthermore,theresultsoftheir

exploratoryPCA

andtheirCFAaredifferentinsofarastheCFAencompassesonlyasubsetoftheindicatorsthatareemployedintheP

CA

.WepresentdetailspertainingtotheirCFAinparentheses.

69by guest on February 26, 2015orm.sagepub.comDownloaded from


4/21


5/21

2005). Common performance indicators, such as growth in market share, product quality, patent

filings, or marketing effectiveness, measure distinct dimensions of operational performance.

In contrast, OP is defined as the economic outcomes resulting from the interplay among an orga-

nizations attributes, actions, and environment (Combs et al., 2005, p. 261). The definition of OP

corresponds to measurement practices in strategic management research because a majority of

researchers assess OP based on economic indicators (Murphy et al., 1996; Richard et al., 2009).

Thus, OP is synonymous with the concepts of financial performance or corporate economic perfor-

mance (Fryxell & Barton, 1990). OP is relevant to both research and practice because in the legal

system (i.e., bankruptcy law or commercial law) and in economic theory, OP (i.e., economic out-

comes) constitutes the final aim of economic activities.

Combs et al. (2005) propose a consistent OP framework with three dimensions: accounting

returns, stock market performance, and growth.

Accounting returns are defined as the historical performance of organizations that is assessed

through the use of financial accounting data as published in annual reports (Fryxell & Barton,

1990). As shown in Table 1, Combs et al. (2005) argue for a single accounting returns dimension,whereas other studies identify several dimensions that are derived from accounting returns indica-

tors. However, we expect at least two separate dimensions to be reflected by accounting returns

indicators. First, a liquidity dimension, which is defined as a firms ability to meet its financial obli-

gations based on cash flows generated from its current operations, is expected (Weygandt, Kimmel,

& Kieso, 2010). Second, a profitability dimension, defined as an organizations efficiency in utiliz-

ing production factors to generate earnings, is expected. Accounting research highlights the differ-

ence between earnings (e.g., net profit) and cash flows that is traced to revenue and expense accruals

(e.g., Dechow, 1994). Accruals mitigate timing and matching problems associated with the alloca-

tion of cash flows to single periods but are subject to distortions caused by discretionary accounting

choices (e.g., a depreciation method or the useful life of assets). Additionally, Rappaport (1993)stresses the divergence between the accounting-based return on investment and the cash flow rate

of return.

Stock market performance reflects the perceptions of investors regarding organizations future

performance (Fryxell & Barton, 1990). This dimension is measured using capital market indicators,

such as total shareholder return (TSR). However, capital market indicators are also influenced by the

momentum and volatility of capital markets, the economy, and psychological effects (Richard et al.,

2009). Stock market performance reflects future OP, in contrast with accounting returns, which

entail a historical perspective. As shown in Table 1, previous studies provide consistent evidence

regarding stock market performance as a distinct OP dimension.

Organizational growth is defined as a change in an organizations size over time. Organizational

growth is a dynamic construct that is commonly evaluated based on three concepts of size: sales,

employees, and assets (Weinzimmer, Nystrom, & Freeman, 1998). As shown in Table 1, previous

studies that investigate the OP dimensions focus on sales growth and disregard employment and

asset growth.

Previous examinations of the dimensionality of OP are subject to three limitations. First, the

number of indicators used is small. For example, Fryxell and Barton (1990) use four indicators, and

Venkatraman and Ramanujam (1987) employ three indicators. However, a small number of indica-

tors may not capture the entire conceptual domain of a construct. Second, indicators are often not

chosen analytically. For example, Murphy et al. (1996) chose 19 OP indicators based on their fre-

quent usage by researchers. These indicators include absolute returns (e.g., net income), return ratios

(e.g., return on assets), size (e.g., number of employees), and ratios of balance sheet items (e.g., debtto equity). Given the conceptual domain of OP, the adequacy of some of these indicators is question-

able; for example, size and static balance sheet items differ conceptually from OP (Combs et al.,

2005; Tosi et al., 2000). If indicators are chosen inadequately, spurious factors may emerge or true

Hamann et al. 71



6/21

factors may be obscured in factor analyses (Fabrigar, Wegener, MacCallum, & Strahan, 1999).

Third, cash flow return indicators are absent in the majority of previous studies. Devinney et al.

(2010) and Rowe and Morrow (1999), who both include a single cash flow return indicator in their

factor analysis (cash flow return on sales and cash flow return on equity, respectively), are excep-

tions. This limitation is important because we expect the single accounting returns dimension pro-

posed by Combs et al. (2005) to divide into two dimensions (i.e., liquidity and profitability) when the

convergence of cash flow returns and profitability indicators is examined systematically.

Research Design

Assessment of Construct Validity

During the process of construct validation, four criteria are evaluated: reliability, convergent valid-

ity, discriminant validity, and nomological validity (Schwab, 2005). We employ CFA to examine

construct validity. First, the theory-testing approach of CFA is appropriate for the evaluation of thetwo competing models, the three-OPdimension model and the four-OPdimension model, that

emerged from our discussion of previous research. Second, this approach enables an examination

of the overall fit of a measurement model to a data set. Third, CFA permits researchers to test the

significance of factor loadings. Fourth, CFA supplies indices that provide insights into reliability,

convergent validity, and discriminant validity (Bagozzi, Yi, & Phillips, 1991; OLeary-Kelly &

Vokurka, 1998). Table 2 presents the methods and indices that are applied to assess the criteria

of construct validity (see also Bagozzi & Yi, 1988).

Prior to the assessment of the construct validity criteria in a CFA, the overall fit of the measure-

ment model to the data must be established (Anderson & Gerbing, 1988). The assessment of the

overall measurement model fit to the data is based on the chi-square statistic, the Comparative FitIndex (CFI), the root mean square error of approximation (RMSEA), the standardized root mean

square residual (SRMR), and the Akaikes Information Criterion (AIC). The methodological liter-

ature criticizes the use of definite cutoff criteria for these goodness-of-fit indices. Goodness-of-fit

indices are sensitive to the misspecification of a model and to sample size, model types, and data

non-normality. Consequently, definite cutoff criteria may yield a high Type I error (i.e., rejecting

acceptable misspecified models) if they are too conservative (Marsh, Hau, & Wen, 2004). We

account for this cutoff criteria ambiguity by differentiating between cutoff criteria for acceptable and

good fits of the measurement model to the data and by reporting more than one goodness-of-fit

index, as recommended by Hu and Bentler (1999).2 We compare the competing models of OP based

on their overall measurement model fit to the data. Hereafter, we employ the best fitting model to

examine the four criteria of construct validity.

Reliability is defined as the ratio of systematic variance to total variance (i.e., the degree to which

an indicator is free of random error). Reliability is a necessary prerequisite for validity (Schwab,

2005). Convergent validity is defined as the extent to which multiple indicators represent a common

construct. A number of indicators of the same construct should exhibit high levels of covariance to

be considered valid measures of the construct in question (Bagozzi et al., 1991). In contrast, discri-

minant validity is defined as the degree of divergence among indicators that are designed to measure

different constructs (Edwards, 2003). The methods that we apply to assess these criteria of construct

validity are presented in Table 2.

Nomological validity is based on evidence pertaining to the relationships between measures of

the construct under investigation and measures of other constructs. This evidence should be consis-tent with relevant theory or with the results of previous empirical studies (Schwab, 2005). Conse-

quently, we test the relationships between the dimensions of OP and the determinants and

consequences of OP. Capon, Farley, and Hoenig (1990) conducted a meta-analysis of the




7/21

Table2

.StatisticsandMethodsThatAreAp

pliedtoAssessConstructValidity

.

StepsinAssessing

ConstructV

alidity

AssessmentCriteria

ExplanationandThresholdsforAcceptability

Overallfito

fthe

measurem

ent

modelto

thedata

Chi-squarestatisticofthe

likelihoodratiotest

H0hypothesisoft

helikelihoodratiotestistheexactfitofaspecifiedmodeltoapop

ulation

(MacCallum,B

row

ne,&

Sugawara,1996

,p.1

32).

AcceptanceofH0

:pvalue>.0

5.

ComparativeFitIndex(CFI)

TheCFIdescribestherelativeimprovementinthefitofthemodelincomparisonwiththefitofthe

independencemodel.Thus,thisindexovercomessa

mplesizeeffects(Bentler,1990

,p

p.245-246)

.

Acceptablefit:CF

I>.9

0;goodfit:CFI>.9

5.

Rootmeansquareerrorof

approximation(RM

SEA)

TheRMSEAmeasuresthediscrepancybetweenthecovariancematrixestimatedfromthemodeland

theobservedmatrix.Thiscriterionadjustsforthemodeldegreesoffreedom(MacCallumetal.,

1996

,p.1

34).

Acceptablefit:RM

SEA .40)a

Cash flow return per employee .648***Cash flow return on sales .692***Cash flow return on assets .692***Return per employee .791***Return on sales .687***Return on assets .748***Employment growth .472***Sales growth .440***Assets growth .638***Total shareholder return .960***Sharpe ratio .723***

Jensens alpha .695***Treynor ratio .777***

Reliability of constructsConstruct reliability (> .60) a .863 .896 .761 .937Average variance extracted (> .50)a .677 .742 .517 .789

Discriminant validity: Fornell-Larcker criterionb

Liquidity .677Profitability .563 .742Growth .010 .047 .517

Stock market performance .017 .027 .048 .789Discriminant validity: Chi-square difference testc

Liquidity 3,241.46*** (3) 8,706.33*** (3) 9,386.71*** (3)Profitability 6,953.09*** (3) 11,473.98*** (3)Growth 14,101.30*** (3)

Nomological validity: Antecedent constructsd

Research and development intensity () .227*** .271*** .043*** .012*Capital investment intensity () .097*** .112*** .051*** .006Market concentration () .022** .018* .034*** .012**Market share () .053*** .054*** .014*** .013***

Nomological validity: Consequent constructsSurvival () .022** .040*** .007 .016***

Note:n 37,272 firm-years.aThe thresholds for item reliability, construct reliability, and average variance extracted are given in parentheses.bThe Fornell-Larcker criterion of discriminant validity is satisfied if the average variance extracted for a factor is greater thanits squared correlations with all other factors. The average variance extracted is presented on the diagonal.cThe chi-square difference test is performed between two two-factor models. In the first model, the correlation between thetwo factors is constrained to 1.0. In the second model, this correlation is freely estimated. A significant chi-square differenceindicates discriminant validity. Differences in degrees of freedom are given in parentheses.dNomological validity is tested in the 4FM-B. In this model, all measures of the antecedent constructs are regressed on eachsingle performance dimension, and each single performance dimension is regressed on survival. Comparative Fit Index (.934),root mean square error of approximation (.036), and standardized root mean square residual (.042) indicate the fit of thismodel. The expected signs are given in parentheses. The regression coefficients are presented in boldface if they are statis-tically significant and display the expected sign.*p< .05. **p< .01. ***p< .001.




15/21

extracted are above the thresholds for all factors, with the lowest values, .761 and .517, respectively,

determined for the growth dimension.

As Table 4 shows, 4FM-B provides evidence of convergent validity for all factors. First, the

factor loadings of all indicators exhibit acceptable convergence (i.e., l> .5). Second, if we consider

the stronger criteria of good convergence (i.e.,l > .7), all indicators of the liquidity, profitability,

and stock market performance factors are above this threshold. The convergence of the three growth

indicators is slightly weaker. The factor-loading estimate of the employment growth indicator (l

.687) and the sales growth indicator (l .663) are both statistically significant but slightly below the

threshold for good convergence.

As shown in Table 5, all factors in model 4FM-B demonstrate discriminant validity. The Fornell-

Larcker criterion holds for all factors. The chi-square difference tests, which compare fixed and

freely estimated two-factor models for all pairs of factors, support this conclusion. As Table 4 shows,

the liquidity and profitability factors exhibit the highest correlation (r .750) among the four fac-

tors. All other correlation coefficients are considerably lower (i.e., r< .25). These results indicate

four dimensions of the OP construct.As shown in Table 5, our analyses provide evidence of nomological validity with regard to the

four OP dimensions. Ten out of 16 regression coefficients of the antecedents of OP are statistically

significant and display the expected signs, according to Capon et al.s (1990) meta-analysis of 320

primary studies. The majority of OP indicators that are included in their meta-analysis belong to the

profitability and liquidity dimensions. Accordingly, all regression coefficients between the four

determinants and these two dimensions are statistically significant and in the expected directions.

Three regression coefficients of survival for the OP dimensions are significantly different from zero

and show the expected (i.e., positive) signs (liquidity, profitability, and stock market performance).

However, growth appears to be unrelated to the survival of companies in our sample.

Robustness Analyses

Table 6 presents the results of our robustness analyses. Our inferences regarding the construct valid-

ity of 4FM-B are stable across industries and time periods. We repeat our primary analyses of the

construct validity for each industry and each year separately. The overall model fit is acceptable for

all three fit indices in all three industries and in 18 out of 21 years.

The model fit is lowest for years with high environmental instability, as indicated by the volatility

and the annual return of the S&P 500 index (e.g., in 2002, after the burst of the dotcom bubble, and in

2008 and 2009, during the financial crisis). In particular, sales growth and employment growth fail

to demonstrate item reliability and convergence for years with high environmental instability. With

regard to the growth dimension, only the asset growth indicator demonstrates acceptable values forreliability and convergence for all years. The average variance extracted for the growth dimension is

below the threshold for almost every year after 2000 except 2005, indicating a weak construct relia-

bility for the last 11 years. The discriminant validity of the dimensions of OP is evident in all indus-

tries. However, for five of the years, the correlation coefficient between the profitability and

liquidity dimensions is high (i.e., r > .8). Consequently, during these years, the two dimensions

do not discriminate as strongly as implied by the primary analysis. Overall, our results generally

remain unchanged across industries and time periods.

Discussion and ConclusionsThe results of this study reveal the existence of four independent OP dimensions: liquidity, profit-

ability, growth, and stock market performance. The evidence of the construct validity of the four-

dimensional OP measurement scheme is strong and consistent across different time periods and

Hamann et al. 81



16/21

Table6

.StabilityoftheMeasurementModel4FMAcrossIndustriesandTime.

Fit

Statistics

Reliability

Validity

S&P500

Group

N

CFI

RMSEA

SRM

R

AIC

Chi-Square

Item

R

eliability

Construct

Reliability

Average

Variance

Extracted

Convergent

Validity

Discriminant

Validity

Volatility

Annual

Return

Industries

ICB2000

16,2

43

.943

.044

.04

7

441,800

1,821.91(56)

13/13

4/4

4/4

13/13

12/12

n/a

n/a

ICB5000

10,0

05

.950

.041

.04

4

282,383

979.99(56)

13/13

4/4

3/4

13/13

12/12

n/a

n/a

ICB9000

11,1

04

.950

.048

.04

4

278.172

1,486.66(56)

13/13

4/4

4/4

13/13

12/12

n/a

n/a

Time

1990

513

.950

.050

.06

2

14,9

86

131.53(57)

11/13

4/4

4/4

11/13

12/12

4.9214

3.1

0%

1991

970

.921

.071

.06

0

25,0

23

333.48(56)

13/13

4/4

4/4

13/13

12/12

5.6230

30.4

7%

1992

1,026

.949

.058

.05

1

27,6

79

252.88(57)

13/13

4/4

4/4

13/13

12/12

2.6822

7.62%

1993

1,097

.960

.054

.04

2

29,6

27

232.52(56)

13/13

4/4

4/4

13/13

12/12

2.9739

10.0

8%

1994

1,163

.944

.063

.02

7

21,9

29

310.88(56)

13/13

4/4

4/4

13/13

12/12

2.0597

0.85%

1995

1,601

.941

.064

.06

1

43,7

31

427.74(56)

12/13

4/4

4/4

13/13

12/12

9.2497

37.0

5%

1996

1,820

.940

.062

.04

9

47,8

70

447.95(56)

13/13

4/4

4/4

13/13

12/12

6.2587

22.9

6%

1997

1,976

.949

.055

.05

8

51,8

27

384.76(56)

13/13

4/4

4/4

13/13

12/12

9.2017

33.3

6%

1998

2,125

.945

.058

.05

2

55,8

34

454.01(56)

13/13

4/4

4/4

13/13

12/12

6.4972

28.5

8%

1999

2,124

.951

.055

.04

4

49,1

37

410.97(56)

13/13

4/4

4/4

13/13

12/12

4.6309

21.0

4%

2000

2,419

.907

.063

.07

3

66,6

79

585.18(56)

11/13

4/4

3/4

13/13

12/12

3.8732

8.8

1%

2001

2,402

.905

.067

.07

1

64,2

64

662.28(56)

12/13

4/4

3/4

12/13

12/12

7.0106

11.89%

2002

2,352

.898

.070

.05

9

65,2

62

697.31(56)

11/13

4/4

3/4

13/13

12/12

11.0

146

22.10%

2003

2,243

.915

.056

.03

7

48,4

33

448.34(56)

13/13

4/4

3/4

13/13

10/12

8.6373

28.6

8%

2004

2,158

.949

.054

.04

7

42,0

20

413.94(56)

12/13

4/4

3/4

13/13

10/12

2.9923

10.8

8%

2005

2,042

.929

.055

.05

7

47,6

12

404.52(56)

13/13

4/4

4/4

13/13

10/12

2.8477

4.77%

2006

2,001

.955

.047

.05

6

52,6

09

306.65(56)

13/13

4/4

3/4

13/13

12/12

4.3365

15.2

3%

2007

1,926

.946

.047

.06

9

52,9

65

290.61(56)

10/13

4/4

3/4

13/13

11/12

3.3149

5.49%

2008

1,829

.855

.072

.06

2

54,7

05

581.51(56)

10/13

4/4

3/4

13/13

12/12

15.1

583

37.00%

2009

1,777

.889

.066

.04

9

37,5

70

492.22(56)

11/13

4/4

3/4

13/13

12/12

12.7

387

26.4

6%

2010

1,698

.909

.066

.04

4

43,0

64

473.18(56)

12/13

4/4

3/4

13/13

11/12

5.2141

15.0

6%

Note:Satisfie

dcriteriaarepresentedinboldface.Re

liabilityandvalidityarereportedasthe

proportionofindicatorsorfactorsthatsatisfyeachcriterion.T

hevolatilityof

theS&P500iscalculated

asthenormalizedstandarddeviationbasedonthed

ailyreturnindexineachyear.C

FI

ComparativeFitIndex;RMSEA

rootm

eansquareerrorofapproximation;SR

MR

standardizedroot

meansquare

residual;AIC

AkaikesInformation

Criterion;ICB

IndustryClassificatio

nBenchmark.

82by guest on February 26, 2015orm.sagepub.comDownloaded from


17/21

industries. We demonstrate that the accounting return dimension that Combs et al. (2005) propose

should be decomposed into the two distinct dimensions of liquidity and profitability. Organizations

differ in their ability to meet financial obligations based on cash flows generated from their current

operations and in their efficiency to utilize production factors to generate earnings. For years with

high environmental instability, the overall fit of the measurement model to the data becomes weaker.

This result contributes to the findings of Fryxell and Barton (1990), who provided evidence on

changes in the structure of the measurement of OP between years with low and high environmental

instability.

Our study contributes to research on OP measurement in three ways. First, regarding the concep-

tual level of the PVF, we clarify the OP construct. Second, regarding the link between the conceptual

and operational levels of the PVF, we establish the construct validity of 13 OP indicators. Third, we

reveal the dynamics of OP measurement over time.

First, construct clarity encompasses definitions, semantic relationships, contextual conditions,

and coherence (Suddaby, 2010). We offer definitions of OP and its dimensions, which capture essen-

tial characteristics, avoid circularity, and are parsimonious. Additionally, we integrate OP and itsdimensions into a hierarchy of related performance constructs, with organizational effectiveness

at the top of this hierarchy. Furthermore, we provide evidence that the nature of the OP construct

is highly sensitive to environmental instability as an important contextual condition. Researchers

who study OP should be aware of the four different (not interchangeable) dimensions and their con-

textual conditions.

Second, we develop a set of OP indicators at the operational level of the PVF. In addition, we test

the construct validity of these OP indicators based on the four OP dimensions. Thus, we contribute to

the establishment of a link between the conceptual and operational levels of an important construct

in strategic management research. We empirically confirm that hybrid indicators should be avoided

when measuring OP and its dimensions, as recommended by Combs et al. (2005). Additionally, wepropose a measurement scheme of 13 OP indicators that measure all four dimensions in a construct-

valid and parsimonious manner.

Third, environmental instability influences the measurement structure of OP. Researchers should

carefully control for this factor if they address OP in their research design. We recommend that

researchers avoid a nonlongitudinal measurement of OP for years characterized by high environ-

mental instability.

Our findings have important implications for future strategic management research. First, strate-

gic management theories that address variations in OP must consider the four OP dimensions as an

entity or only concentrate on selected dimensions. This implication is further underscored by our

evaluation of antecedents of OP. Profitability and liquidity are influenced by all four tested antece-

dents, whereas our results regarding the growth and stock market performance dimensions present a

different picture of these relationships. This issue must be addressed at the conceptual level, and it

offers fruitful avenues for future theoretical research on the determinants of OP. Second, empirical

researchers addressing OP in their work are encouraged to use the four-dimensional OP measure-

ment scheme, for which construct validity has been established. Our study contributes to the reduc-

tion of the plethora of OP indicators that may be employed in future empirical studies and thus may

facilitate an increase in rigor and relevance in strategic management research.

Our study has three limitations. First, we concentrate on listed organizations because the stock

market performance dimension is applicable to only these organizations. Thus, the question of

whether the construct validity of the other three OP dimensions also holds in organizations that are

not active in the capital market remains unanswered. Second, we employ secondary, objective OPdata according to the recommendation of Dess and Robinson (1984). The question of whether the

four OP dimensions are also applicable to other performance data, such as perceptual OP indicators,

remains unanswered. Third, OP is only one of several important performance constructs (e.g.,

Hamann et al. 83



18/21

operational performance or corporate environmental performance). These other performance con-

structs are also subject to measurement issues. Consequently, developing and testing construct-

valid measurement schemes for these constructs offers possibilities for future research.8 Such

research may draw on the methodology of this study and employ the four-dimensional OP measure-

ment scheme to test nomological validity.

Valid measurement is the sine qua non of science. If the measures used in a discipline have not

been demonstrated to have a high degree of validity, that discipline is not a science (Peter, 1979, p.

6). In this respect, our study contributes to the valid measurement of the most important construct in

strategic management research.

Acknowledgments

We would like to thank Mark Orlitzky, Christoph Trumpp, and two anonymous reviewers for their insightful

comments on previous versions of this manuscript. All remaining mistakes are our own.

Declaration of Conflicting InterestsThe author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publi-

cation of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

1. We thank an anonymous reviewer for highlighting the importance of construct clarity.

2. We thank an anonymous reviewer for drawing our attention to this relevant issue.

3. Survival is usually measured by a categorical variable that represents an organizations enduring presence inthe market (Richard, Devinney, Yip, & Johnson, 2009, pp. 732-734). We measure survival as the proportion

of years during which an organization is present in the stock market. We calculate the proportion during each

year of our time period based on the equation s (tey)/(2010 y), withs survival,t

e the year before a

company is delisted or the last year within our time period (i.e., 2010), andy year under consideration.

This operationalization is coarse-grained, but it corresponds with Baker and Kennedys (2002, p. 326) study.

4. We acknowledge that there are alternative measures for assets (e.g., capital employed) and for net profit (e.g.,

EBIT). Thus, ratios that use net profit as the nominator and assets as the denominator are exemplary for an

entire set of potential accounting return indicators. In online supplement 3, we extend our findings to these

indicators.

5. We calculate growth rates in all instances based on the [(tn

tn1)/tn 1] formula (Weinzimmer, Nystrom, &

Freeman, 1998, p. 253).

6. We present an extended sample description including descriptive statistics (e.g., correlations, skewness, kur-

tosis, intraclass correlation coefficients, and design factors) in online supplement 1.

7. Multilevel confirmatory factor analysis (CFA) is another method that accounts for the dependence of our

data. We examine the robustness of our analysis with regard to this methodological decision by conducting

a two-level CFA. Results of this two-level CFA are similar to our primary results and are presented in online

supplement 2. We thank an anonymous reviewer for drawing our attention to this issue.

8. We thank an anonymous reviewer for highlighting this limitation.

References

Akaike, H. (1974). A new look at the statistical model identification.IEEE Transactions on Automatic Control,19(6), 716-723. doi:10.1109/TAC.1974.1100705

Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recom-

mended two-step approach.Psychological Bulletin,103(3), 411-423. doi:10.1037/0033-2909.103.3.411




19/21

Bagozzi, R. P., & Baumgartner, H. (1994). The evaluation of structural equation models and hypothesis testing.

In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 386-422). Cambridge, MA: Blackwell.

Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of

Marketing Science, 16(1), 74-94. doi:10.1177/009207038801600107

Bagozzi, R. P., Yi, Y., & Phillips, L. W. (1991). Assessing construct validity in organizational research.

Administrative Science Quarterly,36(3), 421-458.

Baker, G. P., & Kennedy, R. E. (2002). Survivorship and the economic grim reaper.Journal of Law, Economics,

and Organization, 18(2), 324-361. doi:10.1093/jleo/18.2.324

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin,107(2), 238-246.

doi:10.1037/0033-2909.107.2.238

Bercovitz, J., & Mitchell, W. (2007). When is more better? The impact of business scale and scope on long-term

business survival, while controlling for profitability.Strategic Management Journal,28(1), 61-79. doi:10.10

02/smj.568

Bisbe, J., Batista-Foguet, J.-M., & Chenhall, R. (2007). Defining management accounting constructs: A meth-

odological note on the risks of conceptual misspecification.Accounting, Organizations and Society, 32(7-8),

789-820. doi:10.1016/j.aos.2006.09.010

Boyd, B. K., Gove, S., & Hitt, M. A. (2005). Construct measurement in strategic management research: Illusion

or reality.Strategic Management Journal, 26(3), 239-257. doi:10.1002/smj.444

Burt, R. S. (1976). Interpretational confounding of unobserved variables in structural equation models.

Sociological Methods & Research,5(1), 3-52. doi:10.1177/004912417600500101

Capon, N., Farley, J. U., & Hoenig, S. (1990). Determinants of financial performance: A meta-analysis.

Management Science,36(10), 1143-1159.

Carlson, K. D., & Herdman, A. O. (2010). Understanding the impact of convergent validity on research results.

Organizational Research Methods,15(2), 17-32. doi:10.1177/1094428110392383

Combs, J. G., Crook, T. R., & Shook, C. L. (2005). The dimensionality of organizational performance and itsimplications for strategic management research. In D. J. Ketchen (Ed.),Research methodology in strategy

and management (Vol. 2, pp. 259-286). Amsterdam: Elsevier.

Dechow, P. M. (1994). Accounting earnings and cash flows as measures of firm performance: The role

of accounting accruals. Journal of Accounting and Economics, 18(1), 3-42. doi:10.1016/0165-

4101(94)90016-7

Dess, G. G., & Robinson, R. B., Jr. (1984). Measuring organizational performance in the absence of objective

measures: The case of the privately-held firm and conglomerate business unit. Strategic Management

Journal,5(3), 265-273. doi:10.1002/smj.4250050306

Devinney, T. M., Yip, G. S., & Johnson, G. (2010). Using frontier analysis to evaluate company performance.

British Journal of Management,21(4), 921-938. doi:10.1111/j.1467-8551.2009.00650.xEdwards, J. R. (2003). Construct validation in organizational behavior research. In J. Greenberg (Ed.),

Organizational behavior: The state of the science(2nd ed., pp. 327-371). Mahwah, NJ: Erlbaum.

Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and

measures.Psychological Methods,5(2), 155-174. doi:10.1037/1082-989x.5.2.155

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor

analysis in psychological research.Psychological Methods,4(3), 272-299. doi:10.1037/1082-989X.4.3.272

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobserveable variables and

measurement error. Journal of Marketing Research, 18(1), 39-50.

Fryxell, G. E., & Barton, S. L. (1990). Temporal and contextual change in the measurement structure of finan-

cial performance: Implications for strategy research. Journal of Management,16(3), 553-569. doi:10.1177/014920639001600303

Godfrey, P. C., & Hill, C. W. L. (1995). The problem of unobservables in strategic management research.

Strategic Management Journal,16(7), 519-533. doi:10.1002/smj.4250160703

Hamann et al. 85



20/21


21/21

Venkatraman, N., & Ramanujam, V. (1987). Measurement of business economic performance: An examination

of method convergence. Journal of Management,13(1), 109-122. doi:10.1177/014920638701300109

Weinzimmer, L. G., Nystrom, P. C., & Freeman, S. J. (1998). Measuring organizational growth: Issues, con-

sequences, and guidelines.Journal of Management,24(2), 235-262. doi:10.1177/014920639802400205

Weygandt, J. J., Kimmel, P. D., & Kieso, D. E. (2010). Accounting principles. Hoboken, NJ: Wiley.

Author Biographies

P. Maik Hamann is research assistant and PhD candidate in Management Control/ Strategic Management at

the Technische Universitat Dresden. P. Maik Hamann received his bachelors degree from the University of

Paisley in Scotland and his German diploma degree from the Technische Universitat Dresden in Germany.

He is also lecturer of management accounting at the Technische Universitat Dresden and part-time for Dresden

International University. His main research interests encompass the design of corporate planning systems,

effects of corporate planning at the organizational level, contingency theory, measurement of organizational

effectiveness, construct validity, and philosophy of science.

Frank Schiemann is an Assistant Professor of Accounting at the University of Hamburg, Germany. He

received his PhD degree at the Technische Universitat Dresden, Germany. He was/is lecturer of management

accounting at the Technische Universitat Dresden, University of Hamburg and part-time for Dresden Interna-

tional University. His research focuses on firm valuation models as well as determinants and effects of firms

voluntary and mandatory disclosure decisions via different communication channels. His methodological focus

is on quantitative analysis methods, especially panel data models.

Lucia Bellora is a PhD candidate in Management Accounting and Management Control at the Technische

Universitat Dresden, Germany. She received her German diploma degree from the Technische Universitat in

Dresden. Lucia Bellora is also a lecturer of management accounting at the Technische Universitat Dresden, and

part-time at the Dresden International University and at the International Graduate School Zittau. Her research

interests include the design and performance effects of management control systems, the disclosure of extra-financial information, and the validity of construct measurement. Her methodological interest is directed

especially towards quantitative analysis methods with a focus on structural equation modeling.

Thomas W. Guenther is a professor of Management Accounting and Control at Technische Universitat

Dresden. Thomas Guenther received his PhD and habilitation degree from University of Augsburg, Germany.

He has been a visiting professor several times at the University of Virginia and was/is teaching in MBA and

executive programs at Wirtschaftsuniversitat Wien, Austria; European Business School (EBS), Wiesbaden,

Germany; and Mannheim Business School, Germany. His work covers two fields of research: first, the design

of management control systems within management accounting and strategic management research, and

second, the measurement, valuation, and control of intangibles in financial and management accounting. He

is editor-in-chief of the Journal of Management Control and is on the editorial board of the Business

Administration Review (BARev). He also serves as board member of the Schmalenbach Association. His meth-

odological background is in quantitative analysis, especially structural equation modeling, meta-analyses and

panel data models.

Hamann et al. 87

Documents

Organizational Research Methods 2013 Hamann 67 87