Chasing a Moving Target: Exploitation and Exploration in Dynamic Environments

MANAGEMENT SCIENCEVol. 58, No. 3, March 2012, pp. 587–601ISSN 0025-1909 (print) � ISSN 1526-5501 (online) http://dx.doi.org/10.1287/mnsc.1110.1420

© 2012 INFORMS

Chasing a Moving Target: Exploitation andExploration in Dynamic Environments

Hart E. PosenRoss School of Business, University of Michigan, Ann Arbor, Michigan 48109, [email protected]

Daniel A. LevinthalThe Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104, [email protected]

Acommon justification for organizational change is that the circumstances in which the organization findsitself have changed, thereby eroding the value of utilizing existing knowledge. On the surface, the claim

that organizations should adapt by generating new knowledge seems obvious and compelling. However, thisstandard wisdom overlooks the possibility that the reward to generating new knowledge may itself be eroded ifchange is an ongoing property of the environment. This observation in turn suggests that environmental changeis not a self-evident call for strategies of greater exploration. Indeed, under some conditions the appropriateresponse to environmental change is a renewed focus on exploiting existing knowledge and opportunities.We develop a computational model based on the canonical multiarmed bandit formulation of exploration andexploitation. We endeavor to understand the mechanisms by which environmental change acts to make pur-poseful efforts at organizational adaptation less (or more) valuable.

Key words : adaptation; learning; exploration; exploitation; turbulenceHistory : Received January 21, 2010; accepted May 14, 2011, by Jesper Sørensen, organizations. Published online

in Articles in Advance October 14, 2011.

1. IntroductionA common justification for organizational change isthat the circumstances in which the organization findsitself have changed. This idea is broadly reflectedin the contemporary literature’s focus on, for exam-ple, how the pace of environmental change accentu-ates the need for exploratory search (see, for instance,March 1991) and the importance and desirability ofdynamic capabilities (see, for instance, Teece et al.1997). Claims about the value of organizational actionaimed at adaptation in the face of environmentalchange have a taken-for-granted quality and, on thesurface, seem obvious and compelling. Yet as we willargue, change in the external environment does notnecessarily imply the need for, or benefit of, expandedefforts at organizational adaptation. Indeed, undersome conditions the appropriate response to environ-mental change is a renewed focus on exploiting exist-ing knowledge and opportunities.

In examining the merits of organizational adapta-tion, our work is closely related to March (1991), whoidentifies the need to allocate limited resources acrossboth the exploitation of the known and exploration ofthe novel as a central strategic trade-off. If balancingexploration and exploitation is central to firm perfor-mance, as the consensus now suggests (Gupta et al.2006), then what factors affect the appropriate bal-ance? To answer this question, March (1991) focuses

on the role of environmental change, reflecting a longtradition of interest in the external environment andthe ability and willingness of organizations to adapt(see, for instance, Benner 2009, Hannan and Freeman1984, Henderson 1993, Lant and Mezias 1992, Soren-son and Stuart 2000, Tushman and Romanelli 1985).March (1991, p. 80) concludes that “exogenous envi-ronmental change makes adaptation essential,” but hethen tempers this conclusion by noting that it also“makes learning from experience difficult.”

The broader strategy literature is generally less cir-cumspect. In recognizing the importance of Schum-peterian dynamics, the literature increasingly focusesattention on dynamic capabilities (Teece et al. 1997),high-velocity markets (Brown and Eisenhardt 1997),and hypercompetition (D’Aveni and Gunther 1994).Although in substance this work is nuanced in itstreatment of strategic responses to environmentalchange, in practice it is often interpreted as sug-gesting a simple positive relationship between theextent of environmental change and the merits oforganizational adaptation. This idea is diffused inthe conventional wisdom of the practitioner litera-ture, and managerial sentiment seems to reflect Petersand Waterman’s (1982) call for a “bias for action.”Although we agree with the Schumpeterian focusof much recent research, our analysis suggests thatan exhortation to action in dynamic environments

587

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.

Posen and Levinthal: Chasing a Moving Target588 Management Science 58(3), pp. 587–601, © 2012 INFORMS

is not always appropriate. As such, we endeavor tounderstand the mechanisms by which environmen-tal change makes purposeful efforts at organizationaladaptation less (or more) valuable.

We anchor our formal development on the multi-armed bandit model. This model is the canonical rep-resentation of the exploration–exploitation problemacross literatures as diverse as economics and statis-tics (Berry and Fristedt 1985, Gittins 1979, Robbins1952) and computer science (Holland 1975, Suttonand Barto 1998). In the management literature, March(1991) originally formulated his analysis in terms of aspecific form of a genetic algorithm;1 subsequently, heformulated his discussion of learning in terms of thebandit model (Denrell and March 2001; March 1996,2003, 2010).

Consider the bandit model in the context of a sim-ple organization, where management consists of mak-ing choices under uncertainty. In each period, theorganization must choose from a set of policy alter-natives; it may choose to continue with the policychoice made in the prior period, or it may select adifferent policy choice. The payoff for choosing a par-ticular alternative reflects a draw from a probabil-ity distribution with an unknown, alternative-specific,mean. In pursuing its activity, the organization seeksto maximize the flow of returns over time. The orga-nization makes its choice in each period based onits beliefs about the expected returns to each of thealternatives. If these subjective beliefs were knownto be objectively correct, then the organization wouldchoose to exploit the alternative known to have thehighest expected return. In the absence of such (per-fect) knowledge, the organization employs a strategythat allocates effort between (a) exploiting the alterna-tive currently believed to be superior and (b) explor-ing alternatives that currently seem less promising inhope of identifying a superior alternative to the cur-rent preferred alternative.

We examine this bandit model of organizationalchoice and focus on the implications of environmentalturbulence that alters the returns to particular policyalternatives. Through our model, we highlight twomain observations. First, the appropriate responseto environmental change is not necessarily a strate-gic shift of effort toward exploration—indeed, a shifttoward exploitation is sometimes superior.2 Environ-mental change is a two-sided “blade” with respect

1 The model in March (1991) is a specific form of a genetic algorithmin the sense that individuals do not recombine attributes of otherindividuals directly, but rather they do so indirectly, mediated bythe “organizational code.”2 Keller and Rady (1999) developed a related result of a monopo-list responding to a demand curve that shifts between two states,showing that the monopolist may engage in less experimentationwhen the frequency of the shift between the two states increases.

to the benefits of organizational adaptation. Schol-ars have tended to focus on the diminution of thefirm’s current knowledge stock and the value of exist-ing practices, when change renders “a prior strate-gic orientation 0 0 0no longer effective” (Tushman andRomanelli 1985, p. 180). Yet there is another, less thor-oughly considered, side of the blade. It reflects thepossibility that recurring, though not necessarily pre-dictable, change erodes the future benefits of addi-tional knowledge gained via exploration, suggesting amore circumspect conclusion as to the merits of orga-nizational adaptation in dynamic environments.

Second, organizations are biased toward action inthe face of environmental change, even in the absenceof a strategic shift toward exploration. This actionbias arises endogenously in the process of experi-ential learning where choices are based on beliefsabout the merits of the alternatives. An organiza-tion exploits a particular policy alternative when it isbelieved to be markedly superior to others. Environ-mental change naturally undermines existing beliefs.Alternatives that were believed to be rather good arelikely to seem worse. Moreover, there is the possibilitythat less-examined alternatives may have improved.This compression of beliefs leads to an action bias indynamic environments.

This paper proceeds as follows. In §2, we begin bydiscussing the theoretical background of our model.Then we describe the multiarmed bandit computa-tional model. In §3, we present the results of threesimulation experiments. The first experiment exam-ines the baseline properties of the model by focusingon a simple static environment. The second experi-ment adds turbulence to the baseline experiment. Indoing so, we examine the mechanisms that underliethe action bias and the appropriateness of purpose-ful efforts at adaptation in the face of change. Thethird experiment examines the implications of non-neutral turbulence that not only changes the relativemerits of alternatives but also makes the average bet-ter or worse. In §4, we discuss the implications ofthese results for theory and practice.

2. ModelThe multiarmed bandit model takes its name froma slot machine analogy. In the analogy, a gamblersits in front of a slot machine with N arms, eachhaving different underlying payoff probabilities thatare unknown to the player. The gambler’s objectiveis to maximize the returns to playing the machine.This model has been the subject of significant studybecause its underlying structure closely resembles avariety of realistic economic situations ranging fromresearch settings, such as research and development(Hardwick and Stout 1992), to strategic issues, such

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.

Posen and Levinthal: Chasing a Moving TargetManagement Science 58(3), pp. 587–601, © 2012 INFORMS 589

as product pricing (Bergemann and Välimäki 1996),to consumer choice (Gans et al. 2007). Common to allof these settings is that information about the returnsto an alternative can only be gathered by selecting it.As such, the primary feature of the bandit model isthe process of search and learning (Denrell and March2001, March 2010). As a consequence, as Holland(1975, p. 77) notes, in the bandit model “the trade-offbetween gathering more information and exploitingit appears in the simplest terms.”

Formalizing the discussion above, the bandit modelreflects a sequential choice problem where, at eachpoint in time, t, an organization must choose betweenN alternatives. The realized outcome of a partic-ular choice is a draw from an unknown (alterna-tive specific) probability distribution. If the process isBernoulli, then the choice of alternative i results in anoutcome of � = 41105—with � = 1 reflecting a posi-tive outcome with probability pi and � = 0 reflecting anegative outcome with probability 1−pi. As such, thestate of the environment can be described by the vec-tor of payoffs to the alternatives, Pt = 6p11 t1 0 0 0 1 pN1 t7.In the typical bandit model formulation, the environ-ment is stable across time such that for all t, Pt = P .We relax this assumption to allow for environmentalturbulence.3 Consistent with the definition of turbu-lence as reflecting both the frequency and unpre-dictability of environmental change (Baum and Wally2003, Dess and Beard 1984, Miller 1987, Wholey andBrittain 1989), we model the level of turbulence asthe probability of a shock to the payoff values ofalternatives.

We assume that an organization is a boundedlyrational actor engaged in a process of search andlearning (Denrell and March 2001, March 2010).Choice behavior in the bandit model is driven bythe relationship between two key organizational con-structs: beliefs and strategies. We discuss each in turnbelow.

In the bandit model, the first key feature of organi-zational behavior is that an organization is assumedto make its choice (select an alternative) based on itssubjective assessment of the merits of the alternatives.In employing beliefs to facilitate choice, the modelportrays a boundedly rational organization as pos-sessing a mental model or cognitive map that encap-sulates its understanding of the merits of the availableset of choices (Barr et al. 1992, Gavetti and Levinthal2000, Nelson 2008, Weick 1969). In particular, an orga-nization’s belief about the merits of a given alterna-tive is qi1 t , indexed by alternative i at time t, such that

3 A static environment is typically assumed in the economics lit-erature because of the challenges associated with obtaining closedform solutions. The rare exception here is Whittle’s (1988) “restlessbandit” model.

its set of beliefs across alternatives is characterized byQt = 6q11 t1 0 0 0 1 qN1 t7 with 0 ≤ qi1 t ≤ 1.

Two dimensions of beliefs are particularly rele-vant. First, subjective beliefs may (or may not) haveobjective validity. To the extent that beliefs accu-rately characterize the state of the world and reflectthe true underlying value of the alternatives, thenthey embody knowledge of the choice opportuni-ties that the organization faces. Thus, we character-ize knowledge based on the sum of squared errorsin beliefs relative to the true alternative payoffs as1 −

∑N1 4pi − qi5

2. Second, beliefs may embody strongor weak opinions. The strength of opinions reflectsthe extent to which an organization believes that somealternatives are strongly superior to others. We char-acterize this as the variance across beliefs,

∑N1 4q̄−qi5

2.To refine its beliefs and maximize the value of its

stream of rewards, an organization engages in learn-ing. In the initial period t = 0, the organization hasprior beliefs, Q0 = 6q1101 0 0 0 1 qN107, about the value ofthe N reward outcome probabilities P = 6p11 0 0 0 1 pN 7.In each subsequent period, the organization makes achoice from the set of alternatives. By acting—makinga choice of alternative i—a firm receives feedbackfrom the environment in the form of an outcome�i1 t = 41105 associated with a reward of ri1 t = 4r+1 r−5.This reward may, for example, take the form of salesand profits (or lack thereof), which augments thefirm’s stock of assets such that St =

∑t1 rt .

Equally important, from the learning perspec-tive, performance feedback provides new informationabout action-outcome linkages (the value of alterna-tives) that serve to update beliefs. In particular, beliefsare updated as the organization chooses one option iand observes the outcome, �t = 41105. We assume thatbeliefs at any given point in time reflect the averagereward over an organization’s entire history of trialsof a given alternative (March 1996, Sutton and Barto1998). This simple average updating is a special caseof the more general Bush and Mosteller (1955) updat-ing methodology. In a given period, the firm updatesits beliefs, qi1 t , of the selected alternative followinga Bush–Mosteller fractional adjustment methodologywhere qi1 t+1 = qi1 t + �4�t − qi1 t5 and � is the rate ofupdating. Average updating is the special case of �=

1/4ki+15, where ki is the number of times alternative ihas been tried at time t. For alternatives that are notselected in a given period, qi1 t+1 = qi1 t .

The second key feature of organizational behaviorin the bandit model is that an organization is assumedto possess a search strategy. The organization wouldlike to select, in each period, the alternative withthe highest expected value, max4p11 0 0 0 1 pN 5. How-ever, the expected payoffs are unknown to the orga-nization. Thus, conditional on a given set of beliefs

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


about the relative merits of the alternatives, an organi-zation requires a mechanism for choosing among thealternatives. An organization’s search strategy can beconceived of as a rule or routine (Cyert and March1963, Nelson and Winter 1982) that maps the orga-nization’s beliefs about the merits of alternatives tospecific choices from the set of alternatives.

Under an appropriate set of assumptions, Gittinsand Jones (1974) demonstrate that an optimal strat-egy exists. The basic property of this rational choicestrategy in the bandit model is as follows. Consider aslot machine with two arms; arm 1 has an unknownpayoff probability, and arm 2 has a known payoffprobability (or a fixed payoff). The rational choicesolution involves identifying the optimal stoppingtime—the number of trials to sample arm 1 beforedeciding to select one or the other arm in perpetuity.In the context of a multiarm bandit (with N unknownarms), the solution is not a stopping rule; rather,it is a dynamic allocation index (Gittins index) thatdefines the optimal arm choice in any given period(Gittins 1979).

Although the optimal strategy described by theGittins index is sufficiently computationally inten-sive such that it is intractable except under a set ofrestrictive assumptions (Gans et al. 2007), it is worthpointing out its central characteristic. The “index foran arm is equal to the expected immediate rewardfrom the arm, with an upward adjustment reflect-ing any uncertainty about the prospects of obtainingrewards from the arm, and the possibilities of resolv-ing those uncertainties by selecting that arm” (Gittinsand Wang 1992, p. 1625). The index suggests a patternof choice over time that is sensitive to both the meanof beliefs and the uncertainty surrounding that mean.

Behavioral approaches that seek to identify an orga-nizationally plausible, although not necessarily opti-mal, strategy date back to Schmalensee (1975) andRadner and Rothschild (1975). Although there is awide variety of plausible strategies, perhaps the sim-plest and best-known strategy is that of selecting, ateach point in time, the alternative with the highestbelief, max4q11 0 0 0 1 qN 5, reflecting the highest expectedreward (Auer et al. 2002, Sutton and Barto 1998). Thisrule is referred to as greedy because at each point intime it myopically maximizes the expected reward inthe next period. The greedy rule always exploits cur-rent knowledge; it never explores alternatives that areapparently inferior but may in fact be superior.

As Brezzi and Lai (2002, p. 105) note in their dis-cussion of behavioral approximations to the Gittinsindex, a good rule “introduces adjustments into themyopic (greedy) rule so that some active experi-mentation is used to generate information about theunknown parameters.” Thus, a simple variation onthe greedy rule is to behave greedily a majority of

the time but with some small probability � select analternative at random, independent of beliefs (Suttonand Barto 1998). This rule is typically referred to as�-greedy. However, the �-greedy rule has an impor-tant undesirable feature. The rule is both overly sen-sitive to beliefs and completely insensitive to beliefs.When the organization exploits (1 − � percent ofthe time), it chooses the alternative with the highestbelief, even if that differential to the second highestbelief is trivially small. In contrast, when it explores(� percent of the time), it is equally likely to selectamong all alternatives, independent of beliefs aboutthe merits of a particular alternative. As such, inexploring, the �-greedy rule is equally likely to pickthe worst alternative as it is to pick the next-bestalternative.

A well-defined and examined strategy that ad-dresses the above deficiency is found in the softmaxchoice rule attributable to Luce (1959) and employedwidely (Camerer and Ho 1999, Gans et al. 2007, Sut-ton and Barto 1998, Vermorel and Mohri 2005, Weberet al. 2004). This strategy formulation takes the formof random choice based on a Gibbs (Boltzmann) dis-tribution. Here, the probability of selecting alternativei, mi, is defined as

mi =e4qi/4�/1055

∑Ni=1 e

4qi/4�/10551 (1)

where qi is the belief regarding alternative i, and� is a parameter reflecting the organization’s strategy.4

A strategy, � , maps the organization’s beliefs Q =

6q11 0 0 0 1 qN 7 to choice probabilities M = 6m11 0 0 0 1mN 7where

∑Ni=1 mi = 1. When � → 0 even small differences

in beliefs are weighted heavily. Thus, if one alterna-tive is viewed even marginally more favorably thanare the others, it will be selected with near certainty,and thus we observe a very greedy strategy (highlyexploitative). For example, consider a three-armedbandit with beliefs across arms of Q = 6007100810077.A purely exploitive strategy would map this to a setof choice probabilities of M = 6000110010007 so thatthe firm chooses with probability one the alternativewith the highest beliefs. In contrast, when � → �,differences in beliefs are shrunk to the point thatthe organization is indifferent between alternatives(even if it believes that one alternative is far supe-rior). This leads to exploration as a random choiceamong all alternatives, where each alternative is cho-sen with probability 1/N . Continuing the example,at very high � , the beliefs Q = 6007100810077 wouldmap to the choice probabilities M = 6003310033100337.In such a setting, the organization would be insen-sitive to the fact that its current beliefs suggest that

4 Note that the “/10” in Equation (1) acts simply as a scaling factor.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


alternative two is superior and with probability 0.66would choose an alternative other than the one cur-rently believed best. At moderate levels of � , alterna-tives that appear to be good (high beliefs) are morelikely to be sampled. At intermediate levels of � , alter-natives that appear to be good (high beliefs) are morelikely to be selected.5

Before proceeding with the analysis, we wish tohighlight the relationship between the bandit modeland that of the genetic algorithm model in March(1991). We focus on two key features of strategy inthese models: (a) the mechanism governing organi-zational choice and (b) the meaning of exploration.First, March’s (1991) genetic algorithm and the banditmodel are fundamentally concerned with the trade-off between exploitation of existing knowledge andexploring for new knowledge in the process of orga-nizational learning. Nevertheless, the level of anal-ysis is different. March (1991) generates differencesin the balance between exploration and exploitationby altering internal characteristics of the organization.The strategy features of his model are tunable param-eters reflecting turnover and learning rate at the levelof the individual. In our bandit model, a strategy �encapsulates these internal organizational dynamicsin a single parameter reflecting the sensitivity of orga-nizations to their beliefs. Yet regardless of the level atwhich the tunable strategy parameter is implemented,the implications for tuning the explore-exploit trade-off remain consistent.

Second, March (1991) argues that exploration notonly suggests a willingness to forgo the opportu-nity to engage in an action that one believes supe-rior, but also to privilege alternatives with less certainbeliefs. A search strategy in the bandit model impliesa similar understanding of the notion of explorationbecause choice must focus on the mean value of analternative with “an upward adjustment reflectingany uncertainty” (Gittins and Wang 1992, p. 1625). Itis less obvious that the softmax formulation of strat-egy in our bandit model, although explicitly onlyincorporating the first moment of beliefs (qi), endoge-nously weights the mean belief by uncertainty. Wediscuss the characteristic of our bandit model in theanalysis of the baseline characteristics of the model inthe subsequent section.

3. AnalysisWe use the multiarmed bandit model described aboveas an analytical tool to study the efficacy of alterna-tive adaptation strategies under turbulence. The struc-ture of the simulation requires initializing two sets

5 Organizations are assumed to be risk neutral. They select analternative solely based on the expected value of an alternative.Although per March (1996) and Le Mens and Denrell (2011), riskaversion emerges endogenously in a model of experiential learning.

of variables associated with (a) the environment andturbulence rate and (b) organizational strategies andbeliefs.

The opportunity structure of the environment isdefined by initializing the payoff probabilities foreach alternative. We formulate a 10-arm bandit modelsuch that the vector of alternatives’ payoff probabil-ities is Pt = 6p11 t1 0 0 0 1 p101 t7. The choice of a particularalternative i leads to an outcome, positive/negativesuch that �i1 t = 41105, with probabilities (pi1 t11 − pi1 t)and an associated reward of ri1 t = 411−15. Each alter-native is allocated an initial payoff probability, pi10,that is taken to be a draw from a beta distribution(�= 2 and �= 2). This produces a distribution of pay-off probabilities (across alternatives) that is symmetri-cal with mean of 0.50 and standard deviation of 0.22.The distribution resembles in shape a normal distri-bution but has a distribution over probability valuesthat it is bounded between zero and one.6

Models incorporating turbulence employ a proba-bilistic shock in a given period where the probabilityof a shock, �, ranges from 0.0 to 0.32. When � = 0,the environment is stable, and Pt = P . For � > 0, con-ditional on the occurrence of a shock, the payoff toeach of the alternatives is reset with probability 0.5. Toreset an alternative, we redraw the payoff, pi1 t , fromthe same initial beta distribution (� = 2 and � = 2).As a consequence, such turbulence is neutral in thesense that the expected value of the set of alternativesremains unchanged after a shock, whereas the payoffto individual alternatives change.

All simulations are seeded with 25,000 organiza-tions, each facing a unique environment with alter-natives drawn from a beta distribution (� = 2 and� = 2). We set the initial asset stock to zero andrun the simulation for 500 periods. We specify initialbeliefs across these alternatives, Q0 = 6q1101 0 0 0 1 q10107,to be homogeneous and set the value to 0.5, whichis equal to the mean value of the actual distri-bution of payoffs. The organizations are dividedequally among five different strategies. The strate-gies, � , reflect different balances between explorationand exploitation. The strategies implemented, whichrange from pure exploitation to high exploration, are� = 40001002510051007511005.7

6 We examined alternative initial arm distributions. In particular,we reran the models in this paper with arms drawn from a uniformdistribution. The uniform is a special case of the beta distributionwith (�= 1 and �= 1). The results are qualitatively the same.7 Equation (1) does not permit the use of a value of zero forthe strategy. As � approaches zero, the value of Equation (1)approaches infinity. Therefore, we set as a pure exploitation strat-egy the computationally feasible value of � = 0002. This value issufficiently close to zero that the strategy reflects pure exploitationin the sense that the alternative with the highest expected payoffis chosen in all runs. As such, the figures and discussion in theremainder of the paper simply refer to this as � = 0.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


The analysis proceeds over three experiments inwhich we examine the performance implications ofalternative strategies of balancing exploration andexploitation across different environmental contexts:(1) stable world, (2) turbulent world (mean neutral),and (3) turbulent world (nonneutral: mean devaluingor mean enhancing).

3.1. Experiment 1: Adaptation Strategies in aStable World

We begin our analysis by examining the bandit modelin a stable environment. Our objective is to exam-ine the basic properties of the model and to demon-strate that it generates results that are consistent withthe prior literature, in terms of both the March (1991)model and also the broader bandit literature.

In Figure 1, we examine the implications of alter-ing adaptation strategy, � , across three measures:(a) exploratory choice, (b) performance, and (c) knowl-edge generation. Each is measured at the end of thesimulation. First, the dashed line (right axis) repre-sents the extent to which the choices taken by orga-nizations are exploratory, measured as the probabil-ity of choosing a different alternative in period t + 1than was chosen in period t. As � increases, the levelof exploration increases. At the most exploratory levelof search strategy in our model, � = 1, organizationsselect a new alternative roughly every second period.8

Second, the solid line (left axis) represents perfor-mance measured as the accumulated asset stock. Thegraph highlights that the best strategy, which displaysa moderate level of exploration (� = 005), accumu-lates on average 280 units, which reflects an averagefrequency of positive payoffs across periods of 78%.Third, the dotted line (right axis) reflects the accuracyof knowledge embodied in firms’ subjective beliefsabout the value of alternatives at the end of the finalperiod. We measure knowledge as one minus the sumof squared error of beliefs relative to the true armpayoffs.9 The graph highlights that the level of knowl-edge obtained is increasing with the extent of explo-ration, from 0.55 at � = 0 to 0.72 at � = 1.

8 We also examined alternative metrics of exploration. For example,the probability of not choosing the arm that is believed to offerthe highest payoff. The qualitative results on exploration behaviordo not change with this, or other, alternative measures considered.9 This measure of knowledge can be interpreted as reflecting anexplicit comparison to the level of knowledge that would beobtained by a firm with beliefs chosen randomly. Knowledge iszero if beliefs are set randomly from a beta 42125 distributionmatching reality, and knowledge is one if beliefs perfectly reflectreality such that SSE = 0. Thus, this measure of knowledge hasthe attractive property that knowledge is greater than zero whenthe organization has knowledge above that of an organizationwith random beliefs. Organizations pursuing a search strategy thatembodies a moderate degree of adaptiveness, � = 005, accumu-late, on average, knowledge over the 500 periods of the simulationof 0.59.

Figure 1 Choice, Performance, and Knowledge Across Strategies inStable Environment

00.10.20.30.40.50.60.70.8

210220230240250260270280290

0 0.25 0.50 0.75 1.00

Fra

ctio

n

Per

form

ance

Strategy (�)

Performance Choice (prob. exploring) Knowledge

Taken together, the inverse U-shaped performancecurve across strategies and the near linear increase inknowledge demonstrates the trade-off between gener-ating new knowledge and leveraging existing knowl-edge. At low levels of exploration, little knowledgeis attained and there is little knowledge to exploit.In contrast, at high levels of exploration, significantknowledge is attained, but it is never exploited toenhance performance. Thus, in this static environ-ment, the results reflect March’s (1991, p. 71) conclu-sion that organizations that “engage in exploration tothe exclusion of exploitation are likely to find thatthey suffer the costs of experimentation without gain-ing many of its benefits,” whereas organizations that“engage in exploitation to the exclusion of explorationare likely to find themselves trapped in suboptimalstable equilibrium.”

Although our model replicates the main results ofthe March (1991) model, we demonstrate that that italso replicates key results in the bandit literature. Wefocus on two features of the model: (a) identifying theextent to which exploration in our model embodieschoice based on uncertainty in beliefs and (b) dock-ing our results to a bandit model with an �-greedystrategy.

Gittins and Wang (1992, p. 1625) note that in select-ing an alternative, it is appropriate to consider the“expected immediate reward from the arm, with anupward adjustment reflecting any uncertainty aboutthe prospects of obtaining rewards from the arm, andthe possibilities of resolving those uncertainties byselecting that arm.” This intuition, which arises fromthe Gittins index, is consistent with the notion ofexploration in March (1991) and the broader manage-ment literature. Nevertheless, although the softmaxchoice strategy we employ considers only the meanof beliefs, neglecting direct consideration of uncer-tainty, we demonstrate that it endogenously weightsthe level of uncertainty in its choice behavior.

In the bandit literature, strategy is defined asgreedy if, in each period, it chooses the alternative

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


Figure 2 Exploitation: Choice on the Basis of Mean and Experience

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 0.25 0.50 0.75 1.00

Fra

ctio

n gr

eedy

Strategy (�)

Greedy on experienceGreedy on mean

with the highest mean belief. Analogously, we con-struct a measure of greediness based on experiencewith an alternative (as a proxy for uncertainty) ratherthan on the first moment of beliefs. Greedy on expe-rience measures the probability that an organizationwill choose an alternative with which it has the mostexperience. The results of these measures, at period500, are presented in Figure 2. As exploration strategy,� , increases, organizations become less greedy, on thebasis both of mean and of uncertainty in beliefs. Assuch, organizations are more likely to explore alterna-tives that are presently believed to be inferior and forwhich the beliefs are relatively imprecise.

The intuition behind this result rests on the obser-vation that alternatives that appear to be good aremore likely to be sampled in the process of experien-tial learning (March 1996, Denrell and March 2001).Moreover, the more an alternative is sampled, themore certain the estimate of its mean value. As aresult, there is a positive correlation between themean belief regarding the payoff to a given alterna-tive and the level of experience with that alterna-tive. When an organization chooses a low mean beliefalternative, it is also likely to select an alternative thatis known with less certainty. Exploration strategies(high �) are not only more likely to forgo alterna-tives with high mean belief expectations, they are alsolikely to forgo more certain alternatives in favor ofless certain alternatives.

For the purposes of model docking, we rerunour simulation employing a simple �-greedy strat-egy, rather than our softmax strategy. This enablescomparisons across the two choice mechanisms. Weexpect, and observe, that as � → 0, the performanceresults converge to the greedy strategy with � = 0.Moreover, as expected from the above discussion ofthe characteristics of the softmax choice rule, it per-forms somewhat better than the �-greedy choice rule.Additionally, Sutton and Barto (1998, p. 41) examinean � = 001 greedy strategy on a 10-arm Gaussian(rather than Bernoulli) bandit model and measure thepercentage of optimal action. This measure reflects

the probability that the alternative chosen in a givenperiod is the alternative with the highest payoff prob-ability. We examine this metric at period 500. In ourimplementation, both the � = 0 and � = 0 strate-gies exhibit the same optimal action result of 39%.With � = 001, our model exhibits 63% optimal action,consistent with the Sutton and Barto (1998) result.Thus, the softmax strategy of our bandit model docksto the �-greedy variant of the bandit, and our �-greedy implementation docks to that of Sutton andBarto (1998).

In sum, in a static environment, the properties ofthe bandit model are consistent with March’s (1991)claim that superior performance arises from a strategythat balances exploration and exploitation.

3.2. Experiment 2: Adaptation in aChanging World

In the second experiment, we add turbulence to themodel, at levels ranging from very infrequent to nearcontinuous (0–32%).10 We examine how and whyturbulence alters the optimal level of adaptivenessembodied in a search strategy.

The main results of the turbulence model aredepicted in Figure 3. This figure plots the opti-mal search strategy calculated by fitting a thirdorder polynomial to the results (that is, Figure 1replicated across levels of turbulence) and extract-ing the strategy, � , associated with that peak perfor-mance (maximum stock of accumulated assets). Thezero turbulence result reflects the conclusions fromExperiment 1: Peak performance occurs at a searchstrategy of � = 0056, reflecting a moderate degree ofadaptiveness. As the probabilistic frequency of tur-bulence increases, the optimal level of explorationtakes an inverse U-shaped form. The curve is highlyskewed in the sense that over most of the range ofturbulence examined, the optimal degree of adaptive-ness embodied in a search strategy is decreasing withincreasing frequency of turbulence.

In addition to altering the optimal level of explo-ration strategy, turbulence may also alter the extentof the cost associated with failing to match the opti-mal exploration strategy. To examine this issue, wecalculate the flatness of the performance versus �curves by plotting the second derivative of perfor-mance at the optimal � value (see Figure 3). If thecurve is sufficiently flat, because little or no knowl-edge can be accumulated via experiential learning,then the distinction between exploration and exploita-tion becomes less relevant. Over the interesting rangeof turbulence (0–8%), the curve is significantly con-cave. At very low levels of turbulence (below 0.5%),

10 At higher levels of turbulence, performance becomes invariant toexploration strategy.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


Figure 3 Optimal Exploration Strategy 4� 5 Across Turbulence Levels

–300

–250

–200

–150

–100

–50

0

0.300.350.400.450.500.550.600.650.70

0 0.5 1 2 4 8 16 32

2nd

deriv

ativ

e

Str

ateg

y (�

)

Turbulence

Optimal2nd deriv.@ optimal

the second derivative of performance at the optimal� value is increasingly negative, implying the perfor-mance versus � curve is increasingly concave at theoptimal � . This suggests that increasing turbulenceenhances the rewards to choosing the optimal � . Asturbulence increases further (at levels above 0.5%), thesecond derivative starts to increase (becomes less neg-ative), implying that the performance versus � curveat the optimal � is becoming flatter. This implies thatidentifying the optimal strategy becomes less impor-tant for performance at higher levels of turbulence.11

Because the discussion of turbulence and explo-ration in March (1991) is so prominent in the lit-erature, it is worthwhile to directly compare theseresults to his. In contrast to March (1991), the resultsin Figure 3 suggest a contingency in the relation-ship between turbulence and the level of exploration.However, March (1991) examines only a single, rel-atively modest level of turbulence (2%). His results,like our own, show that organizational performanceis improved by increasing the level of exploration asturbulence increases from 0 to 2%. Like March (1991),we find that under conditions of very infrequent envi-ronmental change, a strategy stressing adaptation viaexploration and new knowledge accretion is optimal;but as change becomes more frequent, the optimalresponse is, in fact, one of greater inertia.

We engage in additional analysis to understand themechanisms driving this response to turbulence. First,we focus on the tension between the obsolescence ofexisting knowledge and the accretion of new knowl-edge. Second, we focus on the impact of turbulenceon the organizations’ strength of opinions surround-ing its beliefs about the relative merits of alternatives.We examine each mechanism in turn below.

Our working hypothesis, addressing the tensionbetween knowledge obsolescence and accretion, is

11 Even a flat curve performance versus � curve does not imply thatan organization should be indifferent between the levels of explo-ration strategy. In our model, exploration and exploitation have noexplicit cost (only opportunity costs). If, in contrast to our model,exploration has a direct cost that is higher than exploitation, thenerring on the side of too little exploration is superior to erring onthe side of too much exploration under high levels of turbulence.

Figure 4 Knowledge Devaluation Across Turbulence Levels

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Kno

wle

dge

Turbulence

Knowledge @ t = 400Knowledge @ t = 500 (w/t = 400 beliefs)Knowledge @ t = 500 (w/updated beliefs)

0 0.5 1 2 4 8 16 32

Note. Holding strategy fixed 4� = 0055.

that the characterization of preferred search strategiesin Figure 3 arises because turbulence erodes knowl-edge asymmetrically. That is, turbulence acts to erodeperformance via two knowledge mechanisms. First,it reduces the future value of existing knowledgebecause it changes the underlying state of the world.Second, turbulence reduces the value of efforts to gen-erate new knowledge because the lifespan of returnsto new knowledge is reduced in a world in whichchange is more frequent. We examine the possibilityof an asymmetry in the effect of turbulence on thesetwo mechanisms in Figures 4 and 5.

We engage in a differences-in-differences analysis ofknowledge levels. Holding beliefs constant at period400 levels, we examine the resulting level of knowl-edge erosion over the subsequent 100 periods. Wethen compare the period 400 knowledge results tothe period 500 knowledge results (assuming constantbeliefs), and the period 500 results assuming learn-ing at a strategy of moderate adaptation, � = 005. Thedashed line in Figure 4 reflects the level of knowl-edge at period 400, across levels of turbulence. In theabsence of turbulence, the knowledge level is 0.59 (asper the analysis in Experiment 1). By the time tur-bulence reaches 8%, knowledge is reduced to 0.44.The solid line presents the results of using period 400beliefs through period 500 (that is, period 400 beliefsare used but not updated). As such, the difference

Figure 5 Knowledge Accretion and Erosion Across Turbulence Levels

–0.08

–0.06

–0.04

–0.02

0

0.02

0.04

0.06

0

Cha

nge

in k

now

ledg

e le

vel

Turbulence

0 0.5 1 2 4 8 16 32

Accretion of new knowledgeErosion of existing knowledgeNet accretion and erosion


INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


between the solid and dashed lines reflects the ero-sion of existing knowledge over the 100-period timespan. Finally, the dotted line reflects knowledge levelsat period 500 if learning (updating of beliefs) contin-ued through to period 500. The difference between thesolid and dotted lines reflects the improved knowl-edge associated with efforts at new knowledge accre-tion over a 100-period time span.

In Figure 5, the solid and dashed lines reflect,respectively, (a) existing knowledge that is eroded byturbulence (the difference between the dashed andsolid lines in Figure 4) and (b) new knowledge accre-tion (the difference between the solid and dotted linesin Figure 4). The dotted line in Figure 5 depicts thenet knowledge change.

Over the range from no turbulence through a fre-quency of 1%, existing knowledge is eroded rapidly.The more rapidly existing knowledge is eroded, otherthings being equal, the greater the impetus to explore.The alternative to employing additional efforts atexploration is using existing knowledge. As such, theerosion of existing knowledge defines the opportu-nity cost of exploring—which is decreasing rapidlyover this range. At the same time, additional explo-ration is generating new knowledge at an increasingrate. With decreasing opportunity cost and increas-ing generation of new knowledge, there is a sub-stantial net knowledge benefit to exploring (positiveslope of the dotted line), and as such the appropriateresponse is one of increased exploration at low levelsof turbulence.

Over the range of turbulence from 1% to 2%, theexisting knowledge erosion level is flattening and,indeed, slightly decreasing. Relative to this low tur-bulence range, this reflects a substantial decrease inthe rate at which opportunity cost is decreasing. Atthe same time, the rate of new knowledge accretionis increasing. However, the increase in the rate ofnew knowledge accretion is substantially less thanthe decrease in the rate of existing knowledge erosion(and increase in opportunity cost). Thus, the appro-priate response, as reflected by the negative slope ofthe net knowledge change curve, is one of decreasedexploration over the 1%–2% turbulence levels.

Over the range of turbulence in excess of 2%,existing knowledge continues to erode less strongly(opportunity cost is increasing), and new knowledgeaccretion levels flatten, then decline. As a result, theappropriate response at higher levels of turbulence isone of further decreases in exploration. At the veryhighest levels of turbulence, there is little knowledgeavailable to erode, and very limited potential for newknowledge accretion.

In sum, the appropriate adaptation strategy de-pends on the effect of turbulence on both the payoff

to utilizing current knowledge and the payoff to gen-erating new knowledge. The value of current knowl-edge reflects the possibilities of continuing to useexisting knowledge into the future and thereby drivesthe opportunity costs associated with engaging inadditional exploration. However, turbulence does notsimply alter the value of existing knowledge. It alsoalters the payoff to efforts at gathering new knowl-edge. The effect of increasing turbulence dependscrucially on the relative consequences of turbulencealong these two dimensions.

The second mechanism underlying the results inthis experiment is that turbulence endogenously givesrise to an action bias, even when the exploration strat-egy itself is fixed. In our discussion of organizations’beliefs, we have focused on the extent these subjec-tive beliefs embody knowledge. In addition, there isanother important dimension of beliefs—the strengthof opinions. The strength of opinions reflects howstrongly a firm believes one alternative is superior toother alternatives. We measure this as the varianceacross an organization’s beliefs. At the start of thesimulation, firms have homogeneous beliefs about thealternatives; they simply believe all alternatives havea 50% payoff probability and, thus, the variance inbeliefs across alternatives is zero. However, organi-zations quickly start to form divergent beliefs aboutthese alternatives.

In Figure 6, we plot, at period 500, the accuracy ofknowledge, strength of opinions, and extent of explo-ration across the range of turbulence for the � = 005strategy. The dotted line highlights the devaluationof knowledge with increasing turbulence. The dashedline plots the strength of opinions (variance acrossbeliefs). The variance in beliefs across alternatives isreduced by nearly one-third as we move from a sta-ble world to one in which turbulence is set at 8%. Itappears turbulence acts not only to devalue knowl-edge but also to homogenize beliefs.

The reduction in the strength of opinions in theface of turbulence has important consequences for

Figure 6 Knowledge, Opinions, and Choices Across TurbulenceLevels

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

Turbulence0 0.5 1 2 4 8 16 32

Knowledge Strength of opinions

Choice (prob. exploring)


INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


the nature of organizational choice. As the strengthof opinions is reduced, organizations endogenouslyincrease the extent to which their choices areexploratory—even holding search strategy constant.The mechanism driving this action bias exists becausein the experiential learning process, in makingchoices, organizations are sensitive to their beliefsabout the merits of the alternatives available to them.Organizations exploit a given policy choice when theybelieve it is markedly superior to others. Environmen-tal change naturally undermines existing beliefs; alter-natives that were believed to be rather good are likelyto seem worse. Moreover, there is the possibility thatless-examined alternatives may have gotten better. Asthe variance across alternatives in beliefs is reducedby turbulence, organizations have fewer bases onwhich to differentiate among alternatives. As a con-sequence of this compression in beliefs, organizationsexhibit an action bias in dynamic environments.

In sum, turbulence acts to alter the optimal level ofadaptation by altering the returns to utilizing existingknowledge and generating new knowledge. Yet as wedemonstrate, turbulence does not necessarily give riseto a requisite shift toward a more exploratory strategy.Indeed, as the level of turbulence increases, the appro-priate balance between exploitation and explorationshifts in the direction of exploitation. In addition,we note a second mechanism at work—turbulencetends to endogenously generate exploratory behavioreven in the absence of an overt change in adaptationstrategy. In the next experiment, we examine furtherimplications of this action bias.

3.3. Experiment 3: Adaptation Strategies UnderNonneutral Turbulence

Turbulence alters the appropriate adaptation strat-egy because it alters organizational knowledge. In theimplementation of environmental change in Experi-ment 2, turbulence reflects a shock to the payoffs tothe alternatives, but the shock is neutral. That is, thepreshock and postshock distribution of payoffs acrossalternatives is drawn from the same beta distribution,and as such the expected value of the set of alter-natives is held constant at 0.5. In this experiment,we examine turbulence that is nonneutral in that itmakes the world more (or less) munificent (increasesor decreases the expected value of the set of alterna-tives). We hold turbulence at the level of a 5% proba-bility of a shock and examine turbulence that changesthe mean payoff values (cumulatively) over the rangefrom +/− 20% of the neutral turbulence level.12

12 We do so by shifting the parameters of the beta distribution.These shocks are cumulative in that a subsequent shock resetsthe payoff probability of an alternative from a progressively worse(better) distribution.

To illustrate munificence-altering turbulence, con-sider the emergence of (sufficiently fast) flash-basedcomputer memory and the consequent changes in thedigital data storage industry. This change reduced thepayoff to producing magnetic spinning disks. It alsoenhanced the returns to a wide variety of applicationsthat were, in the past, deemed to be inferior. Thesenewly enhanced choices include not only solid-statedisk drives as a direct replacement for spinning harddrives but also alternative applications of long-termstorage outside the realm of the traditional PC (e.g.,music player, smart phone, auto). If this turbulencehad been neutral, then to the extent that the returnsto selling spinning hard drives had been reduced, theexpected returns to the alternatives would have beensimilarly enhanced. In this sense, neutral turbulenceshuffles the deck but does not change the expectedreward. Yet the turbulence in the long-term digitalstorage industry was nonneutral. The decrease in theexpected returns to engaging in designing and sell-ing spinning hard drives was more than offset by theenhanced returns to the many new uses for solid-state drives. As such, this turbulence was munificenceenhancing because it not only reshuffled the deck, italso increased the expected return to the set of alter-natives available to organizations in the industry.

Figure 7 displays the optimal search strategy acrossthe range of the munificence of turbulence, hold-ing the level of turbulence constant. Note that thezero percentage point reflects the neutral turbulenceresult from Experiment 2. The main result is thatif turbulence increases the level of munificence, theoptimal level of adaptiveness embodied in a searchstrategy increases. Because we hold the level of turbu-lence constant, the rates of erosion of existing knowl-edge and accretion of new knowledge is held fixed.As such, a different mechanism must be driving thisresult.

To identify the mechanism, we plot in Figure 8 thestrength of organizations’ opinions (dashed line) andthe level of exploration (solid line) across the lev-els of the munificence of turbulence. With the search

Figure 7 Optimal Exploration Strategy 4� 5 Across Munificence ofTurbulence

0.400.420.440.460.480.500.520.540.560.580.60

–20 –10 0 10 20

Str

ateg

y (�

)

Munificence of turbulence (%)

Optimal

Note. Turbulence fixed at 5%.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


Figure 8 Opinions and Choices: Across Munificence of Turbulence

0

0.05

0.10

0.15

0.20

0.25

0.30

36

38

40

42

44

46

48

50

–20 –10 0 10 20

Var

ianc

e in

bel

iefs

Pro

babi

lity

ofex

plor

ing

(%)

Munificence of turbulence (%)

Strength of opinionsChoice (prob. exploring)

Note. Turbulence fixed at 5% and strategy fixed at � = 005.

strategy held constant at � = 005, increasing munifi-cence of turbulence increases the strength of opinions.This occurs because the payoff to all alternatives is,on average, increasing with munificence-enhancingturbulence. However, in the process of experientiallearning, organizations are more likely to select analternative that is known to be good (and as suchthe set of alternatives considered tends to be decreas-ing over time). This raises the possibility that theymay fail to realize, on average, all alternatives haveimproved. Thus, in the experiential learning pro-cess, organizations may interpret the enhanced per-formance results as information about the quality ofthe alternative that they currently favor.

This increase in the strength of opinions undermunificence-enhancing turbulence leads to a decreasein the extent of exploration (per the discussion inExperiment 2). But as turbulence, neutral or munifi-cence enhancing, continues to devalue the returns toexisting knowledge and to new knowledge accretionin the same manner, the appropriate level of � has notchanged. Thus, this endogenous decrease in explo-ration requires a countervailing change in adaptationstrategy.

In the face of munificence-enhancing turbulence,organizations are subject to an “inaction bias.” Thisbias is not driven by any inherent behavioral bias butrather by the nature of belief formation in the experi-ential learning process.13 An increased effort at explo-ration (a more exploratory strategy) is needed to over-come the endogenous reduction in exploration thatis associated with the mechanism described above.Indeed, the opposite is true for munificence-reducingturbulence, which tends to exaggerate the bias towardaction.

In sum, this experiment serves to highlight the ob-servation that turbulence devalues knowledge em-bodied in beliefs. This affects not only the return to

13 This outcome is akin to a competence trap. However, insteadof inertia resulting from accumulated competence with a particu-lar alternative, it results from enhanced positive beliefs about theattractiveness of a given alternative in the absence of informationabout other alternatives.

an explicit strategic shift toward exploration, as dis-cussed in Experiment 2, but also the nature of organi-zational action itself. When organizations are engagedin learning, and their choices are sensitive to theirbeliefs regarding the relative merits of alternatives,turbulence can engender organizational action (that isincreasingly exploratory or exploitative) even in theabsence of a shift in exploration strategy. As a result,the appropriate organizational response to turbulenceis sensitive to the nature of turbulence—not only itsseverity but also the extent to which it enhances ordegrades the value of possible organizational actions.

3.4. Sensitivity AnalysisWe examine the sensitivity of our results to (a) alter-native specifications of the key parameters of the ban-dit model, (b) episodic turbulence, and (c) selection.In each case, the results are robust. First, two keyelements of the bandit model have been altered. Weexamine a bandit model in which belief updatingis implemented following a Bayesian methodologyrather than the simple average updating methodol-ogy employed in the discussion above. We also exam-ine a model where the alternatives are drawn from auniform distribution rather than the near-normal beta(� = 2, � = 2) distribution.14 Second, we examine amodel in which turbulence is episodic. In particular,we examine turbulence that occurs in the middle thirdof the 500-period simulation across a wide range ofseverity. We also examine turbulence that occurs in a50-period window, where the window occurs earlieror later in the simulation. Third, we examine a modelin which there is selection. We examine noncompeti-tive selection in which an organization exits when itsasset stock falls below zero. We also examine com-petitive selection, where an organization’s probabilityof failing is a function of its performance relative toother organizations. In each of these additional exper-iments, the results are robust.

4. DiscussionWe formally consider the effect of changing environ-mental conditions on firm search strategy and, in par-ticular, the extent to which an organization shouldpursue a strategy of adaptation under conditions ofenvironmental change. On the surface, the claim thatorganizations should respond to changing circum-stances by generating new knowledge seems obviousand compelling. This logic is reflected in a broaderexhortation to action in both the academic and popu-lar literatures—and its validity deserves examination.Is the exhortation to action in a dynamic environment

14 Note that the uniform distribution is a special case of the betadistribution parameterized as �= 1 and �= 1.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


always appropriate? Our modeling efforts focus atten-tion on the implications of turbulence for organiza-tional beliefs about the relative merits of alternativesand suggest a more circumspect conclusion about thevalue of organizational adaptation.

Our model points to an inverse U-shaped rela-tionship between the optimal degree of adaptivenessembodied in a search strategy and the frequencyof turbulence. This result stems from two oppos-ing mechanisms. On one hand, intuition suggestsenvironmental change devalues existing knowledge.The failure to respond to change in the environmentimplies decreased performance. This mechanism iswell known in the existing theoretical literature andunderlies much of the wisdom regarding the appro-priate response to change. On the other hand, lesswell developed and understood in the literature is amechanism that acts in the opposite direction. Envi-ronmental change erodes the rewards to exploratoryefforts at accumulating new knowledge. Our analysissuggests that the delicate balance between these twomechanisms, and the possibility of their asymmetri-cal response to turbulence, determines the appropri-ate organizational response to environmental change.

Environmental change has a second, arguably moresubtle, implication. We conceive of strategies asreflecting managerial and organizational attempts tounderstand the world and act appropriately. Thisview is reflected in the bandit model where a strategyis a function that maps knowledge about the worldonto particular choices. Thus, a search strategy doesnot determine a specific action but rather action con-ditional on beliefs about the state of the world. For agiven search strategy, belief structures that are morehighly skewed, that place greater weight on somealternatives and lesser weight on others, will gener-ate choices that are more exploitive. By implication,factors that alter organizational beliefs may lead tochanges in choice behavior even if the search strategyis fixed and unchanging.

When an organization is involved in a process ofexperiential learning, environmental change will tendnot only to erode the extent of knowledge embod-ied in beliefs but also lead to more diffuse, lessdistinct beliefs about the relative merits of alterna-tives. Once-attractive alternatives are likely to becomeless appealing; as a result, once-positive beliefs aboutthese initiatives will tend to be driven lower withrepeated experience. By the same token, should theorganization happen to explore alternatives it previ-ously viewed as less attractive, it may find they haveimproved in the intervening periods. This homoge-nization of beliefs that arises in the experiential learn-ing process under conditions of turbulence leads toan action bias.

Consider the case of adaptation to extreme turbu-lence in the auto industry. In the post-2007 period,oil prices rose markedly and the U.S. housing mar-ket collapsed. This led to rapid changes in consumerpreferences; they shifted from large, heavy, all-wheeldrive vehicles to smaller, lighter vehicles with bettergas mileage.

The taken-for-granted strategy during this periodof turbulence was that of significant action aimedat adaptation. Existing knowledge about designingand building large, heavy, and powerful vehicleswas devalued. The response entailed altering the bal-ance between exploitation and exploration by shift-ing resources toward the development of less wellunderstood technologies (e.g., electric vehicles or lightweight carbon fiber bodies), which were expected tobe a better fit with the new environment.

This strategic response is reflected in the GeneralMotors (GM) effort to bring the all-electric Volt tomarket in the post-2007 period. To allocate resourcesto the Volt, GM reduced efforts to exploit existingknowledge (cutting, at the margin, investment in mid-size fuel efficient vehicles; Welch 2008). The processof developing this highly exploratory technology wasvery slow not because of organizational impedimentsto change but rather because of the limited rate ofnew knowledge accretion that has been an ongoingfeature of this technology domain (batteries in par-ticular). By the time the Volt reached the market in2010, oil prices had dropped to the point where thevehicle was no longer economically viable, and anongoing recession made consumers highly price sen-sitive (Blanco 2011). Thus, ongoing turbulence haddevalued not only GM’s return to deploying existingknowledge and technologies (embodied in large sportutility vehicles) but also the near-term returns to itsexploratory investment in an electric vehicle.15

Although the organizational impetus to act wasstrong, because this turbulence was seen as munif-icence reducing, the returns to exploring weredevalued by ongoing turbulence. Yet there was analternative course of action in the face of this envi-ronmental change. Ford, for example, chose to reduceits efforts at exploration, allocating a larger share ofresources to exploitation. It implemented a strategythat focused more effort on refining existing technolo-gies and designs, for example, deploying gas-electrichybrid technology in key models and modifying thehighly successful European Fiesta for the U.S. market.As CNN reported in January 2009, “In the near term,the carmakers’ [Ford’s] so-called EcoBoost engines—turbocharged engines with highly sophisticated fuel

15 Although subsequent increases in fuel prices may render GM’sinvestments worthwhile, the key point is that the possibility offuture turbulence reduces the expected returns from exploratoryinvestment.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


injection systems—will provide the greater fuel econ-omy Americans want as gas prices rise gradually”(Valdes-Dapena 2009). Ford did not forgo explorationin its entirety, continuing an investment program inelectric vehicle technologies; but whereas GM mas-sively shifted resources toward exploration in the faceof extreme turbulence, Ford shifted resources, at themargin, toward exploitation.

Our theory suggests that the appropriateness ofsuch a strategy is a function of the extent to whichexisting knowledge is devalued by turbulence aswell as the extent to which ongoing turbulence maydevalue investments in exploration aimed at the gen-eration of new knowledge. The challenge for organi-zations is the ex ante assessment of the magnitudes ofthese two competing forces. Organizations should beaware of the bias to act and view the decision to shiftthe exploit or explore balance toward exploration asa strategic choice rather than a strategic imperative.

Our theory, that there are conditions under whichthe appropriate response to environmental change isto heighten the degree of inertia, is broadly consistentwith arguments within the population ecology litera-ture that question the plasticity of organizations. Butthe mechanism in our theory is quite different. Pop-ulation ecology holds that organizations are subjectto inertial pressures that may arise from embeddedorganizational structures, routines, and processes thatinhibit adaptation (Hannan and Freeman 1984). As aresult, inertia may not only be a salient property oforganizations but also may be performance enhancingin the face of environmental turbulence.

Whereas population ecology questions the feasibil-ity of organizational adaptation, our theory questionsthe desirability of adaptation and points to the con-straints on beliefs and their revision given some fixedset of experiences. This distinction between structuralinertia, on one hand, and cognitive limitations on therate of learning, on the other, is important becausethe two problems necessitate different solutions. Ifthe limiting factor is structure, then the managerialimplications are related to designing a more flexi-ble organization, one amenable to modification com-mensurate with the new state of the world (Daviset al. 2009). For example, Brown and Eisenhardt (1997,p. 1) argue for a response that “blends limited struc-ture around responsibilities and priorities with exten-sive communication and design freedom to createimprovisation within current projects.” If, in contrast,the limiting factor is a bound on the rate of learn-ing, then the managerial implications are related toactions that enable organizations to more rapidlyacquire and effectively process new information.16 For

16 Of course, there may be some relationship between interventionsthat address the forces of structural inertia and those that facilitatelearning.

example, this may take the form of building enhancedknowledge management capabilities through addi-tional efforts at environmental scanning. In technol-ogy industries, this may include corporate venturecapital investments that provide enhanced access toinformation on emerging technologies (Benson andZiedonis 2009). Alternatively, this may take the formof purposeful organizational forgetting, selectivelydiscarding old knowledge in order to foster morerapid accretion of new knowledge (Hatch and Dyer2004, Hedberg et al. 1976). McNamara and Baden-Fuller (1999), in a case study of publicly tradedbiotechnology firms, find that newer firms may bemore adaptable not only because they have moreflexible organizational structures but also becausethey learn more rapidly because prior (inappropriate)knowledge does not hinder the process of new knowl-edge accretion.

Our multiarmed bandit model opens up the pos-sibility of a number of potentially interesting exten-sions. Two extensions are of particular interest. First,one might model competitive interactions across orga-nizations. In the model presented here, organiza-tions are effectively competing independently againsta common environment and subsequently compar-ing their performance. More realistically, organiza-tions compete directly with one another. Two typesof competitive interactions are promising additions toour model. Competition may take the form of com-petitive selection, in which an organization’s probabil-ity of failing is a function of its performance relativeto other organizations. Alternatively, competition maytake the form of competitive performance feedbacksuch that instead of receiving direct feedback on thesuccess or failure of a trial, the organization receivesfeedback that is a function of its own trial success orfailure and also the extent to which others in the pop-ulation were successful in a given period. This wouldmake feedback about the true value of an alternativenoisy and in turn alter both the nature of beliefs andsubsequent choices.

Second, one might model organizational awarenessof environmental change. Organizations in our modelare not directly aware of the existence of environmen-tal change and form no inferences about it. Yet theusefulness of our discussion is predicated on a limitednotion of awareness, in the sense that an organizationmust be sufficiently aware so that it can appropri-ately tune its adaptation strategy. One might considera more complex model in which there is second-orderlearning about the existence, magnitude, and munifi-cence of turbulence. Clearly organizations have beliefsregarding such factors, although there does not seemto be strong evidence that organizations reliably fore-cast such matters.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


Finally, additional extensions include (a) hetero-geneity across organizations in the capability to learn;(b) nonfixed search strategies that respond to eitherthe stock of accumulated resources, the nature ofexisting beliefs, or the level of performance aspi-rations; (c) interdependence across choices; and(d) endogenous environmental change that derivesfrom interdependence across organizations.

These possible extensions provide important oppor-tunities to build on the insights provided here, butwould not negate the basic tensions characterized inthe current work. Our objective is to first and fore-most understand how knowledge is eroded by tur-bulence and the consequences of this erosion for theappropriate level of adaptiveness. Adaptive searchis multifaceted. It is not only a strategy but also aset of enacted choices. Strategies map beliefs aboutthe relative merits of alternatives to enacted choices.Examining the interplay between these factors greatlyenhances our understanding of the dynamics ofexploration, and of exploitation, and the rewardsto organizational adaptation. Further, the mannerin which these factors play out, not surprisingly,depends on the context in which they operate, inparticular the degree to which environments arechanging. However, the nature of these dependen-cies is surprising. Environmental change is not a self-evident call for strategies of greater exploration. Thequestion of the appropriate organizational response toenvironmental change is a central challenge for orga-nizations, and unpacking and understanding this ten-sion remains a fertile and important line of inquiryfor organizational theorists and strategy scholars.

AcknowledgmentsThe authors thank seminar participants at Columbia Uni-versity, Duke University, London Business School, Univer-sity of Michigan, University of Minnesota, University ofNorth Carolina, and the University of Western Ontario forhelpful comments on a prior version. Hart Posen is gratefulfor financial support from the 3M Company.

ReferencesAuer, P., N. Cesa-Bianchi, P. Fischer. 2002. Finite-time analysis of the

multiarmed bandit problem. Machine Learn. 47(2–3) 235–256.Barr, P., J. Stimpert, A. Huff. 1992. Cognitive change, strategic

action, and organizational renewal. Strategic Management J.13(S1) 15–36.

Baum, J., S. Wally. 2003. Strategic decision speed and firm perfor-mance. Strategic Management J. 24(11) 1107–1129.

Benner, M. J. 2009. Dynamic or static capabilities? Process manage-ment practices and response to technological change. J. ProductInnovation Management 26(5) 473–486.

Benson, D., R. Ziedonis. 2009. Corporate venture capital as a win-dow on new technologies: Implications for the performance ofcorporate investors when acquiring startups. Organ. Sci. 20(2)329–351.

Bergemann, D., J. Välimäki. 1996. Learning and strategic pricing.Econometrica 64(5) 1125–1149.

Berry, D. A., B. Fristedt. 1985. Bandit Problems: Sequential Allocationof Experiments. Chapman and Hall, London.

Blanco, S. 2011. Consumer Reports: Chevy Volt “is going to be atough sell to the average consumer.” AutoblogGreen (March 1),http://green.autoblog.com/2011/03/01/consumer-reports-chevy-volt-is-going-to-be-a-tough-sell-to-the/.

Brezzi, M., T. L. Lai. 2002. Optimal learning and experimentationin bandit problems. J. Econom. Dynam. Control 27(1) 87–108.

Brown, S., K. Eisenhardt. 1997. The art of continuous change: Link-ing complexity theory and time-paced evolution in relentlesslyshifting organizations. Admin. Sci. Quart. 42(1) 1–34.

Bush, R., F. Mosteller. 1955. Stochastic Models for Learning. JohnWiley & Sons, New York.

Camerer, C., T. H. Ho. 1999. Experience-weighted attraction learn-ing in normal form games. Econometrica 67(4) 827–874.

Cyert, R., J. March. 1963. A Behavioral Theory of the Firm. Prentice-Hall, Englewood Cliffs, NJ.

D’Aveni, R. A., R. E. Gunther. 1994. Hypercompetition: Managing theDynamics of Strategic Maneuvering. Simon and Schuster, NewYork.

Davis, J., K. Eisenhardt, C. Bingham. 2009. Optimal structure, mar-ket dynamism, and the strategy of simple rules. Admin. Sci.Quart. 54(3) 413–452.

Denrell, J., J. March. 2001. Adaptation as information restriction:The hot stove effect. Organ. Sci. 12(5) 523–538.

Dess, G., D. Beard. 1984. Dimensions of organizational task envi-ronments. Admin. Sci. Quart. 29(1) 52–73.

Gans, N., G. Knox, R. Croson. 2007. Simple models of discretechoice and their performance in bandit experiments. Manufac-turing Service Oper. Management 9(4) 383–408.

Gavetti, G., D. A. Levinthal. 2000. Looking forward and lookingbackward: Cognitive and experiential search. Admin. Sci. Quart.45(1) 113–137.

Gittins, J. C. 1979. Bandit processes and dynamic allocation indices.J. Roy. Statist. Soc.. Ser. B 4Methodological5 41(2) 148–177.

Gittins, J. C., D. M. Jones. 1974. A dynamic allocation index forthe sequential design of experiments. J. Gani, ed. Progress inStatistics. North-Holland, Amsterdam, 241–266.

Gittins, J. C., Y. Wang. 1992. The annals of statistics. Ann. Statist.20(3) 1625–1636.

Gupta, A., K. Smith, C. Shalley. 2006. The interplay between explo-ration and exploitation. Acad. Management J. 49(4) 693–706.

Hannan, M., J. Freeman. 1984. Structural inertia and organizationalchange. Amer. Sociol. Rev. 49(2) 149–164.

Hardwick, J. P., Q. F. Stout. 1992. Bandit strategies for ethicalsequential allocation. Proc. 23rd Sympos. Interface, American Sta-tistical Association, Alexandria, VA, 421–424.

Hatch, N. W., J. H. Dyer. 2004. Human capital and learning as asource of sustainable competitive advantage. Strategic Manage-ment J. 25(12) 1155–1178.

Hedberg, B., C. Nystrom, W. Starbuck. 1976. Camping on seesaws—Prescriptions for a self-designing organization. Admin. Sci.Quart. 21(1) 41–65.

Henderson, R. 1993. Underinvestment and incompetence asresponses to radical innovation: Evidence from the photolitho-graphic alignment equipment industry. RAND J. Econom. 24(2)248–270.

Holland, J. 1975. Adaptation in Natural and Artificial Systems: AnIntroductory Analysis with Applications to Biology, Control & Arti-ficial Intelligence. University of Michigan Press, Ann Arbor.

Keller, G., S. Rady. 1999. Optimal experimentation in a changingenvironment. Rev. Econom. Stud. 66(3) 475–507.

Lant, T., S. Mezias. 1992. An organizational learning model of con-vergence and reorientation. Organ. Sci. 3(1) 47–72.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.


Le Mens, G., J. Denrell. 2011. Rational learning and informationsampling: On the naivety assumption in sampling explanationsof judgment biases. Psychol. Rev. 18(2) 379–392.

Luce, R. 1959. Individual Choice Behavior: A Theoretical Analysis.Wiley, New York.

March, J. G. 1991. Exploration and exploitation in organizationallearning. Organ. Sci. 2(1) 71–87.

March, J. G. 1996. Learning to be risk averse. Psychol. Rev. 103(2)309–319.

March, J. G. 2003. Understanding organizational adaptation. Soc.Econom. 25(1) 1–10.

March, J. G. 2010. The Ambiguities of Experience. Cornell UniversityPress, Ithaca, NY.

McNamara, P., C. Baden-Fuller. 1999. Lessons from the Celltechcase: Balancing knowledge exploration and exploitation inorganizational renewal. British J. Management 10(4) 291–307.

Miller, D. 1987. The structural and environmental correlates of busi-ness strategy. Strategic Management J. 8(1) 55–76.

Nelson, R. 2008. Bounded rationality, cognitive maps, and trial anderror learning. J. Econom. Behav. Organ. 67(1) 78–89.

Nelson, R., S. G. Winter. 1982. An Evolutionary Theory of EconomicChange. Belknap Press, Cambridge, MA.

Peters, T. J., R. H. Waterman. 1982. In Search of Excellence: Lessonsfrom America’s Best-Run Companies. Harper & Row, New York.

Radner, R., M. Rothschild. 1975. On the allocation of effort. J. Eco-nomic Theory 10(3) 358–376.

Robbins, H. 1952. Some aspects of the sequential design of experi-ments. Bull. Amer. Math. Soc. 58(5) 527–535.

Schmalensee, R. 1975. Alternative models of bandit selection.J. Econom. Theory 10(3) 333–342.

Sorenson, J., T. E. Stuart. 2000. Aging, obsolescence, and organiza-tional innovation. Admin. Sci. Quart. 45(1) 81–112.

Sutton, R. S., A. G. Barto. 1998. Reinforcement Learning: An Introduc-tion. MIT Press, Cambridge, MA.

Teece, D. J., G. Pisano, A. M. Y. Shuen. 1997. Dynamic capabili-ties and strategic management. Strategic Management J. 18(7)509–533.

Tushman, M., E. Romanelli. 1985. Organizational evolution:A metamorphosis model of convergence and reorientation.L. L. Cummings, B. M. Staw, eds. Research in OrganizationalBehavior. JAI Press, Greenwich, CT, 171–222.

Valdes-Dapena, P. 2009. Ford: No Volt for us. CNNMoney(January 13), http://money.cnn.com/2009/01/12/autos/ford_electric_plans/index.htm.

Vermorel, J., M. Mohri. 2005. Multi-armed bandit algorithms andempirical evaluation. J. Gama, R. Camacho, P. Brazdil, A. Jorge,L. Torgo, eds. Machine Learn.: ECML 2005, LNAI 3720. Springer-Verlag, Berlin, 437–448.

Weber, E., S. Shafir, A.-R. Blais. 2004. Predicting risk sensitivity inhumans and lower animals: Risk as variance or coefficient ofvariation. Psych. Rev. 111(2) 430–445.

Weick, K. 1969. The Social Psychology of Organizing. Addison-Wesley,Reading, MA.

Welch, D. 2008. GM: Live green or die. Bloomberg Businessweek(May 15), http://www.businessweek.com/print/magazine/content/08_21/b4085036665789.htm.

Wholey, D., J. Brittain. 1989. Characterizing environmental varia-tion. Acad. Management J. 32(4) 867–882.

Whittle, P. 1988. Restless bandits: Activity allocation in a changingworld. J. Appl. Probability 25 287–298.

INFORMS

holds

copyrightto

this

article

and

distrib

uted

this

copy

asa

courtesy

tothe

author(s).

Add

ition

alinform

ation,

includ

ingrig

htsan

dpe

rmission

policies,

isav

ailableat

http://journa

ls.in

form

s.org/.

Documents

Chasing a Moving Target: Exploitation and Exploration in Dynamic Environments