how to manage risk (after risk failed).pdf

8/14/2019 how to manage risk (after risk failed).pdf

1/9

How to Manage Risk

(After Risk Management

Has Failed)

FALL 2010 VOL . 52 NO .1

R E P R I N T N U M B E R 5 2 1 0 7

Adam Borison and Gregory Hamm

Please note that gray areas reflect artwork that has been

intentionally removed. The substantive content of the ar-

ticle appears as originally published.


2/9

SLOANREVIEW.MIT.EDU FALL 2010 MITSLOAN MANAGEMENT REVIEW 51

IT IS WELL KNOWNthat over the past decade, and especially over the past few years, a number ofthe worlds most widely respected companies have collapsed. Analysts have cited equally well-known

reasons for these collapses the usual suspects of nonviable business models, greed, incompetent

(and overpaid) management and a lax regulatory environment. Not often mentioned is another key

consideration, something that appears to distinguish collapsed companies strongly from their noncol-

lapsed counterparts. It is the breadth and depth of these companies approach to risk management.

That risk management could be a major (though not sole) cause may seem counterintuitive. The

troubled American International Group Inc., for example, was a leader in risk management and even

R I S K M A N A G E M E N T

How to Manage Risk (AfterRisk ManagementHas Failed)The corporate world has traditionally taken a flawed approachto risk management, but a better alternative is readily available.BY ADAM BORISON AND GREGORY HAMM

THE LEADINGQUESTION

What risk-managementapproachshould com-panies adoptto help themavert futurefailures?

FINDINGSThe traditional fre-quentist approachis based entirely onthe historical record.

The alternativeBayesianapproach incorpo-rates judgmentsto complementhistorical data.

The Bayesian per-spective providesmore powerful andaccurate results.

AIGs former CEO boasted thathis company had the best risk-management [departments] inthe damn industry. In whatways was he wrong?
http://www.sloanreview.mit.edu/http://www.sloanreview.mit.edu/


3/9

52 MITSLOAN MANAGEMENT REVIEW FALL 2010 SLOANREVIEW.MIT.EDU


maintained a risk-management subsidiary. Its former

CEO Maurice R. Hank Greenberg boasted that AIG

had the best risk management [departments] in the

damn industry. Bear Stearns Cos. claimed the best-

in-class processes in analyzing and managing risk;even theNew York Timescited the companys carefully

honed reputation for sound risk management. Fannie

Mae, the Federal National Mortgage Association,

touted its excellent credit culture and risk-manage-

ment capabilities, and Lehman Brothers Holdings Inc.

prided itself on what its leaders called a culture of risk

management at every level of the firm.1

Yet at these companies, and at others with com-

parable cultures, risk management apparently

performed quite dismally. How could this be? We

contend that the answer lies in the concepts andpractices of traditional risk management, which tend

to look for risk in all the wrong places. That is, failure

did not stem from merely paying lip service to risk

management or from applying it poorly, as some

have suggested. Instead, collapse resulted from tak-

ing on overly large risks under the seeming security

of a risk-management approach that was in fact

flawed. The more extensive the reliance on tradi-

tional risk management, we believe, the greater the

risks unknowingly taken on and the higher the

chances of corporate disaster.

This article suggests how the key shortcomings

of traditional risk management can be addressed

by adopting a more sophisticated alternative the

Bayesian approach.

Not By History AloneTwo fundamentally different views have evolved

over the years on how risk should be assessed. The

first view termed the objectivist, or frequentist,

view holds that risk is an objective property ofthe physical world and that associated with each

type and level of risk is a true probability, just as

there is a true atomic number for oxygen. Such

probabilities are obtained from repetitive historical

data, with some of the classic examples (largely for

pedagogic purposes) being coin flips, die rolls and

weather patterns. Based on such data, a frequentist

might say that the probability of flipping a seem-

ingly normal coin and gett ing heads, after having

documented the results of a great many tosses, is

0.5; or that the probability of a high temperature of95 degrees on July 4, 2011, in New York, given the

extensive weather record, is 0.3.

The second view is termed the subjectivist, or

Bayesian, view (named after the Reverend Thomas

Bayes, an English mathematician who made major

contributions to this approach during the 18th cen-

tury). Bayesians consider risk to be in part a

judgment of the observer, or a property of the ob-

servation process, and not solely a function of the

physical world. That is, repetitive historical data are

essentially complemented by other information.

Although classic cases such as coin flips come up

largely in the frequentist context, they can also be

used to contrast the frequentist and Bayesian views.

For instance, suppose a magician pulls what ap-

pears to be a normal coin out of her pocket, allows

you to flip it 10 times, and it comes up heads five of

those times. She then proposes a wager based on

your flipping the coin one more time and getting

heads. What probability do you assign to that out-

come? A frequentist presumably relies on the

historical data from this coin (as well as from any

other normal coin) and assigns a probability of 0.5.A Bayesian takes not only the data into account but

also his judgment about the cleverness, trustwor-

thiness and financial situation of the magician. He

may thus assign a probability very different from

0.5 perhaps as high as 1.0. Another observer

might assign an altogether different probability,

based on other judgments.

A similar argument can be made with respect to

weather patterns, even where there is a great deal of

DAILY RETURNS DURING THE FIRST THREE QUARTERS

These daily trading results for a financial company exhibited modest variations during

much of 2008. Would the fourth quarter show a similar pattern?

Q1

$75

$50

$25

$0

($25)

($100)

($75)

($50)

Q2 Q4

Daily P & L ($)

Q3


4/9

FALL 2010 MITSLOAN MANAGEMENT REVIEW 53SLOANREVIEW.MIT.EDU

repetitive historical data. For example, while the re-

cord over many decades may indicate to a

frequentist that the probability of a high tempera-

ture of 95 degrees in New York City on July 4, 2011,

is 0.3, a Bayesian taking an analysis of global warm-ing into account may assign a probability that is

greater than 0.3. In both cases, the historical record

of the physical world is the same, but the different

probabilities reflect dissimilar judgments about the

present and future of that world.

Although the Bayesian view is well accepted in

some circles, it has not penetrated the risk-manage-

ment world. Traditional risk management has

instead adopted the frequentist view, despite its

three inherent, and major, shortcomings. First, it

puts excessive reliance on historical data and per-forms poorly when addressing issues where

historical data are lacking or misleading. Second,

the frequentist view provides little room and no

formal and rigorous role for judgment built on

experience and expertise. And third, it produces a

false sense of security indeed, sometimes a sense

of complacency because it encourages practitio-

ners to believe that their actions reflect scientific

truth. Many of a corporations most important and

riskiest decisions which often do not fall into the

narrow frequentist paradigm are made without

the help of the more sophisticated and comprehen-

sive Bayesian approach.

An Exceptional Fourth QuarterValue at Risk (VaR), arguably the centerpiece of

traditional risk management, provides a good ex-

ample of the limitations of the frequentist view,

particularly its overreliance on historical data. The

basic idea behind VaR is to calculate the potential

loss within a specified time period typically, a

day. Controls then can be put in place to limit this

loss to a desired level. In particular, if a companyhas identified a $15 million daily loss as the maxi-

mum it should tolerate, and the fourth quarter is

about to begin, what is the probability of exceeding

such a loss during that period? (See Daily Returns

During the First Three Quarters.)

Using data from approximately 200 daily trials

during the first three quarters of 2008, a frequentist

represents the range of daily returns as a normal

distribution with a mean of $1 million and a stan-

dard deviation (or volatility) of $5 million. Based

on this, the probability of a daily loss of more than

$15 million is extremely small, well under 0.1%.

(See Probabilities of Daily Returns, Based on the

Record Alone.) It is a less-than-one-in-1,000 event.

And the probability of a daily loss of more than $25

million is infinitesimal.

The Bayesian approach, in contrast, is to look at

the daily return, like all risks, as a matter of judgment

that is informed by but not limited to the repetitive

historical data. A Bayesian explicitly recognizes that

despite three quarters (or more) of data exhibiting a

volatility of $5 million, the future will not necessarily

replicate the past with certainty. For example, there

could be an unusual end-of-year effect or other

(much larger) socioeconomic forces at work.

Although little, if any, repetitive historical data

underlying these broader phenomena may be avail-

able, a Bayesian can quantify his judgment. For

example, the analyst might see two competing phe-

nomena: Deteriorating market conditions could

cause volatility in the fourth quarter to increase, per-

haps double, while newly imposed regulatory policiesmight cause volatility to decrease. Let us say that the

Bayesian assigns a 30% chance that the volatility will

remain unchanged during the fourth quarter, a 30%

chance that it will double to $10 million and a 40%

chance that it will be halved to $2.5 million.

As might be expected, combining this subjective

information with frequency data in a seamless fash-

ion results in an augmented distribution that is

wider than the one based on the data alone. (See

PROBABILITIES OF DAILY RETURNS,

BASED ON THE RECORD ALONE

The first three quarters results are reflected by this normal distribution.

Given the periods record, the probabilities of large losses are very small.

-20

10%

6%

5%

4%

9%

8%

7%

3%

0%

1%

2%

-15 25

Daily Return

-10 -5 20151050


5/9



Modified Probabilities of Daily Returns.) From

this distribution, the Bayesian determines that the

probability of a daily loss of more than $15 million

is roughly 2.0% over 20 times that of the frequen-

tist approach and that the probability of a daily

loss of more than $25 million is small but noninfi-

nitesmal. (The idea of a Bayesian approach to VaR

was originally suggested in 1997 and followed up in

2000 and 2004.2It has so far gained little traction,

however. Financial analyst Riccardo Rebonatos

2007 book Plight of the Fortune Tellers: Why We Need

to Manage Financial Risk Differently3is one of the

few risk-management volumes that counsel greater

emphasis on a Bayesian approach.)

It is worth noting that the increase in the loss

probability with the Bayesian approach is notbe-

cause the Bayesian thinks that things will get worse.

The increase comes because this approach formally

and precisely reflects a recognition that we have lim-

ited understanding of the world and the important

but nonexclusive role that frequency data play in

that world. The Bayesian view makes room for judg-ment, quantifies that judgment in order to integrate

it with data on an equal footing and acknowledges

the uncertainty that inevitably remains.

The actual daily losses for the fourth quarter of

2008, together with the 99.9% loss limits for the

two approaches, would be $15 million for the fre-

quentist approach and $27 million for the Bayesian

approach. (See Daily Returns During the Full

Year.) If such a limit is accurate, there should be

only a one-in-1,000 chance that it will be exceeded

in any one day. With roughly 50 trials in the

fourth quarter, we expect that this limit should not

be exceeded during that per iod at all. But as we

now know, the fourth quarter of 2008 turned outto be a very turbulent period. Losses grew dramati-

cally and were much more consistent with the

Bayesian than the frequentist view. The frequentist

limit was exceeded 15 times, while even the larger

Bayesian limit was exceeded six t imes. Of course,

this example was developed to make the point that

the Bayesian view is more comprehensive and real-

istic. But despite the artificial construct, we believe

that the broader conclusion holds. Risk manage-

ment built around the reality of judgment

(supported by available data) is superior to r iskmanagement built around the fantasy of fact.

Altered Rainfall Patterns?Weather provides another example for contrasting

the frequentist and Bayesian approaches to risk as-

sessment and for highlighting Bayesian integration

of data and judgment. Consider a company whose

success, and possibly even existence, depends on

rainfall. Such a company could be a supplier of

drinking water, a hydroelectric-based energy utility,

an agricultural operation or a financial enterprise

with rainfall -dependent investments. Suppose

1,000 millimeters is a critical level of rainfall for the

company; that is, it needs a 1,000-millimeter year at

regular intervals ideally, at least every five to 10

years. How can we assess the risk of not receiving

this level of rainfall in the future?

The frequentist approach focuses entirely on the

data, typically applying well-accepted statistical

constructs. In this example, the rainfall pattern can

be matched by a normal distribution with a mean

of 880 millimeters and a standard deviation of 166

millimeters. With this distribution, the probabilityof rainfall of more than 1,000 millimeters in any

one year is about 23%. The probability of going

withouta 1,000-millimeter rainfall level for five

years is then (1.0 0.23)5or about 27%; for 10

years, about 7%; and for the 30 years between 1976

to 2005, less than 0.04%, or one in 2,500. Based on

this frequentist risk assessment, one can say that the

rainfall risk is extremely low.

The Bayesian approach to this issue combines,

MODIFIED PROBABILITIES OF DAILY RETURNS

This distribution reflects both the historical record (the first three quarters results)

together with the observers judgments on how the fourth quarter could differ.

-50

24%

8%

4%

20%

16%

12%

0%-40 50

Daily Return

-30 -20 -10 30 4020100


6/9

FALL 2010 MITSLOAN MANAGEMENT REVIEW 55SLOANREVIEW.MIT.EDU

as always, the available data with judgments about

the broader issues at hand. In this example, the

most important broader issue is the potential effect

of climate change on rainfall a topic of consider-

able controversy and discussion and a Bayesianmight begin with a formal assessment of expert

judgment regarding this effect.

Reflecting a great deal of uncertainty, our expert

estimates that the effect of climate on rainfall ranges

from a decrease of 200 millimeters per year to an

increase of 100 millimeters per year. Combining

this expert assessment with the historical data, we

obtain a distribution for annual rainfall that is

wider and shifted lower than that of the historical

data alone. Specifically, the mean is 830 millimeters,

and the standard deviation is 200 millimeters.Under these conditions, the probability of total

rainfall greater than 1,000 millimeters in any one

year is reduced to about 16%, which means that the

probabilities of going withouta 1,000-millimeter

rainfall year during any particular time interval are

quite different from those of the frequentist case:

about 42% for five years, 17% for 10 years and 0.5%

for all 30 years between 1976 and 2005. The latter

rainfall risk indicated by the Bayesian approach is

low, but not as low as the one-in-2,500 figure based

on the historical data alone. In fact, this probability

is more than 10 times higher.

As it turned out, the actual yearly rainfalls over

the 1976 to 2005 interval were substantially lower

than those of the previous 100 years. There was not

a single year with rainfall over 1,000 millimeters

during that 30-year period. Admittedly, this exam-

ple was chosen, as the first example, to make a

point. Not surprisingly, the Bayesian assessment

appears to be more comprehensive and realistic.

Nevertheless, we believe, as before, that a broad

conclusion holds. Assessing risk by formally inte-

grating both data and judgment leads to moreuseful results.

Learning, and Then Adjusting,ContinuouslyAnother limitation of traditional risk management

that is, of the frequentist approach involves not so

much how risk is defined or measured but how it is

prevented or mitigated. Because risk is assessed solely

by means of the historical record, and this record

changes gradually and subtly, management activity

in the traditional context is largely fixed or static. It

adjusts very slowly and modestly, if at all. Historical

frequency data are collected to establish the facts.

With the facts in hand, extensive rules are estab-lished and controls put in place. These controls

remain essentially undisturbed until there is a seri-

ous failure or disaster, although by then it is too late.

This is a rigid process, and there is no natural sys-

tem for monitoring a wide range of potentially

relevant events, developing insights from those

events and adapting in response.

By contrast, the Bayesian perspective leads natu-

rally to adjustments in the risk-management

activity itself. Because Bayesian risk assessment

combines both data and judgments, its underlyinglogic provides a built-in and rigorous way of updat-

ing assessments as new data arrive or new judgments

emerge. Equally importantly, it provides a natural

way to adjust risk-management activity in response

to this learning.

Commodity prices, such as those involving oil,

provide a good example of the contrast between

these two perspectives with respect to the actual

managementpart of risk management. (See Oil-

Market Prices Over the Past 20 Years, p. 56.) The

pattern has been volatile, particularly in the most

recent years. Consider a company that is interested

in reducing its exposure to this volatility. The typi-

cal frequentist approach is to collect as much of the

oil-price data as possible and parameterize a model

of the pr ice behavior. The model can be simple,

such as classic Brownian motion, or sophisticated,

DAILY RETURNS DURING THE FULL YEAR

With actual results of the fourth quarter of 2008 showing that it was more

turbulent than the first three quarters, the Bayesian approach would have

been particularly useful to risk managers.

Q1

$75

$50

$25

$0

($25)

($100)

($75)

($50)

Q2 Q4

Daily P & L ($)

Q3

99.9% loss limit for frequentist approach

99.9% loss limit for Bayesian approach


7/9



such as a mean-reverting Ornstein-Uhlenbeck pro-

cess. Based on the model chosen, the company

estimates future volatility and exposure, and it then

implements an appropriate hedging strategy.

The Bayesian approach uses not only the histori-cal data on oil prices but also accommodates

judgments about the underlying factors that drive

them. In particular, a Bayesian might believe that

prices are influenced over the long term by two key

structural drivers global economic conditions

and climate policies. As information is gained about

these drivers, judgments regarding oil-price risk will

change, which then leads automatically to a modi-

fied hedging strategy. (See Learning from Recent

Experience.) The company begins with a hedging

strategy for 2010 based on historical data and itsjudgments about the future situation. Some of the

future judgments, namely, 2010 economic condi-

tions, climate policies and oil price, are then revealed.

This 2010 information serves to update judgments

regarding 2011 about economic conditions, climate

policies and oil price. The company then adjusts its

hedging strategy for 2011 based on the revealed 2010

situation and the updated 2011 judgment. The ac-

tual 2011 situation will later be revealed too, and the

hedging strategy for 2012 can similarly be adjusted

in response. This updating and adjustment process

continues to 2013 and beyond.

Once again, the frequentist and Bayesian ap-

proaches show themselves to be substantially

different. The frequentist approach relies on a great

deal of historical data. It adjusts only slowly to

changing conditions, and these adjustments, such

as they are, must essentially be imposed on the

risk-management process. On the other hand, the

Bayesian approach captures both data and judg-ments. It adjusts quickly to changing conditions, as

well as to evolving judgments. And it is inherently

dynamic learning and adjustment are internal

and automatic.

Make Room for BayesiansThe frequentist view that decisions should be based

solely on facts drawn from repetitive historical data

rather than on data complementedby judgments

derived from experience and expertise can be linked

to failed companies errors and subsequent collapse.Why is this distinction between fact alone and

fact-plus-judgment so important? First, because

the fact-alone perspective provides no effective

guidance on issues often those with greatest im-

pact where there are little or no frequency data.

Its a classic case of losing ones keys where it is dark

but looking for them under the street lamp because

the light there is so much better. Second, because

unlimited faith in historical data even large

amounts of it leads to overconfidence and ex-

cessive risk taking. And finally, because a system

based solely on historical fact inevitably lurches

from crisis to crisis.

The Bayesian perspective provides more accu-

rate and powerful results. It recognizes that risk is a

matter of bothdata and judgment, and it uses the

combination in a rigorous manner for identifying,

assessing and managing risk. Where there is a great

deal of relevant data, this information plays a dom-

inant role, with the integration of judgment making

a substantial improvement over the traditional ap-

proach. Where there is little or no relevant data,

judgment plays a dominant role, prov iding valueunder conditions beyond the scope of the tradi-

tional approach.

With the Bayesian approach, risk can be mea-

sured quantitatively, whatever the amount and

quality of the data. And rather than focusing entirely

on the observed world, Bayesian risk assessment

also reflects the consistency, reliability and precision

of the observer. Recognizing the important, some-

times central, role of judgment can lead to more

OIL-MARKET PRICES OVER THE PAST 20 YEARS

The volatility of recent years challenges the frequentist approach, based as that is

on the historical record alone. But the Bayesian approach incorporates the learning

acquired from newly emerging patterns.

1987 1989

$160

$140

$120

$100

$80

$0

$40

$60

$20

1991 1995

Price per Barrel ($)

1993 20071997 1999 20032001 2005

Year


8/9

SLOANREVIEW.MIT.EDU FALL 2010 MITSLOAN MANAGEMENT REVIEW 57

reasonable and realistic behavior in large part be-

cause we realize that judgment is not perfect and

can be refined as more experience is acquired.

Admittedly, obtaining probabilities from subjec-

tive judgments rather than frequency data requiresa great deal of care. The cognitive (unintentional)

and motivational (intentional) biases underlying

probability judgment are well known.4For example,

individuals typically exhibit considerable overcon-

fidence in their probability assessments that is,

the distributions are too narrow. There also are well-

documented biases involving the overweighting of

information that is readily available and easily re-

membered. Fortunately, established and emerging

techniques for probability encoding, such as those

based on expert interviews

5

and prediction mar-kets,6can reduce these biases.

The shortcomings of the frequentist view nar-

rowness of thinking, unwillingness to accept the

possibility of error and the inability to adapt to

changing circumstances clearly played a signifi-

cant role in the recent financial collapses. Companies

such as Citigroup Inc. and Merrill Lynch & Co. con-

tinued to increase their exposure to subprime

mortgages even as evidence regarding deterioration

in the housing market accumulated: During the

early years of the housing boom, default rates on all

mortgages were unusually low. That led bankers

and, more important, rating agencies to build

unrealistic assumptions about future default rates

into their valuation models.7At these companies,

early warning signs were ignored and unrealistic de-

fault rates were not adjusted until it was too late.

With a Bayesian view, such problems may not

have been eliminated altogether, but they could

have been substantially reduced through more

comprehensive and realistic risk assessment and

more dynamic and adaptive risk management.

Many measures are being deployed to recover fromthe collapses and to build a more robust system that

prevents future crises a shift from traditional

risk management to Bayesian risk management

should be a part of this effort.

Adam Borisonis a senior vice president and Greg-

ory Hammis a senior economist at NERA Economic

Consulting in San Francisco. Comment on this arti-

cle at http://sloanreview.mit.edu/x/52107, or contact

the authors at [email protected].

REFERENCES

1.A. Gomstyn, Former AIG CEO Greenberg Defends

Reputation, March 16, 2009, http://blogs/abcnews.com;

Bear Stearns Names Michael Alix Chief Risk Officer and

Robert Neff Deputy Chief Risk Officer, Business Wire,

February 3, 2006; L. Thomas Jr. Bear Stearns Chief

Weathers the Storm, New York Times, June 29, 2007;

Federal National Mortgage Association, Fannie Maes

Marzol to Lead Companys Strategy and Competitive

Analysis Group, press release, August 26, 2004; and

Lehman Brothers, Annual Report, 2.

2.G.A. Holton, Subjective Value at Risk, Financial Engi-

neering News 1 (August 1997): 1, 8-9, 11; K. Dowd,

Estimating Value at Risk: A Subjective Approach, Jour-

nal of Risk Finance 1, no. 4 (2000): 43-46; and T.K. Siu, H.

Tong and H. Yang, On Bayesian Value at Risk: From Lin-

ear to Nonlinear Portfolios, Asia-Pacific Financial

Markets 11, no. 2 (2004): 161-184.

3.R. Rebonato, Plight of the Fortune Tellers: Why We

Need to Manage Risk Differently (Princeton, New Jer-

sey: Princeton University Press, 2007).

4.A. Tversky and D. Kahneman, Judgment Under

Uncertainty: Heuristics and Biases, Science 185, no.

4157 (September 27, 1974): 1124-1131.

5.See, for example, C.S. Spetzler and C.-A.S. Stael Von

Holstein, Probability Encoding in Decision Analysis, Man-

agement Science 22, no. 3 (November 1975): 340-358.

6.See, for example, J. Wolfers and E. Zitzewitz, Predic-

tion Markets, Journal of Economic Perspectives 18, no.

2 (spring 2004): 107-126.

7.S. Tully, Wall Streets Money Machine Breaks Down,

Fortune, Nov. 26, 2007, 64.

Reprint 52107.

Copyright Massachusetts Institute of Technology, 2010.

All rights reserved.

LEARNING FROM RECENT EXPERIENCE

By identifying the drivers of oil price and by incorporating their revealed

values over time into the risk-management model, the Bayesian approach

enables a more effective hedging strategy year after year.

HedgingStrategy

2010

HedgingStrategy

2011

HedgingStrategy

2012

Oil Price2010

EconomicConditions

2010

ClimatePolicies

2010

Oil Price2011

EconomicConditions

2011

ClimatePolicies

2011

Oil Price2012

EconomicConditions

2012

ClimatePolicies

2012
http://www.sloanreview.mit.edu/http://blogs/abcnews.comhttp://blogs/abcnews.comhttp://blogs/abcnews.comhttp://blogs/abcnews.comhttp://blogs/abcnews.comhttp://www.sloanreview.mit.edu/


9/9

PDFs Permission to Copy Back Issues Reprints

Articles published in MIT Sloan Management Review arecopyrighted by the Massachusetts Institute of Technology

unless otherwise specified at the end of an article.

MIT Sloan Management Review articles, permissions,

and back issues can be purchased on our Web site:www.pubservice.com/msstoreor you may order through

our Business Service Center (9 a.m.-7 p.m. ET) at thephone numbers listed below. Paper reprints are available

in quantities of 250 or more.

To reproduce or transmit one or more MIT SloanManagement Review articles by electronic ormechanical means(including photocopying or archivingin any information storage or retrieval system) requires

written permission.To request permission, use our Web site(www.pubservice.com/msstore), call or e-mail:

Toll-free:800-876-5764 (US and Canada)International:818-487-2064

Fax:818-487-4550E-mail:[email protected]

Posting of full-text SMR articles on publicly accessibleInternet sites is prohibited.To obtain permission to post

articles on secure and/or password-protected intranet sites,e-mail your request to [email protected]

Customer ServiceMIT Sloan Management ReviewPO Box 15955

North Hollywood, CA 91615

Documents

how to manage risk (after risk failed).pdf