The New York Times Business Day

The Oracle of Omaha, Lately Looking a Bit Ordinary

APRIL 5, 2014

In four of the last five years, Warren Buffett has underperformed the S.&P. 500-stock index. In the view

of an expert statistician, Mr. Buffett’s recent struggles should underscore the appeal of index funds to

ordinary investors — and the near impossibility of consistently beating the market. Credit Drew

Angerer/Getty Images

Strategies By JEFF SOMMER

Warren Buffett is probably the most famous investor of his generation, and for good reason: His

track record over the long term is a thing of beauty.

He has beaten the market by a wide margin over 49 years, a record so impressive that it’s used in

finance classes as a textbook example of “alpha.”

Alpha is an elusive quality. Very simply put, it is the ability to beat an index fund without adding

risk to a portfolio. Investment managers are always seeking it. If it exists, Warren Buffett surely

has had it.

A new statistical analysis of Mr. Buffett’s long-term record at Berkshire Hathaway has just been

done, and it’s come up with some fascinating insights about his abilities, past and present, and

about the chances that the rest of us have for beating the market. Using a series of statistical

measures, the study suggests that Mr. Buffett has indeed been blessed with an impressively big

dose of alpha over a very long career.

But it also reveals something that isn’t impressive at all: For four of the last five years, Mr.

Buffett has been doing worse than the typical, no-frills Standard & Poor’s 500-stock index fund

— so much worse that it’s unlikely to be a matter of a string of bad luck. Mr. Buffett has begun

to behave like an investor with no alpha at all.

Both sobering facts — Mr. Buffett’s long-term outperformance and his recent stretch of

mediocrity — appear in high relief in the analysis conducted by Salil Mehta, an independent

statistician with deep experience in Washington and on Wall Street. Part of the study appears on

his blog, Statistical Ideas, and he shared the rest of it with me, in an elaborate spreadsheet filled

with more than 30 pages of data and formulas.

Mr. Mehta, who served as director of analytics in the Treasury Department for the $700 billion

Troubled Asset Relief Program, and as director of policy, research and analysis for the Pension

Benefit Guaranty Corporation, says Mr. Buffett’s record provides some humbling lessons about

investment strategies.

“It shows how amazingly difficult it is to keep beating the market, even for a master like Warren

Buffett,” Mr. Mehta said in an interview. “And it suggests that just about everybody else should

just use index funds and not even think about trying to beat the market.”

Mr. Buffett draws the same conclusion about index funds in his recent annual letter to

shareholders of Berkshire Hathaway, of which he is chairman. At 83, he writes that he has given

the following explicit instructions for the money that he is bequeathing in a trust for his wife:

“Put 10 percent of the cash in short-term government bonds and 90 percent in a very low-cost

S.&P. 500 index fund. (I suggest Vanguard’s.) I believe the trust’s long-term results from this

policy will be superior to those attained by most investors — whether pension funds, institutions

or individuals — who employ high-fee managers.”

Mr. Buffett says he has amassed his long-term record by following the precepts of Benjamin

Graham, the value investor who was his mentor and professor at Columbia University. Like Mr.

Graham, Mr. Buffett says he tries to make investments that will last a lifetime.

Among the forms of active management that Mr. Buffett warns about, he includes high-

frequency trading, which is the subject of the new book, “Flash Boys: A Wall Street Revolt,” by

Michael Lewis. (An excerpt appears in The New York Times Magazine.) But he also says he’s

prepared to exploit some of the problems induced by “flash traders.”

A “flash crash” or another extreme drop in the market won’t hurt someone who is a “true

investor if he has cash available when prices get far out of line with values,” Mr. Buffett says in

the letter. “A climate of fear is your friend when investing; a euphoric world is your enemy.”

Such precepts, however, don’t really explain how Mr. Buffett outperformed the market; if they

did, anyone who had read his annual letters could have done it. Mr. Mehta doesn’t explain how

he did it, either. Instead, Mr. Mehta points out how unusual Mr. Buffett really is. From a

statistical standpoint, he is an anomaly.

Consider this, gleaned from Mr. Buffett’s own data on the second page of the Berkshire annual

report: From the beginning of 1965 through the end of 2013, he outperformed his own chosen

benchmark — the S.&P. 500-stock index, including dividends — by 9.9 percentage points,

annualized.

How rare is that? According to Mr. Mehta’s analysis, it puts Mr. Buffett in a vanishingly small

class, comprising far less than 1 percent of the population of investors. This is the tiny group that

is statistically likely to have been able to beat the stock market through that elusive alpha — skill

of some sort, rather than just chance — over a period of 45 to 50 years.

The flip side of Mr. Mehta’s finding is also worth considering. A vast majority of individuals,

including most people now working in finance, do not have alpha, Mr. Mehta says. It doesn’t

matter whether they have studied finance or have prodigious math skills; the statistics show that

they are unlikely to have the ability to beat the market.

That has a serious implication for individual investors, he says: True investing skill is so rare that

the rest of us shouldn’t even try to emulate those who have it. In addition, he says, we probably

shouldn’t bother trying to hire the few outperformers to invest our money. Why? Because we

aren’t likely to be able to identify them. Their talents aren’t always on public display, and there

may be only a few thousand of them in the entire United States.

Mr. Buffett’s talents are widely known. But despite his celebrated past performance, his returns

since the beginning of 2009 have been disappointing.

In four of the last five calendar years, he has underperformed his own benchmark, the S.&P. 500

with dividends, often by significant margins. (In 2011, his return of 4.6 percent beat the

benchmark by 2.6 percentage points.) In addition, data provided by Morningstar shows that he

underperformed the average stock mutual fund investor in four of the five years.)

By contrast, in the previous decades, he had underperformed the S.&P. only six times. Mr.

Mehta said his calculations showed that given such a long period of outperformance, there is

only a 3 percent chance that the recent stretch of underperformance was a matter of bad luck.

What happened? We don’t really know, and Mr. Buffett declined to comment for this column. In

a section of the annual report, Mr. Buffett notes that because he judges investments based on

what he considers to be their intrinsic value, Berkshire may well underperform in periods when

stocks rise rapidly. He also says that as Berkshire has grown, he has increasingly been buying

entire operating companies, as opposed to investing in shares of publicly traded companies, and,

as a result, the long-term strength of his investments may not be fully reflected in his annual

data.

Mr. Mehta won’t hazard a guess, but he does compare Mr. Buffett to Michael Jordan, the

basketball star. “There were essentially two careers,” Mr. Mehta said. “In the first, he was a

superstar. And in the second, late in his career, he just wasn’t one anymore.”

Will Mr. Buffett return to form and trounce the market again? He might or he might not. But Mr.

Mehta says that most people will be better off with a draw: anyone can match the market with an

index fund.

The forever elusive α

Humans have been repeating this inefficient ritual for over 700 years, with the first known origins then

in Europe. There sprung lenders and insurers who assessed the relative merits of individual commercial

risk. The methods were somewhat more crude versus the resources available to people today, but

none-the-less this is the humble birthplace from where modern investment speculation gets its origin.

What should be the effective interest rate to lend an emerging company wanting to complete a

construction project? What should an insurer charge to protect a ship voyaging across a stormy sea, so

that the premium pricing is both attractively profitable yet competitive?

Over time, more information was rapidly made available concerning those who needed capital market

resources. And more ordinary people were able to invest in companies and products. Through the

distribution of personal wealth and technological progress, society experienced episodic bouts of

speculations and manias. The conversion of defined benefit plans in the U.S. to one where American

workers invest their own contributions, combined with draining real median wage growth, created a

force for even greater heterogeneity of outcomes in the desperate and greedy individual pursuit of α

(alpha). And then the digital age took these advances to another level, now allowing virtually everyone

to more quickly and easily trade however they want. But how can these seeming innovations be good

for society, if there is a slimmer portion of risk-adjusted beneficiaries? Let’s explore the outcomes and

difficulties in the great, inefficient search for exceptional alpha.

The true statistical test for outperformance relative to a highly liquid, investable, and specified a-

priori benchmark fully takes into account how likely such performance could have been attained by luck

(this is a favorable outcome by a small segment of generally haphazard performers). After all over any

period of time, there will be separation in the market fates of individual stocks within a basket.

Concurrently, some purely lucky stock holders will too own specific stocks that just uniformly

outperforms the underlying index over this same period of time.

Nonetheless it is worth noting that the difficult statistical standard necessary to warrant the concept of

skill over a long career, or life, has a smaller side effect. And that is that only minorities of those who

speculate will actually have, through skill, statistically outperformed the broader stock index over time.

While we don't aim for precision in this note at any one data point, we attempt to provide the basic

sense of less outpeformance is needed to claim a fixed rank of skill within a population, as one's

investment experience is lengthier. And at the same time, how the probability of outperformance

reduces over time, for a fixed level of outperformance. Put together, even with a small reduction in the

amount of outperformance needed, as one continues to speculate in the markets: the portion of the

population who outperform reduces, and even as within that reduced amount a greater share can be

attributed to skill versus luck. As a different extreme example, if we state flipping a fair coin and having

it land on heads 100% of the time suggests outperformance (we're suspending the break-out here of the

portion with skill), then 25% of people can still do this after 2 flips, however well less than 1/2% can do

this after 50 flips. On the other hand, if we reduce the requirement to stating that landing on heads 54%

of the time suggests outperformance, then still 25% of people can do this after 50 flips. Hence the

requirements are more strict at the short-end of the experiment number of trials, and there could even

be a blend of a slightly greater portion of people demonstrating outperformance at this short-end if the

performance requirements are strengthened but not by enough.

Let’s show how this works, using the time since the recent financial crisis as a baseline frame for this

analysis. From there we’ll expand to a broader set of applications and timeframes. The market has

gone through a large hockey-stick pattern since the height of the financial crisis, 5.5 years ago. Equity

markets initially plummeted through early 2009, but have since smoothly rallied to new highs. Nearly all

holdings have gone up.

If you and your friends had all tried your hand at stock selection and market-timing along the way, then

there is a good chance that you are feeling pretty good right now. Making money is always a welcome

relief, but emotional ego perilously inflates disproportionately with the rise of one’s portfolio. Even

more, in the case of the vast majority of people (those who basically doubled their investments

alongside the market index, instead of perhaps quadrupling it), feeling too good is simply unwarranted.

Humility must substitute for hubris, since if if one outperformed but this was his or her only investment

experience, then luck accounts for a great deal of post-crisis performance. It's a great start of course,

but the odds going forward are stiffer than one may realize.

How likely is it that an investor (or speculator) in U.S. equities over the past 5.5 years has demonstrated

significant investment skills in this asset class? For our test we reduce the investable universe to a

mapping of the current 30 Dow Jones Industrial Average (DJIA) stocks. We're cognizant that money-

weighted returns are much more difficult to achieve (versus the time-weighted we look at) only if one's

portfolio grows rapidly through one's career. Still those just starting to work and save can certainly pick

from a broader basket of smaller stocks, though very quickly if one were outperforming on investment

savings from career wage accumulation, then invariably they would have a greater portion of their

wealth exposed to risk similar to that of large, blue-chip stocks. Note that we'll show by the end of this

note that there are several multiple more American millionaires (excluding value of their primary

residential real estate), as there are skilled investors. See pages 468-469 for the most recent broad

government statistics on same, which they show right next to the tables where they tabulate the

poorest Americans. So with this background, we start with a performance threshold of selecting a

basket of any of the top quarter of these 30 stocks for the past 5.5 years and then expand to a broader

sample over the past 25 years. These top 8 stocks had a minimal monthly outperformance of 1.1% (14%

annualized), with a 0.3% standard deviation. This implies a significantly low, 1% chance of straying that

far from the rest of the DJIA by "luck" alone.

Then being satisfied with our critical threshold, we next solve the probability of continuously selecting a

basket of the annual top quarter of DJIA stocks by chance alone. This is an elementary, compounded

Bernoulli problem, and it comes to less than 1%.

We then use Bayesian probability (see equality below) to determine the portion of the population that

has skill near the required 1.1% monthly outperformance, in order to compensate for the probability of

attaining these results by luck alone. And this portion of the population comes to the low 20s% (see

addendum at bottom). Sometimes those with skill can not outperform due to (bad) luck, just as those

with no skill (the haphazard performer) can sometimes outperform due to (good) luck, hence some

stable threshold needs to be set a-priori.

p(outperform) = p(outperform|luck)*p(luck) + p(outperform|skill)*p(skill)

Which rearranges to the following.

p(skill) = [p(outperform) - p(outperform|luck)*p(luck)] / p(outperform|skill)

While there are empirical differences that would ensue from, not the β (beta) of the 30 DJIA stocks, but

rather from the component of the typical correlation and dispersion components of beta. For example,

when the correlation is high and the dispersion is high, then greater than typical portion of the investing

population at that time would be able to skillfully outperform. And when the opposite parameters

define the investment regime, then less than the typical portion of the investment population would be

able to skillfully outperform.

Theoretically expanding this example to different timeframes, we get the following results. Note that

these examples work for the most common approach to equities speculation: market-timing with a

discretionary allocation towards individual stocks. For 2 years, instead of 5.5 years, the portion of the

population with skill increased to the high 30s%. This is because it is significantly less difficult to

outperform monthly at the same stated 1.1% level, but for a far briefer time period. Instead of focusing

on a precise level for any one person in the middle of the chart, one should again simply appreciate in

this note we are providing a strong sense of the directionally correct pattern for attaining

outperformance that essentially erodes over the long haul. And we discuss some of the confidence

levels to practically consider these probabilities, particularly at the shortest-end of the timeframe where

we need to demand a higher outperformance threshold or perhaps one may want a slightly greater

information ratio.

On the other end of the time spectrum, for 25 years and 50 years of speculation, the portion of

speculators who are able to maintain the same level of statistical evidence of investment skills rapidly

decreases to 1%, and less than 1/2%, respectively. This is shown in blue, on the left axis of the chart

below.

We can also skills-adjust these data, so that we can solve for the level of outperformance that a 2, 25,

and 50 years investment career would need to equate to the same level of difficulty for different time

periods, we adjust to statistically balance for the same difficulty (since we are now looking at briefer or

much lengthier measurement periods), as the 1.1% monthly outperformance associated with a 5.5 years

sample. This comes to 1.5%, 0.5%, and 0.4%, respectively. See the red data below. Incidentally these

monthly outperformances equate to an annual outperformance of about 19%, 7%, and 5%.

We also identify in green, a proxy performance of an extraordinary investor, Warren Buffett. While not

precise for this study, it gives a great idea of how this research note's conclusions square with Warren's

record. In 2001 Warren expressed in Berkshire Hathaway’s (BRK) Chairman’s Letter:

Investors should remember that excitement and expenses are their enemies. And if they insist on

trying to time their participation in equities, they should try to be fearful when others are greedy and

greedy when others are fearful.

This turned out to be even more sage advice in 2008-2009, then it was in 2000-2001. Yet despite his

extraordinarily favorable private placements, which he was able to negotiate during the depths of the

financial crisis, on a market price he barely underperformed during the past 5.5 years. Market price was

used since BRK’s recent months of valuation accounting data is not yet available. We also see in this

chart that he has significantly outperformed over his lengthier, 45-plus year history (which he has

generally illustrated in the initial pages of the annual report), and doesn’t really need the skills-adjusted

handicap we show for those investing for such a long period of time. This reconciles with the fact over

the past 48 years, BRK has outperformed the market in 38 years (roughly besting the market 75%-80%

over a very lengthy time).

To clarify this previous point, we see in that chart above that having only a third, of the recent 5.5-year

skilled monthly outperformance (e.g., from 1.1%, to 0.4%), is needed to correct for the precipitous drop

in the odds of outperforming at that level for over 25 to 50 years. Conversely, a spike higher in monthly

outperformance is needed, during a more brief investment period, to statistically perform the same as

the nearly 24% of people have been capable of showing skill during a briefer 5.5 years.

The confidence interval here is tightest about the 10-year area of the chart above due to the broader

sample going back 25 years, and the large probability errors when estimating at the shortest-end of the

timeframe. And going forward in this analysis, we assume that there are tens of millions of Americans

who are in the labor force, saving, and actively invest (also the major online brokers generally have less

then ten million accounts each so this estimate is in the correct magnitude). This is just to give a rough

sense of how these probability proportions filter through the population and not a deep demographic

analysis on personal financial balance sheets.

Given this, the probability of outperformance would suggest roughly a couple million Americans in their

20s have this sort of investing experience and can feel comfortable with their initial results, even though

a small fraction of this group will actually continue to bear out skillful outperformance over the long-run.

And even early on nearly ¾ of their peers, are already doomed in any pursuit of a 25 to 50 year

statistical outperformance. Since we showed above that only 20s% have succeeded. And Warren

Buffett’s recent unraveling in both performance and active investment confidence isn't a compelling

counter-example for the wishful possibilities of the reverse: a miraculous, late-career revival!

Now on the other end of the age spectrum, nearly a hundred thousand people with 20-30 years of

investing experience have shown the same skill level over this lengthier time frame. And finally of those

in Warren’s age group (45-55 years of investing experience), the number who have also proven to have

skill is just in the thousands. So not a literally a handful, but highly rare in the bigger scheme of things.

Spreading this talent level equally across the 50 states, this latter group would show to be a few dozen

people per state.

Do these numbers such as 50,000 skilled performers aged in their 50s (or 30-40 years of investment

experience), seem like a lot? Well to put this into some perspective, there 8 million Americans who

work in finance related activities. If half of these 50,000 skilled performers were represented by a

spread of the top 10% of those senior workers in finance (now out of 800 thousand), then that would

imply 97% from this top 10% still have not proven investment skill over this period. While obviously the

details of individual performance and strategies are impossible to assess at all times, this statistical

analysis should at least provide a directional picture for the sheer improbability of maintaining skill over

a lengthy period, for all but a few. This is even accounting for a generous handicap on the necessary

outperformance needed for those with longer investment histories.

With such daunting odds, what advice is there for people who dimly choose to speculate anyway, tying

up large amounts of their human capital? There are five specific advice here to impart.

The first advice is that this age-old ritual is exceptionally difficult, and perhaps brutally more so as more people attempt to acquire alpha. With the rise of low-cost index funds, there can be less broad opportunities to achieving it alpha. Just as additional people playing the lottery (or stories about the "nearby store" over the decades having sold multiple winning tickets) can

never increase your ability to attain the winning ticket, nor to outperform the population who more judiciously invests their money.

The second advice is that simply learning the rules of finance or working in the industry hardly increases your absolute chance of outperforming the market (see quote at bottom). This chance we showed in the note is fairly established in probability theory, and it's super low (and likely even lower if one is uncertain within a few years of trying whether they have strong investment skill). Stocks are simply too connected to one another to make distinguishing their fates not as likely. The overall advice here is akin to knowing how to throw a javelin or play chess in junior high school doesn’t imply we should think we can then effectively compete in the Olympics nor play chess against a computer, respectively. It's great to try for a several years, but one should also know that quitting if one doesn't succeed and moving on to Plan B is sometimes more wise.

The third advice is that much more often it is better to simply buy an index fund (and thereby be guaranteed to outperform most of the people who are generally unsuccessful in their attempt to outperform the market), and know that investment capacity is often dear and that human capital are often better spent only entertaining some other pursuits. Better of course to buy the broadest exposure to different asset classes.

The fourth advice is that the very small number of people are skilled investors share some rare talents. They are gifted with an unusual and recognizable super-ability to seamlessly connect specific dots within changing investment problems, over long periods of time, well beyond the abilities of normal smart people. The skills could be in a subset of understanding behavioral finance, consumer sentiment, technical analysis, international public policy, global macro economics, risk, statistics, derivatives, valuation accounting, etc. Of greater importance, they know the many areas of investment knowledge where they are not personally skilled on a global stage, and nimbly have the sense then to avoid those investment risks which trap others.

And the fifth advice is that selecting world-class stocks or a world-class investment manager are both generally difficult, and anyway inefficient. If one can’t properly select the former, then one usually can't skillfully select the latter. And the risk factors are mostly the same. Simply selecting an investment manager based on past results, such as BRK (which at least can proudly prove their long-term record), can often provide a false value for what to infer about an investment manager's future performance. Just see how the past 5.5 years of BRK were, as they were the most wickedly disastrous streak for the company, since 1965! Another example could be one of my college professors (Merton) who won a Nobel prize in economics, yet then went on to co-lead the destruction of a master hedge fund.

We close with a 1998 quote from Warren Buffett. May the wisdom prove promising to those who still

choose to toil away, in pursuit of that magically elusive thing.

Success in investing doesn't correlate with I.Q. once you're above the level of 25. Once you have

ordinary intelligence, what you need is the temperament to control the urges that get other people

into trouble in investing.

March 2014, some additional details:

For the 12 months of 2013, the top quartile security among large companies had an average monthly

relative performance of 1.1%. In a 20% random sample from the past few decades, a top quartile

security in any year, also outperformed the index by about, at least 1.1% monthly average. For this

note, we set a baseline ouperformer performance, which is partially correlated from year to year. This

comes to a probability of outperformance of 11%. There are assumptions in the baseline, in terms of

probability of outperformance and degree of correlation, which could tweak the results slightly to edge

the monthly outperformance up or down, to between 1.0% to 1.2%.

The probability of selecting a top quartile security in any year is just greater than 25% due to discrete

approximation. So over 5.5 years such a haphazard individual would succeed roughly 25%5.5. Also a

baseline critical probability level for a skilled investor to outperform the market with typically top

quartile securities is set to 50%, but as well noted in article above this could be reduced slightly for one's

personal utility and it still doesn't change the storyline of rapidly vanishing skill probability over one's

lifetime. If the hurdle were set too low, then more haphazard people would be considered skilled (a

false positive in the statistical sense). If the hurdle were set too high, then more skilled people would

not get credit for good performance relative to a haphazard investor (a false negative in the statistical

sense).

So p(outperform)=p(outperform|luck)*p(luck)+p(outperform|skill)*p(skill).

Or 11% =0.1%*p(luck)+50%*p(skill).

And p(luck)~75-80%+p(skill)~20-25%=100% would solve this.

One can follow this data on the blue line of the illustration above. Bear in mind that personal circumstances vary per individual so one can calibrate this analysis slightly to their own appetite, though the interlocking movement of the trends in probability and performance over time would still fit the patterns shown here.

Now the statistical “penalty” associated with sample sizes is ∝(1/√time). We see this in the greater

difficulty of outperforming by 1.1% monthly over lengthier periods, or the direct proportional reduction

in the needed α over a lengthier period. So to stretch from 5.5 years, to 22 years, this would be

~[1/√(22/5.5)] ratio of outperformance required. Or from 1.1%, to about 0.5%-0.6%. One can follow this

data on the red line of the illustration above. The actual data in the illustration was solved precisely; we

are just describing an approximate, quick rule-of-thumb here in this paragraph.

The above illustration and note provides a direction on the statistical difficulty in outperforming the

market, and maintaining this for a lengthy period by consistently picking the top securities over multi-

year periods. It works by wrapping theoretical (Bayesian) modeling on top of an empirical sample of

data.

Also other more empirical mathematical analysis, aiming to study specific investment managers’

outperformance, has shown results that in the middle years (e.g., about 10 years) come to roughly

similar small fractions of outperformers. Many of these popular studies, by design, focus on mostly

credibility rules to drive at usually a different type of result: p(outperform), as opposed to p(skill) versus

p(luck).

Now to just see a different extreme illustration of how the requirement to demonstrate outperformance

rises (above 50/50), somewhat related to the exponential style penalty function noted in earlier

paragraphs, as the number of trials is reduced. See this illustration below concerning a fair coin flip (so

now completely unhinged from empirical stock market analysis and the break-out

of haphazardness versus skill). We see the probability of flipping this 50/50 coin consistently with one

outcome (e.g., heads) is inversely proportional to the number of chained attempts (and greatly reduces

well under 1/2% if done over 50 times). Notice the small jaggedness in the red line, due to the discrete

approximation error with the number of coin flips.

Posted 12th February by Salil Mehta

http://statisticalideas.blogspot.com/2014/02/forever-elusive-alpha.html