1
How to Evaluate An Investment Manager
Jessica N. Portis, CFASenior Vice President
Summit Strategies Group8182 Maryland Avenue, 6th FloorSt. Louis, Missouri 63105
2
LiquidityInflation RiskAsset AllocationMarket VolatilityRisk Management
Manager SelectionLiability ManagementFunding Rate Volatility Government RegulationRate of Return and Funding Issues
* Greenwich Associates, Market Trends 2013, USII-13
3
PUBLIC FUND MANAGER HIRING ACTIVITY*
2010
2011
2012
2013
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Expect to HireHave Hired
* Greenwich Associates, Market Trends 2013, USII-13. “Have Hired” refers to mandates or assignments US institutions have hired for – additional or replacement – in the past 12 months. “Expect to Hire” refers to anticipated hiring for mandates or assignments in the next 12 months.
4
CLIENTS FIND IT EASY TO HIRE MANAGERS
Intellectually Challenging Exercise
“Doing”Something
Attention from Managers
It’s Fun to Go Shopping
Thrill of the Chase
Good Source ofInformation for
Personal Accounts
5
CLIENTS FIND IT DIFFICULT TO TERMINATE MANAGERS
● Admitting a mistake.
● Personal relationships established.
— Probably less today than 10 years ago.
● The crowd mentality.
● Politics— Local— Minority
● Timing
— Don’t want to sell at the bottom.
● It’s costly to switch managers.
● Manager’s reason for poor performance or personnel changes appears reasonable.
YOU’RE FIRED!
6
Fama’s Nobel Work Shows Active Managers Fated to Lose
— 10/15/2013 – Charles Stein
Active Managers Stink? Blame These All-Too-Common Fund Flaws
— 10/22/2013 – Focus on Funds by Brendan Conway
Why 'active' investment is a losing bet
— 3/23/2012 – Moneywatch by Larry Swedroe
Medieval Medicine And Active Fund Management
— 7/25/2013 – Personal Finance by Rick Ferri
The Problem for Active Managers— 4/29/2014 – Opinion by Pauline Skypala
ACTIVE MANAGEMENT MAKES FOR AN INTERESTING MEDIA TOPIC
7
-600
-400
-200
0
200
400
600
800
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Med
ian
Exce
ssRe
turn
(Net
of F
ee a
t $50
mill
ion)
Percentage of Rolling 3-Yr Observations Generating ER
US Large Cap ValueThree Year Quarterly Rolling Excess Return Consistency
Summit Top Tier StrategySummit Second Tier
ALPHA DOES EXIST; YOU JUST HAVE TO KNOW WHERE TO FIND IT
Relatively consistent andhigh excess return
Very consistent and positive excess return
Closet index strategies
Very consistent and high excess return
8
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
MULTI DIMENSIONAL APPROACH TO DUE DILIGENCE PROCESS
9
CLIENT CONSTRAINTS = SEARCH CRITERIA
● Desirable Manager Attributes:
— Experience o Firm tenureo Investment team “pedigree”o Product history
— Historical Performanceo Relative to peerso Relative to benchmark
— Level of Assets Under Managemento Minimum sizeo Number of clientso Client diversity
— Preference for local, emerging or minority manager
Candidate Pool
Detail Client Constraints
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
10
DEFINE EXPECTATIONS FOR THE MANDATE AND MANAGER
Alpha Expectations Macro Impact on Sub Strategies
Evaluation Period Tracking Error Budget
Integration with Existing Mandates
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
11
WHY SETTING EXPECTATIONS IS IMPORTANT
● Helps frame your goals and objectives.
● Focus your efforts on meaningful/impactful strategies.
● Improves your odds of being “satisfied” with the manager.
● Narrows down the list of managers you have to perform in-depth due diligence.
— Makes your due diligence process more efficient.
— Tedious process that involves the review of both quantitative and qualitative aspects of firm.
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
12
EFFICIENCY OF ASSET CLASS – ESTABLISHING REALISTIC ALPHA EXPECTATIONS
Emerging MarketsUS Large Cap Value
64%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Dec
-99
Dec
-00
Dec
-01
Dec
-02
Dec
-03
Dec
-04
Dec
-05
Dec
-06
Dec
-07
Dec
-08
Dec
-09
Dec
-10
Dec
-11
Dec
-12
Dec
-13
Perc
enta
ge o
f Uni
vers
e Be
ating
Ben
chm
ark
Annual Chance of Beating Benchmark
52%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Mar
-94
Mar
-95
Mar
-96
Mar
-97
Mar
-98
Mar
-99
Mar
-00
Mar
-01
Mar
-02
Mar
-03
Mar
-04
Mar
-05
Mar
-06
Mar
-07
Mar
-08
Mar
-09
Mar
-10
Mar
-11
Mar
-12
Mar
-13
Perc
enta
ge o
f Uni
vers
e Be
ating
Ben
chm
ark
Annual Chance of Beating Benchmark
Median Annualized Excess Return: 0.18% Median Annualized Excess Return: 1.75%
*Rolling 1-yr Quarterly 1994-201. Source Evestment Alliance, Large Value - 733 Strategies/ Emerging Markets - 497 Strategies (active & inactive)
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
13
UNDERSTANDING WHERE YOU ARE IN THE ECONOMIC CYCLE AND THE ALPHA POTENTIAL FOR SUB STRATEGIES
Annualized ER vs. R1kV (2000-2013) Tracking Error
Deep Value 2.5% 7%Defensive 2.4% 8%Relative Value 1.7% 4%Dividend 1.0% 5%Closet Index 0.9% 3%
Defining Expectations
for the Mandate
Large Cap Value Manager Sub Strategy Performance
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013Deep Value
17.3%Deep Value
8.6%Russell 1000 V
22.2%Yield / Income
5.4%Defensive -18.1%
Deep Value 41.5%
Deep Value 16.7%
Defensive 7.0%
Russell 1000 V 17.5%
Deep Value 36.9%
Russell 1000 V 16.5%
Closet Index 8.4%
Yield / Income 20.9%
Relative Value 5.2%
Yield / Income -30.8%
Relative Value 25.9%
Defensive 15.7%
Yield / Income 6.3%
Closet Index 17.1%
Closet Index 34.2%
Relative Value 15.8%
Relative Value 8.4%
Closet Index 19.6%
Closet Index 4.0%
Relative Value -33.1%
Closet Index 22.7%
Yield / Income 15.5%
Relative Value 1.0%
Deep Value 15.9%
Relative Value 33.2%
Closet Index 15.6%
Russell 1000 V 7.1%
Deep Value 18.8%
Defensive 2.2%
Closet Index -35.1%
Yield / Income 21.1%
Russell 1000 V 15.5%
Closet Index 0.6%
Relative Value 14.9%
Russell 1000 V 32.5%
Defensive 14.4%
Yield / Income 6.5%
Relative Value 18.2%
Deep Value 1.6%
Russell 1000 V -36.8%
Russell 1000 V 19.7%
Relative Value 14.4%
Russell 1000 V 0.4%
Yield / Income 13.3%
Yield / Income 29.5%
Yield / Income 14.0%
Defensive 3.4%
Defensive 16.6%
Russell 1000 V -0.2%
Deep Value -40.8%
Defensive 18.5%
Closet Index 14.1%
Deep Value -3.2%
Defensive 11.9%
Defensive 25.5%
14
IMPACT OF TRACKING ERROR ON RESULTS: WHAT ARE YOU COMFORTABLE WITH?
● Observation: Historically, strategies utilizing a high-conviction investment styles performed best within the US Large Cap Value market segment.
● Takeaway: Align your alpha expectations with your risk tolerance.
Risk Tolerance: Relationship Between Excess Return and Tracking Error.
Relationship between excess return and tracking error. 27,889 rolling 3-year periods for 723 US large cap value strategies 1994-2014. Source: Evestment Alliance.
1-3% 3-5% 5-10% 10%+
-0.5%
0.0%
0.5%
1.0%
1.5%
2.0%
2.5%
3.0%
3.5%
Tracking Error
Med
ian
Exce
ss R
etur
n
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
15
PERFORMANCE EVALUATION DILEMMA
Bottom Quartile
Bottom Decile
0% 10% 20% 30% 40% 50% 60% 70%
% Large Cap Value Managers That Ranked Low for At Least One 3-Year Period in Prior
10 Years
● Good managers sometimes appear “bad”.
● The gut reaction is to terminate poor-performing managers.
● What makes a poor manager?
● How to evaluate track record?
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
16
HOW TO EVALUATE TRACK RECORDS
● Avoid simply relying on recent performance to evaluate a manager – it can be dangerous.
● Incorporate elements of risk and return.— Standard Deviation— Information Ratio— Tracking Error— Batting Average— Up/Down Market Capture
● Rolling period evaluation is much more telling than a single point in time.— Consistency of results.— Helps isolate luck versus skill.— Not constrained by point in time comparisons.— Assess the effect of macro economic conditions on results.
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
17
A CASE STUDY
● Question: Should this manager be terminated?
● Answer: It Depends! Performance only captures one dimension. Need to qualitatively understand the manager to better assess future prospects for success.
1-Year 3-Year 5-Year0%
5%
10%
15%
20%
25%
30%
Manager Index
Trailed 8 of last 12 quarters with no positive alpha
Trailed 13 of last 20 quarters with no positive alpha
Annualized Results Ending 9/30/2007
Mar
-01
Jul-0
1N
ov-0
1M
ar-0
2Ju
l-02
Nov
-02
Mar
-03
Jul-0
3N
ov-0
3M
ar-0
4Ju
l-04
Nov
-04
Mar
-05
Jul-0
5N
ov-0
5M
ar-0
6Ju
l-06
Nov
-06
Mar
-07
Jul-0
7
Manager Index25%
Median
75%
100%
0%One-Year Rolling/Quarterly Percentile Ranking
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
18
MANAGER AVOIDANCE ≥ MANAGER SELECTIONPe
rson
nel I
ssue
sProduct Assessment
Philosophy & Process
Firm
Spec
ific I
ssue
s
Due Diligence Is Focused on Risk Management.
“I’d rather be roughly right than precisely wrong.” - John Meynard Keynes
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
19
QUALITATIVE ASSESSMENT: FIRM
FIRM
Ownership
Leadership
Compensation
Litigation
Product
Integration
Business
Mgmt.
Operational
Infrastru
cture
“I don’t think culture is something you can describe.” – Bill Gates
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
20
QUALITATIVE ASSESSMENT: PRODUCT
AUM
StrategyHistory
Client Type
Diversity
CapacityManagement
Culture
Star vs.
Team
PRODUCT
ASSESSMENT
“I don’t want a lot of good investments; I want a few outstanding ones.” – Philip Fisher
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
21
QUALITATIVE ASSESSMENT: PERSONNEL
Team
Pedigree
Team
Stability
Add’ lTeam
Resources
TeamExperience
TeamSize &
Depth
PERSONNEL
“Coming together is a beginning. Keeping together is progress. Working together is success.” – Henry Ford
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
22
QUALITATIVE ASSESSMENT: PHILOSOPHY AND PROCESSPh
iloso
phy
Investment
Process
Performance
Drivers
Trading
Portfolio
Characteris
tics
Decision Process
Risk Cultu
re
PHILOSOPHY &
PROCESS
“If you don’t have a competitive advantage, don’t compete.” -Jack Welch
Detail Client Constraints
Qualitative Review
Defining Expectations
for the Mandate
Quantitative Review
23
TRIGGERS FOR A FORMAL REVIEW OF A MANAGER
● Firm – Material ownership change or regulatory concern.
● Product – Change in assets under management, style drift, or dilution/orphan of product.
● Personnel – Departure of key investment professional or change in team structure and decision process.
● Philosophy and Process – Change in investment philosophy, shift in market environment not conducive to style, or relaxation of risk controls.
● Performance – Three-year excess return outside of original investment thesis or deterioration in quantitative criteria – information ratio, tracking error, standard deviation, batting average, up-/down- capture ratio, etc.
● Strategic Fit in Client Portfolio – Does the strategy still make sense in terms of your needs?
24
REVISITING THE CASE STUDY
25
Five-Year Rolling/Quarterly Percentile Ranking
Growth of $1.00
Universe: International Value Equity (Separate Accounts & Commingled Funds)
CASE STUDY: THE COMPLETE PICTURE A high conviction manager underperforms badly: do you stay or go?
— Was the firm sold? No.— Had the AUM changed dramatically? No.— Did the team experience turnover? No.— Did the investment process change? No.— Were the markets rewarding their
approach? No! Quality and valuation were out of favor for several years.
26
CONCLUSION
● Clearly define the role the manager and strategy play in the overall portfolio.
● Understand both the opportunities and threats associated with a manager prior to investing.
● Define performance expectations for each manager.— What does the historical pattern of returns for a manager look like?— Understand which strategies work best in the asset class and when.
● When issues arise, investigate thoroughly and compare to documented expectations.
● Act quickly when determining that a termination needs to occur.
QUANTITATIVE REVIEW + QUALITATIVE REVIEW + ONGOING MONITORING = SUCCESS
27
QUESTIONS AND ANSWERS
Disclaimer: Summit Strategies Group (Summit) has prepared this report for the exclusive use by the client for which it was prepared. The information herein was obtained from various sources, such as the client’s custodian(s) accounting statements, commercially available databases, and other economic and financial market data sources. While Summit believes these sources to be reliable, Summit does not guarantee nor shall be liable for the market values, returns or other information contained in this report. The market commentary, portfolio holdings and characteristics are as of the date shown and are subject to change. Past performance is not an indication of future performance. No graph, chart, or formula can, in and of itself, be used to determine which securities or investments to buy or sell. Any forward-looking projection contained herein is based on assumptions that Summit believes may be reasonable, but are subject to a wide range of risks, uncertainties and the possibility of loss. Accordingly, there is no assurance that any estimated performance figures will occur in the amounts and during the periods indicated, or at all. Actual results and performance will differ from those expressed or implied by such forward-looking projections. Any information contained in this report is for information purposes only and should not be construed to be an offer to buy or sell any securities, investment consulting or investment management services.