© Deloitte Consulting, 2005 Predictive Modeling – Panacea or Placebo? Cheng-Sheng Peter Wu, FCAS, ASA, MAAA CAS 2005 Spring Meeting Scottsdale, AZ May

© Deloitte Consulting, 2005

Predictive Modeling – Panacea or Predictive Modeling – Panacea or Placebo?Placebo?

Cheng-Sheng Peter Wu, FCAS, ASA, MAAACheng-Sheng Peter Wu, FCAS, ASA, MAAACAS 2005 Spring MeetingCAS 2005 Spring Meeting

Scottsdale, AZScottsdale, AZMay 16-19, 2005May 16-19, 2005

22


AgendaAgenda

What is Predictive ModelingWhat is Predictive Modeling

A Case Study of Successful Predictive A Case Study of Successful Predictive Modeling - Credit Scoring RevolutionModeling - Credit Scoring Revolution

From Credit Scoring to Predictive ModelingFrom Credit Scoring to Predictive Modeling

What Does Predictive Modeling Mean for What Does Predictive Modeling Mean for Actuaries?Actuaries?

33


What is Predictive ModelingWhat is Predictive Modeling

What is predictive modeling?What is predictive modeling?– Predictive modeling is an application of Predictive modeling is an application of

mathematical and statistical techniques and mathematical and statistical techniques and algorithms to produce a mathematical model algorithms to produce a mathematical model that can effectively predict and segment future that can effectively predict and segment future events events

44


Why is PM a Hot Topic?Why is PM a Hot Topic?

Is it just a new actuarial fashion?Is it just a new actuarial fashion?

Is it just the “flavor of the month” that we all Is it just the “flavor of the month” that we all like to talk about at conferences but nobody like to talk about at conferences but nobody really does?really does?

No to both!No to both!– It is a new addition to the actuary’s toolkit.It is a new addition to the actuary’s toolkit.– It is here to stay.It is here to stay.

55


Why is PM a Hot Topic?Why is PM a Hot Topic?PM is a natural extension of what actuaries PM is a natural extension of what actuaries have done all along.have done all along.– Use data to make predictions and forecasts.Use data to make predictions and forecasts.

It allows us to add statistical rigor and It allows us to add statistical rigor and additional info to traditional areas of additional info to traditional areas of actuarial practice.actuarial practice.– Large scale data mining and multivariate Large scale data mining and multivariate

analysisanalysis– GLM-based ratemakingGLM-based ratemaking– Stochastic Loss reservingStochastic Loss reserving

66


Why is PM a Hot Topic?Why is PM a Hot Topic?

PM also allows us to broaden actuarial PM also allows us to broaden actuarial practice.practice.– Underwriting modelsUnderwriting models– Credit scoringCredit scoring– Retention and cross-sell modelingRetention and cross-sell modeling– Target marketing modelsTarget marketing models– Agency monitoring toolsAgency monitoring tools– Other industriesOther industries

77


What is New About “Today’s” What is New About “Today’s” Predictive Modeling?Predictive Modeling?

Rapid advancement of cheap computing powerRapid advancement of cheap computing power

Moore’s LawMoore’s Law

Year 1980 1985 1990 1995 2000 2004

Storage Cost per Megabyte $190 $ 70 $ 10 $0.90 $0.05 $0.001

Microprocessor Speed, MHz 5-8 16 33 75 200 400

88



Availability of wide range of data from Availability of wide range of data from internal and external sources.internal and external sources.

““Data Mining”:Data Mining”:““Data mining is a process that utilizes predictive Data mining is a process that utilizes predictive

modeling techniques to analyze large modeling techniques to analyze large quantities of internal and external data, in quantities of internal and external data, in order to unlock previously unknown and order to unlock previously unknown and meaningful business relationships” meaningful business relationships”

99



Development of new and powerful modeling and Development of new and powerful modeling and data exploration techniquesdata exploration techniques– Examples: regression, GLM, neural networks, Examples: regression, GLM, neural networks,

decision trees, clustering analysis, MARS, ...decision trees, clustering analysis, MARS, ...– Explore complicated patterns in data such as non-Explore complicated patterns in data such as non-

normality, non-linearity, interactions, etc.normality, non-linearity, interactions, etc.

Statistical analysis is no longer restricted to what Statistical analysis is no longer restricted to what you can do with pencil and paper...you can do with pencil and paper...

……or spreadsheetsor spreadsheets..

1010



Multivariate analysis with large amount of Multivariate analysis with large amount of data and many variablesdata and many variables– Analyze multiple variables “simultaneously” Analyze multiple variables “simultaneously”

instead of one or two at a time.instead of one or two at a time.– Use large amounts of dataUse large amounts of data

No need to use summarized data for actuarial No need to use summarized data for actuarial analyses.analyses.

– Create and analyze novel predictive variables.Create and analyze novel predictive variables.

1111


A Case Study of Successful PMA Case Study of Successful PM

Which Company is This?

0

2

4

6

8

10

12

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

Year

Pre

miu

m, i

n B

illio

ns

1212


Credit Score RevolutionCredit Score Revolution

Progressive vs Industry

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

Year

Gro

wth

Ra

te

80%

85%

90%

95%

100%

105%

110%

115%

Co

mb

ined

Rat

io

Industry Growth Rate

Progressive Growth Rate

Industry Combined Ratio

Progressive Combined Ratio

1313


Why?Why?

Multiple ChoiceMultiple ChoiceProgressive provided foosball tables and Progressive provided foosball tables and free snacks to their trendy, 20-something free snacks to their trendy, 20-something workforceworkforceProgressive built a compound Gamma-Progressive built a compound Gamma-Poisson GLM model to design their class Poisson GLM model to design their class planplanProgressive pioneered the use of credit in Progressive pioneered the use of credit in pricing/underwritingpricing/underwriting

1414



Personal line rating history:Personal line rating history:– Few rating factors before World War II Few rating factors before World War II – Explosion of class plan factors after the WarExplosion of class plan factors after the War– Auto class plans: Auto class plans:

Territory, driver, vehicle, coverage, loss and violation, others, Territory, driver, vehicle, coverage, loss and violation, others, tiers/company…tiers/company…

– Homeowners class plans:Homeowners class plans:Territory, construction class, protection class, coverage, prior Territory, construction class, protection class, coverage, prior loss, others, tiers/company...loss, others, tiers/company...

– Credit scoring introduced in late 80s and early 90sCredit scoring introduced in late 80s and early 90s

1515


Credit Score RevolutionCredit Score RevolutionAbout credit score:About credit score:– First important factor identified over the past 2 First important factor identified over the past 2

decadesdecades

– Composite multivariate score vs. raw credit Composite multivariate score vs. raw credit informationinformation

– Introduced in late 80s and early 90sIntroduced in late 80s and early 90s

– Viewed at first as a “secret weapon”Viewed at first as a “secret weapon”

– Quiet, confidential, controversial, black box, …etcQuiet, confidential, controversial, black box, …etc

““Early believers and users have gained Early believers and users have gained significant competitive advantage!”significant competitive advantage!”

1616


Credit Score RevolutionCredit Score RevolutionCurrent environment of credit score:Current environment of credit score:– Now everyone is using it:Now everyone is using it:

Marketing and direct solicitationMarketing and direct solicitation

New business and renewal business pricing and New business and renewal business pricing and underwritingunderwriting

– Regulatory constraints:Regulatory constraints:

Many states have conducted studies on the true Many states have conducted studies on the true correlation with loss ratio and potential discrimination correlation with loss ratio and potential discrimination issues - WA study, TX study, MO studyissues - WA study, TX study, MO study

Many states have/are considering restricting the use of Many states have/are considering restricting the use of credit scores or certain type of information, MI.credit scores or certain type of information, MI.

More states want the “black box” filed and openedMore states want the “black box” filed and opened

1717


Credit Score RevolutionsCredit Score RevolutionsWhat is “credit score”?What is “credit score”?– A composite score that usually contains 10 to A composite score that usually contains 10 to

40 pieces of credit information 40 pieces of credit information Payment pattern information, account history, Payment pattern information, account history, bankruptcies/liens, collections, inquiries, bad bankruptcies/liens, collections, inquiries, bad debt/defaults…debt/defaults…

Formula scoring or rule-based scoringFormula scoring or rule-based scoring

Industry scores and proprietary scoresIndustry scores and proprietary scores

1818


Credit Score RevolutionsCredit Score RevolutionsWhy “credit score” is so successful?Why “credit score” is so successful?– ““Large scale”Large scale” “ “multivariatemultivariate” scoring using ” scoring using

““external data source”external data source”– Loss ratio lift is significant, a powerful class Loss ratio lift is significant, a powerful class

plan factor or rate tiering factor plan factor or rate tiering factor – ““Brilliant” marketing approach for credit score:Brilliant” marketing approach for credit score:

Benefits/ROI are Benefits/ROI are measurable and measurable and lift curve can be lift curve can be translated into bottom-line benefittranslated into bottom-line benefit

Blind test and independent validation can be done to Blind test and independent validation can be done to verify the benefitverify the benefit

1919


Loss Ratio Lift CurveLoss Ratio Lift Curve

82

66

58

62

7074

78

90

120

50

Credit Score Decile

Loss Ratio

2020



1997 NAIC/Tillinghast Study of 9 Companies' Data

Loss Ratio Relativity of the Best and Worst 20% of Credit Score

Co1 Co2 Co3 Co4 Co5 Co6 Co7 Co8 Co9 Avg

Best 20% -38% -29% -19% -15% -14% -34% -22% -22% -36% -25%

Worst 20% 48% 20% 32% 30% 46% 59% 20% 22% 95% 41%

2121


From Credit Scores to From Credit Scores to Predictive ModelingPredictive Modeling

A credit score is just “A credit score is just “one exampleone example” of an ” of an insurance predictive modelinsurance predictive model

The same methods used to build credit scores The same methods used to build credit scores are used in data mining to build insurance are used in data mining to build insurance predictive models:predictive models:– Fully utilize all sources of internal and external data Fully utilize all sources of internal and external data

sourcessources– Fully utilize all available data Fully utilize all available data

Not just creditNot just credit

– Other lines of business?Other lines of business?


What Does PM Mean for What Does PM Mean for Actuaries?Actuaries?

2323



New ways of analyzing data New ways of analyzing data – New data sourcesNew data sources

– New technologiesNew technologies

– New analytical tools New analytical tools

– True Multivariate analysisTrue Multivariate analysis No longer one or two variables at a timeNo longer one or two variables at a time

– Analysis of risk-, policy-, or HH-level data, Analysis of risk-, policy-, or HH-level data, rather than aggregated data.rather than aggregated data.

2424



New emphasis on the “business” side of the New emphasis on the “business” side of the analytical work and out-of-box thinkinganalytical work and out-of-box thinking

Who thought of credit a decade ago?Who thought of credit a decade ago?How to stay competitive if everyone is using credit and How to stay competitive if everyone is using credit and GLM? GLM? What is the “next” big thing out there?What is the “next” big thing out there?Are you using the same “lift curve” and ROI concept in Are you using the same “lift curve” and ROI concept in your analytical work?your analytical work?How do you tie in your model/analytical work to How do you tie in your model/analytical work to business benefit? business benefit? Can you demonstrate the business benefits of your Can you demonstrate the business benefits of your analytical work through a blind test? analytical work through a blind test? ……etcetc

2525



New challenges to “actuarial” methodologies and New challenges to “actuarial” methodologies and principlesprinciples– Actuarial Ratemaking Principle #1: “A rate is an Actuarial Ratemaking Principle #1: “A rate is an

estimate of the expected value of future costs”estimate of the expected value of future costs”– Actuarial Ratemaking Principle #4: ” A rate is Actuarial Ratemaking Principle #4: ” A rate is

reasonable, not excessive, not inadequate, and not reasonable, not excessive, not inadequate, and not unfairly discriminatoryunfairly discriminatory

– But is that really the way profit-seeking companies But is that really the way profit-seeking companies price their products? Are rates ultimately based on price their products? Are rates ultimately based on costscosts or on what the market will bear? or on what the market will bear?

2626



New challenges to actuarial principles and New challenges to actuarial principles and methodology:methodology:– ““Unfairly discriminatory”:Unfairly discriminatory”:

If we develop a powerful new segmentation model, If we develop a powerful new segmentation model, is it discriminatory to certain risks? is it discriminatory to certain risks?

If we If we don’t don’t introduce it, is it discriminatory to other introduce it, is it discriminatory to other risks?risks?

How do we know if we don’t do the analysis?How do we know if we don’t do the analysis?

Actuaries’ “Static/Equalibrium” Principles vs. Actuaries’ “Static/Equalibrium” Principles vs. Business’ “Ever Changing/Dynamic” PrinciplesBusiness’ “Ever Changing/Dynamic” Principles

2727


Placebo or Panacea?Placebo or Panacea?So which is it?So which is it?Not a placeboNot a placebo– PM is here to stayPM is here to stay– A permanent addition to the actuary’s toolkitA permanent addition to the actuary’s toolkit– Has the power to both deepen and expand Has the power to both deepen and expand

actuarial practice.actuarial practice.

Not a panaceaNot a panacea– PM complements, doesn’t replace PM complements, doesn’t replace

fundamental actuarial principlesfundamental actuarial principles– PM does nothing without sound business PM does nothing without sound business

strategy and implementation.strategy and implementation.

Documents

© Deloitte Consulting, 2005 Predictive Modeling – Panacea or Placebo? Cheng-Sheng Peter Wu, FCAS, ASA, MAAA CAS 2005 Spring Meeting Scottsdale, AZ May