11
Human Capital Score™ Summary and Model Validation Confidential Information

HCS Summary, Validation & FAQ's

Embed Size (px)

Citation preview

Page 1: HCS Summary, Validation & FAQ's

Human Capital Score™

Summary and Model Validation

Confidential Information

Page 2: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 2

Index

Why the Human Capital Score™ is Relevant? ......................................3

How the Human Capital Score™ Works...............................................3

The Human Capital Score™ and Economic Cycles ................................4

Validating the Human Capital Score™ Model ......................................4

Frequently Asked Questions about the Human Capital Score™ ...........8

What is the Human Capital Score? ........................................................................................................... 8

Why hasn't this been done before? .......................................................................................................... 8

What insights does the Human Capital Score™ provide? ......................................................................... 8

How does the Human Capital Score™ address economic condition fluctuations? .................................. 9

On what scale is the Human Capital Score™ provided? ........................................................................... 9

So how does the Human Capital Score™ work? ....................................................................................... 9

What can't the Human Capital Score™ do (aka "the fine print")? .......................................................... 10

Page 3: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 3

Why the Human Capital Score™ is Relevant?

Traditional credit measures (e.g. the FICO© score) are oftentimes inadequate in evaluating the risk of

educational loans since students have had little chance to develop a representative credit history and

may be incorrectly perceived as "high risk" borrowers. The Human Capital Score™ (HCS) is an innovative

proprietary credit risk measure for students developed by People Capital to overcome this information

barrier between student borrowers and educational lenders. The HCS™ is an ideal credit risk measure

for student loans because it uses (just as the traditional credit scores do) standardized and verifiable

attributes; however, these attributes are common to all students (like test scores and majors) instead of

common to all borrowers (like debt repayment history). Instead of predicting future default directly, the

model predicts students’ future incomes. By predicting future income, the HCS™ can better evaluate the

student's ability to repay their student loans in the future and is a more appropriate measure of risk in

student lending.

How the Human Capital Score™ Works

The HCS™ is modeled developed using historical postsecondary education data from several sources.

These data allow us to evaluate the relative effect of certain attributes (such as high school, GPA,

SAT/ACT scores, college, and major) on students' future earnings. We then calibrate these income

estimates to current income levels to predict students' incomes for the 10 years after college

graduation. These income predictions, while very accurate in predicting the ability for students to repay

education debt, will not, however, inform lenders on the propensity of students' to repay education

debt out of what they earn.

The HCS™ model gathers information from the student including standardized test scores, GPA, high

school, college, and major. The HCS calculator in turn provides two pieces of information about the

student: first, a Human Capital Score™, denoting the relative "risk" of students, (this is currently

presented on a scale of 1-9 with "+" or "-" used to denote scores at the high/low end of a range, though

the calculator can provide gradations on a 99 point scale); and second, the projected income path of a

student over the ten years after graduation, which includes a mean predicted income for each year as

well as a predicted income range. This "income band" illustrates the range of possible annual incomes a

student can reasonably expect to earn in a given year. Statistically speaking, a student with a given set of

attributes can expect to earn an income that falls within this income range 80% of the time; we similarly

expect the student to earn an income that falls above this range 10% of the time and below this range

10% of the time.

Page 4: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 4

The Human Capital Score™ and Economic Cycles

The current economic downturn has had a negative effect on the employment prospects of college

graduates, and, therefore, income expectations should be adjusted to account for the subsequent effect

on graduates' incomes. To account for these cyclical effects on expected incomes of college graduates,

the HCS™ model incorporates a "recession adjustment" that is derived from current and historical state

unemployment rates. The unemployment rate is used as an instrument for the effect on potential

earnings because it is affected by a recessionary cycle much the same way college students' earning

potential is affected; in order to earn an income one must be employed. Therefore, we expect that a

higher unemployment rate will result in lower expected earnings. For simplicity, we assume that

students generally find work in the same state as the undergraduate institution from which they

recently graduated. We calculate a "recession weight" derived from the change in each state's

unemployment rate between the current year and the years of the historical data used in the model.

This change in state unemployment rate allows us to control for the current economic downturn while

still using the historical data. We can adjust our predictions since we observe how the unemployment

rate in the current year differs from the unemployment rate experienced by the college students in our

dataset. We essentially adjust the predicted income of students in proportion with the state

unemployment rates of the current recession less the unemployment rate experienced by the students

in the dataset.

Validating the Human Capital Score™ Model

In developing the HCS™ model, the People Capital team has aggregated data from several institutional

and government sources into the dataset used in the model. These data consist of US undergraduate

students and information including their schools, majors, and earnings after college. We then use these

data to predict each student's future earnings for the decade after graduation. Since these data contain

information on realized earnings in the ten years following graduation, we are able to validate the HCS™

model's predictions in two ways using our original sample. First, we evaluate the accuracy of predicted

income as compared to realized income within predicted income deciles for our entire sample. Then, we

split the sample at random into two parts—a "regression" sample and a "test" sample—in order to

simulate an "out of sample" test while still using our original data. In this simulated out of sample test,

we first regress salary on attributes for the "regression" half of our initial overall sample. The results of

that regression are then used to predict the income for the observations within the "test" half of our

initial overall sample. Ideally, the "test" half of the sample would result in predicted incomes as accurate

(i.e. as close to the corresponding realized income) as the "regression" half of the sample.

Figure 1 and Figure 2 both show deciles of HCS™ predicted income and corresponding realized income

for Years 4 and 5 after graduation. In these figures, our measure of "accuracy" is determined by the

proximity of the results to the ideal: that realized income matches the predicted income value so that

Page 5: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 5

points lay on the 45 degree line.1 To illustrate the accuracy of the HCS™ predictions, we first sort the

income predictions into deciles. For each predicted income decile group, we calculate the decile group's

mean predicted income as a fraction of the overall sample's mean realized income (i.e. the proximity of

the mean predicted income within a decile group to the mean realized income over the whole sample).

Similarly, we calculate the decile group's mean realized income as a fraction of the overall sample's

mean realized income. We use these fractions instead of actual income values in order to "normalize"

the results so as to both preserve the integrity of People Capital's intellectual property and also to make

the results comparable from year to year.

There are two relevant values for the deciles: the mean predicted income of the decile group as a

fraction of the mean realized income of the whole sample, and the mean realized income of the decile

group as a fraction of the mean realized income of the whole sample. In Figure 1 and Figure 2, each

point represents a decile group, the x-axis measures the decile group's predicted income fraction and

the y-axis measures the decile group's realized income fraction.

Figure 1: Validating the Human Capital Score™ Income Predictions, Year 4

1 Note: When we refer to realized income, we mean actual income for 4-Year realized income and smoothed

income for 5-year realized income. The data only contain income information on select years—approximately six out of the ten years after graduation (for example, there are no data on 5-year incomes, but there are data on 4-year incomes). Since we require income values for all ten years after graduation, we generate a smoothed income trend for each individual. The smoothed income is calculated as follows: we first assume that income growth is linear, and then, we regress salary on years out of college for each individual to calculate the slope and intercept of each individual's smoothed income trend. Due to our use of smoothed income in the HCS™ model, we provide two versions of the first part of model validation; Figure 1 (Year 4) shows the 4-year predicted income and actual income, whereas Figure 2 (Year 5) shows the individual's 5-year predicted income and smoothed income. We tested alternative patterns of income growth (e.g. log-linear), but the data showed that income growth is best smoothed linearly.

0.70

0.80

0.90

1.00

1.10

1.20

1.30

1.40

0.70 0.80 0.90 1.00 1.10 1.20 1.30 1.40

De

cile

Act

ual

Inco

me

Me

an/O

vera

ll A

ctu

al In

com

e M

ean

Decile HCS™ Predicted Income Mean/Overall Actual Income Mean

YEAR 4, HCS™ Predicted Income vs Realized Income (by Deciles)

Decile Income Fraction

45° Line (Perfect Fit)

Page 6: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 6

Figure 1 illustrates HCS income results for the 4th year after college, and Figure 2 illustrates the HCS

income results for the 5th year after college. Ideally, all ten points would be on the 45° line in Figure 1

and Figure 2, thus indicating that the realized income and predicted income fractions for the decile

group are equal. As Figure 1 shows, the 4-year decile income fractions are fairly close to the 45° line.

Figure 2 similarly shows that the 5-year decile income fractions are also quite close to the 45° line.

Proximity of the decile data points to the 45° line indicates that the average HCS™ income predictions

are almost equal to the average realized income for a decile group in a given year (Year 4 for Figure 1

and Year 5 for Figure 2).

Figure 2: Validating the Human Capital Score™ Income Predictions, Year 5

The second portion of the HCS™ model validation involves an "out of sample" test, which is oftentimes

employed to evaluate whether or not a model can accurately forecast for an "out of sample" dataset

(data not used in the creation of the model). Ideally, we would use a completely different set of data to

perform this test. However, since we do not have data with all of the necessary information, we use

instead the current sample split into two random parts: the first half of our initial sample (Part A) is used

in the regression and to generate the model, while the second half of our initial sample (Part B) is used

to "test" the accuracy of the model. This test evaluates the model's forecasting accuracy because a

resulting income prediction may appear to be a "good fit" for "in sample" (Part A) data, but may not be

accurate in forecasting income for "out of sample" (Part B) data.

To test that the HCS™ income predictions are accurate for individuals outside of the regression sample,

we regress salary on attributes for the "regression" half of our original sample (Part A), and then, using

the results from that regression, we predict the income for observations in the "test" half of our original

sample (Part B). Ideally, the income prediction coefficients based on the regression half of our original

0.70

0.80

0.90

1.00

1.10

1.20

1.30

1.40

0.70 0.80 0.90 1.00 1.10 1.20 1.30 1.40

De

cile

Sm

oo

the

d In

com

e M

ean

/Ove

rall

Smo

oth

ed

Inco

me

Me

an

Decile HCS™ Predicted Income Mean/Overall Smoothed Income Mean

YEAR 5, HCS™ Predicted Income vs Realized Income (by Deciles)

Decile Income Fraction

45° Line (Perfect Fit)

Page 7: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 7

sample (Part A) should produce an accurate prediction for "out of sample" observations from the test

half of our original sample (Part B). This result would indicate that the model not only fits the “in

sample” data, but that it also accurately predicts income for new observations not used in the creation

of the model.

Figure 3 illustrates the accuracy of mean 5-year predicted income and mean 5-year realized income for

the predicted income deciles for two sets of data points—one set represents the predictions for the "in

sample" data (Part A) and another set that represents the predictions for the "out of sample" data (Part

B).

Figure 3: Out of Sample Test to Validate the Human Capital Score™ Income Predictions, Year 5

Ideally, both sets of data points would reside on the 45° line, indicating that the resultant "accuracy" of

the HCS income predictions (closeness to realized income values) does not differ between the in-sample

results and the out-of-sample results. The proximity of both sets data points to the 45° line indicates

that both the "in sample" and "out of sample" mean predicted incomes are close to mean realized

income for each of the predicted income deciles. The "in sample" data points are similar in relevance to

the data points in Figure 1 and Figure 2—they show that the mean HCS™ 5-year predicted income for

each of the deciles is close to the mean 5-year smoothed income value for each of the deciles.

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5

De

cile

Sm

oo

the

d In

com

e M

ean

/Ove

rall

Smo

oth

ed

In

com

e M

ean

Decile HCS™ Predicted Income Mean/Overall Smoothed Income Mean

YEAR 5, HCS™ Predicted Income vs Realized Income, Out of Sample Test

In Sample [Decile Income Fraction]

Out of Sample [Decile Income Fraction]

45° Line (Perfect Fit)

Page 8: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 8

However, a more powerful validation of the HCS™ model emerges from the "out of sample" results.

These points represent the predicted income deciles' aggregate accuracy of the HCS™ 5-year income

predictions for the "test" half of the original sample—which is the closest we have to an "out of sample"

dataset. The "out of sample" income predictions are almost as similar to the "out of sample" realized

incomes as the "in sample" predictions are to the "in sample" realized incomes. In terms of validation,

these results indicate that the model maintains its accuracy when used with observations outside of the

regression sample. This "out of sample" test validates the HCS™ model's ability to forecast income

accurately for an out-of-sample student—perhaps a new prospective student borrower.

Frequently Asked Questions about the Human Capital Score™

What is the Human Capital Score?

The Human Capital Score™ (HCS) is a proprietary credit risk measure developed by People Capital. It calculates future income potential by including variables such as GPA, standardized test scores, college and major. For students — who have short or no credit history — the FICO® Score is unlikely to be an appropriate measure of credit risk. While the current version of the Human Capital Score™ is best calibrated for Bachelor's degrees, we are working on future enhancements that will make the HCS applicable to other degree types.

Why hasn't this been done before?

Traditional credit scores (such as FICO® Score or VantageScore) are based on attributes that are easy to quantify and rank. More delinquencies lower the score; while a longer credit history raises the score. Evaluating student academic attributes is not so simple. How do we know which schools or majors are more likely to correlate with the ability to earn income and the capacity to repay a loan? We must collect, clean, and integrate additional data about schools, majors and such. This requires expertise, time and dedicated resources.

What insights does the Human Capital Score™ provide?

The Human Capital Score™ (HCS) projects the possible income paths of college students in the 10 years after graduation. This allows the classification of students into various risk categories which lenders can use to consider the capacity of a given group of college students to repay loans of long- and short maturities. The projected income shortly after graduation is a good indicator of short term capacity to pay. Longer-term loans can be assessed by looking at predicted income over a longer period. For benchmarking purposes, we provide a Human Capital Score™ for students for the period 2 years and 8 years post-graduation. Note that the current version of the Human Capital Score™ is best calibrated for Bachelor's degrees, we are working on future enhancements that will make the HCS applicable to other degree types.

Projections of the average future income should be considered with full understanding that it is a computer based algorithm that uses historical data and a broad view of future economic conditions to generate a result. As with any score that tries to categorize individual credit capacity, there is a range of possible future results. The Human Capital Score™ offers broad ranking categories as well as measures

Page 9: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 9

of ranges of possible income paths. This makes it possible to evaluate the likelihood that income will fall below a certain threshold in a given year, or that average income will fall below a certain threshold in the 10 years following graduation.

How does the Human Capital Score™ address economic condition fluctuations?

Traditional credit scoring models are based on historical data on defaults and credit attributes (e.g., debt outstanding, number of credit cards, etc.). Attributes linked to low default rates in this historical data are given high scores; attributes linked to high default rates are given low scores. Such traditional models do not seek to understand why, or how, there is a link between an attribute (e.g., number of credit cards) and default; rather, they only reflect patterns and links from their historical data.

Instead of focusing on factors that predict credit default, the Human Capital Score™ uses both academic and credit data to project future earnings potential. Our model identifies the set of earnings paths (around a projected average) possible for a borrower with a given set of attributes (e.g., major, school, SAT score, GPA, etc.). Given these earnings paths, we can determine how often an individual is likely to be able to generate sufficient income to pay off a loan. Because we model the reason why people might default (insufficient income), the model is easy to adjust to projected changes in economic conditions, even to circumstances never seen before in historical data. When income projections fall in response to changing economic conditions, the Human Capital Score™ will reduce income projections.

The Human Capital Score™ model differentiates between people whose earnings paths (under normal conditions) were projected to be often just barely sufficient to make their debt payments from those whose income paths were projected to be more than sufficient. If changing economic conditions reduce our income projections, this will reduce Human Capital Score™ more for the first group than the second.

This flexibility makes the Human Capital Score™ a superior tool for rank ordering students who, as a general matter, have no significant credit history – students' ability to pay is directly dependent on their future earning capacity within a future economic context.

On what scale is the Human Capital Score™ provided?

The Human Capital Score™ is able to generate 99 gradations of income potential, however, People Capital chooses to use a public ranking scale with 9 categories (1-9) with "+" and "-" to denote scores that are at the higher or lower ends of the category.

So how does the Human Capital Score™ work?

The Human Capital Score™ combines credit-risk tools and metrics with academic achievement information to generate possible earnings and insight into each borrower's future creditworthiness.

The model driven calculation is based on statistical data on a large number of students. We know their majors, schools, grades, scores, and a host of other attributes. We know how much these students earned in the years after they graduated. We can use this information to create projections of income for students based on each student's specific academic attributes. In overly simplistic terms, if students in our data who study engineering and have good grades had high and growing incomes after graduation, the Human Capital Score™ will assign high and growing incomes to engineering students with good grades who ask for a Human Capital Score™.

Page 10: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 10

The Human Capital Score™ also incorporates additional information on how much students with various majors earn, the attributes of the various schools, etc. This allows the model to make quality projections of the future potential incomes of students, even when we don't have data on many (or even any) students who went to that school or had that major. The score is not limited by the data provided by students who request a score.

Of course, we can't get data on the incomes of students 10 years after graduation - except from students who graduated at least 10 years ago. To ensure that the Human Capital Score™ reflects the most recent patterns in graduate incomes, we consider the most recent trends in the overall income distribution of college graduates and adjust based on current economic trends. Also, the current version of the Human Capital Score™ is best calibrated for Bachelor's degrees (but we are working on future enhancements that will make the HCS applicable to other degree types).

Because we have individual-level data on many students, we can project both average likely income and the range of possible income paths. Students from a given major and school may all have relatively similar incomes; another major or school may have wide variation in graduates' incomes. The Human Capital Score™ will be able to provide a variety of statistics relevant for repayment, not just expected income. The model can also estimate the likelihood that income will fall below a certain value, or fall in the worst 10 percent group. We can compute the probability that lifetime income will fall below a given threshold.

Note that the current version of the Human Capital Score™ is best calibrated for Bachelor's degrees; we are working on future enhancements that will make the HCS applicable to other degree types.

What can't the Human Capital Score™ do (aka "the fine print")?

While the Human Capital Score™ calculates potential future income, and thus broadly estimates the ability to pay, it does not measure the willingness to pay. If someone with a high income is unwilling to make loan payments, or someone with no income still makes loan payments, this isn't captured by the model. Human Capital Score™ measures ability to pay, not propensity to pay.

While the Human Capital Score™ ranks future income projections for college students in the 10 years after graduation, we do not have a crystal ball. These projections are based on data about the incomes of people who have already graduated from college and are working now. If economic conditions shift in unexpected ways, we won't capture that. Most obviously, if the current recession reduces the incomes of college graduates in the coming years, our estimates of income will be systematically too high. That said, it will continue to show the relative ranking of college students. So it will continue to show which students are relatively better options than others. Note that the current version of the Human Capital Score™ is best calibrated for Bachelor's degrees; we are working on future enhancements that will make the HCS applicable to other degree types.

The Human Capital Score™ can only project income using standardized attributes and ignores specific student interests or plans. An engineering major from MIT with high scores and grades will have a high Human Capital Score™ because past engineers from MIT with high scores and grades have on average enjoyed high incomes after graduation. If this particular student plans to join the circus (no disrespect to this particular career path intended, just that it traditionally affords a lower income level) after graduation, the Human Capital Score™ cannot, and does not attempt to, reflect this. We are not able to

Page 11: HCS Summary, Validation & FAQ's

For additional information, please visit www.people2capital.com or e-mail [email protected]

PRIVATE & CONFIDENTIAL Page 11

reliably verify or validate information about a specific student's work plans or expectations. We can only rely on information that we can verify.

The Human Capital Score™ and any income projections and ranges are solely as statements of opinion and should not be construed as statements of fact. Human Capital Scores are not recommendations to buy, sell or hold any security or to lend to any borrower. People Capital relies on information provided by borrowers and performs only limited verification of this information. The use of the Human Capital Score™ should not be construed as an endorsement of the accuracy of any of the data or conclusions, or as an attempt to independently assess or vouch for the financial condition of any borrower.