Reducing fraud losses with enhanced CNP models …...Falcon users do not currently use temporary blocks. Instead, they rely on an account-based score generally remaining high for the

© 2018 Fair Isaac Corporation. All rights reserved. 1

WHITE PAPER

The 2018 FICO® Falcon® consortium models (both credit and debit) contain new machine learning innovations focused on card-not-present (CNP) transactions.

From a user perspective, you should be aware that the CNP performance improvements in these models are significant. With CNP accounting for up to 90% of card fraud losses in some geographic regions, early detection of fraudulent CNP transactions was the primary focus in development of these new models. You should also be aware that Falcon scores produced by these models for CNP transactions will function very differently than traditional Falcon scores that you are used to managing. Prior to upgrading your models, you should review existing rules and block/unblock strategies to ensure that your fraud strategies and operational workflows are designed to properly accommodate these scoring changes.

This document is intended to provide you with additional information on the changes and highlight the items you should take action on in order to ensure a smooth transition with these model enhancements.

WHITE PAPERReducing fraud losses with enhanced CNP models on the FICO® Falcon® Platform


What are the performance improvements?

The CNP machine learning innovations included in the 2018 consortium models have been tested and evaluated with market data from the FICO® Falcon® consortium. It’s this extensive foundation of global fraud and non-fraud transaction data that allows FICO to develop these new machine learning models and provide empirical evidence of model performance at the time of model release. These improvements can benefit fraud operations in two distinct ways:

1. Reduced overall CNP fraud losses — These advances in machine learning have demonstrated an ability to reduce total CNP fraud losses by upwards of 30% without increasing transaction false positive rates.

2. Greater accuracy on CNP transactions — More CNP fraud will be averted by rapidly detecting the first CNP fraud transaction so that later transactions can be prevented from occurring.

Account vs. Transaction Scoring

Historically, Falcon credit and debit models have been designed to identify accounts in a state of fraud. This design tenet has served as the foundation of Falcon supervised fraud models and is a result of historical, account level fraud reporting into the Falcon consortium. This has since changed. Figure 1 below demonstrates how Falcon models have worked to date when trained on an account level fraud tag. The scores show a gradual increase on each suspicious transaction until the score is elevated high enough to trigger a rule and/or investigation. This account level scoring example is still applicable to card-present transactions in the new Falcon models.

ACCOUNT STATE TRANSITION

FRAUD ALERTS COMMENCE

TIMEt1 t14t13t12t11t10t9t8t7t6t5t4 t15t3t2

990955901893

810

89

10392

112

781742

660

87

225

840

Figure 1



Based on market feedback, the new CNP enhanced models were trained using transaction level tagging and a priority training algorithm to identify the first significant CNP transactions, not accounts in a state of fraud. This boosts CNP fraud detection and reduces losses by detecting fraud earlier in the lifecycle. By focusing on “transaction fraud” as opposed to “account fraud” the scores elevate very quickly on anomalous CNP transactions compared to previous FICO® Falcon® models. Since these models are designed to find the first occurrence of fraud on the account, second and third transactions typically score lower as the model is not designed to detect fraud after the first transaction.

Figure 2 below demonstrates an example of how the Falcon score would behave differently between the new and previous models on suspicious CNP transactions. In particular, we see the first significant CNP fraud transaction shows a score spike. This facilitates faster fraud detection, and therefore fraud savings, if investigated after the first fraud attempt rather than the second and third attempt as is typical of the older, account level models. This example also illustrates that the transaction score declines after the initial spike. The dashed line simulates the trending of an account score, such as those generated by previous Falcon models. Note that the account score would continue to escalate over time in this example.

TIMEt1 t14t13t12t11t10t9t8t7t6t4 t15t3t2 t5

$100

$200

$75$50

$80

$300

$5

$15

$250

$300

$200

$300$340

$370

$250

TRANSACTION SCOREACCOUNT SCORE

840

820

Figure 2



Strategy PreparationDue to changes in how the new CNP enhanced models score CNP transactions, it is essential that you prepare your rule strategies to ensure that case volumes and detection performance are adjusted based on the different scoring behavior of the models. These enhanced CNP models are designed for early fraud detection. Strategies must focus on working CNP cases immediately.

There are five key areas that need specific focus in preparation for and immediately following the new model deployment:

1. Rule segmentation for Point of Sale (POS) modes (CP vs. CNP transactions) Review existing rules where the FICO® Falcon® score is utilized and determine if there is POS mode segmentation (CNP/CP transactions are treated separately). For rules that utilize the Falcon score, manage CP and CNP rules independently given the differing behavior of the scores.

Review whether use of temporary blocks is required, if not already in use. Some Falcon users do not currently use temporary blocks. Instead, they rely on an account-based score generally remaining high for the duration of a fraud episode. With the new Falcon models, CNP transaction scores will decay after showing a spike on the first significant fraudulent CNP transaction. In order to continue declining subsequent fraud transactions until the case is resolved, you need to consider some form of temporary block measures. Existing account-based model rules will not function the same for CP and CNP transactions. You should evaluate the use of temporary blocks after the first significant CNP fraud score given that subsequent transactions will typically decline after the first significant fraud transaction.

In addition, to maintain a case creation rate, we recommend increasing scores for CNP rules initially. Based on your own data in the weeks following the upgrade, the CNP scores can be readjusted as needed.

2. Prepare CNP rules and new rules

The adjusted CNP behavior of the new model will require changes to existing CNP rules and thresholds in order to maintain control over key metrics such as Genuine Decline Rate. Scoring thresholds for CNP transactions should be reviewed and you should consider testing new rule concepts in order to get even more benefit from the CNP enhancements. These rule concepts include:

• Delta Rule — Used to identify a percentage jump, or “spike,” in score from previous CNP transaction to current CNP transaction. This can help identify an acceptable percentage increase threshold.

• New Velocity Rule — Looks for x number of transactions within a short timespan after a spike threshold.

• Score Scaling — Rule strategies can be applied to optimize performance on CNP rapid high score behaviors. Specific score scaling and score fraction thresholds can offer additional performance uplift in Value Detection Rate.

• Use of Max Score — The score of the first significant transaction is persisted and made available on rules on subsequent fraud attempts.



3. Adjust score thresholds based on case multiplier (K)

Review existing rules where the FICO® Falcon® score is utilized and determine adjustments needed to maintain the case creation rate. This step is necessary in order to keep case volumes consistent for your operations. The case multiplier can be used to estimate the number of cases generated at a threshold fraud score. Identify the new threshold score required to keep the case multiplier consistent by comparing the prior model K values to the CNP model values.

For example, if a threshold score of 800 had a case multiplier of 9 in the prior model, go to the new model K table and look up the threshold score corresponding to a K of 9 for the CNP model. If you don’t change your score threshold you may experience increased case volumes, which reduces your ability to effectively work the most likely fraud cases.

4. Monitor score variations

After the new model is live in production, you should monitor the scores to determine if variations in case loads are within acceptable limits. This should include review of real-time decisioning and case creation rule firings. Make sure to allow for appropriate fraud reporting time to capture a representative sample of the fraud and note that the new model will take a few weeks of adjustment in score continuity similar to previous model upgrades.

5. Deploy mitigation strategies, as needed

Evaluate post-go-live results to ensure they are within forecasted levels. Evaluate the use of FICO® Customer Communication Services to resolve cases using two-way, automated consumer interactions. Remember, the model is designed to maximize detection of the first significant CNP fraud transaction. Low dollar (generally equivalent to <$20) CNP transactions may score lower. For appropriate score threshold adjustments, be sure to use the latest model report.



Understanding the MetricsAssessing the performance of a model is a matter of performing a cost-benefit analysis. The cost is the incidence of non-fraud reviews, and the benefit is measured in fraud loss reduction. When comparing one model against another, you should look for an increase in detection of true frauds while the incidence of non-fraud reviews remains constant or declines. For the purpose of model evaluation, the model is considered to identify an account as fraudulent if at least one transaction on the account scores above the defined threshold.

There are three primary performance measures used to evaluate the cost-benefit of the new FICO® Falcon® models:

1. Account Detection Rate (ADR) (Fraud Detection)

The rate of fraudulent accounts that scored above a threshold compared to total fraud accounts (expressed in percent). ADR is an account-based measure, not transaction-based, and measures the percentage of fraud accounts that were identified by the model.

Example: ADR of 40 at a score threshold of 900 means that 40% of fraudulent accounts will be identified at that given threshold.

2. Real-Time Value Detection Rate (RTVDR) (Fraud Detection)

The rate of fraud value that scored above a threshold compared to total fraud value (expressed in percent). RTVDR is a transactional-based measure, not an account measure. Once a transaction scores above a score threshold, all subsequent approved transactions are included in the RTVDR calculation in order to measure how much fraud value was identified by the fraud detection system.

Example: RTVDR of 70 at a score threshold of 700 means that 70% of fraudulent transaction value will be identified at that given threshold.

3. Percent Non-Fraud Transactions1 (%NF) (Customer Service)

The number of genuine transactions from non-fraud accounts that scored above a defined threshold divided by the total number of transactions from non-fraud accounts. The %NF is expressed as a percentage. It is a transaction-based measure used to evaluate the impact of false positives and quantifies the customer impact associated with the fraud model. The %NF can also be thought of as a Genuine Decline Rate (GDR).

Example: A %NF of 0.5% at a score threshold of 800 means that out of a sample of 10,000 legitimate transactions, 50 of those transactions will score greater than 800.

1 Some versions of the Falcon model report may refer to this metric as Transaction False-Positive Rate

FICO and Falcon are registered trademarks of Fair Isaac Corporation in the United States and in other countries. Other product and company names herein may be trademarks of their respective owners. © 2018 Fair Isaac Corporation. All rights reserved. 4627WP_EN 10/18 PDF

NORTH AMERICA +1 888 342 6336 [email protected]

FOR MORE INFORMATION www.fico.com www.fico.com/blogs

LATIN AMERICA & CARIBBEAN +55 11 5189 8267 [email protected]

EUROPE, MIDDLE EAST & AFRICA +44 (0) 207 940 8718 [email protected]

ASIA PACIFIC +65 6422 7700 [email protected]


It’s important to review the relationships between these metrics, as none of them tell a complete story in isolation. For example, %NF measures customer impact, but does not consider fraud detection. Both ADR and VDR measure fraud detection, but do not consider the impact to the customer experience.

A model is considered to perform well when it achieves high ADR and RTVDR while minimizing %NF detected. Together, these metrics can help balance business benefits by answering questions such as, “What percentage of all fraud accounts do you detect using a given score threshold?” and “What percentage of non-fraud transactions do you need to handle to find these fraud accounts using that threshold?”

The FICO® Falcon® Model Performance Reports go into much more detail on these metrics and how they are derived, as well as the various components that make up a Falcon model. It is recommended that you familiarize yourself with your Model Performance Report prior to going live.

Profile MaturationAs with all model releases, the new model will take time to mature before it performs at its full capability. This is due to changes in variables used in the model. FICO’s fraud data scientists have taken care to ensure model score continuity will be maintained to a similar level as previous upgrades, with the expectation that score distributions at a fixed threshold may vary ~25%. Volume of case creation should be based on the expected score distribution, which should be stable at this level.

You may not see a noticeable difference in performance for the first two to three weeks. The model should then start to outperform the prior version and significant lift in detection is expected to be apparent by week four. After week four, the model will continue to show improved performance.

SummaryThe enhanced CNP models include unique score behavior for card-not-present transactions. While card-present scoring remains account-based, the enhanced CNP model is a transaction-based model focused on detection of the first significant CNP fraud transaction. As such, CNP transactions require a different set of strategies to optimally identify the fraud as early as possible and maintain a steady state of operations.

Documents

Reducing fraud losses with enhanced CNP models …...Falcon users do not currently use temporary blocks. Instead, they rely on an account-based score generally remaining high for the