FraudJudger: Real-World Data Oriented FraudDetection on Digital Payment Platforms

Ruoyu DengShanghai Jiao Tong University

Chinadengruoyu@sjtu.edu.cn

Na RuanShanghai Jiao Tong University

Chinanaruan@sjtu.edu.cn

ABSTRACTAutomated fraud behaviors detection on electronic paymentplatforms is a tough problem. Fraud users often exploit thevulnerability of payment platforms and the carelessness ofusers to defraud money, steal passwords, do money launder-ing, etc, which causes enormous losses to digital paymentplatforms and users. There are many challenges for frauddetection in practice. Traditional fraud detection methodsrequire a large-scale manually labeled dataset, which is hardto obtain in reality. Manually labeled data cost tremendoushuman efforts. Besides, the continuous and rapid evolutionof fraud users makes it hard to find new fraud patterns basedon existing detection rules. In our work, we propose a real-world data oriented detection paradigm which can detectfraud users and upgrade its detection ability automatically.Based on the new paradigm, we design a novel fraud de-tection model, FraudJudger, to analyze users behaviors ondigital payment platforms and detect fraud users with fewerlabeled data in training. FraudJudger can learn the latentrepresentations of users from unlabeled data with the help ofAdversarial Autoencoder (AAE). Furthermore, FraudJudgercan find new fraud patterns from unknown users by clusteranalysis. Our experiment is based on a real-world electronicpayment dataset. Comparing with other well-known frauddetection methods, FraudJudger can achieve better detectionperformance with only 10% labeled data.

CCS CONCEPTS• Security and privacy → Economics of security andprivacy.

KEYWORDSreal-world electronic payment data, fraud detection, adver-sarial autoencoder, cluster analysis

1 INTRODUCTIONDigital payment refers to transactions that consumers payfor products or services on the Internet. With the explo-sive growth of electronic commerce, more and more peoplechoose to purchase on the Internet. Different from traditionalface-to-face payments, digital transactions are ensured bya third-party digital payment platform. The security of the

third-party platform is the primary concern. The digital pay-ment brings huge convenience to people’s daily life, butit is vulnerable to cybercrime attacks [25] [28]. There aremany kinds of fraud behaviors. For example, fraudsters maypretend to be a staff in a digital payment platform and com-municate with normal users to steal valuable information.Some fraudsters will use fake identities to transact in theseplatforms. An estimated 73% of enterprises report some formof suspicious activity that puts around $7.6 of every $100transacted at risk [1]. Those frauds cause tremendous dam-age to companies and consumers.

Challenges. Automatic detection for fraud payments is ahot topic in companies and researchers. Many researchersfocus on understanding fraud users’ behavior patterns. It isbelieved that fraud users have different habits comparingwith benign users. The first challenge is how to find usefulfeatures to distinguish fraud users with benign users. Sunet al. [19] use the clickstream to understand user’s behaviorand intentions. Some other features like transaction records[33], time patterns [11] and illicit address information [13],etc, are also proved useful in fraud detection. Fraud usersalways have social connections. Some researchers focus onanalyzing user’s social networks to find suspicious behaviors[4] [21] by graphmodels. They believe fraud users have somecommon group behaviors.The limitation of the above methods is that it is hard

to manually find appropriate features to detect frauds. Intraditional fraud detection methods, researchers should trymany features until the powerful features are found, andthese features may be partial in practice. With the help ofdeep learning, we can automatically learn the best "feature".Autoencoder [15] is an unsupervised model to learn effi-cient data codings. It can get rid of "noise" features and onlyleave essential features. Origin features are encoded to latentrepresentations by autoencoder. Makhzani et al. [17] com-bine autoencoder and generative adversarial network (GAN)[9], and propose a novel model called "adversarial autoen-coder (AAE)". AAE can generate data’s latent representationsmatching the aggregated posterior in an adversarial way.Another challenge is lacking sufficient and convincing

manually labeled data in the real world. Manually labeleddata are always hard to obtain in reality. It costs a vast human

arX

iv:1

909.

0239

8v1

[cs

.CR

] 5

Sep

201

9

, , Ruoyu Deng and Na Ruan

resource to identify fraud users manually [24]. Besides, thehuman’s judgment is sometimes subjective, and it is difficultto detect cunning cheaters. Some researchers use unsuper-vised learning or semi-supervised learning models to detectfrauds [7]. However, for unsupervised learning, it is hard toset targets and evaluate the performance in training models.New deceptive patterns of fraud users appear everyday.

With the development of detection methods, fraud users alsoevolve quickly to anti-detection. It is impossible to find alldetection rules manually. Labeled data are based on historicalexperience, and it is hard to find unseen fraud patterns byusing labeled data. Since unsupervised learning has no pastknowledge [5], we can use unsupervised learning methodsto find new fraud patterns.

In our work, we aim at overcoming these real-world chal-lenges in fraud detection. We aim at analyzing users’ behav-iors and detecting fraud users with a small ratio of labeleddata. Furthermore, we want to play an active role in the com-petition between fraud detection and anti-detection. We aimat detecting potential fraud users who cannot be detected byexisting detection knowledge.

FraudJudger.We propose a fraud detection model namedFraudJudger to detect digital payment frauds automatically.Our detection model contains three steps. First, we mergeusers’ operation and transaction data. Then merged featuresare converted to latent representations by adversarial au-toencoder. We can use these latent representations to classifyusers. It is a semi-supervised learning process, which meanswe only need a few labeled data to train the classificationmodel. Finally, new fraud patterns can be detected by clusteranalysis.

Contributions. In summary, our work makes the follow-ing main contributions:

(1) We design a novel automated fraud detection para-digm for real-world application. Based on the para-digm, we propose a digital payment fraud detectionmodel FraudJudger to overcome the shortcomings ofreal-world data. Our model requires fewer labeled dataand can learn efficient latent features of users.

(2) Our experiment is based on real-world data. The exper-iment result shows that our detection model achievesbetter detection performance with only 10% labeleddata comparedwith otherwell-known supervisedmeth-ods. We propose a new measurement Cluster Recall toevaluate cluster results, and our model outperformsothers.

(3) Our model can discover potential fraud users whocan not be detected by existing detection rules, andanalyzing behaviors of these potential fraud users canhelp companies build a safer payment environment.

Roadmaps. The remainder of the paper is organized asfollows. In Section 2, we present related work. Our improveddetection paradigm is provided in Section 3. Section 4 presentsthe details of our detection model FraudJudger. Our experi-ment is shown in Section 5. Finally, we conclude our researchin Section 6.

2 PRELIMINARIESDigital Payment Fraud DetectionRecently, fraud detection on digital payment platforms be-comes a hot issue in the finance industry, government, andresearchers. There is currently no sophisticated monitoringsystem to solve such problems since the digital payment plat-forms have suddenly emerged in recent years. Researchersoften use financial fraud detection methods to deal withthis problem. The types of financial fraud including creditcard fraud [8], telecommunications fraud [10], insurancefraud [27] and many researchers regard these detection prob-lems as a binary classification problem. Traditional detectionmethod uses rule-based systems [3] to detect the abnormalbehavior, which is eliminated by the industry environmentwhere financial fraud is becoming more diverse and updatedquickly. With the gradual maturity of machine learning anddata mining technologies, some artificial intelligence modelshave gradually been applied to the field of fraud detection.The models most favored by researchers are Naive Bayes(NB), Support Vector Machines (SVM), Decision Tree, etc.However, these models have a common disadvantage that itis easy to overfit the training data for them. In order to over-come this problem, some models based on bagging ensembleclassifier [30] and anomaly detection [2] are used in frauddetection. Besides, there are also researchers who use anentity relationship network [20] to infer possible fraudulentactivity. In recent years, more and more deep learning mod-els are proposed. Generate adversarial network (GAN) [9] isproposed to generate adversarial samples and simulate thedata distribution to improve the classification accuracy, andnew deep learning methods are applied in this field. Zhenget al. [33] use a GAN based on a deep denoising autoencoderarchitecture to detect telecom fraud.

Many researchers focus on the imbalanced data problem.In the real world, fraud users account for only a small por-tion, which will lower the model’s performance. Traditionalsolutions are oversampling minority class [6]. It does not fun-damentally solve this problem. Zhang et al. [32] construct aclustering tree to consider imbalanced data distribution. Li etal. [14] propose a Positive Unlabeled Learning (PU-Learning)model that can improve the performance by utilizing posi-tive labeled data and unlabeled data in detecting deceptiveopinions.

FraudJudger: Real-World Data Oriented Fraud Detection on Digital Payment Platforms , ,

Some researchers choose unsupervised learning and semi-super-vised learning due to lack of enough labeled data in the real-world application. Unsupervised learning methods requireno prior knowledge of users’ labels. It can learn data dis-tributions and have potential in finding new fraud users.Zaslavsky et al. [31] use Self-Organizing Maps (SOM) to ana-lyze transactions in payment platforms to detect credit cardfrauds. Roux et al. [22] proposed a cluster detection basedmethod to detect tax fraud without requiring historic labeleddata.In our work, we use semi-supervised learning to detect

fraud users, and an unsupervised method is applied in ana-lyzing fraud users patterns and finding potential fraud users.

Adversarial AutoencoderAdversarial Autoencoder (AAE) is proposed by Makhzaniet al [17]. AAE is a combination of autoencoder (AE) andgenerative adversarial network (GAN). Like GAN, AAE hasa discriminator part and a generative part. The encoder partof an autoencoder can be regarded as the generative part ofGAN. The encoder can encode inputs to latent vectors. Themission of AAE’s discriminator is discriminating whetheran input latent vector is fake or real. The discriminator andgenerator are trained in an adversarial way. AAE is trained inan unsupervised way, and it can be used in semi-supervisedclassification.

Autoencoder. AE is a feedforward neural network. The infor-mation in its hidden layer is called latent variable z, whichlearns latent representations or latent vector of inputs. Re-cently, AE and its variants like sparse autoencoder (SAE)[18], stacked autoencoder [26] have been widely used indeep learning. Basic autoencoder consists of two parts: theencoder part and the decoder part. Encoders and decodersconsist of two or more layers of fully connected layers. Theencoding process is mapping the original data feature x tothe low-dimensional hidden layer z.

z = p(x) (1)The decoding process is reconstructing the latent variable

z to the output layer x ′ whose dimension is the same as x .

x ′ = q(z) (2)

Finally, it optimizes its own parameters according to theMean Square Error loss (MSE-loss) of x and x ′.

AEloss =1n

n∑i=1

(x − x ′)2 (3)

Generative adversarial autoencoder. GAN is first proposed byIan Goodfellow et al. [9] as a new generation model. Onceproposed, it becomes one of the most popular models in the

deep learning field. The model mainly consists of two parts,generatorG and discriminator D. The model can learn theprior distribution p(x) of training data and then generatedata similar to this distribution. The discriminator model isa binary-classifier that is used to distinguish whether thesample is a newly generated sample or the real sample. GANis proposed based on game theory, and its training processis a process of the mutual game. In the beginning, the gener-ator generates some bad samples from random noise, whichis easily recognized by the discriminator, and then the gen-erator can learn how to generate some samples that makethe discriminator difficult to discriminate or even misjudge.In each training round, GAN will update itself through theprocess of loss back-propagation. After multiple rounds ofthe game, the fitting state is finally reached, that is, the gen-erator can generate samples that hard to be distinguished bythe discriminator. Here is the loss function of GAN:

minG

maxDEx∼pdata [logD(x)]+Ez∼p(z) log (1 − D(G(z))) (4)

where G represents the generator, D represents the dis-criminator, and D(x) represents a neural network that com-putes the probabilities that x is real-world samples ratherthan the generator. p(z) represents the distribution of thenoise samples z, and D(G(z)) represents a neural networkthat computes the probabilities that a sample is generatedby the generator.

Comparing AAE to GAN. Adversarial autoencoder and gen-erative adversarial network have many similarities in unsu-pervised learning. Both of them can learn data distributions.However, when dealing with discrete data, it will be hard tobackpropagate gradient by GAN [29]. Many discrete featuresare encoded by one-hot encoding methods. The discrimina-tor of GAN may play tricks that it only needs to discriminatewhether the inputs are encoding in the one-hot format. AAElearns latent representations of input features and encodesthem as continuous data. The discriminator part of AAE fo-cuses on discriminating latent representations rather thaninitial inputs so that the original data type of input featuresdoes not matter. So AAE has advantages in dealing withdiscrete features comparing with GANs. In digital paymentplatforms, many features are discrete, which need to be en-coded in one-hot format. It is more reasonable to chooseAAE.

3 FRAUD DETECTION PARADIGMIn this section, we present the traditional fraud detectionparadigm on the electronic payment platform first, and thenwe show our improved paradigm.

, , Ruoyu Deng and Na Ruan

Traditional Fraud Detection ParadigmMany digital payment platforms have been devoted to frauddetection for many years. These platforms have their ownfraud users blacklists, and they track and analyze fraud userson the blacklists continuously. They have concluded manydetection rules based on years of experience. As shown inFig 1(a), platforms can use these detection rules to detectnew fraud users and build a larger fraud user blacklist. It isimpossible to gather all kinds of fraud users in this blacklist.There always exists unknown fraud users. These unknownfraud users cannot be detected based on existing detectionrules. Moreover, if fraud users change their behavior patterns,they can escape from being detected by the platforms easily.In the traditional fraud detection paradigm, platforms cannotfind new fraud patterns until new patterns of fraud usersappear after they cause visible accidents on platforms.

The limitations of traditional fraud detection paradigm areobvious. Platforms can only detect fraud users by existingknowledge. It is a passive defending paradigm. Platformscannot update their detection rules to defense unknownfraud users automatically, which makes themselves vulnera-ble when new patterns of fraudsters appear.

(a) Traditional fraud detection paradigm

(b) Improved fraud detection paradigm

Figure 1: Fraud detection paradigms

Improved Fraud Detection ParadigmTraditional fraud detection paradigm cannot update detec-tion rules until visible accidents happen, which will causehuge losses to platforms and other users. It is better to havean improved paradigm that can update its blacklist automat-ically. Platforms can derive new detection rules from newdetected users. The improved fraud detection paradigm isshown in Fig 1(b).The most critical part of the improved paradigm is au-

tomatically updating fraud users blacklist. It requires theparadigm can detect new fraud users from unknown users,especially potential fraud users who can not be detected byexisting detecting rules. Compared with traditional frauddetection paradigm, our improved paradigm can find newpatterns of fraud users before they appear in large numbersand cause huge losses to others. The potential fraud users de-tection is offline, and the cost for the automatically updatingpart will not have much influence in practice.Based on the improved fraud detection paradigm, we de-

sign a creative fraud users detection model, FraudJudger.It can detect fraud users and actively identify fraud usersbeyond existing detecting rules from unknown users.

4 FRAUDJUDGER: FRAUD DETECTION MODELModel OverviewIn this section, we introduce our fraud detection model,FraudJudger. As shown in the new detection paradigm, ourmodel should contain two main functions: building fraudusers blacklist and automatically updating the blacklist. Theoverview of our detection model is shown in Fig 2. The wholedetection model contains three phases:

Phase I:Merging features from raw data. Platforms col-lect two main classes of information: operation data andtransaction data. We first merge the two classes data, andthe merged features are passed to our fraud detection part.

Phase II: Building fraud users blacklist. Merged featuresin Phase I are of high dimensions, which can not be useddirectly. We can omit ineffective and noisy features and getefficient low dimension features with the help of adversarialautoencoder. In the meantime, we can detect fraud users bya few labeled data. It is a semi-supervised learning process.Detected fraud users will be added in the blacklist.

Phase III: Updating fraud users blacklist by cluster anal-ysis. The key idea is to find new fraud users beyond existingdetection rules. In this phase, we train a new AAE networkwithout labels to learn the latent representations of users.Then we cluster these latent variables from the network.After clustering latent variables, different users groups areformed. We detect fraud users with new patterns from thesefraud groups.

FraudJudger: Real-World Data Oriented Fraud Detection on Digital Payment Platforms , ,

Figure 2: The overview of FraudJudger

Merging FeaturesMany electronic payment platforms record users’ operationand transaction information. Operation data contain users’operation actions on payment platforms, such as device in-formation, operation type (changing password, viewing bal-ance, etc), operation time, etc. Transaction data contain user’stransaction information, such as transaction time, transac-tion amount, transaction receiver, etc.

A user may leave many operation and transaction recordson the electronic payment platform. We proposed an appro-priate method to merge a user’s records on the platform.We merge the two kinds of features by the key feature,

which is "user id". It means that features belong to the sameuser will be merged. The value of each feature can be dividedinto two types, numeric and non-numeric. Numeric featurescan be analyzed directly. For non-numeric features, suchas location information or device types, we use one-hot en-coding method to convert them into a numeric vector. Eachnon-numeric feature is mapped into a discrete vector afterone-hot encoding. If two features fi and fj have connections,we will construct a new feature fnew to combine the twofeatures. The new feature fnew contains statistic propertiesof fi and fj .

Building fraud users blacklistIn this phase, our model detects fraud users and build a fraudusers blacklist. The main purposes of our models in thisphase are:(1) Learning latent vectors of users.(2) Detecting fraud users by latent vectors with a small

scale of labels.The dimension of merged features from Phase I is too high

to analyze directly for the following reasons:(1) Raw data contain irrelevant information, which is noise

in our perspective. These irrelevant features will waste

computation resources and affect the model’s perfor-mance.

(2) High dimension features will weaken the model’s gen-eralization ability. Detection model will be easily over-fitted.

We should reduce the dimension of features and onlyleave essential features. Furthermore, if we manually chooseimportant features based on experience, it will be hard tofind new fraud patterns when fraud users change their attackpatterns.

We build a semi-supervised AAE (semi-AAE) based frauddetection model to learn the latent representations of mergedfeatures and classify users. Our detection network containsthree parts: encoder E, decoder E ′ and two discriminatorsD1 and D2. Fig 3 shows the architecture of our detectionnetwork.

Figure 3: Architecture of semi-AAE network for fraud detec-tion in FraudJudger

, , Ruoyu Deng and Na Ruan

Encoder: For an input merged feature x , encoder E willlearn the latent representation z of x . The dimension of thelatent variables z is less than the dimension of the inputx , and the encoder’s network structure determines it. Theencoding procedure can be regarded as dimensionality re-duction. Besides, it will output an extra one-hot variable yto indicate the class of input value, which is a benign useror fraud user in our model. Our model uses y to classifyan unknown user. The inner structure of the encoder is amulti-layer network in our model.

E(x) = (y, z) (5)

Decoder: The purpose of the decoder is learning how toreconstruct the input of the encoder from encoder’s outputs.The decoder’s procedure is the inverse of the encoder. Inputsof the decoder E ′ are outputs of the encoder E. The decoderwill learn how to reconstruct inputs x from y and z. Theoutput of the decoder is x ′. The inner structure of the decoderis also the inverse of the inner structure of the encoder.

E ′(y, z) = x ′ (6)

Loss of Encoder-Decoder: The loss of the encoder andthe decoder Le−d is defined by mean-square loss betweenthe input x of the encoder and output x ′ of the decoder. Itmeasures the similarity between x and x ′.

Le−d = E((x − x ′)2) (7)

Generator: Encoding the classy and latent vectors z fromx can be regarded as the generator. Let p(y) be the priordistribution of y, which is the distributions of fraud usersand benign users in the real world. And p(z) is the priordistribution of z, which is assumed as Gaussian distribution:z ∼ N(µ, σ 2). The generator tries to generate y and z intheir prior distributions to fool the discriminators. The lossfunction of the generator LG is:

LG = −E(loд(1 − D1(z)) + loд(1 − D2(y))) (8)

Discriminator: Like the discriminator of GAN, we usediscriminators in our model to judge whether a variable isreal or fake. Since the encoder has two outputs, y and z,we have two discriminators to discriminate them separately.The discriminators will judge whether a variable is in thereal distribution. The loss function of discriminators LD aredefined as:

LD1 = −E(azloд(D1(z)) + (1 − az )loд(1 − D1(z)))LD2 = −E(ayloд(D2(y)) + (1 − ay )loд(1 − D2(y)))LD = LD1 + LD2

(9)

where az , ay are the true labels (fake samples or real) ofinputs z and y. The total loss of the discriminator part is thesum of each discriminator.

Classifier: We can teach the encoder to output the rightlabel y with the help of a few samples with labels. And theloss function LC is:

LC = −E(a′yloд(y) + (1 − a′y )loд(1 − y)) (10)

where a′y means the right label (fraud samples or benign)for a sample, and y is the output label from the encoder.When the encoder outputs a wrong label, the classifier willback-propagate the classification loss and teach the encoderhow to predict right labels correctly.

Training Procedure: The generator generates like thereal label information y and latent representations z by theencoder network. Two discriminators try to judge whetherthe inputs are fake or real. It is a two-player min-max game.The generator tries to generate true values to fool discrim-inators, and discriminators are improving discriminationaccuracy. Both of the generator and discriminators will im-prove their abilities simultaneously. For samples with labels,they can help to increase the classification ability of ourmodel.

Once the training of the semi-AAE model finishes, we canuse it to classify users and build a fraud users blacklist.

Updating fraud users blacklistThe fraud detection part of FraudJudger can help us identifyusers based on rules we have known. We also hope that ourmodel can detect users beyond existing detecting rules thatwe have known. In this section, we will teach our model howto learn unknown rules. The intuition is that we can identifypotential fraud users with new fraud patterns. The architec-ture of the fraud updating part is shown in Fig 4. It containstwo parts, learning latent representations of unknown usersby AAE network, and finding new fraud patterns from latentrepresentations.

Figure 4: Architecture of detecting new fraud patterns inFraudJudger

FraudJudger: Real-World Data Oriented Fraud Detection on Digital Payment Platforms , ,

Learning latent representations from unknown users. First,we build another adversarial autoencoder network to learnlatent representations of new users without labels. As we cansee from Fig 4, the network of learning latent representationsin this phase has slight changes comparing with the networkin Fig 3.

Since we need to find new fraud patterns from unknownusers, we do not have label information in training data. Wewant to learn appropriate latent representation z of input xso that we can analyze users by z. The first change of the net-work is that the encoder only outputs latent representationz instead of label information y. All information of a user iscontained in the latent representation. The discriminator forlabel information is deleted as well. Since we have no datawith labels, we do not have loss function for classification aswell. The training phase of this network is also a two-playermin-max game.

Cluster analysis to find potential fraud users. After learningusers’ latent representations, we can find potential fraudusers from unknown users by cluster analysis. Algorithm 1shows the procedure of finding potential fraud users.

First, we classify unknown users by the classifier we con-struct in Section 4. Detected fraud users in this step areidentified based on existing detection rules. Our aim is find-ing potential fraud users who have unknown fraud patterns.Potential fraud users will be wrongly classified as benignusers in this step. We cluster users by their latent represen-tations. Fraud users’ behavior patterns are different frombenign users’. Cluster analysis can cluster users with similarbehavior patterns in the same group. We choose k-meansas our cluster analysis method. Users will form k clustersby k-means. Ideally, users can be divided into two groupsby clustering, one is fraud users group, and the other oneis benign users group. However, it does not work in reality.Both of benign users and fraud users contain various kindsof behavior patterns on the electronic payment platforms. Itis not reasonable to only cluster users into two groups. Anappropriate cluster number ncluster should be chosen. Andusers will be partitioned into ncluster groups. Each groupcontains users in a similar behavior pattern. Each group willbe a benign user group or fraud user group depending onthe ratio of detected fraud users. If the ratio is larger than athreshold tf raud , we regard this group as fraud users group.If we find a user who is judged as a benign user in a frauduser group, he has high similarities with fraud users in hisgroup, which indicates that he is more likely to be a frauduser in practice.The intuition of our potential fraud users detection algo-

rithm is that fraud users will gather together by clustering,and the learned latent representations ensure it. In traditionaldetection methods, features are manually chosen based on

Algorithm 1: Finding Potential Fraud UsersInput: Set of unknown users U = {u1,u2, ...,un} ;ClassifierCl f // The classification model we constructed;Cluster numbers ncluster , the threshold of fraud usersratio tf raud ;Output: Potential fraud users set Up

1 Set of users’ predicted labels Y = {y1,y2, ...,yn} ;2 Set of clustering group G = {д1,д2, ...,дncluster } ;3 for i = 1, ...,n do4 yi = Cl f (ui ) // Classify each user by FraudJudger;5 end6 G = Cluster (U,ncluster ) // Cluster users to ncluster

groups;7 for i = 1, ...,ncluster do8 Calculate the ratio of fraud users rf aud ;9 if rf aud > tf raud then

10 for user uj in group i do11 if yj = benign users then12 Add uj to Up

13 end14 end15 end16 end

historical knowledge. We cannot find new fraud patternsin this way. In our FraudJudger model, we can learn latentrepresentations of users without prior fraud detection knowl-edge. Even if fraud users change part of their fraud patterns,our model can still detect them.

If potential fraud users are found, we can update our back-list and new detection rules can be derived by platforms.

5 EXPERIMENTIn this section, we first describe the real-world dataset weuse in Section 5. Then we evaluate FraudJudger’s detectionperformance in Section 5. In order to intuitively showing thelatent representations, we visualize the latent vectors of usersin Section 5. Next, we evaluate the performance of clusteranalysis in Section 5. Finally, we find the best dimension oflatent representations in Section 5.

Dataset DescriptionWe use a real-world dataset from Bestpay, which is a populardigital payment platform. The dataset has been anonymizedbefore we use in case of privacy leakage. The dataset containsmore than 29,000 user’s operation behaviors and transactionbehaviors in 30 days. All users in the dataset are manually la-beled. We regard labels in this dataset as ground truth. In this

, , Ruoyu Deng and Na Ruan

dataset, the amount of fraud users is 4,046, which accountsfor 13.78% of total users. The dataset contains two kinds ofdata, one is operation data, and the other one is transactiondata. There are 20 features in operation data and 27 featuresin transaction data. Some important operation features arelisted in Table 1, and part of important transaction featuresare listed in Table 2.

Table 1: Part features in operation data

Feature Explanation Missing rate

mode user’s operation type 0%time operation time 0%device operation device 29.3%version operation version 19%

IP device’s IP address 18.0%MAC device’s MAC address 89.9%os device’s operation system 0%

geo_code location information 33.9%

Table 2: Part features in transaction data

Feature Explanation Missing rate

time transaction time 0%device transaction device 34.2%

tran_amt transaction amount 0%IP device’s IP address 14.6%

channel platform type 0%acc_id account id 62.1%balance balance after transaction 0%

trains_type type of transaction 0%

As shown in Table 1 and Table 2, there are some commonfeatures in both operation data and transaction data.We havesome essential and useful information, like IP address, time,amounts, etc, to analyze a user’s behavior. After mergingfeatures, we get 2174 dimensions of features for each user.Some features have a high missing rate, like acc_id . We filterout features with a missing rate more than 30%, and get940-dimensional merged features for each user. FraudJudgerwill analyze the 940-dimensional merged features to detectfrauds.

Detection PerformanceIn this experiment, we evaluate the detection ability of Fraud-Judger. First, we compare the model’s performance with dif-ferent proportions of labeled data. Then, we compare Fraud-Judger with other well-known supervised detection methods.

Different proportions of labeled data. We use 20,000 samplesfor training and the rest 9,354 samples for evaluating. Toevaluate the performance of our model, we set five groups ofexperiments with different proportions of labeled samples:

• 1% samples with labels.• 2.5% samples with labels.• 5% samples with labels.• 10% samples with labels.• 25% samples with labels.

The dimension of latent representations in our experimentis 100. We use a five-layer neural network as the structureof our encoder and decoder. The number of neurons in eachhidden layer is 1024. Models are trained in 500 epochs, andthe batch size in training is 200.We use four acknowledged standard performance mea-

sures to evaluate our model, which is precision, accuracy,recall and F-1 score. Precision is the fraction of true detectedfraud users among all users classified as fraud users. Accu-racy is the proportion of users who are correctly classified.Recall is intuitively the ability of the model to find all thefraud samples. F1-score is a weighted harmonic mean ofprecision and recall.

1F1 − score

=1

Precsion+

1Recall

(11)

And the result is shown in Fig 5.

Figure 5: Accuracy, Precision, Recall and F1 Score of models

Fig 5 illustrates our model FraudJudger can achieve betterperformance with more labeled training samples. When theratio of labeled samples is less than 10%, the performanceof the model increases rapidly as the ratio of labeled sampleincreases. When the ratio of labeled samples is more than10%, the increasing speed slows down. As we mentionedbefore, it is hard to obtain enough labeled samples to train ourmodel in practical application. We do not need to use a large

FraudJudger: Real-World Data Oriented Fraud Detection on Digital Payment Platforms , ,

number of manually labeled data. Our experiment resultshows that FraudJudger can achieve excellent classificationperformance with a small ratio of labeled data.

Compared with supervised models. We compare our model’sclassification performance with other supervised classifi-cation models. Three different excellent machine learningmodels are chosen: Linear Discriminant Analysis (LDA), Ran-dom Forest and Adaptive Boosting model (AdaBoost). Weset three groups of FraudJudger models with 5% labels, 10%labels, and 20% labels, respectively. We use the ROC curveto evaluate the result. The result is shown in Figure 6, andthe AUC of each model is shown in Table 3.

Figure 6: ROC of FraudJudger and other models

Table 3: AUC of FraudJudger and other models

Models AUC

FraudJudger-5%labels 0.944FraudJudger-10%labels 0.983FraudJudger-20%labels 0.985

LDA 0.946Random Forest 0.930

AdaBoost 0.975

As we can see from the result, the model’s detection ac-curacy increases with more labeled training data. When theproportion of labeled data is larger than 10%, FraudJudgeroutperforms all other supervised classification models. If weuse fewer labels, FraudJudger still has satisfying performance.Compared with other supervised algorithms, we save morethan 90% work on manually labeling data and we achievebetter performance.

In conclusion, FraudJudger has an excellent performanceon fraud users detection even with a small ratio of labeleddata. Comparing with other supervised fraud detection meth-ods, FraudJudger has a low requirement for the amount oflabeled data. Our model can be applied in more realisticsituations.

Visualization of Latent RepresentationFraudJudger uses learned latent representations to detectfraud users. In order to have an intuitively understandingof the latent representations, we use t-SNE [16] to visual-ize the latent representations learned from FraudJudger. T-SNE is a practical method to visualize high-dimensional databy giving each data point a location in a two-dimensionalmap. Here we choose the dimension of latent representa-tions equals to 100, and the ratio of labeled data is 10%. Thevisualized result of t-SNE is shown in Fig 7.

Figure 7: Visualization of latent representations by t-SNE

As we can see from Fig 7, the red points represent fraudusers, and blue points represent benign users. Fraud usersand benign users are well separated by latent representations.Benign users gather together and benign users are isolated tobenign users. It means that the latent representations learnedfrom FraudJudger can well separate benign users and fraudusers.Furthermore, we cluster latent representations into five

groups by K-means, and we visualize the result in Fig 8.Fig 8 contains five different colors, and five colors repre-

senting five different groups of users. It is hoped that benignusers and fraud users will form different groups after clus-tering, and the cluster result verifies it. The dividing lines

, , Ruoyu Deng and Na Ruan

Figure 8: Visualization of cluster result of latent representa-tions

between different groups are quite apparent. ComparingFig 8 with Fig 7, we can find that most fraud users are clus-tered into the same group in Fig 8. The fraud users in Fig 7are corresponding to the purple group in Fig 8. Benign userswith different behavior patterns are clustered into four differ-ent groups. Fraud users and benign users are well separatedby cluster analysis.

Cluster ResultIn this section, we measure the performance of cluster analy-sis. As we mentioned in Section 4, we use cluster analysis tofind potential fraud users. A more reasonable cluster resultindicates that more likely to find potential fraud users.After clustering, users are clustered into ncluster groups,

and each group contains fraud users and benign users. If agroup’s fraud users ratio is larger than the threshold tf raud ,we regard this group as fraud group. However, differentusers may gather together because of other criterions, suchas age, gender, etc. Different ages or genders of users willgather together instead of fraud users or benign users. Weshould measure whether users gather together with the rightcriterion.

We propose a new measurement R called "Cluster Recall"to measure the performance of the cluster result in gatheringfraud users into the same group. R equals to the ratio of thenumber of fraud users in fraud groups and the number ofall fraud users. A larger cluster recall indicates that morefraud users will gather into the same group, and the latent

variables are better in representing fraud behaviors. A morereasonable learned latent representations will lead to a bettercluster recall.

R = the number o f f raud users in f raud дroups

the number o f total f raud users(12)

We set the threshold tf raud equals to 0.7. And we havefour different models:

(1) Origin features: merged features without dimensional-ity reduction

(2) DAE: latent representations learned by DAE [22] (De-noising Autoencoder)

(3) VAE: latent representations learned by VAE [12] (Vari-ational Autoencoder)

(4) FraudJudger: our proposed model in learning latentrepresentations

In the first group, we cluster origin features directly. Bothof DAE and VAE are well-known unsupervised methodsto learn representations of a set of data. We use them ascomparison models. We use k-means as our cluster methods.We set ncluster = 2, 5, 100, and the result is shown in Table 4.

Table 4: Cluster Recall for different models

Number of cluster 10 50 100

Origin Features 0.05 0.18 0.36DAE 0.14 0.43 0.55VAE 0.09 0.32 0.42

FraudJudger 0.30 0.48 0.59

Table 4 shows that our proposed model FraudJudger hasa better performance than VAE and DAE in all three groupsof different cluster numbers. It indicates that our model canbetter learn behavior patterns of fraud users. When the clus-ter number increases, the cluster recall is larger. However,it is meaningless if we use a too large number of clusters. Ifncluster is too large, the number of users in each group afterclustering will be small. It will be hard for digital paymentplatforms to analyze users’ behavior patterns in this group.All of the dimensionality reduction models have a signifi-cant improvement compared with the group that uses originfeatures without dimensionality reduction. The experimentdemonstrates that fraud users with similar fraud patternswill gather together in our model. We can use our model tofind potential fraud users. If we set a lower tf raud , we canfindmore suspicious potential fraud users, and the credibilityof suspicious potential users will drop. We should balancethis in the real-world application.

FraudJudger: Real-World Data Oriented Fraud Detection on Digital Payment Platforms , ,

Dimension of Latent RepresentationsWe analyze the performance of different dimensions of latentrepresentations in FraudJudger to find the appropriate dimen-sions in practice. We use cluster recall R and adjust mutualinformation (AMI) [23] to measure models’ performance indifferent dimensions.

AMI is a variation of mutual information (MI) to comparetwo clusterings results. A higher AMI indicates the distribu-tions of the two groups are more similar. For two groups Aand B, the AMI is given as below:

AMI (A,B) = MI (A,B) − E(MI (A,B))avд(H (A),H (B)) − E(MI (A,B)) (13)

In our experiment, we calculate the AMI between datadistribution of real data and data distribution of cluster re-sults. We use 10% labeled data when training FraudJudger.Ten groups of different dimensions of latent representationsfrom 0 to 512 are conducted, and the result is shown in Fig 9:

Figure 9: Performance of different dimensions of latent rep-resentations

The result shows that the performance of FraudJudgervaries with dimensions of latent representations. When thedimension is too low (dimension = 1 or 2), cluster recall is 0.Nearly no fraud users gather together in the same group. Itshows that FraudJudger has poor performance in this case.Low dimensions of latent vectors cannot learn data distri-bution well. The model will lose many important featuresdue to too low dimensions. When the dimension is higher,latent vectors contain more information about origin data,and the model’s performance is better. When the dimen-sion is around 128, FraudJudger has the best performanceboth in cluster recall and AMI. When the dimension of la-tent representations is too high, the performance decreases

again. Under this circumstance, the model will learn a lot ofnoise features, which is harmful. Besides, a large dimensionwill lead to high model complexity and over-fitted. Thus,we should choose the dimension of latent representationsaround 128.

6 CONCLUSIONIn this paper, we proposed a novel fraud users detectionmodel FraudJudger on real-world digital payment platforms.FraudJudger can learn latent features of users from origi-nal features and classify users based on the learned latentfeatures. We overcome restrictions of real-world data, andonly a few labeled training data are required. Fraud pat-terns are diverse and evolving, and our proposed methodcan be used in finding potential fraud users from unknownusers, which is useful in anti-frauding. Our experiment isbased on a real-world dataset, and the result demonstratesthat FraudJudger has a good performance in fraud detection.Compared with other well-known methods, FraudJudger hasadvantages in learning latent representations of fraud usersand saves more than 90% manually labeling work. We haveseen broad prospects of deep learning in fraud detection.

REFERENCES[1] Aerospike. 2019. Enabling Digital Payments Transformation. Re-

trieved March 28, 2019 from https://www.aerospike.com/lp/enabling-digital-payments-transformation-ebook

[2] Mohiuddin Ahmed, Abdun Mahmood, and Md Rafiqul Islam. 2015. Asurvey of anomaly detection techniques in financial domain. FutureGeneration Computer Systems 55 (01 2015), 278–288. https://doi.org/10.1016/j.future.2015.01.001

[3] Alejandro Correa Bahnsen, Djamila Aouada, Aleksandar Stojanovic,and Björn Ottersten. 2016. Feature engineering strategies for creditcard fraud detection. Expert Systems with Applications 51 (2016), 134–142.

[4] Alex Beutel, Wanhong Xu, Venkatesan Guruswami, Christopher Palow,and Christos Faloutsos. 2013. Copycatch: stopping group attacks byspotting lockstep behavior in social networks. In Proceedings of the22nd international conference on World Wide Web (WWW). ACM, 119–130.

[5] Fabrizio Carcillo, Yann-Aël Le Borgne, Olivier Caelen, Yacine Kessaci,Frédéric Oblé, and Gianluca Bontempi. 2019. Combining Unsupervisedand Supervised Learning in Credit Card Fraud Detection. InformationSciences (2019).

[6] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W PhilipKegelmeyer. 2002. SMOTE: synthetic minority over-sampling tech-nique. Journal of artificial intelligence research 16 (2002), 321–357.

[7] Daniel de Roux, Boris Perez, Andrés Moreno, Maria del Pilar Villamil,and César Figueroa. 2018. Tax Fraud Detection for Under-ReportingDeclarations Using an Unsupervised Machine Learning Approach.In Proceedings of the 24th ACM SIGKDD International Conference onKnowledge Discovery & Data Mining. ACM, 215–222.

[8] Kang Fu, Dawei Cheng, Yi Tu, and Liqing Zhang. 2016. Credit cardfraud detection using convolutional neural networks. In Proceedings ofthe International Conference on Neural Information Processing (ICONIP).Springer, 483–490.

, , Ruoyu Deng and Na Ruan

[9] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, DavidWarde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014.Generative adversarial nets. In Proceedings of the Advances in neuralinformation processing systems (NIPS). 2672–2680.

[10] Ledisi G Kabari, Domaka N Nanwin, and Edikan Uduak Nquoh.2016. Telecommunications Subscription Fraud Detection Using NaïveBayesian Network. International Journal of Computer Science andMathematical Theory 2, 2 (2016).

[11] Santosh KC and Arjun Mukherjee. 2016. On the temporal dynamicsof opinion spamming: Case studies on yelp. In Proceedings of the 25thInternational Conference on World Wide Web (WWW). ACM, 369–379.

[12] Diederik P Kingma and Max Welling. 2013. Auto-encoding variationalbayes. arXiv preprint arXiv:1312.6114 (2013).

[13] S Lee, C Yoon, H Kang, Y Kim, Yongdae Kim, D Han, S Son, and S Shin.2019. Cybercriminal Minds: An investigative study of cryptocurrencyabuses in the Dark Web. In Network and Distributed Systems SecuritySymposium (NDSS).

[14] Huayi Li, Zhiyuan Chen, Bing Liu, Xiaokai Wei, and Jidong Shao. 2014.Spotting fake reviews via collective positive-unlabeled learning. InProceedings of the IEEE International Conference on Data Mining (ICDM).IEEE, 899–904.

[15] Cheng-Yuan Liou, Wei-Chen Cheng, Jiun-Wei Liou, and Daw-Ran Liou.2014. Autoencoder for words. Neurocomputing 139 (2014), 84–96.

[16] Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing datausing t-SNE. Journal of machine learning research 9, Nov (2008), 2579–2605.

[17] Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow,and Brendan Frey. 2015. Adversarial autoencoders. arXiv preprintarXiv:1511.05644 (2015).

[18] Andrew Ng et al. 2011. Sparse autoencoder. CS294A Lecture notes 72,2011 (2011), 1–19.

[19] Jiao Sun, Qixin Zhu, Zhifei Liu, Xin Liu, Jihae Lee, Zhigang Su, Lei Shi,Ling Huang, and Wei Xu. 2018. FraudVis: Understanding Unsuper-vised Fraud Detection Algorithms. In Proceedings of the IEEE PacificVisualization Symposium (PacificVis). IEEE, 170–174.

[20] Véronique Van Vlasselaer, Tina Eliassi-Rad, Leman Akoglu, MoniqueSnoeck, and Bart Baesens. 2016. Gotcha! Network-based fraud de-tection for social security fraud. Management Science 63, 9 (2016),3090–3110.

[21] Onur Varol, Emilio Ferrara, Clayton A Davis, Filippo Menczer, andAlessandro Flammini. 2017. Online human-bot interactions: Detec-tion, estimation, and characterization. In Proceedings of the Eleventhinternational AAAI conference on web and social media (ICWSM).

[22] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, andPierre-Antoine Manzagol. 2010. Stacked denoising autoencoders:Learning useful representations in a deep network with a local de-noising criterion. Journal of machine learning research 11, Dec (2010),3371–3408.

[23] Nguyen Xuan Vinh, Julien Epps, and James Bailey. 2010. Informationtheoretic measures for clusterings comparison: Variants, properties,normalization and correction for chance. Journal of Machine LearningResearch 11, Oct (2010), 2837–2854.

[24] Bimal Viswanath, M Ahmad Bashir, Mark Crovella, Saikat Guha, Kr-ishna P Gummadi, Balachander Krishnamurthy, and Alan Mislove.2014. Towards detecting anomalous user behavior in online social net-works. In Proceedings of the 23rd USENIX Security Symposium (USENIXSecurity). 223–238.

[25] Jarrod West and Maumita Bhattacharya. 2016. Intelligent financialfraud detection: a comprehensive review. Computers & Security 57(2016), 47–66.

[26] Jun Xu, Lei Xiang, Renlong Hang, and Jianzhong Wu. 2014. StackedSparse Autoencoder (SSAE) based framework for nuclei patch classifi-cation on breast cancer histopathology. In Proceedings of the IEEE 11thInternational Symposium on Biomedical Imaging (ISBI). IEEE, 999–1002.

[27] Wei Xu, Shengnan Wang, Dailing Zhang, and Bo Yang. 2011. Randomrough subspace based neural network ensemble for insurance frauddetection. In Proceedings of the Fourth International Joint Conferenceon Computational Sciences and Optimization. IEEE, 1276–1280.

[28] Yuanshun Yao, Bimal Viswanath, Jenna Cryan, Haitao Zheng, andBen Y Zhao. 2017. Automated crowdturfing attacks and defensesin online review systems. In Proceedings of the 2017 ACM SIGSACConference on Computer and Communications Security (CCS ’17). ACM,1143–1158.

[29] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. Seqgan: Se-quence generative adversarial nets with policy gradient. In Proceedingsof the 31th AAAI.

[30] Masoumeh Zareapoor, Pourya Shamsolmoali, et al. 2015. Applicationof credit card fraud detection: Based on bagging ensemble classifier.Procedia computer science 48, 2015 (2015), 679–685.

[31] Vladimir Zaslavsky and Anna Strizhak. 2006. Credit card fraud detec-tion using self-organizing maps. Information and Security 18 (2006),48.

[32] Youjun Zhang, Guanjun Liu, Lutao Zheng, Chungang Yan, andChangjun Jiang. 2018. A Novel Method of Processing Class Imbalanceand Its Application in Transaction Fraud Detection. In Proceedingsof the IEEE/ACM 5th International Conference on Big Data ComputingApplications and Technologies (BDCAT). IEEE, 152–159.

[33] Yu-Jun Zheng, Xiao-Han Zhou, Wei-Guo Sheng, Yu Xue, and Sheng-Yong Chen. 2018. Generative adversarial network based telecom frauddetection at the receiving bank. Neural Networks 102 (2018), 78–86.