- 1. Managing the Business Risk of Fraud using Sampling and Data
Mining Fall 2009 Mike Blakley Presented to:
2. PWC Global Survey Nov, 2009 Economic crime in a downturn
- Sharp rise in accounting fraud over the past 12 months
- Accounting fraud had grown to 38 percent of the economic crimes
in 2009
- Employees face increased pressures to :
3. Survey findings
- Greater risk of fraud due to increased incentives or
pressures
- More opportunitiesto commit fraud, partially due toreductions
in internal finance staff
- While companies are expecting more fraud, theyhave not done
much
- People who look for fraud are more likely to find it
4. Session objectives
- Understand the framework for managing the business risk of
fraud
- Plan, perform and explain statistical sampling in audits
- Reduce audit costs using data mining, sequential sampling and
other sampling techniques
- Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
- Run, hands-on, the most productive analytic technique
(regression analysis).
- Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
5. Session agenda - 1
- Introduction and the Process for Managing the Business Risk of
Fraud
- Framework of risk management for fraud
- Fundamentals of data mining
- Data mining: The Engine That Drives analysis
-
-
- Sources of Analytics Data
-
-
- Basic and Intermediate ARTs
- IIA Practice Advisory 2320
- The Yellow Book (2007 revision)
- The Guide Managing the Business Risk of Fraud
6. Session Agenda (contd) Sampling refresher
-
- Obtaining and Interpreting the results
- Other Sampling Approaches
7. Session Agenda (contd) Linear regression as anaudit tool
- Charting Regression Seeing Is Believing
-
- Calculation of Statistical Confidence Bounds
- Case Study - Wake County Schools Bus Maintenance
8. Session Agenda (contd) Data mining, or How to test 100%
- Data Conversion and Extraction
-
- Identification of extremes
-
- Major types of data analysis
9. Session Agenda (contd) Excel as an Analytics tool
-
- Adding a little class to your audit
10. Handout (CD)
- CD with articles and software
- More info at www.ezrstats.com
11. Cockroach theory of auditing
12. Cockroach theory of auditing
- There are probably 30 more that you dont see
13. Statistics based roach hunting Many frauds
coulda/woulda/shoulda been detected with analytics 14. Overview
- Fraud patterns detectable with digital analysis
- Basis for digital analysis approach
15. The Why and How
- ACFE/IIA/AICPA Guidance Paper
- Who, What, Why, When & Where
Objective 1 16. Example 1 Wake County Transportation Fraud
- Supplier Kickback School Bus parts
Objective 1a 17. Too little too late
- Understaffed internal audit
- Data on multiple platforms
- Transaction volumes large
Objective 1a 18. Preventable
- Need structured, objective approach
- Need efficient and effective approach
Objective 1a 19. Regression Analysis
- Stepwise to find relationships
Objective 1 20. Dataoutliers
- Sometimes an out and outLiar
Objective 1 21. Data Outliers
- Plot transportation costs vs. number of buses
Objective 1 22. Scatter plot with prediction and confidence
intervals 23. Cost of six types of AIDS drugs Example 2 Objective
1a 24. Medicare HIV Infusion Costs
- South Florida -$2.2Billion
- Rest of the country combined -$.1Billion
Objective 1 25. Pareto Chart Objective 1 26. Typical
Prescription Patterns Example 2 Objective 1a 27. Prescriptions by
Dr. X Example 2 Objective 1a 28. Off-label use
-
- Treat wasting syndrome, side effect of AIDS,OR
-
- Used by body builders for recreational purposes
-
- One physician prescribed $11.5 million worth (12% of the entire
state)
Example 2 Objective 1a 29. Revenue trends Example 3 Objective 1a
30. Dental Billings Example 3 Objective 1a 31. Guidance Paper
- A proposed implementation approach
- Managing the Business Risk of Fraud: A Practical
Guidehttp://tinyurl.com/3ldfza
- Coordinated Investigation Approach
Objective 1b 32. Managing the Business Risk of Fraud: A
Practical Guide
- ACFE,IIA andAICPA Exposure draft issued 11/2007, final
5/2008
- Section 4 Fraud Detection
Objective 1b 33. Guidance Paper
-
- Fraud Investigationand corrective action
34. Risk Governance
- Fraud risk management program
- Written policy managements expectations regarding managing
fraud risk
35. Risk Assessment
- Periodic review and assessment of potential schemes and
events
36. Fraud Prevention
- Establish prevention techniques
- Mitigate possible impact on the organization
37. Fraud Detection
- Establish detection techniques for fraud
- Back stop where preventive measures fail, or
- Unmitigated risks are realized
38. Fraud Investigation and Corrective Action
- Reporting process to solicit input on fraud
- Coordinated approach to investigation
39. 60 Minutes World of Trouble
-
- Fraud Risk Governance one grand wink-wink, nod-nod
-
- Fraud Risk Assessment - categorically false
-
- Fraud Prevention my husband passed away
-
- Fraud Detection -We didn't know? Never saw one.
-
- Fraud Investigationand corrective action - Pick-A-Payment
losses $36 billion
40. Section 4 Fraud Detection
- Proactive Fraud Detection
Objective 1b 41. Proactive Fraud Detection
- Data Analysis to identify:
Objective 1b 42. Fraud Detective Controls
- Operate in the background
- Not evident in everyday business environment
-
- Occur in ordinary course of business
-
- Corroboration using external information
-
- Automatically communicate deficiencies
-
- Use results to enhance other controls
43. Examples of detective controls
- Whistleblower hot-lines (DHHS and OSA have them)
- Process controls (Medicaid audits and edits)
- Proactive fraud detection procedures
44. Specific Examples Cited
- Journal entries suspicious transactions
- Identification of relationships
Objective 1b 45. Data Analysis enhances ability to detect
fraud
- Identify hidden relationships
- Identify suspicious transactions
- Assess effectiveness of internal controls
- Analyze millions of transactions
Objective 1b 46. Continuous Monitoring of Fraud Detection
- Organization should develop ongoing monitoring and
measurements
- Establish measurement criteria (and communicate to Board)
- Measurable criteria include:
47. Measurable Criteria number of
- fraud investigations resolved
- Employees attending annual ethics course
- Whistle blower allegations
- Messages supporting ethical behavior delivered by
executives
- Vendors signing ethical behavior standards
48. Management ownership of each technique implemented
- Each process owner should:
-
- Evaluate effectiveness of technique regularly
-
- Adjust technique as required
-
- Report modifications needed for techniques which become less
effective
49. Practice Advisory 2320-1 Analysis and Evaluation
- International standards for the professional practice of
Internal Auditing
- Analytical audit procedures
-
-
- Differences that are not expected
50. Analytical Audit Procedures
-
- Comparison of amounts with similar information in the
organization
-
- Comparison of amounts with similar information in the
industry
51. Analytical audit procedures
- Performed using monetary amounts, physical quantities, ratios
or percentages
- Ratio, trend andregression analysis
- Period to period comparisons
- Auditors should use analytical audit procedures in planning the
engagement
52. Factors to consider
- Significance of the area being audited
- Adequacy of system of internal control
- Availability and reliability of information
- Extent to which procedures provide support for engagement
results
53. Peeling the Onion Objective 1c Population as Whole Possible
Error Conditions Fraud Items 54. Fraud Pattern Detection Objective
1d Market Basket Stratification Trend Line Holiday Day of Week
Duplicates Univariate Gaps Benfords Law Round Numbers Target Group
55. Digital Analysis (5W)
Objective 1e A little about the basics of digital analysis. 56.
Who Uses Digital Analysis
- Traditionally, IT specialists
- With appropriate tools, audit generalists (CAATs)
- Growing trend of business analytics
- Essential component of continuous monitoring
Objective 1e 57. What - Digital Analysis
- Both numeric and non-numeric data
Objective 1e 58. How - Assessing fraud risk
- Software can do the leg work
- Statistical measures of difference
Objective 1e 59. Why - Advantages
- Handle large data populations
- Objective, quantifiable metrics
- Can be part of continuous monitoring
- Can produce useful business analytics
Objective 1e 60. Why - Disadvantages
- Costly (time and software costs)
- Requires specialized knowledge
Objective 1e 61. When to Use Digital Analysis
- Traditional intermittent (one off)
- Trend is to use it as often as possible
Objective 1e 62. Where Is It Applicable?
- Any organization with data in digital format, and especially
if:
-
- Data structures arecomplex
-
- Potential for fraudexists
Objective 1e 63. Disadvantages of digital analysis
-
- Skills not widely available
64. Objective 1 Summarized
- Who, What, Why, When & Where
Objective 1 65. Objective 1 - Summarized
- Understand the framework for managing the business risk of
fraud
- Plan, perform and explain statistical sampling in audits
- Reduce audit costs using data mining, sequential sampling and
other sampling techniques
- Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
- Run, hands-on, the most productive analytic technique
(regression analysis).
- Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is plan, perform 66. Statistical Sampling
- Attribute Sampling Compliance
- Variable Sampling Numeric Estimates
67. History of Sampling
- Basis is two laws/theorems of probability
68. Law of large numbers 69. Time Line - LLN
- Indian mathematician Bramagupta 600 AD
- Italian mathematician Cardon 1500s
- Statement without proof that empirical statistics improve with
more trials
70. Time line LLN (continued)
- Jacob Bernoulli first to prove in 1713
- Foundation for central limit theorem
71. Central limit theorem Classic measure Mean of a sufficiently
large number of random samples will be approximately normally
distributed. 72. The traditional explanation 73. Central Limit
Theorem
- Average is normally distributed
74. History of Central Limit Theorem
- French mathematician Abraham de Moivre
- 1733 approximate distribution from tossing coin
(heads/tails)
- French Mathematician LaPlace expanded it
75. History of CLT (contd)
- Russian mathematician Lyapunov
76. Industrial revolution Manufacturing Engineering Excitement!
77. Students T William Gosset - 1908 Guinness Brewery 78. SAS
39
- Exposure draft for revision in 2009
79. Attribute sampling
- Few software packages use it
80. Overview
81. Sample size calculation
- Every package different answer
- Need to know the population
- But thats why youre taking a sample!
82. Attribute Sampling Using RAT-STATS Unrestricted populations
83. Session Objectives
- Understand what is attribute sampling and when to use it
- Understand unrestricted populations
- Overview of the process using RAT-STATS
- Understand the formula behind the computations
84. Attribute sampling
- Signatures on approval documents, attachment of supporting
documentation, etc.
85. Statistical approach
- Requires determination of a sample size
86. Overview of process
- Determine the sampling objective
- Determine required sample size
- Identify samples to be selected based upon random numbers
- Compute the sampling results (i.e. estimate of range)
87. How this is done in RAT-STATS
- The sampling parameters are first developed by the auditor
- RAT-STATS is used to compute sample size
- RAT-STATS used to generate random numbers
- Enter results in RAT-STATS to compute estimates
88. Step 1 Develop sampling parameters
89. Step 2 Obtain the random numbers
- Done by entering info into RAT-STATS
- Output can be a variety of sources:
90. Step 3 Pull the sample
- Each random number selected corresponds with an item
- Put the selected item on a separate schedule
91. Step 4 - Test each selected item
- Generally requires reviewing documents
92. Step 5 Compute the results
- Enter summary information into RAT-STATS
- Output can be in a variety of formats:
93. Thats It!
- Now well see an actual demo using the RAT-STATS software
- Excel population of 5,000 invoices
- Results of test of attributes stored in the worksheet
94. Variable Sampling Using RAT-STATS Unrestricted populations
95. Session Objectives
- Understand what variable sampling is and when to use it
- Understand unrestricted populations
- Overview of the process using RAT-STATS
- Understand the formula behind the computations
96. Variable sampling
- Estimating account balances
- Estimating transaction totals
97. Statistical approach
- Requires determination of a sample size
98. Overview of process
- Determine the sampling objective
- Determine required sample size
- Identify samples to be selected based upon random numbers
- Compute the sampling results (i.e. estimate of range)
99. How this is done in RAT-STATS
- The sampling parameters are first developed by the auditor
- RAT-STATS is used to compute sample size
- RAT-STATS used to generate random numbers
- Enter results in RAT-STATS to compute estimates
100. Step 1 Develop sampling parameters
101. Step 1 Develop sampling parameters
102. Step 2 Obtain the random numbers
- Done by entering info into RAT-STATS
- Output can be a variety of sources:
103. Step 3 Pull the sample
- Each random number selected corresponds with an item
- Put the selected item on a separate schedule
104. Step 4 - Test each selected item
- Generally requires reviewing documents
- Example data contains both examined and audited value.
105. Step 5 Compute the results
- Enter summary information into RAT-STATS
- Output can be in a variety of formats:
106. Thats It!
- Now well see an actual demo using the RAT-STATS software
- Excel population of 5,000 invoices
- Audited values stored in the worksheet
107. Attribute Sampling Using RAT-STATS Stratified populations
108. Session Objectives
- Understand what is stratification and when to use it
- Overview of the process using RAT-STATS
109. Stratified sampling
- More efficient in some instances
110. Overview of process
- Determine the sampling objective
- Determine required sample size
- Identify samples to be selected based upon random numbers
- Compute the sampling results (i.e. estimate of range)
111. How this is done in RAT-STATS
- The sampling parameters are first developed by the auditor
- RAT-STATS is used to compute sample size
- RAT-STATS used to generate random numbers
- Enter results in RAT-STATS to compute estimates
112. Step 1 Develop sampling parameters
113. Step 2 Obtain the random numbers
- Done by entering info into RAT-STATS
- Output can be a variety of sources:
114. Step 3 Pull the sample
- Each random number selected corresponds with an item
- Put the selected item on a separate schedule
115. Step 4 - Test each selected item
- Generally requires reviewing documents
116. Step 5 Compute the results
- Enter summary information into RAT-STATS
- Output can be in a variety of formats:
117. Thats It!
- Now well see an actual demo using the RAT-STATS software
- Excel population of 5,000 invoices
- Results of test of attributes stored in the worksheet
118. Variable Sampling Using RAT-STATS Stratified populations
119. Session Objectives
- Understand what stratified sampling is and when to use it
- Populations benefiting from stratified sampling
- Overview of the process using RAT-STATS
- Understand the formula behind the computations
120. Stratified variable sampling
- Narrower standard deviation
121. Overview of process
- Determine the sampling objective
- Determine required sample size
- Identify samples to be selected based upon random numbers
- Compute the sampling results (i.e. estimate of range)
122. How this is done in RAT-STATS
- The sampling parameters are first developed by the auditor
- RAT-STATS is used to compute sample size
- RAT-STATS used to generate random numbers
- Enter results in RAT-STATS to compute estimates
123. Step 1 Develop sampling parameters
124. Step 1 Develop sampling parameters
125. Step 2 Obtain the random numbers
- Done by entering info into RAT-STATS
- Multi-stage random numbers
- Output can be a variety of sources:
126. Step 3 Pull the sample
- Each random number selected corresponds with an item in a
strata
- Put the selected item on a separate schedule
127. Step 4 - Test each selected item
- Generally requires reviewing documents
- Example data contains both examined and audited value.
128. Step 5 Compute the results
- Enter summary information into RAT-STATS
- Output can be in a variety of formats:
129. Thats It!
- Now well see an actual demo using the RAT-STATS software
- Excel population of 5,000 invoices
- Divided into three strata
- Audited values stored in the worksheet
130. Objective 2 - Summarized
- Understand the framework for managing the business risk of
fraud
- Plan, perform and explain statistical sampling in audits
- Reduce audit costs using data mining, sequential sampling and
other sampling techniques
- Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
- Run, hands-on, the most productive analytic technique
(regression analysis).
- Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is cost reduction 131. Techniques for cost reduction
- Optimize sample size (most bang for the buck)
- Skip sampling review 100% of transactions usingc omputera
ssisteda uditt echniques (CAATs)
132. Sample optimization
133. University of Hawaii
134. Sequential sampling
135. 100% test using CAATs
- Provides complete coverage
- Basis for continuous monitoring
136. Objective 3 - Summarized
- Understand the framework for managing the business risk of
fraud
- Plan, perform and explain statistical sampling in audits
- Reduce audit costs using data mining, sequential sampling and
other sampling techniques
- Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
- Run, hands-on, the most productive analytic technique
(regression analysis).
- Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is Yellow Book and SAS 56 137. Yellow book standards
Standards regarding statistical sampling and IT 138. General
standards
- 3.43 Technical Knowledge and competence
- The staff assigned to conduct an audit or attestation
engagement under GAGAS must collectively possess the technical
knowledge, skills, and experience necessary to be competent for the
type of work being performed before beginning work on that
assignment.
- The staff assigned to a GAGAS audit or attestation
- engagement should collectively possess:
139. Stat sampling and IT
- Skills appropriate for the work being performed. For example,
staff or specialist skills in
- (1)statistical sampling if the work involves use of statistical
sampling;
- (2)information technology
140. SAS 56 Analytical procedures
- Requiresuse of analytic review procedures for:
141. SAS 56 Analytical Review
- Encouragesuse of analytical review
A wide variety of analytical procedures may be useful for this
purpose. 142. Objective 4 - Summarized
- Understand the framework for managing the business risk of
fraud
- Plan, perform and explain statistical sampling in audits
- Reduce audit costs using data mining, sequential sampling and
other sampling techniques
- Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
- Run, hands-on, the most productive analytic technique
(regression analysis).
- Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is linear regression 143. Next Metric
144. Trend Busters Does the pattern make sense? 7 - Trends 145.
Trend Busters
- Sales areup , but cost of goods sold isdown
7 Trends 146. Purpose / Type of Errors
- Identify trend lines, slopes, etc.
- Key punch errors where amount is order of magnitude
7 Trends 147. Linear Regression
- Test relationships (e.g. invoice amount and sales tax)
- Perform multi-variable analysis
7 Trends 148. How is it done?
- Estimate linear trends using best fit
- Measure variability (standard errors)
- Sort descending by slope, variability, etc.
7 Trends 149. Trend Lines by Account - Example Results Generally
the trend is gently sloping up, but two accounts (43870 and 54630)
aredifferent .7 Trends 1.89 -0.560 56 54630 2.36 0.340 23 43870
0.36 1.010 32 43517 0.85 1.023 27 32451 4.3 1.070 17 43517 0.87
1.230 18 32451 Std Err Slope N Account 150. Scatter plot with
prediction and confidence intervals 151. Objective 5 -
Summarized
- Understand the framework for managing the business risk of
fraud
- Plan, perform and explain statistical sampling in audits
- Reduce audit costs using data mining, sequential sampling and
other sampling techniques
- Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
- Run, hands-on, the most productive analytic technique
(regression analysis).
- Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is data mining 152. Basis for Pattern Detection
- Isolate the significant few
Objective 6 153. Understanding the Basis
- Stat 101 Counts, Totals, Chi Square and K-S
Objective 2 154. Quantified Approach
- Based onmeasureable differences
Objective 2a 155. Detection of Fraud Characteristics
- Something isdifferentthan expected
Objective 2a 156. Fraud patterns
- Common theme something is different
- Group pattern is different than overall population
Objective 2b 157. Measurement Basis
Objective 2c 158. A few words about statistics(the s word)
- Detailed knowledge of statistics not necessary
- Software packages do the number-crunching
- Statistics used only to highlight potential errors/frauds
- Not used for quantification
Objective 2d 159. How is digital analysis done?
- Comparison of group with population as a whole
- Can be based on either counts or amounts
- Groups can then be ranked using a selected measure
- High difference = possible error/fraud
Objective 2d 160. Demo in Excel of the process
- Based roughly on the Wake County Transportation fraud
- Illustrates how the process works, using Excel
161. Histograms
- Attributes tallied and categorized into bins
Objective 2d 162. Two histograms obtained
Objective 2d 163. Compute Cumulative Amount for each Objective
2d 164. Are the histograms different?
- Two statistical measures of difference
- Both yield a difference metric
Objective 2d 165. Chi Squared
- Classic test on data in a table
- Answers the question are the rows/columns different
- Some limitations on when it can be applied
Objective 2d 166. Chi Squared
- Computationally intensive
Objective 2d 167. Kolmogorov-Smirnov
- Two Russian mathematicians
- Comparison of distributions
- Metric is the d-statistic
Objective 2d 168. How is K-S test done?
-
- For each cluster element determine percentage
-
- Then calculate cumulative percentage
-
- Compare the differences in cumulative percentages
-
- Identify the largest difference
Objective 2d 169. Kolmogorov-Smirnov Objective 2d - KS 170.
Classification by metrics
- Relationships (market basket)
Objective 2e 171. Auditors Top 10 Metrics
Objective e 172. Understanding the Basis
- Stat 101 Counts, Totals, Chi Square and K-S
Objective 2 173. Objective 2 - Summarized
- Understand statistical basis for quantifying differences
- Identify ten general tools and techniques
- Understand examples done using Excel
- How pattern detection fits in
Next are the metrics 174. Its that time! Session Break! 175. The
Top 10 Metrics
- Examples of what it can detect
Objective 3 176. Trapping anomalies Objective 3 177. Fraud
Pattern Detection Objective 3 Market Basket Stratification Trend
Line Holiday Day of Week Duplicates Univariate Gaps Benfords Law
Round Numbers Target Group 178. Outliers / Variability Outliers are
amounts which are significantly different from the rest of the
population 1 - Outliers 179. Outliers / Variability
- Software to analyze z-scores
- Top and Bottom 10, 20 etc.
- High and low variability (coefficient of variation)
1 - Outliers 180. Drill down to the group level
-
- Minimum, maximum and average
- Sort by statistic of interest
-
- Variability (coefficient of variation)
1 - Outliers 181. Example Results Two providers (3478421 and
2356721) had significantly more variability in the amounts of their
claims than all the rest. 1 - Outliers 18.54 2,311 5463122 23.25
3,421 3546789 87.23 4,536 2356721 342.23 3,243 3478421 Coeff Var N
Provider 182. Next Metric
183. Unusual stratification patterns Do you know how your data
looks? 2 - Stratification 184. Stratification - How
2 - Stratification 185. Purpose / types of errors
- Transactions out of the ordinary
- Up-coding insurance claims
- Based on either count or amount
2 Stratification 186. The process?
- Stratify the entire population into bins specified by
auditor
- Same stratification on each group (e.g. vendor)
- Compare the group tested to the population
- Obtain measure of difference for each group
- Sort descending on difference measure
2 Stratification 187. Units of Service Stratified - Example
Results Two providers (2735211 and 4562134) are shown to be much
different from the overall population (as measured by Chi Square).
2 Stratification 0.4632 298 2,503 4237869 0.5231 342 3,410 4321089
0.7453 5,234 8,913 4562134 0.8453 7,453 6,011 2735211 D-stat Chi Sq
N Provider 188. Next Metric
189. Day of Week
- Peak activity mid to late week
3 Day of Week 190. Purpose / Type of Errors
- Identify unusually high/low activity on one or more days of
week
- Dentist who only handled Medicaid on Tuesday
- Office is empty on Friday
3 Day of Week 191. How it is done?
- Programmatically check entire population
- Obtain counts and sums by day of week (1-7)
- For each group do the same procedure
- Compare the two histograms
- Sort descending by metric (chi square/d-stat)
192. Day of Week - Example Results Provider 2735211 only
provided service for Medicaid on Tuesdays.Provider 4562134 was
closed on Thursdays and Fridays.3 Day of Week 0.2189 56 7,905
4237869 0.321 87 5,162 4321089 0.8472 7,746 5,182 4562134 0.9802
12,435 5,404 2735211 D-stat Chi Sq N Provider 193. Next Metric
194. Round Numbers Its about. Estimates! 4 Round Numbers 195.
Purpose / Type of Errors
- Highlight account numbers in journal entries with round
numbers
- Split purchases (under the radar)
- Which groups have the most estimates
4 Round Numbers 196. Round numbers
- Classify population amounts
-
- $5,000 is a round number type 3 (3 zeros)
-
- $10,200 is a round number type 2 (2 zeros)
- Quantify expected vs. actual (d-statistic)
- Generally represents anestimate
4 Round Numbers 197. Round Numbers in Journal Entries - Example
Results Two accounts, 2735211 and 4562134 have significantly more
round number postings than any other posting account in the journal
entries.4 Round Numbers 0.2189 546 9,549 4237869 0.321 768 8,318
4321089 0.97023 35,324 833 4562134 0.9802 54,637 4,136 2735211
D-stat Chi Sq N Account 198. Next Metric
199. Made up Numbers Curb stoning Imaginary numbers Benfords Law
5 Made up numbers 200. What can be detected
- Made up numbers e.g. falsified inventory counts, tax return
schedules
5 Made Up Numbers 201. Benfords Law using Excel
- Basic formula is =log(1+(1/N))
- Workbook with formulae available
athttp://tinyurl.com/4vmcfs
- Obtain leading digits using Left function, e.g.
left(Cell,1)
5 Made Up Numbers 202. Made up numbers
- Check Chi Square and d-statistic
5 Made Up Numbers 203. How is it done?
- Decide type of test (first 1-3 digits, last 1-2 digit etc)
- For each group, count number of observations for each digit
pattern
- Based on total count, compute expected values
- For the group, compute Chi Square and d-stat
- Sort descending by metric (chi square/d-stat)
5 Made Up Numbers 204. Invoice Amounts tested with Benfords law
- Example Results During tests of invoices by store, two stores,
324 and 563 have significantly more differences than any other
store as measured by Benfords Law.5 Made Up Numbers 0.2189 312 74
217 0.321 476 23 432 0.97023 4,735 89 563 0.9802 5,234 79 324
D-stat Chi Sq Hi Digit Store 205. Next Metric
206. Market Basket
- References at end of slides
- Apriori Latin a (from) priori (former)
6 Market Basket 207. Purpose / Type of Errors
- Unexpected patterns and associations
- Based on market basket concept
- Unusual combinations of diagnosis code on medical insurance
claim
6 Market basket 208. Market Basket
- Credit card fraud in Japan taxi and ATM
6 Market basket 209. How is it done?
- First, identify groups, e.g. all medical providers for a
patient
- Next, for each provider, assign a unique integer value
- Create a text file containing the values
6 Market basket 210. Apriori outputs
- For each unique value, probability of other values
- If you see Dr. Jones, you will also see Dr. Smith (80%
probability)
- If you see a JE to account ABC, there will also an entry to
account XYZ (30%)
6 Market basket 211. Next Metric
212. Numeric Sequence Gaps Whats there is interesting, whats not
there is critical 8 - Gaps 213. Purpose / Type of Errors
- Missing documents (sales, cash, etc.)
- Inventory losses (missing receiving reports)
8 Gaps 214. How is it done?
- Check any sequence of numbers supposed to be complete,
e.g.
8 Gaps 215. Gaps Using Excel
- Sequential numbers and dates
8 Gaps 216. Gap Testing - Example Results Four check numbers
aremissing .8 Gaps 1 17548 17546 2 12526 12523 1 10791 10789
Missing End Start 217. Next Metric
218. Duplicates Why is there more than one? Same, Same, Same,
and Same, Same, Different 9 - Duplicates 219. Two types of
(related) tests
- Sameitems samevendor,sameinvoice number,sameinvoice
date,sameamount
- Differentitems sameemployee name,samecity,differentsocial
security number
9 Duplicates 220. Duplicate Payments
- Overriding software controls
9 - Duplicates 221. Fuzzy matching with software
- Regular expression testing in SQL
- Vendor/employee situations
Russian physicist 9 - Duplicates 222. How is it done?
- First, sort file in sequence for testing
- Compare items in consecutive rows
- Extract exceptions for follow-up
9 - Duplicates 223. Possible Duplicates - Example Results Five
invoices may beduplicates .9 - Duplicates 2 1,500.00 2/12/2007
17546 2 2,010.37 8/31/2007 10245 4 3,544.78 6/15/2007 10245 Count
Invoice Amount Invoice Date Vendor 224. Next Metric
225. Date Checking If were closed, why is there Adjusting
journal entry? Receiving report? Payment issued? 10 - Dates 226.
Holiday Date Testing
10 Dates 227. Date Testing challenges
- Floating holidays Friday, Saturday, Sunday, Monday
10 Dates 228. Typical audit areas
10 Dates 229. Determination of Dates
- Transactions when business is closed
- Federal Office of Budget Management
- An excellent fraud indicator in some cases
10 Dates 230. Holiday Date Testing
- Identifying holiday dates:
10 Dates 231. Federal Holidays
- Specific date (unless weekend),OR
10 Dates 232. Federal Holiday Schedule
- Office of Personnel Management
- Example of specific date Independence Day, July 4 th(unless
weekend)
- Example of floating date Martin Luther Kings birthday (3
rdMonday in January)
- Floating Thanksgiving 4 thThursday in November
10 Dates 233. How it is done?
- Programmatically count holidays for entire population
- For each group, count holidays
- Compare the two histograms (group and population)
- Sort descending by metric (chi square/d-stat)
10 Dates 234. Holiday Counts - Example Results Two employees
(10245 and 32325) were off the chart in terms of expense amounts
incurred on aFederal Holiday .10 Dates 0.2189 312 34 24135 0.321
476 18 17546 0.97023 4,735 23 32325 0.9802 5,234 37 10245 D-stat
Chi Sq N Employee Number 235. The Top 10 Metrics
- Examples of what it can detect
Objective 3 236. Objective 3 - Summarized
- Understand statistical basis for quantifying differences
- Identify ten general tools and techniques
- Understand examples done using Excel
- How pattern detection fits in
Next using Excel 237. Use of Excel
Objective 4 238. Excel templates
Objective 4 239. Excel Univariate statistics
- =largest(Range,1), =smallest(Range,1)
- Tools | Data Analysis | Descriptive Statistics
Objective 4 240. Excel Histograms
- Tools | Data Analysis | Histogram
Objective 4 241. Excel Gaps testing
- =if(thiscell-lastcell 1,thiscell-lastcell,0)
Objective 4 242. Detecting duplicates with Excel
- =if(=and(thiscell=lastcell, etc.))
Objective 4 243. Performing audit tests with macros
- Moreefficient and effective
- Examples -http://ezrstats.com/Macros/home.html
Objective 4 244. Using database audit software
- Many built-in functions right off the shelf with SQL
- June 2008 article in the EDP Audit & Control Journal
(EDPACS) SQL as an audit tool
- http://ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf
Objective 4 245. Use of Excel
Objective 4 246. Objective 4 - Summarized
- Understand statistical basis for quantifying differences
- Identify ten general tools and techniques
- Understand examples done using Excel
- How Pattern Detection fits in
Next Fit 247. How Pattern Detection Fits In
Objective 5 248. Where does Fraud Pattern Detection fit in?
- Continuous fraud pattern detection
Right in the middle Objective 5 249. Business Analytics
- Fraud analytics -> business analytics
- Business analytics -> fraud analytics
Objective 5 250. Role in Continuous Monitoring (CM)
- Fraud analytics can feed(CM)
- Continuous fraud pattern detection
- Use output from CM to tune fraud pattern detection
Objective 5 251. Objective 6 - Summarized
- Understand the framework for managing the business risk of
fraud
- Plan, perform and explain statistical sampling in audits
- Reduce audit costs using data mining, sequential sampling and
other sampling techniques
- Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
- Run, hands-on, the most productive analytic technique
(regression analysis).
- Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
252. Links for more information
- http://tinyurl.com/y49sec
- Benfords Lawhttp://tinyurl.com/3qapzu
- Chi Square testshttp://tinyurl.com/43nkdh
- Continuous monitoringhttp://tinyurl.com/3pltdl
253. Market Basket
- Apriori testing for ping ponging
- Temple Universityhttp://tinyurl.com/5vax7r
- Apriori program (open source)http://tinyurl.com/5qehd5
- Article Medical ping ponginghttp://tinyurl.com/5pzbh4
254. Excel macros used in auditing
- Excel as an audit softwarehttp://tinyurl.com/6h3ye7
- Selected macros -http://ezrstats.com/Macros/home.html
- Spreadsheets forever -http://tinyurl.com/5ppl7t
255. Questions? 256. Contact info
- Blog:http://blog.ezrstats.com