What Every IT Auditor Should Know About Sampling

8/12/2019 What Every IT Auditor Should Know About Sampling

1/3

One important aspect of IT audits is sampling

and sampling methodologies. It is important to

understand the different methodologies an auditor

could use and when to use which one. The choice

of methodology also affects the interpretation of

the results. For example, if the auditor discovers

one or two errors in the sample, what does that

mean? It could be that the methodology chosen

has an error rate that allows two errors in that

particular sample (which means there is no needto expand the sample), or it could be that the

methodology chosen allows no errors at all (which

means there is trouble of some sort, even if it is just

a larger sample and more work).

Many auditors rely on one of the standard

audit procedure support systems that provides

standardized forms for performing substantive

tests and includes charts for determining sample

size. It is tempting to rely totally on the packaged

sample information or form (i.e., pull form, find

sample size in chart, pull sample), rather than to

go through a rigorous process to decide whichsampling method

applies and what

the sample size and

potential deviations

mean to the audit.

Also, according to

some experts, the

trend today is to use less-rigorous, nonstatistical

sampling to reduce cost, and there is a risk

that such an approach may be substantially

less capable of detecting a material error than

a statistical approach, such as probability-proportional-to-size (PPS) sampling. The

downside of this rigorous statistical approach is

the complexity of statistical sampling concepts

and process (if done by hand). However, there

are a number of tools, such as Excel worksheets

and plug-ins, to facilitate the process.

Therefore, this article will attempt to

summarize the four most common statistical

methods used in audit, and provide some

guidance in applying those methods.

SAMPLING METHODOLOGIES

There are four basic sampling methodologies:

Attribute samplingThis type of sampling

enables the auditor to estimate the rate of

occurrence of certain characteristics of the

population (e.g., deviations from performance

of a control). It is most often used in

performing tests of controls. A deviation wou

be the failure of a control to function properly

(i.e., an error). Discovery samplingThis type of sampling is

designed to locate a small number of deviation

or exceptions in the population. It is most

often used to detect a fraudulent transaction.

If there is one deviation (i.e., one fraudulent

transaction) in the sample, the auditor must

examine the population. A deviation in

discovery sampling, however, is not the same

as a deviation in other sampling methods. In

the former, it refers to fraud; in the latter,

it refers to an error. Discovery sampling is

used primarily to detect critical deviations.Because they are

considered critical,

the discovery of a

single deviation (e.g.

fraud) is intolerable.

Consequently, if a

critical deviation

is discovered, the auditor may abandon the

sampling procedures and investigate the

population, rather than relying on the sample.

For fraud detection, a fraudulent transaction

or event would be considered critical. Ifusing discovery sampling to detect fraud,

and the auditor uncovers a simple US $300

transposition error in a transaction, that erro

would not be considered critical.

Classical variables sampling (CVS)This

method is used to provide auditors with an

estimate of a numerical quantity, such as the

balance of an account. It is primarily used

by auditors to perform substantive tests.

It includes mean-per-unit estimation, ratio

2009 ISACA. All rights reserved. www.isaca.org ISACA JOURNAL VOLUME 1, 2

Nonstatistical samplingmay be

substantially less capable of detecting a

material error than a statistical approach.

Tommie W. Singleton, Ph.D.,

CISA, CITP, CMA, CPA,is

an associate professor of

information systems (IS) at

the University of Alabama at

Birmingham (USA), a Marshall

IS Scholar and a director

of the Forensic Accounting

Program. Prior to obtaining his

doctorate in accountancy from

the University of Mississippi

(USA) in 1995, Singleton was

president of a small, value-

added dealer of accounting

IS using microcomputers.

Singleton is also a

scholar-in-residence

for IT audit and forensic

accounting at Carr Riggs

Ingram, a large regional

public accounting firm in the

southeastern US. In 1999,the Alabama Society of CPAs

awarded Singleton the

1998-1999 Innovative User of

Technology Award. Singleton

is the ISACA academic

advocate at the University of

Alabama at Birmingham. His

publications on fraud, IT/IS,

IT auditing and IT governance

have appeared in numerous

publications, including the

ISACA Journal.

What Every IT Auditor Should

Know About Sampling


2/3ISACA JOURNAL VOLUME 1, 2009 2009 ISACA. All rights reserved. www.isaca

estimation and difference estimation. For example, this

method would be used to confirm accounts receivable.

Probability-proportional-to-size samplingThis method

develops an estimate of the total monetary amount of

misstatement in a population. PPS uses dollar-unit sampling

or monetary-unit sampling (MUS). Other methods are based

on instances or occurrences, but this method is based on

monetary values, where higher monetary value transactions

have a higher likelihood of being chosen in a samplethus

the name PPS. MUS includes:

a. A tolerable misstatement amount (the total misstatementthe auditor will allow in the population)

b. Acceptable risk of incorrect acceptance (risk that the

sample does not support the conclusion about not being

materially misstated, i.e., a false-positive; generally

5 percent or 10 percent)

c. Acceptable risk of incorrect rejection (opposite of b; sample

shows material misstatement in population when it is not

materially misstated, i.e., a false-negative)

d. Assumption of average percent of misstatement (for items

misstated, the assumed average size of each misstatement

compared to the recorded amount)

MUS is often used in statistical examinations where the

purpose is fraud detection.

The American Institute of Certified Public Accountants

(AICPA) Statistical Sampling Subcommittee prepared an audit

guide in 1983, titledAudit Sampling, that describes PPS. The

audit guide lists several advantages of PPS over CVS.

CHOICE OF SAMPLING METHODOLOGY

The choice of a method depends on the primary purpose of the

sample and substantive test. If the auditor needs to perform a

test of control, the best choice is attribute sampling, generally

speaking. If the purpose of the audit procedure is to detect fraud,then discovery sampling is the best choice, but MUS is a good

choice, too. If the purpose is to look for material misstatements

in an account balance or class of transactions, CVS is a good

choice. But, CVS does tend to require larger samples than

other methods and is, therefore, costly. PPS requires smaller

samples. PPS is designed to be especially effective in the audit of

accounts receivable and inventory, with a few exceptions, and

thus is usually a better choice than classical variables for account

balances such as these. However, PPS is prone to trigger false-

positives, and the auditor must be aware of this possibility.

It is possible to use a different method from that generally

chosen, if there is an extenuating circumstance or objective.

Obviously, discovery sampling has a more stringent requirement

regarding deviations or exceptions, so is usually the prime

choice for fraud detection.

In discovery sampling, a key point is what is meant by critical

deviation. In particular, the standardized audit methodologies

indicate that if the auditor detects a fraudulent transaction, such

as an invoice from a shell (fictitious) vendor, that transaction

is considered a critical deviation. An identified deviation (or

anomaly), therefore, can be classified into two categories: Those that are clearly fraudulent or highly suspicious of fraud

Those that are clearly errors

According to the discovery sampling methodology, if a

fraudulent deviation (i.e., a critical deviation) is detected, then the

review of the sample should be stopped and the entire population

should be reviewed (this method is sometimes referred to as stop-

and-go sampling). The theory behind discovery sampling is that

the goal is zero critical deviations. As defined, that means zero

fraud. Because the purpose is to have zero tolerance for fraud,

the sample sizes tend to be

larger than other sampling

methodologies and, obviously,

have a significantly smaller

allowance for deviations.

However, that does not mean

that, if a deviation that is the

result of error is found, the auditor must stop and review the

population. In fact, the language of authoritative sources says the

auditor maydecide to review the population, not that the auditor

mustdo so.

EXAMPLE OF APPLICATION

What does it mean when a deviation occurs in the sample?

The following is an illustration of what would happen if two

different sampling techniques were used to examine a common

population for the purpose of fraud detection. The set of

circumstances for the illustration is as follows:

The population is 10,000 transactions.

The objective is the effectiveness of antifraud controls.

The IT auditor chose discovery sampling.

A sample size of 483 was taken, based on discovery

sampling table.

Two errors were discovered but neither had any

fraudulent implications.

What does it meanwhen a deviation

occurs in the sample?


3/32009 ISACA. All rights reserved. www.isaca.org ISACA JOURNAL VOLUME 1, 2

According to the discovery sampling rules, the two

occurrences were (minor) errors and, therefore, there were

no critical deviations. The conclusion is that the auditor could

rely upon the sample in assessing the likelihood of fraud, and

there is a 95 percent probability that no critical deviation

exists in this population.

If the auditor had used attribute sampling, because the

auditor was testing controls, the process and sample size would

have been different. If a 1 percent expected deviation rate is

assumed (typical rate), with a 7 percent tolerable deviation rate

and 95 percent confidence interval, the AICPA chart shows asample size of 66 (notice how much smaller the sample size

is for attribute sampling than for discovery sampling), with

one allowable actual

deviation. The 7 percent

is the top end of the low

level of assessed control

risk (2-7 percent), and

within the moderate

control risk (6-12

percent). If none or one

deviation was found

in a sample of 66, then according to attribute sampling, the

assessed level of control risk would not be too low, and the

controls are as effective as assessed. If more than one deviation

occurs in a sample of 66, the interpretation is that actual

control risk is higher than assessed.

Classical variables sampling is not applicable and is based

on monetary amount, or number of occurrences. PPS is

subject to monetary amounts and it is unknown what the

exact sample would have been determined using PPS.

CONCLUSION

According to Practitioners Guide to Audit Sampling, there are

several practical advantages for auditors who use statistical

sampling: less likelihood of over- or under-auditing, more

objective and defensible audit work, better work paper

documentation, and greater confidence in the audit opinion.

Therefore, it is important to understand and properly apply

sampling techniques. This article attempts to discuss the basics

of the four common statistical sampling methods used in IT

audit (and internal and financial audit as well). Auditors need

to take the time to conduct an informed and rigorous thoughtprocess when choosing a statistical method and to achieve

the appropriate interpretation of the results, if there are any

deviations or exceptions in the sample. A thorough approach

to sampling will generally lead to many advantages for the IT

auditor, including efficiency and effectiveness of the audit.

RESOURCES

Guy, Dan M.; D.R. Carmichael; O. Ray Whittingham,

Practitioners Guide to Audit Sampling, John Wiley

& Sons, 1998

Wampler, Bruce; Michelle McEacharn; MUS Using

Excel, CPA Journal Online, May 2005, www.nysscpa.org/

cpajournal/2005/505/essentials/p36.htm

New York State Society of CPAs, Software to Download,

The CPA Journal, www.cpajournal.com/down.htm

AICPA, Audit Guide,Audit Sampling

Yancey, Will; Comprehensive list of references and links

related to Sampling for Financial and Internal Audits,

www.willyancey.com/sampling-financial.htm

Take the time to

conduct an informed

and rigorous thought

process when choosing a

statistical method.

Documents

What Every IT Auditor Should Know About Sampling