A measure of perceived auditor ERP systems expertise: Development, assessment, and uses

Joseph. F. Brazel Department of Accounting

College of Management North Carolina State University Campus Box 8113, Nelson Hall

Raleigh, NC 27695 Telephone: 919-513-1772

Fax: 919-515-4446 e-mail: joe_brazel@ncsu.edu

October 2004

I appreciate the helpful comments of Greg Jenkins, Chris Agoglia, Rick Hatfield, two anonymous reviewers, the editor, and the support of the partners, administrative staff, and audit professionals that participated in this study.

A measure of perceived auditor ERP systems expertise: Development, assessment, and uses

ABSTRACT: The purpose of this study is to develop, assess, and provide uses for a measure of perceived enterprise resource planning (ERP) systems expertise for financial statement auditors. ERP systems are the dominant system used by the public company clients of audit firms. In such settings, the Theory of Planned Behavior suggests that auditor perceptions of their own ERP systems expertise should influence their perceived behavioral control and, in turn, explain auditor behavior. A five-item measure of perceived ERP systems expertise is developed from a review of audit expertise studies and the Theory of Planned Behavior. An assessment of the measure shows that it possesses a high level of reliability and construct validity (i.e., convergent and discriminant validity). Factor analysis results indicate that all five items satisfactorily load on one factor. The results also suggest that auditors perceptions of ERP systems expertise are not simply a by-product of general audit experience. Possible future uses of the measure, including whether the measure explains auditor control testing performance and the customization of the measure for IT audit specialists and internal auditors, are discussed. This studys measure should provide researchers with a tool to examine how auditor self-perceptions of ERP systems expertise can affect auditor behavior and the quality of contemporary audit services. Keywords: auditor, ERP systems, expertise, measure, Theory of Planned Behavior Data Availability: Data are available upon request.

Introduction

The purpose of this study is to develop, assess, and provide possible uses for a measure of

perceived financial statement auditor (hereafter, auditor) enterprise resource planning (ERP)

systems expertise. While the clients of audit firms use a variety of systems to process accounting

transactions, ERP systems are the dominant system environment for auditors servicing public

clients. In 1999, 70% of the Fortune 1000 firms had either implemented or planned to implement

ERP systems in the near future (Cerullo and Cerullo, 2000). In complex audit environments such

as ERP systems, auditors will need to expand their technological knowledge and skills in order to

perform effective and efficient audits (POB, 2000; Kinney, 2001). Ajzens (1991) Theory of

Planned Behavior suggests that auditors who perceive they have higher levels of ERP systems

expertise should perceive they have more behavioral control in ERP settings. Behavioral control

refers to the perception of the ease or difficulty of performing the behavior of interest (e.g.,

auditing in an ERP environment). Prior research in psychology has shown that perceived

behavioral control strongly influences individual behavior (e.g., Bandura, 1982, 1991).

Recent accounting research has shown that, in ERP system settings, (a) auditors are not

apt to recognize heightened inherent and control risks, and (b) auditors with higher perceived

ERP systems expertise are better able to use IT audit specialists and plan the scope of substantive

procedures to mitigate ERP system related risks (Hunton et al., 2004; Brazel and Agoglia, 2004).

Therefore, perceived auditor ERP systems expertise may be a significant determinant of auditor

behavior and audit quality in ERP environments. Unlike variables of interest to researchers that

are characteristic of audit engagements and can be manipulated in an experiment (e.g., integrity

of management), perceived auditor ERP systems expertise is a trait associated with the individual

auditor. As such, if researchers wish to examine the role of this variable in experimental or

2

survey research, a reliable and valid measure of perceived auditor ERP systems expertise must

be developed and assessed. While the investigation of this construct appears to be a fruitful area

for future research, no measure of this variable exists in the literature. Prior research examining

the construct of expertise and Ajzens (1991) Theory of Planned Behavior were reviewed to

create a multiple-item scale designed to measure auditors perceptions of ERP systems expertise.

The multiple-item scale was then assessed for reliability and construct validity with 73 practicing

audit seniors.

Assessment results for the measure indicate that it does possess a high level of reliability

and construct validity (i.e., convergent and discriminant validity). The Cronbachs alpha

associated with the items was found to be satisfactory and there is evidence that the measure is

stable over time. Factor analysis results indicate that all five items intended to measure auditor

self-assessed ERP systems expertise loaded heavily on one factor and this factor was labeled

perceived auditor ERP systems expertise. In addition, the results suggest that this perception is

not correlated with general audit experience. Possible uses of the scale are then discussed,

including the use of the measure in experimental and survey research, the advantages and

disadvantages of pre- and post-experimental measurement of auditors, and how the scale can be

customized for use with other auditors (i.e., internal, information systems) or specific IT settings.

Development of measure

Within complex system environments, auditors will find it necessary to increase their

systems expertise in order to provide them with the sophisticated audit skills required in such

settings (Lilly, 1997; POB, 2000). As noted above, ERP systems appear to be the system used by

the majority of large clients serviced by audit firms. ERP systems are defined as information

3

systems packages that integrate information and information-based processes within and across

functional areas in an organization (Kumar and Hillegersberg, 2000, p. 22). While the positive

benefits associated with ERP system adoption (e.g., compliance with the Sarbanes-Oxley Act of

2002) have led to their widespread implementation (Winters, 2004), the profession has

acknowledged that there are potential audit related risks associated with these complex systems

(e.g., POB, 2000). For example, supervisory review is typically minimal and supplemental

internal control applications are often not properly integrated with the ERP system (Turner,

1999; Wright and Wright, 2002). Given the complexity, pervasiveness, and potential risks

associated with these systems, expertise in ERP systems may explain auditor behavior in these

settings (Wright and Wright, 2002; Guess et al., 2000).

Chi et al. (1982, p. 8) define expertise as the possession of a large body of knowledge

and procedural skill. Prior research has shown that increased levels of expertise in specific

domains can improve the domain specific performances of auditors (Bonner, 1990; Bonner and

Lewis, 1990; Hunton et al., 2004). Expertise studies in auditing have mainly used the

contrastive cross-sectional approach, where groups of experts and novices are compared on

various dimensions (Bedard and Chi, 1993, p. 23). While these studies are useful in identifying

the differences between more and less experienced auditors, they offer little explanation for the

causes of these differences (Bedard and Chi, 1993) and do not examine why variation exists in

the performances of auditors with the same level of general audit experience (Bonner and Lewis,

1990).

Unlike other variables of interest to audit researchers that can be experimentally

manipulated (e.g., the complexity of a clients system), ERP systems expertise is a trait

associated with the individual auditor. Since one cannot readily manipulate factors such as forms

4

of intelligence, a reliable and valid measure of ERP systems expertise must be developed

(Peecher and Solomon, 2001). Bonner and Lewis (1990, p. 2) suggest that researchers should

pay more attention to the criteria used to designate subjects as experts, either directly by the use

of objective performance measures or indirectly by the use of well-specified measures of

knowledge and ability. Similar to other domains of auditor expertise, an objective performance

measure of ERP systems expertise would often be impractical for researchers to obtain

(Abdolmohammadi and Shanteau, 1992). What researchers can obtain from auditor participants

in their studies are auditors self-assessments of ERP systems expertise. Indeed, in their seminal

study of auditor expertise, Bonner and Lewis (1991) measured auditors self-reported expertise

levels and found these measures explained auditor performance. The Theory of Planned

Behavior (Ajzen, 1991) suggests that higher levels of perceived ERP systems expertise should

enhance the perceived ease of performing audit activities in an ERP setting. Such behavioral

control should influence individuals thought patterns, choice of activities, preparation for

activities, and effort expended (Bandura, 1982, 1991). Thus, a measurement of perceived auditor

ERP systems expertise may be useful to researchers attempting to explain auditor behavior in

ERP system environments.

Given that no measure of perceived auditor ERP systems expertise exists in the literature,

prior research examining auditor expertise and Ajzens (1991) Theory of Planned Behavior were

reviewed to create a multiple-item scale designed to measure the construct. Prior research has

focused on the role that domain-specific experience plays in expertise (e.g., Bonner, 1990;

Bonner and Lewis, 1990; Ashton, 1991; Mauldin and Ruchala, 1999). Domain-specific

experience is presumably the basis for obtaining the knowledge needed to become an expert

(Abdolmohammadi and Shanteau, 1992). As such, included in the multiple-item scale were

5

measures of auditors perceptions of their experience levels with auditing ERP systems, their

time spent auditing ERP systems, and how early in their careers they began auditing ERP

systems. Training in a given domain is expected to combine with experience to increase domain-

specific expertise levels (Bonner and Lewis, 1990; Bedard and Chi, 1993). Therefore, the level

of perceived ERP systems training of auditors, both formal and informal, was measured.1

Measurements of perceived experience and training as surrogates for perceived

expertise/behavioral control are consistent with the Theory of Planned Behavior as it predicts

behavioral control and, in turn, behavior, to be dependent on past experiences, resources, and

anticipated impediments and obstacles (Ajzen, 1991). Lastly, Ajzen (1991) states that when

behavioral control perceptions are expected to affect behavior, one must directly measure the

construct that determines the level of behavioral control. Thus, in addition, auditors perceptions

of their ERP systems expertise were directly measured.2

Auditors were asked to evaluate their perceived ERP systems expertise relative to other

in-charge auditors or audit seniors (peers). This reference point is appropriate given that future

research may use the measure to investigate the effects of this perception on the behavior of

auditors with the same general level of experience within audit firms. It is also most applicable to

seniors because their skills or expertise are rated against their fellow seniors during the

promotion and compensation process. Thus, seniors are generally informed with regard to their

expertise levels in relation to those of their peers. While this reference point may vary between

auditors (i.e., auditors have different peer groups), the provision of the reference point allows

auditors a relatively constant frame of reference when responding to the items.3 Researchers

investigating auditor ERP systems expertise with respect to other types of auditors (e.g., internal

auditors, audit managers) can customize the questions to their participants.

6

The five questions and the eight-point Likert scale that were used to measure perceived

auditor ERP systems expertise are provided in the Appendix. Other forms of systems (e.g.,

internal control applications) can be substituted for ERP systems by researchers using the

measure in other settings. Lastly, the Likert scale for the measure contains no mid-point, thus

inducing respondents to rate their expertise either above or below their peers. The intent of not

providing a mid-point (i.e., not allowing participants to rate their expertise as equal to their

peers) was to promote variation between auditors responses to the questions.

Assessment of measure

The multiple-item scale developed in this study was assessed for reliability and construct

validity with 73 practicing audit seniors (participants) from four international and two national

public accounting firms. The offices of the firms that participated in the study were located

throughout the United States. Participants had an average of 3.68 years of audit experience. As

discussed above, five items were used to measure perceived auditor ERP systems expertise.

Participants recorded their responses to these items on eight-point scales. Internal consistency

and factor analysis results indicate that the five-item measure does possess significantly high

levels of both reliability and construct validity, respectively. Cronbachs alpha was calculated to

assess the internal consistency of the measure. Results from the sample provide an alpha level of

0.9542 (non-tabulated), well above the generally accepted threshold of 0.70 recommended by

Nunnally (1978). In addition, randomly splitting the sample into two groups of 36 and 37

participants provides acceptable alpha levels of .9634 and .9472 (non-tabulated), respectively.

The test-retest method of assessing the reliability or stability of the measure over time was

infeasible due to restrictions on accessing the studys participants twice. In such cases, Mason

7

and Levy (2001) suggest comparing the results of this studys sample to that of a prior sample of

auditor participants in order to measure the stability of the instrument over time. Obtaining

similar assessment results between two samples separated by time supports the assertion that the

measure is stable (Mason and Levy, 2001). A pre-test of the measure, consisting of 45 practicing

auditors, occurred approximately one year before this study. The pre-tests Cronbachs alpha was

similar to the alpha level of this study and factor analysis results from the pre-test were

consistent with those presented below.4 Overall, it appears that the measure is reliable.

The assessment of construct validity (i.e., convergent and discriminant validity) for the

five-item measure of perceived auditor ERP systems expertise was performed via exploratory

factor analysis. Participants general audit experience, coded in years (range = 1.00 to 10.92

years), was also included in the analysis to determine whether or not the ERP systems expertise

items and general audit experience appear to measure separate factors (i.e., discriminant

validity). Exploratory factor analysis via the principle components analysis method was

performed on the sample data.

First, a correlation matrix was produced for all six items (i.e., five ERP systems expertise

items and one general audit experience measure). Table I contains the Spearman correlation

take in Table I

matrix. Table I reports that all five items intended to measure perceived auditor ERP systems

expertise were significantly positively correlated (all ps < .001). In addition, none of the five

expertise items were significantly correlated with the general audit experience item (all ps >

.40). The lack of correlation between the ERP systems expertise items and the measure of

general audit experience suggests that auditors perceptions of ERP systems expertise are not

simply a by-product of general audit experience.

8

Next, factors (components) were extracted for all six items via principal components

analysis. Utilizing Kaisers (1960) eigenvalue-one criterion, non-tabulated results show that two

factors were extracted from the data (i.e., the results show two components have eigenvalues

exceeding one). Table II presents the principal components factor matrix, which indicates how

take in Table II

each of the six items loaded on the two factors. Inspection of Table II reveals that Factor

(Component) One loadings for all five items intended to measure perceived auditor ERP systems

expertise were satisfactorily greater than the commonly used threshold of .50 (Nunnally, 1978).

Table II also depicts that the measure of general audit experience (AUDEXP) had satisfactory

loadings greater than .50 on Factor Two. Consistent with the correlation matrix, this finding

indicates that auditors perceptions of ERP systems expertise may not be a factor of career

longevity, but rather more likely affected by the specific client experiences, training, etc. of

auditors.5

Participants mean expertise scores were measured as their mean response to the five

items listed in the Appendix. Table III contains descriptive statistics and the frequency

take in Table III

distribution for the mean expertise scores of the participants. The mean and median scores of the

sample were 3.315 and 3.000, respectively. In addition, the samples mean expertise scores

appear to be fairly well distributed.

To determine the propensity of the measure to dichotomize sample participants into sub-

samples perceiving either low or high ERP systems expertise, participants scoring below and

above the median expertise score of 3.000 were classified as perceiving low and high expertise,

respectively. Due to the subjectiveness of the construct, the median score of the sample was

9

used, as it is impossible to declare with certainty the exact point at which an auditor does or does

not perceive high ERP systems expertise (Baron and Kenny, 1986). Means for the low and high

expertise groups were 1.800 and 4.947, respectively. Based on scale labels for the five items,

responses of 2 and 5 indicate that participants mostly disagreed and mildly agreed,

respectively, that their expertise exceeded their peers. A one-tailed independent-samples t test

(not tabulated) indicated a significant difference between the two groups (p < .001).6 These

differences between the low and high expertise groups indicate that the measure divided

participants into two groups possessing significantly different perceptions of their own ERP

systems expertise.

Lastly, a study by Brazel and Agoglia (2004) provides some evidence supporting the

concurrent validity of the measure. Concurrent validity is the extent to which a measure can

predict performance measured at the same time. They find that the measure provides two groups

of auditors (i.e., auditors perceiving low and high ERP systems expertise) which differ in their

assessments of risk and planning of substantive tests in an ERP setting. Additionally, with the aid

of experienced auditors, it was determined that auditors perceiving high ERP systems expertise

provided higher quality risk assessments and planned more effective substantive tests. Therefore,

there is limited support that the measure behaves as expected.

Uses of measure

This section discusses possible uses for, and other issues concerning, the perceived

auditor ERP systems expertise measure. The measure can be used to capture perceived auditor

ERP systems expertise as part of an experimental study interested in determining its effects upon

auditor performance, audit quality, auditor perceptions, etc. As noted above, Brazel and Agoglia

10

(2004) find that the measure provides two groups of auditors (i.e., auditors perceiving low and

high ERP systems expertise) which differ in their judgments and performance. Results from their

study also suggest that the measure is superior to general audit experience when explaining

auditor behavior in an ERP system setting. Other uses of this measure include studies of the

effects of perceived auditor ERP systems expertise on their identification of control weaknesses

as part of the recently approved standard requiring auditors to review and attest to the

effectiveness of corporate internal controls (PCAOB, 2004). Winters (2004) notes that many

public corporations have adopted ERP systems to assist them in their compliance with the

Sarbanes-Oxley Act of 2002 and its related standards.

Given the apparent lack of correlation between ERP systems expertise and general audit

experience, studies examining the performance differences between auditors with high ERP

systems expertise and auditors with extensive general audit experience might contribute to our

understanding of what experience/expertise factors best explain performance and in a variety of

audit settings. For example, it may be preferable to assign a less experienced auditor with an

ERP systems background to an ERP systems audit environment than a more senior auditor who

may lack this domain of expertise. Survey research might measure a multitude of antecedent

variables such as personal client experiences, specific firm trainings, and the ERP systems

expertise of superiors (in conjunction with this studys measure) to determine their impact on the

construct. Such research might provide insight into how firms can develop ERP systems

expertise (or perceptions of behavioral control in ERP environments) in the initial years of

auditors careers.

The assessment above provides evidence that the measure can either dichotomize a

sample of auditors into groups perceiving either low or high ERP systems expertise or provide

11

substantial variation in mean participant responses (see Table III). Thus, this measure could be

used in either ANOVA or as a continuous variable (i.e., mean participant response score) in

ANCOVA and regression analyses of participant data. Given that perceived ERP systems

expertise is a measured (versus manipulated) variable, researchers should take great care in

measuring other possible explanatory variables in order to evaluate possible causal relationships

between the measure and a particular outcome variable. For example, when investigating the

effects of this perception on auditor risk assessments, researchers should measure participants

experience with assessing risks. If such experience is insignificant between groups, then the

researcher can possibly rule out that variable as one that drives any expertise results. In cases

where such a variable is significant between groups, the researcher can then include the measure

in ANCOVA or as a control variable in regressions to determine whether perceived ERP systems

expertise provides additional explanatory power.

One of the benefits of the measure is that perceived auditor ERP systems expertise can be

measured in very little time. While Choo and Curtis (2000) review of the expertise literature

suggests using far more thorough measures of auditor expertise (e.g., analysis of participant

performance on an ERP system related task), the measure provided in this study may be an

economical way of capturing perceived ERP systems expertise/behavioral control from auditors

who have expended considerable mental effort in an experimental case study or survey. The

provision of an exhaustive measure of ERP systems expertise after an experimental case study

might cause auditors to drop out of the study or complete such a measure with less than their full

attention. Therefore, a more time consuming measure might result in lost or inaccurate data.

Audit professionals are a valuable resource for academic researchers and great care should be

taken to ensure their use in studies is both effective and efficient.

12

The above discussion suggests that the measurement of perceived auditor ERP systems

expertise could occur in a post-experimental questionnaire. This is advisable for two reasons.

First, it is often difficult to obtain double access to audit professionals. Therefore, measuring

auditor expertise and then later randomly assigning participants to experimental conditions can

often be impractical. Second, measuring ERP system perceptions at the beginning of an

experimental case study may sensitize participants to the research questions under investigation

(e.g., the effects of perceived auditor ERP systems expertise on their performance in a given

task). Researchers measuring ERP systems perceptions as an independent variable after

dependent variables have been obtained should be certain that the experimental instrument does

not substantially affect participants expertise scores. In cases where participants are expected to

gain ERP expertise through the completion of experimental materials, it is advisable to pre-

measure auditors expertise levels to enhance any causal interpretations. For studies that simply

induce participants to use their ERP systems expertise in responding to experimental

manipulations and other case study information, the measure can be used post-experimentally

similar to the measurement of other demographic variables (e.g., general audit experience,

industry expertise).

As indicated in the Assessment section, this measure can be customized by future

researchers to measure how other types of auditors perceive their ERP systems

expertise/behavioral control in ERP system settings. Prior research has indicated concerns among

auditors regarding deficiencies in IT audit specialist expertise (Hunton et al., 2004; Brazel and

Agoglia, 2004). Tailoring the measurement questions to factors influencing IT auditor perceived

expertise and relating this perception to their control testing should provide insight into how

these specialists can be used by auditors to meet new control testing standards (PCAOB, 2004).

13

The findings of Brown (1983) and Margheim (1986) indicate that auditors are sensitive to the

expertise of internal audit and rely more on the work of highly competent internal auditors. To

investigate the positive effects of internal auditor ERP systems expertise, the measures

questions could be modified and used in studies examining how their expertise positively

impacts the reliability of system related controls and corporate financial statements. The

measurement items presented in the Appendix measure senior-level auditor perceived ERP

systems expertise. With little customization, the measure could be used by researchers interested

in examining the effects of audit staff, manager, or partner expertise perceptions on their

performance in a variety of other system contexts (e.g., internal control applications). Lastly,

when customizing the measure to other types of auditors or systems, researchers should pre-test

the measure to determine the reliability and validity of the modified measure.

Conclusions and limitations

The purpose of this study was to develop, assess, and provide uses for a measure of

perceived auditor ERP systems expertise. A review of the auditor expertise literature and Ajzens

(1991) Theory of Planned Behavior provided a five-item measure. An assessment of the measure

shows that it possesses a high level of reliability, construct validity, and the results of Brazel and

Agoglia (2004) support the concurrent validity of the measure. Interestingly, the results suggest

that auditor perceptions of ERP systems expertise are not significantly related to general audit

experience. Therefore, it may not be prudent to assume that auditor perceptions of ERP systems

expertise (or perceptions of behavioral control in ERP settings) automatically increase in

conjunction with the longevity of their audit careers. The measure also has the propensity to (a)

dichotomize auditors into distinct groups perceiving either low or high ERP systems expertise

14

for experimental research and (b) provide significant variation in participant responses for use in

survey studies. Possible future uses of the measure include examining its effects on the

performance of ERP system related audit tasks and customizing the measure to determine the

systems expertise levels of other types of auditors. Such research will advance our understanding

of the role perceived auditor ERP systems expertise plays in influencing auditor behavior and

determining the quality of contemporary audit services.

There are several limitations to this study and the measure it has developed. First, the

measure captures auditors perceptions or self-assessments of ERP systems expertise, not their

actual experience, training, etc. with respect to ERP systems. The measure thus lacks objectivity

or does not measure actual behavioral control in an ERP system setting, although Ajzen (1991, p.

183) suggests while the importance of actual behavioral control is self-evident of greater

psychological interest is the perception of behavioral control and its impact on actions. In

addition, any objective measure available to academic researchers (e.g., hours auditing ERP

systems) will still be self-reported by auditor research participants and thus subject to bias. Still,

future research using this studys measure should obtain more objective measures (or

determinants) of auditors ERP systems expertise to (a) evaluate if auditor perceptions of

expertise mirror actual expertise and (b) determine whether more objective or perceived

expertise measures better explain auditor behavior.

Second, the Theory of Planned Behavior (Ajzen, 1991) indicates that intentions can

combine with perceived behavioral control to affect behavior. The construct of auditor intent

with respect to ERP systems was excluded from this study in an attempt to achieve a

parsimonious measure. Given that auditors are part of a profession and thus abide by a code of

conduct, intentions toward auditing may be a perception that lacks the variation present in

15

perceived behavioral control which is affected by personal training/experience. Notwithstanding,

future research could evaluate whether the measure can be improved (i.e., better explain auditor

behavior) by adding questions to capture the construct of intent.

Third, more research is needed to evaluate the predictive and concurrent validity (i.e.,

criterion validity) of the measure and its stability over time. Fourth, the measure relates to a

specific form of system: ERP systems. While this form of system may be the dominant type

encountered by auditors of public companies (Cerullo and Cerullo, 2000), there are a variety of

systems in use and this measure may not properly measure auditor perceptions of expertise with

respect to other IT. Lastly, the audit expertise literature has clearly shown that experience and

training combine to create expertise in auditors. Still, other dimensions of ERP systems expertise

not included in the measure (e.g., ERP implementation knowledge) may be relevant to

determining perceived auditor ERP systems expertise. Future research investigating auditor

interactions with ERP systems may uncover such additional sources of expertise.

16

Table I Spearman correlation matrix for perceived auditor ERP systems expertise items and audit

experience

Spearman Correlation Matrix

Itema

ERP TRAIN

ERP EXP

ERP TIME

ERP BEGAN

ERP EXPERT

AUD EXP

ERP TRAIN

1.00

0.864

0.747

0.728

0.829

-0.097

ERP EXP 1.00 0.704 0.679 0.874 -0.021 ERP TIME

1.00

0.800

0.791

-0.039

ERP BEGAN 1.00 0.755 -0.023 ERP EXPERT

1.00

-0.024

AUD EXP 1.00 Bold = significant correlations at the p < .001 level (two-tailed test). a The five items related to auditing complex and pervasive accounting information systems (e.g., ERP systems) and asked participants to respond on eight-point scales (1 = strongly disagree; 8 = strongly agree). All five items began with the same prompt (Relative to other in-charge auditors at my firm,), before diverging. The divergent wording related to each of the individual items was: I have received more combined informal and formal training during my career (ERPTRAIN) I have more experience (ERPEXP) a larger portion of my time is assigned (ERPTIME) I began at an earlier point in my career (ERPBEGAN) I have a higher level of expertise (ERPEXPERT). Participants were also asked to provide their audit experience in months which was then coded into years (AUDEXP).

17

Table II Component matrix for perceived auditor ERP systems expertise items and audit experience

Component Matrixa Component Component Itemb 1 2 ERPTRAIN .930 .011 ERPEXP .919 .013 ERPTIME .916 -.008 ERPBEGAN .885 .010 ERPEXPERT .955 -.025 AUDEXP .001 1.000 Extraction Method: Principal Component Analysis. a Two components extracted. b The five items related to auditing complex and pervasive accounting information systems (e.g., ERP systems) and asked participants to respond on eight-point scales (1 = strongly disagree; 8 = strongly agree). All five items began with the same prompt (Relative to other in-charge auditors at my firm,), before diverging. The divergent wording related to each of the individual items was: I have received more combined informal and formal training during my career (ERPTRAIN) I have more experience (ERPEXP) a larger portion of my time is assigned (ERPTIME) I began at an earlier point in my career (ERPBEGAN) I have a higher level of expertise (ERPEXPERT). Participants were also asked to provide their audit experience in months which was then coded into years (AUDEXP).

18

Table III Frequency distribution and descriptive statistics for mean perceived auditor ERP systems

expertise measure

Mean Expertise Scorea Frequency Percent

Cumulative Percent

1.0 5 6.85% 6.85% 1.2 2 2.74% 9.59% 1.4 4 5.48% 15.07% 1.6 3 4.11% 19.18% 1.8 4 5.48% 24.66% 2.0 10 13.70% 38.36% 2.2 2 2.74% 41.10% 2.4 2 2.74% 43.84% 2.6 3 4.11% 47.95% 2.8 1 1.37% 49.32% 3.0 3 4.11% 53.42% 3.2 3 4.11% 57.53% 3.4 3 4.11% 61.64% 3.6 1 1.37% 63.01% 3.8 5 6.85% 69.86% 4.0 1 1.37% 71.23% 4.2 1 1.37% 72.60% 4.4 1 1.37% 73.97% 4.8 1 1.37% 75.34% 5.0 2 2.74% 78.08% 5.2 1 1.37% 79.45% 5.4 1 1.37% 80.82% 5.6 2 2.74% 83.56% 5.8 2 2.74% 86.30% 6.0 1 1.37% 87.67% 6.2 3 4.11% 91.78% 6.4 2 2.74% 94.52% 6.6 1 1.37% 95.89% 6.8 1 1.37% 97.26% 7.0 1 1.37% 98.63% 7.2 1 1.37% 100.00%

Total 73 100.00% Mean 3.315 Median 3.000 Std. Deviation 1.815 a Participants mean perceived ERP systems expertise scores were measured as their average response to the five, eight-point scales listed in the Appendix.

19

References

Abdolmohammadi, M.J. and Shanteau, J. (1992), Personal attributes of expert auditors, Organizational Behavior and Human Decision Processes, 53, pp. 158-72.

Ajzen, I. (1991), The theory of planned behavior, Organizational Behavior and Human

Decision Processes, 50, pp. 179-211. Ashton, A.H. (1991), Experience and error frequency knowledge as potential determinants of

audit expertise, The Accounting Review, 66, pp. 218-39. Bandura, A. (1982), Self-efficacy mechanism in human agency, American Psychologist, 37,

pp. 122-147. Bandura, A. (1991). Social-cognitive theory of self-regulation, Organizational Behavior and

Human Decision Processes, 50, pp. 248-87. Baron, R.M. and Kenny, D.A. (1986), The moderator-mediator variable distinction in social

psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, pp. 1173-82.

Bedard, J and Chi, M.T.H. (1993), Expertise in auditing, Auditing: A Journal of Practice and

Theory, 12, pp. 21-45. Bonner, S.E. (1990), Experience effects in auditing: The role of task-specific knowledge, The

Accounting Review, January, pp. 72-92. Bonner, S.E. and Lewis, B. (1990), Determinants of auditor expertise, Journal of Accounting

Research, Supplement, pp. 1-20. Brazel, J.F. and Agoglia, C.P. (2004), The Effects of Computer Assurance Specialist

Competence and Auditor AIS Expertise on Auditor Planning Judgments, Working Paper, North Carolina State University.

Brown, P.R. (1983), Independent auditor judgment in the evaluation of internal audit

functions, Journal of Accounting Research, Autumn, pp. 444-55. Cerullo, M.J., and Cerullo, V. (2000), The internal auditors role in developing and

implementing enterprise resource planning systems, Internal Auditing May/June, pp.25-34.

Chi, M., Glaser, R., and Rees, E. (1982), Expertise in problem solving, in Steinberg, J. (Ed.),

Advances in the Psychology of Human Intelligence, Lawrence Erlbaum Associates, Hillsdale, NJ, 1, pp. 7-75.

20

Choo. F and Curtis, M.B. (2000), Structural assessment in accounting research, Journal of Accounting Literature, 19, pp. 131-157.

Guess, A.K., Louwers, T.J., and Strawser, J.R. (2000), The role of ambiguity in auditors

determination of budgeted audit hours, Behavioral Research in Accounting, 12, pp. 119-38.

Hunton, J.E., Wright A.M., and Wright, S. (2004), Are financial auditors overconfident in their

ability to assess risks associated with enterprise resource planning systems?, Journal of Information Systems (forthcoming).

Kaiser, H.F. (1960), The application of electronic computers to factor analysis, Educational

and Psychological Measurement, 20, pp. 141-51. Kinney, W.R., Jr. (2001), Accounting scholarship: What is uniquely ours?, The Accounting

Review, 76, pp. 275-84. Kumar, K. and Hillegersberg, J.V. (2000), ERP experiences and evolution, Communications of

the ACM, April, pp.22-6. Lilly, R.S. (1997), Client/server database security, IS Audit and Control Journal, 6, pp. 50-1. Margheim, L.L. (1986), Further evidence on external auditors reliance on internal auditors,

Journal of Accounting Research, Spring, pp. 194-205. Mason, J.D. and Levy, L.G. (2001), The use of the latent constructs method in behavioral

accounting research: The measurement of client advocacy, Advances in Taxation, 13, pp. 123-139.

Mauldin, E.G. and Ruchala, L.V. (1999), Towards a meta-theory of accounting information

systems, Accounting, Organizations and Society, May, pp. 317-31. Nunnally, J.D. (1978), Psychometric Theory, McGraw-Hill, New York, NY. Peecher, M.E. and Solomon, I. (2001), Theory and experimentation in studies of audit

judgments and decisions: Avoiding common research traps, International Journal of Auditing, 5, pp. 193-203.

Public Company Accounting Oversight Board (PCAOB) (2004), Auditing Standard No. 2, An

Audit of Internal Control Over Financial Reporting in Conjunction with an Audit of Financial Statements, www.pcaobus.org.

Public Oversight Board (POB) (2000), The Panel on Audit Effectiveness: Report and

Recommendations, www.pobauditpanel.org.

21

Turner, T. (1999), Implementing and Enhancing Data Security in a PeopleSoft Environment, IS Audit and Control Journal, Vol. III, p. 51-1.

Winters, B.I. (2004), Choose the right tools for internal control reporting, Journal of

Accountancy, 197, pp. 34-40. Wright, S. and Wright, A.M. (2002), Information system assurance for enterprise resource

planning systems: Implementation and unique risk considerations, Journal of Information Systems, 16, pp. 99-113.

22

Appendix Perceived auditor enterprise resource planning (ERP) systems expertise measure

The following questions measure perceived auditor ERP systems expertise: 1. Relative to other in-charge auditors at my firm, I have more experience auditing complex and pervasive accounting information systems (e.g., ERP systems). 2. Relative to other in-charge auditors at my firm, a larger portion of my time is assigned to auditing complex and pervasive accounting information systems (e.g., ERP systems). 3. Relative to other in-charge auditors at my firm, I began auditing complex and pervasive accounting information systems (e.g., ERP systems) at an earlier point in my career. 4. Relative to other in-charge auditors at my firm, I have received more combined informal and formal training in relation to complex and pervasive accounting information systems (e.g., ERP systems) during my career. 5. Relative to other in-charge auditors at my firm, I have a higher level of complex and pervasive accounting information systems (e.g., ERP systems) expertise. Participants responded to each of the above questions via the following eight-point Likert scale: 1 2 3 4 5 6 7 8 Strongly Mostly Somewhat Mildly Mildly Somewhat Mostly Strongly

Disagree Disagree Disagree Disagree Agree Agree Agree Agree

23

Endnotes

1. Discussions with audit practitioners indicated that ERP systems training was often

received from firm provided formal training sessions and/or informal (on the job)

training. In order to fully measure perceived auditor ERP systems training, both sources

of training were combined in one item.

2. Prior audit expertise research has focused on the combined role of experience and

training in the development of expertise. As research into auditor interactions with ERP

systems (or other complex systems) progresses, other complementary sources of ERP

systems expertise may be identified and potentially added to this studys measure.

3. By not providing a reference point, albeit a reference point which is perceived by the

auditor and may vary between auditors, the researcher maintains no control over how

auditors interpret the question. For example, one auditor may assess his or her ERP

systems training in relation to his or her peers, while another may assess his or her

training in relation to a competent IT audit specialist. Such confusion would certainly

reduce the explanatory value of the measure.

4. Non-tabulated internal consistency and factor analysis results for the pre-test sample

provided a Cronbachs alpha = 0.9106, well above the generally accepted threshold of

0.70, and all five items satisfactorily loaded on one factor (all factor loadings in excess of

.70) (Nunnally, 1978). General audit experience (in years) and level within their firm

(e.g., fourth year) for the pre-test participants loaded on a separate factor.

5. Orthogonal (Varimax) and oblique (Direct Oblimin) rotations of the factors (non-

tabulated) do not qualitatively change the results.

24

6. Three participants had mean scores of 3.000 and were removed from the original sample

of 73 auditors for the independent-samples t test performed.