Intelligent Decision Aids and the Accounting Professional

Presented by: Micheal AxelsenDirectorApplied Insight Pty Ltd

For CPA AustraliaIT Discussion Group25th May 2011

INTRODUCTION

AcknowledgementsSome of the material presented in this discussion was supported under Australian Research Council's Linkage Projects funding scheme (project number LP0882068)

About this presentation

Objectives• Identify deskilling and whether it occurs• Identify strategies to mitigate effects

Agenda• The what and why of deskilling• Recent research: How does deskilling ‘work’?• Risk factors in deskilling• Findings: Accountants in practice• Mitigating deskilling• Conclusion

Personal motivation

Motivation• Five years on CPA Australia’s Board Education Committee• Twelve years on IT & Management CoE• Observation in practice that accounting partners would gladly

substitute cheap ‘push a button’ IDAs for expensive senior staff• In the long term what does continued use of IDAs mean for the

development of the profession?

Position and qualifications• Director of Applied Insight Pty Ltd; UQ PhD Candidate• Former Chair of CPA Australia IT & M Centre of Excellence• Qualifications:

– Bachelor of Commerce (Hons)– Masters of Information Systems– FCPA

Expectations?

• What are your expectations from today’s session?

THE WHAT AND WHY OF DESKILLING

Key concepts

• Intelligent decision aid: software-intensive systems that integrate the expertise of one or more experts in a given decision domain (Arnold, Collier, Leech & Sutton 2004)

• Deskilling: a decline of auditors’ skills and abilities (Arnold & Sutton 1998)

• Declarative knowledge: definitions, examples and rules that are stored into long-term memory (Anderson 1993; McCall, Arnold & Sutton 2008)

• Procedural knowledge: the ability to apply and extend declarative knowledge and is acquired through experience - a key antecedent to expertise (Anderson 1993; McCall, Arnold & Sutton (2008)

Key concepts

• Reliance: the degree to which a user applies and incorporates the recommendations of the IDA during judgment formulation (Arnold & Sutton 1998; Hampton 2005)

• System restrictiveness: the degree to which and the manner in which a Decision Support System restricts users' decision-making processes to a particular subset of possible processes (Silver 1988) – made up of:– Physical (Chu & Elam 1990; Speier & Morris 2003)– Structural - embedded internally within the system

(Anson et al 1995; Desanctis & Poole 1994)– Process (Kim et al 2002; Wheeler & Valacich 1996)

Ever seen one of these?

Why might deskilling occur?

• Reliance by auditors on computers in the audit process is increasing (Dowling & Leech 2007; Leech 2008; Mascha 2001)

• Many benefits of IDAs are recognised, including training , efficiency, consistency (Elliott & Kielich 1985; O’Leary 1987; Rose & Wolfe 1998; Sutton & Byington 1993)

• Regulations and compliance frameworks are so complex we have to have a computer to remember them all.

• However there may be negative effects: IDA use in the long term that affect accounting expertise (Arnold & Sutton 1998; Dowling, Leech & Moroney 2006; Leech 2008;McCall, Arnold & Sutton 2008; Mitroff & Mason 1989)

• Embedded IDAs affect Tax compliance decisions (Masselli, Ricketts, Arnold & Sutton 2002)

Benefits of IDAs

• Consistency• Reliability• Cover your backside in legal actions demanding rigour in

the work – the ‘Your Honour’ effect• Controlling junior staff, and risk management• Increased efficiency• Enforces good documentation and ‘rationale’• Particularly helps us deal with the issues of an aging and

shrinking workforce

Computer says ‘No’

• Thank you, Carol• However, as a

professional remember it is up to you to use your noggin.

• A professional should be making a professional judgment call – this cannot be made by the computer

RECENT RESEARCH: HOW DOES DESKILLING WORK?

People are strange

• They’re not machines...

Overview

• Braverman (1974) considered that the use of technology leads to degradation of work.

• Fearfull (1992) observed a divide between office workers that developed their skills pre-computerisation and those that developed their skills post-computerisation.

• Some relate the global financial crisis of 208/2009 in the over-reliance by auditors upon the use of technological approaches they did not understand (Fraser & Pong, 2009).

• Reliance by auditors on information systems in the audit process is increasing (Dowling & Leech 2007; Leech 2008; Mascha 2001), and this is confirmed by experience and observation.

Cognitive load theory

• Experts and novices in a problem domain adopt differing problem solving strategies (Sweller 1988):– Novices identified end goals and worked backwards from

those goals through identified sub-goals using means-end analysis

– Experts eliminated the backward-looking phase (Sweller, 1988).

• Experts have acquired schemas as a result of their experience with past problems, whereas novices are forced to use generalised problem-solving strategies as they do not yet possess these schemas (Sweller, 1988).

Cognitive load theory

• Decision aids tend to reduce knowledge acquisition relative to manual environments because aid users can substantially reduce the effort devoted to a task and let the aid "do the work" (Todd and Benbasat 1992; Todd and Benbasat 1994)

• Auditor knowledge is a function of ability, effort and experience (Awasthi & Pratt 1990; Libby & Tan 1994)

• A well established and tested theory of determinants of knowledge that suggests factors separate to those identified by TTD as contributing to deskilling

Cognitive load theory

After Sweller (1988); Libby & Tan (1994)

Adaptive Cognitive Thought - Rational

Definitions, rules, examples

Declarative encoding

Interpretive problem solving

Procedural knowledge acquisition

Declarative knowledge acquisition

Tuning (of production

rules)

Compilation (of production

rules)

Declarative knowledge

Procedural knowledge

After Anderson (1993)

Theory of technology dominance

• Arnold & Sutton (1998) developed this theory in response to the limited success of IDAs in the audit domain, and to see how IDAs might be more effectively designed for audits

• The longer an IDA is used, the higher the level of deskilling in this domain

Continued use of an intelligent decision aid

Level of deskilling in this domain

+ve

Who am I going to believe?

RISK FACTORS IN DESKILLING

Determinants of auditor judgment expertise

• Libby & Luft (1993) and Libby & Tan (1994) consider that auditor judgment performance is a function of the auditor’s ability, experience, motivation and environment:– Ability: capacity to complete information encoding, retrieval,

and analysis tasks and to undertake problem-solving tasks related to the audit domain.

– Experience: the length of time that the auditor has been a professional auditor.

– Motivation: willingness to exert effort to deal with cognitive load (Rose & Wolfe 2000).

– Environment: judgment guidance and technological aid; hierarchical group settings; accountability relationships; sequential, multi-period tasks; and substantial monetary incentives for good performance.

Towards a conceptual model

• Anderson (1982) considers declarative knowledge a pre-requisite to procedural knowledge.

• Libby & Luft (1993) consider ability, experience, motivation and environment to determine auditor expertise - the IDA is part of ‘environment’.

• Sweller (1988) states that experience in addressing the task is necessary to become an expert, whilst Todd & Benbasat (1992) notes that a tendency to ‘let the aid do the work’ reduces knowledge acquisition.

• Arnold & Sutton (1998) propose that continuous use of an IDA deskills the auditor.

Conceptual model

✓✓

✓ ✓

✓

✓

✓✓

Pop Quiz: What matters?

• Does ‘Know-What’ Matter? Or is it ‘Know-How’ that matters?

• What’s more important for an accountant?– Their ability?– Their ‘passion for accounting’ (motivation)?– Their experience?

• Sense check?

CONCLUSION

FINDINGS: ACCOUNTANTS IN PRACTICE

Accounting, do we have a problem?

• The monetising lure of the ‘junior burger’ effect

• Who will write tomorrow’s decision support systems?

• Set it up to ‘push a button’ – little fish in the sea

• Too many spreadsheets?

• ‘It’s like we had to learn how to audit again.’

• Over-purchase of inventory• What could possibly go wrong

with that - VAR• The difference between risk

and uncertainty

Status updates: Interviewee statements (guided by Leximancer)

• Methodology*:– “If I didn't have access to this office's IT enabled audit support tool, I would do the job in half

the time.” (Senior Technical Specialist, Office B, >35 years experience)– “And the IT-enabled audit tool also makes the auditor, if the auditor follows the IT-enabled

audit tool and uses the IT-enabled audit tool properly, the IT-enabled audit tool makes the auditor always consider what the impact on the audit is, or what the risks to material misstatement is, so, there's a reliance on the IT-enabled audit tool in the planning phase, definitely.” (Senior Financial Auditor, Office B, 4 years experience)

• Motivation*:– “The IT enabled audit support tool made the auditors lazy... made the auditors stop thinking

and this office's change over to IT enabled audit support tool highlighted the fact that after 13 years of the prior IT enabled audit support tool that our auditors had to learn to audit again.” (Deputy Auditor-General, Office A, >25 years experience)

• Ability [Time]*:– “I would say yeah, that relying on an IT enabled audit support tool would definitely affect my

ability to remember the details of accounting and auditing standards.”

* These are interviewees’ ‘backfilled’ comments

Status updates: Interviewee statements (guided by Leximancer)

• Judgment*:– “…one of the aspects that I've found interesting… has been around… directors saying

directors no longer think, directors just tick boxes, and auditors saying auditors just fill in forms and do checklists, so the need to exercise judgment's gone away.” (Auditor-General, Office A, >25 years experience)

– “Yeah, that is the difference between the two IT enabled audit support tools, the old IT enabled audit support tool didn't require the auditor to think and the current IT enabled audit support tool does require the auditor to think. I liked the IT enabled support tool that required judgment better.” (Director, Office A, 9 years experience)

* These are interviewees’ ‘backfilled’ comments

Status update – preliminary results

Does reliance on a restrictive IT enabled audit tool deskill an auditor in terms of judgment?

Yes No

Office A 5 2

Office B 2 2

Total: 7/11 4/11

MITIGATING DESKILLING

Mitigation strategies in use

• Unfortunately for my research, people are smart• Intuitively, we do tend to avoid deskilling at the extreme.• When methodologies get too restrictive, they change the

methodologies (i.e. ‘dumb the IDA down’). We rely on the computer less

• Intelligent Decision Aids in public sector audit offices tend to be ‘dumber’ i.e. not as restrictive (but there are some notable exceptions).

• Training is relied upon heavily as a mitigating factor.• Selecting and retaining staff with a ‘joie d’audit’. • Work review by senior staff• Build up understanding from practice with small clients and thence

to big ones.

Other considerations

• Institutional blindness (can’t see the wood for the trees) can lead to a form of ‘creeping deskilling’

• Time pressures allow this less and less in some instances

• No matter what happens, the auditor’s role is in decision making, as is the tax accountant’s, and it comes down to judgment.

Keep thinking!

CONCLUSION

Conclusion

• The evidence is showing that accountants are deskilled by over-reliance on intelligent decision aids.

• Deskilling is real, and deskilling affects professional judgment• Rushing to implement expert systems may affect the development

of professional skills in the long term• Given that IDAs generally become more intelligent in response to

compliance frameworks and regulation, there may be an argument for reducing regulation (yes, I know, good luck with that…)

• In your own workplace, ask yourself if ‘institutional blindness’ is happening

CONTACT DETAILS

Micheal AxelsenDirector, Applied Insight Pty Ltd

m: 0412 526 375t: +61 7 3139 0325e: micheal.axelsen@appliedinsight.com.au

blog: www.michealaxelsen.com

Applied Insight Pty LtdPO Box 603Toowong DC 4066AUSTRALIA