Upload
tatyana-latchford
View
216
Download
2
Tags:
Embed Size (px)
Citation preview
Spreadsheet Development, Auditing, & Control Tools
INFORMS 2004, Denver
Roger GrindeWhittemore School of Business &
EconomicsUniversity of New Hampshire
Context Spreadsheet Errors Happen Regulations
VAT, FDA, SOX Spreadsheet Risk
Likelihood of Errors Impact of Errors
Love them or hate them, spreadsheets exist! “Business always moves too fast for traditional
systems to keep up. The gap is filled by the ubiquitous Microsoft Office suite of desktop programs, particularly Excel.” (Cluster Seven Ltd.)
Recent Errors (Made Public) 10/12/04. Business Council (125 US CEOs) revised
economic forecast. From 0-2% growth, to 2.1-4.5% growth “It was a computational error. The spreadsheet shifted so
the wrong numbers appeared in the wrong columns…It was just one column on the spreadsheet.” “…has never happened before in the survey’s history.”
8/12/04. City of Richmond, VA. Overstated sales revenue from new shopping mall.
From Sept-Dec 2003, sales reported to be $80 million; actual sales were $57 million.
“The city blamed its mistake primarily on an error in a spreadsheet formula, which amplified a subtotal amount.” “It was a clerical error.”
Confluence of Interests
Organizational “embarrassments” Regulations Spreadsheet Auditing & Control
Software Consultant, Developer, and User
Community Academic Research
Literature Spreadsheet Error Research
Empirical studies. Detecting errors in small spreadsheets, building spreadsheets, code inspection, peer- and self-audit, classification of spreadsheet errors
Spreadsheet Auditing Research Field audits, spreadsheet risks, which models
to audit, development of auditing tools, comparison of tools
Spreadsheet Software Research Theoretical models of spreadsheet software,
user-centered approach for functions, alternate paradigms
Current Focus
Tools for related phases of spreadsheet life cycle Design Development/Productivity Auditing: Review & Testing Management & Control
Types of Spreadsheet Users (Jones, Blackwell, Burnett 2003) Basic
Managing & printing a list of information Very simple formulae, e.g., =SUM(A1:A10)
Moderate Understand spreadsheet paradigm fairly thoroughly. Mastered pre-requisites, tackle more ambitious and
long-lived applications e.g., mastered absolute and relative cell references
Advanced Understand Visual Basic and can write user-defined
functions and procedures e.g., mastered array functions
Classification of Spreadsheet Errors References
Panko & Halvorson (1996); Chadwick, Knight, & Rajalingham (2001), Rajalingham, Chadwick, & Knight (2002)
Software Errors User Errors
Quantitative Errors Mechanical Errors Logical Errors Omission Errors
Qualitative Errors Structural Errors Temporal Errors
Design Tools Diagramming/Framing Tools Possible Interface to Spreadsheet Model
Example: Influence DiagramModulesInputs/OutputsInitial Spreadsheet Layout
Alternatives (among others) Visio iGrafx FlowCharter Built-In Office Tools
Currently, not much of a direct link to spreadsheet model design.
Development Tools “Productivity” Tools. Primary purpose is
to ease the development of a spreadsheet model/application
Example Capabilities Assistance in writing formulas, manipulating
data, type conversions, text handling. Range name management Sheet/workbook management Enhanced selection & search tools Conversion tools (e.g., export as JPG or EPS)
Number of available tools with a variety of capabilities. Some fairly specialized.
Auditing Tools Tools to help the modeler perform spreadsheet model
verification. Review (“Code Inspection”) Execution Testing
Example Capabilities Workbook Summaries Formula Analysis, Referencing Problems Precedence/Dependence Analysis Structure analysis, model design Worksheet/book comparisons Execution testing
Native Excel has a level of some of these tools. Several “suites” available as well as a number of
“focused” tools.
Management & Control Tools Tools to assist in the management and
control of spreadsheet models within the firm
Example capabilities Version management & control Access and change control Change logging Security
Sarbanes-Oxley Act of 2002, Section 404 “The requirements under Section 404 of the
Sarbanes Oxley Act increase the focus on controls related to the development and maintenance of spreadsheets.” (PWC, July 2004).
Spreadsheet Tools Sources
Web searches Lists of add-ins Mentioned in papers
Reviewed literature for tools, categorized into one of four categories, assessed feature set at high level
Tools found Design: none (with direct spreadsheet integration) Development/Productivity: 10 Auditing: 10 “suites”, 11 “focused” Control: 4
Overlap between categories exists
Features Precedence & Dependence Browsing Identify unique vs. (apparent) copied
formulas, identify inconsistent formulas Data & referencing problems Workbook summaries Version control and comparison of
worksheets/workbooks. Execution testing Productivity aids Multi-Model management
Feature Mapping: Auditing Tools
ProductPrecedence/De
pendence
ID Unique & Copied
Formulas
Data & Referencing
ProblemsWorkbook Summaries
Version Control / Comparison
Execution Testing
Productivity Aids
Multi-Model Management
Excel Smart Tools Auditor Y N Y Y Y N N NExChecker Y Y Y Y Y N N YOperis Analysis Kit Y Y Y Y Y N Y NRainbow Analyst Y Y Y Y Y N N NSpACE (HMCE)
Spreadsheet Detective Y Y Y Y Y Y N NSpreadsheet Professional Y Y Y Y Y Y N NThe Excel Auditor Y Y N Y N N N NXLSior N N N Y Y Y N NXLSpell (with XDrill) Y Y Y Y N N NCode Tracer Y N N N N N N NDSLimited Tools - dsCopyFinder N Y N N N N N NDSLimited Tools - dsSummarizer N N N Y N N N NExcel Compare N N N N Y N N NExplode Add-In Y N N N N N N NMaster Tools Y Y N N N N Y NReference Browser Y N N N N N N NSpreadsheet xlNavigator Y N N N N N N NSynkronizer 8.0 N N N N Y N N NXdrill Y N Y N N N N NXL Analyst N Y Y Y N N N N
Features: Development Tools
ProductPrecedence/De
pendence
ID Unique & Copied
Formulas
Data & Referencing
ProblemsWorkbook Summaries
Version Control / Comparison
Execution Testing
Productivity Aids
Multi-Model Management
ActiveData N N N Y Y N Y YAppsPro Utilities N N N N N N Y NASAP Utilities N N N N N N Y NEAT Toolpack N N N N N N Y NFastTrack for Excel Y N N N N N Y NModeling ToolPack Y N N N N N Y NNavigator Utilities N N N N N N Y NOZGrid Analysis Collection N N N N N Y Y NPower Utility Pack N N N Y Y N Y NSpreadsheet Composer N N N N N N Y N
ProductPrecedence/De
pendence
ID Unique & Copied
Formulas
Data & Referencing
ProblemsWorkbook Summaries
Version Control / Comparison
Execution Testing
Productivity Aids
Multi-Model Management
CIMCON einfotree N N N N Y N N YClusterSeven XiGence Suite N N N N Y N N YDaCS N N N N Y N N YPerpetual Workbook N N N N Y N N Y
Features: Control Tools
Illustrations: Using Galletta Spreadsheet (1996)
Seeded with Errors
123456789
1011121314151617181920
A B C D E F GCash Budget Assuming 2%
Fall Spring Summer OverallCash, beginning $1,000 $1,360 $673Outflows - School costs $4,468 $4,474 $4,480 $13,422
Living Costs $4,172 $4,213 $4,485 $12,870Inflows - Loans $3,000 $3,000 $3,000 $9,000Support from home $6,000 $5,000 $6,000 $17,000Cash, end $1,360 $673 $980 $3,013
======= ======= ======= =======School (contractual): Tuition $4,115 $4,115 $4,115 $12,345Fees $53 $53 $53 $159School (other): books/supplies $300 $306 $312 $918Living (contractual) MONTHLY MONTHSHousing $450 4 $1,800 $1,800 $1,800 $5,400Insurance $53 4 $212 $212 $212 $636Living Costs (other)Food $330 4 $1,320 $1,346 $1,373 $4,039Entertainment $150 4 $600 $612 $624 $1,836Transportation $40 4 $160 $161 $164 $485Clothing $21 4 $80 $82 $84 $250
inflation/semester
P/D: Spreadsheet Detective Precedent Reports (Galletta example from SS Detective)
The Spreadsheet Detective Precedent ReportDouble click on a row to go to the corresponding cell, press F5 to return to this worksheet
Cell Label Formula Depth
'Model Copy'!F8 `Summer`CashEnd = F3`CashBegig - F4`Out-SchCos - E5`Spring`LiviCosts + F6`Infl-Loans# + F7`SupFroHom +
'Model Copy'!F3 `Summer`CashBegig = E8`Spring`CashEnd ++
'Model Copy'!E8 `Spring`CashEnd = E3`CashBegig - E4`Out-SchCos - E5`LiviCosts + E6`Infl-Loans# + E7`SupFroHom +++
'Model Copy'!F4 `Summer`Out-SchCos = SUM(F10:F12`SchoTuitn:`SchBoo/su) ++
'Model Copy'!F12 `Summer`SchBoo/su = ROUND(E12`Spring * (1 + $D$1`Assumng#) , 0) +++
'Model Copy'!E5 `Spring`LiviCosts = SUM(E14:E20`Housing:`Clothng) ++
'Model Copy'!E14 `Spring`Housing = D14`Fall +++
'Model Copy'!E17 `Spring`Food = ROUND(D17`Fall * (1 + $D$1`Assumng#) , 0) +++
'Model Copy'!E19 `Spring`Transon = ROUND(D19`Fall + (1 + $D$1`Assumng#) , 0) +++
'Model Copy'!E20 `Spring`Clothng = ROUND(D20`Fall * (1 + $D$1`Assumng#) , 0) +++
'Model Copy'!F7 `Summer`SupFroHom = ROUND(1000 + F4`Out-SchCos + F5`LiviCosts - F3`CashBegig - F6`Infl-Loans# , - 3) ++
'Model Copy'!F4 `Summer`Out-SchCos = SUM(F10:F12`SchoTuitn:`SchBoo/su) +++
'Model Copy'!F5 `Summer`LiviCosts = SUM(F12:F19`SchBoo/su:`Transon) +++
'Model Copy'!F3 `Summer`CashBegig = E8`Spring`CashEnd +++
Source/Copied Formula Analysis
123456789
1011121314151617181920
A B C D E F GCash Budget Assuming 2%
Fall Spring Summer OverallCash, beginning $1,000 $1,360 $673Outflows - School costs $4,468 $4,474 $4,480 $13,422
Living Costs $4,172 $4,213 $4,485 $12,870Inflows - Loans $3,000 $3,000 $3,000 $9,000Support from home $6,000 $5,000 $6,000 $17,000Cash, end $1,360 $673 $980 $3,013
======= ======= ======= =======School (contractual): Tuition $4,115 $4,115 $4,115 $12,345Fees $53 $53 $53 $159School (other): books/supplies $300 $306 $312 $918Living (contractual) MONTHLY MONTHSHousing $450 4 $1,800 $1,800 $1,800 $5,400Insurance $53 4 $212 $212 $212 $636Living Costs (other)Food $330 4 $1,320 $1,346 $1,373 $4,039Entertainment $150 4 $600 $612 $624 $1,836Transportation $40 4 $160 $161 $164 $485Clothing $21 4 $80 $82 $84 $250
inflation/semester
Shown with Spreadsheet Detective Add-In Identifies “unique” formulas, and effective copying operations Can reduce amount of manual code inspection required.
Formula Analysis
Formula Reports (Galletta example using SS Detective)
Map and Complexity Measures (Galletta example using Power Utility Pack)
Workbook/Worksheet Reports
Galletta Example using Power Utility Pack
Galletta Example using XLAnalyst
Native Excel
Precedents/Dependents Error Checking Trace Error Evaluate Formula Watch Window Track Changes
Observations “Advanced” Excel features (e.g.,
Solver, Pivot Tables, VBA) are not treated in great detail by the auditing tools.
Lack of design tools that help user/modeler to frame problem and develop initial model structure.
Anticipate a stronger linkage between auditing tools and control tools as regulations become more widespread.
Questions How effective are auditing tools in identifying errors? How effective are auditing tools in helping user
identify errors? How common are false positives and false
negatives? For which users can auditing tools be of benefit? Which models should be audited? (Butler, 2000) Can using these tools help to improve skills of
users/modelers? How much structure can be imposed on the
user/modeler in the design stage (e.g., Lotus Improv)?
Need for Research Detailed survey of tools, capabilities Comparison “Shootout”
Laboratory models Real-world spreadsheets
Empirical Studies Use of audit tools by end users Do users find more errors? Are they faster? Effect of larger models
Spreadsheet Engineering Control tools can help track model development process Tools to help improve a “basic” user’s modeling capabilities. Spreadsheet software improvements and paradigm shifts. Reverse Engineering a “spaghetti” spreadsheet
Use of tools in teaching Auditing tools to assess student spreadsheets Control tools to track development process Student use of auditing tools