How SOX-404 Exposed The Dysfunctional Marriage Between Business And IT...

And How Lawyers Can Help

Gene Kim@realgenekim

genek@realgenekim.me

Where Did The High Performers Come From?

Since 1999, We’ve Benchmarked 1500+ IT Organizations

Source: IT Process Institute (2008) Source: EMA (2009)

Visible Ops: Playbook of High Performers

• The IT Process Institute has been studying high-performing organizations since 1999– What is common to all the high

performers?– What is different between them

and average and low performers?

– How did they become great?

• Answers have been codified in the Visible Ops Methodology

www.ITPI.org

Story of GAIT and SOX-404

• Tell you a story involving IT organizations, businesses, their auditors, the auditors’ regulators– A large and complex problem– How defining two words solved it and made a

difference• My top lessons learned• What I’m doing about it now

Problem Statement

• 2001: Enron fails ($63B market cap), Arthur Andersen dissolution

• 2002: WorldCom (peak $117B market cap)

• Leads to Sarbanes-Oxley Act of 2002

“OMG. 952 IT Deficiencies?!?”

9© 2004 KPMG LLP, the U.S. member firm of KPMG International, a Swiss cooperative. All rights reserved. Printed in the U.S.A.

What were/are people worried about?

IT controls dominate the deficiencies, significant deficiencies, and material weaknesses identified through the S-O 404 assessment.The estimated percentage of deficiencies identified show IT controls accounting for the most (34 percent), followed distantly by revenue (13 percent), procure to pay (10 percent), and fixed assets (10 percent). 

The estimated percentage of significant deficiencies identified again shows IT controls leading the way (23 percent), followed by financial reporting and close (14 percent), procure to pay (13 percent), and revenue (12 percent).  

The estimated percentages of material weaknesses identified include IT controls (27 percent), revenue (18 percent), taxes (11 percent), and financial reporting and close (10 percent).  It is important to note that the results presented here are based on self-reporting by the companies that participated in the survey. Conclusions may be affected by the differing methods companies use to report on various elements of Sarbanes-Oxley compliance.

Holy cow!!! Enron wasn’t caused by a DBA. So, why are the auditors digging here?? --gk

February 2006 Corporate Finance10

PROBLEMS & CHALLENGES

0%

50%

100%

%

EFFORT DEFICIENCIES

IT V NON - IT COMPARISON

IT

NON - IT

Disproportionate Share: Compliance effort. Deficiencies. Non Finance Apps.

Financial Statement Impact: Indirect linkage Least likely impact

Business & IT integration.

0%

50%

100%

%

Fin Apps Non Fin Apps

Applications in Scope

Again, holy cow!!! If the risk isn’t in IT, then auditors are not only generating efforts, but finding

deficiencies that don’t matters… --gk

www.theiia.org

Vision: Create Equivalence to Nine Firm Document on IT

Control ExceptionsGAIT takes the approach used in the nine firm document.

GAIT represents the upfront scoping exercise to appropriately identify the IT controls work relevant to overall internal controls objectives

Chart 3: Evaluating Information Technology General Control (ITGC) Deficiencies, “A Framework for Evaluating Control Exceptions and Deficiencies” (December 20, 2004)

There Had To Be A Better Way

SOX-404 Value Network: Primary Constituencies

www.theiia.org

The Problem

• The IT portions of SOX-404 compliance has frustrated auditors and management– Significant key controls reside inside IT and IT

processes as well as in the business processes– No well-established guidance for scoping IT work

results in inconsistency and the process being overly subjective

– Sometimes result in overly broad scope and excessive testing costs

– Significant risks to financial assertions may be left unaddressed

– Suboptimal use of scarce resources

www.theiia.org

Why Is There A Problem?

• No clear guidance exists to define how IT processes and activities can invalidate financial application processing or financial assertions– COSO provides an accepted construct for defining

overall internal control objectives, assertions, risks and controls, but its application to the IT environmet is ambiguous

– COBIT doesn’t provide a clear mechanism to scope IT processes and controls to the achievement of specific internal control objectives (e.g., COSO objective for internal control over financial reporting)

• Something else is needed…

COSO ERM Cube v2

COBIT

Why Is There A Problem?• No clear guidance exists to define how IT processes and

activities can invalidate financial application processing or financial assertions– COSO provides an accepted construct for defining overall internal

control objectives, assertions, risks and controls, but its application to the IT environment is ambiguous

– COBIT doesn’t provide a clear mechanism to scope IT processes and controls to the achievement of specific internal control objectives (e.g., COSO objective for internal control over financial reporting)

• Something else is needed…

Thought Experiment

• Auditors vs. Management• We can agree that there are two extremes in

spectrum of financial reporting risk– eBay auction settlement business process– Grain elevators

• Extremes are easy… Middle is hard…

www.theiia.org

Language Is Often An Obstacle

• In Newton’s time, there were not concrete terms for several critical concepts:– Force, acceleration, mass, inertia

• In the following slide, note how difficult it was for Newton to frame the “three laws of motion” without these concepts…

www.theiia.org

Early Drafts Of Three Laws Of Motion

• 1. If a quantity once move it will never rest unless hindered by some externall cause.

• 2. A quantity will always move on in the same straight line (not changing the determination nor celerity of its motion) unless some externall cause divert it.

• 3. There is exactly so much required and no more force to reduce a body to rest as there was to put it upon motion.

• Axiom 100: A body once moved will always keep the same celerity, quantity and determination of its motion

• Axiom 103: ...as the body (a) is to the body (b0), so must the power of efficacy vigor strength or virtue of the cause which begets the same quantity of velocity

Source: Isaac Newton, James Gleick.

www.theiia.org

Benchmarks

• Pythagorean theorem: 24 words• Archimedes' Principle: 67 words• Newton’s Three Laws Of Motion: 91 words• The 10 Commandments: 179 words• GAIT Proposed Principles v3.0: 168 words• The Gettysburg Address: 286 words• The Declaration of Independence: 1,300 words • GAIT Principles v1.3: 6,856 words • GAIT Methodology v2.2: 11,348 words• The US Government regulations on the sale of cabbage:

26,911 words

Solution: GAIT…• Released in Feb 2007, Establishes four principles that

– Defines the relevance of IT infrastructure elements to financial reporting integrity

– Define the three types of IT processes that can affect them: change management and systems development, operations and security

– Defines an end-to-end process view of these three processes– Defines an approach to defining objectives and key controls within those

three processes• Provides a methodology and thinking process that

continues the top down, risk based approach started in AS2 to scope IT general controls

• Provides a common context for management and auditors to support and test management’s assessment that the necessary IT controls exist and are effective– Initial target is internal control objectives for financial reporting, but

should extend to operating effectiveness and complying with laws and regulations (as defined by COSO)

GAIT Principle #1

• The only IT infrastructure elements (e.g., databases, operating systems, networks) relevant to ITGC assessment are those that support financially-significant applications and data.

(“What are the relevant IT infrastructure elements?”)

GAIT Principle #2

• The IT processes primarily relevant to ITGC assessment are those that directly impact the integrity of financially-significant applications and data:– Change management and systems development: the processes

around developing, implementing, and maintaining financially significant applications and supporting IT infrastructure

– Operations management: the processes around managing the integrity of production data and program execution

– Security management: the processes around limiting access to information assets

(“What are the relevant end-to-end IT processes?”)

GAIT Principle #3

• Implications to the reliability of financially-significant applications and data, including controls, are based upon the achievement or failure of IT process objectives, not the design and operating effectiveness of the individual controls within those processes.

(“What are the relevant objectives of those IT processes? In other words, we shouldn’t get

carried away when reaching a conclusion when testing a control.”)

GAIT Principle #4

• The basis for identifying key controls in the three IT processes is based on:– Inherent risk of not achieving the IT process objectives– IT process risk indicators

(“How do we select key controls within those IT processes?”)

GAIT Scoping: Step By Step

Evaluate the risks related to (and within) the IT processes which manage the infrastructure & apps

Evaluate overall entity level controls

Identify IT entity level elements and the demonstrated maturity of the process

Identify key financial statement captions

Identify the general ledger accounts related to the key financial statement accounts (significant account)

Identify key transaction processes that affect the general ledger accounts

Identify and understand related business processes

Identify and understand applications and modules that support financially relevant business processes

Analyze the risks within the integrated business process (Identify risks)

Identify manual & automated controls & key functionality within the process that mitigate the risks (Identify key controls)

Identify IT infrastructure elements which support the application (the rest of the stack)

Identify and understand infrastructure that supports the business processes

Validate IT entity level controls

GAIT Starts Here

AS2 begins here

GAIT Tools

• Principles Document• Scenarios and Tutorials

– Online auction settlement process (high IT)– Rebate approval process (med IT)– Option expensing process (low IT)

• Ask Dr. GAIT

Conclusions and Lessons Learned, Continued

► Improved audit comment wording helps to connect to things management cares about:

• “We noted poor change control procedures and were unable to obtain comfort that all changes were authorized and tested as required”

-- vs. -- • “Poor change control practices introduced the risk of

unauthorized or untested changes to key data such as annual threshold amounts for toxic chemical releases. Given the level of precision applied to reviewing the final report downstream, it is unlikely management would detect such errors. Our testing disclosed numerous “break/fix” changes had been made to code or data without supervisory review and approval or notifying the users.”

GAIT Evolution

• Elements of GAIT was incorporated into PCAOB AS-5

• GAIT-R for Business Risk– To me, it's the first really well thought out way of

linking IT to any COSO internal control objective– Unlike ITIL, COBIT: it helps focus on what matters

• The Integrated Auditing Project (“Magic Glasses”)

PCI Problem Definition

• Success of any PCI DSS compliance initiative is very dependent on accurate definition and scoping of the Cardholder Data Environment.

• There is a wide variance in practice, experience and guidance in merchant and QSA community.

• These contribute to scoping errors that result in:– Overly narrow scope that jeopardizes cardholder data– Overly broad scope that adds unnecessary cost and effort

for compliance – Decreased confidence in and frustration with the PCI DSS

standard

33

34

35

36

Source: Gartner RVM Model (Proctor, Smith)

37

38

Top A-Ha Moments

• I love auditors: they have a comprehensive vocabulary that we need – otherwise, we’re stuck in Flatland

• Principles based guidance is great, as long as the words are precisely defined

• Auditors have seen the dead people longer than anyone

• It is possible to make a difference, even in complex social scenarios

• COSO Cube is simple but great

You are only as smart as theaverage

of the top 5 people you hang out with

40

The Prescriptive DevOps Cookbook• “DevOps Cookbook” Authors

– Patrick DeBois, Mike Orzen, John Willis

• Goals– Codify how to start and finish

DevOps transformations– How does Development, IT

Operations and Infosec become dependable partners

– Describe in detail how to replicate the transformations describe in “When IT Fails: The Novel”

“The Goal” by Dr. Eliyahu Goldratt

43

44

Fred Pond, CIO, Columbia Sportswear

• “When you finish that book, everyone on my team will need to read it, as well as my auditors, my boss, and my boss’ boss…”

When IT Fails: The Novel and The DevOps Cookbook

• Coming in July 2012

• “In the tradition of the best MBA case studies, this book should be mandatory reading for business and IT graduates alike.”Paul Muller, VP Software Marketing, Hewlett-Packard

• “The greatest IT management book of our generation.”Branden Williams, CTO Marketing, RSA

Gene Kim, Tripwire founder, Visible Ops co-author

When IT Fails: The Novel and The DevOps Cookbook

• Our mission is to positively affect the lives of 1 million IT workers by 2017

• If you would like the “Top 10 Things Infosec Needs To Know About DevOps,” sample chapters and updates on the book:

– Sign up at http://itrevolution.com – Email genek@realgenekim.me– Hand me a business card

Gene Kim, Tripwire founder, Visible Ops co-author

If you’d like the slides from today’s presentation…

• Text your name, email, website and the number 59871 to +1 (858) 598-3980

• Visit: http://www.instantcustomer.com/go/59871

• Or, scan this QR code:

48