1

Audit Preparation - Evidence

ReliabilityFirst CIP Webinar

Thursday, September 30, 2010Lew Folkerth, Senior Engineer - Compliance

Painting the “Compliance Picture”

All aspects of each requirementThe entire compliance periodAll appropriate assetsAppropriate level of detail to

demonstrate complianceDuring an audit, the picture is

painted by the SMEThe picture is painted with

EVIDENCE2

Evidence Preparation

What is evidence?What constitutes “good” or “quality”

evidence?What is the best way to present evidence?When should evidence be submitted?Can an entity be found compliant with a

requirement based solely on pre-audit evidence submitted?

3

Evidence Preparation

What is evidence?• Physical

Obtained by direct inspection or observation

• DocumentaryProcesses, procedures, databases, written records

• Testimonial Interviews, inquiries, questionnaires, attestations

What constitutes “good” or “quality” evidence? What is the best way to present evidence? When should evidence be submitted? Can an entity be found compliant with a requirement based solely on

pre-audit evidence submitted?4

Evidence Preparation

What is evidence?

What constitutes “good” or “quality” evidence?• Sufficient

Measure of the quantity of evidence that supports the audit objectives – enough to persuade a knowledgeable person that the findings are reasonable

• AppropriateMeasure of the quality of evidence that includes relevance,

validity and reliability What is the best way to present evidence? When should evidence be submitted? Can an entity be found compliant with a requirement based solely on

pre-audit evidence submitted?5

Evidence Preparation

What is evidence? What constitutes “good” or “quality” evidence?

What is the best way to present evidence?• There is no “best” way – whatever fits your evidence

organization• However, here are some tips on how to organize and

present the evidence When should evidence be submitted? Can an entity be found compliant with a requirement based solely

on pre-audit evidence submitted?

6

Evidence Organization and Presentation

PDF documents are greatly preferredPDFs should be searchableIf you want to scan pages to show

signatures, only scan the signature page and include it after the searchable document

7

Suggestions: • One PDF per standard (may be large)• All versions of compliance evidence

applicable to the compliance period• Revisions of each document together, in

reverse chronological order• Two sets of bookmarks:

First set to locate documentsSecond set to locate sections within documents

which satisfy a requirement8

Evidence Organization and Presentation

9

Evidence Organization and Presentation

10

Evidence Organization and Presentation

Evidence Preparation

What is evidence? What constitutes “good” or “quality” evidence? What is the best way to present evidence?

When should evidence be submitted?• Pay careful attention to the audit notification letter• Most evidence will be needed 40 days before the start

of the audit• Provide additional information as requested – sooner

is better than later Can an entity be found compliant with a requirement based solely

on pre-audit evidence submitted?

11

Evidence Preparation

What is evidence? What constitutes “good” or “quality” evidence? What is the best way to present evidence? When should evidence be submitted?

Can an entity be found compliant with a requirement based solely on pre-audit evidence submitted?• YES! At least for some (perhaps many) requirements,

although typically not a majority of requirements.• But, careful and thorough preparation is required to

achieve this.

12

Evidence Presentation

Keep it simple – the audit team will indicate where more detail is needed

Don’t evade questions – “I don’t know” is an acceptable answer, as long as follow-up occurs

Don’t provide slide decks to demonstrate compliance – the audit team is interested in evidence

13

PDF Bookmarks

Demo

14

Questions

Questions should be emailed to Karen Yoder (karen.yoder@rfirst.org) Subject: “CIP WEBINAR”

Questions will be considered in the order they are received

Clarifying questions are welcome and we will do our best to answer during the question period

Challenges to a position should be addressed to the presenter and will be taken offline