Upload
candace-lane
View
220
Download
0
Embed Size (px)
Citation preview
Understanding Performance Audits
Deborah V. Loveless, CPA
November 10, 2015
Objectives
Present Overview of Government Auditing Standards Pertaining to Performance Audits
Highlight the Similarities and Differences Between Financial and Performance Audits
Explain Phases of a Performance Audit Illustrate Use of Performance Audits in
Tennessee
Comptroller of the Treasury Tennessee Comptroller of the Treasury elected
by General Assembly for two-year term Audit Divisions
State Audit Local Government Audit
Broad statutory authority to audit Broad statutory access to records Audit authority in all grants and contracts Fraud Hotline
Performance Audit Definition Government Auditing Standards definition:
Performance audits provide findings and conclusions based on an evaluation of sufficient, appropriate evidence against criteria. Performance audits provide objective analysis to assist management and those charged with governance and oversight in using the information to improve program performance and operations, reduce costs, facilitate decision making by parties with responsibility to oversee or initiate corrective action, and contribute to public accountability.
Matters that may be assessed include the effectiveness, efficiency, and economy with which the program function, system, or process operates. Performance audits are conducted in accordance with accepted standards.
Performance Audit Objectives Performance audit objectives vary widely and
include assessments of Effectiveness, economy and efficiency Internal controls Compliance Prospective analyses
Government Auditing Standards
General Standards Independence Professional Judgment Competence Quality Control and Assurance
Chapter 6 - Performance AuditField Work Standards Introduction (6.01) Reasonable Assurance (6.03) Significance in a Performance Audit (6.04) Audit Risk (6.05) Planning (6.06 – 6.51) Supervision (6.53 – 6.55) Obtaining Sufficient, Appropriate Evidence (6.56 – 6.78) Audit Documentation (6.79 – 6.85)
Reasonable AssuranceThe ultimate goal is to obtain reasonable assurance that evidence is sufficient and appropriate to support the auditors’ findings and conclusions in relationship to the audit objectives.Reasonable assurance varies:
Findings and conclusions Objectives Evidence (sufficiency and appropriateness) Professional judgment is critical
Significance GAGAS definition: relative importance of matter
within the context in which it is being considered, including quantitative and qualitative factors.
Use significance to Decide type of audit work to perform Decide extent of audit work to perform Evaluate results Develop conclusions, findings, & report
Significance v. Materiality
Audit Risk Possibility that the auditors’ findings,
conclusions, recommendations, or assurance may be improper or incomplete
Contributions to increased risk: Evidence is not sufficient or appropriate Inadequate audit process Intentional omissions Misleading information
(last 2 can be due to misrepresentations or fraud)
Planning Auditors must adequately plan and document
the planning of work necessary to address the audit objectives.
Define objectives, scope and methodology based on assessment of: Risk - of something going wrong Significance - of whatever going wrong
Planning is ongoing May have to adjust as we go along
Planning - Risk In planning, we must assess risk and
significance by understanding: Nature of program Potential report user’s needs Internal controls IS controls Legal, regulatory, contract provisions Fraud Ongoing investigations/legal proceedings Prior audit results
Plan the audit to reduce audit risk
Planning – Criteria Use criteria to evaluate performance GAGAS lists examples:
Laws, regulations Contracts, grant agreements Standards Measures, expected performance, benchmarks Defined business practices
Planning - Criteria Attributes of Suitable Criteria
Objectivity – Should be free from bias. Measurability – Should permit reasonably consistent
measurements, qualitative or quantitative, of subject matter.
Completeness – Should be sufficiently complete so that those relevant factors that would alter a conclusion about subject matter are not omitted.
Relevance – Should be relevant to the subject matter.
Planning - Criteria Other considerations:
Criteria identify the required or desired state or expectation with respect to the subject matter that is being measured or evaluated
Criteria provide a context for evaluating evidence and understanding the findings, conclusions, and recommendations included in the auditor’s report
In all cases, the criteria must be identified in the report
Planning - Evidence Evaluate whether information to be used as
evidence will be sufficient, relevant, etc. If available evidence will be insufficient:
Modify scope and methodology Identify additional sources Is that a finding in itself? - i.e., lack of internal
controls
Planning – Using the Work of Others Others = auditors, specialists Auditors should identify the work others have
done Avoid duplication Identify problem areas, recommendations for further
work, ideas, sources, etc.
If using others’ work – document their qualifications, independence, etc.
Planning – Assigning Staff Staff, other resources should be assigned so
that: Team has collective knowledge, experience Sufficient number of staff and supervisors On-the-job training can be provided Specialists available if needed
Must document their scope, etc.
Planning - Communicating Communicating with management, those
charged with governance and others Communicate overview of
Objectives Scope Methodology Timing
Communicate to Auditee management Those charged with governance Individuals contracting/requesting audit If audit based in law, legislative committee
Planning – Written Audit Plan Preparing audit plan
Auditors must prepare a written audit plan for each audit.
Form and content can vary Update plan with significant changes Allows supervisors to monitor:
Likelihood of obtaining sufficient, appropriate evidence Staffing sufficiency
Supervision Audit supervisors…must properly supervise
audit staff Problem with defining “properly” Evidence of supervision:
Supervisory sign-off, ticks Timeliness of sign-off Writing and clearing of coaching notes/points Documentation of team meetings
Obtaining sufficient, appropriate evidence Auditors must obtain sufficient, appropriate
evidence to provide a reasonable basis for their findings and conclusions
Evidence is evaluated as a whole Appropriateness
Relevance Validity Reliability
Hierarchy of Evidence Some evidence is stronger than other evidence
Obtained when controls are strong Obtained through auditor’s direct physical exam,
observation, or computation Obtained from original documents Obtained from interview with no interference Obtained from interview with staff who have direct
knowledge of program Auditor conducted surveys Auditor sampling
Evidence Sufficiency Sufficiency guidelines
As risk increases, need for quality and quantity of evidence increases Stronger evidence might allow for less evidence
Large volume of evidence does not compensate for lack of relevant, valid, or reliable evidence
Overall Assessment of Evidence
Must determine and document overall assessment of evidence for sufficiency and appropriateness used to support findings and conclusions
Evaluate in context of significance Evidence is not sufficient when:
High risk Carries significant limitations Does not address objectives or support findings
Elements of a Finding Finding Elements
Criteria Condition Cause Effect Recommendations
Early Communication
Should inform those charged with governance Significant and urgent deficiencies:
Compliance with laws, grants, contracts Abuse, fraud Internal controls
Early communication does not eliminate need for inclusion of issues in audit report
Audit Documentation Auditors should prepare audit documentation in
sufficient detail to enable an experienced auditor, having no previous connection to the audit, to understand from the audit documentation the nature, timing, extent, and results of audit procedures performed, the audit evidence obtained and its source and conclusions reached, including evidence that supports the auditors’ significant judgments and conclusions.
Documentation Documentation is important
Provides principal support for the report Aids auditors in conducting and supervising Allows for review of audit’s quality
GAGAS requires documentation of Audit’s objectives, scope, and methodology Work performed to support “significant judgments &
conclusions” Evidence of supervisory review
BEFORE report released
Documentation When auditors do not comply with GAGAS
standards Should document departure and impact on audit and
on audit conclusions Sometimes occurs due to
Law, regulation Scope limitations Records access limitations
Chapter 7 - Performance AuditReporting Standards Auditors must issue audit reports
communicating the results of each completed performance audit.
Form of the report Report contents Report issuance and distribution
Reporting Form Form of report can differ based on users’ needs Retrievable Acceptable forms include:
Written reports Letters Briefing slides Presentation materials
Reporting - Purpose Communicate results to
Those charged with governance Auditee officials Oversight officials
Make results less susceptible to misunderstanding
Make results available to public (as applicable) Facilitate follow-up
Report Contents Objectives, scope, and methodology
Provide context and perspective Including any limitations and constraints Detail items tested, populations, period covered,
sampling, etc. Any major assumptions
Report Contents Audit Findings
Including sufficient evidence to support findings Scope of internal control work and any identified
deficiencies Place findings in perspective, include background
information for context Describe report limitations or uncertainties Fraud occurred or is likely to have occurred
If management fails to report to appropriate external parties, auditor must report
Report Contents Conclusions
Based on audit objectives and findings Include logical inferences about the program based
on findings (not just a summary)
Report Contents Recommendations
Correct problems identified during the audit Addressed to those with authority Specific, practical, cost effective, and measurable
GAGAS compliance statement Auditee’s response
Can provide evaluation of response Can obtain oral comments due to time If auditee refuses to provide, indicate such
Report Contents Confidential or sensitive information
Legally protected Classified Public safety and security
Disclose omission and reason for omission Assess impact of public working papers access
Consider handling verbally Example: undercover license plates
Report Distribution Distribution to
Those charged with governance Auditee officials Oversight bodies/organizations arranging for audits
(i.e., legislative requesters) Legal authority or otherwise responsible for acting
on reports Anybody authorized as appropriate
Document any limitation on report distribution
Comparison of Financial Audits and Performance Audits
Similarities: Professional competence, due professional care,
and adequate planning and supervision are required Independence is required Materiality or significance is always considered Auditor may provide recommendations for correcting
a deficiency or improving a condition Auditor forms a conclusion or opinion about the
subject matter’s conformity with specified criteria
Comparison of Financial Audits and Performance Audits More similarities:
In both engagement types, the auditor: Gains an understanding of the subject matter Gains an understanding of internal control if relevant to the
subject matter Obtains reasonable assurance Identifies the criteria in the report Forms a conclusion based on evidence obtained Considers potential independence impairment
Comparison of Financial Audits and Performance Audits Major differences:
Financial Performance
Criteria are defined by the audit standards YES NO Auditor ordinarily decides on the applicable
criteria and may seek agreement from the responsible party about the appropriateness of the criteria for the engagement NO YES
Responsible party asserts as to conformance of the subject matter to the criteria YES NO
Responsible party always makes representation to the auditor YES NO
Comparison of Financial Audits and Performance Audits Reporting Requirements:
Financial Performance Identification of criteria used YES YES Description of significant limitations or
uncertainties that could affect the validity of the auditor’s opinion, conclusion, or findings YES YES
Description of the auditor’s responsibility YES YES Detailed description of work performed NO YES Information to provide context for and
perspective on what is being reported NO YES Engagement can result in a conclusion
rather than an opinion NO YES
Phases of a Performance Audit Audit Assignment/Audit Selection Planning Scoping Detailed Audit Report Writing Distribution
Audit Planning Identify major issues surrounding the agency
Problems? Is the program producing desired outcomes? Answer to a specific question? Achievements?
In order to identify the issues, we need basic background information
Sources Used in Planning Agency website Agency literature News search Prior performance
audits Other recent audits Statutes Rules Legislative history
Interview staff & management
Advocacy groups and stakeholders
Board members Professional literature Statistics/Performance
Measures
Planning During planning keep in mind the five elements
of findings: Condition – What is going on? Cause – Why is it happening? Effect – Why should we care? Criteria – What are we measuring against? Recommendation – What do we want them to do
about it?
Scoping Process of deciding which of the identified
issues to pursue in-depth What makes an issue a high priority?
High dollar High impact – public, other agencies, individuals High profile Agency audit history Inherently high risk Auditor experience
Detailed Audit
Perform fieldwork testing and research In-depth research on only the selected topics Identify rest of the elements Document what we already know or suspect Conduct surveys
Methodology - Interviews Differences between Performance and
Financial Interviews Rarely based on standardized checklists Starting “from scratch” Why and How are as important as What Generally not just updating last audit Different topics/Different ultimate purpose Can be broad vs. specific, known audit topic
Methodology - File Reviews Systematic review of agency records to collect
data Hard copy records Computerized records Both! – Understand overlap
Start analysis = evaluate mid-way Working paper minimum:
Methods – i.e., How selected sample File-by-file results Conclusion/Analysis
Report Writing What makes it a finding?
Almost all findings have recommendations Observations/Comments?
Can have recommendations, but less often the case Positive – agency didn’t do anything “wrong” Legislative interest Event updates Very minor issues Slightly outside of the scope
Agency must respond to findings, but not observations/comments
Report Writing Clear communication is an underlying
Government Auditing Standards principle Audience is diverse and busy Present enough information to…
Convince legislators and agency Understand major topics Answer likely objections
Performance Audits in Tennessee Governmental Entity Review Law
(1977 Sunset legislation) provides a responsible method for reviewing state
government entities to ensure the state regulation is beneficial rather than detrimental to the public interest
assigns a termination date to all state entities created by statute
authorizes program review audits (performance audits) by the Comptroller of the Treasury to aid in legislative review of entities
Performance Audits in Tennessee Sunset Audits
222 entities in initial legislation 260 entities in current legislation Average of 15 public hearings per year Average of 75 entities reviewed per year Average of 4-year extension (rarely the 8-year
extension allowed by statute) 166 terminations (7 in 2014)
Primary reasons for termination: inactive, fulfilled mission, or transfer of function to another entity
Tennessee Government Services Helps 452,000 job seekers and 10,000 veterans find
jobs
Helps 800 children find permanent homes
Answers over 284,000 child abuse, drug abuse, and mental health hotline calls
Administers 1,854,540 immunizations, 42,600 TB tests, and 16,500 lead blood level assessments
Tennessee Government Services Serves more than 1,700 schools, 930,000 students, and
70,000 teaching professionals
Ensures safety on 87,000 miles of state and federal highways
Controls 2,100 wildfires and manages the conservation of 185,000 acres
Disposes of more than 1 million pounds of hazardous household waste
Tennessee Government Services Routinely inspects:
92,014 fuel pumps
22,000 elevators
19,000 bridges
73 public airports
2,933 miles of rail
11,000 school buses and childcare vans
All X-ray equipment
101 credit unions
155 banks
Examples of Performance Audits in Tennessee
Department of Intellectual and Developmental Disabilities
Department of Children’s Services Tennessee Film, Entertainment and Music Commission Department of Labor and Workforce Development Douglas Henry State Museum Commission Tennessee State Board of Accountancy Tennessee Rehabilitative Initiative in Correction Board
Department of Intellectual and Developmental Disabilities (October 2013) Objective: To determine whether the department had
established sufficient services for individuals with developmental disabilities in accordance with statute
Findings: The department had not established sufficient services for individuals with
developmental disabilities obtained data on the number of Tennesseans with
developmental disabilities coordinated with other state agencies to offer services for
individuals with developmental disabilities pursued funding to provide services for individuals with
developmental disabilities
Department of Children’s Services (January 2014) 22 objectives dealing with departmental operations in
Child Safety & Child Health, Juvenile Justice, Child Programs, and Computer Operations
12 findings and 13 observations addressing Child Safety investigation thoroughness child protective investigative operations tracking of child abuse and neglect referrals background checks on individuals providing services to
children in state custody
Department of Children's Services (Continued)
probation and aftercare supervision requirements sufficiency of residential treatment options calculation and analysis of recidivism assessment of foster care placement and monitoring of private
provider placement practices assessment and provision of long-term medical care to
methamphetamine-exposed children in state custody analysis of call wait times for Tennessee Child Abuse Hotline Child Abuse investigative caseloads
Tennessee Film, Entertainment and Music Commission (January 2013) Objectives: to evaluate the effectiveness of the film
incentive program and to evaluate the incentive award process
Findings The Department of Economic and Community Development and the
Department of Revenue have disregarded their statutory responsibility and exercised poor management and administrative oversight of the state’s headquarters film incentive program
The Tennessee Spend, which is eligible expenses used to calculate the incentive payments, is likely to be significantly overstated for reasons including poor internal controls, insufficient policy, and lack of management accountability among the departmentsinvolved with its determination
Department of Labor and Workforce Development (October 2014) Objective
To determine whether there is adequate and appropriate oversight of inspectors and inspections in the Elevator, Boiler, and Amusement Device Units
Finding The department does not yet have a viable amusement device
regulatory unit six years after jurisdiction was transferred from the Department of Commerce and Insurance
Department of Labor and Workforce Development (Continued) Objective
To determine whether there are adequate controls and plans to ensure that adequately trained mine rescue teams are in place as required by statute
Finding The department’s Mine Safety Unit is out of compliance with
state statute in regard to its mine rescue teams’ distance from underground mine operations
Department of Labor and Workforce Development (Continued) Objective
To determine whether the department has made a detailed contingency plan to deal with federal government shutdowns
Observation The department has only minimal guidance to address actions
in case of a federal government shutdown
Douglas Henry State Museum Commission (September 2015) Objectives
To determine if staff had taken adequate steps to preserve its collection
To determine controls over alcohol inventory
Findings Water problems in the museum pose a threat to artifacts on
display or in storage Management lacked internal controls to ensure security of
alcohol stored on site
Tennessee State Board of Accountancy (September 2015) Objectives
Determine the effect of an inactive Peer Review Oversight Committee on the board’s ability to oversee accounting professionals
Determine other states’ requirements governing the confidentiality of peer review results
Finding Under its current structure, the Tennessee State Board of
Accountancy is limited in its ability to oversee the peer review program in Tennessee
Tennessee Rehabilitative Initiative in Correction Board (October 2015) Objectives
Document the controls over the accounts receivable and accounts payable processes, review aging report, and determine process for collecting past-due receivables
Ensure the accuracy of the fiscal year 2014 account balances that were reported to the board of directors, as well as the amounts reported in the state’s CAFR
Determine if TRICOR entered into formal agreements with its Cook Chill customers and assess the Cook Chill program’s financial position
With respect to TIMS, determine if TRICOR management followed information systems’ industry best practices regarding system controls
Tennessee Rehabilitative Initiative in Correction Board (Continued) Findings
TRICOR’s executive director failed to ensure fiscal staff performed key fiscal and financial reporting functions, which led to a pervasive breakdown of control that resulted in material financial misstatements and board decisions that were based on inaccurate financial information
TRICOR and the Department of Correction managements continue to operate the Tennessee Cook Chill program without executed agreements, resulting in unmet expectations and a program net loss of more than $4 million as of March 31, 2015
TRICOR did not provide adequate internal controls in four specific areas
Understanding Performance Audits
Deborah V. Loveless, CPA
November 10, 2015