FM Sector Quality and Results

1

FM Sector Quality and Results

2

FM Quality and Results

Objectives of the session:-

To build a common understanding of

• FM Quality and Results Framework

• Current Issues and Initiatives

• The way forward to enhance quality and results

3


The results we seek:-

• Improved borrowing countries’ financial management performance (client focus/ development objective)

• Acceptable assurance on the use of Bank loan proceeds (corporate focus/ fiduciary objective)

4

FM Quality and Results Framework

FM Practices Manual

Regional Quality Arrangements & Joint Evaluation

Independent Reviews (e.g. QAG, IAD, IEG, IDA 14 Controls Review)

Tools (e.g. ARCS, PRIMA, IRIS)

5

FM Practices Manual Regional Quality

Arrangements & Joint Evaluation




6

• Financial Management Practices Manual

• FM Practices Manual (FMPM) provides implementation guidance on OP/BP 10.02

• FMPM is currently under review and will be covered on other sessions at the Forum


7

QUALITY and RESULTS COMMITTEE- Revision of FM Practices Manual

Ste

p 1

L

AU

NC

H

STATUS AS AT MARCH 24, 2008 PRE-FORUM ACTIVITIES STEPS 1 & 2

Step 3

Drafting

Step 4

Finalize AND DISSEMINATE

Step 2

Discussion and Feedback at FID.

FORUM

LAUNCH Initial discussions at Q&R Committee Done Prepare Survey Questions and Committee Done Review Confirm broad approach with FMSB Done Heads up and alert to FM Group Merged Firm up with IT objectives and survey tool Done Arrange Regional Consultations Done Review of survey questions by FMSB Done Launch Survey Done

DISCUSSION AND FEEDBACK Convene Regional meetings Done R Distill messages and feedback collected from user survey, IDA14 FM , QAG, Legal Done Distill messages from Consultations Done Prepare draft outline of main messages Rev. Done Validate outline with Q&R Committee Done Validate outline with FMSB Done

FIDUCIARY FORUM DISCUSSION AND FEEDBACK Circulate 2-3 page synopsis of proposed revisions Done Discuss and obtain feedback during the forum Mar. 28 Distill feedback and messages Mar. 28

Q&R Committee Discussion Mar. 28

Work Step & End Date January February March April May/June

8

FM Practices Manual





9

• FM work supported by Tools• Audit Reporting Compliance System

(ARCS)

• PRIMA

• IRIS


10

FM Practices Manual





11

• Regional Quality Assurance Arrangements

• Ensure that FM work is carried out in accordance with provisions of the FM Manual

• Supplemented by region specific business procedures as called for by FMPM

• Have been implemented across all regions.


12

• Para. 82 of FMPM:- Joint CSR/FM Network Evaluation of Quality Arrangements

• Compliance :- Objective is to determine whether the Quality Assurance for FM work is sufficiently robust, implemented as designed and in accordance with the FMPM.

• Quality:- whether judgments and conclusions of FM work are being properly supported by evidence

• Efficiency:- work is compliant and of good quality, sufficient to be relied upon by the Controller as a key pillar in disbursement simplification reform.


13

• Joint CSR/FM Network Evaluation of Quality Arrangements

• Detailed Implementation Arrangements are being defined, however here are the broad basic principles:-

• The reviews will be Annual• Anticipated start date: later this calendar year• Arrangements being developed by the FMSB, OPCVP

and CSRVP, through the Quality and Results Committee.

• Risk-based approach envisages two-phase process


14


• Phase I initial high level confirmation of degree of implementation of Regional QA

• Phase II, compliance and substantive testing, grounded in risk based approach and based on accepted peer review principles


15


• To reduce oversight fatigue, the reviews will seek to create synergies between IDA 14 review work, QAG reviews and work performed by IAD

• Team members will be drawn from the Network, Regions and CSR

• At end of one year pilot the roles, responsibilities, methodology will be re-evaluated


16

FM Practices Manual





17

• Other institutional arrangements to enhance quality assurance

• Quality Assurance Group (QAG)

• IDA 14 Controls review

• Internal Audit Department (IAD)

• Independent Evaluation Group (IEG)


18

Issues

Moderately Satisfactory

or Better (1+2+3)

Appropriate level of FM input in supervision activities 91%

FM issues (arising from SA, SOE, FMR and Audit Reports Reviews) dealt with in a timely manner and relevant remedies applied

86%

FM issues adequately identified and followed up appropriately

91%

Task team provided appropriate level of support to the borrower in working out problems and improving its FM arrangements

93%

Compared with 94% in QSA6





• QSA 7

19

QEA 8 conclusions & recommendations:

• The relatively high risks involved in decentralized projects and the diversity of measures taken to mitigate these risks, suggests that quality could be enhanced by wider dissemination of good practice examples and updating the FM/CDD guidelines.

• The continued high level of performance of DPOs can be attributed to the much wider availability of country diagnostics (CFAAs, PERs, ROSCs, PFM reviews); better integration of FM staff into the teams; FM staff having gained more experience with these kind of operations; as well as the clarity of the new OP8.60 policy and accompanying guidelines.

• While in general documentation is of reasonable quality, it remains one of the areas that need further improvement. There is still an inconsistency in the quality of preparation and comprehensiveness of documentation.


20

QEA 8 The QAG Process – Going Forward

• As indicated in previous QEAs, the sustained good performance of FM quality at entry suggests that this is an area that no longer requires specialist assessment in future QEA reviews and that it is timely to consider how the FM network, and the Bank, could maximize benefits from quality review processes in the future. In this regard a number of issues may be considered, including:

• Impact of the Implementation of the Joint Evaluation Mechanisms

• Supervision work should be linked with the updated risk assessment framework to follow up on the operational implications of the risk mitigation measure identified during the preparation phase of the project.


21


IDA 14 Internal Control Review

• Source• IDA 14 Replenishment Report• Required management to “carry out an independent [through

IEG], comprehensive assessment of IDA’s internal control framework, including internal controls over IDA operations and compliance with its charter and policies.”

• Responsible Groups• Management (CSR and OPCS lead and coordinate “self-

assessment”)• IAD reviews management findings/conclusions• IEG evaluates both

• Methodology• COSO framework adapted to address control framework over

operations

22

• Part I: Transaction-level controls• Detailed identification of processes, key controls and attributes

that govern IDA’s primary instruments (CAS, DPL and IL) • Compliance testing of 29 business processes and 115 controls (6

FM controls, 10 procurement, and 13 disbursement)

• Part I completed in FY07; results pointed to the need to:• Reform and rationalize policies, procedures, and controls that

apply to Investment Lending (IL)• Automate work-flow arrangements in the context of IDA

operations to improve efficiency and document accessibility• Review, during Part II, the quality aspects of fiduciary controls

(FM and procurement)

FM Quality and Results IDA 14 Internal Control Review

23


• Part I Findings (FM):• Big problem with timely accessibility of relevant documents

(generally)• Variances in regional implementation of FM manual

• FM control failures• Due to unavailability of documents, unable to determine whether

RMFM (or delegate) reviewed consistency of FMA with PAD• Majority of FM Supervision reports prepared after November

2005 were not in accordance with the FM manual• In about 40% of projects reviewed, there was no documentation

evidencing communication by the FMS to the TTL of the FM rating for the ISR

• In one case, the FM rating provided by the FMS was different from the FM rating in the final ISR; no explanation was attached

24


• Part II: Effectiveness, efficiency and entity-level controls• Focus on identifying, mapping, and assessing entity-level

controls along the 5 COSO components • Address issues deferred from Part I (including AAA, Country

Offices, Information Technology, Fraud and Corruption Risk) and follow-on work on exceptions identified under Part I (quality of fiduciary controls)

• Link the findings and results of Parts I and II and report on overall findings and conclusions on internal controls framework over effectiveness and efficiency of IDA’s operations

• Part II work completed; report in drafting stage

25

Ste

p 1

Id

en

tifi

ca

tio

n

of

FM

Is

su

es

Communication

Step 3

Results Analysis

Step 4

Reporting

Step 2

Assessment of Regions

Identification of FM Issues Prepare fiduciary issues tracker Prepare consolidated issues matrix Analyze common themes Discuss review methodology and objectives

with FM Sector Board (FMSB) Develop draft questionnaire to interview

regional managers, OPCFM & FMSBAssessment of Regions Finalize questionnaire Conduct interviews with RMFM and team Review and assess quality controls

established in each region Obtain evidence to support compliance with

these controls where applicableResults Analysis Identify variances and gaps Evaluate compensating controls where they

exist Evaluate impact of variances and gapsReporting Present summary findings to FMSB for

validation Prepare draft report and recommendations Request management responses Finalize report and action plan

Work Step & Time Frame

26


• Summary of Part II FM findings:• Quality Assurance (QA) arrangements

• All Regions have QA arrangements that have been

documented and disseminated to staff • Difficult to ascertain in some Regions if QA arrangements for

projects under implementation are completely in compliance with the FM manual

• There is a range of supplementary guidance for FM staff

• Projects under preparation • QA arrangements are operating effectively • Variances among Regions appear to be reasonable and

generally responsive to Regional structure & organization

27


• Projects during implementation• Some Regions are not in full compliance with their QA

arrangements (e.g., monitoring of interim financial reports, recording in ARCS of FMS review of audit reports)

• Some gaps in the FM Network QA arrangements are not addressed in the Regional QA arrangements (e.g., FM conditions and legal covenants, audit exemptions)

• There are Regional variances in QA arrangements – many make sense, but in some Regions these variances represent risks and possible control gaps (e.g., documentation of supervision, timing of quality interventions)

• Documentation and communication• Regional variances in level of detail and content of

documentation of FM work• No uniform system of filing (e.g., naming conventions)

28


• Part II concluded that deficiencies exist in the following areas:• Documentation of review of audited financial

statements by an FMS• Management oversight of Audit Report Compliance

Systems (ARCS) data• Documentation of FM supervision• Timing of quality interventions during supervision

• Identification and monitoring of risky projects.

29


• Part II Action Plan agreed with the FMSB• Regions to consolidate and update QA arrangements• FM manual to be reviewed and updated – to consider

IDA 14 findings• Joint CSR/OPCS Evaluation of FM Network Quality

Arrangements to begin – draft methodology to be updated for IDA 14 findings

• ARCS data entry to be brought up to date• FMSB to monitor ARCS regularly to ensure reliability

of this institutional monitoring system• Data entry of baseline information (risk ratings and

interim financial reports) in PRIMA to be completed

30


• Joint CSR/OPCS Evaluation of FM Network Quality Arrangements to specifically monitor improvements to QA arrangements over FM supervision work

• Pending establishment of the institutional Enterprise Content and Document Management system, Q&RC to review existing good practices and develop guidance for staff

• Fiduciary Forum to be used to remind staff about the importance of:

• Documenting FM supervision work• Monitoring interim financial reports• Retention and timely accessibility of FM documents

31


The results we seek:-

• Improved borrowing countries’ financial management performance (client focus/ development objective)

• Acceptable assurance on the use of Bank loan proceeds (corporate focus/ fiduciary objective)

32


Objectives of the session:-

To build a common understanding of

• FM Quality and Results Framework

• Current Issues and Initiatives

• The way forward to enhance quality and results

33


FM Practices Manual




Documents

FM Sector Quality and Results