SUB SERIES NO.MS5.1

Water Quality Management Series

SUB SERIES NO.MS5.1EDITION 1

FRAMEWORKFOR A

WATER QUALITY MANAGEMENT PERFORMANCE ASSESSMENT SYSTEM

(WQMPAS)

Department of Water Affairsand Forestry

June 2000

Final Report - 1 - June 2000

Water Quality

Management Series

SUB SERIES NO. MS5.1

EDITION 1

FRAMEWORK FOR A

WATER QUALITY MANAGEMENT PERFORMANCE ASSESSMENT SYSTEM

(WQMPAS)

Department of Water Affairs and ForestryJune 2000


Published by

the Department of Water Affairs and ForestryPrivate Bag X313PRETORIA, 0001

Republic of South Africa

Tel: (012) 336-7500/+2712 336-7500Fax: (012) 323-0321/+2712 323-0321

Copyright reserved

No part of this publication may be reproduced in any manner without full acknowledgement of the source

This report should be cited as:

Republic of South Africa, Department of Water Affairs and Forestry. 2000.Sub –Series No. MS5.1, Edition 1: Framework for a Water Quality Management Performance Assessment System (WQMPAS). Pretoria.

Coordinated by:

Effectiveness & Performance Centre CC107 Valley StreetClydesdalePRETORIA, 0001Republic of South Africa


Sub-Series MS5.1 Edition 1

DOCUMENT INDEX

MS5.1 Framework for a Water Quality Management Performance Assessment System (WQMPAS).

Final Report - i - June 2000


APPROVAL

TITLE: Framework for a Water Quality Management Performance Assessment System (WQMPAS).

CONSULTANTS: Effectiveness and Performance Centre CC (EPC)

AUTHORS: M. Roos & A. Van Niekerk

PROJECT NAME: Development of a Water Quality Management Performance Assessment System

ISSUE: Edition 1

REPORT STATUS: Final Report

CLIENT: Directorate Water Quality Management, Department of Water Affairs and Forestry

REPORT NO.: MS5.1

FILE NO.: 16/3/4/43

DATE: June 2000

Approved for Effectiveness and Performance Centre CC.

Ms. M. Roos

Approved for the Department of Water Affairs and Forestry

Mr. P. ViljoenDeputy Director: Water Quality Management:

Management Systems

Mr. J.L.J. Van Der WesthuizenDirector: Water Quality Management

Final Report - ii - June 2000


ACKNOWLEDGEMENTS

The following individuals are thanked for their contributions:

Project Management Committee:

Mr. P. Viljoen Project ManagerMr. J.J. Van Wyk Assistant Project ManagerMs. P Moodley Project MemberMs. M. Roos Consultant Team Leader (EPC)Ms. A. Van Niekerk Consultant Project Member (EPC)

Interviewees:

Ms. B. Schreiner Chief Director: Water Use and ConservationDr. H. van Vliet Chief Director: Scientific ServicesMr. Claus Triebel Special AdviserMr. H. Karodia Director: Catchment ManagementMr. R. Makhado Director: Internal AuditMs. M. Hinsch Deputy Director: WQM–Urban Development

& AgricultureMr. L. Bredenhann Deputy Director: WQM–Waste ManagementMs. H. Mackay Assistant Director: Scientific and Ecological

ServicesMr. G. Grobler Assistant Director: WQM: Management

SystemsMr. L. Gravelet-Blondin Deputy Director: KwaZulu-Natal Regional

Office: Water Quality ManagementMr. M. Keet Deputy Director: Gauteng Regional Office:

Water Quality ManagementMr. G. McConkey Deputy Director: Western Cape Regional

Office: Water Quality ManagementMr. S. Van Der Westhuizen Director: Water Quality ManagementMr. L. Van Den Berg Chief Engineer: International ProjectsMr. M.P. Nepfumbada Director: Institute for Water Quality Studies

Final Report - iii - June 2000


EXECUTIVE SUMMARY

The DWAF Directorate Water Quality Management (WQM) has recognised the need to implement a Water Quality Management Performance Assessment System (WQMPAS). The WQMPAS required should focus specifically on performance and the establishment of a performance reporting system.

This document reports the results of the preliminary investigation and development of an initial framework for the development and implementation of the WQMPAS. The development and implementation of the WQMPAS is an iterative process that has several sequential phases. The first phase is now complete and has resulted in a framework that specifies in general what the Performance Assessment System (PAS) must encompass and accomplish. The framework has benefited from the input and insights of Department Water Affairs and Forestry (DWAF) managers and other stakeholders but it is as yet too generic for implementation. As it stands it represents the “bones” around which a user-friendly PAS may be fleshed out.

The objectives for the first phase were: To perform a business analysis of the Water Quality Management (WQM) function of

DWAF, i.e. the determination of what is involved in WQM function of DWAF; to determine performance criteria; to document key performance areas and links to possible performance indicators; to distribute and communicate the above to intended users and stakeholders to

obtain inputs and insights; and the design of a framework for a preliminary WQMPAS.

The above objectives were achieved by a review of pertinent documents, in-depth substantive interviews with intended users from Head Office and three Regional Offices, representatives of the Institute for Water Quality Studies (IWQS) and DWAF senior management. With the support of the Project Committee, the consultants developed a logic framework depicting key performance areas for the water quality management function and provisional schedules for linking performance areas with possible performance indicators. An indication of possible monitoring and audit methods was also sketched. This information was circulated to intended users for comment. A workshop was held in March 2000 where the conceptual framework of the WQMPAS was explained and discussed. Further input was gathered from workshop attendees who included representatives from the Regional Offices, senior management and outside stakeholders.

In short, the WQMPAS must enable the DWAF to demonstrate clearly the extent to which it accomplishes and ensures “the integrated sustainable management of the water quality of all the water resources in the Republic of South Africa”.

Water quality management is a National competency. Responsibility for this competency devolves on the Minister of Water Affairs and Forestry and the Department of Water Affairs and Forestry’s WQM function, which consists of the:

Final Report - iv - June 2000


Department’s Directorate Water Quality Management, which provides policy development, capacity building, specialist support, authorisation and audit services at a strategic level

Department’s nine Regional Offices, which provide policy implementation, operation, control and monitoring services at an operational level; and

Department’s Institute for Water Quality Studies, which provides a scientific support service.

The capacity of the DWAF’s Water Quality Management function will likely be extended with the establishment of Water Management Institutions, specifically Catchment Management Agencies.

Given the differing levels of responsibility outlined above, it is clear that the WQMPAS must be able to deliver performance information at specific levels to different users. It is also important to recognise that the PAS must deliver timeous information to be utilised as a diagnostic tool. It must provide management the opportunity for intervention to ensure a profile of continuous performance improvement. For this to occur, performance targets and benchmarks have to be set for purposes of comparison. The WQMPAS must be compatible with and draw information from the DWAF’s existing information management systems, the Water Management System (WMS) and the Water Authorisation, Registration and Monitoring System (WARMS).

A proposal for the continuance of phases II and III of the WQMPAS project was developed. The second and third phases must be seen as providing a bridge between the development of a WQMPAS and its full implementation. Phase II involves the refinement of the framework developed in phase I. Phase III is the implementation plan and testing of performance information held by DWAF. The anticipated time for completion of the assessment program and implementation plan is December 2000, after which implementation can start.

Final Report - v - June 2000


FRAMEWORK FOR AWATER QUALITY MANAGEMENT PERFORMANCE ASSESSMENT SYSTEM

(WQMPAS)

TABLE OF CONTENTS PAGE

Document Index ................................................................................................ iApproval ............................................................................................... iiAcknowledgements.................................................................................iiiExecutive Summary............................................................................................ iv

1 INTRODUCTION AND CONTEXT..............................................................11.1 Trusteeship of nation’s water resources.............................................11.2 Water Quality Management Function of the DWAF............................11.3 Need for project..................................................................................31.4 Purpose of this document...................................................................3

2 PROJECT SCOPE......................................................................................32.1 Aims of project…………………………………………………………….32.2 Approach/Phasing of project…………………………………………….42.3 Terms of Reference for Phase I………………………………………...42.4 Additions and deviations from Terms of Reference Phase I…………52.5 Linkages with other initiatives………………………………………..… 52.6 Users of a WQMPAS………………………………………………… 6

3 WHAT IS A PERFORMANCE ASSESSMENT SYSTEM……....................73.1 Auditing vs Performance Assessment................................................73.2 Principles and Definitions of a PAS....................................................73.3 Process…..........................................................................................8 3.4 Benefits of WQMPAS.......................................................................10

4 ASSESSMENT OF THE DWAF’s WQM FUNCTION...............................114.1 Policy and legislative requirements/mandates..................................114.2 DWAF’s WQM function: Who are the role players?………...............154.3 Preliminary performance areas identified…......................................174.4 Areas of concern..............................................................................19

5 LOGIC FRAMEWORK..............................................................................195.1 Context…..........................................................................................195.2 The logic of the WQM business........................................................215.3 Performance Assessment of Regional Offices.................................235.4 Issues to be considered in assessing accountability and

Responsibility…................................................................................235.5 Framework for monitoring and evaluation........................................24

6 CONCLUSION…......................................................................................246.1 Limitations........................................................................................24

Final Report - vi - June 2000


7 WAY FORWARD......................................................................................24

APPENDIXES

Appendix A: Summary of results of document analysis…............................26Appendix B: Framework for Monitoring and Evaluation................................48Appendix C: Workshop Inputs......................................................................57

Final Report - vii - June 2000


ALPHABETICAL LIST OF ACRONYMS AND ABBREVIATIONS

C CEC Committee for Environmental Co-ordination

CMA’s Catchment Management Agencies

CMS Catchment Mangement Strategy

CORE Codes of Remuneration System

D DWAF Department of Water Affairs and Forestry

I IWQS Institute for Water Quality Studies

K KPA Key Performance Area

KPI Key Performance Indicator

N NEAF National Environmental Advisory Forum

NEMA National Environmental Management Act

NWA National Water Act, 1998

NWRS National Water Resource Strategy

P PAS Performance Assessment System

R RQO Resource Quality Objective

W WARMS Water Authorisation, Registration and Monitoring System

WMA Water Management Area

WMS Water Management Strategy

WQM Water Quality Management

WQMPAS Water Quality Management Performance Assessment System

WUC Water Use and Conservation

Final Report - viii - June 2000


FRAMEWORK FOR A WATER QUALITY MANAGEMENT PERFORMANCE ASSESSMENT SYSTEM (WQMPAS)

1. INTRODUCTION AND CONTEXT

1.1 Trusteeship of nation’s water resources

As the public trustee of the nation’s water resources the National Government, acting through the Minister of Water Affairs and Forestry must ensure that water is protected, used, developed, conserved, managed and controlled in a sustainable and equitable manner for the benefit of all persons. The Department of Water Affairs and Forestry (DWAF) is ultimately responsible for ensuring that water is allocated equitably and used beneficially in the public interest while promoting environmental values. The DWAF has the power and the duty to regulate the use, flow and control of all water in the Republic.

1.2 Water Quality Management Function of the DWAF

1.2.1 DWAF

The Constitution and the National Water Act, 1998 (Act No. 36 of 1998) assign the legal mandate for water resources management to the Minister of Water Affairs and Forestry (and the DWAF).

The DWAF mandate is embodied in its mission statement “to ensure both the integrated and sustainable development and management of water and forestry resources and the provision of basic community water supply, sanitation and forestry needs. Thus providing a fair service to all users and supporting reconstruction and development towards economic growth, prosperity and harmony of our nation.” The DWAF is structured in Branches, Chief Directorates, Directorates and Sub-Directorates.

The Chief Directorate Water Use and Conservation (WUC) has the mission “ to promote the equitable allocation, beneficial use and sustainability in terms of both quality and quantity, of water resources on an integrated basis through policy development, regulation, facilitation and monitoring.”

The Directorate Water Quality Management (WQM) resorts under WUC and has the mission “to ensure the integrated sustainable management of the water quality of the water resources of South Africa.”

The Institute for Water Quality Studies (IWQS) resorts under the Chief Directorate Scientific Services and supports the water quality management function by providing technical support and information.

1.2.2 Transition

The Directorate WQM has for the past two years been operating in a mode of transition. The Public Service is determined to accelerate recruitment and personnel practices that will result in a work force that mirrors the country’s demographics. At the same time new legislation has propelled water quality management into different directions. Both aspects of transition have a profound impact on how DWAF executes the WQM functions through Regional Offices, the IWQS and the Directorate WQM and the extent to which it is able to accomplish its mission.



1.2.3 Context of the development of a performance assessment system (PAS)

The DWAF Directorate WQM has recognised the need to implement a Water Quality Management Performance Assessment System (WQMPAS). The WQMPAS required should focus specifically on performance and the establishment of a performance reporting system. The performance information needs to be assessed to provide management with the assurance that performance information is valid, accurate and complete.

The Directorate WQM has been pro-active in the initiation of this process. The Interim Constitution and the New Constitution (Act No 108 of 1996, inclusive of the Bill of Rights, Chapter 2 of the Constitution) made it explicitly clear that accountability and representivity form the cornerstones of the new constitutional state. Service delivery is strongly implied and it is listed in detail in sections on Co-operative Government (Chapter 3) and in Schedule 4 of the Constitution. The new Public Finance Management Act, 1999 (Act No. 1 of 1999) implemented in April 2000, requires that government departments report on performance. Another relevant piece of legislation is the White Paper on Transforming Public Service Delivery Batho Pele (People First) launched by the Department of Public Service and Administration during 1997. This White Paper places a considerable emphasis on the delivery of efficient and effective services to communities. Apart from the constitutional obligation to co-operation, good faith and co-ordinated efforts the implementation of the Batho Pele policy aims at effective, people-centred service delivery. Over and above the need to comply with the above-mentioned legislation, performance reporting can be a valuable management tool. By implementing a performance reporting system the Department’s WQM function can respond to their accountability responsibilities and would be able to answer questions, for example:

Is the water quality of rivers in South Africa deteriorating or improving?

Is there a continuous improvement of water quality?

Does the Directorate WQM have the necessary policies and procedures in place to ensure that objectives are achieved?

Is there adequate training of personnel and stakeholders?

Does the Directorate WQM provide the necessary specialist technical support?

Are all water uses impacting on water quality authorised?

Are any services impacting on water quality management duplicated?

Up to now the assessment and auditing functions had been neglected to a certain extent. The implementation of a WQMPAS is one of the tools planned to address this shortcoming. Once a proper system of reporting on performance is established, whereby management is assured of the accuracy and completeness of the information, additional responsibilities and functions can be delegated to the Regional Offices. Consultants were appointed to conduct the reconnaissance phase. The intent is to appoint consultants to conduct the development phase and the compilation of the implementation plan. If the Directorate has adequate resources of appropriate skills the



implementation can be done internally. Alternatively an independent consultant will have to be appointed to conduct the execution of the WQMPAS.

1.3 Need for project

The Directorate WQM initiated this project in response to the recognition that performance of the water quality management function in the DWAF must keep pace with the transition within the Department necessary for the full implementation of the National Water Act, 1998 (Act No. 36 of 1998) (NWA). Further, it was recognised that in the interest of good governance the need for performance based management is a critical factor. The Directorate WQM has a need to manage the WQM function. The function of Water Quality Management is shared by the Regional Offices and Head Office and the IWQS. Furthermore, the provisions of the NWA imply that more resources must be expended in resource directed measures. In the past WQM had concentrated more on source directed measures. This means that WQM must ensure that proper performance information must be held and regularly reported and analysed to ensure that the function can be efficiently and effectively managed.

1.4 Purpose of this document

This document reports the results of the preliminary investigation and development of an initial framework for the development and implementation of a Water Quality Management Performance Assessment System (WQMPAS).

The development and implementation of the WQMPAS is an iterative process that has several sequential phases. The first phase is now complete and has resulted in a framework that specifies in general what the WQMPAS must encompass and accomplish.

This report sets out a brief analysis of the business of the WQM function and explains how information was gathered and applied in order to result in a logic framework depicting the WQM function.

This document also explains how this logic framework formed the basis of a workshop with key role players. The results of the workshop proceedings are included as an appendix to this report.

From the logic framework a preliminary framework for monitoring and evaluation was constructed. It is also appended to this document. The information contained in this report form the background for the continuance of phases II and III of the WQMPAS project.

2. PROJECT SCOPE

2.1 Aims of project

The objective of this project is to assist the Directorate WQM in establishing a WQMPAS to assist the Directorate in fulfilling its responsibility of “ensuring the sustainable integrated protection and management of the water quality of the water resources of South Africa”.



The above objective will be achieved by:

Formulating feasible and practical criteria against which the performance of the Directorate WQM can be measured, and

Developing a WQMPAS.

Although the project was initiated by the Directorate WQM it must be noted that the function for water quality management is diffuse and shared by Head Office, Regional Offices, IWQS and Catchment Management Agencies (CMA‘s) when operable. Thus the WQMPAS must serve the performance information needs of all the parties who share the responsibility for delivering the function.

2.2 Approach/Phasing of project

The development and implementation of a WQMPAS was rationalised into three distinct phases each with specific objectives. They are:

Phase I – Reconnaissance phase

Assessment of the WQM function. Determination and formulation of performance criteria. Development of the Draft WQMPAS. Compilation of a Terms of Reference for Phases II and III of the project.

Phase II – Development phase

Refinement of the framework of WQMPAS developed during phase I. Development of a detailed assessment program. Development of a model of performance information to be reported on. Exploration of information systems and management information. Workshop to familiarise users with WQMPAS.

Phase III – Implementation phase

Run tests of available performance information. Adjust WQMPAS with reference to test results. Perform pilot feasibility tests on assessment steps. Refine WQMPAS with reference to pilot results. Compile an implementation plan. Build capacity within the Directorate WQM.

2.3 Terms of Reference for Phase I

This document represents the completion of phase I as specified in the following Terms of Reference.The purposes of phase I as stated in the Terms of Reference are: Assessment of the WQM function: This step includes an investigation of the WQM’s function’s

“business”, “assessment tools” that are currently available, as well as the identification of possible shortcomings and cost estimates to address the identified shortcomings.

Determination and formulation of performance criteria: This step entails the identification and the formulation of feasible and practical performance criteria that



can be used to measure the effectiveness with which the Department, through the Directorate WQM and its Regional Offices, conducts its WQM “business”.

Compilation of Terms of Reference for phases II and III of the project.

The Terms of Reference further provided for the establishment of a Steering Committee, represented by the Regional Offices and the IWQS, to provide technical input and strategic direction. The Terms of Reference also made provision for the establishment of a project management committee to co-ordinate and manage the project.

The Terms of Reference also stipulated that phase I will incorporate the identification of specific links with other management systems including the Water Management System (WMS), the Water Authorisation, Registration and Monitoring System (WARMS) and the Codes of Remuneration System (CORE).

The final deliverable, the Draft WQMPAS, will consist of an evaluation of the existing performance information held by WQM and include an assessment of the extent to which the information held by WQM allows the reporting of performance of the line function of the Directorate WQM. It will identify any information or systemic gaps impeding the reporting of performance and will identify and assess (including costing of) different options to inform WQM management.

2.4 Additions and deviations from Terms of Reference Phase I

During the conduct of phase I of the project several changes to the approved Terms of Reference were made. The most important deviations are listed below: The Steering Committee represented by the Regional Offices and the IWQS never

came off the ground. The WQM project managers deemed this would be more effective during the subsequent phases of the project.

Upon examination the existing performance information proved to be insufficient to form the basis for a WQMPAS. The project management committee approved a decision to work from the assumption that relevant information held in WMS and WARMS would be accessed by the proposed WQMPAS, but that the consultants should identify key performance areas (KPA’s) and associated key performance indicators (KPI’s) to form the basis of the WQMPAS.

A problem was experienced in the garnering of input from the Regional Offices and other role players based on e-mail documentation. As a consequence the project management committee approved the holding of a workshop to achieve this input. This resulted in a prolongation of the estimated completion time. It also resulted in the expenditure of additional resources.

The costing of options was not possible due to the fact that some of the electronic information systems held by the DWAF were not fully implemented, or experiencing problems, at the time of investigation.

The tasks remaining will be addressed during phase II.

2.5 Linkages with other initiativesIt is important that the WQMPAS must link with existing information and management systems held by the DWAF. Where possible it must feed information from and to the Water Management System (WMS), the Water Authorisation, Registration and Monitoring System (WARMS) and the Codes of Remuneration System (CORE).



The WMS is an electronic database holding water management data. The WQMPAS will be designed to be compatible to, and an adjunct of the WMS.

The WARMS holds data regarding authorisations and registration of water use and licences and the subsequent monitoring of activities under licence or authorisation. It is envisaged that these data (with possible additions or modifications that enhance the DWAF’s ability to report on the performance of the WQM function) would form an essential part of the WQMPAS.

The DWAF has recently adopted the performance-based CORE system to manage and enhance the delivery of clearly specified performance in critical areas by its personnel. The WQMPAS can be a tool to monitor and manage the achievement of CORE performance goals.

2.6 Users of a WQMPAS

The WQMPAS must serve the needs of different categories of users. Water Quality Management, a National competency, is the responsibility of the Minister of Water Affairs and Forestry and the DWAF’s Water Quality Management function is carried out by: Department of Water Affairs and Forestry’s Directorate WQM, which provides policy

development, capacity building, specialist support, authorisation and audit services at a strategic level.

Department of Water Affairs and Forestry’s nine Regional Offices, which provide policy implementation, operation, control and monitoring services at an operational level; and

The Institute for Water Quality Studies that provides a scientific support service.

Each of the above-listed categories has different performance information needs that must all be satisfied by the WQMPAS.

The Directorate WQM will use the WQMPAS to monitor and report on its own efficiency and effectiveness, the implementation of policy, the performance of the Regional Offices WQM components and eventually those of the CMA’s.

The Regional Offices will use the WQMPAS to monitor and report on their own performance efficiency and effectiveness and to monitor the performance of Water Management Institutions, particularly CMA’s.

The IWQS will use the WQMPAS to monitor and report on its efficiency and effectiveness with respect to its mandate in the Water Quality Management function.

Eventually Water Management Institutions and CMA’s will use the WQMPAS to monitor and report on their performance with respect to efficiency and effectiveness.

Senior management will use the WQMPAS to ensure that adequate resources are expended to perform the WQM mission with maximum relevance, efficiency, effectiveness and economy. The WQMPAS is a tool for management to receive timeous performance information in order to make interventions and decisions that will ensure that the mission is carried out.Over and above the Directorate WQM, other Directorates within DWAF can also use the framework as a basis to develop their own PAS.



Other national departments can also use the logic and process followed as a basis for development of their own PAS.

Research Institutions such as universities could use information from the WQMPAS for research purposes.

Clients and the general public can also use the information from the WQMPAS to evaluate the extent to which objectives have been achieved and service delivery standards have been accomplished.

Information from the WQMPAS can be used for making the general public aware of water quality management issues and may possibly be used as a tool to encourage citizens of South Africa to co-operate in the monitoring and protection of the country’s water resources.

3. WHAT IS A PERFORMANCE ASSESSMENT SYSTEM?

3.1 Auditing vs Performance Assessment

Performance assessment is different from an audit. During an audit the main objective is to gather audit evidence to be able to express an opinion on whether the financial information is a fair reflection of the results of the entity. To be able to do this, auditors normally test the control systems or do detail testing on financial figures in the financial statements. An audit provides assurance that one can rely on the financial figures but does not focus on effective, efficient and economic acquisition and utilisation of resources.

In terms of a performance assessment system, management will make representations regarding specific defined performance areas in order to demonstrate the extent to which performance goals are achieved. Management reports on performance and the reported information is analysed and reviewed for accuracy, completeness and validity.

Actual results are compared to targets or benchmarks and variances explained. The performance information held is also audited for completeness, accuracy and validity.

This process is normally referred to as evaluation and monitoring, or performance assessment. Performance assessment is the preferred terminology.

The WQMPAS can be compared to a thermometer. The information generated by the WQMPAS should give a warning of potential problems that militate against good performance. There could be good explanations for it but based on the warning the necessary precautions can be taken to ensure the situation does not deteriorate.

3.2 Principles and Definitions of a PAS

From the above discussion it is clear that a performance assessment system (PAS) has the following goals: It allows the systematic tracking (monitoring) of specific performance areas in a

consistent longitudinal manner, in other words it demonstrates trends.



It must be flexible in recognition that an organisation’s performance goals tend to change with new strategic directions.

It is a tool giving management timeous access to performance information so that interventions ensuring continuous improvement can be readily made.

Appropriate targets or benchmarks against which trends can be measured are critical.

During the development of a WQMPAS the first objective or deliverable will be the identification of key performance areas (KPA’s). KPA’s can be defined as the defined, prioritised, critical performance areas that must be carried out if a mandated mission is to be achieved.

While designing a PAS, the temptation is often to include too much information for measuring and monitoring. The axiom Less is More is valuable. Experience has shown that any organisation (or part of an organisation) should be able to confine its KPA’s to less than eight (as a rule of thumb).

Once KPA’s have been identified and prioritised the next step in the development of a PAS is the identification of appropriate associated key performance indicators (KPI’s).

KPI’s can be defined as things that can be observed or counted and that could provide a diagnostic sense of the extent that a key performance area is being achieved efficiently and effectively.

It is important to keep in mind that only essential performance information should be gathered in a PAS. A phenomenon often observed in organisations, the tendency to measure for the sake of measuring and to lose track of why or to what purpose the measurement occurs must be guarded against.

Satisfactory KPI’s are: Few (choose one indicator as diagnostic of a specific objective). Occur over time (choose an indicator that can be monitored over time – avoid once-

off events). Must be quantifiable. Must be able to measure the indicator against a target or benchmark.

Once KPI’s have been identified they must be challenged and tested before a PAS can be made operational.

3.3 Process

The creation and implementation of a PAS can be divided in three phases namely the planning phase, the conducting of the assessment phase and the reporting phase. The planning phase culminates in delivery of a framework that links KPA’s and KPI’s.

The following diagram (Figure 1) depicts the PAS development and implementation process.Figure 1: Development and Implementation of PAS Process


PLANNING PHASE


The main objectives of the planning phase are to:

Develop an understanding of the assessment entity and its environment; Identify the assessment issues and the criteria by which they will be assessed; and Prepare the assessment program.

Assessment criteria include policies, practices, procedures or requirements against which the actual situation is compared. Based on the assessment criteria a detailed assessment program is developed. An assessment program is a list of procedures to be performed to compare existing systems and/or management practices with assessment criteria and to collect evidence to support assessment observations.

During the conduct phase the objectives are to:


ASSESSMENT PHASE

REPORTING PHASE


Gather and evaluate information to compare actual practices or operations against criteria; and

To obtain sufficient, appropriate evidence to support any opinions and recommendations that will eventually be reported.

Where significant deviations from criteria are identified, the underlying cause and effect will be determined.

Assessment evidence should be of such a quality and quantity that competent evaluators working independently from each other will reach similar findings from evaluating the same assessment evidence.

When the cause and effect of assessment findings have been satisfactorily identified, evaluators come to an opinion, either positive or negative. It represents the evaluation of the assessment finding based on the available evidence.

During the reporting phase the assessment findings are described in a point-form report. The point-form report presents findings in relation to assessment objectives and criteria, and for each finding includes a description of assessment evidence, causes and impact, opinions and recommendations. This report should be cross-referenced to the assessment working papers and supporting evidence. It is important that the point-form report be thoroughly challenged internally to ensure that findings are sound.

After the internal challenge and update of the report the evaluators can develop conclusions. The report should be approved by management. The assessment files are completed and reviewed and the final report issued.

The above described process if followed will result in a fully operational PAS.

3.4 Benefits of WQMPAS

After the WQMPAS is implemented it will allow the DWAF to determine the extent to which the WQM function performs its mission. More specifically this management tool has the following benefits: Management can respond to their accountability obligations. Management can respond to the expectations of governing bodies. Governing bodies

need information to be able to fulfill their roles and responsibilities and a performance information system can provide the necessary information.

Management can respond to emerging expectations from different stakeholders to provide a broader range of information.

Such a system provides an opportunity for management to explain effectiveness in a context that is appropriate, thereby preventing wasted effort by ensuring information focuses on important matters presented at meaningful levels.

Contribute to appropriate decision-making. Assist management with control processes. Ensure compliance with the Public Finance Management Act, 1999 (Act No. 1 of 1999).4 ASSESSMENT OF THE DWAF’S WQM FUNCTION

4.1 Policy and legislative requirements/mandates



The DWAF’s WQM function is carried out within the parameters of several legislative imperatives. The key legislation relevant to this function is briefly described in the following sections of this report.

4.1.1 Constitution 108:1996

Since 1994 all aspects of South African governance have undergone significant change. The Constitution of the Republic of South Africa, 1996, Act 108 of 1996, in Chapter 2: Bill of Rights section 24 describes every citizen’s rights with respect to the environment as follows:(a) to an environment that is not harmful to their health or well-being; and(b) to have the environment protected, for the benefit of present and future

generations, through reasonable legislative and other measures that –(i) prevent pollution and ecological degradation;(ii) promote conservation; and(iii) secure ecologically sustainable development and use of natural resources while

promoting justifiable economic and social development.

With respect to the right to have access to water section 27 stipulates that every citizen has the right to sufficient water and that the state must take reasonable legislative and other measures within its available resources to achieve the progressive realisation of this right.

4.1.2 National Environmental Management Act (NEMA) 107:1998

The NEMA provides umbrella legislation making provision for co-operative environmental governance by establishing principles for decision-making on matters affecting the environment, institutions that will promote co-operative governance and procedures for co-ordinating environmental functions exercised by organs of state; and to provide for matters connected therewith.

The DWAF and particularly the WQM function play an important role within the NEMA ambit. Not only is the DWAF mandated to exercise functions that may affect the environment, it is also mandated to exercise functions that involve the management of

the environment, the WQM function being pertinent in this report.

The NEMA sets out national environmental management principles. Chapter 2 has particular relevance because it sets out the establishment, objectives and functions of the National Environmental Advisory Forum, (NEAF) and the Committee for Environmental Co-ordination, (CEC). The DWAF has key roles and responsibilities in these forums. In terms of the WQM function the objectives under NEMA of the NEAF and the CEC are to ensure that co-operative governance enhances the likelihood that national performance goals can be set and achieved.

4.1.3 National Water Act 36:1998

The legislative measure to ensure the protection of the constitutional rights with respect to water and the aquatic environment is the National Water Act, 1998 (Act No. 36 of



1998). This Act is guided by principles of sustainability and equity in the protection, use, development, conservation, management and control of water resources. As the public trustee of the nation’s water resources the National Government, acting through the Minister of Water Affairs and Forestry must ensure that water is protected, used, developed, conserved, managed and controlled in a sustainable and equitable manner for the benefit of all persons. The DWAF is ultimately responsible for ensuring that water is allocated equitably and used beneficially in the public interest while promoting environmental values. The DWAF has the power and the duty to regulate the use, flow and control of all water in the Republic.

The introduction of the NWA expanded the functions and responsibilities of the water quality management function considerably. The emphasis has now shifted to a more balanced management approach recognising the need for source, remediation and resource directed measures. The NWA introduced several new concepts. Some of the important concepts are:

Water resource is deemed to be a watercourse, surface water, estuary, or aquifer. The concept of a reserve was introduced. Reserve refers to the quantity and quality

of water required;(a) To satisfy basic human needs by securing a basic water supply, as prescribed

under the Water Services Act, 1997 (Act No. 108 of 1997), for people who are now or who will in the reasonably near future be relying upon, taking water from, or being supplied from, the relevant water resource; and

(b) To protect aquatic ecosystems in order to secure ecologically sustainable development and the use of the relevant water resource.

The management of water should take place in a Water Management Area (WMA),

that is the basic management unit in the National Water Resource Strategy (NWRS) within which a Catchment Management Agency (CMA) will conduct the protection, use, development, conservation, management and control of water resources.

The NWA contains stringent pollution prevention measures. Part 4: Pollution prevention Section 19 deals with pollution prevention and in particular the situation where pollution of a water resource occurs or might occur as a result of activities on land. The person who owns, controls, occupies or uses the land in question is responsible for taking measures to prevent pollution of water resources. If these measures are not taken, the CMA concerned (if a CMA does not yet exist, the DWAF) may itself do whatever is necessary to prevent the pollution or to remedy its effects, and to recover all reasonable costs from the persons responsible for the pollution.

Use of water is no longer limited to the abstraction of water, but includes activities having relation to water such as storing of water, disposing waste, discharging water containing waste, altering river banks and beds, recreation and so forth (NWA Section 21). All water uses apart from those mentioned under Schedule I of the NWA (page 152) will ultimately require a form of authorisation or license under the NWA.

Provision is made for public consultation processes in the establishment of strategies.The NWA recognises the unity of the hydrological cycle and the interdependence of its elements, where evaporation and rainfall are linked to groundwater, rivers, lakes, wetlands and the sea. All water is now deemed to be public water and groundwater is no longer viewed as private property.



The complete interconnectedness of water systems demand that the water resources be protected, used, managed, developed and controlled in an integrated and holistic manner through a catchment directed approach.

The final implementation date of the Act was 1 October 1999.

4.1.4 National Water Resource Strategy

The National Water Resource Strategy (NWRS) provides the framework for the protection, use, development, conservation, management and control of water resources for the country. It also provides the framework within which water will be managed at regional or catchment level, in defined WMA’s. The contents of the NWRS are described in Section 6(1) of the Act.

4.1.5 Catchment Management Agencies

The purpose of establishing CMA’s is to delegate water resource management functions to the regional or catchment level and to involve local communities, within the framework of the NWRS. Whilst the ultimate aim is to establish CMA’s for all WMA‘s, the DWAF Regional Offices act as the CMA where one has not been established. The establishment and powers of CMA’s are fully described in Chapter 7 of the NWA.

The Catchment Management Strategy (CMS) provides a framework within which water resources management will be carried out within a WMA.Currently CMA’s are not yet in operation. The Regional Offices continue to assume the functions that may later be carried out by CMA’s.

4.1.6 Water quality objectives

Water quality objectives are described as an important part of Resource Quality Objectives (RQO) in the Act. The purpose of the RQO is to establish clear goals relating to the quality of the relevant water resources. In determining RQO’s a sustainable balance must be sought between the need to protect water resources on the one hand and the need to develop and use them on the other. Provision is made for preliminary determinations of the Class and RQO’s of water resources before the formal classification system is established. Once the Class of a water resource and the RQO’s have been determined they are binding on all authorities and institutions exercising any power or performing any duty under the NWA.

The objectives determined in terms of subsection (1) of Section 13 of Chapter 3 of the NWA may relate to the following: The Reserve; The instream flow; The water level; The presence and concentration of particular substances in the water; The characteristics and quality of the water resource and instream and riparian

habitat; The characteristics and distribution of aquatic biota; The regulation and prohibition of instream or land-based activities which may affect

the quantity of water in or quality of the water resource; and Any other characteristics.



4.1.7 Use of water

Chapter 4 of the Act is significant because it lays the basis for regulating water use. Requirements for regulating water use are specified and include: Taking water from a water resource; Storing water; Impeding or diverting the flow of water in a watercourse; Engaging in a streamflow reduction activity contemplated in section 36; Engaging in a controlled activity identified as such in section 37(1) or declared under

section 38(1); Discharging waste or water containing waste into a water resource through a pipe,

canal, sewer, sea outfall or other conduit; Disposing of water in a manner which may detrimentally impact on a water resource; Disposing in any manner of water which contains waste from, or which has been

heated in, any industrial or power generation process; Altering the bed, banks, course or characteristics of a watercourse; Removing, discharging or disposing of water found underground if it is necessary for

the efficient continuation of an activity or for the safety of people; and Using water for recreational purposes.

4.1.8 Monitoring and assessing

Section 22 of the NWA describes permissible water uses. The Act further refers to monitoring, assessing and information in Chapter 14 where the Minister may require any person to provide information on various aspects of water resources including the quality and quantity of all water resources.

4.1.9 Directorate WQM response to new legislative requirements

Traditionally the Directorate Water Pollution Control and later the Directorate WQM had been responsible for the water quality component of water resources management. The approach adopted was focused on a point-source management approach and emphasis was placed on pollution control. Since 1991 the Directorate has acknowledged the receiving water environment within a catchment context. The previous emphasis on source management had increasingly become challenged within the Department. The Department contributed to the drafting of the new legislation. The DWAF has thus informally been preparing for some time to take on broader responsibilities.

The current structure of the Directorate WQM provides for four sectoral Sub-Directorates and one integrating Sub-Directorate namely: Sub-Directorate Industries; Sub-Directorate Mines; Sub-Directorate Urban Development and Agriculture; Sub-Directorate Waste Management; and Sub-Directorate Management Systems (integrator).

The current structure, while adequate for a point-source directed management focus may not be the most effective delivery system for a resource directed approach to WQM.



The Directorate WQM developed a number of tools to support the WQM function. These tools included the setting of receiving water quality objectives, conducting investigations to assess the impacts of effluent discharges and granting permits for the discharge of effluent. Water quality objectives are specific numeric values set for water quality constituents of concern. In the source directed management process, water quality objectives are used to set appropriate standards for an effluent discharge. The aim is to develop RWQO’s or resource water quality objectives for all catchments. These objectives are set through catchment assessment studies. In the past, the manner in which water quality had to be managed to achieve the set objectives was called the WQM plan. The NWA, however uses the term Catchment Management Strategy (CMS). We may therefore refer to the water quality component of the CMS or to the WQCMS. Although a few catchment studies have been completed much work still needs to be done. Key to our task of preparing a WQMPAS, is the enabling of relevant and effective monitoring and assessment criteria. These will ensure that WQM contribute to the ultimate aims to achieve the sustainable equitable use of water for the benefit of all users; protecting of the quality of water resources, to ensure sustainability and for the integrated management of all aspects of water resources. Where appropriate, the delegation of management functions to a regional or catchment level may occur so as to enable everyone to participate. 4.2 DWAF’s WQM function: Who are the role players?

Within the DWAF the WQM function is shared by a number of role players. The level of performance within the function fluctuates between different role players but each make unique contributions towards achieving WQM objectives. The following section depicts the formal structure (Figure 2) within the DWAF where the WQM function is accommodated. It is important to note that other role players outside of the DWAF also make important contributions to the achievement of water quality management goals. In terms of co-operative governance the NEAF and the CEC (described in section 4.1.2 of

this report) play key roles in ensuring that national water quality performance goals are achieved.

4.2.1 Organogram

The main role players in performance of the WQM function within the DWAF are structured as the following organogram summarises:



4.2.2 Head Office Directorate WQM

The mission of WQM is “to ensure the integrated sustainable management of the water quality of the water resources of South Africa”. The Directorate WQM contributes to the performance of the mission by providing policy development, capacity building, specialist support, authorisation and audit services at a strategic level.



4.2.3 Regional Offices

The DWAF’s nine Regional Offices provide policy implementation, operation, control and monitoring services at an operational level in support of the WQM mission. A regional interpretation of its mission regarding water quality management can be stated as “to ensure that the water resources of the region are fit for use for humans and recognised users and ensure that the health of the water environment is protected and improved on a sustainable basis.”

4.2.4 Institute for Water Quality Studies

The IWQS resorts under the Chief Directorate: Scientific Services with the mission:“To ensure water resource monitoring and development of water resource assessment methodologies, based on multi-disciplinary scientific and technical principles; and to provide an extensive range of water resource quality, quantity and related information in support of the Department’s mandates for water service provision, water resource development and water resource protection”.

The IWQS is responsible for providing water resource quality information collected through a number of national and regional monitoring networks. In support of water resource quality management, the Institute provides technical information, guidelines and procedures for water resource assessment.

4.2.5 Catchment Management Agencies

The purpose of establishing CMA’s is to delegate water resource management functions to the regional or catchment level and to involve local communities, within the framework of the NWRS. Whilst the ultimate aim is to establish CMA’s for all WMA‘s, the DWAF Regional Offices act as the CMA where one has not been established. The establishment and powers of CMA’s are fully described in Chapter 7 of the NWA.

The Catchment Management Strategy (CMS) provides a framework within which water resources management will be carried out within a WMA.

Currently CMA’s are not yet in operation. The Regional Offices continue to assume the functions that may later be carried out by CMA’s.

4.3 Preliminary performance areas identified

In this section the methods used to gather, analyse and present information needed to form the basis of a WQMPAS are briefly discussed. Based on the information gained through interviews with staff and an extensive document analysis preliminary performance areas were identified. A first attempt at selecting key performance areas was tested in a workshop. It is important to note that the key performance areas need

refinement and prioritisation in further iterations before being incorporated as the basis for a final WQMPAS. Obtaining the relevant information to enable the development of the WQMPAS is an iterative process. First, an in-depth understanding of the scope and range of responsibilities and activity areas must be obtained by scrutiny of records and documents. A total of thirty documents were reviewed and key responsibilities and



activities were identified. The document review focussed exclusively on performance areas. Care was taken to be as inclusive as possible. Appendix A provides a summary of the results of the document analysis. On the basis of the knowledge obtained from a thorough document analysis, structured in-depth interviews were held with eighteen designated DWAF personnel.

The preliminary investigation concentrated on obtaining relevant information from Head Office personnel. Three Regional Offices were selected for inclusion, namely Gauteng, KwaZulu-Natal and Western Cape. The information obtained during the interviews was then collated and combined with information and insights gained from the document review.

The above-mentioned information allowed the consultants to formulate a meta level understanding of the responsibilities and activities of the WQM function. From this understanding it was possible to create a logic framework to guide the design of a framework for monitoring and evaluation.

The framework based on the logic model sets out (in a preliminary manner) the key performance areas, associated indicators and monitoring and control steps (assessment). Refer to Appendix B.

Performance areas identified in the documents and interviews include:HEAD OFFICE REGIONAL OFFICES

Planning and Development Policies Strategies Procedures Guidelines Methodologies Regulations Criteria and objectives

Planning and Development Involved in inputs as required

Capacity Building Training Liaison with stakeholders Liaison with other line departments Water Quality Staff satisfaction

Capacity Building Training Involvement of public in participatory processes Liaison with stakeholders Water Quality Staff satisfaction

Operate /technical specialist input Provide support Development of information systems. Involved in assessment of water resource quality

including flow pattern, timing, water quality, habitat and aquatic biota.

Operate /technical specialist input Operational systems Reduce and prevent degradation of water

resources. Meet water quality user requirements. Involved in assessment of water resource quality

including flow pattern, timing, water quality, habitat and aquatic biota.

Abandoned mines (remediation).Control and authorisations Consider licenses Consider authorisations

Control and authorisations Licenses Authorisations All water quality related use must be licensed.

Assessment Assessment system Establishment of a national water monitoring

system for quality, use of water resources, rehabilitation of water resources and compliance with resource quality objectives.

Improved efficiency and effectiveness of water quality management.

Assessment Monitoring Provide input in establishment of a national water

monitoring system for quality, use of water resources, rehabilitation of water resources, and compliance with resource quality objectives. Improved efficiency and effectiveness of water quality management.



Note: Sentences in italic style refer to areas identified during document analysis. Sentences in normal font represent performance areas from interviews.

4.4 Areas of concern

During the interviews interviewees mentioned some areas of concern. The concerns were prompted by questions regarding the extent to which the WQM function will be able to achieve its objectives. The interviewees regard the areas listed as impediments to the effective and efficient carrying out of mandated tasks. Areas of concern include: Lack of policies on the new Act. Attrition of skilled personnel. Lack of detailed information on catchments. Skilled water quality management personnel are in short supply. Different regional offices have different practices. Regional offices at divergent levels in terms of capacity and development. Success in enforcing permits, currently no compliance assessments done. Lack of resources for proper implementation of NWA. Management in terms of RQO’s not yet in place. Inadequate information systems in place. Public participation processes not fully specified. Few successful prosecutions. Large variance in current application and understanding of policies, criteria and

guidelines. Permits issued after the fact or not at all. Lack of support for recording data.

5. LOGIC FRAMEWORK

5.1 Context

This section explains the interpretations that have been placed on the information and findings gathered by the document review and by comprehensive interviews with key managers.The task of the consultants is (a) to determine the key performance areas (in other words, to describe the main business of WQM) and, (b) to establish a system whereby DWAF can determine whether WQM is achieving its intended results. The latter aspect involves as a first step, an analysis of the effectiveness of WQM. In essence, “effectiveness” can be defined as “doing the right things correctly”.

Effectiveness is determined by analysing the extent to which WQM planned achievements in key result areas met expectations in terms of:

The conditions, needs or problems concerned;

Established performance targets;

Past organisational performance; and

Whether WQM is meeting its own prescribed standards (as found in policy, guidelines etc.)



To understand the links between key performance areas and desired outcomes (impacts or effects) it is useful to employ a logic framework.

A logic framework depicting WQM business has been developed and is discussed in section 5.2. At the outset it must be noted that the desired outcome of this exercise, namely a Performance Assessment System for WQM can never be an add-on to the organisation. Rather it must feature as an integral part of WQM’s overall management system. The design of a performance assessment system is an ongoing and interactive process. The structure, responsibilities, practices, processes and resources for implementing WQM policies, objectives and targets should be co-ordinated with existing efforts in other areas (e.g. operations, finance etc.).

A depiction of a performance management system model is provided below for reference.

Figure 3: Performance Management System Model



The management system model illustrates how performance management ensures continual improvement in outcomes. The shaded area (Measurement and evaluation) is where the performance assessment system fits in. Measuring, monitoring and evaluating are key activities of a performance system to ensure that the Directorate WQM, Regional Offices’ WQM components and the Institute for Water Quality Studies to some degree are performing in accordance with the DWAF’s WQM mission.

5.2 The logic of the WQM business



Figure 4: Logic Framework of WQM Business5.2.1 WQM objectives

The mission statement of WQM embodies its overall objectives, namely “to ensure the integrated sustainable management of the water quality of the water resources of South Africa”. This is the task that WQM is mandated to perform.

5.2.2 Core Activities (Implementation)

Core activities are derived from main objectives. The main objectives of the WQM function are:

plan and develop point-source, diffuse source, remediation and resource directed integrated water quality management policies, strategies, procedures, guidelines, methodologies, regulations and criteria;

build capacity; provide specialist technical and strategic support; consider authorisations; monitor and audit the implementation of the set policy; manage WQM related information; promote transparent decision taking through Co-operative Governance and

participative management; ensure the integrated sustainable management of the water quality of the

water resources of South Africa through the application of Source Directed Controls (SDC’s) and Resource Directed Measures (RDM’s); and

ensure fitness for use of South Africa’s surface water, groundwater and estuarine waters for all user sectors, which include: Agriculture; Domestic; Industry; Recreation; as well as Aquatic Ecosystems.

5.2.3 Outputs

Outputs are the distinctive discernible products of resources having been devoted to core activities. Outputs are not the end results, which are outcomes or impacts or effects, but outputs are rather those entities produced which enable outcomes to occur.

The outputs of the core activities (key performance areas) are set out in the third level of the logic framework.

5.2.4 Immediate Impacts

When outputs are produced it is useful to consider that there are different levels of impact. Some results occur immediately. Other impacts and effects can only occur with the passage of time. When designing an overall performance assessment system it can be useful to distinguish three levels of impacts: immediate, intermediate and ultimate. Immediate impacts are those that occur directly as a result of output having been produced.

5.2.5 Intermediate Impacts



Intermediate impacts occur over time and are usually enabled by immediate impacts. In the case of the core activity area of policy it would refer to the ability to take appropriate and relevant actions guided by policy.5.2.6 Ultimate Impacts

The ultimate desired impacts of the WQM function in the Department of Water Affairs and Forestry must embody the mission statement. Thus, following the logic framework, the mission is really the alpha and omega. It is the end to which all the means at the disposal of the DWAF must work.

Monitoring and evaluating the extent to which the Directorate and the Regional Offices in terms of the WQM components are able to perform its mission is thus the main objective of the performance assessment system.

5.3 Performance Assessment of Regional Offices

The central questions are: How do the Regional Offices contribute to the performance of the Department’s WQM

function? How can the Head Office’s Directorate WQM evaluate and ensure regional

performance? How can the Head Office’s Directorate WQM ensure that it provides relevant support

to the Regional Offices?

The logic framework, although drawn up from the Head Office perspective, provides a logical basis to begin to answer the above questions.

Although the core activities differ slightly – policy development being of much lesser priority for the Regional Offices – the logic of progressing from objectives to core activities to outputs to immediate, intermediate and ultimate impacts or effects remains the same.

The key distinction is that there are different levels of performance. Firstly, the Regional Offices have a duty to monitor and sometimes test water use data provided by others. Secondly, the Regional Offices report these data centrally. Thirdly, Head Office has the duty to assess the Regional Offices and to hold them accountable to performance standards in these areas.

The previous sections set out the context and the logic for a Performance Assessment System for WQM.

The relevant inputs of some managers and stakeholders were obtained by way of a workshop. The results of the workshop are attached as Appendix C.

5.4 Issues to be considered in assessing accountability and responsibility

What are the responsibilities and accountability of personnel who manage, perform and verify work affecting water quality, and are these defined and documented?

How do the responsible and accountable personnel: Obtain sufficient training, resources and personnel for implementation? Initiate action to ensure compliance with water quality policy?



Anticipate, identify and record any water quality problem? Initiate, recommend, or provide solutions to those problems? Verify the implementation of such solutions? Control further activities until any water quality deficiency or unsatisfactory

condition has been corrected? Obtain appropriate training to act in emergency situations? Gain an understanding of the consequences of non-compliance? Gain an understanding of the accountability that applies to them? Encourage voluntary action and initiative?

5.5 Framework for monitoring and evaluation

A framework for monitoring and evaluation based on the logic model discussed above and informed by the document review and interviews was developed.

This preliminary framework sets out in tabular form the key performance areas of the WQM function. Possible indicators for the measurement of performance in these areas are listed. In addition sources of information are identified. Finally, standard assessment and monitoring methods are described.

It is stressed that the framework is at a preliminary stage. It would need to be simplified and adjusted following the iterations and inputs of all stakeholders. In addition cost implications have to be evaluated and considered.

The framework is attached as Appendix B.

6. CONCLUSION

With the acceptance of this report, Phase I of the project is concluded. The preliminary framework for a WQMPAS as delineated in this report provides the background and basis for the further development and implementation of the WQMPAS in phases II and III.

6.1 Limitations

The consultants have approached this project from the onset recognising that capacity and resource constraints within the DWAF would necessitate the design of a basic WQMPAS. The aim is getting a basic performance assessment system up and running that could be added to incrementally over time should adequate resources become available.

Regrettably, only limited input was received from the Regional Offices after submission of the draft report. Some input was given by staff from the Regional Offices during the workshop. However, the key performance areas need to be critically assessed and challenged by future users of the WQMPAS.

Big challenges for the next phase of the project will be the further refinement and prioritisation of KPA’s and the selection of appropriate associated KPI’s that will be key to a successful WQMPAS.

7. WAY FORWARD



This concludes Phase I of the project. A proposal for the continuance of Phase II and III was also developed separate from this document. During Phase II the framework will be

refined and a detailed assessment program developed. During Phase III an implementation plan will be completed and implementation will commence. The anticipated time for completion of the assessment program and implementation plan is estimated to take sixteen months to deliver.

The continued improvement of the WQMPAS is considered to be very important and will receive the necessary attention.

The specific objectives for phase II and III of the project are set out below.

Phase II

Refine the framework for the WQMPAS developed during phase I by ensuring that it reflects appropriate key performance areas, associated performance indicators, methods and parameters for the collection of performance information not currently held in DWAF’s information management systems.

Develop a detailed audit program for Head Office, Regional Offices and Catchment Management Agencies, including

- Detailed steps for each performance indicator identified.- The source to be used for audit purposes.- Sample sizes.- Frequency of audits.- Responsible people with reference to any special skills required.

Exploration of information systems and management information. Hold a workshop to familiarise users with PAS.

Phase III

Run tests of available performance information. Adjust PAS with reference to test results. Pilot feasibility testing of audit steps. Refine PAS with reference to pilot results. Compile an implementation plan. Capacity building and training of WQMPAS manager



APPENDIX A

Summary of results of document analysis



WATER QUALITY MANAGEMENT PERFORMANCE ASSESSMENT SYSTEMPossible areas for assessing performance identified through document analysis

Performance area Source1

NWA, White paper on National water policy & Bill of Rights and others

Everyone has the right to an environment not harmful to their health or wellbeing

12 (Bill of rights)

Everyone has the right to have the environment protected for the benefit of present and future generations through reasonable and other measure- prevent pollution & ecological degradation- promote conservation- secure ecologically sustainable development and

use of natural resource while promoting justifiable economic and social development

12 (Bill of rights)

Everyone has the right to sufficient water ((Quality & quantity)

12 (Bill of rights)

Sustainability and equity are guiding principles 10

Non-wqm: meeting basic human needs, present & future 10

Non-wqm: promoting equitable access to water 10,11

Non-wqm: everybody must pay (only a third currently pays non-payer e.g. afforestation, dryland users and disposal of waste

11

_________________________________

1Sources are listed at the end of the table



Performance area Source

Non-wqm: redressing the results of past racial and gender discrimination

10

Non-wqm: promoting efficient, sustainable and beneficial use of water in public interest

10

Non-wqm: water conservation may be a better investment than new dams

11

Facilitate social & economic development 10

Non-wqm: Providing for the growing demand for water use 10

Protecting aquatic ecosystems and their biological diversity

10

Reducing & preventing pollution and degradation of water resources

10

Must classify water resources and resource quality objectives

10 sec 13

Pollution prevention responsibility of owner, user, occupier of land. If not CMA must take responsibility

10

Emergency incidents, remedying rests with person responsible for emergency , otherwise CMA’s responsibility

10




All water use must be licensed unless listed in Schedule I, is an existing lawful use, is permissible under general authorisation or responsible authority waives need for licensing.

10

Non-wqm: meeting international obligations 10

Non-wqm: promoting dam safety 10

Non-wqm: managing floods & droughts 10

Establish suitable institutions with appropriate community, racial & gender representation

10

Non-wqm: establish a national water monitoring system for quantity, quality, use of water resources, rehab of water resources, compliance with resource quality objectives, the health of aquatic ecosystems and atmospheric conditions which may influence water conditions

10

To improve the efficiency and effectiveness in managing water quality

16

To streamline the regulatory system and to clarify its responsibilities in relation to both those of other regulating authorities and of those who contribute to water pollution

16

To enhance public confidence in managing water quality by making it accessible, easy to understand and simple to operate

16

To ensure the regulatory system is receptive to changing technology, to environmental impacts and to prevailing socio-political trends

16




To assess the financial implications of water quality management not only for effluent producers but for the national economy

16

Head Office

Develop policies and strategies for WQM in all main areas (establish national water strategy)

2 (10)

Assessment and control regional implementation of policies and strategies

2

Training of departmental water quality managers 2

Ministerial Investigations 2

Approvals, authorisations, licences or permits to Water Act, Environmental Conservation Act and Mineral Act

2

Permit system to control effluent standards exceeding a minimum standard

2

Management SystemsHead office

Develop policy, regulations, procedures and guidelines on industries and catchment management

2




Ensure that industrial and ICM research take place2

Develop technical expertise to provide regions and the Directorate to ensure ICM executed

2

Develop integrated computerised data management system

2

Ensure the implementation of policy Training for officers in water quality management Training with regard to ICM Ensure implementation of National Water Act Ensure computer management and support systems

are being used by regions and provide training for this

Support the regions with catchment management plans/studies/projects

2

Assessment of implementation of policies and compliance of industries and the performance of water quality management function

2

Regional Offices/CMA’s

Meet user requirements and establish a forum to address water quality issues

2,4

Staff satisfaction 5

Client satisfaction 5

Routine inspections local authority and government institutions

3




Handle pollution incidents3

Reduce faecal pollution impact upon rivers 4

Link with catchment management 4

Acceptable river flow monitoring 5

Dam safety inspections 5

Improvement in water quality 5

Ground Water monitoring Minimise at source the impact of development on

ground water Manage impacts to ensure fitness for use Restore groundwater quality Group according to importance and vulnerability and

where it is the only source of water for communities

6

Regulatory controls focussing on: Water abstraction and dewatering Disturbance and damage to aquifers Waste disposal from urban, commercial, farming

industrial and mining sectors Diffuse sources of pollution associated with urban

and rural development specifically around boreholes Underground storage tanks

6




Permit, exemption and EMP local authority, sect 12B, EMP’s and abattoir 3

Industrial Sector

Regional Offices/CMA’s

Assessment monitoring – industries 3

Permit, exemption and EMP 3

Point source discharge 4

Management of effluent disposal from industrial sites 5

Routine inspections 3

Waste ManagementHead Office

Develop policies and strategies, procedures and standards on waste management

2

Approval of hazardous waste site permits 2

Draft legislation and law enforcement 2




Regulatory approach prescribing best practice standards for prevention of water pollution 2

Capacity Building 2

Development and maintenance of waste management information systems

2

Formulate public participation processes 2

Training 2

Assessment Committees set up in consultation with Department

9

Assessments – 12 month intervals for small sites, 6 for medium, 3 for large sites and monthly for hazardous wasteAssessment programme should include: Checklist of items to be assessed Report on findings Record of performance – also to be submitted to

responsible person and IAP’s.

9

Keep record of complaints received and addressed 9

Planning and design of water pollution control works 2

Construction of water pollution control works 2

Effluent reuse 2




Treatment2

Discharge 2

Waste ManagementRegional Offices

Assessment waste disposal facilities 2

Issue waste disposal site permits/waste management permits for general and hazardous waste

2,4,6

Assessment monitoring sewage works 3

Permit, exemption and Environmental Management Programme (EMP)

3

Point source discharge 4

Disposal of low hazardous waste function 4

Management of effluent disposal from municipal sites 5

Liaison with authorities to upgrade sanitation treatment works

5

Steps for the design of a monitoring (could include site inspections, assessments, data collection, sampling, analysis and interpretation) system:




Obtain information on disposal practices volumes and types of waste

Obtain available information and sample surface and groundwater for chemical analyses

Perform a risk assessment, level of impact and monitoring facilities required

Perform geophysical investigations Drill boreholes Perform water sampling from holes Enter data into computerised database Present to client Include information in application for permit Train on-site personnel in use of database

6,9

Rank aquifers and consider for waste disposal 6

Routine inspections

Early warning monitoring systems

Minimum monitoring requirements at various types of waste management facilities

6

Indicator analysis until undesirable trends are uncovered

6

Regional monitoring systems 6

Monitoring as requirement for Environmental Impact assessment

6




Update of information system – Software Waste Manager/plot or update line graphs and tabulate information according to examples from Department

3,6

Six monthly reporting or at frequency prescribed by waste disposal permit

6

General Waste

Dealt with regional offices

Landfills divided between communal, small, medium and large.

9

Classified in terms of potential to generate leachate 9

Two external assessments a year 9

Hazardous Waste

Deal with Director of Water Quality Management – Head Office

Common sense assessment Information based assessment In-depth assessment

Research and development based assessment

1,9

Report on site sent to Department on a quarterly basis 9

Two external assessments a year 9




Maintain records of any impact caused by landfilling operation on the quality of the water regime

9

Landfills

Water Quality Monitoring Plan

9

Site selection 1

For all new sites and first time monitoring at existing sites a comprehensive analysis is required

Site feasibility and end-user requirements – DWAF confirmation of feasibility. Landfill site investigation to provide pre-disposal background water quality data. Landfill site report submitted to DWAF and results of IAP’s and Site Feasibility report.

Geohydrological Investigation Environmental Impact Assessment Site design. Water quality data expanded to

address other facets of water monitoring Operating plan. Water quality monitoring continues

throughout operation Monitoring plan Permit application report and form Site visit Input into waste disposal site registration system Notify applicant

6,9




Prepare landfill site – DWAF approval9

Control system/monitoring during operation, rehabilitation and closure. Gas monitoring and water quality monitoring

Surface water monitoring - Upstream and downstream

Ground water monitoring Minimum requirements

9

Issue of permits only if minimum requirements are met 4,9

Unpermitted landfills classified and assessed in consultation with DWAF

9

Landfill monitoring committee 9

Water monitoring up to 30 years after closure

1

Programme on waste disposal within the framework of the National Waste Management Strategy

5

Closed landfills

Permit holder apply one year before event Landfill process

1

8

8

Written acceptance of closure design and report issued by Department

9

Inspection by responsible person and Department 9




Closure and implementation of end-use plan assessed at final inspection attended by relevant state departments and monitoring committee

9

Ongoing inspections and maintenance after site closure 9

Post-closure water quality monitoring 6-12 months interval

9

Larger landfills 1

Landfills that produce leachate through ground water

1

Mines

Head Office

Evaluate training2

Evaluation of EMP’s 2

Evaluation of discharge permit applications 2

River diversion permit application 2

Mine closure applications 2




Regional Offices

Routine inspections 3

Manage impact 3

Address point pollution sources 5

Water pollution control measures at abandoned mines

2,4

Urban Development and Agriculture

Head Office

Strategy to manage pollution 2

Strategy to address pollution from the agricultural sector 2

Policy on underground petroleum reservoir tanks 2

Document on classification of aquifers 2

Guidelines for planners for ground water protection 2

Training of regional personnel to use software for the registration of water care works and operators

2

Ensure implementation of policy 2




Assessment of water quality management in South Africa

2

Catchment management plans 2

Catchment management projects 2

DWAF must influence the planning and siting by actively participating in formulating land development objectives

17

Set standards for levels of service 17

Delegation of responsibility for the registration of water care works and operators

2

Regional Offices

Raising awareness in dense settlements communities to maintain services

17

Allow development of settlement management agencies 17

Co-operative governance and community participation 17

Financially viable and sustainable 17

Characterise ito physical, environmental, socio-economic and institutional status

17




Catchment Management

Head Office

Develop catchment management plans and strategies Declare CM areas Classify water resource Determine the reserve Set resource quality objectives Develop water allocation plan

Should represent a programme, responsibilities, guidelines and procedures for implementation and programme of actions in the declared CM areas

2,7,14

Establish the department as a credible body, addressing and understanding the needs of the people

14

Formulate a coherent national CM policy 7

Initiate CM at the catchment level 7

Promote and link integrated water resource management at national and catchment levels

7

Formulate a programme of action 7

Implement regional transformation 7

Build capacity in regional CM staff 7,14




Define budget responsibilities 7

Evaluate CM budget needs of directorates 7

Specify CM Financing mechanisms 7

Develop CM guidelines and procedures 7,14

Develop CM information and Decision support system 7,14

Development of HR and capacity building through training e.g. in-house (water quality skills), departmental e.g. negotiation and communication

14

Conduct CM training awareness programmes 7,14

Establish Pilot CM Area Administrative Arrangements 7

Initiate pilot CM process 7

Water resource classification system 7

Reserve WR objectives and allocation plans 7

Receiving water quality objectives 2

Approval of effluent discharge permits 2




Formulate a programme of action for regional offices 7

Resolve issues concerning DWAF regional offices 7

Assessment the implementation of CM 7,14

DWAF’s role in statutory CM 7

Initiate and guide and promote research

Regional Offices

Perform catchment management functions with consensual participation by representative forums

7

Every catchment management agency to progressively develop a catchment strategy for water resources within its water management area. – must be reviewed at intervals of not more than 5 years

10

10 sec 8

Operate as CMA Highlight CM functions being performed in regional

offices Identify regional office personnel performing CM

functions Regional office budgets and spending to be

separated between CM functions and other functions

Regional director to act as CMA chief executive officer

7




Site analysis2

Water quality management plans 2

Assessment monitoring - river and industries 3

All data put on Polmon 3

Non compliance reported to permit holder 3

Report back twice a year 3

River monitoring 3

Integrated catchment management on river systems 5

Implement of all infrastructural and planning projects involving public at regional level and to create the required forums for consultation with regard to these projects.

14

That h/o reorganises itself to support the regions with those functions which it normally would have undertaken bulk demand satisfaction, catchment level supply and quality planning and permitting

14

Determine the water quality requirements of users 14




Assessment of water quality monitoring system of users14

Determination and characterisation of sources of water pollution

14

Evaluation of the water quality 14

Assessment of legal, institutional and managerial effective management of water quality

14

List of sources

1. Waste Management and the Minimum Requirements First Edition 19982. Water Quality Management: Directorate Business plan 97/983. Water Quality Management Free State Region4. Water Quality Management KwaZulu Natal5. Water Quality Management Mpumalanga6. Minimum requirements for water monitoring at waste management facilities7. Strategic Plan for DWAF for the implementation of Catchment Management in South

Africa8. Minimum Requirements for the handling classification and disposal of hazardous

waste9. Minimum requirements for disposal by landfill10.National Water Act (Act No 36 of 1998)11.White paper on a national water policy for South Africa12.Legal tools WQM orientation course 13.Operational guideline for control over the alteration in the course of a public stream14.Catchment management issues WQM orientation course15.Operational guideline for control over the alteration in the course of a public stream16.Water quality management policies and strategies in the RSA17.Draft strategy for managing the water quality effects of dense settlements

Additional Information

Number of complaints received and followed-up.Link of budget to different activities



APPENDIX B

Framework for monitoring and evaluation



Key Performance Areas

Perfor-mance Indicators

How measured/monitored Assessment Method

Policy Processes (policies, strategies, procedures, guidelines, methodologies)

Yes No Target Date Responsibility/Authorisation

DEVELOPMENT National Water Resources

Strategy Catchment Management Strategy Economic sector: Agriculture Economic sector: Industry Economic sector: Mining Economic sector: Waste disposal Water use: Taking water from a

water resource [S.21(a)] Water use: Storing water [S.21(b)] Water use: Impeding or diverting

the flow of water in a watercourse [S.21(c)]

Water use: Steam flow reduction activities [s.21(d)]

Water use: Controlled activities [S.21(e)]

Water use: Discharging of water [S.21(f)]

Water use: Disposing of water [S.21(g)]

Water use: Disposing of water containing waste, that has been heated, from any industrial process [S.21(h)]

Water use: Altering the bed, banks, course or characteristics of a watercourse [S.21(l)]

Water use: Removing/discharging/disposing of water found underground [s.21(l)]

Water use: Recreational purposes [s.21(k)]

Resource Directed Measures: Classification

YesYesYesYes

Yes

Yes

No

No

No

No

No

No

No

No

No

No

Urban Development & Agriculture

Industries Mines Urban Development &

Agriculture Waste Management None

None

Mines

None

All

All

Waste management

All

Mines

Mines

None

Formal minute meetingsDraftsProgress reports

Verify existence and evaluate completeness and compliance with legislation

Assessment communication evidence i.e. Meetings, minutes



Key Performance Areas Performance Indicators




Resource Directed Measures: RQOs

Water use: sector: Agricultural Water user sector: Domestic Water user sector: Industrial Water user sector: Recreation Water user sector: Aquatic

ecosystem Consideration for licensing:

Existing lawful water uses [S.27(a)] Consideration for licensing: Equity

[S.27(c)] Consideration for licensing: Socio-

Economic impact [S.27(d)] Consideration for licensing: CM

strategy [S.27(f)] Consideration for licensing: Impact

[S.27(f)] Consideration for licensing: Class

and RQOs [S.27(g)] Consideration for licensing:

Current investments [S.27(h)] Consideration for licensing:

Strategic importance [S.27(l)] Consideration for licensing:

Reserve/Int. Obligations [S.27(j)] Consideration for licensing:

Duration of authorization [S.27(k)] Diffuse source management Application of the precautionary

principle Pollution prevention Waste minimisation Cleaner technology Water quality friendly products

Yes

Yes

No

NoNoNoNoNo

No

No

No

No

No

No

No

No

No

No

NoNoNoNo

All

AllAllAllAllAll

None

None

All

All

All

All

None

All

All

None

AllAll

AllAllAllAll

Key Performance Areas Performance How measured/monitored Assessment Method



IndicatorsPolicy Processes (policies, strategies, procedures, guidelines, methodologies)


Best Practice Guidelines Recycling Risk Assessment General WQM framework policy Marine WQM Emergency incidents Rehabilitation policy Information Management Water Service Development Plans Pricing Strategy/Waste discharge

charges Procedures to assess effluent

discharges (Orange Book)

Yes

Yes

Yes

NoNo

NoNoNoNoNo

No

AllAll

AllManagement SystemsAllAllAll

All

All

All




How measured/monitored

Assessment Method


Yes No Target Date

Responsibility/Authorisation

IMPLEMENTATION AND MAINTENANCE Economic sector: Agriculture Economic sector: Industry Economic sector: Mining Economic sector: Urban

development Economic sector: Waste

disposal Water use: Storing water

[S.21(b)] Water use: Impeding or diverting

the flow of water in a watercourse [S.21(c)]

Water use: Steam flow reduction activities [S.21(d)]

Water use: Controlled activities [S21(e)]

Water use: Discharging of water [S.21(f)]

Water use: Disposing of water [S.21(g)]

Water use: Disposing of water containing waste, that has been heated, from any industrial process [S.21(h)]

Water use: Altering the bed, banks, course or characteristics of a watercourse [S.21(l)]

Water use: Removing/discharging/disposing of water found underground [S.21(l)]

Water use: Recreational purposes [S.21(k)]

Formal minute meetings

Changes and updates

Quarterly report-backs from regions

Evaluate completeness and compliance with mandate

Assessment communication evidence i.e. Meeting, minutes

Follow-up of regional concerns Assessment regional compliance to policies Assessment regions compliance to NWA

Key Performance Areas Performance How measured/monitored Assessment Method





Resources directed Measures: Classification

Resources directed Measures: Reserve

Resources directed Measures: RQOs

Water use sector: Agricultural Water user sector: Domestic Water user sector: Industrial Water user sector: Recreation Water user sector: Aquatic

ecosystem Consideration for licensing:

Existing lawful water uses [S.27(a)]

Consideration for licensing: Equity [S.27(b)]

Consideration for licensing: Beneficial use [S.27(c)]

Consideration for licensing: Socio-Economic impact [S.27(d)]

Consideration for licensing: CM strategy [S.27(e)]

Consideration for licensing: Impact [S.27(f)]

Consideration for licensing: Class and RQOs [S.27(g)]

Consideration for licensing: Current investments [S.27(h)]

Consideration for Licensing: Strategic importance [S.27(l)]

Consideration for licensing: Reserve/Int. Obligation[S.27(j)]

Consideration for licensing: Duration of authorisation [S.27(k)]

Diffuse source management Application of the precautionary

principle Pollution preventionKey Performance Areas Performance How measured/monitored Assessment Method





Waste minimisation Cleaner technology Water quality friendly products Best Practice Guidelines Recycling Risk Assessment General WQM framework policy Marine WQM Emergency incidents Rehabilitation policy Information Management Water Service Development

Plans Pricing Strategy/Waste

discharge charges Procedures to assess effluent

discharges (Orange Book)



Policy Processes (policies, strategies, procedures, guidelines,

Needs analysis

Training Numbers



methodologies)

Yes No Target Date Actual Training NumbersGeneral WQM training

Economic Sector: Industry

Economic Sector Agriculture

Economic Sector: Urban development

Economic Sector: Waste Disposal

Transformational, e.g. towards demand management/empowerment

Resource Directed Measures

Remediation Directed Measures

Use of information systems

Course evaluation Verify accuracy of numbers trained

Survey of course attendees

Training and Capacity Building

Needs analysis Yes/No

Strategies/ implementation

Responsibility


Assessment method

E.g. Joint ventures

with tertiary institutions

Bursaries and scholarships

Actions vs planned Verify appointments

Costs of recruitment

Attrition rate


Target % time OJT

Actual % time OJT


Assessment method

Transitional on the job training

Timesheets


Staff Retention How measured/Monitored

Assessment method

Key personnel identified

Attrition rate Replacement

Needs analysis (including equity requirements)

Personnel records and statistics

Verifying accuracy



Key Performance Areas

Policy Processes (policies, strategies,

procedures, guidelines,

methodologies)Gaps / Challenges

Specialist Technical Support

Performance Indicators

# of meetings # of forums # of joint studies # of presentations to forums

Management systems

Economic Sector: Agriculture

Economic Sector: Industry

Economic Sector: Urban development

Economic Sector: Water disposal

Resource Directed Measures

Remediation

Meetings minutesQuarterly reports from regionsForum minutes

Case studies of problems

Verify # of studies



APPENDIX C

Workshop Inputs

Final Report - 57 -June 2000


WORKSHOP ON WQMPAS – 9 MARCH 2000

Background

The Department of Water Affairs and Forestry’s Directorate Water Quality Management (WQM) has initiated the development of a performance assessment system and the establishment of a performance reporting system. The project consists of three distinct phases. EPC was awarded the contract for completion of the first phase. This document is an account of the proceedings of a workshop held at the end of the first phase on 9 March 2000. WQM placed great emphasis on receiving inputs and interactions from a whole array of role-players.

Individuals invited to attend the workshop 42Being:Representatives of regions 20Outside interested parties 5Representatives WQM Head office 6Representatives other Directorates 11

Unfortunately only 21 delegates were able to attend. The workshop programme is attached on annexure I and the attendance list is attached as annexure II.

S Manyaka, A Van Niekerk and M Roos presented the workshop. S van der Westhuizen provided the background and context of the project and P Viljoen described the way forward at the conclusion of the workshop.

Project in perspective

M Roos aimed to put the project in perspective and set the scene for the delegates to participate. She emphasised that the Water Quality Management Performance Assessment System (WQMPAS) was not an audit. The main objective of an audit was described as the gathering of audit evidence to be able to express an opinion on whether the financial information in the financial statements are a fair reflection. To be able to do this, auditors normally test the control systems or do detail testing on financial figures in the financial statements.

The WQMPAS is an assessment system. The difference is that instead of having an external party audit the financial information, management will be responsible for managing their own performance. In terms of an assessment system management will make representations on performance. Management will report on their performance and the information they report on will be reviewed for accuracy, completeness and validity. Actual results are compared to targets and



variances explained. The WQMPAS should be a tool that management use to manage their own performance.

Another very important feature of a performance assessment system (PAS) is that it should not place significant additional administrative burdens on employees. The benefits should outweigh the cost. In as far as possible current systems should be used to generate the information necessary for the WQMPAS.

Objectives of the workshop

M Roos subsequently indicated that the objectives of the workshop were to refine the framework for a WQMPAS that was developed to ensure it is:

Comfortable in the sense that it responds to needs identified.

Specific enough that information can be used to make decisions.

Useful.

Not cumbersome.

Easy to use and understand.

Logic framework

Annalie van Niekerk of EPC presented the session that focussed on the logic framework for WQMPAS.

The thinking behind the generic logic framework included in the workshop documentation was explained. It was stressed that the initial version of the logic framework represented the work of the consultants as informed by the analysis of documents and information gained in interviews. It was explained that this version would at best provide some “ribs” to support the envisioned PAS, but that the main purpose of the workshop was to receive inputs from workshop participants in order to transform the generic logic framework to one that would illustrate more precisely the activities and responsibilities of WQM.

The derivation of the logic framework was explained as follows:

The overall objectives of WQM are encapsulated in the mission statement, To ensure the integrated sustainable management of the water quality of the water resources of South Africa.



To achieve its mission, or overall objectives, WQM undertakes certain core activities, or Key Performance Areas (KPA’s). Head office and the Regional Offices share the responsibility of implementation of the mission through these activities. The core activities identified in the logic framework are:

1. Planning and policy2. Capacity building3. Provision of strategic technical support4. Operation of technical specialist input5. Control/authorisations6. Audit

The differential contribution of Head Office and the Regional Offices to the core activities was illustrated in the provisional logic framework and explained to workshop participants.

The outputs or measurable aspects stemming from the core activities were delineated as follows:

1. Policy2. Trained personnel and stakeholders3. Meetings, forums, joint studies, presentations, and reports4. Licenses and authorisations5. Operational systems (Polmon, CMS, WMS, WARMS)6. Audit system.

From the outputs or measurable aspects flow impacts. Impacts can occur immediately or over time.

Outputs or measurable aspects were related to associated immediate impacts and sketched in the logic framework. Similarly, intermediate impacts (that need some time to pass before becoming visible or measurable) were identified.

Finally, the accounting for and reporting on impacts and effects or outcomes associated with the business of WQM was shown to be a way to demonstrate the achievement of the mission. This, it was explained, is what a PAS must achieve. The PAS will allow any WQM manager to monitor and report on the extent that the mission is being achieved. It can be used as a diagnostic tool that will allow managers to make interventions or apply resources differently in order to be more effective, efficient and relevant in the achievement of objectives.

Key performance areas

It was explained that while designing a PAS, the temptation to include too much information for measuring and monitoring is almost irresistible. The axiom Less is More was stressed. Experience has shown that any organisation (or part of an organisation) should be able to confine its KPA’s to less than eight (as a rule of



thumb). Participants were asked to critically assess the logic framework with this advice in mind.

Key performance indicators

Key performance indicators were defined as things that could be observed or counted that could provide a diagnostic sense of the extent that a key performance area was being achieved efficiently and effectively. Participants were once again warned against the temptation to include too many KPI’s in the PAS. The phenomenon often observed in organisations to measure for the sake of measuring and to lose track of why or to what purpose the measurement occurs, was explained.

Criteria for KPI’s were specified. Satisfactory KPI’s are:I Few (choose one indicator as diagnostic of a specific objective) Occur over time (choose an indicator that can be monitored over time –

avoid once-off events) Must be quantifiable You must be able to measure the indicator against a target or

benchmark

Participants were then tasked with removing to three breakaway groups to wrestle with defining pertinent KPA’s for WQM.

It was decided to have three breakaway groups allocating delegates with similar interest, line functions and responsibilities together. Group I consisted of outside interested parties and representatives of Directorates other than the Water Quality Management Directorate. Group II consisted of representatives from the regions and Group III of representatives from the Directorate Water Quality Management at Head office.

Results of break-away groups

Each individual group identified the following KPA’s.

GROUP I (external)

People/behavior Partnerships Economic/social conditions Awareness

Conditions in aquatic environment/water resources Goals - objectives – actions



Translation of legislation Strategies – guidelines – policy

Functioning – monitor – control Data – information – understanding – knowledge – decisions/control Information on users (social/ecological/ economy) and status of water

resources

Authorisation relevance compliance

Strategies developed 1) need–objectives, plans and tools 2) priorities – actions,

capacity, tasks implemented achieved

Capacity/training/innovation DWAF Users/impactors(communities)

Cooperation DWAF, government and stakeholders

GROUP II (regions)

Source directed measures Licenses (NB) Compliance monitoring(NB) Specialist structures Incident response WQMP’s Law enforcement Registration

Resource directed measures WQMP’s(NB) Water quality objectives Coordination (internal and external) Catchment studies Reserve determination and classification Specialist studies

Information management and audit



Information systems in place

Institutional arrangements Catchment management forums (effective)(NB) Coordination (internal and external) Liaison

Corporate Services Financial management(NB) Administration Human Resource Management and Development Transformation

Policy Guidelines procedures and regulation

GROUP III

Quality of national water resource Water quality objectives Region sensitive/strategic areas – key areas and key indicatorsKPI’s – Water quality status report - trends and interpretation

Public acceptabilitySustainability – outputs linked to quality

Authorisations (number and time) Monitor and enforce Compliance Authorised vs. unauthorised

National Water resource/ catchment management strategies Developed and implemented Standards, norms and procedures

Monitor and enforce

Service delivery to public as measured to policy (internal and external) Public relations Hard products(water quality)

Personal performance

(Not prioritised)

Integration in department (intra, internal and external)Successful prosecutions



Successful remediationEfficiency input vs. outputLiaison - provinces (external and internal)Transformation (equality)Staff retention/ Human Resource Management

If the inputs from the three breakaway groups are critically examined and analysed seven broad KPA’s are revealed. These seven KPA’s encompass the original ones identified by EPC but they are at the same time much more specific in terms of the mission and objectives of WQM.

The KPA’s identified are: Facilitation of awareness

-public-role-players-stakeholders-institutional arrangements-inter/intra/external departments-provinces

Resource Directed measures

Management and Audit-source directed measures-information systems-monitoring-enforcement-control-authorisations/licensing

Strategies, Guidelines and Policies

Capacity building/Training/Innovation-transformation – equality-staff retention

Specialist technical support and input

Corporate support

During the next break-away group session each group was tasked with defining KPI’s associated with the three KPA’s chosen, namely, Institutional arrangements, Resource directed measures and Source directed measures.

The following KPI’s were identified by each group:



GROUP I (external) – KPI’s for Institutional arrangements(CMA’s, WUA, WMI and Forums)The main areas: Intra Department, Inter departments (LA, PG and NG, stakeholders, general public and training institutions). The platform consists of cooperation with government and role players (IAP and outside public). Role-players are domestic, WB, Imp users, environment groups, Agri, Urban, Mines and Industry.

Cooperation Benchmark = constitution Protocols Terms of agreement (MOU ‘s) Rules=time

6 monthly assessmentvery relevant to vision of WQM

GROUP II (regions)- KPI’s for Resource Directed Measures

RDM Plan

Setting of water quality management plans for catchment/management areas

Resource monitoring

Sub-KPI’sTraining/capacity buildingSet up dedicated RDM teamPrioritise QDA’s/Management unitsIdentify studies required

GROUP III KPI’s for Source Directed Measures

Authorisations# issued and enforced in order to WQO

Developed policies – number

Monitor implementation - % compliance of industries

Total load discharge – change per sector and per component (e.g. metal)

% GDP – pollution prevention – industry and region

Public awareness - %



Public/Private partnerships – number and sector specific (e.g. NUMSA and Petroleum)

Compliance/Inspections% compliance in relation to non-compliance

Prosecutions - # and improvement

Spills and pollution incidents, remediation

Emergency clean-ups

Recruitment and retainment of key skills

Re-cap of workshop achievements

The workshop served to demonstrate how complex the design of a suitable PAS is. The defining of more specific key performance areas turned out to be an onerous task. Each of the breakaway groups (first breakaway session) came up with a different list. This shows that at different levels of the organisation, managers have different performance information needs. A logical conclusion is:

The PAS must be capable of delivering the right resolution of information to each level of user.

In the plenary session, prioritisation of the KPA’s put forward by the groups proved challenging. This process was curtailed due to time constraints and a concern of the EPC consultants that participants’ time could be more fruitfully spent on concentrating on devising appropriate KPI’s for one KPA per breakaway group. In the second phase of the project when the PAS is refined the prioritisation exercise must be revisited.

Workshop participants were able to identify KPI’s associated with the three KPA’s chosen, namely, Institutional arrangements, Resource directed measures and Source directed measures. However, in this exercise not all groups were able to reach unanimity. The criteria for KPI’s were not applied across the board. It is clear that more work needs to be done regarding choosing indicators that meet the pre-set criteria.Overall the workshop met its main objective: to get inputs from participants and future users of the PAS in order to ensure that it meets specific as well as more general information needs.

Way forward and closure



P Viljoen closed off the workshop by indicating that the next step would be to summarise the proceedings of the workshop. Thereafter the project would move onto the second phase.



Annexure I – Workshop programme

08H00-08H30 Delegates arrive. Coffee/Tea

08h30-08h40 Welcome – Sakkie van der Westhuizen

08h40-08h50 Background to project – Sakkie van der Westhuizen

08H50-09H05 Project in perspective and objectives of workshop – Mariaan Roos

09H05-09H50 Logic Framework – Annalie van Niekerk

09H50-10H10 Coffee/Tea

10H10-10H20 Setting scene – Solly Manyaka

10H20-11H20 Break-away groups

11H30-12H30 Report- back and wrap-up of morning session – Solly Manyaka

12H30-13H15 Lunch

13H15-13H45 Prioritisation of KPA’s – Solly Manyaka

13H45-14H30 Break-away groups

14H30-15H30 Report-back – Solly Manyaka

15H30-15H40 Re-cap – Annalie van Niekerk

15H40-15H55 Wrap-up –Solly Manyaka

15H55-16H00 Way-forward and closure – Pieter Viljoen

16H00-16H30 Coffee/Tea



Annexure II – Attendance list

NAME & SURNAME POSITIONM. Roos EPC – Project LeaderA. van Niekerk EPC ConsultantH. Bolton EPCL.R. Gravelet-Blondin DD: WQM KwaZulu-NatalB. Schreiner CD: WUC.N. Lesufi DD: WQMS. Manyaka ConsultantJ.L.J. Van Der Westhuizen D:WQMA. Lucas DRD:WQM Eastern CapeH. Abbott AD:WQM Head OfficeG. McConkey DD:WQM Western CapeA. R. Makhadi D:IAJ. Mare AD: WQM Gauteng NorthS.C. Vogel D: North WestP.S. Venter DD: North WestT. Oosthuizen AEMSJ. Van Dyk AD Head OfficeP. Viljoen DD:WQM Head OfficeN. Mohapi AD: Catchment ManagementR. Stassen Business Area Manager: Water

Programme, EnvironmentekV. Mongwe DD:WQM Northern ProvinceL. Bredenhann DD:WQM:WM Head OfficeJ. Van Der Merwe DD:WQM Freestate RegionM. Keet DD: WQM GautengJ. Streit DRD
