Open Society Institute – Assistance

Foundation

A Z E R B A I J A Nbelongs to the Open Society Foundation Network established by famous investor and philanthropist George Soros. The main goal of OSI-Azerbaijan is to foster transition of a closed society to a more open one. Since its establishment in the country in 1996, OSI-Azerbaijan has encouraged the development of the third sector by awarding grants and through operational activity to support civic initiatives in education reforms, communication technologies, human rights and rule of law, mass media, public health, gender equality and arts and culture. Responding to new challenges of the country development, OSI-Azerbaijan has recently concentrated on increasing civil society involvement in the democratization process, good governance and transparency of the use of national resources.

BakuMay 2005

2

INTRODUCTION

Resource-rich states throughout the world face an unusual paradox. Natural resources, once touted as a blessing for poor countries, have more often contributed to poverty, violent conflict, corruption, and repression. The Open Society Institute sees the transparent use of revenues generated by the sale and transport of natural resources as well as a related social and environmental impact as issues of great importance for development and the promotion of civil society. Open Society Institute seeks to build the capacity of local groups to monitor oil revenues and hopes to ensure that existing and future natural resource revenues be invested and expended for the benefit of the public, such as poverty reduction, education, and public health - through the promotion of transparency, civic involvement, and government accountability.

In Azerbaijan Open Society Institute –Assistance Foundation has launched a program of Transparency of Oil revenues and Public Finance in 2002. Through various mechanisms the program aims to generate and publicize research, information, and advocacy on how revenues are being invested and disbursed and how governments and extraction companies respond to civic demands for accountability.

In April, 2004 Open Society Institute –Assistance Foundation and BP Exploration (Caspian Sea) Limited (BP) signed a Memorandum of Understanding for supporting NGO monitoring and capacity building project. OSI-AF and BP acknowledged that construction of the Baku-Tbilisi-Ceyhan Pipeline (BTC) is one of the most significant economic developments in the Azerbaijan Republic. It was agreed that during construction, various monitoring programs shall assess issues related to the BTC including: social issues in communities near the BTC; land ownership issues; environmental protection; historical preservation; local business content and protection of the rights of workers involved in the construction of the BTC.

Subject of this report is evaluation of the monitoring project from the perspective of OSI-AF acted as a principle donor, facilitator and coordinator of NGO monitoring groups. The evaluation will go through identification of strength and weaknesses of each project phases. It will also provide recommendations for second stage of NGO monitoring in 2005 to respond to identify weak points, to improve the process and to further promote capacity of local NGOs.

3

“Monitoring process is aiming to increase public awareness about BTC project development, about challenges and concerns raised during construction period and also serve for the establishment of healthy cooperation between all stakeholders to develop a good practice of civic involvement.”

OSI-AF Press ReleaseBAKU, April 2004

Memorandum of Understanding

Key elements of the Memorandum of Understanding signed by Open Society Institute –Assistance Foundation and BP Exploration (Caspian Sea) Limited (BP) include:

Goals and objectives- Development of experience and proficiency of local NGO’s for conducting

effective BTC pipeline monitoring- Bring successful international standards and practices of monitoring to

Azerbaijan- Improvement of local NGO’s monitoring skills- Continued improvement to BP activities in Azerbaijan via feedback from NGO

monitoring process

Principles of the Project- Integrated process involving both formal training and “learning by doing”- Transparency of the process- Process will reflect international best practice- Number of monitoring group members not more than – 30

BP duties- Support and assistance to participating NGO’s in the form of technical and

organizational support (health, safety, environmental training, transportation)- Discuss monitoring issues with working groups and answer relating to it

questions- When an information request is denied, provide a written explanation stating

the reasons for the denial- Provide an information on methods being employed, current construction

activities and upcoming work

OSI-AF duties- Hold a meeting for selection monitoring group members- Act as the facilitator and coordinator of the NGO monitoring process- Assist monitoring group members during the monitoring and reporting phases- Assist the NGO’s to work as teams rather than individually- Financial assistance to monitoring group activities (trips, salaries, etc. in the

frame of projects)

Monitoring group duties- To work as a team- To select coordinator and evaluator for a each working group- To conduct objective and dispassionate monitoring- Design the work plan together- Produce high-standard reports- Assess their own performance so that they learn from the process

Strengths:- A first for Azerbaijan and the industry- Mechanism to understand Azerbaijan NGOs and their capacity.- NGOs have increased awareness- New practices for OSI to extend to new initiatives

4

- New practices for NGOs to extend to new initiativesEnhanced BP / OSI relationshipWeaknesses: MOU deficient in some areas, for example, confidentiality, data security, archiving, use of photography, audio and video recording.MOU did not provide for the fact that a greater than anticipated amount of training and mentoring would be needed

PROCESS

NGO SELECTION

OSI-AF initiated the process with announcement broadly circulated in national massmedia. The Open Society Institute-Assistance Foundation invited National NGOs

withrelevant experience to participate in monitoring of the BTC oil pipeline. Monitoring by NGOs was set to focus on the following areas:

1. Environment;2. Social problems;3. Human rights (particularly, labour and land rights);4. Conservation of historical monuments;5. Use of local resources.

Eighty six national non-governmental organizations have applied to OSI-AF to participate in the monitoring process. Since there were many NGOs interested in monitoring the BTC pipeline project, OSI-AF in agreement with BP wanted the process to be as inclusive as possible, ie not limited to only a very few NGOs. To ensure more ownership and capacity building for a larger number of NGOs, OSI-AF introduced greater self-regulation into the selection process by providing the following:

- More NGOs had involvement in the process- The NGOs had greater decision making powers and accountability- Capacity building was made available to a greater number of NGOs- The selection process was transparent and the problem of selection of NGOs

by outsiders was removed- Teamwork between NGOs was promoted and maximised.

Five NGO working groups were created, one for each of the five areas listed above. Each working group consisted of four to seven members each, with one representative from each national NGO that was selected to participate. Thus, twenty-seven individuals were identified as local NGO representatives set forward to undertake a BTC pipeline monitoring. Having identified these individuals, the NGO meetings also elected an Evaluator for each Working group. OSI-AF facilitated meetings for Working Groups, hence members for each Working Group voted for a Coordinator.

Strengths:- A transparent process led by an organisation with a commitment to civil

society and capacity building.- A participative and democratic process.Weaknesses: No screening or

assessment process to eliminate unsuitable participants at the start or during the processNot all participants best fitted to a review, monitoring, and audit project

5

CAPACITY BUILDING

Many of the NGOs in Azerbaijan have limited required skills and experience in monitoring (methodology, planning, data collection, interview techniques, data analysis, report writing, presentations etc.). Capacity building was therefore essential to ensure that monitoring activities and reporting are efficient and produce meaningful outputs, which are useful to all parties. Thus, a strong capacity building element was identified as a priority for successful development of the project.

OSI-AF with financial support of BP has brought into a process Catholic Relief Services’ experience in implementing the “Independent Monitoring Project for the Chad Cameroon Oil Pipeline Project” conducted from March 2001 to February 2004. The Chad Cameroon project was the first of its kind in Africa, in that it provided a number of mechanisms for community inclusion in the monitoring and evaluation of construction activities by the people who actually lived and worked alongside the pipeline, and whose lives were directly affected by the changes that occurred during construction. Catholic Relief Services used its institutional expertise delivered a four-day monitoring training, for representatives of 27 local NGOs. The trainings covered the following topics:

1. General Monitoring and Evaluation Methodology and Techniques2. Data Collection Methodology and Techniques3. Data Analysis and Evaluation (Quantitative and Qualitative)4. Teamwork, Planning, and Proposal Writing5. Project Management6. Project Evaluation and Assessment7. Public Relations Methodologies

One of the most important elements that CRS brought to the capacity building process was the inclusion of Mr. Oliver Mokom, Chad Cameroon Pipeline Monitoring Project Manager. Local NGO representatives were provided practical overview of the process and developed a better sense of processes scale.

A need for close mentoring of the workgroup activities and constant training and capacity building exercises were outstandingly fulfilled by contracting Dr. Clive I. Morgan of Ridgeway Environmental Management (based in United Kingdom), this was also possible with BP funding. Dr. Morgan was with workgroups all throughout the monitoring process. As the project progressed new areas of need for capacity building were revealed, such as monitoring techniques, report writing expertise, communication skills etc. A sound capacity building element provided by Dr. Morgan has been one of the key factors for successful development of the project in 2004 and it consisted of the following:

1. An independent perspective on BTC monitoring and audit practice across the pipeline project

2. Mentoring and technical support relevant to monitoring and auditing at the home base of individual project groups in the field

3. Support during NGO monitoring report writing,

6

4. Interact with NGO’s through email, to prepare feedback report on NGO’s performance and individual’s performance,

5. Contribute to the development of a communication strategy for the output of the project

Thus, local NGO involved in the process have gained considerable monitoring capacity as a result of the project in 2004. Success in capacity building element of the project has lead to the emergence of an advance group of civil society representatives. Newly acquired skills of these individuals shall be further refined and strengthened for the ultimate goal of establishing a pool of local experts that could effectively serve as mentors and trainers for local civil society development in future.

Strengths:A significant training componentWeaknesses:Not all essential training needs addressedPROPOSAL DEVELOPMENT

Each Working Group produced a proposal regarding their theme. Proposals were developed within certain boundaries related to monitoring (number of trips to construction site, budget, etc) agreed between OSIAF and BP/BTC and provided to NGOs after the formation of the Working Groups. All proposals included the following items:

- Roles and responsibilities of each member- Geographical coverage of the monitoring- Description of the methodology of conducting the monitoring- Schedule of site visit/meetings/events with BP/BTC- Budget

OSI-AF received all the proposals and reviewed them, to ensure that they are within the financial and practical parameters. After assessing the proposals from each Working Group, and making necessary suggestions for amendments, OSI-AF awarded grants to the organization of a lead member/Coordinator in each Working Group.

Strengths:- NGOs learned a lot through having to correct and improve their proposals- Group effort team-work was involved

Weaknesses:- Variability of group size (smaller groups worked best)- Project proposal review process was not rigorous enough- Several projects with too wide a scope- Too many experts on the fringes of projects- Project objectives often poorly matched to methodology and action plans- Most projects required an audit element but no audit training given.

DEVELOPMENT AND IMPLEMENTATION OF ACTION PLANS

7

Each Workgroup reviewed with Dr. Morgan (Mentor) their proposals to ensure that objectives were consistent with methodologies and action plans set out in the proposals. This analysis revealed some inconsistencies in the original proposals.

Workgroups refined their proposals in the light of this analysis to make them more coherent. At the end of this phase of the process workgroups with help of the Mentor have achieved the following:

- Better definition of the objectives- Better definition of scope of work to be done- More realistic approach to field visits- More detailed action plans- More realistic view of work to be done within the timeframes of the project- Clear definition of roles and responsibilities for Working group members- Identified robust and reproducible methodologies, that were statistically valid

Strengths:- Use of the Project’s Mentor- Opportunity to refine the action plans before actual commencement of the

project- Iterative approach aided process learning by Working group members

Weaknesses:- This phase of the process should have been done prior to proposals

submission at the early stages of the proposals development- Some Working groups were forced to cut back the proposed scope of work to

make the it more achievable- Setback in project implementation by two weeks

MONITORING REALIZATION

Monitoring activities before report writing were scheduled to take place between October 1st and December 10th, 2004. In reality, some groups found it necessary to continue filed work into mid-December and other groups extended data collation and analysis into early February 2005.Monitoring process involved following components:- Document request- Document review and analysis- BP/BTC arranged site visits to CCIC a Spie Petrofac construction contractors- Working group arranged site visits to the communities along the pipeline- Site visits involved making observations, conducting interviews, carrying out

surveys, and making video, audio and photo records- Data collation and analysis completed by Working groups coordinators- Accumulated data on paper and electronic format was consolidated and

prepared for archive storage at OSI-AF

Strengths:- Opportunity of acquiring more information about BP/BTC/Contractor business

procedures and processes- Significant field based component to all projects- Opportunity to observe actual construction facilities- Sufficient support from BP/BTC for site visits- Independent work by Working groups in the 75% of communities along the

BTC pipeline8

- Enhanced ability of NGOs to conduct monitoring of complex and controversial projects

Weaknesses:- Key documentation took longer to source than promised- Some Working groups requested large number of documents but failed to

adequately review them- Lack preparation for visits to construction sites for some of the Working

groups- Presence of pre-conceived ideas about BTC related operations- Lack of proper environment necessary for objective monitoring during some

site visits- Limited number of visits to BTC sites- Limited opportunity for individual or small group visits to BTC sites- Poor coordination of site visit plans between different Working groups- Lack of mentoring during site visits- Lack of sufficient level of monitoring for some of the Working group members- Significant number of case when issues and allegations were accepted

without thorough investigations- Failure to meet original project deadlines

REPORTS

Original process timetable reserved a month for completion of report writing phase of the project, including reports review by OSI-AF. This was to be followed by translations of the reports into English and BP/BTC internal process review of the reports. As stated in the MOU, BP/BTC was to be allowed twenty days for internal review of the reports. By agreement with OSI-AF this was extended to thirty days. This revised timetable assumed public disclosure of the reports in the first week of February 2005. In reality, it proved impossible to meet this revised timetable. Some groups were inefficient to pull information together from different elements of their project. One group had too much data collected in non-systematic way, and another group had serious divisions internally about final content and format of the report, which lead to delays.

Strengths:- Current process and program have generated five reports of a reasonable

quality- The report writing phase clearly demonstrated a need for soundly written

proposals, detailed action plans, correct choice of methodology, well coordinated Working groups, in order to ensure that sound results are generated and reported on to sensible timetables

- Tremendously useful learning experience for NGOs, which will have enhanced their capacity on behalf of the civil society

Weaknesses:- Underestimation of time required for report writing phase of the project- Overestimation of potential of some working group members to produce a

reasonably good copy of a report in agreed time frames- Lack of fully dedicated OSI-AF resource to drive the reporting process to the

original timetable- Mechanism of internal review within OSI-AF was not sufficient for the scale

and the diversity of the project

9

- Difficulty to achieve consensus within some Working groups in order to produce a balanced and objective report

OUTCOMESAs of today, May 24th, 2005, all five reports have been submitted, reviewed and responded by BP/BTC. As the NGO Monitoring Programme drawed to an “Outcome” phase, aims were:- BP/BTC provides a respectful, factual responses to the issues raised by those

commenting on BP/BTC activities,- Those NGO report findings, suggestions and lessons that suggest ways in

which BP/BTC performance can be improved are identified. As a result BP committed to exploring means of operationalizing aspects of these key findings and recommendations,

- Dialogue was maintained between BP/BTC, OSI, and the NGO Working Groups throughout the process,

- BP/BTC and OSI provide assurance that, within the framework of a new round of NGO Monitoring, a system will be created that tracks implementation of these (mutually agreed) suggestions.

In the week commencing May 9th, BP/BTC and NGO Working Group representatives reviewed and discussed each of the main findings and recommendations in the NGO report(s), as well as corresponding BP/BTC response. The BP / BTC response was one of the following:

- BP / BTC accept the finding and / or recommendation (based on pre-agreed criteria, noted below). And where appropriate, indicate actions in response to the recommendation. Actions will be subject to verification by the working groups in the next round of NGO monitoring.

- BP / BTC accept the finding and / or recommendation with qualification.- BP / BTC declines to reject or accept the finding and / or recommendation and

asks that the issue is reviewed at follow-up monitoring in 2005.- BP / BTC rejected the finding and / or recommendation on one or more of the

following grounds (criteria) with justification as appropriate:

a) Factually incorrectb) Misrepresentationc) Poor data / analysisd) Not objectivee) Impracticalf) Inappropriateg) Outside scope of monitoringh) Not a BP / BTC responsibility.

Facilitated by OSI, the objective of these meetings were to:- - Provide NGOs with an opportunity to dialogue with the company regarding

the corporate responses- - Address each of the major NGO findings and corresponding

recommendations in discreet categories:a. Agreementb. Unresolved (an agreement to disagree, but unresolved issue that does not

pass to the new Monitoring round)c. More information required (agreed by both parties)items “a” and “c”, which of these pass become “d”

10

d. Findings/recommendations to be explored in new Monitoring round

In this dialogue process, the NGOs were asked to consider withdrawing any finding or recommendation rejected by BP / BTC and for which a reasonable case is made. The NGOs retained the right to differ with any BP / BTC response. As a result of this dialogue, an agreement made between BP/BTC was recorded in a table, produced for each of the NGO reports. An “agreement” table shall be made available for public disclosure with each report.

Strengths:- Another opportunity for constructive and healthy dialogue- Moving forward from NGO monitoring reports findings to a “case by case”

response- Fair number of recommendations are accepted by BP- Another strong capacity building exercise for NGO- Bridge to future monitoring activities

REPORTS DISCLOSURE

All Workgroups attended two training workshops on “Developing and Implementing and Communication Strategy” and “Handling the Media” provided by Dr. Clive I. Morgan. This enabled them to start working on their communication strategies and prepare for reports disclosure, using expert support provided by OSI-AF.

Outline of communication and reports disclosure phase of the 2004-2005 BTC monitoring project

Main Principles Compliance with “Open Society” values Information made public shall be within the boundaries of the official reports Information must be made available to broad public Information must be well-organized and structured and simplified to be

comprehended by various layers of the society Information shall be presented in balanced and objective manner

Activities

1. Meeting of each BTC Monitoring Working Group(WG) with the group of NGOs that have selected individuals to conduct monitoring within a particular WG. This event shall not be conducted with press present.

2. OSI-AF press conference, highlighting the following: OSI-AF strategy on Transparency of Oil Revenues and Public Finance Disclosure of the OSI-AF report of BTC monitoring process Announcement of the future plans Reference to a date and location of joint WGs press conference

3. Joint WG press conference, highlighting the following Disclosure of the WG’s MONITORING REPORTS of BTC monitoring

process Short presentation by each WG’s coordinator: achievements,

weaknesses, recommendations, BP comments Reference to date and location of communication events to be

conducted by each group

11

4. Activities to be conducted by WG Joint meeting at 3 regional centers (communities, municipalities, NGOs)

Facilitator: Catholic Relief Services Different events by WGs: documentary (social group), presentations at

the museums (archeology)

12

CONCLUSIONS AND FUTURE PROSPECTS

2004-2005 Baku-Tbilisi-Ceyhan pipeline monitoring project is drawing to completion. Already some important conclusions have emerged.

This project has proved to be most valuable as a first opportunity for Civil Society and a Trans-national Organization (BP) engaged in Azerbaijan’s sole vital resource production to cooperate and engage in an equal, healthy, constructive and on-going dialogue in good faith to contribute to the prosperous future of the country.

This project proved to be a challenging but stimulating testing ground for all parties involved. The project has developed a foundation for building a confidence and trust between the parties and recognition of truthful determination of all parties to serve the best interests of Azerbaijan society.

Challenges faced in the execution of the project included:- Lack of capacity in Civil Society- Unconstructive and biased approach of some of the NGOs- Different agendas brought into the project- Scepticism about the value of the pipeline monitoring project- The dominant position of BP/BTC- BP/BTC style of management vs. OSI-AF style of management- BP/BTC’s high standards of management vs. NGO relaxed management- Maintain and demonstrate full compliance with “open society” values

Achievements:- First monitoring project of this type, scale and complexity- Open and transparent self-regulated selection of twenty-seven NGO members- Enhanced capacity of most NGO members- 75 percent of the communities along the pipeline were visited in the course of

the project- Creation of databases of observations and results of interviews and

questionnaire survey- Creation of databases of audio, photo and video archives- Production of five reports- Identification of ten NGO members as an asset for future capacity building

and monitoring projects- Sound recommendations for improvement identified in the reports by

Working groups

Lessons Learnt

- Memorandum of Understanding and its Annexes and Exhibits must more accurately reflect project management requirements

- OSI-AF must have a Program Coordinator dedicated solely to the project- OSI-AF must have greater day-to-day monitoring of NGO activities- Project participants require a larger and better equipped office facilities,

rented specifically for the needs of the project

13

- Selection of the NGO members participating in the project must have more stricter criteria to ensure better quality

- All NGO members selected for the project must sign a Code of Conduct- More training on monitoring and audit and proposal development must be

provided- Proposals must contain more detailed methodology for monitoring- Scope and the objectives must be realistic and match the action plans- More realistic and detailed timetable must be designed for the different

phases of the proposals- OSI-AF must set up a rigorous process for reviewing and approving the

proposals and action plans- BP/BTC/SCP/Contractors must be more flexible to provide sufficient number of

visits to construction sites to accommodate diverse requirements of the Working groups

- BP/BTC/SCP/Contractors must be better guided in regards to handling NGOs, particularly during site visits

- Better communication with communities must be maintained before, during and close out of the monitoring phase of the project

- NGO members participating in the monitoring project must stay in compliance with the a Code of Conduct

- There must be a better coordination of activity between Working groups- Coordinators must be responsible for achieving a consensus on finding and

conclusions- Coordinators must ensure that all copies of reports only go to OSI-AF review- OSI-AF must establish a more rigorous reports review mechanism- BP must receive final reports in Azerbaijani language only from OSI-AF- BP must start translations immediately on receipt of reports- After reviewing reports BP must suggest only amendments on factual errors,

which would be avoided by more rigorous reports review mechanism within OSI-AF

- The table table of report writing, report review and report disclosure of the project must be strictly adhered to

Opportunities for 2005There are considerable opportunities to build on the achievements and lessons learnt in 2004-2005 BTC monitoring project. These include:

- Build on the current initiative by addressing deficiencies with a new project which all current programme participants are eligible to apply for

- Use the most highly developed participants from current project to assist in training and mentoring of participants in a new programme

- Extend the project to capacity build across a wider range of special interest groups in Azerbaijan civil society

14

These opportunities will be realized through a continuation of project in 2005, as illustrated in the following diagram:

15

Longer Term ProspectsThe successful completion of the project in 2005 could lead on to:The advisory and mentoring group continuing to maintain relations with the network of interested civil society representatives and organizations in Baku and regions of the country. This network will have a potential for further institutional development as follows:1. Establishment of Regional Citizens Advisory Councils, which will include NGO members, municipal council members, local community leaders who will:Hold town hall meetings and maintain permanent relations with executive authorities and municipal councils.- Collect concerns and complaints about oil industry projects.- Establish liaison mechanisms with oil industry organizations and meet them

regularly.- Identify with Advisory and Mentoring Group potential monitoring projects and

individuals and organizations to fund and implement them- Consider as a further promotion of their expertise issues of region challenges,

make requests and provide recommendations to local authorities, non-government institutions, parliamentarians, business associations, companies, other institutions regarding regional development issues (infrastructure, local budget, municipal budget, environment, business, arts and culture)

2. Transformation of Advisory and Mentoring Group to a think tank and an Advisory and Mentoring Center (AMC) with the following capacity:- center for accumulation of oil project impact data with public access to data

basis and information resources- training and mentoring capacity for any oil industry monitoring projects- consultancy for RCAC in fundraising and implementation of monitoring

projects- production of annual civil society report about monitoring efforts and results- holding presentations and other events for publicity of RCACs in the capital- maintenance of coordination with other civil society groups working on public

policy issues- maintenance of international relations for monitoring experience exchange

and advocacy- advocacy for policy impact in relation to oil industry development challenges

OSI-AF plans holding meetings and brainstorming sessions for assistance in development of a project on behalf of AMC for OSI-AF funding and additional fundraising.

16