Clean - Final Evaluation Report - Policy Influence Through Community Empowerment Project

7/31/2019 Clean - Final Evaluation Report - Policy Influence Through Community Empowerment Project

1/54

End of Project Evaluation Report

Policy Influence through CommunityEmpowerment

November 2010 - May 2011


2/54

May 2011

TABLE OF CONTENTS

Table of Contents ........................................................................................................ 2

Acknowledgments ...................................................................................................... 3

Affirmations ............................................................................................................... 3

Glossary/Acronyms and Abbreviations ........................................................................ 4

Executive Summary .................................................................................................... 5

Evaluation Introduction/Background ............................................................................ 7

Policy Influence through Citizen Empowerment: Project Designs Document, p 4 .......................8

Evaluation methodology ............................................................................................. 9

Limitations ................................................................................................................ 14

Evaluation findings .................................................................................................... 14

Evaluation objective 1 ............................................................................................... 14Policy Influence through Citizen Empowerment: Project Designs Document, p 2 .....................16

Evaluation objective 2 ............................................................................................... 22




Conclusions and Recommendations .......................................................................... 42

Lessons learned from the Evaluation Process ............................................................ 44

Appendices ............................................................................................................... 45

2


3/54

AcknowledgmentsPolicy Influence through Community Empowerment Project, project summativeevaluation exercise and report is the result of contributions by WVA DME M&E officerAnna Pokhsraryan, WVA DME M&E officer Artak Saghatelyan, WVA DME ManagerAstghik Movsisyan, WV UK PPA Programme Manager Robert Gillen, WVI CV&A learningSpecialist Keren Winterford.

Stepanavan and Alaverdi TDFs did a professional job in collecting data. Special thanksto WVA DME M&E Officer Mikayel Hambardzumyan for provided expertise during theanalysis.

AffirmationsExcept as acknowledged by the references in this paper to other authors andpublications, the end of project evaluation described herein consists of our own work.It was undertaken to determine the projects effectiveness, and results measuredagainst goals, objectives and outputs set forth in the project

Primary quantitative and qualitative data collected throughout the evaluation willremain the property of the communities and families described in this document.

Anna Pokhsraryan, WVA Monitoring and Evaluation OfficerMarch, 2011

3


4/54

Glossary/Acronyms and Abbreviations

ADP Area Development Programme

CAG Community Active GroupCBO Community-based Organization

CBON Community-based Organizations Network

CVA Citizen Voice and Action

CWB Child Well-being

CWBO Child Well-being Outcomes

DME Design, Monitoring and Evaluation

FG Focus group

KII Key Informant Interview

M&E Monitoring and Evaluation

LG Local Government

NO National Office

SO Support Office

TST WV Armenia NO Technical Support Team

WV World Vision

4


5/54

Executive Summary

Application of Policy Influence and Citizen Empowerment project which was basedon the CV&A methodology in Stepanavan and Alaverdi ADPs for the last three yearsresulted in establishment of a more structured approach and social accountabilitymechanisms towards promotion of citizen empowerment and mobilization towardsthe common goal of improved public services.

According to community residents who participated in evaluation process there issignificant improvement of the quality of services if compared with the situationexisting before application of CV&A tool.

According to residents the project established a platform for dialogue between theordinary citizens and public service providers as well engaging in discussions withdecision makers at regional and national levels through community general meetings,town hall meetings and roundtable discussions.

Another achievement of the project implementation is significant increase ofawareness on entitlements for public services among target community residents andservice providers. In depth discussion of entitlements among different servicereceivers groups, even among children groups resulted in several positive changes

initiated by these groups. Community residents can require exact reform referring toan exact law, legal norm or decree.

Application of the CV&A also contributed to the enhancement of facilitation skillsamong the ADP staff and strengthened the capacity in the National Office in terms ofcarrying out evidence based advocacy.

Compared to other ADPs where CV&A method was not applied one can say thatthe participation of ordinary citizens is ensured in a more structured way. Themajority of active groups established by other ADPs (not applying CV&Amethodology) convene meetings upon availability of exact project opportunities or ifthere is a need to address any specific issue. Community action plans are goodexamples of participatory planning tool which has potential of replication while

discussing and planning community budgets.

Nevertheless, several barriers and limitations were identified as the result of theproject evaluation.

The low capacities among community representatives in identifying concretesteps that are needed for improvement of public services in their communities hinderthe quality of action plans. Also given this low capacity of communities, serviceproviders are in a more advantageous position and always try to push their ownagendas. Low capacity comes also from the fact that the majority of communitymembers are not aware of community budgetary funds, distribution of functions andresponsibilities among LG, Marz level administration, etc. There is a risk that service

providers try to push their own agenda (e.g. some of interviewed pupils mentionedthat during the meeting they were asked by teachers or even by principal to raisecertain issues).

The data entered into the CV&A database is not so practical and sometimesremains unutilized.

Not all ADP staff has the sufficient capacity to effectively facilitate communitydiscussions and meetings.

The main partners of WV for facilitating the participatory process were selectedCBOs and community active members. Meanwhile, according to the Nationallegislation on Local Governance community council is the body that is mandated toensure community participation in decision making and represent the communitybest interest in decision making. Given the current situation these communitycouncils were least involved in the CVA processes, which causes limited interest

5


6/54

among the councils to participate and take over, thus ensuring institutionalsustainability of the approach.

It must be stressed that lack of awareness among community members aboutcommunity budgets results in unrealistic action plans. Meanwhile, in some casescommunity councils and heads are reluctant to conduct open discussions of thecommunity budgets.

According to almost all interviewees the only real guarantee of the projectsustainability is WV. That is why, it is especially important to facilitate theinvolvement of community councils into the process.

6


7/54

Evaluation Introduction/BackgroundProgramme/Project 185461 Policy Influence through Citizen Empowerment

projectProject Phase FY08-11

Evaluation Type Summative EvaluationEvaluation Purpose To determine the projects effectiveness, and results

measured against goals, objectives, outputs set forth inthe project. The evaluation will offer strategic andoperational recommendations to ensure future direction,sustainability and effectiveness of this project.

PrimaryMethodologies

Quantitative and qualitative methods of data collection,secondary data review

Evaluation Start andEnd Dates

4 months

Anticipated

Evaluation ReportRelease Date

End of March, 2011

This report presents the summative evaluation of the project Policy Influencethrough Citizen Empowerment project. The aim of the research was to collect dataon the following items: the relevance of the project to local needs, its effectiveness,unintended outcomes, project model impact, integration with ADP practices andsustainability.

The project was designed to address the needs existing in Armenian society in termsof low civil participation in decision making, lack of ownership towards communityresources and development and low level of delivery of basic services. The projectinitially started with the tool of Community-based Performance Monitoring (CBPM),which then, largely contributed by the analysis of the project implementation, wasdeveloped into the tool of Civic Voice and Action (CVA). The major difference betweenthese tools is that the CBPM was more focused on public monitoring of the stateservices, while the CVA has a broader advocacy goal of community empowermentand participation.

To promote democratic processes, citizens of Armenia need both expandedopportunities to participate in civic life and more open governance institutions tochannel participation into a stronger, more democratic system of governance. Many

local citizens are often excluded from engaging with, influencing and benefiting fromthe planning and implementation of poverty reduction and developmentprogrammes. Meanwhile, effective involvement of civil society in poverty reductionwill be enhanced by increased use of social accountability mechanisms, such asCommunity-Based Performance Monitoring (CBPM), to increase the influence of, andaccountability to, poor people in PRS monitoring and the delivery of health, educationand other essential services.1

The goal of this project was to contribute to the empowerment of citizens ofStepanavan and Alaverdi ADP communities to be able to influence goodgovernance through ensuring the civil society participation in elaboration,

1 World Vision, Armenia PRSP Guide; A guide to World Vision engagement with the Poverty

Reduction Strategy Process in Armenia

7


8/54

implementation and monitoring of the community development processesand public services, as well as to use the data collected from the field forthe national level policy influence2.

To serve the above goal the following outcomes were set for the project:

Outcome 1: Increased civil society mobilization and engagement in local/regionallevel development processesOutcome 2: Effective relationships between the LG, CBO, communities and otherdevelopment actors operating in the project sites ensured for further promotion oflocal self-government decentralizationOutcome 3: Improved public service delivery at local level through communityempowermentOutcome 4: Development of good governance promoted at national level

Evaluation TypeThis is summative evaluation to investigate the whole impact chain of efficiency,

effectiveness and consistency.

Evaluation PurposeTo determine the projects effectiveness, and results measured against goals,objectives, results and outputs set forth in the project. The evaluation will offerstrategic and operational recommendations to ensure future direction, sustainabilityand effectiveness of this project.

The evaluation objectives are:Evaluation objectives Key evaluation questions

1. Analyse the

relevance of the projectdesign to the existingneeds

To what extent was/is the project strategy relevantand aligned with the priorities of target areas?

How was the program design process conducted?

Were any unplanned outputs achieved?

How did assumptions affect the program?

2. Analyse the projecteffectiveness, i.e. towhat extent the projectobjectives have beenachieved and how,

including revealing thepossible unintendedoutcomes

To what extent has the project been effective inachieving the objectives and outcomes as stated inthe strategy?

How practical and doable is CV&A methodology?

What unintended outcomes (both positive andnegative) have occurred?

How did the community benefit from the project?

What major changes have been implemented as aresult of the project?

3. Analyse theintegration of theproject with the overallADP design consideringthe ways in which the

How the project supplements the overall ADPdesign?

Did the project objectives contradict with/challengethe overall ADP design?

Did the project objectives overlap with the overall

2 Policy Influence through Citizen Empowerment: Project Designs Document, p 4

8


9/54

project has linked with ADPdesigns and ADPprogramming practice

ADP design?

4. Analyse the impact ofthe project model oncitizen empowerment

How the project did enhance civic participation indecision making process? (Comparing CVA and CAG

models)

3

What barriers were there to participation and howwere these addressed?

What forms of community participation have beenincreased and in what sectors of social activity thelevel of civic participation has especially increasedas the result of the project? (Comparing CVA andCAG models)

What mechanisms of social accountability wereestablished through the project? (Comparing CVAand CAG models)

What was the socio-demographic distribution ofpeople who took part in the project? (gender andage equality in community participation)(Comparing CVA and CAG models)

5. Analyse thesustainability of theproject model

To what extent has the projectestablished/enhanced capacity, processes andsystems that are likely to be sustained?

What capabilities does the ADP/CBO staff have forfacilitating further application of CV&A

What elements of CV&A can be applied as a generalmodel in all ADPs

To what extent is the LG supportive to the CVA

model and willing to sustain the projectachievements

Evaluation methodology

a) Research methodsEvaluation will be based on both qualitative and quantitative research methodologiesselecting appropriate research methods for each criterion, as well as secondary datareview.

Objective 1. Analyse the relevance of the project design to the existingneeds

The evaluation of objective 1 started with analysis of project documentation package,namely project design document, progress reports and monitoring reports, as well asthe documentation package for the target ADPs. Document review took place inparallel with the field stage.After completing questionnaire approbation and interviewers training key informantinterviews (KIIs) with CBO/ADP managers and local government representatives weredone. Focus-groups with community members were conducted as well.

3 CAG, Civic Action Groups, are the community engagement/mobilization model applied by WVArmenia ADPs prior to application of the CVA model and currently utilized in some of the ADPs.

9


10/54

Objective 2. Analyse the project effectiveness, i.e. to what extent theproject objectives have been achieved and how

The evaluation of objective 2 started with analysis of project documentation package,namely program log frame was analyzed in order to find out to what extend theprogram objectives were addressed. Data review included CV&A database to find outthe level of its usefulness in overall project prospective. Project design document,progress reports and monitoring reports, as well as the documentation package forthe target ADPs were analyzed as well. Document review took place simultaneouslywith the field stage.

Focus group discussions (FGD) took place with ADP/CBO staffs. FGD were organizedwith community members as well. KIIs were conducted with LG representatives andservice providers.

Objective 3. Analyse the integration of the project with the overall ADPdesign

The evaluation of objective 3 started with analysis of project documentation package,namely project design document, progress reports and monitoring reports, as well as

the documentation package for the target ADPs. Document review took place prior tothe field stage. KIIs with ADP /CBON managers and staff were conducted as well.Objective 4. Analyse the impact of the project model on citizenempowerment

The evaluation of objective 4 was based on quantitative interviews with communitymembers, using structured interview as a tool. Document review of participants listsand community meetings reports were implemented. FGD with community memberswere conducted. KIIs with CBO and ADP managers were done as well.

Objective 5. Analyse the sustainability of the project modelThe evaluation of objective 5 started with analysis project documentation package,

namely project design document, progress reports and monitoring reports, as well asthe documentation package for the target ADPs. Document review took take placesimultaneously with the field stage. FGD with community members were conducted.KIIs with CBO and ADP managers were done as well.

b) Research tools

Quantitative research:Quantitative interviews were conducted using semi-standardizedquestionnaire (For LG representatives in target ADPs communities) as a research

tool. Questionnaires contained 21 core questions stressing such points as LGsperceptions of citizen participation in general and CVA tool in particular, the degree oftheir openness for civil society, the way how the role of civil society increased incommunity decision making process, etc.

Quantitative research:Key informant interviews were conducted using KII guide (Service providers,including LG representatives of surveyed ADP communities) as a research tool. Thisresearch tool includes open-qualitative questions such as perceptions of citizenparticipation in general and CVA tool in particular, the way how community-serviceproviders relations and services accessibility/quality were changed as a result of theproject, etc.

10


11/54

Group discussions were conducted with FGD guide (with ADP, CBON, CAG staff andcommunity residents4).

In all research tools evaluation objectives were highlighted as separate researchquestion blocks.

The questionnaires used for each of target groups are presented in Appendices.

Document review:Evaluation team reviewed the documentation of implementation process to definewhether activities completed and outputs achieved. Secondary data included suchdata sources as package, namely project design document, progress reports andmonitoring reports, ITT, CVA database as well as the documentation package for thetarget ADPs.

Sampling strategy

The quantitative part of the survey was conducted on sampling criteria of censusas a procedure of systematically acquiring and recording information about all

members of a given social group. That means that all community mayors from targetADPs communities were interviewed, except 3 mayors of surveyed communities pereach ADP, who were already included in qualitative part of the research to get in-depth information over evaluation questions.

Table 1: Sample scheme for quantitative research with community mayorsSTRUCTUREDINTERVIEWS

Target ADPs # of ADPcommunities

Number of interviews

Stepanavan 20 17

Alaverdi 10 7Tavush 20 5

Total 30 24

Within qualitative part of the research KIIs and FGDs were conducted withdifferent target groups. Quantitative research was based on purposive samplestrategy. In case of community residents group interviews, the sample is based onsnowball technique using social networks as group composition mechanisms.

During the research the following methodology of communities selection was used.Together with ADPs and CBONs staff, the CVA databases for each of ADP werereviewed and community rates calculated based on a comparison between numberof identified/ planned activities in CVA action plans and number of improvementcases. Three communities per ADP were selected which are marked as the best,middle and the worst according to obtained scores.In each of selected communities 4 FGDs were conducted with the following socialgroups.

1. Mixed group: In middle level communities mixed groups consist of the poormembers of society.

2. Non-participants3. Women4. School age children

4 The same guide were used as a tool with ADP, CBON managers interviews

11


12/54

KIIs were conducted with the following local service providers:1. LG2. Education (secondary school, kindergarten)3. Health (medical posts)

As for project implementers, the following groups were included in the survey sample:1. ADP managers/staff2. CBON managers/staff3. CAG staff

Four interviews were conducted with Lori marz level authorities from: Head ofeducation department, head of LG department, Deputy Manager of the Department ofHealth and Social Affairs; Deputy Manager of the Department of Analysis andDevelopment

Table 2: Sample scheme for qualitative research (FGD-community residents)Focus group interviews (community residents)

TargetADPs

# ofADP

communities

# ofsurv

eyedcommunities

Surveyedcommuniti

es

# of surveyedsocial groups

# ofintervie

wedcommunitymembers

Stepanavan

20 3 1.the best-Vardablur;2.middle -Pushkino3.theworst-Agarak

Mixed group 3 36

Non-participants

3 34

Women 3 32School age

children

3 32

Alaverdi

30 3 1. Thebest -Haghpat 2.middle -Qarinj3.theworst-Arevatsag

Mixed group 3 36

participantsNon-participants

3 32

Women 3 32School agechildren

3 36

Total 50 6 6 8 24 270

Table 3. Sample scheme for qualitative research (FGD and KII-projectimplementers)

Focus group interviews Key informant interviews

TargetADPs

Targetgroup

# ofsurveyedsocialgroups

Targetgroup

# of surveyedsocial groups

Stepanavan

CBON 1 ADPmanager

1CAG 3

ADP 1

12


13/54

Alaverdi CBON 1 ADPmanager

1

CAG 3ADP 1

Tavush CAG 1 ADPmanager

1ADP 1

Total 12 3

Table 4. Sample scheme for qualitative research (KII- service providers)Key informant interviews

TargetADPs

Target group # of surveyed socialgroups

Targetgroup

# ofsurveyedsocialgroups

Stepanavan

LG 3 ADPmanager

1

Education

Secondary

school

3

Kindergarten

15

Health 3

Alaverdi LG 3 ADPmanager

1

Education

Secondaryschool

3

Kindergarten

3

Health 3

Total 22 2

c) Analysis strategy

Because of the limited sample, the quantitative research data were entered into Exceldatabase developed by WVA M&E officer and analyzed using descriptive statisticscommands.

Analysis of the collected qualitative data was carried out using analytical sketchesdriven from Grounded theory supplemented by categorical-formal models of data

combining and generating

6

.

Upon the preliminary analysis of the evaluation findings the latter were validated withrespective people, including the WV Armenia Operations Manager, Managers of thetarget ADPs, Manager of Sisian ADP where the CVA tool is applied, ProgrammeDevelopment and Quality Manager, DME Manager and two DME M&E Officers, who

5 No kindergarten was operating in 2 of Stepanavan ADP communities.

6Grounded Theory is a research method in which the theory is developed from the data, rather than the other way

around. That makes this is an inductive approach, meaning that it moves from the specific to the more general. The

method of study is essentially based on three elements: concepts, categories and propositions. Grounded Theory is

used as a method of qualitative data analysis.

13


14/54

were responsible for leading the evaluation exercise. During the validation meetingthe initial findings were discussed and agreed upon and respective recommendationsdrawn. In the course of the discussion several new factors affecting the quality andsuccess of the CVA tool application were identified and agreed upon. An illustration tothe latter can be the staff capacity to facilitate that is deemed among the key successfactors by the respective leadership.

Limitations

Target ADP staff were very helpful in providing all necessary information andorganizing field works. Nevertheless, in case of outside advisers engagement inevaluation process it will be reasonable to have more translators involved in theprocess in order to provide gain data circulation between evaluation team membersduring the field stage

Another important limitation is that NO technical staff was not available through thewhole duration of the evaluation. Whilst they were able to be interviewed, this wasshortened, and only limited involvement in the exercise. They also did not participatein the finalisation of this report.

Evaluation findings

Evaluation objective 1The relevance of the project design to the existing needs

Under evaluation objective 1 the following questions were discussed.

To what extent was/is the project strategy relevant and aligned with the

priorities of target areas? How was the program design process conducted?

How did assumptions affect the project?

Were any unplanned outputs achieved?

What major changes have been implemented as a result of the project?

The extent of the project effectiveness in achieving the objectives andoutcomes as stated in the strategy

To reveal the extent to which project strategic objectives are aligned with community

needs one should refer to problems and important objectives raised andprogrammatic achievements stressed by community itself.

(Problem) Low level of public service delivery: (Achievement)Increased delivery of public services

It should be mentioned that both Stepanavan and Alaverdi residents involved in thesurvey were quoted as saying that the general situation with social services improvedin recent years. The majority of residents think that during the past three years therewere numerous significant changes in their communities regarding services both interms of physical access and quality. According to surveyed social groups mainachievements are seen in the following areas:

- Water supply and gasification- Improved preschool and secondary school facilities

14


15/54

- Increased access to health services- Improved waste management

It should be noted that

In terms of evaluation process the common needs of the communities also have beendiscussed. Low employment level, high poverty rate and intensive migrationproblems are mentioned among other issues which according to residents all derivefrom poor socioeconomic development level of the communities. Although theproblem of socioeconomic development per se is not listed in programmatic aims andobjectives it should be mentioned that some targeted social groups believe thatcommunities first of all need to create economic development opportunities.

People do not have either workplace or money for food, but they always call us andask us about our priorities you can help us create jobs in the village so that peoplecan understand something from their lives instead of chatteringif we work and earn money, we can mobilize and put our village in its own feet

Extracts from FGs with Pushkino and Vardablur community members (the

poor)

(Problem) Low level of community mobilization: (Achievement)Increased community mobilization

Increased level of community mobilization is considered as one of the most importantachievements of the project by all targeted social group representatives participatingin the survey.

The life of villager is complicated everyone deals with his/her own problems tomeet ends. During these community meetings we start thinking together and findsolutions

Before we used to complain in the street with no result but now we realized thatif we act togheter our voices will be heard Extracts from FGs with Agarak and Qarinj community members(participants groups)

(Problem) Lack of knowledge of rights and responsibilities towardspublic services: (Achievement) knowledgeable community residents

According to the majority of target community residents there was a significant need

of information on their rights regarding this or that service. The knowledge existingamong the questioned residents was fragmented and of an incomplete nature andconnected to particular service which residents were using at a given moment.

Throughout project implementation residents received significant amount ofinformation on state entitlements and obligations on concrete public services andtheir rights. Nevertheless it should be mentioned that increased awareness aboutrights not always leads to proper implementation ofobligations.

During these meetings people attack us all around with their demands. But are theypaying, for example, the land tax? Pay taxes so that village municipality has fundsto cover existing issues. When you explain their rights please tell them about theirresponsibilities as well

Extracts from FG with Vardablur community mayor (Stepanavan)

15


16/54

Nevertheless, in Alaverdi communities there are several cases when residents paidland taxes couple of days after participating in community meetings.

Problem (Lack of civic dialogue, accountability): (Achievement)Community residents-LG-service providers improved relations

In Armenian context rural environment is generally characterized by informal socialrelations that result in private dialogue between residents and service providers. Nonformal relationships on one hand can enlarge the scope of connections with serviceproviders (people can easily reach them and approach local level service providers).On the other hand, non formal relationships, as a rule, decrease the level of socialresponsibility mechanisms.

When you approach the village mayor with your problem he says, Today ortomorrow it will be solved.... But that today never would come. Communitymeeting is quite another matter, when something is agreed during the meeting it willbe done.Extracts from FG with Vardablur community residents (the poor)

The analysis of the evaluation findings allows to conclude that the projectdesign had been done based on the actual community needs and in linewith the five components of the CVA tool, namely service, information,voice, dialogue and accountability7.

Program design process

Policy influence through Citizen Empowerment project design document providesdetailed analysis of the project overall strategy and its main components. The reviewof Project design document show that the main scope of problems lying in theframework of project proposal is based on target ADPs assessment reports as well ason social-economic and political situation analysis of transition Armenian society. Theoverall goal of the project is to contribute to the empowerment of citizens to influencegood governance, which should enhance civil society model and bring to povertyreduction in target communities. To contribute to the achievement of the goal thefollowing components were stressed within the project:

a) institutional framework enabling community participation and the delivery ofbasic services;b) adequate capacity of local authorities to accept and promote civil societyparticipation in decision making, implementing and monitoring processes;c) adequate capacity of community and civil society to participate in theseprocesses8.

The integration of project objectives into both ADPs overall design (for details,please, see Objective 3, p 26.), collaboration with appropriate government entitiesand local organizations and in depth economic and socio-demographic analysis of thearea, with a special emphasis on the youth and women, resulted in accurate andreasonable compilation of specific areas for intervention in design document.

7 Policy Influence through Citizen Empowerment: Project Designs Document, p 2

8Policy Influence through Citizen Empowerment: Project Designs Document, pp.1-2

16


17/54

Project Design Document paid special attention to the evaluation of voluntary andcommunity active groups activities implemented in target communities which servedas basis for Community Based Organizations Network models development andimplementation as part of the project design?. Group discussion results with CBO andCAG and ADP members involved in current evaluation exercise show thatestablishment of CBO network significantly reinforced CAGs operations, voluntaryinitiatives and overall community cohesion.

With regards to the stakeholders participation in design stage it should be mentionedthat all the questioned CAG, CBO as well as ADP representatives stated that they hadan opportunity to participate in discussions aimed at outlining possible programmaticdirections. Service providers participation in design process is relatively low. Themajority of questioned service providers stated that they have participated inimplementation stage and had rather passive role in design process.

Log frame review showed that in general it had SMART indicators which enabledevaluation team to overcome financial and time costs needed for measurement.Nevertheless, there are two main problems which, to some extent, limited evaluation

process:

1. No M&E plan was designed2. No baseline evaluation was conducted

The extend of assumptions effect on the program

To assess whether the planned outputs led to achievement of outcomes, the verticallogic of the log frame has been analyzed in the framework of the listed assumptionsand risks. The analysis of the rationale underlying causal linkages between theoutputs and outcome of the projects and the factors that may influence the program

or project beyond the management control of the project have been analyzed. Theeffects of outcome and goal level assumptions on the program have been discussedwith ADP/CBON, CAG staff. Discussions show that the majority of underlyingassumptions included in aforementioned levels of log frame has been proven as truewhich ensured output and outcome achievement.

Goal: Contribute to the empowerment of citizens to influence goodgovernanceAssumption: Local Self governance representatives are open for civil society

According to quantitative survey results conducted among village mayors the

majority of respondents (70.5%) highlight the role of citizens participation in makingpublic services accessible.

Total of 80% of village mayors state that community priorities are defined throughjoint meetings with community residents. Moreover qualitative interviews also showthat village mayors have positive attitude towards citizens initiatives promoted in thescope of current project particularly stressing the importance of such programmaticoutcomes as increase of social cohesion, development of community potential andactive participation of citizens not only in statement of priorities but in seekingsolutions.

Group discussions with CBON managers and members show that due to the fact of

organized trainings they possess information which allows them to influence locallevel decision making. Both Alaverdi and Stepanavan CBO members think that theyhave constructive relationships with LG representatives and via systematic

17


18/54

discussions are able promoting community priorities revealed through CVAmethodology into LG agenda.

Village mayors are well aware about us; they cooperate with us and in some casesconsult with us

Extracts from FG CBON staff (Stepanavan)

Nevertheless 2010 first quarter monitoring visits conducted by WVA DME M&E officerrevealed some gaps in cooperation between CBON and LG bodies in Stepanavan.During the reporting period CBO officially approached the mayors for the participationin budget planning, but the letter was not responded. Later, it turned out that therewas misinterpretation of the letter sent by CBO: it was supposed by mayors, thatCBO/ADP wanted to have delegation to intervene in budget planning procedure.

Together with ADP the CBO decided to meet with mayors during budget revisionprocess to be able to see the community development plan and the budget to beassured that the issues discussed have been reflected in both plans and tounderstand what can be the ADP's intentional interference for the given community.

The evaluation process revealed some issues regarding abilities of CBON to influencepolicy making in higher level than communities. CBON representatives state that theirorganization is still in early development stage and as yet has no appropriatepotential, including cooperation experience, fundraising abilities, to influence higherlevel policy and decision making. Specific cases of cooperation have taken placeunder the umbrella of WV Armenia. Interviews show that Stepanavan CBON has lesserlevel of cooperation with Marz level state structures compared to Alaverdi. Though itshould be mentioned that in case of CBON-Authorities cooperation Alaverdi ADP hadmentoring role which is a positive phenomenon when it comes to supporting newestablished organizations but on the other hand that kind of cooperation is mostlyinitiated by the ADP leaving CBON in a passive role. Mentoring role could be valuable

in the initial stages but from the perspective of CVA ideology and sustainability CBONshould act as independent actor in the field.

Outcome 1 Increased civil society mobilization and engagement inlocal/regional level development processesAssumption: Local Self governance representatives are open for civil society

Please, see previous section.

Output 1.1: CBO Network (CBON) promoted in targeted communities

Assumption: Stable legislation on civil society.

The only major change registered in the legislation regulating the Civil Society actorsis the amendment of the law on NGOs. According to the amendment the NGO havingannual budget exceeding one million drams shall publish information on their funds,expenditures, description and geography of implemented projects etc. which cannothave significant impact within the project scope.

Output 2.2 LG - CBON participatory community development planningcontributing to the child well beingAssumption: Willingness of LG to involve CBOs in community development planning

In the course of the entire process there has been no formal engagement of CBONswith Community Development Plans. However, both in Alaverdi and Stepanavan

18


19/54

CBONs representatives have participated in number of Community Council meetingsand maintained regular communication with their representatives.

Also, participatory planning and budgeting was one of the topics covered during thecapacity building training for LGs held in April 2009, where the LG representativesand CBONs discussed the opportunities of engagement in the process of developmentof Community Development Plans. As it was mentioned by CBON and ADP staff duringthe training, representatives of LG agreed that CBON engagement in that processwould be generally beneficial for their communities.9

CAGs and CBONs will cooperate increasingly with regional authorities. Action Planswill become fully integrated with Community Development Plans and aligned withregional processes (i.e. fully institutionalised). WV will also explore the incorporationof other stakeholders such as local businesses in future

Extracts from KII with Alaverdi ADP Manager

Outcome 3: Improved Public Service Delivery at Local Level through

Community EmpowermentAssumption: Public Service providers are open to dialogue

CVA, which was applied in the project communities throughout the past 3 years, hasenabled community members to raise important issues and find solutions for many ofthem. CVA has proven to be an effective tool for solving community issues locally. FGdiscussions with community social groups show that education, health and municipalservice providers are open enough for joint meetings with residents aiming atprioritization of community problems and outlining solutions. The interviews withservice providers also indicate such willingness and ardour. Moreover all interviewedservice providers from all spheres in the communities have been involved indiscussions and design of joint action plans throughout the project.

Output 3.3 Increased influence on local level policy and decision makingprocessesAssumption: LGs and Service providers are open to public monitoring

Please, see previous section.

Major changes identified as a result of the project

Survey results show that the vast majority of target social groups value the end

results of the programmatic activities rather than the process. In other words,different social groups were more desirable to see final project results (mainlyimproved service delivery) and were less interested in the process of how thoseresults would happen. During FGDs when residents were asked to distinguish themain problems they faced three years ago, along with infrastructure problems suchas drinking water or poor condition of school buildings they have been stressing thefactors like lack of information or weak mobilization of community hindering civilsociety development. When asking to describe the main programmatic achievements

9 It must be mentioned that due to September 8 of 10 Alaverdi communities have updated 4

year community development plans according to CVA action plans

19


20/54

throughout these years the residents basically concentrate on actual improvementsregistered in this or that service and mention changes in the sphere of socialsolidarity, civil dialogue, etc as secondary changes.

The vast majority of survey participants belive that the main changes takenplace due to the application of CVA methodology refer to local level projectsin education, health and municipal servic spheres.

The increased level of accountability of service providers is the nextimportant change noted by survey participants.

It is no longer required to wait for hours to receive an information or document fromvillage municipality. Municipality staff already works until 17:00 p.m. since they knowthat people are already demanding.

Extract from FG with Haghpat community members (the poor)

It should be stressed that one of the targeted and achieved results of the project isincreased awareness on state standards and obligations existing in the local level

social services which has crucial role in conducting community based monitoring ofabovementioned services. Yet some CBO and ADP representatives during interviewssuggested considering differentiated dissemination of various state entitlements.Some of the interviewees mentioned that not all entitlements could be used toevaluate the quality of accessibility of certain social services on rural areas.

For example the standard provides that the classroom should be lightened fromappropriate angle or tables and chairs should be situated in fixed distance from eachother, etc. There are standards according to which we shall completely suspend theclasses or terminate the services

Extract from FG with CBON staff (Stepanavan)

The growth of intensity of social contacts is mentioned amongthe key identified changes. As stated by many residents the communitymeetings promote both the communication between the residentsthemselves and service providers and those who receive the service whichactually eases the solution or clarifies many problems at service level.

I always wanted to meet with school headmaster but it was not easy laterparticipated in community meeting and could discuss with him my childs problemsthere in any case it is an opportunity for people to gather and discuss different

issues. Some people think that rural people are not like urban ones and communicatemore actively with each other, but things changed and now each person lives alonewith its family problems.

Extract from FG with Vardablur community members (women)

Though many survey participants prove that community participationinitiatives promote social trust, it should be mentioned that social trust inrural environment remain conditioned by individual interactions. In otherwords, confidence has emphasized personalised nature and confidencetowards any social service means confidence towards the person who

provides that service. Thus while discussing the overall poor quality ofgiven service the FG participants were trying to avoid pointing exact issuesand presenting the service staff personal qualities in positive light.

20


21/54

There are many medicaments that our health post does not posess and we must getthem from Stepanavan or Yerevan, but our doctor is a very good person and alwayshelps us ...

Extract from FG with Pushkino community members (pupils)

Prevalence of positive attitudes towards future and increased socialcohesion are seen as important derivatives of CVA initiatives.

Village residents participated and will participate in improvement of daycare centerservices. Peoples participation is increased, people use the slogan not mine buteveryones

Extract from FG with Arevatsag community education service provider(kindergarten)

To sum up the social perceptions on impact of CVA application one can

conclude that it is about changes aimed at solving concrete problems andstrengthening social capital (aggregate of social trust and mutual assistancenorms) which certainly has crucial role in shaping community empowermentand development perspectives.

21


22/54

Evaluation objective 2Project effectiveness (to what extent the project objectives have been

achieved and how, including revealing the possible unintended outcomes)

Under evaluation objective 2 the following questions are discussed.

To what extent has the project been effective in achieving the objectives andoutcomes as stated in the strategy?

How did the community benefit from the project?

How practical and doable is CV&A methodology?

Project effectiveness in achieving strategy objectives and outcomes

In this section the evaluation report analyses the level of achievement of the projectgoal and outcome level indicators to show to what extent the project was effective inachieving objectives and outcomes stated in Project log frame.

Goal: Contribute to the empowerment of citizens to influence goodgovernanceIndicators:*At least 50% of surveyed local self governmant bodies of targetcommunities state increased role of civil society in community decisionmaking processes

The vast majority of village mayors say that before project implementationcommunity priorities were set based on opinions heard during sporadically organizedmeetings with the participation of community active group members and serviceproviders. Community participation in local level community underwent significantchanges after application of CVA methodology. As a result, 63% (15 out of 24communities of Alaverdi and Stepanavan) of surveyed community mayors stateincreased role of civil society in community decision making processes.

* The CBO Network states its capacity and access to information sufficientfor influencing decision making and policy changes.According to survey results CBO Network rates its capacity and access to informationsufficient for influencing decision making. The majority of CBON members in bothtarget communities stressed that they need more capacities to be involved in policymaking and changing processes. (For more details, please, see Objective 1, p 6)

Outcome 1. Increased civil society mobilization and engagement in

local/regional level development processesIndicator: # of community development issues raised and addressed withactive participation of civil society actorsKIIs and FGs with community members, ADP, CBON staff as well as Stepanavan andAlaverdi ADPs CVA databases reviews showed the following statistics regarding theratio of raised and solved community issues.

22

ADP Totalnumber ofidentifiedissueswith

CVAtechnique

Totalnumberofissuessolve

d

Number of issues solvedwith communityparticipation / %out of totalnumber of identified issues

Number of issues solvedwith communityparticipation and externalorganizations (includingWV) % out of total numberof identified issues

Number of issuesaddressed onlyby WV / % outof total numberof identifiedissues

Alaverdi 355 177 38 / 21.4% 98 / 55.3% 41 / 23.1%

Stepanavan

276 92 12 / 13% 5 / 5.4% 75 / 81.5%

Total 631 269 50 / 18.5% 103 / 38.2% 116 / 43.1%


23/54

Hence, end of project value of the above mentioned indicator is 114.

Outcome 2. Effective relationships between the LG, CBOs, communities andother development actors operating in the project sites ensured for furtherpromotion of local self-government decentralizationIndicator: # of CBON staff, community members participating at LG

operations (e.g. community strategy development, budget hearings, etc.)Alaverdi and Stepanavan ADPs have been actively facilitating dialogue andcommunication between the LGs of 30 project communities and the communitymembers. ADPs have been working closely with LGs through regular meetings toidentify the need of LGs for capacity building and tools necessary for LG staff toaddress problems identified by communities. In order to ensure engagement ofCBONs in community development planning processes Alaverdi and Stepanavan ADPshave been working with heads of LGs to highlight the importance of communitydevelopment plans and the role of community participation in those processes.

As per Project Annual reports about 40 member of CBON staff and 60community members participated at LG operations.

Outcome 3: Improved Public Service Delivery at Local Level throughCommunity EmpowermentIndicator: # of public services delivered to community stated as improvedby surveyed community membersCVA, which was applied in the project communities, has enabled communitymembers to raise important issues and find solutions to many of them. CVA hasproven to be an effective tool for solving community issues locally. Overall, 269issues out of the raised 631 were resolved in the project communities. This data istaken from CVA database review.

Qualitative research tools were used in order to identify community residentsopinions and perceptions over services improvement. Number of interviewedcommunity members is around 270 (4 FG per community with 8-10 participants). Thenumber of solved issues listed by FGs participants is about 24.24 of public services delivered to community stated as improved bysurveyed community members.

Social services delivery and quality assessment by surveyedcommunities

While referring to the issue of the CVA project efficiency, the analysis of the situationwith the quality of the services, their impact on community welfare should be taken

23


24/54

into consideration. Since the communities subject to evaluation have been selectedbased on the level of the CVA application effectiveness, the assessment of servicesshould be presented separately for each community indicating existing similaritiesand differences.

1. The healthcare and education services have been indicated as vital forcommunity life by participants in all communities and received the followingrates: healthcare and education services. In Stepanavan ADP three targetcommunities rated significantly high the healthcare service operationseffectiveness (4.7) while in the case of Alaverdi ADP communities theeducation services have been considered as the highest quality services (4-4.5points mean). The rate of education services in Stepanavan ADP communitiesvary from 3 to 3.5 points which according to survey participants is conditionedby expensiveness or lack of textbooks, scarcity of laboratory equipment,infrastructure for sport events and in some exceptional cases by absence ofquality specialists. The assessment of Alaverdi region health care servicesresulted in almost the same rating points as in Stepanavans education casewhich in its turn is based on personal negative experience.

Figure 1. Social services participatory assessment scores10

(Arevatsag, Alaverdi ADP)11

Figure 2. Social services participatory assessment scores (Vardablur,Stepanavan ADP)

10Figures 1-6 represent data on community services assessment by community target groups. 5-

point scale was used to identify the level of services accessibility, where 1 means absolutely not

accessible and 5 means absolutely accessible.11 The blue line of non-participants cannot be observed on the graph since it repeats the values of the

poor presented in lilac.

24


25/54

Figure 3. Social services participatory assessment scores (Qarinj,Alaverdi ADP)

1. As for the assessment of NGOs the only organization mentioned by target

groups is the World Vision. Political parties rating points are mainly situated in0-1 range which according to target groups is conditioned by virtual abscenseof any influence on community life from their side. Meanwhile poor contingentevaluated the political parties relatively higher than other groups. Thephenomenon stems from the fact that the operations of parties are limited tovarious donations for the poor population during preelections.

2. There are a number of differences between Stepanavan and Alaverdi targetcommunities residents perceptions towards central and self-governmentbodies. Particularly central government body (Marzpetaran) received 2.5 points(mean) by Alaverdi residents while Stepanavan residents evaluated theactivities of the same instance with 1.6 points (mean). The same trend is seenin evaluation of local self government bodies. Alaverdi residents evaluated

their municipalities with 4.25 points while in Stepanavan communities it was3.7. According to the survey results there is correlation between the fact ofparticipation in community meetings and evaluation of municipal services.Thus the groups of people who ever participated in communitymeetings evaluate municipal services higher compared to those whonever participated in any meeting (Nonparticipants: Stepanavan-1.3Alaverdi-2, Participants Stepanavan-2 Alaverdi-3). Existing socialdispositions towards LG bodies are in rather high levels of tolerance scale.Even putting lower grades, people want to emphasize that their village officialsdo their best. Moreover any initiative of LG officials are perceived andinterpreted by people as very important. Compared to other services centralgovernments activities are evaluated with the lowest points. Surveyparticipants condition the fact by such factors as minimum presence ofMarzpetaran officials in community life, tendency to ignore local level issuesand unawareness of residents on Marzpetaran activities

We have seen the governor only once when they need us they come andcall a meeting but when we need them we never see them .... we have appliedto support us with kindergarten with no results

Extract from FG with Vardablur community members (Women)

Service assessment results of the womens and nonparticipants groups inthree target communities of Stapanavan ADP are mainly similar. Whileaccording to Project DD one of the main objectives of the project is to promote

25


26/54

participation of women, however the evaluation could not answer the questionregarding the level of actual participation of women in project implementationto the full extent. The reason is that the qualitative findings received contradictto the quantitative data not allowing for further analysis without an additionalfield exercise with a specific focus and design.

Figure 4. Social services participatory assessment scores (Pushkino,Stepanavan ADP)12

Figure 5. Social services participatory assessment scores (Agarak,Stepanavan ADP)

Assessment results of the poors and nonparticipants in Alaverdi communitiesalso correspond to each other which in its turn may mean that nonparticipant

groups are mainly consisted by poor segment of the population.

Figure 6.Social services participatory assessment scores (Haghpat,Alaverdi ADP)

12 The blue line of non-participants cannot be observed on the graph since it repeats the

values of the poor presented in lilac.

26


27/54

Overall nonparticipant group members tend to put lower points while assessingcommunity services compared to those who participated in this or thatcommunity meeting. This can bring to conclusion that the higher is theawareness level on state standards or obligations regardingcommunity services, the higher is the level of constructive criticism

approach while evaluating social services. If nonparticipants conditiontheir lower assessment points by negative personal experience (for exampleone applied to mayor for job but rejected) the participants assess communityservices taking into consideration qualitative aspects of the service andmaterial features.

I would assess physical conditions of the school with 4 points since we lacksport hall and appropriate laboratories but the quality should be assessed byhighest point 5 since our school has high quality specialists

Extract from FG with Agark community members (Women)

First of all, it should be mentiond that according to survey results there was somemisunderstanding among all interviewed social groups about the format ofcommunity meetings and discussions. At the beginning of interviews social grouprepresentatives mixed the community meetings organized in the scope of CVA projectwith other similar meetings (in some cases organized by other organizations). Itseems like the series of CVA discussions are not considered as separate and moreimportant ones than others. Referring the peculiarities of CVA application in Alaverdiand Stepanavan communities, one can say that Alaverdi communities are moreoptimistic towards the process and the participation level is comparatively higher.

During the evaluation results analysis it turned out that there are several informationsources through we community residents learned of CVA discussion.1. Community people get information on upcoming meeting from ADP staff during

recurrent site visits. No single interviewee pointed any local stakeholder of theprocess as responsible for convening meeting or conducting discussion. WVA isseen as main responsible of the process. The issue is further discussed inObjective 5 of this report (p. 31).

2. Some FG participants told that they have seen announcements about themeeting in their municipalities or municipality staff informed them about themeeting.

3. Informing the community members through school administration is the mostwidespread method of informing residents about meetings. This method can be

considered as effective since it ensures participation of wide scope ofstakeholders in discussions about educational issues but the participantsselection process in some cases has been distorted.

27


28/54

No one tells us that we have right to participate in that discussion. Teacher orheadmaster comes and says You, you and you go to that meeting

Extract from FG with Pushkino community members (pupils)

The success of any initiative on participation is conditioned by the target groupsperception and interpretation of the roles of key components. According to surveyresults the vast majority of target community residents think that civic participationprocess is limited to participation in meetings to define priorities and find solutions.Surveyed community residents and service providers opinions over CVA objectivesare combined in the following two main types:

1. Community based- problem oriented: the vast majority of interviewedresidents think that CVA is designed to help people identify priorities and solvethem through participatory mechanisms.

2. Community based- mobilizing: some part of interviewees think that CVA isto raise community mobilization and make decisions jointly.

The majority of surveyed project implementers stated that CVA methodology is quitepractical and doable. Nevertheless, there are several problems relating to citizenmotivation to participate in community gatherings, meetings

There are two sides of the problem. On the one hand, people often complain thatdecisions have been forced upon them by the local authorities or that they have notbeen consulted in planning at local government level. On the other hand, there areevidences to suggest that on many occasions people do not participate actively inlocal planning even if they have been given the opportunity to do so. A number offactors are important in determining whether people really want to participate inplanning. It seems that participation in local planning is usually restricted to thosecitizens who feel directly threatened or affected by the local government orpersonally involved in its endeavours. It also seems that people are unlikely toparticipate willingly in planning if they feel that their participation will have nosignificant effect on the final outcome. People are also reluctant to participate in localactivities in which they have no particular interest, or which are unlikely to affectthem directly. People tend to participate only in planning when there is some specificinterest for them in its outcome.

Another factor is ignorance. It seems that the average citizen has very littleknowledge of the range of options (and of the implications of these options) open tohim or her in terms of forums which could be used to express his or her views anddesires. A substantial number of citizens therefore do not avail themselves of these

opportunities to shape policy directly because of inertia or indifference.

The third factor is the communication problems. These problems amount to practicaldifficulties such as language problems, differences in attitudes and expectations, andmutual feelings of mistrust, suspicion or resentment.Another issue related to CVA methodology practicality refers to the CVA database. Asit was stated by the managers of Stepanavan and Alaverdi ADPs, the CV&A databaseis less operational and more time consuming exercise. It was suggested to revise theformat of the base and make it more applicable.

28


29/54

Evaluation objective 3Integration of the project with the overall ADP design considering the ways

in which the project has linked with ADP designs and ADP programmingpractice

Under evaluation objective 3 the following questions are discussed

How the project supplements the overall ADP design?

Did the project objectives contradict with/challenge the overall ADP design?

Did the project objectives overlap with the overall ADP design?

Projects linkage with ADP overall designSurvey results show that there are several links to which the project supplements theoverall ADP designs.

Community empowerment and citizen empowerment is one of the main strategicobjectives of WV Armenia strategy to ensure sustainability of projects and programsas well as part of their transition to communities. This approach is applied in all WVArmenia ADPs, including Stepanavan and Alaverdi ADP. The CBPM tool is a cross-cutting for all projects and is integrated in all other projects of the ADP (Education,Health, Economic development).

Collaboration with CBON is not limited by the scopes of the proposed project. On bothStepanavan and Alaverdi CBONs are the pone of the main partners of the ADP ingeneral, and the sponsorship management project in particular. CBPM as one ofmechanisms of social accountability is being widely used by Stepanavan ADPoperated in target communities. CBPM application was also widely applied during thedesign phase of Alaverdi ADP. Facilitation of establishment of a CBO Network, theircapacity building along with capacity building of other role players, including localauthorities through the proposed project added value to the effectiveness of thementioned ADP community development projects aimed at reduction of poverty intargeted communities.

The ADPs supplement the project by the long term presence of WV Armenia intargeted ensuring continuity of efforts to build further the dialogue between allcommunity stakeholders.

In both Alaverdi and Stepanavan ADPs the project contributes to the Civil Society

project mainly, by empowering citizens to be able to influence the decision makingand, if needed, policy changes in their communities.

According to interviewees, CVA definitely helps and complements the project sectoralgoals.No significant contradictions and overlaps were identified with the overall ADPsdesign. Moreover, each year target ADPs amend the programs DIP to address issuesincluded in local Action Plans.

in FGDs with service users we cover training on entitlements and encouragediscussion of decisions. People reflect on the quality of public services (in schools andkindergartens for example). They dont always have these opportunities with

conventional WV service delivery programming approaches

Good facilitators + CVA = Good results. CVA may not be as relevant in its current

29


30/54

form in urban settings as communities are harder to distinguish

Extracts from KII with Alaverdi ADP manager

Evaluation objective 4The impact of the project model on the level of community participation

(comparing CAG13 and CVA models)

Under evaluation objective 4 the following questions are discussed:

How the project did enhance civic participation in decision making process?

What barriers were there to participation and how were these addressed?

What forms of community participation have been increased and in whatsectors of social activity the level of civic participation has especially increasedas the result of the project?

What mechanisms of social accountability were established through theproject?

What was the socio-demographic distribution of people who took part in theproject? (gender and age equality in community participation)

Community participation in decision making process.

Civic participation is one of the principal means by which individuals exercise theirempowerment for the benefit of the locality, often at the same time increasing theirown level of empowerment. It helps to make civic institutions more representative ofand accountable to the local population. It also contributes to concentrating localdecision making more effectively on the issues all members of community believe areimportant, as well as strengthening ties between such bodies and the people they

serve, so building trust.

Survey results show that community involvement in decision-making was promotedthrough CVA so that people know what should be done, by whom and when.

People have an opportunity to gather and discuss priorities in the past we used togather from event to event but now it is done in a more coordinated way

Extract from KII with Community council (Metz Ayrum, Alaverdi)

A show that arguments in favour of enhancing citizen participation frequently focus

on the benefits of the process itself, taking into consideration the point that citizenparticipation is a transformative tool to social change. Nevertheless, there are suchopinions that citizen involvement is intended to produce better decisions and thusmore efficiency in benefits to the rest of community. Hence, we can conclude thatthere are two types of benefits and advantages of citizen participation

13 CAG, Civic Action Groups, are the community engagement/mobilization model applied by

WV Armenia ADPs prior to application of the CVA model and currently utilized in some of the

ADPs.

30


31/54

Table 6. Community participation: advantages for citizens and LG14

Advantages for citizen:Process:

Educated citizens: informed and involved citizens to some extentbecame expert understating technical difficulties of decision makingprocess as well as making community wide solutions.

Persuade and enlighten LGs: informed and involved citizens havelarge possibilities to raise themost important and urgent issues forcommunity and persuade LGs to do certain steps to solve issues.Thus, community activists may have regular contact with key decisionmakers and can persuasively convey their viewpoints.

Gained skills for activist citizenship: due to participation practicescommunity members became more active and can make furthercollaborations to create civil society institutions.

Outcomes:

Priority based decision making: citizen participation ensure theprocess of making local level decisions based on community prioritiesand not on local authorities capriciousness or self-will.

Community based monitoring: citizen participation helpsimplementing monitoring of services and policies.

Advantages for LG:Process

Feedback mechanisms: through regular contact with communityresidents LG representatives learn which activities/projects/policiesare likely to have negative feedback from community and how toavoid such failures.

Political persuasion: when the government truly collaborates withcitizens or when it works to win over citizen sentiment, a key

assumption of successful political suasion is the social influence ofcitizen participants.

Outcomes:

Better policy and implementation decisions: by ensuring citizenparticipation in decision making process government representativescreates bases to elaborate effective policies and make decisions:

Decision making legitimacy: the most important outcome especiallyfor Armenian reality is legitimate political decisions. When communityhave an opportunity to participate in decision making process andwhen their priorities are reflected in policies, legitimate perception of

authorities will be more expended.

Socio-demographic picture of community participation modes

Referring to the analysis of CVA participants socio-demographic characteristics thefollowing should be taken into consideration. As noted both by CVA processparticipant residents and organizers /CBON, ADP/ almost in all communities there aregroups of people always participating in such discussions and groups of peopleavoiding any community initiatives in vast majority of cases.

14 The information mentioned in Table 6, was incorporated on the bases of ADP staff, CBON

staff, CAG members as well as some LG representatives ideas, opinions and perceptions.

31


32/54

The group behaviour is conditioned by its members perceptions. Those who do notparticipate are led by non-intervention strategy. Based on FGDs with ADP staff aswell as with community residents Non-intervention strategy can be explained bythe following reasons:

- Low wellbeing: Tha vast majority of interviewees believe that as a rulevulnerable groups such as poor and lonely women do not participate incommunity level initiatives.

- Need to address basic needs: FG discussions with non participants showthat these people need to meet their basic needs on daily basis and do notwant to waste their time participating in such discussions.

- Low intensivity of social interactions, limited social capital: Thesefamilies are described with passive interactions with other families and aremainly isolated from social life.

- Lack of awareness: Some FG participants noted that these families do notparticipate due to lack of information on upcoming events.

- Indifference towards community initiatives: These groups are mainlyindifferent towards community initiatives which may be explained by low level

of trust towards latters.- Intermediate, nondifferentiated position: Interviewes with the

representatives of nonparticipants groups show that these people as a rule donot have exact requirement to quality of this or that service and do not see theway they influence any development in their community.

-Figure 7. Causal linkage between factors of non-participation social

tactic

To raise the effectiveness of CVA process it is expedient to make the processmaximise inclusion particularly ensuring involvement of vulnerable groups.

Youngsters and middle-age groups representatives are actively involved indiscussion. The picture is a bit contradictory from gender perspective. As it presentedin Evaluation objective 2 the opinions of nonparticipants and women are similar.According to part of FG participants women are participating in discussions moreactively.

Women have more free time thus they participate more actively men cannotleave their job for such a meetings

32


33/54

Extract from FG with Vardablur community members (the poor)

Another part of FG participants think that women participation is limited due to socio-cultural stereotypes on women behavioural norms. The data collected throughvarious evaluation tools on the level of women participation in the communitydiscussions is rather contradictory. This fact limits the ability to analyze it without

additional field exercise with specific focus on the issue. However, at this pointadditional field exercise is deemed not cost-effective. This issue will be addressedthrough the next project cycle (to be funded through the PPA funding) with genderdisaggregated indicators.

Almost all the beneficiary group representatives stated that as a rule the same peopleparticipate in all discussions (predominantly representatives of school staff andstudent councils, and community proactive residents who participate in every event).

Survey participants pointed some issues hindering the effective application of theCVA tool to support civic participation. Accordingly they suggested some changes inthis respect.

Limitations Suggestions

The frequency of meetings is limited andnot all the people have opportunity toparticipate

Increase the frequency of meetings

Nearly the same people participate in alldiscussions

Promote participation of all people andmake the process more transparent15

Some documents designed duringdiscussions have a form of wishlist withno practical implications

Emphasize the practicality of decisionsmade

Unawareness of community budget

possibilities hinder the process of makingrealistic suggestions

Make the process of community

budgeting transparent and moreparticipatory

The following types of participation, varying from one another by intensively andimpact have been established due to implementation of CVA methodology.

1. Community contribution to the development effort, i.e. to theimplementation of the decisions.

In-kind contribution of beneficiaries - local residents contribute their laborin implementation process, along with an assistance of external agency.

Besides WV is seen as a main external assistant agency particularly when itcomes to financial assistance. A financial contribution of the community - A lot of projects have been

implemented with the financial contribution of LG bodies.It should be mentioned that these forms of participation are not appliedseparately. In vast majority of cases implementation of project impliescommunity residents labour force partial financial contribution from LG bodiesand assistance from abroad in terms of funds and assets.

15

For example, Alaverdi ADP manager suggested to put certain criteria of participation, e.g. ineach of new community discussion 40% of FG participants should be previously non

participants.

33


34/54

2. Community-based monitoring: The monitoring of social services also can beconsidered as a form of participation embedded by CVA methodology. Theawareness on state standards on social services is of major importance to sustainthis form of participation.

Social accountability mechanism established through the project

Essential to the well-being of all people is the effective delivery of basic services suchas health, education, water and sanitation. Yet, widespread evidence shows thatservices are failing poor people in a large number of countries with negative impactson human development outcomes. In addressing the failure of services, one keypoint is that the failure of services is not just technical, it is the result of the lack ofaccountability of public, private and non-profit organizations to poor people.

All interviewed project implementers stressed the point that CVA is one of the mosteffective tools in which community participation processes and mechanisms canstrengthen accountability and also affect service delivery outcomes. Citizens canexert their collective voice to influence policy, strategies and expenditure priorities at

local level according to community preferences.

According to interviewed target groups within CVA technique the following model ofsocial accountability was established. This model could be mentioned as short routeof accountability that includes 2 groups of stakeholders: (i) citizens, as clients,directly influence service providers (ii) to improve the quality and accessibility ofdifferent services. As the result citizens became more empowered and servicedelivery is improved.

In terms of the long and short routes, depending on the context, different routes willbe more or less amenable to various forms of community participation. The longroute includes electoral politics, citizen oversight boards, use of media, social

accountability movements that bring together citizen direct action with more formalgovernment mechanisms. The long route is more difficult for communityorganizations, without some types of enabling environment and functional publicinstitutions that can incorporate or engage with community initiatives. In eachcontext, the state/society nexus shapes the room for access on the long route.External Non-Governmental Organizations, especially international NGOs, are not theshort route, but mechanisms for supporting either the short or long route. The shortroute includes direct provision of services, community monitoring and managementof services, contracting with service providers, and projects funded by external NGOsbut managed locally.In relation to service delivery, there are a range of issues related to the role ofcommunity participation and stakeholder involvement in service provision. The short

route of accountability provides for direct community action, both through communityprovision of services and through communities holding providers accountable at thepoint of service delivery.

The evaluation research results show that there are several groups of perceptionsover the role and possible impact of civic participation on social servicesaccessibility.

1. Some of interviewed service providers as well as ADP and CBONrepresentatives think that civic participation and community-basedinterventions have the potential to make service providers be more responsiveto the needs and priorities of beneficiaries (this could be called allocativeefficiency).

2. Other interviewed evaluation research participants are sure that civicparticipation provides designing of community-based projects that are

34


35/54

comparatively cost effective (productive efficiency) because of lower levels ofbureaucracy and better knowledge of local costs. From another point of view,almost all research participants agree that because of the lack in knowledge oflocal budgetary funds the joint action plans that community participants comeup with wish lists rather than realistic and focused steps.

ADP/CBO managers and staffs as well as service providers and LGrepresentatives mentioned several suggestions to provide improvedaccountability:

- overcoming information asymmetry- service providers possess in-depthknowledge on the subject matter compared to ordinary citizens and as a resultduring discussions they easily promote their agenda;

- providing communities with information on service quality through variousforms of Monitoring and Evaluation;

- improving various dimensions of allocative efficiency includes increasedtransparency on budgets and public resources through such mechanisms ascommunity hall meetings, public budgeting.

35


36/54

Evaluation objective 5Sustainability of the project model

Under evaluation objective 5 the following questions are discussed:

What elements of CV&A can be applied as a general model in all ADPs?

To what extent has the project established/enhanced capacity, processes andsystems that are likely to be sustained?

What capabilities does the ADP/CBO staff have for facilitating furtherapplication of CV&A?

What elements of CV&A can be applied as a general model in all ADPs?

To reveal what elements of CVA process can be replicated by other ADPs it isnecessary to refer to activities of experimental groups, i.e. Tavush ADP areascommunity active groups.

According to interviews with different active group members, CAGs in Tavush applydifferent mechanisms for identification and prioritization of community problems.

- CAG and community council cooperation. According to Tavush regionsCAG members their main aim is to reveal community problems, vulnerablegroups and cooperate with community councils for solutions. Some CAGs havecommunity council members in their group. Other CAG members used toparticipate in community councils meetings. Before some CAGs gatheredinformation on community problems through personal inquiries or request sentby individuals.

Currently these CAGs also convene meetings but more sporadically compared to CVAcommunity CAGs. Community priorities are set by CAG members and sent tocommunity councils for discussions.

We try to meet the people in all districts and ask them about their problems. Later

we prioritise issues based on the spread of the issueExtract from FG with CAG staff (Aygehovit, Tavsuh ADP)

- Revealing of problems through social network. Ditavan CAG membersthink that communities are small enough and information sharing is intense sothey easily get information.

CAG group member live in different districts and hear about existing problems indaily basis. Community is small and people are connected to each other

Extract from FG with CAG staff (Ditavan, Tavsuh ADP)

36

We have director of post office, teacher, and veterinarian in our group. We convene

meetings and inform our villagers by announcements

Extract from FG with CAG staff (Aygehovit, Tavsuh ADP)


37/54

CAG-Community council - Village mayor cooperation. Tovuz CAG members toldthat they provide information on problems to community council which laterdiscussed with Community Mayor. In this case CAG acts as support unit to LGinforming on recurrent community issues prioritized by non formal votingmechanisms.

Observation of aforementioned opinions let us to conclude the following:

- experimental group i.e. CAGs have mediated

Documents

Clean - Final Evaluation Report - Policy Influence Through Community Empowerment Project