EUROPE AND CENTRAL ASIA - duinareyes.comduinareyes.com/Duina_Reyes_-_Web_&_Graphic_Designer/Sample_…ECA Regional Results Framework for Youth Development Projects: Final Results May

THE WORLD BANK

ECA Regional Results Framework For Youth Development Projects: Final Results May 2013Social Development

EUROPE AND CENTRAL ASIA

This report has not undergone the review accorded to official World Bank publications. The findings, interpretations, and conclusions expressed herein are those of the authors and do not necessarily reflect the views of the International Bank for Reconstruction and Development/The World Bank and its affiliated organisations, or those of the Executive Directors of The World Bank or the governments they represent.

This report and regional data were compiled by Azra Nukic, under the guidance of Joanna P. de Berry task team leader for the work program on Strengthening the Monitoring and Evaluation of Youth Projects in ECA. The collection of regional data was supported by Sarah Michael, Lola Ibragimova, Bekim Ymeri and Valerie Morrica. This work program was funded by a knowledge grant under the State and Peace Building Fund of the World Bank, with additional funding received under the Social Development Initiative for South East Europe, funded by the Government of Italy.

ECA Regional Results Framework for Youth Development Projects: Final Results

May 2013

Social DevelopmentEUROPE AND CENTRAL ASIA

THE WORLD BANK

ECA REGIONAL RESULTS FRAMEWORK FOR YOUTH DEVELOPMENT PROJECTS: FINAL RESULTS 3

TAbLE OF CONTENTS

Summary .......................................................................................... 5

IntroductIon ..................................................................................... 6

methodology for development of regIonal reSultS framework ............... 7

overvIew of data reSultS ................................................................... 8

Results of the PDO Indicators: Improved Access to Economic Opportunities and Sustainable Youth Services............................. 9

Gender Aspects In Beneficiary Participation Trends .................................. 10

Results of The Outcome Level Indicators .................................................. 12

OUTCOME 1: Quality and Relevant Youth Services are

Available for Young People ................................................................ 12

OUTCOME 2: Youth in ECA have Increased Opportunities

to Develop Business and Employment Skills ...................................... 13

OUTCOME 3: National Youth Organizations and Support

Structures Have Increased Capacity to Implement Youth

Policies and Programming. ................................................................ 14

concluSIonS ................................................................................... 16

4


Between 2007 and 2012 the Social Develop-ment Unit of the World Bank working in the Eastern Europe and Central Asia region sup-

ported youth development projects in Albania, Azer-baijan, Kosovo, North Caucasus, Georgia and Tajiki-stan. Data aggregated from these projects show that the following results were achieved:

Project benefits reached more than 110,000 ■young people in the ECA region;

Across all projects, 48% of beneficiaries were vul- ■nerable youth;

The participation of girls in all project activities ■rose over the course of implementation to end at 46% of beneficiaries being female;

Where data exists, 19% of targeted youth said ■there were good opportunities for young people in their local area after the projects, as compared to 5% saying this before;

SUMMARY

83% of all beneficiaries reported being satisfied ■with the quality of youth services delivered un-der the projects;

Where data exists, 72% of targeted youth said ■they had used skills gained under the projects in their daily life;

751 young people received business develop- ■ment training under the projects;

73 Governmental staff of youth departments and ■Ministries received training under the projects.

This document gives further detail of the results achieved for young people under the projects. It also describes the process to gain a regional picture of these results by supporting the six projects to use the same monitoring indicators and adopt a regional re-sults framework.

6

Today, the Europe and Central Asia region (ECA) is a home to over 170 million young people liv-ing in a changing, technologically demanding

and economically challenging world. Smart invest-ments in the development of these youth are impera-tive for the future of the region. Managing positive social and economic change brought about through youth-focused development programs, and sustain-ing this change over time, requires commitment to managing for results. It also requires evidence-based understanding of what types of youth interventions work and why.

Recognizing the importance of strong Monitoring and Evaluation systems for results-based manage-ment, and in order to enhance the impact of invest-ments in youth development, the ECA Social De-velopment Team launched the “Strengthening of

Monitoring and Evaluation Approaches to Youth Development Initiative”. The initiative consisted of the development of a regional level results framework with common indicators that could measure success within individual country projects and compare re-sults regionally. In doing this, analysis is able to pro-vide better insight into what designs and strategies work best for current and future youth development projects in the region.

This final report presents the findings based on data collected from the regional level results framework of six youth development projects taking place in Geor-gia, Azerbaijan, Tajikistan, Kosovo, Albania and the North Caucasus. The data results provide recommen-dations and highlight challenges for the community of policy-makers and practitioners interested in en-hancing the effectiveness of youth programs.

INTRODUCTION


Since the inception of this initiative in FY10, the World Bank and Project Implementation Units (PIU) from Azerbaijan, Georgia, Kosovo, Ta-

jikistan North Caucasus and more recently Albania, have worked together to strengthen the Monitoring and Evaluation (M&E) capacity of staff and create and implement strong results frameworks based on a joint vision for World Bank funded youth develop-ment projects in the ECA. This initiative started with an in-depth review of the existing Results Frameworks of each project. Significant gaps were revealed in gen-eral M&E skills and data collection methods by many of the Project Implementation Units (PIU) in the existing monitoring and evaluation arrangements. As a result, all Results Frameworks were revised to bet-ter align it’s objectives with the project development objective (PDO) of each youth project and develop comprehensive Monitoring and Evaluation Systems. In addition, a central each Management Information System (MIS) was built and connected to individual project’s MISs to ensure streamlined data collection and reporting. Training needs were identified and addressed through a series of capacity building work-shops for M&E officers and project coordinators.

To support this effort, the initiative funded the or-ganization of the Regional Youth Development workshop which took place from June 9 - 12, 2010 in Istanbul, Turkey. The aim for the workshop was to create a regional vision for youth projects in the ECA region, identify ways to measure success and provide project teams with skills and practical tools to improve M&E of their projects. Specifically, the purpose of the workshop was to:

Identify a joint vision for all youth development ■projects in the ECA and inspire project teams to reach it;

Develop impact level regional framework for ■youth development projects;

Identify common indicators of success that will ■be measured across projects;

Increase M&E capacity of youth project teams. ■

As a result, a joint vision was developed together with an impact level Regional Results Framework (RRF) with common indicators of success that would be measured across the projects. The overall purpose of the RRF was to provide a common platform for data collection across all youth projects and show the scope and results of World Bank’s investment in youth in post-conflict societies in the ECA. The RRF was designed to give expression to the regional vi-sion developed during the Regional Youth Develop-ment Workshop through 18 defined indicators cor-responding to four common objective statements. It was structured to include the hierarchy of objectives (vertical logic) and Monitoring and Evaluation Plan and Reporting Framework (horizontal logic) in one document for easier reference and implementation. This set of common measurable results and corre-sponding indicators would be tracked across all ex-isting youth projects and future World Bank-funded youth projects in the ECA region, which would al-low evidence gathering about program impact across different countries and contributes toward improved quality and accountability.

METHODOLOGY FOR DEVELOPMENT OF REGIONAL

RESULTS FRAMEWORK

8

As a result of the intensive discussions between project teams1 the overarching project devel-opment objective of all youth focused proj-

ects in the report was identified during workshop as: Improved access to economic opportunities and sustain-able youth services in conflict affected countries in ECA. Three outcomes were identified:

Quality and relevant services are available for a. youth;

Youth in the ECA have increased opportunities b. to develop business and employment skills;

National Youth organizations and support struc-c. tures have increased capacity to implement youth policies and programming.

Data on the regional indicators of the six youth proj-ects was collected between January 2011 and March 2012. An interim report covered the period from January 1, 2011 to April 1, 2011 with this final report covering the remaining time period. The projects include: Georgia Youth Development and Inclusion Project, Azerbaijan Internally Displaced Persons Youth Support Project, North Caucasus Youth Empowerment and Security Project, Tajikistan Youth Social and Eco-nomic Opportunity Project, Kosovo Youth Development Project, Albania Youth Empowerment and Community Driven Development Project.

While these youth projects are not identical, they do have similar design features in at least some compo-nents of the programming: (a) an emphasis on the non-formal education activities/programs through the services provided by Youth Centers; (b) a range of activities aimed at promoting youth employment

1 The youth project in Albania was at the early stages of the implementation and none of the PIU members were present at the workshop

OVERVIEW OF DATA RESULTS

focuS of youth development projectS In eca:

Greater economic inclusion of young ■people, particularly those from vulner-able groups and remote regions through livelihood skills such as foreign languag-es, information technology, leadership, entrepreneurship, and business develop-ment.

Greater social inclusion through life – ■skills development, more active participa-tion in community life, and opportunities to lead violence-prevention and peace-building efforts.

Enhanced capacity of Youth Ministries, ■and Federal, Regional, and Local levels of government to better design and imple-ment effective youth programming.

through apprenticeships and entrepreneurial pro-grams; and (c) capacity-building support for insti-tutional structures, such as the Youth Ministries and Departments.

The process of data collection had its challenges. First, all six projects have different starting and clos-ing dates, which meant that while several activities were being implemented in some projects, the same activities were only at the initial stages in others. Sec-ond, the capacity of the PIU to integrate and collect data has been built over time and the quality and ac-curacy of the information varies between both coun-tries and reporting periods. Finally, the last three projects to close, those in Azerbaijan, Georgia and


Albania2 reported some data in an aggregated form between October 2011 and September 2012. This data describes general regional progress but is not useful at the individual country level.

Results of the PDO Indicators: Improved Access to Economic Opportunities and Sustainable Youth Services

Achievement of the PDO was measured against five indicators:

Indicator 1: Percentage of youth in project areas who have participated in project activities (per region).

Youth (aged 14-30) are considered to have partici-pated in project activities if they have used one or more of the project created facilities and services. The targets and achievements for this indicator vary considerably based on the size and density of the regions as well as the size. Therefore it is difficult to present a regional picture from project level data on this indicator.

Azerbaijan reached a high participation rate of 84% of youth in the project areas, followed by Tajikistan with 10.96% in targeted project areas, while the North Caucuses project reached 1.85% of all young people in the region. Two projects (Azerbaijan and N. Caucasus) reached their targets in the level of participation. But all projects show consistent incre-mental increase in participation levels as the projects progressed after an initial slow start in reaching youth beneficiaries. For example, in the aggregate reports for Georgia and Albania, the percentage of partici-pants increased from 2.1% to 3.3% from October 2011 to September 2012. The initial low rates of par-ticipation can be related to difficulties in construct-ing youth centers, causing delays in the initial im-plementation stage and constraining activities with direct benefits for youth participants for months or, in some cases, years. Tajikistan, for one, was not able to incorporate participants until the program was more than half over. The Tajikistan project was able to overcome the initial delay and nearly reach its tar-

2 The Albania Youth Empowerment and Community Driven Development Project is still active.

get in large part due to recruiting efforts, especially by modifying the projects to better fit the needs of the participants (such as reducing barriers to female participation which will be addressed later). In retro-spect, program staff recommended that sequencing of events be taken into careful consideration to be sure that youth participation was incorporated from program start.

The North Caucuses program staff and youth par-ticipants attributed its success in attracting partici-pants to the centers’ roles as places for recreation and leisure as much as places for skill-building. Because of the lack of opportunity and general recreation, which is pervasive throughout many of the areas where these youth development projects took place, youth are often looking for a place to go to or some-thing constructive to do. Youth were attracted to the centers as places to hang out and make friends and engage in activities following that. Where possible, this strategy should be followed to increase social op-portunities at the centers and attract a larger number of participants.

Indicator 2: Percentage of youth in project areas who report an increase in opportunities for young people in their region.

Reporting was inconsistent for this indicator as the indicator was measured only at the end of the project cycle making monitoring of data collection difficult. However, those projects that managed to collect data saw a rise in the percentage of youth reporting an in-crease in opportunities from baseline to final report. Aggregated data from Albania, Georgia and Azerbai-jan indicate an increase from 5% in the fall of 2011 to 19% in the year spanning from spring 2011 to spring 2012. Georgia in particular saw a steady increase of youth reporting an increase in opportunities from 5% in December 2010 to 49% by February 2012. Azer-baijan and the North Caucuses showed the highest percentages of youth reporting an increase in oppor-tunities with levels reaching 92% in the former and 89% in the latter (the target for both was set at 80%).

Indicator 3: % of youth project beneficiaries from vul-nerable/minority groups accessing project services

Each project was designed to be inclusive of vulner-able/minority groups and promote social cohesion between groups. Given that vulnerability is context

10

specific, each project adapted this indicator and used country-specif-ic definitions. A general notion of vulnerability as captured by this indicator is risk of exclusion and marginalization due to a specific characteristic or deficit in ability level that may inhibit or prevent members to fully participate and benefit from services offered by the project. This definition could include ethnic minorities, mar-ginalized groups, youth with dis-abilities and vulnerable youth. Each project has set the target for inclusion of vulnerable/minority youth based on the preliminary assessments of the context.

The available data indicates a strong level of minor-ity/vulnerable youth involvement. The North Cau-casus project had 54% percent of its beneficiaries be-long to vulnerable/minority groups. This value was 20% in Georgia while Azerbaijan project had 88% of its participants being vulnerable or minority youth. In Albania 41% of beneficiaries were from vulner-able/minority groups. Data was not reported for Ta-jikistan.

The Kosovo project was thought to be particularly successful in this area. While there was no target set, a main focus of the program was to serve ethnically mixed populations of young people: three of the thir-teen centers were located in Serb minority populated areas and 80% of project activities were ethnically mixed. This, along with feedback from other pro-grams’ staff, indicates that location of the centers and the make-up of the surrounding populations, often influences the ease of reaching vulnerable/minority participants.

Program staff in the North Caucuses indicated that identifying vulnerable and minority youth proved to be difficult. In order to resolve this challenge, program staff came up with creative ways to identify this population, such as targeted ads in the hopes of reaching poor youth that listen to the radio on their mobile phones as a cheap form of entertainment, us-ing school databases to identify youth with learning and behavioral problems, or having volunteers draw up lists of possible participants from their neighbor-hoods for recruitment.

Target End of Project

30%

75%

20%

N/A

80%

41%

88%

20%

39% 54%

Albania Azerbaijan Georgia Kosovo N. Caucasus

Percent of Vulnerable/Minority Youth Participants

Gender Aspects In Beneficiary Participation Trends

Indicator 4: Direct project beneficiaries (number), of which female (percentage)

Direct beneficiaries include (1) all young people ages 14-30 who use project services and (2) representa-tives of participating SME, NGOs and other partici-pating government representatives, who are directly involved in the project activities. The latter part of the definition would be applicable for representatives of the Youth Ministries, for example, who took part in capacity building trainings.3 In many of the program areas barriers to participation in youth development, and especially business or economic development, exist for women. Disaggregating the numbers by gen-der demonstrates the programs’ ability to overcome those barriers and offer equal opportunities to both genders.

The number of the project beneficiaries steadily grew for all reported projects and continued to grow at a same rate for the three remaining open programs since the interim report. Numbers to date, indicate that the projects have reached more than 110,000 young people in the ECA region.

The interim results report produced in 2011indicated

3 In the process of data collection it became evident that not all direct beneficiaries, particularly those involved in the capacity building trainings or SME, NGO trainings are adequately cap-tured in the supplied data. Nonetheless, the vast majority of the direct beneficiaries—young people – users of the youth centers services are accounted for.


a worrying trend of far more young men participating in projects than young women,. with the exception of the North Caucasus. However, since then there has been notable improvement.

The number of female participants continued to in-crease in the case of Georgia, with the last reporting period in March 2012 consisting of 58% female par-ticipants, for a project total of 55% female participa-tion. In Tajikistan the gap has been fairly consistent throughout project implementation and has slightly risen from 45% to 47.5% in the final exit report. Ko-sovo maintained a participation rate ranging from 36%-45% for females throughout the project cycle. Azerbaijan reported a female participation rate start-ing at about 33% and rising to 38%. The most major increases in female participation were seen in Albania and the North Caucuses. In Albania, female partici-pation rose steadily from 30% in April-June 2011 to 52% in July-September 2011 and had reached 73% by April 2012. This is especially remarkable as the target was only placed at 40%. The North Caucasus also saw an increase in the percentage of female participants from about 25% to 42%.

The projects learned that the inclusion of women can require creativity to reduce barriers. In Tajikistan, to overcome barriers to participation, the program fo-cused on gender sensitive programming, including hiring female staff, reserving spots for women in class-es and providing transportation to the youth centers. When possible, the program also brought together the young men and women to breakdown gender stereo-types. In Azerbaijan, program staff had to work to gain the buy-in of the community before women were gen-erally allowed to participate by hosting community meetings to inform the public and field questions and complaints. Through community participation the program staff gained confidence of the community to allow the young women to participate and let them enjoy the opportunity as their male counterparts. As women saw other women prospering in the program and having certain cultural needs taken care of they felt more comfortable participating.

These techniques take time to develop. This is espe-cially true in cases where communities must be con-vinced to allow women to participate. The incremen-tal increase in female participation is a positive sign of progression both in recruiting efforts and community acceptance of the programs. As programs are replicat-

33% 30% 25%

45% 38%

73%

42% 47.5%

Azer

baija

n

Alba

nia

N. Cau

cuse

s

Tajik

istan

% Female Participation

Interim Data (2011) Final Data (2012)

ed, the process will have to begin anew in each com-munity and programs will face varying challenges and barriers to overcome. Strategies that work should be widely shared as no one tactic will work for all com-munities in turning around gender imbalance.

Indicator 5: Indirect project beneficiaries (number), of which female (percentage)

Indirect beneficiaries include those who benefit from the areas and facilities created by the youth develop-ment project. At the time of the interim report, the number of indirect beneficiaries of the project had reached almost a quarter million people. Since that time, programs in Georgia, Azerbaijan and Albania have continued to increase the amount of indirect beneficiaries reached, as well as the percentage of those that are female. The reach of the programs is quite wide and is affecting much larger numbers than the direct program participants. This is valuable for the indirect beneficiaries and also for the programs themselves. The larger number of beneficiaries these programs have, the more community buy-in they accrue, increasing their chances for sustainable pro-gramming in the future and higher quality results.

In the case of Georgia, the number of indirect proj-ect beneficiaries grew to reach 36,100, 69% of whom were female. The Azerbaijan project surpassed its tar-get and reached 168,757 indirect beneficiaries, 42% female. Finally, while Albania fell short of its target for number of indirect beneficiaries reached, the proj-ect did surpass the target for percentage of female in-

12

direct beneficiaries at 49%, up from just 33% at the time of the interim report. The North Caucasus proj-ect was also particularly successful in this indicator area with the percentage of female indirect beneficia-ries reaching 60%.

Results of The Outcome Level Indicators

outcome 1: Quality and Relevant Youth Services are Available for Young People

Indicator 6: # of youth beneficiaries of project created/supported youth services.

Beneficiaries of project created/supported youth ser-vices include all youth service users. Unlike indicator 4, this indicator includes only youth participants and not SME, NGOs and other participating government representatives, who are involved in the project ac-tivities. Achievement toward targets varied greatly between the different country projects. So far, Alba-nia reached 2030 youth beneficiaries of which 73% were female. Georgia reached a total of 5,350, which is slightly over half of the target number. The North Caucasus was on target with 4,459 as well as Tajiki-stan project which reached 7,500 young people. The Kosovo project was the only one to surpass its target, reaching 238% of the target.

Indicator 7: % of youth beneficiaries who are satisfied with the quality of youth services.

For this indicator satisfaction was defined as “ad-equately service the needs of young people in the

community”. This is noteworthy as the indicator is broad and covers all needs and is not solely a rating of the actual services provided, demon-strating the all-encompassing nature of programming.

With the exception of the Albanian project, whose target level was set at 70%, all project teams set the target level of satisfaction with quality of youth services at 80%. Five coun-tries provided data on the satisfac-tion levels of the quality of youth

services available for project beneficiaries. Tajikistan was the only project that did not meet its target on this indicator, with 45%, 49% and 56% of participants in the three respective program areas who rated the program satisfactory. The rest surpassed the indicator target set. The North Caucasus and Albania reported target levels of satisfaction at 90% and 87% respec-tively. Georgia saw an increase in the percent of ben-eficiaries satisfied with the services from 59% to 96%. Finally, Azerbaijan surpassed its target with 91% of its beneficiaries feeling satisfied with the services.

Indicator 8: % of youth beneficiaries reporting that they had used the skills they had acquired through project ac-tivities in their daily lives.

This indicator includes any skills learned in project activities that are used in daily life. Programs are ex-tremely wide ranging in their subject matter from life skills to vocational skills and business development skills. For instance, in the Tajikistan program, this included computer, English and Russian language, communications and business skills among others. This indicator shows the direct benefit of the projects in the daily lives of the participants.

Four countries provided data for this indicator. The percentage of participants in Georgia stating that they had used skills acquired through project activities rose greatly through the reporting cycle, starting with 33% and ending with 95%, surpassing its target. Tajikistan surpassed its target of 40% in all of its program areas with 45%, 49% and 56%. The North Caucuses also surpassed the target of 70% with 82%. Azerbaijan surpassed its target of 80%, with 84% of participants indicating that they use the skills acquired through the project in their daily lives.

Country Number of Indirect Beneficiaries

% female

Albania 9211 49%

Azerbaijan 168,757 42%

Georgia 36,100 69%

N. Caucasus 121,954.00 60%

Tajikistan 58,565.00 60%

Total 394,587.00 50.60%


Indicator 9: % of territory in each country in which project activities have been imple-mented

The territorial coverage of individual projects varies drastically from country to country. However, this vari-ance is more a result of proj-ect design than achievement of indicator targets. Kosovo project did not set clear tar-gets for this indicator at the design stage. However, after beginning with the two ini-tial resource hubs and several small youth centers, the network of the youth centers in Kosovo is located throughout most municipali-ties, including the conflict area of North Mitrovica. As the project expanded, most of other Kosovo mu-nicipalities benefited from youth resource center and by project close activities has been implemented in 40% of the country.

In Albania, the project was designed to support only two municipalities - Durres and Tirana, hence the target level is only 0.3% of the country’s territory. However, those two municipalities account for al-most 1/3 of the population. The remaining countries indicated a strong presence through their respective territories and all either came close or surpassed the target percentage.

outcome 2: Youth in ECA have Increased Opportunities to Develop Business and Employment Skills

The indicators of the second outcome are focused on a number of activities related to the promotion of employment and other business opportunities for young people in the region. Most of the projects have one to three types of business development ac-tivities: business trainings, support for micro-finance activities and vocational/apprenticeship programs. The Regional Framework captures all three types of activities, however not all projects offered all types of business/employment activities so data is incon-sistent.

0.3%

17%

66%

N/a

34% 40%

0.3% 14%

69%

40%

27%

83%

Albania Azerbaijan Georgia Kosovo N. Caucasus Tajikistan

Percent of Territory in each country in which project activities have been

implemented


Indicator 10: # of youth receiving business development training

This indicator measures the number of young people who achieved at least an 80% attendance rate in busi-ness development training programs. Four countries reported data on the number of youth receiving busi-ness development trainings. A total of 751 young people received business development training, in-cluding: business plans design, legal aspects of start-ing a business and marketing. Even though still in the implementation stage, Albania reached almost half of its target with 121 participants; Azerbaijan accounted for 242 participants, very nearly reaching its target of 250; the North Caucasus exceeded its target of 100 with 138 participants; and Tajikistan reached its tar-get of 250 participants. These numbers represent a great success as all but one very nearly reached or sur-passed its target of providing a full course of business development training.

Indicator 12: # of youth placed in vocational and ap-prenticeship programs.

Vocational training or apprenticeship programs taught youth a skill that can be used to earn an in-come. Some examples include, tailoring, baking, video/camera work and auto repair.

The Albanian program was able to place 227 youth in apprenticeship programs (the target number was just 80), 69% were female. This represents great im-provement from the initial reporting periods when

14

only 10 participants were placed in such programs, 40% of them female. Azerbaijan and Tajikistan both had targets of 250 participants, with Azerbaijan being close to target with 242 and Tajikistan meeting in with 250 youth exactly (these num-bers were not disaggregated by gen-der). The success of Albania should be analyzed to determine exactly how the program was able to raise its numbers to such a degree.

Indicator 13: % of business develop-ment training participants who suc-cessfully pass business training graduation exam.

Four countries reported data on the percent of busi-ness development training participants successfully passing the graduation exam. The results vary by country and range from 100% in Tajikistan to 36% in Albania. Azerbaijan, despite missing its target of 100%, did achieve a high pass rate of 85% while in North Caucuses a pass rate was at 72.4% (the target was 70%).

Both Albania and the North Caucuses programs cite the graduation requirement of making a business plan as a cause of attrition at the end of program and accounting for the drop in numbers from those that attended the course 80% of the time or more (Indica-tor 10) to those that received a certificate. Program staff in the North Caucuses commented that this end requirement acted to weed out less motivated youth and the resulting business plans were high quality work. Azerbaijan, on the other hand, offered a gradu-ation certificate as an incentive to participants in the business development training who passed the busi-ness development test, which may be the cause for the relatively higher pass rate.

outcome 3: National Youth Organizations and Support Structures Have Increased Capacity to Implement Youth Policies and Programming.

An important part of the of the ECA youth develop-ment project is building capacity of the various youth structures to successfully carry out youth projects and policies.

Indicator 14: # of Youth Ministry and Youth Depart-ment staff taking part in youth policy development training.

Only the Georgia, Kosovo, North Caucasus and Ta-jikistan had this as a component of their projects and hence had set targets for Youth Ministry and Youth Department staff attending trainings. The trainings included workshops and other activities directly re-lated to development of youth policy in each country. The Georgia project with 17, Tajikistan, with 13 and Kosovo with 2 Youth Ministry and Youth Depart-ment staff, all surpassed the targets set. Finally, the North Caucasus hosted the largest amount of youth ministry staff at their trainings with 27 and exceeded their target of 25.

The inclusion of Youth Ministry and Youth Depart-ment staff in youth policy development training is vi-tal to the success of this component of youth projects and the incorporation of these officials has proved to be very valuable. In Tajikistan, the relationship was seen to expedite implementation of the project after a long delay and create buy-in for project extension and sustainability. The North Caucuses project was able to use their close relationship to help shape the programmatic orientation and policy of the Youth Ministry.

Indicator 15: # of government structures sup-ported with technical assistance, financial and technical capacity building related to youth.

A government structure is any individual entity with-in the local, regional or national government that

25

1

25

12

27

2

27

17

Georgia Kosovo N. Caucusus Tajikstan

Number of Youth Ministry or Youth Department Staff

attending trainings



receives technical assistance and/or financial and technical capacity support. Support can range from technical assistance to logistics. In the North Cau-cuses, for example, the program supplied basic office furniture to the Youth Ministry and also provided strategic management training. The training was thought to be especially valuable as they provided the same trainer used at the federal level and there-fore, the credential was seen to be more credible and recognized at the federal level. These methods aid in the positive development of Youth Ministries and Departments and strengthen relationships. Five Ko-sovo government structures benefited from capacity building related to youth, 18 in the North Caucasus, 15 in Tajikistan and one in Georgia.

Indicator 16: # of external partnerships with youth stakeholders formed during implementation of the project.

External partnerships include implementing part-ners, advisory boards and project steering commit-tee representatives and other strategic partnerships formed during the course of project implementation. These external partnerships can prove invaluable as they provide resources to the participants that these programs are not able to provide. Examples from Ta-jikistan include The Centre for Economic Opportu-nities, Youth Achievement and several microfinance institutions. At the time of the interim report a total of 66 new partnerships had been formed in the region. By the end of the final reporting period that number had grown to more than 80, with16 more partner-ships in Georgia, a total of 26 in Tajikistan and 33 in the North Caucuses. Because these external part-nerships are so valuable to the youth participants, programs that did not report data on this indicator should focus on forming partnerships in the future.

16

The process of developing and implementing Regional Results Framework highlighted important gaps in the design of the individual project results frameworks for the ECA youth development projects. While this should have been addressed more thoroughly at the design stage of the youth interventions, the creation of the ECA Regional Results Framework aided proj-ects that were on the way and has a potential of being a useful tool to all new programs going forward.

The monitoring and evaluation system that has been set up to monitor progress of youth development projects in the ECA region and provide a metric by which to measure projects against each other provid-ed useful information for the future planning of proj-ects and holds a promise for ongoing utilization. Al-though training was provided, implementing a revised results framework and Management Information System in the middle of the project cycle is difficult and the quality and consistency of the data reflected this. However, the results framework sets out a strong metric to measure program success and generate data across countries to illustrate in a broader sense of the effectiveness of certain designs and strategies.

Despite some difficulties, the new Results Framework has already shown its potential. Analysis of results in the interim report helped to discover issues within the programs and rectify them before the close of

projects. For instance, the data in April 2011 illus-trated the need for stronger emphasis on attracting female participation in the projects as the percentage of female participants was receding. By the time of this final report, female participation was higher than male participation in Georgia, Albania and Azerbai-jan (the three projects that remained open between the interim and final reports).

Feedback on the system is already very positive with project staff indicating that using the RRF has helped them keep track of data and be more results driven and understand reasons for success.

A results framework that is able to draw data across projects that are not identical in design is extremely helpful in deciphering which designs and strategies work best under different circumstances. The Results Framework has and will continue to point out dif-ferences in outcome achievement from which con-clusions can be drawn as to successful strategies for youth engagement and successful programming. The shared connection of using the same Results Frame-work should foster cooperation and idea sharing be-tween the programs to increase the quality of results. The RRF will also aid in deciphering the smartest in-vestments and most sound designs for future World Bank projects.

CONCLUSIONS

THE WORLD BANK1818 H, Street N.W.

Washington, D.C. 20433www.worldbank.org

Documents

EUROPE AND CENTRAL ASIA - duinareyes.comduinareyes.com/Duina_Reyes_-_Web_&_Graphic_Designer/Sample_…ECA Regional Results Framework for Youth Development Projects: Final Results May