Measuring Hope for Children Living in Poverty:

Engaging stakeholders in evaluation at City Kidz

June 13, 2013Rich Janzen & Liliana Araujo

Partners

Purpose

To share the evaluation experience of City Kidz as an example of a collaborative evaluation that followed the principles of community based research.

1. Overview of community based research

2. Overview of the City Kidz evaluation

3. Mechanisms of engagement

4. Products of engagement

“The power of knowledge to impact change” - Since 1982

Centre for Community Based Research

30 years of social innovation in Canada

Over 350 community based research projects

Based on an entrepreneurial spirit in collaboration with many partners to initiate new projects

Hallmarks of Community Based Research

Research that strives to be:

Community situated- begins with a research topic of practical relevance to the community and is carried out in community settings. (Indigenous tradition)

Participatory - community members and researchers equitably share control of the research agenda through active and reciprocal involvement in the research design, implementation and dissemination. (Southern tradition)

Action-oriented - the process and results are useful to community members in making positive social change and to promote social equity. (Northern tradition)

Functions of Community Based Research

Knowledge production

Knowledge mobilization

Community mobilization

Definition

“… a research approach that involves active participation of stakeholders, those whose lives are affected by the issue being studied, in all phases of research for the purpose of producing useful results to make positive changes”

(Nelson, Ochocka, Griffin & Lord, 1998,

p.12)

“Research with,” not “research on” people Training and mentoring Opportunity of meaningful involvement Valuing experiential knowledge Data for advocacy Value-driven approach (Ochocka & Janzen, 2007)

A Growing Trend“Too often, important knowledge remains hidden in academia. Too often, governments develop policies without a full understanding of the big picture and without tracking the consequences of their policies. Too often, civil society organizations implement programs without adequate analysis of the underlying problem and careful consideration of how the program will play out. Solving the complex social, environmental and economic problems we face will require collaborative efforts that are radically inclusive of diverse perspectives and skills. Such collaborations become possible when faculty, staff, and students come to realize that people in community settings have knowledge, experience, and talents that complement their own.”

- Fryer, 2012, University Affairs - emphasis added

Overview of City Kidz Evaluation

World Vision and City Kidz history in Partners to End Child Poverty (PECP)

Desire to replicate to other communities

Purpose• To assess the implementation processes of City Kidz’ core

programs• To assess the outcomes for program participants of City

Kidz’ core programs• To identify future directions for improving on and replicating

City Kidz’ core programs in other sites across Canada

Evaluation Background and Purpose

How are core City Kidz programs presently being implemented?

How and to what extent have core City Kidz programs impacted the well-being of children in low-income communities of Hamilton?

What suggestions would help to improve and replicate core City Kidz programs?

Main Research Questions

Follows the principles of community-based research (community-situated, participatory, action-oriented)

Takes matters of faith seriously

Adopts World Vision’s model of child well-being

Program theory to reflect these program elements

Evaluation Approach

Kinder Kidz•Bus activities/food•Theatre show (4x/Saturday)•Biblical principles/prayer

City Kidz Saturday•Bus activities/food•Theatre show (4x/Saturday)•Biblical principles/prayer

Junior Leadership•Sat. volunteering & connecting•Mid-week training (Kid Lead)•Biblical principles/prayer

Increased self concept and

positive self worth

Increased opportunity for, awareness of & participation in

community activitiesIncreased positive peer influence and

relationships

Increased trust in, credibility of and influence of adult

mentors

Increased prosocial behaviour

Increased understanding of

God’s love

New relationships with adults who love unconditionally

Increased leadership within City Kidz &

community

Healthier lifestyle boundaries

More likely to make positive choices

Weekly Home Visits-Personalized mentorship -Information about activities/events -Prayer

-Encouragement to go to group activities -Support to family members

Discerning God’s Lead•Spiritual retreat

•Daily staff prayer•Openness to miracles

•Miracle Sundays•Organizational tithing

2013 Logic Model

Increased belief that God created me

Increased belief that God loves me

Increased belief that God has a plan for my life

Increased Hope•Better able to dream of a future beyond the cycle of

poverty

Healthier Children•Growth in physical, social, and

mental capacity •Enabled to confront challenges with resiliency and contribute to

society with full potential

Safer Children•Better access to food, safe

housing and stable employment

•Enabled to fully participate in society, now and in future

More Educated Children•Greater knowledge and skill

development•Enabled to achieve in life

and employment preparation

Healthier Neighbourhoods Across Hamilton

Com

mu

nity- L

eve

l

Increased cultural sensitivity and

acceptance

Healthier family dynamics

More likely to imagine

accomplishing great things

Ind

ivid

ua

l-Le

vel

Increased awareness of God-

given potential

Increased self-control and

empowerment

400 Kidz

1,100 Kidz

150 Kidz

2,200 Kidz

Group Activities

Individual Activities

Faith Outcomes

Resiliency Outcomes

Well-Being Outcomes

Improved Well-Being for Children Living in Poverty – 100,000 Canadian kids by 2040

Spiritual Discipline Activities

Increased social sensitivity and

empathy

Internal Outcomes Relationship Outcomes Action Outcomes

Program tracking logs (quantitative tracking of program inputs and outputs)

Survey of participants City Kidz Saturdays: sample of present participants complete short survey

interview with retrospective outcome, implementation, & future directions

Participant focus groups (3 groups with 8-12 sampled participants and/or parents about program implementation, outcomes and future directions)

Staff/volunteer interviews (2 focus groups with sampled staff and volunteers related to program implementation, outcomes and future directions)

Case studies (3 in-depth stories of program impact in the lives of purposively sampled present and past participants. Each story to consider interviews with the participant, one City Kidz staff/volunteer, and another support person in the participant’s life)

Key informant interviews (4 interviews to gain insight into outcomes and factors to consider when replicating core programs in other cities across Canada)

Methods

Mechanisms of Engagement

Stakeholder steering group that guided each step of the evaluation

Training and supporting of “community researchers”

to assist with data gathering

Methodology design that considered multiple stakeholder perspectives

Facilitation of active funder involvement (World Vision Canada) towards using evaluation findings to inform the replication of City Kidz nation-wide

Organizational feedback session to discuss evaluation findings

Products of Engagement

Collaboratively developed products… Comprehensive evaluation framework

Program logic model

Community researcher training manual

Qualitative and quantitative evaluation tools

User-friendly evaluation report that speaks to multiple audiences

Outcome survey based on program theory and informed by theory of hope and resiliency

Future Contact

Rich JanzenResearch Director,

Centre for Community Based Research73 King St. West

Kitchener, Ontario519-741-1318 x 233

rich@communitybasedresearch.ca www.communitybasedresearch.ca