Final Paper

Running head: PROGRAM EVALUATION PAPER1

Program Evaluation Paper

Children In Action Sports Club: Barnabus Club

Jessica L. Hipchen

The University of Georgia

PROGRAM EVALUATION PAPER2

Literature ReviewThe children who participate in the Barnabus Club, a program provided by Children In

Action Sports Club (CIA) located in Brunswick, Ga, are generally from neighborhoods where

drugs, alcohol, and violence are a commonality. The criteria for participant selection in the

program includes: single parent family, one or both parents incarcerated, family in crisis (i.e.

drug/alcohol abuse), low self-esteem, weight/coordination issues, and/or poor school

performance.

Many of these factors have not been measured in the community. There is existing data on

single parent family households within 5-10 miles of our facility. In total, there are an estimated

2,475 single-parent households with children in this 10 mile radius. Something that is important

for these children is programs that are age appropriate and provide and clean and safe

environment (Nowak B., 2014, p. 13).

These children have little to no positive influences in their lives. One of the factors that

influences children’s behavior in the schools is the importance that is placed on an education

within the home and in their social life. Within the ten mile radius mentioned earlier, there is an

estimated total of 3,741 individuals who do not have a high school degree (Nowak, 2014, p. 8).

Additionally, there is an estimated total of 2,580 children between the ages of 6-11 in this area

(Nowak, 2014, p.5). This is not including those that are farther than the 10 mile radius. If they

do not have time invested in them by outside sources such as the personnel within the Barnabus

Club afterschool program, they will likely fall into the cycle of the behaviors mentioned above.

Barnabus Club was created so that these children would have a safe place to go after school.

The kids are given an opportunity for structured activities to learn how to behave appropriately

and to work in a group setting. The goal of this program is to teach the children team building

and behavior, academic, and service skills. This will in turn increase their self-esteem and


motivation, decrease their rate of behavior problems in school and the home, improve their

grades, expand their knowledge of how to work as a team, and give back to the community in

which they live. Most importantly, the program helps the children to understand that Jesus Christ

is the motivator behind all that is done.

Program TheoryThere is no written program theory or guiding philosophy for the Barnabus Club. Their

activities are not based on hard data and/or research. The source of the impetus for the program

and associated assumptions is the Gospel of Jesus Christ. The personnel of Barnabus Club are

Christians who whole heartedly believe that Jesus is the one who gave them a passion for helping

at-risk children such as the ones involved in the program. They are able to make a difference in

the lives of children because they are merely a reflection of Jesus Christ’s character. Barnabus

Club believes that when the children have a circle of influence that reflects the love of Jesus

through activities such as mentoring, group sports, and academics, positive changes such as the

ability to make healthy choices are made within the children. Staff have seen and recorded these

positive changes throughout the length of the program, which will be further discussed in the

data collection portion of the paper.

Program Description:

PurposeThe mission of Children in Action Sports Club (CIA) is “to utilize sports and physical

activities as an outreach to children, youth, and adults to build character, excellence, and

teamwork” (Children In Action Sports Club, 2014). The vision of the organization “is to provide

a favorable environment to experience, accept, and exercise the application of a Christ-centered

lifestyle” (Children In Action Sports Club, 2014). The purpose of the program is to be a Christ-

centered ministry that is intentional in building relationships with children ages seven to twelve


and helping them to make healthy choices. One of CIA’s programs that was created to

encompass this mission and vision and create an impact in the community is the Barnabus Club.

ObjectivesThere are no existing written objectives. I have come up with a list of goals and

objectives that I would suggest creating based on the information provided about the program.

These include:

Goal 1: To promote the gospel of Jesus with participantsOutcome Objective 1a: To increase basic knowledge of the gospel of participants by 100% by 2016.

Outcome Objective 1b: To increase the number of children in the program who believe that Jesus Christ is their Savior by 50% by 2016.

Goal 2: To decrease the number of behavior problems with participants, specifically those who were referred to the program because of behavior issues

Outcome Objective 2a: To decrease the number of behavior problems experienced by participants within the schools by 50% by 2016.Outcome Objective 2b: To decrease the number of behavior problems experienced within the home by 50% by 2016.

Goal 3: To increase self-esteem of participantsOutcome Objective 3a: To decrease the rate of obesity of children involved in the program, specifically those referred to the program because of weight issues, by 20% by 2016. Outcome Objective 3b: To increase the number of children who believe that they can have a successful future by 90% by 2016.

Goal 4: To increase ability of participants to work as a teamOutcome Objective 4: To decrease the amount of negative interactions between/among participants involved in recreational activities by 75% by 2016.

Goal 5: To increase knowledge of participants on how to make healthy decisions

Outcome Objective 5: To increase participants’ knowledge on how to make healthy decisions by 100% by 2016.

Goal 6: To increase activeness of participants in the community


Outcome Objective 6: To increase the amount of participants who spend time in community service activities to total 100% by 2015.

ParticipantsThe participants are children between the ages of seven and twelve who have participated

in the Summer Outreach program, another CIA program, and are identified as high risk. The

criteria for participant selection includes: single parent family, one or both parents incarcerated,

family in crisis (i.e., drug/alcohol abuse), low self-esteem, weight/coordination issues, and/or

poor school performance. The children have shown themselves to be receptive to the

programming (Summer Outreach) yet needing more time with personnel to maintain their

positive behavior and attitude changes. As of 2014, a total of 70 kids participate in the program

with an average of 40 children per day.

StaffingThe program has a total of two full time and four part time staff. In addition to paid staff,

there is an average of five to seven volunteers per week (A. Benner, personal communication,

October 30, 2014).

In addition to the staff and volunteers mentioned above, an evaluator will also be

considered as part time staff in the Barnabus Club’s program. The tasks of the evaluator will be

to collect behavior reports from the school that involve the participants in the program, create

and administer surveys for teachers, parents, and children, provide and distribute log books for

teachers and parents, collect data on progress reports and report cards of the participants, record

the amount of times that a CIA staff member has had to visit the school to mentor a participant,

and lastly to combine all of the information into a structured format that can then be coded and

interpreted for distribution.


SettingThe Barnabus Club program is located in a gymnasium leased by CIA. The gymnasium

includes a kitchen, a dining area for snacks/meals, and additional areas for homework, devotions,

and physical activities.

ActivitiesAs mentioned before, CIA is an organization dedicated to reaching out to high risk

children and youth to build character, a desire for excellence, and an appreciation for teamwork

through physical activity in a team setting which mirrors Jesus Christ in relationship and

character building. In addition, the Barnabus Club, now in its third year of operation, involves

one-on-one mentoring with participants during practices, games, and devotion times as well as

picking children up from schools around Glynn County and transporting them to the facility for

the after school program. The daily program involves 30 minutes of free time for the children to

unwind followed by a break for nutritious snacks and 45 minutes of supervised homework/study

time. At the end of their study time, the program provides a 30 minute devotion time. The final

45 minutes are dedicated to a variety of sports activities where the importance of physical

activity is stressed. Barnabus Club students who experience difficulties in school, academic

and/or behavioral, receive additional support from CIA staff who make school visits and

collaborate with school teachers and other school personnel. Based on their suggestions, CIA

works more directly with the children to help resolve their problems. Additionally, the children

are provided opportunities for community service each week. This may involve doing things

such as serving food to the homeless, raking leaves for the elderly, picking up trash, or

packaging medical supplies at Map International.

Logic modelSee Chart 1 in the Appendix


Focus/Aim of the Evaluation The focus of the evaluation is to determine if CIA’s Barnabus Club program is achieving the

goals and initial, intermediate, and long-term objectives that I created based on the program’s

mission, vision, and activities. There is no written list of goals and objectives; therefore, I created

my own. The results of the evaluation will show whether or not the Barnabus Club program is

having the positive impact on the participants’ behavior and attitudes that are intended. Once the

results have been identified, they can be used to show the areas where impact is abundant or

lacking so that changes can be made to be the most efficient and effective possible.

Research Questions and Hypotheses If students participate in the Barnabus Club then they will gain a greater

understanding of God’s word.IV-Information provided during the huddle/devotion sessionDV- Score on survey

If students participate in Barnabus Club then there will be a decrease in behavior problems at home.

IV-Participation in mentoringDV-Combination of a survey and a parent/guardian log book

If students participate in Barnabus Club then there will be a decrease in behavior problems at school.

IV-Participation in mentoring DV- Combination of behavior reports, a survey, and a teacher logbook

If students participate in the Barnabus Club then their grades in school will increase.

IV-Participation in homework sessionsDV-Combination of progress reports and report cards

If students participate in the Barnabus Club then they will acquire the skills to work on a team.

IV-Participation in sports activitiesDV-Survey

Measurable Indicators. See Chart 2 in Appendix


Design Selection The design selection for this evaluation will be a group research design (GRD). This

strategy is best for the Barnabus Club environment because it allows the program to be evaluated

immediately when a child enrolls in the program and then again a year later. Barnabus Club

would not want to utilize a GRD that delayed the intervention for a child, therefore the strategy

to begin the assessment and data collection at the time of enrollment was chosen. There would be

ethical issues if Barnabus Club were to choose to have a control group. This is due to denying

services that would help the high-risk children. Therefore, all designs containing control groups

were eliminated from the list of options. Furthermore, there will be continuous data kept by

parents, teachers, and staff which will be gathered together and added to the results of the pre-

and post-surveys given to the children to measure different areas of growth. This will show if the

objectives were reached. This is an important factor to show the effectiveness of the program on

the areas intended. There are some threats to internal validity when using a GRD. It is not always

the case that the results are generalizable. To overcome these difficulties there will be a pilot test

before the actual evaluation is conducted. Results cannot be labeled as causal because the

evaluation is not as rigorous as an experimental design (Royce D., Thyer B., Padgett D., p.222).

If necessary, this can be overcome by conducting an evaluation using an experimental research

design. Additionally, in the future, the organization will be able to use the same materials and

evaluation plan. This process is referred to as replication (Royce D., Thyer B., Padgett D., 164).

Data Collection In an effort to control external validity there will be a pilot test completed beforehand.

Data will start being collected upon enrollment into the program. The process of data collection

can be found in Chart 2 of the Appendix.


Data Analysis StrategiesThe surveys will be given to children at the time of enrollment into the program. The

evaluator will distribute the same surveys at the one year mark. Throughout the year, staff will be

responsible for keeping a log book on the behavior of children during group sports activities.

This will be indicated by listing with tally marks the number of times that a child has a negative

behavioral interaction with a team mate. Teachers and parent(s) will also be responsible for

keeping behavior log books and will record the same way as staff for the child in the school or

home. Grades will be collected from the schools at the time of distribution of progress reports

and report cards. Lastly, BMI tests will be conducted at the same times as the surveys. The

results will then be given to the evaluator who will add up the scores and interpret the findings so

that the organization can understand the effectiveness of its program on all levels.

LimitationsThe limitations that the evaluation may come into contact with are that students may not

attend the afterschool program every day. This could skew the data for behavior during sports

activity times. Additionally, each parent, teacher, and staff may have different definitions of

good and bad behavior. This could result in a misuse of the terms when recording data. An

additional issue could be that the parent(s), teacher(s), and staff do not record like they are

supposed to in the log book. If a student involved in the evaluation drops out of the program, the

data for that child will not be included in the results.

Potential Ethical Issues One potential ethical issue to keep in mind is confidentiality. The evaluator and staff

would be required to not mention ongoing data and results that are tied to any one child. This

would be unethical and could threaten the well-being of the child. An additional potential ethical


issue is informed consent. The child would need to give assent and the parent(s)/guardian(s) of

the participant would need to give consent. Because the child is not 18 years of age, this is

required. Not following these guidelines would be unethical and would cause the data to be

thrown out.

Plan for Involving StakeholdersTo involve stakeholders, CIA could host a family night. This would be done so that the

families of the children involved in the program could have a chance to come and visit the

facility and learn about what their children are participating in each day after school and see the

impact that it is having on their child/children’s life. All staff and volunteers would be required

to attend so that the parent(s)/guardian(s) of the children can ask any questions and meet the

hands and feet that are behind the program.

An additional way to involve stakeholders would be to host an appreciation dinner for

staff and volunteers. Food would be provided through donations from local organizations such as

those that already provide the healthy snacks for the children in the Barnabus Club program. The

dinner would be free of charge and would be a time of thanks and community for the staff and

volunteers.

Plan for Disseminating Results The results will be distributed to all key stakeholders of CIA. This includes

parent(s)/guardian(s), teachers involved with the program, staff, board members, volunteers, and

donors. This is so that each stakeholder can see the impact that their time and service are making.

Because board members have a bi-monthly meeting and will soon be switching to a monthly

meeting, results will be disseminated through a PowerPoint presentation during the first board

meeting after the results have been finalized. All staff will be required to attend the board

meeting. Additionally, results will be sent to all board members and staff via email. Teachers,


parent(s)/guardian(s), volunteers, and donors will have a summary of results that will be placed

into a document containing overall percentages and major findings. This is so that the document

will not contain jargon and a multitude of hard data that may not be easily understood. This

document will also be posted on the website.

TimelineSee Chart 2 in Appendix

Budget

Children in Action Sports Club: Barnabus Club program

Requested Funding

Total

Non-Personnel Expenses

Printing (surveys, presentation materials) $700.00 $700.00Log Books $300.00 $300.00BMI Chart $0 $0Total Personnel Expenses $1000

Personnel ExpensesEvaluator (surveys, coding, data documents)

$15,000 $15,000

Total Personnel Expenses $15,000

Total $16,000


References

(2014). About Us: Mission &Vision. Retrieved from http://www.ciasportsclub.net/mission--

vision.html.

Georgia Baptist Convention. (2014). Community Outreach Profile. Comprehensive

Demographic Data and Lifestyle Analysis. 2505 Norwich St., Brunswick,

Ga. :Nowak.

Royse, D., Thyer, B. A., Padgett, D. K., & Logan, T.K. (2010). Program evaluation: An

Introduction (fifth edition). Thomson Brooks/Cole.

http://www.ciasportsclub.net/mission--%09vision.html

http://www.ciasportsclub.net/mission--%09vision.html


Appendix

Chart 1: Logic Model

Inputs Activities Outputs InitialOutcomes

Intermediate Outcomes

Long-term Outcomes

1. Staff2. Funding

Entities3. Volunteers4. Gymnasium5. Sports

Equipment6. Healthy

Food for Snacks

7. Transportation

8. Collaborating Partners

9. Existing Organizational or Interpersonal Networks

10. Time11. Supplies12. Parent(s)/

Guardian(s)13. Computers

1. Huddle/Devotion and Character Development Sessions

2. Homework Sessions

3. Physical Activity Sessions

4. School Visits Sessions

5. Service Learning Activities

1. Number of huddle/devotion and character development sessions held

2. Number of homework sessions held

3. Number of physical activities sessions held

4. Number of school visits made

5. Number of service learning sessions

1. Increased awareness of the gospel

2. Increased awareness of positive character development

3. Changes in attitude about homework

4. Increase in physical activity

5. Changes in attitude about being mentored

6. Increase in participation in service learning activities

1. Increased knowledge of the gospel

2. Increased knowledge of a positive character

3. Increased educational skills

4. Increased ability to participate in sports activities

5. Increased ability to work as a team

6. Increase in positive behavior at school and home

7. Increase in willingness to help others

1. Increase in belief that Jesus is our savior

2. Increase in self-esteem

3. Increased positive character development

4. Increase in grades

5. Ability to work as a team


Chart 2: Measurement Indicators

Time of Outcomes Indicators Target Sample Data Source Data Evaluation Collection

MethodAt registry into program and one year

If students participate in the Barnabus Club then there will be a decrease in problems experienced by participants within the school.

Ongoing mentoring within the program and during school as needed

50% decrease in behavior problems

High-Risk children

School

Teacher

Child

Behavior Reports

Behavior Log-Book

Survey

At registry into the program and one year

If students participate in the Barnabus Club there will be a decrease in the number of behavior problems experienced within the home.

Ongoing mentoring afterschool within the program

50% decrease in behavior problems

High-Risk children

Parent(s)

Child

Behavior Log-Book

Survey


If students participate in the Barnabus Club then there will be a decrease in the rate of overweight/obesity of children involved in the program, specifically those referred to the program because of weight issues.

Organized sports activities during the program

20% decrease in the rate of overweight/obesity

High-Risk children

Child BMI Test


If students participate in the Barnabus Club then

Ongoing mentoring and huddle/devoti

90% increase

High-Risk children

Child Survey


there will be an increase in the number of participants who believe that they can have a successful future.

on sessions


If students participate in the Barnabus Club then there will be a decrease in the amount of negative interactions between participants involved in recreational activities.

Organized sports activities during the program

75% decrease

High-Risk children

Child

Staff

Survey

Behavior Log-Book


If students participate in the Barnabus Club program then there will be an overall increase in knowledge of how to make healthy decisions.

Ongoing mentoring and huddle/devotion sessions

100% will increase their knowledge

High-Risk children

Child Survey


If students participate in the Barnabus Club then there will be an increase in the number of participants who spend time in community service activities.

Weekly Service Learning Activities

100% of students will increase time spent

High-Risk children

Staff Time Log


If students participate in the Barnabus Club then there will be an increase in basic knowledge of

Ongoing huddle/devotion sessions

100% will increase their knowledge

High-risk children

Child Survey


the gospel of participants.


If students participate in the Barnabus Club then there will be an increase in the number of children in the program who believe that Jesus Christ is their Savior.

Ongoing huddle/devotion sessions

50% of children will believe this

High-Risk children

Child Survey

Documents

Final Paper