Project AWARE2015-2016 Evaluation Findings

Presented byMaike & Associates, LLC

November 15th, 2016

An Overview of Project Outcomes: Overall Objectives, School-based Mental Health, Student

Assistance Program, and Youth Mental Health First Aid

State Management Team – November 15, 2016

Mission Statement: Connecting, Detecting, and Responding —

For the Mental Health and Well-being of Washington Students

Three Goals: Build and/or expand capacity at state and local levels to:

1) Improve school climate and safety, 2) Increase access to mental health services, and 3) Increase awareness of mental health issues

Twenty-four Outcome Measures:Capacity building, Collaboration, Discipline Policy, School Climate, School-

based Mental Health Services, Student Assistance Program and Youth Mental Health First Aid

Overall, the Project made excellent progress toward the stated Objectives, meeting, exceeding or making positive

progress on 21/24 measures.

Capacity Building

Outcome Measure 1.1.a. Expand state’s capacity to implement a collaborative, multi-tiered system of support to improve school climate and safety.

Collaboration

• Outcome Measure 1.2.a Collaborate to address student social, emotional learning, and violence prevention.

• Outcome Measure 1.3.1 Improve LEA capacity to implement/expand social, emotional learning and violence prevention strategies.

Discipline Policies, Practices, and ProceduresOutcome Measure 1.1.b. Revise or eliminate discipline policies, practices or procedures that disproportionately impact ethnic, racial or other minority students.

Outcome Measure 1.5. Reduce out of school placement (suspension/expulsion) by 25% in each targeted LEA as compared to baseline (2013-2014) by Project end.

School Climate

Outcome Measure 1.4.a. Improve School Engagement (improved attendance, reduced office discipline referrals [major offenses], improved grades).

Outcome Measure 1.4.b Improve Student-Student Relations subscale of the School Climate Survey.

Outcome Measure 1.4.c. Decrease rates of bullying among students in grades 7, 9, and 11, as measured by the School Climate Survey. (Project)

Outcome Measure 1.6. Improve Total School Climate, SEL Techniques, School Engagement, and Total Bullying Scale Scores of the School Climate Survey for students and staff in grades 3, 5, 7, 9 and 11.

School-based Mental Health Services

GOAL 2: Increase access to mental health services.

School-Based Mental Health ServicesStudent Referrals

Outcome Measure 2.1.a Increase the number of school-aged youth receiving school-based mental health services.

School-Based Mental Health ServicesEnrolled Students

44%

56%

67%

33%

19%

31%

50%

0%

20%

40%

60%

80%

100%

Male Female White Non-White K-5th 6th-8th 9th-12th

STUDENTCHARACTERISTICS:SCHOOL-BASEDMENTALHEALTHSERVICESPercentageofStudentsEnrolledinMHServices(n=149)

GENDER RACE GRADE-LEVEL

Youth of Color

School-Based Mental Health ServicesEnrolled Students

54%

35%

24%

0%5%

32%

40%

30%

6%

15%

12%

24%

37%

78%

67%

2% 1%9%

16% 13%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Risk/ThreatOthers Risk/ThreatSelf ImpairedSchoolFunction Emotional/Behavioral RelationshipProblems

PROBLEMSEVERITYATINTAKEPercentageofstudentsassessedasno,mild,moderate,orsevereproblembehavior

NoProblem Mild Moderate Severe

School-based Mental Health ServicesStudent Outcomes

Outcome Measure 2.1c. Decrease problem severity among highest risk youth by 20%.

0%

3%

3%

1%

2%

18%

47%

49%

71%

60%

82%

50%

48%

28%

39%

Risk/ThreattoOthersn=17

Risk/ThreattoSelfn=34

ImpairedSchoolFunctionn=63

Emotional/Behavioraln=132

RelationshipProblems=111

ChangesInSeverityofProblemBehaviors:HighestRiskParticipantsPercentageofstudentsidentifiedasmoderatetoseverewhose

risklevelincreased,decreasedorremainedunchangedatexit

IncreasedSeverity UnchangedSeverity DecreasedSeverity

School-based Mental Health ServicesHighest Risk Student Outcomes

3.12 3.063.21 3.16 3.15

2.24

2.47

2.682.86

2.74

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Risk/ThreattoOthersn=17 Risk/ThreattoSelfn=34 ImpairedSchoolFunctionn=63 Emotional/Behavioraln=132 RelationshipProblemsn=111

CHANGESINPROBLEMSEVERITYMEANSCORESFORHIGHESTRISKYOUTH:PREVS.POSTMeanscoreatprogramentryandprogramexit

Pre-MeanScore Post-MeanScoreIndicates statistically significantchange

School-Based Mental Health ServicesCommunity-based Referrals and Engagement

Outcome Measure 2.2. Increase the number of students referred to community-based mental health services.

RECOMMENDATIONS

• Referral Systems: Awareness trainings to school staff

• Direct Services: Address access barriers

• Accessibility: Ensure access is equal across subgroups of students

• Effectiveness: Review data, continuous quality improvement

• Community-based Engagement: Improve data collection practices/protocols

Prevention/InterventionStudent Assistance Program

Student Assistance ProgramStudent Characteristics

44%

55%

59%

41%

71%

29%

0%

20%

40%

60%

80%

100%

Male Female White Students of Color 6th-8th 9th-12th

Full Intervention Student Characteristics Percentage of enrolled students (N=271)

Student Assistance Program

43%41%

44%

58%

42%

34%

18%

0%

20%

40%

60%

80%

100%

ATOD-Related Home/Community School Success Behavior/Peer Mental Health Use by Family Use by Others

REASON FOR REFERRALPercentage of students referred by category

Student Assistance ProgramSubstance Use at Program Entry

24%28%

19%

46%

30%

25%

40%

30%

23%

65%

45%

29%

50%

41%

34%

76%

38%

57%

0%

20%

40%

60%

80%

100%

Male Female Middle School High School Students of Color White

Gender Grade Level Race

SUBSTANCE USE BY CATEGORY OF USERSPercentage reporting any past 3 months substance use

Alcohol Marijuana Any AOD

Student Assistance Program

22%

35%

29%

11% 9%

0%

20%

40%

60%

80%

100%

COA/Affected Others Substance Abusers AODA Ed Social Skills JH/MS Users

PEER SUPPORT GROUPSPercentage of students referred

(94% of students referred to one or more support groups)

Student Assistance ProgramChanges in Substance Using Behaviors

31% 31%

44%

27%24%

34%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AlcoholN=187 MarijuanaN=187 AnyAlcohol/MarijuanaN=185

PAST30-DAYSUBSTANCEUSE:ALLPARTICPANTSPercentageofpre/postrespondentsreportinganysubstanceuse

Pre Post

13%reduction

inalcoholusers 21%reduction in

marijuanausers

22%reduction

inusers

RECOMMENDATIONS1) Focus on high-risk students, distribute staff time; 2) Work closely with staff to identify and refer

program participants; 3) Ensure higher engagement in program services

by participants; 4) Develop appropriate materials (e.g., age and

gender specific and culturally relevant) to ensure engagement of program youth;

5) Provide additional staff training in the data collection process; and

6) Monitor program data for quality and adherence to program fidelity.

Youth Mental Health First Aid

Youth Mental Health First Aid (YMHFA)

At the State level, 685 individuals have been trained in YMHFA this year – with 29 receiving training as YMHFA Instructors.

At the the three school communities, an additional 502 individuals have been trained in YMHFA this year – brining the total to 1,187 individuals trained statewide! All sites met Year 2 YMHFA training goals.

These individuals reported encouraging 1,533 youth to seek services, nearly three times the program goal of 554 youth!

Way to go!

YMHFAStories from the field…

• “One of my students had recently starting having [suicidal] ideation, which had resurfaced after a long respite. She had not told anyone, but in my office, she was making comments that reminded me of the YMHFA training. After using the ALGEE model, the student was able to express her emotional distress, gain the awareness of and support of her family, and be connected with local mental health services.”

• “Several of my students had thoughts about suicide so I had to apply ALGEE. One of the students had a plan in place for suicide so I had to report to admin at my school, contact the parent and have the parent take the student to Children's Hospital for evaluation. My student was released and returned to school the next day. My student is still alive today and showing improvement. Thank you!”

• “I have a 6 year old girl who is a worrier. She is a "picker" and also complains about stomach aches pretty consistently. She utilizes the health room a lot. I've talked with her parents, school psych, school nurse... And we have come up with a beginning plan to have her meet weekly with our new health care counselor who has been funded through the Project Aware funding. The girl is excited about meeting with her and I noticed a change this week after she met with her. Happier, more confident, not as anxious...”

25

Connecting, Detecting, and Responding —For the Mental Health and Well-being of Washington Students