1

National EvaluationNational EvaluationData Profile ReportData Profile Report

Findings from the 24-month outcome studyOne Community Partnership

August 18, 2008

Adapted from work by the National Evaluation Team at Macro, International.

2

Key Principles: System of Care

Community-based Collaboration between multiple

service sectors Driven by family voices Individualized support based on

strengths and needs of child and family

Culturally competent Includes a system of ongoing

evaluation to assure data-based accountability.

3

Study Population

Enrolled in Longitudinal Outcom e Study

Enrolled in Dem ographic & Diagnostic Study

Henderson M ental Health Center Youth Case M anagem ent

OCP Target Population: SED Youth

Brow ard County Youth Age 10-18 200,000

20,000

2,000

487

266

Data sources: US Bureau of Census 2000; SAMHSA (estimates SED prevalence rate in children to be 9-13%); Henderson Mental Health Center Youth Case Management Monthly Activity Reports

4

MethodologyDemographic and Diagnostic Study

Describes children and families who enter the system of care and identifies their background factors and service needs.

Derived from data provided by case manager and chart reviews.

Includes 487 youth enrolled in ART, BART, or Connections between July 2004–September 2007 who consented to participate in the FMHI evaluation.

5

Demographic Profile: Youth

Sources:* Enrollment and Demographic Information Form** http://www.fldoe.org/eias/flmove/broward.asp

OCP(N=487)

Broward Schools EBD**(N=1,724)

White NH 41% 43%Black NH 36% 37%Hispanic 24% 16%Other NH 1.8% 3.6%

Numbers in the OCP category do not add to 100% due to differences in how data are collected. Non Hispanic multiracial (n=16) children are counted twice.

6Data source: Demographic and Diagnostic Study, Enrollment and Demographic Information Form

Geographic Distribution(n=482)

7

Insurance type(N= 487)

39%31%

11% 10% 7% 2%0%

10%20%30%40%50%

Med

icaid

Priva

teIn

sura

nce

SCHI

P

Non

e

SSI

Othe

r

Data source: Demographic and Diagnostic Study, Enrollment and Demographic Information Form

8

Presenting Problems at Intake* (n=487)

4%

5%

11%

12%

14%

24%

24%

33%

35%

38%

42%

59%

0% 10% 20% 30% 40% 50% 60% 70%

Specific DDPDD

Psychotic BehaviorSubstance abuse

Learning diffi cultiesSuicide I deation

AnxietyAdjustmentDepression

School perf ormanceHyperactivity

Conduct/ Delinquency

*Because children may have more than one

presenting problem, this chart adds to >100%.

Data source: Data source: Demographic and Diagnostic Study, Enrollment and Demographic Information Form

9

System Involvement at Intake (n=487)

Data source: Demographic and Diagnostic Study, Enrollment and Demographic Information Form

89%72%

19%7% 7% 5%

0%

20%

40%

60%

80%

100%

Mental H

ealth

School-

SPED

J uvenile

J usti

ce*

Child W

elfare

Special

medical

Substan

ce Abus

e

* includes Corrections, Juvenile Court, Probation

10

Referral Source, by Fiscal Year

Data source: Demographic and Diagnostic Study, Enrollment and Demographic Information Form

44%

14%

5%

24%

1%

41%

32%

2%7%9%

21%

4%6%

23%

41%

12% 6%

37%

26%26%

0%

10%

20%

30%

40%

50%

60%

MentalHealth

School J uvenileJ ustice

ChildWelfare

Caregiver

2004 (n=98) 2005 (n=196) 2006 (n=139) 2007 (n=54)

11

Household Risk FactorsHousehold member - history of depression 55% Household member –

history of mental illness (other than depression) 31%

Household member - history of substance abuse 40% Single-adult household 34%Exposed to domestic violence 42%

Risk Factors Prior to Intake (n=251-260)

Data source: National Evaluation Outcome Study, Caregiver Information Questionnaire-Baseline

12

94%

37%

99%91%

35%

98% 95%

42%

99%91%

42%

0%10%20%30%40%50%60%70%80%90%

100%

Caregiver Youth Other Family

2004 (n=93) 2005 Analysis (n=151) 2006 Analysis (n=120) 2007 Analysis (n=43)

Primary Participants in Wraparound

Meetings

Data source: National Evaluation Demographic Study, Enrollment and Demographic Information Form, Chart Reviews, as recorded in HMHC records

13

43%

6%2% 2%

9%

48%

10%

2% 2%5%

13%

1% 3% 2%

33%

19%

4% 2% 4%

42%

0%5%

10%15%20%25%30%35%40%45%50%

Therapist Education ChildWelfare

J J FamilyAdvocate

2004 (n=93) 2005 (n=151)2006 (n=120) 2007 (n=43)

Other Participants at Wraparound Meetings

(n=395)

Data source: Enrollment and Demographic Information Form, Chart Reviews, as recorded in HMHC records

Case manager attendance ranged from 94% in 2004 to 100% in 2007

14

Methodology Longitudinal Outcome Study

Assesses long-term impact of system of care.

Includes subset of demographic sample who agree to participate in longitudinal study.

Families are interviewed at 6-month intervals over a 3-year period.

For this report, youth/families were interviewed at baseline, 6, 12, 18 and 24-month intervals.

15

Stability of living situation – past 6

months

56%68% 68% 67% 77%

44%32% 32% 33% 23%

0%20%40%60%80%

100%

Intake(n=266)

6M(n=190)

12M(n=147)

18M(n=110)

24M(n=92)

1 2 or more

16

School Attendance Intake - 24 Months

(n=44)

50%

14%

36%47%

22%30%

0%

20%

40%

60%

Improved Maintained Declined

OCP National

Data source: Education Questionnaire-Revised

17

Grades –Intake to 24 months

(n=54)

42%31% 27%

45%

30% 26%

0%10%20%30%40%50%

Improved Remained stable Declined

OCP National

Data source: Education Questionnaire-Revised

18Data source: Education Questionnaire-Revised

Disciplinary Actions at Intake

55%

43%

3%

50% 46%

5%

0%

20%

40%

60%

Neither Suspended Expelled

OCP Intake (n=141) National I ntake

19

Disciplinary Actions at 24 months

66%

32% 25%

68%

30%

3%0%

20%

40%

60%

80%

Neither Suspended Expelled

OCP (n=73) National

20Data source: Delinquency Survey-Revised

Juvenile Justice Involvement Arrests Intake - 24 Months

48% 44%37%

26% 30%

9%8%9%10%22%

0%10%20%30%40%50%60%

I ntake(n=81)

6months(n=45)

12months(n=38)

18months(n=27)

24months(n=20)

Perc

ent o

f You

ths

OCP National

21Data source: Delinquency Survey-Revised

Juvenile Justice Involvement On Probation Intake - 24 Months

31%

48%

34% 33%

15%13%13%17%14%17%

0%

10%

20%

30%

40%

50%

60%

I ntake(n=81)

6 months(n=45)

12 months(n=38)

18 months(n=27)

24 months(n=20)

Perc

ent o

f You

ths

OCP National

22

Delinquent Behaviors Intake - 24 Months

0%5%

10%15%20%25%30%35%40%45%

Intake 6M 12M 18M 24M

Hit someone

Out of control

Bullied

Damaged property

Skipped school

In the past 6 months, have you . . . ?

Data source: Delinquency Survey-Revised

23

Self-reported drug or alcohol use

(previous 6 months)28% 29%

26%32% 28%

0%5%

10%15%20%25%30%35%

Intake(n=248)

6 months(n=163)

12 months(n=115)

18 months(n=91)

24 months(n=74)

Data source: GAIN Quick-R

According to the 2006 National Survey on Drug Use and Health 3.9% of 12-13 year olds, 9.1% of 14-15 years and 16% of 16-17 year olds used illicit drugs in the previous. Source: http://www.oas.samhsa.gov/nsduh/2k6nsduh/2k6Results.pdf

24

Degree of Substance ProblemsIntake - 24 Months

41%33%

60%55% 57%

53%58%

30% 31%38%

6% 8% 10% 14%5%

0%

10%

20%

30%

40%

50%

60%

70%

Intake(n=70)

6 months(n=48)

12 months(n=30)

18 months(n=29)

24 months(n=21)

Data source: GAIN Quick-RUrgency of substance problems are only calculated for youth that used in the previous six months.

Higher scores indicate greater dependence.

No/minimal

Moderate

High

25

Internalizing Problem Behaviors and Symptoms Intake - 24 Months

56586062646668

Intake 6 Months 12 Months 18 Months 24 Months

OCP National

Data sources: Total Problem T-score: Child Behavior Checklist (6-18)

Clinical range

Ns range from 261 at Intake to 89 at 24 months

26

Externalizing Problem Behaviors and Symptoms Intake - 24 Months

60

65

70

75

Intake 6 Months 12 Months 18 Months 24 Months

OCP National

Data sources: Total Problem T-score: Child Behavior Checklist (6-18)

Clinical range

Ns range from 261 at Intake to 89 at 24 months

27

23% 20%15% 13% 12%

31% 29% 26% 26% 23%

0%10%20%30%40%

Intake 6 Months 12Months

18Months

24Months%

with

hig

h le

vels

of

anxi

ety

OCP National

AnxietyIntake - 24 Months

Data sources: Anxiety-Revised Children’s Manifest Anxiety Scale.Total T-scores>60 indicate high levels of anxiety on the RCMAS.

Ns ranged from 248 at Intake to 74 at 24 months

28

DepressionIntake - 24 Months

19% 17% 12% 11% 8%

21% 18% 17% 16% 13%

0%

20%

40%

60%

80%

100%

I ntake 6 Month 12 Month 18 Month 24 Month

% sc

oring

in cl

inica

l ran

ge

OCP National

Data sources: Depression-Reynold’s Adolescent Depression Scale 2. Total T-scores > 60 indicate clinical levels of depression on the RADS. Ns ranged from 248 at Intake to 74 at 24 months

29

Caregiver Perspectives of Youth’s Behavioral and Emotional Strengths

Intake - 24 Months

78 80 80 82 827983 85 85 85

50

70

90

110

Intake 6 Months 12 Months 18 Months 24 Months

Aver

age Sc

ore

OCP National

Data Source: Behavioral & Emotional Rating Scale-Caregiver and Youth

Higher scores indicate greater strengths.

Average range

30

Youth’s Perspectives on their Behavioral and Emotional Strengths

Intake - 24 Months

9298 97 97

10193 95 96 97 98

50

70

90

110

Intake 6Months

12Months

18Months

24Months

Aver

age Sc

ore

OCP National

Data Source: Behavioral & Emotional Rating Scale-Caregiver and Youth

Higher scores indicate greater strengths.

Average range

31

Caregiver Global Strain Intake - 24 Months

9.0

8.27.8

7.37.1

9.0

8.08.08.0

8.0

7.07.58.08.59.09.5

10.0

I ntake 6Months

12Months

18Months

24Months

Aver

age

Scor

e

OCP National

Lower scores indicate higher functioning

NOTE: Global Strain includes stresses due to resources (e.g., time & money), and feelings such as anger or resentment, guilt or fatigue due to caregiving responsibilities. Data Source: Caregiver Strain Questionnaire

OCP Ns range from 263 at Intake to 89 at 24 months

32

Service definitionsCase management or service coordination involves finding and

organizing multiple treatment and support services, and may also include preparing, monitoring, and revising service plans; and advocating on behalf of the child and family. Case managers may also provide supportive counseling.

Individual therapy relies on interaction between therapist/clinician and child to promote psychological and behavior change.

Medication treatment and monitoring services typically include the prescription of psychoactive medications by a physician (e.g., psychiatrist) that are designed to alleviate symptoms and promote psychological growth. Treatment includes periodic assessment and monitoring of the child’s reaction(s) to the drug.

33

Child Services Received6 - 24 Months

Data Source: Multi-Sector Service Contacts-Revised-Caregiver

92%

80%

51%

71% 73%

65%

49%

66% 68%61%

66%

50%

0%

20%

40%

60%

80%

100%

Case Mgmt I ndividual Therapy Med Mgmt

6 Months (n=179) 12 Months (n=112)18 Months (n=77) 24 Months (n=56)

34

Service definitionsFamily therapy involves a variety of family members such as caregivers

and/or siblings with or without the child present. Interaction among family members is typically facilitated by a therapist or counselor.

Caregiver or family support services are provided to caregivers or siblings (e.g., family activities, behavior management training, parent classes, support groups, individual therapy for caregiver or other family members). These do not recreational activities, behavioral/therapeutic aide, transportation services, respite care, after-school activities or child care which are described in other questions.

Informal supports are defined as assistance from persons who provide support to the family without compensation from any formal service system. This type of support includes asking a relative or friends to baby sit a child, support received from someone’s church, etc.

35

Family Services Received6 - 24 Months

Data Source: Multi-Sector Service Contacts-Revised-Caregiver

40%

21%

9%

43%

21%14%

31%

17%9%

34%

9% 7%

0%

20%

40%

60%

80%

100%

Family Therapy Family Support I nformal Support

6 Months (n=178) 12 Months (n=112)18 Months (n=77) 24 Months (n=56)

NOTE: Family therapy is typically facilitated by a therapist or counselor. Family support may include support groups, parenting classes,behavior management training for caregivers, etc. Informal support is assistance provided without compensation from any formal service system (e.g., support from friends, church, etc.).

36

Cultural Competency at Intake and 24 Months

55%

80%93% 90% 96% 98%

74% 74%89% 85%

96% 98%

0%20%40%60%80%

100%

Usedknowledge of

family'sculture tomeet our

needs

Told me howto get

additionalservices when

needed

Respectedfamily'sreligiousbeliefs

Understoodmy needs

Spoke ourlanguage or

gotinterpreters

Treated uswith respect

6M (n=166) 24M (n=54)

My primary service provider:

Data Source: Cultural Competence and Service Provision-Caregiver

37

3.53.9 4.2 4.1

4.53.9 4.0 3.8

4.14.4

00.511.522.533.544.55

Outcomes Satisf actionwith Services

Participationin Treatment

Access toServices

CulturalSensitivity

Caregiver (n=51) Youth (N=40)

Caregiver and Youth Perspectives on Services at 24 Months

Very Dissatisfie

d

Very Satisfied

Scores range from 0 (strongly negative) to 5 (strongly positive). Scores above 3.5 are regarded as positive

Data source: Youth Services Survey for Families-Caregiver and Youth Services Survey

38

3.5

3.9

4.2

4.1

4.5

3.9

4.0

3.8

4.1

4.4

0 1 2 3 4 5

Outcomes

Satisfaction with Services

Participation in Treatment

Access to Services

Cultural Sensitivity

Caregiver (n=51) Youth

Caregiver and Youth Perspectives on Services at 24 Months

Very Dissatisfie

d

Neutral Very Satisfied

Scores range from 0 (strongly negative) to 5 (strongly positive).

Data source: Youth Services Survey for Families-Caregiver and Youth Services Survey

39

Key Child Outcomes24 months after entering System of Care

• Most youth continue to live at home with their families.

• Youth’s behavioral and emotional functioning improved and remained stable over time:

At home: decrease in symptomatology, anxiety, and depression;

At school: improvement in grades and attendance; and

In the community: decrease in common delinquent behaviors and arrests

40

Key Family Outcomes24 months after entering System of Care

• Caregivers report less strain over time.

• In general, families were satisfied with the services they received.

41

• Use data to drive community decisions.

• Use data to drive ongoing fidelity of process in terms of training, cross system involvement, etc

• Continue to challenge OCP Oversight Committee to involve families and youth in identifying what families need.

•Support Family Voices to continue family, youth and informal support involvement at all levels of decision making (e.g., governance & family team service planning)

Recommendations