Upload
maxim-munday
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Fostering a Permanent Home:
A Mixed Methods Evaluation of the ZERO TO THREE Court Teams for Maltreated Infants and Toddlers Initiative
Kimberly L. McCombs-Thornton, PhDUniversity of North Carolina at Chapel HillDecember 10 , 2011
Outline
• Background and significance
• ZTT Court Teams program
• Quantitative study
• Qualitative study
• Limitations and Discussion
2
Background and Significance
• Infants experience highest rate of victimization; toddlers second – Infants (< age 1): 20.6 per 1000 – Toddlers (age 1): 11.9 per 1000– Toddlers (age 2): 11.3 per 1000
• Developmental needs of young children are time sensitive
• The caregiver is important for the child’s healthy development
• Infants and toddlers in child welfare are at risk for– Attachment disorders – Poor physical health
3
Background and Significance
• The Adoption and Safe Families Act (ASFA) of 1997 seeks to expedite the time to reach a permanent, stable home for children in child welfare
• A permanent home may include:– family reunification– legal guardianship– or traditional adoption
• More children appear to be reaching permanency since ASFA’s enactment, though the need is still great– Adoptions rose 64.5% in the three years following ASFA’s enactment – Represents only 28% of children eligible for adoption during these years
4
ZTT Court Teams Project
Judge
Community Coordinator
Court Team-DHS case worker-CASA-Attorneys-Providers-Etc.
Monthly Case Reviews
Referral to Child-focused Services
Child-Parent Psychotherapy
Evidence-Based Parent Education
ZTT National Office Activities
-Training and TA-Resource materials-M and E
Core Components
Decrease in Time to Permanency
Reduction in Recurrence of Maltreatment
Improved Child Well-being
LOCAL ACTORS ACTIVITIES LONG-TERM OUTCOMES
TARGET GROUP: CHILDREN ENTERING FOSTER CARE BEFORE AGE 3
5
Previous Evaluation
• JBA completed evaluation in 2009– Process oriented– 95% of closed cases reached permanency– 55% of those reached permanency within one year
• No comparison Problem of causal group inference
6
Study AimsAim 1: Determine the effect of the ZTT Court Teams project on time to permanency.
Aim 2: Assess the influence of the ZTT Court Teams program on how children exit the foster care system.
Aim 3: Examine successful and unsuccessful cases to understand how program components and/or client characteristics contribute to time to permanency.
7
Causal Inference
Q: How do we know outcomes are due to the program?
A: Need a comparison group
Basic Advanced
Pre/post Secondary Matched Randomized Test Data as Comparison Experiment Comparison Group (w/ control group)
8
Quantitative Data
ZTT “Treatment” Group
Dataset:
Court Teams MIS
Data Source: CPS documents, observations in court, etc.
What collected: Child background, service needs and usage, visitation, placements, case status
Study sample: All children in first 4 ZTT sites who entered by end of 12/2009
Number in study:
298 children
Follow up period: One year or more for 94% of cases
9
Quantitative Data
ZTT “Treatment” Group
NSCAW Comparison Group
Dataset: Court Teams MIS National Survey of Child and Adolescent Well-Being (NSCAW)
Data Source: CPS documents, observations in court, etc.
Interview with CPS case worker
What collected: Child background, service needs and usage, visitation, placements, case status
Family risks, child living environments, services needed and received, child behavior, etc.
Study sample: All children in first 4 ZTT sites who entered by end of 12/2009
Children who entered child welfare supervised out of home placement before the age of 3
Number in study:
298 children 511 children
Follow up period: One year or more for 94% of cases
One year or more for all cases 10
Case Characteristics ZTT
(n=298) NSCAW (n=511)
Child Characteristics
Age of child at first out of home placement Mean (months) Infant
9.3 (1.0) 67% (.04)
11.0 (.99) 57% (.04)
Child gender Male
50% (.02)
46% (.04)
Race/Ethnicity African American Caucasian Latino/a
37% (.10) 29% (.10) 14% (.10)
24% (.04) 41% (.05) 18% (.05)
Parent Characteristics Substance Abuse 72% (.06) 59% (.05) Severe Mental Health Needs 17% (.03) 21% (.04) Poverty 95% (.02) 81% (.03) Lack of Employment
35% (.15) 47% (.05)
11
Propensity Score
Propensity score = association between covariates and group membership
12
Selection into ZTT Court Teams
How child selected within a site:- Child under age of 3 at time of entry into child welfare
system- Either all young children assigned to Court Teams judge or
randomly assigned
13
14
Predictors of Time to Permanency
Reasons for Removal
Community Characteristics
Child Characteristics
ParentCharacteristics
Aim 1
Determine the effect of the ZTT Court Teams project on time to permanency.
15
Aim 1
2 Outcome Measures:– Time to “move in” permanency– Time to “official” permanency
Methods:– Time to event analysis with propensity score weights– Test for proportional hazards– Diagnostics
16
Aim 1
Time to “move in” permanency
17
Permanency ZTT (n=298)
NSCAW (n=511)
Move In
Median (SE) 168 (10.5) 210 (25.6)
Mean (SE) 343 (32.1) 398 (23.4)
ZTT cases move into what becomes the permanent home 1.5 to 2 months faster on average
Aim 1Time to “move in” permanency- Kaplan Meier
- Time to Event AnalysisParametric model (Gompertz): HR = 1.06 (CI .67, 1.65)
p=.81
0.00
0.25
0.50
0.75
1.00
0 500 1000 1500 2000Days until move into what becomes the permanent home
ZTT Court Teams NSCAW
Kaplan-Meier survival estimates: Time to Move in Permanency
18
Aim 1
Time to “official” permanency
19
Permanency ZTT (n=298)
NSCAW (n=511)
Official
Median (SE) 383 (10.8) 762 (26.8)
Mean (SE) 466 (20.5) 825 (18.5)
ZTT cases exit foster care one year earlier on average
Aim 1Time to “official” permanency- Kaplan Meier
0.00
0.25
0.50
0.75
1.00
0 500 1000 1500 2000Days until officially discharged from foster care
ZTT Court Teams NSCAW
Kaplan-Meier survival estimates: Time to Official Permanency
20
Aim 1
Time to “official” permanency
- Time to Event Analysis
ZTT cases exit foster care nearly three times as fast as the
comparison group
21
Parametric model (Weibull): HR = 2.67 (CI 1.65, 4.31) p=.000
Aim 2
Assess the influence of the ZTT Court Teams program on how children exit the
foster care system.
22
23
Aim 2
4 ways to exit foster care:
- Reunification
- Adoption
- Relative custodianship
- Non-relative legal guardian
fastest
slowest
Aim 2Type of
Foster Care ExitZTT Court Teams
(n=298)NSCAW sample
(n=511)
pct se pct se
Reunification 37.6% (.047) 29.3% (.042)
Adoption 15.4% (.059) 40.7% (.045)
Relative custodian 24.8% (.085) 8.4% (.033)
Non-relative guardian 3.0% (.017) 1.6% (.007)
Still in foster care at end of study period
19.1% (.068) 20.1% (.036)
24
Aim 2Days until Exit Foster Care, by Type of Exit
25
Type of Exit from Foster Care
ZTT (n=298)
NSCAW (n=511)
Reunification Median 309 547 Mean 340 587
Adoption Median 464 764
Mean 496 800 Relative custodian Median 351 450
Mean 363 487 Non-relative guardian Median 481 878
Mean 467 780
8 months faster on average
10 months faster on average
3-4 months faster on average
10-13 months faster on average
Patterns hold when use propensity scores in a competing risks analysis
Aim 3
Examine successful and unsuccessful cases to understand how program components
and/or client characteristics contribute to time to permanency.
26
Aim 3
Examine successful and unsuccessful cases to understand how program components
and/or client characteristics contribute to time to permanency.
27
Aim 3Methods
• One-on-one open-ended phone interviews with each community coordinator; 2 interviews each
• Approach orientated to unique cases– discuss top and bottom decile in each site (Patton, 1990)
• Community coordinator’s perception of how each component of the program model contributed time to permanency for each case
• Record and transcribed calls
• Coded using Atlas.ti
28
Aim 3
Parents comply with service plan?
Yes
Reunification
“[The mother] is absolutely amazing that, what is she, 20 years old, she had two children with special needs that really require a lot of doctors visits. So she was able to show all the professionals in the system that she was able to follow through with all of these things. And at the same time, she was dealing with her own victimization issues and poor relationship choices…..The children were [ultimately] reunified with the mother.”
29
Aim 3
Parents comply with service plan?
Yes No
Reunification Terminate/Surrender/SuspendParental Rights
Adoption Legal Guardian
“The mother would come into court every time positive for cocaine, refused to get any help….. She didn’t approach it [the service plan]. She just didn’t do anything….This child was only placed in one foster home… [who] adopted the child.” 30
Aim 3
Parents comply with service plan?
Yes NoSomewhat
Reunification Terminate/Surrender/SuspendParental Rights
Adoption Legal Guardian
“An inpatient drug treatment program was recommended for her. She went to one and she completed it. And then when she came out of course they recommended outpatient care and AA meetings and so forth. She started taking those and then slacked off. So that was a discussion in every hearing. Was she or was she not in compliance with the aftercare recommendation….”
31
Aim 3
Parents comply with service plan?
Yes NoSomewhat
Reunification Terminate/Surrender/SuspendParental Rights
Adoption Legal Guardian
Parent attitude/willingness
SOCIAL SUPPORT INFLUENCES
Family, friends, foster parents, and others to provide: transportation, housing, visitation supervision, financial support, care for children, emotional support, etc.
Judge:- monitor case progress- verbal support when parents comply- verbal warning when parents not complying- timekeeper
Service providers:- attempt to meet needs, teach
skills, stabilize condition
Department of Social Services:- design service plan- assistance seeking services- visitation supervision
Parent behavior :- substance abuse issues- mental health issues- relationship issues
Parent personal resources:- education/job skills- poverty- transportation
32
Aim 3
Role of Judge in Time to Permanency:
• Motivate the parents
“[The judge] was very supportive and really wanted the children with their mother. And you know basically would encourage her and would actually praise her and tell her she was doing a good job and tell her to keep it up…assuring her we were going in the right direction… [The judge] is very good about praising when you’ve made progress on your service plan and you’re doing what you’re supposed to do.”
“[The judge] became increasingly frustrated… and really confronted the mom on how the professionals were working harder than she was and her inability to take ownership for her part that she played in this. And confronting mom on not understanding the impact it had on the children.” 33
Aim 3
Role of Judge in Time to Permanency:
• Motivate social support network
• Motivate the case workers
• Role as timekeeper
34
Aim 3
Role of Monthly Case Review in Time to Permanency:
• Keeping all on task“Everybody stayed on task because they knew we were gonna be staffing and we were going to be in court. So there was no room for making, for example, making a referral a week before we go to court because we were always going to court.”
“Usually CPS cases, the hearings are every 90 days. But with this, the parents know that they have to be in court every month. It gets them motivated to get on the ball so they don’t have to go to the judge in 30 days to explain to the judge why they haven’t done what they are supposed to have done 30 days prior. “
• Monitor parental compliance
35
Aim 3
Fast v. Slow case comparison
• Program is consistent across the cases- All have monthly case reviews- Judge has equal contact with cases
36
Aim 3Activity to Strengthen Results Reliability Validity
Within a Site
Used same questions 5 to 6 times in one interview re: fastest cases
Used same questions 5 to 6 times in a second interview on a different day re: slowest cases
Used same questions for fast and slow cases
Validation interviews – with one other professional in each site
Negative case analysis to search for examples contrary to findings
Across Sites Asked same questions across the sites a total of 46 times
Compared findings to the James Bell Associates Court Teams evaluation
Mixed methods – quantitative analysis
37
Conclusion• The ZTT Court Teams program reduces time to official permanency, but not time to move in permanency
• More kids are exiting foster care through reunification and less through adoption
• Though – ZTT Court Teams cases are exiting faster regardless of type of exit
• The program operates consistently for both “fast” and “slow” cases
• The judge and the monthly case review components appear to be most linked to time to permanency
38
Limitations and Next Steps
Quantitative study• Propensity score subject to omitted variable bias
• Only considers child’s first episode in child welfare
• Assess effect of number of court hearings
• Are all types of exits equal?
• Next step – cost effectiveness study
39
Limitations and Next Steps
Qualitative study• Only one person coded and analyzed data
• Sorted by time to move in permanency
• Collected before completed quantitative analysis
• Site differences – fidelity to the model?
40
Reflections
Judge Constance CohenPolk County, Iowa
Judge Ernestine GrayOrleans Parish, Louisiana
Judge Michael McPhailForrest County, Mississippi
41