View
219
Download
4
Category
Tags:
Preview:
Citation preview
CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis
Ed Byrnes, Ph.D.Eastern Washington University
Michael Lawson, M.S.University of California at Davis
Evaluation Questions
1. What is the CPS recidivism rate for Birth & Beyond families in comparison to similar families who were involved with CPS?
2. How does the time to onset of CPS recidivism differ between Birth & Beyond families and similar families who were involved with CPS?
Evaluation Design• Quasi Experimental• External Comparison Group• Fixed Follow Up Periods• Data Sources
– B & B Database– Sacramento County CPS Database
• Between Groups Comparability– Multiple Points of Comparison
• Outcomes– CPS Recidivism and Onset of Recidivism
• Client Predictors of Outcomes– Demographic Characteristics– Assessment Results
Birth and Beyond Group
• 384 B & B Families
• All Had A Prior CPS Referral
• All Had At Least One B & B Home Visit
• All Had A First Home Visit On Or Before July 11, 2006 For One Year Follow Up
CPS Comparison Group• 327 Families• NOT B & B Clients• All Had A First CPS Referral Between January
1, 1999 and July 11, 2006• Sample Trimming
– Children’s Ages– Most Severe CPS Allegation– Children’s Language and Race
• Stratified Random Sampling– Most Severe CPS Allegation– Children’s Gender, Language and Race
Between Groups Comparability• No Significant Differences in
– Children’s Race– Children’s Primary Language – Children’s Gender
• Very Unlikely to Influence Results• Children’s Ages Differed Significantly
– CPS Comparison Group Had More Neonates– CPS Comparison Group Was Then Older– Children’s Age and Group Membership Correlated
at .25, a Weak Relationship– Significance An Artifact of Sample Size of 711
Families• If Children’s Ages Have Any Influence on
Results It Is Quite Small
Between Groups Comparability
Child's Race x BB & CPS Groups
42%
28% 31%33%42%
25%
0%
25%
50%
75%
100%
White Black Latino
Race
Pc
t o
f G
rou
p
CPS Comparison BB Group
Between Groups ComparabilityChild's Language x BB & CPS Groups
90%
10%
97%
3%0%
25%
50%
75%
100%
English Spanish
Language
Pc
t o
f G
rou
p
CPS Comparison BB Group
Between Groups Comparability
Child's Gender x BB & CPS Groups
52% 48%45%55%
0%
25%
50%
75%
100%
Female Male
Gender
Pc
t o
f G
rou
p
CPS Comparison BB Group
Between Groups Comparability
Child's Age x BB & CPS Groups
0
1
2
3
4
0 25 50 75 100
Percentile
Ch
ild's
Ag
e
CPS Comparison BB Group
CPS Recidivism• Data From CPS Database
• All Referrals With A Definitive Finding– Substantiated, Inconclusive and Unfounded– Reflect Contact With The CPS System
• One Year Follow Up Period– From First Home Visit for B & B Group– From First CPS Referral for Comparison
Group– Contrast Voluntary Additional Services with
Usual CPS Processing
CPS Recidivism• Of the 384 B & B families, 140 (36%) had a new
CPS referral during the one year follow up period
• Of the 327 CPS Comparison Group families 159 (49%) had a new CPS referral during the one year follow up period
• This difference was statistically significant• B & B participation versus regular CPS services
was weakly correlated with CPS Recidivism (Phi = .12)
• Participating in B & B home visiting influenced CPS recidivism in the desired direction
CPS Recidivism1 Year CPS Recidivism
49%
36%
0%
25%
50%
75%
100%
CPS Comparison BB Group
Group
Pc
t w
/ Ne
w C
PS
Re
ferr
al
CPS Recidivism Onset
• Data From CPS Database
• Same Criteria As CPS Recidivism
• One Year Follow Up Period
• Survival Analysis
• Time to First CPS Referral– From First Home Visit for B & B Group– From First CPS Referral for Comparison
Group
CPS Recidivism Onset• Average time elapsed from the beginning of
services to their first CPS referral during the one year follow up period– For B & B families was 279 days (sd = 6.5)– For CPS Comparison Group families was 256 days
(sd = 7.45)• This difference was statistically significant• Relationship between group membership and
time to the onset of CPS recidivism was moderate-to-strong– Magnitudes of the test statistics
• Participating in B & B home visiting influenced the time to the onset of CPS recidivism in the desired direction
CPS Recidivism Onset
CPS Recidivism Onset
49%
36%
0%
25%
50%
75%
100%
30 60 90 120 150 180 210 240 270 300 330 360
Days From Start
Cu
mu
lati
ve
CP
S
Re
cid
ivis
m R
ate
CPS Comparison BB Group
CPS Recidivism Onset
• B & B and CPS Comparison Groups have similar patterns of cumulative CPS recidivism rates through the first 150 days
• After 150 days the cumulative CPS recidivism curves diverge
• The relationship between participating in B & B home visiting and a delayed onset of CPS recidivism is more beneficial as time progresses
Predicting CPS Outcomes• Additional Evaluation Questions
– What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within 150 days of the first home visit?
– What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within one year of the first home visit?
• Members of the CPS comparison group never engaged in B & B services they could not be included in these analyses
Predicting CPS Outcomes• Logistic Regression Primer
– Predict the occurrence of an event
• Risk and Protection– Can be quantified using Logistic Regression
• Odds Ratios– Value of 1 means no relationship between a
factor and an event– Value greater than 1 means a risk factor for
an event– Value less than 1 means a protective factor
against an event
Predicting CPS Outcomes• Outcome Variables
– New CPS Referral Within 150 Days– New CPS Referral Within 1 Year– Same Definitions As Recidivism and Onset
• Predictor Variables– Demographics– Services– Assessment
Predicting CPS Outcomes• Model Building Process• Two Stages
– Model by Variable Category for Each Outcome– Final Model for Each Outcome
• Criteria For Final Model Inclusion– Statistical Significance– Odds Ratio Greater Than 2
• Evaluating Models– Goodness of Fit, Improved Case Classification,
Outliers In The Solution v. Over-identification– Sample Size– Explained Variance: How Much Variability In The
Outcome Can Be Attributed To Variables In The Model – 20% or higher is acceptable
Predicting CPS Outcomes• Predicting 150 Day CPS Recidivism• No Service Variables Met Final Model Criteria• Predictors At Intake
– Being a first time parent– Being pregnant– Being under the age of 25– Being a member of the Black race– Being a member of the White race– Being a member of the Hispanic race– Meeting the criteria for clinical depression– Having a high total score on the APPI (indicating
more skillful parenting)– Having a low score on the MSSI (indicating a low
level of social support)
Predicting CPS Outcomes• Predicting 150 Day CPS Recidivism• 85 Cases With Complete Data• 36% Explained Variance• Met Other Model Quality Criteria• Risk Factors
– Pregnant at the Time of Intake– Being Black– Having Low Levels of Social Support
• Protective Factor– Being Under Age 25
Predicting CPS OutcomesRisk & Protection for 150 Day CPS Referral
13.2
0.1
3.22.0
-5.0
0.0
5.0
10.0
15.0
Pregnant atIntake
Under 25 Black* Low SocialSupport*
Factor
Od
ds
Ra
tio
RISK
PROTECTION
* = Not Statistically Significant
Predicting CPS Outcomes• Predicting 1 Year CPS Recidivism• No Service Variables Met Final Model Criteria• Predictors At Intake
– Being fluent in English– Being pregnant– Having a low total score on the APPI (indicating less
skillful parenting)– Experiencing severe domestic violence
• Predictors From Follow Up Assessment– Having a high score on the CAGE questionnaire
(indicating a higher likelihood of alcohol dependence)
– Experiencing severe domestic violence
Predicting CPS Outcomes• Predicting 1 Year CPS Recidivism• 93 Cases With Complete Data• 25% Explained Variance• Met Other Model Quality Criteria• Risk Factors
– Pregnant at the Time of Intake– Having a Higher Final CAGE Questionnaire
Score– Having Low Parenting Skills at Intake
• No Protective Factors
Predicting CPS OutcomesRisk & Protection for 1 Year CPS Referral
4.5
2.2
8.2
-5.0
0.0
5.0
10.0
15.0
Pregnant at Intake Low Total APPI Score* Final CAGE Points
Factor
Od
ds
Ra
tio
RISK
PROTECTION
* = Not Statistically Significant
Dynamic Risk
• Assessment Results Are Not Very Stable Over Time– Pretest – Posttest Correlations Are Weak To
Modest
• Risk and Protective Factors That B & B Clients Experience Are Dynamic Rather Than Static– B & B Home Visitors Must View Assessment
As An Ongoing Process
Dynamic Risk
Chronbach's Alpha and Test-Retest Reliability
Internal Consistency Test-Retest Reliability
pre post r
AAPI-2 0.76 0.81 .48 - .67
MSSI 0.71 0.70 .41 - .51
CES-D 0.82 0.84 .32 - .34
CPS Recidivism Results
• B & B participation was associated with lower aggregate CPS recidivism rates
• B & B participation was associated with a delayed onset of CPS recidivism
• Given the broad ecologies of child maltreatment, these results can be viewed as promising
Discussion Questions• What challenges have you encountered in
conducting quasi experimental evaluation research, and how have you addressed these?
• Since California has a county administered child welfare system what efforts can we make to encourage uniformity of data for greater generalizability of results?
Discussion Questions• What are the trade offs between using
matched (case control) designs versus designs that rely on inclusion and exclusion algorithms for comparison groups?
• Since California and it's counties, like so many other states and locales, are facing deep budget shortfalls, how can we continue to promote evidence based practice as a priority?
Dissemination Plan
• Incorporate feedback about our study from this symposium into a final manuscript.
• Submit the manuscript for publication.
• Peer reviewed journals being considered include:– Children and Youth Services Review;– Research on Social Work Practice.
Follow Up Contact
Ed Byrnes, Ph.D.
Email: ebyrnes@mail.ewu.edu
(509) 455-3422
Michael Lawson, M.S.
Email: malawson@ucdavis.edu
Recommended