View
220
Download
0
Category
Preview:
DESCRIPTION
u Government Performance Results Act of 1993 (GPRA) u Annual Outcomes Report to Congress mandated by Adoption and Safe Families Act (ASFA) of 1997 u Statewide Data Indicators in Child and Family Services Reviews -- a subset of the Annual Outcomes—from National Child Abuse and Neglect Data System (NCANDS) and Adoption and Foster Care Analysis and Reporting System (AFCARS) u California Child Welfare System Improvement and Accountability Act (AB636) became law in 2001 and went into effect in January 2004 Outcomes, outcomes, everywhere
Citation preview
Measuring Child Welfare Agency Performance: Advantages and Challenges of State, County,
& University Collaboration
National Association of Welfare Research and Statistics45th Annual WorkshopMadison, Wisconsin
August 31, 2005
Kelly CrossSan Bernardino County HS
Terry V. ShawCenter for Social Services ResearchUniversity of California at Berkeley
The Performance Indicators Project is funded by the California Department of Social Services and the Stuart Foundation
Measuring Child Welfare Outcomes
Child In Child OutA bunch of stuff happens
*adapted from Lyle, G. L., & Barker, M.A. (1998) Patterns & Spells: New approaches to conceptualizing children’s out of home placement experiences. Chicago: American Evaluation Association Annual Conference
Government Performance Results Act of 1993 (GPRA)
Annual Outcomes Report to Congress mandated by Adoption and Safe Families Act (ASFA) of 1997
Statewide Data Indicators in Child and Family Services Reviews -- a subset of the Annual Outcomes—from National Child Abuse and Neglect Data System (NCANDS) and Adoption and Foster Care Analysis and Reporting System (AFCARS)
California Child Welfare System Improvement and Accountability Act (AB636) became law in 2001 and went into effect in January 2004
Outcomes, outcomes, everywhere
Quarterly distribution of county specific outcome
indicators data• Includes national standards (from AFCARS),
but also draws heavily on previous work done by CWDA and UCB using entry cohort measures
• Mirrors Family to Family Outcomes
• Retains key process measures (e.g., child visits, time to investigation)
Statewide Data Indicators from
AFCARS Stability Of Foster Care
Placement Length Of Time To
Reunification Foster Care Re-entries Length Of Time To Adoption
Why do we use entry cohort measures in addition
to measures from AFCARS?
Who is in AFCARS? AFCARS contains data on children in
foster care during a federal fiscal year
Each reporting period’s submission is a separate dataset. Reporting periods are linked together by the Children’s Bureau to form the annual databases. ANNUAL DATABASES ARE NOT LINKED TO EACH OTHER.
11/02 11/03 11/04
Data snapshots can be biased
Source: Aron Shlonsky, University of Toronto (formerly at CSSR)
California EXAMPLE: Age of Foster Children
(2003 first entries, 2003 exits, July 1 2003 caseload)(2003 first entries, 2003 exits, July 1 2003 caseload)
22
31
22 20
54
30
24 2219
5
24 24
32
16
0
10
20
30
40
50
<1 1-5 6-10 11-15 16+
Age in Years
Perc
ent
EntriesExitsPoint in Time
California EXAMPLE:Median Length of Stay in Months
(1998-2000 first entries, 2001 first spell exits, July 1 2002 first spell caseload)
1821
13
22
14
2126
17 1821
2528
2419
46
0
10
20
30
40
50
All Kin Foster FFA Group
Placement Type
Mon
ths
Entries
Exits
Point inTime
• Federal Measure: Of all children who were adopted during the year, what % had been in care for less than 24 months? (national standard = 32%)
• State enriched: Of all children entering care for the first time, what % are adopted in less than 24 months? (we do not have state standards)
Percent of children exiting care to finalized adoption in less than 24 months
(32% National Standard)
• Baseline: 100 kids exiting to adoption, 33 of them within 24 months=33%. Substantial conformity achieved!
Two pronged approach (1) Faster adoption for 100 children, 50 of them within 24 months=50%,
(2) adoptions for 100 kids in long term care
• 2 years later: 200 kids exiting to adoption, 50 within 24 months=25%. Substantial conformity NOT achieved!?!
Are you getting better or worse? Data from the Multi State Data Archive
Adoption within 24 Months
State A
0%
5%
10%
15%
20%
25%
30%
1990
1991
1992
1993
1994
1995
1996
1997
Ado
pted
FederalState
year
Source: Chapin Hall Center for Children
Why don’t we have state standards ?
The Cycle of Experiences in the Child Welfare The Cycle of Experiences in the Child Welfare SystemSystem
CounterbalancedCounterbalancedIndicators ofIndicators of
SystemSystemPerformancePerformance
PermanencyPermanencyThroughThrough
Reunification,Reunification,Adoption, orAdoption, orGuardianshipGuardianship
LengthLengthOf StayOf Stay
StabilityStabilityOf CareOf Care
Rate of Referrals/Rate of Referrals/Substantiated ReferralsSubstantiated Referrals Home-BasedHome-Based
Services vs.Services vs.Out-of-HomeOut-of-Home
CareCare
Positive Positive AttachmentsAttachments
To Family,To Family,Friends, andFriends, andNeighborsNeighbors
Use of LeastUse of LeastRestrictiveRestrictive
Form of CareForm of Care
Source: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare. Chapel Hill: Jordan Institute for Families,
Reentry to CareReentry to Care
Lack of understanding about the limitations of the national standards, and pressure to
achieve “substantial conformity” (pass), could
drive changes in policy and practice that may not be best
for children and families.
Kelly CrossSan Bernardino County HS
AB636 Components
• Quarterly distribution of county specific outcome indicators data• County Self Assessment• Peer Quality Case Review• County Self Improvement Plan• Continuous monitoring of outcomes
Limitations of Administrative Data
• Adapting a case management system into a mechanism for tracking longitudinal outcomes.
• A three-year cycle of outcome evaluation•Still are not ready to set a baseline•Must refine measurement methods, data clean up and training.
Examples of County Work to Examine Data and Improve Practice
• Changing policy•Social worker monthly contacts•Out of home abuse.
• Confounding policy and data issues•ICWA status•Recurrence of maltreatment
• Associated referrals• “Substantial risk,” & “Sibling abused, child at
risk”
Examples of County Work to Examine Data and Improve Practice
continued
• Complex outcomes—Required health visits•One of several complex measures in development
• Identify CHDP standards• Allow data entry lag• Revise code to account for different time
periods when children are in compliance
• An example of improved practice—Reentry•Examining data helped target those children
most at risk of returning to care after reunification
•Expanded use of Public Health Nurses
AB636=State / County Partnership
• Shifts focus from process measured compliance to outcome based review system, but requires linking outcomes to related processes.
• Data are our friends, not our dictators. • Requires county collaboration with community partners (SIPs signed by Boards of Supervisors).
• Promotes sharing of promising practices among counties.
More Advantages of University Involvement
• Participate regularly on state and county workgroups and committees.
• Share programming code and seek input from county partners on its continual improvement.
• Ensure availability to answer ad hoc
questions
Data and Policy Committees
• Continual refining of measurement process requires both Data and Policy Committees.
• Policy committee—interprets regulatory implications and decides general structure that measurement will conform to (e.g., inclusion of guardianship in measures).
• Data committee—determines specific data
collection & analysis steps necessary to implement measurement guidelines decided by policy group.
Examples of County Work to Examine Data and Improve Practice
continued
• Complex outcomes—Required health visits•One of several complex measures in development
• Identify CHDP standards• Allow data entry lag• Revise code to account for different time
periods when children are in compliance
• An example of improved practice—Reentry•Examining data helped target those children
most at risk of returning to care after reunification
•Expanded use of Public Health Nurses
UCB Website
cssr.berkeley.edu(Child Welfare Services Reports)
includes
age, ethnicity, gender breakoutskin vs non-kin
for all AB636 measures and more
use “Datadude” to examine performance over time
Lessons Learned
•It takes time.
•Keep accurate records of development & policy decisions. •Participant turn over (state, county, & university)
•Discovering new populations
State Websites http://www.dss.cahwnet.gov/cdssweb/
(Child Welfare Systems Improvements)
http://www.childsworld.ca.gov/CDSSCounty_1954.htm
(AB636 Quarterly Reports)
Kelly Cross kcross@hss.sbcounty.gov
(909) 388-0174
Recommended