California Child Welfare Outcomes and Accountability Legislation:Evolving Toward System Improvement

with Longitudinal Data & Analysis

Panel on Increasing Impact and Redistributional Efficiency in Public Child Welfare

Daniel Webster, PhDBarbara Needell, PhD

Terry Shaw, PhD

Center for Social Services ResearchUniversity of California, Berkeley

Association for Public Policy Analysis & Management28th Annual Research Conference

Madison, WINovember 2, 2006

Slides in this presentation originally created by Barbara Needell, Lynn Usher, and Emily Putnam-Hornstein

The Performance Indicators Project at CSSR is supported by the California Department of Social Services and the Stuart Foundation.

Presentation Outline

• Background on Assembly Bill 636

• A Review of Progress–Relative Change on 20 Outcome Indicators–Multivariate Models on Permanency

• The Future of AB636

Point in Time

Exit Cohorts

Entry Cohorts

Data

3 Key Approaches to Data

Caseload Snapshots Versus Entry Cohorts

Jan. 1, 2005

Jan. 1, 2006Jan. 1, 2004

The Cycle of Experiences in the Child Welfare The Cycle of Experiences in the Child Welfare SystemSystem

CounterbalancedCounterbalancedIndicators ofIndicators of

SystemSystemPerformancePerformance

PermanencyPermanencyThroughThrough

Reunification,Reunification,Adoption, orAdoption, orGuardianshipGuardianship

ShorterShorterLengthsLengthsOf StayOf Stay

StabilityStabilityOf CareOf Care

Rate of Referrals/Rate of Referrals/Substantiated ReferralsSubstantiated Referrals Home-BasedHome-Based

Services vs.Services vs.Out-of-HomeOut-of-Home

CareCare

Maintain Maintain Positive Positive

AttachmentsAttachmentsTo Family,To Family,

Friends, andFriends, andNeighborsNeighbors

Use of LeastUse of LeastRestrictiveRestrictive

Form of CareForm of Care

Source: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare. Chapel Hill:  Jordan Institute for Families,

Reentry to CareReentry to Care

Assembly Bill 636Child Welfare Outcomes and Accountability System

• Legislation passed: October 2001

• Baseline Quarterly Report: January 2004

• Provisions of the Act– County self-assessments & self-improvement plans– Quarterly data reports to state and county officials– Public-private collaboration to support reform efforts– Longitudinal data publicly available

The Double-Edged Nature ofPublic Data

• PROS:– Greater performance accountability– Community awareness and involvement, encourages public-private

partnerships– Ability to track improvement over time, identify areas where

programmatic adjustments are needed- County/County and County/State collaboration

• CONS:– Potential for misuse, misinterpretation, and misrepresentation – Available to those with agendas or looking to create a sensational

headline– Misunderstood data can lead to the wrong policy decisions

14.0%

1.8%

4.1%

19.7%

12.5%Re- Entries to Foster Care (- )

Adopted w/ in 24m (+)

Reunified w/ in 12m (+)

1 or 2 Placements w/ in 12m (+)

Recurrence of Maltreatment (- )

California:AB636 Federal Measures,

Percent IMPROVEMENT from Baseline to Most Recent Report Period

Permanency

Safety

Note: (+) indicates a measure where a % increase equals improvement. (-) indicates a measure where a % decrease equals improvement.

17.5%

3.2%

8.0%

8.0%

6.3%

- 2.9%

8.7%

4.6%

4.0%

34.8%

12.0%

3.0%

9.3%

21.9%

Initial Placement Group/ Shelter (- )

I nitial Placement w/ Kin (+)

Placement with All Siblings (+)

Placement with Siblings (+)

Re- Entries w/ in 12m (cohort) (- )

Adopted w/ in 24m (cohort) (+)

Reunified w/ in 12m (cohort) (+)

1 or 2 Placements (at 12m, cohort) (+)

Rate of Children in Foster Care (- )

Rate of First Entry to Foster Care (- )

Recurrence w/ in 12m (- )

Recurrence w/ in 12m of First (- )

Substantiated Referral Rate (- )

Referral Rate (- )

California:AB636 State-Enhanced Measures,

Percent IMPROVEMENT from Baseline to Most Recent Report Period

Permanency

Safety & Participation

Note: (+) indicates a measure where a % increase equals improvement. (-) indicates a measure where a % decrease equals improvement. indicates a measure where performance declined.

Well-Being

Study Limitations & Next Steps

• Shortcomings of Administrative Data

• Study Time Period

• Examine All Performance Indicators

• Analyze Interplay Between Outcomes

• Further Specification for Multivariate Models

Whither AB636 ?

• Early Indication of Positive Change– Results of present study– Attitude shift in public child welfare staff

• A New Landscape with CFSR Round 2– Composites, components, & measures (oh my!)

– Rethink and re-tool quarterly report– Educate child welfare agency staff

• Turning Data into Knowledge– Familiarity/Use of data throughout agency– Making the link from outcome to practice* Developing human capital to use data