Upload
tex
View
38
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Module 1 Slide deck: Child Welfare Data 101. Instructor Notes about this module. This module serves as an entry point for the entire course, it’s purpose is to: Help reduce student anxiety and resistance towards engaging in research and completing the course - PowerPoint PPT Presentation
Citation preview
Using Publicly Available Data to Engage IV-E Students in Research and Statistics: Instructional Modules
MODULE 1 SLIDE DECK:CHILD WELFARE DATA 101
2Module 1: Child Welfare Data 101 2
Instructor Notes about this module This module serves as an entry point for the
entire course, it’s purpose is to: Help reduce student anxiety and resistance towards
engaging in research and completing the course Provide an overview of California's child welfare data Introduce basic statistical concepts that are
informative, accessible, and relatively easy to generate
Increase students’ comfort level with data, thereby instilling confidence in their capacity as informed consumers of data and research
3Module 1: Child Welfare Data 101 3
Agenda Introduction and purpose/goals of the course Data basics:
Overview of sources and uses of data Sampled vs. population-level data Aggregate vs. micro-level data Longitudinal vs. cross-sectional data
Introduction to accessing and using California’s child welfare data
Understanding and computing basic descriptive statistics with child welfare data (e.g. variable descriptions, measures of central tendency, computing percentages)
Engaging Students in Research, possible slides for instructor use to introduce course
Opening Slides
5Module 1: Child Welfare Data 101 5
The Problem MSW students rank research courses as one
of their least favorite classes in the program (anecdotal…) Coursework feels disconnected from practice Most students do not enter with strong math or
stats backgrounds – high anxiety Timeline allows only a superficial coverage of
analytic methods Need to develop “statistical literacy”
Knowledge which enables people to think for themselves, judge independently, and discriminate between good and bad information (Dewey, 1930)
6Module 1: Child Welfare Data 101 6
Competing Course Models
informed consumers of data & research
junior social scientists
difficult (if not impossible) to do well even successful training is lost as not used in post-grad work
technology = data are everywhere!field is increasingly oriented around continuous improvement, outcomesstatistical literacy & critical thinking necessary for EBP and EIP
burdens agencies tasked with helping students access data
7Module 1: Child Welfare Data 101 7
Why we must be skilled consumers of research… Ethical obligation to our clients to be up to date on
the most recent research and skilled in its critical appraisal Cannot only rely on researchers to tell us the
relevant results and findings Necessary for effective advocacy and practice Helps with the efficient translation of research to
practice Critical that practitioners/researchers can speak
the same “language” in order to ensure future research efforts are relevant and that findings are understood in context and translated appropriately
8Module 1: Child Welfare Data 101 8
Managing by DataProvides social workers with the ability to: Compare metrics with agency mission
and practice model Connect to evidence-based practice and
desired outcomes Strategize on what work needs to be
done Focus on what is being achieved Identify what needs attention
9Module 1: Child Welfare Data 101 9
Connecting Data to Practice
Observe
Explain
Strategy
Outcome
We have noted that…
And believe
it is becaus
e..
So we plan to…
Which will
result in…
HYPOTHESIS: A HIGH LEVEL CAUSE AND EFFECT STATEMENT
IN OTHER WORDS…
Slide Developed by NY OCFS
1010
This Course One of the most important you will take here at X
(in my humble opinion) Focuses on practical skills, understanding data,
statistical literacy, consuming research Very connected to your current field placement –
and work post-graduation – yet helps you acquire empirical skills you may not otherwise have the opportunity to develop outside of your graduate studies
Expects that you read, ask questions, think critically, and engage with the material
Requires that you produce a relevant, readable, empirical research report based on publicly available administrative (secondary) child welfare data
11Module 1: Child Welfare Data 101 11
Administrative Data
Collected during the normal course of agency operations
Tabulated/aggregate data are publicly available Full coverage of populations served Free of “reactivity”(data problems are usually
transparent) Analysis of trends over time Performance indicators Social indicators Particularly salient to social work…
12Module 1: Child Welfare Data 101 12
Why a Secondary Analysis of Administrative Data? Data directly support agencies and capture information for
the clients we are working with Increasing emphasis on making these data available to
researchers Technological advances – data storage, frequent refreshes,
web-based/online access and analysis tools Non-intrusive (we work with vulnerable populations and
busy co-workers!) More transferrable to a post-graduate career in social work
Efficient, cheap, available Useful for advocacy efforts, needs assessment, proposals for
new programs (substantiate a service need) These are needed skills in both public and private agencies
13Module 1: Child Welfare Data 101 13
Realities of Administrative Data Analysis
Measures are often crude Sometimes limited documentation Missing data Do you trust what has been entered? Often much more difficult to analyze
than expected…requires careful thinking May be dated Definitions may have changed over time
Sources and Uses of Data
Module 1, Section 1
15Module 1: Child Welfare Data 101 15
Data Sources1. Census data2. Longitudinal surveys of a subsample of
a population3. Cross sectional surveys4. Longitudinal (multi-wave) surveys of a
single sample5. Administrative data6. Multisource data systems
1616
Uses of Data Descriptive
Demographic characteristics of a population, place, office, etc. Trends over time (one period compared to another) Differences/similarities between groups, counties, placement
settings, interventions, etc. Exploratory
Often conducted as pilot studies, attempt to examine feasibility issues (e.g., recruitment), preliminary data to develop fuller hypotheses and research proposals
Explanatory Analysis of the relationship between two events (or two
variables) Looking at the contributions of various factors to some outcomes
(y=a+bX) Evaluation
To evaluate social policies, programs, and interventions The evaluation process encompasses all three uses of data listed
above
17Module 1: Child Welfare Data 101 17
Data Terminology (Review) Qualitative vs. Quantitative
why, how vs. who, what, where, when “All quantitative data is based upon qualitative judgments; and all
qualitative data can be described and manipulated numerically.” (Research Methods Knowledge Base)
Longitudinal vs. Cross-Sectional repeated observations over time vs. a slice
Primary vs. Secondary you collect it vs. someone else collected it
Aggregate vs. Micro-level group tabulations vs. individual units
Deidentified vs. Identified Joe Smith vs. 987334
Sample vs. Census/Population partial vs. full coverage
Sample vs. Census/Population
Module1, Section 2
19Module 1: Child Welfare Data 101 19
Samples vs. Population Can we select a few people or things
for observation and then apply what we observe to a much larger group? Often impractical to gather data from the whole
population, so samples are drawn (this is not relevant to the administrative data we will be using in this course)
Key is the researcher’s ability to generalize findings from the sample to the whole population
Sample=a finite part of a larger population whose properties are studied to gain information about the whole
If all members of a population were identical in all respects – would we need careful sampling procedures?
population
sample
20
Sampling The act, process, or technique of selecting a suitable
sample, or a representative part of a population for determining the parameters (or characteristics) of the whole population
For a sample to provide useful information, it must reflect the same general variations as the overall population
NSCAW data = sample; CWS/CMS data = populationpopulation sample
Use characteristics/observations of sample, to draw conclusions (inferences) about the larger
population
Module 1: Child Welfare Data 101
Aggregate vs. Microdata
Module 1, Section 3
2222
Aggregated data
2323
Micro-data (individuals as units of analysis)
24Module 1: Child Welfare Data 101 24
Working with Aggregated Data…Disaggregate One of the most powerful ways to work with
data… Disaggregation involves dismantling or
separating out groups within a population to better understand the dynamics
Useful for identifying critical issues that were previously undetectedAggregate Permanency Outcomes
Race/Ethnicity
AgeCounty
Placement Type
25Module 1: Child Welfare Data 101 25
The Problem with Summary Statistics:
The average human has one breast and one testicle. *
* ~Des McHale www.quotegarden.com/statistics.html
2000 July-December First Entries California:
Percent Exited to Permanency 132 Months From Entry, by race and placement
Module 1, Section 4Longitudinal vs. Cross-Sectional (Point in Time) Views of Data
30Module 1: Child Welfare Data 101 30
Time Dynamics Cross-Sectional Studies
Examines a phenomenon by collecting/examining a “cross-section” of data at one time (one observation at a point in time) BIG problem: many questions we seek to answer aim to
understand causal processes that occur over time (e.g., children in foster care and mental health)
Longitudinal Studies Based on repeated observations of a given unit over
multiple points in time Trend Studies Cohort /Panel Studies
31Module 1: Child Welfare Data 101 31
Longitudinal Data1988 2006Birth 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Exit
X X X X X X X X X XX X X X X X X X X X X X X
X X X X X X X X X X X X X X X X
Age
Longitudinal Analysis
32Module 1: Child Welfare Data 101 32
3 Key Data Views in Child Welfare
Data
Entry
Cohorts
Exit Cohorts
Point in
Time
33
January 1, 2005 January 1, 2006July 1, 2005Child 1
Child 2
Child 3
Child 4
Child 5
Child 6
Child 7
Child 8
Child 9Child 10
The “View” Matters!
34
Longitudinal vs. Point in Time
Module 1: Child Welfare Data 101
35Module 1: Child Welfare Data 101
36Module 1: Child Welfare Data 101
California’s Child Welfare Data
Module 1, Section 5
38Module 1: Child Welfare Data 101 38
Why do we have these data? In 2001, the California Legislature passed the
Child Welfare System Improvement and Accountability Act (AB 636)
Designed to improve outcomes for children in the child welfare system while holding county and state agencies accountable for the outcomes achieved
The statewide accountability system went into effect January 1, 2004 It is an enhanced version of the federal oversight
system mandated by Congress to monitor states’ performance and provides the legal framework for the California Child and Family Services Reviews
39Module 1: Child Welfare Data 101 39
How are these data used? The foundation for this new oversight system comes
from data obtained from the Child Welfare Services/Case Management System (CWS/CMS)
Each quarter, the state provides county child welfare agencies with county-specific data on 14 outcome measures related to safety, permanency and well-being The baseline performance data is gathered for each county
and also made available to the public Quarterly reports provide counties with quantitative data
and serve as a management tool to track performance over time
Public data is also refreshed quarterly (data available on website is 3-6 months old)
4040
Where are the Data from?
4141
How are the public data configured?
unique characteristics and path through the child welfare system
personal info
aggregate data
tabulations
ageracegenderallegationplacementdispositioncountyyear
42Module 1: Child Welfare Data 101 42
What “type” of information is available?
first allegation of
maltreatment
allegation evaluated out
second allegation of
maltreatment
allegation substantiated
pre-placement family
maintenance services provided
child placed in out-of-home foster care
child reunified
third allegation of
maltreatment
child re-enters foster
care
43
Where can I find these data?
http://cssr.berkeley.edu/ucb_childwelfare
Univariate Statistics, child welfare examples
Module 1, Section 6
45
Variablesvariable
categorical
nominal
ordinal
continuous
exhaustive
mutually exclusive
variability in magnitude
(quantitative in nature)
finite number of values or categories (qualitative in nature)
46Module 1: Child Welfare Data 101
Continuous vs. Categorical The average foster child has 2.6 placements while in
foster care This number makes little sense because the underlying
dimension is discrete (i.e., categorical, discontinuous) 1 2 4 5 6
placements
x2.6 3
Continuous Data Discrete DataAge Days in Care Percentages / Rates
Race/Ethnicity Placement Type Referral Reason
47Module 1: Child Welfare Data 101 47
Descriptive Statistics “Summary” statistics, used to describe what’s
“going on” in our data Describe a situation or condition numerically
by quantifying phenomena In California, there are far fewer children in foster care today
than was true a decade ago. In California, the number of children in foster care today is
47,729 less than was true a decade ago, translating into a 49% decline.
Frequency tables (the distribution), measures of central tendency, measures of variability (the dispersion)
48
Computing a Percent
PERCENT: A proportion in relation to a whole expressed as a fraction of 100.
100totalpart100)(per percent %
100totalpart
100440290
100659.0
%9.65
100reunified # total
12m w/in reunified #
Raw Numbers (counts)
# Reunified w/in 12m# Reunified (total)
= 290= 440
What Percentage of Children reunified in 2005 were reunified within 12 months of entering care?
49
Computing a Rate per 1,000RATE: A proportion in relation to a whole, can be expressed as a fraction of 100, 1000, 100,000, etc.
1000totalpart
1000363,3761,333
100000366.
7.3
1000population child #
care entered #
Raw Numbers (counts)
# Entered Care
# Child Population
= 1,333= 363,376
What was the foster care entry rate in 2005? (i.e., how many children entered care out of all possible children in the population?)
1000totalpart1000per rate
Scales for a meaningful interpretation…
5050
Measures of Central TendencyMean: the average value for a range of data Median: the value of the middle item when the data
are arranged from smallest to largestMode: the value that occurs most frequently within the data
4.168
631715129744 Mean
5.102129 Median
4 Mode
7
= 9
= 9.7
12 4 15 63 7 9 4 17 4 4 7 9 12 15 17 63
51
Measures of Variability
Minimum: the smallest value within the dataMaximum: the largest value within the dataRange: the overall span of the data
4 Minimum
63 Maximum
59463 Range
4 4 7 9 12 15 17 63
51
52Module 1: Child Welfare Data 101 52
The Relationship between Mean and Variability Standard Deviation (represented by the
symbol σ) shows how much variation there is from the mean (average or expected value) Low standard deviation indicates that most
of the data points are close to the mean (less variation)
High standard deviation indicates that data points are spread out over a large range of values (more variation)
53Module 1: Child Welfare Data 101
Standard Deviation
symbol for standard deviation
Sum the difference between each “score” (xi ) and the overall mean (m)
Square the sum
Divide by the count of observations/scores minus 1
Take the square root
54Module 1: Child Welfare Data 101
Standard DeviationScores/Observations: 4 4 7 9 12 15 17 63Mean: 16.375
1. Find the distance between each value and the mean• 4-16.375 = -12.375• 4-16.375 = -12.375• 7-16.375 = -9.375• 9-16.375 = -7.375• 12-16.375 = -4.375• 15-16.375 = -1.375• 17-16.375 = 0.625• 63-16.375 = 46.625
2. Square all Values• -12.375 x -12.375 =
153.1• -12.375 x -12.375 =
153.1• -9.375 x -9.375 = 87.9• -7.375 x -7.375 = 54.4• -4.375 x -4.375 = 19.1• -1.375 x -1.375 = 1.89• 0.625 x 0.625 = 0.39• 46.625 x 46.625 =
2173.9
55Module 1: Child Welfare Data 101
Standard DeviationScores/Observations: 4 4 7 9 12 15 17 63Mean: 16.375
3. Sum the squared values (Sum of Squares: SS)
153.1+153.1+87.9+54.4+ 19.1+1.89+0.39+2173.9 = 2643.9
4. Divide the SS by the count of scores minus 1 (this gives you the Variance)
= 2643.9 / 7= 377.574. Take the square root of the variance (this gives you the
standard deviation)= = 19.4
56Module 1: Child Welfare Data 101 56
Percentage Points vs. Percent Change
Percentage point difference Absolute increase or decrease from one percentage value
to another (calculated by addition or subtraction) The percentage of children in foster family agency (FFA)
care increased by 12 percentage points between 1998 and 2012, from 15% to 27%
Percent change Relative change from one value to another (as a fraction
of the original amount) The proportion of children in foster family agency (FFA)
care increased by 80% between 1998 and 2012, from 15% to 27%
57Module 1: Child Welfare Data 101
Percent Change
Time Period 1
Time Period 2
10 children
12 children
10011 Period2 PeriodChange %
10011.2
1000.2
20%
1001kids 10kids 12
58Module 1: Child Welfare Data 101
Percent ChangeTime Period 1 Time Period 2
10% 12%
% %% %% %% %% %
% %% %% %% %% %
%%
100110%12%Change %
%20
59
Percent Change Calculation
Baseline Referral Rate (time period 1):
7.50100005067.963,637,9
419,488
Comparison Referral Rate (time period 2):
3.4810000483.199,988,9
706,482
Percent Change:
1001Rate Baseline
Rate Comparison
100150.748.3
%7.4100047.0
1001)-.9526(
50.7 48.3 -4.7%
12.0 10.8 -10%
Min
or D
iffer
ence
s du
e to
Ro
undi
ng…
Using Publicly Available Data to Engage IV-E Students in Research and Statistics: Instructional Modules
QUESTIONS? PLEASE CONTACT:
[email protected]@berkeley.edu