Use of the NCSEAM Preschool and Part C Surveys

to Address the SPP/APR Parent/Family Indicators

Batya Elbaum, Ph.D.University of Miami

National Center for Special Education

Accountability MonitoringNational OSEP Early Childhood

ConferenceWashington, D.C.

December 13, 2005

SPP/APR Part C Indicator

“Percent of families participating in Part C who report that early intervention services have helped the family a) know their rights, b) effectively communicate their children’s needs, and c) help their children develop and learn.”

SPP/APR Part B Indicator

“Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities.”

Steps in addressing the SPP/APR parent/family indicators

Select a measurement tool Establish a sampling plan Select mode of administration Work out details of the data

collection process Collect and analyze data Create reports

NCSEAM Parent/Family Involvement Workgroup

Began work in January 2002 to develop a set of valid, reliable, and useful measures of families’ perceptions and involvement in early intervention and special education.

Current status of the NCSEAM Parent/Family Measures

The NCSEAM survey instruments for Part C and Part B school-age were finalized in August 2005

The NCSEAM survey instrument for 619 will be finalized by February 2006

Item development Items were drawn from:

existing survey instruments descriptions of best practices in parent

involvement and family-centered services experts in the field

Stakeholder workgroups were conducted in 6 states (MS, NH, CA, NM, KY, FL)

500 suggested items were reviewed by PACER and other parent groups

Item validation The NCSEAM National Item Validation

Study was conducted between October 2004 and February 2005

The Part C sample consisted of ~ 1750 families from 8 states

The Part B sample consisted of ~ 2500 parents from 6 states, including 71 parents of preschool children

Different but linked item sets were used for Part B and Part C

Further development of items for 619

A 619 workgroup was convened in conjunction with the February 2005 OSEP Early Childhood Conference

Participants suggested additional items and wording changes to ensure that items were appropriate for parents of preschool children

Feedback was solicited via the 619 listserv in spring 2005

Overall, input was provided by 619 staff from over 25 states

Further development of items for 619

Additional feedback was provided by the NCSEAM hParent/Family Involvement Workgroup and a local 619 parent-professional team

Data from Part B parents who participated in the National Item Validation Study were analyzed separately for parents of children K-12 and parents of children receiving preschool special education services

What do the items measure?

Items included in the NCSEAM National Item Validation Study are not all “about” one thing.

They reflect several different dimensions of parents’ perceptions and involvement.

Each dimension is measured by its own set of items.

Each separate set of items constitutes a scale.

What are the dimensions?

Parents’ perceptions of school/program efforts to engage them in meaningful partnerships

Parents’ perceptions of the quality of services provided to their child and family

Parents’ reports of their participation in the early intervention/special education process

Parents’ perceptions of the impact of services on their family

Parents’ perceptions of the impact of services on their child*

Scales for Early Intervention Parents’ perceptions of programs’

partnership efforts and quality of services

Parents’ perceptions of the impact of services on their family

Scales for 619/preschool

Parents’ perceptions of school/program partnership efforts and quality of services

Parents’ perceptions of the impact of services on their family

Parents’ reports of their participation in the preschool special education process

Scales for Part B school-age

Parents’ perceptions of school partnership efforts

Parents’ perceptions of the quality of services provided to their child

Parents’ perceptions of the impact of services on their family

Parents’ reports of their participation in the special education process

Major finding from the National Item Validation Study

Within each scale, the items showed virtually the same order of agreeability for all respondents.

The consistent ordering of items in a scale is the foundation of good measurement.

An example of scale invariance: Dr. B’s Running Fitness Survey

Stand for 5 minutes. Yes Maybe No Walk across a 20-foot room. Yes Maybe No Walk 2 blocks. Yes Maybe No Run 2 blocks. Yes Maybe No Run 2 miles. Yes Maybe No Run 5 miles. Yes Maybe No Run 13.1 miles (half-marathon). Yes Maybe No Run 26.2 miles (marathon). Yes Maybe No Run 66.6 miles/100 k (ultra) Yes Maybe No Run 100 miles. Yes Maybe No Run 250 miles. Yes Maybe No Run 300 miles. Yes Maybe No

Confirmation of Invariance Across alternate forms Across subgroups defined by:

State ethnicity age of child language in which the survey was

administered (English vs. Spanish) method of administration (self-

administered or items read by a facilitator)

Across C, 619, and B school-age (for items with similar content)

Sample 619 Efforts/Quality Items

People from preschool special education, including teachers and other service providers: provide me with information in a

language I understand. give me enough opportunities to

discuss my child’s progress. encourage me to participate in the

decision-making process.

Sample 619 Efforts/Quality Items

We discussed special help my child would get so he/she could learn with children without disabilities.

People from preschool special education are good at working with families.

I am considered an equal partner in planning my child’s preschool special education.

Sample 619 Efforts/Quality Items

People from preschool special education, including teachers and other service providers: provide services to my child in a timely

manner. give me information about the

instructional methods used with my child.

Someone explained to me what options parents have if they disagree with a decision made by people from the preschool special education program.

Sample 619 Efforts/Quality Items

People from preschool special education, including teachers and other service providers, give me options concerning my child’s services and supports.

I was offered special assistance (e.g., child care or transportation) so that I could participate in the IEP/IFSP meeting.

People from preschool special education connect families with organizations that serve parents of children with disabilities.

Sample Part C Impact on Family Items

Over the past year, early intervention services have helped me and/or my family:

Understand my child's special needs. Do things with and for my child that are

good for my child's development. Be more optimistic about my child's

future. Feel that my efforts are helping my child.

Sample Part C Impact on Family Items

Over the past year, early intervention services have helped me and/or my family:

Communicate more effectively with the people who work with my child and my family.

Feel more confident in my skills as a parent.

Understand the roles and responsibilities of the people who work with my child and family.

Sample Part C Impact on Family Items

Over the past year, early intervention services have helped me and/or my family:

Know about my child's and family's rights concerning special education services.

Know where to go for help or support to meet my child's needs.

Make changes in our family routines that will benefit my child with special needs.

Sample Part C Impact on Family Items Over the past year, early intervention

services have helped me and/or my family:

Participate in typical activities for children and families in my community.

Understand how the special education system works.

Know about services in the community.

The school offers parents training . . . [660]

The school explains what options parents have . . . [600]

Written information I receive is . . . understandable. [520]

ITEM CALIBRATIONS ON THE PART B SCHOOL EFFORTS

RULER*

*.95 likelihood of agreement

. .##### . . . . . # # . .## .## .### .#### .#### .#### .#### .#### .#### .##### .##### .##### .####### .######## .############ .########### ########### .############ .######### .######### .####### .####### .##### .##### .#### .## .## ## .## .# . . . . . . . .########

PARENT MEASURES ON THE PART B SCHOOL EFFORTS RULER

In the national sample, the average parent measure on the Part B School Efforts scale was 500. The parent measures ranged from a low of 140 to a high of 880.

In the histogram to the left of the ruler, each # sign represents 13 parents (total n=2634).

Steps in the NCSEAM standard-setting process

In July 2005, NCSEAM convened a stakeholder group with broad representation of families, state and local agencies, advocates, and researchers.

Participants were provided with the list of scale items in calibration (agreeability) order.

Steps in the NCSEAM standard-setting process

The question posed to the group was:

What is the highest item with which you would require an “agree” response in order to have confidence that the meaning of the indicator (i.e., schools are facilitating parent involvement) is being achieved?

Steps in the NCSEAM standard-setting process

Participants had to reach consensus on this question.

The point on the ruler that corresponds to the selected item represents the standard.

“If families don’t agree with this item” – and, by implication, with all those below it – “then we could not say that we had acceptable quality in this area.”

Applying the standard

Performance on the indicator is calculated as the percent of parents or families with measures at or above the established standard.

PART B SCHOOL PARTNERSHIP EFFORTS

S offers parents training about special education issues S gives parents help to play an active role in child's education I organizations that offer information and training for parents I have been asked for my opinion about services I was offered special assistance so I could participate in IEP The school provides information on agencies for transition S gives me choices with regard to services S explains what options parents have if they disagree with school S communicates regularly with me regarding my child's progress We discussed how my child would participate in assessments Written justification given for services outside regular class Teachers and administrators seek out parent input T&A ensure that I have fully understood the Procedural Safeguards I am considered an equal partner with teachers and professionals All of my concerns and recommendations were documented on the IEP T&A show sensitivity to the needs of students with disabilities S offers parents different ways to communicate with teachers T&A encourage me to participate in the decision-making process My child's evaluation report is written in terms I understand We discussed accommodations and modifications for my child T treat me as a team member T&A respect my culture and language Written information I receive is written in an understandable way T are available to speak with me Information is provided to me in a language I understand

The school explains what options parents have if they disagree with a decision of the school. [600]

ITEM CALIBRATIONS ON THE PART B SCHOOL EFFORTS

RULER*

*.95 likelihood of agreement

. .##### . . . . . # # . .## .## .### .#### .#### .#### .#### .#### .#### .##### .##### .##### .####### .######## .############ .########### ########### .############ .######### .######### .####### .####### .##### .##### .#### .## .## ## .## .# . . . . . . . .########

24%

The school explains what options parents have if they disagree with a decision of the school. [600]

ITEM CALIBRATIONS ON THE PART B SCHOOL EFFORTS

RULER*

*.95 likelihood of agreement

. . . . . . . # # . .## .## .### .#### .#### .#### .#### .#### .#### .##### .##### .##### .####### .######## .############ .########### ########### .############ .######### .######### .####### .####### .##### .##### .#### .## .## ## .## .# . . . . . . . .########

30%

PART C IMPACT ON FAMILY Over the past year, Early Intervention services have helped me and/or my family:

participate in typical activities for children and families in my community understand how the Early Intervention system works. keep up friendships for my child and family. know about services in the community. know where to go for support to meet my family's needs. be more effective in managing my child's behavior. know about my child's and family's rights concerning early intervention services. improve my family's quality of life. know where to go for support to meet my child's needs. make changes in family routines that will benefit my child with special needs. do activities that are good for my child even in times of stress. get the services that my child and family need. feel that my family will be accepted and welcomed in the community. communicate more effectively with people who work with my child and my family. feel that my child will be accepted and welcomed in the community. be able to evaluate how much progress my child is making. feel more confident in my skills as a parent. understand the roles of the people who work with my child and family. feel that I can get the services and supports that my child and family need. understand my child's special needs. do things with and for my child that are good for my child's development. feel that my efforts are helping my child.

Over the past year, Early Intervention services have helped me and/or my family:

know about my child’s and family’s rights . . . [600] communicate more effectively . . . [570] do things with and for my child that are good for my

child's development [540]

ITEM CALIBRATIONS ON THE PART C FAMILY OUTCOMES

RULER*

*.95 likelihood of agreement

.############### .## .# .# .## #### .### .#### .#### .#### .######## ###### .####### .##### .##### .### .##### .#### .#### .###### ######### .### .## .## .# .# .# .# . . . . . . . . . .#

Constructing smaller item sets

To reproduce the measures obtained using all the items in a scale, subsets of items need to represent the full range of the measurement ruler.

With appropriate selection, different subsets of items will produce the same measures.

Approximately 25 items per scale are needed to achieve measurement reliability of .90 or above.

How the NCSEAM measures can guide improvement efforts

A state’s measure on the indicator can be directly interpreted with respect to the items that define the scale.

Items below the measure are in large part being accomplished; items above the measure have yet to be accomplished.

Items just above the measure are good targets for improvement.

C -> 619 -> B linkages Example #1: Facilitating parent

networking Part C: Someone from the early intervention

program helped me get in touch with other parents for help and support.

619: People from preschool special education, including teachers and other service providers, connect families with one another for mutual support.

Part B school-age: I was given information about organizations that offer support for parents of students with disabilities

C -> 619 -> B linkages Example #2: Ensuring that parents

know their rights Part C: My family was given information about

the rights of parents regarding early intervention services.

619: People from preschool special education, including teachers and other service providers, ensure that I have fully understood my rights in preschool special education.

Part B school-age: Teachers and administrators ensure that I have fully understood the procedural safeguards (the rules in federal law that protect the rights of parents).

Use of multiple scales

The SPP/APR family indicator for Part C can be addressed by data derived from a single scale, namely, the Impact on Family Scale

The SPP/APR parent indicator for 619 can be addressed by data derived from a single scale, namely, the Efforts/ Quality Scale

However, use of additional scales can provide valuable continuity of measurement as well as inform program improvement.

Further information

Detailed information on the NCSEAM parent surveys is currently posted to the website of the Federal Resource Center for Special Education: www.dssc.org/frc/