RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS)...x The Research Assessment Package for Schools (RAPS), that includes strategies and instruments for measuring four major components

RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS)

STUDENT SUPPORTS AND OPPORTUNITIES IN

SCHOOL: ENGAGEMENT, BELIEFS ABOUT SELF, AND EXPERIENCES OF INTERPERSONAL

SUPPORT

MANUAL FOR ELEMENTARY AND MIDDLE SCHOOL ASSESSMENTS

1998 Edition

¤ 1998, Institute for Research and Reform in Education, Inc.

© 1998, Institute for Research and Reform in Education, Inc. RAPS Manual, Rev. 10/28/98

i

RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS) Student Supports and Opportunities in School: Engagement, Beliefs about

Self, and Experiences of Interpersonal Support

MANUAL FOR ELEMENTARY and MIDDLE SCHOOL ASSESSMENTS

TABLE OF CONTENTS

I. GENERAL INTRODUCTION TO THE RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS) A. Introduction and theoretical framework for the RAPS.........................I-1 B. What is in the RAPS?.........................................................................I-2

1. RAPS-R: An approach to the use of school records data 2. Student Engagement, Beliefs about Self, and Experiences of

Interpersonal Support a. Self-Report: RAPS-S b. Teacher Report: RAPS-T c. Parent Report: RAPS-P

3. T-RAPS: Teacher Engagement, Beliefs about Self, and Experiences of Interpersonal Support

4. Critical Features of School-Site Reform: Perspectives of students (RAPS-SCF) and teachers (RAPS-TCF

C. How should the RAPS be used?........................................................I-3 II. INTRODUCTION TO RAPS COMPONENT 2: Student Supports And

Opportunities in School: Engagement, Beliefs About Self, and Experiences of Interpersonal Support A. Organization of the Manual................................................................II-1 B. Theoretical background......................................................................II-1

1. Engagement as a Precursor of Student Performance and Adjustment

2. Beliefs About Self and Engagement in School 3. Students’ Experiences of Interpersonal Support and the

Development of Student Self-Beliefs 4. Summary


ii

III. DESCRIPTION OF RAPS QUESTIONNAIRES FOR SCHOOLS: RAPS-T and RAPS-S A. Overview...........................................................................................III-1 B. The Teacher Report of Student Engagement for Elementary

School (RAPS-TE)…….....................................................................III-3 1. Item Description 2. Administration Information 3. The RAPS-TE Questionnaire

C. The Student Self-Report for Middle School (RAPS-SE)…………….III-8 1. Item by Construct Description 2. Administration information 3. The RAPS-SE Questionnaire

D. The Teacher Report of Student Engagement for Middle School (RAPS-TM).........................................................................III-26 1. Item Description 2. Administration Information 3. The RAPS-TM Questionnaire

E. The Student Self-Report for Middle School (RAPS-SM).................III-30 1. Item by Construct Description 2. Administration information 3. The RAPS-SM Questionnaire

IV. SCORING AND INTERPRETATION OPTIONS FOR RAPS SCHOOL QUESTIONNAIRES: RAPS-T and RAPS-S A. Elementary Options I: Composite Scores........................................IV-1

1. The RAPS-TE Composite 2. RAPS-SE Composites

B. Elementary Options II: High Risk and Optimal Indicators................IV-5 1. RAPS-TE Indicators 2. RAPS-SE Indicators

C. Middle School Options I: Composite Scores...................................IV-7 1. The RAPS-TM Composite 2. RAPS-SM Composites


iii

D. Middle School Options II: High Risk and Optimal Indicators.........IV-12 1. RAPS-TM Indicators 2. RAPS-SM Indicators

V. RELIABILITY AND VALIDITY INFORMATION FOR RAPS COMPOSITES AND INDICATORS A. RAPS-TE...........................................................................................V-1

1. Characteristics of reliability sample 2. Reliability of the RAPS-TE Student Engagement Composite 3. Assessing the validity of the RAPS-TE Student Engagement

Composite and Indicators B. RAPS-SE..........................................................................................V-.4

1. Characteristics of the reliability sample 2. Reliabilities of the RAPS-SE Composites 3. Assessing the validity of the RAPS-SE Composites and Indicators

C. RAPS-TM........................................................................................V-12 1. Characteristics of reliability sample 2. Reliability of the RAPS-TM Student Engagement Composite 3. Assessing the validity of the RAPS-TM Student Engagement

Composite and Indicators D. RAPS-SM........................................................................................V-15

1. Characteristics of the reliability sample 2. Reliabilities of the RAPS-SM Composites 3. Assessing the validity of the RAPS-SM Composites and Indicators

VI. APPENDICES Biblography for Component 2: Student Supports and Opportunities in School……………………………………………………..VI-1

RAPS-TE Reliability and Validity……………………..……………………VI-9 RAPS-TM Reliability and Validity……………………………………….…VI-13

RAPS-SE Reliability and Validity…………………………………………..VI-17 RAPS-SM Reliability and Validity…………………………………………..VI-33

I-


1

GENERAL INTRODUCTION TO THE RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS)

In this general introduction we will provide a brief overview of the theoretical framework that forms the foundation of the Research Assessment Package for Schools (RAPS), and of each of the separate components of the RAPS. Further, we will review what is in the RAPS, why and how the RAPS measurement tools were developed, and how the assessment package can be used for diverse purposes. The RAPS manual is designed in a modular format, with different components of the RAPS contained within separate modules that can be obtained either together or individually from IRRE. Each module of the RAPS manual goes into considerably more depth on both the theoretical and the research background supporting the individual components. A. Introduction and Theoretical Framework for the RAPS The Institute for Research and Reform has developed this assessment package as part of its overarching approach to the reform of public schools and school systems. This approach has three key elements: x Seven critical features of successful school-site reform that research on adult

and youth development, organizational and educational research and the experience of successful urban schools have shown to yield positive outcomes for diverse groups of students;

x A change framework that links implementation of these changes in schools to

changes in the supports and opportunities experienced by adults and students in schools to changes in student performance and adjustment across elementary, middle and high school, and finally, to increased life chances of youth beyond high school; and

x The Research Assessment Package for Schools (RAPS), that includes

strategies and instruments for measuring four major components of the school-site reform framework: 1) key indicators of student performance and adjustment; 2) the supports and opportunities experienced by students in schools; 3) the supports and opportunities experienced by adults in schools (i.e. teachers, administrators, and other staff members); and 4) the implementation of the seven critical features for school-site reform.

The first two elements of the Institute’s approach to educational reform are described in detail in two other IRRE publications: the seven critical features in First Things First: A Framework for Successful School-site Reform; and the change framework in A Theory of Change Approach to Evaluating Investments in Public Education. Both publications are available from the Institute for Research and Reform. The seven critical features of successful school-site reform are

I-2


summarized in Figure 1 and a simplified illustration of the change framework is presented in Figure 2. B. What Is In The RAPS? Currently, the RAPS includes five separate but integrated measurement tools assessing aspects of three of the four major components described above: RAPS-R, a strategy for analyzing and reporting data from student records (component 1); RAPS-S, a survey given to students to assess their levels of engagement in school, their beliefs about themselves and the interpersonal supports they receive from adults at home and at school (component 2); RAPS-T, a survey given to teachers to obtain their reports of student engagement (component 2); RAPS-P, a parent-report of student engagement in school and the support that the student receives from his or her teachers (component 2); and T-RAPS, a survey given to teachers to assess their levels of engagement and the professional and interpersonal supports they perceive in their schools (component 3). The RAPS-R. The RAPS-R was developed in response to school districts’ need to have a simple, compelling and scientifically credible way to track their students’ performance and adjustment across elementary, middle and high school. The RAPS-R is a strategy for combining information from readily available student records to fill this need. The Student Performance and Commitment Index (SPCI) includes two categories -- high risk and optimal for each school level that were developed based on longitudinal data from the Institute’s data banks and from detailed reviews of other longitudinal research on markers of student performance and adjustment. The RAPS-S. The survey measures included in the RAPS-S were originally designed by a team of researchers to be used in studying specific psychological and interpersonal processes affecting students’ school performance and adjustment. These measures been used by researchers in education, psychology and sociology and results from these measures have been published in numerous peer-reviewed journals and other scholarly articles (see Appendices for sample references). Over the past two years, Institute research staff have revised and abbreviated these measures for use by evaluators and school districts. Measures were retained that showed utility as interim markers of progress toward subsequent outcomes in the change framework shown in Figure 2. The RAPS-T. This teacher survey was originally developed for research purposes to get information about student engagement from teachers that could then be compared to student reports of their own engagement. Subsequently, Institute research staff have abbreviated and revised the measure for use in other applications, including as an important outcome measure for educational

I-3


program and reform efforts targeting student motivation. Again, RAPS-T survey items were selected based on their predictive validity with respect to subsequent student performance and adjustment. The RAPS-P. The RAPS-P is a questionnaire designed to provide additional information about student engagement as well as about how much support the student is receiving from teachers at school. Additional items regarding the parent’s self-perceptions of the support they provided to the student, as well as their own experiences of being supported by the school have been omitted due to low reliability and poor prediction of student self-perceptions, academic performance and personal adjustment. The RAPS-P is currently undergoing additional reliability and validity assessment. The T-RAPS. This measure, tapping the fourth component of the RAPS, was originally developed to assess the full range of supports and opportunities experienced by teachers working in schools. The scales included in the T-RAPS were based on the same motivational theory that guided the construction of the student measures included in the RAPS-S. Subsequent revisions have focused on two components: teachers’ experience of supports for their engagement and their reports of their own levels of engagement. The revised RAPS-T is also currently undergoing additional field testing and validation. The RAPS-CFT and RAPS-CFS. The newest measurement tools included in the assessment package, RAPS-CF (Critical Features) for Teachers and Students, will address the fourth component of the RAPS framework by assessing the extent to which the seven critical features of successful school-site reform are being implemented in schools from the perspectives of students and teachers. These two survey instruments are still being field tested and validated by the Institute. C. How Should the RAPS Be Used? The RAPS is designed to be used in at least four ways: x As a diagnostic instrument that can provide simple, valid and compelling

information about the current status of a particular population of students or adults; a school or set of schools; or an entire district;

x As a program or initiative evaluation tool to examine whether or not a

particular program that focuses on one or more component of the change framework is having its intended effects;

x As a management tool for tracking whether particular change thresholds on

key outcomes have been achieved by a group of individuals, a program or an institution; or

I-4


x As a basic research tool to study change at the individual, classsroom, school

or district level with respect to the relevant elements of the change framework.

Figure 2

II-


1

INTRODUCTION TO RAPS COMPONENT 2

Student Supports and Opportunities in School: Engagement, Beliefs about Self, and

Experiences of Interpersonal Support A. Organization of the Manual The next four sections of this manual focus specifically on the intra- and interpersonal supports students need in order to do well in school. These supports comprise those provided by teachers, parents, and peers; positive beliefs about oneself; and engagement with school and schoolwork. The manual is organized to offer easy access to varying levels of theoretical and empirical information about two RAPS measures of these three constructs: 1) A student self-report measure (RAPS-S), and 2) a teacher-report measure of student engagement (RAPS-T). This section provides theoretical background that explains the importance of the constructs as they relate to educational outcomes for students. The following section includes a brief description of each of the constructs covered by the two measures, including the specific items used to tap each construct. This is followed by a presentation of two approaches to scoring and interpreting responses on each of the measures. The last section pertaining to the RAPS-S and the RAPS-T includes reliability and validity information for the student- and teacher-report measures. Finally, three appendices provide additional information on the RAPS-S and RAPS-T, including a bibliography of related research and theoretical work, and detailed information on the reliability and validity of the RAPS-S and RAPS-T.

B. Theoretical Background A substantial body of existing research explores the conditions that promote school success. Such research has found that students’ academic performance and adjustment are enhanced when the following three conditions exist: 1. Students experience a supportive interpersonal environment that helps

them to believe in themselves and their abilities. A supportive environment is characterized by: a. a structure that is consistent and fair in its rules and expectations

for student performance and conduct; b. support for student autonomy, reflected in the provision of

opportunities for choice and respect for students’ perspectives; and c. adults who demonstrate high levels of interest in and involvement

with students.

II-


2

2. Students have strongly positive feelings about themselves. This includes feeling competent and in control of their environment and of their own behavior. It also includes having generally positive feelings about their relationships with important others such as parents, teachers, and peers.

3. Students are engaged in school. Engagement is reflected in the amount

of attention and effort students apply to schoolwork and homework, the emotions they experience while doing it, and the importance they place on doing well in school.

These three conditions are fundamentally interrelated. They build on each other to create an interpersonal context in the school environment that supports the development of positive self-beliefs, which in turn support engagement in school activities. Student engagement has been found to be the single most robust predictor of students’ performance and personal adjustment in school. This conclusion holds regardless of whether students come from families that are relatively advantaged or disadvantaged economically or socially.i, ii The following pages provide a brief review of the literature regarding these three conditions. We will begin with the condition that has been most directly associated with student academic performance and personal adjustment in school – student engagement – and move backward through our model to student beliefs about themselves and, finally, student experience of interpersonal support. 1. Engagement as a Precursor of Student Performance and Adjustment Defining Engagement: Ongoing Engagement and Reaction to Challenge. Students’ engagement with school is evident both in the flow of ongoing daily activity (e.g., during a particular classroom activity, in the effort expended on homework) and in students’ reactions to challenge (e.g., being given a bad grade, responding incorrectly to a question in class). Ongoing Engagement. Ongoing engagement versus disaffection has to do with students’ typical emotions, thought processes, and behavior evidenced within the course of the school day. Behavioral engagement includes the amount of time students spend on work, the intensity of their concentration and effort, the tendency to stay on task, the level of difficulty of the tasks selected, the demonstration of flexible problem-solving, and the propensity to initiate action when given the opportunity to do so. Emotional components of engagement include heightened levels of positive emotion during the completion of an activity, demonstrated by enthusiasm, optimism, curiosity, and interest. Disaffection from classroom activities is demonstrated by feelings of boredom, discouragement, and anger.iii

II-


3

Reaction to Challenge. Engagement and disaffection in the face of challenge are seen in students’ methods of coping with perceived failures in school. Situations can be perceived as either a threat or a challenge depending on students’ beliefs about their own competence. Students who perceive a situation as challenging actively persist in the face of failure through the use of effort, strategizing, problem-solving, information-seeking, and experimentation. These behaviors tend to be accompanied by an optimistic attitude and attempts to plan and prevent problems from occurring in the future. Conversely, students who feel threatened tend to feel incompetent and full of self-doubt. Typical reactions to a perceived threatening situation include attempting to escape the situation either by mentally or physically withdrawing and by avoiding or putting off the activity as long as possible when it is encountered in the future. These behaviors are invariably accompanied by negative emotions that include anger, blame, denial, anxiety, and hopelessness.iv Linking Engagement to Important Educational Outcomes. A wide array of studies has concluded that school engagement is of utmost importance in minimizing the probability of school dropout and in maximizing the probability of school completion.v Regardless of socioeconomic or family background, student engagement predicts positive student outcomes in such areas as achievement test scores, attendance, grades, disciplinary referrals, retention, and graduation.vi Engaged students are more likely than their disengaged peers to stay in school, learn, demonstrate high levels of achievement, feel good about themselves, and continue on to higher education after high school.vii High levels of engagement can even explain why some high-risk students succeed academically -- a finding confirmed for males and females and for White, Hispanic, and African-American students alike.viii In contrast, students with low levels of engagement who do not develop a sense of belonging are at risk for a wide range of long-term, adverse consequences, including disruptive behavior in class, absenteeism, truancy, juvenile delinquency, and dropping out of school.ix 2. Beliefs about Self and Engagement in School Many researchers have suggested that high levels of engagement are associated with students’ perceptions of a) competence; b) autonomous self-regulation; and c) relatedness to significant others. Competence. Perceived competence, interacting effectively with the environment, has been found to be strongly associated with student performance in school.x In order to succeed in school, students have to know what it takes to be competent and they must believe that they have what it takes to carry out those strategies. Such competence beliefs within the academic domain are strong predictors of students’ behavioral and emotional engagement in the classroom. For example, when competence beliefs are examined as predictors of children’s engagement in the classroom, they account for more than 25% of

II-


4

the variance in teachers’ ratings of children’s engagement and disaffection.xi In addition, many studies have explored the relationship between various aspects of perceived competence (i.e., locus of control, mastery versus helplessness beliefs, causal attributions, and self-efficacy) and performance (i.e., academic achievement). Changes enhancing these aspects of competence beliefs have been found to produce positive changes in the quality and persistence of students’ problem-solving and performance on academic tasks.xii Autonomy. The psychological need for autonomy refers to the need to be in control of one’s own behavior, or to be self-regulating.xiii One way in which perceptions of autonomy can be manifested in the academic domain is through students’ reasons for engaging in classroom activities. If students participate in school activities because they feel forced to or feel guilty about not participating, they will be less motivated and less engaged in the activity than if they choose to take part because it is important to them personally, or because they truly enjoy the activity. Students who perform primarily to please others, avoid guilt, or avoid punishment demonstrate strategies that are timid, stressed, and defensive and even attempt to evade the activity altogether. Conversely, students who participate in school activities because they enjoy them and/or see their value react to challenge with persistence, strategy-testing, and optimism.xiv Relatedness. The psychological need for relatedness includes feelings of being respected by, connected to, and cared for by others.xv In a school setting, students are concerned with whether their teachers, parents, and peers care about and respect them. Perceptions of belonging, of relatedness, and similar social constructs have been found to be positively associated with school adjustment (i.e., less absenteeism, greater social competence, fewer conduct problems, reduced drug use and delinquency),xvi as well as with academic motivation, effort, and achievement.xvii Indeed, some researchers view relatedness as being the most crucial factor in the motivation and engagement of certain categories of at-risk students.xviii 3. Students’ Experiences of Interpersonal Support and the

Development of Student Self-Beliefs The development of competence, autonomy, and relatedness. The development of heightened levels of competence, autonomy, and relatedness in youth requires a variety of interpersonal supports from the school environment.xix For instance, in order to develop high levels of perceived competence, students must be provided with expectations and consequences that are clearly defined and consistently communicated by adults.xx Environments high in structure provide students with this kind of information.xxi In addition, the extent to which students perceive the classroom as autonomy-supportive is a function of their teachers’ ability to create an optimally challenging climate, offer choice, incorporate relevance into the curriculum, and minimize the use of external

II-


5

rewards, controls, and pressures.xxii Finally, the quality of students’ relationships with their teachers is significantly related to students’ beliefs about themselves and their experience of school.xxiii If students perceive their parents, teachers, and peers as dedicating resources such as time and interest to them, and being accepting and affectionate toward them, they will feel satisfied in terms of relatedness – they will express more optimistic and enthusiastic responses when engaged in academic work. If students perceive their teachers and classmates as uncaring and uninterested, they will feel rejected and neglected, and will experience school as frustrating to their sense of relatedness.xxiv Interpersonal supports, student self-beliefs, and engagement. Both research and practice point to the effectiveness of meeting students’ psychological needs relative to alternative strategies for promoting student engagement. Factors such as personal involvement with teachers, academic and conduct standards that are high, clear, fair, and consistent, and being given a voice in the classroom, are forms of social control that influence at-risk students far more than negative consequences such as detention, suspension, retention, and so on. Relative to at-risk students who are ruled by the carrot and stick technique, students whose psychological needs are met are far more likely to demonstrate heightened levels of engagement.xxv Independent of poverty level, students whose psychological needs for competence, autonomy, and relatedness are supported by their interpersonal environments are not only more likely to score high on measures of academic engagement,xxvi but also on measures of personal and social development (e.g., self-esteem, concern for others, liking for school, educational aspirations, and academic aspirations).xxvii These students are also more likely to be successful in school.xxviii Conversely, students who do not receive the contextual supports they need to develop these positive self-beliefs lose their motivation toward and interest in learning, while demonstrating increases in problem behaviors.xxix Furthermore, because the classroom is a social setting in which students and teachers interact with each other, a reciprocal relationship has been found between student engagement and a teacher’s provision of interpersonal supports such as structure, autonomy support, and involvement. Students who are engaged in their learning activities elicit enhanced levels of positive teacher behaviors, which, in turn, produce higher levels of positive student behaviors. In contrast, students who enter school with lower levels of engagement (as is the case with many at-risk students) tend to receive less of the contextual support necessary to enhance motivation, and, therefore, are more likely to disengage completely.xxx It is only a short step between disengagement and dysfunctional behaviors such as absenteeism, disruptive behaviors, suspension, and retention.xxxi Once students begin to engage in such dysfunctional behaviors, dropping out and the negative life outcomes associated with dropping out are highly probable.

II-


6

4. Summary For students to complete school with high levels of academic performance and low levels of at-risk behaviors, they must be engaged in school. For high levels of engagement to occur, students must develop positive perceptions of themselves and their abilities. Interpersonal supports within the school community allow youth to develop more effective strategies for performing in school as well as the beliefs that they have the capacity to execute those strategies. Interpersonal supports can also further students’ understanding of their own and others’ roles in regulating their behavior and can lead to the accumulation of experiences in which significant others are there when students need support. When these supports are consistently available within the school environment and allow positive self-beliefs to develop, students will be more likely to engage fully in school and, ultimately, to graduate creditably.

III- 1


DESCRIPTION OF RAPS QUESTIONNAIRES: RAPS-T AND RAPS-S

A. OVERVIEW

Two types of RAPS instrument are described in this manual: The first and most extensive of these, the RAPS-S, is a self-report measure tapping student engagement in school, student beliefs about self, and student perceptions of interpersonal support. The second measure, the RAPS-T, is a brief teacher-report measure tapping student engagement in school. Both of the instruments are available in forms for middle school (RAPS-TM and RAPS-SM) and for elementary school (RAPS-TE and RAPS-SE).1 The RAPS-S and the RAPS-T are both designed to be used as part of comprehensive school evaluation programs, and they can also be used in basic educational research. They are not designed as diagnostic tests for at-risk students. Both of these measures are described in detail in the following sections. The table below summarizes the constructs tapped by the two measures:

Construct Student-Report Teacher-Report

1. ENGAGEMENT: 1a. Ongoing Engagement X X 1b. Reaction to Challenge X

2. BELIEFS ABOUT SELF: 2a. Perceived Competence X 2b. Perceived Autonomy (Self-

Regulation) X

2c. Perceived Relatedness X

3. EXPERIENCES OF INTERPERSONAL SUPPORT:

3a. Parental Support X 3b. Teacher Support X

1 No separate high school versions of the RAPS are currently available. However, the middle school versions have recently been used successfully in a number of school evaluation projects in urban school districts.

III- 2


A Word About RAPS Questionnaire Administration In addition to describing the RAPS measures, this section provides information about questionnaire administration. The information is based on the experience of IRRE staff and other researchers and educators who have administered these questionnaires in a variety of school settings. Because each setting in which the RAPS might be administered is unique, the information is necessarily very general. IRRE staff are available to provide more detailed information pertaining to RAPS administration upon request by teachers, researchers, administrators, and community members. Requests should include as much specific information as possible about the purposes for which the measures are to be used, and the conditions under which administration will take place. Contact numbers for IRRE may be found in the front of this manual. Confidentiality, Voluntary Participation, and Value Regardless of the purposes for which these questionnaires are being administered, or of who is conducting actual questionnaire administration, data management, data analysis, and report writing, is extremely important that individuals understand that their responses are confidential and voluntary, and that the information that is being provided is valuable. Honest responses can only be expected when the informants completing a questionnaire believe that their answers cannot be used against them, or in their favor, in any way. Similarly, honest and complete responding can only be expected from individuals who do not feel that they are being unduly coerced by those administering the questionnaire. Teachers, parents, and students should be informed that there are no repercussions for refusing to participate, or for skipping questions that make them feel uncomfortable. Parents also need to be given the opportunity to consent to their children’s participation, and a mechanism for obtaining parental consent must be established. Finally, the best way to ensure full and honest responding to RAPS questionnaires is to make it clear to respondents that the information they are providing serves a larger purpose (e.g., helping the school community to establish goals for change, or evaluate how well the school is meeting its established goals).

III- 3


B. THE TEACHER-REPORT OF STUDENT ENGAGEMENT FOR ELEMENTARY SCHOOL (RAPS-TE)

1. Item Description 2. Administration Information 3. The RAPS-TE Questionnaire

III- 4


1. Item Description The RAPS-TE is a very brief (3-item) teacher-report measure of student engagement in school. The three items are: 1. In my class, this student seems tuned in. (teexs52) 2. This student comes to class unprepared. (tebxf2) 3. This student does more than required. (tebxs1) Each of these items is responded to on the following scale:

Very True Sort of True Not Very True Not At All True

(4) (3) (2) (1) 2. Administration Information Instructions for completing the RAPS-TE are fairly straightforward. Typically, teachers are given sufficient forms to provide information for all students in their class. All teachers who are being asked to participate should be given the questionnaires at the same time and should be asked to return the questionnaires by a fixed deadline. The brevity of this measure makes it possible to have teachers report on numerous students without presenting an excessive time burden that could undermine teachers’ willingness to participate.

2 Codes in parentheses are item names that have been used in all IRRE-sponsored and facilitated administrations of this measure. They are included here primarily to facilitate communication with IRRE staff.

III- 5


3. The RAPS-TE Questionnaire

Teacher ID: Research Assessment Package for Schools - Teacher Report On the following pages, please write in a student name or identification number for each of the students in your class who are participating in this study. Then, for EACH STUDENT, tell us how true each of the three items is for THAT STUDENT by circling one of the four answers: VERY TRUE, SORT OF TRUE, NOT VERY TRUE, or NOT AT ALL TRUE. Student ID:

1. In my class, this student seems tuned-in. Very Sort of Not Very Not At All True True True True

2. This student comes to class unprepared. Very Sort of Not Very Not At All True True True True

3. This student does more than required. Very Sort of Not Very Not At All True True True True

Student ID:




III- 6


Student ID:




Student ID:




Student ID:




III-


7

C. THE STUDENT SELF-REPORT FOR ELEMENTARY SCHOOL (RAPS-SE)

1. Item by Construct Description 2. Administration Information 3. The RAPS-SE Questionnaire

III-


8

1. Item by Construct Description This multidimensional measure includes a total of 79 items, tapping three major domains and seven separate subdomains, in a questionnaire format that can be administered to students during a single class period. The majority of these items reflect negative feelings or beliefs. In order to reduce the overall negative tone of the questionnaire, nine positive items have been added. These nine items were selected from the middle school version of the questionnaire, but are not used in constructing summary scores for elementary school students. With one exception, all items are responded to on the following scale:


(4) (3) (2) (1) An item by construct list, including brief descriptions of each construct, is presented on the following pages. The nine items that are not included in the summary scores are presented together in the final table. Engagement Domain The Engagement domain as measured by the RAPS-SE incorporates two separate but interrelated subdomains of student adjustment in school: 1) Ongoing Engagement with school; and 2) Reactions to Challenge (i.e., the strategies students use when faced with negative or stressful school events). Ongoing Engagement Subdomain Ongoing Engagement includes the extent to which the student exerts effort on schoolwork, pays attention in class, prepares for class, and believes that doing well in school is personally important. The RAPS-SE includes a total of six items tapping these aspects of student engagement. These items are listed below:

Item Item Code3 Questionnaire Item Number

I work very hard on my schoolwork. eoxs7 3 I don’t try very hard in school. eoxf3 19 I pay attention in class. eaxs2 35 I don’t work very hard in school. eoxf4 50 When I’m in class I just act as if I’m working. eaxf3 60 How important is it to you to do the best you can in school?

eyxx3 88

3 As with the RAPS-TE, codes are item names that have been used in all IRRE-sponsored and facilitated administrations of this measure. They are included to facilitate communication among users of these measures and with IRRE staff.

III-


9

Eyxx3 is the single item that has a response set that differs from the other items in the questionnaire. It is responded to on the following scale:


(4) (3) (2) (1) Reaction to Challenge Subdomain There are a number of ways in which students may cope with, or react to, negative school-related events. Students may blame negative events on the teacher or other individuals (Projection). Students may cope with negative events by downplaying their importance (Denial). At other times, they may perseverate on the events and worry about them without necessarily doing anything to ensure that such events won’t occur again (Anxiety Amplification). Finally, students may cope by examining their own behavior and attempting to make changes to prevent similar negative events from occurring in the future (Positive Coping). Of these four different reactions to challenge, Projection and Denial appear to be particularly related to positive or negative outcomes for elementary school students and are included in this revised version of the RAPS-SE. One Anxiety Amplification item is included as well. The items are listed below: Item Item Code Questionnaire

Item Number Projection Items:

When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say the teacher didn’t cover the things on the test.

ecjx3 14

When something bad happens to me in school, (like not doing well on a test or not being able to answer an important question in class), I get angry at the teacher.

ecjx4 33

When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say it was the teacher’s fault.

ecjx1 65

Denial Items: When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class),, I tell myself it didn’t matter.

ecdx3 22

When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say it wasn’t important.

ecdx4 42

When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say I didn’t care about it.

ecdx7 71

Anxiety Amplification Item: When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I worry that the other students will think I’m dumb.

ecax1 53

III-


10

Beliefs about Self Domain The Beliefs about Self domain incorporates three separate subdomains: 1) Perceived Competence, 2) Perceived Autonomy, and 3) Perceived Relatedness. Perceived Competence Subdomain The Competence subdomain of the RAPS-SE consists of a total of 16 items. One of these items reflects perceptions of control in the school setting. Seven items reflect the student's beliefs about what kinds of strategies are necessary in order to achieve desirable, or avoid undesirable, outcomes. There are four different strategies represented: 1) Unknown, 2) Powerful Others, 3) Effort, and 5) Luck. A fifth strategy that appeared in earlier versions of the RAPS-SE, Ability, has been omitted because students’ endorsement of ability as a strategy for doing well in school did not appear to be associated with student performance and adjustment in elementary school. The final eight Perceived Competence items reflect the student's belief that he or she has the capacity to enact strategies. There are two capacity items for each strategy except Unknown, including Ability. Thus, although students’ beliefs that ability is necessary to do well in school do not appear to differ among students doing more or less well in school, students do differ in the extent to which they believe that they are smart in school. The Unknown strategy items reflect the student's inability to determine what it takes to do well in school. Thus, it more accurately reflects the lack of a strategy and hence would have no associated feeling of being more or less capable of enacting the strategy in order to do well. The 16 items included in the revised RAPS-SE are listed below: Item Item Code Questionnaire

Item Number Perceptions of Control Item

I can’t do well in school. ccxf3 72 Unknown Strategy Items

I don’t know how to keep myself from getting bad grades. csuf2 5 I don’t know what it takes to get good grades in school. csus3 48

Powerful Others Strategy Items I won’t do well in school if the teacher doesn’t like me. cspf1 36 The best way for me to get good grades is to get my teacher to like me.

csps2 61

Effort Strategy Items Trying hard is the best way for me to do well in school. cses2 43

Luck Strategy Items I have to be luck to do well in school. csls2 1 If I’m unlucky, I won’t do well in school. cslf1 31

III-


11

Item Item Code Questionnaire

Item Number Powerful Others Capacity Items

I can get my teacher to like me. cpps2 29 I can’t get my teacher to like me. cppf2 51

Effort Capacity Items I can work really hard in school. cpes2 32 I can’t work very hard in school. cpef3 80

Ability Capacity Items I’m pretty smart in school. cpas4 10 I’m not very smart in school. cpaf2 40

Luck Capacity Items I’m pretty lucky at getting good grades. cpls2 17 I am unlucky in school. cplf1 45

Perceived Autonomy Subdomain The Perceived Autonomy subdomain of the RAPS-SE is made up of nine items that tap two different types of self-regulation, or reasons for doing schoolwork or homework. Five items tap identified self-regulation (i.e., doing schoolwork because it is seen as important), and four items tap intrinsic self-regulation (i.e., doing schoolwork because it is fun or interesting). These two forms of self-regulation are positively related to each other and are all expected to contribute to students’ academic performance. Two other forms of self-regulation – external (i.e., doing schoolwork to avoid punishment or because it is the rule) and introjected (i.e. doing schoolwork or homework to avoid feeling embarrassed or guilty) were included in previous versions of the RAPS-SE but were omitted from the current revised version because they were unrelated to other indices of student performance and adjustment. Autonomy, or Self-regulation, items are listed below: Item Item Code Questionnaire

Item Number Identified Self-Regulation Items:

I do my homework because I want to understand the subject. ardx1 20 I work on my classwork because I think it is important. ardx9 26 I work on my classwork because I want to learn new things. ardx2 39 I do my homework because I want to learn new things. ardx7 52 I work on my classwork because doing well in school is important to me.

ardx8 64

Intrinsic Self-Regulation Items: I do my homework because I like to do it. arix2 6 I work on my classwork because it’s interesting. arix8 13 I do my homework because it’s fun. arix1 46 I work on my classwork because it’s fun. arix3 70

III-


12

Perceived Relatedness Subdomain The Relatedness subdomain of the RAPS-SE is made up of 14 items that tap four constructs: 1) Feelings about Self (5 items), 2) Parental Emotional Security (3 items), 3) Teacher Emotional Security (3 items), and 4) Peer Emotional Security (3 items). These four constructs differ from those presented in the middle school version, RAPS-SM in that items reflecting Emotional Security with Self and Satisfaction with Self, two separate constructs on the RAPS-SM, are combined into the single Feelings about Self construct. Three additional constructs were included in previous versions of the RAPS-SE: Parental Psychological Proximity Seeking, Teacher Psychological Proximity Seeking, and Peer Psychological Proximity Seeking. Due to low reliabilities and lack of associations with other indices of student performance and adjustment, however, these three components were excluded from the current revision. The Emotional Security constructs each reflect the extent to which positive emotions are present, and negative emotions absent, when in the presence of a relationship partner (i.e., parent(s), teacher, or peers). Feelings about Self reflects the extent to which students feel negatively about themselves and wish that they were different or were someone else. The 14 Perceived Relatedness items are as follows: Item Item Code Questionnaire

Item Number Feelings about Self Items

I wish I were someone else. rswx4 12 When I think about myself, I feel bad. rsef11 18 I wish I felt better about myself. rswx3 37 I wish I liked myself better. rswx5 57 When I think about myself, I feel unhappy. rsef10 69

Parent Emotional Security Items: When I’m with my parents, I feel mad. rpef2 8 When I’m with my parents, I feel unhappy. rpef4 47 When I’m with my parents, I feel ignored. rpef1 83

Teacher Emotional Security Items: When I’m with my teacher, I feel ignored rtef1 2 When I’m with my teacher, I feel mad. rtef2 15 When I’m with my teacher, I feel unhappy. rtef4 41

Peer Emotional Security Items: When I’m with my classmates, I feel ignored. rref1 25 When I’m with my classmates, I feel mad. rref2 44 When I’m with my classmates, I feel unhappy. rref4 78

III-


13

Experiences of Interpersonal Support Domain The Experiences of Interpersonal Support domain of the RAPS-SE includes a total of 27 items tapping two subdomains: 1) Parental Support; and 2) Teacher Support. Items within each subdomain reflect the extent to which the student feels that the adult(s): 1) are involved with them, 2) provide support for student autonomy, and 3) provide structure. Involvement items reflect the extent to which the student feels that the adult(s) know and care about them. Autonomy Support items concern the extent to which students feel that they are allowed to make decisions for themselves, and the extent to which the real-life value of the work that they are asked to do is made clear by the adult(s). Structure items reflect the clarity of the adult(s) expectations regarding student conduct, the extent to which consequences are consistent and predictable by the student when expectations are not met, and the student’s perception of the fairness of the adult(s) expectations. In the RAPS-SE, Parental Support and Teacher Support constitute two separate subdomains, each encompassing involvement, autonomy support, and structure. All items referring to the teacher are specifically about the school setting. Items referring to the parent are either about school and schoolwork (School Context) or do not refer to any specific context (General Context). Experiences of Support from Parents Subdomain Item Item Code Questionnaire

Item Number Parent Involvement - School Context Items

My parents never have enough time to hear about what happens to me in school.

ipxf6 4

Parent Autonomy Support - School Context Items My parents don’t explain why school is important. ypxf15 16 When it comes to school, my parents try to control everything I do. ypxf4 30 When it comes to school, my parents are always telling me what to do.

ypxf5 66

My parents don’t talk about how schoolwork is related to what I want to be.

ypxf14 75

Parent Structure - School Context Items My parents don’t make it clear what they expect of me in school. spxf10 7 My parents want me to do better than I can in school. spxf12 24 I don’t know what my parents expect of me in school. spxf4 85

Parent Involvement - General Context Items My parents don’t seem to have enough time for me. ipgf1 62 My parents don’t seem to know how I feel about things. ipgf3 87

Parent Autonomy Support - General Context Items My parents don’t let me make any of my own decisions. ypgf2 54 My parents trust me. ypgs1 73 My parents try to control everything I do. ypgf1 77

III-


14

Experiences of Support from Parents Subdomain - continued Item Item Code Questionnaire

Item Number Parent Structure - General Context Items

My parents don’t think I can do very much. spgf1 11 I never know what my parents are going to do. spgf3 27 I don’t know what my parents want from me. spgf2 34 My parents are fair with me. spgs4 55

Experiences of Support from Teachers Subdomain Item Item Code Questionnaire

Item Number Teacher Involvement Items:

My teacher doesn’t seem to have enough time for me. itxf5 38 My teacher likes the other kids in my class better than me. itxf3 68

Teacher Autonomy Support Items: My teacher doesn’t explain why we have to learn certain things in school.

ytxf3 28

My teacher doesn’t talk about how schoolwork is related to what I want to be.

ytxf14 56

My teacher interrupts me when I have something to say. ytxf6 63 My teacher tries to control everything I do. ytxf1 81

Teacher Structure Items: My teacher is fair with me. stxs11 9 My teacher’s expectations for me are way off base. stxf12 21 My teacher isn’t fair with me. stxf11 59 My teacher doesn’t make clear what he/she expects of me in school.

stxf8 74

Additional Items The following items were selected from the middle school version of the RAPS-S (the RAPS-SM) in order to balance the predominantly negative orientation of the RAPS-SE items. Each item is presented along with the domain and subdomain with which it is associated on the RAPS-SM.

III-


15


Item Number Beliefs About Self Domain, Perceived Competence Subdomain: 1 item

I can do well in school if I want to. ccxs5 23 Beliefs About Self Domain, Perceived Relatedness Subdomain: 4 items

When I’m with my parents, I feel good. rpes1 58 When I’m with my classmates, I feel good. rres1 67 When I think about myself, I feel proud. rses8 76 When I’m with my teacher, I feel good. rtes5 84

Experiences of Interpersonal Support Domain, Support from Parents Subdomain: 3 items My parents like to talk to me about school. ipxs2 79 My parents know just how well I can do in school. spxs11 82 My parents encourage me to find out how schoolwork could be useful to me.

ypxs14 86

Experiences of Interpersonal Support Domain, Support from Teachers Subdomain: 1 item My teacher cares about how I do in school. itxs6 49

2. Administration Information Time The RAPS-SE has been designed to be administered to groups of students during one class period (approximately 50 minutes). All materials should be ready to distribute immediately at the beginning of the allotted time, in order to ensure that all students will finish. Oral Questionnaire Administration and Training of Administrators Many of the items on the RAPS-SE ask students to discuss their feelings of relatedness with, and experiences of support from teachers. Because of this, students may be hesitant to respond honestly if the administrator of the RAPS-SE is their teacher. If teachers are administering the RAPS-SE, a plan should be established so that each teacher administers the questionnaire to students other than their own. Questionnaire administrators (i.e. teachers, researchers, school administrators or other members of the school community) should read instructions and all items aloud to students, in addition to providing each student with a copy of the questionnaire. In our work in diverse school settings, we have found that reading to students at all levels speeds up the administration process and decreases the possibility that students will misread questions.

III-


16

Oral administration of the RAPS-SE requires adequate training of questionnaire administrators. Among the most important elements of administration that training should cover are the following: 1. Standard instructions. All students should receive exactly the same

instructions at the beginning of the administration period. Examples of standard instructions that may be read to students are available from IRRE.

2. Pacing. Items should be read at a brisk pace, adjusting that pace to a

comfortable rate for the class. We have found that if we keep the pace brisk, more students (of all grade and ability levels) stay on task than if questions are read slowly. Reading at a fairly rapid pace also encourages students to give the first response that comes to mind for each item.

3. Maintaining focus. It is important that administrators do as much as

possible to ensure that students are completing the questionnaires properly. An item’s number should always be read aloud before reading the item itself.

Reading the response scale aloud is also important. The scale (i.e., Very True, Sort of True, Not Very True, Not At All True) should be read aloud for the first 2-3 questions, and should then be repeated aloud periodically throughout the administration. When not reading the answer scale aloud, administrators should be instructed to repeat it to themselves following each question, in order to appropriately pace the reading of the items. Also, the scale should be read aloud for the final questionnaire item, as it has a response set that is different from all other items on the questionnaire.

4. Answering questions. It is important that all questionnaire

administrators be trained to respond in the same way to questions from students. If students do not know what a particular word or question means, time should be taken to explain it (i.e., repeat question, reorder words in question, define words). If individual students experience discomfort with particular questions that is not relieved by reminders about the confidential nature of their responses, they should be allowed to skip those questions. Appropriate administrator training will ensure that all questionnaire administrators have the same understanding of the RAPS-SE items, and respond the same way to students.

III-


17

3. The RAPS-SE Questionnaire Student ID:_________________ Research Assessment Package for Schools -

Elementary School Student Report Read each of the following items. For each one, tell us how true it is for YOU by circling one of the four answers: VERY TRUE, SORT OF TRUE, NOT VERY TRUE, or NOT AT ALL TRUE. There are no right or wrong answers. 1. I have to be lucky to do well in school. A B C D

Very Sort of Not Very Not at all True True True True

2. When I’m with my teacher, I feel ignored. A B C D Very Sort of Not Very Not at all True True True True

3. I work very hard on my schoolwork. A B C D Very Sort of Not Very Not at all True True True True

4. My parents never have enough time to hear about what happens to me in school.

A B C D Very Sort of Not Very Not at all True True True True

5. I don’t know how to keep myself from getting bad grades.


6. I do my homework because I like to do it. A B C D Very Sort of Not Very Not at all True True True True

7. My parents don’t make it clear what they expect of me in school.


8. When I’m with my parents, I feel mad.


III-


18

9. My teacher is fair with me. A B C D Very Sort of Not Very Not at all True True True True

10. I’m pretty smart in school. A B C D Very Sort of Not Very Not at all True True True True

11. My parents don’t think I can do very much.


12. I wish I were someone else. A B C D Very Sort of Not Very Not at all True True True True

13. I work on my classwork because it’s interesting.


14. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I say the teacher didn’t cover the things on the test.


15. When I’m with my teacher, I feel mad. A B C D Very Sort of Not Very Not at all True True True True

16. My parents don’t explain why school is important.


17. I’m pretty lucky at getting good grades. A B C D Very Sort of Not Very Not at all True True True True

18. When I think about myself, I feel bad. A B C D Very Sort of Not Very Not at all True True True True

19. I don’t try very hard in school.


III-


19

20. I do my homework because I want to understand the subject.


21. My teacher’s expectations for me are way off base.


22. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I tell myself it didn’t matter.


23. I can do well in school if I want to. A B C D Very Sort of Not Very Not at all True True True True

24. My parents want me to do better than I can in school.


25. When I’m with my classmates, I feel ignored.


26. I work on my classwork because I think it’s important.


27. I never know what my parents are going to do.


28. My teacher doesn’t explain why we have to learn certain things in school.


29. I can get my teacher to like me. A B C D Very Sort of Not Very Not at all True True True True

30. When it comes to school, my parents try to control everything I do.


31. If I’m unlucky, I won’t do well in school. A B C D Very Sort of Not Very Not at all True True True True

III-


20

32. I can work really hard in school. A B C D


33. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I get angry at the teacher.


34. I don’t know what my parents want from me.


35. I pay attention in class. A B C D Very Sort of Not Very Not at all True True True True

36. I won’t do well in school if the teacher doesn’t like me.


37. I wish I felt better about myself. A B C D Very Sort of Not Very Not at all True True True True

38. My teacher doesn’t seem to have enough time for me.


39. I work on my classwork because I want to learn new things.


40. I’m not very smart in school. A B C D Very Sort of Not Very Not at all True True True True

41. When I’m with my teacher, I feel unhappy.


42. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I say it wasn’t important.


III-


21

43. Trying hard is the best way for me to do

well in school. A B C D Very Sort of Not Very Not at all True True True True

44. When I’m with my classmates, I feel mad. A B C D Very Sort of Not Very Not at all True True True True

45. I am unlucky in school. A B C D Very Sort of Not Very Not at all True True True True

46. I do my homework because it’s fun. A B C D Very Sort of Not Very Not at all True True True True

47. When I’m with my parents, I feel unhappy.


48. I don’t know what it takes to get good grades in school.


49. My teacher cares about how I do in school.


50. I don’t work very hard in school. A B C D Very Sort of Not Very Not at all True True True True

51. I can’t get my teacher to like me. A B C D Very Sort of Not Very Not at all True True True True

52. I do my homework because I want to learn new things.


53. When something bad happens to me in school (like not doing well in a test or not being able to answer an important question), I worry that the other students will think I’m dumb.


III-


22

54. My parents don’t let me make any of my

own decisions. A B C D Very Sort of Not Very Not at all True True True True

55. My parents are fair with me. A B C D Very Sort of Not Very Not at all True True True True

56. My teacher doesn’t talk about how schoolwork is related to what I want to be.


57. I wish I liked myself better. A B C D Very Sort of Not Very Not at all True True True True

58. When I’m with my parents, I feel good. A B C D Very Sort of Not Very Not at all True True True True

59. My teacher isn’t fair with me. A B C D Very Sort of Not Very Not at all True True True True

60. When I’m in class I just act as if I’m working.


61. The best way for me to get good grades is to get my teacher to like me.


62. My parents don’t seem to have enough time for me.


63. My teacher interrupts me when I have something to say.


64. I work on my classwork because doing well in school is important to me.


III-


23

65. When something bad happens to me in

school (like not doing well on a test or not being able to answer an important question), I say it was the teacher’s fault.


66. When it comes to school, my parents are always telling me what to do.


67. When I’m with my classmates, I feel good.


68. My teacher likes the other kids in my class better than me.


69. When I think about myself, I feel unhappy.


70. I work on my classwork because it’s fun. A B C D Very Sort of Not Very Not at all True True True True

71. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I say I didn’t care about it.


72. I can’t do well in school. A B C D Very Sort of Not Very Not at all True True True True

73. My parents trust me. A B C D Very Sort of Not Very Not at all True True True True

74. My teacher doesn’t make clear what he/she expects of me in school.


75. My parents don’t talk about how schoolwork is related to what I want to be.


III-


24

76. When I think about myself, I feel proud. A B C D


77. My parents try to control everything I do. A B C D Very Sort of Not Very Not at all True True True True

78. When I’m with my classmates, I feel unhappy.


79. My parents like to talk to me about school.


80. I can’t work very hard in school. A B C D Very Sort of Not Very Not at all True True True True

81. My teacher tries to control everything I do.


82. My parents know just how well I can do in school.


83. When I’m with my parents, I feel ignored.


84. When I’m with my teacher, I feel good. A B C D Very Sort of Not Very Not at all True True True True

85. I don’t know what my parents expect of me in school.


86. My parents encourage me to find out how schoolwork could be useful to me.


III-


25

87. My parents don’t seem to know how I feel about things.


88. How important is it to you to do the best you can in school?

A B C D Very Sort Of Not Very Not At All Important Important Important Important

III-


26

D. THE TEACHER-REPORT OF STUDENT ENGAGEMENT FOR MIDDLE SCHOOL

(RAPS-TM) 1. Item Description 2. Administration Information 3. The RAPS-TM Questionnaire

III-


27

1. Item Description The RAPS-TM is a very brief (3-item) teacher-report measure of student engagement in school. The three items are: 1. In my class, this student seems tuned in. (teexs54) 2. This student comes to class unprepared. (tebxf2) 3. This student does more than required. (tebxs1) Each of these items is responded to on the following scale:


(4) (3) (2) (1) 2. Administration Information Instructions for completing the RAPS-TM are fairly straightforward. Typically, teachers are given sufficient forms to provide information for all students in a single class. All teachers who are being asked to participate should be given the questionnaires at the same time and should be asked to return the questionnaires by a fixed deadline. The brevity of this measure makes it possible to have teachers report on numerous students without presenting an excessive time burden that could undermine teachers’ willingness to participate.

4 Codes in parentheses are item names that have been used in all IRRE-sponsored and facilitated administrations of this measure. They are included here primarily to facilitate communication with IRRE staff.

III-


28

3. The RAPS-TM Questionnaire Teacher ID: Research Assessment Package for Schools - Teacher Report On the following pages, please write in a student name or identification number for each of the students in your class who are participating in this study. Then, for EACH STUDENT, tell us how true each of the three items is for THAT STUDENT by circling one of the four answers: VERY TRUE, SORT OF TRUE, NOT VERY TRUE, or NOT AT ALL TRUE. Student ID:

1. In my class, this student seems tuned in. Very Sort of Not Very Not At All True True True True



Student ID:




III-


29

Student ID:




Student ID:




Student ID:




III-


30

E. THE STUDENT SELF-REPORT FOR MIDDLE SCHOOL (RAPS-SM)

1. Item by Construct Description 2. Administration Information 3. The RAPS-SM Questionnaire

III-


31

1. Item by Construct Description This multidimensional measure includes a total of 84 items, tapping three major domains and seven separate subdomains, in a questionnaire format that can be administered to students during a single class period. With one exception, all items are responded to on the following scale:


(4) (3) (2) (1) An item by construct list, including brief descriptions of each construct, is presented on the following pages. Engagement Domain The Engagement domain as measured by the RAPS-SM incorporates two separate but interrelated subdomains of student adjustment in school: 1) Ongoing Engagement with school and 2) Reactions to Challenge (i.e., the strategies students use when faced with negative or stressful school events). Ongoing Engagement Subdomain Ongoing Engagement includes the extent to which the student exerts effort on schoolwork, pays attention in class, prepares for class, and believes that doing well in school is personally important. The RAPS-SM includes a total of five items tapping these aspects of student engagement. These items are listed below: Item Item Code5 Questionnaire

Item Number I work very hard on my schoolwork. eoxs7 2 I don’t try very hard in school. eoxf3 21 I pay attention in class. eaxs2 31 I often come to class unprepared. epxf1 54 How important is it to you to do the best you can in school? eyxx3 84

5 As with the RAPS-TM, codes are item names that have been used in all IRRE-sponsored and facilitated administrations of this measure. They are included to facilitate communication among users of these measures and with IRRE staff.

III-


32

Eyxx3 is the single item that has a response set that differs from the other items in the questionnaire. It is responded to on the following scale: Very Important Sort of Important Not Very Important Not At All Important

(4) (3) (2) (1) Reaction to Challenge Subdomain There are a number of ways in which students may cope with, or react to, negative school-related events. Students may blame negative events on the teacher or other individuals (Projection). Students may cope with negative events by downplaying their importance (Denial). At other times, they may perseverate on the events and worry about them without necessarily doing anything to ensure that such events won’t occur again (Anxiety Amplification). Finally, students may cope by examining their own behavior and attempting to make changes to prevent similar negative events from occurring in the future (Positive Coping). Of these four different reactions to challenge, Positive Coping and Projection appear to be particularly related to positive or negative outcomes for middle school students and are included in this revised version of the RAPS-SM. The items are listed below: Item Item Code Questionnaire

Item Number Projection Items:

When something bad happens to me in school, I say the teacher didn’t cover the things on the test.

ecjx3 12

When something bad happens to me in school, I get angry at the teacher.

ecjx4 35

When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say it was the teacher’s fault.

ecjx1 57

Positive Coping Items: When something bad happens to me in school, I try to figure out what I did wrong so that it won’t happen again.

ecpx2 22

When something bad happens to me in school, I try to see what I did wrong.

ecpx4 43

When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I tell myself I’ll do better next time.

ecpx1 68

Beliefs About Self Domain The Beliefs about Self domain incorporates three separate subdomains: 1) Perceived Competence, 2) Perceived Autonomy, and 3) Perceived Relatedness.

III-


33

Perceived Competence Subdomain The Competence subdomain of the RAPS-SM consists of a total of 16 items. Two of these items reflect perceptions of control in the school setting. Six items reflect the student's beliefs about what kinds of strategies are necessary in order to achieve desirable, or avoid undesirable, outcomes. There are four different strategies represented: 1) Unknown, 2) Powerful Others, 3) Effort, and 5) Luck. A fifth strategy that appeared in earlier versions of the RAPS-SM, Ability, has been omitted because students’ endorsement of ability as a strategy for doing well in school did not appear to be associated with student performance and adjustment in school. The final eight Perceived Competence items reflect the student's belief that he or she has the capacity to enact strategies. There are two capacity items for each strategy except unknown, including Ability. Thus, although students’ beliefs that ability is necessary to do well in school does not appear to differ among students doing more or less well in school, students do differ in the extent to which they believe that they are smart in school. The unknown strategy items reflect the student's inability to determine what it takes to do well in school. Thus, it more accurately reflects the lack of a strategy and hence would have no associated feeling of being more or less capable of enacting the strategy in order to do well. The 16 items included in the revised RAPS-SM are listed below: Item Item Code Questionnaire

Item Number Perceptions of Control Items:

I can do well in school if I want to. ccxs5 28 I can’t do well in school. ccxf3 66

Unknown Strategy Items: I don’t know how to keep myself from getting bad grades. csuf2 6 I don’t know what it takes to get good grades in school. csus3 48

Powerful Others Strategy Items: The best way for me to get good grades is to get my teacher to like me.

csps2 64

Effort Strategy Items: If I don’t do well on my schoolwork, it’s because I didn’t try hard enough.

csef3 34

Trying hard is the best way for me to do well in school. cses2 49 Luck Strategy Items:

If I’m unlucky, I won’t do well in school. cslf1 26 Powerful Others Capacity Items:

I can get my teacher to like me. cpps2 23 I can’t get my teacher to like me. cppf2 80

Effort Capacity Items: I can work really hard in school. cpes2 24 I can’t work very hard in school. cpef3 73

III-


34


Item Number Ability Capacity Items:

I’m not very smart in school. cpaf2 42 I’m pretty smart in school. cpas4 82

Luck Capacity Items: I’m pretty lucky at getting good grades. cpls2 15 I am unlucky in school. cplf1 45

Perceived Autonomy Subdomain The Perceived Autonomy subdomain of the RAPS-SM is made up of nine items that tap three different types of self-regulation, or reasons for doing schoolwork or homework. Three items tap introjected self-regulation (i.e., doing schoolwork to avoid feeling embarrassed or guilty), three items involve identified self-regulation (i.e., doing schoolwork because it is seen as important), and three items tap intrinsic self-regulation (i.e., doing schoolwork because it is fun or interesting). These three forms of self-regulation are positively related to each other and are all expected to contribute to students’ academic performance. A fourth form of self-regulation involving doing schoolwork to avoid punishment or because it is the rule (external self-regulation) was included in previous versions of the RAPS-SM but was omitted from the current revised version because it was unrelated to other indices of student performance and adjustment. Autonomy, or Self-Regulation, items are listed below: Item Item Code Questionnaire

Item Number Introjected Self-Regulation Items:

I do my homework because I’ll feel bad about myself if I don’t do it. arjx1 16 I work on my classwork because I’ll be ashamed of myself if it doesn’t get done.

arjx2 51

I work on my classwork because I’ll feel guilty if I don’t do it. arjx7 72 Identified Self-Regulation Items:

I do my homework because I want to learn new things. ardx7 41 I work on my classwork because doing well in school is important to me.

ardx8 60

I work on my classwork because I think it is important. ardx9 83 Intrinsic Self-Regulation Items:

I do my homework because I like to do it. arix2 5 I work on my classwork because it’s interesting. arix8 11 I do my homework because it’s fun. arix1 46

III-


35

Perceived Relatedness Subdomain The Relatedness subdomain of the RAPS-SM is made up of 17 items that tap five constructs: 1) emotional security with self (3 items), 2) satisfaction with self (3 items), 3) parental emotional security (3 items), 4) teacher emotional security (4 items), and 5) peer emotional security (4 items). Three additional constructs were included in previous versions of the RAPS-SM: Parental Psychological Proximity Seeking, Teacher Psychological Proximity Seeking, and Peer Psychological Proximity Seeking. Due to low reliabilities and lack of associations with other indices of student performance and adjustment, however, these three components were excluded from the current revision. The Emotional Security constructs each reflect the extent to which positive emotions are present, and negative emotions absent, when thinking about the self, or in the presence of a relationship partner (i.e., parent(s), teacher, or peers). Satisfaction with self similarly reflects the extent to which students wish that they were different or were someone else. The 17 Perceived Relatedness items are as follows: Item Item Code Questionnaire

Item Number Emotional Security with Self Items:

When I think about myself, I feel bad. rsef11 18 When I think about myself, I feel happy. rses4 50 When I think about myself, I feel proud. rses8 71

Satisfaction with Self Items: I wish I were someone else. rswx4 14 I wish I felt better about myself. rswx3 38 I wish I liked myself better. rswx5 56

Parent Emotional Security Items: When I’m with my parents, I feel mad. rpef2 8 When I’m with my parents, I feel good. rpes1 30 When I’m with my parents, I feel unhappy. rpef4 47

Teacher Emotional Security Items: When I’m with my teacher, I feel good. rtes5 3 When I’m with my teacher, I feel mad. rtef2 17 When I’m with my teacher, I feel unhappy. rtef4 37 When I’m with my teacher, I feel happy. rtes2 69

Peer Emotional Security Items: When I’m with my classmates, I feel ignored. rref1 29 When I’m with my classmates, I feel mad. rref2 44 When I’m with my classmates, I feel good. rres1 62 When I’m with my classmates, I feel unhappy. rref4 74

III-


36

Experiences of Interpersonal Support Domain The Experiences of Interpersonal Support domain of the RAPS-SM includes a total of 31 items tapping two subdomains: 1) Parental Support; and 2) Teacher Support. Items within each subdomain reflect the extent to which the student feels that that the adult(s): 1) are involved with them; 2) provide support for student autonomy; and 3) provide structure. Involvement items reflect the extent to which the student feels that the adult(s) know and care about them. Autonomy Support items concern the extent to which students feel that they are allowed to make decisions for themselves, and the extent to which the real-life value of the work that they are asked to do is made clear by the adult(s). Structure items reflect the clarity of the adult(s) expectations regarding student conduct, the extent to which consequences are consistent and predictable by the student when expectations are not met, and the student’s perception of the fairness of the adult(s) expectations. In the RAPS-SM, Parental Support and Teacher Support constitute two separate subdomains, each encompassing involvement, autonomy support, and structure. All items referring to the teacher are specifically about the school setting. Items referring to the parent are either about school and schoolwork (School Context) or do not refer to any specific context (General Context). Experiences of Support from Parents Subdomain Item Item Code Questionnaire

Item Number Parent Involvement - School Context Items:

My parents never have enough time to hear about what happens to me in school.

ipxf6 4

My parents like to talk to me about school. ipxs2 58 Parent Autonomy Support - School Context Items:

My parents encourage me to find out how schoolwork could be useful to me.

ypxs14 10

My parents don’t explain why school is important. ypxf15 20 My parents don’t talk about how schoolwork is related to what I want to be.

ypxf14 75

Parent Structure - School Context Items: My parents don’t make it clear what they expect of me in school. spxf10 7 My parents know just how well I can do in school. spxs11 33 I don’t know what my parents expect of me in school. spxf4 70

Parent Involvement - General Context Items: My parents enjoy spending time with me. ipgs1 19 My parents do a lot to help me. ipgs2 36 My parents don’t seem to have enough time for me. ipgf1 61 My parents don’t seem to know how I feel about things. ipgf3 76

III-


37


Item Number Parent Autonomy Support - General Context Items:

My parents don’t let me make any of my own decisions. ypgf2 52 My parents trust me. ypgs1 77

Parent Structure - General Context Items: My parents don’t think I can do very much. spgf1 13 My parents are fair with me. spgs4 65 I don’t know what my parents want from me. spgf2 79

Experiences of Support from Teachers Subdomain Item Item Code Questionnaire

Item Number Teacher Involvement Items:

My teacher has plenty of time for me. itxs8 1 My teacher cares about how I do in school. itxs6 27 My teacher doesn’t seem to have enough time for me. itxf5 40 My teacher likes the other kids in my class better than me. itxf3 59 My teacher likes to be with me. itxs10 81

Teacher Autonomy Support Items: My teacher doesn’t explain why we have to learn certain things in school.

ytxf3 32

My teacher thinks what I say is important. ytxs16 55 My teacher interrupts me when I have something to say. ytxf6 63 My teacher tries to control everything I do. ytxf1 78

Teacher Structure Items: My teacher is fair with me. stxs11 9 The rules in my classroom are clear. stxs9 39 My teacher’s expectations for me are way off base. stxf12 25 My teacher isn’t fair with me. stxf11 53 My teacher doesn’t make clear what he/she expects of me in school.

stxf8 67

2. Administration Information Time The RAPS-SM has been designed to be administered to groups of students during one class period (approximately 50 minutes). All materials should be ready to distribute at the beginning of the allotted time, in order to ensure that all students will finish.

III-


38

Oral Questionnaire Administration and Training of Administrators Many of the items on the RAPS-SM ask students to discuss their feelings of relatedness with, and experiences of support from, teachers. Because of this, students may be hesitant to respond honestly if the administrator of the RAPS-SM is their teacher. If teachers are administering the RAPS-SM, a plan should be established so that each teacher administers the questionnaire to students other than his/her own. Whenever possible, questionnaire administrators (i.e. teachers, researchers, school administrators or other members of the school community) should read instructions and all items aloud to students, in addition to providing each student with a copy of the questionnaire. In our work in diverse school settings, we have found that reading to students at all levels speeds up the administration process and decreases the possibility that students will misread questions. Oral administration of the RAPS-SM increases the necessity for adequate training of questionnaire administrators. Among the most important elements of administration that training should cover are the following: 1. Standard instructions. All students should receive exactly the same

instructions at the beginning of the administration period. Examples of standard instructions that may be read to students can be obtained from IRRE.

2. Pacing. Items should be read at a brisk pace, adjusting that pace to a

comfortable rate for the class. We have found that if we keep the pace brisk, more students (of all grade and ability levels) stay on task than if questions are read slowly. Reading at a fairly rapid pace also encourages students to give the first response that comes to mind for each item.

3. Maintaining focus. It is important that administrators do as much as

possible to ensure that students are completing the questionnaires properly. An item’s number should always be read aloud before reading the item itself. This is particularly crucial when answer sheets (either for electronic or hand scoring ) are being used that may not have the questions themselves printed on them.

Reading the response scale aloud is also important. The scale (i.e.Very True, Sort of True, Not Very True, Not At All True) should be read aloud for the first 2-3 questions, and should then be repeated aloud periodically throughout the administration. When not reading the answer scale aloud, administrators should be instructed to repeat it to themselves following each question, in order to appropriately pace the reading of the items. Also, the scale should be read aloud for the final questionnaire item, as it

III-


39

has a response set that is different from all other items on the questionnaire.

4. Answering questions. It is important that all questionnaire

administrators be trained to respond in the same way to questions from students. If students do not know what a particular word or question means, time should be taken to explain it (i.e., repeat question, reorder words in question, define words). If individual students experience discomfort with particular questions that is not relieved by reminders about the confidential nature of their responses, they should be allowed to skip those questions. Appropriate administrator training will ensure that all questionnaire administrators have the same understanding of the RAPS-SM items, and respond the same way to students.

III-


40

3. The RAPS-SM Questionnaire Student ID:_________________ Research Assessment Package for Schools - Student Report Read each of the following items. For each one, tell us how true it is for YOU by circling one of the four answers: VERY TRUE, SORT OF TRUE, NOT VERY TRUE, or NOT AT ALL TRUE. There are no right or wrong answers. 1. My teacher has plenty of time for me. A B C D


2. I work very hard on my schoolwork. A B C D Very Sort of Not Very Not at all True True True True

3. When I’m with my teacher, I feel good. A B C D Very Sort of Not Very Not at all True True True True

4. My parents never have enough time to hear about what happens to me in school.


5. I do my homework because I like to do it. A B C D Very Sort of Not Very Not at all True True True True

6. I don’t know how to keep myself from getting bad grades.


7. My parents don’t make it clear what they expect of me in school.


8. When I’m with my parents, I feel mad. A B C D Very Sort of Not Very Not at all True True True True

III-


41

9. My teacher is fair with me. A B C D Very Sort of Not Very Not at all True True True True

10. My parents encourage me to find out how schoolwork could be useful to me.


11. I work on my classwork because it’s interesting.


12. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I say the teacher didn’t cover the things on the test.


13. My parents don’t think I can do very much.


14. I wish I were someone else. A B C D Very Sort of Not Very Not at all True True True True

15. I’m pretty lucky at getting good grades. A B C D Very Sort of Not Very Not at all True True True True

16. I do my homework because I’ll feel bad about myself if I don’t do it.


17. When I’m with my teacher, I feel mad. A B C D Very Sort of Not Very Not at all True True True True

18. When I think about myself, I feel bad. A B C D Very Sort of Not Very Not at all True True True True

19. My parents enjoy spending time with me. A B C D Very Sort of Not Very Not at all True True True True

III-


42

20. My parents don’t explain why school is important.


21. I don’t try very hard in school. A B C D Very Sort of Not Very Not at all True True True True

22. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I try to figure out what I did wrong so that it won’t happen again.


23. I can get my teacher to like me. A B C D Very Sort of Not Very Not at all True True True True

24. I can work really hard in school. A B C D Very Sort of Not Very Not at all True True True True

25. My teacher’s expectations for me are way off base.


26. If I’m unlucky, I won’t do well in school. A B C D Very Sort of Not Very Not at all True True True True

27. My teacher cares about how I do in school.


28. I can do well in school if I want to. A B C D Very Sort of Not Very Not at all True True True True

29. When I’m with my classmates, I feel ignored.


30. When I’m with my parents, I feel good. A B C D Very Sort of Not Very Not at all True True True True

III-


43

31. I pay attention in class. A B C D Very Sort of Not Very Not at all True True True True

32. My teacher doesn’t explain why we have to learn certain things in school.


33. My parents know just how well I can do in school.


34. If I don’t do well on my schoolwork, it’s because I didn’t try hard enough.


35. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I get angry at the teacher.


36. My parents do a lot to help me. A B C D Very Sort of Not Very Not at all True True True True

37. When I’m with my teacher, I feel unhappy.


38. I wish I felt better about myself. A B C D Very Sort of Not Very Not at all True True True True

39. The rules in my classroom are clear. A B C D Very Sort of Not Very Not at all True True True True

40. My teacher doesn’t seem to have enough time for me.


41. I do my homework because I want to learn new things.


III-


44

42. I’m not very smart in school. A B C D Very Sort of Not Very Not at all True True True True

43. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I try to see what I did wrong.


44. When I’m with my classmates, I feel mad. A B C D Very Sort of Not Very Not at all True True True True

45. I am unlucky in school. A B C D Very Sort of Not Very Not at all True True True True

46. I do my homework because it’s fun. A B C D Very Sort of Not Very Not at all True True True True

47. When I’m with my parents, I feel unhappy.


48. I don’t know what it takes to get good grades in school.


49. Trying hard is the best way for me to do well in school.


50. When I think about myself, I feel happy. A B C D Very Sort of Not Very Not at all True True True True

51. I work on my classwork because I’ll be ashamed of myself if it doesn’t get done.


52. My parents don’t let me make any of my own decisions.


III-


45

53. My teacher isn’t fair with me. A B C D Very Sort of Not Very Not at all True True True True

54. I often come to class unprepared. A B C D Very Sort of Not Very Not at all True True True True

55. My teacher thinks what I say is important. A B C D Very Sort of Not Very Not at all True True True True

56. I wish I liked myself better. A B C D Very Sort of Not Very Not at all True True True True

57. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I say it was the teacher’s fault.


58. My parents like to talk to me about school.


59. My teacher likes the other kids in my class better than me.


60. I work on my classwork because doing well in school is important to me.


61. My parents don’t seem to have enough time for me.


62. When I’m with my classmates, I feel good.


63. My teacher interrupts me when I have something to say.


III-


46

64. The best way for me to get good grades is to get my teacher to like me.


65. My parents are fair with me. A B C D Very Sort of Not Very Not at all True True True True

66. I can’t do well in school. A B C D Very Sort of Not Very Not at all True True True True

67. My teacher doesn’t make clear what he/she expects of me in school.


68. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question), I tell myself I’ll do better next time.


69. When I’m with my teacher, I feel happy. A B C D Very Sort of Not Very Not at all True True True True

70. I don’t know what my parents expect of me in school.


71. When I think about myself, I feel proud. A B C D Very Sort of Not Very Not at all True True True True

72. I work on my classwork because I’ll feel guilty if I don’t do it.


73. I can’t work very hard in school. A B C D Very Sort of Not Very Not at all True True True True

74. When I’m with my classmates, I feel unhappy.


III-


47

75. My parents don’t talk about how schoolwork is related to what I want to be.


76. My parents don’t seem to know how I feel about things.


77. My parents trust me. A B C D Very Sort of Not Very Not at all True True True True

78. My teacher tries to control everything I do.


79. I don’t know what my parents want from me.


80. I can’t get my teacher to like me. A B C D Very Sort of Not Very Not at all True True True True

81. My teacher likes to be with me. A B C D Very Sort of Not Very Not at all True True True True

82. I’m pretty smart in school. A B C D Very Sort of Not Very Not at all True True True True

83. I work on my classwork because I think it is important.


84. How important is it to you to do the best you can in school?

A B C D Very Sort Of Not Very Not At All Important Important Important Important

IV-


1

SCORING AND INTERPRETATION OPTIONS FOR RAPS SCHOOL QUESTIONNAIRES:

RAPS-T and RAPS-S

A. Elementary Options I: Composite Scores

1. The RAPS-TE Composite

A single summary score is derived from this three-item teacher report. One of the items is negative (reflecting lack of engagement) and must be reversed by subtracting each student’s score from five. The Student Engagement composite score can then be created by obtaining the mean of the three items. STUDENT ENGAGEMENT = Mean of teexs5, tebxs1, and tebxf2(rev.) 2. RAPS-SE Composites

The 79 items of the RAPS-SE are combined to create a total of 17 composite scores tapping three major domains and seven subdomains. By domain, the composites are as follows:

A. DOMAIN: ENGAGEMENT - 3 Composites 1. Subdomain: Ongoing Engagement 2. Subdomain: Reaction to Challenge 3. Domain Summary Score: ENGAGEMENT

B. DOMAIN: BELIEFS ABOUT SELF - 3 Composites Subdomain: Perceived Competence - 3 Composites 1. Construct: Promote Competence 2. Construct: Undermine Competence 3. Subdomain Summary Score: Maximize Competence Subdomain: Perceived Autonomy - 2 Composites: 4. Construct: Identified Self-Regulation 5. Construct: Intrinsic Self-Regulation Subdomain: Perceived Relatedness - 5 Composites: 6. Construct: Feelings about Self 7. Construct: Parental Emotional Security 8. Construct: Teacher Emotional Security 9. Construct: Peer Emotional Security

IV-


2

10. Subdomain Summary Score: Relatedness 11. Domain Summary Score: BELIEFS ABOUT SELF

C. DOMAIN: EXPERIENCE OF INTERPERSONAL SUPPORT - 3 Composites

1. Subdomain: Parental Support 2. Subdomain: Teacher Support 3. Domain Summary Score: INTERPERSONAL SUPPORT In creating summary scores reflecting domains, subdomains, and constructs, negative items on scales where some items are negative and others positive must be reversed prior to compositing. Because all items are responded to on a four-point scale, item reversals can be accomplished by subtracting each individual student’s score on that item from five (so that 4 becomes 1, 3 becomes 2, 2 becomes 3, and 1 becomes 4). In the following formulas, item scores that must be reversed are indicated by (rev.). Further, all composite scores should be created so that a high score reflects positive outcomes. In cases where all items in a composite are negative, the entire summary score can be reversed by subtracting each student’s mean score for the composite from five. Composite level reversals are indicated in the tables below.

A. Engagement Domain Composite: Construction: Subdomain Level Composite: Ongoing Engagement

Mean of the 6 ongoing engagement items: eaxs2, eoxs7, eyxx3, eaxf3(rev.), eoxf3(rev.), and eoxf4(rev.)

Subdomain Level Composite: Reaction to Challenge

Reversed mean of the 3 denial, 3 projection, and 1 anxiety amplification: ecdx3, ecdx4, ecdx7, ecjx1, ecjx3, ecjx4, and ecax1

Domain Level Composite: Engagement

Mean of the 2 subdomain level composites: Ongoing Engagement and Reaction to Challenge

IV-


3

B. Beliefs About Self Domain Subdomain: Perceived Competence Composite: Construction: Construct Level Composite: Promote Competence

6 Items: ccxf3(rev.), cses2, cpes2, cpas4, cpps2, cpls2 Formula: Promote Comptence = (ccxf3(rev.) u 4) + (cses2 u cpes2) + (cpas4 u 4) + (cpps2 u 4) + (cpls2 u 4)

Construct Level Composite: Undermine Competence

10 Items: csuf2, csus3, cpef3, cpaf2, cspf1, csps2, cppf2, cslf1, cslf2, cplf1

Formula: csu=mean of csuf2 and csus3 csp=mean of cspf1 and csps2 csl=mean of cslf1 and csls2 Undermine Comptence=(csu u 4) + (cpef3 u 4) + (cpaf2 u 4) + (csp u cppf2) + (csl u cplf1)

Subdomain Level Composite: Maximize Competence

((Promote Competence - Undermines Competence ) + 94) y 40

Subdomain: Perceived Autonomy Composite: Construction: Construct Level Composite: Identified Self-Regulation

Mean of the 5 identified self-regulation items: ardx1, ardx2, ardx7, ardx8, and ardx9

Construct Level Composite: Intrinsic Self-Regulation

Mean of the 4 intrinsic self-regulation items: arix1, arix2, arix3, and arix8

(No subdomain level composite) ---

IV-


4

Subdomain: Perceived Relatedness Composite: Construction: Construct Level Composite: Feelings about Self

Reversed mean of the 5 Feelings About Self items: rsef10, rsef11, rswx3, rswx4, and rswx5

Construct Level Composite: Parental Emotional Security

Reversed mean of the 3 Parental Emotional Security: rpef1, rpef2, and rpef4

Construct Level Composite: Teacher Emotional Security

Reversed mean of the 3 Teacher Emotional Security items: rtef1, rtef2, and rtef4

Construct Level Composite: Peer Emotional Security

Reversed mean of the 3 Peer Emotional Security items: rref1, rref2, and rref4

Subdomain Level Composite: Relatedness

Mean of 4 construct level composites: Feelings About Self, Parental Emotional Security, Teacher Emotional Security, and Peer Emotional Security

Beliefs About Self Domain Summary Composite: Construction: Domain Level Composite: Beliefs About Self

Mean of the 2 subdomain level composite: Control Maximization and Relatedness

C. Experience of Interpersonal Support Domain Composite: Construction: Subdomain Level Composite: Parental Support

Mean of the 17 Parental Support Items: 1 Involvement - School Context item: ipxf6(rev.); 4 Autonomy Support - School Context items: ypxf4(rev.),

ypxf5(rev.), ypxf14(rev.), and ypxf15(rev.); 3 Structure - School Context items: spxf4(rev.), spxf10(rev.),

and spxf12(rev.); 2 Involvement - General Context items: ipgf1(rev.), and

ipgf3(rev.); 3 Autonomy Support - General Context items: ypgs1,

ypgf1(rev.), and ypgf2(rev.); and 4 Structure - General Context items: spgs4, spgf1(rev.),

spgf2(rev.), and spgf3(rev.) Subdomain Level Composite: Teacher Support

Mean of the 10 Teacher Support Items: 2 Involvement items: itxf3(rev.) and itxf5(rev.); 4 Autonomy Support items: ytxf1(rev.), ytxf3(rev.), ytxf6(rev.),

and ytxf14(rev.); and 4 Structure items: stxs11, stxf8(rev.), stxf11(rev.), and

stxf12(rev.) Domain Level Composite: Experience of Interpersonal Support

Mean of the 2 subdomain level composites: Parental Support and Teacher Support

IV-


5

B. Elementary Options II: High Risk and Optimal Indicators 1. RAPS-TE Indicators Indicators of Optimal and High Risk levels of student engagement in school can be derived from the RAPS-TE Student Engagement composite score. These indicators reflect thresholds in the scale of the composite score that have been found to differentiate between elementary school students who demonstrate significantly greater risk for poor academic performance and adjustment difficulties than do other elementary school students, or who are more likely to show successful performance and adjustment than are other students. For both of these indicators (Optimal and High Risk), students with summary scores meeting the criterion for classification receive a score of 1, while all other students receive a score of 0. The criterion levels for classification for each indicator are as follows:

High-Risk Indicator Optimal Indicator

RAPS-TE Student Engagement Summary Score < 2.60

RAPS-TE Student Engagement Summary Score t 3.50

2. RAPS-SE Indicators Indicators of Optimal and High Risk levels of student adjustment in school can also be derived from the composite scores for the three major domains covered by the RAPS-SE measure: Engagement, Beliefs About Self, and Interpersonal Support. Because of the likelihood that support from parents and from teachers may be differentially associated with educational outcomes, separate Optimal and High Risk indicators for Parental Support and for Teacher Support have been established as well. These Optimal and High Risk indicators reflect thresholds in the scales of the composite scores that have been found to differentiate between elementary school students who demonstrate significantly greater risk for poor academic performance and adjustment difficulties than do other students, or who are more likely to show successful performance and adjustment than are other elementary school students. For each of these 10 indicators (5 Optimal and 5 High Risk), students with summary scores meeting the criterion for classification receive a score of 1, while all other students receive a score of 0. The criterion levels for classification for each indicator are as follows:

IV-


6

COMPOSITE High-Risk Indicator Optimal Indicator

1. Engagement Summary Score < 3.25 Summary Score t 3.75 2. Beliefs about Self Summary Score < 3.00 Summary Score t 3.75 3a. Interpersonal Support Summary Score < 2.75 Summary Score t 3.50 3b. Parental Support Summary Score < 3.00 Summary Score t 3.25 3c. Teacher Support Summary Score < 2.50 Summary Score t 3.50

IV-


7

C. Middle School Options I: Composite Scores

1. The RAPS-TM Composite

A single summary score is derived from this three-item teacher report. One of the items is negative (reflecting lack of engagement) and must be reversed by subtracting each student’s score from five. The Student Engagement composite score can then be created by obtaining the mean of the three items. STUDENT ENGAGEMENT = Mean of teexs5, tebxs1, and tebxf2(rev.) 2. RAPS-SM Composites

The 84 items of the RAPS-SM are combined to create a total of 19 composite scores tapping three major domains and seven subdomains. By domain, the composites are as follows: A. DOMAIN: ENGAGEMENT - 3 Composites 1. Subdomain: Ongoing Engagement 2. Subdomain: Reaction to Challenge 3. Domain Summary Score: ENGAGEMENT B. DOMAIN: BELIEFS ABOUT SELF - 13 Composites Subdomain: Perceived Competence - 3 Composites 1. Construct: Promote Competence 2. Construct: Undermine Competence 3. Subdomain Summary Score: Maximize Competence Subdomain: Perceived Autonomy - 3 Composites: 4. Construct: Introjected Self-Regulation 5. Construct: Identified Self-Regulation 6. Construct: Intrinsic Self-Regulation Subdomain: Perceived Relatedness - 6 Composites: 7. Construct: Emotional Security with Self 8. Construct: Satisfaction with Self 9. Construct: Parental Emotional Security 10. Construct: Teacher Emotional Security 11. Construct: Peer Emotional Security 12. Subdomain Summary Score: Relatedness 13. Domain Summary Score: BELIEFS ABOUT SELF

IV-


8

C. DOMAIN: EXPERIENCE OF INTERPERSONAL SUPPORT - 3 Composites 1. Subdomain: Parental Support 2. Subdomain: Teacher Support 3. Domain Summary Score: INTERPERSONAL SUPPORT In creating summary scores reflecting domains, subdomains, and constructs, negative items must be reversed prior to compositing. Because all items are responded to on a four-point scale, item reversals can be accomplished by subtracting each individual student’s score on that item from five (so that 4 becomes 1, 3 becomes 2, 2 becomes 3, and 1 becomes 4). In the following formulas, items that must be reversed are indicated by (rev.).

A. Engagement Domain Composite: Construction: Subdomain Level Composite: Ongoing Engagement

Mean of the 5 ongoing engagement items: eaxs2, eoxs7, eyxx3, eoxf3(rev.) and epxf1(rev.)

Subdomain Level Composite: Reaction to Challenge

Mean of the 3 positive coping and 3 projection items (all projection items reversed): ecpx1, ecpx2, ecpx4, ecjx1(rev.), ecjx3(rev.), and ecjx4(rev.)

Domain Level Composite: Engagement

Mean of the 2 subdomain level composites: Ongoing Engagement and Reaction to Challenge

IV-


9

B. Beliefs About Self Domain Subdomain: Perceived Competence Composite: Construction: Construct Level Composite: Promote Competence

8 Items: ccxs5, ccxf3(rev.), csef3, cses2, cpes2, cpas4, cpps2, cpls2

Formula: ccx=mean of ccxs5 and ccxf3(rev.) cse=mean of csef3 and cses2 Promote Comptence=(ccx u 4) + (cse u cpes2) + (cpas4 u 4) + (cpps2 u 4) + (cpls2 u 4)

Construct Level Composite: Undermine Competence

8 Items: csuf2 csus3, cpef3, cpaf2, csps2, cppf2, cslf1, cplf1 Formula: csu=mean of csuf2 and csus3 Undermine Comptence=(csu u 4) + (cpef3 u 4) + (cpaf2 u 4) + (csps2 u cppf2) + (cslf1 u cplf1)

Subdomain Level Composite: Maximize Competence

((Promote Competence - Undermines Competence ) + 94) y 40

Subdomain: Perceived Autonomy Composite: Construction: Construct Level Composite: Introjected Self-Regulation

Mean of the 3 introjected self-regulation items: arjx1, arjx2, and arjx7

Construct Level Composite: Identified Self-Regulation

Mean of the 3 identified self-regulation items: ardx7, ardx8, and ardx9

Construct Level Composite: Intrinsic Self-Regulation

Mean of the 3 intrinsic self-regulation items: arix1, arix2, and arix8

(No subdomain level composite) ---

IV-


10

Subdomain: Perceived Relatedness Composite: Construction: Construct Level Composite: Emotional Security with Self

Mean of the 3 Emotional Security with Self items: rses4, rses8, and rsef11(rev.)

Construct Level Composite: Satisfaction with Self

Mean of the 3 Satisfaction with Self items (all reversed): rswx3(rev.), rswx4(rev.), and rswx5(rev.)

Construct Level Composite: Parental Emotional Security

Mean of the 3 Parental Emotional Security: rpes1, rpef2(rev.), and rpef4(rev.)

Construct Level Composite: Teacher Emotional Security

Mean of the 4 Teacher Emotional Security items: rtes2, rtes5, rtef2(rev.), and rtef4(rev.)

Construct Level Composite: Peer Emotional Security

Mean of the 4 Peer Emotional Security items: rres1, rref1(rev.), rref2(rev.), and rref4(rev.)

Subdomain Level Composite: Relatedness

Mean of 5 construct level composites: Emotional Security with Self, Satisfaction with Self, Parental Emotional Security, Teacher Emotional Security, and Peer Emotional Security

Beliefs About Self Domain Summary Composite: Construction: Domain Level Composite: Beliefs About Self

Mean of the 2 subdomain level composites and 1 perceived autonomy construct level composite: Control Maximization, Relatedness, and Identified Self-Regulation

IV-


11

C. Experience of Interpersonal Support Domain Composite: Construction: Subdomain Level Composite: Parental Support

Mean of the 17 Parental Support Items: 2 Involvement - School Context items: ipxs2 and ipxf6(rev.); 3 Autonomy Support - School Context items: ypxs14,

ypxf14(rev.), and ypxf15(rev.); 3 Structure - School Context items: spxs11, spxf4(rev.), and

spxf10(rev.); 4 Involvement - General Context items: ipgs1, ipgs2,

ipgf1(rev.), and ipgf3(rev.); 2 Autonomy Support - General Context items: ypgs1 and

ypgf2(rev.); and 3 Structure - General Context items: spgs4, spgf1(rev.), and

spgf2(rev.) Subdomain Level Composite: Teacher Support

Mean of the 14 Teacher Support Items: 5 Involvement items: itxs6, itxs8, itxs10, itxf3(rev.), and

itxf5(rev.); 4 Autonomy Support items: ytxs16, ytxf1(rev.), ytxf3(rev.),

and ytxf6(rev.); and 5 Structure items: stxs9, stxs11, stxf8(rev.), stxf11(rev.), and

stxf12(rev.) Domain Level Composite: Experience of Interpersonal Support

Mean of the 2 subdomain level composites: Parental Support and Teacher Support

IV-


12

D. Middle School Options II: High Risk and Optimal Indicators 1. RAPS-TM Indicators Indicators of Optimal and High Risk levels of student engagement in school can be derived from the RAPS-TM Student Engagement composite score. These indicators reflect thresholds in the scale of the composite score that have been found to differentiate between middle school students who demonstrate significantly greater risk for poor academic performance and adjustment difficulties than do other middle school students, or who are more likely to show successful performance and adjustment than are other middle school students. For both of these indicators (Optimal and High Risk), students with summary scores meeting the criterion for classification receive a score of 1, while all other students receive a score of 0. The criterion levels for classification for each indicator are as follows:

High-Risk Indicator Optimal Indicator

RAPS-TM Student Engagement Summary Score < 2.30

RAPS-TM Student Engagement Summary Score t 3.50

2. RAPS-SM Indicators Indicators of Optimal and High Risk levels of student adjustment in school can also be derived from the composite scores for the three major domains covered by the RAPS-SM measure: Engagement, Beliefs about Self, and Interpersonal Support. Because of the likelihood that support from parents and from teachers may be differentially associated with educational outcomes, separate Optimal and High Risk indicators for Parental Support and for Teacher Support have been established as well. These Optimal and High Risk indicators reflect thresholds in the scales of the composite scores that have been found to differentiate between middle school students who demonstrate significantly greater risk for poor academic performance and adjustment difficulties than do other middle school students, or who are more likely to show successful performance and adjustment than are other middle school students. For each of these 10 indicators (5 Optimal and 5 High Risk), students with summary scores meeting the criterion for classification receive a score of 1, while all other students receive a score of 0. The criterion levels for classification for each indicator are as follows:

IV-


13

COMPOSITE High-Risk Indicator Optimal Indicator

1. Engagement Summary Score < 3.00 Summary Score t 3.75 2. Beliefs about Self Summary Score < 3.00 Summary Score t 3.75 3a. Interpersonal Support Summary Score < 2.75 Summary Score t 3.50 3b. Parental Support Summary Score < 3.00 Summary Score t 3.75 3c. Teacher Support Summary Score < 2.75 Summary Score t 3.50

V-


1

RELIABILITY AND VALIDITY INFORMATION

ELEMENTARY SCHOOL: RAPS-TE AND RAPS-SE COMPOSITES AND INDICATORS

A. RAPS-TE In this section we provide information on the reliability and validity of the composite score and Optimal and High Risk indices developed from the RAPS-TE. As described in a previous section, the RAPS-TE was designed to tap student engagement in school. The validity of the composite Student Engagement measure and of the Optimal and High Risk indicators is assessed by examining the strength of the associations of these measures with the elementary school and middle school performance and adjustment indicators described below. On the following pages, we first present a brief description of the sample of students used to assess the reliability and validity of the RAPS-TE composite and indices. Next is reliability information for the composite and a summary of the validity analyses. Readers who would like more detailed information, including a breakdown for males and females separately, are referred to Appendix B. 1. Characteristics of the Reliability Sample Teacher-report data were obtained from studies conducted in six elementary schools within one urban school district. Teacher report data were matched to administrative record data obtained from the school district. All students with elementary school teacher report data and administrative record data in elementary school, middle school, or both, were included in the resulting data set. The final number of subjects was 2026. Table 1: Demographic characteristics of the reliability sample

Age range 7 through 12 Grades 3rd through 5th Ethnicity 82% African-American

9% Euro-American 9% Hispanic

Gender 51% male 49% female

SES (Based on eligibility for free- or reduced-price school lunches)

85% eligible

V-


2

2. Reliability of the RAPS-TE Student Engagement Composite The three items that comprise the RAPS-TE Student Engagement composite had an average interitem correlation of .59, and an alpha reliability of .81. 3. Assessing the Validity of the RAPS-TE Student Engagement Composite

and Indicators As mentioned above, the validity of the composite RAPS-TE Student Engagement measure and Optimal and High Risk indicators is evaluated by examining the strength of the associations of these measures with elementary school and middle school performance and commitment indicators. The development of High Risk and Optimal indicators of student performance and commitment based on administrative record data from elementary school (SPCI-E) and middle school (SPCI-M) are described in separate technical reports.6,7 Each Optimal and High Risk indicator is based on student attendance records and percentile scores on standardized reading achievement tests. As with the RAPS-TE and RAPS-SE indices described in a previous section, students received a score of 1 on an Optimal or High Risk indicator if they met the threshold criteria for inclusion, and a score of 0 if they did not. The threshold criteria are presented below. Table 2: Outcome Indices

Indicator Threshold Criteria Elementary School (SPCI-E) High Risk a. elementary school attendance rate below 89% or

b. standardized math achievement test percentile score below 35%, or both

Optimal a. elementary school attendance rate of 97% or higher and one or both of the following:

b. standardized reading achievement test percentile score of 70% or higher

c. standardized math achievement test percentile score of 80% or higher Middle School (SPCI-M) High Risk a. middle school attendance rate below 79% or

b. standardized reading achievement test percentile score below 25%, or both

Optimal a. middle school attendance rate of 93% or higher and one or both of the following:


c. standardized math achievement test percentile score of 65% or higher

6 Bridges, L.J., & Connell, J.P. (1998). Student Performance and Commitment Index (SPCI), I: Calibration of Middle School and High School Stars and Flags. Unpublished IRRE technical report. 7 Bridges, L.J., & Connell, J.P. (1998). Student Performance and Adjustment Index (SPCI), II: Calibration of Elementary School Stars and Flags. Unpublished IRRE technical report.

V-


3

Table 3 presents a summary of the strength of the associations between the RAPS-TE Engagement measures and the criterion indices with which they were expected to be associated. The strengths of the associations between the composite score, which is a continuous variable, and the criterion indices, which are in all cases dichotomous (i.e. High Risk v. Other, Optimal v. Other, and High Risk v. Optimal) are assessed using point-biserial correlations. The strengths of the associations between the RAPS-TE Engagement indices and the criterion indices, both of which are dichotomous variables, are assessed as phi coefficients.8 Table 3: Associations between the RAPS-TE Engagement Composite and Indices and Criterion Indices

Criterion Indices SPCI-E (Elementary School) SPCI-M (Middle School)

RAPS-TE Engagement

Predictor

High Risk Optimal High Risk v. Optimal

(Extreme Groups)


(Extreme Groups)

Composite (Continuous) V V V V V V

Indicators (High Risk v.

Other, Optimal v Other, or

High Risk v. Optimal)

V V V V V V

Note 1: In Table 3, High Risk criterion indices are predicted by High Risk

indices, Optimal criterion indices are predicted by Optimal indices, and High Risk v. Optimal criterion indices are predicted by High Risk v. Optimal indices.

Note 2: Validity coefficients represent the strength of the associations

between predictor variables and criterion variables:

V = correlations and phi coefficients of .10 - .49 (all significant at p<.0001)

V =correlations and phi coefficients of .50 and higher (all significant at p<.0001)

8 The phi coefficient (M) is derived from the chi square (F2) statistic and has the same properties as a correlation.

V-


4

B. RAPS-SE In this section we provide information on the reliability and validity of the composite scores and Optimal and High Risk indices developed from the RAPS-SE. As described in a previous section, the RAPS-SE was designed to tap the supports and opportunities available to students in school. Our approach to assessing the validity of the RAPS-SE measures was dictated by our motivational model, in which students’ experience of support directly affects their beliefs about themselves, which in turn affects their engagement with school. Engagement with school, in turn, is the most direct influence on student performance and adjustment in school. Based on this theoretical ordering of the constructs within the model, the criterion measures used to assess validity of the various components of the RAPS-SE were as follows: 1. The validity of Student Engagement composites and Optimal and High Risk

indicators is assessed by examining the strength of the associations of these measures with elementary school and middle school performance and adjustment indicators (SPCI-E Optimal and High Risk and SPCI-M Optimal and High Risk) described on page 2. Also examined is the strength of the associations between the RAPS-SE (self-report) Engagement composites and the teacher-report of student engagement (RAPS-TE) indices.

2. The validity of student Beliefs about Self components and Optimal and High

Risk indicators is primarily evaluated with respect to the strength of the associations these components demonstrate with both the RAPS-TE (teacher-report) and the RAPS-SE Engagement High Risk and Optimal indicators. Further, valid Beliefs about Self components and indicators should continue to demonstrate significant associations with SPCI-E High Risk and Optimal indicators. Thus, these associations are examined as well.

3. Finally, the validity of composites and Optimal and High Risk indicators

reflecting students’ perceptions of Interpersonal Support is evaluated with respect to the associations these composites demonstrate with the RAPS-SE Beliefs about Self High Risk and Optimal indicators. As with the Beliefs About Self components, the associations between Interpersonal Support components and the other criterion indices (SPCI-E Optimal and High Risk, RAPS-TE Engagement Optimal and High Risk, and RAPS-SE Engagement Optimal and High Risk) are also examined.

On the following pages, we first present a brief description of the sample of students used to assess reliability and validity of the RAPS-SE components. Next is reliability information for each composite, followed by a summary of the validity analyses. Readers who would like more detailed information, including a breakdown for males and females separately, are referred to Appendix C.

V-


5

1. Characteristics of the Reliability Sample Student-report data were obtained from studies conducted in six elementary schools within one urban school district. Table 1 presents general characteristics of the reliability sample. Table 1:

Sample size: Total number of students in sample 1846 Number of students with self-report (RAPS-SE) data 1846 Number of students with elementary school administrative records data

1600

Number of students with middle school administrative records data

660

Number of students with teacher-report (RAPS-TE) data

1750

Demographic characteristics: Age range 7 through 13 Grades 3rd through 5th Ethnicity 81% African-American




85% eligible

V-


6

2. Reliabilities of the RAPS-SE Composites Table 2 presents a summary of the alpha reliabilities of each of the composites generated from the RAPS-SE. Table 2: Alpha reliabilities of the RAPS-SE composites

COMPOSITE NUMBER OF ITEMS RELIABILITY (�)

Domain: Engagement 1. Ongoing Engagement 6 .66 2. Reaction to Challenge 7 .73

3. ENGAGEMENT (Summary Score) 13 .71

Domain: Beliefs About Self Perceived Competence

1. Promote Competence 6 .52 2. Undermine Competence 10 .77 3. Maximize Competence (Summary Score) 16 .77

Perceived Autonomy 4. Identified Self-Regulation 5 .78 5. Intrinsic Self-Regulation 4 .80

Perceived Relatedness 6. Emotional Security with Self 5 .68 7. Parental Emotional Security 3 .76 8. Teacher Emotional Security 3 .74 9. Peer Emotional Security 3 .79 10. Relatedness (Summary Score) 14 .83

13. BELIEFS ABOUT SELF (Summary Score) 30 .87

Domain: Experiences of Interpersonal Support 1. Parental Support 17 .82 2. Teacher Support 10 .80

3. INTERPERSONAL SUPPORT (Summary Score) 27 .87

3. Assessing the Validity of the RAPS-SE Composites and Indicators As discussed earlier, the validity of the various RAPS-SE measures is assessed by examining the extent to which each measure is associated with the criterion variables with which it is expected to be associated. The strengths of the associations between the RAPS-SE composite scores, which are continuous variables, and the criterion indices, which are in all cases dichotomous (i.e., High Risk v. Not High Risk, Optimal v. Not Optimal, High Risk v. Optimal) are assessed

V-


7

using point-biserial correlations. The strengths of the associations between the RAPS-SE indices and the criterion indices, both of which are dichotomous variables, are assessed as phi coefficients.9 Validity information is presented at two levels on the following pages: 1. Tables 3 through 5 present information on the associations between

measures reflecting the three major domains tapped by the RAPS-SE (Engagement, Beliefs about Self, and Experience of Interpersonal Support) and the primary criterion variables with which they are expected to be associated. Because of the importance of separate indices reflecting experiences of Parental Support and Teacher Support (as well as the overall Interpersonal Support), these two subdomains are included in the Interpersonal Support table.

2. Tables 6 and 7 present similar information separately for each of the remaining 12 composites from the five additional subdomains tapped by the RAPS-SE (Ongoing Engagement, Reaction to Challenge, Perceived Competence, Perceived Autonomy, and Perceived Relatedness).

As noted earlier, more detailed information on the associations between RAPS-SE measures and outcome indices may be found in Appendix C.


V-


8

Table 3: Associations between the Domain-Level RAPS-SE Engagement Composite and Indices and the Criterion Indices

Criterion Variables SPCI Elementary School SPCI Middle School RAPS-TE (Engagement)

RAPS-SE Engagement

Variable

High Risk v. Other

Optimal v. Other

High Risk v. Optimal

High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


Composite (Continuous) V V V V V V V V V


Other, Optimal v. Other, or


V V V V v V V V V

Note 1: In Table 3 (above), Table 4, and Table 5, High Risk criterion indices are

predicted by High Risk indices, Optimal criterion indices are predicted by Optimal indices, and High Risk v. Optimal criterion indices are predicted by High Risk v. Optimal indices.

Note2: In all tables, validity coefficients represent the strength of the

associations between predictor variables and criterion variables: v = correlations and phi coefficients of less than .10 but significant at

p<.05 V = correlations and phi coefficients of .10 - .49 (all significant at

p<.0001)


V-


9

Table 4: Associations between the Domain-Level RAPS-SE Beliefs about Self Composite and Indices and the Criterion Indices

Criterion Indices RAPS-TE (Engagement) RAPS-SE Engagement

RAPS-SE Beliefs about Self Predictor

High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


Composite (Continuous) V V V V

V V Indicators

(High Risk v. Other, Optimal

v Other, or High Risk v.

Optimal)

V V V V V V

Table 5: Associations between the Experiences of Parental Support (Subdomain-Level), Teacher Support (Subdomain-Level), and Interpersonal Support (Domain-Level) Composites and Indices and the Criterion Indices

Criterion Variables: RAPS-SE Beliefs about Self RAPS-SE Experience of Interpersonal

Support Variable High Risk v.

Other Optimal v. Other High Risk v.

Optimal Experience of Parental Support Composite (Continuous) V

V V Indicators (High Risk, Optimal, or High Risk v. Optimal) V V V Experience of Teacher Support Composite (Continuous) V

V V Indicators (High Risk, Optimal, or High Risk v. Optimal) V V V Experience of Interpersonal Support Composite (Continuous) V

V V Indicators (High Risk, Optimal, or High Risk v. Optimal) V

V V

V-


10

Table 6: Associations between the Subdomain-Level RAPS-SE Engagement Composites and the Criterion Indices

Criterion Indices SPCI Elementary School SPCI Middle School RAPS-TE (Engagement)

RAPS-SE Engagement Subdomain

High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


Ongoing Engagemen

t V V V V V V V V V

Reaction to Challenge V V V V V V V V V

Note: No Optimal and High Risk indicators were created for composites at the subdomain

level or lower, with the exceptions of Experience of Parental Support and Experience of Teacher Support (discussed in Table 4) and are thus not presented in Tables 5 (above) and 6.

V-


11

Table 7: Associations between Subdomain- and Construct-Level RAPS-SE Beliefs about Self Composites and the Criterion Indices

Criterion Variables RAPS-TE (Engagement) RAPS-SE Engagement

RAPS-SE Beliefs about Self

Composite:

High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


Subdomain: Perceived Competence Promote

Competence V V V V V V Undermine

Competence V V V V V V Maximize

Competence (Summary Score)

V V V V V V Subdomain: Perceived Autonomy

Identified Self-Regulation

V V V V V V Intrinsic

Self-Regulation --- v v V V V

Subdomain: Perceived Relatedness Feelings about

Self V V V V V V Parental Emotional

Security V v V V V V Teacher Emotional

Security V V V V V V Peer Emotional

Security v V V V V V

Relatedness (Summary Score) V V V V V V

V-


12

MIDDLE SCHOOL: RAPS-TM AND RAPS-SM COMPOSITES AND INDICATORS

C. RAPS-TM In this section we provide information on the reliability and validity of the composite score and Optimal and High Risk indices developed from the RAPS-TM. As described in a previous section, the RAPS-TM was designed to tap student engagement in school. The validity of the composite Student Engagement measure and of the Optimal and High Risk indicators is assessed by examining the strength of the associations of these measures with the middle school and high school performance and commitment indicators described below. On the following pages, we first present a brief description of the sample of students used to assess the reliability and validity of the RAPS-TM composite and indices. Next is reliability information for the composite and a summary of the validity analyses. Readers who would like more detailed information, including a breakdown for males and females separately, are referred to Appendix B. 1. Characteristics of the Reliability Sample Teacher-report data were obtained from studies conducted in three middle schools within one urban school district. Teacher report data were matched to administrative record data obtained from the school district. All students with middle school teacher report data and administrative record data in middle school, high school, or both, were included in the resulting data set. The final number of subjects was 1553. Table 1: Demographic characteristics of the reliability sample

Age range 10 through 15 Grades 6th through 8th Ethnicity 57% African-American




65% eligible

2. Reliability of the RAPS-TM Student Engagement Composite The three items that comprise the RAPS-TM Student Engagement composite had an average interitem correlation of .69, and an alpha reliability of .87.

V-


13

3. Assessing the Validity of the RAPS-TM Student Engagement Composite and Indicators

As mentioned above, the validity of the composite RAPS-TM Student Engagement measure and Optimal and High Risk indicators is evaluated by examining the strength of the associations of these measures with middle school and high school performance and commitment indicators. The development of High Risk and Optimal indicators of student performance and commitment based on administrative record data from middle school (SPCI-M) and high school (SPCI-H) are described in a separate technical report.10 Each Optimal and High Risk indicator is based on student attendance records and percentile scores on standardized reading achievement tests. As with the RAPS-TM and RAPS-SM indices described in a previous section, students received a score of 1 on an Optimal or High Risk indicator if they met the threshold criteria for inclusion, and a score of 0 if they did not. The threshold criteria are presented below. Table 2: Outcome Indices

Indicator Threshold Criteria Middle School (SPCI-M) High Risk a. middle school attendance rate below 79% or


Optimal a. middle school attendance rate of 93% or higher and one or both of the following:

b. standardized reading achievement test percentile score of 70% or higher, or

c. standardized math achievement test percentile score of 65% or higher. High School (SPCI-H) High Risk a. high school attendance rate below 75% or


Optimal a. high school attendance rate of 89% or higher and one or both of the following:

b. standardized reading achievement test percentile score of 75% or higher, or

c. standardized math achievement test percentile score of 70% or higher.

Table 3 presents a summary of the strength of the associations between the RAPS-TM Engagement measures and the criterion indices with which they were expected to be associated. The strengths of the associations between the composite score, which is a continuous variable, and the criterion indices, which are in all cases dichotomous (i.e. High Risk v. Other, Optimal v. Other, and High Risk v. Optimal)

10 Bridges, L.J., & Connell, J.P. (1998). Student Performance and Commitment Index (SPAI), I: Calibration of Middle School and High School Stars and Flags. Unpublished IRRE technical report.

V-


14

are assessed using point-biserial correlations. The strengths of the associations between the RAPS-TM Engagement indices and the criterion indices, both of which are dichotomous variables, are assessed as phi coefficients.11 Table 3: Associations between the RAPS-SM Engagement Composite and Indices and Criterion Indices

Criterion Indices SPCI-M (Middle School) SPCI-H (High School)

RAPS-TM Engagement

Predictor


(Extreme Groups)


(Extreme Groups)

Composite (Continuous) V V V

V V V




V V V V V V

Note 1: In Table 3, High Risk criterion indices are predicted by High Risk

indices, Optimal criterion indices are predicted by Optimal indices, and High Risk v. Optimal criterion indices are predicted by High Risk v. Optimal indices.

Note 2: Validity coefficients represent the strength of the associations

between predictor variables and criterion variables:

V =correlations and phi coefficients of .10 - .50 (all significant at p<.0001)



V-


15

D. RAPS-SM In this section we provide information on the reliability and validity of the composite scores and Optimal and High Risk indices developed from the RAPS-SM. As described in a previous section, the RAPS-SM was designed to tap the supports and opportunities available to students in school. Our approach to assessing the validity of the RAPS-SM measures was dictated by our motivational model, in which students’ experience of support directly affects their beliefs about themselves, which in turn affects their engagement with school. Engagement with school, in turn, is the most direct influence on student performance and adjustment in school. Based on this theoretical ordering of the constructs within the model, the criterion measures used to assess validity of the various components of the RAPS-SM were as follows: 1. The validity of Student Engagement composites and Optimal and High Risk

indicators is assessed by examining the strength of the associations of these measures with middle school and high school performance and adjustment indicators (SPCI-M Optimal and High Risk and SPCI-H Optimal and High Risk) described on page 2. Also examined is the strength of the associations between the RAPS-SM (self-report) Engagement composites and the teacher-report of student engagement (RAPS-TM) indices.

2. The validity of student Beliefs about Self components and Optimal and High

Risk indicators is primarily evaluated with respect to the strength of the associations these components demonstrate with both the RAPS-TM (teacher-report) and the RAPS-SM Engagement High Risk and Optimal indicators. Further, valid Beliefs about Self components and indicators should continue to demonstrate significant associations with SPCI-M High Risk and Optimal indicators. Thus, these associations are examined as well.

3. Finally, the validity of composites and Optimal and High Risk indicators

reflecting students’ perceptions of Interpersonal Support is evaluated with respect to the associations these composites demonstrate with the RAPS-SM Beliefs about Self High Risk and Optimal indicators. As with the Beliefs About Self components, the associations between Interpersonal Support components and the other criterion indices (SPCI-M Optimal and High Risk, RAPS-TM Engagement Optimal and High Risk, and RAPS-SM Engagement Optimal and High Risk) are also examined.

On the following pages, we first present a brief description of the sample of students used to assess reliability and validity of the RAPS-SM components. Next is reliability information for each composite, followed by a summary of the validity analyses. Readers who would like more detailed information, including a breakdown for males and females separately, are referred to Appendix C.

V-


16

1. Characteristics of the Reliability Sample Student-report data were obtained from studies conducted in three middle schools within one urban school district and one middle school from an adjacent suburban district. Table 1 presents general characteristics of the reliability sample. Table 1:

Sample size: Total number of students in sample 2429 Number of students with self-report (RAPS-SM) data 2429 Number of students with middle school administrative records data

1679 (urban district only)

Number of students with high school administrative records data

681 (urban district only)

Number of students with teacher-report (RAPS-TM) data 1347 (urban district only) Demographic characteristics: Age range 10 through 15 Grades 6th through 8th Ethnicity 44% African-American

39% Euro-American 16% Hispanic 1% Other



Urban district: 58% eligible Suburban district: Unknown

2. Reliabilities of the RAPS-SM Composites Table 2 presents a summary of the alpha reliabilities of each of the composites generated from the RAPS-SM.

V-


17

Table 2: Alpha reliabilities of the RAPS-SM composites

COMPOSITE NUMBER OF ITEMS RELIABILITY (�)

Domain: Engagement 1. Ongoing Engagement 412 .68 2. Reaction to Challenge 6 .69

3. ENGAGEMENT (Summary Score) 10 .77

Domain: Beliefs About Self Perceived Competence

1. Promote Competence 8 .57 2. Undermine Competence 8 .76 3. Maximize Competence (Summary Score) 16 .79

Perceived Autonomy 4. Introjected Self-Regulation 3 .76 5. Identified Self-Regulation 3 .72 6. Intrinsic Self-Regulation 3 .78

Perceived Relatedness 7. Emotional Security with Self 3 .67 8. Satisfaction with Self 3 .71 9. Parental Emotional Security 3 .74 10. Teacher Emotional Security 4 .72 11. Peer Emotional Security 4 .73 12. Relatedness (Summary Score) 17 .82

13. BELIEFS ABOUT SELF (Summary Score) 36 .87

Domain: Experiences of Interpersonal Support 1. Parental Support 17 .86 2. Teacher Support 14 .82

3. INTERPERSONAL SUPPORT (Summary Score) 31 .88

12One of the items included in the five-item Engagement composite was not available in the data set with which reliability and validity information was developed. That item, epxf1 (“I often come to class unprepared”), is included based on results from a separate study involving an independent sample of diverse students in the same general age group. In that study, this item was found to relate strongly to attendance, one of the two components of the four SPCI indices (SPCI-M High Risk and Optimal, and SPCI-H High Risk and Optimal).

V-


18

3. Assessing the Validity of the RAPS-SM Composites and Indicators As discussed earlier, the validity of the various RAPS-SM measures is assessed by examining the extent to which each measure is associated with the criterion variables with which it is expected to be associated. The strengths of the associations between the RAPS-SM composite scores, which are continuous variables, and the criterion indices, which are in all cases dichotomous (i.e., High Risk v. Not High Risk, Optimal v. Not Optimal, High Risk v. Optimal) are assessed using point-biserial correlations. The strengths of the associations between the RAPS-SM indices and the criterion indices, both of which are dichotomous variables, are assessed as phi coefficients.13 Validity information is presented at two levels on the following pages: 1. Tables 3 through 5 present information on the associations between

measures reflecting the three major domains tapped by the RAPS-SM (Engagement, Beliefs about Self, and Experience of Interpersonal Support) and the primary criterion variables with which they are expected to be associated. Because of the importance of separate indices reflecting experiences of Parental Support and Teacher Support (as well as the overall Interpersonal Support), these two subdomains are included in the Interpersonal Support table.

2. Tables 6 and 7 present similar information separately for each of the remaining 14 composites from the five additional subdomains tapped by the RAPS-SM (Ongoing Engagement, Reaction to Challenge, Perceived Competence, Perceived Autonomy, Perceived Relatedness).

As noted earlier, more detailed information on the associations between RAPS-SM measures and outcome indices may be found in Appendix C.


V-


19

Table 3: Associations between the Domain-Level RAPS-SM Engagement Composite and Indices and the Criterion Indices

Criterion Variables SPCI Middle School SPCI High School RAPS-TM (Engagement)

RAPS-SM Engagemen

t Variable

High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


Composite (Continuous

) V V V v --- V* V V V

Indicators (High Risk v. Other,

Optimal v. Other, or


V V V --- --- --- V V V

*p < .05 Note 1: In Table 3 (above), Table 4, and Table 5, High Risk criterion indices are

predicted by High Risk indices, Optimal criterion indices are predicted by Optimal indices, and High Risk v. Optimal criterion indices are predicted by High Risk v. Optimal indices.

Note2: In all tables, validity coefficients represent the strength of the associations

between predictor variables and criterion variables: v =correlations and phi coefficients of less than .10 but significant at

p<.05 V =correlations and phi coefficients of .10 - .50 (all significant at

p<.0001, except where indicated)


V-


20

Table 4: Associations between the Domain-Level RAPS-SM Beliefs About Self Composite and Indices and the Criterion Indices

Criterion Indices RAPS-TM (Engagement) RAPS-SM Engagement

RAPS-SM Beliefs about Self Predictor

High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


Composite (Continuous) V V V V V V




V V V V V V

Table 5: Associations between the Experiences of Parental Support (Subdomain-Level), Teacher Support (Subdomain-Level), and Interpersonal Support (Domain-Level) Composites and Indices and the Criterion Indices

Criterion Variables: RAPS-SM Beliefs About Self RAPS-SM Experience of Interpersonal

Support Variable High Risk v.

Other Optimal v. Other High Risk v.

Optimal Experience of Parental Support Composite (Continuous) V V V Indicators (High Risk, Optimal, or High Risk v. Optimal) V V V Experience of Teacher Support Composite (Continuous) V V V Indicators (High Risk, Optimal, or High Risk v. Optimal) V V V Experience of Interpersonal Support Composite (Continuous) V

V V Indicators (High Risk, Optimal, or High Risk v. Optimal) V V V

V-


21

Table 6: Associations between the Subdomain-Level RAPS-SM Engagement Composites and the Criterion Indices

Criterion Indices SPCI Middle School SPCI High School RAPS-TM (Engagement)

RAPS-SM Engagement Subdomain

High Risk v. Other

Optimal v.

Other


High Risk v. Other

Optimal v.

Other


High Risk v. Other

Optimal v.

Other


Ongoing Engagement V V V V* --- V* V V V

Reaction to Challenge

v V V --- --- --- V V V

*p < .05 Note: No Optimal and High Risk indicators were created for composites at the subdomain

level or lower, with the exceptions of Experience of Parental Support and Experience of Teacher Support (discussed in Table 4) and are thus not presented in Tables 5 (above) and 6.

V-


22

Table 7: Associations between Subdomain- and Construct-Level RAPS-SM Beliefs about Self Composites and the Criterion Indices

Criterion Variables RAPS-TM (Engagement) RAPS-SM Engagement

RAPS-SM Beliefs About Self Composite:

High Risk v. Other

Optimal v. Other


High Risk v. Other

Optimal v. Other


Subdomain: Perceived Competence Promote

Competence V V V V V V Undermine

Competence V V V V V V Maximize

Competence (Summary Score)

V V V V V V Subdomain: Perceived Autonomy

Introjected Self-Regulation

--- --- --- V V V Identified

Self-Regulation V V V V V V

Intrinsic Self-Regulation

v --- --- V V V Subdomain: Perceived Relatedness

Satisfaction with Self

--- --- --- V V V Emotional Security

with Self --- v v V V V

Parental Emotional Security

v --- v V V V Teacher Emotional

Security V V V V V V Peer Emotional

Security v --- v V V V

Relatedness (Summary Score) V v V V V V

VI-


1

BIBLIOGRAPHY for COMPONENT 2 Student Supports and Opportunities in School:

Engagement, Beliefs about Self, and Experiences of Interpersonal Support

Ainsworth, M.D.S. (1989). Attachments beyond infancy. American Psychologist, 44(4), 709-716. Arhar, J.M., & Kromrey, J.D. (1993). Interdisciplinary teaming in the middle level school: Creating a sense of belonging for at-risk middle level students. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, Georgia. Battistich, V., Solomon, D., Kim, D., Watson, M., Schaps, E. (1996). Schools as communities, poverty levels of student populations , and students’ attitudes, motives, and performance. American Educational Journal. Battistich, V., Solomon, D., Watson, M., Schaps, E. (1994). Students and teachers in caring classroom and school communities. Paper presented at the annual meeting of American Educational Research Association. Boggiano, A., Main, D., & Katz, P. (1988). Children’s preference for challenge: The role of perceived competence and control. Journal of Personality and Social Psychology, 54, 134-141. Bridges, L.J., & Connell, J.P. (1998). Student Performance and Commitment Index (SPCI), I: Calibration of middle school and high school stars and flags. Unpublished IRRE technical report. Bridges, L.J., & Connell, J.P. (1998). Student Performance and Commitment Index (SPCI), II: Calibration of elementary school stars and flags. Unpublished IRRE technical report. Bryk, A.S., & Driscoll, M.E. (1988). The school as community: Theoretical foundations, contextual influences, and consequences for students and teachers. Madison: National Center on Effective Secondary Schools. University of Wisconsin. Bryk, A.S., & Thum, Y.M. (1989). The effects of high school organization on dropping out. New Brunswick, N.J.: Center for Policy Research in Education, Eagleton Institute of Politics, Rutgers University. Carnegie Council on Adolescent Development (1989). Turning points: Preparing American youth for the 21st century (Report of the Task force on Education of Young Adolescents). New York: Carnegie Council on Adolescent Development.

VI-


2

Connell, J.P. (1990). Context, self, and action: A motivational analysis of self-system processes across the life-span. In D. Cicchetti (Ed.), The self in transition: From infancy to childhood (pp. 61-97). Chicago: University of Chicago Press. Connell, J.P., Ryan, R.M. (1984). A developmental theory of motivation in the classroom. Teacher Education Quarterly, 11, 64-77. Connell, J.P., & Halpern-Felsher, B.L. (in press). How do neighborhoods affect educational outcomes in middle childhood and adolescence? Connell, J.P., & Wellborn, J.G. (1991). Competence, autonomy, and relatedness: A motivational analysis of self-system processes. In M.R. Gunnar & L.A. Sroufe (Eds.), Self processes in development: Minnesota Symposium on Child Psychology (Vol. 23, pp. 43-77). Chicago: University of Chicago Press. Connell, J.P., & Wellborn, J.G. (1994). Engagement versus disaffection: Motivated patterns of action in the academic domain. Rochester, NY: University of Rochester. Connell, J.P., Spencer, M.B., & Aber, J.L. (1994). Educational risk and resilience in African-American youth: Context, self, action, and outcomes in school. Child Development, 65, 493-506. Connell, J.P., Clifford, E., Crichlow, W., & Halpern-Felsher, B. (1993). Connell, J.P., Halpern-Felsher, B., Clifford, E., Crichlow, W., & Usinger, P. (1995). Hanging in there: Behavioral, psychological, and contextual factors affecting whether African-American adolescents stay in school. Journal of Adolescent Research, 10(1), 41-63. Deci, E.L. (1995). Why we do what we do: The dynamics of personal autonomy. New York: Putnam. Deci, E. L., & Ryan, R.M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press. Deci, E.L., Connell, J.P., & Ryan, R.M. (1985). A motivational analysis of self-determination in the classroom. In C. Ames & R. Ames (Eds.), Research on motivational analysis of self-determination and self-regulation in the classroom (pp. 13-52). San Diego: Academic Press. Deci, E.L., Vallerand, R., Pelletier, L, & Ryan, R. (1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26, 325-346.

VI-


3

Dweck, C.S. (1991). Self-theories and goals: Their role in motivation, personality, and development. In R.A. Dienstbier (Ed.), Nebraska Symposium on Motivation (1990). Lincoln: University of Nebraska Press. Dweck, C.S., & Leggett, E.L.(1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256-273. Eccles, J.S., & Midgley, C. (1988). Stage/environment fit: Developmentally appropriate classrooms for young adolescents. In R.E. Ames & C. Ames (Eds.), Research on motivation in education (Vol. 3, pp. 139-186). New York: Academic Press. Eccles, J.S., Buchanan, C.M., Flanagan, C., Fuligni, A., Midgley, C., & Yee, D. (1991). Control versus autonomy during early adolescence, Journal of Social Issues, 47(4), 53-68. Eccles, J.S., Wigfield, A., & Schiefele, U. (1996). Motivation. In N. Eisenberg (Ed.), Handbook of child psychology: Vol. IV: Social and personality development. Eckert, P. (1989). Jocks and burnouts: Social categories and identity in the high school. New York: Teachers College Press. Fine, M. (1991). Framing dropouts: Notes on the politics of an urban public high school. Albany: State University of New York Press. Finn, J.D. (1989). Withdrawing from school. Review of Educational Research, 59, 117-142. Finn, J.D. (1992, April). Participation among eighth-grade students at risk. Paper presented at a symposium titled “Students at Risk: Results from National Surveys” (chaired by M. C. Wang) at the annual meeting of the American Educational Research Association., San Francisco, CA. Finn, J.D. (1993). School engagement and students at risk. Washington, DC: National Center for Education Statistics. Finn, J.D., & Cox, D. (1992). Participation and withdrawal among fourth-grade pupils. American Educational Research Journal, 29, 141-162. Gamoran, A., & Nystrand, M. (1992). Taking students seriously. In F. Newmann (Ed.), Student engagement and achievement in American secondary schools (pp. 40-61). New York: Teachers College Press.

VI-


4

Goodenow, C. (1993). Classroom belonging among early adolescent students: Relationships to motivation and achievement. Journal of Early Adolescence, 13(1), 21-43. Gottfried, A. (1990). Academic intrinsic motivation in young elementary school children. Journal of Educational Psychology, 82, 525-538. Graham, S., & Barker, G.P. (1990). The downside of help: An attributional-developmental analysis of helping behavior as a low-ability cue. Journal of Educational Psychology, 82, 7-14. Grolnick, W.S., Ryan, R.M., & Deci, E.L. (1989). Parent styles associated with children’s self-regulation and competence in school. Journal of Educational Psychology, 81, 143-154. Hallinger, P., & Murphy, J.F. (1986). The social context of effective schools. American Journal of Education, 94, 328-355. Halpern-Felsher, B.L., Connell, J.P., Duncan, G.P., Aber, J.L., Usinger, P. & Owens, (1995). Harter, S. (1981). A model of mastery motivation in children: Individual differences and developmental change. In W.A. Collins (Eds.), The Minnesota Symposium on Child Psychology (Vol. 14, pp. 215-255). Hillsdale, NJ: Erlbaum. Harter, S., & Connell, J.P. (1984). A comparison of alternative models of the relationships between academic achievement and children’s perceptions of competence, control, and motivational orientation. In J. Nicholls (Ed.), The development of achievement-related conditions and behavior (pp. 219-250). Greenwich, CT: JAI Press. Haslinger, J., Kelly, P., & O’Lare, L. (Sept, 1996). Countering absenteeism, anonymity, and apathy. Educational Leadership, 47-49. Hom, A., & Battistich, V. (1995). Students’ sense of school community as a factor in reducing drug use and delinquency. Paper presented to the Annual Meeting of the American Educational Research Association (San Francisco, CA, April 18-22, 1995). Kim, D. and Others. (1995). Classroom practices that enhance students’ sense of community. Paper presented at the Annual Meeting of the American Educational Research Association (San Francisco, CA, April 18-22, 1995). Knapp, M.S., & Shields, P.M. (1991). Better schooling for the children of poverty: Alternatives to conventional wisdom. San Francisco: McCutchan.

VI-


5

Ladd, G. (1990). Having friends, keeping friends, making friends, and being liked by peers in the classroom: Predictors of children’s early school adjustment? Child Development, 61, 1081-1100. Lambourn, S.D., Brown, B.B., Mounts, N.S., & Steinberg, L. (1992). Putting school in perspective: The influence of family, peers, extracurricular participation, and part-time work on academic engagement. In F.M. Newmann (Ed.), Student engagement and achievement in American secondary schools (pp. 153-181). New York: Teachers College Press. Larson, R.W., & Richards, M.H. (1991). Boredom in the middle school years: Blaming schools versus blaming students. American Journal of Education, 4, 418-442. Lee, V.E., & Smith, J.B. (1993). Effects of school restructuring on the achievement and engagement of middle-grade students. Sociology of Education, 66, 164-187. Lee, V.E., and others (1995). Another look at high school restructuring. More evidence that it improves student achievement and more insight into why. Madison, WI: Center on Organization and Restructuring of Schools. Lovette, O.K., & Jacob, S. (1995). Why do so many high achieving high school students dislike school? NASSP Bulletin, 79, 70-75. MacIver, D., Stipek, D, & Daniels, D. (1991). Explaining within-semester changes in student effort in junior high school and senior high school courses. Journal of Educational Psychology, 83, 201-211. Marks, H.M. (1995). Student engagement in the classrooms of restructuring schools. Madison, WI: Center on Organization and Restructuring of Schools. McLaughlin, M.W., Talbert, J., Kahne, J., & Powell, J. (Nov, 1990). Constructing a personalized school environment. Phi Delta Kappan, 230-235. Midgley, C., Eccles, J.S., & Feldlaufer, H. (1991). Classroom environment and the transition to junior high school. In B.J. Fraser & H.J. Walberg (Eds.) Educational environments: Evaluation, antecedents, and consequences (p 113-139). Oxford: Pergamon Press. Midgley, C., & Feldlaufer, H. (1987). Students’ and teachers’ decision-making fit before and after the transition to junior high school. Journal of Early Adolescence, 7, 225-241.

VI-


6

Mounts, N.S., & Steinberg, L. (1995). An ecological analysis of peer influence on adolescent grade point average and drug use. Developmental Psychology, 31, 915-922. Murphy, J. (1991). Restructuring schools: Capturing and assessing the phenomena. New York: Teachers College Press. Newmann, F.M. (1989). Student engagement and high school reform. Educational Leadership, 46(5), 34-36. Newmann, F.M. (1991). Student engagement in academic work: Expanding the perspective on secondary school effectiveness. In J.R. Bliss, W.A. Firestone, & C.E. Richards (Eds.), Rethinking effective schools: Research and practice (pp. 58-75). Englewood-Cliffs, NJ: Prentice-Hall. Newmann, F.M. (1992). Student engagement and achievement in American secondary schools. New York: Teachers College Press. Newmann, F.M., Marks, H.M., & Gamoran, A. (1995). Authentic pedagogy: Standards that boost study performance. Issues in Restructuring Schools No. 8. Madison, WI: Center on Organization and Restructuring of Schools. Newmann, F.M., Secada, W.G., & Wehlage, G.G. (1995). A guide to authentic instruction and assessment: Vision, standards, and scoring. Madison, WI: Center on Organization and Restructuring of Schools. Nicholls, J.G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, 328-346. Nystrand, M., & Gamoran, A. (1991). Instructional discourse, student engagement, and literature achievement. Research in the Teaching of English, 25(3), 261-290. Patrick, B.C., Skinner, E.A., & Connell, J.P. (1993). What motivates children’s behavior and emotion? Joint effects of perceived control and autonomy in the academic domain. Journal of Personality and Social Psychology, 65(4), 781-791. Pierson, L.H., & Connell, J.P. (1992). Effect of grade retention on children's self-system processes, behavioral adjustment and academic performance. Journal of Educational Psychology, 84(3), 300-307. Pintrich, P.R., & Blumenfeld, P.C. (1985). Classroom experiences and children’s self-perceptions of ability, effort, and conduct. Journal of Educational Psychology, 77(6), 646-657.

VI-


7

Powell, A.G., Farrar, E., & Cohen, D.K. (1985). The shopping mall high school: Winners and losers in the educational marketplace. Boston: Houghton Mifflin. Ryan, R.M., & Connell, J.P. (1989). Perceived locus of causality and internalization: Examining reasons for acting in two domains. Journal of Personality and Social Psychology, 51, 749-761. Ryan, R.M., & Powelson, C.L. (1991). Autonomy and relatedness as fundamental to motivation in education. Journal of Experimental Education, 49-56. Ryan, R.M., & Stiller, J. (1991). The social contexts of internalization: Parent and teacher influences on autonomy, motivation, and learning. In P.R. Pintrich & M.L. Maehr (Eds.), Advances in motivation and achievement: Vol. 7, Goals and self-regulatory process, pp. 115-149. Greenwich, Conn.: JAI Press. Ryan, R.M., Connell, J.P., & Deci, E.L. (1985). A motivational analysis of self-determination and self-regulation in education. In C. Ames & R.E. Ames (Eds.), Research on motivation in education: Vol. 2. The classroom milieu (pp. 13-51). San Diego, CA: Academic Press. Schmitz, B., & Skinner, E.A. (1993). Perceived control, effort, and academic performance: Interindividual, intraindividual, and multivariate time-series analyses. Journal of Personality and Social Psychology, 64(6), 1010-1028. Schneider, E. (Sept, 1996). Giving students a voice in the classroom. Educational Leadership, 22-26. Schunk, D. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26, 207-231. Sedlak, M.W., Wheeler, C.W., Pullin, D.C., & Cusick, P.A. (1986). Selling our students short: Classroom bargains and academic reform in the American high school. New York: Teachers College Press. Skinner, E.A. (1991). Development and perceived control: A dynamic model of action in context. In M.R. Gunnar & L.A. Sroufe (Eds.), Self processes in development: Minnesota Symposium on Child Psychology (Vol. 23, pp. 167-216). Chicago: University of Chicago Press. Skinner, E.A. (1995). Perceived control, motivation, and coping. Thousand Oaks, CA: Sage. Skinner, E.A., & Belmont, M.J. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85(4), 571-581.

VI-


8

Skinner, E.A., Wellborn, J.G., & Connell, J.P. (1990). What it takes to do well in school and whether I’ve got it: The role of perceived control in children’s engagement and school achievement. Journal of Educational Psychology, 82, 22-32. Skinner, E.A., Zimmer-Gembeck, M., & Connell, J.P. (1996). Individual differences and the development of perceived control. Manuscript under review. Stipek, D. J. (1993). Motivation to learn: From theory to practice. Boston: Allyn & Bacon. Solomon, D., & Battistich, V. (1993). Students in caring school and classroom communities. Paper presented at the annual meeting of the American Psychological Association. Solomon, D., Watson, M., Battistich, V., Schaps, E., & Delucchi, K. (1992). Creating a caring community: Educational practices that promote children’s prosocial development. In F.K. Oser, A. Dick, & J.L. Patry (Eds.), Effective and responsible teaching: The new synthesis. (pp. 383-396). San Francisco: Jossey-Bass. Voelkl, K.E. (1995). School warmth, student participation, and achievement. Journal of Experimental Education, 63, 127-138. Wehlage, G., (1989). Dropping out: Can schools be expected to prevent it? In L. Weis, E. Farrar, & H. Petrie (Eds.), Dropouts from school (pp. 1-19). Albany: State University of New York Press. Wehlage, G.G., Rutter, R.A., Smith, G.A., Lesko, N., & Fernandez, R.R. (1989). Reducing the risk: Schools as communities of support. Philadelphia: Falmer. Weiner, B. (1990). History of motivational research in education. Journal of Educational Psychology, 82, 22-32. Weis, L. (1990). Working class without work: High school students in a decentralizing economy. New York: Routledge. Wellborn, J.G. (1991). Engaged and disaffected action: The conceptualization and measurement of motivation in the academic domain. Unpublished doctoral dissertation, University of Rochester, New York. Zachlod, M.G. (Sept, 1996). Room to grow. Educational Leadership, 50-53.

VI- 9


RELIABILITY AND VALIDITY: RAPS-TE

A summary of reliability and validity information for the RAPS-TE is presented in an earlier section of the manual. This appendix provides more detailed information. Following a description of the characteristics of the sample of elementary school students used to assess reliability and validity, Tables 1 and 2 provide the complete set of associations between the RAPS-TE composite and indicators and the outcomes against which they were validated. In addition to information for the sample as a whole, associations are presented for males and females separately. The reliability of the composite, presented earlier for the sample as a whole, is repeated here as well, along with the reliabilities for males and females separately.

Characteristics of the Reliability Sample Data sets used: Teacher-report data were obtained from studies conducted in six elementary schools in one urban school district. Students in grades 3 through 5 were included. A single year of data was available for three schools, two years for two schools, and four years for the remaining school. Teacher-report data were then matched to administrative record data obtained from the school district, covering the same academic years. All students with elementary school teacher report data as well as administrative record data in elementary school, middle school, or both, were included within the resulting data set. The final number of subjects was 2026. Characteristics of the resulting sample: Age and grade: Students for whom teacher-report data was available ranged in age from 7 to 11, and in grade from 3rd to 5th. The latest available year of data was used for each student with multiple elementary school assessments. Approximately 7% were age 7, 20% were age 8, 28% were age 9, 28% were age 10, and 17% were age 11. 31% of the students were in 3rd grade, 30% were in 4th grade, and 39% were in 5th grade. Ethnicity: Approximately 82% of the sample is African-American, 9% is Euro-American, and 9% is Hispanic. Gender: This sample is 51% male and 49% female. SES: 85% of students in this sample were eligible for free or reduced-price school lunches. This percentage is consistent with the percentage for all elementary school students within the school district.

VI- 10


Student Performance and Commitment in elementary school and middle school: Administrative Record outcome data: The development of High Risk and Optimal indicators of student performance and commitment based on administrative record data from elementary school (SPCI-E) and middle school (SPCI-M) are described elsewhere. To assess the validity of the teacher report Student Engagement summary measures, including RAPS-TE High Risk and RAPS-TE Optimal indices, these SPCI-E and SPCI-M High Risk and Optimal performance indicators were used as outcome variables. As described in the text of this manual for the RAPS-TE and the RAPS-SE, students received a score of 1 if they met the threshold criteria for inclusion, and a score of 0 if they did not. The threshold criteria are as follows: 1. High Risk SPCI-E classification:

a. elementary school attendance rate below 89% or b. standardized reading achievement test percentile score below 35%,

or both 2. High Risk SPCI-M classification:

a. middle school attendance rate below 79% or b. standardized reading achievement test percentile score below 25%,

or both 3. Optimal SPCI-E classification:

a. elementary school attendance rate of 97% or higher and one or both of the following:


c. standardized math achievement test percentile score of 80% or higher 4. Optimal SPCI-M classification:

a. middle school attendance rate of 93% or higher and one or both of the following:


c. standardized math achievement test percentile score of 65% or higher

VI- 11


Of the 2,026 students with teacher report data, 1,662 had elementary school administrative record data necessary for determination of SPCI-E High Risk and SPCI-E Optimal status. Of these students, 46% were above the threshold for High Risk classification, and 15% were above the Optimal threshold.14 Slightly more than a third (686) of the students in this sample had administrative record data available for middle school from which High Risk and Optimal status on the SPCI-M could be determined. The probability of being above the High Risk threshold in middle school for those students was 33%, while the probability of being above the Optimal threshold in middle school was 18%.

14 A small number of additional students (26, or less than 2% of the sample) met the criteria for classification in both High Risk and Optimal groups. This was possible only when the student had a standardized reading test score below the 35th percentile but a math test score at or above the 70th percentile and an attendance rate of 97% or higher. The same possibility existed for SPCI-M Optimal and High Risk classification, but only two students in this sample were found to have met both the High Risk and Optimal criteria for middle school.

VI- 12


Details of Reliability and Validity Analyses Table 1: Reliabilities and Associations of Engagement Composite with SPAI-E and SPAI-M Criterion Indices

Average Correlations With Criterion Indices (SPCI-E and SPCI-M) Interitem

Correlation and Reliability

SPCI-E Optimal v.

Other

SPCI-E High Risk v.

Other

SPCI-E Optimal v. High Risk

SPCI-M Optimal v.

Other

SPCI-M High Risk v.

Other

SPCI-M Optimal v. High Risk

All Students: rii = .59, D=.81

Males: rii = .55, D=.78

Females: rii = .62, D=.83

.26

.26

.27

-.31

-.33

-.29

.42

.43

.42

.19

.14+

.24

-.24

-.23

-.26

.35

.31

.39

Note: All correlations in Table 1 are significant at p<.0001 unless otherwise indicated. Correlations denoted with an + are significant at p<.01. Table 2: Associations of Engagement Indices with SPCI-E and SPCI-M Criterion Indices

RAPS-TE Engagement

Group

% of Students in SPCI-E Groups % of Students in SPCI-M Groups

High Risk v. Other High Risk Other High Risk Other

High Risk 64% 36% 45% 55% Other 36% 64% 26% 74%

Chi square (F2) / phi

F2(1) = 127.53, p<.001

phi = .28 F2

(1) = 26.61, p<.001 phi = .20

Optimal v Other: Optimal Other Optimal Other

Optimal 34% 66% 29% 71% Other 12% 88% 15% 85%


F2(1) = 103.23, p<.001

phi = .25 F2

(1) = 18.74, p<.001 phi = .16

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-TE indicator and on the criterion indicator.

High Risk Optimal High Risk Optimal High Risk 91% 9% 79% 21% Optimal 45% 55% 41% 59%


F2(1) = 172.99, p<.001

phi = .50 F2

(1) = 34.24, p<.001 phi = .38

VI- 13


RELIABILITY AND VALIDITY: RAPS-TM A summary of reliability and validity information for the RAPS-TM is presented in an earlier section of the manual. This appendix provides more detailed information. Following a description of the characteristics of the sample of middle school students used to assess reliability and validity, Tables 1 and 2 two tables will provide the complete set of associations between the RAPS-TM composite and indicators and the outcomes against which they were validated. In addition to information for the sample as a whole, associations are presented for males and females separately. The reliability of the composite, presented earlier for the sample as a whole, is repeated here as well, along with the reliabilities for males and females separately.

Characteristics of the Reliability Sample Data sets used Teacher-report data were obtained from studies conducted in three middle schools within one urban school district. Multiple years of data collection were available for each of these schools. Teacher-report data were matched to administrative record data obtained from the school district, covering the same academic years. All students with teacher report data as well as administrative record data in middle school, high school, or both, were included within the resulting data set. The final number of subjects was 1553. Characteristics of the resulting sample Age and grade: Students for whom teacher-report data were available ranged in age from 10 to 15, and in grade from 6th to 8th. The latest available year of data was used for each student with multiple middle school assessments. Approximately 3% were age 10, 14% were age 11, 30% were age 12, 36% were age 13, 16% were age 14, and 1% were age 15. 10% of the students were in 6th grade, 50% were in 7th grade, and 40% were in 8th grade. Ethnicity: Approximately 57% of the sample is African-American, 23% is Euro-American, and 20% is Hispanic. Gender: The sample is 50% male and 50% female. SES: 65% of students in this sample were eligible for free or reduced-price school lunches. This percentage is consistent with that for the school district as a whole. Student performance and commitment in middle school and high school: Administrative record outcome data: The development of High Risk and Optimal indicators of student performance and commitment based on administrative record

VI- 14


data from middle school (SPCI-M) and high school (SPCI-H) are described elsewhere. To assess the validity of the teacher report Student Engagement summary measures, including RAPS-TM High Risk and RAPS-TM Optimal indices, these SPCI-M and SPCI-H High Risk and Optimal performance indicators were used as outcome variables. As with the RAPS-TM and RAPS-SM indices described in this manual, students received a score of 1 if they met the threshold criteria for inclusion, and a score of 0 if they did not. The threshold criteria are as follows: 1. High Risk SPCI-M classification:

a. middle school attendance rate below 79% or b. standardized reading achievement test percentile score below 25%,

or both 2. High Risk SPCI-H classification:

a. high school attendance rate below 75% or b. standardized reading achievement test percentile score below 25%,

or both 3. Optimal SPCI-M classification:

c. middle school attendance rate of 93% or higher and one or both of the following:

d. standardized reading achievement test percentile score of 70% or higher

e. standardized math achievement test score of 65% or higher 4. Optimal SPCI-H classification:

f. high school attendance rate of 89% or higher and one or both of the following:

g. standardized reading achievement test percentile score of 75% or higher

h. standardized math achievement test percentile score of 70% or higher Of the 1553 students with middle school data, 1534 had all administrative records data necessary for determination of both High Risk and Optimal classification. Of these students, 33% (509 students) were above the threshold for High Risk classification in middle school, while 14% (218 students) were above the Optimal threshold.15 Slightly less than half (745) of the students in this sample had administrative record data available for high school from which High Risk SPCI-H status could be 15 A very few additional students (9, or approximately 1% of the sample) actually met the criteria for both SPCI-M High Risk and Optimal classification. This was possible only if they were below the 25th percentile for reading, but at the same time were at or above the 70th percentile for math achievement and had an attendance rate of 93% or higher. Although the same possibility existed for SPCI-H classification, no students were found who met both the Optimal and High Risk classification criteria.

VI- 15


determined. Due to differences in the information necessary for High Risk versus Optimal classification and the fact that not all students in the sample had high school testing data available, SPCI-H Optimal status could be determined for a smaller number of students (697). Of the student whose SPCI-H High Risk status could be determined, 46% were above the High Risk threshold. Among students whose Optimal status could be determined, 13% were above the Optimal threshold.

VI- 16


Details of Reliability and Validity Analyses Table 1: Reliabilities and Associations of Engagement Composite with SPCI-M and SPCI-H Criterion Indices

Average Correlations with Criterion Indices (SPCI-M and SPCI-H) Interitem

Correlation and Reliability

SPCI-M Optimal v.

Other

SPCI-M High Risk v.

Other


SPCI-H Optimal v.

Other

SPCI-H High Risk v.

Other

SPCI-H Optimal v. High Risk

All Students: rii = .69, D=.87

Males: rii = .69, D=.87

Females: rii = .66, D=.85

.29

.33

.28

-.33

-.32

-.34

.51

.52

.52

.28

.33

.26

-.35

-.33

-.36

.44

.47

.42

Note: All correlations in Table 1 are significant at p<.0001 Table 2: Associations of Engagement Indices with SPCI-M and SPCI-H Criterion Indices

RAPS-TM Engagement

Group

% of Students in SPCI-M Groups % of Students in SPCI-H Groups

High Risk v. Other High Risk Other High Risk Other

High Risk 54% 46% 67% 33% Other 24% 76% 37% 63%


F2(1) = 138.84, p<.001

phi = .30 F2

(1) = 56.44, p<.001 phi = .28

Optimal v Other: Optimal Other Optimal Other

Optimal 36% 64% 32% 68% Other 10% 90% 9% 91%


F2(1) = 123.11, p<.001

phi = .28 F2

(1) = 50.10, p<.001 phi = .27

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-TM indicator and on the criterion indicator.

High Risk Optimal High Risk Optimal High Risk 73% 27% 95% 5% Optimal 8% 92% 40% 60%


F2(1) = 193.07, p<.001

phi = .68 F2

(1) = 84.87, p<.001 phi = .61

VI- 17


RELIABILITY AND VALIDITY: RAPS-SE A summary of reliability and validity information for the RAPS-SE is presented in a previous section of the manual. This appendix provides more detailed information. Following a description of the characteristics of the sample of elementary school students used to assess reliability and validity, a series of tables will present the complete set of associations between each RAPS-SE composite and indicator and the outcomes against which each was validated. In addition to information for the sample as a whole, associations are presented for males and females separately. The reliabilities, presented earlier for the sample as a whole, are repeated here as well, along with the reliabilities for males and females separately.

Characteristics of the Reliability Sample Data sets used: Student report data were obtained from studies conducted in six elementary schools within one urban school district. Multiple years of data were available for four of these schools. All students who had participated in at least one assessment were included in the resulting data set. The number of subjects included was 1846. Characteristics of the resulting sample: Age and grade: Students in this sample ranged in age from 7 to 13, and in grade from 3rd to 5th. Approximately 3% were age 7, 21% were 8, 30% were age 9, 30% were age 10, 14% were age 11, and 2% were age 12. A single student was age 13. 31% of the students in the sample were in 3rd grade, 30% of students were in 4th grade, and 39% were in 5th grade at the time of their final assessments. Ethnicity: Approximately 81% of the sample was African-American, 9% was Euro-American, and 10% was Hispanic. Gender: In this sample, 51% of the students were male, 49% were female. SES: 85% of the students in this sample were eligible for free or reduced-price school lunches. High Risk and Optimal SPCI Status in elementary school and middle school: The criteria for inclusion in the SPCI-E High Risk and Optimal groups is provided in Appendix B. Of the 1846 students with elementary school self-report data, a total of 1600 had data necessary for the determination of both Optimal and High Risk classification (i.e., a complete set of attendance and math and reading test scores). Of these, 44% (712 students) were classified as being within the High Risk group and 16% (253 students) were above the threshold for inclusion in the Optimal

VI- 18


group.16 A much smaller subsample of 660 students had the necessary data for determination of SPCI-M High Risk and Optimal classifications. Within this smaller group of students, 31% (202) of the students were above the threshold for High Risk classification in middle school, while 19% (126) were above the Optimal threshold, resulting in a total of 50% of this subsample (328 students) being in one of the two extreme groups. High Risk and Optimal RAPS-TE Classification: A subsample of 1750 students with elementary school self-report data had teacher report data from which the RAPS-TE Optimal and High Risk classifications were derived. In this subsample, 40% were above the threshold for High Risk classification in elementary school, while 22% were above the Optimal threshold, resulting in a total of 1092 students (62%) being in one of the two extreme groups. High Risk and Optimal RAPS-SE Engagement Classification: All of the 1846 students with elementary school self-report data had the data necessary for creation of the RAPS-SE Engagement indices. Among these students, 35% were classified within the RAPS-SE Engagement High Risk group, while 27% were classified within the RAPS-SE Engagement Optimal group. Thus, approximately 62% of the sample (1053 students) were within one of the two extreme groups. High Risk and Optimal RAPS-SE Beliefs about Self Classification. Of the 1846 students with the data necessary for creation of the RAPS-SE Beliefs about Self indices, the probability of being within the High Risk group was 32%, while the probability of being within the Optimal group was 15%, resulting in a total of 48% of this elementary school sample (878 students) being in one of the two extreme groups. High Risk and Optimal RAPS-SE Experiences of Parental Support, Teacher Support, and Interpersonal Support Classification. All 1846 students with self-report data had the data necessary for creation of High Risk and Optimal indices for Parental Support, Teacher Support, and Interpersonal Support. Among these students: 1. the probability of RAPS-SE Interpersonal Support High Risk classification

was 30%, while the probability of being above the Optimal threshold was 23%, resulting in a total of 53% of the sample (980 students) being in one of the two extreme groups;

2. approximately 43% of students were classified as being in the RAPS-SE Parental Support High Risk group, while only 9% were within the Optimal

16 A small number of additional students (29, or 2% of the sample) met the criteria for classification in both High Risk and Optimal groups. This was possible if the student had a standardized reading test score below the 35th percentile both a math test score at or above the 70th percentile and an attendance rate of 97% or higher. The sample possibility existed for SPCI-M Optimal and High Risk classification, but only two students in this sample were found to have met both the High Risk and Optimal criteria for middle school.

VI- 19


group, indicating that 52% of the sample 961 students) were within one of the two extreme groups; and

3. Approximately 22% of students were classified within the RAPS-SE Teacher Support High Risk group, and 34% were within the Teacher Support Optimal group, and a total of 56% of the sample (1034 students) being within one of the two extreme groups.

Details of Reliability and Validity Analyses The tables in this section are presented in the following order: I. Table 1: Engagement Domain Composites A. Subdomain: Ongoing Engagement B. Subdomain: Reaction to Challenge C. Domain Summary Score: Engagement II. Table 2: Beliefs About Self Domain, 1: Perceived Competence Subdomain

Composites A. Construct: Promote Competence B. Construct: Undermine Competence C. Subdomain Summary Score: Maximize Competence III. Table 3: Beliefs About Self Domain, 2: Perceived Autonomy Subdomain

Composites A. Construct: Identified Self-Regulation B. Construct: Intrinsic Self-Regulation IV. Table 4: Beliefs About Self Domain, 3: Perceived Relatedness Subdomain

Composites A. Construct: Feelings about Self B. Construct: Parental Emotional Security C. Construct: Teacher Emotional Security D. Construct: Peer Emotional Security E. Subdomain Summary Score: Perceived Relatedness V. Table 5: Beliefs About Self Domain, 4: Domain Summary Score VI. Table 6: Experiences of Interpersonal Support Domain Composites A. Subdomain: Perceived Parental Support B. Subdomain: Perceived Teacher Support C. Domain Summary Score: Perceived Interpersonal Support

VI- 20


VII. Table 7: Engagement Domain Indicators A. High Risk versus Other B. Optimal versus Other C. High Risk versus Optimal VIII. Table 8: Beliefs About Self Domain Indicators A. High Risk versus Other B. Optimal versus Other C. High Risk versus Optimal IX. Table 9: Experiences of Interpersonal Support Domain Indicators A. Parental Support Subdomain Indicators: 1. High Risk versus Other 2. Optimal versus Other 3. High Risk versus Optimal B. Teacher Support Subdomain Indicators: 1. High Risk versus Other 2. Optimal versus Other 3. High Risk versus Optimal C. Interpersonal Support Domain Indicators: 1. High Risk versus Other 2. Optimal versus Other 3. High Risk versus Optimal

VI- 21


TABLE 1: Engagement Domain Composites

Average Interitem Correlations with Criterion Indices (SPCI-E, SPCI-M, and RAPS-TE) Correlation and

Reliability SPCI-E

Optimal v. Other

SPCI-E High Risk v.

Other


SPCI-M Optimal v.

Other

SPCI-M High Risk v.

Other


RAPS-TE Optimal v.

Other

RAPS-TE High Risk v.

Other

RAPS-TE Optimal v. High Risk

A. Subdomain: Ongoing Engagement

All Students: rii=.25, ? = .66

Males: rii=.25, ? = .67

Females: rii=.23, ? = .63

.13

.12*

.13

-.17

-.15

-.18

.20

.18

.22

.14 *

.15 +

.11,

-.15

-.26

-.07

.24

.32

.17 ,

.22

.16

.26

-.31

-.27

-.33

.36

.28

.41

B. Subdomain: Reaction to Challenge


Males: rii=.26, ? = .72


.12

.12*

.11+

-.18

-.19

-.17

.20

.22

.18

.14*

.14+

.13,

-.17

-.30

-.07 ns

.26

.35

.19,

.15

.12*

.15

-.17

-.18

-.13*

.23

.20

.21

C. Domain Summary Score: Engagement

All Students: rii=.15, �=.71

Males: rii=.20, �=.77

Females: rii=.21, �=.79

.14

.14

.14

-.20

-.20

-.20

.23

.23

.23

.16

.17+

.14+

-.19

-.32

-.09 ns

.29

.39

.22+

.21

.16

.22

-.27

-.26

-.25

.33

.27

.34

Note: Correlations on this table and all subsequent tables are significant at p<.0001 unless otherwise indicated. Correlations denoted with an * are significant at p<.001. Correlations denoted with + are significant at p<.01. Correlations denoted with , are significant at p<.05. Correlations denoted with ns are not significant.

VI- 22


TABLE 2: Beliefs About Self Domain, 1: Perceived Competence Subdomain Composites

Average Interitem Correlations with Criterion Indices (SPAI-E, RAPS-TE, and RAPS-SE Engagement) Correlation and

Reliability SPCI-E

Optimal v. Other

SPCI-E High Risk v.

Other


RAPS-TE Optimal v.

Other


Other


RAPS-SE Engagement Optimal v.

Other

RAPS-SE Engagement High Risk

v. Other

RAPS-SE Engagement Optimal v. High Risk

A. Construct: Promote Competence


Males: rii=.15, ? = .52


.16

.18

.15

-.14

-.14

-.14

.23

.24

.22

.20

.13*

.24

-.23

-.16

-.30

.30

.20

.38

.27

.26

.28

-.33

-.33

-.31

.41

.41

.41

B. Construct: Undermine Competence:


Males: rii=.23, ? = .75


-.19

-.20

-19

.22

.21

.23

-.30

-.30

-.30

-.21

-.17

-.23

.29

.24

.34

-.34

-.26

-.40

-.40

-.38

-.41

.57

.55

.59

-.63

-.61

-.65

C. Subdomain Summary Score: Maximize Competence


Males: rii=.16, �=.75


.21

.24

.19

-.22

-.20

-.23

.31

.33

.30

.24

.17

.28

-.31

-.23

-.37

.38

.27

.45

.41

.38

.42

-.54

-.53

-.54

.62

.61

.63

VI- 23


TABLE 3: Beliefs About Self Domain, 2: Perceived Autonomy Subdomain Composites


Reliability SPCI-E

Optimal v. Other

SPCI-E High Risk v.

Other


RAPS-TE Optimal v.

Other


Other



Other


v. Other


A. Construct: Identified Self-Regulation


Males: rii=.38, ? = .75


.01 ns

.05 ns

-.02 ns

.01 ns

.02 ns

.01 ns

.00 ns

.04 ns

-.04 ns

.10

.05 ns

.13*

-.10

-.05 ns

-.14

.15

.07 ns

.20

.22

.20

.23

-.30

-.29

-.30

.35

.33

.36

B. Construct: Intrinsic Self-Regulation


Males: rii=.48, �=.78


.00 ns

.02 ns

-.02 ns

.07+

.06 ns

.08 ,

-.03 ns

.00 ns

-.06 ns

.07+

.03 ns

.09+

-.03 ns

.01 ns

-.07 ns

.08 ,

.02 ns

.12+

.18

.15

.19

-.17

-.12*

-.21

.25

.21

.28

VI- 24


TABLE 4: Beliefs About Self Domain, 3: Perceived Relatedness Subdomain Composites


Reliability SPCI-E

Optimal v. Other

SPCI-E High Risk v.

Other


RAPS-TE Optimal v.

Other


Other



Other


v. Other


A. Construct: Feelings about Self


Males: rii=.28, ? = .66


.08+

.06 ns

.10+

-.15

-.15

-.15

.15

.13+

.17*

.12

.11*

.11+

-.16

-.19

-.14

.20

.19

.19

.31

.29

.32

-.38

-.41

-.36

.49

.49

.48

B. Construct: Parental Emotional Security


Males: rii=.53, �=.77


.04 ns

.03 ns

.06 ns

-.06 ,

-.07 ns

-.05 ns

.07 ,

.06 ns

.08 ns

.09*

.04

.12*

-.10

-.09+

-.11+

.14

.08 ns

.17

.26

.25

.28

-.37

-.38

-.35

.43

.42

.43

VI- 25


TABLE 4, continued: Beliefs About Self Domain, 3: Perceived Relatedness Subdomain Composites


Reliability SPCI-E

Optimal v. Other

SPCI-E High Risk v.

Other


RAPS-TE Optimal v.

Other


Other



Other


v. Other


C. Construct: Teacher Emotional Security


Males: rii=.49, ? = .74


.08 +

.09 ,

.08 ,

-.14

-.15

-.13*

.15

.17*

.14+

.14

.10 +

.14

-.18

-.17

-.15

.22

.17

.21

.33

.33

.31

-.36

-.39

-.32

.50

.52

.46

D. Construct: Peer Emotional Security


Males: rii=.61, ? = .80


.08*

.05 ns

.12*

-.06,

-.04 ns

-.08,

.12*

.07 ns

.16*

.12

.11+

.13*

-.08+

-.10+

-.06 ns

.15

.15*

.15*

.25

.26

.24

-.28

-.30

-.26

.38

.41

.35

E. Subdomain Summary Score: Perceived Relatedness


Males: rii=.26, �=.83


.11

.08,

.14

-.15

-.13*

-.16

.18

.14

.20

.15

.11*

.18

-.18

-.18

-.17

.24

.19

.26

.40

.38

.40

-.46

-.47

-.45

.59

.59

.57

VI- 26


TABLE 5: Beliefs About Self Domain, 4: Domain Summary Score


Reliability SPCI-E

Optimal v. Other

SPCI-E High Risk v.

Other


RAPS-TE Optimal v.

Other


Other



Other


v. Other


All Students: rii=.19, ? =.87

Males: rii=.18, ? =.87

Females: rii=.19, ? =.87

.14

.13*

.16

-.19

-.18

-.19

.23

.21

.25

.17

.12

.20

-.22

-.20

-.23

.27

.20

.31

.42

.40

.43

-.51

-.51

-.52

.63

.63

.63

VI- 27


Table 6: Experience of Interpersonal Support Domain Composites.

Average Correlations with Criterion Indices (SPAI-E, RAPS-TE, RAPS-SE Engagement, and RAPS-SE Self) Interitem

Correlation & Reliability

SPCI-E Optimal v. Other

SPCI-E High Risk v. Other

SPCI-E High Risk

v. Optimal

RAPS-TE Optimal v. Other

RAPS-TE High Risk v. Other

RAPS-TE High Risk v. Optimal

RAPS-SE Engage.

Optimal v. Other

RAPS-SE Engage. High Risk v. Other

RAPS-SE Engage. High Risk v. Optimal

RAPS-SE Self, Optimal v. Other

RAPS-SE Self,

High Risk v. Other

RAPS-SE Self,


A. Subdomain: Perceived Parental Support All Students:

rii=.21, ? = .82 Males:

rii=.20, ? = .81 Females:

rii=.23, ? = .83

.13

.10+

.16

-.21

-.18

-.23

.24

.20

.28

.17

.11*

.19

-.21

-.16

-.23

.27

.18

.31

.36

.34

.36

-.47

-.48

-.44

.56

.57

.54

.34

.33

.34

-.53

-.55

-.51

.65

.65

.65

B. Subdomain: Perceived Teacher Support All Students:

rii=.29, ? = .80 Males:


rii=.30, ? = .81

.16

.19

.13+

-.21

-.28

-.13+

.26

.34

.18+

.21

.21

.19

-.27

-.30

-.20

.34

.34

.29

.51

.53

.49

-.64

-.65

-.61

.79

.81

.76

.39

.39

.39

-.66

-.70

-.63

.81

.85

.79

C. Domain Summary Score: Perceived Interpersonal Support All Students:

rii=.20, ? = .87 Males:


rii=.20, ? = .87

.14

.13*

.15

-.20

-.21

-.19

.24

.24

.24

.20

.15

.20

-.24

-.22

-.23

.31

.25

.31

.45

.43

.45

-.56

-.57

-.54

.67

.67

.66

.40

.38

.42

-.60

-.62

-.58

.74

.75

.74

VI- 28


Table 7: Engagement Domain Indicators

RAPS-SE Engagement

Group

% of Students in SPCI-E Groups % of Students in SPCI-M Groups % of Students in RAPS-TE Groups

High Risk v. Other:

High Risk Other High Risk Other High Risk Other High Risk 59% 41% 43% 57% 56% 44%

Other 40% 60% 24% 76% 32% 68% Chi square (�2) /

phi �

2(1)=49.25, p<.001

phi = .17 �

2(1)=26.11, p<.001

phi = .20 �

2(1)=96.28, p<.001

phi = .23

Optimal v. Other:

Optimal Other Optimal Other Optimal Other Optimal 25% 75% 25% 75% 34% 66% Other 15% 85% 17% 83% 18% 82%

Chi square (�2) / phi

�2(1)=24.44, p<.001

phi = .12 �

2(1)=4.74, P<.05

phi = .08 �

2(1)=49.96, p<.001

phi = .17

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SE Engagement indicator and the outcome indicator.

High Risk Optimal High Risk Optimal High Risk Optimal High Risk 86% 14% 79% 21% 83% 17% Optimal 59% 41% 45% 55% 45% 55%


�2(1) =56.97, p<.001

phi = .30 �

2(1)=24.27, p<.001

phi = .34 �

2(1)=111.04, p<.001

phi = .40

VI- 29


Table 8: Beliefs About Self Domain Indicators RAPS-SE Beliefs About Self Group % of Students in SPCI-E Groups % of Students in RAPS-TE Groups % of Students in RAPS-SE

Engagement Groups

High Risk v. Other:


Other 42% 58% 34% 66% 20% 80% Chi square (�2) /

phi �

2(1)=33.04, p<.001

phi = .14 �

2(1)=53.47, p<.001

phi = .17 �

2(1)=401.82, p<.001

phi = .47

Optimal v. Other:



�2(1)=17.75, p<.001

phi = .10 �

2(1)=38.25, p<.001

phi = .15 �

2(1)=216.92, p<.001

phi = .34

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SE Beliefs about Self indicator and the criterion indicator.



�2(1) =46.48, p<.001

phi = .31 �

2(1)=75.83, p<.001

phi = .37 �

2(1)=422.21, p<.001

phi = .82

VI- 30


Table 9A: Experiences of Interpersonal Support Domain: Parental Support Subdomain Indicators

RAPS-SE Parental Support Group

% of Students in SPCI-E Groups

% of Students in RAPS-TE Groups

% of Students in RAPS-SE Engagement Groups

% of Students in RAPS-SE Beliefs About Self Groups

High Risk v. Other:

High Risk Other High Risk Other High Risk Other High Risk Other High Risk 57% 43% 50% 50% 58% 42% 57% 43%

Other 39% 61% 33% 67% 18% 82% 14% 86% Chi square (�2) /

phi �

2(1)=47.13, p<.001

phi = .17 �

2(1)=52.37, p<.001

phi = .17 �

2(1)=321.64, p<.001

phi = .42 �

2(1)=382.09, p<.001

phi = .45

Optimal v. Other:

Optimal Other Optimal Other Optimal Other Optimal Other Optimal 23% 77% 30% 70% 44% 56% 30% 70% Other 13% 87% 17% 83% 15% 85% 5% 95%


�2(1)=28.91, p<.001

phi = .13 �

2(1)=39.93, p<.001

phi = .15 �

2(1)=196.60, p<.001

phi = .33 �

2(1)=201.94, p<.001

phi = .33

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SE Parental Support indicator and the outcome indicator.

High Risk Optimal High Risk Optimal High Risk Optimal High Risk Optimal High Risk 84% 16% 77% 23% 84% 16% 94% 6% Optimal 62% 38% 51% 49% 25% 75% 25% 75%


�2(1)=53.07, p<.001

phi = .26 �

2(1)=68.03, p<.001

phi = .27 �

2(1)=359.03, p<.001

phi = .60 �

2(1)=400.73, p<.001

phi = .72

VI- 31


Table 9B: Experiences of Interpersonal Support Domain: Teacher Support Subdomain Indicators

RAPS-SE Teacher Support Group





High Risk v. Other:


Other 43% 57% 36% 64% 25% 75% 21% 79% Chi square (�2) /

phi �

2(1)=25.17, p<.001

phi = .12 �

2(1)=52.10, p<.001

phi = .17 �

2(1)=328.75, p<.001

phi = .42 �

2(1)=366.10, p<.001

phi = .44

Optimal v. Other:



�2(1)=27.58, p<.001

phi = .13 �

2(1)=40.51, p<.001

phi = .15 �

2(1)=289.48, p<.001

phi = .40 �

2(1)=228.94, p<.001

phi = .35

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SE Teacher Support indicator and the criterion indicator.



�2(1)=36.00, p<.001

phi = .26 �

2(1)=73.12, p<.001

phi = .34 �

2(1)=433.20, p<.001

phi = .79 �

2(1)=364.22, p<.001

phi = .81

VI- 32


Table 9C: Experiences of Interpersonal Support Domain Indicators

RAPS-SE Interpersonal Support Group





High Risk v. Other:


Other 42% 58% 34% 66% 20% 80% 15% 85% Chi square (�2) /

phi �

2(1)=32.24, p<.001

phi = .14 �

2(1)=72.67, p<.001

phi = .20 �

2(1)=451.79 p<.001

phi = .49 �

2(1)=556.28, p<.001

phi = .55

Optimal v. Other:



�2(1)=15.77, p<.001

phi = .10 �

2(1)=35.94, p<.001

phi = .14 �

2(1)=263.35, p<.001

phi = .38 �

2(1)=285.91, p<.001

phi = .39

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SE Interpersonal Support indicator and the outcome indicator.



�2(1)=45.53, p<.001

phi = .30 �

2(1)=100.01, p<.001

phi = .41 �

2(1)=492.98, p<.001

phi = .84 �

2(1)=529.47, p<.001

phi = .95

VI- 33


RELIABILITY AND VALIDITY: RAPS-SM A summary of reliability and validity information for the RAPS-SM is presented in Section V of the manual. This appendix provides more detailed information. Following a description of the characteristics of the sample of middle school students used to assess reliability and validity, a series of tables presents the complete set of associations between each RAPS-SM composite and indicator and the outcomes against which each was validated. In addition to information for the sample as a whole, associations are presented for males and females separately. The reliabilities, presented earlier for the sample as a whole, are repeated here as well, along with the reliabilities for males and females separately.

Characteristics of the Reliability Sample Data sets used Student-report data were obtained from studies conducted in three middle schools within one urban school district and one middle school from an adjacent suburban district. Multiple years of data collection were available for each of these schools. All students who had participated in at least one assessment were included in the resulting data set. The number of subjects included was 2429. Characteristics of the resulting sample Age and grade: Students in this sample ranged in age from 10 to 15, and in grade from 6th to 8th. Because the latest available year of data was used for each student with multiple assessments, there were more older students than younger students in the final data set. Approximately 3% were age 10, 16% were 11, 28% were age 12, 36% were age 13, 16% were age 14, and 1% were age 15. 15% of the students in the sample were in 6th grade, 38% of students were in 7th grade, and 47% were in 8th grade at the time of their final assessments. Ethnicity: Approximately 44% of the sample were African-American, 39% were Euro-American, and 16% were Hispanic.. Gender: Approximately 49% of the students were male, 51% were female. SES: Information on SES was not available for the suburban students. Of the students enrolled in the three urban schools, 58% were eligible for free or reduced-price school lunches. High Risk and Optimal SPCI Status in middle school and high school: Of the 2429 students with middle school self-report data, 1661 also had administrative records data from which SPCI-M Optimal and High Risk classifications were

VI- 34


determined. Because administrative records data were unavailable for the suburban school district, SPCI-M indices were only constructed for urban district children. Within this group of students, 30% (499 students) were above the threshold for High Risk classification in middle school, while 16% (269 students) were above the Optimal threshold. Thus, approximately 46% of this urban subsample, or 768 students, were in one of the two extreme groups (High Risk or Optimal) on this indicator. Slightly more than a fourth (269) of the students in this sample had administrative record data available for high school from which High Risk and Optimal status on the SPCI-H could be determined. Once again, all of these students were from the three urban schools. The probability of being above the High Risk threshold in high school for those urban district students was 48%, while the probability of being above the Optimal threshold in high school was 17%. A total of 440 students, or 65% of the sample, were in either the High Risk or the Optimal extreme group. High Risk and Optimal RAPS-TM Classification: A subsample of 1954 students with middle school self-report data had teacher report data from which the RAPS-TM Optimal and High Risk classifications were derived. In this subsample, including only urban district students, 30% were above the threshold for High Risk classification in middle school, while 19% were above the Optimal threshold. In other words, 49% of the sample (661 students) were in one of the two extreme groups on the RAPS-TM. High Risk and Optimal RAPS-SM Engagement Classification: Of the students with any middle school self-report data available, 2377 had the data necessary for creation of the RAPS-SM Engagement indices. Among these students, the probability of being above the RAPS-SM Engagement High Risk threshold was 31%, while the probability of being above the RAPS-SM Engagement Optimal threshold was 15%. A total of 1093 students (46% of the sample) were in an extreme group on this indicator. High Risk and Optimal RAPS-SM Beliefs About Self Classification: Of the 2377 students with the data necessary for creation of the RAPS-SM Beliefs About Self indices, the probability of being above the RAPS-SM Beliefs About Self High Risk threshold was 40%, while the probability of being above the RAPS-SM Beliefs About Self Optimal threshold was 7%, resulting in a total of 47% of the sample (1123 students) being in one of the two extreme groups. High Risk and Optimal RAPS-SM Experiences of Parental Support, Teacher Support, and Interpersonal Support Classifications: Parental Support items were not included in some years of RAPS-SM administration used to create this reliability sample. There was a total of 2362 students with the data necessary for creation of the RAPS-SM Teacher Support indices, 1848 with the parental support data necessary for the creation of the Parental Support indices and 1836 with the

VI- 35


necessary parental and teacher support data to create the RAPS-SM Interpersonal Support composite. Among the students with the necessary data: 1. the probability of being above the RAPS-SM Interpersonal Support High Risk

threshold was 24% and the probability of being above the Optimal threshold was 20%, resulting in a total of 44% of the sample (837 students) being in one of the two extreme groups;

2. the probability of being above the RAPS-SM Parental Support High Risk threshold was 33% and the probability of being above the Optimal threshold was 20%, resulting in a total of 53% of the sample (975 students) being in an extreme group; and

3. the probability of being above the Teacher Support High Risk threshold was 39% while the probability of being above the Teacher Support Optimal threshold was 16%, resulting in a total of 55% of the sample (1291 students) being in one of the two extreme groups.

Details of Reliability and Validity Analyses The tables in this section are presented in the following order: I. Table 1: Engagement Domain Composites A. Subdomain: Ongoing Engagement B. Subdomain: Reaction to Challenge C. Domain Summary Score: Engagement II. Table 2: Beliefs about Self Domain, 1: Perceived Competence Subdomain

Composites A. Construct: Promote Competence B. Construct: Undermine Competence C. Subdomain Summary Score: Maximize Competence III. Table 3: Beliefs about Self Domain, 2: Perceived Autonomy Subdomain

Composites A. Construct: Introjected Self-Regulation B. Construct: Identified Self-Regulation C. Construct: Intrinsic Self-Regulation IV. Table 4: Beliefs about Self Domain, 3: Perceived Relatedness Subdomain

Composites A. Construct: Satisfaction with Self B. Construct: Emotional Security with Self C. Construct: Parental Emotional Security D. Construct: Teacher Emotional Security E. Construct: Peer Emotional Security F. Subdomain Summary Score: Perceived Relatedness

VI- 36


V. Table 5: Beliefs about Self Domain, 4: Domain Summary Score VI. Table 6: Experiences of Interpersonal Support Domain Composites A. Subdomain: Perceived Parental Support B. Subdomain: Perceived Teacher Support C. Domain Summary Score: Perceived Interpersonal Support VII. Table 7: Engagement Domain Indicators A. High Risk versus Other B. Optimal versus Other C. High Risk versus Optimal VIII. Table 8: Beliefs about Self Domain Indicators A. High Risk versus Other B. Optimal versus Other C. High Risk versus Optimal IX. Table 9: Experiences of Interpersonal Support Domain Indicators A. Parental Support Subdomain Indicators: 1. High Risk versus Other 2. Optimal versus Other 3. High Risk versus Optimal B. Teacher Support Subdomain Indicators: 1. High Risk versus Other 2. Optimal versus Other 3. High Risk versus Optimal C. Interpersonal Support Domain Indicators: 1. High Risk versus Other 2. Optimal versus Other 3. High Risk versus Optimal

VI- 37


TABLE 1: Engagement Domain Composites

Average Interitem Correlations with Criterion Indices (SPCI-M, SPCI-H, and RAPS-TM) Correlation and

Reliability SPCI-M

Optimal v. Other

SPCI-M High Risk v.

Other


SPCI-H Optimal v.

Other

SPCI-H High Risk v.

Other

SPCI-H Optimal v. High Risk

RAPS-TM Optimal v.

Other

RAPS-TM High Risk v.

Other

RAPS-TM Optimal v. High Risk

A. Subdomain: Ongoing Engagement


Males: rii=.31, ? = .64


.15

.19

.11+

-.17

-.15

-.17

.26

.28

.24

.07 ns

.09 ns

.05 ns

-.12*

-.11,

-.11,

.13+

.15,

.10 ns

.22

.23

.20

-.28

-.29

-.20

.41

.38

.36

B. Subdomain: Reaction to Challenge


Males: rii=.26, ? = .68


.12

.12*

.13*

-.07 +

-.04 ns

-.09 +

.18

.15 +

.21

.04 ns

.10 ns

.00 ns

-.04 ns

-.02 ns

-.03 ns

.06 ns

.12 ns

-.01 ns

.16

.18

.11+

-.24

-.22

-.19

.32

.31

.28

C. Domain Summary Score: Engagement


Males: rii=.26, �=.77


.16

.18

.14

-.14

-.11+

-.15

.25

.25

.25

.06 ns

.11,

.03 ns

-.09,

-.08 ns

-.08 ns

.10 ,

.16 ,

.05 ns

.22

.24

.18

-.30

-.30

-.23

.42

.40

.36

Note: Correlations on this table and all subsequent tables are significant at p<.0001 unless otherwise indicated. Correlations denoted with an * are significant at p<.001. Correlations denoted with + are significant at p<.01. Correlations denoted with , are significant at p<.05. Correlations denoted with ns are not significant.

VI- 38


TABLE 2: Beliefs about Self Domain, 1: Perceived Competence Subdomain Composites

Average Interitem Correlations with Criterion Indices (SPCI-M, RAPS-TM, and RAPS-SM Engagement) Correlation and

Reliability SPCI-M

Optimal v. Other

SPCI-M High Risk v.

Other


RAPS-TM Optimal v.

Other


Other


RAPS-SM Engagement Optimal v.

Other

RAPS-SM Engagement High Risk

v. Other

RAPS-SM Engagement Optimal v. High Risk

A. Construct: Promote Competence


Males: rii=.13, ? = .55


.18

.19

.19

-.21

-.17

-.24

.34

.31

.39

.17

.16*

.14+

-.27

-.25

-.22

.36

.29

.35

.27

.30

.21

-.43

-.42

-.35

.52

.52

.50

B. Construct: Undermine Competence:


Males: rii=.26, ? = .74


-.22

-.26

-19

.24

.25

.23

-.41

-.46

-.37

-.20

-.16+

-.21

.17

.12,

.16+

-.33

-.24*

-.36

-.30

-.29

-.30

.39

.38

.32

-.55

-.49

-.56

C. Subdomain Summary Score: Maximize Competence


Males: rii=.26, �=.77


.25

.29

.22

-.27

-.26

-.28

.46

.48

.45

.23

.22

.20

-.26

-.21

-.23

.42

.34

.42

.33

.36

.31

-.47

-.47

-.40

.61

.59

.61

VI- 39


TABLE 3: Beliefs about Self Domain, 2: Perceived Autonomy Subdomain Composites

Average Interitem Correlations with Criterion Indices (SPAI-M, RAPS-TM, and RAPS-SM Engagement) Correlation and

Reliability SPCI-M

Optimal v. Other

SPCI-M High Risk v.

Other


RAPS-TM Optimal v.

Other


Other



Other


v. Other


A. Construct: Introjected Self-Regulation


Males: rii=.45, ? = .71


-.04 ns

-.10+

.00 ns

.07+

.13*

.03 ns

-.11+

-.18*

-.02 ns

.04 ns

-.05 ns

.11+

-.04 ns

-.03 ns

-.07 ns

.08 ns

-.04 ns

.17+

.12

.14

.09+

-.13

-.09,

-.11*

.22

.22

.20*

B. Construct: Identified Self-Regulation


Males: rii=.44, ? = .70


.01 ns

.00 ns

.03 ns

-.01 ns

.01 ns

-.01 ns

.02 ns

.00 ns

.04 ns

.12

.11+

.09,

-.18

-.14*

-.17

.24

.20*

.24

.33

.32

.32

-.47

-.46

-.46

.63

.59

.66

C. Construct: Intrinsic Self-Regulation


Males: rii=.49, �=.74


-.09*

-.13*

-.06 ns

.10

.10+

.11+

-.18

-.20

-.16+

.01 ns

.06 ns

-.02 ns

-.06 ns

-.11+

-.01ns

.06 ns

.12 ,

-.01 ns

.22

.22

.19

-.21

-.20

-.22

.39

.37

.39

VI- 40


TABLE 4: Beliefs about Self Domain, 3: Perceived Relatedness Subdomain Composites


Reliability SPCI-M

Optimal v. Other

SPCI-M High Risk v.

Other


RAPS-TM Optimal v.

Other


Other



Other


v. Other


A. Construct: Satisfaction with Self


Males: rii=.33, ? = .59


.08+

.13*

.03 ns

-.12

-.14

-.11+

.18

.23

.13 ,

.03 ns

.05 ns

.04 ns

-.01 ns

-.01 ns

-.04 ns

.03 ns

.06 ns

.07 ns

.11

.15

.13

-.11

-.17

-.10+

.20

.28

.23

B. Construct: Emotional Security with Self


Males: rii=.42, ? = .69


.06 ns

.09 ns

.03 ns

-.04 ns

-.09 ,

-.01 ns

.09 ,

.15 +

.02 ns

.02 ns

.08 ns

-.02 ns

-.09 +

-.10 ,

-.07 ns

.09 ,

.14 ,

.05 ns

.22

.23

.22

-.25

-.33

-.21

.40

.44

.41

C. Construct: Parental Emotional Security


Males: rii=.49, �=.74


.04 ns

.07 ,

.01 ns

-.04 ns

-.10+

.01ns

.07 ,

.13+

.00 ns

.05 ns

.12+

.00 ns

-.08+

-.08 ns

-.08 ns

.11+

.17 +

.07 ns

.26

.24

.27

-.30

-.37

-.24

.47

.45

.48

VI- 41


TABLE 4, continued: Beliefs about Self Domain, 3: Perceived Relatedness Subdomain Composites


Reliability SPCI-M

Optimal v. Other

SPCI-M High Risk v.

Other


RAPS-TM Optimal v.

Other


Other



Other


v. Other


D. Construct: Teacher Emotional Security


Males: rii=.39, ? = .72


.09*

.13*

.06 ns

-.03 ns

.02 ns

-.06 ns

.12*

.13,

.12,

.15

.20

.07 ns

-.22

-.22

-.16

.30

.34

.21+

.29

.33

.29

-.33

-.36

-.28

.52

.57

.52

E. Construct: Peer Emotional Security


Males: rii=.40, ? = .72


.08+

.09,

.07 ns

-.09*

-.09,

-.08,

.06 ns

.11 ns

.01 ns

.04 ns

.08 ns

-.00 ns

-.06 ns

-.05 ns

-.07 ns

.08,

.12+

.07 ns

.14

.14*

.17

-.17

-.20

-.14

.28

.27

.31

F. Subdomain Summary Score: Perceived Relatedness


Males: rii=.21, �=.82


.07+

.08,

.05 ns

.02 ns

-.04 ns

.01 ns

.08,

.11,

.04 ns

.08+

.16

-.01 ns

-.17

-.17

-.11+

.20

.27

.10 ns

.26

.27

.28

-.33

-.39

-.28

.48

.50

.49

VI- 42


TABLE 5: Beliefs about Self Domain, 4: Domain Summary Score


Reliability SPCI-M

Optimal v. Other

SPCI-M High Risk v.

Other


RAPS-TM Optimal v.

Other


Other



Other


v. Other


All Students: rii=.12, ? =.87

Males: rii=.12, ? =.87

Females: rii=.12, ? =.87

.12

.13*

.12*

-.12

-.12+

-.11+

.22

.21

.22

.16

.19

.12+

-.22

-.19

-.20

.32

.30

.29

.37

.40

.40

-.50

-.55

-.49

.67

.69

.73

VI- 43


Table 6: Experience of Interpersonal Support Domain Composites.

Average Correlations with Criterion Indices (SPCI-M, RAPS-TM, RAPS-SM Engagement, and RAPS-SM Self) Interitem

Correlation & Reliability

SPCI-M Optimal v. Other

SPCI-M High Risk v. Other

SPCI-M High Risk

v. Optimal

RAPS-TM

Optimal v. Other

RAPS-TM High Risk v. Other

RAPS-TM High Risk v. Optimal

RAPS-SM Engage.

Optimal v. Other

RAPS-SM

Engage. High Risk v. Other

RAPS-SM

Engage. High Risk v. Optimal

RAPS-SM Self, Optimal v. Other

RAPS-SM Self, High Risk v. Other

RAPS-SM Self,


A. Subdomain: Perceived Parental Support All Students:

rii=.27, ? = .86 Males:


rii=.29, ? = .87

.08*

.12*

.05 ns

-.12

-.13*

-.10 +

.18

.21

.13 +

.11

.13*

.08 ,

-.14

-.14*

-.12 +

.22

.22

.19 +

.31

.29

.31

-.36

-.39

-.31

.56

.53

.56

.29

.30

.29

-.47

-.51

-.43

.59

.58

.60

B. Subdomain: Perceived Teacher Support All Students:

rii=.25, ? = .82 Males:


rii=.25, ? = .82

.08 +

.14

.03 ns

-.05 ,

-.01 ns

-.08 ,

.13*

.16 +

.10 ns

.15

.19

.09 ,

-.20

-.20

-.14*

.30

.32

.21*

.33

.36

.35

-.38

-.41

-.37

.59

.61

.63

.29

.35

.34

-.41

-.42

-.39

.55

.60

.65

C. Domain Summary Score: Perceived Interpersonal Support All Students:

rii=.19, ? = .88 Males:


rii=.19, ? = .88

.09*

.15

.04 ns

-.10

-.08 ,

-.11 +

.18

.21

.14 +

.15

.19

.10 ,

-.20

-.20

-.15

.31

.31

.24

.39

.39

.39

-.45

-.48

-.40

.69

.67

.69

.37

.39

.37

-.53

-.55

-.49

.69

.68

.71

VI- 44


Table 7: Engagement Domain Indicators

RAPS-SM Engagement

Group

% of Students in SPCI-M Groups % of Students in SPCI-H Groups % of Students in RAPS-TM Groups

High Risk v. Other:


Other 27% 73% 41% 59% 22% 78% Chi square (�2) /

phi �

2(1)=24.31, p<.001

phi = .12 �

2(1)=3.28, ns phi = .07

�2(1)=76.34, p<.001

phi = .24

Optimal v. Other:



�2(1)=28.73, p<.001

phi = .13 �

2(1)=0.03, ns phi = .00

�2(1)=44.08, p<.001

phi = .18

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SM Engagement indicator and the criterion indicator.



�2(1) =41.42, p<.001

phi = .34 �

2(1)=0.93, ns phi = .07

�2(1)=102.42, p<.001

phi = .58

VI- 45


Table 8: Beliefs about Self Domain Indicators RAPS-SM Beliefs about Self Group % of Students in SPCI-M Groups % of Students in RAPS-TM Groups % of Students in RAPS-SM

Engagement Groups

High Risk v. Other:


Other 26% 74% 22% 78% 14% 86% Chi square (�2) /

phi �

2(1)=21.82, p<.001

phi = .12 �

2(1)=54.71, p<.001

phi = .21 �

2(1)=490.10, p<.001

phi = .45

Optimal v. Other:



�2(1)=19.65, p<.001

phi = .11 �

2(1)=20.00, p<.001

phi = .12 �

2(1)=237.13, p<.001

phi = .32

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SM Beliefs about Self indicator and the criterion indicator.



�2(1) =39.38, p<.001

phi = .34 �

2(1)=45.89, p<.001

phi = .38 �

2(1)=494.06, p<.001

phi = .86

VI- 46


Table 9A: Experiences of Interpersonal Support Domain: Parental Support Subdomain Indicators

RAPS-SM Parental Support Group

% of Students in SPCI-M Groups

% of Students in RAPS-TM Groups

% of Students in RAPS-SM Engagement Groups

% of Students in RAPS-SM Beliefs About Self

Groups

High Risk v. Other:


Other 27% 73% 25% 75% 19% 81% 22% 78% Chi square (�2) /

phi �

2(1)=14.98, p<.001

phi = .10 �

2(1)=19.80, p<.001

phi = .12 �

2(1)=201.82, p<.001

phi = .33 �

2(1)=345.46, p<.001

phi = .43

Optimal v. Other:



�2(1)=5.47, p<.05

phi = .06 �

2(1)=2.65, ns phi = .05

�2(1)=147.34, p<.001

phi = .28 �

2(1)=217.52, p<.001

phi = .34

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SM Parental Support indicator and the criterion indicator.



�2(1)=17.97, p<.001

phi = .21 �

2(1)=16.90, p<.001

phi = .22 �

2(1)=288.38, p<.001

phi = .77 �

2(1)=347.27, p<.001

phi = .80

VI- 47


Table 9B: Experiences of Interpersonal Support Domain: Teacher Support Subdomain Indicators

RAPS-SM Teacher Support Group





Groups

High Risk v. Other:


Other 29% 71% 21% 79% 18% 82% 26% 74% Chi square (�2) /

phi �

2(1)=1.51, ns phi = .03

�2(1)=55.53, p<.001

phi = .21 �

2(1)=287.60, p<.001

phi = .35 �

2(1)=309.78, p<.001

phi = .36

Optimal v. Other:



�2(1)=0.67, ns phi = .02

�2(1)=10.55, p<.01

phi = .09 �

2(1)=235.67, p<.001

phi = .32 �

2(1)=222.42, p<.001

phi = .31

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SM Teacher Support indicator and the criterion indicator.



�2(1)=6.07, p<.05

phi = .13 �

2(1)=43.97, p<.001

phi = .34 �

2(1)=390.00, p<.001

phi = .75 �

2(1)=419.36, p<.001

phi = .77

VI- 48


Table 9C: Experiences of Interpersonal Support Domain Indicators

RAPS-SM Interpersonal Support Group





Groups

High Risk v. Other:


Other 28% 72% 25% 75% 19% 81% 25% 75% Chi square (�2) /

phi �

2(1)=13.69, p<.001

phi = .09 �

2(1)=32.14, p<.001

phi = .16 �

2(1)=269.22, p<.001

phi = .38 �

2(1)=332.67, p<.001

phi = .43

Optimal v. Other:



�2(1)=6.64, p<.01

phi = .06 �

2(1)=12.29, p<.001

phi = .10 �

2(1)=266.27, p<.001

phi = .38 �

2(1)=296.88, p<.001

phi = .40

High Risk v. Optimal: Analyses include only students who were in one of the two extreme groups (High Risk or Optimal) on both the RAPS-SM Interpersonal Support indicator and the criterion indicator.



�2(1)=21.93, p<.001

phi = .26 �

2(1)=45.79, p<.001

phi = .40 �

2(1)=349.86, p<.001

phi = .88 �

2(1)=360.89, p<.001

phi = .89

VI- 49


i The full citations for all references may be found in Appendix A: Bibliography. ii Arhar & Kromrey, 1993; Bryk & Driscoll, 1988; Connell, Spencer, Aber, 1994; Connell &

Wellborn, 1991; Goodenow, 1993; Hallinger & Murphy, 1986; Halpern-Felsher, Connell, Duncan, Aber, Usinger, & Owens, 1995; Kim, and others, 1995; Lamborn, Brown, Mounts, & Steinberg, 1992; Mounts & Steinberg, 1995.

iii Connell & Wellborn, 1991, 1994; Skinner, 1991, 1995. iv Connell & Wellborn, 1991, 1994; Skinner, 1995; Skinner & Wellborn, 1994. v Grannis, 1991; Hendrick, MacMillan, Balow, & Hough, 1990; Natriello, McDill, & Pallas, 1990 vi Connell, Clifford, Crichlow, & Halpern-Felsher, 1993; Connell, Spencer, & Aber, 1994;

Schmitz & Skinner, 1993; Skinner, Wellborn, & Connell, 1990; Skinner, Zimmer-Gembeck, & Connell, 1996

vii Bryk & Thum, 1989; Connell, Clifford, Crichlow, & Halpern-Felsher, 1993; Connell, Spencer,

& Aber, 1994; Finn, 1993; Finn & Cox, 1992; Finn & Voelkl, 1993; Gamoran & Nystrand, 1992; Marks, 1995; Newmann, Marks, & Gamoran, 1995; Newmann, Secada, & Wehlage, 1995; Nystrand & Gamoran, 1991; Skinner & Belmont, 1993

viii Finn, 1993; Connell, Aber, & Spencer, 1994 ix Finn, 1989; Lee, et al., 1995 x Deci & Ryan, 1985; Connell, 1990; Connell & Wellborn, 1991; Newmann, 1992; Skinner,

1995 xi Connell & Wellborn, 1991; Patrick, Skinner, & Connell, 1993; Skinner, 1991; Skinner,

Wellborn, & Connell , 1990; Skinner, Zimmer-Gembeck, & Connell, 1996 xii Boggiano, Main, & Katz, 1988; Gottfried, 1990; Harter and Connell, 1984; MacIver, Stipek, &

Daniels, 1991; Skinner, 1995; Skinner, Zimmer-Gembeck, & Connell, 1996 xiii Deci & Ryan, 1985; Deci, 1995; Connell & Wellborn, 1991 xiv Connell & Wellborn, 1991; Deci & Ryan, 1985; Dweck & Leggett, 1988; Dweck, 1991;

Nicholls, 1984; Patrick, Skinner, & Connell, 1993; Ryan & Connell, 1989; Schunk, 1991 xv Ainsworth, 1989; Connell & Wellborn, 1991; Deci, Connell, & Ryan, 1985; Weiner, 1990 xvi Battistich, and others, 1996; Hom & Battistich, 1995; Ladd, 1990; Lee, and others, 1995 xvii Arhar & Kromrey, 1993; Battistich, and others, 1996; Goodenow, 1993; Skinner & Belmont,

1993; Voelkl, 1995 xviii Finn, 1989, 1993; Wehlage, 1989; Wehlage, Rutter, Smith, Lesko, & Fernandez, 1989 xix Connell & Wellborn, 1990; Harter & Connell, 1984; Grolnick & Ryan, 1989; Patrick, Skinner,

& Connell, 1993; Pierson & Connell, 1993; Ryan & Powelson, 1991; Ryan & Stiller, 1991; Skinner, Wellborn, & Connell, 1990

VI- 50


xx Pintrich & Blumenfeld, 1985; Skinner, Wellborn, & Connell, 1990 xxi Connell & Ryan, 1984; Deci & Ryan, 1985; Skinner, 1995 xxii Ryan, Connell, & Deci, 1985; Grolnick, Ryan, & Deci, 1989 xxiii Deci, Vallerand, Pelletier, & Ryan, 1991 xxiv Battistich, and others, 1996; Kim, and others, 1995; Skinner & Belmont, 1993 xxv Bryk & Thum, 1989; Haslinger, Kelly, & O’Lare, 1996; Knapp & Shields, 1991; McLaughlin,

Talbert, Kahne, & Powell, 1990; Schneider, 1996; Sedlak and others, 1986; Wehlage and others, 1989; Zachlod, 1996

xxvi Arhar & Kromrey, 1993; Bryk & Driscoll, 1988; Goodenow, 1993; Hallinger & Murphy, 1986;

Kim, and others, 1995 xxvii Battistich, Solomon, Watson, & Schaps, 1994; Solomon, Watson, Battistich, Schaps, &

Delucchi, 1992 xxviii Connell, Spencer, Aber, 1994; Halpern-Felsher, Connell, Duncan, Aber, Usinger, &

Owens, 1995; Lamborn, Brown, Mounts, & Steinberg, 1992; Mounts & Steinberg, 1995 xxix Carnegie Council on Adolescent Development, 1989; Connell & Halpern-Felsher, in press;

Eccles & Midgley, 1988; Eccles, and others, 1991; Eccles, Wigfield, & Schiefele, 1996; Midgley, Eccles, & Feldlaufer, 1991; Midgley & Feldlaufer, 1987

xxx Schmitz & Skinner, 1993; Skinner & Belmont, 1993; Skinner, Wellborn, & Connell, 1990;

Skinner, Zimmer-Gelbeck, & Connell, 1996 xxxi Fine, 1991; Finn, 1989; Powell, and others, 1985; Sedlak and others, 1986; Weis, 1990

Documents

RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS)...x The Research Assessment Package for Schools (RAPS), that includes strategies and instruments for measuring four major components