A2TeaMS Board Page 2011 · and teachers’ instructional knowledge and abilities. In 2010–2011, three A2TeaMS staff worked with 73 teachers across 36 middle and high school campuses

MEMORANDUM October 19, 2011 TO: Board Members FROM: Terry B. Grier, Ed.D. Superintendent of Schools SUBJECT: The Academy of Accomplished Teaching in Math and Science (A2TeaMS).

Program Evaluation Report CONTACT: Carla Stevens, (713) 556-6700

Attached is the 2010 2011 evaluation report on The Academy of Accomplished Teaching in Math and Science (A2TeaMS) program. The program was created to provide ongoing professional development in the areas of content knowledge, research-based pedagogy, and leadership in math and science instructional domains. This type of professional development was implemented by pairing secondary math and science teachers with instructional coaches with math and science teaching expertise. The 2010–2011 academic year was the third and final year of program. The purpose of the current report was to assess the effectiveness of the A2TeaMS program by determining the extent that the program met its goals pertaining to enhancing student achievement and teachers’ instructional knowledge and abilities. In 2010–2011, three A2TeaMS staff worked with 73 teachers across 36 middle and high school campuses. The teacher/coach ratio was approximately one A2TeaMS staff to 24 teachers. On the 2010–2011 Stanford 10 math and science subtests, students who had math/science A2TeaMS teachers who earned 41 or more training hours outperformed their peers who had math/science teachers who participated in less than 41 training hours. Results concerning A2TeaMS teachers' attitudes and beliefs about their instructional knowledge and abilities at the beginning and end of the program indicated that by the end of the program, a greater number of teachers reported higher levels of ability and knowledge in various instructional areas. A2TeaMS teachers reported the greatest growth in their ability to implement the 5E/7E Model of Instruction, in their knowledge of how math-science connections apply to their teaching, and in their ability to apply math-science connections in their teaching to promote student engagement.

__TBG

Attachment

cc: Superintendent’s Cabinet

Nancy Gregory

RESEARCH

The Academy of Accomplished Teaching in Math and Science

(A2TeaMS)

2010–2011

Department of Research and Accountability Houston Independent School District

E d u c a t i o n a l P r o g r a m R e p o r t

2011 Board of Education

Paula M. Harris PRESIDENT Manuel Rodríguez Jr. FIRST VICE PRESIDENT Anna Eastman SECOND VICE PRESIDENT Carol Mims Galloway SECRETARY Michael L. Lunceford ASSISTANT SECRETARY Lawrence Marshall Greg Meyers Harvin C. Moore Juliet K. Stipeche Terry B. Grier, Ed.D. SUPERINTENDENT OF SCHOOLS Carla Stevens ASSISTANT SUPERINTENDENT DEPARTMENT OF RESEARCH AND ACCOUNTABILITY Danya M. Corkin RESEARCH SPECIALIST Kathryn S. Thibodeaux APPLICATION SPECIALIST Venita Holmes, Dr.P.H. RESEARCH MANAGER

Houston Independent School District Hattie Mae White Educational Support Center 4400 West 18th Street Houston, Texas 77092-8501 Website: www.houstonisd.org It is the policy of the Houston Independent School District not to discriminate on the basis of age, color, handicap or disability, ancestry, national origin, marital status, race, religion, sex, veteran status, or political affiliation in its educational or employment programs and activities.

A2TEAMS 2010–2011

1

EXECUTIVE SUMMARY

A2TEAMS EVALUATION 2010–2011

Program Description

Houston Independent School District (HISD) emphasizes teacher training as a means of raising student achievement scores and building the professional skills and talent of teachers within the district. Therefore, the Academy of Accomplished Teaching in Math and Science (A2TeaMS) was created to provide ongoing professional development in the areas of content knowledge, research-based pedagogy, and leadership in math and science instructional domains. This type of professional development was implemented by pairing secondary math and science teachers with instructional coaches with math and science teaching expertise. In order to strengthen the academic programs at participating schools, coaches observed, evaluated, and provided helpful feedback on the daily instructional practices of A2TeaMS teachers. Furthermore, coaches facilitated formal professional development training meetings and workshops. A2TeaMS initially was intended to be a three-year program for all participating teachers. However, the 2010–2011 academic year was the third and final year of program implementation, thus teachers in the second cohort were only exposed to two years of the program. Throughout the program’s existence, A2TeaMS maintained five program goals, which were:

Create a learning organization that provides a coordinated, systematic, district-wide opportunity for consistent on-going mathematics and science professional development on a large scale;

Increase mathematics and science content and pedagogy knowledge of participating teachers as evidenced by rigorous, inquiry-based lessons aligned with the district curriculum, including natural mathematics-science connections;

Increase student achievement in mathematics and science with a focus on student college readiness;

Ensure the written curriculum is taught in the classroom; and Increase administrators’ knowledge of best practices and ability to assess and support best

practices in mathematics and science. Purpose of the Evaluation

The purpose of the evaluation was to assess the effectiveness of the A2TeaMS program by determining the extent that the program met its goals pertaining to enhancing student achievement and teachers’ instructional knowledge and abilities. Therefore, the following research questions were addressed:

1. How was the A2TeaMS program structured in 2010–2011? 2. What were the demographic and professional characteristics of A2TeaMS teachers? 3. Did performance differences exist between the students of A2TeaMS teachers with varying levels

of participation in the program? 4. Did A2TeaMS teachers' attitudes and beliefs regarding instructional knowledge and abilities

change by the end of the program? 5. To what extent did A2TeaMS teachers’ use of math-science connections in their classrooms

influence students’ situational interest for math and science?

HISD RESEARCH AND ACCOUNTABILITY

2

Key Findings 1. How was the A2TeaMS program structured in 2010–2011?

In 2010–2011, three A2TeaMS staff worked with 73 teachers across 36 middle and high school campuses. The teacher/coach ratio was approximately one A2TeaMS staff to 24 teachers.

2. What were the demographic and professional characteristics of A2TeaMS teachers?

Of the 100 A2TeaMS, 60 percent were female, majority were either African American or Asian, and belonged to the second cohort. There were slightly more math teachers (51%) than science teachers (46%) who participated in the program, and most taught at a high school. About half of the teachers were relatively new to the district (<5 years) and 37 percent had less than five years of teaching experience.

3. Did performance differences exist between the students of A2TeaMS teachers with varying levels of participation in the program?

Statistically significant differences in performance on the 2010–2011 Stanford 10 were found

between students who had math teachers that earned more than 41 training hours compared to their peers who had math teachers who participated in less than 41 training hours. The average performance advantage of students’ who had math teachers with greater than 41 training hours ranged between three and nine NCEs.

In regards to the Stanford science results, students who had science teachers who earned 77-102 training hours significantly outperformed the group of students who had science teachers who earned less than 41 training hours by three NCEs, and the group of students who had science teachers who earned 41 to 76 training hours by six NCEs.

A greater percentage of students among teachers who earned more than 40 hours passed the mathematics and science TAKS exam and reached commended levels when compared to students who had math/science courses with teachers who had earned zero to 40 training hours.

Overall findings suggest that there is a slight positive association between the number of A2TeaMS training hours earned and student performance.

4. Did A2TeaMS teachers' attitudes and beliefs regarding instructional knowledge and abilities change by the end of the program?

The overall frequency distribution of responses at the end of the program shifted approximately

1 point on the scale, indicating that a greater number of teachers reported higher levels of ability and knowledge in various instructional areas.

By the end of the program, no teachers reported that they had minimal knowledge or abilities in the instructional areas and strategies taught by the A2TeaMS program.

Over half of the teachers who participated reported that they felt great enough about their knowledge and abilities to help others in the following instructional areas: “knowledge of the TEKS” (67.6%), “knowledge of the TEKS/TAKS alignment” (64.7%), “comfort with managing hands-on lessons” (61.8%), and “ability to teach course content” (55.9%).

Teachers reported developing most in their knowledge and ability to implement the 5E/7E Model of Instruction. The second area that teachers developed most in was in their knowledge of how math-science connections apply to their teaching and having the ability to present these connections to increase student engagement.

A2TEAMS 2010–2011

3

5. To what extent did A2TeaMS teachers’ use of math-science connections in their classrooms


Two forms of situational interest were assessed in the current evaluation: triggered situational interest, which involves eliciting initial interest in a subject (i.e., math) through features of the classroom; and maintained situational interest, which involves students developing deeper interest towards an academic subject and understanding the extent that learning that subject is useful and valuable.

After controlling for a students’ gender and math self-efficacy–two variables associated with math/science interest levels–taking a math course with an A2TeaMS teacher who discussed math-science connections in their classroom significantly increased the likelihood that: 1) features of the math classroom triggered students’ interest in math, 2) students felt interested in the material covered in their math class, and 3) students perceived the material covered in their math class as useful to learn.

After controlling for a students’ gender and self-efficacy for their science class, taking a science course with an A2TeaMS teacher who discussed math-science connections in their classroom significantly increased the likelihood that students’ interest in science were triggered, and that students perceived the science course material as useful to learn.

However, despite being significant, the variance explained by math-science connections on students’ situational interest for math/science was small.

Recommendations

1. Future evaluations of professional development programs should collect information on all aspects of the professional development program (i.e, the number of times teachers received one-on-one consultation from an instructional coach, the number of hours teachers spent collaborating/assisting each other in instructional development, etc.). These additional variables would help explain what aspects of the programs prove most helpful to the teachers, and in turn the students.

2. Based on the current findings, it seems worthwhile that math and science teachers continue to collaborate to create lessons integrating these two subject areas.


4

A2TEAMS EVALUATION 2010–2011

INTRODUCTION

Program Description Houston Independent School District (HISD) emphasizes teacher training as a means of raising

student achievement scores and building the professional skills and talent of teachers within the district. Therefore, the Academy of Accomplished Teaching in Math and Science (A2TeaMS) was created to provide ongoing professional development in the areas of content knowledge, research-based pedagogy, and leadership in math and science instructional domains. This type of professional development was implemented by pairing secondary math and science teachers with instructional coaches with math and science teaching expertise. In order to strengthen the academic programs at participating schools, coaches observed, evaluated, and provided helpful feedback on the daily instructional practices of A2TeaMS teachers. Furthermore, coaches facilitated formal professional development training meetings and workshops. A2TeaMS initially was intended to be a three-year program for all participating teachers. However, the 2010–2011 academic year was the third and final year of program implementation, thus teachers in the second cohort were only exposed to two years of the program. Throughout the program’s existence, A2TeaMS maintained five program goals, which were:

Create a learning organization that provides a coordinated, systematic, district-wide opportunity for consistent on-going mathematics and science professional development on a large scale;

Increase mathematics and science content and pedagogy knowledge of participating teachers as evidenced by rigorous, inquiry-based lessons aligned with the district curriculum, including natural mathematics-science connections;

Increase student achievement in mathematics and science with a focus on student college readiness;

Ensure the written curriculum is taught in the classroom; and Increase administrators’ knowledge of best practices and ability to assess and support best

practices in mathematics and science. Program Personnel Description and Teacher Accomplishments

During the 2010–2011 school year, the A2TeaMS staff facilitated all professional development of the A2TeaMS teachers and provided support in the classroom. Responsibilities of the A2TeaMS staff included coordinating training; conducting monthly school visits to provide support to teachers; coaching A2TeaMS teachers in the implementation of appropriate content and pedagogy; and collecting data on the effectiveness of the implementation of best practices (e.g., Hamilton-Reed, Ngoma, & Jones-Allen, 2009). Throughout the year, A2TeaMS staff provided support in mathematics and science that was unique and based on each campus’ needs. School administrators, regional specialists, and managers attended both the teacher and administrative A2TeaMS workshops.

Throughout the duration of the program, several A2TeaMS teachers were recognized for their teaching excellence. For example, several KBR Science Teacher awards were given to A2TeaMS teachers who effectively implemented the 7E Model of Instruction, an instructional framework that was a main teaching component of A2TeaMS training. Some examples of the awards received by A2TeaMS teachers were:

2011 KBR Science Teacher of the Year Finalist-1st Runner Up 2011 HISD Outstanding Asian Pacific Teacher of the Year 2010 KBR 5/7E Lesson Regional Winner 2009 KBR Science Teacher of the Year Award Regional Finalists

A2TEAMS 2010–2011

5

Purpose of the Evaluation The overall purpose of the current evaluation was to assess the effectiveness of the A2TeaMS

program. This was accomplished through several different analyses of information. First, data were collected on particular facets of the program to determine whether they were associated in enhancing students’ math and science interests and achievement. Furthermore, teachers’ attitudes and beliefs regarding their instructional knowledge and abilities were assessed at the beginning and end of the program to determine the extent that A2TeaMS teachers had developed in various instructional areas (i.e., implementing the 7E Model of Instruction). Specifically, the following questions were addressed:

1. How was the A2TeaMS program structured in 2010–2011? 2. What were the demographic and professional characteristics of A2TeaMS teachers? 3. Did performance differences exist between the students of A2TeaMS teachers with varying levels

of participation in the program? 4. Did A2TeaMS teachers' attitudes and beliefs regarding instructional knowledge and abilities

change by the end of the program? 5. To what extent did A2TeaMS teachers’ use of math-science connections in their classrooms


Review of Literature

Teacher Professional Development

“Effective professional development is not a passive, static process but a reflective and engaging activity that fosters teams, encourages professional development, and moves teachers toward success (Chappuis, Chappuis, Stiggins, 2009). A new way of conceptualizing learning is the team model, which requires teachers to commit to working and learning during and between team meetings (Chappuis et al., 2009). A more successful model of professional development is established that improves practices by emphasizing the career-long process of learning. Teacher incentives become a secondary support to the process of teacher-as-learner (Chappuis et al., 2009). These professional development programs were created to focus on the concrete tasks of teaching through assessment, observation, and reflection (Darling-Hammond & McLaughlin, 1995). A new paradigm is emerging in education that moves towards improving teacher learning opportunities so teachers stay informed of educational advancements (Hawley & Valli, 1999). The National Staff Development Council has created Standards for Staff Development that include organizing adult learners into learning communities, aligning goals with the schools, and developing collaborative efforts to deepen content knowledge (Wei, Darling-Hammond, Andree, Richardson, & Orphanos, 2009). Professional development that is sustained and intense has a greater chance of transforming teaching practices and student learning. Additionally, job-embedded models of support appear to have more of an impact on practice than the traditional model of workshop training (Wei et al., 2009)” (Tucker, 2010, p.4). Student Motivation for Math and Science

Student motivation is associated with numerous positive academic outcomes, such as grades, achievement scores, learning strategy use, persistence, and effort (Bong, 2001; Lau & Nie, 2008; Schraw, Horn, Thorndike-Christ, & Bruning, 1995; Wolters, 2004). An important theory developed by researchers to help explain students’ motivation, particularly for math and science, is Expectancy-Value Theory (EVT; e.g., Eccles (Parsons), Adler, & Meece, 1984; Eccles et al., 1983; Wigfield & Eccles, 2000). One of the main tenets of EVT is that students’ beliefs about their abilities regarding specific tasks within an academic domain and the extent that they value and are interested in a task influences their performance and the choices they make about what academic domains to pursue. Initially, this theory of motivation was developed to understand why females are not as likely to persist in science and math related majors and careers (Eccles et al., 1984). Research utilizing an expectancy-value framework has shown that girls


6

are less likely to enroll in advanced math courses (e.g., Nagy Trautwein, Baumert, Koller, & Garrett, 2006), report lower levels of interest (Watt, 2006), utility value (an individuals’ perception of the usefulness for engaging in a task), and ability perceptions (e.g., Updegraff & Eccles, 1996) in math. In addition, girls have lower intentions than boys to continue to take math classes and pursue math-related careers (Eccles et al., 1984; Meece et al., 1990; Watt, 2006). Supporting the main tenet of EVT, researchers have found that interest in math and/or science (Nagy , 2006; Simpkins, Davis-Kean, & Eccles, 2006; Watt, 2006; Yu, Corkin, & Trenor, 2010), and students’ perceptions of the usefulness for engaging in math and/or science-related academic tasks (Crombie et al., 2005; Updegraff & Eccles, 1996) predict intentions to continue taking math and/or science courses and pursuing math and/or science related careers

Related to EVT, researchers have also developed Interest Theory to explain students’ interest development towards a particular domain and how the development of interest influences students’ achievement-related behaviors (Hidi & Renninger, 2006). Researchers contend that Situational interest, defined as interest elicited by contextual factors (Hidi, 1990; Hidi & Harackiewicz, 2000; Hidi & Renninger, 2006) plays an integral role in learning in that teachers may facilitate its development to increase students’ motivation. Teachers can trigger students’ situational interest in a particular subject matter through external factors such as a lesson (triggered situational interest), which can in turn lead students to further develop and hold these interests (maintained situational interest) by engaging in self-initiated exploration, and making meaningful connections with the subject matter (Hidi & Harackiewicz, 2000).

Given the importance of student motivation as it relates to persistance and achievement, the current evalution examined the extent that A2TeaMs teachers’ promoted situational interest in their classrooms by talking to their students about how math and science topics are connected, a main concept taught through the A2TeaMS program. Two forms of situational interests were assessed in the current evaluation: triggered situational interest, which involves eliciting initial interest through features of the classroom; and maintained situational interest, which involves students developing deeper interest and value towards an academic domain, such as math and science.

Methods

Data Collection Data compiled for this report included student enrollment and individual identification numbers,

teacher identification, and teacher demographic information collected from the Texas Education Agency’s (TEA) Public Education Information Management System (PEIMS). Chancery Student Management System (SMS) databases were also used to verify enrollment of the students in HISD and validate demographic data. In order to assess the degree to which A2TeaMS teachers participated in the program, the number of hours that A2TeaMS teachers’ earned for attending training sessions was recorded using the 2008–2009, 2009–2010, and the 2010–2011 “E-Train” databases. E-Train is an electronic system that schedules and records professional development in HISD. EVAAS data in mathematics and science were also used in the current analyses.

A survey was administered to A2TeaMS teachers that asked them to report their beliefs and attitudes about their instructional knowledge and abilities. Survey data was collected at the beginning of the program (July 2008) and once at the end of the program (May 2011). Student survey data was also collected at the end of the 2010–2011 school year assessing students’ beliefs about their abilities (self-efficacy) and interests in math and science. In addition, students were asked if their teachers talked to them about math and science connections.

Student performance data were collected from the following test assessments: the Stanford Achievement Test (Stanford 10), the Aprenda: La Prueba de Logros en Espanol (Aprenda 3), and the Texas Assessment of Knowledge and Skills (TAKS).

A2TEAMS 2010–2011

7

Study Sample There were 100 teachers who participated in the A2TeaMS program in the three years that the

program was active. The analysis for this report focused on the performance of the 2010–2011 HISD middle and high school students who were enrolled in either a math or a science course with an A2TeaMS teacher. The 2009–2010 performance data was also collected for the same group of students to determine how they were performing prior to having been taught by an A2TeaMS teacher. Comparison groups were created by disaggregating students by the level of training that their A2TeaMS teachers received throughout the duration of the program. The level of training received was based on the number of hours that the teachers earned for attending training related to the A2TeaMS program. Instruments Stanford Achievement Test (Stanford 10)

The Stanford 10 assesses students’ academic achievement in various academic subjects across 12 grade levels (kindergarten through grade 11). This test provides a means of determining the relative standing of students’ academic performance when compared to the performance of students from a nationally representative sample. The normal curve equivalent (NCE; a normalized standard score) for the reading and mathematics subtests is reported in the current evaluation to assess student achievement. The NCE is a normalized standard score most often used when interpolating or averaging scores. Like the National Percentile Rank (NPR), the NCE is a norm-referenced score. In contrast to the NPR, however, the NCE provides an equal-interval scale that allows computations such as averaging or subtraction, which are useful in studying academic progress. NCE scores below 34.4 are considered “below average.” NCE scores between 34.4 and 64.9 are considered “average.” NCE scores above 64.9 are considered “above average” performance.

La prueba de logros en español, Tercera edición (Aprenda 3)

The Aprenda 3 is a norm-referenced, standardized achievement test in Spanish, and is used to assess the level of content mastery for students who receive instruction in Spanish. The Aprenda assesses students’ academic achievement in the same content areas as the Stanford; however, the Aprenda is not a translation of the Stanford. The current evaluation used the Aprenda NCE scores from the reading and mathematics subtests to assess student achievement.

Texas Assessment of Knowledge and Skills (TAKS)

The Texas Assessment of Knowledge and Skills (TAKS) is a state-mandated, criterion-referenced test administered for the first time in the spring 2003 as a means to monitor student performance. The English language version measures academic achievement in reading at grades 3–9; English Language Arts at 10 and 11; writing at grades 4 and 7; social studies at grades 8, 10, and 11; and science at grades 5, 8, 10, and 11. Students in the 11th grade are required to take and pass an exit level TAKS in order to graduate.

Measures Math/Science Self-efficacy

Self-efficacy was assessed by five items from Midgley et al.'s (2000) Patterns of Adaptive Learning Scale (PALS). Ratings were made on five-point scales with response anchors labeled 1 (strongly disagree) and 5 (strongly agree) to assess students’ perceptions of their confidence in doing math/science class work with higher scores indicating higher levels of self-efficacy. Adequate reliability was reported for this scale (α = .78)

Triggered Situational Interest

The Triggered Situational Interest scale (Linnenbrink-Garcia et al., 2010) consists of four items that


8

asked students about certain aspects of the class that they found interesting. Participants rated each item (e.g., “When we do math, my teacher does things that grab my attention.”) on a five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree) with higher scores indicating higher levels of interest towards aspects of their math/science class. Adequate reliability was reported for this scale (α = .86). Maintained Situational Interest-Feeling

The Maintained Situational Interest-Feeling scale (Linnenbrink-Garcia et al., 2010) consists of four items that asked students about how interested they felt about the material they were learning in their math/science class. Participants rated each item (e.g., “I find the math we do in class this year interesting.”) on a five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree) with higher scores indicating higher levels of interest towards math/science course material. Adequate reliability was reported for this scale (α = .92). Triggered Situational Interest-Value

The Maintained Situational Interest-Value scale (Linnenbrink-Garcia et al., 2010) consists of four items that asked students about how useful or important the material they were learning in their math/science class was to them. Participants rated each item (e.g., “What we are studying in math class is useful for me to know.”) on a five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree) with higher scores indicating higher levels of interest towards math/science course material. Adequate reliability was reported for this scale(α = .88). Math-Science Connections To assess whether teachers talked to students about how their math course topics were related to science and vice versa, students were asked to answer “yes” or “no” to the following question related to the A2TeaMS program: “Did your teacher talk to you about the connections between math and science?”

Results

How was the A2TeaMS program structured in 2010–2011?

A2TeaMS staff was available to campus administrators to provide support in mathematics and science instruction and to address training issues unique to each campus. A total of 36 campuses (19 middle schools and 17 high schools; see Appendix A) were supported by the program in 2010–2011. A2TeaMS staff worked under the direction of the Secondary Curriculum, Instruction, and Assessment Department. Teachers participated in professional development during the summer, followed by three half-day Saturday workshops, and three three-hour training sessions after the normal school day throughout the school year. For 2010–2011, of those teachers who were still active in the program, approximately 44 teachers were in their second year of training and 29 teachers were in their third and final year of training. No new teachers entered the program in 2010–2011. In 2010–2011, only 45 training hours were offered. In addition to coordinating professional development, A2TeaMS staff supported teachers in the implementation of best practices through on-site coaching.

In 2010–2011, three A2TeaMS staff worked with 73 teachers across 36 middle and high school campuses. The teacher/coach ratio was approximately one A2TeaMS staff to 24 teachers. In an effort to maintain consistency throughout A2TeaMS trainings and coaching, staff worked with the same teachers throughout the year. This provided a personalized program for each participating teacher.

The A2TeaMS was designed so that each teacher participates for three consecutive years. During the first year of implementation, the focus of the program was on content and pedagogy. During the second year of the program, the focus was on making mathematics-science connections and increasing leadership skills of participating teachers. In the third year of the A2TeaMS program, the focus was to assist teachers

A2TEAMS 2010–2011

9

in analyzing their practices through action research. A2TeaMS staff provided teachers with a theoretical understanding of concepts and content, followed by multiple demonstrations, and opportunities to practice new skills. There was an expectation that skills gained from trainings would be fully integrated into the teaching repertoire of participants. A2TeaMS teachers were also highly encouraged to work together and support each other through the professional development process.

Table 1: Demographic and Professional Characteristics of A2TeaMS Teachers, 2008–2011

(N = 100) n % Gender

Female 60 60.0

Male 32 32.0

Race/Ethnicity

African American 38 38.0

Hispanic 12 12.0

White 16 16.0

Asian 20 20.0

American Indian <5 --

Pacific Islander <5 --

More than 2 Races 5 0.1

Subject/Cohort

Math Cohort 1 21 21.0

Math Cohort 2 30 30.0

Science Cohort 1 15 15.0

Science Cohort 2 31 31.0

Grade Levels Taught (Duplicated)

Grade 6 20 20.0

Grade 7 22 22.0

Grade 8 23 23.0

Grade 9 35 35.0

Grade 10 37 37.0

Grade 11 39 39.0

Grade 12 34 34.0

Years Working in the District

<5 50 50.0

5-10 23 23.0 11-15 10 10.0

16-20 4 4.0 >20 5 5.0

Total Years of Teaching Experience

<5 37 37.0 5-10 22 22.0

11-15 17 17.0 16-20 6 6.0

>20 10 10.0

Notes: All data retrieved from PEIMS 2010–2011 Staff Database. There were approximately 100 teachers who were affiliated with A2TeaMS at some point in the program. Between three and eight teachers had missing data in demographic or professional development fields. The majority of teachers taught multiple grade levels.


10

What were the demographic and professional characteristics of A2TeaMS teachers?

Table 1 presents the demographic and professional characteristics of teachers who participated in the program at some point throughout the three years. Of the 100 A2TeaMS, 60 percent were female. In terms of race/ethnicity, the majority of A2TeaMS were either African American or Asian. The majority of A2TeaMS teachers belonged to the second cohort, and there were slightly more math teachers (51%) than science teachers (46%) who participated in the program. Many of A2TeaMS teachers taught multiple grade levels and most taught at a high school. About half of the teachers who participated in A2TeaMS were fairly new to the district (<5 years) and a greater percentage of A2TeaMS teachers (37%) had less than five years of teaching experience. Did performance differences exist between the students of A2TeaMS -trained teachers with varying levels of participation in the program? Two one-way analyses of variance (ANOVAs) were conducted to test the effects of A2TeaMS teachers’ level of training participation on the Stanford math and Stanford science NCE scores of their students. Four groups were created based on the frequency distribution of students who were in a particular quartile determined by the number of hours earned by their A2TeaMS teachers throughout the three years of the program. Training hours earned by A2TeaMS ranged from zero to 141 hours. The first group consisted of students who were in classrooms with teachers whose number of earned training hours fell in the bottom quartile (x < 41 hours). The second group consisted of students who were in classrooms with teachers whose number of earned training hours were in the second quartile (41-76 hours). The third group consisted of students who were in classrooms with teachers whose number of earned training hours fell in the third quartile (77-102 hours). Finally, the fourth group consisted of students who were in classrooms with teachers whose number of earned training hours were in the top quartile (x > 102 hours). Type III sums of squares (tests of independent effects of variables) were used for significance testing and Tukey's HSD (Honestly Significant Difference) test was used to conduct post-hoc comparisons.

Results of the ANOVAs showed a statistically significant difference between groups in Stanford math F(3,4558) = 50.05, p < .001, and Stanford science F(3,5012) = 23.29, p < .001 scores. The values of eta squared were η2 = .03 and η2 = .01, respectively. It appears that the effect of hours earned by A2TeaMS teachers on their students’ performance was slightly stronger for math. However, the earned training hour effect on Stanford math and Stanford science performance was small. Stanford 10 Posthoc Results

Table 2 presents the number of students who took the Stanford as well as the mean NCE scores and standard deviations by the four established groups. Results indicate that students who had math teachers who had earned 77-102 A2TeaMS training hours throughout the three years of the program significantly outperformed students from the other three groups in the math subtest. Surprisingly, this group of students outperformed their peers with math teachers that earned more than 102 hours of training.

Table 2: Means and Standard Deviations of 2010–2011 Stanford 10 Math and Science Normal Curve Equivalent (NCE) Student Scores by A2TeaMs Teacher Training Hours Earned between 2008–2009 to2010–2011

<41 Hours 41-76 Hours 77-102 Hours >102 Hours

n M SD n M SD n M SD n M SD Stanford Math 687 48.06a*b*c*** 17.08 1,420 52.46ad* 18.28 1,500 57.25bde* 19.46 952 50.50ce 17.13 Science 1,108 47.60f*g*h* 19.93 1,480 51.28fi** 17.21 1,149 53.97gi 19.90 1,276 52.77h 20.45 Note. Differences in means with similar superscripts were statistically significant. Math scores only include the results of students in classrooms with A2TeaMs math teachers, while science scores only include the results of students in classrooms with A2TeaMs science teachers. *p < .001; **p < .013; ***p < .05. NCEs = Normal Curve Equivalents.

A2TEAMS 2010–2011

11

However, it is clear that students with math teachers that earned more than 41 training hours significantly outperformed their peers with math teachers who participated in less than 41 training hours on the math subtest.

In regards to the Stanford science results, students with science teachers who earned 77-102 A2TeaMS training hours throughout the three years of the program significantly outperformed the group of students with science teachers who earned less than 41 training hours and the group of students with science teachers who earned 41 to 76 training hours. Overall results seem to suggest that A2TeaMS training hours earned by math/science teachers were positively associated with student performance on the Stanford math and science exams. TAKS

Table 3 depicts mathematics and science 2010–2011 TAKS scores for students of A2TeaMS teachers who earned more than 40 training hours compared to A2TeaMS teacher who earned minimal to no training hours (0-40). The data reveal that a greater percentage of students among teachers who earned more than 40 hours passed the mathematics and science TAKS exam when compared to students who had math/science courses with teachers who had earned zero to 40 training hours. In addition, the student group enrolled in classes with A2TeaMS teachers who earned more than 40 training hours had a greater percentage of students reaching commended levels when compared to the percent of students reaching commended levels who had teachers with zero to 40 training hours. However, these results should be interpreted with caution given the discrepancy between group sample sizes.

Stanford 10 Results

Paired-samples t-tests were conducted to determine whether there were significant differences between students’ 2009–2010 Stanford math and science performance (the year prior to being enrolled in a class with an A2TeaMS teacher), and students’ 2010–2011 Stanford math and science performance who were enrolled in a class taught by an A2TeaMS teacher. Table 4 presents 2009–2010 and 2010–2011 means, standard deviations, t-statistics, and degrees of freedom on the two Stanford subtests by math and science cohorts.

The data show that the 2010–2011 average NCE on the Stanford math subtest among students who had an A2TeaMS math cohort 1 teacher significantly increased from the year prior to having been taught by an A2TeaMS teacher and after being taught by an A2TeaMS teacher. The average math NCE score of this student group increased by 1.21 NCEs. However, no statistically significant differences were found between this group of students’ 2009–2010 and 2010–2011 average Stanford science NCE score. Cohort 1 science teachers showed a statistically significant increase in their 2010–2011 average science NCE score, but a statistically significant difference was not found in their math performance. The average science score of students of A2TeaMS cohort 1 science teachers increased by 2.78 NCEs.

On both the math and science Stanford subtests, students of cohort 2 mathematics teachers scored significantly higher in 2010–2011 compared to their average NCE scores the year prior to being taught by an A2TeaMS teacher. On the Stanford math subtest, students’ scored about a half NCE better than the year prior. On the Stanford science subtest, these students’ average score increased by 1.94 NCEs.

Table 3: TAKS Math and Science Performance of Students who had a Math or Science Teacher in 2010–2011 Participating in A2TeaMS

Mathematics 2010–2011 Science 2010–2011 Percent

Met Standard

Percent Commended

n Percent Met Standard

Percent Commended

n

Students with A2TeaMs Teachers (>40 hrs. training)

76.3 19.6 7,545 76.2 18.9 4,155

Comparison Group (0-40 hrs. training)

71.9 13.4 2,175 70.5 11.1 840


12

For students of cohort 2 A2TeaMS science teachers, on both the math and science Stanford subtests, students scored significantly higher in 2010–2011 compared to their average NCE scores the year prior to being taught by an A2TeaMS teacher. On the Stanford math subtest, students’ scored about a half NCE better than the year prior. On the Stanford science subtest, these students’ average score increased by 4.1 NCEs.

When combining students from both cohorts together, statistically significant gains were made on both the math and science Stanford subtests. Students taught by an A2TeaMS teacher scored significantly higher in 2010–2011 compared to their average NCE scores the year prior to being taught by an A2TeaMS teacher. On the Stanford math subtest, students’ scored about a half NCE better than the year prior. On the Stanford science subtest, these students’ average score increased by 2.33 NCEs.

Results suggest that students’ taught by cohort 1 teachers tended to perform significantly better only in the respective subject that was taught by their A2TeaMS teachers compared to their performance the year prior to being taught by an A2TeaMS teacher. On the other hand, students taught by cohort 2 teachers performed significantly better in both math and science subtests compared to the year prior to being taught by an A2TeaMS teacher. However, the fact that the sample sizes of these student groups are very large warrants precaution in interpreting the statistically significant findings.

In addition to students’ performance level gains, further information regarding A2TeaMS teachers’ teaching effectiveness can be found in Appendix B, which displays their cumulative 3-year value-added status. Did A2TeaMS teachers' attitudes and beliefs regarding instructional knowledge and abilities change by the end of the program?

At the first summer training of the A2TeaMS teacher training session in 2008, teachers were given a survey regarding their beliefs about their abilities to implement various instructional strategies, increase student engagement, and manage their classrooms. Teachers were also asked about the extent that they had knowledge in pedagogy and course content. A total of 42 teachers filled out the survey in 2008. The same survey was administered to A2TeaMS teachers at the last A2TeaMS meetings in 2011 and 34 teachers completed it. Participants rated each survey item on a five-point scale: 1 = “I have minimal ability/knowledge in this area”; 2 = “Exists but needs improvement”; 3 = “OK, I can do it/I know it”; 4 = “Good, I know what I am doing”; 5 = “Great, and I can help others.” Figure 1 displays the 2008 and 2011 mean ratings on each survey item by ranking them from most to least change in levels of knowledge and abilities in the areas mentioned previously.

Table 4: Mean Differences Between 2009–2010 and 2010–2011 Stanford 10 and Math and Science Normal Curve Equivalent (NCE) Scores by A2TeaMs Cohort Groups

2009–2010 2010–2011 n M SD M SD t df Math Cohort 1 Math 1,589 54.09 16.89 55.30 18.03 -3.76*** 1,588 Science 1,560 52.78 17.59 52.40 18.59 1.07 1,559 Math Cohort 2 Math 2,504 52.45 17.05 53.02 18.42 -2.06* 2,503 Science 2,498 50.84 18.05 52.78 18.19 -6.20*** 2,497 Science Cohort 1 Math 1,046 51.61 17.27 51.19 18.59 0.96 1,045 Science 1,037 49.58 17.47 52.36 18.24 -5.73*** 1,036 Science Cohort 2 Math 3,314 51.44 17.84 51.99 18.16 -2.47* 3,313 Science 3,239 48.90 18.76 53.00 19.18 -15.56*** 3,238 All Cohorts Math 8,778 52.27 17.44 52.79 18.42 -3.65*** 8,777 Science 8,658 50.33 18.31 52.66 18.75 -14.26*** 8,657 Note. ***p < .001; **p < .01; *p < .05. NCEs = Normal Curve Equivalents

A2TEAMS 2010–2011

13

Based on the mean ratings at the beginning of the program, it appears teachers were aware of the instructional strategies they were asked about and had some ability in implementing them (see Appendix C for response frequency distributions). Specifically, the majority of responses ranged from “OK, I can do it/I know it” to “Good, I know what I am doing.” However, a small percentage of the respondents had minimal knowledge about the 5E Model of Instruction and how to implement these instructional strategies in their classrooms (9.8%). In addition, a small percentage of respondents had minimal knowledge of applying math-science connections in their classrooms (9.5%) and minimal ability in applying that knowledge to increase student engagement (4.8%). The overall frequency distribution of responses at the end of the program shifted approximately one point on the scale, indicating that a greater number of teachers reported higher levels of ability and knowledge in various instructional areas. The majority of responses now ranged between “Good, I know what I am doing” to “Great, and I can help others.” By the end of the program, no teachers responded that they had minimal knowledge or abilities in the instructional areas taught by the A2TeaMS program. Over half of the teachers who participated reported that they felt great enough about their knowledge and abilities to help others in the following instructional areas: “knowledge of the TEKS” (67.6%), “knowledge of the TEKS/TAKS alignment” (64.7%), “comfort with managing hands-on lessons” (61.8%), and “ability to teach course content” (55.9%). Instructional areas that teachers developed most were based on the difference in mean ratings at the beginning and end of the A2TeaMS program. Figure 1 shows that teachers developed most in their knowledge and ability to implement the 5E/7E Model of Instruction. The second area that teachers

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Ability to use math-science connections to increase rigor of lessons

Ability to use quality questions with students to increase student engagement

Knowledge of the TEKS for courses students take before and after the class I teach

Ability to help all students learn content

Ability to plan effective lessons

Comfort with managing hands-on lessons

Ability to teach course content this year

Knowledge of the TEKs for courses taught

Ability to use assessment data to redirect instruction

Knowledge of the TEKS/TAKS alignment for courses taught

Ability to use a variety of types of assessments

Ability to use math-science connections to increase student engagement

Knowledge of math-science connections as they apply to classes I teach

Knowledge of the 5E/7E Model of instruction

Ability to implement the 5E/7E Model of Instruction

End of Program Beginning of Program

Figure 1: Mean Ratings on Survey Items Measuring A2TeaMS teacher attitudes and beliefs about their knowledge and teaching abilities at the beginning and end of the A2TeaMS Program.


14

developed most was their knowledge of how math-science connections apply to their teaching and their ability to present these connections to increase student engagement. To what extent did A2TeaMS teachers’ use of math-science connections in their classrooms influence students’ situational interest for math and science?

During the last few weeks of the 2010–2011 academic year, students taught by an A2TeaMS teacher were invited to participate in an on-line survey assessing their math or science self-efficacy and situational interest (see Appendix D). Students who were taking courses with an A2TeaMS math teacher (n = 223) answered questions regarding their math class, and students who were taking courses with an A2TeaMS science teacher (n = 341) answered questions regarding their science class.

A2TeaMS teachers were encouraged throughout the program to integrate mathematics and science concepts and to incorporate the associations between math and science in their lessons. To determine the extent that this instructional practice influenced students’ motivation in math and science, six hierarchical regressions were conducted predicting students’ triggered situational interest, maintained situational interest-feeling, and maintained situational interest-value for their math or science class. In accordance with previous research that has examined variables associated with interests in math and science (e.g., Eccles et al., 1984; Lent et al., 2001), gender and self-efficacy were included as control variables in the regressions. See Appendix E for Cronbach’s alpha values, descriptive statistics, and Pearson correlations among the main variables analyzed. Math Interest A summary of the hierarchical regression results are presented in Table 5. In the hierarchical regression predicting triggered situational interest for math, after controlling for gender and self-efficacy in step 2, the model was significant, F(3,222) = 38.10, p < .001, R2= 52%, Δ R2 = 3%. Math-science connections (p < .01) was a significant positive predictor of students’ triggered situational interest in math even after controlling for student gender and self-efficacy for their math course. In the hierarchical regression predicting maintained situational interest-feeling for math, after controlling for gender and self-efficacy in step 2, the model remained significant, F(3,222) = 81.47, p < .001, R2= 49%, Δ R2 = 3%. Math-science connections (p < .001) was a significant positive predictor of students’ maintained situational interest-feeling in math even after controlling for student gender and self-efficacy for math. Finally, in the hierarchical regression predicting maintained situational interest-value for math, after controlling for gender and self-efficacy in step 2, the model remained significant, F(3,222) = 102.33, p < .001, R2= 52%, Δ R2 = 7%. Math-science connections (p < .001) was a significant positive predictor of students’ maintained situational interest-value in math even after controlling for student gender and self-efficacy for math. Science Interest

In the hierarchical regression predicting triggered situational interest for science, after controlling for gender and self-efficacy in step 2, the model remained significant, F(3,340) = 84.91, p < .001, R2= 41%, Δ R2 = 2%. Math-science connections (p < .01) was a significant positive predictor of students’ triggered situational interest in science even after controlling for student gender and self-efficacy for science. In the hierarchical regression predicting maintained situational interest-feeling for science, after controlling for gender and self-efficacy in step 2, the model remained significant; however, math-science connections was not a significant predictor of maintained situational interest-feeling for science. In the hierarchical regression predicting maintained situational interest-value for science, after controlling for gender and self-efficacy in step 2, the model remained significant, F(3,340) = 100.29, p < .001, R2= 46%, Δ R2 = 1%. Math-science connections (p < .01) was a significant positive predictor of students’ maintained situational interest-value in science even after controlling for student gender and self-efficacy in science.

A2TEAMS 2010–2011

15

Table 5: Summary of Regression Analyses Predicting A2TeaMS Students’ Situational Interest for Math and Science

Math Course Science Course

Variable

Triggered Situational Interesta

Maintained Situational

Interest-Feelingb


Interest-Valuec

Triggered Situational

Interestd


Interest-Feelinge


Interest-Valuef

β β β β β β

Step 1

Gender (female = 1) .04 -.03 -.04 -.02 -.06 .05

Self-efficacy for course .72*** .70*** .72*** .64*** .70*** .69***

Step 2

Gender .04 -.02 -.03 -.03 -.06 .04

Self-efficacy for course .68*** .65*** .64*** .62*** .69*** .68***

Math-Science Connections .17** .18*** .27*** .14** .04 .09*

Note. β indicates standardized regression coefficient. N = 223 for math. N = 341 for science. *p < .05. **p < .01. ***p < .001.

a R2 = .52, p < .001 for Step 1; ΔR2 = .03, p < .01 for Step 2 b R2 = .49, p < .001 for Step 1; ΔR2 = .03, p < .001 for Step 2. c R2 = .52, p < .001 for Step 1; ΔR2 = .07, p < .001 for Step 2. d R2 = .41, p < .001 for Step 1; ΔR2 = .02, p < .01 for Step 2. e R2 = .50, p < .001 for Step 1; ΔR2 = .00, p > .05 for Step 2. f R2 = .46, p < .001 for Step 1; ΔR2 = .01, p < .05 for Step 2.

Overall findings suggest that when students’ report that their A2TeaMS teachers discuss math-science connections, they are more likely to report higher levels of interest for their math/science class, and for the material covered in class. Students are also more likely to report higher levels of utility value for the course, or in other words, tend to report that the course material is useful to know. Consistent with previous studies (e.g., Lent et al., 2008), the current findings indicate that a students’ level of confidence in their abilities to do well in a math/science course (self-efficacy) was significantly associated with students’ interest level in math/science. This is important to mention because this may suggest that by discussing math-science connections, teachers can influence students’ situational interest for the course, above and beyond students’ own math/science self-efficacy. However, future evaluations should include more robost measures to assess the extent that teachers integrate and present math and science topics in their classrooms, given that the current measure only had students report whether teachers did or did not make math-science connections.

Discussion

The A2TeaMS program has provided teachers ongoing professional development in content, pedagogy, leadership, and action research in mathematics and science. For the 2010–2011 school year, the final year of the program, an overall assessment of the program was conducted to determine whether teachers’ participation in the program–over the last three years for cohort one and over the last two years for cohort two–was associated with students’ performance. The teacher training data does seem to suggest that the number of A2TeaMS training hours teachers earned was significantly associated with students’ 2010–2011 Stanford math and science performance. However, the effects of these associations


16

were fairly small. When disaggregating teachers by cohorts, results suggest that students’ taught by cohort 2 teachers

had significant gains in both math and science Stanford performance compared to their performance the prior year (2009–2010) to being taught by an A2TeaMS teacher. Students taught by a cohort 1 teacher only made significant gains in the subject area that was taught by their A2TeaMS teacher. This finding may be explained by the fact that cohort 2 teachers were in their second year of the program when the focus of their instruction was in making math-science connections. It is possible that the emphasis in helping students make these connections, translated into improved test scores in both math and science.

In terms of TAKS performance, a greater percentage of students of teachers with more than 40 hours of training passed the science and math TAKS exam and reached commended levels compared to teachers who received a minimal amount of A2TeaMS training. Overall performance findings do seem to suggest that teachers’ level of participation in the program had a slight positive effect on their students’ performance. Concerning teachers’ attitudes about their abilities and knowledge of the instructional practices taught by A2TeaMS personnel, it does appear that by the end of the program the general consensus of A2TeaMS teachers was that they were more confident in their abilities to implement certain instructional strategies (i.e., 5E/7E Instructional Model, math-science connections) compared to their attitudes at the beginning of the program. Furthermore, support for the fact that teachers were able to successfully implement certain strategies taught by A2TeaMS can been seen by the results involving the positive associations found between teachers’ application of math-science connections and students’ interest and value for math and/or science. In conclusion, findings from this evaluation suggest that the A2TeaMS program benefitted teachers by enhancing their instructional repertoire and overall confidence for applying various teaching strategies. As a result of teachers’ improved teaching skills, students also appear to have benefitted from the program by experiencing academic and motivational gains in the areas of math and science.

Recommendations

1. Future evaluations of professional development programs should collect information on all aspects of the professional development program (i.e, the number of times teachers received one-on-one consultation from an instructional coach, the number of hours teachers spent collaborating/assisting each other in instructional development, etc.). These additional variables would help explain what aspects of the programs prove most helpful to the teachers and in turn the students.

2. Based on the current findings, it seems worthwhile that math and science teachers continue to collaborate to create lessons integrating these two subject areas.

A2TEAMS 2010–2011

17

References

Bong, M. (2001). Between- and within-domain relations of academic motivation among middle and high

school students: Self-efficacy, task value, and achievement goals. Journal of Educational Psychology, 93(1), 23-34. doi:10.1037/0022-0663.93.1.2.

Chappuis, S., Chappuis, J., Stiggins, R. (2009). Supporting teacher learning teams. Educational Leadership, 66(5), 56–60. Crombie, G., Sinclair, N., Silverthorn, N., Byrne, B. M., Dubois, D. L., & Trinneer, A. (2005). Predictors

of young adolescents' math grades and course enrollment intentions: Gender similarities and differences. Sex Roles, 52(5/6), 351-367.

Darling-Hammond, L., & McLaughlin, M. W., (1995). Policies that support professional development in an era of reform. Phi Delta Kappan, 76(8), 597-604. Eccles, J. S., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C. (1983).

Expectancies, values, and academic behaviors. In J. T. Spence (Ed.), Achievement and achievement motivation (pp. 75-146). San Francisco, CA: W. H. Freeman.

Eccles (Parsons), J., Adler, T., & Meece, J. L. (1984). Sex differences in achievement: A test of alternate theories. Journal of Personality and Social Psychology, 46, 26-43.

Hawley, W. & Valli, L. (1999). The Essentials of Effective Professional Development. In Darling- Hammond, L. & Sykes, G (eds.) Teaching as the Learning Profession: Handbook of Policy and Practice. San Francisco: Jossey- Bass Publishers.

Hamilton-Reed, L., Ngoma, B., & Jones-Allen, W. (2009). Student Engagement. Unpublished presentation, Houston Independent School District.

Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational Research, 60, 549-571.

Hidi, S., & Harackiewicz, J. M. (2000). Motivating the academically unmotivated: A critical issue for the 21st century. Review of Educational Research, 70, 151-179.

Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41, 111-127.

Lau, S., & Nie, Y. (2008). Interplay between personal goals and classroom goal structures in predicting student outcomes: A multilevel analysis of person-context interactions. Journal of Educational Psychology, 100(1), 15-29. doi:10.1037/0022-0663.100.1.15.

Lent, R.W., Brown, S.D., Brenner, B., Chopra, S.B., Davis, T., Talleyrand, R., et al. (2001). The role of contextual supports and barriers in the choice of math/science educational options: A test of social cognitive hypotheses. Journal of Counseling Psychology, 48, 474-483.

Lent, R.W., Lopez, A.M., Lopez, F.G., & Sheu, H. (2008). Social cognitive career theory and the prediction of interests and choice goals in the computing disciplines. Journal of Vocational Behavior, 73, 52-62.

Linnenbrink-Garcia, L., Durik, A. M., Conley, A. M., Barron, K. E., Tauer, J. M., Karabenick, S. A., & Harackiewicz, J. M. (2010). Measuring situational interest in academic domains. Educational and Psychological Measurement, 70(4), 647-671. doi:10.1177/0013164409355699.

Meece, J. L., Wigfield, A., & Eccles, J. S. (1990). Predictors of math anxiety and its influence on young adolescents' course enrollment intentions and performance in mathematics. Journal of Educational Psychology, 82(1), 60-70.

Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., Gheen, M., Kaplan, A., Kumar, R., Middleton, M. J., Nelson, J., Roeser, R., & Urdan, T. (2000). Manual for the Patterns of Adaptive Learning Scales (PALS), Ann Arbor, MI: University of Michigan.

Nagy, G., Trautwein, U., Baumert, J., Koller, O., & Garrett, J. (2006). Gender and course selection in upper secondary education: effects of academic self-concept and intrinsic value. Educational


18

Research and Evaluation, 12(4), 323-345. Schraw, G., Horn, C., Thorndike-Christ, T., & Bruning, R. (1995). Academic goal orientations and

student classroom achievement. Contemporary Educational Psychology, 20(3), 359-368. doi:10.1006/ceps.1995.1023.

Simpkins, S. D., Davis-Kean, P. E., & Eccles, J. S. (2006). Math and science motivation: A longitudinal examination of the links between choices and beliefs. Developmental Psychology, 42(1), 70-83.

Tucker, J. (2010). 2009–2010 A2TeaMS Evaluation Report. Unpublished manuscript. Updegraff, K. A., & Eccles, J. S. (1996). Course enrollment as self-regulatory behavior: Who takes

optional high school math courses? Learning and Individual Differences, 8, 239-259. Watt, H. M. G. (2006). The role of motivation in gendered educational and occupational trajectories

related to math. Educational Research and Evaluation, 12, 305-322. Wei, R. C., Darling-Hammond, L., Andree, A., Richardson, N., Orphanos, S. (2009). Professional

learning in the learning profession: A status report on teacher development in the United States and abroad. Dallas, TX. National Staff Developpment Council.

Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68-81.

Wolters, C. A. (2004). Advancing achievement goal theory: Using goal structures and goal orientations to predict students’ motivation, cognition, and achievement. Journal of Educational Psychology, 96, 236-250. doi:10.1037/0022-0663.96.2.236.

Yu, S. L., Corkin, D. M., & Trenor, J. M. (2010, May 3). The role of motivation in choice of engineering among ethnically diverse adolescents. Paper presented at the Annual Meeting of the American Educational Research Association, Denver, CO.

A2TEAMS 2010–2011

19

APPENDIX A List of Schools Participating in A2TeaMS, 2010–2011

School # High School Grade Level 001 AUSTIN HS 9-12 003 DAVIS HS 9-12 010 MADISON HS 9-12 012 REAGAN HS 9-12 017 WESTBURY HS 9-12 018 WHEATLEY HS 9-12 019 WORTHING HS 9-12 020 YATES HS 9-12 024 SCARBOROUGH HS 9-12 027 CHAVEZ HS 9-12 029 CONT LRN CENTER HS 9-12 034 LAW ENF/CRI JUS HS 9-12 043 BURBANK MS 6-8 044 CULLEN MS 6-8 045 DEADY MS 6-8 046 EDISON MS 6-8 049 HAMILTON MS 6-8 051 HARTMAN MS 6-8 054 JACKSON MS 6-8 056 WELCH MS 6-8 059 LONG MS 6-8 060 REVERE MS 6-8 064 PERSHING MS 6-8 067 SMITH, EO MS 6-8 068 GRADY MS 6-8 078 FLEMING MS 6-8 080 RICE MS 6-8 081 SHARPSTOWN MS 6-8 082 WILLIAMS, MC MS 6-8 309 NINTH GRADE COLLEGE PREP ACADEMY 9 310 HOUSTON MATH/SCI/TECH CENTER 9-12 322 CARNEGIE VANGUARD HS 9-12 334 KALEIDOSCOPE SCHOOL 6-8 338 ORTIZ, DANIEL JR MS 6-8

348 HOUSTON ACADEMY FOR INTERNATIONAL STUDIES 9-12

454 EMPOWERMENT SOUTH EARLY COLLEGE 9-10


20

APPENDIX B

Table A: 3-Year Cumulative Value-Added Status of A2TeaMS Teachers by Content Area

Mathematics Science n % n %

Well Above 7 39 7 28 Above 0 0 3 12 NDD 3 17 9 36 Below 3 17 3 12 Well Below 5 28 3 12 Source: 2011 SAS EVAAS Data File. Please note that this data only includes A2TeaMS Teachers who taught these respective subject matters for 3 consecutive years.

A2TEAMS 2010–2011

21

APPENDIX C

Table B. Survey Response Rates of A2TeaMS Teachers at the Beginning of the Program to Attitudes and Beliefs of their Teaching Abilities and Knowledge Questionnaire, 2008–2009

Great and I can help others

Good, I know what I am

doing Ok, I can do

it

Exists but needs

improvement

I have minimal ability/knowledge

in this area Ability to teach course content this year 42.9 35.7 7.1 14.3 0.0 Ability to help all students learn content 19.0 42.9 23.8 11.9 2.4 Ability to plan effective lessons 17.1 34.1 29.3 19.5 0.0 Ability to use a variety of types of assessments 16.7 35.7 21.4 23.8 2.4 Ability to use assessment data to redirect instruction 16.7 38.1 31.0 11.9 2.4 Knowledge of the 5E/7E Model of Instruction 12.2 29.3 26.8 22.0 9.8 Ability to Implement the 5E/7E Model of Instruction 9.8 22.0 29.3 29.3 9.8 Knowledge of the TEKs for courses taught 26.2 42.9 21.4 9.5 0.0 Knowledge of the TEKS/TAKS alignment for courses taught 24.4 41.5 24.4 9.8 0.0 Knowledge of the TEKS for courses students take before and after the class I teach 7.1 42.9 28.6 19.0 2.4 Knowledge of math-science connections as they apply to classes I teach 7.1 14.3 45.2 23.8 9.5 Ability to use math-science connections to increase student engagement 4.8 26.2 31.0 33.3 4.8 Ability to use math-science connections to increase rigor of lessons 14.3 33.3 35.7 16.7 0.0 Comfort with managing hands-on lessons 16.7 40.5 35.7 7.1 0.0 Ability to use quality questions with students to increase student engagement 14.3 42.9 31.0 9.5 2.4


22

APPENDIX C (continued)

Table C. Survey Response Rates of A2TeaMS Teachers at the End of the Program to Attitudes and Beliefs of their Teaching Abilities and Knowledge Questionnaire, 2010–2011

Great and I can help others

Good, I know what I am

doing Ok, I can do

it

Exists but needs

improvement

I have minimal ability/knowledge

in this area Ability to teach course content this year 55.9 32.4 5.9 5.9 0.0 Ability to help all students learn content 38.2 55.9 2.9 2.9 0.0 Ability to plan effective lessons 38.2 55.9 2.9 2.9 0.0 Ability to use a variety of types of assessments 41.2 50.0 5.9 2.9 0.0 Ability to use assessment data to redirect instruction 44.1 50.0 5.9 0.0 0.0 Knowledge of the 5E/7E Model of Instruction 47.1 44.1 8.8 0.0 0.0 Ability to Implement the 5E/7E Model of Instruction 38.2 50.0 8.8 2.9 0.0 Knowledge of the TEKs for courses taught 67.6 32.4 0.0 0.0 0.0 Knowledge of the TEKS/TAKS alignment for courses taught 64.7 35.3 0.0 0.0 0.0 Knowledge of the TEKS for courses students take before and after the class I teach 26.5 52.9 17.6 2.9 0.0 Knowledge of math-science connections as they apply to classes I teach 32.4 50.0 11.8 5.9 0.0 Ability to use math-science connections to increase student engagement 29.4 52.9 11.8 5.9 0.0 Ability to use math-science connections to increase rigor of lessons 23.5 55.9 8.8 11.8 0.0 Comfort with managing hands-on lessons 61.8 26.5 8.8 2.9 0.0 Ability to use quality questions with students to increase student engagement 32.4 58.8 8.8 0.0 0.0

A2TEAMS 2010–2011

23

APPENDIX D

Academic Self-efficacy (Midgley et al., 2000) I can do almost all the work in my science class if I don't give up.

I'm certain I can master the skills taught in my science class. I'm certain I can figure out how to do the most difficult tasks in my science class.

Even if my science class work is hard, I can learn it. I can do even the hardest work in my science class. Triggered Situational Interest (Linnenbrink-Garcia et al., 2010) My science teacher is exciting. When we do science activities, my teacher does things that grab my attention.

This year, my science class is often entertaining. My science class is so exciting it's easy to pay attention. Situational Interest-Feeling (Linnenbrink-Garcia et al., 2010) What we are learning in science class this year is fascinating to me. I am excited about what we are learning in science class this year. I like what we are learning in science class this year. I find the science we do in class this year interesting. Situational Interest-Value (Linnenbrink-Garcia et al., 2010) What we are studying in science class is useful for me to know.

The things we are studying in science this year are important to me.

What we are learning in science this year can be applied to real life.

We are learning valuable things in science class this year.

Math-Science Connections Did your teacher talk to you about the connections between math and science.


24

APPENDIX E

Cronbach Alphas, Means, Standard Deviations, and Pearson Correlations among the Main Variables for Middle and High School Math Courses

Variable α M SD 1 2 3 4 5 1. Gender (female = 1) -- -- -- ---

2. Self-efficacy for course .88 3.70 0.85 -.10 ---

3. Math-Science Connections -- -- -- -.06 .29*** ---

4. Triggered Situational Interest .86 3.45 0.97 -.02 .70*** .36*** ---

5. Maintained Situational Interest- Feeling

.92 3.40 0.93 -.12 .71*** .37*** .82*** ---

6. Maintained Situational Interest-Value

.88 3.84 0.85 -.09 .69*** .46*** .74*** .76***

Notes. N = 223-244; *p < .05. **p < .01. ***p < .001.

Cronbach Alphas, Means, Standard Deviations, and Pearson Correlations among the Main Variables for Middle and High School Science Courses

Variable α M SD 1 2 3 4 5 1. Gender (female = 1) -- -- -- ---

2. Self-efficacy for course .90 3.60 0.86 -.15** ---

3. Math-Science Connections -- -- -- .06 .11 ---

4. Triggered Situational Interest .86 3.53 0.89 -.11* .70*** .20*** ---

5. Maintained Situational Interest-Feeling

.91 3.40 0.93 -.16** .64*** .11* .81*** ---

6. Maintained Situational Interest-Value

.84 3.63 0.83 -.05 .71*** .16** .72*** .79***

Notes. N = 343-347; *p < .05. **p < .01. ***p < .001.

Documents

A2TeaMS Board Page 2011 · and teachers’ instructional knowledge and abilities. In 2010–2011, three A2TeaMS staff worked with 73 teachers across 36 middle and high school campuses