AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean

Connecting Teaching and Learning in Teacher Preparation:

A Closer Look at Teacher Work Samples

AACTE/NCATE Institutional OrientationApril 3, 2008

Hilda Rosselli, Dean

Western Oregon College of Education

[email protected]

April 3-6, 2008 H.Rosselli (AACTE/NCATE) 2

Acknowledgements

All of the faculty, staff, and administrators

who have contributed over the years.

Other colleagues and institutions who have

adapted and furthered TWSMs.

Lead Developer and Advocate of the

Teacher Work Sample Methodology (TWSM)


WOU Context

Small, public comprehensive institution (Undergraduate, MAT, and MSinEd for SPED)

Home of Teacher Work Sample Methodology

H. Del Schalock, TRI mentor to the College(from 1966-2006)

Leadership role in Oregon teacher preparation


Teacher Educators as Professionals

• “…it is not sufficient for teacher educators to simply ‘teach’ prospective teachers about teaching, or about content to be taught.”

• Teacher educators “are obliged…to be sure that teacher candidates can in fact apply what they have learned about teaching with sufficient skill and adaptation that all K-12 students taught by a candidate are able to make appreciable progress in their learning.”

» Schalock & Imig (2000) Shulman’s Union of Insufficiencies +7: New Dimensions of Accountability for Teachers and Teacher Educators.




Our goal…

“…assure that subject content knowledge, teaching skills,and dispositions will effectively advance student learning, not be ends in themselves.” (Elliot, 2005)


State Context• Proficiency based teacher preparation

programs• Standards identified for both initial

teacher licensure and continuing teacher licensure

• Tied to PK-12 student standards• Two Teacher Work Samples are

required for initial licensure


Teacher Work Sample Methodology (TWSM)

Components

• Work Sample Setting• Work Sample Goals and Objectives• Work Sample Unit Plan Rationale• Work Sample Lesson Plans• Work Sample Assessment Plan & Assessments• Analysis and Interpretation of Student Learning• Reflection


The Work Sample Setting

Purpose: Analysis of contextual factors about the “community” that impact teaching and learning.

Standard: The candidate analyzes contextual information to determine relevant factors that influence curriculum, instruction, learning goals and management decisions.


Setting Contextual Factors Affecting Teaching and Learning (Met = 11-12)

3 2 1 0 Knowledge of community, district and school factors

Detailed description and analysis of community, district and school factors that are relevant to teaching and learning

Describes and analyzes community, district and school factors that are relevant to teaching and learning

Little evidence of understanding the community, district and school factors that are relevant to teaching and learning

No evidence of understanding the community, district and school factors that are relevant to teaching and learning

Knowledge of classroom factors

Detailed description and accurate portrayal of relevant classroom climate and physical environment factors that may affect learning

Some description and fairly accurate portrayal of relevant classroom climate and physical environment factors that may affect learning

Little description and/or inaccurate portrayal of relevant classroom climate and physical environment factors that may affect learning

No evidence of understanding of what relevant classroom climate and physical environment factors are that could affect learning

Knowledge of student characteristics

Explains important characteristics of students as learners

Explains some of the important characteristics of students as learners

Little explanation of the important characteristics of students as learners.

Minimal to no inclusion of student characteristics that affect learners

Overall Shows a clear ability to be selective in the data included, to interpret that data, and synthesize the information to accurately describe the context in which the work sample is taught.

Most of the informational data included is selective, interpreted and synthesized in order to accurately describe the context in which the work sample is taught

Some of the informational data included is selective, interpreted and synthesized in order to accurately describe the context in which the work sample is taught

Unable to be selective in the data included, interpret that data, and synthesize the information in order to accurately describe the context in which the work sample is taught


Goals and ObjectivesPurpose: The goals and objectives tell what

learning outcomes are expected of students and how they relate to District, State or National Content Standards and/or Common Curriculum Goals.Standard: The teacher sets developmentally appropriate, challenging, varied learning goals and objectives that are derived from the District, State or National Content Standards and/or Common Curriculum Goals.


Unit Plan Rationale

Purpose: To provide written documentation of the planning used to create a coherent unit.Standard: The teacher demonstrates the ability to plan a unit that connects learning goals and objectives, methods of teaching and assessment and the knowledge of students and their prior knowledge.


Work Sample Lesson Plans

Purpose: To communicate in detail what the teacher will do and what the teacher will have students do for a specific period of time to meet the learning objective.Standard: The teacher develops lessons for a unit that includes activities using a variety of instruction and assessment strategies that help students meet learning outcomes.


Assessment Purpose: Assessment results, formative,

summative and pre/post summative, help determine learning relative to the expected outcomes and influence planning and instruction. Standard: The teacher demonstrates the ability to design valid assessments and analyze assessment data to determine student learning relative to expected outcomes.


Candidate Reflection

Purpose:To think about and communicate in writing insights into your teaching and learning during your field experience to help you improve your teaching and student achievement. Standard: The teacher thoughtfully reflects about the relationship between their own teaching and student learning.


Reflection Checklist

• Reflects upon experiences and shows a developing understanding of the total teaching experience by identifying at least 2 ways to improve teaching

• Identifies successful and unsuccessful goals and objectives, lessons, experiences, activities and assessments and


Reflection Checklist

• Discusses what contributed to what went well, what was learned, and what could have been done differently to improve teaching and improve student learning.

• Refers to own philosophy of education and educational research and theory if/when appropriate.

Lessons Learned


Separate TWS evaluation of content from format

• Typed,neat, on a CD and binder, and on time. • Written clearly and concisely with appropriate

professional language free of errors. • Cover page includes name, unit title, and dates

taught. • All required components are included. • Organized in a coherent and logical fashion that

provides clear indications of the various sections.• If not met work sample is returned to make changes.


When scoring is too holistic…Table XX: Academic and FSFE Competency Evaluations: Selected Items

Special Education Program: SpEd II and SpEd II EI/ECSE from Spring 2004 and Spring 2005Mean Scores and Standard Deviations

3.00 3.00 3.00 3.00 3.003.00 3.00 3.00 3.00 3.003.00 3.00 2.94 3.00 2.943.00 3.00 2.94 3.00 3.00

0

1

2

3

Engaged Students inLearning Activities

Engaged Students atIntellectual Level

Engaged Students at InterestLevel

Provided PositiveReinforcement

Selects Techniques forCognitive Level

(Acad: 4.2 and FSFE 4.3) (Acad: 4.3 and FSFE 4.2) (Acad: 2.2 and FSFE 2.1) (Acad: 4.5 and FSFE 4.5) (Acad: 1.8.4 and FSFE: 2.3)

Academic Sp04 FSFE Sp04

Academic Sp05 FSFE Sp05Scores on Competency Evaluations Gates: Academic Term & FSFE Term

n values based on selected teacher candidates

0 = Not included 2 = Emerging/Included but Needs Work 1 = Incomplete 3 = Acceptable/Adequate

n = 8

n = 21 n = 20

n = 12

(.00) (.00) (.00) (.00) (.00) (.24) (.00) (.00) (.00) (.24)(.00) (.00) (.00) (.00) (.00) (.24) (.00) (.00) (.00) (.00)


Explanation of Scoring for Term 3 Teacher Work Sample

I. Met/Not Met Scores: Passing total score for Term is between 95 and 111.

TWS Component Exceeds Met Partially Met Not Met Setting 11-12 9-10 Below 9 Goals and Objectives 15-18 12-14 Below 12 Unit plan Rationale 18-21 14-17 Below 14 Lesson Plans 16-18 13-15 Below 13 Assessment Plan and Instruments

9-12 7-8 Below 7

Assessment Analysis 11-12 or **13-15 8-10 Below 8 or **10 Proficiency Analysis & Reflection

8-9 6-7 Below 6

Overall Reflection 7-9 5-6 Below 5 Total Possible 111 pts

106-111 **109-114

95-105 **97-108

76-94 Below 76

Note: Exceeds refers to final point total only.

II. Met/Not met Criteria: Throughout the scoring guides there are some criteria that are simply designated as met/not met. These must all be met to receive a passing score. III. A passing score cannot include a score of 1 or 0 for any work sample criteria. IV. One partially met is allowed. IV. Evidence of successful Literacy integration is a requirement to pass.


Example: Assessment

• Developing: creates pre and post assessment items generally aligned with goals and objectives and uses more that one type of assessment tool. Assessment adequately reflects the learning outcomes planned (1/2 to ¾ agreement) About ½ to ¾ of the questions/tasks are free of ambiguity. Good (adequate) alternative variety in assessment strategies.


Example: Assessment

• Proficient: creates pre and post assessment items clearly aligned with goals and objectives and uses more than one type of assessment tool. Assessment adequately reflects the learning outcomes planned (3/4 to full agreement). More than ¾ of the questions/tasks are free of ambiguity. Excellent variety in alternative assessment instruments.


Multi-purposing of TWSM

• Areas that TWSM can help assess– Some elements of content knowledge– Some elements of content pedagogy– Some elements of professional and

pedagogical knowledge and skills– Student learning


Example: Pedagogical Content Knowledge

RATIONALE:• Provides a conceptual overview of this

topic. • Justifies instructional decisions based

on the prior knowledge of the students, explains the sequence of lessons, and discusses the variety of methods used and why.



GOALS & OBJECTIVES:• Selects developmentally appropriate

goals that are clearly written, and derived from the State Content Standards and/or Common Curriculum Goals.

• Includes Objectives that are aligned with State, District and/or Common Curriculum Goals.



LESSON PLANS:

• Lesson plans are detailed and clearly aligned with objectives.

• Designs lessons that represent a cohesive unit rather than a collection of activities.


Meeting the Needs of ALL Students

Candidates are asked to report on specific levels of student learning by IEP, Title I, and LEP status or achievement quartiles.

Some use NCLB sub-populations.


Helping Candidates Become More Effective Developers and Users of Assessment

Ability to:• Adjust instructional pace, • Change the sequence of skill

development,• Select alternative modes of presentation,

revisit a previously taught concept, or• Reconsider a means of motivating certain

students


How do we assess this in TWSM prompts?

• Why they selected a particular pre- and post-assessment,

• How they determined criteria and scoring for each part of an assessment,

• What adjustments were made to a post-assessment based on analysis of pre-assessment results, and

• What flaws they discovered in the assessment instrument with ways to improve its usefulness in the future.


TWS as a Reflective Tool for Faculty

No longer is it good enough for teacher education faculty at accredited institutions to say, “I taught the material.”


An Example• Ana may demonstrate successful

conceptual understanding of course objectives on 3 to 6 assessments in the Adolescent Psychology class

• But what if during student teaching, the candidate is unable to translate key principles of developmental psychology into effective instructional decisions useful in her 7th grade class.


TWS as a Basis for Reflective Learning Community Dialogue

• Candidate with pk-12 Students• Candidate with Candidates• Candidate with Mentor Teacher• Candidates with Faculty• Faculty with Mentor Teachers• Faculty to Faculty


COE Dialogue

TWS allows for the collection and aggregating of data to show that we have taught so that candidates learn.– Individual attainment of proficiencies– Program accountability– Program improvement– Context for research (ORCI)


Limited Capacity of TWSM

Can candidates:

• Work with families?

• Integrate literacy?

• Differentiate for 2nd language learners?

• Integrate technology?

• Demonstrate cultural competence?


Promising Practices

• Content Specific Pedagogical Rubrics– NAEYC, Science, NCTM, Literacy, CEC

• Standards-based Tutorials in Teacher Preparation

• Student Learning as a Context for Continued Professional Development – New Teacher Center


Oregon Literacy content standards: Are there Oregon English/Language Arts Content Standards or other Content area specific standards included as goals for the unit? It is possible for literacy to be integrated without directly addressing the Oregon standards? We want teacher candidates to understand that the English/Language Arts Standards are important for them to use in their planning and teaching as are literacy related standards addressed in other content areas. A Level 3 score indicates that the teacher candidate is able to use the literacy content standards in their teaching either through a focus on one standard in depth or by using several standards in a meaningful way. Objectives: Is the integration of literacy merely an exercise in lower level recall/comprehension with outcomes that are simplistic and non-challenging? Are performance levels all at a simple level or do they include some higher level outcomes that will allow students to achieve the standard? Note that scoring a zero for objectives also means a score of zero for formative and pre/post assessments. Literacy Dimensions (media, visual, spatial, graphic, reading, writing, speaking, and listening): Which dimension(s) of literacy are included in the unit? Note that it is possible for literacy dimensions to be integrated into the unit without specifically including an Oregon English/Language Arts Content Standard (see above). TWS Rationale: In the rationale, teacher candidates must justify instructional decisions and explain how and why literacy is integrated. A level 3 score shows there is clear evidence in the rationale that the learning of content and performance improvement is directly supported by literacy. A Level 2 score shows there is some evidence in the rationale that literacy is integrated and there is justification for the need for a particular dimension, activity, assignment or objective as it relates to learning of content and performance. A Level 1 score shows little evidence in the rationale that clearly links literacy integration to student learning of content and performance. Level 1 may also include an explanation as to why literacy is not integrated into the unit. A Level 0 score means there is no explanation in the rationale as to why literacy is not integrated, which in turn indicates no awareness of the significance of literacy to student learning of content and performance. Activities, assignments in lessons: Is the integration of literacy merely an exercise in lower level recall/comprehension with activities or assignments that are not very supportive of student understanding? Are performance levels simplistic? Or are they engaging, challenging, using critical thinking skills, and/or require higher performance levels? Formative Assessment: Is literacy assessed within the lessons or assignments? Are the literacy objectives aligned with the assessments? Are formative literacy assessments used without having literacy objectives (level 1). Pre/post or Summative Assessment: Is literacy assessed on the pre and post assessment? Are the literacy objectives aligned with the assessments? Is there a summative assessment for literacy objectives (e.g. scoring guides) that involve research projects, presentations, or other performance based targets?

Example: Literacy


Examination of candidates’ decision making in the context of authentic learning environments• More than ever teachers are expected to

think and act systematically on the progress students make or don’t make, bringing into sharper focus the types of decision making required of teachers.– In the TWSM, sequencing of lesson plans and

contextual information about their students or the level of understanding of their students’ prior knowledge based on pre-assessment information.


Student Learning as a Context for Professional Development

When teachers start to grapple with tough questions about their teaching such as “who’s learning? why? who’s not? and why not?” they arrive at a reason for learning more and ultimately the true purpose of professional development: improving performance.


Principle 1

An instructional program needs to be aligned with and supportive of what candidates are asked to do, including the documentation and reporting that is required in completing a work sample.


Principle 2

School contexts that model and are supportive of what candidates are asked to do need to be available for practicum and student teaching placements.


Principle 3

A supervision, evaluation, and feedback system needs to be in place that provides guided practice in applying and carrying out the tasks teacher work sampling demands of candidates.


Principle 4

Judgments about a candidate’s effectiveness as a teacher need to take into account the gains in learning made by every student taught.


A moral question

Can a teacher who has invested time and energy in examining the impact of her teaching simply ignore when the specified goals and objectives have not been achieved by some percentage of her students?


How Good is Good Enough?

• What level of PK-12 student learning is considered acceptable for moving a candidate forward?

• Should judgments about a candidate’s effectiveness as a teacher consider gains in learning made by every student taught?

• What about in more challenging contexts?• What impact might these decisions have on

fields experiencing teacher shortages?


Principle 5

Documentation of a candidate’s effectiveness as a teacher needs to be accompanied by observations of practice and descriptions of context, as well as evidence of learning gains by students.


Principle 6

Multiple lines of evidence need to be considered in reaching a recommendation for licensure, only some of which come through teacher work sampling.


Principle 7

Multiple reviewers of evidence need to be involved in preparing a recommendation for a license to teach, only some of whom represent a teacher education faculty.


Principle 8

Evidence needs to be assembled and reported by a teacher education faculty on the confidence that can be placed in all lines of evidence collected through teacher work sampling that inform a licensing decision (the reliability and validity of information used).


The Rocky Path to Reliable and Valid Data

• Scoring calibrations– Scoring Work Days

• Content validity– Assessment Coordinator

• Time, time, time– Data Analysis Review Event

• Aggregation and analysis of data– EDSMART


Principle 9

A conceptual map is needed to help inform and give meaning to candidates regarding the way in which the previous 8 principles inform the TWSM.


A Recursive Process for Candidates

TeacherWork

Sample

Candidates AssessPK-12 Student Learning

Candidates ReviewResults of TWS

Candidates Reflect and Plan Next Steps


Emotional Implications

Helping candidates cope with the frustrating ambiguity that accompanies the sinking realization that they taught their hearts out but some still didn’t get it.


A Provocative Premise in BOTH K-12 and Higher Ed

“As educators more clearly defined and reached common agreements on what students should know and be able to do at various points in their development, the world of teaching moved from a private act to a more public act.”


TWS Resources

www.wou.edu/provost/extprogram/tws_conference/tws2006conf.html

2009 Summer TWS Conference Coming

2nd book in progress

Documents

AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean