Assessing Foreign Language

Pre-Service Teacher Dispositions in Situated

PracticeKristin Hoyt, Ph.D.

Kennesaw State University The Seventh Symposium on Educator

DispositionsCincinnati, Ohio

November 18, 2010 

Presentation Overview1. Background & context of study2. Presentation of study3. Findings & implications4. Discussion5. Questions & feedback

NNSED - Hoyt

Context• The institutional context:–Who is KSU?• Kennesaw State University

–What is the PTEU model?• Professional Teacher Education Unit

–What is our FL Education program like? NNSED - Hoyt

Background• Background (What prompted the

study?)– Personal interest in reflective practice

(Genetics or environment?)– Revisited program curriculum & program

delivery (à la NCATE / SPA criteria)– Conducted small study re. candidates’

philosophy of teaching statements– Puzzled by nature of assignments /

assessments that inspire critical, reflective thinking in candidates to cultivate dispositional developmentNNSED - Hoyt

Theoretical Connections to this Study

• A teaching and learning framework (Oja & Reiman, 2007)

• Practice-based theory (Oja & Reiman, 2007)

• Situated practice (Johnson & Reiman, 2007; Rieman & Peace, 2002)

• Developmental theory (Breese & Nawrocki-Chabin, 2007; Edick, Danielson & Edwards, 2007; Oja & Reiman, 2007)

• Constructivist principles (Oja & Reiman, 2007) NNSED - Hoyt

Study Design• Inductive, qualitative approach used in case study

as a means to explore and interpret dispositional development in pre-service FL teachers

• Effective teacher dispositions, operationally defined & characterized in five domains that represent key situational contexts of clinical practice

• Content analysis of guided, written assignments that elicit critical reflection related to five dispositional domains, as experienced during clinical practice

• Triangulation with questionnaire responses from university supervisors and evaluation reports by collaborating teachersNNSED - Hoyt

Instrument: FLED-TDI• Foreign Language Education-Teacher

Dispositions Instrument: Assessment rubric used to identify incidences in which pre-service teachers make references in their written reflections to five domains of dispositional development

1. exchange with colleagues2. decision-making3. reflective practice4. interaction with P-12 students5. pedagogical reasoning skills

NNSED - Hoyt

Working Definition of Dispositions

• attitudes or habits of mind that mirror some degree of conscious, voluntary control, apart from coercion, are oriented toward the broad goals of well-prepared teachers, and result in behaviors and practices of effective teachers (Diez, 2007; Edick, et al., 2007; Katz, 1993; Reiman & Johnson, 2003)

NNSED - Hoyt

Three Guiding Premises(1) Encountering challenges and grappling with decisions during real-time experience are central in advancing the development of dispositions. (2) Pre-service teachers’ first significant, relative conscious “undertaking of dispositional development” is realized primarily during their school site-based field experiences - a critical time and place to foster attitudinal change. (3) To cultivate dispositional development during this critical window of time, pre-service teachers need to have specific opportunities …

NNSED - Hoyt

Three Guiding Premises (cont.)

To cultivate dispositional development during this critical window of time, pre-service teachers need to have specific opportunities …(a) to advance pre-dispositional growth through specific & focused reflective assignments during their sequence of university coursework (which should be grounded in a framework of dispositional development); (b) to demonstrate adequate progress in pre-dispositional growth; and (c) to respond in personalized and meaningful ways to the varied dynamics of their clinical experience with specific & focused reflective assignments.

NNSED - Hoyt

Guiding Questions1. What do pre-service FL teachers’

guided written reflections say about how they face their clinical field experience and the kind of meaning it has for them?

2. How might assessment of incidences, in which pre-service FL teachers make dispositional references in their written reflections, inform the evaluation of dispositional development?

NNSED - Hoyt

Participants• Began with six pre-service FL teacher

candidates enrolled in (entirely field-based) pre-student teaching course: FLED TOSS (Teaching of Specific Subjects).

• Three candidates articulated to student teaching course the following semester.

• Limited data analysis to those three candidates for TOSS & Student Teaching.

NNSED - Hoyt

Data Sources• Pre-service teacher candidates:– Site Reflections / Journal Reflections–Written reflection on videotaped

teaching event–Outcome narrative statements

• University supervisors:–Mid-term & final Likert scale

questionnaires• Collaborating teachers:–Written evaluations

NNSED - Hoyt

Data Analysis• Using FLED-TDI, identified & coded

references to five domains in three reflective writing assignments

• One assessor for 3 TOSS reflective writing sources; another assessor for 3 Student Teaching reflective writing sources

• Simple tally-count and means for Supervisor Likert ratings in each domain

• Second assessor also identified references to five domains in Collaborating Teacher evaluations

NNSED - Hoyt

Results:Three Reflective Writing

Assignments

NNSED - Hoyt

Outcome Narratives

Site-Journal Reflections

Videotaped Teaching Reflections

Total Incidences

0 20 40 60 80 100 120 140 160 180 200

All Five Dispositional Domains Ad-dressed in Data Sources in TOSS vs.

Student Teaching

Student Teaching TOSS

Number of Incidences

Results:Five Domains in Written

Reflections

NNSED - Hoyt

Exchange with Colleagues

Decision-Making

Reflective Practice

Interaction with P-12 Learners

Pedagogical Reasoning Skills

Total of Five Domains

0 20 40 60 80 100 120 140 160 180 200

Total Incidences Addressing Five Disposi-tional Domains in Three Data Sources dur-

ing TOSS & Student Teaching

Student Teaching TOSS

Results:Supervisor Ratings

NNSED - Hoyt

Exchange with Colleagues

Decision-Making

Reflective Practice

Interaction with P-12 Learners

Pedagogical Reasoning Skills

3.9 4 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9

Means of Supervisor Likert Scale Ratings during TOSS & Student Teaching in Five

Dispositional Domains

TOSS-ST CombinedStudent TeachingTOSS

Results:Collaborating Teacher

Evaluations

NNSED - Hoyt

Exchange with Colleagues

Decision-Making

Reflective Practice

Interaction with P-12 Learners

Pedagogical Reasoning Skills

Total References

0 5 10 15 20 25 30

References to Five Dispositional Domains in Collaborating Teacher Evaluations during

TOSS & Student Teaching

Student Teaching TOSS

General Findings• Reflective writing assignments

generate references to five domains of disposition.

• All five dispositional domains are evidenced in data from pre-service teacher candidates, university supervisors, and collaborating teachers.

• FLED-TDI assessment instrument ‘holds-up” in initial implementation– Accessible to outside assessor and

produced positive resultsNNSED - Hoyt

Specific Findings• Site & Journal Reflections yield the most

references to five domains.• When considering combined TOSS & ST

incidences in written assignments, Reflective Practice is the most-addressed domain – nearly double the references to other four domains.

• Pedagogical Reasoning Skills yields the highest overall rating from Supervisors in both TOSS & ST.

• Collaborating Teacher evaluations of both TOSS & ST yield equal references for Exchange with Colleagues & Interaction with P-12 Learners. These two domains elicited twice as many references as other three domains.

NNSED - Hoyt

Possible Interpretations• Site & Journal Reflections yield the

most references to five domains.– These assignments inspire and invoke

greater critical reflection.• When considering combined TOSS &

ST incidences. Reflective Practice is the most-addressed domain – nearly double the references to other four domains. – This domain is more comprehensive;

extends into other four domains.NNSED - Hoyt

Possible Interpretations (cont.)• Pedagogical Reasoning Skills yields the

highest overall rating from Supervisors in both TOSS & ST.– Viewed / understood by Supervisors as most

linked to instructional delivery / implementation in the classroom setting.

• Collaborating Teacher evaluations of both TOSS & Student Teaching yield equal references for Exchange with Colleagues & Interaction with P-12 Learners. Both of these domains yielded twice as many references as the other three domains.– This is ‘front & foremost’ to the world of the

classroom teacher.NNSED - Hoyt

General Implications

• Continue / repeat study with revisions

• Share results with program faculty• Elicit their input / feedback as

component of program evaluation• Share results with PTEU faculty, via

newly-formed “Dispositions Task Force”

NNSED - Hoyt

Specific Implications• Revise reflective writing assignments

that yielded lower dispositional response

• Expand data sources to strengthen findings with evidence ‘from the field’– Design program-specific (observation)

evaluation form for Collaborating Teachers• Develop CT Orientation & train Supervisors

–Modify Supervisor evaluation instruments (beyond Likert Questionnaire) with focus on five domains

NNSED - Hoyt

Limitations• Variability in participants– Are they representative of the typical

cohort of pre-service FL teachers? (small n)

• Variability in field experience teaching contexts– Collaborating teachers and P-12 learners

• Variability in data analysis – Need for additional ‘qualified’ raters to

establish interrater reliability. NNSED - Hoyt

Limitations (cont.)• Limited data sources– Need other data sources to strengthen

triangulation– Design of some assignments ‘non-

inspirational’• Variability in data sources– Participant ‘engagement’ in assignment

completion varied over time and across assignments

– Collaborating teachers’ use / lack of use of various different forms & amount of narrative feedback

NNSED - Hoyt

Discussion

I invite your comments, questions, feedback.

Thank you!

NNSED - Hoyt

ReferencesBoud, D., & Walker, D. (1998). Promoting reflection

in professional courses: The challenge of context. Studies in Higher Education, 23(2), 191-206.

Breese, L., & Nawrocki-Chabin, R. (2007). Social-cognitive perspective in dispositional development (pp. 31-52). In M. Diez & J. Raths, J. (eds.), Dispositions in teacher education. Charlotte, NC: IAP-Information Age Publishing.

Edick, N., Danielson, L., & Edwards, S. (2007). Dispositions: Defining, aligning and assessing. Academic Leadership, 4(4). Accessed online at: http://www.academicleadership.org/emprical_research/Dispositions_Defining_Aligning_and_Assessing.shtml. NNSED - Hoyt

References (cont.)Gore, J., & Zeichner, K. (1991). Action research and

reflective teaching in preservice teacher education: A case study from the United States. Teaching and Teacher Education, 7(2), 119-136.

Hoyt, K., & Bernardy, A. (2008). Fostering dispositions in teacher candidates through a written teaching philosophy. Orlando, FL: Session presentation at the ACTFL Annual Meeting.

Johnson, L. E. & Reiman, A. J. (2007). Beginning teacher disposition: Examining the moral/ethical domain. Teaching and Teacher Education, 23, 676-687.

Johnson, L. E. (2008). Teacher candidate disposition: Moral judgment or regurgitation? Journal of Moral Education, 37(4), 429-222.NNSED - Hoyt

References (cont.)Katz, L. G., & Raths J. D. (1985). Dispositions as

goals for teacher education. Teaching and Teacher Education, 1(4), 301-307.

Oja, S. N. & Reiman, A. J. (2007). A constructive-developmental perspective (pp. 91-116). In M. Diez & J. Raths, J. (eds.), Dispositions in teacher education. Charlotte, NC: IAP-Information Age Publishing.

Reiman, A. J. & DeAngelis Peace, A. (2002). Promoting teachers’ moral reasoning and collaborative inquiry performance: A developmental role-taking and guided inquiry study. Journal of Moral Education, 31(1), 51-66.

Wasicsko, Mark M., Callahan, C. J., & Wirtz, P. (2004). Integrating dispositions into the conceptual framework: Four a priori questions. KCA Journal, 23(1), pp. 1-6.

NNSED - Hoyt

Contact Information

Kristin Hoyt, Ph.D.Kennesaw State University

Department of Foreign Languages1000 Chastain Road, Box 1804

Kennesaw, GA khoyt3@kennesaw.edu

NNSED - Hoyt