14
Establishing a portfolio assessment framework for pre-service teachers: a multiple perspectives approach Maria K. Denney a *, Jeanne M. Grier b and Merilyn Buchanan b a School of Special Education, School Psychology, & Early Childhood Studies, University of Florida-College of Education, P.O. Box 117050, Gainesville FL 32611-7050, USA; b School of Education, California State University Channel Islands, One University Drive, Camarillo CA 93012, USA (Received 13 March 2009; final version received 12 September 2011) In the field of initial teacher training, portfolios are widely used to assess pre- service teachers’ performance as well as the outcomes of university-based teacher preparation programmes. However, little is known about the explicit design of portfolio assessment mechanisms in teacher preparation programmes. Issues related to the design and validation of portfolios for pre-service teacher assessment are a critical area of inquiry for the field of initial teacher training. In this study, perspectives were elicited from school trainee teachers and faculty from a secondary teacher preparation programme to examine the relationships identified among core competencies of a portfolio assessment framework and pre- service candidate learning outcomes. Comparative findings are presented about the relationships identified by the trainee teachers and faculty across the secondary education programme’s portfolio core competencies and trainee learning outcomes. A discussion of the findings is presented with implications and future directions highlighted. Keywords: portfolio assessment; secondary teacher preparation; pre-service teacher education; professional development Over the past decade, a number of countries have developed and implemented standards-based professional learning systems that focus on the ongoing process of learning to teach. The standards apply to the professional practice of teachers from teacher preparation, through induction years, the beginning teaching phase, to ongoing, continued career development stages (Coolahan 2002; Harrison 2007). The various sets of standards are described in detail, and have been adopted and implemented by the agencies that monitor and assess teachers throughout their career paths. As teachers work toward attaining the expected standards of professional practice, they are called on to provide evidence of and reflect on the deepening and expansion of their knowledge and skills as they become effective teachers, and to determine areas for their growth and development. However, what comprises the record of evidence that teachers accumulate, how that evidence is best maintained and presented and what learning the teachers perceive they gain as they work to meet each standard is left without guidance. This research presents one framework for the collection of evidence of performance and investigated *Corresponding author. Email: [email protected] Teaching in Higher Education Vol. 17, No. 4, August 2012, 425437 ISSN 1356-2517 print/ISSN 1470-1294 online # 2012 Taylor & Francis http://dx.doi.org/10.1080/13562517.2011.640997 http://www.tandfonline.com

78236966

Embed Size (px)

DESCRIPTION

test

Citation preview

Page 1: 78236966

Establishing a portfolio assessment framework for pre-service teachers:a multiple perspectives approach

Maria K. Denneya*, Jeanne M. Grierb and Merilyn Buchananb

aSchool of Special Education, School Psychology, & Early Childhood Studies, University ofFlorida-College of Education, P.O. Box 117050, Gainesville FL 32611-7050, USA; bSchool ofEducation, California State University Channel Islands, One University Drive, Camarillo CA93012, USA

(Received 13 March 2009; final version received 12 September 2011)

In the field of initial teacher training, portfolios are widely used to assess pre-service teachers’ performance as well as the outcomes of university-based teacherpreparation programmes. However, little is known about the explicit design ofportfolio assessment mechanisms in teacher preparation programmes. Issuesrelated to the design and validation of portfolios for pre-service teacherassessment are a critical area of inquiry for the field of initial teacher training.In this study, perspectives were elicited from school trainee teachers and facultyfrom a secondary teacher preparation programme to examine the relationshipsidentified among core competencies of a portfolio assessment framework and pre-service candidate learning outcomes. Comparative findings are presented aboutthe relationships identified by the trainee teachers and faculty across thesecondary education programme’s portfolio core competencies and traineelearning outcomes. A discussion of the findings is presented with implicationsand future directions highlighted.

Keywords: portfolio assessment; secondary teacher preparation; pre-serviceteacher education; professional development

Over the past decade, a number of countries have developed and implemented

standards-based professional learning systems that focus on the ongoing process of

learning to teach. The standards apply to the professional practice of teachers from

teacher preparation, through induction years, the beginning teaching phase, to

ongoing, continued career development stages (Coolahan 2002; Harrison 2007). The

various sets of standards are described in detail, and have been adopted and

implemented by the agencies that monitor and assess teachers throughout their

career paths. As teachers work toward attaining the expected standards of

professional practice, they are called on to provide evidence of and reflect on the

deepening and expansion of their knowledge and skills as they become effective

teachers, and to determine areas for their growth and development. However, what

comprises the record of evidence that teachers accumulate, how that evidence is best

maintained and presented and what learning the teachers perceive they gain as they

work to meet each standard is left without guidance. This research presents

one framework for the collection of evidence of performance and investigated

*Corresponding author. Email: [email protected]

Teaching in Higher Education

Vol. 17, No. 4, August 2012, 425�437

ISSN 1356-2517 print/ISSN 1470-1294 online

# 2012 Taylor & Francis

http://dx.doi.org/10.1080/13562517.2011.640997

http://www.tandfonline.com

Page 2: 78236966

what pre-service teachers and their university tutors perceived to be the learning that

took place as they met the California state standards for the teaching profession.

Professional standards for teachers in England, the USA, Canada and several

Australian states are based around the belief that quality teaching is critical to

improve students’ learning outcomes, with the standards focusing attention on

the processes, purposes and efforts employed by successful teachers. While thedescriptors and organisation of national and/or state standards differ, they share an

attempt to characterise what effective teachers know and are able to do in a way that

reflects the complexity of teachers’ knowledge and skills combined with a set of

professional dispositions or attributes. Beneath the variations in the descriptors,

there are common domains that include: (1) foundational knowledge and under-

standing of the processes of learning and teaching; (2) professional skills and

competencies as evidenced in teaching practice; and (3) professional dispositions and

values.

Standards for the teaching profession, such as those of other professional bodies

and organisations, provide a framework for a teacher’s career development and

clarify what appropriate professional progression looks like. To progress through the

career stages a teacher demonstrates that standards have been met. Standards also

establish a system for controlling and maintaining quality. Jasman and Barrera

(1998) suggest that professionalism is enhanced when certification for teaching is

contingent on meeting professional standards. It is proposed that the value ofprofessional standards is that teachers gain control in managing their professional

learning (Darling-Hammond 2001). Furthermore, it is asserted that standards might

assist teachers in developing their own professional growth plans and support

ongoing professional learning (Mayer et al. 2002). Ingvarson and Kleinhenz (2003)

identify another value of teacher standards that stems from self-assessment of

teaching performance, as well as feedback from others. The process for assessment

varies depending on both the career phase of the teacher and the standards

concerned.

Decisions about teachers’ competence related to the complexities of the work

frequently rest with more than one assessor and standards need to be written with

clarity and in a way that makes it possible for by several parties to use them

(Ingvarson and Kleinhenz 2003). Teacher preparation providers are held accountable

for assessing pre-service teachers’ levels of attainment of the professional teaching

performance standards, which in turn determines if institutions of higher education

(IHE) can recommend or confer qualified teacher status on the trainee teachers. While

assessment of academic learning is typically the responsibility of university coursetutors, assessment of instructional performance is shared with mentor teachers at the

placement school. Assessment against the standards is, therefore, a matter of skilled

professional judgement made at different times and in different situations, drawing on

evidence collected over time from a number of sources (Training and Development

Agency for Schools 2011). Because of the range of interpretations this gives rise to, it

is asserted that a common understanding of standards be developed across multiple

educational constituents, including school-based tutors, higher education tutors and

teacher trainees (Training and Development Agency for Schools 2011). Likewise, to

ensure clarity about what constitutes a passing standard a scoring system grounded in

benchmark examples, the development of effective evaluation methods for assessing

trainee teachers’ performances is recommended (Ingvarson 2002).

426 M.K. Denney et al.

Page 3: 78236966

Yet, despite cautionary notes and recommendations, the challenge for assessors

remains how to measure the standards, how to implement assessment procedures,

and what to consider as evidence that the standards have been met, and in a way that

is fair, valid and reliable across IHEs and school sites. To better ensure equitable

assessment, rules need to be specified for gathering evidence about practice

(Ingvarson 2002). Keeping records of evidence that demonstrate trainee teachers

are attaining and appropriately applying professional knowledge and skills, and

possess the professional attributes or dispositions that define effective teachers is

called for by the various authorities that wrote the standards. It is anticipated that

drawing trainees’ attention to how and if they are meeting the expected standards of

professional practice will create the opportunity for reflection on the knowledge and

skills they are developing as they become effective teachers, and help determine areas

for professional growth and development.

Portfolios are one mechanism to provide direct evidence of student work and

classroom activity by showing the outcomes of the learning environment planned by

the teacher. Portfolios have been developed and used as an assessment tool in initial

teacher training in recent decades and for multiple purposes and in multiple contexts

(Ledoux & McHenry 2006; Zeichner and Wray 2001). Showcase portfolios are useful

for beginning teachers in search of employment and during job interviews. Portfolios

can be summative�an accumulation of evidence that supports decisions regarding

the alignment of teacher candidate performance, teacher education programme goals

and initial teacher certification (Ledoux and McHenry 2006). Portfolios can also

serve as a formative measure that provides teacher candidates opportunities to select

materials and reflect on their learning to document their growth during the

programme (Zeichner and Wray 2001). Research on the use of portfolios in pre-

service teacher preparation programmes internationally reveals that portfolios are a

medium through which student teachers reflect on their professional development

(Groom and Maunonen-Eskelinen 2006; Loughran and Corrigan 1995). Common to

many of the portfolio models is the collection of trainees’ artifacts, their reflections

and criteria for assessing how these artifacts demonstrate learning outcomes in pre-

service teacher preparation programmes (Dysthe and Engelsen 2004).

In a US study of the preparation of teachers of students with behavioral

disorders, Bloom and Bacon (1995) found that portfolios assisted the students with

self-reflection and assessment of their development as a professional educator. The

authors found that the portfolios provided assistance to the student teachers in the

development of a wide range of skills to aid in decision-making, problem solving and

establishing a connection with the teaching profession. The portfolio process also

gave the student teachers more self-confidence in their abilities as future educators

through the responsibility imposed and control over their own work. The students

were energetic and enthusiastic about having the opportunity for choice in the

development of their portfolios, which led to addressing complex issues in teaching

instead of writing for faculty approval. Some disadvantages included students’

apprehension with the evaluation process, as well as the labour intensiveness of

the assessment process for the faculty. The authors suggested that these issues might

be managed by the active involvement of participating students and faculty in the

portfolio process, as well as the articulation of clear expectations about the

assessment process and required products.

Teaching in Higher Education 427

Page 4: 78236966

In a study of the use of portfolios as a tool for promoting teaching, Klenowski

(2000) focused on the extent to which portfolios supported the development of

reflective practice and teaching skills among students and lecturers in secondary

education. It was found that the use of portfolios not only profoundly impacted the

pedagogic practice among the pre-service teachers, but also among the lecturers. It

was reported that initially the students did not understand the value of the portfolio;

they viewed it as a data filing system. Over time, the assessment procedures and

explanations became clearer; the students began to appreciate the reflective aspects

of the portfolio process. Portfolio exemplars were provided, as well as more specific

evaluation criteria. Lecturers also gained from embedding the portfolio process into

their courses. Changes in their teaching styles were noted through the integration of

reflective practice into their teaching and the connection to their own philosophy of

teaching.

Despite the increase in use of portfolios, there is limited evidence about the

validity of these instruments in initial teacher training (Burns and Haight 2005). As

with traditional paper and pencil measures, it is essential that portfolio assessment

instruments demonstrate similar psychometric standards, especially at a time of

increasing scrutiny and accountability of teacher preparation programmes. In their

discussion about the validation of performance-based assessments, Miller and Linn

(2000) asserted that the criteria for examining validity vary greatly in a high-stakes

educational environment. As the field of initial teacher training grapples with how

best to measure teacher candidate competence�whether through performance-based

assessment, traditional multiple choice measures, classroom observation or portfo-

lios, it is argued that assessment methods need to accurately capture the teaching and

learning process.

As a basis for establishing initial validity of a portfolio framework, this study

focused on the multiple perspectives of secondary education teacher candidates and

faculty to examine how closely the portfolio’s core competencies were aligned with a

set of identified teacher candidate learning outcomes. The primary research

questions guiding this study were: (1) What relationships do trainee teachers and

university faculty members identify among the core competencies demonstrated

through the portfolio framework and candidates’ learning outcomes in the secondary

education programme? and (2) What are the perceived similarities and differences

among the teacher candidates and university faculty perceptions about the required

core competencies and the candidates’ learning outcomes in the portfolio frame-

work?

Method

Research site and participants

This study took place at a public university in Southern California with a total full-

time equivalent enrolment of 3000 students. The university’s initial teacher training

programme constitutes one of the largest campus units with approximately 800

teacher candidates enrolled annually in eight different programmes. Twenty-three

teacher candidates and six programme faculty members gave their informed consent

to participate in the study. The recruitment criteria for this study allowed for the

participation of both first- and second-semester trainee teachers who were enrolled in

428 M.K. Denney et al.

Page 5: 78236966

the university’s secondary education programme. Fifty-two percent of the teacher

candidates were male and 48% were female. The chronological age of the total

sample ranged from 23 to 49 years, with a mean age of 33 (SD �9.5). One hundred

percent of the sample held a held a bachelor degree and 10% earned a master degreebefore entering the secondary teacher preparation programme. As for the single

subject content areas pursued by the teacher candidates, English accounted for the

highest representation (47%), followed by science (29%) and mathematics (24%).

The university faculty represented the subject areas of English, science and

mathematics. The characteristics of the six participating faculty members were as

follows: a student teaching university tutor who did not teach courses in the teacher

training programme; three part time lecturers who taught one course a year; one

tenure-track faculty member who taught one course per semester, and one tenure-track faculty member who taught three courses per semester and was the secondary

teacher education programme coordinator.

Procedure

The teacher candidates and programme faculty were recruited directly from the

university’s secondary education programme. Participants were provided with an oral

and written explanation about the research study. Of the total number of teacher

candidates who were invited to participate, 100% of these candidates agreed to be

included in the study. Of the seven programme faculty who were invited, six faculty

members consented to participate. Written human subject consents were obtainedfrom the participants. The candidates and faculty were asked to complete a written

matching task on one occasion. The matching task took the participants between 10

and 15 minutes to complete.

The matching task

The teacher candidates and faculty members were introduced to a matrix of the

portfolio’s core competencies (i.e. learning environment; instructional process;

learning about students; and professionalism) and a set of candidate learning

outcomes (i.e. actively engage students in their learning) (see Table 1). It should be

noted faculty created the portfolio core competencies across the teacher preparation

programmes for the elementary and secondary education candidate portfolios.The candidate learning outcomes were also identified by the initial teacher

training faculty across the university’s elementary and secondary education teacher

Table 1 Candidate learning outcomes.

Teach all subjects in their area of specialisation

Teach students with English as a first or second language

Relate to the diversity of languages and cultures among students and their families

Meet the diverse needs of all students including those with special needs

Be reflective and deliberative practitioners

Link content and pedagogy

Actively engage students in their learning

Integrate research, theory and best educational practice into their teaching

Teaching in Higher Education 429

Page 6: 78236966

preparation programmes. The participants were asked to individually match the four

portfolio core competencies with each associated candidate learning outcome. For

each portfolio core competency, a total of eight possible associated candidate

learning outcomes could be identified. Each response to the matching task wasscored with a binary scale from 1 (i.e. relation identified) to 2 (i.e. no relation

identified).

Results

This study was designed to examine the perspectives of teacher candidates and

faculty about the relationships among the portfolio’s core competencies and

associated candidate learning outcomes as a basis for establishing initial validity of

the secondary education credential programme’s portfolio framework. It was of

particular interest to examine if the relationships noted among the portfolio’s corecompetencies and candidate learning outcomes were similar or different between the

teacher candidates and programme faculty.

Perspectives of teacher candidates

Table 2 presents a frequency distribution of the teacher candidates’ and faculty

responses to the matching task of the relationships among the portfolio’s core

competencies and the candidate learning outcomes. The teacher candidates were

asked to individually match the four portfolio core competencies (i.e. learning

environment; instructional process; learning about students; and professionalism)with a total of eight possible candidate learning outcomes for each of the related

portfolio’s core competencies.

Of the four portfolio core competencies, instructional process yielded the highest

percentages of affirmative relationships (e.g. range from 70% to 100%) identified for

the entire candidate learning outcomes. The portfolio core competency, learning

environment followed with the next highest percentages with a range from 74% to

96% for affirmative relationships identified for five of the eight candidate learning

outcomes. Of the four portfolio core competencies, learning about students yieldedthe lowest associations with a range from 17% to 30% for four of the eight candidate

learning outcomes. The candidate learning outcome, teach all subjects in their area of

specialization, showed the lowest relation of 17% with the core competency learning

about students.

Perspectives of faculty

Table 2 presents a frequency distribution of the faculty’s responses to the matching

task. As found with the teacher candidates’ data, the core competency instructional

process yielded the highest percentages (e.g. range from 83% to 100%) for seven ofthe eight candidate learning outcomes. The core competency, learning about students

showed the next highest percentages (e.g. range from 83% to 100%) of relationships

with four of the eight candidate learning outcomes. This finding was in direct

contrast to the teacher candidates’ data in which learning about students had the

lowest associations with the candidate learning outcomes. Finally, of the four core

430 M.K. Denney et al.

Page 7: 78236966

Table 2 Portfolio core competencies and candidate learning outcomes.

Trainee

teachers

(n�23)

Faculty

(n�6)

Yes

(%)

No

(%)

Yes

(%)

No

(%)

Learning environment

Teach all subjects in their area of specialisation 43 57 33 67

Teach students with English as a first or second language 96 4 67 33

Relate to the diversity of languages and cultures among

students and their families

96 4 100 0

Meet the diverse needs of all students including those with

special needs

91 9 83 17

Be reflective and deliberative practitioners 39 61 33 67

Link content and pedagogy 74 26 33 67

Actively engage students in their learning 87 13 83 17

Integrate research, theory and best educational practice into

their teaching

61 39 50 50

Instructional process

Teach all subjects in their area of specialisation 91 9 100 0

Teach students with English as a first or second language 78 22 100 0

Relate to the diversity of languages and cultures among

students and their families

70 30 83 17

Meet the diverse needs of all students including those with

special needs

87 13 100 0

Be reflective and deliberative practitioners 91 9 67 33

Link content and pedagogy 100 0 100 0

Actively engage students in their learning 96 4 100 0

Integrate research, theory and best educational practice into

their teaching

96 4 100 0

Learning about students

Teach all subjects in their area of specialisation 17 83 33 67

Teach students with English as a first or second language 100 0 83 17

Relate to the diversity of languages and cultures among

students and their families

100 0 100 0

Meet the diverse needs of all students including those with

special needs

96 4 100 0

Be reflective and deliberative practitioners 26 74 67 33

Link content and pedagogy 30 70 33 67

Actively engage students in their learning 87 13 83 17

Integrate research, theory and best educational practice into

their teaching

30 70 67 33

Professionalism

Teach all subjects in their area of specialisation 83 17 83 17

Teach students with English as a first or second language 48 52 17 83

Relate to the diversity of languages and cultures among

students and their families

57 43 33 67

Teaching in Higher Education 431

Page 8: 78236966

competencies, professionalism revealed the lowest relationships with five of the eight

candidate learning outcomes.

Similarities and differences among teacher candidates and faculty

It was of particular interest to consider if the relationships noted among the

portfolio’s core competencies and candidate learning outcomes were similar or

different between the teacher candidates and faculty. A cut-off percentage range was

used to calculate low (e.g. 0�39%), moderate (e.g. 40�69%) and high (e.g. 70�100%)agreements within each of the four core competencies and candidate learning

outcomes across the teacher candidate and faculty data (Table 2). The highest

agreement across the teacher candidates and faculty was found for the candidate

learning outcomes within the core competencies, instructional process and learning

about students. The most disagreements across the teacher candidates and faculty

were noted for the candidate learning outcomes within the core competencies,

professionalism and learning environment.

Discussion

As a basis for establishing a portfolio assessment framework, the multiple

perspectives of secondary education teacher candidates and faculty were elicited

order to examine how closely the portfolio’s core competencies aligned with a set of

candidate learning outcomes. Similarities and differences between the teacher

candidates’ and the faculty’s perspectives about the relationships among the core

competencies and candidate learning outcomes were of specific interest in this study.

Some interesting findings emerged.

Perspectives of teacher candidates

The teacher candidates, as a group, were very decided in their responses to the

matching task, in fact, very few (6 out of 32 items) were identified as ‘moderate’. As

the ones who have experienced the programme and participated in coursework, their

Table 2 (Continued )

Trainee

teachers

(n�23)

Faculty

(n�6)

Yes

(%)

No

(%)

Yes

(%)

No

(%)

Meet the diverse needs of all students including those with

special needs

83 17 17 83

Be reflective and deliberative practitioners 96 4 100 0

Link content and pedagogy 74 26 33 67

Actively engage students in their learning 48 52 17 83

Integrate research, theory and best educational practice into

their teaching

100 0 83 17

432 M.K. Denney et al.

Page 9: 78236966

ratings of the relationships between the portfolio’s core competencies and the

candidate learning outcomes were a definite yes or no.

Several of the candidate learning outcome results were surprising across the core

competencies from the teacher candidates’ perspectives. The candidate learning

outcome of being a reflective and deliberate practitioner was only deemed strongly

relational (91% and 96%) for the core competencies of instructional process andprofessionalism. It is interesting that the teacher candidates did not perceive that

reflection would benefit them in constructing their learning environment or learning

about students. However, this finding makes sense within the context of the

secondary education programme’s curriculum. The teacher candidates were very

often asked to reflect on their lesson planning and instructional activities. This act is

called a professional responsibility and teacher candidates were not often required to

reflect on other aspects of the teaching process outside of planning.

The candidate learning outcome of integrating research and theory into teaching

follows a very similar pattern as being reflective. The candidates were exposed to the

theory about instructional processes and learning, and this is framed as a

professional activity. However, it was quite surprising that the core competency,

learning about students received a low rating of 30% when so much emphasis was

placed on learning theory in more than one course throughout the secondary

education programme.

The grouping of candidate learning outcomes that addressed teaching students

with English as a first or second language, relating to diversity of languages andcultures, and meeting the needs of all students including those with special needs

were all hallmarks of the secondary teacher training programme. The students rated

these areas very similarly as high across three core competencies: learning

environment; instructional process; and learning about students. However, these

three core competencies were not so similar for the core competency of profession-

alism. It is interesting that teaching students with English as a first or second

language and relating to the diversity of languages was rated as moderate (48% and

57%, respectively) while meeting the diverse needs of students including those with

special needs rated high with 83%. Although, all of these outcomes are considered

the professional responsibility of teachers, one possible explanation for this

discrepancy is that the teacher candidates completing the matching task were all

enrolled in a course entitled Access to Learning: Special Needs Learners. The course

focused on the legal and curricular needs of students with special needs and was

taught by a special education policy specialist.

Finally, it was especially concerning that the teacher candidates did not perceive

the candidate learning outcomes in relationship to the core competency of

professionalism to be more related. Only half of the candidate learning outcomesgenerated a strong relationship rating. The core competency of professionalism was

for two semesters the focus for the secondary education programme faculty to

strengthen their assignments within the curriculum. However, these low relationships

raise questions if these efforts were successful.

Overall, it was gratifying that the teacher candidates had such strong opinions

about the relationships between the candidate learning outcomes and the core

competencies of the programme’s portfolio framework. Their ratings were discrimi-

natory and assisted in the initial design and validation efforts of the secondary

education programme’s portfolio’s framework.

Teaching in Higher Education 433

Page 10: 78236966

Perspectives of faculty

It is difficult to yield reliable faculty findings from only six participating faculty

members. However, with only seven faculty total in the programme, we can state that

our sample was close to a complete population. Regardless of sample size, the faculty

responses on the matching tasks revealed some interesting information.

It was quite surprising that the faculty rated the candidate learning outcome of

being a reflective practitioner as low or moderate in three of the four core portfolio

competencies. Only the core competency, professionalism was rated high with a 100%

agreement. This finding was likely consistent with the overall programme philosophy

that the practice of reflection was important as a professional activity. However, with

the lack of consistent application across the portfolio’s core competencies, this

finding was disconcerting -especially for the core competency of instructional process

that was emphasised throughout the programme’s course- and fieldwork.

In addition, faculty rated the candidate learning outcome of integrating research

and theory into teaching with the following ratings: high in the core competencies of

instructional process (100%) and professionalism (83%); moderate for the core

competencies of learning environment (50%) and learning about students (67%).

These findings suggest that the integration of research and theory into teaching was

highly related to the instructional process. However, the research and theory of

teaching was recognised to a much lesser degree in relationship to the core

competencies of establishing a learning environment and learning about students.

Another interesting finding from the faculty’s perspective about the relationships

between the core competencies and the candidates’ learning outcomes was in the area

of professionalism. The faculty rated only three of the eight candidate learning

outcomes as highly related. Of greater concern were the low ratings for three of the

candidate learning outcomes (e.g. English as a first or second language, meeting the

diverse needs of all students and actively engaging students in their learning).

Overall, these findings imply a combination of factors about the secondary

education programme faculty. The different roles of the six participating faculty and

their level of involvement in the programme might have contributed to the variation

in faculty responses. The familiarity of individual faculty with the programme was

limited to only those aspects of the programme in which they directly participated.

However, by including the multiple faculty perspectives in this study, we hoped that

individual faculty members would take a broader perspective-one outside of their

own familiarity and involvement. These findings suggest that the programme faculty

were not as familiar with the secondary education programme as anticipated due to

their varying levels of participation and involvement.

Similarities and differences among teacher candidates and faculty

In the comparison of candidate learning outcomes as rated either high, moderate, or

low for each of the portfolio’s core competencies, there is much overlap and

agreement between the ratings of teacher candidates and programme faculty. These

strong agreements�both positive and negative�inform our attempt to align the

portfolio core competencies with candidate learning outcomes for the purposes of

further refining and validating the programme’s portfolio assessment framework.

434 M.K. Denney et al.

Page 11: 78236966

In the core competency learning environment, there was agreement of ratings on

four candidate learning outcomes. The faculty and candidates rated three of the

outcomes (i.e. diversity of languages, meeting the diverse needs of all students and

actively engaging students) as high. However, although not in agreement, thecandidate learning outcome of teaching students with English as a first or second

language was very close to having a high rating by both teacher candidates and

faculty members in the programme. In this area, it was almost more telling of what

the two groups labelled as low and not being related to creating and maintaining an

environment for learning. Subject matter knowledge and being reflective were not

rated as important to this core competency. This is quite an interesting finding in that

secondary students and faculty are typically very concerned with the recognition of

their subject matter expertise. The core competency of instructional process wasclearly an area with many similarities of perspective. The candidates rated all eight

learning outcomes as highly related to the instructional process and all faculty

ratings could be considered high if the 67% for being reflective was adjusted due to

the low number of faculty responses. It was very clear from these data that the

instructional process was highly aligned with the entire candidate learning outcomes.

For the core competency of learning about students, the ratings of the teacher

candidates and faculty were similar for all but two of the learning outcomes and

these have been previously discussed. The candidates and faculty rated four of thecandidate learning outcomes as being highly related to learning about students. It is

interesting to note that all four of the learning outcomes identified as high each have

the word student in the phrase.

The core competency of professionalism has been discussed previously for both

candidates and faculty. However, each of the findings informs each other in a quid

pro quo manner. Meaning, if the faculty members do not recognise these candidate

learning outcomes as professional then why should the teacher candidates? Likewise,

it is encouraging that the teacher candidates identified the connections between theircandidate learning outcomes and the core competency of professionalism. They must

have synthesised these relationships themselves even though these were not explicit in

the programme. This will be an area of definite exploration for the future.

Limitations and conclusions

There were a couple of limitations with this study and more research is needed to

extend the generalisation of the findings. The study was conducted at one teacherpreparation programme. The replication of this study with other teacher preparation

programmes internationally might potentially yield different results. The second

limitation was the small sample size of faculty participants in this study. The increase

in the faculty sample size would have been beneficial to match the larger sample of

teacher candidates in the study.

Through a matching task that was completed by secondary education teacher

candidates and faculty, an initial alignment and framework for portfolio validity was

established. Although it was never expected that each portfolio core competencyaddressed every candidate learning outcome, it was expected that every candidate

learning outcome would find a home within at least one core competency. This

indeed was found to be true. However, for the faculty who designed the secondary

education programme and the portfolio structure, it was initially intended that the

Teaching in Higher Education 435

Page 12: 78236966

constructs of reflective and deliberate best practice, and the integration of research

and theory were at least highly related to all four of the portfolio’s core competencies.

However, these connections were not found. Further investigations are needed to

explore these discrepancies both from the perspectives of teacher candidates and

programme faculty within the secondary education programme studied. The

portfolio is a powerful data point in our initial teacher training assessment system.

However, it is not the only factor used when considering recommending teacher

candidates for credentials. It might be prudent for teacher preparation programmes

to consider other assessment methods such as qualitative interviews in addition to

portfolios in order to capture the self-reflections of teacher candidates about their

learning. It is suggested that teacher preparation programmes rely on multiple valid

measures to ensure that candidates are not only prepared, but also qualified to teach

children and youth.

Among the implications for teacher preparation programmes when considering

the use of and implementation of portfolios is the need to be explicit with

programme faculty and students in a few areas. First, it would be important for

all programme faculty to meet regularly about the implementation of portfolios in

order for these faculty to become familiar with all aspects of the programme

curricula beyond the possible one course they might teach within the programme.

Second, it would be beneficial for the intended learning outcomes of the programme

to be made explicit to the students across all courses. It would also be valuable if

teacher preparation programmes would place a greater emphasis on the connections

across reflection, theory, and research. As was found in this study, graduates of

secondary teacher preparation programmes need not only demonstrate specific

competencies, but also be aware of how these are linked to reflective teaching

practices. In the field of initial teacher training, the use of portfolios is widely used to

assess pre-service teacher candidates’ performance as well as, the outcomes of

university-based teacher preparation programmes. The lessons learned from the

confounding findings of this study demonstrate the importance of establishing

validation processes such as a shared understanding (i.e. across faculty and teacher

candidates) about the purpose a portfolio assessment framework and pre-service

candidate learning outcomes when considering this tool as a method of assessment in

the field of initial teacher training. In an era of increasing accountability for teacher

preparation programmes to show how student learning outcomes are linked to

teacher candidate performance, it will be important for programmes to make

informed decisions about how the assessment methods selected accurately reflect the

teaching and learning process�whether through portfolios, classroom observations

and other methods.

References

Bloom, L.A., and E.H. Bacon. 1995. Professional portfolios: An alternative perspective on thepreparation of teachers of students with behavioral disorders. Behavioral Disorders 20, no. 4:290�300.

Burns, M.K., and S.L. Haight. 2005. Psychometric properties and instructional utility ofassessing special education teacher candidate knowledge with portfolios. Initial TeacherTraining and Special Education 28, no. 3/4: 185�94.

Coolahan, J. 2002. Teacher education and the teaching career in an era of lifelong learning.OECD Education Working Papers 2, OECD Publishing, 1�39.

436 M.K. Denney et al.

Page 13: 78236966

Darling-Hammond, L. 2001. Standard setting in teaching: Changes in licensing, certification,and assessment. In Handbook of research on teaching, ed. V. Richardson, 751�76.Washington, DC: American Educational Research Association.

Dysthe, O., and K.S. Engelsen. 2004. Portfolios and assessment in teacher education inNorway: A theory-based discussion of different models in two sites. Assessment &Evaluation in Higher Education 29, no. 2: 239�58.

Groom, B., and I. Maunonen-Eskelinen. 2006. The use of portfolios to develop reflectivepractice in teacher training: A comparative and collaborative approach between two teachertraining providers in the UK and Finland. Teaching in Higher Education 11, no. 3: 291�300.

Harrison, J. 2007. The assessment of ITT standard one, professional values and practice:Measuring performance or what? Journal of Education for Teaching 33, no. 3: 323�40.

Ingvarson, L. 2002. Strengthening the profession? A comparison of recent reforms in the UKand the USA. Teaching Standards and Teacher Evaluation. Australia: Australian Council forEducational Research, Issue 2, 1�16.

Ingvarson, L., and E. Kleinhenz. 2003. A review of standards of practice for beginningteaching. Teaching Standards and Teacher Evaluation. Australia: Australian Council forEducational Research, Issue 4, 1�19.

Jasman, A., and S. Barrera. 1998. Final report: Teacher career structure: Level 3 classroomteachers: Development and implementation of selection processes, professional competencystandards. Perth, Australia: Nexus Strategic Solutions.

Klenowski, V. 2000. Portfolios: Promoting teaching. Assessment in Education 7, no. 2: 215�36.Ledoux, M.W., and N. McHenry. 2006. Electronic portfolio adoption for teacher education

candidates. Early Childhood Education Journal 34, no. 2: 103�16.Loughran, J., and D. Corrigan. 1995. Teaching portfolios: A strategy for developing learning

and teaching in preservice education. Teaching & Initial Teacher Training 11, no. 6: 565�77.Mayer, D., J. Mitchell, D. Macdonald, R. Land, and A. Luke. 2002. Evaluation report of the

education Queensland professional standards for teachers pilot 2002. Brisbane, Australia:University of Queensland.

Miller, M.D., and R.L. Linn. 2000. Validation of performance-based assessments. AppliedPsychological Measurement 24, no. 4: 367�78.

Training and Development Agency for Schools. 2011. Professional standards for teachers forqualified teacher status. London: Training and Development Agency for Schools. http://www.tda.gov.uk/training-provider/itt/qts-standards-itt-requirements/qts-introduction.aspx(accessed June 1, 2011).

Zeichner, K., and S. Wray. 2001. The teaching portfolio in U.S. teacher education programs:What we know and what we need to know. Teaching and Teacher Education 17, no. 5:613�21.

Teaching in Higher Education 437

Page 14: 78236966

Copyright of Teaching in Higher Education is the property of Routledge and its content may not be copied or

emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.

However, users may print, download, or email articles for individual use.