Journal of Personnel Evaluation in Education 5:147-167, 1991 0 1991 Kluwer Academic Publishers, Manufactured in the United States of America
Portfolios for Biology Teacher Assessment
ANGELO COLLINS Florida State University, 203 Carothers, Tallahassee, FL 32306
On a warm, May afternoon, Karen, the high school biology teacher, perched on a stool in the lab reviewing her portfolio. Five years ago her school district had decided to experiment with replacing the annual teacher evaluation by an administrators classroom visit with a one-on-one conference based on the teachers portfolio. During the first year of the experiment, all teacher evaluation had been suspended as the faculty, working alone, in grade level groups, in subject area groups and as a body of the whole, struggled to reach agreement on questions about portfolio development. The first question was, Evidence of what? Other ques- tions were: what documents would provide this evidence; how many documents; what types of documents; and what would be done with the portfolio once it was developed. During the second year, all of the faculty members were busy collecting the documents to provide the evidence for their first portfolios. There had been many conversations among the science teachers about whether this lab was the right one to videotape to display a commitment to student-centered lab work or whether that test really captured ongoing student evaluation. Deeper collegial relationships began to develop as the faculty, focusing on portfolio development, shared their successes and concerns. The faculty wanted this experiment on teacher evaluation to work because the judgment about the quality of teaching would reside among the teachers.
The process of portfolio development had become formalized in a very short time. Karen kept a file drawer of potential documents for evidence-lesson plans, tests, videotapes, student work samples. Attached to each document was a caption. In September she had identified her professional goals for the year and had rehearsed them as she collected and tagged documents. Last week she had assembled the documents that demonstrated her progress toward her goals and arranged them as evidence to tell her story. Yesterday she had met with the other science teachers on the faculty and discussed her portfolio presentation. Now she was reviewing the documents to be submitted in the portfolio. Her chair was writing a letter of attestation to include in the portfolio, which she would deliver to the principal on Friday afternoon. On Monday she and the principal would meet and discuss her portfolio, her ideas for next years goals, and her decision to advance to the rank of Master Teacher after next year.
* * * *
148 A. COLLINS
Of course, Karens story isnt true, but it is a vision of what teacher evaluation might be at some time in the future. Karens story is one of several possible outcomes of the research of BioTAP-the Biology component of the Teacher Assessment Project-conducted at Stanford University from January 1988 to August 1989. The purpose of this article is to report on BioTAP. The article will locate the research of BioTAP in the contexts of educational reform and the entire Teacher Assessment Project, describe how the biology teachers portfolio was designed, developed, and scored, and conclude with results and recommendations.
In 1986, Lee Shulman and a group of colleagues at the Stanford University School of Education initiated the Teacher Assessment Project (TAP) by asking, What do teachers need to know and need to be able to do? How can this knowledge and these skills be assessed? How can a program of assessment be designed that will be adequate to the complexities of teaching while being equitable for all candidates? (Shulman, 1987). These questions framed the research program of TAP as it explored the feasibility of three alternative modes of teacher assessment-simulation exercises, portfolios, and portfolio-based simulations. And these questions were posed in the context of a new educational reform movement in the United States.
The current climate of educational reform is exemplified by the presidential commission report, A Nation at Risk (1983), and the responses to this report. For example, the Carnegie Corporation of New York funded the Forum on Education and the Economy that produced a report, A Nation Prepared (1986). One of the recommendations of this Carnegie report was the formation of a national board for and by teachers to set and maintain high standards of excellence for the teaching profession. The National Board for Professional Teaching Standards (1988) is now a reality. The Carnegie Corporation of New York also funded TAP, a research and development project to explore alternative modes of teacher assessment. While it was intended that the research conducted by TAP inform the deliberations of the National Board, there was never any question that the research would also have an impact on teaching, on state and local education policy, and would contribute to the research on the identification and assessment of the knowledge, skills, and dispositions of exemplary teachers. l
The Teacher Assessment Project at Stanford
From 1986 to 1990, TAP explored three modes of teacher assessment-simulation
BIOLOGY TEACHER ASSESSMENT 149
exercises, portfolios, and portfolio-based simulations-focused on four subject areas-mathematics, history, literacy, and biology-and tested the implications of these new forms of assessment for minority teachers and for teacher education programs (Shulman, Haertel, & Bird, 1988; Shulman, 1990).
Complex. Four assumptions guided the research of TAP. The first was that teaching is a complex task; therefore, the assessment of teachers will require a battery of assessments, some of which will be as complex as teaching itself. No one mode of assessment can be sufficient to capture all the facets of teaching. In the research of TAP there was (and there remains) a dilemma: how to cut up the practice of teaching so that the pieces are large enough to resemble teaching and small enough to focus on the elements of good practice. Attempts to isolate aspects of teaching necessarily reduce fidelity to the act of teaching. However, the credible assessment of teachers must look much like the work of teachers. Portfolios and simulation exercises were modeled on how teachers construct answers to problems that are intrinsic to the act of teaching, like deciding what to teach, or assessing what students know and can do.
Context. A second assumption of TAP was that teaching takes place in a context. One of the elements of the context is the subject matter. Therefore, the assessment procedures that were developed were subject-specific-teaching fractions to fifth graders, the American Revolution as taught in eleventh grade social studies, teaching literacy in third and fourth grade when the instructional emphasis shifts from learning to read to reading to learn, and teaching introductory high school biology, often the first and last course students take in high school science. A second element of context is the students. The assessments provided teachers with opportunities to address how they perform a teaching task for a particular group of students. TAP assumed there can be no one model of teaching because successful teachers adapt their teaching to a context-teaching this topic to these students in this time and place.
Theoretical and practical. A third assumption was that professional teachers have a store of theoretical as well as practical knowledge that supports the decisions they make. Some of this knowledge may have become tacit or routinized through long practice, but this knowledge does exist. The assessments developed were intended to assist teachers in uncovering and explicating their tacit knowledge.
By teachers. A fourth assumption was that the persons who best understand and are qualified to evaluate teaching are teachers. Therefore, all of the work of TAP involved active participation of teachers in the design and implementation of the research.
Other considerations. In addition to these assumptions guiding the research, the
150 A. COLLINS
Project staff attempted to integrate some competing or complementary concerns. One concern is the tension between reforms designed to improve curriculum and instruction and the recognition of the positive aspects of current practice. Therefore, the Project gave weight both to findings from research on teaching and to the views and actions of practicing teachers. While attempting to incorporate desirable advancements of what teaching might be into its assessments, the Project also included reasonable assessments of teaching as it is commonly practiced. Another concern was bias. At the same time that it attempted to embody standards of excellence in its assessments, the Project also tried to avoid potential biases related to ethnicity or gender. The final concern was the recognition that the demands and limitations of a small-scale, low-risk research project would not be isomorphic with the requirements for any large-scale, high-risk implementation of these modes of assessment.
The research of BioTAP
Biology: why and who
The teaching of introductory high school biology was selected as one of the four subject areas in which TAP would concentrate.its efforts. At the secondary school level, where teachers are considered content specialists, biology was selected as the high school science course with the Iargest enrollment, and in contrast to history, which is perceived as having a less problem-solving orientation. Biology instruction was selected also in contrast with the more generalist teaching of elementary mathematics and literacy. The Biology component of TAP explored three modes of assessment, but this report will focus only on the portfolio development aspects of the BioTAP research.
The portfolio development team consisted of the project director, who had experience in teaching high school biology and in research in science education, four university-based research assistants, three of whom had experience as high school biology teachers, and four Bay Area high school biology teachers. The teachers on the development team were nominated by a district or local supervisor, were observed teaching, and were interviewed about their beliefs and practices as biology teachers. Each of the teachers had more than 15 years of experience. They work in schools that represent communities with diverse socioeconomic and ethnic populations. All of the development team members believe that teaching and learning science is more than the mere ability to repeat countless, trivial facts. They believe biology is the process of constructing knowledge to explain and predict phenomena about living systems. In addition to the development team teachers, 16 high school biology teachers were colleagues in the research as they assumed the role of candidate for national recognition, developed portfolios, and shared, through interviews and debriefings, the excitement and frustrations of portfolio develop-
BIOLOGY TEACHER ASSESSMENT 151
ment. These teachers were selected to represent a variety of teaching contexts and many different years of experience, ranging from 26-year veteran to an intern. Last, there was an advisory board of teachers, science educators, and biologists who critiqued the research while it was in process and participated, with the development team teachers, in the rating of the completed portfolios.
One early objective of the BioTAP portfolio development team was to identify and explore aspects of teacher knowledge and practice that could best or only properly be assessed by documenting the biology teachers work in a school or classroom. Evidence of change and growth and of responsiveness to the context were identified as aspects of teaching ideally suited to on-site documentation.
Models. When the BioTAP research began, there were few models of a school- teachers portfolio. However, other professions present their credentials to members of the profession and to the public by means of portfolios, so it was possible to draw from these images as the concept of a teachers portfolio developed (Bird, 1990a). An artists portfolio is a collection of samples of finished, best work that artists agree provide evidence of the knowledge and skill of the artist. The actual samples in the portfolio are interchangeable depending on the goal. The number and variety of documents included in the artists portfolio is significant-having too few documents and having too many is equally reprehensible. Too few signals lack of productivity and experience; too many, an inability to select quality work appropriate to the goal. As a portfolio, a pilots log is an ongoing record of work in progress, not just records of best or typical work. The catalog of a salesperson indicates that the person has access to and can deliver a variety of materials. The badges of a boy or girl scout indicate an accomplishment that has been achieved with the help of a mentor. The badge has much meaning to other scouts, but may have less meaning outside that community, and is awarded with great ceremony and celebration. The intention of BioTAP was to design a portfolio development process for the assessment of teachers that was eclectic, drawing on elements of each of the existing models.
Documents. Another early BioTAP objective was to clarify what the nature of a document might be. A document is defined as an instance or specimen that serves to show, point out, provide evidence of or prove something. There are several classes of documents that a high school biology teacher might prepare as evidence of knowledge and skill of teaching and include in a container called a portfolio. One class of documents is artifacts, actual samples of the usual work of the teacher. These might include lesson plans or notes about a lesson, sample laboratory instructions or samples tests, or letters to parents or administrators. A special group of artifacts is samples of student work. A second class of evidence is reproduc-
152 A. COLLINS
tions-examples of work typically produced in teaching that have no permanence and therefore consciously must be captured in some permanent form for inclusion as a document in portfolio. Reproductions might include a photograph of a bulletin board or chalkboard display, a Xerox copy of student notes, or a videotape of a teacher conducting a lesson. A third class of documents is attestations, reports of the teachers practice prepared by someone other than the teacher. A letter of commendation by an administrator or a parent or a note written by a colleague commenting on a collaborative project or a letter from a former student are examples of attestation. A fourth class of documents is productions, evidence pre- pared especially for the purpose...