Student Portfolios and Authentic Learning Outcomes AssessmentA. Rae Clementz

Ron PittFaculty Development Center

Bryant CollegeNovember 17, 2002

Introduction

There are two parallel initiatives at Bryant College currently under development. The first is that of a student professional portfolio – a mechanism by which students can demonstrate their skills and achievements along a number of dimensions to potential employers and other external and internal constituencies such as advisors, colleagues, parents, scholarship committees, and graduate schools. The second is a form of authentic performance assessment – a mechanism by which Bryant College can determine the degree to which the undergraduate program is meeting its mission and instructional objectives by examining the overall performance of its students. One common tool for conducting this kind of authentic performance assessment and that was strongly suggested in some of the Learning Outcomes Assessment Subcommittee reports in Spring 2002 is a portfolio. This commonality of terms has led to a series of legitimate questions about the purposes, forms, and functions of a somewhat nebulous “Bryant Portfolio.”

The objectives of this paper are to:

Clarify the differences and areas of commonality between these two initiatives Propose a structure and process by which these two initiatives may be pursued while

maintaining a degree of separation as well as connectedness

Process, Form, and Function of the Assessment Portfolio

Although the professional portfolio and portfolio assessment share a word in their names, they represent different physical objects and are surrounded by different purposes, contexts, and procedures. Based on the assessment subcommittee reports, the assessment portfolio needs to have certain characteristics.

1. The skills and qualities associated with the undergraduate mission cut across numerous areas of the curriculum and are not discipline-specific. Rather than design or implement new assessment instruments, the assessment process should build upon existing coursework that is designated to be rich in particular learning outcomes (“course-embedded assessment”).

2. The complete group of stakeholders in a portfolio assessment must be identified, and their interests and concerns must be accommodated in a manner that ensures confidence in the results. In particular, student assessment must not, in any way, become bound up with or confused with faculty or course assessment.

3. The mechanisms by which the assessment portfolios are constructed must be established in a manner that all the stakeholders are comfortable with the types of evidence, the confidentiality of the evidence, and the control over that evidence.

4. There must be either a complete or a representative sampling of the portfolios of the student population, in order for the full range of student learning outcomes to be apparent.

5. A clear set of acceptable types of artifacts or evidence of learning outcomes must be established, so that evaluating the artifacts does not become a case of “comparing apples to oranges to bananas.”

6. Associated with the range of acceptable evidence, quality benchmarks must be identified that determine the quality of student performance.

7. A process must be created by which the level of quality of the evidence can be assessed with objectivity, validity, and repeatability. This may require the involvement of an external, disinterested third party.

8. Finally, a systematic review of the process must be developed and implemented, to ensure that the process meets all needs.

How Students View a Portfolio

While the characteristics described above may serve the needs of a portfolio assessment at Bryant College, they do not necessarily meet the needs and objectives of the students. In general, the student concept of a “portfolio” tends to be that of a professional portfolio, i.e. a tool in gaining employment (Young, 2002). In an informal set of brief, voluntary interviews of students conducted over the lunch hours in the Rotunda during October 2002 by Rae Clementz, Bryant students were asked “In what ways do you think a portfolio would be useful?” All of the thirty-three students interviewed responded with some variation of “to help me get a job,” with one student also indicating that it might be helpful for getting into graduate school. When asked, “What purpose should a portfolio serve?”, again all students interviewed indicated that it should demonstrate their skills or achievements. Interestingly, roughly 30% of the respondents qualified their comments with “of interest to the company I’m interviewing with.” This suggested that rather than an abstract and generic set of learning outcomes, students wanted a portfolio that demonstrated specific, job related skills. What was also implied, although never directly stated, was that students wanted a portfolio that was adaptable, so that different skills could be emphasized depending on the interests and focus of a potential employer.

The student portfolio can be further separated from an assessment portfolio in the types of evidence collected and the ways they are presented. An assessment portfolio is likely to contain whole projects or assignments and may even include associated interim deliverables. When asked what types of items should go into a portfolio, the majority of the students interviewed said “samples of my best work.” Several people even stated “nothing from my freshman or sophomore classes.” This difference was further reflected in their answers to the question “What characteristics would a portfolio have?” Twenty-nine of the thirty-three students indicated that the portfolio should “look very professional.” Somewhat surprisingly, when asked what format or delivery method would be most appropriate for a portfolio, the majority of students said it should be paper-based, although definitely electronically generated. When asked to explain why a paper-based format was best, the explanations revolved around not having access to computers during an interview, a lack of interest and/or lack of time on the part of the interviewer, and a desire to be consistent with perceived corporate practices. The format suggested by the students surveyed was consistent with the format corporate recruiters find most valuable – streamlined, and with specific pieces of evidence of a skill, trait, or ability.

Finally, there is the question of whether a portfolio should be an optional activity or a program requirement – which would require a major restructuring of the curriculum at Bryant College. Both methods are being used at different schools with varying degrees of success. A member of the Educational Technology Development Group at the University of Washington was quoted as saying, “If a student is putting a portfolio together just to satisfy some graduation requirement, he or she might feel resentful about the process, and the end result will probably not be very useful” (Young, 2002). This sentiment was echoed at Bryant where slightly over half of the students interviewed indicated that they would create a portfolio, even if it were optional, and in some cases, especially if it were optional. One student commented that, “making a portfolio is a way to differentiate myself from my peers. I’m willing to go above and beyond what’s required, and that makes me better.” However, almost half the students said they would not create a portfolio unless required to do so. One of the students who was for making the portfolio a requirement explained, “I’d probably hate it while I was doing it, but when it was done, I’d be glad I had done it, and it would probably help me a lot.”

“Portfolios” at Bryant College

It is clear that, between a professional portfolio and an assessment portfolio, there are different needs with different objectives and constituencies that lead to different ends. However, it may be possible to leverage these two processes and objects to assist each other. The following table summarizes the differences and commonalities between implementing a professional portfolio and an authentic performance assessment.

Professional Portfolio Performance AssessmentPrimary Objective

Demonstrate skills desired by potential employers Assess student learning and performance along skills and qualities identified in undergraduate mission statement

Potential Stakeholders Students Faculty Career Services Student Affairs Employers

Students Faculty OPIR Reaccredidation committees Learning-Outcomes Assessment Committee Joint Committee on Advising and First-Year Programs

Potential Audiences Potential Employers Graduate Schools Scholarship Programs Faculty Advisors Parents, Friends, Associates

Faculty Department Chairs President and Vice Presidents Curriculum Committee Learning-Outcomes Assessment Committee OPIR Professional and Faculty Advisors Trustees Accreditation Bodies

Functional Differences Represents students’ best work Streamlined Adaptable to different audiences Professional in content as well as appearance Accessible to outsiders

Represents actual student performance over time, for better or for worse

Structured and assessed around the core objectives Built upon existing classroom activities Representative of all student work Accessible only to designated parties for assessment

purposes

Proposed Next Steps and Timeline

By December 15, 2002 Establish one committee of the major stakeholders, with two sub-committees, to define and answer the

questions regarding the structure and process of these two initiatives

By February 1, 2003 Begin a series of community forums on assessment and portfolios as a way to inform all stakeholders

and initiate these projects

By March 15, 2003 Develop a set of goals, objectives, and activities for each initiative in the coming semester and year

By April 15, 2003 Establish an action plan for the implementation of these two initiatives Begin a series of community forums on the development of assessment criteria (rubrics)

By June 15, 2003 Establish a pilot assessment rubric and identify a review body. Begin a pilot test of the portfolio plan in a single, voluntary summer session course.

By September 1, 2003 Begin implementation of the portfolio initiatives

Throughout Document these activities, and establish a procedure for assessing the progress of these two projects

and recommending changes to the committees

Governance of these two initiatives needs to maintain a degree of separation between them while building on the existence of each other. The attached chart is an attempt to identify the levels of reporting in the creation and functioning of the two committees.

ReferencesYoung, J. (2002, March 8). E-portfolios could give students a new sense of their accomplishments. Chronicle of Higher Education, 48(26), p. A31

Friedman Ben-David, M. (1999) AMEE Guide No. 14: Outcome-based education: Part 3 – Assessment in outcome-based education. Medical Teacher, 21(1), pp. 23-25

Graphical representation of the inheritance structure in the creation and reporting of the two portfolio projects

Information Systems Division

Academic Affairs Division

FacultyAdministration

Student Affairs Division

OPIR

Learning Outcomes Assessment Committee

Joint Committee on Advising and First-

Year Programs

ACE

Faculty Development

Center

Student Senate

President’s Office

Contract Committees

Career Services

IT

Reaccreditation Committees

Administrative Units

Governing Committees

Operational Units

Portfolio Committees

Undergraduate Programs

Professional Portfolio

Committee

Performance Assessment

Portfolio Committee