Documenting Teaching Effectiveness: Sources of Data

Instructor Self Assessment – your perceptions of your teaching1

Course Portfolio – developmental, what you want to think about/know more about2

Teaching Portfolio – evaluative, what reviewers want to know/assess3

Teaching Journal – critically reflective teacher/teaching4

Expanded Teaching Philosophy – to serve as reflective introduction to dossier

Student Perceptions – student perceptions of your teaching Midterm / Early term student feedback End-of-Term Student Ratings of Teaching On-going collection of in-class feedback through short evaluation surveys (eg, new assignments, use of

readings, delivery of class session; Likert & open ended) Critical Incident Questionnaire

o At what point in class this week did you feel most engaged with what was happening?o At what point in class this week did you feel most distanced from what was happening?o What action that anyone (teacher or student) took in class did you find most affirming & helpful?o What action that anyone (teacher or student) took in class did you did most puzzling or confusing?o What about class this week surprised you the most? (From your own reactions to what went on, to

something that someone did, or to anything else that occurs to you as you write now.)

Student Learning – student perceptions of learning as a result of teaching Student Feedback through Consensus5

Develop Classroom Assessment Techniques (CATs) to gather student perceptions of their own learning and of teaching effectiveness related to particular practices recurring in class sessions, across a new segment/unit in course, in activities or assignments (eg, interactive lecture technique, coherence of lecture notes across a new segment of course, use of peer response groups, on grading practices/rubric, on use of suggested resources)

Evidence of student achievement (national exams/benchmark outcomes, grad school placement, thesis completion, student retention/completion; UROP)

Localize National Survey of Student Engagement (NSSE) benchmarks such as use of high impact activities that "require that students engage in substantive matters while actively interacting and collaborating with faculty and their peers"

Develop measures that allow for addressing and analyzing particular forms of resistance to learning and for aligning student/teacher expectations – consider whether students expect that you will address, for example, race, class, sexuality in your course? expect that engaged learning will be the norm for this course? expect to write even if this is not a writing intensive course?

Peer / Colleague Review – peer perceptions of your teaching / learning

Documenting Evidence of Teaching / Page 1

Classroom observation – teaching peers, cohort faculty Lesson Study Project (http://www.uwlax.edu/sotl/lsp/) Review of instructional materials; syllabus, assignments, exams (Peer Review Project:

www.courseportfolio.org/) Review of innovative uses of technology; contributions to distance learning Review of engaged pedagogies/approaches to non-traditional students' and learning (eg, service

learning, study abroad, multicultural teaching and learning) Mentoring activities – whether being a mentor or a mentee Development of teaching- and/or learning-related workshop for peers Peer-reviewed teaching/learning article accepted or grant accepted

Teaching / Learning Resources – research-based perceptions of learning Engagement with disciplinary journals related to teaching, from course design to assessment of

courses, teaching and learning (eg, incorporation into teaching approach, formation of journal club, participation in learning community)

Pursuit of a teaching-related research question to address via Scholarship of Teaching & Learning Participation in faculty professional development programming (eg, career programs, New Faculty

Orientation, teaching with writing or technology workshops, multicultural teaching and learning fellowships/ research programs)

Administrative Perspective – perceptions of teaching role & excellence Dossier Requirements/Data to Document Administrative Guidelines (as set by department, college

and/or university-wide policy) Evaluative Teaching Portfolios Data from curriculum assessment

1. The diagram and its organizing principle are drawn from the Center for Instructional Development and Research at the University of Washington-Seattle; see http://depts.washington.edu/cidrweb/consulting/assessment.html. The suggestions derive from multiple sources, suggestions and experiences shared by CTL consultants.

2. See, for example, http://depts.washington.edu/cidrweb/portfolio/developmental.html or http://www.courseportfolio.org/peer/pages/index.jsp?what=rootMenuD&rootMenuId=3#benchmark.

3. See, for example: http://depts.washington.edu/cidrweb/portfolio/evaluative.html or http://www.courseportfolio.org/peer/pages/index.jsp?what=rootMenuD&rootMenuId=3#inquiry.

4. See Stephen Brookfield's resources on Becoming a Critically Reflective Teacher at http://stephenbrookfield.com/papers.html.

5. University of Minnesota Center for Teaching & Learning: http://www1.umn.edu/ohr/teachlearn/customized/sfc/index.html

Summarizing by Graphic DisplaysAssemble graphs with an explanation: include number of students, dates the courses were taught, qualities that are being evaluated. A main feature of graphs is that they should be clear, not cluttered. Depending on your audience, include all courses taught over a certain period of time (promotion and tenure purposes) or only a select number (applying for a faculty job) in order to showcase the courses you've actually designed. Either way, it will be important to describe which courses you've chosen to include and why. See impage on next page. Credit: http://ftad.osu.edu/portfolio/Feedback.html#graphic.

Documenting Evidence of Teaching / Page 2

Plotting Means of Course EvaluationsAt the Center for Teaching Effectiveness we have found it useful to plot the progress of teaching in a given course over several semesters. By laying out general items along a time line, a professor can document trends in student evaluations. If a single data point is out of line, its impact is lessened by the overview, and the professor may choose to discuss factors in that particular semester that could have contributed to the deviation. http://www.utexas.edu/academic/cte/teachfolio.html:

1 The diagram and its organizing principle are drawn from the Center for Instructional Development and Research at the University of Washington-Seattle; see http://depts.washington.edu/cidrweb/consulting/assessment.html. The suggestions derive from multiple sources, suggestions and experiences shared by CTL consultants.2

? See, for example, http://depts.washington.edu/cidrweb/portfolio/developmental.html or http://www.courseportfolio.org/peer/pages/index.jsp?what=rootMenuD&rootMenuId=3#benchmark. 3

? See, for example: http://depts.washington.edu/cidrweb/portfolio/evaluative.html or http://www.courseportfolio.org/peer/pages/index.jsp?what=rootMenuD&rootMenuId=3#inquiry. 4

? See Stephen Brookfield's resources on Becoming a Critically Reflective Teacher at http://stephenbrookfield.com/papers.html. 5

? University of Minnesota Center for Teaching & Learning: http://www1.umn.edu/ohr/teachlearn/customized/sfc/index.html

Documenting Evidence of Teaching / Page 3

Analyzing Written Comments

Matrix: Laying out a matrix grouping written comments according to the overall course rating given by each student evaluator provides a context for the comments. An instructor can see what kinds of comments were made by students who were in general satisfied with the course, and what kind were made by those who were dissatisfied.. This analysis of written comments sometimes helps to explain certain positive or negative comments, which might be confided to a small subset of a course. See matrix image on next page. Credit: http://www.utexas.edu/academic/cte/teachfolio.html.

Summary of Student Feedback with Reflection: Jane Doe / LinguisticsAt the end of each quarter, students fill out an Evaluation of Instruction Report. They are asked to rate

aspects of both the course and the instructor using a Likert-type scale of 1 (poor) to 5 (excellent). The overall rating at the end of my first quarter teaching was 4.2; at the end of my second quarter, this rose slightly to 4.3

More informative evidence came from students’ written evaluations. In evaluating my first quarter of teaching, student comments were generally favorable, though they sensed my lack of experience as a

Documenting Evidence of Teaching / Page 4

teacher. This was not mentioned after my second quarter of teaching, as I felt more comfortable with both the course content and with leading a classroom. Some students were rather impressed that I learned everyone’s name quickly – not a frequent occurrence at a large state university. A notable aspect of my teaching was that I encouraged them to argue against some of the theories presented. One student described this as “encouraging them to express their own thoughts and feelings.” Other aspects of my teaching that they praise are that I am well-organized, I pay attention to students’ needs and spend extra time helping them when necessary, and I foster a relaxed learning environment in class.

Students' main suggestion for improvement was to incorporate more classroom activities. This corresponds to what I have been learning in professional development workshops: students can have a richer learning experience when they are engaged in the process. Incorporating activities such as demonstrations of experiments, videos, and small-group discussions would accomplish several goals. It would break up the long class time (typically 1 hour, 48 minutes), preventing students from losing attention, and it would provide opportunities for students to work with the material presented in class in a variety of ways. (Last 3 examples from http://ftad.osu.edu/portfolio/.)

Documenting Evidence of Teaching / Page 5