Upload
erik-franklin
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Developing Assessment Methods at Classroom, Unit, and Campus-Wide LevelsPresented atThe Robert Gordon UniversityAberdeen, Scotland
March 19, 2004
Presented by
Trudy W. BantaVice ChancellorPlanning and Institutional ImprovementIndiana University-Purdue University Indianapolis355 N. Lansing St., AO 140Indianapolis, Indiana 46202-2896tbanta@ iupui.eduhttp://www.planning.iupui.edu
Outcomes Assessment
The process of providing credible
evidence of the outcomes of
higher education undertaken for
the purpose of improving
programs and services within the
institution.
Banta, T. W.
ASSESSMENT . . .
“a rich conversation
about student learning
informed by data.”
-- Ted Marchese --
AAHE
Assessment of Individual Student Development
Assessment of basic skills for use in advising• Placement• Counseling
Periodic review of performance with detailed feedback
End-of-program certification of competence• Licensing exams• External examiners
Key Results of Individual Assessment
Faculty can assign grades
Students learn their own strengths and weaknesses
Students become self-assessors
Where is learning satisfactory?
What needs to be retaught?
Which approaches produce the most learning for which students?
Group Assessment Activities
• Classroom assignments, tests, projects
• Questionnaires for students, graduates, employers
• Interviews, focus groups• Program completion and placement• Awards/recognition for graduates• Monitoring of success in graduate
school• Monitoring of success on the job
Use of Results of Group Assessment
• Program improvement
• Institutional and / or state peer review
• Regional and / or national accreditation
Some Purposes of Assessment
1. Students learn content
2. Students assess own strengths
3. Faculty improve instruction
4. Institutions improve programs/services
5. Institutions demonstrate accountability
Outcomes Assessment Requires Collaboration
In setting expected program outcomes
In developing sequence of learning experiences (curriculum)
In choosing measures In interpreting assessment findings In making responsive improvements
Barriers to Collaboration in the Academy
1. Graduate schools prepare specialists
2. Departments hire specialists
3. Much of our scholarship is conducted alone
4. Promotion and tenure favor individual achievements -- interdisciplinary work is harder to evaluate
Campus Interest in Assessment
WHAT WORKS in….
increasing student retention? general education? use of technology in instruction? curriculum in the major?
Good assessment is good research . . .
An important question An approach to answer the
question Data collection Analysis Report
-Gary R. Pike (2000)
Most Faculty Are Not Trained as Teachers
FACULTY DEVELOPMENTCan Help Instructors:
Write clear objectives for student learning in courses and curricula
Individualize instruction using a variety of methods and materials
Ask questions that make students active learners
Develop assessment tools that test higher order intellectual skills
Organizing for Assessment
Goal Course Measure Findings Uses
Write Portfolio
Speak Speech
Think Test
Find Information
Project
Taxonomy of Educational Objectives
(Bloom and Others, 1956)Cognitive domain
categories
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Sample verbs for outcomes
Identifies, defines, describes
Explains, summarizes, classifies
Demonstrates, computes, solves
Differentiates, diagrams, estimates
Creates, formulates, revises
Criticizes, compares, concludes
Learning Outcomes for English Majors
Graduates can:1 Demonstrate how language influences
intellectual and emotional responses2 Apply knowledge of rhetorical context by
writing effectively and appropriately in context
3 Assess critically spoken, written, and visual representations
4 Apply research strategies appropriate to area of study
5 Synthesize diverse issues and responses raised in collaborative discussions of texts
Learning Outcomes in Science
1. Define and explain basic principles, concepts, theories of science
2. Identify characteristics that distinguish math and science from each other and from other ways of obtaining knowledge
3. Illustrate how developments in science can raise ethical issues
4. Solve theoretical or experimental problems in science
5. Evaluate the validity and limitations of theories and scientific claims in interpreting experimental results
6. Evaluate scientific arguments at a level encountered by informed citizens
In each course
Statement
Of
Competence
Teaching/
Learning
Strategy
Assessment
Methods
Critical
Thinking
Group
Project
Written
Paper
Planning for Learning and Assessment
1. Whatgeneraloutcomeare youseeking?
2. Howwould youknow it(theoutcome)if you sawit? (Whatwill thestudentknow orbe able todo?)
3. How willyou helpstudentslearn it?(in classor out ofclass)
4. How couldyoumeasureeach of thedesiredbehaviorslisted in #2?
5. What aretheassess-mentfindings?
6. Whatimprove-mentsmight bebased onassess-mentfindings?
Organizational Levels for Assessment
National
Regional
State
Campus
College
Discipline
Classroom
Student
Direct Measures of LearningAssignments, exams, projects, papers
Indirect MeasuresQuestionnaires, inventories, interviews
- Did the course cover these objectives?- How much did your knowledge increase?- Did the teaching method(s) help you
learn?- Did the assignments help you learn?
GOOD ASSESSMENT INCLUDES BOTH
Fast Feedback(at end of every class)
Most important thing learned Muddiest point Helpfulness of advance reading
assignments for day’s work in class Suggestions for improving class /
assignmentsBateman & Roberts
Graduate School of Business
University of Chicago
Student Suggestions for Improvement
Install a portable microphone Increase type size on
transparencies Leave lights on when using
projector Don’t cover assigned reading in
detail Provide more examples in class
Wingspread Seven Principles for Good Practice in Undergraduate
Education(Chickering & Gamson, 1987)
1. Student-faculty contact
2. Cooperation among students
3. Active learning
4. Prompt feedback
5. Time on task
6. High expectations
7. Respect for diverse talents and ways of learning
Student Learning OrientedCourse Evaluation
1. Learners held high expectations for one another
2. Learners interacted frequently with others
3. Learners participated in learning teams4. Learners respected diverse talents and
ways of learning
-CournoyerAdvances in Social
Work – Fall 2001
Primary Trait Scoring
Assigns scores to attributes (traits) of a task
STEPS Identify traits necessary for success in
assignment Compose scale or rubric giving clear
definition to each point Grade using the rubric
Can Develop a Research Paper
1. Narrows and defines topic
2. Produces bibliography
3. Develops outline
4. Produces first draft
5. Produces final draft
6. Presents oral defense
Out-standing
Accept-able
Unaccept-able
BibliographyOutstanding – References current,
appropriately cited, representative, relevant
Acceptable – References mostly current, few citation errors, coverage adequate, mostly relevant
Unacceptable – No references or containing many errors in citation format, inadequate coverage or irrelevant
Sophomore Competence in Mathematics(Multiple choice responses & supporting work)
Score3
2
1
0
CriterionClear conceptual understanding, consistent
notation, logical formulation, complete solution
Adequate understanding, careless errors, some logic missing, incomplete solution
Inadequate understanding, procedural errors, logical steps missing, poor or no response
Problem not attempted or conceptual understanding totally lacking
Ball State University
Assessment in Sociology and Anthropology
Focus groups of graduating students Given a scenario appropriate to the discipline, a
faculty facilitator asks questions related to outcomes faculty have identified in 3 areas: concepts, theory, methods.
2 faculty observers use 0-3 scale to rate each student on each question
GROUP scores are discussed by all faculty Murphy & Goreham
North Dakota State University
Assessment of Group Interaction
The Student Participant: Listened to others Actively contributed to discussion Challenged others effectively Was willing to alter own opinion Effectively explained concepts/insights Summarized/proposed solutions
5=Consistently excellent
3=Generally satisfactory1=Inconsistent and/or inappropriate
Capstone Course in Business
Comprehensive case study analysis
Graded using scoring rubric covering writing skills knowledge of historic and current theories
and practice ability to integrate tools and techniques
from business specialties in analyzing the case
Olney & Menger
St. Mary’s University
In a Comprehensive Assessment Program...
INVOLVE Students Faculty Student Affairs
Professionals Administrators Graduates Employers
Student Advisory Council at Montevallo
A way to provide continuous student assessment
Student Recommendations1 Develop a statement of expected
ethical behaviors for students2 Add a second research course with lab3 Increase comparative psychology4 Add terminals for statistics lab5 Increase opportunities for research,
writing, and speaking
Collaboration in Evaluation(Harvard and Samford)
Involve a student team in evaluating a course or program Interaction increases Students develop communication skills
teamwork competence
self-confidence
understanding of others’ perspectives
Involvement in learning increases
PRINCIPLES OF UNDERGRADUATE LEARNING
(PULs)1. Core communication and quantitative
skills2. Critical thinking3. Integration and application of knowledge4. Intellectual depth, breadth, and
adaptiveness5. Understanding society and culture6. Values and ethics
Approved by IUPUI Faculty Council May 1998
Student Electronic Portfolio
Students take responsibility for demonstrating core skills
Unique individual skills and achievements can be emphasized
Multi-media opportunities extend possibilities
Metacognitive thinking is enhanced through reflection on contents
- Sharon J. Hamilton
IUPUI
Virginia Commonwealth University
1. First-year English students wrote 1
response / week for 15 weeks
2. Faculty-student affairs teams read
essays
3. Sample of writers interviewed 18
months later
Responses to findings:
Central advising center, new advising
handbook, multicultural workshops
Involving EmployersCombination of survey and focus groups
for employers of business graduates Identified skills, knowledge, personality attributes
sought by employers Encouraged faculty to make curriculum changes Motivated students to develop needed skills Strengthened ties among faculty, students, employers
- Kretovics & McCambridge
Colorado State University
Colorado State UniversityCollege of Business
Curriculum changes based on employer suggestions:
1 credit added to Business Communications for team training and more presentations
Ethics & social responsibility now discussed in intro courses
New Intro to Business course emphasizing career decision-making
More teamwork, oral & written communication, problem-solving in Management survey courses
- Kretovics & McCambridge
Program Course Measurement Findings Action
Objective Objective Method Needed
Apply CT Use Lab exercises 64% of Students
skills in SW statistical ___________ students ask for
practice analysis to (80% of scored 3.0 more lab
inform students will or better time; 5 days
practice earn 3.0 or on lab of lab
decisions better on lab exercises added
exercises
Columbia College
Authentic Assessmentat
Southern Illinois University - Edwardsville
Business - Case Study Analysis with Memo Education - Professional Portfolio Psychology - Poster on Research Project Engineering - Senior Design Project Nursing - Plan of Care for Patient
Responses to Assessmentat
Southern Illinois University-Edwardsville• Business – More case studies and research• Education – More practice in classroom
management• Psychology – Curriculum change in statistics• Engineering – More practice in writing and
speaking• Nursing – Simulation lab with computerized
patients
Assessment in Fine Arts— Portfolio review and/or audition
for every student every semester by
Panel of faculty, students, community representatives, staff or faculty outside fine arts
Results - Students creativity, conceptualization, technique have improved
— Alec Testa— Eastern New Mexico University
Assessment Essentials
1. Plan for assessment2. Involve all stakeholders3. Develop goals and objectives4. Select or design assessment methods5. Report assessment findings6. Promote the use of assessment findings7. Evaluate the assessment program
- Palomba & Banta
Success Factors1 Committed leadership2 Collaboration between faculty and student
affairs leaders3 Teamwork in planning and implementation4 Supportive campus climate
Concern for students, continuous
improvement5 Involvement in design of assessment6 Results effectively communicated7 Conscientious follow-up8 Persistence