Embed Size (px)
Citation preview
Planning For, Interpreting & Using Assessment Data
Gary Williams, Ed.D.Instructional Assessment Specialist, Crafton Hills College
Fred Trapp, Ph.D.Administrative Dean, Institutional Research/Academic Services, Long Beach City College
October, [email protected]
Goals of the Presentation
• “De-mystify” the assessment process
• Provide practical approaches & examples for assessing student learning
• Answer questions posed by attendees that pertain to their assessment challenges
What’s It All About?
• An ongoing process aimed at understanding and improving student learning.
• Faculty making learning expectations explicit and public.
• Faculty setting appropriate standards for learning quality.
What’s It All About?
• Systematically gathering, analyzing and interpreting evidence to determine how well student performance matches agreed upon faculty expectations & standards.
• Using results to document, explain and improve teaching & learning performance.
Tom Angelo
AAHE Bulletin, November 1995
Roles of Assessment
• “We assess to assist, assess to advance, assess to adjust”:– Assist: provide formative feedback to guide student
performance– Advance: summative assessment of student
readiness for what’s next– Adjust: continuous improvement of curriculum,
pedagogy.
- Ruth Stiehl,
The Assessment Primer: Creating a Flow of Learning Evidence (2007)
Formulating Questions for Assessment
• Curriculum designed backwards; Students’ journey forward:– What do students need to DO “out there” that
we’re responsible for “in here?” (Stiehl)
• Subsequent roles in life (work or future study, etc.)
– How do students demonstrate the intended learning now?
– What kinds of evidence must we collect and how do we collect it?
Assessment Questions & Strategies– Factors to consider:
• Meeting Standards– Does the program meet or exceed certain
standards?– Criterion reference, commonly state or
national standards
• Comparing to Others– How does the student or program compare to
others?– Norm reference, other students, programs or
institutions
Assessment Questions & Strategies-Factors to Consider:
• Measuring Goal Attainment– Does the student or program do a good job at
what it sets out to accomplish?– Internal reference to goals and educational
objectives compared to actual performance.– Formative student-center.– Professional judgment about evidence
common.
Assessment Questions & Strategies- Factors to Consider:
• Developing Talent and Improving Programs– Has the student or program improved?– How can the student’s program and learning
experience be improved even further?– Formative and developmental.– Variety of assessment tools and sources of
evidence.
Choosing Assessment Tools
• Depends upon the “unit of analysis”– Course– Program– Degree/general education– Co-curricular
• Also depends upon overall learning expectations
Formulating Assessment Strategies:
Formulating Assessment Strategies:
Formulating Assessment Strategies:
Direct vs. Indirect Evidence
• Direct– What can the student
actually do or demonstrate they know
– Can witness with own eyes
– Setting is structured/ contained
• Indirect– What students say
they can do– Focus on the learning
process or environment
– Things from which learning is inferred
– Setting is not easily contained/structured
Qualitative vs. Quantitative
• Qualitative– Words– Categorization of
performance into groups
– Broad emergent themes
– Holistic judgments
• Quantitative– Numbers– Individual components
and scores– Easier calculations
and comparisons plus presentation to a public audience
Formative vs. Summative
• Assessment for learning
• “In-progress” • Provide corrective
feedback• Establish foundational
learning for next step.
• Assessment for evaluative purpose
• “After the fact” • Determine progress/
achievement/proficiency• Readiness for next step/
role/learning experience
Means of Assessment-(Quantitative Judgments)
• Cognitive– Standardized exams– Locally developed exams
• Attitudes/beliefs– Opinion surveys of students, graduates,
employers
Means of Assessment- (Qualitative Judgments)
• Cognitive – Embedded classroom assignments
• Behavior/performances (skills applications)– Portfolios– Public performances– Juried competitions– Internships– Simulations– Practical demonstrations
• Attitudes/beliefs– Focus groups
Interpreting Results- How Good Is Good Enough?
• Norm Referencing– Comparing student achievement against other
students doing the same task
• Criterion Referencing– Criteria and standards of judgment developed
within the institution
Are Results Valid and Reliable?
• Validity
• Reliability
• Authentic assessment
• Important questions or easy questions
• Inform teaching and learning?
How Does Assessment Data Inform Decision-Making?
• Goal: Making sound curricular and pedagogical decisions, based on evidence
• Assessment questions are tied to instructional goals.
• Assessment methods yield data that is valid & reliable.
• A variety of measures are considered.
• Assessment is an ongoing cycle.
Assessment Process
Student Learning
Assessment Activities
3. Means of Assessment and
Criteria for Success
2. Intended Learning
Outcomes
5. Use of Results
4. Assessment Results
1. Expanded Mission Statement-GE, Program & Course Purposes
Collaboration Among Faculty, Administration & Researchers
• Assessment, the auto, and a road trip: an analogy– Who should drive the car?– Who provides the car, gas, insurance and
maintenance?– Who brings the maps, directions, repair
manual, tool kit, first aid kit, and stimulates the conversation along the journey?
Why Faculty are the Drivers
• Faculty have the primary responsibility for facilitating learning (delivery of instruction)
• Faculty are already heavily involved in assessment (classroom, matriculation)
• Faculty are the content experts
• Who knows better what students should learn than faculty?
Who Provides the Car and Keeps Gas in It?
Administrators!
Role of Administrators
• Establish that an assessment program is important at the institution
• Ensure college’s mission and goals reflect a focus on student learning
• Institutionalize the practice of data-driven decision making (curriculum change, pedagogy, planning, budget, program review)
• Create a neutral, safe environment for dialogue
Where Does IR Sit in the Car?
Roles of Researchers
• Serve as a resource on assessment methods
• Assist in the selection/design and validation of assessment instruments
• Provide expertise on data collection, analysis, interpretation, reporting, and use of results
• Facilitate dialogue - train and explain • Help faculty improve their assessment
efforts
Faculty DON’Ts…
• Avoid the SLO process or rely on others to do it for you.
• Rely on outdated evaluation/grading models to tell you how your students are learning.
• Use only one measure to assess learning
• Don’t criticize or inhibit the assessment efforts of others.
Faculty DOs...
• Participate in SLO assessment cycle
• Make your learning expectations explicit
• Use assessment opportunities to teach as well as to evaluate.
• Dialogue with colleagues about assessment methods and data.
• Focus on assessment as a continuous improvement cycle.
Questions From the Field…
