Upload
jocelin-hood
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
SLO & SAO WorkshopSLO & SAO Workshop
Dr. Deborah DiThomas
Interim Vice Chancellor
Student Services & Operations
Cuesta College Student Services Managers
Dr. Edward C. Bush
Vice President, Student Services
SLO 10 Commandments
1. Acknowledge the SLO/SSO and keep it holy.2. Thou shall not do a SLO/SSO in isolation.3. Thou shalt make SLOs/SSOs relevant. 4. Thou shall not use the word data in vain.5. Thou shall not bite off more than one can chew. 6. Thou shall not covet your neighbor’s SLO/SSO. 7. Thou shall not keep your SLO/SSO on the shelf.8. Thou shall not ever become complacent. 9. Thou shall not kill, hurt, or injure anyone during the
SLO/SSO process, 10. but cursing is okay.
History of RCCD Program Review 05-06 :
Spring 05: New Standards/SLO TrainingDraft Program Reviews
Spring 05: Student Services Retreat Review Accreditation StandardsRevise SLO’sRevise SAO’sWrite Evaluation Plans
Summer 05/Fall 05: Finalize Program Reviews including SAO’s, SLO’s and Assessment Plans
Spring 06: Implement “Assessment Plans” for SAO’s & SLO’s
Summer 06: Begin ‘06 Program Reviews
Better understanding of campus programs
Make our SLO’s and SAO’s more singular—don’t bundle
Pick data that can be measured
We were too broad
Chose one or two specific things to work on
Challenged in serving students at the same time we were assessing students
Write more measurable objectives
See things through the eyes of students
Get staff involved
Follow the template it gives direction
Focus and prioritize
Focus of how we state SLO’s and SAO’s
Need a way to capture data
What We Learned:
Keeping good records so at end it was easy—use timelines
Organization of how we did things—using timelines
Worked well together—shared responsibilities
To truly serve students it takes one on one
Education and dissemination of safety information to college community
Documented needs
Completing the entire process makes us better for next year
Comfort level is higher
Ahead of academics and using us as example
Teamwork
Need to communicate changes to students
Get staff involved
Instruments to measure
Increased communication
What We Did Well:
We will do differently:
Get more input from hourly staff
Don’t wait until end to do evaluation
Better timelines and stick to the timelines (monthly calendars…)
Get all members of department on board
Improve communications with students
Identify specific people to do certain things
Include communication with academic side
More campus specific SLO’s SAO’s
Change format of last year
Defining SLOs & SSOs
Student Learning Outcome - Defines what students should be able to do, think, or know.
Service Area Outcome - Defines how the service area performs.
Defining Outcomes
Detailed, specific, measurable or identifiable, and personally meaningful statements that arederived from the goals and articulate what the end result of a unit, program, course, activity, or process is.
Outcomes should be:• Measurable (not necessarily, countable), or
observable,• Manageable, and• Meaningful.
Standard IIB – WASC
The institution recruits and admits diverse students who are able to benefit from its programs, consistent with its mission. Student support services addresses the identified needs of students and create a supportive learning environment. The entire student pathway through the institutional experience is characterized by a concern for student access, progress, learning, and success. The institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measures in order to improve the effectiveness of these services.
What Does This Mean?
The new standards promote the values of Student Services.
Student Service professionals are leaders in providing responsive services to a diverse and changing student population.
The learning outcomes challenge to Student Services is an opportunity to improve effectiveness and to integrate with instruction.
Standard IIB – continued
……The institution systematically The institution systematically assesses student support servicesassesses student support services using student learning outcomes, using student learning outcomes, faculty and staff input, and other faculty and staff input, and other appropriate measures appropriate measures in order to in order to improve the effectiveness of these improve the effectiveness of these services…services…
Effectiveness is the Ultimate Goal
Learning outcomes focus on what students know and can do rather than what we do to support them.
Shift the paradigm from Student Satisfaction to Student Learning.
Student affairs needs to demonstrate effectiveness thru data.
Campus staff learn how student affairs contributes to student learning.
Perhaps the key question to be asked of student affairs personnel might be expressed as follows…
“Of those students who use the services, programs and facilities, is there any effect on their learning, development, academic success or other intended outcomes, particularly when compared to non-users?”
It is important to note that not all outcomes of interest to student services are learning outcomes; many pertain to the timely, efficient and effective delivery of services.
Source: Upcraft, M. Lee and Schuh, John. Assessment in Student Affairs: A Guide for Practitioners. Jossey-Bass, San Francisco, 2000. ISBN 0-7879-0212-8
Service Area Outcomes Based onCurrently Existing Services
Name of Unit will provide Name of Current Services
improve
decrease
provide
Client will be satisfied with Name of Current Service
Formulating
Service Area Outcomes (SAOs)
SSA Service Verb + Objective
Tutoring Improve…
Students attending Academic Advising will Increase…
Workshops Understand…
Counseling Sessions Know….
“Students participating in the development of a student educational plan will understand the educational requirements for their ed goal and major.”
Formulating Student Learning Outcomes (SLOs)
Student Service Areas
Checklist: Student Learning Outcomes Consistent with Mission Statement. Agreement ~ were staff involved in the
development? Reasonable ~ for the ability of the students. Measurable ~ can be observed, tested. Key Concepts ~ important to the service area,
program. Clarity ~ precise in description. Singular ~ not “bundled” (shown by use of
conjunctions, commas).
R I V E R S I D E C O M M U N I T Y C O L L E G E Moreno Valley Campus Norco Campus Riverside City Campus
Manager: ____________________ Current Service Area Outcome:
Checklist: Service Area Outcomes (SAO’s)
Consistent with Unit Mission Statement. Agreement ~ were staff involved in the development? Reasonable ~ for the ability/staffing of the service area. States what “will” be ~ provided, improved, increased etc. Measurable ~ can be observed, tested. Key Concepts ~ important to the program/service area. Clarity ~ precise in description. Singular ~ not “bundled” (shown by use of conjunctions, commas). Revised Service Area Outcome:
R I V E R S I D E C O M M U N I T Y C O L L E G E Moreno Valley Campus Norco Campus Riverside City Campus
Manager: Current Student Learning Outcome: Checklist: Student Learning Outcomes (SLO’s)
Consistent with Mission Statement. Agreement ~ were staff involved in the development? Reasonable ~ for the ability of the students. Measurable ~ can be observed, tested. Key Concepts ~ important to the course, program. Clarity ~ precise in description. Singular ~ not “bundled” (shown by use of conjunctions,
commas). Revised Student Learning Outcome:
The Assessment Process
4. Utilize Resultsfor Improvement
3. Interpret Findings 2. Gather Evidence
1. Articulate Goals
Good Assessment Techniques
ValidReliableActionableEfficient and cost-effectiveEngaging to students and others Interesting to faculty and other
stakeholdersTriangulated
Types of Assessment
Direct assessments- provide students the opportunity to show what they know. They “…prompt students to represent or demonstrate their learning or produce work so that observers can assess how well students’ texts or responses fit institutional or program-level expectations.” (Maki, p. 88)
Indirect assessments -“…capture students’ perceptions of their learning and the educational environment that supports that learning, such as access to and the quality of services, programs, or educational offerings that support their learning…By themselves, results of indirect methods cannot substitute for the evidence of learning that direct methods provide. They can, however, contribute to interpreting the results of direct
methods…” (Maki, p. 88, 89)
Types of Assessment (cont)
Formative Assessment-is designed to assist the learning process by providing feedback to the learner, which can be used to identify strengths and weakness and hence improve future performance. Formative assessment is most appropriate where the results are to be used internally by those involved in the learning process (students, teachers, curriculum developers).
Summative Assessment-is used primarily to make decisions for grading or determine readiness for progression. Typically summative assessment occurs at the end of an educational activity and is designed to judge the learner’s overall performance. In addition to providing the basis for grade assignment, summative assessment is used to communicate students’ abilities to external stakeholders, e.g., administrators and employers.
Types of Assessment (cont)Qualitative research - uses open-ended questions to gain an in-depth
understanding of the questions being explored. Common qualitative techniques include focus group discussions, mini-groups, and in-depth interviews either face-to-face or via telephone.
Quantitative methods - are distinguished by their emphasis on numbers, measurement, experimental design, and statistical analysis. Researchers typically work with a small number of predetermined response categories to capture various experiences and perspectives of individuals. Often emphasis is on analyzing a large number of cases using carefully constructed instruments that have been evaluated for their reliability and validity (Patton, 1990). Techniques include questionnaires, structured interviews, and tests.” (Palomba and Banta, p. 337)
Both qualitative and quantitative methods bring valuable information to light, but they yield different types of results. Review the type of information needed when selecting a method, and also consider whether qualitative and quantitative research could be used together.
Common Assessment Methods
Performances and Staff Observations
– Presentation, or demonstration Surveys
– Attitudes and perceptions of students and staff Narrative
– Staff and student journals, interviews, focus groups Cumulative
– Portfolios and capstone projects Database-Tracked Academic Behavior
– Grades, graduation, lab usage, persistence
Your Task…
Work with the members of your service area to:
Develop an assessment plan:– Consider types of evidence.– Utilize existing assessment tools if appropriate.– Consider partnering with other student service areas.– Close assessment loop.
Create an annual timeline for assessment.
Delineate people responsible for the various segments of the assessment plan.
RCCD Student Services Assessment of SLO’s and SAO’s
1. Decide on how to assess your SLO’s & SAO’s Can have more than one assessment for any of the
SLO’s or SAO’s
Consider assessment tools that are easy for your area to use
Create own assessment tools, if necessary
Decide where the assessment will happen
Decide who will conduct the assessment
Decide when it will be conducted
Checklist: Means of Assessment
Specific means of measurement identified. Overall score, rating or response level set. Sub-score minimum of a lower scale in any
given area established. Number of students/clients who meet score
given. Administration when and by whom defined. Results judged on what basis stated. Results judged by whom stated.
R I V E R S I D E C O M M U N I T Y C O L L E G E Moreno Valley Campus Norco Campus Riverside City Campus
Manager: Means of Assessment: Checklist: Means of Assessment
Specific means of measurement identified. Overall score, rating or response level set. Sub-score minimum of a lower scale in any given area established. Number of students/clients who meet score given. Administration when and by whom defined. Results judged on what basis stated. Results judged by whom stated. Revised Means of Assessment:
ASSESSMENT WORKSHEET
STUDENT SERVICE AREA: OBJECTIVE:
Student Learning Outcome/ Service Area Outcome Audience
Circumstance/Condition Intervention
Benchmark data (if applicable)
Significant Degree (target and performance level)
Data Collection: Assessment methods and tools
Persons (s) Responsible
Timeline for data collection
SLO/SAO Data Analysis Recommended Improvement
1.1)
Student Learning Outcome/ Service Area Outcome Audience
Circumstance/Condition Intervention
Benchmark data (if applicable)
Significant Degree (target and performance level)
Data Collection Assessment methods and tools
Person (s) Responsible
Timeline for data collection
SLO/SAO Data Analysis Recommended Improvement
Definitions
• Student Learning Outcome - Defines what students should be able to do, think, or know.
• Service Area Outcome - Defines how the service area performs.• Audience - Who is your program/service targeting?• Circumstance/Condition Intervention - Where or under what
circumstances/ conditions will the learning occur?• Benchmark data - Data that serves as a standard by which others may be
measured that serves as a basis for evaluation or comparison• Significant Degree - Target performance level that states how much
learning will be enough?• Data Collection – What assessment methods and tools will be used to
measure learning or performance?• Person (s) Responsible – Identify titles of persons responsible for carrying
out SLO/SAO.• Timeline for data collection – Set a goal for data collection.• SLO/SAO Data Analysis – Summarize the outcome(s) of the study.• Recommended Improvement – How will you use the outcome(s) for
service area improvement?
Implementing Assessment Results
Discuss and disseminate
Make appropriate program improvements
Integrate with program review and strategic planning process
RCCD Student Services Assessment of SLO’s and SAO’s
Assess the objectives and collect the data: Conduct the Assessment Organize the data Summarize your findings Disseminate results
Decide what to do next based on the findings Happy with the results?
You congratulate yourself and select another SLO/SAO to measure…
Not happy with the results? What can you do in your area to make changes?