Online Course Evaluations
Is there a perfect time?
Presenters:
Cassandra Jones, Ph.D., Director of Assessment
Michael Anuszkiewicz, Research Associate
Office of Institutional Effectiveness
• Founded in 1950
• Private, not-for-profit
• Coeducational
• Comprehensive university
• Four Schools• Arts and Sciences• Business Administration• Education and Human Services• Malek School of Health Professions
About Marymount
Student Body (n=3,702)
67%
33%
By Level
Undergrad
Graduate
72%
28%
By Gender
Female
Male
47%
38%
9% 6%By Race/Ethnicity
White
Minority
Nonresident
Unknown
59%15%
4%22%
By Home State
VA
MD
DC
Other/Intn'l
Course Evaluations at MU• Done in-house since Fall 2002
• Paper forms completed in class through Spring 2007
• Online student evaluations since Summer 2007
• Evaluations distributed at the end of each semester– Three terms: Fall, Spring, and Summer– Excluded: Internships; projects; courses of fewer than 2
• Timeline: 3 weeks (since Spring 2008)– Last 2 weeks of class– Exam week
Issues
• Response Rates
• Quality – Nonresponse bias
• New issue – Timing – To exam or not to exam
ISSUE 1:
RESPONSE RATES
A Closer Look at Response Rates
62%
38% 33% 36% 31% 37%
Fall Term Response Rates by Submission Date
Last Day of Class
A Closer Look at Response Rates
62%
38% 33% 36% 31% 37%
0%
54% 51% 49%41%
54%
Fall Term Response Rates by Submission Date
Last Day of Class Including Exams (+)
Issue 2: Nonresponse Bias
• Study Conducted in 2009• Examined predictor of ratings and who
participates: No common predictor
Issue # 3
Timing of Evaluations: To offer or not to offer during exams?
Timing of Responses Received
Term % Before Exams % During ExamsFall 2010 76% 24%Fall 2011 88% 12%Fall 2012 72% 28%
3-Year Average 79% 21%
Undergraduate Students
Feared Distribution of Course Evaluation Completion
F D- D+ C- C C+ C+ B- B B+ A- A0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
During Exams Linear (During Exams)
Perc
ent R
ecei
ved
Feared Distribution of Course Evaluation Completion
F D- D+ C- C C+ C+ B- B B+ A- A0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Before Exams Linear (Before Exams)
Perc
ent R
ecei
ved
How would you rate this course as a learning experience?
Submission Before Exams Submission During Exams
Very Poor
Poor Fair Good Excellent0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Overall
Perc
ent R
ating
Very Poor
Poor Fair Good Excellent0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Overall
Perc
ent R
ating
How would you rate this course as a learning experience?
F D- D D+ C- C C+ B- B B+ A- A.00
.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
Before Exams During Exams
Mea
n Sc
ore
How would you rate the instructor’s teaching?
Submission Before Exams Submission During Exams
Very Poor
Poor Fair Good Excellent0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Overall
Perc
ent R
ating
Very Poor
Poor Fair Good Excellent0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Overall
Perc
ent R
ating
How would you rate the instructor’s teaching?
F D- D D+ C- C C+ B- B B+ A- A.00
.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
Before Exams During Exams
Mea
n Sc
ore
Decided to Examine More Systematically
• Can one predict who responds before or during exams?
• Variables included:– Response: Before or after exams– Course evaluation rating– Course grade– Course level – Semester– Gender– Course Enrollment– Term
• Undergraduates Only
Course Evaluation Questions
• The instructor was well prepared for each class
• The instructor explained course content• The instructor was responsive to student
questions• The instructor used class time well• The instructor encouraged me to take
responsibility .. . .
Findings: Descriptives
• 3 Fall Semesters• 2784 Course Sections• 14055 student records• Composite of Five Questions – Likert scale (Strongly Disagree to Strongly Agree)– Average Course Rating: 21.61 out of 25.00
Findings: Statistical
• HGLM and Logistical Regression/ Same Results– Student Characteristics– Course Characteristics
Findings
• Predictors indicating greater chance of completing before exams– Course grade– Male– Course size– Course Level (up to 400 level course)
Findings
• Predictors indicating greater chance of completing during exams:– Gender (Female)– Higher level of course
• Important finding: Course evaluation rating: Not significant
Conclusions and Implications
• No statistically significant differences in ratings based on when a student responds
• Students who are male, and have higher grades respond before exams
• Students are more likely to complete evaluations before exams in lower level classes
Implications
• Course evaluation ratings are not impacted by timing
• The longer students are in school, the more encouragement they need to complete the evaluations earlier
• Reason to change timing?
Thank you for your time and interest!
Office of Institutional EffectivenessMarymount University2807 North Glebe RoadArlington, VA 22207