Upload
brianne-palmer
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
EVALUATION OF COURSE EVALUATIONS.
by
Prof. Peter Friis-Hansen, Assoc. Prof. Niels Houbak and Prof. Peder Klit
Department for Mechanical Engineering (MEK),Technical University of Denmark (DTU)
DK-2800 Kgs. Lyngby, Denmark.
Evaluation of courses ?
When is a course a good course?• When the lecturer senses the success.• When most students pass the exam• When the students are happy• When the industry are pleased with the candidates• When the students have learned enough to ….
continue
• Attendance• Percentage passing the course exam• Grade average• Student satisfaction• Lecturer satisfaction• Something else?
How do we measure it? (the engineering approach)
Final Evaluation Questionnaire
• 3 sections. A: The course, B: The lecturers, C: Written comments
• Has been used for more than 15 years, the last 3 years in electronic form.
• Must be completed one week before end of term (discussed at the last lecture)
• Is important for the lecturer and for the department management (study board)
• Information is included in overall evaluations of staffs and studies
The Questionnaire.Section A
1. Are the prescribed course prerequisites adequate?
2. How is the course material?
3. Is the form of the course adequate?
4. A standard course has an average workload of 9 hours pr week; how much time did you spend?
5. How many lectures did you attend?
6. What is your general satisfaction with the course?
7. For courses taught in English; did this influence your outcome?
The Questionnaire.Section B
1. How did the (named) lecturer present the subject?
2. Is the lecturer inspiring?
3. How did you experience the dialog/cooperation with the lecturer?
4. How is the lecturer as supervisor?
5. For group works; did you receive criticism for handed in exercises during the course?
Section C
1. I appreciate2. I criticize 3. I suggest
Answers and Weights: Section A.
1 Prerequisites Too many Suitable Too few None
4 10 0 10
2Teaching material
Very bad Bad Acceptable Good Very good
0 2,5 5 7,5 10
3 Form Less suited Reasonable Well suited
0 5 10
4Time
consumptionMuch less Less Normed More Much more
0 4 10 8 3
5 Participation 0-25 26-50 51-75 76-100
0 2,5 7,5 10
6 Satisfaction Very small Small Acceptable Much Very much
0 2 6 8 10
7 English Very negative Negative Not influenced PositiveVery
positive
0 5 10 10 10
Answers and Weights: Section B.
1 Presentation Very bad Bad Acceptable GoodVery good
0 2,5 5 7,5 10
2 Inspiration NoOnly little
To some extent
YesVery much
0 2,5 5 7,5 10
3Dialog,
CooperationVery bad Bad Satisfying Good
Very good
0 2,5 5 7,5 10
4 Supervision Very bad Bad Satisfying GoodVery good
0 2,5 5 7,5 10
5Criticism of
workNo
Only little
To some extent
YesVery much
0 2,5 5 7,5 10
Post processing.
• For each question an average value is calculated
• Questions are weighed against one another
• Average values are multiplied with weights and summed
• A course/lecturer utility number is the outcome
• This is a subjective one number [0-10] evaluation – far from any truth!
• BUT it may contain some information.
Cumulative Curves.Kummuleret fordeling for MEK kurser - skema A
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0,0 1,0 2,0 3,0 4,0 5,0 6,0 7,0 8,0 9,0 10,0
Nytte
Fra
kti
l
Fordeling for MEK kurser - skema A
0
1
2
3
4
5
6
7
8
9
Nytte
An
tal
Kummuleret fordeling for MEK undervisere - skema B
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0,0 1,0 2,0 3,0 4,0 5,0 6,0 7,0 8,0 9,0 10,0
Nytte
Fra
kti
l
Fordeling for MEK undervisere - skema B
0
2
4
6
8
10
12
14
Nytte
An
tal
Time Evolution: Fall term course curves.
Time Evolution: Fall term lecturer curves.
Evaluation action.Reactions from the study board
• Letters to excellent performing lecturers
• Letters to poor performing lecturers – filtered for good excuses. Important to notice, that the tone is: ”What do think you can do about this situation? Can we be of any help?”
• Easy to locate/spot changes in behaviour (monitoring 50+ courses is hard)
Conclusions:
1. It is always worthwhile to evaluate what you are doing.
2. Evaluations can sometimes be used for something differently.
3. The anonymous comparison allows for lecturers to see his/her own performance relative to other staff members.
4. All lecturers notice that their evaluation result has been processed. It is not only the poor performers that are noticed.
5. The weighing factors indicate what is important when improving the teaching quality.
6. The defined “Utility-system” generates a “dynamic normal”, i.e. conscientious staff will try to improve the teaching quality to obtain a good position on the cumulative curves.
7. Tests have shown that radical changes to the weights are needed to form a noticeable change in the ranking