Upload
gisela-randall
View
35
Download
0
Embed Size (px)
DESCRIPTION
How honest are the students when assessing each other’s work? Diogo Montalvão [email protected] Senior Lecturer, Dipl MSc PhD CEng School of Engineering & Technology 2012 Annual L&T Conference, 3 rd May. Outline. Why student’s peer-assessment? Description of a case study at IST - PowerPoint PPT Presentation
Citation preview
1/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
How honest are the students when assessing each other’s work?
Diogo Montalvã[email protected]
Senior Lecturer, Dipl MSc PhD CEngSchool of Engineering & Technology
2012 Annual L&T Conference, 3rd May
2/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
1. Why student’s peer-assessment?2. Description of a case study at IST3. Some of the students’ works4. Peer-assessment results5. Conclusions
Outline
3/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Learn from their peers
Encourages critical analysis of student work
Increased feedback to
students
Helps them perceiving their
own learning
Helps them interpreting assessment
criteria
Why student’s peer-assessment?Student benefits
source: http://www.heacademy.ac.uk/hlst/resources/a-zdirectory/peer_assessment
4/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Summative grading purposes
Reduced marking loads
To get students more involved and to make
them take more responsibility
Assess individual effort in team work
Engagement
Why student’s peer-assessment?Teacher benefits
source: http://www.heacademy.ac.uk/hlst/resources/a-zdirectory/peer_assessment
5/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Why student’s peer-assessment?
Disadvantages may be encountered when students:
• Lack the ability to evaluate each other
• Do not take it seriously
• Fear discrimination
source: http://www.heacademy.ac.uk/hlst/resources/a-zdirectory/peer_assessment
6/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
1. Why student’s peer-assessment?2. Description of a case study at IST3. Some of the students’ works4. Peer-assessment results5. Conclusions
Outline
7/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Description of a Case Study at IST**Instituto Superior Técnico, Technical University of Lisbon
Modules:• Technical Drawing and Geometrical
Modelling I & II
• 1st year students• 2 groups/module, ~50 students/module
• BEng / MEng in Mechanical, Aerospace and Marine Engineering
• 2010/2011 session
8/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Description of a Case Study at ISTTechnical Drawing and Geometrical Modelling I & II - Assessment
In-class tests30%
Group presenta-
tions10%
Group project40%
CAD* assign-ment10%
“Other” tests10%
*CAD – Computer Aided Design
In-class tests30%
Group presenta-
tions10%
Group project40%
CAD* assign-ment10%
“Other” tests10%
“Other” tests 10%
9/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
(“Other” tests – 10%)Every week in-class 15~30 mins assignments marked from 0 to 1
0 1
Wrong Right
Goals:• To engage the students in practice and study• Without significant increase of the marking load
How do we define the boundary in a binary marking scheme for drawings?
This slide is outside the main scope of the presentation (peer-assessment), but is shown for helping the author explain a concept.
10/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
(“Other” tests – 10%)Every week in-class 15~30 mins assignments marked from 0 to 1
0 1
Wrong Very good
Goals:• To engage the students in practice and study• Without significant increase of the marking load
How to differentiate a student that does not deliver the assignment from the others?
.5
Acceptable
This slide is outside the main scope of the presentation (peer-assessment), but is shown for helping the author explain a concept.
11/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
(“Other” tests – 10%)Every week in-class 15~30 mins assignments marked from 0 to 1
0 1
Zero
Goals:• To engage the students in practice and study• Without significant increase of the marking load
.5.25 .75
Poor Accept. Good Very good
2 levels of fail 3 levels of pass
Problems:• Poor feedback• “Ethical” question: marking is based on comparison
This slide is outside the main scope of the presentation (peer-assessment), but is shown for helping the author explain a concept.
12/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
(“Other” tests – 10%)Examples of “other” tests
This slide is outside the main scope of the presentation (peer-assessment), but is shown for helping the author explain a concept.
1 0.75 0.5
13/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Description of a Case Study at ISTTechnical Drawing and Geometrical Modelling I & II - Assessment
In-class tests30%
Grouppresenta-
tions10%
Group project40%
CAD* assign-ment10%
“Other” tests10%
*CAD – Computer Aided DesignLecturer
5%Peer-assessment
5%
In-class tests30%
Public presenta-
tions10%
Group project40%
CAD* assign-ment10%
“Other” tests10%
Group presentations
10%
14/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Description of a Case Study at IST
• Done in-class, during last week
• 10~15 minutes/group
About the public presentation
• Students had to attend all the presentations
• Marks ranged from 1 to 10 (0 for no shows)
• Peer-assessment marks kept confidential by the lecturer
• Non-differentiating peer-assessment marks were rejected
1-2 9-105-63-4 7-8Very Poor Poor Accept. Good Very good
15/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
1. Why student’s peer-assessment?2. Description of a case study at IST3. Some of the students’ works4. Peer-assessment results5. Conclusions
Outline
16/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Mower
Rodrigo Ferreira
Boring mill
Wilson CarreiraHélder Camacho
Filipe Moreira
Some of the students’ works
Reactive suspension
Leonor NetoSoraia Ribeiro
17/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Some of the students’ works
A Racing bike!!! (Solidworks)
(Author deliberately left anonymous)
18/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
1. Why student’s peer-assessment?2. Description of a case study at IST3. Some of the students’ works4. Peer-assessment results5. Conclusions
Outline
19/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
First observation:• Both the teacher’s and students’ marks follow a Gaussian
distribution.
20/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
21/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
The students did not fail their colleagues
22/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
The two students with very poor presentations are given a pass by their
peers...
23/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
... and the four students just below pass were given a comfortable pass by their peers
24/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
72% of the marks given by the
students are in the 7-8 (Good) range
25/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
However, the students seem to
be afraid to overrate too much their colleagues
26/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
• The averaged values are not very different• The teacher’s standard deviation is practically twice as much
as the peers’
27/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Peer-assessment results
The marks are concentrated around the same mode, although the teacher’s assessment “leaks” towards the whole marking range.
28/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
1. Why student’s peer-assessment?2. Description of a case study at IST3. Some of the students’ works4. Peer-assessment results5. Conclusions
Outline
29/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Conclusions
Overall conclusions from this study:
• Students have a tendency to overrate their “mates”
• They are able to make a correct relative judgement between their peers’ performances (and thus themselves)
• Most often, they will avoid to:• Fail their “mates”• Give their peers topper marks
30/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
Conclusions
How honest are the students when assessing each other’s work?
Students move by their own code of conduct and live in a competition environment.
Qualitatively honest Quantitatively biased
31/31How honest are the students when assessing each other’s work?2012 Annual L&T Conference, Diogo Montalvão
THANK YOU
Diogo Montalvã[email protected]
Senior Lecturer, Dipl MSc PhD CEngSchool of Engineering & Technology
Acknowledgments are due to Prof. António Ribeiro, Assistant Professor at IST, for sharing the data from two of their groups of students.