Reflexive learning, socio-cognitive conflict and peer-assessment to improve the quality of feedbacks in online tests

Franck Silvestre, Philippe Vidal, Julien Broisin IRIT, University Toulouse III, France

EC-TEL 2015 September 2015

1

Context of research

• Computer-based assessment

• Online self-assessment tests

• Feedback

• Tsaap-Notes

2

The challenge

How to qualify students’ contributions in order to provide enhanced feedback in online self-assessment tests?

3

Benefits of online tests

• Facilitate the provision of frequent formative assessments [12] [9]

• Provide learners with feedback - any time, - from everywhere, - as often as they want [10]

4

Quality of feedback is a key factor of success for better learning [2][7][8]

Issues about feedback

5

• It is provided by teachers - teacher’s dialect - time consuming

• The same feedback is provided to all learners

Tsaap-Notes [18]• Collaborative note-taking

system

• Micro-blogging, scope and fragment

• Audience response system

• Notes are taken by students after the presentation of the results

6

The « Notes as feedback » approach

• To reuse questions asked during lectures

• To reuse notes taken by students

• In order to generate self-assessment tests providing feedback coming from students notes

7

Semi-automa

tic

How does it work?

8

Result in Moodle

9

Benefits

• Creation of online self-assessment tests is easier

• Feedback is written in students’ dialect

• Promotion of students’ contributions

• Encouraging results from first experimentation [16]

10

Two limits

• The students’ notes are not evaluated nor filtered, there is no ”quality check” of the feedback => qualitative limit

• Only few students participate in the writing of explanations => quantitative limit

11

Toward the NP-Q feature

• Why more than 80% of students do participate in interactive questions, whereas only 25% of them take notes?

12

Success of interactive questionsAudience Response Systems provide

• formative assessments

• immediate feedback for each student

• positive impacts on learning outcomes

[4,6,14,18]

13

How to provide students with immediate feedback about their explanations?

The NP-Q feature

• Submission of an answer in several phases

• Immediate feedback

• Peer assessment of the students’ explanations

• Only best graded contributions are selected for feedback in online tests

15

Phase 1 - First answer submission

16

Phase 2 - Socio-cognitive conflict

17

Alternative answer provided by a peer

The second submission form

Phase 3 - Peer assessment

18

The presentation of the results

The peer assessment form

Final view

19

The list of graded explanations

In Moodle

20

Focus on the algorithm for the explanations’ association

21

a1[ 100; 2; 92 ]

a2[ 0; 3 ; 24 ]

a3[ 100; 3; 0 ]

a4[ 100; 4 ; 65 ]

a5[100; 3 ; 86 ]

a6[ 40; 2 ; 153 ]

a7[ 65; 1 ; 0 ]

a8[ 100;2 ; 38 ]

a1[ 100; 2 ; 92 ]

a4[ 100; 4 ; 65 ]

a5[ 100; 3 ; 86 ]

a8[ 100; 2 ; 38 ]

a2[0; 3 ; 24 ]

a6[40; 2 ; 153 ]

A

AcAc a1[ 100; 2 ; 92 ]

a3[ 100; 3; 0 ]

a4[ 100; 4; 65 ]

a5[ 100; 3 ; 86 ]

a8[ 100; 2 ; 38 ]

a2[0; 3 ; 24 ]

a6[40; 2 ; 153 ]

a7[ 65; 1 ; 0 ]

CcCc

a1[ 100; 2 ; 92 ]

a4[100; 4 ; 65 ]

a5[ 100; 3 ; 86 ]

a8[100; 2 ; 38 ]

a2[0; 3 ; 24 ]

a6[ 2 ; 153 ]

Ac a1[ 100; 2 ; 92 ]

a3[ 100; 3; 0 ]

a4[ 100; 4 ; 65 ]

a5[ 100; 3 ; 86 ]

a8[ 100; 2 ; 38 ]

a2[0; 3 ; 24 ]

a6[40; 2 ; 153 ]

a7[65; 1 ; 0 ]

Cc

Ac

Cc

2

1

fg

First experimentation

• 2 students’ groups

• First year of Computer Science Master

• Two different courses (DCLL and MA)

• 3 sessions of 2 hours

• 5 NP-Q questions

22

Participation rate

23

Socio-cognitive conflict

24

Peer assessment

25

Conclusion

• Students contributions are now qualified

• High participation rate in written tasks

• Encouraging results from the first experimentation

26

Perspectives

• Consideration of faulty peer-assessments

• Consideration of open questions: how to associate answers in this context?

• Scaling: big auditorium, MOOC

27

• Thanks for your attention! • Questions?

Bibliography1. Biggs, J., Tang, C.: Teaching for quality learning at university: What the student does. McGraw-Hill Education (UK) (2011) 2. Black,P.,Wiliam,D.:Assessmentandclassroomlearning.Assessmentineducation 5(1), 7–74 (1998) 3. Boud, D., Cohen, R., Sampson, J.: Peer learning in higher education: Learning from and with each other. Routledge (2014) 4. Caldwell, J.E.: Clickers in the large classroom: Current research and best-practice tips. CBE-Life Sciences Education 6(1), 9–20 (2007) 5. Dochy, F., Segers, M., Sluijsmans, D.: The use of self-, peer and co-assessment in higher education: A review. Studies in Higher education 24(3), 331–350 (1999) 6. Gauci, S.A., Dantas, A.M., Williams, D.A., Kemm, R.E.: Promoting student-centered active learning in lectures with a personal response system. Advances in Physiology Education 33(1), 60–71 (2009) 7. Hattie, J., Jaeger, R.: Assessment and classroom learning: A deductive approach. Assessment in Education 5(1), 111–122 (1998) 8. Higgins, R.: Be more critical: rethinking assessment feedback.. DYE. V, PC4000. (2000) 9. Miller, T.: Formative computer-based assessment in higher education: The effectiveness of feedback in supporting student learning. Assessment & Evaluation in Higher Education 34(2), 181–192 (2009) 10. Nicol, D., Milligan, C.: Rethinking technology-supported assessment practices in relation to the seven principles of good feedback practice. Innovative assessment in higher education pp. 64–77 (2006) 11. QTI, I.: Question and test interoperability (2006) 12. Ricketts, C., Wilks, S.: Improving student performance through computer-based assessment: Insights from recent research. Assessment & evaluation in higher education 27(5), 475–479 (2002) 13. Scouller, K.: The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education 35(4), 453–472 (1998) 14. Shaffer, D.M., Collura, M.J.: Evaluating the effectiveness of a personal response system in the classroom. Teaching of Psychology 36(4), 273–277 (2009) 15. Silvestre, F., Vidal, P., Broisin, J.: Tsaap-notes–an open micro-blogging tool for collaborative notetaking during face-to-face lectures. In: Advanced Learning Tech- nologies (ICALT), 2014 IEEE 14th International Conference on. pp. 39–43. IEEE (2014) 16. Silvestre, F., Vidal, P., Broisin, J.: Semi automatic generation of online tests providing feedbacks based on collaborative note taking. In: Advanced Learning Technologies (ICALT), 2015 IEEE 15th International Conference on. IEEE (to appear) 17. Topping, K.: Peer assessment between students in colleges and universities. Review of educational Research 68(3), 249–276 (1998) 18. Uhari, M., Renko, M., Soini, H.: Experiences of using an interactive audience response system in lectures. BMC Medical Education 3(1), 12 (2003)

29