1. 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html Using game mechanics and social networking to motivate student contributions to a course Edward F. Gehringer Dept. of Computer Science North Carolina State University Your feedback welcome http://www4.ncsu.edu/~efg/form.html

2. Outline

The goal: Authentic assessment

Virtual demo

Social networking and game mechanics

Reputation systems

Summary

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 3. Authentic assessment

What is it?

Why is it hard?

How can we facilitate it?

• Student-generated content

• Peer review

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 4. Student-generated content

gets students working together to improve others learning experiences,

helps them learn, by performing tasks that are similar to real-world responsibilities,

gives them experience in writing their ideas up for an audience of their peers,

allows each cohort to stand on the shoulders of students in earlier classes.

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 5. Outline

The goal: Authentic assessment

Virtual demo

Social networking and game mechanics

Reputation systems

Summary

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 6. Virtual demo

Creating an assignment

Selecting a topic

Reviewing

Feedback from author to reviewer

Teammate review

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 7. Setting up a wiki assignment 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 10/30/09 8. Lets create some teams 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 10/30/09 9. Heres how we import it 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 10. Now there are five teams 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 11. Virtual demo

Setting up an assignment

Selecting a topic

Reviewing

Feedback from author to reviewer

Teammate review

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 12. Selecting a Topic 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 13. Virtual demo

Setting up an assignment

Selecting a topic

Reviewing

Feedback from author to reviewer

Teammate review

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 14. Lets review a wiki page 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 15. Click on the assignment 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 16. then on Others work 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 17. Choose a review 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 18. Read the submitted wiki page 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 19. Then fill out the review template 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 20. Save and view the review 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 21. Virtual demo

Setting up an assignment

Selecting a topic

Reviewing

Feedback from author to reviewer

Teammate review

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 22. Now the reviewee logs in 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 23. and views the scores 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 24. Click on View to expand review 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 25. Notice the link down at the bottom 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 26. Use it to give feedback to the reviewer 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 27. Then the reviewer can view the feedback 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 28. Virtual demo

Setting up an assignment

Selecting a topic

Reviewing

Feedback from author to reviewer

Teammate review

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 29. Review of teammates 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 30. Summary of feedback in Expertiza

Reviews

Feedback to author

Teammate reviews

Metareviews( xreviewsy s review ofz )

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 31. Instructor report 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 32. Outline

The goal: Authentic assessment

Virtual demo

Social networking and game mechanics

Reputation systems

Summary

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 33. How to motivate students?

Runners in a race are motivated by seeing other runners ahead of them.

Players in online games are motivated to catch the leaders.

Can we do the same thing in our peer-review system?

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 34. Motivating Students: Achievement System

Categories in we can track achievement

• Quality of submitted work

• Quality of reviews

• Helpfulness to authors

• Contribution to team

These factors make upreputation.

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 35. Recognizing Achievement: Leaderboards

Students can list their achievements for all to see.

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 36. Now a new review is entered 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 37. and the leaders change 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 38. A new metareview is entered 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 39. and a new leader is recognized 10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 40. What we have here is

a social network!

• Connections among many individuals.

• Information contributed from many sources (albeit coerced!)

• Can be used to discover competencies of individuals.

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 41. Future work: Message boards for extra credit

Studentaposts a question

Studentbresponds

Studentarates studentb s response

Others can rate response too.

• TAs can also respond

Prompt reviews/answers aremore helpful, so reward forthese

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 42. New Features Coming to Expertiza

Levels

• Special powerrearranging messages

• Microtasks/micropayments

• incentivize useful work

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 43. Outline

The goal: Authentic assessment

Virtual demo

Student-generated content

Social networking and game mechanics

Reputation systems

Summary

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 44. Are the evaluations reliable?

Author feedback

Metareviewing helps.

Algorithmic strategies

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 45. Algorithmic approaches

Strategies usequalityandleniencyas metrics for assigning grades.

Quality is dependent on leniency, and vice versa.

Their approach yields a system of simultaneous equations, which can be solved to yield values for quality.

Other researchers have devised similar methodologies.

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 46. Other reputation systems

What we have seen are reputation systems devoted to evaluatingrequiredwork.

Other reputation system deal withvoluntarywork.

Our hope is to blend the two.

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 47. Outline

The goal: Authentic assessment

Virtual demo

Student-generated content

Social networking and game mechanics

Reputation systems

Summary

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html 48. Summary

If students do real tasks, authentic assessment is possible.

Peer assessment provides a scalable approach to authentic assessment.

Incentives such as leaderboards and micropayments can be used to encourage student contributions.

A reputation system is needed for quality control.

10/30/09 Your feedback welcome: http://www4.ncsu.edu/~efg/form.html