The Value of a Self-assessment The Value of a Self-assessment Rubric in Online Discussion Rubric in Online Discussion

ForumsForumsDr. Rafi Davidson

Center of Educational Technology Initiatives

Dr. Annette Kahan Communication & Information Educational Center

Kaye College of Education, Beer-Sheva, Israel

25-28 June 2007, Beer-Sheva, Tel-Aviv

“Knowledge is located neither in the mind nor in any representation of the mind. All of our understandings are situated in complex webs of experience, action, and interaction. Knowledge is a dynamic, evolving phenomenon, a fabric of relations in which one individual is fundamentally entwined with all others in a community” (Martin Ryder , 1999).

http://carbon.cudenver.edu/~mryder/reflect/enactivism.html

The Concept of Collaborative The Concept of Collaborative Knowledge BuildingKnowledge Building

Knowledge building may be defined as “The production and continual improvement of ideas of value to a community, through means that increase the likelihood that what the community accomplishes will be greater than the sum of individual contributions and part of broader cultural efforts” (Scardamalia & Bereiter, 2003).

The use of networked computers provides alternatives to traditional teaching and learning as we move from a single-classroom concept to the concept of a knowledge-building community of learners.

The importance of online forums The importance of online forums

in collaborative knowledge in collaborative knowledge buildingbuilding

Online discussion forums are one of the most common and accepted tools for collaborative learning in virtual courses.

The forums are a shared online environment that enables collaborative knowledge building and supportive social communities (Harasim 1989).

But Mason and others (1993) point out that

dialogue in online forums is disappointing in many instances, since the quality of the learning and the knowledge building is poor.

So One of the greatest challenges is promoting

discourse and developing a high level of discussion and critical thinking in the virtual environment, that will produce meaningful learning, as comprehensive as possible.

The Importance of Self-assessment The Importance of Self-assessment in Teacher Educationin Teacher Education

Use of the self-assessment rubric encourages pre-service and in-service teachers to develop the abilities of critical thinking, reflection and evaluation, abilities that are very important in the teaching profession.

Self-assessment is very effective for learners seeking to improve their knowledge and learning strategies (Shaklee et al. 1997), particularly in a collaborative learning setting.

1!

What is our innovationWhat is our innovation??

Models for online discussion self-assessment usually provide criteria for the learners but do not require their active decision-making and reflection on the quality of the posts.

2!

Therefore we developed a rubric Therefore we developed a rubric for self-assessmentfor self-assessment

To attempt to create a significant online learning community.

To encourage collaborative knowledge building. To assist learners in recognizing their commitment. To encourage re-organization of the discussion in a

relevant manner. To transfer the responsibility of evaluation to the

learner. To develop the abilities of reflection and evaluation. To ease the lecturer’s workload.

3!

The rubric was used in the following courses in a college of education: 1. ICT in Teaching and Learning

2. Information Literacy in Education

3. Teaching Heterogeneous Classrooms

Stages in the Development Stages in the Development of the Self-assessment of the Self-assessment RubricRubric Year %of Final

GradeEvaluation and Evaluator

2000Until

10% - 5%Participation (amount of posts)

Evaluation by lecturer only

200115%A few qualitative and quantitative criteria based on participation and contribution to learning -

Evaluation by lecturer only

200220%Self-assessment rubric prepared by lecturer

Evaluation by lecturer only

200330%Self-assessment rubric improved - based on students’ comments

Self- evaluation by students with monitoring by lecturer (students determined their own grade)

2004-2006

30-60%Self-assessment rubric

Self- evaluation by students with monitoring by lecturer (students determined their own grade)

Steps in the use of the rubric

The rubric is presented to the students at the beginning of the online courses.

During the courses, the students collect their own postings and evaluate their contributions to the knowledge building of their fellow students.

The final grades in the courses are a combination of the student's and the lecturer’s evaluation.

4!

Rubric CategoriesRubric Categories

1. Personal opinion or idea - Offering a reasoned opinion based on reliable sources.

2. Responding to others - Relevant and supported response which adds to what others have said

3. Referral to reliable sources of information 4. Summarizing a topic or issue discussed on

the forum – based on the discussion and contribution of others.

5. Helping peers to solve all kinds of problems6. Asking challenging questions

5!

Rubric Categories (continued)Rubric Categories (continued)

7. Taking personal initiatives in the forum. These may include:

1. Opening a new topic for discussion in a new forum, writing a document on the topic including sources which are presented to the participants

2. Opening a forum and directing the discussion with peers

3. Summarizing the discussion

6!

Student's perspective on the Student's perspective on the value of the self-assessment value of the self-assessment

rubricrubric(Based on content analysis of

students’ responses to the questions on the value of the self assessment

rubric)

Student's perspective on the Student's perspective on the

value of the self-assessmentvalue of the self-assessment

The students referred to the following contributions:

Motivation Transparency and Fairness of Assessment Collaborative Knowledge Building Self Directed Learning Life Skills Professional Development as Teachers

7!

MotivationMotivation

“Gives the student a huge motivation to learn”

“Gives the student a clearer picture of the amount of his participation and his contribution to learning"

“The rubric is a way to give students a push to learn and to evaluate the process they are going through.“

Transparency and Fairness Transparency and Fairness

ofof AssessmentAssessment

“I realized that I have been very active in the various forums.”

“… know where I stand, and if I have done what was expected of me.”

"The student understands exactly why (and how) he got this grade.”

“I very much like the idea of ranking within each criteria “

Collaborative Knowledge Collaborative Knowledge BuildingBuilding

“In this way it is possible to find out the extent of our participation in the various forums and our contribution to other students’ learning”

Self Directed LearningSelf Directed Learning

“Working with the rubric is a process that causes us to look at ourselves critically.”

“Self-assessment changes the status of the student and makes him an active participant, who is held responsible for his contribution.”

“… It is obvious that the rubric is a necessary tool for learning, especially in self-learning.”

Life SkillsLife Skills

“Once they acquire the skill of self-assessment in learning, they can apply it to life in general.”

Professional Development as Professional Development as TeachersTeachers

“The rubric is an excellent opportunity to learn more about our pupil - his real ability to assess himself, his self-confidence…”

“I hope that most of us will try, and also succeed in implementing (self-assessment) in our future classrooms as teachers "

8!

Difficulties Expressed by Difficulties Expressed by Students using the RubricStudents using the Rubric

It takes a long time and a lot of effort to use the self-assessment rubric.

Having the rubric in mind while participating in the forums makes the discussion “forced” – it doesn’t flow freely.

The rubric restricts creativity and free choice.

ConclusionsConclusions

Students believe that the self-assessment rubric contributes to their knowledge building and their training to be teachers, increases social support, and allows greater visibility as to how they are evaluated in the courses.

Preliminary research of content analysis of forum posts shows that the tool raised the level of discussion and increased critical thinking and collaborative knowledge building.

9!

Where do we go from hereWhere do we go from here??

Adapting the rubric to the needs of

other online courses and student populations.

Developing online tools to facilitate the self-assessment process.

It is important to continue It is important to continue research in the following research in the following

areas:areas: Research on the effect of the rubric on

learning and motivation of students.

Research on the effect of the rubric on change in the role of the instructor.

Research on the development of concepts during discussion in forums.

Category

Number of contributionsEach contribution is worth a certain number

of points, based on its quality

Maximum number of points for each post

Maximum number of points for

each category

1 .Personal opinion/idea (in the opening message of the forum, not in response to others)

IContributing a relevant idea/opinion

1 point

IIContributing a relevant reasoned idea/opinionor one based on a reliable source

2 points

IIIContributing a relevant reasoned idea/opinion and basing it on reliable sources

3 points

3 points12 points

2 .Responding to participants

IVRelevant response-without explanation and justification

1 point

VRelevant and supported response which adds to what others have said (like an example or information from experience)

2 points

VISame as V but adding to responses of others and also including referral to additional source/s

3 points

3 points12 points

Category

Number of contributionsEach contribution is worth a certain number

of points, based on its quality

Maximum number of points for each post

Maximum number of points for

each category

3. Adding a post that refers to reliable sources of information

VIIReferral to relevant online source without description1 point

VIIIReferral to relevant online source with comments

2 points

2 points8 points

4. Summarizing a topic discussed in the forum – based on posts

IX Summarizing a topic discussed in the forum

3 points

XSummarizing a topic discussed in the forum andincluding additional information covered in the course

5 points

XISummarizing a topic discussed in the forum andincluding additional information from outside sources

8 points

8 points16 points

Category

Number of contributionsEach contribution is worth a certain number

of points, based on its quality

Maximum number of points for each post

Maximum number of points for

each category

5 .Asking challenging questions

XII Asking a question connected to the course and its purposes1 Point

XIIIAsking a question which received at least one answer from peers

2 Points

XIVAsking a question that received at least two responses from peers3 Points

3 points9 points

6 .Initiating a new discussion forum

XVRaising a topic for discussion

4 Points

XVIRaising a topic for discussion and moderating the discussion

6 Points

XVIIRaising a topic for discussion, moderating the discussion and summarizing the discussion10 Points

10 points30 points

Category

Number of contributionsEach contribution is worth a certain number

of points, based on its quality

Maximum number of points for each post

Maximum number of points for

each category

7 .Helping peers to solve problems – technical and course-related

XVIIIProviding a solution or idea to help resolve a simple problem

1 Point

XIXProviding a solution or idea to help resolve a complex problem

2 Points

2 points6 points