Introduction

Peer Assessment and Teacher AssessmentPeer Assessment and Teacher Assessmentofof

Academic Oral PresentationsAcademic Oral Presentations

Achara Wongsothorn, Ph.D, ProfessorAchara Wongsothorn, Ph.D, Professor, , Chulalongkorn University Language InstituteChulalongkorn University Language Institute

IntroductionIntroduction

• Interaction between learners and teachers is one of the significant variables impacting learning outcomes of students. Interactions among learners and between the learners and their teacher are essential and conducive to learning success. Peer assessment coupled with teacher assessment of students’ English use is one kind of interaction in which students and the teacher jointly assign grades or scores for a task.

Being part of authentic assessment, Being part of authentic assessment, peer assessment allows learners to peer assessment allows learners to

participate in their own as well as their participate in their own as well as their peers’ learning through assessing and peers’ learning through assessing and evaluating targeted performances. The evaluating targeted performances. The process is continuous practice that can process is continuous practice that can create not only interest in learning but create not only interest in learning but

also motivation to improve oneself while also motivation to improve oneself while helping one’s peer know his/her helping one’s peer know his/her

strengths and weaknesses.strengths and weaknesses.

• Oral presentation of research proposals is one component of Chulalongkorn University Language Institute’s Academic English for Graduate Studies course (GE 5500-532). Students are required to develop group research proposals for in-class presentation to be evaluated by peers and the teacher. This group project is a collaboration of students who team together into groups of 2 to 4 members to find research topics of their joint interest.

• From the research topics, they develop research proposals and give group oral presentations to peers and teachers, who meanwhile evaluate their presentations using ten criteria. The activities that the students are engaged in follow the collaborative learning approach. According to Wikipedia, collaborative learning is a method of teaching and learning in which students team together to explore a significant question or create a meaningful project. They discuss and work together in the library, or on the Internet on a shared assignment. Each of them perform a common task, depend on each other and is held accountable to the success of their project.

Research objectivesResearch objectives

• 1. find the relationships among the ten

• oral presentation criteria as rated by peers .

• 2. discover the relationships between the

• teacher assessment and assessment

• by the whole group of students

Research MethodologyResearch Methodology

• Subjects• The subjects in this action research classroom-

based study were 21 students enrolled in the researcher’s CULI’s Academic English for Graduate Studies course in the first semester of the academic year 2549 (referred to as Cohort 3 in the present researcher’s study “Significant Learners’ Factors and English Language Learning: From Ecological Approach to ELT.”) They were graduate students from the Faculty of Education, Psychology, Science, Engineering, Arts, Pharmaceutical Science, and Nursing.

SubjectsSubjects

• The subjects were assigned a group project to present research proposal on the academic topic of their interest. They teamed up into groups of 3-5. The topics ranged from “Memory monitoring and the control of stereotype distortion," Job empowerment and organizational commitment of medical employees,”, “Influence of Khmer art in Thailand’s Northeastern province” to “The role of brief intervention in alcohol reduction among the moderate alcohol drinking community with different per centage of alcohol concentration” and “Teaching and learning of table tennis in multimedia on-line environment.”

Data AnalysisData Analysis

• Data used for the analyses were complete cases. The incomplete cases were dropped from the analyses. During each group presentation, both the teacher-researcher and other students rated oral presentation by each student in each group using the scale of 1 to 5 to signify the level of performance (1=poor, 2=fair, 3=good, 4=very good, 5= excellent) based on the following ten criteria: Evidence of research planning, Encouraging questions, Eye contacts, Appropriate gestures, Audience involvement,, Entertaining, Relevant and useful, Well-organized, Easy to understand, and Confident


• The rating instrument of academic oral presentation of research proposals is a standardized measure adopted for classroom use and for research purposes by the researcher. It is a standardized instrument for rating research project developed for use in this course, i.e., Academic English for Graduate Studies for over a decade. The research done earlier (Wongsothorn, 2006) including the present one confirm the instrument’s reliability and validity. Inter-rater reliability indices range from .80-.89 while content validity of the instrument is safeguarded by reviews of various textbooks and journals on oral rating scales before having the instrument validated by other teacher-researchers using item-objective congruence analysis. The content validity is at .95. The process of establishing the reliability and validity of the rubrics is common in quantitative research and is used for generalizability of the assessment results (Hafner & Hafner, 2003.)


• The data collected were analyzed using Pearson product moment correlations, sum of squares and cross products, and covariance. The set significant level is

• p≤ .05.

ResultsResults

• Research objective 1: find the relationships among the ten oral presentation criteria as rated by peers .

• From descriptive data, Pearson-product moment correlation analyses as presented in the inter-correlation matrices in the following tables confirm the significant relationships among the various features of oral presentation as assessed by students.

Inter-correlationsInter-correlations

It can be concluded from data presented in Tables 1 to 17 that the inter-correlations among the ten features of oral assessment scales as rated by 16 students totaling 720 pairs, 114 pairs are significant at p≤ .05 and 394 pairs are significant at p≤ .01. This means that the Seventy-one per cent of the correlations รis

statistically significant and only 29 per cent is not significant.

Six features of oral presentation Six features of oral presentation that consistently have no significant that consistently have no significant

relationships. They are:relationships. They are:

1. Evidence of research planning and audience

involvement

• 2. Encouraging questions and audience • involvement• 3. Entertaining and relevant and useful• 4. Entertaining and well-organized• 5. Entertaining and easy to understand• 6. Entertaining and confident

The ranking of correlation The ranking of correlation frequencies of the pairs frequencies of the pairs

• 1. 19 cases of significant relationships• Eye contact and audience involvement• Eye contact and entertaining• 2. 18 cases of significant relationships• Eye contact and relevant and useful• Eye contact and well-organized• 3. 17 cases of significant relationships• Evidence of research planning and

encouraging questions• Appropriate gestures and well-organized

4. 16 cases of significant 4. 16 cases of significant relationshipsrelationships

• Evidence of research planning and eye contact• Evidence of research planning and relevant and

useful• Encouraging questions and eye contact• Eye contact and appropriate gestures• Eye contact and easy to understand• Eye contact and confident• Appropriate gestures and audience involvement• Appropriate gestures and entertaining• Appropriate gestures and relevant and useful• Relevant and well-organized• Well-organized and easy to understand• Well-organized and confident• Easy to understand and confident


• Evidence of research planning and well-organized

• Evidence of research planning and easy to understand

• Evidence of research planning and confident

• Encouraging questions and appropriate gestures

• Encouraging questions and entertaining• Relevant and easy to understand

6. 14 cases of significant relationships

Audience involvement and entertaining Audience involvement and relevant

Audience involvement and well-organized

Relevant and useful and confident

7. 7. 11 cases of significant 11 cases of significant relationships relationships

• Audience involvement and easy to understand

• Audience involvement and confident

9. 9. 2 cases of significant 2 cases of significant relationships relationships

• Evidence of research planning and entertaining

• 10. 1 case of significant relationships• Encouraging questions and relevant• Encouraging questions and well-organized• Encouraging questions and easy to understand• Encouraging questions and confident• Appropriate gestures and easy to understand• Appropriate gestures and confident

Research objective 2Research objective 2

• 2. discover the relationships between the teacher assessment and assessment by the whole group of students

• The descriptive statistics report on the means and standard deviations of the rating done by peers and the teacher. The findings reveal that among the 45 pairs there are 32 pairs that are statistically significant—10 are at .05 and 23 are at .01. This means that 71 per cent of the correlations between teacher rating and peer rating is statistically significant and that only 29 per cent is not significant.

Finding of consistent no Finding of consistent no relationships between the features relationships between the features

for both the teacher and peersfor both the teacher and peers

• Entertaining and relevant

• Entertaining and well-organized

• Entertaining and easy to understand

• Entertaining and confident

• The ranking of correlation frequencies of the pairs

1. 19 cases of significant relationshipsEye contact and audience involvementEye contact and entertaining

2. 18 cases of significant relationshipsEye contact and relevant and usefulEye contact and well-organized

3. 17 cases of significant relationshipsEvidence of research planning and encouraging questionsAppropriate gestures and well-organized


•• Evidence of research planning and eye contact• Evidence of research planning and relevant and

useful• Encouraging questions and eye contact• Eye contact and appropriate gestures• Eye contact and easy to understand• Eye contact and confident• Appropriate gestures and audience involvement• Appropriate gestures and entertaining• Appropriate gestures and relevant and useful• Relevant and well-organized• Well-organized and easy to understand• Well-organized and confident• Easy to understand and confident


• Evidence of research planning and well-organized

• Evidence of research planning and easy to understand

• Evidence of research planning and confident• Encouraging questions and appropriate

gestures• Encouraging questions and entertaining• Relevant and easy to understand


• Audience involvement and entertaining

• Audience involvement and relevant

• Audience involvement and well-organized

• Relevant and useful and confident


• Audience involvement and easy to understand

• Audience involvement and confident

•

• 8. 2 cases of significant relationships

• Evidence of research planning and entertaining

9. 1 case of significant relationships9. 1 case of significant relationships

• Encouraging questions and relevant• Encouraging questions and well-

organized• Encouraging questions and easy to

understand• Encouraging questions and confident• Appropriate gestures and easy to

understand• Appropriate gestures and confident

Discussion 1Discussion 1

• It is interesting to note that the finding of consistently no significant relationships for both the teacher and peers’ ratings of academic oral presentation, especially for “entertaining”, which has no relationships with other more academic features. This may be because in the presentations, the students do not have the chance or the skill of entertaining the audience. The entertaining skill seems to be reserved for experienced presenters. The students might be very serious with their research proposals or lack the entertaining skill, which even for academic presentations can enliven the atmosphere and create the sense of friendliness.


• The findings of the two most frequent and consistent relationships (19 and 18 cases) involve “eye contact.” There are 19 cases of significant relationships between “eye contact and audience involvement” and “eye contact and entertaining” while there are 18 cases of significant relationships between “eye contact and relevant and useful” and “eye contact and well-organized.” Student presenters may be able to use “eye contact” effectively. This nonverbal communication emerges as a very significant variable impacting the audience’s perception/judgment of the presentations.


• The only one case of significant relationships found for “encouraging questions” and the “audience-oriented” features such as “appropriate gestures and easy to understand”. The finding may also be due to the fact that the presenters did not ask questions often enough, or not at all.


• The finding of a number of significant relationships between peer rating and teacher rating of student academic oral presentation can be regarded as a confirmation of the reliability of the rubrics and method of assessment as well as the usability of the scales.

RecommendationsRecommendations

• 1. The teacher and peers’ ratings have very frequent significant relationships implying that peers are reliable raters. Peers should be involved in assessing one another’s language use.

• 2. This physical touch seems to be impressive and has important impact on the peer and teacher audience. Student presenters may be able to use “eye contact” effectively. This nonverbal communication emerges as a very significant variable impacting the audience’s perception/judgment of the presentations.

Recommendations (cont.)Recommendations (cont.)

• 3. The proven reliability of the assessment rubrics and the method of peer assessment leads to further investigation of the method of peer assessment in relation to teacher assessment as well as paving the way for more possible application of this assessment method. Student may help lessen the assessment task for the teacher if they can do it reliably.

ReferencesReferences

• Hafner, John C. & Patti M. Hafner (2003). “Quantitative • analysis of the rubric as an assessment tool: an • empirical study of student peer- group rating,”

International Journal of Science Education, 25:12, 1509-1528.

• Wikipedia• Wongsothor, Achara (2006). “Significant Learners’ • Factors and English Language Learning: From • Ecological Approach to ELT.” Paper presented

at the Sixth CULI’s International Conference:Facing EFL Challenges, 27 November 2006 at the Ambassador Hotel, Bangkok.

•

Appendix: The Data for AnalysisAppendix: The Data for Analysis

• Please see details in the text file on the same topic.

Documents

Introduction