Final PPT Balagtas

Literacy Level on Educational Assessment of Students in a Teacher

Education Institution: Basis for a Capability

Building Program

Researchers/PresentorsMarilyn U. Balagtas Antonio G. Dacanay Maville A. Dizon

Rodrigo E. Duque

Research Problems1. What is the level of assessment

literacy of the following identified groups of students in a premiere teacher education institution?

a. Undergraduate BEED studentsb. Undergraduate BSE studentsc. Students Taking Certificate in Teaching

Program d. Graduate Students taking M.A. in

Education2. Is there a significant difference in

the assessment literacy of the four groups of students in a premiere teacher education institution?

Research Problems

3. What capability building program is appropriate for the identified groups based on the results of the study?

Research DesignDescriptive – Survey

ResearchParticipants of the StudyA total of 457 students

participated in this study who were chosen using the convenience sampling technique

Methodology

Participants

Groups of Students Surveyed in Groups of Students Surveyed in this TEI during the SY 2009 -2010this TEI during the SY 2009 -2010

NN %%

Undergraduate BEED Students Undergraduate BEED Students 90 90 19.7 19.7 Undergraduate BSE Students Undergraduate BSE Students 100 100 21.9 21.9 Students taking Certificate in Students taking Certificate in

Teaching Program (CTP) Teaching Program (CTP) 125 125 27.4 27.4

Graduate Students taking MA in Graduate Students taking MA in Education Education

142 142 31.1 31.1

Total Total 457 457 100.0 100.0

Research ProcedurePre- Survey Period – planning of the collaborative research; review and validation of the adapted instrument; tryout of the instrument ; test of reliability of the instrument

Research Instrument35- item Assessment Literacy

Questionnaire adapted from Plake & Impara (1997)

Methodology

Research ProcedureSurvey Period – administering

the assessment literacy questionnaire to the students of the researchers

Methodology

Post - Survey Period – data gathering, organization and interpretation of resultsData Analysis

Frequency Count, Percentage, Mean, SD, ANOVA

Results1. What is the level of assessment literacy of

the following identified groups of students in a premiere teacher education institution?a. Undergraduate BEED students

NMean

(35 items) Std. Deviation Minimum MaximumRank

BEED 90 14.98 3.190 8 23 4

BSE 100 15.08 3.129 8 23 3

CTP 125 15.14 4.250 5 24 2

Graduate 142 15.48 3.558 7 25 1

Total 457 15.20 3.602 5 25

The BEED has the smallest mean score of 14.98, which is slightly lower than the over-all mean score of 15.20. Its standard deviation of 3.19 reveals that the students tested were more homogeneous than the other groups surveyed with a minimum score of 8 and a maximum score of 23.

ResultsAssessment Literacy of Undergraduate BEED Students

NMean


BEED 90 14.98 3.190 8 23 4

BSE 100 15.08 3.129 8 23 3

CTP 125 15.14 4.250 5 24 2

Graduate 142 15.48 3.558 7 25 1

Total 457 15.20 3.602 5 25

Their mean score tells that they answered correctly just about 43% of the items in the test, which has not even reached half of the number of items given in the test. If their score is to be interpreted with reference to the passing score set in Philippine schools, which is 75%, the result reveals the BEED students failed the exam. This implies that the BEED students have not gained the necessary competence to understand their future role as assessors of students’ learning.


Competency No.Competency Statement Minimum Maximum Mean SD

Rank

1 choosing assessment methods appropriate for instructional decisions 0 5 2.33 1.05 2

2 developing assessment methods appropriate for instructional decisions;

0 5 2.51 1.20 1

3 administering, scoring, and interpreting the results of both externally- produced and teacher-produced assessment methods;

0 4 1.90 1.02 6

4 using assessment results when making decisions about individual students, planning instruction, developing curriculum, and improving schools

0 5 2.26 1.01 5

5 developing valid student grading procedures; 0 5 2.27 1.09 3.5

6 communicating assessment results to students, parents, other lay audiences, and other educators;

0 5 1.44 .98 7

7 recognizing unethical, illegal, and other inappropriate methods and uses of assessment information.

0 5 2.27 1.05 3.5

Total Score8 23 14.98 3.19


Competency No. Competency Statement Minimum Maximum Mean SD

Rank


0 5 2.51 1.2 1

This is their strongest area, which could be explained by the fact that the BEED students were surveyed after they have taken their 3-unit assessment course where they were exposed to varied assessment tools and were provided the experience to develop tests, rubrics, and assessment tools as requirements in their course.


Competency No. Competency Statement

Minimum Maximum Mean SD Rank


0 5 1.44 .98 7

This is their weakest area which could be explained by the fact that the BEED students were still at their pre-service stage at the time of the survey, so they had no hands-on experience in applying their lessons in this area of competency.


the following identified groups of students in a premiere teacher education institution?

b. Undergraduate BSE students N

Mean(35 items) Std. Deviation Minimum Maximum

Rank

BEED 90 14.98 3.190 8 23 4

BSE 100 15.08 3.129 8 23 3

CTP 125 15.14 4.250 5 24 2

Graduate 142 15.48 3.558 7 25 1

Total 457 15.20 3.602 5 25

The mean of 15.08 tells that they answered correctly just about 43% of the items in the test, which has not even reached half of the number of items given in the test. If their score is to be interpreted with reference to the passing score set in Philippine schools, which is 75%, the result reveals that the BSE students failed the exam. This implies that the BSE students have not gained the necessary competence to understand their future role as assessors of students’ learning.

Results

Competency No. Competency Statement Minimum Maximum Mean SD Rank

1 choosing assessment methods appropriate for instructional decisions

0 5 2.45 1.086 3


0 5 2.50 .969 1


0 5 1.92 .9506


0 5 2.09 1.065 5

5 developing valid student grading procedures; 0 5 2.46 1.167 2


0 4 1.39 .9847


0 5 2.27 1.325 4

Total Score8 23 15.08 3.129

Assessment Literacy of Undergraduate BSE Students

ResultsAssessment Literacy of Undergraduate BSE Students

Competency No.

Competency Statement Minimum MaximumMean SD

Rank


0 5 2.50 .97 1

This is their strongest area, which could be explained by the fact that just like the BEED, the BSE students were surveyed after they have taken their 3-unit assessment course where they were exposed to varied assessment tools and were provided the experience to develop tests, rubrics, and assessment tools as requirements in their course.

ResultsAssessment Literacy of Undergraduate BSE Students

Competency No.

Competency Statement Minimum Maximum Mean SD Rank


0 4 1.39 .98 7

This is their weakest area which could be explained by the fact that just like the BEED, the BSE students were at their pre-service stage at the time of the survey, so they had no hands-on experience in applying their lessons in this area of competency.



c. Non-Education Degree Holders taking CTP

NMean


BEED 90 14.98 3.190 8 23 4

BSE 100 15.08 3.129 8 23 3

CTP 125 15.14 4.250 5 24 2

Graduate 142 15.48 3.558 7 25 1

Total 457 15.20 3.602 5 25

The mean of 15.14 tells that they answered correctly just about 43% of the items in the test, which has not even reached half of the number of items given in the test. The SD also indicates that the CTP students are the most heterogeneous group of students surveyed. This could be explained by the diverse background that they have. If their score is to be interpreted with reference to the passing score set in Philippine schools, which is 75%, the result also reveals that the CTP students failed the exam. This implies that the CTP students have not gained the necessary competence to understand their future role as assessors of students’ learning.

ResultsAssessment Literacy of CTP Students

Competency No. Competency Statement Minimum Maximum Mean SD

Rank


2 developing assessment methods appropriate for instructional decisions; 0 5 2.43 1.01 2


0 4 2.03 .92 4


0 5 1.94 1.11 6

5 developing valid student grading procedures; 0 5 2.29 1.22 5

6 communicating assessment results to students, parents, other lay audiences, and other educators; 0 5 1.59 1.23 7


0 5 2.46 1.43 1

Total Score

5 24 15.14 4.25

An examination of their performance in the 7 areas of competency reveals that the CTP students have not gained mastery in these areas as indicated by their mean score that is as low as 1.59 and as high as 2.46, all of which are below 2.5. This could be explained by the limited courses where the students could develop deliberately or not the competencies in educational assessment.



d. Graduate Students taking M.A in Education

NMean


BEED 90 14.98 3.190 8 23 4

BSE 100 15.08 3.129 8 23 3

CTP 125 15.14 4.250 5 24 2

Graduate 142 15.48 3.558 7 25 1

Total 457 15.20 3.602 5 25

The mean of 15.48, is the highest among the four groups surveyed, with an SD that indicates that they are the 2nd to the most heterogeneous groups of students surveyed, which could be explained by the diverse background that they have.



d. Graduate Students taking M.A in Education

NMean


BEED 90 14.98 3.190 8 23 4

BSE 100 15.08 3.129 8 23 3

CTP 125 15.14 4.250 5 24 2

Graduate 142 15.48 3.558 7 25 1

Total 457 15.20 3.602 5 25

The mean of 15.48, means that they answered correctly just about 43% of the items in the test, which has not even reached half of the number of items given in the test. If their score is to be interpreted with reference to the passing score set in Philippine schools, which is 75%, the result reveals the graduate students failed the exam. This implies that majority of the graduate students, in spite of the fact that they are all in-service teachers, have not gained the necessary competence to perform effectively their role as assessors of students’ learning.

ResultsAssessment Literacy of the Graduate Students in MA in Education

However, an examination of their performance in the 7 areas of competency reveals that the graduate students, compared with the other groups surveyed, have gained mastery in two areas, which show a mean score that is higher than 2.5. This could be explained by the fact that they are already teaching at the time the study was conducted.

Competency Number Competency Statement Minimum Maximum Mean SD Rank


2 developing assessment methods appropriate for instructional decisions 1 5 2.68 1.062 1

3Administering, scoring, and interpreting the results of both externally – produced and teacher-produced assessment methods

0 4 2.08 0.900 6

4Using assessment results when making decisions about individual students, planning instruction, developing curriculum, and improving schools

0 5 2.12 1.021 5

5 Developing valid student grading procedures 0 5 2.33 1.303 3

6 communicating assessment results to students, parents, other lay audiences, and other educators 0 5 1.30 0.921 7

7Recognizing unethical, illegal and other inappropriate methods and uses of assessment information

0 5 2.32 1.241 4

Total Score 7 25 15.48 3.558

ResultsCompetency No.

Competency Statement Minimum Maximum MeanSD

Rank


1 5 2.68 1.06 1

The graduate students are better in comparison with the other groups when it comes to this competency and this can be attributed to their being experienced teachers in the field. It is worthy to note that 59.78 % of them have at least a teaching experience from one (1) year to five (5) years and 40.22 % of them have teaching experience from six (6) years to twenty five (25) years at the time of the survey, and so they are expected to know better than the undergraduate students or even CTP students when it comes to developing and choosing assessment method appropriate for instructional decisions.

Assessment Literacy of the Graduate Students in MA in Education

Results

Competency No.

Competency Statement Minimum Maximum Mean SD Rank


0 5 1.30 0.92 7

This is their weakest area, which is the same as the other 3 groups surveyed. The result could be explained by the fact that graduate students, even if they are already teaching and that they usually conduct conferences with parents at least once every grading period, may not have the opportunity to explain individually to the parents and other stakeholders the meaning of the grades that their students received because of the big number of classes that they handle.

Assessment Literacy of the Graduate Students in MA in Education

Results

2. Is there a difference in the assessment literacy of the four groups of students in a premiere teacher education institution?

The computed F-value of 0.443 is found not significant at 0.05 level of confidence. This is also true when the four groups were compared in each of the areas of competency. This means that the students surveyed, whether already teaching or not, are weak at assessing students’ learning, which is the most critical role that a teacher performs.

Results 3. What capability building program is

appropriate for the identified groups based on the results of the study?

There is a need to increase the number of units for the course on educational assessment in this institution. The 3-unit course that they offer for educational assessment should be made 6 units at least as recommended in the CMO 30 series of 2004 to give the teachers and the students ample time to discuss all their learning experiences that will enable them to develop all the seven areas of competency needed in assessing learning.

For the undergraduate and CTP students

For the undergraduate and CTP students There is a need to review the emphasis of

the professional education course on assessment of learning so that the topics and the experiences provided to would-be-teachers are geared towards the development of the seven areas of competence in educational assessment . There is a need to provide would-be-teachers opportunities to actually experience what they learn in the classroom to improve their competence in assessing learning.



For the BEED and BSE students only There is a need to review the sequence of

all professional and specialization courses in the teacher education curriculum to ensure that they support each other and make the students become competent in performing their future role as a teacher including their role of assessing their students’ learning.



For the Graduate students

There is a need for a course on assessment of learning in all areas of specialization in the graduate programs to enable the teachers to further develop their competence in assessing their students’ learning.

There is a need to review the curriculum of this institution in their MA in Measurement and Evaluation to ensure that all these competencies are developed and equally emphasized.




There is a need to organize trainings, seminars, workshops, or focused group discussions that could be attended by teachers regardless of their years in teaching at least once a year to have a venue to discuss and reflect on their practice and get feedback that will make them perform effectively their role as assessors of students learning.

There is a need to intensify the promotion of the MA program with specialization in Measurement and Evaluation (M & E) to produce more teachers in the field who could serve as mentors or coaches in assessing learning to non-M &E specialist teachers.



Conclusions1. What is the level of assessment literacy of


Undergraduate students (BEED and BSE)

The undergraduate students (BEED and BSE) in this premiere teacher training institution are strong in developing assessment methods appropriate for instructional decisions.




1) choosing assessment methods appropriate for instructional decisions2) Administering, scoring, and interpreting the results of both externally –

produced and teacher-produced assessment methods;3) Using assessment results when making decisions about individual

students, planning instruction, developing curriculum, and improving schools;

4) Developing valid student grading procedures;

5) Communicating assessment results to students, parents, other lay audiences, and other educators; and

6). Recognizing unethical, illegal and other inappropriate methods and uses of assessment information.

The undergraduate students (BEED and BSE) in this premiere teacher training institution are weak at the following:




In general, the undergraduate students (BEED and BSE) in this premiere teacher training institution who only earned a 3-unit course on Assessing Learning in their Teacher Education Curriculum are not literate enough to perform their future role as assessors of students’ learning.



Non-Education Degree Holders taking CTP

All the non-education degree holders taking a certificate in teaching program (CTP) in this premiere teacher training institution who only earned a 3-unit course on Assessing Learning in their Teacher Education Curriculum are not literate enough to perform their future role as assessors of students’ learning. They are weak in all the seven areas of competence in educational assessment.



Graduate Students in M.A in Education

The graduate students surveyed in this premiere teacher training institution who took their bachelor’s degree in education in different teacher education institutions in the country and who only earned a 3-unit course on Measurement and Evaluation in their Teacher Education Curriculum but who have been in the teaching service for years are competent in two areas of competence such as:

1) developing assessment methods appropriate for instructional decisions; and

2) choosing assessment methods appropriate for instructional decisions.




However, in spite of their teaching experience, they are weak at the following areas of competence:

1) Administering, scoring, and interpreting the results of both externally – produced and teacher-produced assessment methods;

2) Using assessment results when making decisions about individual students, planning instruction, developing curriculum, and improving schools;

3) Developing valid student grading procedures;

4) Communicating assessment results to students, parents, other lay audiences, and other educators; and

5). Recognizing unethical, illegal and other inappropriate methods and uses of assessment information.




In general, the graduate students surveyed in this premiere teacher training institution who took their bachelor’s degree in education in different teacher education institutions in the country and who only earned a 3-unit course on Measurement and Evaluation in their Teacher Education Curriculum are not literate enough to perform their future role as assessors of students’ learning.

Conclusions2. Is there a difference in the assessment

literacy of the four groups of students in a premiere teacher education institution?

The would-be-teachers and in-service teachers surveyed in this premiere teacher education institution do not differ significantly in their assessment literacy regardless of their teacher education program and whether they have or no teaching experience at all.

Conclusions 3. What capability building program is


The present undergraduate teacher education curriculum of this premiere teacher training needs revision to increase its units on educational assessment and to improve the sequence and focus of professional education and specialization courses so that they support the development of the assessment competencies of would-be-teachers .

For the undergraduate and CTP students


Graduate students, regardless of their years in teaching, need trainings, seminars, workshops, or focused group discussions on educational assessment.

All MA programs in Education in this premiere teacher education institution need a course on educational assessment.

The MA program with specialization in Measurement and Evaluation (M & E) is needed to produce more teachers in the field who could serve as mentors or coaches in assessing learning to non-M &E specialist teachers.

Conclusions 3. What capability building program is


Recommendations1. There is a need to revise the present

curriculum of this TEI in all its teacher education programs, whether undergraduate or graduate, to address the need to improve the competency literacy of all its students.

2. There is a need to conduct another study to improve the administration of the assessment literacy test and to include other variables that could explain the status of the assessment literacy of the pre-service and in-service teachers both in public and private schools.

3. There is a need to expose pre-service and in-service teachers to various capability building programs in educational assessment.

Thank you!

Documents

Final PPT Balagtas