Factors Influencing Examination Results in Higher Education for Maximizing Validity and Improving Students’ Pass Rate in Examination System of Tribhuvan

Factors Influencing Factors Influencing Examination Results in Higher Examination Results in Higher Education for Maximizing Education for Maximizing Validity and Improving Validity and Improving Students’ Pass Rate Students’ Pass Rate in Examination System of Tribhuvan in Examination System of Tribhuvan UniversityUniversity

Ram Chandra PokharelRam Chandra PokharelHOD of Statistics DepartmentHOD of Statistics DepartmentPadma Kanys CampusPadma Kanys CampusTribhuvan UniversityTribhuvan University

IntroductionIntroduction

There are five universities in operation in Nepal- There are five universities in operation in Nepal- Kathmandu, Nepal Sanskrit, Pokhara, Purbanchal Kathmandu, Nepal Sanskrit, Pokhara, Purbanchal and Tribhuvan University (TU). They form basis of and Tribhuvan University (TU). They form basis of higher education in Nepal. higher education in Nepal.

Lumbini Buddha University (LBU) has recently been Lumbini Buddha University (LBU) has recently been established but student has not yet been enrolled in established but student has not yet been enrolled in it. it.

Besides, B. P. Koirala Institute of Health Science Besides, B. P. Koirala Institute of Health Science (BPKIHS) and NAMS have also been in operation at (BPKIHS) and NAMS have also been in operation at the university level. Among them, the TU and NSU the university level. Among them, the TU and NSU are entirely financed by the government. are entirely financed by the government.

Tribhuvan UniversityTribhuvan University

In 2007, the TU has 60 constituent and 288 affiliated campuses In 2007, the TU has 60 constituent and 288 affiliated campuses spread all over the country.spread all over the country.

It has 4 faculties (Law, Humanities and Social Sciences, It has 4 faculties (Law, Humanities and Social Sciences, Management and Education) and 5 Technical Institutes (Science Management and Education) and 5 Technical Institutes (Science and Technology, Engineering, Medicine, Forestry, and and Technology, Engineering, Medicine, Forestry, and Agriculture and Animal Science)Agriculture and Animal Science)

In 2006/07, the total student enrollment in TU constituent and In 2006/07, the total student enrollment in TU constituent and affiliated campuses is 1,32,777 and 70,672 respectivelyaffiliated campuses is 1,32,777 and 70,672 respectively

In 2006/07, TU conducted exam for 4,60,716 students including In 2006/07, TU conducted exam for 4,60,716 students including 2,19,186 regular, private and 2,41,530 partial students. 2,19,186 regular, private and 2,41,530 partial students.

TU is providing higher education to around 90 percent of the total TU is providing higher education to around 90 percent of the total students in the country. students in the country.

TU Examination SystemTU Examination System

Centralized and annual examination systems Centralized and annual examination systems are two basic characteristics of Tribhuvan are two basic characteristics of Tribhuvan University in student evaluation system and University in student evaluation system and the tool is essay examination. the tool is essay examination.

Except in case of some technical institute, no Except in case of some technical institute, no other evaluation techniques have been found other evaluation techniques have been found in practice. in practice.

Examination rules and Examination rules and regulation in practiceregulation in practice The same curriculum is adopted for all the students The same curriculum is adopted for all the students

of a particular level. of a particular level. All teachers with specified qualification can teach All teachers with specified qualification can teach

the course to an acceptable standard.the course to an acceptable standard. Students’ achievement in one academic year is Students’ achievement in one academic year is

assessed by the means of a set of few essay assessed by the means of a set of few essay questions in three/four hour’s long examination. questions in three/four hour’s long examination.

The students can demonstrate their achievement in The students can demonstrate their achievement in writing.writing.

The examiners can measure the scope and The examiners can measure the scope and standard of the students' achievement by means of standard of the students' achievement by means of written exam.written exam.

These points make one think that it is easy to earn degrees. But students find it difficult in real world. The success rate in

TU is relatively very low compared to those of other universities.

Students Pass Rate in Students Pass Rate in (2004/05) (2004/05) FacultiesFaculties YearYear Pass %Pass % # of students# of students

Management:Management:I.ComI.ComB.B.SB.B.S

I/III/III/II/IIII/II/III

26.33/27.8026.33/27.8014.27/25.46/45.2714.27/25.46/45.27

36014/2734936014/2734925043/17406/1074725043/17406/10747

Humanities:Humanities:I.A.I.A.B.A.B.A.


21.13/25.2521.13/25.2519.16/18.17/3519.16/18.17/35

64865/4570764865/4570723986/19664/1123223986/19664/11232

Education:Education:I. EdI. EdB. EdB. Ed


14.40/31.6314.40/31.6314.82/33.19/32.4114.82/33.19/32.41

11635/657611635/65761405/8833/55231405/8833/5523

Science:Science:I. Sc.I. Sc.B. Sc.B. Sc.


37.25/49.6637.25/49.6631.48/35.78/48.8531.48/35.78/48.85

6419/50146419/50143786/3057/20863786/3057/2086

Engineering:Engineering:B. E. (Civil)B. E. (Civil)

I/III/IIIII/IVIII/IV

65.88/54.3265.88/54.3257.82/69.5557.82/69.55

334/243334/243211/266211/266

Why these areas as the key Why these areas as the key dimension of the study?dimension of the study? Even pass percentage of TU students is Even pass percentage of TU students is

seriously low, no researches has been seriously low, no researches has been conducted in this area so far. conducted in this area so far.

There are certain factors affecting the result There are certain factors affecting the result which can be improved in such a way that the which can be improved in such a way that the students’ success rate in the exams of TU is students’ success rate in the exams of TU is increased without demolishing the present increased without demolishing the present examination administration system.examination administration system.

Research questionsResearch questions

To what extent does the To what extent does the students’ overall students’ overall readinessreadiness support their achievement level as support their achievement level as reflected in examination result?reflected in examination result?

To what extent does the students’ To what extent does the students’ prior knowledgeprior knowledge (entry behavior) support their examination result? (entry behavior) support their examination result?

To what extent do the To what extent do the teachers’ backgroundteachers’ background and and habits help the students’ achievement in term of habits help the students’ achievement in term of examination results? examination results?

To what extent on TU test items have To what extent on TU test items have validity and validity and reliabilityreliability??

Research questions cont…Research questions cont…

To what extent does the parents’ To what extent does the parents’ financial statusfinancial status affect their children’s score of exams?affect their children’s score of exams?

To what extent do the administrators, teachers, To what extent do the administrators, teachers, students, and guardians/parents students, and guardians/parents satisfiedsatisfied with the with the examination system of TU? examination system of TU?

What measures should be taken in What measures should be taken in improving the improving the student evaluation systemstudent evaluation system? ?

What students’ evaluation and assessment What students’ evaluation and assessment system system would bewould be more appropriatemore appropriate for maintaining quality for maintaining quality and improving students’ success rate for TU? and improving students’ success rate for TU?

Theoretical FrameworkTheoretical Framework

Motivation and AbilityMotivation and Ability

Student’s Student’s readinessreadiness

GovernanceGovernance

Social milieu & Social milieu & Social controlSocial control

TeacherTeacher

CurriculumCurriculum

Students with Entry Students with Entry BehaviorBehavior

Shared MeaningsShared Meanings

Test TheoryTest Theory

Test Test Instrument Instrument DevelopmentDevelopment

AchievementAchievement

Test Test ScoreScore

Valid Valid ExaminationExamination

SystemSystem

Educative Educative MaterialMaterial

AssessmentAssessment

Research Method and Research Method and Procedure Procedure This study employed cross-sectional survey This study employed cross-sectional survey

research method, and used quantitative research method, and used quantitative procedure for data analysis. procedure for data analysis.

This study was based on Tribhuvan This study was based on Tribhuvan University of Nepal. Source of in formation University of Nepal. Source of in formation and variables are given in the diagramand variables are given in the diagram

Variables and their sources Variables and their sources of informationof information

Source of Information Source of Information VariablesVariables StudentsStudents tests scores, entry behavior, test readiness, testtests scores, entry behavior, test readiness, test

satisfaction, attitude towards thesatisfaction, attitude towards theexamination systemexamination system

Teacher Teacher qualification, habits, attitude towards thequalification, habits, attitude towards theexamination system, opinion to improve theexamination system, opinion to improve theexamination systemexamination system

Parents/Guardians Parents/Guardians habits, socio-economic status, investments onhabits, socio-economic status, investments onchild, attitude towards the examination systemchild, attitude towards the examination system

Administrator Administrator attitude towards the examination system, opinion attitude towards the examination system, opinion to improve the examination systemto improve the examination system

Question quality Question quality relation to the test theory, relation of two parallel relation to the test theory, relation of two parallel test scorestest scores

Educators Educators attitude towards the examination system, opinion attitude towards the examination system, opinion to improve the examination systemto improve the examination system

Population and sample Population and sample

Participants in the study were students, Participants in the study were students, guardians, teachers, campus administrators guardians, teachers, campus administrators related to examinations and educators. related to examinations and educators. These four types of units were chosen after These four types of units were chosen after selecting the campus. selecting the campus.

A two- stage; multi-stage sampling procedure A two- stage; multi-stage sampling procedure was employed. First, campuses and then was employed. First, campuses and then students, their teachers, their students, their teachers, their guardians/parents and their exam guardians/parents and their exam administrators and university educators.administrators and university educators.

MethodologyMethodology

Questionnaires were the major tools for data Questionnaires were the major tools for data collection. Four separate sets of questionnaires collection. Four separate sets of questionnaires were developed for the collection of information from were developed for the collection of information from students, parents, teachers and administrators. students, parents, teachers and administrators.

One separate set of questionnaires were prepared One separate set of questionnaires were prepared to collect the information from educational experts. to collect the information from educational experts.

Questionnaires for the students were administrated Questionnaires for the students were administrated in the common classes after the permission of in the common classes after the permission of related campus authority. Instructions were given to related campus authority. Instructions were given to the students to take enough time what they need. the students to take enough time what they need.

Some students returned their questionnaires only Some students returned their questionnaires only next day with their gradients response. next day with their gradients response.

Methodology cont…Methodology cont…

Achievement results of the sample students and Achievement results of the sample students and campuses were collected either from campuses or campuses were collected either from campuses or from the Office of the Controller of Examinations of from the Office of the Controller of Examinations of T.U. according to the researcher’s convenience. T.U. according to the researcher’s convenience.

Information from sample teachers and campus Information from sample teachers and campus administrators were also colleted at the same day administrators were also colleted at the same day when information was collected from students. when information was collected from students.

The questionnaires of parents/guardians were given The questionnaires of parents/guardians were given to students and were requested to pass it to their to students and were requested to pass it to their guardians. The questionnaires were collected in guardians. The questionnaires were collected in following day. following day.

Questionnaires were mail to educators in their Questionnaires were mail to educators in their address and collected from mail. address and collected from mail.

Methodology cont…Methodology cont…

The completed questionnaires were grouped into The completed questionnaires were grouped into students, teachers, parents and administrators. students, teachers, parents and administrators.

After codding the data, variables were created After codding the data, variables were created according to the information collected by students, according to the information collected by students, teachers, campus chief and parents and then data teachers, campus chief and parents and then data were entered in the computer. were entered in the computer.

The Statistical Packages to Social Science (SPSS) The Statistical Packages to Social Science (SPSS) were used for data entry and analysis. were used for data entry and analysis.

Descriptive analysis, Regression analysis, and Descriptive analysis, Regression analysis, and Multivariate analysis were done in this study but due Multivariate analysis were done in this study but due to the small numbers of educators, numerical to the small numbers of educators, numerical analysis was not done in their information. analysis was not done in their information.

Logical Framework for Logical Framework for samplesample

Tribhuvan UniversityTribhuvan University

Faculty:Faculty:1.1. Humanities & Social Science, Humanities & Social Science, 2.2. Education, Education, 3.3. Law and Law and 4.4. ManagementManagement(81 Campuses for BBS 3(81 Campuses for BBS 3rdrd Year) Year)

Institute:Institute:1.1. EngineeringEngineering2.2. MedicineMedicine3.3. ForestryForestry4.4. Agriculture and Animal scienceAgriculture and Animal science5.5. Science and technologyScience and technology(21 campuses for BSc. 3(21 campuses for BSc. 3rdrd Year) Year)

Randomization ofRandomization of13 campuses which have more than 30 students.13 campuses which have more than 30 students.

33rdrd year Bachelor390 students (30 from each campus), 390 sample students’ parents year Bachelor390 students (30 from each campus), 390 sample students’ parents or guardians, all teachers of sample students teaching 3or guardians, all teachers of sample students teaching 3rdrd year Bachelor Level, year Bachelor Level, administrators (C. Chief, Asst. Chief, Chief Administrator, Head of examination administrators (C. Chief, Asst. Chief, Chief Administrator, Head of examination section) of respective sample campuses, professors.section) of respective sample campuses, professors.

Response Rate Response Rate

The students and their guardians/parents both set of The students and their guardians/parents both set of questionnaires were provided for the students in the questionnaires were provided for the students in the sample classrooms and were asked to return in the sample classrooms and were asked to return in the following day with both sets together. following day with both sets together.

After the removal of incomplete set of After the removal of incomplete set of questionnaires, only 30 complete sets (which had questionnaires, only 30 complete sets (which had both students and their guardians) from each both students and their guardians) from each campuses were accepted that were selected campuses were accepted that were selected randomly and had no problem of response rate. randomly and had no problem of response rate.

For teacher, 51 surveys were received from the 58 For teacher, 51 surveys were received from the 58 respondents of 13 campuses giving a response rate respondents of 13 campuses giving a response rate of 87.93% for this research.of 87.93% for this research.

Response RateResponse Rate

Similarly for administrator, 41 surveys were Similarly for administrator, 41 surveys were received from the 63 respondents giving a received from the 63 respondents giving a response rate of 65.08%, relatively high for response rate of 65.08%, relatively high for such survey. such survey.

For educator, question were mailed to five For educator, question were mailed to five respondents (3 professors- education, arts respondents (3 professors- education, arts and science; dean of college of education, and science; dean of college of education, and controller of examination office of TU) and controller of examination office of TU) only 3 were received giving 60% response only 3 were received giving 60% response rate, relatively high for a mailed survey. rate, relatively high for a mailed survey.

Summary of the Findings Summary of the Findings Students’ CharacteristicsStudents’ Characteristics

The regression analysis indicated that enrollment in tuition class, use The regression analysis indicated that enrollment in tuition class, use of guess paper for exams, and having professional license (if any) of guess paper for exams, and having professional license (if any) hindered the students scores but study in library or book centers hindered the students scores but study in library or book centers before the exam helped them positively rather than study at home. before the exam helped them positively rather than study at home.

Class regularity, students’ study habit and interest in the field before Class regularity, students’ study habit and interest in the field before study did not bring any significance difference in the students’ grade study did not bring any significance difference in the students’ grade in the Bachelor’s Level. in the Bachelor’s Level.

Similarly, 8 additional demographic variables and/or the Similarly, 8 additional demographic variables and/or the characteristics that the students can not change easily were also characteristics that the students can not change easily were also significantly related in students’ achievement of exams. The significantly related in students’ achievement of exams. The regression coefficient proved that being regression coefficient proved that being unmarriedunmarried, accepting , accepting Hindu Hindu religion or closed to it, having religion or closed to it, having mother tongue far from Nepalimother tongue far from Nepali, , having high having high mother’s educationalmother’s educational level, having level, having low annual family low annual family incomeincome, having , having SLC education complete from Katmandu valleySLC education complete from Katmandu valley, , malemale and and having no relationship with campushaving no relationship with campus except students except students played the positive roll for students’ achievement in examination. played the positive roll for students’ achievement in examination.

Summary of the FindingsSummary of the Findings

Prior KnowledgePrior Knowledge The beta coefficients of these variables indicated The beta coefficients of these variables indicated

that having high score in junior level, has published that having high score in junior level, has published the name in first list to enroll in campus after the the name in first list to enroll in campus after the entrance exam, boarding schools’ product and have entrance exam, boarding schools’ product and have matching career goal and field of study perfectly that matching career goal and field of study perfectly that student will get high score in Bachelor’s Level.student will get high score in Bachelor’s Level.

English score of a student in junior level were not be English score of a student in junior level were not be the predictors of his/her in Bachelor’s Level. the predictors of his/her in Bachelor’s Level.

Summary of the FindingsSummary of the FindingsTeacher’s BackgroundTeacher’s Background

This research has discovered that lesson plan is very essential for being This research has discovered that lesson plan is very essential for being effective teachers. effective teachers.

Similarly, more than 5 years teaching experience is essential for effective Similarly, more than 5 years teaching experience is essential for effective teaching but above 15 years’ teaching experience of teachers will not be teaching but above 15 years’ teaching experience of teachers will not be helpful for the students. helpful for the students.

Some extra qualification and knowledge of students’ Some extra qualification and knowledge of students’ evaluation/examinations are helpful for teachers’ academic career. evaluation/examinations are helpful for teachers’ academic career.

The teachers division in Master’s Degree had no relationship to satisfy The teachers division in Master’s Degree had no relationship to satisfy their students. Similarly, teachers’ teaching training in TU is not effective. their students. Similarly, teachers’ teaching training in TU is not effective. Higher academic qualification than Master’s Degree does not improve Higher academic qualification than Master’s Degree does not improve the effectiveness of TU teachers.the effectiveness of TU teachers.

In addition, punctuality helps the teacher to be popular and effective but In addition, punctuality helps the teacher to be popular and effective but the number of effective teachers will be always less than the punctual the number of effective teachers will be always less than the punctual and popular teachers in the university. and popular teachers in the university.


Reliability and Validity of Test ItemsReliability and Validity of Test Items It was found that students’ scores in Bachelor’s Level first and It was found that students’ scores in Bachelor’s Level first and

second year were highly correlated (r = 0.81 and p = .00, p <.01). second year were highly correlated (r = 0.81 and p = .00, p <.01). But in contrast, there was very weak relationship (r = .16 and p But in contrast, there was very weak relationship (r = .16 and p = .08, p > .05) between students’ practical score and their final = .08, p > .05) between students’ practical score and their final percentage in Institute of Science and Technology. percentage in Institute of Science and Technology.

Again, researcher had recorded that there were no connection Again, researcher had recorded that there were no connection between TU questions sets and test theory. Question sets had between TU questions sets and test theory. Question sets had provided unnecessary option, have coverage errors and have the provided unnecessary option, have coverage errors and have the problems of out of courses or wrongly formatted/designed. problems of out of courses or wrongly formatted/designed.

This research found that 54.9% students were involved in This research found that 54.9% students were involved in cheating in their last TU exams. cheating in their last TU exams.


Financial StatusFinancial Status This research has found that the children who have This research has found that the children who have

poor family background do better in TU. poor family background do better in TU. Similarly, the children of alcoholic guardians/parents do Similarly, the children of alcoholic guardians/parents do

better in the TU exams. better in the TU exams. The research had found that the amount of The research had found that the amount of

guardians/parents investment (Rupees) on their child guardians/parents investment (Rupees) on their child were not significantly correlated (r = .05, p > .05) with were not significantly correlated (r = .05, p > .05) with their children achievement. their children achievement.

In addition, there was a significant positive correlation In addition, there was a significant positive correlation found between children achievement and their found between children achievement and their guardians/parents’ satisfaction level over them. guardians/parents’ satisfaction level over them.


Satisfaction LevelSatisfaction Level This survey has clearly indicated that the This survey has clearly indicated that the

students, their guardians/parents, teachers students, their guardians/parents, teachers and administrators were not satisfied with TU and administrators were not satisfied with TU examination system. examination system.

It has also indicated that administrators are It has also indicated that administrators are the most negative group and teachers are the the most negative group and teachers are the least negative group among four groups least negative group among four groups towards the university examination system. towards the university examination system.

Summary of the FindingsSummary of the FindingsImproving SystemImproving System

The analysis has clearly indicated that the both groups The analysis has clearly indicated that the both groups (administrators and teachers) had highly positive attitude to improve (administrators and teachers) had highly positive attitude to improve the things that had included in the research (i.e. existing question the things that had included in the research (i.e. existing question setting style, scoring system of students’ answer copies, test setting style, scoring system of students’ answer copies, test administration system of exam hall and the length of time between administration system of exam hall and the length of time between test and results). test and results).

87.8% administrators and 78.4% university teachers were agreed to 87.8% administrators and 78.4% university teachers were agreed to change the present question setting style of TU. Similarly, 87.8% change the present question setting style of TU. Similarly, 87.8% administrators and 56.9% university teachers were agreed to change administrators and 56.9% university teachers were agreed to change the present marking scheming for scoring students’ answer copies. the present marking scheming for scoring students’ answer copies. 100% administrators agreed that TU takes unnecessarily more time 100% administrators agreed that TU takes unnecessarily more time for publishing exam results and 78.4% teachers were simply agreed for publishing exam results and 78.4% teachers were simply agreed the same statement.the same statement.

Summary of the FindingsSummary of the FindingsAppropriate SystemAppropriate System

Professionalism should be developed in all exam related activities. Professionalism should be developed in all exam related activities. Question sets should be developed according to the notion of test Question sets should be developed according to the notion of test

theory such that there can be introduced rechecking of students’ theory such that there can be introduced rechecking of students’ answer copies if they asked for it. answer copies if they asked for it.

Marking schemes should be developed for scoring students’ answer Marking schemes should be developed for scoring students’ answer copies. copies.

Teachers’ involvement in the exam hall should be increased. Teachers’ involvement in the exam hall should be increased. Regional office of the “Office of the Controller of examinations” Regional office of the “Office of the Controller of examinations” should be activated for its examination activities. should be activated for its examination activities.

At least one examination consultant should be hired. Effective At least one examination consultant should be hired. Effective training depending upon the nature of job must be provided for every training depending upon the nature of job must be provided for every staff, teachers or faculty members. staff, teachers or faculty members.

Making transparent in the financial sector of examination, works of Making transparent in the financial sector of examination, works of the staffs, teachers must be paid reasonably. the staffs, teachers must be paid reasonably.

Examination calendar should be published in every academic year Examination calendar should be published in every academic year and should be strictly followed.and should be strictly followed.

Documents

Factors Influencing Examination Results in Higher Education for Maximizing Validity and Improving Students’ Pass Rate in Examination System of Tribhuvan