QUALITY ASSURANCE IN OBJECTIVE TESTING. Isaac Olakanmi Abimbola Department of Science Education University of Ilorin Ilorin, Nigeria [email protected]. OUTLINE. Introduction Types of Test Essay Objective Forms of Objective Test Quality Assurance - PowerPoint PPT Presentation
Citation preview
QUALITY ASSURANCE IN OBJECTIVE TESTINGIsaac Olakanmi Abimbola Whatever is implemented in instruction should guide testing Teachers should only test what was intended for learning and what was actually taught Lesson objectives Instruction Test TYPES OF TEST Answered in writing only TYPES OF TEST (CONTD.) (i) Matching items (ii) Multiple-choice items 1. Two-choice items (i)True/False items Shooting questions Statements to be judged and identified by examinees as True or False Forms of Objective Test (Contd.) 1. Two-choice items Example: The name of the first spacecraft launched in 1957 by the former Soviet Union is Sputnik I _____ (True/False) Forms of Objective Test (Contd.) 1. Two-choice items Ensure balance between true and false items Each statement should be clearly true or false, not partially true or false Forms of Objective Test (Contd.) 1. Two-choice items Measure only one idea in one statement—no double-barreled item Forms of Objective Test (Contd.) 1. Two-choice items Quality Assurance (Contd.) Use one judgement mode at a time—True or False, Yes or No, Correct/Incorrect Avoid direct quotations from books or other materials Do not give clues 1. Two-choice items Example: (i) Matching items The response column is shuffled against the premise column Examinee matches each premise with a corresponding response Measures understanding of concepts, principles and classification schemes Forms of Objective Test (Contd.) 2. More than two-choice items (i) Matching items (Contd.) Example: Match the weather instruments in Column 1 with their uses in Column 2 by writing the appropriate letters (A-F) in the spaces provided: Forms of Objective Test (Contd.) 2. More than two-choice items (i) Matching items (Contd.) 7. 8. 10. F Measures the degree of hotness or coldness of a living body Forms of Objective Test (Contd.) 2. More than two-choice items (Contd.) (i) Matching items (Contd.) Quality Assurance Premise should be on the left-hand column with numbers and responses on the right-hand column with letters Items should be homogeneous Responses should be more than premises Provide no clue in the two columns Forms of Objective Test (Contd.) More than two-choice items (Contd.) (ii) Multiple-choice Items Stem could be a question or statement 3-5 alternatives/options are available with only one correct or appropriate answer (The Key) Incorrect or inappropriate answers are called distractors/distracters/foils Forms of Objective Test (Contd.) More than two-choice items (Contd.) (ii) Multiple-choice Items(Contd.) Scoring is highly objective—lay person/machine Chance for guessing is minimal Forms of Objective Test (Contd.) More than two-choice items (Contd.) (ii) Multiple-choice Items(Contd.) Examples: 1) When human beings breathe out, what do they breathe out? A. air B. oxygen C. nitrogen D. carbon dioxide 2) An example of a living thing is _________. A. water B. fire C. egg D. wind Forms of Objective Test (Contd.) More than two-choice items (Contd.) (ii) Multiple-choice Items(Contd.) All the options should appear plausible Letters of the correct options should vary Avoid “None of the above” and “All of the above “ as options Do not start a stem with a dash QUALITY ASSURANCE IN OBJECTIVE TESTING 1. Statement of lesson objectives 2. Table of Specification/Test Blueprint to ensure fair representation of course content Objectives/Contents can be stated as percentages, number of items or item numbers. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) % of Representation on Test 3. Determine the item format to be used 4. Item Writing (i) Use terms whose definitions are precise from examinees’ point of view (ii) Stem should not start with a dash or blank space (iii) Stem should not contain negatives, especially, double negatives (iv) Use students’ misconceptions and alternative conceptions as distractors QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 4. Item Writing (Contd.) Example (Misconceptions/Alternative Conceptions): Energy is an abstract idea that is not easy to define. Which of the following statements best represents what energy is? A. energy is force B. Energy is a quantity C. energy behaves as a fluid D. energy is fuel QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 4. Item Writing (Contd.) Example (Misconceptions/Alternative Conceptions—Multiple levels): Identify which of the following are characteristic of the vertebrate respiratory surface: 1. moist 2. vascularized 3.semi-permeable 4. freely permeable 5. dry 4. Item Writing (Contd.) A. 1,2,3. B. 1,2,5. C. 2,3,5. D. 2,4,5. E.1,3,5. 5. Review of Test Items Present the test items to others for review. HOD should review the items to spot grammatical errors, omissions and gender friendliness of the items. Regulation on multiple examiners is useful here QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 6.Trial Run of the Test Try out the test on a representative sample of the target population and score the test in readiness for item analysis. 7. Item Analysis Arrange candidates’ scripts according to their scores. Different authors divide the scores into three or four equal parts. (i) Item Difficulty/Facility Index (ii) Item Discrimination Power (iv) Further Item Validity 7. Item Analysis (Contd.) (i) Item Difficulty Index (Contd.) If 15/20 of the upper group and 5/20 of the lower group got item 3 correct (B), therefore, 20 (15+5) persons out of 40 (20+20) got the item correct. Item 3 ITEM 3 7. Item Analysis (Contd.) Item Difficulty = 100x (R/T) = 100x20/40 = 50% where R = the number of persons in the upper and lower thirds combined who got the item right, and T = total number of persons in the upper and lower thirds combined Therefore, item 3 is moderately difficult. Difficulty Index of < 10% =Too difficult; > 90% = Too easy QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 7. Item Analysis (Contd.) (ii) Item Discrimination Power Extent to which each item discriminates between those in the upper third and the lower third, respectively. Item Discrimination Power = Ru – Rl /½ T, where: Ru = no. of those in the upper third who got the item right Rl = no. of those in the lower third who got the item right T = total number of persons in the upper and lower thirds QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 7. Item Analysis (Contd.) The DP for item 3 above is Ru – Rl = 15-5 = 10 =0.50 ½ T ½(20+20) 20 Range of DP = -1.00 to +1.00 Interpretation: 0.40 and above =Very good item 0.30 to 0.39 = Reasonably good but room for improvement 0.20 to 0.29 = Marginal, improve < 0.19 = Poor, reject or revise QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 7. Item Analysis (Contd.) (iii) Effectiveness of Distractors Every option of an item is chosen by at least one person in the upper or lower third of the class while each distractor is chosen more frequently by those in the lower third than in the upper third. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 7. Item Analysis (Contd.) Option D is the correct answer in item 5 Options A, B, & C are all functional distractors with at least one candidate picking each of them Options A & B discriminate rightly while option C discriminates wrongly Item 5 Item 5 Item 5 7. Item Analysis (Contd.) (iv) Coefficient of Item Validity (Yarroch, 1991) This involves a post-test clinical interview of examinees to find out “the relationship between the examinees’ knowledge and the knowledge the examination item was intended to measure” (p. 621). Essay items equivalent to the multiple-choice items could also be administered to the examinees to further test their proper understanding. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 7. Item Analysis (Contd.) Civ = (% correct for correct reason) + (% incorrect for correct reason) QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 8. Amend the items as necessary based on the results of item analysis and reject items that are not suitable 9. Select and compile the final test items duly following rules guiding objective testing. Ethical principles in evaluation The teacher must display a sense of duty towards the stakeholders The method of evaluation must protect the integrity of examinees and examiners The examinees must be provided with adequate information about the test and the method for its preparation Ethical principles in evaluation contd. All tests items should be vetted by the Heads of Department and preferably scored twice by two independent scorers as stipulated in the university regulations There should be sufficient check against errors in adding marks and preparation of mark sheets There should be continuous research on improving the evaluation programme by individual lecturers and other stakeholders The actual test items should never be revealed to any unauthorized persons Ethical principles in evaluation contd. Test material should never be revealed to the students in advance directly or indirectly The test scores or trend in the test performance of a student should not be revealed to an unauthorized person. (Sidhu, 2005, p. 357) CONCLUSION Objectives tests, especially multiple-choice tests, are good for preparing global citizens since they are popular among many countries of the world They can be used to test practically any subject matter, they only require some training Objective tests are fun to prepare, with adequate practice, especially using continuous assessment tests as starting points. They take off the drudgery of marking from lecturers Students have access to their results in good time THANK YOU BIBLIOGRAPHY Abimbola, I. O. (1996). Advances in the development and validation of instruments for assessing students’ science knowledge. In G. A. Badmus & P. I. Odor (Eds.), Challenges of managing educational assessment in Nigeria (pp. 33-40). Kaduna: JAMB, NABTEB, & NBEM. Abiri, J. O. O. (2006). Elements of evaluation, measurement and statistical techniques in education. Ilorin: Library and Publications Committee, University of Ilorin, Ilorin, Nigeria Gillette, F. (n.d.). Multiple choice tests: Composition, assembly and analysis. San Francisco, CA: San Francisco State University. http://www.sfsu.edu . Retrieved 21/9/14. BIBLIOGRAPHY (CONTD.) BIBLIOGRAPHY (CONTD.) Tests: Objective tests. San Luis Obispo, CA: Academic Skills Center, California Polytechnic State University