Trick Question

8/13/2019 Trick Question

1/69

Is This aIs This aIs This aIs This aIs This a

TrickTrickTrickTrickTrickQuestion?Question?Question?Question?Question?

A Short Guideto Writing Effective

Test Questions


2/69

Is This aIs This aIs This aIs This aIs This aTrickTrickTrickTrickTrick

Question?Question?Question?Question?Question?A Short Guide

to Writing EffectiveTest Questions

Designed & Developed by:

Ben ClayKansas Curriculum Center

Formatting & Text Processing by:

Esperanza Root

This publication was developed by theKansas Curriculum Center w ith funds

provided by the Kansas StateDepartment of Education.

First printing:October, 2001


3/69

Preface ............................................................................................ i-iiPre-Test........................................................................................... 1-2Generally........................................................................................ 3-5

General Tips About Testing ............................................... 3-4When to Use Essay or Objective Tests ............................... 4-5Matching Learning Objectives wi th Test Items ...................... 5

Planning the Test .......................................................................... 6-12Cognitive Complexity ........................................................ 6-7Content Quality .................................................................... 8

Meaningfulness .................................................................... 8Language Appropriateness.................................................... 9Transfer and Generalizabil ity ................................................ 9Fairness .............................................................................. 10Reliabil ity ........................................................................... 10How to Defeat Student Guessing ........................................ 11General Test Taking Tips .................................................... 12

Multiple Choice Test Items.........................................................13-19Section Summary ................................................................ 13Test Your Knowledge.......................................................... 14Suggestions for Wri ting Multiple Choice Test Items........ 15-16

Mul tiple Choice Test Taking Tips................................... 17-18Aim for Higher Levels of Learning....................................... 19

True-False Test Items...................................................................20-26Section Summary ................................................................ 20Test Your Knowledge.......................................................... 21Suggestions for Writing True-False Test Items................. 22-23Extreme Modifiers and Qualifiers........................................ 23True-False Test Taking Tips ................................................. 24Variations in Writing True-False Test Items..................... 24-25Aim for Higher Levels of Learning....................................... 26

Matching Test Items.................................................................... 27-33Section Summary ................................................................ 27Test Your Knowledge..................................................... 28-29Suggestions for Writing Matching Test Items .................. 30-31Matching Test Taking Tips .................................................. 32Variations for Creating Matching Tests ................................ 33

Table of ContentsTable of ContentsTable of ContentsTable of ContentsTable of Contents


4/69

Completion or Fill-in-the-Blank Test Items.................................. 34-37Section Summary ................................................................ 34Test Your Knowledge.......................................................... 35Suggestions for Writing Completion Test Items .............. 36-37

Completion Test Taking Tips .............................................. 37

Essay Test Items.......................................................................... 38-44Section Summary ................................................................ 38"I' d Like to Use Essay Tests, But" ..................................... 39Read'Em and Weep Essay Test Items................................... 39Test Your Knowledge.......................................................... 40Suggestions for Writing Essay Test Items ........................ 41-42Four-Step Process in Grading Essay Tests ............................ 43Essay Test Taking Tips......................................................... 44

Additional Types of Test Items.....................................................45-51

Problem Solving ................................................................. 45Using Authentic Assessments......................................... 46-47Grading Authentic Assessments.......................................... 48Rubric Development ..................................................... 48-51

EtcEtcEtc ...........................................................................52-60Purpose of Testing .............................................................. 52Tips on Test Construction ................................................... 52Test Layout Tips.................................................................. 52Returning Tests and Giving Feedback ................................. 53Alternative Testing Modes .................................................. 54Creating Fair Tests and Testing Fairly .................................. 55"I' d Like to Use Essay Tests, But" ................................ 56-57Test Administration Assignment .......................................... 58Cognitive Domain Guide.................................................... 59Affective Domain Guide ..................................................... 60

Bibliography............................................................................... 61-63


5/69i

Pref acePref acePref acePref acePref ace

A notable concern of many teachers is that they frequently havethe task of constructing tests but have relatively little training orinformation to rely on in this task. Is This a Trick Question?is aninformation sourcebook for writing effective test questions. Thecentral focus of the sourcebooks content is derived from standardsdeveloped by the National Center for Research on Evaluation, Stan-dards, and Student Testing (CRESST).

CRESSTs criteria for establishing the technical quality of a testencompasses seven areas: cognitive complexity, content quality,

meaningfulness, language appropriateness, transfer andgeneralizabil ity, fairness, and reliability. Each aspect is discussed inthe sourcebook in a straight-forward, jargon-free style.

Part One contains information concerning general test constructionand introduces the six levels of intellectual understanding: knowl -edge, comprehension, application, analysis, synthesis, and evalua-tion. These levels of understanding assist in categorizing testquestions, with knowledge as the lowest level. Since teachers tendto construct questions in the knowledge category 80% to 90% of thetime, throughout the sourcebook are examples of or suggestions fordeveloping higher order thinking skills. This supports Kansas

current Quality Performance Accreditation initiative which hasestablished content and performance standards that cannot bemeasured by low-level tests.

Part Two of the information sourcebook is devoted to actual testquestion construction. Because of the diversity of assessmentoptions, the sourcebook focuses primarily on paper-and-penciltests, the most common type of teacher-prepared assessment. Fivetest item types are discussed: multiple choice, true-false, matching,completion, and essay. Information covers the appropriate use ofeach item type, advantages and disadvantages of each item type,

and characteristics of well written items. Suggestions for addressinghigher order thinking skills for each item type are also presented.

This sourcebook was developed to accomplish three outcomes:

! Teachers wi ll know and follow appropriate principles for devel-oping and using assessment methods in their teaching, avoidingcommon pitfalls in student assessment.

Research indicates

Teachers tend to use tests thatthey have prepared themselvesmuch more often than any othertype of test. (How Teaching Mat-ters,NCATE,Oct. 2000)

While assessment options are di-

verse, most classroom educatorsrely on text and curriculum-em-bedded questions and tests thatare overwhelmingly classified aspaper-and-pencil(National Com-miss ion on Teaching and

Americas Future,1996).

Formal training in paper-and-pen-cil test construction may occur atthe preservice level (52% of thetime) or as inservice preparation

(21%). A signifi cant number ofprofessional educators (48%) re-port no formal training in devel-oping, administering, scoring, andinterpreting tests (EducationWeek,National Survey of PublicSchool Teachers, 2000 ).

Students report a higher level oftest anxiety over teacher-madetests (64%) than over standard-

ized tests (30%). The top threereasons why: poor test construc-tion, irrelevant or obscure mate-rial coverage, and unclear direc-tions. (NCATE,Summary Dataon Teacher Effectiveness, TeacherQuality, and Teacher Qualifica-tions, 2001.)

(Conti nued on next page)


6/69ii

! Teachers wil l be able to identify and accommodate the limitationsof different informal and formal assessment methods.

! Teachers will gain an awareness that certain assessment ap-proaches can be incompatible with certain instructional goals.

These three outcomes directly support the standards developed bya joint commission established by the National Education Associa-tion, the American Federation of Teachers, and the National Coun-cil on Measurement in Education. The initial standards wereidentified in 1990 and revised in 1999. In May 2001, a new listingwas issued under the title Standards for Teacher Competence inEducational Assessment of Students. The first two standardsdirectly reflect the outcomes of this sourcebook:

! Teachers should be skilled in choosing assessment methodsappropriate for instructional discussion

! Teachers should be skilled in developing assessment methodsappropriate for instructional decisions.

While no one document can thoroughly address the needs andconcerns expressed in all of this information, this sourcebook canbe a valuable resource for any teacher who is interested in measur-ing outcomes of significance, tapping into higher-level thinking andproblem solving skills, and constructing tests that effectively andfairly capture what a student knows.

Ben Clay, CoordinatorKansas Curriculum Center

In Kansas

The Kansas Commission onTeaching and Americas Future(KCTAF), chaired by Dr. AndyTompkins, Kansas Commissionerof Education, proposes to de-velop higher-quality alternativepathways to teaching as well asto reinvent teacher preparationand professional development.As secondary and postsecondaryinstitutions are exploring (out ofnecessity mostly) alternatives to

traditional teacher recruitment,the need for training in assess-ment procedures and paper-and-pencil test construction in par-ticular, become more and moreevident.


7/691

Pre-TestPre-TestPre-TestPre-TestPre-Test

1. Essay exams are easier to constructthan are objective exams. T F ?

2. Essay exams require more thoroughstudent preparation and study timethan objective exams. T F ?

3. Essay exams require writing skill swhere objective exams do not. T F ?

4. Essay exams teach a person howto write. T F ?

5. Essay exams are more subjectivein nature than are objective exams. T F ?

6. Objective exams encourage guess-ing more so than essay exams. T F ?

7. Essay exams limit the extent ofcontent covered. T F ?

8. Essay and objective exams canbe used to measure the samecontent or ability. T F ?

9. Essay and objective exams areboth good ways to evaluate astudents level of knowledge. T F ?

Test Item QuizCircle the correct answer

Answers on next page

T=True F=False ?=Unsure

1.Objective itemswhichrequire students to select thecorrect response from severalalternatives or to supply aword or short phrase to answera question or complete a

statement

2. Subjective or essay itemswhich permit the student toorganize and present anoriginal answer.

Objective items include:! multiple choice! true-false! matching! completion

Subjective items include:! short-answer essay! extended-response essay! problem solving! performance test i tems

Test yourknowledge of

these two itemtypes by

answering thefollowingquestions

Two generalcategories of

test items


8/692

Quiz AnswersQuiz AnswersQuiz AnswersQuiz AnswersQuiz Answers

1. Essay exams are easier to construct than are obj ective exams.

TRUEEssay items are generally easier and less time consuming to construct thanare most objective test items. Technically correct and content appropriate mul-tiple choice and true-false test items require an extensive amount of time to writeand revise.

2. Essay exams require more thorough student preparation and study t ime than

objective exams.

? (QUESTION MARK)According to research findings it is still undetermined whetheror not essay tests require or facilitate more thorough (or even different) studentstudy preparation.

3. Essay exams require wri ting skills where objective exams do not.TRUEWriting skills do affect a students ability to communicate the correct fac-tual information through an essay response. Consequently, students with goodwri ting skill s have an advantage over students who do not.

4. Essay exams teach a person how to w rite.

FALSEEssays do not teach a student how to write but they can emphasize theimportance of being able to communicate through writing. Constant use of essaytests may encourage the knowledgeable but poor writing student to improve his/her writing abili ty in order to improve performance.

5. Essay exams are more subjective in nature than are objecti ve exams.

TRUEEssays are more subjective in nature due to their susceptibility to scoringinfluences. Different readers can rate identical responses differently, the samereader can rate the same paper differently over time, the handwriting, neatness orpunctuation can unintentionally affect a papers grade.

6. Objective exams encourage guessing more so than essay exams.

? (QUESTION MARK)Both item types encourage some guessing. Multiple choice,true-false and matching items can be correctly answered through blind guessing,yet essay items can be responded to satisfactorily through well written bluffing.

7. Essay exams lim it the extent of content covered.

TRUEDue to the extent of time required to respond to an essay question, only afew essay questions can be included on a exam. A larger number of objectiveitems can be tested in the same amount of time, covering more content.

8. Essay and objecti ve exams can be used to measure the same content or abil ity .TRUEBoth item types can measure simi lar content or learning objectives. Re-search has shown that students respond almost identically to essay and objectivetest items covering the same content.

9. Essay and objecti ve exams are both good ways to evaluate a student s level of

knowledge.

TRUEBoth objective and essay test items are good devices for measuring studentachievement. However, as seen in the previous quiz answers, there are particularmeasurement situations where one item type is more appropriate than the other.


9/693

GenerallyGenerallyGenerallyGenerallyGenerally

Length of TestIn theory, the more items a test has, the more reliable it is. On ashort test a few wrong answers can have a great effect on the over-all results. On a long test, a few wrong answers wi ll not influencethe results as much. A long test does have drawbacks. If a test istoo long, and particularly if students are doing the same kind ofitem over and over, they may get tired and not respond accuratelyor seriously. If a test needs to be lengthy, divide it into sectionswith different kinds of tasks, to maintain the student's interest.

Clear, Concise InstructionsIt is necessary to give clear, concise instructions. It is useful toprovide an example of a worked problem, which helps the stu-dents understand exactly what is necessary. What seems to beclear to the writer may be unclear to someone else.

Mix It Up!It is often advantageous to mix types of items (multiple choice,true-false, essay) on a written exam or to mix types of exams (aperformance component with a written component). Weaknessesconnected with one kind of i tem or component or in students test

taking skil ls wil l be minimized.

Test EarlyIt is helpful for instructors to test early in the term and considerdiscounting the first test if results are poor. Students often need apractice test to understand the format each instructor uses and an-ticipate the best way to prepare for and take particular tests.

Test FrequentlyFrequent testing helps students to avoid getting behind, providesinstructors wi th multiple sources of information to use in comput-ing the final course grade (thus minimizing the effect of bad days),

and gives students regular feedback. It is important to test varioustopics in proportion to the emphasis given in class. Students willexpect this practice and wil l study with this expectation.

Check For AccuracyInstructors should be cautious about using tests wri tten by others.Often, items developed by a previous instructor, a textbook pub-lisher, etc., can save a lot of time, but they should be checked foraccuracy and appropriateness in the given course.

General Tips About Testing


Creating a test isone of the mostchallenging tasksconfronting aninstructor.Unfortunately,

many of us havehad little, if any,preparation inwriting tests.

Well constructed tests motivatestudents and reinforce learning.Well constructed tests enableteachers to assess the students

mastery of course objectives.Tests also provide feedback onteaching, often showing what wasor was not communicated clearly.

While alwaysdemanding, testwriting may be

made easier byconsidering thefollowing

suggestions forgeneral test

construction.


10/694

Proofread ExamsOn written exams, it is important to proofread exams carefully and,when possible, have another person proofread them. Tiny mis-

takes, such as misnumbering the responses, can cause big prob-lems later. Collation should also be checked carefully, since miss-ing pages can cause a great deal of trouble.

One Wrong AnswerGenerally, on either a written or performance test, it is wise toavoid having separate items or tasks depend upon answers or skillsrequired in previous items or tasks. A students initial mistake wi llbe perpetuated over the course of succeeding items or tasks, pe-nalizing the student repeatedly for one error.

Special ConsiderationsIt is important to anticipate special considerations that learning dis-abled students or non-native speakers may need. The instructorneeds to anticipate special needs in advance and decide whetheror not students wi ll be allowed the use of dictionaries, extra time,separate testing sites, or other special conditions.

A Little HumorInstructors have found that using a little humor or placing less dif-ficult i tems or tasks at the beginning of an exam can help studentswi th test anxiety to reduce their preliminary tension and thus pro-vide a more accurate demonstration of their progress.

General Tips About Testing(Continued from previous page)

What makes a test

good or bad? The

most basic and obvi ous

answer to that

questi on is that good

tests measure what

you want to measure,

and bad tests do not .

It is always tempt ing

to emphasize the

parts of the coursethat are easiest to

test, rather than the

parts that are

impor tant to test.

When to Use Essay or Object ive TestsWhen to Use Essay or Object ive TestsWhen to Use Essay or Object ive TestsWhen to Use Essay or Object ive TestsWhen to Use Essay or Object ive Tests

Essay tests are appropriate when:! the group to be tested is small and the test is not to be reused.! you wish to encourage and reward the development of student

skil l in writing.! you are more interested in exploring the students atti tudes than in

measuring his/her achievement.

Objective tests are appropriate when:! the group to be tested is large and the test may be reused.! highly reliable scores must be obtained as efficiently as possible.! impartiality of evaluation, fairness, and freedom from possible test

scoring influences are essential.(Conti nued on next page)


11/695

Either essay or objective tests can be used to:! measure almost any important educational achievement

a written test can measure.! test understanding and ability to apply principles.! test abil ity to think cri tically.! test ability to solve problems.

When to Use Essay or Object ive TestsWhen to Use Essay or Object ive TestsWhen to Use Essay or Object ive TestsWhen to Use Essay or Object ive TestsWhen to Use Essay or Object ive Tests(Continued from previous page)

Certain item types are bettersuited than others for measuringparticular learning objectives.For example, learning objectivesrequiring the student to demon-strate or to show, may be bettermeasured by performance testitems, whereas objectives requir-ing the student to explain or todescribe may be better measuredby essay test items.

To further il lustrate this principle,several sample learning objec-tives and appropriate test itemsare provided on the right. Matchthe most suitable test item witheach of the learning objectives.

Matching Learning Objectiveswith Test Items

Instructions: Below are four test item categories labeledA, B, C, and D. Following these test item categories aresample learning objectives. On the line to the left of eachlearning objective, place the letter of the most appropriatetest item category.

A= Objective Test Item (multiple choice,true-false, matching)

B= Performance Test ItemC= Essay Test Item (extended response)D= Essay Test Item (short answer)

____1. Name the parts of the human skeleton

____2. Appraise a composition on the basis ofits organization

____3. Demonstrate safe laboratory skil ls

____4. Cite four examples of satire that Twainuses in Huckleberry Finn

____5. Design a logo for a web page

____6. Describe the impact of a bull market

____7. Diagnose a physical ailment

____8. List important mental attributes necessaryfor an athlete

____9. Categorize great American fiction wri ters

____10. Analyze the major causes of learningdisabilities

The matching oflearning objective

expectations withcertain item typesprovides a highdegree of testvalidity: testingwhat is supposedto be tested

Answers:1-A,2-C,3-B,4-D,5-B,6-C,7-B,8-D,9-A,10-C

Conventional wi sdom

accurately port rays shor t-

answer and essay

examinat ions as the easiest

to w rit e and the most

diff icult to grade,particularly if they are

graded well.


12/696

1.Cognitive Complexity

Planning the TestPlanning the TestPlanning the TestPlanning the TestPlanning the Test

By definition notest can be trulyobjective:existing as anobject of fact,independent of

the mind.

In general ,

test it ems shoul d

! Assess achievement of instruc-tional objectives

! Measure important aspects ofthe subject (concepts and con-

ceptual relations)

! Accurately reflect the empha-sis placed on important aspectsof instruction

! Measure an appropriate levelof student knowledge

! Vary in levels of difficulty

Impl ying that one type of

question is automati cally

objective and the other

necessari ly subjecti ve is a

faul ty assumpt ion, since

bias can occur w ith either

type of test.

Crit eria for Establishing

Technical Quality of a Test*

Standard: The test questions will focus on appropriateintellectual activity ranging from simple recall of facts toproblem solving, critical thinking, and reasoning.

Cognitive complexity refers to the various levels of learning

that can be tested. A good test reflects the goals of theinstruction. If the instructor is mainly concerned with studentsmemorizing facts, the test should ask for simple recall ofmaterial. If the instructor is trying to develop analytic skills, atest that asks for recall is inappropriate and will cause studentsto conclude that memorization is the instructor's true goal.

Refreshing the old bloomDuring the 1948 convention of the American PsychologicalAssociation, a group of educational psychologists decided itwould be useful to classify different levels of understandingthat students can achieve in a course.

In 1956, after extensive research on educational goals, thegroup published its findings in a book edited by Dr. Ben-

jamin S. Bloom, a Harvard professor. BloomsTaxonomy ofEducational Objectiveslists six levels of intellectualunderstanding:

!Knowledge ! Analysis!Comprehension ! Synthesis!Application ! Evaluation

These levels of understanding assist in categorizing test ques-tions. Teachers tend to ask questions in the knowledgecat-egory 80% to 90% of the time. These questions are not bad,but using them all the time is. Try to utilize higher orderlevel of questions. These questions require much morebrainpower. (See the next page for a defini tion and sampl e ques-tion frames for each level of learning.)

*Adapted from material developed by the National Center for Research on Evaluation,Standards, and Student Testing (CRESST).


13/697

1.Cognitive Complexity (continued)

Knowledge

Recognizing and recall ing information,including dates, events, persons, places;terms, definitions; facts, principles,theories; methods and procedures

Sampl e Question Frames

Who invented the?What is meant by?Where is the?

Comprehension

Understanding the meaning of informa-tion, including restating (in own words);translating from one form to another; or

interpreting, explaining, andsummarizing.


Restate in your own words?Convert fractions into?

List three reasons for?

Application

Applying general rules, methods, orprinciples to a new situation, includingclassifying something as a specificexample of a general principle or usinga formula to solve a problem.


How is...an example of... ?How is...related to... ?Why is...significant?

AnalysisIdentifying the organization and patternswi thin a system by identifying its compo-nent parts and the relationships amongthe components.


What are the parts of... ?Classify ...according to...Outline/diagram...

Synthesis

Discovering/creating new connections,generalizations, patterns, or perspectives;combining ideas to form a new whole.


What would you infer from... ?What ideas can you add to... ?How would you create a... ?

Evaluation

Using evidence and reasoned argumentto judge how well a proposal wouldaccomplish a particular purpose;resolving controversies or differencesof opinion.


Do you agree?How would you decide about... ?What priority would you give... ?

See pages 59&60for Cognitiveand Affective Domain Guides.


14/698


Technical Quality of a Test(continued)

2.Content Quality

Standard: The test questions will permit students to demon-strate their knowledge of challenging and important subjectmatter.

Some important questions need to be answered concerningthe content quality of the test. What are the test specifica-tions? What skills do they indicate wi ll be tested. How manyquestions and how many areas will be covered? How manysections wi ll there be? What formats wi ll be used to test?

If an instructor has focused on the War of 1812 in the major-ity of the class sessions and activities, this emphasis shouldbe reflected in the test. A test that covers a much broaderperiod will be regarded as unfair by the students, even if theinstructor has told them that they are responsible for materialthat has not been discussed in class. Students go by instruc-tors' implicit values more than their stated ones.

To Achieve Content Quality

The first activity in planning a testis to outl ine the actual course con-tent that the test wi ll cover. A con-venient way of accomplishing thisis to take a few minutes followingeach class to list on an index cardthe important concepts covered inclass and in assigned reading forthat day. These cards can then be

used later as a source of test items.

An even more conscientious ap-proach would be to construct thetest items themselves after eachclass. The advantage of either ofthese approaches is that the result-ing test is likely to be a better rep-resentation of course activity.

3.Meaningf ulness

Standard: The test questions will be worth students timeand students will recognize and understand their value.

To Achieve Meaningfulness"In my opinion, students should not be forced to guess whatwill be on a test, or psych-out the teacher to decide what tostudy. Research shows that the less able students are heavil ypenalized by a failure to realize what is required for a test.The more able students seem to sense what the teacher wants,but the students most in need of help are likely to floundereven more painfully if they must guess what to study.

"The obvious solution to this problem is to give students spe-cific study questions, then draw the test from the study ques-tions. Sometimes this is criticized as teaching the test,as ifhaving study questions in itself encourages a superficial ap-proach. That may be true if there are very few study ques-tions. However, if a teacher offers questions for all of themost important ideas in an assignment, then teaching thetestis teaching the course."

It is very easy to

writ e items which

require only rote

recall but are

nonetheless di ff icul t

because they aretaken from obscure

passages (foot notes,

for instance).

Russell A. D ewey, PhD

Georgia Southern Un iversity, Statesboro, GA


15/699



4.Language Appropriateness

Standard: The language demands will be clear and appro-priate to the assessment tasks and to students.

Test questions should reflect the language that is used in theclassroom. Test items should be stated in simple, clear lan-guage, free of nonfunctional material and extraneous clues.Test items should also be free of race, ethnic, and sex bias.Beyond these two qualifications, students' language back-

grounds impact their performance on tests. The vocabulary(uncommon usage; nonliteral usage) and the syntax of thetest (atypical parts of speech; complex structures) may createlanguage barriers.

Modifications of the test for students that are limited Englishproficient include: assessment in the native language; textchanges in vocabulary; modification of linguistic complex-ity; addition of visual supports; use of glossaries in native lan-guage; use of glossaries in English; linguistic modification oftest directions; and additional example items/tasks.

Prelim inary fi ndings by the National

Center f or Research on Evaluat ion ,

Standards, and Student Testi ng

(CRESST):

Math and science subsections:67% percent of items had gen-eral vocabulary evaluated as

uncommon or used in anatypical manner; 33% of i temshad syntactic structures eval-uated as complex or atypical intheir construction.

Reading comprehension: Sameas above for vocabulary andsyntax; 50% of items also haddiscourse level demands.

Results of Applying LanguageEvaluation Criteria to

Standardized Content Test Items

5.Transf er and Generalizability

Standard: Successful performance on the test will allow validgeneralizations about achievement to be made.

Presentations, scenarios, projects and portfol ios add dimen-sions to assessment that traditional testing cannot. Teacherscan make valid generalizations about achievement more eas-ily using authentic and performance assessments. These gen-

eralizations may involve instructional placement decisions,formative evaluation decisions and diagnostic decisions. Wellconstructed testswhether they are objective or performanceorientedallow teachers to understand what needs to betaught next. Teachers are also able to monitor a studentslearning, while instruction is underway, and can change theinstruction program as needed.

To reduce frustration for good stu-dents, avoid all of theseandnoneof theseand both a & banswers.These items are acceptable froma theoretical standpoint, but mostprepared test-takers dislike them!As an example, the more subjectmatter a student knows, the easierit is to make arguments in favorof answers that the teacher mightregard as wrong.

True-false questions are the worstof all in this regard. Often thetruth value of an isolated state-ment is quite debatable! It alldepends on how it is interpreted,the definition of a key term, or thecontext.


16/6910



6.Fairness

Standard: Student performance will be measured in a waythat does not give advantage to factors irrelevant to schoollearning; scoring schemes wil l be similarly equitable.

Here are a few basic rules of fairness: test questions shouldreflect the objectives of the unit; expectations should be clearlyknown by the students; each test item should present a clearlyformulated task; one item should not aide in answeringanother; ample time for test completion should be allowed;

and assignment of points should be determined before thetest is administered.

Grading constructively requires the instructor to providefeedback (written and/or oral) that helps the students toappreciate what they achieved and did not achieve by takingthe test. This feedback could include the fol lowi ng:encouraging comments on a test or paper that convey respectfor what the student attempted to accomplish; praise for whatthe student did accomplish and suggestions for improvingperformance.

Five hundred secondary andpostsecondary students

were surveyed for suggestionson how an instructor couldgrade fairly and accurately.

Here are the top 10 responses.

! Consider grading based only onmastery of material and not onpersonalities or perceived effort.

! Do not over emphasize grades.Emphasize learning over grades.

! Keep students informed of theirprogress throughout the term.

! Clearly state grading pol icies andprocedures in the syllabus andreview them with the classduring orientation.

! Avoid modifying policies duringthe term.

! Provide plenty of opportunitiesfor assessment. This will avoidunnecessary pressure and allowfor some mistakes.

! Provide some choice in formator topic when assigning work.

! Keep accurate records of grades.Record numerical grades, rather

than letter grades, whenpossible.

! Consider allowing rewriteson papers.

! If many do poorly on an exam,schedule an exam for the follow-ing week to retest the class.

7.Reliability

Standard: Answers to test questions will be consistentlytrusted to represent what students know.

The whole point of testing is to encourage learning. A goodtest is designed with items that are not easily guessed withoutproper studying. It is possible to construct all types of testquestions which are not readily guessed and therefore requirea student to comprehend basic factual material.

Multiple choice questions are widely scorned as multipleguessquestions. The solution to this problem is to designmultiple choice items so that students who know the subjector material adequately are more likely to choose the correctalternative and students with less adequate knowledge aremore likely to choose a wrong alternative. (On the next pageare suggestions on how to defeat the TEST-WISEstrategies of

students who do not study.)


17/6911

How to Def eat the Common Rules of ThumbHow to Def eat the Common Rules of ThumbHow to Def eat the Common Rules of ThumbHow to Def eat the Common Rules of ThumbHow to Def eat the Common Rules of Thumb

Which Students Use to Guess Correct AnswersWhich Students Use to Guess Correct AnswersWhich Students Use to Guess Correct AnswersWhich Students Use to Guess Correct AnswersWhich Students Use to Guess Correct Answers

Rule of thumb: Pick the longest answer.

Way to defeat this strategy: Make sure the longest answer is right about a fifthof the time (if there are five alternatives for each question).

Rule of thumb: Pick the b alternative.

Way to defeat this strategy: Make sure each answer is used the same numberof times, in random order.

Rule of thumb: Never pick an answer which uses the word always or neverin it.

Way to defeat this strategy:Make sure such answers are correct about a fi fth ofthe time.

Rule of thumb: If there are two answers which express opposites, pick one orthe other and ignore other alternatives.

Way to defeat this strategy: Sometimes offer opposites when neither iscorrect.

Rule of thumb: If in doubt, guess.

Way to minimize the impact of this strategy: Use five alternatives instead ofthree or four.

Rule of thumb: Pick the scientific-sounding answer.

Way to defeat this strategy: Use scientific sounding jargon in wrong answers

Rule of thumb: Do not pick an answer which is too simple or obvious.

Way to defeat this strategy: Sometimes make the simple, obvious answer thecorrect one.

Rule of thumb: Pick a word which you remember was related to the topic.

Way to defeat this strategy: When drawing up distractors (wrong answers) useterminology from the same area of the text as the right answer, but in distractersuse those words incorrectly so the wrong answers are definitely wrong.


18/6912



7.Reliability (continued)Studies have shown that the grade given to an essay testdepend in part upon the neatness of the handwri ting. Thatseems like a poor way to assign a grade. However, i f stu-dents are asked to do the test on a word processor, it is hardto ensure that the work is original. Studies have also shownthat grades for essay tests are influenced by length. If a stu-dent rambles on, there is greater likelihood of hitting a fewpoints that the teacher is looking for. But do we want to

reward verbosity?

Despite all this, essay and short answer tests have many vir-tues. Students need practice formulating arguments, express-ing things clearly, and integrating ideas. Nobody would ar-gue that alltesting should be multiple choice. However, forteachers in many situations, a good objective test is bothfairer and more efficient than an essay or short answer test.One way to ensure reliability is to share wi th your students

1. Tell students to survey the entire test before they begin.This will help them identify which section will be quickand/or easy and which wi ll require more time and thought.

2. Encourage students to underline important words in thedirections such as list, discuss, define, etc.

3. Instruct students that when they take a test, they should dothe easy questions first.

4. Help students schedule their time by estimating the totaltime available compared to the number of questions onthe test. They need to recognize that some types of ques-

tions wi ll take longer than others.5. Suggest that students put a checkmark next to any ques-

tions which they left blank and will need to come back tofor completion later.

6. Prompt students to hold onto their test until they havelooked it over thoroughly. They should make sure theyhave completed each task and have reread the entire testto verify that they have given the answers they intended.

General Test Taking Tips

"Remind, remind,

remind students to

stop and ask for

dir ecti ons or

clari fi cation if there is

something they dont

understand.

D irecti ons are theroadmap to their final

destination."

Suggestion

Encourage students to designtheir own test. This wi ll helpthem anticipate some of thequestions or information to beincluded on the instructors

exam.

Various kinds of objectiveand essay test items are

presented in the followingsections of this document.

Each kind of test item isbriefly described in terms

of advantages andlimitations for use.

General suggestions arealso presented for the

construction of each testitem variation.


19/6913

Multiple Choice Test ItemsMultiple Choice Test ItemsMultiple Choice Test ItemsMultiple Choice Test ItemsMultiple Choice Test Items

Good for:! Application, synthesis,

analysis, and evaluationlevels

Types:! Question/Right answer! Incomplete statement

! Best answer

Advantages:! Very effective! Versatile at all levels! Minimum of writing for

student! Guessing reduced! Can cover broad range

of content

Disadvantages:! Difficult to construct good

test i tems! Difficult to come up with

plausible distractors/alterna-tive responses

"almost any well

defined cogniti ve

object ive can be tested

fairly in a multi ple

choice format."

Section Summary

The multiple choice item consists of the stem, which identifies thequestion or problem and the response alternatives or choices.Usually, students are asked to select the one alternative that bestcompletes a statement or answers a question. For example,

Item Stem: Which of the following is a chemical change?Response Alternatives: a. Evaporation of alcohol

b. Freezing of water

c. Burning of oil "d. Melting of wax

Multiple choice items are considered to be among the most versa-tile of all item types. They can be used to test factual recall as wellas levels of understanding and ability to apply learning. As anexample, the multiple choice item below is testing not only infor-mation recall but also the ability to use judgment in analyzing andevaluating.

Multiple choice tests can be used to test the ability to:1. recall memorized information2. apply theory to routine cases3. apply theory to novel situations4. use judgment in analyzing and evaluating

A. 1 onlyB. 1 and 2 onlyC. 1, 2 and 3 onlyD. 1, 2, 3 and 4 "

Multiple choice items can also provide an excellent basis for post-test discussion, especially if the discussion addresses why the in-correct responses were wrong as well as why the correct responseswere right. Unfortunately, multiple choice items are difficul t andtime consuming to construct well. They may also appear too dis-

criminating (picky) to students, especially when the alternativesare well constructed and are open to misinterpretation by studentswho read more into questions than is there.

Test your knowledge ofmultiple choice tests by taking

the multiple choice teston the next page


20/6914

Circle the Most Correct AnswerCircle the Most Correct AnswerCircle the Most Correct AnswerCircle the Most Correct AnswerCircle the Most Correct Answer

1. Multiple choice items provide highlyreliable test scores because:A. they do not place a high degree of

dependence on the students readingability

B. they place a high degree of depen-dence on a teacher's writing ability

C. they are a subjective measurement ofstudent achievement

D. they allow a wide sampling ofcontent and a reduced guessing factor

2. You should:A. always decide on an answer before

reading the alternativesB. always review your marked examsC. never change an answerD.always do the multiple choice items

on an exam first

3. The above multiple choice item isstructurally undesirable because:

A. a direct question is more desirablethan a incomplete statementB. there is no explici t problem or

information in the stemC. the alternatives are not all plausibleD.all of the aboveE. A & B onlyF. B & C onlyG.A & C onlyH.none of the above

4. The above multiple choice item is

undesirable because:A. it relies on an answer required in a

previous itemB. the stem does not supply enough

informationC. eight alternatives are too many and

too confusing to the studentD.more alternatives just encourage

guessing

5. The right answers in multiple choicequestions tend to be:A. longer and more descriptiveB. the same length as the wrong answersC. at least a paragraph longD.short

6. When guessing on a multiple choicequestion with numbers in the answer:A. always pick the most extremeB. pick the lowest numberC. pick answers in the middle rangeD.always pick C

7. What is the process of elimination in amultiple choice question?A. skipping the entire questionB. eliminating all answers with extreme

modifiersC.just guessingD.eliminating the wrong answers

7. What should you not do when takinga multiple choice test:A. pay attention to patternsB. listen to last minute instructionsC. read each question carefullyD. read all choices

8. It is unlikely that a student who is un-skilled in untangling negative statementswill:A. quickly understand multiple choice

items not written in this wayB. not quickly understand multiple choice

items wri tten in this wayC. quickly understand multiple choice

items wri tten in this wayD.not quickly understand multiple choice

items not written in this way

Answers:1-D,2-B,3-D,4-C,5-A,6-C,7-D,8-C


21/6915

Suggestions For Writ ing Multiple Choice Test ItemsSuggestions For Writ ing Multiple Choice Test ItemsSuggestions For Writ ing Multiple Choice Test ItemsSuggestions For Writ ing Multiple Choice Test ItemsSuggestions For Writ ing Multiple Choice Test Items

1. When possible, state the stem as a direct questionrather than as an incomplete statement.

Undesirable:

Alloys are ordinarilyproduced by

Desirable:

How are alloys ordinarilyproduced?

2. Present a definite, explicit and singular questionor problem in the stem.

Undesirable:

PsychologyDesirable:

The science of mind and be-havior is called

3. Eliminate excessive verbiage or irrelevantinformation from the stem.

Undesirable:

Whi le ironing her formal, Janeburned her hand accidentlyon the hot iron. This was dueto a transfer of heat between...

Desirable:

Which of the following waysof heat transfer explains whyJanes hand was burned aftershe touched a hot iron?

4. Include in the stem any word(s) that might other-wise be repeated in each alternative.

Undesirable:

In national elections in theUnited States the President is

officially:A. chosen by the people.B. chosen by members of

Congress.C. chosen by the House of

Representatives.D. chosen by the Electoral

College."

Desirable:

In national elections in theUnited States the President is

officially chosen byA. the people.B. members of Congress.C. the House of Reps.D. the Electoral college."

! Use at least four alternatives foreach item to lower the probabil-ity of getting the item correct byguessing.

! Use capital letters (A, B, C, D) asresponse signs rather than lower

case letters (a gets confusedwith d and c with a if thetype or duplication is poor).

! Randomly distribute the correctresponse among the alternativepositions throughout the test, hav-ing approximately the same pro-portion of alternatives A, B, C,and D as the correct response.

! Avoid irrelevant clues such asgrammatical structure, wellknown verbal associations or sim-plistic connections between stemand answer.

! When possible, present alterna-tives in some logical order (e.g.,chronological, most to least,

alphabetical).

! Use the alternatives none of theaboveand all of the abovespar-ingly. When used, such alterna-tives should occasionally be usedas the correct response.

Remember


22/6916

5. Use negatively stated stems sparingly. When used,underline and/or capitalize the negative word.

Undesirable:

Which of the following is not

cited as an accomplishment ofthe Kennedy administration?

Desirable:

Which of the following is NOT

cited as an accomplishment ofthe Kennedy administration?

6. Make all alternatives plausible and attractive to theless knowledgeable or skillful student.

Undesirable:

What process is mostnearly the opposite ofphotosynthesis?

A. DigestionB. Assimi lationC. Respiration "D. Catabolism

Desirable:

What process is mostnearly the opposite ofphotosynthesis?

A. DigestionB. RelaxationC. Respiration "D. Exertion

7. Make the alternatives mutually exclusive.

Undesirable:

The daily minimum requiredamount of milk that a 10 yearold child should drink is

A. 1-2 glasses.B. 2-3 glasses. "C. 3-4 glasses. "D. at least 4 glasses.

Desirable:

What is the daily minimumrequired amount of milk a 10year old child should drink?

A. 1 glass.B. 2 glasses.C. 3 glasses. "D. 4 glasses.

8. Make alternatives approximately equal in length.

Undesirable:

The most general cause of lowindividual incomes in the

United States is:A. lack of valuable productive

services to sell. "B. unwillingness to work.C. automation.D. inflation.

Desirable:

What is the most general causeof low individual incomes in

the United States?A. A lack of valuable produc-

tive services to sell. "B. The populations overall

unwillingness to work.C. The nations increased

reliance on automation.D. An i ncreasing nati onal

level of inflation.

Remember

! In testing for definitions, use theterm in the stem rather than asan option.

! List alternatives on separatelines (rather than including theoptions as part of the stem) sothat all options can be clearlydistinguished.

! Keep all alternatives in a simi-lar format (i.e., all phrases, all

sentences, etc.).

! Try to make alternatives for anitem approximately the samelength. (Making the correct re-sponse consistently longer is acommon error.)

! Use misconceptions studentshave indicated in class or errorscommonly made by students inthe class as the basis for incor-rect alternatives.

! Way to judge a good stem: stu-dents who know the contentshould be able to answer beforereading the alternatives.

Multiple choice exams pro-vide easier conditions forcheating than essay tests sincesingle letters or numbers areeasier to see than extensivetext. Cheating can be mini-mized by using alternative testforms and controlling seating.


23/6917

1. Read the directions carefully

The directions usually indicate that some alternatives may bepartly correct or correct statements in themselves, but not when

joined to the stem. The directions may say: choose the mostcorrect answer or mark the one best answer. Sometimes youmay be asked to mark all correct answers.

2. Do the multiple choice items first

If your exam has types of questions other than multiple choice,just reading the stems and alternatives acts is a warm-up to thematerial. (The stem is the question and the alternatives are thechoices). Also, the ideas embedded in these multiple choicequestions wi ll fuel your thinking for doing the other parts of theexam. Use the process of elimination procedure. Eliminate theobviously incorrect alternatives.

3. Read all of the stem and every alternative

Read the stem with each alternative to take advantage of thecorrect sound or flow that the correct answer often produces.Also, you can eliminate any alternatives that do not agree gram-matically wi th the stem.

Some students find it effective to read the stem and anticipatethe correct alternative before actually looking at the alternatives.If you generally do better on essay exams, this strategy may helpyou a great deal.

4. Consider "all of the above" and "none ofthe above"

Examine the above alternatives to see if all of them or none ofthem apply totally. If even one does not apply totally, do notconsider all of the above or none of the above as the correctanswer. Make sure that a statement applies to the question sinceit can be true, but not be relevant to the question at hand!

Attention Students: Mult iple Choice Test Taking TipsAttention Students: Multiple Choice Test Taking TipsAttention Students: Mult iple Choice Test Taking TipsAttention Students: Multiple Choice Test Taking TipsAttention Students: Multiple Choice Test Taking Tips

Pay attention tothe words

! Note quali fying words: usually,often, generally, may,and sel-dom are qualifiers that couldindicate a true statement.

! Words such as every, all, none,always, and onlyare superla-tives that indicate the correctanswer is an undisputed fact. Ingeneral, absolutes are rare.

! If a negative word such asnone,not, never,or neitheris in thestem, assume that the correctalternative must be a fact or ab-solute and that the other alter-natives could be true statements,but not the correct answer.



24/6918

5. Plan your time

Often you are required to answer up to 70 multiple choice ques-tions in an hour or less. This means you may have less than a

minute, on average, to spend on each question. Some ques-tions, of course, will take you only a few seconds, while otherswi ll require more time for thought.

Plan to progress

thr ough the exam

in thr ee ways:

! Read every question carefullybut quickly, answering onlythose of which you are 100%certain. Put a ? on those that

need more thought.

! Then, examine/study the ques-tions not yet answered. Answerthose you are reasonably sureof wi thout pondering too longon each. Erase the ?.

! Finally, study the remaining un-answered questions. If you can-not come to a decision by rea-

soning or if you run out of time,guess. Erase the ?. Note thatsome examinations penalizeguessing by subtractingpoints for incorrect answers.

If there is no penalt y,

then a guess is bet ter

than a blank.

6. Changing answers

Research has shown that changing answers on a multiple choiceor true-false exam is neither good nor bad: if you have a goodreason for changing your answer, change it. The origin of themyth that people always change from right to wrong is thatthose (i.e. the wrong ones) are the only ones you will see when

you review your examyou will not notice the ones youchanged from wrong to right.

Study your marked and returned exam in orderto learn from your successes and mistakes.

1. Examine each question you did get correct. Rememberhow you knew that the information was important whenyou studied. How did you study?

2. Examine each question you did not get correct in order tounderstand the distinction between the correct alternativeand the incorrect alternatives. Ask yourself why the correct

answer is correct and why the other alternatives areincorrect.

3. Determine the level of thought your instructor expects ofyou by reading through all of the questions. Are youexpected to recognize, analyze, synthesize and/or applythe material that has been presented to you? Studyaccordingly for the next exam.

Af ter Your ExamHas Been Returned

This w il l pay dividends on future exams


25/6919

Most teachers find it easier to construct multiple choice items totest recall and comprehension and to use essay items to test higher-level learning objectives. But other possibil ities exist. Mul tiplechoice items that require students to do such things as classify state-ments as fact or opinion go beyond simple recall of facts.

Multiple Choice Test Items: ConclusionMultiple Choice Test Items: ConclusionMultiple Choice Test Items: ConclusionMultiple Choice Test Items: ConclusionMultiple Choice Test Items: Conclusion

Why are thesemultiple choice

questions crummy?

1. How frequently do you take asick day from work?A. never

B. once or twice a yearC. 3 to 5 times a yearD. 6 to 12 times a yearE. at least once a month

2. Identify the issue that you be-lieve is most critical to thiscountry's future.A. the economyB. educationC. integrity in governmentD. national defense

E. some other issue

" Understand that there is

always one clearl y best

answer. My goal is not to

tr ick students or r equire

them to make diff icult

judgment s about two op tions

that are nearly equally

corr ect. My goal is to design

questi ons that students who

understand w ill answer

corr ectly and students who

do not understand wi ll

answer incorrectl y."

John A . John son

Dept. of Psychology,

Penn State University

Ask yourself : Aim for Higher Levels of Learning

Here are tw o examples of mult ipl e choice test

it ems designed for h igher order thi nking skil ls.

A common goal of the Salt March in India, theBoxer Rebellion in China, and the Zulu resistancein southern Africa was to:A. overthrow totali tarian leadersB. force upper classes to carry out land reform

programsC. remove foreign powersD. establish Communist parties to lead the

governments

In western Europe, which development caused theother three?A. decline of tradeB. fall of RomeC. breakdown of central governmentD. rise in the power of the Roman Catholic Church

One way to write multiple choice questions that require more thanrecall is to develop questions that resemble miniature "cases" orsituations. Provide a small collection of data, such as a description

of a situation, a series of graphs, quotes, a paragraph, or any clusterof the kinds of raw information that might be appropriate materialfor the activities of your discipline.

Then develop a series of questions based on that material. Thesequestions might require students to apply learned concepts to thecase, to combine data, to make a prediction on the outcome of aprocess, to analyze a relationship between pieces of the informa-tion, or to synthesize pieces of information into a new concept.


26/6920

True-False Test ItemsTrue-False Test ItemsTrue-False Test ItemsTrue-False Test ItemsTrue-False Test Items

There are many situations

which call for either-or

decisions, such as deciding

whether a specific solut ion

is right or wrong, whether

to continue or to stop,

whether to use a singular

or plural construction, and

so on. For such sit uations,

the true-false it em is anideal measur ing device.

Section Summary

Good for:! Knowledge level content! Evaluating student under-

standing of popular miscon-ceptions

! Concepts with two logicalresponses

Advantages:! Can test large amounts of

content! Students can answer 3-4

questions per minute

Disadvantages:! They are easy! It is difficult to discriminate

between students that knowthe material and studentswho do not

! Students have a 50-50chance of getting the rightanswer by guessing

! Need a large number of i temsfor high reliability

In the most basic format, true-false questions are those in which astatement is presented and the student indicates in some mannerwhether the statement is true or false. In other words, there areonly two possible responses for each item, and the student choosesbetween them. True-false questions are well suited for testing stu-dent recall or comprehension. Students can generally respond tomany questions, covering a lot of content, in a fairly short amountof time.

From the teachers perspective, true-false questions can be written

quickly. They are easy to score. Because they can be objectivelyscored, the scores are more reliable than for items that are at leastpartially dependent on the teachers judgment.

Select or Supply?True-false questions require the students to select a response (trueor false) that shows recognition of correct or incorrect informationthat is presented to them. These are included among the itemsthat are called selection,in contrast to supplyitems in which thestudent must supply the correct information.

Forced Choi ceAnother term applied to true-false items is forced choicebecausethe student must choose between two possible answers. Educa-tional objectives that specify the student wi ll identify,select,andrecognizematerial are appropriately targeted to either forced choicequestions or more complex matching or multiple choice questions.

Much Maligned and AbusedMany educators feel that true-false test items serve little or no mea-surement purposes because true-false items are subject to guess-ing. (But the likelihood of obtaining a substantiall y higher than

chance score by guessing alone is very small). In general, indi-vidual true-false items are less discriminating than individual mul-tiple choice items. There is a tendency to write trivial true-falseitems, which lead students to verbatim memorization. At the sametime, no diagnostic information is available from incorrect responsesto true-false items. Finally, true-false items are not amenable toconcepts that cannot be formulated as propositions.


27/6921

Check Your Knowledge ofTrue-False Test Items

Summarizing the Argument for the

Value of True-False Test I tems

! The essence of educational achievement is the command ofuseful verbal knowledge.

! All verbal knowledge can be expressed in propositions.! A proposition is any sentence that can be said to be true or false.! The extent of students command of a particular area of knowl-

edge is indicated by their success in judging the truth or falsity ofpropositions related to i t.

Ebel and Frisbie (1991)

Since true-false questions tend tobe either extremely easy orextremely difficult, they do notdiscriminate between students ofvarying ability as well as other

types of questions do.

1. Is it recommended to take statements directly from thetext to make good true-false questions?A. YesB. No

2. Two ideas can be included in a true-false statement ifthe purpose is to show cause and effect.A. YesB. No

3. When a true-false statement is an opinion, it should beattributed to someone in the statement.A. Yes

B. No4. Underlining or circling answers is preferable to having

the student write them.A. YesB. No

Circle " Good" if i t describes a good practice in t rue-false question s,

circl e Poor if it characteri zes a poor practice.

5. Complex statements are used to measure higher orderthinking. Good Poor

6. If negatives, such as not, are used, they should behighlighted in some way. Good Poor

7. True and false statements should be approximately thesame length. Good Poor8. There should be a recognizable pattern in the answers,

such as TFTFTFTF. Good Poor9. The following are examples of words that should be

avoided: all, none, never, sometimes, gener-ally, and often. Good Poor

Di rection s: For each question below , circle A or B.! Versatil ityTrue-false items

are adaptable to the measure-ment of a wide variety oflearning outcomes.

! Scoring accuracy andeconomyScoring keys canbe economically applied bymachine or clerical assistants.

! ReliabilityTrue-false teststhat are highly reliable can beconstructed.

! Amenable to item analysisTrue-false items are amenableto item analysis, by means ofwhich they can be improved.

! EfficiencyMore test re-sponses can be obtained froma given amount of writtenmaterial and in a givenamount of time than from

other forms.

! True-false items are useful intesting misconceptions.

! True-false items can be ex-pressed in few words, makingthem less dependent onreading ability.

Making the Case forTrue-False Items

Answers:1-B,2-A,3-A,4-A,5-Poor,6-Good,7-Good,8-Poor,9-Good


28/69


29/6923

Extr eme Modi fi ers:

all nonealways neveronly nobody

invariably no onebest absolutelyworst absolutely noteverybody certainlyeveryone certainly not

Qualifiers:

usually frequentlyoften sometimessome seldommany much

probably a majorityapt to mostmight a fewmay unlikely

5. Avoid the use of extreme modifiers or qualifiers.

Undesirable:

Allsessions of Congress are

called by the President. (F)The Supreme Courtfrequently rules on the con-stitutionality of law. (T)An objective test isgenerallyeasier to score thanan essay test. (T)

Desirable:

The sum of the angles of a

triangle is always180o

. (T)The galvanometer is the in-strument usuallyused for themetering of electrical energyused in a home. (F)

6. Avoid lifting statements from the text, lecture orother materials so that memory alone will not permit a

correct answer.Undesirable:

For every action there is anopposite and equal reaction.

Desirable:

If you were to stand in acanoe and throw a life jacketforward to another canoe,chances are your canoewould jerk backward.

7. Avoid using negatively stated item statements.

Undesirable:

The Supreme Court is notcomposed of nine justices.

Desirable:

The Supreme is composed ofnine justices.

8. Avoid the use of unfamiliar vocabulary.

Undesirable:

According to some politicians,the raison detre for capitalpunishment is retribution.

Desirable:

According to some politicians,justification for capital punish-ment is retribution.

Remember

! Determine that the questions areappropriately answered by

True or False rather than bysome other type of response,such as Yes or No.

! Arrange the statements so thatthere is no discernible pattern ofanswers (such as T , F, T, F, T, Fand T, T, F, F, T, T, F, F) for Trueand False statements.

! Avoid the tendency to add de-tails in true statements to make

them more precise. The answersshould not be obvious to stu-dents who do not know the ma-terial.

! Be sure to include directions thattell students how and where tomark their responses.

Writing Hint

One method for developing true-false items is to write a set oftrue statements that cover the content, then convert approximatelyhalf of them to false statements. Remember: When changingitems to false (as well as in writing the true statements initially),state the items positively, avoiding negatives or double negatives.


30/6924

Attention Students:Attention Students:Attention Students:Attention Students:Attention Students:

True-False Test Taking TipsTrue-False Test Taking TipsTrue-False Test Taking TipsTrue-False Test Taking TipsTrue-False Test Taking Tips

! When you do not know or cannot remember information to determine thetruth of a statement, assume that it is true.

! There are generally more true questions on true-false exams than false questionsbecause instructors tend to emphasize true questions.

! If there is specific detail in the statement, it may also tend to be true. Forexample, the statement "There are 980 endangered species worldwide" hasspecific detail and is li kely to be true.

! Look for extreme modifiers that tend to make the question false. Extreme

modifiers, such as always, all, never, or only make it more likely that thequestion is false.

! Identify qualifiers that tend to make the question true. Qualifiers (seldom,often, many) make the question more likely true.

! Questions that state a reason tend to be false.

! Words in the statement that cause justification or reason (since, because, when,if) tend to make the statement false because they bring in a reason that isincorrect or incomplete.

Variations in Writ ingVariations in WritingVariations in Writ ingVariations in WritingVariations in Writ ing

True-False Test ItemsTrue-False Test ItemsTrue-False Test ItemsTrue-False Test ItemsTrue-False Test Items

The True-False-Correction Quest ion

In this variation, true-false statements are presented with a key word or briefphrase that is underlined. It is not enough that a student correctly identify astatement as being false. To receive credit for a statement labeled false, thestudent must also supply the correct word or phrase which, when used to re-place the underlined part of the statement, makes the statement a true one.

This type of item is more thorough in determining whether students actuallyknow the information that is presented in the false statements. While a studentmight correctly guess that a statement is false, no credit would be given unlessthe student could change the statement to a true one by writing word/words toreplace underlined word(s).



31/6925

Variations in Writ ingVariations in WritingVariations in Writ ingVariations in WritingVariations in Writ ing

True-False Test ItemsTrue-False Test ItemsTrue-False Test ItemsTrue-False Test ItemsTrue-False Test Items(continued)

The teacher decides what word/phrase can be changed in the sentence; if stu-dents were instructed only to make the statement a true statement, they wouldhave the liberty of completely rewriting the statement so that the teacher mightnot be able to determine whether or not the student understood what was wrongwi th the original statement.

If, however, the underlined word/phrase is one that can be changed to its oppo-site it loses the advantage over the simpler true-false question because all thestudent has to know is that the statement is false and change isto is not.

The Yes-No Variation

In the yes-no variation, the student responds to each item by wri ting, circling orindicating yes-no rather than true-false. An example follows:

What reasons are given by students for taking evening

classes? In the li st below , circl e Yesif that is one of the

reasons given by students for enrol li ng in evening classes;

circle Noif that i s not a reason given by students.

Yes No They are employed during the day.Yes No They are working toward a degree.Yes No They l ike going to school.Yes No There are no good television shows to watch.Yes No Parking is more plentiful at night.

The A-B Variation

The example below shows a question for which the same two answers apply.The answers are categories of content rather than true-false or yes-no. This isanother form of forced choice question because for each item the student mustchoose between Aand B.

Indicate whether each type of question below is a selection

type or supply type by circli ng Aif it is selection, Bif itis supply.

Select SupplyA B Multiple choiceA B True-FalseA B EssayA B MatchingA B Short Answer


32/6926

While true-false and other forced choice questions are generallyused to measure knowledge and understanding, they could alsobe used at higher levels. Students could be provided with a set ofinformation new to them, perhaps a portfolio, set of data, or a writ-ten work of some type, then asked various forced choice questionsrelated to the content or the presence/absence of certain character-istics in the work.

Anticipate Scoring Ranges

Scores on true-false items tend to be high because of the ease ofguessing correct answers when the answer is not known. Withonly two choices (true or false) the student could expect to guesscorrectly on half of the items for which correct answers are notknown.

If a student knows the correct answers to 10 questions out of 20and guesses on the other 10, the student could expect a score of15. The teacher can anticipate scores ranging from approximately50% for a student who did nothing but guess on all items to 100%for a student who knew the material.

In the final analysisThe true-false test is probably the best known of the various typesof objective test items. It is the easiest to construct and at the sametime the most abused. The students learn the weaknesses that areinherent in many such i tems and are able to obtain high scores bynoting the grammatical construction, the choice of words or otherclues.

The true-false test can be used effectively as an instructional test topromote interest and introduce points for discussion. This perhapsis the most important use for the plain true-false item. It is a valu-able type of test to use in giving short, daily quizzes that may beused to motivate the students for a new assignment, to review aprevious lesson, to locate points to be retaught or to introduce con-troversial points for class discussion.

True-False Test Items: ConclusionTrue-False Test Items: ConclusionTrue-False Test Items: ConclusionTrue-False Test Items: ConclusionTrue-False Test Items: Conclusion

Why are thesetrue-false questions

crummy?

1. There is no advantage for notusing specific determiners intrue-false items. T F

2. Test validity is a function oftest reliability, which can beimproved by using feweritems. T F

3. A nickel i s larger than adime. T F

4. An eagle's range of sight isprecisely 1,000 ft. T F

5. The telephone was inventeda long time ago. T F

Ask yourself :Aim for Higher Levels of Learning

"A major distinction betweenthe true-false test i tem and itemsin a multiple choice format, isthat the true-false statementcontains no criterion foranswering the question. Eachexaminee must ask the question:True or false w ith respect to

what? Each true-false item mustbe unequivocally true orunequivocally false. It isimperative that proper wordingand the elimination ofextraneous clues are morecrucial wi th the true-false itemthan with any other test format."

Writing Test Items, na, Michigan State UniversityDept. of Education, Dec. 1999


33/6927

Matching questions

prov ide a most effi cient

way to test know ledge in

courses in which events,

dates, names, and places

are impor tant. Matching

questions are also

appropri ate for the

sciences in w hich

numerous experiments,experimenters, resul ts,

and special t erms and

defini ti ons have to be

remembered.

Matching Test ItemsMatching Test ItemsMatching Test ItemsMatching Test ItemsMatching Test Items

Section Summary

Good for:! Knowledge level! Some comprehension level,

if appropriately constructed

Types:! Terms with definitions! Phrases with other phrases! Causes with effects! Parts with larger units! Problems wi th solutions

Advantages:! Maximum coverage at know-

ledge level in a minimumamount of space/preptime

! Valuable in content areas thathave a lot of facts

Disadvantages:! Time consuming for students! May not be appropriate for

higher levels of learning

A simple matching item consists of two columns: one column ofstemsor problems to be answered, and another column of re-sponsesfrom which the answers are to be chosen. Traditionally,the column of stems is placed on the left and the column of re-sponses is placed on the right. An example is given below.

Directions: On the line next to each childrens book inColumn A print the letter of the animal or insect in ColumnB that is a main character in that book. Each animal or insectin Column B can be used only once.

Column A Column B____1. Charlottes Web A. Bear____2. Winnie the Pooh B. Chimpanzee____3. Black Beauty C. Cricket____4. Tarzan D. Deer____5. Pinocchio E. Horse____6. Bambi F. Pig

The student reads a stem (Column A) and finds the correct responsefrom among those in Column B. The student then prints the letterof the correct response in the blank beside the stem in Column A.An alternative is to have the student draw a line from the correctresponse to the stem, but this is more time consuming to score.

In the above example notice that the stems in Column A are as-signed numbers (1, 2, 3, etc.). The items in Column B are desig-nated by capital letters. Capital letters are used rather than lowercase letters in case some students have reading problems. Alsothere are apt to be fewer problems in scoring the students hand-written responses if capital letters are used.

Also in the above example, the student only has to know five ofthe six answers to get them all correct. Since each animal in Col-umn B can be used only once, the one remaining after the fiveknown answers have been recorded is the answer for the sixthpremise. One way to reduce the possibility of guessing correctanswers is to list a larger number of responses than premises.


34/6928

Test Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test Items

1.Problem: Faulty directions.

Directions: "Place the letter of the term in the right hand column on the line to theleft of the definition column."

Circle the letter(s) that describe the best way to revise these directions:A. Add: Match the followingB. Add: Each term may not be used more than onceC. Change the order of the directions providedD. No changes needed

2.Problem: Unrelated topics.

____1. Year in which WWII began A. Joseph Stalin____2.British Prime Minister in WWII B. Franklin D. Roosevelt____3.U.S. President during WWII C. 1939____4. German dictator in WWII D. Winston Churchill

E. Adolf Hitler

Circle the letter(s) that describe the best way to revise this matching test.

A. Change one of the descriptions to read: Russian dictator in WWIB. Add an item to the left hand columnC. Add a description that reads: Year in which WWI began

D.Remove option C. from the right hand columnE. Remove all stimuli and responses that do not concern leaders in WWII

3.Problem: Mixing matching wi th completion.

Directions: On the line to the left of each statement wri te the letter of the atomicparticles from the right hand column that the statement describes. Use each particleonly once.

____1.An ____orbits the nucleus. A. Electron____2.Positively charged particles are called _____. B. Neutron____3.A _____ has no charge. C. Protons____4.The _____ is located in the center of an atom. D. Nucleus

E. Ion

Circle the letter(s) that describe the best way to revise this matching test.A. Edit all the stimuli on the left to be complete statements.B. Remove all the blanks from the stimuli on the left.C. Change the order of the responses on the right.D.Edit the stimuli to be grammatically unbiased (i.e. singular/plural)

Answers:1-C,2-E,3-A,B&D


35/6929

Test Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test ItemsTest Your Knowledge of Matching Test Items

4.Directions: The four statements presented below refer to the structure of thematching test, specifically what elements should be in Column A and what elementsshould be in Column B. At the left of each statement are the lettersAand B. CircleAif Column A is the best choice; circle Bif Column B is the best choice.

A B I. When presenting words and their definitions, which column shouldcontain the definitions, which are longer than the words?

A B 2. Items arranged in chronological order would be found in whichcolumn?

A B 3. Premise is the term applied to the items in which column?A B 4. Items are designated by numbers in which column?

5.Directions: For the four learning objectives listed below, decide whether amatching exercise would be an appropriate method of assessment (Assume thatyou can construct a list of 6-8 items for the matching question.) CircleYESifappropriate; circle NOi f not appropriate.

YES NO A. The student wil l be able to recognize the cities in/near which themajor battles in the American Revolution took place.

YES NO B. The student will be able to di fferentiate between words that arespelled correctly and those spelled incorrectly.

YES NO C. The student wil l be able to identify the elements wi th their symbolsfrom the periodic table.

YES NO D. The student will be able to identify the English words for variousfruits that are represented by their Spanish language counterparts.

6.Directions: On the lines following this matching question supply fourrecommendations to improve this question.

Hitler A. Year in which WWII beganMulroney B. A Canadian Prime MinisterTank C. A German dictator during the WWII1939 D. An armored vehicle used original ly to

break the trench war stalement in WWI

Recommendations:

1._____________________________________________________________________

2._____________________________________________________________________

3._____________________________________________________________________

4._____________________________________________________________________

Answers:4-A,B,A,A;5-Yes,No,Yes,Yes;6-Examples:Needdirections,reverseColumnAandColumnB,makeitemssimilar,increasethenumberofresponses


36/6930

Suggestions For Writ ing Matching Test ItemsSuggestions For Writ ing Matching Test ItemsSuggestions For Writ ing Matching Test ItemsSuggestions For Writ ing Matching Test ItemsSuggestions For Writing Matching Test Items

! Review your teaching objectivesto make sure that a matchingcomponent is appropriate.

! Keep matching items brief, limit-ing the list of stimuli to 10 - 15.

! When possible, reduce theamount of reading time by in-cluding only short phrases orsingle words in the response list.

! Use the more involved expres-sions in the stem and keep theresponses short and simple.

! Arrange the list of responses insome systematic order if possible(chronological, alphabetical).

! Make sure that there are nevermultiple correct responses forone stem (although a responsemay be used as the correct an-swer for more than one stem).

! Avoid breaking a set of items(stems and responses) over twopages. (Students go nuts flippingpages.)

Remember

1. Include directions which clearly state the basis formatching the stimuli with the responses.

Undesirable:

Directions: Match the following.

Desirable:

Directions: On the line to the left of each identifying location

and characteristics in Column I, write the letter of the countryin Column II that is best defined. Each country in Column IImay be used more than once.

Explain whether or not a response can be used more thanonce and indicate where to write the answer.

2. Use only items that share the same foundation ofinformation.

Undesirable:


1. Water A. NaCl2. Discovered Radium B. Fermi3. Salt C. NH34. Ammonia D. 19425. Year of the first E. H

20

Nuclear Fission F. CurieG.1957

Desirable:

Directions: On the line to the left of each compound in Col-umn I, write the letter of the compounds formula presented in

Column II. Use each formula only once.

Column I Column II____1. Water A. H

2S0

4

____2. Salt B. HCl____3. Ammonia C. NaCl____4. Sulfuric Acid D. H

20

E. H2HCl

Unrelated topics included in the same matching item mayallow for obvious matches and mismatches.


37/6931

3. Avoid grammatical or other clues to the correct response.

Undesirable:

Directions: Match the following in order to complete the sentences on the left.____1.Plato insisted that government was A. The Prince.____2.Machiavell i wrote about achieving political B. desirable and inevitable

unity in C. a science requiring experts.____3.Hobbes argued that human nature made D. organized along industrial lines.

absolute monarchy E. Communism.____4.Marx was a German philosopher and

economist who founded

Desirable:

Directions: On the line to the left of each statement write the letter of the philosopher

from the right hand column that the statement describes. Use each philosopher once.

____1.Thought government was a science requiring experts. A. Hobbes____2.Described methods of achieving political unity. B. Marx____3.Founded Communism. C. Machiavelli____4.Believed that human nature made absolute D. Durkheim

monarchy desirable and inevitable E. Plato

4. The column of stimuli on the left should set the question clearly.

Undesirable:


____1. City dwellers A. Wild animals____2. Hunter-gatherers B. Farm____3. Pastoral nomads C. Apartment bui ldings

D. Graze animals

Desirable:

Directions: On the line to the left of each definition, write the letter of the term in theright hand column that is defined. Use each term only once.

____1. Live in areas of high population density. A. Pastoral nomads____2. Move from one place to another in search of wild animals. B. Ranchers____3. Move from one place to another with grazing animals. C. Hunter-gatherers____4. Till land for cash crops. D. City dwellers

E. Farmers


38/6932

Attention Students:Attention Students:Attention Students:Attention Students:Attention Students:

Matching Test Taking TipsMatching Test Taking TipsMatching Test Taking TipsMatching Test Taking TipsMatching Test Taking Tips

! Read the directions. There are usually two lists that need to be matched. Takea look at both l ists to get a feel for the relationships and build your confidence.

! Use one list as a starting point and go through the second list to find a match.This process organizes your thinking. It will also speed your answers becauseyou become familiar with the second list and wi ll be able to go straight to amatch that you saw when looking through the lists a previous time.

! Move through the entire list before selecting a match. If you make a matchwith the first likely answer, you may make an error, because an answer laterin the list may be more correct.

! Cross off i tems on the second list when your are certain that you have a match.This seems simplistic, but it helps yo

Documents

Trick Question