Making a test more reliable

By Miguel Angel Carranza, MsE.

On Reliability.. How to improve a test

Provide clear and explicit instructions Write unambiguous items..

What is a table? The expected answer which table? Do not allow students too much freedom Ensure that tests are well laid out and

perfectly legible: if you include drawings, they should be clear.

How to Make Tests More Reliable (cont.)

Students should be familiar with format and testing technique

Use items that permit scoring which is as objective as possible

Agree acceptable responses and appropriate scores

Train scorers Identify candidates by number, not

name.

More Tips for Test design

Distractors should be parallel in form. Don’t allow one item to give a clue to

another one. All distractors should be about the

same length in terms of number of words.

The use of True-False items is not recommended.

Subjectively scored items should be scored by a sample of raters. /ideally

More Tips for Test design

Review items before testing. There shouldn`t be any typos (typoes?)!!!

If you write the test questions, have a qualified colleague read through them at least once to ensure accuracy (pilot test if possible)

Make the distractors plaussible and attractive to the uninformed.

Analyzing test items (cont.) The teacher is usually silent in this method.Suggestopedia b) Direct Method c)Silent Way

The type of standardized test used to measureAcademic achievement is called ana) Case studyb) Special aptitude testc) Test of intelligenced) Achivement test because formal classroom learning

is measured

Analizing multiple choice items

-- Which of the following persons is the best composer?a) Bach b) Beethoven c) Brahms d) Mozart

This item measures opinion. Bad item

--The fribbled breg will minter best with a a)dignu elma b) erst pinret c) ornt farg d)urtar bink

-The purpose of the cluss in furmpaling is to removea) Tremalisb) cloughs c) cluss-prags d) plumots.The answer is included in the stem. Bad item.

Critiquing Items All but one of the following is not an element. Which

is not?A) carbonb) salt c)sugar d)plastic

The most important city in the southeast is_________Better:The main transportation center for train and airplane

traffic in the Southeast is_____a) Atlanta B) miami c) New Orleans d) Orlando - A figure that has 8 sides is called an____a) pentagon b) quadrilateral c) octagon d) ogive

Another example.. Is anything wrong with it? Who was George Washington?

Ernest Hemingway wrote_______________.

Write an essay in which you consider the future of atomic energy.

Describe what happened to art during the Renaissance

Billi´s mother wanted to bake an apple pie for his aunt and uncle, who were coming for a visit. Billy had not seen them for many months. When Billy`s mother saw that she had no apples in the house, she sent Billy to the store to buy some. Her recipe called for 8 apples to make a pie. If apples at the store cost 30 cents for two, how much money would Billy need to buy eight apples?

A) $.30 B) $.90 C)$1.20 D) $2.40

Switzertand A) is located in AsiaB) produces large quantities of gold.C) has no direct access to the oceanD) is a flat, arid plain.

- As compared to autos of the 1960`s, autos in the 1980s a) more horsepower.b) to use more fuelc) contain more safety featuresd) was less often constructed in foreing countries

Analyzing test itemsWhat method or approach do thefollowing principles correspond to?1. Accuracy is emphasized______2. Grammar is taught inductively______Choose the method for the corresponding

principle1. Teacher motivates the student and makes them

feel comfortablea) Community L. L. b)Suggestopedia c) Silent

Way

What is a Table of Specifications?

A Table of Specifications is a two-way chart which describes the topics to be covered by a test and the number of items or points which will be associated with each topic. Sometimes the types of items are described, as well.

Example

Designing a Table of Specifications

Tables of Specification typically are designed based on the list of course objectives, the topics covered in class, the amount of time spent on those topics, textbook chapter topics, and the emphasis and space provided in the text.

In some cases a great weight will be assigned to a concept that is extremely important, even if relatively little class time was spent on the topic.

How can the use of a Table of Specifications benefit your students?

A Table of Specifications benefits students in two ways. First, it improves the validity of teacher-made tests. Second, it can improve student learning as well.

A Table of Specifications helps to ensure that there is a match between what is taught and what is tested. Classroom assessment should be driven by classroom teaching which itself is driven by course goals and objectives.

Formative and Summative EvaluationFormative evaluations, are ongoing, they provide information to improve

a product or process. Summative evaluations

provide short-term effectiveness or long-term impact information to decide whether or not to adopt a product or process. Summative evaluation can occur just after new course materials were implemented in full (i.e., effectiveness evaluation), or several months to years after the materials were implemented in full.

OR How about if I don’t give you grades until the end of the semester? A passing or failing.