LECTURES OBJECTIVES Identify the different formats of tests
found Distinguish different types of tests: norm-referenced and
criterion-referenced tests Compare and contrast formative and
summative tests Differentiate between objective and subjective
tests (Main reference - Brown, H. Douglas, 2004. Language
Assessment: Principles and classroom practices. ) 2
Slide 3
NORM-REFERENCED & CRITERION REFERENCED TESTS 3
Slide 4
NORM-REFERENCED TESTS To rank each student with respect to the
achievement of others in broad areas of knowledge. Normed using
large groups of test takers. Compares one taker to another. Measure
achievement, predicts future performance. Each individual is
compared with other examinees and assigned a score--usually
expressed as a percentile, a grade or equivalent score. Student
achievement is reported for broad skill areas, although some
norm-referenced tests do report student achievement in specific
sub-areas. 4
Slide 5
NORM-REFERENCED TEST Measures broad skill areas sampled from a
variety of textbooks, syllabi, and the judgments of curriculum
experts. Each skill is, usually, tested by less than four items.
Items vary in difficulty. Items are selected that discriminate
between high and low achievers. If too many people get a question
correct, or too many score well, then test questions are thrown out
until they achieve a normal curve again. 5
Slide 6
CRITERION-REFERENCED TEST Criterion-referenced tests, also
called mastery tests, compare a person's performance to a set of
objectives. Anyone who meets the criterion can get a high score.
Everyone knows what the benchmarks / objectives are and can attain
mastery to meet them. It is possible for ALL the test takers to
achieve 100% mastery. Measure a student against a specific set of
knowledge (criterion). 6
Slide 7
CRITERION-REFERENCED TEST To determine whether each student has
achieved specific skills or concepts. To find out how much students
know before instruction begins and after it has finished. Measures
specific skills which make up a designated curriculum. These skills
are identified by teachers and curriculum experts. Each skill is
expressed as an instructional objective. Each individual is
compared with a preset standard for acceptable achievement. The
performance of other examinees is irrelevant. Each skill is tested
by at least four items in order to obtain an adequate sample of
student performance and to minimize the effect of guessing. The
items which test any given skill are parallel in difficulty. 7
Slide 8
NORM & CRITERION REFERENCED TESTS
DimensionCriterion-Referenced Tests Norm-Referenced Tests Purpose
To determine whether each student has achieved specific skills or
concepts. To find out how much students know before instruction
begins and after it has finished. To rank each student with respect
to the achievement of others in broad areas of knowledge. To
discriminate between high and low achievers. Content Measures
specific skills which make up a designated curriculum. These skills
are identified by teachers and curriculum experts. Each skill is
expressed as an instructional objective. Measures broad skill areas
sampled from a variety of textbooks, syllabi, and the judgments of
curriculum experts. The following is adapted from: Popham, J. W.
(1975). Educational evaluation. Englewood Cliffs, New Jersey:
Prentice-Hall, Inc. 8
Slide 9
NORM & CRITERION REFERENCED TESTS
DimensionCriterion-Referenced Tests Norm-Referenced Tests Item
Characteris- tics Each skill is tested by at least four items in
order to obtain an adequate sample of student performance and to
minimize the effect of guessing. The items which test any given
skill are parallel in difficulty. Each skill is usually tested by
less than four items. Items vary in difficulty. Items are selected
that discriminate between high and low achievers. 9
Slide 10
NORM & CRITERION REFERENCED TESTS
DimensionCriterion-Referenced Tests Norm-Referenced Tests Score
Interpre- tation Each individual is compared with a preset standard
for acceptable achievement. The performance of other examinees is
irrelevant. A student's score is usually expressed as a percentage.
Student achievement is reported for individual skills. Each
individual is compared with other examinees and assigned a
score--usually expressed as a percentile, a grade equivalent score,
or a stanine. Student achievement is reported for broad skill
areas, although some norm-referenced tests do report student
achievement for individual skills. 10
Slide 11
Uses of Test Results for Teachers Two main ways that test
results can be used by teachers: For revising instruction for
entire class. For developing intervention strategies for individual
students. Standardized test results have not typically been used to
aid teachers in making instructional decisions. Data-driven
decision making takes some practice and experience for classroom
teachers. NORM & CRITERION REFERENCED TESTS 11
Slide 12
Norm-referenced General ability Range of ability Large groups
Compares people to people-comparison groups Selecting top
candidates Criterion-referenced Mastery Basic skills Prerequisites
Affective Psychomotor Grouping for instruction COMPARING NORM &
CRITERION-REFERENCED TESTS 12
Slide 13
COMMON CHARACTERISTICS OF NRT & CRT *Require a relevant and
representative sample of test items *Require specification of the
achievement domain to be measured *Use the same type of test items
*Use the same rules for item writing *Judged by the same qualities
(validity and reliability) *Useful in educational measurement
13
Slide 14
ADVANTAGES AND DISADVANTAGES OF NRT Advantages They easy for
instructors to use They work well in situations requiring rigid
differentiation among students They are generally appropriate in
large courses Disadvantages An individual's grade is determined not
only by his/her achievements, but also by the achievements of
others. no indication of prerequisite knowledge for more advanced
material has been mastered less appropriate for measuring affective
and psychomotor objectives encourages competition and comparison
scores 14
Slide 15
ADVANTAGES AND DISADVANTAGES OF CRT Advantages Students are not
competing with each other Students are thus more likely to actively
help each other learn. A student's grade is not influenced by the
caliber of the class. Disadvantages It is difficult to set a
reasonable standard for students Most experienced faculty set
criteria based on their knowledge of how students usually perform
Criterion-referenced systems often become fairly similar to norm-
referenced systems. absolute standards difficult to set in some
areas standards tend to be arbitrary not appropriate comparison
when others are valuable 15
Slide 16
FORMATIVE & SUMMATIVE TESTS 16
Slide 17
THE GARDEN ANALOGY If we think of our children as plants
Summative assessment of the plants is the process of simply
measuring them. It might be interesting to compare and analyze
measurements but, in themselves, these do not affect the growth of
the plants. Formative assessment, on the other hand, is the
equivalent of feeding and watering the plants appropriate to their
needs - directly affecting their growth. Formative and summative
assessment are interconnected. They seldom stand alone in
construction or effect. 17
Slide 18
FORMATIVE ASSESSMENT Assessment for learning Taken at varying
intervals throughout a course to provide information and feedback
that will help improve the quality of student learning the quality
of the course itself The purpose is: To promote further improvement
of student learning during the learning process To involve students
in the ongoing assessment of their own achievement Provides
information on what an individual student needs To practice To have
re-taught To learn next 18
Slide 19
KEY ELEMENTS OF FORMATIVE ASSESSMENT 1. The identification by
teachers & learners of learning goals, intentions or outcomes
and criteria for achieving these. 2. Rich conversations between
teachers & students that continually build and go deeper. 3.
The provision of effective, timely feedback to enable students to
advance their learning. 4. The active involvement of students in
their own learning. 5. Teachers responding to identified learning
needs and strengths by modifying their teaching approach(es). Black
& Wiliam, 1998 19
Slide 20
BENEFITS OF FORMATIVE ASSESSMENT FOR TEACHERS (Boston, 2002)
Teachers are able to determine what standards students already know
and to what degree. Teachers can decide what minor modifications or
major changes in instruction they need to makes so that all
students can succeed in upcoming instruction and on subsequent
assessments. Teachers can create appropriate lessons and activities
for groups of learners or individual students. Teaching can inform
students about their current progress in order to help them set
goals for improvement. 20
Slide 21
BENEFITS OF FORMATIVE ASSESSMENTS FOR STUDENTS Students are
more motivated to learn. Students take responsibility for their own
learning. Students become users of assessment. Students learn
valuable lifelong skills such as self- evaluation, self-assessment,
and goal setting. Student achievement can improve from 21-41
percentile points. (marzano 2003; stiggens et. al, 2006) 21
Slide 22
SUMMATIVE ASSESSMENT Assessment of learning Generally taken by
students at the end of a unit or semester to demonstrate the "sum"
of what they have or have not learned. Summative assessment methods
are the most traditional way of evaluating student work. "Good
summative assessments--tests and other graded evaluations--must be
demonstrably reliable, valid, and free of bias" (Angelo and Cross,
1993). 22
BALANCED CLASSROOM ASSESSMENT SYSTEM FORMATIVE ASSESSMENT
SUMMATIVE ASSESSMENT A process used by teachers and students during
instruction that provides feedback to adjust ongoing teaching and
learning to help students improve their achievement of intended
instructional outcomes. A tool used after instruction to measure
student achievement which provides evidence of student competence
or program effectiveness. students are evaluated upon completion of
the work and the focus is on the final product. 24
Slide 25
FORMATIVE SUMMATIVE Occurs During Instruction Not Graded
Process Descriptive Feedback Continuous Occurs at the end Graded
Product Evaluative Feedback Periodic Sort students in rank order
COMPARISON OF ASSESSMENTS 25
Slide 26
A Fine Sieve Formative assessment informs both teachers and
students about student understanding at a point when timely
adjustments can be made. These adjustments help to ensure students
achieve targeted standards-based learning goals within a set time
frame. A course sieve Summative assessments happen too far down the
learning path to provide information at the classroom level and to
make instructional adjustments and interventions during the
learning process 26 COMPARISON OF ASSESSMENTS
Slide 27
OBJECTIVES & SUBJECTIVES TESTS 27
Slide 28
OBJECTIVE AND SUBJECTIVE ASSESSMENT Objective assessment is a
form of questioning which has a single correct answer. Subjective
assessment is a form of questioning which may have more than one
current answer (or more than one way of expressing the correct
answer). 28
Slide 29
OBJECTIVE TEST Objective tests include multiple choice,
true-false, matching, and fill-in questions. They tend to focus
more on specific facts than on general ideas and concepts Questions
on a tests that only have one correct answer Objective tests
require far more careful preparation than subjective tests
Objective examination can be part of formative (diagnostic) and
summative (final assessment) exams. Most popular objective exam is
Multiple Choice Questions (MCQ). (the method of scoring is the only
factor that distinguishes an objective test from a subjective
test)
Slide 30
Advantages of multiple choice question: 1. The ability to
create a test item bank 2. Quick grading can be easily computer
scored 3. If written well, high reliability - only one possible
answer 4. Objective grading 5. Wide coverage of content 6. Can be
used for mass testing 7. Precision in providing information
regarding specific skills and abilities. 8. Students are familiar
with the item type directions are easy to understand.
Slide 31
Weaknesses of multiple choice question: 1. Difficult and time
consuming to construct 2. Low validity 3. Mainly tests recognition
knowledge and recall of facts. 4. Guessing may have considerable
effect 5. Cheating may be facilitated 6. Sometimes skills and areas
are tested because they are testable than important 7. Places a
high degree of dependence on students reading ability and teachers
writing ability. 8. It may limit beneficial washback. 9. This
technique strictly limits what can be tested.
Slide 32
SUBJECTIVE TEST Subjective tests include essay, short-answer,
vocabulary, and take-home tests Questions on a test that have more
than one correct answer. Each examiner uses his own judgment in
evaluating performance and awarding marks.
Slide 33
Strengths: 1. Easy to set 2. High validity 3. Can assess
affective and interpretive aspects of language skills 4. allow a
candidate to express originality of thought 5. allow the examiner
to assess the candidate's quality of written expression.
Weaknesses: 1. Marking is time consuming 2. Reliability is low 3.
Inter-rater as well as intra-rater variability are probable. 4.
Dependence on presentation.- good hand writing vs bad handwriting
5. Question evasion - possible for the candidate to avoid questions
in areas of the curriculum in which they are weak. SUBJECTIVE
TEST
Slide 34
OBJECTIVE VS. SUBJECTIVE TEST Objective short answer closed
response mostly recognition, limited production difficult to write
well quick and easy to grade reliable workload up front Subjective
long answer open response emphasis on production relatively easy to
write difficult to grade time-consuming inter-rater reliability not
as reliable workload post test 34
Slide 35
REFERENCES Classroom Assessment: Basic Concepts. Formative
vs.Summative Assessments. Retrieved October 20, 2008 from
http://fcit.usf.edu/assessment/basic/basica.html
http://fcit.usf.edu/assessment/basic/basica.html Formative vs.
Summative Evaluation. Retrieved October 20, 2008 from
http.jan.ucc.nau.edu/edtech/etc/667/proposal/evaluation/
summative_vs_formative.htm Formative and Summative Assessment.
Retrieved October 20, 2008 from
http://www.krauseinnovationcenter.org/ewyl/modules/module6-
3.html.http://www.krauseinnovationcenter.org/ewyl/modules/module6-
Classroom Assessment: Basic Concepts. Formative vs.Summative
Assessments. Retrieved October 24, 2008 from
http://fcit.usf.edu/assessment/basic/basica.html
http://fcit.usf.edu/assessment/basic/basica.html Pawlas, G., Oliva,
P. (2008) Supervision for Todays Schools, Sixth Edition. New York:
John Wiley and Sons 35
Slide 36
Arter, Judith, and Jay McTighe. Scoring Rubrics in the
Classroom. Thousand Oaks, CA: Corwin Press, INC., 2001. Marzano,
Robert J., Debra Pickering, and Jay McTighe. Assessing Student
Outcomes. Alexandria, VA: Association for Supervision and
Curriculum Development, 1993. Schoenbach, Ruth, et al. Reading for
Understanding, A Guide to Improving Reading in Middle and High
School Classrooms. San Francisco, CA: Jossey-Bass, Inc., 1999.
36
