Upload
carlo-magno
View
482
Download
0
Embed Size (px)
Citation preview
Accountability in Developing Assessment Tools
Carlo Magno
MCL Consultant
Objectives
• 1. Check the alignment between the objectives set in the syllabus and the items contained in the assessment.
• 2. Distinguish in using paper and pencil or performance-based needed when creating assessment tasks.
• 3. Follow the rules in writing types of items and performance-based tasks.
Assessment Process
Objectives in the syllabus
Create Assessment
tasks
Content Validity Review
assessment tasks
Revision
Data Gathering Data Analysis
Teacher Teacher
Teacher
Teacher
Teacher
Supervisor
•Needs of students and society •Mission and vision of the school •National and International Standards •Existing programs
What to check? C
on
ten
t V
alid
ity Check for alignment between the course
objectives and the competency covered in the task
Check if the assessment is appropriate with the form assessment task made (paper and pen vs. performance-based)
Face
V
alid
ity
Minor: Rules in writing the tasks
Minor: Syntax (grammar)
Minor: Suitability for learners
Task!
• Objective: Prepare a financial report
• Assessment task?
• Objective: Create a promotional material for the Chocolate Hills
• Assessment Task?
• Objective: calculate the total assessable benefits of an employee
• Assessment task?
Task!
• Objective: Interpret information obtained from a radar
• Assessment task:
• Objective: Give the advantage of multithreading.
Task!
• Objective: Classify whether the building pictures belongs to traditional, industrial and post-industrial style.
• Assessment Task:?? • Objective: Illustrate Maxwell's Theorem of
Reciprocal Displacements. • Assessment task:?? • Objective: Graphs a circle and other geometric
figures on the coordinate plane • Assessment Task:
Task
• Objective: Compare accommodation theory and face negotiation theory.
• Assessment task?
• Objective: Classify soluble and insoluble compounds
• Assessment task?
V. Matching Items with Criterion Item
• Instruction: Indicate whether the matching of the objective and item is suitable.
• 1. Objective: Given a performance of an instrumental or vocal melody containing a melodic or rhythmic error, and given the score for the melody, be able to point out the error.
• Criterion item: The instructor will play the melody of the attached musical score on the piano and will make an error either in rhythm or melody. Raise your hand when the error occurs.
• Is the item Suitable? ___ Yes
____ No ____Can’t tell
V. Matching Items with Criterion Item
• 2. Objective: Given mathematical equations containing one unknown, be able to solve for the unknown.
• Criterion Item: Sam weighs 97 kilos. He weighs 3.5 kilos more than Barrey. How much does Barry weigh?
• Is the item Suitable? ___ Yes
____ No
____Can’t tell
V. Matching Items with Criterion Item
• 3. Objective: Be able to demonstrate familiarity with sexual anatomy and physiology
• Criterion Item: Draw and label a sketch of the male and female reproductive systems.
• Is the item Suitable? ___ Yes
____ No
____Can’t tell
V. Matching Items with Criterion Item
• 4. Objective: Given any one of the computers in our product line, in its original carton, be able to install and adjust the machine, preparing it for use. Criteria: The machine shows normal indication, and the area is free of debris and cartons
• Criterion item: Select one of the cartons containing one of
our model XX computers, and install it for the secretary in Room 45. Make sure it is ready for use and the area is left clean.
• Is the item Suitable? ___ Yes ____ No ____Can’t tell
V. Matching Items with Criterion Item
• 5. Objective: When given a set of paragraphs (that use words within your vocabulary), some of which are missing topic sentences, be able to identify the paragraph without topic sentences.
• Criterion Item: Turn to page 29 in your copy of Silas
Marner. Underline the topic sentence of each paragraph on that page.
• Is the item Suitable? ___ Yes ____ No ____Can’t tell
Observe the two sets of objectives
• Sequence the evolution of different OS
• Recognize specific conditions that will put people at risk in a building
• Select from the list what information is useful when performing a fitness assessment
• Convert moles to kilograms
• Create a program that will compute the grades of students using java.
• Construct a pole that can supporting a 30 sq flat wood on top.
• Demonstrate how to convince tourist to visit the old church.
• Sell a new kind of coffee among customers in a coffee shop.
Paper and pen vs. Performance-based tasks
• Format: binary, multiple choice, matching type, short answer
• Measure discreet skills (cognitive)
• Group administration
• Format: demonstrations, performances, actual work (authentic)
• Measure complex integrated skills (cognitive, psychomotor, affect)
• Individual or several groups
Constructive Alignment Objective: Interpret a table showing the population density of
the world (Understanding) Test Item: Given a bar graph, report the years where the
population increased and decreased, describe the pattern of growth each year and provide possible explanations for the growth pattern.
Objective: Draw a picture using the principle of balance. (Creating,)
Performance task: Draw a picture in the box showing the principle of balance.
Objective: Recognize fallacies in an argument (Understanding) Test item: Read the given article and list down five fallacious
statements found. Objective: Run in a 100-yard dash (psychomotor) Performance Assessment: Students need to complete a run in
100 yard dash.
Determine if the objectives will require paper and pen or performance-based task
• 1. State the domain and range of a given function • 2. Solve systems of linear equations in two variables by the
graphical method • 3. Draw the slope of the given equation • 4. Simplify complex fractions • 5. Deliver a speech to persuade people to donate for the
orphanage • 6. Multiply and divide rational algebraic expression • 7. List hazards of working with chemicals • 8. Classify the properties of substances as to physical or chemical
and extrinsic or intrinsic • 9. Cite evidence of chemical changes • 10. Present ways to prevent walls from easily cracking during
earthquakes
Paper and pencil assessment
• Binary type – remembering - understanding
• Matching type - remembering
• Multiple choice – remembering, understanding, applying, analyzing, evaluating
• Supply type - remembering
Binary Type
• 1. Avoid the use of “specific determiners”
• FAULTY: No picture-no sound in a television set may indicate a bad 5U4G.
• IMPROVED: A bad 5U4G tube in a television set will result in no picture sound.
Why is the item faulty?
Binary Type
• 2. Base true-false items upon statements that are absolutely true or false, without qualifications or exceptions.
• FAULTY: World War II was fought in Europe and the Far East.
• IMPROVED: The primary combat locations in terms of military personnel during World War II were Europe and the Far East.
Why is the item faulty?
Binary Type
• 3. Avoid negative stated items when possible and eliminate all double negatives.
• FAULTY: It is not frequently observed that copper turns green as a result of oxidation.
• IMPROVED: Copper will turn green upon oxidizing.
Why is the item faulty?
Binary Type
• 4. Use quantitative and precise rather than qualitative language where possible.
• FAULTY: Many people voted for Gloria Arroyo in the 2003 Presidential election.
• IMPROVED: Gloria Arroyo received more than 60 percent of the popular votes cast in the Presidential election of 2003.
Why is the item faulty?
Binary Type
• 5. Avoid stereotypic and textbook statements.
• FAULTY: From time to time efforts have been made to explode the notion that there may be a cause-and-effect relationship between arboreal life and primate anatomy.
• IMPROVED: There is a known relationship between primate anatomy and arboreal life.
Why is the item faulty?
Binary Type
• 6. Avoid making the true items consistently longer than the false items.
• 7. Avoid the use of unfamiliar or esoteric language. • FAULTY: According to some peripatetic politicos, the
raison d’etre for capital punishment is retribution. • IMPROVED: According to some politicians,
justification for the existence of capital punishment can be traced to the Biblical statement, “An eye for an eye.”
Why is the item faulty?
Binary Type
• 8. Avoid complex sentences with many dependent clauses.
• FAULTY: Jane Austen, an American novelist born in 1790, was a prolific writer and is best known for her novel Pride and Prejudice, which was published in 1820.
• IMPROVED: Jane Austen is best known for her novel Pride and prejudice.
Why is the item faulty?
Binary Type
• 9. It is suggested that the crucial elements of an item be placed at the end of the statement.
• FAULTY: Oxygen reduction occurs more readily because carbon monoxide combines with hemoglobin faster than oxygen does.
• IMPROVED: Carbon monoxide poisoning occurs because carbon monoxide dissolves delicate lung tissue.
Why is the item faulty?
Matching Type Match List A with List B. You will be given one point
for each correct match. List A List B a. cotton gin a. Eli Whitney b. reaper b. Alexander Graham Bell c. wheel c. David Brinkley d. TU54G tube d. Louisa May Alcott e. steamboat e. None of these
•Directions failed to specify the basis for matching •List are enumerated identically •Responses not listed logically •Lacks homogeneity •Equal number of elements •Use of “None of the above”
Why is the item faulty?
Matching Type
IMPROVED: Famous inventions are listed in the left-hand column and inventors in the right-hand column below. Place the letter corresponding to the inventor in the space next to the invention for which he s famous. Each match is worth 1 point, and “None of these” may be the correct answer. Inventors may be used more than once.
Inventions Inventors __ 1. steamboat a. Alexander Graham-Bell __ 2. cotton skin b. Robert Fulton __ 3. sewing machine c. Elias Howe __ 4. reaper d. Cyrus McCormick e. Eli Whitney
Multiple Choice
1. Salvador Dali is
a. a famous Indian.
b. important in international law.
c. known for his surrealistic art.
d. the author of many avant-garde plays.
•It is recommended that the stem be a direct question. •The stem should pose a clear, define, explicit, and singular problem.
Why is the item faulty?
Multiple Choice
IMPROVED: With which one of the fine arts is Salvador Dali associated?
a. surrealistic painting
b. avant-garde theatre
c. polytonal symphonic music
d. impressionistic poetry
Multiple Choice
2. Milk can be pasteurized at home by
a. heating it to a temperature of 130o
b. Heating it to a temperature of 145o
c. Heating it to a temperature of 160o
d. Heating it to a temperature of 175o
•Include in the stem any words that might otherwise be repeated in each response.
Why is the item faulty?
Multiple Choice
IMPROVED: The minimum temperature that can be used to pasteurize milk at home is:
a. 130oC
b. 145oC
c. 160oC
d. 175oC
Multiple Choice 3. Although the experimental research, particularly that by
Hansmocker must be considered equivocal and assumptions viewed as too restrictive, most testing experts would recommend as the easiest method of significantly improving paper-and-pencil achievement test reliability to
a. increase the size of the group being tested.
b. increase the differential weighting of items.
c. increase the objective of scoring.
d. increase the number of items.
e. increase the amount of testing time.
Items should be stated simply and understandably, excluding all nonfunctional words from stem and alternatives.
Why is the item faulty?
Multiple Choice
IMPROVED: Assume a 10-item, 10-minute paper-and-pencil multiple choice achievement test has a reliability of .40. The easiest way of increasing the reliability to .80 would be to increased
a. group size
b. scoring objectivity
c. differential item scoring weights
d. the number of items
e. testing time
Multiple Choice
4. None of the following cities is a state capital except
a. Bangor
b. Los Angeles
c. Denver
d. New Haven
•Avoid negatively stated items
Why is the item faulty?
Multiple Choice
IMPROVED: Which of the following cities is a state capital?
a. Bangor
b. Los Angeles
c. Denver
d. New Haven
Multiple Choice
5. Who wrote Harry Potter and the Goblet of Fire?
a. J. K. Rowling
b. Manny Paquiao
c. Lea Salonga
d. Mark Twain
•If possible the alternatives should be presented in some logical, numerical, or systematic order. •Response alternatives should be mutually exclusive.
Why is the item faulty?
Multiple Choice
IMPROVED: Who wrote Penrod?
a. J. K. Rowling
b. J. R. R. Tolkien
c. V. Hugo
d. L. Carrol
Multiple Choice
6. Which of the following statements makes clear the meaning of the word “electron”?
a. An electronic tool
b. Neutral particles
c. Negative particles
d. A voting machine
e. The nuclei of atoms
•Make all responses plausible and attractive to the less knowledgeable and skillful student.
Why is the item faulty?
Multiple Choice
IMPROVED: Which of the following phrases is a description of an “electron”?
a. Neutral particle
b. Negative particle
c. Neutralized proton
d. Radiated particle
e. Atom nucleus
Multiple Choice
7. What is the area of a right triangle whose sides adjacent to the right angle are 4 inches long respectively?
a. 7
b. 12
c. 25
d. None of the above
•The response alternative “None of the above” should be used with caution, if at all.
Why is the item faulty?
Multiple Choice
IMPROVED: What is the area of a right triangle whose sides adjacent to the right angle are 4 inches and 3 inches respectively?
a. 6 sq. inches
b. 7 sq. inches
c. 12 sq. inches
d. 25 sq. inches
e. None of the above
Multiple Choice 8. As compared with the American factory worker in
the early part of the 19th century, the American factory worker at the close of the century
a. was working long hours
b. received greater social security benefits
c. was to receive lower money wages
d. was less likely to belong to a labor union.
e. became less likely to have personal contact with employers
Make options grammatically parallel to each other and consistent with the stem.
Why is the item faulty?
Multiple Choice
IMPROVED: As compared with the American factory worker in the early part of the century, the American factory worker at the close of the century
a. worked longer hours.
b. had more social security.
c. received lower money wages.
d. was less likely to belong to a labor union
e. had less personal contact with his employer
Multiple Choice 9. The “standard error of estimate’
refer to
a. the objectivity of scoring.
b. the percentage of reduced error variance.
c. an absolute amount of possible error.
d. the amount of error in estimating criterion scores.
Avoid such irrelevant cues as “common elements” and “pat verbal associations.”
Why is the item faulty?
Multiple Choice
IMPROVED: The “standard error of estimate” is most directly related to which of the following test characteristic?
a. Objectivity
b. Reliability
c. Validity
d. Usability
e. Specificity
Multiple Choice 10. What name is given to the group of
complex organic compounds that occur in small quantities in natural foods that are essential to normal nutrition?
a. Calorie
b. Minerals
c. Nutrients
d. Vitamins
In testing for understanding of a term or concept, it is generally preferable to present the term in the stem and alternative definitions in the options.
Why is the item faulty?
Multiple Choice
IMPROVED: Which of the following statements is the best description of a vitamin?
Short Answer Type
1. Require short, definite, clear-cut, and explicit answers
FAULTY: Earnest Hemingway wrote______
IMPROVED: The Old Man and the Sea was written by _______.
Who wrote The Old man and the Sea?
Why is the item faulty?
Short Answer Type
2. Avoid multimutilated statements
FAULTY: _____ pointed out in ____ the freedom of thought in America was seriously hampered by ___, ____, & __.
IMPROVED: That freedom of thought in America was seriously hampered by social pressures toward conformity was pointed out in 1830 by ______.
Why is the item faulty?
Short Answer Type
• 3. If several equal answers equal credit should be given to each one.
• 4. Specify and announce in advance whether scoring will take spelling into account.
Short Answer Type
5. In testing for comprehension of terms and knowledge of definition, it is often better to supply the term and require a definition than to provide a definition and require the term.
FAULTY: What is the general measurement term describing the consistency with which items in a test measure the same thing?
IMPROVED: Define “internal consistency reliability.”
Why is the item faulty?
Short Answer Type
6. It is generally recommended that in completion items the blanks come at the end of the statement.
FAULTY: A (an) ________ is the index obtained by dividing a mental age score by chronological age and multiplying by 100.
IMPROVED: The index obtained by dividing a mental age score by chronological age and multiplying by 100 is called a (an) ________
Why is the item faulty?
Short Answer Type
7. Minimize the use of textbook expressions and stereotyped language.
FAULTY: The power to declare war is vested in ______
IMPROVED: Which national legislative body has the authority to declare war?
Why is the item faulty?
Short Answer Type
8. Specify the terms in which the response is to be given.
FAULTY: Where does the Security Council of the United Nations hold its meeting?
IMPROVED: In what city of the United States does the Security Council of the United Nations hold its meeting?
FAULTY: If a circle has 4-inch diameter, its area is_____ IMPROVED: A circle has 4-inch diameter. Its area in
square inches correct to two decimal places, is _____
Why is the item faulty?
Short Answer Type
9. In general, direct questions are preferable to incomplete declarative sentences.
FAULTY: Gold was discovered in California in the year ___
IMPROVED: In what year was gold discovered in California?
Why is the item faulty?
Short Answer Type
10. Avoid extraneous clues to the correct answer
FAULTY: A fraction whose denominator is greater than its numerator is a _____
IMPROVED: Fractions whose denominator are greater than their numerators are called _____
Why is the item faulty?
• Ability to Recognize the Relevance of Information
58
• Ability to Recognize Warranted and Unwarranted Generalizations
59
• Ability to Recognize Inferences
60
• Ability to Interpret Experimental Findings
61
• Ability to Apply Principles
62
• Ability to Recognize Assumptions
63
Reading comprehension • Bem (1975) has argued that androgynous people are
“better off” than their sex-typed counterparts because they are not constrained by rigid sex-role concepts and are freer to respond to a wider variety of situations. Seeking to test this hypothesis, Bem exposed masculine, feminine, and androgynous men and women to situations that called for independence (a masculine attribute) or nurturance (a feminine attribute). The test for masculine independence assessed the subject’s willingness to resist social pressure by refusing to agree with peers who gave bogus judgments when rating cartoons for funniness (for example, several peers might say that a very funny cartoon was hilarious). Nurturance or feminine expressiveness, was measured by observing the behavior of the subject when left alone for ten minutes with a 5-month old baby. The result confirmed Bem’s hypothesis. Both the masculine sex-typed and the androgynous subjects were more independent (less conforming) on the ‘independence” test than feminine sex-typed individuals. Furthermore, both the feminine and the androgynous subjects were more “nurturant” than the masculine sex-typed individuals when interacting with the baby. Thus, the androgynous subjects were quite flexible, they performed as masculine subjects did on the “feminine” task.
64
35. What is the independent variable in the study? a. Situations calling for independence and
nurturance b. Situation to make the sex type react c. Situations to make the androgynous be
flexible d. Situations like sex type, androgynous and
sex role concepts 36. What are the levels of the IV? a. masculine attribute and feminine attribute b. rating cartoons and taking care of a baby c. independence and nurturance d. flexibility and rigidity
Interpreting Diagrams Instruction. Study the following illustrations and answer the following
questions.
Figure 1
65
Pretest Posttest
101. Which group received the treatment? a. group A b. group B b. c. none of the above 102. Why did group B remain stable across the experiment? a. there is an EV b. there is no treatment c. there is the occurence of ceiling effect 103. What is the problem at the start of the experiment? a. the groups are nonequivalent b. the groups are competing with each other c. the treatment took place immediately
Workshop 1
• Work in groups by department and write one course objective and write one item to match the objective.
• Ensure that the items is aligned with the objective • Conditions: Skills should cover undertstanding,
applying, analyzing, evaluation, use multiple choice • Organize the presentation in a table: • Group presentation and feedback will follow.
Course Objective Item
Characteristics of performance-based assessment
• Students perform, create, construct, produce, or do something.
• Deep understanding and/or reasoning skills are needed and assessed.
• Involves sustained work, [often days and weeks].
• Calls on students to explain, justify, and defend.
• Performance is directly observable.
• Involves engaging in ideas of importance and substance.
• Relies on trained assessor’s judgments for scoring
• Multiple criteria and standards are prespecified and public
• There is no single correct answer.
• If authentic, the performance is grounded in real world contexts and constraints.
Learning Targets
• Skills – Communication and presentation skills – Ex: Speaking
1. Speaking clearly, expressively, and audibly a. Using voice expressively b. Speaking articulately and pronouncing words correctly c. Using appropriate vocal volume
2. Presenting ideas with appropriate introduction, development, and conclusion
1. Presenting ideas in an effective order 2. Providing a clear focus on the central idea 3. Providing signal words, internal summaries, and transitions
Learning Targets
3. Developing ideas using appropriate support materials
a) Being clear and using reasoning processes
b) Clarifying, illustrating, exemplifying, and documenting ideas
4. Using nonverbal cues
a. Using eye contact
b. Using appropriate facial expressions, gestures, and body movement
5. Selecting language to a special purpose
a. Using language and conventions appropriate for the audience
Learning Targets
• Psychomotor skills – Fine motor: cutting papers with scissors, drawing a
line tracing, penmanship, coloring drawing, connecting dots
– Gross motor: Walking, jumping, balancing, throwing, skipping, kicking
– Complex: Perform a swing golf, operate a computer, drive a car, operate a microscope
– Visual: Copying, finding letters, finding embedded figures, identifying shapes, discrimination
– Verbal and auditory: identify and discriminate sounds, imitate sounds, pronounce carefully, blend vowels
Learning Targets
• Products
– Write promotional materials
– Report on a foreign country
– Playing a new song
Variation of authenticity
Relatively authentic Somewhat authentic Authentic
Identify the materials used in dressing a wound
Give the steps in dressing a wound
Dresses the wound of a patient
Tell the use of a thermometer
Records temperature in a chart
Uses the thermometer on a patient and records it.
Explain the steps on taking blood pressure
Show how to take the blood pressure using a sphygmomanometer
Get the blood pressure of a patient.
Complexity of task
• Restricted-type task – Narrowly defined and require brief responses – Task is structured and specific – Ex:
• Construct a bar graph from data provided • Demonstrate a shorter conversation in French about what is on a
menu • Read an article from the newspaper and answer questions • Flip a coin ten times. Predict what the next ten flips of the coin will
be, and explain why. • Listen to the evening news on television and explain if you believe
the stories are biased. • Construct a circle, square, and triangle from provided materials
that have the same circumference.
Constructing Performance Based tasks
• Extended-type task – Complex and elaborate – Often include collaborative work with small group of
students. – Requires the use of a variety of information – Examples:
• Design a playhouse and estimate cost of materials and labor • Plan a trip to another country: Include the budget and itinerary,
and justify why you want to visit certain places • Conduct a historical reenactment (e. g. impeachment trial of
ERAP) • Diagnose and repair a car problem • Design an advertising campaign for a new or existing product
Identifying Performance Task Description
• Prepare a task description • Listing of specifications to ensure that essential if
criteria are met • Includes the ff.:
– Content and skill targets to be assessed – Description of student activities
• Group or individual • Help allowed
– Resources needed – Teacher role – Administrative process – Scoring procedures
Performance-based Task Question Prompt
• Task prompts and questions will be based on the task descriptions.
• Clearly identifies the outcomes, outlines what the students are encourage dot do, explains criteria for judgment.
Performance Criteria
• What you look for in student responses to evaluate their progress toward meeting the learning target.
• Dimensions of traits in performance that are used to illustrate understanding, reasoning, and proficiency.
• Start with identifying the most important dimensions of the performance
• What distinguishes an adequate to an inadequate demonstration of the target?
Example of Criteria
• Learning target: – Students will be able to write a persuasive paper to
encourage the reader to accept a specific course of action or point of view.
• Criteria: – Appropriateness of language for the audience – Plausibility and relevance of supporting arguments. – Level of detail presented – Evidence of creative, innovative thinking – Clarity of expression – Organization of ideas
Example of Performance-based task
• Final Product • Chemists are often required to determine the
concentration of unknown acidic or basic solutions. You will be provided with an unknown solution that is either acidic or basic. Your problem is to design an experiment, using the materials (and/or others) listed below, to determine the concentration of the unknown solution provided. Express your concentration of the unknown material in moles of H+ (aq) per liter or moles of OH- (aq) per liter of solution.
Task • a) Under the heading PROCEDURE list in order the steps of the procedure
you will use to solve the problem. You may include a diagram to help illustrate your plans for the experiment. Include any safety procedures you would follow.
• b) Construct a DATA TABLE or indicate any other method that you could use to record the observations and results that will be obtained.
• Materials
• 2 beakers, 100 ml 1 burette, 50 ml 4 Erlenmeyer flasks, 250 ml
• 1 filter funnel, small 1 graduated cylinder, 50 ml 1 burette stand
• safety goggles calculator 1 wash bottle of distilled watre
• 1 white tile hydrochloric acid, 0.1 mol/L sodium hydroxide, 0.1 mol/L
• phenolphthalein indicator bromothymol blue indicator unknown solution
Criteria Yes NO A. Procedures for investigation
1. Detailed
procedure/experimentally
feasible
Method includes details which will clearly
result in accurate determination of
concentration of unknown, including steps
below and repeated trials.
2. Sequence to plan Suggests method to determine molarity of
unknown solution
3. General strategy
First identifies unknown solution as an acid
or base using appropriate indicator, then
suggests titration using neutralizing solution.
4. Safety procedures Safety goggles should be worn; care in
handling acids and bases.
5. Use of equipment/diagram
Appropriate equipment and materials are
suggested, especially use of burette and
indicator.
Criteria Yes No B. Plan for recording and
organizing observations/data
1. Space for manipulation of data
or qualitative description
Could include space for averaging data;
stipulation of indicator used.
2. Matched to plan
Plan is comprehensive so that all
observations and data generated by plan
may be recorded; e.g., space for repeated
trials is included.
3. Organized sequentially Organized so that recording follows data
as generated.
4. Labelled fully (units included) All columns and rows are identified;
correct units of measurement are used.
5. Variables identified Solution used to titrate unknown is
indicated as acid or base.
Rating Scales
• Indicate the degree to which a particular dimension is present.
• Three kinds:
– Numerical
– Qualitative
– Combined Qualitative and Quantitative
Rating Scale
• Numerical Scale
– Numbers of a continuum to indicate different level of proficiency in terms of frequency or quality
Example: Complete Understanding 5 4 3 2 1 No understanding
Clear organization 5 4 3 2 1 No organization
Fluent reader 5 4 3 2 1 Emerging reader
Rating Scale
• Qualitative scale
– Uses verbal descriptions to indicate student performance.
– Provides a way to check the whether each dimension was evidenced.
• Type A: Indicate different gradations of the dimension
• Type B: Checklist
Rating Scale
• Example of Type A: – Minimal, partial, complete – Never, seldom, occasionally, frequently, always – Consistent, sporadically, rarely – None, some, complete – Novice, intermediate, advance, superior – Inadequate, needs improvement, good excellent – Excellent, proficient, needs improvement – Absent, developing, adequate, fully developed – Limited, partial, thorough – Emerging, developing, achieving – Not there yet, shows growth, proficient – Excellent, good, fair, poor
Rating Scale
• Example of Type A: Checklist
Rating Scale
• Holistic scale – The category of the scale contains several criteria, yielding
a single score that gives an overall impression or rating
Example level 4: Sophisticated understanding of text indicated with constructed meaning
level 3: Solid understanding of text indicated with some constructed meaning
level 2: Partial understanding of text indicated with tenuous constructed meaning
level 1: superficial understanding of text with little or no constructed meaning
Example holistic scale
Analytic scale
• Analytic Scale
– One in which each criterion receives a separate score.
Example
Criteria Outstanding 5 4
Competent 3
Marginal 2 1
Creative ideas
Logical organization
Relevance of detail
Variety in words and sentences
Vivid images
Rubrics
• When scoring criteria are combined with a rating scale, a complete scoring guideline is produced or rubric.
• A scoring guide that uses criteria to differentiate between levels of student proficiency.
Example of a rubric
• Rubrics should answer the following questions: – By what criteria should performance be judged?
– Where should we look and what should we look for to judge performance success?
– What does the range in the performance quality look like?
– How do we determine validity, reliability, and fairly what scores should be given and what that score means?
– How should the different levels of quality be described and distinguished from one another?
Rubric
Workshop
• Work in groups by department and create an performance task for one objective.
• Ensure that the performance task is aligned with the objective
• Organize the presentation in using the following: – Objective – Describe the final output – Describe what the students will do – List the criteria
• Group presentation and feedback will follow.