Chapter 7

Types of Educational Measures

Mary ErsinghausPan Zhang

New Mexico State University

2

Topics - Chapter 7

•Classifying educational measures•4 types of educational measures usedin quantitative research•Locating and evaluating educational measures•Criteria for evaluating instruments

3

Create a Cognitive Map

Based on those topics, let's summarize our understanding of the chapter...

4

Classification of Measures1. Cognitive or non-cognitive

– Cognitive - focus on what a person knows– Non-cognitive measures focus on affective traits

2. Commercially prepared or locally developed– Commercially - focus on technical merit– Locally prepared - for specific situations

3. Self-report or observations by others– Self-report - subjects to supply the response – Observations - subjects to be observed by others

5

Types of Quantitative Measures

Four Types

• Tests

• Questionnaires• Observations• Interviews

6

Tests

Standard questions of cognitive knowledge or skillsScores

Norm-referenced Criterion-referenced

Standardization Uniform procedures for administering and scoring Types – achievement, aptitude, standards-based

Standard scores

7

Discussion Question

What is the difference between standardized achievement tests and standardized aptitude tests?

8

QuestionnairesDefinition, concept, and usage..Series of statements or questions used to gather information,

stimulate responses, and measure traits, reactions, knowledge, and experience of participants..

● Types● Scales● Checklist● Ranked Order● Problems

9

Questionnaire TypesPersonality Assessment1. As used by psychologists, counselors, and clinicians● Holistic and focused on total individual; non cognitive

and affective traits, and cognitive characteristics2. As used by teachers and researchers● Measure important individual traits as related to

learning and motivation● Designed in a way educators without clinical training

can utilize● Major focuses are self-concept and self esteem

10

Questionnaire TypesAttitude, value, and interest questionnaires1. Attitudes: predispositions, likes and dislikes, predict

behavior2. Interests: similar to attitude measure of favorable or

unfavorable response to activities3. Values: multiple types necessary to identify kinds of

values being examined in surveys, e.g., social values, personal values, and educational values, etc

*Importance of preference: influence motivation, goals, and achievement

11

Scales...Series of gradations used to measure responses...

Types of scales: Likert scale, semantic differentialLikert scale:

● true Likert scale: statement indicating a value or positive or negative direction towards something

● Likert-type rating: beginning with neutral statement and direction or gradation provided as response opinions

Semantic differential: A 7-point scale with adjective pairs as end points

11

12

ChecklistsChecklists: provides number of options from which to

choose; requires one or more choices of several alternatives

e.g. Check as many as apply. The topics in research that I find very useful are:

_____measurement_____qualitative design_____quantitative design_____reviewing literature

13

Ranked OrderRanked order: respondent places a limited number of

categories into sequential order, basing on importance, liking, degree of experience, etc...

14

ProblemsProblems in measuring non cognitive traits● non cognitive instruments, such as questionnaires,

with lower reliability and less evidence for validity..● sources of error in non cognitive measures1. respond set--tendency of participants to respond in

the same way, regardless of the content of the items2. faking--participants giving deliberately inaccurate

indications of their attitudes, personality, or interests

15

Discussion Question What types of questionnaires are mostly

commonly used in educational research, briefly describe them and give some examples?

15

16

Observations

● Limitations of self-report in tests and questionnaires

● Advantages of observations--firsthand data and description of natural behavior

● Types of observations

● Observer affects

17

Observation TypesInference1. High-inference--make inferences about behavior2. low-inference--no interpretation of behavior

Laboratory Observation-controlled environment and structured procedure

experiment; possibility of biased responses

Structured field observation1.define behaviors to be observed2.identify system of coding and recording

3.measure frequency and/or duration

17

18

Observer Effect

Observer bias:

caused by background, expectations, etc of the observer

Contamination:caused by previous knowledge of the study of the

observer

Halo effect:influenced by the initial impression about a person or a

group by the observer18

19

Interviews

• Orally questioning participants and recording their responses

• Advantages Greater depth and richness of information Used for all ages Clarify responses through additional questioning Allow for flexibility Reduce “no response” and/or “neutral” responses

20

Interviews

• Disadvantages– Time consuming

– Expensive

– Small samples

– Subjective

21

Types of Interview Questions

• Structured– Subject chooses from responses provided

• Semi-structured– Open-ended, specific in intent

• Unstructured– Open-ended, can be highly subjective

• Leading– Bias results

22

Interviewer Effects

• Several potential sources of error:– Bias, subjectivity– Match interviewer to participants– Response recording methods

23

Criteria for Evaluating Instrumentation

1. Evidence for validity should be stated clearly

2. Evidence for reliability should be stated clearly

3. The instruments should be clearly described

4. The procedures for administering the instrument should be clearly described

5. Norms should be specified for norm-referenced interpretations

6. Procedures for setting standards should be indicated for criterion-referenced interpretation

23

24

Criteria for Evaluating Instrumentation

7. The scores used in reporting results should be meaningful

8. Measures of non cognitive traits should avoid problems of response set and faking

9. Observers and interviewers should be trained

10. In high-inference observations the qualifications of the observers to make sound professional judgments should be indicated

11. The effect of the interviewer or observer should be minimal

24