MEASUREMENT IN RESEARCH報告者：許之馨授課老師：任維廉 教授

2

自我介紹許之馨台北人運管系大四興趣：看電影、吃美食

3

OUTLINE1. Measurement2. Validity

• Content validity• Criterion-related validity• Construct validity

3. Reliability• Standard error measurement• Internal consistency• Equivalence• Stability

4. Response set

4

MEASUREMENT All descriptive and experimental research

studies involve some kind of measurement.

Two related terms Test Evaluation

Formative evaluation Summative evaluation

5

VALIDITY The tools used in descriptive research

Test Questionnaires Interview guide

The extent to which a research tool measures what it intends to measure.

6

VALIDITY Three major type

Content validity Criterion-related validity Construct validity

7

CONTENT VALIDITY Item in research tool

Related to the subject matter tested / stated objectives of a course of study or program?

Reflecting the emphasis placed in a course or a program?

Representative of the universe of items?

Non-statistical

8

CONTENT VALIDITY Test blueprint

Two-way chart ( related specific objective to specific content areas )

http://www.utexas.edu/provost/sacs/pdf/Test%20Blueprint%20handout.pdf

9

CRITERION-RELATED VALIDITY Empirical (statistically)

Comparison the scores on the to-be-validated test and the scores on a criterion measure Correlation coefficient

10

CRITERION-RELATED VALIDITY Concurrent validity

The to-be-validated test and the criterion test were made at same time or after a short interval

Predicative validity A much longer time interval exists between the

two test situations or the two assessment situations

11

CONSTRUCT VALIDITY Construct

Psychological traits or characteristic Not directly observable but be inferred on the basis of

overt behavior

To the extent of measuring a theoretical construct or trait

Controversial

12

RELIABILITY The consistency of getting the same or similar

responses The accuracy of the score of one person

The person’s score obtained on a test is not the person’s true score

Standard error measurement(SEM) The consistency of score of a group of people

Internal consistency Equivalence Stability

13

STANDARD ERROR MEASUREMENT Test someone with all our tremendous

number of equivalent forms. Then we will find that scores are not the same. The distribution of scores familiar are resemble the familiar “ normal ” curve.

The average of scores is his true score.

14

STANDARD ERROR MEASUREMENT The standard deviation as a measure od the

variation of observed scores around the true score

SD : standard deviation of obtained scores of a group : the reliability coefficient computed for the same group

15

INTERNAL CONSISTENCY Reliability derived from the administration of a

single test (questionnaire, interview)

Methods of determining reliability correlation coefficient Split half

Divided into two equal halves. Correlation coefficient the of scores of two half-tests. The Spearman-Brown formula to get the reliability of the

entire test r : reliability of the full test

r1/2 : reliability of the two half tests

16

INTERNAL CONSISTENCY Kuder-Richardson

: estimate of reliability based on Kuder-Richardson n : number of items in the test

17

EQUIVALENCE Parallel test form are administration to a

group of people at the same time or with very little time lapse.

The correlation coefficient is computed between the scores of the two tests.

18

STABILITY The same test after substantial time lapse

One test version here or now and equivalent test version after substantial time interval

19

RESPONSE SET A consistent tendency to follow a certain

pattern in responding to items in a test (questionnaire, interview)

Interference with getting usable data Avoiding extreme response option Socially expected response Faking

20

RESPONSE SET Weaken validity

Halo Effect Generosity Error Error of Central Tendency

21

Thanks for your attention

Q&A