Upload
morris-bishop
View
217
Download
1
Embed Size (px)
Citation preview
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Data and the Nature Data and the Nature of Measurementof Measurement
Graziano and RaulinGraziano and RaulinResearch Methods: Chapter 4Research Methods: Chapter 4This multimedia product and its contents are protected under copyright law. The This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.lease, or lending of the program.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Research VariablesResearch Variables
VariableVariable: Any characteristic that can : Any characteristic that can take on more than one valuetake on more than one value– Examples: speed, level of hostility, Examples: speed, level of hostility,
accuracy of feedback, reaction timeaccuracy of feedback, reaction time Research is the study of the Research is the study of the
relationship among variablesrelationship among variables– Therefore, there must be at least two Therefore, there must be at least two
variables in a research study (or there is variables in a research study (or there is no relationship to study)no relationship to study)
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Measuring VariablesMeasuring Variables
MeasurementMeasurement: Assigning : Assigning numbers to indicate the level of a numbers to indicate the level of a variablevariable– Sometimes the number assignment Sometimes the number assignment
is easy to understand (e.g., time is easy to understand (e.g., time measured in seconds)measured in seconds)
– Sometimes it is more arbitrary (e.g., Sometimes it is more arbitrary (e.g., 1 for male and 2 for female)1 for male and 2 for female)
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Scales of MeasurementScales of Measurement
Based on how closely the Based on how closely the measurement scale matches the real measurement scale matches the real number systemnumber system
Scales of Measurement (Stevens, Scales of Measurement (Stevens, 1946, 1957)1946, 1957)– NominalNominal– OrdinalOrdinal– IntervalInterval– RatioRatio
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Nominal ScalesNominal Scales
Naming scaleNaming scale– Each number reflects a categoryEach number reflects a category– ExamplesExamples: diagnostic categories, : diagnostic categories,
political affiliationspolitical affiliations Produces nominal or categorical Produces nominal or categorical
datadata Mathematical propertiesMathematical properties
– IdentityIdentity
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Ordinal ScalesOrdinal Scales
Scale indicating rank orderScale indicating rank order– Reflects the order, but not the Reflects the order, but not the
amount amount ExampleExample: order of finish in : order of finish in a race, class rankingsa race, class rankings
Produces ordered dataProduces ordered data Mathematical propertiesMathematical properties
– Identity Identity – MagnitudeMagnitude
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Interval ScalesInterval Scales
Scale with equal intervalsScale with equal intervals– The scale indicates amount, but with no The scale indicates amount, but with no
zero pointzero point– ExamplesExamples: temperature on the Celsius : temperature on the Celsius
scale, most psychological testsscale, most psychological tests Produces score dataProduces score data Mathematical propertiesMathematical properties
– IdentityIdentity– MagnitudeMagnitude– Equal intervalsEqual intervals
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Ratio ScalesRatio Scales
Scale that fits the number system wellScale that fits the number system well– Includes equal intervals and a true zero Includes equal intervals and a true zero – ExamplesExamples: time, distance, frequency : time, distance, frequency
Produces score dataProduces score data Mathematical properties Mathematical properties
– IdentityIdentity– MagnitudeMagnitude– Equal intervalsEqual intervals– True zeroTrue zero
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Psychological TestsPsychological Tests
Most psychological test fall Most psychological test fall somewhere between an ordinal and somewhere between an ordinal and a ratio scalea ratio scale– Ordinal in that the distance between Ordinal in that the distance between
scores may not be equalscores may not be equal– Ratio in that one can view the test Ratio in that one can view the test
score as the number of correct itemsscore as the number of correct items Norm is to assume such tests Norm is to assume such tests
represent an interval scalerepresent an interval scale
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Measurement ErrorMeasurement Error
Decreases the accuracy of Decreases the accuracy of measurementmeasurement
Possible sources of measurement Possible sources of measurement errorerror– Response set biasesResponse set biases– Inconsistent measurement proceduresInconsistent measurement procedures– Sloppy proceduresSloppy procedures– Unreliable measuresUnreliable measures
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Operational DefinitionsOperational Definitions
Specific procedures for measuring Specific procedures for measuring and/or manipulating a variableand/or manipulating a variable
Every variable should be Every variable should be operationally definedoperationally defined
The more careful and complete The more careful and complete the operational definition, the the operational definition, the more precise the measurement of more precise the measurement of the variablethe variable
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
ReliabilityReliability
The consistency of measurementThe consistency of measurement Consistency can be conceptualized in Consistency can be conceptualized in
different waysdifferent ways– Therefore, there are different types of Therefore, there are different types of
reliabilityreliability Usually measured with a correlationUsually measured with a correlation
– Covered in Chapter 5Covered in Chapter 5– Sensitive to the consistency of rank Sensitive to the consistency of rank
orderings of participantsorderings of participants
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Types of ReliabilityTypes of Reliability
Interrater reliabilityInterrater reliability: degree of : degree of agreement between two independent agreement between two independent ratersraters
Test-retest reliabilityTest-retest reliability: degree of : degree of consistency over timeconsistency over time
Internal consistency reliabilityInternal consistency reliability: : degree to which the items of a degree to which the items of a measure all measure the same thingmeasure all measure the same thing
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Perfect ReliabilityPerfect Reliability
Reliability is a Reliability is a measure of measure of consistency.consistency.
Perfect reliability Perfect reliability means that the means that the scores are scores are perfectly perfectly consistent.consistent.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Good ReliabilityGood Reliability
Reliability Reliability deteriorates deteriorates when the when the consistency is consistency is lost.lost.
Here the rank Here the rank orderings are still orderings are still reasonably reasonably consistent.consistent.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Fair ReliabilityFair Reliability
As reliability As reliability deteriorates deteriorates further, the rank further, the rank orderings from orderings from the two testings the two testings are less similar.are less similar.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Poor ReliabilityPoor Reliability
When you reach When you reach the point where the point where the rank the rank orderings have orderings have no relationship to no relationship to one another, one another, your reliability your reliability (i.e., consistency) (i.e., consistency) is poor.is poor.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Effective RangeEffective Range
The range in which a measure gives an The range in which a measure gives an accurate indication of the level of the accurate indication of the level of the variablevariable
Critical to select measures with an Critical to select measures with an effective range appropriate foreffective range appropriate for– Your sample Your sample – Your studyYour study– ExampleExample: Using a calculus test to measure : Using a calculus test to measure
math skills in second graders would NOT math skills in second graders would NOT workwork
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Scale Attenuation Scale Attenuation EffectsEffects Results from a restriction of the Results from a restriction of the
range of the measurerange of the measure– Therefore, people above or below the Therefore, people above or below the
effective range are not measured effective range are not measured accuratelyaccurately
Two types of scale attenuation Two types of scale attenuation effectseffects– Floor effectsFloor effects– Ceiling effectsCeiling effects
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Illustrating These Illustrating These EffectsEffects SituationSituation
– Assume that we Assume that we are measuring are measuring the weight of 9 the weight of 9 men using a men using a standard standard bathroom scalebathroom scale
Floor EffectFloor Effect– The needle is The needle is
stuck so that it stuck so that it never reads never reads below 175below 175
Ceiling EffectCeiling Effect– The needle is The needle is
stuck so that it stuck so that it never reads never reads above 200above 200
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Weights for 9 MenWeights for 9 Men
0
50
100
150
200
250
300
1 2 3 4 5 6 7 8 9
Individuals
Weig
ht
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Floor EffectFloor Effect
Range on the bottom of the scale is Range on the bottom of the scale is insufficient to measure people who insufficient to measure people who score near the bottomscore near the bottom
If the scale needle was stuck so that it If the scale needle was stuck so that it never read below 175 poundsnever read below 175 pounds– There would be a floor at 175There would be a floor at 175
Error due to this floor effect is shown Error due to this floor effect is shown in dark blue in the next slidein dark blue in the next slide– Weights recorded for these people are too Weights recorded for these people are too
highhigh
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Floor EffectFloor Effect
0
50
100
150
200
250
300
1 2 3 4 5 6 7 8 9
Individuals
Weig
ht
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Ceiling EffectCeiling Effect
The range on the top of the scale is The range on the top of the scale is insufficient to measure people who insufficient to measure people who score near the topscore near the top
If the scale could not read above 200 If the scale could not read above 200 poundspounds– There would be a ceiling at 200 (everyone There would be a ceiling at 200 (everyone
above 200 would be reported as weighing above 200 would be reported as weighing 200)200)
Error due to this ceiling effect is shown Error due to this ceiling effect is shown in dark blue in the next slidein dark blue in the next slide
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Ceiling EffectCeiling Effect
0
50
100
150
200
250
300
1 2 3 4 5 6 7 8 9
Individuals
Weig
ht
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Source of These Source of These EffectsEffects Result of psychological scale with Result of psychological scale with
an insufficient range to measure an insufficient range to measure the range of performance in the the range of performance in the samplesample– Example: a measure of memory that Example: a measure of memory that
is either too easy or too hardis either too easy or too hard
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
ValidityValidity
A scale is valid if it measures what it is A scale is valid if it measures what it is supposed to measuresupposed to measure
Validity also refers to how well a scale Validity also refers to how well a scale predicts other variablespredicts other variables– Example: An IQ test is likely to be a valid Example: An IQ test is likely to be a valid
predictor of grades in school.predictor of grades in school.– When used this way, the scale is called the When used this way, the scale is called the
predictor measurepredictor measure and the measure and the measure predicted is called the predicted is called the criterioncriterion
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Measuring ValidityMeasuring Validity
Validity is usually measured with a Validity is usually measured with a correlationcorrelation– The correlation is between The correlation is between
The measure andThe measure and A SPECIFIED criterion measureA SPECIFIED criterion measure
– Always list the criterion when reporting the Always list the criterion when reporting the level of validity level of validity
For example, the IQ test is a valid predictor of For example, the IQ test is a valid predictor of school grades, but not a valid predictor of school grades, but not a valid predictor of athletic ability.athletic ability.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Perfect ValidityPerfect Validity
Validity is the Validity is the degree to which degree to which one measure one measure predicts another.predicts another.
With perfect With perfect validity, the rank validity, the rank orderings on the orderings on the predictor and predictor and criterion criterion measures are measures are identical.identical.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Good ValidityGood Validity
If the rank If the rank orderings from orderings from the predictor and the predictor and criterion criterion measures are measures are similar, you have similar, you have good validity.good validity.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Fair ValidityFair Validity
But the less the But the less the rank orderings rank orderings from the from the predictor and predictor and criterion criterion measures agree, measures agree, the lower the the lower the validity.validity.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Poor ValidityPoor Validity
When you reach When you reach the point that the the point that the rank orderings of rank orderings of predictor and predictor and criterion criterion measures are measures are unrelated, you unrelated, you have essentially have essentially zero validity.zero validity.
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
Objective Objective MeasurementMeasurement The hallmark of scienceThe hallmark of science
– Objective measures produce the same Objective measures produce the same result no matter who does the result no matter who does the measuringmeasuring
– Therefore, scientific principles will apply Therefore, scientific principles will apply no matter who tests these principlesno matter who tests these principles
Objective measures reduce biases Objective measures reduce biases that could distort results that could distort results (see Chapters 8 (see Chapters 8 and 9)and 9)
Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)
SummarySummary
Measuring variables is central to Measuring variables is central to researchresearch
Several scales of measurement existSeveral scales of measurement exist Reduce measurement error with Reduce measurement error with
carefully developed operational carefully developed operational definitionsdefinitions
Enhance reliability and validity of your Enhance reliability and validity of your measuresmeasures
The goal is to produce objective, The goal is to produce objective, accurate measures of your variablesaccurate measures of your variables