Upload
elyssa-gladman
View
228
Download
0
Tags:
Embed Size (px)
Citation preview
Scales and Measurement
Chapters 11, 12
Constructs and Measurement
• Construct Development• Identifying and defining what is to be measured• A construct is a hypothetical variable composed of
different elements that are thought to be related (e.g., 5 questions tapping brand loyalty)
• Measurement• Figuring out how to measure what you want to measure• Measure needs to be reliable and valid
Different Types of Reliability• Internal Reliability
• Extent to which items on a scale “hang together” or are correlated with one another
• Cronbach’s alpha (covered in last class)• Split-half reliability (split measure into halves, correlate)
• Test-Retest Reliability• Extent to which scores are stable over time• Have people complete questionnaire twice and correlate scores
Validity: Overview of Key Definitions• Validity (in general)
• The extent to which conclusions drawn from a study are true
• Internal Validity• When a researcher can clearly identify cause and effect relationships (i.e.,
there are no confounds)
• External Validity• The extent to which what you find in your study can be generalized to your
target population
• Construct Validity• Extent to which your constructs of interest (e.g., sensation seeking) are
accurately and completely identified (measured)• In other words, the extent to which you are actually measuring what you say
you are measuring (your sensation seeking scale really does measure the true construct of sensation seeking)
Other Forms of Validity• Content Validity (Face Validity)
• Extent to which a measure is appropriate according to experts in the domain of interest
• Concurrent Validity (Convergent Validity)• Extent to which one measure of a construct overlaps with other similar
measures of that construct• Discriminant Validity
• Extent to which a measure of one construct does not overlap with measures of different constructs
• Predictive Validity• Extent to which a measure of a construct can predict theoretically-
relevant outcomes• Nomological Validity
• How a construct fits within a broader set of related constructs
Key Idea
• To develop reliable and valid measures which we can subsequently (and appropriately) use in statistical analyses, we must understand:
• Properties of scales
• How to design good questions that do not lead to biased or inconsistent responses
Measurement/Scaling Properties
• Assignment• You can assign objects to categories
• Order (Magnitude)• You can order objects in terms of having more or less of some quality
• Distance (Equal Intervals)• The distance between adjacent points on the scale is identical
• Origin (Absolute Zero Point)• Zero “means something” (absence of a given quality)
Types of Scales• Nominal Scale
• Has Assignment Only (Political Party)
• Ordinal• Has Assignment, Order (Rank Order of Finish in a Race)
• Interval• Has Assignment, Order, Equal Intervals (Temperature)
• Hybrid Ordinally-Interval Scale• Like an ordinal scale, but researcher “pretends” it is an interval scale (e.g.,
assumes 1 to 7 scale is an interval scale); commonly used in questionnaires
• Ratio• Has Assignment, Order, Equal Intervals, Absolute Zero (Number of Cars)
What Type of Scale?• Number of Sweaters Purchased This Year? _______
• What is Your Ethnicity?
• To what extent do you agree or disagree that Congress should have approved the $700 bailout? (1 = strongly disagree to 7 = strongly agree)
• Please rank the following issues from most to least important (Iraq, Health Care, Economy, Environment)
• What is your income? (5-10k; 11-15k; 16-20k; 20-25k; 25-30k)
On the Importance of Attitudes
I think McCain has a good grasp
of national security issues I feel a strong
connection with Obama
I am inclined to
vote for McCain
I’m planning on voting for Obama
I think Obama’s Plan
for Health Care Reform
is a good one
I believe both candidatesbring strengths to the table
I feel McCain would be
a good leader
Three Components of Attitudes• Cognitive Component
• How a person thinks about an attitude object (product, issue, candidate, idea)
• Affective Component• How a person feels about an attitude object
• Behavioral– A person’s behavioral predisposition to respond to an attitude
object in a certain way
Three Components of Attitudes?
I think McCain has a good grasp
of national security issues I feel a strong
connection with Obama
I am inclined to
vote for McCain
I’m planning on voting for Obama
I think Obama’s Plan
for Health Care Reform
is a good one
I believe both candidatesbring strengths to the table
I feel McCain would be
a good leader
Measuring Attitudes• While attitudes not perfect predictors of behavior, still very important
• We need to understand how to measure attitudes accurately
• Today, we’ll look at some creative ways to measure attitudes
• And, some standard approaches taken in marketing research
• As we do this, think about the types of questions we might want to ask in
our restaurant concepts study
Creative Measures of Attitudes
• Projective techniques (partially structured)
• Physiological (GSR; Heart Rate; Blood Pressure)
• Functional magnetic resonance imaging (fMRI)
• Flushing toilets
• Bogus pipeline
• Lost letters and emails*
• Implicit association test (IAT)*
You’ve Got Mail!
Dear Peter Price,
Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies.
Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application.
We are happy to inform you that you have been selected to receive a Glassner Scholarship.
We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.
You’ve Got Mail!
Dear Mohammed Hameed,
Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies.
Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application.
We are happy to inform you that you have been selected to receive a Glassner Scholarship.
We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.
You’ve Got Mail!
Dear Mohammed Hameed,
Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition four four years at a state-funded university. There is also an additional $500 per year for academic supplies.
Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application.
We regret to inform you that you have not been selected to receive a Glasser Scholarship.
We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.
You’ve Got Mail!(Method)
• Bushman & Bonacci (2004, JESP)
• Pretested on Arab-American Prejudice
• 2 weeks later, receive email intended for a different person
• 2 IVs
• Intended recipient had European-American (Peter Price) vs. Arab-American name (Mohammed Hameed)
• Intended recipient won or didn’t win a scholarship (4 years support)
• DV = willingness to return the email to the sender to indicate it was incorrectly delivered (must be done in 2 days)
You’ve Got Mail!(Results)
00.10.20.30.40.50.60.70.8
Low Med High Low Med High
Arab-American Prejudice
Od
ds
of
Re
turn
ing
Em
ail
European Name Arab Name
Good News! Bad News
You’ve Got Mail!(Results)
00.10.20.30.40.50.60.70.8
Low Med High Low Med High
Arab-American Prejudice
Od
ds
of
Re
turn
ing
Em
ail
European Name Arab Name
Good News! Bad News
You’ve Got Mail!(Results)
00.10.20.30.40.50.60.70.8
Low Med High Low Med High
Arab-American Prejudice
Od
ds
of
Re
turn
ing
Em
ail
European Name Arab Name
Good News! Bad News
You’ve Got Mail!(Results)
GOOD NEWS! YOU WON!!
• P (returning email) went down as prejudice went up, especially when the intended recipient had an Arab-American name
BAD NEWS…YOU DIDN’T WIN.
• European-American: P (returning email) went down as prejudice went up
• Arab-American: P (returning email) went up as prejudice went up.
• People with Arab-American prejudice were actually more likely to return the bad news than the good news email when recipient was Arab American
WHAT MIGHT EXPLAIN SUCH ARAB-AMERICAN PREJUDICE?
Terror Management TheoryTerror Management Theory
Basic AssumptionBasic Assumption Humans have developed “anxiety buffering mechanisms” against
fear of death (cultural worldview, self-esteem, relationships)
Typical MethodTypical Method Write about own death (mortality salience) or dental pain Filler task Judge others from ingroup vs. outgroup
Typical ResultTypical Result We judge worldview threatening others more harshly under
conditions of mortality salience Support for Bush (vs. Kerry) goes up after MS prime
Implicit Association Task (IAT)• An implicit attitude is an attitude which people are not conscious of (or
would not want to admit to) but which can be assessed via the associations people hold in their minds between evaluative words (good, bad) and attitude objects (Caucasian, Arab)
• The IAT is a reaction time (RT) task in which subjects categorize words (positive, negative) and attitude objects (pictures of Caucasians, Arabs) on the left or right side of screen
• To simplify, some trials involve categorizations that are consistent with an implicit attitude, others are inconsistent with the attitude
• If there is an implicitly negative attitude toward Arabs, then reaction times should be slower to the inconsistent trials
The following set of trials is consistent
with an implicitly negativeattitude toward Arabs
Reaction times should berelatively fast if there
is an implicitly negative attitudetoward Arabs
Good Word orCaucasian
Bad Word orArab
If you see a good word or a Caucasian, hit “e” (left)
If you see a bad word or an Arab, hit “i” (right)
Good Word orCaucasian
Bad Word orArab
Dirt
Good Word orCaucasian
Bad Word orArab
Good Word orCaucasian
Bad Word orArab
Happy
Now Change CategorizationNow Change Categorization
(The following trials areinconsistent with implicit negative
attitude toward Arabs)
Reaction times should be slower on these trials if there is an implicitly
negative attitude toward Arabs
Bad Word orCaucasian
Good Word orArab
Categorization Switched
Bad Word orCaucasian
Good Word orArab
Bad Word orCaucasian
Good Word orArab
Joy
Bad Word orCaucasian
Good Word orArab
Death
What are we saying?!
• Sensitive attitudes (e.g., racial prejudice) are sometimes hard to assess with a self-report scale (due to social desirability concerns)
• The IAT measures associations (automatic attitudes)
• Implicitly negative attitudes are not uncommon, but…
• People can also override these automatic responses with controlled processes (so behavior is not discriminatory)
• For more information, visit https://implicit.harvard.edu/implicit/ where you can take an IAT for the 2008 election (or other issues)
Verbal Rating Scales
How satisfied were you with today’s meal?
1 2 3 4 5 6 7
Extremely Dissatisfied Somewhat Neither Somewhat Satisfied ExtremelyDissatisfied Dissatisfied Satisfied Satisfied
Issues in Designing Verbal Rating Scales
• Many measures taken by marketing researchers are verbal ratings
• What do we need to consider when we develop verbal rating scales?– Number of categories– Forced vs. unforced scale– Balanced or unbalanced scale– Extent of verbal description– Should response categories be numbered or not– Comparative vs. noncomparative scale– Scale direction
Number of Response Categories?• To what extent are you satisfied with your current MP3 player?
• Most researchers suggest between 5 and 7 categories; for example:
1 2 3 4 5 6 7
Extremely Dissatisfied Somewhat Neither Somewhat Satisfied ExtremelyDissatisfied Dissatisfied Satisfied Satisfied
• Too few does not give you enough information
• Too many and it will be hard for people to discriminate between the options (e.g., a 100-point scale)
Forced vs. Unforced Scale?• How likely would you be to buy a car manufactured in Brazil?
• Forced Scale (even number of options forces the respondent to lean one way or the other):
1 2 3 4 5 6
Very Unlikely Somewhat Somewhat Likely VeryUnlikely Unlikely Likely Likely
• Unforced scale gives people a neutral option:
1 2 3 4 5 6 7Very Unlikely Somewhat Neither Somewhat Likely Very
Unlikely Unlikely Likely Likely
Balanced vs. Unbalanced Scale?• How satisfied are you with your current hair stylist?
• Balanced scale (same number of positive and negative options):
1 2 3 4 5 6 7
Extremely Dissatisfied Somewhat Neither Somewhat Satisfied ExtremelyDissatisfied Dissatisfied Satisfied Satisfied
• Unbalanced scale (here all options are positive):
1 2 3 4 5 6 7Somewhat Very
Satisfied Satisfied
• Unbalanced scale can give biased results; unless distribution is naturally skewed to one side of the scale, should use balanced scale
Extent of Verbal Description?• The U.S. should invest in wind powered energy
• Label endpoints or label all options?
1 2 3 4 5 6 7Strongly Strongly
Disagree Agree
1 2 3 4 5 6 7Strongly Moderately Slightly Neither Agree Slightly Moderately Strongly
Disagree Disagree Disagree or Disagree Agree Agree Agree
• Labeling all options can aid in interpretation.
Should Categories be Numbered?• Toyota is an Environmentally Friendly Company
Strongly Moderately Slightly Neither Agree Slightly Moderately StronglyDisagree Disagree Disagree or Disagree Agree Agree Agree
1 2 3 4 5 6 7-3 -2 -1 0 1 2 3
• Numbers can help respondents understand scale
• 1 to 7 scale quite common
• But -3 to +3 can help interpretation of scale (disagree is negative, agree is positive); it may, however, overemphasize negativity
• Judgment call; pretesting both scales could help identify problems
Should we havenumbers here?
Comparative vs. Noncomparative?• Noncomparative question
• How would you evaluate Mintifresh toothpaste?
• Comparative question• Compared to your current brand, how would you evaluate
Mintifresh toothpaste?
• Comparative questions establish the referent and can be useful if you need to know how your product compares to a specific competitor or the customer’s current brand
• Noncomparative have the advantage of allowing the respondent to create their own referent, which can potentially improve accuracy
Direction of Scale?• Typical direction (lower values, negative connotation on left):
Strongly Moderately Slightly Neither Agree Slightly Moderately StronglyDisagree Disagree Disagree or Disagree Agree Agree Agree
1 2 3 4 5 6 7
• Some scales are not valenced, so must be careful about positioning. Here we see a semantic differential scale, with amusing positioning:
Unpleasant -2 -1 0 1 2 Pleasant
Flimsy -2 -1 0 1 2 SturdyMale -2 -1 0 1 2 Female
• Hmmm…this arrangement suggests that males are to be evaluated negatively; must be careful in designing scales so as not to bias results
Other Types of Scales
Rank order the restaurants above where 1 = most preferred, 3 = least preferred
Rank-Order andPaired Comparison Techniques
• Rank-order technique• Advantages: easy to understand, typically what we do in real-
life (vs. ratings)• Disadvantage: it may be that a person dislikes all of the
options, so ranking not that informative• Paired comparisons
• Take n products, compare each one to every other (in each pair, pick the one you prefer)
• Gives you direct comparisons, but # paired comparisons can be very large as number of products to be compared increases
pairings 62
)14(4
2
)1(pairings ofNumber products, 4With
nn
Semantic Differential Scale• Please provide your impression of Politician X:
Dumb _____ _____ _____ _____ __X__ Smart
Cold _____ _____ _____ __X__ _____ Warm Boring _____ _____ _____ _____ __X__ Funny Mean _____ _____ _____ __X__ _____ Nice
• Please provide your impression of Politician Y:Dumb _____ __X__ _____ _____ _____ Smart
Cold __X__ _____ _____ _____ _____ Warm Boring _____ _____ __X__ _____ ____ Funny Mean _____ __X__ _____ _____ _____ Nice
• Profile Analysis of Politician X vs. Y:Dumb _____ _____ _____ _____ _____ Smart
Cold _____ _____ _____ _____ _____ Warm Boring _____ _____ _____ _____ _____ Funny Mean _____ _____ _____ _____ _____ Nice
X
Y
Stapel Scale• Please provide your impression of Shoe Store X. (Use negative numbers
if you feel it is inaccurate; positive numbers if you feel it is accurate):
-5 -4 -3 -2 -1 Fast Service 1 23 4 5
-5 -4 -3 -2 -1 Friendly 12 3 4 5-5 -4 -3 -2 -1 Knowledgeable 12 3 4 5
• Can draw comparative profile analysis (e.g., of various shoe stores) as we did with semantic differential scale (previous slide)
Questionnaire Design
Chapter 13
Some Key Issues
Questions: Common Pitfalls• Incomprehensible
• When you consider a new stadium, is it possible that part of your determining factors might rest on the fact that you sometimes choose to forgo entertainment in favor of more pedestrian activities like walking your dog?
• Unanswerable• Will building a new stadium in Boise cost too much?
• Leading• Wouldn’t you agree that it is a great idea to build a new multiuse
facility in Boise?• Double-barreled
• Do you think the old stadium needs to be replaced and a new stadium should be built downtown?
Purpose of Questions• Descriptive
• Describe characteristics of your sample
• Predictive (Hypothesis Testing)• What demographic factors are associated with support for the stadium?
• Every question should be designed to provide useful information• What is our primary goal in the stadium study?• What questions are we trying to address?
• How do our questions stack up? • Good, bad questions? • Do they each answer a question? Descriptive? Predictive?
Arrangement of Questions: Flowerpot Approach
Exhibit 13.5