Upload
phamquynh
View
215
Download
0
Embed Size (px)
Citation preview
TESTING FOR HIERARCHICAL STRUCTUREAND PRIMING EFFECTS AMONG
INDIVIDUAL VALUE CHOICES
William G. JacobyDepartment of Political Science
Michigan State University303 South Kedzie HallEast Lansing, MI 48824
A preliminary report on a study supported by the programfor Time-Sharing Experiments in the Social Sciences
February 2006
This paper is a preliminary report from a study of value choices within the American mass
public. Longstanding psychological theories have stressed that individual value preferences
are very stable, highly structured and broadly applicable to virtually all situational contexts.
But, recent research suggests that value choices may be relatively malleable and susceptible to
contextual effects. Furthermore, the apparent existence of value ambivalence raises questions
about citizens’ abilities to differentiate among values in the first place. At the present time, it
is not clear which of these two general perspectives provides the most accurate representation
of core values.
The program for Time-Sharing Experiments in the Social Sciences (TESS) provided sup-
port for a study that used some new items on an internet survey in order to to address the
important issues raised by these potentially conflicting perspectives. The empirical results
show that most people do make consistent pairwise choices among values. Furthermore, most
individuals’ sets of pairwise choices are fully transitive, meaning that they can be assembled
into hierarchical structures. Finally, value choices are largely impervious to priming effects.
SUBSTANTIVE BACKGROUND
For present purposes, “values” can be defined as an individual’s abstract, general concep-
tions about the desirable and undesirable end-states of human life ( 1973). As such, values
provide criteria for evaluating external stimuli and interacting with other elements of the
social environment. They effectively define what is ”good” and ”bad” in the world.
Human values have been the focus of an enormous amount of research across a variety
of disciplines, including political science (Kuklinski 2001; Feldman 2003), sociology (Spates
1983), economics (Katona 1975), marketing (Kamakura and Mazzon 1991), and philosophy
(Hansson 2001). However, the modern, empirical study of values really began in psychology,
particularly with the pioneering work of Milton Rokeach (1973; 1979). Subsequent research
has proceeded in a wide variety of directions (Seligman, Olson, Zanna 1996). But, there
are two general ideas drawn from this research tradition that deserve particular emphasis:
First, a critical distinguishing feature of values is their applicability across different contexts
(Schwartz and Bilsky 1987; Schwartz 1996). Values provide individuals with general evalu-
ative standards that can always be applied, regardless of the specific circumstances under
which the values are invoked. Second, people are rarely affected by single values, in isolation
from other values. Instead, hierarchically-organized value structures are the key to under-
standing human behavior (e.g., Schwartz and Bilsky 1987; Schwartz 1992; 1996; Verplanken
and Holland 2002).
There is a clear consensus in the psychological literature regarding the contextual sta-
bility and hierarchically structured nature of individual values. However, recent research–
particularly, in political science– has taken very different, and potentially conflicting, posi-
tions with respect to these value characteristics. For example, theories of value pluralism
(Tetlock 1986), issue framing (Nelson, Clawson, Oxley 1997; Grant and Rudolph 2003),
and campaign effects (McCann 1997) all hold that value choices are context-specific and
changeable in different circumstances. In a somewhat different vein, theories of value am-
bivalence (Feldman and Zaller 1992; Alvarez and Brehm 2003) and indifference (Maio and
Olson 1998) suggest that people may have trouble making and/or expressing reliable and
consistent choices between values in the first place.
The apparent variability and ambivalence in human values pose serious challenges for tra-
ditional psychological theories. If individual value preferences are changeable with contextual
factors, then the ”transsituational” (Schwartz 1996) nature of values is badly compromised
(Seligman and Katz 1996). Similarly, ambivalence about values challenges the existence of
transitive choices and, thereby, largely precludes the existence of individual value structures
(Maio and Olson 1998; Jacoby 2002). Thus, it is critically important to determine whether
the recent findings have merit, or whether the traditional theoretical understanding provides
the more accurate depiction of individual values.
The existing empirical evidence is ambiguous and largely indeterminate. For example,
studies that seem to reveal framing effects usually examine differences in reactions to single
values. In fact, preference orders across multiple values do not vary systematically across
2
issue frames (Nelson et al. 1997, p. 576). Similarly, most studies of ambivalence never test
directly individual willingness or ability to make choices between specific values; instead,
they examine indirect evidence based upon the supposed consequences of value ambivalence
(Feldman and Zaller 1992; Alvarez and Brehm 2002).
Finally, and perhaps most fundamentally, the very existence of hierarchical value struc-
tures has never been rigorously tested. Again, some studies simply focus on single values,
taken separately from other values (e.g., Feldman 1988). In other cases, researchers ask
subjects to rank-order a set of values (e.g., Rokeach 1973; Schwartz 1992). Both of these
approaches bypass any direct assessment of consistency or transitivity in value choices. The
one line of work that does attempt to construct preference orders from pairwise value choices
(Jacoby 2002; 2006) is based upon a very limited dataset which precludes evaluation of the
measurement error that undoubtedly exists in expressed statements of choices between values
(Miethe 1985; McCarty and Shrum 2000).
STUDY DESIGN AND DATA COLLECTION
The problems outlined in the previous section are addressed by using a new set of items,
administered as part of an internet survey that was supported by the program for Time-
Sharing Experiments in the Social Sciences. The project focuses on individual choices among
five important values: Liberty, economic security, equality, morality, and social order. These
particular values were selected because of their salience in American political culture and
their relevance to current policy controversies. The values were presented to the survey
respondents under varying experimental conditions aimed at priming particular values and
value conflicts. Respondents made choices among subsets of the values in a manner that
enables empirical tests of hierarchical value structuring. Such tests are impossible with more
traditional data collection strategies (i.e., rating single values or rank-ordering the full set of
values).
The overall plan of the study is laid out in Figure 1. Respondents were randomly assigned
to one of eight conditions. Seven of these conditions involve two survey questions that either
3
emphasize a specific value (five conditions) or the conflict between a pair of values (two
conditions). A control group was given two questions that do not mention values. The
questions used in each of the conditions are shown in Part I of the Appendix.
Next, all respondents were asked to place themselves along the liberal-conservative con-
tinuum using a relatively standard seven-point scale (Part II in the Appendix). This item
served as a distraction task for respondents in the seven experimental conditions. The re-
sponses to this question are also intrinsically important because ideology is one of the factors
that is hypothesized to affect value ambivalence (Feldman and Zaller 1992) and, hence, the
degree of hierarchical structuring among individual value choices.
The value choices were elicited next, using the method of triads (Coombs 1964; Weller
and Romney 1988). Respondents were first shown a screen that introduced, and provided
a brief definition for, each of the five values (Part IIIA in the Appendix). After that, they
were shown a series of ten screens. On each screen, a distinct combination of three values
(i.e., “a triad”) was listed. For each triad, respondents were asked to indicate which one of
the three values is most important, and which one of the three is least important. Given the
nature of the internet survey, respondents could make the selections very easily, by clicking
radio buttons with the computer mouse. The ten triads (i.e., all possible subsets of three
values from the five) are shown in Part IIIB of the Appendix. The order of the triads was
varied randomly, as was the order of the three values within each triad.
The internet survey was conducted by Knowledge Networks, Inc.; it is designated ”TESS
37 Jacoby.” Data collection was carried out from June 16 through June 21, 2005. A total of
1000 surveys were fielded, and 649 respondents completed the interview schedule. Thus, the
response rate was about 65 %.
CONSISTENCY AND TRANSITIVITY IN VALUE CHOICES
Let us begin by considering choices among pairs of values. The individuals’ responses
to each triad can be broken down into three pairwise choices. For example, assume a triad
4
containing values A, B, and C, with a respondent stating that A is most important and
C is least important. This, in turn, implies that A is more important than B, B is more
important than C, and A is more important than C.
In the full set of ten triads, each pair of values appears in three different triads. There-
fore, three replications of each respondent’s choice between each value pair are recovered.
This allows an assessment of consistency in pairwise choices (i.e., how many times does the
respondent make the same choice for each pair?) and permits the determination of that
person’s “dominant” choice (i.e., the value chosen two or three times) for each value pair.
Consistency in Pairwise Choices
Table 1 presents basic descriptive data on the pairwise value choices. Each row of the
table represents one of the ten distinct value pairs that can be derived from the set of five
values. Beginning at the left side of the table, the first four columns represent the number of
times the first-mentioned value in that row is chosen over the second-mentioned value, across
the three replications for that pair. The rightmost column within each row summarizes the
amount of consistency in choices involving that value pair; that is, the percentage of times
respondents chose one or the other value consistently across all three replications of that
particular value pair.
The entries in Table 1 show that the respondents are highly consistent in their value
choices. In every case, 70% or more make the same pairwise choice across all three replica-
tions. Stated somewhat differently, the percentage of respondents who exhibit some degree
of inconsistency ranges from a high of 29.58% (for choices between equality and morality)
or just under one-third of the sample, to a low of 19.57% (for choices between morality and
social order) or just under one-fifth of the sample
As a standard of comparison for the entries in Table 1, consider an admittedly unrealistic
situation in which everyone is indifferent between two values (due to ambivalence, failure to
recognize any potential conflict between the values, and so on). In that case, the probability
5
of choosing one value over the other on a single choice is 0.50. And, the probability of
making consistent choices across all three replications is only 0.25. In other words, if people
did not maintain real preferences between a given pair of values, then only one-fourth of the
sample should be exhibiting consistent choices across the three replications of each choice.
But, again, the empirical levels of consistency are much higher than this stringent standard.
This, in turn, suggests that most people really do have crystallized preferences across the
values examined in this study.
Variability in Value Importance
The replicated pairwise choices can be used to determine each person’s “dominant” choice
for each value pair; that is, the value that is chosen two or three times (i.e., a majority) across
the replications. These dominant pairwise choices can, in turn, be used to determine each
respondent’s beliefs about the relative importance of the separate values. This is accom-
plished by assigning each value a score (for each survey respondent), showing the number of
times that value is the dominant choice over other values in that person’s full set of choices
(Peffley, Knigge, Hurwitz 2001). As a simple example, consider three values: A, B, and C.
Assume that these are presented to respondents in triads (probably along with other values),
and that an arbitrary respondent has A dominating B (i.e., A was the selected over B on at
least two of the three choices), A also dominating C, and B dominating C. For this person,
A would receive a score of 2, since he/she has it dominating two other values (B and C); B
would be scored 1 (it only dominates C); and C would receive a 0, since it never dominates
any other values (even though it may have been chosen in one of the replicated pairwise
choices). With the Knowledge Networks data, there are five values, so the scores for each
value can range from zero to four.
Table 2 shows the distribution of importance scores for each of the four values. The most
striking feature of the table is the wide variability among the preferences. At the individual
level, there does not seem to be general agreement about which values are most (or least)
important. “Morality” received the largest number of high ( ”4”) scores but even so, only
6
about one-third of the respondents rated this value as more important than any other. And,
very few people rated “equality” or “social order” very highly; less than eight percent of the
respondents placed each of these values at the top of their rankings. At the other extreme,
”social order” received the largest number of zero scores. But, again, only about 30% of
the respondents ranked this value in last place. Overall, it is fair to say that each of the
four values shows up in every possible position within the value hierarchies of a substantial
number of people.
The dot plot shown in Figure 2 summarizes succinctly the aggregate distribution of
importance ratings for each value. Specifically, the horizontal position of the point plotted
within each row corresponds to the mean importance score for that value; the solid bar
around each point represents a 95% confidence interval. From the figure, it can be seen that
morality, economic security, and liberty are considered to be the most important values.
Their mean importance scores are 2.36, 2.32, and 2.29, respectively. However, as can be
discerned from the overlapping confidence intervals in Figure 2, the differences between
these means are not statistically significant. The remaining two values, equality and social
order, have significantly lower mean importance scores, at 1.58 and 1.45, respectively. Again,
the difference between these two means is not statistically significant.
At the aggregate level, Americans do not seem to differentiate fully among the different
values. Instead, public opinion sorts values into two distinct sets: One group that is relatively
more important, composed of morality, liberty, and economic security and a second group
that is usually viewed as less important, equality and social order.
Are Individual Value Choices Transitive?
For each survey respondent, the full set of dominant pairwise choices can be examined
for transitivity. If transitivity holds in a respondent’s value choices then, for any set of three
values (again, say A, B, and C), choices on two of the pairs imply the choice that will be
made on the third pair. So, if A is chosen over B, and B is chosen over C, then transitivity
implies that A is also chosen over C. If an individual’s choices among the values are fully
7
transitive (i.e., the transitivity relation holds for all possible subsets of three values), then
each of the values will receive a distinct score. Therefore, it is possible to assemble that
person’s pairwise choices into a complete rank-ordering of the values. Figure 3 provides
a simple example showing how the dominant pairwise pairwise choices are obtained from
a hypothetical individual’s responses to the triads, and how the dominant choices used to
construct an overall preference ordering for that person.
Intransitivity exists in a set of choices among three values whenever the third choice is
not implied by the other two choices. For example, A is chosen over B, B over C, and then C
is chosen over A. Here, each value is only selected over a single other value, so it is impossible
to construct a rank-ordering of the three; hence, the three choices form an intransitive set.
Thus, the relative prevalence of transitivity and intransitivity in the data constitutes a direct
empirical test for the existence of individual value hierarchies.
Table 3 provides the basic data on transitivity among individual value choices. The first
row of the table shows that only 12.48% of the respondents exhibited any intransitivity
among all of their dominant pairwise choices. Stated differently, nearly nine-tenths of the
respondents maintain fully-ordered (i.e., completely transitive) value preferences. The re-
maining rows in the table give the proportion of intransitive choices on all subsets of three
values. Parenthetically, note that some people are intransitive on more than one triad; hence
the proportions intransitive on the particular triads sum to more than the 12.48% figure given
above.
The information in Table 3 shows a very obvious pattern: On any given triad, only a
tiny minority of the respondents give intransitive choices among the values. The number of
intransitivities never rises above about three percent on any value triad, and it is generally
smaller than that. Clearly, the vast majority of the respondents have no difficulty providing
a complete ordering of the five values. This, in turn, provides strong confirmation for the
existence of fully-differentiated hierarchical value structures within the mass public.
8
CAN VALUE CHOICES BE PRIMED?
The experimental manipulation in this study randomly assigned survey respondents to
eight different conditions. Five of these conditions are each intended to prime one particular
value. Specifically, respondents were asked to indicate the degree to which the Bush and
Kerry presidential campaigns emphasized one of the values (See sections IA through IE in
the Appendix for question wording). Two other conditions emphasized value conflict, by
asking respondents to indicate the degree to which the presidential campaigns emphasized
liberty versus social order (see section IF in the Appendix) and liberty versus equality (see
section IG in the Appendix). Finally, the eighth condition did not prime any values; instead,
respondents were asked about partisanship in the presidential campaigns (Appendix section
IH). After the experimental manipulation, all respondents were asked to place themselves
along a seven-point liberal-conservative scale. Then, they completed the battery of value
choice items.
If values are the “transsituational” phenomena argued by Rokeach (1973), Schwartz
(1996) and others, then priming should have no discernible effect on pairwise value choices.
Alternatively, if feelings about values can be manipulated through environmental conditions
(e.g., Tetlock 1986; Nelson, Clawson, Oxley 1997; Maio and Olson 1998), then priming
particular values should enhance the degree to which those values are chosen when they are
juxtaposed against other values. The design of the current study enables multiple tests of
the priming hypothesis.
With five values, there are ten pairwise choices. For each pair, the dependent variable
is the probability that one value is the dominant choice over the other. It is important
to emphasize that our empirical measurement of these choices is particularly reliable, since
dominant choices result from the choices made over the three replications of each value pair
from the triad items. For each of the ten value pairs, there are two independent variables.
These are dummies, scored one if one of the two values in the pair was primed, and zero
otherwise. For example, assume that the dependent variable is the probability of choosing
9
equality over economic security. In that case, people who received the equality prime are
scored one on the first predictor, and people who received the economic security prime are
scored one on the other predictor. According to the priming hypothesis, the first predictor
should increase the probability of choosing equality, while the second predictor should de-
crease that probability. Note that respondents in either of the two value conflict conditions
are coded as having been primed for both values in the pair.
Table 4 contains the results from this analysis. Each row of the table contains the
maximum likelihood estimates for a logistic regression predicting a value choice using primes
for the two values included in that particular pair. Notice first that the equations do not fit
the data very well. All of the pseudo R-squared values are very small (i.e., all but one have
zeroes out to at least the second decimal place). Turning to the individual value primes,
a number of the coefficients have the wrong sign. That is, five of the coefficients for the
first prime are negative (they should be positive) and three of the coefficients for the second
prime are positive (they should be negative). Note, however, that only two of the coefficients
are statistically different from zero at the 0.05 level (directional test): The equality prime
increases the probability of choosing that value over economic security and also over social
order. In all of the remaining instances, exposure to a value prior to a pairwise choice
situation has no systematic impact on the probability of choosing that value over another
value.
Furthermore, even the two seemingly-significant coefficients in Table 4 should not be
taken too seriously. Since each of the predictors in each of the equations constitutes a
potential prime, there are actually 20 tests contained in the empirical results. The existence
of these multiple tests should be taken into account when assessing statistical significance.
The Bonferroni adjustment for performing n one-sided tests at the α level is α/n. Here, that
produces a probability value of 0.0025; in other words, the latter value is the probability level
that should be required to reject the null hypothesis at the 0.05 level, across 20 different
tests. This would, in turn, require a coefficient that is 2.81 times larger than its standard
10
error. Neither of the significant coefficients in Table 4 meet this more stringent (but more
valid) standard. Thus, there is virtually no reliable evidence in these results for priming
effects in value choices.
CONCLUSION
The TESS Program provided a unique opportunity for an empirical analysis of of individ-
ual value choices. The preliminary results described in this report lead to several conclusions–
all, admittedly, somewhat tentative at this point. First, the successful completion of the data
collection, itself, shows that the method of triads is a reasonable strategy for obtaining repli-
cated pairwise comparisons among values. This is important because it provides reliable
information on value choices. The latter can be used to test for individuals’ hierarchical
value structures. Without this capability, the existence of such rank-ordered structures must
remain an untested assumption.
Second, the data from the internet survey show that people do, generally, make consis-
tent choices among values. That, in turn, attests to the viability of values as a psychological
construct. If values were only “truisms,” without strong cognitive and affective underpin-
nings, then we would expect people to move frequently from one to the other when asked
to choose between them. But, that just did not happen with these data: The vast majority
of the survey respondents made the same choice across the three replications of each value
pair. So, this provides strong evidence against criticisms that have been leveled against the
values concept in the recent literature.
Third, the distribution of value preferences reveals both consensus and conflict in the
ways that Americans think about fundamental principles like liberty and equality. There
appears to be wide agreement that morality, liberty, and economic security comprise the most
important values, with equality and social order falling at somewhat less salient positions
in public esteem. But, these aggregate patterns emerge from enormous variability among
individual choices among the values. All five of these values are highly important for some
subsets of the public, and less so for others.
11
Fourth, individual value choices are usually transitive. This means that people really do
differentiate among values. And, that is consistent with the existence of hierarchical value
structures throughout the mass public.
Fifth, the data show no evidence of priming effects in value choices. Experimental ma-
nipulations intended to enhance the accessibility of one value relative to another simply had
no effect on the probability of choosing one value over the other. Such a finding is consistent
with the view that values are “transsituational” phenomena, or judgmental standards that
are relatively immune to external influences.
In conclusion, the initial evidence from this study vindicates psychological theories and
scholarly understandings that have been widely accepted, but seldom tested, over the years.
Of course, further analysis is necessary in order to deal with questions like the following:
What factors influence individual value choices? What causes the inconsistent and/or intran-
sitive value choices that, although relatively rare, do arise in the data? Are there identifiable
subsets of the public whose value choices can be affected by environmental factors? Of
course, I intend to address all of these questions in the immediate future. But, even at this
early point, it is clear that the TESS program’s support of this study has enabled important
insights regarding value choices in the American mass public.
12
REFERENCES
Alvarez, R. Michael and John Brehm. (2002) Hard Choices, Easy Answers. Princeton, NJ:Princeton University Press.
Coombs, Clyde H. (1964) A Theory of Data. New York: Wiley.
Feldman, Stanley. (2003) “Values, Ideology, and the Structure of Political Attitudes.” InOxford Handbook of Political Psychology. David O. Sears, Leonie Huddy, Robert Jervis(Editors). New York: Oxford University Press.
Feldman, Stanley and John Zaller. (1992) “The Political Culture of Ambivalence: Ide-ological Responses to the Welfare State.” American Journal of Political Science 36:268-307.
Grant, J. Tobin and Thomas J. Rudolph. (2003) “Value Conflict, Group Affect, and theIssue of Campaign Finance.” American Journal of Political Science 47: 453-469.
Hansson, Sven Ove. (2001) The Structure of Values and Norms. Cambridge, UK: Cam-bridge University Press.
Jacoby, William G. (2002) Core Values and Political Attitudes. In Barbara Norrander andClyde Wilcox (Editors), Understanding Public Opinion (Second Edition). Washington,DC: Congressional Quarterly.
Jacoby, William G. (2006) “Value Choices in the American Public.” American Journal ofPolitical Science. Forthcoming.
Kamakura, Wagner A. and Jose Alfonso Mazzon. (1991) “Value Segmentation: A Modelfor the Measurement of Values and Value Systems.” Journal of Consumer Research18: 208-218.
Katona, George. (1975) Psychological Economics. New York: Elsevier.
Kuklinski, James H. (2001) Introduction: Political Values.” In Citizens and Politics: Per-spectives from Political Psychology. James H. Kuklinski (Editor). Cambridge, UK:Cambridge University Press.
Maio, Gregory R. and James M. Olson. (1998) Values as Truisms: Evidence and Implica-tions.” Journal of Personality and Social Psychology 74: 294-311.
McCann, James A. (1997) Electoral Choices and Core Value Change: The 1992 PresidentialCampaign.” American Journal of Political Science 41: 564-583.
McCarty, John A. and L. J. Shrum. (2000) “Measurement of Personal Values in Research.”Public Opinion Quarterly 64: 271-298.
Miethe, Terance D. (1985) “The Validity and Reliability of Value Measurements.” Journalof Psychology 119: 441-453.
Nelson, Thomas E.; Rosalee A. Clawson; Zoe M. Oxley. (1997) “Media Framing of a CivilLiberties Conflict and Its Effect on Tolerance.” American Political Science Review 91:567-583.
Rokeach, Milton. (1973) The Nature of Human Values. New York: Free Press.
Rokeach, Milton. (1979) Understanding Human Values: Individual and Societal. New York:Free Press.
Schwartz, Shalom H. (1992) “Universals in the Content and Structure of Values: TheoreticalAdvances and Empirical Tests in 20 Countries.” In Mark P. Zanna (Editor), Advancesin Experimental Social Psychology. Orlando, FL: Academic Press.
Schwartz, Shalom H. (1996) “Value Priorities and Behavior: Applying a Theory of Inte-grated Value Systems.” In Clive Seligman, James M. Olson, Mark P. Zanna (Editors),he Psychology of Values: The Ontario Symposium, Volume 8. Mahwah, NJ: LawrenceErlbaum Associates.
Schwartz, Shalom H. and Wolfgang Bilsky. (1987) “Toward a Universal PsychologicalStructure of Human Values.” Journal of Personality and Social Psychology 53: 550-562.
Seligman, Clive and Albert N. Katz. (1996) “The Dynamics of Value Systems.” In ThePsychology of Values: The Ontario Symposium, Volume 8. Clive Seligman, James M.Olson, Mark P. Zanna (Editors). Mahwah, NJ: Lawrence Erlbaum Associates.
Seligman, Clive; James M. Olson; Mark P. Zanna. (1996) The Psychology of Values: TheOntario Symposium, Volume 8. Mahwah, NJ: Lawrence Erlbaum.
Spates, James L. (1983) “The Sociology of Values.” Annual Review of Sociology 9: 27-49.
Tetlock, Philip E. (1986) “A Value Pluralism Model of Ideological Reasoning.” Journal ofPersonality and Social Psychology 50: 819-827.
Verplanken, Bas and Rob. W. Holland. (2002) “Motivated Decision Making: Effects of Ac-tivism and Self-Centrality on Values in Choices and Behavior.” Journal of Personalityand Social Psychology 82: 434-447.
Weller, Susan C. and A. Kimball Romney. (1988) Systematic Data Collection. NewburyPark, CA: Sage.
14
APPENDIX
ITEM LIST FOR INTERNET SURVEY
I. Experimental Manipulation. The general objective is to encourage respondents to thinkabout values without explicitly invoking their own value choices. To this end, the items askrespondents to assess the degree to which values were apparent in the 2004 major partypresidential campaigns (this, of course, assumes that the survey would go into the field after theelection). There are eight different conditions total: Five conditions prime a single value; twoconditions prime conflict between two values; and one condition is a control which does notprime any values. In each condition, respondents are asked two questions. Note that the orderin which the candidates are mentioned will be varied randomly.
A. “Liberty” condition:
In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of liberty, or guaranteeing the widest freedom possible foreveryone to act and think as they consider most appropriate.
Please use the following scale to indicate how much you think the Bush campaignemphasized liberty. Just use the mouse to click on the appropriate scale value:
1 2 3 4 5 6 7
The Bush campaigndid not emphasize liberty at all
The Bush campaignplaced a great deal of
emphasis on liberty
How about the Kerry campaign? How much did the Kerry campaign emphasize liberty?
1 2 3 4 5 6 7
The Kerry campaigndid not emphasize liberty at all
The Kerry campaignplaced a great deal of
emphasis on liberty
16
B. “Equality” condition:
In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of equality, or making sure that everybody has the samechance to get ahead in life.
Please use the following scale to indicate how much you think the Bush campaignemphasized equality. Just use the mouse to click on the appropriate scale value:
1 2 3 4 5 6 7
The Bush campaigndid not emphasize equality at all
The Bush campaignplaced a great deal ofemphasis on equality
How about the Kerry campaign? How much did the Kerry campaign emphasize equality?
1 2 3 4 5 6 7
The Kerry campaigndid not emphasize equality at all
The Kerry campaignplaced a great deal ofemphasis on equality
17
C. “Economic Security” condition:
In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of economic security, or guaranteeing a steady job and adecent income to everyone.
Please use the following scale to indicate how much you think the Bush campaignemphasized economic security. Just use the mouse to click on the appropriate scale value:
1 2 3 4 5 6 7
The Bush campaigndid not emphasize economic security at all
The Bush campaignplaced a great deal
of emphasis oneconomic security
How about the Kerry campaign? How much did the Kerry campaign emphasize economicsecurity?
1 2 3 4 5 6 7
The Kerry campaigndid not emphasize economic securityat all
The Kerry campaignplaced a great deal
of emphasis oneconomic security
18
D. “Morality” condition:
In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of morality, or people living according to the rules thatconstitute decent human behavior.
Please use the following scale to indicate how much you think the Bush campaignemphasized morality. Just use the mouse to click on the appropriate scale value:
1 2 3 4 5 6 7
The Bush campaigndid not emphasize morality at all
The Bush campaignplaced a great deal ofemphasis on morality
How about the Kerry campaign? How much did the Kerry campaign emphasize morality?
1 2 3 4 5 6 7
The Kerry campaigndid not emphasize morality at all
The Kerry campaignplaced a great deal ofemphasis on morality
19
E. “Social Order” condition:
In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized the idea of social order, or being able to live in a safe and peacefulsociety where the laws are respected and enforced.
Please use the following scale to indicate how much you think the Bush campaignemphasized social order. Just use the mouse to click on the appropriate scale value:
1 2 3 4 5 6 7
The Bush campaigndid not emphasize social order at all
The Bush campaignplaced a great deal of
emphasis on social order
How about the Kerry campaign? How much did the Kerry campaign emphasize social order?
1 2 3 4 5 6 7
The Kerry campaigndid not emphasize social order at all
The Kerry campaignplaced a great deal of
emphasis on social order
20
F. “Liberty versus Social Order” condition (note that the order in which liberty and social orderare mentioned will be varied randomly across respondents in this condition):
In the recent presidential election campaign, the candidates and parties emphasized thedifferences between certain themes and ideas. We are interested in your feelings about thehow much the two candidates emphasized the idea of liberty (that is, guaranteeing thewidest freedom possible for everyone to act and think as they consider most appropriate)versus the idea of social order (that is, being able to live in a safe and peaceful societywhere the laws are respected and enforced).
Please use the following scale to indicate how much you think the Bush campaignemphasized liberty versus social order. Just use the mouse to click on the appropriate scalevalue:
1 2 3 4 5 6 7
The Bush campaignplaced much more emphasis on libertythan on social order
The Bush campaignemphasized liberty
and social orderjust about equally
The Bush campaignplaced much moreemphasis on social
order than on liberty
How about the Kerry campaign? How much did the Kerry campaign emphasize libertyversus social order?
1 2 3 4 5 6 7
The Kerry campaignplaced much more emphasis on libertythan on social order
The Kerry campaignemphasized liberty
and social orderjust about equally
The Kerry campaignplaced much moreemphasis on social
order than on liberty
21
F. “Liberty versus Equality” condition (note that the order in which liberty and equality arementioned will be varied randomly across respondents in this condition):
In the recent presidential election campaign, the candidates and parties emphasized thedifferences between certain themes and ideas. We are interested in your feelings about thehow much the two candidates emphasized the idea of liberty (that is, guaranteeing thewidest freedom possible for everyone to act and think as they consider most appropriate)versus the idea of equality (that is, making sure that everybody has the same chance to getahead in life).
Please use the following scale to indicate how much you think the Bush campaignemphasized liberty versus equality. Just use the mouse to click on the appropriate scalevalue:
1 2 3 4 5 6 7
The Bush campaignplaced much more emphasis on libertythan on equality
The Bush campaignemphasized libertyand equality just
about equally
The Bush campaignplaced much more
emphasis on equality than on liberty
How about the Kerry campaign? How much did the Kerry campaign emphasize libertyversus equality?
1 2 3 4 5 6 7
The Kerry campaignplaced much more emphasis on libertythan on equality
The Kerry campaignemphasized libertyand equality just
about equally
The Kerry campaignplaced much more
emphasis on equality than on liberty
22
E. Control Group, or “Political Party” condition:
In the recent presidential election campaign, the candidates and parties both emphasizedcertain themes and ideas. We are interested in your feelings about the extent to which thecandidates emphasized their ties to their own political parties.
Please use the following scale to indicate how much you think the Bush campaignemphasized its ties to the Republican party. Just use the mouse to click on the appropriatescale value:
1 2 3 4 5 6 7
The Bush campaigndid not emphasize the Republican party at all
The Bush campaignplaced a great dealof emphasis on the
Republican Party
How about the Kerry campaign? How much did the Kerry campaign emphasize theDemocratic party?
1 2 3 4 5 6 7
The Kerry campaigndid not emphasize the Democraticparty at all
The Kerry campaignplaced a great dealof emphasis on the
Democratic Party
23
II. Distraction Task. The idea here is to encourage respondents to think about something otherthan the value or values that were included in their version of the experimental manipulation.The latter should still remain accessible in short-term memory. But, this distraction is intendedinhibit the degree to which their own expressed value choices (obtained immediately afterward)are based directly upon their responses to the experimental manipulation items.
Many people use the labels “liberal,” “moderate,” or “conservative” to describe their ownpolitical views. Which one of the following labels best describes your own political ideas?Please use the mouse to click the check box next to the appropriate label:
Q Extremely conservative
Q Conservative
Q Sightly conservative
Q Moderate, middle of the road
Q Slightly liberal
Q Liberal
Q Extremely liberal
Q I don’t generally think ofmyself in these terms.
24
III. Value Triads. The objective here is to elicit respondents’ value preferences in a manner thatprovides repeated measurement of each pairwise value comparison and does not presuppose theexistence of a fully-ranked value hierarchy.
A. Instruction Screen:
We’d like to ask you about some things that are important for our society, such as liberty,equality, economic security, morality, and social order. First, here is what we mean by theseideas:
By LIBERTY we mean a guarantee of the widest freedom possible for everyone to actand think as they consider most appropriate.
By EQUALITY we mean narrowing the gap in wealth and power between the rich andthe poor.
By ECONOMIC SECURITY we mean the gurarantee of a steady job and a decentincome.
By MORALITY we mean people living according to the rules that constitute decenthuman behavior.
By SOCIAL ORDER we mean being able to live in an orderly and peaceful societywhere the laws are respected and enforced.
All five of these ideas are important, but sometimes we have to choose between what is moreimportant and what is less important. And, the specific choices we make sometimes dependupon the comparisons we have to make.
On the next few screens, we will show you these ideas in sets of three. For each set, pleaseuse the mouse to indicate the idea that you think is most important of the three, and also theidea that you think is least important of the three. In some cases, you might think all threeof the ideas are very important, but please try to indicate the ones you think are most andleast important if you had to choose between them.
If you absolutely cannot decide which ideas are most or least important, then just click thebox at the bottom of the screen, and move on to the next set of three ideas.
25
B. Value triads. Each set of three values will appear on a separate screen. Note that the orderof the triads, as well as the order of the three values within each triad, will be variedrandomly.
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Liberty
Q Q Equality
Q Q Economic Security
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Equality
Q Q Morality
Q Q Social Order
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Economic Security
Q Q Morality
Q Q Social Order
26
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Liberty
Q Q Economic Security
Q Q Morality
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Liberty
Q Q Morality
Q Q Equality
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Social Order
Q Q Equality
Q Q Economic Security
27
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Equality
Q Q Social Order
Q Q Liberty
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Morality
Q Q Economic Security
Q Q Equality
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Economic Security
Q Q Social Order
Q Q Liberty
28
The MOSTIMPORTANTidea from these
three is:
The LEASTIMPORTANTidea from these
three is:
Q Q Liberty
Q Q Morality
Q Q Social Order
Table 1: Consistency in pairwise value choices.
Number of times the first value is chosen overthe second value across the three replications Percentage making
consistent choicesacross all three
replicationsZero One Two Three
Liberty versus equality 25.12% 11.56% 15.72% 47.61% 72.73%
Liberty versus economic security 38.67% 12.02% 12.33% 36.98% 75.65%
Liberty versus morality 41.91% 10.02% 10.63% 37.44% 79.35%
Liberty versus social order 21.57% 10.48% 11.40% 56.55% 78.12%
Equality versus economic security 52.39% 14.64% 10.02% 22.96% 75.35%
Equality versus morality 56.09 10.02% 19.57% 14.33% 70.42%
Equality versus social order 33.13% 12.79% 12.63% 41.45% 74.58%
Economic security versus morality 44.68% 7.24% 12.48% 35.59% 80.27%
Economic security versus social order 22.03% 11.71% 15.87% 50.39% 72.42%
Morality versus social order 25.27% 8.32% 11.25% 55.16% 80.43%
Note: In each row, the entries in the four leftmost columns indicate the percentage of respondents that selected thefirst-mentioned value zero, one, two, or three times across the three replications of the pairwise choice. Therightmost column gives the percentage of respondents that made fully consistent choices across all threereplications. Sample size is 649.
Table 2: The distribution of importance scores for each value.
Value
Liberty EqualityEconomicSecurity Morality
SocialOrder
ImportanceScore
Zero: 14.48% 23.88% 9.09% 16.18% 30.20%
One: 15.10% 27.43% 20.80% 15.41% 25.89%
Two: 20.96% 23.57% 22.65% 16.80% 19.72%
Three: 26.19% 17.41% 23.88% 19.11% 16.80%
Four: 23.27% 7.70% 23.57% 32.51% 7.40%
Note: Cell entries give the percentage of respondents with each importance rating for each value. The number ofobservations is 649.
Table 3: Transitivity in pairwise value choices.
Value triad
Percentage whose choices across valuepairs are intransitive
Any intransitive choices across all ten value triads 12.48%
Liberty, equality, economic security 2.47%
Liberty, equality, morality 3.39%
Liberty, equality, social order 1.85%
Liberty, economic security, morality 0.77%
Liberty, economic security, social order 2.16%
Liberty, morality, social order 1.08%
Equality, economic security, morality 2.16%
Equality, economic security, social order 2.77%
Equality, morality, social order 1.08%
Economic security, morality, social order 1.69%
Note: The entry in the first row shows the percentage of respondents with any intransitivity on any ofthe subsets of three values (i.e., triads). The entries in the remaining rows of the table give thepercentage of respondents whose choices on that specific triad are intransitive. All percentagesare based upon 649 observations.
Table 4: The impact of value priming on pairwise choices between values.
Maximum likelihood logistic regression coefficients (standard errors in parentheses)
Pseudo R2Value choice: Prime for firstvalue in pair
Prime for secondvalue in pair
Liberty over equality
-0.068(0.196)
-0.086(0.220)
0.000
Liberty over economic security
0.019(0.187)
0.093(0.242)
0.000
Liberty over morality
0.272(0.187)
-0.188(0.250)
0.004
Liberty over social order
-0.082(0.203)
-0.220(0.223)
0.001
Equality over economic security
0.466*(0.215)
0.404(0.249)
0.007
Equality over morality
0.340(0.215)
0.235(0.255)
0.004
Equality over social order
0.589*(0.220)
-0.159(0.209)
0.010
Economic security over morality
-0.308(0.242)
-0.014(0.244)
0.002
Economic security over social order
-0.053(0.254)
-0.154(0.216)
0.001
Morality over social order
-0.133(0.256)
-0.083(0.218)
0.000
Note: Table entries are maximum likelihood coefficients for a logistic regression equation predicting the valuechoice listed in the leftmost column. The number of observations is 649 in each equation. From the left, thefirst coefficient in each row is for a dummy variable indicating that the first-mentioned value is primed. Thesecond coefficient is for a dummy variable indicating that the second-mentioned value is primed. Accordingto the value priming hypothesis, the first coefficient should be positive, and the second coefficient should benegative. Coefficients marked with an asterisk are significantly different from zero at the 0.05 level in adirectional test.
EXPERIMENTAL MANIPULATION(Eight Conditions, Two Items in Each)
Five conditions, each priming one specific value
Two conditions, each priming a conflictbetween two specific values
One control group receives no value priming
DISTRACTION TASK(All Respondents, One Item)
Liberal-conservative self-placement
INSTRUCTIONS FOR TRIADS(All Respondents)
One screen, providing a one-sentence definition for eachvalue, and instructions for responding to the triad items
VALUE TRIADS(All Respondents, Ten Items)
Ten screens, each of which displays three values. Respondentsselect the most- and least-important value on each screen
Figure 1: Overall Structure of Proposed Study
Mean importance score
0 1 2 3 4
Social Order
Equality
Liberty
Economic Security
Morality
Figure 2: Mean value importance scores.
Note: Points in the dotplot represent mean importance scores (calculated across 649 internet surveyrespondents) for each value. The error bars around each point represent the 95% confidenceinterval for the mean.
Figure 3: A simple example showing how triads are used to obtain a full preference order (and,in so doing, test for transitivity). The example assumes only four values (A, B, C, andD) and follows a single, hypothetical, person’s choices.
A. Triads: All possible subsets of three values (i.e., “triads”) are presented to the respondent. For eachtriad, the most-preferred value is underlined and the least-preferred value is crossed out.
A B CA B DA C DB C D
B. Dominant Pairwise Choices: With four values, each value pair occurs in two different triads(e.g., A and B appear in the first two triads). For all six possible value pairs, examine the choicesmade between them.
B is chosen over A twice (in the first two triads), hence B dominates AC is chosen over A twice (in the first and third triads), hence C dominates AA is chosen over D twice (in the second and third triads), hence A dominates DB is chosen over C twice (in the first and fourth triads), hence B dominates CB is chosen over D twice (in the second and fourth triads), hence B dominates DC is chosen over D twice (in the third and fourth triads), hence C dominates D
Based upon the preceding results, the hypothetical respondent is fully consistent in his/her pairwisechoices (i.e., identical choices occur on both comparisons for each value pair).
C. Obtain the Preference Order from the Dominant Choices: Count the number of times eachvalue dominates another value. The resultant numeric scores give the preference order for therespondent.
A dominates one other value (D), so it is assigned a 1B dominates three other values (A, C, and D), so it is assigned a 3C dominates two values (A and D), so it is assigned a 2D dominates no other values, so it is assigned a 0
For this respondent, the preference ordering for the values (based upon the dominant pairwisechoices) is B (most preferred), C, A, D (least preferred).
If the scores assigned to the values are fully-ordered (as they are in this example), then all of thechoices are transitive and a complete preference ordering for the values can be constructed. If thereare any ties in the scores, then at least some of the pairwise choices are intransitive, and a full rankingof the values is impossible.