Effects of Filter Questions in Public Opinion

Effects of Filter Questions in Public Opinion Surveys

GEORGE F BISHOP ROBERT W OLDENDICK A N D ALFRED J TUCHFARBER

Do you have an opinion on this or not Have you been interested enough in this to favor one side over the other Where do you stand on this issue or havent you thought much about it These questions typify the filters that have long been used in the American National Election Studies to screen out the politically uninformed a practice which stems from the early concerns of Philip Converse (1964 1970) and his colleagues with the problem of non-attitudes in political opinion surveys

Only recently however has anyone attempted to examine sys- tematically the consequences of using such filter questions Schuman and Presser (1978 1981) Comparing filtered with unfiltered questions

Abstract Extending previous work the authors find that the wording of a filter question can make a significant difference in the percentage of dont know (DK) responses elicited by an item especially with topics that are more abstract or less familiar to survey respondents They also find however that the content of an item can have a substantial independent effect on DK or no opinion responses regardless of how the filter question is worded In general it appears that the less familiar the issue or topic the greater the increase in DK responses produced by adding a filter Even more important the analysis shows that filtering can in some instances dramatically affect the conclusions a pollster would draw about the distribution of public opinion on an issue Indeed such effects may occur more often than has previously been sus-pected though the circumstances under which they emerge remain elusive The authors suggest that such effects may become amenable to analysis by probing respondents about what they had in mind as they answered the question

George F Bishop is Associate Professor of Political Science and a Senior Research Associate at the Institute for Policy Research at the University of Cincinnati Robert W Oldendick is Assistant Director and Alfred J Tuchfarber is Director of the Institute for Policy Research University of Cincinnati This research was supported by a grant from the National Science Foundation (SOC 78-07407) The authors want to thank the anonymous reviewers for their useful comments and suggestions for revising the original manuscript

Public Opinion Quarterly Vol 47528-546 0 1983 by the Trustees of Columbia Univers~ty Published by Elsevier Science Publ~shmg Co Inc 0033-362Xl8310047-528$250

529 EFFECTS OF FILTER QUESTIONS

about political and social issues they found that a filter will generally increase the percentage of dont know (DK) or no opinion re-sponses to an item by about 20-25 percent Furthermore their analysis indicates that these increments in DK responses do not depend on the content of an issue and they did not find any relationship between the percentage of DKs which were volunteered on an issue in the absence of an explicit filter question (the standard form) and the percentage of respondents removed by adding one Yet they did discover that the wording of a filter can make a substantial difference in the percentage of respondents who say they have no opinion A filter that is which emphasizes the frequency or ac- ceptability of not having an opinion on an issue will screen out many more people than one which does not

Surprisingly perhaps their research also suggests that in most instances filtering will have little impact on the distribution of substantive responses to an item once the DKs are excluded from the analysis The use of filter questions in their experiments moreover did not appear to have any significant influence on the magnitude of association between substantive responses to issues and such demographic variables as age sex and education A researcher would in other words draw essentially the same conclusion about the nature and determinants of public opinion on an issue on the basis of either a filtered or an unfiltered form (see Schuman and Presser 1981 126-28 137-41)

Independently the authors have investigated many of the same effects of filtering in a series of experiments and replications Here we will try to generalize and extend the work of Schuman and Presser at least in part using a different set of issues and a somewhat greater variety of filter questions However we should make it clear that our investigation was not designed initially as a replication of theirs it originated rather in a broader study of changes in the wording and format of questions on domestic and foreign policy issues in the American National Election Studies (Bishop et al 1982) Our analysis then is more in the nature of an independent probe of the same conceptual territory though it does include a partial replication of two split-ballot experiments from their project

We use DK or DK response throughout the paper as a generic term for various types of nonsubstantive responses On a standard (unfiltered) item this would indicate a volunteered dont know whereas on a filtered form it would refer to saying no to such questions as Do you have an opinion on this or not and Have you thought much about this issue Where appropriate we distinguish between these different meanings

BISHOP OLDENDICK AND TUCHFARBER

Research Design

SURVEYS I A N D I1

Our first experiments were part of two larger random-digit-dialed (RDD)2 telephone surveys (1) a study of public opinion about the responsiveness of local government in Cincinnati Ohio carried out in July and August of 1978 and (2) an omnibus survey on assorted topics conducted in November and early December of 1978 in the Greater Cincinnati metropolitan area3 In both surveys we ran a number of split-ballot experiments which pitted variously worded filtered forms of an issue item against a standard (unfiltered) form (see Appendix and Table 1) With the exception of the SALT issue in Survey I on which everyone was asked the same filter question (see Form A below) we randomly assigned respondents to one of four or five groups depending upon the issue

Form A In this condition respondents were exposed on all issues to a filter adapted from a version used originally in the SRC election studies prior to 1964 Do you have an opinion on this or not (see eg the American National Election Study 1960)

Form B The filter on all questions asked of this group read Have you been interested enough in this to favor one side over the

In these and our other surveys we used pure random-digit-dialing to select a sample of telephone households With this method a potential number is generated by randomly selecting an exchange and then appending a random number between 0001 and 9999 Subsequent numbers are created by repeating these two steps (Klecka and Tuchfarber 1978 106)

Approximately 63 percent of the respondents in the city survey in JulyIAugust 1978 were members of a panel being interviewed for a second time the rest (37 percent) being from a previously uninterviewed control group The reinterview rate for the panel group (14 months after the base period) was 627 percent-ie completed interviews including partials The completion rate (including partials) for the control group was 677 percent Additional information on the subcategories of nonresponse (see below) was not available

In the Greater Cincinnati Survey of NovemberIDecember 1978 the percentage of fully completed interviews with known households (N= 1584) was 736 percent The refusal rate was 157 percent The remainder consisted of partially completed interviews (33 percent) potential interviews that could not be completed because of a language barrier a hearing problem illiteracy senility or physical illness (42 percent) or because the selected respondent was away on vacation a business trip etc (32 percent)

Prior to November 1981 records of busys and no answers to calls were not maintained and included in our reports of response rates for the Greater Cincinnati Surveys The reader should also note that for representativeness the data for this and all other Greater Cincinnati Surveys are weighted inversely in proportion to the number of independent telephone lines in the household and inversely to the number of people-18 years old and over-living in the household (technical documentation available from the authors)

EFFECTS OF FILTER QUESTIONS 531

Table 1 Summary of Split-Ballot Experiments with 12 Issues in 5 Independent Surveys

Survey III Survey I Survey 11 (Spring Survey IV Survey V

(Summer 1978) (Full 1978) 1979) (Fall 1979) (Spring 1980) Form Form Form Form Form

Std A B C D Std A B C D Std A Std A C E Std A C E

Govt vs private solu- tlon of US national problems X X X X X X X X X X - - X X X - X X X -

Aftirmative actlon for blacks on jobs and education X X X X X X X X X X - - - - - - - - - -

Tax cut vs jobs of publicemployees X X X X - X X X X - - - - - - - - - - -

Government-paid vs private health insurance X X X X X X X X X X - - X X X - X X X -

1975 Publ~c Affairs Act X X X X - X X X X - X - X X X - X X X -

Reestablishing diplomatic relations with Cuba X X X X - X X X X - X X X X X - X X X -

Resum~ng arms shipments toTurkey X X X X - X X X X - X X X X X - X X X -

SALTlSoviet interfer- ence in Africanaffairs - X - - - X X X X - - - - - - - - - - -

Russian leaders trying to get along with1 dominate America - - - - - - - - - - x x - - - - - - - -

Arab nations trying to defeatlwork for real peace with Israel - - - - - - - - - - x x - - - - - - - -

Constitutional amendment to balance federal budget - - - - - - - - - - - - X X X - X X X -

SALT I1 agreiment with Soviet Union - - - - - - - - - - - - X - X X X - X X

NOTE X included in experiment - not included

other-a form also borrowed from the SRC election surveys (see eg the American National Election Study 1964)

Form C Here respondents were screened on each item with the question Have you thought much about this issue This represented one of two variations we created (see Form D) of a filter first introduced by the Center for Political Studies in 1970 to accompany a 7-point scaled format for political issues Where would you place yourself on this scale or havent you thought much about it (see the American National Election Study 1970)

Form D On three of the issues (see Table 2) respondents in this condition were asked Where do you stand on this issue or havent you thought much about it But on all other topics they received From C The reader should also note that where both Forms C and D were used the number of cases assigned to each of these conditions was approximately half the size of those allotted to Forms A B and the standard form Our assumption was that Forms C and D were

- - -

532 BISHOP OLDENDICK AND TUCHFARBER

Table 2 Percentage Giving DK Response to Standard and Filtered Forms of Questions About Domestic and Foreign Policy Issues by Survey

Filtered Forms Filter Diff

Std Form Form Form Form Form Form

Issue (unfilt) A B C D E x2 DF P

Surveys I and I1 (combined) Government vs private

solution of US prob- 43 267 303 331 362 - 744 3 059 lems (477) (468) (426) (223) (234) -

Affirmative action for blacks on jobs and 29 80 89 74 84 - 052 3 915 education (475) (466) (429) (224) (234) -

Taxcut vs jobs ofpublic 78 268 258 298 - - 199 2 368 -employees (475) (466) (427) (457) -

Government-paid vs private health insur- 66 147 185 239 219 - 1061 3 014 ance (472) (466) (427) (224) (232) -

1975 Public Affairs Act 651 934 931 958 - - 338 2 184 (475) (465) (426) (454) - -

Diplomatic relations 117 372 428 449 - - 601 2 049 with Cuba (474) (464) (429) (458) - -

Arms shipments to 132 455 494 601 - - 2087 2 000 Turkey (475) (461) (429) (457) - -

SALTISoviet interfer- 94 264 314 347 - - 494 2 084 ence in African affairsa (3 14) (306) (273) (299) - -

Survey I11 Diplomatic relations 166 487 - - - - - - -

with Cuba (561) (575) - - - - - - -Arms shipments to 195 534 - - - -

Turkey (558) (575) - - - - - - -Russian leaders trying to

get along with vs 175 312 - - - - - - -dominate America (561) (574) - - - - - - -

Arab nations trying to defeat vs work for 161 411 - - - - - - -real peace with Israel (559) (575) - - - - - - -

Surveys IV and V (combined) Govt vs private s o h - 134 340 - 312 - - 127 1 260

tion of US problems (768) (793) - (777) - -

Constitutional amendment to balance fed- 114 367 - 458 - - 1311 1 000 era1 budget (764) (794) - (781) - -

Government-paid vs private health insur- 72 188 - 213 - - 145 1 229 ance (762) (795) - (781) - -

1975 Public Affairs Act 734 915 - 953 - - 862 1 003 (757) (793) - (780) - -

Diplomatic relations 160 368 - 382 - - 027 1 601 with Cuba (761) (793) - (777) - -

Arms shipments to 188 461 - 559 - - 1462 1 000 Turkey (759) (791) - (777) - -

SALT I1 agreement with 206 - - 422 - 528 1721 1 000 Soviet Union (792) - - (778) - (792)

a Data from Survey I1 only (see Table 1)


essentially equivalent variations of one another and could probably be combined in analysis though that assumption needed to be tested

Standard Form The respondents in this group were all administered the standard form of the issue questions-that is without a filter

To summarize the first survey (summer 1978) included split-ballot experiments on seven issues with 4 to 5 different question forms plus one issue-SALT-asked in filtered form (Form A) of all respondents In the follow-up survey (fall 1978) we replicated these first seven split-ballots and added one for the SALT issue Altogether then there was a total of eight original manipulations and seven replications in the first two studies

SURVEY I11

In an omnibus RDD telephone survey of the same metropolitan area during the spring of 1979 we repeated the questions on reestablishing diplomatic relations with Cuba and arms shipments to Tur- key4 In addition we replicated two items (in forced-choice form) from an experiment by Schuman and Presser (1981) (1) a question about whether respondents believed that the Russian leaders were basically trying to get along with vs dominate America and (2) one about whether respondents thought the Arab nations were trying to defeat or work for a real peace with Israel (see Appendix) Unlike the items in Surveys I and 11 however all four items in this study were administered in just two versions the standard form and Form A- largely to maximize cases for analysis Finally as part of an inquiry into the sources of opinion-giving in surveys we repeated the question about repealing The 1975 Public Affairs Act Only this time it was asked of everyone in the standard (unfiltered) form

SURVEYS IV AND V

Replications of five items also appeared in two later RDD omnibus surveys of the same metropolitan region one of which was conducted in the fall of 1979 the other in the spring of 19805 Repeated were the

The percentage of fully completed interviews with known households (N= 1482) in this survey was 746 percent the refusal rate 107 percent partially completed interviews 47 percent unable to complete because of language barrier hearing problem illiteracy senility or illness 56 percent unable to complete during field period because selected respondent was on vacation away on business etc 43 percent These data were also weighted as described in footnote 3

The percentage of fully completed interviews with known households (N=1540) in the fall 1979 survey was 728 percent the refusal rate 108 percent partially completed interviews 41 percent unable to complete because of language barrier hearing problem illiteracy senility or illness 75 percent unable to complete during field period because selected respondent was on vacation away on business etc 48


questions on governmental vs private solutions of national problems government-paid vs private health insurance The 1975 Public Af- fairs Act diplomatic relations with Cuba and arms shipments to Turkey Furthermore we added and replicated split-ballot experiments for two new items (1) one about whether the respondent favored or opposed an amendment to the Constitution which would require the federal government to balance its budget every year and (2) another about whether the respondent favored or opposed the strategic arms limitation agreement (SALT 11) between the United States and the Soviet Union (see Appendix)

Six of these seven items were administered in one of three forms the standard form Form A and Form C The one exception occurred on the question about the SALT agreement with the Soviets which was the last item in the sequence There we asked it in standard form Form C and in place of Form A in a version adapted from an NBC NewsIAssociated Press poll on the issue Have you already heard or read enough about it to have an ~ p i n i o n ~ Below we identify this more strongly worded filter as Form E (see Table 2)

Table 1 summarizes the various split-ballot experiments conducted in each of the five surveys Let us turn now to the results

Effects of Filtering on DK Responses

Table 2 shows the percentage of DK responses given to an issue by question form in Surveys I and I1 (combined) Survey 111 and Sur- veys IV and V ( ~ o m b i n e d ) ~ The figures there suggest that the effect

percent The percentage of fully completed interviews with known households (N=1520) in the spring 1980 survey was 749 percent the refusal rate 153 percent partially completed interviews 30 percent unable to complete because of language barrier hearing problems illiteracy senility or illness 48 percent unable to complete during field period because selected respondent was on vacation away on business etc 20 percent In both surveys the data were again weighted as described in footnote 3

See NBC NewsIAssociated Press National Poll 125 October 1979 (available from NBC News 30 Rockefeller Plaza New York NY 10020)

The data for Surveys I and I1 were combined because the experiments in each were exact replications of each other and because there were no significant variations in DK responses to the various items by survey (data not shown) The same was true of the experiments in Surveys IV and V (which were also exact replications of each other) with one minor exception a significant difference in DK responses to the Public Affairs Act In Survey IV the percentage of DKs given to Form A was 922 percent to Form C 933 percent (x2 - 019 df = 1p = 659) But in Survey V the percentage of DKs for Form A was 908 percent for Form C 974 percent (x2 = 1409 df = 1 p = 002) Furthermore the likelihood ratio chi-squared value for the three-way interaction of response by form by survey was statistically significant (x2 = 667 df = I p = 01) Since this was the only exception and because the magnitude of the effect was relatively small we have treated it as a chance fluctuation in combining the data from the two surveys


of a filter depends upon both the content of the item and the wording of the filter Take for example the issue of affirmative action for blacks on jobs and education (see Surveys I and 11) Here we find that a filter question regardless of how it is worded makes but a small difference in the percentage of DK responses adding only another 5-6 percent on the average to the baseline level of DKs volunteered on the standard form In contrast on an obscure foreign affairs issue such as arms shipments to Turkey the addition of a filter increases the DK percentage by anywhere from 27 to 48 percent depending upon the survey and how it is worded Indeed the range of increments in DK responses due to adding a filter tends to be substantially greater in our experiments (45 to 469 percent) than that observed by Schuman and Presser in theirs (roughly 10 to 36 percent) And while the average increment in DKs resulting from a filter appears to be fairly similar in their studies and ours (about 22 percent) this gross resemblance conceals some important differences between them

EFFECTS O F DIFFERENCES IN FILTER WORDING

The last three columns of Table 2 show a chi-squared test of the significance of the differences in DK responses among the variously worded filter questions (ie excluding the standard form) In Surveys I and I1 five out of the eight discrepancies either reach or approach significance And in each of these instances Forms B C and D consistently screen out more respondents than Form A in particular on the more obscure matters such as arms shipments to Turkey Similarly in Surveys IV and V where four of the seven differences are statistically significant Form C invariably removes more respondents than Form A Finally on the SALT I1 issue at least the data indicate that Form E represented an even stronger filter than Form C for in Surveys IV and V the difference between them was sizable (106 percent) and highly significant More generally the results in Table 2 suggest that the more abstract or remote the issue the greater will be the effect of a more strongly worded filter question

Our findings converge moreover with Schumans and Pressers observation that the willingness to give a DK response varies with the strength of a filter question or as they put it by the degree of encouragement provided by a filter Strictly speaking their experiment on the wording of filters involved manipulating both the lead-in statement to the item which emphasized the frequency and ac-ceptability of not having an opinion as well as the location of the filter question within the item (see Schuman and Presser 1981125) Though their results are thus confounded by these simultaneous


manipulations they nevertheless would seem to concur with the evidence reported in Table 28

EFFECTS O F ISSUE CONTENT

Table 3 summarizes the relation between the percentage of DKs given to an issue in the absence of an explicit filter (the standard form) and the increment in no opinion responses created by the Form A filter which was used with all but one issue (SALT 11) Here we have combined the data from the five surveys to make an overall estimate of the effects of issue content Clearly it makes a substantial difference In general the greater the number of DKs that an item elicits voluntarily on the standard form the greater the effect of adding a filter on that topic (rho= 62 p lt05) Schuman and Presser (1978 1981) however found no evidence for any such association in their analysis a discrepancy which may be due to the somewhat

Table 3 Relation Between DK Responses (in Percent) Volunteered on the Standard Form of an Issue and the Percentage Increase in No Opinion Responses Produced by the

Form A Filter in Surveys I-V (Combined)

Issue Std Form Rnk Form A Rnk

Government vs private solution 99 5 212 6 of US problems (1246) (1 262)

Affirmative action for blacks 29 1 51 1 on jobs and education (475) (466)

Tax cut vs jobs of 78 3 190 5 public employees (475) (466)

Government-paid vs private 69 2 101 2 health insurance (1235) (1261)

1975 Public Affairs Act 702 11 218 7 (1232) (1258)

Di~lomatic relations with Cuba 150 7 255 10 (1796) (1832)

Arms shipments to Turkey 175 95 306 11 (1792) (1827)

SALTISoviets in Africa 94 4 170 4 (314) (306)

Russian leaders trying to get 175 95 137 3 along with vs dominate America (561) (574)

Arab nations trying to defeat 161 8 250 8 vs work for real peace with Israel (559) (575)

Constitutional amendment to 114 6 252 9 balance federal budget (764) (794)

NOTE Spearmans rho = 62 t = 237 df = 9 p lt 05 (two-tailed test)

We should also note however that there seems to be little difference in DK responses between what Schuman and Presser would call a quasi-filter (Form D) and a full filter (Form C) at least on the three topics for which comparisons are available in Surveys I and 11 suggesting that their experiment on this aspect of filtering was indeed confounded by the lead-in statement to their question


broader range of issues sampled in our experiments In any case our findings suggest that the number of DK responses given to an item voluntarily reflects (among other things) the general familiarity of an issue to respondents which essentially is what a filter question is intended to measure And the less the familiarity of an issue the greater will be the impact of a filter question to that effect--eg Have you thought much about this issue or Do you have an opinion on this or not In other words are you familiar with this topic

But a dont know or no opinion response may of course indicate something other than lack of familiarity with an issue nota- bly ambivalence (cf Coombs and Coombs 1978 Faulkenberry and Mason 1978) In fact ambivalence could well explain the most prominent deviation in Table 3 namely the large difference in rank- i n g ~which occurred on the question about whether the Russian leaders were trying to get along with or dominate America Intuitively it does not seem plausible that the content of this item would be un- familiar to even the most politically apathetic among us Yet it evoked a considerable number of DKs spontaneously on the standard form (175 percent) to which a filter added little (137 percent) relatively speaking The reason we suspect is that this issue represents a genuine conflict of two basic sentiments optimism vs pessimism about the prospects of peace between the two nations Only by probing people about what they mean when they say dont know or no opinion in response to such issues can we directly test these interpretations-a task for another occasion

Effects of Filtering on Substantive Responses

Though a filter question generally screens out about a fourth or fifth of the sample Schuman and Presser conclude that filtering does not appreciably change the inferences a researcher would make about public preferences on a given issue Indeed they found that filtering had little or no effect on their conclusions about the distribution of substantive responses to various questions though occasionally it made an important difference (Schuman and Presser 1981 126-28) Our analysis however suggests that filtering may make more than just an occasional difference In fact we found that on roughly half of the topics we studied a filter question created a statistically significant difference in the marginals (see Tables 4 5 6 and 7)9 Let us begin with the effects on issues concerning domestic affairs

In Table 4 we have combined the filtered forms for each issue since there were no

- - - - - - - -

----

--


Table 4 Substantive Response to Questions about Domestic Policy Issues by Form and Survey

Survey I Survey I1 Survey IV Survey V (Summer 1978) (Fall 1978) (Fall 1979) (Spring 1980)

Filrered Filtered Filrered Filtered Std Formamp Srd Formamp Std Formamp Std Formamp

Isstre and response Form (Combined) Form (Combined) Form (Combrned) Form (Combined)

Govt vs private solution o f US national ~rob lems

Leave things to individuals and private businesses 405 539 546 658 569 617 591 644

Govt should do more to solve countrys problems 595 461 454 342 431 383 409 356

Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1

Cutting federal taxes i f it means public employees lose jobs

Agree (with tax cut) 401 448 483 583 - - - -Disagree 599 552 517 417 - - - -Total 1000 1000 1000 1000 ( N ) (147) (348) (291) (631)

x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005

DOMESTIC AFFAIRS ISSUES

Consider first the findings from Surveys I and I1 in Table 4 on the issue of governmental versus private sector approaches to solving the nations problems Using the standard form in Survey I we would infer that a substantial majority of citizens (595 percent) favored a governmental approach whereas with the filtered form(s) we would reach just the opposite conclusion-ie that a majority (539 percent) supported a more individualistic private sector solution The data show a similar patern in Survey 11 respondents receiving the filtered form(s) appear to be much more favorable toward the private sector alternative than those administered the standard form1deg Leaving aside the thorny matter of which form constitutes the more valid measure these results demonstrate that filtering can seriously affect

significant differences in substantive responses among them in any of the surveys with one minor exception in Survey I V on issue (a) which failed to replicate in Survey V (data not shown) To conserve space we have not shown the data for the issue questions on which filtering had little or no effect (ie affirmative action government-paid vs private health insurance relations with Cuba arms shipments to Turkey and the constitutional amendment to balance the federal budget)

O Notice however that the level of support for private sector initiatives was generally higher on both forms in Survey I1 than it was in Survey I This variation stems largely it would seem from the fact that we had sampled a better educated more affluent and evidently more conservative population in Survey 11 the general metropolitan area which includes the suburbs of the city of Cincinnati sampled in Survey I


the judgments a pollster would make about the nature of public opinion on this fundamental issue

Yet as the figures for Surveys IV and V indicate the magnitude of the filtering effect on such issues may vary over time for reasons that are not immediately obvious In both of these studies we find again that respondents given the filtered forms were more supportive of private sector initiatives than those given the standard form But in neither survey did the differences which were relatively small in each case (about 5 percent) come close to being statistically significant

The results for the tax cut issue in Table 4 though not as dramatic illustrate some related policy consequences In both Surveys I and 11 respondents administered the filtered form(s) were more likely to agree with the idea that the federal government should cut taxes even if it means that a lot of public employees would lose their jobs Only in Survey 11 however was the difference between forms sizable (10 percent) as well as statistically significant ( x 2 = 758 df = 1p lt 01) The difference nonetheless falls in the same substantive direction as that observed on the previous issue In each case the effect of using a filter question is to increase the percentage in favor of the more conservative alternative-ie the private sector approach to solving national problems and cutting federal taxes even if it means eliminating the jobs of many public employees The policy implications of using (or not using) a filter question on such an issue should thus be obvious

Responses to a fictitious issue do not of course have any policy significance in the real world (cf Bishop et al 1980) Yet the data in Table 5 tell us that a filter question can for whatever reason affect

One possible explanation lies in the unanticipated but significant change in DK responses to this issue over time (see Table 2) For whatever reason the percentage of DKs given voluntarily on the standard form of the national problems item increased dramatically between Survey I1 in the fall of 1978 (43 percent) and Survey IV in the fall of 1979 (170 percent) it then dropped off in Survey V during the spring of 1980 (99 percent) though not to its previous low level (cf Surveys I and 11) Other things equal we would expect this rise in DK responses to the standard form in Surveys IV and V to diminish the difference in substantive responses between forms because it tends to (1) increase the percentage in favor of the private sector solution in much the same way that a filter would (ie by removing cases) and (2) reduce the increment in DKs due to adding a filter The effects of filtering on substantive results for such issues may thus depend in part upon the magnitude of the increment in DK responses produced by a given form-ie the greater the increment the greater the impact--other things being equal Other things of course are not usually equal in particular the content of the issue So the question is whether our proposition about the size of DK increments can be generalized to other items (see the Discussion below)


Table 5 Substantive Response to Question about Repealing The 1975 Public Affairs Act by Form in Surveys I 11 IV and V (Combined)

Standard Filtered Forms Response Form (Combined)

Agree (should be repealed) 484 588 Disagree 516 412

Total 1000 1000 (N1 (368) (182)

substantive reactions to even the most obscure mattersI2 In each of the four surveys taken separately (data not shown) moreover we found that the percentage agreeing that the Public Affairs Act should be repealed was consistently higher with the filtered form(s) than with the standard form So the difference between forms seems to be fairly reliable nor is it inconsequential in magnitude averaging a little more than 10 percent As to why more people should agree with repealing a fictitious statute when given a filtered form we can only speculate In large part we think it reflects a form of acquies- cence among respondents who feel pressured to give an opinion when faced with a filter question such as Do you have an opinion on this or not but who cannot think of a reason for disagreeing with the statement that it should be repealed (see Bishop et al 1982)

FOREIGN AFFAIRS ISSUES

Table 6 gives the results for the two foreign affairs items which we adapted from experiments by Schuman and Presser (1978 1981) On

Table 6 Substantive Response to Questions About RussianIAmerican and ArabIIsraeli Relations by Form in Survey 111 (Spring 1979)

Issue and Resuonse Standard Form Form A

RussianIAmerican Relations Russian leaders are trying to get along with America 408 Russian leaders are trying to dominate America 592 Total 1000 (N) (463) xz=734 d f = 1p=006

ArabIIsraeli Relations Arab nations trying to defeat Israel 624 Arab nations trying to work for real peace with Israel 376 Total 1000 (N) (470) xZ=276 d f = 1 p = 096

l 2 Because of the extremely small number of respondents who gave an opinion on this issue in any single survey we have combined the four surveys where both standard and filtered forms were used and collapsed the latter forms together


the issue of RussianIAmerican relations we discover once more that a filter question can make a significant and sizable difference in the marginals For whatever reason the percentage believing that the Russian leaders are basically trying to dominate America tended to be significantly higher on the filtered form than on the standard form Similarly on the question of ArabiIsraeli relations we find a larger proportion of respondents on the filtered form who think that the Arab nations are trying to defeat Israel than on the standard form though the size of the difference is modest (about 6 percent) and only approaches statistical significance On this issue then as well as the previous one the filtered form makes it appear that respondents are more pessimistic about foreign relations So while we have evidence from just a single experiment with each item the substantive similar- ity of the two results provides us with an important conceptual rep-lication

In contrast Schuman and Presser (1981) found that filtering had no significant effect on responses to questions about the same two topics In their filtering experiments however these questions were worded in an agreeidisagree formatI3 Furthermore because no filter question was used in their experiment with the forced-choice version of either item direct comparisons with the results of the present study cannot be made

Table 7 shows the effects for the two different questions which were asked about the SALT issue In each case filtering has a substantial impact on the marginal~ On the first version of this issue for example we would infer that public support for continuing the SALT negotiations would be even higher had we used either form B or C of the filter instead of Form A or the standard form The variation in responses among the three filtered forms (A vs B vs C) moreover was statistically significant (x2 = 738 df = 2 p = 024) Thus it would seem on this item at least that the more strongly worded the filter the greater the effect on substantive responses For it was Forms B and C that produced the greatest increments in DK re-

l 3 There was also a minor difference in the wording of the filter used in their experiments and ours In the experiments by Schuman and Presser it read Do you have an opinion on that whereas in our study it reads Do you have an opinion on this or not In addition there are differences in the lead-in statements to the items In the Schuman-Presser studies respondents were told Here are some questions about other countries Not everyone has opinions on these questions [emphasis added] If you do not have an opinion just say so [emphasis added] The Russian leaders (see Schuman and Presser 1981116) In our survey the lead-in statement was much shorter Here are some questions about foreign affairs So there were potentially important variations between their experiments and ours in the degree to which respondents were encouraged to give an opinion on this issue which also makes the two experiments somewhat less comparable


Table 7 Substantive Response to Questions about Strategic Arms Limitation Talks (SALT) by Form and Survey

Survey 11 Survey I V Survey V (Fall 1978) (Fal l 1979) (Spring 1980)

Srd Form Form Form Srd Form Form Std Form Form Issue und Response Form A R C Form C E Form C E

SALT Negotiations if Soviets lnterferlng in African Af- fairs

Discontinue SALT until Soviets stop interfering In American affairs 324 326 247 213 - - - - - -

Continue SALT because of im~ortanceof limiting arms race 676 674 753 787 - - - - - ----

Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)

SALT I1 Agreement w ~ t h Soviet Union

Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015

sponses (see Table 2) And it is these two forms that generated the largest substantive differences on the issue

We find much the same pattern on the other version of the SALT issue But here the effect on the response distributions apparently falls in the opposite ideological direction In both Surveys IV and V public approval of the SALT agreement decreases when the filtered forms are used Furthermore in each of these experiments it is the more strongly worded filter-Form E (see Table 2)-which creates the largest difference in the substantive responses Had we used only the standard form in Survey IV for instance we would have con- cluded that a clear majority of citizens favored the agreement (571 percent) whereas with Form E we would have inferred that a majority (539 percent) was opposed to it In Survey V the contrast be- comes even sharper For there with the standard form we would think that the public was rather evenly divided on the issue (497 percent vs 503 percent) but with Form E that an overwhelming majority (638 percent) were against the agreement In either case a policy maker reading these poll results would probably make rather different judgments about public support for various policy options (cf Robin- son and Meadow 1982 Roshco 1978)

In addition we think it is worth emphasizing that the direction of the differences due to filtering on this issue depends upon the wording and format of the substantive question When presented as a one- sided statement about the SALT I1 agreement to which respondents are expected to answer either favor or oppose the effect of the


filter was to increase opposition But when offered a two-sided format which linked the SALT issue to the behavior of the Soviet Union in African affairs the addition of a filter apparently increased support for the SALT process These questions however actually represented different aspects of the SALT issue at different points in time Some would even argue that the two questions are indeed distinct from one another We should not therefore expect a filter to give rise to similar substantive implications on each question since that so clearly depends upon the framing of the issue

Discussion and Conclusions

What have we learned First of all we know that the wording of a filter can make a significant difference in the percentage of DK responses elicited by an item One which asks respondents how interested they are in an issue or how much they have thought or read about it will generally screen out more people than one which asks simply whether they have an opinion And the more abstract or remote the issue the greater this question wording effect will tend to be

Furthermore we now know that the content of an issue can have a substantial independent effect on the DK response When the issue is a highly familiar or emotive one such as affirmative action for blacks the addition of a filter question will probably have little impact on the DK proportions But the more obscure or less familiar the topic the greater will be the increase in DKs produced by adding a filter-ie other things being equal in particular the degree of ambivalence evoked by an item

More important we have learned that filtering can in some instances dramatically affect the conclusions we would draw about the distribution of public opinion on an issue Indeed such effects may occur more often than has previously been suspected the circumstances under which they will emerge however remain somewhat puzzling On several issues the effects appear to be related to the remoteness of the issue the strength of the filter and the size of the DK increment it produces (eg SALT 11) but on other topics (eg Cuba Turkey) no such relation is evident

Nor can we explain the variation among issues by controlling a seemingly relevant third variable like education In most cases education does correlate significantly with giving a DK response to an issue especially when a filter question is asked (cf Schuman and Presser 1981137-41) And while it also tends to correlate with the substantive responses given to many items the relationship is often not consistent On the SALT I1 issue for instance education was


strongly associated with giving a DK response to a filter question (average gamma = 42) but it was negligibly related to substantive responses on both the filtered forms (average gamma = - 07) and the standard form (average gamma = -01) So it could not possibly account for the differences in the marginal distributions for this item (Table 7) This crude demographic variable cannot therefore provide us with an overall explanation though it may help account for effects on certain issues

What else can The answer we believe lies in reconceptualizing how respondents think about the questions we ask them Consider again the issue of the SALT I1 agreement It is not exactly a topic we would expect many respondents to have thought about an expecta- tion that is strongly confirmed by the large percentage of respondents who acknowledged that they had not thought or read much about it (see Table 2) But what do such respondents do in the absence of an explicit filter question Some of course volunteer a dont know but most do not We suspect that the majority of them seize upon whatever words or cues in the item seem most familiar they then respond in terms of the information that is most accessible to them in memory about those stimulus cues On the SALT I1 item for example the most plausible cue would be the Soviet Union which in turn would evoke such associations as the Russians and Communism It is toward these symbols then that the less informed respondent would be expressing his or her opinion on the standard form as opposed to the policy issue of the SALT I1 agreement So while the issue per se may not be familiar to them the attitude object-the Soviet Union-would be And that too perhaps is why we may in some instances observe a sizable number of DK responses to an issue asked in filtered form and yet find little or no effect on the substantive responses namely because respondents are answering the question on both forms largely in terms of the familiarity of the object con- tained in the item (eg Cuba) rather than the issue itself-ie re-establishing diplomatic relations with Cuba Such items may in other words be multidimensional

Thus the effect of a filter on substantive responses may be a function not only of opinions about the issue or situation described in the item (eg resuming arms shipments to Turkey) but also of affect toward the object(s) included in it (Turkey) Only by probing respondents about what they had in mind as they answered the question can we tell what was being expressed the attitude toward the object the attitude toward the policy situation or some combina- tion thereof And only then will we learn what it is that a filter does or does not screen out and with what consequences

EFFECTS OF FILTER QUESTIONS

Appendix

Issue Questions in Standard Form

1 Now some people think that the government in Washington is trying to do too many things that should be left to individuals and private businesses Others disagree and think that the government should do even more to solve our countrys problems

What is your o p i n i o n 4 0 you think the government is trying to do too many things that should be left to individuals and private businesses or do you think the government should do even more to solve our countrys problems

2 Some people say that in order to make up for past discrimination black people should be given preferential treatment in getting jobs and education Others disagree and say that a persons ability should be the main consideration

What is your o p i n i o n 4 0 you think that in order to make up for past discrimination black people should be given preferential treatment in getting jobs and education or do you think that a persons ability should be the main consideration

3 Some people say that the government in Washington should cut taxes even if it means that a lot of public employees would lose their jobs

Do you agree or disagree with this idea 4 Some people feel that there should be a government insurance plan which would

cover all medical and hospital expenses Others feel that medical expenses should be paid by individuals and through private insurance like Blue Cross

What is your f e e l i n g 4 0 you think there should be a government insurance plan which would cover all medical and hospital expenses or do you feel that medical expenses should be paid by individuals and through private insurance like Blue Cross

5 Some people say that the 1975 Public Affairs Act should be repealed Do you agree or disagree with this idea 6 Some people say that the United States should re-establish diplomatic relations

with Cuba Do you favor or oppose this idea 7 Some people say that the United States should resume arms shipment to Turkey Do you approve or disapprove of this idea 8 Some people say that we should refuse to continue the Strategic Arms Limitation

Talks with the Soviet Union until they stop interfering in African affairs Other people think that limiting the arms race is so important that we should continue the talks despite the involvement of the Soviet Union in African affairs

What is your feeling-do you think we should refuse to continue the Strategic Arms Limitation Talks until the Soviet Union stops interfering in African affairs or do you think that limiting the arms race is so important that we should continue the talks despite the involvement of the Soviet Union in African affairs

9 Some people feel that the Russian leaders are basically trying to get along with America Others think that they are basically trying to dominate America

What is your f e e l i n g 4 0 you think the Russian leaders are basically trying to get along with America or do you think that they are basically trying to dominate America

10 Some people feel that the Arab nations are basically trying to defeat Israel Others think that they are trying to work for a real peace with Israel

What is your f e e l i n g 4 0 you think that the Arab nations are trying to defeat Israel or do you think that they are trying to work for a real peace with Israel

11 Some people favor an amendment to the Constitution which would require the federul government to balance its budget every year

What is your f e e l i n g 4 0 you favor or oppose an amendment which would require the federal government to balance its budget every year

12 Recently the United States and the Soviet Union reached an agreement on a new strategic arms limitation treaty usually called SALT 11

What about you-do you favor or oppose this new SALT agreement


References Bishop George F Robert W Oldendick and Alfred J Tuchfarber

1982 Effects of presenting one versus two sides of an issue in survey questions Public Opinion Quarterly 4669-85

Bishop George F et al 1980 Pseudo-opinions on public affairs Public Opinion Quarterly 44198-209

Campbell Angus Philip E Converse et al 1960 American National Election Study 1960 (ICPSR 7216) Ann Arbor Inter-

University Consortium for Political and Social Research university of Michi- gan

Center for Political Studies 1970 American National Election Study 1970 (ICPSR 7298) Ann Arbor Inter-

university Consortium for Political and Social Research university of Michi- gan

Converse Philip E 1964 The nature of belief systems in mass publics Pp 206-261 in David E Apter

(ed) The Quantitative Analysis of Social Problems Reading Mass Addison-Wesley

1970 Attitudes and non-attitudes continuation of a dialogue Pp 168-189 in Ed- ward R Tufte (ed) The Quantitative Analysis of Social Problems Reading Mass Addison-Wesley

Coombs Clyde H and Lolagene C Coombs 1977 Dont know item ambiguity or respondent uncertainty Public Opinion

Quarterly 40497-514 Faulkenberry G David and Robert Mason

1978 Characteristics of nonopinion and no opinion response groups Public Opinion Quarterly 42533-43

Klecka William R and Alfred J Tuchfarber 1978 Random digit dialing a comparison to personal surveys Public Opinion

Quarterly 42 105-14 Politival Behavior Program and the Survey Research Center

1964 American National Election Study 1964 (ICPSR 7235) Ann Arbor Mich Inter-University Consortium for Political and Social Research University of Michigan

Robinson John P and Robert G Meadow 1982 Polls Apart Cabin John MdWashington DC Seven Locks Press

Roshco Bernard 1978 The polls polling on Panama-si dont know hell no Public Opinion

Quarterly 4231-62 Schuman Howard and Stanley Presser

1978 The assessment of no opinion in attitude surveys Pp 241-275 in Karl Schuessler (ed) Sociological Methodology 1979 San Francisco Jossey- Bass

1981 Questions and Answers in Attitude Surveys Experiments on Question Form Wording and Context New York Academic Press

You have printed the following article

Effects of Filter Questions in Public Opinion SurveysGeorge F Bishop Robert W Oldendick Alfred J TuchfarberThe Public Opinion Quarterly Vol 47 No 4 (Winter 1983) pp 528-546Stable URL

httplinksjstororgsicisici=0033-362X2819832429473A43C5283AEOFQIP3E20CO3B2-Y

This article references the following linked citations If you are trying to access articles from anoff-campus location you may be required to first logon via your library web site to access JSTOR Pleasevisit your librarys website or contact a librarian to learn about options for remote access to JSTOR

[Footnotes]

2 Random Digit Dialing A Comparison to Personal SurveysWilliam R Klecka Alfred J TuchfarberThe Public Opinion Quarterly Vol 42 No 1 (Spring 1978) pp 105-114Stable URL

httplinksjstororgsicisici=0033-362X2819782129423A13C1053ARDDACT3E20CO3B2-3

References

Effects of Presenting One Versus Two Sides of an Issue in Survey QuestionsGeorge F Bishop Robert W Oldendick Alfred J TuchfarberThe Public Opinion Quarterly Vol 46 No 1 (Spring 1982) pp 69-85Stable URL

httplinksjstororgsicisici=0033-362X2819822129463A13C693AEOPOVT3E20CO3B2-D

Pseudo-Opinions on Public AffairsGeorge F Bishop Robert W Oldendick Alfred J Tuchfarber Stephen E BennettThe Public Opinion Quarterly Vol 44 No 2 (Summer 1980) pp 198-209Stable URL

httplinksjstororgsicisici=0033-362X2819802229443A23C1983APOPA3E20CO3B2-9

httpwwwjstororg

LINKED CITATIONS- Page 1 of 2 -

NOTE The reference numbering from the original has been maintained in this citation list

Dont Know Item Ambiguity or Respondent UncertaintyClyde H Coombs Lolagene C CoombsThe Public Opinion Quarterly Vol 40 No 4 (Winter 1976-1977) pp 497-514Stable URL

httplinksjstororgsicisici=0033-362X281976242F19772429403A43C4973A22KIAOR3E20CO3B2-V

Characteristics of Nonopinion and No Opinion Response GroupsG David Faulkenberry Robert MasonThe Public Opinion Quarterly Vol 42 No 4 (Winter 1978) pp 533-543Stable URL

httplinksjstororgsicisici=0033-362X2819782429423A43C5333ACONANO3E20CO3B2-U

Random Digit Dialing A Comparison to Personal SurveysWilliam R Klecka Alfred J TuchfarberThe Public Opinion Quarterly Vol 42 No 1 (Spring 1978) pp 105-114Stable URL


The Polls Polling on Panama-Si Dont Know Hell NoBernard RoshcoThe Public Opinion Quarterly Vol 42 No 4 (Winter 1978) pp 551-562Stable URL

httplinksjstororgsicisici=0033-362X2819782429423A43C5513ATPPOPD3E20CO3B2-U

httpwwwjstororg




about political and social issues they found that a filter will generally increase the percentage of dont know (DK) or no opinion re-sponses to an item by about 20-25 percent Furthermore their analysis indicates that these increments in DK responses do not depend on the content of an issue and they did not find any relationship between the percentage of DKs which were volunteered on an issue in the absence of an explicit filter question (the standard form) and the percentage of respondents removed by adding one Yet they did discover that the wording of a filter can make a substantial difference in the percentage of respondents who say they have no opinion A filter that is which emphasizes the frequency or ac- ceptability of not having an opinion on an issue will screen out many more people than one which does not

Surprisingly perhaps their research also suggests that in most instances filtering will have little impact on the distribution of substantive responses to an item once the DKs are excluded from the analysis The use of filter questions in their experiments moreover did not appear to have any significant influence on the magnitude of association between substantive responses to issues and such demographic variables as age sex and education A researcher would in other words draw essentially the same conclusion about the nature and determinants of public opinion on an issue on the basis of either a filtered or an unfiltered form (see Schuman and Presser 1981 126-28 137-41)

Independently the authors have investigated many of the same effects of filtering in a series of experiments and replications Here we will try to generalize and extend the work of Schuman and Presser at least in part using a different set of issues and a somewhat greater variety of filter questions However we should make it clear that our investigation was not designed initially as a replication of theirs it originated rather in a broader study of changes in the wording and format of questions on domestic and foreign policy issues in the American National Election Studies (Bishop et al 1982) Our analysis then is more in the nature of an independent probe of the same conceptual territory though it does include a partial replication of two split-ballot experiments from their project

We use DK or DK response throughout the paper as a generic term for various types of nonsubstantive responses On a standard (unfiltered) item this would indicate a volunteered dont know whereas on a filtered form it would refer to saying no to such questions as Do you have an opinion on this or not and Have you thought much about this issue Where appropriate we distinguish between these different meanings


Research Design

SURVEYS I A N D I1





























- - -

































SURVEY I11


SURVEYS IV AND V












































- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg




Research Design

SURVEYS I A N D I1





























- - -

































SURVEY I11


SURVEYS IV AND V












































- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg
























- - -

































SURVEY I11


SURVEYS IV AND V












































- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg



- - -

































SURVEY I11


SURVEYS IV AND V












































- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg







SURVEY I11


SURVEYS IV AND V












































- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg











































- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg


































- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg





























- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg









- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg



- - - - - - - -

----

--









Total 1000 1 0 0 ~ 1000 1000 1000 1000 1000 1000 ( N ) (153) (330) (303) (609) (310) (535) (356) (523)

x=699df=I X2=1028df=1 x2=171df=1 x2=228df=1



x2=0 74 d f= l x2=7 58 d f=I p = 388 p = 005















Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg













Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg







Total 1000 1000 (N1 (368) (182)





















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg















Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg










Total 1000 1000 1000 1000 (l4 (284) (225) (187) (195)


Favor - - - - 571 505 461 497 458 362 Oppose - - - - 429 495 539 503 542 638

Total 1000iimi1000 100010001000 (N) (290) (233) (196) (313) (217) (178)

x2=602 df= 2 x2= 839 df = 2 p = 049 p = 015
















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg















Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg








Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg




Appendix
















































[Footnotes]



References





httpwwwjstororg











httpwwwjstororg





























[Footnotes]



References





httpwwwjstororg











httpwwwjstororg







[Footnotes]



References





httpwwwjstororg











httpwwwjstororg











httpwwwjstororg



Documents

Effects of Filter Questions in Public Opinion