AAPOR 2016 Reshaping the Research Landscape: Public Opinion and Data Science

Respondent Reactions to a

Sexual Victimization Survey in a

Multi-mode Pilot Study

Suzanne Kaasa, Ph.D.

Darby Steiger, M.A.

Leanne Heaton, Ph.D.

David Cantor, Ph.D.

Rose McAloon, M.A.

AAPOR 2016

Sexual Victimization: What are the Stats?

AAPOR 2016

Pilot Study to Test Alternative Survey Methods

Phone (CATI) vs. laptop (ACASI)

Over 10,000 females from 5

locations across the country

AAPOR 2016

Challenges to Sensitive Survey Data Collection

(e.g., Jaffe et al., 2015; Baldor, 2013)

AAPOR 2016

Methodological Precautions for Sensitive Surveys

Train interviewers to be sensitive and identify

respondent distress

Assess respondent reactions through self-report

Provide sexual victimization resources to respondents

AAPOR 2016

Prior Research on Respondent Reactions

Sensitive surveys produce more self-reported

positive reactions than negative reactions.

Most respondents who report some feelings of

harm do not regret taking the survey.

(e.g., McClinton, 2015; Newman et al., 2006)

The Questions

AAPOR 2016

Research Questions

1. How likely were respondents to experience positive and negative

reactions, and regret participation in the study?

2. Does survey methodology affect respondent reactions?

3. Are interviewers able to identify respondent distress?

4. Do respondents find sexual victimization resources helpful?

(e.g., McClinton, 2015; Newman et al., 2006)

The Measures

AAPOR 2016

• You experienced intense emotions while

completing the survey.

• The research made you think about things you

didn’t want to think about.

• The research raised unpleasant emotional

issues for you that you had NOT expected.

Measuring Reactions to the Survey

• You believe you have been able to help others

by participating in this study.

• You were glad to have had the opportunity to

participate.

• You feel you gained something positive from

participating.

Negative Reactions Scale Positive Reactions Scale

(No) Regret Now that you know what the survey is about, you would have made the same choice to participate.

Chronbach Alpha = .80 Chronbach Alpha = .77

AAPOR 2016

Strongly Disagree Disagree Neutral Agree Strongly Agree

1 2 3 4 5

Higher Mean Scores

More negative reactions

More positive reactions

Less regret

Response Scale

AAPOR 2016

Did you use any of the resources that

were shared with you?

[If yes] Did you find these resources

helpful?

Resources

AAPOR 2016

Sample Size

Interview Re-Interview

n = 3,053 n = 359

n = 4,984 n = 503

The Answers

AAPOR 2016

Emotional Reactions to the Survey

22.7

48.3

25.3

32.7

83.2

69.1

79.3

77.3

51.7

74.7

67.3

16.8

30.9

20.7

(No regret) Would have made the same choice to participate

Gained something positive from participating

Glad to have had the opportunity to participate

Believe you have been able to help others

Raised unpleasant emotional issues

Thought about things you didn’t want to think about

Experienced intense emotions

Percent of Respondents Reporting Emotional Reactions (ACASI)

No Yes

Neg

ativ

e P

osi

tive

*

*

*

AAPOR 2016

Emotional Reactions to the Survey

*

*

* 13.1

42.3

21.6

20.8

79.4

67.5

79.1

86.9

57.7

78.4

79.2

20.6

32.5

20.9

(No regret) Would have made the same choice to participate

Gained something positive from participating

Glad to have had the opportunity to participate

Believe you have been able to help others

Raised unpleasant emotional issues

Thought about things you didn’t want to think about

Experienced intense emotions

Percent of Respondents Reporting Emotional Reactions (CATI)

No Yes

Ne

ga

tive

Po

sitiv

e

AAPOR 2016

Self-reported Negative Reactions by Interviewer Observed Distress

3.2

3.7

2.6 2.6

ACASI CATI

Signs of Distress No Signs of Distress

Interviewer Observations of Respondent Distress

Resp

on

dent S

elf-r

ep

ort

ed

Nega

tive

Rea

ctio

ns

* *

AAPOR 2016

Reactions to Sexual Victimization Resources

86.5%

12.9% 0.6%

Reactions to Resources (ACASI)

Did not use

Did use and foundhelpful

Did use and didn't findhelpful

94.5%

5.1% 0.4%

Reactions to Resources (CATI)

Did not use

Did use and foundhelpful

Did use and didn't findhelpful

AAPOR 2016

Discussion

1. Respondents reported more positive reactions than negative reactions

or regret.

2. Interviewers were able to accurately assess and respond to distress.

3. Although resources were rarely used, they were found to be helpful.

4. Survey mode did not affect respondent reactions or interviewer ability

to detect signs of distress. Mode did affect use of sexual victimization

resources.