Embed Size (px)
Citation preview
1
Surveys, Field Research & Secondary Data
Research Methods
2
Week 6: Lecture outline
1. Survey data collection methods
2. Types of Surveys
3. Field Research
4. Secondary Data
5. Class Round on Methods
Topics appropriate to survey research Counting crime – asking people about
victimization counters problems of data collected by police
Self-reports – dominant method for studying the etiology of crime
» Frequency/type of crimes committed
» Prevalence (how many people commit crimes) committed by a broader population
3
Topics appropriate to survey research Perceptions and attitudes – to learn how
people feel about crime and CJ policy
Policy proposals – search for ways to respond to crime that are supported by the general public
Targeted victim surveys – used to evaluate policy innovations & program success
Other evaluation uses – e.g., measuring community attitudes, citizen responses, etc.
4
Open- and close-ended questionsOpen-ended – respondent is asked to provide his or her
own answer» What were your initial expectations of the program?
________________________________________________
Closed-ended – respondent selects an answer from a list or scale» On a scale of 1 to 5, where 1 is low and 5 is high, rate your initial
expectations of the program.
5
1 2 3 4 5
Low High
Open versus closed? Choice between these question types will
depend on the aim of your research, the study design, the population being study, etc.
Advantages and disadvantages to both.
Possible gains in reliability (closed-ended) balanced with possible losses in breadth of information and validity.
One strategy is to combine open- and close-ended questions.
6
Designing questions… Make items clear – avoid ambiguous questions; do not
ask “double-barreled” questions
Short items are best – respondents like to read and answer a question quickly
Avoid negative items – leads to misinterpretation
Avoid biased items and terms – do not ask questions that encourage a certain answer
Closed question response categories must be exhaustive and mutually exclusive.
7
Survey data collection methods
1. Self-administered
2. In-person interview
3. Telephone interviews
Focus: Australia’s National Crime Victimization Survey Methodology
8
Australia’s Crime Victimization Survey The MPHS was conducted as a supplement to the monthly LFS. Each month one eighth of the dwellings in the LFS sample were rotated out of the
survey. In 2012-13, all of these dwellings were selected to respond to the MPHS each month . In these dwellings, after the LFS had been fully completed for each person in scope and
coverage, a person aged 15 years and over was selected at random (based on a computer algorithm) and asked the various MPHS topic questions in a personal interview.
If the randomly selected person was aged 15–17 years, permission was sought from a parent or guardian before conducting the interview. If permission was not given, the parent or guardian was asked the crime questions on behalf of the 15–17 year old.
Questions relating to sexual assault, alcohol or substances contributing to the most recent physical or face-to-face threatened assault were not asked of proxy respondents. Only those persons aged 18 years and over were asked questions on sexual assault.
Data was collected using Computer Assisted Interviewing, whereby responses were recorded directly onto an electronic questionnaire in a notebook computer, usually during a telephone interview.
9
A Link to the USA National Crime Victimization Survey http://www.bjs.gov/index.cfm?ty=dcdet
ail&iid=245
10
UCR and NCVS UCR data are based on reported criminal
acts (offender characteristics) NCVS data based on individuals actually
victimized (characteristics of victims)
Assessment of NCVS Document a massive amount of crime that goes
unreported Underestimate crime rate Insignificant crimes tend to be forgotten Victims of several crimes may also forget about all the
crimes Females do not report victimization if her abuser live
in the same household Whites and college graduates are more likely to report
being victimized
NCVS respondents are interviewed every six months (7 interviews)
Reported victimization rates usually decease with each interview (awareness of victimization)
Overestimation of some crimes Respondents might mistakenly interpret some
noncriminal events as crimes “Telescoping “ effect
Assessment of NCVS
Created to complement UCR and NCVS Interviews or questionnaires Demonstrate the prevalence of offending (the
proportion of respondents who have committed a particular offence)
Incidence of offending (the average number of offences per person in the study)
Self-reports data
Samples for self-reports Target: Adult inmates of jails and prisons:
PREA findings on sexual assault Target: Adolescents, usually high school
students The most important finding: delinquency is very
common Middle-class youth commit as much crime as
working-class youth Testing criminological theories: Travis Hirschi’s
Research Study testing Control Theory( compared to subculural & Strain)
Assessment of self-report studies
Focus on minor and trivial offenses (truancy, running away from home, minor drug and alcohol use)
Although recent studies (NYS) asked subjects about rape and robbery
Respondents might not to tell the truth (reliability issues)
If respondents lie…. Self-report data can be checked against police records,
school records, interviews with teachers and parents The use of, or threat of , polygraph validation (20%
change their initial responses when threatened with a “lie detector”)
Subsequent interviewing of subjects permits probing regarding the details and context of acts
Use of “lie scales”
Focus: Self Reported Substance Use Among Detainees in Australia http://www.aic.gov.au/media_library/co
nferences/evaluation/mcgregor.pdf
http://192.190.66.70/documents/7/E/8/%7B7E8D4A8E-A5AF-4D3B-8821-ED8A1BA489B6%7Drpp93.pdf
19
UCR, NCVS, and self-reports None of the three is perfect For the best estimates of the actual number of crimes,
NCVS data are preferable For the best estimates of offender characteristics, self-
reports and NCVS are preferable UCR are superior for understanding the geographical
distribution of crime
Self-administered questionnaires Can be home-delivered
» Researcher delivers questionnaire to home of sample respondent, explains the study, and then comes back later
Mailed (sent and returned) survey is most common
» Researchers must reduce the trouble it takes to return a questionnaire
Goal is High Response Rate21
Computer-based self-administration
Via email, website Issues:» representativeness»mixed in with, or mistaken for, spam» requires access to Web» sampling frame?
22
Week 7: More Survey Research What we will Cover: 1.Interviewing techniques and Focus Groups2. A Comparison of Survey Methods: phone/ computer( survey monkey), mail, and in-person
3. Field Research: The good, the bad, and the ugly4. Agency Records, Content Analysis and Secondary Data
23
In-person interview survey
Typically achieve higher response rates than mail surveys (80-85% is considered good)
Demeanor and appearance of interviewer should be appropriate; interviewer should be familiar with questionnaire and ask questions precisely
When more than one interviewer administers, efforts must be coordinated and controlled
Practice interviewing
24
Specialized interviewing
•Two variations:» General interview guide – less structured, lists
issues, topics, questions you wish to cover; no standardized order
» Standardized open-ended interview – more structured, specific questions in specific order; useful in case studies, retrieves rich detail in responses
25
Telephone surveys 94% of all households now have telephones Random-Digit Dialing» Obviates unlisted number problem» Often results in business, pay phones, fax lines
Saves money and time, provides safety to interviewers, more convenient
may be interpreted as bogus sales calls; ease of hang-up
26
Comparison of the three methods
Self-administered questionnaires are generally cheaper, better for sensitive issues than interview surveys
Using mail – local and national surveys are same cost Interviews – more appropriate when respondent
literacy may be a problem, produce fewer incompletes, achieve higher completion rates
Validity low in survey research; reliability high Surveys are also inflexible, superficial in coverage
27
Tips on self-report items
Convince subjects you will guarantee confidentiality and anonymity
Minimize possible social undesirability you are asking respondents to admit
Phrase questions in non-judgmental manner
Bear in mind “fading memory” when setting time frame
28
Focus groups 12-15 people brought together to engage in guided
group discussion of some topic( e.g. addicts& recovery)
Members are selected to represent a target population, but cannot make statistical estimates about population
Most useful when precise generalization to larger group is not necessary
May be used to guide interpretation of questionnaires following survey administration
Examples: Drug addicts and reentry, homeless sex offenders, teenagers and sexting
29
30
Getting Out
Field Research
Field Research•Field research encompasses two different methods of obtaining data:
•Direct observation
•Asking questions
•May yield qualitative and quantitative data
•Often no precisely defined hypotheses to be tested
•Can be used to make sense out of an ongoing process
Topics Appropriate to Field Research
•Gives comprehensive perspective – enhances validity
•Go directly to phenomenon, observe it as completely as possible
•Especially appropriate for topics best understood in their natural setting
•How street-level drug dealers distinguish customers
•Studies of ‘vice’, e.g., prostitution and drug-use.
•Aspects of physical settings, Disney World, social control
Various Roles of the Observer (Gold, 1969)
•Complete participant – participates fully; true identity and purpose are not known to subjects
•E.g., posing as a bar patron; becoming a police officer; or corrections worker
•Participant-as-observer – make known your position as researcher and participate with the group
•E.g., study of active drug users; Julie Mueler and the Guardian Angels
•Observer-as-participant – make known your position as a researcher; do not actually participate
•E.g., Observational study of police patrol—”ride-along” ; research on gangs
•Complete observer – observes without becoming a participant
•E.g., court observation, Chicago neighbourhood study( windshield studies)
Observer status Be aware of, and document role of researcher
(extent of participation)
Be aware that all observation is subjective.
Be aware of the possible “effect of participation”.
Be aware of often competing ethical and ‘scientific’ values related to all observational studies.
34
Asking Questions
•Field research is often a matter of going where the action is and simply watching and listening
•Also a matter of asking questions & recording answers
•Field research interviews are much less structured than survey interviews
•Ideally set up and conducted just like a normal, casual conversation
Preparing for the Field
•Access to formal organizations
•Find a sponsor, write a letter to executive director, arrange a phone call, arrange a meeting
•Access to subcultures
•Find an informant (e.g., person who works with offenders), use that person as your “in”
•Snowball sampling is useful as informant identifies others, who identify others, etc.
Sampling in Field Research
•Controlled probability sampling used rarely; purposive sampling is common
•Bear in mind two stages of sampling:
•To what extent are the situations available for observation representative of the general phenomena you wish to describe and explain?
•Are your actual observations within those total situations representative of all observations?
Recording Observations
•Note taking, tape recording when interviewing and when making observations
•Videotaping or photographs can make records of “before” and “after” some physical design change
•Field notes – observations are recorded as written notes, often in a field journal; first take sketchy notes and then rewrite your notes in detail
•Structured observations – observers mark closed-ended forms, which produce numeric measures
Linking Field Observations and Other Data
•Useful to combine field research with surveys or data from official records
•Baltimore study of the effects of neighborhood physical characteristics on residents’ perceptions of crime problems (Taylor, Shumaker, & Gottfredson, 1985)
•Perceptions – surveys;
•Physical problems – (1) observations, (2) actual population and crime information - census data & crime reports from police records
Strengths and Weaknesses of Field Research
•Provides great depth of understanding
•Flexibility (no need to prepare much in advance)
•More appropriate to measure behavior than surveys
•High validity; quant. measures – incomplete picture
•Low reliability – often very personal
•Generalizability – personal nature may produce findings that may not be replicated by another
•Precise probability samples can’t normally be drawn
41
Other Sources to Consider
Agency Records, Content Analysis and Secondary
Data
Secondary Data•Data from agency records – agencies collect a vast amount of crime and CJ data
•Secondary analysis – analyzing data previously collected
•Content analysis – researchers examine a class of social artifacts (typically written documents)
Topics Appropriate for Agency Records
•Most commonly used in descriptive or exploratory studies
•Content analysis often center on links between communication, perceptions of crime problems, individual behavior, CJ policy
Types of Agency Records
•Published Statistics – gov’t organizations routinely collect and publish compilations of data (e.g., ABS, BOCSAR, AIC) often available in libraries and online
•Nonpublic Agency Records – agencies produce data not routinely released (e.g., police departments, courthouses, correctional facilities)
•New Data Collected by Agency Staff – collected for specific research purposes; less costly + more control
Units of Analysis in Criminal Justice Data
Criminal Activity•Incidents•Crimes violated•Victims•Offenders
Court Activity•Defendants•Filings•Charges and Counts•Cases•Appearances•Dispositions•Sentences
Apprehension•Arrests•Offenders•Charges•Counts
Corrections•Offenders•Admissions•Returns•Discharges
Sources of Reliability and Validity Problems
•Virtually all CJ record keeping is a social process – “social production of data”
•Records reflect decisions made by CJ personnel as well as actual behavior by juveniles and adults
•Discretion factors in to recordkeeping
•CJ organizations are more interested in keeping track of individual cases than in examining patterns
•Potential for clerical errors due to volume of data
Content Analysis
•Systematic study of messages – can be applied to virtually any form of communication
•Decide on operational definitions of key variables
•Decide what to watch, read, listen to & time frame
•Analyze collected data
•Well suited to answer “who says what, to whom, why, how, and with what effect?”
Aspects of Sampling and Coding in Content Analysis 2
•Reminders:
•Remember operational definition of variables, and their mutually exclusive & exhaustive attributes
•Pretest coding scheme
•Assess coding reliability via intercoder reliability method and test-retest method
Secondary Analysis•Sources – websites, libraries
•AIC, http://www.aic.gov.au/
•BOCSAR, http://www.bocsar.nsw.gov.au/
•BJS, NCVS, ICPSR, NACJD
•Advantages – cheaper, faster, benefit from work of skilled researchers
•Disadvantages – data may not be appropriate to your research question; least useful for evaluation studies (which are designed to answer specific questions about specific programs)
Class Round
What’s the most appropriate method(s) for investigating your research evaluation question(s)?
