Researching Sex Offenders: A Workshop on Conceptualizing and Implementing Sex Offender Research...

Researching Sex Offenders: A Workshop on Conceptualizing and Implementing Sex

Offender Research Projects

R. Karl HansonPublic Safety Canada

Presentation at the 13th Annual Conference of the NYATSA, Saratoga Springs, NY, May 14, 2008

Big Questions

• Impact of criminal justice and social policy interventions (e.g., community notification, residency restrictions, civil commitment, risk assessment)

• Assessment of reduced risk in high risk offenders

• Active components in sexual offender treatment

• Social policy for prevention on onset

Evaluation of

Sex Offender Treatment Programs

Collaborative Outcome Data Committee

• Formed in 1997

Goals:– Define standards for research on treatment

outcome for sexual offenders (develop consensus)

– Organize existing sexual offender outcome studies

– Promote high quality evaluations

CODC contributors

• Anthony Beech• Darren Bisshop• Guy Bourgon• Dawn Fisher• R. Karl Hanson• Andrew Harris• Calvin Langton• Roxanne Lieb

• Janice Marques• Michael Miner• William Murphy• Michael Seto• Vernon Quinsey• David Thornton• Pamela Yates

CODC Study Quality Guidelines

• Structured rating scale

• Definition of study quality – “…judgement of minimal bias can be made with high

confidence.”

• 20 items (plus 1 additional item rated for cross-institutional designs)– Items fall under 7 categories– Items assess either confidence OR bias (including direction of

bias)

I) Administrative control of independent variables

• Defining treatment (confidence)

• Defining comparison group (confidence)

• Miscellaneous incidental factors (bias)

II) Experimenter expectancies

• Experimenter involvement (bias)

• Blinding in data management (bias)

III) Sample size

• Sample size of treatment (confidence)

• Sample size of comparison (confidence)

• Sample size of institutions (confidence) (for cross-institutional designs only)

IV) Attrition

• Subject selection (bias)

• Program attrition (bias)

• Intent-to-treat (bias)

• Attrition in follow-up (bias)

V) Equivalency of groups

• A priori equivalency of groups (bias)

• Adequacy of search of differences (confidence)

• Findings on group differences (bias)

VI) Outcome variables

• Length of follow-up (confidence)

• Validity/reliability of recidivism information (confidence)

• Equivalency of follow-up (bias)

VII) Correct comparison conducted

• Data dredging (confidence)

• Effectiveness of statistical controls (confidence)

• Computation of least bias comparison (bias)

Global Rating• All items considered in making overall

judgment of bias and confidence – Same three-point scales as individual

items

• Bias and confidence are considered separately

Global Rating Categories• Strong

– High confidence AND negligible bias

• Good– High confidence and some bias, OR– Some confidence and negligible bias

• Weak– Some confidence and some bias

• Reject– Little confidence, OR– Considerable bias

Reliability Study 1: Students

• 2 senior undergrad students

• Approximately one week training (8 practice studies)

• 10 real studies rated independently

Results: Global Ratings

• Overall 9/10 (ICC = 0.95)

• Global confidence 10/10 (ICC = 1.00)

• Global bias 9/10 (ICC = 0.69)• Direction of bias 7/10

Reliability Study 2: Experts

• 12 Experts in sex offender research evaluation• No training on guidelines• 10 hypothetical studies ranging in quality • Rated 1-6 studies each (3 ratings per study)

Expert Raters

• Guy Bourgon• Andrew Harris• Grant Harris• Niklas Langstrom• Roxanne Lieb• Ruth Mann

• Robert McGrath• William Murphy• Vernon Quinsey• Marnie Rice• David Thornton• Pamela Yates

Expert Reliability

– Some agreement on individual items

– No agreement on global ratings

www.publicsafety.gc.ca

• Collaborative Data Outcome Committee. (2007). Sex offender treatment outcome research: Guidelines for Evaluation (CODC Guidelines). Part 1: Introduction and overview. Corrections User Report No 2007-02. Ottawa: Public Safety Canada.

• Collaborative Data Outcome Committee. (2007). The Collaborative Outcome Data Committee’s Guidelines for the evaluation of sexual offender treatment outcome research. Part 2: CODC Guidelines. Corrections User Report No 2007-03. Ottawa: Public Safety Canada.