PhaseII Report Draft - Risk Resilience Researchrisk-resilience.berkeley.edu/sites/default/files/... · ! 4! Executive)Summary! Context’ The!mission!of!the!CaliforniaDepartmentof!Juvenile!Justice!(DJJ)!is!to!protectpublic!safety,!

1

08 Fall 08 Fall

1

CA-‐YASI Construct Validity To what extent do the domains measure the risk factors they’re supposed to measure?

C o n t a c t : s k e e m @ u c i . e d u ; ( 9 4 9 ) 2 9 4 -‐ 1 4 7 2

Jennifer Skeem, PhD Patrick Kennealy, PhD Isaias Hernandez, MSW

2

2

Table of Contents Executive Summary .............................................................................................................................. 4 Context ................................................................................................................................................................ 4 Objective ............................................................................................................................................................ 4 Method ................................................................................................................................................................ 5 Results & Conclusion ...................................................................................................................................... 5 Recommendations .......................................................................................................................................... 6

Rationale & Method .............................................................................................................................. 7 Evaluation overview ....................................................................................................................................... 7 How important is construct validity? ....................................................................................................... 8 Cross-‐structure analysis of construct validity ....................................................................................... 8 Study procedure .............................................................................................................................................. 9 Study participants ........................................................................................................................................... 9 Scoring the CA-‐YASI ..................................................................................................................................... 10 CA-‐YASI domains: Definitions, reliability, and inter-‐correlations .............................................. 11 Overview of criterion measures .............................................................................................................. 15

Results ................................................................................................................................................... 15 CA-‐YASI as a Whole ...................................................................................................................................... 16 Violence-‐Aggression .................................................................................................................................... 18 Attitudes .......................................................................................................................................................... 19 Social-‐Cognitive Skills ................................................................................................................................. 21 Social Influences ........................................................................................................................................... 23 Family ............................................................................................................................................................... 24 Community Stability .................................................................................................................................... 25 Education/Employment ............................................................................................................................. 27 Substance Use ................................................................................................................................................ 28 Health ............................................................................................................................................................... 30 Legal History .................................................................................................................................................. 31 Correctional Response ............................................................................................................................... 32

Conclusions .......................................................................................................................................... 34 Recommendations ............................................................................................................................. 35 System-‐wide ................................................................................................................................................... 35 Case by Case ................................................................................................................................................... 36

References ............................................................................................................................................ 38

Appendix A: Results Using Orbis Scoring System .................................................................... 43

Appendix B: Psychometrics of Criterion Measures ................................................................ 47 Brief Symptom Inventory (BSI) ............................................................................................................... 47 Communities that Care (CTC) ................................................................................................................... 47 Conduct Disorder ......................................................................................................................................... 47 Neighborhood Disorganization ............................................................................................................... 47 Family Background Questionnaire (FBQ) ............................................................................................ 48 Go/No Go ......................................................................................................................................................... 48 Head Injury (HI) ............................................................................................................................................ 48 Meanness ........................................................................................................................................................ 48 Peer Delinquent Behavior Scale (PDBS) .............................................................................................. 49

3

Psychological Inventory of Criminal Thinking Styles (PICTS) ...................................................... 49 Psychopathy Checklist, Youth Version (PCL:YV) ............................................................................... 49 Pubertal Development Scale (PDS) ........................................................................................................ 50 Revised Childhood Manifest Anxiety Scale (RCMAS) ....................................................................... 50 Substance Abuse Subtle Screening Inventory-‐A2 (SASSI-‐A2) ....................................................... 50 School Connection Scale (SCS) ................................................................................................................. 50 Social Information Processing (SIP) scale ........................................................................................... 51 Tower of London (ToL) .............................................................................................................................. 51 Wechsler Abbreviated Intelligence Scale (WASI) ............................................................................. 51

ADDENDUM: Testing domain components created post hoc by Orbis ............................ 53

4

Executive Summary

Context The mission of the California Department of Juvenile Justice (DJJ) is to protect public safety, partly by providing youth with “a range of training and treatment services” that could help them desist from crime. DJJ contracted with Orbis Partners Incorporated (“Orbis”) to adapt their Youth Assessment and Screening Instrument (YASI) to the unique needs of the DJJ population. The new assessment tool, CA-‐YASI, is used by DJJ to help structure its decision-‐making about, and treatment of, youth. Among tools currently on the market, the CA-‐YASI is appealing option for reducing risk because it ostensibly taps dynamic, or changeable, factors that reliably predict recidivism. In this court-‐mandated, independent evaluation, we test whether the CA-‐YASI is a good tool for assessing risk and informing risk reduction efforts. Our specific aims are to (1) examine the extent to which DJJ staff are able to reliably score the CA-‐YASI, (2) evaluate how well the CA-‐YASI assesses risk factors it purports to assess, and (3) assess the utility of this tool in predicting future infractions and re-‐arrest. We addressed Aim One in our first report, where we found that 60% of DJJ staff were able to score the CA-‐YASI with adequate reliably, at the total score level. We address Aim Two in the present report. Here, we use only reliable staff to assess the CA-‐YASI’s construct validity; that is, whether the CA-‐YASI domains measure the risk factors they are supposed to measure. We will address Aim Three in our future report, where we will test how well the CA-‐YASI predicts recidivism.

Objective A risk assessment can be conducted for one of two ultimate purposes – to predict recidivism or to inform recidivism reduction efforts (see Heilbrun, 1997). In selecting the CA-‐YASI to assess risk, DJJ signaled its interest in informing risk reduction efforts. The CA-‐YASI requires trained staff to integrate interview-‐ and file-‐ information to rate over 100 items that assess different aspects (including strengths) of twelve different risk factor domains– a process that requires at least 2.5 hours per case. If its ultimate purpose in assessing risk was merely to predict recidivism, DJJ could have selected a more efficient tool. Because DJJ wishes to go beyond predicting recidivism to reduce its likelihood, it is critical to assess the construct validity of the CA-‐YASI. Construct validity is the extent to which a tool measures the abstract concept(s) that it purports to measure. Theoretically, complex tools like the CA-‐YASI add value to simple measures of risk by assessing constructs that help explain the process that leads to recidivism. Specifically, they assess “criminogenic needs” like anger, poor self control, and antisocial attitudes that maintain criminal behavior. If they measure these constructs validly, then such tools can inform risk reduction efforts by specifying risk factors to

5

target in treatment and monitor during supervision. Thus, our objective is to assess the extent to which the CA-‐YASI domains assess the risk factors they are supposed to assess.

Method In this cross-‐structure analysis, we examine the relationships between the CA-‐YASI domains and well-‐validated measures of constructs that theoretically are strongly related (and not strongly related) to those domains. For each domain, we assess whether there is evidence of convergent validity (i.e., strong correlations with measures of theoretically similar constructs) and discriminant validity (i.e., weak correlations with measures of theoretically dissimilar constructs). For example, if it is valid, the CA-‐YASI “Attitudes” domain should be more strongly associated with a validated measure of criminal thinking styles (PICTS) than a validated measure of general intelligence (the WASI). Simply put, if the CA-‐YASI Attitude scale actually assesses procriminal attitudes, a youth’s score on that scale will tell us more about his level of criminal thinking than his level of intelligence. This study required two assessments of each youth: a CA-‐YASI assessment by a reliable DJJ staff member (i.e., CA-‐YASI Total ICC >.65), and an assessment of criterion measures by well-‐trained UC Irvine staff. So, youth were defined as eligible to participate in this study if they had been assessed by a reliable DJJ staff member within the past two months. Using IRB-‐approved procedures, UC Irvine staff recruited and assessed 237 eligible male youth at four DJJ facilities (OHC/CHAD, Preston, Southern, and Ventura). During each three-‐hour assessment, UC Irvine staff administered over 30 well-‐validated criterion measures via semi-‐structured interview, self-‐report, and performance test methods. The convergent validity measures varied by target construct (see Appendix B for evidence of reliability and validity for each measure). In contrast, across most domains, the divergent validity measures assessed somatization, head trauma, intelligence, and pubertal development. Correlational analyses were conducted to compare the strength of association between each CA-‐YASI Domain and its convergent and discriminant criterion measures. Internal consistency and inter-‐rater reliability (for the subset of reliable staff) was also calculated for each domain.

Results & Conclusion The results represent a “best case scenario” for the construct validity of the CA-‐YASI, given that this study (a) excluded the 40% of DJJ staff who could not reliably score the tool, and (b) used simple scores that sum all items in each domain, rather than Orbis-‐based scores that delete some items to maximize predictive utility (sometimes at the expense of construct validity; see Appendix A). Even under these ideal conditions, we found very limited evidence that the CA-‐YASI domains most relevant to evidence-‐based treatment for delinquent youth assess the risk factors they are meant to assess. There was evidence that the CA-‐YASI Substance Use and mental Health domains tapped their target constructs. However, the domains that ostensibly tap robust

6

individual criminogenic needs – Violence-‐Aggression, Attitudes, and Social Cognitive Skills – showed no specific associations with target constructs of anger/hostility, procriminal thinking, and executive function deficits. There was a similar lack of support for domains that ostensibly tap important contextual risk factors like antisocial peer influence (Social Influences), family problems including abuse and poor monitoring (Family), and inadequate involvement in Education/Employment. In contrast, we found strong evidence that the CA-‐YASI captures relatively static individual risk factors for crime. The scales with the strongest support, by far, were Legal History and Correctional Response, which generally distill past criminal behavior. Similarly, there was evidence that the Violence-‐Aggression domain captures general antisocial traits and behavior (more than dynamic risk factors like anger and hostile attribution bias). Like the Violence-‐Aggression Domain, CA-‐YASI Total scores were strongly associated with a well-‐validated measure of social deviance, i.e., “Factor 2” of the Psychopathy Checklist: Youth Version. This bodes well for the CA-‐YASI’s utility in predicting misbehavior, given that this scale – like simpler measures of criminal history – robustly predicts recidivism. We found little evidence for the notion that the CA-‐YASI adds value to measures that simply characterize risk by assessing constructs that help explain the process that leads to recidivism. Because the CA-‐YASI cannot specify strong risk factors to target in treatment to reduce recidivism (with the possible exception of Substance Use), its utility as a risk reduction tool seems limited.

Recommendations As a system, DJJ reportedly is using CA-‐YASI scores to inform risk reduction efforts in a narrow manner. That is, they assign youth with high Violence-‐Aggression scores to Anger Interruption Training. The results of this study suggest that youth with high Violence-‐Aggression scores are likely to have antisocial traits that place them at high risk for recidivism … but not necessarily problems with anger. If their goal is to match youth to a relevant intervention, we recommend that DJJ use a validated measure of anger to assign youth to Anger Interruption Training. As individuals, DJJ staff may be using CA-‐YASI scores to inform risk reduction efforts on a case-‐by case basis. The results of this study suggest that the CA-‐YASI may be more useful for informing intervention efforts that follow the “risk” principle than the “need” principle. That is, CA-‐YASI Total scores (like Violence-‐Aggression scores) seem to identify antisocial, high risk youth. Based on the “risk” principle of correctional intervention, higher risk youth should receive relatively intensive services and supervision. So, CA-‐YASI Total scores could be used to assign higher risk youth to more services. However, we recommend limited use of the CA-‐YASI to inform the nature of those services. The CA-‐YASI domains of Substance Use and Health can be used to flag youth in need of substance abuse and mental health services, respectively. However, the remaining domains – including those that ostensibly tap the strongest criminogenic needs for youth (e.g., antisocial attitudes

7

and peers; Simourd & Andrews, 1994)– should not be interpreted as indicators of specific treatment targets.

Rationale & Method

Evaluation overview Few ideals have greater traction in current discourse than “evidence-‐based practice.” According to this ideal, the best research informs practice that improves outcomes. Across the United States, budget cuts are fueling interest in evidence-‐based corrections. Policymakers wish to spend limited dollars wisely, in the manner that will best protect public safety. How can they do so? First, by supporting the use of well-‐validated, structured tools to assess offenders’ risk of recidivism and inform sentencing and placement decisions. Research has established that validated risk assessment tools significantly improve professionals’ ability to predict future criminal behavior. Increasingly, these tools are being applied in response to regulations that require assessments to identify “high risk” individuals for detention or “low risk” individuals for release. Second, by supporting correctional programs that (a) match the intensity of services and supervision to an offender’s level of risk, and (b) target changeable risk factors for crime (e.g., procriminal attitudes) rather than variables that are less crime-‐relevant (e.g., low self esteem). Programs that follow these principles have been shown to significantly reduce recidivism. Increased interest in evidence-‐based corrections has created an active market for risk assessment tools. Tools that measure changeable risk factors and can therefore inform risk reduction efforts are particularly appealing. A handful of companies are selling tools to corrections agencies across the nation. However, the evidence base for these tools varies considerably – some tools that are popular are not well-‐validated. The mission of the California Department of Juvenile Justice (DJJ) is to protect public safety, partly by providing youth with “a range of training and treatment services” that could help them desist from crime. In 2008, DJJ hired Orbis Partners Incorporated (“Orbis”) to customize an assessment model for DJJ youth based on the Youth Assessment and Screening Instrument (YASI) that had been primarily used and validated in juvenile probation settings. The new tool called CA-‐YASI, is used to help structure its decision-‐making about, and treatment of, youth. Among tools currently on the market, the CA-‐YASI is appealing option for reducing risk because it ostensibly taps dynamic, or changeable, factors that reliably predict recidivism. However, it has not been extensively evaluated. In this court-‐mandated, independent evaluation, we assess whether the CA-‐YASI is a good tool for making placement and release decisions (i.e., assessing risk), identifying supervision and intervention targets (i.e., identifying criminogenic needs), and capturing change in risk over time. Our specific aims are to (1) examine the extent to which DJJ staff are able to reliably score the CA-‐YASI, (2) evaluate how well the CA-‐YASI assesses risk factors it purports to assess (e.g.,

8

substance abuse; antisocial atttitudes), and (3) assess the utility of this tool in predicting future infractions and re-‐arrest. We addressed Aim One in our past Phase One report (finalized May, 2011), where we found that 60% of 78 DJJ staff were able to score the CA-‐YASI reliably, at the total score level. We address Aim Two in the present Phase Two report. Here, we use reliable staff to assess the CA-‐YASI’s construct validity, that is, whether the CA-‐YASI domains measure the risk factors they are supposed to measure. We will address Aim Three in our future Phase Three report, where we will test how well the CA-‐YASI predicts recidivism.

How important is construct validity? Most evaluations of risk assessment tools focus exclusively on predictive validity, i.e., the extent to which total scores predict recidivism. This is appropriate when the ultimate purpose of risk assessment is merely to characterize a youth’s likelihood of recidivism, compared to other youth. Usually, this is to inform a single event decision where there is no opportunity to modify the risk estimate based on future behavior (see Heilbrun, 1997). In this case, what the tool assesses is irrelevant because there is no interest in explaining risk. For example, if a tool that that assesses accuracy in playing street dice correlates highly with recidivism, then the tool is valid for characterizing risk (see Nunnally, 1978). In contrast, when the ultimate purpose of risk assessment is to reduce a youth’s likelihood of recidivism, construct validity also becomes relevant. Construct validity is the extent to which a tool measures the abstract concept(s) that it purports to measure. Theoretically, complex tools like the CA-‐YASI add value to simple measures of risk by assessing constructs that help explain the process that leads to recidivism. Specifically, they assess “criminogenic needs” like anger, poor self control, and antisocial attitudes that maintain criminal behavior. If (and only if) they measure these constructs validly, then such tools can inform risk reduction efforts by (a) specifying risk factors to target in treatment, and (b) capturing any changes in risk over time to inform ongoing decisions about supervision and treatment. If DJJ merely wished to characterize a youth’s level of recidivism risk to make a single event decision, they could select a more efficient tool. The CA-‐YASI requires trained staff to integrate interview-‐ and file-‐ information to rate over 100 items that assess different aspects of twelve different risk and protective factors. This reportedly takes a minimum of 2.5 hours per case (see Phase I report). In an era of severe resource limitations, it is critical to assess whether the CA-‐YASI actually adds value that could inform risk reduction efforts. Does the tool assess the criminogenic needs it purports to measure?

Cross-‐structure analysis of construct validity The goal of Phase Two is to assess the extent to which the CA-‐YASI domains assess the risk factors they are supposed to assess. Because “construct validation ultimately rests on studying relations between the construct in question and other constructs of variables in a theoretical context” (Pedhazer & Schmelkin, 1991), the Phase Two method is cross-‐structure analysis.

9

Specifically, we examine the relationships between the CA-‐YASI domains and well-‐validated measures of constructs that theoretically are strongly related (and unrelated) to those domains. For each of the domains, we assess whether there is evidence of convergent validity (i.e., strong correlations with measures of theoretically similar constructs) and discriminant validity (i.e., weak correlations with measures of theoretically different constructs). For example, if it is valid, the CA-‐YASI domain called “Substance Use” should correlate much more strongly with a well-‐validated measure of youth’s substance abuse problems (SASSI) than a well-‐validated measure of intelligence (the WASI).1 It is important to attend to both convergent and discriminant measures because “everything is correlated with everything else, more or less.” It is in examining the pattern of correlations that one can determine whether a scale measures what it purports to measure.

Study procedure The cross-‐structure analysis involves two assessments of a sample of DJJ youth. The first is a routine CA-‐YASI assessment completed by a DJJ staff member who demonstrated at least adequate interrater reliability (CA-‐YASI Total Score ICC > .60) in Phase I of this evaluation. (Because reliability is a necessary but not sufficient condition for validity, unreliable staff were excluded from Phase II.) The second is a comprehensive assessment of over 30 well-‐validated criterion measures completed by reliable UC Irvine staff. To ensure that associations between the CA-‐YASI and criterion measures were not attenuated by change over time, criterion assessments were completed within two months of the CA-‐YASI assessment. Using IRB-‐approved procedures, trained UC Irvine staff approached eligible male youth at four DJJ institutions (i.e., OHC/CHAD, Southern, Ventura) throughout the state and invited them to participate in the study. They enrolled youth when the youth provided assent and his parent/guardian provided informed consent for research participation. Youth and guardians were informed that participation (or lack thereof) would not affect the youth’s supervision or treatment at DJJ, and that all responses would be kept confidential. Youth were not paid for their time or offered any incentive to participate. Enrolled youth completed a 3 hour assessment that included a semi-‐structured interview, self-‐report measures, and performance tests (including computerized tasks). These assessments were conducted by UC Irvine staff who had trained to reliability on all measures and procedures. After the interview, UC Irvine reviewed youths’ records to code information that was relevant to completing the criterion measures.

Study participants Study ineligibility criteria for DJJ youth included: (a) female gender, (b) no CA-‐YASI assessment by a reliable DJJ staff member within the past two months, (c) non-‐English speaking (n = 3), (d) age greater than 23 years of age (n = 23), and (e) transfer or discharge during the recruitment

1 When conducting a cross-‐structure analysis, it is important to recognize that the strength of correlation

10

window (n= 54). We excluded girls from this study because there were too few of them in the DJJ population (n = 44; CDCR, 2012) to adequately power the separate statistical analyses that would be needed for this subpopulation. As shown in the figure, of the 487 DJJ youth who were eligible to participate in this study, 325 were recruited (the remaining 162 were not, given budget limitations). Based on DJJ policies, UC Irvine staff was not allowed to offer any incentive to youth to participate. Of those recruited, 27% of youth or their parents refused to participate. Compared to study participants, those who refused were modestly younger (mean = 17.74 years old, SD = 1.62; t (323) = 2.12, p < .05), but did not differ significantly in terms of ethnicity. Participants were 237 incarcerated adolescent male offenders recruited from four sites: OHC (35.4%; n = 84), Ventura (32.5%; n = 77), Chad (21.1%; n = 50) and Southern (11.0%; n = 26). Given that there were no significant site-‐related differences in youth’s CA-‐YASI Total scores, data were pooled across sites for analyses. Study participants were representative of the DJJ population with respect to age, ethnicity, and criminal history. For example, even though UC Irvine staff oversampled youth who were age 16 and younger, the average age of study participants was 18.16 years (SD = 1.57, range: 15 to 23), which is similar (d = -‐.15; r=.07) to the average age of the DJJ population (18.42 years, SD = 1.86; CDCR, 2012). Similarly, UC Irvine staff made explicit efforts during recruitment to ensure that the study sample matched the DJJ population in ethnicity and criminal history. Participants were 56.1% Hispanic, 27.0% African American, 11.4% Caucasian and 5.5% other, which is not significantly different from the ethnic distribution of the DJJ population.

Scoring the CA-‐YASI As noted in our Phase I report, scoring the CA-‐YASI is not simple. Three scoring systems – “old,” “new,” and “simple” are available. The old system, which is currently programmed into DJJ CA-‐YASI software, weights items in a manner that Orbis has implied is outdated and unlikely to strongly predict recidivism. The new system, which was recently developed by Orbis, assigns weights to items based on how strongly they predicted recidivism for a sample of DJJ youth in a recent analysis they conducted. This new system "neutralizes" several items (i.e., effectively

Eligible Youth n = 487

Refused n = 88

Youth Refusal n = 83

Parental Refusal n = 5

Not Recruited n = 162

Participated n = 237

11

deletes them) because they did not predict recidivism. In contrast with Orbis’s old and new system, the UC Irvine simple scoring system (introduced in the Phase I report) uses virtually all CA-‐YASI items. It applies unit weights to each item, and then simply sums across appropriate items to calculate domain and total scale scores. Throughout the main text of this report, we use the simple scoring system. Why? Because the aim is to assess construct validity, i.e., whether the CA-‐YASI domains assess the needs they ostensibly assess. Even though the new scoring system neutralizes many items, Orbis plans to retain all items on the CA-‐YASI because as a whole, these items theoretically assess needs relevant to treatment and supervision. Use of our simple scoring system presents a "best case scenario" for the construct validity of the CA-‐YASI domains. With only one exception (out of 32 possibilities), scores yielded by our simple system related as strongly, or more strongly, to the concurrent measures than Orbis-‐based scores. This is apparent by reviewing Appendix A, where we describe construct validity results for the new Orbis-‐based scoring system, which was designed to predict recidivism (more than to assess constructs). After consulting with DJJ research, we chose the new system (not old) because it represents the “best case scenario” for the predictive utility of the CA-‐YASI and will be the focus of our upcoming Phase III cross-‐validation study. That report will help DJJ assess whether it's worthwhile to continue the contract with Orbis and update its software to the new system.

CA-‐YASI domains: Definitions, reliability, and inter-‐correlations The CA-‐YASI domains are listed Table 1, along with definitions adapted from Orbis (these definitions focus on the risk rather than protective pole of each dimension).

Table 1: CA-‐YASI Domains & Reliability

CA-‐YASI Domain

Definition ICC α

Legal History early onset, frequent, varied criminal behavior .81 .62

Correctional Response

noncompliance with rules of institutional or community placement, including misconducts, technicals, and new offenses

.71 .65

Violence-‐Aggression

past violent behaviour (static risk); anger/hostility, callousness, attitudes supportive of aggression

.66 .86

Social Influences

attachment to antisocial peers, absence of constructive adult role models

.64 .89

12

CA-‐YASI Domain

Definition ICC α

Substance Use frequent alcohol and drug use that can impair functioning .57 .52

Attitudes antisocial attitudes, including minimization of responsibility, denial of harm, poor attitudes toward the justice system/authority

.79 .87

Social-‐Cognitive Skills

poor decision-‐making skills (consequential thinking, goal setting, problem-‐solving) and/or interpersonal skills (perspective taking) relevant to antisocial behavior

.57 .93

Family poor family relationships or role modeling .79 .75

Education-‐Employment

poor educational achievement, employment potential, or motivation

.84 .80

Health mental health problems * .49

Community Linkages

lack of relevant services to address criminogenic needs in the community

.85 .65

Community Stability

poor finances, accommodation, or transportation .68 .62

*There was insufficient information in the reliability case material to rate mental health Because a domain cannot be valid if it is not reliable, Table 1 also reports two different indices of reliability. The first index is inter-‐rater reliability, or the agreement between DJJ staff and expert ratings of items for each domain. These data were drawn from Phase I, but focus only on the subsample of reliable DJJ staff included in Phase II -‐-‐i.e., those with CA-‐YASI Total ICCs > .65. The average CA-‐YASI Total ICC for this subsample of reliable staff is an excellent .85. However, using our threshold of ICC > .65, three specific domains have inadequate inter-‐rater reliability even in this generally reliable subsample (Social Cognitive Skills; Substance Use, and, to a lesser extent, Social Influences). The second index of reliability is a Cronbach’s alpha, which indicates the extent to which the items of each domain “hang together” and appear to assess the same construct. The results indicate that internal consistency is acceptable (i.e., alpha > .70) for six of the twelve CA-‐YASI domains; questionable (i.e., .70 > alpha >.60) for four domains; and poor/unacceptable for two domains. Specifically, CA-‐YASI items that ostensibly assess mental health or substance abuse do not appear to measure unitary constructs. Notably, however, the mental Health and Substance Abuse domains are the only ‘variable’ domains that performed relatively well in this

13

study (which helps offset Orbis concerns that heterogeneous domains are the basis for poor construct validity results). The degree to which the domains of the CA-‐YASI are associated with another, and with total scores on the measure, is shown in Table 2 on the following page. Cohen (1988, 1992) provides the following guidelines for interpreting correlation coefficients for the social sciences:

• “small” or weak, r = 0.1 − 0.23; • “medium” or moderate, r = 0.24 − 0.36; • “large” or strong, r = 0.37 or greater.

As shown in Table 2, as a whole, CA-‐YASI domains tend to be moderately related to each another. Although this is partially because some domains measure similar constructs, it is probably also because of shared method, i.e., the fact that one DJJ staff member predominantly rated the youth across all 12 domains (see Footnote 1, above). If that professional’s general impression of the youth (e.g., his likeability, attractiveness) affected her or her specific ratings across ostensibly different domains, then the correlations partly reflect halo error. This is one reason that a cross-‐structure analysis of construct validity is critical – it allows one to assess whether the CA-‐YASI domains are more strongly associated with measures of the same construct measured via a different method than with measures of a different construct assessed within the CA-‐YASI. For example, the CA-‐YASI domain of Substance Use should correlate more strongly with the SASSI (a self report substance abuse measure) than with the CA-‐YASI domain of Family. In other words, the correlations between the CA-‐YASI and validated measures must be more a function of shared constructs than shared method. Beyond the general moderate strength of relationships among domains, two other points are important to note in Table 2. First, two groups of domains are so strongly correlated with one another that one might question whether they really get at separable constructs: (1) Legal History and Correctional Response (which both emphasize criminal history); and (2) Violence-‐Aggression, Attitudes, Social-‐Cognitive Skills, and Social Influences (which emphasize antisocial traits and peers). Second, two domains (Health and Community Linkages) are so weakly correlated with the rest of the scale that it raises a question about whether they belong in the instrument. These data on the reliability of, and correlations among, the CA-‐YASI domains provide important contextual information for interpreting the cross-‐structure analyses. They will be referenced later, as relevant.

14

Table 2. CA-‐YASI Cross-‐Domain & Domain-‐Total Correlations

CA-YASI Domain

Legal Hx

Correc Resp

Vio-‐Agg

Soc Infl

Sub Use

Atti-‐tudes

Soc-‐Cog

Family Educ-‐Employ

Health Comm Link

Comm Stab

Legal History 1.0

Correctional Response

.64** 1.0

Violence-Aggression

.31** .41** 1.0

Social Influences .35** .35** .67** 1.0

Substance Use .24** .26** .38** .38** 1.0

Attitudes .19** .31** .68** .65** .10 1.0

Social-Cognitive Skills

.13* .18** .65** .49** .03 .68** 1.0

Family .13* .33** .38** .30** .20** .25** .29** 1.0

Education-Employment

.13* .26** .47** .41** .04 .53** .52** .34** 1.0

Health .11 .25** .08 -‐.07 .09 .02 .02 .25** .22** 1.0

Community Linkages

.11 .09 .05 .26** .10 .11 .07 .15* .14* .01 1.0

Community Stability

.23** .19** .32** .32** -‐.05 .15* .20** .38** .27** .05 .12 1.0

Domain/Total Correlation

.46** .58** .83** .79** .48** .76** .71** .56** .70** .22** .23** .47**

** p< .01, *p <.05

15

Overview of criterion measures Criterion measures are grouped into concurrent measures (that will relate strongly to the CA-‐YASI, if it is valid) and divergent measures (that will relate weakly to the CA-‐YASI, if it is valid). The primary concurrent validity measures vary by CA-‐YASI domain, and are described -‐-‐ domain by domain -‐-‐ in the results section. As a whole, the concurrent measures were chosen because they (a) assessed a construct that was similar to, or the same as, a CA-‐YASI domain, and (b) were well-‐validated (i.e., reliable and valid) measures of that construct. To control for the influence of method factors, we strove to include concurrent measures that represented a range of methods (e.g., structured clinical ratings, youth self report, performance tests, laboratory measures). The primary divergent validity measures are the same across CA-‐YASI domains. These were chosen because they (a) ostensibly assessed a construct that was dissimilar to virtually all CA-‐YASI domains, and (b) were well-‐validated. The divergent measures and rationales are provided in Table 3. Evidence for the reliability and validity of all measures (including divergent measures) are provided in Appendix B.

Table 3: Discriminant Measures for All Domains

Measure Definition & rationale

Somatization (Brief Symptom Inventory)

distress about perceived bodily dysfunction (e.g., dizziness, nausea, hot/cold spells, shortness of breath). Theoretically unrelated to CA-‐YASI domains, possibly excepting Health (weak assn..).

Head Trauma (Schubert et al., 2004)

presence and seriousness of brain injuries. Theoretically unrelated to CA-‐YASI domains, possibly excepting Social/Cognitive (weak assn).

Wechsler Abbreviated Scale of Intelligence

estimated intelligence quotient, based on verbal and performance tests. Theoretically unrelated to CA-‐YASI domains, excepting Social/Cognitive (moderate assn) and possibly legal history (weak assn.).

Pubertal Development Scale

degree of pubertal development (based on body hair growth, voice change, facial hair growth; pre-‐, early, mid-‐, late-‐ or post-‐pubertal). Theoretically unrelated or very weakly related to most CA-‐YASI domains.

Results In this section, we describe and interpret the results of the cross-‐structure analysis. To provide context, we begin with analysis of the CA-‐YASI scale as a whole. Then, we present evidence

16

relevant to the construct validity of each CA-‐YASI domain. We cover domains in the order by which we grouped them. That is, domains that ostensibly assess:

1. Robust individual criminogenic needs, including Violence-‐Aggression; Attitudes; and Social Cognitive Skills. These domains are directly relevant to evidence-‐based risk-‐reduction programs, including cognitive behavioral anger management and behavioral/skills-‐based programs. Indeed, DJJ uses Violence/Aggression to assign youth to Anger Interruption Training;

2. Contextual risk factors for offending, including peer group associations (Social Influences), the quality of parenting (Family), and -‐-‐ to a lesser extent -‐-‐ need for community/financial support post-‐incarceration (i.e., Community Stability);

3. School or work involvement (Education/Employment); 4. Substance use and mental health (Health is a largely noncriminogenic, but important

need); 5. Criminal history, including Legal History and Correctional Response.

Although evidence-‐based programs for young offenders vary tremendously, they often target robust individual criminal needs (#1 above), family functioning, peer group associations, and school/work involvement. In this evaluation, we placed the most emphasis on rigorously evaluating these risk-‐reduction-‐relevant domains. We now turn to the nature of the CA-‐YASI as a whole.

CA-‐YASI as a Whole The CA-‐YASI as a whole is not designed to assess a construct. Instead, like other purpose-‐built risk assessment tools, it is designed to assess a specific set of risk/protective factors that, as a whole, predict recidivism. Nevertheless, CA-‐YASI Total Scores should manifest concurrent validity, in the sense that they should be at least moderately correlated with total scores on well-‐validated risk assessment tools. Why? A robust body of evidence indicates that (a) well-‐validated risk assessment tools tend to be highly associated with one another, and (b) have levels of predictive utility that are essentially interchangeable (for a review, see Skeem & Monahan, 2011). This is probably because these validated tools all tap –albeit in different ways -‐-‐ shared dimensions of risk (e.g., criminal history; antisocial attitudes; irresponsible lifestyle; psychopathic features; see Kroner, Mills, & Reddon, 2005). For these reasons, we examined the association between CA-‐YASI Total Scores and the Youth Version of the Psychopathy Checklist (PCL:YV; Forth, Kosson, & Hare, 2003). We selected this measure of psychopathy instead of a purpose-‐built risk assessment tool to assess concurrent validity because the PCL:YV and its parent measure, the PCL-‐R (a) predict recidivism as strongly as purpose-‐built tools, (b) are often selected by experts above purpose-‐built tools to assess risk, and (c) assess constructs that are relatively well-‐understood. The PCL:YV consists of 20 items that are scored by a trained rater, on the basis of a semi-‐structured interview and file review (see Appendix B for psychometrics). The PCL:YV consists of two basic scales: the Interpersonal-‐Affective scale assesses core features of psychopathy, or

17

the “selfish, callous, and remorseless use of others” (Hare, 1991), whereas the Social Deviance scale assesses a “chronically unstable and antisocial lifestyle” (Hare, 1991). The predictive utility of the PCL measures is largely attributable to its Social Deviance scale (Skeem & Mulvey, 2001; Walters, 2003). This may be partly because of Meehl’s maxim that past behavior is typically the best predictor of future similar behavior (Gendreau et al., 2003) and partly because the scale taps broad traits like antagonism, anger, and impulsivity that are not specific to psychopathy, but place people at risk for violence and other criminal behavior (Skeem, Miller, et al., 2005). The relation between CA-‐YASI total scores and main PCL:YV scores is shown in GREEN in Figure 1. As a whole, the CA-‐YASI is strongly associated with the PCL:YV, but this association is solely attributable to the PCL:YV Social Deviance scale. That is, the CA-‐YASI overlaps with the PCL:YV not in its assessment of core interpersonal and affective features of psychopathy (rpartial = -‐.05, ns), but instead in its assessment of an impulsive and irresponsible lifestyle and dense history of criminal behavior and conduct problems (rpartial = -‐.05, p <.001). This provides some evidence for the concurrent validity of the CA-‐YASI as a whole, in the sense that the tool is associated with a validated measure of antisocial traits and behavior that has been shown to predict recidivism relatively strongly among youth.

The relation between CA-‐YASI scores and the discriminant measures is shown in RED in Figure 2. Most of the associations were – as they should be—weak. However, CA-‐YASI scores were moderately associated with low intelligence. Individual differences in motivation during IQ testing can spuriously inflate the association between measures of intelligence and antisocial behavior (Duckworth et al., 2011). However, IQ scores were significantly more strongly associated with the CA-‐YASI than our well-‐validated measure of antisocial traits and behavior

0.37

0.47

0.16 0.15

0.04

0.33

0.2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 1. CA-‐YASI Total: Concurrent & Discriminant Associations

18

(PCL:YV Social Deviance, r = .18, p <.01; t [df] = 2.35, p <.01). This raises concern that the CA-‐YASI taps some irrelevant variance. That concern is only partially offset by the fact that the CA-‐YASI Total Scores correlate more strongly with the primary concurrent measure (PCL:YV Social Deviance) than with intelligence, t (df) = 1.92, p < .05, or any other divergent measure. Given this overview of the relation of the CA-‐YASI as a whole to a major concurrent measure and our discriminant measure, we now turn to the CA-‐YASI domains. The domains are the primary focus of the construct validity analysis, given that the domains are meant to tap particular risk factors or criminogenic needs.

Violence-‐Aggression The Violence-‐Aggression domain is one of the longest scales in the CA-‐YASI. It ostensibly assesses past violent behavior (static risk), along with anger/hostility, callousness, and attitudes supportive of aggression. The concurrent measures and rationales for this domain are provided in Table 4. Evidence for their reliability and validity is provided in Appendix B.

Table 4: Concurrent Measures for Violence-‐Aggression


Social Information Processing Scale (SIP)

tendency to perceive that peers have hostile intent, even when it is absent (e.g., misinterpret benign comments as threats); tendency to generate aggressive responses to social dilemmas, and attitudes supportive of aggression. The SIP assesses social-‐cognitive problems that robustly predict youths’ aggression and are directly relevant to intervention.

Hostility (BSI) anger (i.e., emotional reactivity that involves annoyance and irritation, a tendency to argue, urges to destroy property or hurt others, uncontrollable temper outbursts). This scale robustly predicts violence and is directly relevant to anger-‐focused interventions.

Meanness (Patrick, 2010)

tendencies toward callousness, cruelty and predatory aggression. Relevant to Violence-‐Aggression’s focus on features relevant to predatory aggression

PCL:YV Social Deviance

assesses an impulsive and irresponsible lifestyle, as well as chronic criminal behaviour. Explains most of the PCL:YV’s utility in predicting violence. Relevant to static aspects of Violence-‐Aggression.

The relationships between the Violence-‐Aggression domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 2. The Violence-‐Aggression domain was unassociated with professional ratings of hostile attribution bias (intent and response), even though the domain explicitly includes hostile attribution bias. Violence-‐Aggression was only weakly associated with self reported anger and attitudes supportive of violence, even though it includes those constructs as well. In contrast, this domain was

19

moderately associated with more static features like meanness and antisocial traits/behavior. Generally, the scale appears loaded toward static factors more than dynamic, treatment-‐relevant features (see Appendix A for evidence that this remains true, even when the analysis is restricted to ostensibly dynamic items on this scale). Figure 2 also shows the associations between the Violence-‐Aggression domain and measures of discriminant validity in RED. Most of these associations were – as they should be – weak. As

was the case with CA-‐YASI Total scores, however, the Violence-‐Aggression domain was moderately associated with low intelligence. In fact, Violence-‐Aggression was not significantly less strongly associated with intelligence than with Social Deviance, t(df) = 1.04, ns. In summary, the Violence-‐Aggression domain was weakly associated with both the concurrent measures (r= .18) and discriminant measures (r= .13), on average. These results provide little support for the construct validity of this scale, and are consistent with earlier suggestions that this scale has a lack of specificity. That is, as shown in Table 2, Violence-‐Aggression scores were very strongly associated with three other CA-‐YASI domains -‐-‐ and were so highly correlated with CA-‐YASI Total scores (r=.83) that they arguably measure the same thing. In short, the Violence/Aggression scale appears to assess risk factors for criminal behavior (which happens to include violence) more than dynamic risk factors for violence per se.

Attitudes The attitudes scale ostensibly assesses attitudes that are supportive of criminal behavior, including minimization of responsibility, denial of harm, and poor attitudes toward the justice system/authority. The domain is construed as entirely dynamic. The concurrent measures and

0.07 0.08

0.23 0.15

0.24

0.36

0.06 0.06

0.28

0.14

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 2. Aggression-‐Violence: Validity Associations

20

rationales for this domain are provided in Table 5. Evidence for their reliability and validity is provided in Appendix B.

Table 5: Concurrent Measures for Attitudes


Psychological Inventory of Criminal Thinking Styles (PICTS)2

thinking styles considered essential to the maintenance of a criminal lifestyle; including Proactive Criminal Thinking (e.g., feelings of entitlement to special treatment or goods) and Reactive Criminal Thinking (e.g., perceived inability to tolerate stress or solve problems; impulsive, angry, externalizing reactions).

Attitudes supportive of aggression (SIP)

see above (Violence-‐Aggression domain). Given that aggression is one form of criminal behavior, supportive attitudes are relevant to the Attitude domain.

The relationships between the Attitude domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 3. The Attitudes domain was unassociated or inversely associated with a validated measure of criminal thinking; and only weakly associated with attitudes supportive of aggression. (Unlike the Violence-‐Aggression domain, the Attitudes domain was only weakly associated with PCL:YV Social Deviance [r=.15, p<.01], suggesting less emphasis on static factors.) Figure 3 also shows the associations between the Attitudes domain and measures of discriminant validity in RED. All of these associations were – as they should be – weak. In summary, the Attitudes domain was more weakly associated with the concurrent measures (r= .01) than the discriminant measures (r= .14), on average. These results challenge the construct validity of the Attitudes scale. Although it is possible that the CA-‐YASI domain would show a stronger relation with a measure of the content of criminal thinking (i.e., the CSSM) rather than its process (PICTS), it seems unlikely that our use of the PICTS explains the poor concurrent validity results given that (a) measures of criminal thinking content and process are moderately associated (Morgan et al., 2010 data, CCSM Total with PICTS Proactive & Reactive r =.24 & .21, p<.001), (b) the Attitudes scale relates weakly to the SIP Attitudes scale, which focuses on content, and (c) Attitudes scores were very strongly associated with three other CA-‐YASI domains (Table 2), raising further questions about domain specificity. There is little evidence that the Attitudes domain assesses procriminal thinking or attitudes per se. 2 Because of an administrative error, PICTS scores are available for only 86 youth (36% of the full sample). Nevertheless, data appear to be missing at random, i.e., there are no significant differences between youth who did-‐ and did not complete the PICTS in age, ethnicity, CA-‐YASI Total scores, social deviance (PCL:YV), or intelligence (WASI). Moreover, a sample size of 85 provides sufficient power (.80) to detect a medium to large effect at p<.05 (Cohen, 1992). For these reasons, we used the PICTS data, which should relate strongly to Attitudes, but emphasize effect sizes rather than significance levels.

21

Social-‐Cognitive Skills The Social-‐Cognitive Skills domain ostensibly assesses poor decision-‐making skills (consequential thinking, goal setting, and problem-‐solving) and – to a lesser extent -‐ interpersonal skills (perspective taking) thought relevant to antisocial behavior. The domain is construed as entirely dynamic. The concurrent measures and rationales for this domain are provided in Table 5. Evidence for their reliability and validity is provided in Appendix B.

Table 6: Concurrent Measures for Social-‐Cognitive Skills


Tower of London (ToL)

commonly used neuropsychological measure of executive function, i.e., cognitive processes that allow for self-‐regulation and socially appropriate behaviour. ToL assesses impulsivity (time to first move) and problems with cognitive flexibility/working memory (errors/excess moves). Executive function are associated with, and thought to underpin, antisocial behavior. Social-‐Cognitive Skills is meant to tap deficits that relate both to executive function and antisocial behaviour.

Go/No Go Task response inhibition, an element of executive function that is specifically relevant to learning via punishment (i.e., passive avoidance learning). Relates strongly to criminal behaviour (For rationale, see ToL).

-‐0.18

0.05

0.16 0.09

0.03

0.23 0.2

-‐0.3 -‐0.2 -‐0.1 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

PICTS Reactive

PICTS Proactive

SIP Attitudes*

Somatization Head Injury Lo IQ*** Lo Pubertal*

Figure 3. Attitudes: Validity Associations

22

Intelligence (WASI)

see above (Discriminant measures). Because much of the Social-‐Cognitive domain is meant to assess neuropsychological problems relevant to offending, and intelligence relates to such problems, the WASI is better construed as a convergent than discriminant measure for this domain. However, social-‐cognitive scores should relate weakly to intelligence, or at least less strongly than measures of executive function (particularly impulsivity and response inhibition).

The relationships between the Social Cognitive Skills domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 4. The Social-‐Cognitive Skills domain was unassociated or weakly associated with indices of executive function and intelligence. The domain was more related to intelligence than response inhibition, which counters theory-‐based expectations (as response inhibition is more specific to both executive functioning and antisocial behavior). As shown in RED in Figure 4, the domain was unrelated weakly to the discriminant measures.

In summary, the Social Cognitive Skills domain was essentially unassociated with both the concurrent measures (r= .08) and discriminant measures (r= .09), on average. These results challenge the validity of this scale. As shown in Table 2, Social Cognitive Skills scores were very strongly associated with two other CA-‐YASI domains, raising further questions about domain specificity. Moreover, as shown in Table 1, even for the subset of “reliable” staff included in this project, interrater reliability is very poor for this domain…probably because the items require so much judgement. In short, there is little evidence that the Social-‐Cognitive domain specifically assesses cognitive problems that research has shown is relevant to antisocial behavior (chiefly, problems related to executive functioning).

0.17

0.05

-‐0.11

0.2 0.12

-‐0.03

0.17

-‐0.2

-‐0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ToL Errors* ToL Impuls Go/No Go Lo IQ* Somatization Head Injury Lo Pubertal*

Figure 4. Social-‐Cognitive Skills: Validity Associations

23

Social Influences The Social Influence domain ostensibly assesses attachment to antisocial peers and, to a lesser extent, absence of constructive adult role models in the community. There is also an emphasis on gang involvement (a specific class of antisocial peers). The domain is construed as entirely dynamic. The concurrent measures and rationales for this domain are provided in Table 7. Evidence for their reliability and validity is provided in Appendix B.

Table 7: Concurrent Measures for Social Influences


Peer Delinquent Behavior (PDB)

two aspects of peer delinquency: antisocial peer behavior (e.g., “How many of your friends have sold drugs?”) and antisocial peer influence (“How many of your friends have suggested that you should sell drugs?”). Directly relevant to core of Social Influences domain.

Neighborhood Disorganization

degree of disorganization in the youth’s most recent neighborhood, as indexed by census tract data on poverty, unemployment, and cultural heterogeneity. Neighborhood disorganization relates to crime and theoretically relates at least weakly to the availability of prosocial community role models.

The relationships between the Social Influences domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 5. The Social Influences domain was weakly associated with indices of peers’ antisocial behavior and neighborhood organization, but unassociated with indices of peers’ influence on the youth’s antisocial behavior. The relationships between the Social Influences domain on one hand, and measures of discriminant validity on the other, are shown in RED in Figure 5. Generally, the domain was unassociated or weakly associated with these measures.3 In summary, the Social influences domain is weakly associated with both the groups of convergent (r=.16) and discriminant (r=.11) measures, on average. Nevertheless, compared to the CA-‐YASI domains evaluated above, there is at least some evidence that Social Influences taps construct relevant variance. That variance is more descriptive of having peers who engage in antisocial behavior than “peer influence” per se. If this domain could be rated more reliably (even reliable staff have difficulty; see Table 1), its validity might improve.

3 Although susceptibility to peer influence relates to psychosocial maturity, pubertal development was retained as a discriminant measure for Social Influences because the domain focuses on affiliation with antisocial peers (not influence or susceptibility to such).

24

Family The Family domain ostensibly assesses poor family relationships and poor family role modeling of prosocial behavior. The domain is meant to include both static (e.g., history of abuse) and dynamic (e.g., current level of conflict) factors. The concurrent measures and rationales for this domain are provided in Table 8. Evidence for their reliability and validity is provided in Appendix B.

Table 8: Concurrent Measures for Family


Family Background Questionnaire (FBQ)

psychological abuse (including exposure to criminal behaviour), physical abuse, and exposure to domestic violence. These variables relate to adolescents’ risk of criminal behavior, and are theoretically relevant to the Family domain (both static and dynamic features).

Family Management Scale (FMS, in Communities That Care)

poor parental monitoring and discipline, which is a strong risk factor for delinquent and criminal behavior. This variable is relevant to the Family domain, if it is meant to capture dynamic risk factors relevant to risk reduction via parent training

The relationships between the Family domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 6. The Family domain was unassociated or weakly associated with the concurrent measures, with the weakest relationship to psychological abuse

0.22

0.05

0.21

0 0.06

0.26

0.12

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

PDB Behavior**

PDB Inhluence

Neigh Disorg**

Somatization Head Injury Lo IQ** Lo Pubertal

Figure 5. Social Inhluences: Validity Associations

25

(and its opposite on this scale, which includes affection, support, and positive role modeling), and strongest relationship to family monitoring. The relationships between the Family domain on one hand, and measures of discriminant validity on the other, are shown in RED in Figure 6. Generally, the domain was unassociated or weakly associated with these measures (more Somatization than Intelligence).

In summary, the Family domain is weakly associated with both the groups of concurrent (r=.12) and discriminant (r=.11) validity measures, on average. Its strongest associations with members of both the concurrent (Family Monitoring) and discriminant (Somatization) groups are essentially interchangeable with one another. This provides little support for the notion that this domain taps family-‐related static or dynamic risk factors for crime.

Community Stability As suggested earlier, it was difficult (if not impossible) to isolate concurrent validity measures for the Community Linkages scale (given that release plans were often unknown when youth were assessed; as was the availability of appropriate services by census tract). Although it was also difficult to identify an appropriate concurrent validity measures for the Community Stability scale, we attempted to do so. The Community Stability domain ostensibly taps poor finances, accommodation, or transportation. The domain is meant to be wholly dynamic. The concurrent measure and

0

0.13 0.16 0.19 0.18

0.02

0.14 0.08

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 6. Family: Validity Associations

26

rationale for this domain are provided in Table 9. Evidence for its reliability and validity is provided in Appendix B.

Table 9: Concurrent Measure for Community Stability


Neighborhood Disorganization

See above (Peer Influences). Although we have no measure of socioeconomic status specific to the youth, neighbourhood disadvantage is associated with family/individual disadvantage. Youth from more disorganized neighborhoods should have lower scores on Community Stability.

The relationship between the Community Stability domain and the concurrent measure of Neighborhood Disorganization concurrent was weak, as shown in GREEN in Figure 7.

The relationships between the Community Stability domain on one hand, and measures of discriminant validity on the other, are shown in RED in Figure 7. The domain was moderately associated with intelligence, weakly associated with head injury, and unassociated with somatization and pubertal status. In summary, the Community Domain is weakly associated with both the concurrent measure (r=.13) and discriminant (r=.12) measures, on average. As shown in Table 1, this domain also has questionable levels of internal consistency. Although the present evaluation should not be

0.13

0.02

0.12

0.29

0.03

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Comm Disorg Somatization Lo head Injury Lo IQ*** Pubertal

Figure 7. Community Stability Validity Associations

27

viewed as a strong test of this scale (given that well-‐validated measures of directly relevant constructs are lacking), it provides little support for its unidimensionality and relationship to general socioeconomic disadvantage.

Education/Employment The Education/Employment domain ostensibly assesses poor educational achievement, employment potential, or motivation related to either. The domain is meant to include both static (e.g., achievement) and dynamic (e.g., motivation) factors. The concurrent measures and rationales for this domain are provided in Table 10. Evidence for their reliability and validity is provided in Appendix B.

Table 10: Concurrent Measures for Education/Employment


School Connection Scale

for those currently in school, assesses commitment (e.g., “I wish I could drop out of school”, reverse coded) and belongingness (e.g., “I feel close to at least one of my teachers”). Directly relevant to dynamic aspects of Education/Employment domain.

Impulsive & Irresponsible Lifestyle (PCL:YV)

impulsive and irresponsible lifestyle (a subscale of Social Deviance), which includes poor educational and work motivation and achievement, and a lack of long-‐term goals relevant to those domains. Relevant to both static and dynamic aspects of Education/Employment.

Intelligence see above (discriminant measures). Low intelligence relates to poor educational and work attainment and, for that reason, is construed as a concurrent rather than discriminant measure for Education/Employment. Associations should be weaker for intelligence, compared to the other concurrent measures.

The relationships between the Education/Employment domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 8. The Education/Employment domain was weakly associated with the main concurrent measures (tapping school involvement and an irresponsible lifestyle), and just moderately associated with intelligence.

28

The relationships between the Education/Employment domain on one hand, and measures of discriminant validity on the other, are shown in RED in Figure 8. Generally, the domain was unassociated or weakly associated with the discriminant measures, with the exception of low pubertal status (notably, pubertal status was not associated with School Connection). In summary, the Education/Employment domain is weakly associated with both the groups of concurrent (r=.17) and discriminant (r=.20) validity measures, on average. This provides little support for the notion that this domain taps educational and/or vocational problems relevant to intervention.

Substance Use The Substance Use domain ostensibly taps frequent alcohol and drug use that can impair functioning. The domain is meant to include both static (e.g., historical) and dynamic (e.g., current) factors. The concurrent measures and rationales for this domain are provided in Table 11. Evidence for their reliability and validity is provided in Appendix B.

Table 11: Concurrent Measures for Substance Use


Face Valid Alcohol (SASSI-‐A2)

alcohol use, motivation and consequences of usage, and loss of control

0.1 0.17

0.24 0.22

0.06

0.31

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

School Cnxn Imp-‐Irresp Lifestyle**

Lo IQ** Somatization** Head Injury Lo Pubertal**

Figure 8. Education/Employment: Validity Associations

29

Face Valid Other Drugs (SASSI-‐A2)

other drug use, motivation and consequences of usage, and loss of control

The relationships between the Substance Use domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 9. The Substance Use domain is moderately associated with a well-‐validated self-‐report measure of alcohol and other drug dependence problems.

The relationships between the Substance Use domain on one hand, and measures of discriminant validity on the other, are shown in RED in Figure 9. Generally, the domain was unassociated with the discriminant measures, with the exception of low intelligence, which was weakly related to the domain. In summary, the Substance Use domain is moderately associated with the concurrent measures (r=.28) and weakly associated with the discriminant (r=.10) measures, on average. As shown in Table 2, this domain is not very strongly associated with other CA-‐YASI domains. This provides some support for the notion that this domain has some discriminant validity. As shown in Table 1, however, the Substance Use domain is one of the least reliable scales in the CA-‐YASI – with unacceptable levels of inter-‐rater reliability even among the subset of reliable DJJ staff, and unacceptable levels of internal consistency. If Orbis could improve the reliability of this scale (i.e., transparency and consistency of items), it is likely that it its validity would improve as well. As Orbis has indicated, however, there is also an interest in breadth of coverage in each domain…and this domain performed better than most.

0.25 0.3

0.05 0.1

0.22

0.02 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Alcohol** Drug*** Lo Somatiz Head Injury Lo IQ** Lo Pubertal

Figure 9. Substance Use: Validity Associations

30

Health The Health domain ostensibly taps mental health problems as an important noncriminogenic need (the domain yields service “flags” for this domain, whose items are not included in risk calculations). The domain is meant to be wholly dynamic. The concurrent measures and rationales for this domain are provided in Table 12. Evidence for their reliability and validity is provided in Appendix B.

Table 12: Concurrent Measures for Substance Use


Mental Health Problems (BSI)

global severity index is a well-‐validated measure of current mental health problems and general psychological distress. Taps essence of ostensible target for Health.

Anxiety (RCMAS) anxiety, including worry, social concerns, and physiological correlates. An important (and relatively common) index of psychological distress among youth

The relationships between the Health domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 10. The Health domain is weakly associated with total mental health problems (BSI) and just moderately associated with anxiety.

0.21 0.24

0.09 0.05 0.05

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Total BSI** Anxiety RCMAS** Head Injury Lo IQ Lo Pubertal

Figure 10. Health: Validity Associations

31

The relationships between the Health domain on one hand, and measures of discriminant validity on the other, are shown in RED in Figure 10. Notably Somatization was excluded because it is a component of the BSI measure of general mental health. Health was unassociated with the remaining discriminant measures. In summary, the Health domain is weakly associated with the concurrent measures (r=.23) and unassociated with the discriminant (r=.06) measures, on average. As shown in Table 2, this domain is not very strongly associated with other CA-‐YASI domains. As shown in Table 1, however, Health domain has unacceptable levels of internal consistency. If Orbis could improve the consistency of items on this scale, it is likely that it its validity would improve as well. (As noted in the Substance Abuse section, however, there is also an interest in breadth of coverage and the Health domain performed better than most in the CA-‐YASI.)

Legal History Unlike mental health, criminal history is a strong and robust risk factor for crime. The History domain ostensibly assesses criminal behavior (frequent, varied, serious, and with early onset). It is entirely static. The concurrent measures and rationales for this domain are provided in Table 13. Evidence for their reliability and validity is provided in Appendix B.

Table 13: Concurrent Measures for Legal History


Antisocial Behavior (PCL:YV)

antisocial behavior (a subscale of Social Deviance); specifically, frequent, varied criminal behaviour with an early onset. Directly relevant to Legal History, as it ostensibly taps the same construct.

Conduct Disorder (CBCL & DIS)

symptom count for diagnoses of conduct disorder. Assesses a variety of criminal behaviors, prior to age 15, which overlap with the Legal History domain. Correlations are expected to be moderate (i.e., weaker than those for Antisocial Behavior, which extends beyond age 15).

The relationships between the Legal History domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 11. Legal history was very strongly associated with PCL:YV Antisocial Behavior and weakly associated with Conduct Disorder.

32

Figure 11 also shows the associations between the Legal History domain and measures of discriminant validity in RED. Generally, Legal History was (as it should be) unassociated or weakly associated with the discriminant measures (including IQ). In summary, the Legal History domain was moderately associated with the concurrent measures (r= .33) and unassociated with the discriminant measures (r= .04), on average. There is also some support for its specificity in Table 2. These results provide strong support for the construct validity of this scale – the strongest support observed in this evaluation. This scale also attains very good inter-‐rater reliability among reliable DJJ staff (Table 1).

Correctional Response The Correctional Response domain ostensibly assesses noncompliance with rules of institutional or community placement, including misconducts, technicals, and new offenses. It is entirely static. The concurrent measures and rationales for this domain are provided in Table 14. Evidence for their reliability and validity is provided in Appendix B.

0.5

0.15

0.01 0.02

0.11

0.01 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Antisoc Beh*** Conduct D/O* Somatization Head Injury Lo IQ Lo Pubertal

Figure 11. Legal History: Validity Associations

33

Table 14: Concurrent Measures for Correctional Response


Antisocial Behavior (PCL:YV)

see above (Legal History). This facet includes criminal behavior that occurs during correctional supervision (e.g., escapes, breaches) and is therefore relevant to Correctional Response.

AWOL/Escape indicates, based on a record review completed by reliable raters, whether the youth ever went AWOL or attempted escape.

The relationships between the Correctional Response domain on one hand, and measures of concurrent validity on the other, are shown in GREEN in Figure 12. Legal history was very strongly associated with PCL:YV Antisocial Behavior and moderately associated with AWOL/Escape.

Figure 12 also shows the associations between the Legal History domain and measures of discriminant validity in RED. Correctional History was weakly associated with intelligence and somatization, and unassociated with the remaining discriminant measures. In summary, the Correctional Response domain was strongly associated with the concurrent measures (r= .42) and weakly associated with the discriminant measures (r= .11), on average. This provides support for the construct validity of this scale. Notably, the specificity of this scale

0.52

0.32

0.16

0.05

0.19

0.02 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 12. Correctional Response: Validity Associations

34

could not be strongly tested in this evaluation, which will be important in the future because correlations between this scale and indices of general criminal history are very large (PCL:YV Antisocial Behavior and CA-‐YASI Legal History, see Table 2).

Conclusions Unlike the Phase I study, which focused on the ability of DJJ staff to reliably score the CA-‐YASI, the present study focuses squarely on the CA-‐YASI itself to evaluate whether its domains assess the risk factors they ostensibly assess. The results represent a “best case scenario” for the construct validity of the CA-‐YASI, given that this study (a) excluded the 40% of DJJ staff who could not reliably score the tool, and (b) used simple scores that sum all items in each domain, rather than Orbis-‐based scores that delete some items to maximize predictive utility (sometimes at the expense of construct validity; see Appendix A). Even under these ideal conditions, we found little evidence that the CA-‐YASI domains most relevant to evidence-‐based treatment for delinquent youth assess the risk factors they are meant to assess. As shown in Table 15, there was evidence that the CA-‐YASI Substance Use and mental Health domains tapped their target constructs. However, the domains that ostensibly tap robust individual criminogenic needs – Violence-‐Aggression, Attitudes, and Social Cognitive Skills – showed no specific associations with target constructs of anger/hostility, procriminal thinking, and executive function deficits. There was a similar lack of support for domains that ostensibly tap important contextual risk factors like antisocial peer influence (Social Influences), family problems including abuse and poor monitoring (Family), and inadequate involvement in Education/Employment.

Table 15: Average Convergent/Discriminant Associations by Domain* CA-‐YASI Domain M correlation,

convergent measures M correlation, divergent measures

Legal History (“Static”) .33 .04

Correctional Response (“Static”) .42 .11

Violence-‐Aggression .18 .13

Social Influences .16 .11

Substance Use .28 .10

Attitudes .01 .14

Social-‐Cognitive Skills .08 .09

Family .12 .11

35

CA-‐YASI Domain M correlation, convergent measures

M correlation, divergent measures

Education-‐Employment .17 .20

Health .23 .06

Community Linkages -‐-‐ -‐-‐

Community Stability .13 .12

* Domains that performed relatively well are highlighted in green. In contrast, we found strong evidence that the CA-‐YASI captures “static” risk factors for crime. The scales with the strongest support, by far, were Legal History and Correctional Response, which tap the general domain of past criminal behavior, and past criminal behavior that occurs while under supervision, respectively. Similarly, there was evidence that the Violence-‐Aggression domain captures general antisocial traits and behavior (more than dynamic risk factors like anger and hostile attribution bias). Like the Violence-‐Aggression Domain, CA-‐YASI Total scores were strongly associated with general antisocial traits and behavior that are captured by a well-‐validated measure of social deviance (the PCL:YV). This bodes well for the CA-‐YASI’s utility in predicting misbehavior, given that this measure of social deviance – like simpler measures of criminal history – robustly predicts recidivism. In our future Phase III study, we will directly assess whether the CA-‐YASI adequately characterizes a youth’s likelihood of recidivism. In the present Phase II study, we assessed whether the CA-‐YASI adds value to simple measures of risk by assessing constructs that help explain the process that leads to recidivism. We found little evidence that it does. Because the CA-‐YASI cannot specify risk factors to target in treatment to reduce recidivism (with the possible exception of Substance Use), its utility as a risk reduction tool is limited.

Recommendations

System-‐wide As a system, DJJ reportedly is using CA-‐YASI scores to inform risk reduction efforts in only a narrow manner. That is, they assign youth with high Violence-‐Aggression scores to Anger Interruption Training. The results of this study suggest that youth with high Violence-‐Aggression scores are likely to have antisocial traits that place them at high risk for recidivism (including violent recidivism)…but will not necessarily have problems with anger or hostile attribution bias. If the goal is to treat youth with those specific problems, we recommend that DJJ use a validated measure of anger to assign youth to Anger Interruption Training. We will recommend treatment-‐specific, well-‐validated measures of particular needs upon request.

36

Case by Case As individuals, DJJ staff may be using CA-‐YASI scores to inform risk reduction efforts in a broader manner, on a case-‐by case basis. The results of this study suggest that CA-‐YASI Total scores (like Violence-‐Aggression scores) often identify antisocial, high risk youth. Based on the “risk” principle of correctional intervention, higher risk youth should receive relatively intensive supervision and services. Assuming that the results of our Phase III study support its predictive utility, it seems acceptable to interpret high CA-‐YASI total scores as an indicator of high risk, and to assign higher risk youth to more services. However, we recommend limited use of the CA-‐YASI to inform the nature of those services. Because many domains do not specifically assess the criminogenic needs they purport to assess, the CA-‐YASI provides little help in actualizing the “need” principle of correctional intervention. More specifically, the CA-‐YASI domains of Substance Use and Health can be used to flag youth in need of substance abuse and mental health services, respectively. However, the remaining domains – including those that ostensibly tap the strongest changeable risk factors for recidivism among youth (e.g., antisocial attitudes and peers; Simourd & Andrews, 1994)– should not be interpreted as indicators of specific treatment targets. Staff should not rely solely upon specific domain scores to assign youth to cognitive-‐behavioral therapy for criminal thinking and/or antisocial peers; family therapy or parent training; or even educational-‐vocational programs.4 Resource Requirements Despite a number of unanticipated barriers that are well-‐documented in progress reports (e.g., having limited access to youth during business hours; delays in receiving CA-‐YASI data; closing DJJ facilities), the UCI Team managed to absorb a number of costs, obtain supplemental funding from DJJ to cover additional costs, and ultimately conduct an ambitious assessment of construct validity. Three months were spent obtaining UCI and state IRB approval and hiring and training research staff (including volunteers) to reliability on all of the measures. The team spent over one year collecting data at four facilities across the state, with each visit requiring substantial coordination and travel time. Outside of several weeks spent negotiating a contract extension, the remainder of the period was spent on (a) data entry, cleaning, and analysis, and (b) interpretation and report-‐writing. Throughout Phase Two, the UCI Team held weekly staff meetings. Although the UCI Team planned to reinstate monthly meetings that included both DJJ research and executive staff,

4 This is not to say that there is no value in using a structured process for collecting information and developing rapport with the youth. For example, the CA-‐YASI probably helps staff obtain useful information about a youth’s job history. But most domains that ostensibly tap strong variable risk factors (including Education/Employment) have little evidence of construct validity…and scores on those domains should not be used to inform specific treatment decisions.

37

those meetings did not materialize. Once Steve Lesch was identified as the main contact for the UCI Team, Dr. Skeem and Mr. Lesch met as needed. The core UCI Team for Phase Two consisted of five individuals. Two of these individuals were paid consistently for their time, including (a) the project manager , who coordinated travel, conducted interviews, and drafted quarterly progress reports that were delivered to DJJ research, and (b) a research assistant, who assisted the project coordinator with the above tasks as well as assisted in data input and analysis. Two half time graduate students were paid for a portion of the time they spent on the project, and volunteered many additional hours – these students completed interviews, established databases, and conducted some analyses. One additional graduate student completed interviews as needed, volunteering all of his hours to the project. The Principal Investigator was paid for a portion of the time she spent on this project, and volunteered many additional hours – she oversaw all aspects of planning, training, data collection, and data analysis, and report writing.

38

References Achenbach, T. M. (1991). Integrative guide for the 1991 CBCL/4-‐18, YSR, and TRF Profiles. Burlington, VT: University of Vermont, Department of Psychiatry. Arthur, M. W., Hawkins, J. D., Pollard, J. A., Catalano, R. F., & Baglioni Jr., A. J. (2002). Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Curvey. Eval. Rev., 26, 575-‐601. Berg, W., & Byrd, D. (2002). The Tower of London spatial problem-‐solving task: Enhancing clinical and research implementation. Journal of Clinical and Experimental Neuropsychology, 24(5), 586-‐604. Bradshaw, C. P., Rodgers, C., Ghandour, L. & Garbarino, J. (2009). Social-‐cognitive mediators of the association between community violence exposure and aggressive behavior. School Psychology Quarterly, 24(3), 199-‐210. Brown, R. (1999). The influence of extracurricular activity participation upon youth problem behavior: School connection as a mediator. Washington, DC: Education Resources Information Center. (Eric Document Reproduction Service No. ED430167) Brown R., & Evans, W. P. (2002). Extracurricular activity and ethnicity: creating greater school connection among diverse student populations. Urban Education, 37, 263-‐286. Bulten, E., Nijman, H., & van der Staak, C. (2009). Measuring criminal thinking styles: The construct validity and utility of the PICTS in a Dutch prison sample. Legal and Criminal Psychology, 14, 35-‐49. Chung, H. L., & Steinberg, L. (2006). Relations Between Neighborhood Factors, Parenting Behaviors, Peer Deviance, and Delinquency Among Serious Juvenile Offenders. Developmental Psychology, 42 (2), 319-‐331. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Second Edition. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Cohen, J. (1992). Quantitative methods in psychology: A power primer. Psychological Bulletin, 112(1), 155-‐159. Costello, A. J., Edelbrock, C., Kalas, R., Kessler, M. D. & Klaric, S. H. (1982). The NIMH Diagnostic Interview Schedule for Children (DISC). Unpublished interview schedule, Department of Psychiatry, University of Pittsburgh. Crone, E. A., & Dahl, R. E. (2012). Understanding adolescence as a period of social–affective engagement and goal flexibility. Nature Reviews Neuroscience,13(9), 636-‐650.

39

Derogatis, L. & Melisaratos, N. (1983). The Brief Symptom Inventory: An introductory report. Psychological Medicine, 13, 595-‐605. Dolan, M. (2012). The neuropsychology of prefrontal function in antisocial personality disordered offenders with varying degrees of psychopathy. Psychological Medicine, 1(1), 1-‐11. Duckworth, A. L., Quinn, P. D., Lynam, D. R., Loeber, R., & Stouthamer-‐Loeber, M. (2011). Role of test motivation in intelligence testing. Proceedings of the National Academy of Sciences, 108(19), 7716-‐20. Elliott, D. S., Wilson, W. J., Huizinga, D., Sampson, R. J., Elliott, A., & Rankin, B. (1996). The effects of neighborhood disadvantage on adolescent development. Journal of Research in Crime and Delinquency, 33(4), 389-‐426. Forth, A. E., Kosson, D. S., & Hare, R. D. (2003). The Psychopathy Checklist: Youth Version. Toronto, ON: Multi-‐Health Systems. Gendreau, P., Goggin, C., & Smith, P. (2002). Is the PCL-‐R really the ‘‘unparalleled’’ measure of offender risk? A lesson in knowledge accumulation. Criminal Justice and Behaviour, 29, 397–426. Hare, RD (1991). The Hare Psychopathy Checklist-‐Revise. Niagara Falls, NY: Multi-‐Health Systems. Heilbrun, K. (1997). Prediction versus management models relevant to risk assessment: The importance of legal decision-‐making context. Law and Human Behavior, 21, 347–359. Kim-‐Cohen, J., Moffitt, T. E., Taylor, A., Pawlby, S. J., & Caspi, A. (2005). Maternal depression and children’s antisocial behavior: nature and nurture effects. Archives of General Psychiatry, 62(2), 173-‐181. Kroner, D., Mills, J., & Morgan, B. (2005). A coffee can, factor analysis, and prediction of antisocial behavior: The structure of criminal risk. International Journal of Law & Psychiatry, 28, 360–374. McGee, R. A., Wolfe, D. A., & Wilson, S. K. (1997). Multiple maltreatment experiences and adolescent behavior problems: Adolescents’ perspectives. Development and Psychopathology, 9(1), 131–149. Melchert, T. P. (1998). Testing the validity of an instrument for assessing family of origin history. Journal of Clinical Psychology, 54(7), 863-‐875.

40

Miller, Franklin G.; Lazowski, Linda E. (2005). Substance Abuse Subtle Screening Inventory for Adolescents-‐Second Version. In Grisso, Thomas (Ed); Vincent, Gina (Ed); Seagrave, Daniel (Ed). (2005). Mental health screening and assessment in juvenile justice. (pp. 139-‐151). New York, NY, US: Guilford Press. xvii, 397 pp. Monahan, K. C., Steinberg, L., & Cauffman, E. (2009). Affiliation with antisocial peers, susceptibility to peer influence, and desistance from antisocial behavior during the transition to adulthood. Developmental Psychology, 45(6), 1520-‐1530. Morgan, R. D., Fisher, W. H., Duan, N., Mandracchia, J. T., & Murray, D. (2010). Prevalence of criminal thinking among state prison inmates with serious mental illness. Law & Human Behavior, 34, 324-‐336. Muris, P., Merckelbach, H., Ollendick, T. H., King, N. J., & Bogie, N. (2002). Three traditional and three new childhood anxiety questionnaires: their reliability and validity in a normal adolescent sample. Behaviour Research and Therapy, 40(7), 753-‐772. Myers, K. M., & Winters, N. C. (2002). Ten-‐year review of rating scales, I: Overview of scale functioning, psychometric properties, and selection. Journal of the American Academy of Child & Adolescent Psychiatry, 4(2)1, 114–122. Newman, J. P., & Kosson, D. S. (1986). Passive avoidance learning in psychopathic and nonpsychopathic offenders. Journal of Abnormal Psychology, 95(3), 257-‐263. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-‐Hill. Odgers, C. L., Reppucci, N. D., & Moretti, M. M. (2005). Nipping psychopathy in the bud: An examination of the convergent, predictive, and theoretical utility of the PCL:YV among adolescent girls. Behavioral Sciences and the Law, 23, 743–763. Ogilvie, J. M., Stewart, A. L., Chan, R. C. K., & Shum, D. (2011). Neuropsychological measures of executive function and antisocial behavior: A meta-‐analysis. Criminology, 49, 1063–1107. Patrick, C. J. (2010). Description of brief scales for assessment of boldness, meanness, and disinhibition. Unpublished manual. Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Erlbaum. Petersen, A. C., Crockett, L., Richards, M., & Boxer, A. (1988). A self-‐report measure of pubertal status: Reliability, validity, and initial norms. Journal of Youth and Adolescence, 2, 117-‐133. Reynolds, C. R., & Richmond, B. O. (1997). What I Think and Feel: A revised measure of children’s manifest anxiety. Journal of Abnormal Child Psychology, 25(1), 15-‐20.

41

Sanzen (2012). Tower of London Test. Software, manual, and normative data available at: https://neuropsychological-‐assessment-‐tests.com/sanzen-‐tower-‐london-‐test Schmitz, K. E., Hovell, M. F., Nichols, J. F., Irvin, V. L., Keating, K., Simon, G. M., et al. (2004). A validation study of early adolescents’ pubertal self-‐assessments. Journal of Early Adolescence, 24, 357–384. Schubert, C. A., Mulvey, E.P., Steinberg, L., Cauffman, E., Losoya, S., Hecker, T., Chassin, L., et al. (2004). Operational Lessons from the Pathways to Desistance Project. Youth Violence and Juvenile Justice, 2 (3), 237-‐255. Shallice T (1982). Specific impairments in planning. Philosophical Transactions of the Royal Society of London 298, 199–209. Simourd, L., & Andrews, D. A. (1994, January). Correlates of delinquency: A look at gender differences. In Forum on Corrections Research (Vol. 6, No. 1, pp. 26-‐31). Skeem, J. L., Hernandez, I., Kennealy, P. J. & Rich, J. (June, 2011). CA-‐YASI Reliability: How adequately do staff in California’s Division of Juvenile Justice rate youths’ risk of recidivism? Prepared for the California Division of Juvenile Justice. Skeem, J. L., Miller, J. D., Mulvey, E., Tiemann, J., & Monahan, J. (2005). Using a five-‐factor lens to explore the relation between personality traits and violence in psychiatric patients. Journal of Consulting and Clinical Psychology, 73(3), 454-‐465. Skeem, J., & Monahan, J. (2011). Current directions in violence risk assessment. Current Directions in Psychological Science, 20, 38-‐42. Skeem, J., & Mulvey, E. (2001). Psychopathy and community violence among civil psychiatric patients: Results from the MacArthur Violence Risk Assessment Study. Journal of Consulting and Clinical Psychology, 69, 1-‐23. Skeem, J.L., Schubert, C.A., Odgers, C., Mulvey, E.P., Gardner, W. Lidz, C. (2006) Psychiatric Symptoms and Community Violence Among High-‐risk patients: A test of the relationship at the weekly level. Journal of Consulting and Clinical Psychology, 74(5), 967-‐979. Steinberg, L. (2010). A dual systems model of adolescent risk-‐taking. Developmental Psychobiology, Special Issue: Psychobiological models of adolescent risk, 52(3), 216-‐224. Steinberg, L., Albert, D., Cauffman, E., Banich, M., Graham, S., & Woolard, J. (2008). Age differences in sensation seeking and impulsivity as indexed by behavior and self-‐report: Evidence for a dual systems model. Developmental Psychology, 44(6), 1764-‐1778.

42

Thornberry, T. P., Lizotte, A. J., Krohn, M. D., Farnworth, M. & Jang, S. J. (1994). Delinquent peers, beliefs, and delinquent behavior: A longitudinal test of interactional theory. Criminology, 32: 47-‐83. Walters, G. D. (2003). Predicting institutional adjustment and recidivism with the Psychopathy Checklist factor scores: A meta-‐analysis. Law and Human Behavior, 27(5), 541-‐558. Walters, G. D. (2006c). Proactive and reactive composite scales for the Psychological Inventory of Criminal Thinking Styles (PICTS). Journal of Offender Rehabilitation, 42, 23-‐36. Walters, G. D. (2012). Criminal thinking and recidivism: Meta-‐analytic evidence on the predictive and incremental validity of the Psychological Inventory of Criminal Thinking Styles (PICTS). Aggression and Violent Behavior. Walters, G. D., Frederick, A. A., & Schlauch, C. (2007). Postdicting arrests for instrumental and reactive aggression with the PICTS Proactive and Reactive composite scales. Journal of Interpersonal Violence, 22, 1415-‐1430. Walters, G. D., & Geyer, M. D. (2005). Construct validity of the Psychological Inventory of Criminal Thinking Styles in relationship to the PAI, disciplinary adjustment, and program completion. Journal of Personality Assessment, 84, 252-‐260. Wechsler, D. (1999). Wechsler Abbreviated Scale of Intelligence. The Psychological Corporation: Harcourt Brace & Company. New York, NY.

43

Appendix A: Results Using Orbis Scoring System YASI Total Score (Risk – Protective) Criterion Measure YASI Total Score PCL:YV Total .34*** PCL:YV Social Deviance .45*** PCL:YV Int-‐Affective .14* Somatization .14* Head Injury .02 Low IQ .33*** Low Pubertal .23*** Section A: Legal History Criterion Measure Static PCL:YV Antisocial .44*** Conduct Disorder Symptoms .12* Somatization .08 Head Injury -‐.04 Low IQ .13* Low Pubertal .06 Section B: Correctional Response Criterion Measure Static PCL:YV Antisocial .52*** AWOL/Escape .35*** Somatization .15* Head Injury .05 Low IQ .19** Low Pubertal .00 Section C: Aggression-‐Violence Criterion Measure Static Dynamic Protective SIP Intent .06 .03 -‐.09 SIP Response .08 .03 -‐.10 SIP Attitudes .20** .17** -‐.16** Hostility .14* .14* -‐.06 Meaness .15* .13* .18** PCL:YV Social Deviance .25*** .22*** -‐.28*** Somatization .05 .03 -‐.10 Head Injury -‐.02 .04 -‐.09 Low IQ .19** .14* .19** Low Pubertal .06 .17** -‐.18**

44

Section D: Social Influences Criterion Measure Dynamic Protective PDB Behavior .22*** -‐.23*** PDB Influence .06 -‐.06 Neighborhood Disorg. .25*** -‐.07 Somatization -‐.06 -‐.02 Head Injury .07 -‐.04 Low IQ .23*** -‐.28*** Low Pubertal .09 -‐.12* Section E: Substance Abuse Criterion Measure Static Dynamic SASSI Alcohol .02 .08

SASSI Drug .02 .10 Somatization .02 .00 Head Injury .09 -‐.01 Low IQ .01 .25*** Low Pubertal .00 .14* Section F: Attitudes Criterion Measure Dynamic Protective PICTS Reactive -‐.07 -‐.04 PICTS Proactive -‐.20* -‐.04 SIP attitudes .16** -‐.21** Somatization .03 -‐.07 Head Injury .02 .00 Low IQ .22** -‐.29*** Low Pubertal .15** -‐.18** Section G: Social-‐Cognitive Skills Criterion Measure Dynamic Protective TOL errors .15* -‐.17* TOL Impulsivity .00 -‐.04 Go/No Go -‐.06 08 Low IQ .20** .18** Somatization .12* -‐.10 Head Injury -‐.05 .01 Low Pubertal .14* -‐.18**

45

Section H: Family Criterion Measure Static Dynamic Protective FBQ Psychological -‐.08 -‐.07 .18** FBQ Physical -‐.03 -‐.02 05 FBQ Dom Vio -‐.01 .04 .00 Family Monitoring -‐.07 -‐.20*** .11 Somatization .02 .14* -‐.14* Head Injury .03 .09 .02 Low IQ .03 -‐.14* .18** Low Pubertal -‐.05 .05 -‐.18** Section I: Education/Employment Criterion Measure Static Dynamic Protective School Connection .03 -‐.12 -‐.01 PCL:YV Interpersonal-‐Lifestyle

.14* .17** -‐.17**

Low IQ .26*** .19** .19** Somatization .12* 18** -‐.19** Head Injury -‐.04 -‐.02 .00 Low Pubertal .01 .30*** -‐.22*** Section J: Health Criterion Measure Dynamic BSI Total Symptoms .09 RCMAS Anxiety .07 Head Injury -‐.06 Low IQ .07 Low Pubertal .05 Section L: Community Stability Criterion Measure Dynamic Protective Community Disorganization .07 -‐.11 Somatization -‐.03 -‐.14* Head Injury -‐.06 .20*** Low IQ .15* -‐.12 Low Pubertal .04 -‐.08

46

47

Appendix B: Psychometrics of Criterion Measures

Brief Symptom Inventory (BSI) The BSI (Derogatis & Melisaratos, 1983) is a 53 item self report measure of mental health problems and psychiatric distress. It has been used in the Pathways to Desistance Study, the largest longitudinal study of serious adolescent offenders conducted to date (see http://www.pathwaysstudy.pitt.edu/codebook/bsi-‐sf.html ). The BSI manifests good test-‐retest reliability (r = .68-‐.91), internal consistency (alpha = .71-‐.85) and concurrent validity with the Minnesota Multiphasic Personality Inventory (e.g., Derogatis & Melisaratos, 1983). The BSI is also sensitive to change in symptoms over time (for a review, see Skeem, Schubert, Odgers, Mulvey, Gardner & Lidz, 2005). In the present study, we used Total BSI scores as a measure of general mental health problems and psychiatric distress. We also used two BSI subscales that consistently emerge in factor analyses of the measure, and have been shown to manifest discriminant validity (unlike some other BSI subscales; see Skeem et al., 2006): Somatization and Hostility. The BSI Hostility scale has been shown to assess anger and predict violence among high risk individuals (see Skeem, Schubert, et al., 2005).

Communities that Care (CTC) We used two scales from the CTC (Arthur et al., 2002): Community and Family. These scales consist of 26 self report items that assess the risk for potential social, drug and behavioral problems in family and local community settings. Reliability is suggested by adequate internal consistency across all eight of the measure’s domains in samples of early-‐ and mid-‐adolescents. Validity of CTC is suggested by correlations of scales with outcome measures including substance use and delinquent behavior.

Conduct Disorder This self report scale assesses the lifetime prevalence and frequency of involvement in delinquent activities including theft, assault and public disorder. Consistent with the work of Kim-‐Cohen and colleagues (2005), questions were drawn from the Child Behavior Checklist (Achenbach, 1991), Teacher’s Report Form (Achenbach, 1991) and Diagnostic Interview Schedule for Children (Costello, Edelbrock, Kalas, Kessler, & Klaric, 1982) to ensure adequate coverage of all 15 CD symptoms. Previous research suggests that these items demonstrate good internal consistency and relate as they should to measures of other constructs (Kim-‐Cohen, et al., 2005).

Neighborhood Disorganization Uses census tract data to characterize participants’ neighborhood, based on their most recent home address. Neighborhood disorganization summarizes: per capita income (reverse scored), percent of households on public assistance, percent non-‐white only households, percent of female-‐headed households, and percent of people unemployed. Similar census-‐based

48

measures have been used in a variety of studies that demonstrate a link between disadvantage and crime/violence (e.g., Elliott, Wilson, Huizinga, et al., 1996).

Family Background Questionnaire (FBQ) We used an adapted version of the FBQ (see McGee, Wolfe, & Wilson, 1997) to assess exposure to maltreatment and domestic violence. This 28-‐item scale includes subscales that index physical abuse, psychological abuse (e.g., harsh criticism; lack of affection; exposure to criminal behavior), “How often and exposure to domestic violence. Consistent with previous research (Odgers, Reppucci & Moretti, 2005), the three subscales mentioned were used to form a cumulative index of abuse. Because the subscales had unequal numbers of items, each was standardized prior to summing the total score (M Z score = -‐.01, SD = 2.50). The FBQ has demonstrated test-‐retest reliability (.96 over a two-‐week interval; Melchert, 1998). Validity is suggested by the pattern of relations with criterion-‐related variables including history of incest, parental chemical dependency, clinical status, socioeconomic status, and birth order (Melchert, 1998).

Go/No Go To assess capacity for punishment-‐based learning (AKA passive avoidance learning), a GoNoGo Task (GNG; Newman & Kosson, 1986) was administered via a laptop computer. Participants were presented pairs of images (colored drawings of animals) on the computer screen. The task featured two blocks, each featuring 50 trials. For each pair, the youth was asked to select one of the images by pressing a keyboard button and received either positive (win 100 points) or negative (lose 100 points) feedback. The feedback received was dependent on each image’s reinforcement probability. During the first half of a block’s trials, one image was positively reinforced 80 percent of the time and the other image was reinforced 20 percent of the time. Approximately half way through a block the reinforcement ratios were switched. The outcome of interest on this task was the number of times the participant pressed the button for punished response (an error of commission). Among measures of executive function, Go/No Go scores relate relatively strongly to antisocial behavior and traits (d = .55; Ogilvie, Steward, Chan, & Shum, 2011).

Head Injury (HI) The HI (Schubert et al., 2004) is a 4-‐item self report measure of the presence and seriousness of brain injury for adolescents. It was developed by researchers on the Pathways To Desistance Study in collaboration with two established neuropsychologists, and relates in a theoretically coherent manner to other variables assessed in that study.

Meanness Meanness (Patrick, 2010) is a 19-‐item self report scale designed to gauge tendencies “…toward callousness, cruelty, predatory aggression, and excitement seeking” (Patrick, 2010, p. 2). The scale has been shown to be internally consistent, and to manifest a coherent pattern of relationships with measures of psychopathic and other personality features.

49

Peer Delinquent Behavior Scale (PDBS) The PDBS (Thornberry, Lizotte, Krohn, Farnworth, & Jang, 1994) is a 19-‐item self-‐report measure which captures two aspects of peer delinquency: antisocial peer behavior (e.g., “How many of your friends have sold drugs?”) and antisocial peer influence (“How many of your friends have suggested that you should sell drugs?”). The scale has high rates of internal consistency (alpha = .90; Chung & Steinberg, 2006), and manifests a theoretically coherent pattern of associations with individual offending and neighborhood social factors (Chung & Steinberg, 2006; Monahan, Steinberg, & Cauffman, 2009).

Psychological Inventory of Criminal Thinking Styles (PICTS) The short form of the PICTS (Walters, 2006) is a 35-‐item self-‐report measure of thinking styles considered essential to the maintenance of a criminal lifestyle. This form assesses (a) Proactive Criminal Thinking including entitlement (e.g., “On the streets I would tell myself I needed to rob or steal in order to live the life I had coming“) and Reactive Criminal Thinking including “cutoff,” (e.g., “When pressured by life’s problems, I’ve said ‘the hell with it,’ and followed up by using drugs or engaging in crime”). Good internal consistency was observed in the present study (Reactive α = .84; Proactive α = .82), and the short form has manifested good test-‐retest reliability in past research (Walters, 2006). The full PICTS manifests a theoretically coherent pattern of relationships with general personality traits and internalizing/externalizing disorders (Bulten, Nijman, & van der Staak, 2011; Walters & Guyer, 2005). The short PICTS manifests a similar pattern of relationships with measures of antisocial and psychopathic traits (Walters, 2006). The Proactive and Reactive scales assessed by the short form significantly and robustly predict general recidivism (Walters, 2012), and are moderately associated with a validated measure of the content of criminal thinking (the CSSM; r= .24 and .21, p<.001; data from Morgan et al., 2001). There is weaker evidence that the two scales differentially postdict instrumental and reactive aggression, as well (Walters, Frederick, & Schlauch, 2007).

Psychopathy Checklist, Youth Version (PCL:YV) The PCL: YV (Forth, Kosson, & Hare, 2003) is a 20-‐item rating scale that is completed based on a clinical interview and review of relevant file information. Each item is rated as “0” (does not apply), “1” (applies somewhat), or “2” (definitely applies) by trained raters. The PCL: YV renders a total score (M = 21.91, SD = 6.85) and Interpersonal-‐Affective (M = 7.32, SD = 3.40) and Social Deviance (M = 13.02, SD = 4.01) scale scores. High reliability is suggested by alpha and kappa coefficients and test-‐retest reliability correlations as reported in the PCL: YV Manual (Forth et al., 2003). Validity is suggested by concurrent associations with personality-‐based and lab measures along with its predictive utility for criminal justice outcome as reported in the PCL: YV Manual (Forth et al., 2003). Before conducting PCL: YV interviews, raters completed PCL: YV training. This included reviewing and scoring several practice videotapes. When treating the project coordinator’s ratings as the criterion, acceptable levels of inter-‐rater reliability on total scores (Mean ICC = .89) were obtained for 4 cases, with each rater (n = 5) reaching an ICC of at least .80. To ensure reliable PCL: YV interviewing and scoring during the course of the study, the project coordinator (the author) made secondary PCL: YV ratings based on audio recordings of interviews and file review notes. When treating the project coordinator’s ratings as the

50

criterion, acceptable levels of inter-‐rater reliability were observed for total (ICC = .81), Interpersonal-‐Affective (ICC = .78) and Social Deviance (ICC = .88) scores for 10 cases.

Pubertal Development Scale (PDS) The PDS (Petersen, Crockett, Richards, & Boxer, 1988) is a four-‐item self-‐report measure assessing pubertal status via questions about perceived changes in skin, height, underarm hair, and voice. Each item has four options (has not yet started, barely started, definitely started but not finished, and definitely completed). Strong reliability is indicated by alpha coefficients (range = .68 to .83; average across items = .77) and minimal regressions in score magnitude across repeated administrations (Petersen et al., 1988). Construct validity of the PDS is suggested by significant correlations with physician examinations (rs: .61-‐.67) and the Tanner staging measure, which is based on physician examination (Schmitz et al., 2004). Notably, the relations among pubertal status (including testosterone changes), frontal-‐cortical functioning, and risk-‐taking behavior are complex (see Crone & Dahl, 2012). Although puberty may play a role in antisocial behavior, this may only be true in certain social contexts. In short, there is little reason to expect any strong bivariate relationships between pubertal status and CA-‐YASI domains.

Revised Childhood Manifest Anxiety Scale (RCMAS) The RCMAS (Reynolds & Richmond, 1985, 2000) is a 37-‐item self-‐report instrument designed to measure the nature and level of anxiety. Reliability is indicated by high internal consistency (alpha = .89; Reynolds & Richmond, 1997) and good test-‐retest reliability (rs range: 0.63-‐0.85; Myers & Winters, 2002). Validity is indicated by high correlation with another validated measure of anxiety -‐ the STAIC (r = .88) – and a theoretically coherent pattern of relations with other measures (Muris, Merckelbacj, Ollendick, King, & Bogie, 2002).

Substance Abuse Subtle Screening Inventory-‐A2 (SASSI-‐A2) The SASSI-‐A2 (Miller & Lazowski, 2005) is a self-‐report measure for substance use for adolescents. We used 32 items of the SASS-‐A2 that comprise the Face Valid Alcohol and Face Valid Other Drugs scales, which were shown to identify individuals diagnosed by clinicians with alcohol dependence or other drug dependence disorders, respectively. These scales have demonstrated reliability (internal consistency and test-‐retest all above .90) and validity (prediction of dependence disorders; accuracy of 94%), in both criminal and non-‐criminal samples.

School Connection Scale (SCS) The SCS (Brown, 1999) is a 16-‐item self report measure that gauges school commitment and belongingness. Research indicates suggest reliability with good internal consistency (alpha = .86) and validity via predictive utility for academic performance and participation in extracurricular activities (Brown & Evans, 2002).

51

Social Information Processing (SIP) scale The SIP (Bradshaw, Rodgers, Ghandour, & Garbarino, 2009) is designed to assess hostile attribution bias, a tendency to attribute hostile intent to peers. The SIP involves presenting four hypothetical social situations to the youth, who is asked to imagine being personally involved (e.g., “Imagine that a peer hits you hard in the back with a ball”). For each vignette, the tester asks open-‐ended questions designed to assess hostile attribution bias (e.g., “why do you think he did this?”) and aggressive responses (e.g., “what woud you do?”). Trained raters than review youth’s responses across vignettes to rate “Intent” and “Response.” High inter-‐rater reliability has been manifested in in prior research (ICC=.81-‐.94; Bradshaw et al., 2009) and was observed in the present study, based on 6 training cases (ICC = .80-‐.95). The SIP also includes four self report items that assess attitudes supportive of aggressive behavior (e.g., “it’s ok for me to hit someone if they hit me first”; alpha in this study=.70). The SIP has been shown to relate to theoretically relevant variables (like exposure to community violence) and to predict aggressive behavior (Bradshaw et al., 2009).

Tower of London (ToL) The ToL (Shallice, 1982) is a neuropsychological test commonly used to assess executive functioning, including problems with planning, cognitive flexibility, and working memory. We used a computerized version of the ToL with 21 trials (Sanzen, 2012; formerly Colorado Assessment Tests). On each trial, the participant is required to move colored beads across pegs of different length from a current arrangement (on the left side of the screen) to match a goal arrangement (on the right side of the screen). The images for the first trial are shown below. The participant’s task is to re-‐arrange the balls in the work space to match the goal state as quickly and efficiently as possible. Over 21 trials, problems increase in difficulty (with additional balls and rods). Each trial has a specified minimum number of moves (from 2 to 5). We calculated two scores: (1) the average excess moves above the minimum required to solve the problem, across trials; and (2) the average time to the first move, across trials, in milliseconds (Berg & Byrd, 2002; Steinberg, Albert, Cauffman, Banich, Graham, & Woolard, 2008). Although the ToL assesses abilities that are not specific to executive function (like visual spatial reasoning), it has some support for tapping cognitive flexibility and working memory. ToL scores are also significantly associated with antisocial behavior (Dolan, 2012; see also Steinberg, 2010).

Wechsler Abbreviated Intelligence Scale (WASI) The version of the WASI (Wechsler, 1999) used in this study estimates general intellectual ability based on two (Vocabulary and Matrix Reasoning) subtests. High reliability is indicated by subtest level alpha coefficients (range = .84-‐.98) and high test-‐retest reliability correlations across 2 to 12 weeks (rs = 0.87 to 0.92). IQ estimates based on the two-‐subtest version of the

52

WASI are very strongly correlated with those based on full intelligence tests like the WISC-‐III (r=.81; Wechsler, 1999).

53

ADDENDUM: Testing domain components created post hoc by Orbis After the first draft of this report was distributed, DJJ and Orbis asked that we conduct further analyses to determine whether the construct validity of the CA-‐YASI could be improved by isolating a more homogeneous sub-‐set of items or a “component” within each domain. We agreed to conduct these analyses for one domain -‐-‐ Aggression-‐Violence -‐-‐ which is most relevant to DJJ-‐policy. Months later, Orbis sent us a list of not one, but two item sub-‐sets within the Aggression-‐Violence domain. They also listed a number of additional item sub-‐sets for five other domains. It is not feasible for us to test multiple item subsets within six different domains (and arguably would be inappropriate, given the post hoc nature of this request – knowledge about the specific measures used to assess construct validity allows Orbis to select items that specifically match those measures, rather than aiming for an abstract construct). Nevertheless, we did as Orbis asked for the first three domains. We believe this provides a fair and representative test of potential improvement in the tool’s construct validity, with selection of item subsets. Orbis believed that the components they created within CA-‐YASI domains (as “micro-‐constructs”) would relate more strongly to the concurrent measures selected for this evaluation than the original domains (as “broad categories”). This was not the case, as shown in Tables X, Y and Z. If anything, the newly created components perform more poorly than the original domains. This helps rule out the possibility that low estimates of construct validity for CA-‐YASI domains is a function of domain multidimensionality (for the domains tested, there was no evidence of “micro-‐constructs”). Our basic conclusion remains – with the two exceptions outlined in Table 15 (i.e., Substance Abuse, and mental Health) -‐-‐ this study provides little evidence that the CA-‐YASI domains (or even components within domains, identified posthoc) validly assess the variable risk factors or constructs after which they are named. Table X. Aggression/Violence Domain, post hoc results Original

Aggression/Violence Domain

Post-‐hoc Anger Component (items 14, 15, 18)

Post hoc Aggression Component (items 16, 17, 19)

SIP Intent .07 .11 .05 SIP Response .08 .11 .06 SIP Attitudes .23*** .14* .22** Hostility .15* .16* .13 Meanness .24** .15* .23** Soc. Deviance .36*** .26*** .27***

54

Somatization .06 .07 .05 Head Injury .06 .02 .03 Low IQ .28*** .17* .23** Low Pubertal Dev. .14* .16* .20** Notes: n = 180-‐235. *** p < .001, ** p < .01, * p < .05. Table Y. Attitudes Domain, post hoc results Original

Attitudes Domain

Post-‐hoc Responsibility Component (items 1, 2)

Post-‐hoc Law Abiding Component (items 3, 4, 5)

PICTS Reactive -‐.18 -‐.19 -‐.20 PICTS Proactive .05 -‐.02 .08 SIP Attitudes .16* .11 .19** Somatization .09 .07 .09 Head Injury .03 -‐.03 .05 Low IQ .23*** .21** 24** Low Pubertal Dev. .20* .18** .17* Notes: n = 86-‐235. *** p < .001, ** p < .01, * p < .05. Table Z. Social-‐Cognitive Domain, post hoc results Original

Social Cognitive Domain

Post hoc Impulsivity Component (items 1, 2, 4, and 8)

Post hoc Interpersonal Component (items 3, 6, 7)

ToL Errors .17* .17* .13 ToL Impulsivity .05 .03 .04 Go/No Go -‐.11 -‐.09 -‐.11 Low IQ .20* .23** .12 Somatization .12 .1 .12 Head Injury -‐.03 -‐.06 .02 Low Pubertal Dev. .17* .18** -‐.16* Notes: n = 190-‐235. *** p < .001, ** p < .01, * p < .05.

Documents

PhaseII Report Draft - Risk Resilience Researchrisk-resilience.berkeley.edu/sites/default/files/... · ! 4! Executive)Summary! Context’ The!mission!of!the!CaliforniaDepartmentof!Juvenile!Justice!(DJJ)!is!to!protectpublic!safety,!