28
A Revision of the Multicultural Counseling Awareness Scale Joseph G. Ponterotto, Denise Gretchen, Shawn 0. Utsey, Brian P. Rieger, and Richard Austin This article reports the results of 2 studies designed to test and revise the Multicultural Counseling Awareness Scale (J. G. Ponterotto et al., 1996). Collective results support the 2-factor extraction (Knowledge and Aware- ness) as the best fit model and provide initial indices of validity and internal consistency reliability for the newly titled Multicultural Counseling Knowl- edge and Awareness Scale. Este articulo informa 10s resultados de 2 estudios diseriados para probar y revisar la Escala de Conciencia de Consejeria Multicultural (J. G. Ponterotto et al., 1996). Los resultados colectivos sostienen la extraccion de 2 factores (el Conocimiento y la Conciencia) como el mejor modelo y proporcionan 10s indices iniciales de validez y fiabilidad de la consistencia interna para la nuevamente titulada Escala de Conocimiento y Conciencia de Consejeria Multicultural. he 1990s witnessed multicultural counseling rising to assume, perhaps, the central core of the counseling profession’s identity. In fact, a recent T Delphi Poll on the future of counseling psychology over the next 10 years in the United States identified “commitment to issues of diversity” as the single greatest core identification of the profession through the year 2010 (Neimeyer & Diamond, 2001, p. 57). One vibrant research focus within this core identity is the subject of multicultural counseling competency develop- ment and assessment (Constantine & Ladany, 2001; Fuertes, Bartolomeo, & Nichols, 2001; Pope-Davis & Coleman, 1997; Pope-Davis, Coleman, Liu, & Toporek, in press; Sue et al., 1998). Despite the increasing use of multicultural competency measures in system- atic research and program evaluation, the collective set of measures is still considered to be in the early stages of development and validation (Ponterotto, Fuertes, & Chen, 2000). At present, there is a strong need to devote more focused attention to the testing and possible revision of paper-and-pencil self- report measures of multicultural counseling competence (Constantine & Ladany, Joseph G . Ponterotto, Denise Gretchen, and Richard Austin, Counseling Psychology Program, Fordham University at Lincoln Center; Shawn 0. Utsey, Howard University; Brian f! Rieger, State University of New York Health Science Center, Syracuse. Correspondence concerning this article should be addressed to Joseph G. Ponterotto, Division ofPsychologica1 BEducationalServices, Room 1008, Fordham University at Lincoln Center, 113 West 60th Street, New York, N Y 10023-7478 (e-mail:[email protected]). JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 153

A Revision of the Multicultural Counseling Awareness Scale

Embed Size (px)

Citation preview

Page 1: A Revision of the Multicultural Counseling Awareness Scale

A Revision of the Multicultural Counseling Awareness Scale

Joseph G. Ponterotto, Denise Gretchen, Shawn 0. Utsey, Brian P. Rieger, and Richard Austin

This article reports the results of 2 studies designed to test and revise the Multicultural Counseling Awareness Scale (J. G. Ponterotto et al., 1996). Collective results support the 2-factor extraction (Knowledge and Aware- ness) as the best fit model and provide initial indices of validity and internal consistency reliability for the newly titled Multicultural Counseling Knowl- edge and Awareness Scale.

Este articulo informa 10s resultados de 2 estudios diseriados para probar y revisar la Escala de Conciencia de Consejeria Multicultural (J. G. Ponterotto et al., 1996). Los resultados colectivos sostienen la extraccion de 2 factores (el Conocimiento y la Conciencia) como el mejor modelo y proporcionan 10s indices iniciales de validez y fiabilidad de la consistencia interna para la nuevamente titulada Escala de Conocimiento y Conciencia de Consejeria Multicultural.

he 1990s witnessed multicultural counseling rising to assume, perhaps, the central core of the counseling profession’s identity. In fact, a recent T Delphi Poll on the future of counseling psychology over the next 10

years in the United States identified “commitment to issues of diversity” as the single greatest core identification of the profession through the year 2010 (Neimeyer & Diamond, 2001, p. 57). One vibrant research focus within this core identity is the subject of multicultural counseling competency develop- ment and assessment (Constantine & Ladany, 2001; Fuertes, Bartolomeo, & Nichols, 2001; Pope-Davis & Coleman, 1997; Pope-Davis, Coleman, Liu, & Toporek, in press; Sue et al., 1998).

Despite the increasing use of multicultural competency measures in system- atic research and program evaluation, the collective set of measures is still considered to be in the early stages of development and validation (Ponterotto, Fuertes, & Chen, 2000). At present, there is a strong need to devote more focused attention to the testing and possible revision of paper-and-pencil self- report measures of multicultural counseling competence (Constantine & Ladany,

Joseph G . Ponterotto, Denise Gretchen, and Richard Austin, Counseling Psychology Program, Fordham University at Lincoln Center; Shawn 0. Utsey, Howard University; Brian f! Rieger, State University of New York Health Science Center, Syracuse. Correspondence concerning this article should be addressed to

Joseph G. Ponterotto, Division ofPsychologica1 BEducationalServices, Room 1008, Fordham University at Lincoln Center, 113 West 60th Street, New York, N Y 10023-7478 (e-mail: [email protected]).

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 153

Page 2: A Revision of the Multicultural Counseling Awareness Scale

2000,2001; Kocarek, Talbot, Batka, & Anderson, 2001 ; Worthington, Mobley, Franks, & Tan, 2000).

The purpose of the present series of studies was to revise and test the validity of a popular measure of multicultural counseling competencies-the Multicultural Counseling Awareness Scale (MCAS; Ponterotto et al., 1996). Our introduc- tion begins with a brief overview of competency measurement in multicultural counseling, followed by a review of the development and uses of the MCAS. The section closes with a review of criticisms and research needs on the MCAS and with a delineation of the specific goals of the present sequence of studies.

measurement of multicultural c o & ~ competence

Landmark work on multicultural competency assessment was conducted by LaFromboise and colleagues with their development and validation of the Cross- Cultural Counseling Inventory (CCCI; Hernandez & LaFromboise, 1985) and the Cross-Cultural Counseling Inventory-Revised (CCCI-R; LaFromboise, Coleman, & Hernandez, 1991). The CCCI-R is a 20-item instrument com- pleted by an evaluator who observes a counselor working with a client of a different racial/ethnic background.

Subsequent to work on the observer-report format of the CCCI, research began on the development of counselor self-report assessments of perceived multicultural competence. In the early 199Os, three geographically dispersed research teams, working independently, developed the following self-report instruments: the Multicultural Awareness/Knowledge/Skills Survey (MAKSS; D’Andrea, Daniels, & Heck, 1991) in Hawaii, the Multicultural Counseling Inventory (MCI; Sodowsky, 1996; Sodowsky, Taffe, Gutkin, &Wise, 1994) in Nebraska, and the Multicultural Counseling Awareness Scale (MCAS; Ponterotto et al., 1996; Ponterotto, Sanchez, & Magids, 1991) in New York.

A conceptual base for all three instruments was the frequently cited Sue et al. (1982) multicultural counseling competency report. This report defined multicultural counseling competence as consisting of three distinct, yet interre- lated, components: awareness of one’s own cultural socialization and accom- panying biases, knowledge of the worldviews and value patterns of cultur- ally diverse populations, and specific skills for intervention with these popu- lations. Continuing conceptual work on the competency report has resulted in periodic updates and expansions (Arredondo et al., 1996; Sue, Arredondo, & McDavis, 1992; Sue et al., 1998). Furthermore, the competency model has been integrated as a crucial component of the comprehensive working theory of multicultural counseling and therapy (Sue, Ivey, & Pedersen, 1996). A more complete developmental history of the competency model and accom- panying measures can be found in a series of review chapters (Constantine &

154 JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 3: A Revision of the Multicultural Counseling Awareness Scale

Ladany, 2001; Ponterotto et al., 2000; Pope-Davis & Dings, 1995; Pope-Davis & Nielson, 1996).

Because of agrowing interest in multicultural training (Pope-Davis & Coleman, 1997) and given the convenient self-report, anonymous format of multicultural competency measures, these assessment tools have reached more widespread use in the last half decade (Ponterotto et al., 2000; Pope-Davis & Dings, 1995). However, a number of researchers have raised strong caution in using and inter- preting these instruments, given their relatively recent development and the lack of validation and replication research available (Ponterotto et al., 2000). Psychometric reviews of the instruments (Constantine & Ladany, 2001; Ponterotto, Rieger, Barrett, &Sparks, 1994; Pope-Davis &Dings, 1995; Pope-Davis &Nielson, 1996), along with direct empirical comparisons of the measures (Constantine & Ladany, 2000; Kocarek et al., 2001; Pope-Davis & Dings, 1994; Worthington et al., 2000) suggest that although holding promise for research and training, the instruments need closer psychometric scrutiny and likely revision.

The present study focuses on the construct validity of the MCAS. This instru- ment was chosen for scrutiny for the following reasons: (a) It has been cited as a self-report measure with strong promise for multicultural competency research (Kocarek et al., 2001; Pope-Davis & Dings, 1995)) (b) it is of moderate length facilitating higher response rates (comparatively the MAKSS is 60 items), and (c) previous critiques of the instrument have provided clear directions for needed instrument revision (e.g., Kocarek et al., 2001; Pope-Davis &Dings, 1994, 1995).

early development and factor structure of the MCAS

The 45-item MCAS is a subject-centered scale (also known as individual dif- ference scales) in which scores reflect differences among respondents in terms of their standing on the scale’s dimensions (see Dawis, 1987). Likert-type scal- ing methods (Likert, 1932) were used in designing the MCAS (see Dawis, 2000, for a review of scaling methods).

Ponterotto et al. (1991) developed the MCAS using what Dawis (1987) la- beled the rational-empirical approach. The rational approach included initial item development and selection, independent card sorts, a focus group discus- sion of items, and a content validity assessment. The empirical approach in- volved item analysis and sequenced factor analytic procedures. The authors began with 135 items and through the rational approach reduced the pool to 70 items. Then placing the 70 items on a 7-point Likert-type scale, the authors recruited a sample of 126 counseling students and professionals for the empiri- cal assessment. Following item and factor analyses, the scale was reduced to its current 45 items. The MCAS includes (a) 28 Knowledge/Skills items that query both general multicultural knowledge and specific familiarity with the pub-

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 155

Page 4: A Revision of the Multicultural Counseling Awareness Scale

lished work of leading scholars in the area, (b) 14 Awareness items tapping a subtle Eurocentric worldview bias of the counseling relationship and thera- peutic goals, and (c) a 3-item social desirability cluster.

In their initial preparation of MCAS items, Ponterotto et al. (1991) attempted to tap into the three-dimensional model (awareness, knowledge, and skills) of multicultural competence posited by Sue et al. (1982). However, both rational (or qualitative) procedures (independent card sorts) and empirical (or quantita- tive) methods (principal component analyses) found that the items were best explained by a two-factor model, with the Knowledge and Skills items loading together and the Awareness items loading on their own subscale. Sue and his colleagues (Sue et al., 1992) have maintained that awareness (defined as beliefs and attitudes), knowledge, and skill are distinct constructs, although it is clear that the MCAS items did not capture the tripartite conceptualization.

To date, the MCAS has been used in a number of studies, and moderate to good support has been found for its psychometric properties. In six of seven samples, coefficient alpha for the Knowledge/Skills subscale was .90 or higher, and the alpha for the Awareness subscale was in the .70s or .80s for six of seven samples (Kocarek et al., 2001; Manese, Wu, & Nepomuceno, 2001; ear- lier studies reviewed in Ponterotto & Alexander, 1996). Regarding test-retest reliability, Manese et al. found 10-month stability coefficients of .70 for the Knowledge/Skills subscale and .73 for the Awareness subscale.

In support of convergent validity, Ponterotto et al. (1996) found the Knowledge/ Skills subscale (but not the Awareness subscale) to be significantly correlated with the self-report version of the CCCI-R (LaFromboise et al., 1991), a mea- sure of general multicultural knowledge. Also, as predicted, the Awareness subscale (but not the Knowledge/Skills subscale) was significantly correlated with the New Racism Scale, a popular measure of racial bias (Jacobson, 1985). Moderate to strong criterion-related validity has been found particularly for the Knowledge/Skills subscale through positive correlations with training vari- ables (Kocarek et al., 2001; Pope-Davis, Dings, & Ottavi, 1995; Pope-Davis, Reynolds, & Dings; 1994) and through gain scores in multicultural classes (Ponterotto et al., 1996) and internship training programs (Manese et al., 2001). Regarding construct validity support, Vinson and Neimeyer (2000) found that levels of multicultural Knowledge/Skills and Awareness were related to levels of racial identity development in theoretically expected directions.

Finally, the correlation between the Knowledge/Skills subscale and the Awareness subscale has been low to moderate, averaging .37 across studies (Ponterotto et al., 1996; Pope-Davis & Dings, 1995) and supporting the ob- lique nature of the factors. Ponterotto et al. (1996) found nonsignificant corre- lations between the MCAS subscales and the Marlowe-Crowne Social Desir- ability Scale (SDS; Crowne & Marlowe, 1960).

Despite some initial indices of psychometric support, a number of concerns have been raised regarding the MCAS. These concerns, echoed both by the

156 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 5: A Revision of the Multicultural Counseling Awareness Scale

scale developers themselves (e.g., Ponterotto et al., 1996; Ponterotto et al., 1994) and independent researchers (Kocarek et al., 2001; Pope-Davis & Dings, 1994, 1995), focus on four areas: (a) definitional clarity of the named subscales, (b) the inclusion of items that query knowledge of specific scholars in the field, (c) some weaker items (psychometrically speaking), and (d) the utility of the three-item social desirability cluster. A number of these researchers have cau- tioned that the initial two-factor model posited in Ponterotto et al. (1996) was based on a limited sample (N= 126) and have called for more rigorous factor- structure examination of the MCAS items.

The purpose of the present two-part study was first to examine the factor struc- ture underlying MCAS items, using a relatively large national sample; second, to revise the MCAS based on this analysis and in conjunction with previously cited concerns of the instrument; and third, to subject the revised instrument to both confirmatory factory analysis and tests of validity and reliability.

SAMPLE

Participants for this study included 525 students and professionals in counsel- ing and counseling psychology. Dawis (2000) highlighted that for exploratory factor analysis, large samples of at least 400 respondents are ideal. The mean age for the sample was 35 years (SD = 9.9); the median was 34 years, with the range from 21 to 69 years of age. The sample was composed of 66.5% (n= 349) women and 33.5% (n= 176) men. Eighty-three percent (n= 436) of the sample participants were White, 7% (n = 37) African American, 6% (n = 32) Hispanic American, 2% (n = 10) Asian American, 1% (n = 5) Native American, and 1% (n = 5) other. The gender and majority-minority sample breakdown matches very closely the disproportionate national data in the counseling profession (see Dinsmore & England, 1996; Ponterotto & Casas, 1991). The highest degree held by participants at the time of the study was as fol-

lows: 42% (n=220) held the bachelor’s degree, 39% (n= 205) held the master’s or post-master’s degree (nondoctorate), and 19% (n = 100) held the doctoral degree. Forty-five percent (n = 236) of the sample participants were enrolled in graduate school at the time of the survey; of this number, 83% (n = 196) were in master’s degree programs, with the (17%, n = 40) engaged in doctoral study or post-master’s specialty programs in counseling.

Regarding multicultural training in counseling, 33% (n= 173) had never com- pleted course work in the area, and 67% (n= 352) had completed at least some course work in multicultural counseling. Furthermore, 35% (n = 184) of the sample had completed specific training workshops on multicultural counsel- ing issues. Finally, 32% (n = 168) of the sample had received direct clinical supervision of their work with raciallethnic minority clients.

JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 157

Page 6: A Revision of the Multicultural Counseling Awareness Scale

INSTRUMENT

The MCAS (Ponterotto et al., 1996; Ponterotto et al., 1991) includes 45 items presented on a 7-point Likert-type scale. The following instruction precedes the items: “Using the following scale, rate the truth of each item as it applies to you.” The Likert-type range is from 1 (not at all true) to 7 (totally true); a midpoint anchor is also presented at 4 (somewhat true).

The Knowledge/Skills subscale includes 28 items, all positively worded. The majority of Knowledge/Skills items query the respondent’s knowledge of basic multicultural counseling issues (e.g., Item 12: “I am aware of the individual differ- ences that exist within members of a particular ethnic group based on values and beliefs, and level of acculturation”). Six of the Knowledge/Skills items ask about the respondent’s familiarity with an identified leading scholar in multicultural counseling (e.g., Item 1: “I am familiar with the research and writings ofJanet E. Helms and I can discuss her work at length spontaneously”). Rationales for the items as well as selection criteria for the six named items (leading scholars) are presented in Ponterotto et al. (1991).

The Awareness subscale includes 14 items, of which 10 are negatively worded (i.e., a low score indicates high awareness). An example of a negatively worded Awareness item is “I believe all clients should maintain direct eye contact during counseling” (Item 2). These items are reverse scored before all data analysis pro- cedures. Early pilot testing and content validity assessments (Ponterotto et al., 1991) found that Awareness items were more amenable to reverse wording. Fur- thermore, the Awareness items also measure a subtle Eurocentric bias (cf. Ponterotto & Casas, 1991), and the scale developers were concerned that these items would be more sensitive to social desirability contamination.

Finally, the MCAS includes three social desirability test items in an initial at- tempt to screen for nonvalid profiles. Two of the items measure general social desirability (e.g., Item 38: “At this point in my professional development, I feel I could benefit little from clinical supervision of my multicultural client caseload); and one item measures outright faking: “I am familiar with the research and writ- ings of Michael Santana-DeVio, and I can discuss his work at length spontane- ously’’ (Item 22); there is no such person.

Internal consistency coefficients (alpha) for the Knowledge/Skills subscale across seven studies ranged from .78 to .93, with a median alpha of .91. The coefficient alphas for the Awareness subscale across the same studies ranged from .67 to .83, with amedian alpha of .76 (see review in Kocarek et al., 2001; Manese et al., 2001; Ponterotto & Alexander, 1996). Convergent and criterion-related validity of the MCAS subscales has been presented and was reviewed earlier in this article.

PROCEDURE

Attempts were made to accrue a sample that was heterogeneous in terms of geographic location, degree level, professional experience level, and multicultural

158 JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 7: A Revision of the Multicultural Counseling Awareness Scale

training level. Graduate student subsamples (n = 287) were recruited from faculty contacts in five geographically dispersed counseling programs. Profes- sors were contacted by the first author and asked to distribute the instrument in select classes. To ensure good response rates, the professors were asked to distribute and collect the surveys during class time. The response rate from the varied classes was 100%. Subsamples of our data set were borrowed from Donald Pope-Davis (Pope-Davis et al., 1995; Pope-Davis et al., 1994) who studied MCAS scores of practicing professional counselors (n = 126, response rate = 60%) in Iowa and a national sample of predoctoral interns (n = 112, response rate = 64%) in counseling psychology. (Data borrowed from Donald B. Pope- Davis, director of the University of Notre Dame Multicultural Research Center, had not been subject to previous factor analysis examination.)

On the '/-point Likert-type scale used in the MCAS, the grand mean across the 45 items was 4.7 (SD= 1.4). Means and standard deviations for the Knowledge/ Skills subscale scores (M= 4.14, SD = 1.4) and the Awareness subscale scores (M= 5.70, SD = .71) indicate a negative skew of Awareness scores. Coefficient alpha for the 28-item Knowledge/Skills subscale scores was .92 and for the 14-item Awareness subscale scores, .79. These coefficients are in the range reported in previous studies (see review in Ponterotto & Alexander, 1996).

EXPLORATORY PRINCIPAL COMPONENT ANALYSIS

A principal component analysis yielded nine eigenvalues greater than unity. A plot of the eigenvalues indicated a clear scree at three factors. Both orthogo- nal and oblique rotations were conducted. Although the two extractions were highly similar in terms of factor structure and loadings, the orthogonal (varimax) rotation provided the clearest and most interpretable extraction. The intercorrelations among the three factors were low and supported the orthogonal (rather than oblique) factor extraction method. Specifically, Factor 1 corre- lated .20 with Factor 2 and .28 with Factor 3. Factor 2 correlated -.01 with Factor 3. Using a more stringent (than Ponterotto et al., 1996) minimum factor loading cutoff of .40, 40 of the 45 items had unique high loadings on one of the three factors. Table 1 presents the results of the three-factor orthogonal extraction.

Twenty of the 22 general Knowledge/Skills items loaded on Factor 1 as did 1 Awareness item. Twelve of the 14 Awareness items loaded on Factor 2. The 6 specific Knowledge/Skills items focusing on familiarity with a particular leading scholar (Numbers 1, 4, 10, 16, 18, and 35) in the field all loaded clearly on Factor 3. Also, the one social desirability test item (Number 22) that asks about familiarity with a fictitious scholar also loaded on Factor 3. Two items (Numbers

JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 159

Page 8: A Revision of the Multicultural Counseling Awareness Scale

TABLE 1

Rotated Factor Loading Matrix, Communalities, Item-Total Correlations, and Item Means and Standard Deviations for the

Multicultural Counseling Awareness Scale

Factor Item-Total Item Number and Original Subscale 1 2 3 hZ Correlation M SD

1. Knowledge 2. (Awareness) 3. Knowledge 4. Knowledge 5. Knowledge 6. (Awareness) 7. Knowledge 8. Awareness 9. Knowledge

10. Knowledge 11. (Awareness) 12. Knowledge 13. Knowledge 14. (Awareness) 15. (Awareness) 16. Knowledge 17. Knowledge 18. Knowledge 19. Knowledge 20. Knowledge 21. Knowledge 22. Social Desirability 23. Knowledge 24. Knowledge 25. Knowledge 26. (Awareness) 27. Knowledge 28. (Awareness) 29. Knowledge 30. Social Desirability 31. Knowledge 32. Knowledge 33. Knowledge 34. (Awareness) 35. Knowledge 36. (Awareness) 37. Awareness 38. Social Desirability 39. Awareness 40. Knowledge 41. Knowledge 42. Awareness 43. (Awareness) 44. Knowledge 45. Knowledge

.I9

.I6

.47

.29

.51

.I8 5 2 .34 5 9 .19 .07 .55 .48 .oo .01 . I 3 .37 .20 .60 .69 5 8 .03 .53 5 8

- -

- -

- -

- - - - - - .73

-.07 .61

-.08 .71 5 7

.65 5 2

-.07 .14 .01 .31

-.09 .42 .63 S O .33 .I2 - .62 5 1

- - - .49 -

- -

-

.20 - 5 9 .03 .24 .23 - .48

.16

.15

.08 - .47 .oo .37 .61 .63

.24

.I3 -.03

.09

.24 -.20

.31 -.04 -.02 - .64

-.17 - 5 2 .oo

-.21 .03 .I3 .44 - .40 .08 - .44 .55

-.34 .27 -.05

.30 - .48

.04

.26

-.17

- -

-.06

-

- .53

.68

.08

.32

.69

.10

.07

.06

.03a

.I2

.71

.03

.06

.10

.16

.06

.76

.05"

.71

.I2

.21 -.03 50 .I0 .43b .08 .26 .29

.24

.31

.05

.13

.01b

.04

.64 -.07 -.lo

.13"

.01

.35

.06 -.09

.28

.07

.08

-.17

.53

.38

.33

.62

.32

.27

.31

.14

.39

.55

.22

.30

.38

.40

.40

.60

.20 5 6 .37 5 2 .40 .29 .39 .52 5 4 .48 .48 .31 5 6 .46 .24 .24 .47 .I7 .44 .20 .41 .I4 .24 .53 .35 .35 .37 .39 .33

.44

.40

.49 57 .51 .37 .33 .34 .57 .45 .26 .41 5 4 .32 .28 .34 .38 .45 .47 .63 5 2 .15 .55 5 7 .55 .32 50 .09 .63 .48 .39 .39 57 .I 2 .39 .17 .41

-.16 .43 5 7 5 2 .40 .39 .49 5 1

1.62 5.86 4.36 2.40 4.90 5.75 4.34 5.20 5.19 1.89 6.24 5.79 5.30 5.97 6.27 1.46 4.74 1.77 4.82 4.64 5.36 1.18 4.87 3.47 4.96 4.66 3.28 6.58 4.38 3.12 4.96 5.09 5.63 5.62 1.33 6.71 5.97 1.88 5.10 3.84 5.45 6.16 4.29 5.21 4.87

1.36 1.55 1.86 1.83 1.81 1.26 1.59 1.54 1.65 1.46 1.23 1.40 1.70 1.40 1.12 1.12 1.51 1.47 1.41 1.58 1.37 0.63 1.76 1.73 1.48 1.66 1.71 1 .oo 1.56 1.63 1.46 1.56 1.34 1.25 0.87 0.80 1.40 1.60 1.48 1.61 1.42 1.20 1.92 1.44 1.72

(table continued on next page)

160 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 9: A Revision of the Multicultural Counseling Awareness Scale

TABLE 1 (Continued)

Rotated Factor Loading Matrix, Communalities, Item-Total Correlations, and Item Means and Standard Deviations for the

Multicultural Counseling Awareness Scale

Item Number and Factor Item-Total Original Subscale 1 2 3 h2 Correlation M SO

Eigenvalue 10.4 4.0 2.8 % Common variance 23.2 8.9 6.3 % Cummulative

common variance 32.2 38.5

Note. h2 =community estimate. Factor 1 = Knowledge/Skills subscale; Factor 2 = Awareness subscale; Factor 3 = KnowledgelSkiIls name-specific subscale. Items in parentheses are connotatively inconsistent (negatively worded) and are reverse scored. Factor loadings that are underlined represent those selected for the revised two-factor Multicultural Counseling Knowledge and Awareness Scale.

8 and 17) had no loadings above the .40 criterion established. Finally, 2 items had multiple high loadings: Item 24 loaded on Factor 1 and Factor 3, and Item 33 loaded on Factor 1 and Factor 2.

The total common variance accounted for by the three-factor varimax ex- traction was 38.5%. Factor 1 (eigenvalue = 10.4) accounted for 23.2% of the common variance and focused on general multicultural counseling knowledge/ skills. Factor 2 (eigenvalue = 4.0) accounted for 8.9% and focused on multicultural awareness. Finally, Factor 3 (eigenvalue = 2.8) accounted for 6.3% of the com- mon variance and focused on knowledge of leading well-published scholars in the multicultural counseling field.

high loading. bMultiple high loading.

SCALE REVISIONS

As noted earlier, critics of the MCAS cited earlier expressed concern in a number of areas. First, the inclusion of six items that query familiarity with an individual scholar in the field has been questioned on both conceptual and practical grounds. Second, the use of and need for the three-item social desir- ability cluster have been questioned. Finally, some items of the MCAS appear overly long and include multiple clauses. Closer scrutiny of select items has been suggested (Kocarek et al., 2001; Pope-Davis & Dings, 1995). On the basis of these collective criticisms and in conjunction with the results of the explor- atory components analysis, the following revisions were made to the MCAS.

1. Factor 3 items, those inquiring of the respondent’s knowledge of a speci- fied scholar, were eliminated. Previous criticisms have been raised regarding these items. One concern deals with the items themselves; they include mul-

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 161

Page 10: A Revision of the Multicultural Counseling Awareness Scale

clauses and may be confusing to the respondent (see Pope-Davis & Dings, 1995). Another concern is the subjectivity of the six scholars chosen for the items. Although Ponterotto et al. (1991) used some objective criteria in select- ing the six scholars (e.g., productivity ratings and citation counts), one could easily argue with the selection. For example, would the six scholars identified as national leaders in 1991 be so identified in 2002? What about newer, emerging leaders in the field?

2. The three social desirability items (Items 22,30, and 38) were eliminated. Although Ponterotto et al. (1991) provided a recommendation for a minimum cutoff score for a valid inventory, the social desirability cluster never received adequate testing (see Pope-Davis & Dings, 1995).

3. Specific item analyses led to the following revisions. Five items either had no high (.40 or greater) loadings (Items 8, 17, and 38) or multiple high loadings (Items 24 and 33). One of these items (38) was also a social desirability cluster item. One item (39) did not load on its specified factor. In deciding whether to retain or discard these items, we examined the statistical properties of the items as they performed in the present factor analysis and the original structure first presented in Ponterotto et al. (1991) and published in Ponterotto et al. (1996). Table 2 presents the item number along with the factor loadings and commu- nality estimates across both studies. Given Item 38 was a social desirability control item, it was eliminated in Step 2 above and is therefore not included in Table 2. Item 8 and Item 39 had their highest loading on different factors across both studies. Furthermore, their communality estimates were relatively low across samples. These items are clearly problematic and were dropped from the MCAS.

Item 17 did not have a unique high loading in the present sample and barely reached a substantive loading in the 1991 study. A close scrutiny of this item reveals that it contains two separate clauses (“I am aware of my limitations in

TABLE 2

Factor Loadings and Communality Estimates for Multicultural Counseling Awareness Scale (MCAS) Item Numbers 8,17,24,33,

and 39 Across Two Studies

Present Study ( N = 525) Ponterotto et al., 1991 ( N = 126) MCAS Item Factor 1 Factor 2 Factor 3 ha Factor 1 Factor 2 ha

8 .34 .16 .03 .14 .31 .42 .22 17 .37 .24 .05 .20 .42 .25 .20 24 58 .04 .43 .52 .76 .01 .64 33 52 .44 .01 .47 .47 .32 .26 39 .42 .27 .01 .24 .33 .40 .21 ~~~ ~

Note. See Table 1 Note.

162 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 11: A Revision of the Multicultural Counseling Awareness Scale

cross-cultural counseling and could specify them readily.”) As noted by Pope- Davis and Dings (1995), double clauses were a concern for the Factor 3 items discussed in Step 1. Also, the communality estimates for Item 17 across studies were low. This item was eliminated. Item 33 had multiple high loadings across Factors 1 and 2 in the present study, although it loaded on Factor 1 in the 1991 study. The lack of a clear loading in the present sample is problematic, and the item was dropped.

Finally, Item 24 loaded highly on Factor 1 and Factor 3, loading .15 higher on Factor 1. This was not one of the named items that was eliminated in Step 1. The item reads “I am knowledgeable of acculturation models for various ethnic minority groups.” In the 1991 study, this item was a clear and strong Factor 1 loading. In the present study, the item loaded on Factor 3 as well as Factor 1, its intended factor, probably because it queries rather specific knowl- edge. The communality estimates across studies are high, and we are confi- dent that with Factor 3 (the six named items) eliminated, this item would clearly load on Factor 1 in a two-factor extraction. Therefore, this item was retained as a Factor 1 (Knowledge) item.

4. Consistent with the recommendation of Pope-Davis and Dings (1995), we recommend changing the name of Factor 1, Knowledge/Skills, to Knowledge to more accurately reflect the content of the items. Finally, we recommend that the inventory name be changed from the Multicultural Counseling Aware- ness Scale (MCAS) to the Multicultural Counseling Knowledge and Awareness Scale (MCKAS).

The revised MCKAS consists of 20 Knowledge items and 12 Awareness items. All 20 Knowledge items are positively scored (high scores indicate high per- ceived knowledge). Ten of the 12 Awareness items are negatively worded and therefore are reverse scored so that high scores indicate high awareness. The nature of the subtle biases and attitudes reflected in the Awareness subscale support this reverse-wording format. The revised Knowledge and Awareness subscale items selected for the MCKAS are indicated by underlined factor loadings in Table 1. The content of the items can be viewed in the Appendix.

The goal of Study 2 was to test the goodness of fit of the two-factor (Knowl- edge and Awareness) MCKAS model on a new sample. Furthermore, initial tests of convergent, criterion-related, and discriminant validity were performed along with a test of internal consistency reliability. We hypothesized that a two-factor orthogonal model would be the best fit factor structure and that subscales of the MCKAS would correlate positively with convergent measures of self-perceived multicultural competence and ethnic identity.

JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT. July 2002 Vol. 30 163

Page 12: A Revision of the Multicultural Counseling Awareness Scale

SAMPLE

Participants included 199 counselors-in-training recruited from five universities in the Northeast. The mean age for the sample was 30 years (SD= 8.9), with the median age falling at 26 years. Seventy-five percent (n = 149) of the sample participants were female, and the racial/ethnic breakdown was as follows: 45% (n = 90) White/Not Hispanic, 18% (n = 36) African American, 16% (n = 32) Hispanic American, 2% (n = 4) Asian AmericadPacific Islander, and 1% (n = 2) Native American (the remainder was listed as either other or unspecified).

Seventy-five percent (n =149) of the participants were enrolled in master’s programs in counseling; 4% (n = 8) in post-master’s, nondoctoral programs; 8% (n = 16) in doctoral programs; and remainder unspecified. Twenty percent (n = 40) of the sample had never completed a multicultural counseling course, 39% (n = 78) had not completed a course but reported covering multicultural issues in other classes, 23% (n = 46) had completed one course, and 6% (n= 12) had com- pleted two or more multicultural-specific counseling courses. Roughly 60% (n= 119) of the sample indicated no supervised experience with culturally diverse clients, whereas 25% (n = 50) did receive such supervision (remainder unspecified).

INSTRUMENTS

In addition to the 32-item MCKAS described in the previous study, partici- pants also completed the MCI (Sodowsky et al., 1994), the Multigroup Ethnic Identity Measure (MEIM; Phinney, 1992), and the SDS (Crowne & Marlowe, 1964). The MCI is considered one of the stronger competency instruments from a psychometric perspective (Ponterotto & Alexander, 1996; Pope-Davis & Dings, 1995), and this instrument was chosen as a convergent validity test for the revised MCKAS. Positive and significant correlations were expected between the MCI Knowledge and Skills subscales and the MCKAS Knowl- edge subscale. Because previous correlational research has found that the MCI and MCAS Awareness subscales seem to measure different components of multicultural awareness, this correlation was expected to be low (see Kocarek et al., 2001; Ponterotto et al., 2000; Pope-Davis & Dings, 1994).

As a measure of ethnic identity commitment and attitudes toward other groups, the MEIM was hypothesized to relate conceptually to multicultural knowl- edge and awareness in counselors. Therefore, positive and significant relation- ships were expected between MEIM subscales and MCKAS subscales. Fi- nally, the SDS was incorporated to test for discriminant validity of the MCKAS.

MCI. The MCI (Sodowsky, 1996; Sodowsky et al., 1994) is one of the more widely used paper-and-pencil self-report measures of perceived multicultural counseling competence (see Pope-Davis & Nielson, 1996). The instrument has been used in a number of published studies and multiple doctoral disserta- tions, and a goodly amount of empirical data is available on the measure. The

164 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 13: A Revision of the Multicultural Counseling Awareness Scale

40-item MCI consists of four, factor analytically derived subscales, all arranged on a 4-point Likert-type scale: Skill (11 items), Awareness (10 items), Counseling Relationship (8 items), and Knowledge (1 1 items). In a review of MCI scores with seven samples, coefficient alphas ranged from .77 to .83 (mean alpha = .80) for Skill scores, .74 to 3 0 (mean alpha = .78) for Knowledge scores, .76 to 3 3 (mean alpha = .78) for Awareness scores, and .65 to .78 (mean alpha = .68) for Counseling Relationship scores (Pope-Davis & Nielson, 1996). The MCI and its separate subscales have evidenced moderate levels of construct validity using exploratory and confirmatory factor analyses and moderate levels of convergent and criterion-related validity through expected correlations with multicultural training and experience levels (for author review see Sodowsky, 1996; for independent reviews see Constantine & Ladany, 2001; Ponterotto et al., 2000; Ponterotto et al., 1994; Pope-Davis & Nielson, 1996; Worthington et al., 2000). MEIM. The MEIM (Phinney, 1992) consists of an Ethnic Identity subscale (14

items) that assesses ethnic identity achievement, a sense of ethnic affirmation and belonging, and adherence to culturally traditional practices, and an Other Group Orientation subscale (6 items) that assesses attitudes toward interactions with in- dividuals representing other (than one’s own) racidethnic groups. MEIM items are scored on a 4-point Likert-type scale. Phinney’s factor analyses on both high school and college student samples supported this two-factor conceptualization. Recently, Ponterotto, Gretchen, Utsey, Stracuzzi, and Saya (in press) reviewed 12 published studies using the MEIM and reported a mean coefficient alpha of .86 (median = .85, range = 3 1 to .92) for the Ethnic Identity subscale scores and a mean alpha of .69 (median = .75, range = .35 to .82) for the Other Group Orien- tation subscale scores. In a confirmatory factor analysis of MEIM scores with late adolescents, Ponterotto et al. (in press) found the two-factor model to be the best fit model, reaching a goodness-of-fit index of .92 and a relative noncentrality in- dex of -93. In addition to the construct validity indicated by the confirmatory factor analysis, the Ponterotto et al. (in press) review indicated moderate levels of both criterion-related and convergent validity. SDS. The SDS (Crowne & Marlowe, 1960, 1964, 1973) includes 33 true-false

items that tap one’s need to respond in a socially and culturally appropriate manner. Internal consistency of the SDS was reported at .88 with a l-month stability coefficient of .89 (Crowne & Marlowe, 1960). Paulhus (1991) reported internal consistency coefficients ranging from .73 to .88. In more recent multicultural counseling competency-focused studies, SDS scores had coefficient alphas of .86 (Constantine, Juby, & Liang, 2001; Constantine & Ladany, 2000).

PROCEDURE

Procedures for data collection were similar to those in Study 1. Contact was made with a professor in each of the five universities located in the states of New York and New Jersey. The professors were asked to distribute the surveys

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 165

Page 14: A Revision of the Multicultural Counseling Awareness Scale

in courses during class time and collect the surveys as a group. Survey packets included a demographic form, the MCKAS, and, in some cases, one of the companion instruments described earlier. All surveys distributed to students were returned completed. Professors used the instruments to stimulate class discussion on multicultural counseling, although the courses were not neces- sarily multicultural in title.

qtudv 3, results SCORE DISPERSION AND RELIABILITY

Table 3 presents the means, standard deviations, cell sizes, coefficient alphas, and relevant subscale intercorrelations for Study 2. The means for the MCKAS Knowledge and Awareness subscales were 4.96 and 5.06, respectively, indi- cating a slight negative skew on the 1-7-point Likert-type scale. Averaging the items, respondents tended to indicate that the competencies listed were some- what true as applied to them. The MCI means for the Skills, Awareness, Coun- seling Relationship, and Knowledge subscales were 3.31,3.15, 2.19, and 3.16, respectively, on the 1 to 4 scaling, indicating a negative skew and thus positive self-perceptions on the Skills, Awareness, and Knowledge subscales. Finally,

TABLE 3

Means, Standard Deviations, Coefficient Alpha, and Intercorrelations Between Multicultural Counseling Knowledge and Awareness Scale

(MCKAS) and Comparison Instruments

Correlations Scale/ Coefficient Su bscale n M SO a MCKAS-K MCKAS-A

MCKAS-K 196 4.96 0.80 .85 1 .o 0.04 MCKAS-A 196 5.06 1.14 .85 0.04 1 .oo MCI-S 42 3.31 0.36 .73 0.43" -0.10 MCI-A 42 3.15 0.44 .70 0.44" -0.06 MCI-R 42 2.19 0.70 .78 -0.1 2 0.74" MCI-K 42 3.16 0.36 .66 0.49'' 0.10 ME I M-El 42 3.12 0.42 .75 0.31' 0.03 MEIM-OGO 42 2.91 0.51 .42 0.12 0.20 SDS 27 1.53 0.20 .81 -0.39' 0.07

Note. MCKAS-K = MCKAS Knowledge subscale; MCKAS-A = MCKAS Awareness subscale; MCI-S = Multicultural Counseling Inventory Skills subscale; MCI-A = Multicultural Counseling Inventory Awareness subscale; MCI-R = Multicultural Counseling Inventory Relationship subscale; MCI-K = Multicultural Counseling Inventory Knowledge subscale; MEIM-El = Multigroup Ethnic Identity Measure Ethnic Identity subscale; MEIM-OGO = Multigroup Ethnic Identity Measure Other Group Orientation subscale; SDS = Social Desirability Scale. ' p < .05. ** p < .01.

166 JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 15: A Revision of the Multicultural Counseling Awareness Scale

the MEIM Ethnic Identity and Other Group Orientation subscales received means of 3.12 and 2.91, respectively, on the l-Crange, again indicating anegative skew (see Table 3).

After a brief review of recommended minimally acceptable internal consis- tency levels, Ponterotto (1996) established .70 as a satisfactory coefficient al- pha for scores on paper-and-pencil, group-administered research measures in psychology. A review of Column 4 in Table 3 shows that the majority of subscale scores across instruments reached this floor level. Alphas for the MCKAS Knowledge and Awareness subscale scores were .85 and .85, respectively. With the present sample, the MCI subscale scores all reached the .70 floor with the exception of the Knowledge subscale scores, which had an alpha of .66. Fi- nally, whereas the MEIM Ethnic Identity subscale scores and the Social De- sirability scale scores had acceptable alphas, the alpha for the MEIM Other Group Orientation subscale scores at .42 was unacceptable.

CONSTRUCT VALIDITY

A goal of Study 2 was to use confirmatory factor analysis (CFA) to test the goodness-of-fit index of the two-factor MCKAS. Hoyle (2000) highlighted the usefulness of CFA to evaluate the construct validity of measures. Our sample size of N = 199 meets requirements for minimal sample of 5-10 participants per scale item recommended by Bentler and Chou (1987). Hoyle and Quintana and Maxwell (1999) recommended a minimum of 200 participants, and our sample size was just under that total at N = 199.

CONFIRMATORY METHODS AND CRITERIA

Confirmatory maximum likelihood factor analysis was used to test the under- lying factor structure of the MCKAS (see Hoyle, 2000, for a recent CFA re- view). Three theoretical models were chosen for comparison with each other and with a baseline null model: (a) one global factor, (b) two independent (orthogonal) factors, and (c) two correlated (oblique) factors. The null model represents a hypothetical worst case in which no underlying relationships between items are stipulated. The purpose of the null model is to establish a baseline against which competing theoretical models can be compared for improvement in fit.

Theoretical justification for a unidimensional factor structure (one global fac- tor) rests on the hypothesis that knowledge, skills, and awareness are indistinct aspects of multicultural counseling competence. Research into the factor structure of the CCCI-R (LaFromboise et al., 1991) has provided some support for a unidimensional construct underlying multicultural counseling competence. The proposed two-factor models stem from the results of Study 1 and the results of prior principal component analyses of the MCAS (Ponterotto et al., 1996). In

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 167

Page 16: A Revision of the Multicultural Counseling Awareness Scale

the orthogonal model, the Knowledge/Skills and Awareness factors are stipu- lated to be independent of each other, whereas in the oblique model, they are assumed to be correlated.

LISREL 7 (Joreskog & Sorbom, 1989) was used for all analyses, and raw data were converted into a covariance matrix using the PRELIS program (Joreskog & Sorbom, 1988). In the null model, each of the 32 items was constrained to load on one of 32 factors, and factor intercorrelations were fvred at zero. Similarly, in the three theoretical models, items were free to load on only one factor while loadings on other factors (when necessary) were fured at zero. For each factor, one loading was always constrained to equal 1 to establish a metric. For the two-factor oblique model, the correlation between factors was free to be estimated.

Following established procedures for structural equation modeling (Bollen, 1989; Hu & Bentler, 1995) the different models were compared using several indices of fit. To begin with, chi-square for each model was computed and tested for signifi- cance. The chi-square statistic is used to determine whether sample parameters differ significantly from values specified in the theoretical model, with the null hypothesis being that all the population covariance has been extracted by the hypothesized number of factors (see Hoyle, 2000). Also examined were the good- ness-of-fit index (GFI) and the adjusted GFI (AGFI). The GFI varies from 0 to 1 and is an estimate of the amount of variance accounted for by the model, similar to R2 in multiple regression. Higher (i.e., .go+) GFI values indicate better fit (see Hu & Bentler, 1999). The AGFI incorporates a penalty for the addition of more parameters because this naturally tends to inflate the GFI.

Another recommended method of examining fit is to measure the relative improvement offered by a proposed model in comparison with a logical worst case or null model in which there are no underlying relationships between observed values (Bollen, 1989; Hu & Bentler, 1995). Hu and Bentler (1995) recommended using two indices of incremental fit: the Tucker-Lewis index (TLI; Tucker & Lewis, 1973) and the relative noncentrality index (RNI; McDonald & Marsh, 1990). Bentler and Bonett (1980) suggested that incre- mental index values of .90 or greater indicate adequate fit. Finally, we also examined the root mean square residual (RMSR) as recommended by Quintana and Maxwell (1999). According to these authors, the RMSR is less affected by sample size and therefore has more descriptive value across various sample sizes. Although somewhat arbitrary, Quintana and Maxwell reviewed research noting that a RMSR of less than .05 is indicative of a good fit, .05 to .08 a fair fit, .08 to .10 a mediocre fit, and greater than .1 a poor fit.

CON F I RM AT0 RY RESULTS

The various goodness-of-fit indicators are presented in Table 4. The chi-square for all models was significant, indicating a significant difference between ob- served and specified models. However, this is not unexpected given the large

168 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 17: A Revision of the Multicultural Counseling Awareness Scale

TABLE 4

Goodness-of-Fit Indicators for the Multicultural Counseling Knowledge and Awareness Scale Null Model and Competing

Theoretical Models Using Unaggregated and Aggregated Items for Study 2 (N = 199)

Model x2 df GFI AGFl TLI RNI RMSR

Unaggregated (20 Knowledge items; 12 Awareness items) .22 Null model 2529.41* 496 .40 .36

1 Global factor 1681.44' 464 .50 .43 .36 .40 .I7 - -

2 Orthogonal factors 968.81' 464 .74 .70 .74 .75 .09 2 Oblique factors 968.81. 463 .74 .70 .73 .75 .09

Aggregated (7 Knowledge items; 4 Awareness items)

.34 Null model 1007.33' 55 .43 .31 1 Global factor 496.59' 44 .66 .48 .41 .52 .20 2 Orthogonal factors 114.08' 44 .90 .85 .91 .93 .07 2 Oblique factors 114.08' 43 .go .85 .91 .93 .07

- -

Note. GFI = goodness-of-fit index; AGFl = adjusted goodness-of-fit index; TLI =Tucker Lewis index; RNI = relative noncentrality index; RMSR = root mean square residual. ' p c .001.

sample size and associated power of the chi-square test and does not necessar- ily indicate a poor fit in practical terms (Bollen, 1989; Fassinger, 1987). Re- cently, Quintana and Maxwell (1999), in their integrative review of structural equation modeling procedures, highlighted that chi-square tests for an exact fit between observed and specified models rather than a close or satisfactory fit, and therefore this statistic is overly restrictive and limited.

Table 4 demonstrates that the GFI, AGFI, TLI, and RNI are nearly equiva- lent for the two-factor orthogonal and two-factor oblique models and that they hover in the low .70s, indicating an unsatisfactory fit to the sample data. This unsatisfactory fit is also supported by the RMSR, falling at .09, indicating a mediocre fit (Quintana & Maxwell, 1999).

These results suggest that the proposed one- and two-factor structures of the MCKAS are unsatisfactory. However, some recent confirmatory analyses have shown that an unsatisfactory fit is not unexpected in models with more than four or five items per factor and a large sample size (Bagozzi & Heatherton, 1994; Pope-Davis, Vandiver, & Stone, 1999; Rahim & Magner, 1995; Utsey & Ponterotto, 1996). This is due to the high levels of random error associated with typical scale items and to the many parameters that must be estimated (Bagozzi & Heatherton, 1994). To compensate for these problems, Bagozzi and Heatherton recommended creating aggregate variables (also known as itemparcels) within each factor. In scales in which there are nine or more items per factor, three or more aggregate variables are appropriate. The first aggre-

JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 169

Page 18: A Revision of the Multicultural Counseling Awareness Scale

gate item for each factor is created from the items with the highest and lowest factor loadings, whereas the remaining items are randomly allocated. The smaller set of aggregate variables is then submitted to a new CFA. There is now ample support for aggregate item procedures because this method reduces the number of variables used and consequently improves the sample size-to-variable ratio. Furthermore, aggregate procedures improve the reliability of measures and mini- mize violations of assumptions ofmultivariate normality (see Kishton & Widaman, 1994; Marsh, Antill, & Cunningham, 1989; Pope-Davis et al., 1999; Russell, Kahn, Spoth, & Altmaier, 1998; Utsey & Ponterotto, 1996).

Following the procedures outlined by Bagozzi and Heatherton (1994) and mod- eled in Utsey and Ponterotto (1996), new aggregate items were created for the MCKAS subscales. Seven aggregate Knowledge items were created: K1 Items 20, 29,31; K2 Items 7, 12,40; K3 Items 3,9,25; K4 Items 13, 19,27; K5 Items 523, 41; K6 Items 21, 24, 44; and K7 Items 32, 45. Four aggregate Awareness items were created: A w l Items 14,26,36; AW2 Items 2, 15,37; AW3 Items 11,28,42; and AW4 Items 6,34, 43 (refer to Table 1). Coefficient alphas for the two aggre- gate factors were .91 and .80, respectively.

Table 4 presents a summary of the aggregate fit indexes. As expected, the aggre- gate fit improved with the GFI, TLI, and RNI at or above .90 and with the RMSR falling at .07. This pattern of fit indexes indicates an overall satisfactory fit for the two-factor models. It is interesting that the two-factor oblique and orthogonal fits were virtually identical; this finding is supported by the negligible correlation (r= .04) between the Knowledge and Awareness subscales (refer to Table 3).

CO NVE RG E NT, C R ITE R I0 N , AN D D I SC R I M I N ANT VAL I D ITY

The convergent, criterion-related, and discriminant validity of the MCKAS was examined through a pattern analysis of the correlations listed in Table 3. Effect sizes were exam’ined for all correlations using standards established by Cohen (1988) in which r< .3 is a small effect size, r= .3 to .49 is a medium effect size, and r2.5 is a large effect size. Convergent validity for MCKAS scores was examined through correlations with the MCI subscale scores. The MCKAS Knowledge subscale evidenced significant correlations and medium effect sizes with the MCI Knowl- edge ( r = .49), Skill ( r = .43), and Awareness (.44) subscales. The significant correlation between MCKAS Knowledge scores and MCI Knowledge subscale scores and Skills subscale scores was expected.

The MCKAS Awareness subscale correlated highly and significantly with the MCI Counseling Relationship subscale (r= .74; large effect size) because both subscales focus to some degree on the counselor’s perceived comfort working with culturally diverse clients. The Awareness subscales of the two instruments were not correlated ( r = -.06; small effect size) because the items in the MCKAS Awareness subscale focus on subtle Eurocentric bias, whereas the MCI Awareness items focus on the counselor’s understandinghowledge of issues outside the counseling relationship.

170 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 19: A Revision of the Multicultural Counseling Awareness Scale

The model of multicultural counseling competencies (Sue et al., 1998) posits that counselors with high levels of competency are likely to have a strong internal sense, knowledge, and appreciation of their own ethnic heritage (identity achievement) as well as positive and embracing attitudes toward interethnic interaction in both their personal and professional lives (Ponterotto et al., 2000). Therefore, MCKAS subscale correlations with the MEIM were used as a cri- terion validity check. MCKAS Knowledge scores were moderately ( r = .31; medium effect size) and significantly correlated with MEIM Ethnic Identity scores, as expected. It was also expected that the MCKAS Awareness scores would correlate with the MEIM Other Group Orientation subscale; however, this correlation ( r = .20; small effect size) did not reach significance. Finally, MCKAS Knowledge subscale scores did correlate significantly with the SDS scores ( r= -.39; medium effect size).

n The MCAS is one of the more frequently used multicultural competency self- report measures (Pope-Davis & Dings, 1995). Although exhibiting some strong initial indices of reliability and validity (reviewed in Ponterotto & Alexander, 1996), due caution has been expressed regarding the limitations of the MCAS. An important research need for the MCAS was to conduct an exploratory factor analysis using a large national sample. Accordingly, Study 1, sampling 525 participants, found that a two-factor extraction identified in early work on the MCAS (Ponterotto et al., 1996; Ponterotto et al., 1991) was the best fit model underlying MCAS items.

The results of Study 1, coupled with MCAS revision suggestions made by in- dependent researchers (e.g., Kocarek et al., 2001; Pope-Davis & Dings, 1995), led to various item eliminations and a more concise MCKAS. Study 2, using a new sample of 199 counselors-in-training, incorporated aggregate confirmatory procedures (i.e., Bagozzi & Heatherton, 1994) to test three competing factor structure models. This study found both the two-factor oblique and orthogonal models reaching multiple satisfactory goodness-of-fit indexes. Previous studies of the MCAS found scores on the Knowledge/Skills and Awareness subscales to be moderately correlated (averaging .37), and thus fostering an oblique solution, but the present study found that this correlation was negligible (at .04; refer to Table 3), thus supporting an orthogonal relationship among subscales.

Study 2 also provided moderate indices of convergent validity for the MCKAS Knowledge subscale in its anticipated significant correlation with the Knowledge and Skills subscales of the MCI. Furthermore, the MCKAS Awareness subscale was significantly correlated with the MCI Counseling Relationship subscale, both of which assess to some degree the counselor’s sensitivity and comfort with cross- cultural value variations. That the MCKAS and MCI Awareness subscales were unrelated is not surprising because this finding replicates previous research (e.g.,

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 171

Page 20: A Revision of the Multicultural Counseling Awareness Scale

Pope-Davis & Dings, 1994), and item content analyses revealed that that the awareness construct operationalizations varied across instruments (Ponterotto et al., 2000; Pope-Davis & Dings, 1995). Specifically, the awareness construct of the MCI seems to parallel the Knowledge items on the MCKAS. For example, MCI Item 25 queries respondents’ “understanding of specific racial and ethnic minority groups,” whereas Item 27 taps one’s “understanding of the importance of the legalities of immigration” (Sodowsky, 1996, pp. 298-299). These types of items clustered in the MCKAS Knowledge subscale (see Ponterotto et al., 1996) because they query knowledge of specific multicultural topics. Clearly, more definitional clarity and construct validity research are needed on the awareness construct across measures.

A degree of criterion-related validity was established for the MCKAS Knowl- edge subscale in its expected significant correlation with the MEIM’s Ethnic Identity subscale. We expected a significant correlation between the MCKAS Awareness subscale and the MEIM Other Group Orientation subscale, although this expectation did not materialize. The low coefficient alpha of the MEIM subscale (a = .42), along with the small subsample (n = 42) that completed both instruments, likely attenuated the potential criterion relationship. Clearly more research is needed to test the criterion-related validity of both MCKAS subscales.

Study 2 also found discriminant validity for the Awareness subscale of the MCKAS, but the Knowledge subscale was found to correlate significantly with the SDS. This finding is in contrast to two recent studies using the new MCKAS (Constantine et al., 2001; Constantine & Ladany, 2000) that found MCKAS Awareness subscale scores to be significantly negatively correlated with social desirability scores. One hypothesis for this significant correlation posited by Constantine and colleagues (Constantine et al., 2001; Constantine & Ladany, 2000) was that respondents who were high in multicultural Awareness had a particularly low need to appear socially desirable and, thus, the significant negative correlation. Similarly, in the present sample it could be that respon- dents who were high in multicultural Knowledge were internally secure and had a low need to appear socially desirable.

However, although it is true that anonymous self-report measures yield lower levels of respondent acquiescence or socially desirable responding than do questions asked in face-to-face interviews (see review in Krosnick, 1999), so- cially desirable responding has been cited as a concern for all measures of self- report multicultural competencies and needs to be addressed in further re- search (Constantine & Ladany, 2001; Pope-Davis & Dings, 1994; Sodowsky, Kuo-Jackson, Richardson, & Corey, 1998; Worthington et al., 2000).

Limitations of the present studies are numerous and can serve as guides for future validation research on the MCKAS and other self-report measures of multicultural counseling competencies. First, sampling procedures were not random, and Study 2 participants were limited to the northeastern region of the United States. More systematic random sampling across all training and

1 72 JOURNAL OFMULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 21: A Revision of the Multicultural Counseling Awareness Scale

professional levels will help researchers more closely scrutinize MCKAS strengths and limitations. Furthermore, given the MCKAS no longer includes items that assess familiarity with the work of specific counseling psychology researchers, it would be appropriate to test the instrument with social workers and clinical and school psychologists, because the Sue et al. (1992, 1998) model purports to transcend mental health domains.

A major limitation of the MCKAS and other self-report measures is that criterion-related validity linking instrument scores with actual success in multicultural counseling practice is limited. Specifically, Constantine and Ladany (2000) and Ladany, Inman, Constantine, and Hofheinz (1997) found scores on various multicultural competency instruments to be unrelated to written case conceptualization skills. Worthington et al. (2000) used a creative analog design with video vignettes to document a limited relationship between self-reported multicultural competencies (using the MCI) and observer-report competency evaluations (using the CCCI-R). There is a strong need for field-based, criterion- related validity research on the MCKAS using both concurrent and predic- tive procedures.

Regarding construct validity, despite the satisfactory goodness-of-fit indexes found with aggregate (Bagozzi & Heatherton, 1994) CFA procedures, the per- centage of common variance accounted for by the two-factor MCKAS model was only 32% (refer to Table 1). More research is needed on the construct validity of the MCKAS. For example, how does perceived multicultural coun- seling competency relate to perceived general counseling competency? What degree of overlap exists between these two constructs, and, collectively, how can they predict counseling process and outcome variables? In relation to nu- merous potential predictors of successful outcome, what part does multicultural competence play relative to all racial/ethnic groups (including Anglo Saxon and White Ethnic groups) in a heterogeneous society? Accordingly, we sug- gest the use of additional structural equation modeling, particularly path models, to further test the role and salience of the multicultural competence construct and its related measuring devices (see Quintana &Maxwell, 1999; Quintana, Troyano, & Taylor, 2001).

The present study, as well as others (e.g., Constantine, Gloria, & Ladany, in press), has called into question the tripartite (awareness, knowledge, and skills) model of Sue et al. (1998). It might be that this model needs conceptual revi- sion given the results of factor analytic studies, and perhaps new models should be tested (see new models in Constantine & Ladany, 2001, and Fuertes & Gretchen, 2001). It may also be that traditional psychometric research may not be able to capture the interacting complexity of the tripartite model and that more varied research approaches are in order.

Recently, qualitative research methods have been recommended for coun- seling research (Ponterotto, 2002), and we urge researchers to incorporate varied qualitative approaches to the study of multicultural counseling competence.

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 173

Page 22: A Revision of the Multicultural Counseling Awareness Scale

Recent examples of such designs are beginning to appear in the literature with greater frequency (e.g., Ancis & Szymanski, 2001; Fuertes, Mueller, Chauhan, Walker, & Ladany, in press; Ponterotto, Jackson, & Nutini, 2001; Pope-Davis et al., 2002). Morrow, Rakhsha, and Castaneda (2001) provided a thorough review of the place of qualitative methods for multicultural counseling research.

The present series of studies represented only an initial effort to test and revise an increasingly popular measure of multicultural counseling compe- tence. As such, the results are preliminary, and the MCKAS should not be used for any individual evaluative purposes. However, preliminary evidence is emerging that suggests the new MCKAS is a potentially useful instrument. For example, Constantine and colleagues (Constantine, 2000; Constantine et al., in press; Constantine et al., 2001) reported coefficient alpha reliability es- timates for the MCKAS Full scale scores, Knowledge subscale scores, and Awareness subscale scores to be in the .75 to .91 range.

Regarding construct validity, Constantine (2000) found affective and cognitive empathy together (and affective empathy alone) contributed significant positive variance to both MCKAS Knowledge and Awareness scores. Furthermore, Constantine et al. (2001) found that after controlling for social desirability and previous multicultural training, self-reported racism and White racial identity attitudes together accounted for a significant amount of the variance in per- ceived multicultural knowledge and awareness among marital and family thera- pists. Specifically, higher pseudoindependence attitudes were related to higher self-reported multicultural knowledge, and higher reintegration attitudes and racism scores were related to lower levels of multicultural awareness. In another recent study supporting the construct validity of the MCKAS, Bidell and Casas (2001) found subscale scores to be significantly correlated in expected directions with the Attitudes Toward Lesbians and Gay Men Scale (Herek, 1998) scores and with subscale scores of the new Sexual Orientation Counselor Competence Scale (Bidell & Casas, 2001).

As multicultural emphases in psychology continue to gamer professional and political weight (Heppner, Casas, Carter, & Stone, 2000; Neimeyer & Diamond, 200l), the use of easily administered self-report instruments will be increasingly called upon by the profession and state/federal practice monitors (see also Pope- Davis & Dings, 1995). In the profession’s efforts to become more culturally ef- fective and pluralistic in practice, it is essential that adequate empirical attention be given to both the competency model itself and the instrumentation designed to operationalize the model (cf. Ponterotto et al., 2000). It is hoped that this article will stimulate further quantitative and qualitative research in this regard.

references Ancis, J. R., & Szymanski, D. M. (2001). Awareness of White privilege among White counselor

trainees. TEe Counseling Psychologist, 29,548-569.

174 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 23: A Revision of the Multicultural Counseling Awareness Scale

Arredondo, P., Toporek, R., Brown, S., Jones, J., Locke, D. C., Sanchez, J., et al. (1996). Operationalization of the multicultural counseling competencies. Journal ofMulticultural Coun- seling and Development, 24, 42-78.

Bagozzi, R. P., & Heatherton, T. F. (1994). A general approach to representing multifaceted personality constructs: Application to state self-esteem. Structural Equation Modeling, 7,35-67.

Bentler, P. M., & Bonett, D. G. (1980). Significance of tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.

Bentler, P. M., & Chou, C. P. (1987). Practical issues in structural equation modeling. Sociological Methods and Research, 16, 78-1 17.

Bidell, M. P., & Casas, J. M. (2001, August). Measuring counselor competence with lesbian/gay/bi- sexual clients: Implicationsfor multiculhual training. Paper presented at the annual meeting of the American Psychological Association, San Francisco.

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley. Cohen, J. (1988). Statisticalpower analysisfor the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Constantine, M. G. (2000). Social desirability attitudes, sex, and affective and cognitive empa-

thy as predictors of self-reported multicultural counseling competence. f i e Counseling Psy- chologist, 28, 857-872.

Constantine, M. G., Gloria, A. M., & Ladany, N. (in press). The factor structure underlying three self-report multicultural counseling competence scales. Cultural Diversity andEthnicMinority Psycho logy.

Constantine, M. G., Juby, H. L., & Liang, J. J. C. (2001). Examining multicultural counseling competence and race-related attitudes among White marital and family therapists. Journal of Marital and Family fierapy, 27, 353-362.

Constantine, M. G., & Ladany, N. (2000). Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization ability. Journal of Counseling Psychology, 47, 155-164.

Constantine, M. G., & Ladany, N. (2001). New visions for defining and assessing multicultural counseling competence. In J. G. Ponterotto, J. M. Casas, L. A. Suzuki, & C. M. Alexander (Eds.), Handbook of multicultural counseling (2nd ed., pp. 482-498). Thousand Oaks, CA: Sage.

Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psycho- pathology. Journal of Consulting Psychology, 24, 349-354.

Crowne, D. P., & Marlowe, D. (1964). T7ze approval motive. New York: Wiley. Crowne, D. P., & Marlowe, D. (1973). Social Desirability Scale. InJ. P. Robinson & P. R. Shaver

(Eds.), Measures of socialpsychological attitudes (pp. 727-732). Ann Arbor, MI: Survey Research Center, Institute for Social Research.

D’Andrea, M., Daniels, J., & Heck, R. (1991). Evaluating the impact of multicultural counseling training. Journal of Counseling d Development, 70, 143-150.

Dawis, R. V. (1987). Scale construction. Journal of Counseling Psychology, 34,481-489. Dawis, R. V. (2000). Scale construction and psychometric considerations. In H. E. A. Tinsley &

S. D. Brown (Eds.), Handbook ofapplied multivariate statistics and mathematical modeling(pp. 65- 94). San Diego, CA: Academic Press.

Dinsmore, J. A., & England, J. T. (1996). A study of multicultural counseling training at CACREP- accredited counselor education programs. Counselor Education and Supervision, 36,58-76.

Fassinger, R. E. (1987). Use of structural equation modeling in counseling psychology research. Journal of Counseling Psychology, 34, 425-436.

Fuertes, J. N., Bartolomeo, M., & Nichols, C. M. (2001). Future research directions in the study of counselor multicultural competencies. Journal of Multicultural Counseling and Development,

Fuertes, J. N., & Gretchen, D. (2001). Emerging theories of multicultural counseling. In J. G. Ponterotto, J. M. Casas, L. A. Suzuki, & C. M. Alexander (Eds.), Handbook of multicultural counseling (2nd ed., pp. 509-541). Thousand Oaks, CA: Sage.

Fuertes, J. N., Mueller, L. N., Chauhan, R. V., Walker, J. A,, & Ladany, N. (in press). An inves- tigation of European-American therapists’ approach to counseling African-American clients. f i e Counseling Psychologist.

29, 3-12.

JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 175

Page 24: A Revision of the Multicultural Counseling Awareness Scale

Heppner, P. P., Casas, J. M., Carter, J., & Stone, G. L. (2000). The maturation of counseling psychology: Multifaceted perspectives, 1978-1998. In S. D. Brown & R. W. Lent (Eds.), Handbook ofcounselingpsychology (3rd ed., pp. 3-49). New York: Wiley.

Herek, G. M. (1998). Z?ie Attitudes Toward Lesbians and Gay Men (AZZG) Scale. Thousand Oaks, C A Sage. Hernandez, A. G., & LaFromboise, T. D. (1985, August). Z?E deuelopmt ofthe Cross -Cullural Counseling

Inventmy Paper presented at the annual meeting ofthe American Psychological Association, Los Angeles. Hoyle, R H. (2000). C o h a t o r y factor analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of

applied multivariak slatistia and p he ma tical modeling (pp. 465-497). San Diego, C A Academic Press. Hu, L., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation mod-

eling: ConcepLs, issues, and applications (pp. 76-99). Thousand Oaks, CA: Sage. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Con-

ventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. Jacobson, C. K. (1985). Resistance to affiiative action: Self-interest or racism. Journal of Conflict

Resolution, W, 306-329. Joreskog, K. G., &Sorbom, D. (1988). PRELISApreprocRcForjrLISREL [Computer program manual].

Chicago: Scientific Software. Joreskog, K. G., & Sorbom, D. (1989). LISREL 7: A guide to theprogram and applications [Computer

program manual]. Chicago: Scientific Software. Kishton, J. M., & Widaman, K. F. (1994). Unidimensional versus domain representative parceling of

questionnaire items: An empirical example. Educational and Psychological Meaturemenl, 54,757-765. Kocarek, C. E., Talbot, D. M., Batka, J. C., & Anderson, M. Z. (2001). Reliability and validity of

three measures of multiculhual competency. Journal of Counseling t3 Development, 79,486-496. Krosnick, J. A. (1999). Maximizing questionnaire quality. In J. P. Robinson, P. R. Shaver, &

L. S . Wrightsman (Eds.), Measures ofpolitical attitudes : Val. 2. Measures of social psychological attitudes (pp. 37-57). San Diego, CA: Academic Press.

Ladany, N., Inman, A. G., Constantine, M. G., & Hofheinz, E. W. (1997). Supervisee multicultural case conceptualization ability and self-reported multicultural competence as functions of supervi- see racial identity and supervisor focus. Journal ofCounseling Psychology, 44,284-293.

LaFromboise, T. D., Coleman, H. L. K., & Hernandez, A. (1991). Development and factor structure of the Cross-Cultural Counseling Inventory-Revised. Professional Psychology: Research and Rutice, 22,380-388.

Likert, R. (1932). A technique for the measurement of attitudes. Archives ofPsychology (No. 140). Manese, J. E., Wu, J. T., & Nepomuceno, C. A. (2001). The effect of training on multicultural coun-

seling competencies: An exploratory study over a ten-year period. Journal ofMulticultural Counsel- ing and Development, 29, 31 -40.

Marsh, H. W., Antill, J. K., & Cunningham, J. D. (1989). Masculinilty and femininity: A bipolar construct and independent constructs. Journal of Personality, 57,625-663.

McDonald, R. P., & Marsh, H. W. (1990). Choosing a multivariate model: Noncentrality and good- ness-of-fit. Psychological Bulletin, 707,247-255.

Morrow, S. L., Rakhsha, G., & Castaneda, C. L. (2001). Qualitative research methods for multicultural counseling. In J. G. Ponterotto, J. M. Casas, L. A. Sumki, & C. M. Alexander (Eds.), Handbook of multicultural counseling (2nd ed., pp. 575-603). Thousand Oaks, CA: Sage.

Neimeyer, G. J., & Diamond, A. K. (2001). The anticipated future of counselling psychology in the United States: A Delphi poll. Counselling Psychology Quarterly, 74,49-65.

Paulhus, D. L. (1991). Measurement and control of response bias. In J. B. Robinson, P. R Shaver, & L. S . Wrightsman (Eds.), Measures ofpersonality and social psychological attitudes (pp. 17-59). San Diego, CA: Academic Press.

Phinney, J. S. (1992). The Multigroup Ethnic Identity Measure: A new scale for use with diverse groups. Journal ofAdolescent Research, 7, 156-176.

Ponterotto, J. G. (1996). Evaluating and selecting research instruments. In F. T. L. b o n g & J. T. Austin (Eds.), Z?E psychology research handbook: A guidejr graduate students and researh assistants (pp. 73-84). Thousand Oaks, C A Sage.

Ponterotto, J. G. (2002). Qualitative research methods: The fifth force in psychology. Zhe Counseling Aychologist, 30,394-406.

176 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 25: A Revision of the Multicultural Counseling Awareness Scale

Ponterotto, J. G., &Alexander, C. M. (1996). Assessing the multicultural competence of counselors and clinicians. In L. A. Suzuki, P. Meller, & J. G. Ponterotto (Eds.), Handbook ofmulticultural assessment (pp. 651-672). San Francisco: Jossey-Bass.

Ponterotto, J. G., & Casas, J. M. (1991). Handbook ofraciaUethnic minority counseling researdr Springfield, I L Thomas.

Ponterotto, J. G., Fuertes, J. N., & Chen, E. C. (2000). Models of multicultural counseling. In S. D. Brown & R W. Lent (Eds.), Handbook ofcnunselingpsyhlou (3rd ed., pp. 639-669). New York Wiley.

Ponterotto, J. G., Gretchen, D., Utsey, S. O., S t r a d , T, & Saya, R (in press). A critique and confirma- tory test of the Multigroup Ethnic Identity Measure (MEIM). EduGationolandPSrdrolo~GalMearurement.

Ponterotto, J. G., Jackson, M. A., & Nutini, C. D. (2001). Reflections on the life stories of pioneers in multicultural counseling. In J. G. Ponterotto, J. M. Casas, L A. Suzuki, & C. M. Alexander (Eds.), Handbook ofmulticultural counseling (2nd ed., pp. 138-161). Thousand Oaks, CA: Sage.

Ponterotto, J. G., Rieger, B. T., Barrett, A., Harris, G., Sparks, R, Sanchez, C. M., et al. (1996). Development and initial validation of the Multicultural Counseling Awareness Scale. In G. R. Sodowsky & J. C. Impara (Eds.), Multicultural assessment in counseling and clinical psychology (pp. 247-282). Lincoln, NE: Buros Institute of Mental Measurements.

Ponterotto, J. G., Rieger, B. T., Barrett, A., & Sparks, R. (1994). Assessing multicultural counseling competence: A review of instrumentation. Journal of Counseling B Development, 72, 316-322.

Ponterotto, J. G., Sanchez, C. M., & Magids, D. (1991, August). Initial development oftheMulticultura1 Counseling Awareness Scale. Paper presented at the annual meeting of the American Psychological Association, San Francisco.

Pope-Davis, D. B., & Coleman, H. L. K. (Eds.). (1997). Multicultural counseling competence: Assessment, education and training, and supervision Thousand Oaks, CA: Sage.

Pope-Davis, D. B., Coleman, H. L. K., Liu, W., & Toporek, R. (in press). Handbook ofmulticultural competencies. Thousand Oaks, C A Sage.

Pope-Davis, D. B., & Dings, J. G. (1994). An empirical comparison of two self-report multicultural counseling competency inventories. Measurement and Evaluation in Counseling and Development, 27,

Pope-Davis, D. B., & Dings, J. G. (1995). The assessment of multicultural counseling competencies. In J. G. Ponterotto, J. M. Casas, L. A. Suzuki, & C. M. Alexander (Us.), Handbook of multicullural counseling (pp. 287-311). Thousand Oaks, C A Sage.

Pope-Davis, D. B., Dings, J. G., & Ottavi, T. M. (1995). The relationship of multicultural counseling competency with demographic and educational variables. Iowa Psychologist, 4ql) , 12-13.

Pope-Davis, D. B., & Nielson, D. (1996). Assessing multicultural counseling competencies using the Multicultural Counseling Inventory: A review of research. In G. R. Sodowsky & J. C. Impara (Eds.), Multiculturalassarsment in counselingandclinicalpsychology (pp. 325-343). Lincoln, N E Buros Institute of Mental Measurements.

Pope-Davis, D. B., Reynolds, A. L., & Dings, J. G. (1994). Multicultural competencies of doctoral interns at university counseling centers: An exploratory investigation. Profissional @chology: Re- search and Practice, 25,466-470.

Pope-Davis, D. B., Toporek, R. L., Ortega-Villalobos, L., Ligiero, D., Brittal-Powell, C. S., Liu, W. M., et al. (2002). Client perspectives of multicultural counseling competence: A qualitative examination. n e Counseling Psychologist, 30,355-393.

Pope-Davis, D. B., Vandiver, B. J., &Stone, G. L. (1999). White racial identity attitude development: A psychometric examination of two instruments. Journal of Counseling Psychology, 46,70-79.

Quintana, S . M., & Maxwell, S. E. (1999). Implications of recent developments in structural equation modeling for counseling psychology. n e Counseling Psychologist, 27, 485-527.

Quintana, S. M., Troyano, N., & Taylor, G. (2001). Cultural validity and inherent challenges in quantitative research methods for multicultural counseling. In J. G. Ponterotto, J. M. Casas, L. A. Suzuki, & C. M. Alexander (Eds.), Handbook of multicultural counseling(2nd ed., pp. 604- 630). Thousand Oaks, CA: Sage.

Rahim, M. A., & Magner, N. R. (1995). Confirmatory factor analysis of the styles of handling interpersonal conflict: First-order factor model and its invariance across groups. Journal of Applied Psychology, 80, 122-132.

93-101.

JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 177

Page 26: A Revision of the Multicultural Counseling Awareness Scale

Russell, D. W., Kahn, J. H., Spoth, R., & Altmaier, E. M. (1998). Analyzing data from experi- mental studies: A latent variable structural equation modeling approach. Journal of Counseling

Sodowsky, G. R. (1996). The Multicultural Counseling Inventory: Validity and applications in multicultural training. In G. R. Sodowsky & J. C. Impara (Eds.), Multicultural msessment in counseling and clinical psychology (pp. 283-324). Lincoln, NE: Buros Institute of Mental Mea- surements.

Sodowsky, G. R., Kuojackson, P. Y., Richardson, M. F., & Corey, A. T. (1998). Correlates of self-reported multicultural competencies: Counselor multicultural social desirability, race, social inadequacy, locus of control racial ideology, and multicultural training. Journal of Counseling

Sodowsky, G. R., Taffe, R. C., Gutkin, T., &Wise, S. L. (1994). Development of the Multicultural Counseling Inventory: A self-report measure of multicultural competencies. Journal of Coun- seling Psychology, 4 7, 13 7- 148.

Sue, D. W., Arredondo, P., & McDavis, R. J. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal ofMulticultura1 Counseling and Development,

Sue, D. W., Bernier, J. E., Durran, A., Feinberg, L., Pedersen, P., Smith, E. J., et al. (1982). Position paper: Cross-cultural counseling competencies. The Counseling Psychologist, 10,

Sue, D. W., Carter, R. T., Casas, J. M., Fouad, N. A., Ivey, A. E., Jensen, M., et al. (1998). Multicultural counseling competencies: Individual and organiwtional development. Thousand Oaks, CA: Sage.

Sue, D. W., Ivey, A. E., & Pedersen, P. B. (1996). A theory of multicultural counseling and therajy. Pacific Grove, CA: Brooks-Cole.

Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analy- sis. Psychometrika, 38, 1-10.

Utsey, S. O., & Ponterotto, J. G. (1996). Development and validation of the Index of Race Related Stress (IRRS). Journal of Counseling Psychology, 43, 490-501.

Vinson, T. S., & Neimeyer, G. J. (2000). The relationship between racial identity development and multicultural counseling competence. Journal of Multicultural Counseling and Development,

Worthington, R. L., Mobley, M., Franks, R. P., & Tan, J. A. (2000). Multicultural counseling competencies: Verbal content, counselor attributions, and social desirability. Journal of Coun- seling Psychology, 47, 460-468.

PsycholOgy, 45, 18-29.

PsyChOlOgy, 45, 256-264.

20,45-52.

45-52.

28, 177-192.

ITEM CONTENT OF THE MULTICULTURAL COUNSELING KNOWLEDGE AND AWARENESS SCALE

1. I believe all clients should maintain direct eye contact during counseling. (A) 2. I check up on my minority/cultural counseling skills by monitoring my

functioning-via consultation, supervision, and continuing education. (K) 3. I am aware some research indicates that minority clients receive “less

preferred” forms of counseling treatment than majority clients. (K) 4. I think that clients who do not discuss intimate aspects of their lives are

being resistant and defensive. (A)

178 JOURNALOF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30

Page 27: A Revision of the Multicultural Counseling Awareness Scale

5. I am aware of certain counseling skills, techniques, or approaches that are more likely to transcend culture and be effective with any clients. (K)

6. I am familiar with the “culturally deficient” and “culturally deprived” depictions of minority mental health and understand how these labels serve to foster and perpetuate discrimination. (K)

7. I feel all the recent attention directed toward multicultural issues in counseling is overdone and not really warranted. (A)

8. I am aware of individual differences that exist among members within a particular ethnic group based on values, beliefs, and level of accul- turation. (K)

9. I am aware some research indicates that minority clients are more likely to be diagnosed with mental illness than are majority clients. (K)

10. I think that clients should perceive the nuclear family as the ideal social unit. (A)

11. I think that being highly competitive and achievement oriented are traits that all clients should work towards. (A)

12. I am aware of differential interpretations of nonverbal communication (e.g., personal space, eye contact, handshakes) within various racial/ ethnic groups. (K)

13. I understand the impact and operations of oppression and the racist concepts that have permeated the mental health professions. (K)

14. I realize that counselor-client incongruities in problem conceptualization and counseling goals may reduce counselor credibility. (K)

15. I am aware that some raciaVethnic minorities see the profession of psychology hnctioning to maintain and promote the status and power of the White Establishment. (K)

16. I am knowledgeable of acculturation models for various ethnic minority

17. I have an understanding of the role culture and racism play in the devel-

18. I believe that it is important to emphasize objective and rational thinking

19. I am aware of culture-specific, that is culturally indigenous, models of

20. I believe that my clients should view the patriarchal structure as ideal. (A) 21. I am aware of both the initial barriers and benefits related to the cross-

22. I am comfortable with differences that exist between me and my

23. I am aware of institutional barriers which may inhibit minorities from

24. I think that my clients should exhibit some degree of psychological

groups. (K)

opment of identity and world views among minority groups. (K)

in minority clients. (A)

counseling for various raciaVethnic groups. (K)

cultural counseling relationship. (K)

clients in terms of race and beliefs. (K)

using mental health services. (K)

qindedness and sophistication. (A)

JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30 179

Page 28: A Revision of the Multicultural Counseling Awareness Scale

25. I believe that minority clients will benefit most from counseling with a majority counselor who endorses White middle class values and norms. (A)

26. I am aware that being born a White person in this society carries with it certain advantages. (A)

27. I am aware of the value assumptions inherent in major schools of counseling and understand how these assumptions may conflict with values of culturally diverse clients. (K)

trary to their own life experiences and inappropriate or insufficient to their needs. (K)

29. I am aware that being born a minority in this society brings with it certain challenges that White people do not have to face. (A)

30. I believe that all clients must view themselves as their number one responsibility. (A)

31. I am sensitive to circumstances (personal biases, language dominance, stage of ethnic identity development) which may dictate referral of the minority client to a member of hidher own racial/ethnic group. (K)

32. I am aware that some minorities believe counselors lead minority students into non-academic programs regardless of student potential, preferences, or ambitions. (K)

28. I am aware that some minorities see the counseling process as con-

Note. The following items are reverse scored: 1,4, 7, 10, 11, 18,20,24,25, and 30. The Knowledge items are designated by the symbol K after the item, and the Awareness items are designated by the symbol A after the item.

180 JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT July 2002 Vol. 30