Embed Size (px)
Citation preview
An Item Response Theory Integration of Normal and AbnormalPersonality Scales
Douglas B. SamuelYale University School of Medicine
Leonard J. SimmsUniversity at Buffalo, State University of New York
Lee Anna ClarkUniversity of Iowa
W. John LivesleyUniversity of British Columbia
Thomas A. WidigerUniversity of Kentucky
The Diagnostic and Statistical Manual of Mental Disorders (DSM–IV–TR) currentlyconceptualizes personality disorders (PDs) as categorical syndromes that are distinctfrom normal personality. However, an alternative dimensional viewpoint is that PDsare maladaptive expressions of general personality traits. The dimensional perspectivepostulates that personality pathology exists at a more extreme level of the latent traitthan does general personality. This hypothesis was examined using item responsetheory analyses comparing scales from two personality pathology instruments—theDimensional Assessment of Personality Pathology-Basic Questionnaire (DAPP-BQ;Livesley & Jackson, in press) and the Schedule for Nonadaptive and Adaptive Person-ality (SNAP; Clark, 1993; Clark, Simms, Wu, & Casillas, in press)—with scales froman instrument designed to assess normal range personality, the NEO PersonalityInventory–Revised (NEO PI-R; Costa & McCrae, 1992). The results indicate thatrespective scales from these instruments assess shared latent constructs, with the NEOPI-R providing more information at the lower (normal) range and the DAPP-BQ andSNAP providing more information at the higher (abnormal) range. Nevertheless, theresults also demonstrated substantial overlap in coverage. Implications of the findingsare discussed with respect to the study and development of items that would providespecific discriminations along underlying trait continua.
Keywords: IRT, psychopathology, dimensional, personality disorder, assessment
The Diagnostic and Statistical Manual ofMental Disorders (DSM–IV–TR; AmericanPsychiatric Association [APA], 2000) repre-sents “the categorical perspective that Personal-ity Disorders are qualitatively distinct clinicalsyndromes” (p. 689). However, an alternativeperspective is that personality disorder criteriaare maladaptive, extreme versions of generalpersonality structure (Clark, 2007; Livesley,
2005; Widiger & Samuel, 2005). According tothis hypothesis, items from instruments assess-ing the DSM–IV personality disorder criteriaassess the same underlying constructs as generalpersonality inventories, albeit at more extremelevels. While much research has demonstratedthat instruments assessing personality pathol-ogy and those assessing normal personalitytraits do share common latent dimensions
Douglas B. Samuel, Department of Psychiatry, Yale Uni-versity School of Medicine; Leonard J. Simms, Departmentof Psychology, University at Buffalo, State University ofNew York; Lee Anna Clark, Department of Psychology,University of Iowa; W. John Livesley, Department of Psy-chiatry, University of British Columbia; Thomas A. Widi-ger, Department of Psychology, University of Kentucky.
This research was partially supported by a fellowship(F31MH074245-02) from the National Institute of Mental
Health, awarded to Douglas B. Samuel. Douglas B. Samuelis the author of the Dimensional Assessment of PersonalityPathology-Basic Questionnaire. Lee Anna Clark is affiliatedwith the Schedule of Nonadaptive and Adaptive Personal-ity. She has no financial interest in the product.
Correspondence concerning this article should be ad-dressed to Douglas B. Samuel, VA Connecticut HealthcareSystem–Building 35, 950 Campbell Avenue (151-D), WestHaven, CT 06516. E-mail: [email protected]
CORRECTED JUNE 24, 2010; SEE LAST PAGE
Personality Disorders: Theory, Research, and Treatment © 2010 American Psychological Association2010, Vol. 1, No. 1, 5–21 1949-2715/10/$12.00 DOI: 10.1037/a0018136
5
(Markon, Krueger, & Watson, 2005; O’Connor,2005; Schroeder, Wormsworth, & Livesley,1992), there has been very little research thathas tested whether personality disorder instru-ments assess the shared traits at more extremelevels than general personality instruments.However, such a comparison is possible usingitem response theory (IRT).
The field of psychological assessment hasbeen based largely in classical test theory(CTT), but significant advances in psychomet-rics have led to improved techniques for devel-oping and evaluating assessment instruments. Aprimary example of these advances is the appli-cation of IRT (for a detailed history and descrip-tion of IRT, see Embretson & Reise, 2000). IRTwas first introduced to psychology by way ofeducational testing, as a method of developingmore efficient measures of educational attain-ment or achievement. Only recently has it beenapplied to personality assessment, primarily todevelop computerized adaptive testing (CAT)versions of existing measures. For example, Re-ise and Henson (2000) reported on a CAT ver-sion of the NEO Personality Inventory–Revised(NEO PI-R; Costa & McCrae, 1992) using areal-data simulation, and Simms and Clark(2005) developed and validated an IRT-basedCAT for the Schedule for Nonadaptive andAdaptive Personality (SNAP; Clark, 1993).
Another potentially useful extension of IRTto the study of personality and personality pa-thology is its ability to compare the amount ofinformation that existing instruments provide atdifferent levels of a latent trait (Reise & Hen-son, 2000).1 Items typically vary in the amountof information they provide across levels of atrait. For example, some items may providelittle information at low levels of a trait (e.g., allpersons within the lower range provide the sameanswer), but a great deal of information athigher levels (i.e., persons at the higher levels ofthe trait respond differentially to the item).Thus, as long as items from different measurescan be shown to load on the same latent dimen-sion, they can be compared in terms of thelevels of that latent trait where they provide thegreatest discrimination. It is this aspect of IRTthat could be used to compare where measuresof normal and abnormal personality functioningprovide more or less information along an un-derlying latent continuum.
We are aware of only two published studiesthat have used IRT in this manner (Walton,Roberts, Krueger, Blonigen, & Hicks, 2008;Zickar, Russell, Smith, Bohle, & Tilley, 2002;and the latter is not directly pertinent to thecurrent study as it addresses time-of-day pref-erences for working). Walton and colleaguesadministered the Psychopathic Personality In-ventory (PPI; Lilienfeld & Andrews, 1996) andthe Multidimensional Personality Questionnaire(MPQ; Tellegen, 1982). Subsets of items fromeach instrument that were deemed to assess thesame underlying construct were subjected toIRT analysis. Information curves were calcu-lated for each instrument and presented on acommon graph. Walton and colleagues con-cluded from a visual inspection of these curvesthat, inconsistent with the hypothesis that psy-chopathy items were assessing extreme variantsof normal personality, the instruments did notappear to be assessing different regions of thelatent trait. This may be because the PPI as-sesses personality traits that are risk factors forpsychopathy rather than psychopathic personal-ity per se. These two studies have demonstrateda novel use of IRT techniques; however, giventhat the results of Walton et al. (2008) wereinconsistent with hypotheses, a closer examina-tion of normal and abnormal personality scalesis warranted.
One of the most heavily researched models ofgeneral personality functioning is the Five Fac-tor Model (FFM; McCrae & Costa, 1999),which has the five broad dimensions of extra-version (vs. introversion), agreeableness (vs.antagonism), neuroticism (vs. emotional stabil-ity), conscientiousness (vs. undependability),and openness (vs. closedness to experience).Over the past two decades, the FFM has pro-vided a useful dimensional framework for un-derstanding the DSM personality disorders, andwell over 50 published studies support the linkbetween them (Widiger & Costa, 2002). Ameta-analysis of a number of these studies(Samuel & Widiger, 2008), reviews of this re-search (Clark, 2007; Livesley, 2001), and aninterbattery factor analysis of published data
1 In a literal sense, the phrase “IRT-based analyses”should be used (rather than simply “IRT”) in this sentenceand many others in this article. However, it is most commonin the literature to refer to both the theory and its applicationsimply by “IRT” and we shall follow this usage.
6 SAMUEL ET AL.
sets that examined relations between the FFMand the personality disorders (O’Connor, 2005)all have led to the conclusion that there arestrong and robust links between the DSM–IVPD formulations and dimensions of normal per-sonality. Thus, the FFM is a compelling candi-date to assess general personality traits withinan IRT-based comparison.
Two instruments measuring maladaptive per-sonality traits that would lend themselves to anintegrated IRT study of the common latentstructure underlying normal and abnormal per-sonality are the Dimensional Assessment ofPersonality Pathology–Basic Questionnaire(DAPP-BQ; Livesley & Jackson, in press) andthe SNAP (Clark, 1993; Clark et al., in press).Both instruments were derived through an iter-ative process that included factor analyses ofpersonality disorder symptomatology.
Exploratory factor analytic studies have dem-onstrated empirically that the dimensions ofmaladaptive personality functioning assessedby the DAPP-BQ and the SNAP are well inte-grated with at least four of the five domains ofthe FFM (Clark & Livesley, 2002). For exam-ple, Schroeder and colleagues (1992) exam-ined the DAPP-BQ and the NEO PI (Costa &McCrae, 1985) in a sample of 300 communitymembers, and found that four components of afive-component solution mapped cleanly ontothe FFM (i.e., neuroticism, extraversion, agree-ableness, and conscientiousness), whereas the“openness” domain included a considerableloading of scales assessing extraversion.
Clark and colleagues (in press) report theresults of three principal factor analyses of theSNAP and various FFM measures, each ofwhich yielded a five-factor solution that closelymirrored four of the five domains of the FFM.An openness factor emerged more strongly thanin the FFM/DAPP-BQ analyses, but still wasdefined inconsistently across the three samples.Thus, Clark and colleagues (in press) concludedthat openness was not well represented in theSNAP item pool. Clark, Livesley, Schroeder,and Irish (1996) also provided evidence for theconvergence of the DAPP-BQ and the SNAPvia joint exploratory factor analysis of the twoinstruments. Five factors were extracted, four ofwhich corresponded well to neuroticism, extra-version, agreeableness, and conscientiousness,whereas, similar to the previous studies, open-
ness appeared not to be well represented ineither instrument.
Finally, Markon et al. (2005) conducted aseries of exploratory factor analyses of a meta-analytically derived correlation matrix as wellas new data sets that included the DAPP-BQ,SNAP, NEO PI-R, and other measures of nor-mal and abnormal personality functioning. Thisstudy explored how normal and abnormal per-sonality scales might be integrated within acommon hierarchical structure. The authorsconcluded that their “results reinforce the posi-tion that the Big Five represent a crucial level ofanalysis for normal personality research andextend this position to include psychopathologyresearch as well” (p. 154). Of specific relevanceto the current study was the further empiricaldocumentation of a common underlying traitstructure among the DAPP-BQ, SNAP, andNEO PI-R scales.
In summary, extensive research supports theview that these three measures of general andmaladaptive personality functioning share acommon four-factor structure and thereforecould be amenable to IRT analysis. These IRTanalyses will provide data on whether person-ality pathology instruments assess the sharedlatent traits at more extreme levels than generalpersonality measures. For example, to the ex-tent that the DAPP-BQ compulsivity and SNAPworkaholism scales measure maladaptive ex-treme variants of NEO PI-R conscientiousness,the compulsivity and workaholism items shouldprovide more psychometric information athigher (i.e., more severe) levels of the underly-ing trait than the NEO PI-R items. In turn, theNEO PI-R items should provide more informa-tion at lower (i.e., less severe) levels of the trait.Comparable hypotheses can be made for otherDAPP-BQ, SNAP, and NEO PI-R items withrespect to the three additional broad latent fac-tors underlying their integration identified inprevious research.
Method
Samples and Participants
The data for the current study were drawnfrom two separate data collections. The firstincluded 920 individuals who were adminis-tered the DAPP-BQ and the NEO PI-R as partof an adult community sample collected in Brit-
7AN IRT INTEGRATION OF PERSONALITY SCALES
ish Columbia, Canada (Jang, Livesley, & Ver-non, 2002). The sample was predominantly fe-male (63%) and had a mean age of 33.6 years(SD � 13.8). A second sample included 680students at the University of Kentucky with amean age of 19.8 years (SD � 4.4) who com-pleted the SNAP and the NEO PI-R to fulfillcourse credit. The majority was female (62%)and Caucasian (85%), with 10% African Amer-icans and 5% other ethnic groups. Portions ofthis dataset have been used in previously pub-lished studies (e.g., Mullins-Sweatt, Jamer-son, Samuel, Olson, & Widiger, 2006). De-scriptive statistics for the scales from each sam-ple are presented in online Appendix A.
Measures
NEO PI-R. The NEO PI-R (Costa & Mc-Crae, 1992) is a measure of the FFM and con-tains 240 items that are rated on a Likert-typescale ranging from 1 (strongly disagree) to 5(strongly agree). This instrument is comprisedof five broad domain scales—neuroticism, ex-traversion, openness to experience, agreeable-ness, and conscientiousness—each of which isassessed by six underlying facet scales. Internalconsistency is high for the domains (coefficientalphas range from .86 to .95) and moderate tostrong (.56 to .81) for the facet scales (Costa &McCrae, 1992). The NEO PI-R has evidencedstrong temporal stability, with values rangingfrom .76 to .84 over a 7-year period (Costa,Herbst, McCrae, & Siegler, 2000).
DAPP-BQ. The DAPP-BQ (Livesley &Jackson, in press) contains 560 statements towhich an individual responds on a 5-point Lik-ert-type scale ranging from “strongly disagree”to “strongly agree.” The DAPP-BQ includes 18scales developed in part through factor analysisof personality disorder symptomatology; for ex-ample, affective lability, social avoidance, con-duct problems, and compulsivity. These scalesare internally consistent with coefficient alphasranging from .83 to .94 and reliable over 3-weekintervals with test–retest reliability rangingfrom .81 to .93 (Larstone, Jang, Livesley, Ver-non, & Wolf, 2002).
SNAP-2. The SNAP-2 (Clark et al., inpress) is a 390-item instrument that uses a True-False format. The SNAP includes 12 lower-order trait scales developed in part through fac-tor analyses of personality disorder symptom-
atology, such as self-harm, aggression,manipulation, and workaholism. It also includesthree “temperament” scales that assess, respec-tively, the core of three higher order personalitydomains, Negative Temperament, PositiveTemperament, and Disinhibition, but thesescales were not used in the current study. SNAPscales are internally consistent (median coeffi-cient alphas ranged from .76 to .92; median �.81 in samples of patients and nonpatient col-lege students and adults) and are stable overshort (1-week; retest r range � .81 to .93;median � .88) and moderate (up to 4 months;retest r range � .76 to .89; median � .85)intervals (Clark et al., in press).
Domain Configuration
We first arranged the items from each instru-ment into the four domains consistent with theprior factor analytic research discussed earlier:(1) emotional instability, (2) antagonism, (3)introversion, and (4) constraint. In accordance,all items were keyed in the direction of theseconstructs. This grouping was done at the scalelevel because each of the four domains typicallywas comprised of all items from one NEO PI-Rdomain and all items from related DAPP-BQscales. For example, one of the four domainsidentified by the joint factor analytic studiesincluded the neuroticism scale from the NEOPI-R as well as the anxiety, suicidal ideation,insecure attachment, affective lability, identitydisturbance, and submissiveness scales from theDAPP-BQ (Markon et al., 2005; Schroeder etal., 1992). Similarly, the items from the SNAPscales of detachment and exhibitionism werejoined with NEO PI-R extraversion scale toform a group hereafter referred to as the intro-version domain.
Analytic Procedures and Results
Unidimensionality Assessment
An assumption underlying IRT models is thatitems being analyzed form a unidimensionallatent construct. This was particularly importantin the current study because the items wereobtained from different instruments. Becausethe SNAP utilizes a dichotomous, “true-false”format, the NEO PI-R items were also dichot-omized so that they could be compared in a
8 SAMUEL ET AL.
straightforward manner; specifically, responses“strongly disagree” and “disagree” were re-coded as false, whereas “agree” and “stronglyagree” were recoded as true. Also, because therecoding of the NEO PI-R’s “neutral” responseoption has the potential to affect our results, wechose to recode “neutral” as “false” to provide aconservative and more stringent test of our hy-potheses (i.e., recoding in this way makes theitems more “difficult” in an IRT framework andthus biases them slightly toward abnormality).
Stout (1987, 1990) has argued that what isrequired for IRT is not the absence of anysubfactors, but the presence of a single, domi-nant factor that is common to the items. Thus,we sought to demonstrate that the underlyingtraits were essentially unidimensional for thepurposes of IRT, meaning that a broad, generaldimension underlies all item responses.
We used the MicroFACT 2.0 (Waller, 2002)software program to compute three statisticsassessing the fit of a one-factor model to thedata in each of the four domains. Consistentwith past studies, we first calculated the ratio ofthe first to second eigenvalue of the polychoriccorrelation matrix to assess the presence of adominant first factor. Additionally, we calcu-lated the goodness-of-fit index (GFI), an indi-cator of absolute fit for a one-factor solution.Values over .90 are considered to be evidenceof good fit and those over .95 indicative of anexcellent fit (Hu & Bentler, 1999). Finally, wealso calculated the root-mean-square residual(RMSR), for which lower values indicate bettermodel-to-data fit and those under .10 suggestessential unidimensionality. Although there isno infallible statistical indicator of latent struc-ture, these three measures taken together areconsidered to provide adequate information toevaluate the assumption of essential unidimen-sionality (Stout, 1987).
These unidimensionality indices indicatedthat only one of the newly sorted domains (e.g.,DAPP-BQ/NEO PI-R emotional instability) ev-idenced essential unidimensionality; the otherswere insufficient for IRT analyses. When uni-dimensionality was not clearly evidenced, itemswith low (typically �.50) factor loadings wereremoved. We examined these deleted itemsfor possible inclusion in another domain, butnone was retained. Following these deletions,the remaining items were reassessed for unidi-mensionality.
Table 1 presents the final unidimensionalityresults for each domain from the two sets of com-parisons. The values in Table 1 are generally at orabove the criteria indicating essential unidimen-sionality (Stout, 1987) and are comparable tothose reported in previous IRT studies with mea-sures of personality (Jane et al., 2007; Reise &Henson, 2000; Reise, Smith, & Furr, 2001; Simms& Clark, 2005). The ratio of the first to secondeigenvalues in the current study ranged from 4.1to 6.7, which compares well with previous reportsof this statistic (Bolt, Hare, Vitale, & Newman,2004; Cooke & Michie, 1997; Jane et al., 2007;Reise et al., 2001; Smith & Reise, 1998). Wefurther investigated the presence of a dominantfirst factor by examining a scree plot for each ofthe domains (see online Appendix B). Togetherwith the unidimensionality values presented inTable 1, these scree plots indicate that the newlycreated domains met Stout’s (1990) criteria foressential unidimensionality and were amenable toIRT analysis.
Content Analysis
The process of refining these scales inherentlyreduced the number of items that were analyzed.While this reduction was helpful in satisfying unidi-mensionality, it raised questions about whether theresulting scales were sufficiently similar in content to
Table 1Unidimensionality Assessment Values
Ratio GFI RMSR
DAPP-BQEmotional instability (140) 6.7 0.94 0.076Antagonism (114) 4.7 0.89 0.091Introversion (31) 5.6 0.96 0.089Constraint (73) 5.0 0.93 0.083
SNAPEmotional instability (30) 4.4 0.93 0.111Antagonism (51) 4.6 0.91 0.107Introversion (26) 5.5 0.96 0.091Constraint (37) 4.1 0.93 0.108
Note. Ratio � the ratio of the first to second eigenvalue;GFI � goodness-of-fit index; RMSR � root mean squared;DAPP-BQ � the Dimensional Assessment of PersonalityPathology–Basic Questionnaire; SNAP � Schedule forNonadaptive and Adaptive Personality. The first headingindicates the instruments being compared with the NEOPersonality Inventory–Revised (NEO PI-R), and the in-dented heading indicates the domain being examined. Thenumber within the parentheses is the final number of itemsincluded within the domain for that particular analysis.
9AN IRT INTEGRATION OF PERSONALITY SCALES
the originals. To investigate this possibility, we ex-amined the content from those items that were re-tained and compared it to those items that wereexcluded. In other words, we examined whether (a)the remaining items within the SNAP/NEO PI-Rand DAPP-BQ/NEO PI-R comparisons were faith-ful to the original item pools and (b) the remainingitems within these two comparisons were similar toeach other. To do so, we first counted the number ofitems that were retained from each facet of the NEOPI-R and the SNAP and DAPP-BQ scales (see on-line Appendix C). From these counts, it appearedthat certain scales and facets were more stronglyrepresented than others. Not surprisingly, it appearedthat the items from NEO PI-R facets most closelyrelated in content to the DAPP-BQ and SNAP scaleswere more likely to be retained.
For example, these content analyses evidencedthat the remaining emotional instability construct forthe SNAP and NEO PI-R appears to be largelydefined by negative mood and hopelessness and issomewhat narrower than the broader construct in-cluding anxiousness, anger, and impulsiveness thatemerged from the comparison of the DAPP-BQ andNEO PI-R. For the introversion domain it appearedthat the construct was quite similar across both com-parisons and was characterized by social withdrawaland emotional coldness. Although there were differ-ences in the conceptualizations of antagonism, de-pending largely on the content of the DAPP-BQ andSNAP, it appeared that the constructs were quitesimilar across the two comparisons and the remain-ing items had high fidelity with the original content.Finally, some dissimilarity was again noted for theconstraint domain as a majority of the items fromboth measures were retained for the DAPP-BQ/NEO PI-R comparison, suggesting that the meaningof the combined dimension changed little from itscomponent parts. However, a number of items wereexcluded across the NEO PI-R facets and SNAPscales such that the SNAP/NEO PI-R comparisonwas perhaps more heavily laden with impulsivitythan the DAPP-BQ/NEO PI-R comparison.
IRT Analyses
We chose Samejima’s (1969) graded re-sponse model to estimate the item parametersfor the analyses of the DAPP-BQ and the NEOPI-R within this comparison because both in-struments use a 5-point Likert scale. The gradedresponse model is an extension of the two-parameter logistic model for polytomous items
and is commonly used for IRT analyses of per-sonality instruments with Likert-type scales(e.g., Reise & Henson, 2000). As the SNAP andNEO PI-R items were both dichotomous for thiscomparison (using the procedures outlined ear-lier), these analyses were conducted using thetwo-parameter logistic model. The IRT param-eters for all analyses were estimated using Mul-tilog 7.03 (Thissen, Chen, & Bock, 2003).
The primary results of interest for the currentstudy are the item information curves (IICs),which show the amount of psychometric infor-mation that each item provides at all levels ofthe latent trait. An important property of iteminformation curves is that they can be summedor averaged to provide an overall estimate ofmeasurement precision for a complete scaleacross all levels of the underlying construct.Because total information curves are sensitiveto scale length, simply summing the informa-tion curves for a scale would have producedresults that were biased by the number of itemsretained for the scale. In order to place eachscale and instrument on a more level playingfield we chose to average the IICs to control forlength. We termed these “mean informationcurves” (MICs). Within the DAPP-NEO andSNAP-NEO domains, separate MICs were cal-culated for each of the scales that comprised thedomain. For example, separate MICs were cal-culated for the SNAP scales of detachment andexhibitionism as well as for the NEO PI-R do-main of extraversion. The Multilog softwareprovides an estimate of the psychometric infor-mation at levels of theta ranging from �3.0to 3.0, at intervals of .02. Thus, the mean iteminformation values were tested among scales ateach interval through a series of one-way anal-yses of variance (ANOVAs), with Tukey posthoc contrasts. This allowed for a statistical com-parison at each interval of theta to determinewhether scales were providing different levelsof information. Because space limitations pre-clude presentation of all possible MICs, weprovide only a few illustrative examples. Whenexamining these MICs we were looking fornotable differences in the height of the curve atdifferent points along the continuum. Thegreater the distance between the two lines thegreater the difference between the amount ofinformation that each respective scale is provid-ing at that particular level of the trait.
10 SAMUEL ET AL.
Figure 1 presents the MICs for selected scalesfrom the DAPP-NEO antagonism domain. Inthis case, the NEO agreeableness curve ap-peared to be relatively flat, indicating that it wasproviding roughly equivalent information at alllevels of antagonism. By contrast, the twoscales from the DAPP-BQ provided increasingpsychometric information as the level of thetaincreased. Statistical comparisons indicated thatthe curve for the NEO PI-R agreeableness scalewas significantly higher than both DAPP-BQscales from thetas of �3.0 to �2.6. Similarly,the DAPP-BQ conduct problems scale providedsignificantly more information than eitherDAPP-BQ interpersonal disesteem or NEOPI-R agreeableness at levels of theta from �1.2to 0. Finally, the curves for both of theDAPP-BQ scales were significantly higher thanthe NEO PI-R agreeableness curve at levels oftheta above 0.4, indicating that these DAPP-BQitems provided more information at the highestlevels of the latent trait.
Figure 2 presents the MICs for the scalescomprising the SNAP-NEO introversion do-main. A visual inspection of this figure indi-cated that the curves for all three of these scalespeak at roughly the same level of the latent trait.However, while their location along theta wascomparable, there were differences between the
scales in terms of the amount of informationprovided. This was particularly evident at levelsof theta ranging from 0.4 to 1.4, where theSNAP detachment scale provided significantlymore information than the NEO PI-R extraver-sion scale, whereas the NEO PI-R scale pro-vided more information than the SNAP exhi-bitionism scale. Thus, the findings from thisparticular analysis did not suggest that theSNAP and NEO PI-R provided information atdifferent levels of the latent trait. Rather, theyindicated that the SNAP detachment scale pro-vided more fidelity in assessing introversionwithin a selected range of theta.
Figure 3 presents selected MICs from theDAPP-NEO emotional instability domain. Avisual inspection of these curves suggestedthat DAPP-BQ affective lability and NEOPI-R neuroticism covered the latent trait ofemotional instability in very similar ways. Infact, both of these scales evidenced a moderateassessment of the latent trait across all levels oftheta and were not significantly different fromone another. However, the curve for the suicidalideation scale from the DAPP-BQ provided vir-tually no psychometric information at the lowerlevels of theta before spiking upward to providea great deal of information at the highest levelsof theta. The results of a one-way ANOVA
Figure 1. DAPP-BQ and NEO PI-R antagonism mean information curves.
11AN IRT INTEGRATION OF PERSONALITY SCALES
indicated that the DAPP-BQ suicidal ideationcurve was significantly lower than bothDAPP-BQ affective lability and NEO PI-Rneuroticism at the levels of theta from �3.0to 0.4. Conversely, the suicidal ideation curvewas higher than both of these scales at alllevels of theta above 1.4. Given this curve, it
appeared that the suicidal ideation scale pro-vided discrimination only among individualswho were quite extreme on the trait of emo-tional instability.
In lieu of providing MICs for every scalewithin each set of comparisons, Tables 2 and3 present the mean alpha and beta parameters
Figure 2. SNAP and NEO PI-R introversion mean information curves.
Figure 3. DAPP-BQ and NEO PI-R emotional instability mean information curves.
12 SAMUEL ET AL.
Tab
le2
Com
pari
sons
ofB
eta
Par
amet
ers
for
NE
OP
erso
nali
tyIn
vent
ory–
Rev
ised
(NE
OP
I-R
)an
dD
imen
sion
alA
sses
smen
tof
Per
sona
lity
Pat
holo
gy–B
asic
Que
stio
nnai
re(D
AP
P-B
Q)
Scal
es
Alp
hab1
b2b3
b4
MSE
MSE
MSE
MSE
MSE
DA
PP-N
EO
Em
otio
nal
Inst
abili
ty1.
NE
ON
euro
ticis
m(4
8/48
)1.
02.1
1�
3.80
.44
�0.
76.1
80.
54.2
03.
50.5
62.
DA
PPIn
secu
reA
ttach
men
t(1
6/16
)0.
97.1
0�
1.22
.16
0.40
.16
1.85
.30
3.55
.54
3.D
APP
Anx
iety
(16/
16)
1.57
.13
�1.
34.1
0�
0.25
.08
0.76
.12
1.96
.21
4.D
APP
Aff
ectiv
eL
abili
ty(1
6/16
)1.
14.1
1�
1.82
.17
�0.
42.1
10.
91.1
72.
38.3
35.
DA
PPId
entit
yD
istu
rban
ce(1
6/16
)1.
47.1
2�
0.89
.08
0.30
.10
1.45
.18
2.64
.32
6.D
APP
Subm
issi
vene
ss(1
6/16
)0.
90.0
9�
2.33
.26
�0.
52.1
41.
46.2
63.
81.5
87.
DA
PPSu
icid
alId
eatio
n(1
2/12
)1.
51.2
21.
39.2
31.
91.3
02.
53.3
93.
32.5
8Si
gnifi
cant
diff
eren
ces
with
inco
lum
n3
�1,
2,4,
67
�1,
2,3,
4,5,
67
�1,
2,3,
4,5,
67
�1,
3,4,
5,6
3�
1,2
5,7
�1,
2,6
2,3,
4,5,
6,7
�1
2,5,
7�
12
�1,
3,4
2,5,
6,7
�1
DA
PP-N
EO
Intr
over
sion
1.N
EO
Ext
rave
rsio
n(1
3/48
)1.
30.1
1�
1.59
.13
0.62
.11
1.75
.19
3.59
.44
2.D
APP
Intim
acy
Prob
lem
s(3
/16)
1.12
.11
�0.
45.0
90.
91.1
42.
34.2
73.
68.4
63.
DA
PPR
estr
icte
dE
xpre
ssio
n(5
/16)
1.21
.11
�1.
19.1
10.
12.1
01.
39.1
62.
76.3
04.
DA
PPSo
cial
Avo
idan
ce(1
0/16
)2.
02.1
4�
0.73
.06
0.13
.06
1.07
.18
2.05
.18
Sign
ifica
ntdi
ffer
ence
sw
ithin
colu
mn
4�
1,2,
32,
4�
1ns
2�
41,
2�
4D
APP
-NE
OA
ntag
onis
m1.
NE
OA
gree
able
ness
(38/
48)
0.89
.11
�2.
69.3
40.
56.2
72.
26.4
15.
07.8
82.
DA
PPN
arci
ssis
m(1
5/16
)1.
12.1
1�
1.85
.16
�0.
66.1
10.
67.1
72.
30.3
43.
DA
PPSu
spic
ious
ness
(14/
14)
1.45
.13
�0.
93.1
70.
26.1
31.
40.2
02.
71.4
04.
DA
PPC
ondu
ctPr
oble
ms
(16/
16)
1.19
.15
0.18
.14
1.02
.22
1.85
.31
2.84
.46
5.D
APP
Rej
ectio
n(1
5/16
)1.
01.1
12.
40.2
8�
0.81
.15
0.77
.19
2.78
.45
6.D
APP
Inte
rper
sona
lD
ises
teem
(16/
16)
1.33
.13
�0.
59.1
00.
66.1
51.
77.2
62.
99.4
4Si
gnifi
cant
diff
eren
ces
with
inco
lum
n4,
6�
14,
6�
1,2,
51,
4,6
�2,
51
�2,
51
�2,
3,4,
53
�1,
53
�1,
5D
APP
-NE
OC
onst
rain
t1.
NE
OC
onsc
ient
ious
ness
(45/
48)
1.13
.11
�3.
95.5
9�
1.27
.22
0.02
.15
2.68
.30
2.D
APP
Com
puls
ivity
(13/
16)
1.19
.11
�2.
92.4
0�
1.45
.20
0.29
.13
2.09
.24
3.D
APP
Pass
ive
Agg
ress
ive
(15/
16)
1.38
.12
�2.
62.3
0�
1.31
.16
�0.
17.0
91.
03.1
1Si
gnifi
cant
diff
eren
ces
with
inco
lum
nns
2,3
�1
nsns
1�
3
Not
e.ns
�no
tsig
nfica
nt.S
igni
fican
ceva
lue
seta
tp�
.05;
b1,b
2,b3
,and
b4�
1st,
2nd,
3rd,
and
4th
beta
para
met
ers,
resp
ectiv
ely.
The
prop
ortio
nin
the
pare
nthe
ses
follo
win
gea
chsc
ale
nam
ein
dica
tes
the
num
ber
ofite
ms
reta
ined
(num
erat
or)
from
the
num
ber
ofite
ms
inth
eor
igin
alsc
ale
(den
omin
ator
).
13AN IRT INTEGRATION OF PERSONALITY SCALES
for each scale (for complete item-level informa-tion, please consult online Appendixes Dthrough K). IRT analyses estimate two param-eters for each item, “alpha” and “beta.” Alpha,which is also referred to as the slope or discrim-ination parameter, corresponds to the item’sability to discriminate between individuals andcan be analogized to the item’s quality. Betacorresponds to the level of the latent trait that isrequired for an individual to endorse a givenresponse with a 50% probability. Within intel-lectual assessment, beta is often analogized asthe item’s “difficulty” but within personalityand psychopathology assessment it might more
accurately be referred to as an item’s “extrem-ity” or “severity.” Also note that the DAPP-BQ/NEO PI-R comparison contains four beta val-ues, while the SNAP/NEO PI-R comparisononly has a single beta. This difference is a resultof the different response formats employed. Thedichotomous items within the SNAP/NEO PI-Rcomparisons indicate the point along each traitat which the probability of responding “true”begins to exceed that of responding “false.”Conversely, the Likert-type items from theDAPP-BQ/NEO PI-R comparison have five re-sponse options and the four beta values corre-spond to the interval between each of theseoptions. For example, b1 within Table 2 indi-cates the level of the latent trait at which thelikelihood of responding “disagree” becomeshigher than that of responding “strongly dis-agree” to the average item within each scale.Much like the MICs, we calculated the valueswithin Tables 2 and 3 by averaging the betaparameters from each item within each respec-tive scale as well as the standard errors for eachbeta.
Tables 2 and 3 also include a summary oftests to compare the relative magnitudes of themean beta parameters. For these tests, we con-ducted a series of one-way ANOVAs withineach domain such that items were treated ascases, each scale’s membership was treated asthe independent variable, and beta was the de-pendent variable. This was followed withTukey’s post hoc comparisons for each set ofscales. Table 2 contains a legend specifying theinstances for which the differences in the meanbeta parameters were significantly different be-tween two scales ( p � .05). For example, whencomparing the SNAP and NEO PI-R within thedomain of emotional instability, the main effectwas significant, F(2, 27) � 4.05. Post hoc testsrevealed that the SNAP self-harm mean (1.41)was significantly higher than the mean value forthe NEO PI-R neuroticism items (0.84), but notdifferent from the SNAP dependency mean(1.15).
This finding is readily interpretable for thedichotomous items within the SNAP and NEOPI-R comparisons. The difference in these meanbetas indicates that the level of the latent traitrequired to endorse an item with a 50% proba-bility is significantly higher for the SNAP self-harm items than for the NEO PI-R neuroticismitems. Table 3 also indicates that the differences
Table 3Comparisons of Beta Parameters for NEOPersonality Inventory–Revised (NEO PI-R) andSchedule for Nonadaptive and Adaptive Personality(SNAP) Scales
Alpha Beta
M SE M SE
SNAP-NEO Emotional Instability1. NEO Neuroticism (12/48) 1.50 .22 0.84 .142. SNAP Self-Harm (12/16) 1.70 .28 1.41 .183. SNAP Dependency (6/18) 1.04 .19 1.15 .23Significant differences within
column 1,2 � 3 2 � 1SNAP-NEO Introversion
1. NEO Extraversion (12/48) 1.68 .23 0.68 .102. SNAP Detachment (7/18) 2.17 .29 0.92 .103. SNAP Exhibitionism (7/16) 1.10 .18 0.74 .16Significant differences within
column 2 � 1 � 3 nsSNAP-NEO Antagonism
1. NEO Agreeableness (18/48) 1.26 .20 0.44 .152. SNAP Manipulativeness
(11/20) 1.40 .23 0.78 .153. SNAP Entitlement (3/16) 1.05 .19 1.05 .214. SNAP Mistrust (4/19) 1.08 .18 0.72 .175. SNAP Aggression (15/20) 1.68 .28 1.00 .15Significant differences within
column 5 � 1,4 5 � 1SNAP-NEO Constraint
1. NEO Conscientiousness(24/48) 1.68 .47 �1.05 .43
2. SNAP Impulsivity (9/19) 1.19 .25 �0.64 .293. SNAP Workaholism (4/18) 0.80 .36 �0.60 .58Significant differences within
column 1 � 2,3 2 � 1
Note. ns � nonsignificant. Significance value set at p �.05. The proportion in the parentheses following each scalename indicates the number of items retained (numerator)from the number of items in the original scale (denomina-tor).
14 SAMUEL ET AL.
between the mean beta values for NEO PI-Rextraversion, the SNAP detachment, and SNAPexhibitionism scales were not significant, F(2,23) � .88. In contrast, the main effect for theSNAP-NEO antagonism ANOVA was signifi-cant, F(4, 46) � 2.60. Post hoc tests revealedthat the SNAP aggression scale had a signifi-cantly higher beta than the NEO PI-R agree-ableness scale. Finally, the main effect for theSNAP-NEO constraint comparison was signifi-cant, F(2, 34) � 4.38, and Table 3 indicates thatthe SNAP Impulsivity scale had a significantlyhigher beta than did the NEO PI-R conscien-tiousness scale.
The interpretation of the four beta parameterswithin the DAPP-NEO comparison in Table 2 issomewhat more complex so we chose to focuson the third beta parameter (i.e., b3), whichcorresponds to the point at which we dichoto-mized the NEO items for the SNAP compari-son. Within the emotional instability domain,the main effect was significant, F(6,133) � 13.38, and post hoc tests revealed thatthe DAPP-BQ suicidal ideation scale had a b3value that was significantly higher than thosefrom all other scales except DAPP-BQ insecureattachment which, in turn, also was significantlyhigher than the DAPP-BQ anxiety and affectivelability scales. The NEO PI-R neuroticism scalewas generally lower than the values for mostDAPP-BQ scales and these differences weresignificant for suicidal ideation, submissive-ness, identity disturbance, and insecure attach-ment. Within the introversion domain, the onlysignificant difference found at the third beta wasthe DAPP-BQ intimacy problems scale havinga higher beta than the DAPP-BQ social avoid-ance scale, F(3, 27) � 3.44. The DAPP-NEOcomparison was significant, F(5, 108) � 4.90and post hoc tests indicated the agreeablenessscale had a mean beta value that was signifi-cantly higher than both the DAPP-BQ narcis-sism and rejection scales. There were no signif-icant differences found within the constraintdomain among the third beta values for theDAPP-NEO comparison.
Discussion
A great deal of research has suggested thatinstruments assessing maladaptive personalitytraits share a common higher order structurewith four of the five FFM domains (Clark, 1993,
2007; Clark et al., 1996; Clark & Livesley,2002; Livesley, 2003; Markon et al., 2005;O’Connor, 2005; Schroeder et al., 1992; Widi-ger & Samuel, 2005; Clark et al., in press).However, to date, the research has been con-fined to exploratory factor analytic studies. Inthe current study, scales and items from theDAPP-BQ, SNAP, and NEO PI-R were sortedinto four higher order domains: emotional in-stability, antagonism, introversion, and con-straint. The items within each domain then weresubjected to nonlinear factor analysis to inves-tigate the degree to which the different mea-sures assessed a common, unidimensional latenttrait. The results provided evidence that scalesassessing normal personality and abnormal per-sonality traits share a common dimensionalstructure. For example, when the NEO PI-Ritems assessing extraversion were pooled withthose from the SNAP scales of exhibitionismand detachment, the resulting scale appeared toshow essential unidimensionality (Stout, 1987),indicating that the items all assess a sharedlatent construct identified in the current study asintroversion. Comparable findings also were ob-tained when the NEO PI-R scales and thosefrom the DAPP-BQ and SNAP were combinedinto domains identified herein as emotional in-stability, antagonism, and constraint. Thus,these findings are consistent with previous evi-dence that items assessing personality pathol-ogy and normal personality traits form at leastfour unidimensional domains. In addition, theresults of the current study go beyond theseanalyses to apply an IRT perspective to supportthe hypothesis that the maladaptive traits as-sessed by the DAPP-BQ and SNAP are extremeversions of general personality structure (Clark,2007; Livesley, 2005; Widiger & Samuel,2005).
However, it should be pointed out that, notsurprisingly, the DAPP-BQ and SNAP did notoperationalize the four latent constructs in pre-cisely the same way. Previous research has in-dicated that DAPP-BQ and SNAP scales can beunderstood with respect to four common higherorder domains (Clark & Livesley, 2002; Clarket al., 1996; Markon et al., 2005), but the re-spective scales from each inventory do not de-fine these four domains in precisely the samemanner. As a result, content analysis revealedthat NEO PI-R items retained with the respec-tive DAPP-BQ and SNAP scales also varied
15AN IRT INTEGRATION OF PERSONALITY SCALES
somewhat across the two sets of factor analyses.For example, nearly all the items were retainedfrom the DAPP-BQ/NEO PI-R constraint com-parison, making it quite faithful to the originaldomain as assessed by each instrument. How-ever, the content analysis revealed that the cor-responding SNAP/NEO PI-R construct wassomewhat narrower and more heavily ladenwith impulsivity, in part because content as-sessed in that domain in the SNAP (e.g., thepropriety scale) is not well represented in theNEO PI-R. In other cases where the resultinglatent trait did differ, this was also a result of thepreexisting content within the SNAP or theDAPP-BQ. This finding is not surprising be-cause, despite their similarities, these instru-ments are not identical in coverage and content.Thus, one would not expect them to defineshared latent constructs in precisely the samemanner.
However, a strength of the current study isthe examination of two different measures ofmaladaptive personality, rather than relyingsolely on just one of them. Conducting separateanalyses with both measures allows for a con-trast that would not have been possible had onlyone measure been included. By examining theSNAP and DAPP-BQ within in the same manu-script we can consider similarities and differ-ences in their conceptualizations more directly.Nonetheless, future research that administers allof these instruments within the same samplewould be useful both to determine whether ourfindings replicate and also how the domainswould be defined when all three measures areconsidered conjointly.
Scale Comparisons
A considerable body of research has sug-gested that measures of normal and abnormalpersonality traits are closely related to one an-other and involve common, underlying traits(Clark, 2007; Livesley, 2005; Widiger & Sam-uel, 2005). The IRT analyses allow further ex-plication of this relationship, and indicate thatnormal and abnormal personality scales occupydifferent locations on these underlying traits.More specifically, the NEO PI-R generally pro-vided more psychometric information at thelower levels of the latent trait, whereas theDAPP-BQ and SNAP generally provided moreinformation at the higher ends of the underlying
trait dimensions. These findings are consistentwith the dimensional view of PD, which pro-poses that PD symptomatology represents notonly maladaptive variants of normal personalitytraits but also extreme (elevated) variants ofthese same traits (Clark, 2007; Widiger &Samuel, 2005). In this view, for example, theaffective lability and self-injurious behaviorsassociated with borderline PD are more extrememanifestations of the dispositional trait of neu-roticism assessed by normal personality inven-tories (Trull, Widiger, Lynam, & Costa, 2003).
Many prior studies have failed to support acategorical distinction between normal and ab-normal personality functioning (e.g., Roths-child, Cleland, Haslam, & Zimmerman, 2003)and have demonstrated meaningful associationsbetween normal and abnormal personality func-tioning (Samuel & Widiger, 2008; Widiger &Costa, 2002). The current findings provide em-pirical support for the hypothesis that the pri-mary difference between instruments designedto assess normal versus pathological personalitytraits is the location along the shared dimen-sions where they provide the most psychometricinformation. This indicates that personality pa-thology instruments provide better fidelity inassessing individuals with high levels of a latenttrait, such as antagonism, whereas normal per-sonality instruments provide more discrimina-tion among individuals at the agreeable end ofthe dimension. The NEO PI-R can contribute tothe assessment of personality disorder, such as“borderline” traits (Trull et al., 2003) and psy-chopathy (Miller & Lynam, 2003), but our IRTanalyses suggest that, not surprisingly, mea-sures of personality pathology provide moreinformation regarding maladaptive functioningwhereas general personality inventories havegreater value within the normal range. In sum,the results presented in Tables 2 and 3, as wellas illustrated in Figures 1–3, support the viewthat personality disorder is a maladaptive ex-pression of normal personality traits.
Nevertheless, a visual inspection of Figures1–3 suggested that despite significant differ-ences, the curves from these instruments alsoshowed a great deal of overlap. In some cases,such as the DAPP-BQ suicidal ideation scale,the differences were unmistakable and consis-tent with theoretical expectations. However, inothers the differences among curves were lesssubstantial or even nonsignificant (e.g., SNAP
16 SAMUEL ET AL.
exhibitionism). Thus, it appears that despite thedifferences noted above, there may be overlapas well as distinction between some assessmentsof normal and maladaptive personality traits.This implies that some scales from the SNAP,DAPP-BQ, and NEO PI-R are providing similarinformation regarding individuals’ standing onthe four identified higher order dimensions.
It is important to note that this does not mean,for example, the SNAP exhibitionism scalelacks utility. Rather, because the SNAP wasdeveloped expressly to provide lower order,specific trait information (vs. higher order, gen-eral factor information) its scales likely are lesssaturated with the latent trait variance beingmodeled in these analyses (e.g., introversion/extraversion). To the degree that any item orscale is not assessing the exact latent, higherorder construct, its ability to provide psycho-metric information from an IRT perspectivewill be limited. Although only those items thatloaded sufficiently on the latent trait were re-tained, it is necessarily true that some items willload more strongly than others. To this extent,there may be items (and thus scales) that will be“favored” or “disfavored” a priori relative to thehigher order dimension.
Nonetheless, we do contend that the overlapindicates that the NEO PI-R is not best classi-fied as simply a measure of normal personalityfunctioning. Although it was constructed as ameasure of normal personality traits (Costa &McCrae, 1992), it appears that it is better un-derstood as a measure of general personalitystructure that, in some cases, clearly extendsinto the realm of abnormal personality function-ing. Consider, for instance, the NEO PI-R neu-roticism scale. Endorsing items keyed in thedirection of depressiveness, anxiousness, self-consciousness, or vulnerability is, in large part,an endorsement of maladaptive personalityfunctioning (e.g., “I am easily frightened,”“Sometimes I feel completely worthless,” and“At times I have been so ashamed I just wantedto hide”). These items are not appreciably dif-ferent in context or coverage from respectiveitems from the DAPP-BQ (e.g., “I tend to over-react to minor problems”) and the SNAP (e.g.,“I haven’t made much of my life”).
Haigler and Widiger (2001) demonstratedempirically that 98% of the NEO PI-R neurot-icism items assess maladaptive personalityfunctioning when keyed in the direction of high
neuroticism. Equally important for the purposesof this study, they also found that 83% of theNEO PI-R items keyed in the direction of an-tagonism (corresponding to DAPP-BQ conductproblems and interpersonal disesteem, see Fig-ure 1) and 90% of the NEO PI-R items keyed inthe direction of introversion (corresponding toSNAP detachment, see Figure 2) concern ab-normal, maladaptive personality functioning. Itis perhaps not surprising then that the IRTcurves demonstrated considerable overlap ofthe NEO PI-R scales with the respective scalesfrom the DAPP-BQ and SNAP, even though thescales were constructed with quite different pur-poses in mind: The NEO PI-R to assess person-ality traits evident within the general (normal)population, and the DAPP-BQ and SNAP toassess abnormal personality traits (particularlythose underlying the DSM–IV–TR personalitydisorders) within clinical (and more general)populations. In sum, the NEO PI-R may providemore information about the maladaptive rangeof traits assessed by the DAPP-BQ and SNAPthan might be expected from its developmentand description within the literature as a mea-sure of normal personality functioning. Never-theless, the IRT analyses do suggest that boththe DAPP-BQ and the SNAP are more success-ful (again, not surprisingly, given the purposefor their development) than the NEO PI-R incovering the highest, most maladaptive range ofpersonality functioning.
Behavioral Specificity
The most distinct findings obtained for eithera DAPP-BQ or a SNAP scale were those forDAPP-BQ suicidal ideation. Within Figure 3,the curve for the DAPP-BQ suicidal ideationscale is visibly different from both the NEOPI-R neuroticism scale and even the DAPP-BQaffective lability scale. While most of the curvespeak around an average to moderately highlevel of theta, this MIC peaks well to the rightof the figure at a theta of approximately 2.2.
One explanation for this finding is the ex-tremely low endorsement rates for the suicidalideation items. In fact, the base-rates have attimes been so low for this scale that it has beenexcluded from past factor analyses of theDAPP-BQ (e.g., Larstone et al., 2002). It isperhaps self-evident that suicidal ideationwould be an extreme variation of more general
17AN IRT INTEGRATION OF PERSONALITY SCALES
NEO PI-R depressiveness or DAPP-BQ affec-tive lability. It stands to reason that an individ-ual who would endorse an item such as “I havetried to end my life more than once” would bemore extreme on the trait of emotional instabil-ity than would an individual who would onlyendorse a NEO PI-R item such as “Sometimes Ifeel completely worthless.” The suicidal ide-ation items, however, are not only more dys-functional; they also tend to be more behavior-ally specific. For example, two more items fromthe suicidal ideation scale are “I have taken anoverdose when I was very upset” and “I havetried to commit suicide.” These items are quitebehaviorally specific, relative to simply endors-ing the presence of depressed mood or evensuicidal ideation. It is perhaps the behavioralspecificity of these items that also contributes tothe more distinctive locations along the under-lying trait dimension. It is also noteworthythat—despite their specificity—they still con-tain sufficient general trait variance that theywere retained in the joint factor analyses. Itshould also be noted that the parameter esti-mates for these items are not based upon only ahandful of subjects. Even within the communitysample there was a notable prevalence of sui-cidality (e.g., 47 persons indicated that endingtheir lives seems to be the only way out, and 42persons indicated that they have tried to killthemselves).
One of the unique strengths of the IRT ap-proach to scale construction and evaluation is itsability to identify items that provide informa-tion at a specific location along an underlyingdimension (Embretson & Reise, 2000). An idealassessment of personality structure from theperspective of IRT would be to have items thatprovide specific assessments at all levels of thetrait, analogous to items of an achievement orability test providing precise discriminations ateach point along increasing levels of ability.The findings from the current study clearly in-dicate that the DAPP-BQ, NEO PI-R, and theSNAP do a good job of assessing the broadrange of four primary personality dimensions,although their strengths do lie in somewhat dif-ferent ranges. Nevertheless, it would be usefulfor future research to explore the developmentof items that are specific to each severity levelof the latent trait. Minimally, it would be usefulto have items whose information curves arespecific to the normal range or abnormal range
of the trait, respectively, and ideally to haveitems that provide specific discriminationswithin each of these ranges.
It is quite possible that such individual itemsalready exist within the scales of the DAPP-BQ,SNAP, or NEO PI-R. However, it was beyondthe scope of the current study to report the itemresponse curves for the approximately 1,000items from these three instruments. Moreover,there is reason to doubt that a large number ofsuch items are present within the existing mea-sures, because previously published IRT analy-ses of these instruments have not identified anappreciable number of items unique or specificto particular points along the respective traitdimensions (Reise & Henson, 2000; Simms &Clark, 2005). One common thread among theseinstruments is the reliance on classical test the-ory (CTT) methodologies for their constructionand validation. While this strategy has assuredlyproduced reliable and valid measures of person-ality dimensions, it may have also led to theexclusion of the more extreme, and perhapsbehaviorally specific, items that may be useful,if not necessary, for specific discriminationsalong a respective trait dimension. Thus, futureresearch should address this hypothesis by us-ing IRT to evaluate existing items from variouspersonality pathology measures as well as de-veloping new, experimental items to determinewhether better discrimination can be obtainedalong specific trait dimensions. While the de-velopment of more behaviorally specific itemsmay increase the measurement range of a scale,it is also possible that these items would evi-dence weaker loadings with a broad, generalfactor. For this reason, the potential inclusion ofsuch items must be carefully weighed againstthe unidimensionality requirement (assumingthe goal of measuring the general domain vs.more specific traits). Nonetheless, the potentialfor behaviorally specific, low base-rate items toincrease the measurement range of personalitymeasures appears to be fertile ground for futurestudy.
Limitations
The two samples were each relatively largeby traditional standards (i.e., over 600 subjects),but these sample sizes are only adequate for IRTpurposes. Additionally, they were communityand undergraduate samples that were largely
18 SAMUEL ET AL.
Caucasian. Therefore, future research that rep-licates these findings within larger, more ethni-cally diverse, samples that vary more widely interms of severity may be useful. Furthermore,the current study employed the self-reportmethodology exclusively. Although self-reportis the most commonly used method within re-search and clinical practice (Widiger & Boyd,2009), the accepted gold standard for assess-ment is the structured interview (Rogers, 2001).Although there is not an a priori reason tobelieve that other assessment methods wouldevidence different results (as the SNAP andDAPP-BQ were designed to assess the sameinformation, albeit via self-report), it will none-theless be important for future research to rep-licate these findings using structured interviewsto assess the DSM–IV–TR (or DSM–V) person-ality disorders and the FFM.
In addition, the method we used for testingdifferences among the betas is limited by itsdependence on the number of items within agiven domain. In these ANOVA analyses, wetreated the items as cases, which makes thedetection of significant differences among thebeta values much more difficult for domainsthat retained fewer scales or items. This couldbe an explanation for why the DAPP-NEO emo-tional instability comparison (with 140 items)had several significant findings whereas theSNAP-NEO introversion comparison (with 26items) had none. Perhaps had we included theSNAP “temperament” scales in our analysis,the latter result would have been more similarto the former. Finally, a potential limitation—orat least complication—of this study is the con-trasting response formats employed by each in-strument. The dichotomous items of the SNAPare consistent with the derivation of IRT tech-nologies within intellectual and academic test-ing formats that rely almost exclusively on “cor-rect” (vs. incorrect) response options. Thisproperty gives them the advantage of beingreadily interpretable, particularly in the case ofthe beta parameters. Either a given item (orscale) is more extreme or it is not. However, onebenefit inherent to polytomous items, such asthose present on the DAPP-BQ and the NEOPI-R, is the further differentiation of individualswith respect to multiple response categories.This same principle may also allow polyto-mous items to provide greater psychometricinformation at the extreme levels of the latent
traits. The potential influence of response for-mat on IRT analyses is an important area forfuture research.
Conclusions
This study provided a demonstration, usingIRT methods, that scales from personality pa-thology and general personality instruments canbe combined onto a common metric and that thepersonality pathology items provide more infor-mation at extreme levels of these traits than dogeneral personality items. Self-report scales as-sessing personality pathology and general per-sonality traits were shown to lie along commonunderlying continua, with the two sets of scalesgenerally differing significantly in terms of theirrespective locations along the latent trait.Whereas the normal personality scales tended toprovide greater information at the lower levelsof theta, the personality pathology scales con-sistently provided more information at the up-per levels. This evidence supports both a dimen-sional conceptualization of personality disorderand the utility of IRT in future instrument de-velopment and evaluation.
References
American Psychiatric Association. (2000). Diagnos-tic and statistical manual of mental disorders (4thed., revised). Washington, DC: Author.
Bolt, D. M., Hare, R. D., Vitale, J. E., & Newman,J. P. (2004). A multigroup item response theoryanalysis of the Psychopathy Checklist-Revised.Psychological Assessment, 16, 155–168.
Clark, L. A. (1993). Manual for the Schedule forNonadaptive and Adaptive Personality. Minneap-olis, MN: University of Minnesota Press.
Clark, L. A. (2007). Assessment and diagnosis ofpersonality disorder. Perennial issues and anemerging reconceptualization. Annual Review ofPsychology, 58, 227–257.
Clark, L. A., & Livesley, W. J. (2002). Two ap-proaches to identifying the dimensions of person-ality disorder: Convergence on the five-factormodel. In P. T. Costa, Jr. & T. A. (Eds.) Person-ality disorders and the five-factor model of person-ality (2nd ed., pp. 161–176). Washington, DC:American Psychological Association.
Clark, L. A., Livesley, W. J., Schroeder, M. L., &Irish, S. L. (1996). Convergence of two systemsfor assessing specific traits of personality disorder.Psychological Assessment, 8, 294–303.
19AN IRT INTEGRATION OF PERSONALITY SCALES
Clark, L. A., Simms, L. J., Wu, K. D., & Casillas, A.(in press). Manual for the Schedule for Nonadap-tive and Adaptive Personality (SNAP-2). Minne-apolis, MN: University of Minnesota Press.
Cooke, D. J., & Michie, C. (1997). An item re-sponse theory analysis of the Hare PsychopathyChecklist-Revised. Psychological Assess-ment, 9, 3–14.
Costa, P. T., & McCrae, R. R. (1992). RevisedNEO Personality Inventory (NEO PI-R) andNEO Five-Factor Inventory (NEO-FFI) profes-sional manual. Odessa, FL: Psychological As-sessment Resources.
Costa Jr., P. T., Herbst, J. H., McCrae, R. R., &Siegler, I. C. (2000). Personality at midlife: Sta-bility, intrinsic maturation, and response to lifeevents. Assessment, 7, 365–378.
Costa Jr., P. T., & McCrae, R. R. (1985). The NEOpersonality inventory manual. Odessa, FL: Psy-chological Assessment Resources.
Embretson, S. E., & Reise, S. P. (2000). Item responsetheory for psychologists. Mahwah, NJ: Erlbaum.
Haigler, E. D., & Widiger, T. A. (2001). Experimen-tal manipulation of NEO PI-R items. Journal ofPersonality Assessment, 77, 339–358.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fitindices in covariance structure analysis: Conven-tional criteria versus new alternatives. StructuralEquation Modeling, 6, 1–55.
Jane, J. S., Oltmanns, T. F., South, S. C., & Turkhei-mer, E. (2007). Gender bias in diagnostic criteriafor the personality disorders: An item responsetheory analysis. Journal of Abnormal Psychology,116, 166–175.
Jang, K. L., Livesley, W. J., & Vernon, P. A. (2002).The etiology of personality function: The Univer-sity of British Columbia Twin Project. Twin Re-search, 5, 342–346.
Larstone, R. M., Jang, K. L., Livesley, W. J., Vernon,P. A., & Wolf, H. (2002). The relationship be-tween Eysenck’s P-E-N model of personality, thefive-factor model, and traits delineating personal-ity dysfunction. Personality and Individual Differ-ences, 33, 25–37.
Lilienfeld, S. O., & Andrews, B. P. (1996). Devel-opment and preliminary validation of a self-reportmeasure of psychopathic personality traits in non-criminal populations. Journal of Personality As-sessment, 66, 488–524.
Livesley, W. J. (2001). Conceptual and taxonomicissues. In W. J. Livesley (Ed.), Handbook of per-sonality disorders: Theory, research, and treat-ment (pp. 3–38). New York: Guilford Press.
Livesley, W. J. (2003). Diagnostic dilemmas in clas-sifying personality disorder. In K. A. Phillips,M. B. First, & H. A. Pincus (Eds.), AdvancingDSM: Dilemmas in psychiatric diagnosis (pp.
153–190). Washington, DC: American PsychiatricAssociation.
Livesley, W. J. (2005). Behavioral and moleculargenetic contributions to a dimensional classifica-tion of personality disorder. Journal of PersonalityDisorders, 19, 131–155.
Livesley, W. J., & Jackson, D. (in press). Manual forthe Dimensional Assessment of Personality Pa-thology–Basic Questionnaire. Port Huron, MI:Sigma Press.
Markon, K. E., Krueger, R. F., & Watson, D. (2005).Delineating the structure of normal and abnormalpersonality: An integrative hierarchical approach.Journal of Personality and Social Psychology, 88,139–157.
McCrae, R. R., & Costa, P. T., Jr. (1999). A Five-Factor theory of personality. In L. A. Pervin &O. P. John (Eds.), Handbook of personality: The-ory and research (2nd ed., pp. 139–153). NewYork: Guilford Press.
Miller, J. D., & Lynam, D. R. (2003). Psychopathyand the five-factor model of personality: A repli-cation and extension. Journal of Personality As-sessment, 81, 168–178.
Mullins-Sweatt, S. N., Jamerson, J. E., Samuel, D. B.,Olson, D. R., & Widiger, T. A. (2006). Psychometricproperties of an abbreviated instrument of the five-factor model. Assessment, 13, 119–137.
O’Connor, B. P. (2005). A search for consensus onthe dimensional structure of personality disorders.Journal of Clinical Psychology, 61, 323–645.
Reise, S. P., & Henson, J. H. (2000). Computeriza-tion and adaptive administration of the NEO PI-R.Assessment, 7, 347–364.
Reise, S. P., Smith, L., & Furr, R. M. (2001). Invari-ance on the NEO PI-R neuroticism scale. Multi-variate Behavioral Research, 36, 83–110.
Rogers, R. (2001). Diagnostic and structured inter-viewing. A handbook for psychologists. New York:Guilford Press.
Rothschild, L., Cleland, C., Haslam, N., & Zimmer-man, M. (2003). A taxometric study of borderlinepersonality disorder. Journal of Abnormal Psy-chology, 112, 657–666.
Samejima, F. (1969). Estimation of latent ability us-ing a response pattern of graded scores. Psy-chometrika, 34(Suppl. 17), 1–100.
Samuel, D. B., & Widiger, T. A. (2008). A meta-analytic review of the relationships between thefive-factor model and DSM–IV–TR personality dis-orders: A facet level analysis. Clinical PsychologyReview, 28, 1326–1342.
Schroeder, M. L., Wormsworth, J. A., & Livesley, W. J.(1992). Dimensions of personality disorder and theirrelationships to the Big Five dimensions of person-ality. Psychological Assessment, 4, 47–53.
Simms, L. J., & Clark, L. A. (2005). Validation of acomputerized adaptive version of the Schedule for
20 SAMUEL ET AL.
Nonadaptive and Adaptive Personality (SNAP).Psychological Assessment, 17, 28–43.
Smith, L. L., & Reise, S. P. (1998). Gender differ-ences on negative affectivity: An IRT study ofdifferential item functioning on the Multidimen-sional Personality Questionnaire stress reactionscale. Journal of Personality and Social Psychol-ogy, 75, 1350–1362.
Stout, W. F. (1987). A nonparametric approach forassessing latent trait unidimensionality. Psy-chometrika, 52, 589–617.
Stout, W. F. (1990). A new item response theorymodeling approach with applications to unidimen-sionality assessment and ability estimation. Psy-chometrika, 55, 293–325.
Tellegen, A. (1982). Multidimensional PersonalityQuestionnaire. Unpublished manuscript, Univer-sity of Minnesota, Minneapolis.
Thissen, D., Chen, W. H., & Bock, D. (2003). Mul-tilog 7.03. Lincolnwood, IL: Scientific SoftwareInternational.
Trull, T. J., Widiger, T. A., Lynam, D. R., & Costa,P. T., Jr. (2003). Borderline personality disorderfrom the perspective of general personality func-tioning. Journal of Abnormal Psychology, 112,193–202.
Waller, N. G. (2002). WinMFact 2.0. Minneapolis,MN: Author.
Walton, K. E., Roberts, B. W., Krueger, R. F., Bloni-gen, D. M., & Hicks, B. M. (2008). Capturingabnormal personality with normal personality in-ventories: An item response theory approach.Journal of Personality, 76, 1623–1648.
Widiger, T.A., & Boyd, S. (2009). Personality disor-der assessment instruments. In J. N. Butcher (Ed.),Oxford handbook of personality assessment (3rded., pp. 336–363). New York: Oxford UniversityPress.
Widiger, T. A., & Costa, P. T., Jr. (2002). Five-factormodel personality disorder research. In P. T.Costa, Jr. & Widiger, T. A. (Eds.), Personalitydisorders and the five-factor model of personality(2nd ed., pp. 59–87). Washington, DC: AmericanPsychological Association.
Widiger, T. A., & Samuel, D. B. (2005). Diagnos-tic categories or dimensions: A question forDSM-V. Journal of Abnormal Psychology, 114,494 –504.
Zickar, M. J., Russell, S. S., Smith, C. S., Bohle, P.,& Tilley, A. J. (2002). Evaluating two morning-ness scales with item response theory. Personalityand Individual Differences, 33, 11–24.
21AN IRT INTEGRATION OF PERSONALITY SCALES
Correction to Samuel et al (2010)
In the article, “An Item Response Theory Integration of Normal and Abnormal PersonalityScales” by Douglas Samuel, Leonard Simms, Lee Anna Clark, John Livesley, and TomWidiger (Personality Disorders: Theory, Research, Treatment, 2010, Vol. 1, No. 1, pp.5-21), in the acknowledgments, Douglas Samuel was incorrectly listed as the author of theDAPP-BQ instrument. John Livesley is the correct author of the DAPP-BQ instrument.
