Personality Profiling of Fictional Characters using Sense-Level Linksbetween Lexical Resources

Lucie Flekova† and Iryna Gurevych†‡

† Ubiquitous Knowledge Processing Lab (UKP-TUDA)Department of Computer Science, Technische Universitat Darmstadt

‡ Ubiquitous Knowledge Processing Lab (UKP-DIPF)German Institute for Educational Research

www.ukp.tu-darmstadt.de

“Always be yourself, unless you canbe Batman. Then always be Batman.”

– Bill Murray

Abstract

This study focuses on personality predic-tion of protagonists in novels based onthe Five-Factor Model of personality. Wepresent and publish a novel collaborativelybuilt dataset of fictional character person-ality and design our task as a text classifi-cation problem. We incorporate a rangeof semantic features, including WordNetand VerbNet sense-level information andword vector representations. We evalu-ate three machine learning models basedon the speech, actions and predicatives ofthe main characters, and show that espe-cially the lexical-semantic features signifi-cantly outperform the baselines. The mostpredictive features correspond to reportedfindings in personality psychology.

1 Introduction

Recent progress in NLP has given rise to the fieldof personality profiling - automated classificationof personality traits based on written, verbal andmultimodal behavior of an individual. This re-search builds upon findings from classical person-ality psychology and has applications in a widerange of areas from medicine (suicide prevention)across security (forensics, paedophile detection,cyberbullying) to marketing and sales (recommen-dation systems, target group profiles). The goldstandard labels for an objective evaluation of per-sonality are mostly obtained by means of personal-ity tests of the Five Factor Model (FFM) (McCraeand Costa, 1987; Goldberg, 1990), which is well-known and widely accepted in psychology andother research fields. The FFM defines personality

along five bipolar scales: Extraversion (sociablevs. reserved), Emotional stability (secure vs. neu-rotic), Agreeableness (friendly vs. unsympathic),Conscientiousness (organized vs. careless) andOpenness to experience (insightful vs. unimagi-native). Psychologists have shown that these fivepersonality traits are stable across individual lifes-pan, demographical and cultural differences (Johnand Srivastava, 1999) and affect many life aspects.(Terracciano et al., 2008; Rentfrow et al., 2011).

It has been shown that the personality traits ofreaders impact their literature preferences (Tirreand Dixit, 1995; Mar et al., 2009). Psychologyresearchers also found that perceived similarityis predictive of interpersonal attraction (Montoyaet al., 2008; Byrne, 1961; Chartrand and Bargh,1999). More explicitly, recent research (Kaufmanand Libby, 2012) shows that readers of a narrativedevelop more favorable attitudes and less stereo-type application towards a character, if his differ-ence (e.g. racial) is revealed only later in the story.We therefore hypothesize that readers might havea preference for reading novels depicting fictionalcharacters that are similar to themselves. Finding adirect link between reader’s and protagonist’s per-sonality traits would advance the development ofcontent-based recommendation systems. As a firststep to explore this hypothesis further, it needs tobe determined if we are able to construct a per-sonality profile of a fictional character in a similarway as it is done for humans, and which aspectsof personality profiling can be exploited to autom-atize such procedure.

In this paper, we open this research topic bypresenting a novel collaboratively built datasetof fictional character personality in Section 3,which we make available on our website.1 Fram-ing the personality prediction as a text classifica-tion task, we incorporate features of both lexical-

1https://www.ukp.tu-darmstadt.de/data/personality-profiling/

resource-based and vector space semantics, in-cluding WordNet and VerbNet sense-level infor-mation and vectorial word representations. Weevaluate three machine learning models based onthe speech (Section 4), actions (Section 5) andpredicatives (Section 6) of the protagonists, andshow that especially on the direct speech andaction data the lexical-semantic features signifi-cantly outperform the baselines. Qualitative anal-ysis reveals that the most predictive features corre-spond to reported findings in psychology and NLP.

2 Related work

Research in the the area of content-based recom-mendation systems have shown that incorporat-ing semantic information is valuable for the userand leads to measurable improvements (Passant,2010; Di Noia et al., 2012; Heitmann and Hayes,2010). De Clercq et al. (2014) incorporated se-mantic frames from FrameNet into the recommen-dation system for books. They represent the plotof each book with a sequence of ca. 200 seman-tic frames and has shown that the frame informa-tion (such as Killing - Revenge - Death) outper-forms the bag-of-words approach.Recent NLP ex-periments begin to reveal the importance of entity-centric models in a variety of tasks. Chambers(2013) show improvement in event schema induc-tion by learning entity-centric rules (e.g., a victimis likely to be a person). Bamman et al. (2014) andSmith et al. (2013) present latent variable modelsfor unsupervised learning of latent character typesin movie plot summaries and in English novels,taking authorial style into account. However, eventhe state-of-the-art NLP work rather describes per-sonas of fictional characters by their role in thestory - e.g., action hero, valley girl, best friend,villain etc. - or by their relations to other char-acters, such as mother or daughter (Elson et al.,2010; Kokkinakis and Malm, 2011), rather thanby their inner preferences and motivations. It isimportant to note here that determining a person-ality of a character is a very different task fromdetermining its role in the story. Psychologicalunderstanding of personality, in contrast to role at-tribution requires a certain detached objectivity -even outright villains may have traits considereddesirable in real life. For example, the devil hasin many tales a very high aspiration level, appear-ing highly conscientious and agreeable. We hy-pothesize that these deeper personality aspects are

those which drive reader’s affiliation to the char-acter, thus deserve to be examined closer.

Also literary scholars formulate ad hoc person-ality descriptions for their experiments, for exam-ple to test hypotheses from evolutionary psychol-ogy (Johnson et al., 2011) or examine fictionalportrayals of physicists (Dotson, 2009). These de-scriptions are usually adjusted to the experimentfocus (e.g. emotions, relationships, ambitions).As McCrae et al. (2012) point out, a standard setof personality traits, that encompass the full rangeof characteristics found in all characters in litera-ture (p.77), is needed for a better comparison.

Hence we base our present study primarily onthe previous NLP research on personality predic-tion of human individuals. Correlations betweenlexical and stylistic aspects of text and the fiveFFM personality traits of the author have beenfound in numerous experiments, with extraver-sion receiving the most attention (Pennebaker andKing, 1999; Dewaele and Furnham, 1999; Gill andOberlander, 2002; Mehl et al., 2006; Aran andGatica-Perez, 2013; Lepri et al., 2010). The LIWClexicon (Pennebaker et al., 2001) established itsposition as a powerful mean of such analysis.

The first machine learning experiments in thisarea were conducted by Argamon et al. (2005),Oberlander and Nowson (2006) and Mairesseet al. (2007). Researchers predicted the fivepersonality traits of the authors of stream-of-conscientiousness essays, blog posts and recordedconversation snippets. Given balanced data sets,Mairesse et al. (2007) report binary classificationaccuracy of 50-56% on extraversion in text and47-57% in speech, using word ngrams, LIWC,MRC psycholinguistic database (Coltheart, 1981)and prosodic features. Additional improvementis reported when the extraversion was labeled byexternal judges rather than by self-testing. Ex-tended studies on larger datasets achieve accu-racies around 55% (Nowson, 2007; Estival etal., 2007). More recent work in this area fo-cuses on the personality prediction in social net-works (Kosinski et al., 2013; Kosinski et al.,2014) and multimodal personality prediction (Bieland Gatica-Perez, 2013; Aran and Gatica-Perez,2013). These trends emphasized the correlation ofnetwork features and audiovisual features with ex-traversion, giving rise to the Workshop on Compu-tational Personality Recognition (for an overviewsee (Celli et al., 2013; Celli et al., 2014).

3 Data set construction

Traditionally, the gold standard for this supervisedclassification task is obtained by the means of per-sonality questionnaires, used for the Five-FactorModel, taken by each of the individuals assessed.This poses a challenge for fictional characters.However, strong correlations have been found be-tween the self-reported and perceived personalitytraits (Mehl et al., 2006). Our gold standard bene-fits from the fact that readers enjoy discussing thepersonality of their favourite book character on-line. A popular layman instrument for personal-ity classification is the Myers-Brigggs Type Indi-cator (Myers et al., 1985), shortly MBTI, whichsorts personal preferences into four opposite pairs,or dichotomies, such as Thinking vs. Feeling orJudging vs. Perceiving. While the MBTI validityhas been questioned by the research community(Pittenger, 2005), the Extraversion scale is show-ing rather strong validity and correlation to similartrait in the Five-Factor Model (McCrae and Costa,1989; MacDonald et al., 1994). Our study hencefocuses on the Extraversion scale.

Our data was collected from the collabora-tively constructed Personality Databank2 wherethe readers can vote if a book character is, amongother aspects, introverted or extraverted. While thereaders used codes based on the MBTI typology,they did not apply the MBTI assessment strate-gies. There was no explicit annotation guidelineand the interpretation was left to readers’ intuitionand knowledge.3 This approach of gold standardcollection has several obvious drawbacks. First,the question is posed as dichotomic, while in real-ity the extraversion is a normally distributed traitin human population (Goldberg, 1990). Second,users can view the vote of previous participants,which may influence their decision. While we ad-dress both of these issues in our ongoing data col-lection project based on the Five-Factor Model, weconsider them acceptable for this study due to theexploratory character of our pilot research.

We have collected extraversion ratings for 298book characters, of which 129 (43%) are rather ex-traverted and 166 (56%) rather introverted. Rated

2http://www.mbti-databank.com/3MBTI defines extraversion as “getting energy from ac-

tive involvement in events, having a lot of different activities,enjoying being around people.” In the NEO Five-Factor In-ventory (Costa and McCrae, 1992), underlying facets of ex-traversion are warmth, gregariousness, assertiveness, activity,excitement seeking and positive emotion.

characters come from a wide range of novels thatthe online users are familiar with, often coveringclassical literature which is part of the high schoolsyllabus, as well as the most popular modern fic-tion, such as the Harry Potter series, Twilight, StarWars or A Game of Thrones. A sample of the mostrated introverts and extraverts is given in table 1.The rating distribution in our data is strongly U-shaped. The percentage agreement of voters in ourdata is 84.9%, calculated as:

P =1

N

N∑i=1

k∑j=1

nij(nij − 1)

n(n− 1)

where k = 2 (introvert, extravert), N is the num-ber of book characters and n the number of votesper character. Voters on the website were anony-mous and cannot be uniquely identified for addi-tional corrections. There is no correlation betweenthe extraversion and the gender of the character.

Character Book E ITyrion Lannister Game of Thrones 52 1Cersei Lannister Game of Thrones 48 7Joffrey Baratheon Game of Thrones 41 1Ron Weasley Harry Potter series 37 4Jamie Lannister Game of Thrones 38 9Draco Malfoy Harry Potter series 33 4Anakin Skywalker Star Wars series 30 6Robert Baratheon Game of Thrones 28 2Gimli Lord of the Rings 19 2Jar Jar Binks Star Wars series 12 2Harry Potter Harry Potter series 1 71Severus Snape Harry Potter series 1 65Gandalf Lord of the Rings 1 59Yoda Star Wars series 0 58Jon Snow Game of Thrones 1 47Albus Dumbledore Harry Potter series 4 46Ned Stark Game of Thrones 0 41Aragorn Lord of the Rings 1 41Frodo Lord of the Rings 1 40Bran Stark Game of Thrones 1 36

Table 1: Extraverts (E) and introverts (I) with the highestnumber of user votes.

Our set of English e-books covered 220 of thecharacters from our gold standard. We have builtthree systems to assess the following:

1. Direct speech: Does the style and content ofcharacter’s utterances predict his extraversionin a similar way as it was shown for livingindividuals?

2. Actions: Is the behavior, of which a characteris an agent, predictive for extraversion?

3. Predicatives and adverbs: Are the explicit(John was an exhibitionist) or implicit (Johnshouted abruptly) descriptions of the charac-ter in the book predictive for extraversion?

In the next three sections we present the experi-mental settings and results for each of the systems.

4 Direct speech of fictional characters

The system for the direct speech resembles themost to the previous systems developed for authorpersonality profiling, e.g. on stream of conscious-ness essays (Mairesse et al., 2007) or social mediaposts (Celli et al., 2013) and therefore providesthe best opportunity for comparison between hu-man individuals and fictional characters. On topof the comparison to previous research, we exploitthe sense links between WordNet and VerbNet toextract additional features - an approach which isnovel for this type of task.

4.1 Extraction and assignment of speech

We process the book text using freely availablecomponents of the DKPro framework (Gurevychet al., 2007). The most challenging task in build-ing the direct speech data set is assigning to the di-rect speech utterance the correct speaker. We ben-efit from the epub format of the e-books whichdefines a paragraph structure in such a way, thatonly the indirect speech chunk immediately sur-rounding the direct speech can be considered:

<p> John turned to Harry."Let’s go," he said.</p>

Given the large amount of text available in thebooks we focus on precision and discard all utter-ances with no explicit speaker (i.e., 30-70% of theutterances, dependent on the book), as the perfor-mance of current systems on such utterance typesis still fairly low (O’Keefe et al., 2012; He et al.,2013; Iosif and Mishra, 2014). Similarly, conven-tional coreference resolution systems did not per-form well on this type of data and were thereforenot used in the final setup. We adapt the StanfordNamed Entity Recognizer(Finkel et al., 2005) toconsider titles (Mr., Mrs., Sir...) as a part of thename and to treat the first person I as a named en-tity. However, identifying only the named entityPERSON in this way is not sufficient. On our eval-uation sample consisting of A Game of Thronesand Pride and Prejudice books (the former anno-tated by us, the latter by He et al. (2013)), 20%of utterances with explicit named speaker werenot recognized. Of those correctly identified as aPerson in the adjacent indirect speech, 17% werenot the speakers. Therefore we implemented a

custom heuristics (Algorithm 1), which addition-ally benefits from the WordNet semantic classesof verbs, enriching the speaker detection by grab-bing the nouns . With this method we retrieve89% of known speakers, of which 92% is assignedcorrectly. Retrieved names are grouped based onstring overlap (e.g. Ser Jaime and Jaime Lannis-ter), excluding the match on last name, and cor-rected for non-obvious groupings (such as Mar-garet and Peggy).Algorithm 1 Assign speaker1: nsubj← subjects in adjacent indirect speech2: if count(nsubj(i) = PERSON) = 1 then speaker ←

nsubj3: else if count(nsubj(i) = PERSON) ≥ 1 then

speaker ← the nearest one to directSpeech4: else if directSpeech preceded by

VERB.COMMUNICATION then speaker ← thepreceding noun(s)

5: else if directSpeech followed byVERB.COMMUNICATION then speaker ← thefollowing noun(s)

6: else if directSpeech followed by gap &VERB.COMMUNICATION then speaker ← the noun(s)in gap

7: else if directSpeech preceded by gap &VERB.COMMUNICATION then speaker ← the noun(s)in gapreturn speaker

Our experimental data consists of usable directspeech sets of 175 characters - 80 extraverts (E)and 95 introverts (I) - containing 289 274 words in21 857 utterances (on average 111 utterances forE and 136 for I, as I are often central in books).4

4.2 Classification approach for direct speech

All speech utterances of one book character arerepresented as one instance in our system. Weuse the leave-one-out classification setup due tothe relatively small dataset size, using the supportvector machines (SVM-SMO) classifier, whichperforms well on comparable tasks (Celli et al.,2013). The classification is performed through theDKPro TC Framework (Daxenberger et al., 2014).

Lexical features As a bottom-up approach weuse the 1000 most frequent word uni-, bi- and tri-grams, 1000 dependency word pairs, 1000 charac-ter trigrams and 500 most frequent verbs, adverbs,adjectives and interjections as binary features.

Semantic features Since the top-down ap-proach, i.e. not focusing on individual words, has

4The data set size is comparable to ongoing personalityprofiling challenges - see http://pan.webis.de

been found more suitable for the personality pro-filing task on smaller data sets (Celli et al., 2013),we aim on capturing additional phenomena on ahigher level of abstraction. The main part of ourfeatures is extracted on sense level. We use themost frequent sense of WordNet (Miller, 1995)to annotate all verbs in the direct speech (a sim-ple but well performing approach for books). Wethen label the disambiguated verbs with their se-mantic field given in WordNet (WordNet defines14 semantic classes of verbs which group verbsby their semantic field) and we measure frequencyand occurence of each of these classes (e.g. cogni-tion, communication, motion, perception)5. Ad-ditionally, we use the lexical-semantic resourceUBY (Gurevych et al., 2012) to access the Word-Net and VerbNet information, and to exploit theVerbNet sense-level links which connects Word-Net senses with the corresponding 273 main Verb-Net classes (Kipper-Schuler, 2005). These aremore fine-grained (e.g. pay, conspire, neglect, dis-cover) than the WordNet semantic fields. WordNetcovered 90% and VerbNet 86% of all the verb oc-curences.

On word level, we extract 81 additional fea-tures using the Linguistic Inquiry and Word Count(LIWC) tools (Pennebaker et al., 2001), whichconsists of lexicons related to psychological pro-cesses (cognitive, perceptual, social, biological,affective) and personal concerns (achievement, re-ligion, death...) and other categories such as fillers,disfluencies or swear words6. Additionally, sinceemotion detection has been found predictive inprevious personality work (Mohammad and Kir-itchenko, 2013), we measure overall positive andnegative sentiment expressed per character, usingSentiWordNet (Esuli and Sebastiani, 2006) andNRC Emotion Lexicon (Mohammad and Turney,2010) for the word lookup, inverting sentimentscores for negated dependency sub-tree given bythe Stanford Parser.

Stylistic features Features of this group cap-ture the syntactic and stylistic properties of the ut-terances of a character, disregarding the content.Starting from the surfacial properties, we measurethe sentence, utterance and word length, includingthe proportion of words shorter than 4 or longerthan 6 letters, frequency of each punctuation mark,

5https://wordnet.princeton.edu/man/lexnames.5WN.html

6For complete overview refer to www.liwc.net

and endings of each adjective as per Corney et al.(2002). On the syntax level we measure the fre-quency of each part of speech as well as the 500most frequent part-of-speech bi-, tri- and quadri-grams, and the frequency of each dependency ob-tained from the Stanford Parser. We additionallycapture the frequency of superlatives, compara-tives and modal verbs, the proportion of verbs inpresent, past and future tense, and the formality ofthe language as per the part-of-speech-based for-mality coefficient (Heylighen and Dewaele, 2002),and measure the average depth of the parse trees.

Word embeddings as features Since vectorspace semantics has been beneficial for predictingauthor’s personality in previous work (Neumanand Cohen, 2014), we use a pre-trained word vec-tor model created by the GloVe algorithm (Pen-nington et al., 2014) on English Wikipedia. GloVeemploys a global log-bilinear regression modelthat combines the advantages of the global matrixfactorization and local context window methods.We assign the resulting 300-dimensional vectorsto the words in character’s direct speech, exclud-ing stopwords, and calculate an average vector foreach character. We calculate for each test charac-ter the cosine similarity to the mean vector of ex-travert, resp. introvert, in the training data, and toeach character in the training set individually us-ing the DL4J NLP package7. We consider both thefinal scalar outcome and the difference of each ofthe individual vector dimensions as features.

4.3 Classification results on direct speechTable 2 shows the precision, recall, F1-score andaccuracy for extraversion and introversion as aweighted average of the two class values.

ID Feature set P R F A1 - (baseline) .295 .543 .382 .5432 Ngrams .519 .514 .515 .5143 LIWC .555 .560 .552 .5604 WordNet .527 .548 .528 .5485 VerbNet .649 .617 .572 .6176 Style .560 .581 .558 .5817 Sentiment .524 .543 .419 .5438 Vectors .295 .543 .382 .5439 All .550 632. .588 .632Percentage human agreement: .849

Table 2: Weighted precision (P), recall (R), F-score (F) andaccuracy (A) for a direct speech system, in each line us-ing only the given group of features. WordNet stands forWordNet semantic labels, VerbNet setup uses the WordNet-VerbNet links to retrieve VerbNet labels. Highlighted F-scores differ from the majority baseline significantly(p <0.05), using an approximate randomization test.

7http://deeplearning4j.org/

IntrovertFeat.group Features Meritunigrams reason, trouble, strange, indeed .24-.19bigrams this time, tell me, I hope .19-.16LIWC Negate, Discrepancy,

Insight, Exclusion .18-.13WordNet stative, creation, cognition .15-.09VerbNet lodge, hunt, defend .23-.19Style modal verbs, neg, sbar, articles .19-.14ExtravertFeat.group Features Meritngrams we, hurry, fat, dirty .24-.19LIWC We, Inclusion, Pronoun, Body .18-.09WordNet motion, contact, communication,

body, perception, change .14-.07VerbNet get, talk, substance emission .18-.15Style pronoun We, whadjp,

type-token ratio., interjections .20-.14

Table 3: The most predictive features for each group forspeaker’s extraversion and introversion. Correlation merit,as per the correlation feature selection in WEKA, evaluatesPearson’s correlation between the feature and the class

Similarly to previous research (Mairesse et al.,2007; Celli et al., 2013), the bottom-up wordbased approach is outperformed by top-down se-mantic approaches which employ a more abstractfeature representation. As in previous work,LIWC features exhibit good performance. How-ever, the highest performance is achieved employ-ing the VerbNet verb classes with WordNet word-sense disambiguation. Also stylistic features con-tribute substantially to the classification despitethe mixture of genres in our book corpus - es-pecially frequencies of modal verbs and part-of-speech ratios were particularly informative. Themost predictive features from each group are listedin Table 3 together with their correlation merit(Hall, 1999), and compared with previous work inTable 4.

Feature I/E Ref Feature I/E RefPredictive also in our data: No effect in our data:Pronoun ’we’ -/+ [3] Neg. emot. +/- [1]Tentative, unsure +/- [1] Pos. emot. -/+ [1]Exclusive +/- [1] Self-ref. -/+ [1]Inclusive -/+ [1] Formality +/- [2]Insight +/- [1] Elaborated +/- [3]Nouns, articles +/- [2] Long sent. +/- [3]Lexical richness +/- [2] Social -/+ [3]Negations +/- [2]Body functions -/+ [2]Interjections -/+ [3]

Source ID Author[1] Pennebaker and King (1999)[2] Dewaele and Furnham (1999)[3] Mairesse et al. (2007)

Table 4: Comparison of our results to previously reportedpredictive features for speaker’s extraversion (E), resp. intro-version (I). We list publications where these features were, toour knowledge, reported as novel.

In accordance with the experiments of Pen-nebaker and King (1999), we observe more fre-quent exclusions (e.g. without, but), hedging andnegation expressed by introverts, and inclusion(e.g. with, and) by extraverts. Extraverts talk morein first person plural, use more back-channels andinterjections, and talk more about aspects relatedto their body. Introverts show more rationalizationthrough insight words and more factual speech us-ing less pronouns.

Additionally, the semantic features in Table 3confirm the broad psychological characteristics ofboth types in general, i.e., for introverts the ra-tionalization, uncertainty and preference for indi-vidual or rather static activities, and for extravertstheir spontaneity, talkativeness and preference formotion. Furthermore, we observe certain direct-ness in extraverts’ speech - note the predictivewords fat and dirty and frequent descriptions ofbody functions.

Discussion Exploiting the links between lexical-semantic resources (performing WordNet word-sense disambiguation and using VerbNet verbclasses linked to the disambiguated senses) wasparticularly beneficial for this task. WordNetsemantic fields for verbs alone are too coarse-grained to capture the nuances in direct speech,and experiments with fine-grained VerbNet classeswithout WSD resulted in noisy labels. We did notconfirm the previously reported findings on emo-tional polarity - we observe that the genre of thebooks (e.g. love romance vs horror story) haveblurred the subtle differences between individualcharacters, unfortunately the dataset size did notallow for genre distinctions. Furthermore, a per-ceived extravert in our case can be a pure villain(Draco Malfoy, Joffrey Baratheon...) as well as afriendly companion (Gimli, Ron Weasley...), whilethe evil extravert types are possibly rarer in the ex-periments on human writing, or are more likely tofit under the MBTI definition of extraversion thanFFM facets. Another potential cause, based on theerror analysis, is the different target of the samesentiment for extraverts and introverts. For exam-ple, the ngram ”I fear” is highly predictive for anintrovert in our data while extraverts would ratheruse formulations to imply that others should fear.Similarly to Nowson et al. (2005), we did not findany difference in the formality measure of Hey-lighen and Dewaele (2002). Neither we did in thecomplexity of sentences as per the parse tree depth

and sentence length. It is probable that these as-pects were also impacted by our broad variety ofauthor style (F. Dostoyevsky vs J. K. Rowling).Our basic vector-based features carried no usefulinformation in our case, in contrast to the person-ality research of Neuman and Cohen (2014). Weobserved that the factual content of the stories con-tributed to the character similarity measure morethan the subtle personality differences.

5 Actions of fictional characters

While psycholinguists and consequenlty NLP re-searchers analyzed the relation between speech,resp. writing, and personality of an individual,psychologists often evaluate extraversion throughbehavioral personality questionnaries (Costa andMcCrae, 1992; Goldberg et al., 2006). We hypoth-esize that similar behavior shall be predictive forextraversion of fictional characters as perceived bythe readers.

5.1 Action extraction

For our purpose we define actions as the subject,verb and context of a sentence, where the subjectis a named entity Person and the context is eithera direct object in relation dobj to the verb or a firstchild of the adjacent verb phrase in a parse tree.After grouping the actions per character, the sub-ject name is removed. For example, a sample ofactions of the character Eddard Stark of Game ofThrones would be: X paused a moment, X studiedhis face, X changed his mind, X unrolled the paper,X said etc., visualized in Figure 1. We obtained 22030 actions for 205 characters (102 E, 116 I), withon average 100 actions for E and 101 for I. Notethat also actions for those characters who do nottalk enough in the books (often first-person per-spectives) could be used.

Figure 1: A revealing word cloud of the most frequent wordsfrom the actions of which Eddard Stark (Game of Thrones) isa subject. Size is proportional to the frequency of a word.

5.2 Action classification setupIn the system based on actions we use only a sub-set of the features described in 4.2. From the lex-ical features we focus on the 500 most frequentverbs and dependency word pairs. Semantic fea-tures are used the same way as in 4.2, profitingfrom LIWC, WordNet, Verbnet and the sentimentlexicons. Word embedding vectors for book char-acters are in this case computed by taking only theverbs into account rather than all content words.From the stylistic features we use the part-of-speech bigrams and trigrams, verb modality andverb tense.

5.3 Classification results on actionsTable 5 shows the performance of the classifica-tion models based on the protagonists’ actions, us-ing different feature groups. The overall perfor-mance is higher than for the direct speech model.

ID Feature set P R F A1 - (baseline) .267 .517 .352 .5172 Ngrams .539 .506 .505 .5073 LIWC .600 .577 .567 .5774 WordNet .517 .518 .517 .5185 VerbNet .599 .583 .578 .5836 Style .573 .601 .553 .6017 Sentiment .357 .453 .382 .4538 Vectors .504 .497 .451 .4979 All .600 .623 .598 .623Percentage human agreement: .849

Table 5: Weighted precision (P), recall (R), F-score (F) andaccuracy (A) for actions - in each line for a system using onlythe given group of features. WordNet stands for WordNetsemantic labels, VerbNet setup uses the WordNet-VerbNetlinks. Highlighted F-scores differ from the majority baselinesignificantly (p <0.05), using an approx. randomization test.

Due to the lack of previous NLP experimentson this task, we compare our features to the ac-tions measured in the International PersonalityItem Pool (Goldberg et al., 2006), frequently usedpersonality assesment questionnaire (Table 6).

The most predictive features of this model cap-ture the activity and excitement seeking facetsof extraversion. Stylistic features reflect thecomplexity difference of the verb phrases (Johnjumped vs. John thought about it), extraverts be-ing characterized by plain verbs. Semantic fea-tures exhibit higher precision than stylistic ones.Sense-linked semantic classes of VerbNet demon-strate the preference of extraverts for being ac-tive and expressing themselves - they jump, fight,shout, run in and run out, eat and drink, see andhear and get easily bored. Extraverts in books also

ExtravertInternational Personality Item Pool:likes to party, feels comfortable around people,starts conversations, talks to many people, enjoys beinga center of attention, makes friends easily, takes charge,captivates people, feels at ease with a company,is skilled in handling social situationsOur experiment:bring (VN), consume (VN), contiguous location(VN),holding (VN), social (WN), motion (WN), emotion (WN)Leisure (LIWC), Home (LIWC), Family (LIWC), fight,march, care, take, jump, shriek, clear throat, bore, get to,come in, agree, hold, hear, inform, sell, come forwardIntrovertInternational Personality Item Pool:Doesn’t talk much, stays in the background, has littleto say, does not draw attention, has difficulties toapproach others, is quiet around strangers, feelsuncomfortable around others,does not show feelings,is a private person, waits to be leadOur experiment:snooze (VN), conceal (VN), wish (VN), stative (WN),creation (WN), walk, sleep, lay, know, maintain, expect,hope, find out, might, help, explain

Table 6: Characteristic actions for extraverts and introvertsas assessed in the IPIP personality questionaire, compared toour most informative features

often bring or hold something. Introverts, on theother hand, seem to favor slow movements - whilethey are thinking, reflecting, creating, looking forexplanations and find out solutions, they tend to liedown, sit or walk, eventually even sleep or snooze.The uncertainty typical for introverts is also no-table in their actions, as they often hope or wishfor something they might like to do. Addition-ally, semantic classes Social and Family, reportedas correlated to extraversion by Pennebaker andKing (1999) and not confirmed in our first model,became predictive in protaonists’ actions.

5.4 Discussion

Also in this task, the VerbNet classes brought sig-nificant improvement in performance. The clas-sification model based on actions outperformsnot only the direct speech model, but also thestate-of-the-art systems predicting authors’ ex-traversion from the stream-of-consciousness es-says (Mairesse et al., 2007; Celli et al., 2013;Neuman and Cohen, 2014). While surely notdirectly comparable, this result hints to the factthat the personality is easier to detect from be-havior than from person’s verbal expression. Thiswould correspond to the findings of Mairesse etal. (2007), Biel and Gatica-Perez (2013) and Aranand Gatica-Perez (2013) on multimodal data sets.

6 Predicatives of fictional characters

Our third extraversion prediction system is sub-ordinate to how fictional characters are describedand to the manners in which they behave. We arenot aware of a previous NLP work predicting ex-traversion using descriptive adjectives of the per-sons in question. We thus juxtapose the most pre-dictive features of our system to the adjectival ex-traversion markers developed by Goldberg (1992).

6.1 Extraction of descriptive properties

In this setup we extract predicatives of the namedentities PERSON in the books - relations amod(angry John) and cop (John was smart). As theseexplicit statements are very sparse in modern nov-els, we additionally include adverbial modifiers(advmod) related to person’s actions (John saidangrily). We extract data for 205 characters, withon average 43 words per character.

(a) Master Yoda(Star Wars)

(b) Sansa Stark(Game of Thrones)

Figure 2: Frequency word clouds for character descriptions

6.2 Classification setup

This system uses similar set of lexical, semanticand vectorial features similarly as in 5.2, this timewith the focus on adjectives, nouns and adverbsinstead of verbs. Stylistic and VerbNet featuresare hence not included, word vectors are as in 4.2.

6.3 Classification results on descriptions

Table 7 reports on the performance of individualfeature groups. With only few words per charactersemantic lexicons are less powerful than ngrams.

ID Feature set P R F A1 - (baseline) .267 .517 .352 .5172 Ngrams .686 .657 .648 .6573 LIWC .645 .601 .586 .6014 WordNet .518 .545 .528 .5455 Sentiment .375 .463 .384 .4636 Vectors .267 .517 .352 .5177 All .692 .698 .693 .698Percentage human agreement: .849

Table 7: Weighted precision, recall, F-score and accuracy.Highlighted F-scores differ from the majority baseline sig-nificantly (p <0.05).

Table 8 displays the most predictive features inour system contrasted to the adjectival markers.

ExtravertGoldberg (1992) :adventurous,mischievous, playful, rambunctious,dominant, forceful, demonstrative, exhibitionistic,flamboyant, brave, courageous, daring, assured,...Our experiment :excited, restlessly, stubbornly, restless, beloved, eager,abruptly, defiantly, darkly, eagerly, loudly, reluctant,stubborn, unwise, ruthless, quickly, abruptly, right,change (WN), social (WN)IntrovertGoldberg (1992) :bashful, shy, timid, inhibited, restrainedunadventurous, unaggressive, uncompetitivebitter, joyless, melancholic, moody, morose,...Our experiment :anxious, patiently, hesitantly, backward, softly,warily, coldly, helplessly, respectfully, slowly,politely, thoughtfully, nervously, silent, carefully,gratefully, dryly, sheepishly, politely, weary, calm,gently, sadly, sideways, stative (WN)

Table 8: Characteristic adjectives for extraverts and intro-verts as reported by L. Goldberg, compared to our most in-formative features as per the correlation merit

6.4 Discussion on errors

All our systems had issues with characters rated byless than five readers and with protagonists withlow agreement. Other challenges arise from au-thorial style, age of the novel and speech individ-uality of characters (e.g. Yoda). Varied length ofinformation for different characters poses issues inmeasuring normally distributed features (e.g. ra-tio of jumping verbs), being in shorter texts lessreliable. Ongoing and future work on this task ad-dresses the limitations of these initial experiments,especially the data set size and the gold standardquality. Extending the data will also enable us toexamine different book genres as variables for thepersonality distribution and feature impact. It willbe worth examining the relations between charac-ters, since we observed certain patterns in our data,such as the main introvert character supported byhis best friend extravert. Additionally, we wantto verify if the system in Section 6 is overly opti-mistic due to the data size.

7 Conclusion and future work

Automated personality profiling of fictional char-acters, based on rigorous models from personal-ity psychology, has a potential to impact numer-ous domains. We framed it as a text classifica-tion problem and presented a novel collaborativelybuilt dataset of fictional personality. We incor-

porate features of both lexical resource-based andvectorial semantics, including WordNet and Verb-Net sense-level information and vectorial wordrepresentations. In models based on the speechand actions of the protagonists, we demonstratedthat the sense-linked lexical-semantic features sig-nificantly outperform the baselines. The most pre-dictive features correspond to the reported find-ings in personality psychology and NLP experi-ments on human personality. Our systems basedon actions and appearance of characters demon-strate higher performance than systems based ondirect speech, which is in accordance with recentresearch on personality in social networks (Kosin-ski et al., 2014; Biel and Gatica-Perez, 2013), re-vealing the importance of the metadata. We haveshown that exploiting the links between lexical re-sources to leverage more accurate semantic infor-mation can be beneficial for this type of tasks, ori-ented to actions performed by the entity. How-ever, the human annotator agreement in our taskstays high above the performance achieved. Con-sidering that most of the sucessful novels wereproduced as movies, we cannot exclude that ourannotators based their decision on the multimodalrepresentation of the protagonists. In the future weaim on collecting a more detail and rigorous goldstandard through gamification and expanding ourwork on all five personality traits from the Five-Factor Model and their facets, and ultimately ex-tend our system to a semi-supervised model deal-ing with notably larger amount of data. We alsoplan to examine closer the differences betweenperceived human and fictional personality, and therelationship between the personality of the readerand the characters.

Acknowledgments

This work has been supported by the Volkswa-gen Foundation as part of the Lichtenberg Pro-fessorship Program under grant No. I/82806 andby the German Research Foundation under grantNo. GU 798/14-1. Additional support was pro-vided by the German Federal Ministry of Educa-tion and Research (BMBF) as a part of the Soft-ware Campus program under the promotional ref-erence 01-S12054 and by the German Institutefor Educational Research (DIPF). We also warmlythank Holtzbrinck Digital GmbH for providing asubstantial part of the e-book resources, and theEMNLP reviewers for their helpful comments.

References[Aran and Gatica-Perez2013] Oya Aran and Daniel

Gatica-Perez. 2013. Cross-domain personality pre-diction: from video blogs to small group meetings.In Proceedings of the 15th ACM on Internationalconference on multimodal interaction.

[Argamon et al.2005] Shlomo Argamon, SushantDhawle, Moshe Koppel, and James W Pennebaker.2005. Lexical predictors of personality type. InProceedings of the Joint Annual Meeting of theInterface and the Classification Society of NorthAmerica.

[Bamman et al.2014] David Bamman, Ted Underwood,and Noah A Smith. 2014. A bayesian mixed effectsmodel of literary character. In Proceedings of the52nd Annual Meeting of the Association for Compu-tational Linguistics.

[Biel and Gatica-Perez2013] J Biel and Daniel Gatica-Perez. 2013. The youtube lens: Crowdsourcedpersonality impressions and audiovisual analysisof vlogs. Multimedia, IEEE Transactions on,15(1):41–55.

[Byrne1961] Donn Byrne. 1961. Interpersonal attrac-tion and attitude similarity. The Journal of Abnor-mal and Social Psychology, 62(3):713.

[Celli et al.2013] Fabio Celli, Fabio Pianesi, DavidStillwell, and Michal Kosinski. 2013. Workshop oncomputational personality recognition (shared task).In Proceedings of the Workshop on ComputationalPersonality Recognition.

[Celli et al.2014] Fabio Celli, Bruno Lepri, Joan-IsaacBiel, Daniel Gatica-Perez, Giuseppe Riccardi, andFabio Pianesi. 2014. The workshop on computa-tional personality recognition 2014. In Proceedingsof the ACM International Conference on Multime-dia. ACM.

[Chambers2013] Nathanael Chambers. 2013. Eventschema induction with a probabilistic entity-drivenmodel. In EMNLP, volume 13, pages 1797–1807.

[Chartrand and Bargh1999] Tanya L Chartrand andJohn A Bargh. 1999. The chameleon effect:the perception–behavior link and social interac-tion. Journal of personality and social psychology,76(6):893.

[Coltheart1981] Max Coltheart. 1981. The mrc psy-cholinguistic database. The Quarterly Journal ofExperimental Psychology, 33(4):497–505.

[Corney et al.2002] Malcolm Corney, Olivier de Vel,Alison Anderson, and George Mohay. 2002.Gender-preferential text mining of e-mail discourse.In Proceedings of 18th Annual Computer SecurityApplications Conference. IEEE.

[Costa and McCrae1992] Paul T Costa and Robert RMcCrae. 1992. Professional manual: revised NEO

personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI). Odessa, FL: Psycho-logical Assessment Resources.

[Daxenberger et al.2014] Johannes Daxenberger, OliverFerschke, Iryna Gurevych, and Torsten Zesch. 2014.Dkpro tc: A java-based framework for supervisedlearning experiments on textual data. In Proceed-ings of the 52nd Annual Meeting of the Associationfor Computational Linguistics. System Demonstra-tions, pages 61–66.

[De Clercq et al.2014] Orphee De Clercq, MichaelSchuhmacher, Simone Paolo Ponzetto, andVeronique Hoste. 2014. Exploiting framenet forcontent-based book recommendation. In CBRecSysat ACM RecSys, number 1613-0073, pages 14–21.CEUR-WS.

[Dewaele and Furnham1999] Jean-Marc Dewaele andAdrian Furnham. 1999. Extraversion: The unlovedvariable in applied linguistic research. LanguageLearning, 49(3):509–544.

[Di Noia et al.2012] Tommaso Di Noia, Roberto Mi-rizzi, Vito Claudio Ostuni, Davide Romito, andMarkus Zanker. 2012. Linked open data to supportcontent-based recommender systems. In Proceed-ings of the 8th International Conference on SemanticSystems, I-SEMANTICS ’12, pages 1–8, New York,NY, USA. ACM.

[Dotson2009] Daniel Dotson. 2009. Portrayal ofphysicists in fictional works. CLCWeb: Compara-tive Literature and Culture, 11(2):5.

[Elson et al.2010] David K Elson, Nicholas Dames, andKathleen R McKeown. 2010. Extracting social net-works from literary fiction. In Proceedings of the48th Annual Meeting of the Association for Compu-tational Linguistics.

[Estival et al.2007] Dominique Estival, Tanja Gaustad,Son Bao Pham, Will Radford, and Ben Hutchinson.2007. Author profiling for english emails. In Pro-ceedings of the 10th Conference of the Pacific Asso-ciation for Computational Linguistics.

[Esuli and Sebastiani2006] Andrea Esuli and FabrizioSebastiani. 2006. SentiWordNet: a publicly avail-able lexical resource for opinion mining. In Pro-ceedings of the 5th International Conference onLanguage Resources and Evaluation, volume 6.

[Finkel et al.2005] Jenny Rose Finkel, Trond Grenager,and Christopher Manning. 2005. Incorporatingnon-local information into information extractionsystems by gibbs sampling. In Proceedings of the43rd Annual Meeting on Association for Computa-tional Linguistics, pages 363–370. Association forComputational Linguistics.

[Gill and Oberlander2002] Alastair J Gill and Jon Ober-lander. 2002. Taking care of the linguistic featuresof extraversion. In Proceedings of the 24th AnnualConference of the Cognitive Science Society.

[Goldberg et al.2006] Lewis R Goldberg, John A John-son, Herbert W Eber, Robert Hogan, Michael C Ash-ton, C Robert Cloninger, and Harrison G Gough.2006. The international personality item pool andthe future of public-domain personality measures.Journal of Research in personality, 40(1):84–96.

[Goldberg1990] Lewis R Goldberg. 1990. An alterna-tive description of personality: the Big-Five factorstructure. Journal of personality and social psychol-ogy, 59(6):1216.

[Goldberg1992] Lewis R Goldberg. 1992. The devel-opment of markers for the Big-Five factor structure.Psychological assessment, 4(1):26.

[Gurevych et al.2007] Iryna Gurevych, MaxMuhlhauser, Christof Muller, Jurgen Steimle,Markus Weimer, and Torsten Zesch. 2007. Darm-stadt Knowledge Processing Repository Based onUIMA. In Proceedings of the First Workshop onUnstructured Information Management Architectureat Biannual Conference of the Society for Com-putational Linguistics and Language Technology,Tubingen, Germany.

[Gurevych et al.2012] Iryna Gurevych, Judith Eckle-Kohler, Silvana Hartmann, Michael Matuschek,Christian M Meyer, and Christian Wirth. 2012.Uby: A large-scale unified lexical-semantic resourcebased on LMF. In Proceedings of the 13th Confer-ence of the European Chapter of the Association forComputational Linguistics. Association for Compu-tational Linguistics.

[Hall1999] Mark A Hall. 1999. Correlation-based fea-ture selection for machine learning. Ph.D. thesis,The University of Waikato.

[He et al.2013] Hua He, Denilson Barbosa, and Grze-gorz Kondrak. 2013. Identification of speakers innovels. In Proceedings of the 51st Annual Meetingon Association for Computational Linguistics, pages1312–1320.

[Heitmann and Hayes2010] Benjamin Heitmann andConor Hayes. 2010. Using linked data to buildopen, collaborative recommender systems. In AAAIspring symposium: linked data meets artificial intel-ligence.

[Heylighen and Dewaele2002] Francis Heylighen andJean-Marc Dewaele. 2002. Variation in the Contex-tuality of Language: An Empirical Measure. Foun-dations of Science, 7(3):293–340.

[Iosif and Mishra2014] Elias Iosif and Taniya Mishra.2014. From speaker identification to affective anal-ysis: A multi-step system for analyzing children sto-ries. EACL 2014, pages 40–49.

[John and Srivastava1999] Oliver P John and SanjaySrivastava. 1999. The Big Five trait taxonomy:History, measurement, and theoretical perspectives.Handbook of personality: Theory and research,2(1999):102–138.

[Johnson et al.2011] John A Johnson, Joseph Carroll,Jonathan Gottschall, and Daniel Kruger. 2011. Por-trayal of personality in victorian novels reflects mod-ern research findings but amplifies the significanceof agreeableness. Journal of Research in Personal-ity, 45(1):50–58.

[Kaufman and Libby2012] Geoff F Kaufman andLisa K Libby. 2012. Changing beliefs and behaviorthrough experience-taking. Journal of personalityand social psychology, 103(1):1.

[Kipper-Schuler2005] Karin Kipper-Schuler. 2005.VerbNet: A broad-coverage, comprehensive verblexicon. Ph.D. thesis, University of Pennsylvania.

[Kokkinakis and Malm2011] Dimitrios Kokkinakis andMats Malm. 2011. Character profiling in 19th cen-tury fiction. Language Technologies for Digital Hu-manities and Cultural Heritage.

[Kosinski et al.2013] Michal Kosinski, David Stillwell,and Thore Graepel. 2013. Private traits andattributes are predictable from digital records ofhuman behavior. Proceedings of the NationalAcademy of Sciences, 110(15):5802–5805.

[Kosinski et al.2014] Michal Kosinski, YoramBachrach, Pushmeet Kohli, David Stillwell,and Thore Graepel. 2014. Manifestations ofuser personality in website choice and behaviouron online social networks. Machine learning,95(3):357–380.

[Lepri et al.2010] Bruno Lepri, Ramanathan Subrama-nian, Kyriaki Kalimeri, Jacopo Staiano, Fabio Pi-anesi, and Nicu Sebe. 2010. Employing social gazeand speaking activity for automatic determination ofthe extraversion trait. In International Conferenceon Multimodal Interfaces and the Workshop on Ma-chine Learning for Multimodal Interaction, ICMI-MLMI ’10, pages 7:1–7:8, New York, NY, USA.ACM.

[MacDonald et al.1994] Douglas A MacDonald, Pe-ter E Anderson, Catherine I Tsagarakis, and Cor-nelius J Holland. 1994. Examination of the rela-tionship between the Myers-Briggs Type Indicatorand the NEO personality inventory. PsychologicalReports, 74(1):339–344.

[Mairesse et al.2007] Francois Mairesse, Marilyn AWalker, Matthias R Mehl, and Roger K Moore.2007. Using linguistic cues for the automatic recog-nition of personality in conversation and text. Jour-nal of artificial intelligence research.

[Mar et al.2009] Raymond A Mar, Keith Oatley, andJordan B Peterson. 2009. Exploring the link be-tween reading fiction and empathy: Ruling out indi-vidual differences and examining outcomes. Com-munications Journal, 34(4):407–428.

[McCrae and Costa1987] Robert R McCrae and Paul TCosta. 1987. Validation of the five-factor model ofpersonality across instruments and observers. Jour-nal of personality and social psychology, 52(1):81.

[McCrae and Costa1989] Robert R McCrae and Paul TCosta. 1989. Reinterpreting the myers-briggstype indicator from the perspective of the five-factor model of personality. Journal of personality,57(1):17–40.

[McCrae et al.2012] Robert R McCrae, James FGaines, and Marie A Wellington. 2012. Thefive-factor model in fact and fiction. Handbook ofpsychology. 2nd ed. New Jersey: John Wiley & SonsInc.

[Mehl et al.2006] Matthias R Mehl, Samuel D Gosling,and James W Pennebaker. 2006. Personality in itsnatural habitat: manifestations and implicit folk the-ories of personality in daily life. Journal of person-ality and social psychology, 90(5):862.

[Miller1995] George A Miller. 1995. WordNet: a lex-ical database for English. Communications of theACM, 38(11):39–41.

[Mohammad and Kiritchenko2013] Saif M Moham-mad and Svetlana Kiritchenko. 2013. Using nu-ances of emotion to identify personality. arXivpreprint arXiv:1309.6352.

[Mohammad and Turney2010] Saif M Mohammad andPeter D Turney. 2010. Emotions evoked by com-mon words and phrases: Using Mechanical Turk tocreate an emotion lexicon. In Proceedings of theNAACL HLT 2010 Workshop on Computational Ap-proaches to Analysis and Generation of Emotion inText, pages 26–34.

[Montoya et al.2008] R Matthew Montoya, Robert SHorton, and Jeffrey Kirchner. 2008. Is actual sim-ilarity necessary for attraction? a meta-analysis ofactual and perceived similarity. Journal of Socialand Personal Relationships, 25(6):889–922.

[Myers et al.1985] Isabel Briggs Myers, Mary H Mc-Caulley, and Robert Most. 1985. Manual, a guideto the development and use of the Myers-Briggs typeindicator. Consulting Psychologists Press.

[Neuman and Cohen2014] Yair Neuman and YochaiCohen. 2014. A vectorial semantics approach topersonality assessment. Scientific reports, 4.

[Nowson et al.2005] Scott Nowson, Jon Oberlander,and Alastair J Gill. 2005. Weblogs, genres and in-dividual differences. In Proceedings of the 27th An-nual Conference of the Cognitive Science Society,volume 1666, page 1671. Citeseer.

[Nowson2007] Scott Nowson. 2007. Identifying morebloggers: Towards large scale personality classifica-tion of personal weblogs. In In Proceedings of theInternational Conference on Weblogs and Social.

[Oberlander and Nowson2006] Jon Oberlander andScott Nowson. 2006. Whose thumb is it anyway?:classifying author personality from weblog text.In Proceedings of the COLING/ACL on Mainconference poster sessions.

[O’Keefe et al.2012] Tim O’Keefe, Silvia Pareti,James R Curran, Irena Koprinska, and MatthewHonnibal. 2012. A sequence labelling approachto quote attribution. In Proceedings of the 2012Joint Conference on Empirical Methods in NaturalLanguage Processing and Computational NaturalLanguage Learning, pages 790–799. Associationfor Computational Linguistics.

[Passant2010] Alexandre Passant. 2010. dbrec musicrecommendations using DBpedia. In The SemanticWeb–ISWC 2010, pages 209–224.

[Pennebaker and King1999] James W Pennebaker andLaura A King. 1999. Linguistic styles: languageuse as an individual difference. Journal of personal-ity and social psychology, 77(6):1296.

[Pennebaker et al.2001] James W Pennebaker,Martha E Francis, and Roger J Booth. 2001.Linguistic inquiry and word count: LIWC 2001.Mahway: Lawrence Erlbaum Associates, 71.

[Pennington et al.2014] Jeffrey Pennington, RichardSocher, and Christopher D Manning. 2014. Glove:Global vectors for word representation. Proceedingsof the Empiricial Methods in Natural Language Pro-cessing (EMNLP 2014), 12.

[Pittenger2005] David J Pittenger. 2005. Cautionarycomments regarding the myers-briggs type indica-tor. Consulting Psychology Journal: Practice andResearch, 57(3):210.

[Rentfrow et al.2011] Peter J Rentfrow, Lewis R Gold-berg, and Daniel J Levitin. 2011. The structure ofmusical preferences: a five-factor model. Journal ofpersonality and social psychology, 100(6):1139.

[Smith et al.2013] Noah A Smith, David Bamman, andBrendan OConnor. 2013. Learning latent personasof film characters. In Proceedings of the 51st An-nual Meeting of the Association for ComputationalLinguistics.

[Terracciano et al.2008] Antonio Terracciano,Corinna E Lockenhoff, Rosa M Crum, O JosephBienvenu, and Paul T Costa. 2008. Five-factormodel personality profiles of drug users. BmcPsychiatry, 8(1):22.

[Tirre and Dixit1995] William C Tirre and SharvariDixit. 1995. Reading interests: Their dimension-ality and correlation with personality and cogni-tive factors. Personality and Individual Differences,18(6):731–738.