Metaphoria: An Algorithmic Companion for Metaphor Creation · Metaphor Creation Katy Ilonka Gero Columbia University [email protected] Lydia B. Chilton Columbia University [email protected]

Metaphoria An Algorithmic Companion forMetaphor Creation

Katy Ilonka GeroColumbia Universitykatycscolumbiaedu

Lydia B ChiltonColumbia University

chiltoncscolumbiaedu

ABSTRACTCreative writing from poetry to journalism is at the cruxof human ingenuity and social interaction Existing creativewriting support tools produce entire passages or fully formedsentences but these approaches fail to adapt to the writerrsquosown ideas and intentions Instead we posit to build tools thatgenerate ideas coherent with the writerrsquos context and en-courage writers to produce divergent outcomes To explorethis we focus on supporting metaphor creation We presentMetaphoria an interactive system that generates metaphori-cal connections based on an input word from the writer Ourstudies show that Metaphoria provides more coherent sug-gestions than existing systems and supports the expressionof writersrsquo unique intentions We discuss the complex issueof ownership in human-machine collaboration and how tobuild adaptive creativity support tools in other domains

CCS CONCEPTSbull Human-centered computing rarr Interactive systemsand tools Natural language interfaces bull Applied comput-ingrarr Arts and humanities

KEYWORDShuman-computer collaboration co-creativity generative artwriting support natural language processing

ACM Reference FormatKaty Ilonka Gero and Lydia B Chilton 2019 Metaphoria An Al-gorithmic Companion for Metaphor Creation In CHI Conferenceon Human Factors in Computing Systems Proceedings (CHI 2019)May 4ndash9 2019 Glasgow Scotland UK ACM New York NY USA12 pages httpsdoiorg10114532906053300526

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copiesare not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page Copyrightsfor components of this work owned by others than the author(s) mustbe honored Abstracting with credit is permitted To copy otherwise orrepublish to post on servers or to redistribute to lists requires prior specificpermission andor a fee Request permissions from permissionsacmorgCHI 2019 May 4ndash9 2019 Glasgow Scotland UKcopy 2019 Copyright held by the ownerauthor(s) Publication rights licensedto ACMACM ISBN 978-1-4503-5970-21905 $1500httpsdoiorg10114532906053300526

Figure 1 A poet using Metaphoria to find metaphorical con-nections between america and wood

1 INTRODUCTIONCreative writing from poetry to journalism is at the crux ofhuman ingenuity and social interaction It conveys not onlyinformation but also experience emotion and beauty Whilecomputation has opened a floodgate of creative tools formusic and the visual arts little of that fervor has transferredto text Word processors that detect grammatical errors areuseful but do not support the creative elements of writingPast work in computational support for creative writing

has focused on suggesting next sentences while writing sto-ries [3 26 36] or fully generating a creative output based ona topic [11] These ideas have potential but current systemsfail to provide strong coherence with the intention of thewritermdasheither the text that they have already written or theirintention for the entire output Since these tools are not user-centric they are most useful during ideation when there arefewer constraints In this case a systemrsquos failure to providecoherence can be seen as a feature a random suggestion canhelp a writer move in an unexpected direction

CHI 2019 May 4ndash9 2019 Glasgow Scotland UK Katy Ilonka Gero and Lydia B Chilton

We can improve tools for creative writing by designingthem from a user-centric perspective To do so we proposefocusing on the building blocks of creative writing in whichwriters have more specific goals Instead of providing com-plete sentences generally applicable to wherever the writeris we can improve the relevance of our support by constrain-ing the idea space to a specific writing goal and allow itto be used at more points in the writing process We focuson metaphor which famously conveys complex or abstractideas succinctly and is used in everything from poetry tojournalism to science education [19 28 29]

Creating unconventional and expressivemetaphors is chal-lenging [10] requiring divergent and lateral cognitive pro-cesses [13] We present Metaphoria an interactive systemthat generates potential metaphorical connections for any in-put word Metaphoria uses an open source knowledge graphand a modified Word Moverrsquos Distance algorithm to finda large ranked list of suggested metaphorical connectionsThese suggestions are embedded in an interactive interfacethat allows writers to generate ideas for any input Figure 1shows the system while used by a professional poetWe ran three studies to evaluate Metaphoria First we

compare our method for generating suggestions to state-of-the-art systems and show it performs better across threemetrics for metaphor quality Second we have novices writeextended metaphors with and without Metaphoria and showthat Metaphoria generates meaningful and inspirational sug-gestions given a specific writing task Third we have pro-fessional poets write poems with Metaphoria and show therange of expression using the system In the Discussion wereport on issues of ownership that arise when a computa-tional system produces ldquohuman-likerdquo output and suggestfuture work to mitigate these concerns

We make the following contributions

bull A computational method for producing metaphoricalconnections better than state-of-the-art algorithms

bull Metaphoria an interactive system for collaborativelywriting metaphors with a computer

bull User studies with novice and expert writers showingthat Metaphoria gives people useful and inspirationalsuggestions and increases the diversity of responses

bull Design implications for ownership in co-creative sys-tems more generally

2 RELATEDWORKWriting supportWriting support has a long history editing has existed per-haps as long as writing and the introduction of dictionariesand thesauri gave writers external tools they could use ontheir own Experimental writing movements such as theDadaists with their cut-up technique and the Oulipo with

their constrained methods employed algorithmic ideas totrigger inspiration pre-dating the advent of computers

One of the early successes of computation was the devel-opment of spell-check [33] and grammar-checking remainsan active area of research today [20] Recent computationalwork has leveraged cognitive apprenticeship models to im-prove writing with highly specific goals such as an emailto request help [15] an essay for a standardized test [2] ora piece of journalism [25] Work on collaborative writing[1 17 39] has shown that writing can be broken into micro-tasks in which individuals can contribute usefully withoutaccess to the full writing documentThis success suggests applying user-centric ideas to cre-

ative writing Support for creative writing has focused ongenerating next sentences for a story [3 36 38] or generatingentire poems given a topic [11 31] While this paradigm haspotential to trigger inspiration similar to the earlier experi-mental movements we focus on providing more coherentsuggestions by responding to the need for rhetorical devicesWe provide support for metaphor creation a common butchallenging rhetorical device [10] This narrowing of thegoal similar to previous HCI work on writing allows us toachieve the coherence necessary to move beyond randomassociation and support the creation of meaning

Creativity support and co-creativityCreativity support tools have flourished for music and thevisual arts from the widespread adoption of software for gen-eration and editing to the development of medium-specificprogramming languages [22 34 45] These tools are begin-ning to tackle how to be compatible with existing manualpractices [16] as well as how to be more compatible withcurrent artificial intelligence frameworks [6 30]

The way in which creativity support tools integrate withan artistrsquos practice is at the heart of these issues When asupport tool provides more complete or conceptual contribu-tions or provides contributions without a request from theartist (as in mixed-initiative user interfaces [14]) the termco-creativity is often used Critically Davis defines human-computer co-creativity as when the ldquoprogram is adaptingto the input of the userrdquo [5] This distinguishes co-creativesystems from more procedural contributions in which anartist either has a high level of control over the outputs asin a synthesizer or little to no control over the outputs as ina computer-generated poem based on a topic [11]

It is essential to think about tools as supporting artists intheir desired practice rather than replacing aspects deemedcomputationally tractable Support for creativewriting shouldalign with the lsquowide wallsrsquo design principle of creativity sup-port tools in which tools aim to ldquosupport and suggest a widerange of explorationsrdquo [35] Unlike more specified writing

Metaphoria An Algorithmic Companion for Metaphor Creation CHI 2019 May 4ndash9 2019 Glasgow Scotland UK

tasks (such as writing an email to request help) creative writ-ers do not want tools that will make their writing sound thesame as others [38] Thus in co-creative domains systemsshould be conducive to divergent outcomes

Metaphor generation algorithmsMetaphor generation is a version of conceptual blending[7] that has been correlated with general fluid intelligence[37] and is considered an important challenge in artificialintelligence [44]

Current metaphor generation systems find properties thatcan be attributed to the two concepts in the metaphor Twoprominent algorithms are Thesaurus Rex [40 42] and Inter-secting Word Vectors [8] Thesaurus Rex [40 42] is a webservice that provides shared attributes and categories for in-put concepts For example inputting coffee amp cola producesresults such as acidic food and nonalcoholic beverage The-saurus Rex is explicitly designed to support metaphor gener-ation [41 43] Intersecting Word Vectors [8] is a metaphorgeneration algorithm in which connector words are foundusing word embeddings Connector words are those foundin the intersection of the 1000 words closest to each of theconcept words For example connector words for storm ampsurrender include barrage and onslaught These systems arestrong baselines for metaphor generation from the artificialintelligence and natural language processing communitiesTheories of metaphor often conform to structural align-

ment theory [9] in which analogies are discovered by findingisomorphic sections of knowledge graphs where each edge isa structural relation between concepts Work on using analo-gies for product design [12] has focused on the differencebetween structural and functional aspects of products forideation We draw on these ideas of structural and functionalconnections as a search function for concept attributes

3 DESIGN OF METAPHORIADesign GoalsBased on our literature review coherence to context is thebiggest barrier to use for creative writing support tools [3 2636] Secondarily writers do not want tools that make theirwriting sound the same as others [38] Thus suggestions thatresult in divergent outcomes for writers is crucial Thesegoals map to previous methodology in HCI for the evaluationof generative drawing tools Jacobs et al [16] evaluate theirdrawing tool on compatibility (coherence to context) andexpressiveness (ability to express a divergent set of ideas) A system that is coherent to context provides sugges-

tions that are relevant to the task at hand If writers cometo the system with an idea or intention the system shouldgenerate metaphorical phrases coherent with this contextand should be flexible enough to be coherent for a wide range

high envy is used for getting attention like a bellenvy is for alerting you to something like a bellenvy is used to toll like bell

low envy is for playing music like a bellTable 1 Examples of connections with high and lowrelevance for the seed envy is a bell

of writer ideas and intentions A system that encourages di-vergent outcomes provides many compelling options andincreases the variation in writersrsquo work rather than propelall writers toward similar metaphors

To address coherence to context we focus on generatingmetaphorical connections for a given ldquoseed metaphorrdquo Seedmetaphors are of the form [source] is [vehicle] eg envy isa bell where envy is the source and bell is the vehicle Byfocusing on connections between the words such as lsquoenvycan sound the alarm like a bellrsquo rather than the selection ofthe seed words we leave open the possibility that the writerinputs one or both words of the seed metaphor

To address divergent outcomes we generate and presentmultiple distinct suggestions for each seed metaphor Thisapproach allows writers to select a suggestion salient forthem in particular

Generating coherent connectionsStarting with a seed metaphor our approach is to first gen-erate many features of the vehicle (bell) and then rank thesefeatures by how related they are to the source (envy) Thisaligns with traditional metaphor usage in which features ofthe vehicle are used to explain the sourceTo find features of the vehicle we use ConceptNet [24]

an open-source knowledge graph as a source of structuraland functional properties of words Structural properties areelements that define or compose an object For example abell has a clapper and a mouth In ConceptNet we select forstructural features by querying the ldquoHasArdquo relations of thevehicle Functional properties focus on an objectrsquos actionsand purpose For example a bell can make noise and be usedfor alerting In ConceptNet we select for functional featuresby querying the ldquoUsedForrdquo and ldquoCapableOfrdquo relations To-gether structural and functional properties provide a largeset of potential connections from the vehicle to the sourceNot all features of the vehicle (bell) will metaphorically

map to the source (envy) To find the most relevant oneswe rank how related the vehicle features (eg used for get-ting attention) are to the source (envy) To rank suggestionswe use GloVe word embeddings [32] trained on Wikipedia2014 + Gigaword 5 Word embeddings are a common way tomeasure the semantic similarity between words [27] Here

we use them to measure the semantic similarity betweenthe vehicle property and source word Examples of vehicleproperties with high and low relevance are found in Table 1

To find the semantic distance between vehicle features andthe source word we use a modified Word Moverrsquos Distance(WMD) [18] WMD is an algorithm for finding the smallestdistance between two documents ie sets of words in a wordembedding space It formulates distance between documentsas a transportation problem we denote c(i j) as the distancebetween words xi and x j where c(i j) is the cosine distancebetween the two word vectors Given two documents D1 andD2 we allow each word i in D1 to be transformed into anyword in D2 in total or in parts We denote Ti j as how muchof word i in D1 is transformed to word j in D2 thereforesum

i j Ti j = 1We can define the distance between two documents as the

minimum cumulative cost of moving all words in D1 to allwords in D2 This is equivalent to solving the linear program

minsumi j

Ti j lowast c(i j) (1)

for which specialized solvers have been developed Forexample this would find the shortest distance from makingnoise to envy1 From this ranking of connections we canselect the top n as the most coherent

Selecting multiple distinct connectionsIn order to promote diverse outcomes our systems presentswriters with 10 coherent suggestions that are semanticallydistinct For instance get attention and getting peoplersquos at-tention may both be coherent yet they give effectively thesame idea to the writer For this reason as we build our listof suggestions to show the writer we throw out any featurethat is too close to any of the features already ranked Thiscloseness is again calculated with the Word Moverrsquos Dis-tance this time between two features Through observationwe find a distance of less than 4 indicates two features arenot semantically distinct

Additional coherence with valence rankingThe word embedding space is not sensitive to antonymsand thus some highly ranked features have a mismatchedsentiment with the source concept Pilot testing showedthat people found mismatched sentiments to be jarring andcaused them to lose faith in the system However peoplewho are first shown more intuitive features were more likelyto appreciate the antonym features Thus we first select thesuggestions as shown above and then re-rank them by howsimilar the valence of each one is to the source concept

1In this usage D2 is always a single word the source concept although ourimplementation allows for natural expansion into multi-word sources

Figure 2 Screenshot of Metaphoria with suggestion for jeal-ousy is a garden expanded

Valence is the positive or negative connotation of a wordand we assign valence scores to all words based on Warrineret alrsquos database [46] We denote the valence of the source asVsource and the valence of word i in the feature Vi for words1 n Then we define the valence distance as

Vdist = |Vsource minus avg(V1 Vn)| (2)We can then reorder the suggestions from the smallest

valence distance to the largestFinally we rephrase all connections into a suggestion for

the writer given the source envy the vehicle bell and theconnecting featuremaking noise the suggestion is presentedas lsquoenvy is used for making noise like a bellrsquo

Additional distinctness with suggestion expansionGreat metaphors are specific wewant to support writing spe-cific metaphors by expanding them to include more detailsof how the source and vehicle are connected If envy makesnoise like a bell we can expand on the details of the noise abell makes (eg vibrato reverberation highlow pitch) and howthese details relate to envy For example the noise of a bellhas reverberation and envy has lasting bitterness Metaphoriaprovides multiple detailed metaphoric expansions for eachsuggestion to give writers more diverse optionsTo generate the expanded metaphors we first split each

suggestion into two parallel sentences one about the vehicle(bells make noise) and one about the source (envymakes noise)We want to find several alternative words to replace noisein each sentence To generate these words we again rely onword embeddings This time however we want to discoverwords that will syntactically match the sentencendashfor thisreason we use word embeddings trained using a dependencyparse as the context [21] This results in similar words alsohaving a similar part of speech We use the word embeddingsto create list of 60 words similar to the content word (noise)and 60 words similar to source (envy) Then we order thesewords by similarity to the vehicle (bell) and original word(noise) respectively and return the 10 most related words

in each case Figure 2 shows the interface where a writerselects the suggestion ldquojealousy is for growing flowers like agardenrdquo and can click through suggested expansions such asldquojealousy is for growing sorrowrdquo

InteractivityThe above methods are embedded in a Flask-based web ap-plication as shown in Figure 1 Writers can input their ownsource and click through a set of common vehicles Eachcombination will generate a list of up to 10 suggestions andeach suggestion can be expandedThe design of Metaphoria has our goals of coherence

to context and divergent outcomes in mind By allowingwriters to input a source and change the vehicle we adaptto the intention of the writer allowing greater coherenceShowing writers 10 semantically relevant suggestions andenabling writers to lsquoshiftrsquo the suggestions with the detailwords enables a diversity of ideas and hopefully responses

4 STUDY 1 SUGGESTION QUALITYThis study evaluates the quality of the suggestions Metapho-ria generates To achieve coherence to context suggestionsshould make sense given their seed metaphor and enact prin-ciples of high quality writing

MethodologyTo evaluate the suggestions we compare them to two otherstate-of-the-art metaphor generation algorithms ThesaurusRex [42] and IntersectingWord Vectors [8] These algorithmsare described fully in the Related Works section As our sys-tem produces a ranked set of suggestions we also compareboth the highest ranked suggestions with the lowest to eval-uate the effectiveness of the ranking algorithmThesaurus Rex produces shared attributes for example

envy amp bell produces attributes such as loud Intersecting sim-ilarly produces connector words for envy amp bell it produceswords such as behold In both cases we formulate these intosentences comparable with Metaphoria suggestions Table 2shows examples of this

For each systemwe select the top three ranked suggestionsRanking for Metaphoria is done using the WMD distance tothe source concept (as explained in the Design section) bothThesaurus Rex and Intersecting generate ranked lists

To compare the systems we define three metrics for eval-uating metaphor strength The first is aptness in which ametaphor accurately describes a connection between theconcepts this is the ground level of metaphors The secondis specificity in which a metaphor describes a connectionunlikely to be transferable other concepts The third is im-ageability in which a metaphor describes a connection thereader can visualize

Metaphoriaenvy is used for getting attention like a bellenvy is for alerting you to something like a bell

Thesaurus Rexenvy is loud like a bellenvy is audible like a bell

Intersectingenvy is shiny like a bellenvy can behold like a bell

Table 2 Examples of metaphors fromMetaphoria andtwo comparable state-of-the-artmetaphor generationalgorithms for the seed envy is a bell

We expect that Intersecting will not be particularly apt asit relies solely on the embedding space to provide meaningand embedding spaces notoriously lack consistent discretesemantics [23] Thesaurus Rex uses textual evidence so weexpect its connections to be apt but because of this we alsoexpect it to be less imageable and specific as it may only findhigher level and thus vaguer attributes

We have three hypothesesndash H1 Metaphoria suggestions are more apt than Inter-secting and at least as apt as Thesaurus Rex

ndash H2 Metaphoria suggestions are more specific thanThesaurus Rex and Intersecting

ndash H3Metaphoria suggestions aremore imageable thanThesaurus Rex and Intersecting

Additionally we want to know if top-ranked Metapho-ria suggestions are more apt than bottom-ranked ones Forthis we compare the top three and bottom three rankedsuggestions Our hypothesis is

ndash H4 Top-ranked Metaphoria suggestions aremore aptthan bottom ranked ones

We have two professional writers with an MFA in CreativeWriting act as annotators We consider 12 different seedmetaphors eg hope is a stream and for each generate thetop 3 metaphor suggestions from each system Additional wegenerate the bottom 3 metaphor suggestions for MetaphoriaThis results in 144 suggestions total

The annotators consider each metaphor suggestion andmark whether it is apt specific and imageable They are toldthat all suggestions are generated by computers but theyare not told anything about how or the fact that they comefrom different systems They are shown the suggestions foreach seed metaphor in random order

In addition to definitions of the metrics annotators werealso provided with examples of positive and negative casesfor each category as shown found in Table 3

Apt makes sensestrong example Love can come on unexpectedlyweak example Love is a weather event

Specific uniquely belongingstrong example Love can last through the nightweak example Love is dark

Imageable evokes visualstrong example Love can rain down on our headsweak example Love can scare people

Table 3 Examples of strong and weak sentences foreach of the metaphor evaluation metrics All sen-tences are based on the seed metaphor love is a storm

Apt Specific Imageable

Metaphoria (M) 97 82 100Thesaurus Rex (TR) 100 47 100Intersecting (I) 49 43 53

Table 4 While both Metaphoria and Thesaurus Rexgenerate apt and imageablemetaphors onlyMetapho-ria consistently produces specific metaphors

As in any evaluation of linguistic artifacts it is not clearthat there are precise or correct rankings for all of theseattributes Instead there are general trends that most nativeEnglish speakers would agree with We first have the annota-tors evaluate suggestions for 2 seed metaphors together anddiscuss their evaluation in order to establish common under-standings of the metrics They then individually evaluate thesuggestions for the 12 seed metaphors

ResultsWe report the percent agreement between the two annotatorsfor apt specific and imageable (and the Cohenrsquos Kappa corre-lation coefficients) to be 85 (063) 83 (067) and 88 (064)respectively Given the natural ambiguity of metaphors andcreative writing this is a high level of agreementThe following results are determined by combining the

evaluations of the two annotators the higher evaluation isused in cases of disagreement Table 4 shows the percentof times a given systemsrsquo suggestions was marked as aptspecific or imageable While Metaphoria and Thesaurus Rexmetaphors are both consistently apt and imageable Metapho-ria outperforms all systems on specificityTo test H1-3 we perform paired t-tests (Bonferonni cor-

rected) on the relevant pairs and disprove the null hypothesisfor H1 and H2 However it is clear that H3 does not hold as

Hypothesis diff t-value p-value

H1a M more apt than I 048 583 28e-08H1b TR more apt than I 051 616 48e-09

H2a M more specific than TR 034 336 27e-03H2b M more specific than I 038 355 67e-04

H3a M more imageable than TR 000 na naH3b M more imageable than I 047 559 14e-09

Table 5 T-tests confirm that Metaphoria is as goodor better across all metrics than state-of-the-artmetaphor generation algorithms P-values are Bonfer-onni corrected

Top-ranked 97 82 100Bottom-ranked 78 85 89

Table 6 Top-ranked metaphors perform significantlybetter than bottom-ranked metaphors on aptness andimageability there is no significant difference forspecificity

both Metaphoria and Thesaurus Rex were 100 imageableThe results of the statistical tests can be found in Table 5

Surprisingly Thesaurus Rex metaphors were as imageableas Metaphoria ones In general the annotators found adjec-tives like hard more imageable than we expected HoweverMetaphoria still outperforms other systems on specificityWe also consider the difference between the top and bot-

tom rankedMetaphoria suggestions Table 1 shows examplesTable 6 shows the percent of times a given systemsrsquo sugges-tions was marked as apt specific or imageable Top rankedsuggestions are more apt than bottom ranked ones (t = 249p-value = 001) which confirms H4 There is no significantdifference for specificity (t = -030 p-value = 076) Howevertop ranked suggestions are slightly more imageable thanbottom ranked suggestions (t = 209 p-value = 004) It couldbe that aptness makes it easier visualize the suggestion

This shows thatMetaphoria creates high qualitymetaphorsand can provide strong suggestions to writers

5 STUDY 2 NOVICE USERSThis study evaluates the quality of the suggestions Metapho-ria generates in the context of a specific writing task writingextended metaphors This allows us to test coherence tocontext as well as if Metaphoria supports divergent out-comes when writers are given the same list of suggestions

MethodologyWe recruited 16 undergraduates 8 female 8 male with anaverage age of 195 (σ 2 = 12) Each participant did a writingtask and a semi-structured interviewEach participant was asked write a metaphor that ex-

presses a connection between an abstract concept and con-crete object presented to them They are given the followingexample for the seed love is a stream

Love is something that just drags me along Likea stream it just takes me in whatever directionit is going

We present each participants with six seed metaphors Themetaphors are generated by combining a random word froma set of poetic themes (eg love) with a random word from aset of concrete nouns (eg stream) [8] Participants are askedto write about these seed metaphors one at a timendash3 withMetaphoria and 3 without All participants were given thesame seed metaphors in the following order

bull gratitude is a streambull peace is a kingbull jealousy is sandbull consciousness is a shadowbull loss is a wingbull friendship is snow

To counterbalance the experiment half the participantscould use Metaphoria with the first three metaphors andhalf use it with the last three metaphors Figure 3 shows howthe interface is presented in each caseAfter the participant completes the task the first author

conducts a semi-structured interview in which all partici-pants are asked the same set of core questions with follow-upquestions asked as specific issues come up During the inter-view the participant or interviewer could use the interfaceto go back and look at what the participant wrote or interactwith the suggestions again

In this study we are testing Metaphoria for coherence tocontext If the suggestions are not coherent participants willnot be able to use them to write coherent sentences which istheir goal Thus usage is a strong signal for coherence Wealso test for divergent outcomes by looking at the varietyof responses If Metaphoria does not support divergent out-comes metaphors written across participants will be moresimilar when using Metaphoria than not

ResultsCoherence to context 12 of 16 participants used the sugges-tions to the complete the task Although all participants weregiven the same suggestions in the same order they used avariety of different suggestions For instance given the seedmetaphor peace is a king P10 used the suggestion lsquopeace is for

Figure 3 Interface for constrained writing task in whichparticipantswrote extendedmetaphorswithout suggestions(a) and with suggestions (b) Figure includes responses fromP12 (a) and P10 (b)

leading the people like a kingrsquo while P6 used the suggestionlsquopeace is for rallying the troops like a kingrsquo

Some participants were inspired by multiple suggestionslike P1 who used two suggestions lsquofriendship is for beautifulvistas like snowrsquo and lsquofriendship often arrives with a stormlike snowrsquo to write the following metaphor

Friendship often breaks out from kindness It isa snow that often falls around christmas

Many participants were impressed by the quality of thesuggestions like P8 who said

ldquoI like lsquoyou can use gratitude to wash somethinglike a streamrsquo Thatrsquos something I wish I hadcome up with Thatrsquos creativerdquo

Several of these participants acknowledged that the qualityof the suggestions varied P3 said that although some of themetaphors didnrsquot make immediate sense they thought thatthe metaphors could make immediate sense to someone else

All participants were asked to choose one suggestion thatwas bad in some way and discuss why Most participantsspent some time rereading suggestions to select one Duringthis process several participants discovered that a suggestionthey previously thought did not make sense they could infact interpret P4 said

ldquoWith this one I was sort of a little confusedlsquopeace is for moving forward and backwardsin checkers like a kingrsquo I guess it makes sense

now that I say it out loud Itrsquos saying that peacedoesnrsquot have any limits on itrdquo

Of the 4 participants who did not use the suggestions 3said this was because the suggestions did not make senseThey often said the suggestions were too literal or simplynonsensical However P12 said the suggestions did makesense but she did not want to use them because she wantedto demonstrate that she could write creative metaphors onher own We come back to this in the Discussion section

Divergent outcomes The suggestions may be coherent butif participants end up writing very similar responses thenMetaphoria is not supporting divergent outcomes for writersWe report both quantitative and qualitative results

To quantitatively measure this we measure the variationof responses across all participants when they did or did notuse Metaphoria Here we define variation as the distributionof distances between all responsesndashhigh variation means allresponses were very different from all other responses Wemeasure distance as the Word Moverrsquos Distance betweentwo responsesThe responses without Metaphoria act as a baseline for

the variance we expect to see in the responses If participantswere staying close the suggestions as opposed to expandingor shifting the ideas we would expect there to be less vari-ation with Metaphoria Less variation means similar ideaswords and phrasing As a reminder all participants receivedthe same suggestions when they had access to Metaphoria

Our hypothesis is as follows

ndash H5 The variation in responses with Metaphoria is asleast as large as the variation in responses without

We compare the variation per seed metaphor with andwithout Metaphoria There is no significant difference inthe variation of the responses for 4 of the 6 seed metaphorsFor consciousness is a shadow there is significantly greatervariation with Metaphoria for jealousy is sand there is sig-nificantly greater variation withoutTable 7 shows examples from participants who said they

were inspired by the same suggestion demonstrating thewide range of directions participants took the idea as wellas examples of the more convergent responses

Qualitatively participants did not feel like the suggestionsboxed them in but rather inspired them to come up withnew ideas P4 expressed well how he would be inspired by asuggestion

ldquoI saw lsquogratitude is for bathing like a streamrsquo andthat made me think well how big is a streamIt started making me think about its sizerdquo

To demonstrate how far he took this idea here is his finalresponse to gratitude is a stream

lsquogratitude is for bathing like a streamrsquoP6 Like a stream you can bathe in gratitude and as the

stream cleans your body gratitude cleans your soulP13 A stream to me is rapid and powerful and has the

ability to sweep you away Gratitude offered by afriend or even a stranger is a stream in this way ithas the unexpected power to swell your heart withpositive emotions and completely sweep you away

lsquojealousy can irritate skin like sandrsquoP16 Jealousy is a sand It finds a way to irritate and con-

flict trouble of mind upon those whom it possessesP2 Jealousy can itch and irritate your mental behavior

similar to the sand that clings on to your clothesand feet

Table 7 Metaphoria mostly resulted in distinct re-sponses even when writers used the same suggestionas in the lsquogratitudersquo examples But sometimes sugges-tions resulted in very similar responses as in the lsquojeal-ousyrsquo example

Gratitude can be difficult to feel or to noticemuch like a stream that runs down the gutterof the road in a rainstorm And like all streamsit can easily run dryndashand you might not realizeitrsquos gone until itrsquos too late

We were worried that certain suggestions would be farmore coherent than others or that there would be a strong or-dering effect and therefore participants would always choosethe same suggestions and write similar responses Howeveras seen in the above analysis this was not the case Evenwhen participants chose the same response they would writeradically different things

6 STUDY 3 EXPERT WRITERSThis study evaluates if Metaphoria can adapt to a writerrsquosown goals and tests the system on inputs we did not ex-pect Our previous studies show Metaphoria is coherent tocontext and produces divergent outcomes now we tacklewhether these properties hold in real tasks which span awide range of writer intentions

MethodologyWe gave three professional poets a 15 minute tutorial ofMetaphoria and then asked them towrite a poem on a subjectof their own choosing using Metaphoria in some way Thepoets wrote for around 30 minutes each We then conducteda semi-structured interview and utilized having Metaphoriaavailable to discuss their process and response

In this study we gave participants access to the full inter-activity of Metaphoria they could enter in their own sourceconcept as well as a generate new vehicles which are drawnrandomly from a list of common poetic vehiclesThe poets were recruited through a mailing list for cur-

rent and past MFA in Creative Writing students at a localuniversity All had a regular writing practice were publishedpoets and one also held a teaching position in which theytaught poetry writing workshops to undergraduates

ResultsCoherence to context All poets used several of the sugges-tions in their poem Part of each poem is reproduced inTable 8 where words they input into Metaphoria are high-lighted in pink and phrases from the suggestions they usedare highlighted in greenThe context each poet brought to Metaphoria was very

different PO1 initially entered the word island the first lineof their poem was inspired by the suggestion lsquoisland can filla glass like winersquo though they first spent several minuteswith other suggestions like lsquoisland can travel over water likea shiprsquo and lsquoisland can age over time like winersquo PO2 wasinitially inspired by suggestions for the seed metaphor workis a garden where work was input during the tutorial severalwords in the first stanza came from the suggestions for thisseed Later they input the words swaying and she

PO3 brought a very different type of context They inputmany more words than the other two poets more interestedin finding interesting suggestions than crafting a poem witha particular direction almost every line derives from somepart of Metaphoria They first input sales then marketingbefore exploring the word metaphor Their first line is in-spired by the suggestion lsquometaphor is for restoring quietlike a bellrsquo Later they input words like time guns historyelections laughter and stone to mention only a small numberAll poets found suggestions that resonated with them

though they were discriminate and often searched throughseveral seeds before finding something they used Howeverthere were clearly different styles of use PO1 and PO2 com-posed poems with some kind of linear narrative or thoughtand used Metaphoria on words they had already written of-ten finding a suggestion that would finish the line they wereworking on In contrast PO3 input words they thought mightbe make for interesting metaphors or words they simplyoverheard (we met in a coffee shop) many of which nevermade it into the poem PO3rsquos use was more like collectinginteresting phrases which they then arranged and edited

Divergent outcomes The resulting poems were of dramati-cally different styles both due to each poetrsquos differing usageof Metaphoria and their different writing styles When ex-plicitly asked about the expressiveness of the system all

poets noted that established writers have their own style andthe system was unlikely to dramatically change it Both PO2and PO3 thought Metaphoria would increase the creativityof amateur poets who tend to get stuck in cliche languagethey thought the unexpectedness of the word combinationswas likely to help

However PO2 did bring up concerns of ownership Whilethey did not think that Metaphoria limited them they wereconcerned about using suggestions from Metaphoria thatwere too different from their intention even if these sugges-tions were very good PO3 used Metaphoria most liberallyyet had no such concerns They drew a comparison betweenMetaphoria and Instagram noting that while Instagram hasproduced a genre of photography that is very recognizableand thus the photos are somewhat similar it has also pro-duced unexpected and creative artworks They speculatedthat Metaphoria might create a genre of Metaphoria-style po-ems but would also allow poets to move in new and excitingdirections We analyze these concerns in the Discussion

7 DISCUSSIONOwnership concerns and cognitive models of usageOwnership is extremely important to writers It is essentialthat writers feel like they own their material andMetaphoriawas designed to augment writerrsquos abilities not replace themTo tackle this head on we asked all participants about howmuch ownership they felt for what they wrote Each poet inthe expert study discussed their relationship to Metaphoriausing a different cognitive model

PO1was unconcerned about the influence of the system ontheir writing they thought of Metaphoria ldquolike a calculatorfor wordsrdquo They used Metaphoria as a cognitive offload-ing tool outsourcing specific moments of word generationand allowing them to focus on other goals like the overalldirection of the poem and the flow of the lines

PO2 was concerned about using Metaphoria when it pro-duced particularly good images For example they thoughtthe line lsquoshe is used for currency and jewelryrsquo was ldquoan amaz-ing line of poetryrdquo but ldquodefinitely altered the direction ofthe poemrdquo which worried them In this case they treatedMetaphoria as a co-creative partnerwho contributed moreto the poem than PO2 felt comfortable with

PO3 used Metaphoria much more liberallyndashwith no partic-ular intended direction they were more playful and wantedto uncover interesting Metaphoria-style combinations Inthis case Metaphoria was used as a casual creator [4] an in-teractive system that encourages exploration in the creationor discovery of surprising new artifactsIn the novice study 4 of the 16 participants said that

they felt less ownership over the final results because someamount of work was being done by the system this reaction

PO1rsquos response PO2rsquos response PO3rsquos response

My island fills glasses like wine

irsquots vines wrap around my

new mouth like grapes

This new America is used to building things

anew strange comfort like the rest of an air-bed

at dusk

How new is new

Garden Work

with my mother her tulips flaming blueand yellow laboring to bloom beneathher palms the soft lawn grating againstearly spring We are wasting time lingeringunder the porch light before dark flirtingwith enemy weeds before my father returnshome drunk and swaying like a storm

She is used for currency and jewelryand lighting the pathway She is formaking flowers rise up to collidewith her hands

Metaphor for restoring quietUse a gun to paint a roomAddiction can clog a sink drain like hairHistory can win a warThe garden of wasted timeFear to extinguish a fire like sandice is for finding the source of lightswimming is like snow it is for childrenYou can use caution to build fear in a movieYou can use witchcraft to listen to music like an earCorruption can outrun you like a horse

Table 8 Part of responses from three professional poets working with Metaphoria Words highlighted in pinkwere input into Metaphoria by the poets while words and phrases highlighted in green were suggestions thatpoets used

was strongest in those that thought the suggestions wereparticularly good In this case likely they saw Metaphoria asa co-creative partner contributing too much to their work

Thus algorithmic suggestions are used differently depend-ing on the cognitive model users projectndasha offloading toolthat does grunt work (like a dictionary or thesaurus) a truepartner that can do too much or too little or a casual creatorthat allows the user to explore Systems designers shouldbe aware of different cognitive models and build tools thatsupport creators without threatening their agency

Design implications from ownership concernsAll participants in the novice and expert studies acknowl-edged that they happily accept prompts ideas feedback andedits from people (both teachers and peers) without feel-ing loss of ownership For machines to become acceptableco-creative partners there are two design avenues

Increased transparency can make the mechanisms ofthe machine more apparent This way it feels more like alsquoword calculatorrsquo than a system trying to outsmart you Pre-sentation of the suggestions maymatter more studies shouldbe done on how this affects perceived ownership It could bethat for some writers full sentences (even ones constructednaively from templates) are more threatening than a keydangling phrase

Increased interactivity integrates the person into thecreation process Themore interaction themore themachinecan be seen as a causal creator that helps explore new spacesThis interactionwith a computational system can give peoplecomfort and agency similar to howwe learn to converse withpeople offering us advice Systems could draw suggestionsfrom different contexts or genres that writer can pick orspecify such as a particular novel technical text or set of

tweets and include tunable parameters such as suggestionlength vocabulary sophistication connotative constraints(like negativepositive) or phonetic features

Limitations and future workInteraction with Metaphoria is limited to inputting a sourceword and requesting a new the vehicle word This does nottake into consideration what a writer has previously writteneither the text of whatever they are currently working onor past work that might be relevant To make systems morepersonalized we could highlight how suggestions relate to awriterrsquos previous work or phrase suggestions in a syntacticstyle specific to the writerAdditionally Metaphoria can be expanded to other do-

mains like journalism For example we can provide sugges-tions to metaphorically explain scientific concepts for laypeople ldquoCRISPR can cut genes like scissors can cut paperrdquo Wecan adapt the system by training a custom word embeddingto provide representations for words in specialized domainslike medical research technology or law

8 CONCLUSIONMotivated by past work on user-centric creativity supportwe created Metaphoria an interactive interface for generat-ingmetaphorical connections Our evaluations demonstratedthat Metaphoria generates suggestions coherent to contextand supports divergent outcomes for writers We discussownership and cognitive models in human-computer collab-oration and present future work for more interactive andtransparent systems that can further empower creators

ACKNOWLEDGEMENTSKaty IlonkaGero is supported by anNSFGRF (DGE - 1644869)

REFERENCES[1] Michael S Bernstein Greg Little Robert C Miller Bjoumlrn Hartmann

Mark S Ackerman David R Karger David Crowell and KatrinaPanovich 2010 Soylent A Word Processor with a Crowd InsideIn Proceedings of the 23Nd Annual ACM Symposium on User InterfaceSoftware and Technology (UIST rsquo10) ACM New York NY USA 313ndash322httpsdoiorg10114518660291866078

[2] Jill Burstein Beata Beigman Klebanov Norbert Elliot and Hillary Mol-loy 2016 A Left Turn Automated Feedback and Activity Generationfor Student Writers In Language Teaching Learning and Technology6ndash13 httpsdoiorg1021437LTLT2016-2

[3] Elizabeth Clark Anne Spencer Ross Chenhao Tan Yangfeng Ji andNoah A Smith 2018 Creative Writing with a Machine in the LoopCase Studies on Slogans and Stories In 23rd International Conference onIntelligent User Interfaces (IUI rsquo18) ACM New York NY USA 329ndash340httpsdoiorg10114531729443172983

[4] Kate Compton and Michael Mateas 2015 Casual Creators In ICCC228ndash235

[5] Nicholas Davis 2013 Human-computer co-creativity Blending hu-man and computational creativity In Ninth Artificial Intelligence andInteractive Digital Entertainment Conference

[6] Nicholas Davis Chih-Pin Hsiao Kunwar Yashraj Singh and BrianMagerko 2016 Co-creative drawing agent with object recognitionIn Twelfth Artificial Intelligence and Interactive Digital EntertainmentConference

[7] Gilles Fauconnier andMark Turner 2008 The way we think Conceptualblending and the mindrsquos hidden complexities Basic Books

[8] Andrea Gagliano Emily Paul Kyle Booten and Marti A Hearst 2016IntersectingWord Vectors to Take Figurative Language to NewHeightsIn Proceedings of the Fifth Workshop on Computational Linguistics forLiterature 20ndash31

[9] Dedre Gentner 1983 Structure-Mapping A Theoretical Frameworkfor Analogy Cognitive Science 7 2 (1983) 155ndash170 httpsdoiorg101207s15516709cog0702_3

[10] Katy Gero and Lydia Chilton 2018 Challenges in FindingMetaphoricalConnections In Proceedings of the Workshop on Figurative LanguageProcessing 1ndash6

[11] Marjan Ghazvininejad Xing Shi Yejin Choi and Kevin Knight 2016Generating topical poetry In Proceedings of the 2016 Conference onEmpirical Methods in Natural Language Processing 1183ndash1191

[12] Karni Gilon Joel Chan Felicia Y Ng Hila Liifshitz-Assaf Aniket Kitturand Dafna Shahaf 2018 Analogy Mining for Specific Design Needs InProceedings of the 2018 CHI Conference on Human Factors in ComputingSystems (CHI rsquo18) ACM New York NY USA Article 121 11 pageshttpsdoiorg10114531735743173695

[13] Sam Glucksberg Matthew S McGlone Yosef Grodzinsky and KatrinAmunts 2001 Understanding figurative language From metaphor toidioms Number 36 Oxford University Press on Demand

[14] Eric Horvitz 1999 Principles of Mixed-initiative User Interfaces InProceedings of the SIGCHI Conference on Human Factors in ComputingSystems (CHI rsquo99) ACM New York NY USA 159ndash166 httpsdoiorg101145302979303030

[15] Julie S Hui Darren Gergle and Elizabeth M Gerber 2018 IntroAssistA Tool to Support Writing Introductory Help Requests In Proceedingsof the 2018 CHI Conference on Human Factors in Computing Systems(CHI rsquo18) ACM New York NY USA Article 22 13 pages httpsdoiorg10114531735743173596

[16] Jennifer Jacobs Joel Brandt Radomiacuter Mech and Mitchel Resnick 2018Extending Manual Drawing Practices with Artist-Centric Program-ming Tools In Proceedings of the 2018 CHI Conference on Human Factorsin Computing Systems (CHI rsquo18) ACM New York NY USA Article

590 13 pages httpsdoiorg10114531735743174164[17] Joy Kim Justin Cheng and Michael S Bernstein 2014 Ensemble Ex-

ploring Complementary Strengths of Leaders and Crowds in CreativeCollaboration In Proceedings of the 17th ACM Conference on ComputerSupported Cooperative Work amp38 Social Computing (CSCW rsquo14) ACMNewYork NY USA 745ndash755 httpsdoiorg10114525316022531638

[18] Matt Kusner Yu Sun Nicholas Kolkin and Kilian Weinberger 2015From word embeddings to document distances In International Con-ference on Machine Learning 957ndash966

[19] George Lakoff and Mark Turner 2009 More than cool reason A fieldguide to poetic metaphor University of Chicago Press

[20] Claudia LeacockMartin ChodorowMichael Gamon and Joel Tetreault2010 Automated grammatical error detection for language learnersSynthesis lectures on human language technologies 3 1 (2010) 1ndash134

[21] Omer Levy and Yoav Goldberg 2014 Dependency-based word embed-dings In Proceedings of the 52nd Annual Meeting of the Association forComputational Linguistics (Volume 2 Short Papers) Vol 2 302ndash308

[22] Zach Lieberman T Watson and A Castro 2015 openFrameworkshttpopenframeworksccabout Accessed 2018-09-19

[23] Tal Linzen 2016 Issues in evaluating semantic spaces using wordanalogies CoRR abs160607736 (2016) arXiv160607736 httparxivorgabs160607736

[24] H Liu and P Singh 2004 ConceptNet mdash A Practical CommonsenseReasoning Tool-Kit BT Technology Journal 22 4 (01 Oct 2004) 211ndash226httpsdoiorg101023BBTTJ0000047600454216d

[25] Neil Maiden Konstantinos Zachos Amanda Brown George BrockLars Nyre Aleksander Nygaringrd Tonheim Dimitris Apsotolou andJeremy Evans 2018 Making the News Digital Creativity Supportfor Journalists In Proceedings of the 2018 CHI Conference on HumanFactors in Computing Systems (CHI rsquo18) ACM New York NY USAArticle 475 11 pages httpsdoiorg10114531735743174049

[26] Enrique Manjavacas Folgert Karsdorp Ben Burtenshaw and MikeKestemont 2017 Synthetic literature Writing science fiction in aco-creative process In Proceedings of the Workshop on ComputationalCreativity in Natural Language Generation (CC-NLG 2017) 29ndash37

[27] Tomas Mikolov Ilya Sutskever Kai Chen Greg S Corrado and JeffDean 2013 Distributed Representations of Words and Phrases andtheir Compositionality In Advances in Neural Information ProcessingSystems 26 C J C Burges L Bottou M Welling Z Ghahramani andK Q Weinberger (Eds) Curran Associates Inc 3111ndash3119

[28] Jeffery Scott Mio 1997 Metaphor and Politics Metaphor and Symbol12 2 (1997) 113ndash133 httpsdoiorg101207s15327868ms1202_2arXivhttpsdoiorg101207s15327868ms1202_2

[29] Kai Niebert Sabine Marsch and David F Treagust 2012 Under-standing needs embodiment A theory-guided reanalysis of the roleof metaphors and analogies in understanding science Science Ed-ucation 96 5 (2012) 849ndash877 httpsdoiorg101002sce21026arXivhttpsonlinelibrarywileycomdoipdf101002sce21026

[30] Changhoon Oh Jungwoo Song Jinhan Choi Seonghyeon Kim Sung-woo Lee and Bongwon Suh 2018 I Lead You Help but Only withEnough Details Understanding User Experience of Co-Creation withArtificial Intelligence In Proceedings of the 2018 CHI Conference onHuman Factors in Computing Systems (CHI rsquo18) ACM New York NYUSA Article 649 13 pages httpsdoiorg10114531735743174223

[31] Hugo Gonccedilalo Oliveira 2012 PoeTryMe a versatile platform forpoetry generation Computational Creativity Concept Invention andGeneral Intelligence 1 (2012) 21

[32] Jeffrey Pennington Richard Socher and Christopher Manning 2014Glove Global vectors for word representation In Proceedings of the2014 conference on empirical methods in natural language processing(EMNLP) 1532ndash1543

[33] James L Peterson 1980 Computer Programs for Detecting and Cor-recting Spelling Errors Commun ACM 23 12 (Dec 1980) 676ndash687httpsdoiorg101145359038359041

[34] Casey Reas and Ben Fry 2004 Processing httpprocessingorgAccessed 2018-09-19

[35] Mitchel Resnick Brad Myers Kumiyo Nakakoji Ben ShneidermanRandy Pausch Ted Selker and Mike Eisenberg 2005 Design principlesfor tools to support creative thinking In NSF Workshop Report onCreativity Support Tools Citeseer 25ndash36

[36] Melissa Roemmele andAndrew S Gordon 2018 Automated Assistancefor Creative Writing with an RNN Language Model In Proceedings ofthe 23rd International Conference on Intelligent User Interfaces Compan-ion (IUI rsquo18 Companion) ACM New York NY USA Article 21 2 pageshttpsdoiorg10114531803083180329

[37] Paul J Silvia and Roger E Beaty 2012 Making creative metaphorsThe importance of fluid intelligence for creative thought Intelligence40 4 (2012) 343 ndash 351 httpsdoiorg101016jintell201202005

[38] Robin Sloan 2016 Writing with the machine httpswwwrobinsloancomnoteswriting-with-the-machine Accessed 2018-09-19

[39] Jaime Teevan Shamsi T Iqbal and Curtis von Veh 2016 Support-ing Collaborative Writing with Microtasks In Proceedings of the 2016CHI Conference on Human Factors in Computing Systems (CHI rsquo16)ACM NewYork NY USA 2657ndash2668 httpsdoiorg10114528580362858108

[40] Tony Veale [n d] Thesaurus Rex httpngramsucdietherex3Accessed 2018-09-19

[41] Tony Veale 2013 Less Rhyme More Reason Knowledge-based PoetryGeneration with Feeling Insight and Wit In ICCC 152ndash159

[42] Tony Veale and Yanfen Hao 2007 Comprehending and generating aptmetaphors a web-driven case-based approach to figurative languageIn AAAI Vol 2007 1471ndash1476

[43] Tony Veale and Guofu Li 2016 Distributed Divergent CreativityComputational Creative Agents at Web Scale Cognitive Computation8 2 (01 Apr 2016) 175ndash186 httpsdoiorg101007s12559-015-9337-9

[44] Tony Veale Ekaterina Shutova and Beata Beigman Klebanov 2016Metaphor A computational perspective Synthesis Lectures on HumanLanguage Technologies 9 1 (2016) 1ndash160

[45] GeWang 2008 The ChucK Audio Programming Language An Strongly-timed and On-the-fly Environmentality PhD Dissertation PrincetonUniversity

[46] Amy Beth Warriner Victor Kuperman and Marc Brysbaert 2013Norms of valence arousal and dominance for 13915 English lemmasBehavior Research Methods 45 4 (01 Dec 2013) 1191ndash1207 httpsdoiorg103758s13428-012-0314-x

Abstract

1 Introduction

2 Related Work

Writing support

Creativity support and co-creativity

Metaphor generation algorithms

3 Design of Metaphoria

Design Goals

Generating coherent connections

Selecting multiple distinct connections

Additional coherence with valence ranking

Additional distinctness with suggestion expansion

Interactivity

4 Study 1 Suggestion Quality

Methodology

Results

5 Study 2 Novice Users

Methodology

Results

6 Study 3 Expert Writers

Methodology

Results

7 Discussion

Ownership concerns and cognitive models of usage

Design implications from ownership concerns

Limitations and future work

8 Conclusion

References