Iconic prosody in story reading - umu.diva-portal.orgumu.diva-portal.org/smash/get/diva2:759405/FULLTEXT02.pdf · Iconic prosody in story reading ... Bryant and Fox Tree, 2002; Cosmides,

ICONICPROSODY 1

Iconicprosodyinstoryreading

MarcusPerlman

UniversityofWisconsin‐Madison

DepartmentofPsychology

NathanielClark

UniversityofCalifornia‐SantaCruz

PsychologyDepartment

MarleneJohanssonFalck

UmeåUniversity

DepartmentofLanguageStudies

Keywords:prosody;vocalgesture;speechproduction;iconicity

Marcus Perlman

1202 E. Johnson St.

University of Wisconsin, Madison

Madison, WI 53706

Email: [email protected]

ICONICPROSODY 2

Abstract

Recentexperimentshaveshownthatpeopleiconicallymodulatetheirprosody

correspondingwiththeirutterance’ssemanticmeaning(e.g.Shintel,etal.,2006).This

articlereportsfindingsfromastoryreadingtaskthatexpandstheinvestigationoficonic

prosodytoabstractmeaningsinadditiontoconcreteones.Participantsreadstoriesthat

contrastedalongconcreteandabstractsemanticdimensionsofspeed(e.g.afastdrive,slow

careerprogress)andsize(e.g.asmallgrasshopper,animportantcontract).Participants

readfaststoriesatafasterratethanslowstories,andbigstorieswithalowerpitchthan

smallstories.Theeffectofspeedwasdistributedacrossthestories,includinginportions

thatwereidenticalacrossstories,whereasthesizeeffectwaslocalizedtosize‐related

words.Overall,thesefindingsenrichthedocumentationoficonicityinspokenlanguage,

andbearonourunderstandingoftherelationshipbetweengestureandspeech.

ICONICPROSODY 3

1.Introduction

Aspeopletalk,theycommonlyusetheirhandstoproduceiconicgesturesthatare

relatedtothemeaningtheyareexpressingverbally(Kendon,2004;McNeill,1992).

Typically,theiconicformsofthesegesturesareunderstoodtoreflectsomemore‐or‐less

detailedaspectofthesensorimotorimagerythatisassociatedwiththemeaningofthe

spokenutterance.Forexample,considertheutteranceofmycologistDavidArora,inwhich

hedescribesthe“longnicestem”ofaparticularvarietyofmushroom

(http://www.youtube.com/watch?v=VSn3aGGzG1M).Asthespeakerarticulatestheword

“long”,hesimultaneouslydepictsthestemwithhishands,curlingthefingersofhisright

handandtracingthestem’slengthdownwardfromhisraisedleftpalm.

Notably,inthisexample,Aroraalsoappearstodepicttheimageofthelongstem

throughthedynamic“shape”ofhisvoice.Incoordinationwiththemanualgesture,he

salientlyextendsthespokendurationoftheadjective“long”,stretchingthewordiniconic

reflectionofthestem’slength.Thisobservationsuggeststhattheconceptualizationof

length—inthiscaseinstantiatedinthecontextofamushroomstem—canbeiconically

realizednotjustinco‐speechmanualgestures,butalsoiniconicmodulationsofthe

temporalpatterningofspeech.

Recentexperimentalstudiesshowthattheuseoficonicprosodyisnotjustlimited

toisolatedinstancesinassociationwithmanualiconicgestures.Forexample,inthelab,

Englishspeakershavebeenshowntoreliablyproduceiconicmodulationsoftheirspeech

ratewhendescribingthespeedofanevent,andintheirpitchwhendescribingthevertical

ICONICPROSODY 4

movementofanobject(Perlman,2010;Shintel,Nusbaum,&Okrent,2006).Thesestudies

showamotivatedrelationshipbetweenthemeaningthatpeopleareexpressingandthe

prosodicformsthattheyproduceintheirspeech.Specifically,participantstendtotalk

fasterwhendescribingafasteventandslowerwhendescribingaslowevent.Theytalk

withahigherpitchwhenreferringtoanobjectthatismovingupwardandalowerpitch

whenreferringtoonethatismovingdownward.Inthispaper,werefergenerallytosuch

motivatedcorrespondencesbetweentheprosodicformofanutteranceanditsmeaningas

iconic,andnotethaticoniccorrespondencesmaybevariablydetailed,abstract,schematic,

cross‐modal,metaphoric,metonymic,etc.(Similargeneralusageisrecognizedinthe

manualgestureliterature,e.g.,Cienki&Müller,2008;McNeill,2005)

Todate,researchoniconicprosodyislargelyexploratory.Ithasbeeninvestigated

injustasmallsetofsemanticdomains,withtheapplicationofonlyafewmethodsand

analyses.Remarkablylittleisknownabouthowspeakersuseiconicprosodyinthewild.

Yetthisresearchissignificantasitcontributestothegrowingdocumentationofthe

prevalenceoficonicityinspokenlanguages,inadditiontosigned.Inlanguagesacrossthe

world,iconicityisincreasinglyreportedinphenomenalikephonologicalorsound

symbolism,phonesthemes,onomatopoeia,andideophones(Dingemanse,2012;Nuckolls,

1996;Perniss,Thompson,&Vigliocco,2010).Linguistsandanthropologistsdescribe

substantialiconiclexiconsfromBantulanguagesinAfrica(Childs,1994),non‐Pama‐

NyunganAustralianAboriginallanguages(Alpher,2001;McGregor,2001;Schultze‐Berndt,

2001),Japanese,KoreanandSoutheastAsianlanguages(Diffloth,1972;Watson,2001),

QuechalanguagesofSouthAmerica(Nuckolls,1996),andBalto‐Finniclanguages(Mikone,

2001).Moreover,theiconicityintheseconventionalspokenformsrelatestoawiderange

ICONICPROSODY 5

ofmeanings,including,forexample,shape,mannerofmotion,texture,size,brightness,

distance,psychologicalandmentalstates,andtemporalaspect.Theserichiconiclexicons

pointtomoredynamicspokeniconicgestures,suchasiconicprosody,intheprocessesof

theirformation(Perlman,Dale,&Lupyan,2014).

Thepresentstudybuildsonpreviousresearchoniconicprosodyintwoprimary

ways.First,itprovidesfurtherevidenceofspeedrelatedmodulationsofspeechratewithin

amorenaturalisticspeakingtaskthanthatusedbyShintelandcolleagues(2006).Second,

itextendstheinvestigationoficonicprosodytothesemanticdomainofsize.Inboththe

speedandsizedomains,ourresultscontributefinerdetailsrelatedtothetemporal

patterningoficonicprosodyandthekindsofconcreteandabstractmeaningsthatmay

elicitit.

1.1.Iconicityinprosody

Mostpreviousresearchonprosodyinspeechproductionhasnotexaminedhow

variablesassociateddirectlywiththesemanticmeaningofanutterancemightinfluenceits

prosodicform.Althoughboundariesbetweensemanticsandpragmaticsmaybe

contentious,welimitouruseof“semantic”heretomeaningsthatareovertlyexpressedby

thewordsandphrasesofanutterance,suchasmannerofmotion(e.g.fast/slow),theshape

ofanentity(e.g.big/small),oranentity’sspatialpositionanddirectionofmovement(e.g.

up/down).Incontrasttothesekindsofsemanticdistinctions,pastresearchhasgenerally

focusedonquestionsrelatedtohowprosodydemarcatesthesyntacticstructureofan

utterance(e.g.,Ferreira,1993),howitdirectsattentionthroughstressandpitchaccenton

ICONICPROSODY 6

focusedelements(e.g.,Levelt,Roelofs,&Meyers,1999),andhowitexpresses

paralinguisticinformationaboutaspeaker’sinternalstateslikeemotionandattitude(e.g.,

Bolinger,1986;BryantandFoxTree,2002;Cosmides,1983).Overall,thislargebodyof

researchshowsthattheprosodicformofanutteranceissignificantlydeterminedand

constrainedbyamultitudeoffactors,includingthesyntaxandphonologyoftheutterance,

aswellasparalinguisticfactorslikethespeaker’sattention,emotion,andattitude.For

instance,Levelt(1989:180)summarizesthatprosodyisdeterminedbytheinteractionsof

thephrasalorganizationofwordstodetermineintonationalunitboundaries,themoodand

modalityoftheutterancetodetermineintonationalpatternsandboundarytones,andthe

assignmentofpitchaccenttofocusedelements.Accordingtothisview,theprocessesthat

determineprosodyarenotdirectlyinfluencedbytheconceptualizationoftheutterance.

However,somerecentresultsshowthat,undersomeconditions,prosodyisalso

measurablyinfluencedbyvariablesrelatingtothesemanticmeaningoftheutterance.One

pioneeringstudyrecordedparticipantsastheydescribedthedirectionofmotionofan

animateddotonacomputerscreen,usingthecarrierphrase,“Thedotismoving

[left/right/up/down]”(Shinteletal.,2006).Inafirstexperiment,participantswerefound

toincreaseordecreasetheirfundamentalfrequencyastheydescribedanupwardor

downwardmovingdot,respectively.Inasecondexperiment,thedotmovedtotheleftor

right,whilealsotravelingateitherafastoraslowrate.Participantsspokefasterwhen

describingtheleftwardorrightwardmovementoffastmovingdotscomparedtoslow

ones,eventhoughthespeedofthedotwasincidentaltothecommunicativetask.

Anotherstudyinvestigatedwhetheradultsspeakingininfantdirectedspeech(IDS)

wouldmodulatetheprosodyoftheirvoiceastheyspokesentencesrelatingtothe

ICONICPROSODY 7

antonymicpairsofmeaningshappy/sad,hot/cold,big/small,tall/short,yummy/yucky,

andstrong/weak(Nygaard,Herold,&Namy,2009).Threespeakerswereshownanonce

word,like“foppick”or“tillen,”foreachmeaning,andtheninstructedtosay,“Canyouget

the[nonceword]one?”asiftoaninfant.Analysisrevealedconsistentdifferencesalong

differentcombinationsofacousticparameterswithineachpairofantonyms.Forexample,

themeaningbigwascharacterizedbylowerpitch,longerduration,andhigherintensity

thansmall,whiletallwascharacterizedbyalongerdurationandgreaterpitchvariation

thanshort.Theseeffectsweregenerallyfoundatthelevelsofbothwordandsentence.

Speakersalsoproducesemanticallymotivatedmodulationsoftheirprosodyinmore

spontaneouscontexts.Perlman(2010)askedparticipantstowatchaseriesofshortvideo

clipsshowingfastorslow‐pacedeventsandthendescribethemopen‐endedlytoan

experimenter.Onaverage,speakersspokefasteracrosstheircompletedescriptionsoffast

eventscomparedtoslowevents.Inadditiontotheseoveralldifferences,theyalso

produceddistinctmodulationsintempowhenarticulatingadverbialphrasesaboutspeed,

suchasreallyfastorveryslowly.

Adifferentflavorofexperimentaskedparticipantstoproducethevowel/a/ina

go/no‐gotaskinresponsetostimulithatvariedalongthreedimensions:shape(trianglevs.

dodecagon),luminance(whitevs.black),andsize(smallvs.large;PariseandPavani,

2011).Thestudyfoundthatparticipantspronouncedthesyllableindifferentways

accordingtothestimuli,articulatingitwithhigherintensitywhenrespondingto

dodecagonscomparedtotrianglesandwhiteshapescomparedtoblackones,andwitha

higherthirdformantfortrianglescomparedtododecagons.Size,however,wasnotfound

tohaveaneffectonarticulation.

ICONICPROSODY 8

Takentogether,theresultsofthesestudiesshowthatpeoplehaveatendencyto

modulatetheirprosodyiniconiccorrespondencewithcertainmeanings.Yet,ourempirical

knowledgeoficonicprosodyremainsextremelylimited.Forexample,weknowlittleabout

thekindsofmeaningsthatarelikelytoinfluenceprosody.Wealsoknowlittleabouthow

iconicprosodymanifestsinanutterance,includingtheacousticcharacteristicsthatare

mostrelevant,aswellasitstemporalpatterning.

Gesturescholarshaveproposedthatonereasonforthisempiricalgapmaybe

relatedtothemethodologicalchallengeofstudyingiconicprosody(Duncan,2003;McNeill,

2005).Whereasmanualgesturesaredistinctlyidentifiableascommunicativemovements

(Kendon,2004),prosodyisproducedwithinthearticulatorymovementsofconventional

speechforms.Theprosodicformisfurtherdeterminedbyahostofotherfactorssuchas

syntaxandthespeaker’semotional,attitudinalandattentionalstate.Asaconsequence,it

canbechallengingtoidentifyandmeasureiconicprosodyandseparateitfromtheseother

variables.

2.Presentstudy

Towardsovercomingthischallenge,thepresentarticlereportstheresultsfroman

experimentthatusedastoryreadingtasktoinvestigateiconicprosodyinthesemantic

domainsofspeedandsize.Participantswererecordedastheyreadaloudaseriesofshort

storiestoapartner.Onesetofstoriesinvolvedeitherfastorslowspeedofmovement,and

theother,smallorlargesize.Wealsoexaminedwhetherabstractmeaningsmightelicit

iconicprosody,includingstorieswithineachsetthatinvolvedabstractinstantiationsofthe

ICONICPROSODY 9

targetedsemanticdomain(e.g.slowprogressorabigcontract).Wegenerallyexpected

thatparticipantswouldexpressthespeedofthestoriesthroughtherateofspeech,

whereassizewouldbeexpressedthroughitsintonationalpattern.Inparticular,

participantswouldspeakwithafasterarticulationrateinfaststoriescomparedtoslow

ones,andwithahigherpitchinsmallstoriescomparedtobigones.Thetaskalsoallowed

ustoexaminemorepreciselythedynamicsofhowtheconceptsofspeedandsizemightbe

manifestedinthetemporalandintonationalpatternsofspeech,forexample,whether

iconicallyfasterorhigher‐pitchedspeechoccursonlyintheimmediatecontextofwords

like‘fast’or‘small,’orwhetheritisdispersedmorewidelythroughoutanutterance.

Ourpredictionsregardingspeedandarticulationratewerebasedonpriorresearch

showingthatspeakerssometimesproducesloworfastmodulationsofarticulationrate

whendescribing,respectively,asloworfastevent(Perlman,2010;Shinteletal.,2006).

Priorresearchalsomotivatedourpredictionsregardinganiconicrelationshipbetween

sizeandpitch,asinphenomenalikesize‐soundsymbolism(Jacobson&Waugh,1979;

Ohala,1984;Tsur,2006;butseeParise&Pavani,2011).Peopleroutinelyassociatehigh‐

pitchedsoundswithsmallsizeandlow‐pitchedsoundswithlargesize.Inonewell‐

documentedcase,frontvowelswithhighsecondformants(e.g./i/)areassociatedwith

small,whilebackvowelswithlowsecondformants(e.g./ɔ/)areassociatedwithlarge(e.g.

Sapir,1929;andseeUltan,1978fortheprevalenceofthisassociationacrossthelexiconsof

variouslanguages).Additionally,experimentalresultsshowthatadultsspontaneously

modulatetheirpitchasaniconicexpressionofsizewhenreadingsentencesininfant

directedspeech(Nygaardetal.,2009).

ICONICPROSODY 10

2.1.Method

2.1.1.Participants

Sixty‐sevenundergraduatestudents(35forspeed,32forsize)attendingthe

UniversityofCalifornia,SantaCruzparticipatedinthestudyinexchangeforcoursecredit.

Allparticipantswereself‐reportednativespeakersofEnglish.

2.1.2.Stimulianddesign

Eightcarefullymatchedpairsofshortstoriesinstantiatingsemanticcontrastsof

speed(fastvs.slow)andsize(bigvs.small)werecreatedandformattedintoMicrosoft

PowerPointpresentations(SeeTable1).Thefirstslideofeachstorypresentedan

identifyingcharacterandtitle,andthestory’scontentwascontainedinthreesubsequent

slidesthatincludedanintroduction,aplot,andaconclusion.Onlytheintroductionand

plotcontainedwordsexplicitlyreferringtospeedorsize;theconclusionswereidentical

betweenpairedstories.Thusthestoriescontainedasetofcontrastingphrasesthatdiffered

betweenconditions,andtheremainingnon‐contrasting,sharedcontentthatwasidentical

betweenconditions.

Inadditiontothemainsemanticcontrast,thetargetstoriesalsovariedinmapping

type.Twoofthefourstoriesforeachdomainwereconcrete,concerningobservableevents

andphysicalobjects.Forspeed,thismeantrealmovementtakingplaceatameasurable

rate,suchasfastrunningversusslowwalking,andforsize,thismeantlargeorsmall

physicalobjects,likehouses.Theothertwostoriesineachdomainwereabstractand

concernedmoreabstractmeaningsdescribedintermsofspeedandsize,suchascareer

ICONICPROSODY 11

progress(e.g.“afasttracktosuccess”)orthesignificanceofanevent(e.g.,“areallybig

deal”).

Table1:Examplestorymaterialsforspeedcontrast

Fast Slow Syl. Sectionlabelandinfo

Martha Martha Character‐‐notanalyzed

ARapidRun ASluggishWalk Title‐‐notanalyzed

Marthaisafastrunner(4).Shealwaysexercises,andtodaysheisoutthere.

Marthaisaslowwalker(4).Sherarelyexercises,buttodaysheisoutthere.

21 Intro‐‐analyzed,instantiatescontrast,italictextanalyzedforcontrastingphrases

Shetakesoffquickly(4)throughtheneighborhood.Shedashespastsomehouses(7)andcontinuestowardsaschool.Shespeedstoanear‐sprint(5)andreachesthefinalstretch.

Shelaborsslowly(4)throughtheneighborhood.Shestrugglespastsomehouses(7)andcontinuestowardsaschool.Sheslowstoanearstop(5)butreachesthefinalstretch.

37 Plot‐‐analyzed,instantiatescontrast,italictextanalyzedforcontrastingphrases

Sheisreachingtheend. Sheisreachingtheend. 6 Conclusion‐‐analyzed,doesnotinstantiatecontrast

Fivefillerstorieswerealsocreatedforeachsemanticdomain.Thefillersparalleled

thetargetstoriesinstructure,butwereunrelatedtospeedorsizeandalsodifferedin

meaningfromeachother.Thetargetandfillerstorieswerecompiledtogetherinto

PowerPointpresentationsoffourcounterbalancedlistseachforthesizeandspeed

domains.Eachlistcontainedfourtargetstoriesrepresentingthefactorialcombinationof

semanticvalence(eitherfastvs.sloworbigvs.small)andconcreteness(concretevs

abstract).Thesewereinterspersedamongthedomain’sfivefillers,foratotalofnine

storiesperlist.Afulllistofstimuliappearsintheappendix.TableA‐1displaysstories

involvingspeed,andTableA‐2displaysstoriesinvolvingsize.

2.1.3.Procedure

ICONICPROSODY 12

Subjectsparticipatedinthestudyinpairs,butwerescheduledindependentlyand

didnotknoweachother.Onememberofthepairwasassignedtoalistcontainingspeed

stories,andtheothertoalistforsize.Participantsreadtheirrespectivelistofstoriesto

theirpartnerslide‐by‐slideviaaself‐pacedPowerPointpresentation,whichwasdisplayed

tothereaderonalaptopcomputer.Inthreeinstances,onlyoneparticipantarrivedfora

scheduledsession.Inthesecases,theparticipantreadfromoneofthespeedliststoan

undergraduateconfederate.

Tomakethetaskmorecommunicativeandmeaningful,participantswereinstructed

“tobeaninterestingstoryteller,”andweretoldthattheywouldbeansweringquestions

aboutthestoriestheirpartnerreadtothem.Tohelpthereadergainfamiliaritywiththe

storyandengagewithitsmeaning,eachstorywasrepeatedtwiceconsecutivelyinthe

presentation.Thefirsttimethroughparticipantswereinstructedtoreadthestorysilently,

andthesecondtimetoreaditaloudtotheirpartner.Afteronepartnerreadthroughtheir

listofstories,thepairswitchedroles,andthenewreaderreadtheirstoriesfromtheother

domain.Participantsconcludedthesessionbyfillingoutapost‐experimentquestionnaire.

Intotal,atypicalsessionlastedlessthan25minutes.

2.1.4.Analysis

Thereadingswererecordedwithaflatboundarymicrophonethatwassetonthe

tableinfrontofthereaderandconnectedtoadigitalrecorder.Therecordingswere

analyzedusingPraatphoneticanalysissoftware(Boersma,2001).

Foreachstory,theboundariesofsectionsandcontrastingtargetphraseswere

markedinaPraatTextGrid.Disfluencies,suchasfalsestartsandrepetitions,weremarked

ICONICPROSODY 13

andlaterexcisedfromtheanalysis.Therecordedreadings(.wavfile)andassociated

TextGridannotationswerefedintoaPraatscriptthatcomputedthedurationsandpitches

ofeachmarkedinterval,includingbothcontrastingandsharedphrases.Articulationrates

werecalculatedbydividingthetotalnumberofsyllablesoftherelevantportionsbythe

totaldurationofthoseportions(seeAppendixtablesA‐1andA‐2forsyllablecountsof

storysections).

Pitchforthecontrastingintervalsofeachstorywascomputedastheduration‐

weightedaveragepitchofallcontrastingphrases;pitchforthesharedelementswasa

duration‐weightedaverageofthepitchesoftheremaining,sharedintervals.Duration‐

weightedaverageswerecomputedbymultiplyingeachphrase’smeanpitchbyitsduration,

summingtheseproducts,anddividingthesumbythetotaldurationofthephrases.In

Praat,theautocorrelationalgorithmforF0measurementsissensitivetosourcesofnoise,

likeaccidentaldoublingorhalvingofthefundamentalfrequencyduringmodalvoicing,and

inaccuratemeasurementsofamodalvoicing(likecreakyvoicingorwhispering,whichare

inherentlynoisier).DuetothesedifficultiesofaccuratelymeasuringF0,eachparticipant’s

recordingwasanalyzedtwice.Thefirstpassofthepitchtrackerusedthedefaultsettings

ontherawrecordingsfromeachparticipant.Thisservedtoprovideabaselinepitchfor

eachparticipant,basedontheiraveragepitchacrossthewholetask,whichwasusedinthe

secondpassofthepitchtrackertorefinethesettingsformoreaccuratetracking.

Forthesecondpass,severaladjustmentsweremadetotherecordingandtothe

defaultsettingsofPraat’spitchtrackingalgorithm.First,eachparticipant’srecordingwas

low‐passfiltered,withtheupperboundofthepassbandsetateightsemitones(2/3

octave)abovethespeaker’saveragepitchacrossthewholerecording.Thisfilter

ICONICPROSODY 14

guaranteesanabsenceofdoublingfortheupperoctaveofthespeakers’constrainedvocal

range,startingat4semitonesbelowtheirbaselinepitch.Thefilteralsoremovesformants,

reducingthepotentialinfluenceofsupralaryngealarticulationontrackedF0.Further,to

excludeamodal(breathy,creaky)voicingandreducepitchhalving,theSilenceThresholdof

Praat’spitchtrackingalgorithmwassetto0.075(fromadefaultof0.03)andthefloorof

thepitchtrackerwindowwassetto8semitonesbelowthespeaker’spreviously

determinedbaseline(Boersma,1993).

2.2.Resultsanddiscussion

2.2.1.Speedstories

Thedatafromoneparticipantwasdiscardedbecauseofexcessivedisfluenciesand

failuretoreadaportionofastory,leaving34readers.Tables2and3showsummary

statisticsforarticulationratesandpitchesofthesestories.

Table2:Summarystatisticsfromarticulationratedata(syl/s)fromspeedstories

StoryType

Condition Sections Overall Concretestories Abstractstories

Overall 5.00(0.20) 4.79(0.22) 5.20(0.29)

Fast‐‐Mean(SD) Sharedphrases 4.79(0.30) 4.63(0.36) 4.95(0.36)

Contrastingphrases 5.20(0.44) 4.95(0.47) 5.45(0.56)

Overall 4.86(0.20) 4.58(0.25) 5.13(0.33)

Slow‐‐Mean(SD) Sharedphrases 4.67(0.20) 4.37(0.26) 4.97(0.27)


Note:n=34participants.

First,athree‐wayrepeatedmeasuresANOVAwasperformed,witharticulationrate

asthedependentvariableandsemanticvalence(itemsfromfastorslowstories),

concreteness(itemsfromconcreteorabstractstories),andcontrast(contrastingorshared

ICONICPROSODY 15

storyintervals)asfixedwithin‐subjectfactors.Thetestrevealedasignificantmaineffect

forspeed,F(1,32)=4.24,p=.048,partialη2=.11,butnointeractionsofspeedwith

mappingorcontrast.Onaverage,participants’articulationrateswerefasterwhenreading

thefaststoriescomparedtotheslowones.

Table3:Summarystatisticsfrompitchdata(Hz)fromspeedstories

StoryType


Overall 173.13(2.30) 171.50(3.70) 175.21(3.65)

Fast‐‐Mean(SD) Sharedphrases 170.74(3.15) 169.31(3.64) 172.25(5.69)


Overall 172.56(2.29) 172.49(3.39) 172.68(3.65)

Slow‐‐Mean(SD) Sharedphrases 170.82(2.48) 171.37(3.01) 170.33(4.20)



Athree‐wayrepeatedmeasureANOVAwasalsoperformedwithpitchasthe

dependentvariableandthesamesetoffixedwithin‐subjectfactors.TheANOVAshowed

nomaineffectofspeedandnointeractionsbetweenspeedandtheotherfactors.

Tosummarize,wepredictedthatparticipantswouldspeakwithafasterarticulation

ratewhenreadingstoriesinthefastcondition,butthattheirreadingsoffastandslow

storieswouldnotdifferintermsofpitch.Thispredictionissupportedbyourdata:stories

inthefastconditionwerereadaloudatasignificantlyfasterarticulationratethanstories

inthecorrespondingslowcondition,andtherewerenosignificantdifferencesinpitch

betweenfastandslowstories.Further,thelackofinteractionbetweensemanticvalence

andcontrastsuggeststhatthemodulationsinarticulationratearenotconcentratedwithin

explicitphrasesaboutspeed,butareinsteadmoredispersedacrossthestory.

2.2.2.Sizestories

ICONICPROSODY 16

Threeparticipantswereremovedbecauseofexcessivedisfluencies,leaving29

speakers.Tables4and5showsummarystatisticsofthepitchesandarticulationratesfrom

thesestories.BecausepitchmeasurementsexpressedinHertzaregenerallylog‐normally

distributed(Johnson,1997),Table4reportsthegeometricratherthanarithmeticmeanand

standarddeviation.

Table4:Summarystatisticsforpitchdata(Hz)fromsizestories

StoryType


Overall 170.92(2.03) 173.37(5.54) 168.51(4.00)

Small‐‐Mean(SD) Sharedphrases 172.43(3.36) 175.78(6.82) 169.13(4.27)


Overall 168.15(2.01) 169.44(5.92) 166.87(4.21)

Big‐‐Mean(SD) Sharedphrases 171.76(3.27) 173.74(6.90) 169.79(4.43)



Table5:Summarystatisticsforarticulationratedata(syl/s)fromsizestories

StoryType


Overall 4.94(0.23) 4.85(0.32) 5.02(0.40)

Small‐‐Mean(SD) Sharedphrases 5.08(0.52) 5.02(0.68) 5.13(0.55)


Overall 4.97(0.23) 4.83(0.21) 5.11(0.42)

Big‐‐Mean(SD) Sharedphrases 4.95(0.55) 4.87(0.63) 5.03(0.68)



First,athree‐wayrepeated‐measuresANOVAwasperformed,withlog‐transformed

pitchasthedependentvariable,andsemanticvalence(itemsfrombigorsmallstories),

concreteness(itemsfromconcreteorabstractstories),andcontrast(contrastingorshared

phrases)asfixedwithin‐subjectfactors.

Thethree‐wayANOVArevealedasignificantmaineffectforsize,F(1,27)=13.70,p

<.001,partialη2=.33.Therewasalsoasignificantinteractionbetweensizeandcontrast,

ICONICPROSODY 17

F(1,27)=7.22,p=.021,partialη2=.21.Followupt‐testsshowedthat,forcontrasting

phrases,participants’pitchesweresignificantlyhigherinthesmallconditioncomparedto

thebigcondition,t(28)=3.86,p<.001,whileforthesharedphrases,therewasno

significantdifferenceinpitchbetweenbigandsmallstories.

Nextweconductedathree‐wayANOVAwitharticulationrateasthedependent

variable,andthesamefixedfactorsasabove.Theresultsdidnotshowareliablemain

effectofsize,noraninteractionwithmappingtype.Therewas,however,asignificant

interactionbetweensizeandcontrast.Follow‐upt‐testsshowedthat,whilethedirection

ofdifferencewasreversedbetweensharedandcontrastingphrases,neithersharednor

contrastingphrasesofsizestoriesdifferedsignificantlyinarticulationrate.

Tosummarizetheseresults,wepredictedthatparticipantswouldspeakwitha

higherpitchwhenreadingstoriesinthesmallcondition,butthattheirreadingsofsmall

andbigstorieswouldnotdifferintermsofarticulationrate.Insupportofthisprediction,

wefoundthatcontrastingphrasesofsmallstorieswerespokenwithahigherpitchthan

thoseof‘large’stories,overbothconcreteandabstractinstantiationsofsize.Theshared

sectionsofthesestoriesdidnotdifferacrossbigandsmallstories.Thusthepitcheffect

associatedwithbigvs.smallwasrestrictedtotheimmediatecontextinwhichtheconcept

wasmentioned,andwasnotpresentforthepartsofthestorythatweresharedacross

conditions.Forarticulationrate,althoughwefoundaninteractionbetweensemantic

valenceandcontrast,follow‐uptestsfailedtorevealanysignificantpairwisedifferences

betweenthecontrastingorsharedelementsofbigandsmallversions,indicatingno

systematicinfluenceofsemanticvalenceonarticulationrate.

2.2.3.Alternativeexplanations

ICONICPROSODY 18

Althoughinlinewithourpredictions,thefactthatonlycontrastingsizephrases

variedinpitchpresentsthepossibleconfoundthatthedifferenceisanartifactofdifferent

phonologicalcontentbetweentheconditions,ratherthanthecontrastingmeanings.(Note,

however,thatthiscannotaccountforthearticulationrateeffectofspeedstories,which

occurredinthephonologically‐identicalsharedphrasesaswellascontrastingones.)There

areintrinsicpitchdifferencesassociatedwithconsonantvoicing.Forexample,/ta/hasa

higherintrinsicF0than/da/(Ohde,1984),thoughthiseffectissmall,shortinduration,and

localizedtohigh‐pitchedcontextsearlyintheutterance(Hanson,2009).Moreimportantly,

vowelcharacterisalsoassociatedwithintrinsicpitchdifferences.Forexample,/bi/hasa

higherintrinsicF0than/ba/(WhalenandLevitt,1995).Thusitisimportanttotest

whetherthepitcheffectcouldhavebeenduetolow‐levelmechanicsofarticulation,rather

thantothesemanticdifferencesbetweenconditions.

Toaddressthispossibleconfound,wereasonedthat,ifthephonological

characteristicsofcontrastingphrasesareresponsibleforthepitchdifferences,thenphrase

pairswhosephonologicalcontentbiasesthesmallmemberofthepairtowardshigherpitch

shouldhavesubstantiallygreaterdifferencesthanphrasepairswhosephonological

contentisdistributedwithanoppositebias.Forexample,thephrasepair,“itty‐bitty

houses”versus“gargantuanhouses”shouldhavealargerpitchdifferencethanthepair,“a

heftypricetopay”versus“asmallpricetopay.”

Thiswastestedbyfirstquantifyingthedegreeofphonologicalcontrastalongthe

potentiallytroublesomefeatures.Aphonologicalbiasscorewascomputedforeachphrase

pairasaratiooftheoddsofthehigher‐versuslower‐pitchedphonetypesforthesmall

phraseofeachpair,versusthesameoddsforthebigphraseofeachpair(seeTable6;χ2

ICONICPROSODY 19

testsshowednosignificantdifferenceinthedistributionofvowelandconsonanttypes

betweensmallandlargephrases).Forvoicelessness,weusedthecountsofvoicelessand

voicedconsonantsinthesmallandbigphrasesofeachofthe13pairstocalculatetheodds

ratiosofvoicelessconsonantspreferentiallyoccurringinthesmallphraseofeachpair.To

avoidundefinedodds‐ratios,weaddedacountof0.1toanycategorythathadrecordeda0,

andsubtracted0.1fromthecomplementarycategory.(Forexample,thephrase“reallybig

deal”containsnovoicelessconsonants,butitwascountedashaving0.1voiceless

consonantsand5.9voicedones.)Wealsocomputed13similaroddsratiosforvowels,

usingthecountsofhigh‐fronttoothervowelsinsmallversusbigmembersofthepair.

Next,pitchdifferencescoreswerecalculatedforeachphrase.Thefillerstoriesin

eachparticipant’srecordingprovidedabaselinelog‐transformedpitchwithwhichto

normalizeforindividualdifferencesacrossproductionsofagivenphrase,andthenall

participants’productionsofthatphrasewereaveragedtogether.Thedifferencescorewas

computedasthedifferencebetweentheaveragelog‐transformedpitchofthesmall

memberofthepairandtheaveragelog‐transformedpitchofthebigmemberofthepair.

Finally,thesepitchdifferencescoreswereregressedonthelogarithmsofthe

consonantoddsratiosandthevoweloddsratios,andtheirinteraction.Thesethree

predictorsfailedtoaccountforasignificantportionofthevarianceinpairwisedifferences,

multiple‐R2=.09,F(3,9)=0.30,n.s.Thus,theregressionfailedtoprovideevidencethat

phonologicaldifferencesbetweenthephrasepairsofsizestoriesrelatedtotheobserved

differencesinpitchacrossthepairs.

ICONICPROSODY 20

Table6.Distributionofconsonantsandvowelsacrosscontrastingmaterialfromsizestories

Small Big Odds‐ratio

Consonants Voiceless 47 38 1.50

Voiced 52 63

Vowels High‐front 14 14 1.06

Other 17 18

Note:Theoddsratiocomparestheoddsofthehigher‐pitchedfeatureoccuringinsmallvsbigphrases.

Anotherpossibleexplanationforourresultsisthatparticipantsmighthavebeen

consciouslyawareofthespeedandsizemanipulationsinthestories.Giventhe

experimentalcontextofreadingstoriestooneanother,theymighthavethendeliberately

modifiedtheirspeechiniconicemphasisofthesemeanings.However,post‐experiment

questionnairesmitigatethisconcerntosomeextent.Whenparticipantswereaskedwhat

theythoughtthestudywasaboutandwhethertheynoticedanythinginparticularabout

thestories,themajoritymadenomentionofnoticingparticularmeaningsinthestories.

The34%whodidsuggestedthatthestorieshadsomethingtodowithemotionalcontrasts

likehappyvs.sad,andnotasingleparticipantnoticedthesemanticcontrastsofspeedor

size.Thus,theresultsofthequestionnairesuggestthatparticipantsemployedtheseiconic

modulationsspontaneouslyinthetask,withoutconsciousdeliberation.

3.Generaldiscussion

Thepurposeofthepresentstudywastoadvanceinvestigationintothe

phenomenonoficonicprosodyinspeechproduction.Ourstoryreadingtaskfocusedon

meaningsrelatedtotheconceptsofspeedandsize,consideringbothconcreteandabstract

ICONICPROSODY 21

sensesoftheseterms.Wefoundthatwhenparticipantsreadaloudstoriesrelatingto

speed,theirreadingsoffaststoriesweresignificantlyfasterthanslowcounterpartstories.

Thisdifferencewassignificantacrossbothcontrastingandsharedphrases.Thelackof

interactionbetweenspeedandcontrastsuggeststhattheinfluenceofspeedwas

distributedacrossthestory,includingportionsinwhichspeedwasnotexplicitly

mentioned.Participants’readingsofsizestoriesalsoshowedaniconiceffectinprosody,

withsmallstoriesreadwithahigherpitchthanlargestories.Inthiscase,thedifference

waslocalizedtophrasesmakingexplicitmentionofsize;sharedsectionsofsizestoriesdid

notsignificantlydifferinpitch.Notably,wedidnotfindpitcheffectsinthespeedstories,or

articulationrateeffectsinthesizestories.

Asawhole,theseresultsreinforcepreviousresearchshowingthatspeakers

spontaneouslymodulatetheirprosodyiniconicrelationtothemeaningtheyare

expressing.Further,thedistinctionbetweenthewidely‐dispersedarticulationrate

differencesforspeedstories,andthemorenarrowlyconcentratedpitchdifferencesforsize

storiessuggeststhatthemodulationsarenotduesimplytoautomatic,linearactivationof

wordmeanings,whichwouldgenerateonlylocalizedeffectsinthecontrastingphrases.

Rather,itappearsthatthedifferentconceptsofspeedandsizemaylendthemselvesto

differenttemporalpatternsoficonicprosody.Onepossibleexplanationofthesedifferent

temporalpatternsisthattheyreflecttheconceptualizationofspeedasadynamic,ongoing

propertyofeventsthatlaststhroughoutthestories,andsizeasamorestatic,localized

propertyofobjectsandotherentitiesthatisrelevantonlywhentheyareinfocus.

Anadditionalgoalofthestudywastoexaminewhetherpeopleproduceiconic

modulationsofprosodywhenexpressingabstractsensesofspeed(e.g.fast/slowcareer

ICONICPROSODY 22

progress)andsize(e.g.,small/bigdeal).Wefailedtofindsignificantinteractionsof

concretenesswitheitherspeedorsize.Thus,althoughtheconcretesenseselicited

numericallylargereffects,especiallyforspeed,theresultsprovidetentativeevidencethat

abstractmeaningshavethepotentialtoelicitconceptuallymotivatedmodulationsof

prosody.

Whilethesmallnumberofitemsinthepresentexperimentdoesnotpermitmuch

generalization,theseresultsontheuseoficonicprosodytoexpressabstractsensesof

concretetermsoffersomedirectionforfurtherinquiry.Onepossibilityisthatsome

abstractconceptsmaytendtomanifestmoredistinctlyiniconicprosodythanothers.For

example,vocaliconicityforconcreteobjectpropertieslikesize,wherewefoundevidence

thattheactivatedconceptisconcentratedonthearticulationofparticularsize‐related

words,andwhichhasawell‐establishedsymbolicphonetics,maybemoreflexiblyapplied

torelatedabstractconcepts.Alternatively,itmaybethatthevocaliconicityunderlying

moreabstractmeaningsisgenerallyattenuatedindegreeoractivatedwithmore

variability(cf.Bergenetal.,2007).

3.1.Originsandsemanticscopeoficonicprosodyasgesture

Somescholarshavepreviouslyproposedthaticonicmodulationsofprosody,such

asthoseexaminedhere,maybeconsideredqualitativelyasaformofgesture(Liddel,2003;

McNeill,2005;Okrent,2002;Perlman,2010;Shintel,Nusbaum,&Okrent,2006).This

accountposesthatspeakersconceptualizeandexpressnon‐vocaldomainslikespeedand

size,inpart,byiconicmovementsoftheirvocaltract.Indeed,theintroductoryexampleof

ICONICPROSODY 23

DavidAroragesturingandvocalizingtodepictthelengthofamushroom’s“longstem”

hintsatacloseconnectionbetweeniconicmanualgesturesandiconicprosody.However,

thisraisesthequestionofhowiconicprosodymightcometobeincorporatedintoone’s

conceptualizationofnon‐vocaldomainslikespeedandsize.Wehypothesizethatsuch

associationsarelikelytodevelopbetweennon‐vocalconceptsandprosodicqualitiesofthe

voicetotheextentthattheyarecorrelatedinexperience—includingexperiencewith

culturalconventions—andlendthemselvestoabstractstructuralcorrespondences(cf.,

primarymetaphors;Grady,1999).

Forexample,peoplemightdevelopamappingbetweenspeedandspeechrateas

thesedomainsarelikelytobecorrelatedinexperience.Aspeopleactfast,theyare

generallymoreinclinedtotalkfast—acorrelationthatfollowsfromthefindingthat

physiologicalarousalisassociatedwithafasterspeechrate(BanseandScherer,1996).In

supportofthisidea,ithasbeenfoundthatradioplay‐by‐playofliveactioninasoccer

matchroughlytracksthepaceofactioninthegame.Play‐by‐playnarrationexhibitsa

fasterspeechratethanthatofspeechprovidingbackgroundinformationorcolor

commentary,withspeechrateincreasing(accomplishedbyshorterandlessfrequent

pauses)assuspensebuilds,andnormalizingafterdramaticeventslikegoals(Kern,2010).

Thiscorrelationisalsosupportedbyworkonlisteners’processingofspeechrateiconicity,

whichshowsthatlistenersaresensitivetothearticulationrateofthespeech,withshorter

reactiontimesinatrue‐falsejudgmenttaskafterfastspeech,butonlywhenthespeech

describedanurgentorhurriedscenario(ShintelandNusbaum,2008).Theseresultspoint

tohowanacceleratedorslowedarticulationratemightcometoembodyone’smore

generalconceptoffastorslowmotion.Thus,similartotheproductionofmanualiconic

ICONICPROSODY 24

gestures,whenpeoplethinkandtalkaboutfastorslowspeed,theirconceptualization

mightmanifestthroughiconicprosodythatreflectsthiscorrespondence.

Aplausibleaccountalsoexistsfortheexperientialcorrelationbetweenpitchand

size,whichisthoroughlyreviewedbyOhala(1984).Asthebasisforthiscorrelation,the

physicsofsoundentailthatlargerobjectsgenerallymakelower‐frequencysoundsthan

smallerones.Thepatternalsoappliestothepitchofanimalvocalizations,withlarger

animalsgenerallyproducinglowerpitchedsoundsandsmalleranimalsproducinghigher

pitchedsounds(afactlexicalizedinthephrase“pip‐squeak”).Moreover,thisexperiential

correlationhasbeenarguedtobesignificantintheevolutionofritualizedvocalsignalsin

manyvertebrates(Morton,1977).Acrossspeciesasdiverseasdogs,chickadees,

rhinoceroses,andfrogs,lower‐pitchedsoundsareusedforthreatdisplays(whenan

individualmightbenefitfromothers’reactingasifitislargerthanitreallyis),while

higher‐pitchedsoundsareusedinnonaggressivecontexts(whenitcanbebeneficialtobe

smallandnon‐threatening).Thesepatternssuggestthattheassociationbetweenpitchand

sizemaybegainedthroughexperience,butmayalsohaveareinforcingbasisthatis

inheritedfromourevolution.

Thus,throughsuchexperientialcorrelations,people’sconceptsofqualitieslike

speedandsizemightcometobeembodiedinthearticulatorymovementsoftheirvocal

tract.Weproposethat,likemanualiconicgestures,thevocalembodimentofthese

conceptualizationsbecomesactivewhenpeopletalkandthinkaboutconceptslikespeed

andsize,andthaticonicprosodyisproducedwhenthisactivityreachesacertainthreshold

(cf.Hostetter&Alibali,2008).

ICONICPROSODY 25

Consideredtogether,ourresults,alongwiththoseofotherrecentstudies,pointto

severalmappingsthatappeartobeembodiediniconicprosody,includingspeed,

verticality,andnowsize.Yetwhatothersemanticdomainsmightelicittheproductionof

iconicprosody?Comparedtothevisuospatialmodalityutilizedbymanualgestures,some

scholarshavemadetheargumentthatthevocal‐auditorymodalityisseverelylimitedinits

potentialforiconicrepresentation(Armstrong&Wilcox,2007;Hockett,1978;Liddell,

2003;Tomasello,2008;alsoseePinker&Jackendoff,2005onvocalimitation).AsHockett

reasons(1978:274):

Whenarepresentationofsomefour‐dimensionalhunkoflifehastobe

compressedintothesingledimensionofspeech,mosticonicityisnecessarily

squeezedout.Inone‐dimensionalprojection,anelephantisindistinguishable

fromawoodshed.Speechperforceislargelyarbitrary.

Whilesuchreasoningmayseemintuitive,itisuncleartheextenttowhichitisbiasedbythe

methodological(andcasualobservational)challengeofidentifyingiconicprosodyand

separatingitfromthe“non‐iconic”partofthespeechsignal,includingsyntacticand

phonologicalfactors.Indeed,onemightcounterthatspeechtooismultidimensional,

consistingofvariablepropertieslikefundamentalfrequency,intensity,duration,aswellas

morecomplexqualitiesoftimbrelikeharmonics‐to‐noiseratio.

Incontrasttothisview,substantialexperimentalworkinsoundsymbolismand

crosslinguisticresearchontherichsystemsofonomatopoeiainvariouslanguagespointsto

manyotherpotentialdomainsforiconicprosody(Dingemanse,2012;Hinton,Nichols,&

Ohala,1994;Nuckolls,1999;Pernissetal.,2010).Thisresearch,whichimplicatesarolefor

vocaliconicityinthediachronicdevelopmentoflanguages,hasfoundthattheconventional

ICONICPROSODY 26

gesturesofspeechareusedintheiconicexpressionofadisparatearrayofmeanings

relatingtoshape,mannerofmotion,physicaltexture,brightness,distance,andgender,

amongnumerousotherconcepts.Furtherresearchshouldaimtoobservethewaysin

whichthesevariousconceptsmightmanifestmoredynamicallyintheiconicprosodyof

speech(aswellasinexaggerationofthephoneticfeaturesofspeech;Feist,2013;Perlman,

2010).

4.Conclusion

Achallengeforresearchersistodevelopmethodsandanalysesthatfacilitatethe

identificationandmeasurementoficonicprosody,especiallywithinmorenaturalistic

contextsandacrosslanguages.Thegoalofthepresentstudywastoexpandthescopeof

semanticdomainsexaminedinresearchoniconicprosodyandtoenableamorefine‐

grainedexaminationofthephenomenon.Notably,theresultsshowdifferenttemporal

patternsoficonicprosodybetweentheconceptualdomainsofspeedandsize,which

presentsasignificantconsiderationasresearchersseektounderstandhowiconicprosody

mightmanifestoverthetimecourseofanutterance.Moregenerally,thefindings

contributetoasmallbutgrowingsetofstudiessuggestingthaticonicprosodyinthe

expressionofspeedisreadilyelicitedinEnglishspeakersinarangeofcontexts,fromthe

productionofcannedphrasesdescribingamovingdotinacomputerexperiment,to

readingstories,tothespontaneousdescriptionofvideoclips.Thesestudiesadditionally

indicatethaticonicprosodyrelatingtosomeotherconceptualdomains,likesizeand

verticality,mightalsobeelicitedwithsomefacility.

ICONICPROSODY 27

Yetbeyondthesefewexperiments,verylittleisknownabouticonicprosodyinthe

wild.Weknowalmostnothing,forexample,regardingtheonlinerelationshipbetween

manualiconicgestureandiconicprosodyduringtheproductionofspeech,suchastheir

relativefrequencyofoccurrenceorwhethertheymighttendtoco‐occurwitheachother.

Ultimately,ifwearetounderstandtherelationshipsbetweenspeech,gesture,andthe

conceptualprocessesthatareinvolvedintheirproduction,itiscrucialthatwegaina

deeperandmorecomprehensiveunderstandingoficonicprosodyandothervocaliconic

phenomena.

References

Armstrong,D.F.&Wilcox,S.(2007).Thegesturaloriginoflanguage.NewYork:Oxford

UniversityPress.

Banse,R.&Scherer,K.R.(1996).Acousticprofilesinvocalemotionexpression.Journalof

PersonalityandSocialPsychology70(3),614‐636.

Bergen,B.,Lindsay,S.,Matlock,T.,&Narayanan,S.(2007).Spatialandlinguisticaspectsof

visualimageryinsentencecomprehension.CognitiveScience,31,733‐764.

Boersma,P.(1993).Accurateanalysisofthefundamentalfrequencyandtheharmonics‐to‐

noiseratioofasampledsound.InstituteforPhoneticScience,Universityof

Amsterdam,Proceedings17,97‐110.

Boersma,P.(2001).Praat,asystemfordoingphoneticsbycomputer.GlotInternational,5,

341‐345.

ICONICPROSODY 28

Bolinger,D.(1986).Intonationanditsparts:MelodyinspokenEnglish.PaloAlto,CA:

StanfordUniversityPress.

Bryant,G.A.&FoxTree,J.E.(2002).Recognizingverbalironyinspontaneousspeech.

MetaphorandSymbol,17(2),99‐117.

Cienki,A.&Müller,C.(2008).Metaphor,gesture,andthought.InR.W.Gibbs,Jr.(Ed.),The

Cambridgehandbookofmetaphorandthought.Cambridge,UK:Cambridge

UniversityPress,483‐501

Cosmides,L.(1983).Invariancesintheacousticexpressionofemotionduringspeech.

JournalofExperimentalPsychology:HumanPerceptionandPerformance,9,864‐881.

Dingemanse,M.(2012).Advancesinthecross‐linguisticstudyofideophones.Language

andLinguisticsCompass,6,654‐672.

Duncan,S.D.(2003).Gestureinlanguage:issuesforsignlanguageresearch.InK.Emmorey

(Ed.),Perspectivesonclassifierconstructionsinsignedlanguages.Mahwah,NJ:

LawrenceErlbaumAssociates,259‐268.

Ferreira,F.(1993).Creationofprosodyduringsentenceproduction.PsychologicalReview,

100,233‐253.

Feist,J.(2013).“Soundsymbolism”inEnglish.JournalofPragmatics,45,104‐118.

Grady,J.(1999).Atypologyofmotivationforconceptualmetaphor:correlationvs.

resemblance.InG.Steen,R.Gibbs(Eds.),MetaphorinCognitiveLinguistics.

Philadelphia:JohnBenjamins,79–100.

Hanson,H.M.(2009).Effectofobstruentconsonantsonfundamentalfrequencyatvowel

onsetinEnglish.JournaloftheAcousticalSocietyofAmerica,125(1),425‐441.

ICONICPROSODY 29

Hinton,L.,Nichols,J.&Ohala,JJ.(Eds.).(1994).Soundsymbolism.Cambridge,UK:

CambridgeUniversityPress.

Hockett,C.F.(1978).InsearchofJove'sbrow.Americanspeech,53(4),243‐313.

Hostetter,A.&Alibali,M.W.(2008).Visibleembodiment:Gesturesassimulatedaction.

PsychonomicBulletin&Review,15,495‐514.

Jakobson,R.&Waugh,L.R.(1979).Thesoundshapeoflanguage.Bloomington:Indiana

UniversityPress.

Johnson,K.(1997).Auditoryandacousticphonetics.Oxford:Blackwell.

Kendon,A.(2004).Gesture:Visibleactionasutterance.Cambridge,UK:Cambridge

UniversityPress.

Kern,F.(2010).Speakingdramatically:Theprosodyinradiolivecommentariesoffootball

games.InSelting,M.,Barth‐Weingarten,D.,Reber,E.(eds.),ProsodyinInteraction.

Amsterdam:JohnBenjamins,217‐238.

Levelt,W.J.M.(1989).Speaking:Fromintentiontoarticulation.Cambridge,MA:MITPress.

Levelt,W.,Roelofs,A.,&Meyer,A.(1999).Atheoryoflexicalaccessinspeechproduction.

BehavioralandBrainSciences,22,1‐75.

Liddell,S.K.(2003).Grammar,gesture,andmeaninginAmericanSignLanguage.

Cambridge,UK:CambridgeUniversityPress.

McNeill,D.(1992).Handandmind:Whatgesturesrevealaboutthought.Chicago:Chicago

UniversityPress.

McNeill,D.(Ed.).(2000).Languageandgesture.Cambridge,UK:CambridgeUniversity

Press.

ICONICPROSODY 30

McNeill,D.(2005).Gestureandthought.Chicago:ChicagoUniversityPress.

Morton,E.W.(1977).Ontheoccurrenceandsignificanceofmotivation‐structuralrulesin

somebirdandmammalsounds.AmericanNaturalist,111,855‐869.

Nuckolls,J.(1999).Thecaseforsoundsymbolism.AnnualReviewofAnthropology,28,

225‐252.

Nygaard,L.C.,Herold,D.S.,&Namy,L.L.(2009).Thesemanticsofprosody:Listeners’use

ofacousticcorrelatestowordmeaning.CognitiveScience,33,127‐146.

Ohala,J.J.(1984).Anethologicalperspectiveoncommoncross‐languageutilizationofF0

ofvoice.Phonetica,41,1‐16.

Ohde,R.N.(1984).Fundamentalfrequencyasanacousticcorrelateofstopconsonant

voicing.JournaloftheAcousticalSocietyofAmerica,75(1),224‐230.

Okrent,A.(2002).Amodality‐freenotionofgestureandhowitcanhelpuswiththe

morphemevs.gesturequestioninsignlanguagelinguistics.InR.P.Meier,K.

Cormier,&D.Quinto‐Pozos(Eds.),Modalityandstructureinsignedandspoken

languages.Cambridge,UK:CambridgeUniversityPress,175‐198.

Parise,C.V.&Pavani,F.(2011).Evidenceofsoundsymbolisminsimplevocalizations.

ExperimentalBrainResearch,214,373‐380.

Perlman,M.(2010).Talkingfast:Theuseofspeechrateasiconicgesture.InF.Perrill,V.

Tobin,&M.Turner(Eds.),Meaning,Form,andBody(pp.245‐262).Stanford,CA:

CSLIPublications.

Perlman,M.,Dale,R.,&Lupyan,G.(2014).Vocalcharades:Theemergenceofconventions

invocalcommunication.InE.A.Cartmill,S.Roberts,H.Lyn,&H.Cornish(Eds.),The

ICONICPROSODY 31

EvolutionofLanguage:Proceedingsofthe10thInternationalConference

(EVOLANGX).NewJersey:WorldScientific.

Perniss,P.,Thompson,R.L.,&Vigliocco,G.(2010).Iconicityasageneralpropertyof

language:Evidencefromspokenandsignedlanguages.FrontiersinPsychology,1,1‐

14.

Pinker,S.&Jackendoff,R.(2005).Thefacultyoflanguage:What’sspecialaboutit?

Cognition,95,201‐236.

Shintel,H.&Nusbaum,H.C.(2008).Movingtothespeedofsound:Contextmodulationof

theeffectofacousticpropertiesofspeech.CognitiveScience,105(6),1063‐1078.

Shintel,H.,Nusbaum,H.C.,&Okrent,A.(2006).Analogacousticexpressioninspeech

communication.JournalofMemoryandLanguage,55,167‐177.

Tomasello,M.(2008).Originsofhumancommunication.Cambridge,MA:MITPress.

Tsur,R.(2006).Size‐soundsymbolismrevisited.JournalofPragmatics,38,905‐924.

Ultan,R.(1969).Size‐soundsymbolism.InJ.H.Greenberg,C.A.Ferguson,andE.A.

Moravcsik(Eds.)UniversalofHumanLanguage,vol.2:Phonology(pp.527‐568).

Stanford:UniversityPress.

Whalen,D.W.,&Levitt,A.G.(1995).TheuniversalityofintrinsicF0ofvowels.Journalof

Phonetics,23,349‐366.

Acknowledgements

ICONICPROSODY 32

Thestudywasdonewhilethethirdauthor,MarleneJohanssonFalck,wasfundedbythe

SwedishResearchCouncil(VR37127).Hercontributiontowritingthismanuscriptwas

fundedbytheRoyalSwedishAcademyofLetters,HistoryandAntiquities,supportedbya

grantfromtheKnutandAliceWallenbergFoundation(KAW2009.0295).

ICONICPROSODY 33

Appendix1TableS‐1:Speedstimuliwithsyllablecounts

Section Syl. Fasttext Slowtext

Character Martha Martha

Title ARapidRun ASluggishWalk

Intro 21 MarthaisafastrunnerShealwaysexercises,andtodaysheisoutthere.

MarthaisaslowwalkerSherarelyexercises,buttodaysheisoutthere.

ContrastingPhrase 4 afastrunner aslowrunner

Plot 37 Shetakesoffquicklythroughtheneighborhood.Shedashespastsomehousesandcontinuestowardsaschool.Shespeedstoanear‐sprintandreachesthefinalstretch.

Shelaborsslowlythroughtheneighborhood.Shestrugglespastsomehousesandcontinuestowardsaschool.Sheslowstoanearstopbutreachesthefinalstretch.

ContrastingPhrase 4 takesoffquickly laborsslowly

ContrastingPhrase 5 dashespastsomehouses strugglespastsomehouses

ContrastingPhrase 5 speedstoanear‐sprint slowstoanearstop

Conclusion 6 Sheisreachingtheend. Sheisreachingtheend.

Character Bob Bob

Title RacingthroughTraffic InchingThroughTraffic

Intro 18 Bobisamiddle‐agedprofessionalman.Heisdrivingtowork,butseemstobeflying

Bobisamiddle‐agedprofessionalman.Heisdrivingtowork,butseemstobecreeping

ContrastingPhrase 6 butseemstobeflying butseemstobecreeping

Plot 39 Thetrafficislight,andBobquicklyracesdownthehighway,dartinginandoutbetweentheothercars.Hekeepsracinguntilheseestheofficebuilding.

Thetrafficisthick,andBobslowlyinchesdownthehighway,crawlinginandoutbetweentheothercars.Hekeepsinchinguntilheseestheofficebuilding.

ContrastingPhrase 5 Bobquicklyracesdown Bobslowlyinchesdown

ContrastingPhrase 5 dartinginandout crawlinginandout

Conclusion 5 Heisgettingthere. Heisgettingthere.

Character Diana Diana

Title SpeedingDownaRoadtoNowhere LimpingDownaPathtoNowhere

Intro 25 Dianaisawomaninherforties.Herlifehassuddenlytakensomeunfortunateturns.

Dianaisawomaninherforties.Herlifehasgraduallytakensomeunfortunateturns.

ContrastingPhrase 6 hassuddenlytaken hasgraduallytaken

Plot 51 SheisspeedingdownaroadtonowhereTheeconomyisbad.Hercompanyhasgoneoutofbusiness.Herskillsarenolongerneededonthejobmarket.Sheblazesontowardsanuncertainfuture.

SheislimpingdownapathtonowhereTheeconomyisbad.Hercompanyhasgoneoutofbusiness.Herskillsarenolongerneededonthejobmarket.Shehobblesontowardsanuncertainfuture.

ContrastingPhrase 7 sheisspeedingdownaroad sheislimpingdownapath

ICONICPROSODY 34

ContrastingPhrase 4 sheblazeson shehobbleson

Conclusion 6 Herlifeischanging. Herlifeischanging.

Character Peter Peter

Title TheFastTracktoSuccess TheSlowPathtoSuccess

Intro 22 Peterisa35‐year‐oldbusinessman.Heistakingafasttracktosuccess.

Peterisa35‐year‐oldbusinessman.Heistakingaslowpathtosuccess.

ContrastingPhrase 3 afasttrack aslowpath

Plot 41 HiscareerisquicklymovingforwardHeisabouttobepromoted.Hehasanicelifeandapleasanthome.Heisheadingswiftlyintherightdirection.

HiscareerisslowlymovingforwardHeisabouttobepromoted.Hehasanicelifeandapleasanthome.Heisheadingslowlyintherightdirection.

ContrastingPhrase 6 quicklymovingforward slowlymovingforward

ContrastingPhrase 4 headingswiftly headingslowly

Conclusion 6 Heisreachinghisgoals. Heisreachinghisgoals.

Note:"Bob"and"Martha"itemswereconsideredconcrete,while"Diana"and"Peter"itemswereconsideredabstract.

ICONICPROSODY 35

Appendix2

TableA‐2:SizestimuliwithphonoligcalbiasmeasuresforcontrastingContrastingPhrases

Section Syl Bigtext Smalltext C‐Bias V‐Bias

Character Phyllis Phyllis

Title AGiantElephant ATinyGrasshopper

Intro 19 Phyllisisahumungouselephant.Sheisontheplainsearchingforfood.

Phyllisisaminisculegrasshopper.Sheisontheplainsearchingforfood.

ContrastingPhrase 7 ahumongouselephant aminisculegrasshopper 1.00 1.25

Plot 17 Phyllislumbersthroughthegrass,castingamassiveshadowasshegoes.

Phyllishiphopsthroughthegrass,castingatinyshadowasshegoes.

ContrastingPhrase 5 lumbersthroughthegrass hiphopsthroughthegrass 149.50 5.33

ContrastingPhrase 5 amassiveshadow atinyshadow 199.00 1.50

Conclusion 6 Shekeepslookingforfood. Shekeepslookingforfood.

Character Bonnie Bonnie

Title AGargantuanHome AnItty‐BittyHome

Intro 14 Bonnieisathome.Shelivesinaverygianthouse.

Bonnieisathome.Shelivesinaverytinyhouse.

ContrastingPhrase 5 verygianthouse veryteenyhouse 4.00 1.00

Plot 28 Thehugehouseisinaneighborhoodwithothergargantuanhouses.Eachhouseisonagreatbigplotofland.

Thetinyhouseisinaneighborhoodwithotheritty‐bittyhouses.Eachhouseisonalittleplotofland.

ContrastingPhrase 2 hugehouse tinyhouse 199.00 1.00

ContrastingPhrase 6 gargantuanhouses itty‐bittyhouses 398.00 3.50

ContrastingPhrase 6 agreatbigplotofland alittleplotofland 0.50 1.29

Conclusion 5 Thishouseisherhome. Thishouseisherhome.

Character Joe Joe

Title AReallyBigDeal A PrettySmallDeal

Intro 20 Joeisthinkingaboutsigningarecordcontract.Itisareallybigdeal.

Joeisthinkingaboutsigningarecordcontract.Itisaprettysmalldeal.

ContrastingPhrase 4 reallybigdeal prettysmalldeal 1.00 359.40

Plot 23/22 Itwillhaveahugeimpactonhiscareer.Joeismakingamonumentaldecision.

Itwillhavelittleimpactonhiscareer.Joeismakingatrivialdecision.

ContrastingPhrase 5 haveahugeimpact havelittleimpact 4.00 0.60

ContrastingPhrase 8/7 amonumentaldecision atrivialdecision 1.00 1.33

Conc. 5 Hesignsthecontract. Hesignsthecontract.

Character George George

Title AHeftyPricetoPay ASmallPricetoPay

Intro 25 Georgeisageneral.Hehasjustsentabattalionintoheavybattletotakeoverabridge.

Georgeisageneral.Hehasjustsentabattalionintoalightskirmishtotakeoverabridge.

ICONICPROSODY 36

ContrastingPhrase 4 heavybattle lightskirmish 1.00 2.00

Plot 23/22 Thecasualtieswillbevast.Itisaheftypricetopay,buttheenemymustbestopped.

Thecasualtieswillbeslight.Itisasmallpricetopay,andtheenemymustbestopped.

ContrastingPhrase 3 willbevast willbeslight 1.00 1.00

ContrastingPhrase 7/6 aheftypricetopay asmallpricetopay 0.01 0.24

Conclusion 5 Theymovetowardsthebridge. Theymovetowardsthebridge.

Note:"Phyllis"and"Bonnie"itemswereconsideredconcrete,while"Joe"and"George"itemswereconsideredabstract.Note:Consonantbiasscore(C‐bias)istheratiooftheoddsofvoicelessconsonantsinthesmallphrasetotheiroddsinthebigphrase.Vowelbiasscore(V‐Bias)istheratiooftheoddsofhigh‐frontvowelsinthesmallphrasetotheiroddsinthebigphrase.

Documents

Iconic prosody in story reading - umu.diva-portal.orgumu.diva-portal.org/smash/get/diva2:759405/FULLTEXT02.pdf · Iconic prosody in story reading ... Bryant and Fox Tree, 2002; Cosmides,