CS388:NaturalLanguageProcessing
GregDurre8
Lecture20:Informa>onExtrac>on,SRL,etc.
Administrivia
‣ Finalprojectpresenta>onslotsannounced
‣ Project2duetoday
ThisLecture
‣ Howdowerepresentinforma>onforinforma>onextrac>on?
‣ OpenInforma>onExtrac>on
‣ Rela>onextrac>on
‣ Slotfilling
‣ Seman>crolelabeling/abstractmeaningrepresenta>on
Represen>ngInforma>on
Seman>cRepresenta>ons
Examplecredit:AsadSayeed
personBrutusCaesarObamaBush…
stabBrutus Caesar
…
‣ “World”isasetofen>>esandpredicates
president
Obama
Bush…
BrutusstabsCaesar stab(Brutus,Caesar)=>true
‣ Statementsarelogicalexpressionsthatevaluatetotrueorfalse
Caesarwasstabbed ∃xstab(x,Caesar)=>true
Neo-DavidsonianEvents
Examplecredit:AsadSayeed
BrutusstabbedCaesarwithaknifeatthetheaterontheIdesofMarch
∃estabs(e,Brutus,Caesar)
‣ Letsusdescribeeventsashavingproper>es‣ Unifiedrepresenta>onofeventsanden>>es:
∃xdriver(x)∧clever(x)∧loca>on(x,America)
somecleverdriverinAmerica
∧with(e,knife) ∧loca>on(e,theater)
∧>me(e,IdesofMarch)
RealText
BarackObamasignedtheAffordableCareactonTuesday.HegaveaspeechlaterthataCernoononhowtheactwouldhelptheAmericanpeople.SeveralprominentRepublicanswerequicktodenouncethenewlaw. whichTuesday?
‣ Needtoimputemissinginforma>on,resolvecoreference,etc.
‣ S>llunclearhowtorepresentsomethingspreciselyorhowthatinforma>oncouldbeleveraged(severalprominentRepublicans)
who?
???
whichaaernoon?
∃esign(e,BarackObama)∧pa>ent(e,ACA)∧>me(e,Tuesday)
OtherChallenges
BobandAlicewerefriendsunGlhemovedawaytoaHendcollege
‣ Howtorepresenttemporalinforma>on?
‣ Represen>ngtrulyopen-domaininforma>onisverycomplicated!Wedon’thaveaformalrepresenta>onthatcancaptureeverything
∃e1∃e2friends(e1,Bob,Alice)∧moved(e2,Bob)∧end_of(e1,e2)
BobandAlicewerefriendsunGlaroundthe+mehemovedawaytoaHendcollege
(Atleast)ThreeSolu>ons
‣ En>ty-rela>on-en>tytriples:focusonen>>esandtheirrela>ons(notethaten>>esispre8ybroad:canincludeeventslikeWorldWarII,etc.)
(LadyGaga,singerOf,BadRomance)
‣ Craaedannota>onstocapturesomesubsetofphenomena:predicate-argumentstructures(seman>crolelabeling),>me(temporalrela>ons),…
‣ Slotfilling:specificontology,populateinforma>oninapredefinedway
(Earthquake:magnitude=8.0,epicenter=centralItaly,…)
OpenIE‣ En>ty-rela>on-en>tytriplesaren’tnecessarilygroundedinanontology
BarackObamasignedtheAffordableCareactonTuesday.HegaveaspeechlaterthataCernoononhowtheactwouldhelptheAmericanpeople.SeveralprominentRepublicanswerequicktodenouncethenewlaw.
(BarackObama,signed,theAffordableCareact)
(SeveralprominentRepublicans,denounce,thenewlaw)
‣ Extractstringsandletadownstreamsystemfigureitout
IE:TheBigPicture
‣ Howdowerepresentinforma>on?Whatdoweextract?
‣ Slotfillers‣ En>ty-rela>on-en>tytriples(fixedontologyoropen)
‣ Seman>croles
‣ Abstractmeaningrepresenta>on
Seman>cRoleLabeling/ AbstractMeaningRepresenta>on
Seman>cRoleLabeling
LadyGagaperformedaconcertforstudents
‣ Performingevent
‣ Subject:LadyGaga‣ Object:aconcert‣ Audience:students
AconcertwasperformedbyLadyGagaforstudents
‣ Sameeventdescribedbuttherepresenta>onlooksdifferent
VARG1 ARG0 ARG2
VARG0 ARG1 ARG2
‣ Verb(predicate)associatedwithseveralarguments(roles):“Agent”,“Theme”,and“Beneficiary”
VerbNet
verbs.colorado.edu
‣ Definestheseman>csofverbs,argumentsforeveryverbinEnglish
Seman>cRoles
‣ Relatedtothetarolesinlinguis>cs
‣ Agent(~subject),pa>ent/theme(~object),goal(~indirectobject)
‣ “Postprocessing”layerontopofdependencyparsingthatexposesusefulinforma>on,canonicalizesacrossgramma>calconstruc>ons
ARG0 ARG1 ARG2+(seman>csvary)
Seman>cRoleLabeling
‣ Iden>fypredicate,disambiguateit,iden>fythatpredicate’sarguments
‣ VerbrolesfromPropbank(Palmeretal.,2005)
FigurefromHeetal.(2017)
quicken:
Seman>cRoleLabeling
FigurefromHeetal.(2017)
‣ Iden>fypredicates(love)usingaclassifier(notshown)
‣ Iden>fyARG0,ARG1,etc.asataggingtaskwithaBiLSTMcondi>onedonlove
‣ Othersystemsincorporatesyntax,jointpredicate-argumentfinding
SRLforQA
ShenandLapata(2007)
‣ Ques>onandseveralanswercandidates
Q:Whodiscoveredprions?
AC1:In1997,StanleyB.Prusiner,ascienGstintheUnitedStates,discoveredprions…
AC2:Prionswereresearchedby…
Scorebymatchingexpectedanswerphrase(EAP)againstanswercandidate(AC)
AbstractMeaningRepresenta>on
‣ Graph-structuredannota>on
Theboywantstogo
Banarescuetal.(2014)
‣ SupersetofSRL:fullsentenceanalyses,containscoreferenceandmul>-wordexpressionsaswell
‣ F1scoresinthe60s:hard!
‣ Socomprehensivethatit’shardtopredict,buts>lldoesn’thandletenseorsomeotherthings…
AbstractMeaningRepresenta>on
Flaniganetal.(2016),Lyuetal.(2018)
‣ Firstpredictmappingfromconceptstographnodes(many-to-many)
‣ Thenuseanedgescoringmodulesimilartodependencyparserstopredictedges
‣ Predic>ngacoherentgraphishard,lotsofconstraintsonitandnodynamicprogram
Summariza>onwithAMR
Liuetal.(2015)
‣MergeAMRsacrossmul>plesentences
‣ Summariza>on=subgraphextrac>on
‣ Norealsystemsactuallyworkthisway(morewhenwetalkaboutsummariza>on)
SlotFilling
SlotFilling‣Mostconserva>ve,narrowformofIE
IndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyiniGallysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.
magnitude time
epicenter
Speaker:AlanClark
“GenderRolesintheHolyRomanEmpire”
AllagherCenterMainAuditorium
Thistalkwilldiscuss…
speakertitle
location
FreitagandMcCallum(2000)
‣ Oldwork:HMMs,laterCRFstrainedperrole
SlotFilling:MUC
HaghighiandKlein(2010)
‣ Keyaspect:needtocombineinforma>onacrossmul>plemen>onsofanen>tyusingcoreference
SlotFilling:Forums
‣ Extractproductoccurrencesincybercrimeforums,butnoteverythingthatlookslikeaproductisaproduct
Portnoffetal.(2017),Durre8etal.(2017)Notaproductinthiscontext
Rela>onExtrac>on
Rela>onExtrac>on‣ Extracten>ty-rela>on-en>tytriplesfromafixedinventory
ACE(2003-2005)
DuringthewarinIraq,AmericanjournalistsweresomeGmescaughtinthelineoffire
Located_In
‣ Systemscanbefeature-basedorneural,lookatsurfacewords,syntac>cfeatures(dependencypaths),seman>croles
Na>onality
‣ Problem:limiteddataforscalingtobigontologies
‣ UseNER-likesystemtoiden>fyen>tyspans,classifyrela>onsbetweenen>typairswithaclassifier
HearstPa8erns
Hearst(1992)
Xsuchas[list] ciGessuchasBerlin,Paris,andLondon.
‣ Syntac>cpa8ernsespeciallyforfindinghypernym-hyponympairs(“isa”rela>ons)
Berlinisacity
otherciGesincludingBerlin
YisaX
otherXincludingY
‣ Totallyunsupervisedwayofharves>ngworldknowledgefortaskslikeparsingandcoreference(BansalandKlein,2011-2012)
DistantSupervision
Mintzetal.(2009)
[StevenSpielberg]’sfilm[SavingPrivateRyan]islooselybasedonthebrothers’story
‣ Iftwoen>>esinarela>onappearinthesamesentence,assumethesentenceexpressestherela>on
‣ Lotsofrela>onsinourknowledgebasealready(e.g.,23,000film-directorrela>ons);usethesetobootstrapmoretrainingdata
Allisonco-producedtheAcademyAward-winning[SavingPrivateRyan],directedby[StevenSpielberg]
Director
Director
DistantSupervision
Mintzetal.(2009)
‣ Learndecentlyaccurateclassifiersfor~100Freebaserela>ons‣ Couldbeusedtocrawlthewebandexpandourknowledgebase
FewRel‣ Treatsrela>onclassifica>onasafew-shotclassifica>onproblem
‣ 100classesx700instances,goalistogeneralizetoeachclasswithjustafewinstances
‣ BERTcanhandlethisfairlywell(Soaresetal.,2019)
‣ “FewRel2.0”:newdatasetwith“noneoftheabove”type,whichmakesthingsmuchharder
Hanetal.(2018),Gaoetal.(2019)
En>tyTracking/ProceduralText
En>tyTracking
Slidecredit:AdityaGupta
RecipesDataset
Kiddonetal.(2016),Bosselutetal.(2018)
‣ Informa>onextrac>onfor“proceduraltext”:textdescribingsomekindofprocess
‣ Forarecipe:whatingredientsareinvolvedateach>mestep?
‣ Involvesglobalconstraintsandbeingabletomodelcomplexen>tyinterac>ons
En>tyTracking
Slidecredit:AdityaGupta
ProParaDataset
Dalvietal.(2018),GuptaandDurre8(2019)
‣ Processparagraphs:predictwhenobjectsarecreated,moved,ordestroyedinascien>ficprocess
‣ Structuredpredic>onproblem,>edtothepar>cularinforma>onconveyedintheseparagraphs
‣ UseaneuralCRFtomakeacoherentpredic>onforeachen>ty
OpenIE
OpenInforma>onExtrac>on
‣ Typicallynofixedrela>oninventory
‣ “Open”ness—wanttobeabletoextractallkindsofinforma>onfromopen-domaintext
‣ Acquirecommonsenseknowledgejustfrom“reading”aboutit,butneedtoprocesslotsoftext(“machinereading”)
TextRunner‣ Extractposi>veexamplesof(e,r,e)triplesviaparsingandheuris>cs
BarackObama,44thpresidentoftheUnitedStates,wasbornonAugust4,1961inHonolulu
=>Barack_Obama,wasbornin,Honolulu
‣ TrainaNaiveBayesclassifiertofiltertriplesfromrawtext:usesfeaturesonPOStags,lexicalfeatures,stopwords,etc.
‣ 80xfasterthanrunningaparser(whichwasslowin2007…)
Bankoetal.(2007)
‣ Usemul>pleinstancesofextrac>onstoassignprobabilitytoarela>on
Exploi>ngRedundancy
Bankoetal.(2007)
‣ Concrete:definitelytrueAbstract:possiblytruebutunderspecified
‣ Hardtoevaluate:canassessprecision ofextractedfacts,buthowdoweknowrecall?
‣ 9Mwebpages/133Msentences
‣ 2.2tuplesextractedpersentence,filterbasedonprobabili>es
ReVerb
Faderetal.(2011)
‣Moreconstraints:openrela>onshavetobeginwithverb,endwithpreposi>on,becon>guous(e.g.,wasbornon)
‣ Extractmoremeaningfulrela>ons,par>cularlywithlightverbs
ReVerb
Faderetal.(2011)
‣ Foreachverb,iden>fythelongestsequenceofwordsfollowingtheverbthatsa>sfyaPOSregex(V.*P)andwhichsa>sfyheuris>clexicalconstraintsonspecificity
‣ Findthenearestargumentsoneithersideoftherela>on
‣ Annotatorslabeledrela>onsin500documentstoassessrecall
QAfromOpenIE
Choietal.(2015)
Takeaways
‣ Rela>onextrac>on:cancollectdatawithdistantsupervision,usethistoexpandknowledgebases
‣ Slotfilling:>edtoaspecificontology,butgivesfine-grainedinforma>on
‣ OpenIE:extractslotsofthings,buthardtoknowhowgoodorusefultheyare
‣ Cancombinewithstandardques>onanswering
‣ Addnewfactstoknowledgebases
‣ SRL/AMR:handleabunchofphenomena,butmoreorlesslikesyntax++intermsofwhattheyrepresent
‣Many,manyapplica>onsandtechniques