Time for Text Mining and Information Retrievaljstroetge/talks/jannik-stroetgen... · Text Mining...

Time forText Mining and Information Retrieval

Jannik Strötgen

jannik.stroetgen@mpi-inf.mpg.de

Joint Lecture Series – Saarbrücken – January 11, 2017

Why Time? Temporal Tagging HeidelTime Applications Summary

Why temporal information?

Joint Lecture Series – January 11, 2017 c© Jannik Strötgen 1 / 50

Motivating example

From: student@uni.deTo: prof@uni.deDate: January 11, 2017

Subject: meeting

Dear Prof,

I am sorry that I couldn’tmake it on Monday. Butcould we meet this Friday?I don’t care about the time.What about 2 pm?

Thank you and kind regards,student

Motivating example

Subject: meeting

Dear Prof,

JANUARY09 Monday10 Tuesday

11 Wednesday

12 Thursday13 Friday14 Saturday15 Sunday16 Monday

Motivating example

Subject: meeting

Dear Prof,

JANUARY09 Monday10 Tuesday11 Wednesday12 Thursday

13 Friday

2 pm – 2:30 pm

student

14 Saturday15 Sunday16 Monday

Motivating example

Subject: meeting

Dear Prof,

Motivating example

Subject: meeting

Dear Prof,

I am sorry that I couldn’tmake it on Monday. But

could we meet this Friday ?

I don’t care about the time.What about 2 pm?

13 Friday

Motivating example

Subject: meeting

Dear Prof,

13 Friday

2 pm – 2:30 pm

student

Time: crucial role in many types of documents

news articles Wikipedia documents

Time: crucial role in many types of documents

short messages scientific publications literary works

Time: important key characteristicstime is well-defined:

expressions can becompared with each other

time can be organizedhierarchically:

different granularities

relations as defined in Allen’sinterval algebra [Allen, 1983]

2000 before March 2001equalduring...

2015 2016

2016-03

... 2016-03-12

2016-04 ...

2015 2016

2016-03

... 2016-03-12

2016-04 ...

2015 2016

2016-03

... 2016-03-12

2016-04 ...

2015 2016

2016-03

... 2016-03-12

2016-04 ...

Time: important key characteristics

temporal information can be normalized:expressions with same semantics → same value

temporal information is term- and language-independent

t2017-01-11

tref2017-01-10

tomorrow

t2017-01-11

tref2017-01-10

tomorrowJanuary 11, 2017

t2017-01-11

tref2017-01-10 2017-01-16 2017-02-11

tomorrowlast Wednesday

one month ago

January 11, 2017

t2017-01-11

tref2017-01-10 2017-01-11 2017-01-16 2017-02-11

tomorrow

last Wednesday

one month ago

January 11, 2017

Time: valuable for many applications

information retrieval (search engines)information extraction

text summarizationmachine translationquestion answering. . .

prerequisite: temporal tagging

information retrieval (search engines)information extractiontext summarizationmachine translationquestion answering. . .

Questions addressed in this talk

What’s temporal tagging?

Why domain-sensitive temporal tagging?

What applications can exploit temporal information and how?

The two tasks of a temporal tagger

1. extraction of temporal expressions

main challengeambiguities, e.g.,

may, march, fall, 2000

May I march 2000 milesbefore I fall?

1. extraction of temporal expressions2. normalization of temporal expressions

tonight → 2011-09-20TNIyesterday → 2011-09-19next week → 2011-W39Sept. 20, 2011 → 2011-09-20next month → 2011-10

main challengenormalization of relative andunderspecified expressions

1. extraction of temporal expressions2. normalization of temporal expressions

tonight → 2011-09-20TNIyesterday → 2011-09-19next week → 2011-W39Sept. 20, 2011 → 2011-09-20next month → 2011-10

main challengenormalization of relative andunderspecified expressions

Types of temporal expressions

dates– June 2001

times– yesterday morning

durations– several months

sets– every day

Realizations of date expressions

explicit → easy to normalize– June 24, 2013– the 20th century

implicit

– Christmas 2012– Columbus Day 2006

relative

– two weeks later– yesterday

underspecified

– Monday– June 24

often challengingnormalization of relative and underspecified expressions

implicit → additional knowledge required– Christmas 2012– Columbus Day 2006

relative

– two weeks later– yesterday

underspecified

relative → reference time required– two weeks later– yesterday

underspecified

underspecified → reference time and relation to it required– Monday– June 24

Temporal tagging example

Document Creation Time: 2000-12-26. . . On Thursday , the Census Bureau will publish the official pop-ulation count for the United States, including the state-by-state to-tals required under the Constitution to determine how many seatseach state is allocated in the House. The figures, eagerly awaitedby many state government officials, are the first in a wave of re-leases of demographic data based on the 2000 census. . . .Population estimates issued periodically by the Census Bureau in-dicate that as of October , 275,843,000 people were living in . . .Additional seats are then assigned to each state based on aperson-to-House-member ratio that changes every 10 years be-cause the country’s population keeps growing . . .

Temporal tagging of news(-style) documents

normalization typically with document creation time

Temporal tagging of narrative documents

Soviet-Afghan War 2017-01-101979: Soviet deploymentThe Afghan government, having secured a treaty in the spring and summerof 1979. They requested Soviet troops to provide security and to assist inthe fight against the mujahideen rebels. On April 14, 1979, the Afghan gov-ernment requested that the USSR send 15 to 20 helicopters with their crewsto Afghanistan, and on June 16 . . . The operation was fully complete by themorning of December 28, 1979. . . . According to the Soviet Politburo theywere complying with the 1978 Treaty of Friendship, Cooperation and GoodNeighborliness . . . Soviet ground forces, under the command of Marshal SergeiSokolov, entered Afghanistan from the north on December 27. In the morning,the 103rd Guards . . .

WikiWars corpus, 18 SovietsInAfghanistan.key.xml

Soviet-Afghan War 2017-01-10

1979 : Soviet deployment

The Afghan government, having secured a treaty in the spring and

summer of 1979 . They requested Soviet troops to provide security and

to assist in the fight against the mujahideen rebels. On April 14, 1979 ,

the Afghan government requested that the USSR send 15 to 20 helicopters

with their crews to Afghanistan, and on June 16 . . . The operation was fully

complete by the morning of December 28, 1979 . . . . According to the

Soviet Politburo they were complying with the 1978 Treaty of Friendship,Cooperation and Good Neighborliness . . . Soviet ground forces, under thecommand of Marshal Sergei Sokolov, entered Afghanistan from the north on

December 27 . In the morning , the 103rd Guards . . .

reference time identification in the text required

Temporal tagging of colloquial documents

(a) Yo! Rem to come for lab tmr:-) no need bring anything rite? 2010-09-23

Time4SMS corpus, 0019314.key.xml

(b) Whats it u wanted 2 say last nite? Did anna contact u? 2010-01-10

(c) Dear people, andy ng is availableat 10 am in his office. 2011-02-16T12:42

(d) Meet sunshine place on Thurs? 2010-10-05

(a) Yo! Rem to come for lab tmr :-) no need bring anything rite? 2010-09-23

(b) Whats it u wanted 2 say last nite ? Did anna contact u? 2010-01-10

(d) Meet sunshine place on Thurs ? 2010-10-05

noisy language and missing context information

Temporal tagging of scientific documents

Supplementation with all three macular carotenoids 2011-10-06Subjects consumed one tablet per day containing 10.6 mg MZ, 5.9 mg L and 1.2mg Z (Intervention, I group) or placebo (P group). . . . Subjects were assessedat baseline, two and four months. Clinical pathology analysis was performed atbaseline and four months.

Time4SCI corpus, 21979997.key.xml

Supplementation with all three macular carotenoids 2011-10-06

Subjects consumed one tablet per day containing 10.6 mg MZ, 5.9 mg L and

1.2 mg Z (Intervention, I group) or placebo (P group). . . . Subjects were as-

sessed at baseline , two and four months . Clinical pathology analysis was

performed at baseline and four months .

no real points in time, but local time frame

domaingroup of documents with

same characteristics relevant for temporal tagging

news- news articles- letters- formal emails

narrative- Wikipedia- descriptive

documents

colloquial- SMS- tweets- user-g. content

autonomic- scientific

documents- literary works

differences between documents of different domainsnormalization of relative and underspecified expressionstype of language (formal, scientific, historic, colloquial)different types of temporal expressions

documents

differences between documents of different domainsnormalization of relative and underspecified expressionstype of language (formal, scientific, historic, colloquial)

different types of temporal expressions

documents

corpus comparisonwith manually annotated corpora of four domains

Date Time Duration Set

HALLOHAlloHalloHallo

HALLOhallo

[%] news

narrative

HALLOhallo

[%] news

narrativecolloquial

HALLOhallo

[%] news

narrativecolloquialscientific

State-of-the-art in temporal tagging 2010

some research on temporal taggingannotation standards (e.g., TimeML [Pustejovsky et al. (2005)])

temporal taggers– only few available (e.g., GUTime [Verhagen & Pustejovsky (2008)])– if available, hardly extensible– focus on English– research only on news documents

there was a need for a domain-sensitive temporal tagger

Domain-sensitive normalization strategies

Remember: domain-sensitive normalizationWhy Time? Temporal Tagging HeidelTime Applications Temponyms Summary

Temporal tagging of news(-style) documents

normalization typically with document creation time

Why Time? Temporal Tagging HeidelTime Applications Temponyms Summary

noisy language and missing context information

no real points in time, but local time frame

reference time detection for underspecified expressions

underspecified(e.g., “December”)

news, colloquial

narrative

previouslymentionedexpression

autonomic

local time frame(previous expression

or new TPZ)

news, colloquial

narrative

autonomic

or new TPZ)

news, colloquial

narrative

autonomic

or new TPZ)

news, colloquial

narrative

autonomic

or new TPZ)

reference time detection for relative expressions

relative

context-independent(e.g., “today”)

news,colloquial

narrative

autonomic

local time frame(previous expres-sion or new TPZ)

context-dependent(e.g., “the following day”)

autonomic

local time frame(previous expres-sion or new TPZ)

news,colloquial,narrativepreviouslymentionedexpression

relation between underspecified expression and reference time

news, colloquial(ref=DCT)

no tense

before

colloquial

before

present/ future

narrative(ref 6=DCT)

autonomic(local time frame)

after(if not new TPZ)

no tense

before

colloquial

before

present/ future

no tense

before

colloquial

before

present/ future

no tense

before

colloquial

before

present/ future

no tense

before

colloquial

before

present/ future

no tense

before

colloquial

before

present/ future

The temporal tagger HeidelTime

Julian Zell MichaelGertz

[SemEval’10,’13, LREC’12,’14, LRE’13][EVALITA’14, EACL’14, TALIP’14, EMNLP’15]

characteristicsrule-based, multilingual, domain-sensitive

extractionbased on regular expressionslinguistic features (POS, POS of next token, ...)knowledge resources (names of months, holidays, ...)

normalizationlinguistic clues (tense in sentence, ...)domain-specific normalization strategies

4 domains: news, narrative, colloquial, autonomic

13 languagesmanually created resourcessome added by colleagues, some by other researchers

200 languagesautomatically created resources [EMNLP’15]

publicly available (UIMA, GATE, Java standalone, online demo)quite widely usedfirst domain-sensitive temporal taggeronly tagger for some of the languages

[Moriceau & Tannier (2014),Camp & Christiansen (2012),Skukan et al. (2014)]

easy to extend tofurther languages

AyserArmiti

Hui Li CanhVan Tran

[Moriceau & Tannier (2014),Camp & Christiansen (2012),Skukan et al. (2014)]

easy to extend tofurther languages

HeidelTime – extensive evaluationstate-of-the-art system for extraction & normalization

on all publicly available corporafor (almost) all languages and domains

winner of research competitions– TempEval-2 (en), TempEval-3 (en, sp), Evalita’14 (it)

[Verhagen et al. (2010), UzZaman et al. (2010), Caselli et al. (2014)]

no detailed numbers, but . . .

on all publicly available corporafor (almost) all languages and domainswinner of research competitions

– TempEval-2 (en), TempEval-3 (en, sp), Evalita’14 (it)[Verhagen et al. (2010), UzZaman et al. (2010), Caselli et al. (2014)]

on all publicly available corporafor (almost) all languages and domainswinner of research competitions

– TempEval-2 (en), TempEval-3 (en, sp), Evalita’14 (it)[Verhagen et al. (2010), UzZaman et al. (2010), Caselli et al. (2014)]

Value of HeidelTime’s domain-sensitive strategies

cross-domain evaluation

corpus strategy extraction normalization

narrative narratives

Don’t trust a tagger developed for news if you want toprocess narratives (e.g., Wikipedia)

now: some details to explain the “200 languages extension”

news news 91.1 78.6

narrative narratives

news news 91.1 78.6

narrative news 87.9 56.9narratives

news news 91.1 78.6

narrative news 87.9 56.9narratives 87.9 78.7

news news 91.1 78.6narratives 91.1 61.5

HeidelTime’s architecture

ResourcesSource Code

patternsnormalization knowledgerules

→ language-dependent

well-defined rule syntax (outside of source code)

domain-specificnormalization strategiesresource interpreter

→ language-independent

HeidelTime’s language resources

Pattern files:frequently used terms // Pattern resource

// for// month names// reMonthNameJanuaryFebruaryMarch...

// Pattern resource// for// month abbrev.// reMonthShortJan\.?Feb\.?Mar\.?....

// Pattern resource// for// month numbers// reMonthNumber1011120?[1–9]

Normalization files:contain normalizedvalues of such terms

// Normalization resource// for// month patterns// normMonth“January”,“01”“Jan\.?”,“01”“0?1”,“01”...

Pattern files:frequently used terms // Pattern resource

// for// month names// reMonthNameJanuaryFebruaryMarch...

// Pattern resource// for// month abbrev.// reMonthShortJan\.?Feb\.?Mar\.?....

// Pattern resource// for// month numbers// reMonthNumber1011120?[1–9]

Normalization files:contain normalizedvalues of such terms

// Normalization resource// for// month patterns// normMonth“January”,“01”“Jan\.?”,“01”“0?1”,“01”...

After read by HeidelTime, accessible by rules:

reMonthLong = (January|February|...)reMonthShort = (Jan.?|Feb.?|Mar.?|...)reMonthNumber = (10|11|12|0?[1-9])...

normMonth(January) = 01normMonth(Jan) = 01normMonth(1) = 01...

Rule files:every rule contains at least:

– rule name– extraction part – calls pattern– value normalization part – calls normalization functions

Temporal tagging of 200 languages [EMNLP’15]

so far: for 13 languagesmanual resource development for each added language

disadvantages:labor and time intensivelanguage knowledge requiredthere are many more languages not yet addressed

developing language resources automaticallysimplified English resources as starting point for translationsresource development process for “all languages”language-independent rules

Developing language resources automatically

resource development process for “all languages”

normMonthLong''January'',''01''''February'',''02''...

Simplified English resources

for each language

// spanish// reMonthLong

// spanish// normMonthLong

// german// reMonthLong

// german// normMonthLong

for each language

extractpatterns

JanuaryFebruary

for each language

extractpatterns

JanuaryFebruary

Wiktionary

February

translations = { german: Januar spanish: enero...}

January

for each language

// spanish// reMonthLong// ''January'',''01''enero// ''February'',''02''febrero

extractpatterns

JanuaryFebruary

// spanish// normMonthLong// ''January'',''01''''enero'',''01''// '' February'',''02''''febrero'',''02''

// german// reMonthLong// ''January'',''01''Januar// ''February'',''02''Februar

// german// normMonthLong// ''January'',''01''''Januar'',''01''// ''February'',''02''''Februar'',''02''

add patterns add normalizations

Wiktionary

February

January

language-independent rules (based on English rules)no language-dependent wordsadd “creative rules”, e.g., permutations of patterns

assumptionobviously, not all rules required for all languagesbut: “unnecessary” rules are unlikely to harm results

evaluationvery promisingbaseline for most of the 200 languages

The email-calendar application

Subject: meeting

Dear Prof,

13 Friday

Time for search engines

temporal information needs are frequentquery log analysis [Zhang et al., 2010]: 13.8% explicitly temporal

example information need: topic=“Olympics” time=“[1965,1974]”

web search with time constraints is difficult

temporal expressions considered as standard words1965 and 1974 make documents relevant1970 will not contribute to relevanceten years later [1970] not at all

Event-centric search [ICTIR’16]

DhruvGupta

KlausBerberich

ideaevents important in many textsevents as 〈 time, place, persons 〉

example search query: Max Planck Institute for Informatics

standardranked list ofdocuments

1. doc-12. doc-53. doc-8

our approachdocuments clustered along

important events

1990 (foundation): doc-1, doc-52000 (10th anniv.): doc-2, doc-12015 (25th anniv.): doc-8, doc-5

DhruvGupta

KlausBerberich

1. doc-12. doc-53. doc-8

important events

DhruvGupta

KlausBerberich

1. doc-12. doc-53. doc-8

important events

Time for summarization

relations between temporal expressions are ignored

documentIn 2015, something unimportant happened.One year later, something important happened.

useful summariesno underspecified/relative expressions without reference timeadd normalized values (e.g., One year later [2016])

useful summariesno underspecified/relative expressions without reference time

add normalized values (e.g., One year later [2016])

Time-centric corpus analysis [DHd’14, DH’15]

FrankFischer

world literatureGutenberg / Gutenberg.de corporaabout 550 authors, 2735 works (1510 – 1950)

When does (the German) literature take place?analysis of explicit month vs. day-month mentions

quiz:which month occurs most frequently?

FrankFischer

When does (the German) literature take place?

complete corpus

Sub-intervals Authors

The “Calendar boy” of world literature

Jules Verne

tiwoli – Today in World Literature

ThomasBögel

FrankFischer

select a day e.g., today

birthday invitation“I usually celebrate

my birthday onJanuary 11 with afew friends. Wouldyou like to come

along?”Hermann Hesse

tiwoli – Today in World Literature

ThomasBögel

FrankFischer

select a day e.g., today

birthday invitation“I usually celebrate

my birthday onJanuary 11 with afew friends. Wouldyou like to come

along?”Hermann Hesse

What about latent temporalinformation in texts?

Temponyms [WWW’16]

ErdalKuzey

VinaySetty

GerhardWeikum

not only “obvious” temporalexpressions as defined in

TimeML

ideastandard text phrases may be associated with temporal scopestemporal scopes can be found in KBs (or Wikipedia)

example phrases

the Cuban Revolutionary War 2008 Mexico City plane crash

temporal tagging results

–the Cuban Revolutionary War

20082008 Mexico City plane crash

temponym tagging results

[1953-07-26,1959-01-01]the Cuban Revolutionary War

2008-11-042008 Mexico City plane crash

temponym tagging results

[1953-07-26,1959-01-01]the Cuban Revolutionary War

2008-11-042008 Mexico City plane crash

temponyms add new or more precise temporal information

Temponyms

approachesunambiguous temponyms:

– HeidelTime extension [TempWeb’16]

all types of temponyms:– ILP-based approach with joint inference [WWW’16]

examples

his presidencyfederal election...

his presidencyClinton’sObama’sBush (jr)’sBush (sen)’s

federal electionCA 2008US 2008US 2012

Temponyms

examples

Temponyms

examples

ILP-based approach with joint inference [WWW’16]

first steps

identify temponym candidates– his presidency– federal election

find KB candidatesextract and normalize

– context entities– context temporal expressions

ILPs to find optimal temponymsbased on coherence and relatedness

measures

find KB candidatesClinton’s presidencyObama’s presidencyBush (jr)’s presidencyBush (sen)’s presidencyCA federal election ’08US federal election ’08US federal election ’12

contextBush, Washington2009, 2008,

first steps

measures

first steps

measures

first steps

measures

Summary

temporal information– frequent, important

temporal tagging– extraction and normalization of temporal expressions

domain-sensitive temporal tagging– different domains, different challenges– taggers tailored towards one domain, quality drop on others

many applications can benefit– in multiple ways

latent temporal information in texts– challenging, but useful to further enrich documents

Summary

Time for question answering

ZhenJia

GerhardWeikum

AbdAbujabal

RishiSaha Roy

MohamedYahya

various types of temporal questionsWhen was Bill Clinton born?Who won last year’s Superbowl?What money did they use in Spain before 2002?

this is ongoing work ...

Time for question answering

ZhenJia

GerhardWeikum

AbdAbujabal

RishiSaha Roy

MohamedYahya

various types of temporal questionsWhen was Bill Clinton born?Who won last year’s Superbowl?What money did they use in Spain before 2002?

this is ongoing work ...

Time for further information

github iOS & Android app CS library

Thank you! – Time for your questions...

Time for further information

github iOS & Android app CS library

Thank you! – Time for your questions...

Time for Text Mining and Information Retrievaljstroetge/talks/jannik-stroetgen... · Text Mining...

Documents

- UNTERSUCHUNG VON REALITÄTSNAHEN PROBLEMEN MIT FUNKTIONEN Christian, Jannik, Stefan EXTREMWERTPROBLEME

Feuerwehr Gundelfingen Jahresbericht 2014 · Johannes Haarhaus Jannik Häußler Manuel Löffler Daniel Palmtag Manuel Tschöpe Zur Oberfeuerwehrfrau: Madlin Huber ... Florian Hiesel

Jannik, Tun, Julie, Eva H1 Götter - Götter: Brahama, Vishnu, Shiva: - Brahama Welt erschaffen - Vishnu beschützt - Shiva Weltzerstörer

VERKNÜPFUNG VON ZWEI FUNKTIONEN Benjamin, Christian, Jannik Auswirkung auf Stetigkeit, Differenzierbarkeit und Integrierbarkeit

Pädagogischer Einsatz von Lernprogrammen im Unterricht der Grundschule Referat von Annika Drackert, Jannik Porstner und Corinna Schäfer

Mining-mining indicators

Texte : Jannik (sauf avis contraire) Mise en page et photos : Jannik et Michel … · 2010. 3. 21. · Nombre de milles parcourus : 4700 D’île en île en Grèce Ionienne Texte

Karine Mordal-Manet, Jannik Laval, Stéphane Ducasse To

Niels Emil Jannik Bjerrum-Bohr Niels Bohr International Academy Niels Bohr Institute

Mining Explained Glossary Mining 101 Mining Finance Guide

Environmental Dosimetry - HPS Chaptershpschapters.org/src/2014/1 Jannik-Environmental Dosimetry...1) Introduction to Environmental Dosimetry •Fields of Environmental Dosimetry and

Jannik Lindbæk jr.: Statoil i en norsk og global mediehverdag

Un calendario de la literatura española (aplicación para ... · 7 de octubre de 2015 Un calendario de la literatura española Fischer & Strötgen 21 / 21. Title: Un calendario de

Midtlandets Musik- og Teaterforening€¦ · KONCEPT & KOREOGRAFI - Jannik Elkær, Kristoffer Louis Andrup Pedersen PERFORMANCE & KOREOGRAFI - Petras Lisauskas, Jeppe Kaas Vad, Jesper

INDHOLD - WordPress.com · Aarhus Kommune/ Bo Fristed, ITK/ Projektejer Martin Vellerup/ Projektledelse Marianne Krogbæk/ Kreativ Proces Jannik Mulvad/ Biblioteksudvikling Sidsel

JANNIK GIGER Krypta - paladino store

#HACK4DK "Tal i tiden – data om samfund og sundhed fra 1960’erne til i dag" af Jannik Vestergaard Jensen

Virtuelle Forschungsumgebungen in Digitalen Bibliotheken Robert Strötgen

Fleksibel styring af SKATs inddrivelsesprocesser med ... · PDF fileFleksibel styring af SKATs inddrivelsesprocesser med regelmotorer: Evaluering af Frameworket Drools Jannik Lind

Indhold - Aarhus Bibliotekerne · aarhus kommune/ bo Fristed, itk/ projektejer martin Vellerup/ projektledelse marianne krogbæk/ kreativ proces Jannik mulvad/ biblioteksudvikling