26
EURECOM July 2012 Agora: putting museum objects into their art-historic context Marieke van Erp [email protected]

Agora: putting museum objects into their art-historic context

Embed Size (px)

DESCRIPTION

The digital era has presented big challenges, but also great opportunities for the museum world. One of these opportunities is the way museums can open up their collections to the public. Many museums are now actively exploring possibilities to present their collections online for visitors who cannot come to the museum, or to show objects for which they do not have space in the exhibition halls. Often they will put together themed Web sites for online exhibitions in which objects are presented in a certain context. However, these themed Web sites usually only cover only a small part of their collection. For the majority of the objects, the context is not made explicit. In the Agora project, we aim to make this context explicit in an automatic way in order to help users understand and interpret museum objects. We do this by linking museum objects to historical events and explicitly presenting these links in an event-driven browsing environment. In the first part of my talk, I will explain the theoretical framework we have developed in the Agora project to represent historical contexts as well as the general challenges to the project. In the second part of my talk, I will focus on the particular challenges in information extraction for building the event thesaurus and linking museum objects. These slides are from a presentation given at the Eurecom seminar on July 20 2012

Citation preview

Page 1: Agora: putting museum objects into their art-historic context

EURECOM July 2012

Agora: putting museum objects into their art-historic context

Marieke van [email protected]

Page 2: Agora: putting museum objects into their art-historic context

Introduction

• BA, MA & PhD Computational Linguistics/Information Extraction @Tilburg University

• Since 2009: SemWeb group @VU University Amsterdam

Page 3: Agora: putting museum objects into their art-historic context

Overview

• The Agora Project

• Digital Hermeneutics

• Building an Event Thesaurus for Dutch• Experiments & Results• Outlook

Image src: http://www.artrage.com.au/dreamgirl/filesend/223/EarthFromAbove_EXPOTVDC212_prog.jpg

Page 4: Agora: putting museum objects into their art-historic context

• Collaboration VU CS & History departments, Netherlands Institute for Sound and Vision and Rijksmuseum Amsterdam

• Facilitate and investigate digitally mediated public history

The Agora Project

Page 5: Agora: putting museum objects into their art-historic context

Digitising Heritage

• Galleries, libraries, archives and museums (GLAMS) are digitising their data and presenting it online• This changes the role of GLAMS

from information interpreters to information providers• In the online setting, objects can

easily start to lead their own lives

Image source: http://terracebay.library.on.ca/wp-content/uploads/2011/04/clip_image002.jpg

Page 6: Agora: putting museum objects into their art-historic context
Page 7: Agora: putting museum objects into their art-historic context

Digital Hermeneutics

• An object on its own has no meaning; event descriptions provide historical context• A single event only gives part

of the historical context; chains of events (narratives) provide a more complete overview

Image src: http://3.bp.blogspot.com/-7nXcVdW0_wc/Th0JDRIT1GI/AAAAAAAAIEk/IoPReKrojkY/s1600/42st.jpg

Page 8: Agora: putting museum objects into their art-historic context

sem:Actor

sem:Place

Yogyakarta

sem:Event

19/12/1948

The Attack on Yogyakartardf:type

rdf:type

rdf:type

sem:hasActor

agora:depictsEvent

sem:hasBeginTimeStamp

Netherlands

Painting: Three Fighter Aircraft in the Sky

Mohammed Toha Paints "Three Fighter Aircraft in the Sky"

agora:createsEvent

sem:Actor

rdf:type

sem:hasActor

Mohammed Toha

sem:Eventrdf:type

rma:creationDate

sem:hasPlacesem:hasPlace

sem:hasBeginTimeStamp

rma:maker

rma:creationPlace

Event Dimension

Page 9: Agora: putting museum objects into their art-historic context

Indonesia

1945 - 1946

The Attack on Yogyakarta

KNIL

sem:hasPlace

sem:eventType

Armed Conflict

Sumatra

19/12/1948 - 31/12/1948

Operation Crow

KNIL

sem:hasPlace

sem:eventType

Armed Conflict

Yogyakarta

01/03/1949

The Attack on Yogyakarta

KNIL

sem:hasPlace

sem:eventType

Attack

agora:hasBiographicalRelation

agora:hasBiographicalRelation

sem:hasActor

sem:hasActor

sem:hasActor

sem:hasTimeStamp

sem:hasTimeStamp

sem:hasTimeStamp

Narratives

Page 10: Agora: putting museum objects into their art-historic context

Event-driven Browsing

Page 11: Agora: putting museum objects into their art-historic context

Event-driven Browsing

Page 12: Agora: putting museum objects into their art-historic context

Event-driven Browsing

Page 13: Agora: putting museum objects into their art-historic context

Building an Event Thesaurus

• There are no extensive structured event descriptions• Rijksmuseum Amsterdam has a

flat list of 1,693 ‘events’: only names and very much focused on 17th century Holland • Our goal: • create a list of historically

relevant events• provide actors, locations,

times & types for each event

Image src: http://www.collinsdictionary.com/static/graphics/default.png

Page 14: Agora: putting museum objects into their art-historic context

First Attempt• Pattern based event-name

extraction• In Dutch Wikipedia we

found 2,444 event candidates • 1209 (56.3%) correct• 169 (13.9%) partially

correct• Off-the-shelf named entity

recognition (P/R/F1)• Person 77/77/77• Location 75/58/66• Organisation 32/37/34

Image src: http://www.spaceg.com/multimedia/collection/explosions/atomic%20explosion%205.jpg

Page 15: Agora: putting museum objects into their art-historic context

First Attempt• Co-occurrence based event-

relation finder• only actor, location and/

or date found for 392 events• 49.6% actor is correct• 41.1% location is correct• 51.5% date is correct

Image src: http://www.spaceg.com/multimedia/collection/explosions/atomic%20explosion%205.jpg

Page 16: Agora: putting museum objects into their art-historic context

First Attempt

• Problems event element recognition:• Shallow grammatical

processing (post-war rebuilding and during the North sea flood recognised as 1 event)

• Missing locations (Battle of LOC pattern fails)

• No distinction between entities and action nouns (German Occupancy vs German Occupants look the same for the approach)

• Named Entity Recogniser not suited for domain

Image src: http://www.spaceg.com/multimedia/collection/explosions/atomic%20explosion%205.jpg

Page 17: Agora: putting museum objects into their art-historic context

First Attempt

• Problems event relation finder:• Relies on redundancy in

the data, only works for ‘popular’ events• Too coarse-grained (who

were the actors/locations in WWII)• Evaluation is hard!

Image src: http://www.spaceg.com/multimedia/collection/explosions/atomic%20explosion%205.jpg

Page 18: Agora: putting museum objects into their art-historic context

Back to the drawing board...• Analysis of event names

• Combinations of sortal nouns with a PP and a named entity e.g., Battle of Stalingrad, Death of John Lennon

• Combinations of nominalized verbs with a PP and a named entity e.g, Excavation of Troy, Election of Obama.

• Combinations of a referential adjective with an event type and named entity e.g., the American invasion of Iraq.

• Transparent proper names: Great War

• Opaque proper names: Event names that can not be decomposed on morphological grounds e.g., Holocaust, Spanish Fury

Image src: http://www.northescambia.com/wp-content/uploads/2010/01/molinotrashfire10.jpg

Page 19: Agora: putting museum objects into their art-historic context

Back to the drawing board...

• Improve Named Entity Recognition• Add gazetteers for

historical names• Post-processing for titles

and improved NE boundaries

Image src: http://www.northescambia.com/wp-content/uploads/2010/01/molinotrashfire10.jpg

Page 20: Agora: putting museum objects into their art-historic context

Back to the drawing board...

• Finding Event Relations• Use structure Wikipedia/

DBpedia• Shallow parsing• Hierarchies of actors &

locations

Image src: http://www.northescambia.com/wp-content/uploads/2010/01/molinotrashfire10.jpg

Page 21: Agora: putting museum objects into their art-historic context

Current Work

Spotlight (P/R/F)

Person

Location

Organisation

54.05/7.52/13.20

64.52/30.77/41.67

0/0/0

Stanford (P/R/F1)

58.60/34.46/43.40

67.19/66.15/66.67

9.78/25.71/14.17

Freire (P/R/F1)

79.17/71.16/74.95

80.00/61.54/69.57

89.66/74.29/81.25

• Still some work to be done, but Freire et al. (2012) shows that smart features can work with small amounts of training data• Combine classifiers• Add post-processing• MISC Class remains to be done...

Page 22: Agora: putting museum objects into their art-historic context

Current WorkWord POS CHUNK NERU.N. NNP I-NP I-ORG official NN I-NP O Ekeus NNP I-NP I-PER heads VBZ I-VP O for IN I-PP O Baghdad NNP I-NP I-LOC . . O O

focus,minthree,mintwo,minone,plusone,plustwo,fnfreq,lnfreq,ncfreq,orgfreq,geo,n,v,a,adv,pn,cap,allcaps,beg,end,length,capfreq,class"is","wood",")","and","painted","dark",0,0,0,2.45253198865684,0,0,0,1,0,0,0,0,0,0,2,0,"O""painted",")","and","is","dark","grey",0,0,0,0,0,0,0,0,1,0,0,0,0,0,7,0,"O""dark","and","is","painted","grey",".",0,0,0,0.493875418347986,0,0,1,0,1,0,0,0,0,0,4,0,"O""grey","is","painted","dark",".","William",0,0,0,0.0768052510316108,0,1,1,1,1,0,0,0,0,0,4,0,"O"".","painted","dark","grey","William","Herschel",0,0,0,2.36647279037729,0,0,0,0,0,0,0,1,0,0,1,0,"O""William","dark","grey",".","Herschel","made",8.2034429051892,3.27892030900003,0,4.67158565874127,0,0,0,0,0,0,1,0,0,0,7,0,"B-PER""Herschel","grey",".","William","made","many",2.36726761611533,2.39936346938848,0,0.443930767784,0,1,1,0,0,0,1,0,0,0,8,0,"I-PER""made",".","William","Herschel","many","telescopes",0,0,0,0.493875418347986,0,0,0,1,1,0,0,0,0,0,4,0,"O""many","William","Herschel","made","telescopes","of",0,0,0,0.0768052510316108,0,0,0,0,1,0,0,0,0,0,4,0,"O""telescopes","Herschel","made","many","of","this",0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,0,"O"

[CoNLL2003]

[Freire et al. 2012]

Page 23: Agora: putting museum objects into their art-historic context

Current Work

• Build smarter extractors for event names• First focus on ‘regular’

event names (e.g., Battle of LOC, War of YEAR) • Use knowledge about

action nouns vs static nouns (WordNet)

Page 24: Agora: putting museum objects into their art-historic context

The Story So Far

• It takes time to learn to communicate in an interdisciplinary project• Don’t try to solve too much

in one go • Cycles of error analysis • Domain adaptation is difficult:

optimise for precision

Page 25: Agora: putting museum objects into their art-historic context

Outlook

• Redesign of Agora demo (new version autumn/winter) • Include different perspectives

(together with Semantics of History)• Ship model use case• Historical Named Entity

Recognition for English & Dutch • 2nd round user studies (spring

2013)