Upload
liliana-m-melgar-estrada
View
110
Download
0
Embed Size (px)
Citation preview
A conceptual model for the annotation of
audiovisual heritage in a media studies context
Liliana Melgar, Marijn Koolen, Jaap Blom, Eva Baaren, Roeland Ordelman
Workshop “Audiovisual Data And Digital Scholarship: Towards Multimodal Literacy”
Digital Humanities
Krakow, 2016
What do “wish lists” and research have in common?
“Digital humanists are
motivated annotators” (Walkowsky & Barker, 2014)
Annotating is one of
the so-called
“scholarly primitives” (Unsworth, 2000)
A (oral) historian’s case
How cultural
activism is used
in the claim for
identity of “x”
ethnic group?
© AAC Library
© southernfoodways.org
...“My focus is on the
representations of the past as
acts of storytelling (or personal
“narratives”)”
The “Bricolage” of qualitative data analysis
Buster Keaton. 'Sherlock Jr.' (1924)
© Tumblr.com
“Moving images are a blind medium”(Sandom & Enser, 2001)
Added challenges of AV media
Analysis of our research use cases
Media
aesthetics
Media
representations
Cross-media
analysis
Social history of
the media
Social and
cultural history
Fan culture
studies
Genre studies
Discursive
representations
(media linguistics)
Visual
representations
Public debates
War and
conflict
Journalism
(data-driven,
narrative)
Postcolonial
studies
Migration
Desmet case
Reality tv
Memories from
the Moroccan-
Berber activists in
NL/FL
Muslims/islam
representations
Amateur
film/video
Youtube
broadcasts
World War IIIndonesian
decolonisation
Public contention
Refugee crisis
RQ:
How to provide
system support
for manual
annotation tasks
of AV media in
the context of
Digital
Humanities
infrastructures?
Requirements
analysis
Research use
cases
Method
Use case / requirements analysis: one to one interviews with media scholars
Literature review
Analysis of the data models, functionalities and interface features of current
tools that support video annotation
CAQDAS VIDEO ANALYSISPROFESSIONAL
VIDEO EDITORS
DOMAIN SPECIFIC
APPLICATIONS
GENERIC WEB AV
ANNOTATION
➔ Transana
➔ Nvivo
➔ Atlas.ti
➔ Anvil
➔ Advene
➔ Elan
➔ Lignes du temp
➔ Final Cut Pro
➔ Adobe’s
Premiere ProCC
➔ Cinemetrics
➔ Linked TV
➔ ArtTube
➔ Youtube
➔ Synote
Preliminary Outcomes
Study of the information
annotating behavior of
media scholars
Analysis of existing tools for
video annotation
A process model
(use case-driven)
AV media-centered data
model
A concept model
Study of the information
annotating behavior of
media scholars
Analysis of existing tools for
video annotation
A process model of
annotation in a research
workflow (use case-
driven)
A media-centered model
of AV annotation
(data-oriented)
A concept -integrative
model of annotation
Preliminary Outcomes
Conceptual
model
Object:
Media
Document
Actor
Annotation
(output)
Process
Task
Motivation
Context
Conceptual
model,
dimensions
Scholar/
(oral)
historian
Thematization
ManualInterpreting
Themes / Narratives
Oral history interviews
Academic research
Current Open
Annotation Data Model
Current Open Annotation Data
Model for scholarly research
The Open Annotation
Collaboration (OAC)
Project (Hunter et al., 2010)
Adds context and agent = Actor
Method, task, and purpose (in stages)
Previous work has identified stages in the
scholarly research process
Bron et al., 2012. “Overview of the phases in the media studies
research cycle with associated search processes and changes in
the research question (RQ). Arrows indicate possible sequences.”
Traditional scholarly
research process in
the previous use
cases include an
“analysis” phase,
which is mostly done
through manual
annotations, named
as:
-Coding (Grounded
theory analysis)
-Thematization
(narratives)
-Content analysis
Previous work has also identified stages in search process
(Huurdeman & Kamps, 2014; Koolen…)
?RQ
-Storytelling
-Reporting
Exploratory data
analysis
Data
preparationData enrichment
Explore (other)
datasets
Research phases & annotation process
Pre-focused annotation➔ Bookmarking
➔ Open coding (initial coding)-
-tagging
➔ Commenting (memos,
tasks)
Focused annotation➔ Focused coding
➔ Classification/
Thematization /
Narrative
➔ Codebook
➔ Commenting
(analytical memos)
Annotation preparation
➔ Defining coding
schemes (layers)
➔ Select data
enrichment services
per layer
Based on studies about research stages (Bron et al., 2015), search stages (Huurdeman & Kamps, 2014, Koolen et al., 2015); Qualitative data analysis theory (Charmaz, 2006); Concept of
annotation (Agosti et al., 2012; Melgar, 2016); CLARIAH use cases (interviews with media scholars) and discussions during CLARIAH WP5 requirement analysis.
Corpus selection Corpus analysis & enrichment
ExplorationContextualization
(Assembling)PresentationAnalysis
A (oral) historian’s case
How cultural
activism is used
in the claim for
identity of “x”
ethnic group?
Data
preparationExplore
(other)
datasets
Data enrichment
Exploratory data
analysis
Focused
annotation
Corpus selection Pre-Focused
annotation
Conclusion and future work
• We need to approach “annotation” in a broader scope
• We have come up with three models for annotation:
• A conceptual model of annotation
• A process model(s)
• A media-centered, data-oriented model
• Future work includes
• Implementation of our model in the CLARIAH services
• Evaluation with media scholars
• New requirements and further development for each
research use case
References
Bron, Marc, Jasmijn van Gorp, and Maarten de Rijke. “Media Studies Research in the Data-Driven Age: How Research Questions
Evolve.” Journal of the Association for Information Science and Technology, 2015, n/a-n/a. doi:10.1002/asi.23458.
Huurdeman, Hugo C., and Jaap Kamps. “From Multistage Information-Seeking Models to Multistage Search Systems.” In Proceedings of
the 5th Information Interaction in Context Symposium, 145–154. IIiX ’14. New York, NY, USA: ACM, 2014.
doi:10.1145/2637002.2637020.
Koolen, Marijn, Toine Bogers, Antal van den Bosch, and Jaap Kamps. “Looking for Books in Social Media: An Analysis of Complex
Search Requests.” In Advances in Information Retrieval - 37th European Conference on IR Research, ECIR 2015, Vienna, Austria,
March 29 - April 2, 2015. Proceedings, edited by Allan Hanbury, Gabriella Kazai, Andreas Rauber, and Norbert Fuhr, 9022:184–
196. Lecture Notes in Computer Science, 2015. doi:10.1007/978-3-319-16354-3_19.
Sandom, Christine, and P.G.B. Enser. “VIRAMI: Visual Information Retrieval for Archival Moving Imagery.” Milano, Italy: Archives &
Museum Informatics, 2001. http://www.archimuse.com/publishing/ichim01_vol1/sandom.pdf.
Unsworth, John. “Scholarly Primitives: What Methods Do Humanities Researchers Have in Common, and How Might Our Tools Reflect
This?” London: King’s Collegue, 2000. http://people.brandeis.edu/~unsworth/Kings.5-00/primitives.html.
Walkowski, Niels-Oliver, and Elton T.E. Barker. “Digital Humanists Are Motivated Annotators.” Laussane, Switzerland, 2014.
http://dharchive.org/paper/DH2014/Paper-296.xml