Upload
herbert-furness
View
218
Download
1
Embed Size (px)
Citation preview
A Stepwise Modeling Approach for Individual Media Semantics
Annett Mitschick, Klaus MeißnerTU Dresden, Department of Computer Science, Multimedia Technology Group
„Applications of Semantic Technologies“ (AST' 06)
06.10.2006 AST 2006 2
Outline
• Introduction• Approach:
- Semantics for Personal Media Management
- Information Instantiation and Integration
- Implemented System Architecture
• Conclusion and Future Work
06.10.2006 AST 2006 3
Introduction
• Media Management @ Home:- Growing amount of digital assets (text,
images, video, audio)• complex, high-dimensional
- User: • non-/semi-professional• individual conceptualizations • usually minimum efforts to
annotate/organize
- Tools mostly apply administrative (objective) metadata vs. user interpretation/conceptualization
• Goal: automated modeling of essential semantic descriptions from existing metadata to ease annotation work
06.10.2006 AST 2006 4
Introduction
• Solutions:- Automated feature extraction and indexing techniques to manage
complexity of huge multimedia collections• low-level features Similarity-based Search (Query-by-Example, etc.)• higher-level features Pattern Recognition/Classification (objects,
people, genres, etc.) requires prototype features (user context, user feedback)
“Semantic Gap”
- Media descriptions based on common metadata standards to enable interoperability & comparability
• MPEG-7 Multimedia Description Schema• RDF/OWL representation for semantic modeling
- Semantic Desktop• Personal Information Management based on Semantic Web Technologies
06.10.2006 AST 2006 5
Approach :: Semantics for Personal Media Management
• Knowledge representation - Allow description of background-knowledge (context information like
events, activities, actors, locations, etc.)
- Allow integration of domain models and individual extensions (e.g. integration of MPEG-7 MDS, FOAF for description of people, etc.)
- Manageable, straightforward for non-professionals
• ABC [Hunter, et al.]- “facilitate interoperability between metadata ontologies“, “conceptual
basis for automated mapping …”, “guidance to communities…”
- Base classes to attach domain-specific entities MPEG-7 MDS
- Temporal semantics and related actualities (who, what, when, where) evolution/transition of objects
06.10.2006 AST 2006 6
Approach :: Semantics for Personal Media Management
• Acquisition of as much information as possible from content + context
- File information, administrative metadata, structural features, etc.
• Information sources depend on application context- naming, availability/accessibility, semantics, etc. of features/attributes
e.g. “Date Time Original”/”Date Created”/…, “Creator”/”Author”/…, etc.
- properties might be relevant, or redundant in some contexte.g. “EXIF:ExposureTime” and “EXIF:ShutterSpeedValue”
Metadata/FeatureExtraction
EXIFIPTC MPEG-7 ...
Metadata/FeatureExtraction
ID3CDDBMPEG-7 ...
Metadata/FeatureExtraction
DCXMPBibTex ...
Metadata/FeatureExtraction
SMILSMPTEMPEG-21 ...
06.10.2006 AST 2006 7
• First step: - Apply sets of mapping rules to construct media-centric description
(MDS)
- Mapping rules expandable regarding supported metadata schemes( developer)
Approach :: Information Instantiation and Integration
MDS
Media-Analysis
EXIF XMP IPTCDC ...
R2
III
R1
III
ABC
RDF-Repository
Make=FUJIFILMModel=FinePix F601
[(urn:Exif:Make rdf:value ?j) (urn:Exif:Model rdf:value ?k) (urn:Filepath rdf:value ?l) hashCode(?l, ?t) makeTemp(?x) -> (?t mds:creationTool ?x) (?x mds:make ?j) (?x mds:model ?k) (?x rdf:type mds:Device)]
<rdf:Description rdf:about="urn:Exif:Make"> <rdf:value …>FUJIFILM</rdf:value></rdf:Description><rdf:Description rdf:about="urn:Exif:Model"> <rdf:value …>FinePix F601</rdf:value></rdf:Description>
<rdf:Description rdf:about="#ins61737146"> <mds:creationTool rdf:nodeID="A0"/> ...</rdf:Description> ...<rdf:Description rdf:nodeID="A0"> <mds:make>FUJIFILM</mds:make> <mds:model>FinePix F601</mds:model> <rdf:type rdf:resource="#Device"/></rdf:Description>
06.10.2006 AST 2006 8
Approach :: Information Instantiation and Integration
• Next steps:- Apply derivation rules (schema level)
to establish relationships to actualities and temporalities( developer, user)
- Apply classification rules (instance level)to refine description model regardingknown pattern ( user)
MDS
Media-Analysis
EXIF XMP IPTCDC ...
R2
III
R1
III
ABC
RDF-Repository
06.10.2006 AST 2006 9
Approach :: Implemented System Architecture
• Design rationale:- Automated import and indexing of media items triggered by file system
actions
- Registry of media type specific analyzing components providing metadata and feature extraction
- Processing of semantics and ontology model wrapped by generic model API independent from RDF/OWL storage and querying solution (Jena, Sesame, …)
- Extensible, flexible service architecture plug-in based setup, extension at run-time(OSGi service platform)
MediaSemantics
Image-Analyzer
Text-Analyzer
Audio-Analyzer Model
Image-Semantics
Text-Semantics
Audio-Semantics
Jena RDF Framework
Rule Manager
RDF-Repository
MediaImporter
GUI
Media Repository
06.10.2006 AST 2006 10
Conclusion and Future Work
• Our approach:- Part of the K-IMM (Knowledge through Intelligent Media Management)
project managing private media collections with the help of semantic technologies
- Stepwise modeling of resource descriptions to separate distinct problems of construction of media semantics
• Building objective media descriptions from various sources• Deriving subjective, user-oriented semantics
• Future Work:- Strategies to extend and refine basic rule sets by means of user
feedback, context capture- Distribution of modeling steps (Web Services/Agents)- Detailed, comprehensive evaluation
06.10.2006 AST 2006 11
Thank you for your attention!
Annett [email protected]
Multimedia Technology Group, Department of Computer Science, TU Dresdenhttp://www-mmt.inf.tu-dresden.de