Upload
marco-brambilla
View
1.062
Download
0
Embed Size (px)
DESCRIPTION
As the Web becomes a platform for multimedia content fruition, audiovisual search assumes a central role in providing users with the content most adequate to their information needs. A key issue for enabling audiovisual search is extracting indexable knowledge from opaque media. Such a process is heavily constrained by scalability and performance issues and must be able to flexibly incorporate specialized components for educing selected features from media elements. This paper shows how the use of a model-driven approach can help designers specify multimedia indexing processes, verify properties of interest in such processes, and generate the code that orchestrates the components, so as to enable rapid prototyping of content analysis processes in presence of evolving requirements.
Citation preview
Model-Driven Design of Audiovisual Indexing Processes
for Search-Based Applications
Alessandro Bozzon, Marco Brambilla, Piero Fraternali
http://dbgroup.como.polimi.it/brambilla/mdd-search-apps
CBMI 2009, June 5th, Chania, Crete.
2
Model-driven design of search applications
Agenda
• Introduction• Search-based applications (SBAs)• The Content Provision and Annotation (CPA) process
Definition Verification of properties
• Conclusions
3
Model-driven design of search applications
3Introduction
• Search has become the prominent paradigm for information seeking across both online spaces and enterprises.
• Search-Based Applications (SBAs)
searching over heterogeneous data constitutes the predominant user interaction paradigm
Search Engine
•canned applications
•fixed behavior•homogeneous data source
•simple content processing flows
•basic query flows
SBA
• tailor-made solution, depending on varying data and user's needs nature
• search engines are part of a complex system
• data source integration
• content analysis
• querying
• Web-mediated social interactions
4
Model-driven design of search applications
4Motivations
• SBAs are complex application integrating: complex front-end: query expression, result presentation, personalization,
adaptation complex back-end: content provisioning, annotation, indexing, distributed query
execution
• SBAs embody data-intensive and process-intensive tasks
• content items, queries, and result lists are first-class objects, subject to a life cycle
• Content Provisioning and Annotation process (CAI process)– indexation of contents coming from the application data sources (data
retrieval from external sources, transformation or aggregation, analysis, indexation)
• Query and Result Presentation process (QRP process)
– operations related to query analysis, execution, orchestration and result-set composition
• User Interaction process (UI process)
– the way users interact with the application's functionalities ( statement of information needs, result navigation, social interactions)
5
Model-driven design of search applications
5Thesis
• Current practices lack complete tools and methodologies• rely either on very simple models or on the programming skills
of the developers low separation of concerns among involved actors low productivity in the implementation phase problems in managing and maintaining applications over time
• SBA development demands for the same evolution in methods and tools that has characterized in the recent past the progress of Software Engineering for Web applications clear separation of concerns among the involved actors central roles of models as key development artifacts automatic code generation, etc.
SBA Model Driven Development Framework
6
Model-driven design of search applications
6The contribution
A modeling framework
• an intuitive Rich Process Model to specify the schema of the CPA process for a given application,
• use of formal methods to investigate properties of interest of his design
• code generation techniques to implement and deploy the process
7
Model-driven design of search applications
7Background
• Information Retrieval ”... the process of finding contents (e.g., textual documents, multimedia items, etc.)
that satisfy an information need from within large collections (usually stored on computers)...”
• Business Process Design representing processes (of heterogeneous nature) in terms of related, structured
activities or tasks that produce a specific service or product several proposals for visual modeling languages (e.g., UML, YAML, BPMN)
• Model Driven Web Engineering raise of abstraction (separation of platform independent and platform dependent
concerns) in Web application design and development use of models (and model transformations) as the key artifacts for application
developments several proposals (e.g., UML, Hera, OOHDM, UWE, W2000, WebML)
• Formal methods Formal semantics to models Automatic verification
8
Model-driven design of search applications
8Background Models
BPMN WebML
Business Process Model Notation•Activities
•FlowsConstraints (OR-XOR-AND gateways)
•Artifacts (Data Objects and data associations)
•Events
Web Modeling Language•Domain ModelsNavigation Models ( content publication and manipulation, link behavioral semantics)
•Service composition and orchestration Models ( Web service invocation and publication, XML management)
9
Model-driven design of search applications
9Modeling Approach
Process Models: BPMN (CAI, QRP), Interaction Pattern Composition (UI)
Domain data and process metadata: ER/UML
Application Models: WebML
Model To Model Transformation: Java / ATL
Model To Code Transformation: Java
10
Model-driven design of search applications
10CPA Process - Domain Model
Content model: the objects that relate to the Content Items indexed by a search application
Annotation model: structure of the annotations associated with searchable Content Items during the indexing process
Usage model: user, roles, permissions
Index model:
abstraction for the actual physical implementation of search engine indexes
11
Model-driven design of search applications
11CPA Process - Process and Application design
Coarse CPA process model Content Registration Content Analysis Content Indexation
Fine-grained process model Analysis of audiovisual content
trough face recognition and identification technologies
Application model Invocation algorithm of annotation
technologies
Re
fin
em
en
tM
2M
Tra
ns
form
ati
on
M2
T T
ran
sfo
rma
tio
n
Running CPA process Console trace of the working
annotation technology Process advancement control UI
Full Size
Video
12
Model-driven design of search applications
12CPA - Examples of complex processes
Analysis of audiovisual content
Analysis of audio-visual content with textual annotations
13
Model-driven design of search applications
13Search-specific extensions to the process model (BPMN*)
• BPMN does not provide support for:
• process data• data flows and dependencies among activities
does not convey domain-specific information about the modeled process for activity’s business logic
• Formalized extensions to the BPMN language, introducing: typed attributes (content, annotation, process) typed activities activity properties
• type of produced and consumed data• type of enforced operations (classified w.r.t. their input/output)
finer-grained data flows• mapping of links with guard conditions and parameters
14
Model-driven design of search applications
14BPMN* > WebML transformation
• More detailed transformation rules finer-grained Application Model, needing less (or no) refinement by the
designer
• typed activities enables reusable PIM models
• data dependencies are specified at an higher level less errors in Application Model design fastest SBA development
ATL Transformation
Example of ATL BPMD-WebML transformation rule for activities
15
Model-driven design of search applications
15Verification of properties for CPA processes
• Assessment of CPA process outputs Algebraic representation
• Expression of an activity depend on the type (number and type of input/output) of the activity and on the schema (activity pre-condition, mutual exclusions or indetermination in setting activity inputs) of the process
The expression evaluation requires assigning the semantics to each activity• functions from input to output values, described extensively, as tabular expression (or
ground-truth)
Variation of activities’ business logics (or in the process schema) produces variations in the representativeness of annotations assessment of produced annotation quality, validation against expected outcomes, compare
outcomes of different processes
indexAnnotation(AggregateAnnotation(
AnalyzeAudio(TranscodeAudioVideo(RetrieveVideo()).audioOut),
AnalyzeFaces(TranscodeAudioVideo(RetrieveVideo()).videoOut)))
16
Model-driven design of search applications
16Implementation
• Extensions to a commercial tool for the automatic generation of Web applications
• Creation of a Web-based BPMN editor for the specification of the CAI and QRP process models
• Set of model to model and model to text transformations
17
Model-driven design of search applications
17Experiences
• PHAROS (Platform for searcH of Audiovisual Resources across Online Spaces) project First-hand experience on a large scale SBA application
• Adoption of models in the platform design
– Domain model
– Query and result presentation process
– User interaction process
– Experiments with Content Analysis and
Indexing process Developer evaluation
Pros Cons
Complete set of design dimensionsHigh-quality prototypesQuick prototyping cyclesFast development time
Lack of reverse transformations from application to process modelsAvailability of only simple guard condition expressions
18
Model-driven design of search applications
18Conclusions and future work
A modeling framework
• an intuitive and visual design of the CPA process,
• Helps the initial design of the process
• Helps for its evolution when requirements change
• the extensibility of the model, based on a plugin approach
• the amenability to automatic verification
• for checking properties
• for estimating the characteristics of a specific CPA configuration
• the availability of well-established code generation technology
• great improvement in productivity
Future work
• extension of the validation capabilities
• industrial implementation
19
Model-driven design of search applications
19
Thank You!
Marco [email protected]
www.dei.polimi.it
http://www.pharos-audiovisual-search.eu/
19