Upload
vrt-medialab
View
1.803
Download
0
Embed Size (px)
Citation preview
PISAProduction, Indexing and Search
of Audio-visual Material
Computer Assisted ManufacturingDieter Van Rijsselbergen, IBBT – MMLab – UGent
35
PISA - Overview
Abstract
Information
Footage
Concept
Virtual
Model
Model Driven Development:
• Setting (Stage properties, light)
• Character
• Synthetic Speech
• Sound effects
• Character animation
• Virtual camera
VirtualModelling
AutomatedProductionRealisation
• Ingest
• Editing
• Mastering
• Reproduction to alternative distribution channels
Computer Assisted Design
Computer Assisted Manufacturing
Script Editing
• Parse scenario
• Shooting script editor
• Storyboard
Script Editing
Reverse Engineering
• Shot segmentation
• Video footprint and reuse detection
• Biometric face detection
• Background analysis
• Speech-to-text
Interpretation
• Character identification
• Background categorisation
and identification
• Topic and eventdetection
Intelligent Analysis and
Quantization
Quantization
Analysis
Indexing
Retrieval
• Timecode based indexing
• Geo-temporal reference
• Taxonomy based indexing and search
• Facetted search
Search Engine
36
Computer Assisted Manufacturing
! Computer Assisted Manufacturing is about automation
of media production processes
! Based on extensive production metadata
! high level Product Engineering for narratives
! low level Manufacturing Engineering for actual
manufacturing specifications
! Using clever algorithms to fill in the gaps
! supported by Trouvaille
37
Drama Manufacturing Processes
Product Engineering
Manufacturing Engineering
Manufacturing
Synopsis
Writing
3-D
Previsualization
Dialogue
Writing
Shooting
ScriptingEditing
Quality
Assurance
Acquisition? ? ?
38
Objectives
! Extend manufacturing workflow with 3-D previsualization
! Build evolving products
! Re-use previsualization parameters
during manufacturing
! Increase efficiency and automation in production
processes
! 1) Rough Cut Editing based on 3-D previsualization
! 2) Repurposing for multiple output channels
Computer Assisted
Manufacturing
Rough Cut Editing
40
Rough Cut Editing
! Inclusion of 3-D previsualization into thedrama manufacturing process
! Use previsualization as shooting script
! Integration into the MAMS and Craft Editing systems
! Building evolving products
!mixture of previsualized and shot footage
! evolve from all-previsualized to all-shot composition
41
Rough Cut Editing Processes
Product Engineering
Manufacturing Engineering
Manufacturing
Synopsis
Writing
Previz.
Rendering
Dialogue
Writing
EditingQuality
Assurance
Acquisition
3-D
Previsualization
Computer Assisted
Manufacturing
Repurposing
43
Computer Aided Repurposing (CAR)
! Automated Repurposing of Drama
! Different output channels
!HDTV
!SDTV
! iPod/Cellular
! True Universal Media Experience
! requires semantically sensible scaling/cropping
44
Repurposing Concept
! Automated Repurposing of Drama
! Spatial adaptation using semantic Objects of Interest
! as in “focus on Capt. Kirk”
! decided by creative staff in function of the narrative
! is a form of Manufacturing Engineering
! can be defined in early 3-D Previsualization
! preparation not only of shooting script
but also of adaptations
! parameters refined in Quality Assurance
45
Repurposing: How?
! 1) Determine where the semantic elements are
! Semantic Objects of Interest
are linked with regions of interest
!manual annotation
! automated
algorithms
! aided by
Product
Engineering
metadata!Jenny Waldek
46
Repurposing: How?
! 2) Provide reframing instructions, e.g.:
! Object Orientation and Range
! “The Bad must be shown on the right…”
! Object Size
! “… and I want it to be a full shot of The Bad.”
0.75
Frame SizeObject of Interest
Size: (30%, 90%)
47
Repurposing: Channels
! HDTV
! Originally acquired
high definition material
! SDTV
! Legacy applications
! Common pan/scan
! Editing
(AVID)
! Mobile (iPod)
! Semantically
repurposed
48
Repurposing Processes
Product Engineering
Manufacturing Engineering
Manufacturing
Synopsis
Writing
3-D
Previsualization
Previz.
Rendering
Dialogue
Writing
EditingQuality
Assurance
AcquisitionEDL
RenderingRepurposing
Repurposing
Scripting
49
Workflow (1): Synopsis
50
Workflow (2): Management
51
Workflow (3): 3-D Previz. – Scenario
52
Workflow (4): 3-D Previz. – Shooting Script
53
Workflow (5): 3-D Previz. Editing
54
Workflow (6): Acquisition
55
Workflow (7): Editing
56
Workflow (8): Quality Assurance & Analysis
Plane
57
Workflow (9): Repurposing
58
Workflow (10): Conform & Publish
59
Computer Aided Manufacturing: Workflow