Digital Media Production

Gastcollege – Nieuwe Media en Maatschappij

Digital Media Production

© VRT-medialab: onderzoek en innovatie2

Quid VRT-Medialab?

• +/- 22 experts – research and development

• Member of international associations EBU (P/SCAIE, P/MAG, P/NP, P/HDTV,…) AMWA, SMPTE, IPTC, W3C

• Deliverables Expertise Proof of Concept Technology transfer


http://medialab.vrt.be


Table of Content

• Background – Digital Media Production Anno 2004 - The Digital Media Factory Reconciliation of requirements

• Creative staff Case 1 – Media Asset Management

• The consumer Case 2 – Non-linear Distribution

• The near future Case 3 – Computer Integrated Manufacturing


2004...


Background – Proof of Concept


• Objectives Tape-less production Re-purposing of material Integrated system for radio,TV and online

• Strategy - Generic if possible, specialised if required

Central storage services are the core of the architecture The MAM system implements the central warehouse processes State-of-the-art production systems are integrated with the MAM system Legacy archive system provides search and retrieval functionality

Objectives anno 2004Objectives anno 2004


Central storage

INGEST

MediaAsset Mgmt

IP network

CentralStorage

Draft Processing

Concurrentengineering

Browse Clients Processing duringingest/transfer

POSTProduction

Craft editors

Subtitling

Archiving

Analysis

Annotation

Follow-upcontinuity Editor-in-Chief

Post-sonorisation

PLAY-OUT

8 GB P2 card= 32 min videoDV25 (news)

Go Live - 17 June 2007


Lessons Learned

1. There is a mismatch between business requirements and state-of-the-art technology;

2. Media technology suppliers don’t acknowledge the limitations of IT technology;3. IT suppliers don’t sufficiently understand the requirements of (media)

production;4. Re-using applications in a context which they have not originally been designed

for is usually a sub-optimal solution;5. Proper reasearch and development are quintessential.


Online searching, re-viewing and cutting


One-click re-purposing of content


2009 - Reconciliation of Requirements

Computer Integrated Manufacturing CAD – Computer Assisted Design CAM – Computer Assisted Manufacturing

Media Asset Mgnt. - Effective re-use Harvesting and Annotation Search and Retrieval

State-of-the-Art Single system for radio, tv and online File-based production Re-use of content

Non-linear distribution on demand Personalisation

But – Needle in the haystack Unstructured information 10 times more items No native file-based tools available


Lessons Learned – Principles of System Integration

Everything should be made as simple as possible, but not one bit simpler.


Media Asset Management

Production PlatformProduction Platform


Infrastructure – Networks and Storage

Production and distribution


MAM system‘Wires’ - (Telex Services)(IPTC7901)

EBU popserver(NMS)

Web SitesArchives

Eurovision (Superpop – NMS)

DocumentationServices

Correspondents(email)

Reuters(NewsML1.1)

Lack of an effective standard Unstructured or semi-structured information

Redundant information – Overflow

Manual processingNewsroom Computer System

The problem (1) – Production Context


The problem (2) – video processing

Allegory of Fortune, Dosso Dossi (Courtesy of the J.Paul Getty museum)

« Prosperity in life is transitory and dependant on luck »

The semantic gap

-> interpretation is unique to audiovisual media

-> The source of valuable metadata is Product Engineering

-> Computer vision techniques can only recognise lower-level features

http://www.getty.edu/art/gettyguide/artObjectDetails?artobj=943






For example…


Master Data Management




The State of the Art

archiefnummer : ALG 20010813 1

fragmentnummer : 1

reeks : 1000 ZONNEN EN GARNALEN

bandnummer : E03024404

formaat : DBCM

fragmenttitel : 1000 ZONNEN & GARNALEN

beeld : KL/PALPLUS

fragmentduur : 18 20

tekst : 0'00" TOERISTISCH REPORTAGEMAGAZINE OVERZICHT

ONDERWERPEN GENERIEK TOERISTISCH REPORTAGEMAGAZINE,

OVERZICHT ONDERWERPEN

0'50" VANDAAG : KUNSTENAAR LUC HOFKENS ONTWIERP EEN OASE

OP ZIJN DAKTERRAS IN BORGERHOUT DIE DOET DENKEN AAN DE

GRAND CANYON INTERVIEW MET LUC EN ZIJN VROUW

MARILOU BUITENBEELD DAK MET OMGEVING BUITENKANT

ARBEIDERSWONING, PANO OVER ROTSWANDEN, KRATEN MET WATER,

BEPANTING, FOTOALBUM MET VERLOOP WERKEN

4'00" JUNIOR : KLAARTJE ALAERTS, 13 JAAR WIL ASTRONAUTEN

WORDEN ZE BEZOEKT HETEUROSPACE CENTER METRUIMTEVEREN,

RAKETTEN SIMULATIE IN RUIMTEVEER, INTERVIEW, HEEFT EEN

UFO GEZIEN MAAKT ZELF KLEIN RAKETJE, SCHIET HET AF

7'50" DE SCHEURKALENDER : ARCHIEF RECLAMEFILM IBM

INTERVIEW MAURICE DE WILDE, EERSTE PERSOONLIJKECOMPUTER

trefwoorden : BELGIE; BORGERHOUT; ARTIEST; OASE; KUNST; GRAND

CANYON (NATUURGEBIED); DAK; TERRAS; INTERVIEW; EURO

SPACE CENTER; RUIMTEVAART; PC; BOOTTOCHT; RIJKDOM;

PASSAGIER; GASTRONOMIE; RESTAURANT; PERSONEEL;

VAKANTIE; BINNENBEELD; SCHIP; BECKERS LEEN; VRT;

LOTTO; RADIOOMROEPSTER; KLANKSTUDIO; UITVINDING;

BARBECUE; BETONMOLEN; IBM; RECLAMESPOT

rechthebbende : VRT

Opzoekscherm FILM Set: 16 Aantal: 1

blz 1 van 3

trefwoorden: ibm and vrt

archiefnummer: -

uitzendjaar: maand: dag:

fragmentnummer: fragmentduur:

reeks:

formaat: bandnummer:

aflevering: afleveringsnummer:

programma: uitzenddatum:

fragmenttitel:

tekst:

kategorie:

opnamedatum: opnamenummer:

journalist: rechthebbende:

SETS

The strings required for the operation are not defined

F11 F12 F13 F14 F17 F18 F19 F20 Ent

Eindigen Sets Refset Toon Vorige Volg/Leeg Thesaurus Commando Opzoeken



Proof of Concept


Shot Segmentation and Scene Recognition


Face recognition and identification


Video copy detection

Identify dupplicates – Trace dependencies Linking stories over time Copy detection - Intellectual Property Protection







Media Asset Management Issues

• Production Environment• No upfront selection - information overload;

• Multiple sources of more or less structured metadata;

• Parallel sources cause dupplicates;

• International context - multilingual

• Professional Requirements• An item is relevant or it is not;

• Retrieve all relevant items;

• No false positives

• Video Processing• 80% of the context is in the speech

• Machines may recognise more objects

• Machine processing is more robust



Conclusions

A consistent application framework and proper user-interfaces are the key to effective re-use and

supply chain integration in general.

!

(≠ Digitizing analogue and disintegrated workflows)

• Scripts contain the essential master data – integration with creative processes is a must;

• In professional context, it is key to trace dependencies between objects. Hence, as opposed to any conventional approach, a search engine must integrate information rather than to index multiple sources;

• Computer Vision does not replace formal annotation, but substantially accelerates the process and increases the accuracy


Case 2 – Non-linear Distribution





D a t a G e n e r a l






MetaData

MetaData

Communication(Information)


Data

Digital Broadcasting via DVB, IP or other

Digital Broadcasting via DVB, IP or other

Video Audio





1914 – First radio broadcast


1937 – Die Volksempfaenger


1955 - Mobile radio...


Digital Broadcasting – Que?


2004 – Live Streaming


Porting the radioplayer...


Porting the radioplayer...


Stats Launch feb 2009 Currently over 60.000 installations 2500 users per day Average duration 30”


Business Case

IP Distributie (radio)

• +/- 5.000.000 luisteruren per dag

• 5.000.000 uur à 128 kbps, 250 TB

• 90 PB per jaar à 70.000€ per PB

• Totaal – 6.300.000€ per jaar

Vgl. deredactie.be

• Homepage 2MB

• 128 seconden high-def audio(!)


Broadcast...

A customer can have a car painted any colour that he wants so long as it is black.

(Henry Ford)


Watch anytime - BBC iPlayer


Good, better,...

Portable - Anywhere

• Radio-player application

• Quick win – “in series” live transcoding

Broadband distribution - Anytime

• “Podcast”, “iPlayer”

• On demand complements live offering

• Major impact – master data power-lifting

The Perfect Storm…

• Mobile – Portable and impulse-driven

• Internet - Seamless integration of live and on demand

• An excellent user interface…?


Why it might work this time...


Why it might work this time...

Radio and Telephone

• Radio and telecommunication are on the cold side

• Short attention span, high level of attention

• Impulse-driven

• Mobile was the killer app…


Proof of Concept - The Ultimate Radio Exp(eriment)

myRadio @time of writing

• Broadband connection enables non-linear distribution

• Adaptive streaming enables bandwidth-friendly distribution

• Excellent quality where possible, lower bit-rate if necessary

• Open distribution framework

http://myradio.vrt.com

http://myradio.vrt.com/


Proof of Concept (Ongoing)

Seamless integration of Live and On demand

• “Bookmark” by tagging

• Pause, Back and Fast Forward

Work in Process – Track each individual item

• Audio segmentation

• Speech recognition and indexing

• Metadata harvesting (!!!)

Personalisation

• Radio search engine

• Assemble myRadio


Personalisation by Interactivity

User profile

Personalisation

A customizable product and tracking the user profile based on data mining techniques will

allow us to create the illusion of personalisation


Trend - From « metadata » to CAD/CAM

?


Case 3 – Computer Integrated Manufacturing


State of the Art


Use Case – Drama Production

Ani


Proof of Concept - Scoop

http://www.youtube.com(search for “scoop”, “ibbt”, “CAD”)

http://www.youtube.com/

© VRT-medialab: onderzoek en innovatie54 54

http://medialab.vrt.be

http://myradio.vrt.be

[email protected]

http://medialab.vrt.be/

http://medialab.vrt.be/