1
Traitement, Modélisation d ’Images et Communications
(Processing, Modelling and Communication of images)
TEMICS
Evaluation 2001-2005
The Team and its Evolution
Nov. 2005
Project Leader C. Guillemot (INRIA)
Team Assistant H. Béchu (INRIA)
Researchers/Teachers T. Furon (INRIA) C. Fontaine (CNRS) L. Morin (Univ. Rennes-1) G. Rath (INRIA) + S. Marinkovic (INRIA), on leave, Sept.2004
Research Engineer L. Guillo (CNRS, 40%)
Temporary engineers: 3 Postdocs: 3 Ph.D Students: 9
Departures L. Amsaleg (creation of TexMex, 02) S. Pateux (secondment from FT, 03) P. Siohan (secondment from FT, 03) H. Nicolas (Univ Bordeaux 05)
Arrivals T. Furon (CR-INRIA, Oct. 02) L. Guillo (IG CNRS 40%, Oct. 02) S. Marinkovic (CR-INRIA, Oct. 02) G. Rath (CR-INRIA, Oct. 03) C. Fontaine (CR-CNRS, Oct. 05)
A. Roumy (CR-INRIA, Jan. 06)
Temporary engineers (2001-2005): 12 Postdocs (2001-2005): 11 Ph.D (2001-2005): 14
Changes
General Objectives
Design and development of theoretical frameworks, methods and algorithms in the areas of analysis, modelling, coding, communication and watermarking of images and video signals
1. Representation and compression with functionalities (navigation, manipulation, scalability, enabling feature extraction / indexing, …)
2. Communication over heterogeneous, wired & wireless networks
3. Information hiding under constraints of invisibility, capacity, robustness and security
1. Augmented reality, interaction / navigation with/in content
2. Networked (wired/wireless IP) multimedia
3. Copyright, content enrichment, copy protection, authentication
Application DomainsProblems
Research Axes and Methodology
Area 1: Video analysis and 3D modelling Computer vision Analysis-Synthesis based on meshes
Area 2: Scalable video representation Texture/geometry sparse representation Complete/over-complete transforms
Area 3: Joint source-channel coding Statistical and Turbo Estimation Joint Source-Channel Statistical Codes Frame Theory
Area 4: Watermarking and security Communication with side information Game theory Shannon cryptanalysis
Highlights of a few contributions
Challenges: Modeling of large scenes via a sequence of 3D models Automatic 3D modeling from unconstrained video Temporal consistency Multi-resolution low rate representation
Synthesis:• Temporal continuity of connectivity by construction,
• Temporal continuity of geometry via morphing
• Software transfer to France Telecom, contribution to MPEG/3D AV
Contribution: Analysis:
• Rate-distortion constrained construction of dense motion field + camera path depth map
• Consistent connectivity dissociated from geometry
• Synthesis wavelets on the low-res. mesh
Area 1 – Main Achievements: 3D Model-based scalable video coding
Contribution:
Local adaptation of orientations of 1D lifting steps on multi-resolution Quincunx grids Capture anisotropic geometrical structures
• Better energy compaction in low-frequencies. Outperforms in compression and denoising the other « *lets » (contourlets, curvelets, bandelets, …). Orientation optimization based on Markov fields in de-noising
Challenges: Compact (sparse) signal representation space Sparsity of the representation provided by WT
limited by the isotropy of the basis functions High energy coefficients cluster around edges.
Oriented Wavelet Transform - Anisotropic StructuresArea 2 – Main Achievements:
Challenges:
Sparse signal representation spaces Amenability to other processing tasks (motion analysis, feature extraction, local description)
Contributions:
Methods to tune the redundancy in Laplacian
Pyramid (LP) representations LP is a special type of frames
Reconstruction algorithms Foreseen Impact in the SVC context Patent filed, transfer under discussion.
Method-1
Method-2
Filters need not be bi-orthogonal or orthogonal
Area 2 – Main AchievementsBridging the gap between Wavelets and Laplacian Pyramids
Challenges: • High sensitivity to noise of VLCs • Optimal decoding in presence of noise • Quadratic complexity of optimal decoding solutions
Without errorsPSNR = 33.9 dB0.5 bpp
Hard dec. PSNR = 16.4 dB
Eb/N0=5 dB.
Soft dec PSNR = 25.1 dB
Contributions: • Estimation/soft synchro of VLC-coded sources
• Adopted in JPEG2K part 11
• Patent transferred/licensed to Thomson
• New State models based on state aggregation
• State model = (Nk,Tk mod T)• Complexity O(T.L) instead of O(L**2)• best paper award
Area 3 – Main Achievements: Statistical Estimation of VLC-encoded sources
Challenges: • Sub-optimality of classical VLCs in presence of noise
• Need for unequal error protection
• Sub-optimality when followed by a channel code
Contributions: • New Families of error-resilient source codes
• Multiplexed (stationary and 1st-order) codes: joint statistical coding of 2 sources• Reaching the source entropy• Synchronous decoding of high priority information• Patent transferred/licensed to Thomson
Area 3 – Main Achievements: New statistical codes for memoryless and memory sources
• Formalism based on re-writing rules extending the class of VLCs
leading to codes exhibiting a set of properties (compression, uniform marginal bit probability, reduced state space, …)
S l b
FLC
6.19 bpp
16.79 db
0.005 BER
Huf1.71 bpp15.59 db
0.005 BER
Mul
1.71 bpp
24.82 db
0.005 BER
Challenges: • Signal representation resilient to noise • Diversity in multi-channel transmission• Multiple description codes
Contributions: Signal expansion on redundant bases of functions (frames) A complete framework with a set of erasure recovery and error localization/recovery algos
• Frame theoretic (using the dual frame)
• Coding theoretic (syndrome) approaches
• Subspace-based approaches
• Hypothesis testing approaches
A large class of Frames (based on block codes, on filter banks, tree-structured wavelets)
IntroducingStructure in
signal
Sourcecoder channel
Sourcedecoder
ReconstructAlgorith
Frame = a bigger basis (overcomplete)
Redundancy structured by proper design
Syndromes can be expressed as a sum of complex sinusoids in noise.
9 Journal papers
Area 3 – Main Achievements: Over-Complete Frame Expansions and Decoding Methods
Contributions: Communication under constraints
Area 4 – Main Achievements: Robustness and Security of Watermarking
Challenges: Improving trade-off invisibility / capacity / robustness Assessing/improving watermarking security
8 bits hidden, 38 dB Scaling + JPEG (22dB)
Capacity: Channel coding with side information Robustness: Game theory (min-max game):
Security: Adaptation of Shannon cryptanalysis to watermarking• Measure information about secret key leaking from WM content• Theoretical analysis of security of main techniques (Fisher Information) (Number of observations / complexity needed) • Tools implementing the attack based on blind source separation
« Security » attack: 36 dB
xyxyb DD
N
E
iiiW
'0,
minmax
Best paper award
Patent
Visibility and Impact
14 Ph.D Theses 44 accepted journal publications + 6 submitted 7 book chapters 132 conference publications Thesis (2) and papers (2) awards
Organization of PCS 2003 Co-chairing CORESA 2005 Editorial board of IEEE Trans. on IP (00-04) Editorial board of IEEE Trans. on CSVT (04-06) TEMICS is present in International scientific
committees (IEEE-IMDSP, IEEE-MMSP, …) TEMICS is present in programme committees of a
large number of conferences (see synthesis doc.) Numerous expertises (see synthesis doc.)
5 INRIA patents
• 3 patents transferred/licensed to Thomson• Transfer of the 2 others under discussion
3 patents filed by industrial partners with
TEMICS members as co-authors
Software transfer
• 3D codec + 2D mesh-based codec transferred
to France Télécom (FT)
• WAVIX transferred to Thomson: Starting
point for joint reply to MPEG/SVC cfp.
• WULL provided to several companies
• MuxCode library transferred to Thomson
• AC estimation integrated in OpenJPEG and
transferred to Thomson
10 contributions to standardization
Scientific In the Application Domains2001-2005
Project Positioning within INRIA
Specificities w.r.t. teams in the area of 3D modelling (e.g., MOVI): unconstrained monocular sequences, constraints of description costs & scalability
Complementarity/collaboration with PLANETE on problems of cross-layer design: PLANETE focuses on MAC and network issues whereas TEMICS focuses on signal processing issues.
Complementarity/collaboration with TEXMEX on issues as tracing of copyrighted content: TEXMEX uses indexing techniques whereas TEMICS uses watermarking techniques; Study of transforms for joint compression - feature point / descriptor extraction
Strategic Objective:
« Develop multimedia data and multimedia information processing … To process multimedia data and transmitting it over heterogeneous networksThe objective of this research concerns modeling, processing and indexing large scale, changing data masses-signals, sounds, images, etc.-often associating several source modalities and several communication channel and data protection models. »
International / National Positioning
USA + Canada + Japan: • Berkeley (K. Ramchandran, A. Zhakor), Boston (J. Konrad), Rochester (M. Tekalp), Stanford (B. Girod), UIUC (P. Moulin, T. Huang), Tohoku Univ (K. Deguchi)
Europe: • Collaborations with UCL (B. Macq, L. Vandendorpe), EPFL (M. Kunt), Vigo (F. Perez- Gonzales), Sienna (M. Barni), Geneva (T. Pun), … + many others via 6 EU projects.
France: • Collaborations with CNRS-L2S, I3S, LIS-Grenoble, ENST-Paris
TEMICS strengths:• Problematic at the crossroad of information theory, signal processing, and vision
• Research spectrum capitalizing on dualities between a range of problems
• Scientific (publications) and applicative (patents,standardization) achievements
Peer / Competing Groups
Summary
Recommendations of last evaluation• Focus on telecommunication problems (« on the hottest research areas of visual communication»)
• Contribute to standardization
• Patents and software transfer
• Re-enforce the team on the main focus of the project
High visibility and high impact on scientific and applicative aspects
The team forms a very good mix of signal processing, information theory and vision skills needed for the problems addressed at the crossroad of the three domains.
2001 - 2005
Smooth continuation and evolution of the most promising research directions as well as new challenging research directions to address the three following problems:
1. Representation and compression with functionalities (navigation, manipulation, scalability, enabling feature extraction / indexing, …)
2. Communication over heterogeneous, wired & wireless networks
3. Information hiding under constraints of invisibility, capacity, robustness, security
Objectives for next period
5 Research Axes
Area 1: Video analysis and 3D modelling
Area 2: Scalable video representation
Area 3: Joint source-channel coding
Area 4: Watermarking and security
Area 5: Distributed Source Coding
New Challenges (1/2)
Area 1: Video Analysis and 3D Modelling Vision for compression and communication Higher level/multimodal information (e.g., GIS data) with a range of fundamental and methodological issues (registration, extraction of correlated features, joint analysis and representation)
Area 2: Scalable Representation/Coding Multi-resolution and sparse representations: trade-offs sparsity, decay order, invariance, coherence across scales, space and directions, frequency selectivity, …. Multi-resolution spatio-temporal texture analysis/synthesis
Area 3: Joint Source-Channel Coding Signal processing (e.g., frames, framelets) and information theoretic (statistical codes, JSC codes, iterative/turbo decoding) approaches for video communication (including multiple channels)
Area 4: Watermarking and Security Extend our work on watermarking security assessment considering other hiding primitives (authentication, fingerprinting, steganography) Address the poor trade-off robustness/security of watermarking techniques: Is there a fundamental reason (theoretical bounds) for the poor trade-off robustness/security ?
New Challenges (2/2)
Area 5: Distributed Source Coding (New, started in 2004) A large range of open fundamental and applied problems
• Side information modelling (3D for exploiting scene geometrical constraints)
• Code design accounting for source memory (Frame theory, FSM) and channels
Thanks for listening!
Questions ?