Upload
others
View
16
Download
0
Embed Size (px)
Citation preview
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Compression for 3DTV Compression for 3DTV -- with special with special focus on MPEG standardsfocus on MPEG standards
Dr.Dr.--Ing. Aljoscha Ing. Aljoscha SmoliSmolićć
Fraunhofer HHIFraunhofer HHIImage Processing DepartmentImage Processing Department
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Overview
� Introduction, terms, market and standards
� 3D scene representation
� Conventional stereo video coding
� Video plus depth coding
� Multi-view video coding (MVC)
� Multi-view video plus depth (MVD) coding
� (3D mesh compression)
� Summary & Conclusions
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
3DV/FVV Processing Chain
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
3DV and FVV as Functionalities
� 3D Video (3DV, also known as stereo) and Free Viewpoint Video (FVV) describe functionalities
� Expand the user‘s sensation beyond what is offered by classical 2D video
� They do not exclude each other , can be combined in a single system
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Free Navigation
� Arbitrary selection of viewpoint and direction within real world audio-visuel scenes
� Well-known from computer graphics , virtual reality, computer games
� Synthetic scenes, objects or static views of real objects, 2D video
� Or restricted to predefined viewpoints (Formula 1 on Premiere)
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Free Navigation in Real Scenes
� New technology allows generation of intermediate views in between real existing camera view
� Image-Based rendering, 3D reconstruction
� Allows e.g. viewing sports or theatre from arbitraryviewpoints
� New technology for the whole chain:� Acquisition, signal processing, modelling, representation,
compression, transmission, rendering, display
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Free Viewpoint Video
� Same functionality as CG objects� free navigation, can be
viewed from any viewpoint/direction
� Integration into complete scenes (virtual/augmented/real)
� But: depict appearance, motion, deformation of real world objects
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Generation of a 3D depth impression from seperate viewsfor each eye
� Well-known e.g. from IMAX
3DV / Stereo Video
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Demo
3DV / Stereo Video
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Applications
� 3DV: Extension of 2D video to 3D with depth impress ion� DVD, broadcast, cinema, mobile, specific applications
(medicine, military, etc.)� Buy a DVD of a Hollywood movie in 3D
� FVV: Extension for interactive viewpoint selection� Production, special effects� DVD, broadcast, cinema, mobile, specific applications
(medicine, military, etc.)� Buy a DVD of a concert, opera, theatre, sports event in
FVV� E.g. final of FIFA worldcup
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Market Potential
� 3DV and FVV will create huge markets for equipment (production and consumer side) and content in the near future
� 3D displays� Set-top boxes, SW� Content� Equipment for acquisition� Equipment/SW for content creation� …
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Standards� Development of consumer markets (esp.) requires
interoperability of systems and equipment from different providers
� This means standard formats for content
� Consider impact of MPEG-2 video and mp3 on media related industry and society
� Note: these are “just” digital and compressed formats of their analog counterparts, basically nothing new
� FVV and 3DV expand the sensation
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MPEG/VCEG Standards for FVV and 3DV
� Standards are necessary to make 3DV and FVV happen
� MPEG investigated the needs for related standards in a sub-group called 3DAV
� Work continues in the JVT of MPEG and VCEG
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� A lot of things are already possible with MPEG-4, which is a rich multimedia framework
� Additions have been made to the computer graphics specifications (MPEG-4 AFX) to support specific FVV systems
� 2 new standards are finalized or being created� Auxiliary Video Data Representation� Multi-view Video Coding (MVC)
� More to come, e.g. MVD
MPEG/VCEG Standards for FVV and 3DV
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Overview
� Introduction, terms, market and standards
� 3D scene representation
� Conventional stereo video coding
� Video plus depth coding
� Multi-view video coding (MVC)
� Multi-view video plus depth (MVD) coding
� 3D mesh compression
� Summary & Conclusions
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Scene Representations
More geometry-based More image-based
StaticTexture,SingleGeometry
View-dep.Texture,SingleGeometry
StaticTexture,View-dep.Geometry
View-dep.Texture,View-dep.Geometry
Sprites
Layered-Depthimages
Lumigraph
Light-fieldrendering
Incarnations of 3D model
Conventional graphics pipeline Warping Interpolation
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Scene Representations
More geometry-based More image-based
StaticTexture,SingleGeometry
View-dep.Texture,SingleGeometry
StaticTexture,View-dep.Geometry
View-dep.Texture,View-dep.Geometry
Sprites
Layered-Depthimages
Lumigraph
Light-fieldrendering
Incarnations of 3D model
Conventional graphics pipeline Warping Interpolation
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Scene Representations
More geometry-based More image-based
StaticTexture,SingleGeometry
View-dep.Texture,SingleGeometry
StaticTexture,View-dep.Geometry
View-dep.Texture,View-dep.Geometry
Sprites
Layered-Depthimages
Lumigraph
Light-fieldrendering
Incarnations of 3D model
Conventional graphics pipeline Warping Interpolation
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Scene Representations
More geometry-based More image-based
StaticTexture,SingleGeometry
View-dep.Texture,SingleGeometry
StaticTexture,View-dep.Geometry
View-dep.Texture,View-dep.Geometry
Sprites
Layered-Depthimages
Lumigraph
Light-fieldrendering
Incarnations of 3D model
Conventional graphics pipeline Warping Interpolation
Video “Per-Pixel” Depth
255
0
Znear
Zfar
+
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Scene Representations
� Suitable for FVV and 3DV
� Pure FVV: rendering of one view from the representation
� 3DV: rendering of a stereo pair corresponding to eye positions
� A lot of very different data types are used for FVV and 3DV
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Overview
� Introduction, terms, market and standards
� 3D scene representation
� Conventional stereo video coding
� Video plus depth coding
� Multi-view video coding (MVC)
� Multi-view video plus depth (MVD) coding
� 3D mesh compression
� Summary & Conclusions
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Generation of a 3D depth impression from separate views for each eye
� Well-known e.g. from IMAX
� 2 video signals are used
Conventional Stereo Video
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Stereo images are very similar
� Images are related through scene geometry and camera properties
� Predict one view from the other
� Displacement of a pixel with respect to other image: disparity
Conventional Stereo Video
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Disparity
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Analogy: dense motion field� Use principles of motion estimation/compensation for
prediction
� Specific differences:� Different statistics of disparity and motion vectors� Disparities can be very large� Adjustments of entropy coding� Temporal adjacent images are more similar� Disocclusion more evident� Problems of white & color balancing� Scene lighting, surface reflections
Conventional Stereo Video
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Inter-view Prediction
� Algorithms well known, standards available (MPEG-2 MV Profile)
� Limited gain, temporal prediction works well
� Significant gain only for I frames
� No commercial relevance
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Downsampling
� Downsample one view
� Does not degrade 3D impression significantly
� Toggle between left/right
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Limitations of Conventional Stereo Video� Problem of conventional stereo video (2 fixed views):
� head motion parallax viewing is not supported
� Viewer expects occlusions/dis-occlusions when watching 3D scenery
� Unnatural impression when moving head (impression that whole scene moves)
� Can be overcome adding free viewpoint functionality :� View is adapted to user motion
�Tracking�Multi-view displays
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Overview
� Introduction, terms, market and standards
� 3D scene representation
� Conventional stereo video coding
� Video plus depth coding
� Multi-view video coding (MVC)
� Multi-view video plus depth (MVD) coding
� 3D mesh compression
� Summary & Conclusions
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Depth-Based Rendering
Image Depth
3D Warp
Virtual view
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MPEG-C Part 3
We would like to thank the Image Based Realities Gr oup of Microsoft Research for providing the Breakdancers and Ballroom data sets.
� Depth: monochromatic 8 bit video signal
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Generation of 2 views for each eye from one video and per pixel depth information
� Rendering of a stereo pair at the decoder� Virtual views can be generated within a certain operating
range => head motion parallax viewing� Requires tracking of user
Interactive Stereo Video
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Interactive Stereo Video
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� 3D impression without glasses� Automatic gaze tracking with built-in camera sensors� Automatic adjustment of 3D rendering due to user motion
FhG-HHI Autostereoscopic Display
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� 3D impression is individually adjustable due to rendering at decoder� Strength of depth impression� Position in relation to the screen (in front, behind,
distance)
� ATTEST:� Efficient compression of depth data is possible� Only 10% of related rate for video signal without
noticeable artifacts
Interactive Stereo Video
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Coding of „Video plus Depth“
Video
Depth
+
0 250 500 750 1000 1250 150032
34
36
38
40
42
44
46
48
50
52
20% of base layer bitrate
`Interview’
Bitrate [kbit/s]
H.264/AVC (v6.1a)MPEG-4 (R/D opt.)MPEG-4 (MS-Ref.)MPEG-2 (TM-5)
PS
NR
[dB
]
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
3D-TV System of the ATTEST Project
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Handling of Disocclusions
• Best approach: Pre-process (smooth) depth-images such thatno disocclusions appear !
Original Depth2D Gaussian Filter Smoothed Depth
* =
( ) ( ) ( )222 222
1 σ
πσ/, vuevuG +−=
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Handling of Disocclusions
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Limitations of Single Video plus Depth
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Disocclusion artifacts increase with distance of virtual view from available original view
Does not support wide range multi-view 3D displays
Very limited free viewpoint navigation
Problem of depth estimation
Limitations of Single Video plus Depth
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Video + depth as data representation for 3DTV
Enables 3DV with limited FVV
Very simple but efficient
Results from ATTEST: depth can be compressed very efficiently using video coders such as AVC (5-10% bitrate compared to color signal)
MPEG-C Part 3
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Initiative driven by Philips, FhG-HHI and other partners:
Define a simple container format that includes vide o + depth + auxiliary data (stereo config)
Fast solution: no long standardization process
MPEG-C Part 3
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Full specification has been proposed, evaluated and accepted by MPEG
January 2007 FDIS
ISO/IEC 23002-3 Representation of Auxiliary Video a nd Supplemental Information
ISO/IEC 13818-1:2003 Carriage of Auxiliary Data
MPEG-C Part 3
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Overview
� Introduction, terms, market and standards
� 3D scene representation
� Conventional stereo video coding
� Video plus depth coding
� Multi-view video coding (MVC)
� Multi-view video plus depth (MVD) coding
� 3D mesh compression
� Summary & Conclusions
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Multi-view Video Coding (MVC)
� FVV and 3DV representations require transmission of multiple synchronized video signals that show the same scenery from different viewpoints
� Huge amount of data that need to be compressed efficiently
� Contains a large amount of inter-view statistical dependencies
We would like to thank the Image Based Realities Gr oup of Microsoft Research for providing the Breakdancers and Ballroom data sets.
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC Applications
� Multi-view displays, e.g. MERL, Philips
� Autostereoscopic for multiple users
� Support head motion parallax viewing
� Integral imaging
� FVV based on ray-space, light-field
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Multi-view Video Coding (MVC)
� Evaluations have shown that specific MVC exploiting inter-view statistical dependencies provides significant coding gain
� Only for dense camera settings, not for dome-type arrangements
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Camera Settings
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Räumlich-zeitliche Prädiktion I
1st order neighbors
• T (temporal)
• S (inter-view)
• L/R (combined)
Multi-view Prediction
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
J JT better than S better than R/L
Influencing factorsFrame rate, camera distance, complexity of Content (motion, spatial detail)
Räumlich-zeitliche Prädiktion II
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC Standard
� MPEG decided to launch a new standard based on H.264/MPEG4-AVC
� MPEG-4 Part 10, Amd. 4
�Evaluation of Call for Proposals January 2006�WD July 2006�PDAM January 2007�FPDAM July 2007�FDIS January 2008
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Test Data & Conditions
� 8 MV data sets with 5-16 sequences each
� Different frame rates, image resolutions, camera distances, camera setups
� Representative test set for MV applications
� 3 bitrates, low, medium, high quality
� Over 200 sequences to be generated
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC Call for Proposals
� Current reference model is based on a proposal made by FhG-HHI
� Proved best performance out of 8 proposals in response to a CfP in formal subjective tests
HHI: Hierarchical B Pictures
12345678
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC Reference Model• Fully compatible to H.264/MPEG4-AVC• Reorganization of input images into a single stream prior to
encoding• Uses hierarchical B-pictures combined in temporal and inter-
view dimension
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11
1
2
3
4
5
6
7
T0
0
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11
1
2
3
4
5
6
7
8 9 10 11 12 13 14
T0
0 15
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11
1
2
3
4
5
6
7
16
8
17
9
18
10
19
11
20
12
21
13
22
14
23
T0
0 15
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11
1
2
3
4
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
T0
0 15
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11
1
2
3
4
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
72 73 74
T0
0 15
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11
1
2
3
4
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74
T0
0 15
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
1
2
3
4
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
1
3
4
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
3
4
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
B1
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
3
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
B1
P0
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
5
6
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
B1
P0
B1
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
5
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
B1
P0
B1
P0
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
7
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
B1
P0
B1
P0
B1
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
B1
P0
B1
P0
B1
P0
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
11
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
I0
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
9
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
13
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
I0
B1
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
8
17
25
33
41
49
57
65
18
26
34
42
50
58
66
10
19
27
35
43
51
59
67
20
28
36
44
52
60
68
12
21
29
37
45
53
61
69
22
30
38
46
54
62
70
14
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
7415I0
P0
B1
P0
B1
P0
B1
P0
I0
B1
B2
B2
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
17
25
33
41
49
57
65
18
26
34
42
50
58
66
19
27
35
43
51
59
67
20
28
36
44
52
60
68
21
29
37
45
53
61
69
22
30
38
46
54
62
70
23
31
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
17
25
33
41
49
57
65
18
26
34
42
50
58
66
19
27
35
43
51
59
67
20
28
36
44
52
60
68
21
29
37
45
53
61
69
22
30
38
46
54
62
70
23
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
P0
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
17
25
33
41
49
57
65
18
26
34
42
50
58
66
19
35
43
51
59
67
20
28
36
44
52
60
68
21
29
37
45
53
61
69
22
30
38
46
54
62
70
23
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
P0
B1
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
24
32
40
48
56
64
17
33
41
49
57
65
18
26
34
42
50
58
66
19
35
43
51
59
67
20
28
36
44
52
60
68
21
37
45
53
61
69
22
30
38
46
54
62
70
23
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
P0
B1
B2
B2
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
32
40
48
56
64
17
33
41
49
57
65
18
34
42
50
58
66
19
35
43
51
59
67
20
36
44
52
60
68
21
37
45
53
61
69
22
38
46
54
62
70
23
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
32
40
48
56
64
17
33
41
49
57
65
18
34
42
50
58
66
19
35
43
51
59
67
20
36
44
52
60
68
21
37
45
53
61
69
22
38
46
54
62
70
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
32
40
48
56
64
17
33
41
49
57
65
18
34
42
50
58
66
35
43
51
59
67
20
36
44
52
60
68
21
37
45
53
61
69
22
38
46
54
62
70
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
16
32
40
48
56
64
33
41
49
57
65
18
34
42
50
58
66
35
43
51
59
67
20
36
44
52
60
68
37
45
53
61
69
22
38
46
54
62
70
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
32
40
48
56
64
33
41
49
57
65
34
42
50
58
66
35
43
51
59
67
36
44
52
60
68
37
45
53
61
69
38
46
54
62
70
39
47
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
32
48
56
64
33
49
57
65
34
50
58
66
35
51
59
67
36
52
60
68
37
53
61
69
38
54
62
70
39
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
48
56
64
49
57
65
50
58
66
51
59
67
52
60
68
53
61
69
54
62
70
55
63
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
48
64
49
65
50
66
51
67
52
68
53
69
54
70
55
71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
64 65 66 67 68 69 70 71
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
64 65 66 67 68 69 70
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
P0
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
64 65 66 68 69 70
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
P0
B1
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
64 66 68 70
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
P0
B1
B2
B2
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
80
88
96
104
112
120
128
72
81
89
97
105
113
121
129
73
82
90
98
106
114
122
130
74I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
P0
B1
B2
B2
B3
B3
B3
B3
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
80
88
96
104
112
120
128
81
89
97
105
113
121
129
82
90
98
106
114
122
130
I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
P0
B1
B2
B2
B3
B3
B3
B3
B2
B3
B3
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
96
104
112
120
128
97
105
113
121
129
98
106
114
122
130
I0
P0
B1
P0
B1
P0
B1
P0
B1
B2
B3
B3
B2
B3
B3
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
b4
b4
b4
b4
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
B1
B2
B3
B3
B2
B3
B3
P0
B2
b4
b4
b4
b4
B1
B3
B3
P0
B1
B2
B2
B3
B3
B3
B3
B2
B3
B3
B2
B3
B3
B3
b4
b4
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
S0
S1
S2
S3
S4
S5
S6
S7
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11T0
I0
B1
B2
B3
B3
B2
B3
B3
P0
B1
B2
B3
B3
B2
B3
B3
B2
B3
b4
b4
b4
b4
P0
B1
B2
B3
B3
B2
B3
B3
B2
b4
b4
b4
b4
P0
B1
B2
B3
B3
B2
B3
B3
B2
b4
b4
b4
b4
P0
B2
B3
B3
B2
B3
B3
B3
b4
b4
B2
B3
B3
B3
b4
b4
B2
B3
B3
B3
b4
b4
I0
P0
P0
P0
P0
B1
B1
B1
B1
B1
B1
B3
B3
B3
B3
B3
B1
B2
B3
B3
B2
B3
B3
B2
B3
B3
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Objective Results Ballroom
30
31
32
33
34
35
36
37
192 256 320 384 448 512 576 640Avg. Rate [kbps]
Avg
. PS
NR
Y [d
B]
Inter-view PredicitionSimulcastAnchor
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
34.0
35.0
36.0
37.0
38.0
39.0
40.0
41.0
256 384 512 640 768 896
Avg. Rate [kbps]
Avg
. PS
NR
Y [d
B]
Inter-view PredictionSimulcastAnchor
Objective Results Race1
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Summary Objective Results
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Ballro
om Exit Uli
Race1
Flamen
co2
Break
danc
ers
Rena
Akko&
Kayo
Avg
. PS
NR
ga
in [d
B]
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Subjective Results
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Subjective Results
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Subjective Results
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Results
� Fully H.264/MPEG4-AVC compatible bitstream
� Decoder needs to de-interleave the decoded pictures into MV video streams
� Sophisticated temporal+inter-view prediction structures can be implemented based on hierarchical B-picture syntax
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Full MVC Structure
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Simulcast
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
KS_IPP
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
KS_PIP
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
KS_IBP
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Full MVC Structure
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Objective Results
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Objective Results
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC High-level Syntax
• Improved random access
• Low delay
• Memory optimization
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
View Interpolation Prediction
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Illumination Compensation
Adopted for JMVM, technology under consideration
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC Core ExperimentsSpecific tools under investigation in Core
Experiments:
• Disparity vector prediction
• Disparity estimation/compensation
• Inter-view direct mode
• Motion skip mode
• Deblocking filter
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVC StandardMPEG-4 Part 10, Amd. 4
• High-level syntax will be included, part of WD
• IC is part of JMVM, under consideration
• Specific tools under investigation in Core Experime nts
• Inclusion of signal processing tools requires major redesign of H.264/AVC implementations
• Will only be done if “sum of improvements” exceeds a reasonable threshold (e.g. 30% bitrate saving)
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Overview
� Introduction, terms, market and standards
� 3D scene representation
� Conventional stereo video coding
� Video plus depth coding
� Multi-view video coding (MVC)
� Multi-view video plus depth (MVD) coding
� 3D mesh compression
� Summary & Conclusions
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Up to 45 views are displayed at the same time� Others: Philips, MERL
Holografika Wide Range 3D Display
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
FVV Examples
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
FVV Examples
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Requirement
�These systems and applications require an input format that allows
� rendering a wide range (quasi) continuum of output views
�at the decoder
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� MVC� Usage of N views� No continuum� For large N very inefficient
� MPEG-C Part 3� Disocclusion artifacts increase with distance of virtual
view from available original view� Does not support wide range multi-view 3D displays� Very limited free viewpoint navigation
Extension to MVD
We would like to thank the Image Based Realities Gr oup of Microsoft Research for providing the Breakdancers and Ballroom data sets.
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Multi-view Video plus Depth (MVD)
•Usage of N videos with depth information•Interpolation of virtual intermediate views along all cameras (pair-wise switching)
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
< 180°
N originalcamera views
arbitraryview area
Multi-view Video plus Depth (MVD)
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
6 Original Views
Example – Synthesis of Motion Parallax
45 Virtual Intermediate Views
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Virtual View Interpolation
Camera 1 Camera 7
……………………………
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Virtual View Interpolation
……………………………
Camera 1 Camera 6
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVD System Overview
…
Transmission
CameraViews
3D Scene
DepthEstimation
3DRepresentation
3D Coding
Encoder
3D Decoding
Scene Rendering
Decoder
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVD System Overview
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MVD Signal Processing
N original images
Rectification
Disparity Matching
N-1 rectified image pairs
N-1 rectified pairs video plus disparity
Depth Map Creation
Analysis
De-Rectification
N video-plus-depth representations
• • • •
• •
• • • •
• •
• • N-1 rectified pairs video plus depth
Rectification
Parallax map Creation
M rectified video-plus-depth images
M rectified video- plus-parallax images
Virtual view rendering
M rectified display views
Rendering
De-Rectification
N video-plus-depth representations
M display views
• • • •
• •
• •
• •
• •
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Alternative: LDV
• Layered Depth Video (LDV)– One video– One associated depth map– One background video layer (those parts that are occluded by
original pixels)– One background depth layer
• May be more efficient (less data)
• May be more difficult to generate (occlusion layers)
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Initiative in JVT� There are advanced 3D video applications and systems, which are
not efficiently supported by standards
� These require a format that allows rendering a wide range (quasi) continuum of output views at the decoder
� MVD is a format that would support this, LDV as alternative
� Can be regarded as extension of MVC, MPEG-C Part 3
� Proposal: start a study to investigate how MVD can be efficiently supported by standards
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Work Plan
• Specification of requirements,
• Evaluation of available ISO and ITU standards rega rding efficient support of MVD,
• Collection and generation of MVD test data,
• Specification of test conditions,
• Coding experiments, including available standards and new algorithms,
• Subjective tests including 3D displays and free vi ewpoint functionality.
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Overview
� Introduction, terms, market and standards
� 3D scene representation
� Conventional stereo video coding
� Video plus depth coding
� Multi-view video coding (MVC)
� Multi-view video plus depth (MVD) coding
� 3D mesh compression
� Summary & Conclusions
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Capturing a real dynamic scene with N cameras� Transformation into a special data representation� Interpolation of arbitrary intermediate views� Allows free navigation within the scene (within practical
limits)
Free Viewpoint Video
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
� Courtesy of: Stephan Wuermlin et al., ETH Zuerich, Switzerland
FVV Acquisition
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
FVV Segmentation
� Courtesy of: Stephan Wuermlin et al., ETH Zuerich, Switzerland
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
3D Model Reconstruction and Rendering
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Object ReconstructionShape-from-Silhouette-Approacha) Hierarchical Voxel Modelingb) Marching-Cubes-Approachc) Surface Smoothingd) Mesh Reduction
T. Lewiner et al.: Efficient Implementation of Marching Cubes Cases with Topological Guarantee", Journal of Graphics Tools, Vol. 8, pp. 1-15, 2003.
G. Taubin: Curve and Surface Smoothing Without Shrinkage, Proc. IEEE International Conference on Computer Vision (ICCV 95), 1995.H. Hoppe, Efficient Implementation of Progressive Meshes, Computers & Graphics, Vol. 22, No. 2, pp. 27-36, 1998 / Tool in DirectX
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Free Viewpoint Video
� Same functionality as CG objects� free navigation, can be
viewed from any viewpoint/direction
� Integration into complete scenes (virtual/augmented/real)
� But: depict appearance, motion, deformation of real world objects
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Integrated Scene with FVV
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MPEG-4 Representation
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
MPEG-4 Representation
� Geometry: 3D mesh
� Textures (video streams): H.264/AVC
� Rendering: view-dependent texture mapping
� Supported by MPEG-4 due to recent extension of AFX developed by Fraunhofer HHI
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Coding Results for Textures
28
30
32
34
36
38
40
42
50 100 150 200 250 300
Bitrate [kbit/s]
PSNR [dB]
MPEG-4 (Part 2) Core Profile without shape
MPEG-4 (Part 2) Core Profile with object shape
H.264/AVC High Profile
H.264/AVC High Profile with pre-processing
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
D3DMC
Intra/InterSwitch
OctreeClustering
Scal./Quant.
m t( )+
+
MPEG-43DMC
Reconstr./Inv Scal.
OctreeReconstr.
)(ˆ to
o t( )d t( ) Arithmet.Coding
Memory
-y t( )
0)(ˆ td)(ˆ tm
)1(ˆ −tm
)1(ˆ −tm
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Differential 3D Mesh Coding (D3DMC)
• Tri-linear interpolation of eight estimated motion vectors
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Differential 3D Mesh Coding (D3DMC)
• Subdivision of space and resulting octree structure
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Arithmetic Coding
• Linear Quantization of the estimatedmotion vectors
• Entropy coding of the discreteestimated motion vectors. Usage of thewell-known Context-Based Adaptive Binary Arithmetic Coding (CABAC) framework of H.264/AVC
1
100
10000
-2000 -1500 -1000 -500 0 500 1000 1500 2000
X Y Z
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
RD-optimized D3DMC
• Block diagram of the encoder
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation
• Humanoid sequence provided by Vrije Universiteit Brussel (VUB) consisting of 117 keyframes in different resolutions
498 vertices 1940 vertices 7646 vertices
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation – Humanoid, 498 vertices
Original AFX-IC 22,7 kbit/s
D3DMC23,0 kBit/s
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation – Humanoid, 498 vertices
0,0001
0,001
0,01
0 5 10 15 20 25 30 35 40 45 50
Bitrate [kBits/s]
Dis
tort
ion
DA
Fixed D3DMC AFX IC RD-optimized D3DMC
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation – Humanoid, 1940 vertices
Original AFX-IC 60,1 kbit/s
D3DMC62,7 kBit/s
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation – Humanoid, 1940 vertices
0,0001
0,001
0,01
0 20 40 60 80 100 120 140 160 180
Bitrate [kBits/s]
Dis
tort
ion
DA
Fixed D3DMC AFX IC RD-optimized D3DMC
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation – Humanoid, 7646 vertices
Original AFX-IC 128,1 kbit/s
D3DMC86,4 kBit/s
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation – Humanoid, 7646 vertices
0,0001
0,001
0,01
0 100 200 300 400 500 600
Bitrate [kBits/s]
Dis
tort
ion
DA
Fixed D3DMC AFX IC RD-optimized D3DMC
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Evaluation – Chicken, 3030 vertices
AFX-IC at 109 kbit/s D3DMC at 108 kbit/s
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Improvements
• Skinning: INT, France
• Scalable: University of Hannover, Stefanoski, Ostermann
• New standard for dynamic mesh compression under development in MPEG
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Message• FVV and 3DV are new functionalities that expand the user’s
sensation beyond classical 2D video
• Both can be combined
• Technology is available and being optimized
• Standards are available and being developed
• There is the potential for huge new markets
• => THIS IS READY FOR TAKE OFF
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
What needs to be done?
• Optimization of algorithms and processes (esp. with regard to error prone vision tasks as segmentation, depth estimation, etc.)
• Finalization of standards and addition of missing components (e.g. efficient format for MVD, LDV)
• Development of convincing applications and systems
• MARKETING
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Credits
• Besides credits on individual slides this presentation is based on work in the Image Processing and Interactive Media Departments @ Fraunhofer HHI:
Nicole Atzpadin Eddy CookePeter Eisert Christoph FehnPeter Kauff Matthias KautznerPhilipp Merkle Karsten MüllerMarkus Müller Sabine LukaschikJürgen Rurainsky Ralf SchäferOliver Schreer Klaas SchüürRalf Tanger Thomas Wiegand
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Literature
• A. Smolic, P. Merkle, K. Muller, C. Fehn, P. Kauff, and T. Wiegand: Compression of multi-view video and associated data . In press in "Three-Dimensional Television: Capture, Transmission, and Display." Editors: Haldun M. Ozaktas and Levent Onural, Springer, Heidelberg, 2007.
• A. Smolic, R. Sondershaus, N. Stefanoski, L. Vasa, K. Mueller, J. Ostermann, and T. Wiegand: A survey on coding of static and dynamic 3D meshes, In press in "Three-Dimensional Television: Capture, Transmission, and Display." Editors: Haldun M. Ozaktas and Levent Onural, Springer, Heidelberg, 2007.
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
ThankThank youyou !!
[email protected]://ip.hhi.de/~smolic/
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
Aljoscha Smolic, 3DTV-CON ‘07 Compression for 3DTV - with special focus on MPEG standards
PDF Created with deskPDF PDF Writer - Trial :: http://www.docudesk.com