1

Video Coding ConceptKai-Chao Yang

2

Video Sequence and Picture

Video sequence Large amount of temporal redundancy

Intra Picture/VOP/Slice (I-Picture) (p.295)

Encoded without referencing others All MBs are intra coded

Inter Picture/VOP/Slice (P-Picture, B-Picture) (p.295)

Encoded by referencing other pictures Some MBs are intra coded, and some are inter

coded

Intra 0 Inter 1 Inter 2 Inter 3 Inter 4 Inter 5

3

Group of Pictures

Group of Pictures (GOP) (p.295, p.455-457)

GOP in H.264/AVC Multiple reference frames B-pictures can be referenced Variable number of B-pictures

GOP in Scalable Video Coding (SVC) of H.264/AVC Hierarchical prediction structure

I B B P B B P … B B I B B P …

Encoding order:

Frame order: 0 1 2 3 4 5

0 2 3 1 5 6

6

4

Video stream

GOP GOP GOP…

Picture, Slice, MacroBlock, and Block

H.264/AVC Variable MB size 4x4 transform block

4

Picture

SliceMBblock

8

8

8

8

5

Flow Chart of Video Coding (p.293-295)

Transform Quantization

Motion Estimation

Entropy Coding

Quantization-1

Inverse Transform

Motion Compensation

Decoder

MB (spatial frequency) (lossy compression) (lossless compression)

(frame reconstruction)

(estimating similar blocks)

(removing temporal redundancy)

-

+

High frequency

Low frequency

Reference frameCurrent frame

6

Coding of I-Slice

Example

Original block Transformed block Quantization matrix

15 0 -2 -1 -1 -1 0 …Bit-stream

Zig-zag scanEntropy coding

DCT

7

Coding of P-Slice

Example

=

=

+Motion Vectors

Motion Estimation

Residual

Motion Compensation

Original current frame

Reconstructed reference frame

Frame buffer

-

8

Structure of H.264/AVC Encoder

Representation of video content

Formats of VCL representation

9

H.264/AVC Flow Chart

EntropyCoding

Scaling & Inv. Transform

Motion-Compensation

ControlData

Quant.Transf. coeffs

MotionData

Intra/Inter

CoderControl

Decoder

MotionEstimation

Transform/Scal./Quant.-

InputVideoSignal

Split intoMacroblocks16x16 pixels

Intra-frame Prediction

De-blockingFilter

OutputVideoSignal

10

New Features of H.264/AVC

Multiple reference frames Variable block size 1/4 and 1/8 motion vector precision Weighted prediction 4x4 transform block Intra prediction De-blocking filter CABAC & CAVLC Various block sizes and shapes

8x8

0

4x8

0 10 12 3

4x48x4

108x8

Types

0

16x16

0 1

8x16MB

Types

8x80 12 3

16x8

1

0

11

FMO and ASO

Examples of Flexible MB Order (FMO)

Arbitrary Slice Order (ASO) sending and receiving the slices of the picture in

any order relative to each other

Slice Group #2

Slice Group #1

Slice Group #0Slice Group #0

Slice Group #1

Slice #0

Slice #1

Slice #2

12

H.264/AVC Profiles

13

H.264/AVC FRExt Amendment

Completed in July 2004 For coding of high-fidelity

video material Professional film production,

video post production, or high-definition TV/DVD

Higher quality, higher rates Professional extensions

Fidelity Range extensions (FRExt)

http://plusd.itmedia.co.jp/lifestyle/articles/0407/09/news074.html

14

Profiles of H.264/AVC FRExt

High profiles High profile (HP)

Supporting 8-bit with 4:2:0 sampling High 10 profile (Hi10P)

Supporting 10-bit with 4:2:0 sampling High 4:2:2 profile (H422P)

Supporting 10-bit with 4:2:2 sampling High 4:4:4 profile (H444P)

Supporting 12-bit with 4:4:4 sampling, and efficient lossless coding and an integer residual color transform for coding RGB video

References

T. Wiegand, G.J. Sullivan, G. Bjntegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard,” in IEEE Transaction on Circuits and Systems for Video Technology, 2003.

Iain E. G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia, Wiley, 2003.

 G. J. Sullivan, P. Topiwala, and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions,” in SPIE Conference on Applications of Digital Image Processing, 2004.

15