Introducing MPEG-5 Part 2 LCEVC - ITU · Introducing MPEG-5 Part 2 LCEVC A codec to improve other codecs Guido Meardi Geneva, ITU Workshop on the Future of Media ... The longer-term

Private and confidential

Introducing MPEG-5 Part 2 LCEVC

A codec to improve other codecs

Guido Meardi

Geneva, ITU Workshop on the Future of Media

8/10/2019

Copyright © 2019 V-Nova Ltd. All rights reserved 3Copyright © 2016 V-Nova Ltd. All rights reserved Copyright © 2016 V-Nova Ltd. All rights reserved

Why a Low Complexity Enhancement Video Coding Standard?

David Ronca

Director of Video Encoding Facebook(previously Netflix)

(…)

Encoder complexity is outpacing Moore’s law.

(…)

The longer-term answer to video encoding can not be to simply add more CPU capacity. This is an unsustainable model; both financially, and environmentally.

(…

Codec research must emphasize compression efficiency AND computational efficiency. Here's a thought exercise: If the next video encoder maintained the efficiency of VVC/AV1 but reduced computational complexity by 50%, would we consider that a successful new codec?

(from “Encoder complexity hits the wall”, 7/10/2019)

“

“

Copyright © 2019 V-Nova Ltd. All rights reserved 4

Codec Evolution – Better Compression, at a Cost

MPEG-2 AVC/H.264 HEVC

VP9

AV1

VVC

EVC

File size

(compression efficiency)

Time to encode

(computational complexity)

Legacy (2003), but still

large majority of video

Copyright © 2019 V-Nova Ltd. All rights reserved 5

File size

(compression efficiency)

Time to encode

(computational complexity)

MPEG-2 AVC/H.264 HEVC

VP9

VVC

AV1

EVC

A New Approach – Helpful whatever the choice


Low Complexity Enhancement Video Coding – What it is

Codec-agnostic codec of enhancement data, where enhancing n-th generation codec results in

Compression performance as close as possible to (n+1)-th generation codec

Encoding and decoding complexity in line with n-th generation codec

Demux

Standard CompliantStream

Standard Decoder*

Video stream

Ancillarydata stream

Enhanceddecoder

Output video(full resolution)

* Any standard format: AVC, HEVC, VVC, …

Capability Extension of Existing Video Codec(codec agnostic)


What is a sub-layer of enhancement residual data?

Sparse highly

detailed

information


Beware of super-resolution, fidelity requires the actual data

Sparse highly

detailed

information


How LCEVC works: Two Sub-layers of Residual Data

Decoded

Base Picture

(Layer 0)

Preliminary

Intermediate

Picture

First layer of

Residuals

(Enhancement

Sub-Layer 1)

Preliminary

Output Picture

Combined

Intermediate

Picture

Second layer

of Residuals

(Enhancement

Sub-Layer 2)

Combined

Output Picture

+ =

=+

+

Temporal

Buffer


Signal flow

base layer

sub-layer 0

sub-layer 1

enhancementlayer

½ Resolutionimage stream


A standard that it helpful for the old, the current, and the future Compression benefits

Improving the quality of AVC … … as well as HEVC … … as well as VVC

-43%

-45%

-45%

BD-rate-VMAF LCEVC AVC vs. AVC:

-44.96%

BD-rate-VMAF LCEVC HEVC vs. HEVC:

-34.20%

-32%

-37%

-38%

74.00

76.00

78.00

80.00

82.00

84.00

86.00

88.00

90.00

4000 5500 7000 8500 10000 11500 13000 14500

VM

AF

Bitrate (Kbps)

ParkRunning UHDp50 - LCEVC VTL vs. VTL - VMAF

LCEVC+VVC VVC

BD-rate-VMAF LCEVC VVC vs. VVC:

(coming soon)

-19%

-14%


What it means in practice – Example CampFire sequence

HEVC UHDp30 @ 8 Mbps LCEVC HEVC UHDp30 @ 8 Mbps


A standard that it helpful for the old, the current, and the future Encoding complexity benefits

Reducing the complexity of AVC … … as well as HEVC … … as well as VVC

100

37.6

LCEVC HEVC Encoding Time, as % of HEVC Encoding Time

100% = 736 sec

x265 LCEVC x265

100

41.3

LCEVC AVC Encoding Time, as % of AVC Encoding Time

100% = 74 sec

x264 LCEVC x264

100

21.3

LCEVC VVC Encoding Time, as % of VVC Encoding Time

100% = 1,412,882 sec

VTL LCEVC VTL

2.7x2.4x 4.8x


V-Nova CfP Submission 1 (P11 ) V-Nova CfP Submission 1 (P11 )

P11 JM vs. JM P11 JM vs. JM

BD-rate-PSNR BD-rate-MOS

ParkRunning3 -32.68% -51.70%

Campfire -18.32% -39.30%

Average Class A -25.50% -45.50%

LCEVC

LCEVC JM vs. JM

BD-rate-PSNR

ParkRunning3 -43.52%

Campfire -28.54%

Average Class A -36.03%

LCEVC

LCEVC JM vs.P11 JM

BD-rate-PSNR

ParkRunning3 -17.28%

Campfire -13.80%

Average Class A -15.54%

The collaborative work performed in the past months produced notable

improvements

• The good results of the CfPimproved materially!

• Specification, definitionsand overall clarity alsoimproved significantly

(MOS still to be done)

(MOS still to be done)

Thank You

[email protected]

Documents

Introducing MPEG-5 Part 2 LCEVC - ITU · Introducing MPEG-5 Part 2 LCEVC A codec to improve other codecs Guido Meardi Geneva, ITU Workshop on the Future of Media ... The longer-term