77
High Dynamic Range Master Class Matthew Goldman Senior Vice President Technology, TV & Media Ericsson & Executive Vice President, Society of Motion Picture & Television Engineers

High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

Embed Size (px)

Citation preview

Page 1: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

High Dynamic RangeMaster Class

Matthew GoldmanSenior Vice President Technology, TV & Media

Ericsson&

Executive Vice President, Society of Motion Picture & Television Engineers

Page 2: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 2

Do we see or do we “make”?

© Ericsson 2016 | 2016-08-31 | Page 2

Page 3: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 3

Are these cubes

A) All straight

B) Some curved or

C) All curved?

Let’s find out….

try this …

Page 4: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 4

What color are the tiles?

Page 5: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 5

Surprised?We “make” color we don’t “see” color

Page 6: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 6

› UHDTV discussions often don’t consider the

human visual system (HVS)

› The eye is relatively low resolution, so the

HVS builds detail over time via saccadic

motion and eye tracking and using contrast

and color, combining input data with internal

reference models based on memory

› This is not at all how cameras work

wE “make” imageswe do not “see” them like a camera

Page 7: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 7

Recap: 5 Ultra-HD Immersive Viewing Image Technologies

High Dynamic Range

Wide Color

Gamut

10-bit Sampling

8b = Visible Banding

High Frame Rate

Image Resolution

8K UHD 7680x4320

4K UHD 3840x2160

HD 1920x1080

SD

Page 8: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 8

Visual perception - Resolution

1 arc minute*

*limit of Fovea Centralis 0.5 arc minute

Page 9: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 9

Proper Viewing Distance to “See” Spatial Resolution

HDTV field-of-view ~30°

4K UHDTV field-of-view ~60°

H

H

Proper Viewing Distance (D)HD (1080p) ~= 3H

4K UHD (2160p) ~= 1.5H

x°x°

WD

D = (W/2)/tan(x)

Screen size = √(H2+W2)

1 arc minute

Page 10: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 10

Screen Size vs. Viewing Distance

Source:

http://www.rtings.com/info/4k-ultra-hd-uhd-vs-1080p-full-hd-tvs

(2160p)

Page 11: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 11

High Dynamic Range (HDR)› HDR immersion not limited to strict viewing distance

– Benefits large screens (including HD) and tablets and phones

› From transmit side, HDR is potentially more economically viable to deploy than

4K UHDTV

› Once you have seen HDR, you realize how much better than current TV it is

› Cameras can capture HDR now, but we can’t see it at home

Pictures are richer, more lifelike and sharper with HDR. Seeing is believing.

Page 12: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 12

HDR and perceived resolution

Low contrast image looks ‘softer’ as

some detail is harder to see

More dynamic range can reveal more

detail – especially edges – and looks

sharper (although the pixel resolution

is the same)

Which image has higher resolution?

Page 13: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 13

HDR and perceived resolution

Low contrast image looks ‘softer’ as

some detail is harder to see

More dynamic range can reveal more

detail – especially edges – and looks

sharper (although the pixel resolution

is the same)

Page 14: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 14

Contrast effects on resolution

Pelli-Robson chart:

Impact of contrast on resolution

Snellen chart:

Impact of size/distance

on resolution

Page 15: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 15

This is (simplistically) how it works

SD/HD/4K TV Today:

Low dynamic range means subtle

contrast differences in the original

content (and which many cameras

can capture) are not maintained –

detail is missing.

HDR TV:

High dynamic range means

subtle contrast differences in

the original content can be

captured and transmitted to

the consumers, revealing

previously hidden detail.

Page 16: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 16

Dynamic range and the HVS

Light

Grey

Dark

Grey

Page 17: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 17

Dynamic range and the HVS

Light

Grey?

Dark

Grey?

Page 18: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 18

Is ‘A’ or ‘B’ lighter?

Another example

50//60, 100/120 fps

10/12 bit sample depth

30 fps

8 bit sample depth

Page 19: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 19

They are the same

Another example

50//60, 100/120 fps

10/12 bit sample depth

30 fps

8 bit sample depth

Page 20: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 20

“HDR+” … for any image resolution

High Dynamic Range

Wide Color Gamut

10-bit

Sampling

8b = Visible Banding

The combination of HDR, WCG and higher sample precision technologies – acts as a single feature!

3840 x 2160p 1920 x 1080p

or

Whether

Page 21: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 21

Candela per square meter

(cd/m2) or “nit”

Cinema today: 48 cd/m2

– In dark viewing environment

Reference white for TV

production: 100 cd/m2

– Rec. ITU-R BT.1886

– Based on 1930s CRT!

Typical LCD TV today

(standard dynamic range, SDR): 300-400 cd/m2

HDR TVs, now to future: 1,000 to 4,000 cd/m2

Luminous Intensity

Page 22: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 22

Comparing SDR to HDR

Standard Dynamic Range,

Lowlight Exposure

High Dynamic Range,

(simulated by tone

mapping)

Images source: K. McCoy. Licensed under CC BY-SA 3.0 via Wikimedia Commons

Standard Dynamic Range,

Highlight Exposure

Page 23: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 23

› SDR: Video generally ≤ 1.25x; Cinema generally ≤ 2.7x

› HDR: May be up to 100x

HDR is not about brighter display!

Source: Report ITU-R BT.2390

Page 24: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 24

HDR: Specular light ImpactImages courtesy of Dolby Laboratories

© Ericsson 2016 | 2016-08-31 | Page 24

Page 25: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 25

Human Visual System Capability

10-610-8 10-4 10-2 100 102 104 106 108

cd/m2 (nit)

TV today

Simultaneous dynamic range

Digital Cinema

Light Level

Cinema: 48 cd/m2 peak white

in dark viewing environment

Television: 100 cd/m2 peak white

(Rec. ITU-R BT.1886)

Based on 1930s CRT

Page 26: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 26

Human Visual System Capability

10-610-8 10-4 10-2 100 102 104 106 108

cd/m2 (nit)

TV today

Simultaneous dynamic range

Digital Cinema

Light Level

Cinema: 48 cd/m2 peak white

in dark viewing environment

Television: 100 cd/m2 peak white

(Rec. ITU-R BT.1886)

Based on 1930s CRT

HDR TV4Q15: 1000-1200 cd/m2

Future: Likely much higher

Page 27: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 27

Wide Color Gamut (WCG)Capture more of reality – richer colors

Outer triangle: UHDTV primaries

Rec. ITU-R BT.2020

Inner triangle: HDTV primaries

Rec. ITU-R BT.709

Source: Report ITU-R BT.2246

Page 28: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 28

WCG & HDR are closely linked

X

Y

Y X

Z

Outer triangle: UHDTV primaries

Rec. ITU-R BT.2020

Inner triangle: HDTV primaries

Rec. ITU-R BT.709

BT.709

+ Z

BT.2020 + Z

Page 29: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 29

› Today, all direct-to-consumer digital TV uses 8-bit sampling

› Banding (posterization) with 8b, especially in plain areas

– Sky, backgrounds, graphics, logo

– Very noticeable with slow changes, such as fades

› Significantly improved PQ with 10-bit sample bit depth

– No bandwidth cost in the compressed domain

– HEVC Main-10 Profile allows 8-bit or 10-bit operation

› HDR and WCG exacerbates issues with 8-bit sampling

Visual Quality: sample Bit depth

Visible banding

8-bit 10-bit

Page 30: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 30

› To see what’s happening from a far distance, the scene needs to be captured, transmitted to a remote location, then reconstituted

› Cameras convert scene light to an electrical signal, suitable of being transmitted over long distances, using an opto-electronic transfer function (OETF)

› Display convert an electrical signal back to scene light using an electro-optical transfer function (EOTF)

› For over 60 years, the cathode ray tube (CRT) was the universal display technology used

› The response of a CRT to an input signal is not linear and its EOTF is commonly known as gamma

Television = tele + visionFrom camera to display

Transmission MediumScene

Capture

Scene

Display

OETF EOTF

L = Vγ

where γ = 2.35

for HDTV

(ITU-R BT.709)

Page 31: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 31

› Step size / Luminance (dL/L) is the measure of visibility

› Levels below the Barten’s contrast sensitivity function (dashed curve) are masked from the HVS

› Mapping signal levels to display luminance (EOTF) is known as the gamma curve (a straight line in log space)

› 8-bit gamma-coded has large, visible steps across the range

› 10-bit gamma-coded reduces this dramatically

Sample Bit Depth

Page 32: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 32

› ST 2084:2014 – High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays

– Defines “display referred” curve with absolute luminance values based on human visual model– Called Perceptual Quantizer (PQ)

› ST 2086:2014 – Mastering Display Color Volume Metadata Supporting High Luminance and Wide Color Gamut Images

– Specifies mastering display primaries, white point, and min/max luminance

› Draft ST 2094-x – Content-Dependent Metadata for Color Volume

Transformation of High Luminance and Wide Color Gamut Images– Specifies dynamic metadata used in the color volume transformation of source content mastered with

HDR and/or WCG imagery, when such content is rendered for presentation on a display having a smaller color volume

› New project: HDR & WCG Signaling on Streaming Interfaces– To define a signaling representation & carriage mechanism for real-time interfaces to identify HDR and

WCG content so that it is properly processed in a production facility as well as correctly displayed in professional reference displays using SMPTE interface standards

Recent / on-going StandardsSMPTE

Page 33: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 33

› Rec. ITU-R BT.2100 – Parameter values for high dynamic range television systems for production & international programme exchange

› Report ITU-R BT.2390 – High dynamic range television for production and international programme exchange (companion report to BT.2100)

› Interim Report from RG-24 HDR-TV – HDR signalling requirements for programme production and international exchange arising from Recommendation ITU-R BT.2100

– Possibly modify Rec. ITU-R BT.1120-8 Digital interfaces for HDTV studio signals, and Rec. ITU-R BT.2077-1 Real-time serial digital interfaces for UHDTV signals

– Transfer function signaling could enable auto-selection of appropriate EOTF by a display:› BT.1886 EOTF if the SDR-TV (i.e., BT.709 HD or BT.2020 UHD) flag is received› BT.2100 PQ EOTF if the PQ flag is received› BT.2100 HLG EOTF if the HLG flag is received

Recent / on-going Standards ITU-R WP6C

Page 34: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 34

› Production reference for CRT peak white level is 100 nits

– Now referred to as standard dynamic range (SDR)

– SDR camera OETF comes from a desire to simplify analog TV electronics› Inverse of CRT gamma (EOTF)

› Rec. ITU-R BT.2100 defines 2 HDRtransfer functions:

– SMPTE ST 2084 Perceptual Quantization (PQ) EOTF

– Hybrid Log Gamma (HLG) OETF› (originally from BBC/NHK)

– Pros & Cons for each› Consequences for the production chain

HDR Transfer functions

10-bit levels over wider range

Page 35: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 35

Where:

E´ denotes a nonlinear color value {R’, G’, B’} or { L’, M’, S’} in PQ space

[0,1]

FD is the luminance of a displayed linear component {RD, GD, BD} or YD or ID,

in cd/m2

Y denotes the normalized linear color value, in the range [0:1]

So that when R’=G’=B’, the displayed pixel is achromaticm1 = 261016384 =0.1593017578125

m2 = 25234096×128=78.84375

c1 = 34244096=0.8359375=c3-c2+1

c2 = 24134096×32=18.8515625

c3 = 23924096×32=18.6875

Perceptual Quantization (PQ) EOTF (SMPTE ST 2084)

1

2

21

1

32

1

10,max

10000EOTF

m

m

m

D

Ecc

cEY

YEF

Page 36: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 36

Hybrid Log Gamma (HLG) OETF

Source: Report ITU-R BT.2390

EcbEa

EEEE

1ln

102OETF

Where:

E is the signal for each colour component {Rs, Gs, Bs} proportional to scene linear light

and scaled by camera exposure, normalized to the range [0:12]

E´ is the resulting non-linear signal {R’, G’, B’} in the range [0:1]

a = 0.17883277, b = 0.28466892, c = 0.55991073

Page 37: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 37

Both End-to-End HDR Systems

Source: BBC Research & Development

HLG OETF

SMPTE ST 2084

PQ EOTF

End-to-end HDR systems, both identical in mastering environment

Rec. ITU-R BT.2100 – Parameter values for high dynamic range television systems for production & international programme exchange

• Common reference opto-optical transfer function (OOTF); compensates for non-linearity between displayed light and the light captured by the camera

Page 38: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 38

EOTF

ITU-R BT.1886

(peak white 100 nits)

OETF

ITU-R BT.709

Live Broadcast Chain – SDR

AcquisitionCompression & Transmission

DisplayScene

Page 39: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 39

Live Broadcast Chain – HDR+

High Dynamic Range Display

? ?

High Dynamic Range Content

>16 f-stops

?

AcquisitionCompression & Transmission

DisplayScene

Page 40: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 40

HDR+: the wider ecosystem

Camera

sensor

OETF &

Map to

Color

Space

Graphics

&

Production

Compress

(DTH)

Full Size

&

Down-

converted

Compress

content

exchange)

Profess-

ional

Decode

Ad Insert Consumer

DecodeEOTF,

Remap

&

Display

STB

Graphics

Overlay

Page 41: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 41

SDR Production Today

Racking

Racking

Video

VideoVideo

Router

Vision

Mixer

(Switcher)

Graphics

Other

sources

CCU ctrl

CCU ctrl Video To Studio

Page 42: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 42

› The reference monitor dilemma

› HDR Reference Monitors today do not have a standard

› So we can either:

– Define the standard (as with SDR TV today), or

– Signal what was used

› The choice affects interoperability of content

HDR Production

Page 43: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 43

Joint HDR/SDR Production

HDR to SDR

ConversionHDR SDR

These all need to have matched levels ..... or you can never create the SDR version

Page 44: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 44

Backward compatibility checklist1. What exactly is meant by backward compatibility?

2. Which technologies are we trying to address?1. Dynamic range

2. Color gamut / Color space

3. Sample bit depth

4. Spatial resolution

5. Temporal resolution

6. Video coding standard for delivery-to-consumer (DTH)

3. What is “good enough” picture quality?1. For legacy, conventional HD service

2. For new Ultra HD service (1080p or 2160p HDR+)

4. What new costs will backward compatibility create?1. In production

2. In post-production

3. In distribution

4. In delivery to consumer

Before we all sign up to a

backward compatibility

scheme, have we really

understood what will happen

at each stage of the content

chain?

Page 45: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 45

HDR+ “backward compatibility”

› Q1: Are we trying to make an HDR signal that also can be

viewed on any conventional SDR TV/display?

(stream backward compatibility)

Or

› Q2: Are we trying to make a signal that can be converted by

an intermediate step to be shown on any conventional SDR

TV/display?

(display backward compatibility)

Or

› Q3: Are we planning to simulcast, as we do today with

HD/SD, and are planning to do with UHD/HD?

Historical note: Prior to HD being launched, backward compatibility

was a big concern – after in-depth review, not so much

Page 46: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 46

We could …

› Create a signal that works with existing HD/UHD STBs/DTVs

– But then we’re restricted to 8-bit AVC 1080i

› Create a signal that decodes with a new STB but looks “okay” on an SDR TV

› Create a signal that is optimal for HDR (1080p or 2160p)

– And then either:

› Use the existing transmission for SDR, or

› Convert in new STBs

Backward Compatibility –Options?

Page 47: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 47

› Existing HD SDR TV services will remain– Large populations of SDR-only or AVC-only STBs/DTVs

› What does this mean?

SDR HD transmission will remain unless there is an effective way to do

backward compatibility with a single service that does not

compromise either the new HDR image or the legacy SDR image …

What About Legacy HD Services?

Cost-effectively producing both

HDR and SDR means

one production chain

HDR must be suitable for

deriving SDR

SDR HD transmission

will remain and so is available

for use by SDR-only UHDTVs

Page 48: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 48

› Dynamic range: ITU-R BT.2100 (PQ or HLG) BT.709/BT.1886 (Gamma)

› Color space: ITU-R BT.2020 BT.709

› Sample bit depth: 10b 8b (for delivery-to-consumers)

› Spatial resolution: 3840x2160 1920x1080 or 1280x720 (as applicable)

› Temporal resolution: 50-60 fps progressive 25-30 fps interlaced

› Video coding standard: HEVC to AVC or MPEG-2 (for delivery-to-consumers)

› With “broadcast quality” images in both HDR and SDR formats

– For legacy conventional HD service

– For new Ultra HD service (1080p or 2160p HDR+)

Simulcast required unless all of these conditions are met*

*Note: the above still excludes modulation, transport, and audio BC!

Backward Compatibilityto legacy HD

Page 49: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 49

HDR+ Approaches: “PQ10”-based

› SMPTE ST 2084 PQ HDR + Rec. ITU-R BT.2020 color– One of two HDR+ formats defined in new Rec. ITU-R BT.2100 Image parameter values for

high dynamic range television for use in production and international programme exchange

› Single layer baseline is non backwards compatible

› HDR10 = PQ10 + reference display metadata

– Metadata = SMPTE ST 2086 HDR static metadata + MaxCLL + MaxFALL

– Specified by Blu-ray® Disc Association, DECE, CTA, UHD Alliance for pre-produced content

– Uses HEVC Color Remapping Information SEI message

Page 50: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 50

So Called “HDR10” media profileBDA: Ultra HD Blu-ray® Video Characteristics

Page 51: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 51

HDR+ Approaches: “PQ10”-based (2)

› Backwards compatibility possible by using proprietary add-on schemes

involving dual layers or single layer + “HDR enhancement” metadata

› Some examples

– Dolby Vision: Dual layer, HDR baseline + “SDR reconstruction” enhancement layer

– Dolby Vision Live: Single layer, HDR baseline + optional “Display Adaptation” metadata

(SMPTE ST 2094-1 & 2094-10 HDR dynamic metadata)› Also optional ICtCp color space + optional closed-loop Re-shaper

– Technicolor “SL-HDR1” (formerly “Prime”): Single layer, SDR baseline + “HDR

Reconstruction” metadata (SMPTE ST 2094-1 & 2094-30 HDR dynamic metadata)

– Qualcomm: HDR10 + “Dynamic Range Adjustment” metadata

Page 52: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 52

HDR+ Approaches: “HLG10”-based

› Hybrid-Log Gamma (HLG) HDR + Rec. ITU-R BT.2020 color

– One of two HDR+ formats defined in new Rec. ITU-R BT.2100 Image parameter values for

high dynamic range television for use in production and international programme exchange

› Single layer with no metadata

› Backwards compatible for Live TV (if BT.2020 color space container

maintained)

› Possibly backward compatible to HD (BT.709) color space with tone mapping

algorithm

– But some in industry do not believe doing BC this way is necessary for 4K

(very limited in-field)

– Instead, plan to simulcast conventional HD SDR with 4K HDR+

Page 53: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 53

› Fewer constraints than live

production

– Post-production can produce for just

that specific content

– Knows about pictures “in the future”

– A much more controlled environment

› Could exist in multiple formats if

needed

– Although more economical if there is

only 1 format for all

HDR+ for On-Demand Assets

Page 54: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 54

› Live TV ecosystems have special needs

› To get on-air in early adoption (2017-2019?), must keep the solution simple and “forgiving”, to work as best as possible with existing live workflows

› Some proposed HDR schemes require metadata or dual layer streams

› This information may get dropped until the HDR Live TV ecosystem matures (islands of implementations always occur in technology displacements)

› For early Live TV workflows, use HDR schemes that do not require metadata or dual layers so that if lost/missing/not produced, renderer is still able to produce “broadcast quality” HDR images

› “Bread & butter” will be conventional HD for a long time–Simulcast likely required in early deployments

Live TV challenges

Page 55: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 55

Live TV Content delivery is not likeon-demand, blu-ray, or cinema

Several HDR+ schemes had been proposed, based on discussions with Hollywood studios / Blu-ray Disc

Association / display manufacturers. None of whom, however, produce Live TV programming!

Jean Wake Split Time 1:22:12

RWT Live

Jean Wake Split Time 1:22:12Jean Wake Split Time 1:22:12

An impressive swim from wake

Page 56: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 56

Live TV Content production

Jean Wake Split Time 1:22:12

Now in real time we are adding lower thirds …This is the basic picture …

Page 57: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 57

Live TV Content production

Jean Wake Split Time 1:22:12Jean Wake Split Time 1:22:12Jean Wake Split Time 1:22:12

Adding transitions and logos, freeze frames,

slow motion, repositioning, etc.

Jean Wake Split Time 1:22:12

RWT Live

Jean Wake Split Time 1:22:12Jean Wake Split Time 1:22:12

Adding logos and captioning downstream

An impressive swim from wake

Page 58: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 58

Live TV Content delivery

This content often needs

further downstream

image manipulation

(mixing, wipes, fades,

keying, graphics) …

Logo replacement

“Squeeze & Tease”

Up Next

Down-conversion

Format conversion

Page 59: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 59

In Live TV, what/where is the “final encoder”?

“Just insert at the final encode”

Subsequent Broadcaster

(e.g., international distribution*)

Broadcast

Affiliate/Remote

Use Case

1

Use Case

2

Use Case

3Broadcast

Affiliate/Remote

Broadcast

Network/Source

Broadcast

Network/Source

Broadcast

Network/Source

*or MVPD (Cable or Satellite Provider ) in some regions

Page 60: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 60

Dual layer and single layer + “HDR enhancement” metadata

are difficult to apply to Live TV workflows

If there are dual-layers or enhancement metadata that need to be

reconstituted into a single stream to be processed and re-encoded,

then how do I do the following?:

– Transition between two sources, for example in a live studio program?

– Apply effects (which layer am I applying them to?)

– Add graphics (which layer am I applying them to?)

– Format or standards convert (which layer am I converting and how?)

– Add logos or closed captions

– Cut between programs and adverts

Dual-layer & “HDR Enhancement” metadata operational problems

Page 61: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 61

› Remember audio loudness issues?

Video “Loudness”!

Page 62: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 62

› Advertisers may use the

opportunity to grab attention by

introducing huge steps in light

levels

Video “Loudness”!

Page 63: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 63

Uncompressed Compressed

(consumer-grade)

4K (2160p) vs. 1080i HD 400% circa 250%

“HDR+” (HDR+WCG+10bit) 25-30% circa 0-20%

HFR (50-60fps 100-120fps) 200% circa 30%

Bandwidth impacts

In some cases, bandwidth also required to simulcast

legacy HD bitstreams in addition to new UHD HDR+ bitstreams

Page 64: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 64

› If bandwidth constraints prevent a broadcaster from offering all of

the new technologies, then focus on the “best bang for the bit”

–1080p50/60 HDR+

› Take advantage of all modern displays’ ability to up-convert 1080p

to 4K (2160p)

–Of course, HDR+ support required to render HDR+

So what about 1080p HDR+?

Page 65: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 65

› UHDTV is all about the consumer experience

– But UHD-1 Phase 1 does not include immersive technologies!

› UHD-1 Phase 2 defined commercially … technical specs expected end 2016

UHD Roadmap (DVB)

* The specification also includes

fractional variants of the 60fps

family (1/1.001)

Source: DVB

CM-UHDTV

2160p100/120*UHD-1 Phase 2

CP “B” (HFR)Bit depth: 10-bit

Colour Space: BT.2020

Dynamic Range: HDR

Backwards compatibility

options: Desired

Audio: Next generation audio

(e.g. Object-based )

2019/2020

2017/2018

2014/2015

2160p50/60*

Bit depth: 10 bit

Colour Space: BT.2020

Dynamic Range: HDR

Backwards compatibility options:

Desired

Audio: Next generation

audio (e.g. Object-based)

UHD-1 Phase 2

CP “A” (HDR+)

2160p50/60*

Bit depth: 8-bit, 10-bit,

Colour Space: Rec. BT.709 and

BT.2020 (signalled)

Dynamic Range: Standard

Audio: Existing DVB Toolbox

Hooks for High Frame Rate compatibility

included

UHD-1 Phase 1

Page 66: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 66

› CTA

– Draft CTA-861-G – A DTV Profile for Uncompressed High Speed Digital Interfaces (core of HDMI). To add HDR dynamic metadata & HLG OETF to HDR support already in CTA-861.3

› Ultra HD Forum

– Ultra HD Forum Guidelines (for UHD Phase A)

– Supports 1080p and 2160p HDR+

– PQ10 and HLG10 for Live TV workflows

› UHD Alliance

– UltraHD Premium specs and logo certification program

– Broadcasting Sub-Group to consider the definition or recommendation of a reference display environment

› ATSC, SCTE, others …

– Standardizing the use of HDR+ in direct-to-home/consumer systems

Other Recent/on-going standards activities

High Dynamic Range Display

Peak Brightness: >1000 nits

Black Level: <0.05 nits

or

Peak Brightness: >540 nits

Black Level: <0.0005 nits

Page 67: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 67

› Call for Evidence for HDR/WCG (February 2015)– Input contributions showed that low bitrate applications did not perform as expected,

including poor texture rendition, color shifts after compression, color boundary overflow & chroma sub-sampling issues

› January 2016: Decision made not to add new HDR tools to HEVC due to lack of significant improvements of candidate proposals over improved encode-side only techniques (no changes to bitstream syntax or reference decoder)

› 3 techniques improved the “anchors” (source images used in evaluation)– Luma Adjustment (Ericsson)

› Preprocessing before encoding› Removes subsampling artifacts

– Chroma QP offset (Ericsson)› Encoder optimization› Removes chrominance artifacts

– Luma QP offset (Ericsson & Sharp)› Encoder optimization› Increases detail

› Next step: “Best practices” recommendation & verification testing

MPEG HEVC HDR “Fast TracK”

Page 68: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 68

Original 4:4:4Conventional 4:2:0

(no compression)Ericsson 4:2:0

(no compression)

Luma adjustment (ERICSSON)

© Ericsson 2016 | 2016-08-31 | Page 68

Page 69: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 69

Luma adjustment (ERICSSON)

Original 4:4:4Conventional 4:2:0

(no compression)Ericsson 4:2:0

(no compression)© Ericsson 2016 | 2016-08-31 | Page 69

Page 70: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 70

Chroma qp offset (ERICSSON)

Anchor v3.2 matching bit-rateCfE Anchor

Page 71: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 71

Chroma qp offset (ERICSSON)

CfE Anchor

Green Artifacts

Anchor v3.2 matching bit-rate

Page 72: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 72

Chroma qp offset (ERICSSON)

Red Artifacts

CfE Anchor Anchor v3.2 matching bit-rate

Page 73: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 73

Luma QP offset (ERICSSON & SHARP)

CfE Anchor Anchor v3.2 matching bit-rate

Page 74: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 74

Luma QP offset (ERICSSON & SHARP)

CfE Anchor Anchor v3.2 matching bit-rate

More

Texture

Detail

Page 75: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 75

Luma QP offset (ERICSSON & SHARP)

CfE Anchor Anchor v3.2 matching bit-rate

Sharper

Edges

Page 76: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 76

› HDR has the interest to be a major success

– Highly visible improvement in the immersive viewing experience

› Both 1080p HD and 2160p 4K UHD initially … and 4320p 8K UHD some day

› A lot of the confusion surrounding the standards and proposals is now resolving

› Simplicity and content interoperability are key

› Initially, PQ10 and HLG10 for Live TV workflows

Summary

Page 77: High Dynamic Range Master Class - set6.tempsite.wsset6.tempsite.ws/eventos/palestras/matthew_goldman_set_expo_2016… · High Dynamic Range Master Class ... H H Proper Viewing Distance

© Ericsson 2016 | 2016-08-31 | Page 77