1 Metadata for Formatting with Multiple Aspect Ratios Graham Jones National Association of Broadcasters Chair – SMPTE S22 Image Formatting AHG HPA Technology

1

Metadata for Formattingwith Multiple Aspect Ratios

Graham JonesGraham JonesNational Association of BroadcastersNational Association of Broadcasters

Chair – SMPTE S22 Image Formatting AHGChair – SMPTE S22 Image Formatting AHG

HPA Technology RetreatHPA Technology RetreatFebruary 24, 2006February 24, 2006

2Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006

Metadata for Image Formatting

• Joint initiative by SMPTE, ATSC, and CEA, to produce an end-to-end system solution for image formatting as related to aspect ratio

• Harmonized standards for format of necessary metadata, metadata carriage, and recommended practices for applications

• Work in progress – estimated completion mid-2006


Issues

• Optimize display of the image for the television viewer– Multiple production aspect ratios– At least two display aspect ratios

• Provide guidance for the format conversion process– In the receiver and display– In post and pre-broadcast

• Minimize changes to images created by the original program producer

• Maintain critical image content


Image Aspect Ratios4:3

16:9

>16:91.852.39etc.

14:9


Displays


Displayed Images

16:9 4:3

4:316:916:9

4:34:3


Undesirable Results

Produce

4:3 4:3 using letterbox

Viewer DisplayBroadcast

16:9 using pillarbox

“Postage stamp”


Undesirable Results

Produce

16:9 16:9 using pillarbox


4:3 using letterbox

“Postage stamp”

Other undesirable images – stretched, squashedor vital picture information lost due to cropping


Undesirable Results

Produce

16:9 16:9 using pillarbox


4:3 using letterbox

“Postage stamp”X


Image Formatting Metadata

• AFD – Active Format Description• Bar Data• Pan-Scan Information• Individual specifications already exist in standards

– AFD: MPEG, ETSI/DVB, ATSC– Bar Data: ATSC– Pan-scan: SMPTE, MPEG-2 (also AVC and VC-1)

• Currently no carriage in HD baseband or MXF• Limited guidelines on applications and use


US Network Distribution

LocalStation

ATSCDigital

NTSCAnalog

RemoteProduction

& PostVenues Contribution

HD or SDNetworkReleaseCenter

DTV

NTSC

CEAATSCMPEGSMPTE


AFD, Bar Data, and Pan-Scan Carriagein Professional Environment

• SMPTE standards in preparation for carriage of AFD, Bar Data, and Pan-Scan in VANC of SDI and HD-DSI for the studio environment

• These standards also define KLV encoding for carriage in MXF, AAF, etc.

Active Picture Area

Vertical ANCillary data space

Hor

iz. A

NC

illa

ry d

ata

spac

e


DTV System Block Diagram

RFTransmission

System

Ancillary Data

ChannelCoding

Modulation

Control Data

Audio Subsystem

AudioSource Coding& Compression

Transport

Service Multiplex &Transport

Ser

vice

Mu

ltip

lex

Video Subsystem

VideoSource Coding& Compression

Studio/Master Control

Audio

VideoR

oute

r/M

C S

wit

cher

SMPTE 259M: 601 video (525/625) (SDI)SMPTE 292M: 1080i/p & 720p (HD-SDI)


Active Format Description - AFD

• Four-bit code that defines the active image area and protected image areas for various combinations of aspect ratios – AFD is independent of image resolution

• Intended to guide receivers and STBs in how to optimize the displayed image– Also applications in the post and pre-broadcast chain

• Recommended practice for use of AFD and Bar Data now in preparation by CEA


In 4:3 coded frame DescriptionIllustration of described format

Active Format

AFD = ‘0010’Box 16:9 (top)

AFD = ‘0011’Box 14:9 (top)

AFD = ‘0100’Box >16:9 (centre)

AFD = ‘1010’16:9 (centre)

AFD = ‘1011’14:9 (centre)

AFD = ‘1101’4:3 (with alternate14:9 center)



AFD = ‘1000’Full frame

Image with a 16:9 aspect ratio as letterbox atthe top of a 4:3 coded frame.


Image with aspect ratio greater than 16:9 asa vertically centered letterbox in a 4:3 codedframe.

Image is full frame, with an aspect ratio that isthe same as the 4:3 coded frame.

Image with a 16:9 aspect ratio as a verticallycentered letterbox in a 4:3 coded frame.

Image with 14:9 aspect ratio as a verticallycentered letterbox in a 4:3 coded frame.

Image with a 4:3 aspect ratio and with analternate 14:9 center in a 4:3 coded frame.

Image with a 16:9 aspect ratio and with analternate 14:9 center as a vertically centeredletterbox in a 4:3 coded frame.



Active Format

AFD = ‘0010’Box 16:9 (top)

AFD = ‘0011’Box 14:9 (top)


AFD = ‘1010’16:9 (centre)

AFD = ‘1011’14:9 (centre)







Image with aspect ratio greater than 16:9 asa vertically centered letterbox in a 4:3 codedframe.

Image is full frame, with an aspect ratio that isthe same as the 4:3 coded frame.

Image with a 16:9 aspect ratio as a verticallycentered letterbox in a 4:3 coded frame.

Image with 14:9 aspect ratio as a verticallycentered letterbox in a 4:3 coded frame.

Image with a 4:3 aspect ratio and with analternate 14:9 center in a 4:3 coded frame.



AFD Codes in a 4:3 Coded Frame


AFD Codes in a 16:9 Coded Frame


Image with aspect ratio greater than 16:9as a vertically centered letterbox in a16:9 coded frame.


Active Format

AFD = ‘1111’16:9 (with alternate4:3 centre)

AFD = ‘1011’14:9 (centre)



Image with a 14:9 aspect ratio as ahorizontally centered pillarbox imagein a 16:9 coded frame.

Image with a 4:3 aspect ratio and with analternate 14:9 center as a horizontallycentered pillarbox image in a 16:9 codedframe.

Image with a 16:9 aspect ratio and withan alternate 14:9 center in a 16:9 codedframe.



Image is full frame, with an aspect ratiothat is the same as the 16:9 codedframe.

AFD = ‘1010’16:9 (with complete16:9 image protected)

Image is full frame, with a 16:9 aspectratio and with all image areas protected.

AFD = ‘1001’4:3 (centre)



Image with aspect ratio greater than 16:9as a vertically centered letterbox in a16:9 coded frame.


Active Format


AFD = ‘1011’14:9 (centre)




Image with a 4:3 aspect ratio and with analternate 14:9 center as a horizontallycentered pillarbox image in a 16:9 codedframe.




Image is full frame, with an aspect ratiothat is the same as the 16:9 codedframe.

AFD = ‘1010’16:9 (with complete16:9 image protected)

Image is full frame, with a 16:9 aspectratio and with all image areas protected.

AFD = ‘1001’4:3 (centre)



Format Conversion with AFD

16:9 Letterbox Postage stamp Full Frame

SD 4:3

UP

Without AFD(undesirable)

With AFD(desirable)

HD 16:9

With AFD, original image information is maintained throughout conversion processes, while optimal viewing area is reached.

AFD code 1010Letterbox 16:9 image vertically centered in the frame


Image Aspect Ratio >16:9

AFD code 0100aspect ratio >16:9

4:3 16:9


Bar Data

• Signaled with AFD 0100 (aspect ratio >16:9) to indicate the exact width of the top and bottom bars

• May also be signaled with AFD 0000 (undefined) to indicate the width of the bars for any other non-standard active image area

• Bar Data signals one pair of bars only– Top/bottom or left/right

– Not intended to signal a postage stamp - such transmitted images should be discouraged


Bar Data Example

2.40:1 ACTIVE IMAGE(1280 x 534)

IN A 16:9 FRAME

TOP BAR

BOTTOM BAR

1280 Luma Samples

534Lines

1280 x 720Progressive Frame

Line Number(SMPTE 296M)

26118

653745

720Lines

AFD code 0100 (aspect ratio >16:9)


Pan-Scan Information

• Used to signal how to extract a different aspect ratio viewport from a source image

• In broadcast world intended for professional use


Pan-Scan Information

SOURCE

VIEWPORT

V-Size

Source Center

Viewport CenterH-Size

Offset-V

Offset-H


Pan and Scan

© Warner Bros.Screen shot courtesy of Letterbox and Widescreen Advocacy Page

www.widescreen.org


US Network DistributionPan-Scan Application

LocalStation

ATSCDigital

NTSCAnalog

RemoteProduction


HD or SD

NetworkCenter

NTSC

DTV



LocalStation

ATSCDigital

RemoteProduction


HD or SD

NetworkCenter

For MPEG links, Pan-Scan information is carried in display extensions of the video sequence header

DTV

NTSCAnalog



LocalStation

ATSCDigital

RemoteProduction


HD or SD

NetworkCenter

DTV

For MPEG links, Pan-Scan information is carried in display extensions of the video sequence header


Output Documents

• CEA – recommended practice for use of AFD and Bar Data by receivers

• ATSC – minor revisions to A/53 for semantics of AFD and Bar Data for consistency with CEA RP

• SMPTE standards for carriage of AFD, Bar Data, and Pan-Scan information in VANC of SD and HD video and as KLV data

• SMPTE – recommended practice for operational use of AFD, Bar Data, and Pan-Scan information– Contributions welcome


Related papers at NAB Broadcast Engineering Conference

Adam Goldberg:The Importance of Video Format Metadata

Bill Miller:Canceling the Postage Stamp


Thank you

[email protected]@nab.org