Upload
brandy-hodgkinson
View
243
Download
3
Tags:
Embed Size (px)
Citation preview
Basics of Video
Courtesy of Professor Yao WangPolytechnic University, Brooklyn, NY11201
Adapted from Yao Wang, 2004
Video Basics
Outline
• Video capture– Photometric model– Geometric model
• Analog video– Progressive vs. interlaced rasters in analog TV system– Different color representations: YUV/YIQ
• Digital video– Sampling/quantization– Y’CbCr format
• Video display– Spatial/temporal/bit-depth resolution
Adapted from Yao Wang, 2004
Video Basics 3
Photometric Model of Video Capture
Courtesy of Onur Guleryuz
Adapted from Yao Wang, 2004
Video Basics 4
Geometric Model of Video Capture
Cameracenter
Imageplane
2-Dimage
3-Dpoint
The image of an object is reversed from its 3-D position. The object appears smaller when it is farther away.
Z
YFy
Z
XFx ,
Adapted from Yao Wang, 2004
Video Basics 5
Implication of Models in Analog World
Miniature building Explosion from The Mummy Lighting in Filmmaking
Adapted from Yao Wang, 2004
Video Basics 6
Progressive and Interlaced Raster Scans
Field 1 Field 2
Progressive Frame Interlaced Frame
Interlaced scan is developed to provide a trade-off between temporal and vertical resolution, for a given, fixed data rate (number of line/sec).
Horizontal retrace
Vertical retrace
Adapted from Yao Wang, 2004
Video Basics 7
Color TV Broadcasting and Receiving
Lum ina nc e ,C hro m ina nc e ,Aud io M ultip le xing
M o d ula tio n
De -M o d ula tio n
De -M ultip le xing
YC 1C 2--->RG B
RG B--->
YC 1C 2
Adapted from Yao Wang, 2004
Video Basics 8
Why not using RGB directly?
• R,G,B components are correlated– Transmitting R,G,B components separately is redundant– More efficient use of bandwidth is desired
• RGB->YC1C2 transformation– Decorrelating: Y,C1,C2 are uncorrelated– C1 and C2 require lower bandwidth– Y (luminance) component can be received by B/W TV sets
Color transformation is a compromised solution, but the ultimate one
Adapted from Yao Wang, 2004
Video Basics 9
YIQ in NTSC
• I (in-phase): orange-to-cyan
• Q (quadrature): green-to-purple (human eye is less sensitive)– Q can be further bandlimited than I
• Phase=Arctan(Q/I) = hue, Magnitude=sqrt (I^2+Q^2) = saturation
• Hue is better retained than saturation
Recall: Quadrature amplitude modulation (QAM) in digital communication
Color Image Y image
I image (orange-cyan) Q image (green-purple)
Adapted from Yao Wang, 2004
Video Basics 11
I and Q on the color circle
I: orange-cyan
Q: green-purple
Adapted from Yao Wang, 2004
Video Basics 12
Conversion between RGB and YIQ
Y = 0.299 R + 0.587 G + 0.114 BI = 0.596 R -0.275 G -0.321 BQ = 0.212 R -0.523 G + 0.311 B
R =1.0 Y + 0.956 I + 0.620 Q,G = 1.0 Y - 0.272 I -0.647 Q,B =1.0 Y -1.108 I + 1.700 Q.
• RGB -> YIQ
• YIQ -> RGB
Adapted from Yao Wang, 2004
Video Basics 13
YUV in PAL
Adapted from Yao Wang, 2004
Video Basics 14
YUV/RGB Conversion
Y = (( ( 66 * R + 129 * G + 25 * B + 128) >> 8) + 16)U = ( ( -38 * R - 74 * G + 112 * B + 128) >> 8) + 128V = ( ( 112 * R - 94 * G - 18 * B + 128) >> 8) + 128
Numerical approximations
Adapted from Yao Wang, 2004
Video Basics 15
YIQ/YUV Comparison
Adapted from Yao Wang, 2004
Video Basics 16
Different Color TV Systems
Parameters NTSC PAL SECAM
Field Rate (Hz) 59.95 (60) 50 50
Line Number/Frame 525 625 625
Line Rate (Line/s) 15,750 15,625 15,625
Color Coordinate YIQ YUV YDbDr
Luminance Bandwidth (MHz) 4.2 5.0/5.5 6.0
Chrominance Bandwidth (MHz) 1.5(I)/0.5(Q) 1.3(U,V) 1.0 (U,V)
Color Subcarrier (MHz) 3.58 4.43 4.25(Db),4.41(Dr)
Color Modulation QAM QAM FM
Audio Subcarrier 4.5 5.5/6.0 6.5
Total Bandwidth (MHz) 6.0 7.0/8.0 8.0
Adapted from Yao Wang, 2004
Video Basics 17
Who uses what?
From http://www.stjarnhimlen.se/tv/tv.html#worldwide_0
Adapted from Yao Wang, 2004
Video Basics 18
Digital Video
Taken from EE465: Image Acquisition
Sampling Quantization
Adapted from Yao Wang, 2004
Video Basics 19
BT.601* Video Format
480
line
s
525
line
s
12 2p e l
16p e l
85 8 p e ls
72 0 p e ls
Ac tiveAre a
52 5/60 : 60 fie ld /s57
6 lin
es
625
line
s
86 4 p e ls
13 2p e l
12p e l
72 0 p e ls
Ac tiveAre a
62 5/50 : 50 fie ld /s
* BT.601 is formerly known as CCIR601
Adapted from Yao Wang, 2004
Video Basics 20
RGB <--> Y’CbCr
Analog video
Digital video
Adapted from Yao Wang, 2004
Video Basics 21
YUV vs. Y’CbCr
Adapted from Yao Wang, 2004
Video Basics 22
Chrominance Subsampling Formats
4 :2 :0F o r ev er y 2x 2 Y P ix els
1 C b & 1 C r P ix el(S u b s am p l in g b y 2 :1 b o thh o r i zo n ta l l y an d v e r t i c al l y )
4:2:2 Fo r ev er y 2x 2 Y P ix e ls
2 C b & 2 C r P ix e l(S u b s am p lin g b y 2:1
h o r izo n tal l y o n ly )
4 :4:4 Fo r ev er y 2x 2 Y P ix e ls
4 C b & 4 C r P ix e l(N o s u b s am p l in g )
Y P ix el C b an d C r P ix el
4:1:1Fo r ev er y 4x 1 Y P ix e ls
1 C b & 1 C r P ix e l(S u b s am p lin g b y 4:1
h o r izo n tal ly o n ly )
Adapted from Yao Wang, 2004
Video Basics 23
Digital Video Formats
Video Format Y Size Color Sampling
Frame Rate (Hz)
Raw Data Rate (Mbps)
HDTV Over air. cable, satellite, MPEG2 video, 20-45 Mbps SMPTE296M 1280x720 4:2:0 24P/30P/60P 265/332/664 SMPTE295M 1920x1080 4:2:0 24P/30P/60I 597/746/746 Video production, MPEG2, 15-50 Mbps BT.601 720x480/576 4:4:4 60I/50I 249 BT.601 720x480/576 4:2:2 60I/50I 166 High quality video distribution (DVD, SDTV), MPEG2, 4-10 Mbps BT.601 720x480/576 4:2:0 60I/50I 124 Intermediate quality video distribution (VCD, WWW), MPEG1, 1.5 Mbps SIF 352x240/288 4:2:0 30P/25P 30 Video conferencing over ISDN/Internet, H.261/H.263, 128-384 Kbps CIF 352x288 4:2:0 30P 37 Video telephony over wired/wireless modem, H.263, 20-64 Kbps QCIF 176x144 4:2:0 30P 9.1
Adapted from Yao Wang, 2004
Video Basics 24
4:2:0 YUV Video
Y: 288-by-352
U: 144-by-176 V: 144-by-176
Adapted from Yao Wang, 2004
Video Basics 25
Tricky Photometric Situations
Shadow causes problem to background extraction
Video enhancement
Adapted from Yao Wang, 2004
Video Basics 26
Geometric Invariance
Adapted from Yao Wang, 2004
Video Basics 27
Video Display
• High-end– If the resolution of display device is higher than that of video
sequence, what can we do?
– Tradeoff between quality and complexity
– Subjective evaluation of video quality
• Low-end– If the resolution of display device is lower than that of video
sequence, what can we do?
– What if the bit-depth resolution is lower? (e.g., display video on PDAs and portable DVDs)
It is the last and the least-researched component in visual communication systems
Adapted from Yao Wang, 2004
Video Basics 28
Resolution, Resolution, Resolution
spatial
temporal
Bit-depth
1M 10M
30fps
300fps
8bpp
32bpp
Adapted from Yao Wang, 2004
Video Basics 29
High Dynamic Range Imaging
Q: Can we generate a HDR image (16bpp) by a standard camera?A: Yes, adjust the exposure and fuse multiple LDR images together
Adapted from Yao Wang, 2004
Video Basics 30
HDR Display (after Toner Mapping)
Note that any commercial display devices we see these days are NOT HDR