View
41
Download
0
Category
Tags:
Preview:
DESCRIPTION
A Hybrid Scheme for Low Bit-Rate Coding of Stereo Images. Jianmin Jiang, Eran A. Edirisinghe University of Bradford, BradfordLoughborough University, Leicestershire IEEE TRANSACTIONS ON IMAGE PROCESSING, FEB 2002. OUTLINE. INTRODUCTION (Stereo Image) AN OVERVIEW OF THE HYBRID SCHEME - PowerPoint PPT Presentation
Citation preview
A Hybrid Scheme for Low Bit-Rate Coding of Stereo Images
Jianmin Jiang, Eran A. EdirisingheUniversity of Bradford, Bradford Loughborough University, Leicestershire
IEEE TRANSACTIONS ON IMAGE PROCESSING, FEB 2002
OUTLINE
INTRODUCTION (Stereo Image) AN OVERVIEW OF THE HYBRID
SCHEME CONTOUR ANALYSIS OF STEREO
IMAGES FREE-FORM OBJECT ENCODING EXPERIMENTAL RESULTS
INTRODUCTION
Stereo Image Stereo images are constructed by
simulating human viewing effect upon observing objects through two horizontally separated positions
Every stereo image can be represented two frames labeled as left frame and right frame.
INTRODUCTION (cont.) If these left and right frames are to be
compressed and transmitted independently, the bandwidth required would need to be double.
Efficient compression can be achieved by exploiting redundancies Intra-frame redundancy
Two constituent images Inter-frame redundancy
Left and right frames
INTRODUCTION (cont.) Estimation for disparity or binocular para
llax Intensity based method
Look for correspondence between luminance values (eg. Block matching) Fixed blocks can’t reflect the true disparity betwe
en those free-form objects. Object based method
Determine objects in both images and seeks correspondence between the two sets (eg. The proposed scheme)
AN OVERVIEW OF THE HYBRID SCHEME
Left
Right
JPEG
iJPEG
ObjectExtraction
&match R-Obj
L-ObjBlock
Classify
Boundarycoder
Interiorcoder
DCTcoder
Backgroundcoder
Block Disparity
Object Disparity
CONTOUR ANALYSIS OF STEREO IMAGES
Contour Extraction Laplacian-of-Gaussian (LoG) operator ,σ is the standard deviation The LoG response will be positive on the dar
ker side, and negative on the lighter side
2
22
22
22
4]
21[
1),(
yx
eyx
yxLoG
The left hand graph shows a 1-D image, 200 pixels long, containing a step edge.
The right hand graph shows the response of a 1-D LoG filter with Gaussian=3 pixels
intensity
Contour Extraction (cont.) Let sx and sy be the slops of the LoG of
the image along both x and y directions for some zero-crossing point.
An edge strength at (x, y) is defined as
The contour points are chosen by using a pre-set threshold
CONTOUR ANALYSIS OF STEREO IMAGES
1
463
2141
21 1
-1-2
-3
-2-4
CONTOUR ANALYSIS OF STEREO IMAGES
Contour Matching Closed contours Open contours Background = + + …
We use chain codes {ai 0, 1, …, 7} to represent the contours
04
CONTOUR ANALYSIS OF STEREO IMAGES
Contour Matching Closed contours (hui Li, Sanjit k. Mitra, 1995)
Shifting and smoothing the chain codes We convert the standard chain code {ai} into a modi
fied code {bi} by a shifting operation defined by
e.g. {7,0,7,0,7,0} -> {7,8,7,8,7,8} e.g.
{2,0,7,4} {4,2,1,6}
{2,0,-1,-4} {4,2,1,-2}
Mean = -1 1 {3,1,0,-3}-Mean
CONTOUR ANALYSIS OF STEREO IMAGES
Chain code
Chain code
smoothing & shifting
smoothing & shifting
CONTOUR ANALYSIS OF STEREO IMAGES Contour Matching (Closed contours)
Measure of correlation between two n-point segments, one starting at index k of contour L and the other starting at index l of contour R, is defined as:
Where
{li} and {ri} be the chain code representations of two contours L and R, which corresponds to the left and right frames, and let NL and NR be their lengths
The similarity function FLR = max {MCkl} If FLR≧ FL’R, where L’ represents all similar contours to R, the b
est match is selected.
1
0
'' )(4
cos1 n
j
jljkkl rln
MC
1
0
mod)(mod)(1
'n
j
NjkNikik LL ln
ll
n pointssegments
RL
lk 80
4MCkl=1 when perfect match
L R
CONTOUR ANALYSIS OF STEREO IMAGES
Contour Matching (open contours) Measure of curvature at the ith point for a co
ntour of length n with chain code {li}
The contour segments surrounding the salient points are then used as 1-D templates in finding the corresponding matches in the other image
|}}||,{max{|max 131
jijijijij
i llllC
If Ci ≧Ck for all k [i-p, i+p], where p is a constant, we say this point is a salient point
2p
2p
L R
CONTOUR ANALYSIS OF STEREO IMAGES
Left
Right
JPEG
iJPEG
ObjectExtraction
&match R-Obj
L-ObjBlock
Classify
Boundarycoder
Interiorcoder
DCTcoder
Backgroundcoder
Block Disparity
Object Disparity
CONTOUR ANALYSIS OF STEREO IMAGES
Contour Blocking and Classification8x8 pixels block
Internal block
External block
Boundary block
Bounding box
Object Bounding Rectangle (OBR)
We extend the bounding box such that its width and height are multiples of eight pixels
FREE-FORM OBJECT ENCODING
Padding of left OBR Extensive experiments indicated that
certain trends exist with respect to pixel value variations for boundary blocks.
We want to use a linear equation to predict those exterior pixels immediately next to to the boundary pixels in a row (projected pixel).
(E.A. Edirisinghe, J. Jiang and C. Grecos 1999)
Interior pixel
Projected pixel
FREE-FORM OBJECT ENCODING
Padding of left OBR (cont.) In a given row, assume there exist n (1 < n <
N) consecutive pixels inside the object,which are bounded by a projected pixel on the left or on the right . Let these n pixel values be represented by Pn.
BXAP nn Where Xn represents the column number of Pn with respect to the projected pixel
n = 2
N
N
FREE-FORM OBJECT ENCODING Padding of left OBR (cont.)
1. When Pn are bounded by two projected pixels, both projected pixels will be determined using the same linear equation.
2. If a projected pixel is flanked by interior pixels on both sides, the above process is performed in both directions and the average of the two results is taken.
3. If n = 1, the projected pixel value is taken to be equal to the single interior pixel value.
4. Remaining exterior pixels are padded with traditional technique.
Padding of left OBR (cont.) After all the boundary blocks are padded,
the exterior blocks immediately next to the boundary blocks are filled by replicating the samples at the border of boundary blocks with the priority
1. Boundary block to the left of exterior block2. Block at the top3. Block to the right4. Block at the bottom (lowest priority)5. Remaining blocks are filled with value of 128
FREE-FORM OBJECT ENCODING
3 P PP P
3 P 1P 2
3 P P P P P 1P 5
2 P P P P 515
5 3 P P 1 4 44 4
3 P P P P P
5 3 P P P P PP P
1
FREE-FORM OBJECT ENCODING
Left
Right
JPEG
iJPEG
ObjectExtraction
&match R-Obj
L-ObjBlock
Classify
Boundarycoder
Interiorcoder
DCTcoder
Backgroundcoder
Block Disparity
Object Disparity
FREE-FORM OBJECT ENCODING
Disparity-Based Prediction for Both Boundary Blocks and Internal Blocks Boundary blocks
Search range [-7,7]. Alpha plane (L unpadded)
Square error (L padded)
W = [-7,+7]
Left obj. Right obj.
{),(,1..,0
),(Llkif
wolkAlphaL
{
),(,1..,0
),(Rsrif
wosrAlphaR
8
1,
2)()_,_(ji
Rijij AlphayxblockLblockRSE
)_,_(min__
blockLblockRSEblockLwblockL
match
0011
0011
0111
1111
-( )2 x
RL
The best match
Since the stereo pair is produced by observation of the same scene from two
horizontally separated positions, disparity between the two frames will only occur
horizontally rather than vertically
FREE-FORM OBJECT ENCODING
Disparity-Based Prediction for Both Boundary Blocks and Internal Blocks Boundary blocks
Error block : The shape of the right object would be similar t
o that of its matching left object, so we won’t encode the shape of right OBRs.
R
= {sij}
)()( LijLijij AlphamAlphasc
= {mij}
L (padded)
FREE-FORM OBJECT ENCODING
Disparity-Based Prediction for Both Boundary Blocks and Internal Blocks Internal blocks
Use the two blocks to be pioneering blocks, one at the top and one at the left of the block to be encoded
No overhead bits needed for informing the decoder of which block is chosen in the left frame
(J. Jian, E.A. Edirisinghe and H. Schroder 1997)
Rij-1
RijRi-1j
XX
FREE-FORM OBJECT ENCODING
Rij-1
RijRi-1j
Lij-1
LijLi-1jBlock to be encoded
Pioneering blocks
Best match
predictor
Search window|-----N x 8-----|
L R
11
_ijji
RRR
blockP ij α,β (0,1], the weight of each block
1. Use Rij-1 and Ri-1j to compute Rij
2. Find the best match Lij in L, where L is produced by Lij-1 and Li-1j
FREE-FORM OBJECT ENCODING
Internal texture smooth detection Calculate the error block and MSE between
two matching regions. If MSE is less then some threshold, the error block won’t be encoded.
If MSE is too large, object extraction would be proceeded within current object.
matching coding
EXPERIMENTAL RESULTSE.C.R = (bit_NoJ – bit_NoP)/bit_NoJ
(Bit_NoJ and bit_NoP: total bits produced in compressing right frames by JPEG and by the proposed algorithm)
DCTDP : disparity compensated transform domain predictive coding(block based)
EXPERIMENTAL RESULTS
The reference image quality is poor, so matches between objects will produce larger error for both boundary and internal areas.
EXPERIMENTAL RESULTS
Visual inspection for example (right frame)
original DCTDP proposed
Recommended