Upload
adele-hawkins
View
218
Download
0
Embed Size (px)
Citation preview
1
Byung Cheol Song
Shin-Cheol Jeong
Yangl im Choi
Video Super-Resolution Algorithm Using Bi-directional
Overlapped Block Motion Compensation
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 3, MARCH 2011
2
IntroductionMotion-compensated SR
Basic concept Hierarchical motion estimation Bi-directional OBMC Result
Learning-based SRHybrid super-resolutionExperimental Results
OUTLINE
3
For the last few decades, many image interpolation algorithms have been developed to display high quality scaled images on cutting-edge digital consumer application such as HDTV,DSC.
Traditional interpolation methods usually suffer from several types of visual degradation.
To overcome the problem, many algorithms are proposed. But there are many problem.(Ex. structurally weak against textures, computational complexity)
INTRODUCTION
4
Brandi et al. presented an interesting SR approach. They defined the so-called key frames (KF) that sparsely exist in a video
sequence and have HR resolution. The remaining frames in the video sequence, i.e., non-key frames(NKF) had LR resolution
Brandi et al. took advantage of the fact that few KF (encoded at HR) may provide enough HF information to the up-scale NKF (encoded at LR)
However, it rarely found true motion information because it made use of the conventional full-search motion estimation.
This paper presents a hybrid SR algorithm where each LR patch is adaptively selected between a temporally super-resolved patch and a spatially super-resolved patch using adjacent HR KFs. Temporally super-resolved patch MSR Spatially super-resolved patch LSR
INTRODUCTION
5
A target frame in LR video sequence is interpolated by using forward and backward HR key-frame(KF).
MOTION-COMPENSATED SR Basic Concept
• The KF interval N can be constant or variable.• These paper employ cubic convolution for Up-Scaling.
UnHR U
nHRUHR0U
NHR
6
First, the MV for overlapping M×M matching block is searched by using a rate-constrained ME.
The conventional rate-constrained ME find the best v for each matching block by minimizing the rate as well as distortion
In order to maximize the ME performance and to concurrently reduce the computational burden, we adopt a rate-constrained fast full search algorithm presented in [15].
MOTION-COMPENSATED SR Hierarchica l Moti on Esti mati on
λ : Lagrange multipier
7
Assume that MVs between up-scaled LR frames are statistically very similar to those between corresponding HR frame.
Replace the unknown MVs for HR frames with the MVs obtains from the up-scaled LR frames.
Employ BOBMC based on bi-directional MVs .the BOBMC is performed on a 4×4 block basis.
Each MV may be a forward or backward MV. The direction is determined according to SAD
MOTION-COMPENSATED SR B i -d irecti onal OBMC
9
It cannot often provide acceptable visual quality due to non-translational motion, occlusion, inaccurate motion estimation and limited motion search range.
When such temporal motion compensation does not work well, we employ a learning-based SR in order to avoid the degradation of visual quality.
MOTION-COMPENSATED SR Drawback
10
On-the-fly learning stage Construct a trained dictionary using adjacent LR KFs and HR KFs.
Inference stage The best-matched patch for the is exhaustively searched from the
dictionary.(the patch having minimum SAD is chosen as best-match) Produce a spatially super-resolved HR patch from
LEARNING-BASED SR
Produce Laplacian patch
K-means clustering
Seek optimal
LMS algorithm
Optimal dictionary
11
COMPARE MSR & LSR
MSR LSR
Background is well- motion-compensated
Seldom be well- motion-compensated
Better Quality
12
HYBRID SUPER-RESOLUTION
• The boundary between the temporally super-resolved patches and spatially super-resolved patches, we can observe blocking artifact because they are derived from different frames.
• We apply a simple smoothing filter to the boundary pixels
13
PSNR performance of the MSR according to N
EXPERIMENTAL RESULTS
Without/with little global motion N seldom affects
With large global motion
14
EXPER
IMEN
TAL R
ESULTS
N = 30M =16L = 4Search range =±64
MSR Hybrid SR
[1] X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Trans. Image Process., vol. 10, no. 10, pp. 1521–1527, Oct. 2001.[12] W. Fan and D. Y. Yeung, “Image hallucination using neighbor embedding over visual primitive manifolds,” in Proc. CVPR, 2007, pp. 1–7.[14] F. Brandi, R. Queiroz, and D. Mukherjee, “Super-resolution of video using key frames and motion estimation,” in Proc. IEEE ICIP, Oct.2008, pp. 321–324.
Fan’s[12]Barsiu’s
Brandi’s[14]
NEDI[1]
BLI Bi-cubic