36
Dipartimento di Elettronica e Informazione Motion Estimation from a Single Blurred Image Image Restoration: De-Blurring Build a “Blur Map” Adapt Existing De-blurring Techniques to real blurred images

Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Motion Estimation from a Single Blurred Image

Image Restoration: De-Blurring• Build a “Blur Map”• Adapt Existing De-blurring Techniques to real blurred images

Page 2: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Analysis, Reconstruction and 3D reconstruction from a single blurred image

Example

More info:[email protected]

Page 3: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

Restoration of Radial Blur

Page 4: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

Restoration of Radial blur

Page 5: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

Rotational Blur Estimation From a single Image

Page 6: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

Rotational Blur Estimation From a single Image

Page 7: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

Rotational Blur Estimation From a single Image

Blur is Space Variant

Estimation of • 3d camera rotation

axis

• Angular speed

From a single image

Page 8: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

And then….

Blur Removal

Denoising!

i.e. a restoration algorithm

Page 9: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

Restoration of Rotational Blur

Page 10: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blurred Image Analysis and Restoration

Restoration of Rotational Blur

Page 11: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blur Analysis and Restoration -- Projects

Implement a Rotational Blur Removal algorithm that takes into account also noise.

Deblurring on spherical surfaces

More info:[email protected]://tinyurl.com/3wqmeo

Page 12: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blur classification in a single blurred image

Given a Blurred Image, determine if the image is blurred

and in that case, if the blur is due to

object motion

camera shake

out of focus

Depth of field

These information are necessary for image restoration

Page 13: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Depth of Field – Blurred Object Segmentation

Page 14: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Depth of Field – Blurred Object Segmentation

Page 15: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Depth of Field – Blurred Object Segmentation

Page 16: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Blur classification and Segmentation -- Projects

Improve the segmentation algorithm, exploiting image priors

Embed the blur estimation algoirthm

Restoration: blur inversion + noise removal

Blur Classification

More info:

giacomo.

boracchi

@polimi.

it

Page 17: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Low-light Image Restoration

Taking satisfactory pictures at low-light conditions is challenging.

Pictures acquired with a short exposure-time have low SNR and are very noisy because of a high gain (ISO number).

Page 18: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Low-light Image Restoration

Page 19: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Low-light Image Restoration

Page 20: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Introduction

Taking satisfactory pictures at low-light conditions is challenging.

Pictures acquired with a short exposure-time have low SNR and are very noisy because of a high gain (ISO number).

Typically the exposure time is increased in order to improve the SNR of the acquired image.

But this also increases the risk of blur, because of movements occurring in the extended exposure.

Page 21: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Low-light Image Restoration

Page 22: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Low-light Image Restoration

A variety of solutions:• Lenses Stabilization • Different Acquisition Strategies

In particular [Tico06] and [Yuan07] proposed two methods that use differently exposed images• one with a long exposure time (blurred but with negligible noise)• one with a short exposure time (noisy but with negligible blur)

The noisy image is used to estimate the blur PSF allowing to restore the blurred image (deblurring)

[Tico06] Tico, M., "Estimation of motion blur point spread function from differently exposed image frames," Proc. 14th Eur. Signal Process. Conf., EUSIPCO 2006, Florence, Italy, September 2006[Yuan07] Yuan, L., J. Sun, L. Quan, and H.-Y. Shum, "Image deblurring with blurred/noisy image pairs," ACM Trans. Graph., vol. 26, no. 3, July 2007

Page 23: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Low-light Image Restoration

Acquire a sequence of short exposure images (frames) and jointly denoise them, using a video denoising algorithm

Page 24: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

BM3D DenoisingMore info:[email protected]://www.cs.tut.fi/~foi/BM-3D-PRESENT/index.html

Page 25: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

BM3D – Aggregation

Page 26: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

BM3D – Aggregation

Page 27: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

BM3D – Aggregation

Page 28: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

Long exposure, camera shaked image

Page 29: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

One of the short exposure, noisy image

Page 30: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

A detail from restored with Tico et al. algorithm

Visible artifacts due to mismatchesbetween assumed blur model (invariant PSF, linearity) and real blur.

Page 31: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

A detail from image

restored with

our algorithm

There are less artifacts.

Modeling is accurate.

Denoising is less ill-posed than deblurring.

Page 32: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

A detail from imagerestored with Tico et al. algorithm

Page 33: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

A detail from image restored with our algorithm

Not all details can be recovered by denoising because SNR is too low.

Page 34: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

A detail from the noisy image

Page 35: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

A detail from image restored with our algorithm

Not all details can be recovered by denoising because SNR is too low.

Page 36: Motion Estimation from a Single Blurred Image · 2009. 4. 14. · Motion Estimation from a Single Blurred Image ... Taking satisfactory pictures at low-light conditions is challenging

Dipartimento di Elettronica e Informazione

Denoising vs Deblurring

A detail from image restored with our algorithm

Not all details can be recovered by denoising because SNR is too low.

More info:[email protected]://tinyurl.com/4pcb6v